SQLite vs ForestDB: Pros and Cons and future development

Hi all,

Today I’ve been reading your documentation about the storage engines supported by Couchbase: SQLite and ForestDB.

Concerning your documentation, it seems that ForestDB is much better than SQLite (it’s faster, has better concurrency handling and so on).

However, what’s the reason of not using it by deafult? Is there any issues about ForestDB? Isn’t it enough stable and mature? Are you going to push it as the default storage engine in the future (v.1.4 and v2.0)?

Actually, while we were able to make some initial gains on forest db, in the end it was not sustainable and so for mobile we are deprecating it with 1.4 and removing it in 2.0. We may revisit it again in the future if we make improvements.

1 Like

Thank you very much for your reply!

In that case, I’ll go with SQLite :smiley:

In version 1.3.1 I made some minor changes to the way the iOS version uses SQLite, which enables concurrent reads along with a writer. This should help a lot in common situations like when your app is displaying data while the replicator is pulling updates into the db on its background thread.

ehm, is there an easy way to migrate from ForestDB to SQLite?

No, we never wrote an updater for that direction. You can do it by creating a new database, enumerating all docs in the original db, and adding them to the new db. (Make sure you use the method that adds a doc with an existing revID, instead of creating a new revision; that was added in 1.3.)

I tried to find the code for the SQLite -> ForestDB migration but without luck. Can you point it out please?

I also wrote my migration function and it seems to work, but I think the couchbase API is a bit inconsistent so I don’t have high confidence :slight_smile:. For instance, revision.attachments returns an array of CBLAttachment while attachments in putExistingRevision(withProperties:attachments:revisionHistory:) requires a dictionary. :confused:

Anyway, here is the function I wrote (invoked from the couchbase queue):

    private func dbMigration() {
        let documentsPath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0]
        let couchbaseFolder = documentsPath + "/couchbase.cblite2"
        // If the old couchbase ForestDB folder is not there, no migration is needed.
        guard FileManager.default.fileExists(atPath: couchbaseFolder) else {
            return
        }
        do {
            // Create a new temporary manager to open the old ForestDB database.
            let tmpManager = try CBLManager(directory: documentsPath, options: nil)
            tmpManager.dispatchQueue = couchbaseQueue
            tmpManager.storageType = kCBLForestDBStorage
            tmpManager.excludedFromBackup = true
            let oldDb = try self.manager.databaseNamed("couchbase")

            // Query to get all documents in ForestDB
            let query = oldDb.createAllDocumentsQuery()
            let result = try query.run()
            while let row = result.nextRow() {
                guard let documentId = row.documentID,
                    let oldDocument = row.document,
                    let revision = oldDocument.currentRevision,
                    let properties = revision.properties else {
                        continue
                }

                // Create a document in the SQLite database with the same id it had in ForestDB.
                let document = database.document(withID: documentId)

                // Get attachments.
                var attachments: [String: URL] = [:]
                revision.attachments?.forEach { (attachment) in
                    attachments[attachment.name] = attachment.contentURL
                }

                // Get revision ids.
                var revisionHistory: [CBLRevision] = []
                do {
                    revisionHistory = try revision.getHistory()
                } catch let error as NSError {
                    print(error)
                }
                var revisionIds: [String] = []
                revisionHistory.forEach { (revision) in
                    if let id = revision.revisionID {
                        revisionIds.append(id)
                    }
                }
                // Revision ids must be in reverse chronological order.
                revisionIds.reverse()

                if revisionIds.count > 0 {
                    _ = try? document?.putExistingRevision(withProperties: properties, attachments: attachments, revisionHistory: revisionIds, from: nil)
                }
            }
            try FileManager.default.removeItem(atPath: couchbaseFolder)
        } catch let error as NSError {
            print(error.localizedDescription)
        }
    }