May I ask you 1Password developers a technical question about cross-platform data sync strategies?

ahmiao
ahmiao
Community Member

I am a developer who wants to develop an app across multiple platforms (iOS, OS X, Android, Windows etc). And I have no idea about the way to store data and synchronize them between platforms reliably.

Use SQLite to store the data and synchronize the database file by Dropbox?

If a SQLite database is in used, it seems impossible to use Core Data in iOS and OS X.

Any ideas about the question?

Really appreciate your help.


1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Sync Type: Not Provided

Comments

  • Hi @ahmiao,

    What works for one app won't necessarily work for another, but I'm happy to give you general info about what we've done that I think might be worth mimicking.

    For starters...I think it's important to separate the data the app uses directly vs what's used for sync. These two things have different constraints, and trying to find a single solution that works for everything, and all platforms is going to be very difficult. The cool part about doing this is that you're then free to find the best solution for each platform. We use sqlite on iOS and OS X, but we could just as easily use CoreData if we wanted. On Android we use sqlite as well, but it actually has a different database schema that works best for it.

    The more interesting challenge is defining a sync format. There's a million ways to go about this. You can either sync a copy of the data, or something more general like the equivalent of the database queries that every other device should run. The latter is actually how iCloud/CoreData sync works. That's a super tough problem to tackle though. Syncing a copy of the data is significantly easier, and is what we do. In our case your AgileKeychain file, or your OPVault file, or what gets stored in iCloud/CloudKit is basically just a duplication of the data we store locally in a sqlite database. In those 3 cases what's effectively stored is a JSON file for each item. What's nice about JSON is that it supported basically everywhere, and it's loose format so if we want to add things over time we can do that without much worry about older compatibility. You then need a translator to convert to/from your internal database format to the sync format.

    The fun (read: maddening) part about sync is detecting when multiple devices have edited the same piece of data, and handling that gracefully. Detecting this can be done all sorts of ways, but usually it boils down to having some kind of version number/identifier for the data. Then you've gotta determine how to merge it and save it back out. This will entirely depend on the purpose of the app.

    Sync is hard, but it's a really fun challenge. Best of luck on your app.

    Rick

  • ahmiao
    ahmiao
    Community Member

    @rickfillion Great thanks to your answer. It will help me a lot.

  • AGAlumB
    AGAlumB
    1Password Alumni

    :) :+1:

This discussion has been closed.