Compress opvault/agilekeychain on sync
You should have an option to compress the opvault/agilekeychain vaults prior to sync in all clients. It's really painful to watch the vaults synchronize over dropbox or folder sync when there are hundreds of changed files. You are not utilizing bandwidth effectively by making all of those round trips. If it was a single file per vault, you could download the updates much faster.
1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Sync Type: Not Provided
Referrer: kb-search:zip, kb:agilekeychain-zip, ug:mac/new-computer-elsewhere, kb:switch-to-opvault, kb:opvault-design
Comments
-
You should have an option to compress the opvault/agilekeychain vaults prior to sync in all clients.
@NoOneHere: Compression is something we can consider adding in the future, but...
It's really painful to watch the vaults synchronize over dropbox or folder sync when there are hundreds of changed files.
Unless you're only syncing very rarely, you probably won't have hundreds of items that need updating at once. And of course the initial sync only needs to be done the first time, and after that it's incremental. And I'm not sure why you'd want to watch. Granted, I do this myself too, but I have no idea why. :lol:
You are not utilizing bandwidth effectively by making all of those round trips. If it was a single file per vault, you could download the updates much faster.
Requests for files from the server are no more bandwidth intensive than going to a webpage that loads additional resources (javascript, images, etc.) And keep in mind that when 1Password contacts Dropbox for updates (as an example), it isn't always requesting individual files, since it doesn't necessarily know what new data the server has for it. iCloud on the other hand manages this all itself at the OS level, so it often already has all new data before 1Password even asks for it. It can be more efficient in this regard since it isn't file-based.
In regard to 1Password specifically, individual items are — at most — kilobytes each. My largest is roughly half a megabyte, whereas most are much smaller, all the way down to roughly 200 bytes. My main vault is roughly 21.6MB uncompressed. Zipped, it's 21.2MB: not a significant decrease. Finally, even my 1Password backups (which are .zip files containing my smaller secondary vaults as well) are actually a slightly larger 31.6MB.
I think you may be overlooking a few things here:
- Any reduction in bandwidth usage will come at the expense of additional energy usage (due to processing compressed data), which isn't really a big concern on a computer, but then again cellular data is less of a a concern on a computer than it is on a mobile device.
- Encrypted data such as your 1Password vault is less efficient to compress, because we can't take advantage of advances in modeling that allow specific data types (text or images, for example) to be compressed more efficiently.
- Sending over the vault in compressed form will allow for some savings in bandwidth (at the expense of energy, as I mentioned above), but this is really only useful for the initial sync, so that's really the only area where this is worth looking into.
- Decreased granularity (which seems to be at least part of what you're advocating) is actually less efficient when it comes to bandwidth, since a minor change to a single item then requires resending either the entire vault over again or a 'chunk' of it, which contains data other than the bits (literally, ha) that were actually updated. This can also increase the likelihood of sync conflicts, if different data from the same set is being updated at the other end as well.
Now, if you're talking about batching and compressing only updated items, that could be slightly more efficient...but of course the sync service itself would have to have the logic built in to do this. Perhaps this is something that will be more feasible in the future. :)
0 -
I wasn't suggesting you'd gain a tremendous amount of compression. I was actually largely going for storage mode--more of a tarball. A good cipher-text shouldn't be terribly compressible past it's plain text size. I was more aiming for TCP overhead and not reaching peak transfer speed performance.
There is a large inefficiency in transferring a lot of 1000 byte files in comparison to one larger compressed archive (let's say 10 MB for my vault in its current state or your 21 MB vault). The larger file has the ability to be transferred faster since by the time you reach any sort of speed on a representative example file contained within the large set of smaller files that transfer has abruptly ended and you're now requesting a new part.
As for uncompressing, it's going to happen so quickly on any modern device that it's not going to be an issue--especially if we're simply using storage mode. But that's just my opinion. Loading your average news website is going to be a heck of a lot more strenuous on your device.
Your team is going to approach the problem in a way that makes sense to them. Whether that takes the form of a delta of the vaults contents or sending the entire thing is entirely up to you (though, liking my options, I'd say make it a checkbox in advanced 8-)).
0 -
I wasn't suggesting you'd gain a tremendous amount of compression. I was actually largely going for storage mode--more of a tarball. A good cipher-text shouldn't be terribly compressible past it's plain text size. I was more aiming for TCP overhead and not reaching peak transfer speed performance.
@NoOneHere: Thanks for clarifying. It sounds like you're more concerned about protocol efficiency rather than bandwidth. Sorry for misunderstanding!
But realize that while this is a tradeoff you're willing to accept (increased efficiency at the cost of power and/or bandwidth), that may not be true for everyone. For example, many (most?) people are on metered mobile data plans. And while electricity is relatively cheap, cellular data certainly isn't :dizzy:
Your team is going to approach the problem in a way that makes sense to them. Whether that takes the form of a delta of the vaults contents or sending the entire thing is entirely up to you (though, liking my options, I'd say make it a checkbox in advanced 8-)).
Indeed, but we really appreciate your perspective! Especially as we develop future versions of 1Password (and 1Password for Teams in particular, which we have a lot more control over when it comes to data transfer) we'll continue to reevaluate what makes sense. Who knows? Maybe by the time we're working on 1Password 10(?) data will be cheap (or free!) Thanks so much for taking the time to let us know this is an area you'd like us to work on. :chuffed:
0
