Merging multiple vaults
Comments
-
Okay, I'm having this same problem, too. I'm not 100% certain if my issue is what yours is, but I think I might have tracked something similar down.
First off, I don't remember how I set this up originally, but I do know that I started with a single valued Primary on my Mac. I bought a brand new phone and installed Dropbox, and then 1Password.
For months, changes made (i.e. new items) on my iPhone weren't sync'ing with my Mac at all. Dropbox seemingly wasn't even showing that the files on my Mac had been updated (though they showed up on the site). I'm not 100% sure if something on my Dropbox is busted, but I was able to force a new sync and it then showed that the 1Password files had been updated... now I know that it is at least getting over to the device.
What I just now realized is that there are two vaults on my iPhone. There's Primary and then there is 1Password. I have no idea where 1Password came from, but on the vaults screen, it was set to store my changes to that vault. Not the Primary vault that syncs to my computer. When I managed to find and load the 1Password vault on my Mac, sure enough, all of the changed/added items started to show up.
So now I have (seemingly) two completely different vaults and I have no idea how to merge them together—though I was able to find my missing items.
0 -
@nvahalik: Aha! Thanks for the detailed description. It sounds like you probably setup 1Password with a new vault at some point when installing it on your (perhaps new) iPhone, and then added a secondary vault. Since it isn't actually posable to create an additional vault in 1Password for iOS, this would have had to originate in Dropbox. Let's work on merging your data!
Now, if it's just a few items, it may be simplest to view their details and move them to the other vault. However, it sounds like you may have been using this second vault for a while. In that case, merging it with the other will be much more efficient. First we'll need to establish a few things:
- What is the path to your Primary vault in 1Password for Mac Preferences > Sync?
- Do you have more than one vault in 1Password for Mac?
- What are the paths shown for each vault in 1Password for iOS Settings > Sync > (Vault) > Sync Service?
If I understand you correctly, ultimately what you'll need to do is disable sync for the iOS vault you want merged with your Mac data, and then re-enable sync for the iOS vault, choosing the same vault you're syncing with Dropbox from the Mac. You'll probably be warned of a vault mismatch, and then all you need to do is tap More Info and then Merge them. Please let me know what you find. We'll get this sorted out for you! :)
0 -
The path to the Primary vault in OSX is ~/Dropbox/1Password/1Password.agilekeychain.
I do have more than one vault listed in OSX. I have another shared vault that isn't on my iPhone.
The other keychain in question, the "1Password" vault, is in ~/Dropbox/Private User Data/1Password.agilekeychain.The 1Password vault is the oldest one, and I guess at some point I just never removed it and now it's confused the iOS version?
There are ~1.8K items in each vault. What's tough is that likely there are probably a couple hundred entries that are mismatched between the two.
0 -
There are ~1.8K items in each vault. What's tough is that likely there are probably a couple hundred entries that are mismatched between the two.
@nvahalik: Wow. That is certainly daunting! It sounds like what you need to do is merge these to end up with One Vault to Rule Them All...but there are a few challenges we can foresee here:
- You may end up overwriting some data you want with some data you don't
- You may end up with duplicates.
- Going through the merged vault to verify either would likely be time-consuming (at the very least).
Ultimately there isn't any way for 1Password to determine your intent and sort all of that out for you, but there are a few things that can help ease the burden:
- Backing up all of your 1Password data beforehand gives you a fallback if needed.
- In the case of legitimately identical items (matching filename/UUID), Dropbox will have the newer take precedence.
- You can also save a copy of each original vault separately from the new, merged vault to refer to in a pinch temporarily, until you're confident that you have everything you need in the new one.
With all of that in mind, this is what I would do in your situation:
- Make a full backup of the data on each device.
- Make a copy of each vault saved in Dropbox and store them in a safe place (probably not Dropbox for now, to avoid confusion — but you'll still need the originals there at first to do the merge).
- Rename these copies to something meaningful for you, such as "MacPrimary.agilekeychain" and "iOSVault.agilekeychain".
- Note the vault path listed in 1Password for iOS, and then disable sync win 1Password for Mac for the vault you want to merge there (you can also do it in the other direction if you prefer).
- Enable Dropbox Sync for the same vault 1Password for Mac to point at the vault stored in Dropbox which is being used on iOS.
- When asked, verify the path first, and click the Merge button to start the process.
- Give Dropbox some time to finish syncing from your Mac to the Dropbox server (you'll know it's finished when the Dropbox icon stops animating).
- Open 1Password for iOS again and let the changes sync there as well.
- Delete the vault in Dropbox which is no longer being used (in this example, the original Mac vault).
- (Optionally) move the copies of both original vaults to Dropbox. This way, you can open them on either device as a Secondary vault if needed for reference.
- Use your new merged vault to determine that all of your data is there and up-to-date.
- (Optionally) retrieve data from the original vaults as needed.
I hope this helps. Please let me know how it turns out! :)
0 -
This will take some time—as the iOS device was defaulting to 1Password and OSX was defaulting new items to Primary. I was REALLY hoping you'd say I could use Apple's scripting interface to enumerate and go through duplicates like I've done with iTunes in the past... 8-)
0 -
This will take some time—as the iOS device was defaulting to 1Password and OSX was defaulting new items to Primary.
@nvahalik: Ouch. Okay that makes sense. I had thought perhaps this was a single vault at one point that had diverged over time. :ohnoes:
I was REALLY hoping you'd say I could use Apple's scripting interface to enumerate and go through duplicates like I've done with iTunes in the past... 8-)
Hmm. I wonder if that may actually be possible with @MrC 's converter...
@MrC: If he exported both vaults to 1PIF, is there duplicate detection that might help in merging the two? :innocent:
0 -
A de-dupping tool was something I'd been considering, but was awaiting the right customer who had a real need, and would be engaged in working through the details.
The converter can read 1PIF (it does so for the onepif2html converter), and could be used as the basis to compare records for de-duping. The issue becomes - what defines a duplicate? Since these were different vaults, the UUID of each entry is not going to match between supposed duplicate records.
So, we have to resort to string matching across fields. Is a duplicate simply the string used as the Title? Or is it the combination of Title, Website, Username and Password for Logins, but something else for other categories?
And there can be fuzzy-matching too - where a field has only a slight variation between two records (a corrected typo, for instance).
And that brings us to a second challenge. How should the duplicate candidates be presented to a user for selection and merging? Since conversion is a one-shot tool generally, I don't have all the bells and whistles of a UI to present pretty views of two records side by side, so a user can select the record, and possibly the fields from each record.
@nvahalik - perhaps you can tell me what would be "good enough" to get you de-duped?
0 -
So, we have to resort to string matching across fields. Is a duplicate simply the string used as the Title? Or is it the combination of Title, Website, Username and Password for Logins, but something else for other categories?
@MrC: That was my concern as well. I'm not sure that it's possible to do this in an automated fashion in such a way that the humans would be satisfied with the results, but I figured if anyone would know it'd be you. :p
0 -
@MrC, I haven't given it a whole lot of thought, but ultimately 90% of the two vaults should be completely equal. After that, it should be roughly like this:
if (item is unique) then include that item else get the item with the latest modification date include that item end if
0 -
I think the difficulty MrC is trying to convey is that whole "is unique" concept. Does every single field need to be evaluated, or is it enough to compare the title, URL, username, and password (for login items)? What about other categories of items? It could very quickly get very complex to try and match every possible field on every possible item type (especially when you start getting into custom categories).
It has been quite a while since I've done any programming myself, so there may be a solution that I'm not thinking of.... perhaps all of the fields in each item could be turned into an array, and a loop could be written to cycle through all fields of items with the same title to compare them to each other. If there are differences they are both retained. If they are identical one is discarded. But even this, if possible, would not be a perfect solution.
It would be interesting, I think, to see how many item UUIDs are different between your two vaults, and if perhaps sorting them out that way would help... These are unique within vaults but not necessarily across vaults.
Ben
0 -
Ah, okay I see what you're saying @bwoodruff. I'd think for my purposes, website and timestamp are sufficient enough to determine if something is unique or not. I didn't know if maybe 1Password had some sort of hash that could be used.
0 -
Okay, I think it was actually way better than I imagined. I was able to determine that the vault was just a few months old and that there were only approximately 6 new entries added on my phone. I have since been able to disable the extra vault and it looks like everything is working smoothly.
0
