syncing shared secondary vault on Dropbox creates many files

jacknutting

The company I work for has 1Password licenses for all of us, and we have a shared vault for all of us to use. Some time today, we started seeing a proliferation of files in .dropbox.cache/2015-06-10. At this point I'm seeing the Dropbox sync notification on my mac pop up every 5 or 6 seconds, telling me that anywhere between 1 and 50 files were updated. The reason I'm pretty sure 1password is involved is that many of the files have names like:

FFAD8ECC9F71485EBDD5049670FAA64B (deleted 435d59c28177cf028b367315084c8f04).1password

I'm running 1Password 5.3.1, but don't really know what all my colleagues are using.

Any idea how we can solve this? We've been using 1Password for a long time now, and I can't recall this happening before.

---

1Password Version: 5.3.1
Extension Version: Not Provided
OS Version: 10.10.3
Sync Type: Dropbox

Vee_AG

Hi @jacknutting,

Have you noticed an increase in the size of your 1Password.agilekeychain file in Dropbox? And are you seeing any error messages?

jacknutting

It looks like it has been growing, but I can't say exactly when it started, or by how much it's grown. The shared keychain has 266 items, and is 231.4MB in size.

Compared to my personal keychain with 486 items at 12.5MB, it seems like the shared one is overly-large.

So I took a look inside the data/default directory in the keychain, and I find that there's a lot of action going on. Clearly the files in there have also been changing on Dropbox a lot, there are just so many (thousands) that I lost track of where I was. In data/default, I have lots of files with names like this:

contents (User Name's conflicted copy 2015-06-10 (XXX)).js

Where "User Name" is one of our dozens of users, and "XXX" is a number from 1 up to 286. That's the highest number I've seen, but there's some variation from user to user and I haven't checked all users, just skimmed over the list of files.

Along with those, there is another set of similarly-named files:

FFAD8ECC9F71485EBDD5049670FAA64B (User Name's conflicted copy 2015-06-10 (XXX)).1password

There don't seem to be quite as many of those, but still over 100 for many users.

Looking more closely, there are more sets of files that follow the same format but have a different ID string at the start:

FF7801B4797A45E88F7C881CC0459CC2
FEC0CB9BBC8A42D8A00088929DB01BD6
FDB3A0EA754747E1B7D3EBD56BB3D3CA
FD91644A24D9452388BA10B481F387CD
FC3A722B98944D04AD987B92F6C2BC4B
FC340AB3C1B940C998841BB443632068
FB45E9B9902B46C78E2EF44B4BEB8D5A
FA57EB3B8743414C966E736E7E7CB28C
FA14E0F19B754693A1D701DFAD2F3A70
F978A7A59A9D4D8FABEF86D34D3EB188
F81E963475C0414B88F06672E71A03E7
F782A8471B4C42829D40E83070ACE3D3
F73EF905BD954A02BAB66A3D7DB7D21F
F72AF60F1ACF4873B7F1FE78112CAC2A
F5E0C353D5D14EEA9AA8F321FB4BA079
F4B507EFEB004E7F982246C4FBF239B4

Etc, etc. I could go on and on. Each of those also has a matching F4B507EFEB004E7F982246C4FBF239B4.1password file.

At this point I've turned of my Dropbox syncing because it was just too much to handle. I'd appreciate any tips you can give us.

jacknutting

Another point of data: Scrolling through the file listing, I see about 22 user names coming back again and again. That's probably a third or a fourth of our total number of users, so it's not every user. I'll start asking around a bit about who's running what version of 1Password.

jacknutting

I just checked with two colleagues who are in the list of "conflicted copy" copies, and they are each running 5.3. I'm running 5.3.1, and my name isn't among the conflicted copies. Just spitballing here, but could it be that the new version 5.3.2, which presumably one or more of my colleagues have already updated to, is triggering a weird interaction with keychains in use by version 5.3?

littlebobbytables

Hi @jacknutting,

Versions 5.3.1 and 5.3.2 were quite minor, with small changes specific just to the AgileBits Store version of 1Password for Mac so it won't be anything in there. As far as all syncing code goes it will be identical to that found in 5.3.

I see one of two possibilities and I'm not sure which you will view as easier so I'm going to cover both.

One approach is to make a note of the 22 people that seem to have stood out. Have them delete the secondary vault from their copy of 1Password and hopefully Dropbox will settle down. I wish I could supply a timeframe but it's hard to say given the reliance on a cloud system and it sounds like you have quite a few users interacting with this single Dropbox account. Once 1Password/Dropbox isn't kicking up a storm you can have them re-add the vault which is as simple double clicking on the .agilekeychain in Dropbox. I'm hoping these 22 are enough to stop the rampant conflicts going on.

The second approach is to delete the secondary vault's keychain completely from Dropbox. With the .agilekeychain missing 1Password an all machines will disable sync for the shared vault. One person could then re-enable Dropbox sync for the secondary vault, creating a brand new .agilekeychain. You can either have people delete the secondary vault completely and re-add it by double clicking on the .agilekeychain or by re-enabling Dropbox syncing and pointing it to the new .agilekeychain. Personally I suspect the deletion of the secondary vault and re-adding it might be an easier pill to swallow

It's tough to say which is better. If the first works then you've only inconvenienced 22 people for a small period of time. If it doesn't work you'll be looking at option 2.

I believe the problem you're experiencing here is while Dropbox is fantastic, 1Password is interacting with the .agilekeychain in an automated fashion and that's each copy of 1Password. As long as there aren't any blips that's okay but if something happens you can end up in a bit of a vicious spiral as copies of 1Password don't communicate with each other and instead are all reacting to the same events and what you need is a single copy of 1Password to say "I've got this, give me a moment and I'll resolve the conflicts" which 1Password isn't capable of doing.

Thankfully this is rare given it will be a pain to resolve. I do apologise that there isn't an easier and more elegant solution that I can think of.

You may have a bunch of questions or thoughts so please do post back.

jacknutting

Thanks for getting back to me. The second option does sound less painful, for sure.

So now I've got a vault that contains many thousands of files in the data/default directory. Which of them should I keep? If I filter out everything with the word "conflicted" in the name, I'm left with these:

16 -rwxr-xr-x@ 1 jack staff 6552 Feb 26 15:53 .1password.keys
8 -rw-r--r--@ 1 jack staff 8 Feb 26 15:53 .password.hint
0 -rw-r--r--@ 1 jack staff 0 Nov 19 2014 .ws.agile.1Password.write
16 -rwxr-xr-x@ 1 jack staff 6552 Feb 26 15:53 1password.keys
88 -rwxr-xr-x@ 1 jack staff 43567 Jun 11 08:18 contents.js
16 -rwxr-xr-x@ 1 jack staff 6041 Feb 26 15:53 encryptionKeys.js

As well as a bunch of files (405 of them) like this, but with varying hex strings:

FFAD8ECC9F71485EBDD5049670FAA64B.1password

Should I presume that if I just keep those files, this vault will be OK?

littlebobbytables

Hi @jacknutting,

Personally I would recommend deleting the entire .agilekeychain folder for the shared, anything else and the results would be unpredictable.

What might reassure you is the entire .agilekeychain is nothing more than the sync data - deleting the entire .agilekeychain from Dropbox won't remove the vault from any person's Mac as the vault is locally stored and the local copy is the primary. With the deletion of the entire .agilekeychain and then creation of a brand new one (yet still representing the same data) each Mac would, as I mentioned, disable syncing as although the two .agilekeychains would have the same filename, 1Password knows they aren't the exact same copy. Everybody would still have access to the copy of the shared vault that is stored in their OS X user account and so during the period of re-establishing synchronisation they could still access the Login items (as long as nobody goes on a password changing spree at the same time).

Obviously please do keep asking questions until you're happy.

jacknutting

Well, that pretty much worked, but then a few days later we ran into the same issue. I guess that someone's locally-stored shared keychain contains something that conflicts with "the truth" (the vault that I shared as a new shared keychain), in a way that causes Dropbox to not know what to do besides start creating more "conflicted copies" to pass around to everyone, forever. So I'm going to re-do this, but this time ask everyone to actually delete their old shared vaults, and just add back in the new one that I will freshly export.

Probably that will work, but overall I feel like this method of syncing files is fraught with problems. Because who knows when it will fail again?

One thing I'm starting to consider is to stop using Dropbox for this, and instead sync our keychain via a shared git repository. We are a highly technical company and this won't be a problem for the majority of our users. Most of us are using git all day anyway, so it will probably feel pretty natural. Can you think of any potential downsides to this approach?

littlebobbytables

Hi @jacknutting,

What you've experienced. It can happen and I've dealt with a couple of cases but I don't think it's common at all. Sometimes somebody will say this is the first problem they've had since we introduced multiple vaults but when it happens it stands out because it really does a number on the .agilekeychain. I don't know if it will have had any influence but it might be worth making sure all of your machines are correctly contacting a NTP server too. I realised recently that my Mac Mini's time clock was minutes out compared to my other devices despite the fact that they're all set up to calibrate their time clocks. Turns out the server I was using on the Mac Mini wasn't responding and who knows how long that might have been happening for. That sort of thing could be potentially messing up Dropbox.

I'm not well equipped enough to comment on using GIT I'm afraid. I've used GIT and in theory it should work but I feel certain nobody here has tested that configuration. I take it you're thinking that with the manual aspect of GIT that you will feel you have more control and awareness of what is going on, and that if one machine starts to run amok you'll be better equipped to resolve it? It would mean remembering to pull down changes unless your GIT tool is better than whatever I played with the last time I used GIT. It's an interesting idea but as I say, I'm not sure anybody here would be able to offer much insight into what to expect.

donnoman

We have experienced the same thing, we have multiple dropbox vaults and we have begun naming this event a 'sync storm'. We don't know why it starts, or how it ends. The beginings and endings are equally mysterious, and there is no fix. I've done the vault blow away and make a new one, and support just keeps telling me "it wasn't meant for a group of users, just small families". Dropbox blames 1password, 1password blames dropbox. Nobody seems to have a solution.

Drew_AG

Hi @donnoman,

I'm sorry to hear about the problems you've been having with that!

I see that you've already been receiving help with this via email, and that you replied to that email thread again earlier today. Someone will respond to that as soon as possible, and we'll keep the conversation going through email so we don't duplicate efforts or cause confusion between here and there. Thanks so much for your patience! :)

ref: RYQ-19275-379