syncing thru dropbox not working correctly on new installs/devices

jjspreij

Hi there,

I've got 1Password running on 3 Macs, 1 iPad and 1 iPhone ; the vault is being synced thru dropbox. Running 1Password 4 on Mountain Lion, 1Password 5 on Yosemite, and newest versions of iOS & 1P on the iDevices

This has worked perfectly for many years. But now, any new install of 1Password will only get the most recent additions/edits to the vault, nothing before a certain date.

So two of the Macs have the complete vault and sync perfectly (currently 994 items in vault); but the newly installed Mac is syncing all recent items in the vault, but not receiving anything older than April 2014 (it has 117 items in vault). Already tried: completely deleting the OS X account, but the same thing happened in the new OS X account.

And in very similar fashion, my iPhone5 was syncing perfectly running iOS 8.1 ; but just now I've updated to an iPhone 6, and it is only receiving the same 117 items as the Mac mentioned above.

So, any new passwords are syncing to all devices; but the "new" devices are not receiving the older password entries.

What next? ;-)

Megan

Hi @jjspreij‌

I'm sorry to hear that you're having trouble keeping things synced up lately. I've got a few basic things that we want to check regarding Dropbox first, before we get into the serious troubleshooting:

• Please ensure that Dropbox is installed and running on your computer
• Check that you are logged in with the same account on all computers and devices
• Confirm that Dropbox sync is selected in 1Password > Preferences > Sync on your Mac
• Confirm that Dropbox is selected in Settings > Sync on your iOS device

If all of those items check out, let's run a test and see where things are getting tripped up:

• Create a test entry on your Mac ("Test, Mac") and add some data
• Create a test entry on each of your iOS devices ("Test, iPhone/iPad") and add some data
• Does "Test for Mac" show up on either of your iOS devices?
• Does "Test for iPhone" and/or "Test for iPad" show up on your Mac?
• Log in to the Dropbox website and click on the 1Password.html file inside your 1Password.agilekeychain folder
• Enter your Master Password
• Do either or any of your test entries show up in your database here?

Please let me know the results of this little experiment - once we have a better idea of where things are going sideways, we should be able to get this straightened out simply for you :)

jjspreij

• Please ensure that Dropbox is installed and running on your computer
• Check that you are logged in with the same account on all computers and devices
• Confirm that Dropbox sync is selected in 1Password > Preferences > Sync on your Mac
• Confirm that Dropbox is selected in Settings > Sync on your iOS device

checked all of the above, all ok & in order

• Create a test entry on your Mac ("Test, Mac") and add some data
• Create a test entry on each of your iOS devices ("Test, iPhone/iPad") and add some data
• Does "Test for Mac" show up on either of your iOS devices?
• Does "Test for iPhone" and/or "Test for iPad" show up on your Mac?

tested with 2 Macs, 1 iPhone, 1 iPad; created a new unique "Reward Program" entry on each; all of the new entries sync correctly to all of the devices

• Log in to the Dropbox website and click on the 1Password.html file inside your 1Password.agilekeychain folder
• Enter your Master Password
• Do either or any of your test entries show up in your database here?

No, they don't! And logging in there gives an error:

I can login thru the Dropbox website; and I can see a lot but not all of my entries listed; but I can't get actual details on them, only the list entry

Also the login screen looks old, not in the new stye:

• Please let me know the results of this little experiment - once we have a better idea of where things are going sideways, we should be able to get this straightened out simply for you :)

Hope so!

One more bit of info: there are two Macs, both running latest versions of OS X & 1P. The 1Password.agilekeychain file/bundle (inside the Dropbox folder) on both of them is exactly the same size. The recently cleanly installed Mac is missing all the older 1P entries; the Mac which has been upgraded over the years has all the 1P entries. Both of them get all new or newly edited entries.

Hope this helps, JJ

Megan

Hi @jjspreij‌

Thanks for getting back to me with those results!

tested with 2 Macs, 1 iPhone, 1 iPad; created a new unique "Reward Program" entry on each; all of the new entries sync correctly to all of the devices

This is a pretty great sign, actually. Sounds like everything should be in order here. I notice you left one of the Macs out of the rotation here - any reason for that?

I can login thru the Dropbox website; and I can see a lot but not all of my entries listed; but I can't get actual details on them, only the list entry

Also the login screen looks old, not in the new stye:

1PasswordAnywhere does need to be access from the Dropbox website. Can you tell which entries are missing - is it the same entries that are missing on your new Mac? Because 1PasswordAnywhere was designed for 1Password 3, it will look slightly differently than the 1Password 5 interface, and some details (such as any data stored in custom fields) will not be visible.

jjspreij

hi Megan,

Thanks for getting back!

This is a pretty great sign, actually. Sounds like everything should be in order here. I notice you left one of the Macs out of the rotation here - any reason for that?

Based on earlier testing, I knew what the result on the third Mac would be - but I doublechecked just now, and yes, all new entries sync fine to all devices, also the third Mac.

It's just on the newly installed devices (new iPhone6, MacBook with new clean install of OS X), which have fresh 1P installs, that all 1P entries older than April 2014 are missing. If I then edit & save one of the missing entries on another device, that entry will show up on all devices...

1PasswordAnywhere does need to be access from the Dropbox website. Can you tell which entries are missing - is it the same entries that are missing on your new Mac?

Very sorry, on closer inspection 1PasswordAnywhere does show all entries in the list - BUT, not all of them are "viewable". In fact, exactly the entries that are completely missing from the new 1P installs, those are not viewable in 1PasswordAnywhere.

A couple of screenshots showing number of entries per category will help to explain:

1PasswordAnywhere thru Dropbox website

1P v5.0.2 install on Yosemite

1P v5.1.2 on iPhone 6 (new install)

So for instance 1PasswordAnywhere has 142 Secure Notes; 1P on Yosemite also has 142 Secure Notes; but 1P on the iPhone has only 24 Secure Notes. But, exactly those 24 are the only ones which can be viewed in 1PasswordAnywhere; clicking on any other Secure Note results in this popup error:

Rad

Hi @jjspreij,

I’m sorry to hear that you’re still experiencing issues with Dropbox sync on all your devices. It seems that you got into a very rare and strange but which we currently are trying to nail down.

I strongly suspect that your 1Password.agilekeychain is somehow corrupted, so to eliminate it from our list of suspects, let’s try the steps bellow to try and fix this problem:

1. On the Mac that contains all of your data

   a. In 1Password 4 for Mac go to Preferences > Sync and disable Dropbox sync.

   b. Your 1Password.agilekeychain file is normaly located in Users/Your_Username/Dropbox/1Password/. Please rename it from 1Password.agilekeychain to 1Password_old.agilekeychain and move it somewhere else on your hard drive for backup purposes. Please make sure that there is no other agilekeychain in your Dropbox directory.

   c. Enable Dropbox sync. This should create a fresh new 1Password.agilekeychain.

2. On your other Macs (if applicable),

   a. Disable Dropbox sync.

   b. Quit both 1Password and 1Password mini.

   c. Trash the local database “~/Library/Application Support/1Password 4/Data” and empty the trash.

   d. Reboot you Mac as a safety precaution.

   e. Launch 1Password and please make sure that it uses the newly created 1Password.agilekeychain from Dropbox with all your data

3. Please perform the steps bellow on all your iOS devices using 1Password.

   a. Open 1Password and go to Settings > Advanced and erase your local data by choosing "Erase data and Settings". This will only erase the local data, not your Dropbox data

   b. Reopen 1Password and choose Dropbox sync. 1Password for iOS will now use the newly created 1Password.agilekeychain.

This should do it. Please let us know if this helped solve the issue.

Cheers!

Rad

ref: OPM-2608

rickfillion

Hi @jjspreij,

If you have not yet followed @Rad‌'s advice (or if you still have the 1Password_old.agilekeychain file around), I'd love to get more information from you in hopes of better understanding what happened to your data in the AgileKeychain (if there's a bug, I'd like to fix it). For every item in your vault, there's a corresponding file inside of the AgileKeychain. It sounds to me like the files corresponding to the older entries are either not present, or corrupted, or otherwise unreadable. The AgileKeychain also contains a list of all items with the vary basic info, which is why it's showing up in 1PasswordAnywhere, but then when you click on it it can't read the data. So then the question becomes... what happened to these files? I would greatly appreciate getting more information to help look for a bug.

Rick

jjspreij

Hi Rick,

as luck would have it I hadn't followed @Rad's advice yet, was just about to when I saw your note. Would be glad to help! Let me know what I can do. I'm at home in the Terminal if that helps.

What puzzles me is how the older 1P installs are still able to read all the entries, working with the same identical (?) Dropbox data...

--JJ

jjspreij

Just had a look inside a copy of the file 1Password.agilekeychain.

In the folder 1Password.agilekeychain/data/default there are only 166 items (that folder looked like the logical one to contain all entries):

• contents.js (dated yesterday)
• 1password.keys (dated 3 nov 2008)
• encryptionKeys.js (dated 3 nov 2008)

• and 163 *.1password files (most dated from current back to 18 march 2014, and just three from 2012 and 2013)

• the "older" 1P install on my main Mac shows 997 entries behind "All Items" at the top (so a lot more than 163, and all readable)

• the new 1P install on iPhone6 and MacBook show 132 items under "All Items" (so a bit less than 163)

I opened the oldest *.1password files (from 2012 and 2013) in TextWrangler, and recognized one as a 1P Note based on a string at the end. Searching for the title of that note:

• in 1P on my main Mac finds it as expected. In the filesystem the modified timestamp is 11 april 2012; but in 1P it is 22 april 2012
• in 1P on iPhone does not find it

Hope this helps!

jjspreij

Yet more info: I also have a TimeMachine backup system going back 1 year. Looking at the contents of 1Password.agilekeychain/data/default at different moments in backup-time, I found the "switch-moment" - a backup with 1000+ files in that directory, and one week later only 100+ files. This happened somewhere at the beginning of October 2014.

Here is a comparison of the before & after showing listings from Terminal:

https://dropbox.com/s/bbbga7pqa5pmrfz/1p-analyse.txt?dl=0

("Tardis" is my TimeMachine Volume)

I'm surprised that 1P on my main Mac still works perfectly, with all those files now missing!

littlebobbytables

@jjspreij‌ Thank you and I've notified Rick of your response so he should pop in here at some point to review your observations.

Your surprise regarding your main Mac I think I can explain depending on when you first started syncing it. If the first sync happened prior to this drastic change to your agilekeychain then it would have had the chance to populate its internal database from the sync repository. Prior to 1Password 4 the agilekeychain was the database and syncing devices via Dropbox was handled by simply moving the vault into Dropbox - nice and simple. Since then your vault is actually an encrypted sqlite database file and your sync data is now a second copy essentially. This has a few advantages, one of which is if anything goes wrong with Dropbox you've lost nothing. Personally I prefer the new approach.

Now if the initial sync of the two machines with the seemingly correct vault happened after this then actually I'm surprised too.

jjspreij

The first sync happened beginning of 2008, in antique times, far before the recent drastic change ;-) So that explains it, thanks!

rickfillion

Hi @jjspreij‌,

My gut tells me that @littlebobbytables‌ has nailed it here, and it would explain all of the behavior you're seeing. It would be worth us reviewing on our side how we migrate from a 1P3 agilekeychain in dropbox to 1P4/5's sqlite database + agilekeychain sync. I created a case in our bug tracker so that we can look more deeply into the migration process (OPM-2646).

Can you confirm that this time machine machine backup difference happened at or around the time you upgraded from 1Password 3 to 1Password 4 or 5? If it doesn't coincide then we're probably looking at the wrong thing.

Can you look on the dropbox website... they have a feature called 'Events' where you can see what happened with every sync of dropbox. If you can find the dropbox sync event where these files went away, it should tell us what else changed with this, and which device triggered this change. It would help confirm or deny that this was an upgrade migration issue.

This is what an Event detail looks like here, and from this I can tell that I modified an item by deleting an attachment (other edits may have taken:

If you manage to find that dropbox event where this took place, if you could save it as a PDF and get it to us, that'd be great. I could imagine that you would not want to be posting that directly onto a public forum, so feel free to private message that to me.

Rick

ref: OPM-2646

jjspreij

hi @rickfillion,

Can you confirm that this time machine machine backup difference happened at or around the time you upgraded from 1Password 3 to 1Password 4 or 5?

No, I upgraded to 1P v4 & v5 quite soon after they were available. So by 7 October 2014 I must have been using v4 everywhere.

But here is the Dropbox event where most of the entry files dissappeared:

Just before it happened, I linked a new MacBook to the Dropbox account. So what I think more or less happened:

• the MacBook just had a new SSD with a clean empty system installed
• created a new OS X account on the MacBook
• needed to install quite a few apps, for which serialnrs are needed, which are in 1P, which first requires Dropbox...
• installed Dropbox, which then needed to sync about 3 GB
• installed 1P and linked it to Dropbox

Maybe I was too soon with the last step, and pointed 1P at the 1Password.agilekeychain file in Dropbox before it was fully synced. Then it might have read the partial contents, and rewritten back that partial set, deleting the other entry files which had by then arrived. Either that, or the newly installed 1P had some other issue with the 1Password.agilekeychain file...?

jjspreij

Hi @Megan, @Rad, @rickfillion,

Just wanted to let you know that I followed Rad's advice and all is working perfectly again - thanks!

(If I can be of any more help, let me know ;-)

--jj

sjk

Thanks for those additional details, @jjspreij, which I've included with the bug report that @rickfillion previously filed. I also changed (at)Rick to @rickfillion in your last two replies so they're referring to the "right Rick". :)

Glad to hear that Dropbox syncing of your 1Password data is now working perfectly again after following @Rad's instructions!

We'll let you know if there's anything else you can help with. Thanks for that offer and all of your help so far!

rickfillion

Hi @jjspreij‌,

That would make sense. Since the AgileKeychain is comprised of many many files, and if I remember correctly dropbox tends to download newest to oldest when installing on a new machine. I wonder if maybe 1Password should be doing a sanity check of the AgileKeychain to ensure everything is reasonably setup in there before allowing a user to setup sync to it. In your case simply making sure contents.js exists, then every item listed in contents.js should have a .1password file or a tombstone corresponding to it... that would have caught it. I'll have to think about that for a bit before convincing myself that that's a good idea.

I'm quite relieved that it wasn't the 1P3->1P4/5 transition that did this. I'm sorry that this happened to you though. And if possible I'd like to try to make sure it doesn't bite someone else in the future.

Rick

jjspreij

Since the AgileKeychain is comprised of many many files, and if I remember correctly dropbox tends to download newest to oldest when installing on a new machine

Indeed, that makes a lot of sense @rickfillion ;-) Sanity check might be a good idea; good luck, and thanks for all your help! --JJ

rickfillion

I filed a case yesterday (OPM-2654) so that we can consider the implications of the sanity check and possibly build it into a future release.

Thank you for your help.

Rick

ref: OPM-2654