Data loss after migration to OPVault
I have been a 1Password user since 2010. I have more than 600 entries in my vault. I use 1PW on my iPhone, iPad, Windows PC (work) and on my Mac (at home).
I decided to switch from Agile Keychain Vault to OPVault in Windows using the instructions at https://discussions.agilebits.com/discussion/39875/getting-your-data-into-the-opvault-format because of the recent discussions regarding the security and privacy of the old keychain.
After the migration, I ended up with significant amounts of data missing:
1) Many software licenses that had been created in the early years (2010) were almost "naked": they had their name, but no licence code and nothing else. Items that have been created more recently apparently were not affected.
2) A lot of attachments were missing.
I stopped investigating the issue and I do not know if I lost even more data. I reverted back to my Agile Keychain backup and got rid of the incomplete OPVault data.
You really need to implement a reliable process to validate and ensure the consistency and integrity of the OPVault after migrating from the Agile Keychain. Right now, I do not see myself migrating to the new vault.
I highly encourage everybody to check thoroughly, if your OPVault contains all your data. Otherwise, you might be in for a horrible surprise. 1Password contains very sensitive and important data. It has to be secured against unauthorized access. But it has to be secured against data loss, too.
1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Sync Type: Not Provided
Comments
-
@Christian_H Hey, I did change my vault from agilekeychain to OPVault yesterday and did notice some missing software licenses and attachments as well. In regards to the licenses the problem, in my case, was that I had multiple licences with the same name differentiated just by the version number. I then changed the name to display the version number as well and every license was then exported and imported correctly to OPVault.
In case of the missing attachment I don't know what is causing the issue but I had one attachment missing which I then attached manually afterwards again. All other attachments are in place correctly. I did notice one strange behaviour: After exporting all the objects in from my old agilekychain I had a folder named attachments in my documents folder. There were all attachments of my secure notes stored after the export. I deleted the folder securely but was shocked to see sensitive attachments just stored on my computer without me granting the permission or having the knowledge that 1Password did something like that. Maybe an official Agilebits member can look into and report on this....
0 -
Hi guys,
I do apologize about this issue, we're looking into this and it is strange that it is not importing data correctly for items that are older than a specific timeframe. I believe what is happening is that we might be exporting certain items with its older format version and the import only handles the newer format. If my assumption is correct, that means once we add the older format to the import process, it should then import the rest of the data properly but I need your help to confirm this.
@Christian_H and @JohnnyFJohnsson, if you don't mind helping us out with this, could you please do this:
- For Johnny, switch back to the original agilekeychain vault via the File Menu > Open 1Password Vault. Chris, skip this as you're back on the original vault.
- Find your pre-2010 software license items that were missing the rest of the data. Select one of them and then go to the File Menu > Export, select 1PIF format and this time, select Selected items. Save it to its own folder in your Documents > 1Password folder.
- Open the export folder, open the 1PIF file with NotePad, does it show all of your fields or just some metadata about your note and attachment?
- Delete the export folder after this.
2) A lot of attachments were missing.
Do you recall if they were also pre-2010 or not a single attachment was imported?
In case of the missing attachment I don't know what is causing the issue but I had one attachment missing which I then attached manually afterwards again
Do you recall if there's anything special about that attachment, was there a weird character in its name, what type was it, was it an image? Any details you can give us will help us improve this.
After exporting all the objects in from my old agilekychain I had a folder named attachments in my documents folder
That's part of the export process, all attachments are exported to its own attachments folder. Inside, you'd see folders with names that matches the item UUIDs for which the attachments belong to.
We didn't do a good job to explain you need to delete the attachments folder. I've updated the original instructions to explain this, we do apologize for this.
0 -
I solved the issue by doing the migration on my Mac using 1Password for Mac. It worked flawlessly with the Mac version. I did not even loose icons (in logins or software). Unfortunately, I do not have the old Agile Keychain available to me (I deleted it after checking the integrity of the new OPVault).
As far as I recall, the missing attachment issue also affected at least one software license that had been purchased in 2013.
If you want me to, I might be able to recover the old Agile Keychain from a backup in order to do further checks.
0 -
@MikeT The missing attachment was for a secure note with no special characters in its title or anything. The only thing I can think of is that the note is called "... Wiederherstellungsschlüssel" using the german letter ü (ue). That might have caused the problem migrating to OPVault.
Thanks for explaining. I was wondering what the attachment folder might be and if it is safe for me to delete it because (except for the one attachment detailed above) all the attachments were migrated sucessfully to the OPVault without me even touching the attachments folder.
0 -
Hi guys,
If you want me to, I might be able to recover the old Agile Keychain from a backup in order to do further checks.
Can you hold on to that backup, in case we want to come back to you. We have another user with the same issue and we're working with him to see if we can reproduce it and have a fix out soon.
Mac. It worked flawlessly with the Mac version.
I'm glad to hear that. If you're willing to help us test this further, so we can improve our import process on Windows, please let us know. I know it takes a lot of time to test this for us but we'd be curious to know if the export from the Mac version works to import within the Windows version.
The only thing I can think of is that the note is called "... Wiederherstellungsschlüssel" using the german letter ü (ue). That might have caused the problem migrating to OPVault.
I can confirm this issue, we'll investigate this and fix it.
Thank you so much for reporting this!
0 -
@Christian_H you're very welcome
0 -
Hi @JohnnyFJohnsson,
I assume you meant me, not Christian.
Unfortunately, I got some bad news. While I was able to reproduce it initially, further attempts didn't reproduce it. We've tried several dozens of combinations and still can't get it to break. This may explain why it didn't affect everyone.
Any chance you can try exporting it from an old vault or backup and try again?
0 -
Can you hold on to that backup, in case we want to come back to you.
Ok, I will keep my old Agile Keychain for the time being. Feel free to contact me.
I know it takes a lot of time to test this for us but we'd curious to know if the
export from the Mac version works to import it in the Windows version's import process.Ok, I will do so. I don't have access to the PC until tomorrow, I will get back to you then.
0 -
Hi @Christian_H,
Ok, I will keep my old Agile Keychain for the time being. Feel free to contact me.
Thanks, we appreciate it.
Ok, I will do so. I don't have access to the PC until tomorrow, I will get back to you then.
Be careful with the 1PIF file, do not use any cloud based syncing tool as the export file is not protected.
0 -
Be careful with the 1PIF file, do not use any cloud based syncing tool as the export file is not protected.
No problem, I am using a USB stick for this transfer. I just did the export from the "old" Agile Keychain on my Mac and will try to import the 1PIF file on the PC tomorrow.
0 -
That's great! Just make sure you don't lose it. :smile:
0 -
I don't know if this is relevant, but I had discovered and reported that 1Password 4 for Windows would fail to import 1PIFs created by 1Password 2.
0 -
Ok, here is the story:
I exported my data from the old Agile Keychain on my Mac to the unencrypted 1pif data format using the latest 1Password for Mac version.
During the import of the Mac-generated 1pif data, 1Password for Windows seemed to have crashed ("not responding"). After ignoring the according Windows message for about 2 minutes, 1Password signalled that the import of X items was successful after all. 1Password did not crash.
In contrast to Wednesday's import of 1PW for Win-generated 1pif data, no attachments were missing when importing the Mac-generated data. Slight bug though: the Mac export generated folders with attachments and Mac OS generated their invisible file system counterparts [attachment: file.pdf, counterpart: .UNDERSCOREfile.pdf]) like Mac OS does. Maybe, 1Password for Windows should realize that ".UNDERSCORE-files" with the same filename should not be imported. Importing those invisible files as attachments does not make sense.
No icons "survived" the import in 1Password for Windows (logins, applications/software), they are gone (did not happen on the Mac) for some reason. At least some (or all, didn't really check thoroughly) of the icons survived the export and import, when being done on the Mac.
In short:
The data loss problem seems to originate from the 1pif export out of 1Password for Windows, because using 1PW for Win to import Mac-generated 1pif data seems to work ok.0 -
Hi @Christian_h,
Importing those invisible files as attachments does not make sense.
These are resource forks created by OS X. We will investigate to see if there is a better alternative that is more cross platform and if not, we will try to avoid importing the resource forks.
No icons "survived" the import in 1Password for Windows (logins, applications/software), they are gone (did not happen on the Mac) for some reason. At least some (or all, didn't really check thoroughly) of the icons survived the export and import, when being done on the Mac.
That's intentional, no rich icons are saved in your data file, these are downloaded on the fly from our image service into a local cache on your drive. This allow us to keep your data files lean, leave the bigger rich icons on our server.
The exception is when you add a custom icon of your own, that gets saved to your data file and that should be in the export file. Do you recall if you ever added your own custom icons or did you let 1Password download everything?
For 1Password on Windows, try this: open the main 1Password program, unlock, and go to the File Menu > Download Rich Icons.
Our Windows version does not support Rich icons for the software license, that's coming in the future.
The data loss problem seems to originate from the 1pif export out of 1Password for Windows, because using 1Password for Win to import Mac-generated 1pif data seems to work ok.
Huge thanks, that will help us narrow it down.
0
