1PIF import is broken

zfil

Hello

I wanted to move to OPVault format from my existing Agile Keychain so I followed the procedure described here: https://support.1password.com/switch-to-opvault/win.html

The resulting new vault lost data:

• Logins with multiple url: only the first url is kept
• Identity: everything is lost besides the title
• All icons are lost
  There may be other things lost I didn't do an exhaustive search.

Moreover during the import the deduplication code is broken, for instance:

• 2 similar Login in different folders are considered duplicates (in my case one was in the trash)
• 2 Software with the same Title are considered duplicates (2 differents versions ...).

Instead of going trough an intermediary file format (with all the risk that gives) why not building a dedicated format converter that does the conversions and give the user confidence of what was actually done (ie. stats on the object converted, diffs etc ...)

I trust 1Password to store valuable information for me and I don't want to loose anything.

Regards
Philippe

---

1Password Version: 4.6.0.BETA-591
Extension Version: Not Provided
OS Version: Windows 7 SP1
Sync Type: Dropbox

Maroder

Hi,

I just tried to change to OPVault format in Windows (Dropbox Sync).

After the Import I was missing multiple items in the new vault only because they had the same URL or name. Also some accounts where missing completely for whatever reason. Furthermore other "Accounts" where listed in the password section after migration.

The export and import to OPVault in its current state is completely unreliable. Why is a export/import so hard to accomplish within the same application and even version?

How you find it save to give Dropbox (hello NSA) the possibility to read the URLs of my password safe is beyond me. The whole situation does have a very negative connotation for an application I entrust with all my password and I will keep my eyes open for a new password manager.

Regards
Mark

MikeT

Hi guys,

@zfil,

Logins with multiple url: only the first url is kept

I can confirm this and when I pulled the data out of the item in OPvault, it did have the second URL, so the data is there but the UI isn't showing it. I'll file a bug report to fix this.

ref: OPW-514

• Identity: everything is lost besides the title

Can you tell me if all of them are older than 2012? I tried exporting all of my Identities from agilekeychain and imported it in OPvault, they all showed up fine.

Can you try something for me:

1. Select the affected Identity item and go to the Help Menu > Advanced > Copy JSON
2. Open Notepad, do not open any text editors that may create files in cloud folders.
3. Paste the item's JSON by pressing Control + V or go to the Edit Menu > Paste.

Do you see all of your Identity data? If yes, delete the content and close Notepad without saving anything.

All icons are lost

Are we talking about Rich Icons that 1Password downloads automatically for you or do you mean custom icons you've added manually to your items?

Rich Icons are not exported since 1Password would re-download it for you as the icons are not stored in your data file but in a local cache.

2 similar Login in different folders are considered duplicates (in my case one was in the trash)

I'm not sure why they should not be considered duplicates, the folder and trash are just organization methods that do not change the item itself. Do you mean you have two Login that has the same titles with different data in it beside the folder location?

• 2 Software with the same Title are considered duplicates (2 differents versions ...).

Did you get the second prompt to not overwrite these duplicates?

The reason we have a second prompt is that some folks use 1PIF to update items and overwrite them based on the titles and if they want to keep it, they can choose to import it as duplicates instead of overwriting.

Instead of going trough an intermediary file format (with all the risk that gives) why not building a dedicated format converter that does the conversions and give the user confidence of what was actually done (ie. stats on the object converted, diffs etc ...)

That's something we're working on as mentioned in the blog post but it will take quite a bit of time, we have a lot of edge cases because of the various format upgrades over the years and we have to find them all first before we can do a better job with the automated tool with more detailed changes.

MikeT

Hi @Maroder,

After the Import I was missing multiple items in the new vault only because they had the same URL or name.

We made a mistake in our instruction and we will update it. When asked to replace items based on identical titles, it should be No to all. I've updated it here and requested our docs team to update it in the Knowledge Base.

We will clarify this in a future update.

Also some accounts where missing completely for whatever reason

Do you see titles only with the rest of the data is empty? We're seeing reports of old items showing up like that and are investigating. Based on what we're finding, the data is in there but the interface does not show it. We might be able to fix it by updating the interface to show these data. To confirm this, can you try this:

1. Select the affected Account item and go to the Help Menu > Advanced > Copy JSON
2. Open Notepad, do not open any text editors that may create files in cloud folders.
3. Paste the item's JSON by pressing Control + V or go to the Edit Menu > Paste.

Do you see all of your account data? If yes, delete the content and close Notepad without saving anything.

We tested this with relatively recent databases and that's why we missed these bugs, they were with much old databases as far as we can see. We do apologize and will fix these ASAP.

Furthermore other "Accounts" where listed in the password section after migration.

Can you explain what you mean by this?

How you find it save to give Dropbox (hello NSA) the possibility to read the URLs of my password safe is beyond me.

We make no excuses but we've explained the rationale in not encrypting the metadata and the agilekeychain format were available long before any cloud services including Dropbox were available. Our formats are available for everyone to review and to study, including our recent OPvault format here.

We're working around the clock to get OPvault out to everyone sooner but we have to make sure our changes are not causing issues like the ones reported here.

zfil

Hi @MikeT

Thanks for the explanation to handle the duplicates.

For the missing Identity I can confirm that the element was created in 2009 and last modified 22/08/2012.
The cut and paste JSON contains all my datas.

For the icons I agree it is the case for the sites (but it is a pain to have to re-download), but for the applications or the identities the icons are not downloaded, so it is not great to lose them!

Anyway thanks for looking at this feature, I'm going to wait you have finished the migration tool :) But please make sure you provide way to compare the diffs before/after the migration!

Thanks

MikeT

Hi @zfil,

For the missing Identity I can confirm that the element was created in 2009 and last modified 22/08/2012. The cut and paste JSON contains all my datas.

Ok, that suggests the interface is the issue or we need to convert it differently. Would you mind waiting for us to figure out a solution and get in touch with you with a potential fix?

but for the applications or the identities the icons are not downloaded, so it is not great to lose them!

Ok, these are custom icons then, not Rich Icons. Custom icons are stored in your items and I just tested, it is the same issue with the Identity item, it is still there but the program is not displaying it. You can do the JSON test to confirm this if you want, you'd see it in the "customicon" field.

I've filed a separate bug report to get this fixed.

ref: OPW-515

Thanks for reporting these issues, we appreciate it. If you see anything else, please let us know.

zfil

I don't have the customicon field in the JSON, and to be honest I don't even know how I've added these icons ...

I guess I've done it with the OS X version (I don't have a Mac anymore ...). Speaking of that when the windows version will be on par feature wises ;) ie. tag support is quite crappy on the windoze side ...

Otherwise yes fell free to contact me if you need to test things ...

Cheers

MikeT

Hi @zfil,

I don't have the customicon field in the JSON, and to be honest I don't even know how I've added these icons ...

Wait, do you see it in the agilekeychain vault on Windows?

I guess I've done it with the OS X version (I don't have a Mac anymore ...).

Did you export the 1PIF file from OS X or was this done on Windows?

Can you clarify all the steps you took and the versions of 1Password you're using, it'll help us narrow this down.

Speaking of that when the windows version will be on par feature wises ;) ie. tag support is quite crappy on the windoze side ...

You can get the list of tags if you press Alt + Down in the search field.

We don't release details until we're ready as per our policy but we do plan to improve this in a big way.

Otherwise yes fell free to contact me if you need to test things ...

Thanks!

zfil

Hi @MikeT,

Wait, do you see it in the agilekeychain vault on Windows?
Did you export the 1PIF file from OS X or was this done on Windows?
Can you clarify all the steps you took and the versions of 1Password you're using, it'll help us narrow this down.

Initially the vault was created on OS X (don't remember the version but it was in 2009). Now since at least 2 years I think I don't use anymore OS X. Only windows and iOS ...
I synchronize with Dropbox.

So I see the custom icon for my identity (actually this my photo) on Windows version 4.6.0.BETA-591 (but not on iOS version Pro 6.0.2). I see as well custom icons for the Software on Windows and iOS but this is not symmetrical. For the most part the same icons are displayed but for some case there is a custom icon in Windows and not iOS and vice versa ...

I've exported and imported the 1PIF on Windows.

Tell me if you need more details ...

Cheers

MikeT

Thanks for the extra details, we're still looking into it but we have an idea what it can be.

As for the custom icon, it looks like it is a separate issue from the other one with the different formats. We have a couple of ideas on how to fix this and we're walking through them to pick the best solution. Stay tuned.