1PIF file format documentation?

jimgarrison

I see from a quick examination that the .1pif format is just JSON. Is the structure of the JSON objects documented?

I need to convert a database of about 500 entries from SafeWallet to 1Password. I have examined the existing converter written by MrC and find that it supports SafeWallet up to version 2.4.1.2, which exports CSV. My version (3.0.7) will only export XML.

What I would like to do is get documentation for the .1pif format so I can write an XSL transform to convert SafeWallet XML directly to .1pif JSON. If necessary I could reverse engineer the JSON object structure from MrC's Perl code but that'll probably take longer than just re-entering all my data, and since this is a one-time operation it would not have much future value other than as an intellectual exercise.

Is the .1pif structure documented anywhere?

DBrown

You're in luck, @jimgarrison‌!

A generous forum member, @MrC, has contributed exactly what you need. You can read all about it in this article in our general knowledgebase.

Please let us know how it goes.

jimgarrison

Thanks. I've already seen that article, which describes the utility but not the .1pif JSON object layout. As I noted, MrC's converter will work only for previous versions of SafeWallet, so I will need to write my own XSL transform to read the XML exported by SafeWallet 3 and convert it to JSON (.1pif). In order to do so I need to know how to structure the JSON objects.

I was hoping to find a document describing the JSON object layout. If this does not exist it may not be worth my time reverse-engineering it from the Perl code, since it's unlikely to help anybody else.

rob

Hi, @jimgarrison‌.

We do not currently have the 1PIF format documented, but if you create a demo vault in 1Password (under the 1Password menu), you can export all the items in the 1PIF format and see how things are structured.

Edit: Sorry, my instructions were for 1Password for Mac, but you can still create demo data in 1Password for Windows and export it as a 1PIF file to see the structure.

DBrown

As I noted, MrC's converter will work only for previous versions of SafeWallet...

Yup, I totally missed that. Sorry, @jimgarrison!

MrC

@jimgarrison‌,

No worries, I think I have you covered. You want the 1.01 version of the converter which supports:

Version 1.01:
- New: Update SafeWallet converter to support the Windows 3.x XML version.

Look in the _testing directory of my AgileBits Utility (link mentioned in the main converter thread).

jimgarrison

I downloaded the latest and ran it. As a developer with over 30 years experience, with about 20 years writing Perl, I tip my hat to your Perl coding and software engineering skills. This is very well organized and modularized, and eminently readable (which for Perl says a lot :-). My earlier reluctance to reverse-engineer from the source was unfounded. Your code can serve as documentation for the 1PIF format for any Perl developer.

I do have some questions:

• The converter assigned 1P "types" in a way that doesn't match how I was using types in SafeWallet. Almost all entries ended up as "Server Accounts" in 1P. Is there a way to change an entry's type in 1P after it is created? If not, how can I better control the mapping?
• In SW you could have entries with the same title in different folders; the import process won't import two entries with the same name. I guess I'll have to clean up the source before converting.
• Speaking of folders, how hard would it be to copy over the folder structure into the 1P database?
• For SW entries with custom fields, the converter merged them all into the notes field. Is there a way to have it create similar custom fields in the 1P entry?

Is the 1.01 source on GitHub? I could fork it locally and look at making some of these enhancements.

MrC

Thanks for the very nice feedback @jimgarrison,

I'll reply to your questions above; but let's take any additional programmatic type questions regarding the converter offline, since they probably won't be of much use to anyone but a couple of gear-heads like you and me. :) My email is at the top of the script files.

The converter assigned 1P "types" in a way that doesn't match how I was using types in SafeWallet. Almost all entries ended up as "Server Accounts" in 1P. Is there a way to change an entry's type in 1P after it is created? If not, how can I better control the mapping?

You can't change the type after they've been placed into the 1PIF or imported into 1Password, so this needs to be handled before conversion.

Each converter has its own method for determining an entry's category (type). In the case of SafeWallet, categories (types) are not exported, so these are determined based on the presence of the entry's field names. Some field names are unique to an export type, and thus, can be used as a (semi-) reliable means to determine the type. The %card_field_specs table in Converters/Safewallet.pm has the list of exported types, along with the stock field names for the SafeWallet item. You'll see some 0's and 1's down a column in the table - those indicate if the field's presence can set the type. If you've customized your SafeWallet, you may have added a field name that is currently being used to detect a type. The 'server' type allows the string 'IP' or the string 'RCON' to set the 'server' type. I'm guessing you have added an IP field somewhere, and so those items are being detected as 'server'. If I've guessed correctly, you can then just set the corresponding 1 value in the 'IP' field under 'server' to a 0, and IP won't be used to select the type any longer. And unless you have an 'RCON' field set, no server types will be detected. The server category is a tough one, because there aren't enough fields to uniquely identify it and distinguish it from others.

There are a couple of other SafeWallet types that get converted to 1Password 'server' categories. These are SafeWallet's 'irc' and 'password' categories. The same logic mentioned above applies, but these entries have a "type_out" value set to coerce the output type. Examine the fields for these categories, esp. those that allow the type to be set (those values with a '1' just left of the RE used for matching field names).

In SW you could have entries with the same title in different folders; the import process won't import two entries with the same name. I guess I'll have to clean up the source before converting.

1Password for Windows gives you the option to import items with the same names - you have to click the Yes to All button on import. I don't care for how the Windows version ignores the UUID and instead uses the item's title field for duplicate detection; it is broken by design, and I've reported this.

Speaking of folders, how hard would it be to copy over the folder structure into the 1P database?

The converter places a password manager's folder structure or groups into 1Password's Tags. Creating folders in 1Password is not officially supported, and cannot be done safely if folders already exist in the vault since there's no way to query and obtain existing folder UUIDs and names. Another user asked about the possibility of creating folders for empty vaults, and this is something I'll look into (but it requires possibly some larger changes to Utils/PIF.pm than I was willing to undertake these past few weeks). It is on my To Do list to explore.

For SW entries with custom fields, the converter merged them all into the notes field. Is there a way to have it create similar custom fields in the 1P entry?

Yup, that's correct. Custom field names can be added to an entry, but you have to do so with care. The basic idea is that you tell PIF.pm about the new field entry via the add_new_field() function. This is done in the converter's do_export() function. See the examples there in Converters/Safewallet.pm and see the function add_new_field() in PIF.pm. There are existing section names in 1Password, and these are specified at the top of PIF.pm (see $sn_branchInfo or $sn_main, for example). You can create your own sections too, just pass a unique section key/title pair as a dot separated string (see examples in Converters/Ewallet.pm). Be sure the field name key you use doesn't already exist in PIF.pm for some other value, esp. for the specific category. Some types (e.g. $k_menu or $k_date) require special data handing - so you have to take care here. And some fields should be concealed, or not present a password generator, etc. These flags can be found in other add_new_field() examples in the various converters, and of course in the %pif_table table inside PIF.pm (key => value pairs after the t=text label column).

If you want to see more revealing data as the converter is running, add the -d option and pipe the output to a pager such as less.

I'm sure this should be enough to get you started...