MrC's Convert to 1Password Utility (mrc-converter-suite)

BenWaldman

Thank you , MrC. I eventually got things to work, but my situation was a bit different than described in your document. After pasting, I was asked for the password to the "Local Items" keychain (I have no keychain with this name - perhaps it is a cache of the iCloud keychain that the OS maintains?) - my account login was the password. Then, for each item, I was asked for the password to the iCloud keychain (which again, was my local account password, not my iCloud password) - I was never asked for the password to the local-iCloud keychain which I had created. Then I used a modified script based on your description of the race conditions (I made the delay 1.0, and also added a delay between entering the text and pressing the ok button, which seemed to avoid errors). I still had one more class of errors - some items would just not copy from the iCloud keychain - it turns out they were all application passwords that ended with .accountsync (for email accounts and Twitter). (I wonder if logging out of iCloud or turning off iCloud passwords would have solved all this). Anyway, just thought I'd pass on my experience - thanks again for all your work.

MrC

Hi @BenWaldman,

Thanks for the follow-up. I wish I knew how to create the out-of-sync Keychain condition, so that I could document how to recover from that scenario. Alas, I suppose users will be left to resolve the situation themselves until then.

Good to hear that you are now converted and imported into 1Password. Enjoy!

buggypac

Hi MrC, I tried converting my LastPass export (.txt) using the Apple Script but none of the URLs seemed to have made it when importing into the latest 1Password. Nothing customized. Any ideas?

MrC

Hi @buggypac ,

Can you explain how you created the export file?

It doesn't make sense that URLs are missing, as they are just cells value in the CSV file. I'm guessing the export file was not stored in the correct format, and you copied/pasted from a separate browser window. Also, what browser did you use?

buggypac

Hi @MrC, I created it as instructed in the readme. I used Chrome to export the info, but since it creates an HTML file, I copied the contents and saved it as a .txt instead.

MrC

Hi @buggypac ,

As per the README:

Note: The LastPass extension for Chrome incorrectly opens the export as an HTML document. If you are using Chrome as your browser, do not use the extension to start the export. Instead, either use the LastPass extension in another browser, or use the Tools > Advanced > Export item in the left sidebar when your vault is open in the browser (you will have to use the copy / paste method mentioned in the note above).

Don't use the Chrome-created HTML - it is not in the right format. You must use one of the alternate methods mentioned.

buggypac

Hi @MrC, the second method was the one I used:

or use the Tools > Advanced > Export item in the left sidebar when your vault is open in the browser (you will have to use the copy / paste method mentioned in the note above).

MrC

@buggypac,

I just tested the method you said you used. In the Chrome browser, I used the left-side sidebar, and selected Tools > Advanced Tools > Export, entered my password when asked, and it opens a new window labeled https://lastpass.com/export.php. Then, I selected all the text on the page, Copied, and then opened TextEdit and I made sure that TextEdit is in Plain Text mode (TextEdit > Preferences > New Document > Format set to Plain Text), and then Pasted the text in the new document, and finally saved is is pm_export.txt onto my Desktop. The converter runs fine:

$ perl convert_to_1p4.pl lastpass ~/Desktop/1P_import.1pif -v Imported 20 items Exported 1 email item Exported 2 note items Exported 1 creditcard item Exported 1 driverslicense item Exported 1 passport item Exported 1 socialsecurity item Exported 1 wireless item Exported 2 server items Exported 3 login items Exported 1 software item Exported 2 bankacct items Exported 1 database item Exported 3 membership items Exported 20 total items You may now import the file /Users/MrC/Desktop/1P_import.1pif into 1Password

Try running the converter in debug mode, to capture the first few lines. Run the following, replacing the name/path to your export file as appropriate

perl convert_to_1p4.pl lastpass ~/Desktop/pm_export.txt -v -d  |  grep print_fileinfo

I get the following:
print_fileinfo : Export file info print_fileinfo : size: 4514 print_fileinfo : kind: UTF-8 Unicode English text print_fileinfo : mime: text/plain; charset=utf-8

What is your result?

Edit: But now I'm also wondering. Did everything else but the URLs get imported correctly?

What are the results of your verbose output (like mine above)?

sunrocket

MrC, thank you for all your work on this.

I just downloaded both the "stable bits" and "testing bits" zip files, but cannot decompress the files. Error message for both: "A failure has occurred. The structure of the archive is damaged."

Can you please advise, and forgive me if you've already covered this in earlier posts.

Want to convert Mac Keychain v. 9.0 to 1Password 5.4.1

Thanks in advance.

MrC

Hi @sunrocket ,

Perhaps you have some A/V or other software interfering? I just tested a download of the Testing Bits and the archive is fine. You might have to clear your browsers cache, and re-download. Perhaps trying another browser.

sunrocket

Thanks for your ultra-prompt response, MrC -- I downloaded in Opera Beta (vs. Safari), and archive opened fine.

Regards.

MrC

@sunrocket. Bingo!

afrederickson

Hi MrC,

I have tried the conversion utility, but i have so many custom types created, it seems to be importing the wrong data into the wrong fields. I can rearrange the field types for each one i created, but I was wondering how should i modify the .pm script to make this work. I have copied certain types and given it my own names and field types, but trying to make this easy so i dont have to retype this all into 1Password. Thanks in advance.

MrC

Hi @afrederickson ,

Can you tell me which password manager you are converting from?

afrederickson

sorry, that would be helpful, mSecure, v3.5.4

MrC

@afrederickson ,

There is a table at the top of the Msecure.pm converter, called %card_field_specs. It is a table of category and field definitions for mSecure. At the top of each category entry in that table is a textname value, which is the string used for comparison against the exported entry (column 2). Each field definition within a category entry maps to the columns in an exported entry, and defines the mapping of export to 1Password internal per-category field names. The number of fields in the category entry must match the number of columns output in the CSV (after removing the first couple of common columns).

Example exported entry, split by comma for easier reading:

Business,
Bank Accounts
Wells Bank
what a shady bank^Kthis is.
12345
9999
shady sam
west branch
555-1212

Columns 1 through 4 have common meaning to all entries in the mSecure export.

Column 1 is the Grouping or Tag, and it is "Business".

Column 3 is the entry's title "Wells Bank"

Column 4 is the Notes column.

Column 2 specifies the (stock) card type, and here it is "Bank Accounts". It matches the textname 'Bank Accounts' in %card_field_specs, so the import type is 'bankacct' and the fields in this definition are used (shown below).

The first four columns are shifted out, and the remaining columns are defined by the relevant %card_field_specs category entry:

    bankacct =>                 { textname => 'Bank Accounts', fields => [
        [ 'accountNo',          0, 'Account Number', ],
        [ 'telephonePin',       0, 'PIN', ],
        [ 'owner',              0, 'Name', ],
        [ 'branchAddress',      0, 'Branch', ],
        [ 'branchPhone',        0, 'Phone No.', ],
    ]},

The first column now in the row is the Account Number, and it maps to the 1Password bank account field with the keyname 'accountNo'. Likewise, the next column is the PIN and it maps to the internal keyname 'telephonePin'. The remainder of the mappings should be fairly obvious.

So, if you've changed the name of the category (e.g. from Bank Accounts to My Bank Accounts, then you'll have to change the textname accordingly. If you happen to have both stock names and new names, you can duplicate the entry and change the textname in the new entry, and you'll also need to rename the %card_field_specs key 'bankacct' for your new entry to something unique such as 'mybanking'. Finally, you must add a mapping called type_out that specifies where 'banking' entries should go in 1Password, and that would look like:

    mybanking =>                 { textname => 'My Bank Accounts', type_out => 'bankacct', fields => [
        [ 'accountNo',          0, 'Account Number', ],
        [ 'telephonePin',       0, 'PIN', ],
        [ 'owner',              0, 'Name', ],
        [ 'branchAddress',      0, 'Branch', ],
        [ 'branchPhone',        0, 'Phone No.', ],
    ]},

There's more to it, but that's the basic idea. Also, the names are localized from a localization string table if it exists. If you are using English, there's nothing to worry about.

Hope this helps!

afrederickson

Thanks for this. If i need to know the keyname of other fields within 1Password, how to do i get those?

MrC

@afredrickson,

The file Utils/PIF.pm has a table called %pif_table, which defines every field in every 1Password category. It defines a field's data type, keyname, text label, the location within the record's sections, and additional options.

The connecting component between input and output fields is the key. The %card_field_specs table specifies the key value, and this key value must reside in the category's entry within the %pif_table for the data to be mapped into an existing field. Otherwise, the field gets mapped to Notes as a field: value pair (unless the -addfields option is specified, in which case custom fields are created using the field's label).

There are some caveats with some data "kinds", such as dates, addresses, monthyears, etc. Such fields require validation or certain data formatting to be valid. But for simple string types ($k_string), the most common type, there's nothing to worry about.

gagga

I have an export from HandySafe 5.10 (NOT Handysafe Pro). This is a txt file, not a XML file.

I tried to use your converter but assume it only supports the XML export? Any idea how to import my text file into 1password?

Full text file (with sample data) here:
https://dropbox.com/s/9qyyjdwgjgnhvog/handysafesampleexport.txt?dl=0

MrC

Hi @gagga,

Where can I find this software? Is this for OS X or Windows? I don't recognize that version number.

gagga

Hi Mr.C
It's Windows. Very old. From 2008. It's the predecessor of Handysafe Pro from Epocware. I don't even have the installer file anymore but valuable data in one of these export files.

MrC

@gagga,

I'm unable to find a download from the Internet Archive's Wayback machine.

Looking at the text file output, it is essentially free-format text, with some rudimentary delineations. I have no idea about the comprehensive list of possible card types/categories, their fields and field semantics, label uniqueness, quoting schemes, etc. It may not possible to parse this file unambiguously (certain data could trigger incorrect parsing of entries).

I think your best bet would be to get this data into a spreadsheet, placing entries into the required column format so that you could export to CSV. You'd want to format the data as required for 1Password CSV importing.

Since this is such an old password manager, it isn't worth spending much time on this. However, it would be easy to write a quick converter for that export data to convert to CSV, but it may have errors, since the Notes section data may contain lines that begin with a : or even possibly "[Card, Icon: ##]" (but this pattern seems unlikely). You could manually adjust the resulting CSV, or even adjust the input until the results are satisfactory. That would be the best that can be done, without knowing the exact field names, field semantics, etc.

afrederickson

Hi MrC,

I am picking this up after the holidays :-)

My 1Password just did an update to 5.4.2

I slightly modified the msecure.pm file and now get this error message:

---

Error: failed to load converter module 'msecure'
syntax error at Converters/Msecure.pm line 5, near "I# mSecure CSV export converter
#

Copyright 2014 Mike Cappella (mike@cappella.us)

package Converters::Msecure 1.00"
BEGIN not safe after errors--compilation aborted at Converters/Msecure.pm line 11.
Compilation failed in require at convert_to_1p4.pl line 38.

Usage: convert_to_1p4.pl

converters:
clipperz csv dataguardian datavault essentialpim ewallet handysafe ironkeyim keepass2
keepassx keychain lastpass licensekeeper msecure msecure copy nortonis onepif2html
passpack passwordagent passworddepot passwordwallet safeincloud safewallet spbwallet
splashid vcard

specify one of the converters above on the command line to see complete options

---

just trying to make sure the change i made to the .pm file is not the cause vs. the upgrade.

afrederickson

Nevermind, :-0

I had made a syntax error..

MrC

@afrederickson ,

It looks you have some errant character in front of the # character on line 5.

MrC

I've promoted version 1.08 as Stable Bits. The list of changes and new features is large:

Version 1.08:
New Converters:
- RoboForm converter now available.
- Keeper Desktop converter now available.
- Txt2notes converter now available, converts text files into Secure Notes.
- F-Secure KEY converter now available.
- Passwords Plus converter now available (OS X).

New Features:
- Option --addfields or -a will create custom fields for non-mappable fields instead of placing them into notes.
- Option --tags or -t will add the specified comma-separated list of tags to each record.
- The keepassx converter will decode and convert an entry's attachment. It is placed in a folder
named 1P4_Attachments in the same location that the 1P4_import.1pif file will be created. An entry's attachment is placed in a sub-directory named with the entry's Title.
- The vcard converter option --securenote will place vCard data into a Secure Note, with custom fields similar to an Identity.
- The 1PIF module supports icons if either of the Perl 'GD' or 'Image::Magick' libraries are available.
- The vCard converter will include icons when the --icon option is specified (and the required library is available - see previous note).
- Added more import types to handysafe converter; added a multiple field-matching system to assist category detection.
- The handysafe converter now has support for localalization (lang nl-nl). More languages will be added as requested.
- Languages directory for per-converter language translation files.
- Added output diagnostic to onepif2html converter to indicate the name of the created output file.
- Add support for 1Password's 'Password' category.
- The csv converter now accepts a Tags column. The comma-separated values within that column will be imported as 1Password Tags.

Changed:
- The --help option is a little more instructive.
- Converter ewallet's combolock and voicemail categories go to the Password category now.
- Converter splashid's frequentflyer now maps to the 1Password Rewards category.

Fixed:
- Several fixes for the onepif2html print converter: strip the BOM on Windows 1PIF exports; gracefully handle unknown 1PIF typeNames; support the legacy typeNames allowed in the Windows UI.
- Minor Perl syntax changes throughout the code to satisfy Perl versions > 5.16.
- Remove extraneous, but harmless setting of imptypes in essetialpim converter.
- Remove extraneous comment in msecure converter; fixed duplicate key in frequentflyer definition.
- Splashid converter was not honoring --imptypes for user-defined types (user defined types are specified as 'userdefined'.
- Splashid converter was not converting \b end of line characters into newline characters in an entry's notes section (Windows only).
- The debug function unfold_and_chop() was not honoring its maxlen parameter.
- The debug function print_record was not outputing notes.
- Date parsing was broken in some cases for converters ewallet and lastpass.
- Several ewallet converter fixes.
- The safewallet converter was incorrectly detecting a web login category as an irc category.
- Several vcard converter fixes: 1) prevent vcard converter from importing an encoded photo into the notes section; 2) converter was bailing after the first entry in a list of vcards. 3) Uses a new module for parsing / packaging vCard data (Text::vFile::asData). 4) Handles vCard exports via iCloud.
- The handysafe converter was not picking up root-level nodes.
- Some debug output errors in Utils::PIF.pm.
- A date converter routine for some converters could cause an error under anomalous conditions - affected converters: dataguardian, ironkey, keepass2, keepassx, keychain, passwordagent, passworddepot, spbwallet.
- The datavault converter on Windows could botch some multiline entries in the notes section.
- The onepif2html converter was not outputing a label for unlabeled URLs.
- The onepif2html converter was not handling the Password category.
- Converter keychain now avoids outputting a 0x0 (nil) hexvalue as a URI protocol.
- Converter keychain export instructions for iCloud keychain export was missing part of the fully-qualified keychain name (e.g. local-icloud should have been local-icloud.keychain).

Internal Changes:
- Miscellaneous code refactoring and consolidating.
- New module Utils::Normalize.
- Simplify the lastpass converter's %card_field_specs table.
- convert_to_1p4 will now accept and pass to a converter more than a single filename argument.
- Modified the %card_field_specs table to include custom field definitions within the table.
- Made the normalize_card_data() function universal, and moved it to Utils::PIF.pm.
- Code updates to work with Perl versions newer than the old requirement of Perl 5.16.
- Debug output will include info about the export file supplied on the command line.
- Many code cleanups.

stace3

Hi there, I am trying to covert from Spb Wallet but receive the following error message:
C:\Users\Stacey\Desktop\convert_to_1p4>perl convert_to_1p4.pl spbwallet -v ..\My_Wallet.txt
Enter your SPB Wallet password:
DBD::SQLite::db prepare failed: no such table: spbwlt_Template at Converters/Spbwallet.pm line 232, line 1.
Can't call method "execute" on an undefined value at Converters/Spbwallet.pm
line 233, line 1 (#1)
(F) You used the syntax of a method call, but the slot filled by the
object reference or package name contains an undefined value. Something
like this will reproduce the error:

    $BADREF = undef;
    process $BADREF 1,2,3;
    $BADREF->process(1,2,3);

Uncaught exception from user code:
Can't call method "execute" on an undefined value at Converters/Spbwallet.pm line 233, line 1.
Converters::Spbwallet::do_import('..\My_Wallet.txt', undef) called at convert_to_1p4.pl line 116

I am using Spb Wallet Version 2.1.2 Build 12118

This is all beyond my skillset but your step by step instructions have been very helpful to this point. Any advice on what to do now?

MrC

Hi @stace3,

The spbwallet converter is unique in the conveters - it works on the actual wallet file itself, not a text export. Instead of the path "..\My_Wallet.txt", provide a path to your .swl wallet file. I don't know where that is on you system.

From the SPB Wallet section in Section 3:

● SPB Wallet

There is no need to export any data from SPB Wallet. The converter will read and decrypt the data directly from the SPB Wallet .swl file. It might be easiest to move your .swl wallet file to your Desktop. When you enter your command line, use your .swl wallet file’s name as name_of_export_file.

If would prefer not to move the .swl file, you’ll have to specify a path to your wallet file in place of the relative path ..\name_of_your_spbwallet_file.

dszp

I didn't see https://discussions.agilebits.com/discussion/comment/263822/#Comment_263822 prior to using the converters to convert my Mom's LastPass data to 1Password. Fortunately, saving the "text" page to a text file using Copy/Paste to Notepad mostly worked--the only issue I've found is that all ampersands (&) anywhere in the document (including in titles and passwords) were converted to "&" (the HTML entity) in 1Password, which I manually fixed in the few places I could find it. Hopefully no other issues; haven't seen any so far anyway :-) I don't recall seeing a "don't use Chrome" note in the readme files while running the conversion tool, guess I should have reviewed the forums first as well. Thanks for the converters!

MrC

Hi @dszp,

Thanks for the nice remarks, and I'm most pleased that you were able to help our your Mom! I wrote the very first converter for my Mom, so here's to good, 'ol Moms everywhere.

FYI: From the LastPass export notes in Section 3 of the README:

Note: The LastPass extension for Chrome incorrectly opens the export as an HTML document. If you are using Chrome as your browser, do not use the extension to start the export. Instead, either use the LastPass extension in another browser, or use the Tools > Advanced > Export item in the left sidebar when your vault is open in the browser (you will have to use the copy / paste method mentioned in the note above).

It is the Chrome extension that incorrectly creates an HTML document (and hence, the HTML entities); 1Password has no role in this. The data is already HTML encoded incorrectly by LastPass.