MrC's Convert to 1Password Utility (mrc-converter-suite)

ivory

Thanks for the fast response @MrC ! I had to admit that I did not change the language :( but still for 455 login details, only 125 are exported correctly, the rest (330) are exported as notes. Software, servers and credit cards are fine too. Could it be due to the notes present in some of the logins?. It also says "(1 item expanded to 2 items)". Thanks

MrC

@ivory

The enpass converter detects categories based on certain keywords in the export. The converter has a table of fields for each stock Enpass category. Example:

    creditcard =>               { textname => '', fields => [
        [ 'cardholder',         1, qr/^Cardholder$/, ],
        [ 'type',               0, qr/^Type$/, ],
        [ 'ccnum',              0, qr/^Number$/, ],
        [ 'cvv',                1, qr/^CVC$/, ],
        [ 'pin',                0, qr/^PIN$/, ],
        [ '_expiry',            0, qr/^Expiry date$/, ],
        [ '_validfrom',         0, qr/^Valid from$/, ],
        [ 'creditLimit',        1, qr/^Credit limit$/, ],
        [ 'cashLimit',          0, qr/^Withdrawal limit$/, ],
        [ 'interest',           0, qr/^Interest rate$/, ],
        [ 'bank',               1, qr/^Issuing bank$/, ],
        [ '_tpassword',         1, qr/^Transaction password$/, ],
        [ '_issuedon',          0, qr/^Issued on$/, ],
        [ '_iflostphone',       0, qr/^If lost, call$/, ],
        [ 'url',                0, qr/^Website$/,               { type_out => 'login' } ],
        [ 'username',           0, qr/^Username$/,              { type_out => 'login' } ],
        [ 'password',           0, qr/^Password$/,              { type_out => 'login' } ],
    ]},

Each item in the export contains a set of Key/Value pairs. It is the Key name that is used to hint at the category. When a sufficient number of specific key names are found in a record, the category is then set.

When users customize their categories, or the field names, automatic conversion then becomes more challenging, since each user's export is then unique and understood only by that user.

I can likely help you by customizing the converter to detect your export better. Contact me offline for this - my email is at the top of the convert_to_1p4.pl script. I've helped many users this way, and we can avoid polluting this thread with back and forth details uninteresting to others.

About the 1 item expanded to 2 items - this occurs when a password manager's single category really works better in 1Password as two or more categories. 1Password really wants Login's (i.e. website, username, password) to be in the Login category, so that auto-fill can work. Likewise Credit Cards, and a few others. Some password managers put all this type of information into a single record. The converter will split these into the appropriate 1Password categories. You can see in the table above the instruction type_out => 'login' - this is what controls this, and generally it should not be changed (or 1Password really won't work as designed and you'll end up moving the data, or creating additional records manually).

pet

is the source code publicly available? If not, would you please put it in a public repo? :)

MrC

@pet,

The code is, and has always been, located at the links mentioned in the first thread of this forum post. It is script code so that folks like you can read it and be certain it isn't doing anything malicious.

TC1

Question re: Keepass2 data import to 1Password!

• Latest 1Password build for Windows
• Windows 10
• Stablebits utility

Hello! Thanks for the tool MrC, I've been able to correctly utilize it thus far to convert and import my data, with one exception I am requesting assistance for!

Upon converting and importing, I find that all 209 of my records are stored in a lump list alphabetically. I then tried the -f extension with the script only to find the converter simply segregated the list into sections separated by the first letter in the name of the record "a" "b" etc etc and so on.

Keepass2 is arranged for me into toplevel "sections" or "folders" such as "Career" "Internet" "Homebanking" and so on. I am attempting to have your converter replicate this subdivision of data yet have been unable to find a way to correctly manipulate the script or the data within Keepass to allow the converter to do its job correctly.

Any ideas?

MrC

Hi @TC1

I think you're wanting version 1.11, in Testing Bits mentioned in the first post of this thread, because of this change:

Version 1.11:
Changed:
- keepass2: Changed group separator from :: to / so that it will nest as Tags using 1Password's tag hierarchy.

Recent 1Password's have eliminated the Folders, and moved to Tags.

TC1

Hi MrC,

Thank you for the quick (less than 5 minute) reply! I will re-download the Test version of the utility and begin the process again. Will report back on my success or lack thereof.

On a side note: It is unfortunate that 1Password still does not allow the removal of the primary "Personal" Vault in Windows. I spent a bit of time attempting to find a way to remove the 209 records from the online Vault that was created when I initially set up the import, only to find it was impossible other than item by item. I could create a secondary vault and work on the conversion within it (which I did), but the primary would remain incorrectly imported and limited in ability to quickly edit large number of records or delete the 209 records altogether to prep for a re-import attempt.

Additionally, should I have attempted a second import (for example having used the -f modifier) into the primary vault, the data is duplicated (an anticipated result). However, the Windows version does not provide the functionality (that exists in the Mac version) to remove duplicate entries :(

My solution was to cancel my entire account and delete 1Password, and start over. The only way I could find to create a fresh start environment after having created the initial Vault database. Ah well, I'm sure these features will be added at a later date?

MrC

Hi @TC1 ,

You're welcome - hopefully the updated version addresses your issue.

I don't manage my data via the web version (I use the Desktop version for this), so can't comment much on its interface. Perhaps Agilebits will address this soon.

I do a lot of import testing with 1Password, and find the best method is to either use a separate vault, or I add a Tag using the --tag option:

 -t or --tags <list>        # add one or more comma-separated tags to the record

Then I can select the entire lot and delete them in one shot. Don't use -f for what you are trying to do - Folder names are not discoverable by the converter, so it just blindly adds a new Folder to the 1PIF, and 1Password dutifully creates a new (same-named) one.

1Password used to add an import timestamp tag to items just imported - I don't think it does this anymore.

TC1

Thanks for the tip, @MrC.

Wanted to note a weird quirk while downloading and extracting the Testing version of your utility.

Downloaded the utility. During attempted extraction, "invalid archive" errors and the zip appears empty. I can move through the subfolders in your dropbox fine and see the 644kb file in my downloads. But for some reason the archive shows as empty? Windows... ugh.

Anyway, quickly downloaded 7zip and it extracted just fine. Not sure if anyone else ever ran into that but thought it was odd considering the Stable release of the utility extracted just fine with Windows embedded extraction tool.

Anyway, back to converting...

TC1

@MrC

I completed the conversion using 1.11 Test app (this time into a new vault named TEST). The list is still bulk and arranged in alphabetical categories. However, I can now see that the Tags are present "GENERAL/EXAMPLE" example being one of many such as Homebanking, Shopping, Internet, etc.

I can View the entries by both Category and Title, but neither properly sort the list. How might I utilize the Tags (which show the correct division of data, aside from the unnecessary "General" top level folder) to move the entries into Categories or sub-categories?

The Tags appear to be clickable, but doing so within the desktop app 1PW does not group the entries by Tag. Apologies if this is getting out of your wheelhouse.

MrC

TC1,

I'm sorry about the zip troubles. Something is broken with creating zip archives that work under Windows via macOS Mojave's command line zip utility. I've been building these zip files for ages, and had no idea these were broken on Windows (since my recent Mojave update). The Stable Bits archive works because I haven't updated that archive since macOS High Sierra.

I'm using a different archiver now, and it appears that compatible zip files are now created. I've updated Testing Bits.

RE: your other questions - these would be better asked on the platform-specific forum area, since they are not really relevant to the converter, and each platform has slightly different features and support specialists. But some quick replies below.

I think I had misunderstood your issue. When you say "the list", I am guessing now that you mean the list of Items in your vault, vs the Folders or Tags list.

The Items sort is controlled by the Sort selector (it is limited on the Windows app):

The top level "General" tag comes from your Keepass2 data. You can just move the items or rename the tag. It's a bit easier on the Mac side.

You cannot move 1Password items into new categories. An item's category is selected at the time of creation.

I don't know about the click-ability of tags in the Windows app.

AGAlumB

@TC1: If you'll start a new discussion in the Windows category of the forum I'm sure we can answer any questions you have about that app. Sorry to do that to you, but I don't want to get this discussion off topic, and to be honest I'm not sure off the top of my head what you mean by "does not group the entries by Tag". I'm sure if you'll go into a bit more detail we can offer some insight though. :)

TC1

Fair enough, and thank you both for the attention to this forum, gents!

AGAlumB

99.99% of the thanks on this topic goes to MrC, but I'm happy to contribute in any way I can. :)

ivory

Just stopping by to say THANK YOU @MrC for the script and helping me out, really appreciate this! :) :) AWESOME work

MrC

@ivory ,

It was my pleasure. Thanks for the follow up.

Enjoy 1Password!

kakaarsch

I want to export my icloud keychain to a local keychain in order to import it to 1Pass but I keep gettint this error:
An error has occurred. Unable to add an item to the current keychain. The contents of this item cannot be retrieved.

I am not sure how to "reset" my icloud keychain as I´m not even using icloud. Is there anything I can do about this?

MrC

This is an update to my earlier posts about using the Get_Safari_Passwords scripts to copy your Safari passwords and place them into a 1Password importable 1PIF file. Instructions are below for both Safari 12 and Safari 11.

In Testing Bits, mentioned in the first post of this converter suite thread, there are two Applescripts named Get_Safari11_Passwords and Get_Safari12_Passwords. Use the one that matches your Safari version. You can open and examine these with Script Editor if you have concerns about how they operate.

Instructions

1. For Safari 12, download Get_Safari12_Passwords
   For Safari 11, download Get_Safari11_Passwords
2. Open the script with Script Editor
3. Launch Safari
4. Open Safari's Preferences and navigate to the Passwords tab
5. Unlock the dialog so your list of credentials is visible
6. Switch back to Script Editor and hit its Run button

If all is successful, it will cycle through your rows of login credentials, and export them to a file on your Desktop named pm_export.csv. The script will then close Safari's preferences dialog, and open a dialog to inform you that the export has completed. You can now convert that pm_export.csv file with the csv converter in version 1.11 of the conversion suite (also in Testing Bits).

Note: You may need to allow Script Editor to access Safari's UI elements. To do this, find the Security & Privacy area in System Preferences and allow Script Editor to control your computer:

Once I've worked out any kinks, I'll incorporate the scripts into the AppleScript_Conversion_Helper.

I have not yet updated the README.pdf instructions for this version (or the original version.

I'd be interested in any feedback or issues - please let me know if there are any issues, and even a quick post here telling of your success.

kakaarsch

I get this error when trying to convert the csv file:
Uncaught exception from user code: malformed JSON string, neither array, object, number, string or atom, at character offset 0

MrC

@kakaarsch ,

I'm sorry, can you please give me some more context? If you were using the command line, what was the exact command you used?

There are no JSON strings or JSON code in the csv converter. Add the -d option to the command to see more context.

I presume the export from Safari using the Get_Safari##_Passwords script worked.

kakaarsch

@MrC
This is the whole output. After I use this command:
'perl convert_to_1p4.pl onepif -v ../pm_export.csv'

Uncaught exception from user code: malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "www.joyclub.de,www.j...") at /Users/anooob/Desktop/convert_to_1p4_1/./Utils/PIF.pm line 1006. JSON::PP::decode_error('malformed JSON string, neither array, object, number, string ...') called at /Users/anooob/Desktop/convert_to_1p4_1/Modules/JSON/PP.pm line 1045 JSON::PP::word() called at /Users/anooob/Desktop/convert_to_1p4_1/Modules/JSON/PP.pm line 728 JSON::PP::value() called at /Users/anooob/Desktop/convert_to_1p4_1/Modules/JSON/PP.pm line 688 JSON::PP::PP_decode_json('JSON::PP=HASH(0x7fa69c1ed510)', 'www.joyclub.de,www.joyclub.de,,XXX', 0) called at /Users/anooob/Desktop/convert_to_1p4_1/Modules/JSON/PP.pm line 148 JSON::PP::decode('JSON::PP=HASH(0x7fa69c1ed510)', 'www.joyclub.de,www.joyclub.de,,XXX') called at /Users/anooob/Desktop/convert_to_1p4_1/Modules/JSON/PP.pm line 110 JSON::PP::decode_json('www.joyclub.de,www.joyclub.de,,XXX') called at /Users/anooob/Desktop/convert_to_1p4_1/./Utils/PIF.pm line 1006 Utils::PIF::get_items_from_1pif('../pm_export.csv') called at /Users/anooob/Desktop/convert_to_1p4_1/./Converters/Onepif.pm line 92 Converters::Onepif::do_import('../pm_export.csv', undef) called at convert_to_1p4.pl line 139

MrC

@kakaarsch ,

I see the problem. You're using the onepif converter, instead of the csv converter.

Use:

perl convert_to_1p4.pl csv -v ../pm_export.csv

Chubstr

Hey @MrC - super basic question - I used the Convert utility and it outputted a .1pif file, but I can't import it into 1password since it's not a CSV. I'm guessing I missed something in the process to convert the file to a CSV? Can you tell me what I need to do from here? Thank you!

MrC

1PIF is the input format you want for the Desktop version - it is a more robust, cross-platform 1Password file format.

Which version of 1Password are you using, and which platform, @Chubstr ?

Chubstr

@MrC - problem solved! I was trying to import to the web version, not the desktop version. Thanks for your help!

MrC

Cheers @Chubstr. Enjoy 1Password.

sengelking

Hi, when using the "Get_Safari12_Passwords" scrip I get the following error:
error "„System Events“ hat einen Fehler erhalten: „text field 1 of sheet 1 of window \"Passwörter\" of application process \"Safari\"“ kann nicht gelesen werden. Ungültiger Index." number -1719 from text field 1 of sheet 1 of window "Passwörter" of application process "Safari"

I've changed "Passwords" to "Passwörter" as my Mac is setup in German. The error occurs when it tries to copy the data from a website that is set to never save the password

MrC

@sengelking ,

Thanks for reporting. Are you saying there is an entry in Safari's passwords for the particular website, but the value of the password is empty?

sengelking

There’s an entry for the website.
The username field just contains a - and in the password field it said it is never saved. Both fields are greyed out in the dialog and are not editable

MrC

@sengelking ,

Ok, I'll take a look and post an update. Shall I ping you when the update is posted?