MrC's Convert to 1Password Utility (mrc-converter-suite)

sjk

@tajchiu: It would be awesome if the 1Password importer gave me an error indicating that I need to remove the header from the file!

@MrC: It would be even more awesome-er if 1Password just removed the header for you!

@tajchiu: That's fair - I was sort of thinking the same thing!

Thanks for the suggestions, guys! I've filed a new issue that 1Password for Mac recognize/skip headers when importing 'popular' CSV format files (which also refers to an existing issue about displaying more detailed import failure messages). I agree both would be useful improvements. :+1:

ref: OPM-1306, OPM-5762

MrC

@bpiec,

I've updated the csv converter in the 1.10 version of the converter suite to remove a possible BOM, and have updated the README.pdf to indicate the CSV file should be UTF-8 (and that a BOM is not required, but tolerated).

Thanks again.

MrC

Today I updated version 1.10 of the converter suite, currently in Testing Bits, to include the ability to check for known compromised passwords. From the Changes.txt file:

• The converter suite now has the ability to safely test for insecure password using the Troy Hunt
  Pwned Passwords list. This feature is enabled using the --checkpass option. See the README.pdf
  for more details. The code used to perform these checks is not loaded when the option is not
  supplied.

and from the README.pdf:

I'm hopeful this will be very useful to many.

AGAlumB

@MrC: So great! Thank you! :love:

Lars

@MrC - Oh, that's awesome! 👍🏻

Azurite

Just a note that the script linked to on GitHub is slightly different from the current (as of Feb 28, 2018) script in the Readme.pdf I was using as I attempted to migrate an iCloud keychain to a local keychain for conversion purposes. The password "set" is in the middle, after the "try" and that's what helped the script work for me! :chuffed:

ETA: However, for some strange reason, the AppleScript Conversion Helper app isn't seeing my local-icloud.keychain in the Keychains folder. Do I need to do something to get it to see it?

MrC

@Azurite ,

The GitHub repository is version 1.09. Testing Bits is version 1.10, and you should be using it.

I don’t think I understand what you mean by the set and try? Can you clarify?

What is the output of the command:

security list-keychains

run in a command shell?

Azurite

The Testing Bits is what I grabbed, but the code in the Readme.pdf that came with it (for the Script Editor) didn't work for me. This did:
tell application "System Events" repeat while exists (processes where name is "SecurityAgent") tell process "SecurityAgent" set frontmost to true if (count of windows) > 0 then set window_name to name of front window end if try keystroke "password" delay 0.1 keystroke return delay 0.1 on error -- do nothing to skip the error end try end tell delay 0.5 end repeat end tell

The slight difference is in where the password is defined and when it's used. For whatever reason, for me, on 10.13.3, the script in the Readme just kept running and running while the SecurityAgent window was opened-- it never actually "typed" anything. (And I made sure to have Accessibility enabled, the correct folder on the Desktop, etc).

The adjustment to the code is what worked for me.

I was also able to fix the Helper app not seeing my keychain; I had the Keychain on my Desktop, rather than in the default location of ~/Library/Keychains. I needed to import it to Keychain Assistant since I guess that app only sees what's in the Keychains folder. I needed to restart in order for Keychain Assistant to look in the folder again.

MrC

@Azurite ,

Sorry for the troubles.

I'm totally fed up with the problems created by the key-pressing AppleScript trying to get around the security command's prompting. So, I'm decrypting the keychain myself, and am re-working the converter to decrypt a keychain natively. Standby....

FYI: I've already fixed the problem with the AppleScript helper not showing all the keychains - the next update will show all that security knows about.

DavidMBrown

I am astonished to discover that @MrC is still providing this invaluable service to 1Password users. Kudos to you, sir.

Lars

@DavidMBrown - Awww man, DB -- how are you? Long time, no hear from. :) Hope all is well.

MrC

Thanks DB, long time indeed! Yup, I'm still grinding away, slave-drivers these Agilebits customers are!

DavidMBrown

Missing much about the interaction with folks like you, @Lars and @MrC, but doing fine.

Azurite

@MrC, I want to thank you so so so so much for providing this invaluable tool. Any issues I had with the script or the helper I think were entirely of my own misunderstanding of how to do things, and I ultimately figured everything out and successfully got the Helper app to take my local-icloud Keychain and convert it to a 1Password pif file that I imported into my account. I mainly wanted to post my experience in the event someone else experienced the same issues and wanted to know what I did to fix it.

Thank you again! :)

AGAlumB

@DavidMBrown: <3

@Azurite: I myself understand almost nothing until I try — and usually fail — to do it a few times. So glad to hear that you persevered, and MrC's converter helps you as well. Cheers! :chuffed:

MrC

The keychain converter now natively decrypts macOS keychains!

This eliminates the need to use the security command to dump decrypted data, along with its gazillions of prompts, and of course this eliminates the horribly clumsy Applescript code to enter the user's password ad infinitum.

And its fast - for me, the best case (when a user has certain modules available), it takes about 1/2 a second. Worst case, less than 4 seconds. Rejoice, Sing, Frolic.

@Azurite / @brenty

AGAlumB

@MrC: Frolic, sing, and rejoice I will! That's freaking amazing!!! :love:

MrC

I have just posted an update to the 1.10 version of the converter suite. It now includes a native chrome converter. It will decrypt and convert your Chrome form fill data (Logins, Credit Cards). Both macOS and Windows is supported.

AGAlumB

So great! Thank you! :chuffed: :+1:

Eamonh

I've just added a pull request to the github repository (https://github.com/agilebits/onepassword-utilities) to fix a bug with the keepass2 converter and add support for exporting the createdAt field.

It looks like that repository has not been updated in a year so just wondering if there is a better place to submit my changes?

Eamonh

Ok, so just discovered that support for the createdAt field exists already in the "testing bits". It would be great if this code could be made available on the github repo as a another branch, would have saved me a couple of hours trying to understand the code and re-implementing the feature :(

However, it appears that the typo still exists in "testing bits", specifically line 203 of Keepass.pm it has "LastModificationtime" instead of "LastModificationTime" (capital T for Time) which causes the check at line 82 to fail. I also think line 126 needs to be updated too.

dannye23

I've been through this a few times and cant seem to get past the error Can't open perl script "convert_to_1p4.pl": No such file or directory

I created a local keychain and it finds it but then always get this error. Same with when i try to convert from Chrome. I'm on high sierra, is that the problem,? Apple script 1.10

Thanks

Daniel

MrC

Hi @Eamonh ,

I post updates to Agilebits when I'm able/ready. Their repository is just that - theirs. I have, and will always have, the master code base. I make very frequent changes, and Agilebits cannot keep up with the pull requests I'd need to submit. So, by definition, their repository will always be dated. I could have my own Git repository, but like Testing Bits, users like yourself likely would not have found it directly either.

Re: Keepass2's LastModificationtime vs. LastModificationTime: thanks. I've just updated Testing Bits. Thank you for the assist on this.

MrC

Hi @dannye23 ,

It sounds like you are using the AppleScript helper?

Where did you locate the convert_to_1p4 folder itself? Did you unzip the archive and place it directly on your Desktop?

dannye23

Yes. It automatically unzipped into download folder and I dragged and dropped to desktop. (Downloaded from test bit's as instructed) Same error when I tell it to use Chrome option. Could it be high sierra blocking it somehow?? Or I'm being a doofus and missing something?

MrC

@dannye23 ,

This is one of those cases which is always hard to assist users remotely, since I don't have eyes on the results/problem, and have to rely on user interpretations, and their working environment is different than mine.

For security, macOS runs the AppleScript in a private safe area. When it does this, the AppleScript was not seeing the folder convert_to_1p4 on your Desktop because I recently made a change (to help my testing), to allow the folder to be anywhere. I'd forgotten about the macOS change of folders to the private area.

I've just updated the 1.10 version in Testing Bits mentioned in the first post of the converter suite thread. Please grab that version and use it, replacing what you have, and put the convert_to_1p4 directly on the Desktop. Later, I'll add some code that allows you to tell the AppleScript where the convert_to_1p4 folder is if it is not found on the Desktop.

Sorry for the troubles.

dannye23

No need to apologise it's amazing work you do here. Ok I'll give it a go when home and report back, thanks!

dannye23

Success! Thank you! amazing work, thanks again!

MrC

@dannye23 ,

Thanks, and you're welcome. I've already updated the Applescript helper to ask you to locate the convert_to_1p4 folder if it is not found on the Desktop.

Edit: I see you posted just before me. Nice! Enjoy 1Password!

Edit 2: You may very well be the first person to use the new keychain converter's raw decryption code, and the new chrome converter. Good to get a report it works! :-)

MrC

I have just posted an update to the 1.10 version of the converter suite. It now includes an enpass converter. I've currently only tested it on macOS, but will get to Windows shortly.

Please note, while the Enpass CSV export is valid CSV, there are several issues:

1. There is no information in the records to indicate the category, so the converter uses key field matching heuristics to detect a category.
2. The CSV export eliminates empty fields from the export, so some category detection may be impossible under certain circumstances.
3. Some records duplicate certain field names. The converter tries to map these fields to the correct 1Password field, but due to 2 above, such fields may not map as you'd like or expect.
4. If you've renamed any of the key field labels, the converter may not detect the category. The converter can be easily customized to handle this, and I'm happy to help you do that if necessary.
5. Enpass does not do any date validation - dates are simply random string values in Enpass, and therefore are meaningless. The converter does not attempt to parse any date values, and they will be placed into the Notes section of the record.

Enpass has a ridiculous number of categories / record templates (some of which make no sense for a password manager, such as Flight Information and Hotel Reservations). I've handled all of them with the exception of a few esoteric categories beneath the Other grouping. This just means they will go to Secure Notes, and you cannot specify them using the --include or --exclude options.