MrC's Convert to 1Password Utility (mrc-converter-suite)

AGAlumB

Holy cow! That's amazing! :love:

g76sCk8Cb

Signed up just to leave a comment:

MrC, you mentioned this

"A few others have in the past had trouble getting the Date::Calc module to build on Windows 10, and I'm not sure why. I'm able to build it in both 32 and 64-bit environments. The problem is with Bit::Vector.

Which Perl version exactly did you downoad?
What is your Windows architecture?"

This can be solved by using cpanm instead of cpan to install the module.

MrC

@g76sCk8Cb ,

Thanks for taking the time to post about this issue. I have not seen anyone report the issue in quite a while now, and since that time, I'd created a batch script to help Windows users do the module installation.

Without being able to reliably reproduce the problem, I can't advise on solutions, since any such "solution" is not testable. Do you have a means to reproduce the issue using the latest Strawberry Perl portable packages?

devstuff

Would it be possible to add CSV-style support for the PasswordSafe converter?

On my Mac (Sierra) I'm using the file-format-compatible pwSafe UI which only offers the CSV export option.

MrC

@devstuff ,

I have no idea what that mysterious UI-only app exports, so could not comment. I'd have a lot of questions to ask, but before we go down that path, have you determined if the csv converter can handle the export?

ya_icloud_victim

@MrC,
What field names should I use to import createdAt and updatedAt with the CSV importer?

MrC

@ya_icloud_victim ,

Those are special time stamps that are handled as meta-data, and not as simple fields. They must be in stored in the 1PIF as special Unix epoch time values, and the CSV converter can have no idea what date formats are stored in a user's CSV cell. So they are not currently supported by the csv converter.

Where did you're dates come from, and what is the format of their value?

ya_icloud_victim

@MrC
They've come from mangled keychains that I've hand de-duped. I've converted them to 1PIF time format.

MrC

@ya_icloud_victim ,

Why not just use the newly updated keychain converter, which manages this for you?

I suppose I could add support for the Epoch time stamps as CSV fields. I would have to do some strict checking of the values present, and do some range checking.

ya_icloud_victim

@MrC Thanks for the prompt responses! I had two mangled keychains with ~500 records so I de-duped them in a spreadsheet before dumping to a CSV. I'd be happy with a --dontbestupidwithtimestamps flag if you wanted to skip the input checks :-)

MrC

@ya_icloud_victim ,

I'll take a look at adding these date fields to the csv converter. I still have to verify that the values are sensible, or an import can crash 1Password. Bad values in a 1PIF have been known to crash 1Password in the past, and it is hard for users to figure out why, and I don't want to spend time trying to debug these types of issues.

MrC

@ya_icloud_victim ,

I've just updated the 1.10 version in Testing Bits, mentioned in the first post of the converter suite thread. It updates the csv converter to support two new columns: modified and created. These are Unix epoch values. If present, and valid, they will be used to set the time stamps.

ya_icloud_victim

Awesome! Thanks MrC now I'm all sorted (once I fix my mistake of exporting localtime rather than UTC from the source.) Some trivial suggestions: (a) create_pif_record doesn't report the two new fields in debug mode, (b) you might make note that the times should be UTC, not localtime, in the docs (I'm amazed that your release notes and README.pdf were both updated already!), and (c) you might consider stripping commas, periods and quotes from the fields. My first export didn't parse because I had the numbers as formatted text. I read almost all of these 40 pages before making my request to you. You are an extraordinary asset to this community. Thank you for helping yet another complete stranger!

MrC

@ya_icloud_victim ,

You're welcome.

You've discovered several of reasons I had not supported this metadata in the csv converter. User's need to know about Unix epoch time, 1Password's internal expectations for these values, about the differences between a string representation of an integer vs. a spreadsheet's pretty-printed version, how to properly construct a CSV and ensure that data exported is in the format expected, etc. It's all too much for almost all users.

I'm not going to do any clean-ups on these fields - there are too many hidden gotchas, and for any user who wants to use them, they are going to have to really understand the issues above.

I don't anticipate much usage of these fields by csv converter users.

mbierman

Hi!

I wanted to get my old passwords from Chrome (Version 65.0.3325.181 (Official Build) (64-bit)) and import them into 1Password (6.8.8) on a Mac (10.13.4).

I exported a CSV from Chrome and it comes with the headers:
name,url,username,password

but the conversion doesn't seem to like that... saying, "CSV column names do not match expected names"

I will go try to read through the Perl and see if I can reverse engineer what it is looking for, but I thought I'd ask since no doubt someone knows the answer.

Update:

Looks like I found the answer:

[ 'title',      0, qr/^title$/i, ],
[ 'url',        1, qr/^website|url$/i, ],
[ 'username',       1, qr/^username$/i, ],
[ 'password',       1, qr/^password$/i, ],
[ 'notes',      0, qr/^notes$/i, ],
[ 'tags',       0, qr/^tags$/i, ],

Thanks!

MrC

Hi @mbierman ,

No need!

The 1.10 version of the converter has a chrome converter that decrypts and converts the Chrome form-fill data - no need to export anything. It is in the 1.10 release, in Testing Bits, mentioned in the first post of the converter suite thread.

mbierman

I didn’t see that in the reader but I’m all set thanks.

MikeSafari

Man, I am having nothing but problems getting this to run. My "Desktop" folder is actually redirected to my D:\ drive as my C:\ drive is a smallish SSD, which I suspect is what's causing the problems.

I can run portableshell.bat from C:\myperl without issue, but the install_modules.bat step doesn't work 100% since I think it's looking for the Desktop folder in C:\ rather than D:. I've manually changed this to the correct path and it works fine and installs all the modules. I've got my KeePass exported XML file in the same Desktop folder to which the rest are extracted.

After that, though, I simply can't run the perl convert_to_1p4.pl command that actually starts the conversion.

I've manually put the files in %USERPROFILE%\Desktop\ with no change. When I run it from there, I get:

Can't locate Utils/PIF.pm in @INC (you may need to install the Utils::PIF module) (@INC contains: C:/myperl/perl/site/lib C:/myperl/perl/vendor/lib C:/myperl/perl/lib) at C:\Users\USERNAME\Desktop\convert_to_1p4\convert_to_1p4.pl line 15.
BEGIN failed--compilation aborted at C:\Users\USERNAME\Desktop\convert_to_1p4\convert_to_1p4.pl line 15.

Any ideas?

I'm using 1PW 6.8.534.0 (downloaded and installed today), importing from KeePass 2.38 on Windows 10, 1709.

MrC

Hi @MikeSafari ,

Sorry for the troubles.

First, be sure to use the 1.10 version in Testing Bits, mentioned in the first post of the converter suite thread.

Second, it doesn't matter where you place the convert_to_1p4 folder - so long as you've changed directories within the portableshell.bat launched command shell. I routinely run it from my O: drive on Windows. You can just change drives in the command shell by typing D: and hitting Enter, and then doing a cd command to the path where you've placed it.

Third, the install_modules.bat is just a convenient helper. Just adjust the cd command inside, or perform the steps manually. The modules only need to be installed once, and the cd afterwards just gets you ready for the convert_to_1p4.pl command you'll run.

I think these there adjustments should be all you need.

MikeSafari

Hey @MrC, that did it! I was changing directories slightly wrong and also using an outdated version of the converter that I found through the official 1Password import/export page.

Thanks!

MrC

@MikeSafari ,

Outstanding. Enjoy 1Password!

cindyhoskey

HI, @MrC I'm hoping you can help me.

I'm trying to convert a chrome export. I have been following the directions in the ReadMe file in convert_to_1p4. I have the latest 1Password installed. I downloaded your Testing Bits version. I downloaded and installed Strawberry Perl as per the readme. I've been working my way through various issues with directories, etc. and I finally seem to have the right things in the right place, but now I'm getting errors. Here is what I ran and the output:

**C:\Users\cindy\Desktop\convert_to_1p4>**perl convert_to_1p4.pl chrome -v C:\Users\cindy\Desktop\ChromePasswords.csv

DBD::SQLite::db prepare failed: database is locked at C:/Users/cindy/Desktop/convert_to_1p4/./Converters/Chrome.pm line 94.
Can't call method "execute" on an undefined value at
        C:/Users/cindy/Desktop/convert_to_1p4/./Converters/Chrome.pm line 95 (#1)
    (F) You used the syntax of a method call, but the slot filled by the
    object reference or package name contains an undefined value.  Something
    like this will reproduce the error:

        $BADREF = undef;
        process $BADREF 1,2,3;
        $BADREF->process(1,2,3);

Uncaught exception from user code:
        Can't call method "execute" on an undefined value at C:/Users/cindy/Desktop/convert_to_1p4/./Converters/Chrome.pm line 95.
        Converters::Chrome::get_db_entries(undef) called at C:/Users/cindy/Desktop/convert_to_1p4/./Converters/Chrome.pm line 153
        Converters::Chrome::do_import("C:\\Users\\cindy\\Desktop\\ChromePasswords.csv", undef) called at convert_to_1p4.pl line 140

Is this something you can help me with?

Thank you in advance.

MrC

@cindyhoskey ,

This will happen if Chrome is still open/launched. Please be sure to quit Chrome before running the converter.

cindyhoskey

So it's pulling from Chrome directly and not from the export file I specified in the command?

MrC

@cindyhoskey ,

That is correct. You do not need to export from Chrome. It reads the raw Chrome form fill database, and with your password (which you supply), it decrypts the database for you.

From the README.pdf:

cindyhoskey

Yup, I didn't read it as thoroughly as I thought I had.

So, I completely shut down chrome, then i completely killed it in Task Manager. Now when I run the following and get this error:

C:\Users\cindy\Desktop\convert_to_1p4>perl convert_to_1p4.pl chrome

Uncaught exception from user code:
Win32::API::Call: parameter 7 had a buffer overflow at C:/Users/cindy/Desktop/convert_to_1p4/./Converters/Chrome.pm line 205.
Converters::Chrome::decrypt("\x{1}\x{0}\x{0}\x{0}\x{d0}\x{8c}\x{9d}\x{df}\x{1}\x{15}\x{d1}\x{11}\x{8c}z\x{0}\x{c0}O\x{c2}\x{97}\x{eb}\x{1}\x{0}\x{0}\x{0}\x{93}\x{e}\x{b}\x{cf}\x{10}\x{b3}:M\x{a1}\x{a6}\x{9a}\x{bd}\x{19}\x{ce}\x{a7}\x{91}\x{0}\x{0}\x{0}\x{0}\x{2}\x{0}\x{0}\x{0}\x{0}\x{0}\x{10}f\x{0}\x{0}\x{0}\x{1}\x{0}\x{0} \x{0}\x{0}"..., undef) called at C:/Users/cindy/Desktop/convert_to_1p4/./Converters/Chrome.pm line 102
Converters::Chrome::get_db_entries(undef) called at C:/Users/cindy/Desktop/convert_to_1p4/./Converters/Chrome.pm line 153
Converters::Chrome::do_import(undef, undef) called at convert_to_1p4.pl line 140

If it needs a password it didn't request one. I'm on Windows 10.

MrC

@cindyhoskey ,

Oh, darn. Let's see if we can figure this out. Which version / bit-depth of Windows are you running?

cindyhoskey

Windows 10 Home, 64 bit. It says for windows Version 1709.

cindyhoskey

@MrC I can try a csv file instead of Chrome. I have them in a csv, too.

MrC

@cindyhoskey ,

OK, I can reproduce it on Win 10 x64. I was testing on Win 7.

For now, so that you are not wasting time waiting for me, free free to use a csv import if you need to.