MrC's Convert to 1Password Utility (mrc-converter-suite)

MrC

@Donaldd,

I doubt it is the images, since I have attached pics and the conversion works. The images are ignored. I suppose I should export these some day.

It could be the Chinese characters. Can you reply here with some sample Chinese text I might try?

Donaldd

@MrC
Please try these words: 密码管理器 （Password manager in Chinese) :)
I also tried to import the csv file to 1password (use the built-in import), which I exported form Safeincloud, the entries could be shown in 1password except the secure notes.

MrC

@Donaldd,

No problem importing a conversion with those characters.

Can you describe exactly how you're importing, and how you are responding to the dialogs that are presented?

What might help, is if you do the conversion, make a copy of the .1pif file, edit the copy, remove 1/2 the entries, and try to import that 50% file. If that works, try the other half. Divide and conquer this way until you find the offending entry.

Not that a .1pif file has a record divider string:

***5642bee8-a5ff-11dc-8314-0800200c9a66***

Be sure to remove the string for each record you remove.

Donaldd

@MrC
Thank you for the kind help :)
Finally I could import the 1pif file into 1password with your help:
If I try to import the converted file directed in 1password, it will be "0 items", but I could import all the entries when removing the first entry in notepad.exe. However the first entry is quite common, what's the point?
Then I tried to convert the file again, opened the 1PIF with notepad and simply click "save"... Then the file could be imported....
:(

So the problem is I need to open that 1PIF file in notepad and save before importing...

Anyway, the problem is solved! Thank you so much, MrC.

MrC

@Donaldd,

First, great to hear you have the items imported!

If you still have the patience, I'd like to understand what caused the problem. Since you can import the file after saving it with Notepad, this implies that either the encoding format, line ending format, or the addition of what is called the BOM, changed and allowed the import. I think the removal of the first entry was not a factor, rather, as you've implied, the simple save via Notepad allowed import.

I presume you did the export, conversion, and import all on Windows.

The converter on Windows generates the 1PIF as UTF-8 with CR-LF line endings.

If you opened the file in Notepad and saved it, it would have read the file as ANSI (by default) and saved it in the same way the 1PIF was created, that is UTF-8 with CR-LF line endings. But it is possible now that your Chinese characters are botched.

If you opened and read the file as UTF-8 (not the Notepad default, you would have changed this in the Open dialog) then the file would be UTF-8 with CR-LF line endings, but it also now includes a BOM. 1Password for Windows doesn't care about a BOM, but it is possible that some characters in your 1PIF tripped up 1Password's importer.

Any additional info, based on what I mention above?

Donaldd

@MrC
Hi, I tried to open and save this file in Notepad++ which can choose the format to save ("UTF-8 without BOM" or UTF-8).
The result is:
UTF-8 without BOM = 0 item, UTF-8 = all the items could be imported properly
Hope these info could help :)

AGAlumB

@Donaldd: Thanks for bringing this to our attention! We'll have to investigate that text encoding issue further. :)

CamJN

I'm seeing perl puke when trying to convert my exported keychain.

cd '/Users/camdennarzt/Desktop/convert_to_1p4/' && /usr/bin/perl5.16 convert_to_1p4.pl keychain '/Users/camdennarzt/Desktop/pm_export-icloud.txt' -v
~/Desktop/convert_to_1p4 ~/Desktop/convert_to_1p4
Odd number of elements in hash assignment at Converters/Keychain.pm line 146 (#1)
    (W misc) You specified an odd number of elements to initialize a hash,
    which is odd, because hashes come in key/value pairs.

Use of uninitialized value in list assignment at Converters/Keychain.pm line
    146 (#2)
    (W uninitialized) An undefined value was used as if it were already
    defined.  It was interpreted as a "" or a 0, but maybe it was a mistake.
    To suppress this warning assign a defined value to your variables.

    To help you figure out what was undefined, perl will try to tell you
    the name of the variable (if any) that was undefined.  In some cases
    it cannot do this, so it also tells you what operation you used the
    undefined value in.  Note, however, that perl optimizes your program
    anid the operation displayed in the warning may not necessarily appear
    literally in your program.  For example, "that $foo" is usually
    optimized into "that " . $foo, and the warning will refer to the
    concatenation (.) operator, even though there is no . in
    your program.

Examined 214 items
Skipped 214 non-login items
Skipped 0 duplicate items
Imported 0 items
Exported 0 total items

I'm on os x 10.11.5, any advice?

MrC

@CamJN,

Use the 1.09 version in Testing Bits.

marty_b

Can this convert a generic XML file to CSV or to 1p4? I am struggling to find something to do this for me that isn't an "online converter" where I have to upload the file to some random unknown website.

MrC

Hi @marty_b ,

An XML file is just structured data. It is meaningless without content definitions. So it cannot be converted into anything meaningful without knowing how to interpret the structure.

Can you explain where the file came frome, or give some more details?

marty_b

Well I got some crappy plugin that is supposed to export CSV or XML passwords from a browser but CSV doesn't work only XML. It's crystal clear how to interpret it if you open it up it's just the wrong format to do anything useful with. There are too many entries for me to do it by hand but I've wasted so much time looking for a simple XML to CSV conversion that maybe I already could have done a large chunk of it.

They look like

Instead of having that I need a CSV with columns that say host, user, password, etc. with the content in each column.

There are some more types but I dont' really need them like formSubmitURL="http://somesite.com" and httpRealm="" and userFieldName="username" and passwordFieldName="password".

All I really care about getting are my hostname, username, and password into a usable format.

MrC

Hi @marty_b ,

Dont waste any more time with this. Ill help you in the morning, California time.

marty_b

OK thanks.

Megan

Hi @marty_b,

I’m so glad to see that MrC is helping you out here. He’s our friendly neighbourhood conversion guru, and he’s worked a lot of magic with all sorts of data formats. You’re in good hands. :)

Gerhard0007

Problem - starting the conversion of a sbpwallet file !

Dear MRC !

Thank you for your work on the 1password Import converter tool !

I tried to get your converter Tool working on a W7 and a W10 Machine. I also tried to use it with strawberry perl as portable app and as installed Version - but on both machines i always get the follwoing error after entering the spbwallet Password:

DBD::SQLite::db prepare failed: no such table: spbwlt_Template at Converters/Spbwallet.pm line 230, line 1.

Meanwhile:
The last attempts to start the conversion always ending up in:
incorrect password

I use the same password to unlock the swl wallet by program.

Installed Versions:
1Password 4.6.0.604
Spbwallet 2.1.2.

Do i have to start / close the applications while the conversion process ?

Hope you can give me some hint to overcome the problem.

Many thanks in advance,
Gerhard.

MrC

Hi @Gerhard0007 ,

Thanks for your kind words.

Can you show me the command line you used?

I made some clarifications to the export text for SPB Wallet, and I'm wondering if a) you're using the 1.08 version of the converter which doesn't have those clarifications and so b) your command might not be correct. The 1.09 version in Testing Bits has the updated text:

● SPB Wallet
There is no need to export any data from SPB Wallet. The converter will read and decrypt the data directly from the SPB Wallet .swl file. It might be easiest to move your .swl wallet file to your Desktop. When you enter your command line, your name_of_export_file will be ..\name_of_your_spbwallet_file.swl.
If would prefer not to move the .swl file, you’ll have to specify a path to your wallet file as your name_of_export_file.

Gerhard0007

Hi MRC !

Thanks for the fast response ;-)

I always used a copy of my original gerhard.swl file (without exporting the data) to check the converter tool version 1.08.

The commandline i used is (before i changed the directory using "cd" to get into the converter directory - where i also placed a copy of my the gerhard.swl file):
convert_to_1p4 spbwallet gerhard.swl

The last attempts always ended in: incorrect password

I also tried to change the password of the gerhard.swl file by SPBWallet for testing purposes from real password to:
"123456" and then to "gerhard" - but also with no success.

Perhaps i have to "reset" or "rebuild" the mysql structure created in the previous attempts - but i am not familiar with perl scripting or mysql ...

Many thanks again,
Gerhard

MrC

Hi @Gerhard0007 ,

Please download and use the 1.09 version in Testing Bits mentioned in post 1 of this thread. The error you are seeing:

DBD::SQLite::db prepare failed: no such table: spbwlt_Template at Converters/Spbwallet.pm line 230, <STDIN> line 1.

in the 1.08 version is likely misleading, and probably means your password was incorrect. The 1.09 version handles incorrect password situations better and tells you so.

There's no need to copy your .swl file, or rebuild it, or change the password - it should work fine as is, and the file is only read, not written to (so its safe).

Please try it again, and if it doesn't work, show your actual full command and output (copy and paste, or screenshot), since that's the only thing I have to use for diagnostics.

Also, please add -v to your command line so you get a little more info. And finally, you can add -d to enable even more diagnostic output (caution: it may contain sensitive information). Your command and output should look like:

perl convert_to_1p4.pl spbwallet ..\..\_Wallets\spb_wallet.swl -v
Enter your SPB Wallet password:
Imported 33 items (10 items expanded to 20 items)
Exported 1 passport item
Exported 1 email item
Exported 3 creditcard items
Exported 1 rewards item
Exported 12 note items
Exported 13 login items
Exported 1 driverslicense item
Exported 1 identity item
Exported 1 bankacct item
Exported 2 server items
Exported 5 membership items
Exported 1 socialsecurity item
Exported 1 software item
Exported 43 total items
You may now import the file C:\Users\MrC\Desktop\1P_import.1pif into 1Password

jornhenkes

Hi MrC,
Here's my report on an issue I'm encountering and noticed you about on Github.

I'm trying to convert my Keepass2 passwords to 1password, with the converter. I've followed all the steps and tried the different options, but I'm getting this same error every time:

Failed to inflate compressed data: 
Use of uninitialized value $outfile in open at Utils/PIF.pm line 697 (#1)
    (W uninitialized) An undefined value was used as if it were already
    defined.  It was interpreted as a "" or a 0, but maybe it was a mistake.
    To suppress this warning assign a defined value to your variables.

    To help you figure out what was undefined, perl will try to tell you
    the name of the variable (if any) that was undefined.  In some cases
    it cannot do this, so it also tells you what operation you used the
    undefined value in.  Note, however, that perl optimizes your program
    anid the operation displayed in the warning may not necessarily appear
    literally in your program.  For example, "that $foo" is usually
    optimized into "that " . $foo, and the warning will refer to the
    concatenation (.) operator, even though there is no . in
    your program.

Use of uninitialized value $outfile in concatenation (.) or string at
    Utils/PIF.pm line 697 (#1)
Cannot create 1pif output file: 
No such file or directory

I'm using the Mac converting tool, but I exported my Keepass2 file on a Windows pc (KeepassX doesn't have an xml export option)

Any idea what might go wrong?

Thanks! Jorn

MrC

Hi @jornhenkes ,

Thanks for reporting the issue here and understanding why I ask folks to discuss matters here rather than on Github. A centralized place for Q&As really helps, as I can link answers and keep users on-site rather than being redirected to the foreign and typically unfamiliar GitHub site.

There's no problem converting the Keepass2 / Windows XML file on OS X. It is platform agnostic.

First, you should update to the 1.09 version in Testing Bits. For keepass2, it adds support for historic passwords, and has other general fixes.

Second, can you show me your command line (exactly). The error you're getting is indicating that the 1PIF file cannot be made (at the default Desktop location), and this is unusual.

If you're not allowed to write to your Desktop, you can override this location and provide your own using the -o option. Here's an example creating the file converted.1pif in the current directory:

$perl convert_to_1p4.pl keepass2 -v ../../_TestFiles/keepass2.xml -o ./converted.1pif
Attachment created: 1P4_Attachments/Sample Entry #2/A Text File.txt
Attachment created: 1P4_Attachments/Sample Entry #2/A UTF8 file.txt
Attachment created: 1P4_Attachments/Sample Entry #2/cabinet cooling.pdf
Imported 7 items
Exported 5 login items
Exported 2 note items
Exported 7 total items
You may now import the file ./converted.1pif into 1Password

$ ls -l converted.1pif
-rw-r--r--  1 cappella  staff  3617 Jun 26 12:07 converted.1pif

And now any attachments will be in the 1P4_Attachments directory just created in the local directory also.

jornhenkes

Hi @Mrc,

Thanks for all the info. I've download the 1.09 version and made an export of my Keepass again, this time with .xml as extension. I used .txt before. Unfortunately this all didn't help, I'm keep getting the same error. Here is a screen dump of my console:

In this case I used --folders, but I also tried it without the --folders tag, same issue again.

I hope you can find something!

Gerhard0007

Hi MRC !

I used 1.09 now with no success:

C:\Users\XXXX\Desktop\convert_to_1p4\onepassword-utilities\convert_to_1p4>con
vert_to_1p4.pl spbwallet Container.swl -v -d
main : Command Line: spbwallet Container.swl -v -d
main : Output file: C:\Users\XXXX\Desktop\1P_import.1pif
print_fileinfo : Export file info
print_fileinfo : size: 206848
Enter your SPB Wallet password:
do_import : *** Connecting to database file
do_import : --- Decoding templates
do_import : --- Done decoding templates
do_import : --- Decoding card field templates
do_import : --- Done decoding card field templates
*** Disconnecting from database file

Incorrect password

C:\Users\XXXX\Desktop\convert_to_1p4\onepassword-utilities\convert_to_1p4>

I recreated a new swl File with new Password but also with no success. Perhaps my full version of the PERL Installation is faulty ?

Thanks in advance,
Gerhard

MrC

@Gerhard0007,

Sorry for your troubles. Something doesnt look right to me with the debug output. I'll take a look in the morning to try to make sense of the issue.

Gerhard0007

Hi MRC !

For further testing i uninstalled the 64bit version of Strawberry perl and installed the 32bit Version. No success so far. The result always is "Incorrect Password".

I can upload my Test .swl file if useful for your analysis.

Many thanks again,
Gerhard

MrC

Hi @Gerhard0007

Are your entries within Spb Wallet in a non-English language? If so, which? German?.

The converter keys detection of record types based on the stock English names. Since I can't know that a password is correct, I use the decoding of one field as the basis for detecting correct decoding.

If your stock card fields are non-English, I'll need some translations to properly detect and convert the categories.

MrC

@jornhenkes ,

The screenshot was perfect. I see what is happening now. One of the attachments is failing to be "inflated" (uncompressed), and the inflation code has a bug is and is incorrectly returning from the routine that creates the entries used to make the 1PIF. Because it is returning, the routine isn't passing back the proper data to the calling routine above, and that's why the error occurs about the empty output file name. The fix for this was easy.

Furthermore, the converter was not properly checking for some cases where the attachment was decompressed correctly, and as such, it thought there was an error, causing the problem above.

I've updated the 1.09 package - please download it and let's see If I'm correct and you can now convert.

jornhenkes

Hi @MrC ,
Thanks for all the work. I've downloaded the updated version, but I got a new error this time. My perl is fluent, so I'm not sure what going wrong this time. Here's a screenshot of the output I'm getting:

Many thanks for the help again!

MrC

@jornhenkes ,

I'm sorry, somehow I'd cut a character out of the code. Duh. The update was just posted with the fix.

AGAlumB

Truly, it happens to the best of us. Were it me, I'd still be scratching my head. Nice catch! :lol: