MrC's Convert to 1Password Utility (mrc-converter-suite)

dszp

@MrC Thanks for the pointer. I'm definitely not trying to say it wasn't there in the instructions, just that I didn't see it--I have a history with Perl scripting so I'm sure I skimmed the directions more lightly than I should have :-) But hey, I can apparently clean up my own messes as well at least...thanks again! Instead of diving in on the converter for my own data, I used the Mac 1Password to import from LastPass for myself back in November, which worked mostly fine except I had to do two separate types (two imports) and, now that I think about it, also ran into issues with Mac Chrome's export and had to use another browser to get an importable file. Is your memory supposed to start slipping in your early 30s? :-)

MrC

@dszp - gottcha.

I really wish I had a more dynamic wizard-like documentation language / flow, where users could actively engage with the documentation, as they follow the steps, and the documentation would tailor itself to their situation.

Does your memory start to fade in your 30s? I forget.

dszp

@MrC Yeah, it seems like the standardized argument list would work well with a basic GUI wrapper/wizard, but then there's the whole GUI wrapper to write and maintain...Robocopy has the RobocopyGUI is kind of what I'm thinking, but that's kind of hypocritical of me since I prefer the Robocopy CLI to the GUI wrapper I guess :-)

stace3

@MrC I appreciate your speedy response. I tried your method but received this error:

C:\Users\Stacey\Desktop\convert_to_1p4>perl convert_to_1p4.pl spbwallet -v ..\My_Wallet.swl
Enter your SPB Wallet password:
DBD::SQLite::db prepare failed: no such table: spbwlt_Template at Converters/Spbwallet.pm line 232, line 1.
Can't call method "execute" on an undefined value at Converters/Spbwallet.pm
line 233, line 1 (#1)
(F) You used the syntax of a method call, but the slot filled by the
object reference or package name contains an undefined value. Something
like this will reproduce the error:

    $BADREF = undef;
    process $BADREF 1,2,3;
    $BADREF->process(1,2,3);

Uncaught exception from user code:
Can't call method "execute" on an undefined value at Converters/Spbwallet.pm line 233, line 1.
Converters::Spbwallet::do_import('..\My_Wallet.swl', undef) called at convert_to_1p4.pl line 116

C:\Users\Stacey\Desktop\convert_to_1p4>

Any possible next steps? Would love to get this data into 1Password without manually entering everything.

MrC

Hi @stace3,

Sorry for your troubles. Let's see if we can figure out what's going wrong.

Run the same command, but add the additional command line voodoo:

 -d | grep print_fileinfo

so your full command will be:

perl convert_to_1p4.pl spbwallet -v ..\My_Wallet.swl  -d | grep print_fileinfo

The command will hang waiting for your password to be entered - enter it and the command will complete (but fail of course). Copy and paste the print_fileinfo lines here in a reply.

stace3

Thank you. I think I did this correctly but receive the following error message:

C:\Users\Stacey\Desktop\convert_to_1p4>perl convert_to_1p4.pl spbwallet -v ..\My_Wallet.swl -d | grep print_fileinfo
'grep' is not recognized as an internal or external command,
operable program or batch file.

I copied and pasted from your message above.

MrC

Ah, darn, I wasn't paying attention. This is Windows. Just rerun your command adding the -d option, and copy and paste only the three print_fileinfo lines. Those lines won't contain any confidential information. It will look like this:

print_fileinfo : Export file info
print_fileinfo : size: 223232
print_fileinfo : kind: SQLite 3.x database
print_fileinfo : mime: application/octet-stream; charset=binary

stace3

I'm sorry, you've lost me. And I realize I have made a mistake in my initial run. The file name is not My_Wallet.swl it is My Wallet.swl This is the error I receive:

C:\Users\Stacey\Desktop\convert_to_1p4>perl convert_to_1p4.pl spbwallet -v ..\My Wallet.swl -d
main : Command Line: spbwallet -v ..\My Wallet.swl -d
Missing export_text_file name - please specify the file to convert
Usage: convert_to_1p4.pl

converters:
clipperz csv dataguardian datavault essentialpim ewallet handysafe ironkeyim keepass2
keepassx keychain lastpass licensekeeper msecure nortonis onepif2html passpack
passwordagent passworddepot passwordwallet safeincloud safewallet spbwallet splashid
vcard

options:
-d or --debug # enable debug output
-D or --dump # dump category / field table
-e or --exptypes # comma separated list of one or more export types from list below
-f or --folders # create and assign items to folders
-h or --help # output help and usage text
-i or --imptypes # comma separated list of one or more import types from list below
-o or --outfile # use file named ofile.1pif as the output file
-v or --verbose # output operations more verbosely
--nowatchtower # do not set creation date for logins to trigger Watchtower checks

supported import types:
bankacct contact creditcard driverslicense emailacct frequentflyer hosting idcard
librarycard membership note onlineshoppingacct passport personalinsurance server
socialsecurity software website
supported export types:
bankacct creditcard driverslicense email identity login membership note passport rewards
server socialsecurity software

It seems it will not recognize the file with the space in the name? Is that correct? I apologize, this is quite beyond anything I have done before.

MrC

@stace3,

Ok, add double quotes around your file name. That might just be the issue. Try this first:

 perl convert_to_1p4.pl spbwallet -v "..\My Wallet.swl"

If that fails, then add the -d and report the print_fileinfo lines:

 perl convert_to_1p4.pl spbwallet -v "..\My Wallet.swl"  -d

The double quotes are required because your filename has a space in it, and we have to tell the windows command shell that it is part of the parameter being passed to the perl command (and not the two separate words "..\My" and "Wallet.swl").

You're doing great, by the way!

stace3

It worked! Thank you so much! You have saved me so much work.

MrC

@stace3 - You're awesome! Nice work. Enjoy 1Password.

jrsforums

@MRC. I am using Win 10 and Roboform 7.9.17.5. Installed latest 'stable bits' and output html file for logins. On running, got following error message.

Unexpected duplicate key 'passed-username' in card_field_space{login}

I can send html file, but it is ~2.4 mb

MrC

Hi @jrsforums ,

The converter checks to be sure there are not multiple identical field names in a record. I commented about this in another thread. Are you able to change one of the field names in the record in question and rerun the converter?

For this converter, I may have to handle duplicate field names specially, but need to think about it. I could use help in knowing which website(s) trigger this.

Please never send your exported data! I won't read it, and more importantly, you don't want it traversing the Internet unencrypted.

MrC

Hi @jrsforums,

I have more info to add. I'd forgotten about how the RoboForm converter matches username and password field names. It uses patterns, and so what I think is happening is that your 'passed-username' pattern is matching one of the username patterns. If you can tell me the two field names in that record that appear to look like or "username" fields, I can tighten up the pattern that matches these. This is something I was going to do before the 1.08 release but forgot about that. There isn't a set list of patterns - rather, it will vary based on each user's web site logins, since these field labels vary across the web in no formal manner.

onemico

Hi,

I have attempted this several times over the last day or so, to convert on OS X El Capitan, using Keychain, and I keep getting the following:

XXXXXXXX-MacBook-Pro:convert_to_1p4 XXXXX$ perl5.16 convert_to_1p4.pl keychain -v ../../../Desktop/pm_export-icloud.txt
Odd number of elements in hash assignment at Converters/Keychain.pm line 146 (#1)
(W misc) You specified an odd number of elements to initialize a hash,
which is odd, because hashes come in key/value pairs.

Use of uninitialized value in list assignment at Converters/Keychain.pm line
146 (#2)
(W uninitialized) An undefined value was used as if it were already
defined. It was interpreted as a "" or a 0, but maybe it was a mistake.
To suppress this warning assign a defined value to your variables.

To help you figure out what was undefined, perl will try to tell you
the name of the variable (if any) that was undefined.  In some cases
it cannot do this, so it also tells you what operation you used the
undefined value in.  Note, however, that perl optimizes your program
anid the operation displayed in the warning may not necessarily appear
literally in your program.  For example, "that $foo" is usually
optimized into "that " . $foo, and the warning will refer to the
concatenation (.) operator, even though there is no . in
your program.

Examined 378 items
Skipped 378 non-login items
Skipped 0 duplicate items
Imported 0 items
Exported 0 total items

Can anyone confirm what I am doing wrong, followed both the PDF guide, and the explicit OS X guide on the "Extra Help" page on the first post?

Any help greatly appreciated, pulling my hair out here :)

MrC

@onemico,

What version of the converters are you running? Your command seems to follow some instructions from older versions.

If you are not using the 1.08 version of the converters, please update. If you are, try the following command and report the output:

 perl convert_to_1p4.pl keychain -v ../../../Desktop/pm_export-icloud.txt  -d  | grep print_fileinfo

I'll take a look at the code in the morning to see under what circumstances you might get this error.

onemico

@MrC

Hi, I am using 1.08, attached is the output you requested, hopefully this will help, thanks very much.

Also to add, it does create the 1P_import.1pif file, but it is a blank 0 bytes file

xxxxxx-MacBook-Pro:convert_to_1p4 xxxxx$ perl convert_to_1p4.pl keychain -v ../../../Desktop/pm_export-icloud.txt -d | grep print_fileinfo
print_fileinfo : Export file info
print_fileinfo : size: 316432
print_fileinfo : kind: ASCII text, with very long lines
Odd number of elements in hash assignment at Converters/Keychain.pm line 146 (#1)
(W misc) You specified an odd number of elements to initialize a hash,
which is odd, because hashes come in key/value pairs.

Use of uninitialized value in list assignment at Converters/Keychain.pm line
146 (#2)
(W uninitialized) An undefined value was used as if it were already
defined. It was interpreted as a "" or a 0, but maybe it was a mistake.
To suppress this warning assign a defined value to your variables.

To help you figure out what was undefined, perl will try to tell you
the name of the variable (if any) that was undefined.  In some cases
it cannot do this, so it also tells you what operation you used the
undefined value in.  Note, however, that perl optimizes your program
anid the operation displayed in the warning may not necessarily appear
literally in your program.  For example, "that $foo" is usually
optimized into "that " . $foo, and the warning will refer to the
concatenation (.) operator, even though there is no . in
your program.

print_fileinfo : mime: text/plain; charset=us-ascii

MrC

Hi @onemico,

Thanks for the info. Now we can move onto the next steps. I'll need you to download the 1.09 version in Testing Bits. I've added some debug code to help isolate the issue.

Run your command with the -d debug option included;

perl convert_to_1p4.pl keychain -v ~/Desktop/pm_export-icloud.txt -d

There will be a lot of output, but the converter should abort when it encounters your error. It will output the name of the attribute which is causing the problem, as well as the entire list of attributes for the record. It will look like this:

Unexpected undefined value in class=inet attribute '0x00000008 <blob>'
    0x00000007 <blob>="YouTube"
    0x00000008 <blob>=
    "acct"<blob>="gomer@example.com"
    "atyp"<blob>=<NULL>
    "cdat"<timedate>=0x32303135303132373035343731335A00  "20150127054713Z\000"
    "crtr"<uint32>=<NULL>
    "cusi"<sint32>=<NULL>
    "desc"<blob>=<NULL>
    "icmt"<blob>=<NULL>
    "invi"<sint32>=<NULL>
    "mdat"<timedate>=0x32303135303132373035343731335A00  "20150127054713Z\000"
    "nega"<sint32>=<NULL>
    "path"<blob>=<NULL>
    "port"<uint32>=0x00000000
    "prot"<blob>=<NULL>
    "ptcl"<uint32>=0x00000000
    "scrp"<sint32>=<NULL>
    "sdmn"<blob>=<NULL>
    "srvr"<blob>="YouTube"
    "type"<uint32>=<NULL>

What I need to understand is which attribute name is causing the issue. In this case, it is the attribute named 0x00000008 <blob>, and as you can see above, there is nothing after the = character for it in the attribute list. Don't report any values in your reply, but reply with the error line and the srvr or svce attribute, as in:

Unexpected undefined value in class=inet attribute '0x00000008 <blob>'
    "srvr"<blob>="YouTube"

Let's see if this gets us closer to the error.

onemico

Thanks @MrC but I am not getting that output with 1.09 version, below is my output"

xxxxxx-MacBook-Pro:convert_to_1p4 xxxxx$ perl convert_to_1p4.pl keychain -v ~/Desktop/pm_export-icloud.txt -d
main : Command Line: keychain -v /Users/xxxxx/Desktop/pm_export-icloud.txt -d
main : Output file: /Users/xxxxx/Desktop/1P_import.1pif
print_fileinfo : Export file info
print_fileinfo : size: 316432
print_fileinfo : kind: ASCII text, with very long lines
print_fileinfo : mime: text/plain; charset=us-ascii
do_import : Entry 1
do_import : raw attibute list:
/Users/xxxxx/Library/Keychains/local-icloud.keychain
Unexpected undefined value in class=/Users/xxxxx/Library/Keychains/local-icloud.keychain attribute '/Users/xxxxx/Library/Keychains/local-icloud.keychain'
/Users/xxxxx/Library/Keychains/local-icloud.keychain

MrC

@onemico,

Ok, that is progress then! So it appears the block that represents an entry is not in a format I'm expecting, and it is the first entry you're having problems with. A block should look like this, in this format:

keychain: "/Users/fredsmith/Library/Keychains/login.keychain"
class: "inet"
attributes:
    0x00000007 <blob>="YouTube"
    0x00000008 <blob>=<NULL>
    "acct"<blob>="gomer@example.com"
    "atyp"<blob>=<NULL>
    "cdat"<timedate>=0x32303135303132373035343731335A00  "20150127054713Z\000"
    "crtr"<uint32>=<NULL>
    "cusi"<sint32>=<NULL>
    "desc"<blob>=<NULL>
    "icmt"<blob>=<NULL>
    "invi"<sint32>=<NULL>
    "mdat"<timedate>=0x32303135303132373035343731335A00  "20150127054713Z\000"
    "nega"<sint32>=<NULL>
    "path"<blob>=<NULL>
    "port"<uint32>=0x00000000
    "prot"<blob>=<NULL>
    "ptcl"<uint32>=0x00000000
    "scrp"<sint32>=<NULL>
    "sdmn"<blob>=<NULL>
    "srvr"<blob>="YouTube"
    "type"<uint32>=<NULL>
data:
"My Secret Password"

It should start with the keychain name, followed by the class type, followed by the (indented) attributes name=value pairs, and then followed by a data block, where the value is on the next line.

Can you comment on what you're seeing?

onemico

Thanks, this is the first block is pasted below, with the obvious changes made to username and password
keychain: "/Users/xxxxx/Library/Keychains/local-icloud.keychain" version: 256 class: "genp" attributes: 0x00000007 <blob>="AirPort" 0x00000008 <blob>=<NULL> "acct"<blob>="TP-LINK_55FF68" "cdat"<timedate>=0x32303136303132323038323434375A00 "20160122082447Z\000" "crtr"<uint32>=<NULL> "cusi"<sint32>=<NULL> "desc"<blob>=<NULL> "gena"<blob>=<NULL> "icmt"<blob>=<NULL> "invi"<sint32>=<NULL> "mdat"<timedate>=0x32303136303132323038323434375A00 "20160122082447Z\000" "nega"<sint32>=<NULL> "prot"<blob>=<NULL> "scrp"<sint32>=<NULL> "svce"<blob>="AirPort" "type"<uint32>=<NULL> data: "Password"

MrC

@onemico,

You're an awesome debug assistant! I've spotted the issue and have updated the 1.09 converter in Testing Bits. Give the new one a try!

onemico

@MrC

You are a legend, working now perfectly. Thank you very much. And thanks, not a coder, but do work in IT, reading debugs all day, so always try to format OK and reply ASAP :)

Thanks again for the great tool and for helping troubleshoot individual issues like this

MrC

@onemico,

Outstanding! Enjoy 1Password.

jrsforums

@MrC Not exactly sure what to look at in the data. Am i looking for a duplicate or problem tag or data in a tag? Any indication of what tag? Would it be possible for you error routine to indicate which record the problem is in or to bypass the record. Obviously, I have a lot of records in the 2.4 mb of data.

MrC

Hi @jrsforums,

It is a little hard to describe. Roboform, unlike other password managers doesn't have fixed field names for a record, such as Username, Password, URL, Notes, etc., or fixed positions in an export. Instead, it uses the actual names within the HTML web page as the field names. So the converter uses patterns to try to find the name of these fields so that it can assign them to 1Password's fields.

You've discovered one field name - it is passed-username. But this is probably matching some other field name in that record, perhaps it is simply username or user, and then the converter complains about duplicate fields.

The problem is that if i make the field detection too restrictive, many of your username and password fields won't get mapped correctly. If I make the patterns too relaxed, it will incorrectly detect those extra fields that RF grabbed from the web page.

I'm going to change the code today to allow you to more easily enter patterns that can be used to detect these fields, as well as prevent certain patterns from being used. That way, you can just add your field such as passed-username into a table, and it will no longer be considered as a candidate for Username, and therefore won't trigger the duplicate field detector.

I'll have something for you this evening (PST).

jrsforums

@MrC Searched html file with notepad and crimson editor. Could not find string passed-username

MrC

Hi @jrsforums,

Since you reported that the duplicate field was named passed-username, that was the term I used. I don't know what you actually have, so you'll have to double check your results.

I just updated the 1.09 version in Testing Bits. Can you try that out?

jrsforums

I copied, exactly, the error message. Yes, I will try 1.09

jrsforums

@MrC. To go to 1.09 does one need to do a full install or just bring in new modules? It may be in doc, but I do not remember seeing.