MrC's Convert to 1Password Utility (mrc-converter-suite)

MrC

Ok, it sounds like your PATH was not modified by the Perl installer. Use the full path in the command instead of just perl. Your command will start like the following:

 C:\Perl64\bin\perl.exe convert_to_1p4.pl ....

Russty

Bingo! You're a geniuis! Thanks MrC. This is Russty with 2 s's signing off!

Ben

Hi @Russty‌

Thanks for the update here! Glad to hear you've got it working.

@MrC, thanks for your help in this matter, as always. :)

Michael Zhang

Please help.
Thanks in advance.

MrC

Hi @Michael Zhang‌ ,

This should be an easy one. From the README file:

Some additional modules are required for Windows users - the commands below only need to be run once, after you've installed ActivePerl. In the cmd.exe window, type the following command and hit Enter:

ppm install xml-xpath

Follow that instruction and try the conversion again.

Michael Zhang

Thanks for your prompt reply. still can not.

MrC

Can you explain which version of ActivePerl you installed, and how and where? Something does not seem right with the installation.

Try running the command to modify your path:

PATH=%PATH%;C:\Perl64\bin

and then run the ppm command to instal xml-xpath.

Michael Zhang

Finally, it works.
Thanks very much.
Geniuis!

Michael Zhang

But now got another problem, after import on desktop, when i sync with dropbox to iphone6, 1password on iphone6 crash, and can not launch now.

MrC

Finally, it works. Thanks very much. Geniuis!

Great, so that's good progress. Thanks for the nice feedback.

But now got another problem, after import on desktop, when i sync with dropbox to iphone6, 1password on iphone6 crash, and can not launch now.

>

So that I'm clear, your import into 1Password for Windows was successful and seemed correct (the number of records looks correct, a quick glance at the data seems fine, etc.)?

If that's the case, then it think your issue is a separate matter, having to do with syncing 1Password for Windows with Dropbox, and then getting that data into the iPhone. I recall that records can take a little while to propagate to and from Dropbox the first time. You should post your issue in perhaps the IOS or Windows forum, and the AgileBits folks can work with you on that matter.

Steve_J

Hi MrC

I've exported my SafeWallet data to xml and tried to import it via your conversion utility. However, I'm getting 'Imported 0 cards'.

Investigating further, it appears that I'm running version 3.0.5 whilst I believe your conversion utility is for 3.0.7.

I assume that there's been a slight change to the xml format that's causing the issue.

Would it be possible to send me a dummy SafeWallet 3.0.7 xml file so I can check what's different and then correct my version?

Thanks for your help!

Cheers

Steve

MrC

Hi @Steve_J,

Your theory of format changes makes sense and is possible. If you send me an email, I'll send you my test data file created with SafeWallet 3.0.7 on Windows.

If there are format changes, I'll need to get an idea of how that formatting has changed, so might have to ask you for an XML export of some dummy data to review what has changed and make any modifications to the converter module. My email address is at the top of the converter script.

Edit: it turns out the issue was not due to any substantive XML export differences between the 3.0.5 vs. 3.0.7 versions. The issue was that the data format in the SafeWallet 3.x series changed from the 2.x series. Version 1.01 of the conversion suite handles these changes (it is in the _testing directory of my AgileBits Utilities folder - see link in post #1).

Steve_J

Just to confirm that V1.01 handles the XML export from the Windows version of SafeWallet without any issues.

All my cards are now safely migrated to 1Password.

Thank you very much!

test011

Thanks for the wonderful tool.

Does this work with UTF-8 charset? I am converting safewallet 3.0.7 export file and many card names are in UTF-8, asian characters.
Most of card names, but not all, are corrupted in resulting 1pif and card types are messed up as I get only one CreditCard convert though I have more in the export file. Other BankAccount and CreditCard cards are converted into something else, probably to Server Items since I am getting 23 of them. Should I install some perl module to handle UTF-8? (using 1p4 1.01 version)

1password work well with manually typed UTF-8 entries.

Update:
I opened resulting 1pif file and confirmed it seems OK. no corrupted entries. I guess 1password is the one corrupting them while import. Sorry for the false alarm.

Update2:
The "secureContents" part of the resulting 1pif file is very much different from one 1pif I exported from 1password after manually typed in some cards. the converted 1pif has all the information in "notesPlain". I guess my safewallet cards are too heavily customized to be converted. They were imported into Safewallet from SPBwallet.

Final Update:
1password import/export cannot handle UTF-8.
I manually inputted 10 cards with UTF-8 titles, export 10 of them in 1pif, created a new vault, imported that 1pif file, resulting a mess.

MrC

Hi @test011,

1PIF handles UTF-8 just fine. And the required version of Perl handles Unicode fine too.

The SafeWallet converter, however, currently only supports English field names simply because I do not have the translations for other languages. Since field names are matched (using pattern recognition), only those that are coded into the card type and field name table are recognized (another converter uses translation tables to handle localization, but I don't have access to those for SafeWallet).

Unfortunately, my SafeWallet demo has now expired, and there is no way to register or extend the trial, so I can't quickly go test Asian characters. However, my test data does include Unicode values and those were properly handled and encoded into the 1PIF, whereby they were imported into 1Password.

I'm not sure what is going wrong in your situation. If you can create for me a test data file for SafeWallet, using the stock English field names, and including Asian characters in the values (or in custom fields), I can test out what might be going wrong.

I have no idea what the SPBWallet --> SafeWallet conversion did, so can't really comment on that aspect.

A 1PIF export from 1Password will include additional encoded fields and values, which are not present in a 1PIF produced by the converter (there is no need to included some data, as it is generated on import). It appears you're seeing something else though, as you mention all the data is going into the notesPlain section. This would indicate that the field names are not matching. The %card_field_specs table at the beginning of the Converters/Safewallet.pm file can be customized to match the card field patterns. For example, the City field in an Address card is matched with:

[ 'city',               0, qr/^City$/, ],

but could be customized to match the Spanish word for city:

[ 'city',               0, qr/^Ciudad$/, ],

or both:

[ 'city',               0, qr/^City|Ciudad$/, ],

If I had the translation tables, I could automate this (as it is done in the mSecure converter). It's a little more complicated, possibly, than this, since the uniqueness of these string values across all the cards dictates whether or not a card category can be safely assumed.

If you want to discuss how your conversion might be handled, let's talk offline. I'll need a sample export (not your private data), and I'd need you to provide a translation table for your language (I'll explain what's required). My email address is at the top of the script.

seanspicer

thankyou so much for doing this. i have used the script to export my wifes windows eWallet file and am trying to run the conversion script on my mac. i have counted that there are 168 records in total but the output states:
Imported 0 cards
Exported 0 total items

any ideas where i should look ?

MrC

Hi @seanspicer,

You're welcome.

Can you tell me about your eWallet's version and platform? Is it possible you were running eWallet on Windows, and exported it there, and then moved the export file to a Mac to convert?

seanspicer

Hi @MrC‌

Can you tell me about your eWallet's version and platform?

Windows 8
eWallet version 7.4.x

Is it possible you were running eWallet on Windows, and exported it there, and then moved the export file to a Mac to convert

yes this is exactly what i did.

what do you think is the problem ?

MrC

Great, that's what I thought.

You'll have to export and convert on the same platform. Please follow the Windows instructions and see how you fare.

seanspicer

thanks. i will do and report back here

madeio

@MrC

I'm using mSecure 3.5.4 and try to use your converter. I personilized in mSecure the type name. Therefor I changed the converter file in the following way

    login =>            { textname => 'Web-login', type_out => 'login', fields => [
    [ 'url',        0, 'URL', ],
    [ 'username',       0, 'Username', ],
    [ 'password',       0, 'Password', ],

But I'm still getting the following error message. What is the failure?

PS C:\Users\...\Desktop\onepassword-utilities\convert_to_1p4> perl convert_to_1p4.pl msecure -v pm_export.txt
Use of uninitialized value $notes in string ne at Converters/Msecure.pm line
        231, <$io> line 1 (#1)
    (W uninitialized) An undefined value was used as if it were already
    defined.  It was interpreted as a "" or a 0, but maybe it was a mistake.
    To suppress this warning assign a defined value to your variables.

    To help you figure out what was undefined, perl will try to tell you
    the name of the variable (if any) that was undefined.  In some cases
    it cannot do this, so it also tells you what operation you used the
    undefined value in.  Note, however, that perl optimizes your program
    anid the operation displayed in the warning may not necessarily appear
    literally in your program.  For example, "that $foo" is usually
    optimized into "that " . $foo, and the warning will refer to the
    concatenation (.) operator, even though there is no . in
    your program.

Use of uninitialized value $msecure_type in exists at Converters/Msecure.pm
        line 237, <$io> line 1 (#1)
Use of uninitialized value $msecure_type in concatenation (.) or string at
        Converters/Msecure.pm line 242, <$io> line 1 (#1)
Renamed card type '' is not a default type, and is being mapped to Secure Notes

Use of uninitialized value $msecure_type in join or string at
        Converters/Msecure.pm line 244, <$io> line 1 (#1)
Use of uninitialized value $notes in string ne at Converters/Msecure.pm line
        231, <$io> line 2 (#1)
Use of uninitialized value $msecure_type in exists at Converters/Msecure.pm
        line 237, <$io> line 2 (#1)
Use of uninitialized value $msecure_type in concatenation (.) or string at
        Converters/Msecure.pm line 242, <$io> line 2 (#1)
Renamed card type '' is not a default type, and is being mapped to Secure Notes

Use of uninitialized value $msecure_type in join or string at
        Converters/Msecure.pm line 244, <$io> line 2 (#1)
Use of uninitialized value $notes in string ne at Converters/Msecure.pm line
        231, <$io> line 3 (#1)
Use of uninitialized value $msecure_type in exists at Converters/Msecure.pm
        line 237, <$io> line 3 (#1)
Use of uninitialized value $msecure_type in concatenation (.) or string at
        Converters/Msecure.pm line 242, <$io> line 3 (#1)
Renamed card type '' is not a default type, and is being mapped to Secure Notes

Use of uninitialized value $msecure_type in join or string at
        Converters/Msecure.pm line 244, <$io> line 3 (#1)
Use of uninitialized value $notes in string ne at Converters/Msecure.pm line
        231, <$io> line 4 (#1)
Use of uninitialized value $msecure_type in exists at Converters/Msecure.pm
        line 237, <$io> line 4 (#1)
Use of uninitialized value $msecure_type in concatenation (.) or string at
        Converters/Msecure.pm line 242, <$io> line 4 (#1)
Renamed card type '' is not a default type, and is being mapped to Secure Notes

Use of uninitialized value $msecure_type in join or string at
        Converters/Msecure.pm line 244, <$io> line 4 (#1)
Imported 4 cards
Exported 4 note items
Exported 4 total items<

MrC

Hi @madeio‌,

Since you are customizing the converter, let's discuss this offline. Send me an email and I'll follow-up (my email is at the top of the script). I don't see, nor would expect to see, the error you get by the simple customization you've indicated above. The error would indicate that perhaps additional customizations have been done to the Web-login category, or there is some other factor I can't deduce from the the output above.

In case it helps, the error means that the expected Notes part of the Web-login is missing, and that should never be the case (it could be empty, but not missing from the cell data).

Also, you don't need to specify the type_out => 'login' since the specific card_field_spec category is already 'login'.

Best,
MrC

crixlet

@MrC‌ worked like a charm. Thanks for your hard work on this!

MrC

@crixlet,

You're welcome! Thanks for the nice feedback. Out of curiosity, from whence did your data come?

madeio

@‌MrC

I found the problem. mSecure is exporting my csv file with semicolon instead of comma. My PC has German as primary language and that's why it is using a semicolon for csv files. I just had to change the region and language settings to English.

MrC

Thanks for the update @madeio,

FYI: the separator character can be changed in the script:

        sep_char => ',',

would be changed to:

        sep_char => ';',

I have no idea how well mSecure exports using this character, so your choice to change the region settings was probably your best and safest bet. It sounds like you are good to go.

seanspicer

Hi @MrC‌

Just to let you know that the import from eWallet@Windows to 1Password4@Windows worked fine. A couple of useful tips to you and others:

1) Location field in 1Password4 does not get updated

This has been mentioned in other posts. Whilst the URL field is successfully updated, the Location field is empty. The location field appears to be used as the domain to match a Login entry to a website, as well as form the search key for Watchtower Security Audits. A successful workaround was to simply open and close each Login record to refresh the Location field. There did not seem to be any need to actively Save the record and I only had to use the Return Key (open/close) and arrow key (up/down) to achieve this.

2) Security Audit (Watchtower/Heartbleed) displays 'no vulnerabilities'

This has been mentioned in other posts. The problem is that 1Password thinks imported logins were created on the date of import and therefore after affected websites have been patched. A successful workaround is to set the system date to sometime in the past (e.g. 1st Jan 2001) before importing into 1Password, and once the import has completed set it back to the correct date and time. Hey Presto - all passwords have an old created date and will trigger a security audit flag notifying you to change your password.

Hope this helps. Thank you for a great tool

seanspicer

MrC

Great, and thanks so much for the feedback and helpful tips!. It's nice to hear you were able to import successfully. In sure the AgileBits folks well address the 1Password for Windows issues you mention as soon as they are able.

mishaw

First of all hello to all and thank you for the great tool!
Now, I'm partially through the convertion process from LastPass, but the problem is I lost "URL" column when converting the LastPass csv file.
How do I fix that?

I run Windows 7 64bit with 1Password 4.1.0.530 in demo mode and LastPass for Chrome 3.1.69.
The data format in export file is following:

url,username,password,extra,name,grouping,fav

When exporting, the current LastPass generates the html page with all the data that I copy-paste to the notepad. Can it damage the data structure and cause the problem?

Thanks in advance!

MrC

Hi @mishaw,

It sounds like the issue you have encountered is issue #1 mentioned by @seanspicer above. Does the description makes sense to you and seem to be the same as described?

Please also be sure to try version 1.01 in the _testing folder of my AgileBits Utilities share (link at the top of this thread).