MrC's Convert to 1Password Utility (mrc-converter-suite)

Gerhard0007

Hi MRC !

You´re right so far - the language of SPBWallet is german - sorry, did not recognize that this could make a difference.

I do not have any english SPBWallet version for download.

I made 2 screenshots for better identification ..

best regards,
Gerhard

jornhenkes

@MrC Superb, it all worked great this time! Thanks for all your effort!

MrC

@jornhenkes - Thank you for your perseverance, patience and excellent help! Enjoy 1Password!

@Gerhard0007 - Darn, this is unfortunate. I'll going to rummage through the binary to see if I can dig-up the language translations. If not, I'm going to need your help with this. I'd need exact English to German translations for the names you see near the top of the file Converters\Spbwallet.pm. The table looks like this:

I'd need the translations for the things pointed to by the arrows. Those category names, and field names, are used to detect the record type. Without these mappings, your entries all end up as Secure Notes.

The spbwallet converter provides a dump facility. You'll be able to run the conversion, adding the --dump option, and get the table of categories and field names. But first we have to do a quick fix to allow your password to be accepted. Due to our timezone differences, I'll tell you how to modify the Converters\Spbwallet.pm file to allow the conversion to proceed further.

Edit the file with a plain text editor like Notepad. In the file, there is a test to see if the decryption using your password was successful:

# There should be a card field named 'Password' in the TemplateField table - if this isn't found,
# then assume the user's password was incorrect and the decryption produced jibberish.
#
unless (grep { $templatefield{$_}{'Name'} eq 'Password'} keys %templatefield) {

Change the quoted word 'Password' to your language's version (I think its 'Passwort'). The characters must be exact. Save the file, and re-run the conversion. If that works, add the --dump option and you'll see the list of items for which you have templates and I need translations for.

The dump will look like this:

$ perl convert_to_1p4.pl spbwallet ../../_Wallets/spb_wallet.swl --dump
Enter your SPB Wallet password:
Cards: Frequent Flyer Account
    Phone
    Account #
    Web Site
    Airline
    Issued
    Card Type
    User Name
    Password
    Status
    Expires
    CVV
    Award
    Owner
    PIN
Cards: ID Card
    Expires
    PIN/Code
    Full Name
    Country
    Phone
    Card Title
    Issued
    ID #

Because this is a tedious job, when I write new converters and add new categories and their fields to the table, I do them based on which categories are most frequently used by users. That way, at least users can get started, and I finish up the more esoteric categories later. You might consider the same approach - let's get your Login items work, Credit Cards, etc.

Let's see if this gets your further down the line...

Edit: I'm not able to get the strings from the binary. They are not readily accessible and I'm spending too much time trying discover how they are stored. So, our best bet is the --dump above. That may be enough to help me along.

Gerhard0007

Hi MRC !

Great work ! The "Passwort" changes everything ;-)

I attached the wetransfer.com link https://we.tl/rKCoWQrFdj from --dump output as file - created by adding ">result.txt" to the command line:
convert_to_1p4>convert_to_1p4.pl spbwallet Container1.swl -v -d --dump >result.txt

Until now i used a test file without real data inside - i think i have to contact my customer now to do the same job with his "real" .swl file so that we can get all used categories and field names before me make any changes in the convert script.

Regards,
Gerhard

MrC

@Gerhard0007 ,

Great!

With your Categories / Fields dump, I can quickly create the required languages file. I'll go ahead and add the few changes I need to support languages in the spbwallet converter.

tintagel

+1 on the fixes to Converters/Keychain.pm in v1.09! Thank you for these great converters!

AGAlumB

I think we should put MrC up for UNESCO World Heritage. :chuffed:

jmknoble

Hi, all.

I'm trialling a conversion from KeePassX v0.4.3 to 1Password 6 on OS X 10.11.5 (El Capitan).

I've exported my kdb to XML and am using MrC's much-lauded convert_to_1p4.pl utility to convert to 1PIF for import. However, my KeePassX database seems to have a number of date fields with the value "Never", which the converter happily passes to Time::Piece::strptime(), which doesn't grok "Never":

    $ /usr/bin/perl ./convert_to_1p4.pl keepassx --verbose --folders --modified --nowatchtower ~/Desktop/onepassword-utilities-master/jmknoble1-kdb.xml
    Uncaught exception from user code:
        Error parsing time at /System/Library/Perl/5.18/darwin-thread-multi-2level/Time/Piece.pm line 469.
        Time::Piece::strptime('Time::Piece', 'Never', '%Y-%m-%dT%H:%M:%S') called at Converters/Keepassx.pm line 138
        Converters::Keepassx::parse_date_string('Never') called at Converters/Keepassx.pm line 146
        Converters::Keepassx::date2epoch('Never') called at Converters/Keepassx.pm line 85
        Converters::Keepassx::do_import('/Users/jmknoble/Desktop/onepassword-utilities-master/jmknoble...', undef) called at ./convert_to_1p4.pl line 123`

"Never" can apparently appear for any of the date fields provided by KeePassX:

    $ fgrep '>Never<' ~/Desktop/onepassword-utilities-master/jmknoble1-kdb.xml |sed -e 's/^  *//' -e 's/  *$//' |sort |uniq
    <creation>Never</creation>
    <expire>Never</expire>
    <lastaccess>Never</lastaccess>
    <lastmod>Never</lastmod>

I can modify Converters/Keepassx.pm to handle these fields, but I don't know what 1Password expects to mean "Never" for these sorts of dates. Should the dates be left out of the 1PIF file? Or should they be provided, but with some sort of "magic" value?

MrC

Hi @jmknoble,

"much-lauded" - blush.

Sorry for the inconvenience, I had not seen the value in those fields. I'll update the code. Hold on and I'll post the update.

MrC

@jmknoble ,

Thanks for the excellent debugging and diagnostic report. I've updated the Testing Bits code.

Here's all you need to change, if you want to do it manually, in the file Converters/Keepassx.pm, add line 138 below to your code in that location:

134 sub parse_date_string {
135     local $_ = $_[0];
136     my $when = $_[1] || 0;                                      # -1 = past only, 0 = assume this century, 1 = future only, 2 = 50-yr moving window
137
138     return undef if $_ eq 'Never';
139
140     if (my $t = Time::Piece->strptime($_, "%Y-%m-%dT%H:%M:%S")) {       # KeePassX dates are in standard UTC string format, no TZ
141         return $t;
142     }

On import, you'll see the unmappable date stamps in your Notes area:

lastaccess: Never
creation: Never
expire: Never

5andwich

I exported Keychain Items as described in the PDF and followed the instructions precisely. I have done the entire process 3 times now. When I go to convert the Keychain file to a 1PW file, the terminal output didn't look promising.

Examined 1232 items
Skipped 1232 non-login items
Skipped 0 duplicate items
Imported 0 items
Exported 0 total items

I can assure you, that the majority of those are not non-login items.

When I attempted to import into 1PW, I got the expected error "No items found"

Mac OS X 10.11.5
1PW Stable

On trial, would like to buy, but not if I lose my passwords. Slightly discouraging. Thanks!

Edit: Looks like I can do this in LastPass without any issues or price tag..simply using the Chrome extension...hmmm

MrC

Hi @5andwich ,

This is a known issue with the 1.08 version. I corrected the problem in the 1.09 version in Testing Bits, mentioned in the first post of the converter suite thread.

Sorry about the troubles. I didn't know about a particular line in a Keychain entry until a user brought it to my attention. Please don't hold AgileBits to blame for my mistakes or oversights.

5andwich

Nobody is to blame! I know this is a community supported item, and programming has bugs, I am dealing with some of my own right now :(

That said, being able to import logins is very important, it's an objective reality.

Thanks for the clarification, I'll give 1.09 a try now! :)

Edit: Worked like a charm, thanks @MrC !

Megan

Hi @5andwich,

I’m so glad to see that MrC was able to get things working for you - whoo hoo! I hope that all your data is now safely inside 1Password.

If you have any further questions or concerns, please don’t hesitate to reach out, we’re always here to help. :)

benklebe

So the Keychain Access application seems very buggy, at least to me. Copying and pasting doesn't work at all for me unless both keychains being copied too and from are unlocked, and even then it will fail to copy the first item but do all the other ones. However, after it has done so it will appear as if only the tokens and network passwords have been copied over until I delete one of the entries after which it flashes and everything else appears. This is very weird and kind of untrustworthy. Has anyone else had these issues?

MrC

Hi @benklebe ,

Sorry, I've not heard of that problem. Nobody has ever mentioned it before.

Do you have any clipboard software installed, or anything else that might be causing some interference?

benklebe

@MrC no, I do not have any clipboard software installed. I've googled the error I get on the first attempt, and it is this: "The contents of this item cannot be retrieved." After that it goes smoothly except for a few more times when the script fails on this error again, and I have to manually reset it before continuing. I am now attempting a reboot to see if that helps.

MrC

@benklebe ,

Which script is getting the error? The AppleScript to help press the Yes button? If so, don't use it and just do lots of manual button presses.

Also, be sure to use the 1.09 version of the converter in Testing Bits, mentioned in the converter suite thread.

benklebe

@MrC , the script is not getting the error. Keychain Access is throwing an error that looks like this:

This is the AppleScript that is supposed to enter your password lots of times to authenticate the transfer of iCloud entries into the local keychain. I could enter my password hundreds of times but that seems error prone and this really isn't the script's fault, it seems to be a Keychain Access bug. (I am a programmer myself)

I was just wondering if anyone here had this problem and/or knew a good workaround. The reboot did nothing to help it.

MrC

@benklebe,

The only time I've seen any types of errors along these lines are when the Keychain is corrupted. This can happen when your local keychain store and the iCloud stores are out of sync. The only way I've been able to resolve the issue is to delete the iCloud keychain and re-sync. This is done by disabling Keychain syncing on the device and resyncing. This is from memory, and I don't recall the exact procedure, but a webby search would be best since others have provided step-by-step instructions to work through this.

The Keychain syncing seems very fickle.

benklebe

@MrC I deleted the iCloud keychain and re-synced it. After a few tries with the AppleScript it's working perfectly. The corruption was really tripping it up. I should be using the tester, not the stable converter, right?

benklebe

Update: Everything went swimmingly and my old keychain is all imported! What magnificent work you've done, @MrC

MrC

@benklebe,

Excellent!

Yes, the 1.09 version in Testing Bits is what I was referring to. It fixes a bug with an unanticipated extra line in the keychain output.

Congratulations on working out the issue. Enjoy 1Password.

Megan

Hi @benklebe,

I’m so glad to see that MrC was able to help you get your data into 1Password. If you ever have any other questions, we’re always here for you. :)

marcuscent

Hi. I would like to use 1password but I have a lot of legacy data in Password Agent. I have followed all the steps to try and migrate my data but I'm not getting a valid file at the end of the process. Using the command - perl convert_to_1p4.pl passwordagent -v ..\passwords.xml, I do get an output file but it processes 0 records so the import file is empty. There is no other error message I can see. My XML export file from PasswordAgent does include all the data so I'm not sure where it's going wrong. Any help appreciated.

Megan

Hi @marcuscent,

Unfortunately we don’t have a built-in importer for Password Agent, so the conversion tools are the best way to get your data imported. @MrC is our friendly neighbourhood import guru and I’m sure he’ll be able to figure out where things are getting tangled up here. :)

MrC

Hi @marcuscent ,

When you exported your data, were you sure that you checked the All selector above the fields listings? The converter is expecting all the fields to be exported.

Which version of Password Agent are you using?

You can get additional diagnostics adding the -d option. Example:

$ perl convert_to_1p4.pl passwordagent -v -d ../../_TestFiles/passwordagent_win.xml
main                : Command Line: passwordagent -v -d ../../_TestFiles/passwordagent_win.xml
main                : Output file: /Users/cappella/Desktop/1P_import.1pif
print_fileinfo      : Export file info
print_fileinfo      :   size: 1302
print_fileinfo      :   kind: XML  document text
print_fileinfo      :   mime: application/xml; charset=iso-8859-1
do_import           : Group: A Group
do_import           :       field: name = an empty note
do_import           :       field: account =
do_import           :       field: password =
do_import           :       field: link =
do_import           :       field: note =
do_import           :       field: date_added = 6/30/2015
do_import           :       field: date_modified = 7/2/2015
do_import           :       field: date_expire =
normalize_card_data :   pushed to notes: date_added: 6/30/2015
normalize_card_data :   pushed to notes: date_modified: 7/2/2015
print_record        : title: an empty note
                      tags:  A Group
                      notes: date_added: 6/30/2015<CR>date_modified: 7/2/2015
do_import           : Group: A Group::A Sub Group
do_import           :       field: name = an Account
do_import           :       field: account = sam@example.com
do_import           :       field: password = supersecret
do_import           :       field: link = www.example.com
do_import           :       field: note = some,
super
"
<
>
secret
à é ø ß C

DONE
do_import           :       field: date_added = 7/2/2015
do_import           :       field: date_modified = 7/2/2015
do_import           :       field: date_expire =
normalize_card_data :   pushed to notes: date_added: 7/2/2015
normalize_card_data :   pushed to notes: date_modified: 7/2/2015
print_record        : title: an Account
                      tags:  A Group::A Sub Group
                      key(url): link = www.example.com
                      key(password): password = supersecret
                      key(username): account = sam@example.com
                      notes: date_added: 7/2/2015<CR>date_modified: 7/2/2015<CR><CR>some,<CR>super<CR>"<CR><<CR>><CR>secret<CR>à é ø ß C<CR><CR>DO...

Perhaps you can examine the output and compare it to the sample output above. Do you see any differences that might be relevant?

marcuscent

Thanks. I am using password agent lite x64 2016.6.1. I don't see an option to select "all". The export function only allows me to select CSV or XML. There are no other options. Perhaps a limitation of the free version?

MrC

@marcuscent ,

The version I'd written the converter for was 2.6.3. Your version is a new major version bump, as per their changes notes, This new version is completely re-written and contains many changes. So, I'll have to take a look at the output of that version and see how it has changed. I'll take a look today.

MrC

@marcuscent ,

Ok, the XML data format is entirely different. I'll update the converter to handle the new format. Give me a day or two, perhaps even today if I get through my workload quickly enough. I'll update this post and ping you when its ready.