Clipperz Converter for 1Password 4

MrC
MrC
Volunteer Moderator
edited October 2014 in Mac

I've created a script to convert your Clipperz data into a format that can be imported by 1Password 4. The script can be run on OS X or Windows.

Edit: this version of the converter has been obsoleted. The new and improved version is located here:

https://discussions.agilebits.com/discussion/30286/mrcs-convert-to-1password-utility/p1

Comments

  • ash_m
    ash_m
    Community Member

    Thanks for this MrC!

    I've just run an import from Clipperz and the clipperz_to_1p4.pl script has output the following as unmapped fields:

    UNMAPPED FIELD Callback URL
    UNMAPPED FIELD callbackPW
    UNMAPPED FIELD Signature Fields
    UNMAPPED FIELD email
    UNMAPPED FIELD pass
    UNMAPPED FIELD website
    UNMAPPED FIELD user
    UNMAPPED FIELD Bank
    UNMAPPED FIELD Online banking ID
    UNMAPPED FIELD Security Number
    UNMAPPED FIELD Account number
    UNMAPPED FIELD Online banking password
    UNMAPPED FIELD Name
    UNMAPPED FIELD Key
    UNMAPPED FIELD Company
    UNMAPPED FIELD Encryption password
    UNMAPPED FIELD Web address
    UNMAPPED FIELD App Secret
    UNMAPPED FIELD App ID
    UNMAPPED FIELD Signature Field
    UNMAPPED FIELD Callback url
    UNMAPPED FIELD Payment Response password
    UNMAPPED FIELD Label 3
    UNMAPPED FIELD Label 2
    UNMAPPED FIELD Label 1

    I've checked over a handful of these and the data appears to have been imported just fine as secure notes which includes the field labels. Nothing to be concerned about here, right?

  • MrC
    MrC
    Volunteer Moderator
    edited June 2014

    Great, thanks for the feedback.

    The warnings are remnants from another conversion script where I needed to know about unmapped fields. In your case, everything unmapped goes to its own field or notes. So long as you have all your data, that's the good thing. I'll update the script to remove these extraneous warnings. Sorry about the noise.

    Edit: Script update posted.

  • MrC
    MrC
    Volunteer Moderator

    I've posted an update which sets each items Created time to 1/1/2000 to help trigger WatchTower checks. This can be defeated with the --nowatchtower option. See the last Note in the Read Me file.

  • scoter
    scoter
    Community Member

    When I run the script, I get this error:

    Wide character in subroutine entry at clipperz_to_1p4.pl line 149 (#1)
    (S utf8) Perl met a wide character (>255) when it wasn't expecting
    one. This warning is by default on for I/O (like print). The easiest
    way to quiet this warning is simply to add the :utf8 layer to the
    output, e.g. binmode STDOUT, ':utf8'. Another way to turn off the
    warning is to add no warnings 'utf8'; but that is often closer to
    cheating. In general, you are supposed to explicitly mark the
    filehandle with an encoding, see open and "binmode" in perlfunc.

    Uncaught exception from user code:
    Wide character in subroutine entry at clipperz_to_1p4.pl line 149.
    JSON::PP::PP_decode_json('JSON::PP=HASH(0x7fd201015110)', '[{"label":"24 Hour Fitness","data":{"currentVersionKey":"4da9...', 0) called at /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/JSON/PP.pm line 148
    JSON::PP::decode('JSON::PP=HASH(0x7fd201015110)', '[{"label":"24 Hour Fitness","data":{"currentVersionKey":"4da9...') called at /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/JSON/PP.pm line 110
    JSON::PP::decode_json('[{"label":"24 Hour Fitness","data":{"currentVersionKey":"4da9...') called at clipperz_to_1p4.pl line 149
    main::import_json() called at clipperz_to_1p4.pl line 133

    I'm using the script from GitHub.

    I saved the Clipperz JSON export as UTF-8 with Unix (LF) on a Mac.

    -pt

  • MrC
    MrC
    Volunteer Moderator
    edited September 2014

    Hi @scoter. Sorry for the trouble. I've just updated the script and posted it at the link in post #1.

    I'd just noticed and fixed this yesterday, but didn't have the time to post the update, I'm in the middle of overhauling the script to better support more card types. Currently all but Login types go to Secure Notes.

    See how the just-posted one works for you. If you don't like how it imports, I'll have the new version done in the next couple of days.

  • scoter
    scoter
    Community Member

    Hi @MrC, That works much better. It produced the 1P4_import.1pif file without issue.

    If you are going to have a better version in a couple of days, I will wait to do the import into 1Password.

    Thanks!

  • MrC
    MrC
    Volunteer Moderator
    edited September 2014

    Hi scoter,

    I've posted the updated script, and have tested it in OS X and Windows.

    The clipperz web program has the following stock types: Web password, Bank account, Credit card, Address book entry, Custom card, and Direct login. The first three are directly handled by the conversion script. In addition, for Bank account and Credit cards which have username, password, or URL fields, an additional 1P4 Login card is created (those fields will not be in the imported Bank account / Credit card).

    The other clipperz types will just go into Secure Notes, as there is no direct mapping.

    The converter is flexible, in that it was built to handle custom cards - these will require card and field definitions that map to existing 1P4 categories. Depending upon how many of these you have, the creation of these definitions may not be worth the time and effort.

  • scoter
    scoter
    Community Member

    Thanks, @MrC! I will try it out soon.

  • scoter
    scoter
    Community Member

    The script worked great. 633 cards imported! I couldn't have made the transition without it.

    The only challenge was I needed to install the latest version of Perl (on my Mac), plus Date/Calc.pm and Carp/Clan.pm.

    Thanks much!

  • MrC
    MrC
    Volunteer Moderator
    edited September 2014

    Nice!

    Thanks for the feedback and kind words.

    I'm curious which version of OS X you are running to not have those modules installed. They are stock in Mavericks w/Perl version 5.12 and 5.16:

    $ ls -l /System/Library/Perl/Extras/5.1[26]/{Date/Calc.pm,Carp/Clan.pm}
-rw-r--r--  1 root  wheel  8494 Oct 22  2013 /System/Library/Perl/Extras/5.12/Carp/Clan.pm
-rw-r--r--  1 root  wheel  2135 Oct 22  2013 /System/Library/Perl/Extras/5.12/Date/Calc.pm
-rw-r--r--  1 root  wheel  8494 Oct 22  2013 /System/Library/Perl/Extras/5.16/Carp/Clan.pm
-rw-r--r--  1 root  wheel  2135 Oct 22  2013 /System/Library/Perl/Extras/5.16/Date/Calc.pm

    Or maybe you're using your own Perl install.

  • scoter
    scoter
    Community Member

    I'm running OSX 10.9.5.

    When I ran the earlier script, I got this error:

    $ perl clipperz_to_1p4.pl -v ../clipperz_export_20071217.json
    Perl v5.14.0 required--this is only v5.12.4, stopped at clipperz_to_1p4.pl line 7.

    BEGIN failed--compilation aborted at clipperz_to_1p4.pl line 7.

    So I installed Perl 5.16 using perlbrew and it worked fine with the script you posted on 18 September.

    But when I ran the latest script you posted on 20 September, I got an error about Date/Calc.pm missing:

    $ perl clipperz_to_1p4.pl -v ../clipperz_export.txt
    Can't locate Date/Calc.pm in @INC (@INC contains: /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/darwin-2level /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0 /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/darwin-2level /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0 .) at PIF.pm line 18.
    BEGIN failed--compilation aborted at PIF.pm line 18.
    Compilation failed in require at clipperz_to_1p4.pl line 22.
    BEGIN failed--compilation aborted at clipperz_to_1p4.pl line 22 (#1)
    (F) You said to do (or require, or use) a file that couldn't be found.
    Perl looks for the file in all the locations mentioned in @INC, unless
    the file name included the full path to the file. Perhaps you need
    to set the PERL5LIB or PERL5OPT environment variable to say where the
    extra library is, or maybe the script needs to add the library name
    to @INC. Or maybe you just misspelled the name of the file. See
    "require" in perlfunc and lib.

    Uncaught exception from user code:
    Can't locate Date/Calc.pm in @INC (@INC contains: /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/darwin-2level /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0 /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/darwin-2level /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0 .) at PIF.pm line 18.
    BEGIN failed--compilation aborted at PIF.pm line 18.
    Compilation failed in require at clipperz_to_1p4.pl line 22.

    BEGIN failed--compilation aborted at clipperz_to_1p4.pl line 22.

    I installed that and then got the error about Carp/Clan.pm missing:

    $ perl clipperz_to_1p4.pl -v ../clipperz_export.txt
    Can't locate Carp/Clan.pm in @INC (@INC contains: /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/darwin-2level /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0 /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/darwin-2level /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0 .) at /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/Date/Calc/PP.pm line 18.
    BEGIN failed--compilation aborted at /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/Date/Calc/PP.pm line 18.
    Compilation failed in require at /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/Date/Calc.pm line 37.
    BEGIN failed--compilation aborted at /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/Date/Calc.pm line 43.
    Compilation failed in require at PIF.pm line 18.
    BEGIN failed--compilation aborted at PIF.pm line 18.
    Compilation failed in require at clipperz_to_1p4.pl line 22.
    BEGIN failed--compilation aborted at clipperz_to_1p4.pl line 22 (#1)
    (F) You said to do (or require, or use) a file that couldn't be found.
    Perl looks for the file in all the locations mentioned in @INC, unless
    the file name included the full path to the file. Perhaps you need
    to set the PERL5LIB or PERL5OPT environment variable to say where the
    extra library is, or maybe the script needs to add the library name
    to @INC. Or maybe you just misspelled the name of the file. See
    "require" in perlfunc and lib.

    Uncaught exception from user code:
    Can't locate Carp/Clan.pm in @INC (@INC contains: /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/darwin-2level /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0 /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/darwin-2level /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0 .) at /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/Date/Calc/PP.pm line 18.
    BEGIN failed--compilation aborted at /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/Date/Calc/PP.pm line 18.
    Compilation failed in require at /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/Date/Calc.pm line 37.
    BEGIN failed--compilation aborted at /Users/Patrick/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/Date/Calc.pm line 43.
    Compilation failed in require at PIF.pm line 18.
    BEGIN failed--compilation aborted at PIF.pm line 18.
    Compilation failed in require at clipperz_to_1p4.pl line 22.

    BEGIN failed--compilation aborted at clipperz_to_1p4.pl line 22.

    Once I installed that everything worked fine.

    Note, I'm not a Perl guy, so I may have been doing something stupid.

  • MrC
    MrC
    Volunteer Moderator
    edited September 2014

    I deleted the duplicate posting for you.

    It is possible that your homebrew install has changed your PATH, and how it deals with default binaries. You should have this on your Mavericks system (this one is a fresh 10.9.4 VM, with no additional tools installed):

    $ sw_vers -productVersion 
10.9.4

$ ls -l /usr/bin/perl*
-rwxr-xr-x   1 root  wheel  58608 Apr 17 10:10 /usr/bin/perl
-rwxr-xr-x   1 root  wheel  35440 Aug 25  2013 /usr/bin/perl5.12
-rwxr-xr-x   1 root  wheel  35440 Apr 17 10:10 /usr/bin/perl5.16
-rwxr-xr-x  39 root  wheel    811 Aug 25  2013 /usr/bin/perlbug
-rwxr-xr-x   1 root  wheel  46074 Aug 25  2013 /usr/bin/perlbug5.12
-rwxr-xr-x   1 root  wheel  41971 Aug 25  2013 /usr/bin/perlbug5.16
-rwxr-xr-x  39 root  wheel    811 Aug 25  2013 /usr/bin/perldoc
-rwxr-xr-x   1 root  wheel    259 Aug 25  2013 /usr/bin/perldoc5.12
-rwxr-xr-x   1 root  wheel    259 Aug 25  2013 /usr/bin/perldoc5.16
-rwxr-xr-x  39 root  wheel    811 Aug 25  2013 /usr/bin/perlivp
-rwxr-xr-x   1 root  wheel  12489 Aug 25  2013 /usr/bin/perlivp5.12
-rwxr-xr-x   1 root  wheel  10807 Aug 25  2013 /usr/bin/perlivp5.16
-rwxr-xr-x  39 root  wheel    811 Aug 25  2013 /usr/bin/perlthanks
-rwxr-xr-x   1 root  wheel  46074 Aug 25  2013 /usr/bin/perlthanks5.12
-rwxr-xr-x   1 root  wheel  41971 Aug 25  2013 /usr/bin/perlthanks5.16

$ which perl
/usr/bin/perl

$ /usr/bin/perl --version

This is perl 5, version 16, subversion 2 (v5.16.2) built for darwin-thread-multi-2level
(with 3 registered patches, see perl -V for more detail)

...

$ /usr/bin/perl -e 'use Date::Calc; use Carp::Clan; print "It works\n"'
It works

    Anyway, you have it working, and I don't think I need to worry about anything.

    FYI: the Date::Calc module was required after the update I made to do proper date parsing, validation, and conversion. This allows the source wallet's date fields to be placed into 1Password's date-related fields, whenever possible.

  • scoter
    scoter
    Community Member

    Yes, everything is working and I made the transition. Clipperz served me well for a long time, but got slower and slower as my number of cards increased. I'm happy with 1Password so far and love the integration with my Mac and iOS devices. Thanks again for your help.

This discussion has been closed.