MrC's Convert to 1Password Utility (mrc-converter-suite)
Comments
-
Looks like a module that I use is not present in Strawberry Perl. I'll add it to the module list to include.
Easy fix. For you, in the portableshell.bat command shell window, type:
cpan install -f Win32::Unicode::File
and hit Enter. It will install the module, and you should be able to try the converter again.
0 -
Thank you very much. I tried to install the modul, but it failed because of a test. After installing it with force option like you suggested, your script runs fine. It decoded all my Firefox passwords and stored it in the export file.
This saved so much time :-)
0 -
I can't copy anything from iCloud Keychain to 'local-cloud' without getting this error, "An error has occurred. Unable to add an item to the current keychain. One or more parameters passed to a function were not valid." This is so frustrating, I just want my passwords OUT of iCloud Keychain!
0 -
@mkhoury ,
Please see this post. Make sure you have version 1.11 of the converter suite. The scripts are included with the archive now, and not in the Testing Bits area.
The README and converter docs are being updated to explain more about the issues with copying iCloud keychain in macOS versions Sierra and beyond. A preview:
0 -
Thanks! That did the trick!
0 -
I've posted a significant update to the converter suite.
- The convert_to-1p4 naming is retired - the suite is now called the mrc-converter-suite.
- The Stable Bits / Testing BIts areas are retired. Instead, I will update the single, dated archive when I make changes.
- The README.pdf is retired, replaced by an much better, easy flowing README.html.
- All documentation has been re-written and much of it auto-generated.
- For Windows users, the installer version of Strawberry Perl is now prescribed instead of the portable version.
- See the Changes.txt file for a list of changes.
I've tested every step, reread the docs a dozen times, but most certainly I've overlooked something.
0 -
I'm using the converter, since importing from CSV is mysteriously not working anymore. The converter outputs the .1pif file but everything imports as NOTES and not Logins.
Update: importing the same CSV file via the website works.
0 -
@mkhoury ,
I'm not seeing any issues here:
$ cat _TestFiles/CSV/login-utf8.csv "title","login username","login password","login url","notes","tags","cust col 1","created","modified" "Joe's Login","Joe Blow","supersecret","http://www.example.com","what notes?","TAG1","cust val 1","1522630494","1522604494" "Sally's Login","Sally Smith","supersecret","http://xxx.example.com",Üñîçø∂é,"TAG2","cust val 1","1522630494d","1522604494x" $ perl convert.pl csv -v _TestFiles/CSV/login-utf8-win.csv Imported 2 items Exported 2 login items Exported 2 total items You may now import the file /Users/MrC/Desktop/1P_import.1pif into 1Password $ perl convert.pl csv -v _TestFiles/CSV/login-utf8.csv Imported 2 items Exported 2 login items Exported 2 total items You may now import the file /Users/MrC/Desktop/1P_import.1pif into 1Password $ perl convert.pl csv -vd _TestFiles/CSV/login-utf8.csv main : Runninng script from '/Users/MrC/Documents/Coding/mrc-converter-suite' main : Command Line: csv -vd _TestFiles/CSV/login-utf8.csv main : Output file: /Users/MrC/Desktop/1P_import.1pif print_fileinfo : Input file info: "_TestFiles/CSV/login-utf8.csv" print_fileinfo : size: 360 print_fileinfo : kind: UTF-8 Unicode text print_fileinfo : mime: text/plain; charset=utf-8 find_card_type : type detected as 'login' do_import : ROW: 1 normalize_card_data : field: login username normalize_card_data : field: login url normalize_card_data : field: cust col 1 normalize_card_data : pushed to notes: cust col 1: cust val 1 normalize_card_data : field: login password print_record : title: Joe's Login tags: TAG1 key(password): login password = supersecret key(url): login url = http://www.example.com key(username): login username = Joe Blow notes: cust col 1: cust val 1<CR><CR>what notes? do_import : ROW: 2 do_import : Invalid modified epoch date: 1522604494x do_import : Invalid created epoch date: 1522630494d normalize_card_data : field: modified normalize_card_data : field: created normalize_card_data : field: login password normalize_card_data : field: login url normalize_card_data : field: cust col 1 normalize_card_data : pushed to notes: cust col 1: cust val 1 normalize_card_data : field: login username print_record : title: Sally's Login tags: TAG2 key(username): login username = Sally Smith key(url): login url = http://xxx.example.com key(password): login password = supersecret key(created): created = 1522630494d key(modified): modified = 1522604494x notes: cust col 1: cust val 1<CR><CR>Üñîçø∂é Imported 2 items create_pif_record : Title: Joe's Login create_pif_record : key test(username): Joe Blow create_pif_record : key test(url): http://www.example.com create_pif_record : key test(password): supersecret create_pif_record : notes: cust col 1: cust val 1<CR><CR>what notes? create_pif_record : tags: TAG1 create_pif_record : Title: Sally's Login create_pif_record : key test(modified), Not found create_pif_record : key test(created), Not found create_pif_record : key test(password): supersecret create_pif_record : key test(url): http://xxx.example.com create_pif_record : key test(username): Sally Smith create_pif_record : notes: cust col 1: cust val 1<CR><CR>Üñîçø∂é create_pif_record : tags: TAG2 create_pif_record : *unmapped card field pushed to notes: modified create_pif_record : *unmapped card field pushed to notes: created Exported 2 login items Exported 2 total items You may now import the file /Users/MrC/Desktop/1P_import.1pif into 1PasswordWhat platform?
Are you using the new mrc-converter-suite package?
Do you have the correct column labels?0 -
Used the conversion script to take password out of Safari and into CSV (like I've done on two other computers now). Then I tried to import the CSV into the 1Password macOS app. It kept throwing an error. Tried the conversion script and that wouldn't convert anything but the stuff in the CSV into "notes" (didn't want that). Thought about trying the CSV importer via the web, that worked like a charm.
0 -
Aha!
@mkhoury - I found a bug and fixed it. Can you give the update I just posted a try?
2019-01-25 -----------------------------------------------------------------------------------------
- Fix [csv]
Fix issue mishandling special columns that were not present in the CSV (e.g. tags, notes, created,
modified).
0 - Fix [csv]
-
That did the trick!
0 -
Hi @MrC.
Thank you for writing such a wonderful and well-documented conversion tool!
I get the error message below when trying to convert from a KeePass 2 XML file. For context, the process to create the file was as follows:
- Windows XP KeePass 1.x file (.kdb) imported into Windows KeePass 2.x (.kdbx)
- Exported Windows KeePass 2.x (.kdbx) to KeePass 2 XML file
- XML file is saved to desktop and shared between Windows XP VM and Mac via Parallels
- Converter is run on Mac using macOSConvertHelper to generate command in Terminal
Error message:
Uncaught exception from user code:
Error parsing time at /System/Library/Perl/5.18/darwin-thread-multi-2level/Time/Piece.pm line 469.
Time::Piece::strptime('Time::Piece', '0002-11-30T08:00:00Z', '%Y-%m-%dT%H:%M:%SZ') called at /Users/roopesh/Desktop/mrc-converter-suite/./Converters/Keepass2.pm line 278
Converters::Keepass2::parse_date_string('0002-11-30T08:00:00Z') called at /Users/roopesh/Desktop/mrc-converter-suite/./Converters/Keepass2.pm line 286
Converters::Keepass2::date2epoch('0002-11-30T08:00:00Z') called at /Users/roopesh/Desktop/mrc-converter-suite/./Converters/Keepass2.pm line 124
Converters::Keepass2::do_import('/Users/roopesh/Desktop/pm_export.txt', undef) called at convert.pl line 1690 -
@MrC,
I figured, and tried searching within KeePass to find the problematic item but it doesn’t return any results. Perhaps the export process modifies the date formats. I suppose I could regenerate the XML and try searching directly. But since you have an update I’ll give that a try!
Do you accept donations anywhere for your efforts?
0 -
It is a curious one, and I've never encountered such an issue w/KeePass 2, so have no idea how that date was derived. You can turn on debugging if you are curious to find it. You'd look for lines such as:
get_entrydata_from_entry: key: LastModificationTime: '2015-04-30T03:14:20Z' get_entrydata_from_entry: key: CreationTime: '2015-04-30T03:12:20Z'
Debugging is enabled with
-dor--debug:perl convert.pl keepass2 -dv TestFiles/keepass2.xml
But its no big deal, as testing for the failed date parsing is something that I should do anyway. One simple reason I had not done this - I want to get feedback like yours, since I can't create all the possible permutations of user data (I do try to test all the edge cases I can think of).
Your compliments and offer have been a sufficient donation to me.
0 -
@MrC if I quit the apple script before it completes, are the passwords that it has captured saved somewhere? I want to ensure they get deleted.. I decided to go a different route than copying plain text.
0 -
@djshockd - That's great news. Enjoy 1Password!
@d2theustin - Which script - there are 3 AppleScripts - the macOSConvertHelper and two Get_Safari##_Passwords scripts. Just so I'm clear.
The two Get_Safari... scripts do not write out their contents until all rows have been processed. The internally collected data dies with script termination.
0 -
Hi MRC,
First, like most of us, a HUGE immense thank you for your work on these conversion tools.
Here's the issue I'm encountering trying to run the convert.pl for chrome
$ perl convert.pl -v chrome Error: failed to load converter module 'chrome' Can't locate PBKDF2/Tiny.pm in @INC (you may need to install the PBKDF2::Tiny module) (@INC contains: /Users/username/Desktop/mrc-converter-suite/. /Users/username/Desktop/mrc-converter-suite/lib /Library/Perl/5.18/darwin-thread-multi-2level /Library/Perl/5.18 /Network/Library/Perl/5.18/darwin-thread-multi-2level /Network/Library/Perl/5.18 /Library/Perl/Updates/5.18.2 /System/Library/Perl/5.18/darwin-thread-multi-2level /System/Library/Perl/5.18 /System/Library/Perl/Extras/5.18/darwin-thread-multi-2level /System/Library/Perl/Extras/5.18 .) at /System/Library/Perl/5.18/if.pm line 13. BEGIN failed--compilation aborted at /Users/debbienordstrom/Desktop/mrc-converter-suite/./Converters/Chrome.pm line 23. Compilation failed in require at convert.pl line 53. Usage: convert.pl <options> chrome <export file> Options: Select one of the converters above and add it to the command line to see more complete options. Example: perl convert.pl ewallet --helpI've tried using /usr/bin/perl5.16 or /usr/bin/perl5.16 convert.pl -v chrome with the same results.
Similar issue with csv option by the way (I was able to work around that one by using a different computer running Mojave to convert the .csv file, a Safari password export)I can't use that same work around with Chrome as it doesn't use an export file that aI can copy to another computer.
Any ideas?Merci beaucoup!
System Info:
Downloaded mrc-convert-suite Today (Latest I imagine)
macOS 10.11.6
Chrome 71.0.3578.98 Official Build 64bit0 -
Hi MRC,
First, like most of us, a HUGE immense thank you for your work on these conversion tools.
Here's the issue I'm encountering trying to run the convert.pl for chrome
$ perl convert.pl -v chrome Error: failed to load converter module 'chrome' Can't locate PBKDF2/Tiny.pm in @INC (you may need to install the PBKDF2::Tiny module) (@INC contains: /Users/username/Desktop/mrc-converter-suite/. /Users/username/Desktop/mrc-converter-suite/lib /Library/Perl/5.18/darwin-thread-multi-2level /Library/Perl/5.18 /Network/Library/Perl/5.18/darwin-thread-multi-2level /Network/Library/Perl/5.18 /Library/Perl/Updates/5.18.2 /System/Library/Perl/5.18/darwin-thread-multi-2level /System/Library/Perl/5.18 /System/Library/Perl/Extras/5.18/darwin-thread-multi-2level /System/Library/Perl/Extras/5.18 .) at /System/Library/Perl/5.18/if.pm line 13. BEGIN failed--compilation aborted at /Users/username/Desktop/mrc-converter-suite/./Converters/Chrome.pm line 23. Compilation failed in require at convert.pl line 53. Usage: convert.pl <options> chrome <export file> Options: Select one of the converters above and add it to the command line to see more complete options. Example: perl convert.pl ewallet --helpI've tried using /usr/bin/perl5.16 or /usr/bin/perl5.16 convert.pl -v chrome with the same results.
Similar issue with csv option by the way (I was able to work around that one by using a different computer running Mojave to convert the .csv file, a Safari password export)I can't use that same work around with Chrome as it doesn't use an export file that aI can copy to another computer.
Any ideas?Thanks amillio
System Info
Downloaded mrc-convert-suite Today (Latest I imagine)
macOS 10.11.6
Chrome 71.0.3578.98 Official Build 64bit0 -
Hi MRC,
When running macOSConvertHelper to convert the Safari passwords CSV, I keep getting the error in the attached screenshot.
I'm running 10.13.6 on an iMac Pro.
0 -
Importing the CSV file (vs converting it) seemed to work fine.
However, when reviewing duplicate passwords, one of the first items that I updated brought in 4 additional sites with the same domain. The first password I changed updated 2 domains. The next password I changed updated 1 domain, and I still 2 domains to go buried in the notes of the imported item, resulting in a bunch of copy/pasting and logging in/out of multiple browser tabs. Therefore, I would definitely prefer to have each (Safari) shared domain broken out as 5 separate items (in this case), as the 1:1 changes would be more streamlined, and less confusing to track.
0 -
Hi @markaceto ,
Let me see if I can replicate the issue you see on 10.13 with macOSConvertHelper. You can just use the command line for now if necessary - the macOSConvertHelper job is to just build it for you anyway.
RE: the multiple web-host names that can come out of Safari when it groups like-items, it is an annoying Safari behavior. I don't think I want to have the AppleScript code split records for several reasons. Rather, what I'd prefer to do is create a special CSV column and place those items in that column. Then program the csv converter to know about that special column, and based on an option, let the converter do the record-splitting. This is a more generic solution that could be applicable to other CSV exports, and frankly, the converter suite's scripting language is much agile and easier to work with than AppleScript (which is kind of like teaching a child to ride a bike by instructing, now flex this muscle slightly, elongate this limb a little, lean to your left side, push, turn the handlebars 2 degrees right, lean...)
How does that sound?
Edit: I believe the issue with the script being unable to a Terminal window is caused if you don't Allow Script Editor access to Terminal. macOS will prompt you to Allow access. Perhaps that dialog was not noticed, or cancelled.
0 -
Thanks for the quick reply!
Unless I'm mistaken, the conversion is an extra step because I can just import the CSV directly into 1p 6.8.9 (not sure about v7).
RE: the multiple web-host names that can come out of Safari... programming in AppleScript sounds like a nightmare, so your proposed solutions sounds good. I'm concerned about (my own) user error when I get lost in the minutia, so whatever is the most bulletproof solution works for me :)
BTW:
1. Scrolling through the list of passwords in Safari prefs, I have roughly 60 grouped items (ranging from 1-4 other instances each) out of 340 "subtotal" so I actually have upwards of 400 "grand total". I would definitely screw up a few of those, even when working in small batches.
2. I also have the issue of a blank website reappearing from iCloud sync, and I also have to quad-click it to open the detail sheet. The first double-click does nothing. The second double-click opens the sheet.I'm really excited about getting out of Safari/iCloud/Keychain, and into a more user-serviceable system!
0 -
You're welcome.
Yes, you can import directly. The CSV generated by the applescript is very simple - there's not much data there - just a few cells / row. The converter proper provides more abilities, but they may be inconsequential for your needs. I just don't want to spend much more time working on both of the AppleScripts (which seem to require updates each macOS / Safari release), and the script flow is based on one CSV row at a time - and I certainly don't want to start adding options processing (some users will want the URLs grouped).
Since 1Password has the ability to handle multiple URLs per Login record, the right want to deal with this is for the csv converter to place them into those supplementary URL areas rather than notes. This way, I can add an option (like --addurls, I'll think about a good name), and make it user's choice. With this, inside of 1Password you have one record, but you can click on any of the links. I could also provide the ability to add the URLs to Notes, or split the records, so you can choose how to handle them.
Eh, gads. I've almost never saved passwords in browsers. I didn't know about Safari grouping items until someone reported the issue, and I worked very hard to figure out how they those entries were created. Never occurred to me Safari would group them by itself automatically (because its basically the wrong approach, and error prone). Nor did I know how to create blank entries, but eventually figured it out (bug in older macOS'). And the blank entries not opening without effort, or them even being stored at all, is clearly a Safara / iCloud bug. The applescript tries to work around this issue. Is it failing for you?
0 -
Sorry, which version of Safari are you using on 10.13 where you ran the script?
0 -
@mrc,
If Apple hadn't broken iCloud Keychain export, I'd be using that for sure. My plan is to get all the passwords out of Safari, change the duplicate/weak/vulnerable ones, and then delete them from Safari. That should also purge them from iCloud Keychain, leaving a significantly more manageable batch for me to start pulling into 1p. My goal is to strip everything but system level stuff out of the Keychain. Process of elimination...
I'm using Safari 12.0.3
0 -
Yup, I get it.
I need to create a VM environment on 10.13 to test out the issue w/Safari 12. I have 11 on my VM currently. I'll see what I can find.
0
