Importing saved passwords from Firefox is too painful
I recently decided to start using 1password rather than relying on the Firefox password manager.
I have over 300 stored username/password pairs stored in the Firefox password manager and so obviously wanted to migrate those over to 1password.
However the process of doing so was extremely painful!
After some digging in these support forums I found instructions for using this [1] firefox addon to export a csv file, as well as the convert_to_1p4 utilities.
Unfortunately that isn't enough. The convert_to_1p4 utilities require the csv to be hand edited to fix up the column names, which is a relatively small problem. A much bigger problem is that the convert_to_1p4 utilities expect each entry to have a "title", which the firefox addon does not export since firefox password manager doesn't track website names, just website URLs.
After some digging in the source code for [1] I also noticed that it uses unconventional escaping for certain characters, which convert_to_1p4 doesn't deal with since it just has generic csv support.
So I wrote a python script which is specialized to deal with the csv files output by [1]. This script both deals with the unorthodox escaping, as well as generates a title for each username/password using the URL of the website.
I'm trying to attach this python script here, but for some reason it's not working. Let me know if there's any other way I can contribute this script.
[1] https://addons.mozilla.org/en-US/firefox/addon/password-exporter/
1Password Version: 6.5.3
Extension Version: 4.6.3.b1
OS Version: Not Provided
Sync Type: Not Provided
Referrer: forum-search:Importing saved passwords from Firefox is too painful
Comments
-
I'm still unable to attach the python script here. So I put it up in a github gist instead:
https://gist.github.com/sicking/fd38436a56f5363012d78fa8645cb8fb
0 -
Can you tell me what you mean by "unconventional escaping for certain characters" ? And I don't know what you mean by "generic csv support". I'm happy to resolve any issues you've found.
You can certainly use 1Password's native CSV import, but that requires columns to be in a certain order.
The alternative, which you've mentioned, is to label the columns as required to use the csv converter. The required Title column can just be a duplicated URL (or other) column. It is also easy enough to modify the conversion table in convert_to_1p4.pl to help customize the eventual Title value as well.
I can't do much about the difficulty in getting data out of other software packages, its just the way it is - few software companies care about or have any vested interest in help you leave their product. As far as 1Password's inability to directly import from Firefox, I'll just say on Agilebit's behalf, there are hundreds of password management software packages out there, and writing export instructions, converters, importers, etc. for all of them is a huge job, one for which almost no customers are willing to pay a premium.
0 -
The bad escaping that the exporter tool does is here:
I.e. it replace " with %22, < with %3C, etc.
The main problem with using the URL as the title is that it's requires modifying each row of the csv file to add that extra column. That's a lot of work to do for 300+ passwords in a text editor. It would be doable by using excel of similar spreadsheet tool, but since I'm only using web-based spreadsheets these days, I didn't want to upload all my passwords anywhere, even temporarily.
The other problem with using the URL as the title is that the "http://", "https://" and "www." makes it a bit hard to scan for and find resulting entries in the 1password UI. Though admittedly this might be less of a problem in practice.
I agree though that the lack of better exporting abilities in Firefox is hard to fix on 1password's end. The decision about which password managers to support should obviously be based on which ones are commonly used. If people rarely use the built-in password managers in browsers then obviously it's not worth prioritizing building a converter. It would surprise me if the browser based password managers are so rarely used, but I don't have data so happy to defer to AgileBits here.
Anyhow, I wasn't expecting anyone to do the work for me here. I've already written the converter tool and wanted to check if there was interest in me contributing it to the set of existing converters.
(For what it's worth, Firefox stores it's usernames/password in a sqlite database. This makes exporting non-trivial since it can't be done in a simple perl/python script, but rather requires linking in a binary. I'd be happy to help figure out the exact schema and file location of the sqlite database though if there's interest)
0 -
Oh, I see, you'd meant that the Firefox exporter does not do HTML entity decoding. Got it. Perhaps I should just make a converter for this to add to the suite. That would resolve all the issues.
Perl has SQL DB modules, and they come native on OS X so I can use those. Windows users would need to install one more module above the basic Strawberry Perl installation they do not, so its not a big deal.
If you'd like to work out the schema, I'll implement the converter - that would save me a fair amount of time, and we can do away with the clumsy method now required.
0 -
Sorry for being slow. I'll get you a schema on Monday.
0 -
Sadly it appears that I was entirely wrong. The passwords aren't stored in a sqlite database at all, but rather in a .json file. Unfortunately it appears that the usernames/passwords are encrypted, even if a master password is not enabled. Presumably they are encrypted with a known key though. But I'm having little luck with figuring out what crypto algorithm is used, and what key is used.
I'm still looking into what algorithm and key is used. However this was sadly a lot more complex than I had hoped.
0 -
Ok, I managed to get hold of some of the relevant engineers at mozilla. It appears that the username/password is first DES3 ("triple-des") encrypted, and then ASN.1 encoded.
Unfortunately ASN.1 encoding is not a specific encoding, but appears to be a more general framework.
The specific template used by mozilla is described here:
https://dxr.mozilla.org/mozilla-central/rev/5e17f9181c6cb0968966280d1c1d96e725702af1/security/nss/lib/pk11wrap/pk11sdr.c#25-32More information about ASN.1 here: http://luca.ntop.org/Teaching/Appunti/asn1.html
Then there's the matter of finding the key that's actually used. It might be located in the key3.db file, but likely that file uses some form of encoding.
So yeah, this was dramatically more complex than I had hoped.
Most likely the actual code would not be so complex. But figuring out what code needs to be written feels pretty non-trivial. And then some library would be needed to do the triple-des decryption.
It'd likely be dramatically easier to add the functionality to the 1password browser addon, since the browser has internal APIs which return the decoded username/password. But obviously that's a whole different ballgame.
0 -
Thanks for the update on what you were able to find out about that! It sounds like it all turned out to be much more complicated than you expected.
If you still need help to import that data into 1Password, perhaps @MrC will have some suggestions. Please don't hesitate to let us know if you have more questions. Have a great weekend! :)
0
