306 million breached passwords made available by Troy Hunt. Useful for 1Password somehow?

pervel

Security expert Troy Hunt has made 306 million breached passwords available here:
Introducing 306 Million Freely Downloadable Pwned Passwords

Do you think 1Password could potentially make use of this? Maybe as an extension to the existing security audits (e.g. Watchtower)? I guess one argument against it is the size of the file: 5.3GB compressed.

In any case, it's pretty cool stuff.

---

1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Sync Type: Not Provided

XIII

If not, maybe someone (@MrC?) can write a script that runs SHA1 on passwords exported (from 1Password) in plain text and performs a grep on the local 11 GB dump provided by Troy Hunt?

(I have no idea how long it would take to verify hundreds of passwords for inclusion in such a large file)

XIII

Additional thought: for a bit more security AgileBits might consider an (advanced) option to export SHA-1 hashes of passwords instead of the passwords themselves in plain text for this purpose?

XIII

Hm, a standard grep on the command line took about 6 minutes for a single SHA-1 hash...

I guess a binary search might be (much) faster.

pervel

I guess a binary search might be (much) faster.

Yea, it needs to be indexed for fast searching. If you try his online version, the search is almost instant: https://haveibeenpwned.com/Passwords

3rdparty

I came here to post this - so instead I'll add a very useful +1 request to this. Tricky due to size though.

pervel

Just found out about the look command. Since the file is already sorted, you can use that to do a really fast lookup in the file. Try it instead of grep.

AGAlumB

Great discussion! I'll preface this by saying that Troy Hunt is awesome and does great work...but it's worth mentioning that entering any passwords into a website isn't usually a great idea. But in this case, provided we can establish and verify a secure connection to https://haveibeenpwned.com we're at least only trusting his site. And ultimately these will probably be passwords we want to change anyway. ;)

As far as 1Password using something like this, while the file size is certainly a concern, it's also worth bearing in mind that this is a much more valuable resource when it comes to reused/weak passwords. If you have long, strong, unique passwords generated by 1Password itself for sites, finding that one has been compromised is helpful for that account but not others. So I'm not sure adding this to 1Password would give users much "bang for the buck", so to speak. It's an interesting idea though. :)

XIII

@pervel Thanks so much! I did not know about the look command; it's exactly what I needed :)
@brenty Troy agrees about entering passwords into a website; that's why you can use the SHA-1 hash instead. Even better: do it locally.

Turns out it is not so hard to do a local check manually on macOS; a script is thus not required.

Prerequisites:

• Pwnd Passwords downloaded and unpacked in ~/Downloads
• RAM disk created as /Volumes/RAMDisk
• Passwords in 1Password exported as CSV (plain text; that's why I use a volatile RAM disk!), with only the password column selected, in /Volumes/RAMDisk/1p.csv Note: you have to manually remove the starting and trailing " quotes and the trailing comma ,. Also make sure there are no empty lines, except for the last line, which should be empty. I used (regular expressions in) SublimeText to achieve this.

Generate SHA-1 hashes
while read -r line; do echo -n $line | openssl sha1; done < /Volumes/RAMDisk/1p.csv > /Volumes/RAMDisk/sha1.txt

Check occurrence of SHA-1 hashes in Pwnd Passwords
while read -r line; do look -df $line ~/Downloads/pwned-passwords-1.0.txt; done < /Volumes/RAMDisk/sha1.txt

My export of several hundreds of passwords had 10 hits... By matching the line number of the reported SHA-1 hashes in both /Volumes/RAMDisk/sha1.txt and /Volumes/RAMDisk/1p.csv I could backtrack which passwords were involved. All of them were small digit-only pincodes.

So thanks to 1Password (and LastPass and KeePass) for generating strong and unique passwords! :)

AGAlumB

@brenty Troy agrees about entering passwords into a website; that's why you can use the SHA-1 hash instead. Even better: do it locally.

@XIII: 100%! But I don't think a lot of people are going to download gigabytes of data and run a check locally, so I thought it was worth mentioning. :lol:

Thanks so much for sharing your method so that others can do that too if they wish. That's a nice breakdown. Cheers! :)