Memorable password generator: Random vs. non-random choice and entropy loss/gain

@gr1516 - great questions! I can't say what people tend to do because we intentionally know nothing about the generation of your password(s); all that logic is kept client-side (in your 1Password app or browser). However, it would not surprise me to learn that there are indeed people who will "skip" a generated password if it contains words with which they're unfamiliar (on the theory that it would be easier to forget/misspell such words, perhaps).

Entropy can be thought of as the "cost" of brute-forcing the correct password (no shortcuts, just straight guessing one after the next until the correct password is discovered). For a password with an entropy of n bits, your attacker would need to check 2^(n-1) passwords before hitting the correct one (the "minus one" here being half, since we're using binary logarithm). Assuming the generated passwords are equally probable (i.e. the generator is a well-designed cryptographically secure pseudorandom number generator, which ours is), then you have n bits of entropy when there are 2^n possible passwords.

Unlike the formula for random-character passwords, which has to take into account character-space (which varies with allowed symbols, etc), word-list password entropy-estimation is considerably simpler. If you have (like we do) a random wordlist of what appears today to be 18,176 words (if I counted correctly, heh), then 2^n = 18176*, and therefore n = 14.1497, or, in non-math-speak English: just under 14.15 bits of entropy per word. From there, it's a simple matter of multiplying that by the number of words. Three words? 42.45 bits of entropy. Four words? 56.6. Five words? 70.75. And so on.

So the question of how much entropy will be lost when a person self-limits the word pool as you've suggested (or by any other method) is an equally simple two-parter:

1. Change the calculation of 2^n = 18176 into whatever you think (or know) the word list will be shortened into. If you shave off, say, 1,500 words, then it would be 2^n = 16676, or n= 14.0255.
2. Again, how many words are chosen.

As you may have guessed, it is that latter point - how many words are chosen - which will make the most significant difference. Four words at "full strength" would be 56.6bits of entropy, while at the reduced total wordlist of 16676, it would be 56.102. That may seem like a small amount, but remember, each additional bit is TWICE the "size," so small fractions definitely make a difference. But you can very easily address that reduction by simply adding another word, even using the smaller list. Allowing the generator to randomly choose five words instead of four increases the entropy to 70.1275 bits, which is vastly greater in real terms than four words using either list.

So, to sum up: if you're worried about lost entropy (and there is indeed some), simply increase the number of (randomly chosen) words in your passphrase. Hope that's helpful.

@gr1516 - we agree; the work done there by that Redditor is quite good, as comments by both our founder @dteare and our principal security architect, @jpgoldberg in that Reddit thread attest to.

I can't say how many words the average person knows, but I would strongly urge people not to reject 90% of the list just because they don't like the words being offered. We did do some pruning a while back of words that either clearly were or could arguably be construed as offensive or objectionable. There might be some others, and there might be a few words that are more uncommon and therefore less well-known, such as abscissa or breccia (both on our list), but that group doesn't even come close to approaching 90% of the list. Most words are more like cabinet and matter and similar words that I would imagine the vast majority of people are familiar with.

Even at 10% where, yes, four words would be not expensive enough to resist cracking, using your calculations, six words appears to still be well into the trillions of dollars, all-but-impossible for virtually everyone. Sure, six words is harder to remember than four...but there are options: don't restrict the word list so severely, or if you must, use more words and - as the now-famous xkcd comic puts it, use a simple mnemonic that makes it very difficult to forget, even for people with memory issues: