Diceware dictionary size and feature request

dfz
dfz
Community Member

Can you give some specs on your Diceware solution:
1. Number of dictionary entries (e.g. Diceware is 7776. How long is yours?)
2. word size range (shortest to longest) Looks like 3-8, correct?
3. What's the entropy value for each word in the password string.

Requests:

Can you add "none" to the delimiter list? Some of us like to just type word strings quickly. A little more readability difficulty is not an issue in most cases

Can you add ";" (semicolon) to the delimiter list?
This is the fastest and easiest delimiter to type since it's right under the right hand 4th finger.

Can you add initial caps such as: BreakSettlerButtonDestination
This might be lowest priority; a user survey might help determine whether to include this.

I know there's a risk of making the list excessive, but I think, at least, the first 2 would be popular additions.


1Password Version: 6.0
Extension Version: 4.4.4
OS Version: OSX 10.11.1
Sync Type: iCloud & Dropbox
Referrer: forum-search:diceware

Comments

  • Hi @dfz,

    There are currently approximately between 18000 and 18500 words in the list. Yes, the size range is 3-8 letters. Regarding the entropy, I will have an employee more versed in the password entropy add to this topic to answer that question.

    I have submitted your request for separators to our developers. We will likely not add too many options because it can be confusing, but we're not opposed to reiterating, especially considering your use cases for those options.

    Thanks for your feedback! Requests like these help us make 1Password even better!

    Cheers,
    Kevin

  • julie-tx
    julie-tx
    1Password Alumni

    Hi, @dfz -

    It's always great getting this sort of question.

    I can't make any promises about the features you've requested, but I'm a big fan of camelCase and I'd love to have 1PW generate word-based passwords WithThatCapitalizationScheme. My personal master password uses it, just sayin'. There are problems with the "no delimiter" option if some means of distinguishing words isn't present because combining two short words could wind up making one valid longer word, which would include words in other dictionaries.

    As @ag_kevin explained, our word list is somewhere between 18,000 and 18,500 words. All words are chosen with equal randomness - we're not trying to make poetry or grammatically correct sentences, or even sentences that might make sense. Of course, that doesn't mean you can't come up with some clever mnemonic device, but we're not doing anything to make that easier -- random is random.

    With no fewer than 18,000 words in our word list -- and you should always select the lower bound so you are getting a conservative estimate of strength -- each word contributes about 14.1 bits of entropy. Assuming you created "BreakSettlerButtonDestination" using 1Password and our word list, that would be about 56.4 bits of entropy.

    Now, about that "none" option. Because short words can be recombined to make longer words, there would still have to be some form of delimiter. There's nothing keeping you from editing the passwords we create and adding your own personal twist to make it easier for you to remember, but you have to protect the entropy we've given you. I try to make word-based passwords into their own little sentences. For example, when I write about the XKCD comic, I like to write that as "Correct! Horse: Battery Staple" because I love adding punctuation and capitalization.

    I hope I've answered your questions. If not, please feel free to write back. And if I did answer your questions, please make up more questions about passwords because they are my favorite subject to write about.

  • dfz
    dfz
    Community Member

    Thanks, Kevin.
    Based on that, my computations show:

    An item entropy of 14.1 - 14.2 bits based on your 18,000-18,500 items.
    So, a 4 word password would have a total entropy of 56-57 bits, not bad for a casual site
    and a 5 word password would have a total entropy of 70-71 bits, pretty good for important stuff.

    At 1 trillion hashes/sec, which is what Edward Snowdon recommended as a possibility for the most powerful cracking systems (i.e. NSA grade), it would take about 30 years, on average, to brute-force crack such a password.

    That compares to regular dice ware with 65 bits of entropy for a 5 word password, crackable in ½ year, so good work folks and thanks for the enhancement.

    Adding this capability is one of the best things you could have done to improve security for everyone.

  • dfz
    dfz
    Community Member
    edited January 2016

    I've been using diceware style passwords for a while, considering it the best scheme. However, there are 2 hitches in this:

    1. The sites that REQUIRE upper/lower case, digits, and sometimes, special characters. For those, we have to manually add prefixes to the 1Password auto-generated password. Perhaps your developers could consider providing some assistance there to optionally auto-insert a fixed prefix string like "1A." for those situations. It doesn't matter what it is or if it's the same everywhere. We're depending on the diceware string length for our security, not character type variation.

    2. The sites that limit the size of passwords to 12-15 characters, thwarting the use of secure diceware based passwords. Not much to be done here except write to the operators and encourage change. I did that at my bank and they said their service provider was in the process of expanding passwords to 20 characters. Yea! So, I think folks out there may be starting to get on board that length (and randomness) is the most important attribute for password security.

    1Password 6 is definitely a major leap forward. Thanks for the great work, everyone.

  • dfz
    dfz
    Community Member

    Thanks for the additional info, Julie.

    I can understand your hesitancy regarding the "none" delimiter. The number of occasions when that would occur should be extremely rare, but I'll leave that to your judgement. My priority would be to then add the semicolon, as it's the easiest delimiter for those cases where it has to be manually typed. My vote would be to replace the underscore if a choice had to be made to keep the list short. Thanks for the consideration.

  • jpgoldberg
    jpgoldberg
    1Password Alumni

    Thank you @dfz!

    You are absolutely correct that diceware-like passwords often do not meet various website password requirements. This poses a difficult, but all too typical, problem for us or any developer. We can either

    1. Add a boat load of options hoping that some combination of them will cover most of needs
    2. Have few options, and accept that the automatic output won't always do what people need it to do

    There is no absolute right or wrong for which approach is better. A lot of it is sort of judging "bang for the buck" kinds of things. We ask ourselves:

    Is there something that we can do that still keeps things simple for the user while helping out a lot of them?

    When we build something like a password generator, the underlying code may allow for lots of advanced features and options that end up never being used or presented to the user. But don't take that negatively. This also means that often adding a desired feature doesn't require large changes to the underlying computation, but focuses only on how the feature is to be presented within the app.

    So, I make no promises whatsoever, but it is possible that some of the things that you have suggested (or things like those) may show up in the future.

  • dfz
    dfz
    Community Member
    edited January 2016

    I completely understand, jpgoldberg. It's nice to have lots of options, but at some point you start making things too complex to understand and use. There's a lot to be said for simplicity and limited choice, both for ease of use and, more importantly, software reliability and integrity. The highest priority for you folks is NO BUGS! with passwords. I'll leave it to you to make the best judgement on that.

  • Aleen
    Aleen
    1Password Alumni

    Finding the right blend of security and convenience is a tough job and we constantly evaluate and reevaluate our choices! :chuffed: Thanks so much for all of your great questions and feedback, @dfz; they definitely help make us think!

  • braindump
    braindump
    Community Member

    Great thread here. I do have a follow-on question....

    As I understand it, the level of entropy is dependent upon the selection of words being truly random. I've read through as many of the forum posts related to this feature I could find, but haven't seen a discussion about how 1Password chooses the set of real words for a passphrase, and whether that mechanism is truly random.

    Is there any information that can be shared regarding how 1Password's choice of real words for a passphrase is actually random?

    Thanks!

  • Hi @braindump,

    That's an excellent question. 1Password randomness is based The operating system's SecRandomCopyBytes call. And we do not employ modulo bias to generate random numbers within a range. More detailed information is available in this post:

    https://discussions.agilebits.com/discussion/comment/122096/#Comment_122096

    I don't know if that's the level of detail you were looking for. If you have further questions, please reply.

    Cheers,
    Kevin

  • dfz
    dfz
    Community Member

    A useful argument for the "no delimiter" option:

    Sometimes, another person may get a glance at your password. A string of 4-5 words would be very easy to remember even with just a 1 second peek. If there were no delimiters, it would look, at first glance, like a long string of letters. It would take some time and effort to parse the string into words and remember them, by which time your view would hopefully be gone.

  • That's a good point, @dfz.

    Rick

  • My1
    My1
    Community Member

    Adding to the topic,is the wordlist stored somewhere for us to look at, just curious?

  • AGAlumB
    AGAlumB
    1Password Alumni

    @My1: Yep! You can find a lot more details in @jpgoldberg 's overview here:

    1Password Wordlist

    But I'm glad you brought it up, because the location has changed slightly in 1Password 6:

    1Password 6.app/Contents/Frameworks/AgileLibrary.framework/Versions/A/Resources/AgileWords.txt

    Or, if you're using the Mac App Store version, the app will just be called 1Password.app. I hope this helps. Be sure to let me know if you have any other questions! :)

  • My1
    My1
    Community Member
    edited October 2016

    oh, i completely overlooked that we are on the mac forum. where is this thing in Windows?

    in the meantime I got a better idea an tried and actually got the mac app DL'ed and got myself the list, but it seems that there are quite a few 1 or 2 letter words in there (like "i" or "or" n++ counts 51, regex search \n..?\n )
    maybe they were overlooked, no idea.
    judging the line counter (and ignoring the final blank line) we have 18.436 (14,170 bits) passwords in total subtracting the 51 from that we still have 18.385 (still 14,166 bits per word, no big change...)

  • AGAlumB
    AGAlumB
    1Password Alumni
    edited October 2016

    @My1: Well, the new Wordlist started out in 1Password for Mac, so this is where the discussion originated. No worries. :)

    On Windows it gets a bit complicated. While the new 1Password 6 Windows desktop app uses the AgileBits Wordlist, there isn't a notion of "bundles" there, so it's embedded in the executable. And 1Password for Windows version 4 actually uses the standard Diceware list, not the new Wordlist.

    You make an interesting point about short words. We'll certainly continue to make modifications to the Wordlist as needed, but it's important to keep in mind that entropy is what determines the strength of a password. It's counterintuitive, but having a short word show up in a word-based generated password is equal to a long word since it has the same chance of appearing and is therefore equally difficult to guess. After all, word-based passwords aren't built for length. A character-based password of the same length will always be stronger. We're looking for the odds that any given combination of words will show up, as illustrated by your calculations. Fun stuff! :sunglasses:

  • My1
    My1
    Community Member

    Well i was talking 1pw6 anyway so no problems. I don't have problems with small words i thought that they were left in there in error coz ag_kevin explicitly states 3-8 letters per word.

    I am aware that word based pws ain't in for the length (at least not for the length of the individual words, but rather the length of the wordlist :-P )

    Well for the probability of any given words(s) to appear the calculations are fairly easy (at least when multi-usage of a word is allowed which does better in the entropy anyway) it's w*c/l with w being the number of words we are caring about (in the case of 1 or 2 letter words 51) c is the count of words in the password (et's take the standard 4) and l is the length of the wordlist (18.436) making the probability 204/18.436 or about 1,1 percent according to my quick phone calculations. In the case of not allowing multiple it's a bit higher but not as easy to calculate, especially on the phone where you just have basics.

  • AGAlumB
    AGAlumB
    1Password Alumni

    Well i was talking 1pw6 anyway so no problems. I don't have problems with small words i thought that they were left in there in error coz ag_kevin explicitly states 3-8 letters per word.

    @My1: Ah, sorry for the misunderstanding. Originally we had some longer words too, but we've made some changes to the Wordlist over time to try to improve it where we can. It's important that we don't use super-obscure words if we can help it. There simply aren't a lot of 1-2 letter words, relatively speaking, and frankly only serious Scrabble players know most of them. ;)

    I am aware that word based pws ain't in for the length (at least not for the length of the individual words, but rather the length of the wordlist :-P )

    Fair enough. I wasn't trying to condescend. I just didn't want total that for granted, and I thought it best to be explicit for anyone else who might find this discussion. :)

    Well for the probability of any given words(s) to appear the calculations are fairly easy (at least when multi-usage of a word is allowed which does better in the entropy anyway) it's w*c/l with w being the number of words we are caring about (in the case of 1 or 2 letter words 51) c is the count of words in the password (et's take the standard 4) and l is the length of the wordlist (18.436) making the probability 204/18.436 or about 1,1 percent according to my quick phone calculations. In the case of not allowing multiple it's a bit higher but not as easy to calculate, especially on the phone where you just have basics.

    Indeed. And while entering a word-based password is often easier on a mobile device than something character-based, it can definitely still be a chore if it's a long one. Cheers! :)

  • My1
    My1
    Community Member

    "It's important that we don't use super-obscure words if we can help it. "
    of course, coz them we get into the remembering aspect yet again which is the whole point os using wordlist in the first place.

    "I just didn't want total that for granted, and I thought it best to be explicit for anyone else who might find this discussion."
    true, completely forgot about that Idea (especially since I crashed right into here myself)

    "it can definitely still be a chore if it's a long one"
    not wrong, especially since in password inputs any good keyboard kicks autocomplete right into oblivion (for an obvious reason)

    also are there any usage restrictions or is it allowed to use the (honestly pretty intresting) wordlist for other purposes than in 1pw, especially than in 1pw mac it's literally lying around in the open?

  • AGAlumB
    AGAlumB
    1Password Alumni

    Indeed. We haven't officially "open sourced" the Wordlist or anything (though perhaps we will one day), but I wouldn't want to discourage you from having some fun with it. As ingenious a concept as it is, my favourite thing about Diceware has always been the "dice" part. ;)

  • My1
    My1
    Community Member
    edited October 2016

    because my personal Idea would have been for example that a website with 2fa that instead of generating a recovery "password" that's awful to type is that the user gets a bunch of lists and can select the needed words from there (okay the idea would be awful on phones without some nice javascript extras but even without js 10 (or more, remembering isnt the problem here and it is an emergency recovery measure so you wont type it every day) words are easier to type compared to something like the 1pw account key (especially if there isnt a numbers row on your keyboard (I hate that on the PS Vita) not forgetting that the symbols a user can type on his/her keyboard (and the difficulty of typing those) vary from one input language to another.

    of course the real diceware is a very nice concept (especially with the out of band RNG) although you would need another dice/roll but are way above what's needed with an 18,5k list but 46,7k combinations for 6 dice/rolls

  • AGAlumB
    AGAlumB
    1Password Alumni

    Ah, interesting. :)

This discussion has been closed.