Deep-technical: Trying to understand, I mean, really understand, client unlocking
(This is probably a question for @jpgoldberg, as he's answered similarly detailed questions in the past....)
Hello!
I've been trying to assess the risks presented by 1Password on a badly compromised system. In particular, I'm trying to determine just what can be done if someone's installed a keylogger that was able to get the user's Master Password.
Unfortunately, the Security Design Whitepaper still has a lot of holes in it, and the deeper I dig into that (and vaults on Windows and macOS), the more questions I have.
I'm very much in a "Trust, but Verify" mode -- I have (and will continue to) recommended 1Password without reservations, but when someone asks me for technical details, with very specific scenarios in mind...I like to be able see the validity of my responses with my own eyes...
I understand vaguely what the 1Password keylogging protections entail, but more clarity there would definitely be helpful.
More client-level detail would be INCREDIBLY helpful. I'd like to be able to directly demonstrate the key derivation steps, and vault decryption, to demonstrate that the system is behaving the way we expect.
A post from September 2016 (https://discussions.agilebits.com/discussion/69566/white-paper-clarifications) helped with this somewhat, but not completely. Like, for example, where is the MUK-salt (I can't remember exactly what you call it) stored? Where is the Secret Key stored? And how is the secret part of the secret key thrown into the derivation mix? (like, as an ascii string, or converted somehow to binary via some unusual number base conversion)? Like the participant in that conversation, I'd love to see some test vectors...
As a more specific example, on Windows, I see an "EncryptedMasterKey" in the "config" table (which looks like it may just be raw AES + HMAC). My suspicion is that the 2SKD process (salt, email, Secret Key, and Master Password) is used to decrypt this EMK (I assume this is how it works, but then it might simply be a million rounds of SHA-256 on the ascii text of the entered password....).
It looks like that EMK also decrypts the "enc_login" field in "accounts", which looks (from experimentation, since I can't yet decrypt it personally) like it must contain the Secret Key and the text of the Master Password. This is logically required so that the app can directly authenticate to the server for multiple team / family accounts.
I've assumed that this EMK also directly decrypts the "mp" in keysets, which then decrypts the private keys, and from that the vaults, etc... Though it may be possible that the EMK only decrypts enc_login, and then the key derivation process is run again for every account using the data pulled out of enc_login. (in fact, experimentation on Windows shows that this may be the case -- breaking the emc_login string by mucking with the raw B64 caused the initial app login to work, but then it couldn't read any items, and I had to sign-in again.) (and it didn't pre-fill the secret key field either).
Through further experimentation on Windows, I'm also wondering where the key derivation process gets the user's email address from -- because even when I changed it in the accounts table, authentication still worked.
Somewhere, there must be a structure that includes Salt, Secret Key, and Email, as used by the client to unlock the whole system (it seems to reset that to whatever the current "first" account is, so if I set up with team A, add team B, then later delete team A, it'll seamlessly shift to using Team B's password to unlock the app). The security of the entire system (assuming a keylogged password) depends on how much of a pain it is to retrieve that data.
I have very similar questions for macOS. The enc_login field looks very different in structure (but probably identical in usage), but I don't see the EncryptedMasterKey field anywhere.
In a comment here (https://discussions.agilebits.com/discussion/comment/385433#Comment_385433) JP says that "the 'local' data store Key Derivation Function hasn't been updated as it should have been." Or is it possible that the EncryptedMasterKey" on windows isn't actually dependent upon the 2SKD process? Has that since been fixed (or is the fix coming in version 7)?
Obviously, a lot of this may change with version 7. (and I really love that you're using the Secure Enclave private key service, too. Is there any reason you couldn't use that to encrypt the local copy of the Secret Key too?)
Ultimately, everything boils down to this:
How strong are the keylogging protections on Windows and macOS?
Assuming an attacker bypasses those protections, how easy is it to extract Secret Key, Email, and Salt (the other items required for the key derivation)? (for the moment, I'm ignoring whether the user has logged in via the web...the Secret Key is trivial to pull from browser Local Storage).
Assuming they get all that, how much of a pain in the neck is it for the attacker to then begin decrypting things? (I suspect "not easy" but also suspect that a determined attacker could work it out in a few days) (if a tool isn't already on GitHub).
We're currently working on adding 1Password MFA support internally, so that a compromise of Secret Key and Master Password can't simply be rolled into an attacker logging in through his own web browser. That's a huge step forward, but the "being able to 'simply' decrypt the local vault" is still a big question mark for us, risk-wise. Having the admin immediately mark the user as "on travel" (to remove the vaults) is an additional good step, provided the compromised system is still attached to the network and the attacker hasn't already copied the data.
I greatly appreciate everything AgileBits does to make this product as open and as transparent as possible, but I really wish you could clarify bits of the White Paper (and fill in the many gaps which remain). I'm struggling to explain the whole system at a deep, "here's exactly how it works" level. But every time I think I've got a good diagram of the system (and it's multiple pages already), I run across a forum post or database entry that throws my understanding into doubt.
Thanks!
(has anyone done a talk on exactly how all this works? I've given a similar talk for iOS encryption...and am really thinking this'd be a fun thing to do...but then I'm kinda weird.)
1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Sync Type: Not Provided
Comments
-
Hi @darthnull,
I can answer some of these questions for you while we wait for a more complete answer from @jpgoldberg. He's out at a conference this week so he'll be slower to answer than normal. There's a lot of questions in your post and before we dive into the deep stuff I'd like to take a detour to the higher level.
I'll actually need to split this up into multiple comments because as one comment it was too large and I couldn't submit.
0 -
Compromised Systems
The overarching question you have is about using 1Password on a compromised system. You should avoid doing that. You should also avoid using email, or doing much of anything on a compromised system. If you look at macOS, it does a fair amount to help you not get into that position. It has GateKeeper that lets you make sure that you're only running software that's been codesigned by "trusted" developers. It has SIP to limit the damage that can be done in more dire scenarios. We encourage everyone to keep up to date with OS updates as that's the best way to stay protected. I'm sure you know that, and I'm preaching to the choir, but the best way to not have to deal with an ugly scenario is by not getting into an ugly scenario to begin with.
That's not the answer you came here for though. You specifically want to hear about ugly scenarios, so let's talk ugly.
Not all compromises are created equal, and it does us no good to treat them all equal. Let's peel back the layers of the onion and see where we start shedding tears. I'm going to focus on macOS cause that's where my expertise lies.
The vast majority of malware on systems comes in the form of an app that's running some extra curricular activities. This is the best way for an attacker to get onto a system: find a way to infect an app that a large number of users are using. Maybe it's a video player that was downloaded from the internet. In general GateKeeper would protect against this, but for sake of argument let's say we disabled GateKeerper. The video app still plays video, but it also does more nefarious things. In general this level of malware is going to be limited to what it can do. Let's look at what those things might be.
Reading data on disk. This is probably the most common thing I've seen them do. If they know where interesting data lies, they can read it. In the case of 1Password, your data on disk isn't all that interesting. It's encrypted with a truly random key, and getting to that random key requires the key derivation that you mentioned. We've gone out of our way to make an attack of that data very difficult by using 100,000 iterations of a slow password hashing function (PBKDF2). There's no shortcuts, each try has to go through all 100,000 rounds. When you limit an attacker to doing only a few tries per second, they just won't get anywhere fast.
Key logging. Any user-level app can request to get keystrokes from the system. This is how fantastic tools like TextExpander work, it detects that I've typed a series of characters like "agminidr" and automatically expands that to be a snippet with instructions for how to send us a diagnostics report. All this to say that there are perfectly legitimate reasons for why an app would want to be able to read every keystroke on the system. It can absolutely be used for bad though. An app could read every keystroke I type as I email my doctor about my latest health problem. That's pretty scary. Apple recognizes that sometimes we need a way to be able to hide keystrokes from that global event tap and has provided us with something called Secure Input. When Secure Input is enabled, keystrokes are treated differently and they bypass the event tap. They're delivered directly to the app/window with focus. If you're a TextExpander user, you'll notice that their system bar icon changes dims whenever you're in a password field anywhere on the system. TextExpander knows when Secure Input has been enabled, and it knows that expansions won't be possible while that's the case. 1Password for Mac makes extensive use of Secure Input. Obviously we use it for the Master Password field, but less obviously we actually use it for all text entry fields when editing an item, even if it's a normal text field like maybe a name field on an Identity item. We decided that it was best to treat all of the data you put into 1Password as sensitive.
Reading data on disk and keylogging are the most common things malware tend to do, and these user-level malware apps are by far the most common type. In general 1Password's handling of it is reasonable (if there's anything more we could do, we'd love to hear about it as we're always looking for more ways to secure the app). That still doesn't mean that I'd want to be using 1Password or anything on such a system. Let's go deeper though.
There's another level of compromised systems, those that they tend to refer to as "rooted". Being "rooted" means that the malware is no longer running as a user level process but instead gets to run as the super-user on the system. On the Mac this is relatively rare, but it does exist. Malware running as root can do more interesting things.
Key logging. User-level key loggers are limited to seeing the keystrokes in that global event tap and if the system decides to bypass that then they don't see the keys. A root level keylogger could read keystrokes at a lower level, so the protections of Secure Input no longer matter. This means that every keystroke can be read, and so yes the Master Password could be read as it's typed by a root level keylogger. That's pretty damn scary, but ultimately not as useful a tool as it may seem because of the next thing.
Reading arbitrary memory. Why go through the pain of reading every keystroke and trying to figure out what's what when you can just read the whole memory of a system and get the data that way. Eventually to be processed by the CPU data needs to be loaded into memory. Being able to read arbitrary memory means that there's effective nowhere to hide anything. This is why attacks like Specter and Meltdown were so bad (though those attacks did not require root, which made them far far worse).
A rooted system is security game-over. There are no protections that anyone can do that will cause software to be safe when running on a rooted system. It's not a matter trying harder, the malware will always have the inside track if it can read the whole memory. To go back to my terrible onion analogy... this is where we cry.
0 -
Key Derivation / Unlock Process
Time for a palette cleanser. Let's have ourselves some fun and talk tech.
I love this part because I'm allowed and encouraged to explain any part of 1Password. We have no secrets.
When 1Password is in a locked state, what's required to unlock it? Many things, but what's the one thing that the user needs to provide? The Master Password. This means that the rest needs to be available to 1Password even when locked. There are no derived encryption keys yet to play with, and so there's nothing to truly protect the rest of those parameters. If you open up the sqlite database (B5.sqlite in the case of 1Password for Mac) you'll see most of them sitting in the clear.
You might reasonably ask "isn't there some way to protect those things?" And the answer is a combination of "sort of but not really" and "yes, on some systems". For example we could store them in the keychain. Unfortunately the keychain on the Mac is... not very nice to users. It's too easy to put data into the keychain lose access to it because the system will prompt the user to ask them if the app (1Password) should have access to the data that it itself put into the keychain and the user can say no. Once 1Password loses access to it, the wheels fall off the cart. There's a new and improved keychain for use on the Mac and we use it for some things, but it comes with its own set of downsides which makes it not a great fit for this. On Macs with the Touch Bar, there's a Secure Enclave, and this would be a real solution. Unfortunately very few Macs have the Touch Bar and so it wouldn't protect many people.
The unlock chain of events looks something like this:
- Read the encrypted active keyset of the master account (first one in the db). It's the one marked as being encrypted_by "mp".
- Get salt, iterations from the encrypted keyset
- Read Secret Key, email, from the database for the master account
- Get Master Password from user
- Use all of those to derive the Master Unlock Key
- Attempt to decrypt the encrypted active keyset with the MUK. If it fails, then the master password is incorrect. Let's assume it decrypts and the password was right.
- Use the decrypted active keyset to decrypt all other keysets associated with this account (for every group that you're a part of there will be an additional keyset)
- Use the Master Unlock Key to decrypt the data that's in the master account's enc_login. Inside of enc_login we store two things: the account's master unlock key (not particularly useful in the case of the master account as we derived it) and SRP x for the account.
At this point we've fully decrypted/unlocked the master user account, and because we have SRP x we can even securely communicate with our server and create a new encrypted session there. You might think we're done, but we're just getting started.
- Using the master account's MUK, we can decrypt the other accounts' enc_login field. This gives us the other accounts' MUK and SRP x
- Using the other accounts' MUK we can decrypt their encrypted active keyset (again, the one marked encrypted_by "mp")
- Using the decrypted active keyset we can decrypt their other keysets
Now we have all account MUKs and keysets and SRP x: the app is unlocked and can communicate with the servers for any accounts. You still don't have any data though.
- Fetch the vaults from the database, each one has an encrypted vault key marked as having been encrypted with one of the keysets
- Use the correct keyset to decrypt the vault key
- Use the vault key to decrypt the vault items
Finally we're able to play with real data.
I can understand that you might want to write an implementation of this to verify that this is truly what's going on. It's certainly doable and we can provide specifics for anything that'd be needed, but let me be blunt: it's a lot of damn work. The last implementation of this we did was the command-line tool and it took over a week of full time development by someone who was very familiar with all of these concepts.
I would love to see us open source the command line tool so that anyone can inspect the process. As you say, Trust but Verify. Right now the best way to verify would be to look at the code of the webapp (the javascript is pretty readable) or to decompile our Mac app. A tool like hopper disassembler would probably make rather readable code out of it and there's only a handful of classes involved.
I hope this helps. I've likely not answered all of your questions, but I hope I've answered some.
Cheers.
Rick
0 -
Right now the best way to verify would be to look at the code of the webapp (the javascript is pretty readable) or to decompile our Mac app.
/me smacks forehead
I got lost looking at the Mac app, and at op in hopper (side note -- for some reason, op is totally hanging my work Mac at the moment. I'll dig more and open a new conversation/ticket if necessary...it's probably just me). I know enough about hopper and IDA to be dangerous, but not enough to easily make progress there. COMPLETELY forgot about the web app, even though you'd suggested it directly.
[ ... four hours later ...]
Success. Decrypted enc_sym_key and enc_login for macOS, and verified the computed SRP-x verifier against the contents of enc_login. (verified the verifier...heh).
I'm out of time for the day, but will return tomorrow and review where I stand, see what questions are remaining. I remember that yesterday, it all made sense and seemed helpful (but then I obviously forgot useful specifics today...) Thanks for the help so far!! This is definitely helping me understand everything much better.
david.
(and sorry about the silly forum handle -- I'd assumed that there'd be a "real name" component that I could fill out with my real name...)
0 -
/me smacks forehead
I got lost looking at the Mac app, and at op in hopper (side note -- for some reason, op is totally hanging my work Mac at the moment. I'll dig more and open a new conversation/ticket if necessary...it's probably just me). I know enough about hopper and IDA to be dangerous, but not enough to easily make progress there. COMPLETELY forgot about the web app, even though you'd suggested it directly.@darthnull: Hey, without doing stuff like that, none of us would have the benefit of "eureka!" moments, which are super enjoyable. I'm glad that SuperRick was able to point you in the right direction. :)
[ ... four hours later ...]
Success. Decrypted enc_sym_key and enc_login for macOS, and verified the computed SRP-x verifier against the contents of enc_login. (verified the verifier...heh).:chuffed: :+1:
I'm out of time for the day, but will return tomorrow and review where I stand, see what questions are remaining. I remember that yesterday, it all made sense and seemed helpful (but then I obviously forgot useful specifics today...) Thanks for the help so far!! This is definitely helping me understand everything much better.
Yesterday... I remembered the specifics I forgot today...
Sorry, couldn't resist. Anyway, sounds good. We're here if you have any other questions, comments, or feedback. :)
david.
(and sorry about the silly forum handle -- I'd assumed that there'd be a "real name" component that I could fill out with my real name...)No worries! Doesn't matter to us. We're just glad you're here. But I'll shoot you a private message here in a minute in case you want to change it. However, I'm pretty sure "david" is already taken. Cheers! ;)
0 -
Okay, I'm very comfortable with the macOS side of things now. :) (though I'm still having issues with the op command line tool). Of course, I was pretty comfortable with it before but at least now I'm able to see through all the data for myself (even down to the item level -- once I found a way to convert the JWK-formatted RSA keys into something PyCryptoDomeX could handle...).
I'm still wondering about the Windows data. I see the EncryptedMasterKey field, but can't figure out how to decrypt it, and obviously, don't know what's inside. From length, it looks like just a pair of 256-bit blocks (a key and a HMAC check feels likely). I've tried to decrypt both that, and enc_login, using both the derived MUK for the account, and the mp that that MUK unlocks, but neither worked.
So my current guess is that the user's password derives some new key (not the same MUK as is derived on macOS or in the web client), which decrypts EncryptedMasterKey, which then decrypts enc_login. The data in enc_login then fully unlocks the account, as it either directly contains that account's MUK, as it does on macOS, or just the user's password and secret key. From experimentation it looks like it's got the password in plaintext (compared two different accounts with different password lengths, and the the difference in ciphertext length matches the difference in password lengths).
Anyway, I think that's the last stumbling block I have -- just how is the Windows EncryptedMasterKey and enc_login data encrypted, and, ultimately, where is the Secret Key stored. Because in the end, it all comes down to how easily an attacker can get the Secret Key and Master Password. All the other risk analysis hinges on that, and that's the one thing I just can't figure out. (doesn't help that I'm a Mac geek and fairly windows-phobic....)
So if anyone can chime in on these windows details, that'd be incredibly helpful. :).
And thanks to everyone for the help so far! The openness is huge (personally, and professionally, transparency has always been a big thing for me). And at a geeky level, just being able to directly appreciate the elegance of the overall vault format is incredibly satisfying. Almost as satisfying as the iOS key structure...(though 1P is a lot easier to play with).
david.
0 -
@darthnull I can help with Windows side of things. EncryptedMasterKey has your app master key encrypted. When user unlocks app, we use master password to generate unlocking key that is used to decrypt EncryptedMasterKey to get master key that is used to get into enc_login, protected settings, vaults, pretty much everywhere.
To your question "how easily an attacker can get the Secret Key and Master Password" - to my knowledge only by brute force when your app master password is weak. Your master password can not be 1234. Worth noting that master password used to unlock app never stored or submitted anywhere, there is only one storage allowed for it - user's head :)
1Password for Windows also measures time used to derive "unlocking key" and if your computer is too quick, app will re-encrypt master key immediately. That is to ensure that database created in 2015 is still hard to crack in 2025.
Hope that helps, let me know if you want more details.
0 -
@SergeyTheAgile Well, yes, I understand that at a theoretical level. I'm trying to be able to replicate it in practice, so I can observe firsthand exactly how the encryption is handled, etc.
Is the EncryptedMasterKey dependent upon only the user's Master Password? Or is the Secret Key a factor there too? if the number of iterations changes based on the speed of the machine, where is that parameter stored too? (it looks like there may be a binary header in front of the opdata01 block in the EncryptedMasterKey data...that seems like a logical place for that sort of information).
And how will all of this change with version 7? (which I should throw onto my VM and check out directly, too...)
0 -
@darthnull EncryptedMasterKey structure is 64-bit number of iterations + 32-bit salt length + salt + 32-bit encrypted key data + encrypted key data. Master password together with iterations and salt is used to derive PBKDF2 key to decrypt encrypted master key (opdata01). Secret key is not in game here yet, it's used later in communications between your device and 1password.com, but not for app unlock.
Iterations are calibrated to make derivation roughly 1 second long. It kicks in when derivation took less than 500ms or when it takes more than 5000ms.
Version 7 uses same approach, there is no change. The only technical difference that will happen is that instead of calling .NET's API for cryptography we are moving closer to native Windows API direct calls. .NET libraries are using the same API, but they are aging and replacement is not coming for quite some time, so instead of waiting for miracles we just going to call BCrypt set of Windows API functions ourselves.
0 -
Thanks, that helps a lot! (though I think you may have an error -- the structure on my box looks like it's a 32-bit iteration count).
Your comment about .NET makes me wonder exactly what hash PRF you're using -- I'm seeing some comments that indicate that the .NET library only supports PBKDF2.HMAC_SHA1, not SHA256 (using the Rfc2898DeriveBytes call?)
Also the EMK structure includes an additional 16 bytes at the end that don't make sense. In my case, I see:
- iteration count (4 bytes, not 8 as you said)
- salt length (4 bytes, equals 16 in my structure) (and probably most)
- salt (16 bytes)
- Payload length (0x90)
- payload:
- opdata01 header (8 bytes)
- plaintext length (8 bytes, == 0x40 here)
- IV (16 bytes)
- cipher text (64 bytes)
- HMAC-SHA256 signature (32 bytes) [should there be a key for this too? does it come out of PBKDF2 also? (like an additional 32 bytes of material...)]
- ???? (16 bytes)
Adding up the opdata counts, we get 8+8+16+64+32+16 = 144 bytes, or 0x90, as expected.
Once this structure is decrypted (AES-256-CBC, with IV as provided), it should give me two 32-byte blocks. I'd assumed these would be the account's Master Unlock Key, but it's not matching the MUK I produce on the Mac. (and neither the MUK nor the mp the MUK decrypts from the keyset can decrypt enc_login). So I'm guessing that this isn't actually MUK, but is another key that decrypts enc_login (again, an opdata01 structure, so AES-256-CBC + IV). The decrypted enc_login should then allow me to extract (or generate) the actual account's MUK. I'm guessing.
In which case, I'm doing something wrong, because I can't decrypt enc_login with either of the 32-byte blocks I think I'm decrypting from EMK. So either my KDF is wrong, or my decryption of either (or both) opdata structures is wrong. Or both, I suppose.
On the one hand, this is kind of irrelevant, since you've at least confirmed to me that you're not using the Secret Key in the windows client unlock. On the other hand, I really want to see this for myself, just because it's crypto and crypto is fun.....(and because it's always nicer to see for certain...)
Again, thanks so much for helping on this. And to everyone at Agile for making the choice to be so open about how it all works. It's really a great way to build trust in the product!
0 -
Got it!
sigh Damned padding. Totally forgot that for an even block multiple, you prepend 16 bytes of random text in opdata01. So I was deriving the password-based key properly, but the key I pulled out of the EMK structure was half key, half random garbage.
For the record, it looks like it's using PBKDF-HMAC-SHA512.
And with that...I think I've got it all figured out. Woohoo. Now to write it all up so I don't forget.
Thanks again for your help! (and patience!!)
0 -
Haha likewise, this is fascinating stuff, so thanks for giving us the opportunity. After all, if no one asks these questions, does our openness make a sound? In the crypto...forest..? I don't know. :lol:
0 -
@darthnull oh, right, sorry, i forgot to mention that it's PBKDF2HMAC512, we are using native BCryptDeriveKeyPBKDF2() there.
0
