Using 1Password with git

[Deleted User]

Somewhat a specific request, but I'm wondering if anyone has integrated 1Password CLI with git on the command line?

Git comes with support for credential.helpers and includes support for the macOS keychain by default. I looked through the example in the git documentation as well as the 1Password CLI docs, and it seems it should be possible.

On a side note, in the two 1Password help pages on the CLI (Getting Started and Full Docs), neither mention that the feature is only available on Team or Business accounts. Only came to suspect that when the Getting Started page mentioned a "Sign-in Address" and I started searching for what that is. Might be a nice update to the docs to clarify.

Are there any plans for the CLI being available for personal accounts?

---

1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Sync Type: Not Provided

cohix

@Angusc Thanks for writing in!

We would love to see someone integrate op with git, I think that would be a fantastic use case :)

The CLI is indeed available for any 1Password membership account, including individual and families accounts. For those, just use the my.1password.com sign-in address :)

angusl

Ok, playing around with this, will def have some feedback for the thread about shaping future of the CLI.

One other question I can't find in the docs is what exactly get item is searching on. Specifically I'm trying to search on domain but not having any luck. Searching "domain.com" doesn't match, which seems strange.

Is it an exact match search on any website field? That's not what I would assume from the term "domain".

angusl

Well that turned into a interesting days diversion. You can probably guess it's cold outside here.

Python Script
I have a working python script integrated with the git credentials.helper. It works, but there's one big gotcha. For some reason when OP requests a password to signin when launched via my script launched via git, OP gives an error before any input is even received:

[LOG] 2019/01/22 06:46:56 (ERROR) inappropriate ioctl for device

The strange thing is if I run the plython script myself and it launches OP, there's no problem entering the password to get a new token. Anyone at agile shed some light on what OP is seeing when it kicks out that error? It's clearly from OP, as the format matches exactly with all the other errors like bad passes, expired tokens, etc.

The script is available for anyone's use at git-credential-1password.

tl;dr

Unless you want to get into all the details, skip the rest.

To work around that problem I modified the script to include a flag to require saved tokens, and it works with git pretty seamlessly, as long as you refresh your token before (or after the script reminds you it's invalid).

Why python? I actually don't know python well (and now have a distinct dislike for languages that are so touchy about whitespace), but it was one of two contenders since macOS ships with a JSON parser for python by default, and I wanted to create something that didn't require additional installs if possible (Marie Kondo for computers!)

I mentioned two contenders, and I actually started with the unmentioned one first before running into a roadblock there first. But since I've now worked around that problem in the same way as in Python, you get two for the price of one.

Swift Script

You might not have known this, but you can run swift code like a perl or python script from the command line without compiling a binary. Yet you still have full access to all the macOS API calls, which is fun. Simply include a shebang at the top of the file and make the swift file executable like any other scripting language.

I started down the Swift path first because it has an amazing JSON parsing functionality which I wanted to make use of. (Side note for the technically inclined, it took 9 lines of code to parse the JSON response, and 7 of those were simply defining structs. One of the best JSON parsing experiences.)

Everything was going ok until I got to calling OP and it wanted to prompt for a password. For some unknown reason when launched through swift Process APIs, it wants to echo the password as it's types...which I didn't want to happen. I dug around and found how to make a call to get input in "secret" mode like typically happens in the terminal, but since it's actually OP giving the prompt, I'm not sure how much control my script has over the interactions.

The Swift script is working interactively, but I haven't tried integrating it directly with git as it's still very rough compared to the python one. But it's available in the same Github respository.

In the end both scripts got hung up in roughly the same place, but for two totally different reasons. Having worked through these I have some input on what might make the CLI more useful, but I'll add that to the pinned thread looking for feedback.

MrC

My guess... op is trying to set raw mode for the controlling terminal to blank the password, but your program is on a pipe, so op's STDIN is not connected to the terminal - hence the inappropriate ioctl() call.

angusl

My script is using a pipe, but that wouldn't explain why it works when my script launches op, but not when git launches my script which launches op.

MrC

Does the git command close the controlling terminal, or use a PTY?

Inappropriate ioctl's on a terminal usually mean that the terminal cannot be placed into modes such as RAW, or cannot seek (because of a pipe).

angusl

Git doesn't control the terminal, and doubt it's using a PTY, it's pretty basic. The git command (which is initiated by the user in the terminal) launches the script I wrote, and my script is calling OP. The git command relies on stdout and stdin to first send some data to my script, and then receive the final info back from my script, so I would suspect it's using pipes, though I didn't look through it's code in detail.

My first uneducated suspicion was that op was checking some condition of the terminal to make sure that something wasn't trying to steal the master pass as it was being entered, since git is waiting for some output when op prompts for a password. But I would think git's stdin is piped to my script's stdout, which is where is expects the final data to come from.

Looking at the git code, here's what git is doing:

• It launches my script in function run_credential_helper() using start_command()
• start_command() uses a standard fork()
• it then writes some data with a fdopen() call, quickly closing that stdout handle
• then starts waiting for data on stdin using another fdopen() in run_credential_helper() and uses a while loop in credential_read()

So that's the state it's in when that error is popping up

I'm not actually using a pipe for the stdin of op subprocess, in fact I wasn't doing anything with stdin since I didn't think I needed to. I was setting up a pipe for op's stdout to get the results of looking up an item. I did experiment with a few settings for stdin, but it didn't change anything.

As for the stdin of my script, there is a readline loop on stdin earlier to get the first info from git, but it's over long before calling op, and I didn't create a pipe, just used Python's sys.stdin.readline(), which is likely to be using a pipe under the surface.

cohix

@angusl This is an interesting one. Seems like this is a (somewhat) common edge case that's been seen before, for example: https://github.com/golang/go/issues/19909

I'll go through and see if I can tweak our handling of stdin to hit this case.

Thanks for going into such detail, and I'm very excited to review your Git integration (especially the Swift version!)

angusl

@cohix Yea, but it gets a bit confusing with the three parts. My script (talking python here) isn't doing anything with stdin, and I have to assume the python Popen() call is using the typical BSD popen() under the hood and isn't doing anything on it's own, but that's a guess. git is listening on stdin for the user/pass from the helper, but that's not directly connected to OP or it's in/outs.

Since you're interested in the swift implementation, here's some details I didn't put in the GitHub repo. It's broken in exactly the same place as python, but very different way (see details below). Works if session token in already in your ENV though. I simply set up an alias to refresh the OP session token and use that when needed.

When the swift script calls OP and OP prompts for the master password, input is being echoed. I thought it just wasn't set properly to not echo but would still work (I never tried). I finally tried to send some random text expecting it to fail, when I discovered nothing is happening. Input echos into terminal, but it's seems to not be passed to OP. Basically it's just hung there, and have to control-c to kill it. I played with all the different possibilities with pipes to stdin when launching OP in swift (using Foundation Process() but could never get OP to respond to input on password prompt when launched via swift (either compiled or not).

One other side note. I ran into one really interesting problem when using swift. I kept getting weird errors trying to launch OP from the swift script after it had previously worked. Turns out I had always run it from the command line previously, and launching from inside Xcode or a playground triggered macOS Gatekeeper on OP. Launching OP from the command line or from python worked fine, but after Xcode had that problem, I couldn't even launch OP from the uncompiled swift script again. Launching OP through the finder via "Open..." or manually removing the quarantine extended attributes fixed the problem, but took me a while to track that down, errors weren't helpful. Not sure why I didn't add that to the readme, I'll fix that in the instructions.

cohix

@angusl One potential workaround would be to collect the MP using non-echoing input in your Swift script, then pass that value into stdin on op (something like echo {masterpass} | op signin {shorthand} --output=raw, which would give you the same value that's saved into env. You could then pass that session token into any other commands with the global --session flag. I think this would circumvent your issue (and you could generalize it to either pull the token from env or create a new token and just use it as a local var for that invocation)

I'll poke around some more to see if we're doing anything weird with the input there.

angusl

@cohix I considered that. While I didn't try it, figured it might work. But it's nice if my script (or any script) never has to handle the MP, leaving it only to OP. I don't want the power!

It also raises the issue of then not being able to save the token for the next git command, as all those processes run in their own environment. That was one of the main reasons is why I didn't bother trying to echo the MP out via my script. It actually makes the UX worse having to type the MP every time without the ability to save the session token for reuse.

Storing the session token for use across multiple envs is not straightforward, but it's possible if you don't use ENV. My script (or any OP script) could write the current session token out to a file, say ~.op_session. Then any script could check there for a valid token and use it if needed. Would be best if OP checked for a token in the same place by default, and perhaps even included a flag to save it there automatically. This runs into potential security issues with access to that file, since an ENV var is by nature more transient than a file on disk. But really not other way to share something between envs whether they be forked executions or second terminal windows.

And as long as I'm dreaming, would also be helpful if there was an OP command to see if a token was valid without trying to use it in a query. Can Make the logic flow easier if you can do it that way, but that can be worked around.

cohix

@angusl you're very right to never want your code to have the master password, I love to hear that. Until we figure out what's going on here though, it may be the best (temporary) solution

As for the session file, we specifically did not do this for the exact reason you mentioned, we don't want the session information and the encryption key for that information in essentially the same place, hence the use of env vars.