R (programming) and Shell "Handle is invalid"

jonlin
jonlin
Community Member
in CLI

I am trying to interact with the CLI using R (statistics oriented programming language). I've ran into an error message that i can't seem to look up.

When I run a shell command directly from "R Console" (different than interactive shell/command prompt), I don't get an opportunity to interact with the prompts. For example:

> shell('op signin <EMAIL>')
Enter the password for <EMAIL> at <SECRET KEY>: [ERROR] invalid account key format length 0.

One of my thoughts was trying to 'chain' commands. However, it seems to return a different error message:

> shell('op signin <EMAIL> <SECRET KEY> --raw && <PASSWORD>')
Enter the password for <EMAIL> at <SECRET KEY>: [ERROR] The handle is invalid.

The CLI error messages are different than the interactive shell and command prompt. Specifically, in the interactive shell, if my is blank, it just asks for it again. If the password is blank, I get a 401: Unauthorized. These errors are different than the R Console.

I've also posted this on the R forums to see if anyone has any R specific insight (https://community.rstudio.com/t/interacting-with-1password-cli/81657). I have confirmed that my CLI 1.7.0 is installed correctly and I can successfully perform actions via interactive shell and also command prompt.

Does anyone have any thoughts on whether or not trying to figure out chaining is the right approach, or if there may be something unexpected that is being passed to the CLI?

1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Windows
Sync Type: Not Provided
Referrer: forum-search:Handle is invalid

Comments

  • zcutlip
    zcutlip
    Community Member
    edited September 2020

    I suspect op is trying to read the password from standard in, but whatever the R runtime does to invoke a shell command, it's not providing a stdin file descriptor (the "handle" in the error message) that can be read from.

    If R's shell() actually executes the program in a bourne-compatible shell environment (which is a security vulnerability), you might try piping your password to op:

    shell('echo "<PASSWORD>" | op signin <EMAIL> <SECRET KEY> --raw')

    Alternatively, if there's an alternative to shell() that allows you to specify stdin explicitly, that might be better. I do something similar in python here:
    https://github.com/zcutlip/pyonepassword/blob/41bf4bb3e211f9178aa3bfbb84a2edb3e6021c35/pyonepassword/pyonepassword.py#L179

  • SvenS1P
    SvenS1P
    edited September 2020

    Indeed, @zcutlip is right here @jonlin; op by default reads from stdin. When you run a command from an interactive shell (like PowerShell or bash), stdin is normally associated with the keyboard. How exactly it does this is a bit beyond the scope of my answer, but when you pipe a command into another command, the output (stdout) of the first command will be transferred into stdin of the next command.

    Firstly, are you using RStudio by any chance? I experimented a little bit and it seems that the console in RStudio does not associate the keyboard with stdin for programs running with shell(). Actually, it appears no file descriptor is provided by default like @zcutlip mentioned. You can provide one in a temporary file by setting input=. So, something like the following should work:

    shell('op signin [<sign_in_address> [<email_address> [<secret_key>]]] --raw', input='<password>')

    Keep in mind that op would normally mask your password when you enter it, but this method won't mask it.

    However, if you start R from an interactive terminal session (for example bash, PowerShell, or even the Terminal tab in RStudio), it appears that it will link the keyboard to stdin. For example, on PowerShell I can start an R session by running C:\path\to\R.exe. I can then use the following to sign in to 1Password:

    shell('op signin [<sign_in_address> [<email_address>]] --raw')

    And I then get prompted for my secret key and password like normal:

    Enter the Secret Key for <email_address> at <sign_in_address>:
Enter the password for <email_address> at <sign_in_address>:

    I'd recommend trying the second option if you can. Let us know how you get on. :smile:

  • jonlin
    jonlin
    Community Member
    edited September 2020

    Reading both your responses was helpful - thanks @zcutlip and @Matthew_1P.

    I made some progress and found a different R function to send command prompts to. The below works reasonably well when there is only one input. It won't work for 2FA, or if I put the secret key as an input first, so my next step is trying to figure out how I can pass multiple input arguments into it.

    system2('op', args = c('signin', R_DOMAIN, R_EMAIL, R_SECRET_KEY, '--raw'), input = R_PASSWORD, stdout = TRUE)

    I had some follow-up questions:

    1. I am admittedly a bit novice in the shell space. Can you expand as to if R was using a bourne shell, how that could be a vulnerability? I would not want to code or design something that could introduce that! The echo suggestion did work, but if thats bad practice, I figure I should avoid it.

    2. Could you expand the response on "stdin file descriptor" a bit more (or point me to something I could learn from?) Even though the above system2() works reasonably well, I can't seem to pass multiple responses to it. For example, I am supposed to be able to pass a character vector and have each element passed as an input to command, but I instead get an error that again that seems to indicate its not passing the correct input:

    # Secret key and password as inputs


  system2('op', args = c('signin', R_DOMAIN, R_EMAIL, '--raw'), input = c(, R_SECRET_KEY, R_PASSWORD), stdout = TRUE)


[1] "Enter the Secret Key for EMAIL at DOMAIN: Enter the password for EMAIL at DOMAIN: [ERROR] inappropriate ioctl for device"

# Secret key specified as argument, 2 factor as input


  system2('op', args = c('signin', R_DOMAIN, R_EMAIL, R_SECRET_KEY, '--raw'), input = c(R_PASSWORD, R_TWOFACTOR), stdout = TRUE)


[1] "Enter your six-digit authentication code: [ERROR] incorrect One-Time Password length. expected 6"

    To expand (a bit beyond the scope of the question), the intention is to make a R package that can interact with 1Password op. So while I could spin up a terminal, the end objective is to use R console, and either ask the user to pass the password as a object, or enter it at runtime. Under normal R usage, I would store my passwords in a environment variable. I figured that if I could store the master password as the environment variable, then I could unlock the vault and look up my other passwords.

  • SvenS1P
    SvenS1P
    edited September 2020

    I am admittedly a bit novice in the shell space.

    @jonlin: Then I should confess that this is the first time I've used R, so I'll have to adopt some pseudocode for parts of my answer! :smile:

    Can you expand as to if R was using a bourne shell, how that could be a vulnerability? I would not want to code or design something that could introduce that! The echo suggestion did work, but if thats bad practice, I figure I should avoid it.

    The issue here is one of shell injection vulnerabilities. In short, if you handle user input of any kind, you need to treat it as untrustworthy. As an example, say you have a Login item called WiFi Password in a Shared vault. If you run the following pseudocode:

    item_title = getItemTitle() # Get the WiFi Item title
shell.run("op get item " + item_title)

    item_title will be set to WiFi Password, and the command op get item WiFi Password will be run in the shell. op will give you back your item details like normal.

    But if someone were to change that same Login item title to WiFi Password ; echo Shell Injection!, item_title will also be set to that. This means the shell now runs op get item WiFi Password ; echo Shell Injection!. The semicolon is part of the shell's language and tells it to run a second command after the first. Not only will we get our item details back from op, but we'll also see Shell Injection! printed to the terminal:

    {...JSON results from op...}
Shell Injection!

    In this case, echoing back Shell Injection! is pretty harmless. But shell injections are quite serious and allow an attacker to run arbitrary commands using the shell. They can also use some pretty advanced techniques too.

    In python you can escape user input for use in a shell command with shlex.quote(). It looks like the equivalent in R is shQuote(), but I haven't taken a close look at it. Generally speaking though it's best to avoid using the shell when possible.

    The good news is that, with your switch to system2() from shell(), it looks like you are directly executing op rather than running it through a shell. Please see my latest answer on this. It doesn't look like this is the case, so you may need to escape user input still.

    Could you expand the response on "stdin file descriptor" a bit more (or point me to something I could learn from?)

    Briefly speaking, a lot of things in Windows and Unix are treated as special files on a lower-level, even if we don't see that when programming on a higher level. For example, to communicate with some old printers, you'd write to the LPT1 file in any folder. As I write this post, on Windows you still can't call a file LPT1 or a variation like LPT1.txt because of this backwards compatibility.

    In the same way, stdin, stdout and stderr are all treated as files to a program even though they don't really exist themselves on the disk. Some languages will even let you read and write to them like normal files. For example, Microsoft have an example of writing to stdin and stdout as if they were files in C.

    If you want to read more about the background and some of the more technical details, Wikipedia seems to have a good starting point here: https://en.wikipedia.org/wiki/Standard_streams

    Even though the above system2() works reasonably well, I can't seem to pass multiple responses to it. For example, I am supposed to be able to pass a character vector and have each element passed as an input to command, but I instead get an error that again that seems to indicate its not passing the correct input:

    I'll be honest, this does seem like something we may be aware of. I found this thread which talked about a similar issue: https://1password.community/discussion/100528/using-1password-with-git#latest.

    The exact details vary slightly, but I'll still pass this on to our developers.

    If you can, I would let op handle obtaining the Secret Key and Master Password itself:

    system2('op', args = c('signin', R_DOMAIN, R_EMAIL, '--raw'))

    I was able to get this working in PowerShell, Fedora Remix (WSL2) and the RStudio Terminal (starting the R Console first by typing R and hitting return). The only place I couldn't get it working was the RStudio Console, and I think that's because it's not an interactive terminal. I realise there are quite a few similar terms there! :sweat:

  • jonlin
    jonlin
    Community Member

    @Matthew_1P - Thank you again for the great response and lots of Sunday reading. I couldn't have gotten this far without your feedback.

    I do agree, while it is preferable for R to allow it to interact with the prompts as presented by op, I haven't found any solutions by anyone that seem to let R interact with any application when prompts are presented. That is probably an R limitation, moreso than anything else.

    I did some debugging and saw what was being passed via the system2() command in R (I entered system2 in the R console to see the function). It looks like its still parsing the arguments (i.e. shQuote) before passing it to system(). I did notice its doing two things:

    1. It is writing the inputs to a temporary file (and destroying it upon exit of function). Therefore, system2(..., input = c(R_SECRET, R_PASSWORD, R_TWOFACTOR)) has these three elements written to a temporary file (blahblah, as shown below). Each line of this file temporary file has the inputs, followed by a blank line at the end.

    R_SECRET
    R_PASSWORD
    R_TWOFACTOR
    (blank line)

    1. Once this temporary file is written, it is then calling the OS command directly:

    'op' signin <R_DOMAIN> <R_EMAIL> <R_SECRET> --raw 2>&1 < '/var/folders/blahblah'" when system2(... sterr = TRUE), or

    'op' signin <R_DOMAIN> <R_EMAIL> <R_SECRET> --raw < '/var/folders/blahblah'" when system2(... sterr = FALSE)

    Would it be reasonable to believe that op is somehow not reading the subsequent lines in this temporary file that is created, when called in this manner?

  • SvenS1P
    SvenS1P

    Having had another look, it looks like system2 calls the shell still. I originally read "invokes the OS command" in the docs as meaning it directly invokes the command. Looking at the R source code this doesn't seem to be the case though, and it still gets called through the shell.

    One way to solve this is to escape each of your arguments as I mentioned before. It might also be worth checking in with an expert in R to see if there's a way to directly call a program without going through a shell.

    Would it be reasonable to believe that op is somehow not reading the subsequent lines in this temporary file that is created, when called in this manner?

    I actually tested it out using this method directly from the shell when trying to reproduce it before (i.e. running op signin {sign_in_address} {email_address} < op.txt, where op.txt contained my Secret Key and Master Password). Although it might be related to the issue I mentioned last time, I'm not certain. Either way, I have passed it on to our developers, so it's in safe hands. :smile::+1:

  • zcutlip
    zcutlip
    Community Member

    Hi @jonlin

    Can you expand as to if R was using a bourne shell, how that could be a vulnerability?

    It's generally advised not to use calls that pass a string to directly to a shell invocation. In Python or C, examples of this would be system(). The reason is that this string could be interpreted by the shell as a subcommand to execute, even if you only intend for it to be data. And input to the shell is nearly impossible to sanitize. This is a class of vulnerability called command injection. Even if you can rely in the input not being malicious it's still easy to find a situation where it breaks unexpectedly because something got interpreted in a special way by the shell.

    So let's say someone had the very clever password $(touch ~/foo)123. If that string was passed to the shell, the first part inside the parenthesis would be executed in a subshell and then be replaced with its output (which would be empty). So what you'd end up with is:

    • An empty file in your home directory called 'foo'
    • A password of just '123'

    It gets worse if the command injection is malicious like: $(rm -rf ~/)456 or $(cat /path/to/secret.txt | mail -s "stole your secrets hahaha")789

    Safer methods of calling external commands, such as Python's subprocess.Popen() never invoke a shell, but rather pass an array of arguments to fork() and exec() or whatever the equivalent is on your platform.

    Under normal R usage, I would store my passwords in a environment variable

    Be very thoughtful about this. It's generally advised not to keep secrets in memory for any longer than necessary. And environment variables are an especially poor place to store sensitive things. Any process (or child of a process) that has access to that environment could read your password. There are ways to sanitize environment variables and to ensure that variable only gets set for the process that needs it but that's extra work that you must be certain you're doing correctly. Otherwise you risk leaking your password to arbitrary processes.

    Your operating system may provide some sort of secure store (such as macOS's keychain) that is designed to only unlock when a user is logged on locally, so that might be worth researching for a way to retrieve a password programmatically, while ensuring it's stored safely.

    Regarding 2FA, I don't think you'll find a way to do this non-interactively (and if you did, that's sort of a breakdown in 2FA's role), but the good news is you only have to do that once, manually.

    It sounds like the thing you're doing with R is similar to a Python project I've been working on. Here's where I do sign-in:
    https://github.com/zcutlip/pyonepassword/blob/fba462c458d558c9c5a9ff658661c137f59805fa/pyonepassword/_py_op_cli.py#L133

    And here's an example using the whole thing to look up a password:
    https://github.com/zcutlip/pyonepassword/blob/fba462c458d558c9c5a9ff658661c137f59805fa/examples/example-signin-get-item.py

    Good luck!
    Zach

  • SvenS1P
    SvenS1P

    Thanks for sharing, you raise some great points here!

    When I was taking a look at R, I wasn't able to find any function that always calls op directly. If you're only going to run this on Windows though, the docs for R state that system() (not system2()) might work for your purposes.

    If you need it to work on all platforms, then I don't think any of the three included functions (shell(), system() and system2()) will work for you. In that case, it might be worth exploring whether you could write this using foreign function interfaces, but keep in mind that they come with a whole host of other considerations and might add quite a bit of complexity to what you're trying to achieve.

  • jonlin
    jonlin
    Community Member

    Thanks @zcutlip and @Matthew_1P again. Again, thanks to your leading points and education, I did some additional research last week. I think other than the .C() call that Matthew mentioned (which is beyond my depth!), there isn't any other direct ways to call op via R, except via system2().

    Given purpose of the package, I needed to introduce some validation and safeguards. I've made some adaptations so far to help detect such issues:

    • shQuote() to have a good sanitization on inputs that causes injection issues on my test case.
    • Add some basic expectation validation for specific scenarios
    • Added injection examples by Zach into testing

    Thanks to both of you for your help, this was a very educational experience for me. Theres some way to go for ratcheting it down further for security sake. For personal use, this should be appropriate. I'll keep watching to see if theres more to learn or apply.

  • ag_yaron
    ag_yaron
    1Password Alumni

    That's great to hear @jonlin !
    I also learned quite a bit just by reading this entire discussion :)

    If you do find any other valuable information or learn new things with R and our CLI, do share them here as it might help others :+1:

  • jpgoldberg
    jpgoldberg
    1Password Alumni

    This might be irrelevant, but if you are using RStudio it is possible to invoke a shell from the Rmarkdown. This might not be where you want it, but do take a look at https://bookdown.org/yihui/rmarkdown-cookbook/eng-bash.html

This discussion has been closed.