How do I use the AWS Shell Plugin with Terraform?

Options
dejanz
dejanz
Community Member

I set up the 1password shell plugin for aws as shown here: https://blog.1password.com/shell-plugins/

It works great with aws (super cool btw!!!)

But It doesn't work with Terraform. For Terraform, I still have to have the credentials in my ~/.aws/credentials file.

Am I missing something? Or would a separate plugin for Terraform have to exist?

I know that there is a 1Password Terraform provider, but it's not quite what I was imagining.


1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Browser:_ Not Provided
Referrer: forum-search:aws terraform

Comments

  • lshevtsov
    lshevtsov
    Community Member
    Options

    Also interested in this.

    Similarly, if i have a script that needs AWS credentials to run, I can't supply the credentials via the 1Password AWS plugin.

    What i'd like to see is a feature to run op plugin run -- aws with my command instead of the aws command.

  • Hi @dejanz and @lshevtsov:

    Great question! What may work in this case is using the AWS environment variable functionality of Terraform: Terraform hashicorp/aws

    Set the relevant environment variables to 1Password CLI secret references (op://<vault>/<item>/<field>), then use op run -- terraform ... to use your credentials from 1Password.

    Let me know how you get on with that!

    Jack

  • lshevtsov
    lshevtsov
    Community Member
    Options

    I see, so for the aws command, I can use the plugin, and for the rest I can populate env vars from the same 1Password item with op run.

    That solves my issue, thanks!

  • MerelyGin
    MerelyGin
    Community Member
    Options

    @Jack.P_1P your answer seems to have solved the issue for @lshevtsov which is awesome, I have a similar problem running terragrunt commands.
    Could you please expand/explain what you mean by the following?

    Set the relevant environment variables to 1Password CLI secret references (op:////), then use op run -- terraform ... to use your credentials from 1Password.

    I am not sure how I would set the env vars?

  • MerelyGin
    MerelyGin
    Community Member
    Options

    Please ignore previous comment, I have found out how to do this using https://developer.1password.com/docs/cli/secrets-environment-variables/

    I think I have a bigger problem. Terragrunt uses a function get_aws_account_id() and this doesn't work when I am trying to use a set of assume a role via my root "security" account. Terragrunt assumes I have a ~/.aws/credentials file yet when using the op aws plugin it suggests I can remove this file.

    If anyone has experience of using terragrunt with aws op plugin, I'm excited to see how you achieved it

  • Hey folks, have you heard about our Terraform shell plugin being available in beta? Download our latest beta and it's as easy as:

    cd ./your-tf-aws-project
    op plugin init terraform; <select your AWS credentials>
    terraform plan

    Let us know what you think!

  • lantrix
    lantrix
    Community Member
    Options

    This is great. I've configured the terraform AND the was plugins, but my terraform is using an AWS S3 backend, and it still can't find the credentials.

    Output of the inspect for both plugins:

    op plugin inspect aws
    
    AWS CLI
    Credential type: Access Key
    
    Configured Aliases
    
    ✔ Alias for "aws" configured
    ✔ Aliases sourced (/Users/lantrix/.config/op/plugins.sh)
    
    Configured Credentials
    
    ✔ "AWS Access Key (NewProject)" (vault: "Private")
      Configured for directory "/Users/lantrix/repos/terraform-infrastructure". (takes precedence)
    
    ✔ "AWS Access Key" (vault: "Private")
      Configured as global default.
    

    and

    op plugin inspect terraform
    
    Terraform CLI
    
    Configured Aliases
    
    ✔ Alias for "terraform" configured
    ✔ Aliases sourced (/Users/lantrix/.config/op/plugins.sh)
    
    Configured Credentials
    
    ✔ "AWS Access Key (NewProject)" (vault: "Private")
      Configured for directory "/Users/lantrix/repos/terraform-infrastructure".
    

    But on init of the backend, still can't find the credentials. I'm assuming it's because the alias is being used to wrap op.

    terraform init \
        -backend-config="region=ap-southeast-2" \
        -backend-config="workspace_key_prefix=newproject-infra" \
        -backend-config="bucket=terraform-state-${accountId}" \
        -backend-config="key=newproject-infra-terraform.tfstate" \
        -backend-config="dynamodb_table=terraform-state"
    
    Initializing the backend...
    Initializing modules...
    ╷
    │ Error: error configuring S3 Backend: no valid credential sources for S3 Backend found.
    │ 
    │ Please see https://www.terraform.io/docs/language/settings/backends/s3.html
    │ for more information about providing credentials.
    │ 
    │ Error: NoCredentialProviders: no valid providers in chain. Deprecated.
    │       For verbose messaging see aws.Config.CredentialsChainVerboseErrors
    │ 
    │ 
    │
    

    Terraform main.tf is configured for the backend too. This works with normally placed AWS CLI credentials in $HOME/.aws/credentials

    terraform {
      required_providers {
        aws = {
          source  = "hashicorp/aws"
          version = "5.16.1"
        }
      }
      backend "s3" {
        workspace_key_prefix = "newproject-infra"
        bucket               = "terraform-state-123456789012"
        key                  = "newproject-infra-terraform.tfstate"
        region               = "ap-southeast-2"
        dynamodb_table       = "terraform-state"
      }
    }
    
  • Maelstromeous
    Maelstromeous
    Community Member
    Options

    It appears the terraform plugin has been yoinked, I don't see it in the plugin list :-/

  • Maelstromeous
    Maelstromeous
    Community Member
    edited September 2023
    Options

    I've figured out a decent workaround for now until the TF plugin is restored.

    In order to use TF with AWS, you need to inject the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY dynamically. This unfortunately cannot be done via MFA via 1password as far as I know, so you need to have a IAM user created with credentials specifically for Terraform. This can be done via the below process:

    1. Make sure you've followed the AWS CLI setup guide, it shows you how to properly create a credential.
    2. Create a shell script called "1passwordcreds.sh" and put it in your home dir
    3. Insert the following into the file:
    #!/bin/bash
    
    # Get your AWS access key ID and secret access key from 1Password
    AWS_ACCESS_KEY_ID=$(op read "op://<VAULT NAME>/<1PASSWORD ENTRY>/access key id")
    AWS_SECRET_ACCESS_KEY=$(op read "op://<VAULT NAME>/<1PASSWORD ENTRY>/secret access key")
    
    # Export the AWS credentials as environment variables
    export AWS_ACCESS_KEY_ID
    export AWS_SECRET_ACCESS_KEY
    
    # Print the AWS credentials to the console
    echo "AWS Access Key ID: $AWS_ACCESS_KEY_ID"
    
    1. Replace <VAULT_NAME> and <1PASSWORD_ENTRY> according to your needs. e.g. mine in my "Dev" vault with the item called "AWS - Terraform" reads:
    AWS_ACCESS_KEY_ID=$(op read "op://Dev/AWS - Terraform/access key id")
    AWS_SECRET_ACCESS_KEY=$(op read "op://Dev/AWS - Terraform/secret access key")
    
    1. chmod +x ~/1passwordcreds.sh
    2. Run source ~/1passwordcreds.sh
    3. If you've set up the credential properly in 1Password, you'll be prompted for your biometrics.
    4. To confirm you now have credentials, run printenv | grep -i aws, you should see your credentials. You should see both the access key ID and secret key.
    5. Run terraform apply etc.

    Note because you are sourcing the file, the creds will "delete" themselves when the terminal ends, there is no storage of the credentials anywhere in the file system or in e.g. ./aws/credentials.

    To ease debugging, here's my terraform file which works:

    terraform {
      backend "s3" {
        bucket = "<bucket>"
        key    = "terraform.tfstate"
        region = "eu-west-2"
      }
    }
    
    provider "aws" {
      region = "eu-west-2"
    }
    

    No special keys or anything are needed.

  • lantrix
    lantrix
    Community Member
    Options

    @Maelstromeous the terraform plugin is in the beta https://developer.1password.com/docs/cli/shell-plugins/terraform/

    I used your workaround and it works. Thanks.

This discussion has been closed.