How do I use the AWS Shell Plugin with Terraform?

dejanz

I set up the 1password shell plugin for aws as shown here: https://blog.1password.com/shell-plugins/

It works great with aws (super cool btw!!!)

But It doesn't work with Terraform. For Terraform, I still have to have the credentials in my ~/.aws/credentials file.

Am I missing something? Or would a separate plugin for Terraform have to exist?

I know that there is a 1Password Terraform provider, but it's not quite what I was imagining.

---

1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Browser:_ Not Provided
Referrer: forum-search:aws terraform

lshevtsov

Also interested in this.

Similarly, if i have a script that needs AWS credentials to run, I can't supply the credentials via the 1Password AWS plugin.

What i'd like to see is a feature to run op plugin run -- aws with my command instead of the aws command.

Jack.P_1P

Hi @dejanz and @lshevtsov:

Great question! What may work in this case is using the AWS environment variable functionality of Terraform: Terraform hashicorp/aws

Set the relevant environment variables to 1Password CLI secret references (op://<vault>/<item>/<field>), then use op run -- terraform ... to use your credentials from 1Password.

Let me know how you get on with that!

Jack

lshevtsov

I see, so for the aws command, I can use the plugin, and for the rest I can populate env vars from the same 1Password item with op run.

That solves my issue, thanks!

MerelyGin

@Jack.P_1P your answer seems to have solved the issue for @lshevtsov which is awesome, I have a similar problem running terragrunt commands.
Could you please expand/explain what you mean by the following?

Set the relevant environment variables to 1Password CLI secret references (op:////), then use op run -- terraform ... to use your credentials from 1Password.

I am not sure how I would set the env vars?

MerelyGin

Please ignore previous comment, I have found out how to do this using https://developer.1password.com/docs/cli/secrets-environment-variables/

I think I have a bigger problem. Terragrunt uses a function get_aws_account_id() and this doesn't work when I am trying to use a set of assume a role via my root "security" account. Terragrunt assumes I have a ~/.aws/credentials file yet when using the op aws plugin it suggests I can remove this file.

If anyone has experience of using terragrunt with aws op plugin, I'm excited to see how you achieved it

andi.t_1P

Hey folks, have you heard about our Terraform shell plugin being available in beta? Download our latest beta and it's as easy as:

cd ./your-tf-aws-project
op plugin init terraform; <select your AWS credentials>
terraform plan

Let us know what you think!

lantrix

This is great. I've configured the terraform AND the was plugins, but my terraform is using an AWS S3 backend, and it still can't find the credentials.

Output of the inspect for both plugins:

op plugin inspect aws

AWS CLI
Credential type: Access Key

Configured Aliases

✔ Alias for "aws" configured
✔ Aliases sourced (/Users/lantrix/.config/op/plugins.sh)

Configured Credentials

✔ "AWS Access Key (NewProject)" (vault: "Private")
  Configured for directory "/Users/lantrix/repos/terraform-infrastructure". (takes precedence)

✔ "AWS Access Key" (vault: "Private")
  Configured as global default.

and

op plugin inspect terraform

Terraform CLI

Configured Aliases

✔ Alias for "terraform" configured
✔ Aliases sourced (/Users/lantrix/.config/op/plugins.sh)

Configured Credentials

✔ "AWS Access Key (NewProject)" (vault: "Private")
  Configured for directory "/Users/lantrix/repos/terraform-infrastructure".

But on init of the backend, still can't find the credentials. I'm assuming it's because the alias is being used to wrap op.

terraform init \
    -backend-config="region=ap-southeast-2" \
    -backend-config="workspace_key_prefix=newproject-infra" \
    -backend-config="bucket=terraform-state-${accountId}" \
    -backend-config="key=newproject-infra-terraform.tfstate" \
    -backend-config="dynamodb_table=terraform-state"

Initializing the backend...
Initializing modules...
╷
│ Error: error configuring S3 Backend: no valid credential sources for S3 Backend found.
│ 
│ Please see https://www.terraform.io/docs/language/settings/backends/s3.html
│ for more information about providing credentials.
│ 
│ Error: NoCredentialProviders: no valid providers in chain. Deprecated.
│       For verbose messaging see aws.Config.CredentialsChainVerboseErrors
│ 
│ 
│

Terraform main.tf is configured for the backend too. This works with normally placed AWS CLI credentials in $HOME/.aws/credentials

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "5.16.1"
    }
  }
  backend "s3" {
    workspace_key_prefix = "newproject-infra"
    bucket               = "terraform-state-123456789012"
    key                  = "newproject-infra-terraform.tfstate"
    region               = "ap-southeast-2"
    dynamodb_table       = "terraform-state"
  }
}

Maelstromeous

It appears the terraform plugin has been yoinked, I don't see it in the plugin list :-/

Maelstromeous

I've figured out a decent workaround for now until the TF plugin is restored.

In order to use TF with AWS, you need to inject the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY dynamically. This unfortunately cannot be done via MFA via 1password as far as I know, so you need to have a IAM user created with credentials specifically for Terraform. This can be done via the below process:

1. Make sure you've followed the AWS CLI setup guide, it shows you how to properly create a credential.
2. Create a shell script called "1passwordcreds.sh" and put it in your home dir
3. Insert the following into the file:

#!/bin/bash

# Get your AWS access key ID and secret access key from 1Password
AWS_ACCESS_KEY_ID=$(op read "op://<VAULT NAME>/<1PASSWORD ENTRY>/access key id")
AWS_SECRET_ACCESS_KEY=$(op read "op://<VAULT NAME>/<1PASSWORD ENTRY>/secret access key")

# Export the AWS credentials as environment variables
export AWS_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY

# Print the AWS credentials to the console
echo "AWS Access Key ID: $AWS_ACCESS_KEY_ID"

4. Replace <VAULT_NAME> and <1PASSWORD_ENTRY> according to your needs. e.g. mine in my "Dev" vault with the item called "AWS - Terraform" reads:

AWS_ACCESS_KEY_ID=$(op read "op://Dev/AWS - Terraform/access key id")
AWS_SECRET_ACCESS_KEY=$(op read "op://Dev/AWS - Terraform/secret access key")

5. chmod +x ~/1passwordcreds.sh
6. Run source ~/1passwordcreds.sh
7. If you've set up the credential properly in 1Password, you'll be prompted for your biometrics.
8. To confirm you now have credentials, run printenv | grep -i aws, you should see your credentials. You should see both the access key ID and secret key.
9. Run terraform apply etc.

Note because you are sourcing the file, the creds will "delete" themselves when the terminal ends, there is no storage of the credentials anywhere in the file system or in e.g. ./aws/credentials.

To ease debugging, here's my terraform file which works:

terraform {
  backend "s3" {
    bucket = "<bucket>"
    key    = "terraform.tfstate"
    region = "eu-west-2"
  }
}

provider "aws" {
  region = "eu-west-2"
}

No special keys or anything are needed.

lantrix

@Maelstromeous the terraform plugin is in the beta https://developer.1password.com/docs/cli/shell-plugins/terraform/

I used your workaround and it works. Thanks.