export specific secret value as an environment variable

Nadros

Hello 1Password Community :)

I am trying to fetch and export a specific secret value as an environment variable:

what I tried: - Note that I replaced the actual gcp location with [gcp location]

export AIRFLOW_SERVICE_ACCOUNT=$(op item get --cache --format=json [gcp location] | jq -r '.fields[]|select(.id="secret")|.value')

Then I fetch the environment variable as follows:

airflow_service_account = os.environ["AIRFLOW_SERVICE_ACCOUNT"]
service_account_json = "service-account.json"
with open(service_account_json, "w") as f:
    f.write(airflow_service_account)

Error:

raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 5)

when I run the following in the terminal:
op item get --cache --format=json [gcp location] | jq -r '.fields[]|select(.id="secret")|.value'

I get the following result (note that I replaced private data with the word bla

null
{
"type": "service_account",
  "project_id": "bla",
  "private_key_id": "bla",
  "private_key": "-----BEGIN PRIVATE KEY-----\nbla\n-----END PRIVATE KEY-----\n",
  "client_email": "bla",
  "client_id": "bla",
  "auth_uri": "bla",
  "token_uri": "bla",
  "auth_provider_x509_cert_url": "bla",
  "client_x509_cert_url": "bla"
}

I hope I was clear enough in my description.
Does anyone know what modifications I need to do to properly fetch this concerned service-account dict?

Thank you for your time :)

---

1Password Version: 7.9.4
Extension Version: Not Provided
OS Version: macOS 12.3.1

Justin.Yoon_1P

Hey @Nadros ,

It looks like you are saving a JSON object directly as a value in 1Password. At first glance, it seems like your code is having trouble parsing the JSON. Could you print what echo $AIRFLOW_SERVICE_ACCOUNT returns for you after you set that var (with the info redacted with "bla" of course)? I would like to see if there are any strings that we add for formatting that may break in your code.

Additionally, I do think there may be an alternative - to set each key:value in the JSON as its own field in the item.

If you take this approach, each of the values can be accessed by our secret reference syntax, which in conjunction with our op run command to start your app, can load distinct env vars for use during runtime.

Nadros

Hello @Justin.Yoon_1P .

Thank you for getting back to me.

Small context about the secret: is it used to access an account on Airflow, which is kind of a task orchestrator.

The output of the echo $AIRFLOW_SERVICE_ACCOUNT command is the following:

null
{
"type": "service_account",
  "project_id": "bla",
  "private_key_id": "bla",
  "private_key": "-----BEGIN PRIVATE KEY-----
                  bla
-----END PRIVATE KEY-----
",
  "client_email": "bla",
  "client_id": "bla",
  "auth_uri": "bla",
  "token_uri": "bla",
  "auth_provider_x509_cert_url": "bla",
  "client_x509_cert_url": "bla"
}

which I believe is the same as calling the op command as per my 1st message.

when I run the following command op item get --cache --format=json [gcp location] this is the entire dict:

{
  "id": "bla",
  "title": "bla",
  "version": 1,
  "vault": {
    "id": "bla",
    "name": "Engineering"
  },
  "category": "SECURE_NOTE",
  "last_edited_by": "bla",
  "created_at": "bla",
  "updated_at": "bla",
  "fields": [
    {
      "id": "notesPlain",
      "type": "STRING",
      "purpose": "NOTES",
      "label": "notesPlain"
    },
    {
      "id": "secret",
      "type": "CONCEALED",
      "label": "secret",
      "value": "{\n\"type\": \"service_account\",\n  \"project_id\": \"bla\",\n  \"private_key_id\": \"bla\",\n  \"private_key\": \"-----BEGIN PRIVATE KEY-----\\nbla\\n-----END PRIVATE KEY-----\\n\",\n  \"client_email\": \"bla\",\n  \"client_id\": \"bla\",\n  \"auth_uri\": \"bla\",\n  \"token_uri\": \"bla\",\n  \"auth_provider_x509_cert_url\": \"bla\",\n  \"client_x509_cert_url\": \"bla\"\n}"
    }
  ]
}

and the part that I am trying to isolate is the entire value inside the key 'value' near the end.

Horia.Culea_1P

Hey @Nadros, thank you for reaching out to us!
Quick question, before diving deeper into troubleshooting this: Have you tried the op read command? By glancing through the thread, I think this may actually be a simpler way to achieve the same result. The syntax would be: op read op://<vault-name>/<item-name>/<field-name>.
More documentation can be found at https://developer.1password.com/docs/cli/reference/commands/read/
Let us know if this helps you!
Looking forward to hearing from you.

Best,
Horia

Nadros

Hello @Horia.Culea_1P ,
I do not think that the secret in question is hosted within a vault folder, because when I run
op read op://gcp/bla/iam/jw-airflow-serviceaccount, I get the following error:

could not read secret op://gcp/bla/iam/xm-airflow-serviceaccount: could not get item gcp/bla: "gcp" isn't a vault in this account. Specify the vault with its ID or name.

and when I run op read op://gcp/bla/iam/xm-airflow-serviceaccount/secret

(the only thing that is present in the concerned OP folder is secret), I get the following:

too many '/': secret references should match op://<vault>/<item>[/<section>]/<field>

I am checking this with the devops team as we speak.

Thank you for your time.
Respectfully,

Maher.

Horia.Culea_1P

Hmm let me make sure I understand the issue correctly:

When you run op item get --cache --format=json [gcp location], the [gcp location] part is, in fact, the name of your item, is this correct?

The issue here may be that this name contains the "/" character, which is not valid in secret references.

So, to reiterate, in order to create a valid secret reference, you would need:
1. the vault name or ID (Engineering, from the json output I see you provided above)
2. the item name or ID - if your item name contains unsupported characters, you can use the item ID, that you replaced above with "bla"
3. the field name or ID - secret, in your case

So I think a valid secret reference should be op://Engineering/<your-item-id>/secret.
Can you please let us know what does op read, given the above as an argument, return?

Thank you!

Best,
Horia

Nadros

ok now I got to receive the isolated value

old command that was almost getting me the value of the secret:

export AIRFLOW_SERVICE_ACCOUNT=$(op item get --cache --format=json [gcp location] | jq -r '.fields[]|select(.id="secret")|.value')

new command:

export AIRFLOW_SERVICE_ACCOUNT=$(op item get --cache --format=json [gcp location] | jq -r '.fields[]|select(.id=="secret")|.value')

only difference is the double "=" sign in the "select" field.

Horia.Culea_1P

Glad you got it working!

Please do let us know if you encounter any other hurdles, we're here to help :)

Best,
Horia