performance question with strange use case, maybe?

cttwapps

Hello, I'm trying to use op get item populate environment variables at login rather than storing secrets and API tokens in clear text as part of .zshrc or .bash_profile. I'm noticing that it takes about 2.6 - 2.7sec to get the results of op get item --fields <filed> <item_name>. While this is not a huge concern when getting one item I plan on getting at least five to create a full dev environment. For example, the item Google-Dev has standard username and password fields, as well as an API token which is saved to a text field named "Personal Access Token". I'm using the following command export the environment variable:

export YOUTUBE_DEV_KEY=$(op get item --fields "Personal Access Token" google-dev)

I have two major questions related to my use case:

1. Am I using a known inefficient means of getting a field from an item? This seemed more direct than getting an entire JSON document and using jq to get what I really need.
2. Is this a reasonable response time on an op get item request and I'm just impatient?

Any help or guidance is greatly appreciated. Thank you.

---

1Password Version: 1.6.0
Extension Version: Not Provided
OS Version: OS X 10.15.6
Sync Type: Not Provided

DevOma

I try also to read a bunch of vaults/secrets and the performance is really poor - I can't understand why every command takes so long - in huge environments it is not usable - for testing I created 3 vaults and add 3 to 4 secrets - to read the data it takes about 30 seconds!

cttwapps

A bit of follow-on with some details:

I have been storing the private part of ssh-keys into 1password as Secure Notes (base64 encoded text) using a uniforming naming scheme and I fetch them with the following:

op list items --categories "Secure Note" | op get item - --fields title | grep -i "ssh-key"

This takes an average of 2.2s to return a list of four items, half of which is the 'list items' (I'm sure that as I create more Secure Notes this will skew :( ). However, fetching the contents of a single note, writing it to disk, decoding that output and extracting the pub key via ssh-keygen takes on average 1.7s to complete, approximately 94% of which is the following:

op get item --fields notesplain

The take way is that it takes an average of 1.7s to manipulate 2240 bytes of text when working with ssh-keys. My original thread is about manipulation of an average of 39 bytes of text. From my testing it seems time spent during 'get item' requests is the response time from the 1password.com API. :(

cttwapps

A bit of follow-on:

Sadly, I am also able to duplicate the poor performance with Ubuntu 20.04.1 LTS as well as the original MacOS 10.15.6 system.

felix_1p

Hi @cttwapps,

the tl;dr is: There will always be a certain performance overhead but there are things you can do to make the commands faster.

The is always a performance overhead when initiating a TLS connection to a server. On my system it always takes at least 500ms to make a successfully roundtrip to an HTTPS server. If you add the additional verification/encryption overhead for how the command line tool communicates with the server we are at least at 700ms.

However, fetching an item by name only is the slowest way to request an item. Since item names are encrypted we actually have to download all items, decrypt them and look for the item locally.
The best you can do right now is to reference the item via its UUID and also set the vault UUID:

op get item <item_uuid> --vault <vault_uuid>

That's basically an instant lookup on the server.

If you want to fetch a couple of items you can pass multiple item UUIDs via standard input. This is also a lot faster then fetching each item individually since you only pay the connection setup cost once.

cttwapps

Hi @felix_1p,

Understood and thanks for the suggestion. I'll try using the uuid and vault_uuid to see if fetches are faster; it will require a little more work upfront when adding new things to the dev environment but if I only have to pay that cost, as well as the connection setup cost, once I'm totally okay with that.

Thank you.

cttwapps

Hi again @felix_1p,

Happy to report that using vault and item uuid brings the item fetch time down to an average of 0.75s for Secure Notes larger than 2000 bytes. Since I don't expect the vault uuid to ever change, unless I do something destructive, terrible, or stupid, I can store that value in the initial environment setup and use it to my heart's content!

@DevOma you might want to look at Felix's suggestion for your use case. Hopefully, it yields the same decrease in overhead.

Thank you again!

SvenS1P

On behalf of Felix, you're very welcome @cttwapps! Let us know if there's anything else we can help with. :chuffed: