Ansible Lookup Plugin?
Has anybody worked on creating a lookup plugin for Ansible using the new CLI?
I may work on one myself if not, but I don't want to duplicate work that's already done (and open sourced) either. It seems like the Lastpass lookup plugin that's already distributed with Ansible could be used as a starting point:
https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/lookup/lastpass.py
1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Sync Type: Not Provided
Comments
-
I'm not aware of such a plugin being developed by anyone. Ansible seems neat though, I hadn't heard of it before today.
Rick
0 -
FYI, I've gone ahead and built this myself. Will be open sourcing it shortly and/or contributing it to Ansible core. Just a heads up to anybody who finds this thread.
0 -
That's awesome. We'd love to hear about it when it's available.
Rick
0 -
I have submitted this as a pull request, hopefully to be distributed with Ansible:
https://github.com/ansible/ansible/pull/37207If it gets rejected for some reason, then I will bundle it up as a plugin for others to install via Galaxy.
0 -
Woo! Here's to hoping they merge the pull request. :)
Rick
0 -
I’m an Ansible Core developer. I'll review the PR and see about getting it merged.
0 -
Since you can also edit items with the 1Password CLI it would also be feasible to create modules to create, update, and delete 1Password records. This would make 1Password a first-class secret management tool alongside other "big name" secret stores. I like the implications of this.
0 -
Unfortunately we can't actually edit items yet with
op. Most of the lower level work needed to make that happen is there (it's so similar to creation). One of the things we've been having trouble with is just how to expose this to users. To be honest, I'm not a big fan of howop create itemworks, and I think we need to reconsider how it works. Every item has effectively 3 parts: the item metadata (createdAt, updatedAt, etc...) which isn't encrypted, the item overview (contains things like title, urls, tags) which is encrypted, and item details (everything really interesting goes in there) which is also encrypted. Currentlyop create itemhas you give it only the details, and then specify a few of the fields you can put in the overview as flags.My gut says that we need to redo things all the way back to
op get templateand have that command return you a structure that looks just likeop get itemwould, except empty. Then you'd pass that whole structure (which includes metadata, overview, details) over toop create item. If we did that, thenop update itemcould work exactly the same and the world would be a beautiful place.Rick
0 -
Yes! @rickfillion ,
I've been contemplating the road path to providing my converter suite with the ability to submit entries directly via
op create item, but I don't think the CLI is ready for that yet. There does need to be a structure that supports packaging the metadata, key field data, notes, and perhaps even attachments (+ internal linking).0 -
God that'd be hot, @MrC.
Rick
0 -
FYI, the Ansible lookup plugin for 1PW was merged today. Thanks to @sam.doran for help with that. Planned for Ansible 2.6, which is anticipated to release as stable on June 28.
0 -
Way to go @scottsb and @sam.doran! That's awesome.
Rick
0 -
Just want to thank @scottsb and @sam.doran for this. Something like hashicorp vault is overkill for some of the things I'm doing and this makes my setup a lot easier.
0 -
@eby Glad to hear!
I added login capability to the lookups for Ansible 2.7 (release date is 2018-10-04), plus a community member contributed a facts module to Ansible, allowing 1Passwod items to be used as Ansible facts.
Right now I’m working on unifying the interface across them all so it feels consistent.
0 -
That's super cool!
0 -
Hey @scottsb, @sam.doran,
I am looking for some feedback on how 1Password works with ansible.
We are currently in the way of leaving LastPass due to issues we encounter with Ansible and our other DevOps scripts. Mainly we are facing two huge issues with LastPass:
- ID of items change when you move them from one vault to another
- SSL verification fails randomly causing various CLI calls to fail and so causing the script where the lastpass cli call is done to fail also. This could be the run of our Ansible playbooks or also the lost of some passwords due to synchronization failures.
The "good" thing with LastPass is that his CLI have good performance, around 130ms to query our vault with 1500 entries.
So, how does 1Password plays with your playbook. We manage the hosting infrastructure of our customers with Ansible and have hundred of lookup query in our playbook. So we are looking for a fast and reliable solution.
Do you advice 1Password to handle such an ansible usage ?
Do you benchmark the time of a lookup query ?
Are you using 1Password with the cloud service or with a local vault ?
Thanks for your feedbacks,
0 -
Do you advice 1Password to handle such an ansible usage?
1Password would probably work fine but you would need to change your strategy. If you have hundreds of lookups per play, I would recommend using the
onepassword_factsmodule to get everything you need in one query, rather than making hundreds of queries. Note that once those items are stored as facts, you will need to safeguard that data. Ansible does not treat data from 1Password any different than other variables. Be especially careful if you have fact caching enabled. (This is not unique to 1Password — this applies to data from any lookup plugin).Do you benchmark the time of a lookup query?
I did during initial development but don't recall the exact numbers. Rather than focus on total time per query I would find ways to do fewer queries. I just did a few tests on macOS 10.14.4 with
opversion 0.5.5 by runningtime op get item fooand the results range between0.78and0.95Are you using 1Password with the cloud service or with a local vault?
I did my testing and development using the cloud service. I never tested using a local vault.
If you need a truly high performance secret store, you could also look at Hashicorp Vault. Ansible has a lookup plugin for that as well.
0 -
Are you using 1Password with the cloud service or with a local vault ?
Sam's answers are good, but one key note: the Ansible plugin uses the
opcommand line tool under the surface, and that only works with the cloud service, not local vaults.0 -
@sam.doran Thanks a lot for that quick and detailed answer. You exposed a really interesting strategy, I wasn't aware of
onepassword_factsthis could be definitely the solution. I will dig into this. Thanks so much ;)However, I am really surprised by the performance of your CLI, which version are you using ? I have exposed my bencharks on this post https://discussions.agilebits.com/discussion/comment/502630/ and I am around 4-5s :dizzy: Are you using a future unreleased version of the CLI ?
Regarding Hashicorp Vault, yeah, this could be the solution for ansible, but this won't fill all the others features that can give 1Password: outstanding UI, apps everywhere, usable by non technical users, pricing as we don't want to handle the hosting ourselves...
@scottsb thanks for your clarification, we don't intend to use local vaults especially if they can't synchronize with online service
0 -
Not sure why you're seeing such long times. Here is my result from running your test. Version 0.5.5 does seem to be a bit faster than previous versions.
0 -
@sam.doran Correct me if I am wrong, I have inspected the
onepassword_factssource code. But as far as I understand, each fact is still gathered from a separate call of theopcli in theget_rawmethod.So, the profit is "only" to prevent calls to the lookup module each time the group/host config need to be accessed. This is fine but with a response of 4s by each
opquery the run of ansible would be delayed by 20 minutes for 300 entries. Still unacceptable sadly :(0 -
@sam.doran I am currently thinking about a way to use the native ansible vault tocache 1Password secrets.
Ideally, if a var is not defined, ansible would call a custom plugin and try to lookup the value in 1Password depending on the naming scheme for example. If the value can be grabbed, it will be stored in the vault.
Do you have any advices if this could be possible ? I identified some Python projects that helps in updating Ansible vault but currently have no clue on how to handle the fallback if the var is not defined.
A workaround could be to use a specific lookup plugin that will search in the vault before querying 1Password, but at the expense of a more verbose syntax.
0 -
It looks like this is currently limited to items and not capable of retrieving documents at the moment. If I were to submit a PR adding that functionality, would the preference be to amend the current
onepasswordlookup, or to create a separateonepassword_doclookup specifically for documents?The semantics are similar but not matching...
usage: op get document <document> [--vault=<vault>] > <filename>
vs...usage: op get item <item> [--vault=<vault>] [--include-trash]0 -
@sam.doran is the Ansible core maintainer amongst us, so he would be best suited to answer. :-)
0 -
That sounds like it would be entirely doable :)
0 -
@jamesdh There is an open pull request to add a
onepassword_doclookup plugin. I had some issues with it in my testing and left some feedback in the review. I'm waiting on the PR author to address those comments.0 -
FYI,
I just published the Ansible lookup module that uses my 1Password local database search module.
I don't have finished to migrate to 1Password so I didn't use it in production so far but Molecule tests are passing ;)
Code is here: https://github.com/mickaelperrin/ansible-onepassword-local-lookup-plugin
ping @sam.doran
0
