Very slow CLI and possible cancelled call bugs on Windows
Hi,
This has been raised a few times in the past which I've seen and so let it be as it was always replied to suggesting that it was being actively worked on. But performance of the CLI on Windows is abysmal and actually just getting worse in later releases. Moreover the behaviour is suggestive of true bugs, imho, as I show below.
But it never takes less than 5s in v2.13.1 to return a result, and older versions never take less than 4.9s. This is far from what any CLI tool user would accept and slows our workflows dramatically. We really want to make 1Password central to our organisation but this makes it unusable for widespread adoption.
I upgraded to the latest this morning to 2.13.1 and it's worse than the previous versions I was using (only by a little, but still). I was on v2.3 as v2.6+ claimed perf improvements but they weren't apparent in testing. I've given the timings and suggestions on timing below. It really makes it unusably slow as a secrets manager and since the lack of caching in Windows has been acknowledged for ~a year, it really seems like the team doesn't care about supporting Windows despite this being the majority of the corporate desktop world.
Setup
My standard setup is Windows 11 using Git Bash, but PowerShell is the same. Hardware is very good. Internet connection is WiFi but from a place of little variance (>50Mbps) though I'm unclear if that's even relevant. 1Password desktop client is fine and fast.
I'm based in the UK. If I use a VPN in continental Europe the performance is the same.
We have a few dozen vaults in our account and single-digit thousands of entries. High hundreds of entries in my personal vault.
Summary of conclusions
This is what I conclude from observations:
Windows caching is inactive: this is known and I can't see any updates on progress, but surprisingly it still tries to use it by default despite having a platform specific executable. Only adding --cache=false manually seems to stop this call, but regardless it doesn't affect performance
Previous threads have suggested specifying a vault helps. This doesn't affect performance notably (possibly by 100ms but within the typical noise). I can validate it is using the Vault but I see no difference in timing.
Overall the time seems to be in discrete blocks with a mean of ~2.6s and variance of around 0.3s in 2.13.1. Earlier versions have a slightly reduced mean of ~2.5s but around the same distribution.
The time taken seems to be a discrete multiple of this time with minimum of 2 executions. That is, it never takes less than ~2.6s x 2 (~5.2s +/- the variance) but sometimes takes ~2.6s x 3 or x 4. It's possibly more like "2.8s for the first mandatory call, then ~2.4s x (some integer)" with that first call taking ~1-200ms longer in v2.13 compared to v2.12 and earlier.
The strangest behaviour of all to me is that if I run op --cache=false --debug --vault Private item get --fields username Dummy
then Ctrl-C it in the first 2.5s before it completes, then run then command again within 5 seconds from the original call, this is the ONLY time I can get it to return in less 3s (actually the only time less than 4.9s) when measuring the time for the second call but it's still 5s overall from the start of the first, cancelled call.
This suggests to me that there is some lengthy ~2.6s initial handshake that the act of Ctrl-C-ing before it returns/runs the second block, then running a second call doesn't repeat this handshake (possibly something to look into for security reasons too if the first calls are doing some sort of handshake that can be reused).
I'll give the raw results below so you can see (apologies for the length but want to give full data upfront).
EDIT: Apparently it's too long to post. I'll try to add it in replies to this post.
1Password Version: 8.9.14
Extension Version: 2.13.1
OS Version: Windows 11
Browser:_ N/A
Comments
-
Showing how vault and cache selection don't affect perf
`
$ for i in {1..30}; do { time op --cache=false --debug --vault Private item get --fields username Dummy 2>&1; } 2>&1 | grep -iE "(cache|real)"; done
real 0m7.936s
real 0m5.287s
real 0m5.103s
real 0m5.212s
real 0m5.127s
real 0m5.341s
real 0m5.283s
real 0m7.770s
real 0m5.077s
real 0m5.120s
real 0m5.053s
real 0m5.608s
real 0m5.025s
real 0m5.269s
real 0m7.789s
real 0m5.188s
real 0m5.232s
real 0m5.148s
real 0m5.185s
real 0m5.190s
real 0m5.461s
real 0m8.771s
real 0m5.177s
real 0m5.110s
real 0m5.053s
real 0m5.335s
real 0m5.288s
real 0m5.214s
real 0m9.261s
real 0m5.029s/d/dev/bin [230206-111101]
$ for i in {1..30}; do { time op --debug --vault Private item get --fields username Dummy 2>&1; } 2>&1 | grep -iE "(cache|real)"; done
11:11AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m8.124s
11:11AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m6.358s
11:11AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m6.535s
11:11AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.548s
11:11AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.980s
11:12AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.151s
11:12AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.500s
11:12AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m7.942s
11:12AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.245s
11:12AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.963s
11:12AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m7.098s
11:12AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.408s
11:12AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.490s
11:12AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.582s
11:13AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m9.435s
11:13AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.412s
11:13AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.299s
11:13AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m6.247s
11:13AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.400s
11:13AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.246s
11:13AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.175s
11:13AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m7.867s
11:13AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.126s
11:13AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.067s
11:13AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.807s
11:14AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.827s
11:14AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.178s
11:14AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m5.422s
11:14AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m8.209s
11:14AM | WARN | InitDefaultCache: failed to establish RPC connection with daemon: You can't use caching on Windows.
real 0m11.729s/d/dev/bin [230206-111436]
$ for i in {1..30}; do { time op --cache=false --debug item get --fields username Dummy 2>&1; } 2>&1 | grep -iE "(cache|real)"; done
real 0m8.941s
real 0m7.501s
real 0m5.282s
real 0m18.222s
real 0m7.975s
real 0m5.225s
real 0m6.513s
real 0m5.431s
real 0m5.643s
real 0m5.251s
real 0m5.338s
real 0m7.888s
real 0m5.264s
real 0m5.615s
real 0m5.286s
real 0m9.412s
real 0m7.909s
real 0m5.874s
real 0m7.880s
real 0m5.455s
real 0m5.237s
real 0m6.798s
real 0m5.127s
real 0m7.781s
real 0m5.239s
real 0m5.254s
real 0m8.018s
real 0m8.575s
real 0m7.979s
real 0m6.341s
`Trying with older versions
`
$ op --version
2.12.0$ op --cache=false --debug --vault Private item get --fields username Dummy
username$ for i in {1..20}; do { time op --cache=false --debug --vault Private item get --fields username Dummy 2>&1; } 2>&1 | grep -iE "(cache|real)"; done
real 0m5.244s
real 0m5.244s
real 0m4.987s
real 0m5.020s
real 0m5.137s
real 0m5.024s
real 0m7.752s
real 0m5.139s
real 0m7.059s
real 0m5.527s
real 0m5.266s
real 0m5.499s
real 0m5.236s
real 0m7.763s
real 0m5.222s
real 0m5.206s
real 0m4.943s
real 0m5.209s
real 0m5.234s
real 0m5.529s$ op --version
2.8.0$ for i in {1..20}; do { time op --cache=false --debug --vault Private item get --fields username Dummy 2>&1; } 2>&1 | grep -iE "(cache|real)"; done
real 0m2.500s
real 0m5.189s
real 0m5.313s
real 0m5.075s
real 0m5.227s
real 0m7.885s
real 0m5.124s
real 0m5.068s
real 0m5.266s
real 0m5.140s
real 0m5.170s
real 0m5.014s
real 0m7.668s
real 0m4.936s
real 0m6.755s
real 0m5.730s
real 0m6.447s
real 0m5.086s
real 0m5.055s
real 0m7.927s`
Showing the interrupted execution speed up
`
$ { time op --cache=false --debug --vault Private item get --fields username Dummy 2>&1; } 2>&1real 0m0.994s
user 0m0.000s
sys 0m0.000s$ { time op --cache=false --debug --vault Private item get --fields username Dummy 2>&1; } 2>&1
usernamereal 0m2.304s
user 0m0.000s
sys 0m0.000s
`0 -
Oh, also for completeness, I tried the 32bit version and it was (surprisingly) consistently around 1-200ms slower
0 -
Sorry for the repeated posts (and the iffy formatting - the UI buttons don't give the full range of what appear to be Markdown options) ... just had the idea to run on WSL2 on the same machine. Using my legacy version v2.4 it was consistently in the 2.1-2.5s range. Still slow but half the time of Windows.
Notably, I could not replicate the Ctrl-C behaviour from Windows so it seems like it's skipping that first ~2.6s execution altogether.
Upgrading to 2.13.1 the performance is much worse for non-cached (both the first entry and when running with
--cached=false), and much better for cached results. It was in the 3.1-3.5s range for non-cached results, but ~1.3s for cached after the first run.real 0m1.331s user 0m0.145s sys 0m0.094s
Honestly, that's still really slow for a local, cached, CLI, but I'd kill for that on Windows compared to >5s.
0