Unable to login to SCIM Bridge [GCP]

huyngo

Hello! We recently set up a 1Password SCIM Bridge on Google Cloud Platform (GCP) through marketplace. It had been working fine up until today, when the bearer token no longer worked. The error we're getting is:

Error occured when logging in.
failed to create session

We have health monitoring turned on, and the status is good and actively checking. However, new users being added to a managed group aren't being added to 1Password and, as mentioned, we also can't login to the SCIM bridge to look at logs/syncs.

---

1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Browser:_ Not Provided

laz.h_1P

Hi huyngo, my apologies for the late response. I'm Laz, a developer on the SCIM bridge. Its very unusual for the bearer token to suddenly stop working. Are you the only administrator of the account? If not, its possible that someone else went onto the account and accidentally invalidated / regenerated the bearer token. Regardless, if you go into the integration page you are able to regenerate a new bearer token and then configure it on your IdP and then swap out the token on the bridge. Sorry to hear that this has happened to you! I wish I could offer more direct advise, but as you said its hard to determine anything 100% without seeing the logs

timchambers

We've also encountered this a couple times now, running 2.6.2. The first time this happened, I ended up performing a complete redeploy on GCP to get this working again with Google Workspace. This has now happened again. This usually manifests first when I noticed that group changes haven't synced correctly to 1Password, then I attempt to sign into the SCIM bridge to manually sync things, except that I cannot because of this "failed to create session" error.

I'm now attempting to regenerate credentials. Could you provide documentation on the specific steps you mentioned: "and then configure it on your IdP and then swap out the token on the bridge"?

laz.h_1P

After some more digging into this and speaking to colleagues there may be an alternative, simpler workaround (until we get in a permanent fix for this, of course). If you are able go into the bridge UI and disable Workspace (this can be done by removing your credentials file or configuration), then re-enable it by replacing the same data it will cause the Workspace server to restart and likely fix the issue.

If this doesn't work, the specific documentation can be found in the SCIM troubleshooting guide. Hope this helps!

timchambers

@laz.h_1P For what it's worth, when in this state, it's not possible to sign into the SCIM bridge at all. I ended up removing and recreating the cluster and redeploying the SCIM bridge to get things working again—I'm getting pretty good at it due to this issue. Do you anticipate a fix to this issue so that we can reliably use the SCIM bridge going forward? It's hard to justify paying the ongoing expense of a cluster that sits unused an not syncing our users and groups after some unpredictable period of time.

laz.h_1P

@timchambers I'm very sorry to hear about the continual problems we're having. There's a clear issue with our current deployment and we are working to get a permanent fix release ASAP. There is something that we suspect is causing this issue related to filesystem permissions. Since I can't see your logs, its impossible for me to be 100% certain that this is the same problem. Regardless, you can try this fix below:

# Upgrade to 2.6.2 (if you need to)
kubectl set image deploy/op-scim-bridge-1 op-scim-bridge-1=1password/scim:v2.6.2

# Patch initContainer command args
kubectl patch deploy/op-scim-bridge-1 -p='{"spec":{"template":{"spec":{"initContainers":[{"name":"opuser-home-permissions","args":["mkdir -p /home/opuser/.op && chown -R 999 /home/opuser && chmod 700 /home/opuser && chmod 700 /home/opuser/.op && umask 177 /home/opuser/.op"]}]}}}}'

# Change existing config file permissions
kubectl exec -it deploy/op-scim-bridge-1 -c op-scim-bridge-1 -- chmod 600 /home/opuser/.op/config

If after running the commands, you're still having trouble with the SCIM bridge, download your SCIM bridge logs following the instructions here (doesn't require your bearer token): https://support.1password.com/cs/scim-logs/

Attach them in a message to businesssupport@1password.com, and my colleagues will be able to take a closer look. We're currently tracking this problem for other customers and would love to give direct help as it will also help us permanently resolve the issue faster.

Thanks a ton for your patience, and our apologies again for the broken version.

chravtacque

Hi there,

Same issue. Got the error 'cannot read config at \"/home/opuser/.op/config\": Can't continue. We can't safely access \"/home/opuser/.op/config\" because its permissions are too broad. Change its permissions to 600 and try again.'

I was able to resolve the issue by issuing 'chmod 600 /home/opuser/.op/config' inside the container, however, the change is not persistent.

laz.h_1P

Hi @chravtacque,

Thanks for writing in. Sorry to hear that you are also experiencing this issue. For now, the best that I can suggest is the temporary workaround as you found. We have identified the root issue and will be including a permanent fix in the next release. Thank you for your continued patience

huyngo

Hi, sorry for the late response! Thanks for following up. We ended up doing the same thing as @timchambers, completely redeployed a new cluster and scim bridge.

Right now we're running into a problem where we can still log into the scim bridge and everything looks fine, but user provisioning isn't happening in 1Password. The scim bridge is working, and health monitoring for the Google Workspace integration is good, but the managed groups aren't provisioning new users.

@laz.h_1P any advice on this?

Jack.P_1P

Hi @huyngo:

In that case, reaching out to us directly via businesssupport@1password.com would be your best bet. Include logs from your SCIM bridge, and we'll be able to take a closer look.

https://support.1password.com/cs/scim-logs/

Jack

huyngo

@Jack.P_1P Ahh, thanks for the quick response, Jack! We managed to fix it by going into the scim bridge, then deleting the credentials json and reuploading after looking through the logs and seeing this error:

{"level":"error","version":"2.6.2","build":"206022","application":"op-scim","request_id":"cdmnvg4hhkfl5rpsrnkg","error":"Server: (failed to GetCurrentUserWithGroupMemberships), Wrapped: (failed to Account.GetInfo), session is no longer valid, or missing credentials to authenticate","time":"2022-11-18T23:13:59Z","message":"failed to SyncGroups"}

We figured that deleting/readding the JSON would trigger a session refresh and it did. Not sure if sessions should be refreshed automatically or not, but we'll just periodically refresh it manually/whenever the problem shows up again.

Jack.P_1P

Hi @huyngo:

You're very welcome! We're investigating this behavior, so please do reach out directly if you run into trouble again so we can take a closer look.

Jack

chravtacque

@Jack.P_1P Any ideas when the new release will be available?

Jack.P_1P

Hi @chravtacque:

Version 2.7.0 of 1Password SCIM Bridge contains the fix for the issue my colleague Laz referred to earlier. Let me know if you're still having trouble with the SCIM bridge, and I'll be able to take a closer look.

Jack

huyngo

@Jack.P_1P Thank you! Going to upgrade now. Incidentally, we lost access to the SCIM bridge dashboard again (says the bearer token is no longer valid), so perfect opportunity to upgrade. I'll update if anything looks off afterwards.

tnam10

Hey @Jack.P_1P ,

I'm having a similar issue to the post above. The Bridge logs show the error as:
{"level":"error","version":"2.7.0","build":"207001","application":"op-scim","request_id":"ces89eq03vucv0bl270g","error":"Server: (failed to GetCurrentUserWithGroupMemberships), Wrapped: (failed to Account.GetInfo), session is no longer valid, or missing credentials to authenticate","time":"2023-01-06T21:28:51Z","message":"failed to SyncGroups"}

I set up our Bridge SCIM configuration using GCP this morning and was successful (or so I thought). After about 10-20 min it would stop syncing. Following @huyngo advice, I also deleted/recreated the JSON key, which got it working again for another 30 min or so before it stopped syncing with the same error. Could you please assist me with this?

I sent an email to businesssupport@1password.com and included the Bridge logs, GCP logs as well as a link to this page (+ my username) just now.

Thank you for the help!

Tenzin

Jack.P_1P

Hi @tnam10:

Thanks for emailing everything in! We'll be in touch soon, so keep an eye out.

Jack