1Password SCIM Bridge: Incorrect bearer token

nrose
nrose
Community Member
edited February 2021 in SCIM Bridge

I'm trying to deploy the SCIM Bridge as an Azure Kubernetes service following these instructions: https://support.1password.com/scim-deploy-azure .

  • Every time I get to the Step 4: Test the SCIM bridge, it kicks back the error {"detail":"failed to create session","schemas":["urn:ietf:params:scim:api:messages:2.0:Error"]}.
  • When I attempt to test directly at the DNS URL, I see the 1Password SCIM Bridge Login requesting Enter your OAuth bearer token:, it responds Incorrect bearer token.

I've confirmed it's successfully applied a Let's Encrypt SSL certificate.

I've regenerated the credentials (3) times, each time deleting the scimsession 'secret' and replacing it with the newly generated one.

Any ideas what I am doing wrong here?

[EDIT]: I don't think this is relevant, but I should note this is a brand new deployment still under a 'Business' trial.

Comments

  • 1P_Amanda
    1P_Amanda

    Hi @nrose!

    The first place to check is your SCIM Bridge logs - what are they showing?

    When you run kubectl get secrets does it show something like

    NAME TYPE DATA AGE
    scimsession Opaque 1 11m

    with the age you're expecting?

    When you update the scimsession file, are you ensuring that you use the new updated bearer token as well?

    Let me know!
    Amanda

  • nrose
    nrose
    Community Member

    Hey @1P_Amanda , appreciate the response. I see the scimsession secret listed.

    NAME TYPE DATA AGE
    default-token-fz8pw kubernetes.io/service-account-token 3 44h
    scimsession Opaque 1 40h

    ...and yes, each time I've started over, I've intentionally deleted out the 1Password secret saved in to my vault, deleted the downloaded scimsession file, and made sure to use the new ones provided. I kept second guessing myself so I went through this process 4 times before giving up for the day.

  • nrose
    nrose
    Community Member

    @1P_Amanda - How do I access the SCIM Bridge logs? I'm not seeing where those can be pulled from.

  • nrose
    nrose
    Community Member

    @1P_Amanda - I just took a lame guess at /status in front of the SCIM URL and now see the status page. At the bottom, it says:

    "Could not retrieve log files. Make sure redis is running and try again."

    Everything else is in an error state.

  • ag_alice_t
    ag_alice_t

    Hi there @nrose - could you check to make sure that "Cloud Providers" is not part of the "denied" group in your 1Password Advanced Protection Firewall? That would be under Security -> Firewall settings.

  • nrose
    nrose
    Community Member

    @ag_alice_t -- I did. (It originally was, but now it's not). Either way, I wouldn't think that would cause an issue accepting the token though. BUT yes - it's been removed.

  • 1P_Amanda
    1P_Amanda

    In order to get the logs from the status page, you need to authenticate using the bearer token from the SCIM Bridge landing page. From the command line you can run kubectl get pods, grab the name of your SCIM Bridge pod, then run kubectl logs pod/op-scim-bridge-<id> using the name you just grabbed, and that will output the SCIM Bridge logs.

  • nrose
    nrose
    Community Member

    @1P_Amanda : I feel as though it's trying to tell me something. heh

    Entire logs are lit up with:
    2021/02/08 18:19:13 http: TLS handshake error from 10.244.0.1:50781: EOF
    2021/02/08 18:19:19 http: TLS handshake error from 10.244.0.1:4658: EOF
    2021/02/08 18:19:25 http: TLS handshake error from 10.244.0.1:33543: EOF
    2021/02/08 18:19:31 http: TLS handshake error from 10.244.0.1:2303: EOF
    2021/02/08 18:19:37 http: TLS handshake error from 10.244.0.1:8804: EOF

  • 1P_Amanda
    1P_Amanda

    Unfortunately, that's not a real error - azure's health checker hits the SCIM Bridge via the IP address instead of the url, which causes that to show up in the logs. Somewhere in all those lines there should be something more insightful.

  • nrose
    nrose
    Community Member

    @1P_Amanda : Only other things I see are:

    [LOG] [1.6.1] 2021/02/08 16:12:48 (ERROR) AuthWrap failed to check session, trying to generateNewSession: failed to touch session: session not initialized
    [LOG] [1.6.1] 2021/02/08 16:12:48 (INFO) session invalid, generating new session
    [LOG] [1.6.1] 2021/02/08 16:12:49 (ERROR) generateNewSession failed to PerformAuthWithSRPCredentials: This 1Password application does not support Duo authentication
    [LOG] [1.6.1] 2021/02/08 16:12:49 (ERROR) AuthWrap failed to generateNewSession: This 1Password application does not support Duo authentication
    [LOG] [1.6.1] 2021/02/08 16:12:49 (INFO) GET /login/session from 10.244.0.1:34521 - 500 (Internal Server Error) in 929ms

    [LOG] [1.6.1] 2021/02/08 17:01:25 (INFO) GET /status from 10.244.0.1:34521 - 200 (OK) in 0ms
    [LOG] [1.6.1] 2021/02/08 17:01:25 (INFO) GET /static/common/common.css from 10.244.0.1:34521 - 200 (OK) in 0ms
    [LOG] [1.6.1] 2021/02/08 17:01:25 (INFO) GET /static/status/status.css from 10.244.0.1:34521 - 200 (OK) in 0ms
    [LOG] [1.6.1] 2021/02/08 17:01:25 (INFO) GET /static/img/scim-bridge.svg from 10.244.0.1:34521 - 200 (OK) in 0ms
    [LOG] [1.6.1] 2021/02/08 17:01:25 (INFO) GET /static/img/healthy.svg from 10.244.0.1:34521 - 200 (OK) in 0ms
    [LOG] [1.6.1] 2021/02/08 17:01:25 (INFO) GET /static/status/status.js from 10.244.0.1:34521 - 200 (OK) in 0ms
    [LOG] [1.6.1] 2021/02/08 17:01:25 (INFO) GET /static/img/unhealthy.svg from 10.244.0.1:34521 - 200 (OK) in 0ms
    [LOG] [1.6.1] 2021/02/08 17:01:25 (INFO) GET /static/img/unknown.svg from 10.244.0.1:34521 - 200 (OK) in 0ms
    [LOG] [1.6.1] 2021/02/08 17:01:25 (INFO) GET /static/img/favicon.ico from 10.244.0.1:34521 - 200 (OK) in 0ms
    [LOG] [1.6.1] 2021/02/08 17:01:25 (INFO) GET /static/img/favicon.svg from 10.244.0.1:34521 - 200 (OK) in 0ms

    [LOG] [1.6.1] 2021/02/08 18:02:58 (INFO) Health Service Reports:
    [LOG] [1.6.1] 2021/02/08 18:02:58 (INFO) {ChallengeServer 2021-02-08 17:35:59.65845115 +0000 UTC m=+149580.921137147 2021-02-08 17:35:59.65845115 +0000 UTC m=+149580.921137147 healthy}
    [LOG] [1.6.1] 2021/02/08 18:02:58 (INFO) {ProvisionWatcher 2021-02-07 00:02:58.743638828 +0000 UTC m=+0.006324725 2021-02-07 00:12:58.743638828 +0000 UTC m=+600.006324725 unknown}
    [LOG] [1.6.1] 2021/02/08 18:02:58 (INFO) {RedisCache 2021-02-08 18:02:58.922591434 +0000 UTC m=+151200.185277331 2021-02-08 18:02:58.922591434 +0000 UTC m=+151200.185277331 healthy}
    [LOG] [1.6.1] 2021/02/08 18:02:58 (INFO) {SCIMServer 2021-02-08 17:36:00.708466243 +0000 UTC m=+149581.971152140 2021-02-08 17:36:00.708466243 +0000 UTC m=+149581.971152140 healthy}

    I also saw just a few of these but they were very sporadic, mixed in amongst the others:

    2021/02/08 18:44:57 http: TLS handshake error from 10.244.0.1:16796: acme/autocert: missing server name
    2021/02/08 18:44:58 http: TLS handshake error from 10.244.0.1:60896: acme/autocert: missing server name
    2021/02/08 18:44:58 http: TLS handshake error from 10.244.0.1:14594: acme/autocert: missing server name
    2021/02/08 18:44:58 http: TLS handshake error from 10.244.0.1:15709: acme/autocert: missing server name
    2021/02/08 18:44:59 http: TLS handshake error from 10.244.0.1:37211: acme/autocert: missing server name
    2021/02/08 18:44:59 http: TLS handshake error from 10.244.0.1:9318: acme/autocert: missing server name
    2021/02/08 18:45:00 http: TLS handshake error from 10.244.0.1:33911: acme/autocert: missing server name
    2021/02/08 18:45:01 http: TLS handshake error from 10.244.0.1:42462: acme/autocert: missing server name

  • 1P_Amanda
    1P_Amanda

    Now we're getting somewhere :) It looks like you have enforced MFA enabled - if you disable it does it work? Also, what do you see when you go to "Provisioning" under settings?

  • nrose
    nrose
    Community Member

    @1P_Amanda : PROGRESS, although I'm confused by it. I disabled Duo and it finally connected, yet very bizarrely -- when I reenabled Duo, it KEPT working. I'm not sure why this would be? Leaves me concerned that the problem will come back. With Duo enabled, how are you supposed to allow the SCIM connection to circumvent MFA?

  • 1P_Amanda
    1P_Amanda

    @nrose YES! I love progress. So, we recently updated how we did provisioning in order to allow people to enforce MFA, and now you're an early adopter (Congratulations!). I think you ended up in a bit of a weird state during the transition, but I don't anticipate the problem coming back. If it does, email us and we will be happy to hop on a call to sort it out.

  • nrose
    nrose
    Community Member

    @1P_Amanda : Fair enough! As long as I'm not falling outside of expected behavior, I can live with that. I think I'll still throttle my MFA time down to 1 day just to make sure the issue doesn't come back in ~24 hours, but assuming it doesn't - I think I'll be comfortable moving forward...

    Thank you!

  • nrose
    nrose
    Community Member

    @1P_Amanda : Already right back to the same place, unfortunately. Getting the Duo authentication error in the logs again.

  • 1P_Amanda
    1P_Amanda

    Dang it! I am going to try and reproduce / crowd source and get back to you.

  • 1P_Amanda
    1P_Amanda

    @nrose - I think I have the answer. Enable "bypass" for that specific user in Duo - add the Provision Manager's automatically generated email into Duo, and then set it to bypass MFA encryption within Duo. Let me know if it works!

  • nrose
    nrose
    Community Member
    edited February 2021

    @1P_Amanda : That's one thing I've been confused about since the very beginning. I keep seeing reference to the Provision Manager account / e-mail, but I don't know what that is or see where to find it. Nowhere in the instructions (that I remember) called for creating an account. I don't see any accounts clearly defined in the logs.

    [EDIT]: Hold. I may have found it in the 1Password portal Activity Logs. Will update after testing.

  • 1P_Amanda
    1P_Amanda

    @nrose - Sorry about that, we used to create a provisioning manager user in the setup of the SCIM Bridge, but recently changed it to a different kind of account and haven't quite updated all our documentation. You're looking for the "Automated User Provisioning" - if you go to the group "Provision Managers" it should be in there. Click on "Automated User Provisioning" and on the left there should be an email associated with them. That's the email you should use for the bypass. You can also go check out "Activity Log" on the right and see if it shows up there.

  • nrose
    nrose
    Community Member

    @1P_Amanda : Pretty sure that is going to have been the fix. After configuring that user for "Bypass" in our Duo Portal, I checked the Duo logs and see three authentications for the last few hours -- meaning it IS hitting Duo, and IS getting passed through correctly. Gonna give it another 24 hours to make sure, but I think we're gonna be solid on this one! (Thank you!)

  • nrose
    nrose
    Community Member

    @1P_Amanda : A day later and I'm still seeing the bypass being used in our Duo audit logs. Provisioning Logs in Azure are still running / updating regularly. Waiting on one final test, but I'm 95% confident that did the trick.

    (Update documentation to reflect recent updates!!!)

  • nrose
    nrose
    Community Member

    Yup -- final test went through. We're solid. Thank you again @1P_Amanda !

  • 1P_Amanda
    1P_Amanda

    Excellent! We've made a note to review the documentation, cheers!

This discussion has been closed.