SCIM deploy using a single Docker container (without Redis)

edited July 22 in SCIM Bridge

Hi,

I'm trying to deploy SCIM bridge in a docker container over EC2 AWS instance.
I don't want to use Redis, because I have a valid certificate for my domain in AWS Certificate Manager. This certificate is assigned in my public load balancer.

I wrote following docker-compose file:

`version: "3"

services:
scim:
image: 1password/scim:v1.4.3
container_name: 1password-scim
restart: always
entrypoint: ["/op-scim/op-scim"]
env_file: scim-base64.env
ports:
- 3002:3002`

QUESTIONS:

  • Do I need expose port 3002 in my EC2 instance for incoming traffic from load balancer?
  • Do I need expose port 3002 in load balancer to public internet?
  • Do I need a trafic redirection from 443 to 3002 port?
  • Ports 80 and 443 also should be opened in load balancer and server if I'n not using Redis?

1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Sync Type: Not Provided

Comments

  • graham_1Pgraham_1P

    Team Member

    Hi @leonardopech,

    The redis container is a hard dependency for the SCIM Bridge. It does not just hold the TLS certificate. It also holds persistent log files as well as ExternalID mappings for AzureAD.

    Assuming you were not using AzureAD, it is in theory possible for us to modify the code to make it detachable. However that would be not a small undertaking. What is your rationale for not wanting to use the redis container?

    To answer your questions explicity:

    Do I need expose port 3002 in my EC2 instance for incoming traffic from load balancer? Yes. By default the SCIM Bridge listens on this port for unsecured traffic when running without LetsEncrypt.
    Do I need expose port 3002 in load balancer to public internet? No. Terminate TLS connections at the load balancer, and route the resulting unsecured traffic through your network to the SCIM Bridge on port 3002.
    Do I need a trafic redirection from 443 to 3002 port? Yes. You have the right idea.
    Ports 80 and 443 also should be opened in load balancer and server if I'n not using Redis? If we were able to detach it somehow, no ports would change. The service would still receive traffic on 443.

    Let me know what questions you have.

    Graham

  • Perfect!!

    My final docker-compose file was:

    `version: "3"

    services:
    scim:
    image: 1password/scim:v1.4.3
    container_name: 1password-scim
    restart: always
    depends_on:
    - redis
    ports:
    - 3002:3002
    entrypoint: ["/op-scim/op-scim"]
    env_file: scim-base64.env
    redis:
    image: redis:latest
    container_name: 1password-redis
    restart: always`

    Load balancer receive traffic in 443 and is redirected to porto 3002 in my EC2 instance and is also redirected to port 3002 in SCIM container. It works so good!

  • ag_anaag_ana

    Team Member

    On behalf of graham_1P, you are welcome @leonardopech! It's great to hear :)

    If you have any other questions, please feel free to reach out anytime.

    Have a wonderful day :)

  • I would like to hop on this feature request, I was actually wondering about this as well. Here are my thoughts and requirements:

    Given that I want to run the bride on AWS with ALB and fargate, with a SSL cert issued by AWS.
    No AzureAD if that makes a difference.

    • Will you log everything to stdout so I can just use my default log setup to store the logs (required for compliance and security)
    • Do you actually need the reddis instance to be persistent and backed up? Is there any data that must not be lost?
    • Do you say you store log data in reddis? Do you rotate logs or will it just be full at some point?

    In general I would really prefer a simple docker image here with no hard requirements on a specific reddis container unless you really need it. At least on AWS running reddis in a container is an anti-pattern if you need a durable data storage and backup.

    And also the less moving parts the better, right? In the end this is a critical piece of infrastructure.

    Hope I could explain my case, looking forward to your suggestions,

    Jan

  • graham_1Pgraham_1P

    Team Member

    Hi @jan222,

    Thanks for explaining your use case so thoroughly! It helps us understand the why behind your request.

    I'll answer your questions explicitly first and then talk a bit more generally after.

    Will you log everything to stdout so I can just use my default log setup to store the logs (required for compliance and security)

    Yes we already do.

    Do you actually need the reddis instance to be persistent and backed up? Is there any data that must not be lost?

    No, not at all; the contained redis instance is entirely ephemeral. In all cases, everything we put in there can be regenerated via the Identity provider or LetsEncrypt. For your case, that would just be the log data, the loss of which will not impact the functionality of the SCIM Bridge.

    Do you say you store log data in reddis? Do you rotate logs or will it just be full at some point?

    Yes we persist a few days of logs in redis. By default, that is 3 days. You can configure that time period with the argument of --expire-cached-logs=n where n must be an integer value. If you set it to 0 or less no logs will be kept. In this case, you should set up a log capture mechanism on stdout.

    To speak more generally, we completely agree about the critical nature of the SCIM Bridge to a stable identity configuration, and accordingly reliability is one of our top priorities. For that reason we designed the SCIM Bridge to be able to go down completely and then automatically recover when deployed in a kubernetes cluster. I also agree fewer moving parts means fewer opportunities for failure. That being said, we do use the redis instance in very simple way where it is a helpful aid to make our work with the IDPs faster. Fewer moving parts do mean fewer failure points, but it also means less resiliency for the more error-prone portions of the application.

    Quite frankly I'm not yet convinced removing redis would improve the reliability of the SCIM Bridge. However I'll keep an open mind and an open eye to how we may be able to detach redis in the future and how that could improve reliability.

    Perhaps we can add a running mode which is predicated on the assumption the SCIM Bridge is behind a load balancer, is in a kubernetes cluster, and already has a log capture mechanism hooked up. There would be fewer edge cases to worry about in that scenario. I'll make a note and continue to consider it into the future.

    One other item to note: We do not lock to a specific redis version. Instead we just retrieve the latest version (https://github.com/1Password/scim-examples/blob/master/kubernetes/redis-deployment.yaml#L17). You can find similar references in our other downstream applications such as our Helm chart: https://github.com/1Password/op-scim-helm/blob/main/op-scim-bridge/values.yaml#L4

    Graham

  • Hi Graham, thanks so much for your feedback.

    It is really good to hear that the redis store can be ephemeral. I would suggest to add this to the documentation. I was actually worried it keeps some kind of mapping of the users and if lost nobody can use 1password anymore.

    I think for an ephemeral it is fine to just keep it and also easy enough to setup with any of the docker orchestration tools. So no need to remove it from my side.

    Also great that we don't have to rely on redis as log storage but can just capture stdout. That makes it really easy to integrate.

  • graham_1Pgraham_1P

    Team Member

    I'm very glad to hear that satisfied all of your concerns! I'll take a look and see where we can add that information to the documentation.

    Let me know if you have any follow up questions!

    Graham

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file