SCIM deploy using a single Docker container (without Redis)

leonardopech

Hi,

I'm trying to deploy SCIM bridge in a docker container over EC2 AWS instance.
I don't want to use Redis, because I have a valid certificate for my domain in AWS Certificate Manager. This certificate is assigned in my public load balancer.

I wrote following docker-compose file:

`version: "3"

services:
scim:
image: 1password/scim:v1.4.3
container_name: 1password-scim
restart: always
entrypoint: ["/op-scim/op-scim"]
env_file: scim-base64.env
ports:
- 3002:3002`

QUESTIONS:

• Do I need expose port 3002 in my EC2 instance for incoming traffic from load balancer?
• Do I need expose port 3002 in load balancer to public internet?
• Do I need a trafic redirection from 443 to 3002 port?
• Ports 80 and 443 also should be opened in load balancer and server if I'n not using Redis?

---

1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Sync Type: Not Provided

graham_1P

Hi @leonardopech,

The redis container is a hard dependency for the SCIM Bridge. It does not just hold the TLS certificate. It also holds persistent log files as well as ExternalID mappings for AzureAD.

Assuming you were not using AzureAD, it is in theory possible for us to modify the code to make it detachable. However that would be not a small undertaking. What is your rationale for not wanting to use the redis container?

To answer your questions explicity:

Do I need expose port 3002 in my EC2 instance for incoming traffic from load balancer? Yes. By default the SCIM Bridge listens on this port for unsecured traffic when running without LetsEncrypt.
Do I need expose port 3002 in load balancer to public internet? No. Terminate TLS connections at the load balancer, and route the resulting unsecured traffic through your network to the SCIM Bridge on port 3002.
Do I need a trafic redirection from 443 to 3002 port? Yes. You have the right idea.
Ports 80 and 443 also should be opened in load balancer and server if I'n not using Redis? If we were able to detach it somehow, no ports would change. The service would still receive traffic on 443.

Let me know what questions you have.

Graham

leonardopech

Perfect!!

My final docker-compose file was:

`version: "3"

services:
scim:
image: 1password/scim:v1.4.3
container_name: 1password-scim
restart: always
depends_on:
- redis
ports:
- 3002:3002
entrypoint: ["/op-scim/op-scim"]
env_file: scim-base64.env
redis:
image: redis:latest
container_name: 1password-redis
restart: always`

Load balancer receive traffic in 443 and is redirected to porto 3002 in my EC2 instance and is also redirected to port 3002 in SCIM container. It works so good!

ag_ana

On behalf of graham_1P, you are welcome @leonardopech! It's great to hear :)

If you have any other questions, please feel free to reach out anytime.

Have a wonderful day :)

jan222

I would like to hop on this feature request, I was actually wondering about this as well. Here are my thoughts and requirements:

Given that I want to run the bride on AWS with ALB and fargate, with a SSL cert issued by AWS.
No AzureAD if that makes a difference.

• Will you log everything to stdout so I can just use my default log setup to store the logs (required for compliance and security)
• Do you actually need the reddis instance to be persistent and backed up? Is there any data that must not be lost?
• Do you say you store log data in reddis? Do you rotate logs or will it just be full at some point?

In general I would really prefer a simple docker image here with no hard requirements on a specific reddis container unless you really need it. At least on AWS running reddis in a container is an anti-pattern if you need a durable data storage and backup.

And also the less moving parts the better, right? In the end this is a critical piece of infrastructure.

Hope I could explain my case, looking forward to your suggestions,

Jan

graham_1P

Hi @jan222,

Thanks for explaining your use case so thoroughly! It helps us understand the why behind your request.

I'll answer your questions explicitly first and then talk a bit more generally after.

Will you log everything to stdout so I can just use my default log setup to store the logs (required for compliance and security)

Yes we already do.

Do you actually need the reddis instance to be persistent and backed up? Is there any data that must not be lost?

No, not at all; the contained redis instance is entirely ephemeral. In all cases, everything we put in there can be regenerated via the Identity provider or LetsEncrypt. For your case, that would just be the log data, the loss of which will not impact the functionality of the SCIM Bridge.

Do you say you store log data in reddis? Do you rotate logs or will it just be full at some point?

Yes we persist a few days of logs in redis. By default, that is 3 days. You can configure that time period with the argument of --expire-cached-logs=n where n must be an integer value. If you set it to 0 or less no logs will be kept. In this case, you should set up a log capture mechanism on stdout.

To speak more generally, we completely agree about the critical nature of the SCIM Bridge to a stable identity configuration, and accordingly reliability is one of our top priorities. For that reason we designed the SCIM Bridge to be able to go down completely and then automatically recover when deployed in a kubernetes cluster. I also agree fewer moving parts means fewer opportunities for failure. That being said, we do use the redis instance in very simple way where it is a helpful aid to make our work with the IDPs faster. Fewer moving parts do mean fewer failure points, but it also means less resiliency for the more error-prone portions of the application.

Quite frankly I'm not yet convinced removing redis would improve the reliability of the SCIM Bridge. However I'll keep an open mind and an open eye to how we may be able to detach redis in the future and how that could improve reliability.

Perhaps we can add a running mode which is predicated on the assumption the SCIM Bridge is behind a load balancer, is in a kubernetes cluster, and already has a log capture mechanism hooked up. There would be fewer edge cases to worry about in that scenario. I'll make a note and continue to consider it into the future.

One other item to note: We do not lock to a specific redis version. Instead we just retrieve the latest version (https://github.com/1Password/scim-examples/blob/master/kubernetes/redis-deployment.yaml#L17). You can find similar references in our other downstream applications such as our Helm chart: https://github.com/1Password/op-scim-helm/blob/main/op-scim-bridge/values.yaml#L4

Graham

jan222

Hi Graham, thanks so much for your feedback.

It is really good to hear that the redis store can be ephemeral. I would suggest to add this to the documentation. I was actually worried it keeps some kind of mapping of the users and if lost nobody can use 1password anymore.

I think for an ephemeral it is fine to just keep it and also easy enough to setup with any of the docker orchestration tools. So no need to remove it from my side.

Also great that we don't have to rely on redis as log storage but can just capture stdout. That makes it really easy to integrate.

graham_1P

I'm very glad to hear that satisfied all of your concerns! I'll take a look and see where we can add that information to the documentation.

Let me know if you have any follow up questions!

Graham

dzeidzei

Hello,
I've now tried to get 1password scim bridge runnin on AWS ECS Fargate without no success. I have one service with one task containing op-scim and op-redis containers and op-scim is set to be depend on redis. All goes well at first and service is starting and started to register to load balancer's target group, but after a short while service start to terminate and after that restart itselt continuously.

I think that both containers should be able to communicate with each other, like they do in docker desktop where that same combo is workin fine.

Can someone here give me a hint what might be wrong? I'm quite stucked with this.

1P_Amanda

Hi @dzeidzei!

Just to make sure we're on the same page, are you using this to get started or did you spin it up without using a template? https://github.com/1Password/scim-examples/tree/master/aws-ecsfargate-terraform

Let me know!
Amanda

dzeidzei

Hi @1P_Amanda,
I'm using that github template and have build op-redis and op-scim images on Docekr dekstop with same docker-compose.yml which leonardopech defined above and then pushed them to Our AWS ECR.
Accordint to my investications so far, I think the problem is that op-scim and op-redis are unable to comminicate inside ESC Fargate task.
I've got it runnig on EC2 linux instance like leonardopech, but prefer to run this kind of task on ECS Fargate if possible.

1P_Amanda

Do you set the environment variable "OP_REDIS_HOST" to localhost somewhere? Also, just curious, is there a specific reason you're using that docker-compose as opposed to the one in scim.json?

Amanda

dzeidzei

No, I haven't set OP_REDIS_HOST environment variable. I'd like to have scim_bridge running in private subnet with load ballancer with my own certificate in public subnet, and that compose is suitable for that.

What is the method of comminication between op-scim and redis inside docker?

dzeidzei

dzeidzei

Thanks a lot for grat help @1P_Amanda! Setting OP_REDIS_HOST to localhost for op-scim container did the trick and the whole thing is runnig on ECS Fargate now!

Time to celebrate!

1P_Amanda

I'm so glad to hear that, have a great weekend!

Amanda

dzeidzei

Great weekend to You too!

dzeidzei

ag_ana

:)