Problems with first time kube setup
I am working to get the automation set up in our kube environment at work. We are using the helm chart here:
https://github.com/1Password/connect-helm-charts/tree/main/charts/connect
After working through initial problems I have the onepassword-connect and onepassword-connect-operator pod in healthy running state and the operator pod scanning all my namespaces for the 1Password Annotations. For several days I had an issue with the initial set up of the Connect sever because of a permissions issue where I could not add vaults to the connect server but now our corp admin has helped and everything looks correct in the web gui for the the integration with respect to the access token and the required vault having read,write access.
My problem is that I do not think my connect server is making the connection to 1password and I have added an annotation and it is failing. In the web gui when I look at my secrets automation is says at the top "Your Connect server hasn't authenticated with 1Password yet"
When our admin created this he saved the json and token off to our shared vault in 1passwd and I see those values in the secrets in my pods (we use hashicorp vault to load sensitive data today and that looks like it is working correctly)
Well, I just found a big clue while typing this. I see this error in the logs for the connect-sync container in the onepassword-connect pod:
[bpeatman@mgmt001 ~]$ [k8s:ine2-aks-tools-admin] ~ $ k logs -n 1password -c connect-sync onepassword-connect-bdb4df46-dgn5p | head -40
{"log_message":"(I) disabling bus peer auto-discovery","timestamp":"2023-02-17T17:45:40.21623277Z","level":3}
{"log_message":"(W) did not initialize bus connection to peer localhost:11220. If the peer is currently booting, it may initialize the connection while starting. Details: [grav] failed to transport.CreateConnection: [transport-websocket] failed to Dial endpoint: dial tcp [::1]:11220: connect: connection refused. ","timestamp":"2023-02-17T17:45:40.216810693Z","level":2}
{"log_message":"(I) starting 1Password Connect Sync ...","timestamp":"2023-02-17T17:45:40.2170009Z","level":3}
{"log_message":"(I) no existing database found, will initialize at /home/opuser/.op/data/1password.sqlite","timestamp":"2023-02-17T17:45:40.217207608Z","level":3}
{"log_message":"(I) database initialization complete","timestamp":"2023-02-17T17:45:40.221409669Z","level":3}
{"log_message":"(I) ### syncer credentials bootstrap ### ","timestamp":"2023-02-17T17:45:40.221596076Z","level":3}
{"log_message":"(E) Server: (unable to get credentials and initialize API, retrying in 500ms), Wrapped: (failed to FindCredentialsUniqueKey), Wrapped: (failed to loadCredentialsFile), Wrapped: (LoadLocalAuthV2 failed to credentialsDataFromBase64), illegal base64 data at input byte 0","timestamp":"2023-02-17T17:45:40.221636977Z","level":1}
So I think even though I have the ENV variable with the correct json in that connect-api container there is a problem with it somehow.
It is:
OP_SESSION : secretKeyRef(op-credentials.1password-credentials.json) and I see the correct data in that secret.
I see the forum strips leading white space but this yaml still may be helpful, it shows that I am accepting defaults for almost everything
this is my yaml for the application deployment with the helm chart:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
annotations:
argocd.argoproj.io/sync-wave: "2"
finalizers:
- resources-finalizer.argocd.argoproj.io
name: 1password
namespace: argo-cd
spec:
destination:
namespace: 1password
server: 'https://kubernetes.default.svc'
project: default
source:
chart: connect
helm:
releaseName: 1password
values: |
connect:
version: 1.6.1
operator:
create: true
autoRestart: true
version: 1.6.0
repoURL: 'https://1password.github.io/connect-helm-charts/'
targetRevision: "*"
syncPolicy:
automated:
prune: true
selfHeal: true
Thanks in advance for any help. I think I am getting close and this is very exciting :)
I think maybe I need to convert my secret into base64 and I saw docs on how to do that.
1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Browser:_ Not Provided