gitops-connector pod in CrashLoopBackoff state after executing helm upgrade

[colinrippeyfinarne]

HI,

I have an OKdo ROCK 4 SE running Ubuntu. I've installed Rancher k3s and connected the cluster to Arc.

I've successfully executed the first set of steps from this tutorial to get a cluster-config GitOps up and running and happy that I can commit changes to the namespaces.yaml and see the new namespaces deployed to the cluster:

https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/tutorial-gitops-flux2-ci-cd#connect-the-gitops-repository-1

I've then tried to follow the next set of steps to get the GitOps connector installed:

https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/tutorial-gitops-flux2-ci-cd#install-gitops-connector-1

After running the helm upgrade step when I do a check of the pods using kubectl get pods -n flux-system I see:

NAME READY STATUS RESTARTS AGE
notification-controller-df5b79cc5-v5vtl 1/1 Running 0 137m
source-controller-758ffb89cf-bbchm 1/1 Running 0 137m
helm-controller-65d59c8c87-vzt9k 1/1 Running 0 137m
kustomize-controller-64f6855bbc-jvm46 1/1 Running 0 137m
fluxconfig-controller-769fd4df9d-gq2g9 2/2 Running 0 137m
fluxconfig-agent-54648b6887-9ztfn 2/2 Running 0 137m
gitops-connector-76f779b899-5nc7w 0/1 CrashLoopBackOff 25 (56s ago) 104m

If I run kubectl logs -l=app=gitops-connector -n flux-system -f I see:

exec /usr/bin/sh: exec format error

If I run kubectl describe pod gitops-connector-76f779b899-5nc7w -n flux-system I see:

Name: gitops-connector-76f779b899-5nc7w
Namespace: flux-system
Priority: 0
Service Account: default
Node: rock4se/10.0.0.40
Start Time: Mon, 14 Aug 2023 09:36:22 +0100
Labels: app=gitops-connector
pod-template-hash=76f779b899
Annotations:
Status: Running
IP: 10.42.0.43
IPs:
IP: 10.42.0.43
Controlled By: ReplicaSet/gitops-connector-76f779b899
Containers:
connector:
Container ID: containerd://1f3bf4a836d5944f4e332d2805fa90b8a7ba2a1215bd70b27ff31dc3f58cd966
Image: ghcr.io/azure/gitops-connector:1.2.1
Image ID: ghcr.io/azure/gitops-connector@sha256:d120265b9a8d68b5759e8415da49e88e2a3b389898b1f1f764e70d40179b9917
Port: 8080/TCP
Host Port: 0/TCP
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Mon, 14 Aug 2023 11:20:08 +0100
Finished: Mon, 14 Aug 2023 11:20:08 +0100
Ready: False
Restart Count: 25
Environment:
GIT_REPOSITORY_TYPE: <set to the key 'GIT_REPOSITORY_TYPE' of config map 'gitops-connector-cm'> Optional: false
CICD_ORCHESTRATOR_TYPE: <set to the key 'CICD_ORCHESTRATOR_TYPE' of config map 'gitops-connector-cm'> Optional: false
GITOPS_OPERATOR_TYPE: <set to the key 'GITOPS_OPERATOR_TYPE' of config map 'gitops-connector-cm'> Optional: false
GITOPS_APP_URL: <set to the key 'GITOPS_APP_URL' of config map 'gitops-connector-cm'> Optional: false
GITHUB_GITOPS_REPO_NAME: <set to the key 'GITHUB_GITOPS_REPO_NAME' of config map 'gitops-connector-cm'> Optional: false
GITHUB_GITOPS_MANIFEST_REPO_NAME: <set to the key 'GITHUB_GITOPS_MANIFEST_REPO_NAME' of config map 'gitops-connector-cm'> Optional: false
GITHUB_ORG_URL: <set to the key 'GITHUB_ORG_URL' of config map 'gitops-connector-cm'> Optional: false
PAT: <set to the key 'PAT' in secret 'gitops-connector-secret'> Optional: false
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-xnrf5 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
kube-api-access-xnrf5:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional:
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors:
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message

---

Warning BackOff 94s (x481 over 106m) kubelet Back-off restarting failed container connector in pod gitops-connector-76f779b899-5nc7w_flux-system(9a6fe095-0be6-493e-981d-78f3218d61bb)