Cannot override runAsUser on azwi-proxy

[ldardick]

Describe the bug
The admission webhook injects azwi-proxy as a sidecar container. The injected container defines a securityContext that defines that the container will run as non-root.

      securityContext:
        capabilities:
          drop:
            - ALL
        privileged: false
        runAsNonRoot: true
        readOnlyRootFilesystem: true
        allowPrivilegeEscalation: false

However, in some scenarios this may violate the existing non-root policy defined by the pod causing the sidecar container not being able to progress.

containerStatuses:
    - name: azwi-proxy
      state:
        waiting:
          reason: CreateContainerConfigError
          message: >-
            container's runAsUser breaks non-root policy (pod:
            "[redacted]([redacted])",
            container: azwi-proxy)

Version 0.15 (that we were using prior to the release of 1.10) didn't have the securityContext defined

Steps To Reproduce
[edited to add repro steps that doesn't depends on any third party chart]

1. Install version 1.1.0 of the admission webhook
2. Create a Pod and a Service Account with the following config:

apiVersion: v1
kind: Pod
metadata:
  name: static-web
  namespace: test-wi-proxy
  labels:
    azure.workload.identity/use: 'true'
  annotations:
    azure.workload.identity/inject-proxy-sidecar: 'true'
spec:
  containers:
    - name: web
      image: nginx
  securityContext:
    runAsUser: 0
    fsGroup: 65534
  serviceAccountName: my-app-kubesa

---

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app-kubesa
  namespace: test-wi-proxy
  labels:
    azure.workload.identity/use: 'true'
  annotations:
    azure.workload.identity/client-id: [redacted]

Expected behavior
The admission webhook chart should allow configuring if the sidecar proxy will run as root or not.

Logs
N/A, the container won't be able to progress

Environment

• Kubernetes version (use kubectl version): 1.25.6
• Cloud provider or hardware configuration: AKS
• OS (e.g: cat /etc/os-release): Ubuntu 22.04.2 LTS
• Kernel (e.g. uname -a): 5.15.0-1039-azure
• Install tools: N/A
• Network plugin and version (if this is a network-related bug): Azure CNI
• Others: N/A

Additional context
N/A

[ldardick]

Edited the original repro steps (which explained how to replicate using ExternalDNS) to replace them with steps that doesn't depend on a third party chart and can be tested isolated.

[SaiVamsi95]

@ldardick FYI:
The same issue is happening with Velero deployment as well. Unable to store PVC data with restic when using azure workload identity to access Azure storage account.

[ldardick]

@ldardick FYI: The same issue is happening with Velero deployment as well. Unable to store PVC data with restic when using azure workload identity to access Azure storage account.

For what it's worth, if you downgrade to 1.0.0, that version doesn't have the security context hardcoded and works

[kengibous]

@SaiVamsi95

FWIW i got my node agent pods to start by adding the following to my values.yaml

nodeAgent:  
  podSecurityContext:
    runAsUser: 1000
    runAsNonRoot: true