Setup sealer: cannot fetch certificate: error trying to reach service: Address is not allowed

[rowi1de]

I'm currently facting the following issue:

2023/05/09 07:22:56 Connection to sealed secrets with (Namespace: infrastructure / ServiceName: sealed-secrets)
2023/05/09 07:22:56 Setup sealer: cannot fetch certificate: error trying to reach service: Address is not allowed

• sealed-secrets 2.8.1
  with:

 repoURL: 'https://bitnami-labs.github.io/sealed-secrets'
  targetRevision: 2.8.1
  helm:
    values: |-
      ingress:
        enabled: true
        hostname: sealed-secrets.something.com
        pathType: Prefix
        ingressClassName: "nginx"
        annotations:
          nginx.ingress.kubernetes.io/auth-type: basic
          # name of the secret that contains the user/password definitions
          nginx.ingress.kubernetes.io/auth-secret: sealed-secrets-cert-basic-auth
          # message to display with an appropriate context why the authentication is required
          nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - sealed-secrets-cert'

destination:
  server: 'https://kubernetes.default.svc'
  namespace: infrastructure
syncPolicy:
  automated:
    prune: true
    selfHeal: true
  syncOptions:
    - CreateNamespace=true

• sealed-secrets-web 3.0.7

project: infrastructure
source:
  repoURL: 'https://charts.bakito.net'
  targetRevision: 3.0.*
  helm:
    values: |
      sealedSecrets:
        namespace: infrastructure
        serviceName: sealed-secrets
      env:
        sealedSecretsControllerNamespace: infrastructure
      disableLoadSecrets: true
  chart: sealed-secrets-web
destination:
  server: 'https://kubernetes.default.svc'
  namespace: infrastructure
syncPolicy:
  automated:
    prune: true
    selfHeal: true
  syncOptions:
    - CreateNamespace=true

I can't use the certURL because of #141
Anyway using cluster internal connection would be preferred.

The namespace & service mentioned in the error exist Namespace: infrastructure / ServiceName: sealed-secrets
any hints for further debugging welcome

[bakito]

Hi @rowi1de

Can you please verify that the k8s service name of your sealed-secrets installation is sealed-secrets if not, you might need to set the sealedSecrets.serviceName property in the helm chart.

[rowi1de]

@bakito yes :)

 kubectl get service sealed-secrets -n infrastructure
NAME             TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)           AGE
sealed-secrets   ClusterIP   10.100.216.31   <none>        8080/TCP,80/TCP   33d

kubectl get service sealed-secrets -n infrastructure -o yaml

apiVersion: v1
kind: Service
metadata:
  annotations:
  creationTimestamp: "2023-04-06T12:54:54Z"
  labels:
    app.kubernetes.io/instance: sealed-secrets
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: sealed-secrets
    app.kubernetes.io/part-of: sealed-secrets
    app.kubernetes.io/version: v0.20.2
    helm.sh/chart: sealed-secrets-2.8.1
  name: sealed-secrets
  namespace: infrastructure
  resourceVersion: "352387267"
  uid: ced7b2e5-e75c-428e-a5f1-185142f0571b
spec:
  clusterIP: xxxxxx
  clusterIPs:
  - xxxxx
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: http
    port: 8080
    protocol: TCP
    targetPort: http
  - name: http-80
    port: 80
    protocol: TCP
    targetPort: http
  selector:
    app.kubernetes.io/instance: sealed-secrets
    app.kubernetes.io/name: sealed-secrets
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

[rowi1de]

@bakito regarding the error "cannot fetch certificate: error trying to reach service: Address is not allowed" do you know how the cluster internal fetching works? How the Endpoint is constructed?

[bakito]

@rowi1de no, not really.
This is done, as mentioned in #141, inside the slealed-secret client.

Probably they use k8s default.

..svc.cluster.local

Or so

[rowi1de]

Good Hint, so I can curl "sealed-secrets.infrastructure.svc.cluster.local:8080/v1/cert.pem" ...
so setting certURL: "http://sealed-secrets.infrastructure.svc.cluster.local:8080/v1/cert.pem" fixed it.

I've the feeling something is wrong in here https://github.com/bitnami-labs/sealed-secrets/blob/67c2699797425dff2d39f550b42b1782d2ba406e/pkg/kubeseal/kubeseal.go#LL120C2-L120C40 will need to dig deeper. Thanks

[rowi1de]

@bakito one more idea:
You are passing "TODO" for the context on OpenCert, could this be a reason
https://github.com/search?q=repo%3Abakito%2Fsealed-secrets-web+TODO&type=code

[bakito]

@rowi1de context.TODO shoudl not make a difference here.
Anyway good hint as I wanted to change this quite some while.
I changed it in #143. Once the merge request is merged and build you could try the newest ghcr.io/bakito/sealed-secrets-web:main to check if it makes any difference.

[bakito]

When using sealedSecrets.namespace and sealedSecrets.serviceName the service is queried by on the k8s api.
If the service account of the ssw pod needs to have rbac to read the services.
Is rbac created with the helm chart in your case?

[rowi1de]

I’m using the default for both helm charts regarding rbac => this means rbac resources are created in both charts

[rowi1de]

Networking issue https://medium.com/@denisstortisilva/kubernetes-eks-calico-and-custom-admission-webhooks-a2956b49bd0d

[phoenix-bjoern]

To fix the permission issue for the SA I had to extend the Role:

rules:
  - apiGroups:
      - bitnami.com
    resources:
      - sealedsecrets
    verbs:
      - get
      - list
  - apiGroups:
      - ''
    resources:
      - secrets
    verbs:
      - get
  - apiGroups:
      - ''
    resources:
      - services/proxy
    verbs:
      - get
  - apiGroups:
      - ''
    resources:
      - services
    verbs:
      - get

Maybe this is also caused by a change in kubeseal...