Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

helmGlobals is not used when HelmChartInflationGenerator is used in generators field #5685

Open
LarsStegman opened this issue Apr 30, 2024 · 1 comment
Labels
kind/bug Categorizes issue or PR as related to a bug. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.

Comments

@LarsStegman
Copy link

LarsStegman commented Apr 30, 2024

What happened?

I am having trouble getting my deployments of a Helm chart to multiple sites/deployments working.

The setup

/
├─ charts/
│  ├─ my-app/
├─ my-app/
│  ├─ helm/
│  │  ├─ helm.yaml
│  │  ├─ kustomization.yaml
│  ├─ overlays/
│  │  ├─ site1/
│  │  │  ├─ common/
│  │  │  │  ├─ kustomization.yaml
│  │  │  ├─ deployments/
│  │  │  │  ├─ deployment1/
│  │  │  │  │  ├─ kustomization.yaml
│  │  │  │  ├─ kustomization.yaml

I have this structure because I need to deploy my app to multiple sites. Some
of the values need to be the same across all sites, like the application
version, for example. Some of the values need to be the same across 1 site,
like some credentials/domain names. Some of the values are specific per
deployment, like the name.

This is the contents of the /my-app/helm folder. It is used as the base
config for all deployments.

# /my-app/helm/helm.yaml
apiVersion: builtin
kind: HelmChartInflationGenerator
metadata:
  name: my-app
name: my-app
# /my-app/helm/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - helm.yaml

Then, per site I kustomize the HelmChartInflationGenerator from
/my-app/helm/helm.yaml to provide some valuesInline.

# /my-app/overlays/site1/common/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ../../../helm

patches:
  - patch: |-
      apiVersion: builtin
      kind: HelmChartInflationGenerator
      metadata:
        name: my-app
      valuesInline:
        k8sSecret:
          secretName: my-secret
        outputs:
          - url: "http://database.site1.company.global"
            tokenSecretKey: write1
            db: "db1"

Then in /my-app/overlays/site1/deployments/deployment1/kustomization.yaml I kustomize the
same HelmChartInflationGenerator another time to add additional
valuesInline values.

# /my-app/overlays/site1/deployments/deployment1/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../common-config

patches:
  - patch: |-
      apiVersion: builtin
      kind: HelmChartInflationGenerator
      metadata:
        name: my-app
      releaseName: deployment1-my-app
      valuesInline:
        requireActiveAccessToSensors: true
        socketListeners:
          - protocol: UDP
            port: 2000
        appConfig: |
          [config]
            key1 = "value1"

Finally, I collect all the different deployments from the site in
/my-app/overlays/site1/deployments/kustomization.yaml.

# /my-app/overlays/site1/deployments/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
helmGlobals:
  chartHome: ../../../charts

generators:
  - ./deployment1

The problem

When I try to build this kustomization I get the following:

> cd my-app/overlays/site1/deployments
> kustomize build . --enable-helm --load-restrictor LoadRestrictionsNone
Error: no repo specified for pull, no chart found at ''

It looks like helmGlobals.chartHome is not passed to the
HelmChartInflationGenerator at all. Even if I change
it to foo, I still get the same error. I would expect it to say
no chart found at 'foo' then.

Am I doing something wrong, or is this a bug?

What did you expect to happen?

The Helm chart is inflated multiple times, once per sites per deployment.

How can we reproduce it (as minimally and precisely as possible)?

Files

# /my-app/helm/helm.yaml
apiVersion: builtin
kind: HelmChartInflationGenerator
metadata:
  name: my-app
name: my-app
# /my-app/helm/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - helm.yaml
# /my-app/overlays/site1/common/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ../../../helm

patches:
  - patch: |-
      apiVersion: builtin
      kind: HelmChartInflationGenerator
      metadata:
        name: my-app
      valuesInline:
        k8sSecret:
          secretName: my-secret
        outputs:
          - url: "http://database.site1.company.global"
            tokenSecretKey: write1
            db: "db1"
# /my-app/overlays/site1/deployments/deployment1/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../common-config

patches:
  - patch: |-
      apiVersion: builtin
      kind: HelmChartInflationGenerator
      metadata:
        name: my-app
      releaseName: deployment1-my-app
      valuesInline:
        requireActiveAccessToSensors: true
        socketListeners:
          - protocol: UDP
            port: 2000
        appConfig: |
          [config]
            key1 = "value1"
# /my-app/overlays/site1/deployments/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
helmGlobals:
  chartHome: ../../../charts

generators:
  - ./deployment1

Command

> cd my-app/overlays/site1/deployments
> kustomize build . --enable-helm --load-restrictor LoadRestrictionsNone
Error: no repo specified for pull, no chart found at ''

Expected output

Inflated Helm Chart per site per deployment

Actual output

An error saying the Helm chart cannot be found

Kustomize version

5.3.0

Operating system

Linux

@LarsStegman LarsStegman added the kind/bug Categorizes issue or PR as related to a bug. label Apr 30, 2024
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

SIG CLI takes a lead on issue triage for this repo, but any Kubernetes member can accept issues by applying the triage/accepted label.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label Apr 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.
Projects
None yet
Development

No branches or pull requests

2 participants