[Feature] Allows custom job templates on helm chart

[gabriel-milan]

Summary

This allows setting a custom job template directly on the values.yaml file.

Importance

In our use case, the only thing we needed to do was to set environment variables for all our jobs using envFrom. With this change (tested and working for us) it allows us to change the job template directly on the values.yaml file, making it much easier to manage.

Checklist

This PR:

• adds new tests (not applicable, I guess. please correct me if I'm wrong)
• adds a change file in the changes/ directory (wip)

[zanieb]

Can you share an example of using this? It seems like the user experience would not really be great from a helm-chart perspective as you'd have to ensure the file is available.

[gabriel-milan]

Sure! In our deployment, we're setting agent.jobTemplate to a YAML hosted on GCS. The YAML looks like this:

apiVersion: batch/v1
kind: Job
spec:
  template:
    spec:
      containers:
        - name: flow
          envFrom:
            - secretRef:
                name: gcp-credentials

This allows us to set many environment variables from existing secrets in our different namespaces, which eases management of dev/staging/prod environments.

[zanieb]

Perhaps we should name it "jobTemplateFilePath" or something instead in values.yaml. I expect to be able to inline some yaml as currently named.

[gabriel-milan]

That makes sense

[zanieb]

#323 will resolve the CI failures

[zanieb]

@gabriel-milan I don't have permission to update your branch

[gabriel-milan]

That's weird. I've enabled the Allow edits by maintainers. Is there anything else I can do?

[zanieb]

@gabriel-milan that works! If this passes CI I'll get it into #324