Skip to content

Commit

Permalink
Merge pull request #6 from kaweezle/feature/kustomization-generator
Browse files Browse the repository at this point in the history
✨ Added kustomization generator
  • Loading branch information
antoinemartin committed Jan 31, 2023
2 parents 31a2a21 + 2886e90 commit f39ac6a
Show file tree
Hide file tree
Showing 15 changed files with 184 additions and 21 deletions.
1 change: 1 addition & 0 deletions .github/workflows/release-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ jobs:
args: --snapshot --skip-publish --rm-dist
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SCOOP_BUCKET_GITHUB_TOKEN: ${{ secrets.REPO_ACCESS_TOKEN }}
# - name: Build APK repo
# uses: ./.github/actions/make-apkindex
# with:
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ jobs:
args: release --rm-dist
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SCOOP_BUCKET_GITHUB_TOKEN: ${{ secrets.REPO_ACCESS_TOKEN }}
- name: Trigger APK repo build
uses: peter-evans/repository-dispatch@v2
with:
Expand Down
16 changes: 16 additions & 0 deletions .goreleaser.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ builds:
archives:
- format: binary
name_template: '{{ .Binary }}_v{{ .Version }}_{{ .Os }}_{{ .Arch }}{{ with .Arm }}v{{ . }}{{ end }}{{ with .Mips }}_{{ . }}{{ end }}{{ if not (eq .Amd64 "v1") }}{{ .Amd64 }}{{ end }}'
format_overrides:
- goos: windows
format: zip
checksum:
name_template: "SHA256SUMS"
snapshot:
Expand Down Expand Up @@ -73,3 +76,16 @@ dockers:
- --label=org.opencontainers.image.created={{ time "2006-01-02T15:04:05Z07:00" }}
- --label=org.opencontainers.image.revision={{ .FullCommit }}
- --label=org.opencontainers.image.licenses=Apache-2.0
scoop:
bucket:
owner: kaweezle
name: scoop-bucket
branch: main
token: "{{ .Env.SCOOP_BUCKET_GITHUB_TOKEN }}"
commit_author:
name: Antoine Martin
email: [email protected]
commit_msg_template: "Scoop update for {{ .ProjectName }} version {{ .Tag }}"
homepage: https://github.com/kaweezle/krmfnbuiltin
description: "Kustomize SOPS KRM function"
license: Apache-2.0
64 changes: 62 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ transformation in your kustomize projects.
<li><a href="#configmap-generator-with-git-properties">ConfigMap generator with git properties</a></li>
<li><a href="#heredoc-generator">Heredoc generator</a></li>
<li><a href="#extended-replacement-in-structured-content">Extended replacement in structured content</a></li>
<li><a href="#kustomization-generator">Kustomization generator</a></li>
</ul>
</li>
<li><a href="#installation">Installation</a></li>
Expand Down Expand Up @@ -351,6 +352,24 @@ metadata:
With these annotations, the generated config map will be saved in the
`local-config.yaml` file in the configuration directory.

If the file name is empty, i.e. the annotation is:

```yaml
config.kaweezle.com/path: ""
```

The generated resources will be saved each in its own file with the pattern:

```text
<namespace>/<kind>_<name>.yaml
```

For instance:

```text
kube-flannel/daemonset_kube-flannel-ds.yaml
```

## Extensions

### Remove Transformer
Expand Down Expand Up @@ -735,7 +754,7 @@ metadata:
config.kaweezle.com/local-config: "true"
config.kubernetes.io/function: |
exec:
path: ../../krmfnbuiltin
path: krmfnbuiltin
data:
sish:
# New properties
Expand All @@ -751,7 +770,7 @@ metadata:
config.kaweezle.com/prune-local: "true"
config.kubernetes.io/function: |
exec:
path: ../../krmfnbuiltin
path: krmfnbuiltin
replacements:
- source:
kind: LocalConfiguration
Expand Down Expand Up @@ -810,6 +829,47 @@ will become
HostName target.link
```

### Kustomization generator

`KustomizationGenerator` is the kustomize equivalent to
`HelmChartInflationGenerator`. It allows generating resources from a
kustomization.

Example:

```yaml
apiVersion: builtin
kind: KustomizationGenerator
metadata:
name: kustomization-generator
annotations:
config.kaweezle.com/path: "uninode.yaml"
config.kubernetes.io/function: |
exec:
path: krmfnbuiltin
kustomizeDirectory: https://github.com/antoinemartin/autocloud.git//packages/uninode?ref=deploy/citest
```

If this function is run with the following command:

```console
> kustomize fn run --enable-exec --fn-path functions applications
```

It will generate a file named `uninode.yaml` in the `applications` directory.
With:

```yaml
config.kaweezle.com/path: ""
```

One file will be created per resource (see
[Keeping or deleting generated resources](#keeping-or-deleting-generated-resources)).

**IMPORTANT** `krmfnbuiltin` runs from the directory where the
`kustomize run fn` command has been launched, and **not from the function
configuration folder**. Any relative path should take this into consideration.

## Installation

With each [Release](https://github.com/kaweezle/krmfnbuiltin/releases), we
Expand Down
4 changes: 3 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ require (
github.com/stretchr/testify v1.8.1
golang.org/x/tools v0.5.0
sigs.k8s.io/kustomize/api v0.12.1
sigs.k8s.io/kustomize/kyaml v0.13.10
sigs.k8s.io/kustomize/kyaml v0.13.9
sigs.k8s.io/yaml v1.2.0

)
Expand All @@ -33,6 +33,7 @@ require (
github.com/go-openapi/swag v0.22.3 // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/google/gnostic v0.5.7-v3refs // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/imdario/mergo v0.3.13 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
Expand All @@ -51,6 +52,7 @@ require (
github.com/spf13/pflag v1.0.5 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
github.com/xlab/treeprint v1.1.0 // indirect
go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 // indirect
golang.org/x/crypto v0.3.0 // indirect
golang.org/x/mod v0.7.0 // indirect
golang.org/x/net v0.5.0 // indirect
Expand Down
10 changes: 8 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a h1:idn718Q4
github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY=
github.com/bwesterb/go-ristretto v1.2.0/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cloudflare/circl v1.1.0 h1:bZgT/A+cikZnKIwn7xL2OBj012Bmvho/o6RpRvv3GKY=
github.com/cloudflare/circl v1.1.0/go.mod h1:prBCrKB9DV4poKZY1l9zBXg2QJY7mvgRvtMxxK7fi4I=
Expand Down Expand Up @@ -89,6 +92,7 @@ github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/
github.com/google/gofuzz v1.1.0 h1:Hsa8mG0dQ46ij8Sl2AYJDUv1oA9/d6Vk+3LG99Oe02g=
github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=
github.com/google/uuid v1.1.2 h1:EVhdT+1Kseyi1/pUmXKaFxYsDNy9RQYkMWRH68J/W7Y=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
Expand Down Expand Up @@ -173,6 +177,7 @@ github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9de
github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 h1:+FNtrFTmVw0YZGpBGX56XDee331t6JAXeK2bcyhLOOc=
go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5/go.mod h1:nmDLcffg48OtT/PSW0Hg7FvpRQsQh5OSqIylirxKC7o=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
Expand Down Expand Up @@ -217,6 +222,7 @@ golang.org/x/sync v0.1.0 h1:wsuoTGHzEhffawBOhz5CYhcrV4IdKZbEyZjBMuTp12o=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191002063906-3421d5a6bb1c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
Expand Down Expand Up @@ -319,8 +325,8 @@ k8s.io/utils v0.0.0-20220210201930-3a6ce19ff2f9 h1:HNSDgDCrr/6Ly3WEGKZftiE7IY19V
k8s.io/utils v0.0.0-20220210201930-3a6ce19ff2f9/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA=
sigs.k8s.io/kustomize/api v0.12.1 h1:7YM7gW3kYBwtKvoY216ZzY+8hM+lV53LUayghNRJ0vM=
sigs.k8s.io/kustomize/api v0.12.1/go.mod h1:y3JUhimkZkR6sbLNwfJHxvo1TCLwuwm14sCYnkH6S1s=
sigs.k8s.io/kustomize/kyaml v0.13.10 h1:htPMvrk7ZDfTDyrgXIm/2mfmcYJHEmRb6s+yCLgtNms=
sigs.k8s.io/kustomize/kyaml v0.13.10/go.mod h1:PzDV8gSaY8mwdd7nR9zg7Pw5yh9fu8G+ElAXoQVzBq8=
sigs.k8s.io/kustomize/kyaml v0.13.9 h1:Qz53EAaFFANyNgyOEJbT/yoIHygK40/ZcvU3rgry2Tk=
sigs.k8s.io/kustomize/kyaml v0.13.9/go.mod h1:QsRbD0/KcU+wdk0/L0fIp2KLnohkVzs6fQ85/nOXac4=
sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw=
sigs.k8s.io/yaml v1.2.0 h1:kr/MCeFWJWTwyaHoR9c8EjH9OumOmoF9YGiZd7lFm/Q=
sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc=
2 changes: 1 addition & 1 deletion main.go
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ func main() {

cmd := command.Build(processor, command.StandaloneDisabled, false)
command.AddGenerateDockerfile(cmd)
cmd.Version = "v0.3.0" // <---VERSION--->
cmd.Version = "v0.3.1" // <---VERSION--->

if err := cmd.Execute(); err != nil {
os.Exit(1)
Expand Down
54 changes: 54 additions & 0 deletions pkg/extras/KustomizationGenerator.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
package extras

import (
"sigs.k8s.io/kustomize/api/krusty"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/filesys"
"sigs.k8s.io/yaml"
)

// KustomizationGeneratorPlugin configures the KustomizationGenerator.
type KustomizationGeneratorPlugin struct {
Directory string `json:"kustomizeDirectory,omitempty" yaml:"kustomizeDirectory,omitempty"`
}

// enablePlugins adds to opts the options to run exec functions
func enablePlugins(opts *krusty.Options) *krusty.Options {
opts.PluginConfig = types.EnabledPluginConfig(types.BploUseStaticallyLinked) // cSpell: disable-line
opts.PluginConfig.FnpLoadingOptions.EnableExec = true
opts.PluginConfig.FnpLoadingOptions.AsCurrentUser = true
opts.PluginConfig.HelmConfig.Command = "helm"
opts.LoadRestrictions = types.LoadRestrictionsNone
return opts
}

// runKustomizations runs the kustomization in dirname (URL compatible) with
// the filesystem fs.
func runKustomizations(fs filesys.FileSystem, dirname string) (resources resmap.ResMap, err error) {

opts := enablePlugins(krusty.MakeDefaultOptions())
k := krusty.MakeKustomizer(opts)
resources, err = k.Run(fs, dirname)
return
}

// Config reads the function configuration, i.e. the kustomizeDirectory
func (p *KustomizationGeneratorPlugin) Config(
h *resmap.PluginHelpers, c []byte) (err error) {
err = yaml.Unmarshal(c, p)
if err != nil {
return err
}
return err
}

// Generate generates the resources of the directory
func (p *KustomizationGeneratorPlugin) Generate() (resmap.ResMap, error) {
return runKustomizations(filesys.MakeFsOnDisk(), p.Directory)
}

// NewKustomizationGeneratorPlugin returns a newly Created KustomizationGenerator
func NewKustomizationGeneratorPlugin() resmap.GeneratorPlugin {
return &KustomizationGeneratorPlugin{}
}
5 changes: 3 additions & 2 deletions pkg/plugins/builtinplugintype_string.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions pkg/plugins/factories.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ const (
ReplacementTransformer
GitConfigMapGenerator
RemoveTransformer
KustomizationGenerator
)

var stringToBuiltinPluginTypeMap map[string]BuiltinPluginType
Expand Down Expand Up @@ -122,6 +123,7 @@ var GeneratorFactories = map[BuiltinPluginType]func() resmap.GeneratorPlugin{
SecretGenerator: builtins.NewSecretGeneratorPlugin,
HelmChartInflationGenerator: builtins.NewHelmChartInflationGeneratorPlugin,
GitConfigMapGenerator: extras.NewGitConfigMapGeneratorPlugin,
KustomizationGenerator: extras.NewKustomizationGeneratorPlugin,
}

func MakeBuiltinPlugin(r resid.Gvk) (resmap.Configurable, error) {
Expand Down
12 changes: 7 additions & 5 deletions pkg/utils/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,12 +61,14 @@ func TransferAnnotations(list []*yaml.RNode, config *yaml.RNode) (err error) {
if local {
annotations[FunctionAnnotationLocalConfig] = "true"
}
annotations[kioutil.LegacyPathAnnotation] = path
annotations[kioutil.PathAnnotation] = path
if path != "" {
annotations[kioutil.LegacyPathAnnotation] = path
annotations[kioutil.PathAnnotation] = path

curIndex := strconv.Itoa(startIndex + index)
annotations[kioutil.LegacyIndexAnnotation] = curIndex
annotations[kioutil.IndexAnnotation] = curIndex
curIndex := strconv.Itoa(startIndex + index)
annotations[kioutil.LegacyIndexAnnotation] = curIndex
annotations[kioutil.IndexAnnotation] = curIndex
}
delete(annotations, FunctionAnnotationInjectLocal)
delete(annotations, FunctionAnnotationFunction)
r.SetAnnotations(annotations)
Expand Down
10 changes: 10 additions & 0 deletions tests/kustomization/functions/kustomization-generator.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
apiVersion: builtin
kind: KustomizationGenerator
metadata:
name: kustomization-generator
annotations:
config.kaweezle.com/path: ""
config.kubernetes.io/function: |
exec:
path: ../../krmfnbuiltin
kustomizeDirectory: https://github.com/antoinemartin/autocloud.git//packages/uninode?ref=deploy/citest
Empty file.
12 changes: 8 additions & 4 deletions tests/test_krmfnbuiltin.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,14 @@ for d in $(ls -d */); do
cp -r original applications
echo " > Performing kustomizations..."
kustomize fn run --enable-exec --fn-path functions applications
for f in $(ls -1 applications); do
echo " > Checking $f..."
diff <(yq eval -P expected/$f) <(yq eval -P applications/$f)
done
if [ -d expected ]; then
for f in $(ls -1 applications); do
echo " > Checking $f..."
diff <(yq eval -P expected/$f) <(yq eval -P applications/$f)
done
else
echo " > No expected result. Skipping check"
fi
cd ..
done
echo "Done ok 🎉"
12 changes: 8 additions & 4 deletions tests/test_krmfnbuiltin_kpt.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,14 @@ for d in $(ls -d */); do
mv $temp_file_2 $temp_file
done
cat $temp_file | kpt fn sink applications
for f in $(ls -1 expected); do
echo " > Checking $f..."
diff <(yq eval -P expected/$f) <(yq eval -P applications/$f)
done
if [ -d expected ]; then
for f in $(ls -1 applications); do
echo " > Checking $f..."
diff <(yq eval -P expected/$f) <(yq eval -P applications/$f)
done
else
echo " > No expected result. Skipping check"
fi
cd ..
done
echo "Done ok 🎉"

0 comments on commit f39ac6a

Please sign in to comment.