Problem with gitops-connector concept and hard to adapt for multiple applications scenario

[kauemendes]

We are trying to implement the solution using GitOps Connector and we face some issues related to the solution.
We have found out that the solution is designed to host one app repository with one manifest repository, but our scenario is different.

We have several projects in one manifest repository, therefore we can have multiple commit-ids or pull-requests entering the git history at the same time. Flux reconciles the latest commit id updated with the source controller, meaning that we can miss several commit-ids and pull-requests in the way of the gitops-connector notification.

Following scenario:

We have gitops source controller with commit-id

123 --- as latest

After several commits

123 <- latest applied by kustomization
321
432
212
122 <- latest commit not yet applied

SourceController updates to 122 hash and then kustomization will apply… Of course, all in between will apply as well since Git is cumulating changes, but the gitops-connector will only execute on top of 122 the latest commit notified by flux, if we have multiples pipelines waiting for gitops connector response git commit hash such as 212, 432 or 321, will not be notified and the pipeline will wait for approval until the task timeout expires.

IRL Scenario:

Giving one manifest repo to multiple applications pipelines, Is there an option or solution of configuration using GitOps-Connector?

[eedorenko]

Nope. The problem is that Flux doesn't reconcile those intermediate commits. It doesn't know if they are successful or not. It doesn't know if they even exist and therefore there is no notification callback on them.

The solution would be to have different repos or different branches. In the latter case Flux will have different GitRepositories that will be handled separately.

[joeshadbolt]

@eedorenko apologies if I am interpreting your response here incorrectly. But, are you saying the gitops connector essentially does not support FluxCD when using a 'repo per environment' repository structure because flux's source controller doesn't necessarily create an artifact per commit?

[eedorenko]

If repo per environment means that a repo contains in a single branch manifests for different applications with different promotion lifecycles, then, yes, gitops connector doesn't support that schema.

[pareion]

Will it support this use case in the future?

In our case, our dev and test repositories contain their manifests, so we would have x amount of GitOps connectors instead of a single one.
However, for sanity's sake, we have opted for one repository in our staging and production case, which this use case supports.