CI Test Coverage Calculation

[daemon1024]

Feature Request

We should calculate test coverage for tests across all the CI test suites we run.

Is your feature request related to a problem? Please describe the use case.

Currently each coverage calculation is overwritten by the new coverage data in subsequent test suites in a PR.

Describe the solution you'd like

Uploading coverage data to codecov in a seperate job extractiong information from each test suite.
https://about.codecov.io/blog/uploading-code-coverage-in-a-separate-job-on-github-actions/ this article should help out.

[anonymousr007]

Hello,

I am new in KubeArmor, I saw this is a good first issue. So this is perfect for me as a beginner. Can you please guide me more about that issue? I am also interested in this project #789.

Thanks & Regards
Rishabh

[daemon1024]

Hey @anonymousr007, This issue would require familiarity with Github Actions in general. Github has an excellent documentation about Github Actions @ https://docs.github.com/en/actions.

Our coverage calculation, sends the coverage data to codecov in the test suite here

KubeArmor/.github/workflows/ci-test-local.yml

Lines 72 to 85 in b38a0a5

	- name: Measure code coverage
	if: ${{ always() }}
	run: |
	go install github.com/modocache/gover@latest
	gover
	go tool cover -func=gover.coverprofile
	working-directory: KubeArmor
	env:
	GOPATH: /home/runner/go
	- uses: codecov/codecov-action@v2
	if: ${{ always() }}
	with:
	files: ./KubeArmor/gover.coverprofile

.

We need to move this to a seperate job so as a collective coverage can be calculated.
As mentioned in the issue, https://about.codecov.io/blog/uploading-code-coverage-in-a-separate-job-on-github-actions/ this article should guide you about how can we achieve that.

Thanks a lot for showing interest in this. Feel free to ask questions here in this issue thread or on KubeArmor Slack, happy to help out.

[anonymousr007]

Hello @daemon1024, Thanks for assigning me this issue.
I would like to ask a question about this issue.

• Is This YAML file code a separate job in the same file? File Name: KubeArmor/.github/workflows/ci-test-local.yml

[daemon1024]

This YAML file code a separate job in the same file? File Name: KubeArmor/.github/workflows/ci-test-local.yml

This is a step of the job build, we want to move it to a separate job, yes.
https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions#understanding-the-workflow-file

[anonymousr007]

Hello @daemon1024, I make a WIP-PR. You guide me in it. Thanks.

[daemon1024]

Hey @anonymousr007 Can you link me the PR? I cannot seem to find it.

[anonymousr007]

Please review it

[rahulk789]

Hello! I would love to work on this issue. I have gone through the articles sent above. My understanding of the issue is that the coverage task is currently a step in KubeArmor/.github/workflows/ci-test-local.yml and must be converted to a seperate job, is there any other file that i should be editing? I think removing codecov step from github/workflows/ci-test-ginkgo.yml makes sense.

[daemon1024]

Thanks for picking this up @rahulk789.

I think removing codecov step from github/workflows/ci-test-ginkgo.yml makes sense.

From my understanding it would still need some form of codecov step that helps aggregating the coverage files across the test suite. I will check the PR and let's continue the discussion there.

[rahulk789]

Since Achref's pr (#1029) had removed local tests, we no longer need to combine the code coverage from two ci tests since we have only ginkgo test suit now. That being said, the issue seems no longer relevant and should I close my pr? correct me if I am wrong. @daemon1024

[daemon1024]

Yes @rahulk789, this is still valid.
Even if we only have a single test suite, it runs across multiple matrices like docker,container and crio which would result in different coverages (once generated) which is needed to be aggregated.

Also we would need a way to generate coverage in Ginkgo test suite itself too. So the work item is still relevant imo. WDYT?

[Pushkarm029]

Working on this one. Please assign.

[mohammad-siraj]

Hello! I would love to work on this issue. @rahulk789