Run functional testing via scripts and generate a report which is uploaded as artifact to the repo

[iranzo]

We're currently doing installation of a Spoke based on a HUB, but we're not testing several steps that have raised issues like the ones we saw recently:

• ACM didn't finished the installation
• CSR for adding the worker weren't added
• Containers fail to pull because they don't accept the self-signed certificate from the internal registry
• etc

The goal will be to create some testing, based on simple scripts that test the UI, test running some pods, etc and upload the final artifact of the generated report to the repo

For example: reuse and extend https://github.com/RHsyseng/openshift-checks

Please, comment with suggested checks and we can elaborate a list:

• ACM installation finished
• No CSRS pending
• No PullBack pods
• Start a new deployment on the servers to validate

[fbac]

As mentioned, in order to perform a real e2e we're lacking some test. In addition to what was already commented:

• Test final customer procedures that are going to be executed through UI, such as:

  • IP change
  • domain change
  • certificate addition/change
  • any other procedure performed by the customer through UI

• Checks per component:

  • kubernetes and ocp api health
  • etcd health (including some disk speed checks?)
  • router and routes health
  • registry
  • any other critical component test

• Deploy of a mock application with a workflow expected for a customer:

  • deploy a simple webserver
  • expose the service
  • check the route, the app, etc
  • scale replicas
  • any other missing usage test

In general we lack real e2e; this is needed but as well it will be expensive in time and resources.

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days