Our Shopozor product needs a reliable CI/CD system. This repository consists of its configuration. In essence, we need to continuously
- unit test
- acceptance test
- end-to-end test
the Shopozor services.
This CI configuration provides following output:
- our Github Pull Requests get feedback on the Shopozor applications' tests
- our Github repositories get feedback on the Shopozor applications' current build statuses
The resulting CI/CD Jelastic environment is the entrypoint to any deployment and test.
We took the decision to implement our CI/CD with Jenkins, even though there are much more sexy alternatives in the development ecosystem out there. The main reason behind that decision is our very little development budget. After the first release, our customer will take over the CI/CD costs.
- CircleCI is very nicely integrated into Github and is very easy to configure
- The hobbyist plan is pretty restrictive on the amount of docker containers we can run within a test: it could be fine for our frontends’ acceptance tests but definitely not for our backend’s acceptance tests and all our e2e tests
- External repositories not supported in the free plan
- We could host our repos on Gitlab but then we’d have to pay for its hosting, which would need to be a 24/7 hosting, making it therefore a pretty expensive choice
- Very easy to configure
- Completely free
- We can live with a partially available Jenkins server, some kind of on-demand Jenkins until the product's first release; as Jenkins only triggers builds, it's more than fine.
Each of our services is unit- and integration-tested in isolation of the other services. This is done inside of the Jenkins docker container which is provided with the relevant docker tools.
The two major use cases of our product, the Consumer and the Management use cases, are tested end-to-end. That means that all the necessary services are started on their own Jelastic environments and the tests are run on them. The test results are then gathered from those environments and referenced on our Jenkins job.
In its simplest configuration, our CI environment looks like this:
It runs a docker image of jenkins as well as a Postgresql database. The database is required for some of our backend tests.
Out of the box, the jenkins docker image isn't provided with all the tools we need for our CI/CD. Additionally, we need the following software:
- docker, because we build docker images
- jq, because we interact with Jelastic environments
in our pipelines.
The list of necessary plugins can be found here. Among others,
- ghprb is responsible for triggering the relevant tests on Jenkins upon pull request
- embeddable-build-status is responsible for making the Jenkins builds statuses available so that they can be displayed in the corresponding repositories'
README
s
The Github user(s) involved in our CI/CD jobs / pipelines need to fulfill the following requirements:
- full repository permissions, in order to be able to modify pull requests
- 2FA disabled
- absolutely needs ghprb shared secret: the section
<githubAuth>
<org.jenkinsci.plugins.ghprb.GhprbGitHubAuth>
<serverAPIUrl>https://api.github.com</serverAPIUrl>
<credentialsId>GITHUB_CREDENTIAL_ID</credentialsId>
<id>GHPRB_TRIGGER_AUTH_ID</id>
<description>Github pull request builder</description>
<secret>GHPRB_SHARED_SECRET</secret>
</org.jenkinsci.plugins.ghprb.GhprbGitHubAuth>
</githubAuth>
of the GhprbTrigger config needs to be filled up with the secret provided in the Github repository's webhook:
Note that the webhook gets automatically created in the Github repository upon pull request creation. The secret will not be filled correctly though. Therefore, the first time a PR is made, no Jenkins job will be triggered, but the webhook will be created. Then it just needs to be fed up with the relevant secret. After that, the following comment in the pull request
retest this please
will trigger the Jenkins build accordingly.
- the trigger phrases are documented here
The global GHPRB plugin configuration can be found here. Upon a pull request, the repository's
- unit
- acceptance
tests are triggered. Their results are returned back to the pull request upon completion:
In the PR comments,
retest this please
will run the above mentioned tests againtest consumer e2e
will run the consumer end-to-end teststest management e2e
will run the management end-to-end tests
The jobs are setup automatically by means of the Jenkins CLI. Their configuration can be found in config/*.xml
:
Jenkins pipelines can nicely be configured by code. Extensive documentation can be found here.
Because our e2e pipelines share some code, they can do so over:
The configuration of the global shared libraries happens here.
Any push to our master
or dev
branch will be rejected, except it was initiated by our softozor-deployer
user:
As soon as a pull request is issued, the webhook is automatically added to the repository's webhooks:
It is critical that the webhook's secret be filled with the corresponding value set in Jenkins global configuration of the GHPRB plugin. In addition, the webhook needs to be triggered by everything.
The list of recent deliveries clearly shows whether the webhook gets triggered and with what payload and result.
The Jelastic installation requires following form to be filled: