This tool was born from a need to track build tool changes for a bunch of repositories. Due to various reasons, a
microservices based project I manage needed to convert from Maven builds to Gradle builds. We had over 20 repositories
that had varying levels of compliance, which made it more difficult to get them set up in our CI/CD pipelines. This
tool allows us to scan each of the repositories on demand and get a report based on the current develop
baseline. We
also were able to police when a team marked the pipeline to skip tests. I hope you'll be able to use it as well.
In a microservices environment where there are many repositories, one for each service, I needed a way to test the consistency of the layout. I created this tool primarily to enforce that the work branch is consistent, and the required files are present across all the repositories. There's a lot this tool can be expanded to do in the future, but this was the immediate need.
You control actions by specifying the YAML file that describes your environment. By default, it is called
scan-repos.yaml
. This file provides the list of files you want to ensure exist, and the list of repositories.
work-branch: develop
auth:
mode: url
api-token: api.token
rejected-files:
- LICENSE.txt
required-files:
- name: LICENSE
not-empty: true
includes: Apache License
- name: CODE_OF_CONDUCT.md
not-empty: true
- name: CONTRIBUTING.md
not-empty: true
- name: PULL_REQUEST_TEMPLATE.md
not-empty: true
repositories:
- https://github.com/D-Haven/DHaven.Faux.git
- https://github.com/D-Haven/spa-server.git
- https://github.com/D-Haven/demo-projects.git
- https://github.com/D-Haven/Ark.git
- https://github.com/D-Haven/DHaven.uService.Core.git
- https://github.com/D-Haven/DHaven.Api.Gateway.git
- https://github.com/D-Haven/DHaven.LoadBalance.git
- https://github.com/D-Haven/DHaven.Faux.git
- https://github.com/D-Haven/BibleUtilities.git
Specifies the work branch where development happens. In many organizations, it is develop
, but others just use main
or master
.
Specifies the authentication to use if it's not a public API. If authentication is specified, then the sub structure needs to be:
auth:
mode: basic | url
username: username
api-token: path to token file
Mode must be "basic" for standard HTTP basic authentication, or "url" for url encoded authentication. GitHub and GitLab don't care what the username is, so that is optional. Bitbucket requires the username.
Optional field to specify the specific username. This is required for Bitbucket, but other hosts don't require it. If not supplied, the default username of "repo-scanner" will be used.
Specifies the filename for your API Token when interacting with GitHub. The file should only include the content of the API token.
NOTE: we are planning more authentication methods to support SSH connections and Bitbucket. You can help out here!
The list of files that should not exist in a repository. Useful for migrating away from one build too suite to a new version. There are no further validations, because the only criteria is that these files do not exist in the work branch.
required-files:
- name: LICENSE
not-empty: true
includes: Apache License
excludes: GNU GENERAL PUBLIC LICENSE
The list of files that must exist, and the further constraints we want to check. Those constraints currently include ensuring the file contains content, that the file contains text, or that the file does not contain text.
The name of the file that must exist
Flag to enforce the file contains something (i.e. not an empty file).
Text to ensure exists in the file. Allows you to check your "Jenkinsfile" is using the build tool, etc. Only one positive test allowed currently.
Text to ensure does not exist in a file. Allows you to check your "Jenkinsfile" is not skipping tests. Only one negative test allowed currently.
The list of repository URLS to scan. The tool will clone just the work branch for each repository in memory, scan the work tree for the required files, and ensure they have more than 1 byte each. If there is an error pulling that branch in a repository, the tool will mark the error and move on.