We expect and require all contributions to our products to go a merge request with a formal review. As such, we follow the Merge Request workflow and code review guidelines articulated in GitLab's developer documentation. We would, however, like to highlight a few items from these documents and add a few additional considerations for reviewers and authors.
The secure analyzers verify merge requests by running a new commit against downstream test projects for their supported languages/frameworks (i.e. the
gemnasium analyzer of
Dependency Scanning will trigger tests against
go, and several other test projects). The verification is done by comparing the generated report output against an expected report committed to the analyzer's repository. If analyzer behavior has changed, then the pipeline will fail because the contents of the expected and generated reports will no longer match.
The expectations for the
Secrets Detection analyzers are stored in the project's
qa/expect directory with a sub-directory for each particular framework/language type.
If a change to an analyzer changes the report output, then the expected report must change as well.
Secret Detection the expected reports are stored with the analyzer in a path corresponding to the test project in a subdirectory for the language/framework. For example, the expected report for scanning the dependencies of
java-maven is stored in gemnasium-maven/qa/expect. This way if an analyzer's behavior changes, the expectation can be changed and packaged in the same commit.
License Compliance, expectations are stored in the test project's
qa/expect directory (e.g. ruby-bundler/qa/expect).
More information about can be found in the Security Products test projects repository.
We currently do not have automated tests for OpenShift. If you want to see how a change affects the analyzer behavior on OpenShift,
you can test it using the Secure OpenShift Instance.
You can find the credentials to login to this instance in 1Password under the name
If there is not an existing test project for the feature which you would like to test, then it is recommended to mirror an existing test repository on GitLab.com. To do this, go to New Project -> Import Project -> Repo by URL and paste the Git repository URL of the repository that you would like to mirror. Enable the Mirror repository checkbox so that updates to the test respository on GitLab.com will be automatically synced to the OpenShift instance. Set the project visibility to public so that that any links that you leave to these projects on your merge requests are visible to others.
Once you have finished creating the project, go to CI/CD -> Pipelines and click the Run Pipeline button
in order to start a new pipeline. The job will be picked up by OpenShift runners installed on the instance. Optionally, you can set an env var for the analyzer image (e.g.
SAST_ANALYER_IMAGE) set to a tmp image built on a branch on gitlab.com to pull in changes and iterations.