Gitlab APIs along with Continuous Integration can be very helpful when executing certain bulk tasks.
Consider this requirement derived from a real-world scenario
- Company XYZ possess several repositories that have been organized under a Gitlab group
- The company needs to test the building of projects in bulk using new hardware (Runner with different CPU Architecture) that will bring down execution costs, whenever the build in each of the projects fails an issue must be automatically created.
- Lastly, all the issues that were automatically created whenever a project built failed, should be collected in bulk and reported back to a Wiki
How do we test the building of those several projects and create issues and reports about its execution automatically? Let's use Gitlab CI and APIs.
1. Company groups and projects Structure
In this case, the set of projects were grouped under a single group, following this structure:
2. Automatically creating Issues leveraging Gitlab CI and API
In order to create issues using Gitlab API we will use the Issues API an example of that can use the following cURL command:
The API Call:
curl --request POST --header "PRIVATE-TOKEN:$ISSUE_API_KEY" "https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/issues?title=Build%20Failed&labels=ARMbuild&description=Project%20Tests%20Failed%20on%20ARM"
The previous Gitlab API call can be configured to be executed whenever a job fails. Let's dissect this API Call to understand its parameters so you can potentially customize it for your project environment
- Base URL: https://gitlab.com/api/v4/projects
- Project where we want to add the issue: $CI_PROJECT_ID Notice this ID is unique and corresponds to the project where the CI/CD pipeline runs
- Issues: Endpoint we use to tell Gitlab we want to add an issue to the project
- Parameters:
- Title: How we want the issue to be titled
- Labels: Helpful to group issues by label or type, They help you organize and tag your work so you can track and find the work items you’re interested in.
- Description: Field to explain the nature of the issue if needed
The request is of type POST, because we are sending data to our receiver service. For this call to be successful it requires authentication for which we will use PRIVATE-TOKEN header
The private token can be generated by following these steps How-to-generate-token
When we execute the above API call, we create an issue in the corresponding Gitlab project
Great, so once the multi-project pipeline has run, each of the projects that failed in its building stage will create an issue warning us to double check why it failed while documenting the failure and labeling it for future follow-up.
3. Automatically collecting all the issues from Gitlab Group
Thanks to Gitlab CI and APIs we can collect all the issues created and report them back, by adding this script in your pipeline stage
Let's dissect again the main API call:
curl --header "PRIVATE-TOKEN:$GROUP_ISSUE_LIST" "https://gitlab.com/api/v4/groups/9123625/issues
- Base url: https://gitlab.com/api/v4/
- Group resource: /groups/9123625
- Issues resources: /issues
The previous API call will return a json object, the one we will save as an artifact when executing our pipeline job. Notice this artifact is created and saved automatically by Gitlab CI Great! So far we created issues per failed project, and collected them all in one single step
4. Reporting back to Wiki Project
For convenience, the json report was transformed to markdown, then using the following script we publish the markdown report to the Wiki of an specific project
curl --data "format=markdown&title=$CI_JOB_ID&content=$results" --header "PRIVATE-TOKEN:$API_WIKI" "https://gitlab.com/api/v4/projects/20852684/wikis"
Let's breakdown again the API call:
- Base url: https://gitlab.com/api/v4/
- Project resource ID : /projects/20852684
- Wiki resource: /wiki
- Parameters:
- Data format: markdown. We want to publish a markdown table
- Title: Title of the Wiki entry, we use the environment variable corresponding to the CI_JOB that was executed
- Content: The markdown table generated with the issues collection
Finally, when the last API call has been executed, this is an example of the output we can get:
Let's recapitulate, by using Gitlab CI in a multi project pipeline along with APIs we were able to test and report automatically x-number of projects and its compatibility with a new hardware CPU architecture. More information about the APIs utilized for this project here:
Issues-api Collect-group-issues WikisAPI
If you’d like to see GitLab’s API in action, watch this video.
For more information, visit LEARN@GITLAB.
Cover image credit:
Cover image by Mohanan on Unsplash
“” – William Arias
Click to tweet
Guide to the Cloud
Harness the power of the cloud with microservices, cloud-agnostic DevOps, and workflow portability.
Learn more