How GitLab is automating release generation in .gitlab-ci.yml

Why does release generation need to be automated?

Release Management, a group within the Release stage, is about unblocking users as they continuously deliver value to their customers. A part of this can be seen in how users create Release tags to track production deployments and in that adoption of the Release features, users are deploying more with GitLab. We can also see a correlation between users adopting Releases to deploy with GitLab and leveraging more paid features.

We discovered the automation of creating these releases for our users was difficult and documented the needs for Release generation from the .gitlab-ci.yml file in our epic, gitlab&2510. The highlights of the user pains around the release process include a:

1. desire to avoid leaving the CLI to complete release jobs or tasks
2. need for the tag creation to be associated with a pipeline
3. simpler way to include release creation within GitLab as a part of the CI/CD processes

Not only do users want to automatically create releases within their CI/CD, many of the other solutions such as Azure Pipelines, XebiaLabs, and Cloudbees, all feature a way to deploy and create releases from the YAML.

How did we discover that we needed a tool to create the release?

The Release Management team underwent a few calls with a GitLab Maintainer, Kamil Trzciński, to help decide the correct path forward. There were several scenarios we considered.

Option	Pros	Cons
Generate Release from Rails	1. Simple addition of token and yaml job	1. Rails would be too heavy 2. High potential for errors and slowness 3. Least performant option
Create an independent CLI tool	1. Independent and users can use it without downloading the whole Runner command 2. Can be owned by the Release team 3. The intention and domain of the CLI is very specific and users won't get confused. 4. We can have the code in any repo	1. Maybe some code duplication from gitlab-runner-helper command to auth with CI_JOB_TOKEN
Create a runner helper	1. Much quicker since it's just copy/pasting a file and changing the command name and URL	1. The user have to download the gitlab-runner-helper binary 2. Lots of actions for the user to take to use 3. Not as discoverable 4. Coupled and dependent on Runner team 5. It would require to follow the same release version scheme as GitLab Runner does

We learned quickly that Rails was not the right place to generate a release as a result of performance and we also learned the logic was too much for the GitLab Runner to handle. This was discussed in a chat with Steve Azzopardi from the Runner Team, Sean Carroll from Release Management, and of course the product teammates.

The technical takeaways from this call included:

• Runner was more fragile and likely would be taxed by the logic that creating a release from the yaml would cause
• Rails would be slow and not as performant when creating a release
• a command line tool would be extensible and allow the Release Management team to wholly own the product

Thus, the team moved forward with the independent CLI and began developing this tool.

How does the release-cli work?

The release-cli tool is written in Go and can be called directly to create a Release via the API, given the right job token and parameters are provided. The more likely way users will interact with this tool will be in the YAML file as a job:

release_upload:
  image: registry.gitlab.com/gitlab-org/release-cli:v0.1.0
  script:
    - gitlab-releaser create --name="My Release" --description="My Release description"

Steps for the release-cli

1. The release-cli will instruct the release node of .gitlab-ci.yml

• This will convert the actions of creating the release into commands and pass those to the Runner.

2. YAML exposed as Steps

• The release node of the .gitlab-ci.yml configuration file is converted into Steps and made available via API endpoint.

3. Runner calls CLI on Job success

• The Runner executes the Job, and upon success calls the Release CLI. The evaluation of the Release must be created is made by the CLI, so this step implies the Runner always calls the Release CLI at the end of a successful Job.

4. CLI retrieves Release Steps

• The Release CLI calls the Rails API to retrieve the release configuration (as Steps).

5. CLI creates Release

• The Release CLI makes an API call to Rails to create the Release.

What is next for the release-cli?

We currently have the first iteration of the release-cli complete, which looks like:

This MVC implemented via gitlab#199253, has enabled:

• use of the GitLab Release CLI docker image
• the --name and --description parameters for a Release
• calling the Releases API to create a Release with name and description
• access to all CI variables, including $CI_JOB_TOKEN for authentication to the GitLab Releases API and $CI_PROJECT_ID to create the Release

The Release Management team will expand the tool to be able to manage many tasks for the Release generation. These features include:

1. Expose :release yaml as steps via gitlab#199250
2. Support file uploads to Releases via gitlab#17838
3. Support binary files in Releases via gitlab#35893
4. Automatically create Release notes in Releases via gitlab#15563