When you develop software that users rely on, effective communication about changes with each release is essential. By keeping users informed about new features and any modifications or removals, you ensure they maximize the software's benefits and avoid encountering unpleasant surprises during upgrades.
Historically, creating release notes and maintaining a changelog has been a laborious task, requiring developers to monitor changes externally or release managers to sift through merge histories. With the GitLab Changelog API, you can use the rich history provided in our git repository to easily create release notes and maintain a changelog.
In this post, we'll delve into automating releases with GitLab, covering the generation of release artifacts, release notes, and a comprehensive changelog detailing all user-centric software modifications.
Releases in GitLab
First, let's explore how releases work in GitLab.
In GitLab, a release is a specific version of your code, identified by a git tag, that includes details about changes since the last release (and release notes) and any related artifacts built from that version of the code, such as Docker images, installation packages, and documentation.
You can create and track releases in GitLab using the UI by calling our Release API or by defining a special release job inside a CI pipeline. In this tutorial, we'll use the release job in a CI/CD pipeline, which allows us to extend the automation we're using in our pipelines for testing, code scanning, etc. to also perform automated releases.
To automate our releases, we first need to answer this question: Where are we going to get the information on changes made for our release notes and our changelog? The answer: Our git repository, which provides us with a rich history of development activity through commit messages and merge commit history. Let's see if we can leverage this rich history to automatically create our notes and changelogs.
Introducing commit trailers
Commit trailers are structured entries in your git commits, created by adding simple <HEADER>:<BODY> format messages to the end of your commit. The git CLI tool can then parse and extract these for use in other systems. An example you might have already used is git commit --sign-off to sign off on a commit. This is implemented by adding a Signed-off-by: <Your Name> trailer to the commit. We can add any arbitrary structured data here, which makes it a great place to store information that could be useful for our changelog.
In fact, if we use a Changelog: <added/changed/removed> trailer in our commits, the GitLab Changelog API will parse these and use them to create a changelog for us automatically!
Let's see this in action by making some changes to a real codebase and performing a release, and generating release notes and changelog entries.
Our example project
For the purposes of this blog, I'm using a simple Python web app repository. Let's pretend Version 1.0.0 of the application was just released and is the current version of the code. I've also created a 1.0.0 release in GitLab, which I did manually because we haven't created our automated release pipeline yet:
Making our changes
We're in rapid development mode, so we're going to be working on releasing Version 2.0.0 of our application today. As part of our 2.0.0 release, we're going to be adding a new feature to our app: A chatbot! And we're also going to be removing the quantum blockchain feature, because we only needed that for our first venture capital funding round. Also, we're going to be adding an automated release job to our CI/CD pipeline for our 2.0.0 release.
First, let's remove unneeded features. I've created a merge request that contains the necessary removals. Importantly, we need to ensure we have a commit message that includes the Changelog: removed trailer. There's a few ways to do this, such as including it directly in a commit, or performing an interactive rebase and adding it using the CLI. But I think the easiest way in our situation is to leave it until the end and then use the Edit commit message button in GitLab to add the trailer to the merge commit like so:
If you use this method, you can also change the merge commit title to something more succinct. I've changed the title of my merge commit to 'Remove Unused Features', as this is what will appear in the changelog entry.
Next, let's add some new functionality for the 2.0.0 release. Again, all we need to do is open another merge request that includes our new features and then edit the merge commit to include the Changelog: added trailer and edit the commit title to be more succinct:
Now we're pretty much ready to release 2.0.0. But we don't want to create our release manually this time. So before our release we're going to add some jobs to our .gitlab-ci.yml file that will perform the release for us automatically, and generate the respective release notes and changelog entries, when we tag our code with a new version like 2.0.0.
Note: If you want to enforce changelog trailers, consider using something like Danger to perform automated checks for MR conventions.
Building an automated release pipeline
For our pipeline to work, we need to create a project access token that will allow us to call GitLab's API to generate changelog entries. Create a project access token with the API scope, and then store the token as a CI/CD variable called CI_API_TOKEN. We'll reference this variable to authenticate to the API.
Next, we're going to add two new jobs to our gitlab-ci.yml file:
prepare_job:
stage: prepare
image: alpine:latest
rules:
- if: '$CI_COMMIT_TAG =~ /^v?\d+\.\d+\.\d+$/'
script:
- apk add curl jq
- 'curl -H "PRIVATE-TOKEN: $CI_API_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/changelog?version=$CI_COMMIT_TAG" | jq -r .notes > release_notes.md'
artifacts:
paths:
- release_notes.md
release_job:
stage: release
image: registry.gitlab.com/gitlab-org/release-cli:latest
needs:
- job: prepare_job
artifacts: true
rules:
- if: '$CI_COMMIT_TAG =~ /^v?\d+\.\d+\.\d+$/'
script:
- echo "Creating release"
release:
name: 'Release $CI_COMMIT_TAG'
description: release_notes.md
tag_name: '$CI_COMMIT_TAG'
ref: '$CI_COMMIT_SHA'
assets:
links:
- name: 'Container Image $CI_COMMIT_TAG'
url: "https://$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA"
In the above configuration, the prepare_job uses curl and jq to call the GitLab Changelog API endpoint and then passes this to our release_job to actually create the release. To break it down further:
- We use the project access token created earlier to call the GitLab Changelog API, which performs the generation of the release notes and we store this as an artifact.
- We're using the
$CI_COMMIT_TAGvariable as the version. For this to work, we need to be using semantic versioning for our tags (something like2.0.0for example), so you'll notice I've also restricted the release job using arulessection that checks for a semantic version tag.- Semantic versioning is required for the GitLab Changelog API to work. It uses this format to find the most recent release to compare to our current release.
- We use the official
release-cliimage from GitLab. The release-cli is required to use thereleasekeyword in a job. - We use the
releasekeyword to create a release in GitLab. This is a special job keyword reserved for creating a release and populating the required fields. - We can pass a file as an argument to the
descriptionof the release. In our case, it's the file we generated in theprepare_job, which was passed to this job as an artifact. - We've also included our container image that is being built earlier in the pipeline as a release asset. You can attach any assets you'd like from your build process, such as binaries or documentation by providing a URL to wherever you've uploaded them earlier in the pipeline.
Performing an automated release
With this setup, all we need to do to perform a release is push a tag to our repository that follows our versioning scheme. You can simply push a tag using the CLI, this example uses GitLab's UI to create a tag on the main branch. Create a tag by selecting Code -> Tags -> New Tag on the sidebar:
On creation, our pipelines will start to execute. The GitLab Changelog API will automatically generate release notes for us as markdown, which contains all the changes between this release and the previous release. Here's the resulting markdown generated in our example:
## 2.0.0 (2023-08-25)
### added (1 change)
- [Add ChatBot](gl-demo-ultimate-bridley/super-devsecops-incorporated/simply-notes-release-demo@0c3601a45af617c5481322bfce4d71db1f911b02) ([merge request](gl-demo-ultimate-bridley/super-devsecops-incorporated/simply-notes-release-demo!4))
### removed (1 change)
- [Remove Unused Features](gl-demo-ultimate-bridley/super-devsecops-incorporated/simply-notes-release-demo@463d453c5ae0f4fc611ea969e5442e3298bf0d8a) ([merge request](gl-demo-ultimate-bridley/super-devsecops-incorporated/simply-notes-release-demo!3))
As you can see, GitLab has extracted the entries for our release notes automatically using our git commit trailers. In addition, it's helpfully provided links back to the merge request so readers can see more details and discussion around the changes.
And now, our final release:
Creating the changelog
Next, we want to update our changelog (which is basically a collated history of all your release notes). You can use a POST request to the changelog API endpoint we used earlier to do this.
You can do this as part of your release pipeline if you like, for example by adding this to the script section of your prepare job:
'curl -H "PRIVATE-TOKEN: $CI_API_TOKEN" -X POST "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/changelog?version=$CI_COMMIT_TAG"
Note that this will actually modify the repository. It will create a commit to add the latest notes to a CHANGELOG.md file:
And we are done! By utilizing the rich history provided by git with some handy commit trailers, we can leverage GitLab's powerful API and CI/CD pipelines to automate our release process and generate release notes for us.
