This post is a customer story by Adfinis SyGroup.
We've decided to use GitLab CI to build Debian packages automatically. GitLab CI allows users to execute tasks based on definable events, such as Git tags.
We've created a generic Docker container which contains the base package-building tools and is used by GitLab to build the package. Updates can be easily installed in the build environment, since the Docker container can be simply replaced with a new one.
The following shows the automated packaging of the GoAccess log analysis tool. Many tools are not packaged in their latest version and thus have to be created manually.
Prepare the Debian Package
First, the files which control the building of the Debian package are created. In the case of GoAccess, these are:
debian/changelog # Changes to the package and software debian/compat # Compatibility level for debhelper debian/control # Package-specific information such as dependencies and description debian/rules # Instructions for debhelper
Debian themselves already offer detailed documentations to simplify the introduction to packaging.
Prepare the Docker Container
On a host system, a container must be prepared in which a package can then be built. Start by creating a
FROM debian:wheezy ADD setup.sh /opt/ RUN /bin/bash /opt/setup.sh
Dockerfile (official documentation) is indicated which base image is to be used. In this case, it's Debian Wheezy. After that, the
setup.sh script is copied into the
/opt/ directory of the container. In
setup.sh, the mirror which is going to be used is configured, and the most basic dependencies are installed, which can be used in any build:
#!/bin/sh # change to our own mirror echo "deb http://pkg.adfinis-sygroup.ch/debian/ wheezy main non-free contrib" > /etc/apt/sources.list echo "deb http://security.debian.org/ wheezy/updates main" >> /etc/apt/sources.list echo "deb http://pkg.adfinis-sygroup.ch/debian/ wheezy-updates main contrib non-free" >> /etc/apt/sources.list # requirements apt-get update apt-get -y install git dh-make build-essential autoconf autotools-dev
As soon as these files have been prepared, we can build the Docker container:
docker build -t generic-package-build-runner:v1
The container is now created and ready for use.
Configure GitLab CI
Now, the prepared Docker container has to be registered for the current project, in which a package is to be built:
gitlab-ci-multi-runner register \ --non-interactive \ --url "$(GITLAB_URL)" \ --registration-token "$(CI_TOKEN)" \ --description "Generic debian wheezy package build runner" \ --executor "docker" \ --docker-image "generic-package-build-runner:v1"
The GitLab URL and the CI token can be found in the GitLab project on the page Settings > Runners. Each project has its own CI token.
In order for GitLab CI to know which commands in the container should be executed, the file
.gitlab-ci.yml is created within the repository.
# Is performed before the scripts in the stages step before_script: - source /etc/profile # Defines stages which are to be executed stages: - build # Stage "build" run-build: stage: build script: - apt-get install -y libncurses5-dev libglib2.0-dev libgeoip-dev libtokyocabinet-dev zlib1g-dev libncursesw5-dev libbz2-dev - autoreconf -fvi - cp COPYING debian/copyright - dpkg-buildpackage -us -uc - mkdir build - mv ../goaccess*.deb build/ # This stage is only executed for new tags only: - tags # The files which are to be made available in GitLab artifacts: paths: - build/*
The most important part of this file is the
run-build stage. This part defines which actions are executed, when they are executed and the locations of the files created in the build.
Since a generic Docker container was created, the necessary dependencies have to be installed in the first step.
After that, the building procedure is prepared with
autoreconf. Among other things, this results in the creation of the Makefile, which is indispensable for the build. Since we're using the copyright from the package, we'll copy it to
The building process is then started with the command
dpkg-buildpackage. The package is compiled and the Debian package is created. These packages are then moved to the
build directory that was created and uploaded to GitLab.
As soon as we have a new release, a Git tag is created. This Git tag starts a new build in GitLab, which builds the package using the latest version. The package that is created will then be made available in the web interface of GitLab, where it can be downloaded.
Ideally, the packages or artifacts built should be processed automatically, for example by uploading them to a mirror. In our case, we're using a bot which, when instructed by a GitLab Webhook, downloads the artifacts onto a target server, adds them to a Aptly repository and publishes the repository, such that the process can be fully automated from package build to publication. The final result can be freely viewed on our Aptly mirror.