DRY development: A cheatsheet on reusability throughout GitLab

Joe Randazzo, Noah Ing ·
Jan 3, 2023 · 7 min read · Leave a comment

More than 20 years ago, the book The Pragmatic Programmer brought attention to the DRY principle, or “Don’t Repeat Yourself." This principle is defined as every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

The main problem to solve here is minimizing duplication. As a development project is bombarded with new requests or changing requirements, DevOps teams must balance between development of net-new features or maintaining existing code. The important part is how to reduce duplicate knowledge across projects.

This tutorial explores the mechanisms throughout GitLab that leverage the DRY principle to cut down on code duplication and standardize on knowledge. To see working examples of reusability in action, take a look at this repository.

Minimizing duplication in CI/CD

include

include can be used to transform a single .gitlab-ci.yml file into multiple files to improve readability and minimize duplication. For example, testing, security, or deployment workflows can be broken out into separate templates. This also allows ownership of the files.

include:
  - template: CI/Build.gitlab-ci.yml
  - template: CI/Test.gitlab-ci.yml
  - template: CI/Security.gitlab-ci.yml
  - template: CD/Deploy.gitlab-ci.yml

YAML anchors

YAML anchors can be used to reduce repeat syntax and extend blocks of CI workflow, including jobs, variables, and scripts.

.test_template: &test_suite
  image: ruby:2.6

unit_test:
  <<: *test_suite
  script:
    - echo "Running a test here"

end_to_end_test:
  <<: *test_suite
  script:
    - echo "Running a test here"

smoke_test:
  <<: *test_suite
  script:
    - echo "Running a test here"

extends

extends is similar to anchors with additional flexibility and readability. The major difference is it can be used with includes.


.prepare_deploy:
  stage: deploy
  script: 
    - echo "I am preparing the deploy"
  only:
    - main

deploy_to_dev:
  extends: .prepare_deploy
  script:
    - echo "Deploy to dev environment"
  environment: dev

deploy_to_production:
  extends: .prepare_deploy
  script:
    - echo "Deploy to production environment"
  when: manual
  environment: production

!reference

!reference enables the selection of keyword configuration from other job sections and reuse in the current session.

.vars:
  variables:
    DEV_URL: "http://dev-url.com"
    STAGING_URL: "http://staging-url.com"

.setup_env:
  script:
    - echo "Creating Environment"

.teardown_env:
  after_script:
    - echo "Deleting Environment"

integration_test:
  variables: !reference [.vars, variables, DEV_URL]
  script:
    - !reference [.setup_env, script]
    - echo "Run Test"
  after_script:
    - !reference [.teardown_env, after_script]

performance_test:
  variables: !reference [.vars, variables]
  script:
    - !reference [.setup_env, script]
    - echo "Run Test"
  after_script:
    - !reference [.teardown_env, after_script]  

Downstream pipelines

Downstream pipelines enable the breakout of microservices and their pipelines. A .gitlab-ci.yml file can be used for each service, and when a file or directory is changed, only that pipeline needs to be triggered improving the awareness and readability of what’s deploying.

ui:
  trigger:
    include: ui/.gitlab-ci.yml
    strategy: depend
  rules:
    - changes: [ui/*]

backend:
  trigger:
    include: backend/.gitlab-ci.yml
    strategy: depend
  rules:
    - changes: [backend/*]

Dynamic child pipeline

CI/CD variables

CI/CD variables can be scoped to a specific level, including the project, group, instance level, or .gitlab-ci.yml level. The values can be stored and reused across a group for project inheritance or overwritten at the project level.

variables:
  PROJECT_LEVEL_VARIABLES: "I am first in line in precedence"
  GROUP_LEVEL_VARIABLES: "I am second in line"
  INSTANCE_LEVEL_VARIABLES: "I am in third place"
  GITLAB_CI_YML_LEVEL_VARIABLES: "I am last in line of precedence"

Creating consistent code reviews across multiple teams

Description templates

Description templates enable teams to define a consistent workflow for issues or merge requests. For example, the MR template can define a checklist for rolling out to a feature to ensure it’s documented, quality tested, and reviewed by appropriate team members. Here are MR templates that GitLab team members use daily.

<!-- These templates can be set at the instance or group level to share amongst the organization: https://docs.gitlab.com/ee/user/project/description_templates.html#set-instance-level-description-templates -->

## What does this MR do?

<!-- Briefly describe what this MR is about. -->

## Related issues

<!-- Link related issues below. -->

## Create a checklist for the author or reviewer
- [ ] Optional. Consider taking this writing course before publishing a change.
- [ ] Follow the documentation process stated here.
- [ ] Tag this user group if this applies.


<!-- Quick Actions - See https://docs.gitlab.com/ee/user/project/quick_actions.html#issues-merge-requests-and-epics for a list of all the quick actions available. -->

<!-- Add a label to assign a specific workflow using scoped labels -->
/label ~documentation ~"type::maintenance" ~"docs::improvement" ~"maintenance::refactor"

<!-- Apply draft format automatically -->
/draft 

<!-- Assign myself or a usergroup -->
/assign me

Project templates

Project templates can be used to define an initial project structure for when new services are being developed. This gives a consistent starting point for projects that come equipped with the latest file configurations and defaults.

File templates

File templates are similar to project templates but are default files to choose from when adding a new file to your repository. The team then can quickly choose from files that have best practices baked in and organization defaults.

Defining a Pipeline Center of Excellence project for CI/CD workflows

As you 'productionize' your CI/CD workflows, it’s recommended to create a “Pipeline Center of Excellence” project that contains templates, containers, or other abstracted constructs that can be adopted throughout the organization. This project contains file or CI/CD templates that have the best practices or well-formed workflows defined for development teams to quickly adopt (includes) without recreating the wheel. To explore this in practice, visit Pipeline COE documentation written by the GitLab Professional Services team.

Have a reusable component to suggest or that we missed? Add a comment to this blog post or suggest a change to this file!

Cover image by Federico Beccari on Unsplash.

“To extract the most value from DevOps, it's important not to repeat yourself. Our cheatsheet shows you how to follow the DRY concept for CI/CD.” – Joe Randazzo, Noah Ing

Click to tweet

Open in Web IDE View source