Automation is essential for successful DevOps teams, and CI/CD pipelines are a big part of that journey. At its most basic level, a pipeline gets code from point A to point B. What makes a better pipeline is how quickly and efficiently it accomplishes this task.
A CI/CD pipeline automates steps in the SDLC like builds, tests, and deployments. When a team takes advantage of automated pipelines, they simplify the handoff process and decrease the chance of human error, creating faster iterations and better quality code. Everyone can see where code is in the process and identify problems long before they make it to production.
Before we dive in, let's cover some basics:
The GitLab pipeline glossary
Commit: A code change.
Job: Instructions that a runner has to execute.
Pipeline: A collection of jobs split into different stages.
Runner: An agent or server that executes each job individually that can spin up or down as needed.
Stages: A keyword that defines certain stages of a job, such as
deploy. Jobs of the same stage are executed in parallel.
Pipelines are configured using a version-controlled YAML file,
.gitlab-ci.yml, within the root of a project. From there, you can set up parameters of your pipeline:
- What to execute using GitLab Runner
- What happens when a process succeeds or fails
Not all jobs are so simple. For larger products that require cross-project interdependencies, such as those adopting a microservices architecture, there are multi-project pipelines.
In GitLab 9.3 we made it possible to display links for upstream and downstream projects directly on the pipeline graph, so developers can check the overall status of the entire chain in a single view. Pipelines continue to evolve, and in our CI/CD product vision we’re looking into making pipelines even more cohesive by implementing Multiple Pipelines in a single
.gitlab-ci.yml in the future.
Pipeline as code
In the early iterations of CI/CD, DevOps tools set up pipelines as point-and-click or through a GUI. This originally presented a number of challenges:
- Auditing was limited to what was already built in
- Unable to collaborate
- Difficulty troubleshooting
Something as simple as rolling back to the last known config was an exercise in futility. CI/CD pipelines during this time were prone to breaking, lacked visibility, and were difficult to change.
The pipeline as code model corrected a lot of these pain points and offered the flexibility teams needed to execute efficiently. With source code, teams could use Git to search and introspect changes.
Today, many tools have adopted YAML configuration as a best practice. GitLab CI/CD has used code, rather than GUI, since the beginning for pipeline configuration. "Pipeline as code" comes with many of the same benefits the other "as code" trends have:
- Version control – keep track of changes over time and revert to previous configurations easily
- Audit trails – know when and what changes were made to the source code
- Ease of collaboration – code is available to the team for improvements, suggestions, and updates
- Knowledge sharing – import templates and code snippets so teams can share best practices
- Built-in Lint tool – ensures YAML file is valid and assists new users
The principles of software development apply not only to the applications we deliver but also to how we build them. The pipeline as code model creates automated processes that help developers build applications better and faster. Having everything documented in a source repository allows for greater visibility and collaboration so that everyone can continually improve processes, which is what DevOps is all about.
Building better pipelines with Auto DevOps
Even though CI/CD pipelines have automated so much of the development process, the initial work to build and configure them in your environment takes time. But what if you aren’t sure what all the parts of your CI/CD pipeline should be? What are the best practices you should know at every stage?
In the past, there have only been two choices: Time-consuming configuration from scratch with complete customization, or an easier auto-configuration with much less flexibility. Developers have longed for the moment where they could click a button and have a complete pipeline with code quality, language detection, and all scripts included with very little manual work.
Auto DevOps is our solution to this problem. It is a pre-built, fully-featured CI/CD pipeline that automates the entire delivery process. Instead of having to choose between time and flexibility, GitLab offers both. In addition to the Auto DevOps template, GitLab offers several CI templates that can be modified as necessary, or you can override specific settings. Want all the power of Auto DevOps for a custom test job? Just override the
script block for the
test job and give it a try. Since templates are also modular, teams have the option to pull in only the parts they need.
We hope this blog post gives you some insight into how we approach pipeline as code and our larger vision for how we’re improving the CI/CD pipeline experience in the future. Automated pipelines increase development speed and improve code quality, and we’re actively working on making them even better and easier to use.
Cover image by Gerrie van der Walt on Unsplash
“How @GitLab is making a better pipeline with Auto DevOps” – Chrissie Buchanan
Click to tweet