Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Product Section Direction - CI/CD

CI/CD Overview

The CI/CD section focuses on the following stages of the DevOps Lifecycle:

You can see how the CI/CD stages relate to each other in the following infographic:

Pipeline Infographic

CI/CD is a large portion of the DevOps market with significant growth upside. As of late 2019, the application release automation/CD and configuration management (mainly aligned to the Release stage) segment supported 984.7MM USD revenue in 2019, projected by 2022 to grow to 1617MM. The Continuous Integration, integrated quality/code review/static analysis/test automation, and environment management (mainly aligned to the Verify stage) represented an even larger 1785MM market in 2019, projected to grow to 2624MM. With a combined projected total of 4241MM by 2022, it's critical that we continue to lead here. (market data, GitLab internal link)

Team size and tracking against plan can be seen on our hiring chart. Maturity for each of the stages can be found on our maturity page.

Monthly Q&A Video

We have a monthly internal Q&A where we discuss strategy and how things are going. You can see the latest video below:

Competitive Space and Positioning

CI/CD solutions have continued to rapidly innovate. Kubernetes itself, a massive driver of industry change, just turned five years old in 2019. Docker, now considered a mature and foundational technology, was only released in 2013. We do not expect this pace to slow down, so it's important we balance supporting the current technology winners (Kubernetes, AWS, Azure, and Google Cloud) with best-of-breed, built-in solutions, while at the same time avoiding over-investment on any single technology solution that could become legacy. Our Multi-Platform Support theme highlights the technologies we're monitoring and investing in.

There are dangers to falling out of balance; Pivotal resisted embracing Kubernetes early and has suffered for it. At the same time, technologies like Jenkins that deeply embraced what are now considered legacy paradigms are enjoying the long tail of relevancy, but are having to scramble to remain relevant for the future.

Trend Towards Consolidation

Continuing apace in 2019 after Microsoft's 2018 acquisition of GitHub, the trend to consolidate DevOps companies seems here to stay. In January 2019, Travis CI was acquired by Idera, and in February 2019 we saw Shippable acquired by JFrog. Atlassian and GitHub now both bundle CI/CD with SCM, alongside their ever-growing related suite of products. In January 2020, CollabNet acquired XebiaLabs to build out their version of a comprehensive DevOps solution.

It's natural for technology markets go through stages as they mature: when a young technology is first becoming popular, there is an explosion of tools to support it. New technologies have rough edges that make them difficult to use, and early tools tend to center around adoption of the new paradigm. Once the technology matures, consolidation is a natural part of the life cycle. GitLab is in a fantastic position to be ahead of the curve on consolidation, but it's a position we need to actively defend as competitors start to bring more legitimately integrated products to market. Our Single-Application CI/CD theme is geared towards getting the most out of and defending our lead in this space.

Making Continuous Delivery Real

Our overall product positioning for CI/CD is aligned to the one set forth in Jez Humble's classic Continuous Delivery book. Reducing risk, getting features and fixes out quickly, and reducing costs/creating happier teams by removing the barriers to getting things done are principles that have stood the test of time. Modern CD pushes this even further by supporting more ways to microtarget users and incrementally make changes, and Progressive Delivery has emerged as a set of best-practices that we're embracing in our product. Still, the 5 principles at the heart of continuous delivery remain illustrative of the approaches we want to enable:

  1. Build quality in
  2. Work in small batches
  3. Computers perform repetitive tasks, people solve problems
  4. Relentlessly pursue continuous improvement
  5. Everyone is responsible

The other side of the coin here is our Do Powerful Things Easy theme - a huge focus for 2020 is to dive deep with our users on how onboarding works in the product and ensure people are doing amazing things with GitLab CI/CD as quickly as possible.

Challenges

Opportunities

Strategy

Our strategy for CI/CD right now is all about enabling easy-to-discover workflows that support doing powerful, complex CI/CD actions with a minimum of manual configuration. We want to take advantage of our single application so that, while each team may have their own views or dashboards in the product that support their day to day, the information about what is deployed where is available everywhere and to everyone, embedded naturally into the product where it's relevant. For example, a person thinking about upcoming releases may interact mostly with an environment-based overview that helps them see upcoming changes as they flow environments, but that same information exists everywhere it is relevant:

The end result is that even complex delivery flows become part of everyone's primary way of working. There isn't a context shift (or even worse, a switch into a different tool) needed to start thinking about delivery - that information is there, in context, from your first commit. The centerpiece of this strategy is our Do Powerful Things Easy theme.

Plan & Themes

Given our vision and strategy, our plan for the next year has been organized around a number of themes that support our direction:

Single Application CI/CD

DevOps tooling continues its trend towards consolidation, driven by exhaustion from the administrative effort needed to wire individual solutions together in order to get any benefit, and still never quite achieving a well-integrated user experience. Point solutions are struggling because they suffer from the fundamental problem that CI/CD is not an island: in order to compete effectively they are forced to maintain integrations with various stacks, which is expensive in terms of focus and results that they are unable to innovate as fast as competitors that are providing a unified solution. It's is a priority for us that we retain flexibility for integrations if unbundling becomes a trend or a high quality point solution emerges within any segment, but we can do this in a way that still takes advantage of our unified platform where available, and overall right now it's clear that our approach is winning.

In the end, our single application solution is what will ensure that GitLab is a Robot Cat instead of a Sheepdog. By embedding CI/CD naturally into everyone's workflows, as a first-class part of working with issues, source control, merge requests, etc., CI/CD isn't a box under someone's desk that nobody knows how to work while on vacation - it's a part of how everyone works together, and the glue that binds everyone around actually delivering.

Related Epics:

Multi-Platform Support

Improving the ability and options available for GitLab to support multiple platforms, such as Windows, Mac, ARM, and even mainframes, GPUs, TPUs, FPGAs, RISC-V, and so on, is a top priority because it enables more of our users to get be able to take advantage of the GitLab single application. A lot of the work around adding platforms is related to the GitLab Runner, which we want you to be able to run literally anywhere, and build any kind of code. You can see more about what we plan to do there on the Runner direction page. We will be adding not just support for new platforms from bring-your-own-runner, but adding these to our shared fleet as well.

Truly being multi-platform also means making sure that GitLab interacts well with other platforms and tools you use as part of your DevOps ecosystem. One important element of this for CI/CD in particular is making sure that GitLab CI/CD remains a viable, performant option even when using external repositories, but also means for example supporting different testing platforms, packaging systems, or other automation tools that tend to be platform-heavy or even platform-specific.

Related Epics:

Speedy, Reliable Pipelines

GitLab CI/CD was built from the ground up with speed and scalability in mind. This is reflected both in the product design as well as product vision (automated testing, testing all the things). To enable our users to go from monthly shipping releases to truly enabling continuous delivery, everything that we do must be concerned with the scalability and speed of continuous integration and testing. Not that sword fighting on office chairs isn't fun, but we do want people to be able to contribute at the speed of development. We will achieve this through automated tuning where possible, and faster feedback on potential issues where not.

Wall clock time for a pipeline is an important measure here. You feel this many times every day - when you make a commit and a pipeline starts, but you need to go have lunch before you see the result, that's not fun for anyway. Issues like our auto-determining the test parallelization factor and running tests likely to fail first represent how we want GitLab CI/CD to automatically find and solve these problems for you. We are building more speed into the workflow itself with features like the dependency proxy to help speed up the tax you you pay on every job invocation setting up the environment, fetching dependencies, and so on.

Related Epics:

Do Powerful Things Easily

One of the core principles of GitLab CI/CD is the idea that we want to give people powerful tools to do their job; flexibility is key, and we want to ensure that getting complex things done with small, easy-to-use, but powerful primitives is the GitLab way. This also helps us remain flexible and avoid hewing too closely to a specific technology solution, ensuring we retain our flexibility to embrace future innovation. This includes providing modern and powerful project templates for emerging technologies like ML/AI, and for popular but sometimes complex pipelines like ones for mobile application development.

An interesting measure for this is how long it takes to get to your first green build from the time you create your project, for both new and seasoned pipeline users. We are going to start tracking this as a success metric for how easy it is to get in and get your work done; GitLab's built-in functionality will make this quick and easy as possible.

Related Epics:

Progressive Delivery

Put simply, Progressive Delivery is a set of emerging CD best practices, oriented around being able to control and monitor deployments incrementally over time, and in an automated and safe way. It takes Continuous Delivery to the next level by providing more options around how and when different users get access to your new features, making software delivery a continuous activity rather than frequent but punctuated moments in time.

Additional articles on Progressive Delivery can be found on the LaunchDarkly blog, RedMonk and The New Stack, as well as our own blog posts highlighting Feature Flags and Review Apps.

Related Epics:

Compliance as Code & Secure Secrets

Compliance as Code is the idea that requirements for auditing and compliance teams can be collected with zero additional cognitive load from developers and other team members participating in software delivery. Because GitLab has an end-to-end view of the artifacts that are associated with releases (issues, merge requests, feature flags, and so on) we are uniquely positioned to provide a comprehensive view of compliance-related activity, collected on an ongoing basis throughout the build, test, and release process. Our vision here can be found in our strategy page for Release Governance.

In addition to collecting compliance evidence along the path to production, our plan is to improve the way GitLab pipelines interact with secure secrets, typically held in a Vault or other secrets store. Our plan is to first allow GitLab to integrate with your existing Vault, implemented via gitlab#28321. In the future, however, we have a broader vision for bundling Vault with GitLab and making secrets management a part of the GitLab product in general; this will expand to include storing GitLab's own secrets in an external Vault.

Related Epics:

Stages & Categories

The CI/CD section is composed of three stages, each of which contains several categories. Each stage has an overall strategy statement below, aligned to the themes for CI/CD, and each category within each stage has a dedicated vision page (plus optional documentation, marketing pages, and other materials linked below.)

Verify

The three year goals of the Verify group are:

  1. Be the tool of choice for groups that want to reduce their lead time for changes to under an hour. Part of how we will achieve this is by taking CI/CD analytics to the next level.
  2. Provide 100% automated setup, configuration, scaling and management of GitLab Runners for medium to large enterprises.
  3. Make ML/AI workflows easier, including features like adding the ability to install industry leading applications like Kubeflow to your Kubernetes clusters.

The details of how we'll achieve different elements of this can be found in the individual category pages below:

Continuous Integration (CI)

Gain the confidence to ship at blistering speed and immense scale with automated builds, testing, and out-of-the-box security to verify each commit moves you forward. This category is at the "lovable" level of maturity.

Learn moreDocumentationStrategy

Templates

Get new projects hitting the ground running with extensible, reusable templates. This category is planned, but not yet available.

DocumentationStrategy

GitLab Runner

GitLab Runner is the execution agent that is used to run your CI jobs and send the results back to GitLab. This category is planned, but not yet available.

DocumentationStrategy

Code Quality

Automatically analyze your source code to surface issues and see if quality is improving or getting worse with the latest commit. This category is at the "minimal" level of maturity.

DocumentationStrategy

Unit Testing

Unit testing ensures that individual components built within a pipeline perform as expected, and are an important part of a Continuous Integration framework. This category is at the "minimal" level of maturity.

DocumentationStrategy

Load Testing

Be confident in the performance of your changes by ensuring that they are validated against real world load scenarios. This category is planned, but not yet available.

Strategy

Web Performance

Ensure performance of your software in-browser using automated web performance testing. This category is at the "minimal" level of maturity.

DocumentationStrategy

System Testing

Modern software is often delivered as a collection of (micro)services to multiple clouds, rather than a single monolith to your own data center. Validating complex interactions to ensure reliability of the system as a whole is more important than ever. This category is planned, but not yet available.

Strategy

Usability Testing

Testing to ensure usability flows work and are actually understandable and valuable to your users. This category is at the "minimal" level of maturity.

DocumentationStrategy

Accessibility Testing

Beyond being a compliance requirement in many cases, accessibility testing is the right thing to do. This category is planned, but not yet available.

Strategy

Package

The goal of the Package group is to build a set of features that, within three years, will allow ninety percent of our customers to store all of their packages in GitLab. In order to accomplish this goal, we must:

  1. Unify the Container Registry and the Package Registry to create a seamless user experience for building, testing and releasing code. We will achieve this through improving the usability of our existing registries and standardizing authentication and permissions for each package manager, as well as creating visibility and transparency into the Package and Container Registry itself.
  2. Work our customers and the GitLab Community to drive contributions. This is the only and best way to achieve adding support for all of the most common package manager formats in a timely manner.
  3. Improve the performance and reliability of builds by reducing reliance on external dependencies.

The details of how we'll achieve different elements of this can be found in the individual category pages below:

Package Registry

Every team needs a place to store their packages and dependencies. GitLab aims to provide a comprehensive solution, integrated into our single application, that supports package management for all commonly used languages and binary formats. This category is at the "viable" level of maturity.

DocumentationStrategy

Container Registry

A secure and private registry for Docker images built-in to GitLab. Creating, pushing, and retrieving images works out of the box with GitLab CI/CD. This category is at the "viable" level of maturity.

DocumentationStrategy

Helm Chart Registry

Kubernetes cluster integrations can take advantage of Helm charts to standardize their distribution and install processes. Supporting a built-in helm chart registry allows for better, self-managed container orchestration. This category is at the "minimal" level of maturity.

Strategy

Dependency Proxy

The GitLab dependency proxy can serve as an intermediary between your local developers and automation and the world of packages that need to be fetched from remote repositories. By adding a security and validation layer to a caching proxy, you can ensure reliability, accuracy, and auditability for the packages you depend on. This category is at the "minimal" level of maturity.

DocumentationStrategy

Dependency Firewall

GitLab ensures that the dependencies stored in your package registries conform to your corporate compliance guidelines. This means you can prevent your organization from using dependencies that are insecure or out of policy. This category is planned, but not yet available.

Strategy

Jupyter Notebooks

Jupyter Notebooks are a common type of code used for data-science use cases. With GitLab you can store and version control those notebooks in the same way you store packages and application code. This category is at the "minimal" level of maturity.

DocumentationStrategy

Git LFS

Git LFS (Large File Storage) is a Git extension, which reduces the impact of large files in your repository by downloading the relevant versions of them lazily. Specifically, large files are downloaded during the checkout process rather than during cloning or fetching. This category is at the "minimal" level of maturity.

DocumentationStrategy

Release

The three year target for Release is:

The details of how we'll achieve different elements of this can be found in the individual category pages below:

Continuous Delivery

Deliver your changes to production with zero-touch software delivery; focus on building great software and allow GitLab CD to bring your release through your path to production for you. This category is at the "complete" level of maturity.

Learn moreDocumentationStrategy

Pages

Use any static site generator to create websites that are easily managed and deployed by GitLab. This category is at the "complete" level of maturity.

Learn moreDocumentationStrategy

Review Apps

Get a full production like environment for every merge request that updates on each commit. See code running and enable user acceptance testing before you merge. This category is at the "complete" level of maturity.

Learn moreDocumentationStrategy

Merge Trains

Keeping master green and ensuring the stability of collaboration on branches is vitally important. GitLab has introduced Merge Trains as an important way to accomplish this. This category is at the "viable" level of maturity.

DocumentationStrategy

Incremental Rollout

Mitigate the risk of production deploys by deploying new production code to small subset of your fleet and then incrementally adding more. This category is planned, but not yet available.

DocumentationStrategy

Feature Flags

Feature flags enable teams to achieve CD by letting them deploy dark features to production as smaller batches for controlled testing, separating feature delivery from customer launch, and removing risk from delivery. This category is at the "viable" level of maturity.

DocumentationStrategy

Release Orchestration

Management and orchestration of releases-as-code built on intelligent notifications, scheduling of delivery and shared resources, blackout periods, relationships, parallelization, and sequencing, as well as support for integrating manual processes and interventions. This category is at the "minimal" level of maturity.

DocumentationStrategy

Release Governance

Release Governance includes features such as deploy-time security controls to ensure only trusted container images are deployed on Kubernetes Engine, and more broadly includes all the assurances and evidence collection that are necessary for you to trust the changes you're delivering. This category is at the "minimal" level of maturity.

DocumentationStrategy

Secrets Management

Use Vault embedded in GitLab for managing your secrets both inside and outside of GitLab. This category is at the "minimal" level of maturity.

DocumentationStrategy

What's Next

It's important to call out that the below plan can change any moment and should not be taken as a hard commitment, though we do try to keep things generally stable. In general, we follow the same prioritization guidelines as the product team at large. Issues will tend to flow from having no milestone, to being added to the backlog, to being added to this page and/or a specific milestone for delivery.

12.8 (2020-02-22)

Verify

Package

Release

12.9 (2020-03-22)

Verify

Package

Release

12.10 (2020-04-22)

Verify

Package

Release

13.0 (2020-05-22)

Verify

Release

13.1 (2020-06-22)

Verify

Package

Release