Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Category Direction - Continuous Integration Scaling

Continuous Integration Scaling

In FY22, we have committed to SaaS First. In the Verify Stage, this means prioritizing the scale of Continuous Integration and ensuring our users on GitLab.com are leveraging a reliable and available service. We are focused on a goal of 20M builds per day as it represents not only a target to drive our future architecture, but a volume that we expect to achieve within several quarters.

Investment Allocation

Currently, we are staffing this Category with 10% of the Verify output, or equivalent to ~3 engineers in Verify. After we establish the load testing effort, we will evaluate if additional headcount will be required to support this effort.

We are expecting to on-ramp additional headcount in FY22-Q3 per product#2178, raising the percentage to ~24% of the Verify headcount.

Additional Resources

For specific information and features related to authoring/defining pipelines, check out Pipeline Authoring. You may also be looking for one of the following related product direction pages: GitLab Runner.

What's Next & Why

To begin working towards our goal, we will focus on the challenges section outlines in the Next CI/CD scale target: 20M builds per day by 2024 blueprint. These challenges revolve around various known issues with the ci_builds table. Challenges are listed below with related epics or issues capturing the work to be done. They are listed in priority order.

  Challenge Related Epics & Issues GitLab.com Self-Managed
1 We are running out of the capacity to store primary keys gitlab#325618 Complete Complete
2 Queuing mechanisms are using the large table gitlab&5909 14.4 14.5
3 Moving big amounts of data is challenging gitlab&6009 Complete Complete
4 The table is too large gitlab#328603 14.5 14.6
5 Development velocity is negatively affected TBD TBD  
6 Create the capability to load testing the GitLab CI system* gitlab#328736 TBD TBD

Roadmap View

Queue Improvements PI

As the new queuing mechanism rolls out, performance changes will be visible in the Duration of the builds queue retrieval using the big query SQL graph in Prometheus.

Vision Items

Our top vision items we have defined include:

  1. Improve GitLab CI/CD data model
  2. Improve Runner job queuing
  3. GitLab Runner Autoscaler architecture
  4. Create the capability to load testing the GitLab CI system
Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license