The following page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features or functionality remain at the sole discretion of GitLab Inc.
By 2025, our vision for GitLab Runner Core is to provide GitLab Runner administrators and developers with an almost zero-friction experience. We see this vision for Runner Core as enabling the broader GitLab product vision of continuing to mature the GitLab DevOps platform rapidly.
A critical goal to enable the Runer Core zero-friction vision is to eliminate CI job execution errors and customer-reported bugs directly related to the current Runner architecture. Today, the Runner handles both the lifecycle of the CI job execution environment and the execution of the build. This comingling of concerns has increased complexity and negatively impacted our innovation velocity.
So a critical pillar in our strategy is to address these architectural limitations. With that in place, we aim to deliver a near zero-friction experience for Runner autoscaling on public cloud provider-hosted virtual machines and public cloud or on-premise Kubernetes clusters.
Check out our Ops Section Direction "Who's is it for?" for an in depth look at the our target personas across Ops. For Runner, our "What's Next & Why" are targeting the following personas, as ranked by priority for support:
| Operating Systems | Compute Architectures | |
|---|---|---|
| Linux | x86_64, ARM32, ARM64, ppc64le, s390x | |
| macOS | x86_64, M1 | |
| Windows | x86_64 |
| Container Builds | Docker, Podman, Buildah |
| Container Orchestration | Kubernetes, Red Hat Open Shift, AWS EKS, GCP ECS |
The table below represents the current strategic priorities for runner core. This list will change with each monthly revision of this direction page.
| Theme-Category | Item | Why? | Target delivery QTR |
|---|---|---|---|
| Security and Compliance | GitLab Runner Tokens Architecture Evolution | A GitLab Runner can be described as a worker process that executes the CI/CD pipeline jobs you define for your project. A runner has access to the source code in your project repository, so you must implement security controls to reduce risk. One of our strategic goals is to enhance the security architecture of the runner token mechanism to reduce security risks and simplify compliance management. | Q1-FY24 |
| Supply Chain Security | Secure Software Supply Chain - Runner Workstream | Supporting GitLab's supply chain security direction will require new features and capabilities in GitLab Runner. These include but are not limited to, generating industry standard attestations on completing a CI job and enabling support for natively signining build artifacts. | Q1-FY24 |
| Platform Enablement | Docker Machine Replacement - GitLab Runner Autoscaling plugins for public cloud providers | The Next Runner Scaling Architecture is the architectural blueprint that serves as the foundation for replacing the Docker Machine-based Runner Autoscaler for scaling Runner on public cloud-hosted virtual machines. Based on the architectural blueprint, the solution we landed on for the MVC introduces three new concepts to GitLab Runner: Fleeting, Taskscaler, and Fleeting-plugins. With its minimal API focused on VM creation, connection, and deletion, Fleeting is the abstraction layer for public cloud providers' instance groups. Taskscaler is the new GitLab-developed autoscaler for provisioning VM instances using Fleeting and autoscaling instance groups. Taskscaler allocates and assigns tasks to execute each VM. Finally, Fleeting-plugins are cloud provider specific, and it is how customers can adopt the Next Runner Scaling on the target public or private cloud of their choosing. The first plugins that we are planning to deliver are for AWS and Google Compute Engine. This prioritization of Google Compute Engine enables us to test the new architecture at scale on internal GitLab projects. However, since the abstraction layer will be well defined and documented, community contributions may help accelerate the development of plugins for the other major cloud platforms. | Q1-FY24; Q2-FY24 |
| Platform Maturity | GitLab Runner Next Generation | The near ubiquity of GitLab Runner — the build agent that takes a user's GitLab CI job and executes the job commands on a computing environment — has enabled the GitLab community and customers to use GitLab CI on many types of computing infrastructure. So while the current Runner architecture has been a solid foundation, it's becoming increasingly clear that a new approach to Runner is needed to support our user's and customers' needs in the future with the increased use and adoption of GitLab CI | Q3-FY24 |
The summary list below includes a few popular items that we have decided not to prioritize.
| Item | Why Not? |
|---|---|
| Local runner execution MVC | The gitlab-runner exec feature was the initial solution in GitLab that allowed users to validate and test changes to a pipeline without needing to run the pipeline on a GitLab instance. However, as additional features and capabilities were introduced to GitLab CI, it was no longer tenable to replicate all GitLab CI features into gitlab-runner exec. So, while this seems like a simple feature request, implementing GitLab CI job debugging in a local runner is quite complex. To summarize, we will need to duplicate the CI logic handled in the Rails application. As a result of the level of effort and complexity, we will not invest in adding this feature set to the Runner. The Verify Pipeline Authoring team is considering a feature to provide the ability to test or simulate sections of any pipeline configuration. Finally, we are planning to fully remove the gitlab-runner exec command from GitLab Runner in the 16.0 release. You can provide feedback on this removal plan in this issue. |
| Sticky Runners MVC | In this issue, users need to improve CI job performance in scenarios where each job can generate intermediate build elements hundreds of GBs in size. In the current GitLab CI model, a significant amount of pipeline execution time is due to the uploading and downloading of intermediate build elements between jobs in a pipeline. Given the current Runner executor implementation, i.e., we support several executor types out of the box (shell, docker, Kubernetes), changing the CI job execution paradigm in GitLab is a significant architectural change. One option on the table is to restrict this feature to Runners using the shell executor. The Sticky Runners MVC feature is not in the FY24 roadmap due to higher priority features in the Runner core and the Verify stage. |
| OS | Zero Friction Setup and Configuration |
|---|---|
| Linux | Viable |
| macOS | Viable |
| Windows | Viable |
| Kubernetes - Runner setup and management option | Zero Friction Setup and Configuration (Day 0) | Zero Friction Operations and Management |
|---|---|---|
| GitLab Runner Helm Chart | Viable | Viable |
| GitLab Runner Operator | Viable | Viable |
| Cloud | VM Autoscaling Maturity | Target for Lovable |
|---|---|---|
| AWS | Complete - however the solution uses legacy Docker Machine technology. | FY24 Q4 |
| GCP | Complete - however the solution uses legacy Docker Machine technology. | FY24 Q4 |
| AZURE | Complete - however the solution uses legacy Docker Machine technology. | FY24 Q4 |
Runner core is comprised of various components, features, and capabilities. This section aims to provide clarity regarding the Runner Core architecture direction at a more fine-grained level.
| Category | Strategy |
|---|---|
| Shells | Today the current philosophy behind GitLab CI/CD job execution is that everything is a shell script. The use of shell scripts for job execution has benefits. Still, there are also significant drawbacks in maintenance costs and complexity, which in some cases has negatively impacted our ability to deliver new features quickly. In this issue, Manager/Taskrunner design, which is currently confidential, we are discussing the architectural underpinning of Runner. Those discussions will guide the evolution of the core GitLab Runner CI job execution mechanism. It could become the basis of what we are currently referring to as GitLab Runner Next Generation. |
| Helm Chart | The gitLab Runner Helm Chart has been the traditional method to install GitLab Runner on Kubernetes. However, with the release of the GitLab Runner Operator and the GitLab Kubernetes agent, we need to carefully consider and define our long-term maintenance and development strategy for the Helm Chart and the Operator. The current thought process is to add critical functional pieces to the Operator to be fully on par with the Helm Chart. Then this will mean we will aim to deprecate the Helm Chart install option for GitLab Runner in the 17.0 release. Follow along with the discussion in this issue. |
When you run a continuous integration pipeline job or workflow, the code in that pipeline must execute on some computing platform to complete your software's building, testing, and deployment. Terms used to describe the software that handles the pipeline code execution include worker, agent, or runner.
So while the basic functionality of pipeline code execution is table stakes in the industry, the ability to efficiently build software on multiple compute platforms with low operational maintenance overhead is critical to the value proposition for self-managed GitLab.
For customers who need to run CI/CD workloads on environments that they manage (self-managed), GitLab runner includes a wide array of features and capabilities positioned competitively in the marketplace.
| Solution | CI/CD Agent naming convention/brand | Self-Managed Option Availablity | Notes |
|---|---|---|---|
| GitHub Actions | Runners | Available | GitHub released self-hosted runners in late 2019. Since then, GitHub has continued to invest in features and capabilities. We also notice similar investment themes as GitHub continues to target market segments requiring a self-managed platform. For example, GitHub self-hosted runners have added support for Apple M1 silicon. GitHub is also investing in self-hosted runner auto-scaling with plans to add support for Kubernetes in CY23-Q1. The autoscaling roadmap issue also includes an interesting note about queue visibility and tooling to deploy runners from the UI, which is very much related to our vision for zero friction runners. |
| Jenkins | Agent | Available | A Jenkins agent is an executable residing on a node, whether virtual, bare-metal or a container that the Jenkins controller tasks to run a job. While installing the Jenkins agent on a target platform does require Java, the agent capability enables distributed builds in Jenkins and is flexible from a deployment standpoint. The Jenkins agent architecture is scalable; however, there will be ongoing maintenance overhead for organizations that self-manage large-scale Jenkins installations. |
| Harness.io | Harness Delegate | Available | Harness currently provides the following types of Delegate: Kubernetes, Shell Script, AWS ECS, Helm, Docker. Though the Delegates perform a similar essential function as GitLab Runner, i.e., executes tasks provided by the Harness Manager, the Delegates' primary purpose is to deploy software to the target platform. In this regard, the value proposition of the GitLab Agent for Kubernetes is a critical consideration when evaluating capabilities in GitLab for developer frictionless cloud-native deployment. |
| Codefresh | Codefresh Runner | Available* | The Codefresh Runner, which handles getting tasks from the Codefresh SaaS platform and executing them, is available only for Kubernetes. |
| CircleCI | CircleCI Runner | Available | The CircleCI self-hosted runner, released in November 2020, is supported on Linux, Windows, macOS, and Kubernetes but is only available to customers on CircleCI's Scale Plan. In the near term, CirlceCI is adding support for additional platforms. Extending platform support is an expected and necessary by-product of targeting customers who cannot run CI/CD workloads on a SaaS solution. |
| Bitbucket | Runners | Available | Users can self-host Bitbucket Runners on Linux x64, Windows 2K19, or macOS Catalina. On Windows and macOS, a pre-requisite to using the runner is OpenJDK11. |
| TeamCity | Build Agent | Available | The TeamCity build agent, developed in Java uses a polling mechanism to retrieve commands from the associated TeamCity server - TeamCity Cloud or TeamCity On-Premises. TeamCity's build agent executes the build process on the target computing platform. The build agent is compatible with Linux, Windows, and macOS, and there is support for installing the build agent via the TeamCity Agent Push option from the TeamCity server. The TeamCity agent setup capabilities for Amazon EC2, Kubernetes, and VMWare vSphere are mature and easy to use. |
| Cirrus CI | Cirrus CI agent | Available | Cirrus CI supports Linux and Windows Containers, macOS, and Free BSD virtual machines. In addition, their approach to task execution, where all configuration options are in the cirrus.yml file, including the VM image version and Docker Container image version, is a compelling value proposition. For example, you can specify the specific Google Cloud compute type needed for each task in your workflow. |
| Semaphore | CI/CD agents | Available | Semaphore offers self-hosed agents for the Semaphore On-Premise solution. The agents are available for Linux, Windows, macOS and support container-based builds with Docker. Autoscaling on AWS is delivered via the Semaphore agent AWS stack, a solution that deploys a fleet of Semaphore self-hosted agents into an AWS account. |
The pace of change and innovation in DevOps is high. New entrants will likely challenge current paradigms and disrupt the market. An example of that is onedev, an open-source project that relies solely on Kubernetes to execute CI jobs with Linux and Windows containers support. The long-term potential here is clear. Kubernetes continues to be the leading container orchestration platform. Assuming that continues and organizations develop a deep bench of expertise to manage Kubernetes at scale, then we can make the following hypothesis. Having a CI/CD runner solution that is easy to install, maintain and operate on Kubernetes, coupled with predictive DevOps capabilities, will be critical to long-term market success.
So, as we head into FY24 and beyond, we will continue to focus on adding key features to Runner Core to maintain our pace of innovation and competitive position
The near features highlighted here represent just a subset of the features and capabilities that have been requested by the community and customers. If you have questions about a specific runner feature request or have a requirement that's not yet in our backlog, you can provide feedback or open an issue in the GitLab Runner repository.
This direction page was revised on: 2022-12-27