Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Verify Stage Direction

Everyone can contribute! Feel free to comment on our async AMA issue, email Jackie Porter, and propose an MR to this page!

MANAGE SECURE PLAN RELEASE PACKAGE DEV OPS CREATE VERIFY CONFIGURE PROTECT MONITOR
Verify

Overview

What is Verify?

The Verify Stage is responsible for executing on the market needs for continuous integration. From our Continuous Integration use case:

When practicing Continuous Integration, teams collaborate on projects by using a shared repository to store, modify and track frequent changes to their codebase. Developers check in, or integrate, their code into the repository multiple times a day and rely on automated tests to run in the background. These automated tests verify the changes by checking for potential bugs and security vulnerabilities, as well as performance and code quality degradations. Running tests as early in the software development lifecycle as possible is advantageous in order to detect problems before they intensify.

Market Landscape for Continuous Integration Tools

The total addressable market (TAMkt) for DevOps tools delivering against the Verify stage is $1.5B in 2020 and is expected to grow to $3.45B by 2024 (19.93% CAGR) (i). The Verify Stage makes up 32% of the Ops market and represents a significant portion of GitLab's expanding addressable market.

GitLab Continuous Integration Current Position

The Verify Stage has continued to maintain a premium experience for individual and small teams of Software and DevOps Engineers with market share increasing each month as evidenced in our Verify product performance indicators (internal).

Delivering on the Enterprise use case is steadily increasing as evidenced in our Verify Paid user-product performance indicators (internal). To continue this growth, the Verify Stage needs to invest more in the scaling requirements for large organizations, deliver on a solution for building secure and compliant software, as well as prioritize the usability of our core CI capabilities.

SWOT

Combining GitLab's Biggest Risks with the Verify Stage perspective, the Continuous Integration vision has some significant strengths, weaknesses, opportunties, and threats to becoming the leading platform for building, testing, and optimizing code:

STRENGTHS
Internal resources to exploit
WEAKNESSES
Internal Risks to mitigate
OPPORTUNITIES
External resources to exploit
THREATS
External Risks to mitigate
We are one of the core adoption paths for our users at GitLab

Developer first approach for experiences

Meaningful insights from use of the DevOps platform
Lack of usage data-informed product decisions

Ineffectively managed Technical debt/bugs

Over indexing to Enterprise Product Management
Reduce friction between all functions of development in a single-platform

Empower developers to manage operations, quality, and security by baking those activities into GitLab
Competition is by far the largest external risk for GitLab

GitHub

Circle CI

JFrog

HashiCorp

Public cloud providers

Core competencies for Continuous Integration

As organizations migrate to a cloud-first strategy, Continuous Integration must work to adapt to any changing needs in scale, performance, and usability. The Verify Stage must simultaneously support the trend toward microservices architecture and infrastructure as code, while balancing the needs of monorepos. With new regulations appearing around secure supply chains in software and demands for baked in compliance, our offering has to deliver on those needs. In order to adequately perform on these expectations, the core competencies Continuous Integration must meet for Enterpises are as follows: build and test automation, pipeline configuration management, visibility into CI performance, and built-in compliance with security.

Competitive Landscape

Our top competitors for the Verify Stage are as follows:

  1. GitHub Actions
  2. Jenkins
  3. Cloudbees Electric flow

Secondarily, there are emerging competitors in the continuous integration space we are watching carefully:

  1. Waypoint
  2. Tekton
  3. JFrog
  4. Harness

Continuous Integration Mission

Our mission for GitLab Continuous Integration is to empower all users to easily contribute to the building, testing, and optimization of code across teams, organizations as well as the Open Source Community.

Continuous Integration Vision

From this mission, the GitLab Continuous Integration three-year vision is to become the leading platform for Software Engineers, DevOps Engineers, and Development Team leads for automatically building, testing, as well as optimizing code.

We will execute against this vision by:

  1. Delivering on the Ops as code promise via the pipelines as code experience, improving how Software alongside DevOps Engineers author pipelines and configure their .gitlab-ci.yml files.
  2. Empowering Ops for all by building in visibility to changes made to code before production, CI/CD minute consumption, as well as natively compliant, secure supply chain workflows to support the building, testing, and optimizing of code from a single CI tool.
  3. Leverage the data available in the end-to-end GitLab DevOps platform to inform optimizations that will drive speedy, reliable pipelines.
  4. Growing GitLab.com users by investing in GitLab Hosted First and scaling our capabilities to meet Enterprise-level needs.

Continuous Integration Jobs To Be Done

Each of our product groups have specific JTBDs:

  1. Pipeline Execution JTBD
  2. Pipeline Authoring JTBD
  3. Testing JTBD
  4. Runner JTBD

Some of the core JTBDs for our three year vision and strategies are as follows:

Strategy

Verify

Enabling the Continuous Integration use case is currently spread across four product groups (Pipeline Authoring, Pipeline Execution, Testing, and Runner). In order to define our top focuses and intiatives, the Verify Stage needs to have a concerted perspective on what GitLab will offer for Continuous Integration. We are targeting the Ops Section Direction personas and will support these users with our Continuous Integration mission, vision, and one-year direction.

One-year direction

The Continuous Integration vision is a longer term investment. In the next year, we expect to make progress on the following categories to support our target of being the leading CI platform of choice:

  1. Pipeline Authoring - Unlock Ops as code and Operations for all by reducing friction for authoring, linting, and building .gitlab-ci.yml files.
  2. Merge Trains - Help organizations obtain speedy, reliable pipelines by making the Merge Train experience "Complete".
  3. Runner Cloud - Lay the foundation for scale by focusing on GitLab Hosted First] to support speedy, reliable pipelines at large organizations.
  4. Continuous Integration - Move this category to "Lovable" by unblock the Enterprise adoption of CI by investing in the pipelines as code experience and CI/CD minute visibility to inform team decisions on pipeline optimizations, delivering an Ops for all capability.
  5. CI Scaling - Execute on a GitLab Hosted First to extend the upper bounds of jobs, pipelines, and queuing limits on GitLab.com
  6. Review Apps - Invest in the Ops for all capability for teams to preview changes to code before deploying.
  7. Build Artifacts - Support GitLab Hosted First by empowering users to manage artifacts easily.
  8. Runner Enterprise Management - Drive optimizations for pipelines to help GitLab Admins and DevOps Engineers achieve speedy, reliable pipelines with a single view for managing Runners.
  9. Code Testing and Coverage - Build on the smart feedback loop created within pipelines of test results and coverage and use that data to surface trends from tests to software engineers.

What's next

What we aren't focused on now

There are important things we won't work on to focus on our one year plan.

  1. Jenkins Importer - While Jenkins is definitely apart of the competitive landscape, in the year we need to prioritize our scale and position against GitHub actions.
  2. Usability Testing - By focusing on Review Apps, we hope to lay the foundation for user testing and then improve the Visual Reviews later.
  3. Accessibility Testing - There has not been immense demand for supporting Accessibility Testing in the product. More research is needed to understand the impact this offering will have on our user base.
  4. Performance Testing - Our viable offering seems to be meeting the base requirements for users and more research is needed to validate this requirements for this category.

Verify Investment Cases

In the Verify Stage, we have several investment cases highlighted as top priorities for future investment in the Ops Section. Additionally, there are investment cases beyond the ones identified stack ranked by potential return on investment as indicated by priority below:

  1. Priority 1 Investment Cases
  2. Priority 2 Investment Cases
  3. Priority 3 Investment Cases

Verify Stage Categories

Continuous Integration (CI)

Gain the confidence to ship at blistering speed and immense scale with automated builds, testing, and out-of-the-box security to verify each commit moves you forward. This category is at the "complete" level of maturity.

Learn moreDocumentationDirection

Continuous Integration Scaling (CI Scaling)

Watch us reach 20M builds per day on GitLab.com.

DocumentationDirection

Pipeline Authoring

Category containing features associated with editing/authoring your .gitlab-ci.yml.

Learn moreDocumentationDirection

GitLab Runner

GitLab Runner is the execution agent that is used to run your CI jobs and send the results back to GitLab. This category is at the "lovable" level of maturity.

DocumentationDirection

Jenkins Importer

The Jenkins Importer helps unblock teams from migrating to GitLab CI. This category is at the "viable" level of maturity.

Direction

Code Testing and Coverage

Code testing and coverage ensure that individual components built within a pipeline perform as expected, and are an important part of a Continuous Integration framework. This category is at the "viable" level of maturity.

DocumentationDirection

Build Artifacts

Keep build artifacts under control and actionable.

DocumentationDirection

Performance Testing

Be confident in the performance of your changes by ensuring that they are validated against real world scenarios. This category is at the "minimal" level of maturity.

DocumentationDirection

Usability Testing

Testing to ensure usability flows work and are actually understandable and valuable to your users. This category is at the "minimal" level of maturity.

DocumentationDirection

Accessibility Testing

Beyond being a compliance requirement in many cases, accessibility testing is the right thing to do. This category is at the "viable" level of maturity.

DocumentationDirection

Merge Trains

Keeping master green and ensuring the stability of collaboration on branches is vitally important. GitLab has introduced Merge Trains as an important way to accomplish this. This category is at the "viable" level of maturity.

DocumentationDirection

Review Apps

Get a fully functional pre-production environment for every merge request that updates on each commit. See code running, and enable user acceptance testing and automated smoke tests before you merge. This category is at the "complete" level of maturity.

Priority: low • Learn moreDocumentationDirection

Pricing

The Verify group is at the entrypoint to the funnel in our user adoption journey, so our features are an critical enabler for users seeking a complete DevOps platform. While we do try to drive adoption in order to support multi-stage growth at least partially through features at the free tier, it's also important for us that we have features at the paid tiers. For our group, these will typically be cross-department and cross-company views related to CI templates, quality and pipelines. See below for the how we are thinking about each of the tiers, and the kinds of features that will be included.

Free

The foundation of the Free strategy for Verify is that enhancements to baseline CI YAML features will be available in this tier by default. The rationale for this approach is that we want to preserve a consistent experience. Users must always be able to use the same .gitlab-ci.yml in all tiers.

Beyond this, features that drive broad adoption and help with onboarding (including generally making it easier to get started with GitLab CI) are also good candidates for inclusion in this tier. Providing solutions to simplify the deployment and management of Runners at scale for self-managed is also critical for all tiers of users.

Premium

The default paid tier for enterprises, Premium will cater to directors operating a medium to large instance. We will focus on features that solve for typical entry-level enterprise needs: reporting and analytics supporting Ops Insights, operational efficiency driving effective project management, and supporting visibility in consumption related to 10,000 CI/CD minutes per month medium to large organizations.

You can see features that we are considering including in this tier in this issue search.

Ultimate

Using GitLab CI across hundreds or thousands of projects in large enterprises means a greater need for portfolio management and consistency of experience in development practices. This is accomplished by making templates discoverable via gitlab#320698 and a consistent authoring experience such as in these issues. Additionally, the larger the organization the greater the need for regulation and compliance which is where features like Protected Runners captured in gitlab&6058 or better integrations with Compliance Pipelines become especially attractive. Lastly, our core promise for Ultimate tier is enabling users to effectively monitor and best use their 50,000 CI/CD minutes in visualizations such as a energy consumption dashboard via gitlab#325852.

You can see features that we are considering including in this tier in this issue search.

Letters from the Editor

As a part of our annual planning, we want to transparently convey our progress to our goals and what lies ahead for each of the groups in the Verify Stage. We use a format called "Letter from the Editor", a play on the Letter to the Editor, which positions the writer of this letter to the community of GitLab rather than the community writing to GitLab. We use this method to broadcast our priorities and updates in the product group.

Pipeline Execution Letter from the Editor

Pipeline Authoring Letter from the Editor

Runner Letter from the Editor

Testing Letter from the Editor

Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license