The following page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features or functionality remain at the sole discretion of GitLab Inc.

Introduction and how you can help

Thanks for visiting this category page on Continuous Verification in GitLab. This page belongs to the Respond group of the Monitor stage, and is maintained by Alana Bellucci who can be contacted directly via email. This vision is a work in progress and everyone can contribute. Sharing your feedback directly on issues and epics at GitLab.com is the best way to contribute to our vision. If you’re a GitLab user and have direct knowledge of your need for continuous verification, we’d especially love to hear from you.

Overview

Continuous verification is the collection and analysis of observability data to ensure the application performance is within predetermined quality limits after a deployment. We plan to enable teams to practice continuous verification alongside GitLab's CI/CD, using GitLab's new observability capabilities. We also recognize that teams have alternative preferences for their monitoring or observability platform. We plan to start building a vendor-agnostic experience that enables the same continuous verification experience all within GitLab.

Vision

GitLab Continuous Verification enables users to confidently verify successful deployments or quickly identify anomalies. Users are already using GitLab Observability or another third party tool to monitor their software. With Continuous Verification users get the added benefit from using monitoring tools under a single UI, and can efficiently deploy with confidence. Additionally, GitLab machine learning helps users automate deployments, minimizing urgent human intervention.

Challenges

As we invest R&D in building a Continuous Verification solution at GitLab, we are faced with the following challenges:

Users may manually monitor multiple tools (ex: DynaTrace, AppDynamics, Jira) and make the connection around issues that arise with deployments.
Monitoring vendors position themselves as the single pane of glass and many already offer before and after deployment data. We are tracking how different monitoring vendors know about deployments.
Continuous verification is dependent on multiple third party monitoring vendors, we need an integration path and prioritization strategy from the beginning.

Opportunities

As a DevOps platform we are uniquely positioned to take advantage of the following opportunities:

One of the most important things to monitor is when something changes. With GitLab Continuous Verification, users will have a single interface to identify the change, manage (multiple monitoring vendors), respond to, and remediate issues that arise post deployment in GitLab.
Continuous Verification allows Platform Engineers and Release Managers to automate repetitive portions of deployment by taking advantage of observability data.
Machine learning models rely on observability data. These models can predict failures, proactively enable remedial automation, and empower teams to run pre-defined playbooks.

Target Audience and Experience

Continuous verification is built for Ingrid (Infrastructure Operator) and Sasha (Software Developer).

Ingrid uses GitLab continuous verification to set up the 3rd-party observability vendors integrations. She builds and enables the deployment automation, taking advantage of the observability data, so that her development teams can take advantage of safe and efficient deployments.

When Sasha is executing a deployment for her application, she uses GitLab to monitor the application in GitLab. She knows when things are going well or when things need attention. After she's manually deployed a few times, she enables the automatic deployment step because she's confident the deployment will be successful.

Jobs to be Done

Continuous Verification

When deploying an application, I want to verify that the application is in a performant and healthy state, so I can be assured that the deployment isn't causing any abnormalities.

Job statements	Maturity	Confidence	Source
When planning to deploy changes to an application, I want to verify that the application is currently performant and in a healthy state, so that I can have confidence that known issues won't interfere with my deployment.		Researched	Issue
Immediately after a deployment, I want to verify that the application is performant and in a healthy state, so that I can automate the rollout or rollback of the changes.		Researched	Issue

Strategy

Today - (FY23Q1)

Validate the requirements for a viable Continuous Verification solution is in GitLab.

Through problem validation, analyst research and feedback from internal users, we know the requirements for the first iteration of Continuous Verification.
Designs are ready for engineering to complete a technical breakdown and start work on development.

How are we tracking success?

Each of the items outlined above are complete.
We are receiving internal, customer, and community feedback on issues, MRs and direction pages.

Near Term - (6 months - 1 year, ~ FY23Q3 - FY24Q1)

Continuous Verification is in `viable` maturity.

There is a viable solution for Continuous Verification in GitLab

How are we tracking success?

Increase the number of deployments per customer
Our deployment team is dogfooding our solution and providing feedback

Maturity Plan

We are currently working to mature the Continuous Verification category from planned to viable. Definitions of these maturity levels can be found on GitLab's Maturity page. While we are still working on validating the requirements for this first iteration, we'd love your input on the Continuous Verification: Viable Maturity Plan epic.

What is Next & Why?

For our first minimum viable change (MVC) we plan to create a heatmap of a specified time period that looks at GitLab alerts.

Marketing & Sales Enablement

As we work towards making Continuous Verification viable, we will host feedback sessions with internal teams to iterate on requirements. Once Continuous Verification is viable, we will work with Product Marketing on an email campaign, primary feature release post and use cases.

Pricing

Functionality	Free	Premium	Ultimate
GitLab Alerts	✅	✅	✅
Integration with one or more observability tool		✅	✅
Actionable automation		✅	✅
Supervised machine learning			✅

Competitive Landscape

Harness - founded in 2016, Harness has their own solution for Continuous Verification
Datadog - founded in 2010, Datadog has a deployment tracking solution
Verica - founded in 2018, Verica is using the term continuous verification which proactively discovers system weaknesses and security flaws

Analyst Landscape

As of early 2022, Continuous Verification doesn't have it's own magic quadrant, but the term "continuous verification" is seen in one-off mentions throughout publications. We will be working with analysts to better understand what the trends are for observability data are around automation and machine learning. These will help inform future iterations of GitLab's solution for Continuous Verification.

Category Direction - Continuous Verification