Product Vision - Verify

This is the product vision for Verify. The Verify stage of the DevOps pipeline covers the CI lifecycle as well as testing (unit, integration, acceptance, performance, etc.). Our mission is to help developers feel confident in delivering their code to production. Please read through to understand where we're headed and feel free to jump right into the discussion by commenting on one of the epics or issues, or by reaching out directly to our product manager leading this area: Jason Lenny (E-Mail | Twitter)

Continuous Integration Landscape

It's an exciting time in the world of Continuous Integration. Technologies like Kubernetes have created a huge splash and are driving innovation forward; serverless, microservices, and cloud native in general represent important evolutions as well. Monitoring technology also continues to advance, making the promise of technologies like automated rollbacks based on impact a reality.

We also know that Continuous Delivery is a journey - we have users everywhere on the spectrum from facing transformational challenges moving away from legacy stacks all the way to those looking to squeeze the last bits of efficiency out of highly automated DevOps delivery platforms. By delivering our features more purposefully in the context of DevOps maturity levels, we are going to be able to do better bringing everyone on the journey to DevOps success.

The Journey to DevOps Maturity

We're really taking the idea of bringing GitLab users on the CI/CD journey seriously, and have used the great model here for our inspiration (though we have modified it slightly and will continue to do so over time.)

  Advanced Level Intermediate Level Baseline Level Beginner Level
Process & Organization Self organised and cross functional: The team can solve any task, releases continuously, and is doing continuous improvement. DevOps! Pull-based process: Measurement and reduction of cycle time. Continuous focus on process improvement. Always production ready code. Agile 101: Improved communication with business. Releases after each iteration. Developers have access to production logs. Silo organisation: People who in some way depend on each others work do not have effective ways of working together. Infrequent releases. Developers do not have access to production logs.
Technology Loosely coupled architecture: It is easy to replace technology for the benefit of something better (Branch by abstraction). Simple technology: In depth knowledge about each technology; why it is used and how it works. All of the technologies used are easy to configure and script against. Technology that makes it simple to roll back and forth between database versions. Best-of-breed: Chooses technology stack based on what is best for each purpose. Preference for Open Source. Avoids products that causes vendor lock-in. "Enterprise" infrastructure: Technology that can only be configured via a GUI. Large "enterprise" suites that do not work well in tandem and don't deliver what was promised.
Quality Assurance All testing automated: Almost all testing is automated, also for non-functional requirements. Testing of infrastructure code. Health monitoring for applications and environments and proactive handling of problems. Automated functional tests: Automated acceptance and system tests. Tests are written as part of requirements specification. All stakeholders specify tests. Automated technical tests: Automated unit and integration tests. Manual testing: Test department. Testing towards the end, not continuously. Developers do not test.
Deployment Routines One-click deploy: Everybody (including the customer) can deploy with one click. 0-downtime deploy. Feedback on database performance and deployment for each release. Automated deploy: Same process for deploy to all environments. Feature toggling to switch on/off functionality in production. Release and rollback is tested. Database migration and rollback is automated and tested for each deploy. Database performance is monitored and optimised. Repeatable deploy: Documented and partially automated deploy. Database changes are scripted and versioned. Manual deploy: Deployments require many manual steps. Manual and unversioned database migrations.
Configuration Management Infrastructure as code: Fully automated provisioning and validation of environments. Orchestration of environments. Application configuration control: All application configuration in version control. The application is configured in one place. Self service of development- and test environments. Dependency control: Dependencies and libraries are defined in version control. Manual configuration: Manual configuration in each environment and on each server.
Build & Continuous Integration Build/deploy pipeline: Same binary is deployed to all environments. Continuous improvement and automation of repeating tasks. Optimised for rapid feedback and visualisation of integration problems. Continuous integration: Continuous integration of source code to mainline. All changes (code, configuration, environments, etc.) triggers the feedback mechanisms. Artifact repository. Reuse of scripts and tools. Generation of reports for the build. Builds that fail are fixed immediately. CI-server: Automation of builds/tests on CI server. Can recreate builds from source code. Manual routines: Manual routines for builds. Lack of artifact repository. Lack of reports.

Important Concepts

In order to accelerate CI in this new world, there are a few particular ideas we are keeping close as north stars to guide us forward:

☁️ Innovating with Cloud Native Capability

Our platform must stay current with evolving trends in platform architecture. Microservices, Kubernetes, and Serverless will continue to lead the way here, and our CI/CD solutions must address the unique needs of these approaches by offering solutions that facilitate the technological and cultural transformations these teams are going through. These technologies represent a wave driving DevOps forward, and we want to be on the crest of that wave helping companies to deliver using GitLab.

💡 Delivery Insights to Unlock DevOps Success

As Peter Drucker says, "if you can't measure it - you can't improve it." Using the data in our CI/CD platform to help teams get the most out of their delivery pipeline gives us a unique advantage in offering DevOps insights to our users. Where competitors must integrate with a variety of other tools, attempting to normalize and understand data structures that can change at any time, we build a single application solution where process, behavioral, and other insights can flow seamlessly throughout, facilitating organizational transformation. Value stream mapping, wait time, retries, failure rate, batch size, job duration, quality, resource usage, throughput; these (and more) are all great metrics we already have in the system and can increase visibility to.

❤️ More Complete (Minimally Lovable) Features to Solve Complex Problems

V1 feature iterations are how we build software, but at the same time we need to continue to curate those features that have proven their value into complete, lovable features that exceed our users expectations. We will achieve this by growing individual features, solving scalability challenges that larger customers see, and providing intelligent connections between individual features. Doing this lets us solve deeper, more subtle problem sets and - by being focused on real problems our users face - we'll leave the competition behind.

Upcoming Focus

To achieve our goals in the CI domain, we're looking at making big investments over the medium term in several key areas, including the following. We've aligned each quarter in 2019 to a step along the journey where we'll focus on improving that part of the path, though that doesn't mean we aren't looking at the big picture the whole year - these just represent periods where we're giving these areas special attention.

What we hope to achieve with this sequencing is taking our existing capabilites, which meet the need of a variety of users but tend to focus on baseline/intermediate capability, and pushing them forward with advanced, deep features to drive the entire product forward in the first quarter. Then, over the remainder of 2019, we'll peel the onion back ensuring there's a breadcrumb trail for all users to reach that level of maturity (not just in these features, but for all advanced features in GitLab CI/CD.)

We have a strong focus heading into each quarter:

After Q4 we've of course already begun thinking about where the product is headed. Watch this space soon for updates on where we see the product going in 2020 and beyond.

Finally, it's important to mention that this is our vision on the product at this time. All of this can change any moment and should not be taken as a hard commitment, though we do try to keep things generally stable and not change things all the time.

Prioritization Process

In general, we follow the same prioritization guidelines as the product team at large. Issues will tend to flow from having no milestone, to being added to the backlog, to being added to this page and/or a specific milestone for delivery.

You can see our entire public backlog for Verify at this link; filtering by labels or milestones will allow you to explore. If you find something you're interested in, you're encouraged to jump into the conversation and participate. At GitLab, everyone can contribute!

Issues with the "direction" label have been flagged as being particularly interesting, and are listed in the sections below.



In addition to the scheduled items below, we also track several key categories within the Verify stage. You can read more about our strategy for each of these at the following epics:

11.5 (2018-11-22)

11.6 (2018-12-22)

11.7 (2019-01-22)

11.8 (2019-02-22)

12.0 (2019-03-22)

12.1 (2019-04-22)

12.2 (2019-05-22)

12.3 (2019-06-22)

2019 Q3

2019 Q4

Other Interesting Items

There are a number of other issues that we've identified as being interesting that we are potentially thinking about, but do not currently have planned by setting a milestone for delivery. Some are good ideas we want to do, but don't yet know when; some we may never get around to, some may be replaced by another idea, and some are just waiting for that right spark of inspiration to turn them into something special.

Remember that at GitLab, everyone can contribute! This is one of our fundamental values and something we truly believe in, so if you have feedback on any of these items you're more than welcome to jump into the discussion. Our vision and product are truly something we build together!