| Stage | Maturity |
|---|---|
| Manage | Minimal |
Thank you for visiting the category direction page for Value Stream Management and Analytics. This page is maintained by the Product Manager for the Analytics group.
This strategy is a work in progress and everyone can contribute by sharing their feedback directly on GitLab.com, via e-mail, or Twitter.
Over the years, software has become more powerful and specialized; however, the teams who create, build, deliver, and analyze the performance of products are more disconnected, despite the increasing time spent on updates and status reports. In order to scale and speed up the delivery of quality products, silos have to be broken but the business, product managers and engineers still use different tools glued together by imperfect integrations. We strongly believe that having a single application for the entire software development lifecycle is already a huge step forward, but we are only just starting to explore ways to surface valuable insights and recommendations, which will help organizations increase transparency and productivity across teams and connect the business with engineering.
Value Stream Mapping in Software Development focuses on understanding and measuring the value added by the flow of activities in the software development lifecycle. In the time of technological disruption we are in, success will be largely dependent on the ability of enterprises to define, connect and manage software and business value streams. Often times, this will coincide with a culture shift requiring many enterprises to adapt the way they work. At GitLab, we are making a small step towards connecting the business with engineering by using issues and MRs with labels associated with OKRs, for example.
Our first attempt at helping organizations get a better understanding of their flow was the introduction of Cycle Analytics for all customers that follow the GitLab Flow. Cycle Analytics got a lot of attention and we quickly understood that while we are striving to define best practices, different organizations have different value streams and workflows and we need to support the ability to define and measure these customized workflows in GitLab in order to move the industry forward and serve our customers better. As of June 2019, Cycle Analytics is officially part of our VSM solution.
We are building GitLab with the goal of having teams manage their entire development life cycle and we believe that an integrated solution will enable teams to be faster and more efficient. However, you cannot measure what you cannot see, so we want to make it easy for Engineering Leadership to quickly get a unified view of the end-to-end flow of how software value is created by their teams. We believe we can capture the vast majority of the processes metadata within GitLab since we already have tools for ideation, planning, design, development, testing, code review, security testing and release.
In order to ensure consistent visibility across the value stream at different levels of granularity, we want to facilitate consistent language and definitions across teams. Using GitLab events, such as issue_creation, merge_request_last_build_finished, merge_request_closed, test_job_finished, we can enable teams to define and split their value stream stages as they see fit. While we have only a few events available now, we will continue to expand that list. We will enable users to define, save and share the process reports they create together with any relevant charts. This will ensure that our customers use a single source of data to display both detailed information for daily organization and aggregated statistics for management roll-ups.
By providing customers with the ability to define their processes at different levels of granularity and to calculate the time it takes to complete each stage of the process, users should be able to quickly spot where their relative bottlenecks are. However, in order to get to the bottom of a problem, we need to allow users to drill-down. This is why we will build a catalog of configurable chart widgets, which users could quickly add to their dashboard to gain quick insights of possible causes of slow-moving stages in their process.
Let's take cycle time for our coding stage as an example. We drilled down and found out that some merge requests took 6 weeks to be completed. Why was that? Our issues were labeled as 'features', but 80% of the code seems to be refactoring! How many MRs exhibited the same problem? Was it the same file? By connecting with our code hotspots heatmap, the team leads can quickly identify potential tech debt that is slowing the team down.
Let's take code review as another example - while a certain level of code review is necessary, extensive long cycles of waiting could be a huge source of waste preventing valuable features getting to production. We are planning to provide histograms showing the time taken to incorporate merge requests, size and complexity thereof as well as other graphs displaying pipeline times and particular individuals involved. By allowing filtering by date, authors, assignees, milestones or weights an engineering manager should be able to find patterns of suboptimal operations, which they can address. In order to measure the value of said review process, one could compare the quality/performance issues and rework coming from that particular place in the codebase in the future. However, a better alternative might be an automated review process.
In general, through value stream mapping, we hope to be able to show you the steps where you can improve your practices by segregating the value added activities from those that contribute to waste. We are going to get to a stage where we compel customers to ask the important questions of how much value each activity adds compared to the time and resources it takes. By visualizing hand-offs identified from our unified data model, it should become clear where teams can improve.
Some of the questions we hope our charts can initially answer are:
Through our product, we have worked with a vast host of engineering teams, which has helped us identify common patterns of success and failure in DevOps. We are planning to encode this knowledge and given the data we enable users to store in their instance, we would like to automatically highlight process bottlenecks and generate recommendations on how to minimize their impact. We would also like to enable customers to set targets for the different stages of their process, which we can follow up on with reminders and suggestions. A couple of examples, below:
Recommendations will be ranked in order of importance by the value they bring in reducing time to production and increasing quality. Through your feedback on the quality of those recommendations and industry developments, we will continue to improve our algorithm to provide more tailored suggestions.
It's important that data and performance is evaluated with a focus on outcomes. Whether we succeed in enabling customers to visualize the bottlenecks in their process can be objectively measured by faster time to production. According to the 2018 DORA report, Elite DevOps performers take less than an hour to get from code committed to production, while low performers - anywhere between one and six months. We will show you trends of how well you are doing over time plotted against industry benchmarks or your own organizational targets. Being fast, however, should not come at the cost of wasteful coding practices creating a lot of quality and instability issues. High and elite performers in DevOps show high correlation between faster time to production and rare, quickly identified and resolved issues. We will provide KPIs coming from our monitoring stage such as MTTD, MTTR to provide the full picture of the health of your value stream.
During the minimal and viable states of the category, we aim to cater to the needs of Development Teams Leadership (primarily) and Product Managers (secondarily) in their efforts to improve velocity and predictability of their value streams. As we build out the category, Department Leads and Senior Executives will have one place where they can oversee the progress of their teams and identify best practices and negative pattens that inspire improvements across their organizations.
Since many organizations are just starting to explore analytics in order to increase efficiency and predictability in the software development cycle, we expect the category to change quickly in response to feedback.
Nevertheless, over the next year, there are a couple of big areas we would like to tackle that should enable us to deliver on our 3 year strategy.
Create a one stop analytics hub, where customers are able to aggregate and visualize information across individuals, projects, groups and subgroups
Allow customers to filter each dashboard down to specific individuals, labels, milestone, dates
Allow users to own multiple dashboards, which they can save and share
Allow customers to export more of their data in .csv format, through logs using our integration with ELK or through our API
Q2 2019 (May, Jun, Jul) 12.0
12.2
Q3 2019 (Aug, Sep, Oct)
12.3
12.4 (TBC)
12.5 (TBC, under UX and product discovery)
Q4 2019 (Nov, Dec, Jan)
Q1 2020 (Feb, Mar, Apr)
Q2 2020 (May, Jun, Jul)
This category is currently at the Minimal maturity level and our next maturity target is Viable by October 2019. Please see our definitions of maturity levels and related epics.
TaskTop is exclusively focused on Value Stream Management and allows users to connect more than 50 tools together, including Atlassian's JIRA, GitLab, GitHub, JamaSoftware, CollabNet VersionOne, Xebia Labs, and TargetProcess to name a few. Tasktop serves as an integration layer on top of all the software development tools that a team uses and allows for mapping of processes and people in order to achieve a common data model across the toolchain. End users can visualize the flows between the different tools and the data can be exported to a database for visualization through BI tools.
While we understand that not all users of GitLab utilize all of our stages, we believe that there is already a lot of information across planing, source code management and continuous integration and deployment, which can be used to deliver valuable insights.
We are starting to build dashboards, which can help end users visualize a custom-defined value stream flow at a high level and drill down and filter to a specific line of code or MR.
CollabNet VersionOne provides users with the ability to input a lot of information, which is a double edged sword as it can lead to duplication of effort and stale information when feeds are not automated. It does however, allow a company to visualize project streams from a top level with all their dependencies. End users can also create customizable reports and dashboards that can be shared with senior management.
Plutora Analytics seem to target mainly the release managers with their Time to Value Dashboard. The company also integrates with JIRA, Jenkins, GitLab, CollabNet VersionOne, etc but there is still a lot of configuration that seems to be left to the user.
Targetprocess tries to provide full overview over the delivery process and integrates with Jenkins, GitHub and JIRA. The company also provides customizable dashboards that can give an overview over the process from ideation to delivery.
Although GitPrime doesn't try to provide a value stream management solution, it focuses on productivity metrics and cycle time by looking at the productivity of a team. It exclusively uses only git data.
Naturally, Azure is working on adding analytics that can help engineering teams become more effective but it's still in very early stages. It has also recently acquired PullPanda.
Similarly to GitPrime, Code Climate focuses on the team and uses git data only.
Similarly to GitPrime, Gitalytics focuses on the team and uses git data only.
XebiaLabs' analytics are predominantly focused on the Release Manager and give useful overviews of deployments, issue throughput and stages. The company integrates with JIRA, Jenkins, etc and end users can see in which stage of the release process they are.
CloudBees DevOptics is focused on giving visibility and insights to measure, manage and optimize the software delivery value stream. It allows comparisons across teams and integrates with Jenkins and Jira and SVM /VCS solutions.
CA Agile Central combines data across the planning process in a single integrated page with custom applications available to CA Agile Central users. The applications can be installed in custom pages within CA Agile Central or on a dashboard.
Forrester's New Wave: Value Stream Management Tools, Q3 2018 uncovered an emerging market with no leaders. However, vendors from different niches of the development pipeline are converging to value stream management in response to customers seeking greater transparency into their processes.
Forrester’s vision for VSM includes:
Additional functionality, requested by clients includes:
We are just getting started, but senior management from the development organization has expressed interest in what we are building and we are looking for their feedback. Every single issue in the space we create should be something that the Engineering Executives find helpful. An initial list of issues includes:
Relevant links: Development Metrics Working Group, Development Metrics Agenda
