Dev Section Performance Indicators

1. You are here:
2. Handbook
3. Product Handbook
4. Dev Section Performance Indicators

On this page

• Section PI Overview
• Regular Performance Indicators
  • Dev - Section CMAU - Estimated & Recorded Sum of Dev Section SMAU
  • Manage, Manage:Optimize - SMAU, GMAU - MAU Viewing Analytics Features
  • Manage:Access - Paid GMAU - MAU Using Paid SAML
  • Manage:Compliance - Paid GMAU - MAU Viewing Compliance Features
  • Manage:Compliance - Other - Projects Marked with a Compliance Framework
  • Manage:Import - GMAU - MAU importing
  • Manage:Import - Other - Share of New Projects Imported
  • Plan, Plan:Project Management - SMAU, Paid GMAU - MAU Interacting With Issues
  • Plan:Product Planning - Paid GMAU - MAU creating Epics each Month
  • Plan:Certify - Paid GMAU - MAU Interacting with Requirements
  • Plan:Certify - Other - Number of Requirements Created Monthly
  • Create - SMAU - Unique users writing to a Git repository
  • Create:Source Code - GMAU - Unique users writing to a project Git repository
  • Create:Source Code Secondary - GMAU - Unique users approving merge requests
  • Create:Source Code Tertiary - GMAU - Unique users pushes rejected by push rules
  • Create:Code Review - GMAU - Unique users using Merge Requests
  • Create:Code Review - Other - Code Review/SCM Ratio by UMAU and Subscription Tier
  • Create:Code Review - Editor Extension (VS Code) - Category MAU
  • Create:Gitaly - Other - Availability
  • Create:Gitaly - Other - Unique instances with Gitaly Cluster enabled
  • Create:Editor - GMAU - Unique users committing via any web editor
  • Create:Editor - AMAU - Unique users committing by web editor
  • Create:Editor - Other - Number of Commits per Month from the Web IDE
  • Create:Editor - Other - Number of new Snippets per Month
  • Create:Editor - AMAU - Unique users committing via the Static Site Editor
  • Create:Editor - AMAU - Uniques users of Wiki
  • Create:Ecosystem - GMAU - MAU of any integration
  • Create:Ecosystem - Other - Total active project integrations
  • Create:Ecosystem - Other - Integrations managed at Group or Instance level
• Other PI Pages
• Legends
  • Health
• How to work with pages like this
  • Pages
    • Helper Functions
  • Data
  • Guidelines

Note: This page is dynamically generated via the Dev Section performance indicators .yml file.

Section PI Overview

Stage/Group	PI Type	PI	Health	SaaS / SM	Implementation Status
Dev	Section CMAU	Estimated & Recorded Sum of Dev Section SMAU	Okay	Both	Complete
Manage, Manage:Optimize	SMAU, GMAU	MAU Viewing Analytics Features	Okay	Both	Complete
Manage:Access	Paid GMAU	MAU Using Paid SAML	Okay	SaaS	Complete
Manage:Compliance	Paid GMAU	MAU Viewing Compliance Features	Attention	Both	Complete
Manage:Import	GMAU	MAU importing	Unknown	Both	Dashboard
Plan, Plan:Project Management	SMAU, Paid GMAU	MAU Interacting With Issues	Okay	Both	Complete
Plan:Product Planning	Paid GMAU	MAU creating Epics each Month	Okay	Both	Complete
Create	SMAU	Unique users writing to a Git repository	Okay	Both	Complete
Create:Source Code	GMAU	Unique users writing to a project Git repository	Okay	Both	Complete
Create:Code Review	GMAU	Unique users using Merge Requests	Okay	Both	Complete
Create:Gitaly	Other	Availability	Unknown	Both	Instrumentation
Create:Editor	GMAU	Unique users committing via any web editor	Attention	Both	Complete
Create:Ecosystem	GMAU	MAU of any integration	Unknown	Both	Instrumentation

Regular Performance Indicators

Dev - Section CMAU - Estimated & Recorded Sum of Dev Section SMAU

A sum of all SMAUs across the three stages in the Dev Section (Manage, Plan, and Create) for All users and Paid users.

SaaS/SM: Both

Target: Estimated CMAU growth 4% MoM; 2.73M by end of FY21Q4 for CMAU, 603K by end of FY21Q4 for Paid CMAU

Is Primary PI: true

Metric Name: issues, action_monthly_active_users_project_repo, analytics_unique_visits_for_any_target

Chart (Sisense↗)

Secondary Chart (Sisense↗)

Tertiary Chart (Sisense↗)

Health: Okay

• CMAU is fully instrumented
• Dev Section Jan 2021 CMAU remained flat MoM and remained down 3.5% from Nov 2020
• Dev Section exceeded CMAU target of 2.73M by 1.4% by end of FY21
• Dev Section Jan 2021 SaaS CMAU bounced back from its Dec 2021 value
• Dev Section Jan 2021 Self-Managed CMAU continues to grow MoM
• Plan Stage Jan 2021 CMAU increased and exceeded its previous highest CMAU value
• Create Stage Jan 2021 CMAU increased and is trending upwards
• Manage Stage Jan 2021 CMAU decreased MoM

Implementation Status: Complete

Focus This Month (March):

• Work with Data team to better calculate "Estimated Self-Managed Uplift" to help balance out calculations
• Analyze Manage Stage CMAU drop and formulate plan to recapture lost CMAU

Manage, Manage:Optimize - SMAU, GMAU - MAU Viewing Analytics Features

The number of unique users viewing any Analytics page.

SaaS/SM: Both

Target: 116050

Is Primary PI: true

Metric Name: analytics_total_unique_counts_monthly

Chart (Sisense↗)

Secondary Chart (Sisense↗)

Tertiary Chart (Sisense↗)

Health: Okay

• Insights - The Optimize data for January is inaccurate due to this usage ping issue. Many page views for self managed are showing as negative.
• Insights - Page views by SaaS users continued a downward trend that started in December. SaaS usage dropped by 10% in December, and again by 13% in January. Views are down for all Optimize pages except Group Insights, which increased by 45%.
• Improvements - The new DevOps Adoption feature was rolled back in January to pivot from the concept of Segments to only supporting groups in the table (https://gitlab.com/gitlab-org/gitlab/-/issues/296972). There is strong interest in this feature from enterprise customers wanting insights into how GitLab is being used in their organizations. Refactoring work commenced in 13.9 and will continue into 13.10 with the goal of renabling the feature in 13.10 or 13.11. This is expected to result in an increase in DevOps Reports usage. After this work is completed, the feature will be added at the group level, further increasing availability and adoption.
• Improvements - Group-level VSA is being refactored to create more space on the page for new features and to simplify navigation. This work is scheduled to be completed in 13.9. Next up we are adding the ability to sort items in each VSA stage by cycle time, which will allow users to identify aging items and take action to reduce overall cycle time in their value streams. Additionally, we are refactoring project-level VSA code to use the group-level code which will improve usability and add functionality when viewing VSA at the project level, as well as allow us to iterate faster because we won't be maintaining two sets of code. All of these improvements combined will improve the usability of VSA and is expected in increase adoption as we announce each improvement. These are the foundational steps we are taking before embarking on our FY22 vision for VSA which will add many new features and a much richer value stream management experience.
• SaaS data is not available for the paid SMAU chart due to limitations of the redis counter that is used.

Implementation Status: Complete

Manage:Access - Paid GMAU - MAU Using Paid SAML

The number of unique users logging into GitLab using SAML SSO for Groups.

SaaS/SM: SaaS

Target: 30000. See https://gitlab.com/gitlab-com/Product/-/issues/1832 for further detail.

Is Primary PI: true

Metric Name: user_auth_by_provider (Group_SAML)

Chart (Sisense↗)

Health: Okay

• Improvements - In 13.10, Auditor users will have read-only access to groups with SSO enabled. This will allow the support organization to reduce the number of Administrators for GitLab.com and meet their OKR!
• Improvements - In 13.10, GitLab will begin enforcing SSO for Git activity. This will provide the same level of protection to Git as the Web UI and been requested by several large customers and prospects.
• Improvements - In 13.10, we will increase security for GitLab.com by lowering the timeout for SAML SSO sessions from 7 days to 1 day.
• Insights - We continue to see a healthy upward trend for group SAML logins per month! This is a combination of decreasing the SSO timeout and enteprise customer growth for GitLab.com . My analysis shows that decreasing the SSO timeout increased Group SAML usage by 10%.

Implementation Status: Complete

Manage:Compliance - Paid GMAU - MAU Viewing Compliance Features

The number of unique users viewing Audit Events, Compliance Dashboard, or Credential Inventory.

SaaS/SM: Both

Target: TBD (expected [April 2021](https://gitlab.com/gitlab-org/manage/compliance/general/-/issues/13))

Is Primary PI: true

Metric Name: i_compliance_audit_events, g_compliance_audit_events, g_compliance_dashboard, i_compliance_credential_inventory, usage_data_g_compliance_dashboard, usage_data_a_compliance_audit_events_api

Chart (Sisense↗)

Secondary Chart (Sisense↗)

Tertiary Chart (Sisense↗)

Health: Attention

• We're still building out the compliance features that are tracked by this metric. This improvement work will take longer than desired to complete since we do not have enough people to ship any faster. We expect iterations to ship on a continuous basis across each of these features.
• Setting a target is premature since we're still collecting initial data. We would like to collect a total of 6 months worth of data before setting a target.
• Result - Increased GMAU by ~25% in December by making Audit Events available to Developers and making the feature more discoverable.
• Adding a Change Management view to the Compliance Dashboard. Planned for Q2 2021.
• Improving PAT & SSH credential management. Ongoing with expected epic completion in Q2 2021.
• Improving discoverability of Compliance features. Planned for Q2 2021. - Adding additional credentials to the credential inventory. Planned for Q3 2021.

Implementation Status: Complete

Manage:Compliance - Other - Projects Marked with a Compliance Framework

New and cumulative number of projects using compliance frameworks.

SaaS/SM: Both

Target: 1,030 total new projects in Q4 using compliance frameworks (SaaS and self-managed combined)

Metric Name: projects_with_compliance_framework

Chart (Sisense↗)

Health: Attention

• In Q3 we saw 936 new projects use compliance frameworks. These were a limited selection of 5, hard-coded options. We are targeting a 10% increase in adoption by making these frameworks customizable
• Additionally, we're using a conservative target for now while we focus on several improvements to the underlying feature that drives this metric.
• We're continuing iteration on compliance frameworks to give them more utility and solve several standardization and enforcement problems for customers.
• We're adding an improvement to give organizations better control of who can add or modify these frameworks at the project level in 13.8.
• We expect the next iteration, consisting of frontend and backend additions to ship in 13.9. This iteration is particularly important for the reason below.
• By 13.10 we expect compliance frameworks to fully support one of our most requested solutions, group compliance pipelines, as a required step to implement it, which we expect to drive even more adoption of compliance frameworks.

Implementation Status: Complete

Manage:Import - GMAU - MAU importing

A monthly count of the unique number of users performing an import

SaaS/SM: Both

Target: TBD - See Health section below for details

Is Primary PI: true

Metric Name: unique_users_all_imports

Health: Unknown

• No target defined yet.
• Target will be defined after the metric instrumentation is implemented and we get a baseline of data.
• Improvement - Reducing friction for GitLab-to-GitLab migrations - The MVC for the Group Migration feature was enabled in production in 13.9. This feature will enable a much easier migration of an entire GitLab group from one instance to another.

Implementation Status: Dashboard

• Instrumentation has been implemented in 13.9.
• Dashboard update is in progress.

Manage:Import - Other - Share of New Projects Imported

Share of projects created via import within 90 days of the namespace's creation.

SaaS/SM: Both

Target: TBD - See Health section below for details

Metric Name: TBD

Health: Unknown

• No target defined yet.
• Target will be defined once the dashboard is updated with the improved metric (for both Saas and self-managed).
• Improvement - Reducing friction for GitLab-to-GitLab migrations - The MVC for the Group Migration feature was enabled in production in 13.9. This feature will enable a much easier migration of an entire GitLab group from one instance to another.

Implementation Status: Dashboard

• Instrumented for both GitLab.com and self-managed in 13.6.
• Dashboard update is currently in progress.

Plan, Plan:Project Management - SMAU, Paid GMAU - MAU Interacting With Issues

Number of unique users interacting with an issue (open, close, reopen, adjust label, adjust milestone, adjust weight, comment)

SaaS/SM: Both

Target: 484,510 for All, 148,176 for Paid

Is Primary PI: true

URL(s)

• https://about.gitlab.com/direction/plan/issue_tracking/
• https://app.periscopedata.com/app/gitlab/775807/Plan-xMAU-(monthly)

Metric Name: redis_hll_counters.issues_edit.issues_edit_total_unique_counts_monthly

Chart (Sisense↗)

Secondary Chart (Sisense↗)

Tertiary Chart (Sisense↗)

Health: Okay

• Achieved - FY21 Q4 targets of 437,490 for All, 101,835 for Paid. New targets set for FY22 Q1.
• Driver - For Paid, continue high level progress on achieving feature parity between an Instance, Group, and Project, making issues more extensible, and continuing to make incremental progress on iterations.
• Improvement - For All, work towards addressing the problem areas that emerged out of the Category Maturity Scorecard exercise for Issue Tracking.
• Improvement - For All, continue making progress on quality of life improvements for existing features.
• Improvement - For Paid, creating multi-level Epics spanning a Group hierarchy is clunky and non-intuitive. Improving this, along with making the Roadmap more interactive and intuitive would be a huge catalyst to drive Product Planning adoption.
• Improvement - For Paid, continue making incremental, iterative progress towards Plan's core JTBD as outlined in the Issue Tracking direction.

Implementation Status: Complete

• xMAU data models are now updated with the new counter that aggregates unique users across many different issue actions.
• New Plan xMAU dashboard is live.

Lessons Learned (February):

• Only 28% of instances are currently on a version with the new Plan xMAU metric counters.
• The ratio of net new Paid GMAU to churned Paid GMAU improved from 3.75 in December to 3.99 in January, which indicates net growth and retention for Plan Paid users is healthy.
• MAU interacting with Iterations increased by 52% from December to January. The next step to drive feature adoption is more holistically integrating Iterations into Boards.

Focus This Month (March):

• Driver - Determine technical feasibility for allowing an issue to be assigned to multiple groups, which would likely solve Plan adoption blockers (priorities 1 and 3).
• Continue making progress on the 13.9 release plan and roll into 13.10 goals.
• Open > 3 documentation MRs in support of better articulating Plan use cases.
• Complete the validation track for saved filters/searches, customizable issue types, and interacting with a full issue from within a Board
• Merge architectural blueprint for consolidating Groups and Projects on the backend.

Plan:Product Planning - Paid GMAU - MAU creating Epics each Month

The number of unique users creating Epics each month.

SaaS/SM: Both

Target: 3,900 Combined Unique Users for Self Managed and SaaS

Is Primary PI: true

Metric Name: epics

Chart (Sisense↗)

Tertiary Chart (Sisense↗)

Health: Okay

• Achieved - Target of 3,900 (3,943 actual GMAU for epic creation)
• Insight - Epic creation for SaaS continues to grow. Self-Managed was stable, but that could be due to the Janauary tracking bug.
• Improvement - Epic Boards MVC 1 is planned to ship in 13.10 and should continue the trend into the end of FY21 by enabling Portfolio-level planning.
• Improvement - Epic Swimlanes shipped in 13.6
• Closed - original PI dashboard issue for epic creation

Implementation Status: Complete

• We consider our Epic Creation GMAU as "Complete". However, our next target will be set after a second instrumentation.
• Instrument "all epic interactions" during 13.9 - 13.10. Instrumentation issue and dasboard issue.
• A new target will be selected when we have a baseline.

Plan:Certify - Paid GMAU - MAU Interacting with Requirements

The number of unique users interacting (create, view, reference) each month

SaaS/SM: Both

Target: TBD

Metric Name: TBD

Health: Unknown

• Note - Requirement interaction tracking issue is complete. Data is in the process of being aggregated. Early data shows healthy adoption of test report linking. Tracking issue
• Improvement - UX improvements for Requirements Management based on completed research study, Requirement Export MVC.

Implementation Status: Dashboard

• Dashboarding - Requirement Creation is instrumented is complete. Tracking issue for gmau dashboard here

Plan:Certify - Other - Number of Requirements Created Monthly

A monthly count of the number of requirements created.

SaaS/SM: Both

Target: 1000 by end of March 2021

URL(s)

• https://gitlab.com/gitlab-data/analytics/-/issues/5270

Metric Name: requirements_created

Chart (Sisense↗)

Health: Okay

• Improvement - Requirement Export MVC feature to release in 13.8
• Insight - Requirement creation has decreased. Investigation shows a substantial set of users are Universities, which explains the downturn for the holidays.
• Insight - Self-managed instances creating requirements continues to grow.

Implementation Status: Dashboard

• Dashboarding - accurate instrumentation on Self Managed, still needs verification for SaaS - issue

Create - SMAU - Unique users writing to a Git repository

A rolling count of the number of unique users who performed any Git write operation within the last 28 days.

SaaS/SM: Both

Target: Our target growth rate is 3% higher than the previous month for SMAU, and 4% for Paid SMAU.

Is Primary PI: true

URL(s)

• https://app.periscopedata.com/app/gitlab/728380/WIP-Create-Stage-Dashboard

Metric Name: action_monthly_active_users_project_repo

Chart (Sisense↗)

Secondary Chart (Sisense↗)

Tertiary Chart (Sisense↗)

Health: Okay

• Observation - December 2020 SMAU is less than November 2020 SMAU. A similar drop can be observed in GMAU as well, suggesting a common cause. The cause is likely seasonality consistent with the end of year. For this reason January SMAU and GMAU is expected to return to November 2020 levels and continue growing.
• Insight - on the median GitLab instance, 33% of users perform a Git write operation each month (source).
• Insight - on the median Paid GitLab instance (self-managed), 50% of users perform a Git write operation each month (source).

Implementation Status: Complete

• Improvement - AMAU is currently over counted. A deduplicated counter was implemented gitlab#234027 in GitLab 13.6. The charts do not yet reflect the new counter gitlab#291943.

Create:Source Code - GMAU - Unique users writing to a project Git repository

A rolling count of the number of unique users who performed any Git write operation within the last 28 days

SaaS/SM: Both

Target: Our target growth rate is 3% higher than the previous month for SMAU, and 4% for Paid SMAU.

Is Primary PI: true

URL(s)

• https://app.periscopedata.com/app/gitlab/728380/WIP-Create-Stage-Dashboard?widget=9520571&udv=0
• https://gitlab.com/gitlab-org/gitlab/-/issues/234027

Metric Name: action_monthly_active_users_project_repo

Chart (Sisense↗)

Secondary Chart (Sisense↗)

Tertiary Chart (Sisense↗)

Health: Okay

• Starting in GitLab 13.6 AMAU for this metric takes into consideration all repository types.
• Improvement - work towards providing users more insights while navigating their code with Code Intelligence
• Improvement - work towards providing more control over managing repository size

Implementation Status: Complete

• Enabled on GitLab.com
• Released for self-managed in GitLab 13.3

Create:Source Code Secondary - GMAU - Unique users approving merge requests

A rolling count of the number of unique users who approved any merge request within the last 28 days

Target: TBD

URL(s)

• https://gitlab.com/gitlab-org/gitlab/-/issues/293014

Health: Unknown

• Not currently instrumented
• Currently planning instrumentation and evaluating if this is a more relevant metric for the Source Code group

Create:Source Code Tertiary - GMAU - Unique users pushes rejected by push rules

A rolling count of the number of unique users who had one or more push rejected by any push rule within the last 28 days

Target: TBD

URL(s)

• https://gitlab.com/gitlab-org/gitlab/-/issues/293016

Health: Unknown

• Not currently instrumented
• Currently planning instrumentation and evaluating if this is a more relevant metric for the Source Code group

Create:Code Review - GMAU - Unique users using Merge Requests

This is defined as users who perform one or more of the following actions in a merge request - create, merge, or close a Merge Request.

SaaS/SM: Both

Target: Our target growth rate is 3% higher than the previous month for SMAU, and 4% for Paid SMAU.

Is Primary PI: true

URL(s)

• https://app.periscopedata.com/app/gitlab/526749/Create---Code-Review-category-dashboard

Metric Name: merge_requests_users

Chart (Sisense↗)

Secondary Chart (Sisense↗)

Tertiary Chart (Sisense↗)

Health: Okay

• Target - This aligns with our current understanding that Code Review is about 50% of the MAU of the stage metric and continuing to maintain that difference.
• Insight - Paid users of Code Review make up about one-third of total Code Review users - this provides an opportunity to convert to paid and increase users.
• Improvement - Support for custom commit messages when applying a suggested change will allow more teams with strict commit message guidelines to adopt this feature and maintain conformity with the project.
• Improvement - Mark changes as viewed in merge requests is a foundational step towards tracking and communicating progress on merge requests to other participants.
• Improvement - SCM <-> Code Review <-> Verify correlation charts now defined in new charts

Implementation Status: Complete

• Enabled on GitLab.com 2020-07-04
• Enabled for self-managed on GitLab 13.2
• Working on more comprehensive MAU tracking in &4948

Create:Code Review - Other - Code Review/SCM Ratio by UMAU and Subscription Tier

SaaS/SM: Both

Target: TBD - Primarily this metric is used for monitoring the relationship between SCM GMAU and Code Review GMAU to make sure it doesn't not fall out of alignment.

URL(s)

• https://app.periscopedata.com/app/gitlab/801102/SCM-%3C%3E-Code-Review-%3C%3E-Verify
• https://app.periscopedata.com/app/gitlab/801102/SCM-%3C%3E-Code-Review-%3C%3E-Verify?widget=10728688&udv=1218043

Metric Name: merge_requests_users

Chart (Sisense↗)

Secondary Chart (Sisense↗)

Health: Unknown

• Target - this metric is primarily monitored to remain at a consistent level.
• Insight - Paid Instances with 10+ users more closely track to 75% of SCM Users using Code Review
• Insight - Free Instances and those with less than 10 users drive down the SCM<->Code Review Ratio as most have less than 50% adoption

Implementation Status: Complete

Create:Code Review - Editor Extension (VS Code) - Category MAU

This is defined as users who use [GitLab Workflow](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow) for VS Code and make at least one API request to a GitLab instance.

SaaS/SM: Both

Target: TBD

Health: Unknown

• Instrumentation in 13.9
• Improvement - GitLab Workflow 3.11.0 released with auto completion for GitLab CI Variables
• Improvement - GitLab Workflow 3.10.0 released with viewing code review comments on merge request diffs

Implementation Status: Instrumentation

• MAU Collection Issue

Create:Gitaly - Other - Availability

Percentage of time during which Gitaly is fully operational and providing service to users within SLO parameters. Availability is measured according by Apdex (latency) and error rate. Gitaly is considered available when at least 50% of users are experiencing satisfactory latency, _and_ 50% of requests are completing successfully. If either of these conditions is not met, the service is experiencing an outage.

SaaS/SM: Both

Target: 99.95%, consistent with GitLab.com availability service level objectives

Is Primary PI: true

URL(s)

• https://dashboards.gitlab.net/d/gitaly-main/gitaly-overview
• https://gitlab.com/gitlab-org/gitlab/-/issues/216683

Metric Name: gitaly_apdex

Health: Unknown

• Instrumentation for this metric started in 13.4 and is still ongoing.

Implementation Status: Instrumentation

• Rolling 7 day metrics available in Grafana for GitLab.com. Current availability per Adpex is 99.97%.
• Improvement - self-managed availability metrics are planned

Create:Gitaly - Other - Unique instances with Gitaly Cluster enabled

Number of recorded GitLab instances with Gitaly Cluster enabled

SaaS/SM: Self-Managed

Target: TBD

URL(s)

• https://dashboards.gitlab.net/d/gitaly-main/gitaly-overview

Metric Name: gitaly_clusters

Chart (Sisense↗)

Health: Attention

• No target defined as the feature is still being rolled out.
• Improvement - introduce Variable Replication Factor and auto-reblancing within a cluster to enable more enterprise-level adoption.

Implementation Status: Complete

• Insight - Seeing impressive growth in Gitaly Cluster adoption.
• Instrumentation for reporting on instances with Gitaly Cluster enabled was introduced in 13.0 with the GA of Gitaly Cluster.

Create:Editor - GMAU - Unique users committing via any web editor

A rolling count of the number of unique users who performed a Git write operation to any respository using the Web IDE, Web Editor or Snippet within a stage in the last 28 days

SaaS/SM: Both

Target: 4% growth month over month for All

Is Primary PI: true

URL(s)

• https://app.periscopedata.com/app/gitlab/741875/Create:Editor-xMAU

Metric Name: action_monthly_active_users_ide_edit

Chart (Sisense↗)

Secondary Chart (Sisense↗)

Tertiary Chart (Sisense↗)

Health: Attention

• New target GMAU to be set once we get cumulative, de-duped data from newly aligned categories.
• Insight - Rebound from the holiday season to nearly November levels. We'll be monitoring to ensure consistent organic growth in subsequent months.
• Improvement - No targeted efforts in the next few releases to specifically grow GMAU, but currently working to migrate the Web IDE to use Editor Lite which will bring consistency across editors and a moderate performance improvement.

Implementation Status: Complete

• Instrumentation of Static Site Editor and Wiki to roll up into GMAU completed in 13.7, waiting for data to be reflected in the charts.

Create:Editor - AMAU - Unique users committing by web editor

A rolling count of the number of unique users who performed a Git write operation to any respository using the Web IDE, Web Editor or Snippet within a stage in the last 28 days - users grouped by editor used

SaaS/SM: Both

Target: TBD

URL(s)

• https://app.periscopedata.com/app/gitlab/741875/Create:Editor-xMAU

Metric Name: action_monthly_active_users_web_ide_edit, action_monthly_active_users_sfe_edit, action_monthly_active_users_snippet_editor_edit

Chart (Sisense↗)

Health: Unknown

• Data collection is beginning, but it's too early to define a target as instances continue to upgrade and report data.
• Insight - Editing through the Web Editor (single file editor) is 2-3 to 1 for users of the Web IDE. This confirms data recognized previously based on navigation paths that the single file editor is more widely used. We need to continute monitoring this to determine future investments in the Web IDE.
• Note - Important to call out that these are not unique users, so the same user can be reported as using multiple web editors.

Implementation Status: Completed Dashboard

• Instrumentation is in place for both self-managed and SaaS.
• Updating dashboards to for Paid / All breakouts

Create:Editor - Other - Number of Commits per Month from the Web IDE

The total number of commits made via the Web IDE.

SaaS/SM: Both

Target: Maintain organic growth, 650K by end of Q1 2021

URL(s)

• https://app.periscopedata.com/app/gitlab/723366/Create:Editor-Expanded-for-Embed

Metric Name: web_ide_commits

Chart (Sisense↗)

Health: Attention

• Insight - Continued decrease in January which is inconsistent with the GMAU charts above. We'll investigate whether all the data is in (as of 2021-02-05)

Implementation Status: Complete

• Instrumentation working and charts updated to include both self-managed and SaaS data.
• Updating dashboard for Paid / All breakout.

Create:Editor - Other - Number of new Snippets per Month

The number of new Snippets created each month.

SaaS/SM: Both

Target: 4% Growth month-over-month to 82k Snippets per Month by Q2 2021

URL(s)

• https://gitlab.com/gitlab-data/analytics/-/issues/4400

Metric Name: snippets_change

Chart (Sisense↗)

Health: Attention

• Inisght - Continued decrease in January which is inconsistent with the GMAU charts above. We'll investigate whether all the data is in (as of 2021-02-05)

Implementation Status: Complete

• Instrumentation for number of snippets is complete for self-managed and SaaS.

Create:Editor - AMAU - Unique users committing via the Static Site Editor

Monthly unique users that have made at least one commit via the static site editor.

SaaS/SM: Both

Target: 150 AMAU by June 30, 2021

URL(s)

• https://app.periscopedata.com/app/gitlab/684213/WIP-Static-Site-Editor-Usage-Metrics
• https://about.gitlab.com/direction/create/static_site_editor/

Metric Name: static_site_editor_create_commit

Secondary Chart (Sisense↗)

Health: Attention

• Goal - drive early adoption, with a focus on internal GitLab team members editing the Handbook
• Insight - Moderate growth month over month (10% but that only amounts to 3 new users).
• Insight - Growth will be slow until we can expand the use cases to support creating new pages and make the editor compatible with more static site generators.

Implementation Status: Complete

• Improvement - Instrumentation for Usage Ping completed in 13.7

Create:Editor - AMAU - Uniques users of Wiki

Monthly unique users that have viewed or edited a wiki

SaaS/SM: Both

Target: 10% higher than current month (SaaS and self-managed combined)

URL(s)

• https://gitlab.com/gitlab-org/gitlab/-/issues/230481

Metric Name: action_monthly_active_users_wiki_repo

Chart (Sisense↗)

Secondary Chart (Sisense↗)

Health: Okay

• No target defined

Implementation Status: Complete

• Insight - Estimated Self-Managed lift issue was fixed and now our primary chart looks more realistic, bringing our estimated total MAU for January to 804k. Secondary graph still needs to be updated.
• Improvement - Planning to slice the data to show how much group-level Wikis are contributing to growth.

Create:Ecosystem - GMAU - MAU of any integration

Monthly Unique Users of any GitLab integration.

SaaS/SM: Both

Target: TBD

Is Primary PI: true

URL(s)

• https://gitlab.com/groups/gitlab-org/-/epics/5181
• https://gitlab.com/gitlab-org/gitlab/-/issues/297401
• https://gitlab.com/gitlab-data/analytics/-/issues/5850

Metric Name: TBD

Health: Unknown

• No target defined
• Improvement - Adding Group and Instance integration will allow larger sets of projects to be integrated easily
• Improvement - Improvements and new functionality in the Jira integration should increase adoption, expanding on the success of our most popular integration

Implementation Status: Instrumentation

• Awaiting implementation of data in Usage Ping
• Improvement - (February 2021) First individual tracking counter (Jira) has shipped and is live on GitLab.com. Team will observe and ensure stability/performance, then enable the feature flag for the 13.10 release.
• Awaiting implementation of Sisense dashboard (blocked by Usage Ping implementation)

Create:Ecosystem - Other - Total active project integrations

The total number of active project service integrations across all usage ping data (which includes GitLab.com)

SaaS/SM: Both

Target: Maintain flat adoption rate relative to overall GitLab growth

Metric Name: projects_asana_active, projects_assembla_active, projects_bamboo_active, projects_bugzilla_active, projects_buildkite_active, projects_campfire_active, projects_confluence_active, projects_custom_issue_tracker_active, projects_discord_active, projects_drone_ci_active, projects_emails_on_push_active, projects_external_wiki_active, projects_flowdock_active, projects_github_active, projects_gitlab_slack_application_active, projects_hangouts_chat_active, projects_hipchat_active, projects_irker_active, projects_jenkins_active, projects_jira_active, projects_jira_cloud_active, projects_jira_dvcs_cloud_active, projects_jira_dvcs_server_active, projects_jira_issuelist_active, projects_jira_server_active, projects_mattermost_active, projects_mattermost_slash_commands_active, projects_microsoft_teams_active, projects_packagist_active, projects_pipelines_email_active, projects_pivotaltracker_active, projects_prometheus_active, projects_pushover_active, projects_redmine_active, projects_slack_active, projects_slack_notifications_active, projects_slack_slash_active, projects_slack_slash_commands_active, projects_teamcity_active, projects_unify_circuit_active, projects_webex_teams_active

Chart (Sisense↗)

Health: Okay

• Observation - Growth tracks fairly evenly against overall usage growth (total number of projects created), demonstrating flat relative adoption rate of integrations
• Observation - GitLab.com users integrate at ~1/3 the rate that self-managed users do

Implementation Status: Dashboard

• Good overall tracking of project integrations, detailed breakdown available per-integration
• Updating dashboards to for Paid / All breakouts

Create:Ecosystem - Other - Integrations managed at Group or Instance level

The total number of active project integrations that are being defined by a Group or Instance integration

SaaS/SM: Both

Target: Not defined

URL(s)

• https://gitlab.com/gitlab-org/gitlab/-/issues/204802#note_392267011
• https://gitlab.com/gitlab-data/analytics/-/issues/5849

Metric Name: TBD

Health: Unknown

• No data so far

Implementation Status: Dashboard

• Usage ping implemented, dashboard not yet built

Other PI Pages

• Performance Indicator Maturity Table
• Engineering Function Performance Indicators
  • Customer Support Department Performance Indicators
  • Development Department Performance Indicators
    • Dev Sub-Department Performance Indicators
    • Ops Sub-Department Performance Indicators
    • Secure Sub-Department Performance Indicators
    • Threat Management Sub-Department Performance Indicators
    • Growth Sub-Department Performance Indicators
    • Enablement Sub-Department Performance Indicators
  • Infrastructure Department Performance Indicators
  • Quality Department Performance Indicators
  • Security Department Performance Indicators
  • UX Department Performance Indicators
• Product Performance Indicators
  • Dev Section Performance Indicators
  • Ops Section Performance Indicators
  • Sec Section Performance Indicators
  • Enablement Section Performance Indicators
  • Stage and Group Performance Indicators
• Corporate Finance Performance Indicators
• Recruiting Performance Indicators
• People Success Indicators
• KPI Index

Legends

Health

Value	Level	Meaning
3	Okay	The KPI is at an acceptable level compared to the threshold
2	Attention	This is a blip, or we’re going to watch it, or we just need to enact a proven intervention
1	Problem	We'll prioritize our efforts here
0	Unknown	Unknown

How to work with pages like this

Pages

Pages, such as the Engineering Function Performance Indicators page are rendered by an ERB template that contains HTML code.

• Any changes to the Performance Indicator (PI) ERB template will cascade to all links under the Other PI Pages section
• To add, delete, or update (K)PI information (including title, description, etc), make changes to the corresponding Performance Indicators data file. See more details in the Data section below.
• To update the way each individual (K)PI displays:
  • make changes to the Performance Indicators List ERB file for how the (K)PI is displayed on the KPI Index page
  • make changes to the Performance Indicators ERB file for any other properties and its format
• When adding a new PI page,
  • create the new PI page in the business division's handbook section (like the Engineering Division page)
  • include the relative link to the PI page in the Performance Indicators List ERB file
  • add all the (K)PI information into the corresponding Performance Indicators data file

Helper Functions

These ERB templates calls custom helper functions that extract and transform data from the Performance Indicators data file.

• The kpi_list_by_org(org) helper function takes a required string argument named org (deparment or division level) that returns all the KPIs (pi.is_key == true) for a specific organization grouping (pi.org == org) from the Performance Indicators data file.
• The pi_maturity_level(performance_indicator) helper function automatically assigns a maturity level based on the availability of certain data properties for a particular PI.
• The pi_maturity_reasons(performance_indicator) helper function returns a reason for a PI maturity based on other data properties.
• The performance_indicators(org) takes a required string argument named org (deparment or division level) that returns two lists - a list of all KPIs and a list of all PIs for a specific organization grouping (department/division).
• The signed_periscope_url(data) takes in the sisense_data property information from Performance Indicators data files and returns a signed chart URL for embedding a Sisense chart into the handbook.

Data

The heart of pages like this are Performance Indicators data files which are YAML files. Each - denotes a dictionary of values for a new (K)PI. The current elements (or data properties) are:

Property	Type	Description
name	Required	String value of the name of the (K)PI. For Product PIs, product hierarchy should be separate from name by " - " (Ex. {Stage Name}:{Group Name} - {PI Type} - {PI Name}
base_path	Required	Relative path to the performance indicator page that this (K)PI should live on
definition	Required	refer to Parts of a KPI
parent	Optional	should be used when a (K)PI is a subset of another PI. For example, we might care about Hiring vs Plan at the company level. The child would be the division and department levels, which would have the parent flag.
target	Required	The target or cap for the (K)PI. Please use Unknown until we reach maturity level 2 if this is not yet defined. For GMAU, the target should be quarterly.
org	Required	the organizational grouping (Ex: Engineering Function or Development Department). For Product Sections, ensure you have the word section (Ex : Dev Section)
section	Optional	the product section (Ex: dev) as defined in sections.yml
stage	Optional	the product stage (Ex: release) as defined in stages.yml
group	Optional	the product group (Ex: progressive_delivery) as defined in stages.yml
category	Optional	the product group (Ex: feature_flags) as defined in categories.yml
is_key	Required	boolean value (true/false) that indicates if it is a (key) performance indicator
health	Required	indicates the (K)PI health and reasons as nested attributes. This should be updated monthly before Key Meetings by the DRI.
health.level	Optional	indicates a value between 0 and 3 (inclusive) to represent the health of the (K)PI. This should be updated monthly before Key Meetings by the DRI.
health.reasons	Optional	indicates the reasons behind the health level. This should be updated monthly before Key Meetings by the DRI. Should be an array (indented lines starting with dashes) even if you only have one reason.
urls	Optional	list of urls associated with the (K)PI. Should be an array (indented lines starting with dashes) even if you only have one url
funnel	Optional	indicates there is a handbook link for a description of the funnel for this PI. Should be a URL
sisense_data	Optional	allows a Sisense dashboard to be embeded as part of the (K)PI using chart, dashboard, and embed as neseted attributes.
sisense_data.chart	Optional	indicates the numeric Sisense chart/widget ID. For example: 9090628
sisense_data.dashboard	Optional	indicates the numeric Sisense dashboard ID. For example: 634200
sisense_data.shared_dashboard	Optional	indicates the numeric Sisense shared_dashboard ID. For example: 185b8e19-a99e-4718-9aba-96cc5d3ea88b
sisense_data.embed	Optional	indicates the Sisense embed version. For example: v2
sisense_data_secondary	Optional	allows a second Sisense dashboard to be embeded. Same as sisense data
sisense_data_secondary.chart	Optional	Same as sisense_data.chart
sisense_data_secondary.dashboard	Optional	Same as sisense_data.dashboard
sisense_data_secondary.shared_dashboard	Optional	Same as sisense_data.shared_dashboard
sisense_data_secondary.embed	Optional	Same as sisense_data.embed
public	Optional	boolean flag that can be set to false where a (K)PI does not meet the public guidelines.
pi_type	Optional	indicates the Product PI type (Ex: AMAU, GMAU, SMAU, Group PPI)
product_analytics_type	Optional	indicates if the metric is available on SaaS, SM (self-managed), or Both.
is_primary	Optional	boolean flag that indicates if this is the Primary PI for the Product Group.
implementation	Optional	indicates the implementation status and reasons as nested attributes. This should be updated monthly before Key Meetings by the DRI.
implementation.status	Optional	indicates the Implementation Status status. This should be updated monthly before Key Meetings by the DRI.
implementation.reasons	Optional	indicates the reasons behind the implementation status. This should be updated monthly before Key Meetings by the DRI. Should be an array (indented lines starting with dashes) even if you only have one reason.
lessons	Optional	indicates lessons learned from a K(PI) as a nested attribute. This should be updated monthly before Key Meetings by the DRI.
lessons.learned	Optional	learned is an attribute that can be nested under lessonsand indicates lessons learned from a K(PI). This should be updated monthly before Key Meetings by the DRI. Should be an array (indented lines starting with dashes) even if you only have one lesson learned
monthly_focus	Optional	indicates monthly focus goals from a K(PI) as a nested attribute. This should be updated monthly before Key Meetings by the DRI.
monthly_focus.goals	Optional	indicates monthly focus goals from a K(PI). This should be updated monthly before Key Meetings by the DRI. Should be an array (indented lines starting with dashes) even if you only have one goal
metric_name	Optional	indicates the name of the metric in Self-Managed implemenation. The SaaS representation of the Self-Managed implementation should use the same name.

Guidelines

• Each KPI chart should be a timeseries chart.
  • Use Purple bars to denote values.
  • Use a Red stepped-line for timeseries target.
  • Directional targets will be used:
    • Above ...
    • Below ...
    • At ...
    • At or above ...
    • At or below ...
  • Optional: Use a single Green bar for the current month (in progress). Preferred for metrics which accumulate through a month but optional due to the performance impact to render the chart.
  • Optional: Use a Black line for rolling average.
• Each Sisense dashboard for KPIs should consider the following settings to ensure timely updates:
  • Setting up auto-refresh for a frequency that fits the KPI
  • Excluding Dashboards from Auto Archive
• Each KPI should have a standalone dashboard with a single chart representing the KPI and a text box with a link back to the handbook definition.
  • In Sisense, create a shared dashboard link to get the shared dashboard ID.
  • In Sisense, use the Share Link action of the chart to get the chart (widget_id) and the dashboard ID.
  • Add the shared_dashboard, chart , and the dashboard key-value pairs to the corresponding Performance Indicators data file under the sisense_data property
• Multi-series performance indicators should consider the following guidelines:
  • If series are mutually exclusive, use stacked bars for each series with a monthly time series
  • If series are not mutually exclusive, use grouped bars for each series with a monthly time series
  • Do not graph any targets in the chart.
  • Current month styling guidelines will not apply
• Avoid : in strings as it's an important character in YAML and will confuse the data parsing process. Put the string in "quotes" if you really need to use a :