Dev Section Performance Indicators

1. You are here:
2. Handbook
3. Product Handbook
4. Dev Section Performance Indicators

On this page

• Regular Performance Indicators
• Other PI Pages
• Legends
  • Health
  • Instrumentation
• How to work with pages like this
  • Pages
    • Helper Functions
  • Data
  • Guidelines

Regular Performance Indicators

Create SMAU - Git Write Operations

The Create SMAU (Stage Monthly Active Users) is the number of users who perform a Git write operation within a stage in a 28 day rolling period.

Target: TBD

URL(s)

• https://gitlab.com/gitlab-org/gitlab/-/issues/219956

Status: Unknown

• Instrumentation not yet available.

Instrumentation: Problem

• Instrumentation not yet available.

Create:Gitaly MAU - Git Write Operations

The Create Gitaly MAU (Monthly Active Users) - group and category - is the number of users who perform a Git write operation within a stage in a 28 day rolling period.

Target: TBD

URL(s)

• https://gitlab.com/gitlab-org/gitlab/-/issues/219956

Status: Unknown

• Instrumentation not yet available.

Instrumentation: Problem

• Instrumentation not yet available.

Create:Gitaly - Availability

Percentage of time during which Gitaly is fully operational and providing service to users within SLO parameters. The availability of Gitaly is measured according to two metrics. Apdex, for latency and error rate, for errors. A service is considered available when at least 50% of users are experiencing satisfactory latency, _and_ 50% of requests are completing successfully. If either of these conditions is not met, the service is experiencing an outage.

Target: 99.95%

URL(s)

• https://dashboards.gitlab.com/d/gitaly-main/gitaly-overview
• https://gitlab.com/gitlab-org/gitlab/-/issues/216683

Status: Attention

• For GitLab.com, none of the data used for these metrics is in the data warehouse yet, therefore we cannot embed Sisense charts in the handbook.
• For self-hosted instances, data is not being collected.

Instrumentation: Attention

• For GitLab.com, we are currently tracking this data via Prometheus, and is published on Grafana.
• For self-hosted instances, these metrics are also available, but there is not yet a way to summarize them for the usage ping.

Create:Source Code - Percentage of users per instance using Merge Requests

Proportion of users per each instance (self-managed) or userspace (GitLab.com) making use of Merge Requests. This is defined as users who perform one or more of the following actions in a merge request - create, merge, or close a Merge Request.

Target: TBD

URL(s)

• https://gitlab.com/gitlab-data/analytics/-/issues/4633

Status: Unknown

• Instrumentation in-progress
• Sisense chart is being constructed

Total active project integrations

The total number of active project service integrations across all usage ping data (which includes GitLab.com)

Target: Not yet set

URL(s)

• https://gitlab.com/gitlab-data/analytics/-/issues/4476

Chart (Sisense↗)

Status: Attention

• Clearly indicates healthy adoption of integrations.
• Unclear what this means relative to adoption of the related stages. (i.e Plan vs Jira)

Create:Web IDE - Number of Commits per Month from the Web IDE

The total number of commits made via the Web IDE.

Target: Unknown until we reach maturity level 2

URL(s)

• https://gitlab.com/gitlab-data/analytics/-/issues/4398

Chart (Sisense↗)

Status: Problem

• Target not defined

Create:Snippets - Number of new Snippets per Month

The number of new Snippets created each month.

Target: Unknown until we reach maturity level 2

URL(s)

• https://gitlab.com/gitlab-data/analytics/-/issues/4400

Chart (Sisense↗)

Status: Problem

• Target not defined

Plan:Portfolio Management - Unique Users of Epics each Month

The number of unique users interacting with Epics each month.

Target: Targeting xk Unique Users for Self Managed and yk Unique Users for SaaS

URL(s)

• https://gitlab.com/gitlab-data/analytics/-/issues/5280

Chart

Status: Attention

• Final Chart in progress

Plan:Certify - Number of requirements created

A monthly count of the number of requirements created.

Target: Unknown until we reach maturity level 2

Status: Problem

• Target not defined

Instrumentation: Problem

• Updated data collection to measure interactions in progress
• Currently only measuring self-managed instances

Other PI Pages

• Performance Indicator Maturity Table
• Engineering Function Performance Indicators
  • Customer Support Department Performance Indicators
  • Development Department Performance Indicators
    • Growth Section Performance Indicators
  • Infrastructure Department Performance Indicators
  • Quality Department Performance Indicators
  • Security Department Performance Indicators
  • UX Department Performance Indicators
• Product Performance Indicators
  • Dev Section Performance Indicators
  • Ops Section Performance Indicators
  • Secure & Defend Section Performance Indicators
  • Enablement Section Performance Indicators
• Corporate Finance Performance Indicators
• Recruiting Performance Indicators
• People Success Indicators
• Data Team KPI Index

Legends

Health

Value	Level	Meaning
3	Okay	The KPI is at an acceptable level compared to the threshold
2	Attention	This is a blip, or we’re going to watch it, or we just need to enact a proven intervention
1	Problem	We'll prioritize our efforts here
0	Unknown	Unknown

Instrumentation

Value	Level	Meaning
3	Okay	We are correctly capturing and measuring data for GitLab.com and self-managed instances.
2	Attention	Missing one of: GitLab.com or self-managed data.
1	Problem	Missing both: GitLab.com and self-managed data.
0	Unknown	Unknown

How to work with pages like this

Pages

Pages, such as the Engineering Function Performance Indicators page are rendered by an ERB template that contains HTML code.

• Any changes to the Performance Indicator (PI) ERB template will cascade to all links under the Other PI Pages section
• To update the options for the maturity table, make changes to the Performance Indicators Maturities ERB file
• To update the way each individual (K)PI displays:
  • make changes to the Performance Indicators List ERB file for how the (K)PI is displayed on the KPI Index page
  • make changes to the Performance Indicators ERB file for any other properties and its format
• When adding a new PI page,
  • create the new PI page in the business division's handbook section (like the Engineering Division page)
  • include the relative link to the PI page in the Performance Indicators List ERB file
  • add all the (K)PI information into the Performance Indicators data file

Helper Functions

These ERB templates calls custom helper functions that extract and transform data from the Performance Indicators data file.

• The kpi_list_by_org(org) helper function takes a required string argument named org (deparment or division level) that returns all the KPIs (pi.is_key == true) for a specific organization grouping (pi.org == org) from the Performance Indicators data file.
• The pi_maturity_level(performance_indicator) helper function automatically assigns a maturity level based on the availability of certain data properties for a particular PI.
• The pi_maturity_reasons(performance_indicator) helper function returns a reason for a PI maturity based on other data properties.
• The performance_indicator_target(name) helper function returns the target value from the target property for a specific PI from the Performance Indicators data file.
• The performance_indicators(org) takes a required string argument named org (deparment or division level) that returns two lists - a list of all KPIs and a list of all PIs for a specific organization grouping (department/division).
• The signed_periscope_url(data) takes in the sisense_data property information from the Performance Indicators data file and returns a signed chart URL for embedding a Sisense chart into the handbook.

Data

The heart of pages like this is the Performance Indicators data file which is in YAML file. Each - denotes a dictionary of values for a new (K)PI. The current elements (or data properties) are:

Property	Type	Description
name	Required	String value of the name of the (K)PI
base_path	Required	Relative path to the performance indicator page that this (K)PI should live on
definition	Required	refer to Parts of a KPI
parent	Optional	should be used when a (K)PI is a subset of another PI. For example, we might care about Hiring vs Plan at the company level. The child would be the division and department levels, which would have the parent flag.
target	Required	The target or cap for the (K)PI. Please use Unknown until we reach maturity level 2 if this is not yet defined.
org	Required	the organizational grouping (Ex: Engineering Function or Development Department)
is_key	Required	boolean value (true/false) that indicates if it is a (key) performance indicator
health	Required	has two additional elements/property - level (inclusive value between 0-3) and reasons. This should be updated monthly before Key Meetings by the DRI.
urls	Optional	list of urls associated with the (K)PI
sisense_data	Optional	contains elements/properties related to Sisense, including chart (numeric Sisense widget ID), dashboard (numeric Sisense dashboard ID), shared_dashboard (Sisense shared dashboard ID), embed (v2)
public	Optional	boolean flag that can be set to false where a (K)PI does not meet the public guidelines.
instrumentation	Optional	has two additional elements/property - level (inclusive value between 0-3) and reasons. This should be updated monthly before Key Meetings by the DRI.

Guidelines

• Each KPI chart should be a timeseries chart.
  • Use Purple bars to denote values.
  • Use a single Green bar for the current month (in progress).
  • Use a Red stepped-line for timeseries target.
  • Optional: Use a Black dashed-line for rolling average.
• Each KPI should have a standalone dashboard with a single chart representing the KPI and a text box with a link back to the handbook definition.
  • In Sisense, create a shared dashboard link to get the shared dashboard ID.
  • In Sisense, use the Share Link action of the chart to get the chart (widget_id) and the dashboard ID.
  • Add the shared_dashboard, chart , and the dashboard key-value pairs to the Performance Indicators data file under the sisense_data property
• Avoid : in strings as it's an important character in YAML and will confuse the data parsing process. Put the string in "quotes" if you really need to use a :
• urls: should be an array (indented lines starting with dashes) even if you only have one url
• Both maturity.level and health.level display a value between 0 and 3 (inclusive). The health value is a manual input while the maturity is an automated value based on other discrete (K)PI data properties.