| KPI | Health | Status |
|---|---|---|
| Unique Wider Community Contributors per Month | Attention |
|
| Meetups per month | Problem |
|
| Wider Community merged MRs per release | Attention |
|
| Developer Evangelism Monthly Impressions | Okay |
|
| GitLab for Education Quarterly Active Seats | Attention |
|
| MRARR | Unknown |
|
Unique Wider Community Contributors based on merged contribution by month. Contributors are unique and only counted once for multiple MRs from the same contributor.
Target: Above 120 contributors per month
Health: Attention
DRI: Nick Veenhof
A GitLab Meetup is defined as a meetup with a presentation given by a GitLab team member or a wider community member about GitLab or a tangential topic. It does not include meetups without GitLab content where GitLab only provides support. This KPI is tracked by counting the number of issues in the GitLab.com data with the Meetups label in the Marketing group.
Each event should have one associated issue with the Meetups label. The issue due date should be set to the event date. The combination of Meetups label and due date will be used by our data dashboard to count the number of meetups each month. This method of tracking can results in errors when meetup issues are created in other groups or when GitLab-related presentations at meetups are not shared with the Developer Relations team.
Health: Problem
The Wider community contributions per milestone metric reflects the number of merge requests (MRs) merged from the wider GitLab community for each milestone from our GitLab.com data. To count as a contribution the MR must have been merged, have the Community contribution label, have a GitLab milestone set (e.g 11.11, 12.0, etc.) and be against one of the monitored gitlab-org group projects.
Wider community metrics are more reliable after 2015 when the Community contribution label was created, and lastly, there is ongoing work to provide deeper insights into this metric in the Wider Community Sisense dashboard. Further time period representations are provided on the Contributions and contributors over time dashboard
Health: Attention
Lessons Learned (May):
The Developer Evangelism team aims to build thought leadership accross several mediums, primarily through content while exploring various other means. We track impressions of content generated to understand the reach of the generated content, currently we are tracking Twitter impressions, while exploring ways to track other mediums in subsequent iterations.
Health: Okay
The GitLab for Education team aims to facilitate and drive adoption of GitLab at educational institutions around the world and build an engaged community of GitLab Evangelists in the next generation of the workforce. We track the number of active seats in the GitLab for Education Program on a quarterly basis as a gauge of program enrollment and rentention. The number of active seats is determied by the number of seats from New Business, Add-ons, and Renewals.
Health: Attention
MRARR (pronounced "mer-arr," like a pirate) is the measurement of Merge Requests from customers, multiplied with the revenue of the account (ARR) from that customer. This measures how active our biggest customers are contributing to GitLab. We believe the higher this number the better we'll retain these customers and improve product fit for large enterprises. The unit of MRARR is MR Dollars (MR$). MR Dollars is different than the normal Dollars which is used for ARR. We are tracking current initiatives to improve this in this epic.
Target: Identified in SiSense Chart
Health: Unknown
DRI: Nick Veenhof
URL(s)
The Wider community contributions per month metric reflects the number of merge requests (MRs) merged from the wider GitLab community for each month from our GitLab.com data. To count as a contribution the MR must have been merged, have the Community contribution label and be against one of the monitored gitlab-org group projects.
Wider community metrics are more reliable after 2015 when the Community contribution label was created. This PI is also represented by the below Bitergia chart. Further time period representations are provided on the Contributions and contributors over time dashboard
Health: Attention
Lessons Learned (May):
The First time contributors per month metric reflects the number of wider GitLab community members that contribute for the first time to GitLab on a given month. To count as a first time contributor, their MR must have been merged, have the <a href="https://about.gitlab.com/handbook/marketing/developer-relations/code-contributor-program/#contribution-labels>1st-time-contributor label</a> and be against one of the monitored gitlab-org group projects.
Wider community metrics are more reliable after 2018 when the 1st-time-contributor label was created. This PI is also represented by the below Bitergia chart. Further time period representations are provided on the Contributions and contributors over time dashboard
Note - if you cannot see this chart, please go to the original Bitergia chart.
Health: Problem
This is a subset of an existing KPI. Please see the definition for the parent KPI.
We track the number of new intitutions joining the GitLab for Education Program on a quarterly basis. The number of new institutions is a gauge for program awareness and expansion.
Health: Attention
This is a subset of an existing KPI. Please see the definition for the parent KPI.
We track the number of new seats in GitLab for Education Program on a quarterly basis. The number of new seats is a gauge for GitLab adoption.
Health: Attention
The Developer Relations team supports the GitLab Community Forum as an official place for the wider community to seek technical support and assistance with GitLab. We track the average number of likes on comments across the forum over the quarter as a measure of helpful comments and impressions. We are also exploring other ways to track forum success.
Health: Okay
The Developer Relations team supports the GitLab Subreddit r/gitlab as a place to ask questions, get technical support, and projects related to GitLab. We track the number of unique visitors per quarter to gauge the engagement of growth of our subscriber community.
Health: Okay
The Developer Relations team engages with the community on HackerNews. It's important to build karma on content unrelated to GitLab as well and be a member of the HackerNews community. We track the cumulative karma of our team over time.
Health: Problem
The GitLab Collective on Stack Overflow is a space where the Developer Relations team can share content related to GitLab, users can collaborate, and share expertise. We track the ratio of questions to answers to measure contributor engagement over the quarter.
Health: Problem
OCMA (pronounced "ock-mah") measures the median time of all open MRs as of a specific date. In other words, on any given day, this calculates the number of open MRs and median time in an open state for open MRs at that point in time.
Target: Below 100 days
Health: Attention
URL(s)
Leading Organizations are entitled to a Review-Response SLO of 4 working days.
Target: Below or equal to 4 working days
Health: Okay
URL(s)
Returning vs new contributors. Having more returning contributors, in combination with a growing amount of unique wider community members, is a sign of a healthy contributor community.
Target: TBD
Health: Attention
URL(s)
The number of MR Coaches defined by team.yml role
Target: Above 50 coaches per month
Health: Attention
URL(s)
Percentage of Community Contributions that are related to feature development
Target: Above 20%
Health: Okay
This is the ratio of community contributions with the number of merged product MRs. As we grow as a company we want to make sure the community scales with the company.
Target: Above 8% of all MRs
Health: Attention
| Value | Level | Meaning |
|---|---|---|
| 3 | Okay | The KPI is at an acceptable level compared to the threshold |
| 2 | Attention | This is a blip, or we’re going to watch it, or we just need to enact a proven intervention |
| 1 | Problem | We'll prioritize our efforts here |
| -1 | Confidential | Metric & metric health are confidential |
| 0 | Unknown | Unknown |
Pages, such as the Engineering Function Performance Indicators page are rendered by an ERB template that contains HTML code.
Other PI Pages sectionThese ERB templates calls custom helper functions that extract and transform data from the Performance Indicators data file.
kpi_list_by_org(org) helper function takes a required string argument named org (deparment or division level) that returns all the KPIs (pi.is_key == true) for a specific organization grouping (pi.org == org) from the Performance Indicators data file.pi_maturity_level(performance_indicator) helper function automatically assigns a maturity level based on the availability of certain data properties for a particular PI.pi_maturity_reasons(performance_indicator) helper function returns a reason for a PI maturity based on other data properties.performance_indicators(org) takes a required string argument named org (deparment or division level) that returns two lists - a list of all KPIs and a list of all PIs for a specific organization grouping (department/division).signed_periscope_url(data) takes in the sisense_data property information from Performance Indicators data files and returns a signed chart URL for embedding a Sisense chart into the handbook.The heart of pages like this are Performance Indicators data files which are YAML files. Each - denotes a dictionary of values for a new (K)PI. The current elements (or data properties) are:
| Property | Type | Description |
|---|---|---|
name |
Required | String value of the name of the (K)PI. For Product PIs, product hierarchy should be separate from name by " - " (Ex. {Stage Name}:{Group Name} - {PI Type} - {PI Name} |
base_path |
Required | Relative path to the performance indicator page that this (K)PI should live on |
definition |
Required | refer to Parts of a KPI |
parent |
Optional | should be used when a (K)PI is a subset of another PI. For example, we might care about Hiring vs Plan at the company level. The child would be the division and department levels, which would have the parent flag. |
target |
Required | The target or cap for the (K)PI. Please use Unknown until we reach maturity level 2 if this is not yet defined. For GMAU, the target should be quarterly. |
org |
Required | the organizational grouping (Ex: Engineering Function or Development Department). For Product Sections, ensure you have the word section (Ex : Dev Section) |
section |
Optional | the product section (Ex: dev) as defined in sections.yml |
stage |
Optional | the product stage (Ex: release) as defined in stages.yml |
group |
Optional | the product group (Ex: progressive_delivery) as defined in stages.yml |
category |
Optional | the product group (Ex: feature_flags) as defined in categories.yml |
is_key |
Required | boolean value (true/false) that indicates if it is a (key) performance indicator |
health |
Required | indicates the (K)PI health and reasons as nested attributes. This should be updated monthly before Key Reviews by the DRI. |
health.level |
Optional | indicates a value between 0 and 3 (inclusive) to represent the health of the (K)PI. This should be updated monthly before Key Reviews by the DRI. |
health.reasons |
Optional | indicates the reasons behind the health level. This should be updated monthly before Key Reviews by the DRI. Should be an array (indented lines starting with dashes) even if you only have one reason. |
urls |
Optional | list of urls associated with the (K)PI. Should be an array (indented lines starting with dashes) even if you only have one url |
funnel |
Optional | indicates there is a handbook link for a description of the funnel for this PI. Should be a URL |
sisense_data |
Optional | allows a Sisense dashboard to be embeded as part of the (K)PI using chart, dashboard, and embed as neseted attributes. |
sisense_data.chart |
Optional | indicates the numeric Sisense chart/widget ID. For example: 9090628 |
sisense_data.dashboard |
Optional | indicates the numeric Sisense dashboard ID. For example: 634200 |
sisense_data.shared_dashboard |
Optional | indicates the numeric Sisense shared_dashboard ID. For example: 185b8e19-a99e-4718-9aba-96cc5d3ea88b |
sisense_data.embed |
Optional | indicates the Sisense embed version. For example: v2 |
sisense_data_secondary |
Optional | allows a second Sisense dashboard to be embeded. Same as sisense data |
sisense_data_secondary.chart |
Optional | Same as sisense_data.chart |
sisense_data_secondary.dashboard |
Optional | Same as sisense_data.dashboard |
sisense_data_secondary.shared_dashboard |
Optional | Same as sisense_data.shared_dashboard |
sisense_data_secondary.embed |
Optional | Same as sisense_data.embed |
public |
Optional | boolean flag that can be set to false where a (K)PI does not meet the public guidelines. |
pi_type |
Optional | indicates the Product PI type (Ex: AMAU, GMAU, SMAU, Group PPI) |
product_analytics_type |
Optional | indicates if the metric is available on SaaS, SM (self-managed), or Both. |
is_primary |
Optional | boolean flag that indicates if this is the Primary PI for the Product Group. |
implementation |
Optional | indicates the implementation status and reasons as nested attributes. This should be updated monthly before Key Reviews by the DRI. |
implementation.status |
Optional | indicates the Implementation Status status. This should be updated monthly before Key Reviews by the DRI. |
implementation.reasons |
Optional | indicates the reasons behind the implementation status. This should be updated monthly before Key Reviews by the DRI. Should be an array (indented lines starting with dashes) even if you only have one reason. |
lessons |
Optional | indicates lessons learned from a K(PI) as a nested attribute. This should be updated monthly before Key Reviews by the DRI. |
lessons.learned |
Optional | learned is an attribute that can be nested under lessonsand indicates lessons learned from a K(PI). This should be updated monthly before Key Reviews by the DRI. Should be an array (indented lines starting with dashes) even if you only have one lesson learned |
monthly_focus |
Optional | indicates monthly focus goals from a K(PI) as a nested attribute. This should be updated monthly before Key Reviews by the DRI. |
monthly_focus.goals |
Optional | indicates monthly focus goals from a K(PI). This should be updated monthly before Key Reviews by the DRI. Should be an array (indented lines starting with dashes) even if you only have one goal |
metric_name |
Optional | indicates the name of the metric in Self-Managed implemenation. The SaaS representation of the Self-Managed implementation should use the same name. |
Please reference the Engineering Metrics Page for guidelines on chart visualization formatting.