Developer Relations Department Performance Indicators
Executive Summary
| KPI | Health | Status |
|---|---|---|
| Unique Wider Community Contributors per Month | Attention |
|
| Wider Community merged MRs per release | Attention |
|
| Developer Relations Monthly Outreach | Attention |
|
| Active Community Members | Attention |
|
| GitLab for Education Quarterly Active Seats | Attention |
|
| MRARR | Unknown |
|
Key Performance Indicators
Unique Wider Community Contributors per Month
Unique Wider Community Contributors based on merged contribution by month. Contributors are unique and only counted once for multiple MRs from the same contributor. (To view the dashboard your browser must allow third-party cookies)
Target: Above 170 contributors per month Health:Attention
- Seeing an increase of returning contributors
- MR Backlog decreased ~8.2% during the 2022-12-01 <> 2023-01-26
- All time high of 466 review activity from GitLab Team Members to Wider Community in December
- All time high of 88 review activity between the Wider Community in December
Chart (Tableau↗)
Wider Community merged MRs per release
The Wider community contributions per milestone metric reflects the number of merge requests (MRs) merged from the wider GitLab community for each milestone from our GitLab.com data. To count as a contribution the MR must have been merged, have the Community contribution label, have a GitLab milestone set (e.g 11.11, 12.0, etc.) and be against one of the monitored gitlab-org group projects.
Wider community metrics are more reliable after 2015 when the Community contribution label was created, and lastly, there is ongoing work to provide deeper insights into this metric in the Wider Community Tableau dashboard. Further time period representations are provided on the Contributions and contributors over time dashboard (To view the dashboard your browser must allow third-party cookies)
Target: Health:Attention
- Under target, but showing steady increasing trend, amplified by the collaboration with the Quality Department.
- Reached baseline for making current contribution process more effective. Ongoing longer term focus on improving the overall contributor journey.
Chart (Tableau↗)
Developer Relations Monthly Outreach
The Developer Relations team aims to build thought leadership accross several mediums, primarily through content while exploring various other means. We track the reach of content generated to understand it’s impact. We track pageviews of blog posts authored by the Developer Relations team, views of YouTube videos uploaded by our team members, and the reach of our social media shares.(To view the dashboard your browser must allow third-party cookies)
Target: Health:Attention
- This is a new way of measuring this metric and we are still learning about the best way to track these numbers.
Active Community Members
The Developer Relations team seeks to grow engagement with our community across a number of channels. We measure this activity through the “Active Community Members” metric in Common Room. This metric reflects the number of people actively posting, commenting, or engaging with GitLab content across a number of channels including GitLab.com.
Target: Health:Attention
- This is a new way of measuring this metric and we are still learning about the best way to track these numbers.
GitLab for Education Quarterly Active Seats
The GitLab for Education team aims to facilitate and drive adoption of GitLab at educational institutions around the world and build an engaged community of GitLab Evangelists in the next generation of the workforce. We track the number of active seats in the GitLab for Education Program on a quarterly basis as a gauge of program enrollment and rentention. The number of active seats is determied by the number of seats from New Business, Add-ons, and Renewals.
Target: Health:Attention
- All growth prior to FY22Q1 has been organic. Outreach efforts are underway to promote the program.
- Ongoing longer term focus on outreach (new) and enablement for existing program members (renewals).
MRARR
MRARR (pronounced “mer-arr,” like a pirate) is the measurement of Merge Requests from customers, multiplied with the revenue of the account (ARR) from that customer. This measures how active our biggest customers are contributing to GitLab. We believe the higher this number the better we’ll retain these customers and improve product fit for large enterprises. The unit of MRARR is MR Dollars (MR$). MR Dollars is different than the normal Dollars which is used for ARR. We are tracking current initiatives to improve this in this epic.
Target: Identified in SiSense Chart Unknown
- Trending up and stabilizing
- Automated identification of Leading Organizations and Review Time SLO implemented
- Switched effort to on-board new contributing customers
- Customer contributions are now being identified
URL(s):
Regular Performance Indicators
Wider Community merged MRs per month
The Wider community contributions per month metric reflects the number of merge requests (MRs) merged from the wider GitLab community for each month from our GitLab.com data. To count as a contribution the MR must have been merged, have the Community contribution label and be against one of the monitored gitlab-org group projects.
Wider community metrics are more reliable after 2015 when the Community contribution label was created. Further time period representations are provided on the Contributions and contributors over time dashboard (To view the dashboard your browser must allow third-party cookies)
Target: Health:Attention
- Under target, but showing steady increasing trend, amplified by the collaboration with the Quality Department.
- Reached baseline for making current contribution process more effective. Ongoing longer term focus on improving the overall contributor journey.
Chart (Tableau↗)
First time contributors per month
The First time contributors per month metric reflects the number of wider GitLab community members that contribute for the first time to GitLab on a given month. To count as a first time contributor, their MR must have been merged, have the 1st-time-contributor label and be against one of the monitored gitlab-org group projects.
Wider community metrics are more reliable after 2018 when the 1st-time-contributor label was created. Further time period representations are provided on the Contributions and contributors over time dashboard
Note - if you cannot see this chart, please go to the original Tableau chart. (To view the dashboard your browser must allow third-party cookies)
Target: Health:Problem
- Unknown at this time, further research is required.
Chart (Tableau↗)
GitLab for Education Quarterly New Institutions
We track the number of new intitutions joining the GitLab for Education Program on a quarterly basis. The number of new institutions is a gauge for program awareness and expansion.
Target: Health:Attention
- All growth prior to FY22Q1 has been organic. Outreach efforts are underway to promote the program.
GitLab for Education Quarterly New Seats
We track the number of new seats in GitLab for Education Program on a quarterly basis. The number of new seats is a gauge for GitLab adoption.
Target: Health:Attention
- All growth prior to FY22Q1 has been organic. Outreach efforts are underway to promote the program.
GitLab Community Forum Likes Per Quarter
The Developer Relations team supports the GitLab Community Forum as an official place for the wider community to seek technical support and assistance with GitLab. We track the average number of likes on comments across the forum over the quarter as a measure of helpful comments and impressions. We are also exploring other ways to track forum success.
Target: Health:Okay
- This metric is new and is performing at the same threshold as last year.
Open Community MR Age
OCMA (pronounced “ock-mah”) measures the median time of all open MRs as of a specific date. In other words, on any given day, this calculates the number of open MRs and median time in an open state for open MRs at that point in time. (To view the dashboard your browser must allow third-party cookies)
Target: Below 100 days Health:Attention
- Above target. Our backlog of open MRs is decreasing at a steady rate
- Some stages/groups hold more than 25% of the open wider community MRs
- Collaborating with Runner team to address old community contribution
Chart (Tableau↗)
URL(s):
Leading Organizations MR Time-to-review
Leading Organizations are entitled to a Review-Response SLO of 4 working days. (To view the dashboard your browser must allow third-party cookies)
Target: Below or equal to 4 working days Health:Okay
- Above target at 5 working days
- Holiday break caused a spike that is visible in January.
Chart (Tableau↗)
URL(s):
Returning vs new contributors
Returning vs new contributors. Having more returning contributors, in combination with a growing amount of unique wider community members, is a sign of a healthy contributor community.(To view the dashboard your browser must allow third-party cookies)
Target: TBD Health:Attention
- Slightly under target. At 56% for current month
- Low total amount of new contributors since September `22
Chart (Tableau↗)
URL(s):
Community MR Coaches per Month
The number of MR Coaches defined by team.yml role (To view the dashboard your browser must allow third-party cookies)
Target: Above 50 coaches per month Health:Attention
- Increased to 40
- Improved automation and more self-service bot commands allow MR Coaches to focus on the more difficult cases
- Contributor Success team expanded, and every member of that team is an MR coaches by default
Chart (Tableau↗)
URL(s):
Feature Community Contribution MRs
Percentage of Community Contributions that are related to feature development (To view the dashboard your browser must allow third-party cookies)
Target: Above 20% Health:Okay
- Above target for 5 consecutive months
Chart (Tableau↗)
Community MR Percentage
This is the ratio of community contributions with the number of merged product MRs. As we grow as a company we want to make sure the community scales with the company. (To view the dashboard your browser must allow third-party cookies)
Target: Above 8% of all MRs Health:Attention
- Percentage went down to 5%
- The Wider Community is not growing as fast as the increase of GitLab team members
Chart (Tableau↗)
Legends
Health
| Value | Level | Meaning |
|---|---|---|
| 3 | Okay | The KPI is at an acceptable level compared to the threshold |
| 2 | Attention | This is a blip, or we’re going to watch it, or we just need to enact a proven intervention |
| 1 | Problem | We'll prioritize our efforts here |
| -1 | Confidential | Metric & metric health are confidential |
| 0 | Unknown | Unknown |
How pages like this work
Data
The heart of pages like this are Performance Indicators data files which are YAML files. Each - denotes a dictionary of values for a new (K)PI. The current elements (or data properties) are:
| Property | Type | Description |
|---|---|---|
name |
Required | String value of the name of the (K)PI. For Product PIs, product hierarchy should be separate from name by " - " (Ex. {Stage Name}:{Group Name} - {PI Type} - {PI Name} |
base_path |
Required | Relative path to the performance indicator page that this (K)PI should live on |
definition |
Required | refer to Parts of a KPI |
parent |
Optional | should be used when a (K)PI is a subset of another PI. For example, we might care about Hiring vs Plan at the company level. The child would be the division and department levels, which would have the parent flag. |
target |
Required | The target or cap for the (K)PI. Please use Unknown until we reach maturity level 2 if this is not yet defined. For GMAU, the target should be quarterly. |
org |
Required | the organizational grouping (Ex: Engineering Function or Development Department). For Product Sections, ensure you have the word section (Ex : Dev Section) |
section |
Optional | the product section (Ex: dev) as defined in sections.yml |
stage |
Optional | the product stage (Ex: release) as defined in stages.yml |
group |
Optional | the product group (Ex: progressive_delivery) as defined in stages.yml |
category |
Optional | the product group (Ex: feature_flags) as defined in categories.yml |
is_key |
Required | boolean value (true/false) that indicates if it is a (key) performance indicator |
health |
Required | indicates the (K)PI health and reasons as nested attributes. This should be updated monthly before Key Reviews by the DRI. |
health.level |
Optional | indicates a value between 0 and 3 (inclusive) to represent the health of the (K)PI. This should be updated monthly before Key Reviews by the DRI. |
health.reasons |
Optional | indicates the reasons behind the health level. This should be updated monthly before Key Reviews by the DRI. Should be an array (indented lines starting with dashes) even if you only have one reason. |
urls |
Optional | list of urls associated with the (K)PI. Should be an array (indented lines starting with dashes) even if you only have one url |
funnel |
Optional | indicates there is a handbook link for a description of the funnel for this PI. Should be a URL |
sisense_data |
Optional | allows a Sisense dashboard to be embeded as part of the (K)PI using chart, dashboard, and embed as neseted attributes. |
sisense_data.chart |
Optional | indicates the numeric Sisense chart/widget ID. For example: 9090628 |
sisense_data.dashboard |
Optional | indicates the numeric Sisense dashboard ID. For example: 634200 |
sisense_data.shared_dashboard |
Optional | indicates the numeric Sisense shared_dashboard ID. For example: 185b8e19-a99e-4718-9aba-96cc5d3ea88b |
sisense_data.embed |
Optional | indicates the Sisense embed version. For example: v2 |
sisense_data_secondary |
Optional | allows a second Sisense dashboard to be embeded. Same as sisense data |
sisense_data_secondary.chart |
Optional | Same as sisense_data.chart |
sisense_data_secondary.dashboard |
Optional | Same as sisense_data.dashboard |
sisense_data_secondary.shared_dashboard |
Optional | Same as sisense_data.shared_dashboard |
sisense_data_secondary.embed |
Optional | Same as sisense_data.embed |
public |
Optional | boolean flag that can be set to false where a (K)PI does not meet the public guidelines. |
pi_type |
Optional | indicates the Product PI type (Ex: AMAU, GMAU, SMAU, Group PPI) |
product_analytics_type |
Optional | indicates if the metric is available on SaaS, SM (self-managed), or Both. |
is_primary |
Optional | boolean flag that indicates if this is the Primary PI for the Product Group. |
implementation |
Optional | indicates the implementation status and reasons as nested attributes. This should be updated monthly before Key Reviews by the DRI. |
implementation.status |
Optional | indicates the Implementation Status status. This should be updated monthly before Key Reviews by the DRI. |
implementation.reasons |
Optional | indicates the reasons behind the implementation status. This should be updated monthly before Key Reviews by the DRI. Should be an array (indented lines starting with dashes) even if you only have one reason. |
lessons |
Optional | indicates lessons learned from a K(PI) as a nested attribute. This should be updated monthly before Key Reviews by the DRI. |
lessons.learned |
Optional | learned is an attribute that can be nested under lessonsand indicates lessons learned from a K(PI). This should be updated monthly before Key Reviews by the DRI. Should be an array (indented lines starting with dashes) even if you only have one lesson learned |
monthly_focus |
Optional | indicates monthly focus goals from a K(PI) as a nested attribute. This should be updated monthly before Key Reviews by the DRI. |
monthly_focus.goals |
Optional | indicates monthly focus goals from a K(PI). This should be updated monthly before Key Reviews by the DRI. Should be an array (indented lines starting with dashes) even if you only have one goal |
metric_name |
Optional | indicates the name of the metric in Self-Managed implemenation. The SaaS representation of the Self-Managed implementation should use the same name. |
e5a40db2)
