| KPI | Health | Reason | Next Steps |
| Hiring Actual vs Plan | Okay | Engineering is on plan. But we are lending some of our recruiters to sales for this quarter. And we just put in place a new "one star minimum" rule that might decrease offer volume. | |
| Throughput | Attention | We are current seeing appropriate growth trends. We want to continue to monitor this for further action. | |
| Average MRs/Dev/Month | Attention | The metric has not fully recovered to ~10. We are seeing positive trends, but need to address growth negative correlation with this metric. | |
| Mean time to merge (MTTM) | Problem | Rough charting tells us that the 85th percentile over the last 7 months takes upwards of 10+ days to merge. We are doing more investigating into cycle time of engineering, increasing maintainers. Working group established. | |
Are we able to hire high quality workers to build our product vision in a timely manner? Hiring information comes from BambooHR where employees are in the division `Engineering`.
URL(s)
Health: Okay
Engineering is on plan. But we are lending some of our recruiters to sales for this quarter. And we just put in place a new "one star minimum" rule that might decrease offer volume.
Maturity: Level 2 of 3
We have charts driven off of team.yml
Next Steps
Throughput shows the number of Merge Requests (MRs) on a month by month basis. It’s important because it shows the overall development teams velocity. The Senior Director of Development is the DRI on what projects are included.
URL(s)
Health: Attention
We are current seeing appropriate growth trends. We want to continue to monitor this for further action.
Maturity: Level 3 of 3
We currently have automation and dashboard, thresholds need to be established. Need to move metric to Periscope.
Next Steps
Average MRs per Developer per month is a monthly evaluation of how MRs on average an author performs. It’s important because it measures productivity. The Senior Director of Development is the DRI on what projects are included.
URL(s)
Health: Attention
The metric has not fully recovered to ~10. We are seeing positive trends, but need to address growth negative correlation with this metric.
Maturity: Level 2 of 3
We currently have automation and dashboard, thresholds need to be established. We need to move metric to periscope.
Next Steps
To be aligned with CycleTime from Development. Monthly mean time to merge MRs, it tells us on average how long it takes from submitting code to being merged. The Senior Director of Development is the DRI on what projects are included.
URL(s)
Health: Problem
Rough charting tells us that the 85th percentile over the last 7 months takes upwards of 10+ days to merge. We are doing more investigating into cycle time of engineering, increasing maintainers. Working group established.
Maturity: Level 2 of 3
In Periscope. We need to determine thresholds for success/failure.
Next Steps
Diversity is one of our core values, and a general challenge for the tech industry. GitLab is in a privileged position to positively impact diversity in tech because our remote lifestyle should be more friendly to people who may have left the tech industry, or studied a technical field but never entered industry. This means we can add to the diversity of our industry, and not just play a zero-sum recruiting game with our competitors.
URL(s)
Health: Attention
Engineering is now at the tech benchmark for gender diversity (~16%), but our potential is greater and we can do better. 20% should be our floor in technical roles. Other types of diversity are unknown.
Maturity: Level 2 of 3
The content is shared only in a closed metrics review, and does not have granularity. It’s not visualized, or in time series.
Next Steps
The handbook is essential to working remote successfully, to keeping up our transparency, and to recruiting successfully. Our processes are constantly evolving and we need a way to make sure the handbook is being updated at a regular cadence.
URL(s)
Health: Unknown
Unknown. But my sense is we are not doing enough. For instance, we have not been able to fully update the handbook after the development department re-org (dev backend, and ops backend are still present. Although many of the new teams do have their own pages already)
Maturity: Level 2 of 3
We currently just have contribution graphs, which are a poor proxy for this.
Next Steps
People are a priority and attrition comes at a great human cost to the individual and team. Additionally, recruiting (backfilling attrition) is a ludicrously expensive process, so we prefer to keep the people we have :)
URL(s)
Health: Okay
I seem to recall our attrition is now below 10% which is great compared to the tech benchmark of 22% and the remote benchmark for 16%, but the fact that I can’t just look at a simple graph makes me nervous...
Maturity: Level 2 of 3
There is manually curated data in a spreadsheet from PO
Next Steps
Average MRs per Developer per month is a monthly evaluation of how MRs on average an author performs. Growth represents how fast we are growing as an organization. It’s important because time spent getting new members up to speed can affect a teams velocity.
URL(s)
Health: Problem
We know this is causal, but we don’t have a good quantification of the impact at this point.
Maturity: Level 2 of 3
We currently manually pull this information.
Next Steps
Measurement of time from Community member MR proposed till GitLab response. It’s important because it shows our commitment to community and engagement.
URL(s)
Health: Unknown
We haven’t started measuring it yet.
Maturity: Level 1 of 3
We haven’t started measuring it yet.
Next Steps
BE Unit Test coverage shows the unit test coverage of our code base. As an example 95% represents that 95% of the LOC in our BE software is unit tested. It’s important as it shows how much code is tested early in the development process.
URL(s)
Health: Okay
This metric's threshold for action is around 95%.
Maturity: Level 3 of 3
In Periscope and being updated.
Next Steps
FE Unit Test coverage shows the unit test coverage of our code base. As an example 95% represents that 95% of the LOC in our FE software is unit tested. It’s important as it shows how much code is tested early in the development process.
URL(s)
Health: Attention
Definite room for improvement, though it needs to be caveated with legacy vs. new code. Move from Karma to jest and lack of combination of testing has led to most recent decline. Jest testing is at approximately 16% which when added shows improvement to ~70%.
Maturity: Level 3 of 3
In Periscope and being updated
Next Steps
Increased value in customer bookings on gitlab.com by acquiring new customers and upselling existing customers to a higher tier through data driven initiatives that help surface our existing capabilities to prospective and existing users.
URL(s)
Health: Unknown
Need to validate current health here.
Maturity: Level 2 of 3
Data now dashboarded in Periscope.
Next Steps
Reduced customer churn on gitlab.com by identifying and re-engaging customers that have created accounts but have not adopted our product.
URL(s)
Health: Unknown
No metrics collected.
Maturity: Level 2 of 3
Data now dashboarded in Periscope.
Next Steps
| Level | Meaning |
| Level 3 of 3 | Measurable, time series, identified target for the metric, automated data extraction, dashboard in Periscope available to the whole company (if not the whole world) |
| Level 2 of 3 | About two-thirds done. E.g. Missing one of: automated data collection, defined threshold, or periscope dashboard. |
| Level 1 of 3 | About one-third done. E.g. Has one of: automated data collection, defined threshold, or periscope dashboard. |
| Level 0 of 3 | We only have an idea or a plan. |
| Level | Meaning |
| Okay | The KPI is at an acceptable level compared to the threshold |
| Attention | This is a blip, or we’re going to watch it, or we just need to enact a proven intervention |
| Problem | We'll prioritize our efforts here |
| Unknown | Unknown |
The heart of pages like this is a data file called /data/performance_indicators.yml which is in YAML format. Almost everything you need to do will involve edits to this file. Here are some tips:
: in strings as it's an important character in YAML and will confuse the data parsing process. Put the string in "quotes" if you really need to use a :urls: should be an array (indented lines starting with dashes) even if you only have one urlmaturity.level and health.level take a number between 0 and 3 (inclusive)Pages like /handbook/engineering/performance-indicators/ are rendered by and ERB template.
These ERB templates call the helper function performance_indicators() that is defined in /helpers/custom_helpers.rb. This helper function calls in several partial templates to do it's work.
This function takes a required argument named org in string format that limits the scope of the page to a portion of the data file. Possible valid values for this org argument are listed in the orgs property of each element in the array in /data/performance_indicators.yml.
