| KPI | Health | Reason | Next Steps |
| Hiring Actual vs Plan | Okay | Engineering is on plan. But we are lending some of our recruiters to sales for this quarter. And we just put in place a new "one star minimum" rule that might decrease offer volume. | |
| GitLab.com Availability | Attention | We’re above the SLO threshold, but we also know the data needs to be better | |
| Infrastructure Cost per GitLab.com Monthly Active Users | Attention | Met savings goals last quarter in working group. | |
| GitLab.com Performance | Attention | We are experiencing occasionally slowness of both the frontend and git operations. | |
| Infrastructure cost vs plan | Attention | Met savings goals last quarter in working group. | |
Are we able to hire high quality workers to build our product vision in a timely manner? Hiring information comes from BambooHR where employees are in the division `Engineering`.
URL(s)
Health: Okay
Engineering is on plan. But we are lending some of our recruiters to sales for this quarter. And we just put in place a new "one star minimum" rule that might decrease offer volume.
Maturity: Level 2 of 3
We have charts driven off of team.yml
Next Steps
Percentage of time during which GitLab.com is fully operational and providing service to users within SLO parameters.
URL(s)
Health: Attention
We’re above the SLO threshold, but we also know the data needs to be better
Maturity: Level 2 of 3
We have a good understanding of the metric and are currently collecting it via Pingdom, but we need to implement it as a proper SLO.
Next Steps
This metric reflects the dollar cost necessary to support one user in GitLab.com. It is an important metric because it allows us to estimate Infrastructure costs as our user base grows. Infrastructure cost comes from Netsuite; it is all expenses with the department name of `Infrastructure`, excluding account 6999 (Allocation). This cost is divided by [MAU](/handbook/product/metrics/#monthly-active-user-mau)
URL(s)
Health: Attention
Met savings goals last quarter in working group.
Maturity: Level 2 of 3
We don’t have a good way to accurately track this cost yet.
Next Steps
This metric needs to reflect the performance of GitLab as experienced by users. It should capture both frontend and backend performance. Even though the Infrastructure will be responsible for this metric they will need other departments such as Development, Quality, PM, and UX to positively affect change.
URL(s)
Health: Attention
We are experiencing occasionally slowness of both the frontend and git operations.
Maturity: Level 1 of 3
We need better tooling, better data capture, and to wire up this metric to the prioritization of performance issues.
Next Steps
Tracks our actual infrastructure against our planned infrastructure costs for GitLab.com. We need this metric to manage our financial position. [Description of what this metric tells us, and why it’s important]
URL(s)
Health: Attention
Met savings goals last quarter in working group.
Maturity: Level 1 of 3
We don’t have a good way to accurately track this cost yet.
Next Steps
Diversity is one of our core values, and a general challenge for the tech industry. GitLab is in a privileged position to positively impact diversity in tech because our remote lifestyle should be more friendly to people who may have left the tech industry, or studied a technical field but never entered industry. This means we can add to the diversity of our industry, and not just play a zero-sum recruiting game with our competitors.
URL(s)
Health: Attention
Engineering is now at the tech benchmark for gender diversity (~16%), but our potential is greater and we can do better. 20% should be our floor in technical roles. Other types of diversity are unknown.
Maturity: Level 2 of 3
The content is shared only in a closed metrics review, and does not have granularity. It’s not visualized, or in time series.
Next Steps
The handbook is essential to working remote successfully, to keeping up our transparency, and to recruiting successfully. Our processes are constantly evolving and we need a way to make sure the handbook is being updated at a regular cadence.
URL(s)
Health: Unknown
Unknown. But my sense is we are not doing enough. For instance, we have not been able to fully update the handbook after the development department re-org (dev backend, and ops backend are still present. Although many of the new teams do have their own pages already)
Maturity: Level 2 of 3
We currently just have contribution graphs, which are a poor proxy for this.
Next Steps
People are a priority and attrition comes at a great human cost to the individual and team. Additionally, recruiting (backfilling attrition) is a ludicrously expensive process, so we prefer to keep the people we have :)
URL(s)
Health: Okay
I seem to recall our attrition is now below 10% which is great compared to the tech benchmark of 22% and the remote benchmark for 16%, but the fact that I can’t just look at a simple graph makes me nervous...
Maturity: Level 2 of 3
There is manually curated data in a spreadsheet from PO
Next Steps
Each service at GitLab has two general metrics SLOS. "Apdex Score" is simply put, this is a measure of the percentage of requests to that service that complete within a satisfactory amount of time. The thresholds are defined per service, so for some services it will be in microseconds for others it could be seconds. "Error Ratio" is the percentage of requests to a service which end in error. For each service in the system we define an acceptable threshold for these values. For apdex we want the actual score to be above the threshold, for error ratio, we want it to be below the threshold. We don’t expect the apdex to be a perfect 100%, and we don’t expect the error rate to be a perfect 0%, but we would like these values to be within their predefined thresholds 100% of the time. The actual amount of time that they adhere to their SLO thresholds is far below this currently
URL(s)
Health: Unknown
TBD
Maturity: Level 0 of 3
TBD
Next Steps
Measures the mean amount of time elapsed between incidents that affect GitLab.com’s availability.
URL(s)
Health: Problem
We can’t properly evaluate the KPI.
Maturity: Level 2 of 3
We currently have to calculate this from issues in the production queue, and even that’s not entirely accurate because we don’t yet have strict definitions of start and end times for incidents. Currently, we only review this data at the end of a quarter to generate error budgets.
Next Steps
Tracks the total number of deployments to production over the course of the month due to regressions and/or outages, which allows us to measure our deployment speed to production.
URL(s)
Health: Attention
CD is Delivery's highest priority; we just started collecting data, so no targets are set for the KPI yet.
Maturity: Level 2 of 3
Data is currently being collected in a JSON file and will move to a Graphana dashboard by the end of Q2.
Next Steps
Tracks number of failed deployments to production over the course of the month due to regressions and/or outages, which allows us to measure our production deployment readiness.
URL(s)
Health: Problem
We are not yet tracking this KPI
Maturity: Level 1 of 3
We have not yet tracking this KPI.
Next Steps
Tracks time to recover full operational status in case of a catastrophic incident in our primary production environment.
URL(s)
Health: Unknown
DR environment is not operational.
Maturity: Level 1 of 3
The DR environment is not fully up and running yet due to various bugs that have delayed the roll out, and we need to establish regular fire drills to establish a baseline.
Next Steps
| Level | Meaning |
| Level 3 of 3 | Measurable, time series, identified target for the metric, automated data extraction, dashboard in Periscope available to the whole company (if not the whole world) |
| Level 2 of 3 | About two-thirds done. E.g. Missing one of: automated data collection, defined threshold, or periscope dashboard. |
| Level 1 of 3 | About one-third done. E.g. Has one of: automated data collection, defined threshold, or periscope dashboard. |
| Level 0 of 3 | We only have an idea or a plan. |
| Level | Meaning |
| Okay | The KPI is at an acceptable level compared to the threshold |
| Attention | This is a blip, or we’re going to watch it, or we just need to enact a proven intervention |
| Problem | We'll prioritize our efforts here |
| Unknown | Unknown |
The heart of pages like this is a data file called /data/performance_indicators.yml which is in YAML format. Almost everything you need to do will involve edits to this file. Here are some tips:
: in strings as it's an important character in YAML and will confuse the data parsing process. Put the string in "quotes" if you really need to use a :urls: should be an array (indented lines starting with dashes) even if you only have one urlmaturity.level and health.level take a number between 0 and 3 (inclusive)Pages like /handbook/engineering/performance-indicators/ are rendered by and ERB template.
These ERB templates call the helper function performance_indicators() that is defined in /helpers/custom_helpers.rb. This helper function calls in several partial templates to do it's work.
This function takes a required argument named org in string format that limits the scope of the page to a portion of the data file. Possible valid values for this org argument are listed in the orgs property of each element in the array in /data/performance_indicators.yml.
