Development Department Performance Indicators

1. You are here:
2. Handbook
3. Engineering
4. Development Department
5. Development Department Performance Indicators

On this page

• Executive Summary
• Key Performance Indicators
• Regular Performance Indicators
• Other PI Pages
• Legends
  • Maturity
  • Health
• How to work with pages like this
  • Data
  • Pages

Executive Summary

KPI	Health	Reason(s)
Development Hiring Actual vs Plan	Attention	Development is projected to not hit quarterly goal (46 vs. 54). We overshot Q2 which keeps us on track (63 vs 42). We have seen some variability in sourcing which has affected us. Health: Monitor health closely
Development Average Location Factor	Attention	We are at our target of 0.58 exactly overall, but trending upward. We need to get the target location factors in the charts. It will probably be cleaner if we get this in periscope. We need to set the target location factors on our vacancies and make sure recruiting is targeting the right areas of the globe on a per role basis.
Throughput	Attention	Move to single code base affected throughput by roughly 20%. One time charge associated with the transition. What we need to do next to raise maturity. Update the chart to include all repos, but not dev. First pass of periscope updates made. Need to fix ETL bug.
Average MRs/Development Engineers/Month	Problem	Trends are showing negative impact. Single code base has slowed this metric and we haven't recovered. Issue open to explore this 'https://gitlab.com/gitlab-com/www-gitlab-com/issues/5375' Health: What we need to do next to improve metric - compare to the future trend of hiring/onboarding rates.
Mean time to merge (MTTM)	Okay	Because of our desire to encourage early development we are putting a high ceiling on this metric - 14 days for average.

Key Performance Indicators

Development Hiring Actual vs Plan

Are we able to hire high quality workers to build our product vision in a timely manner? Hiring information comes from BambooHR where employees are in the division `Engineering`.

Target: 276 by February 1, 2020

URL(s)

• /handbook/hiring/charts/development-department/

Health: Attention

• Development is projected to not hit quarterly goal (46 vs. 54). We overshot Q2 which keeps us on track (63 vs 42). We have seen some variability in sourcing which has affected us.
• Health: Monitor health closely

Maturity: Level 2 of 3

• Needs periscope data

Development Average Location Factor

We remain efficient financially if we are hiring globally, working asynchronously, and hiring great people in low-cost regions where we pay market rates. We track an average location factor by function and department so managers can make tradeoffs and hire in an expensive region when they really need specific talent unavailable elsewhere, and offset it with great people who happen to be in low cost areas.

Target: 0.58

URL(s)

• /handbook/hiring/charts/development-department/

Health: Attention

• We are at our target of 0.58 exactly overall, but trending upward.
• We need to get the target location factors in the charts.
• It will probably be cleaner if we get this in periscope.
• We need to set the target location factors on our vacancies and make sure recruiting is targeting the right areas of the globe on a per role basis.

Maturity: Level 2 of 3

• Needs periscope data

Throughput

Throughput shows the number of Merge Requests (MRs) on a month by month basis. It’s important because it shows the overall development teams velocity. The Senior Director of Development is the DRI on what projects are included.

Target: 20% increase quarter of quarter

URL(s)

• https://quality-dashboard.gitlap.com/groups/gitlab-org/monthly_merged_mrs

Chart (Periscope↗)

Health: Attention

• Move to single code base affected throughput by roughly 20%. One time charge associated with the transition.
• What we need to do next to raise maturity. Update the chart to include all repos, but not dev.
• First pass of periscope updates made. Need to fix ETL bug.

Maturity: Level 3 of 3

Average MRs/Development Engineers/Month

Average MRs per Development Engineer (an IC in the Development Department) per month is a monthly evaluation of how MRs on average an author performs. It’s important because it measures productivity. We include external contributions from the wider community. We want to be efficient at accepting community contributions, as a result, the data is analyzed on a per category/group basis and never per GitLab team or team members. The Senior Director of Development is the DRI on what projects are included.

Target: 8 MRs per Development Engineer per Month

Chart (Periscope↗)

Health: Problem

• Trends are showing negative impact. Single code base has slowed this metric and we haven't recovered. Issue open to explore this 'https://gitlab.com/gitlab-com/www-gitlab-com/issues/5375'
• Health: What we need to do next to improve metric - compare to the future trend of hiring/onboarding rates.

Maturity: Level 3 of 3

Mean time to merge (MTTM)

To be aligned with CycleTime from Development. Monthly mean time to merge MRs, it tells us on average how long it takes from submitting code to being merged. The Senior Director of Development is the DRI on what projects are included.

Target: On average under 14 days

Chart (Periscope↗)

Health: Okay

• Because of our desire to encourage early development we are putting a high ceiling on this metric - 14 days for average.

Maturity: Level 3 of 3

Regular Performance Indicators

Average MRs per Author vs Growth

Average MRs per Developer per month is a monthly evaluation of how MRs on average an author performs. Growth represents how fast we are growing as an organization. It’s important because time spent getting new members up to speed can affect a teams velocity.

Target: 0% growth 10 MRs per Engineer, 10% growth 8 MRs per Engineering, 15% growth 7 MRs per Engineer

URL(s)

• https://app.periscopedata.com/app/gitlab/463858/Engineering-KPIs

Health: Attention

• We know this is causal, we have a good understanding of the impact at this point.
• What we need to do next to raise maturity - Rebaseline measurement based on new headcount.
• What we need to do next to improve metric - We are attempting to hold the line the performance as we hire. We will continue to monitor.
• Further automation of pulling of this metric.

Maturity: Level 2 of 3

• Needs periscope data

Response to Community SLO

Measurement of time from Community member MR proposed till GitLab response. It’s important because it shows our commitment to community and engagement.

Target: No Target Set

URL(s)

• https://gitlab.com/gitlab-com/www-gitlab-com/merge_requests/20165

Health: Unknown

• We haven’t started measuring it yet.
• What we need to do next to raise maturity - start measuring.
• What we need to do next to improve metric - unknown.
• Import into Periscope for better tracking.

Maturity: Level 2 of 3

• Needs periscope data

Backend Unit Test Coverage

BE Unit Test coverage shows the unit test coverage of our code base. As an example 95% represents that 95% of the LOC in our BE software is unit tested. It’s important as it shows how much code is tested early in the development process.

Target: 95%

URL(s)

• https://app.periscopedata.com/app/gitlab/475029/Source-Code-KPI's

Health: Okay

• This metric's threshold for action is around 95%.
• We could spend further time on coded tests, it’s not recommended at this point unless we are finding significant defect correlation in our non-unit-test coverage areas.

Maturity: Level 2 of 3

• Needs periscope data

Frontend Unit Test Coverage

FE Unit Test coverage shows the unit test coverage of our code base. As an example 95% represents that 95% of the LOC in our FE software is unit tested. It’s important as it shows how much code is tested early in the development process. We are currently converting from Karma to Jest organically, so the performance indicator needs to show total coverage of the combined coverage.

Target: 75%

URL(s)

• https://app.periscopedata.com/app/gitlab/475029/Source-Code-KPI's

Health: Attention

• Definite room for improvement, though it needs to be caveated with legacy vs. new code. Move from Karma to jest and lack of combination of testing has led to most recent decline.
• What we need to do next to raise maturity - Establish a bar for this work as some legacy factors affect code coverage.
• What we need to do next to improve metric - Provide adequate code coverage on new code and any particularly problematic areas in legacy.

Maturity: Level 2 of 3

• Needs periscope data

Other PI Pages

• Performance Indicator Maturity Table
• Engineering Function Performance Indicators
  • Development Department Performance Indicators
    • Growth Section Performance Indicators
  • Infrastructure Department Performance Indicators
  • Quality Department Performance Indicators
  • Security Department Performance Indicators
  • Support Department Performance Indicators
  • UX Department Performance Indicators
• Data Team Metrics Overview

Legends

Maturity

Level	Meaning
Level 3 of 3	Has a description, target, and periscope data.
Level 2 of 3	Missing one of: description, target, or periscope data.
Level 1 of 3	Missing two of: description, target, or periscope data.
Level 0 of 3	Missing a description, a target, and periscope data.

Health

Level	Meaning
Okay	The KPI is at an acceptable level compared to the threshold
Attention	This is a blip, or we’re going to watch it, or we just need to enact a proven intervention
Problem	We'll prioritize our efforts here
Unknown	Unknown

How to work with pages like this

Data

The heart of pages like this is a data file called /data/performance_indicators.yml which is in YAML format. Almost everything you need to do will involve edits to this file. Here are some tips:

• Avoid : in strings as it's an important character in YAML and will confuse the data parsing process. Put the string in "quotes" if you really need to use a :
• urls: should be an array (indented lines starting with dashes) even if you only have one url
• Both maturity.level and health.level take a number between 0 and 3 (inclusive)

Pages

Pages like /handbook/engineering/performance-indicators/ are rendered by and ERB template.

These ERB templates call the helper function performance_indicators() that is defined in /helpers/custom_helpers.rb. This helper function calls in several partial templates to do it's work.

This function takes a required argument named org in string format that limits the scope of the page to a portion of the data file. Possible valid values for this org argument are listed in the org property of each element in the array in /data/performance_indicators.yml.