Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Development Department Performance Indicators

On this page

Executive Summary

KPI Health Reason Next Steps
Hiring Actual vs Plan Okay Engineering is on plan. But we are lending some of our recruiters to sales for this quarter. And we just put in place a new "one star minimum" rule that might decrease offer volume.
  • Health: Monitor health closely
  • Maturity: Get this into periscope
  • Throughput Okay We are current seeing appropriate growth trends. We want to continue to monitor this for further action.
  • What we need to do next to raise maturity. Update the chart to include all repos, but not dev.
  • First pass of periscope updates made. Need to fix ETL bug.
  • Average MRs/Dev/Month Attention We are recalibrating to set headcount based on senior leadership guidance. We need to establish new goal and expectations relative to current growth plans. With additional vacation and starts in August original metric dropped below 8.
  • Health: What we need to do next to improve metric - compare to the future trend of hiring/onboarding rates.
  • Maturity: Establish new bar with historical data and manage
  • Mean time to merge (MTTM) Okay Because of our desire to encourage early development we are putting a high ceiling on this metric - 14 days for average.
  • None
  • Key Performance Indicators

    Hiring Actual vs Plan

    Are we able to hire high quality workers to build our product vision in a timely manner? Hiring information comes from BambooHR where employees are in the division `Engineering`.

    URL(s)

    Health: Okay

    Engineering is on plan. But we are lending some of our recruiters to sales for this quarter. And we just put in place a new "one star minimum" rule that might decrease offer volume.

    Maturity: Level 2 of 3

    We have charts driven off of team.yml

    Next Steps

    Throughput

    Throughput shows the number of Merge Requests (MRs) on a month by month basis. It’s important because it shows the overall development teams velocity. The Senior Director of Development is the DRI on what projects are included.

    URL(s)

    Health: Okay

    We are current seeing appropriate growth trends. We want to continue to monitor this for further action.

    Maturity: Level 3 of 3

    We currently have automation and dashboard, thresholds need to be established.

    Next Steps

    Average MRs/Dev/Month

    Average MRs per Developer per month is a monthly evaluation of how MRs on average an author performs. It’s important because it measures productivity. The Senior Director of Development is the DRI on what projects are included.

    URL(s)

    Health: Attention

    We are recalibrating to set headcount based on senior leadership guidance. We need to establish new goal and expectations relative to current growth plans. With additional vacation and starts in August original metric dropped below 8.

    Maturity: Level 2 of 3

    We currently have automation and dashboard, thresholds need to be recalibrated. Denominator (dev) needs to be calibrated.

    Next Steps

    Mean time to merge (MTTM)

    To be aligned with CycleTime from Development. Monthly mean time to merge MRs, it tells us on average how long it takes from submitting code to being merged. The Senior Director of Development is the DRI on what projects are included.

    URL(s)

    Health: Okay

    Because of our desire to encourage early development we are putting a high ceiling on this metric - 14 days for average.

    Maturity: Level 3 of 3

    In Periscope.

    Next Steps

    Regular Performance Indicators

    Diversity

    Diversity & Inclusion is one of our core values, and a general challenge for the tech industry. GitLab is in a privileged position to positively impact diversity in tech because our remote lifestyle should be more friendly to people who may have left the tech industry, or studied a technical field but never entered industry. This means we can add to the diversity of our industry, and not just play a zero-sum recruiting game with our competitors.

    URL(s)

    Health: Attention

    Engineering is now at the tech benchmark for gender diversity (~16%), but our potential is greater and we can do better. 20% should be our floor in technical roles. Other types of diversity are unknown.

    Maturity: Level 2 of 3

    The content is shared only in a closed metrics review, and does not have granularity. It’s not visualized, or in time series.

    Next Steps

    Handbook Update Frequency

    The handbook is essential to working remote successfully, to keeping up our transparency, and to recruiting successfully. Our processes are constantly evolving and we need a way to make sure the handbook is being updated at a regular cadence.

    URL(s)

    Health: Unknown

    Unknown. But my sense is we are not doing enough. For instance, we have not been able to fully update the handbook after the development department re-org (dev backend, and ops backend are still present. Although many of the new teams do have their own pages already)

    Maturity: Level 2 of 3

    We currently just have contribution graphs, which are a poor proxy for this.

    Next Steps

    Team Member Retention

    People are a priority and attrition comes at a great human cost to the individual and team. Additionally, recruiting (backfilling attrition) is a ludicrously expensive process, so we prefer to keep the people we have :)

    URL(s)

    Health: Okay

    I seem to recall our attrition is now below 10% which is great compared to the tech benchmark of 22% and the remote benchmark for 16%, but the fact that I can’t just look at a simple graph makes me nervous...

    Maturity: Level 2 of 3

    There is manually curated data in a spreadsheet from PO

    Next Steps

    Average MRs per Author vs Growth

    Average MRs per Developer per month is a monthly evaluation of how MRs on average an author performs. Growth represents how fast we are growing as an organization. It’s important because time spent getting new members up to speed can affect a teams velocity.

    URL(s)

    Health: Attention

    We know this is causal, we have a good understanding of the impact at this point.

    Maturity: Level 2 of 3

    We can now see information in periscope via the development department headcount.

    Next Steps

    Response to Community SLO

    Measurement of time from Community member MR proposed till GitLab response. It’s important because it shows our commitment to community and engagement.

    URL(s)

    Health: Unknown

    We haven’t started measuring it yet.

    Maturity: Level 1 of 3

    We haven’t started measuring it yet.

    Next Steps

    Backend Unit Test Coverage

    BE Unit Test coverage shows the unit test coverage of our code base. As an example 95% represents that 95% of the LOC in our BE software is unit tested. It’s important as it shows how much code is tested early in the development process.

    URL(s)

    Health: Okay

    This metric's threshold for action is around 95%.

    Maturity: Level 3 of 3

    In Periscope and being updated.

    Next Steps

    Frontend Unit Test Coverage

    FE Unit Test coverage shows the unit test coverage of our code base. As an example 95% represents that 95% of the LOC in our FE software is unit tested. It’s important as it shows how much code is tested early in the development process.

    URL(s)

    Health: Attention

    Definite room for improvement, though it needs to be caveated with legacy vs. new code. Move from Karma to jest and lack of combination of testing has led to most recent decline.

    Maturity: Level 2 of 3

    In Periscope and being updated manually.

    Next Steps

    Acquisition and Conversion (IACV)

    Increased value in customer bookings on gitlab.com by acquiring new customers and upselling existing customers to a higher tier through data driven initiatives that help surface our existing capabilities to prospective and existing users.

    URL(s)

    Health: Unknown

    Need to validate current health here.

    Maturity: Level 2 of 3

    Data now dashboarded in Periscope.

    Next Steps

    Expansion and User Retention

    Reduced customer churn on gitlab.com by identifying and re-engaging customers that have created accounts but have not adopted our product.

    URL(s)

    Health: Unknown

    No metrics collected.

    Maturity: Level 2 of 3

    Data now dashboarded in Periscope.

    Next Steps

    Other PI Pages

    Legends

    Maturity

    Level Meaning
    Level 3 of 3 Measurable, time series, identified target for the metric, automated data extraction, dashboard in Periscope available to the whole company (if not the whole world)
    Level 2 of 3 About two-thirds done. E.g. Missing one of: automated data collection, defined threshold, or periscope dashboard.
    Level 1 of 3 About one-third done. E.g. Has one of: automated data collection, defined threshold, or periscope dashboard.
    Level 0 of 3 We only have an idea or a plan.

    Health

    Level Meaning
    Okay The KPI is at an acceptable level compared to the threshold
    Attention This is a blip, or we’re going to watch it, or we just need to enact a proven intervention
    Problem We'll prioritize our efforts here
    Unknown Unknown

    How to work with pages like this

    Data

    The heart of pages like this is a data file called /data/performance_indicators.yml which is in YAML format. Almost everything you need to do will involve edits to this file. Here are some tips:

    Pages

    Pages like /handbook/engineering/performance-indicators/ are rendered by and ERB template.

    These ERB templates call the helper function performance_indicators() that is defined in /helpers/custom_helpers.rb. This helper function calls in several partial templates to do it's work.

    This function takes a required argument named org in string format that limits the scope of the page to a portion of the data file. Possible valid values for this org argument are listed in the orgs property of each element in the array in /data/performance_indicators.yml.