Engineering Function Performance Indicators

1. You are here:
2. Handbook
3. Engineering
4. Engineering Function Performance Indicators

Maintained by:

Eric Johnson

@edjdev

On this page

• Executive Summary
• Key Performance Indicators
  • Engineering Hiring Actual vs Plan
  • Engineering Non-Headcount Plan vs Actuals
  • Engineering Average Location Factor
  • Engineering Recruiting Average Top-of-Funnel Location Factor
  • Engineering Handbook Update Frequency
• Regular Performance Indicators
  • Diversity
  • Team Member Retention
  • Engineering Discretionary Bonus Rate
  • Engineering Division Member MR Rate
• Other PI Pages
• Legends
  • Health
  • Instrumentation
• How to work with pages like this
  • Pages
    • Helper Functions
  • Data
  • Guidelines

Executive Summary

Key Performance Indicators

Engineering Hiring Actual vs Plan

This is a subset of an existing KPI. Please see the definition for the parent KPI.

Employees are in the division `Engineering`.

Target: 525 engineering members by Jan 31, 2019

Chart (Sisense↗)

Status: Okay

• Engineering is on plan

Engineering Non-Headcount Plan vs Actuals

We need to spend our investors' money wisely. We also need to run a responsible business to be successful, and to one day go on the public market.

Target: Unknown until FY21 planning process

URL(s)

• https://app.periscopedata.com/app/gitlab/633290/Engineering-Non-Headcount-BvAs

Status: Okay

• We are below our budget significantly

Engineering Average Location Factor

We remain efficient financially if we are hiring globally, working asynchronously, and hiring great people in low-cost regions where we pay market rates. We track an average location factor by function and department so managers can make tradeoffs and hire in an expensive region when they really need specific talent unavailable elsewhere, and offset it with great people who happen to be in low cost areas.

Target: .58 average location factor

Chart (Sisense↗)

Status: Attention

• This chart appears broken so we need to talk to the data team

Engineering Recruiting Average Top-of-Funnel Location Factor

We need to be proactive in measuring our location factor, starting with candidates who are at the top of the recruiting funnel.

Target: 0.58

Chart (Sisense↗)

Status: Problem

• We need to set the target
• The chart is not actionable as-is. We need to take it to the data team to fix.
• We're not certain how this value is going over 1 periodically.

Engineering Handbook Update Frequency

The handbook is essential to working remote successfully, to keeping up our transparency, and to recruiting successfully. Our processes are constantly evolving and we need a way to make sure the handbook is being updated at a regular cadence. This data is retrieved by querying the API with a python script for merge requests that have files matching `/source/handbook/engineering/**` or `/source/handbook/support/**` over time.

Target: Unknown until we reach maturity level 2

Chart (Sisense↗)

Status: Okay

• Above target and trending up

Regular Performance Indicators

Diversity

Diversity, Inclusion & Belonging is one of our core values, and a general challenge for the tech industry. GitLab is in a privileged position to positively impact diversity in tech because our remote lifestyle should be more friendly to people who may have left the tech industry, or studied a technical field but never entered industry. This means we can add to the diversity of our industry, and not just play a zero-sum recruiting game with our competitors.

Target: It's against company policy to set diversity quotas (but we may add benchmarks to compare ourselves against).

URL(s)

• https://docs.google.com/presentation/d/1Rj6brdP2hKU1zgj0ll1poywBw0aR9jIAdT4GsLZfIIU/edit#slide=id.g4bb9f44e0b_6_160
• https://docs.google.com/spreadsheets/d/1Jef9oopd8AeZRYM2P9B8ye2TxfdzxI0FY6ixhlyzW_c/edit#gid=1035217495

Status: Attention

• Engineering is now at the tech benchmark for gender diversity (~16%), but our potential is greater and we can do better. 20% should be our floor in technical roles. Other types of diversity are unknown.
• Get data wired up and visualized in periscope.
• Clear data sharing with legal.
• Get better data about racial and country diversity.

Team Member Retention

People are a priority and attrition comes at a great human cost to the individual and team. Additionally, recruiting (backfilling attrition) is a ludicrously expensive process, so we prefer to keep the people we have :)

Target: 90%

URL(s)

• https://docs.google.com/spreadsheets/d/1Jef9oopd8AeZRYM2P9B8ye2TxfdzxI0FY6ixhlyzW_c/edit#gid=1405344200

Status: Okay

• I seem to recall our attrition is now below 10% which is great compared to the tech benchmark of 22% and the remote benchmark for 16%, but the fact that I can’t just look at a simple graph makes me nervous…
• Get into a system (BambooHR?) and visualized in periscope
• Break out voluntary and involuntary

Engineering Discretionary Bonus Rate

Discretionary bonuses offer a highly motivating way to reward individual GitLab team members who really shine as they live our values. Our goal is to award discretionary bonuses to 10% of GitLab team members in the Engineering function every month.

Target: 10%

URL(s)

Status: Unknown

• We currently track bonuses at the department level, but not at the Engineering-function level.

Engineering Division Member MR Rate

Engineering MR Rate is a key indicator showing how productive our team members are based on the average MR merged per team member. We currently count all members of the Engineering Division (Director, EMs, ICs) in the denominator because this is a team effort and we want to be cognizant of managerial overhead.

Target: 6 MRs per Month

Chart (Sisense↗)

Status: Problem

• We have found some bugs in the data, so we need to confirm it's accurate before taking any action
• We need to pick a reasonable target

Other PI Pages

• Performance Indicator Maturity Table
• Engineering Function Performance Indicators
  • Customer Support Department Performance Indicators
  • Development Department Performance Indicators
    • Growth Section Performance Indicators
  • Infrastructure Department Performance Indicators
  • Quality Department Performance Indicators
  • Security Department Performance Indicators
  • UX Department Performance Indicators
• Product Performance Indicators
  • Dev Section Performance Indicators
  • Ops Section Performance Indicators
  • Secure & Defend Section Performance Indicators
  • Enablement Section Performance Indicators
• Corporate Finance Performance Indicators
• Recruiting Performance Indicators
• People Success Indicators
• Data Team KPI Index

Legends

Health

Instrumentation

How to work with pages like this

Pages

Pages, such as the Engineering Function Performance Indicators page are rendered by an ERB template that contains HTML code.

• Any changes to the Performance Indicator (PI) ERB template will cascade to all links under the Other PI Pages section
• To add, delete, or update (K)PI information (including title, description, etc), make changes to the Performance Indicators data file. See more details in the Data section below.
• To update the way each individual (K)PI displays:
  • make changes to the Performance Indicators List ERB file for how the (K)PI is displayed on the KPI Index page
  • make changes to the Performance Indicators ERB file for any other properties and its format
• When adding a new PI page,
  • create the new PI page in the business division's handbook section (like the Engineering Division page)
  • include the relative link to the PI page in the Performance Indicators List ERB file
  • add all the (K)PI information into the Performance Indicators data file

Helper Functions

These ERB templates calls custom helper functions that extract and transform data from the Performance Indicators data file.

• The kpi_list_by_org(org) helper function takes a required string argument named org (deparment or division level) that returns all the KPIs (pi.is_key == true) for a specific organization grouping (pi.org == org) from the Performance Indicators data file.
• The pi_maturity_level(performance_indicator) helper function automatically assigns a maturity level based on the availability of certain data properties for a particular PI.
• The pi_maturity_reasons(performance_indicator) helper function returns a reason for a PI maturity based on other data properties.
• The performance_indicator_target(name) helper function returns the target value from the target property for a specific PI from the Performance Indicators data file.
• The performance_indicators(org) takes a required string argument named org (deparment or division level) that returns two lists - a list of all KPIs and a list of all PIs for a specific organization grouping (department/division).
• The signed_periscope_url(data) takes in the sisense_data property information from the Performance Indicators data file and returns a signed chart URL for embedding a Sisense chart into the handbook.

Data

The heart of pages like this is the Performance Indicators data file which is in YAML file. Each - denotes a dictionary of values for a new (K)PI. The current elements (or data properties) are:

Guidelines

• Each KPI chart should be a timeseries chart.
  • Use Purple bars to denote values.
  • Use a single Green bar for the current month (in progress).
  • Use a Red stepped-line for timeseries target.
  • Optional: Use a Black dashed-line for rolling average.
• Each KPI should have a standalone dashboard with a single chart representing the KPI and a text box with a link back to the handbook definition.
  • In Sisense, create a shared dashboard link to get the shared dashboard ID.
  • In Sisense, use the Share Link action of the chart to get the chart (widget_id) and the dashboard ID.
  • Add the shared_dashboard, chart , and the dashboard key-value pairs to the Performance Indicators data file under the sisense_data property
• Avoid : in strings as it's an important character in YAML and will confuse the data parsing process. Put the string in "quotes" if you really need to use a :
• urls: should be an array (indented lines starting with dashes) even if you only have one url
• Both maturity.level and health.level display a value between 0 and 3 (inclusive).