Engineering Function Performance Indicators

1. You are here:
2. Handbook
3. Engineering
4. Engineering Function Performance Indicators

On this page

• Executive Summary
• Key Performance Indicators
• Regular Performance Indicators
• Other PI Pages
• Legends
  • Maturity
  • Health
• How to work with pages like this
  • Data
  • Pages

Executive Summary

KPI	Health	Reason(s)
Engineering Hiring Actual vs Plan	Okay	Engineering is on plan. But we are lending some of our recruiters to sales for this quarter. And we just put in place a new "one star minimum" rule that might decrease offer volume. Health: Monitor health closely
Non-headcount budget vs plan	Unknown	Currently finance tells me when there is a problem, I’m not self-service. Get the budget captured in a system Chart budget vs. actual over time in periscope
Engineering Average Location Factor	Attention	We are at our target of 0.58 exactly overall, but trending upward. We need to get the target location factors in the charts. It will probably be cleaner if we get this in periscope. We need to set the target location factors on our vacancies and make sure recruiting is targeting the right areas of the globe on a per role basis.
Handbook Update Frequency	Unknown	Unknown. But my sense is we are not doing enough. For instance, we have not been able to fully update the handbook after the development department re-org (dev backend, and ops backend are still present. Although many of the new teams do have their own pages already) We’d like to query the API for merge requests that have files matching /source/handbook/engineering/** over time Also facet by Department Visualize in Periscope

Key Performance Indicators

Engineering Hiring Actual vs Plan

Are we able to hire high quality workers to build our product vision in a timely manner? Hiring information comes from BambooHR where employees are in the division `Engineering`.

Target: 525 engineering members by Jan 31, 2019

URL(s)

• /handbook/hiring/charts/engineering-function/

Health: Okay

• Engineering is on plan. But we are lending some of our recruiters to sales for this quarter. And we just put in place a new "one star minimum" rule that might decrease offer volume.
• Health: Monitor health closely

Maturity: Level 2 of 3

• Needs periscope data

Non-headcount budget vs plan

We need to spend our investors' money wisely. We also need to run a responsible business to be successful, and to one day go on the public market.

Target: Unknown until FY21 planning process

URL(s)

• https://docs.google.com/spreadsheets/d/15iPViTjJvLHUoU10lAUq5Rp3hm0xquC987cGISimm88/edit#gid=137507826

Health: Unknown

• Currently finance tells me when there is a problem, I’m not self-service.
• Get the budget captured in a system
• Chart budget vs. actual over time in periscope

Maturity: Level 2 of 3

• Needs periscope data

Engineering Average Location Factor

We remain efficient financially if we are hiring globally, working asynchronously, and hiring great people in low-cost regions where we pay market rates. We track an average location factor by function and department so managers can make tradeoffs and hire in an expensive region when they really need specific talent unavailable elsewhere, and offset it with great people who happen to be in low cost areas.

Target: .58 average location factor

URL(s)

• /handbook/hiring/charts/engineering-function/

Health: Attention

• We are at our target of 0.58 exactly overall, but trending upward.
• We need to get the target location factors in the charts.
• It will probably be cleaner if we get this in periscope.
• We need to set the target location factors on our vacancies and make sure recruiting is targeting the right areas of the globe on a per role basis.

Maturity: Level 2 of 3

• Needs periscope data

Handbook Update Frequency

The handbook is essential to working remote successfully, to keeping up our transparency, and to recruiting successfully. Our processes are constantly evolving and we need a way to make sure the handbook is being updated at a regular cadence.

Target: Unknown until we reach maturity level 2

URL(s)

• https://gitlab.com/edjdev
• https://gitlab.com/clefelhocz1
• https://gitlab.com/glopezfernandez
• https://gitlab.com/meks
• https://gitlab.com/tcooney
• https://gitlab.com/clenneville

Health: Unknown

• Unknown. But my sense is we are not doing enough. For instance, we have not been able to fully update the handbook after the development department re-org (dev backend, and ops backend are still present. Although many of the new teams do have their own pages already)
• We’d like to query the API for merge requests that have files matching /source/handbook/engineering/** over time
• Also facet by Department
• Visualize in Periscope

Maturity: Level 2 of 3

• Needs periscope data

Regular Performance Indicators

Engineering Pulse Survey

A pulse survey is a brief, frequently sent survey that provides near-real-time information about satisfaction in time series. It usually consists of just 1-2 NPS-style questions like “How likely are you to recommend GitLab to your friends as a place to work?” , with an optional comment box for qualitative answers. This is unlike an annual, or bi-annual engagement survey that has much more content, but also has lower submission rates, and can only be sent infrequently.

Target: Unkown, until we send the first one

Health: Unknown

• We probably need our new PO business partner to start
• Need faceting by team/department/function while preserving anonymity
• We need to choose the right tool. Qualtrics, 6Q, CultureAmp, HazelHQ (Slack integration)
• The right tool needs to have an API, or otherwise be able to get data into Snowplow

Maturity: Level 2 of 3

• Needs periscope data

Diversity

Diversity & Inclusion is one of our core values, and a general challenge for the tech industry. GitLab is in a privileged position to positively impact diversity in tech because our remote lifestyle should be more friendly to people who may have left the tech industry, or studied a technical field but never entered industry. This means we can add to the diversity of our industry, and not just play a zero-sum recruiting game with our competitors.

Target: It's against company policy to set diversity quotas (but we may add benchmarks to compare ourselves against).

URL(s)

• https://docs.google.com/presentation/d/1Rj6brdP2hKU1zgj0ll1poywBw0aR9jIAdT4GsLZfIIU/edit#slide=id.g4bb9f44e0b_6_160
• https://docs.google.com/spreadsheets/d/1Jef9oopd8AeZRYM2P9B8ye2TxfdzxI0FY6ixhlyzW_c/edit#gid=1035217495

Health: Attention

• Engineering is now at the tech benchmark for gender diversity (~16%), but our potential is greater and we can do better. 20% should be our floor in technical roles. Other types of diversity are unknown.
• Get data wired up and visualized in periscope.
• Clear data sharing with legal.
• Get better data about racial and country diversity.

Maturity: Level 2 of 3

• Needs periscope data

Team Member Retention

People are a priority and attrition comes at a great human cost to the individual and team. Additionally, recruiting (backfilling attrition) is a ludicrously expensive process, so we prefer to keep the people we have :)

Target: 90%

URL(s)

• https://docs.google.com/spreadsheets/d/1Jef9oopd8AeZRYM2P9B8ye2TxfdzxI0FY6ixhlyzW_c/edit#gid=1405344200

Health: Okay

• I seem to recall our attrition is now below 10% which is great compared to the tech benchmark of 22% and the remote benchmark for 16%, but the fact that I can’t just look at a simple graph makes me nervous...
• Get into a system (BambooHR?) and visualized in periscope
• Break out voluntary and involuntary

Maturity: Level 2 of 3

• Needs periscope data

Other PI Pages

• Performance Indicator Maturity Table
• Engineering Function Performance Indicators
  • Development Department Performance Indicators
    • Growth Section Performance Indicators
  • Infrastructure Department Performance Indicators
  • Quality Department Performance Indicators
  • Security Department Performance Indicators
  • Support Department Performance Indicators
  • UX Department Performance Indicators
• Data Team Metrics Overview

Legends

Maturity

Level	Meaning
Level 3 of 3	Has a description, target, and periscope data.
Level 2 of 3	Missing one of: description, target, or periscope data.
Level 1 of 3	Missing two of: description, target, or periscope data.
Level 0 of 3	Missing a description, a target, and periscope data.

Health

Level	Meaning
Okay	The KPI is at an acceptable level compared to the threshold
Attention	This is a blip, or we’re going to watch it, or we just need to enact a proven intervention
Problem	We'll prioritize our efforts here
Unknown	Unknown

How to work with pages like this

Data

The heart of pages like this is a data file called /data/performance_indicators.yml which is in YAML format. Almost everything you need to do will involve edits to this file. Here are some tips:

• Avoid : in strings as it's an important character in YAML and will confuse the data parsing process. Put the string in "quotes" if you really need to use a :
• urls: should be an array (indented lines starting with dashes) even if you only have one url
• Both maturity.level and health.level take a number between 0 and 3 (inclusive)

Pages

Pages like /handbook/engineering/performance-indicators/ are rendered by and ERB template.

These ERB templates call the helper function performance_indicators() that is defined in /helpers/custom_helpers.rb. This helper function calls in several partial templates to do it's work.

This function takes a required argument named org in string format that limits the scope of the page to a portion of the data file. Possible valid values for this org argument are listed in the org property of each element in the array in /data/performance_indicators.yml.