Gitlab hero border pattern left svg Gitlab hero border pattern right svg

UX Department Performance Indicators

Executive Summary

KPI Maturity Health Reason(s)
UX Hiring Actual vs Plan Level 2 of 3 Okay
  • UX hiring is on plan, but we just put in place a new "two-star minimum" rule that might decrease offer volume.
  • Health: Monitor health closely
UX Non-Headcount Plan vs Actuals Level 2 of 3 Okay
  • Chart budget vs. actual over time available in periscope with clear variance
UX Average Location Factor Level 2 of 3 Attention
  • Our average location factor trended upward to .68 due to hiring for more complex stage groups.
  • We are actively working with recruiting to target the right areas of the globe on a per role basis, and we expect to see this average go down during FY21.
UX Recruiting Average Top-of-Funnel Location Factor Level 2 of 3 Unknown
  • We need to get this as a chart in periscope.
  • We have set the target location factors on our vacancies and make sure recruiting is targeting the right areas of the globe on a per role basis.
UX Handbook Update Frequency Level 3 of 3 Unknown
  • Unknown.
Perception of system usability Level 3 of 3 Attention
  • Perceived usability rates as a B- to a low B.
  • Continue to action on UX Scorecards for every stage group.
Ratio of proactive vs reactive UX work Level 3 of 3 Okay
  • In Q4 FY21, we had 28 product Designers on staff and 58 total validation issues. This is two higher than the target of 56.

Key Performance Indicators

UX Hiring Actual vs Plan

Are we able to hire high quality workers to build our product vision in a timely manner? Hiring information comes from BambooHR where employees are in the division `Engineering` and department is `UX`. This KPI is tracked and reported on a monthly basis but year to date (headcount v plan) is also measured.

Target: 67 by February 1, 2021

URL(s)

Health: Okay

Maturity: Level 2 of 3

UX Non-Headcount Plan vs Actuals

This is a subset of an existing KPI. Please see the definition for the parent KPI.

We need to spend our investors' money wisely. We also need to run a responsible business to be successful, and to one day go on the public market.

Target: Unknown until FY21 planning process

URL(s)

Health: Okay

Maturity: Level 2 of 3

UX Average Location Factor

We remain efficient financially if we are hiring globally, working asynchronously, and hiring great people in low-cost regions where we pay market rates. We track an average location factor by function and department so managers can make tradeoffs and hire in an expensive region when they really need specific talent unavailable elsewhere, and offset it with great people who happen to be in low cost areas.

URL(s)

Chart (Sisense↗)

Health: Attention

Maturity: Level 2 of 3

UX Recruiting Average Top-of-Funnel Location Factor

We need to be proactive in measuring our location factor, starting with candidates who are at the top of the recruiting funnel.

Target: 0.58

URL(s)

Health: Unknown

Maturity: Level 2 of 3

UX Handbook Update Frequency

This is a subset of an existing KPI. Please see the definition for the parent KPI.

The handbook is essential to working remote successfully, to keeping up our transparency, and to recruiting successfully. Our processes are constantly evolving and we need a way to make sure the handbook is being updated at a regular cadence. This data is retrieved by querying the API with a python script for merge requests that have files matching `/source/handbook/engineering/**` or `/source/handbook/support/**` over time.

Target: 50

Chart (Sisense↗)

Health: Unknown

Maturity: Level 3 of 3

Perception of system usability

The System Usability Scale (SUS) is an industry-standard survey that measures overall system usability based on 10 questions. Moving a SUS score upward even a couple of points on a large system is a significant change. The goal of this KPI is to understand how usability of the GitLab product rates against industry standards and then track trends over time. Even though UX will be responsible for this metric, they will need other departments such as PM and Development to positively affect change. See Table 1 for grading details.

Target: B+

URL(s)

Chart (Sisense↗)

Health: Attention

Maturity: Level 3 of 3

Ratio of proactive vs reactive UX work

Use customer research to validate solutions. This includes feedback from the wider GitLab community gathered via multiple comments, a survey, usability study, card sort/tree test, and/or user interviews. Hypothesis that there is a connection between this KPI and SUS KPI.

Target: 2 validation issues per Product Designer per quarter

Chart (Sisense↗)

Health: Okay

Maturity: Level 3 of 3

Regular Performance Indicators

UX Discretionary Bonus Rate

Discretionary bonuses offer a highly motivating way to reward individual GitLab team members who really shine as they live our values. Our goal is to award discretionary bonuses to 10% of GitLabbers in the UX department every month.

Target: 10%

URL(s)

Health: Unknown

Maturity: Level 2 of 3

UX debt

UX Debt means that for a given issue, we failed to meet defined standards for our Design system or for usability and feature viability standards as defined in agreed-upon design assets. When we fail to ship something according to defined standards, we track the resulting issues with a "UX debt" label. Even though UX will be responsible for this metric, they will need other departments such as PM and Development to positively affect change.

Target: Under 50 open "ux debt" issues

Chart (Sisense↗)

Health: Attention

Maturity: Level 3 of 3

Throughput of Technical Writing team documentation MRs

This KPI tracks the number of ~documentation MRs merged every month across all GitLab projects which have involvement (review, collaboration, or authoring) from the ~"Technical Writing" team. The goal is to increase velocity over time as the team grows.

Target: 25 MRs per technical writer per month

Chart (Sisense↗)

Health: Okay

Maturity: Level 3 of 3

Distribution of Technical Writing team documentation effort

Tracks the type of documentation changes involving the ~"Technical Writing" team, based on `docs::` scoped label applied. Labels include feature, new, improvement, fix, revamp, or housekeeping. Our goal is to increase the proportion of efforts where Technical Writers proactively improve content, instead of just responding to a new feature or fixing a bug (improvement, new, and revamp).

Target: 50% of MRs are focused on proactively making our docs better ([improvement, revamp, new](https://gitlab.com/gitlab-org/gitlab/-/labels?utf8=%E2%9C%93&subscribed=&search=docs%3A%3A))

Chart (Sisense↗)

Health: Attention

Maturity: Level 3 of 3

Other PI Pages

Legends

Maturity

Level Meaning
Level 3 of 3 Has a description, target, and Sisense embed (if public) or URL (or not).
Level 2 of 3 Missing one of: description, target, or Sisense embed (if public) or URL (or not).
Level 1 of 3 Missing two of: description, target, or Sisense embed (if public) or URL (or not).
Level 0 of 3 Missing a description, a target, and Sisense embed (if public) or URL (or not).

Health

Level Meaning
Okay The KPI is at an acceptable level compared to the threshold
Attention This is a blip, or we’re going to watch it, or we just need to enact a proven intervention
Problem We'll prioritize our efforts here
Unknown Unknown

How to work with pages like this

Data

The heart of pages like this is a data file called /data/performance_indicators.yml which is in YAML format. Almost everything you need to do will involve edits to this file. Here are some tips:

Two flags to note:

Pages

Pages like /handbook/engineering/performance-indicators/ are rendered by and ERB template.

These ERB templates call the helper function performance_indicators() that is defined in /helpers/custom_helpers.rb. This helper function calls in several partial templates to do it's work.

This function takes a required argument named org in string format that limits the scope of the page to a portion of the data file. Possible valid values for this org argument are listed in the org property of each element in the array in /data/performance_indicators.yml.