| KPI | Maturity | Health | Reason(s) |
|---|---|---|---|
| Security Hiring Actual vs Plan | Level 2 of 3 | Okay |
|
| Security Non-Headcount Plan vs Actuals | Level 2 of 3 | Unknown |
|
| Security Average Location Factor | Level 2 of 3 | Attention |
|
| Security Recruiting Average Top-of-Funnel Location Factor | Level 2 of 3 | Unknown |
|
| Security Handbook Update Frequency | Level 3 of 3 | Unknown |
|
| MTTM (Mean-Time-To-Mitigation) for S1-S2-S3 security vulnerabilities | Level 1 of 3 | Okay |
|
| Blocked Abuse Activity | Level 1 of 3 | Attention |
|
| HackerOne budget vs actual (with forecast of 'unknowns') | Level 1 of 3 | Attention |
|
Are we able to hire high quality workers to build our product vision in a timely manner? Hiring information comes from BambooHR where employees are in the division `Engineering` and department is `security`. This KPI is tracked and reported on a monthly basis but year to date (headcount v plan) is also measured.
URL(s)
Chart (Sisense↗)
Health: Okay
Maturity: Level 2 of 3
This is a subset of an existing KPI. Please see the definition for the parent KPI.
We need to spend our investors' money wisely. We also need to run a responsible business to be successful, and to one day go on the public market.
Target: Unknown until FY21 planning process
URL(s)
Health: Unknown
Maturity: Level 2 of 3
We remain efficient financially if we are hiring globally, working asynchronously, and hiring great people in low-cost regions where we pay market rates. We track an average location factor by function and department so managers can make tradeoffs and hire in an expensive region when they really need specific talent unavailable elsewhere, and offset it with great people who happen to be in low cost areas. Data comes from BambooHR and is the average location factor of all team members in the Security department.
URL(s)
Chart (Sisense↗)
Health: Attention
Maturity: Level 2 of 3
We need to be proactive in measuring our location factor, starting with candidates who are at the top of the recruiting funnel.
Target: 0.58
URL(s)
Health: Unknown
Maturity: Level 2 of 3
This is a subset of an existing KPI. Please see the definition for the parent KPI.
The handbook is essential to working remote successfully, to keeping up our transparency, and to recruiting successfully. Our processes are constantly evolving and we need a way to make sure the handbook is being updated at a regular cadence. This data is retrieved by querying the API with a python script for merge requests that have files matching `/source/handbook/engineering/security` over time.
Target: 100
Chart (Sisense↗)
Health: Unknown
Maturity: Level 3 of 3
The MTTM metric is an indicator of our efficiency in mitigating security vulnerabilities, whether they are reported through HackerOne bug bounty program (or other external means, such as security@gitlab.com emails) or internally-reported. The average days to close issues in the GitLab CE project (project_id = '278964') that are have the label `security` and S1, S2, or S3; this excludes issues with variation of the security label (e.g. `security review`) and S4 issues. Issues that are not yet closed are excluded from this analysis. This means historical data can change as older issues are closed and are introduced to the analysis. The average age in days threshold is set at the daily level.
URL(s)
Health: Okay
Maturity: Level 1 of 3
This metric is focused around the efficacy of our Abuse Handling and Abuse Operations. The Abuse Team is responsible for tracking, automating, and reporting on blocked abusive accounts on GitLab.com, and tracking reduction in cloud spend where the activities are due to abuse.
URL(s)
Health: Attention
Maturity: Level 1 of 3
We currently run a public bug bounty program through HackerOne, and this program has been largely successful - we get a lot of hacker engagement, and since the program went public, we have been able to resolve nearly 100 reported security vulnerabilities. The bounty spend is however, a budgeting forecast concern because of the unpredictability factor from month to month.
Health: Attention
Maturity: Level 1 of 3
Discretionary bonuses offer a highly motivating way to reward individual GitLab team members who really shine as they live our values. Our goal is to award discretionary bonuses to 10% of GitLabbers in the Security department every month.
Target: 10%
URL(s)
Health: Unknown
Maturity: Level 2 of 3
The true mark of a successful public bug bounty program is partially tied to hacker outreach and engagement. We’ve already implemented quite a bit of security automation to engage hackers. For example, when findings are submitted by hackers, an automated dynamic message is generated by our automation scripts and they receive a response with expected ETA on a reply (this ETA is dependent on how many findings are in our ‘to be triaged’ bucket at that time, so the ETA changes). This ensures the hacker knows 1) we received the report, 2) a human will be reading the report within a certain ETA, and 3) a human will be replying within an ETA. Our program went public in mid-December 2018, and we are now focusing on outreach and incentive programs to retain the top hackers to our program - about 90%+ of the most impactful reports are currently submitted by our Top 5-10 hackers, so it is imperative we retain those individuals.
Health: Attention
Maturity: Level 1 of 3
Although the Security Department is always working towards minimizing friction to day-to-day company operations, it is never possible to be 100% frictionless when adding an additional step into an existing process. As cumbersome as this may feel (despite our best efforts), this is usually a necessary step to maturing our security posture and meeting customer requirements (especially large enterprise customers). We plan to measure Security response times to new processes such as access request approval issues and security reviews for 3rd party vendor requests to highlight where bottlenecks are to the current process, and use a data-driven approach to iterate on better processes.
Health: Unknown
Maturity: Level 1 of 3
| Level | Meaning |
|---|---|
| Level 3 of 3 | Has a description, target, and Sisense embed (if public) or URL (or not). |
| Level 2 of 3 | Missing one of: description, target, or Sisense embed (if public) or URL (or not). |
| Level 1 of 3 | Missing two of: description, target, or Sisense embed (if public) or URL (or not). |
| Level 0 of 3 | Missing a description, a target, and Sisense embed (if public) or URL (or not). |
| Level | Meaning |
|---|---|
| Okay | The KPI is at an acceptable level compared to the threshold |
| Attention | This is a blip, or we’re going to watch it, or we just need to enact a proven intervention |
| Problem | We'll prioritize our efforts here |
| Unknown | Unknown |
The heart of pages like this is a data file called /data/performance_indicators.yml which is in YAML format. Almost everything you need to do will involve edits to this file. Here are some tips:
: in strings as it's an important character in YAML and will confuse the data parsing process. Put the string in "quotes" if you really need to use a :urls: should be an array (indented lines starting with dashes) even if you only have one urlmaturity.level and health.level take a number between 0 and 3 (inclusive)Two flags to note:
parent can be used when a PI or KPI is a subset of another PI. For example, we might care about Hiring vs Plan at the company level. The child would be the division and department levels, which would have the parent flag.public can be set to false where a PI or KPI meets our not public guidelines.Pages like /handbook/engineering/performance-indicators/ are rendered by and ERB template.
These ERB templates call the helper function performance_indicators() that is defined in /helpers/custom_helpers.rb. This helper function calls in several partial templates to do it's work.
This function takes a required argument named org in string format that limits the scope of the page to a portion of the data file. Possible valid values for this org argument are listed in the org property of each element in the array in /data/performance_indicators.yml.
