| KPI | Health | Status |
|---|---|---|
| MTTR (Mean-Time-To-Remediation) by severity for security vulnerabilities over the past 30 days | Problem |
|
| Security Engineer On-Call Page Volume | Okay |
|
| Security Control Health | Attention |
|
| Security Impact on Net ARR | Okay |
|
| Cost of Abuse | Attention |
|
The MTTR metric is an indicator of our historical efficiency as a company remediating security vulnerabilities, whether they are internally reported, via the HackerOne bug bounty program, or through other external means. Vulnerabilities that are not yet closed are excluded from this metric. For Security purposes, please view this chart directly in Sisense.
Target: Time to remediate
URL(s)
Health: Problem
This metric is focused around the volume and severity of paged incidents to the Security Engineer On-Call. For Security purposes, please view this chart directly in Sisense.
Target: Number of pages/month does not exceed +50% of monthly average of the last 12 months for 3 consecutive months
URL(s)
Health: Okay
The objective of this KPI is to minimize exposure to unexpected security risks through continuous control testing. GCF security controls are conitnuously tested as parts of the Compliance team's continuous monitoring program, internal audits and external audits. A clear indicator of program health is directly reflected in the control health effectveness rating (CHER). Observations are a result of GCF security failure, indicating that the control is not implemented, designed effectively or is not operating effectively. These observations indicate a gap that requires remediation in order for the security control to be operating and audit ready.
Target: 60% of executed control tests with a CHER of 1 (Fully Effective)
Chart (Sisense↗)
Health: Attention
The Field Security organization functions as a sales and customer enablement team therefore a clear indicator of success is directly reflected in the engagement of their services by Legal, Sales, TAMs and customers themselves.
Target: $4M net ARR per month
URL(s)
Chart (Sisense↗)
Health: Okay
This metric is focused around the financial impact of abusive accounts and their activity. For Security purposes, please view this chart directly in Sisense.
Target: Cost of abuse is below $10K/Mo
URL(s)
Health: Attention
This metric groups security incidents by incident category. For Security purposes, please view this chart directly in Sisense.
Target: Number of security incidents in any category does not exceed +50% of the individual category's 3-month average.
URL(s)
Health: Okay
This metric groups security incidents by the impacted organization, service, or part of the business. For Security purposes, please view this chart directly in Sisense.
Target: Number of security incidents in any organization does not exceed +50% of the individual organization's 3-month average.
URL(s)
Health: Okay
The average age of open vulnerabilities gives us a current-state snapshot of how fast we are scheduling and fixing the vulnerabilities found post-Production deploy. This is important because it can help indicate what our future MTTR will look like and whether we are meeting our defined SLOs. The average age is measured in days, and the targets for each severity are defined in the Security Handbook. For Security purposes, please view this chart directly in Sisense.
Target: Time to remediate
URL(s)
Health: Attention
Due to the inherent risk of running 3rd-party code and the current focus on GitLab’s dependency management, this maturity model was created to guide and measure the progress. Over the last 4 quarters work has been done to strengthen the foundation for a world-class dependency management system, reduce the risk associated with software supply chain attacks, and create a market differentiating feature set for GitLab. A lower level must be achieved before the next level up can be achieved, however work and planning preparing for the next level up can occur in parallel. Level 1 - Basic Risk Detection and Manual Processes. Level 2 - Advanced Risk Detection and Manual Processes. Level 3 - Automated Enforcement and Risk Mitigation. For Security purposes, please view this chart with the link provided.
Target: Achieve Level 2 maturity in main GitLab repo by end of FY22Q3
URL(s)
Health: Okay
Operational risk management enables organizations to proactively identify and mitigate operational security risks that may impact Organizational Output, Brand Reputation, Business Continuity, Customers & Stakeholders, Legal & Regulatory and/or Financials.
Target: TBD, this will be determined upon Sisnse integration for detailed dashboarding
Chart (Sisense↗)
Health: Attention
An indicator of information system and process risk, there are multiple inputs that lead to identification of Observations to include Security Compliance continuous control testing, Internal Audit control testing, third party (vendor) assessments, external audits and customer assessments.
Target: TBD, this will be determined upon Sisnse integration for detailed dashboarding
Chart (Sisense↗)
Health: Problem
An indicator of third party risk, third party risk assessments proactively identify potential vendor security risks as part of onboarding or contracting, enabling business owners to make risk based decisions throughout the vendor lifecycle.
Target: TBD, this will be determined upon Sisense integration for detailed dashboarding
Chart (Sisense↗)
Health: Attention
All Security Automation OKRs should reach 80% or higher by the end of any active quarter. This measurement indicates how well the Security Automation team is executing on the highest priorities of the quarter which are aligned with business needs and goals.
Target: 80
Chart (Sisense↗)
Health: Okay
We attempt to complete 7 issues or more every two weeks. The measurement indicates how well the Security Automation team is scoping milestones and provides a view of the team velocity over the last 10 milestones. While our average iteration is 2 weeks in length, in an effor to maintain flexibility, we allow for 1 week and 3 week long iterations as well, thus the data in the chat is measured by 1 week long increments.
Target: 7
Chart (Sisense↗)
Health: Okay
We attempt to complete 7 issues or more every two weeks. The measurement indicates how well the Security Automation team is scoping milestones over the last 10 iterations and provides a view of the average team velocity.
Target: 7
Chart (Sisense↗)
Health: Okay
Discretionary bonuses offer a highly motivating way to reward individual GitLab team members who really shine as they live our values. Our goal is to award discretionary bonuses to 10% of GitLab team members in the Security department every month.
Target: At or above 10%
Chart (Sisense↗)
Health: Attention
This is a subset of an existing KPI. Please see the definition for the parent KPI.
Employees are in the division "Engineering" and department is "Security".
Target: 0.9
URL(s)
Chart (Sisense↗)
Health: Okay
This is a subset of an existing KPI. Please see the definition for the parent KPI.
We need to spend our investors' money wisely. We also need to run a responsible business to be successful, and to one day go on the public market. For Security purposes, please view this chart directly in Sisense.
Target: Unknown until FY22 planning process
URL(s)
Health: Attention
This is a subset of an existing KPI. Please see the definition for the parent KPI.
We remain efficient financially if we are hiring globally, working asynchronously, and hiring great people in low-cost regions where we pay market rates. We track an average location factor for team members hired within the past 3 months so hiring managers can make tradeoffs and hire in an expensive region when they really need specific talent unavailable elsewhere, and offset it with great people who happen to be in more efficient location factor areas with another hire. The historical average location factor represents the average location factor for only new hires in the last three months, excluding internal hires and promotions. The calculation for the three-month rolling average location factor is the location factor of all new hires in the last three months divided by the number of new hires in the last three months for a given hire month. The data source is BambooHR data.
Target: Less than 0.70
Chart (Sisense↗)
Health: Okay
We remain efficient financially if we are hiring globally, working asynchronously, and hiring great people in low-cost regions where we pay market rates. We track an average location factor by function and department so managers can make tradeoffs and hire in an expensive region when they really need specific talent unavailable elsewhere, and offset it with great people who happen to be in low cost areas. Data comes from BambooHR and is the average location factor of all team members in the Security department.
Target: Less than 0.70
URL(s)
Chart (Sisense↗)
Health: Okay
This is a subset of an existing KPI. Please see the definition for the parent KPI.
The handbook is essential to working remote successfully, to keeping up our transparency, and to recruiting successfully. Our processes are constantly evolving and we need a way to make sure the handbook is being updated at a regular cadence. This data is retrieved by querying the API with a python script for merge requests that have files matching `/source/handbook/engineering/security` over time. time.
Target: 1
Chart (Sisense↗)
Health: Okay
This PI is in support of the engineering organization’s overall MR Rate objective however, this should not be considered a reflection of the performance or output of the Security Department whose work is primarily handbook driven. Thus, there is no current target for Security Department MR Rate.
Target: 0
URL(s)
Chart (Sisense↗)
Health: Okay
| Value | Level | Meaning |
|---|---|---|
| 3 | Okay | The KPI is at an acceptable level compared to the threshold |
| 2 | Attention | This is a blip, or we’re going to watch it, or we just need to enact a proven intervention |
| 1 | Problem | We'll prioritize our efforts here |
| 0 | Unknown | Unknown |
Pages, such as the Engineering Function Performance Indicators page are rendered by an ERB template that contains HTML code.
Other PI Pages sectionThese ERB templates calls custom helper functions that extract and transform data from the Performance Indicators data file.
kpi_list_by_org(org) helper function takes a required string argument named org (deparment or division level) that returns all the KPIs (pi.is_key == true) for a specific organization grouping (pi.org == org) from the Performance Indicators data file.pi_maturity_level(performance_indicator) helper function automatically assigns a maturity level based on the availability of certain data properties for a particular PI.pi_maturity_reasons(performance_indicator) helper function returns a reason for a PI maturity based on other data properties.performance_indicators(org) takes a required string argument named org (deparment or division level) that returns two lists - a list of all KPIs and a list of all PIs for a specific organization grouping (department/division).signed_periscope_url(data) takes in the sisense_data property information from Performance Indicators data files and returns a signed chart URL for embedding a Sisense chart into the handbook.The heart of pages like this are Performance Indicators data files which are YAML files. Each - denotes a dictionary of values for a new (K)PI. The current elements (or data properties) are:
| Property | Type | Description |
|---|---|---|
name |
Required | String value of the name of the (K)PI. For Product PIs, product hierarchy should be separate from name by " - " (Ex. {Stage Name}:{Group Name} - {PI Type} - {PI Name} |
base_path |
Required | Relative path to the performance indicator page that this (K)PI should live on |
definition |
Required | refer to Parts of a KPI |
parent |
Optional | should be used when a (K)PI is a subset of another PI. For example, we might care about Hiring vs Plan at the company level. The child would be the division and department levels, which would have the parent flag. |
target |
Required | The target or cap for the (K)PI. Please use Unknown until we reach maturity level 2 if this is not yet defined. For GMAU, the target should be quarterly. |
org |
Required | the organizational grouping (Ex: Engineering Function or Development Department). For Product Sections, ensure you have the word section (Ex : Dev Section) |
section |
Optional | the product section (Ex: dev) as defined in sections.yml |
stage |
Optional | the product stage (Ex: release) as defined in stages.yml |
group |
Optional | the product group (Ex: progressive_delivery) as defined in stages.yml |
category |
Optional | the product group (Ex: feature_flags) as defined in categories.yml |
is_key |
Required | boolean value (true/false) that indicates if it is a (key) performance indicator |
health |
Required | indicates the (K)PI health and reasons as nested attributes. This should be updated monthly before Key Meetings by the DRI. |
health.level |
Optional | indicates a value between 0 and 3 (inclusive) to represent the health of the (K)PI. This should be updated monthly before Key Meetings by the DRI. |
health.reasons |
Optional | indicates the reasons behind the health level. This should be updated monthly before Key Meetings by the DRI. Should be an array (indented lines starting with dashes) even if you only have one reason. |
urls |
Optional | list of urls associated with the (K)PI. Should be an array (indented lines starting with dashes) even if you only have one url |
funnel |
Optional | indicates there is a handbook link for a description of the funnel for this PI. Should be a URL |
sisense_data |
Optional | allows a Sisense dashboard to be embeded as part of the (K)PI using chart, dashboard, and embed as neseted attributes. |
sisense_data.chart |
Optional | indicates the numeric Sisense chart/widget ID. For example: 9090628 |
sisense_data.dashboard |
Optional | indicates the numeric Sisense dashboard ID. For example: 634200 |
sisense_data.shared_dashboard |
Optional | indicates the numeric Sisense shared_dashboard ID. For example: 185b8e19-a99e-4718-9aba-96cc5d3ea88b |
sisense_data.embed |
Optional | indicates the Sisense embed version. For example: v2 |
sisense_data_secondary |
Optional | allows a second Sisense dashboard to be embeded. Same as sisense data |
sisense_data_secondary.chart |
Optional | Same as sisense_data.chart |
sisense_data_secondary.dashboard |
Optional | Same as sisense_data.dashboard |
sisense_data_secondary.shared_dashboard |
Optional | Same as sisense_data.shared_dashboard |
sisense_data_secondary.embed |
Optional | Same as sisense_data.embed |
public |
Optional | boolean flag that can be set to false where a (K)PI does not meet the public guidelines. |
pi_type |
Optional | indicates the Product PI type (Ex: AMAU, GMAU, SMAU, Group PPI) |
product_analytics_type |
Optional | indicates if the metric is available on SaaS, SM (self-managed), or Both. |
is_primary |
Optional | boolean flag that indicates if this is the Primary PI for the Product Group. |
implementation |
Optional | indicates the implementation status and reasons as nested attributes. This should be updated monthly before Key Meetings by the DRI. |
implementation.status |
Optional | indicates the Implementation Status status. This should be updated monthly before Key Meetings by the DRI. |
implementation.reasons |
Optional | indicates the reasons behind the implementation status. This should be updated monthly before Key Meetings by the DRI. Should be an array (indented lines starting with dashes) even if you only have one reason. |
lessons |
Optional | indicates lessons learned from a K(PI) as a nested attribute. This should be updated monthly before Key Meetings by the DRI. |
lessons.learned |
Optional | learned is an attribute that can be nested under lessonsand indicates lessons learned from a K(PI). This should be updated monthly before Key Meetings by the DRI. Should be an array (indented lines starting with dashes) even if you only have one lesson learned |
monthly_focus |
Optional | indicates monthly focus goals from a K(PI) as a nested attribute. This should be updated monthly before Key Meetings by the DRI. |
monthly_focus.goals |
Optional | indicates monthly focus goals from a K(PI). This should be updated monthly before Key Meetings by the DRI. Should be an array (indented lines starting with dashes) even if you only have one goal |
metric_name |
Optional | indicates the name of the metric in Self-Managed implemenation. The SaaS representation of the Self-Managed implementation should use the same name. |
Above ...Below ...At ...At or above ...At or below ...shared_dashboard, chart , and the dashboard key-value pairs to the corresponding Performance Indicators data file under the sisense_data property: in strings as it's an important character in YAML and will confuse the data parsing process. Put the string in "quotes" if you really need to use a :