Product usage playbooks with usage metrics

Maintained by:

GitLab collects product usage metrics via service ping from self-managed customers. For each of the GitLab use-cases, we have at least a single usage metric reported in the service ping. These metrics are used to trigger Product Usage playbooks in Gainsight to prompt CSMs to reach out to customers in a consultative context to help the customer adopt GitLab features as use-case solutions and maximize their return on investment with GitLab. Below is a list of the metrics used to trigger each of the current product usage playbooks. Furthermore, there is information defining the triggers for each playbook and discovery questions that CSMs can use to guide the consultation process with the customer.

CI/CD Use Case

Metric	Triggers	Definition	Discovery
`ci_runners`	Less than one pipeline, one month after onboarding	Total configured runners on GitLab instance	This number staying at zero implies that the customer does not have any GitLab runners configured hence not using GitLab CI. It is important to understand the customer's reasoning behind this; do they not use CI/CD in their workflow all together or does GitLab not meet their needs?
`ci_internal_pipelines`	Drops by at least 20% month on month or remains below 50% of `projects` on GitLab instance three months after onboarding	Total pipelines in GitLab repositories	A drop in this number shows a lessening use of CI pipelines. That in itself might not be necessarily a bad thing as this could also happen as a result of projects consolidating to run fewer efficient pipelines instead. Understanding the reason behind the change can help guide the CSM's next steps.
`ci_external_pipelines`	Higher than `ci_internal_pipelines` month on month for three months	Total pipelines in repositories hosted outside of GitLab	Growth in this metric is suggestive of GitLab CI being used for an increasing number of projects outside of GitLab. This is likely during a migration between different SCM tools. To maximize the customers return on investment in GitLab CSM's can use these numbers to understand how a migration is progressing and well as ensuring that GitLab's source code management is meeting the customer's needs.
`projects_jenkins_active`	Higher than 20% of projects on GitLab instance after onboarding or growth month on month	Count of projects with active integrations for Jenkins	Ideally this metric should trend down as CSMs help onboard customers onto GitLab's single DevSecOps platform. If the metric is trending up instead, then it is important to understand the customer's motivation and liaise with product to ensure that the customer can benefit from GitLab CI/CD.
`projects_teamcity_active`	Higher than 20% of projects on GitLab instance after onboarding or growth month on month	Count of projects with active integrations for Teamcity CI	Ideally this metric should trend down as CSMs help onboard customers onto GitLab's single DevSecOps platform. If the metric is trending up instead, then it is important to understand the customer's motivation and liaise with product to ensure that the customer can benefit from GitLab CI/CD.
`projects_bamboo_active`	Higher than 20% of projects on GitLab instance after onboarding or growth month on month	Count of projects with active integrations for Bamboo CI	Ideally this metric should trend down as CSMs help onboard customers onto GitLab's single DevSecOps platform. If the metric is trending up instead, then it is important to understand the customer's motivation and liaise with product to ensure that the customer can benefit from GitLab CI/CD.

User Engagement Use Case

Metric	Triggers	Definition	Discovery
`billable_user_count`/`license_user_count`	Drops by at least 10% month on month or remains below 50% a month after onboarding	Ratio of active users versus licensed users	A ratio less than one implies that the customer is not fully utilizing their license which is usually the case during onboarding or a consolidation of different GitLab instances into one. If this number drops month on month it is important to understand the reasons behind the drop from the customer and ensure that GitLab is meeting their needs.

Secure Use Case

Metric	Triggers	Definition	Discovery
`container_scanning_job`	Less than 50 jobs, one month after onboarding	Count of Container Scanning jobs run	This number staying at zero implies that the customer(Ultimate tier) does not have any containers which need to be scanned. It is important to understand the customer's reasoning behind this; Are building and deploying containers part of their workflow? Were errors found when including the container scanning template into their CI file? Conduct additional research to understand what is the cause of low adoption.
`dast_jobs`	Less than 50 jobs, three months after onboarding	Count of DAST jobs run	This number remaining below 5 implies that the customer(Ultimate tier) does not have many web applications which need to be scanned. Confirm if the customer understands how the DAST scanning capability works. Does the customer provision Review App environments? How does the customer currently uncover vulnerabilities that occur in their app environments? Conduct research to understand what is the cause of low adoption.
`dependency_scanning_jobs`	Less than one job, one month after onboarding	Count of Dependency Scanning jobs run	This number remaining below 5 implies that the customer(Ultimate tier) does not have many projects where dependency scanning is necessary. It is important to understand the customer's reasoning behind this; Do key projects rely on dependencies? Were errors found when including the dependency scanning template into their CI file? Conduct research to understand what is the cause of low adoption.
`secret_detection_scans`	Less than 50 jobs, one month after onboarding	Number of Secret Detection security scans run	This number remaining below 50 implies that the customer(Ultimate tier) does not have many projects which need to be scanned for leaked secrets. It is important to understand the customer's reasoning behind this; Has there been any attempt to enable secret detection? Are there any applications that require sensitive credentials in order to function? Conduct research to understand what is the cause of low adoption.