To drive use case enablement and expansion with customers, we need to define exactly what it means to adopt a use case at GitLab. These health measures will appear in the Product Usage scorecard section in Gainsight. For more, see the Gainsight Scorecard Attributes and Calculations.
| 2-6 Months | 6-9 Months | > 9 Months | |
|---|---|---|---|
| < 10% | Red | Red | Red |
| 10-50% | Yellow | Red | Red |
| 51-75% | Green | Yellow | Red |
| > 75% | Green | Green | Green |
User Engagement is intended to measure the number of users logging in each month / billable users.
| Red | Yellow | Green | |
|---|---|---|---|
| unique active users - L28D / Billable Users | <50% | ≥50 - <80% | ≥80% |
This looks at all users who actively log in on a 28 day basis divided by the amount of total users that have been deployed on the account.
Limitations:
As the customer progresses through the lifecycle, user engagement is a measure of the "solidity" consumption of licenses. For example, arre the users that have found their way to the platform sticking around and regularly utilizing it?
Why it matters: User Engagement should prove to be a great way to build a more comprehensive view of renewal risk. It'll be less efficacious for customers that have dormant-user-deactivation enabled, as un-engaged users should be being removed from the billable count on an ongoing basis.
How to use it:
Position as a way for the customer to get value out of the seats they've already paid for, and help the account team ensure more predictable renewal outcomes. It becomes riskier the closer we are to renewal.
| Use Case (Stage) | Purpose / Adoption Level | Description |
|---|---|---|
| SCM (Create) | Basic Adoption | Is my customer using the basic toolset? Most customers should adopt these features pretty quickly into their GitLab journey |
| CI (Verify) | Product Stickiness | As part of their continued adoption and customer journey, we want to help our customers adopt CI, as well as helping their central DevOps teams to better understand their organization's adoption of CI. Use these metrics to help determine progress towards adoption |
| DevSecOps (Secure) | Enablement & Expansion | For customers using our security features or who are trialing and wanting to shift left, use these metrics to help identify adoption and track growth |
| CD (Release) | Enablement & Expansion | How much has my customer adopted GitLab for deployments? The next path along the customer journey is the CD use case |
SCM is considered one of the initial land use cases. To that end, we want to ensure the customer is using it appropriately. Adoption timeline: 1 months after license purchase
| Red | Yellow | Green | |
|---|---|---|---|
| Git Operations - Users L28D / Licenses Sold | < 25% | ≥ 25 - < 50% | ≥ 50% |
This looks to all active users who performed any Git operation (read/write/push) / Licenses Sold.
CI is considered both an initial purchase reason or, in the case of SCM, an expansionary use case (one after the initial purchase intent has been solved). Adoption timeline: 1 months after license purchase
| Red | Yellow | Green | |
|---|---|---|---|
| CI Pipelines Utilization % (CI Pipelines - User L28 / Licenses Sold) | < 25% | ≥ 25% - < 50% | ≥ 50% |
| Deployments Per User L28D (Deployments L28D (event) / Licenses Sold) | < 3% | ≥ 3% - < 8% | ≥ 8% |
| User Deployments Utilization % (Deployments - User L28D / Licenses Sold) | < 5% | ≥ 5% - < 12% | ≥ 12% |
These DevSecOps metrics are available for all customers. Adoption timeline: 1 months after license purchase
| Red | Yellow | Green | |
|---|---|---|---|
| Secure Scanner Utilization % (Secure Scanners - Users L28D / Licenses Sold) | ≤ 5% | > 5% - < 20% | ≥ 20% |
| Container Scanning Jobs Utilization % (Container Scanning Jobs - User L28D / Licenses Sold) | ≤ 3% | > 3% - < 10% | ≥ 10% |
| Secret Detection Jobs Utilization % (Secret Detection Jobs - User L28D / Licenses Sold) | ≤ 6% | > 6% - < 20% | ≥ 20% |
CD is considered an expansionary use case (one after the initial purchase intent has been solved). Adoption timeline: 1 months after license purchase
| Red | Yellow | Green | |
|---|---|---|---|
| User Deployments Utilization % (Deployments - User L28D / Licenses Sold) | < 5% | 5-12% | > 12% |
| Deployments Per User L28D (Deployments L28D (event) / Licenses Sold) | < 2 | 2 - 7 | > 7 |
| Successful Deployments % (Successful Deployments - L28D / (Successful Deployments - L28D + Failed Deployments - L28D)) | < 25% | 25% - 80% | > 80% |
licenses sold as the denominator. This means that while we can see the OVERALL value an account is paying for, we will not see if a new or recently upgraded subscription finds value with a small portion of their billable users. E.g., if a customer purchased 500 licenses, but has only deployed 50 licenses (billable users), then the account will show red for user-based health scores
Billable Users was a metric introduced in 14.0. Any customers on an older (self-managed) instance will not have this value and License Utilization will appear as NULL (note: this is a non-issue for SaaS customers)
unique_active_user metric debuted in 15.2 and only exists for self-managed instances. This health score will be NULL for all SaaS customers and any self-managed customer on 15.1 or earlier
"How many use cases has a customer adopted?"
In Gainsight, scorecards track customer adoption of Gitlab use cases.
A green score signifies that a customer is adopting that specific use case. On the Customer Health Dashboard, in the Use Case Adoption Count chart, we count the number of green scores for each customer to visualize the count of customers adopting null, 1,2, 3 and 4 use cases.
Use this chart to understand how many use cases each of your customers have adopted.
Gainsight calculation rules
Gainsight Rules mark boolean fields as true on Company object for accounts with green scores. These boolean fields are named SCM Adoption, CI Adoption, CD Adoption and DevSecOps Adoption.
Once marked, the number of “true” booleans for each account are summed. If an account has a green SCM, CI, CD and DevSecOps, this would be a 4 score. If none of the use cases are green, this would be 0 and if all of the use case scores are N/A, this would be NULL to mean no usage stats have been recorded.
Using the Gainsight Rules Engine, we have created a mechanism for a Call to Action (CTA) to be created every time a Product Health Score drops from Green to Yellow/Red or from Yellow to Red. The CTA is assigned to the Customer Success Manager (CSM) for follow up, and no playbooks will be associated with the CTA. The CSM will have the option to manually create/add tasks to the CTA in order to keep track of actions taken towards correcting the decrease in score.
Through this CTA, the CSM is notified quickly when a customer may be decreasing their usage of an area of the product, so that they can investigate, ask discovery questions, and triage early, in order to help customers adopt and avoid potential churn or contraction down the line.
While there may be some false positives (for example holiday breaks when no one is working), we prefer to be extra cautious when it comes to potential risk, so we ask CSMs to do their due diligence when receiving these CTAs to ensure their customer is not facing new blockers, company changes, etc. that could affect their renewal, and if they are to begin the triage process immediately.
The CSM may also be able to spot trends of where customers may have lagging usage either over time or across their books of business and suggest best practices to their customers to help with expectations and adoption.
This logic applies to the following Scores:
Notes:
License Utilization is calculated on a subscription level. In Gainsight, we roll up stats from all subscriptions under an account and display it at the account level. Billable User Count comes from Operational Metrics for both SaaS and Self-Managed customers (Self-Managed stats are limited to customers in version 14.0 or later). Subscription stats are brought over from Zuora.
If you believe there is inaccurate stats for an account, see how to report bad usage stats below.
There are three main fields we use at the Instance and Namespace level (generally subscription-level, too) for License Utilization stats:
Billable User Count/Licensed Users represented as a percentage.NOTE: these exist on the Product Usage Metrics object. This will be represented per Instance or Namespace.
Gainsight then rolls up these statistics to the account level, and you can see the aggregated stats for your accounts (see below).
On the Customer 360, you can view the following fields under the User Adoption Metrics section:
Please note that there may be situations where fields are blank or don't seem to include utilization stats from all subscriptions. This might occur because:
If the reporting look good (no missing stats), see how to report bad usage stats below.
If you believe you found an inaccuracy with the usage stats, here are several steps to confirm and then report. First, the integration works by passing data from the data warehouse to Salesforce to Gainsight. In Gainsight, the health scorecard is based on subscriptions where we know both the Billable Users and Total Licenses Sold; if we don't have either one then that subscription is excluded.
If you want to report bad usage stats, please refer to this handbook page on Triaging Data Quality.