To guide users in how to use the customer's product usage reporting within Gainsight, review use case adoption strategies, and understand how the data connects and what to do with data quality concerns.
For a richer explanation of the purpose and intended outcomes, see the Product Usage Reporting Vision page for more information.
For the overall 3-year vision, see Product Usage Reporting Roadmap (internal GitLab document).
For a high level overview (7 minutes), see the Using Product Usage Reporting in Gainsight - Introduction video.
Use the Product Usage Reporting - v2 Dashboard to see the full list of your accounts.
When using the Account C360 page, these topics are most relevant for evaluating usage:
| Report Name | Description | Application |
|---|---|---|
| Summary | Shows Company-wide License Utilization and Total Licenses Sold | Use this to quickly identify the depth of adoption, if the account is Reporting |
| User Adoption Metrics | Contextually-based metrics to graph the adoption percentage of the account (e.g., users who ran a pipeline divided by monthly active users). | Metrics to help the account team understand monthly use case and license usage to assess for expansion or enablement decisions. Toggle through the different reports to see different graphs. |
| Product Usage Trends | Month-over-Month trend analyses for use cases such as SCM, CI, DevSecOps, and License Utilization | Monthly data points for use case-specific metrics. To be used to help the account team learn feature and deployment depth and adoption — use this in conjunction with the Monthly Distilled Metrics. Toggle through the reports to analyze the top metrcs on a per use case basis. |
| Instance and Namespace Details | List of all instances related to the account with Instance Type meta data |
Use Updating Self Managed Instance Type to update the hostname |
There will be several limitations with the MVC deployment. As you come across use cases, please open an issue or request a new metric as needed. Here are several potential use cases for utilizing data:
| Topic | Description | Questions to Address | References |
|---|---|---|---|
| Understand my customer's usage | With usage data, quickly look up accounts to see which instances we are receiving usage data. User story: see all instances and namespaces related to your account (Production and non-Production) and activity. *Note- : Must be tied to active subscriptions. |
- Who is sending their service ping data? • Are they sending Production data? • Is their activity in line with my expectations? • Does the activity agree with what I know about their usage? |
• VIDEO: Using Product Usage Data in Gainsight - Introduction • VIDEO: What is Product Usage Data and How to Find it in Gainsight |
| C360: Instance and Namespace Details | I need to know which GitLab version they're on to help them upgrade or patch their self-managed instance. | • Which version(s) are my customers on? • If multiple instances, how do I know the version for their instance? • What is their namespace? |
C360: Instance and Namespace Product Usage Data Dashboard ( Self Managed Instances - Current GitLab Version Details report) |
| How many licenses has my customer deployed? | Understand my customers' License Utilization (see above) to know how many licenses have been deployed billable_user_count/licensed seats. Example: a customer purchased 200 licenses and deployed 80 after 9 months, 80/200 = 40%. |
• What trends can I see? • How do I understand my customer's License Utilization? |
• VIDEO: How to Find License Utilization and What it Means in Gainsight • C360: User Adoption Metrics • Product Usage Data dashboard • License Utilization Handbook • VIDEO: Using Product Usage Data in Gainsight - Introduction |
| Understand my customer's GitLab adoption | Know the metrics per use case: SCM, CI, CD, and DevSecOps to understand their adoption. Use the Use Case Adoption guide for use case adoption definitions. See the GitLab Adoption Journey for an explanation on the adoption of SCM, CI, CD, and DevSecOps. |
• Which use cases has my customer adopted? • What degree have they adopted? • Which features have they adopted?</li><li>Does feature adoption align to customer purchase intent? |
C360: Scorecard Product Usage Data dashboard |
| How do I understand the health score with product usage reporting? | Use Gainsight for a quick, high level roll-up of the customer's adoption. Identify if the customer is on track per GitLab use case. | • My customer’s Use Case health is red, yellow, or green — how is that calculated? • How is Product Usage Data weighted? • What is "good" health? |
Use Case Adoption methodology Health Score Measure Weightings VIDEO: What is Product Usage Reporting and How to Find it in Gainsight Usage Trends Dashboard |
Remember, this is an MVC — please create an issue to suggest new metrics, different ways to evaluate the customer's journey, or other ideas.
For specifics on use cases and their health methodology, see Use Case Adoption.
A variety of product usage statistics are pushed back from Gainsight to Salesforce. To see a complete list of customer health related fields that are synced back from Gainsight to Salesforce, please review Customer Health within the Using Gainsight Data in SFDC page.
For GitLab and customers, we must know which instance(s) are used by our customers as their Production instance where they develop their production code. This is required to ensure we are accurately helping customers appropriately adopt GitLab (Product Usage Reporting Vision).
As a general rule, each subscription has one production instance or namespace attached to it. Covered below is how the instances are determined, labeled, and the data is applied within GitLab.
A customer’s server lacks identification of the type (production, test, non-production, mirror, etc.). When GitLab receives a customer’s service ping, we do not know if it is used to deploy production code, a test server, or as a mirror.
Problem: this is critical for self-managed customers because a customer may have anywhere from one to ten instances tied to a single subscription and GitLab could be receiving data from one production instance, several, or none based on whether the customer is air-gapped, blocking our IP ports, etc.
For namespaces, this is irrelevant because the customer pays directly for the namespace and any testing can be done in various projects within the namespace. For SaaS, it is generally solved before it becomes a problem.
For self-managed customers, a CSM labels their customers' instances as Production, Staging, or Obsolete. Steps:
Instance Type, select the proper optionNotes:
While the above process works for any account, we do require automation for the smaller accounts that do not have CSM ownership. The automation in Gainsight falls under two methods:
Production based on License Utilization or UMAU utilization of 20% or greater. This is intended to quickly identify new instances while excluding test instances attached to a single subscription that is only used by a few usersNamespaces are automatically labeled within Gainsight, using the Gainsight Rule “Load to Instance Data - Label SaaS Instances as Production”.
New self-managed instances come online all the time. The different types include:
To make sure we're correctly identifying Production vs. other types, use these instructions to see a full list of instances yet to be labeled:
CSM Burn-Down Dashboard* Unknown Instances - CSM Owned reportProduct Usage TrendsNOT Production Instance Usage reportInstance and Namespace Details section, label the instance accordinglyThis process is critical, as a customer can have multiple subscriptions and each subscription can have multiple instances. See the example diagram:
The instance types are synced from Gainsight to Snowflake weekly and updated Sunday nights PST. Gainsight collects the data and uploads it to an S3 bucket using the rule Admin - Drop Instance Type to S3 for Snowflake Pickup and then Snowflake processes and updates tables every Sunday night.
When an account has multiple GitLab instances identified as Production (Instructions on how to Update Self-Managed Instance Type), the Product Usage health measures the most recently updated instance instead of the primary instance, causing scoring inconsistencies.
Note: this is less than 5% of the time because the vast majority of accounts have a single production instance.
Video Instructions to update instance data in Gainsight to include only one instance in Product Usage health measure.
Included in Health Measures to EXCLUDE the instance section. NOTE: Make sure you select “Opt-Out” rather than null, or the system may overwrite your update. Then click UpdateBest Practices:
Because the DevSecOps health measure looks to the account as "Ultimate", this step was added to make sure the correct production instance is scored in the case of multiple subscriptions under a given account.
If a CSM has marked a production instance under a Premium subscription, DevSecOps health will appear as be “NA”. Meaning, even if there are two subscriptions with one Premium and another Ultimate, as long as the CSM marked the Premium one for health scoring, you will no longer see a DevSecOps health score (generally red) on the account.
Gainsight Rules:
NEW: Admin: Update Plan Name on Product Usage Instance Metrics
Plan Name from the Customer Subscription object to the Product Usage Instance Metrics objectSet Score: DevSecOps Adoption Individual Measures
Plan Name on the Product Usage Instance Metrics object instead of the Products Purchased on the Company objectThe Product Stage definitions have been extracted from the Metrics Dictionary. For more information on Stage metrics, please review the dictionary.
Eventually, the metrics list and definitions will be embedded directly in the handbook. As a first iteration, the list of metrics and their definitions are in the Data Mart - Table Definitions spreadsheet.
See our technical documentation for our instance of Gainsight's Adoption Explorer.
See the User & Project Adoption Metrics tab on Product Usage Reporting for Gainsight Definitions for calculated metrics.
Data is integrated from Snowflake to Gainsight on a monthly basis. Over time, this will move to bi-weekly and then weekly.
Data Quality Problem issue template.
If your customer IS sending data but it does not appear in Gainsight, here are the instructions to manually re-map it.
Note: this process may take several days or a week, given schedules and load. Please be patient as this is an entirely manual process until automation resolves these issues.
Because of the data integrations (listed above), there are certain delays from the moment the customer sends data via Service Ping (self-managed) or collected via Snowplow (SaaS) to when it is visible and usable in systems such as Gainsight and Salesforce. This delay is 1-2 days from when the Service Ping is received from the customer.
Data from SaaS customers is collected on Mondays at 00:00 UTC (Sundays at 5:00 pm PST) each week. Once collected it makes its way through the Snowflake lineage and is then ingested by Gainsight on Mondays at 15:00 UTC (Mondays at 8:00 am PST).
Data from Self-Managed customers also comes in on a weekly basis but each instance has its own day and time when the data is collected. Since this process is not standardized the way SaaS collection is, longer delays are expected based on the time the data is collected (between 1 and 2 days).
Below are the various data sources, their definitions, and uses.
When they activate with Cloud Licensing, customers share Subscription Data, which contains basic license usage and instance version information. This data helps to automate activation, provisioning, co-terms and renewals. The sharing of Subscription Data is a standard part of GitLab's subscription agreement. Operational Metrics contains more detailed product usage metrics and is a subset of Service Ping. This data enable us to serve and support our customers through guiding them with use case adoption scores, assisting with best practices, offering guidance, and assisting with upgrade recommendations. See this 7-min video on the data (internal only) for more information. Customers are able to seek an exemption of sharing Operational Metrics, if national security is a risk.
Subscription Data
References:
We utilize Service Ping to derive self-managed customer usage reporting. For more details, see Service Ping FAQs. Any references to "Service Ping" in Gainsight explicitly refers to self-managed product usage data (licenses + feature use).
SaaS customer usage is in Gainsight and collected via the Snowplow collector.
When licenses are automatically generated (either via WebStore or Deal Desk) a Zuora subscription ID is mapped to a license. This mapping enables the data to link in Gainsight to create a complete picture of an account.
If there is ever an issue where that data is not mapped, follow the steps below:
Example Issue where this issue was discovered: https://gitlab.com/gitlab-data/analytics/-/issues/8518
To request a new metric, please open an issue in the Product Analytics project and at-mention Product Analytics PM. You can see Add per project count as an example. However, before you create an issue, please confirm the metric does not already exist in the Event Dictionary.
Examples of new metrics can include:
If the customer is self-managed, check if they have an instance labeled Production. They must have an instance labeled as Production to appear in the Usage Trends report.
The purpose of the Instance and Namespace Details report section is to show:
If a metric was implemented in a later release, it will not appear. Check the Data Mart Table definitions to identify the release for the metric in question. You can check the instance and namespace details to confirm which release version the customer is on. Example: Billable Users was instrumented in 14.0, and if a customer is on 13.9 or earlier, then this field will appear as NULL.
Service Ping is a GitLab process that collects customer analytics on self-managed instances and sends a weekly payload to GitLab. The payload provides important high-level statistics that helps our product, support, and sales teams understand how GitLab is used.
The only way to confirm if they have opted into Service Ping is if we have data for them. See the Service Ping Guide for more information.
Yes.
No, unless they do a contractual exemption (limited to certain PubSec orgs).
Operational metrics.
Usage Statistics are received and collected weekly, and those stats are added to Gainsight every week. Even though the metrics are received and added weekly, the metrics are still shown as monthly, such as Last 28 Days (L28) or All-Time.
Example:
No. Please refer to How do Last 28 Days metrics work?
You may notice usage stats missing for the first week of a month up until a ping is collected from the customer. The ping snapshot date will differ for each customer.
There’s no DevSecOps-specific override, however, there is one option to override which is the CSM Sentiment. The CSM can change that to Red, making the overall account red. HB Reference. CSM Sentiment overall weighting is 25%, whereas Product is 50%
Check the Product Usage Data for Gainsight Definitions.
Check the Product Usage Data for Gainsight Definitions.
UUID is the GitLab-assigned ID of a server. There can be more than one server for one hostname.
Telemetry has the connotation of third-party analytics, so we avoid using this word. See more information and alternatives on the Top Misused Terms Handbook Page.
The following reports are not effective for accounts with multiple hostnames and subscriptions: