Data is another way designers and researchers at GitLab can understand user behavior. Analytics can provide valuable input throughout the Product Development Flow. By using data, we can understand and quantify the impact of the iteration that we shipped.
We should not depend entirely on data to make decisions, but it should be an essential input to decision making. To learn more about quantitative data/research, see the Using Data to Find Insights handbook page.
Part of the design process is to have a strong hypothesis to guide our work.
Ideally, the hypothesis will be based on information from user research.
For example:
We believe
storing information about how an incident was resolved, how long the resolution took, and what the outcome was in a way that’s easy
forengineers responsible for incident management
to access will achievea 20% faster resolution time for incidents
.
Here are possible ways one could use data to understand whether a 20% faster resolution time for incidents
was achieved or not:
These data points would be hard to obtain during solution validation but when measured they help connect the dots from research, iteration, to impact.
By observing and measuring, it should spark further questions to help generate more possible iterations in the future.
To generate reports and dashboards, we use a third party tool called Sisense to visualize the data captured.
The data source determines the table names used in Sisense queries. We have three primary data sources that are useful from a product perspective: usage ping, product database, and Snowplow.
Our goal is to analyze product usage. NOT to track individual users. This means on the frontend we respect browser settings of "do not track" and allow opting out of usage ping. In addition to that, the Analytics Instrumentation team is responsible for data pseudonymization so that there no personally identifiable information saved. This video highlights how Snowplow, usage ping, and pseudonymization work together.
Key Data Sources for Product Managers at GitLab elaborates on how each data source is used and queried.
These visualizations will help you understand how the systems work together:
The issues and merge requests below are examples of how we have used data for decisions.
#data
.