The Acquisition, Expansion, Conversion and Retention groups took part in a Fast Boot in September 2019. The planning issue contains the proposal for Fast Boot, and this page documents our discussions and outcomes.
Monthly Product/Growth metrics review with E-Group (VP Product leads, with support from Growth Director)
Every 6 weeks Growth Group Conversation (PM + Eng Director lead)
Weekly engineering meeting
Weekly ideation & prioritization meeting
On Monday, check in on experiments to identify which can be concluded, and to collect data for review meeting on Tuesdays
Group Level
Groups will organize/meet as needed
How do we run Growth experiments?
Development board labels/process - continue on & follow handbook
A/B testing process, artifacts, expectations
Each experiment will set confidence level - 95%-99%
Test summary written by data analyst in the issue (add a “a/b test complete” label for searching later)
Expectations for working in another teams’ code base
Emphasize that the point is to gather data, not ship production code
Challenge: review process too slow. Reviewers vs Maintainers. Only a small number of maintainers.
Goal: one deliverable picked up per week
Feature Ownership
Fulfillment owns underlying billing, licensing, and transactional system, as well as current portal user experience.
Acquisition owns the New Signup ARR growth KPI + the new signup UX
Conversion owns the New Free to Paid Trial ARR growth KPI + the trial UX
Expansion owns the Current customer seat expansion + current customer upgrade ARR growth KPI + the upgrade flow & true-up flow
Retention owns the Gross Retention KPI + renewal flow
Growth PM’s will own issues for key flows; engineering leaders will help assign work
Long Term: the vision is for the fulfillment team to own the billing and licensing platform, with a set of API’s and SLA’s, so the growth teams can build and curate user experiences on top of them to drive great customer experiences (and resulting KPI’s).
Deep Dives
Teardown/journey map existing flow. Leverage demos, videos, screenshots. Ask “experts” to join remotely to demo and share their pain points/opportunities. Session to be led by the product manager.
Acquisition Overview
Discussion led by Jensen, PM for Acquisition, topics covered include:
During this time we walked through Telemetry as a team and reviewed where we were headed. The main item we covered was the Vision Page in the handbook and how we need to clearly communicate what telemetry is and where to find more information.
Topics covered (which will also be added to the handbook)
Vision
SMAU – what it is and how it's managed
Dashboards – what they are, self-serve and how to request one
Data fields available for tracking and what to do if you need a new one
Tracking and instrumentation – how to's for GitLab.com and Self-hosted
Key outcome is to work with the Release team to improve our existing Feature Flags capability in GitLab in order to successfully A/B test Growth experiments.
Right now in order to run an experiment, we need to do additional work to ensure we can measure the results. This work will be bespoke for each experiment while we improve our capability.
We can investigate third party solutions if needed.
Days 3-5
At this time the individual Growth teams split into their groups to create and work on issues in their respective areas.
