Develop a respectful and privacy focussed data collection framework that allows us to make informed product decisions which improve the way our customers sign up to GitLab, manage and renew their licenses, and upgrade their accounts to higher tiers.
The Growth sub-department consists of groups that eliminate barriers between our users and our product value.
Managing customer licensing and transactions
The Fulfillment Group is responsible for:
Licensing
Transactions
Deliver telemetry data that improves our product
The Telemetry Group is responsible for:
Collection
Analysis
Drive value for the business and our users by improving activation, retention, upsell, and per-stage adoption
We focus on validating ideas with data across the following four Groups:
We work closely with the Data Team along with our Product Team counterparts to design and implement experiments that measure the impact of changes to our messaging, UX, and overall experience of using GitLab.
The Growth sub-department uses the ~"devops::growth" label and the following groups for tracking throughput and ownership of issues and merge requests.
| Group name | Group label |
|---|---|
| Fulfillment | ~"group::fulfillment" |
| Telemetry | ~"group::telemetry" |
| Acquisition | ~"group::acquisition" |
| Conversion | ~"group::conversion" |
| Expansion | ~"group::expansion" |
| Retention | ~"group::retention" |
The following people are permanent members of the Growth sub-department:
| Person | Role |
|---|---|
| Bartek Marnane | Director of Engineering, Growth |
| James Lopez | Backend Engineering Manager, Fulfillment |
| Chris Baus | Engineering Manager, Fulfillment |
| Jerome Ng | Backend Engineering Manager, Growth:Acquisition and Conversion and Telemetry |
| Phil Calder | Backend Engineering Manager, Growth:Expansion and Retention |
The following people are permanent members of groups that belong to the Growth sub-department:
| Person | Role |
|---|---|
| Jerome Ng | Backend Engineering Manager, Growth:Acquisition and Conversion and Telemetry |
| Alex Buijs | Senior Fullstack Engineer, Growth:Acquisition |
| Nicolas Dular | Senior Fullstack Engineer, Growth:Acquisition |
| Person | Role |
|---|---|
| Jerome Ng | Backend Engineering Manager, Growth:Acquisition and Conversion and Telemetry |
| Alper Akgun | Senior Fullstack Engineer, Growth:Conversion |
| Dallas Reedy | Fullstack Engineer, Growth:Conversion |
| Person | Role |
|---|---|
| Phil Calder | Backend Engineering Manager, Growth:Expansion and Retention |
| Jackie Fraser | Fullstack Engineer, Growth:Expansion |
| Doug Stull | Senior Fullstack Engineer, Growth:Expansion |
| Person | Role |
|---|---|
| Phil Calder | Backend Engineering Manager, Growth:Expansion and Retention |
| Jeremy Jackson | Senior Fullstack Engineer, Growth:Retention |
| Jay Swain | Senior Fullstack Engineer, Growth:Retention |
| Person | Role |
|---|---|
| Rubén Dávila | Backend Engineer, Fulfillment |
| James Lopez | Backend Engineering Manager, Fulfillment |
| Mark Chao | Backend Engineer, Fulfillment |
| Amparo Luna | Backend Engineer, Fulfillment |
| Corinna Wiesner | Backend Engineer, Fulfillment |
| Tyler Amos | Senior Backend Engineer, Fulfillment |
| Vladlena Shumilo | Backend Engineer, Fulfillment |
| Person | Role |
|---|---|
| Vitaly Slobodin | Senior Frontend Engineer, Fulfillment |
| Chris Baus | Engineering Manager, Fulfillment |
| Ragnar Hardarson | Senior Frontend Engineer, Fulfillment |
| Ammar Alakkad | Frontend Engineer, Fulfillment |
| Person | Role |
|---|---|
| Kirstie Cook | Backend Engineer, Fulfillment |
| Chris Baus | Engineering Manager, Fulfillment |
| Shreyas Agarwal | Senior Backend Engineer, Fulfillment |
| Person | Role |
|---|---|
| Alishan 'Ali' Ladhani | Backend Engineer, Telemetry |
| Jerome Ng | Backend Engineering Manager, Growth:Acquisition and Conversion and Telemetry |
| Alina Mihaila | Backend Engineer, Telemetry |
| Person | Role |
|---|
The following table shows who will provide cover if one or more of the Growth Engineering management team are unable to work for any reason.
| Team Member | Coverered by | Escalation |
|---|---|---|
| Bartek Marnane | Christopher Lefelhocz | Eric Johnson |
| Chris Baus | James Lopez | Bartek Marnane |
| James Lopez | Chris Baus | Bartek Marnane |
| Jerome Ng | Phil Calder | Bartek Marnane |
| Phil Calder | Jerome Ng | Bartek Marnane |
If an Engineer is unavailable the Engineering Manager will reassign open issues and merge requests to another engineer, preferably in the same group.
Some people management functions may require escalation or delegation, such as BambooHR and Expensify.
This can be used as the basis for a BCP Communication Plan and Role Assignments, as well as a general guide to Growth Engineering continuity in the event of one or team members being unavailable for any reason.
The following members of other functional teams are our stable counterparts:
As part of the wider Growth stage we track and work on issues with the label ~"devops::growth".
Our team follows the Product Development Flow utilizing all labels from ~workflow::start to ~workflow::verification.
We adhere to the Completion Criteria and Who Transitions Out outlined in the Product Development Flow to progress issues from one stage to the next.
We use workflow boards to track issue progress throughout a milestone. Workflow boards should be viewed at the highest group level for visibility into all nested projects in a group.
There are three GitLab groups we use:
| gitlab-org | gitlab-com | gitlab-services | all groups |
|---|---|---|---|
| Growth Workflow | - | Growth Workflow | - |
| Acquisition Workflow | - | Acquisition Workflow | - |
| Conversion Workflow | - | Conversion Workflow | - |
| Expansion Workflow | - | Expansion Workflow | - |
| Retention Workflow | - | Retention Workflow | - |
| Fulfillment Workflow | - | - | - |
| Telemetry Workflow | - | Telemetry Workflow | - |
Before work can begin on an issue, we should estimate it first after a preliminary investigation. This is normally done in the monthly planning meeting.
| Weight | Description (Engineering) |
|---|---|
| 1 | The simplest possible change. We are confident there will be no side effects. |
| 2 | A simple change (minimal code changes), where we understand all of the requirements. |
| 3 | A simple change, but the code footprint is bigger (e.g. lots of different files, or tests effected). The requirements are clear. |
| 5 | A more complex change that will impact multiple areas of the codebase, there may also be some refactoring involved. Requirements are understood but you feel there are likely to be some gaps along the way. |
| 8 | A complex change, that will involve much of the codebase or will require lots of input from others to determine the requirements. |
| 13 | A significant change that may have dependencies (other teams or third-parties) and we likely still don't understand all of the requirements. It's unlikely we would commit to this in a milestone, and the preference would be to further clarify requirements and/or break in to smaller Issues. |
In planning and estimation, we value velocity over predictability. The main goal of our planning and estimation is to focus on the MVC, uncover blind spots, and help us achieve a baseline level of predictability without over optimizing. We aim for 70% predictability instead of 90%. We believe that optimizing for velocity (MR throughput) enables our Growth teams to achieve a weekly experimentation cadence.
Our work is planned and delivered on a monthly cycle using milestones. Our team follows the Product Development Timeline utilizing all dates including from M-1, 4th: Draft of the issues to M+1, 4th: Public Retrospective.
We use milestone boards for high level planning and roadmapping across several milestones.
The Growth UX team has a handbook page which includes Growth specific workflows.
To help our team be efficient, we explicitly define how our team uses issues.
We aim to create issues in the same project as where the future merge request will live. For example, if an experiment is being run in the Customers application, both the issue and MR should be created in the Customers project.
We emphasize creating the issue in the right project to avoid having to close and move issues later in the development process. If the location of the future merge request cannot be determined, we will create the issue in our catch-all growth team-tasks project.
To support Iteration Growth engineering:
5 and higher should be reassigned to the Product Manager to make sure they can be split into smaller MVCs.
When this is not possible, the Product Manager will create a spike or research issue so that engineering can break it down and close the original.In an effort to prevent accidental outages to business critical processes, whenever we touch code which may have a direct or indirect impact on data flowing into Salesforce.com, we should assign @jbrennan1 to the MR as a reviewer. Note, a group is being created to replace the single point of failure.
We follow a four step process for running experiments as outlined by Andrew Chen's How to build a growth team.
Each week, we provide progress updates and talk about our learnings in our Growth Weekly Meeting.
The duration of each experiment will vary depending on how long it takes for experiment results to reach statistical significance. Due to the varying duration, there will be some weeks where we have several experiments running concurrently in parallel.
Experiments are tracked on Growth - Experiments boards by group:
| gitlab-org | gitlab-com | gitlab-services | all groups |
|---|---|---|---|
| Growth Experiments | Growth Experiments | Growth Experiments | - |
All growth experiments consist of two issues:
~"growth experiment" label, and a scoped experiment:: label.For tracking the status of experiments we also use the following scoped experiment:: labels:
~"experiment::pending"~"experiment::active"~"experiment::blocked"~"experiment::validated"~"experiment::invalidated"~"experiment::inconclusive"In order to deploy an experiment, experiments need to be tagged with a milestone and aligned to GitLab's release schedule.
Release schedules vary depending on where an experiment is being conducted.
Experiments are generally excluded from monthly release posts as they are behind feature flags and usually made available on GitLab.com. An experiment feature can be added to the release post only after it has been implemented for the long run.
Every week, engineers in the Growth sub-department meet to discuss topics related to growth engineering. Discussion topics include how to track experiments, A/B testing, changes in customers application, changes in gitlab application, etc. Growth Engineers are encouraged to bring discussion topics to the meeting and to them to the agenda.
To get the most time zone coverage, these meetings alternate fortnightly between:
Team members are encouraged to attend the meeting that matches their time zone.
The Acquisition, Expansion, Conversion and Retention groups took part in a Fast Boot in September 2019. The planning issue contains the proposal for Fast Boot, and outcomes are available in the Growth Fast Boot Page.
#s_growth in Slack (GitLab internal)#g_fulfillment in Slack (GitLab internal)#g_telemetry in Slack (GitLab internal)