Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Acquisition and Conversion Group

Vision

The Acquisition and Conversion Groups are part of the Growth section. The Acquisition Group is responsible for promoting the value of GitLab and accelerating site visitors transition to happy valued users of our product. The Conversion Group is responsible for continually improving the user experience when interacting with locked features, limits and trials.

I have a question. Who do I ask? Questions should start by @ mentioning the Product Manager for the Acquisition group, the Conversion group, or by creating an issue in the Growth issues board.

How we work

Workflow

Our development process follows the Engineering workflow.

Cadence

There are two cadences we operate on: a Monthly Milestone Cadence for feature development and a Weekly Experiment Cadence for high-velocity experimentation.

Each group will decide which cadence to follow depending on the work defined by the Product Manager:

Monthly Milestone Cadence

In a monthly milestone cadence, we plan and deliver work on a monthly cycle using milestones and following the product development timeline.

To ensure we adhere to this monthly cycle, we pay special attention to these dates:

We plan and groom issues in advance of finalizing a milestone commitment on M-1, 13th and before beginning development on M-1, 18th.

Milestone Commitment

A milestone commitment is a list of issues our team aims to complete in the milestone.

Our milestone commitments are based off of our team's estimated capacity, which is the sum of issue weights completed in the last milestone. If there is a large variation in the estimated capacity of the last milestone and the one before it, we will use an average estimated capacity of the last few milestones.

Once issues are groomed, estimated, and prioritized, we begin loading issues into a milestone commitment until the sum of the loaded issue weights equals our team's estimated capacity. The loaded issues are our milestone commitment and are tagged with the correct milestone label.

Weekly Experiment Cadence

In a weekly experiment cadence, we plan and deliver work on a weekly cycle.

We follow a four step process as outlined by Andrew Chen's How to build a growth team.

  1. Form Hypotheses: Define ideas our team wants to test.
  2. Prioritize Ideas: Decide which ideas to test first.
  3. Implement Experiments: Do the Product, Design, Engineering, Data, and Marketing work to execute the experiment.
  4. Analyze Results: Dive into the results data and prove or disprove our hypotheses.

Each week, we provide progress updates and talk about our learnings in our Growth Weekly meeting.

Our aim is to implement one new experiment per week. The duration of each experiment will vary depending on how long it takes for experiment results to reach statistical significance. Due to the varying duration, there will be some weeks where we have several experiments running concurrently in parallel.

Broader Team Integration

Some of our work will need to be integrated into the monthly milestone cadence which the broader engineering and product organizations.

Examples of this include:

To help with this integration, our experiments will still be tagged with the correct milestones.

How We Use Issues

To help our team be efficient, we explicitly define how our team uses issues.

Issue Boards

We use group level issue boards for visibility into all nested projects in a group. We use workflow boards to track issue progress throughout a milestone cycle. We use milestone boards for high level planning across several milestones.

gitlab.com/gitlab-org

The gitlab.com/gitlab-org group includes the Gitlab, Customers, and License projects.

gitlab.com/gitlab-services

The gitlab.com/gitlab-services group includes the Versions project.

Issue Creation

We aim to create issues in the same project as where the future merge request will live. For example, if an experiment is being run in the Customers application, both the issue and MR should be created in the Customers project.

We emphasize creating the issue in the right project to avoid having to close and move issues later in the development process. If the location of the future merge request cannot be determined, we will create the issue in our catch-all growth team-tasks project.

We aim to maintain a 1:1 ratio between issues and merge requests. For example, if one issue requires two merge requests, we will split the issue into two issues. We aim for a 1:1 ratio as we believe it emphasizes keeping MRs small, it makes estimation on issues straight forward, and it provides more visibility into an issue's progress.

Issue Hierarchy

There are two ways to organize issue hierarchy: Epics and Parent Issues.

Depending on their preference, each group's Product Manager will decide which issue hierarchy to use:

Epics

Epics are a Gitlab feature that allows easy linking of higher-level themes across issues. Epics have a roadmap gantt chart which is useful for visualizing work.

Parent Issues

Parent issues are normal Gitlab issues that are used to group several child issues together. Since Parent issues are normal issues, they have the full functionality of issues including the ability to add labels, add milestones and assign team members.

Issue Labels

We follow the issue label definitions in the issue workflow in addition to the ones mentioned below:

Workflow Labels

Our groups utilize a set of three newly created experiment workflow labels in conjunction with the existing workflow labels.

Experiment Results Labels

To help our group track the results of our experiments, we utilize the following labels:

Milestone Week Labels

To help our group prioritize experiments within a milestone, we utilize the following labels:

Release Scoping Labels

When following a monthly milestone cadence, the top 70% of a milestone's issues will be labelled as Deliverable while the remaining 30% of issues will be labelled as Stretch.

If a single deliverable is broken in to mutliple smaller issues, one of these issues should be labelled as the Deliverable while the other issues are dependencies and are linked to the Deliverable issue. Once all dependency issues have been delivered, the issue with the Deliverable label can be closed.

Prioritization

Prioritization is a collaboration between Product, UX, Data, and Engineering. We use the ICE framework for experiments. Each Deliverable issue will have an ICE score.

Estimation

We use a light-weight estimation process using a fibonacci scale with a maximum weight of 5. Anything over a 5 (large) indicates the work should be broken down into smaller, clearly defined issues.

Weight Description (Engineering)
1 Trivial: The simplest possible change. We are confident there will be no side effects.
2 Small: A simple change (minimal code changes), where we understand all of the requirements.
3 Medium: A simple change, but the code footprint is bigger (e.g. lots of different files, or tests effected). The requirements are clear.
5 Large: A more complex change that will impact multiple areas of the codebase, there may also be some refactoring involved. Requirements are understood but you feel there are likely to be some gaps along the way.

In planning and estimation, we value velocity over predictability. The main goal of our planning and estimation is to focus on the MVC, uncover blind spots, and help us achieve a baseline level of predictability without over optimizing. We aim for 70% predictability instead of 90%. We believe that optimizing for velocity (MR throughput) enables our Growth teams to achieve a weekly experimentation cadence.

Daily Standups (Async)

We have daily asynchronous standups using status hero's Slack integration. The purpose of these standups are to allow team members to have visibility into what everyone else is doing, allow a platform for asking for and offering help, and provide a starting point for some social conversations.

Three questions are asked at each standup:

Meetings (Sync)

Each group holds synchronus meetings twice a week to gain additional clarity and alignment on our async discussions.

The agenda for each meeting follows this structure:

Meeting rules:

Measuring Throughput

One of our main engineering metrics is throughput which is the total number of MRs that are completed and in production in a given period of time. We use throughput to encourage small MRs and to practice our values of iteration. Read more about why we adoped this model.

We aim for 10 MRs per engineer per month which is tracked using our throughput metrics dashboard.

Team Members

The following people are permanent members of the Acquisition and Conversion Group:

Person Role
Jerome Ng Engineering Manager, Growth:Acquisition and Conversion
Alex Buijs Senior Fullstack Engineer, Growth:Acquisition
Nicolas Dular Senior Fullstack Engineer, Growth:Acquisition
Alper Akgun Senior Fullstack Engineer, Growth:Conversion