Blog Small experiments, significant results and learnings
Published on: April 7, 2021
6 min read

Small experiments, significant results and learnings

How our Growth team validates design solutions with the smallest experiments possible


A while ago, I worked closely with the Growth:Expansion team on improving the experience of inviting users to GitLab. I first went through the existing experience of a user that’s inviting their team to GitLab and found a few opportunities for improvements. The work could have ended there, but I felt uneasy about it in the days after completing it. I felt that there was more to it, so I dug in again. This time, I wanted to explore what the experience was across many users involved in the process, instead of just the inviting user.

Multi-user journey map

So instead of mapping out a single-user journey map, I mapped out the journey as it was then for all the users involved. I came up with the following:

Multi-user journey map

Take a look at the multi-user journey map on Mural.

I wanted to see what it was like for non-admin users to invite their team to GitLab or assign some work to them (not all managers and leads are admin users). So a non-admin user wants to assign an issue to a team member that isn’t on GitLab yet. There are three users involved: the non-admin user trying to assign some work, the admin user who is the only one who can invite new users and the user that is being invited. The main conclusion of this multi-user journey map? There are many interruptions and a lot of waiting time between the steps. Such a simple task as assigning an issue to a team member can span across days because of these interruptions.

The other conclusion of this work was that it was hard to find out how to invite users to GitLab, especially for new teams trying to adopt GitLab.

So we came up with a problem to solve:

Can we make it easier for new teams to invite their team members?

and a question to answer:

Would non-admin users request invitations to their team members if they could?

Making it easier for new teams to invite their team members

We started with the smaller problem as it was a great candidate to do a MVC (minimal viable change) experiment and learn a lot from it. The concept of the solution was simple: increase the discoverability of the invite members feature. After some thought, I realized that the best place for this was the Assignee dropdown that we use on issues and merge requests. It’s at the top of the right sidebar, which means it’s quite prominent, but more importantly, it is a commonly used feature related to team management.

So we decided that the most minimal experiment we could do was to add the “Invite members” link to the bottom of that dropdown and link directly to the SettingsMembers page of the project. That’s the page where admin users can invite new users.

Assignee dropdown

Here’s the prototype of the experience we tested. After crunching the numbers of the experiment we saw the following results:

  • Only a 0.16% click-through rate on the “Invite members” link in the dropdown
  • But a 2% increase in namespaces with two or more users This was significant because we only showed the new “Invite members” link to admin users. So the low click-through rate makes sense as the majority of users viewing the assignee are not admins and therefore did not see our test "invite members" option. However, even with a low click-through rate, the change in the metric that mattered most saw a 2% increase. Which is a considerable increase on its own! But we’re just getting started.

Do non-admin users want to invite their team members?

Now we come to the question that we uncovered during the mapping of the multi-user journey. There’s a lot of waiting time and disruptions in the process of inviting a new user to GitLab. Especially when a non-admin user wants to do it. So we decided to run a similar experiment where we show the “Invite members” link in the assignee dropdown to non-admin users too. Following our MVC approach to conducting experiments, we wanted to run a minimal experiment that would help us answer this question. Instead of taking the time to build a complete experience for non-admin users requesting invitations from admin users, we decided to show a modal explaining that this feature isn’t available yet. We also added a link that would take the non-admin user to the SettingsMembers page, where they can see who the admin is and contact them outside of GitLab (for now).

Modal not ready It’s not the ideal experience, but the potential for learning justified it. Plus, we only show an experiment like this to a fraction of our users. The experiment has only been running for a few weeks, so it’s too early for conclusions. But we’re seeing encouraging results already, some suggest even up to a 20% increase in namespaces with two or more users so yes, it seems that non-admin users do want to invite their team members.

Other improvements and follow-up experiments

One major improvement that our engineers have been working on is the “Invite members” modal. Instead of taking the user out of their workflow and into the SettingsMembers page, they’ll be able to invite team members within their current workflow.

Modal invite form

This is a prototype of what the experience would be like with the invite modal.


These experiments are the first among many that we want to conduct. We’re also thinking about allowing non-admin users to request a free trial, activation of a feature, switching to a higher plan from their admin all while potentially giving the admin the ability to turn this functionality off and on as needed. The experiments we conducted so far are indicating that there’s a demand for non-admin users to be able to request things limited to admins. And most importantly, they were minimal experiments that led to significant results and great learnings.

For more details about these experiments check

Photo by Evgeni Tcherkasski on Unsplash

We want to hear from you

Enjoyed reading this blog post or have questions or feedback? Share your thoughts by creating a new topic in the GitLab community forum. Share your feedback

Ready to get started?

See what your team could do with a unified DevSecOps Platform.

Get free trial

Find out which plan works best for your team

Learn about pricing

Learn about what GitLab can do for your team

Talk to an expert