Gitlab hero border pattern left svg Gitlab hero border pattern right svg

UX Researcher Workflow

On this page

UX Researcher onboarding

If you are just starting out here at GitLab, welcome! Make sure to review all the pages here in the UX Research section of the handbook to help get you get oriented. There is also a specific page dedicated to UX Researcher onboarding.

How we decide what to research

UX Researchers collaborate with Product Managers to determine the scope and priority of research studies. UX Researchers should sit in on monthly planning calls for their relevant product stages to stay informed about what Product feels are the most important initiatives at GitLab. Additionally, UX Researchers should proactively offer ways in which they can assist in the delivery of such initiatives.

UX Researcher tools

All UX Researchers have individual accounts for the following tools:

SurveyMonkey - All UX Researchers should utilize SurveyMonkey for surveys. Please use the GitLab Theme to style your survey. The GitLab logo should be positioned in the top left hand corner of every page (applied automatically, resize to small).

UsabilityHub - Used to build design evaluations, such as first click tests, preference tests and five second tests. UsabilityHub should not be used for surveys.

MailChimp - Used to send campaigns to subscribers of GitLab First Look.

OptimalWorkshop - Used for card sorts and tree testing. We do not have an ongoing subscription to OptimalWorkshop. We purchase a monthly license as and when required.

Zoom Pro Account - We use Zoom to run usability testing sessions and user interviews.

Research methods

We use a wide variety of research methods that include (but are not limited to):

Reviewing a design evaluation (guidance for UX Researchers)

  1. UX Researchers should familiarize themselves with the process outlined in How to create a design evaluation (guidance for Product Designers).

  2. When sending a design evaluation to GitLab First Look users, ensure standard processes are followed; for example, distributing the study to a test sample of users and sending the study to GitLab First Look subscribers who have opted into design evaluations from the relevant product stage.

Incentives

Amazon gift cards are country specific. When purchasing a gift card, ensure you use the appropriate Amazon store for a user's preferred country.

How we work

Like other departments at GitLab, UX Researchers follow the GitLab Workflow and the Product Development Timeline.

We follow a monthly research cycle and use milestones to schedule our work. A typical monthly research cycle is as follows:

8th of each month (or next business day)

8th to 21st of each month.

22nd of each month.

22nd to the last day of the month.

1st to 7th of each month.

Depending on the scope of each research study, a UX Researcher typically works on 1-3 studies per research cycle. Sometimes, larger studies may need to span more than 1 research cycle.

Scheduling issues

Working on a research study

  1. Update the Research proposal with the following:
    • Label the issue with the area of GitLab you’re testing (for example, navigation), the status of the issue (in progress) and the Product stage (for example, manage).
    • Add a milestone to the issue.
    • Mark the issue as confidential until the research is completed, so it doesn’t influence user behavior.
    • Assign the issue to yourself.
    • Add a checklist of actions that you plan to undertake. Next to each item, add an estimated deadline of when you plan to complete it. This makes it easier for people to understand where the research is up to.
    • Add related issue numbers.
  2. Conduct the research. Ensure you keep the checklist up to date.
  3. Document the study's findings within the UXR_Insights repository
    • Refer to the repository's ReadMe for instructions on how to do this.
    • Use the repository's issue templates to guide you through the process of adding your findings.
  4. Schedule a wash-up meeting.
    • Ensure you link to the study's Epic in the calendar invitation at least 24 hours prior to the meeting.
  5. Record and facilitate the wash-up meeting.
  6. Add the next steps/recommendations as agreed upon in the wash-up meeting to the Epic description.
  7. Update the Research proposal with the following:
    • Link to the Epic.
    • Unmark the issue within the UX research project as confidential. (In some cases the issue may need to remain confidential if sensitive information is shared. If you’re unsure of whether an issue should remain confidential, please check with Sarah O’Donnell @sarahod).
    • Update the status of the issue to done.
    • Close the issue. You should stay assigned to closed issues so it's obvious who completed the research.

Qualtrics

Creating a survey
  1. Select Blank survey project
  2. Give your project a name.
  3. Enter survey questions. On the right hand side of the screen you will find:
  4. Update the Look & Feel of your survey.
  5. Adjust your Survey Options as appropriate.
  6. Under the Tools menu, you will find a host of useful features such as:
    • Collaborate - Allows you to share your survey with another Qualtrics user so you can edit it and/or analyze the results together.
    • Spell check - Supports English (US) only.
    • Generate test responses - Generate automated dummy responses for your survey to see what the dataset and report will look like before sending the survey to actual participants.
    • Check survey accessibility - Determine whether your survey is accessible for respondents who use screen readers, and receive suggestions for improving your survey’s accessibility.
    • Analyze Survey - Open ExpertReview‘s suggestions for your survey.
    • Import/export survey - From/to a QSF, TXT or DOC file.
  7. When you have finished building your survey, Preview it to ensure that everything is working the way you expect it to.
  8. Once you are ready to launch your survey, click Publish.
Look & Feel settings

The following settings should be applied to your survey:

  1. Theme: Blank
  2. Layout: Modern
  3. General
    • Progress bar: Without Text
    • Progress bar position: Top
  4. Style
    • Primary colour: #554488
    • Secondary colour: #554488
    • Font: Arial
    • Foreground contrast: Medium
    • Question spacing: Compact
    • Question text: 14px bold
    • Answer text: 14px
  5. Motion
    • Page transition: None
    • Additional check boxes: Unticked
  6. Logo
    • GitLab First Look Logo (stored in UX Research library)
    • Style: Banner
    • Placement: Left
    • Max height: 70px
    • Mobile scaling: 66%
  7. Background
    • Background type: Color
    • Background color: #ffffff
    • Foreground contrast: Medium
    • Questions container: Off
Distributing a survey to GitLab First Look
  1. Select Contacts (top right menu).
  2. Select Lists (far left menu).
  3. Select GitLab First Look
  4. Select List Options
  5. From the dropdown menu, select Create Sample From List.
  6. Give your sample a name.
  7. Select your sample size (the number of people who you ideally want to distribute your survey to).
    • It is advisable to send your survey to a small test sample of users to begin with, to ensure that you receive the kind of responses you are looking for.
    • When you distribute your survey to a larger audience, to avoid contacting the same users who have already answered your survey as part of your test sample, click Advanced Sampling Criteria, ensure the sampling criteria statement reads: All of the following are true, change the Contact Activity filter to Survey and change the Select survey filter to the survey you have already distributed.
  8. Click Add Sampling Criteria
  9. Ensure the sampling criteria statement reads: All of the following are true
  10. Select Embedded data, select an Embedded Data field, select an Operator and enter/select an Embedded Data Value.
    • For every sample you create, you must always add sampling criteria for Stage groups and Types of research. When a person signs up to GitLab First Look, they specify what they would like to receive studies about (stage groups) and what type of research they would like to take part in. It’s extremely important that when you contact users, you only email them in relation to their preferences.
    • Remember the embedded data for Stage groups and Types of research can contain multiple values (it’s rare that users sign-up to receive emails abouts one particular stage group or want to take part in one form of research). Therefore, your Operator will always be Contains.
    • For more information on what embedded data is available, please refer to the embedded data section.
  11. Click New Condition to add additional sampling criteria.
  12. Click Create once you have finished entering your sampling criteria.
  13. A progress bar will appear, this indicates that Qualtrics is building your sample.
  14. Once the sample is ready, click Go to Sample (Alternatively you can navigate to Lists and your new sample will be visible under All Lists. Click on your sample to access it).
  15. The number of contacts in your sample will show on the left hand side of the screen.
  16. If you are happy with the number of contacts in your sample, you are now ready to send your survey to users. To do this, go to Projects
  17. Select the project/survey you want to distribute to users.
  18. Click Distributions
  19. Click Compose Email
  20. In the To field, navigate to: Group Library: UX Research -> GitLab First Look -> Samples and select the sample you created earlier.
  21. Update the From field to: firstlook@gitlab.com
  22. Update the From Name to: GitLab First Look
  23. The Reply-To Email can either be your own, personal GitLab email address or firstlook@gitlab.com
    • If you are planning to leave a survey running while you are on vacation, please use firstlook@gitlab.com as this will automatically forward to all UX Researchers who may be able to assist with any user queries in your absence.
  24. Update the When field to the time you would like to send the email.
  25. Enter your subject line.
    • A standard subject line we use for campaigns is: Quick, new research study available! however, if you wish, you may experiment with this. Please ensure that you use a subject line checker to evaluate your subject line.
  26. Delete the standard text/links which appear in the WYSIWYG text area.
  27. Click Load Message next to the Message field.
  28. Navigate to Group Library: UX Research and select the template you would like to use.
  29. Select Use Fixed Text option from the Load Message dropdown after the template has loaded. This stops any changes you make to the template from being saved permanently.
  30. Edit the message text as appropriate.
    • By default, the call to action always links to the current project/survey you are planning to send. As does the screening survey text link found in the body of templates for Usability testing and User interviews.
  31. When you have finished building your email, click Send Preview Email and enter your GitLab email address. A copy of the email will be sent to you for review.
  32. If you are happy with your email, click Send
Embedded data

When users sign-up to GitLab First Look, we automatically collect the following embedded data:

  1. Stage groups
    • 1 - Manage
    • 2 - Plan
    • 3 - Create
    • 4 - Verify
    • 5 - Package
    • 6 - Release
    • 7 - Configure
    • 8 - Monitor
    • 9 - Secure
    • 10 - Defend
    • 11 - Enablement & Growth
  2. Types of research
    • Beta testing
    • Cards sorts
    • Design evaluations
    • Surveys
    • Usability testing
    • User interviews
  3. GitLab user - Whether or not the person is a GitLab user.
    • Yes - the person is a GitLab user.
    • No - the person is not a GitLab user.
  4. Job title - What the person's job title is.
    • Back-end Engineer/Developer
    • Designer
    • DevOps Engineer
    • Executive (VP of Engineering, CTO, CEO, etc)
    • Front-end Engineer/Developer
    • Full-stack Engineer/Developer
    • Infrastructure Engineer
    • Operations Engineer
    • Penetration Tester
    • Product Manager
    • Project Manager
    • Quality Assurance Engineer
    • Researcher
    • Security Analyst
    • Security Professional
    • Scientist
    • Site Reliability Engineer
    • Software Engineer/Developer
    • Student
    • Systems Administrator/Engineer
    • Unemployed
    • Other
  5. Organisation size - How many people work within the user's organisation? (Added August 2019)
    • 0-10 people
    • 11-100 people
    • 101-500 people
    • 501-1000 people
    • 1001-10,000 people
    • 10,000+ people
  6. Team size - How many people work within the user's team (Added August 2019)
    • 0 - it's just me!
    • 1-5 people
    • 6-10 people
    • 11-20 people
    • 21-30 people
    • 30+ people
  7. SaaS (GitLab.com) package
    • Free
    • Bronze
    • Silver
    • Gold
    • I don't know
  8. Self managed package
    • Core
    • Starter
    • Premium
    • Ultimate
    • I don't know
What to do if your sample size is lower than expected

There could be a couple of reasons why your sample number is lower than expected:

  1. The contact frequency for GitLab First Look has been exhausted. By default, no member of GitLab First Look receives an email more than once a week or four times a month. If this is the case, you will need to either delay sending your survey (you may only need to delay sending your survey by a day - reach out to the UX Research Coordinator for confirmation) or source users outside of the GitLab First Look panel.
  2. There aren’t enough GitLab First Look members that match your sampling criteria.
  3. There’s an error in your sampling criteria.

Wash-up meetings

After a research initiative, the UX Researcher schedules a "Wash-up Meeting" with all stakeholders to help communicate findings and collaboratively agree on a list of improvements/actions to take forward.

Checklist templates

The following are examples of checklists that you may want to add to a Research proposal.

Usability Testing

User Interviews

Surveys

Design Evaluations

Card Sorts