At GitLab, our UX Research Operations Coordinators work closely with those requesting research to make sure the people they are gathering feedback from are the right ones. If you are filling in for a UX Research Operations Coordinator, use this page as a reference.
A UX Researcher or UX Research Operations Coordinator may collaborate with you throughout the process, and we’re always available to answer questions.
Collecting the data that will help you answer your questions is essential. The only way to do that is to ensure you speak with the right people. Ask yourself the following questions to help define your target audience:
The answers to these questions should flow directly from the goals and objectives that you’ve previously identified for your study. It’s important that everyone involved in the research study agrees on these and that they’re finalized before we start recruiting. It’s better to start with clear objectives than to try to shoehorn participants into a study for which they simply aren't a match.
It’s best to think of tasks and responsibilities instead of job titles, because often a person’s job title doesn’t fully correspond to the tasks they perform.
The screener has a specific function - it’s meant to identify the people who are your target demographic, so that you can ask them the things you really want to know in the study. A best practice is to copy the Screener Template (internal link only) in Google Docs and collaborate with your stakeholders until all questions are finalized. For tips on how to write an effective screener, check out this page. Then, you’ll create a screener in Qualtrics that includes the questions. If you don’t have access to Qualtrics, request it. Recruiting will not begin until the finalized screener is created in Qualtrics and shared with the UX Research Operations Coordinators.
Your questions on the screener must match your participant criteria. This allows the UX Research Operations Coordinator to review your desired criteria and know which answers you want to see on the screener. The more abstract or open-ended you get in the screener, the harder it is for UX Research Operations Coordinators to parse which answer it is you’re looking for. A best practice is to avoid using open-ended questions in screeners.
Participants must agree to each of these questions to take part in moderated studies. You must include these questions in every screener:
Determine if you need the IP Assignment and/or GitLab's Individual Contributor License Agreement
Common questions we include are:
There are copies of these questions with standard phrasing available in the UX Research library in Qualtrics. There is also a generic template in Qualtrics based on the screener template noted above. To use this template, create a new project in Qualtrics and select the following options:
Use a survey from your libraryUX Research & ProductUXR Generic Screener (from the Templates folder)After selecting Create project you will be redirected to the survey builder within Qualtrics and will be able to edit the screener however you see fit.
Sometimes, we need to recruit participants from the same organization. In these cases, consider adding a question at the end of the screener to ask potential participants to recruit additional prospects (called snowball sampling). Example:
We are also interested in talking to people from the same organizations/teams, but who have different roles than yourself. Everyone's interviews will be done separately and everyone will be paid the same amount, once their session is over. Your participation in the session is not contingent on anyone else participating. If you know of any colleagues who would be interested in taking part in this study, please provide their email and their role/job title, and we will send them a link to this screener.
Sometimes, we are able to combine efforts by using a common screener across different studies. This is called a Common Screener.
It’s important to remember that a screener is not the same as a survey. There can be a temptation to pile on extra questions that you want to know the answer to, but aren’t strictly necessary for determining whether that participant should get through the door. You will benefit from including these questions in your discussion guide or script instead.
To schedule participants, you will need to create a Calendly event. The UX Research Operations Coordinator will send individual emails to qualified participants and ask them to book a time with you. Set up Calendly here.
Open a Recruiting request issue in the UX research project, using the Recruiting Request template. This template auto-assigns the issue to the UX Research Operations Coordinator, who will distribute the work depending on their individual workload and work with you to clarify any of your recruitment criteria.
The UX Research Operations Coordinator will only begin the recruiting process once the finalized screener is created in Qualtrics and shared with them. If you are not ready for recruitment to start and are opening the issue to get organized, mark your issue as WIP or DRAFT. The UX Research Operations Coordinator will not work on the request until WIP/DRAFT is removed and the request is ready for participants. If you are unsure if the UX Research Operations Coordinator can support your project, reach out on Slack in #ux_research or directly to @Caitlin.
When your request is ready to go, the UX Research Operations Coordinator will decide the best recruiting method to use, depending on your recruiting criteria and your research timeline. These options include:
@cfaughnan or the data team. After that, allow a couple of weeks for the emails in Qualtrics to generate your desired number of participants.Recruitment typically takes 2 weeks for target participants who are plentiful in our research panel, such as Engineers and DevOps professionals. Recruitment for low-incidence users, like security professionals or users of a newly released feature, may take more than 2 weeks. In rare cases, when your target participants cannot be found, the UX Research Operations Coordinator will suggest other options.
If you are planning to recruit users through a promotional game or contest (e.g., Opportunity to win 1 of 3 $30 (or equivalent currency) Amazon Gift cards), please review the following information in the handbook and consult with legal. For information on contacting legal, pelase refer to how to reach us in the Legal Team handbook page.Engaging legal for approval and creating an incentive request must be completed before conducting research involving promotional game or contests.
In all cases, the UX Research Operations Coordinator will keep you updated on the progress of the recruitment effort by commenting on the recruiting issue.
The UX Research Operations Coordinator is responsible for launching recruitment campaigns, prioritizing requests, scheduling participants, reimbursing them for their time, and ensuring NDAs (as needed) are signed and stored properly.
Once the screener has been sent to potential participants, the UX Research Operations Coordinator will monitor the results and share them with the person who requested the recruiting. This is done via a Google spreadsheet. They will suggest which potential participants best fit your recruiting criteria. The DRI for the research may also provide a list of names to the UX Research Operations Coordinator from the responses that they would like invited.
Once potential participants are agreed upon, they will send the invitations to schedule to qualified participants. The UX Research Operations Coordinator relies on you to update them in a timely manner as to whether any bookings have been made. You should plan to alert them within a couple of business days if no bookings have been made on your calendar, so they can send reminders or fresh invitations. They will repeat this cycle until the request is fulfilled.
If you have already opened a recruitment request and are actively working with the UX Research Operations Coordinator on your project, they will process incentives as required for participants that have been taking part.
If you are recruiting participants on your own, and need to process a gratuity, open an Incentives request issue in the UX research project, using the Incentives Request template. Be sure to @ mention the UX Research Operations Coordinator.
The person requesting the recruiting needs to fill out a spreadsheet, provided by the UX Research Operations Coordinator, to indicate which participants receive a gratuity for moderated research. Make sure to provide the participant’s name, email, amount of gratuity, and which currency it needs to be in.
The UX Research Operations Coordinator will send a gratuity to participants who have completed research sessions or who have been randomly selected to win a prize through completing a survey. Once all participants have been sent gifts, the UX Research Operations Coordinator will close the issue.
If we share information that is Confidential, as described in the GitLab Code of Business Conduct and Ethics, we may ask research participants to sign an NDA. Typically, however, we don’t ask participants to sign an NDA. A best practice is to check with your Product Manager to see if an NDA is needed for your research. NDAs must be signed by both the participant AND a member of GitLab’s legal team.
If one is needed, reach out to @asmolinski2 and let him know. He will set up a spreadsheet to document the participants who need to sign an NDA. He will create the NDA and send it to the participants. It is the responsibility of the person conducting the research to check the status of their participant’s NDA prior to conducting their session. If an NDA is not signed by all parties, the session will need to be rescheduled.
To understand how the above process works in action, the following research study is a great example:
Accessibility solution validation research issue
The stated goals of the study were:
As part of our continued effort to improve the automated testing capabilities of GitLab, we want to ship a new feature for accessibility testing. We want to validate some solutions for this feature. Questions we have are:
- How exactly users want to see Accessibility issues introduced during their development shown to them.
- Which of the proposed designs suffice their use case for Automated Accessibility Testing as part of the CI process.
How many participants did they need?
It was a usability test, so around five users is generally the sweet spot.
What was the timeline?
The Product Designer stated that they were ready to begin any time, and notified the UX Research Operations Coordinator of their vacation time.
Who did they want to speak with?
Frontend engineers was submitted, but that's too broad. Instead, focusing on their day-to-day activities helped the UX Research Operations Coordinator find accurate matches.
In this example, the key questions that recruiting hinged on were really clear: the UX Research Operations Coordinator expected to see a question about job title, common tasks participants performed or job responsibilities they had, and something about how they perceived the importance of accessibility compliance. The UX Research Operations Coordinator also expected to see things that we almost always include, like company size and industry, which we try to get a mixture of when they’re not the object of the study.
This study is unusual in that we're not asking about what tools people use. Often, we explicitly want to see what someone with or without GitLab experience thinks. It wasn't relevant for this study, so we're not collecting that information.
Keeping things short and sweet increases the completion rate of screeners.
Check out the finished screener
A lot of the conversation above occurred in the recruitment request.
The UX Research Operations Coordinator took great care in creating a segment from our research panel (GitLab First Look). Because the study was about accessibility, a topic about which many people are passionate, this study was a good one to use custom subject lines and email copy. The UX Research Operations Coordinator sent out an email with the subject line New study - let's talk accessibility. After two emails, we received over 20 responses.
The UX Research Operations Coordinator highlighted in green the people who said they do accessibility testing in their jobs, and strongly agreed that accessibility compliance is important (4 total). They highlighted in orange people who only fit the latter criterion (9). Altogether, this is a surplus of participants, however, we now have some participants that we can reach out to directly when the next accessibility study is available.