I joined GitLab around two and a half years ago. At the time, I was GitLab’s first and only UX researcher. I’d hunt out issues in the GitLab CE project where I felt I could add value. Usually, a member of the Product or UX/Product Design team would open the issue, which was often sparked by user feedback from social media, during a customer meeting, or even in response to a prior issue. It was my responsibility to help the teams determine how we could best address user needs, motivations, and pain points, or if the request was an edge case. I documented my research questions and insights in the associated issues living in the CE project. However, this approach had some problems:
Back then, the formatting options available for issues was in its infancy. It was difficult to structure and share data in a clear and concise way. Like most good researchers, I’d always learn more than what I intended to during a study. I created new issues for the insights we weren’t previously aware of and didn’t have documented. Epics didn’t exist yet, so there was no way to collectively group issues from the same study. I could label issues, but we discovered (maybe ironically) with UX research that GitLab’s search functionality needed improvements. The GitLab CE project contains more than 50,000 issues, so trying to find and action an insight was like trying to find a needle in a haystack.
Enter the UX Research repository and research reports. As the Product and UX/Product Design teams grew, so did the demand for UX research. Product managers and UX/product designers needed greater visibility into what I was working on so they had a sense of my availability for projects, and I needed a way to manage incoming research requests. We had success by creating a dedicated repository for UX Research requests and then using checklists within issues to track my progress against each request. However, I still had my original problem of needing to store and disseminate research insights. I resorted to Google Docs and began producing reports of my insights. This worked well for a little while, but then the cracks started to show.
The problem with reports
They are not searchable
Whenever anybody asked me if I had witnessed users experiencing a particular problem, I’d rack my brain trying to work out which research report might contain the answer. I’d sift through multiple reports, scanning everything I had previously written. The situation became worse when we added new UX researchers to the team who began producing their own reports. I had a vague idea of what was in my own reports, but I didn't know where to start with reports produced by other UX researchers.
They create research silos
As I searched through dozens of reports, I realized research findings were inaccessible. UX researchers were spending a large part of their days searching through past insights, when their time would be better spent speaking with users and uncovering new insights. Everybody should be able to find research swiftly and easily without needing a researcher to find it for them.
They are not actionable
At GitLab, we use issues to solve problems, develop ideas, and collaborate. One of our values is iteration: We do the smallest thing possible and get it out as quickly as possible. UX research reports were not small; they often contained many insights. Just one insight could lead to multiple, iterative changes to the user interface. We ended up copying parts of our reports into issues, which felt like a duplication of effort.
They quickly become outdated
Our research reports directly addressed the research questions formed with the Product and UX/Product Design teams and were extremely focused on a topic or feature. GitLab is a rapidly growing product; consequently, our research reports became outdated very quickly. Reports that felt ‘old’ or ‘stale’ were rarely revisited, but the reports contained insights that could be triangulated with more recent research. Reports didn’t provide an easy way to access this important data in the future.
Finding a solution
I wanted to confirm whether people outside of the UX Research team also felt these problems. I set up 1:1 interviews with every product manager at GitLab. In these interviews, I learned reports weren’t working for our product managers either. If something requires their attention, they want it in an issue.
I read (lots) of articles on Atomic Research and realized we could use a similar approach for managing our insights. Better yet, I felt we could dogfood our approach.
Introducing the UXR Insights repository
The UXR Insights repository is the new single source of truth for all user insights discovered by GitLab’s UX researchers and UX/product designers. Instead of reports, we use issues to document key findings from research studies.
You may be wondering why we reverted to issues, given the problems of a couple of years ago. GitLab’s issue functionality has improved immensely since then. There’s now a range of formatting options for issues, and our search functionality includes the ability to search by labels.
We use labels to tag and organize insights. This allows anyone to quickly search and filter through issues to find the insights they need. Unlike in a report, insights are continually added. This means that you’ll receive a dynamic list of results when searching through the repository.
We use epics and GitLab’s related issues functionality to track issues from the same research study. The epic description usually contains our research methodology and any background information about users.
Open issues and epics indicate that the research study is still in progress and the UX researcher and/or UX/product designer is still adding insights to the repository. Closed issues and epics indicate that the research study is finished.
Each insight is supported with evidence, typically in the form of a video clip or statistical data. Unlike the atomic research approach, some lightweight research synthesis takes place before insights are added to the repository (which is why we also call them ‘insights’ rather than ‘nuggets’ or ‘observations’). While every issue within the repository contains a single insight on a particular topic, the insight can relate to multiple users.
For example: We’re conducting some usability testing. Four out of the five users we tested with experienced the same problem. Rather than open four separate issues, we’ll create one issue, but we’ll include four supporting pieces of evidence (four video clips – one for each user) in the single issue.
We’re also experimenting with using the UXR Insights repository for quantitative forms of research, such as surveys. Each survey insight focuses on a key theme/question (for example: mobile usage) and is supported by data derived from the survey results.
Challenges and what the future holds
Our biggest challenge was transferring all our research reports into the UXR Insights repository. The team has collected a lot of data over the years, so it was a mammoth task. We never envisioned moving our research to an insights repository when we originally wrote and formatted our reports. Retrospectively adding insights means we’ve had to make some compromises; we haven’t always been able to use the insight structure that we want to use going forward.
A second challenge is training new and existing members of the UX department how to use the insights repository. We believe everyone can contribute. The UX Research team are not gatekeepers to research. We want everyone to be able to conduct research effectively and to be able to accurately add their findings to the insights repository. As a starting point, we’ve added templates to the repository that guide users through the process of adding insights.
We decided to keep our insights separate from the GitLab CE and EE projects, which is where our Product and UX/Product Design teams typically work. Not all of our insights are necessarily actionable right away – sometimes more evidence is required (especially with the gems we unintentionally discover during our studies). We needed a place where we could store and share these insights, while continuing to discuss and research them. The UXR Insights repository is within the GitLab.org group, meaning that product managers who create issue boards at a group level to manage their workflow can simply add an insight to their board when they are ready to act on it. Or they can cross-link to the insight in a supporting issue or epic.
This is our first iteration of the UXR Insights repository. We expect improvements will be required along the way, and the UX team is planning to review how the repository is working after 90 days. However, early signs indicate that (unsurprisingly) no UX researchers are missing writing reports!
Cover image by chuttersnap on Unsplash
