GitLab task lists are a list of checkboxes that you can include anywhere in GitLab where you can have GitLab Flavored Markdown (GFM). This includes issue descriptions and comments, as well as merge requests and epics. They can be used for a list of items to consider when building a feature, tracking tasks for new employees to complete when onboarding, or even managing that list of materials to purchase for your next home renovation. You can use them as todo lists, and so checking off an item should be quick and satisfying.
More checkboxes, more problems
In the past, task lists with several items, even dozens, worked fairly well. Check an empty checkbox, and a database record gets updated. The checkbox is then displayed as checked. Done.
However, as the number of items increases, and the consequent markdown becomes more complex and longer, problems begin to appear. For example, visually the checkbox appears checked, but because updating the backend takes a longer time, if you checked another checkbox, the screen would refresh several seconds later and the checkbox might then be unchecked. It soon became next to impossible to go down a list and check off items without waiting 10 seconds between each one. Yet another problem was that if other users were also checking items on the list, your change could be erased by them checking their item – they were overwriting your data.
In GitLab 11.8 (released on Feb. 22, 2019), we significantly increased the performance of task lists, as well as making them much more robust. Here's how we did it:
Essentially we wanted:
- Checking a checkbox to be as fast as possible.
- Many users to concurrently interact with checkboxes in the same task list, without overwriting each other.
Both the performance and data integrity issues stemmed from the fact that we were updating the complete markdown. This meant that we changed the markdown source in the browser with the updated checkbox, sent it to the backend, where it was saved to the database, and then re-rendered so that we could cache the new and send it back to the user.
A scalable solution
But what if we could update a single checkbox, and send only that to the backend? That might allow multiple users to check off as many tasks as they wanted, without clobbering each other. And what if we didn't have to do any markdown rendering at all? We wouldn't have to do any markdown processing, or process embedded issue links, or query if labels have changed, or any of the other advanced things that go on when updating an issue. Performance would definitely increase in this case.
Frontend work
On the frontend, with only a small modification to the
deckar01/task_list
gem we use, we were able to pass the exact text and line number in the markdown source
for the clicked task.
Wrap this piece of information
in a new update_task parameter for our update endpoint, and send it to the backend.
Backend work
On the backend, we needed to verify that the task we were interested in still existed in exactly the same format – the text had to match the exact line number in the source. This meant that even if someone changed text above or below the task item, as long as our line matched exactly, we could update that line in the latest source and save it without losing changes.
In order to update our cached HTML so that we wouldn't have to re-render it, we turned on
the SOURCEPOS flag of the CommonMark renderer, which adds a data-sourcepos attribute to the HTML.
For example, a task item's might look like this:
<li data-sourcepos="1:1-1:12" class="task-list-item">
<input type="checkbox" class="task-list-item-checkbox" disabled> Task 1
</li>
With a little Nokogiri magic we were able to find the correct line and toggle the checked attribute.
Since we updated the cache directly, we completely bypassed any markdown rendering, processing of special attributes, etc. Performance dramatically increased. However, since we are not able to get it down to zero, we disabled the checkboxes while the request was in flight to ensure we weren't getting clicks on other tasks.
The result: a much more satisfying task list.
Brett Walker worked on the backend changes and Fatih Acet worked on the frontend changes in this improvement. See more details in the GitLab issue.
Photo by Glenn Carstens-Peters on Unsplash
