Real-time Editing of Issue Descriptions (REID) was a Single-Engineer Group (SEG) investment within GitLab’s Incubation Engineering Department. This SEG effort has ended due to a team member departure, effective March 16, 2023.
The goal of REID was to ship an MVP to enable real-time collaborative editing of issue descriptions. This would also serve as the foundation for building additional real-time collaborative text-editing use cases at GitLab. A user should never have to leave the GitLab application for tasks like pair programming, collaborating on isssues, merge requests, tech designs and RFCs. REID has been developed following the principle of progressive enhancement.
While a working solution to real-time editing of issue descriptions has not yet been implemented, the SEG laid the foundations for completing this work at a future date by delivering the following key results:
Y.js (implementation of the YATA CRDT for conflict-free state replication between many participants).These libraries are currently in proof-of-concept state and have yet to be integrated and validated to work well with a production application such as GitLab.
REID is a complex subject and many challenges were faced along the way:
Awareness feature
was built and an integration attempt was made, but ultimately
did not ship because of open questions around how to slot it into GitLab's overall IA.A hand-over with the Application Performance group has been conducted in this issue.
The following artifacts were produced by this SEG. Some of these are ongoing efforts and will require more time to be completed and considered ready for production.
GitLab's software stack enables real-time collaboration via Websockets, in particular GraphQL subscriptions. Real-time comes with certain complexities when users want to collaboratively edit text, in particular rich text. In order to allow multiple parties to edit the same text concurrently, we must ensure a conflict-free replication mode between all participating clients. There are several ways to achieve this, but we utilize CRDTs. CRDTs allow us to concurrently edit state from 1, 2, … n clients and eventually end up with a consistent representation of the document on all clients.
Every participant in the editing process is a client (in comparison to Operational Transformation). The CRDT we picked (YATA) has a popular frontend implementation Y.js and we have created bindings for its Rust port y-crdt. This allows our Ruby on Rails backend to just act as one more client.
y-rb gemCurrent State: Published to RubyGems.
This Ruby gem is developed under the y-crdt umbrella (project link) and brings basic and complex shared types (Array, Map, Text, XML) to Ruby. The encoded updates and state vectors are 100% compatible with Y.js and therefore state can be synced between the JavaScript frontend and the Ruby backend in a seamless way. It is implemented as Ruby language bindings on top of yrs,
the Rust implementation of YATA, but adds more idiomatic interfaces that Ruby developers would be accustomed to.
The reference implementation of YATA is written in JavaScript, including the server-side components implementing the transport (websockets in our case.) In the communication model used in the reference implementation, the server is just another peer in the synchronization network, but plays a special role: it is the SSoT for what the latest state is, where and how it is perstisted, and what should happen should clients disconnect and need to catch up on lost updates. This necessitates the server, which is a Rails app (i.e. written in Ruby), to be able to understand and transform state updates, so a Ruby implementation of YATA was needed.
Trade-offs, limitations, open issues:
yrs bindings. Familiarity with magnus
(used for Rust-Ruby bindings) and rb-sys (writing C-Ruby extensions in Rust) are necessary.y-rb_redis gemCurrent State: Started
This Ruby gem is developed under the y-crdt umbrella (project link). It is a thin abstraction layer to allow persisting changesets to Redis. The updates produced by y-crdt documents are stored as lists using Ruby's Array binary encoding.
This allows us to persist and restore documents efficiently.
In the server-centric model, the server acts as a state mediator between clients and allows them to catch up on lost updates in the event of network partitions or similar problems. This necessitates the server to own a copy of the latest YATA document with all state updates applied. If a client reconnects, the server can push the latest state vector into the subscriber without the client having to wait for another client propagating a new update.
Trade-offs, limitations, open issues:
y-rb_actioncable gemCurrent State: Started
This Ruby gem is developed under the y-crdt umbrella (project link).
It implements the Y sync protocol on top of ActionCable by providing
a SyncChannel that clients can subscribe to in order to receive and request state updates.
On top of using y-rb_redis to store YATA state, it utilizes Redis Streams
to keep track of which updates were last seen by any given client.
GitLab has settled on ActionCable to implement websockets on top of the
Redis PubSub adapter.
This implementation has no delivery guarantees (we look for at-least-once), and no atomic broadcast mechanism.
This means that if a client disconnects then reconnects, it may lose out on updates other clients may have sent to the
server. Upon reconnection, a client is now able to sync its state with the server and catch up on lost messages.
Trade-offs, limitations, open issues:
SyncChannel implemented in this library (YATA updates) or via the existing GraphqlChannel where updates propagate
by posting a GQL mutation and/or by triggering a GQL query in response to changes being made to the underlying model.
It may depend on the individual use case which approach makes more sense: if the database is the source of truth and
conflicts or data races are unlikely, GraphqlChannel suffices. An example could be updating the issue label set.
For more complex use cases such as RT text editing, data flows at much higher frequency and needs to be represented
efficiently on the wire (using a binary encoding) and YATA provides us assurance that updates are free of conflicts.
It is conceivable to implement GraphQL mutations and queries on top of SyncChannel, but GraphQL does
not support binary wire formats (we deliver JSON payloads) and sending every YATA update as a mutation will likely
create unncessary overhead. Both approach could therefore co-exist.Current State: Started
Starting a collaborative editing session should be exactly the same as opening and editing an issue today. In case a second user starts to work on an issue, the backend will create a collaborative editing session users automatically and makes sure that all changes (deltas) are synced between all participating parties.
Awareness helps us to see who we are collaborating with and how. It helps us answering the following questions:
Trade-offs, limitations, open issues:
Current State: Started
To determine and replicate a change to the description field, we need to encode everything that a user does as a change event (delta). Y.js uses a binary encoded representation to optimize payloads, but essentially it is a list of operations. One example of how this looks like is the Quill Delta Format.
The GitLab UI does not rely on a uniform text editor but instead offeres a mix of textarea, tiptap/prosemirror, Web IDE, …, multiple editor modes (Raw, WYSISWYG), and post-edit transforms (Markdown parser → references). We need to build a transform layer to broadcast changes made to a text field in a conflict-free format, and reversly, apply incoming changes seamlessly in the editor.
Trade-offs, limitations, open issues:
Write a binding (transform) that allows to incrementally receive and apply updates in any text editing mode. This could be accomplished by first translating markdown into an intermediate AST that could act as a universal interchange format between different editors:
| Date | Topic | Description | Video | |
|---|---|---|---|---|
| 14 | 2022-12-21 | An ActionCable based Y.js WebSocket and BroadcastChannel provider | Editor bindings | https://www.youtube.com/watch?v=g–9JLOMXeQ |
| 13 | 2022-12-12 | Rust available in gdk and base images | Adding Rust | https://www.youtube.com/watch?v=xmj7SzYIo1o |
| 12 | 2022-11-17 | Adding Rust to GitLab Pt.2 | Adding Rust to gdk, CI, etc. | https://www.youtube.com/watch?v=K_vidq8VdAs |
| 11 | 2022-11-02 | Adding Rust to GitLab | Shipping a gem with a native extension is tricky and extends the scope of the project. | https://www.youtube.com/watch?v=IQS81adVWWg |
| 10 | 2022-10-19 | Message protocol, Editor bindings, Awareness Pt. 2 | Two busy weeks, and lots of complexity. | https://www.youtube.com/watch?v=wJnvhW4eBKQ |
| 9 | 2022-10-06 | Real-time demo (y-rb usage) | Celebrate the first table y-rb multip-platform release with a sandbox demo. |
https://www.youtube.com/watch?v=1gfYfUMh3dU |
| 8 | 2022-09-16 | Change of plans | Awareness feature not to ship to current issue sidebar and y-rb multi-platform releases. | https://www.youtube.com/watch?v=x47YnULTiro |
| 7 | 2022-08-18 | Ruby+Rust | Awareness feature about to ship, and a little intro to how Ruby+Rust bindings work. | https://www.youtube.com/watch?v=ng20jSo2TIs |
| 6 | 2022-08-05 | Progress & Roadmap | Progress on the work to ship the Awareness widget, and an outlook for what is coming next. | https://www.youtube.com/watch?v=ZSdpj_YxsCE |
| 5 | 2022-07-22 | JTBD and Progress Update | JTBD (Jobs to be done) and Iteration on the Awareness UI | https://www.youtube.com/watch?v=S0s-s45zVv8&list=PL05JrBw4t0KpPmRsaVaDOoWyIp1iKacZo&index=2 |
| 4 | 2022-07-06 | Awareness UI/UX and a CRDT database | Where and how to add the presence indicators on the issue page. A database for CRDTs is usually not necessary, but can become interesting for cases where we need a simple way to fetch the initial document state. | https://www.youtube.com/watch?v=B094U_DvL-s&list=PL05JrBw4t0KpPmRsaVaDOoWyIp1iKacZo&index=3 |
| 3 | 2022-06-22 | The Awareness feature | Technical challenges on rolling out real-time features at scale. Memory and resource concerns and how to mitigate them. | https://www.youtube.com/watch?v=ZPH9tvcPDfc&list=PL05JrBw4t0KpPmRsaVaDOoWyIp1iKacZo&index=4 |
| 2 | 2022-06-09 | Tech design & terminology | A proposal for real-time collboration at GitLab. The presented tech design aims to be a comprehensive guide and peer-reviewed document | https://www.youtube.com/watch?v=crzCYkMk5XQ&list=PL05JrBw4t0KpPmRsaVaDOoWyIp1iKacZo&index=5 |
| 1 | 2022-05-25 | First update | The y-rb Ruby gem and how we are going to use it to bring real-time collaboration to GitLab. | https://www.youtube.com/watch?v=5mTS64Y-rX0&list=PL05JrBw4t0KpPmRsaVaDOoWyIp1iKacZo |