GitLab might move to a single Rails codebase

Feb 21, 2019 · 6 min read
Marin Jankovski GitLab profile

A single repository with no license changes

Before we go into the details of the proposed changes, we want to stress that:

What are the challenges with having two repositories?

Currently the Ruby on Rails code of GitLab (the majority of the codebase) are maintained in two repositories. The gitlab-ce repository for the code with an open source license and the gitlab-ee repository containing code with a proprietary license which is source available.

Feature development is difficult and error prone when making any change at GitLab in two similar yet separate repositories that depend on one another.

Below are a few examples to demonstrate the problem:

Duplicated work during feature development

This frontend only Merge Request required a backport to CE repository. Backporting included creating duplicate work to avoid future conflicts as well as changes to the code to support the feature.

A simple change can break master

A simple change in a spec in CE repository failed the pipeline in the master branch. After hours of investigation, an MR reverting the change was created, as well as a second to address the problem.

Conflicts during preparation for regular releases

This concerns preparation for a regular release, e.g. 11.7.5 release. Merge requests preparing the release for both the CE repository and EE repository need to be created and once the pipelines pass, the EE repository requires a merge from the CE repository. This causes additional conflicts, pipeline failures, and similar delays requiring more manual intervention during which the CE distribution release is also delayed.

Between these three examples, days of engineering time has been spent on busy work, delaying the delivery of work that brings actual value. Only three examples are highlighted, but this type of work occurs daily. Whether writing a new feature available in Core, or any of the enterprise plans, all are equally affected.

More details on the workflows and challenges can be found in the working in CE and EE codebases blueprint document.

What have we done to improve the situation?

We've invested significant development time to try and keep the two repositories separate:

Pre-2016: Manual merges for each release

Prior to 2016, merging the CE repository into the EE repository was done when we were ready to cut a release; the number of commits was small so this could be done by one person.

2016-2017: Daily merges by a team of developers

In 2016, the number of commits between the two repositories grew so the task was divided between seven (brave) developers responsible for merging the code once a day. This worked for a while until delays started happening due to failed specs or difficult merge conflicts.

2017-2018: Automated merges every three hours

At the end of 2017, we merged an MR that allowed the creation of automated MRs between the two repositories, mentioning individuals to resolve conflicts. This task ran every three hours, allowing for a smaller number of commits to be worked on. You can read more about our automated CE to EE merge here.

Present: Further automation with Merge Train

By the end of 2018, the number of changes going into both the CE and EE repositories grew to thousands of commits in some cases, which made the automated MR insufficient. The Merge Train tool was created to automate these workflows further, by automatically rejecting merge conflicts and preferring changes from one repository over the other. The edge cases we've encountered are requiring us to invest additional time in improving the custom tool.

This last attempt turned out to be a bit of a crossroads. Do we invest more development time in improving the custom tooling, knowing that we will never get it 100 percent right, or do we need to take some more drastic measures that are going to save countless hours of development time?

What are we proposing?

One of GitLab's core values is efficiency. As previously mentioned, merging the gitlab-ce Rails repository into the gitlab-ee Rails repository is proving to be inefficient.

The Rails repository is one of many base repositories of which GitLab consists. The gitlab-ce repository is a part of a gitlab-ce distribution package which offers only the Core feature set. Similarly, the gitlab-ee repository is part of a gitlab-ee distribution package which has a larger feature set available. See the image below:


The change we are proposing would merge the gitlab-ce and gitlab-ee repositories into a single gitlab repository. This change is reflected below:


The design for merging two codebases outlines the required work and process changes in detail. The proposed change would pertain only to the Ruby on Rails repository, and I've summarized it below.

So, what changes?

What remains unchanged?

What are the possible downsides?

We want to be clear about the possible downsides of this approach:

What's next?

We currently think that the efficiency gains and clearer naming outweighs these disadvantages. Our stewardship of GitLab is an important aspect of GitLab's success as a whole, so we would love to know:

We invite you to share your suggestions in issue 2952, which was an inspiration for the proposal as it currently stands. We look forward to hearing your thoughts!

Cover image from Unsplash

“.@gitlab might move to a single @rails codebase – here's why” – Marin Jankovski

Click to tweet

Edit this page View source