The following page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features or functionality remain at the sole discretion of GitLab Inc.
Last updated: 2022-09-27
| Section | Stage | Maturity |
|---|---|---|
| Dev | Create | Loveable |
The Source Code Management direction page belongs to the Source Code group of the Create stage, and is maintained by Torsten Linz, who can be reached at tlinz@gitlab.com.
This direction is a work in progress, and everyone can contribute. Sharing your feedback directly is the best way to contribute to our strategy and vision. Please, comment and contribute in the linked issues and epics or raise an issue yourself in gitlab-org/gitlab if you don't find existing feedback that matches your thoughts.
Source code management (SCM) is the foundation of an organization's software development practice.
Building great software depends on teams working well together. Teams can rarely be divided into areas of complete independence. As cross-functional security, compliance, and growth teams are formed, or new services and libraries are created, effective coordination and collaboration are essential.
To achieve this, teams need to protect production while making it easy for everyone to contribute. Source Code Management provides the controls to define the rules and workflows for this purpose:
While the principles have been around for half a century, SCM is not without challenges. SCM needs to address a large set of requirements that are partly contradicting:
Git is the leading Version Control System (VCS) and is loved by developers. It excels at tracking changes in source code and makes it easy and transparent to merge changes from different developers into one code base. GitLab's source code management builds on top of Git adding functionality that aims to address the above requirements. For example, access control to code repositories or requiring code reviews before merging changes. Source code management and the create stage represent the most popular features in GitLab.
Yet, despite their appeal, neither Git nor GitLab SCM are perfect. Here are the current main shortcomings:
Therefore, the vision for Source Code Management at GitLab consists of three pillars:
Note: SCM is not only the most used function in GitLab, but also the one with the longest history as it has been there from the beginning. As a result, we get a lot of feedback and have a long backlog of issues. Therefore, we need to spend a considerable share of our teams’ capacity on issues that are not in the center of this vision but address bugs, stability, security, and scalability to keep our users and customers happy.
Source code management mainly targets software engineers but also anyone who is contributing to any types of project. To that end, we target all the user personas we describe in our handbook, with a special focus on the following:
Sasha (Software Developer): targets full time contributors to all types of projects (commercial, OSS, data science, etc.). These users expect and need a high level of reliability and speed in their interactions with both project files and Git.
Delaney (Development Team Lead): targets users who often times have elevated roles which allow for the management of project settings, such as access control, security, commit strategies, and mirroring.
Devon (DevOps Engineer): targets engineers tasked with supporting and enabling software teams. Their tasks often revolve around platform creation and maintenance, where GitOps] workflows are crucial.
The Source Code Management category is expansive and encompasses a broad set of features. Which features are leveraged, how they are leveraged, and to what extent depends on the size of development team and the complexity of the product that they are building. We find that as the team size grows, as does the complexity of the software product. For these reasons, it is challenging to define a single user journey that captures how our users move through Source Code. That being said, there are five main buckets we can use to group the jobs to be done of our users. Understanding this general workflow helps to focus product development and discovery when exploring how to streamline the Source Code experience
We intend to define and document the user journeys our specific personas take and how those differ based on the size of the development and the industry in which they work.
The following initiatives are currently in validation as we refine the problem to solve and develop solutions.
Report number of lines per language in repository charts
Currently, in repository analytics, we report the used programming languages only as relative percentages and base these percentages on the file sizes in bytes. As the size of files depends largely on the programming style, this is not very helpful. Also, it is not comparable to nor aggregatable with other work that is usually reported in lines of code. We have received tons of great feedback in the comments of this issue and our analysis of these now focuses on understanding which personas have which specific needs to address the problem(s) in the right way.
Large repositories are difficult to manage in Git because they are difficult to scale, have performance concerns, require large amounts of storage, and suffer from loss of per-service semantic versioning. We see large repositories in situations where a company has a lot of large files, such as binary assets or they are operating monolithic repository, more commonly known as monorepos. Monorepos are a software development approach where code for many projects is stored in a single repository. Monorepos provide several advantages such as reduced complexity, code reuse, easier collaboration amongst teams, and streamlined dependency management. However, they have several drawbacks as aforementioned. We have the opportunity to improve workflows for large repos via improvements to partial clone and sparse checkout. WIP opportunity canvas can be found here - only accessible to internal GitLab team members.
Since the second half of 2021, we placed special emphasis on strengthening our SaaS offering by focusing on ensuring feature parity with our market-leading self-managed offering. For the Source Code group, this meant delivering solutions that are scalable, performant, and secure. Therefore, until the end of Q1 2022 (14.8), we focused our attention to issues of these types:
Infradev: Protect GitLab.com's availability from infrastructure failure.
Performance: Ensure GitLab.com performs well at scale as well as provides a great developer experience on key workflows and actions. Focus on resolving existing failure scenarios and technical debt.
Security: Ensure GitLab.com and self-managed GitLab instances are secure.
Since the beginning of Q2 2022, we started returning to feature work to address larger topics and resolving individual issues raised by users and customers. However, we continue to spend a portion of our capacity on the above-mentioned bug types to assure our SaaS offering continues to be secure and performant.
Here are some of the features that we recently shipped:
FIPS 140-2 Compliant - FIPS compliance is a requirement for the US Govt to utilize a piece of software. It is required for any FISMA or FedRAMP system and cannot be waived. For GitLab to be directly usable within the US Govt, we need to be compliant. To contribute to GitLab's goal to meet these demands, the Source Code group contributed their share of work.
Ability to enforce IP address restrictions for Git over SSH - Limiting access to requests from a trusted set of IP addresses may improve security. Until now, only the API and UI supported such access restrictions; SSH access was blocked entirely. SSH now also adheres to this restriction, and grants access only to requests coming from IP addresses in your list.
Improved and faster file browsing and syntax highlighting - We made multiple changes to improve the experience of browsing a repository and viewing source files. Rather than reloading the full page when moving from folder to folder, we now display repository files and folders with a single VUE application. Our repository tree view is more performant and stable now and working with files is faster. We have also switched from server-side transformations for files to client-side syntax highlighting. The rendering time for large files is now up to 66% faster.
Here are some of the customer/user issues that we recently addressed:
Block Git access protocols at group level - To improve security, you can now block Git access protocols that you don’t use at the group level. This is similar to the GitLab administrator setting, but can now be set per group. By default, both HTTP(S) and SSH are enabled. In your group’s Settings > General > Permissions, scroll to Enable git access protocols and remove any protocols you don’t use.
Gitlab API for Protected tags: use group_id or user_id to protect tags - Enhancement to the Gitlab API for Protected tags to support setting protected tags using group_id or user_id like it is possible for protected environments API and protected branches API.
Project Creation Using Group Template Defaults Path to Template Path - Addressing a bug that led to newly created projects to land in the template sub-group when created from a template.
MVC: view source code rules organized by branches - We are bringing the settings for protected branches and merge request approvals conveniently in one single place under Settings > Repository > Branch rules. Moreover, as the name suggests, they will be viewed through the lens of a branch as they naturally relate to such. While no functionality will be changed this significantly improves discoverability and intelligibility of your settings. Previously, these settings were in far apart places making them hard to understand as we learned from user feedback including a recent SUS study. This is the first iteration of this work and will allow viewing only. To change the settings you will need to use the existing settings as you know them. Next, we intend to allow the editing of rules in this single place that organizes the rules by branches.
GitLab is working on making GitLab.com FedRamp compliant and the source code team it contributing its share of work (internal link).
We have missed our group's availability targets of 99.95% every month since April. (Worst case was in June with 99.87%.) The root cause has been identified to be a recurring CPU saturation at peak times in underlying infrastructure. It needs to be fixed on multiple levels by the source code team, the code review team and the scalability team. The work is expected to finalize around 15.4. By then it is expected that the availability hits the targets again.
Next to such bigger efforts, we also spend capacity on smaller issues raised by customers as we have a long list of such and want to start burning these downs after having focused on GitLab.com stability and security for a long time.
As mentioned above, we also continue to spend a significant portion of our capacity on scalability of GitLab.com, performance and security to keep up the good level that we have achieved after massive work in 2021 and early 2022 in these domains.
In our milestone planning issues under our prioritization & Planning epic you find a detailed view of work done in each iteration. In the same epic, you also find our working list for prioritization.
Framework for Source Code Rules
GitLab offers a number of controls that can be implemented as safeguards. These controls such as protected branches, approval rules, code owners' approvals and many more can be put in place to enforce adherence to policies that ensure quality. A recent SUS study and other feedback suggest that users struggle finding the different controls and struggle understanding which settings interact how. Framework for Source Code Rules aims at making it easier and more intuitive to administer your Source Code management tool. To do so we are bringing the different controls together into one single place and will organize them on a per-branch basis.
The MVC: view source code rules organized by branches is currently being worked on. It enables viewing only.
After its release, we intend to add the ability to also edit the rules in this new branch-view. Beyond this, we intend to also move security approvals under branch rules.
Long-term we will consider also moving settings under per-branch view that currently don't relate to branches but might benefit from attaching them to branches such as status checks or merge checks.
There are different ways for development teams to set up merge strategies depending on their software development, code review, and compliance practices. While we encourage customers practice GitLab Flow, we also need to support different strategies as there is no one right way for our customers. This effort streamlines the settings that allow users to create different flows and adds the option.
Improvements to Code owners
Large organizations with many projects and large projects need to enforce review policies so that they can ensure the correct teams and individuals review changes that impact them. File owners will be automatically added to related Merge Requests (separate feature), but it is also necessary to add controls to prevent changes directly to important branches without approval.
We should continue to improve on the first iteration of code owners. WIP opportunity canvas can be found here - only accessible to internal GitLab team members.
Improvements to the list of commits
The Source Code group is not investing in the following opportunities in the immediate future:
Limiting which branches a user can read in a Git repository is possible in a basic sense, by only advertising a subset of refs, but it is not possible to guarantee that unreachable objects will not be sent to the client. This means that branch read access controls would be very weak, since they could not prevent exfiltration of data they do not have permission to read.
Path-level read access controls
From a commit, Git expects all trees and blobs to be reachable. Although Git supports partial clone and spares checkout, which allow data to be excluded from fetch and checkout, Git expects to be able to fetch missing objects on demand. Deliberately excluding objects by path is likely to cause unexpected failures.
This category is currently at the Loveable maturity level (see our definitions of maturity levels).
However, specific aspects are not yet loveable:
The primary performance indicator (PI) for our group is the number of unique users writing to a project Git repository. We want to ensure all the features in our group provide a great experience that ultimately will allow everyone to contribute more often. A great experience in our group is a critical starting point for this.
Aligning with our SaaS first and product depth direction, we are also working to make performance our secondary indicator (see related issue). The intent here is to track the most heavily used services in our group and track how they improve over time. We firmly believe speed is the killer feature and as such will work to provide a speedy experience to set a great stage for new and existing users.
See more detail in the Create:Source Code PI page section.
The Source Code category of GitLab offers the features where the creative process begins. Here authors will not only consume existing project contents but will also author new content that will eventually move through the DevOps lifecycle. Additionally, many of the features in Source Code are consumed in the Code Review stage of the software developement lifecycle. Consider the following examples:
Because of this close relationship, the Source Code Management group must work closely with the Code Review group in order to ensure the developer experience is cohesive and efficient. This experience is core to providing a great first impression for users.
Important competitors are GitHub and Perforce, and increasingly Azure DevOps.
For public open source projects, GitHub is our primary competitor, with millions of active users having chosen GitHub before the first version of GitLab ever existed.
In most source code management capabilities, GitLab compares favorably to GitHub, the most notable exception being the maturity of forking workflows which GitHub pioneered. GitHub has a highly polished and fast product, which makes tasks like browsing and managing projects fast and easy.
For users of SVN (Apache Subversion) intending to migrate to Git, GitHub is a significant competitor, particularly because GitHub supports hosting SVN repositories.
Perforce competes with GitLab primarily on its ability to support enormous repositories, however, Perforce also competes on the basis of being a Centralized Version Control System. This means that Perforce not only supports granular write permissions, but granular read permissions on a branch and file path basis. While fine grained read permissions are important to some customers, large monolithic repositories may be split into smaller repositories allowing read controls and easier management.
Large file support (see Gitaly direction) is an ongoing area of interest because it blocks certain segments of software development from using Git.
Similarly, extremely large repository support (see Gitaly direction) is also an area of interest for the same reason.