Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Category Strategy - Memory

🛠️ Memory group

Last updated: 2021-09-27

Introduction and how you can help

Please reach out to Yannis Roussos, Product Manager for the Memory group (Email) if you'd like to provide feedback or ask any questions related to this product category.

This strategy is a work in progress, and everyone can contribute. Please comment and contribute in the linked issues and epics on this page. Sharing your feedback directly on GitLab.com is the best way to contribute to our strategy and vision.

Overview

GitLab should be as lighweight as possible and perform at any scale. GitLab is used on Raspberry Pis for only a few users, large organizations with thousands of users and up to the scale of GitLab.com. The Memory team is responsible for application-wide concerns that impact the availability, reliability and performance of GitLab at any scale. The Memory team analyzes the behavior of GitLab, for example with regards to memory consumption, recognizes bottlenecks, proposes changes to the application and works on overall improvements to ensure that GitLab works well for self-hosted customers and our SaaS offering alike.

The Memory team regularly collaborates with teams across GitLab to support high-impact initiatives that reduce memory consumption, change the overall design of the application or helps other teams with the design of new features.

The Memory team focuses on the MVC to iterate as fast as possible, strict prioritization on only the highest-impact areas and clean handover to other teams. Learn more about how the team works.

Differences to GitLab's Scalability team

The Scalability team is primarily concerned with the performance and availability of GitLab.com and focuses on helping other teams become better at scalability. The Memory Group is concerned with application-wide concerns at any scale, ranging from very small installations to GitLab.com.

Where we are headed

The Memory team's short term goals are to continuously identify the main bottlenecks that affect GitLab's performance and availability and address those - either as a stand-alone project or in collaboration with the teams that are directly responsible for that area.

In medium to long term the Memory team wants to provide better tooling, metrics and visibility to development teams that allow them to make better data-driven decisions when developing new features or iterating on existing ones. Using this approach, we want to shift performance, memory and application issues further left so that they can be addressed before being implemented. This will enable GitLab itself to become more proactive during development. Ultimately, these features should be integrated into GitLab itself so that our users can benefit from the same tooling.

What's Next & Why

Scalability improvements for Redis

GitLab as an application relies heavily on Redis to provide shared state, cluster-wide atomic locking, message queueing (for Sidekiq) and caching services to the application.

Our redis-persistent is growing at a steady pace with its memory usage approaching saturation in a September incident. We also witnessed a noticeable increase in CPU usage on redis-cache when RackAttack rate-limiting was enabled.

The Memory Team will switch its focus starting in 14.4 and for a few milestones to helping the Scalability Team address the top Redis related scalability priorities: Create a dedicated Redis (Cache) instance for Rate Limiting data and Create a dedicated Redis (persistent) instance for session keys

Move GitLab to Ruby 3

Ruby has three major areas planned for improvements: Performance, Concurrency and Static Analysis. The Memory Team is driving the process of updating the supported Ruby version to 3.0, but we have to pause the effort starting in 14.4 to focus on the Redis improvements. We will resume the process once the initial work on the Redis related scalability improvements has been concluded.

As we progress towards updating the supported Ruby version to 3.0 we will identify opportunities to optimize our adoption of Ruby 3.0 to make the best use of these performance and scalability improvements. This may include implementation changes, tooling and/or documentation to provide best practices for our development community.

Medium term

The Memory team is currently evaluating the following major initiatives for prioritization:

Only load needed code

Split the application into functional parts to ensure that only needed code is loaded with all dependencies, also referred to as Composable Codebase. This initiative will allow for reduction of resources as well as creating new points of scalability going forward.

We have completed an initial investigation and a POC and we plan to evaluate how to move forward with implementing and releasing the changes in the application.

Improve the Rails boot-up time

When Rails starts, whether it be the app, a console, etc, it takes nearly 1 minute to start. When used with other wrapper scripts such as those used to power the application in Docker Containers, only adds to this amount of time. Recent improvements, such as the use of Bootsnap, have made Kubernetes start up times quicker, though those improvements only impact our Docker images. And, due to the use of multiple initContainers, many of which require the Rails application to start, add up to lengthy start time in Kubernetes. This has the potential to lead to large issues.

You can check the related issue for more information.

Improve GitLab's object storage backend

Object storage is widely used in Gitlab's application layer. We want to focus on improving our object storage support as any code that interacts with object storage can affect the performance of the application or even affect the availability of specific GitLab features.

Improve how GitLab runs in memory constrained environments.

GitLab is a complete solution for DevOps and more features are added frequently to address missing capabilities. This often means increases in memory consumption, which makes it harder to run GitLab on small instances with small amounts of RAM. Currently, the minimum required to run GitLab is 4GB; we want to reduce this as much as possible or at least keep the minimum requirements as low as possible while new features are added to GitLab.

We have concluded the initial implementation of changes that reduce the overall memory footprint of the application. We plan to re-evaluate the minimum resource requirements for Gitlab and then continue our efforts on implementing memory improvements in memory constrained environments. Those updates are essential for small instances but can also benefit the performance of larger instances as well.

Improve internal tools

We want to make data-driven decisions and that requires sharp tooling to understand memory consumption and other performance related bottlenecks.

At the same time, this will allow us to dogfood the implementation of tools and metrics and can lead us to enabling other development teams as well in making better data-driven decisions when developing new features or iterating on existing ones.

In a year

Metrics

Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license