The following page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features or functionality remain at the sole discretion of GitLab Inc.
Last updated: 2021-09-27
Please reach out to Yannis Roussos, Product Manager for the Memory group (Email) if you'd like to provide feedback or ask any questions related to this product category.
This strategy is a work in progress, and everyone can contribute. Please comment and contribute in the linked issues and epics on this page. Sharing your feedback directly on GitLab.com is the best way to contribute to our strategy and vision.
GitLab should be as lighweight as possible and perform at any scale. GitLab is used on Raspberry Pis for only a few users, large organizations with thousands of users and up to the scale of GitLab.com. The Memory team is responsible for application-wide concerns that impact the availability, reliability and performance of GitLab at any scale. The Memory team analyzes the behavior of GitLab, for example with regards to memory consumption, recognizes bottlenecks, proposes changes to the application and works on overall improvements to ensure that GitLab works well for self-hosted customers and our SaaS offering alike.
The Memory team regularly collaborates with teams across GitLab to support high-impact initiatives that reduce memory consumption, change the overall design of the application or helps other teams with the design of new features.
The Memory team focuses on the MVC to iterate as fast as possible, strict prioritization on only the highest-impact areas and clean handover to other teams. Learn more about how the team works.
The Scalability team is primarily concerned with the performance and availability of GitLab.com and focuses on helping other teams become better at scalability. The Memory Group is concerned with application-wide concerns at any scale, ranging from very small installations to GitLab.com.
The Memory team's short term goals are to continuously identify the main bottlenecks that affect GitLab's performance and availability and address those - either as a stand-alone project or in collaboration with the teams that are directly responsible for that area.
In medium to long term the Memory team wants to provide better tooling, metrics and visibility to development teams that allow them to make better data-driven decisions when developing new features or iterating on existing ones. Using this approach, we want to shift performance, memory and application issues further left so that they can be addressed before being implemented. This will enable GitLab itself to become more proactive during development. Ultimately, these features should be integrated into GitLab itself so that our users can benefit from the same tooling.
GitLab as an application relies heavily on Redis to provide shared state, cluster-wide atomic locking, message queueing (for Sidekiq) and caching services to the application.
Our redis-persistent is growing at a steady pace with its memory usage approaching saturation in a September incident. We also witnessed a noticeable increase in CPU usage on redis-cache when RackAttack rate-limiting was enabled.
The Memory Team will switch its focus starting in 14.4 and for a few milestones to helping the Scalability Team address the top Redis related scalability priorities: Create a dedicated Redis (Cache) instance for Rate Limiting data and Create a dedicated Redis (persistent) instance for session keys
Ruby has three major areas planned for improvements: Performance, Concurrency and Static Analysis. The Memory Team is driving the process of updating the supported Ruby version to 3.0, but we have to pause the effort starting in 14.4 to focus on the Redis improvements. We will resume the process once the initial work on the Redis related scalability improvements has been concluded.
As we progress towards updating the supported Ruby version to 3.0 we will identify opportunities to optimize our adoption of Ruby 3.0 to make the best use of these performance and scalability improvements. This may include implementation changes, tooling and/or documentation to provide best practices for our development community.
The Memory team is currently evaluating the following major initiatives for prioritization:
Split the application into functional parts to ensure that only needed code is loaded with all dependencies, also referred to as Composable Codebase. This initiative will allow for reduction of resources as well as creating new points of scalability going forward.
We have completed an initial investigation and a POC and we plan to evaluate how to move forward with implementing and releasing the changes in the application.
When Rails starts, whether it be the app, a console, etc, it takes nearly 1 minute to start. When used with other wrapper scripts such as those used to power the application in Docker Containers, only adds to this amount of time. Recent improvements, such as the use of Bootsnap, have made Kubernetes start up times quicker, though those improvements only impact our Docker images. And, due to the use of multiple initContainers, many of which require the Rails application to start, add up to lengthy start time in Kubernetes. This has the potential to lead to large issues.
You can check the related issue for more information.
Object storage is widely used in Gitlab's application layer. We want to focus on improving our object storage support as any code that interacts with object storage can affect the performance of the application or even affect the availability of specific GitLab features.
GitLab is a complete solution for DevOps and more features are added frequently to address missing capabilities. This often means increases in memory consumption, which makes it harder to run GitLab on small instances with small amounts of RAM. Currently, the minimum required to run GitLab is 4GB; we want to reduce this as much as possible or at least keep the minimum requirements as low as possible while new features are added to GitLab.
We have concluded the initial implementation of changes that reduce the overall memory footprint of the application. We plan to re-evaluate the minimum resource requirements for Gitlab and then continue our efforts on implementing memory improvements in memory constrained environments. Those updates are essential for small instances but can also benefit the performance of larger instances as well.
We want to make data-driven decisions and that requires sharp tooling to understand memory consumption and other performance related bottlenecks.
At the same time, this will allow us to dogfood the implementation of tools and metrics and can lead us to enabling other development teams as well in making better data-driven decisions when developing new features or iterating on existing ones.
