The Tenant Scale group (formerly Pods or Sharding group) is part of the Data Stores stage. We offer support for groups, projects, and user profiles within our product, but our main focus is a long-term horizontal scaling solution for GitLab.
This page covers processes and information specific to the Tenant Scale group. See also the direction page and the features we support per category.
To get in touch with us, it's best to create an issue in the relevant
project (typically GitLab) and add the
~"group::tenant scale" label, along with any other appropriate labels.
For urgent items, feel free to use the Slack channel (internal): #g_tenant-scale.
There are multiple proposals and ideas to increase horizontal scalability via solutions such as database sharding and tenant isolation. The objective of this group is to explore, iterate on, validate, and lead implementation of proposals to provide a solution to accommodate GitLab.com's daily-active user growth.
As we brainstorm and iterate on horizontal scalability proposals, we will provide implementation details, prototypes, metrics, demos, and documentation to support our hypotheses and outcomes.
The executive summary goals for the Tenant Scale group include:
The following people are permanent members of the Tenant Scale group:
The following members of other functional teams are our stable counterparts:
| Person | Role |
|---|---|
| Christina Lohr | Senior Product Manager, Tenant Scale |
| Mike Nichols | Staff Product Designer, Tenant Scale |
| Amelia Bauerly | Senior Product Designer, Tenant Scale |
| Lorena Ciutacu | Technical Writer, Tenant Scale |
Exception Ratio: 1 Distinguished Engineer, multiple Staff Engineers.
Justification: The Tenant Scale group was formed to implement a sharding solution to solve our database limitations as outlined by the Database Scalability Working Group. The project requires deep knowledge of both PostgreSQL and the application itself. As the working group has noted, this problem cannot be solved to meet our needs and requirements if we limit ourselves to the database: we must consider careful changes in the application to make it a reality. Given the complexity of the task, we have carefully crafted a team with a mix of interests and expertise to achieve success. We also anticipate the team will continue to require the expertise of multiple staff+ level engineers to accomplish our horizontal scalability goals.
Where we can we follow the GitLab values and communicate asynchronously. However, there have a few important recurring meetings. Please reach out to the #g_tenant-scale Slack channel if you'd like to be invited.
With the globally distributed nature of this team, it is unlikely that we will have a synchronous meeting where everyone will attend. When we have synchronous meetings we will record the meetings and share written summaries with the links to the recordings. Currently we have the following recurring meetings scheduled.
We follow the GitLab engineering workflow guidelines. To bring an issue to our attention please add the ~"group::pods" label along with any other relevant labels. If it is an urgent issue, please reach out to the Product Manager or Engineering Manager listed in the Stable Counterparts section above.
~"group::pods"
Currently the Tenant Scale group is in the early stages of working out what the finished product will be. The task ahead of us is vast and complex. The only way we are going to be able to break this down and iterate on it is to build proofs-of-concept.
Currently we are in the proof-of-concept stage, in most cases we will need to take them to their logical conclusion. We might reach out to other GitLab teams to help us validate some of the assumptions we have had to make to in order to continue this exploration process. We are not afraid to determine that a proof-of-concept should not be continued. We will take what we learn from the process and apply it to the next one so we can iterate on PoC's in an efficient way.
We have short toes and welcome contributions, we are also very happy to talk about Cells with anyone that is interested. If you have a question, comment or just want generally know what we are up to and how we are going about it feel free to contact us, we'll make time.
We are working on building ourselves a rolling four quarters roadmap to enable transparency about what we are working on. Please visit our planning page to see how this process works.
There are a few factors influencing some of our working agreements here. First, there is a lot of excitement about the goals and expected outcomes of this team. We are working on a really difficult problem, we have lofty expectations as to what we can deliver and many different potential solutions to investigate. Because of the critical nature of what we are going to deliver we need to be able to consistently and concisely describe our current efforts. Second, we are a newly formed team and we need to be explicit about how we work together to ensure success.
Currently we are asking team members to update their issues with:
~pods::active to issues that we know we will be working on in the near future. This helps us to keep our Pods: Build tidy, especially the Open and Closed columnsWe can continue to iterate on our working agreement as a team to ensure that we are a successful team and exceed expectations.
It is expected that we share progress early and often by creating recorded demos. There will be specific requests for demonstration topics, but unscheduled demonstrations are welcome too. When recording a demo it does not need to be polished or perfect. Guidelines below:
We should be aiming for sharing demos on a weekly basis.
There will be many opportunities for us to undertake proofs of concept (POC). When we do, we agree to timebox the POC to 4 weeks and ideally to line up with milestone boundaries. When we need to make exceptions to this agreement we will employ multimodal communication to ensure all stakeholders are properly apprised.
In the early phases of our work we will create implementation plans for review. As we create these implementation plans we will list them below.
We use Ruby on Rails to build our product backed by a PostgreSQL database. You'll be a great fit for the team if you have strong database performance and scaling experience. Tenant Scale group members have the chance to put their mark on the end product and how it will be implemented as well as being at the forefront of building the solution.
Our goal is to ensure all candidates we hire are in the best position to make fantastic contributions to GitLab, using their technical knowledge and creative expertise in solving problems.
We hold the bar high in hiring quality engineers but also understand the interview process can feel like a long process. Below are the stages for interviewing for Tenant Scale group that balances the need for hiring for success and expediating the process.
We aim to set up and book in all interviews in at the beginning of the interview process so candidates can see and prepare for what lays ahead for them. Feedback from candidates on the process is welcomed and a channel for that feedback will be provided at the beginning.
(Sisense↗) We also track our backlog of issues, including past due security and infradev issues, and total open System Usability Scale (SUS) impacting issues and bugs.
(Sisense↗) MR Type labels help us report what we're working on to industry analysts in a way that's consistent across the engineering department. The dashboard below shows the trend of MR Types over time and a list of merged MRs.