The Pods Group (formerly Sharding group) is focused on a long-term horizontal scaling solution for GitLab. There are multiple proposals and ideas to increase horizontal scalability via solutions such as database sharding and tenant isolation. The charter of this group is to explore, iterate on, validate, and lead implementation of proposals to provide a solution to accommodate Gitlab.com's daily-active user growth.
As we brainstorm and iterate on horizontal scalability proposals, we will provide implementation details, prototypes, metrics, demos and documentation to support our hypotheses and outcomes.
Note: As we iterate towards a rename of the Sharding group to Pods, this page will contain references to both Pods and Sharding.
The executive summary goals for the Pods Group include:
The following people are permanent members of the Pods Group:
| Person | Role |
|---|---|
| Barry Firman | Backend Engineering Manager, Pods |
| Omar Qunsul | Backend Engineer, Pods |
| Rutger Wessels | Senior Backend Engineer |
| Thong Kuah | Staff Backend Engineer, Pods |
The following members of other functional teams are our stable counterparts:
| Person | Role |
|---|---|
| Fabian Zimmer | Group Product Manager, Enablement |
We use Ruby on Rails to build our product backed by a PostgreSQL database. The Pods Group currently has open positions for Ruby Backend Engineers, particularly those with strong database performance and scaling experience.
This is a particularly exciting time to be joining the Group. We are at the beginning of our journey thinking about and creating proofs-of-concept, determining and demonstrating the feasibility and value to the customer of our ideas. Essentially a greenfield initiative in GitLab, Pods Group members have the chance to put their mark on the end product and how it will be implemented as well as being at the forefront of building the solution.
Our goal is to ensure all candidates we hire are in the best position to make fantastic contributions to GitLab, using their technical knowledge and creative expertise in solving problems.
We hold the bar high in hiring quality engineers but also understand the interview process can feel like a long process. Below are the stages for interviewing for Pods Group that balances the need for hiring for success and expediating the process.
We aim to set up and book in all interviews in at the beginning of the interview process so candidates can see and prepare for what lays ahead for them. Feedback from candidates on the process is welcomed and a channel for that feedback will be provided at the beginning.
Exception Ratio: 1 Distinguished Engineer, multiple Staff Engineers : Team
Justification: The Pods team was formed (originally as the Sharding team) to implement a sharding solution to solve our database limitations as outlined by the Database Scalability Working Group. The project requires in-depth knowledge of PostgreSQL as well as into the application itself. To borrow some phrasing from the working group: we recognize this problem cannot be solved to meet our needs and requirements if we limit ourselves to the database: we must consider careful changes in the application to make it a reality. With that being said, the DRI had to be selective to ensure a small team of engineers can achieve the above requirements to cover every corner of our database and codebase. All candidates were selected based on the required skills profile for the success of the project, to make the best mix and match by covering a spectrum of interests with a minimal group. There have also been multiple Staff engineers borrowed from other teams at various times to execute on these goals. Going forward, we anticipate the team will continue to require the expertise of multiple staff+ level engineers to accomplish our horizontal scalability goals.
Where we can we follow the GitLab values and communicate asynchronously. However, there have a few important recurring meetings. Please reach out to the #g_pods Slack channel if you'd like to be invited.
With the globally distributed nature of this team, it is unlikely that we will have a synchronous meeting where everyone will attend. When we have synchronous meetings we will record the meetings and share written summaries with the links to the recordings. Currently we have the following recurring meetings scheduled.
We follow the GitLab engineering workflow guidelines. To bring an issue to our attention please add the ~"group::pods" label along with any other relevant labels. If it is an urgent issue, please reach out to the Product Manager or Engineering Manager listed in the Stable Counterparts section above.
~"group::pods"
Currently the Pods Group is in the early stages of working out what the finished product will be. The task ahead of us is vast and complex. The only way we are going to be able to break this down and iterate on it is to build proofs-of-concept.
Currently we are in the proof-of-concept stage, in most cases we will need to take them to their logical conclusion. We might reach out to other GitLab teams to help us validate some of the assumptions we have had to make to in order to continue this exploration process. We are not afraid to determine that a proof-of-concept should not be continued. We will take what we learn from the process and apply it to the next one so we can iterate on PoC's in an efficient way.
We have short toes and welcome contributions, we are also very happy to talk about Pods with anyone that is interested. If you have a question, comment or just want generally know what we are up to and how we are going about it feel free to contact us, we'll make time.
Listed below are the pieces of work that are top of mind for the Pods Group, whilst not exclusively what we are working they give an indication of the direction the team are going in:
| Epics |
|---|
| Phase 8: Clean Up Decomposition |
| Decomposition: Support for self-managed customers |
| Pods - Proof of Concepts |
There are a few factors influencing some of our working agreements here. First, there is a lot of excitement about the goals and expected outcomes of this team. We are working on a really difficult problem, we have lofty expectations as to what we can deliver and many different potential solutions to investigate. Because of the critical nature of what we are going to deliver we need to be able to consistently and concisely describe our current efforts. Second, we are a newly formed team and we need to be explicit about how we work together to ensure success.
Currently we are asking team members to update their issues with:
~pods::active to issues that we know we will be working on in the near future. This helps us to keep our Pods: Build tidy, especially the Open and Closed columnsWe can continue to iterate on our working agreement as a team to ensure that we are a successful team and exceed expectations.
It is expected that we share progress early and often by creating recorded demos. There will be specific requests for demonstration topics, but unscheduled demonstrations are welcome too. When recording a demo it does not need to be polished or perfect. Guidelines below:
We should be aiming for sharing demos on a weekly basis.
There will be many opportunities for us to undertake proofs of concept (POC). When we do, we agree to timebox the POC to 4 weeks and ideally to line up with milestone boundaries. When we need to make exceptions to this agreement we will employ multimodal communication to ensure all stakeholders are properly apprised.
In the early phases of our work we will create implementation plans for review. As we create these implementation plans we will list them below.
(Sisense↗) We also track our backlog of issues, including past due security and infradev issues, and total open System Usability Scale (SUS) impacting issues and bugs.
(Sisense↗) MR Type labels help us report what we're working on to industry analysts in a way that's consistent across the engineering department. The dashboard below shows the trend of MR Types over time and a list of merged MRs.