The Gitaly team is responsible for building and maintaining systems to ensure that the git data storage tier of GitLab instances, and GitLab.com in particular, is reliable, secure and fast. For more information about Gitaly, see our Direction page.
Provide a durable, performant, and reliable Git storage layer for GitLab.
The following people are permanent members of the Gitaly Team:
| Person | Role |
|---|---|
| Tim Zallmann | Senior Director of Engineering, Dev |
| Andras Horvath | Manager, Engineering, Gitaly |
The following members of other functional teams are our stable counterparts:
| Person | Role |
|---|---|
| Mark Wood | Senior Product Manager, Plan:Certify & Create:Gitaly |
| John McDonnell | Senior Software Engineer in Test, Create:Gitaly |
Gitaly team members do not carry pagers, but we live around the world and there's a good chance that someone is available during their working hours. There is no coverage for weekends; instead, we strive to empower incident responders to mitigate any circumstance.
These issues relate to ongoing production outages or similar. They interrupt our process used to schedule work and get attention as soon as possible. Please only interrupt us sparingly, in these cases:
Getting attention to an urgent, interrupting issue
@gl-gitaly (the whole team) on the issue.We use the standard engineering workflow to schedule work. To get Gitaly team
work on something, it's best to create an issue on the Gitaly issue tracker
and add the ~"group::gitaly" and ~"workflow::problem validation" labels,
along with any other appropriate labels. Then, feel free to tag the relevant
Product Manager and/or Engineering Manager as listed above.
For information requests and other quick one-offs, feel free to use #g_create_gitaly on Slack to get attention on the issue.
Infradev labelsThese are typically Corrective Actions or other followup items that have strict SLO tracking. They will be scheduled through either of the above paths, by EM and/or PM polling these dashboards:
A weekly call is held between the product manager and engineering manager, which is listed in the "Gitaly Team" calendar. Everyone is welcome to join and these calls are used to discuss any roadblocks, concerns, status updates, deliverables, or other thoughts that impact the group.
To have a constant communication flow about planned changes, updates and maybe breaking changes we have the #g_create_gitaly_consumers Slack channel. In the channel we will provide updates for all teams using the service but also ask for assistance to provide feedback and insights about planned changes or improvements.
To support this pro-active communication additionally there is also an individual counterpart on the consumer side to help with research in the codebases and coordination with all the teams consuming Gitaly. The DRI on Consumer side is Igor Drozdov.
The Gitaly consumers are:
We use a lightweight system of issue weighting to help with capacity planning. These weights help us ensure that the amount of scheduled work in a cycle is reasonable, both for the team as a whole and for each individual. The "weight budget" for a given cycle is determined based on the team's recent output, as well as the upcoming availability of each engineer.
Since things take longer than you think, it's OK if an issue takes longer than the weight indicates. The weights are intended to be used in aggregate, and what takes one person a day might take another person a week, depending on their level of background knowledge about the issue. That's explicitly OK and expected. We should strive to be accurate, but understand that they are estimates! Change the weight if it is not accurate or if the issue becomes harder than originally expected. Leave a comment indicating why the weight was changed and tag your EM so that we can better understand weighting and continue to improve.
The weights we use are:
| Weight | Description |
|---|---|
| 1: Trivial | The problem is very well understood, no extra investigation is required, the exact solution is already known and just needs to be implemented, no surprises are expected, and no coordination with other teams or people is required. Examples are documentation updates, simple regressions, and other bugs that have already been investigated and discussed and can be fixed with a few lines of code, or technical debt that we know exactly how to address, but just haven't found time for yet. |
| 2: Small | The problem is well understood and a solution is outlined, but a little bit of extra investigation will probably still be required to realize the solution. Few surprises are expected, if any, and no coordination with other teams or people is required. Examples are simple features, like a new API endpoint to expose existing data or functionality, or regular bugs or performance issues where some investigation has already taken place. |
| 3: Medium | Features that are well understood and relatively straightforward. A solution will be outlined, and most edge cases will be considered, but some extra investigation will be required to realize the solution. Some surprises are expected, and coordination with other teams or people may be required. Bugs that are relatively poorly understood and may not yet have a suggested solution. Significant investigation will definitely be required, but the expectation is that once the problem is found, a solution should be relatively straightforward. Examples are regular features, potentially with a backend and frontend component, or most bugs or performance issues. |
| 5: Large | Features that are well understood, but known to be hard. A solution will be outlined, and major edge cases will be considered, but extra investigation will definitely be required to realize the solution. Many surprises are expected, and coordination with other teams or people is likely required. Bugs that are very poorly understood, and will not have a suggested solution. Significant investigation will be required, and once the problem is found, a solution may not be straightforward. Examples are large features with a backend and frontend component, or bugs or performance issues that have seen some initial investigation but have not yet been reproduced or otherwise "figured out". |
Anything larger than 5 should be broken down if possible.
Security issues are typically weighted one level higher than they would normally appear from the table above. This is to account for the extra rigor of the security release process. In particular, the fix usually needs more-careful consideration, and must also be backported across several releases.
At the beginning of each release, the Gitaly EM will create a retrospective issue to collect discussion items during the release. The first weekly Gitaly meeting after the 18th that issue will be used to discuss what was brought up.
Additionally, a weekly review is conducted by the Geekbot Slack integration. Each
member of the #gitaly-standup channel on Slack will be prompted to fill out the
weekly retrospective to review what you've learned and what can be improved.
When a new developer joins Gitaly, their responsibility will include maintaining
the Gitaly project from their first day. This means that the developer will get
Maintainer access to the repository and will be added to the
gitlab.com/gl-gitaly group so they appear in merge request approval group.
Career development conversations in the Create:Gitaly BE team are centered around a Career Development Sheet that is based on the Engineering Career Matrix for Individual Contributors. The sheet lists the expected current level behaviors on the left, the next level behaviors on the right, and uses colored columns in between to visually represent the extent to which the individual has shown to have grown from the current level to the next. Columns to the right of a next level behavior are used to collect specific examples of that behavior, which serve as evidence of the individual's growth.
Both the matrix and the sheet are Works in Progress; the development of the career matrix is tracked in an epic, and as the matrix evolves, the sheet will be updated to match.
Maintainer rights are revoked, and to remove the developer from the list of
authorized approvers, remove them from the gl-gitaly GitLab.com group.