Create:Code Review BE Team

Team Vision & Mission

Please refer to the category directions for Code Review Workflow and GitLab CLI for the current strategy and 1 year plan for these products.

Core Responsibilities

The Create:Code Review BE team is responsible for all backend aspects of the product categories that fall under the Code Review group of the Create stage of the DevOps lifecycle:

• Code Review Workflow, mainly the Merge Requests features
• GitLab CLI

Group Members

The following people are permanent members of the Create:Code Review Group:

Name	Role
François Rosé	Engineering Manager, Create:Code Review
Sincheol (David) Kim	Senior Backend Engineer, Create:Code Review
Gary Holtz	Backend Engineer, Create:Code Review
Kerri Miller	Staff Backend Engineer, Create:Code Review
Marc Shaw	Senior Backend Engineer, Create:Code Review
Patrick Bajao	Senior Backend Engineer, Create:Code Review

Stable Counterparts

The following members of other functional teams are our stable counterparts:

Name	Role
Amy Qualls	Senior Technical Writer, Create (Editor Extensions, Code Review)
André Luís	Frontend Engineering Manager, Create:Source Code, Create:Code Review, Delivery & Scalability
Kai Armstrong	Principal Product Manager, Create:Code Review
Phil Hughes	Staff Fullstack Engineer, Create:Code Review
Stanislav Lashmanov	Senior Frontend Engineer, Create:Code Review
Thomas Randolph	Senior Frontend Engineer, Create:Code Review

Common Links

• GitLab Team Handle: @code-review-be
• Slack Channel: #g_create_code-review
• Slack Handle: @code_review_be
• Issue Tracker: create-stage/code-review-be

Commonly Monitored Issue Lists

• Planning Issues
• OKRs
• Release Board
• Workflow Board (be sure to add a milestone filter)
• Reliability
• Security
• InfraDev Issues

Metrics and KPIs

These KPIs are a subset of the Development Department Performance Indicators.

You can find our dashboards here:

• Centralized Engineering Metrics
• Development Department Performance Indicators

Team OKRs

Objectives and Key Results (OKRs) help align our team towards what really matters. These happen quarterly, align up through the stage, and are based on company OKRs and the engineering OKR process. We check in on the status of our progress routinely throughout the quarter, at least on a monthly basis, to determine whether we are on track or need to pivot in order to accomplish or change these goals.

For a list of current and past OKRs for our team, use this link.

How We Work

See the work section of the main Code Review page.

Working with our team

As stewards for some of the more high profile features - Merge Request pages, Approval Rules, etc - we receive a large number of inquires and requests for assistance or information about them, as well as the dependent features that it encompasses. We welcome them, as issues or Slack inquires, and we strive to be responsive to these, in the interest of Collaboration and Results, but we also must balance the value of Efficiency.

Our target is to respond to incoming requests within 2 working days, although we will frequently respond more quickly, depending on team member availability, experience, and workload.

To contact the Code Review BE team in GitLab, you can use the @code-review-be handle.

Meeting Schedule

Team members meet monthly and are encouraged to join in order to to interact with peers, solve technical challenges, and discuss freely with each other. You can check for the next meeting on the Code Review Group calendar.

Planning

If you haven’t read the code, you haven’t investigated deeply enough

– Nick Thomas

To assign weights to issues in a future milestone, on every 4th of the development month (or the next working day if it falls on a holiday or weekend), BE engineers look at the list of issues that are set for next milestone. These are assigned by the engineering manager. To weight issues, before the 15th of the month, they should:

1. See if there is already a discussed backend solution/plan or none yet.
2. If the discussed backend solution/plan isn’t that clear, clarify it.
3. If there’s no solution/plan yet, devise one. Doesn’t need to be a detailed solution/plan. Feel free to ask other people to pick their brains.
4. If there’s a need to collaborate with a stable counterpart to devise a solution/plan, add a comment and tag relevant counterparts.
5. Give the issue a weight if there’s none yet or update if the existing weight isn’t appropriate anymore. Leave a comment about why a certain weight is given.
6. It’s strongly encouraged to spend no more than 2 hours per issue. Give it your best guess and move on if you run out of time.
7. Label the issue as ~“workflow::ready for development”.
8. Unassign yourself or keep it assigned if you want to work on the issue.

Follow-up issues

You will begin to collect follow-up issues when you’ve worked on something in a release but have tasks leftover, such as technical debt, feature flag rollouts or removals, or non-blocking work for the issue. For these, you can address them in at least 2 ways:

• Add an appropriate future milestone to the follow-up issue(s) with a weight and good description on the importance of working this issue
• Add the issue(s) to the relevant planning issue

You should generally take on follow-up work that is part of our definition of done, preferably in the same milestone as the original work, or the one immediately following. If this represents a substantial amount of work, bring it to your manager’s attention, as it may affect scheduling decisions.

If there are many follow-up issues, consider creating an epic.

Backend and Frontend issues

Many issues require work on both the backend and frontend, but the weight of that work may not be the same. Since an issue can only have a single weight set on it, we use scoped labels instead when this is the case: ~backend-weight::<number> and ~frontend-weight::<number>.

What to work on

The primary source for things to work on is the Code Review backend issue board for the current iteration cycle (don’t forget to filter by milestone!), which lists all of the Deliverable and Stretch issues scheduled for this cycle in priority order.

The lists are compiled by the Product Manager following the product prioritization process, with input from the team, engineering managers, and other stakeholders. The iteration cycle lasts from the 18th of one month until the 17th of the next, and is identified by the GitLab version set to be released.

There is also the Code Review backend assignment issue board (again, don’t forget to filter by milestone!), which shows the same Deliverable and Stretch issues, now grouped by assignee, with the left-most list listing issues not currently assigned to any backend engineer. On each list, the issues are again ordered by priority.

What to work on first

Deliverables are considered top priority and are expected to be done by the end of the iteration cycle on the 17th, in time for the monthly release.

These top priority issues are assigned to engineers on or ahead of the 18th of the month, when the iteration cycle is kicked off, and it is their responsibility to make a best effort to get them done during that cycle, and to inform their engineering manager if anything is standing in the way of their success. You can find the issues assigned to you on the Code Review backend assignment issue board (again, don’t forget to filter by milestone!).

Many things can happen during a month that can result in a deliverable not actually being completed by the end of a cycle, and while this usually indicates that the engineering manager was too optimistic in their estimation of the issue’s weight, or that an engineer’s other responsibilities ended up taking up more time than expected, this should never come as a surprise to the engineering manager.

The sooner this potential outcome is anticipated and communicated, the more time there is to see if anything can be done to prevent it, like reducing the scope of the deliverable, or finding a different engineer who may have more time to finish a deliverable that hasn’t been started yet. If this outcome cannot be averted and the deliverable ends up missing the cycle, it will simply be moved to the next cycle to be finished up, and the engineer and engineering manager will have a chance to retrospect and learn from what happened.

Generally, your deliverables are expected to take up about 75% of the time you spend working in a month. The other 25% is set aside for other responsibilities (code review, community merge request coaching, helping people out in Slack, participating in discussions in issues, etc), as well as urgent issues that come up during the month and need someone working on them immediately (regressions, security issues, customer issues, etc).

What to work on next

If you have time to spare after finishing your deliverables and other activities, you can spend the remaining time working on Stretch issues, which can also be found on the Code Review backend issue board and Code Review backend assignment issue board (again, don’t forget to filter by milestone!).

These lower priority issues are not expected to be done by the end of the iteration cycle, but are to be Deliverables in the next cycle, so any progress made on them ahead of time is a bonus.

Stretch issues are usually not directly assigned to people, except in cases where there is clearly a most appropriate person to work on them, like in the case of technical debt, bugs related to work someone did recently, or issues someone started on before but hasn’t had a chance to finish yet.

If no Stretch issues are assigned to you yet, you can find new ones to pick up in the left-most list of the Code Review backend assignment issue board (again, don’t forget to filter by milestone!), which lists all issues not currently assigned to any backend engineer. As the issues are ordered by priority, they should be picked up starting at the top. When you assign an issue to yourself to indicate you’re working on it, it will move to your list and out of the left-most unassigned list, and the second issue will rise to the top for other engineers to pick up.

If anything is blocking you from getting started with the top issue immediately, like unanswered questions or unclear requirements, you can skip it and consider a lower priority issue, as long as you put your findings and questions in the issue, so that the next engineer who comes around may find it in a better state.

Instead of picking up Stretch issues, you may also choose to spend any spare time working on anything else that you believe will have a significant positive impact on the product or the company in general. As the general guidelines state, “we recognize that inspiration is perishable, so if you’re enthusiastic about something that generates great results in relatively little time feel free to work on that.”

We expect people to be managers of one and prefer responsibility over rigidity, so there’s no need to ask for permission if you decide to work on something that’s not on the issue board, but please keep your other responsibilities in mind, and make sure that there is an issue, you are assigned to it, and consider sharing it in #g_create_code-review.

Deep Dives

The groups in the Create stage organize regular Deep Dive sessions to share our domain specific knowledge with others in the stage, the organization, and the wider community. All existing Deep Dives can be found on GitLab Unfiltered with slides and recordings in the descriptions. For Create specific Deep Dives, there is a different playlist. To find more information on upcoming sessions, or to propose a new topic, please see the epic.

Performance Monitoring

The Create:Code Review BE team is responsible for keeping some API endpoints and controller actions performant (e.g. below our target speed index).

There are Grafana and Kibana dashboards available that the team can check to ensure we are meeting these targets.

Here are the Kibana dashboards that give a quick overview on how they perform:

• Create::Code Review Controller Actions (internal only)
• Create::Code Review: API Endpoints (internal only)
• Create::Code Review: Sidekiq Workers (internal only)

These tables are filtered by the endpoints and controller actions that the group handles and sorted by P90 (slowest first) for the last 7 days by default.

The Grafana dashboard shows more specific details about each actions and endpoints.

To see the specific details for certain actions/endpoints, it can be filtered by action and/or controller.

Issue identification process

1. Every week we check the kibana and grafana dashboards as per the calendar invite
2. If we identfiy an endpoint or worker that meets our severity criteria we create an issue (if not created already) and label it based on our severity and priority criteria
   • If an issue is already created, check whether the severity/priority needs to be updated
3. This issue should contain as much information on why it is the severity/priority as logs are often held for only 7 days
4. Using a specific performance issue board we can keep track of our current issues and they will be prioritized accordingly during planning

Priority and Severity

More details can be found in the handbook for severity and priority

Calendar Invites

There are calendar invites that act as a reminder for backend team members to check these dashboards weekly on Monday:

• APAC: Check performance dashboards - set on Monday 12AM UTC
• AMER: Check performance dashboards - set on Monday 4PM UTC

All backend team members are invited to appropriate calendar invites. This does not necessarily mean that we only need to take a look at them at those specific times.

If a team member sees that an action or endpoint does not meet our target, they should create a performance issue if there’s no existing one. If there’s an existing one, team member can either update the issue, investigate or work on the issue (if they have spare time).

Responsibilities of Issue Triage

• Do a weekly review of the triage board and make sure these issues are investigated by assigning them to developers or performing the investigation
• Coordinate with Product Manager to get those issues assigned to milestones so that they can be worked on in timely manner
• Check issues in each column in the order of severity. ~missed-SLO -> ~approaching-SLO -> ~severity::1 -> ~severity::2 -> ~severity::3 -> ~severity::4
• Ensure the severity label is accurate based on the problem investigated
• Our aim is getting to 0 issues in ~missed-SLO column eventually and handle issues well before they get labeled as ~missed-SLO and ~approaching-SLO

Engineering Onboarding

This example onboarding issue will eventually be turned into an onboarding issue template.

Other Related Pages

• Create:Code Review BE Engineer Resources, e.g. team building and career development
• Create:Code Review BE Engineering Manager Responsibilities, e.g. milestone planning, talent assessment, and project management

View page source - Edit this page - please contribute.