Engineering Workflow

This document explains the workflow for anyone working with issues in GitLab Inc. For the workflow that applies to everyone please see PROCESS.md.

GitLab Flow

Products at GitLab are built using the GitLab Flow.

Broken master

If you notice that the tests for the master branch of GitLab CE or EE are failing (red) or broken (green as a false positive), fixing this takes priority over everything else development related, since everything we do while tests are broken may break existing functionality, or introduce new bugs and security issues.

Create an issue, post about it in #development so that other developers are aware of the problem and can help. If the problem cannot be fixed by you within a few hours, @mention the relevant Engineering Leads and CTO in the issue and on Slack, so that resources can be assigned to fix it as quickly as possible.

Security issues

If you find or are alerted of a security issue, major or small, fixing this takes priority over everything else development related.

Create a confidential issue mentioning the Security and the relevant Engineering Leads, as well as the VP of Engineering, and post about it in #security.

Basics

Start working on an issue you’re assigned to. If you’re not assigned to any issue, find the issue with the highest priority you can work on, by relevant label. You can use this query, which sorts by priority for the started milestones, and filter by the label for your team.
If you need to schedule something or prioritize it, apply the appropriate labels. See below for details.
You are responsible for the issue that you're assigned to. This means it has to ship with the milestone it's associated with. If you are not able to do this, you have to communicate it early to your manager. In teams, the team is responsible for this (see below).
You (and your team, if applicable) are responsible for:
- ensuring that your changes apply cleanly to GitLab Enterprise Edition.
- the testing of a new feature or fix, especially right after it has been merged and packaged.
Once a release candidate has been deployed to the staging environment, please verify that your changes work as intended. We have seen issues where bugs did not appear in development but showed in production (e.g. due to CE-EE merge issues).

For general guidelines about issues and merge requests, be sure to read the GitLab Workflow.

Working in Teams

For larger issues or issues that contain many different moving parts, you'll be likely working in a team. This team will typically consist of a backend developer, a frontend developer, a UX designer and a product manager.

Teams have a shared responsibility to ship the issue in the planned release.
1. If the team suspects that they might not be able to ship something in time, the team should escalate / inform others as soon as possible. A good start is informing your lead.
2. It's generally preferable to ship a smaller iteration of an issue, than ship something a release later.
Consider starting a Slack channel for a new team, but remember to write all relevant information in the related issue(s). You don't want to have to read up on two threads, rather than only one, and Slack channels are not open to the greater GitLab community.

Choosing something to work on

Start working on things with the highest priority in the current milestone. The priority of items are defined under labels in the repository, but you are able to sort by priority.

After sorting by priority, choose something that you’re able to tackle and falls under your responsibility. That means that if you’re a frontend developer, you work on something with the label Frontend.

To filter very precisely, you could filter all issues for:

Milestone: Started
Assignee: Unassigned
Label: Your label of choice. For instance CI, Discussion, Edge, Frontend, or Platform
Sort by priority

Use this link to quickly set the above parameters. You'll still need to filter by the label for your own team.

If you’re in doubt about what to work on, ask your lead. They will be able to tell you.

Workflow labels

Labels are described in our Contribution guide.

Getting data about GitLab.com

GitLab.com is a very large instance of GitLab Enterprise Edition. It runs release candidates for new releases, and sees a lot of issues because of the amount of traffic it gets. There are several internal tools available for developers at GitLab to get data about what's happening in the production system:

Performance data

There is extensive monitoring publicly available for GitLab.com. For more on this and related tools, see the monitoring handbook.

More details on GitLab Profiler are also found in the monitoring performance handbook.

Error reporting

Sentry is our error reporting tool
log.gitlap.com has production logs
prometheus.gitlab.com has alerts for the production team

Scheduling issues

GitLab Inc has to be selective in working on particular issues. We have a limited capacity to work on new things. Therefore, we have to schedule issues carefully.

Product Managers are responsible for scheduling all issues in their respective product areas, including features, bugs, and tech debt. Product managers alone determine the prioritization. The UX Lead and Engineering Leads are responsible for allocating people making sure things are done on time. Product Managers are not responsible for these activities, they are not project managers.

Direction issues are the big, prioritized new features for each release. They are limited to a small number per release so that we have plenty of capacity to work on other important issues, bug fixes, etc.

If you want to schedule an Accepting Merge Requests issue, please remove the label first.

Any scheduled issue should have a team label assigned, and at least one type label.

Requesting something to be scheduled

To request scheduling an issue, ask the responsible product manager

We have many more requests for great features than we have capacity to work on. There is a good chance we’ll not be able to work on something. Make sure the appropriate labels (such as customer) are applied so every issue is given the priority it deserves.

Process improvement

There is an informal scheduling process discussion in the #scheduling Slack channel. Anyone can join and suggest improvements to our scheduling process.

Product development timeline

Teams (Product, UX, Engineering) continually work on issues according to their respective workflows. There is no specified process whereby a particular resource should be working on a set of issues in a given time period. However, there are specific deadlines that should inform team workflows and prioritization. Suppose we are talking about milestone m that will be shipped in month M (on the 22nd). We have the following deadlines:

By month M-1, 4th: Release scope is established. In-scope issues marked with milestone m.
By month M-1, 7th: m issues are updated with docs starter blurb. Release post (WIP merge request) created with m issues and docs starter blurbs.
On month M-1, 8th (or next business day): Kickoff call with WIP release post.
By month M, 7th: Completed m issues with docs have been merged into master. Unstarted or unfinished m issues are de-scoped from m, with m being removed from them.
On month M, 22nd: Release shipped to production. Release post published.

Refer to release post due dates for additional deadlines.

Note that release timelines are overlapping. For example, when a release is shipped to production on the 22nd, the scope for the following release has already been established earlier in that same month.

An in-scope issue already has a docs starter blurb (written by the PM) by the time engineers start development. Engineers should create and merge in the updated docs as part of completing an issue by the 7th. PMs revise the starter docs blurbs in the release post appropriately before the release post is published.

Refer to Feature freeze on the 7th for the release on the 22nd for further timeline details of code releases, including major/minor version releases, as well as patch releases.