Multi-Large Working Group

1. You are here:
2. Meet the GitLab team
3. Organizational Structure
4. Working Groups
5. Multi-Large Working Group

On this page

• Attributes
  • Charter
  • Scope and Definitions
    • Sequence Order Of Deliverables
      • Deprecate NFS
      • Helm Charts
      • Stateful Nodes
  • Horizontal Scaling Across Multiple Instances
• Roles and Responsibilities
• Related Links

Attributes

Charter

The charter of this working group is to improve the operational efficiency of running multiple, large, independent GitLab sites without needing linear growth on staffing.

Scope and Definitions

In this context,

• GitLab sites means GitLab instances under a GitLab domain managed by GitLab (i.e, not operated on behalf of customers)
  • Automation developed should be applicable and usable to self-managed installations in the future
  • The scope excludes customer instances to focus on the problem of running multiple large sites fully under our control, where we are the ultimate DRIs for what policies and solutions we adopt as a company to apply to those instances.
• Multiple means up to 10 primary instances
• large means instances that start at the 1K users Reference Architecture and can scale beyond the 50K users Reference Architecture to a million active users
• independent means they're fully isolated from each other, they don't have a common virtual operations centre

An example of this type of instance would include GitLab.eu. The working group may possibly consider GitLab Federal, although this has to be an explicit decision, as operating federal entails clearances and such.

Sequence Order Of Deliverables

The following list outlines the order of deliverables per Multi-large Working Group Agenda on 2020-07-13, and then further refined on 2021-02-01. The goal of this list is to define deliverables for the Working Group such that its charter can be finite.

Deprecate NFS

1. Make Web/API work without NFS (issue tracker)
2. Make CI/CD live traces work without NFS (issue tracker)
3. Make Pages work without NFS (Scalability: https://gitlab.com/groups/gitlab-org/-/epics/3980)

Helm Charts

1. Move Web/API nodes of .com to Helm charts (issue tracker)
2. Move Pages nodes of .com to Helm Charts (issue tracker)

Stateful Nodes

1. Gitaly to K8s
2. Redis to K8s

Note that not all stateful nodes are scoped here, notably PostgreSQL, and this is intentional.

Horizontal Scaling Across Multiple Instances

For clarification, this working group is focused on horizontal scaling across multiple GitLab sites. Single-site scaling (e.g., GitLab.com) is already covered under the charter of the Scalability Team.

Roles and Responsibilities

Related Links

• Working group kick off meeting
• Defining Day-2 Operations - DZone Agile
• Scalability Team - GitLab