Infrastructure

Maintained by:

Mission

The Infrastructure Department enables GitLab (the company) to deliver a single DevOps application, and GitLab SaaS users to focus on generating value for their own businesses by ensuring that we operate an enterprise-grade SaaS platform.

The Infrastructure Department does this by focusing on availability, reliability, performance, and scalability efforts. These responsibilities have cost efficiency as an additional driving force, reinforced by the properly prioritized dogfooding efforts.

Many other teams also contribute to the success of the SaaS platform because GitLab.com is not a role. However, it is the responsibility of the Infrastructure Department to drive the ongoing evolution of the SaaS platform, enabled by platform observability data.

Vision

The Infrastructure Department operates a fast, secure, and reliable SaaS platform to which (and with which) everyone can contribute.

Integral part of this vision is to:

Build a highly performant team of engineers, combining operational and software development experience to influence the best in reliable infrastructure.
Work publicly in accordance with our transparency value.
Use our own product to prepare, build, deliver work, and support the company strategy.
Align our strategy with the industry trends, company direction, and end customer needs.

Direction

The direction is accomplished by using Objectives and Key Results (OKRs).

Other strategic initiatives to achieve this vision are driven by the needs of enterprise customers looking to adopt GitLab.com. The GitLab.com strategy catalogs top customer requests for the SaaS offering and outlines strategic initiatves across both Infrastructure and Stage Groups needed to address these gaps.

We are also Product Development

Unlike typical companies, part of the mandates of our Security, Infrastructure, and Support Departments is to contribute to the development of the GitLab Product. This follows from these concepts, many of which are also behaviors attached to our core values:

As such, everyone in the department should be familiar with, and be acting upon, the following statements:

We should all feel comfortable contributing to the GitLab open source project
If we need something, our first instinct should be to get it into the open source project so it can be given back to the community
Try to get it in the open source project first, rather than later, even if it's 2x harder
We should be using the whole product to do our jobs
We are all familiar with our Dogfooding process and follow it
We should not expect new team members to join the company with these instincts, so we should be willing to teach them
It is part of managers' responsibility to teach these values and behaviors

Design

The Infrastructure Library contains documents that outline our thinking about the problems we are solving and represents the current state for any topic, playing a significant role in how we produce technical solutions to meet the challenges we face.

Dogfooding

The Infrastructure department uses GitLab and GitLab features extensively as the main tool for operating many environments, including GitLab.com.

We follow the same dogfooding process as part of the Engineering function, while keeping the department mission statement as the primary prioritization driver. The prioritization process is aligned to the Engineering function level prioritization process which defines where the priority of dogfooding lies with regards to other technical decisions the Infrastructure department makes.

When we consider building tools to help us operate GitLab.com, we follow the 5x rule to determine whether to build the tool as a feature in GitLab or outside of GitLab. To track Infrastructure's contributions back into the GitLab product, we tag those issues with the appropriate Dogfooding label.

Handbook use at the Infrastructure department

At GitLab, we have a handbook first policy. It is how we communicate process changes, and how we build up a single source of truth for work that is being delivered every day.

The handbook usage page guide lists a number of general tips. Highlighting the ones that can be encountered most frequently in the Infrastructure department:

The wider community can benefit from training materials, architectural diagrams, technical documentation, and how-to documentation. A good place for this detailed information is in the related project documentation. A handbook page can contain a high level overview, and link to more in-depth information placed in the project documentation.
Think about the audience consuming the material in the handbook. A detailed run through of a GitLab.com operational runbook in the handbook might provide information that is not applicable to self-managed users, potentially causing confusion. Additionally, the handbook is not a go-to place for operational information, and grouping operational information together in a single place while explaining the general context with links as a reference will increase visibility.
Ensure that the handbook pages are easy to consume. Checklists, onboarding, repeatable tasks should be either automated or created in a form of template that can be linked from the handbook.
The handbook is the process. The handbook describes our principles, and our epics and issues are our principles put into practice.

Projects

Classification of the Infrastructure department projects is described on the infrastructure department projects page.

The infrastructure issue tracker is the backlog and a catch-all project for the infrastructure teams and tracks the work our teams are doing–unrelated to an ongoing change or incident.

In addition to tracking the backlog, Infrastructure Department projects are captured in our Infrastructure Department Epic as well as in our Quarterly Objectives & Key Results

Supporting Product Features

We have a model that we use to help us support product features. This model provides details on how we collaborate to ship new features to Production.

Stable Counterparts

Infrastructure SREs may be aligned with stage groups as stable counterparts.

Stable Counterparts are used as a framework for managing reliable services at GitLab. The framework provides guidelines for collaboration between Stage Groups and Infrastructure Teams.

Interviewing

The Infrastructure department hires for a number of different technical specialisms and positions across its teams. This Infrastructure Interviewing Guide offers more detail on some of our regular openings, interview process and other useful information related to applying to jobs with us. More information on our current openings can be found on the careers page.

Infrastructure Office Hours

Join us for our interactive Infrastructure Office Hours call! This monthly gathering, held every second Tuesday of the month, provides a platform for open dialogue between the Infrastructure department and the rest of the company. Whether you have questions, suggestions, or insights to share, this call is an opportunity to collaborate and drive positive change. Everyone is invited to participate and discuss topics such as reliability engineering, incident management, observability, scalability, delivery and deployments, cloud costs, error budgeting, service-level objectives or any other infrastructure-related topics.

Check the Infrastructure calendar in Google Calendar for exact times. The agenda for the meeting is available in Google Docs.

The call is livestreamed onto the GitLab Unfiltered channel.

Common Links

General Issue Trackers	General Slack Channels	Team Slack Channels	Resources
Infrastructure issue queue	#production	#g_delivery	Production Architecture
Production incidents, and changes	#infrastructure-lounge	#g_scalability	Operational Runbooks
Delivery	#incident-management		Environments
Scalability	#announcements		Monitoring
	#feed_alerts-general		Readiness Reviews
			Infrastructure Standards