Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Quality Department

On this page

Child Pages

On-boarding
Guidelines
Performance Indicators
Project Management
Roadmap
Test Engineering
Triage Operations

Vision

Ensure that GitLab consistently releases high-quality software across the growing suite of products, developing software at scale without sacrificing quality, stability or velocity. The quality of the product is our collective responsibility. The quality department makes sure everyone is aware of what the quality of the product is, empirically.

To execute on this, we categorize our vision into the following areas.

Culture of Quality

Champion an early culture of quality in the development process

Timely feedback loop

Test Coverage

Test framework reliability and efficiency

Improve test coverage

Self-managed usecase

Improve Quality in release process

Engineering Productivity

Quality Engineering structure

Teams within Quality Engineering

Dev QE team
Ops & CI/CD QE team
Secure & Enablement QE team
Engineering Productivity team

Department members

Person Role
Rémy Coutable Staff Backend Engineer, Engineering Productivity
Mark Fletcher Backend Engineer, Engineering Productivity
Jen-Shin Lin Backend Engineer, Engineering Productivity
Kyle Wiebers Senior Backend Engineer, Engineering Productivity
Dan Davison Senior Test Automation Engineer, Configure
Mark Lapierre Senior Test Automation Engineer, Create:Source Code
Sanad Liaquat Senior Test Automation Engineer, Manage
Tomislav Nikić Test Automation Engineer, Create:Knowledge, Create:Editor, Gitter
Zeff Morgan Senior Test Automation Engineer, Verify
Walmyr Lima e Silva Filho Senior Test Automation Engineer, Plan
Aleksandr Soborov Test Automation Engineer, Secure
Nailia Iskhakova Test Automation Engineer, Enablement
Grant Young Senior Test Automation Engineer, Enablement
Jennie Louie Test Automation Engineer, Enablement
A.S. Backend Engineer, Engineering Productivity
Désirée Chevalier Test Automation Engineer, Manage

Stable counterparts

Every Test Automation Engineer is aligned with a Product Manager and is responsible for the same features their Product Manager oversees. They work alongside Product Managers and engineering at each stage of the process: planning, implementation, testing and further iterations. The area a Test Automation Engineer is responsible for is part of their title; for example, "Test Automation Engineer, Plan." as defined in the team org chart.

Every Quality Engineering Manager is aligned with an Engineering Director in the Development Department. They work at a higher level and align cross-team efforts which maps to a Development Department section. The area a Quality Engineering Manager is responsible for is part of their title; for example, "Quality Engineering Manager, Dev" as defined in the team org chart. This is with the exception of the Engineering Productivity team which is based on the span of control.

Full-stack Engineering Productivity Engineers develop features both internal and external which improves the efficiency of engineers and development processes. Their work is separate from the regular release kickoff features per areas of responsibility.

Headcount planning

We plan headcount as follows:

Quality Engineering processes

Failure Management Rotation

Every member in the Quality team shares the responsibility of analysing the daily QA tests against master and staging branches. More details can be seen here

Development and infrastructure collaboration

We currently have 2 venues of collaboration with Development and Infrastructure departments.

Availability and performance grooming

To mitigate performance issues, Quality Engineering will triage and groom performance issues for Product Management and Development via a weekly Availability & Performance Grooming. The goal is to make the performance of various aspects of our application empirical with tests, environments, and metrics.

Quality Engineering will ensure that performance issues are identified and/or created on the board with the label ~performance-grooming. These issues that are surfaced to the grooming meeting will be severitized according to our definitions.

Identifying issues

Quality Engineering will focus in identifying issues in the following areas:

Grooming

A manager in the Quality Engineering department will lead grooming with issues populated before hand in the board. Issues are walked through from high to low severity covering ~S1, ~S2 and ~S3 performance bugs.

Deliverable of grooming each issue:

Please see the Development department's Infrastructure and Quality collaboration handbook section.

Development requests

Quality Engineering will track productivity, metric and process automation improvement work items in the Development-Quality board to service the Development department. Requirements and requests are to be created with the label ~dev-quality. The head of both departments will review and groom the board on an on-going basis. Issues will be assigned and worked on by an Engineer in the Engineering Productivity team team and communicated broadly when each work item is completed.

Meetings

We try to have as few meetings as possible. We currently have 3 recurring meetings for the whole department. Everyone in the Department is free to join and the agenda is available to everyone in the company. Every meeting is also recorded.

  1. Quality Department Weekly: This is where the whole department comes together weekly to discuss our day-to-day challenges, propose automation framework improvements and catchup on important announcements. This is also a place to connect socially with the rest of the department. This meeting happens in 2 parts to accommodate our team members across multiple timezones.
    • Part 1 - Wednesday 0730 UTC
    • Part 2 - Wednesday 2030 UTC (adjusts per PDT)
  2. Engineering Productivity Team Weekly: The Engineering Productivity team meets weekly to discuss engineering wide process improvements. This meeting is scheduled for every Tuesday at 1300 UTC.
  3. Quality Engineering Staff Weekly: This weekly brings together Quality Engineering's management team to discuss directional plans, long-term initiatives, hiring goals and address issues that require attention. This meeting is scheduled for every Wednesday at 1430 UTC (adjusts per PDT).

Team retrospective

The Quality team holds an asynchronous retrospective for each release. The process is automated and notes are captured in Quality retrospectives (GITLAB ONLY)

AMA sessions

Every quarter the Quality team will host an AMA session. The idea is to keep everyone informed about what's new, our challenges and to answer questions related to the test framework and etc.

The next sessions are scheduled for 2019/09/20, 2019/12/20, and 2020/03/20.

Note: the dates mentioned above can change, but we will try to keep this document updated.

Release process overview

Moved to release documentation.

Quality Engineering initiatives

Triage Efficiency

Due to the volume of issues, one team cannot handle the triage process. We have invented Triage Packages to scale the triage process within Engineering horizontally.

More on our Triage Operations

Test Automation Framework

The GitLab test automation framework is distributed across three projects:

Architecture overview

See the GitLab QA Documentation and current architecture overview.

Installation and execution