Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Quality Department

Child Pages

On-boarding
Guidelines
Performance Indicators
Performance and Scalability
Project Management
Roadmap
Test Engineering
Triage Operations

FY21 Direction

Achieve world class enterprise grade readiness and empowering product and development teams to ship software at scale without sacrificing quality, stability or velocity. Enable Engineering and Product organizations to account for quality proactively in the planning process. Relentless focus on internal test and tooling stability for maximum productivity of our Engineering organization.

The quality of the product is our collective responsibility, we the quality department makes sure everyone is aware of what the quality of the product is, empirically.

To execute on this, we categorize our direction into the following areas.

Culture of quality

Test automation coverage

Tooling reliability and efficiency

Productivity, metrics and workflow optimization

Self-managed usecase

Quality Engineering structure

Teams within Quality Engineering

Dev QE team
Ops & CI/CD QE team
Growth QE team
Secure & Enablement QE team
Engineering Productivity team

Department members

Director and managers

Person Role
Mek Stittri Director of Quality Engineering
Ramya Authappan Quality Engineering Manager, Dev
Joanna Shih Quality Engineering Manager, Ops & CI/CD
Tanya Pazitny Quality Engineering Manager, Secure & Enablement
Kyle Wiebers Interim Quality Engineering Manager, Engineering Productivity
Vincy Wilson Quality Engineering Manager, Growth

Individual contributors

Person Role
Rémy Coutable Staff Backend Engineer, Engineering Productivity
Mark Fletcher Backend Engineer, Engineering Productivity
Jen-Shin Lin Backend Engineer, Engineering Productivity
Dan Davison Senior Software Engineer in Test, Configure
Mark Lapierre Senior Software Engineer in Test, Create:Source Code (primary) & Create:Gitaly (secondary)
Sanad Liaquat Senior Software Engineer in Test, Manage:Access (primary) & Manage:Spaces (secondary)
Tomislav Nikić Software Engineer in Test, Create:Knowledge
Zeff Morgan Senior Software Engineer in Test, Verify:Testing
Walmyr Lima e Silva Filho Senior Software Engineer in Test, Plan:Project Management (primary) & Plan:Portfolio Management (secondary)
Tiffany Rea Software Engineer in Test, Verify:Continuous Integration
Sofia Vistas Software Engineer in Test Monitor:APM (primary) & Monitor:Health (secondary)
Grant Young Senior Software Engineer in Test, Enablement:Memory
Jennie Louie Software Engineer in Test, Enablement:Geo Replication
Nailia Iskhakova Software Engineer in Test, Enablement:Distribution
Erick Banks Senior Software Engineer in Test Enablement:Search
Albert Salim Senior Backend Engineer, Engineering Productivity
Désirée Chevalier Software Engineer in Test, Manage:Import
Will Meek Senior Software Engineer in Test, Secure:Composition Analysis (primary) & Secure:Exposure Analysis (secondary)
Anastasia McDonald Software Engineer in Test, Create:Editor
N.W. Senior Software Engineer in Test, Enablement:Distribution

Stable counterparts

Every Software Engineer in Test (SET) takes part in building our product as a DRI in GitLab's Product Quad DRIs. They work alongside Development, Product, and UX in the Product Development Workflow. As stable counterparts, SETs should be considered critical members of the core team between Product Designers, Engineering Managers and Product Managers.

Every Quality Engineering Manager is aligned with an Engineering Director in the Development Department. They work at a higher level and align cross-team efforts which maps to a Development Department section. The area a Quality Engineering Manager is responsible for is defined in the Product Stages and Groups and part of their title in team org chart. This is with the exception of the Engineering Productivity team which is based on the span of control.

Full-stack Engineering Productivity Engineers develop features both internal and external that improves the efficiency of engineers and development processes. Their work is separate from the regular release kickoff features per areas of responsibility.

Staffing planning

We staff our department with the following gearing ratios:

Quality Engineering processes

Task management and tracking

We have top level boards (at the gitlab-org level) to communicate what is being worked on for all teams in quality engineering. Each board has a cut-line on every column that is owned by an individual. Tasks can be moved vertically to be above or below the cut-line. The cut-line is used to determine team member capacity, it is assigned to the Backlog milestone. The board itself pulls from any milestone as a catch-all so we have insights into past, current and future milestones. The cut-line also serves as a healthy discussion between engineers and their manager in their 1:1s. Every task on the board should be sized according to our weight definitions.

How to use the board and cut-line

Discussion on the intent and how to use the board

Team boards

The boards serve as a single pane of glass view for each team and help in communicating the overall status broadly, transparently and asynchronously.

Quality department pipeline triage rotation

Every member in the Quality Department shares the responsibility of analyzing the daily QA tests against master and staging branches. More details can be seen here

Development and infrastructure collaboration

We currently have 2 venues of collaboration with Development and Infrastructure departments.

Availability and performance grooming

To mitigate performance issues, Quality Engineering will triage and groom performance issues for Product Management and Development via a weekly Availability & Performance Grooming. The goal is to make the performance of various aspects of our application empirical with tests, environments, and metrics.

Quality Engineering will ensure that performance issues are identified and/or created on the board with the label ~performance-grooming. These issues that are surfaced to the grooming meeting will be severitized according to our definitions.

Identifying issues

Quality Engineering will focus in identifying issues in the following areas:

Grooming

A manager in the Quality Engineering department will lead grooming with issues populated before hand in the board. Issues are walked through from high to low severity covering ~S1, ~S2 and ~S3 performance bugs.

Deliverable of grooming each issue:

Please see the Development department's Infrastructure and Quality collaboration handbook section.

Development requests

Quality Engineering will track productivity, metric and process automation improvement work items in the Development-Quality board to service the Development department. Requirements and requests are to be created with the label ~dev-quality. The head of both departments will review and groom the board on an on-going basis. Issues will be assigned and worked on by an Engineer in the Engineering Productivity team team and communicated broadly when each work item is completed.

Meetings

We try to have as few meetings as possible. We currently have 3 recurring meetings for the whole department. Everyone in the Department is free to join and the agenda is available to everyone in the company. Every meeting is also recorded.

  1. Quality Department Weekly: This is where the whole department comes together weekly to discuss our day-to-day challenges, propose automation framework improvements and catchup on important announcements. This is also a place to connect socially with the rest of the department. This meeting happens in 2 parts to accommodate our team members across multiple timezones.
    • Part 1 - Wednesday 0730 UTC
    • Part 2 - Wednesday 2030 UTC (adjusts per PDT)
  2. Engineering Productivity Team Weekly: The Engineering Productivity team meets weekly to discuss engineering wide process improvements. This meeting is scheduled for every Tuesday at 1300 UTC.
  3. Quality Engineering Staff Weekly: This weekly brings together Quality Engineering's management team to discuss directional plans, long-term initiatives, hiring goals and address issues that require attention. This meeting is scheduled for every Wednesday at 1430 UTC (adjusts per PDT).

Team retrospective

The Quality team holds an asynchronous retrospective for each release. The process is automated and notes are captured in Quality retrospectives (GITLAB ONLY)

Release process overview

Moved to release documentation.

Quality Engineering initiatives

Triage Efficiency

Due to the volume of issues, one team cannot handle the triage process. We have invented Triage Reports to scale the triage process within Engineering horizontally.

More on our Triage Operations

Test Automation Framework

The GitLab test automation framework is distributed across three projects:

Architecture overview

See the GitLab QA Documentation and current architecture overview.

Installation and execution

Performance and Scalability

The Quality Department is committed to ensuring that self-managed customers have performant and scalable configurations. To that end, we are focused on creating a variety of tested and certified Reference Architectures. Additionally, we have developed the GitLab Performance Toolkit, which provides several tools for measuring the performance of any GitLab instance. We use the Toolkit every day to monitor for potential performance degradations, and this tool can also be used by GitLab customers to directly test their on-premise instances. More information is available on our Performance and Scalability page.