Ensure that GitLab consistently releases high-quality software across the growing suite of products, developing software at scale without sacrificing quality, stability or velocity. The quality of the product is our collective responsibility. The quality department makes sure everyone is aware of what the quality of the product is, empirically.
To execute on this, we categorize our vision into the following areas.
Culture of Quality
Ensure a culture of quality early on in the development process.
Review product requirements and designs to identify risk and steer the team on quality strategy.
Partner with all engineering teams to ensure that our products have testability built in that can be leveraged by the test infrastructure.
Be a champion for better software design, promote proper testing practices and bug prevention strategies.
Create a timely feedback loop on Quality.
Use the data from test results to improve test gaps in our test suites.
Conduct test gap analysis from bugs that are filed by GitLab users.
Be a sounding-board for our end users and non-engineering stakeholders by integrating their feedback on quality into the development process.
Test Automation Expertise
Own and build out test automation frameworks and infrastructure.
Own and expand end-to-end automated testing framework GitLab QA.
Own and build functional performance tests.
Own and build third-party service testing capabilities.
Own and build our Continuous Integration & Delivery test pipelines.
Continuously improve GitLab test coverage, reliability and efficiency.
Groom the test pyramid coverage and ensure that the right tests run at the right place.
Improve on the duration and de-duplication of the GitLab test suites.
Improve stability by eliminating flaky tests and transient failures.
Improve test results tracking and reporting mechanisms that aid in triaging test results.
Coach developers (internal & external) on contributing to GitLab QA test scenarios.
Continuously improve the release process where Quality is a stakeholder.
Eradicate obstacles in the release process.
Ensure that we have a repeatable and consistent Quality process for every release.
Make metrics-driven suggestions to improve engineering processes, velocity, and throughput.
Build productivity tooling to help speed up development efforts.
Build automated tools to ensure the consistency and quality of the codebase and merge request workflow.
The need to ensure performance and quality will not diminish as we scale, so we may consider adding a dedicated team in the future.
Currently this is being handled by each test area owners.
Functional Performance tests
An emphasis on the performance of on-premise GitLab installations
Browser and mobile browser coverage
Functionality testing against all supported customer browsers
Visual regression testing with real browsers
Workload planning and assignment
Please look at our Project Management page for an overview of the projects we currently own and contribute to.
Product alignment teams (Dev stage & Ops stage) are stable-counterparts that are embedded as part of each product area's functional group. The scheduling is guided by each product area's product manager.
The Engineering Productivity team which is focused on improving internal processes schedules work independently but is driven by the productivity severity of the problem we are solving.
Submitting and reviewing code
For test automation changes, it is crucial that every change is reviewed by at least one Senior Test Automation Engineer in the Quality team.
We are currently setting best practices and standards for Page Objects and REST API clients. Thus the first priority is to have test automation related changes reviewed and approved by the team. For test automation only changes, the quality team alone is adequate to review and merge the changes.
It is strongly recommended not to re-assign or ask someone else to take over a Merge Request if the original engineer is not able to complete it immediately (e.g., due to vacation, illness, etc). An exception might be if the Merge Request can be completed quickly and easily (e.g., it requires only minor changes that someone else can easily make).
The Quality team holds a once-a-month 30 minute meeting for the team's retrospective. This happens one week before the public GitLab team retrospective. The retrospective is for us to reflect as a team; what went well, what went wrong and how can we improve. Emotions are not only allowed in retrospectives, they are encouraged, this is a safe environment that allows the team to discuss freely. The meeting is private only to the Quality team, however, the agenda items are public within GitLab. The agenda items are populated ahead of time, come meeting time, we discuss the items and identify a positive plan of action as needed. Important findings here can make its way into the public GitLab team retrospective in the following week.
It should be made clear that this is a retrospective and we encourage frank and sincere self-reflection. As a result, we allow passionate and emotional comments in the agenda but we ensure that they are steered towards a positive plan of action.
Person A: I was really disappointed by the progress of x because y. => Person B: I noticed that too, in the future let's make sure that we prevent y from happening with this plan etc.
Person C: I was really angry that this happened, we are not being efficient here => Person D: should we change the way we currently do things, what improvements can be made.
The Quality team retrospective agenda is only available for viewing inside GitLab. The public team retrospective contains information that is open to the public.