Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Test Engineering Process

On this page

The Quality Engineering department helps facilitate the test planning process for all things related to Engineering work.

Child Pages

Test Design Heuristics

Test Planning

The goal of our test planning process is to ensure quality and manage risk in an efficient and effective manner.

To achieve that goal when delivering a change or new feature, we have conversations that link test requirements to product requirements, identify risks and quality concerns, and review and plan test coverage across all test levels.

The output or deliverables of the test planning process are:

The deliverables are considered complete when the tests have been added/updated and the merge request has been merged.

Process

At release kickoff we highlight some of the changes scheduled for the next release. The majority of our test planning work starts in the issues relevant to those changes, although this process can be applied to any change. Here is an overview of the process:

The following guidelines provide more detail, as well as suggested responsibilities for various roles.

As a feature issue author:
As an Engineer who will implement the change, or a Test Automation Engineer contributing to the change:
As a merge request author (i.e., the Engineer who will implement the test):
As a Test Automation Engineer:

Finally, once all test deliverables are completed, the feature issue can be closed (along with a test plan, if one was created).

Everyone in engineering is expected to contribute to Quality and keep our test pyramid in top shape. For every new feature we aim to ship a new slice of the pyramid so we don't incur test automation debt. This is what enables us to do Continuous Delivery.

TestPyramid.png

Test Plan

We do not require a test plan for every feature or epic. Test plans are expensive to create and we would only limit this to high-impact and cross functional changes. There is no strict guideline for this and we defer this decision to each engineering group or team.

Examples of work that's likely to warrant a test plan:

GitLab's test plan is based on Google’s 10 min test plan. This test plan uses the ACC Framework (Attribute, Components and Capabilities matrix)

Currently we have a test plan template that is available for test planning in GitLab CE & EE. A test plan can be created by anyone involved in a change.

You can see the all the existing test plans here:

Security

In addition to quality, security is also one of our top concerns. With every change and every new feature we need to consider security risks and mitigate them early in the development process.

See also when and how to request a security review.

The test plan template has a risk column dedicated to security, however, whether or not a test plan is not created, it is essential that a discussion about security risks takes place as early as possible.

Test Coverage Tracking

We are currently evaluating using GitLab mechanics to track test coverage in gitlab-org/quality/testcases.

Our previous iteration can be seen at DEPRECATED - GitLab Blackbox Integration & E2E Test Coverage.

An explanation of why we chose to do it this way is explained in this issue: Test case management and test tracking in a Native Continuous Delivery way.

To summarize, we want to track our tests in a Native Continuous Delivery way.

An overview of the test management view is shown below.

TestCaseManagement.png