Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Test Design Heuristics

On this page

Heuristic Testing

Why Care?

Testing can sometimes be reduced to the simplest of checks for a specific behavior of a specific function. We often do this when we're under a time constraint. But is that how our customers always use our software?

By taking just a little extra time in our test approach to think about testing in general, we can save hours of work rolling back a release or producing hotfixes. Here are some simple approaches to enhancing our testing without adding a significant amount of time to our process.

This is especially applicable to any exploratory testing performed by both test automation engineers and backend/frontend engineers alike. They can also be helpful to consider when designing automated tests as they help eliminate bias.

Discovering Variations

Even simple solutions can have infinite variations to test when we consider the data, timing, environmental or platform factors, configuration, and user behavior following varied sequences actions. We should have as our goal to find the most important and most interesting variations to test, since we cannot possibly test them all.

Identifying Variables

Some variables are obvious, some are not. Some are accessible to the end user, some are indirectly accessible, modified only indirectly through changing other variables. For example, size or number of files in a database or number of users logged into a system may affect how a feature functions but are outside of the feature itself and may not be directly modified without specific environments or tools.

So our goal here is to identify the variables that may play a part in how a particular feature functions and use these as touch points for testing that feature. Since we may have a number of variables to consider, it would be wise to create simple notes to track those variables to make our testing more effective.

Heuristics to Help Identify Variables
What are heuristics?

The definition of a heuristic is varied based on where we look, so we'll use the simple definition: a heuristic is a fallible method for solving a problem or making a decision. They are not algorithms, so they do not always give us the results we want. We should think of them as excellent thinking tools to help us navigate unknowns, such as "did my change introduce a bug?". They are dependent upon context, so not every heuristic is applicable to every problem, just like algorithms.

Here are some heuristics that may help us find variables:

How can we use these heuristics? Are there others? The next section answers those questions.

Heuristic Glossary

Add Your Own

This is not an exhaustive list of heuristics. If you know others you have found useful, please consider contributing them to this document.

Accreditation to these ideas belongs to the general test community, many who have contributed to and compiled these ideas in different ways. For further explanation of applying heuristics to testing, see Elisabeth Hendrickson, VP of R&D at Pivotal Software, Inc., and her book titled Explore It! or many other various blogs and sites, e.g. Satisfice, DevelopSense, or Ministry of Testing. For an in-depth explanation of heuristics in the context of software testing, see Heuristics for Understanding Heuristics.