What is fuzz testing?

Fuzz testing, or application fuzzing, is a software testing technique that allows teams to discover security vulnerabilities or bugs in the source code of software applications. Unlike traditional software testing methodologies – SAST, DAST, or IAST – fuzzing essentially “pings” code with random inputs in an effort to crash it and thus identify faults that would otherwise not be apparent. Those code faults (or problems with business logic) represent areas that are potentially at high risk for security threats.

When a fault or vulnerability is found, a fuzzer — a tool that identifies the potential causes of the crash — can be used to zero in on specific vulnerabilities in the source code. Fuzzers are most effective at uncovering vulnerabilities that can be exploited by attacks such as SQL injection and cross-site scripting, where hackers disable security to steal information or take down a system. Fuzzers are less effective at identifying vulnerabilities that are unrelated to system crashes, such as spyware or Trojans.

Fuzz testing’s supporters praise it for being fully automated and able to find obscure weaknesses, while its detractors complain it can be difficult to set up and prone to deliver unreliable results.

Fuzz testing stands out in another way as well: there’s actually a story about how the concept was discovered. In 1988, University of Wisconsin – Madison Professor Barton Miller was trying to access code remotely using a dial-up system, but feedback from a thunderstorm kept causing the program to crash. The idea that external “noise” couldn’t be tolerated by code became the inspiration for Miller and his student’s work. They discovered that Unix, Mac, and Windows programs would routinely crash when pinged by random unexpected inputs. Miller is one of the authors of Fuzzing for Software Security Testing and Quality Assurance.

There are two main types of fuzzing: coverage-guided and behavioral.

Coverage-guided fuzzing focuses on the source code while the app is running, probing it with random input in an effort to uncover bugs. New tests are constantly being generated and the goal is to get the app to crash. A crash means a potential problem, and data from the coverage-guided fuzz testing process will allow a tester to reproduce the crash, which is helpful when trying to identify at-risk code.

Behavioral fuzzing works differently. Using specs to show how an application should work, it uses random inputs to judge how the app really works; the difference between the expected and the reality is generally where bugs or other potential security risks can be found.

Why is fuzz testing important for DevSecOps? Because of the random nature of fuzz testing, experts say it’s the methodology most likely to find bugs missed by other tests. It’s also seen as an incredibly low-effort testing methodology, or what some like to call “set it and forget it.” Once the test harness is created fuzz testing is fully automated and will run indefinitely. It can be scaled easily by spinning up more machines and is a good choice for regression testing. Fuzzing also speeds up the development process by maximizing code coverage — how much of the code has been executed by the fuzzer — without introducing false positives. Higher code coverage means more thorough testing.

Fuzz testing is also ideal to work alongside a manual testing team, as both sets of inputs will educate the other.

Two main challenges face practitioners looking to implement fuzz testing: setup and data analysis. Fuzz testing isn’t necessarily easy to set up; it requires complex testing “harnesses” that can be even more tricky to create if the fuzz testing isn’t actually located within an existing toolchain. Plus, fuzz testing can generate a lot of data, including potentially false positives. So it’s critical to ensure that a testing team is prepared to deal with the onslaught of information.

Also, while less easy to document, negative attitudes toward the “vague” nature of fuzz testing do persist in the QA community.