Fuzz testing, or fuzzing, is the act of inputting unexpected, malformed and/or random data to measure response or stability of an application or service. This is accomplished by monitoring for unexpected behavior or service / application crashes. The goal of fuzzing is to discover software defects that need to be addressed as these defects may lead to (exploitable) vulnerabilities.
Fuzz testing can be performed using one of the two fuzz algorithm strategies:
GitLab envisions two types of fuzzing that will become part of our offering.
A large benefit of source-aware fuzzing is that it can be done without requiring a Review App to be created for live testing. This means that you can integrate fuzzing earlier into your SDLC and shift security further left. Getting these results sooner means that developers can act on them sooner, reducing cycle times.
There are two primary target audiences:
There is an additional role which may be included in the target audience. Product verification (PV) engineers may leverage fuzz testing as part of their unit or regression tests. If the software development team does not directly handle their unit test creation, a PV engineer may be partnered with a software development team to create and maintain unit tests on behalf of the software development team.
Web API providers - Web APIs are a natural fit for us to address with fuzzing. GitLab has many users with this use case who use our product already. It will be a short jump for them to add fuzzing.
This is beneficial since we can lean on our core strengths to bring fuzz testing to this use case and we will have a large number of existing users we can start to get feedback from.
Another benefit of this is that GitLab will be able to begin dogfooding this very quickly, as we publish many web APIs ourselves.
Security research and penetration testing teams - Security labs and penetration testing firms generally make use of fuzzing in some form or another as part of their test engagements. GitLab has a large opportunity in this market with our fuzzing solution to begin working with these organization.
Specifically, GitLab can provide value with our existing product to help security firms manage their repositories and pipelines, as part of a single DevOps applicaiton. What will really differentiate us here though is that our fuzzing technology will live inside GitLab. This will help these orgs work more effectively, as opposed to having to use a standalone fuzzing product and then being required to manually interface with their SCM tools to store and update results.
Medical device and regulated industries - GitLab has an opportunity to introduce fuzzing as a new piece of technology to the medical device and regulated hardware industry. Because these devices can cause serious injury if they malfunction, the ability to find bugs with fuzzing before they are found in the field will resonate quite well.
This will be a tougher set of use cases to enter for a number of reasons. Because these industries are already very regulated, they likely have very mature workflows, which GitLab would need to find a way to make ourselves a part of. We have an opportunity to introduce GitLab as a single application for their whole DevOps lifecycle, which will help these companies move quicker in addition to providing them fuzzing, but this will take time.
Because this is a relatively new space for GitLab, we need to be cognizant of the time needed to enter the space, but it is quite a large opportunity. To make sure we can move forward iteratively, we will attempt to find an initial user to work with and make sure we understand the medical and regulated device use cases and workflows. Once they are successful, we can then use them as a case study for additional companies.
Automotive systems - Similar to medical devices, introducing fuzz testing as part of automotive systems is a big opportunity for GitLab. Automotive systems undergo long testing cycles before deployment into production, which means they can take advantage of long, "soak-test" fuzzing sessions to find bugs.
A potential challenge for this use case will be ensuring our fuzz test and GitLab platform fit the existing technology stacks that automotive manufacturers use. To ensure that we are successful in this space, we will canvas several different automotive customers, learn more about their specific technology stacks, and work closely with one we are compatible with as aprt of a future iteration. Based on the results and feedback, we can then decide to either expand into additional technology stacks or to focus more on the ones we already have.
Embedded device manufacturers - A broad and challenging domain we can bring fuzzing to is the embedded device space. These hardware devices typically connect to sensors and control units over specific interfaces, some of which are not always secured before being deployed. Fuzz testing can provide the ability to locate these bugs and vulnerabilities before deploying the devices.
Embedded devices generally will have toolchains that are less commonly used than web or desktop software, since those toolchains are intended solely for developing hardware-based applications. This means that fuzzing will provide a large amount of value since the platforms have not been as rigorously tested by as many people, but also will present GitLab a challenge in that there will be far more choices than we can realistically support.
To ensure we can be successful in this space, we will start with hardware devices that communicate over channels and protocols that are close to our existing fuzzing tools and do not require the use of custom hardware to communicate. Devices that speak HTTP over a wired or wirless connection are a good example of what we could quickly target. Devices that use a proprietary hardware interface with a proprietary protocol GitLab does not support are a good example of the type of devices we would not attempt to start with.
Each target audience has different challenges that need to be addressed with our fuzz testing solution:
Traditionally, fuzzing has been difficult to implement inside of organizations because it can be time-consuming to setup and run as well as require a large amount of technical expertise to do so correctly and to properly interpret results. This has largely limited the adoption of fuzzing to large enterprises with very mature security organizations.
GitLab's aspirations with fuzzing is that it becomes a security technique that all of our users can take advantage of to find and fix issues in their apps before attackers can exploit them.
Since fuzzing is traditionally difficult to use, our emphasis will be on making fuzzing an accessible technology that our users can take advantage of without becoming technical experts in the space nor requiring them to form a dedicated fuzzing team.
GitLab will provide its users with application robustness and security visibility testing solutions which validate the entire application stack. This will be provided by verifying the user’s application or service both while at rest (white-box testing) as well as while running (black-box testing). This will include historical trending and recommendations for next steps to provide peace of mind to our customers.
Furthermore, GitLab will provide real-time remediation for user solutions running in production. This will be possible by analyzing the user issues found in GitLab Secure and apply “virtual patches” to the user’s production application or service leveraging GitLab Defend. This will allow your organization to be secure and continue functioning until the underlying vulnerability can be remediated in your code.
GitLab ultimately envisions our fuzzing solutions becoming a critical part of every organization's DevSecOps workflow, whether that is for web APIs, traditional desktop software, or for hardware devices. Our long-term vision is to provide fuzzing solutions for all these use cases. We will start with web API and web appliation use cases, which are already well supported in GitLab, as part of an iterative approach to delivering fuzzing.
The next step is to provide a minimal support for fuzzing as part of GitLab. Users can set up dedicated pipelines and run fuzzing sessions. This is the very minimal goal to dig into this category.
Our initial focus is on network applications and services, so we are not focusing on fuzzing local desktop or mobile applications at this time.
Wireless, Bluetooth, and Automotive based fuzzing as these types of fuzz testing solutions require special hardware to accomplish. This makes these types of fuzz testing out of scope for now.
The fuzz testing competitive landscape is composed of both commercial and open source testing solutions. The following outlines the competitive landscape for fuzz testing solutions:
GitLab already uses OWASP ZAP today to power the DAST capability within the GitLab Secure offerings. ZAP is a web application security scanner, leveraging a proxy to perform testing, and does not perform protocol-level fuzzing. All fuzzing by ZAP is performed within the context of the web application being tested.
Analysts consider fuzzing to be part of Application Security Testing and is generally discussed as part of those reports, rather than as a standalone capability. There are challenges to make it part of the DevSecOps paradigm, and they are interested to hear more about that.
Gartner wrote "Outsourcing or leveraging managed services to perform fuzzing is the recommendation in the absence of internal subject matter expertise and staffing." as part of their "Structuring Application Security Practices and Tools to Support DevOps and DevSecOps" research. This recommendation is a result of the high-level of manual configuration many fuzzers need. This is a pain point GitLab should be mindful of as we work on bringing our own fuzzing solutions out so that users can be successful without needing to be a fuzzing expert nor need to do a large amount of configuration.
Gartner also published their How to Deploy Application and Perform Application Security Testing where they say "You should reserve fuzzing for nonweb applications." We disagree with this conclusion and think that it comes primarily from the difficulty normally associated with doing fuzz testing. This underscores the importance of our emphasis of making fuzz testing straightforward and easy to use.
Additionaly, as APIs become more and more of a staple of modern application development, they should be prioritized for fuzz testing. As one of the primary interfaces exposed to end-users, fuzz testing can reveal critical bugs and security issues before they are exposed to the world, despite them not being traditional web applications.
By combining our fuzz testing solution with our DAST, IAST, and SAST solutions into a GitLab Secure suite, GitLab can be placed into the Gartner Magic Quadrant for Application Security Testing. This will give GitLab additional exposure as well as show security thought leadership.
Other analyst firms include 451 Research and IDC as they have focused security practices in which GitLab Secure can be highlighted and show leadership.
