Note: I resurrected this entry from an older blog. So if it looks familiar, that’s why.
This week I was reminded of how fun fuzz testing software applications was.
At a previous job, there were many bugs that were found by simply feeding completely random data into the programs. Even more bugs were found when it started with valid data that had a small amount of the data randomized, or fuzzed.
So I wanted to write an entry about what it is, and how to use it.
What Is It?
Fuzz testing is a testing method, where randomized data is fed into your application to quickly find faults.
It’s not the same as testing for an explicit input and output, such as a simple program that adds two numbers–if you give it “2” and “5”, you’d expect it to return “7”.
Instead, this type of test checks for certain conditions or events to not happen. It’s widely accepted, for example, that a program should never crash, regardless of input.
The cool part is, since we don’t care so much about the input data, we can fuzz an application as fast as the application will respond. The test will fire information at it until a certain number of tries has been reached, or until it poops its pants, whichever comes first.
There are two broad categories of fuzzing: random and quasi-random
Random fuzzing is just bombarding the application with complete garbage–any amount of any kind of data. If you’re on a Linux system, using /dev/urandom is handy.
You’ll likely find bugs the data begins being processed. Many programs make assumptions about how the data will be structured, such as, “The data will be no more than 50 characters long,” or, “The data will only contain certain kinds of characters.” Throwing random data quickly ferrets out those assumptions.
Quasi-random fuzzing is a step up from straight random fuzzing–you start with valid data, but make a small to it, then send it through.
What this does is give you data that can probably get past the data processing part, and get into deeper parts of the code.
The real meat of a program will have more complex business logic, and bugs resulting from things like forgotten requirements, multiple developers working on the same code, unhandled cases, etc.
What Can Be Fuzzed?
Practically any program. All programs act on some kind of input. It could be:
- A file
- Network packets
- A webservice payload
- A webpage
- Manually-entered data from a user
- A database
Why Fuzz Test?
This may seem an unsporting way to find bugs, but the fact is, this type of test finds places where assumptions are made in the code.
It finds where assumptions are made by finding where programs don’t fail gracefully. Failing gracefully is an important part of software development.
Many programs are either customer-facing, or they run behind the scenes.
For customer-facing programs, you don’t want an ugly crash showing up where instead a friendlier “Oops!” message can appear.
For backend programs, having your program log a verbose message saying what happened is better than weeding through a stacktrace for a problem.
Release The Hounds!
Hope this has helped you and that you enjoy trying this method of testing. I think you’ll find that the ease of constructing fuzz tests plus the versatility of using them against similar programs is really satisfying.
It’s not a one-stop test–there’s still a lot of other types of testing to do–but as far as testing types go, this one will get some mileage.