DeepState is a framework that provides C and C++ developers with a common interface to various symbolic execution and fuzzing engines. Users can write one test harness using a Google Test-like API, then execute it using multiple backends without having to learn the complexities of the underlying engines. It supports writing unit tests and API sequence tests, as well as automatic test generation. Read more about the goals and design of DeepState in our paper.
To put it another way, DeepState sits at the intersection of property-based testing, traditional unit testing, fuzzing, and symbolic execution. It lets you perform property-based unit testing using fuzzing or symbolic execution as a back end to generate data, and saves the results so that what DeepState finds can easily be used in deterministic settings such as regression testing or CI.
DeepState currently targets Linux, with macOS support in progress (the fuzzers work fine, but symbolic execution is not well-supported yet, without a painful cross-compilation process).
First make sure you install Python 3.6 or greater. Then use this command line to install additional requirements and compile DeepState:
sudo apt update && sudo apt-get install build-essential gcc-multilib g++-multilib cmake python3-setuptools python3-dev libffi-dev z3 sudo apt-add-repository ppa:sri-csl/formal-methods sudo apt-get update sudo apt-get install yices2 git clone https://github.com/trailofbits/deepstate deepstate mkdir deepstate/build && cd deepstate/build cmake ../ make sudo make install
Assuming the DeepState build resides in
$DEEPSTATE, run the following commands to install the DeepState python package:
virtualenv venv . venv/bin/activate python $DEEPSTATE/build/setup.py install
$PATH should now include two executables:
deepstate-angr. These are executors, which are used to run DeepState test binaries with specific backends (automatically installed as Python dependencies). The
deepstate-manticore executor uses the Manticore backend while
deepstate-angr uses angr. They share a common interface where you may specify a number of workers and an output directory for saving backend-generated test cases.
If you try using Manticore, and it doesn't work, but you definitely have the latest Manticore installed, check the
.travis.yml file. If that grabs a Manticore other than the master version, you can try using the version of Manticore we use in our CI tests. Sometimes Manticore makes a breaking change, and we are behind for a short time.
You can check your build using the test binaries that were (by default) built and emitted to
deepstate/build/examples. For example, to use angr to symbolically execute the
IntegerOverflow test harness with 4 workers, saving generated test cases in a directory called
out, you would invoke:
deepstate-angr --num_workers 4 --output_test_dir out $DEEPSTATE/build/examples/IntegerOverflow
out directory should look something like:
out └── IntegerOverflow.cpp ├── SignedInteger_AdditionOverflow │ ├── a512f8ffb2c1bb775a9779ec60b699cb.fail │ └── f1d3ff8443297732862df21dc4e57262.pass └── SignedInteger_MultiplicationOverflow ├── 6a1a90442b4d898cb3fac2800fef5baf.fail └── f1d3ff8443297732862df21dc4e57262.pass
To run these tests, you can just use the native executable, e.g.:
$DEEPSTATE/build/examples/IntegerOverflow --input_test_dir out
to run all the generated tests, or
$DEEPSTATE/build/examples/IntegerOverflow --input_test_files_dir out/IntegerOverflow.cpp/SignedInteger_AdditionOverflow --input_which_test SignedInteger_AdditionOverflow
to run the tests in one directory (in this case, you want to specify which test to run, also). You can also run a single test, e.g.:
$DEEPSTATE/build/examples/IntegerOverflow --input_test_file out/IntegerOverflow.cpp/SignedInteger_AdditionOverflow/a512f8ffb2c1bb775a9779ec60b699cb.fail--input_which_test SignedInteger_AdditionOverflow
In the absence of an
--input_which_test argument, DeepState defaults
to the first-defined test. Run the native executable with the
argument to see all DeepState options.
You can also try out Deepstate with Docker, which is the easiest way to get all the fuzzers and tools up and running on any system.
The build may take about 40 minutes, because some fuzzers require us building huge projects like QEMU or LLVM.
$ docker build -t deepstate-base -f docker/base/Dockerfile docker/base $ docker build -t deepstate --build-arg make_j=6 -f ./docker/Dockerfile . $ docker run -it deepstate bash [email protected]:~/deepstate$ cd build/examples [email protected]:~/deepstate/build/examples$ deepstate-angr ./Runlen [email protected]:~/deepstate/build/examples$ mkdir tmp && deepstate-eclipser ./Runlen -o tmp --timeout 30 [email protected]:~/deepstate/build/examples$ cd ../../build_libfuzzer/examples [email protected]:~/deepstate/build_libfuzzer/examples$ ./Runlen_LF -max_total_time=30 [email protected]:~/deepstate/build_libfuzzer/examples$ cd ../../build_afl/examples [email protected]:~/deepstate/build_afl/examples$ mkdir foo && echo x > foo/x && mkdir afl_Runlen2 [email protected]:~/deepstate/build_afl/examples$ $AFL_HOME/afl-fuzz -i foo -o afl_Runlen -- ./Runlen_AFL --input_test_file @@ --no_fork --abort_on_fail [email protected]:~/deepstate/build_afl/examples$ deepstate-afl -o afl_Runlen2 ./Runlen_AFL --fuzzer_out
Check out docs folder:
All accepted PRs are awarded bounties by Trail of Bits. Join the #deepstate channel on the Empire Hacking Slack to discuss ongoing development and claim bounties. Check the good first issue label for suggested contributions.
We have not yet applied DeepState to many targets, but it was responsible for finding the following confirmed bugs (serious faults are in bold):
DeepState is released under The Apache License 2.0.