VUzzer Applicationaware Evolutionary Fuzzing Sanjay Rawat Vivek Jain

VUzzer: Application-aware Evolutionary Fuzzing Sanjay Rawat, Vivek Jain, Ashish Kumar, Lucian Cojocar, Cristiano Giuffrida, and Herbert Bos Victor van der Veen

VUzzer Your takeaway message of today Smart fuzzing without symbolic execution Extract application features for meaningful mutation VUzzer 30 K inputs: 403 crashes AFL 30. 000 K inputs: 238 crashes VUzzer @ NDSS – March 1, 2017 2

PROBLEM STATEMENT What are we trying to solve anyway? VUzzer @ NDSS – March 1, 2017 3

AFL. . . read(fd, buf, size); if (buf[5] == 0 x. D 8 && buf[4] == 0 x. FF) // interesting code here else pr_exit(“Invalid file”); AFL will run for hours on this VUzzer @ NDSS – March 1, 2017 4

AFL. . . read(fd, buf, size); if (buf[5] == 0 x. D 8 && buf[4] == 0 x. FF) // interesting code here else pr_exit(“Invalid file”); AFL will run for hours on this 1. Has to figure out that offset 4 and 5 are of interest (where) VUzzer @ NDSS – March 1, 2017 5

AFL. . . read(fd, buf, size); if (buf[5] == 0 x. D 8 && buf[4] == 0 x. FF) // interesting code here else pr_exit(“Invalid file”); AFL will run for hours on this 1. Has to figure out that offset 4 and 5 are of interest (where) 2. Needs to guess 0 x. FFD 8 (what) VUzzer @ NDSS – March 1, 2017 6

More Problems Handling ‘Complex’ Code Structures • Magic bytes Certain values shall be placed at pre-determined offsets • Deeper execution Many inputs will end up in less interesting error-handling code • (multibyte) Markers Not at fixed offsets: if (strstr(&buf, “MAZE”)). . . VUzzer @ NDSS – March 1, 2017 7

VUZZER Our mutation-, coverage-based, greybox fuzzer VUzzer @ NDSS – March 1, 2017 8

VUzzer Where to mutate, what to insert • Evolutionary fuzzing Mutate/select most promising paths • Magic byte detection Find possible magic byte values to reach deeper into the binary • (limited) Input type detection Aid mutation by detecting input bytes of certain types (integers) Avoid non-scalable techniques • No symbolic execution • Limited use of Dynamic Taint Analysis VUzzer @ NDSS – March 1, 2017 9

Feature Extraction Data-flow features Control-flow features VUzzer @ NDSS – March 1, 2017 10

Feature Extraction Data-flow features Information about relationship between input data and program computations Extracted using static analysis / dynamic taint analysis • • Example: cmp instructions on x 86 • Offsets: which input bytes are compared against? (taint analysis) • Magic values: immediate operands for cmp (static analysis) Example: lea instructions • Integer types: is index operand tainted? Control-flow features VUzzer @ NDSS – March 1, 2017 11

Feature Extraction Data-flow features Control-flow features Information about importance of certain execution paths • Identify error-handling blocks (heuristics based) • Rank basic blocks to prioritize hard-to-reach code • Each basic block gets a weight depending on how deep it is nested • Error-handling blocks get a negative weight VUzzer @ NDSS – March 1, 2017 12

VUzzer @ NDSS – March 1, 2017 13

CMP Immediates Rank Basic Blocks VUzzer @ NDSS – March 1, 2017 14

Seed Inputs (known valid) 15

Dynamic Taint Analysis Seed Inputs (known valid) 16

Dynamic Taint Analysis + Error-handling code + Magic bytes + LEA offsets VUzzer @ NDSS – March 1, 2017 17

Fitness(executed code) High scores for inputs that execute highly ranked basic blocks

Dynamic Taint Analysis only when new code is covered Mutate and loop VUzzer @ NDSS – March 1, 2017 19

VUZZER VUzzer @ NDSS – March 1, 2017 20

Evaluation Darpa Cyber Grand Challenge • VUzzer / AFL • 13 binaries, 6 hours per binary • Far fewer VUzzer inputs VUzzer @ NDSS – March 1, 2017 AFL: 22 k inputs VUzzer: 400 inputs 21

LAVA-M Dataset • LAVA: inject hard-to-reach faults to (e. g. , ) evaluate fuzzers Program uniq base 64 md 5 sum Total Bugs FUZZER 28 44 57 who • Evaluation 2136 SES VUzzer 7 7 2 0 9 0 27 17 1 0 18 50 VUzzer hits significantly more bugs than • FUZZER: coverage-based • SES: symbolic execution / SAT-based VUzzer @ NDSS – March 1, 2017 22

Various Applications • Comparison against AFL on vanilla Ubuntu 14. 04 Program mpg 321 gif 2 png pdf 2 svg tcpdump tcptrace Djpeg • Evaluation Crashes (AFL) 19 7 0 Crashes (VUzzer) 337 127 13 0 238 0 3 403 1 Inputs (AFL) Inputs (VUzzer) 883 K 24 K 1. 84 M 43 K 923 K 5 K 2. 89 M 3. 29 M 35. 9 M 78 K 30 K 90 K More bugs with fewer inputs VUzzer @ NDSS – March 1, 2017 23

Various Applications • Comparison against AFL on vanilla Ubuntu 14. 04 Program mpg 321 gif 2 png pdf 2 svg tcpdump tcptrace Djpeg • Evaluation Crashes (AFL) 19 7 0 Crashes (VUzzer) 337 127 13 0 238 0 3 403 1 Inputs (AFL) Inputs (VUzzer) 883 K 24 K 1. 84 M 43 K 923 K 5 K 2. 89 M 3. 29 M 35. 9 M 78 K 30 K 90 K More bugs with fewer inputs VUzzer @ NDSS – March 1, 2017 24

Various Applications Evaluation • Comparison against AFL on vanilla Ubuntu 14. 04 • Crash faster • Consistent progress VUzzer @ NDSS – March 1, 2017 25

VUzzer Conclusion • Novel fuzzing technique based on evolutionary approach • Application-aware fuzzer by exploiting data-flow and control-flow features • Prioritize hard-to-reach code paths • Deprioritize error-handling code • Significantly more bugs with orders of magnitude fewer inputs in less time VUzzer @ NDSS – March 1, 2017 26

Open Source Final Remarks https: //github. com/vusec/vuzzer VUSec Project Page https: //vusec. net/projects/fuzzing s. rawat@vu. nl