Write a Blog >>
SPLASH 2019
Sun 20 - Fri 25 October 2019 Athens, Greece
Fri 25 Oct 2019 12:07 - 12:30 at Attica - Test Generation Chair(s): Sasa Misailovic

Despite much recent interest in randomised testing (fuzzing) of compilers, the practical impact of fuzzer-found compiler bugs on real-world applications has barely been assessed. We present the first quantitative and qualitative study of the tangible impact of miscompilation bugs in a mature compiler. We follow a rigorous methodology where the bug impact over the compiled application is evaluated based on (1) whether the bug appears to trigger during compilation; (2) the extent to which generated assembly code changes syntactically due to triggering of the bug; and (3) whether such changes cause regression test suite failures, or whether we can manually find application inputs that trigger execution divergence due to such changes. The study is conducted with respect to the compilation of more than 10 million lines of C/C++ code from 309 Debian packages, using 12% of the historical and now fixed miscompilation bugs found by four state-of-the-art fuzzers in the Clang/LLVM compiler, as well as 18 bugs found by human users compiling real code or as a by-product of formal verification efforts. The results show that almost half of the fuzzer-found bugs propagate to the generated binaries for at least one package, in which case only a very small part of the binary is typically affected, yet causing two failures when running the test suites of all the impacted packages. User-reported and formal verification bugs do not exhibit a higher impact, with a lower rate of triggered bugs and one test failure. The manual analysis of a selection of the syntactic changes caused by some of our bugs (fuzzer-found and non fuzzer-found) in package assembly code, shows that either these changes have no semantic impact or that they would require very specific runtime circumstances to trigger execution divergence.

Slides of OOPSLA presentation (compilerbugs-oopsla-19.pdf)3.29MiB

Fri 25 Oct

Displayed time zone: Beirut change

11:00 - 12:30
Test GenerationOOPSLA at Attica
Chair(s): Sasa Misailovic University of Illinois at Urbana-Champaign
11:00
22m
Talk
CLOTHO: Directed Test Generation for Weakly Consistent Database Systems
OOPSLA
Kia Rahmani Purdue University, Kartik Nagar Purdue University, Benjamin Delaware Purdue University, Suresh Jagannathan Purdue University
DOI Pre-print
11:22
22m
Talk
Coverage Guided, Property Based Testing
OOPSLA
Leonidas Lampropoulos University of Pennsylvania, University of Maryland, Michael Hicks University of Maryland, Benjamin C. Pierce University of Pennsylvania
DOI
11:45
22m
Talk
FuzzFactory: Domain-Specific Fuzzing with Waypoints
OOPSLA
Rohan Padhye University of California, Berkeley, Caroline Lemieux University of California, Berkeley, Koushik Sen University of California, Berkeley, Laurent Simon Samsung Research America, Hayawardh Vijayakumar Samsung Research America
DOI Pre-print
12:07
22m
Talk
Compiler Fuzzing: How Much Does It Matter?
OOPSLA
Michaël Marcozzi Imperial College London, Qiyi Tang Imperial College London, Alastair F. Donaldson Imperial College London, Cristian Cadar Imperial College London
Link to publication DOI Pre-print Media Attached File Attached