Write a Blog >>
SPLASH 2019
Sun 20 - Fri 25 October 2019 Athens, Greece
Wed 23 Oct 2019 14:22 - 14:45 at Olympia - Testing

Distributed and concurrent applications often have subtle bugs that only get exposed under specific schedules. While these schedules may be found by systematic model checking techniques, in practice, model checkers do not scale to large systems. On the other hand, naive random exploration techniques often require a very large number of runs to find the specific interactions needed to expose a bug. In recent years, several random testing algorithms have been proposed that, on the one hand, exploit state-space reduction strategies from model checking and, on the other, provide \emph{guarantees} on the probability of hitting bugs of certain kinds.

These existing techniques exploit two orthogonal strategies to reduce the state space: \emph{partial-order reduction} and \emph{bug depth}. Testing algorithms based on partial order techniques, such as RAPOS or POS, ensure non-redundant exploration of independent interleavings among system events by imposing an equivalence relation on schedules and ideally exploring only one schedule from each equivalence class. Techniques based on bug depth, such as PCT, exploit the empirical observation that many bugs are exposed by the clever scheduling of a small number of key events. They bias the sample space of schedules to only cover all executions of small depth, rather than the much larger space of all schedules. At this point, there is no random testing algorithm that combines the power of both approaches.

In this paper, we provide such an algorithm. Our algorithm, \emph{trace-aware} PCT (taPCTCP), extends and unifies several different algorithms in the random testing literature. It samples the space of low-depth executions by constructing a schedule \emph{online}, while taking dependencies among events into account. Moreover, the algorithm comes with a theoretical guarantee on the probability of sampling a trace of low depth—the probability grows exponentially with the depth but only polynomially with the number of racy events explored. We further show that the guarantee is optimal among a large class of techniques.

We empirically compare our algorithm with several state-of-the-art random testing approaches for concurrent software on two large-scale distributed systems, Zookeeper and Cassandra, and show that our approach is effective in uncovering subtle bugs and usually outperforms related random testing algorithms.

This program is tentative and subject to change.

Wed 23 Oct

splash-2019-oopsla
14:00 - 15:30: OOPSLA - Testing at Olympia
splash-2019-oopsla14:00 - 14:22
Talk
August ShiUniversity of Illinois at Urbana-Champaign, Milica Hadzi-TanovicTechnische Universitat Munchen, Lingming ZhangThe University of Texas at Dallas, Darko MarinovUniversity of Illinois at Urbana-Champaign, Owolabi LegunsenUniversity of Illinois at Urbana-Champaign
splash-2019-oopsla14:22 - 14:45
Talk
Burcu Kulahcioglu OzkanMax Planck Institute for Software Systems (MPI-SWS), Rupak MajumdarMax Planck Institute for Software Systems (MPI-SWS), Simin OraeeMax Planck Institute for Software Systems (MPI-SWS)
splash-2019-oopsla14:45 - 15:07
Talk
Dowon SongKorea University, Myungho LeeKorea University, Hakjoo OhKorea University
splash-2019-oopsla15:07 - 15:30
Talk
Ranadeep BiswasIRIF, University Paris Diderot & CNRS, Constantin EneaIRIF, University Paris Diderot & CNRS