Write a Blog >>
SPLASH 2019
Sun 20 - Fri 25 October 2019 Athens, Greece
Wed 23 Oct 2019 14:00 - 14:22 at Olympia - Testing

Regression test selection (RTS) aims to speed up regression testing by rerunning only tests that are affected by code changes. RTS can be performed using static or dynamic analysis techniques. Our recent study showed that static and dynamic RTS perform similarly for medium-sized Java projects. However, the results of that recent study also showed that static RTS can be unsafe, missing to select tests that dynamic RTS selects, and reflection was the only cause of unsafety observed among the evaluated projects.

In this paper, we investigate five techniques—three purely static techniques and two hybrid static-dynamic techniques—that aim to make static RTS safe with respect to reflection. We implemented these reflection-aware (RA) techniques by extending the reflection-unaware (RU) class-level static RTS technique in a tool called STARTS. To assess these RA techniques, we compared their end-to-end times with the RU technique, and with RetestAll which reruns all tests after every code change. We also compared safety and precision of the RA techniques with Ekstazi, a state-of-the-art dynamic RTS technique.

Our evaluation on 1173 versions of 24 open-source Java projects showed negative results. The RA techniques improve the safety of RU but at very high costs. The purely static techniques were safe in our experiments but decrease the precision of the RU technique, with end-to-end time at best 85.8% of RetestAll time, compared to 69.1% for RU. A hybrid static-dynamic technique provides better precision, is safer than RU and incurs lower end-to-end time—75.8% of RetestAll, but can still be unsafe in the presence of test-order dependencies. Our study highlights the challenges involved in making static RTS safe with respect to reflection.

This program is tentative and subject to change.

Wed 23 Oct

splash-2019-oopsla
14:00 - 15:30: OOPSLA - Testing at Olympia
splash-2019-oopsla14:00 - 14:22
Talk
August ShiUniversity of Illinois at Urbana-Champaign, Milica Hadzi-TanovicTechnische Universitat Munchen, Lingming ZhangThe University of Texas at Dallas, Darko MarinovUniversity of Illinois at Urbana-Champaign, Owolabi LegunsenUniversity of Illinois at Urbana-Champaign
splash-2019-oopsla14:22 - 14:45
Talk
Burcu Kulahcioglu OzkanMax Planck Institute for Software Systems (MPI-SWS), Rupak MajumdarMax Planck Institute for Software Systems (MPI-SWS), Simin OraeeMax Planck Institute for Software Systems (MPI-SWS)
splash-2019-oopsla14:45 - 15:07
Talk
Dowon SongKorea University, Myungho LeeKorea University, Hakjoo OhKorea University
splash-2019-oopsla15:07 - 15:30
Talk
Ranadeep BiswasIRIF, University Paris Diderot & CNRS, Constantin EneaIRIF, University Paris Diderot & CNRS