To demonstrate that a compiler optimization, a memory management algorithm, or a synchronization technique is useful, a VM researcher needs benchmarks that demonstrate the desired behavior and, at the same time, capture representative aspects of real-world applications. During the last decade, multiple new programming paradigms have appeared on the JVM, including functional programming, big-data processing, parallel and concurrent programming, message passing, stream processing, and machine learning. The JVM has evolved as a platform, too: new features, such as e.g. method handles, variable handles, the invokedynamic instruction, lambdas, atomic and relaxed memory operations, present new challenges for JIT compilers and runtime environments. Existing benchmark suites do not capture these new features, because they were made in a time when such workloads did not exist.

To enable the pending Renaissance for VM and compiler research, new benchmarks are required. The Renaissance Suite is an ongoing, open-source effort to collect representative real-world workloads and advance the research and development of VMs. In this talk, we compare Renaissance against existing benchmark suites and we show how our benchmarks have helped us identify new optimizations for the Graal JIT compiler. We conclude by summarizing the future goals of the Renaissance Suite.

Lubomír is an assistant professor at the Department of Distributed and Dependable Systems, Charles University, Prague, Czech Republic. His primary research interests include performance-related topics focusing on performance evaluation, testing, and monitoring. In addition to performance, his research interests include also dynamic program analysis, with specific focus on making the programs running on the Java (and Dalvik) Virtual Machines more observable. He holds an MSc. from the Czech Technical University in Prague, and a PhD from the Charles University in Prague. He is a member of the ACM.