Tool Competition

This year as well we are pleased to announce the nineth edition of the unit testing tool competition. The competition has the goal to experiment with testing tools for a diversified set of traditional and emerging systems and domains.

Java tool competition: As for recent years, we invite researchers to participate in the competition with their unit test generation tool for Java. Tools will be evaluated against a benchmark with respect to code coverage and mutation score.

Java test example

Cyber-physical systems (CPS) testing competition: In addition to the traditional Java tool competition, we also organize a CPS testing competition on self-driving cars simulation environments. Specifically, in collaboration with the BeamNG research team, this competition focuses on the generation of scenarios using BeamNG self-driving cars simulator.

Beam-NG Visual 1
Beam-NG Visual 2

For future editions of the SBST Tool competition (e.g., from 2022) we foresee the addition of a further "Python testing tool competition" where:

  • an infrastructure will be provided to generate the tools
  • it is conducted in an evaluation setting similar to the one performed for the traditional Java testing tool competition

How to participate in the two competitions?

Java tool competition:

The infrastructure concerning the Java tool competition is available on GitHub and can also be tried using Docker: https://github.com/JUnitContest/junitcontest. It is important to note that some changes are expected to the infrastructure during the end of 2020 and during the beginning of 2021.

To participate:

CPS testing competition

The infrastructure concerning the CPS tool competition can be found here: https://github.com/se2p/tool-competition-av.

To participate:

Important Dates

  • Tool submission: February 12
  • Benchmark results communicated to authors: March 2
  • Submission of camera-ready paper: March 12

Organizing Committee

Thumbnail Image

Sebastiano Panichella

Zurich University of Applied Science (ZHAW)

Thumbnail Image

Alessio Gambi

Passau University, Germany

Thumbnail Image

Fiorella Zampetti

University of Sannio

Thumbnail Image

Vincenzo Riccio

University of Lugano