Mutation Testing

🥇

Dear Participants, we now have the final results for the competition. Based on the mutation analysis results, we congratulate the team developing BandFuzz for having achieved both the highest number of mutant kills in total, as well as the highest average mutation score. We further note that BandFuzz also achieved the highest coverage of mutants.

With Best Regards,
SBFT Fuzzing Competition Team

The figure below demonstrates that BandFuzz has the highest number of mutations killed in mutation tests.

The figure below demonstrates that BandFuzz outperforms other fuzzers in almost every target.

The table below demonstrates that BandFuzz shows great improvement in mutation test.

The number of mutants each fuzzer killed for each benchmark:

Benchmark	bandfuzz	fox	libafl	libfuzzer	mystique	pastis	tunefuzz
freetype2_ftfuzzer	6814	5944	6172	5131	5900	6158	5924
jsoncpp_jsoncpp_fuzzer	1150	650	638	679	651	686	672
lcms_cms_transform_fuzzer	1709	1522	1547	1527	1570	1560	1461
libpcap_fuzz_both	2095	2443	1811	1636	2031	1852	2096
libxml2_xml	7259	6880	6160	6125	6212	6240	6582
re2_fuzzer	6651	3672	3635	3687	3650	3670	5967
stb_stbi_read_fuzzer	1597	1550	1562	1474	1592	1647	1643
zlib_zlib_uncompress_fuzzer	361	352	326	355	326	359	333