The Relevance of Classic Fuzz Testing: Have We Solved This One?

As fuzz testing has passed its 30th anniversary, and in the face of incredible progress techniques tools, question arises if classic, basic technique is still useful applicable? In that tradition, we have updated tools scripts applied them to a large collection Unix utilities on Linux, FreeBSD, MacOS. before, our failure criteria was whether program crashed or hung. We found 9 crash hang out 74 15 78 12 76 A total 24 different failed across three platforms. note these rates are somewhat higher than previous 1995, 2000, 2006 studies reliability command line utilities. debugged each utility categorized causes failures. Classic categories failures, such as pointer array errors not checking return codes, were broadly present current results. addition, couple new failures appearing. examples illustrate programming practices allowed happen. side note, tested limited number available modern language (Rust) be no better standard ones.