Reliable detection of compressed and encrypted data

Abstract Several cybersecurity domains, such as ransomware detection, forensics and data analysis, require methods to reliably identify encrypted fragments. Typically, current approaches employ statistics derived from byte-level distribution, entropy estimation, However, modern content types use compression techniques which alter distribution pushing it closer the uniform distribution. The result is that exhibit unreliable encryption detection performance when compressed appear in dataset. Furthermore, proposed are typically evaluated over few fragment sizes, making hard assess their practical applicability. This paper compares existing statistical tests on a large, standardized dataset shows consistently fail distinguish both small large sizes. We address these shortcomings design EnCoD , learning-based classifier can data. evaluate of 16 different file sizes ranging 512B 8KB. Our results highlight outperforms by wide margin, with accuracy $$\sim 82\%$$ ∼ 82 % for fragments up 92\%$$ 92 8KB Moreover, pinpoint exact format given fragment, rather than performing only binary classification like previous approaches.