Towards Fingerprinting Malicious Traffic

The primary intent of this paper is detect malicious traffic at the network level. To this end, we apply several machine learning techniques to build classifiers that fingerprint maliciousness on IP traffic. As such, J48, Naı̈ve Bayesian, SVM and Boosting algorithms are used to classify malware communications that are generated from dynamic malware analysis framework. The generated traffic log files are pre-processed in order to extract features that characterize malicious packets. The data mining algorithms are applied on these features. The comparison between different algorithms results has shown that J48 and Boosted J48 algorithms have performed better than other algorithms. We managed to obtain a detection rate of 99% of malicious traffic with a false positive rate less than 1% for J48 and Boosted J48 algorithms. Additional tests have generated results that show that our model can detect malicious traffic obtained from different sources. c © 2011 Published by Elsevier Ltd.