Machine Learning for Host-based Anomaly Detection

Machine Learning for Host-based Anomaly Detection by Gaurav Tandon Dissertation Advisor: Philip K. Chan, Ph.D. Anomaly detection techniques complement signature based methods for intrusion detection. Machine learning approaches are applied to anomaly detection for automated learning and detection. Traditional host-based anomaly detectors model system call sequences to detect novel attacks. This dissertation makes four key contributions to detect host anomalies. First, we present an unsupervised approach to clean training data using novel representations for system call sequences. Second, supervised learning with system call arguments and other attributes is proposed for enriched modeling. Third, techniques to increase model coverage for improved accuracy are presented. Fourth, we propose spatio-temporal modeling to detect suspicious behavior for mobile hosts. Experimental results on various data sets indicate that our techniques are more effective than traditional methods in capturing attack-based host anomalies. Additionally, our supervised methods create succint models and the computational overhead incurred is reasonable for an online anomaly detection system.