Custom Load Shedding for Non-Cooperative Network Monitoring Applications

Network monitoring applications are prone to continuous overload situations due to the extreme and highly variable data rates of network traffic, traffic anomalies and network attacks. These overload situations can have a severe and unpredictable impact on the accuracy of monitoring applications, right when they are most valuable to network operators. Load shedding via traffic sampling has recently been proposed as an effective alternative to overprovisioning for efficiently handling overload situations in network monitoring. However, not all monitoring applications are robust against sampling and often other techniques can be devised to shed load more effectively. In order to provide a generic solution for arbitrary applications, we propose a load shedding scheme that, besides supporting standard sampling techniques, allows network monitoring applications to define custom load shedding methods. The main novelty of our approach is that the monitoring system can delegate the task of shedding excess load to applications in a safe manner and still achieve robustness in the presence of non-cooperative and selfish users. We use real-world packet traces and deploy a real implementation of our load shedding scheme in a large university network in order to show the robustness of our monitoring system against deliberate traffic anomalies and queries that fail to shed load correctly. Our results also show that the monitoring system is able to preserve a high degree of accuracy and fairness during extreme overload conditions.