OFBench: Performance Test Suite on OpenFlow Switches

Performance issues of OpenFlow switches are attracting a lot of attention owing to the potential large-scale deployment of software-defined devices. This paper presents theOFBench which is an automatic test suite for evaluating the performance of OpenFlow switches. The design, as part of the Automation Control Test System (ACTS) development, is based on a controller-agent architecture which allows the development of test cases that are written in a high-level script language. In addition to the end-toend measurement methodology, novel methods are proposed to further profile the internal performance metrics, which are difficult to get due to the black-box nature of the device under test. The prototype of this suite currently comprises five test cases to evaluate five performance metrics, which are action time, pipeline time, buffer size, pipeline efficiency, and timeout accuracy. OpenFlow switches are evaluated and three issues are observed associated with switches during the testing. First, some switches may not be well implemented in the design of apply-action instructions. Second, some switches suffer from random crashes with a high volume of bursty packet-in traffic. Finally, the timer of idle-timeout is often not reset properly with matching flow entry.