Current publicly available OpenFlow controller benchmarks focus on measuring throughput, latency, and variation of controller's core IO and processing loop. Beacon offers a number of bundles offering advanced functionality such as topology discovery, layer 2 shortest path routing, web UI framework, etc, that are enabled in the default distribution. These are not required for learning switch-style operation, and should be turned off for benchmarking. The following configuration guidelines will yield the most accurate benchmarking results.
32 vs 64 bit OS
Beacon performs dramatically better under a 64 bit JVM running on a 64 bit OS, I've personally seen a factor of 2x+ improvement in throughput going from 32 to 64bit. Both the JVM and the core Java classes use a large number of 64 bit data fields, so performance is much better on 64bit systems. Do not use a 32 bit system for comparative benchmarks.
Use the most recent 6 series Oracle-based JRE/JDK, as of writing this is 6 update 38, you can find it here: http://www.oracle.com/technetwork/java/javase/downloads/index.html. Java 7 and OpenJDK are unsupported at this time and should be avoided for benchmarking. Use of older versions will result in measurably slower performance, please use the latest Oracle JVM.
If you want to bind Beacon to specific processor cores when using Linux, you can prepend the above command and use taskset.
If you are benchmarking for latency, Beacon has a specific mode that trades global throughput for improved latency, you can enable this by setting the following in your beacon.properties file:
It is generally advisable to set the threadCount parameter to match the number of cores in your system.
beacon/beacon.ini (Linux specific)
If you find the performance to have a high variance it is possible that the JVMs garbage collector is thrashing internally due to a lack of memory, it should on average be consuming only 1-2% of CPU time, tools such as JVisualVM can attach to the running JVM and show you what is happening. If you see a lot of CPU time being spent in the garbage collector try increasing -Xmx3000M to larger values (RAM permitting).
Feel free to contact me if you have any questions or comments.