• Management
Skip to end of metadata
Go to start of metadata

Management and Measurements

1. Introudction

This document describes an OpenFlow measurement infrastructhre that has been deployed at Stanford. In typical OpenFlow network, there are some important performance metrics, that was not particually interested in the legacy netwrok, such as, flow setup time, the num ber of active flows, flow arrival rate, etc. There is also software compornents introduced in OpenFlow network, such as, OpenFlow controller and FlowVisor. The infrastructure measure those metrics and present them to be useful for network operators to monitor and debug OpenFlow networks.

2. Overview

Fig.1 illustrates the overall architecture of the measurement infrastucture. It consists of three main compornents. (1) OpenFlow measuremnt boxes (2) Collector scripts and (3) Graphing Scripts.

  1. Measurement Box: OpenFlow measuremnt boxes are small-factor linux PCs equipped with multiple ethernet interfaces. They are distributed in OpenFlow network to measure the datapath performance, such as, end-to-end flow setup time, round trip delay and packet loss.
  2. Collector Scripts: Collector scripts consists of scripts that gather measurement data from the measurement boxes and also monitor the status of the components, such as, OpenFlow switches, OpenFlow controller software. All the corrected data are stored in the database as a time synced time-series.
  3. Graphing Scripts: Graphing scripts generates two types of graphs for each performance metric, time-series and CDF(cumulative distribution function), for a specified time period. Time-series graphs are useful for debugging. The operator can correlate multiple metrics (in time) to get a hint what is the root cause of the issue (we'll talk about this in detail later). CDF is useful to judge the overall performance in a certin time period. In Stanford, we show the past 6 hours of all metrics in real time in a single screen to debug/monitor network in regular basis. We also generate graphs specifiing past 7 days for weekly performance reports. graphs specifiing past 7 days for weekly performance reports.

3. Measurement Box

Any regular Linux PC can be used as a measurement box. We consume two ethernet ports per OpenFlow network(i.e., OpenFlow VLAN) per switch. If you like to monitor multiple OpenFlow networks and/or multiple switches that are located closely, you probably prefer a a single PC with many ethernet ports (rather than having lots of PCs). In Stanford, we use a small-factor linux PC equipped with eight ethernet interfaces and we distribute 8GB flash image to make them use as a measurement box. To buy this hardware, please refer to Appendix.

Measurement box measures three different performance, Flow setup time, RTT and Wget delay.

  • Flow setup delay (per switch): A typical OpenFlow controller installs a flow reactively, i.e., the first pakcet of a flow goes to the controller and the controller installs the flow entries along the path as well as delivers the packets to the destination. We refer this delay as flow setup time. It is a unique performance metric in OpenFlow network and arguably one of the most important metric to monitor, because the process of flow setup involves almost all the compornents in OpenFlow network. To monitor flow setup delay, we use 'ping' with a longer interval than a flow expiriation time. SNAC controller use the flow expiration time of 5 sec. So we use ping with 10sec interval to measure this.
  • RTT time to a target IP address : This is an end-to-end ping RTT (and loss rate) to specific IP address. It is good to specify an IP address of the host that is located outside of OpenFlow network so that we can monitor the end-to-end performance.
  • Web browsing delay (Wget delay): We picked Web browsing delay as a user-perceived performance of the network. We use 'wget' command to measure the time for a browser to get an entire weg page (including embedded images). From the OpenFlow network point of view, this involves a burst of flow setups, whose delay may be different from the single flow setup delay measured in flow setup delay.

3.1. Setting up Measurement Box

3.1.1. Overview

Fig. 2 illustrates the internal configuration of a measurement box. We use eth0 for control access to the host OS (base Linux). Linux namespace is used to create 'virtual host' on each interface from eth1 to eth7, which allows us to use each interface as an indivual Linux host and let them ping each other (the ping packet is actually sent out from the box rather than forwarded internally). You can login from the host OS (base Linux) to the virutual machine via ssh.

Fig.2: Measurement Box

3.1.2. Creation of Flash Image

Downlaod 8GB flash image from this link and unzip and then write to your 8GB flash as follows.$ md5sum OFMbox.r0.2.img.gz
2278e35ecce67ead73f114dd994d3ab7 OFMbox.r0.2.img.gz
$ unzip OFMbox.r0.2.img.gz
$ sudo dd if=OFMbox.r0.2.img of=/dev/

Unknown macro: {your flash device name}

bs=20M
(you can login with username root, password root. Please change it once you boot the box).

3.1.3. Example Network

We explain how to configure the measurement box using an example. Fig. 3 is an example network configuration.
http://yuba.stanford.edu/foswiki/pub/OpenFlow/Deployment/Measurement/Measurement-Example.jpgFig.3: Example Measurement Configuration
There are two OpenFlow switches A and B, and two OpenFlow VLANs (vlan 100 and vlan 200). We setup the measurement box to monitor the following switch/vlan combination.

SWITCH

VLAN

Ports on the measurement box

OpenFlow Switch A

100

eth1 and eth2

OpenFlow Switch B

100

eth3 and eth4

OpenFlow Switch B

200

eth5 and eth6

Here is the IP address assignent of each interface of the measurement box

Interface

VLAN

IP

mask

Gateway

Configuration File

eth0

non-OF

192.168.0.1

255.255.255.0

192.168.0.1

/etc/network/interfaces

eth1

100

192.168.100.101

255.255.255.0

192.168.100.254

/etc/ofm.conf

eth2

192.168.100.102

eth3

192.168.100.103

eth4

192.168.100.104

eth5

200

192.168.200.105

255.255.255.0

192.168.200.254

eth6

192.168.200.106

Here are the list of measurement that each interface will do

Type of the measurement

Target

How

Flow Setup Delay

OpenFLow A, VLAN100

10 sec interval ping between eth1 and eth2

Flow Setup Delay

OpenFlow B, VLAN100

10 sec interval ping between eth3 and eth4

Flow Setup Delay

OpenFlow A, VLAN200

10 sec interval ping between eth4 and eth5

RTT

OpenFlow A, VLAN100

Ping to between eth1 and gateway

RTT

OpenFlow B, VLAN100

Ping to gateway eth3 and gateway

RTT

OpenFlow B, VLAN200

Ping to gateway eth5 and gateway

Web browsing delay

OpenFLow A, VLAN100

Recursive wget to certain URL from eth1

Web browsing delay

OpenFLow B, VLAN100

Recursive wget to certain URL from eth3

Web browsing delay

OpenFLow B, VLAN200

Recursive wget to certain URL from eth5

3.1.4. Configuration

Follow Step 1-5 to configure a measurement box. The example configuration is the one explained in 3.1.4.

Step1. Edit /etc/network/interfaces file to configure IP setting for eth0 (regular IP setting in Debian Linux).auto eth0
iface eth0 inet static
address 192.168.0.1 netmask 255.255.255.0
gateway 192.168.0.254
Step2. Edit /etc/ofm.conf to configure all other interfaces and measurement setup.

Here is the parameters needs to be configured.

  • BASEDIR, DATADIR, OFMBIN .. don't need to change
  • INTERFACES: List of interfaces (eth*) to be used
  • SWITCHES: List of interfaces used to measure RTT and Web browsing performance
  • PINGx: Ping taget for VLAN x (referred later)
  • VMIPx: IP address for interface ethx
  • VMMASKx: IP mask for interface ethx
  • VMGWx: IP gateway for interface ethx
  • SWNAMEx: switch nickname to be measured by interface x
  • SWVLANx: vlan number for interface x
  • PINGTGTx: ping target (for RTT measurement) for interface ethx
  • WGETTGTx: wget target URL (for Web delay measurement) for interface ethx

Here is an example ofm.conf that is for the configuration explained in 3.1.3.

      1. OpenFlow Measurement Box Configuration
        BASEDIR="/ofm"
        DATADIR="$BASEDIR/data"
        OFMBIN="$BASEDIR/bin"
        ###############################
        INTERFACES="1 2 3 4 5 6"
        SWITCHES="1 3 5"
        PINGTGT100="172.24.74.229"
        PINGTGT200="172.27.69.1"
        WGETTGT="http://testurl"

VMIP1=192.168.100.101
VMMASK1=255.255.255.0
VMGW1=192.168.100.254
SWNAME1=switchA
SWVLAN1=100
PINGTGT1=$PINGTGT100
WGETTGT1=$WGETTGT

VMIP2=192.168.100.102
VMMASK2=255.255.255.0
VMGW2=192.168.100.254

VMIP3=192.168.100.103
VMMASK3=255.255.255.0
VMGW3=192.168.100.254
SWNAME3=switchB
SWVLAN3=100
PINGTGT3=$PINGTGT100
WGETTGT3=$WGETTGT

VMIP4=192.168.100.104
VMMASK4=255.255.255.0
VMGW4=192.168.100.254

VMIP5=192.168.200.205
VMMASK5=255.255.255.0
VMGW5=192.168.200.254
SWNAME5=switchB
SWVLAN5=200
PINGTGT5=$PINGTGT200
WGETTGT5=$WGETTGT

VMIP6=192.168.200.206
VMMASK6=255.255.255.0
VMGW6=192.168.200.254
Step3. Edit crontab file

All the measurements are invoked from cron. /ofm/bin/gencrontab.sh command will automatically create a crontab file using /etc/ofm.conf file. Open crontab file by 'crontab -e' and paste the output of gencrontab.sh.0 * * * * ssh -o StrictHostKeyChecking=no vm1 /ofm/bin/ping10.sh /ofm/data/vm1/fsetup/data switchA 100 192.168.100.102
0 * * * * ssh -o StrictHostKeyChecking=no vm3 /ofm/bin/ping10.sh /ofm/data/vm3/fsetup/data switchB 100 192.168.100.104
0 * * * * ssh -o StrictHostKeyChecking=no vm5 /ofm/bin/ping10.sh /ofm/data/vm5/fsetup/data switchB 200 192.168.200.206
0 * * * * ssh -o StrictHostKeyChecking=no vm1 /ofm/bin/ping.sh /ofm/data/vm1/rtt/data switchA 100 172.24.74.229
0 * * * * ssh -o StrictHostKeyChecking=no vm3 /ofm/bin/ping.sh /ofm/data/vm3/rtt/data switchB 100 172.24.74.229
0 * * * * ssh -o StrictHostKeyChecking=no vm5 /ofm/bin/ping.sh /ofm/data/vm5/rtt/data switchB 200 172.27.69.1
0,5,10,15,20,25,30,35,40,45,50,55 * * * * ssh -o StrictHostKeyChecking=no vm1 /ofm/bin/wget.sh /ofm/data/vm1/wget/data switchA 100 http://testurl
1,6,11,16,21,26,31,36,41,46,51,56 * * * * ssh -o StrictHostKeyChecking=no vm3 /ofm/bin/wget.sh /ofm/data/vm3/wget/data switchB 100 http://testurl
2,7,12,17,22,27,32,37,42,47,52,57 * * * * ssh -o StrictHostKeyChecking=no vm5 /ofm/bin/wget.sh /ofm/data/vm5/wget/data switchB 200 http://testurl
Step 4. Reboot the box (run `shutdown -r now`)

Step 5. Run /ofm/bin/runnow.sh command. Because some of the measuremnts are only invoked at zero o'clock, we kick off the measurement now. (This can be done by /etc/rc.local file).$ /ofm/bin/runnow.sh

4. Collector Scripts

4.1. Installation

Collector scripts can run any Linux host. They were invoked from cron and gather measurement data from the measurement boxes or collect info from OpenFlow switches (via SNMP) or SNAC (via RESTful interface).

Here is the pre-requisite software installed in the server

  • Python
  • NetSNMP (snmpwalk program)
  • wget
  • rsync
  • awk

After installing the above prerequisites, download the script from this link and expand it. Here we assume all the data is stored at /of-measurement (in the following, we refer it as OFMROOT).
tar zxvf of-measurement.tar.gz
It will creates the following directoriesOFMROOT
+--conf

+-- ofm-server.conf

+-- bin

+-- cpu.sh

+-- flow.sh

+-- traffic.sh

+-- get-cpu.sh

+-- get-traffic.sh

+-- get-cpu.sh

+-- getdata-ofmbox.sh

+--cpu

+-- data

+--traffic

+-- data

+--fsetup

+-- data

+--rtt

+-- data

+--wget

+-- data

+--keys

+--id_rsa.ofmbox

+--rtplot
+--bin

+-- cpu-plot.sh

+-- flow-plot.sh

+-- flow_rate-plot.sh

+-- fsetup-plot.sh

+-- rtt-plot.sh

+-- wget-plot.sh

+-- traffic-plot.sh

+-- generic-plot-day.sh

+-- generic-plot-hr.sh

+--conf

+-- ofm-plot.conf

+--cache

+-lib

+--graph

+--template

+--html
+-- index-openflow.html
+-- wide-openflow.html

4.2. Configuration

There is an example configuration file in at

Unknown macro: {OFMROOT}

/conf/ofm-server.conf. Copy this to /etc dir (or make a symbolic link to this file) and then configure:

OFM_BASEDIR

Set this to the OFMROOT dir. Example:OFM_BASEDIR=/of-measurement
OFM_BOX_IPS

Set this to the list of IP adddresses of measurement boxes. Example:OFM_BOX_IPS="172.27.75.11 172.24.98.28 172.24.98.33 172.27.75.11 172.27.75.12 172.27.75.13 172.27.75.14 172.27.75.15"
OFM_CPU_TARGET

This parameter is specifying the list of switches to get CPU usage. Switches have to be configured to allow SNMP access from the server. Each item has switch_name:switch_vendor format, where switch_name is DNS resolvable switch name and switch_vendor is the vendor name (currently nec and hp are supported).OFM_CPU_TARGET="necsw5:nec hpsw2:hp hpsw3:hp necsw6:nec necsw:nec necsw2:nec necsw7:nec necsw4:nec cis-hpswitch-11:hp cis-hpswitch-22:hp cisx-hpswitch-01:hp cisx-hpswitch-11:hp cisx-hpswitch-21:hp cisx-hpswitch-31:hp"
OFM_TRAFFIC_TARGET

This parameter is specifying the list of switches to get the total traffic volume. Switches have to be configured to allow SNMP access from the server. Each item has switch_name:switch_vendor:number_of_ports:nickname format, where switch_name is DNS resolvable switch name, switch_vendor is the vendor name (currently nec and hp are supported), number_of_ports is the number of ports of the switch and nickname will be used for the file prefix to store the data (use switch_name).
OFM_TRAFFIC_TARGET="necsw4:nec:48:necsw4 necsw6:nec:48:necsw6 necsw5:nec:24:necsw5 hpsw2:hp:48:hpsw2 hpsw3:hp:48:hpsw3 necsw6:nec:48:necsw6 necsw:nec:24:necsw necsw2:nec:24:necsw2 necsw3:nec:48:necsw3 necsw7:nec:48:necsw7"
OFM_FLOW_TARGET and OFM_SNAC_LOCATION

Only available if you're using SNAC. OFM_SNAC_LOCATION is the server name (DNS resolvable name) where SNAC is running. OFM_FLOW_TARGET is the list of "switch_name:vlan" where switch_name is registered in SNAC and vlan is the VLAN ID of the switch.OFM_FLOW_TARGET="necsw6:74 hpsw3:74"
OFM_SNAC_LOCATION=openflow1.stanford.edu

4.2. Setup Crontab

Register the following entries to your crontab.
OFM_BASEDIR=(set your OFMROOT dir)
*/5 * * * * $OFM_BASEDIR/bin/getdata-ofmbox.sh > /dev/null 2>&1
*/5 * * * * $OFM_BASEDIR/bin//cpu/cpu.sh > /dev/null 2>&1
*/1 * * * * $OFM_BASEDIR/bin/flow.sh > /dev/null 2>&1
*/5 * * * * $OFM_BASEDIR/bin/traffic.sh > /dev/null 2>&1

5. Graphing Scripts

5.1. Configuration

There is an example configuration file at

/rtplot/conf/ofm-plot.conf. Copy this to /etc dir (or make a symbolic link to this file) and then configure:

OFM_BASEDIR

Set this to the OFMROOT dir (be consistent with ofm-server.conf). Example:OFM_BASEDIR=/of-measurement
OFM_RTPLOT_HOUR

Graphing scripts plots time-series graph (of many stats) of some time period. This parameter specify the time length (in hour). Default is 6 hours.
OFM_RTPLOT_HOUR=6
OFM_RTPLOT_CPU_SW

List of switch names to plot CPU graph (name has to be consistent with OFM_CPU_TARGET parameter in ofm-server.conf).
OFM_RTPLOT_CPU_SW="hpsw3 necsw6"
OFM_RTPLOT_TRAFFIC_SW

List of switch names to plot traffic graph (name has to be consistent with OFM_TRAFFIC_TARGET parameter in ofm-server.conf).
OFM_RTPLOT_TRAFFIC_SW="hpsw3 necsw6"
OFM_RTPLOT_FSETUP_SW, OFM_RTPLOT_RTT_SW, OFM_RTPLOT_WGET_SW and OFM_RTPLOT_FLOW_SW

List of switch names to flow setup time delay, rtt, wget delay and flow stats graphs. Format of each items has switch_name.vVLAN format, where switch_name has to be consistent with SWNAME? and SWVLAN? in ofm.conf in measurement box, respectively.
OFM_RTPLOT_FSETUP_SW="hpsw3.v74 necsw6.v74 nonof.v74"
OFM_RTPLOT_RTT_SW="hpsw3.v74 necsw6.v74 nonof.v74"
OFM_RTPLOT_WGET_SW="hpsw3.v74 necsw6.v74 nonof.v74"
OFM_RTPLOT_FLOW_SW="hpsw3.v74 necsw6.v74"

5.2. Setup Crontab

Register the following entries to your crontab.OFM_BASEDIR=(set your OFMROOT dir)
*/5 * * * * $OFM_BASEDIR/rtplot/bin/fsetup-plot.sh > /dev/null 2>&1
*/5 * * * * $OFM_BASEDIR/rtplot/bin/rtt-plot.sh > /dev/null 2>&1
*/5 * * * * $OFM_BASEDIR/rtplot/bin/wget-plot.sh > /dev/null 2>&1
*/5 * * * * $OFM_BASEDIR/rtplot/bin/cpu-plot.sh > /dev/null 2>&1
*/5 * * * * $OFM_BASEDIR/rtplot/bin/traffic-plot.sh > /dev/null 2>&1
*/5 * * * * $OFM_BASEDIR/rtplot/bin/flow-plot.sh > /dev/null 2>&1
*/5 * * * * $OFM_BASEDIR/rtplot/bin/flow_rate-plot.sh > /dev/null 2>&1

6. Debugging OpenFlow Networks

Under construction.

Appendix 1. Where to buy the hardware

We bought our measurement box from Soekris Enginnering. Net5501 motherboard comes with four port 10/100 ethenret ports. There is a kit comes with enclosure (see "net5501-70 Board and 1 Slot standard case" in this link). We added four port PCI ethernet card (lan1741), which is also sold inthis link). For 8GB flash CF, buy SanDisk brand Flash. We had difficulty to boot from CF cards of other brands.

  • No labels