Copyright © 2012 Nicira, Inc.

Licensed under the Apache License, Version 2.0 (the ``License''); you may not use this file except in compliance with the License.  You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an ``AS IS'' BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the License for the specific language governing permissions and limitations under the License.

TITLE
*****

This manual is for Open vSwitch, version 1.9.

Introduction
************

Open vSwitch is an Ethernet switch implemented in software.  It is well suited to environments that call for basic switching features such as VLANs and bonding.  However, Open vSwitch is built as a platform to open up network forwarding to programmatic extension, so it really shines in environments that demand fast, flexible control over the network.

Open vSwitch is most often deployed today as a purely software switch, that is, on computers without hardware specialized for packet switching.  In this class of uses, Open vSwitch most often runs on virtual server hosts, to forward virtual machine network traffic to and from a physical network.  Virtualization is not a requirement: Open vSwitch can also forward traffic among physical interfaces in a ``bare metal'' environment.

Open vSwitch is designed and internally layered to be amenable for use within special-purpose switch hardware, and a number of network hardware manufacturers have used it for that purpose.  This allows Open vSwitch to control high-performance hardware switches as well as software switches, although hardware is inherently less flexible than software.

Open vSwitch runs on top of a wide variety of hardware, operating systems, and hypervisors, and it has been integrated with multiple cloud management systems.  Please refer to Open vSwitch and vendor documentation for details.

Architecture
============

This section describes the components of Open vSwitch at a high-level and how they interact with each other and with components not part of Open vSwitch.

The most important component of Open vSwitch is the virtual switch daemon, ovs-vswitchd.  This daemon implements Open vSwitch core switching and programmability functions.  In a software switch, this daemon is a ``soft real-time'' component that handles the first packet of each new flow that enters the network.  In a hardware switch, it instead configures the hardware's packet processing features.

The virtual switch daemon has two important external interfaces.  The first of these speaks a protocol called OVSDB to the second major Open vSwitch daemon, the ovsdb-server database server.  This daemon's configuration database tracks typically slowly changing management plane state, such as the set of configured switches and their ports.  The database server can also talk to controllers on remote servers over the same OVSDB protocol.

ovs-vswitchd's other major external interface is the control plane interface.  This interface allows controllers on remote servers to directly control the treatment of packets that arrive on the switches configured in the database.  Control can be reactive, with the controller deciding what should be done for each new flow as it arrives at the switch, or proactive, with the controller specifying in advance the treatment of every packet that might happen to arrive, or some combination.  Open vSwitch uses the OpenFlow protocol for this control plane interface.

Open vSwitch also includes a number of utility programs. ovs-vsctl is an interactive command-line interface to ovsdb-server for configuring and querying Open vSwitch configuration.  It is also used heavily in shell scripts that integrate Open vSwitch with hypervisor and cloud management systems.

ovs-ofctl is a command-line tool that speaks the OpenFlow protocol.  Unlike ovs-vsctl, which is often used in production, it is most useful for troubleshooting and prototyping.  We will often use it in examples in this book.

ovs-appctl is a command-line tool to send commands to ovs-vswitchd and ovsdb-server.  It is occasionally useful for special purposes, especially in testing and debugging.

Testing
*******

Testing is essential to writing reliable software, so as network operation becomes programmable, testing becomes essential to designing a network.  This chapter discusses different ways that one can test a network containing Open vSwitch elements.

The goal of testing is to model a network setup that one would use in production, to allow one to assess important properties of that setup, such as its correctness, performance, and scalability.  How the model is constructed determines which properties can be assessed and how accurately.  The model also determines properties of the testing itself, such as how easy or difficult the testing is to set up, to replicate, and to change, how expensive and time-consuming the testing is, and how easy or difficult the results are to interpret.

Physical Test Models
====================

The most obvious way to test Open vSwitch is to use a model that replicates the production system, or to use the production system for testing.  This form of modeling allows accurate performance measurements, but in most other respects, it is inconvenient.  Setting up a new network topology may require physically replugging network cables, which is time-consuming and error-prone.  Recovering from errors in the network programming, which are sure to be encountered during testing, may require setting up an extra out-of-band network. Finally, running multiple tests in parallel requires replicating the entire setup.

Simple setups, such as those that involve only one machine acting as a switch, can often be tested satisfactorily as physical models.  At least one Open vSwitch developer tests his changes this way exclusively.

Virtual Test Models
===================

Virtualization is one way to ease the inconvenience of testing with a physical model, that is, to use virtual machines instead of physical ones, and virtual switches instead of physical switches.  Because virtual machines and virtual switches are easier to reconfigure than physical machines and physical switches, this makes running different tests easier.  Virtual environments also reduce the need for physical hardware and at the same time, it is easier to replicate virtual environments, so taken together it becomes cheaper to run multiple tests in parallel.

A virtual environment doesn't perfectly reproduced the performance of the physical environment that it models.  A virtual environment usually performs slightly worse than the corresponding physical setup, so one may assume that, if the performance of the virtual model is adequate, then the performance of the corresponding physical setup would also be acceptable.  (This makes it difficult, of course, to have confidence in the physical performance if the virtual environment does not perform acceptably.)

One important way that a virtual environment can perform better than a physical one is that a virtual network within a single virtual server host may outperform a typical physical network.  The aggregate performance of virtual network connections between collocated VMs, for example, can easily exceed 5 Gb/s, whereas the most common physical Ethernet connections have a maximum bandwidth of 1 Gb/s.  At the same time, many implementations of quality-of-service (QoS) features to limit bandwidth, including the Linux QoS features that Open vSwitch has built-in support for configuring, are designed for physical Ethernet connections and do not work well within a virtual server host.

Using a virtual model can become more difficult if the network setup that one wishes to test uses virtualization itself.  Sometimes it is possible to ``factor'' the model so that a simple virtualization setup still adequately models the physical setup.  For example, one might decide that how virtual machines are assigned to physical hypervisors is not an important characteristic of the physical environment, so that one may model these virtual machines of the physical environment as virtual machines in the test model, leaving the hypervisors of the physical environment out of the model.

In situations where factoring the physical hypervisors out of the test model is not acceptable, one may resort to nested virtualization, that is, running hypervisors inside hypervisors. This decision is not to be made lightly, because nested virtualization has disadvantages beyond those of ordinary virtualization.  First, not all hypervisors support nested virtualization. (In my experience, nested virtualization works best with VMware ESX as the outer hypervisor.)  Second, nested virtualization compounds performance issues.  It is much more common to see poor performance and timeouts that cannot be reproduced in a corresponding physical setup in a virtual environment that includes nested hypervisors.

Dummy Test Models
=================

* Test setups. ** Physical ** Virtual *** start-test-vm **** non-persistent **** install ethtool, tcpdump, strace, valgrind, hping3, netperf (from non-free) **** set up serial port in inittab (just uncomment the T0:... line) **** runme, put this in /etc/rc.local and mark it executable: **** NFS setup (mount -o ro,udp,timeo=1 10.0.2.2:/home/blp/ovs /mnt) #!/bin/sh mount -t iso9660 /dev/sr0 /cdrom exec sh -x /cdrom/runme exit 0

** Nested virtual ** Dummy-based. ** test-vm based *** not for performance simulation (including QoS)

* Test techniques. ** ofproto/trace ** netdev-dummy/receive ** write some tools for arp, ping, more?

* Introduction to OpenFlow. * Packet processing pipeline. ** The ``normal'' action's pipeline. * Examples of proactive programming. ** Hub (flood) ** Virtual switch ** Ingress-only firewall * Limitations of proactive programming in OF1.0 ** Can't do MAC learning (solution: NX or reactive programming) ** Cartesian product is expensive (OF1.1+ or NX or reactive programming) * Reactive programming. ** Slow ** High latency ** Handles disconnection badly ** Hard to scale * Hybrid approaches ** Proactive with a few packets to controller ** Proactive with local controller

* Management plane. ** Ports and interfaces. ** Bonding. *** How to use through OpenFlow, and limitations. *** Future directions. *** How to emulate with OpenFlow? ** VLANs. *** Interaction with OpenFlow. ** MAC learning. *** Reading and flushing the MAC learning table. *** Preventing MAC learning. *** Interaction with bonding. *** ** Mirrors. *** Limitations with OpenFlow. *** How to emulate with OpenFlow (avoiding duplicates). ** Spanning tree. *** Limitations for nonstandard setups. ** CFM *** Tradeoffs. *** Caveats. ** Policing. ** QoS.

* Control plane programming techniques. ** Resubmit. ** Registers. *** NAND gate. ** Local controller.

* Proactive control plane programming. ** Resubmit. *** Firewall with ingress and egress rules. **** Handling IP fragments. ** Learning. *** A MAC-learning switch. **** Reading and flushing the MAC learning table. **** Monitoring changes to the MAC learning table. *** Stateful firewall. ** VXLAN control plane. ** Routing. ** Spoof prevention. *** IPv6. ** PVLAN.

* Reactive control plane programming? ** In-band control as example?

* In-band control. ** How it works. ** Limitations. ** Rolling your own.

* Optimization. ** Separate tables. *** Tables with a single type of flow (see learning). ** Minimizing types of flows. ** Minimizing lookups. ** Minimizing flow table changes. *** Avoiding timeouts.