Currently in Migration - Jason Edelman's Old Blog
  • Home
  • About
  • Contact

Open vSwitch 101

8/8/2013

8 Comments

 
I’ve written in the past about how the virtual switch is an SDN war zone.  Well, it still seems to be the early days for Software Defined Networking (SDN) no matter how much time goes by and I realized there isn't a whole lot of documentation out there, especially for the new guy or gal on the block, when it comes to Open vSwitch when compared to the vendor offerings from Cisco and VMware.  Over the coming weeks, I’m going to hope to write more about Open vSwitch, Linux networking, and Open Stack Networking (Neutron, formerly Quantum).  On that note, this post is meant to be an easy to read, longer than expected, concise introduction to Open vSwitch (OVS).
If there is any information that is incorrect or should be updated, please let me know.  I do not hide the fact that I’m new to Open vSwitch myself.

If you’re even reading this blog, you’ve probably at least heard of Open vSwitch.   Yes, it is an open source virtual switch.  And when comparing it to the Cisco Nexus 1000V, the VMware standard (local) vSwitch, or the VMware distributed vSwitch, it is really more like the VMware standard switch.  There is no central controller, configuration manager, or virtual supervisor module coordinating between Open vSwitches on different physical hosts.  It can run on Linux based hypervisors including KVM, Xen, Virtual Box, etc.  Check out the official FAQ for all that it supports.

From an architectural standpoint, there are three main components of Open vSwitch.

  • Kernel module (Fast path) – I would equate the kernel module to ASICs on a hardware switch.  It is where all of your packet processing takes place, i.e. data plane.  The Virtual Ethernet Module (VEM) in the Nexus 1000V would be comparable to the kernel module in Open vSwitch.

  • vSwitch daemon (ovs-vswitchd) – also called a controller in some documentation, is the Linux process that runs in user space (not the kernel) that dictates how the kernel  module will be programmed – think of this as a locally significant control plane.  This daemon runs on every physical host.  Because I am seeing this called a controller in some docs, it is important to understand, this is not a SDN or OpenFlow controller by any means and it’s just a controller in the sense that it controls a local data plane module.  Update:  See discussion in the comments section below on ovs-switchd not being a control plane, but rather an extension of the data plane.

  • Database Server – More officially, it is called the Open vSwitch Database (OVSDB) server.  If you choose to run the vSwitch daemon, you will need this database.  This is the database that will store all of your cool and working configurations.  Luckily this isn’t OpenStack and you don’t need to go out and learn MySQL or anything like that (I kid, I kid).  This server uses JSON for its database schema and it communicates externally and to the vSwitch daemon using a management protocol, namely OVSDB, which is using JSON-RPC.

Below is a high level view of the various components just described.  The control cluster shown at the top is not part of Open vSwitch (OVS) natively.  That would be an add-on, likely not open source, to allow far greater scale.
Picture
Source: http://openvswitch.org/slides/LinuxSummit-2011.pdf
Aside from the general architecture, here is some other good information worth pointing out.

  • You actually don’t require the vSwitch daemon to use the OVS kernel module.  If this tickles your fancy, you’ll be left with a dumb data plane that can’t even perform MAC learning.  Remember, the vSwitch daemon is the control plane of this switch.  But, you can use an OVS utility called ‘ovs-dpctl’ to manually program flows directly to the kernel.  You should only use this utility if you are not running the vSwitch daemon.  Update:  See discussion in the comments section below on ovs-switchd not being a control plane, but rather an extension of the data plane.
  • You do need the OVS kernel module, or some form of it, to run the vSwitch daemon.  This kernel module can run in user space as well still serving as the data plane.  It may be required if the hypervisor doesn't natively support OVS.
  • You do need the OVSDB-server to run the vSwitch daemon. 
  • If you aren’t using a management platform that currently uses the OVSDB protocol to communicate with the OVSDB-server to centrally manage OVS instances, which you are probably aren’t, unless you are using a VMware/Nicira solution, you can use an OVS utility called ‘ovs-vsctl’ to manually configure Open vSwitch.  Using this tool is directly interfacing with the OVSDB-server.  The db server directly updates the vSwitch daemon.   You can also use a utility called ‘ovs-ofctl’ to configure flows when using the OVS daemon and kernel module.   ‘ovs-ofctl’ is a general utility, not just for OVS, which can be used with any OpenFlow switch.
  • As I previously stated, the vSwitch daemon is not an OpenFlow/Controller, but guess what, it has the ability to speak OpenFlow.  This is how OVS plugs into SDN environments – each vSwitch daemon would communicate to the kernel module (southbound) and to an OpenFlow controller (northbound).  But in reality OpenFlow is the southbound protocol of the controller being used.  Confusing terminology if you aren't good with directions like me :).  Just having fun and consider this your gas station.
  • A single vSwitch daemon can control multiple logical and independent switches/bridges on the same physical server.  This would be the same as multiple VMware vSwitches on a single host.
  • Upon booting up a physical server, switch configurations are pulled from the OVSDB-server and into the vSwitch daemon – then the kernel module is programmed from this data.
  • Many of the start-ups bringing in all of the VC money in the network virtualization space are using Open vSwitch, but in different forms.  This includes using all three components of OVS, but with vendor-specific extensions for added functionality – call this OVS++, but others only leverage the data plane module (in kernel and/or user space), and then fully write their own equivalent to the vSwitch daemon and OVSDB-server.  


I hope to write more soon on related topics.  If you have any specific questions, feel free to write in and we can explore together.

Follow up post going into a bit more detail: Open vSwitch 201 & 301

Thanks,
Jason

Twitter: @jedelman8


8 Comments
Ivan Pepelnjak link
8/8/2013 05:07:12 am

Just a few nits to pick:

* ovs-vswitchd is an OpenFlow client (or agent, whatever the terminology);
* it has no real control plane (ovsdb-server is the closest OVS gets to a control plane), but works together with the kernel module to implement the data plane.

Think of the kernel module as fast switching and ovs-vswitchd as process switching. Whenever a flow is not in a fast switching cache, the packet gets punted to the process switching (remember MLS?).

Here's a blog post I wrote about this topic a while ago:

http://blog.ipspace.net/2013/04/open-vswitch-under-hood.html

BTW, most implementations (apart from NVP) use ovs-vsctl to configure the ovsdb-server, including the Open vSwitch Quantum plugin. Yeah, imagine that - they wrote a python agent that runs on the hypervisor host and executes ovs-vsctl commands ... but then there are other people writing OpenFlow controllers in JavaScript, so maybe I shouldn't complain :D

Looking forward to future posts!
Ivan

Reply
Jason link
8/8/2013 06:38:30 am

Hey Ivan,

Thanks for the comments. I definitely agree that there is an OpenFlow agent in ovs-vswitchd, but it's more than just being an agent, isn't it? It seems it is giving the intelligence of MAC learning (as an example), but also serves as an OF agent should you choose to use it. You even reference it as a control plane, if you wish, in your post. I also recall reading a thread on the OVS website that had stated if the ovs-vswitchd process is killed, everything dies even if OpenFlow is not being used. That told me ovs-vswitchd is performing MAC learning (slow path for first packet) to program the kernel.

For the fast path/ slow path, fully agree. It seems I even commented on your post! VSG is a great analogy here for network folks as you used in your post, but I didn't use it b/c my post was getting too long already.

I could have simply said the ovs-switchd has an OF agent included, but I had too much fun with directional jokes. :) Need to work on being short and to the point!

Also, it is interesting other implementations are just using ovs-vsctl. Are you aware of any others besides the OVS Quantum plugin using python scripts? Other vendors seem to be scrapping vswitchd/ovsdb.

Thanks again,
Jason

Reply
Ivan Pepelnjak link
8/9/2013 12:24:56 am

Hi!

MAC learning in bridging is data-plane functionality. The only control-plane protocol ovs-vswitchd runs is LACP.

As for "if it dies, everything goes with it" - kernel module does solely flow-based forwarding and ovs-vswitchd is consulted for every new flow. If it dies, you can't establish a single new TCP (or UDP) session through the OVS.

Finally, thanks for spotting the bug in my blog post. Fixed ;)

Ivan

Reply
Jason link
8/9/2013 02:40:40 am

I had another exchange with Ivan because I was still having a gripe with this. Turns out my issue may be with terminology. I was trying to make it black and white for control vs. data plane separation, but clearly it's not that easy in certain designs.

The last response Ivan gave me was this:
"MAC learning is done on-the-fly (by gleaning the contents of the forwarded packets), so it's clearly a data plane functionality regardless of where it's done.

Remember: if you punt packets to the controller, you're actually offloading part of your data plane to the controller (even though it's called CONTROLler)."

Key part to take note of is the line that states, "offloading part of your data plane to the controller." This helped me --- hope it helps others as well.

-Jason

Bhargav Bhikkkaji
3/31/2014 05:22:04 am

**You do need the OVS kernel module, or some form of it, to run the vSwitch daemon

I think one does not need OVS kernel module to run vSwitch daemon.

http://git.openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=INSTALL.userspace;hb=HEAD

Reply
Jason Edelman link
5/23/2014 04:20:06 pm

Hi Bhargav,

Thanks for sharing the link, but that's why I said "or some form of it." I was alluding to running OVS on a VM solution (as some network virtualization vendors do and did).

Good to know it can run fully in user space too without a VM-type solution :)

Reply
Steven Iveson link
6/4/2014 11:05:30 pm

Great article.

Love the whole control/management/data plane debate, I feel a blog post coming on. Cheers

Reply
Pratik
3/15/2015 10:15:04 pm

Hey,
I am newbie as well as u to this OpenvSwitch (OVS). i underrstood your blog but i've some questions related to OVS.
Can you explain me that how OVSDB connects to OVS relay agent and parse the data to the vSwitch daemon using OF.??

Reply



Leave a Reply.

    Author

    Jason Edelman, Founder of Network to Code, focused on training and services for emerging network technologies. CCIE 15394.  VCDX-NV 167.


    Enter your email address:

    Delivered by FeedBurner


    Top Posts

    The Future of Networking and the Network Engineer

    OpenFlow, vPath, and SDN

    Network Virtualization vs. SDN

    Nexus 7000 FAQ

    Possibilities of OpenFlow/SDN Applications 

    Loved, Hated, but Never Ignored #OpenFlow #SDN

    Software Defined Networking: Cisco Domination to Market Education

    OpenFlow, SDN, and Meraki

    CAPWAP and OpenFlow - thinking outside the box

    Introduction to OpenFlow...for Network Engineers


    Categories

    All
    1cloudroad
    2011
    2960
    40gbe
    7000
    Arista
    Aruba
    Big Switch
    Brocade
    Capwap
    Christmas
    Cisco
    Controller
    Data Center
    Dell Force10
    Embrane
    Extreme
    Fex
    Hadoop
    Hp
    Ibm
    Isr G2
    Juniper
    Limited Lifetime Warranty
    Meraki
    Multicast
    N7k
    Nexus
    Nicira
    Ons
    Opendaylight
    Openflow
    Openstack
    Presidio
    Qsfp
    Quick Facts
    Routeflow
    Sdn
    Sdn Ecosystem
    Security
    Ucs


    Archives

    May 2015
    April 2015
    February 2015
    January 2015
    December 2014
    November 2014
    October 2014
    September 2014
    August 2014
    June 2014
    May 2014
    March 2014
    February 2014
    January 2014
    December 2013
    November 2013
    October 2013
    September 2013
    August 2013
    July 2013
    June 2013
    May 2013
    April 2013
    March 2013
    February 2013
    January 2013
    December 2012
    November 2012
    October 2012
    June 2012
    May 2012
    April 2012
    March 2012
    February 2012
    January 2012
    December 2011
    November 2011


    RSS Feed


    View my profile on LinkedIn
Photo used under Creative Commons from NASA Goddard Photo and Video