Currently in Migration - Jason Edelman's Old Blog
  • Home
  • About
  • Contact

SDN Applications – Going beyond RESTful APIs

10/30/2013

7 Comments

 
Yesterday was an interesting day in that I attended a full day ONUG academy session that was all about writing SDN applications.  Big thanks to Matt Davy and Chuck Black for leading the session.  While we weren’t hacking on code, there was a lot of discussion around APIs, network programmability, and the approach to take when building SDN applications [that leverage northbound APIs of a controller].  I’ve made it pretty public that I’ve been working with onePK building my own controller (using the term controller very loosely here) communicating directly with network devices as opposed to natively integrating with an existing controller like OpenDaylight, Floodlight, etc. and leveraging their northbound APIs.  
Picture
There are definitely different schools of thought on programming to a device directly (ala onePK) vs. solely programming to northbound interfaces on a controller --- that is a conversation for a different day, but please remember they are two different models.  I’m using onePK development as a foundation and an example to articulate northbound API requirements for controllers.   Combining the material from yesterday with what I’ve been doing with onePK has made me think on how to marry the two together in what I’d call an ideal platform to develop on in controller networks.  

onePK High Level

With onePK, you are able to build an application and leverage native C or Java libraries.  These libraries, call them APIs if you wish, allow you to extract data from a device.  You can extract what you want when you want.  For example, you can extract things like interfaces on the devices, interface status, serial numbers, CPU utilization, routing tables, and literally a plethora of other things.  This may seem trivial and some may say, how is that different than SNMP or screen scraping?   Well, the result is the same of course, but it is a programmatic interface working over a persistent TCP connection between the onePK application and the onePK agent loaded on Cisco network devices.  Does it matter it’s a persistent connection?  Maybe, maybe not, and it depends on the application consuming the data.  But remember APIs and programming are great for machine to machine interaction and CLI was meant for user interaction, so screen scraping will not scale, but for basic polling --- SNMP could work just fine.  Of course, onePK also allows you to make changes to the network device (not just extract/get data) although I’m not going to focus on that any more in this post.  

We now know onePK offers the ability to extract data, make changes to the network, but what about real time events that are needed to create this feedback loop everyone talks about?

Real Time Events

In the example I gave, the application is pulling or extracting data as needed.  It is pulling real time state information when the application needs it, but it’s still a pull model and a pull model is not event driven.  Ideally, the network device is also able to communicate real time state based on events back to the application.  Just so happens onePK is able to do this as well.  I was able to test this leveraging the Java libraries by configuring “listeners” on the network devices.  Example I tested included CLI, CDP, and Interface listeners.  If a “CLI listener” is deployed and a command is entered on the device, a CLI event is generated and the event is sent in real-time back to the application.  The event message sent to the application also carries the command that was executed so the application knows a command was executed, but also knows the command itself the user entered.  This data can be used for further processing throughout the application.  Same holds true for the other types of listeners – I only listed three, but onePK supports many more than three listeners.  In general, there will be events that are generated with listeners and associated metadata sent back to the application about that event.  The bottom line is that listeners, or an equivalent, will be very important going forward (in my opinion) for looking at and building applications that require real time state of the network.  

Proactive vs. Reactive Applications

Up until now, there has been a lot of discussion around proactive vs. reactive flows, i.e. punting packets to the controller and all of that fun stuff, but what about proactive vs. reactive applications that leverage the network?  Have you ever thought about what type of API is needed to build a particular type of application?  Maybe you aren’t thinking about writing an application just yet, but if the foundation of your network is a particular device or SDN controller, starting to think about what is possible couldn’t be more important.

RESTful APIs vs.  Listener APIs

As an aside, if you are reading this and you’re a hard core developer, I may not be using the correct software development vocabulary, but hope my points get across.  Feel free to write in and correct me.  Anyway, these listener APIs would be leveraged through a platform like onePK, but more abstract, it would insinuate that a native programming language was being used to communicate to the network controller.  The dilemma that I see is having a RESTful APIs seems like, or is being touted as gold when controllers are talked about these days, and right now, I would say “so what?” to that.  

RESTful APIs are making web calls into a controller and seem to be great for two things --- pulling the data or extracting the data when you want from the network AND making a configuration change to the network.  Both of these would be categorized under the umbrella of a Proactive application.  These are proactive in the sense you know what you want to get or configure in the network.

On the other hand, Listener APIs by means of a native programming language allow you to gain the advantage of leveraging real time events like I’ve described.  These events can be processed and then you can have an action be taken on the network creating the holy grail of a feedback loop.  The action, or change to the network, can actually be sent back to the controller via native language API or REST.  These types of APIs allow for reactive SDN applications.

Don’t controllers offer native language APIs?

Yes and No.  I would argue no right now, but I haven’t looked at every controller.  Don’t forget I could also be totally off base and wrong :).  So, does your controller of choice simply allow you to add your own classes and methods or is it really a de-coupled SDK or development environment in that you don’t have to be hacking on controller code ON the controller?  It’s my understanding at this point you would be hacking on controller code.

While my example with onePK is being used as reference, I am advocating for that model to be integrated northbound of the controller, not just southbound or as a standalone application.  This means we should be able to write an application in Java (as an example) de-coupled from the controller where the application would reside on a separate server.  Said a different way…there would be an “agent” on the controller and the SDN application would create a persistent connection to that agent to get the benefits gained from real time event driven notifications through the use of listeners, and also have the application be de-coupled from the controller to allow for a more modular and scalable application/controller design.

I do acknowledge there are other ways for applications to receive real time events, i.e. from data sources “other” than the controller.  In that design, a REST only API would work just fine --- the application would receive events from the “other” application and then program/make changes to the controller via REST.  I am not advocating against this because that may be required to leverage alerts and to integrate into 3rd party systems, but I do not think that’s the way it should have to be to get real time events from the network that the controller natively manages.  

To sum it all up, there is more to APIs than just REST, and if I were ranking controllers, those that offered RESTful APIs, but ALSO a solid framework and SDK to build applications, that can be de-coupled from the actual controller code base would score higher than those who could not.  Going a step further, imagine these kits or libraries working cross-controller as well --- this could be another layer of abstraction to make it easier to interact in real time across different controllers (and port between them).  

Thanks,
Jason

Twitter: @jedelman8

7 Comments
Dave Tucker link
10/31/2013 20:38

Hi Jason,

*** Full disclosure - I work for HP, words are my own ***

Nice article... you raise some interesting points.
Here are my unedited thoughts...

Calling an API "RESTful" is like calling a Car "Manual". Knowing the transmission type tells you, the user, what type of shifter you will be using, but doesn't tell you anything about the engine.

REST isn't just asymmetric, it can be symmetric too - see the "distributed observer pattern" - and could be used to implement a publish/subscribe model to create feedback loops between the network, controller and applications.

If we don't use REST, as Web Services aren't right for all situations, a Message Queue (like RabbitMQ) could be used to implement this model instead, but thats a whole other topic!

IMO, you can comfortably use REST where your application doesn't require access to state in realtime - its ideal for integration with business applications or for orchestration. Applications built on the controller using a Native SDK are much more suited to realtime operations.

HP's controller, or any controller that uses something like OSGi, provides separation between applications and the controller itself. Your application communicates with the controller (or other applications for that matter) through a set of OSGi services (Java interfaces). This way, your application source is completely separated from the controller's, and OSGi can handle the dependencies.

So in summary, I think the architecture you describe exists today, but I may be wrong :)

@dave_tucker

Reply
Jason Edelman link
11/01/2013 06:48

Hi Dave,

Thanks for the comment. This is just the type of dialogue I was hoping for. I am clearly not familiar with the distributed observer pattern, but it sounds like maybe I should be --- and agree a MQ can be held for a different conversation.

I also agree REST can be used where the application doesn't require real time state information. You say it's good for provisioning and orchestration and I'd argue we need to start bringing real time state into those types of applications. Wouldn't orchestration platforms (and almost any other app) provide a bit more value if they had real time insight into the network?

As far as native controller SDKs and even a controller that is using OSGi, it was still my impression they weren't separated. I asked this question to HP on Wednesday at ONUG and thought the answer I received was that the application could not be de-coupled from the controller source. I could be wrong. But I do admit, I'm not an expert in OSGi!

But all good stuff and now I have some more areas to look into thanks to you.

Thanks again,
Jason

Reply
Dave Tucker link
11/01/2013 07:13

About network state in applications...
I'm strongly of the the opinion that we should never provide network state directly to applications. This mandates that the application has a level of network awareness (and I don't trust application developers with this). Instead the interaction ought to be abstracted such that an application, that traditionally consumes networking as a service, can express its demands in a language other tha "networkese"

To give an example of provisioning.... I push a request to the SDN controller which says "give me a network that looks like this". The SDN controller can implement the abstract network in any which way it wants (VLAN, VXLAN, NVGRE etc...) and even make adjustments on the fly to ensure that performance conforms with the description of the network it was given.

For business application this is much the same.... They would make a request to the controller which says "I'm going to be starting a big backup that needs to be completed in X minutes and is X priority". The policy engine on the SDN controller can then decide whether it should prioritise this traffic and which path it should take through the network, perhaps even signalling to the hypervisor to move the VM closer to the backup server ;)

When I'm writing apps for the HP Controller, it's a completely separate project. It has its own Eclipse workspace, it's under its own version control system, it compiles as an independent artifact etc...
When it gets uploaded to the controller, OSGi checks that the services I need (e.g the SDN Controller) are available before the artifact can be started.

So while there is a dependency on the HP SDK, as I'm using interfaces from it, my code is decoupled. If I design my application well enough (e.g provide an abstraction layer above the controller services) I should be able to swap this out for any other SDK.


@dave_tucker

Jason Edelman link
11/01/2013 07:30

I agree on the abstraction layer especially for any higher level business applications. For tooling and other IT systems integration, I'm not 100% in sure yet b/c it really does come down to what the application will do. We may need a network event that is generated for an application -- maybe a security monitoring system.

The basic premise of where I was going is many tools today either provision...OR monitor (via polling). Tools can be smarter if they are start to do both (with real time knowledge) and are integrated across technology stacks. But it's easier just to pick one :)

Definitely not advocating at any point to have a random developer treat the network as a sandbox. That's misconstrued.

Thanks for the comments again. Definitely need to understand OSGi better.

-Jason

Reply
Greg Ferro link
11/04/2013 05:03

I think that I disagree with your perspective here. While Listener APIs look attractive in terms of pseudo-real time features and capabilities ultimately you are implementing features that are highly specific to a device or function. These features may solve an immediate requirement or problem but in the long run, they won't survive the interoperability challenge that lies ahead.

REST can be used for real time communication using trigger events or similar concepts and generally uses less resources.

I would not consider using Listener APIs unless absolutely necessary and certainly for any core functions because they are likely to "die back" and REST interfaces dominate.

Reply
Jason Edelman link
11/04/2013 06:52

Greg, thanks for the comment. Why are you insinuating listener APIs = implementing highly specific features for particular devices, etc.? The goal should be to have common northbound interfaces generally speaking --- this includes common REST APIs and common native language APIs. Real time events could be anything from interface status, counters, neighbor events, etc. The list is long and many of which are not specific to a device or vendor.

Reply
kishore
09/15/2014 23:08

Hi , Any one can you help me how to create REST web services in Openday light controller. I'm struggling to create my own REST services in opendaylight controller. Guys pls help me.

Reply



Leave a Reply.

    Author

    Jason Edelman, Founder of Network to Code, focused on training and services for emerging network technologies. CCIE 15394.  VCDX-NV 167.


    Enter your email address:

    Delivered by FeedBurner


    Top Posts

    The Future of Networking and the Network Engineer

    OpenFlow, vPath, and SDN

    Network Virtualization vs. SDN

    Nexus 7000 FAQ

    Possibilities of OpenFlow/SDN Applications 

    Loved, Hated, but Never Ignored #OpenFlow #SDN

    Software Defined Networking: Cisco Domination to Market Education

    OpenFlow, SDN, and Meraki

    CAPWAP and OpenFlow - thinking outside the box

    Introduction to OpenFlow...for Network Engineers


    Categories

    All
    1cloudroad
    2011
    2960
    40gbe
    7000
    Arista
    Aruba
    Big Switch
    Brocade
    Capwap
    Christmas
    Cisco
    Controller
    Data Center
    Dell Force10
    Embrane
    Extreme
    Fex
    Hadoop
    Hp
    Ibm
    Isr G2
    Juniper
    Limited Lifetime Warranty
    Meraki
    Multicast
    N7k
    Nexus
    Nicira
    Ons
    Opendaylight
    Openflow
    Openstack
    Presidio
    Qsfp
    Quick Facts
    Routeflow
    Sdn
    Sdn Ecosystem
    Security
    Ucs


    Archives

    May 2015
    April 2015
    February 2015
    January 2015
    December 2014
    November 2014
    October 2014
    September 2014
    August 2014
    June 2014
    May 2014
    March 2014
    February 2014
    January 2014
    December 2013
    November 2013
    October 2013
    September 2013
    August 2013
    July 2013
    June 2013
    May 2013
    April 2013
    March 2013
    February 2013
    January 2013
    December 2012
    November 2012
    October 2012
    June 2012
    May 2012
    April 2012
    March 2012
    February 2012
    January 2012
    December 2011
    November 2011


    RSS Feed


    View my profile on LinkedIn
Photo used under Creative Commons from NASA Goddard Photo and Video
✕