re
for Python. If you make it this far, you become an expert in using expressions like this: ([A-Z])\w+
. And that’s not even a hard one! Regex party, anyone? I’ll pass.
TextFSM to the Rescue
TextFSM is a state machine (basically like a templating engine) that is purpose built to simplify working with regular expressions and getting structured data out of traditional network devices. Nifty.
For the more formal introduction taken directly from the TextFSM website:
"TextFSM, originally developed to allow programmatic access to information given by the output of CLI driven devices, such as network routers and switches, it can however be used for any such textual output."
How does it work?
An Example
show vlan
on a Cisco switch:
id
, name
, and status
for each VLAN.So, a template that gets this data would look like the following:
VLAN_ID
, NAME
, and STATUS
. Next to each variable name is the regular expression that represents the value being extracted. So, we can see here that VLAN_ID
will be one or more digits.Note: If I wanted to, I could have been more granular with some of these regex’s, but there was no need for this example.
Under the variables that are defined is where you then create the regular expression(s) that are used to evaluate each line of raw text. When a line from the raw text matches the regex, the next line of text is then evaluated, and so on. When there is a match, you can do nothing (as shown in the first regex above) or create a Record and store the data (as shown in the second regex above). The reason for the first regex
^\d+\s+enet\s+CE
is this matches entries in the second VLAN table shown in the show vlan
output. I’m basically disregarding that one, not recording anything there, and then storing all records for the first table. There is much more to TextFSM, so definitely take a look at the official docs if you are interested in learning more.If we use the
textfsm.py
script that comes with the install, the usage looks like this:
After seeing this, I wanted to do three things:
- Convert the individual lists to a list of dictionaries such that each existing row is it’s own dictionary and the keys are equal to the column headers as currently shown.
- Make it possible to use this approach by using real-time state data and not just local text stored in offline files
- Integrate with Ansible to eliminate the user from having to deal with the list to dict conversion, etc. and this way, it’ll just seem like structured data is being returned from every device!
What was the result?
netget
. It supports several parameters, but the main ones are: connection
, command
, file
, and template
. template
is required and you would put the path to the TextFSM based template here. command
and connection
go together as this would be the show command
or equivalent and type of connection you will be making. Today, it only supports SSH, but it’s open to support other mechanisms like telnet and console connections in the future. Last, file
can be used instead of command
if you want to just work with offline files rather than collect data in real-time from devices.One major point about this type of approach is the user can create whatever type of template they want without creating new Ansible modules! Just create a new template and get structured data back!
Another Example - Gathering neighbor data from a switch
Running an Ansible playbook using netget:
Almost forgot - here is the Playbook:
Since I am not regex guru, I’ve come to use the following site quite a bit to test and troubleshoot regular expressions: http://www.regexr.com/. There are plenty of others, but this one has been doing the trick for me.
Side note: I haven’t posted the code yet for the netget module mainly because it doesn’t have all the functionality I’d like it to have yet. And I started using Paramiko as the library, but think it probably makes more sense to use the library, called netmiko, that Kirk Byers has been building since it’s multi vendor, etc., which also depends on Paramiko anyway.
Given that the industry has kind of standardized on a common CLI, this method should even work across multiple vendors, which is pretty damn cool. TextFSM is definitely not limited to network gear though, so feel free to use it on whatever device you want - even those native Linux operating systems :)
What do you think? What other functionality would you like to see in something like this?
Thanks,
Jason
Twitter: @jedelman8