pyATS & Genie – Beneath the Surface

Today, we’ll take you behind the scenes and inspect the iceberg below the surface: how the framework and its libraries are built, and how you can take advantage of its APIs in Python.

Network Automation & Testing

pyATS | Genie was initially developed as the next-generation test infrastructure for Cisco Engineering. But wait – are we not talking about networking automation and NetDevOps?

We are. Upon closely inspecting and comparing test and network automation, we can identify a high degree of behavior overlap: they both programmatically drive network devices, only to different expectations. Whereas tests have passing criteria, network automation is built around business logic that acts and reacts on input conditions.

In other words – they share the same network automation libraries.

For the past 20 years, Cisco has invested in automated testing. With pyATS | Genie released externally through DevNet, it allows everyone to make use of the great libraries and scripts that have been created as part of this ongoing engineering effort. So, while our engineers rigorously test the next platform/release, you may leverage the same libraries for your own network automation needs.

So, do I use pyATS or Genie?

In short, you use both. They are like two sides of the same NetDevOps coin.

pyATS is the foundation of this ecosystem. As a powerful and highly-pluggable Python test framework, it is designed to provide maximum flexibility to developers, and standardizes the boilerplate requirements:

◈ define topologies and device/interconnects

◈ programmatically interact with various devices

◈ write, execute and report on test scripts

On the other hand, Genie is pyATS’s library and development-kit that focuses on building reusable network automation libraries and testcases. Built on top of pyATS, Genie features:

◈ parsers: converting/formatting command output into Pythonic data structures

◈ models: OS/platform agnostic Python classes that represents feature/protocol configuration state and operational status

◈ triggers & verifications: reusable pool of data-driven testcases

Together, pyATS | Genie provides you with all the tools & libraries necessary for network automation. By picking and choosing the right APIs and testcases, all you have to do is:

1. build your own business logic that makes use of the libraries

2. integrate it into the rest of your automation system, be it Jenkins, Ansible, ROBOT Framework or the likes.

“What do you call a pyATS developer that leverages Genie? A pyATS Genius.”

Parsers & Models

So far, you have seen that you can use Genie and parse CLI commands in shell, eg:

bash$ genie parse “show interfaces” --testbed-file testbed.yaml

Behind the scenes, this invokes Genie’s parsing capability:

1. connects to the testbed device

2. performs a search for the most-appropriate parser to use, based on input CLI and the connected device’s OS and platform information

3. invoke the parser to process the output

4. return the parsed Python dictionary (displayed as JSON in Genie CLI).

Parsers are the lowest library layer in Genie. Each parser is responsible for:

1. issuing the right command on device, collecting output

2. convert/scrape/format the output, based on context, into a schema-controlled dictionary output.

The use of a schema with each parser ensures that each parser is self-describing, self-documenting, and self-testing.

The 1000+ parsers currently featured in Genie give you the basic ability to view, compare and analyze your device’s operational states in straight-up Python dictionary format. As awesome as that sounds, they do come with a few caveats:

◈ each parser processes only one command, and narrowly represents only a slice of the overall operational state of a feature/protocol.

◈ commands between different OS and platforms often differ, and as such, building business logic around parsers does not scale if you have a variety of devices in your network.

This is where models come in.

Genie models are the next-layer-up above parsers: YANG-inspired Python classes that implements a whole feature/protocol agnostically. They’re called YANG-inspired because the development team studies the YANG models of various platforms and crafted their own. Why? Because YANG is a machine-to-machine descriptor, and NETCONF XML comes with its own angle bracket tax…

Built to be human-friendly and engineered to works across different platforms and OSes, Genie models enables users to interact with network devices/protocols in a holistic, high-level and Pythonic fashion.

Take interface for example. To build the interface operational state model, our resident CCIE engineer, @tahigash3 studied YANG interface models across a variety of platforms, and came up with one top-level structure. With it, when you invoke Genie to learn “interface”, eg:

bash$ genie learn interface --testbed-file testbed.yaml

or if you are using Python directly:

The engine automatically issues the following commands for each reference platform.

IOS-XE IOS-XR NXOS

show interfaces

show vrf detail

show ip interface

show ipv6 interface

show interface switchport

show etherchannel summary

show interfaces [intf]

accounting

IOS-XR

show interfaces detail

show vlan interface

show vrf all detail

show ipv4 vrf all interface

show ipv6 vrf all interface

show bundle

show interfaces [intf]

accounting

NXOS

show interface

show vrf all interface

show ip interface vrf all

show ipv6 interface vrf all

show interface switchport

show routing ipv6 vrf all

show routing vrf all

These command outputs are then parsed, using Genie parsers, and reconstructed together into the new data structure that represents the entire operational state of this device’s interfaces:

In addition, each operational model is accompanied by a list of keys that are naturally “less interesting.”  When a diff is performed – the list ensures things like “uptime” and “keep-alive-sent” (e.g., data that is ever changing/incrementing and of minimal value) do no pollute the output, and that you can focus on just the things that matter.

Besides operational status, this design around high-level, holistic model approach applies to device configuration as well: Genie conf models enable users to configure and unconfigure network devices just by setting Python object attributes. The rest is handled for you automatically.

Because these models remain structurally consistent across different OS/platforms, automation built around Genie models are portable across your network: write them once and use them across different topologies and device types.

Can it get even better? Of course! Genie’s opens source library implementations are not limited to just Cisco devices. Whilst the team here is focused on building support for Cisco platforms (duh!), it is 100% possible to support 3rd party vendors and even competitor platforms through library extensions and plugins.

Sky’s the limit

With pyATS | Genie, you have free rein over your network automation. By harnessing the power of parsers and models, you can build true data-driven, portable and agnostic network automation that scales along with your network.