SD-Routing: Unlock Agility and Efficiency for the Secure WAN Edge

Many Cisco enterprise customers have decades of Cisco Catalyst routing and security capabilities functioning at branch locations. However, many of their traditional network management solutions can’t keep up with the demands of cloud adoption, remote work, and ever-growing user expectations. This translates to poor user experience, sluggish applications, and possible security vulnerabilities. These factors are driving the need for a transformation across applications, networks, and security.

This operational paradigm shift aims to seamlessly connect users anywhere to any application and secure user access by protecting against evolving threats. The answer to these operational challenges is Cisco’s software-defined routing (SD-Routing) solution. It goes beyond traditional per-device-based management by enabling full frictionless lifecycle device management, monitoring, configuration, and troubleshooting—as well as robust, next-generation firewall security integrations—from a single dashboard that doesn’t require any changes to your existing environment.

Figure 1. SD-Routing solution overview

Let’s explore some key use cases of SD-Routing that can transform your network:

Frictionless device lifecycle management. Simplify and prepare your network for the future with one management platform. SD-Routing, controlled through the Cisco Catalyst SD-WAN Manager dashboard, can:

• Unify management: Manage device software upgrades, monitoring, and troubleshooting through the intuitive Catalyst SD-WAN Manager dashboard. This simplifies network operations and empowers you to manage both traditional routing and Catalyst SD-WAN environments.
• Tame legacy challenges: Simplify complex legacy operations with SD-Routing. Basic troubleshooting tools within the manager help you maintain and optimize performance. Continuous updates ensure your network stays ahead of the curve.
• Combat configuration drift: Manage and track changes with a unified platform. Use the manager to create configuration templates for standardized deployments and future SD-WAN migration.

Network administrators might be using homegrown automation or third-party vendor tools to solve these problems. You can continue to use these tools, but you don’t need to invest further. Rather, take advantage of SD-WAN Manager, which comes as a part of Catalyst licensing.

Security

Configuring diverse IOS XE security features through the command-line interface (CLI) or customized ad hoc scripts has historically been a complex, labor-intensive process that is prone to errors. This is especially true for defining granular security policies across zones and containers. With the introduction of SD-Routing guided security workflows, customers aiming to implement robust, next-generation firewall (NGFW) security on their on-premises routers will find this a valuable addition, allowing for consistent policy application across deployments. Many customers want Direct Internet Access (DIA) at their branch offices, but security concerns hold them back. SD-Routing can streamline secure DIA deployment on WAN edge routers, offering a simpler approach to securing distributed networks.

Cloud on-ramp for multicloud

Traditional network teams often struggle to securely extend their WANs to cloud providers, where key enterprise applications may reside. SD-Routing simplifies this process, especially for those who are hesitant to adopt it. With SD-Routing, you can securely connect to cloud providers like AWS and Azure following best practices, without months of learning complex, cloud-specific configurations. This empowers you to seamlessly connect to cloud providers and focus on your business outcomes.

As you tackle the modern network challenges, explore SD-Routing to simplify, streamline, secure, and future-proof your WAN environment. The single management platform for Catalyst SD-WAN and SD-Routing saves time and operational expenses with agile and automated workflows that quickly respond to network changes.

Beyond these immediate benefits, SD-Routing also can help strategically position your network for simplified future migrations to SD-WAN, depending on where you are in your digital transformation journey.

Whether you have existing enterprise networking equipment in your WAN or are considering a future purchase of Cisco Catalyst 8000 Edge Platforms, Cisco 1000 Series Integrated Service Routers, Cisco 1000 Series Aggregation Service Routers, or Industrial Routers, SD-Routing can unlock their full potential. Even better, if you’re already using Cisco Catalyst SD-WAN Manager, you can leverage the same platform to manage your SD-Routing deployments.

Source: cisco.com

Continue reading

Common Database Infrastructure in Cisco IOS XE Software Simplifies 160+ Enterprise Devices

Developed by a global team of more than 3000 software engineers, Cisco IOS XE Software powers more than 160 Cisco enterprise platforms for access, distribution, core, WAN, and wireless — with many different form factors and combinations of hardware and software. One of the main reasons the software stack can encompass such a large portfolio of enterprise networking products is due to a common database and database-centric programming model across all platforms.

It started with the Cisco 1000 Series Aggregation Services Router (ASR 1000) in 2004, where every state update to the data path went into and out of an in-memory database. Since 2015 and Cisco IOS XE version 16.1.1, many more platforms have been added, due in large part to the software stack’s consolidated database features that work across all platforms. From one platform supported by IOS XE to 160 in six years is an incredible industry run rate.

Here are some of the most useful and robust database features used across all Cisco devices that run Cisco IOS XE.

In-memory Database Power and Capturing Application Intent

Configuration and operational data in IOS XE devices are stored in in-memory NoSQL graph databases. In addition to providing atomicity, consistency, isolation, and durability (ACID) functionality, IOS XE supports validation and default values, dependency management, replication, notifications, subscriptions, and consolidation.

Application database intent ― including schema, defaults, validation, and graph model ― are captured in a Domain Specific Language (DSL) called The Definition Language (TDL) that was developed by Cisco. Using TDL, developers can describe what they want to do, what data they want to model, and the rules for validation. Then the TDL compiler generates database interaction code in the language of choice for the application (e.g., C, Java, Python), as shown in Figure 1. If developers want to use a new language, they can still use the intent captured in TDL to generate code.

Figure 1. Utilizing DSL to Capture Database User Intent

Decoupling intent from implementation code provides tremendous architectural flexibility. For IOS XE, the back end is written in C to provide optimal performance. The front end uses a formal query system and can be in any language. We use a custom compiler with a Model-View-Controller (MVC)-based architecture to perform the magic of converting intent to front-end APIs.

This approach eliminates the need for data conversion for clients querying the database. As shown in Figure 2, applications can natively interact with the database through APIs regardless of the language of choice. The database can also be read by other applications and/or infrastructure (e.g., Web UI, CLI-based show commands, and other monitoring services).

Figure 2. Cisco IOS XE Applications Natively Interact with the Database

Runtime Infrastructure for Cisco IOS XE

Although the database infrastructure in IOS XE can use secondary storage as the database store, most of the applications use in-memory databases that reside in RAM. A transactional engine specifies ACID guarantees (e.g., a process launched by some user must request modifying the database and signal when it is done modifying it). Failure to complete the process results in the database being rolled back so it is never in an inconsistent state.

Figure 3. Runtime Infrastructure for Cisco IOS XE

The raw lookup data structure layer includes the infrastructure for indexing algorithm tables (e.g., hash tables, binary search trees). The graph layer is where user-specific database configurations like table connections, default values, and validation enforcement are performed. For example, a Wireless Lan Controller (WLC) tracks Access Points (AP) and clients connected to it. Clients are connected to the WLC through the AP. This wireless operational state may be modeled as AP and client tables, with each record in the AP table connected to a client table. It is important to note this is the internal state of the application. With IOS XE database runtime, this state can now be consolidated, exported, replicated for SSO, etcetera, while being performant enough to support the high-scale requirements for wireless.

Other Functions Enhanced with IOS XE Database Features

◉ Fast reload – On reload, a persistent, version-aware, binary configuration can be read faster than any text representation. In the past, reloading software on Cisco platforms could take up to 7 minutes. With Extended Fast Software Upgrade (xFSU), it takes 30 seconds or less. The hardware is never powered off and traffic keeps flowing while the control plane is maintained in an operational state during the reload process.

◉ Stateful Process Restart – Externalizing an IOS XE device’s configuration and operational state allows stateful restart processes. By saving the device’s state externally, it can be restarted and will continue where it left off.

◉ Horizontal Scaling – Consolidation of a device’s operational state allows for the elastic and horizontal scaling of processes based on changing application traffic patterns. There may be multiple copies of the same process, each with its own database, but Cisco enables databases to be consolidated into a single database, providing a global view, which makes it easier to spawn more processes horizontally.

◉ Stateful Switchover (SSO) – Databases on active and standby devices in a high availability configuration are continuously synchronized through replication to keep the standby device in a hot state, able to become active in case of a failure. Like stateful process restart, at the device level, SSO synchronizes one device through replication continuously.

◉ In-Service Software Upgrade (ISSU) – To ensure that versions of Cisco IOS XE that are running are correct across supervisor engines and other devices, databases in Cisco IOS provide per-object versioning support with build time checking for violations. This helps ensure a reliable ISSU.  ISSU orchestrates the upgrade on standby and active processors one after the other and then switches between them in the control plane so that there is zero effective downtime and zero traffic loss.

◉ Monitoring and Global Device View – A device running IOS XE provide a global view of its complex and varied operations, based on the consolidation of databases, which allows for greater real-time insights into configuration and operational data. Analysts can subscribe to specific data sets and request to be alerted when any changes occur to monitor the device more proactively.

Summary of Database Benefits in Cisco IOS XE

Database features in Cisco IOS XE allow devices to be reloaded in seconds, to maintain a state during restart and switchover. Applications can consume database records natively without any translation required. Intent can be gathered and code generated in any development language, ensuring resilience to regressions. Databases used by each device are consolidated into a global view, enabling the horizontal scaling of processes. The system supports version skew operation with per-object versioning.

It’s all relatively seamless across all 160+ Cisco IOS XE devices.

Source: cisco.com

Continue reading

Cisco Champions the Powerful, Evolving Networking Software Stack

With the interconnection of billions of devices in public and private networks and many applications and services moving to the cloud, software is increasingly becoming independent of and abstracted from hardware. At public cloud vendors like Amazon Web Services, Google Cloud Platform, and Microsoft Azure, hardware has been commoditized and software has taken center stage.

At Cisco, resellers and enterprise customers put complex solutions together using our products. The integration of switches, routers, and other gear with software used to require up to a one-year qualification cycle. But with the cloud providers, it’s immediate. Today, more native cloud concepts have been added to Cisco IOS XE software. Quarter by quarter, our enterprise software is becoming more efficient and cost-effective, more automated, and more programmable.

From Physical to Virtual to Cloud Native 

The first incarnation of Cisco enterprise cloud-enabled products was the virtualization of physical hardware devices in the cloud as virtual machines. They had all the existing concepts and features customers were used to in existing physical Cisco platforms.

In recent years we’ve been moving from physical to virtual to cloud-native products. As customers are becoming more aware and ready to consume cloud-native features, Cisco IOS XE is being enriched to provide those features. At 190 million lines of code―more than 300 million when vendor software development kits (SDKs) and open-source libraries are added―Cisco IOS XE runs 80+ platforms for access, distribution, core, wireless, and WAN layers. It facilitates a myriad of combinations of hardware and software, forwarding, and physical and virtual form factors.

Why Cisco? 

Prospective Cisco customers and competitors may ask, why spend $5000 for an enterprise switch when you can spend $1000? The answer is that our customers know that buying a cheaper switch may lack the features they need. Less expensive gear will also potentially add to their maintenance costs because the components may not be as good as Cisco’s.

Another reason to buy Cisco is due to the breadth of our enterprise portfolio. Any one company can do one vertical market well. With IOS XE, we have integrated everything across the networking software stack, and across the entire enterprise network, and we’re working to keep it simple across multiple network domains.

Efficiency and Cost-effectiveness 

With networking becoming increasingly feature-rich and complex, simpler networking software translates to greater efficiency, a smaller headcount, and fewer onsite visits to fix problems. For example, Cisco IOS XE provides simplified app hosting using a Docker image in a container and deployment using device controller tools. It supports third-party, off-the-shelf applications built using Linux toolchains that allow business apps to run at the network edge.

Other examples include the simplification of development, debugging, and device validation with Cisco Platform Abstraction (CPA) and unified software tracing that integrates traces from software running anywhere in a network for more complete visibility into 100+ processes in real-time. Another example of Cisco IOS XE simplicity is virtualization technology that runs over optical fiber, enabling switches to be physically located up to thousands of miles away from each other.

The Power of Automation 

Cisco IOS XE is becoming more and more self-driving. Cisco developers are increasingly taking away the manual tasks required to manage the network by automating them. That makes networks easier and cheaper to maintain and faster to debug.

Examples include the automation of image upgrades using Cisco DNA Center and support for programmable microservices to replace manual device upgrades, repurposing, and management. Other automated processes include streaming telemetry and analytics in all layers of software that run at the speed of events observed (e.g., faster than two million route updates per second) to handle the huge scale of networking operations.

Programmability 

Systems administrators in enterprise companies are constantly upgrading, repurposing, and managing thousands of switches. An advanced networking software stack must be able to manage multi-vendor networks using native and open-source data models. Cisco IOS XE supports a suite of Google Remote Procedure Call (gRPC)-based microservices that simplify and lighten workloads with programmability. They allow administrators to programmatically manage Cisco enterprise devices.

The IOS XE Development Environment  

A lot of enterprise software takes years to develop. The Cisco software development environment rolls out new solutions in months.

Developers spend 60-70% of their time developing software instead of application logic. The IOS XE development environment is automating as many common capabilities (like show commands, tracing, telemetry, export for dashboard, hand wiring HA code, testing base ISSU compatibility checks, and mocking for unit tests) as possible to avoid the need to hand code them. With hand coding, every one of these features would require developers to generate two-to-three times as much code. Hand coding is also not amenable to automated, flexible deployments and in the current development trajectory will not fit into the low-footprint devices we ship.

The Cisco Enterprise Networking software development team works at a solution level, conducting pre-qualification testing and providing the tools to control an entire enterprise dashboard from a single dashboard.

Source: cisco.com

Continue reading

Navigating supply chain disruptions for agile retail

Business continuity is so important for retailers that face disruption to their supply chain. Between overseas shipments decreasing and physical stores closing, it’s no secret that the retail industry is coming face to face with the changing business landscape. Retailers that have solely relied on in-store offerings are now looking to quickly take their business to the web with a secure network. Luxury brands that rely heavily on global supply chains are particularly looking to creatively pivot their business model. Some sectors of retail are experiencing an uptick, particularly those leveraging delivery services and curbside pickup (such as quick service restaurants). The surge in remote workers and the need for visibility across both the network and across business operations, calls for agility for all retailers alike.

The largest question for retail brands around COVID-19 quickly becomes how to manage resources when they see interruptions to their supply chain. The best way to combat these disruptions is to engage employees, associates, suppliers and consumers with the right digital solutions. The goal of course being to find business continuity and a rhythm across the value chain, however manageable. Here’s a look at what how retailers can leverage the power of IT, and even look ahead to the increasingly evolving end consumer.

◉ Information and communication Real-time insight around product availability levels is especially important, as the continuous flow of updates requires up-to-the-minute inventory management. Retailers that can leverage a unified communication platform to support timely product information is key for all workers across the distribution center, to the customer service centers, to those managing stock in stores, and newly remote workers supporting retail operations from home.  For broader internal communications, collaboration and video conferencing enables retailers to ensure alignment around priorities and company strategy. Ramping up additional licenses to scale efforts aimed at helping organizations move as a unit has become top of mind as well.

◉ A whole new network A timely approach to setting up virtual firewalls is another beneficial move for retailers. A newly mobile workforce that can leverage a VPN connection allows workers to more effectively help customers in their online purchase journey while keeping transactional data secure. That in mind, threat detection and protection for an increasingly remote workforce is the best way to not only protect customer information but, to keep integrated retail systems secure as well across the network.

◉ Lock down on secure data Increased online presence for shoppers who are no longer visiting physical stores means online retail platforms may have to support a higher volume of traffic. An agile IT infrastructure and the right data storage for your network can up-level digital capabilities to grow that ecommerce channel and improve overall customer sentiment.

◉ Unified commerce The right contact center solution can help get customers the answers to their delivery or pickup questions more quickly, and location awareness can speed up the pickup process. From an omni-channel perspective, direct to consumer communication such as click to chat features can improve that customer experience and awareness around their order. This simply pushes the envelope around click-and-collect fulfillment methods that the industry has seen consumers gravitate to for years. What was a convenience has just become more of a necessity. Retailers that can support unified commerce are able to remain fluid during dynamic and uncertain times, as the consumer behaviors continue to evolve.

Continue reading

Running NetBeez Agents on Cisco Catalyst Switches

I am happy to announce a new powerful integration between NetBeez and Cisco. Starting with Cisco IOS-XE version 16.12.1, Cisco users can install the NetBeez docker agent on Cisco Catalyst 9000 series switches. This new integration is part of the Cisco application hosting framework, which enables third-party off-the-shelf applications to run on top of Cisco devices. As you’ll read in the next paragraphs, NetBeez and Cisco users will have a lot to gain from this integration. If you are new to NetBeez, let me tell you more about it.

Wide Area Network Monitoring with NetBeez

NetBeez is a distributed network monitoring solution that enables network engineering teams to monitor remote Wide Area Network locations via dedicated hardware or software agents, called Beez. The Beez run active monitoring tests, such as ping, traceroute, and iperf, as well as DNS and HTTP checks against web and cloud applications. Like a canary in a coal mine, the Beez proactively detect remote performance issues that impact end-users and business operations. The performance data logged by the Beez is sent real-time to the NetBeez central server, where it’s processed for alerting, displayed on the user dashboard, or consumed by third-party applications via the available APIs.

With the Cisco App Hosting integration, the Catalyst 9000 is capable of hosting NetBeez agents and run network performance tests from the user perspective. In this scenario, the NetBeez server is still needed to manage the Beez running on the switches and to collect the network performance data generated.

Benefits of Cisco App Hosting

Traditionally, the Beez runs on top of a Raspberry Pi that is plugged into the access switch at remote WAN sites. Companies that need to monitor large WANs have to invest considerable time and resources to ship and deploy the hardware appliances at remote locations. The Cisco App Hosting removes this “physical barrier” in the deployment and maintenance process of the Beez. Catalyst owners can now easily install via the Cisco CLI the NetBeez docker agent on their switches. Let’s see what this procedure looks like …

Configuring Catalyst for App Hosting

Configuring a Catalyst 9000 series switch to host a NetBeez docker agent is fairly simple. Before you begin, make sure you meet the following requirements:

◈ A Cisco Catalyst 9000 switch with IOS-XE version 16.12.1

◈ A USB SSD-120G for Catalyst 9000 series switches

◈ A NetBeez server running version 2.0

◈ The NetBeez docker agent v2.0.5

The procedure will have you:

1. Create a user VLAN that will be used by the NetBeez docker agent as uplink

2. Map the user VLAN to one of the switch’s access or trunk ports

3. Create an AppGigabitEthernet interface that is an internal bridge between the eth0 interface on the NetBeez agent and the user VLAN mentioned at step 1

4. Define configuration parameters needed by the NetBeez docker agent to connect to the server.

The following diagram illustrates how these different components relate to each other.

Continue reading

Enterprise Streaming Telemetry and You: Getting Started with Model Driven Telemetry

Why Streaming Telemetry?

Cisco IOS XE is the Network Operating System for the Enterprise. It runs on switches like the Catalyst 9000, routers like the ASR 1000, CSR1000v, and ISR 1000 and 4000’s, Catalyst 9800 Wireless LAN controllers, as well as a few other devices in IoT and Cable product lines. Since the IOS XE 16.6 release there has been support for model driven telemetry, which provides network operators with additional options for getting information from their network.

Traditionally SNMP has been highly successful for monitoring enterprise networks, but it has limitations: unreliable transport, inconsistent encoding between versions, limited filtering and data retrieval options, as well as the impact to the CPU and memory of the running device when multiple Network Monitoring Solutions poll the device simultaneously. Model-Driven Telemetry addresses many of the shortfalls of legacy monitoring capabilities and provides an additional interface in which telemetry is now available to be published from.

Yes, this is a push based feature. No longer do we need to poll the device and ask for operational state. Now we just decide what data we need, how often we need it, and where to send it. Once the configuration is in place the device happily publishes the telemetry data out to the 3rd party collectors, your monitoring tools, big data search and visualization engines like Splunk and Elastic, or even to a simple text file – it’s totally configurable what you do with the data. In the example below we use Telegraf + InfluxDB + Grafana to receive, store, and visualize the data.

Once common use case is to monitor the CPU utilization of a device. Let’s understand where and how we can get this data from our Cisco Catalyst 9300 running IOS XE 16.10

YANG Models

YANG Models are at the heart of Model-Driven Telemetry: Yet Another Next Generation! These human-readable text-based models define the data that is available not just telemetry publication but also for programmatic configuration as well. These data models reside within the IOS XE device and can easily be downloaded when using tooling like YANG-Explorer. All of the models are also published on the YangModels Github page which makes them easy to access and analyze.

The YANG-explorer tooling is available on the CiscoDevNet Github page which can download the YANG models directly from the IOS XE device over the NETCONF or RESTCONF interfaces and quickly show which data is available and from which model.

This example from the YANG Explorer shows that we have already downloaded and loaded the Cisco-IOS-XE-process-cpu-oper.yang model and began to explore it. It shows that the one-minute, five-minute, and 5-second “Busy CPU-Utilization” is available as a percent, as well as the Interrupt 5-second (five-second-intr) metric. It also shows some metadata about the model that includes the XPath, Prefix, Namespace, and a description with some other details.

Cisco IOS XE MDT Configuration

Now that we know which YANG data model contains the needed information let’s enable sending this telemetry from the device.

It is possible to configure and verify telemetry subscriptions from the traditional CLI as well as through the NETCONF, RESTCONF, and gNMI programmatic interfaces using YANG. When using CLI the show commands are available with the ‘show telemetry ietf’ set of commands, and is configured similarly with ‘telemetry ietf’ commands when in configure mode. When using YANG, the “Cisco-IOS-XE-mdt-cfg.yang” and “Cisco-IOS-XE-mdt-oper.yang” YANG models are available for both configuration and operational datasets.

Lets look at a configuration example from a Catalyst 9300 switch running Cisco IOS XE 16.10 This configuration enables telemetry subscription ID 501 and encoding is set to “kvgbp” which is a self describing JSON key-value pare Google Protocol Buffers format. The data that we want sent is defined by the filter xpath and we used YANG Explorer and the YANG models earlier to find it. The xpath filter prefix for the Cisco-IOS-XE-process-cpu-oper.yang model is “process-cpu-ios-xe-oper.yang”, and the specific datapoint or KPI we want is the 5-second CPU Utilization. The source address and source VRF are set so that the device knows which port or interface to send telemetry from. The update policy is set in centiseconds so every 5 seconds (500 centiseconds) the device will publish an update. Finally, the IP and port that the receiver is listening on is set, as well as the to use gRPC over TCP as the protocol.

Cat9300# show run | sec tel

telemetry ietf subscription 501

 encoding encode-kvgpb

 filter xpath /process-cpu-ios-xe-oper:cpu-usage/cpu-utilization/five-seconds

 source-address 10.60.0.19

 source-vrf Mgmt-vrf

 stream yang-push

 update-policy periodic 500

 receiver ip address 10.12.252.224 57000 protocol grpc-tcp

Let’s see what this looks like with some of the show commands: ‘show telemetry ietf subscription 501 detail’ and ‘show telemetry ietf subscription 501 receiver’:

The output shows that the CPU Utilization XPath has been set and that the telemetry receiver has connected successfully.

Telemetry Receiver

The open source software stack that allows easy reception, decoding, and processing of the “kvgbp” telemetry is referred to as the TIG stack. TIG represents three separate software components: Telegraf which receives the telemetry data, InfluxDB which stores it, and Grafana which is responsible for visualizations and alerting.

Telegraf has the “cisco_telemetry_mdt” input plugin that receives and decodes the gRPC payloads that the IOS XE device sends. It also has an output plugin that sends this data into the InfluxDB where it is stored. The configuration for Telegraf is simple and static because once it’s setup it rarely needs to be reconfigured or modified. Simply define a few global parameters, the input, the output, and then start the telegraf binary or daemon process.

In this example we configure the gRPC input listener on port 57000 – this is the port that IOS XE will publish telemetry to. We have also configured where to send the data out to: InfluxDB running on the localhost, port 8086, as well as the database, username, and password to use for the data base storage.

# telegraf.conf

# Global Agent Configuration

[agent]

hostname = "telemetry-container"

flush_interval = "15s"

interval = "15s"

# gRPC Dial-Out Telemetry Listener

[[inputs.cisco_telemetry_mdt]]

transport = "grpc-dialout"

service_address = ":57000"

# Output Plugin InfluxDB

[[outputs.influxdb]]

database = "telegraf"

urls = [ "http://127.0.0.1:8086" ]

username = "telegraf"

password = "your-influxdb-password-here"

InfluxDB and Grafana can run inside Docker containers or natively on Linux, and there is excellent getting started documentation on the official InfluxDB and Grafana websites. I recommend following the official guides in order to setup InfluxDB and Grafana in your environment as needed.

Visualization with Grafana

Grafana is the visualization engine that is used to display the telemetry data. It calls into InfluxDB to access the data that is stored there, which is the same data that Telegraf received from IOS XE. In this example there are three unique queries for each of the CPU metrics: five-seconds, five-minutes, and one-minute, and those are shown in the chart. We can see that the 5-second CPU average in green is between 1% and 2%, while the 1 minute CPU average in blue remains at 1%.

Continue reading

Cisco Trusted Platforms

Service Provider networks serve as critical infrastructure, and the security and trustworthiness of the network infrastructure is essential and the Trusted Infrastructure video from Mobile World Congress. Providers of digital infrastructure must be able to verify whether the hardware and software that comprise their infrastructure are genuine, uncompromised, and operating as intended.  As shown in Figure 1 below, there are security and trust requirements at every layer of the Network Operating System. I will address each of these layers in this and a subsequent blog.

Figure 1: Network Operating System Layers and Security Requirements

Note that not all the features listed here are available on all Service Provider platforms. Please contact the sales team for details.

Foundations of Trust

The ability to verify that a Cisco device is genuine and running uncompromised code depends on Cisco Secure Boot and Trust Anchor module (TAm).  Cisco uses digitally-signed software images, a Secure Unique Device Identifier (SUDI), and a hardware-anchored secure boot process to prevent inauthentic or compromised code from booting on a Cisco platform.

Hardware Root of Trust

A trusted element in the scope of system software is a piece of code that is known to be authentic.  A trusted element must either be immutable (stored in such a way as to prevent modification) or authenticated through validation mechanisms. Cisco anchors the root of trust, which initiates the boot process, in tamper-resistant hardware.  The hardware-anchored root of trust protects the first code running on a system from compromise and becomes the root of trust for the system.

Trust Anchor module

The Trust Anchor Module (TAm) is a proprietary, tamper-resistant chip that features non-volatile secure storage, Secure Unique Device Identifier (SUDI), and crypto services including random number generation (RNG).  See below for additional information on SUDI.

Image signing

Image signing is a two-step process that creates a unique digital signature for a given block of code. First, a hashing algorithm, similar to a checksum, is used to compute a hash value of the block of code. The hash is then encrypted with a Cisco private key, resulting in a digital signature that is attached to and delivered with the image. Signed images can be checked at runtime to verify that the software has not been modified.

Chain of Trust

A chain of trust exists when the integrity of each element of code on a system is validated before that piece of code is allowed to run. A chain of trust starts with a root of trust element. The root of trust validates the next element in the chain (usually firmware) before it is allowed to start, and so on. Through the use of image signing and trusted elements, Cisco hardware-anchored secure boot establishes a chain of trust which boots the system securely and validates the integrity of the software.

Secure Boot

Cisco Secure Boot helps ensure that the code that executes on Cisco hardware platforms is genuine and untampered. A typical UEFI-based boot process starts at the UEFI firmware and works up to the boot loader and the operating system. A tampered UEFI firmware can result in the entire boot process being compromised.

Using a hardware-anchored root of trust, digitally-signed software images, and a unique device identity, Cisco hardware-anchored secure boot establishes a chain of trust which boots the system securely and validates the integrity of the software.  The root of trust (aka. microloader), which is protected by tamper-resistant hardware, first performs a self-check and then verifies the UEFI firmware, and thus kicks off the chain of trust leading up to the integrity verification of the entire IOS XR operating system.

Secure Unique Device Identifier (SUDI)

The SUDI is an X.509v3 certificate and an associated key-pair which are protected in hardware in the Trust Anchor module (TAm).  The SUDI certificate contains the product identifier and serial number and is rooted in Cisco Public Key Infrastructure. This identity can be either RSA or ECDSA based. The key pair and the SUDI certificate are inserted into the Trust Anchor module during manufacturing, and the private key can never be exported. The SUDI provides an immutable identity for the router that is used to verify that the device is a genuine Cisco product, and to ensure that the router is well-known to the customer’s inventory system.

The SUDI-based identity can be used to perform authenticated and automated configuration using Zero Touch Provisioning (ZTP). A backend system can issue a challenge to the router to validate its identity and the router will respond to the challenge using its SUDI based identity. This allows the backend system to not only verify against its inventory that the right router is in the right location but also provide encrypted configuration that can only be opened by the specific router, thereby ensuring confidentiality in transit.

Secure Storage

Cisco’s Trust Anchor technology provides a mechanism to securely store secrets on the router. The encryption of the storage space is tied to the hardware root of trust, and data cannot be decrypted without the specific hardware that was used to encrypt it. The secrets that can be stored include user passwords, customer credentials for authentication protocols such as RADIUS or TACACS, customer certificates, and any type of keys.

The combination of SUDI-based ZTP and secure storage provide very strong protection of customer configuration and secrets.

Hardware Fingerprint

Tampered hardware, particularly in transit, is a clear vector of attack. This is especially a concern when the hardware is in transit from Cisco to our customers and partners; or when a Service Provider ships their router from a holding center to the deployment center. A malicious agent can intercept the hardware in transit and tamper the hardware in a non-detectable manner.

Cisco’s Hardware Fingerprinting technology provides the ability to detect tampered hardware using the Trust Anchor. Cisco fingerprints the critical hardware elements of a router, such as CPUs and ASICs, during manufacturing and stores the fingerprint in the tamper resistant Trust Anchor. This fingerprint is not only immutable once it is inserted into the Trust Anchor but it also cannot be read back from the Trust Anchor.

When the router boots up, UEFI firmware fingerprints the hardware elements of the router at boot and creates a fingerprint of the hardware elements. This fingerprint is sent to the Trust Anchor hardware, which will compare it against the master fingerprint stored inside the hardware. UEFI firmware will only boot the router if the Trust Anchor hardware can successfully verify the observed fingerprint at bootup against the master fingerprint.

As threats evolve, Cisco continues to enhance the security and resilience of our solutions.  While no vendor can guarantee security, we are committed to transparency and accountability and to acting as a trusted partner to our customers to address today’s, and tomorrow’s, security challenges.

Continue reading

Trust in IOS XR

I looked at the hardware Trust Anchor module (TAm) that enhances the security of Cisco Service Provider products and provides visibility into the authenticity and integrity of the platforms. In this blog I will go over the software functionalities in IOS XR that enhance the security posture of the router, defends the router against common attacks, and provides evidence of trust.

Buffer Overflow Protection

A buffer overflow attack involves a common error by developers where the input to an allocated buffer (a memory region) is not validated, and the input overflows the allocated memory. This attack can lead to execution of arbitrary code. A similar attack involves prior knowledge of where critical data is loaded into memory and then targeting that memory location.

IOS XR uses multiple runtime defenses (RTDs) to protect from such errors.

1. W^X (Write XOR Execute): This is a feature in Linux where any page of memory can either be written to or executed but not both. In the scenario where an input overflows the buffer, the overflow data exists in a memory region that can be written to but cannot be used to execute arbitrary malicious code.

2. Address Space Layout Randomization (ASLR): This is a Linux feature wherein the memory locations of running processes are randomized each time. This prevents critical data from always being loaded at the same location in memory, and makes it more difficult for an attacker to launch malicious operations on a specific, well-known memory location.

3. Object Size Checking (OSC): This is a compiler technique used to identify the size of objects, even at compile time for specific types of objects, and then detecting if the data being written will overflow the allocated memory. The compiler will flag such errors at compile time, if the errors can be detected at compile time, or it will add additional instructions to raise exceptions at run time.

4. Safe-C: Many library functions used in C are known to be quite difficult to use safely when it comes to certain memory related operations. Developers working on IOS XR use a safer and secure variant of the library functions, called the Safe-C Library. This provides an alternative to the standard C library calls, where memory accesses, particularly writes to memory locations, are first verified to be within bounds before data in memory is read from or written to. Note that not all modules in IOS XR fully utilize Safe-C libraries due to the maturity of the code base. Critical modules use Safe-C libraries and we migrate to Safe-C libraries in other modules as appropriate.

The above 4 features provide a much safer environment in which to run IOS XR software and mitigate a very common class of security problems.

Integrity Measurement Architecture (IMA)

Cisco hardware-anchored Secure Boot verifies the integrity of the image, including all firmware, to prevent inauthentic or compromised code from booting on a Cisco device. Once a router has booted up, it typically runs for months without a reboot. A malicious actor could get access to the router and tamper a binary at runtime and this would not be detected for a long time. To prevent such tampering of binaries at runtime, we are bringing Linux Integrity Measurement Architecture into IOS XR.

Linux IMA is a kernel security module which checks the integrity of every binary loaded into memory at runtime. Every binary carries a Cisco-issued signature. The Linux kernel validates this signature using Cisco’s Public IMA Certificate that is stored in the hardware-based Trust Anchor module. In the IMA measurement mode, the hashes of all binaries launched are logged into a secure location. In the IMA appraisal mode, if the signature validation fails, the binary will not be allowed to launch. Thus any accidental or malicious modification of a binary at runtime is detected and its execution prevented.  This significantly enhances security and allows the integrity the running software to be verified.

Remote Attestation

We have built a significant number of security controls into our Service Provider product and one of the important aspect of building trust in a product is the ability to verify that the router is in fact doing what it claims to be doing. If an attacker were to tamper the system, the very first action by the attacker will be to remove all evidence of the attack and to present the router as untampered. The only means of verifying the integrity of the router cannot be the router itself.

Remote Attestation will allow the operator to cryptographically verify that the router’s boot keys, boot configuration and all launched software have not been tampered. Cisco’s Trust Anchor supports Remote Attestation functionality where every aspect of the boot up process – starting from the verification of the root of trust and extending throughout the entire secure boot process – as well as the runtime IMA measurements are extended into Platform Configuration Registers (PCRs) in the Trust Anchor. Cisco’s software release process will provide Known Good Value (KGV) hashes for every software, firmware, and key material data shipped with the router.

Once the router is up and running, a verifying party will be able to request an attestation quote from the router. The Trust Anchor hardware can output the audit log and a PCR quote, signs the quote using an Attestation Private Key for that specific router and responds back to the verifying party. The verifying party will be able to use Cisco provided Known Good Value hashes and the Attestation Public Certificate to verify the attested PCR quotes and audit logs. This verification is protected against replay attacks using a nonce, and ensures the attestation is specific to a particular router by using Attestation key pairs that are unique to each router. Thus, one will be able to trust that the router hardware, boot keys, boot configuration and the running software have not been tampered.

Conclusion

Figure 1 Security Technologies at All Layers

We set a target of fulfilling security requirements at every layer of the Cisco Hardware and IOS XR Network OS. Figure 1 shows the various technologies in use that satisfy these requirements. Cisco’s Trust Anchor provides a foundation of trust for the Next Generation of Security in Service Provider routers that allows service providers to deploy Trusted Platforms, especially when deployed in remote and open locations. The security features in IOS XR software provide a strong defensive environment to run Cisco and Customer applications. Together, the union of Cisco’s Trust Anchor hardware and IOS XR software provide truly Trustworthy Solutions, where the trustworthiness of the system can be measured using Remote Attestation.

As new attacks emerge, Cisco is dedicated to further strengthen security and trustworthy solutions. We are committed to transparency and accountability, acting as a trusted partner to our customers to address evolving security threats.

Continue reading

Automate Device Provisioning with Cisco IOS XE Zero Touch Provisioning

When new hardware is ordered and it arrives on site, it’s an exciting time. New hardware! New software! … But new challenges too!  But the age-old challenge of getting new devices on the network doesn’t need to be one of them. Sitting in the lab pre-provisioning devices is no longer required if you’re using Cisco IOS XE, because of features like Cisco Network Plug-n-Play (PnP) and Zero Touch Provisioning (ZTP). PnP is the premium solution made possible with Cisco DNA Center, while Zero Touch Provisioning (ZTP) is for the do-it-yourself customers who don’t mind investing more time in configuring and maintaining the infrastructure required to bootstrap devices. IOS XE runs on the enterprise hardware and software platforms that includes Catalyst 9000 series of switches and wireless LAN controllers, and the ISR 1000 and 4000 series routers.

DHCP Configuration to enable Zero Touch Provisioning

ZTP works when the DHCP client on the IOS XE device gets a DHCP Offer that includes option 67. This options, also called the “bootfile name,” tells the device which file to load and from where it’s available. Lets look at a few examples of how we can configure this on either the ISC DHCP Server or on the Cisco IOS DHCP Server.

The configuration example for the Linux ISC DHCP dhcpd.conf is below:

subnet 192.168.69.0 netmask 255.255.255.0 {
option bootfile-name "http://192.168.69.1/ztp.py"; }

For the Cisco IOS DHCP Server:

ip dhcp pool ZTP_DEMO
 network 192.168.69.0 255.255.255.0
 option 67 ascii http://192.168.69.1/ztp.py

In these examples option 67 points to a HTTP URL and a Python script. When the device see’s the python file in the option 67 then it downloads and executes it during bootup when no other configuration is present and without any manual user intervention. Pretty neat!

Next let’s take a look at a packet capture, which shows interesting things.

You might notice that the Client Identifier (option 61) is not consistent between DHCP Discovers. This is not a mistake, or multiple devices, but actually the intended behavior. Since IOS XE 16.8 the Client Identifier alternates between the device serial number, and the MAC address of the management port. When the DHCP server issues the DHCP Offer in packet 8, the bootfile-name (option 67) is set to http://192.168.69.1/ztp.py – This is the Python script that will be downloaded and run once the device is ready.

Once the device is ready?

Yes, a few things need to happen before the Python script can run. Specifically, the Guest Shell container must start up. This is a linux container that runs within the IOS XE platform. This container has limited access to the IOS XE subsystem. This means that if you decide to run a crypto-miner within Guest Shell (not recommended!) that the IOS XE device will still handle the routing and switching of packets without any problems as the resource allocations for Guest Shell are separate from those responsible for the core capabilities of the device.

The power of the CLI, now with the flexibility of Python

Guest Shell does have some nice feature, specifically the Python API. This API allows the Guest Shell to send commands to the IOS XE operating system. What kind of commands? Show commands are supported with the Python CLI module cli.cli. Show commands are great for displaying information about the device, but are limited when it comes to making device configuration changes. To really harness the power of the Python API, the cli.execute and cli.configure modules provide a great deal of flexibility when it comes to device configuration. We can interact with the device through Python using the traditional “configure terminal” (“conf t”) interface and even send exec commands as needed. All the power of the CLI, now with the flexibility of Python.

So our container has started, and it knows which file to access. Lets look at an example ztp.py script next.

This simple Python script sets the Vlan1 interface IP, enables some AAA settings, turns on the NETCONF-YANG programmatic interface, and creates a user. After, it prints the output to the screen, so that if we have a console cable connected we get visual confirmation that the device has been set successfully.

This device has now been automatically configured using ZTP. It received an IP address on the management port from DHCP, the Python configuration file from a webserver via DHCP option 67, it started up Guest Shell container and configured remote access. We are now free to carry on with our day-to-day tasks as the device is online and in a known and managed state.

Continue reading

On-Box Python for Cisco Devices – the Why, What, and How

As a junior network engineer at a university I wrote a lot of management scripts in Perl.  I had scripts to do things such as check switchport configurations and upgrade switch code. Times have changed a lot since then. The university’s web server now runs in the cloud, rather than on my personal workstation, and Python has surpassed Perl as the scripting language du jour. Network automation with Python is now a major focus, making Python an extremely important tool.

Today I’m going to show you how to use Python scripts hosted on the box and integrated into IOS. This is far more powerful than my earlier-career scripts, and I have some simple examples for PCI compliance, Dynamic DNS ACL updates, and configuration validation.

As with many things in IT, we seem to be continually oscillating between “centralized” and “distributed.”  On-box hosting of Python scripts is an example of moving back toward distributed. My view on the argument is that it’s never about the extremes, but more about the balance—a bit like a pendulum constantly swinging as technology advances change what’s possible and practical.

Today, I want to demonstrate why Python scripts running on-box can be awesome.  I also want to explain how easy it is, based on the application hosting environment we’ve just released.  In addition, I’ll give some examples of how Python is even more powerful when combined with some of the existing IOS infrastructure such as Embedded Event Manager (EEM).

Why on-box Python?

There are three main advantages for Python scripts running on the device itself rather than externally.

◈ Scale: If I have a “sanity” script that I need to run regularly and it takes six seconds per device * 1000 devices, that would be 6,000 seconds, or one hour and 40 minutes.I could run them in parallel, but that still consumes resources on my management station, processing the data. It also potentially transports lots of data back to the management station, only to discard much of it.  An alternative is to distribute the work to the devices and get them to provide an update when they’re done.

◈ Security: Instead of having “utility” logins that connect into devices and export information for external processing, you can have the device process its data locally and just export the summary state. Data stays on the device, and less external connections are required.

◈ Autonomy: The biggest limitation of centralized processing is that it needs a network connection to the device. There is a set of use cases to modify device behaviour when it loses connectivity to other devices.  This can only be done on-box.

To illustrate the “what,” I’ve provided some sample scripts and use cases for the three points above. The code is published @  https://github.com/aradford123/on-box-python.git

To get started, we’re going to want to make sure Git is installed on your network device, run the following commands (after you’ve enabled guestshell; see section at the end for details).  The reason for using /flash/gs_script is that it’s a persistent directory and will be available on a switch stack switchover.

# install git

[guestshell@guestshell ~]$ sudo yum install git

# now install scripts into /flash/gs_script

[guestshell@guestshell ~]$ git clone https://github.com/aradford123/on-box-python.git /flash/gs_script

Example 1 – PCI compliance

Here’s an example of a use case that I think you’ll find interesting. One of our customers had a PCI requirement to ensure that any switch ports that were unused for more than seven days were disabled. (This was to prevent people from plugging in unauthorized devices.)

The script looks at all interfaces on the switch, and those that have been inactive (no traffic send/received) for more than seven days are shutdown. All interfaces that were shut down in a logged in a Cisco sparkroom. The interface description is updated with a message indicating the time/date it was shutdown by the PCI-check application.

To run the script, we’ll use the Embedded Event Manager.  EEM is a really powerful piece of IOS infrastructure that can be used to schedule the Python script to run.  The EEM cron job runs the Python script at 15 minutes past the hour, Monday to Friday.

event manager applet PCI-check

 event timer cron cron-entry "15 * * * 1-5"

 action 1.0 cli command "enable"

 action 1.1 cli command "guestshell run python bootflash:gs_script/src/pci-tool/pci_check.py --apply"

This script can now be run hourly (instead of weekly). It’s an example of scaling using on-box Python.

Example 2 – DNS ACL

Another customer had a requirement to keep an ACL updated with the latest DNS entries. For example, they wanted the ACL to reflect the real IP addresses of www.cisco.com and www.amazon.com.

This script automatically schedules its next execution based on the minimum Time To Live (TTL) of the DNS response. It ensures we don’t attempt to update more often than necessary.

The script logs entries that are added to the ACL via syslog, but that could be Cisco Spark or any other notification mechanism.

show logging

*Jul 12 11:44:11.174: %SYS-5-USERLOG_NOTICE: Message from tty4(user id: ): "Looking up cisco.com"

*Jul 12 11:44:11.253: %SYS-5-USERLOG_NOTICE: Message from tty4(user id: ): "Looking up amazon.com"

*Jul 12 11:44:11.341: %SYS-5-USERLOG_NOTICE: Message from tty4(user id: ): "adding IP: 72.163.4.161 to ACL: status: Success"

*Jul 12 11:44:11.351: %SYS-5-USERLOG_NOTICE: Message from tty4(user id: ): "adding IP: 54.239.25.208 to ACL: status: Success"

*Jul 12 11:44:11.360: %SYS-5-USERLOG_NOTICE: Message from tty4(user id: ): "adding IP: 54.239.17.6 to ACL: status: Success"

*Jul 12 11:44:11.370: %SYS-5-USERLOG_NOTICE: Message from tty4(user id: ): "adding IP: 54.239.26.128 to ACL: status: Success"

*Jul 12 11:44:11.379: %SYS-5-USERLOG_NOTICE: Message from tty4(user id: ): "adding IP: 54.239.17.7 to ACL: status: Success"

*Jul 12 11:44:11.389: %SYS-5-USERLOG_NOTICE: Message from tty4(user id: ): "adding IP: 54.239.25.200 to ACL: status: Success"

*Jul 12 11:44:11.398: %SYS-5-USERLOG_NOTICE: Message from tty4(user id: ): "adding IP: 54.239.25.192 to ACL: status: Success"

*Jul 12 11:44:17.605: %SYS-5-USERLOG_NOTICE: Message from tty5(user id: ): "reschedule in : 557 seconds: status: Success"

Here is the resulting ACL. Notice how the remarks are used to indicate the time the ACL changed. The last two entries were added at a later date.

9300#show run | sec canary

ip access-list extended canary_ip_in

remark Added 72.163.4.161 @Wed Jul 12 11:44:11 2017

permit ip any host 72.163.4.161

remark Added 54.239.25.208 @Wed Jul 12 11:44:11 2017

permit ip any host 54.239.25.208

remark Added 54.239.17.6 @Wed Jul 12 11:44:11 2017

permit ip any host 54.239.17.6

remark Added 54.239.26.128 @Wed Jul 12 11:44:11 2017

permit ip any host 54.239.26.128

remark Added 54.239.17.7 @Wed Jul 12 11:44:11 2017

permit ip any host 54.239.17.7

remark Added 54.239.25.200 @Thu Jul 13 16:34:37 2017

permit ip any host 54.239.25.200

remark Added 54.239.25.192 @Thu Jul 13 16:34:37 2017

permit ip any host 54.239.25.192

deny ip any any

This script uses a different type of EEM trigger, a countdown timer. The script self-updates the trigger based on the TTL of the DNS response. In the case below, it will fire in the next 557 seconds.

9300#show run | sec even

event manager applet DNS_update

 event timer countdown time 557

 action 1.0 cli command "enable"

 action 1.1 cli command "guestshell run python bootflash:gs_script/src/dns-update/DNS_update.py cisco.com amazon.com"

This is an example of security using on-box Python. No external access to the device is required.

Example 3 – Configuration change

This example uses an EEM event to look for a syslog message and execute a Python script. In this case, it looks for a configuration event, and fires the script.

This script will do two things:

◈ A sanity check. This example is a simple test to see if an IP address is reachable, but it could be more sophisticated. If the sanity check fails, then the configuration is rolled back.

◈ Log the changes to the configuration in a spark room.

This screenshow shows the configuration diff posted to a spark room.

Here’s an example of the sanity check:  I have a very simple sanitiy check that is looking for connectivity to 1.1.1.1.   While this example is trivial, the sanity check could be much more sophisticated (checking for OSPF neighours, number of connected hosts, etc.).

I shut down the loopback address 1.1.1.1, which is being checked by the sanity function.  The sanity function fails, triggering configuration rollback, and the current (working) configuration is restored.

9300(config)#int loopback 2

9300(config-if)#shut

9300(config-if)#exit

9300(config)#end

9300#

Jul 14 12:01:04.859: %SYS-5-CONFIG_I: Configured from console by vty0 (10.61.215.206)

Jul 14 12:01:15.648: %MLANG-3-LOG: config_check.py: Sanity 

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds:

.....

Success rate is 0 percent (0/5)

Jul 14 12:01:15.690: Rollback:Acquired Configuration lock.

Jul 14 12:01:15.690: %SYS-5-CONFIG_R: Config Replace is Done 

Jul 14 12:01:16.579: %MLANG-3-LOG: config_check.py: 

Total number of passes: 1

Rollback Done

Jul 14 12:01:18.127: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback2, changed state to up

Jul 14 12:01:18.127: %LINK-3-UPDOWN: Interface Loopback2, changed state to up

There are lots of other options for this script, including checking into a git repository and more enhanced sanity checks.

This Python script uses EEM in a different way. The first line of the script embeds an EEM registration. If there is a syslog message with pattern “CONFIG_I” in it, the script will be executed. (NOTE: This is actually a Python script, and the normal Python code is after this.)

::cisco::eem::event_register_syslog pattern "CONFIG_I" maxrun 60

# this is an example of an EEM policy trigger

# based of Joe Clarke version

#https://github.com/CiscoDevNet/python_code_samples_network/blob/master/eem_configdiff_to_spark/sl_config_diff_to_spark.py

I then tell EEM to look for the registration in the config_check.py script.

event manager directory user policy flash:

event manager policy config_check.py

This is an example of autonomy; if the configuration is changed, a sanity check is run to make sure the device is still functioning (and connected) to the network. If the sanity check fails, the configuration is rolled back.

Upgrade sanity check

It’s pretty simple to extend the use case above to check the status of the device before downloading and installing new software. Once the new code is installed, it will re-run the sanity check and either remove the old version of code or roll back depending on the status of the sanity check.

This is another example of autonomy using on-box Python.

How does this really work?

Python runs in a guestshell on the device.  The guestshell is CentOS or Montevista shell running as an application on the device. In order to enable application hosting you need to use the application hosting framework IOX.  To enable IOX is quite easy:

9300# conf t

9300(config)#iox

You then need to enable guestshell. This will take a few seconds.

9300# guestshell enable

Management Interface will be selected if configured

Please wait for completion

On switches running 16.8.1 and later, you need to configure a management interface, as shown below. This was added to allow access to guest shell from ports other than the mangement interface (GigabitEthernet0/0).

9300# conf t

app-hosting appid guestshell

 app-vnic management guest-interface 0

end

9300# guestshell enable

Management Interface will be selected if configured

Please wait for completion

Once it has started, you can either run a command or get an interactive shell session.

9300#guestshell run echo "hello world"

hello world

9300#guestshell

[guestshell@guestshell ~]$ 

The very first thing you will do is update the DNS settings. You can use vi, or just a simple echo statement.

echo -e "nameserver 8.8.8.8\ndomain cisco.com" > /etc/resolv.conf

Now to install some Python modules. This is pretty simple. Just use pip install. I am using the “-E” option as my switch needs a proxy to get to the internet.

[guestshell@guestshell ~]$ sudo -E  pip install netaddr

Collecting netaddr

  Downloading netaddr-0.7.19-py2.py3-none-any.whl (1.6MB)

    100% |################################| 1.6MB 257kB/s 

Installing collected packages: netaddr

Successfully installed netaddr-0.7.19

DevOps

The next question you’re asking is how do I keep the scripts on the device updated? It would be a pain to have to copy/install new scripts all the time.

The solution is pretty simple. Store the scripts in a git repository (so you have full version control), then use an EEM script to “git pull” regularly to keep them updated.

Here’s a simple git update script:

[guestshell@guestshell ~]$ cat /flash/gs_script/utils/update_git.sh 

#!/bin/bash

(cd /flash/gs_script; git pull)

All that’s required is another EEM cron job to keep the device updated with the latest git repository.

event manager applet GIT-sync

 event timer cron cron-entry "0,30 * * * *"

 action 1.0 cli command "enable"

 action 1.1 cli command "guestshell run bootflash/gs_script/src/util/update_git/sh"

Continue reading