Scaling the Adoption of Private Cellular Networks

1. Private Networks

Private networks are essential to every enterprise. Enterprises use private networks to integrate information systems into their operations and to continue their digital transformation through technology integration into business processes. Over the past twenty years, Wi-Fi has become an essential component of nearly every private network. Wi-Fi accelerates digital transformation and supports a wide variety of enterprise-specific value propositions.

Back in the early 2000s, Cisco’s own analysis estimated that Wi-Fi adoption by its employees was resulting in staff being 86 minutes more productive per day than their tethered counterparts. More recently, analysis of Wi-Fi adoption by retailers indicates improvements in top and bottom lines, with positive impact on customer loyalty, increased insights through the use of wireless network analytics and increased sales. Other examples include industrial predictive maintenance use cases that are delivering 10-20% increases in equipment uptime and 5-10% decreases in overall maintenance costs. One report indicates that Wi-Fi is being used in 34% of such deployments across different industry sectors. Finally, in sports and entertainment, digitization is transforming the fan experience. At the SoFi stadium, the private network uses a massive deployment of more than 2500 Cisco Access Points to deliver the fastest and most reliable fan experience, that is reported to have resulted in the most digitally engaged set of spectators.

Across all verticals, from carpeted office, through to retail, manufacturing and sports and entertainment, Wi-Fi based private networks have proved themselves adept at supporting the widest range of business needs and value chains.

2. Complementary wide-area cellular technology

In parallel with enterprise adoption of local-area Wi-Fi networks, several industry segments have integrated cellular wide-area technology into their business processes. The earliest use cases adopting wide-area cellular technology have focused on the benefits offered by the wide area coverage offered by public cellular providers. In contrast to the local-area private Wi-Fi networks, public cellular coverage supports nationwide service. Phone based systems that connect vehicle users have always been an important segment for public cellular providers. But now we see integration of cellular modem technologies into the latest utility meter offerings, where the cellular connectivity is able to provide near real time visibility of energy consumption to utility customers. The wide area coverage ensures that a uniform solution can be offered across a particular geography.

Transportation systems that integrate cellular modems leverage the same wide area capability. The latest connected warning signs now benefit from secure connectivity from road-side control cabinets to the central data centre. Fleet management solutions use wide area cellular connectivity to improve vehicle maintenance, lower fuel consumption as well as automated logging of odometers, rev-meters and accelerometers.

Over the years, public cellular providers have adapted their product and services to enable a range of different verticals to integrate cellular modems that benefit from wide area connectivity into their business processes while supporting a range of different business relevant value propositions.

3. The emergence of private metropolitan-area cellular networks

The coverage advantage of public cellular systems has driven adoption by those use cases that necessitate national or international coverage. So called “metropolitan area network” use cases can similarly benefit from this coverage advantage. One of the earliest examples of such is the Australian regulator ACMA that permits use of 3GPP defined 1800 MHz cellular frequencies for supporting point-to-multipoint systems for private networks in regional and remote areas of Australia. This has led to the adoption of private cellular networks by mining and energy companies that have operations that span over significant distances and where the increased range of cellular based point-to-multipoint systems offer clear advantages compared to local Wi-Fi based unlicensed alternatives.

In the US, many utility companies used to operate private metropolitan-area networks based on WiMAX technology. These have now transitioned to private LTE based systems, enabled by the recent innovation in spectrum licensing associated with CBRS. Now airports are using these new licenses to operate private LTE networks, leveraging the extended range offered by cellular frequencies to enable better coverage of the apron where aircraft are parked to support baggage and maintenance use-cases.

In the UK, from 2019, Ofcom took the decision to augment its approach to licensing spectrum for cellular operation, with the introduction of shared access to spectrum for a newly introduced 5G band. The specific 5G band covers 400 MHz of spectrum between 3.8 and 4.2 GHz. Ofcom’s rationale for the novel approach was to “enable the deployment of private networks with greater control over security, resilience and reliability”. Ofcom has made two types of local license available:

◉ a low power license that authorizes the licensee to deploy as many radio access points within a 50 metre radius of a defined reference point. The radio access points have a maximum emitted power of 24 dBm (for a 20 MHz carrier) and an antenna height limited to 10 metres above ground.

◉ a medium power licensed that authorizes the licensee to deploy a single radio access point at a defined rural location where the radio access point has a maximum emitted power of 42 dBm (for a 20 MHz carrier).

Previously businesses wanting to benefit from integrating cellular service into their business operations had to engage with public cellular operators that had been licensed exclusive spectrum. Now, these new regulatory approaches are allowing businesses to deploy local and metropolitan cellular systems independently of public operators.

4. Standardization of 3GPP Non-Public Networks

5G is targeted at fulfilling the requirements from different industrial segments. In order to meet such expectations, 3GPP Release 16 defines enhancements to the 5G system to support Non-Public Networks (NPNs). This introduces two new cellular identifiers, a Non-Public Network Identity (NID) and a Closed Access Group Identity (CAG-ID), enabling devices to perform non-public network identification, discovery and selection as well as enabling the NPN to implement access controls. In release 16, the NPN can be deployed in two different configurations:

◉ “stand-alone” mode (S-NPN) where the NPN is deployed in isolation of a public cellular network, and

◉ in“public network integrated” mode (PNI-NPN) where the NPN leverages 5GS functionality delivered by the public cellular network, including SIM/identity management.

The PNI-NPN deployment can, subject to agreed policies, enable an enterprise device to seamlessly transition between the NPN access network and the public cellular network. In contrast, the Release 16 S-NPN is considered isolated from other networks. However, release 17 has seen further enhancements with the ability for a device to access the S-NPN using credentials owned by a separate credential holder (CH) entity. The credential holder can be a private enterprise, or can be a public cellular operator, enabling a SIM-based public cellular identity to be used to authenticate a device on an S-NPN. Note, whereas such a scenario would conventionally be referred to as “roaming”, 3GPP’s use of roaming is limited to using another public cellular operator’s visited network and hence 3GPP refers to authentication between S-NPN and CH as “interworking”.

These latest NPN capabilities, when coupled with the new approaches to licensing cellular frequencies, are specifically aimed at broadening the applicability of private cellular networks to the widest range of businesses.

5. Operating inter-connected networks

Operating interconnections between networks, be that peering interconnect, an ISP service or roaming, always requires a technical framework and a financial framework that are referenced in terms defined in legal agreements agreed between parties.

The GSM Association came into existence to drive matters essential for the implementation of a pan European roaming service. Since its inception back in the 1990s, GSMA’s remit has since broadened to address services and solutions that underpin interoperability and make mobile work across the world. Serving its operator members, GSMA defines how to operationalize the roaming reference points defined by 3GPP to enable their operator members to support international roaming. This includes defining international roaming agreements, operating systems to enable collecting and sharing roaming related business and technical information, and procedures that enable the exchange of roaming signalling between different operators.

In contrast to the unified inter-operator cellular system operationalized by GSMA, historically the private wireless industry has taken a decentralized approach, with each individual wireless hotspot provider defining their own legal terms and getting end-users to agree to those before being able to access via the private network. This decentralized approach has not inhibited private wireless hotspot adoption, with some estimates of over 500 million Wi-Fi hotspots available worldwide. However, more recently it has inhibited usage, as users avoid the required user engagement necessary to accept the hotspot’s legal terms.

6. Scalability

How to scale interconnect is a significant issue for private networks. While GSMA has been successful in scaling roaming between the 800 public cellular operators, there are still challenges in scaling GSMA interconnect. This requires the use of roaming hub providers to scale operations. Importantly, such hub models are predicated on the use of financially settled service that can be used to pay for the services of the roaming hub provider. In contrast, the businesses that have deployed private wireless networks frequently do not require financial remuneration from another enterprise in exchange for providing access, be that from a third party private enterprise or a public cellular operator. Without financial remuneration to enable conventional hub models, an alternative approach to scaling may be required for private networks.

Another key aspect of scaling private networks is related to the dimensioning of inter-connected signalling that is a function of the geographical coverage of the private wireless access network and the number of subscribers served by a particular credential holder. Public cellular networks provide nationwide coverage to 10s of millions of subscribers. Such scale drives significant roaming signalling traffic between cellular providers that enable assumptions related to longevity of signalling connections to be embedded into technical procedures that support bidirectional signalling between all public cellular operators. In contrast, early data from the Wireless Broadband Alliance (WBA) on adoption of its OpenRoaming federation, a system designed to operate with private wireless networks, indicates that dimensioning in private deployments may be as low as one thousandth of that experienced by a conventional public cellular network.

With some forecasting 1 million private cellular networks by the end of the decade, a thousand times the current number of public cellular networks, we can anticipate the future scalability challenges of being able to support 1000 times more networks, each with 1/1000th of the signalling load.

7. Interconnecting 3GPP Non-Public Networks

The opportunity of being able to interconnect 3GPP Non-Public Networks with third party systems is aimed at fulfilling 5G’s opportunity at serving different industrial segments. The challenges faced include defining the technical framework to simplify adoption of interconnect functionality, agreeing procedures that are amenable to the administrators of information technology (IT) and operation technology (OT) systems in separate businesses while simultaneously supporting the unique scaling attributes of private networks and separate credential holders.

Complementing the technical framework, a legal framework that enables legal teams in private enterprises, individual credential holders and public cellular operators to scale is required. The legal terms need to ensure cellular devices, be that end-user smartphones or embedded cellular modems, experience a great service when using the private wireless networks. Finally, the interconnect systems should not assume that financial remuneration for providing wireless service is going to be available to fund the operation of hubs to scale interconnect across the millions of private networks.

Simplification and scaling of private 5G solutions is going to be critical to ensure the full potential of 5G can be harnessed. The 5G DRIVE (Diversified oRAN Integration & Vendor Evaluation) project led by Virgin Media O2 and part-funded by the UK DCMS, Cisco and co-partners is targeted at defining the use of the new 5G Security Edge Protection Proxy (SEPP) roaming interface to connect public and private 5G networks. Cisco is invested in solving the key problem of how best to integrate private 3GPP Non-Public Networks with established public cellular networks, affordably, securely and at scale. Cisco will use its membership of the 5GDrive project to showcase its 5G-as-a-Service offer that is aimed at lowering the barriers to adoption for 3GPP Non-Public Networks as well sharing key learnings from its incubation of the OpenRoaming systems from an internal Cisco proof-of-concept to an industry standard supporting roaming across over a million private hotspots. Watch out for upcoming blogs where we will be sharing more information about proof of concept demonstrations of how SEPP-based roaming could be adapted to lower barriers to adoption for private enterprises.

Source: cisco.com

Continue reading

How to Use Presence Web Services

Presence of mind

Jabber is so last decades. Webex and its competition are the best modern means of messaging. But Cisco IM&P, a companion server to Cisco Call Manager, is still the best way to subscribe to user presence updates.

Suppose you have a group of employees to whom you assign tasks as they come in. If you can watch the presence of that group, you’ll know who is available, who is away, who is on the phone, etc. You can build an application that automatically assigns tasks according to the presence of the users.

The Presence Web Services (PWS) API, a feature of Cisco IM&P, is ideal for this kind of application. In my experience as a former developer support engineer, I noticed many developers don’t quite understand how to use PWS properly. I hope that by the time you’re done reading this, you’ll have a good grasp of everything involved in making PWS work for you.

Here’s a condensed breakdown of the steps:

1. Log in an application user with app username and password

a. This operation returns the application user session key

2. Use the application user session key to log in an end user

a. This operation returns an end user session key

3. Create a web service to handle presence notifications

a. Run this web service to listen on a common port, e.g., 8080

4. Use the application user session key to register the URL of your web service as an endpoint

a. This returns an endpoint ID

5. Use the end user session key to subscribe to one or more end user contacts

a. This returns a subscription ID

6. Create a script to fetch the subscribed presence, using the subscription ID

a. For example, get_subscribed_presence.py

In steps 1 and 2, there’s a choice called “force=”. If you set “force=true”, the server will return a new session key every time. I recommend you use “force=false”, so that it keeps re-using the same session key. This covers a multitude of programming sins.

In Step 3, it is important to use a common port, like 80, 82, 8080, etc. If your web service is based on Python and you use the Flask library, the default port for Flask is 5000, which will not work. You must tell flask to use one of the common ports, instead.

Once you have completed steps 1 through 5, any change in the presence of the contacts in your step 5 subscription will trigger a REST GET operation on the endpoint. The GET will pass two parameters: The subscription id which should always be 1 with these scripts, and etype, which should always be “PRESENCE_NOTIFICATION”.

Your application should then use the subscription ID to fetch all the presence changes for that subscription. The API for that is getSubscribedPresence. The script that invokes getSubscribedPresence is, coincidentally, get_subscribed_presence.py.

The sample scripts use REST, but you can also use SOAP.

No problemo!

A common problem occurs when you run your endpoint after a contact’s presence already changed. The server will send a presence notification to the endpoint, but the endpoint isn’t running, so that notification never gets to the endpoint, and the endpoint doesn’t fetch the subscribed presence information. This is a problem because, if for any reason you don’t fetch the presence values on that subscription, the server will stop sending future notifications until you do.

So, the script you create in Step 6 is a fail-safe. Suppose a contact, Carlotta Tendant, switches from AVAILABLE to AWAY. The server will notify the web service at the endpoint URL that a change in presence occurred. If your endpoint isn’t active, or it does not pick up the notification and fetch the presence information, the server will stop sending presence notifications until you fetch that presence information.

It is important to know that the presence notification doesn’t send any contact information or the fact that Carlotta is now AWAY; it just notifies the web service that a presence has changed for one or more contacts for that subscription. Your web service must fetch the information about the contact and the contact’s presence.

To avoid the possibility of missed notifications, run the get_subscribed_presence.py script once everything is set up and ready and your endpoint is running. This grabs the information for the users and their presence, and thus clears the queue for the server to send new presence notifications.

There is another reason the web service may not receive a notification. If the Cisco IM&P server CPU usage reaches 80% or higher, the server stops sending notifications until the CPU usage drops below 80%. Here’s how to compensate for that possibility. Write your app to perform a get subscribed presence at an interval of every 10 minutes (or whichever seems best), just to make sure that if, for any reason, your application did not act on a presence notification, the queue will clear, and notifications will continue.

Scripts

WARNING: Don’t use my sample scripts on a production server. These are for instructional purposes only.

My sample scripts are as follows:

pws-create.py

pws-delete.py

endpoint.py

get_subscribed_presence.py

And there are some data files the script uses to get information about the server, the host for the endpoint, app user, end user, and the contacts for your presence subscription.

serverparams.json (points to your Cisco IM&P server and the host IP address for the endpoint)

appuser.json (has the application username and password)

enduser.json (has the end user name. You use the session key from your application user login)

contacts.list (the list of contacts for which you will subscribe to get presence notifications)

Order Up

Here’s how you run the scripts, in order.

1. python3 pws-delete.py

    1. This removes all endpoints and subscriptions so you can start fresh

2. python3 pws-create.py

    1. This sets up the endpoint and subscribes to the presence of contacts in list. It uses serverparams.json to identify your Cisco IM&P server and the IP address of the host where your endpoint will run.

3. python3 endpoint.py

    1. This is the endpoint script. It uses the Flask Python library to work as a web service.

4. python3 get_subscribed_presence.py 1 BASIC_PRESENCE (or RICH_PRESENCE)

   1. You run this after the endpoint web service is up and running. This clears out any pending subscription updates and notifications so that the queue is empty and future notifications will work.

If you look at the code in the sample endpoint script, for the web service endpoint doesn’t include the code to fetch the subscription presence. I put all that into the get_subscribed_presence.py script. My endpoint simply executes the script externally like so:

subprocess.run("python3 get_subscribed_presence.py "+id+" "+etype, shell=True)

The endpoint will know the value of id and etype and pass the values when it runs get_subscribed_presence.py. If you want to run the script yourself, however, you need to pass values at the command line, for example:

python3 get_subscribed_presence.py 1 BASIC_PRESENCE

You can also use RICH_PRESENCE instead if that’s what you want. If you’re done everything correctly, the subscription id will always be 1, which is why you pass the number 1 to the script at the command line.

The sample script doesn’t do anything with the presence information. It prints it to the console where you run the endpoint web service. Your application must perform your needed task, such as updating a display of contacts and their presence. 

Source: cisco.com

Continue reading

Cisco Champions the Powerful, Evolving Networking Software Stack

With the interconnection of billions of devices in public and private networks and many applications and services moving to the cloud, software is increasingly becoming independent of and abstracted from hardware. At public cloud vendors like Amazon Web Services, Google Cloud Platform, and Microsoft Azure, hardware has been commoditized and software has taken center stage.

At Cisco, resellers and enterprise customers put complex solutions together using our products. The integration of switches, routers, and other gear with software used to require up to a one-year qualification cycle. But with the cloud providers, it’s immediate. Today, more native cloud concepts have been added to Cisco IOS XE software. Quarter by quarter, our enterprise software is becoming more efficient and cost-effective, more automated, and more programmable.

From Physical to Virtual to Cloud Native 

The first incarnation of Cisco enterprise cloud-enabled products was the virtualization of physical hardware devices in the cloud as virtual machines. They had all the existing concepts and features customers were used to in existing physical Cisco platforms.

In recent years we’ve been moving from physical to virtual to cloud-native products. As customers are becoming more aware and ready to consume cloud-native features, Cisco IOS XE is being enriched to provide those features. At 190 million lines of code―more than 300 million when vendor software development kits (SDKs) and open-source libraries are added―Cisco IOS XE runs 80+ platforms for access, distribution, core, wireless, and WAN layers. It facilitates a myriad of combinations of hardware and software, forwarding, and physical and virtual form factors.

Why Cisco? 

Prospective Cisco customers and competitors may ask, why spend $5000 for an enterprise switch when you can spend $1000? The answer is that our customers know that buying a cheaper switch may lack the features they need. Less expensive gear will also potentially add to their maintenance costs because the components may not be as good as Cisco’s.

Another reason to buy Cisco is due to the breadth of our enterprise portfolio. Any one company can do one vertical market well. With IOS XE, we have integrated everything across the networking software stack, and across the entire enterprise network, and we’re working to keep it simple across multiple network domains.

Efficiency and Cost-effectiveness 

With networking becoming increasingly feature-rich and complex, simpler networking software translates to greater efficiency, a smaller headcount, and fewer onsite visits to fix problems. For example, Cisco IOS XE provides simplified app hosting using a Docker image in a container and deployment using device controller tools. It supports third-party, off-the-shelf applications built using Linux toolchains that allow business apps to run at the network edge.

Other examples include the simplification of development, debugging, and device validation with Cisco Platform Abstraction (CPA) and unified software tracing that integrates traces from software running anywhere in a network for more complete visibility into 100+ processes in real-time. Another example of Cisco IOS XE simplicity is virtualization technology that runs over optical fiber, enabling switches to be physically located up to thousands of miles away from each other.

The Power of Automation 

Cisco IOS XE is becoming more and more self-driving. Cisco developers are increasingly taking away the manual tasks required to manage the network by automating them. That makes networks easier and cheaper to maintain and faster to debug.

Examples include the automation of image upgrades using Cisco DNA Center and support for programmable microservices to replace manual device upgrades, repurposing, and management. Other automated processes include streaming telemetry and analytics in all layers of software that run at the speed of events observed (e.g., faster than two million route updates per second) to handle the huge scale of networking operations.

Programmability 

Systems administrators in enterprise companies are constantly upgrading, repurposing, and managing thousands of switches. An advanced networking software stack must be able to manage multi-vendor networks using native and open-source data models. Cisco IOS XE supports a suite of Google Remote Procedure Call (gRPC)-based microservices that simplify and lighten workloads with programmability. They allow administrators to programmatically manage Cisco enterprise devices.

The IOS XE Development Environment  

A lot of enterprise software takes years to develop. The Cisco software development environment rolls out new solutions in months.

Developers spend 60-70% of their time developing software instead of application logic. The IOS XE development environment is automating as many common capabilities (like show commands, tracing, telemetry, export for dashboard, hand wiring HA code, testing base ISSU compatibility checks, and mocking for unit tests) as possible to avoid the need to hand code them. With hand coding, every one of these features would require developers to generate two-to-three times as much code. Hand coding is also not amenable to automated, flexible deployments and in the current development trajectory will not fit into the low-footprint devices we ship.

The Cisco Enterprise Networking software development team works at a solution level, conducting pre-qualification testing and providing the tools to control an entire enterprise dashboard from a single dashboard.

Source: cisco.com

Continue reading

Cisco Secure Firewall on AWS: Build resilience at scale with stateful firewall clustering

Organizations embrace the public cloud for the agility, scalability, and reliability it offers when running applications. But just as organizations need these capabilities to ensure their applications operate where needed and as needed, they also require their security does the same. Organizations may introduce multiple individual firewalls into their AWS infrastructure to produce this outcome. In theory, this may be a good decision, but in practice—this could lead to asymmetric routing issues. Complex SNAT configuration can mitigate asymmetric routing issues, but this isn’t practical for sustaining public cloud operations. Organizations are looking out for their long-term cloud strategies by ruling out SNAT and are calling for a more reliable and scalable solution for connecting their applications and security for always-on protection.

To solve these challenges, Cisco created stateful firewall clustering with Secure Firewall in AWS.

Cisco Secure Firewall clustering overview

Firewall clustering for Secure Firewall Threat Defense Virtual provides a highly resilient and reliable architecture for securing your AWS cloud environment. This capability lets you group multiple Secure Firewall Threat Defense Virtual appliances together as a single logical device, known as a “cluster.”

A cluster provides all the conveniences of a single device (management and integration into a network) while taking advantage of the increased throughput and redundancy you would expect from deploying multiple devices individually. Cisco uses Cluster Control Link (CCL) for forwarding asymmetric traffic across devices in the cluster. Clusters can go up to 16 members, and we use VxLAN for CCL.

In this case, clustering has the following roles:

Figure 1: Cisco Secure Firewall Clustering Overview

The above diagram explains traffic flow between the client and the server with the insertion of the firewall cluster in the network. Below defines the roles of clustering and how packet flow interacts at each step.

Clustering roles and responsibilities 

Owner: The Owner is the node in the cluster that initially receives the connection.

◉ The Owner maintains the TCP state and processes the packets. 

◉ A connection has only one Owner. 

◉ If the original Owner fails, the new node receives the packets, and the Director chooses a new Owner from the available nodes in the cluster.

Backup Owner: The node that stores TCP/UDP state information received from the Owner so that the connection can be seamlessly transferred to a new owner in case of failure.

Director: The Director is the node in the cluster that handles owner lookup requests from the Forwarder(s). 

◉ When the Owner receives a new connection, it chooses a Director based on a hash of the source/destination IP address and ports. The Owner then sends a message to the Director to register the new connection. 

◉ If packets arrive at any node other than the Owner, the node queries the Director. The Director then seeks out and defines the Owner node so that the Forwarder can redirect packets to the correct destination. 

◉ A connection has only one Director. 

◉ If a Director fails, the Owner chooses a new Director.

Forwarder: The Forwarder is a node in the cluster that redirects packets to the Owner. 

◉ If a Forwarder receives a packet for a connection it does not own, it queries the Director to seek out the Owner.  

◉ Once the Owner is defined, the Forwarder establishes a flow, and redirects any future packets it receives for this connection to the defined Owner.

Fragment Owner: For fragmented packets, cluster nodes that receive a fragment determine a Fragment Owner using a hash of the fragment source IP address, destination IP address, and the packet ID. All fragments are then redirected to the Fragment Owner over Cluster Control Link.  

Integration with AWS Gateway Load Balancer (GWLB)

Cisco brought support for AWS Gateway Load Balancer (Figure 2). This feature enables organizations to scale their firewall presence as needed to meet demand.

Figure 2: Cisco Secure Firewall and AWS Gateway Load Balancer integration 

Cisco Secure Firewall clustering in AWS

Building off the previous figure, organizations can take advantage of the AWS Gateway Load Balancer with Secure Firewall’s clustering capability to evenly distribute traffic at the Secure Firewall cluster. This enables organizations to maximize the benefits of clustering capabilities including increased throughput and redundancy. Figure 3 shows how positioning a Secure Firewall cluster behind the AWS Gateway Load Balancer creates a resilient architecture. Let’s take a closer look at what is going on in the diagram.

Figure 3: Cisco Secure Firewall clustering in AWS

Figure 3 shows an Internet user looking to access a workload. Before the user can access the workload, the user’s traffic is routed to Firewall Node 2 for inspection. The traffic flow for this example includes:

User -> IGW -> GWLBe -> GWLB -> Secure Firewall (2) -> GLWB -> GWLBe -> Workload

In the event of failure, the AWS Gateway Load Balancer cuts off existing connections to the failed node, making the above solution non-stateful.

Recently, AWS announced a new feature for their load balancers known as Target Failover for Existing Flows. This feature enables forwarding of existing connections to another target in the event of failure.

Cisco is an early adaptor of this feature and has combined Target Failover for Existing Flows with Secure Firewall clustering capabilities to create the industry’s first stateful cluster in AWS.

Figure 4: Cisco Secure Firewall clustering rehashing existing flow to a new node

Figure 4 shows a firewall failure event and how the AWS Gateway Load Balancer uses the Target Failover for Existing Flows feature to switch the traffic flow from Firewall Node 2 to Firewall Node 3. The traffic flow for this example includes:

User -> IGW -> GWLBe -> GWLB -> Secure Firewall (3) -> GLWB -> GWLBe -> Workload

Source: cisco.com

Continue reading

Introducing Cisco Cloud Network Controller on Google Cloud Platform – Part 3

Part 1 and Part 2 of this blog series covered native cloud networking and firewall rules automation on GCP, and a read through is recommended for completeness. This final post of the series is about enabling external access for cloud resources. More specifically, it will focus on how customers can enable external connectivity from and to GCP, using either Cloud Native Router or Cisco Cloud Router (CCR) based on Cisco Catalyst 8000v, depending on use case.

By expanding previous capabilities, Cisco Cloud Network Controller (CNC) will provision routing, automate VPC peering between infra and user VPCs, and BGP IPSec connectivity to external networks with only a few steps using the same policy model.

Scenario

This scenario will leverage the existing configuration built previously represented by network-a and network-b VPCs. These user VPCs will be peered with the infra VPC in a hub and spoke architecture, where GCP cloud native routers will be provisioned to establish BGP IPSec tunnels with an external IPSec device. The GCP cloud native routers are composed by the combination of a Cloud Router and a High-availability (HA) Cloud VPN gateway.

The high-level topology below illustrates the additional connections automated by Cisco CNC.

Provisioning Cloud Native Routers

The first step is to enable external connectivity under Region Management by selecting in which region cloud native routers will be deployed. For this scenario, they will be provisioned in the same region as the Cisco CNC as depicted on the high-level topology. Additionally, default values will be used for the IPSec Tunnel Subnet Pool and BGP AS under the Hub Network representing the GCP Cloud Router.

The cloud native routers are being provisioned purposely on a different region to illustrate the ability of having a dedicated hub network with external access. However, they could have been deployed on the same region as the user VPCs.

Note: a brief overview of the Cisco CNC GUI was provided on Part 1.

Enabling External Networks

The next step is to create an External Network construct within the infra tenant. This is where an external VRF is also defined to represent external networks connected to on-premises data centers or remote sites. Any cloud VRF mapped to existing VPC networks can leak routes to this external VRF or can get routes from it. In addition to the external VRF definition, this is also where VPN settings are entered with the remote IPSec peer details.

The configuration below illustrates the stitching of the external VRF and the VPN network within the region where the cloud native routers are being provisioned in the backend. For simplicity, the VRF was named as “external-vrf” but in a production environment, the name should be defined wisely and aligned to the external network as to improve operations.

The VPN network settings require public IP of the remote IPSec device, IKE version, and BGP AS. As indicated earlier, the default subnet pool is being used.

Once the external network is created, Cisco CNC generates a configuration file for the remote IPSec device to establish BGP peering and IPSec tunnels with the GCP cloud native routers. Below is the option to download the configuration file.

Configuring External IPSec Device

As the configuration file provides most of the configuration required for the external IPSec device, customization is needed only on tunnel source interface and routing settings where applicable to match local network requirements. In this example, the remote IPSec device is a virtual router using interface GigabitEthernet1. For brevity, only one of the IPSec tunnels config is shown below along with all the other config generated by Cisco CNC.

vrf definition external-vrf

    rd 100:1

    address-family ipv4

    exit-address-family

interface Loopback0

    vrf forwarding external-vrf

    ip address 41.41.41.41 255.255.255.255

crypto ikev2 proposal ikev2-1

    encryption aes-cbc-256 aes-cbc-192 aes-cbc-128

    integrity sha512 sha384 sha256 sha1

    group 24 21 20 19 16 15 14 2

crypto ikev2 policy ikev2-1

    proposal ikev2-1

crypto ikev2 keyring keyring-ifc-3

    peer peer-ikev2-keyring

        address 34.124.13.142

        pre-shared-key 49642299083152372839266840799663038731

crypto ikev2 profile ikev-profile-ifc-3

    match address local interface GigabitEthernet1

    match identity remote address 34.124.13.142 255.255.255.255

    identity local address 20.253.155.252

    authentication remote pre-share

    authentication local pre-share

    keyring local keyring-ifc-3

    lifetime 3600

    dpd 10 5 periodic

crypto ipsec transform-set ikev-transport-ifc-3 esp-gcm 256

    mode tunnel

crypto ipsec profile ikev-profile-ifc-3

    set transform-set ikev-transport-ifc-3

    set pfs group14

    set ikev2-profile ikev-profile-ifc-3

interface Tunnel300

    vrf forwarding external-vrf

    ip address 169.254.0.2 255.255.255.252

    ip mtu 1400

    ip tcp adjust-mss 1400

    tunnel source GigabitEthernet1

    tunnel mode ipsec ipv4

    tunnel destination 34.124.13.142

    tunnel protection ipsec profile ikev-profile-ifc-3

ip route 34.124.13.142 255.255.255.255 GigabitEthernet1 192.168.0.1

router bgp 65002

    bgp router-id 100

   bgp log-neighbor-changes

    address-family ipv4 vrf external-vrf

        network 41.41.41.41 mask 255.255.255.255

        neighbor 169.254.0.1 remote-as 65534

        neighbor 169.254.0.1 ebgp-multihop 255

        neighbor 169.254.0.1 activate

Verifying External Connectivity status

Once configuration is applied, there are a few ways to verify BGP peering and IPSec tunnels between GCP and external devices: via CLI on the IPSec device itself and via Cisco CNC GUI on the External Connectivity dashboard.

In the GCP console (infra project), under Hybrid Connectivity, it shows both the IPSec and BGP sessions are established accordingly by the combination of a Cloud Router and an HA Cloud VPN gateway automated by Cisco CNC, upon definition of the External Network. Note that the infra VPC network is named as overlay-1 by default as part of the Cisco CNC deployment from the marketplace.

Route Leaking Between External and VPC Networks

Now that BGP IPSec tunnels are established, let’s configure inter-VRF routing between external networks and existing user VPC networks from previous sections. This works by enabling VPC peering between the user VPCs and the infra VPC hosting VPN connections, which will share these VPN connections to external sites. Routes received on the VPN connections are leaked to user VPCs, and user VPC routes are advertised on the VPN connections.

Using inter-VRF routing, the route is leaked between the external VRF of the VPN connections and the cloud local user VRFs. The configuration below illustrates route leaking from external-vrf to network-a.

The reverse route leaking configuration from network-a to external-vrf is filtered with Subnet IP to show granularity. Also, the same steps were performed for network-b but not depicted for brevity.

In addition to the existing peering between network-a and network-b VPCs, now both user VPCs are also peered with the infra VPC (overlay-1) as depicted on the high-level topology.

By exploring one of the peering connection details, it is possible to see the external subnet 41.41.41.41/32 in the imported routes table.

On the remote IPSec device, the subnets from network-a and network-b VPCs are learned over BGP peering as expected.

remote-site#sh bgp vpnv4 unicast vrf external-vrf

<<<output omitted for brevity>>>

     Network          Next Hop            Metric LocPrf Weight Path

Route Distinguisher: 100:1 (default for vrf external-vrf)

 *>   41.41.41.41/32   0.0.0.0                  0         32768 i

 *    172.16.1.0/24    169.254.0.5            100             0 65534 ?

 *>                    169.254.0.1            100             0 65534 ?

 *    172.16.128.0/24  169.254.0.5            100             0 65534 ?

 *>                    169.254.0.1            100             0 65534 ?

remote-site#

Defining External EPG for the External Network

Up to this point, all routing policies were automated by Cisco CNC to allow external connectivity to and from GCP. However, firewall rules are also required for end-to-end connectivity. This is accomplished by creating an external EPG using subnet selection as the endpoint selector to represent external networks. Note that this external EPG is also created within the infra tenant and associated to the external-vrf created previously.

The next step is to apply contracts between the external EPG and the previously created cloud EPGs to allow communication between endpoints in GCP and external networks, which in this scenario is represented by 41.41.41.41/32 (loopback0 on remote IPSec device). As this is happening across different tenants, the contract scope is set to global and exported from the engineering tenant to the infra tenant and vice-versa, if allowing traffic to be initiated from both sides.

To the cloud connectivity

From the cloud connectivity

On the backend, the combination of contracts and filters translates into proper GCP firewall rules, as covered in details on Part 2 of this series. For brevity, only the outcome is provided below.

remote-site#ping vrf external-vrf 172.16.1.2 source lo0

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 172.16.1.2, timeout is 2 seconds:

Packet sent with a source address of 41.41.41.41 !!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 84/84/86 ms

remote-site#ping vrf external-vrf 172.16.128.2 source lo0

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 172.16.128.2, timeout is 2 seconds:

Packet sent with a source address of 41.41.41.41 !!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 132/133/138 ms

root@web-server:/home/marinfer# ping 41.41.41.41

PING 41.41.41.41 (41.41.41.41) 56(84) bytes of data.

64 bytes from 41.41.41.41: icmp_seq=1 ttl=254 time=87.0 ms

64 bytes from 41.41.41.41: icmp_seq=2 ttl=254 time=84.9 ms

64 bytes from 41.41.41.41: icmp_seq=3 ttl=254 time=83.7 ms

64 bytes from 41.41.41.41: icmp_seq=4 ttl=254 time=83.8 ms

root@web-server:/home/marinfer# 

root@app-server:/home/marinfer# ping 41.41.41.41

PING 41.41.41.41 (41.41.41.41) 56(84) bytes of data.

64 bytes from 41.41.41.41: icmp_seq=1 ttl=254 time=134 ms

64 bytes from 41.41.41.41: icmp_seq=2 ttl=254 time=132 ms

64 bytes from 41.41.41.41: icmp_seq=3 ttl=254 time=131 ms

64 bytes from 41.41.41.41: icmp_seq=4 ttl=254 time=136 ms

root@app-server:/home/marinfer#

Advanced Routing Capabilities with Cisco Cloud Router

Leveraging native routing capabilities as demonstrated may suffice for some specific use cases and be limited for others. Therefore, for more advanced routing capabilities, Cisco Cloud Routers can be deployed instead. The provisioning process is relatively the same with CCRs also instantiated within the infra VPC in a hub and spoke architecture. Besides having the ability to manage the complete lifecycle of the CCRs from the Cisco CNC, customers can also choose different tier-based throughput options based on requirements.

One of the main use cases for leveraging Cisco Cloud Routers is the BGP EVPN support across different cloud sites running Cisco CNC, or for hybrid cloud connectivity with on-prem sites when policy extension is desirable. The different inter-site uses cases are being documented on specific white papers, and below is a high-level topology illustrating the architecture.

Source: cisco.com

Continue reading

Introducing Cisco Cloud Network Controller on Google Cloud Platform – Part 2

Part 1 of this blog series demonstrated how Cisco CNC can automate cloud networking within GCP independently of security policies. Part 2 goes over additional capabilities pertaining to contract-based routing and firewall rules automation by extending the same policy model.

One of the reasons for decoupling routing and security is to give customers more flexibility. Often, organizations may have different teams responsible for cloud networking and security policies definitions in the cloud. However, for those use cases where policy consistency is a top priority followed by more governance of cloud resources, a common policy model is a must.

Policy Model Translation

Below is a high-level one-to-one mapping of the Cisco CNC policy model to native GCP cloud constructs.

Essentially, a tenant maps to a project and is the top-level logical container holding all the other policies. For cloud networking, Cisco CNC translates the combination of VRF and Cloud Context Profile into global VPC networks and regional subnets. In the scenario below, Cisco CNC will also translate security policies by combining cloud EPGs (Endpoint Groups) with contracts and filters into firewall rules and network tags in GCP.

By definition, a cloud EPG is a collection of endpoints sharing the same security policy, can have endpoints in one or more subnets and is tied to a VRF.

Scenario

This scenario has two VRFs: network-a and network-b. Additionally, cloud EPGs Web & App will be created and associated to contracts with specific security policies defined by filters. A Cloud External EPG will also be created as Internet EPG to allow internet access on network-a.

On GCP, these policies are translated into proper VPC networks, subnets, routing tables, peering, firewall rules, and network tags. Note that for this scenario, VPCs and subnets were already pre-provisioned.

Contract-based Routing

On Part 1 of this blog series, a route leak policy was created to allow inter-VRF routing between network-a and network-b. For this scenario, only contract-based routing will be enabled, which means contracts will drive routing where needed. Therefore, the leak route policy created previously was removed and peering between VPCs disconnected.

Contract-based Routing is a global mode configuration available in the Cloud Network Controller Setup. Note that when contract-based routing is enabled, the routes between a pair of internal VRFs can be leaked using contracts only in the absence of a route leak policy.

Note: a brief overview of the Cisco CNC GUI was provided on Part 1.

Firewall Rules Automation

The configuration below illustrates the creation of Web and Internet EPGs tied to network-a, along with their associated endpoint selectors. Those are used to assign endpoints to a Cloud EPG, and can be based on IP address, Subnet, Region, or Custom tags (using a combination of key value pairs and match expressions).

For the Web EPG, a key value pair is used with specific tags to be matched (custom: epg equals web). For the Internet EPG, a subnet selector is used allowing all traffic. Furthermore, Internet EPG needs to be type External as internet access will be allowed on network-a.

Web EPG

Internet EPG

The Cloud EPG App configuration is not depicted for brevity but is similar to that of cloud EPG Web. However, it is tied to network-b and set with its unique endpoint selector (custom: epg equals app).

On GCP, these policies get translated to dedicated ingress firewall rules and network tags for Web and App as highlighted using the following format: capic-<app-profile-name>-<epg-name>.

Note: Rebranding from Cloud APIC to Cloud Network Controller is covered on Part 1.

In the example below, cloud endpoints instantiated in GCP with labels matching the endpoint selectors are assigned to network tags and firewall rules automated by Cisco CNC.

Associating Contracts to EPGs

Now, let’s associate the web-to-app contract between Web and App EPGs using the concept of consumer and provider to define rules direction.

Upon associating the contract, additional ingress and egress firewall rules are programmed depending on the consumer and provider relationship specified. Specifically, these firewall rules are updated based on security policies defined through contracts and filters. For brevity, all traffic is allowed but granular filters can be added per requirements. On another note, these rules are only programmed once cloud endpoints matching the rules are instantiated.

Wait, what about peering between these VPCs? Since contract-based routing is enabled, it also drives routing by enabling peering and auto generating routes to each other accordingly.

Lastly, let’s allow internet access to web services residing on network-a by adding the internet-access contract between Internet and Web EPGs.

As soon as the contract is associated, Cisco CNC adds an ingress firewall rule with network tags representing the Web EPG which allows internet access to endpoints behind it.

From this point on, internet access to web-server is allowed as well as connectivity from the web-server to the app-server.

root@web-server:/home/marinfer# ifconfig ens4

ens4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1460

        inet 172.16.1.2  netmask 255.255.255.255  broadcast 172.16.1.2

        inet6 fe80::4001:acff:fe10:102  prefixlen 64  scopeid 0x20<link>

        ether 42:01:ac:10:01:02  txqueuelen 1000  (Ethernet)

        RX packets 19988  bytes 3583929 (3.4 MiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 17707  bytes 1721956 (1.6 MiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

root@web-server:/home/marinfer# ping 172.16.128.2

PING 172.16.128.2 (172.16.128.2) 56(84) bytes of data.

64 bytes from 172.16.128.2: icmp_seq=1 ttl=64 time=58.3 ms

64 bytes from 172.16.128.2: icmp_seq=2 ttl=64 time=56.0 ms

64 bytes from 172.16.128.2: icmp_seq=3 ttl=64 time=56.0 ms

64 bytes from 172.16.128.2: icmp_seq=4 ttl=64 time=56.0 ms

Cloud Resources Visibility

Using a cloud-like policy model, Cisco CNC provides a topology and hierarchical view of cloud resources on a per tenant basis with drill down options. Moreover, application profile containers group together cloud EPGs and associated contracts for easy visibility of policies and dependencies.

More granular visibility is provided all the way to cloud endpoints. Firewall rules are also visible via Cisco CNC GUI under Ingress and Egress Rules.

Source: cisco.com

Continue reading