Cisco User Defined Network: Redefining Secure, Personal Networks

Connecting all your devices to a shared network environment such as dorm rooms, classrooms, multi-dwelling building units, etc. may not be desirable as there are too many users/devices on the shared network and onboarding of devices is not secure. In addition, there is limited user control; that is, there is no easy way for the users to deterministically discover and limit access to only the devices that belong to them. You can see all users’ devices and every user can see your device. This, not only results in poor user experience but also brings in security concerns where users knowingly or unknowingly can take control of devices that may belong to other users. 

Cisco User Defined Network (UDN) changes the shared network experience by enabling simple, secure and remote on-boarding of wireless endpoints on to the shared network to give a personal network-like experience. Cisco UDN provides control to the end-users to create their own personal network consisting of only their devices and also enables the end-users the ability to invite other trusted users into their personal network. This provides security to the end-users at the same time giving them ability to collaborate and share their devices with other trusted users. 

Solution Building Blocks

The following are the functional components required for Cisco UDN Solution. This is supported in the Catalyst 9800 controllers in centrally switched mode.

Figure 1. Solution Building Blocks

Cisco UDN Mobile App: The mobile app is used for registering user’s devices onto the network from anywhere (on-prem or off-prem) and anytime. End-user can log in to the mobile app using the credentials provided by the organization’s network administrator. Device on-boarding can be done in multiple ways. These include: 

◉ Scanning the devices connected to the network and selecting devices required to be onboarded

◉ Manually entering the MAC address of the device

◉ Using a camera to capture the MAC address of the device or using a picture of the mac address to be added

In addition, using mobile app, users can also invite other trusted users to be part of their private network segment. The mobile app is available for download both on Apple store and Google play store.

Cisco UDN Cloud Service: Cloud service is responsible for ensuring the registered devices are authenticated with Active Directory through SAML 2.0 based SSO gateway or Azure AD. Cloud service is also responsible for assigning the end-users and their registered devices to a private network and provides rich insights about UDN service with the cloud dashboard.

Cisco DNA Center: Is an on-prem appliance which connects with Cisco UDN cloud service. It is the single point through which the on-prem network can be provisioned (automation) and provides visibility through telemetry and assurance data. 

Identity Services Engine (ISE): Provides authentication and authorization services for the end-users to connect to the network.

Catalyst 9800 Wireless Controller and Access Points: Network elements which enables traffic containment within the personal network. UDN is supported on wave2 and Cisco Catalyst access points.

How does it work?

Cisco UDN solution focuses on simplicity and secure onboarding of devices. The solution gives flexibility to the end-users to invite other trusted users to be part of their personal network. The shared network can be segmented into smaller networks as defined by the users. Users from one segment will not be able to see traffic from another user segment. The solution ensures that broadcast, link-local multicast and discovery services (such as mDNS, UPnP) traffic from other user segments will not be seen within a private network segment. Optionally, unicast traffic from other segments can also be blocked. However, unicast traffic within a personal network and north-south traffic will be allowed. 

Workflows

There are three main workflows associated with UDN:

1. Endpoint registration workflow: User’s endpoint can register with the UDN cloud service through a mobile-app from anywhere at any time (on-prem or off-prem). Upon registration, the cloud service ensures that the endpoint is authenticated with the active directory. Cloud service then assigns a private segment/network to the authenticated users and assigns a unique identity – User Defined Network ID (UDN-ID). This unique identity (UDN-ID) along with the user and endpoint information (mac address) is pushed from cloud service to on-prem through DNAC. The unique private network identity along with the user/endpoint information is stored in ISE 

2. Endpoint on-boarding workflow: When the endpoint joins the wireless network using one of the UDN enabled WLANs, as part of the authorization policy, ISE will push the private network ID associated with the endpoint to the wireless controller. This mapping of endpoint to UDN-ID is retrieved from ISE. The network elements (wireless LAN controller and access point), will use the UDN-ID to enforce traffic containment for the traffic generated by that endpoint

3. Invitation workflow: A user can invite another trusted user to be part of his personal network. This is initiated from the mobile app of the user who is inviting. The invitation will trigger a notification to the invitee through the cloud service. Invitee has an option to either accept or reject the invitation. Once the invitee has accepted the request, cloud service will put the invitee in the same personal network as the inviter and notify the on-prem network (DNAC/ISE) about the change of the personal room for the invitee. ISE will then trigger a change of authorization to the invitee and notify the wireless controller of this change. The network elements will take appropriate actions to ensure that the invitee belongs to the inviter’s personal room and enforces traffic containment accordingly

The following diagram highlights the various steps involved in each of the three workflows.

Figure 2. UDN Workflows

Traffic Containment

Traffic containment is enforced in the network elements, wireless controller and access points. UDN-ID, an identifier for a personal network segment, is received by WLC from ISE as part of access-accept RADIUS message during either client on-boarding or change-of-authorization. Unicast traffic containment is not enabled by default. When enabled on a WLAN, unicast traffic between two different personal networks is blocked. Unicast traffic only within a personal network and north-south traffic will be allowed. Wireless controller enforces unicast traffic containment. The traffic containment logic in the AP ensures that the link-local multicast and broadcast traffic is sent as unicast traffic over the air to only the clients belonging to a specific personal network. The table below summarizes the details of traffic containment enforced on the network elements.

Figure 3. UDN Traffic Containment

The WLAN on which UDN can be enabled should have either MAC-filtering enabled or should be an 802.1x WLAN. The following are the possible authentication combinations on which UDN can be supported on the wireless controller:

For RLAN, only mDNS and unicast traffic can be contained through UDN. To support LLM and/or broadcast traffic, all clients on RLAN needs to be in the same UDN.

Monitor and Control

The end-to-end visibility into the UDN solution is enabled through both DNA cloud service dashboard and DNAC assurance. In addition, DNAC also enables configuring the UDN service through a single pane of glass. 

DNA Cloud Service provides rich insights with the cloud dashboard. It gives visibility into the devices registered, connected within a UDN and also information about the invitations sent to other trusted users etc. 

Figure 4. Insights and Cloud Dashboard

On-Prem DNAC enables enablement of UDN through automation workflow and provides complete visibility of UDN through Client 360 view in assurance.

Figure 5. UDN Client Visibility

Cisco UDN enriches the user experience in a shared network environment. Users can bring any device they want to the Enterprise network and benefit from home-like user experience while connected to the Enterprise network. It is simple, easy to use and provides security and control for the user’s personal network.

Continue reading

Introduction to Terraform and ACI – Part 5

If you haven’t already seen the Introduction to Terraform posts, please have a read through. Here are links to the first four posts:

1. Introduction to Terraform

2. Terraform and ACI​​​​​​​

3. Explanation of the Terraform configuration files

4. Terraform Remote State and Team Collaboration

So far we’ve seen how Terraform works, ACI integration, and remote state. Although not absolutely necessary, sometimes it’s useful to understand how the providers work in case you need to troubleshoot an issue.​​​​​​​ This section will cover at a high level how Terraform Providers are built, using the ACI provider as an example. 

Code Example

https://github.com/conmurphy/intro-to-terraform-and-aci-remote-backend.git

For explanation of the Terraform files see Part 3 of the series. The backend.tf file will be added in the current post.

Lab Infrastructure

You may already have your own ACI lab to follow along with however if you don’t you might want to use the ACI Simulator in the Devnet Sandbox.

ACI Simulator AlwaysOn – V4

Terraform File Structure

As we previously saw, Terraform is split into two main components, Core and Plugins. All Providers and Provisioners used in Terraform configurations are plugins.  ​​​​

Data sources and resources exist within a provider and are responsible for the lifecycle management of the specific endpoint. For example we will have a look at the resource, “aci_bridge_domain“, which is responsible for creating and managing our bridge domains.

The code for the Terraform ACI Provider can be found on Github

https://github.com/CiscoDevNet/terraform-provider-aci

There are a number of files in the root directory of this repo how the ones we are concerned with are “main.go“, the “vendor” folder, and the “aci” folder. 

◉ main.go: This file creates a valid, executable Go binary that Terraform Core can consume.

◉ vendor: This folder contains all of the Go modules which will be imported and used by the provider plugin. The key module for this provider is the ACI Go SDK which is responsible for making the HTTP requests to APIC and returning the response.

◉ aci: This is the important directory where all the resources for the provider exist.

Within the ACI  folder you’ll find a file called “provider.go“. This is a standard file in the Terraform providers and is responsible for setting the properties such as username, password, and URL of the APIC in this case.

It’s also responsible for defining which resources are available to configure in the Terraform files, and linking them with the function which implements the Create, Read, Update, and Delete (CRUD) capability.

In the aci folder you’ll also find all the data sources and resources available for this provider. Terraform has a specific structure for the filename and should start with data_source_ or resource_

Let’s look at the resource, “resource_aci_fvbd“, used to create bridge domains.

◉ On lines 10 and 11 the ACI Go SDK is imported. 

◉ The main function starts on line 16 and is followed by a standard Terraform configuration

    ◉ Lines 18 – 21 define which operations are available for this resource and which function should be called. We will see these four further down the page.

◉ Lines 29 – 59 in the screenshot are setting the properties which will be available for the resource in the Terraform configuration files.

TROUBLESHOOTING TIP: This is an easy way to check exactly what is supported/configurable if you think the documentation for a provider is incorrect or incomplete. 

We’ve now reached the key functions in the file and these are responsible for implementing the changes. In our case creating, reading, updating, and destroying a bridge domain.

If you scroll up you can confirm that the function names match those configured on lines 18-21 

Whenever you run a command, e.g. “terraform destroy“, Terraform will call one of these functions. 

Let’s have a look at what it’s creating.

First the ACI Go SDK has to be setup on line 419

Following on from that the values from your configuration files are retrieved so Terraform can take the appropriate action. For example in this screenshot the name we’ve configured, “bd_for_subnet“, will be stored in the variable, “name“. 

Likewise for the description, TenantDn, and all other bridge domain properties we’ve configured.

Further down in the file you’ll see the ACI Go SDK is called to create a NewBridgeDomain. This object is then passed to a Save function in the SDK which makes a call to the APIC to create the bridge domain

Continuing down towards the end of the create function you’ll see the ID being set on line 726. Remember when Terraform manages a resource it keeps the state within the terraform.tfstate file. Terraform tracks this resource using the id, and in the case of ACI the id is the DistinguishedName.

It’s not only the id that Terraform tracks though, all the other properties for the resource should also be available in the state file. To save this data there is another function, setBridgeDomainAttributes, which sets the properties in the state file with the values that were returned after creating/updating the resource. 

So when Terraform creates our bridge domain, it saves the response properties into the state file using this function.

TROUBLESHOOTING TIP: If resources are always created/updated when you run a terraform apply even though  you haven’t changed any configuration, you might want to check the state file to ensure that all the properties are being set correctly.

Source: cisco.com

Continue reading

The Best Kept Secret in Mobile Networks: A Million Saved is a Million Earned

I’m about to let you in on a little secret, actually it’s a big secret. What if I told you that you could save millions – even tens of millions – of dollars in two hours or less? If you were the CFO of a Mobile Operator, would this get your attention? What if I told you that the larger and busier your mobile network is, the more money you could save? What if the savings were in the hundreds of millions or even billions of dollars?

Now that I have your attention, let me tell you a bit about the Cisco Ultra Traffic Optimization (CUTO) solution. I could tell you that this is a vendor-agnostic solution for both the RAN and the mobile packet core. I could tell you how CUTO uses machine learning algorithms or about proactive cross-traffic contention detection. I could tell you about elephant flows and how the CUTO software optimizes the packet scheduler in RAN networks. I could write a whole blog on the CUTO technology and how it works, but I won’t. I will leave out the technical details for another time to share.

What I want to share today are two important facts about our CUTO solution:

1) We have helped multiple operators install and turn on CUTO, networkwide in less than 2 hours.

2) Real world deployments are demonstrating material savings, including a recent trial with a large Tier 1 operator that resulted in a calculated savings of several billions of dollars.

Out of all the segments of the network that operators are investing in, spectrum and RAN tend to be a top priority, both from a CAPEX and an OPEX standpoint. This is because mobile network operators have thousand’s, if not tens of thousands, of cell towers with accompanying RAN equipment. The amount of equipment required, and the cost of a truck roll per site leads to enormous expenses when you want to upgrade or augment the RAN network. With network traffic growing faster every year, the challenge to stay ahead of demand also grows. Major RAN network augmentations can take months, if not years to complete, especially when you factor in governmental regulations and permitting processes. This is where one aspect of the CUTO solution stands out, its ease and speed of deployment. CUTO’s purpose is to optimize the RAN network and improve the efficiency of the use of the spectrum. Instead of sending an army of service trucks and technicians to each and every cell tower, CUTO is deployed in the Core of the network, which is a very small number of sites (data centers). Making things even easier, CUTO can be deployed on Common Off the Shelf Servers (COTS), or better yet it can be deployed on the existing Network Function Virtual Infrastructure (NFVI) stack already in the mobile network core. Installing and deploying CUTO is as easy as spinning up a few virtual machines. In less than 2 hours, real operators on live networks have managed to install and deploy CUTO networkwide. Service providers talk a lot about MTTD (mean time to detect an issue) and MTTR (mean time to repair it), but with CUTO they can talk about MTTME – mean time to millions earned (saved).

If you look at the present mode of operation (PMO) for most mobile operators, there’s a very typical workflow that occurs in RAN networks. Customer’s consumption of video and an insatiable appetite for bandwidth eventually leads to a capacity trigger in the network, alerting the mobile network operator that they have congestion in a cell site. To handle the alert, operators typically have three options:

1) They might be able to “re-farm” spectrum and transition 3G spectrum to 4G spectrum. If this is an option, it typically leads to about a 40% spectrum gain at the price tag of about 22K in CAPEX per site and with very nominal OPEX costs. This option is relatively quick, simple, and leads to a good capacity improvement.

2) They might be able to deploy a new spectrum band or increase the Antenna Sectorization density. This option leads to about 20%-30% of the capacity gain but comes at a price tag of about 80K in CAPEX per site and about 20K in OPEX. This is a relatively long and costly process for a marginal capacity improvement and finding high quality “beachfront” spectrum is impossible in many markets around the world.

3) If neither option 1 or 2 are possible, the Operator would need to build a new cell site and cell split (tighten reuse). A new site leads to about 80% capacity gain depending on the site’s placement, user distribution, terrain, shadowing, etc., and comes at a cost of roughly 250K in CAPEX and about 65K/year in OPEX. This is an extremely long process (permitting, etc.) and very expensive.

Unfortunately, the spectrum is a finite resource, the opportunity for operators to choose option 1 or 2 is becoming scarce. Just five years ago, about 50% of cell sites with congestion were candidates for option 1, 30% are candidates for option 2, and only 20% required option 3. Five years from now virtually no cell sites will be candidates for option 1, maybe 10% will be candidates for option 2, and over 90% will require the very time-consuming and costly option 3.

CUTO offers mobile operators an alternative, helping to optimize traffic and reduce the congestion in the cell sites, which can significantly reduce the number of sites requiring capacity upgrades. During a recent trial to measure the efficacy of CUTO in the real-world and at scale, we deployed it with a Tier 1 operator. Near immediately, we saw a 15% reduction in the number of cell sites triggering a capacity upgrade due to consistent congestion. Here are some real-world numbers of how we were able to calculate the savings based on that 15% reduction:

The operator in this use case has:

• ~40M subscribers

• 10,000 sites triggering a need for a capacity upgrade

• Annual data traffic growth rate of 25%

The assumptions we agreed to with the operator were:

• Blended Incremental CAPEX/Upgrade (options 1, 2, and 3) = $100K CAPEX

• Blended Incremental OPEX/Site/Year (options 1, 2, and 3) = $20K OPEX/year

In this real-world example, you can see that CUTO saves this operator over $1.8B of CAPEX and $837M of OPEX over 5 years. I like to consider that as “adult money.” I recognize that these numbers are enormous and because of that, you may be skeptical. I was skeptical until I saw the results of the real-world trial for myself. I expect there to be plenty of FUD coming from the folks that are at risk of missing out on significant revenue because of your deployment of CUTO. Here’s my answer to those objections:

• Even if you cut these numbers in half or more, my guess is that you are looking at a material impact on your P&L.

• See it firsthand, don’t just take my word for it, ask your local Cisco Account Manager for a trial of CUTO and look at the savings based on your network, your mix of options 1, 2, or 3, and your costing models.

We all know that 5G will be driving new use cases, the need for more bandwidth, and we know that video will only become more ubiquitous. Mobile operators will require more and more cell sites and those sites will continue to fill up in capacity over time. Why not get ahead of the problem, and see what sort of MTTME your organization is capable of when your organization embraces a highly innovative software strategy?

Continue reading

Introduction to Terraform with ACI – Part 4

If you haven’t already seen the Introduction to Terraform post, please have a read through. This section will cover the Terraform Remote Backend using Terraform Cloud.​​​​​​​

1. Introduction to Terraform

2. Terraform and ACI​​​​​​​

3. Explanation of the Terraform configuration files

Code Example

https://github.com/conmurphy/intro-to-terraform-and-aci-remote-backend.git

For explanation of the Terraform files see the following post. The backend.tf file will be added in the current post.

Lab Infrastructure

You may already have your own ACI lab to follow along with however if you don’t you might want to use the ACI Simulator in the Devnet Sandbox.

ACI Simulator AlwaysOn – V4

Terraform Backends

An important part of using Terraform is understanding where and how state is managed. In the first section Terraform was installed on my laptop when running the init, plan, and apply commands. A state file (terraform.tfstate) was also created in the folder in which I ran the commands. ​​​​​​​

It’s fine when learning and testing concepts however does not typically work well in shared/production environment. What happens if my colleagues also want to run these commands? Do they have their own separate state file?​​​​​​​

These questions can be answered with the concept of the Terraform Backend.

“A backend in Terraform determines how state is loaded and how an operation such as apply is executed. This abstraction enables non-local file state storage, remote execution, etc.

Also Read: 300-420: Designing Cisco Enterprise Networks (ENSLD)

Here are some of the benefits of backends:

◉ Working in a team: Backends can store their state remotely and protect that state with locks to prevent corruption. Some backends such as Terraform Cloud even automatically store a history of all state revisions.

◉ Keeping sensitive information off disk: State is retrieved from backends on demand and only stored in memory. If you’re using a backend such as Amazon S3, the only location the state ever is persisted is in S3.

◉ Remote operations: For larger infrastructures or certain changes, terraform apply can take a long, long time. Some backends support remote operations which enable the operation to execute remotely. You can then turn off your computer and your operation will still complete. Paired with remote state storage and locking above, this also helps in team environments.”

https://www.terraform.io/docs/backends/index.html

As you can see from the Terraform documentation, there are many backend options to choose from.

In this post we’ll setup the Terraform Cloud Remote backend.

https://www.terraform.io/docs/backends/types/remote.html

We will use the same Terraform configuration files as we saw in the previous posts, with the addition of the “backend.tf “ file. See the code examples above for a post explaining the various files.

For this example you will need to create a free account on the Terraform Cloud platform

◉ Create a new organization and provide it a name

◉ Create a new CLI Driven workspace

◉ Once created, navigate to the “General” page under “Settings”

◉ Change the “Execution Mode” to “Local”

You have two options with Terraform Cloud

◉ Remote Execution – Let Terraform Cloud maintain the state and run the plan and apply commands

◉ Local Execution – Let Terraform Cloud main the state but you run the plan and apply  commands on your local machine

In order to have Terraform Cloud run the commands you will either need public access to the endpoints or run an agent in your environment (similar to Intersight Assist configuring on premises devices)

Agents are available as part of the Terraform Cloud business plan. For the purposes of this post Terraform Cloud will manage the state while we will run the commands locally.

◉ Navigate back to the production workspace and you should see that the queue and variables tabs have been removed.

◉ Copy the example Terraform code and update the backend.tf file (the Terraform files can be found in the Github repo above)

◉ Navigate to the Settings link at the top of the page and then API Tokens

◉ Create an authentication token

◉ Copy the token

◉ On your local machine create a file (if it doesn’t already exist) in the home directory with the name .terraformrc

◉ Add the credentials/token information that was just create for your organization. Here is an example

CONMURPH:~$ cat ~/.terraformrc

credentials "app.terraform.io" {

  token="<ENTER THE TOKEN HERE> "

}

◉ You should now have the example Terraform files from the Github repo above, an updated backend.tf file with your organization/workspace, and a .terraformrc file with the token to access this organization

◉ Navigate to the folder containing the example Terraform files and your backend.tf file

◉ Run the terraform init command. If everything is correct you should see the remote backend initialised and the ACI plugin installed

◉ Run the terraform plan and terraform apply commands to apply to configuration changes.

◉ Once complete, if the apply is successful have a look at your Terraform Cloud organization.

◉ In the States tab you should now see the first version of your state file. When you look through this file you’ll see it’s exactly the same as the one you previously had on your local machine, however now it’s under the control of Terraform Cloud​​​​​​​

◉ Finally, if you want to collaborate with your colleagues, you can all run the commands locally and have Terraform Cloud manage a single state file. (May need to investigate locking depending on how you are managing the environment)

Source: cisco.com

Continue reading

Introduction to Terraform with ACI – Part 3

If you haven’t already seen the previous Introduction to Terraform posts, please have a read through. This “Part 3” will provide an explanation of the various configuration files you’ll see in the Terraform demo.

Introduction to Terraform

Terraform and ACI

Code Example

https://github.com/conmurphy/terraform-aci-testing/tree/master/terraform

Configuration Files

You could split out the Terraform backend config from the ACI config however in this demo it has been consolidated. 

config-app.tf

The name “myWebsite” in this example refers to the Terraform instance name of the “aci_application_profile” resource. 

The Application Profile name that will be configured in ACI is “my_website“. 

When referencing one Terraform resource from another, use the Terraform instance name (i.e. “myWebsite“).

config.tf

Only the key (name of the Terraform state file) has been statically configured in the S3 backend configuration. The bucket, region, access key, and secret key would be passed through the command line arguments when running the “terraform init” command. See the following for more detail on the various options to set these arguments.

https://www.terraform.io/docs/backends/config.html

terraform.tfvars

variables.tf

We need to define the variables that Terraform will use in the configuration. Here are the options to provide values for these variables:

◉ Provide a default value in the variable definition below

◉ Configure the “terraform.tfvars” file with default values as previously shown

◉ Provide the variable values as part of the command line input

$terraform apply –var ’tenant_name=tenant-01’

◉ Use environmental variables starting with “TF_VAR“

$export TF_VAR_tenant_name=tenant-01

◉ Provide no default value in which case Terraform will prompt for an input when a plan or apply command runs

versions.tf

Source: cisco.com

Continue reading

Win with Cisco ACI and F5 BIG-IP – Deployment Best Practices

Applications environments have different and unique needs on how traffic is to be handled. Some applications, due to the nature of their functionality or maybe due to a business need do require that the application server(s) are able to view the real IP address of the client making the request to the application.

Now, when the request comes to the F5 BIG-IP, it has the option to change the real IP address of the request or to keep it intact. To keep it intact, the setting on the F5 BIG-IP ‘Source Address Translation’ is set to ‘None’.

As simple as it may sound to just toggle a setting on the F5 BIG-IP, a change of this setting causes significant change in traffic flow behavior.

Let us take an example with some actual values. Starting with a simple setup of a standalone F5 BIG-IP with one interface on the F5 BIG-IP for all traffic (one-arm)

◉ Client – 10.168.56.30

◉ BIG-IP Virtual IP – 10.168.57.11

◉ BIG-IP Self IP – 10.168.57.10

◉ Server – 192.168.56.30

Scenario 1: With SNAT

From Client: Src: 10.168.56.30           Dest: 10.168.57.11

From BIG-IP to Server: Src: 10.168.57.10 (Self-IP)     Dest: 192.168.56.30

In above scenario, the server will respond back to 10.168.57.10 and F5 BIG-IP will take care of forwarding the traffic back to the client. Here, the application server has visibility to the Self-IP 10.168.57.10 and not the client IP. 

Scenario 2: No SNAT

From Client: Src: 10.168.56.30           Dest: 10.168.57.11

From BIG-IP to Server: Src: 10.168.56.30       Dest: 192.168.56.30

In this scenario, the server will respond back to 10.168.56.30 and here is where comes in the complication, as the return traffic needs to go back to the F5and not the real client. One way to achieve this is to set the default GW of the server to the Self-IP of the BIG-IP and then the server will send the return traffic to the BIG-IP. BUT what if the server default gateway is not to be changed for whatsoever reason.  Policy based redirect will help here. The default gateway of the server will point to the ACI fabric, and the ACI fabric will be able to intercept the traffic and send it over to the BIG-IP.

With this, the advantage of using PBR is two-fold

◉ The server(s) default gateway does not need to point to F5 BIG-IP, but can point to the ACI fabric

◉ The real client IP is preserved for the entire traffic flow

◉ Avoid server originated traffic to hit BIG-IP, resulting BIG-IP to configure a forwarding virtual to handle that traffic. If server originated traffic volume is high, it could result unnecessary load the F5 BIG-IP

Before we get deeper into the topic of PBR below are a few links to help you refresh on some of the Cisco ACI and F5 BIG-IP concepts

◉ Cisco ACI fundamentals

◉ SNAT and Automap

◉ F5 BIG-IP modes of deployment

Now let us look at what it takes to configure PBR using a Standalone F5 BIG-IP Virtual Edition in One-Arm mode.

To use the PBR feature on APIC – Service graph is a MUST

Details on L4-L7 service graph on APIC

To get hands on experience on deploying a service graph (without pbr)

Configuration on APIC

1) Bridge domain ‘F5-BD’

◉ Under Tenant->Networking->Bridge domains->’F5-BD’->Policy

◉ IP Data plane learning – Disabled

2) L4-L7 Policy-Based Redirect

◉ Under Tenant->Policies->Protocol->L4-L7 Policy based redirect, create a new one

◉ Name: ‘bigip-pbr-policy’

◉ L3 destinations: F5 BIG-IP Self-IP and MAC

◉ IP: 10.168.57.10

◉ MAC: Find the MAC of interface the above Self-IP is assigned from logging into the F5 BIG-IP (example: 00:50:56:AC:D2:81)

3) Logical Device Cluster- Under Tenant->Services->L4-L7, create a logical device

◉ Managed – unchecked

◉ Name: ‘pbr-demo-bigip-ve`

◉ Service Type: ADC

◉ Device Type: Virtual (in this example)

◉ VMM domain (choose the appropriate VMM domain)

◉ Devices: Add the F5 BIG-IP VM from the dropdown and assign it an interface

◉ Name: ‘1_1’, VNIC: ‘Network Adaptor 2’

◉ Cluster interfaces

◉ Name: consumer, Concrete interface Device1/[1_1]

◉ Name: provider, Concrete interface: Device1/[1_1]

4) Service graph template

◉ Under Tenant->Services->L4-L7->Service graph templates, create a service graph template

◉ Give the graph a name:’ pbr-demo-sgt’ and then drag and drop the logical device cluster (pbr-demo-bigip-ve) to create the service graph

◉ ADC: one-arm

◉ Route redirect: true

5) Click on the service graph created and then go to the Policy tab, make sure the Connections for the connectors C1 and C2 and set as follows:

◉ Direct connect – True

◉ Adjacency type – L3

6) Apply the service graph template

◉ Right click on the service graph and apply the service graph

◉ Choose the appropriate consumer End point group (‘App’) provider End point group (‘Web’) and provide a name for the new contract

◉ For the connector

◉ BD: ‘F5-BD’

◉ L3 destination – checked

◉ Redirect policy – ‘bigip-pbr-policy’

◉ Cluster interface – ‘provider’

Once the service graph is deployed, it is in applied state and the network path between the consumer, F5 BIG-IP and provider has been successfully setup on the APIC

Configuration on BIG-IP

1) VLAN/Self-IP/Default route

◉ Default route – 10.168.57.1

◉ Self-IP – 10.168.57.10

◉ VLAN – 4094 (untagged) – for a VE the tagging is taken care by vCenter

2) Nodes/Pool/VIP

◉ VIP – 10.168.57.11

◉ Source address translation on VIP: None

3) iRule (end of the article) that can be helpful for debugging

Few differences in configuration when the BIG-IP is a Virtual edition and is setup in a High availability pair

1) BIG-IP: Set MAC Masquerade

2) APIC: Logical device cluster

◉ Promiscuous mode – enabled

◉ Add both BIG-IP devices as part of the cluster

3) APIC: L4-L7 Policy-Based Redirect

◉ L3 destinations: Enter the Floating BIG-IP Self-IP and MAC masquerade

Configuration is complete, let’s look at the traffic flows.

Client-> F5 BIG-IP -> Server

Server-> F5 BIG-IP -> Client

In Step 2 when the traffic is returned from the client, ACI uses the Self-IP and MAC that was defined in the L4-L7 redirect policy to send traffic to the BIG-IP.

iRule to help with debugging on the BIG-IP

Output seen in /var/log/ltm on the BIG-IP, look at the event <SERVER_CONNECTED>

Scenario 1: No SNAT -> Client IP is preserved

If you are curious of the iRule output if SNAT is enabled on the BIG-IP – Enable AutoMap on the virtual server on the BIG-IP

Scenario 2: With SNAT -> Client IP not preserved.

Continue reading