Cisco and Google Cloud Partner to Bridge Applications and Networks: Announcing Cisco SD-WAN Cloud Hub with Google Cloud

Today, Cisco and Google Cloud are announcing their intent to develop the industry’s first application-centric multicloud networking fabric. This automated solution will ensure that applications and enterprise networks will be able to share service-level agreement settings, security policy, and compliance data, to provide predictable application performance and consistent user experience.

Our partnership will support the many businesses that are embracing a hybrid and multicloud strategy to get the benefits of agility, scalability and flexibility. The platform will enable businesses to optimize application stacks by distributing application components to their best locations. For example, an application suite could support a front end running on one public cloud to optimize for cost, an analytics library on another cloud to leverage its AI/ML capabilities, and a financial component running on-prem for optimal security and compliance.

The connective fabric for these modern enterprise apps (and their distributed users) is the network. The network fabric needs to be able to discover the apps, identify their attributes, and adapt for optimal user experience. Likewise, applications need to respond to changing needs and sometimes enterprise-sized shifts in load, while maintaining availability, security and compliance, through the correlation of application, user and network insights.

In our newly-expanded partnership, Cisco and Google Cloud will build a close integration between Cisco SD-WAN solutions and Google Cloud. Network policies, such as segmentation, will follow network traffic across the boundary between the enterprise network and Google Cloud, for end-to-end control of security, performance, and quality of experience.

“Expansion of the Google-Cisco partnership represents a significant step forward for enterprises operating across hybrid and multi-cloud environments,” says Shailesh Shukla, Vice President of Products and General Manager, Networking at Google Cloud. “By linking Cisco’s SD-WAN with Google Cloud’s global network and Anthos, we can jointly provide customers a unique solution that automates, secures and optimizes the end to end network based on the application demands, simplifying hybrid deployments for enterprise organizations.”

With Cisco SD-WAN Cloud Hub with Google Cloud, for the first time, the industry will have full WAN application integration with cloud workloads: The network will talk to apps, and apps will talk to the network. Critical security, policy, and performance information will be able to cross the boundaries of network, cloud, and application. This integration will extend into hybrid and multicloud environments, like Anthos, Google Cloud’s open application platform, supporting the optimization of distributed, multicloud microservice-based applications.

Towards Stronger Multicloud Apps

Today, applications do not have a way to dynamically signal SLA requests to the underlying network.  With this new integration, applications will be able to dynamically request the required network resources, by publishing application data in Google Cloud Service Directory. The network will be able to use this data to provision itself for the appropriate SD-WAN policies.

For example, a business-critical application that needs low latency would have that requirement in its Google Cloud Service Directory entry. The appropriate SD-WAN policies would then be applied on the network. Likewise, as Cisco’s SD-WAN controller, vManage, monitors network performance and service health metrics, it could intelligently direct user requests to the most optimal cloud service nodes.

Cisco SD-WAN can also proactively provide network insights to a distributed system, like Google Anthos, to ensure availability of applications. For example, based on network reachability metrics from Cisco SD-WAN, Anthos can make real-time decisions to divert traffic to regions with better network reachability.

With Cisco SD-WAN Cloud Hub for Google cloud, customers can extend the single point of orchestration and management for their SD-WAN network to include the underlay offered by Google Cloud backbone. Together with Cisco SD-WAN, Google Cloud’s reliable global network provides enterprise customers with operational efficiency and the agility to scale up for bandwidth.

This integration will promote better security and compliance for enterprises. Using Cisco Cloud Hub, policies can be extended into the cloud to authorize access to cloud applications based on user identity. With Cisco SD-WAN Cloud Hub, any device, including IoT devices, will be able to comply with enterprise policy across the network and app ecosystem.

The partnership to create the Cisco SD-WAN Cloud Hub with Google Cloud will lead to applications with greater availability, improved user experience, and more robust policy management.

A Partnership Solution for Tomorrow’s Networks

For enterprise customers who deploy applications in Google Cloud and multicloud environments, Cisco SD-WAN Cloud Hub with Google Cloud offers a faster, smarter, and more secure way to connect with them and consume them. The Cloud Hub will increase the availability of enterprise applications by intelligently discovering and implementing their SLAs through dynamically orchestrated SD-WAN policy. The platform will decrease risk and increase compliance, offering end-to-end policy-based access, and enhanced segmentation from users to applications in the hybrid multicloud environment.

Cisco and Google Cloud intend to invite select customers to participate in previews of this solution by the end of 2020. General availability is planned for the first half of 2021.

Continue reading

Keeping applications safe and secure in a time of remote work

Businesses around the world have quickly moved to a remote worker initiative, with more users accessing critical workloads outside the traditional workplace than ever before. New attack vectors are inadvertently being introduced to businesses as users are accessing their workloads outside the traditional 4-walls of the workplace and the security protection those 4 walls provide.

To combat the uncertainty and risks introduced by mobilizing a greater than normal remote workforce, it is critical that IT maintains visibility into the network and application behavior from the source of both the users’ remote access machine as well as the critical workloads they’re accessing in the data center, cloud or both (Figure 1). Additionally, it is critical for cybersecurity operators to be able to move to a whitelist/zero-trust segmentation model for network traffic they deem critical for the business to function and do so in a way that can be implemented in a matter of minutes.

Figure 1 – Example of hybrid deployment with remote access to critical workloads

Cisco Tetration and Cisco AnyConnect are paired together now to provide comprehensive security for workload protection needs that customers are having during these volatile times.  These technologies allow IT operators to mitigate many risks to their critical workloads introduced by having an increased attack surface at the “access” layer of their network and also enforce policies to secure the edge and the workloads.  Let’s take a look at the two most relevant use-cases:

Use Case 1 – Gain visibility to network and application behavior of the remote workforce and the workloads they’re accessing. Figure 2 shows exactly how AnyConnect and Tetration work together by sharing telemetry to provide granular level visibility:

Figure 2 – Tetration and AnyConnect Integration

Use-Case 2 –  Easily implement whitelist network policies that protect access to workloads themselves. Figure 3 demonstrates Tetration enforcing enterprise wide policies that affect the organization as a whole. Figure 4 shows Tetration enforcing policies based on application and workload behavior remain compliant. Having these policies across workloads running anywhere (on-prem, cloud or both) adds the needed protection that stretches beyond perimeter security. With workloads being remotely accessed, micro-segmentation prevents later movement of threats reducing surface attacks.

Figure 3 – Enterprise wide policies on Tetration

Figure 4 – Policies on workload based on workload behavior

Now let us dive into the critical elements that help you maintain full visibility and monitor your security policies as your environment evolves. Note all images below are demonstration derived from running Tetration.

1. Visibility is key—quickly see what applications are being accessed by remote users (Fig.5).

Figure 5 – Applications are access by remote users

2. Gain control— with deeper insights you have more power to take better IT decisions. Get an understanding of your workload data flow without the added overhead of manual interrogation (Fig.6). With the help of Tetration agent running on each workload, you also have the log on the processes that have been accessed (Fig7).

Figure 6 – Details flow data

Figure 7 – Get the accessed process using Tetration Agent

3. Search optimization—get granular search results using user details. Historically, this has been a challenge, but with this capability, it will save you time of deeper intervention (Fig.8). Go further by filtering allowed communication policies amongst workloads by searching AD groups (Fig. 9)

Figure 8 – Filter based on AD user

Figure 9 – Filter allowed communications based on AD Group

Cisco Tetration and AnyConnect can help you ramp up your remote access goals securely.

Continue reading

Imagining a world with ubiquitous Remote Expertise – well at least in Banking

We recently returned from Cisco Live where this year’s theme was IMAGINE. Innovation was everywhere, there was even an IMAGINE Lab. That theme ignited some memories for me about when Cisco initially came out with a solutions called Remote Expert. At that time, several years ago, it was the cutting edge of what we could do with video within the business world. I was taken by the impact this technology could have, and began to IMAGINE how someday this concept and technology could be used globally by society to help individuals or groups answer critical questions and solve big problems. That video would become pervasive as it has, but also there would be a way to curate expertise, where you could do a live video chat to research or find answers from those who have that specific knowledge. Well, we are on our way with Siri and Alexa, but not there yet. Someday!

The Remote Expert solution at Cisco has continued to evolve and has been a very effective approach for Retail Banking. In a bank the Remote Expert provides a fully immersive, virtual face-to-face interaction that replicates the “physical expert in the room” experience by connecting a pool of centralized experts based in a call center or another office with the customer sitting in a branch or brokerage. Why is this a need in banking? The bank branch is a critical aspect of the Omni channel and needs to provide the entire set of products and services offered. However, it’s definitely not cost effective to have an expert in all the services from personal and business accounts through mortgage and investments.

Having Remote Expert capability allows the bank to match the erratic customer demand for the expertise they need across an entire physical branch network without having an expert sitting within each branch. It allows the bank to increase availability of experts at peak times to augment what expertise currently sits in the branch. In addition, the bank can then offer their entire expansive product line in all branches at all times at a price the customer can afford. Having expertise physically located in each branch is totally cost prohibitive. The price of services without Remote Expert would be so high, no one could afford them.

In addition, as financial institution are burdened with more and more regulation, the risk of selling complex advisory products increases. The banks are looking for ways to meet regulatory requirements while reducing risk and lowering costs. Remote Expert will do both. An added benefit is how Remote Expert lowers the cost of training and supervising a dispersed physical sales force across a large geographical area. This technology allows the bank to keep everyone updated and trained using the existing video capability. Just another cost saving benefit.

Cisco, along with our partner Moderro are delivering Remote Expert solutions to bank branches, as well as in the form of a standalone unit. The autonomous unit allow the bank to go to the customer, extending reach to wherever there’s market need.

The Remote Expert solution allows the bank to:

◉ Maintain the customer intimacy and satisfaction while introducing new product lines to existing branches without deploying a physical sales force

◉ Reducing/replacing a physical sales force with a centralized sales force to optimize expert usage

◉ Deliver smaller footprint branches, with fewer staff members selling more complex products at a price that is acceptable to the bank

◉ Increase assisted self service capability in banking hall to reduce service staff

◉ Reduce sales distribution risk (the risk of miss-selling)

◉ Reduce costs

◉ Increase revenues

Continue reading

Configuring Cisco ACI and Cisco UCS Deployments with Cisco Action Orchestrator

DevNet Developer Advocates, Unite!

Cisco Action Orchestrator (CAO) has been getting some attention from the DevNet Developer Advocates recently. So, I’m going to jump in and do my part and write about how I used CAO to configure Cisco ACI and Cisco UCS.

It was kind of interesting how this plan came together. We were talking about a Multi-domain automation project/demo and were trying to determine how we should pull all the parts together. We all had different pieces: Stuart had SD-WAN;  Matt had Meraki;  I had UCS and ACI;  Kareem had DNAC; and Hank had the bits and pieces and what-nots (IP addresses, DNS, etc.)

I suggested Ansible, Matt and Kareem were thinking Python, Stuart proposed Postman Runner, and Hank threw CAO into the mix. All valid suggestions. CAO stood out though because we could still use the tools, we were familiar with and have them called from a CAO workflow. Or we could use the built-in capabilities of CAO like the “HTTP Request” activity in the “Web Service” Adapter.

Gentlemen Start Your Workflows

CAO was decided upon and we all got to work on our workflows. The individual workflows are the parts we needed to make “IT” happen. Whatever the “IT” was.

I had two pieces to work on:

1. Provision a UCS Server using a Service Profile
2. Create an ACI tenant with all the necessary elements for policies, networks, and an Application Profile

I knew a couple of things my workflows would need – an IP Address and a Tenant Name. Hank’s workflows would give me the IP Address and Matt’s workflows would give me the Tenant Name.

Information in hand I got to making my part of the “IT” happen. At the same time, the others, knowing what they needed to provide, got to making their part of the “IT” happen.

Cisco ACI – DRY (Don’t Repeat Yourself)

A pretty standard rule in programming is “don’t repeat yourself.” What does it mean? Simply put, if you are going to do something more than once, write a function!

I already wrote several Ansible Playbooks to provision a Cisco ACI Tenant and create an Application Profile, so I decided to use them. The workflow I created in CAO for ACI is just a call to the “Execute Linux/Unix SSH Script” activity in the “Unix/Linux System” Adapter.

For the Workflow I had to define a couple of CAO items.

◉ A target – In this case, the “target” is a Linux host where the Ansible Playbook is run.
◉ An Account Key – The “Account Key” is the credential that is used to connect to the “target”. I used a basic “username/password” credential.

The workflow is simple…

◉ Get the Tenant Name
◉ Connect to the Ansible control node
◉ Run the ansible-playbook command passing the Tenant Name as an argument.

Could more be done here? Absolutely – error checking/handling, logging, additional variables, and so on. Lot’s more could be done, but I would do it as part of my Ansible deployment and let CAO take advantage of it.

Cisco UCS – Put Your SDK in the Cloud

The Cisco UCS API is massive – thousands and thousands of objects and a multitude of methods to query and configure those objects.  Once connected to a UCS Manager all interactions come down to two basic operations, Query and Configure.

I created an AWS Lambda function and made it accessible via an AWS API Gateway. The Lambda function accepts a JSON formatted payload that is divided into two sections, auth, and action.

The auth section is hostname, username, password and is used to connect to the Cisco UCS Manager

The action section is a listing of Cisco UCS Python SDK Modules, Classes, and Object properties. The Lambda function uses Python’s dynamic module importing capability to import modules as needed.

This payload connects to the specified UCS Manager and creates an Organization called Org01

{

        "auth": {

            "hostname": "ucsm.company.com",

            "username": "admin",

            "password": "password"

        },

        "action": {

            "method": "configure",

            "objects": [

            {

                    "module": "ucsmsdk.mometa.org.OrgOrg",

                    "class": "OrgOrg",

                    "properties":{

                        "parent_mo_or_dn": "org-root",

                        "name": "Org01"

                    },

                    "message": "created organization Org01"

                }

            ]

        }

    }

This payload connects to the specified UCS Manager and queries the Class ID computeRackUnit

  {

        "auth": {

            "hostname": "ucsm.company.com",

            "username": "admin",

            "password": "password"

        },

        "action": {

            "method": "query_classid",

            "class_id": "computeRackUnit"

        }

    }

To be honest there is a lot of DRY going on here as well. The Python code for this Lambda function is very similar to the code for the UCS Ansible Collection module, ucs_managed_objects.

In CAO the workflow uses the Tenant Name as input and a Payload is created to send to the UCS SDK in the Cloud. There are two parts to the UCS CAO workflows. One workflow creates the Payload and another workflow processes the Payload by sending it to the AWS API Gateway.  Creating a generic Payload processing workflow enables the processing of any payload that defines any UCS object configuration or query

For this workflow I needed to define a target in CAO

◉ Target – In this case the “target” is the AWS API Gateway that accepts the JSON Payload and sends it on to the AWS Lamba function.

The workflow is simple…

◉ Get the Tenant Name and an IP Address

◉ Create and process the Payload for a UCS Organization named with the Tenant Name

◉ Create and process the Payload for a UCS Service Profile named with the Tenant Name

Pulling “IT” all together

Remember earlier I indicated that we all went off and created our “IT” separately using our familiar tools and incorporating some DRY. When the time came for us to put our pieces together it didn’t matter that I was using Ansible and Python and AWS, my workflows just needed a couple inputs. As long as those inputs were available, we were able to tie my workflows into a larger Multi-domain workflow.

The same was true for Stuart, Matt, Kareem and Hank. As long as we knew what input was needed and what output was needed, we could connect our individual pieces. The pieces of “IT” fit together nicely, and within less than forty minutes on a Webex call we had the main all-encompassing workflow created and running without error.

Continue reading

Cloud vs. Hybrid Collaboration. Which is Right for You?

Understanding Cloud and Hybrid Collaboration Solutions

The world is changing, and the way organizations collaborate and communicate is now different. In-office meetings or traveling to meet with colleagues or customers have been replaced with video conferencing solutions. Organizations are relying on collaboration tools to get their work done, whether they are working from home, in-office or on the go.

Most businesses have a mix of hardware, software and data centers that have been added over time. It is important to look deeper and understand the solutions you currently have available and the changes required to prepare for your new collaboration solution. It will be valuable to define your company’s needs and the capabilities required to best serve your employees and customers and make a decision on the necessary hardware and software required for the transition.

The Right Collaboration Solution with Cisco Webex 

The right collaboration solution accelerates the pace of business and increases productivity and engagement within your workplace and allows you to attract and retain the best talent. Cisco Webex provides a single, unified platform integrating all your collaboration needs to deliver a simple, intelligent, and delightful experience for customers. Our platform enables you to call, meet and message from one single application.

Webex integrates seamlessly into many industry-leading applications, allowing your team to keep their current workflow without missing a beat. Our integrated solution provides complete collaboration without compromising your convenience, security, and privacy. We provide flexible deployments giving you the option to take advantage of our cloud or hybrid collaboration solutions based on your business needs.

Cloud Collaboration Deployments

Cloud Collaboration deployments allow customers to move their resources, data storage, applications, servers and networks to the cloud. Cloud deployments have been proven to be secure, reliable and cost-effective. More and more organizations are moving their infrastructure to the cloud because cloud collaboration solutions provide lower upfront costs, less maintenance, and reduced infrastructure requirement, with the option to scale up and down as needed. They also enhance team collaboration and increase employee productivity and engagement.

With cloud video conferencing, your system can be up and running in minutes, allowing you to connect with your colleagues and customers with high-quality audio and video. Users are able to seamlessly communicate face to face, while administrators have an easier way to manage and provide effective collaboration.  Video conferencing solutions transform your meeting experience and provide you with the freedom to communicate effectively with multiple locations and collaborate with remote stakeholders.

Capabilities You Can Trust 

Cisco Webex provides a modern and easy to use interface to meet, call and message and it seamlessly integrates with your current workflow and applications. Cisco Webex delivers powerful AI and machine learning capabilities across the Webex portfolio. With capabilities such as voice intelligence, real-time transcription, People Insights, Facial Recognition, and Webex Assistant, Cisco Webex helps teams work smarter and better, no matter where they are. Administrators can take advantage of the intuitive, single-pane-of-glass management portal with real-time actionable insights to scale up and down, and provision, administer and manage their Cisco Webex services. Our built-in security and compliance requirements provide you with strong encryption, compliance visibility, and control while collaborating inside and outside your organization.

Hybrid Collaboration Deployments

Moving to the cloud does not have to be an all or nothing solution. Many organizations have made significant on-premises investments and they leverage the hybrid approach, transitioning some systems to the cloud while continuing to run others on their on-premises infrastructure. Hybrid deployments allow you to make the transition to the cloud at the pace suitable for your organization while maximizing your existing investments. According to Nemertes Research, “Hybrid enables organizations to adopt cloud strategically, delivering less disruption and faster access to specific capabilities like meetings and team collaboration from the cloud.”

About the Best Migration Path for Your Business

Cisco Webex allows customers to select the best migration path suited for their business needs and goals. Our flexible deployments enable customers to connect to the Webex Cloud and take advantage of the latest cloud innovations while protecting their existing investment and infrastructure. Cisco Webex Cloud is secure, reliable and highly available with unmatched performance.

The Webex Cloud is built and optimized for real-time media, with a global network that is engineered for effective meetings. It provides greater quality, reliability, and security that is impossible to achieve on the open and public internet.

Continue reading

Cisco is Named a Leader in Aragon Globe for Unified Communications and Collaboration 2020

This week, industry analyst firm Aragon Research published its fourth annual 2020 Aragon Research Globe™ for Unified Communications and Collaboration (UCC), and I’m thrilled that Cisco was identified again as leader.

Cisco was noted for key areas such as brand awareness, product strategy, market understanding, management team, marketing, product innovation and understanding of our customers’ needs.

Aragon’s report offers timely insights, which are even more pertinent in the 2020 and beyond.

The Foundation: Security, Reliability, and Scalability

From Aragon’s report:

“Because business leaders need to count on the platform provider, they need to consistently deliver—even in times of network congestion —  will cause the selection to be looked at more completely than just on the basis of feature comparisons.”

Communications has always been considered essential to business continuity. But as the recent massive demand for remote work has demonstrated, the complete set of collaboration tools are now also mission-critical. They must be engineered for uncompromising resilience and performance, even under extreme circumstances. This is especially true of cloud solutions — where enterprises may be exposed to failures and security breaches, not in their control.

At Cisco, we are singularly committed to security by default, privacy by design and complete transparency if any issues arise. In March 2020, the Webex cloud platform experienced a 4X volume surge in EMEA, 3X surge in APJC, 2.5X surge in the Americas and delivered over 14B meeting minutes – with a high level of reliability, security and data privacy rigorously enforced globally at all times.

The Need for a Complete Collaboration Platform in the Post-COVID World

From Aragon’s report:

“The platform play is the key criteria for enterprises. Providers still focus on core strengths, but the evaluation should be on the combination and the overall strength of the platform.”

Fully integrated business communications — including, calling, messaging, meetings, team collaboration, and contact centers — is now essential to productivity, user experience and total cost of ownership, especially when working remotely. So the overall strength of the platform is now a critical consideration.

We at Cisco have been laser-focused on platform integration and unified user experience for all collaboration functions. A single Webex cloud platform now supports all workloads, allowing the integration of all UCC functions, devices, analytics, service management, and external business system interfaces. And we have also unified the user interfaces across all applications with deep device and business application integrations to deliver an intuitive and consistent user experience.

The Path to the Cloud

From Aragon’s report:

“With Cisco’s broad UCC portfolio and the Cisco Collaboration Flex Plan, enterprises have a range of options that combine on-premises, hybrid, and public cloud deployments, as well as the flexibility to mix, match, and integrate services and deployment models as the business evolves.”

The shift to cloud-based collaboration has been underway for some time and following COVID-19 will accelerate even further. In fact, Aragon recommends that all enterprises consider adopting the cloud.

But while some enterprises will consider a flash transition to the cloud, for many the optimal path depends on individual needs and priorities. Enterprises may want to balance continuing to leverage the investment in their on-premises PBX while adding calling, meetings and team collaboration via the cloud. The transition path may be phased by moving specific workloads to the cloud, or by geographic and scalability considerations.

Cisco is in the privileged position of leading in both on-premises and cloud-based calling, meetings, and contact centers markets — so we are focused on enabling full transition flexibility for our customers.

We have integrated our premises-based Unified Communications Manager with Webex cloud collaboration via Webex Edge for Calling to enable hybrid deployments and unified user experience and management. We have also introduced Cisco UCM Cloud, providing dedicated hosted cloud deployment for enterprises with extensive integrations or special regulatory compliance. And we offer full commercial flexibility via our Collaboration Flex Plan and with our Webex Hardware as a Service offers.

Overall, I believe the key takeaways are: Communication and collaboration must be seamlessly integrated in  a single platform with unified experience and management, backed by uncompromising reliability and security, and flexibly delivered via both the cloud and on-premises based on individual enterprise needs.

Continue reading

Time Series Analysis with ARIMA: Part 3

XI. Cisco Use Case – Forecasting Memory Allocation on Devices

Background: Cisco devices have a resource limit (rlimit), based on their OS and platform type. When a device reaches its resource limit, it might indicate harm in the network or a leak. High water mark is used as a buffer to the resource limit. It is a threshold, that if it reaches the resource limit, indicates a problem in the network. For more information on high water mark, here is a useful presentation on what it means for routers and switches.

Problem: The client wants to track and forecast high water mark over a two-year period. We need to determine if/when high water mark will cross the resource limit two years into the future.

Solution: To solve this problem, we used time series forecasting. We forecasted high water mark over a two-year period.

A lot of effort went into thinking about the problem. I will put the general steps we took in solving this before going into the details. Notice that most of these steps were mentioned or highlighted in the first two blogs!

1. First, we looked at and plotted data
2. Second, we formulated hypotheses about the data and tested the hypotheses
3. Third, we made data stationary if needed
4. Fourth, we trained many separate model types and tested them against future values for validation
5. Finally, we gave conclusions and recommendations moving forward

1) Looked at and Plotted Data

When we initially received the dataset, we received numerous variables in addition to high water memory utilization over time. We received 65 devices and for each device, we were given about 1.5 years of data in monthly snapshots. Below is information on one example Cisco device. Note the variables labeled X and Y (timestamp and “bgp_high_water”). These are the variables we are interested in using for our forecast. The “bgp_high_water” variable is represented in bytes.

We then plotted data of all devices. Below is an example device highlighting the importance of visualization. Depending on how you look at the data can make immense difference in the interpretation of it. The graph on the right is plotted with respect to the resource limit, which makes the variability in bytes look much less extreme than it appears on the left. These graphs inform us the steepness of the rise of high water mark over time and became critical when interpreting forecasts and predictions with different models.

2) Formulated hypotheses about the data and test the hypotheses

We faced two big challenges when we looked at these data points on a graph.

Challenge #1: The first challenge we faced was related to the number of data points per device. We only had monthly snapshots of memory allocation over ~1.5 years. This means we had about 15 data points on average per device. For a successful time series analysis, you need least 2 to 3 years’ worth of data to predict 2 years out, which would mean we would need at the very least 24-36 data points per device. Because we didn’t have much data, we could not confidently or reliably trust seasonality and trend detection tests we ran. This issue can be resolved with more data, but at the time we were given the problem, we could not trust the statistics completely.

Challenge #2: The second challenge we faced was related to how we could use extra variables as leading indicators for a multivariate model. The red continuous variables below were all of the variables we could consider as leading indicators for high water mark. We had a wealth of choices and decisions to make, but we had to be extremely cautious about what we could do with these extra potential regressors.

Based on these sets challenges, we made 4 formal hypotheses about the data given.

Hypothesis 1: Given the current dataset, we cannot reliably detect trend or seasonality on a device.

We failed to reject the null hypothesis that there is no trend. Statistically, we found trending data in some devices – upward and downward. However, because of the number of data points, we felt there was not a strong enough signal to suggest there was a trend in the data. After removing trend, the data became statistically stationary. We detected no seasonality.

To make this conclusion, we plotted our data and ran the ADF and KPSS tests (mentioned in part 2 of the blog series) to inform our decisions. For example, let’s take a look at this particular device below. Visually, we see the data has some trend down, but it is not by many bytes. Additionally, we could see that for seasonality there is not much there. As mentioned before, we needed at least 2-3 solid years to detect seasonality or at least definitively say there is seasonality in the data. When we ran the ADF and KPSS tests, the results suggested that the data was non-stationary, but because there was so little data at the time, we believed it would not make a difference for our models if we made data stationary or non-stationary.

Hypothesis 2: There is no significant correlated lag between the PATH variables and BGP High Water

We found 3 features that were highly correlated with high water mark. They were IPV4, IPV6, and Speaker AS Paths. Because they were highly correlated, we thought these features could be used as leading indicators to high water mark. However, on closer inspection, that was not the case. For a variable to be a leading indicator, we needed the variable of interest to be highly correlated with high water mark at a lag point. For example, say the point at IPV4 rises by 1000 bytes in month 10, and high water mark will rise by some amount in Month 12. For IPV4 to be a leading indicator of high water mark, you would need to see this 2 month rise consistently across time.

Notice the plots below on the right. For the IPV4 variable, we lagged the data points by each month and then looked at the correlation coefficients to high water mark. Notice that you see a high correlation between the IPV4 points and the high water mark at the first data point, also called lag 0. The correlation coefficient is around 0.75. However, this correlation won’t tell us anything other than that when IPV4 paths rise, high water mark rises at the exact time and vice versa. If you look at the data points after the first one, you will see the correlation with high water mark go down and there is a slight dip before going back to no correlation at all. Because of this, we ruled IPV4 out as a leading indicator.

If you look at the relation between IPV6 and high water mark, at around 12 months lag, you see a correlation rise to about 0.6. IPV6 looks like an interesting variable at first sight, but with a little domain knowledge, you would understand that this is also not possible. Does it really seem possible for the example device below, that when IPV6 paths increase, high water mark increases a year later? If we had 2 years of data, and we saw that rise happen again we might think there is something there, but we did not have enough data to make that call. Now think about all of this for forecasting. We could not predict out a month, let alone 2 years, if the indicator we were basing your model on rose at the same time as the target variable. We therefore did not consider any variables for multivariate forecasting, but we wanted to leave little doubt about it. This leads to hypothesis 3.

Hypothesis 3: High water mark is a function of the paths: IPV4, IPV6, and Speaker_AS, so cannot be used as explainers/regressors for High Water Mark

The high correlation of these variables also might be indicating that they are a direct function of high water mark, meaning that if we added these variables together times some constant, we would get the high water mark number in bytes. The histograms below highlight the variability in this constant for each device. If this variability is low, it is an indication that the variables of interest to the high water mark are probably direct functions of high water mark. We confirmed this with the following formula:

(IPV4+IPV6+Speaker)*X = high water mark or IPV4+IPV6+Speaker = High Water / X, where X is the constant of interest.

Notice the plots below for three example devices. We plotted X at each time point as a histogram. Notice there is very little variability in X. Each device’s constant is centered around a particular mean. We therefore concluded that with the leading indicators of interest, we could not use a multivariate model to predict high water mark. We would need a leading indicator that is not directly related to our target variable. A good example of a leading indicator in this scenario would be the count of how many IP Addresses were on the network at a single time.

Hypothesis 4: There will be no significant difference in MAE and MAPE from a Baseline ARIMA Model ARIMA (1, 1, 0) or ARIMA(1, 0, 0)

We then decided that we would try some different model types on our train-test split to see which model had the best model performance. We made our metric for best performance the lowest mean absolute error and mean absolute percentage error on the test set of a train-test split. We then evaluated the models further by forecasting 2 years out and evaluated the MAE and MAPE up to current data point at the time we received the data (6 months). The next sections highlight the outcomes of hypothesis 4.

3) Made data stationary if needed

While we did not think we could reliably detect trend, we decided to run a model on differenced “stationary” datasets anyways. We also made models with the multivariate approach to see performance. We made a prior assumption that it would perform worse than a baseline ARIMA. Below is an example device plotted with its non-differenced and differenced data. All other device plots follow similar patterns.

4) Trained many separate model types and tested them against future values for validation

Below are the results of our initial analysis given the data on each device. We tried many different approaches, including ARIMA with a baseline model, Facebook’s Prophet model, ARIMAX, and an exponential smoothing model called local linear trend. Given the constraints of the data at the time, our best approach was a baseline ARIMA model. Results showed that across all devices, models using a baseline ARIMA with parameters p=1, d=0, and q=0 had the lowest MAE and MAPE. Given more data and time to detect the systematic components, we would likely have seen better results with more complex and smoother models. However, given the data we had so far, a simple ARIMA model performed really well!

The baseline ARIMA was not without its flaws. Given that most of the data constituted random components, it was very difficult to predict out 2 years for as little data as we had. Thus, you will see forecasting patterns similar to the examples below. Training and testing yields relatively low and good mean absolute errors, whereas forecasts yield higher and worse mean absolute error over time.

Below is an example of a sample prediction on a test set for 2-3 months out using ARIMA(1,0,0). Notice that both the training and testing Mean Absolute Error and Mean Absolute Percentage error are very low.

Next, is an example forecast done 6 months out without validation data using ARIMA(1,0,0). Notice the forecast is a straight line and visually you can see that the forecast error is much higher. It shows that a baseline ARIMA cannot adjust forecasts to the random change in the data. Because there was no reliable detected seasonality, cycle, or trend in the data per device, the model was trying to predict random components.

5) Conclusions and Recommendations

As a result of our initial analyses, we chose a baseline ARIMA model over other more complicated models. In our conclusion, we recommended to the client that they probably shouldn’t be forecasting out more than a couple of months. Instead, they can do rolling predictions out every 2 months as new data comes in and then scale to more complex models once trends or seasonality are spotted in the data.

For some good news on the project, we have recently gotten more data, and we have been able to integrate more complex models that utilize trend data. In our latest dataset for this use case, we have been able to spot trends and been able to make 2 year predictions with much more confidence using a model approach called local linear trend. We used this in our initial model approach but did not have much confidence in it given the data constraints at the time. Fortunately, time series models in general are very flexible given that the data input is simply time and a target variable. Model adaptations are therefore very fast and easy to implement once you understand the basics of time series analysis and have enough data to spot trends and seasonality.

XII. Designing Your Own Time Series Forecasts

Let’s end the blog with some rules you can follow to develop your own time series analysis. This will give you the confidence to start thinking about time series in a new way.

1) Always plot and decompose your data

Always plot and decompose your data. Even if you don’t think there are any trends or seasonality, it’s always good to look at your data!

2) Construct hypotheses about your data after looking at it.

Always construct hypotheses about your data after you look at it. You want to do this so that you can formalize what you think is going on in your datasets and test your assumptions about them. For example, suppose you were asked to predict out 2 years with only monthly snapshots of data. Maybe you don’t think you have enough data to detect trend or seasonality. It is possible that you will find no systematic patterns of trend and seasonality in your dataset. If you do not, know you will most likely be predicting only the random components of your dataset. This will help you make strong, testable conclusions later when presenting to stakeholders.

3) For forecasting, make sure your data are stationary. For some smoothing models you may not need to do this.

Remember the definition and rules of stationarity when using models like ARIMA!

4) If multiple variables are present in your data, try to determine the usefulness of using them in your multivariate time series model.

Can other continuous variables be used as leading indicators or signals to help you predict the next data point? If not, don’t use them! They likely won’t help your forecasts, and might actually make them worse.

5) Choose models that make sense for the data you are given, and don’t be afraid to experiment with model parameters.

The autoregressive and the moving average terms may sometimes cancel each other out under certain conditions. Refer to the post below to help you understand your data better and pick the appropriate lag parameters.

6) Compare model parameters and model types. You will likely find that simpler models are usually better.

Always try using simple models first as a baseline, like ARIMA(1,0,0) or ARIMA(1, 1, 0). Then you can add more complexity if your data has complex systematic components like trend and seasonality. You will likely get better prediction accuracy and lower forecast error as you get more data. Additionally, if you don’t need multivariate analysis, don’t use multivariate analysis. Only use it if you think it will improve your forecasts.

Continue reading