DevOps with CloudCenter Suite and Kubernetes in a Multicloud Environment – Part 2

This post is the 2nd part our series on DevOps and will focus on a CI/CD demo based on Cisco Multicloud Portfolio. You can find part one here. For our demo environment, we are using resources from 3 Kubernetes clusters, on-premises and in AWS.

Our lab

We have built a simple microservice-based application as shown by the picture below.

 The source code of the 5 components is stored in a github repository, where new versions of the application are committed (uploaded) by developers. At each commit, the Jenkins orchestrator gets the source code and compiles it, building the container images ready to deploy the application.

The images are saved in a shared container registry (Harbor, see next picture) where Cisco CloudCenter (or Cisco CloudCenter Suite, as per the official new title) will be able to retrieve them when asked by Jenkins to deploy the application. Based on input parameters provided by Jenkins, Cisco CloudCenter will target the deployment to the most appropriate environment for the current phase of the project.

In our demo lab, the environments are “integration test”, “performance test” and “production”.

They correspond to three different Kubernetes clusters that have been created on-premises (integration and performance test) and in AWS (production).

Each environment has different set of policies, that will be inherited by every application that is deployed there: policies for security, networking, autoscaling, etc.

The 3 Kubernetes clusters mentioned above have been quickly deployed by the Cisco Container Platform (CCP) without having to manually create them on each side.

The value in using CCP here is simple: in few minutes we created and deployed 3 production-ready clusters, fully integrated with networking, storage, security, monitoring and logging without even touching the K8s installer or the underlying infrastructure.

The 2 clusters named “integration test” and “performance test” were created automatically inside VM in a local VMware environment, while the cluster named “production” was created in AWS (CCP uses the API exposed by AWS’s Managed Kubernetes Service (EKS) to do everything automatically, including the integration with AWS’s Identify and Access Management (IAM) for authentication, authorization and access control)

The automated deployments will repeat, in the three environments, in a sequence that tests each version before moving it to the next deployment environment, ensuring the quality of the release. In the real world you might want to run more complex testing activities (such as code quality inspection, security, resiliency etc) than the 2 tests in this example (functional and performance).

Demo flow

◈ The next picture is a sequence diagram showing all the actions that we have automated; we used a color code to represent the phases that are commonly referred to as Continuous Integration (the green part) and Continuous Deployment (the orange part).

CCC stands for Cisco CloudCenter, where K8s dev, test and production represent the 3 Kubernetes clusters mentioned above.

◈ The entire process is completely automated and brings a new version of the application to the production deployment without any human intervention. This complete automation is often referred to as Continuous Deployment and – although very useful and adopted by big players like Facebook (their pipeline is more complex than our simplified demo) – is not very common among the customers I generally meet.

Those that adopted DevOps still prefer to have some human checks in between the activities, so that they feel they have a better control on the process and its quality.

When they have more experience, they will probably be confident enough to delegate every check to the automation tools.

Implementation

The automation is based on Jenkins, an open source orchestrator that benefits from the availability of hundreds of plugins; it can automate almost every component in your IT ecosystem, including Cisco CloudCenter of course.

In the Jenkins dashboard you can build different projects, like in the picture below. A project is a sequence of steps, using plugins to drive activities in the systems you want to automate (e.g. pull the source code from the repository, compile it, build container images, trigger a cloud deployment through Cisco CloudCenter, etc.).

Projects can call other projects, to make your orchestration modular and reusable. In the picture above, the project ‘TheWall’ (that is the name of our demo application) calls the other 5 projects in a sequence, checking that the outcome is positive before calling the next one.

◈ With this we are able to automate the deployments on those 3 Kubernetes clusters and run the functional test and the performance test of the application using an external tool (here we are using another open source product called Apache Jmeter).

◈ The functional test (which happens on the integration test cluster) is a sequence of user transactions, executed by the test tool using a pool of user identities and a pool of input data such as simulated clicks and text inputs, where assertions about the expected result are validated automatically. If the page generated by the application differs from the expected result, an error is logged, and the test can be considered failed. So, the functional test ensures that the application behaves as expected from a functional standpoint (and you can avoid a manual test for user acceptance).

The performance test (which happens on the performance test cluster), executed by the same tool, stresses the application and the infrastructure from a performance standpoint. A large number of concurrent users are simulated by the tool, invoking a sequence of user transactions with random wait time, reproducing a situation similar to the workload in a production environment. Response times are tracked and so are eventual errors, allowing the tool to declare whether the test is successful or not.

Based on the outcome produced by Jmeter, Jenkins will continue with the Continuous Deployment pipeline or abort it, notifying the developers that something went wrong, requiring a correction. In the latter scenario, the CI/CD cycle will start from the beginning: new modified source code modified committed, application built and deployed to the first environment, test executed, application promoted to next environment and tested… until the pipeline is completely executed without any warning or error and the application is released automatically in production.

The next picture shows the execution of the Jenkins pipeline for three different builds of the application. The most recent execution failed because the modification of the source code introduced an error that blocked the build. The other two executions succeeded, as demonstrated by the green color of every step in the pipeline.

Jenkins logs all the activities, so that you can check what happened during the automated process.

The next picture shows the output of the sub-project named ‘TheWall_Deploy_Test’, that is the 7th stage in the pipeline in previous picture.

In order for us to ensure that governance policies are applied during deployment (such as access control, reporting, cost control etc), we have inserted CloudCente in the process. Jenkins will use the API exposed by Cisco CloudCenter to deploy the application ‘TheWall’ to the test environment.

Note that the performance test environment needs to be robust enough to sustain the workload of the performance test, whileon the contrary, the functional test can be executed in a smaller cluster with less computing power.

You don’t have to code the API calls, because Cisco CloudCenter ships a plugin for Jenkins that integrates into its user interface graphically. But if you prefer, Jenkins can run scripts and commands from the CLI for you.