Hyperconverged Infrastructure with Harvester: The start of the Journey

Deploying and running data center infrastructure management – compute, networking, and storage – has traditionally been manual, slow, and arduous. Data center staffers are accustomed to doing a lot of command line configuration and spending hours in front of data center terminals. Hyperconverged Infrastructure (HCI) is the way out: It solves the problem of running storage, networking, and compute in a straightforward way by combining the provisioning and management of these resources into one package, and it uses software defined data center technologies to drive automation of these resources. At least in theory.

Recently, a colleague and I have been experimenting with Harvester, an open source project to build a cloud native, Kubernetes-based Hyperconverged Infrastructure tool for running data center and edge compute workloads on bare metal servers.

Harvester brings a modern approach to legacy infrastructure by running all data center and edge compute infrastructure, virtual machines, networking, and storage, on top of Kubernetes. It is designed to run containers and virtual machine workloads side-by-side in a data center, and to lower the total cost of data center and edge infrastructure management.

Why we need hyperconverged infrastructure

Many IT professionals know about HCI concepts from using products from VMWare, or by employing cloud infrastructure like AWS, Azure, and GCP to manage Virtual Machine applications, networking, and storage. The cloud providers have made HCI flexible by giving us APIs to manage these resources with less day-to-day effort, at least once the programming is done. And, of course, cloud providers handle all the hardware – we don’t need to stand up our own hardware in a physical location.

Multi-node Harvester cluster

However, most of the current products that support converged infrastructure tend to lock customers to using their company’s own technology, and they also usually come with licensing fees. Now, there is nothing wrong with paying for a technology when it helps you solve your problem. But single-vendor solutions can wall you off from knowing exactly how these technologies work, limiting your flexibility to innovate or react to issues.

If you could use a technology that combines with other technologies you are already required to know today – like Kubernetes, Linux, containers, and cloud native – then you could theoretically eliminate some of the headaches of managing edge compute / data centers, while also lowering costs.

This is what the people building Harvester are attempting to do.

Adapting to the speed of change

Cloud providers have made it easier to deploy and manage the infrastructure surrounding applications. But this has come at the expense of control, and in some cases performance.

HCI, which the cloud providers support and provide, gets us some control back. However, the recent rise of application containers, over virtual machines, changed again how infrastructure is managed and even thought of, by abstracting layers of application packaging, all while making that packaging lighter weight than last-generation VM application packaging. Containers also provide application environments that are  faster to start up, and easier to distribute because of the decreased image sizes. Kubernetes takes container technologies like Docker to the next level by adding in networking, storage, and resource management between containers, in an environment that connects everything together. Kubernetes allows us to integrate microservice applications with automation and speedy deployments.

Kubernetes offers an improvement on HCI technologies and methodologies. It provides a better way for developers to create cloud agnostic applications, and to spin up workloads in containers more quickly than traditional VM applications. Kubernetes did not aim to replace HCI, but it did make a lot of the goals of software deployment and delivery simpler, from an HCI perspective.

In a lot of environments, Kubernetes runs inside VMs. So you still need external HCI technology to manage the underlying infrastructure for the VMs that are running Kubernetes. The problem now is that if you want to run your application in Kubernetes containers on infrastructure you have control of, you have different layers of HCI to support.  Even if you get better application management with Kubernetes, infrastructure management becomes more complex. You could try to use vanilla Kubernetes for every part of your edge-compute / data center stack and run it as your bare metal operating system instead of traditional HCI technologies, but you have to be ok migrating all workloads to containers, and in some cases that is a high hurdle to clear, not to mention the HCI networking that you will need to migrate over to Kubernetes.

The good news is that there are IoT and Edge Compute projects that can help. The Rancher organization, for example is creating a lightweight version of Kubernetes, k3s, for IoT compute resources like the Raspberry Pi and Intel NUC computers. It helps us push Kubernetes onto more bare metal infrastructure. Other orgs, like KubeVirt, have created technologies to run virtual machines inside containers and on top of Kubernetes, which has helped with the speed of deployment for VMs, which then allow us to use Kubernetes for our virtual networking layers and all application workloads (container and VMs). And other technology projects, like Rook and Longhorn, help with persistent storage for HCI through Kubernetes.

If only these could combine into one neat package, we would be in good shape.

Hyperconverged everything

Knowing where we have come from in the world of Hyperconverged Infrastructure for our Data Centers and our applications, we can now move on to what combines all these technologies together. Harvester packages up k3s (light weight Kubernetes), KubeVirt (VMs in containers), and Longhorn (persistent storage) to provide Hyperconverged Infrastructure for bare metal compute using cloud native technologies, and wraps an API / Web GUI bow on it to for convenience and automation.

Source: cisco.com

Continue reading

Focus on HyperFlex: Sizing A New Cluster Using the Sizer and Profiler Tools

In this ‘Focus on HyperFlex’ blog, we’ll zero-in on different aspects of the Cisco HyperFlex (‘HX’) hyperconverged system and ways to make HX work best for you and your organization. This edition will illustrate on how to size a cluster when you might not have all the details of the workload worked out. In this situation, HyperFlex Profiler is the right approach to learn more about the workloads.

During my time in sales, teams often asked me to size a HyperFlex cluster and provide a customer quote. It was customary to have many more questions than the team or customer could answer about the application. Normally, they would provide me with an Excel sheet with some CPU and memory values. That is a great start, and it gave me deep insight into the customer’s application. However, an application profile is not only about averages of CPU and memory.  There are several more parameters needed, including the performance and latency peaks. With the customer’s permission, I would run a HyperFlex Profiler in their environment to gain more information about their application. Before installing the OVA on their vCenter, I would explain what HyperFlex Profiler is and how it helps with sizing their new HyperFlex environment. 

HyperFlex Profiler 

If there is no historical insight into the potential clustered application environment, then start with HyperFlex Profiler. HyperFlex Profiler will gather data on the vCenter environment and consolidate that mass of data to a single, easily digestible file. This file will quickly size the cluster after importing it into the HyperFlex Sizer tool and paint a clearer picture of the environment and workloads. 

However, profiling the environment is not a quick hit in a short period of time. The best approach is to run the HyperFlex Profiler for at least seven days or, preferably, 30 days. A longer measuring period ensures you capture data when “end of the month reports” are run. Of course, don’t just measure the environment during a weekend when there is little traffic! Be sure to capture at least one logical business cycle for that application. 

The HyperFlex Profiler is an OVA installed on a VMware environment. The only configuration is to provide (read-only) access to the vCenter environment and define which servers the HyperFlex Profiler will monitor. Multiple servers and/or clusters can be selected. For environments running different types of workloads, it is recommended to isolate them by selecting the servers in their environments – for instance, the VDI or the SQL environment. Of course, selecting all servers and workloads is also an option. Keep in mind that you will have more overhead this way. 

When it is monitoring the environment, you will see the following: 

More details about the environment can be shown in other tables and graphics, that can be exported in pdf format. 

It is essential to see the peaks of the environment. This way, you can make sure the new HyperFlex designed cluster can handle the workload, and ensure there is room for expansion. 

There are different graphs in the HyperFlex Profiler. Here you can see the metrics of the storage reads. Here, you have more insight into the frequently used block-size of the environment. This is one of them:

The next step is to use the power of the HyperFlex ProFiler to create a bill of material out of all the information from vCenter.  The data of the HyperFlex Profiler can be manually or automatically downloaded and uploaded to the HyperFlex Sizer. 

HyperFlex Sizer 

The HyperFlex Sizer is an online tool (https://hyperflexsizer.cloudapps.cisco.com/) accessible to both partners and customers. With this tool, you can add your personal, most commonly used workloads to a HyperFlex cluster and have the sizing tool decide the best option. HyperFlex Sizer takes the HyperFlex best practices into account when calculating the optimal solution. 

Furthermore, you can customize the sizing tool, using only the preferred components the customer wishes to see in the new HyperFlex environment. Different elements, like CPU, memory, types and sizes of drives, and more, can be customized. 

Uploading the data of the HX Profiler to the HX Sizer is entirely straightforward. After claiming the HX Profile into the HX Sizer, the tool will produce a practical Bill of Materials that can be the baseline of a  discussion with your partner about the best solution for your applications. 

Here is a screenshot of the HyperFlex Sizer where a HyperFlex cluster is calculated with the requested workload:

A variation of HyperFlex clusters may be advised. This is possible when you want to have different workloads, each with their characteristics. 

It shows the amount of HyperFlex or compute nodes, what type of nodes and  includes all the parts that are needed to create the solution. This way, you don’t have to configure everything manually, eliminating human errors.  

Create an estimate

Once you know the total size of the new HyperFlex cluster, partners or Cisco experts can easily upload the Bill of Material to Cisco Commerce Workspace (CCW) and estimate the HyperFlex cluster.

In CCW, the estimate can be converted to an order.

Source: cisco.com

Continue reading

Modernizing to Oracle 19c with Hyperconverged Infrastructure

Oracle Database 19c is a long-term support release of the database, offering customers the best performance, scalability, reliability, and security for all their operational and analytical workloads. The core aim for 19c is stability, as it forms a foundation for next phase of autonomous database optimizations. These optimizations include the ability for the database to automatically create indexes, which allow for the database to self-optimize and maintain optimal configuration.

Additional optimizations in Oracle Database 19c include real-time statistics for all operations and the ability to automatically quarantine problematic sequel. It also includes key unique innovations for core enterprise capabilities. For a lot of customers who run standby databases, Oracle 19c accepts updates to those standby databases, thereby turning the standby from a read-only to a read-mostly asset. In order to deal with streaming data or to have IOT type workloads, 19c provides a new in-memory rows store and API to provide very high speed and high volume data ingest.

Below is the list of some of the new features in Oracle Database 19c:

◉ General – Flashback Standby database when primary database is flashed back
◉ Performance – SQL Quarantine
◉ RAC & Grid Infrastructure – Zero-Downtime Oracle Grid infrastructure patching
◉ Availability – Dynamically change Fast-Start Failover (FSFO) target
◉ Application Development – REST Enabled SQL Support
◉ Automatic Indexing
◉ Database IN-Memory
◉ REAL-TIME Statistics Collections
◉ High Availability

Oracle on A Hyperconverged Infrastructure

The rapidly emerging world of hyperconverged infrastructure (HCI) promises many technical and financial benefits. The marketplace has been quick to recognize and validate the advantages of HCI, which combines the main features of a three-tier architecture – compute, storage, and networking – into a single solution. Having superior, vendor-supplied software is crucial to facilitate easy and efficient management of resources. In addition to managing the essentials of compute, storage and networking layers, HCI solutions also provide features to handle DR and the ability to scale up, with additional nodes as needed.

For an Oracle customer, the benefits of HCI from a technical optimization perspective are clear. You reduce unused hardware capacity through better resource management, eliminate network devices, and scale up your Oracle deployments easily and rapidly in response to rising demand. Adding nodes and moving workloads becomes seamless.

Various considerations while choosing an HCI environment:

◉ A Simple, integrated software stack which allows to do more with less
◉ Consistent, high performance that satisfies demanding business critical applications
◉ Flexible deployment models that align IT expertise with business priorities
◉ Advanced automation capabilities that enable IT agility
◉ Hypervisors that offer confidence and lower risk

Cisco’s Hyperconverged Infrastructure

Businesses across all verticals are seeing benefits behind this tight integration with virtual technologies as well. HCI reduces complexity and fragmentation around having to manage resources sitting on heterogeneous systems; it can reduce data center footprints; and it can greatly reduce deployment risks with validated deployment architectures.

There’s clear demand in the market. Consider this: according to the latest Gartner Magic Quadrant for Integrated Systems report, “hyperconverged integrated systems will represent over 35 percent of the total integrated system market revenue by 2019.” This makes it one of the fastest-growing and most valuable technology segments in the industry today.

With a co-engineered hardware and software solution Cisco has become a Gartner Magic Quadrant leader in HCI with Cisco HyperFlex. Taking in all the considerations listed above while choosing an HCI environment, Cisco’s HyperFlex is a great solution and a great technology to consider while running oracle on. That said, there are numerous critical features that set this technology apart from any other HCI solution out there. One of those aspects revolves around the fact that HyperFlex comes with full network fabric integration. This type of integration allows administrators to create QoS policies and even manage vSwitch configurations that scale throughout the entire fabric interconnect architecture. This approach provides data reliability and fast database performance. Cisco HyperFlex integrates servers, storage systems, network resources, and storage software to provide an enterprise-scale environment for an Oracle Database deployment. This highly integrated environment provides reliability, high availability, scalability, and performance to handle large-scale transactional workloads.

Oracle Databases and Real Application Clusters (RAC) are the core of many enterprise applications, including online transaction processing (OLTP), data warehousing, business intelligence, report generation, and online analytical processing (OLAP). As the amount and types of data increase, you need flexible and scalable systems with predictable performance to address database sprawl. By deploying Cisco HyperFlex with All Flash or All NVMe nodes, you can run your Oracle Database and RAC deployments on an agile platform that delivers insight in less time and at less cost.

Cisco HyperFlex systems with Oracle Database and RAC:

◉ Closely match the needs of databases and applications
◉ Reduce Storage footprint
◉ Optimize storage costs
◉ Deliver predictable database performance
◉ Keep enterprise applications and database available

The first fully engineered hyperconverged appliance based on NVMe storage delivers more of what you need to propel mission-critical workloads:

◉ It provides 71 percent more I/O operations per second (IOPS) and 37 percent lower latency than our previous-generation all-flash node.
◉ Also provides 15% more storage efficiency due to less storage needed when using the Cisco HyperFlex Acceleration Engine.

All-NVMe solutions support the most latency-sensitive applications with the simplicity of hyperconvergence. Our solutions provide the first fully integrated platform designed to support NVMe technology with increased performance and RAS.

As mentioned above, Cisco HyperFlex systems best suites Oracle Database and RAC deployments, best performance at a cost of very low latency. To help you deploy Oracle 19c and Oracle RAC, we deployed Oracle 19c Database on a 4 node HyperFlex cluster and tested it with various configuration profiles. Below are the links to the Oracle whitepaper references containing validation results tested internally. From the test results, it’s clearly evident that Cisco HyperFlex systems can handle OLTP highly intensive workloads by delivering best performance at a very low cost and latency.

Continue reading

Cisco Hyperflex: The Time for HCI Compromise is Over

Hyperconverged infrastructure (HCI) – it’s a bit of a mouthful but the truth is it’s not half as complicated as it sounds.

While the concept has only been mainstream for a few years, HCI is the fastest-growing segment of the market for integrated systems.

In fact, Gartner predicts it will grow from a niche industry in 2012 to a $5 billion industry by 2019[i], while a recent ESG Lab report states that 85% of businesses currently use, or are planning to use, HCI solutions.

Continue reading