Using the Knowledge Store on Cisco Observability Platform

Build custom observability solutions

Cisco Observability Platform (COP) enables developers to build custom observability solutions to gain valuable insights across their technology and business stack. While storage and query of Metric, Event, Log, and Trace (MELT) data is a key platform capability, the Knowledge Store (KS) enables solutions to define and manage domain-specific business data. This is a key enabler of differentiated solutions. For example, a solution may use Health Rules and FMM entity modeling to detect network intrusions. Using the Knowledge Store, the solution could bring a concept such as “Investigation” to the platform, allowing its users to create and manage the complete lifecycle of a network intrusion investigation from creation to remediation.

In this blog post we will teach the nuts and bolts of adding a knowledge model to a Cisco Observability Platform (COP) solution, using the example of a network security investigation. This blog post will make frequent use of the FSOC command to provide hands-on examples. If you are not familiar with FSOC, you can review its readme.

First, let’s quickly review the COP architecture to understand where the Knowledge Store fits in. The Knowledge Store is the distributed “brain” of the platform. The knowledge store is an advanced JSON document store that supports solution-defined Types and cross-object references. In the diagram below, the Knowledge Store is shown “connected” by arrows to other components of the platform. This is because all components of the platform store their configurations in the knowledge store. The Knowledge Store has no ‘built-in’ Types for these components. Instead, each component of the platform uses a system solution to define knowledge types defining their own configurations. In this sense, even internal components of the platform are solutions that depend on the Knowledge Store. For this reason, the Knowledge Store is the most essential component of the platform that absolutely nothing else can function without.

To add a more detailed understanding of the Knowledge Store we can understand it as a database that has layers. The SOLUTION layer is replicated globally across Cells. This makes the SOLUTION layer suitable for relatively small pieces of information that need to be shared globally. Any objects placed inside a solution package must be made available to subscribers in all cells, therefore they are placed in the replicated SOLUTION layer.

Solution Level Schema

Get a step-by-step guide

From this point we will switch to a hands-on mode and invite you to ‘git clone git@github.com:geoffhendrey/cop-examples.git’. After cloning the repo, take a look at https://github.com/geoffhendrey/cop-examples/blob/main/example/knowledge-store-investigation/README.md which offers a detailed step-by-step guide on how to define a network intrusion Type in the JSON store and how to populate it with a set of default values for an investigation. Shown below is an example of a malware investigation that can be stored in the knowledge store.

Malware Investigation

The critical thing to understand is that prior to the creation of the ‘investigation’ type, which is taught in the git repo above, the platform had no concept of an investigation. Therefore, knowledge modeling is a foundational capability, allowing solutions to extend the platform. As you can see from the example investigation below, a solution may bring the capability to report, investigate, remediate, and close a malware incident.

If you cloned the git repo and followed along with the README, then you already know the key points taught by the ‘investigation’ example:

1. The knowledge store is a JSON document store
2. A solution package can define a Type, which is akin to adding a table to a database
3. A Type must specify a JSON schema for its allowed content
4. A Type must also specify which document fields uniquely identify documents/objects in the store
5. A solution may include objects, which may be of a Type defined in the solution, or which were defined by some different solution
6. Objects included in a Solution are replicated globally across all cells in the Cisco Observability Platform.
7. A solution including Types and Objects can be published with the fsoc command line utility

Provide value and context on top of MELT data

Cisco Observability Platform enables solution developers to bring powerful, domain specific knowledge models to the platform. Knowledge models allow solutions to provide value and context on top of MELT data. This capability is unique to COP. Look for future blogs where we will explore how to access objects at runtime, using fsoc, and the underlying REST APIs. We will also explore advanced topics such as how to generate knowledge objects based on workflows that can be triggered by platform health rules, or triggers inside the data ingestion pipeline.

Source: cisco.com