Part 1: The Technology of Trust
It wasn’t that long ago that most enterprise resources resided in heavily-protected on-premises data centers surrounded by vigilant layers of security. Trust was placed in securing the computing environment itself—the physically isolated data center that only trained and vetted personnel could physically access and the network that strictly controlled connections from enterprise endpoints.
Now all the physical and virtual security layers built into data centers that we relied for years have changed. Data “centers” are wherever the data and applications are located—in public clouds, SaaS providers, branch sites, and edge compute and storage. Every employee uses mobile devices—some of which are personally owned—for accessing and manipulating corporate resources. Virtualized workloads move among cloud or IaaS providers to improve responsiveness to a workforce in constant motion. Branch sites quickly need new direct internet connections to work with cloud applications. Global black swan events cause enormous shifts as whole populations rapidly move to working remotely. The attack surface has grown exponentially, literally overnight.
Another evolutionary and significant change in the attack surface results from transformations in the basic architecture of router and switch hardware. Just a decade or so ago, most routers and switches that formed the foundation of corporate and world-wide networks were built with proprietary silicon, hardware, and specialized operating systems. Back then, hackers were content to focus on the population of billions of x86-based and Windows-operating PCs and Linux servers, taking advantage of the same vulnerabilities and exploitive toolsets to invade and infect them. They largely ignored routers and switches because they required specialized tools to attack vendor-specific vulnerabilities—too much work for threat actors with such a target-rich environment of x86 machines to troll.
Hackers Turn to the Network for Access
But the choice of targets changed as hackers increasingly turned their attention to attacking enterprise data sources spread over distributed data centers, cloud platforms, and a mobile workforce. The opportunity for attacks became more prevalent as routers and switches gradually standardized on commodity hardware, Linux, and modules of open source code. Some enterprises and cloud providers are even trying white box and bare metal boxes to shave a few dollars off data center networking CAPEX. However, it’s the shared x86 and ARM code, Linux, and open source code that draws the attention of hackers, providing a potential foothold into corporate data-rich networks. By focusing on gaining access to network devices through common attack routes, malevolent hackers may be able to steal security keys and IP addresses among other sensitive data, opening more pathways into the network and the treasure-troves of corporate data. This attack vector can be particularly insidious when open source code that is integrated into the Network Operating System has Common Vulnerabilities and Exposures (CVEs) that go unpatched, offering hackers convenient documented doorways.
Own Networking Devices, Own the Traffic.
As network components became a more attractive target for hackers and nation-state spies looking for access to corporate resources, the sophistication of attacks increases to not only gain control over routers, but to hide the evidence of infiltration. One particular advancement is the development of persistent malware that can survive hard resets because the malicious code alters the BIOS or even the lower level boot loaders to gain control of the root. Without built-in protection at the hardware root layer that prevents altered low-level code from loading and infecting the entire OS, detecting and eliminating these persistent malware infections is next to impossible. An infected networking device—router, switch, access point, firewall—becomes an open gateway to all the network traffic.
With the multitude of dangers constantly testing the gateways of networks, there should be no such concept of “implicit trust”. At the core of the defensive network is the principle of proven trustworthy hardware and software working in conjunction to protect network devices from attack. By building on a foundation of trustworthy networking components and secure software, the connected sources of data and applications distributed through cloud services, branch sites, and a mobile workforce are also protected.
What are the fundamentals of building trusted network systems and what should CIOs and CSOs be asking of their vendors?
It’s interesting to examine the paradox of enterprise CIOs and CSOs insisting that the hardware and operating systems of data center components are verifiably trustworthy while relying on a network fabric based on commodity components. To secure critical data and application resources, trust must permeate the hardware and software that run networks from the data center to campus to branch and cloud. Needless to say, that covers a very large territory of intertwined components, each of which must reinforce the trustworthiness of the complete network. Some of the major points to research when selecting network vendors and choosing components are:
◉ Is the network hardware authentic and genuine?
◉ What is the origin of all the software installed, including BIOS and open source modules?
◉ Have components or installed software been modified while the unit was in transit from manufacturing to customer?
◉ Are network devices running authentic code from the vendor?
◉ Are there any known CVEs with the software, including any open source modules?
◉ Are software and security patches available and applied?
◉ Is enterprise sensitive information, such as encryption keys, stored on network devices protected?
To ensure trust in a network, security-focused processes and technologies must be built into the hardware and software across the full lifecycle of solutions. That high level of engineering and supply chain control is very difficult to accomplish on low-margin, bare metal hardware. If the choice comes down to savings from slightly less costly hardware versus increase in risk, it’s worthwhile remembering the average cost of a single stolen record from security breaches is $155 (in the U.S.), while the cost of the loss of customer trust and theft of intellectual property is incalculable.
A Chain of Trust should be built-in, starting with the design, sourcing of parts, and construction phase of both hardware and root-level software. It continues throughout the Secure Development Lifecycle (SDL), all the way to end of life for secure disposal of routers and switches, which can have sensitive data still stored in memory.
Security Embedded in Hardware
Hardware engineers must have an overriding mindset of security starting from sourcing of parts from reliable and certified manufacturers; designing secure boot strategies using Secure Unique Device Identifiers (SUDI) embedded in Trust Anchor modules (TAm); and Root of Trust tamper-resistant chips for secure generation and storage of cryptographic key pairs.
The SUDI is an X.509v3 certificate with an associated key-pair protected in hardware. The SUDI certificate contains the product identifier and serial number and is rooted to the Cisco Public Key Infrastructure. This identity can be either RSA- or ECDSA-based. The key pair and the SUDI certificate are inserted into the TAm during manufacturing so that the private key cannot be exported. The SUDI provides an immutable identity for the router or switch that is used to verify that the device is a genuine product. The result is hardware that can be verified as authentic and untainted, maximizing the security of the traffic flowing through the network.
Secure Software Development Lifecycle
Software developers must strictly follow Secure Development Lifecycle guidelines for coding the network operating systems with a combination of tools, processes, and awareness training that provides a holistic approach to product resiliency and establishes a culture of security awareness. From a trust perspective, the SDL development process includes:
◉ Product security requirements
◉ Management of third-party software, including open source code
◉ Secure design processes
◉ Secure coding practices and common libraries
◉ Static analysis
◉ Vulnerability testing
Security and Vulnerability Audits provide assurance that as problems are uncovered during the software development and testing cycles, they cannot be ignored. Ideally, the audit team reports not to engineering management but to the office of CSO or CEO to ensure that problems are completely fixed or the release red-lighted until they are remediated. This is an example of a culture of trust that permeates across functional departments all the way to the C-level—all in service of protecting the customer.
Once trust is established for the development and release of networking software, the next step is to ensure that the software images delivered to customers’ controllers are original and untainted. To make a downloadable image trustworthy, it can be protected with a hash created with SHA-512, encrypted with a private key, and combined with the software image as a digital signature package. That image is downloaded directly to customers’ network controllers. The controllers use a matching public key to decrypt the digitally-signed hash and image package to verify nothing has been altered from the original image. Only software images that pass this critical test can be loaded on devices, preventing any malicious code from booting on a controller, router, or switch. This step ensures that only the vendor’s approved and unaltered code is running on network devices.
Reporting on CVEs Is Only One Way to Measure Solution Maturity
As previously discussed, open source and third-party code and APIs are commonly integrated into all modern network operating systems. Likewise, the core OS for network devices is commonly based on open source Linux kernels. All of these externally-developed code modules need to be certified by their creators and tested by neutral third-party organizations as well as the ultimate end-use vendor. As the number of products, software packages, and connected devices in networks continue to rise, it’s inevitable that more security vulnerabilities will also increase. Ironically perhaps, one reason for the increase in Common Vulnerabilities and Exposures (CVEs) is that the industry is getting better at finding and reporting them.
To ensure trustworthy networking, vendors must commit to Coordinated Disclosure, starting with a system for tracking every piece of open source and third-party code incorporated into network products. In addition, every CVE discovered either internally or externally needs to be tracked and fixed by a responsible entity in the trusted value chain of partners and suppliers. Customers, partners, and researchers need to be consistently notified of every recently discovered and fixed CVE and the potential for risk. Transparency improves customer security. That’s why at Cisco we report and provide fixes for CVEs that are found by our own internal testing, our customers, and third-party researchers.
A Third-Party Software Digitization (TPSD) program is another example of a corporate-wide initiative for managing third party software—including royalty bearing commercial software as well as free and open source software. The system automates the tracking and tracing of “Where Used, Where Is, Where Distributed” for compliance, contributions, security, quality, and customer accountability. A TPSD ensures that when a CVE is uncovered in open source or third-party code, the exact usage of the code is immediately known so that fixes can be applied in every instance.
As with the Security and Vulnerability Audit team, a Product Security Incident Response Team (PSIRT) that is independent from engineering is critical to keeping an unbiased watchful eye on all internally and externally developed code. PSIRT members continuously monitor open source and third-party code used in NOSs for reported CVEs. In addition, customers and researchers need a documented method of reporting to PSIRTs any CVEs that they discover. In turn, vendors need a transparent process for reporting on fixes and patches and level of severity back to customers, partners, and security researchers.
An important way to collaborate with industry peers on CVEs is through the Forum of Incident Response and Security Teams (FIRST) organization, which authors the PSIRT Framework. The Framework identifies core responsibilities of PSIRT teams, provides guidance on how to build capabilities to investigate and disclose security vulnerabilities and their remediations to customers in a transparent manner. As you consider the trustworthiness of your networking ecosystem, it’s worthwhile considering your vendors involvement with organizations like FIRST and the Industry Consortium for Advancement of Security on the Internet (ICASI)—which are strong indicators of a fundamental commitment to transparency.
Building in Trust Throughout the Product Lifecycle
All the processes and capabilities I have described in this post are integrated into Cisco’s hardware and software development processes. Cisco embeds security and resilience throughout the lifecycle of our solutions including design, test, manufacturing, distribution, support, and end of life. We design our solutions with built-in trustworthy technologies to enhance security and provide verification of the authenticity and integrity of Cisco hardware and software. We use a secure development lifecycle to make security a primary design consideration—never an afterthought. And we work with our partner ecosystem to implement a comprehensive Value Chain Security program to mitigate supply chain risks such as counterfeit and taint.
Trustworthy Networks Are More Important Than Ever
It wasn’t that long ago that most enterprise resources resided in heavily-protected on-premises data centers surrounded by vigilant layers of security. Trust was placed in securing the computing environment itself—the physically isolated data center that only trained and vetted personnel could physically access and the network that strictly controlled connections from enterprise endpoints.
Now all the physical and virtual security layers built into data centers that we relied for years have changed. Data “centers” are wherever the data and applications are located—in public clouds, SaaS providers, branch sites, and edge compute and storage. Every employee uses mobile devices—some of which are personally owned—for accessing and manipulating corporate resources. Virtualized workloads move among cloud or IaaS providers to improve responsiveness to a workforce in constant motion. Branch sites quickly need new direct internet connections to work with cloud applications. Global black swan events cause enormous shifts as whole populations rapidly move to working remotely. The attack surface has grown exponentially, literally overnight.
Another evolutionary and significant change in the attack surface results from transformations in the basic architecture of router and switch hardware. Just a decade or so ago, most routers and switches that formed the foundation of corporate and world-wide networks were built with proprietary silicon, hardware, and specialized operating systems. Back then, hackers were content to focus on the population of billions of x86-based and Windows-operating PCs and Linux servers, taking advantage of the same vulnerabilities and exploitive toolsets to invade and infect them. They largely ignored routers and switches because they required specialized tools to attack vendor-specific vulnerabilities—too much work for threat actors with such a target-rich environment of x86 machines to troll.
Hackers Turn to the Network for Access
But the choice of targets changed as hackers increasingly turned their attention to attacking enterprise data sources spread over distributed data centers, cloud platforms, and a mobile workforce. The opportunity for attacks became more prevalent as routers and switches gradually standardized on commodity hardware, Linux, and modules of open source code. Some enterprises and cloud providers are even trying white box and bare metal boxes to shave a few dollars off data center networking CAPEX. However, it’s the shared x86 and ARM code, Linux, and open source code that draws the attention of hackers, providing a potential foothold into corporate data-rich networks. By focusing on gaining access to network devices through common attack routes, malevolent hackers may be able to steal security keys and IP addresses among other sensitive data, opening more pathways into the network and the treasure-troves of corporate data. This attack vector can be particularly insidious when open source code that is integrated into the Network Operating System has Common Vulnerabilities and Exposures (CVEs) that go unpatched, offering hackers convenient documented doorways.
Own Networking Devices, Own the Traffic.
As network components became a more attractive target for hackers and nation-state spies looking for access to corporate resources, the sophistication of attacks increases to not only gain control over routers, but to hide the evidence of infiltration. One particular advancement is the development of persistent malware that can survive hard resets because the malicious code alters the BIOS or even the lower level boot loaders to gain control of the root. Without built-in protection at the hardware root layer that prevents altered low-level code from loading and infecting the entire OS, detecting and eliminating these persistent malware infections is next to impossible. An infected networking device—router, switch, access point, firewall—becomes an open gateway to all the network traffic.
With the multitude of dangers constantly testing the gateways of networks, there should be no such concept of “implicit trust”. At the core of the defensive network is the principle of proven trustworthy hardware and software working in conjunction to protect network devices from attack. By building on a foundation of trustworthy networking components and secure software, the connected sources of data and applications distributed through cloud services, branch sites, and a mobile workforce are also protected.
What are the fundamentals of building trusted network systems and what should CIOs and CSOs be asking of their vendors?
What CIOs and CSOs Need to Ask About Trustworthiness
It’s interesting to examine the paradox of enterprise CIOs and CSOs insisting that the hardware and operating systems of data center components are verifiably trustworthy while relying on a network fabric based on commodity components. To secure critical data and application resources, trust must permeate the hardware and software that run networks from the data center to campus to branch and cloud. Needless to say, that covers a very large territory of intertwined components, each of which must reinforce the trustworthiness of the complete network. Some of the major points to research when selecting network vendors and choosing components are:
◉ Is the network hardware authentic and genuine?
◉ What is the origin of all the software installed, including BIOS and open source modules?
◉ Have components or installed software been modified while the unit was in transit from manufacturing to customer?
◉ Are network devices running authentic code from the vendor?
◉ Are there any known CVEs with the software, including any open source modules?
◉ Are software and security patches available and applied?
◉ Is enterprise sensitive information, such as encryption keys, stored on network devices protected?
To ensure trust in a network, security-focused processes and technologies must be built into the hardware and software across the full lifecycle of solutions. That high level of engineering and supply chain control is very difficult to accomplish on low-margin, bare metal hardware. If the choice comes down to savings from slightly less costly hardware versus increase in risk, it’s worthwhile remembering the average cost of a single stolen record from security breaches is $155 (in the U.S.), while the cost of the loss of customer trust and theft of intellectual property is incalculable.
Building a Chain of Trust
A Chain of Trust should be built-in, starting with the design, sourcing of parts, and construction phase of both hardware and root-level software. It continues throughout the Secure Development Lifecycle (SDL), all the way to end of life for secure disposal of routers and switches, which can have sensitive data still stored in memory.
Security Embedded in Hardware
Hardware engineers must have an overriding mindset of security starting from sourcing of parts from reliable and certified manufacturers; designing secure boot strategies using Secure Unique Device Identifiers (SUDI) embedded in Trust Anchor modules (TAm); and Root of Trust tamper-resistant chips for secure generation and storage of cryptographic key pairs.
The SUDI is an X.509v3 certificate with an associated key-pair protected in hardware. The SUDI certificate contains the product identifier and serial number and is rooted to the Cisco Public Key Infrastructure. This identity can be either RSA- or ECDSA-based. The key pair and the SUDI certificate are inserted into the TAm during manufacturing so that the private key cannot be exported. The SUDI provides an immutable identity for the router or switch that is used to verify that the device is a genuine product. The result is hardware that can be verified as authentic and untainted, maximizing the security of the traffic flowing through the network.
Secure Software Development Lifecycle
Software developers must strictly follow Secure Development Lifecycle guidelines for coding the network operating systems with a combination of tools, processes, and awareness training that provides a holistic approach to product resiliency and establishes a culture of security awareness. From a trust perspective, the SDL development process includes:
◉ Product security requirements
◉ Management of third-party software, including open source code
◉ Secure design processes
◉ Secure coding practices and common libraries
◉ Static analysis
◉ Vulnerability testing
Security and Vulnerability Audits provide assurance that as problems are uncovered during the software development and testing cycles, they cannot be ignored. Ideally, the audit team reports not to engineering management but to the office of CSO or CEO to ensure that problems are completely fixed or the release red-lighted until they are remediated. This is an example of a culture of trust that permeates across functional departments all the way to the C-level—all in service of protecting the customer.
Download Verified Software Directly to Customer Controllers
Once trust is established for the development and release of networking software, the next step is to ensure that the software images delivered to customers’ controllers are original and untainted. To make a downloadable image trustworthy, it can be protected with a hash created with SHA-512, encrypted with a private key, and combined with the software image as a digital signature package. That image is downloaded directly to customers’ network controllers. The controllers use a matching public key to decrypt the digitally-signed hash and image package to verify nothing has been altered from the original image. Only software images that pass this critical test can be loaded on devices, preventing any malicious code from booting on a controller, router, or switch. This step ensures that only the vendor’s approved and unaltered code is running on network devices.
Reporting on CVEs Is Only One Way to Measure Solution Maturity
As previously discussed, open source and third-party code and APIs are commonly integrated into all modern network operating systems. Likewise, the core OS for network devices is commonly based on open source Linux kernels. All of these externally-developed code modules need to be certified by their creators and tested by neutral third-party organizations as well as the ultimate end-use vendor. As the number of products, software packages, and connected devices in networks continue to rise, it’s inevitable that more security vulnerabilities will also increase. Ironically perhaps, one reason for the increase in Common Vulnerabilities and Exposures (CVEs) is that the industry is getting better at finding and reporting them.
To ensure trustworthy networking, vendors must commit to Coordinated Disclosure, starting with a system for tracking every piece of open source and third-party code incorporated into network products. In addition, every CVE discovered either internally or externally needs to be tracked and fixed by a responsible entity in the trusted value chain of partners and suppliers. Customers, partners, and researchers need to be consistently notified of every recently discovered and fixed CVE and the potential for risk. Transparency improves customer security. That’s why at Cisco we report and provide fixes for CVEs that are found by our own internal testing, our customers, and third-party researchers.
A Third-Party Software Digitization (TPSD) program is another example of a corporate-wide initiative for managing third party software—including royalty bearing commercial software as well as free and open source software. The system automates the tracking and tracing of “Where Used, Where Is, Where Distributed” for compliance, contributions, security, quality, and customer accountability. A TPSD ensures that when a CVE is uncovered in open source or third-party code, the exact usage of the code is immediately known so that fixes can be applied in every instance.
As with the Security and Vulnerability Audit team, a Product Security Incident Response Team (PSIRT) that is independent from engineering is critical to keeping an unbiased watchful eye on all internally and externally developed code. PSIRT members continuously monitor open source and third-party code used in NOSs for reported CVEs. In addition, customers and researchers need a documented method of reporting to PSIRTs any CVEs that they discover. In turn, vendors need a transparent process for reporting on fixes and patches and level of severity back to customers, partners, and security researchers.
An important way to collaborate with industry peers on CVEs is through the Forum of Incident Response and Security Teams (FIRST) organization, which authors the PSIRT Framework. The Framework identifies core responsibilities of PSIRT teams, provides guidance on how to build capabilities to investigate and disclose security vulnerabilities and their remediations to customers in a transparent manner. As you consider the trustworthiness of your networking ecosystem, it’s worthwhile considering your vendors involvement with organizations like FIRST and the Industry Consortium for Advancement of Security on the Internet (ICASI)—which are strong indicators of a fundamental commitment to transparency.
Building in Trust Throughout the Product Lifecycle
All the processes and capabilities I have described in this post are integrated into Cisco’s hardware and software development processes. Cisco embeds security and resilience throughout the lifecycle of our solutions including design, test, manufacturing, distribution, support, and end of life. We design our solutions with built-in trustworthy technologies to enhance security and provide verification of the authenticity and integrity of Cisco hardware and software. We use a secure development lifecycle to make security a primary design consideration—never an afterthought. And we work with our partner ecosystem to implement a comprehensive Value Chain Security program to mitigate supply chain risks such as counterfeit and taint.
Posts
0 comments:
Post a Comment