How to put workload scanning into practice in Kubernetes (and how AI can help)

Kubernetes is used by some of the world’s biggest companies, including Google, Spotify, Tinder, and Airbnb, mostly because it can reduce costs and enable very flexible scaling. 

In many ways, it’s the ideal platform for the distributed cloud infrastructure we all rely on today. However maintaining the security of your k8s workloads requires special attention. As we’ll explore, workload scanning is essential, and AI tooling can help.

What is Kubernetes (k8s)?

Kubernetes (k8s) is a powerful open-source platform for deploying and managing containerized applications. Although Google originally developed k8s, the company released it into the wild. 

K8s has now become a popular open-source platform for container orchestration, managed by the Cloud Native Computing Foundation (CNCF). Today, more than 5 million developers use k8s to deploy and manage applications. 

The wider implications of a distributed architecture

An important feature of cloud computing and cloud application platforms (like k8s) is the resilient nature of their distributed infrastructure. Kubernetes, for example, runs containerized applications across a ‘cluster’, which is a collection of machines (physical or virtual), called nodes. It gets a little more in-depth once you start looking at the different node types found in a cluster, but for now it’s enough to understand that the distributed architecture is a core part of k8s’ advantages, and its weaker points too.

K8s makes containerized applications easy to scale when demand fluctuates, which is why they’re used by large companies for services like video streaming, or where data processing can oscillate over short periods. You can also easily extend functionality, and roll-back changes when issues are detected.

Containerized applications in k8s are highly portable, and have a secure design that isolates them from each other and the operating systems they run on. But this is a lot less secure than you might think. 

Why workload scanning is 100% necessary

The same distributed architecture that makes k8s so versatile and resilient, also makes it challenging to manage. 

There is very little visibility over your workloads, and the complexity of these distributed systems (which is a product of its extensibility and flexibility) means that it’s also harder to keep track of potential vulnerabilities.

Many people think of k8s as being ‘secure by design’, because the Linux-based containers are separated from the OS. In theory, if a cluster is breached an attacker can only access the containers on it and not the entire infrastructure. Even then, each container would need dedicated effort to gain access. 

However, this misses the point, and ignores a major security flaw: the majority of cloud attacks involve the use of credential theft – allowing cybercriminals to breeze in, just like they were an authorized developer. 

In this situation, the ‘secure design’ doesn’t help, because the hackers have the keys. The only way to protect your systems is by actively and constantly scanning workloads.

Another reason for workload scanning is that the increased ‘surface area’ of your workloads makes them more vulnerable to malicious attacks. The open-source nature of k8s means that third-party components can easily become misconfigured, unsupported, out of date, or prone to ‘known vulnerabilities’. 

A small update could lead to a massive issue, and supply chain attacks are becoming more common too. Even when vulnerabilities are known, it’s hard to detect if you’re affected without taking active measures. This is why workload scanning is essential for maintaining the security of your k8s workloads.

How can you secure your Kubernetes workloads?

There are several tactics for securing k8s workloads. For example, you can monitor IP addresses for user access to clusters, and Kubernetes API calls for unusual activity. 

Identity and Access Management (IAM) is also an important part of securing cloud infrastructure. And, of course, the right Cloud Security Posture Management (CSPM) tool can help ensure cloud security policies are aligned across your infrastructure.  

Due to the complex nature of k8s workloads, performing these checks manually would be wholly ineffective and time-consuming. For this, you need a powerful toolkit that meets all your requirements.

Tools for k8s workload scanning

Thanks to the huge community supporting k8s, there are many tools available for securing workloads. The one drawback is that they tend to offer fragmented functionality, so it’s common to use a variety of tools for different purposes. For example:

Kubescape – Used for scanning clusters and highlighting non-compliant YAML files.

Datree – Analyzes manifest files for misconfigurations.

Trivy – Scans workloads for vulnerabilities, using known CVEs for reference.

Google Kubernetes Engine – Can scan workloads for known vulnerabilities within the Security Posture Dashboard (doesn’t work for Windows Server containers though).

And there are many more tools too. However, with multiple dashboards and often limited coverage it’s still less than ideal, and it’s hard to keep an overview as a result. There are also issues with multicloud environments, which might require a bigger collection of tools to ensure you’re totally covered.

It’s also important to have something that can monitor behavior, and flag anything that deviates from ‘normal’ access patterns. For this, you need to use more sophisticated security tooling that leverages machine learning and ‘smart’ algorithms or artificial intelligence (AI).

AI tooling is one of the most interesting developments in securing cloud environments. It is being used to help overcome the shortfall in k8s expertise, and it can improve the overall performance of your applications too.

Using AI tooling to scan k8s/AWS workloads and more

There are some interesting AI tools already available for developers looking to save some time and resources while enhancing their overall security.

CastAI, for example, provides a platform for automated cost management and performance optimization, but it can also check for configuration issues and scan for vulnerabilities.

Likewise, a relatively new tool is k8sGPT, which uses a language model to allow ‘natural language’ interaction with your k8s infrastructure, and this uses connectors to integrate with other tools like Trivy. So, you could build your own security toolkit around a management tool like this.

This kind of option might be quite appealing if you’re starting out with Kubernetes and want to make some progress quickly without learning how to use it the ‘old-fashioned way’.

One tool that can’t be ignored, though, is AWS GuardDuty, which is also AI powered, and can give you the visibility you need over all your AWS accounts.

How AWS GuardDuty helps with workload scanning and much more

Amazon provides a powerful Security Hub that can really help with securing your workloads, and a key part of this is Amazon GuardDuty, which can be used in combination with Amazon Inspector.

Amazon Inspector is a powerful agent-based vulnerability scanner, which can cover pretty much everything the above tools can offer. It looks for things like misconfigurations and CVEs. We use this as part of our security toolkit, in combination with our own custom code.

By contrast, Amazon GuardDuty is an agentless ‘AI-powered’ automated tool that can detect threats across your entire AWS infrastructure, based on unusual usage and known threat IP addresses. It’s totally inobtrusive, and highly effective.

What’s great about both of these tools is that they’re easily integrated with the AWS Security Hub. This means that you have a high level of visibility across your entire infrastructure with a single dashboard. Whenever an issue is found, it’s sent to the Security Hub as a ‘finding’, with recommended action included.

How does Amazon GuardDuty use AI to protect workloads (and more)?

GuardDuty uses machine learning to detect anomalous behavior, using a combination of threat intelligence feeds and behavioral modeling that senses malicious intent via behavior. Whenever a threat is detected, you can also create automated mitigation responses, so remedial action can happen even when you’re asleep.

While Amazon Inspector is hard at work continuously scanning your workloads and container images, GuardDuty uses its AI-powered skills to detect active threats in your systems, looking for malware, weird access patterns, or anything else ‘out of the ordinary’, and letting you know if something is concerning. The great thing about AWS GuardDuty is that it can cover all your AWS Accounts, as well as instances, container workloads, serverless workloads, databases, storage, and users.

Cloud security: Is one tool ever enough?

It would be fantastic if one tool could offer all the security requirements your cloud infrastructure needs. Amazon GuardDuty certainly gets very close to this, but it still needs to be used intelligently and must match your requirements (with a bit of customization).

Want to learn more about this important topic? Check out our webinar, in which we discuss the most effective (and efficient) way you can perform workload scanning for k8s/AWS environments.

Have questions, or want to get in touch? Contact us here, and let’s talk about how we can make your project fly.