Cybersecurity

Cloud-Native Security: Securing Kubernetes and Containers

Kubernetes 1.25 has just removed PodSecurityPolicy, and many clusters are running without a replacement. We walk through the layers of cloud-native security, the misconfigurations we find most often, and the controls every production cluster should have.

By EthicalHat Research · · 3 min read
Infographic: The four layers of cloud-native security

Containers and Kubernetes have become the default way to run new applications. They also introduce an attack surface that many security teams are still learning. In a typical cluster, a single misconfigured pod, an over-privileged service account or an exposed dashboard can turn one vulnerable application into control of every workload on the platform.

This is a good moment to review cluster security. Kubernetes 1.25, released on 23 August, removed PodSecurityPolicy (PSP), the built-in mechanism many clusters relied on to stop privileged pods. Clusters upgraded without a replacement lose that protection.

The four layers

The Kubernetes documentation describes cloud-native security as four nested layers, the "4Cs": Cloud, Cluster, Container and Code. Each layer depends on the ones outside it. A perfectly hardened container does not help if anyone on the internet can reach an unauthenticated API server.

Cloud (or data centre)

  • Keep the Kubernetes API server private, or restrict it to known IP ranges.
  • Give nodes minimal cloud IAM permissions, and block pod access to the instance metadata service where the cloud provider allows it.
  • Use managed node images and patch them regularly.

Cluster

  • RBAC: avoid cluster-admin bindings for humans and service accounts; scope roles to namespaces.
  • Admission control: enforce what pods are allowed to do (see below).
  • Network policies: default-deny, then allow only the traffic each service needs.
  • Secrets: enable encryption at rest for Secrets in etcd, or use an external secrets manager.
  • Audit logging: turn on API server audit logs and ship them to your SIEM.

Container

  • Build from minimal base images (distroless or Alpine) and scan them.
  • Run as a non-root user with a read-only root filesystem.
  • Drop all Linux capabilities and add back only what is needed.
  • Never run privileged: true or mount the host's Docker socket.

Code

Everything from our DevSecOps article applies: dependency scanning, SAST and no secrets in images.

The misconfigurations we find most often

In Kubernetes security audits this year, the same issues come up again and again:

  1. Service accounts with cluster-wide permissions. Often granted "temporarily" to get a Helm chart working. A compromise of that pod becomes a compromise of the cluster.
  2. Privileged pods and host mounts. Monitoring agents and CI runners are frequent offenders. hostPath: / or a mounted /var/run/docker.sock is equivalent to root on the node.
  3. No network policies. Every pod can talk to every other pod, including databases in other namespaces.
  4. Exposed dashboards and APIs. The Kubernetes dashboard, Kubelet read-only ports, or tools like Argo and Jenkins reachable from the internet. Tesla's 2018 cryptojacking incident started with an unauthenticated Kubernetes console.
  5. Secrets in environment variables and images, visible to anyone who can describe the pod or pull the image.

Replacing PodSecurityPolicy

Kubernetes now ships Pod Security Admission, stable in 1.25, which enforces the three Pod Security Standards (privileged, baseline and restricted) per namespace using labels:

yaml
apiVersion: v1
kind: Namespace
metadata:
  name: payments
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: latest
    pod-security.kubernetes.io/warn: restricted

A pod that satisfies the restricted profile looks like this:

yaml
apiVersion: v1
kind: Pod
metadata:
  name: api
  namespace: payments
spec:
  automountServiceAccountToken: false
  securityContext:
    runAsNonRoot: true
    seccompProfile: { type: RuntimeDefault }
  containers:
    - name: api
      image: registry.example.com/payments-api@sha256:4f1c...   # pinned by digest
      securityContext:
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true
        capabilities: { drop: ["ALL"] }
      resources:
        limits: { cpu: "500m", memory: "256Mi" }

For finer-grained rules, such as "images must come from our registry" or "every deployment needs an owner label", a policy engine such as Kyverno or OPA Gatekeeper complements the built-in standards.

Default-deny networking

Start every namespace with a policy that blocks all traffic, then allow what is needed:

yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
  namespace: payments
spec:
  podSelector: {}
  policyTypes: [Ingress, Egress]

Remember to allow DNS egress. It is the most common reason a new default-deny policy "breaks everything".

Runtime visibility

Preventive controls reduce risk; runtime detection catches what gets through. Tools such as Falco watch system calls for suspicious behaviour, for example a shell spawned in a container, a new binary executed or a read of a service account token. Combined with API server audit logs, they give your SOC visibility into what is happening inside the cluster.

A quick self-assessment

  • Can anyone on the internet reach your API server?
  • Which service accounts can create pods or read secrets cluster-wide?
  • Which pods run privileged, as root or with host mounts?
  • Does every namespace have a default-deny network policy?
  • What replaced PodSecurityPolicy on your 1.25 clusters?

If any of those questions took more than a few minutes to answer, our Kubernetes security and container security will give you a complete, prioritised picture.

EthicalHat Research

The EthicalHat research team: penetration testers, SOC analysts and engineers writing about security, software engineering and AI.

Want an expert opinion on your own systems?

Email info@ethicalhat.com or call +1-844-838-4422. Our security engineers are happy to talk it through.