Container and Kubernetes Hunting

Why Containers Need a Different Hunting Model

Every technique covered in Chapters 1 through 9 assumes a single operating system instance with its own kernel, its own process tree, and Sysmon or an EDR agent watching it directly. Kubernetes breaks that model in three specific ways that change what a hunter should look for and where.

Three Ways Kubernetes Breaks the Model

• Containers share a kernel. Unlike a hypervisor-based VM, a container has no hardware-enforced isolation boundary. A kernel vulnerability, a misconfigured capability, or a mounted host resource can let a process inside a container directly affect the host or every other container running on it. This is the entire premise of T1611, Escape to Host.
• The orchestrator is itself an API-reachable attack surface. The Kubernetes API server is a control plane that can create, delete, and exec into workloads across the entire cluster. A credential that grants API access (a stolen kubeconfig, an over-permissioned service account token) is functionally equivalent to domain admin in a Windows environment, except the "domain" is every workload the cluster runs.
• Workloads are ephemeral by design. A compromised pod might live for minutes before the orchestrator reschedules it. Traditional incident response assumptions (the compromised host is still there tomorrow for forensic imaging) do not hold. Hunting has to work from the audit trail and runtime telemetry, because the pod itself may already be gone.

Applying the Same ATT&CK Discipline

This chapter maps to the same ATT&CK-driven hypothesis discipline from Chapter 2, using the ATT&CK for Containers matrix published by the Center for Threat-Informed Defense, plus the container-relevant sub-techniques already present in TTPHUNT: T1053.007 (Container Orchestration Job) and T1059.013 (Container CLI/API). Where this chapter goes further is the cluster-level control plane: RBAC, the API server audit log, and container runtime behavior.

ATT&CK for Containers Overview

The Center for Threat-Informed Defense published ATT&CK for Containers as an extension to the Enterprise matrix, adding techniques specific to container and orchestration abuse rather than forcing container attacks into host-centric technique definitions that did not quite fit.

Five Anchor Techniques

Kubernetes Audit Log Hunting

The Audit Log as Primary Data Source

The Kubernetes API server can emit a structured audit log (audit.k8s.io/v1) for every request it processes, if an audit policy is configured. This is the single most important data source in this chapter: every kubectl command, every controller reconciliation, and every service account API call passes through the API server and can be captured here.

Core fields to build hunt queries around: verb (get, list, create, update, patch, delete, deletecollection), objectRef (the resource type, namespace, and name being acted on, including a subresource field for things like exec and portforward), user (the identity making the call, a human via SSO, or a service account), and sourceIPs.

Example Audit Log Entry

{
  "apiVersion": "audit.k8s.io/v1",
  "kind": "Event",
  "level": "Metadata",
  "verb": "create",
  "objectRef": {
    "resource": "pods",
    "namespace": "production",
    "name": "web-frontend-7c9f8d-xk2lp",
    "apiVersion": "v1",
    "subresource": "exec"
  },
  "user": { "username": "jane.doe@corp.com", "groups": ["system:authenticated"] },
  "sourceIPs": ["203.0.113.45"],
  "requestURI": "/api/v1/namespaces/production/pods/web-frontend-7c9f8d-xk2lp/exec?command=/bin/sh"
}

Three Hunt Patterns

These cover most of the value in this log source:

• Exec into production pods. Filter for objectRef.subresource: "exec" or "attach" against namespaces that should never need interactive debugging (production, especially anything handling regulated data). A legitimate on-call engineer execs in occasionally; a pattern of repeated execs across many pods from one identity in a short window looks like discovery or lateral movement, not debugging.
• Anonymous or unauthenticated API access. A user.username of system:anonymous succeeding against any resource indicates the API server (or an exposed component like the kubelet's read-only port) is reachable without authentication, a critical exposure regardless of what specific resource was accessed.
• RBAC self-escalation. Watch for create/update verbs against clusterrolebindings, rolebindings, or clusterroles performed by an identity that is not a documented cluster administrator. Binding oneself (or a controlled service account) to cluster-admin is the single most direct privilege escalation path in Kubernetes.

Runtime Detection with Falco

Why Falco Fills the Gap

The Kubernetes audit log tells you what was requested through the API. It has no visibility into what a process does once it is already running inside a container: a reverse shell spawned by an exploited application, a write to a sensitive host-mounted path, or a process reading the pod's own service account token to exfiltrate it. That gap is covered by syscall-level runtime security tooling, of which Falco (a CNCF graduated project) is the de facto standard.

Falco rules match on syscall events using a condition expression built from fields like proc.name, container.id, fd.name, and evt.type, reusable via macros and lists. The canonical example, "Terminal shell in container," illustrates the pattern every container runtime hunt should understand:

Example Falco Rule

- rule: Terminal shell in container
  desc: Detect a shell being spawned in a container
  condition: > spawned_process and container and shell_procs and proc.tty != 0
  output: > Shell spawned in container (user=%user.name
    container_id=%container.id container_name=%container.name
    shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)
  priority: WARNING

This single rule is disproportionately valuable because most production container images are built without an interactive shell need in mind: a shell spawning inside a minimal application container, attached to a TTY, is rarely a legitimate operational action and very frequently the first sign of a working remote code execution exploit.

More Falco Hunt Patterns

Container Escape and Privileged Pod Hunting

Why Pod Specs Matter More Than Exploits

Most real-world container escapes (T1611) do not require a kernel zero-day. They require a pod spec that was granted more access than the workload needed, deployed by a team optimizing for "it works" over least privilege. Hunting for escape risk is largely a matter of hunting the pod specification itself, before any exploit ever runs.

Escape-Enabling Pod Spec Settings

Hunting the Spec, Not the Log

Because these are static configuration facts rather than runtime events, the hunt is a periodic sweep, not a log query: enumerate every pod spec in the cluster (via kubectl get pods -o json across all namespaces, or an admission-controller policy engine's audit mode) and flag any workload carrying one of the settings above without a documented, reviewed exception.

The Docker socket mount is worth tracing end to end, since it is one of the least obvious of these paths:

Worked Example

Image and Registry Hunting

Why Image Provenance Matters

A container is only as trustworthy as the image it was built from. T1610 (Deploy Container) and T1612 (Build Image on Host) both center on getting an adversary-influenced image running, whether pulled from an untrusted registry or built directly against the runtime's API to skip registry-based scanning entirely.

Four Hunts to Run

• Hunt for images pulled from registries outside the organization's approved allow-list. A workload pulling directly from Docker Hub's public namespace, or from a registry that has never appeared in the cluster's image-pull history before, is a lead worth checking, especially for images with generic, imitation-prone names (nginx-official, redis-cache) that mimic legitimate ones.
• Hunt for images without a corresponding CI/CD provenance record. A mature pipeline signs or attests every image it builds. A running workload backed by an image with no matching build record, commit hash, or signature is either a manual deploy that bypassed the pipeline (a process gap worth closing) or something more concerning.
• Hunt for direct daemon/API build requests (T1612). A build request sent straight to the Docker or containerd API, rather than through the CI/CD pipeline's normal build step, bypasses every image-scanning gate that only inspects images at registry-push time.
• Watch for resource-consumption anomalies consistent with cryptomining. A generic-looking image that spikes CPU utilization to near 100% shortly after deployment, with no corresponding legitimate workload change, is one of the most common real-world outcomes of a compromised or malicious container image: financially motivated actors overwhelmingly prioritize compute theft over data theft in container environments, because it monetizes instantly and often goes unnoticed for weeks in a busy cluster.

Building Container and Kubernetes Hunt Hypotheses

Applying ABLE to Containers

The same ABLE structure and PEAK hunt-plan template from earlier chapters applies directly, with the Kubernetes audit log and Falco runtime alerts as the primary data sources in place of Sysmon.

Hunt Plan Template

HUNT PLAN
Analyst: [name]
Hypothesis: An adversary with a stolen or over-permissioned service account
  token used the Kubernetes API to exec into a running pod and enumerate
  secrets across namespaces it should not have access to (T1609, T1613,
  T1552.007)
ATT&CK: T1609 - Container Administration Command

Scope:
- Cluster: production-eks-01, all namespaces
- Time window: Last 30 days
- Data sources: Kubernetes API server audit log, Falco alerts, RBAC bindings

Expected evidence if hypothesis TRUE:
- Audit log: pods/exec subresource calls from a service account identity
  (not a human SSO user) outside its owning namespace
- Audit log: get/list against the secrets resource across multiple
  namespaces from the same identity in a short window
- No corresponding CI/CD job or documented debugging ticket for the activity

Expected evidence if hypothesis FALSE:
- All cross-namespace secret access traces to a documented, RBAC-scoped
  operator/controller with a legitimate multi-namespace function

Planned queries:
1. Filter audit log for objectRef.subresource == "exec" grouped by user.username
2. Cross-reference identities with type service-account against their
   owning namespace vs. the namespace of the objectRef
3. Filter audit log for verb in (get, list) against resource == "secrets",
   group by user, count distinct namespaces touched
4. Cross-reference flagged identities against the cluster's RoleBinding/
   ClusterRoleBinding definitions to confirm intended scope

Three Ready-to-Adapt Hypotheses

For a first pass at a container/Kubernetes hunt program:

• Privileged pod sweep: "A workload in this cluster is running with privileged: true, a sensitive hostPath mount, or hostPID/hostNetwork without a documented exception." (T1611). Test via a periodic pod-spec sweep, not a log query.
• RBAC self-escalation: "An adversary with limited namespace access created or modified a ClusterRoleBinding to grant themselves or a controlled service account cluster-admin." (T1078.004-adjacent, container-specific privilege escalation). Test against the audit log filtered on RBAC resource verbs.
• Cryptomining via unauthorized deploy: "An adversary used T1610 to deploy a container from an unapproved registry that consumes anomalous CPU/GPU resources shortly after creation." Test by correlating the audit log's pod-create events against a resource-utilization baseline per namespace.