B4.run

Menu

Site

Documentation

Kubernetes Sandbox

Use kubernetesSandbox when the B4.run runtime already runs in Kubernetes and each Agent Protocol thread needs an isolated Pod-backed workspace. It isolates each thread in a container, not a microVM. Before production, you need a dynamic volume provisioner, a compatible sandbox image and enough Pod and PVC quota. If NetworkPolicy is part of your security boundary, you also need a policy-enforcing CNI.

The application process needs Kubernetes API credentials to create and exec into sandbox workloads. The sandbox Pods don't: B4.run sets automountServiceAccountToken: false on every provider-managed Pod.

Install the sandbox infrastructureCopy link to section: Install the sandbox infrastructure

The b4-sandbox-infra chart creates the target namespace, the orchestrator ServiceAccount and RBAC, a default-deny egress backstop, resource controls, Pod Security Standard labels and a PVC reaper. Keep its Helm release, the B4.run app and application credentials in the separate b4-app management namespace. Sandbox resources live in b4-sandboxes.

For a fresh release, prepare a values file that lists every planned application ServiceAccount. For an existing release, export its effective values first. Either way, orchestrator.subjects must contain the complete intended subject list: preserve every existing item and append each application ServiceAccount as a cross-namespace subject. Helm replaces arrays, so never write a guessed numeric subject index. Follow Kubernetes deployment for the complete ordered workflow.

Install the infrastructure once per cluster or environment:

bash
helm upgrade --install b4-sandbox-infra \
  oci://ghcr.io/cacheplane/charts/b4-sandbox-infra \
  --namespace b4-app \
  --create-namespace \
  --values b4-sandbox-infra-values.yaml

The Helm release namespace does not control where sandbox workloads run. The chart's namespace.name defaults to b4-sandboxes. Review the values before you change namespace ownership or disable a control.

Configure the providerCopy link to section: Configure the provider

Point the provider at the namespace the infrastructure chart manages:

b4.config.ts
import { config } from "@b4run/cli"
import { kubernetesSandbox } from "@b4run/sandbox"
 
export default config({
  sandbox: {
    provider: kubernetesSandbox({ scope: "my-app",
      image: "node:24-slim",
      namespace: "b4-sandboxes",
    }),
    network: { mode: "deny" },
    resources: { memoryMb: 512, cpus: 1, diskGb: 2 },
  },
})

storageClass and startupTimeoutMs are optional. resources.diskGb becomes the PVC storage request, so confirm the selected StorageClass can provision it.

scope is required and identifies the application and environment within the provider. The provider hashes scope and the exact logical thread ID into stable resource names, including NetworkPolicy selectors. Keep scope stable across deployments. Different scopes address different resources in the same namespace, but they do not replace tenant authorization, RBAC, or namespace trust boundaries. Sharing scope also does not coordinate concurrent replicas. See Application scope for the breaking storage-name change.

The provider replaces the image entrypoint with sleep infinity and runs shell and filesystem operations through sh -c. The image must provide a POSIX sh, a sleep implementation that accepts infinity, and the core utilities used by the filesystem backend (cat, mkdir, dirname, ls, realpath, stat, rm, and touch). It also needs timeout when resources.timeoutMs is set. The hardened default runs as numeric UID/GID 1000:1000. The image doesn't need an /etc/passwd entry, but its executables and filesystem permissions must work for that identity. When they don't, set a compatible UID/GID through security.runAsNonRoot.

Wire application RBACCopy link to section: Wire application RBAC

The app Pod's ServiceAccount must be a subject of the infrastructure chart's b4-orchestrator RoleBinding. That Role covers only the provider's Pod, Pod exec, PVC and NetworkPolicy operations, with no Secret access.

Add the management-namespace app ServiceAccount to the complete orchestrator.subjects list before you deploy the app, so the app is never running without authorization. Follow Kubernetes deployment for the ordered RoleBinding and application-chart commands. Keep these three values aligned:

  • b4-sandbox-infra's namespace.name
  • kubernetesSandbox({ scope: "my-app", namespace: "b4-sandboxes" })
  • the application chart's informational sandboxNamespace

Changing sandboxNamespace alone does not move workloads or change RBAC.

Pod and workspace lifecycleCopy link to section: Pod and workspace lifecycle

Each distinct provider resource key receives one keeper Pod and one ReadWriteOnce PVC mounted at /workspace.

  • acquire() creates the PVC and Pod or reattaches the live pair.
  • release() deletes the Pod and retains the PVC for the next turn.
  • destroy() deletes the Pod, per-thread NetworkPolicy, and PVC.

Idle reap and server shutdown both call release(), so their Pods go away and their PVCs remain unreferenced. With the chart reaper enabled, that storage has a limited life. The reaper has no B4.run thread metadata and considers every unreferenced B4.run PVC eligible.

The reaper runs hourly by default. Each run does two things:

  • It marks an unreferenced PVC that has no valid marker.
  • It clears the marker from a PVC it observes referenced by a Pod.

A scheduled reaper run deletes a currently unreferenced PVC when its stored marker is older than reaper.ttlHours (default 168). A reattachment resets the marker only if a reaper run observes the PVC referenced by a Pod. So a short reattachment entirely between scheduled runs can leave an old marker in place, and after the Pod is released, the next run can delete the PVC after recent use.

This includes a still-live thread released by idle reap or shutdown. If its PVC is deleted, the next acquire() provisions a replacement and starts with an empty workspace. If you need thread workspaces to survive longer, tune reaper.ttlHours or disable the reaper and own cleanup another way.

The provider applies non-root execution, dropped capabilities, no privilege escalation, RuntimeDefault seccomp, and writable scratch mounts around its read-only-root default. The chart enforces Restricted Pod Security Standards by default. Baseline is needed only for workloads or settings that actually violate Restricted admission, such as security.runAsNonRoot: false. Setting security.readOnlyRootFilesystem: false alone remains Restricted-compatible and does not require lowering the namespace to Baseline. The chart never downgrades automatically, and the cluster admission controller decides whether an image and policy are accepted. Validate your image under the profile you enforce.

ResourceQuota caps aggregate namespace consumption. LimitRange supplies CPU, memory, and ephemeral-storage defaults. Monitor reaper deletions alongside Pod and PVC lifecycle so you can tell storage-retention failures apart from application behavior.

NetworkPolicy and DNSCopy link to section: NetworkPolicy and DNS

The infrastructure chart installs a default-deny egress backstop for Pods labeled app.kubernetes.io/managed-by=b4, with UDP/TCP port 53 allowed to kube-system. So deny mode is not zero egress: DNS remains allowed, along with the DNS-tunneling risk that comes with it.

NetworkPolicy objects have no effect unless the cluster uses a policy-enforcing CNI such as Calico or Cilium. The chart does not install or configure a CNI. Test DNS and policy behavior on your actual cluster.

The chart backstop and provider policy are additive. With networkPolicy.defaultDenyEgress=true, a provider setting of network: { mode: "allow" } cannot override the chart policy and does not reopen egress. In allow mode, the provider emits no per-thread NetworkPolicy and does not enforce denylist. The chart backstop is the only B4.run-supplied egress restriction. If you disable that backstop without an operator-owned replacement, egress is open. In deny mode, the provider creates a per-thread policy with DNS and configured CIDR exceptions.

PID and resource exhaustionCopy link to section: PID and resource exhaustion

Kubernetes has no namespaced LimitRange or ResourceQuota field for process count. PID limits are a node/runtime concern: configure kubelet podPidsLimit (or its runtime equivalent) on nodes that schedule sandbox Pods. The chart does not supply PID limits, and security.pidsLimit is not enforced by the Kubernetes provider.

CPU, memory, ephemeral-storage, PVC count, and requested storage remain chart-managed controls. Monitor quota pressure, Pending Pods, failed volume provisioning, evictions and reaper activity. None of these failures shows up as model behavior.

Preflight and end-to-end verificationCopy link to section: Preflight and end-to-end verification

b4 check calls the provider's preflight(). It runs a SelfSubjectAccessReview for every Kubernetes API operation the provider can perform: create/get/delete Pods; create/get/delete PVCs; create/get pods/exec; and create/get/list/update/delete NetworkPolicies. It checks the complete set even when one review is denied or fails, then reports missing permissions, authorization-review failures and API transport failures separately. Only after every required permission is granted does it check NetworkPolicy enforcement; an unconfirmed policy-capable CNI produces a warning rather than a successful enforcement claim.

The compatibility command preflights storage selection but does not create a claim PVC. Dynamic RWO provisioning is a runtime prerequisite verified by the lifecycle, not proven by preflight.

The policy pins Kind v0.32.0, kubectl v1.35.6, and Calico v3.32.1 across these patch releases:

  • Kubernetes 1.34 (1.34.8)
  • Kubernetes 1.35 (1.35.5)
  • Kubernetes 1.36 (1.36.1)

Run the portable lifecycle against the cluster already selected in kubeconfig:

bash
pnpm verify:k8s:compat -- --target <1.34|1.35|1.36> --context <exact-context> [--storage-class <name>] [--keep-on-failure]

The target must match the server minor version, and the context must exactly match the current context. The command never switches contexts.

B4.run's Kind/Calico coverage does not certify managed Kubernetes services, other CNI implementations, or storage drivers.

Use both gated lanes as combined evidence. sandbox-k8s-e2e, enabled with B4_TEST_SMOKE_E2E=1, proves the built app and provider are wired together. It creates a Pod, PVC, and per-thread NetworkPolicy, executes as a non-root user, and removes the Pod and PVC on thread deletion. The sandbox-k8s-e2e lane does not test DNS or blocked egress.

The separate sandbox-k8s integration lane runs against kind with Calico and proves DNS resolution plus blocked egress, including the chart backstop around an allow-mode sandbox. Neither substitutes for verifying the StorageClass, CNI, DNS, admission, quotas, and cleanup settings in your production cluster.

Operational referencesCopy link to section: Operational references