Skip to content

Custom Helm Chart

You can create your own Helm chart to be used by the k8s_sandbox package instead of the built-in agent-env chart.

Chart requirements

  • Have one container per Pod, or if multiple containers are required on a Pod, the first container must represent the sandbox environment in which operations will be performed by Inspect.
  • Accept a top-level annotations object in its values schema (will typically include annotations.inspectTaskName). Apply these annotations to Pods and other resources.
  • In the metadata section of each Pod, set the app.kubernetes.io/instance label to the name of the Helm release i.e. .Release.Name. This is an 8 character string supplied by the k8s_sandbox package. This is used by the k8s_sandbox package for discovering the sandbox environments.
  • In the metadata section of each Pod which represents a sandbox environment, set the inspect/service label. The value is used for naming the sandbox environments presented to Inspect (e.g. "default" or "victim"). Pods which do not have an inspect/service label set are not presented to Inspect as sandbox environments.

Be aware that the release name generated by the k8s_sandbox package may begin with a digit, so ensure that any resource names which include this result in a valid Kubernetes resource name.

A basic chart's template file might look like this:

apiVersion: v1
kind: Pod
metadata:
  name: my-custom-chart-pod-{{ .Release.Name }}
  labels:
    app.kubernetes.io/name: {{ .Chart.Name }}
    app.kubernetes.io/instance: {{ .Release.Name }}
    inspect/service: default
  annotations:
    {{- toYaml $.Values.annotations | nindent 4 }}
spec:
  containers:
  - name: default-container
    image: python:3.12-bookworm
    command: ["sleep", "infinity"]

Using your chart

Note

Only local charts (i.e. ones available in a local directory) are currently supported. In future, we hope to support charts from a remote repository.

To use the custom Helm chart, pass a SandboxEnvironmentSpec containing a K8sSandboxEnvironmentConfig as the sandbox parameter to the Task or Sample constructor.

Task(
    ...
    sandbox=SandboxEnvironmentSpec(
        "k8s",
        K8sSandboxEnvironmentConfig(
            chart="path/to/your/chart",
            values=Path("path/to/your/values.yaml"),
        ),
    ),
)

The values can be None to use the chart's default values.

Chart readiness

The k8s_sandbox package uses the --wait flag when installing the Helm chart so won't begin the eval until the chart is deemed ready by Helm. If this is not sufficient, consider using a Helm post-install hook which waits for a condition to be met.