Skip to content

Kubernetes Support

To use LPUs in an existing Kubernetes cluster, prepare the driver and toolkit on the nodes, add ha-lpu-operator to the cluster, and request the number of LPUs in your workloads.

Location What to add or change
LPU nodes Install hyperaccel-lpu and configure CDI in the runtime
Cluster The ha-lpu-operator Helm chart
Existing Pods and Deployments Add hyperaccel.ai/lpu to the container's resources.limits

Preparing the nodes and cluster

In a cluster running Kubernetes 1.31 or later, check the following on each node with an LPU:

  • Is hyperaccel-lpu installed so the driver and toolkit are available?
  • Is the existing containerd or CRI-O runtime configured to read CDI specifications? See Container support.

If cert-manager manages the webhook certificates, it must already be available in the cluster. If you provide your own certificate, use the settings below.

Adding the operator to the cluster

Install ha-lpu-operator if it is not already present in the cluster.

The chart and the component images are pulled with the account HyperAccel issued. Log Helm in first, and create the Secret the component images are pulled with. The chart looks it up by the name harbor-secret.

1
2
3
4
helm registry login public.hyperaccel.net
kubectl create namespace bertha-system
kubectl -n bertha-system create secret docker-registry harbor-secret \
  --docker-server=public.hyperaccel.net --docker-username=<user> --docker-password=<password>
1
2
helm install ha-lpu-operator oci://public.hyperaccel.net/hyperaccel-public/charts/ha-lpu-operator \
  --version 0.1.8 --namespace bertha-system --create-namespace

The chart creates the operator and a default BerthaCluster. The operator manages the LPU components, including device-plugin. The chart includes NFD (node-feature-discovery). If it is already installed, check its status rather than installing it again.

1
2
3
kubectl get berthacluster -n bertha-system
kubectl get nodes -L hyperaccel.ai/lpu.count,hyperaccel.ai/lpu-max-contiguous
kubectl describe node <node_name> | grep hyperaccel.ai/lpu

Once BerthaCluster is Ready and the node advertises the hyperaccel.ai/lpu resource, Pods can request LPUs. The validator checks the node's driver and device files before allowing device-plugin and feature-discovery to run.

Using existing certificate settings

To change the defaults, update berthaCluster.spec in the Helm values. Use the webhook certificate path that fits your environment:

Setting Certificate
webhook.tlsSecretName A Secret you provide
webhook.certManager.issuerRef The specified cert-manager Issuer
Neither (default) The operator creates a self-signed ClusterIssuer

Using cards in a Pod

In spec.containers[] of an existing Pod or spec.template.spec.containers[] of a Deployment, add hyperaccel.ai/lpu to resources.limits. Keep the existing CPU and memory settings.

1
2
3
resources:
  limits:
    hyperaccel.ai/lpu: 1

To check access to a single card using the SDK image, use this Pod. The namespace you create it in needs harbor-secret too:

lpu-test.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
apiVersion: v1
kind: Pod
metadata:
  name: lpu-test
spec:
  containers:
    - name: sdk
      image: public.hyperaccel.net/hyperaccel-public/hyperaccel-sdk:v0.0.6
      command: ["python3", "-c", "import hart; print(hart.get_device_count())"]
      resources:
        limits:
          hyperaccel.ai/lpu: 1
  imagePullSecrets:
    - name: harbor-secret
  restartPolicy: Never
1
2
3
kubectl apply -f lpu-test.yaml
kubectl wait --for=jsonpath='{.status.phase}'=Succeeded pod/lpu-test --timeout=180s
kubectl logs lpu-test      # 1

The connection check passes when the Pod completes and prints a card count of 1.

Serving with vLLM

In your existing Deployment, change the vLLM container's image, execution options, and LPU request. The example below shows only the fields to change in spec.template.spec.containers[]. To pull the image, add harbor-secret to spec.template.spec.imagePullSecrets and create the same Secret as above in that namespace.

1
2
3
4
5
image: public.hyperaccel.net/hyperaccel-public/vllm-hyperaccel:0.0.6
args: ["vllm", "serve", "Qwen/Qwen3-8B", "--max-model-len", "2048", "--no-enable-prefix-caching", "--no-async-scheduling"]
resources:
  limits:
    hyperaccel.ai/lpu: 1

Set the model and context length to suit your existing workload. To split the model across two LPUs, set both hyperaccel.ai/lpu: 2 and --pipeline-parallel-size 2. For serving options, see the vLLM configuration guide.

Check these points in your existing operational configuration:

Existing setting What to check when using LPUs
Model cache Use the existing volume. If you reference a PVC, it must be available in the same namespace. Check storage constraints to determine whether it can also be used on another node
Startup and readiness checks Allow for model downloading and kernel preparation in startupProbe. Startup check failures can trigger a restart; readinessProbe failures remove the Pod from traffic but do not themselves restart it
Update strategy If the existing Pod holds all LPUs, a new Pod may wait for resources. Updates without downtime require spare LPU and storage capacity for the new Pod

To replace the existing Pod by stopping it first when no spare LPU is available, add the following under the Deployment's spec. The service will be unavailable until the new model is ready.

1
2
strategy:
  type: Recreate

Node labels

feature-discovery adds these labels. Use them directly in nodeSelector or affinity rules.

Label Example Meaning
hyperaccel.ai/lpu.present true The node has cards
hyperaccel.ai/lpu.count 8 Card count
hyperaccel.ai/lpu-max-contiguous 2 Maximum number of cards under a single PCIe switch

For Pods requesting two or more cards, the webhook adds a preference for nodes whose lpu-max-contiguous is at least the requested card count. When cards share a switch, communication between them stays within that switch. This is a preference, so the Pod can still run if no node matches.

Metrics

Add ha-lpu-exporter as a scrape target in your existing Prometheus Operator setup. The exporter serves /metrics from Pods in bertha-system on the port named metrics (9401).

Set metadata.namespace and metadata.labels in the PodMonitor below to match your existing Prometheus podMonitorNamespaceSelector and podMonitorSelector. The example values monitoring and release: kube-prometheus-stack are for environments using those selection criteria. spec.namespaceSelector points to bertha-system, where the exporter runs. If you changed the BerthaCluster name, update app.kubernetes.io/instance to the actual value as well.

ha-lpu-exporter-podmonitor.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: ha-lpu-exporter
  namespace: monitoring
  labels:
    release: kube-prometheus-stack
spec:
  namespaceSelector:
    matchNames:
      - bertha-system
  selector:
    matchLabels:
      app.kubernetes.io/name: exporter
      app.kubernetes.io/instance: ha-lpu-operator
  podMetricsEndpoints:
    - port: metrics
      path: /metrics
      interval: 15s
1
kubectl apply -f ha-lpu-exporter-podmonitor.yaml

Include this PodMonitor without overwriting the selection criteria for existing scrape targets. In Prometheus Targets, check that the exporter is UP, then query lpu_device_up. UP indicates a successful scrape; lpu_device_up indicates whether a card responds.

Metric Meaning
lpu_device_up Whether the card responds
lpu_device_wedged Whether the card is stuck
lpu_fw_errors_total Firmware error count
lpu_temperature_celsius Temperature
lpu_power_watts Power
lpu_dram_used_bytes Card memory usage
lpu_pcie_link_degraded Whether the PCIe link is running below its maximum speed
lpu_pod_lpu_count Number of cards allocated per Pod

Troubleshooting

Symptom What to check
Pod is Pending Events in kubectl describe pod. If the node has no hyperaccel.ai/lpu, check kubectl get berthacluster -n bertha-system and the validator logs
BerthaCluster is not Ready kubectl get pods -n bertha-system. If the validator fails, start with the driver on that node
No /dev/ha* inside the container Run ha-ctk precheck on the node and check the runtime's CDI configuration
Allocatable count is lower than the card count Check whether device-plugin has excluded faulty cards from new allocations. Check the lpu_device_up and lpu_device_wedged metrics
Exporter is missing from Prometheus PodMonitor namespace and label selectors, and permission to list Pods. If the target is DOWN, check the metrics port, path, and network