Skip to content

Installation

This page describes the procedure for installing the driver ha.ko. The driver is distributed as the deb package hyperdriver-dkms. When you install it, the module is loaded automatically at boot, and the udev rules and the operations tools (hyperaccel-smi, ha-hotreset, ha-fw-update, ha-led, ha-wipe) are installed with it. When the installation finishes, /dev/haN and /sys/class/ha/haN/ appear for each LPU device. After installing, follow Verifying the installation to check that everything works correctly.


Before installing

What you need

Item Requirement
Privileges root
Running kernel A kernel on which the driver builds
Headers of the running kernel Install them beforehand
dkms Installed as a dependency
python3 Installed as a dependency
pciutils Installed as a dependency
  • python3 is used by hyperaccel-smi, ha-fw-update, ha-led, and ha-wipe. ha-fw-update requires Python 3.7 or later.
  • lspci and setpci from pciutils are used by ha-hotreset. ha-fw-update and ha-wipe also go through ha-hotreset when they reset the card.
  • ha.ko is built on the machine where you install it, so a compiler (gcc) and make are required. The dkms package installs both as dependencies.

Check that the devices are visible

1
lspci -nn -d 20fb:

One line should appear per card. If nothing appears, no device nodes are created even after you install the driver. Check first that the cards are seated and powered.

Secure Boot

A machine with Secure Boot enabled loads only modules signed with an enrolled key. Check the state with mokutil --sb-state.

dkms signs ha.ko with its own key, so you must enroll that key once. See Enrolling the Secure Boot key.


Installing with the deb package

The hyperdriver-dkms package installs the driver source through DKMS. It is used on Ubuntu and Debian.

Item Value
Package name hyperdriver-dkms
Architecture all
Dependencies dkms, linux-headers-generic or linux-headers-amd64, python3

The CDI spec is created not by this package but by the HyperAccel container toolkit (ha-container-toolkit). See Container support.

Procedure

Installing hyperaccel-lpu from the HyperAccel apt repository also installs this package. For how to register the repository, see Quick install. To install directly from a deb file that you have downloaded, run the following.

1
2
3
sudo apt install linux-headers-$(uname -r)
sudo apt install ./hyperdriver-dkms_<version>_all.deb
lsmod | grep -w ha

The header package in the dependencies is a meta package, so it may not be the headers of the running kernel. The install script requires the headers of the running kernel, so install those headers first. If they are missing, the script prints the following and the installation fails.

1
2
hyperdriver-dkms: kernel headers for <kernel version> not found.
  install them, e.g. apt install linux-headers-<kernel version>

What the install script does

  1. Checks that the headers of the running kernel exist.
  2. First checks in a temporary DKMS tree that the new version builds on the running kernel. If it does not build, the script fails the installation without touching the installed driver. See Upgrading the deb package.
  3. Removes all previously registered versions and registers the new version with DKMS.
  4. Builds and installs ha.ko for the running kernel.
  5. Also builds ha.ko for those other installed kernels (such as a kernel waiting for a reboot) that have headers. For a kernel without headers, it only prints a warning.
  6. Reloads the module and applies the udev rules.

The warning in step 5 is as follows.

1
hyperdriver-dkms: no headers for installed kernel <kernel version> - ha.ko will be MISSING if you boot it.

If you plan to boot that kernel, install the headers of that kernel before booting and run sudo dkms autoinstall -k <kernel version>. It is done when dkms status -m ha shows that kernel as installed.

Check that the module is loaded

The install script finishes the installation even if the modprobe ha in step 6 fails. This happens on a machine where the Secure Boot key has not been enrolled. After installing, check with lsmod | grep -w ha that the module is loaded.


Enrolling the Secure Boot key

dkms signs ha.ko with the same key of its own every time it builds. Once you enroll this key as a MOK (Machine Owner Key), modules rebuilt for new kernels also load.

The location of the key file differs by distribution.

1
2
3
4
# Ubuntu
sudo mokutil --import /var/lib/shim-signed/mok/MOK.der
# Other distributions
sudo mokutil --import /var/lib/dkms/mok.pub
  1. mokutil asks you to set a one-time password.
  2. When you reboot, the MOK management screen appears. Press a key within about 10 seconds, select Enroll MOK, and enter the password.
  3. The module is loaded automatically at boot, so after the boot finishes, check with lsmod | grep -w ha.

On Ubuntu, dkms may already have opened a dialog during installation and asked for this password. In that case, mokutil reports that the key is already in the enrollment request, so you only need to reboot.


Installed files

Path Contents
ha.ko Installed by DKMS for each kernel
/usr/src/ha-<version>/ DKMS source
/etc/modules-load.d/ha.conf Configuration that loads the ha module at boot
/lib/udev/rules.d/99-ha.rules udev rules
/usr/bin/hyperaccel-smi, /usr/bin/ha-hotreset, /usr/bin/ha-fw-update, /usr/bin/ha-led, /usr/bin/ha-wipe Operations tools

You can check the actual path of the ha.ko that will be loaded with modinfo -n ha. For how to use the tools, see hyperaccel-smi, ha-hotreset, ha-fw-update, ha-led, and ha-wipe.


Automatic loading at boot

The package installs /etc/modules-load.d/ha.conf. Its contents are the single line ha, which causes the ha module to be loaded at boot. If there is no ha.ko that matches the kernel you booted, the module is not loaded. For kernel updates, see When you update the kernel.


Module parameters

The installation procedure does not set any module parameters, so the module is loaded with the default values. To change a value, create a configuration file under /etc/modprobe.d/.

1
2
# /etc/modprobe.d/ha.conf
options ha <parameter>=<value>

This file is read when the module is loaded, so it takes effect only after you reload the module or reboot. On distributions that build the initial ramdisk with dracut, such as the RHEL family and Fedora, applying it by rebooting may require rebuilding the initial ramdisk as well. See Setting values in an option file. Some parameters can also be changed while the module is running. For how to change them, see Configuring and using parameters, and for the list of parameters, see Parameter reference. The removal procedure does not delete this file.


Device node permissions and udev rules

The udev rules that are installed (99-ha.rules) are as follows.

1
2
3
SUBSYSTEM=="ha", MODE="0666"
SUBSYSTEM=="ha", ACTION=="add", RUN+="/bin/chmod a+rw /sys/class/ha/%k/bypass_range_check"
SUBSYSTEM=="ha", ACTION=="change", RUN+="/bin/chmod a+rw /sys/class/ha/%k/bypass_range_check"
Line What it does
1 Makes /dev/haN readable and writable by all local users (0666)
2–3 Makes the debug node bypass_range_check readable and writable by all users

For bypass_range_check, see sysfs nodes.

The default rules leave access open

  • Any local user can open /dev/haN and use the device.
  • Anyone can write 1 to bypass_range_check to turn off the memory access range check of that device.

On a machine used by several users, restrict the device nodes to a group as shown below, and remove the two bypass_range_check lines so that only root can use bypass_range_check.

Restricting access

Example rule. The group name hyperaccel is an example.

1
SUBSYSTEM=="ha", MODE="0660", GROUP="hyperaccel"

After you edit the rule, create the group and apply the rules again.

1
2
3
4
sudo groupadd -f hyperaccel
sudo usermod -aG hyperaccel <user>
sudo udevadm control --reload-rules
sudo udevadm trigger --subsystem-match=ha
  • A user added to the group gets the group permission only after logging in again.
  • The permission that was already changed on bypass_range_check returns to the default (writable by root only) when you reload the module or reboot.

The file to edit is /etc/udev/rules.d/99-ha.rules. Copy /lib/udev/rules.d/99-ha.rules to that path and edit the copy. A file with the same name in /etc/udev/rules.d/ takes precedence, and a package upgrade does not change this file.


See also