Skip to content

Upgrading and removal

This page covers the procedures for replacing the installed driver with a new version, handling kernel updates, and removing the driver. When a new version is installed, the installer first checks whether it can be used with the running kernel, and if it cannot, the installed driver is left as it is. However, if processes that use the device remain, the old module keeps running in the kernel, so you must stop the workloads before upgrading and check the running version after the upgrade.


Before upgrading

Stop the workloads

If even one process has /dev/haN open, the module cannot be unloaded. If you upgrade in this state, the new version is installed on disk, but the old version keeps running in the kernel until you reboot or reload the module.

1
2
sudo lsof /dev/ha*
lsmod | grep -w ha

lsof must print nothing, and the third column (use count) of the ha line in lsmod must be 0. The use count includes all processes that have the device open, including processes inside containers. If lsof is not available, you can see the same information with sudo fuser -v /dev/ha*.

Record the current version

1
cat /sys/module/ha/version

Check that the firmware matches the new driver

If the card firmware's command protocol is lower than the minimum version that the new driver requires, the upgraded driver does not probe that card and does not create /dev/haN. Check the current firmware's protocol as <N> in ABI protocol v<N> in the kernel log.

1
sudo dmesg | grep 'ABI protocol'

Compare <N> with the firmware requirements in Supported environments. If the firmware does not meet the requirements, contact HyperAccel support before upgrading the driver.

Upgrade order

  1. Check that the firmware matches the new driver.
  2. Stop the workloads that use the devices.
  3. Upgrade the deb package.
  4. Check the running version.
  5. Also match the SDK that the application uses to the version distributed with this driver.
  6. Restart the workloads.

Upgrading the deb package

1
2
sudo apt install linux-headers-$(uname -r)
sudo apt install ./hyperdriver-dkms_<new version>_all.deb

The installation script tries building the new version in a temporary DKMS tree before it replaces the previous version. Until then, the previous version stays registered and loaded. If the build fails, the installation fails after printing the following, and the previous driver is left as it is for all kernels.

1
2
3
hyperdriver-dkms: ha/<new version> failed to stage or build for kernel <kernel version>.
  <kernel version>: loaded <loaded version>, installed <installed version>
  build log: <log path>

The second line shows the version currently loaded in the running kernel (loaded) and the version that will be loaded next (installed). If there is no driver, it shows none.

A package whose installation failed is left unconfigured. Until this state is cleared, other apt operations also first try to configure this package again and fail, so clean it up in one of the following ways.

  • After resolving the cause, run sudo dpkg --configure -a. If it failed because the headers are missing, install the headers of the running kernel; if the build failed, check the build log and check that the running kernel is listed in Supported environments.
  • Reinstall the previous version's deb. See Reverting to a previous version.

When upgrading from a 2.4.0 or earlier package

The behavior of keeping the previous driver applies only if the package that was installed before the upgrade has it too. A 2.4.0 or earlier package removes its own driver first when the upgrade starts. So if the build of the new version fails, you are left with no driver, and installed in the failure message shows none. On such systems, always install the headers of the running kernel before upgrading, and check that the kernel is listed in Supported environments.

If the installation finished but the previous module could not be unloaded because of processes using the device, the following warning appears. Stop the workloads and reload the module or reboot.

1
2
3
4
hyperdriver-dkms: WARNING - installed <new version>, but the kernel is still running <previous version>.
  the device was in use, so the old driver could not be unloaded.
  free the device (stop workloads) and reload, or reboot:
    modprobe -r ha && modprobe ha

Checking the running version

1
2
cat /sys/module/ha/version
modinfo -F version ha

The first command shows the version currently loaded in the kernel, and the second shows the version of the module file installed for the running kernel. If the two values differ, the old module is still loaded. After stopping the workloads, reload the module or reboot. On distributions that build the initial RAM disk with dracut, such as the RHEL family and Fedora, rebuild the initial RAM disk with sudo dracut -f --regenerate-all before rebooting. Otherwise, a different version of ha.ko contained in the initial RAM disk may be loaded.

For the remaining checks, follow Verifying the installation.


Reloading the module

1
2
sudo modprobe -r ha
sudo modprobe ha
  • If modprobe: FATAL: Module ha is in use. appears, a process is still using the device. See Stop the workloads.
  • When the module is unloaded, all /dev/haN and /sys/class/ha/haN/ disappear, and they are created anew when it is loaded again. The number (haN) is assigned anew on each load, so identify cards by PCI address. See Multiple devices.
  • Module parameters placed in /etc/modprobe.d/ are applied when the module is loaded again.
  • Reloading the module does not reset the card. To revive a stopped card, see FATAL and recovery.

When you update the kernel

When a new kernel is installed, DKMS builds ha.ko for that kernel. The headers of that kernel must be installed as well. When the package is installed or upgraded, it builds for every installed kernel that has headers.

To boot one of the other kernels that were already installed when the package was installed (such as a kernel waiting for a reboot) and that had no headers, install that kernel's headers and build for that kernel before booting.

1
2
sudo dkms autoinstall -k <kernel version>
dkms status -m ha

It is done when dkms status shows that kernel as installed.

  • DKMS skips the build on 16K and 64K page kernels. The driver supports only 4K page kernels. See Supported environments.
  • On systems with Secure Boot enabled too, dkms signs the new module with the same key, so if you have enrolled the key once, you do not need to enroll it again. See Enrolling the Secure Boot key.

Reverting to a previous version

1
sudo apt install --allow-downgrades ./hyperdriver-dkms_<previous version>_all.deb

When reverting to a 2.4.0 or earlier version

The deb of a 2.4.0 or earlier version removes the installed driver first and then builds. If that version does not build on the running kernel, you are left with no driver. Before reverting, check in the Release notes whether that version builds on the running kernel.

After reverting, also check the running version.


Removal

First stop the workloads that use the devices. See Stop the workloads.

1
sudo apt purge hyperdriver-dkms

The removal script unloads the module, deletes all registered DKMS versions, and then deletes the files the package installed. apt remove also removes the driver in the same way.

Deleting only the DKMS registration

1
2
dkms status -m ha
sudo dkms remove ha/<version> --all

dkms remove deletes only the module of that version. The auto-load configuration, udev rules, and tools remain. To remove the driver completely, use the removal procedure above.

What remains after removal

  • Module parameter configuration files you created yourself in /etc/modprobe.d/
  • /etc/udev/rules.d/99-ha.rules, which you created yourself to modify the rules
  • A module that could not be unloaded. If processes were using the device, the module stays loaded even though the files are deleted. End the processes and run sudo rmmod ha, or reboot.
  • On distributions that build the initial RAM disk with dracut, such as the RHEL family and Fedora, the ha.ko and the auto-load configuration inside the initial RAM disk, if it has been rebuilt since the driver was installed (for example, by a kernel update). If left as they are, the driver is loaded again at reboot, so run sudo dracut -f --regenerate-all after the removal.

Verifying the removal

1
2
3
4
5
lsmod | grep -w ha
ls /dev/ha*
ls /etc/modules-load.d/ha.conf
dkms status -m ha
modinfo ha

lsmod and dkms status must print nothing, and for the others, the file or module must not be found.


See also