Examples and tutorials¶
The tasks you perform often while operating the driver are organized here so that you can follow them from start to finish. Each tutorial includes example scripts and settings that you can use as they are or adapt. Why the features behave as they do is explained in Operations guide; command options and exit codes are on the page of each tool (hyperaccel-smi, ha-hotreset, ha-fw-update); and the definitions of the nodes are in sysfs nodes.
Tutorials¶
| Tutorial | What you will be able to do | Example |
|---|---|---|
| Collecting telemetry with Prometheus | Collect hwmon sensor values and the card state, memory, error counters, and utilization with Prometheus, and set up alerts | ha-prom-textfile.sh, systemd timer, alert rules |
| Building a card health check | Check that the driver is loaded, the number of cards, and the health and fw_version of each card, and report the result through the exit code |
ha-healthcheck.sh, systemd timer |
| Collecting diagnostic information for a problem | Before resetting a card, save the information to send to the support team in a single archive | ha-collect-diag.sh |
| Recovering an unresponsive card | Follow the process from checking the symptom through resetting one card with ha-hotreset to confirming the recovery |
Command examples |
| Updating firmware on several cards in sequence | Update the cards one at a time with the same firmware, and stop on the spot if a problem occurs | ha-fw-update-all.sh |
When a card has a problem, collect the diagnostic information first and then recover the card. This is because a reset erases the state that shows the cause.
Before you use the example scripts¶
- The example scripts are not included in the driver package. Save the content of the page to a file and use it.
- The scripts are written in POSIX
shsyntax, so they also work with a/bin/shthat is not bash (for example,dash). - The health check and monitoring examples only read sysfs and do not open the device nodes, so they can run alongside workloads. The health check runs without root privileges.
- In an environment where the
/sysof the host is mounted at a different path (for example, a container), you can tell the scripts that path with the environment variableSYSFS. - Adapt values such as paths, the number of cards, and intervals to your environment.