Support Matrix
Items verified with hapti 0.0.6. Anything not listed here is not verified.
Legend — ✅ supported · 🟡 limited · ❌ not supported
Full stack combinations, including OS, kernel, driver, and firmware, are in the
SDK Installation Guide. This page covers what the profiler itself supports.
| Item |
Status |
Notes |
| Linux, x86_64 |
✅ |
Works through LD_PRELOAD |
| Python 3.10 – 3.12 |
✅ |
|
Runtime (hart) 0.0.6 |
✅ |
|
| Device tracing |
✅ |
|
| Windows, macOS |
❌ |
|
| Statically linked targets |
❌ |
Cannot be intercepted with LD_PRELOAD |
Collection
| Feature |
Status |
Notes |
| Runtime API activity |
✅ |
|
| Memory allocation activity |
✅ |
Included in runtime activity with a byte count. The allocating device is not recorded |
| Lower-layer call activity |
✅ |
|
| Device kernel timeline |
✅ |
--runtime-trace or --device-trace |
| Device memory transfer timeline |
✅ |
|
| Memory transfer bandwidth track |
✅ |
H2D and D2H, 4 KiB or more |
| Per-kernel hardware counters |
🟡 |
--device-trace. Default is 20 of core 0, up to 32 |
| Signal logger |
✅ |
--device-trace |
| Host-device correlation |
✅ |
|
| User annotations |
✅ |
haprof path |
| Profiler overhead |
❌ |
The record type exists but is not recorded |
| Per-domain activity selection |
❌ |
Flags are parsed but ignored |
| Profiling an already running process |
❌ |
Configuration is fixed at process start |
| Start and stop during execution |
❌ |
haprof works per process. Use the Kineto plugin for region-level control |
| More than one device per trace |
❌ |
Collects from one device only. Select it with HAPTI_TOOL_DEVICE_ID |
CLI Options
The status and effect of each option are in CLI: Option Status.
| Option |
Status |
Notes |
--runtime-trace, --device-trace |
✅ |
|
--profile-children, --preload, --no-execv |
✅ |
|
-o, -d, -f, --log-level |
✅ |
|
Per-domain --*-trace, --external-correlation |
❌ |
Parsed but silently ignored |
haprof merge |
✅ |
|
Output
| Format |
Status |
Notes |
Perfetto .pftrace |
✅ |
|
Chrome JSON .json |
✅ |
|
| Selecting a single format |
✅ |
-f |
| Custom path or name |
✅ |
-d, -o |
Counters
|
|
| Counter blocks |
6 — AXIBRG, CDMA, DMC, UTIL, CGDMA, MPU |
| Events |
41 |
| Registered per execution |
Up to 32, 20 by default |
| Sampling |
Per kernel execution |
| Representation |
Applies to |
| Per-execution delta |
AXIBRG, CDMA, CGDMA, MPU |
| Raw value |
DMC |
Percentage of sample_total |
UTIL |
| Derived bandwidth and stall breakdown |
CGDMA, AXIBRG |
| Bank skew |
DMC |
The catalog and formulas are in the Counter Reference.
Multi-Process
| Feature |
Status |
Notes |
| fork and exec children |
✅ |
--profile-children. One trace per process |
| Merging traces from the same machine |
✅ |
haprof merge. Aligns exactly because the clock is shared |
| Straggler report |
✅ |
haprof merge. Requires per-rank .json |
| Merging traces across machines |
🟡 |
Concatenation works but clocks are not corrected. The report prints a cross-node: yes warning and per-process clock offsets |
Integrations
| Integration |
Status |
Notes |
| Perfetto UI |
✅ |
|
| Chrome tracing |
✅ |
|
| PyTorch profiler through Kineto |
✅ |
hapti.kineto.register() |
| Device timeline on the Kineto path |
✅ |
|
| Hardware counters on the Kineto path |
❌ |
Use haprof --device-trace |
hapti annotations on the Kineto path |
❌ |
Use torch.profiler.record_function() |
Using haprof and the Kineto plugin together |
❌ |
Only one device record ring can be open |
| Writing a custom collection tool (C++ API) |
❌ |
No public API is provided. Use haprof or the Kineto plugin |
| Trace loading or query API |
❌ |
Use Perfetto |
Known Limitations
- Per-domain trace flags are parsed but not passed through. They are silently ignored, and host-side
activity kinds are always collected in full. Choose whether to include device data with
--runtime-trace or
--device-trace.
- Configuration is fixed at process start. You cannot profile an already running process, start or stop during execution, or change settings. If you
need region-level collection, use the
profile() region of the
Kineto plugin, or mark the analysis range with annotations.
- One device per trace. Select another device with
HAPTI_TOOL_DEVICE_ID, and for multi-rank runs with one
device per rank, produce per-rank traces with --profile-children.
- The trace is written at process exit. A forced kill leaves no usable result. Send an interrupt signal instead
of
SIGKILL so the process exits normally.
- Profiler overhead is not recorded. Compare against a run without the profiler to estimate the overhead.
- Memory allocation records do not include the allocating device. In a multi-device process, the allocation
location cannot be distinguished from the trace alone, and there is currently no workaround.
- CGDMA bandwidth and UTIL utilization are estimates, not measurements. Use them only to compare executions,
and cross-check UTIL utilization against the stall breakdown metrics.
- The first execution is not a measurement target. It includes kernel loading, and counter deltas have no valid
baseline. Profile steady-state executions after warm-up.
- In a Kineto session that uses multiple devices, only the lowest-numbered device gets named rows. Many
combinations are unverified, so
haprof --profile-children is recommended for multiple devices.
The impact and workaround for each limitation are summarized in a table in the
Known Issues section of the release notes.
See Also