Luna Flow
English

perf_support tutorial

This page shows contributors how to list benchmark cases, run one case and check its result from MoonBit code, for example when investigating a kernel change. The method is described in the perf_support design and in bench/README.md.

Quick start

perf_support is used inside this repository. Its tests run with:

LINEAR_ALGEBRA_TEST_BENCH=1 ./run_test.sh

or directly:

moon test -p perf_support --target native

Everyday tasks

List the cases

for name in @support.sample_case_names() {
  println(name)
}

case_names() lists all registered cases.

Run one case and compare checksums

let prepared = @support.find_prepared_case("chol_baseline_spd_64").unwrap()
let before = @support.run_prepared_case_once(prepared)
// ... change a kernel, rebuild ...
let after = @support.run_prepared_case_once(prepared)
assert_eq(before, after) // the result bits did not change

Equal checksums mean the result is bit-for-bit unchanged; different checksums after a kernel change are expected when the summation order changed, and then the results must be compared numerically.

Regenerate fixtures

Delete bench/datasets/cases/<id>.json or run python3 bench/generate_fixtures.py; the next run recreates the inputs from the registry.

Going further

To add an operation, add cases to the manifest, regenerate the registry with bench/generate_fixtures.py, and extend the match in run_prepared_case_inplace.

Common pitfalls

  • Running on non-native targets. Fixture loading needs file-system access.
  • Editing generated_registry.mbt by hand. It is generated; change the manifest and regenerate.
  • Comparing checksums across targets. Kernels differ per target, so result bits may differ.

Next steps