Views
No views yet
SuperexponentialAI/relu. It was built to
be used with the kernels library.1# make sure `kernels` is installed: `pip install -U kernels`
2from kernels import get_kernel
3
4kernel_module = get_kernel("SuperexponentialAI/relu", version=3)
5relu = kernel_module.relu
6
7relu(x)relumetal and rocm are
declared in kernels-community/relu upstream but are intentionally
not included here — no Apple or AMD hardware was available to build
or validate them, and this repo only ships backends that were actually
optimized and benchmarked.kernels-community/relu is fp32-only). Semantics match upstream exactly,
including NaN → 0 (not PyTorch's NaN → NaN) — this is a pre-existing
upstream behavior, intentionally preserved. All backends verified
bit-identical to torch.nn.functional.relu on non-NaN inputs.F.relu). Upstream (kernels-community/relu) is fp32-only, so other
dtypes have no Hub baseline. v3 corrects earlier 4090 claims (fp16/
bf16 "1.85x/1.81x", fp32 "1.20x DRAM") that came from a position-biased
shared-buffer interleave, and fixes a real (32 MiB, L2] launch-config
cliff — full story in L2_BAND_CLIFF.md; methodology in
PERFORMANCE.md.| target | dtype | vs torch | vs upstream Hub |
|---|---|---|---|
| RTX 4090 (sm_89) | fp32 | ~parity (0.95–1.09x) | 0.96–1.22x L2, parity DRAM |
| RTX 4090 | fp16 / bf16 | 1.17–1.27x base, ~parity large | — |
| RTX 4090 | int8 | 1.14–1.37x | — |
| H100 (sm_90) | fp32 | 1.52x L2, ~parity DRAM | 1.24–1.44x DRAM |
| H100 | fp16 / bf16 | 1.38x L2, ~1.0–1.20x DRAM | — |
| H100 | int8 | 1.23–1.61x | — |
| CPU (i9-13900K) | fp32/fp16/bf16/int8 | at DRAM bandwidth ceiling | — |
| XPU (UHD 770) | fp32/fp16/bf16/int8 | 1.01x / 0.96x / 1.16x / 1.06x | — |
__ldcs/__stcs except in the pre-Hopper (32 MiB, L2] band, which
uses default loads + streaming stores (v3 fix — evict-first loads there
ran 0.34–0.65x vs torch by discarding L2 input residency; see
L2_BAND_CLIFF.md), and Hopper+ below 16 MB, which uses unroll-2 /
stream-store-only / 256-thread blocks (+4.6–9.3%, symmetric pairwise
A/B). int8 additionally uses __vmaxs4 SIMD byte-max. Tuned via
per-arch empirical sweeps, a 5-agent design fan-out, and roofline
analysis (OPTIMIZATION.md for sm_89, H100_BENCHMARK.md for sm_90 —
including which standalone-sweep "wins" failed in-harness verification
and were not shipped).torch.relu (v3, fair buffer handling — see
L2_BAND_CLIFF.md for why v2's shared-buffer interleave numbers were
retracted): fp32 ~parity at every size (0.95–1.09x, both at the ~944
GB/s DRAM wall when DRAM-bound); fp16/bf16 1.17–1.27x at 1024²,
~parity at 4096²; int8 1.14–1.37x. In the band the v2 kernel lost
2–3x, v3 measures 0.99–1.06x.kernels-community/relu (fp32, bit-identical
output): 0.96–1.22x at L2-resident sizes (run-sensitive), parity at
the DRAM wall — both kernels saturate GDDR6X there. Upstream's real
gaps are dtype coverage (fp32-only) and the 1-D launch collapse
below.DTYPE_THROUGHPUT.md).H100_BENCHMARK.md):torch (separate-buffer symmetric pairs): fp32 1.52x at
L2-resident 1024², ~parity DRAM-bound (both at the wall); fp16/bf16
1.38x at 1024², 1.0–1.20x at 4096²; int8 1.23x at 1024²,
1.61x at L2-resident 4096². The pre-Hopper L2-band cliff does not
occur on Hopper (measured 1.43–1.55x over torch at band sizes with
the streaming config; L2_BAND_CLIFF.md).grid = numel/size(-1) blocks, which collapses to a
single block on flat 1-D tensors — measured 140x slower than this
kernel at 4096² elements on H100 (relu(x.view(-1))-shaped calls).torch.relu (torch 2.12.0+xpu, Intel UHD 770 —
a consumer iGPU, not an officially-supported PyTorch XPU tier; both
implementations are bound by the same shared-DDR5 wall, so gains are
modest compared to CUDA/CPU):| dtype | torch (GB/s) | ours (GB/s) | ratio (ours vs torch) |
|---|---|---|---|
| fp32 | 40.6 | 40.8 | 1.01x (tied) |
| fp16 | 43.8 | 42.1 | 0.96x (torch ~4% faster) |
| bf16 | 36.6 | 42.5 | 1.16x faster |
| int8 | 78.0 | 82.9 | 1.06x faster |
F.relu. See AITESTING.md
for the environment setup and hardware-driver caveats needed to run
this (a separate torch==2.12.0+xpu venv, Level Zero runtime install,
device ACLs, and a CPU-generate-then-copy workaround for an on-device
fp16/bf16 creation bug on this GPU).benchmarks/: ReluBenchmark mirrors
upstream's fp32 base/large workloads (directly comparable), and
ReluDtypeBenchmark adds the fp16/bf16/int8 workloads upstream lacks.kernels benchmark ./relu-build (all workloads verify
against F.relu).kernels benchmark SuperexponentialAI/relu --version 3)
requires this repo to be hosted as a kernel-type repo; it is
currently model-type (kernel-repo creation is restricted for this
account), so use the local form until it migrates. The same caveat
applies to the get_kernel snippet above — load via
snapshot_download + sys.path in the meantime.ReluDtypeBenchmark uses the kernel's preallocated out= form and times
the reference the same way — the allocating form alternates two live
output blocks and pushes L2-boundary workloads off the L2 cliff for
whichever side allocates (details in L2_BAND_CLIFF.md). For pure GPU
kernel time, the nsys numbers above remain the ground truth
(INVESTIGATION_L2.md).OPTIMIZATION.md, DTYPE_THROUGHPUT.md,
and INVESTIGATION_L2.md in this repository.