Eager execution: backends and kernels

Resolution is by priority: Native (100 — whichever of FFM/JNI/cinterop applies to the target) → Panama / Wasm SIMD128 (50) → Scalar (0). The best available provider that carries the kernel wins; otherwise it cascades down. At most one native tier is compiled into a given target, so the native columns are mutually exclusive per platform rather than stacked. The Wasm SIMD128 column is the sidecar pack (skainet-backend-wasm-simd), present only after WasmSimd.install(); every matmul kernel in it has an explicit SIMD128 body (the K-quant int8 dots on i32x4.dot_i16x8_s, a 4×8 register-blocked dense GEMM, eight-wide BF16/FP16 widening), and the dense kernels thread over the sidecar’s pthread pool like the packed ones when the page is cross-origin isolated. The ternary kernels in the sidecar are the portable mirror of the vendored NeoGPU kernel — the same 256-entry decode table as a compile-time constant, read four codes at a time in SIMD128 — registered under the same dispatch keys as the FFM and JNI ternary packs (WasmTernaryKernelPack).

The dense x · Wᵀ row is the product a Linear layer computes. The JVM and the wasm tier multiply the [out, in] weight in place through skainet_fp32_matmul_wt (KernelProvider.matmulFp32Wt; the JVM makes a heap weight resident off-heap once, the sidecar keeps it in its linear memory) and never materialise the transposed copy the Panama path caches per weight.

The fused row kernels — softmax / log-softmax over the last dimension, layer norm and RMS norm (what LayerNormalization calls through the optional FusedNormOps hook), and scaled dot-product attention with its structural masks and grouped-query heads — are one C pass each (f32_fused.c), served by the JVM FFM provider (softmaxFp32, normFp32, attentionFp32) and the wasm sidecar alike. Where neither is present the ops layer runs its common implementation: attention scalar per head and parallel over heads through the schedule, the normalisation as its decomposed elementwise chain. The decomposition is also what a recording or differentiating op set always sees, so the hook changes no recorded graph and no gradient.

Packed weights reach a kernel through the registry by declaring their block order; a canonical weight is relayouted once, not per call.

Packed-quant matmul works on every target: those formats have a commonMain scalar kernel and DefaultCpuOpsBase dispatches packed weights through the registry.