Kernel × platform support matrix

Kernels that read the weight in canonical row-major GGUF file order straight from off-heap bytes (mmap’d pages or a direct buffer) — no heap staging, no prepack copy (#1189). This is what lets a model larger than the managed-heap cap decode on Android. Dense Float32 tensors are also served from the mapping (as element views, not a matmul kernel) — see StorageCapabilities.mappedServableEncodings for the authoritative set the memory plan budgets against the page cache. An empty cell means the format heap-stages under WeightResidency.MAPPED on that platform.

The browser has no files to map, so the wasm tier keeps weights resident in the SIMD128 sidecar’s linear memory instead: each weight is uploaded once on its first matmul and every later call reads it in place (WasmKernels.weightHandle). This is the JS/WASM counterpart of mapped serving above — no per-call copy of the weight, no prepack — bounded by the wasm32 4 GB linear memory rather than the page cache. The other platforms serve from the heap, off-heap or the mapping and have no cell here.

Kernels that are not reached through the matmul accessors above: the ternary formats are served by ViewKernelPack`s keyed in `KernelDispatch (BitNet-b1.58 with exact FP32 activations, the fused multi-plane BitNet-planes lm_head, TQ2_0 with int8 activations), and the dense x · Wᵀ of a Linear layer by the matmulFp32Wt accessor, which multiplies a row-major [out, in] weight in place instead of transposing and caching a copy. The fused row kernels — softmax / log-softmax, layer norm / RMS norm and scaled dot-product attention with its structural masks and grouped-query heads (softmaxFp32, normFp32, attentionFp32) — run those ops in one pass where the ops layer otherwise runs its scalar loops or the normalisation’s decomposition. The convolutions run as im2col + GEMM: on the JVM inside the ops layer itself (JvmVectorKernels, declared as the panama-vector tier), on wasm as the sidecar’s skainet_f32_conv. The scalar reference kernels serve every cell that is empty here (—), at reference speed; the common row-dot serves x · Wᵀ there.

See also the eager backends & kernels mindmap (Eager execution: backends and kernels) for the narrative overview and gaps.