Architecture Overview

The DSL-Centric Core

Model architectures are declared, not hand-coded: every generative family is a thin <f>Network() builder over the shared decoderTransformerNetwork() in llm-core (package sk.ainet.lang.nn.dsl.decoder), which composes the SKaiNET engine’s DSL primitives (embedding, RMSNorm, MHA with RoPE/KV-cache/QK-norm/biases, SwiGLU or family-specific FFNs) into a compute graph. The same package owns the shared loading machinery — DecoderGgufWeightLoader (a thin wrapper over the engine’s StreamingGgufParametersLoader; no per-family quantization or packing code exists in this repository), GgufDecoderMetadata, DecoderTensorNames, and the family-neutral SafeTensors half. Hand-coded runtimes survive only where the DSL cannot express the architecture yet (Gemma 3n’s AltUp/per-layer embeddings, T5’s encoder-decoder loop) — see DSL Networks vs Hand-Coded Runtimes.

Every family module follows the same file scheme (issue #346): <F>NetworkDef.kt, <F>NetworkLoader, <F>WeightLoader, <F>RuntimeWeights/<F>TensorNames, <F>ConfigParser, plus a k<f> runtime facade where one exists. Adding a family is mechanical — see Add a New Model Architecture.

Module Structure

transformer-core            NN primitives (attention, KV-cache, embedding, norms, RoPE, FFNs) — all targets incl. androidNative
llm-core                    Decoder DSL core (sk.ainet.lang.nn.dsl.decoder), Tokenizer, InferenceRuntime,
                            OptimizedLLMRuntime, ModelRegistry; re-exports transformer-core
llm-agent                   Chat templates, tool calling, AgentLoop, ChatSession
llm-inference/
  llama/                    Llama/Mistral family (+ the deprecated legacy LlamaRuntime, retiring)
  qwen/                     Qwen 2/2.5/3 family (attention biases, QK-norm)
  gemma/                    Gemma 3/4 family (DSL lane, GGUF + SafeTensors)
  gemma3n/                  Gemma 3n family (hand-rolled runtime: AltUp, per-layer embeddings)
  functiongemma/            FunctionGemma StableHLO/IREE export
  apertus/                  Apertus family (xIELU, QK-norm, ungated FFN)
  bitnet/                   BitNet b1.58 family (packed I2_S ternary path)
  smollm2/                  SmolLM2 StableHLO/IREE export
  bert/                     BERT embeddings (DSL)
  t5/, vec2text/            T5/GTR encoder-decoder + embedding inversion
  voxtral/                  Voxtral TTS (architecture code only)
  moonshine/, whisper/      Speech-to-text encoders
llm-runtime/
  kllama/                   Runtime facade + CLI backend for the llama-compatible set (Llama/Mistral/Qwen)
  kgemma/                   Gemma runner CLI (gemma3/gemma4/gemma3n variants)
  kgemma3n/                 Gemma 3n runtime facade
  kapertus/                 Apertus runtime facade
  kbitnet/                  BitNet runtime facade
  gemma-iree/, iree-android/  Compiled (StableHLO → IREE) on-device path
llm-apps/
  skainet-cli/              Unified CLI (auto-detects architecture, routes every GGUF family)
  kllama-cli/               Llama-family CLI with agent/tool-calling mode
  kbert-cli/                BERT CLI
  skainet-decode/           GGUF decode + GenerationMetrics CLI (JVM leg, SKaiNET#1129)
  skainet-decode-core/      Shared DecodeSession flow behind both decode legs (KMP: jvm + android)
  skainet-decode-android/   On-device decode/metrics activity (Android leg, SKaiNET#1244)
  kllama-java-sample/       Java interop sample
llm-performance/            Benchmarking module

Dependency Graph

Key Interfaces

InferenceRuntime<T>: Minimal inference contract: forward(tokenId): Tensor and reset().
OptimizedLLMRuntime<T>: The runtime every DSL family runs on — takes the module built by an <F>NetworkLoader, executes it eagerly (DIRECT mode) or through the traced/fused pipeline, and provides the greedy/sampled generate() loop. The golden-token parity gates certify this exact path against reference implementations.
Tokenizer: Encode/decode text with eosTokenId, bosTokenId, vocabSize. Obtained via TokenizerFactory.fromGgufFields(…); implementations include GGUFTokenizer, HuggingFaceBPETokenizer, TekkenTokenizerAdapter.
ChatTemplate: Format conversation messages into prompt strings and parse tool calls from output. Implementations: QwenChatTemplate, Llama3ChatTemplate, GemmaChatTemplate, ChatMLTemplate.
DecoderRuntime<T>: Template-method base class of the legacy hand-coded runtimes (LlamaRuntime is deprecated and unreferenced by the CLIs; Gemma 3n’s runtime is the remaining production user). New families never implement it — they declare a network and run on OptimizedLLMRuntime.