Run LLM inference from Java — moved

This how-to moved. The KLlama / KBert Java inference APIs described here (sk.ainet.apps.kllama.java.KLlamaJava, sk.ainet.apps.bert.java.KBertJava, sk.ainet.apps.kllama.chat.java.JavaAgentLoop, and friends) now live in the sibling SKaiNET-transformers repository. Mainline SKaiNET kept the engine layer only — tensors, graph IR, backends, and model loading / tokenization.

Start here instead:

What stayed in mainline and is still useful for LLM workflows:

  • Load models (GGUF, SafeTensors, ONNX) — StreamingGGUFReader, StreamingSafeTensorsReader, zero-copy file-backed loads, quantization-preserving paths.

  • The TokenizerFactory in skainet-io-core dispatches to the right implementation per model architecture — Qwen / GPT-2 byte-level BPE via QwenByteLevelBpeTokenizer, LLaMA / Gemma / TinyLlama SentencePiece via SentencePieceTokenizer. Both verified against HuggingFace reference token IDs. Usable from Java via TokenizerFactory.fromGguf(fields) or fromTokenizerJson(json).

  • The TensorEncoding metadata on TensorSpec carries Q4_K / Q8_0 / TernaryPacked quant layout through the graph IR, so backends can dispatch on it — see the 0.19.0 release notes.