SKaiNET
Toggle table of contents
0.20.0-SNAPSHOT
common
Target filter
common
Switch theme
Search in API
Skip to content
SKaiNET
skainet-io-gguf
/
sk.ainet.io.gguf
/
GgufModelMetadata
/
tokenizerTokens
tokenizer
Tokens
val
tokenizerTokens
:
List
<
String
>
?
(
source
)
Full vocab as stored in
tokenizer.ggml.tokens
(index = token id).