SKaiNET
Toggle table of contents
0.20.0-SNAPSHOT
common
Target filter
common
Switch theme
Search in API
Skip to content
SKaiNET
skainet-io-core
/
sk.ainet.io.tokenizer
/
SentencePieceTokenizer
/
Companion
/
WHITESPACE_ESCAPE
WHITESPACE_
ESCAPE
const
val
WHITESPACE_ESCAPE
:
Char
=
'\u2581'
(
source
)
SentencePiece whitespace-escape character:
▁
(U+2581).