skainet-lang-core/sk.ainet.lang.tensor.ops.turboquant/BitPacker

BitPacker

object BitPacker(source)

Bit-packing and unpacking for TurboQuant codes.

Packs signed N-bit integer codes into compact byte arrays for storage. Supports 2, 3, 4, and 8-bit packing. Codes are stored as unsigned offsets (biased by 2^(bits-1)) to simplify packing.

Packing is append-friendly: codes can be packed incrementally per token without re-packing the entire cache.

Functions

Link copied to clipboard
fun pack(codes: ByteArray, bits: Int): ByteArray

Pack signed codes into a compact byte array.

Link copied to clipboard
fun packedSize(count: Int, bits: Int): Int

Compute the byte size needed to pack count codes at bits per code.

Link copied to clipboard
fun unpack(packed: ByteArray, count: Int, bits: Int): ByteArray

Unpack a byte array back to signed codes.

Generated by Dokka
© 2026 Copyright