skainet-lang-core/sk.ainet.lang.tensor.storage/DefaultKvCacheStore

DefaultKvCacheStore

class DefaultKvCacheStore(config: KvCacheConfig) : KvCacheStore(source)

Default KV cache implementation using dense FP32 storage.

This is the reference/baseline implementation that stores K/V as uncompressed float arrays. Quantized implementations (Q8_0, TurboQuant) will override appendToken and readKeys/readValues with encode-on-write / decode-on-read paths.

Internal layout per layer:

keys: FloatArray(numHeads * maxSeqLen * headDim) — numHeads, maxSeqLen, headDim
values: FloatArray(numHeads * maxSeqLen * headDim) — numHeads, maxSeqLen, headDim

Append writes to position currentSeqLen; read returns a contiguous slice.

Constructors

DefaultKvCacheStore

constructor(config: KvCacheConfig)

Properties

open override val currentSeqLen: Int

Current number of tokens stored in the cache.

open override val headDim: Int

Dimension per head.

open override val keyEncoding: TensorEncoding

Encoding used for key storage.

open override val maxSeqLen: Int

Maximum sequence length this cache can hold.

open override val numHeads: Int

Number of KV heads per layer.

open override val numLayers: Int

Number of transformer layers in this cache.

open override val placement: Placement

Placement intent for the cache buffers.

open override val valueEncoding: TensorEncoding

Encoding used for value storage.

Functions

open override fun appendToken(layer: Int, key: FloatArray, value: FloatArray)

Append a single token's K/V projections for one layer.

open override fun clear()

Reset the cache, clearing all stored tokens.

open override fun evict(fromPos: Int)

Evict all cached tokens from position fromPos onward.

open override fun memoryReport(): KvCacheMemoryReport

Memory report for the entire cache.

open override fun readKeys(layer: Int, startPos: Int = 0, endPos: Int = currentSeqLen): FloatArray

Read cached keys for a layer, dequantized to float.

open override fun readKeyStorage(layer: Int, startPos: Int = 0, endPos: Int = currentSeqLen): TensorStorage

Read raw (possibly compressed) key storage for a layer as TensorStorage.

open override fun readValues(layer: Int, startPos: Int = 0, endPos: Int = currentSeqLen): FloatArray

Read cached values for a layer, dequantized to float.

readValueStorage

open override fun readValueStorage(layer: Int, startPos: Int = 0, endPos: Int = currentSeqLen): TensorStorage

Read raw (possibly compressed) value storage for a layer as TensorStorage.