storeKeyValue

fun storeKeyValue(layer: Int, key: FloatArray, value: FloatArray)(source)

Store K/V projections for a new token, compressing as configured.

Parameters

layer

Layer index

key

Key projection numHeads, headDim

value

Value projection numHeads, headDim