KvCacheMemoryReport

data class KvCacheMemoryReport(val numLayers: Int, val numHeads: Int, val headDim: Int, val maxSeqLen: Int, val currentSeqLen: Int, val keyEncoding: TensorEncoding, val valueEncoding: TensorEncoding, val placement: Placement, val keyPhysicalBytes: Long, val valuePhysicalBytes: Long, val keyLogicalBytes: Long, val valueLogicalBytes: Long)(source)

Memory report for a KV cache instance.

Constructors

constructor(numLayers: Int, numHeads: Int, headDim: Int, maxSeqLen: Int, currentSeqLen: Int, keyEncoding: TensorEncoding, valueEncoding: TensorEncoding, placement: Placement, keyPhysicalBytes: Long, valuePhysicalBytes: Long, keyLogicalBytes: Long, valueLogicalBytes: Long)