skainet-lang-core/sk.ainet.lang.tensor.storage/KvCacheAnnotationResolver

KvCacheAnnotationResolver

object KvCacheAnnotationResolver(source)

Resolves KvCache annotations to KvCacheStore instances.

Used by skainet-transformers to create KV caches declaratively. When a model layer is annotated with @KvCache(preset = "balanced"), this resolver creates the appropriate compressed or dense cache.

Example:

// In skainet-transformers attention layer:
@KvCache(preset = "balanced")
class SelfAttention(val numHeads: Int, val headDim: Int, ...) {
    val cache = KvCacheAnnotationResolver.resolve(
        annotation = this::class.annotations.filterIsInstance<KvCache>().first(),
        numLayers = modelConfig.numLayers,
        numHeads = numHeads,
        headDim = headDim,
        maxSeqLen = modelConfig.maxSeqLen
    )
}

Functions

resolve

fun resolve(preset: String, numLayers: Int, numHeads: Int, headDim: Int, maxSeqLen: Int): KvCacheStore

Resolve a preset name string to a KvCacheStore.

fun resolve(annotation: KvCache, numLayers: Int, numHeads: Int, headDim: Int, maxSeqLen: Int): KvCacheStore

Resolve a KvCache annotation to a KvCacheStore.