skainet-io-safetensors/sk.ainet.io.safetensors/SafeTensorsIndexParser

SafeTensorsIndexParser

Parser for model.safetensors.index.json files.

Index files are used for sharded SafeTensors models to map tensor names to their containing shard files.

Usage:

val index = SafeTensorsIndexParser.parse(jsonContent)
println("Shards: ${index.shardCount}")
println("Total size: ${index.metadata.totalSize}")

val shardForTensor = index.getShardForTensor("model.embed_tokens.weight")

Functions

deriveIndexFilename

fun deriveIndexFilename(shardFilename: String): String?

Derive the expected index filename from a shard filename.

isIndexFile

fun isIndexFile(path: String): Boolean

Check if a file path looks like a SafeTensors index file.

isShardedFilename

fun isShardedFilename(filename: String): Boolean

Check if a filename matches the sharded SafeTensors pattern.

parse

fun parse(jsonContent: String): SafeTensorsIndex

Parse a SafeTensors index from JSON content.

parseShardFilename

fun parseShardFilename(filename: String): Pair<Int, Int>?

Extract shard information from a sharded filename.