skainet-io-safetensors/sk.ainet.io.safetensors/SafeTensorsWriter

SafeTensorsWriter

class SafeTensorsWriter(source)

Writer for creating SafeTensors format files.

SafeTensors format:

8 bytes: header size (little-endian u64)
N bytes: JSON header with tensor metadata
Remaining: raw tensor data at specified offsets

Usage:

SafeTensorsWriter.write(sink) {
    metadata("format", "pt")
    metadata("framework_version", "2.0.0")

    tensor("layer1.weight", "F32", listOf(10, 20)) { floatArrayToBytes(weights) }
    tensor("layer1.bias", "F32", listOf(20)) { floatArrayToBytes(bias) }
}

Types

object Companion

Functions

fun metadata(key: String, value: String)

Add custom metadata key-value pair.

fun tensor(name: String, dtype: String, shape: IntArray, dataProvider: () -> ByteArray)

Add a tensor with Int shape dimensions.

fun tensor(name: String, dtype: String, shape: List<Long>, dataProvider: () -> ByteArray)

Add a tensor to the file.

fun tensorBF16(name: String, shape: List<Long>, data: FloatArray)

Add a BFloat16 tensor from FloatArray. Values are converted from Float32 to BFloat16.

fun tensorBool(name: String, shape: List<Long>, data: BooleanArray)

Add a Bool tensor from BooleanArray.

fun tensorF16(name: String, shape: List<Long>, data: FloatArray)

Add a Float16 (half-precision) tensor from FloatArray. Values are converted from Float32 to Float16.

fun tensorF32(name: String, shape: IntArray, data: FloatArray)

Add a Float32 tensor from FloatArray with Int shape.

fun tensorF32(name: String, shape: List<Long>, data: FloatArray)

Add a Float32 tensor from FloatArray.

fun tensorF64(name: String, shape: List<Long>, data: DoubleArray)

Add a Float64 tensor from DoubleArray.

fun tensorI16(name: String, shape: List<Long>, data: ShortArray)

Add an Int16 tensor from ShortArray.

fun tensorI32(name: String, shape: IntArray, data: IntArray)

Add an Int32 tensor from IntArray with Int shape.

fun tensorI32(name: String, shape: List<Long>, data: IntArray)

Add an Int32 tensor from IntArray.

fun tensorI64(name: String, shape: List<Long>, data: LongArray)

Add an Int64 tensor from LongArray.

fun tensorI8(name: String, shape: List<Long>, data: ByteArray)

Add an Int8/Byte tensor from ByteArray.

fun tensorU16(name: String, shape: List<Long>, data: ShortArray)

Add a UInt16 tensor from ShortArray (interpreted as unsigned).

fun tensorU32(name: String, shape: List<Long>, data: IntArray)

Add a UInt32 tensor from IntArray (interpreted as unsigned).

fun tensorU64(name: String, shape: List<Long>, data: LongArray)

Add a UInt64 tensor from LongArray (interpreted as unsigned).

fun tensorU8(name: String, shape: List<Long>, data: ByteArray)

Add a UInt8 tensor from ByteArray (interpreted as unsigned).