Cartridge β a concept in the SKaiNET universe
The cartridge is the deployment unit of the SKaiNET universe: the bridge between a model and a real application on real hardware. This page defines the term and its contract, ecosystem-wide β not tied to any one application.
Definition
[working]
A cartridge is a self-contained, application-agnostic software package that runs one model on given hardware, exposing a minimal inputβoutput API that any application can call.
It is the deployment unit of the SKaiNET universe: the bridge between a model (authored, trained, or imported in SKaiNET) and a real application on real hardware. A cartridge knows its model, its runtime, and its hardware. It knows nothing about any application.
The name is literal. Like a game or print cartridge: a sealed, swappable unit you slot into a host. The host doesn’t reach inside the cartridge; the cartridge doesn’t know the host. You change capability by swapping the cartridge, not by rewiring the host.
Why the concept exists
[working]
The SKaiNET universe spans model authoring (a KMP tensor framework), export (ONNX / GGUF / HLO), hardware compilers (a vendor NPU SDK via its cartridge builder, ONNX Runtime, IREE HAL bridges), and a growing set of on-device targets (NPUs, CPUs of varied ABIs, GPUs). Applications want "run this model, fast, on this device" without absorbing any of that.
Without a named unit, every app re-integrates a model+runtime+hardware combination from scratch, and every new model or backend is a fresh integration. The cartridge is the seam that makes the model+runtime+hardware a product you can hand to an application β build it once, reuse it everywhere, swap it without touching the app.
Two forces it resolves:
-
Reuse. One cartridge, many applications. The same Whisper-on-NPU cartridge serves a voice assistant, a CLI, or a test harness unchanged.
-
Replaceability. A faster model, a new quantization, a different backend β a new cartridge behind the same seam. The app is untouched.
Replaceability has two planes
[drafted]
"Swap the cartridge" means two different operations depending on what changes, and the spec treats them separately throughout:
-
Binary plane β the new cartridge has the same io contract (same descriptor
ioblock, samefamily): swapping is a package swap. Nothing recompiles; generated typed bindings (typed facade) stay valid. Example: the Whisper-on-NPU cartridge with theenmodel set β the same cartridge with ademodel set; or a requantized weights drop of the same model. -
Profile plane β the new cartridge has a different io shape but the same
task: swapping happens behind a versioned task profile. The adapter (or runner) compiles against the profile, not the cartridge; the capability delta (languages, streaming, endpointing) is machine-readable as a descriptor diff and must be an accepted tradeoff, never a silent loss. Example: batch Whisper-on-NPU β streaming Moonshine.
A swap that changes the task is not a swap; it is a different capability.
The cartridge contract
[drafted] β v0.2 revises the native-speed and endpointing clauses; see Changelog.
A cartridge MUST
-
Be self-contained β bundle its own native runtime and model artifact(s). Nothing outside itself except the OS and the hardware driver. A dependency on a specific driver build or version (e.g. a patched NPU driver) is permitted but MUST be declared in
requirements.driverand MUST be checkable at runtime via the ABI’s preflight (ctg_preflight) β a build machine cannot see the fleet’s drivers. -
Use the fastest execution path available for its declared
execution_mode. Forexecution_mode: native, that means the HW-specific path the target offers (an NPU vendor compiler; ORT with the ABI’s best kernels; IREE for a supported target). Forexecution_mode: eager, portability is the point: a correct, unoptimized reference path that runs anywhere is contract-compliant β the eager tier is a legitimate cartridge, not a degraded one (see Packaging, target resolution, and the eager tier). In both modes, performance is honestly reported as measured (see Anatomy of a cartridge), never promised. -
Be application-agnostic β its API is model in β model out. It contains no application types, no product contracts, no domain logic belonging to any one product. This is a structural requirement, about the interface and code, not a licensing one; whether the model weights are open, commercially licensed, or encrypted-and-gated is a separate, per-cartridge choice (Licensing: two rules).
-
Expose the Cartridge Runtime ABI β the C call-level contract defined in the Runtime ABI page: lifecycle, batch and streaming io, cancellation, preflight, error model, buffer ownership, and threading rules. Package-level metadata makes a cartridge routable; the ABI is what makes it replaceable as a unit. A language-native facade (JNI/Kotlin, a CLI) MAY sit on top, but the C ABI is the portable seam.
-
Be replaceable as a unit β on the binary plane by package swap, on the profile plane behind a task profile (Replaceability has two planes).
-
Declare its capabilities β a machine-readable descriptor (Anatomy of a cartridge) so hosts can select among cartridges without hard-coding.
-
Be verifiable β ship with (or resolve to) a signed manifest of content digests covering every artifact it bundles, so a host can prove this is the original, unmodified cartridge before it ever runs (Integrity, provenance, and weight-only updates). Designed in from the first cartridge, not retrofitted.
-
Declare the licenses it stands on β the manifest records a license per artifact and the cartridge’s effective license, which MUST satisfy every upstream obligation (Licensing: two rules). An artifact with no declared license is a packaging error, not a default.
A cartridge MUST NOT
-
Contain application policy: session lifecycle, business rules, intent/command formats, product-specific vocabularies, or any closed/proprietary logic. Those belong to the adapter.
-
Make endpointing decisions β but it MAY contain signal-level endpointing machinery. The ruling (ADR-003): a cartridge MAY embed VAD / speech segmentation when it is intrinsic to how its model streams, and MUST surface it as data β segment boundaries, endpoint events, speech probabilities on its output stream β never as decisions. "Speech ended at t=3.1s" is cartridge output; "therefore the session is over" is adapter policy. The descriptor declares which machinery is inside via
attributes.endpointing: none | signal | semantic. Grounding: the ORT Moonshine cartridge runs Silero VAD + utterance segmentation internally and emitsLineCompletedevents (endpointing: signal); the SKaiNET/IREE Moonshine v2 engine has no VAD and relies on the host’s stop signal (endpointing: none). Both are contract-compliant; hiding either fact would not be.
|
Boundary test. If a line of code in a cartridge references both a runtime primitive and an application concept (a session, a command schema, a product intent), it is in the wrong layer β and, because cartridges ship publicly, it has also leaked closed application logic into a public component. |
Licensing: two rules
[drafted]
Rule 1 β app-agnosticism and open weights are separate axes
App-agnostic (structural: no product logic inside) and open-source (a licensing/business choice about the weights) are two different axes, and the contract only requires the first. A cartridge can be app-agnostic with open weights, or app-agnostic with commercially licensed / encrypted weights (Integrity, provenance, and weight-only updates, "Encrypted weights") that only decrypt once the host resolves a valid entitlement key. What must never happen is licensing terms leaking into the cartridge as code β a license-check, a call-home, a feature gate keyed off a product identity. Those are application policy and belong in the adapter or a provisioning service.
Rule 2 β the effective license must fulfill every upstream license
A cartridge is a derived work of everything it bundles: model weights (usually derived from a
published checkpoint), calibration data, vendored runtime code, tokenizers. The manifest
therefore records, per artifact, a license (SPDX expression or a marker like proprietary /
community-restricted) and derived_from (what upstream it descends from). The cartridge’s
declared effective license MUST be satisfiable given all of them β attribution carried, share-
alike honored, notices shipped, usage caps respected. A build-time license gate enforcing this
is Phase-2 tooling (roadmap); the manifest fields exist now.
Worked reality checks from the existing cartridges:
| Case | Fact |
|---|---|
|
Has no license file at all: unlicensed first-party code, Whisper-derived NPU weights
with no OpenAI/MIT attribution, and Apache-2.0 vendor sample code modified in-tree without
the modification notices Β§4(b) requires. The public description of it as "Apache-2.0" is
currently aspirational. Remediation is a Phase-3 roadmap item; until then its descriptor
carries |
|
The bundled |
The descriptor’s license field states the effective license: an SPDX id for open weights;
proprietary or a license/entitlement URL for gated ones (paired with an encryption block on
the relevant artifact); NOASSERTION for a not-yet-remediated package β which conformance
tooling treats as a failure, by design.
The three layers
[working]
Application any app (a voice assistant, a CLI, a test harness, another product)
β
βΌ
Adapter app-specific glue: the application's contract + policy
β (session lifecycle, NLU, command assembly, fallback) β CLOSED, per-app
βΌ
Cartridge self-contained; runs one model on given HW
(native runtime + model + ABI + capabilities) β PUBLIC, app-agnostic
-
Cartridge β public, reusable, app-unaware. Model in β model out.
-
Adapter β the thin, app-specific layer that plugs a cartridge into one application’s contract, via a task profile where one exists. It is where the closed, product-specific code lives, and the only layer that knows both "cartridge" and "application". (In the first consumer app: a voice-engine implementation that drives a cartridge, runs NLU, and assembles the app’s command format.)
-
Application β owns the user, the UI, the product decisions.
The split is a distribution boundary as much as an architectural one: a cartridge’s code, interface, and manifest are always public and app-agnostic; adapters and applications are app-specific and may be closed source. Whether a cartridge’s weights are also free is a separate, per-cartridge licensing choice β Licensing: two rules.
Anatomy of a cartridge
[drafted] β descriptor schema v0.2; the quality and model blocks are new.
| Part | What it is |
|---|---|
Native runtime |
the inference engine for the target: an NPU driver stack, ORT, IREE, β¦ packaged as the cartridge’s own libraries |
Model artifact(s) |
the compiled/quantized model in the runtime’s format (a vendor NPU blob, |
Runtime ABI |
the C call-level contract (Cartridge Runtime ABI v0.1) every cartridge exports; language facades may wrap it |
Capability descriptor |
machine-readable metadata a host routes on β schema:
|
Signed manifest |
content digests + per-artifact licenses + signatures (Integrity, provenance, and weight-only updates) β schema:
|
Capability descriptor (v0.3 shape)
spec_version : "0.3"
id, version : identity (OQ3 naming) + semver
execution_mode : native | eager
family : model-family identity β the binary-plane grouping (same family + same io
= package-swappable; also scopes weight-update applicability)
model : name, family, variant, params, checkpoint, upstream {source, license}
license : EFFECTIVE license (see Licensing, Rule 2)
modality, task : audio/asr, image/detection, β¦
io : input/output tensor or stream contract β not application terms
target : hardware + ABI (+ accelerator) the cartridge runs on
performance : measured, not promised β rtf, latency, measured_on, methodology (a pointer
to the workload/benchmark definition that produced the number)
quality : task-level quality, measured β for asr: WER + dataset + harness refs.
MANDATORY for task: asr. Without it, fallback tolerance (OQ9) and
weight-update regression gates are unanswerable.
attributes : task-specific β languages, streaming, endpointing (none|signal|semantic),
context window, label set, β¦
requirements : accelerator?, driver name+version, memory, os β split by WHERE they are
checkable: build-time (abi, packaging) vs runtime preflight (driver, memory)
flavors : (ADR-014) additional measured variants sharing this identity β another
bundled language, runtime tier, or ABI β each its own id, overriding
whichever of attributes/model/performance/quality/target/requirements
actually differ; everything else (task/family/io/license/β¦) is shared
Performance and quality are measured on the stated target and reported honestly, including
when they are not good enough (a cartridge may be RTF β 2 on one ABI and β 0.25 on another β
both are true, both are stated). measured_on names the device/run; methodology points at
the workload definition so the number is falsifiable, and internal infrastructure details
(device IPs, hostnames) stay out of public descriptors.
flavors (ADR-014) is how one package covers several independently-addressable,
independently-measured identities β Moonshine ships one AAR with an English base and a
German flavor (different upstream checkpoints, different measured RTF); FunctionGemma ships
one AAR with a Vulkan base and a CPU flavor (4Γ slower, no accelerator required). The base
descriptor’s own fields stay the default flavor unchanged, so a host that predates ADR-014
still resolves correctly against them β flavors is purely additive. A flavor field, when
present, replaces the base’s field of the same name in full (no deep merge); a host treats
{base} βͺ {each flavor} as the complete set of identities the package exposes, applying
every per-descriptor rule (mandatory quality for task=asr, etc.) to each one individually.
Cartridge as a service
[direction]
Producing a cartridge is hardware-specific compilation: a model + a target β a validated, packaged cartridge. That is inherently backend-plural β one NPU vendor’s path is its own toolchain, other NPUs/targets need other compilers, CPU needs ORT, some targets go through IREE.
The NPU cartridge builder is already a cartridge builder for one backend β the vendor NPU compiler: model β NPU blob β validated with the vendor’s inspection tool β verified on-device over adb. Its calibrate / convert flow, and the resulting NPU blob + tokenizer + mel filters, are exactly a Whisper-on-NPU cartridge’s model side.
The general form β "cartridge as a service" β is a build service with pluggable compiler
backends (a vendor NPU compiler, ORT, IREE, β¦) that takes (model, target) and emits a cartridge with its
capability descriptor filled from the build. A forward direction, not a current deliverable.
Integrity, provenance, and weight-only updates
[drafted] β mechanism proven by runnable example; v0.2 fixes the update-verification gate. Threat model and deployment shapes: Threat model β who verifies what, where.
A cartridge runs with native-code and NPU privileges on the host. Before a host loads one it must be able to prove two things: this is the artifact the publisher actually built (integrity) and, once weight-only updates exist, this update legitimately descends from that artifact (provenance/lineage).
Content addressing
Every artifact a cartridge bundles (native runtime, model weights, tokenizer, calibration side
data, the capability descriptor itself) is identified by a SHA-256 digest, not by name or path
alone. A manifest lists every artifact’s role, path, digest, size, and license
(Licensing: two rules, Rule 2), plus provenance (producer, build time, build inputs). The manifest’s
own digest is computed over its canonical JSON with the signatures field excluded β that
digest is what gets signed, and it is also how one manifest pins another: by digest, never by a
mutable tag.
Granularity rule: an artifact is addressable only if it is a distinct file. A blob baked into
a compiled binary (the ORT Moonshine cartridge bakes its 21.4 MB Silero VAD model in as a C
array) is covered only by the containing binary’s digest β meaning it cannot be
weight-updated or license-tracked independently. Cartridge builds SHOULD keep model data as
file artifacts; where baking in is unavoidable, the manifest records the containing binary’s
derived_from accordingly.
Signing
The manifest carries one or more detached signatures (keyid, alg, signer, signature,
signed_at). Ed25519 over the manifest digest is the default choice: small keys and
signatures, fast to verify on a constrained device. A host verifies a cartridge before load by:
recomputing every artifact digest and comparing it to the manifest, recomputing the manifest
digest, and checking the signature(s) against a trust store of keys it already holds for that
cartridge id β pinned at provisioning time or delivered via the registry (OQ4), never shipped
inside the artifact it is meant to authenticate. Where verification actually runs, per
deployment shape, is defined in the threat model β for the common
build-time-staged shape it is the CI staging step, not the device, that runs this check.
A runnable proof lives in examples/: schema, signed example manifests, and
verify_manifest.py (stdlib+openssl, no crypto dependency). Legitimate manifests verify;
flipping a single field breaks the signature.
Weight-only updates (federated learning and beyond)
[direction] β modeled and schema-supported; no producing pipeline exists yet in this ecosystem. The design is here so the first fine-tune or federated round lands in a prepared shape, not so anyone believes it runs today.
The runtime + tokenizer + code of a cartridge changes rarely; the weights are expected to change often β a new fine-tune, a quantization pass, or a federated-learning aggregation round. Weight-only updates are modeled as a distinct, separately-signed object:
-
kind: weights-update, carrying only the changed artifact(s) (a full weights blob, or later a delta/adapter such as a LoRA update β OQ7). -
base: {kind, id, version, digest}β pins the exact base cartridge (or prior update) it applies to, by digest. A host refuses to apply an update whosebase.digestdoesn’t match what is currently installed β this stops rollback and base-substitution attacks and makes a chain of rounds an auditable, tamper-evident lineage. -
provenance.trainingrecords round number, aggregation method, participant count, optionally a differential-privacy budget. -
provenance.evaluationrecords the regression gate (below).
Verification gate (v0.2 β replaces the v0.1 wording). A weight update is supposed to change outputs β "parity with the previous weights" is a vacuous check. What a weight update MUST pass before publication:
-
Structural parity of the new weights against the float reference of the new checkpoint, within stated tolerance (same check a base cartridge runs β the Minerva host-verification pattern).
-
Task-metric regression gate against the previous weights on a pinned, digest-identified evaluation set: the task metric (WER for asr) must not degrade beyond a stated threshold. The result β dataset digest, metric, new value, baseline value β is recorded in
provenance.evaluation. A degradation may be shipped only as an explicit, recorded waiver, never silently.
This also motivates a second trust role: the party allowed to sign a base cartridge (the
publisher) is not necessarily the party allowed to sign a weight update for it (a
training/aggregation service). A host’s trust policy authorizes keys per role per cartridge
id β the shape TUF uses for root vs. targets keys (OQ6). The worked example signs the base
with a publisher key and a federated round-17 update with a separate aggregator key.
Encrypted weights
Signing proves authenticity and integrity; it does not provide confidentiality. For weights
that are commercially sensitive, the manifest supports an optional encryption block on a
weights artifact: an algorithm (e.g. AES-256-GCM) and a key reference β never the key itself
β that the host resolves through its own provisioning (a secure element, a device key store).
Encryption and signing are independent controls.
This pattern already exists in this codebase for a different target: SKaiNET’s Minerva
secure-MCU export (skainet-compile-minerva in
SKaiNET) provisions a device key
outside the generated bundle, redacts key material from its manifest, and verifies inference
against that key. The cartridge manifest generalizes exactly that pattern β external key,
redacted from the shipped artifact, resolved by the host at load β to NPU/ORT/IREE cartridges.
Packaging, target resolution, and the eager tier
[drafted] β the staging step itself is Phase-2 tooling (Roadmap).
A cartridge is built per (model, target); an application is usually built for several
targets at once. Resolving "which cartridge artifact for which target" is a build-time
concern β with one deliberate exception: requirements only a device can answer (driver
version, free memory, accelerator presence) are checked at runtime via
ctg_preflight, before first load, not discovered as an inference
crash. The build step resolves and verifies what is knowable at build time; preflight verifies
the rest where it is knowable.
A build-time staging step is expected, not a shortcut
Packaging needs an explicit build step (a Gradle task in the JVM/Android world) that per declared target: resolves the matching cartridge variant, verifies its manifest (Integrity, provenance, and weight-only updates), stages its artifacts into the app bundle, and generates the typed bindings (typed facade) from the descriptor. Precedents already working in SKaiNET: Minerva’s gated host-verification task and the NPU cartridge builder’s convert-then-verify flow. If a declared target has no compatible cartridge for a required capability, the build step is where that surfaces β as a build-time warning or failure, not a runtime crash on a user’s device.
An artifact marked delivery in the manifest (ADR-009: Google Play Asset Delivery, Apple
On-Demand Resources/Background Assets) is verified here exactly like any other β the only
difference is where the staging step puts it afterward: a platform asset pack instead of
the main bundle. See threat-model.adoc, Shape Aβ² for what changes
(and what doesn’t) about verification once such an artifact lands on-device.
Resolution and the eager tier
Selection among candidate cartridges for a required capability, per target:
-
Hard constraints β
taskmatches; every required attribute is satisfied (required language must be inlanguages, required streaming mode, β¦);targetcompatible with the build target;requirementssatisfiable. -
Tier preference β
execution_mode: nativefor the target when available; otherwiseexecution_mode: eager(a portable reference implementation, the universal fallback tier). A tier downgrade is substituted automatically but reported loudly β a build warning, so a silent perf downgrade stays visible. -
Soft preferences β measured quality, latency, footprint.
family (v0.2) replaces v0.1’s fallback_of. fallback_of made the fallback enumerate the
natives it stood in for β so every new native cartridge forced an edit + re-sign + re-release
of its fallback, and the worked example (Moonshine standing in for Whisper) violated the
"same model family" definition anyway. Under v0.2:
-
familystates model-family identity (e.g.whisper-tiny) β it scopes binary-plane swaps and weight-update applicability. It does not encode fallback relationships. -
Fallback is not declared at all; it is resolved β any same-
taskcartridge that passes the hard constraints is a candidate, tier preference picks native over eager. Whether a capability-diminished candidate (fewer languages, worse WER) is acceptable is exactly the hard-constraint set the host declares β OQ9 becomes "which attributes does the host mark required", a per-host policy, not a cross-cartridge annotation.
Developing a cartridge
[working] β the loop below is how the two existing cartridges were actually produced.
1. Author / import the model SKaiNET DSL, a traced forward pass, or an ONNX import
2. Gather REAL calibration data representative inputs, not random β random data calibrates
to noise (Whisper precedent: real command-shaped audio
via FLEURS, the NPU builder's calibration step)
3. Compile for the declared target backend-specific: the NPU builder, an ORT export (CPU), an
IREE HAL bridge β produces the model artifact
4. Assemble the package runtime + model + side data + ABI implementation;
draft the descriptor with performance AND quality BLANK
5. Verify (a) host-side parity: reference float output vs. compiled/
quantized output, within a stated tolerance
(b) on-device functional run on the real target hardware
(the NPU builder's on-device verify over adb)
(c) task-level quality on a pinned eval set (WER harness
for asr) β parity within tolerance does NOT measure
task quality; both are required
6. Fill in performance + quality from 5(b) and 5(c) β never invented, never a spec sheet
7. Build the manifest and sign it content-addressed digests + per-artifact licenses +
Ed25519 signature(s); production keys live in CI (OQ6)
8. Publish via a hub, and/or as buildable source, and/or pre-built
internal β see Consuming below
Weight-only updates loop back into this cycle without restarting it: new weights re-enter at
step 2 (new training data) or step 4, run the v0.2 verification gate from Integrity, provenance, and weight-only updates
(structural parity vs. the new float reference plus the task-metric regression gate vs. the
previous weights), and produce a weights-update manifest pinned to the published base’s
digest β signed by whichever key is authorized for that role.
Consuming a cartridge
[drafted]
A consumer gets a cartridge one of three ways. All three resolve to the identical on-disk shape β descriptor + artifacts + manifest β so the staging step never needs to know which one supplied a given cartridge, and none of the three change what gets verified before load β only how the bytes got there and how much curation sits on top.
Mode A β a cartridge hub [direction]
A portal that lists cartridges (searchable by descriptor fields) and serves them:
content-addressed artifact storage, a resolvable "channel" per cartridge id (latest-stable,
round-latest) that a consumer resolves to one immutable, signed digest before ever trusting
it β the channel name is never itself what’s verified, the same discipline a container tag
needs against --digest β and a directory of trust roots (OQ4, OQ6). For a commercially
licensed cartridge, the hub is the entitlement gate: the descriptor stays publicly listed, but
resolving the weights artifact’s key_ref requires a valid entitlement β it gates bytes, not
metadata.
Mode B β clone a source repo and build it yourself [working]
No hub required. A cartridge’s source repo (the shape the two reference
cartridges already follow) bundles the packaging step itself, so
git clone && <build> produces a conformant package locally.
This unlocks a trust path independent of signatures: if the build is reproducible, a consumer can rebuild from source and check that their own artifact digests match the digests in a manifest signed upstream, verifying the cartridge without trusting the channel it arrived through at all. Sign what a consumer can’t feasibly rebuild (encrypted commercial weights, a large training run); let reproducibility carry the rest. Whether today’s builders are actually deterministic end-to-end is OQ11.
Mode C β pre-built, internal distribution [drafted]
For a single company shipping to its own products, the common case: build once (through step 7 β package + signed manifest) and drop the package, as-is, onto whatever artifact storage the company already operates β Artifactory/Nexus, a GitLab/GitHub Package Registry, an S3 bucket β no dedicated search UI, entitlement gate, or discovery portal. Consumers already know which cartridge id/version they need and pull it directly.
This is Mode A with the discovery-and-entitlement layer removed, not a lesser version of it. What survives unchanged is the only part that matters for trust: the manifest and its signature(s) still get verified before load. For a single-publisher internal setting the trust store is trivial: pin the company’s own key(s) once β baked into build images, provisioned on managed devices. This is the practical answer to OQ4 for the internal audience.
Multiple cartridges β selection is the host’s job
[drafted]
Once more than one cartridge exists for overlapping capability, a host must choose which cartridge runs. That routing is application/adapter policy β cartridges are peers that know nothing about each other; a cartridge must never contain "if not me, use the other one". The host routes on capability descriptors per the resolution rules in Packaging, target resolution, and the eager tier.
Two layers of compatibility
"Can I replace Whisper with Moonshine?" splits into two questions with different answers β the two planes made concrete.
Cartridge-level (binary-plane) compatibility rarely holds across model families, and isn’t
the goal. The Whisper-on-NPU cartridge and the Moonshine engines are all task: asr, but their io
shapes differ structurally: batch 4 s log-mel window vs. streaming PCM chunks; [en, de] vs.
[en]; different tokenizers. None of that is a bug β the descriptor exists so the difference
is machine-readable and honest, not so unrelated cartridges fake an identical tensor API.
Profile-plane compatibility is the actual swap point. The application never talks to a
cartridge’s raw io; the adapter absorbs the difference behind one stable, per-task interface β
the task profile. In the first consumer app today both adapters implement the same
closed voice-engine contract and the app discovers whichever is wired in by reflection; the public
asr/v1 profile formalizes exactly that seam without the app-specific residue.
What a swap costs. The profile hides the shape difference; it cannot hide a capability difference. Moving from Whisper to an English-only streaming engine means losing German and changing the UX from one final transcript to incremental results β a real product change, surfaced as a descriptor diff and accepted explicitly, never silently.
Existing engines and their cartridge status
[working] β corrected in v0.2; the v0.1 text conflated the two Moonshine engines.
| Engine | Runtime / target | Cartridge status |
|---|---|---|
Whisper-tiny ASR / vendor NPU β |
vendor NPU, 32-bit ARM SoC / armeabi-v7a; |
Working cartridge in fact; RTF < 1 on the device; en (+ de code path, de models not yet committed). ABI + licensing retrofit = roadmap Phase 3. |
Moonshine streaming ASR / ORT β |
ONNX Runtime, CPU (v7a/v8a/x86_64); |
Working, genuinely app-agnostic ( |
Moonshine v2 streaming ASR / IREE β SKaiNET-DSL engine ( |
IREE/Vulkan (Mali GPU), armeabi-v7a; no VAD ( |
The engine actually wired into the first app’s Moonshine adapter today β via a vendored
|
Important correction (v0.2): the first app’s Moonshine adapter does not consume the ORT Moonshine cartridge β the streaming feature moved to the SKaiNET/IREE Moonshine v2 engine for performance on the 32-bit device; the ORT engine is superseded, its architecture carried over. The ORT cartridge remains the eager-tier ASR cartridge and the VAD-bearing exemplar.
The NPU vision demos (MobileNetV2, YOLOv8) are cartridge-shaped and could be promoted the same way.
What is not a cartridge
[working]
-
A raw model file (
.onnx, GGUF) β no runtime, doesn’t run itself. -
A framework β SKaiNET itself authors and runs models; it is the universe the cartridge lives in, not a cartridge.
-
A compiler / build tool β the NPU cartridge builder produces cartridge model artifacts; it is not one.
-
An app engine that embeds product logic β that is an adapter, not a cartridge.
Relationship to the rest of the SKaiNET universe
[working]
SKaiNET (KMP tensor framework) author / import / run models; calibration + execution β model + export (ONNX / GGUF / HLO) + calibration data / observed tensors βΌ Cartridge build ββ NPU builder Β· ORT Β· IREE HAL bridges ββ HW-specific compile + package β βΌ Cartridge (public, HW-native, app-agnostic) β exports the Runtime ABI β βΌ Adapter (app-specific, may be closed) β Application
-
SKaiNET is where models are defined, imported, and run, and where representative tensors / calibration come from. It is also the source of the type vocabulary the typed cartridge facade mirrors (task-profiles.adoc#typed). See the SKaiNET documentation.
-
The NPU builder and the IREE/ORT backends are the compile/package side β the makers of cartridges.
-
The cartridge is the packaging that makes any of the above consumable by an application.
-
The adapter (e.g. the first app’s ASR engines) is the app-specific consumer.
Open questions
[drafted] β statuses updated for v0.2; resolved items point at their ADR.
-
OQ0 β Runtime ABI. Resolved (drafted) by Cartridge Runtime ABI v0.1 / ADR-002. Was the spec’s largest hole: package metadata was normative, the call-level contract wasn’t.
-
OQ1 β Descriptor schema. First formal draft shipped in v0.1; v0.2 adds
model,quality,family,endpointing,methodology,spec_version. Remaining: per-task attribute vocabularies beyondasr. -
OQ2 β Packaging format(s). Open. Likely per-platform (AAR,
.so+assets, KMP artifact), with a common descriptor+manifest. -
OQ3 β Naming & versioning.
<task>-<model>-<runtime>-<target>+ semver, used throughout. Still to formalize: what constitutes a breaking (major) change β an io change is; is a language drop? -
OQ4 β Registry. Settled for internal distribution (Mode C: reuse existing artifact storage). Open for an external/multi-tenant audience.
-
OQ5 β Conformance. Now specifiable (the ABI gives a conformance test something to invoke):
ctg-conformis Phase-2 tooling β schema-validate, verify manifest, dlopen, ABI version check, smoke inference against bundled vectors. -
OQ6 β Key management / trust roots. Publisher vs. trainer/aggregator keys per role per cartridge id β likely a TUF-shaped root/targets split. Note v0.2 threat model adds the missing base-cartridge anti-rollback concern (TUF snapshot/timestamp shape) to this question.
-
OQ7 β Weight-update semantics. Full replacement blob vs. delta/adapter (LoRA); fixed convention or per-cartridge choice.
-
OQ8 β Revocation. Compromised key, bad round, withdrawn version, on hosts that may be offline. Parked (
direction, see roadmap). -
OQ9 β Fallback tolerance. Reframed by v0.2 resolution rules: fallback is resolved, not declared; tolerance = which attributes the host marks required. Remaining: a sensible default required-set per task profile.
-
OQ10 β Commercial licensing field(s). Formalize the closed/commercial shape of
license
entitlement pointers. v0.2 adds the concrete worked case (Moonshine Community License: revenue-capped commercial use, display duty) that the field structure must be able to carry. -
OQ11 β Reproducible builds. Whether the NPU builder / an ORT export / an IREE compile can be made bit-for-bit deterministic β prerequisite for the Mode-B trust path to mean anything beyond "I trust my own laptop."
-
OQ12 β Task profiles. Resolved (drafted) by Task profiles / ADR-005 for
asr/v1and ADR-008 foryolo/v1(detection);embeddingandttsremain open. -
OQ13 β On-demand delivery failure as a first-class capability? ADR-009 (platform asset delivery) leaves open whether an
on-demand-timed artifact that never arrives (offline device, user declines) deserves descriptor-level vocabulary of its own, or stays a plain application-level failure β Multiple cartridges β selection's existing hard-constraint framing may already be enough.
Changelog
v0.3 (2026-09-09) β descriptor flavors (ADR-014): one package can now cover several
independently-addressable, independently-measured identities (a bundled language, a runtime
tier, an ABI) sharing one task/family/io/license identity, instead of needing one descriptor
file per variant. Migrated in the same pass: Moonshine’s en/de descriptors and
FunctionGemma’s Vulkan/CPU descriptors, each collapsed from two files to one;
the atlas generator updated to expand flavors into one doc per
identity. ADR-007 (structured output + LLM model metadata) also flipped proposed β accepted
this pass.
v0.2 (2026-08-25) β Runtime ABI introduced as a MUST (OQ0, Cartridge Runtime ABI v0.1); two-plane
replaceability made normative; native-speed MUST made conditional on execution_mode
(eager tier is contract-compliant); endpointing ruling: signal-level VAD allowed as data, never
as decisions (attributes.endpointing); fallback_of replaced by family + resolution rules;
descriptor v0.2 (model, quality β mandatory for asr, methodology, requirements.os,
spec_version); manifest v0.2 (per-artifact license + derived_from, effective license,
provenance.evaluation); weight-update verification gate corrected (parity-vs-previous
replaced by float-parity + task-metric regression gate); licensing Rule 2 (license composition)
added with worked cases; threat model split out (Threat model β who verifies what, where) with deployment
shapes and verification placement; task profile asr/v1 drafted (Task profiles);
two-Moonshine-engines correction; maturity tags added; internal device IP scrubbed from
examples; ADR log started.
v0.1 (2026-07) β initial concept definition, contract, descriptor + manifest first drafts, signing example, consumption modes A/B/C, compatibility layers, OQ1βOQ12.