Reflective Documentation for SKaiNET Operators

Explain why documentation is part of product design for SKaiNET.

Introduce the “Operator” abstraction and the need for reflective docs that track interface shape and backend coverage.

State the pain points this solves (code drift, stale status matrices, ambiguous backend coverage).

Define “reflective documentation”: combining machine-generated metadata (from the codebase) with human-written narrative.

Summarize how projects like Sphinx (Python) treat API docs and cross-links as a pipeline; use Sphinx only as an analogy (no Python implementation).

Contrast traditional docgen vs. duplex-reflective docs (two-way awareness between code + docs).

Describe using Kotlin Symbol Processing (KSP) to scan Operator interfaces and functions.

Declare two annotations for backend implementation status: [source,kotlin] ---- @Target(AnnotationTarget.CLASS, AnnotationTarget.FUNCTION) annotation class NotImplemented(vararg val backends: String)

  @Target(AnnotationTarget.CLASS, AnnotationTarget.FUNCTION)
  annotation class InProgress(
      vararg val backends: String,
      val owner: String = "",
      val issue: String = ""  // e.g., "GH-1234"
  )
  ----
- Semantics:
  • If neither annotation is present for a given backend, treat it as Implemented.
  • @InProgress overrides Implemented for listed backends.
  • @NotImplemented overrides both and marks listed backends as not available.
- KSP output: generate a single JSON metadata artifact per module per build (e.g., build/generated/{module}/operators.json).

Provide a concrete JSON Schema for the metadata (keep realistic but concise). Include at least:

Explain a Gradle task that converts the KSP JSON into AsciiDoc fragments (one per Operator and optionally one per function).

Show how fragments embed:

Provide example AsciiDoc fragment: [source,adoc] ---- // generated: do not edit == TensorOps.matmul

[source,kotlin]
----
fun matmul(a: Tensor, b: Tensor): Tensor
----

.Backend status
|===
| Backend | Status | Notes

| CPU   | Implemented   |
| GPU   | Implemented   |
| TPU   | Implemented   |
|===

See xref:theory/matmul.adoc#definition[MatMul semantics] and xref:examples/matmul.adoc#examples[Examples].
----

Demonstrate combining generated and human-written docs via include:: and xref:, with a small folder layout: [source,text] ---- docs/ modules/ operators/ generated/TensorOps.adoc generated/TensorOps.matmul.adoc theory/ matmul.adoc examples/ matmul.adoc index.adoc ----

Define patterns where human and generated content coexist:

Show a synchronization flow as a Mermaid diagram:

[mermaid]
----
flowchart LR
  A[Operator Interfaces (KMP)] --> B[KSP Processor]
  B --> C[operators.json]
  C --> D[Doc Generator (JSON -> AsciiDoc)]
  D --> E[_generated_ Fragments]
  E --> F[Master Docs (AsciiDoc)]
  F --> G[AsciiDoctorJ/Dokka Output]
  G --> H[Website / PDF]
  F -->|xref| E
----

Optionally include a PlantUML sequence to illustrate “human + generated” merge logic (optional).

Keep the pipeline within Kotlin/JVM/Gradle:

Provide sample Gradle snippets: [source,gradle] ---- plugins { id("com.google.devtools.ksp") version "2.2.21-2.0.5" id("org.jetbrains.dokka") version "2.1.0" id("org.asciidoctor.jvm.convert") version "3.0.0" }

dependencies {
  ksp(project(":tools:ksp-ops"))
  implementation(project(":tools:docgen"))
}

tasks.register("generateOperatorDocs") {
  dependsOn("kspKotlin")
  inputs.file(layout.buildDirectory.file("generated/ksp/operators.json"))
  outputs.dir(layout.buildDirectory.dir("docs/_generated_"))
  doLast {
    // call JSON->AsciiDoc generator
  }
}

tasks.named("asciidoctor") {
  dependsOn("generateOperatorDocs")
  inputs.dir("$buildDir/docs/_generated_")
}

tasks.named("dokkaHtml") {
  dependsOn("generateOperatorDocs")
}
----

Explain how Dokka can:

Provide an example Operator interface and annotations:

[source,kotlin]
----
package ai.{FRAMEWORK_NAME}.ops

interface TensorOps {
    fun matmul(a: Tensor, b: Tensor): Tensor

@InProgress("Metal", owner="ops-team", issue="GH-1234")
fun relu(x: Tensor): Tensor

    @NotImplemented("Experimental")
    fun conv2d(input: Tensor, kernel: Tensor, stride: Int = 1, padding: Int = 0): Tensor
}
----

Show the KSP-produced JSON excerpt and the corresponding generated AsciiDoc fragment for at least one function (e.g., relu).

Give a minimal human-written math section for MatMul (dimensions, shapes, complexity), and show how it is included via xref: from the generated fragment.

Summarize benefits:

Call out risks and mitigations: