What governs this run?

ModelArchitecture

Dimensions and numerical conventions that define the supported GPT computation.

Fixed within an identified model artifact.

Owns: Dimensions and numerical conventions that define the supported GPT computation., Declared explanatory stage templates for supported runtimes; these are not captured observations or timing.. Excludes: request text, generation limits, inspection position.

ModelArchitecture

vocabulary

Supported baseline value.

width

Channels per position.

ffn

Intermediate channels.

context

Maximum input positions in a forward pass.

layers

Executed decoder blocks; each owns its weights and KV state.

heads

Supported baseline value.

epsilon

Supported baseline value.

position

Supported baseline value.

normalization

Supported baseline value.

activation

Supported baseline value.

precision

Supported baseline value.

LlamaArchitecture

vocabulary

integer

width

integer

ffn

integer

context

integer

layers

integer

heads

integer

position

string

normalization

string

activation

string

precision

string

epsilon

number

ArchitectureStage

owner

string

label

string

scope

string

detail

string

inputs

Earlier stage indices in this single-block schematic; residual branches remain explicit.

operations

array

related

array

inspection

string

ArchitectureWalkthrough

stages

array

KVState

Immutable actual keys and values for a prefix, bound to checkpoint, tokenizer, positions, and precision.

Immutable snapshot; continuation creates a new snapshot. Residency is separate.

Owns: Immutable actual keys and values for a prefix, bound to checkpoint, tokenizer, positions, and precision.. Excludes: resource placement, cached answers, service timing.

KVState

checkpoint_sha256

string

tokenizer_sha256

string

precision

string

token_ids

array

keys

Layer, prefix position, channel.

values

Layer, prefix position, channel.

KVStateOperation

name

string

ModelArtifact

An immutable inference artifact binding numerical weights to architecture, tokenizer identity, and preparation provenance.

Immutable for an artifact identity. Preparation seed is provenance, never a request control.

Owns: The full inference payload and its lightweight identity reference., Preparation provenance attached to the artifact.. Excludes: request text, generation limits, inspection position.

ModelArtifact

config

ModelArchitecture · owned by ModelArchitecture

format

string

manifest_sha256

string

parameters

ModelWeights · owned by ModelWeights

provenance

Declared alternatives

ModelArtifactReference

checkpoint_sha256

SHA-256 identity of the exact artifact bytes or canonical checkpoint.

tokenizer_sha256

SHA-256 identity of the exact artifact bytes or canonical checkpoint.

architecture

ModelArchitecture · owned by ModelArchitecture

InitializationProvenance

initializer

string

kind

string

seed

integer

TrainingProvenance

kind

string

seed

integer

preparation_run_sha256

string

step

integer

selection

string

development_loss

number

data_sha256

string

source_sha256

object

ModelAsset

path

string

sha256

string

bytes

integer

ModelStorage

base_url

Relative URL or HTTP(S) object-storage prefix; identical fetch protocol locally and remotely.

ModelDescriptor

id

string

label

string

role

string

parameters

integer

architecture

Declared alternatives

weights

ModelAsset · owned by ModelArtifact

tokenizer

ModelAsset · owned by ModelArtifact

seed

integer

inspection

string

stop_ids

array

normalization

string

provenance

string

weight_format

string

tokenizer_format

string

ModelCatalog

storage

ModelStorage · owned by ModelArtifact

models

array

ModelSelection

model_id

string

ModelLoadResult

model

ModelDescriptor · owned by ModelArtifact

load_ms

number

token_pieces

array

parameters

Declared alternatives

ModelSource

id

string

role

string

seed

integer

provenance

string

weights

string

tokenizer

string

weight_format

string

tokenizer_format

string

ModelSources

storage

ModelStorage · owned by ModelArtifact

models

array

ModelWeights

Numerical tensors consumed by the model: embeddings, projection weights, biases, and normalization scales.

Immutable within its identified artifact or recorded execution.

Owns: Numerical tensor values indexed by model operation; block tensors have an explicit layer axis.. Excludes: Architecture dimensions and execution conventions, Request and generation configuration, Per-request KV state.

ModelWeights

attention_output

Layer-indexed block tensor.

attention_output_bias

Layer-indexed block tensor.

ff_down

Layer-indexed block tensor.

ff_down_bias

Layer-indexed block tensor.

ff_up

Layer-indexed block tensor.

ff_up_bias

Layer-indexed block tensor.

key

Layer-indexed block tensor.

key_bias

Layer-indexed block tensor.

lm_bias

array

lm_head

array

ln1_bias

Layer-indexed block tensor.

ln1_scale

Layer-indexed block tensor.

ln2_bias

Layer-indexed block tensor.

ln2_scale

Layer-indexed block tensor.

ln_final_bias

array

ln_final_scale

array

position_embedding

array

query

Layer-indexed block tensor.

query_bias

Layer-indexed block tensor.

token_embedding

array

value

Layer-indexed block tensor.

value_bias

Layer-indexed block tensor.

InferenceSystem

The configured execution boundary that accepts inference requests and produces their outputs.

Fixed for an identified system configuration.

Owns: Composition of separately owned execution, scheduling, cache, transfer and compute-service configurations., System-wide supported request and output-token ceilings., The shared containment hierarchy of ontology-owned logical components and mechanisms, with cross-component relationships., Declared dispatch boundary; the baseline uses one direct in-process execution endpoint.. Excludes: Constituent mechanisms and their observations., Caller-requested generation policy and response publication., Model weights and measured results., Containment does not claim separate processes, execution order or support for configuring every component..

InferenceMap

nodes

array

relationships

array

InferenceMapNode

id

string

parent

['string', 'null']

owner

string

kind

string

view

string

expanded

boolean

InferenceMapRelationship

source

string

target

string

label

string

KVResidencySystemConfig

max_requests

integer

max_new_tokens_cap

integer

scheduling

RequestSchedulingConfig · owned by RequestScheduling

kv_cache

KVCacheConfig · owned by KVCache

transfer

StateTransferConfig · owned by StateTransfer

compute_service

ComputeServiceConfig · owned by ComputeService

GenerationPolicy

Per-request output-token budget and supported selection and termination rules.

Editable per request before submission; immutable in that request’s run binding.

Owns: Per-request output-token budget and selection and termination settings., Recorded identity and explanation of the policy used by an execution.. Excludes: model initialization, system capacity, Backend execution strategies such as speculative drafting and verification, observed outcomes.

GenerationPolicy

max_new_tokens

Maximum number of selected IDs, including an end token. May stop earlier.

selection

Supported baseline value.

stopping

Stop at the configured model end token or output budget; context capacity also bounds execution.

InferenceRequest

One invocation with input and per-request generation settings, identified at its declared arrival.

Editable before submission; immutable in a run binding.

Owns: Request input and per-request generation settings., Request identity and declared arrival time; arrival does not imply admission or execution.. Excludes: model initialization, system capacity, observed outcomes.

InferenceRequest

text

User text; normalization is declared by the selected tokenizer.

generation

GenerationPolicy · owned by GenerationPolicy

RequestArrival

request_id

Request identity

arrival_us

Declared virtual arrival in microseconds

request

InferenceRequest · owned by InferenceRequest

GenerationInput

execution

ExecutionSpec · owned by ExecutionSpec

workload

Workload · owned by Workload

GenerationPrepared

prompt_ids

array

pieces

array

input

GenerationInput · owned by InferenceRequest

encoding

Declared alternatives

GenerationStep

execution

ExecutionSpec · owned by ExecutionSpec

prefix_ids

array

output_ids

array

generation

GenerationPolicy · owned by GenerationPolicy

KVExperimentInput

system

KVResidencySystemConfig · owned by InferenceSystem

workload

Workload · owned by Workload

ontology_binding

OntologyBinding · owned by ExperimentSpec

Workload

An ordered collection of request arrivals; array order breaks equal-time ties.

Immutable in an identified execution record.

Owns: An ordered collection of request arrivals; array order breaks equal-time ties.. Excludes: Actual admission or execution order chosen by the scheduler, Generated outputs and measured completion times.

Workload

kind

Arrival policy

requests

Ordered requests

InferenceResponse

The output published for one request and its observed completion outcome.

Immutable execution evidence, accumulated as output is published and the request finishes.

Owns: Selected token IDs published to the modeled output collector and their diagnostic display contribution., Request completion, selection and publication counts, unused budget, and termination reason.. Excludes: Unpublished selections, including TURN_END, and any future speculative candidates, Internal tensors and KV state, Network receipt or acknowledgment; current publication is in-process publication..

RunOutcome

selected_tokens

Includes terminal controls.

published_tokens

Every selected ID except TURN_END, including diagnostic control/byte/hex tokens.

unused_budget

Requested ceiling minus actual selected ID count.

stop_reason

Recorded by the execution loop.

complete

True only if TURN_END was selected.

RequestEndpoint

Accept requests and publish responses at the inference service boundary.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: Request and response contracts, streaming and cancellation semantics., Its typed boundary in the shared configuration and explanation hierarchy.. Excludes: Routing decisions, model computation, and customer-side receipt., The boundary declaration alone provides no executable experiment or numerical evidence..

RequestEndpointBoundary

id

string

parent

['string', 'null']

owner

string

kind

string

view

string

expanded

boolean

RequestRouter

Select an execution destination for an admitted service request.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: Destination selection and affinity to model availability, capacity and resident state., Its typed boundary in the shared configuration and explanation hierarchy.. Excludes: Scheduling work within a selected engine; no configurable router is implemented., The boundary declaration alone provides no executable experiment or numerical evidence..

RequestRouterBoundary

id

string

parent

['string', 'null']

owner

string

kind

string

view

string

expanded

boolean

InferenceEngine

Coordinate request scheduling, model execution and decoding against a selected model artifact.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: Composition of request scheduler, tokenizer, model executor and sequence decoding., Its typed boundary in the shared configuration and explanation hierarchy.. Excludes: Public API transport, physical device ownership, and claiming a separate process or vendor product., The boundary declaration alone provides no executable experiment or numerical evidence..

InferenceEngineBoundary

id

string

parent

['string', 'null']

owner

string

kind

string

view

string

expanded

boolean

StateManager

Coordinate the lifetime, residency and movement of inference state.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: Composition of sequence-state representation, KV cache management and state transfer., Its typed boundary in the shared configuration and explanation hierarchy.. Excludes: Model equations, kernel implementation and fabric topology; no generic state service is implemented., The boundary declaration alone provides no executable experiment or numerical evidence..

StateManagerBoundary

id

string

parent

['string', 'null']

owner

string

kind

string

view

string

expanded

boolean

ResourcePool

The compute, memory and interconnect resources against which inference execution is constrained.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: Composition of the resource graph and declared compute-service assumptions., Its typed boundary in the shared configuration and explanation hierarchy.. Excludes: Placement decisions, measured hardware calibration and claims that graph topology is executed., The boundary declaration alone provides no executable experiment or numerical evidence..

ResourcePoolBoundary

id

string

parent

['string', 'null']

owner

string

kind

string

view

string

expanded

boolean

ComputeKernel

An executable implementation of a bounded numerical operation on a compute device.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: The boundary for algorithm, precision, fusion and memory-access choices in an operation implementation., Its typed boundary in the shared configuration and explanation hierarchy.. Excludes: CUDA as a platform, model architecture, scheduler policy, and an implemented kernel selector., The boundary declaration alone provides no executable experiment or numerical evidence..

ComputeKernelBoundary

id

string

parent

['string', 'null']

owner

string

kind

string

view

string

expanded

boolean

DeviceRuntime

The execution services used to launch device work and manage its memory and synchronization.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: The boundary between model execution and device launch, memory and synchronization facilities., Its typed boundary in the shared configuration and explanation hierarchy.. Excludes: Compute-kernel algorithms, physical hardware capacity, and claiming a CUDA runtime in this browser., The boundary declaration alone provides no executable experiment or numerical evidence..

DeviceRuntimeBoundary

id

string

parent

['string', 'null']

owner

string

kind

string

view

string

expanded

boolean

CollectiveCommunication

Coordinate a communication operation across a declared group of execution participants.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: The boundary for collective operation, algorithm and participant choices, such as all-reduce., Its typed boundary in the shared configuration and explanation hierarchy.. Excludes: NCCL as a universal implementation, point-to-point state transfer, and any implemented collective simulation., The boundary declaration alone provides no executable experiment or numerical evidence..

CollectiveCommunicationBoundary

id

string

parent

['string', 'null']

owner

string

kind

string

view

string

expanded

boolean

ExperimentSpec

One declared intervention on an ontology-owned mechanism or artifact in the inference journey.

Editable before execution; the consumed specification is frozen in the execution record.

Owns: One mechanism or artifact control and its requested settings., Primary and related ontology subjects, resolved against admitted concepts and preserved in execution.. Excludes: Observed outcomes and execution identity, Causal events and numerical measurements.

OntologyBinding

primary

Primary subject; must name an admitted ontology concept.

related

Other associated concepts; excludes the primary subject.

ExperimentSpec

control

string

value

['string', 'boolean', 'object']

ontology_binding

OntologyBinding · owned by ExperimentSpec

ExperimentExecution

One identified execution of the consumed controls against customer input, retaining its events and observations for inspection and replay.

Evidence accumulates during execution; the completed record is immutable.

Owns: Execution records joining the consumed specification, events, observations, and request outcomes., Numerical traces and shared execution provenance identifying source bytes, Python/Git version, and the actual backend; provenance is distinct from requested system settings., Resolved model, system and customer-input bindings actually consumed; these are frozen evidence, not an independent editable specification.. Excludes: Editable experiment drafts, Inspection cursor or playback position, Claims that a local run predicts production performance.

GenerationEvidence

input

GenerationInput · owned by InferenceRequest

model

ModelDescriptor · owned by ModelArtifact

prediction

GenerationPrediction · owned by ExperimentObservation

CacheRun

id

string

declaration

KVRunBinding · owned by ExperimentExecution

checkpoint

ModelArtifact · owned by ModelArtifact

provenance

ExecutionProvenance · owned by ExperimentExecution

events

array

computations

array

measurements

array

outcomes

object

states

object

evidence

string

ExecutionProvenance

python

string

revision

string

source_sha256

string

sources

object

backend

ExecutionBackend · owned by ExecutionBackend

ExecutionReplayInput

record_json

Exact exported JSON bytes; preserve numeric lexemes for replay.

KVRunBinding

model

ModelArtifactReference · owned by ModelArtifact

system

KVResidencySystemConfig · owned by InferenceSystem

workload

Workload · owned by Workload

ontology_binding

OntologyBinding · owned by ExperimentSpec

ResourceExecution

input

GenerationInput · owned by InferenceRequest

run

CacheRun · owned by ExperimentExecution

outputs

object

decoding_notes

Request-indexed malformed or incomplete decoding diagnostics; output text remains unmodified.

ExperimentEvent

An occurrence within an execution, with explicit order, causal links, and the applicable clock interpretation.

Recorded when an occurrence happens; earlier events are not rewritten.

Owns: Causal cache event envelopes with request identity and virtual timestamps., The current cache event envelope records cross-mechanism scheduling, cache and service transitions; it is not their semantic owner.. Excludes: A global sequential step shared by concurrent requests, Inferring causality from adjacency or equal timestamps, Treating virtual time or execution order as measured wall time, Response publication and completion payloads, owned by InferenceResponse.

CacheEvent

event_id

string

request_id

string

time_us

number

kind

string

state_id

['string', 'null']

bytes

integer

device_reserved_bytes

integer

host_retained_bytes

integer

detail

string

causes

array

phase

['string', 'null']

transfer_direction

['string', 'null']

ExperimentObservation

Recorded tensors or local measurements attached to executed work; mechanism-specific decisions and computations have separate owners.

Captured during execution and retained unchanged with its evidence.

Owns: Recorded intermediate tensors and their dependencies., Local wall-time measurements with their measurement boundaries.. Excludes: Requested settings and hypothetical service rates, Equating selected tokens with published response tokens, Treating a host measurement as target-accelerator performance.

TensorObservation

axes

array

dtype

string

id

string

inputs

array

layer

['integer', 'null']

positions

array

request_id

string

run_id

string

shape

array

step

integer

values

array

operation

Declared alternatives

CacheMeasurement

event_id

string

kind

string

elapsed_us

number

bytes

integer

GenerationPrediction

model_id

string

weights_sha256

string

tokenizer_sha256

string

sequence_ids

array

output_ids

array

text

string

pieces

array

logits

array

selected_id

integer

stop_reason

['string', 'null']

compute_ms

number

nodes

array

decoding_note

Malformed output or incomplete terminal encoding. Valid pending byte or UTF-8 prefixes while generation can continue have no diagnostic.

request

GenerationStep · owned by InferenceRequest

id

string

contract_sha256

string

probabilities

array

work

GenerationWork · owned by ModelExecution

ExecutionSpec

A composition of experiment controls used together to execute customer input. The list specifies one setup, not a sweep.

Editable before execution; retained unchanged with the execution evidence.

Owns: Composition of ontology-bound controls, with one assignment per control., Execution setup independently of customer input and observed results.. Excludes: Customer text and request arrivals, owned by Workload and InferenceRequest, Observed outcomes, Silently overriding conflicting controls.

ExecutionSpec

experiments

array

GenerationDefaults

execution

ExecutionSpec · owned by ExecutionSpec

workload

Workload · owned by Workload

ExperimentDesign

A named investigation, or lab, that tests one question with ExecutionSpec settings, states which controls it still lacks, and carries its abstraction contract.

Authored with the repository; applying it replaces the editable ExecutionSpec before execution.

Owns: The question, ontology subjects, settings and missing controls of one experiment., The abstraction contract and evidence references that bound its conclusions.. Excludes: Execution evidence and observed outcomes, owned by ExperimentExecution, Free-form pages or computations outside the declared execution operations.

ExperimentDesign

id

string

name

string

question

string

ontology_binding

OntologyBinding · owned by ExperimentSpec

settings

Controls applied together. An experiment with no missing controls must form a valid ExecutionSpec.

missing_controls

Undeclared controls the question needs; any entry makes the experiment planned, not executable.

contract

AbstractionContract · owned by ExperimentDesign

sources

array

evidence

Repository paths of evaluation results or research records supporting current conclusions.

AbstractionContract

target_regime

string

preserved

string

omissions

string

evidence_plan

string

limits

string

ExperimentSource

title

string

url

string

ExperimentCatalog

experiments

array

InspectionState

One selected position in an identified accepted token sequence, with one shared diagram/detail surface and architecture or observation selection. Selection and journey replay never execute inference.

Changes on inspection; reset when input is applied or output is reset. Execution identity is null until a computation exists.

Owns: Sequence position shared by token highlighting, derived recorded computation and the next submitted raw prefix., Shared diagram/detail focus and journey selection, distinguishing declared architecture from captured observations., Executed observation selection; inspecting an operand never changes the generation prefix.. Excludes: input text, model or generation configuration.

InspectionState

run_id

Current accepted computation identity; null before the first step or after reset.

position

Single selected position in the accepted sequence; determines the exact prefix for Next Token. The recorded computation is derived from this position.

observation_id

Exact observation ID within the selected pass, or null for request and output inspection.

surface

Single detail and diagram focus; model requires an observation in the selected computation.

architecture_step

Index in the selected architecture walkthrough; never an observation ID.

computation_index

Selected computation in a resource execution; null for ordinary token generation.

ApiError

An explicit transport admission or execution failure message.

Produced for a rejected or failed service operation; a completed execution record need not exist.

Owns: An explicit transport admission or execution failure message.. Excludes: Successful inference output, A structured recovery or retry policy, Evidence that an experiment completed.

ApiError

error

Explicit rejection reason

ExecutionBackend

The runtime, kernel and communication implementations used to execute model work; recorded backend evidence identifies actual compute, storage, library and precision.

Fixed in the recorded run; a different backend requires a new run binding.

Owns: Actual compute device, state-storage implementation, library, and precision observed for execution.. Excludes: hypothetical service rates, logical capacity budgets, worker placement policy.

ExecutionBackend

compute_device

string

device_state_storage

string

host_state_storage

string

library

string

precision

string

ResourceGraph

Hypothetical execution locations with local memory and directed point-to-point transfer links.

Immutable scenario input; changing it defines a different scenario.

Owns: Each node identifies one exclusive compute/local-memory budget; each directed link identifies one exclusive capacity., Rates use FLOPs or bytes per virtual microsecond; capacity uses bytes. Values are assumptions, not calibrated profiles., A reverse link is independent capacity. Shared duplex pools or multiple aliases for the same physical resource are outside this initial representation.. Excludes: No routing, placement, resource scheduling, contention algorithm or topology execution is implemented., No model/device compatibility, target-model scaling, weight/workspace allocation or vendor performance claim..

ResourceNode

id

string

memory_capacity_bytes

integer

compute_flops_per_us

number

memory_bytes_per_us

number

ResourceLink

id

string

source

string

target

string

bandwidth_bytes_per_us

number

startup_us

number

ResourceGraph

basis

string

nodes

array

links

array

Tokenizer

The identified vocabulary and codec that frame conversation input and map source bytes to inspectable token IDs.

Vocabulary and codec identity are fixed for an encoding; turns are inputs and spans/results are recorded evidence.

Owns: Vocabulary entries, role-tagged framing inputs, and the resulting token sequence., Source-byte/token spans and identities required to explain and reproduce encoding., Identity of the token-ID input supplied to model computation; these IDs are not embedding vectors.. Excludes: Embedding vectors and model computation, Generation policy and serving decisions, Diagnostic output display is not a guarantee that arbitrary generated IDs form valid text..

Token

id

integer

kind

string

piece

string

group

string

Turn

role

string

text

string

Span

kind

string

reason

string

byte_start

['integer', 'null']

byte_end

['integer', 'null']

token_start

integer

token_end

integer

raw_bytes

array

ids

array

turn

['integer', 'null']

Encoding

ids

array

spans

array

input_sha256

string

input_kind

string

manifest_sha256

string

context_limit

['integer', 'null']

codec_version

string

unicode_version

string

TokenizerOperation

name

string

Embedding

Look up token vectors and combine the declared input representations.

A model operation identity; observations are captured per execution.

Owns: Look up token vectors and combine the declared input representations.. Excludes: Positional vectors are supplied by PositionEncoding; later attention mixing is separate..

EmbeddingOperation

name

string

PositionEncoding

Represent sequence position according to the model architecture; the current implementation uses learned absolute vectors.

A model operation identity; observations are captured per execution.

Owns: Represent sequence position according to the model architecture; the current implementation uses learned absolute vectors.. Excludes: Does not establish extrapolation or compatibility when a mechanism is replaced..

PositionEncodingOperation

name

string

Normalization

Normalize channels using the declared statistics, epsilon, scale, and bias.

A model operation identity; observations are captured per execution.

Owns: Normalize channels using the declared statistics, epsilon, scale, and bias.. Excludes: Does not own residual addition or feed-forward projections..

NormalizationOperation

name

string

Attention

Query and mix sequence information, using accessible key/value history or a recurrent summary. The GPT baseline implements causal softmax attention.

A model operation identity; observations are captured per execution.

Owns: Query-based sequence mixing and the state-update rule of recurrent attention.. Excludes: KVState owns numerical K/V payloads; KVCache owns retention and reuse management., Position mechanisms are owned by PositionEncoding..

AttentionOperation

name

string

ResidualAddition

Add a branch output to its preserved residual input.

A model operation identity; observations are captured per execution.

Owns: Add a branch output to its preserved residual input.. Excludes: Does not own either branch computation or normalization..

ResidualAdditionOperation

name

string

FeedForward

Apply per-position channel projections and the declared activation.

A model operation identity; observations are captured per execution.

Owns: Apply per-position channel projections and the declared activation.. Excludes: Does not mix token positions or own residual addition..

FeedForwardOperation

name

string

OutputProjection

Project final hidden vectors to vocabulary scores.

A model operation identity; observations are captured per execution.

Owns: Project final hidden vectors to vocabulary scores.. Excludes: Does not select, deliver, or decode output tokens..

OutputProjectionOperation

name

string

ModelExecution

Execute model forward computations over input positions and available sequence state, producing logits and intermediate values.

Configuration is fixed for a run; observations identify the work that actually executed.

Owns: Numerical forward work, its precision and completion evidence., Declared full-prefix or cached execution implementation and numerical backend selection.. Excludes: Caller selection and stopping requirements., Draft acceptance, output publication, request scheduling, and resource-service timing..

CacheComputation

work_id

string

request_id

string

step

integer

input_ids

array

state_before

KVState · owned by KVState

state_after

KVState · owned by KVState

nodes

array

selection

Declared alternatives

matmul_flops

integer

logical_memory_bytes

integer

reference_max_abs_error

number

phase

string

reference_matmul_flops

integer

GenerationWork

processed_positions

integer

reused_positions

integer

matmul_flops

Executed dense and attention matrix multiply/add operations, counted as two FLOPs per multiply-add. Excludes normalization, activation and softmax.

Decoding

Use model predictions to advance a generated sequence while honoring the requested generation policy.

Configuration is fixed for a run; observations identify the work that actually executed.

Owns: Prediction-step evidence and the selected token; the implemented algorithm is autoregressive greedy selection., The algorithmic boundary for future draft proposal, target verification, acceptance and rollback.. Excludes: Requested generation policy, owned by GenerationPolicy., Forward tensor equations, response transport and inter-request scheduling., Declaring this boundary does not implement speculative decoding or distribution-preserving sampling..

TokenSelection

decoding

string

probabilities

array

raw_logits

array

rng_state

null

rule

string

selected_id

integer

RequestScheduling

Decide which arrived requests may become active and which eligible work runs next under resource constraints.

Configuration is fixed for a run; observations identify the work that actually executed.

Owns: Active-request admission slots; the current KV engine admits FIFO and rotates active requests between serialized service steps.. Excludes: Arrival declarations are workload input, not a scheduling decision., KV allocation and eviction policy, tensor equations, and token selection., No batching, priority, preemption or fair-sharing guarantee is implemented..

RequestSchedulingConfig

max_active_requests

integer

KVCache

Manage allocation, residency, retention and valid reuse of KV-state payloads under declared capacities.

Configuration is fixed for a run; observations identify the work that actually executed.

Owns: Logical device and host KV budgets and exact whole-prefix LRU retention., Cache validity, lookup, allocation and eviction decisions in the current KV lab.. Excludes: The numerical keys and values belong to KVState., Attention equations, transfer service costs and request scheduling., No partial-prefix paging or model-quality claim..

KVCacheConfig

device_kv_bytes

integer

host_kv_bytes

integer

retention_policy

string

StateTransfer

Move an identified sequence-state payload across an execution or memory boundary before dependent work uses it.

Configuration is fixed for a run; observations identify the work that actually executed.

Owns: Declared payload-transfer bandwidth and startup delay; current modeled duration is startup plus bytes divided by rate., The handoff boundary for future lossless prefill-to-decode KV movement.. Excludes: Endpoint placement, routing and topology selection., State interpretation or attention math; cache retention and eviction choices., No vendor conversion, network contention or physical-performance calibration is represented..

StateTransferConfig

link_bytes_per_us

number

link_latency_us

number

ComputeService

Translate declared operation and memory-traffic counts into conditional service time under explicit resource rates.

Configuration is fixed for a run; observations identify the work that actually executed.

Owns: Hypothetical compute and memory rates; the KV lab charges the maximum of operation time and memory-traffic time.. Excludes: Numerical model results and actual device identity., Measured physical performance, transfer service, and request scheduling., No calibrated accelerator or multi-resource overlap model..

ComputeServiceConfig

compute_flops_per_us

number

memory_bytes_per_us

number

CustomerInteraction

Declared client submissions, cumulative output consumption and cancellation on one scenario clock.

Immutable scenario input; changing it defines a different scenario.

Owns: Ordered customer actions; equal-time array order breaks ties. Each submission creates one invocation., Consumption describes client demand; cancellation expresses intent, not acknowledgment or immediate server termination.. Excludes: No browser rendering, API protocol, network delivery, retry grouping or agent tool orchestration is implemented., These inputs are validated scenarios, not accepted by existing inference engines..

CustomerSubmission

kind

string

at_us

number

request_id

string

request

InferenceRequest · owned by InferenceRequest

CustomerConsumption

kind

string

at_us

number

request_id

string

output_bytes

Cumulative application-output bytes consumed; excludes protocol framing. A declared action, not proof that bytes were available.

CustomerCancellation

kind

string

at_us

number

request_id

string

CustomerInteraction

clock

string

actions

array