Chunked and Resumable Transfer

1. The real problem: resume progress is not artifact trust

Consider a 12 GiB model artifact with expected digest sha256:D1 and exact size S1. The connection drops mid-transfer and the client is no longer certain which local ACK was last durable. Resuming from a guessed local offset can duplicate or skip bytes, while checking only the final size can miss corruption. The owned artifact is Resumable Artifact Transfer Session & Integrity Harness v1.

2. Lesson boundary: control transport, not topology or licensing

AIE-014 consumes immutable artifact identity from AIE-013 and owns only the resumable transfer protocol. A completed transfer grants no license permission; AIE-015 owns that policy. Node placement belongs to AIE-016 and GPU/media runtime health belongs to AIE-017. The bounded outcome here is receiving the exact expected bytes and publishing them safely.

3. Bind every session to expected digest and size

At session creation, record at least session_id, target_artifact_digest, and expected_size. A friendly name or filename is not sufficient identity. If the target artifact changes, the old session must not be reused for the new artifact even when the display name stays the same. Session binding prevents a valid transport session from becoming valid evidence for the wrong content.

4. The server is the progress authority after interruption

After a disconnect, the client must not treat its last local offset as authoritative. Resume from the range/offset confirmed by the transfer service. OCI chunked upload follows this pattern: successful chunk responses report the received range, and a GET to the upload location can return the current valid upload state. In the harness, resume_offset = server_ack_end + 1, not the byte the client merely believes was durable.

5. Chunks are ordered; gaps and invalid overlaps are rejected

The next chunk must start exactly after the last server-confirmed byte. A chunk that jumps ahead, overlaps improperly, or otherwise breaks ordering must not advance acknowledged state. The harness reports REJECT_CHUNK, keeps the acknowledged range unchanged, and preserves the partial blob as NON_RUNNABLE.

6. Retry must be bounded and identity-safe

Network interruption is normal, but blind retry is unsafe. A retry must remain attached to the same valid session and target identity, start from server-confirmed state, and record retry count and interruption reason. If the session is expired or bound to another target, the correct outcome is RESTART_REQUIRED, not an invented offset.

7. A partial blob is never a runnable replica

Receiving most bytes—or even receiving all chunk acknowledgements—does not yet create a runnable replica. Partial data must carry an explicit NON_RUNNABLE state and remain outside the executable replica catalog. Transport progress answers how much data arrived; it does not yet prove content identity.

8. Size verification is required but not sufficient

OCI descriptors carry both digest and byte size for referenced content, and retrieved content should be checked against those values before trust. In this design, final_size == expected_size is a hard gate. But equal-length payloads can contain different bytes, so a size match alone can never authorize publication.

9. Final digest verification closes content identity

After the candidate is complete, recompute the digest over the final bytes and compare it with target_artifact_digest. If size matches but digest does not, the artifact is QUARANTINE. Historical offsets and successful ACKs do not override this failure because they are transport evidence, not content-identity proof.

10. Publish atomically and only after verification

Until both final size and digest pass, the candidate stays in a temporary/non-runnable location. Only then may an atomic operation expose it as a replica and create the replica_id plus audit event. Downstream systems must never observe an intermediate state as if it were a complete runnable artifact.

11. Quarantine is a typed failure, not an ambiguous file

Record a precise quarantine_reason, such as FINAL_DIGEST_MISMATCH, FINAL_SIZE_MISMATCH, or SESSION_IDENTITY_MISMATCH. A quarantined candidate is neither automatically destroyed nor runnable; it remains inspectable evidence for diagnosis. This prevents a transport failure from being silently coerced into success.

12. The audit trail must reconstruct the decision

Record session ID, target digest/size, chunk index and sent range, acknowledged range, resume offset, upload location, retry count, interruption reason, out-of-order result, final digest/size, and publish/quarantine state. A reviewer should be able to reconstruct why the system resumed, rejected, restarted, quarantined, or published without relying on hidden reasoning.

13. OCI is a reference pattern, not a MAKNOW architecture mandate

The current OCI Distribution Specification documents the POST → PATCH → PUT chunked blob-upload pattern and server-side progress retrieval for interrupted uploads. MAKNOW does not need to be an OCI registry. The reusable safety properties are explicit sessions, server-confirmed progress, ordered chunks, explicit closure, and final integrity verification. The exact harness state machine remains a MAKNOW design decision.

14. Lab: eight fixtures designed to break shortcuts

Case 1 makes the client ACK ahead of server state and requires server-confirmed resume. Case 2 rejects an out-of-order chunk. Case 3 has correct size but wrong final digest and must quarantine. Case 4 fails the final size gate. Case 5 is the only clean publish path. Case 6 restarts an expired session. Case 7 refuses a session bound to a different artifact identity. Case 8 proves that a session for the old digest cannot be reused after a changed-context artifact update.

15. Transfer and final checklist

For transfer, mutate one chunk while preserving the total byte count. Offsets and size can still look correct, but the final digest must reveal the corruption and keep the candidate unpublished in quarantine. Final checklist: exact identity; session binding; server-confirmed resume; ordered ranges; bounded retry; partial=NON_RUNNABLE; final size; final digest; atomic publish; quarantine reason; replica ID only after publication; audit event; and requalification when target identity or transfer semantics change.