Accessible Multimedia Learning UX

When “has captions” is mistaken for “is accessible”

Consider a multimedia lesson with narrated video, an interactive transcript, a word-synced playback control, and a slide deck. It looks polished and the video has captions. Yet the captions omit important non-speech audio, focus jumps unpredictably after seeking, and one instructional diagram is explained only visually. A QA check that asks only “can I turn captions on?” misses three real barriers.

This lesson builds Accessible Multimedia Experience Audit Contract v1. Each experience records media type, purpose, caption/transcript/audio-description/text-alternative states, keyboard and focus evidence, contrast, resize/reflow, pause/stop control, barriers, WCAG references, severity, remediation owner, and verification state. The strongest positive state is not automatic release; when evidence is sufficient, the contract reaches MULTIMEDIA_ACCESS_REVIEW_REQUIRED. Missing decisive evidence or unresolved critical barriers must fail closed.

Route orientation and ownership boundary

MKN-011 owns Accessible Multimedia Learning UX. MKN-007 is the hard conceptual prerequisite and MKN-010 is soft. MKN-010 owns TTS, Forced Alignment, and Word Sync; this lesson does not reteach alignment quality or speech generation. MKN-012 owns the visual/slide production pipeline; this lesson does not absorb slide authoring. Our object is access to media plus its controls and alternative paths.

Foundation learners receive a field map and a worked case. Repair learners isolate the broken evidence dimension and rerun only the affected gate. Experienced learners defend severity and a bounded decision without supplied decision labels. The checkpoints are F1 boundary retrieval, F2 negative-case diagnosis, and F3 a complete recomputation for an audio-first mobile experience with a hardware keyboard.

Mental model: inventory → access need → evidence → test → severity → repair/exception → verification

The pipeline is:

content inventory → access needs → WCAG/evidence mapping → media + interaction tests → barrier severity → remediate or governed exception → independent verification → review/hold

For audio/video, ask whether speech and required non-speech meaning are represented by captions, transcripts, or an appropriate alternative. For visual information, ask whether needed meaning is available through description or equivalent text. For interaction, verify a complete keyboard path and meaningful focus behavior through seek/play/pause and other state changes. For presentation, verify contrast and reflow in the target context rather than inferring them from a screenshot.

UNKNOWN, BARRIER, HOLD, and REVIEW_REQUIRED are valid states. The contract must not coerce incomplete evidence into success because the product “mostly works.”

Foundation contract: seventeen evidence dimensions

The core dimensions are experience_id, media_type, content_purpose, caption_state, transcript_state, audio_description_state, text_alternative_state, keyboard_path, focus_order_state, contrast_check, resize_reflow_state, pause_stop_control, error_or_barrier, wcag_ref, severity, remediation_owner, and verification_state.

Minimum invariants: media and purpose are explicit; synchronized-media captions include required speech and non-speech information; transcripts do not automatically replace every synchronized-media requirement; important visual meaning has a usable text/description path; keyboard evidence covers the full interaction path rather than mere tab availability; focus is tested after state changes; contrast and reflow are measured in the target context; pause/stop controls are explicit when applicable; severity is local MAKNOW policy rather than a WCAG label; remediation or exceptions have an owner; and a changed context triggers recomputation.

Deep explanation: keep FACT, INFERENCE, and DESIGN separate

WCAG 2.2 — FACT. WCAG 2.2 provides testable, technology-independent success criteria under the principles Perceivable, Operable, Understandable, and Robust. Relevant areas include prerecorded media alternatives, captions, audio description, keyboard access, focus order, contrast, resize/reflow, and pause/stop behavior.

W3C WAI media guidance — FACT. WAI describes captions as a text version of speech plus non-speech audio needed to understand content, transcripts as text versions of audio information, and description as a way to convey visual information needed for understanding. WAI presentation guidance also recommends describing pertinent visual information and providing accessible materials.

Product interpretation — INFERENCE. A media player with a caption toggle is therefore not sufficient evidence for the complete learning experience. Media, alternatives, interaction state, and presentation context need distinct evidence.

MAKNOW release policy — DESIGN. Barrier severity, release blocking, exception approval, and remediation ownership are local project authority. External standards do not automatically authorize these product decisions.

Worked examples: baseline, invalid shortcut, and transfer

Baseline. EXP-VID-01 combines narrated video, transcript, and a word-synced seek control. Captions contain speech but omit an alarm sound required to understand the scenario; seeking breaks focus continuity; the summary diagram has no alternative. The result is HOLD_CRITICAL_ACCESS_BARRIERS. A transcript does not erase caption, focus, or visual-alternative failures.

Wrong approach. A team removes non-speech cues to make captions “cleaner.” caption_state changes from COMPLETE_REVIEWED to INCOMPLETE_NON_SPEECH, so the gate returns HOLD_CAPTION_COMPLETENESS.

Transfer. The lesson becomes audio-first on a narrow mobile screen at 200% text with a hardware keyboard. Audio description may become NOT_APPLICABLE if there is no decisive visual channel, while transcript readability, reflow, focus order, and playback controls become more important. The desktop outcome is not an oracle; a healthy transfer fixture recomputes to MULTIMEDIA_ACCESS_REVIEW_REQUIRED.

Failure modes that must fail closed

Do not treat the experience as release-ready when captions omit required speech or non-speech information; a required transcript is missing or stale; instructional visual meaning exists only in color/shape/image; the keyboard path skips controls or traps focus; focus moves incorrectly after seek/modal/state changes; contrast or reflow is unknown but a pass is asserted; required pause/stop control is absent; a high-severity barrier has no remediation owner; an exception lacks traceable owner/reason/approval; or a desktop result is copied into mobile transfer.

A HOLD is a correct result when evidence is not strong enough.

Alternatives and trade-offs: captions, transcripts, description, and alternative presentation

Captions keep text synchronized with media and can carry required non-speech cues. Transcripts support scanning, search, and asynchronous reading but do not automatically replace caption or description obligations. Audio description brings needed visual meaning into an audio path; for instructional video, integrated description in the main narration can sometimes be preferable. Text alternatives or long descriptions can make diagrams understandable without vision, but they must convey instructional meaning rather than merely visual appearance.

Trade-offs include timing, cognitive load, screen size, player capability, learner preferences, localization cost, and maintenance. The contract should state which alternative serves which content purpose and how it was verified.

Experienced bridge: treat accessibility as a state machine, not an end-stage checklist

A production system can model each media experience as a small state machine. Caption and transcript states are versioned; keyboard and focus evidence is rerun after important interactions; verification records who checked which revision. A player update, seek-behavior change, or viewport change can drift one dimension, and one dimension can still be release-blocking.

Automation can check file presence, schema, some contrast measurements, focus traversal logs, or caption-track versions. It usually cannot decide by itself whether a description conveys the instructional meaning of a diagram or whether a non-speech sound is necessary for understanding. Human judgment remains visible rather than being hidden behind an automated PASS.

Guided and independent practice

Guided. Run the baseline fixture and identify its three barriers separately. Repair only the non-speech caption cue and rerun; focus and alternative failures should remain. Repair focus next, then add the text alternative. Do not hide distinct causes behind one vague “fix accessibility” change.

Independent. Create a podcast + transcript + chapter-control fixture. Mark audio_description_state as NOT_APPLICABLE only when no visual channel exists and the rationale is visible. Then mutate the fixture by making the transcript stale or removing the owner from a high-severity barrier; the gate must reject success.

Experienced. For a real player, decide which events should produce focus evidence—initial play, seek, transcript-row activation, captions menu, fullscreen exit, and error recovery—and which checks need human verification.

Verification evidence: what must be reproducible?

The package must show, without hidden reasoning, that the frozen Bible hash is unchanged, both locales contain all 13 semantic roles, the three-scenario table covers all 17 dimensions, the synthetic lab is deterministic and offline, negative cases emit typed states, external claims reference real source IDs, and transfer is recomputed without a baseline oracle.

Machine validation is not proof of human quality, but it detects structural drift. A human reviewer should be able to reproduce the same bounded state from the visible fixture and rules and understand why a barrier blocks or remains review-required.

Assessment and transfer: desktop video to audio-first mobile

Assessment has four layers: retrieval explains the MKN-011 boundary; application builds the baseline contract; diagnosis evaluates the mutation that removes non-speech caption information; transfer changes modality and interaction context.

On a narrow screen at 200% text, reflow may become decisive. A hardware keyboard makes focus evidence more prominent. An audio-first design can change some visual requirements, but it does not erase transcript or control semantics. Copying the desktop state produces HOLD_BASELINE_ORACLE_REUSE.

Passing means explaining which evidence changed because the context changed and why the decision had to be rebuilt.

References, freshness, and authority boundaries

Primary external sources are WCAG 2.2, W3C WAI Accessible Presentations, and WAI Making Audio and Video Media Accessible. Rechecked on 2026-09-13, WCAG 2.2 remains a W3C Recommendation with testable, technology-independent criteria; WAI continues to define captions, transcripts, and description around access to speech, non-speech audio, and needed visual information.

These sources define FACT boundaries. Severity, remediation ownership, exception workflow, and release states remain MAKNOW DESIGN authority. A material standards/platform change triggers freshness review and a new package version; the accepted Golden is not mutated in place.