Localization QA and AI-Assisted Translation Review

1. First screen: fluent is not the same as correct

A release candidate contains a polished German sentence produced by an AI-assisted translation workflow. It reads naturally. The spelling is clean. The model can even explain why its wording sounds idiomatic. Yet the string silently drops {amount}, uses an obsolete product term, and formats a date as if the target user were still in the source locale. A second Arabic candidate is linguistically plausible but arrives without explicit language and direction metadata. Nothing about surface fluency proves that these strings are safe to ship.

That is the bounded problem owned by this lesson. Localization QA needs several evidence layers that answer different questions. Automated checks can catch structural defects such as lost variables, malformed message syntax, forbidden terminology, or an obviously mismatched locale-format fixture. Human review remains accountable for meaning, context, style, cultural fit, and the final accept/edit/reject decision. AI output is a candidate in that workflow, never the authority that certifies itself.

Microsoft's current localization-testing guidance still separates functional, visual, and linguistic validation and says linguistic review should examine accuracy, context, style, terminology, consistency, cultural appropriateness, and correct data formats. [S-MS-L10N-TEST] Its in-context review guidance also makes the running product and product context part of quality review. This lesson converts that idea into an inspectable local record rather than a subjective "looks good" approval.

The evidence artifact is Localization QA & AI Review Record v1. Each row binds message identity, source intent, AI/MT candidate, automated structural results, terminology evidence, locale-format evidence, contextual review, accountable human decision, mutation/repair evidence, and one residual limitation.

2. Route orientation: automate what is mechanically observable, review what is semantic

Foundation route: begin with five questions. Did the candidate preserve required variables? Did it respect protected or forbidden terminology? Does the fixture show target-locale formatting instead of source-locale formatting? Is language/direction metadata present when the target string needs it? Finally, does a human reviewer judge the candidate correct for the actual product context? The first four can be bounded by deterministic fixtures; the last cannot be delegated to a boolean from the same model that generated the text.

Repair route: start with one injected defect. Predict the failure code before running the checker. Then execute the local audit, inspect the evidence row, repair only that boundary, and rerun. If {amount} disappeared, restore the variable and do not rewrite unrelated wording. If the product term violates the approved glossary, fix the term but still leave meaning/context review to a human. If an Arabic string lacks direction metadata, restore metadata and record that this is presentation/interoperability evidence, not proof of linguistic accuracy.

Experienced route: skip introductory tool vocabulary and inspect the review system as an acceptance contract. Ask whether a release owner can answer who made the decision, which checks were automated, which claims required human judgment, what source/context/terminology evidence was available, which deliberate mutation was detected, and what remains unproven.

3. Mental model: one candidate, multiple evidence boundaries

Use a pipeline with explicit handoffs:

source + intent + context + terminology → AI/MT candidate → automated structural checks → human linguistic/context review → accept/edit/reject → regression evidence.

The pipeline matters because each stage has different authority. Source intent tells the reviewer what the message is supposed to accomplish. Terminology constrains domain words but does not prove the sentence. The candidate may come from a model, a machine-translation service, a translation memory suggestion, or a human draft; its origin does not change the evidence requirement. Automated checks observe mechanical properties. Human review evaluates meaning and contextual suitability. The final decision must cite both kinds of evidence when both are relevant.

Do not collapse these stages into a single "quality score." A structural pass can coexist with a context mistranslation. A glossary-compliant sentence can still reverse the intended meaning. A fluent sentence can omit a variable. A human can also make mistakes, so the record should preserve decisions, evidence, reviewer role, and regression tests rather than treating human review as mystical proof.

For right-to-left strings, language and direction metadata form another bounded layer. The W3C String Metadata Working Draft says the language and direction of a natural-language string should be determinable and recommends explicit metadata instead of relying only on heuristics. [S-W3C-STRINGMETA] In this lesson, missing lang/dir evidence is a detectable risk; restoring it still does not certify the translation's meaning.

4. Foundation layer: design the review row before the checker

The minimum review row contains message_id, source, source_intent, candidate, source_locale, target_locale, required variables, candidate variables, terminology constraints, locale-format observation, context note, language/direction metadata, automated findings, human decision, reviewer role, evidence references, and residual risk.

Separate observation from inference. Observation: the source contains {amount} and the candidate does not. Inference: the localized message cannot preserve the runtime amount substitution. Decision: EDIT_REQUIRED. Observation: the German candidate contains Arbeitsbereich while the current product glossary requires Projektbereich. Inference: terminology policy is violated. Decision: edit the term and still perform contextual review. This structure prevents an AI explanation from being mistaken for runtime evidence.

The glossary is a control surface, not an oracle. Azure Translator's current glossary documentation describes context-specific terminology, explicit source/target language pairs, and regular review/update of glossary entries. [S-AZ-GLOSSARY] That supports using a glossary to constrain known terms, but it does not support the stronger claim that glossary compliance guarantees the whole sentence.

The fixture therefore stores terminology evidence separately from the human meaning decision. It also stores whether a message is AI-assisted. That field is provenance only; it never grants or removes correctness.

5. Deep explanation: QA dimensions are orthogonal, not interchangeable

A useful localization review examines multiple dimensions because different failures survive different checks.

Accuracy and meaning. Does the target preserve the intended proposition, action, warning, or commitment? A model can choose a fluent but wrong sense of a polysemous word. This is a human/contextual judgment in the bounded workflow.

Contextual adherence. Is the wording correct for the UI surface, audience, product state, and surrounding action? "Save" in a document editor is not the same context as "save" in a rescue message. Resource-level review without product context can miss this.

Style and terminology. A style guide constrains voice; a glossary constrains domain terms. Both improve consistency, but neither replaces semantic review. Microsoft lists style-guide adherence, terminology, vocabulary, and consistency as separate linguistic-validation aspects. [S-MS-L10N-TEST]

Locale-specific formatting. Dates, numbers, currency, names, and other data should follow target conventions when the product contract requires localized presentation. The fixture uses explicit expected/observed values so the learner can prove the local check without pretending to have tested every locale or platform.

Structural integrity. Variables and protected tokens must survive. This is ideal for automation because the property is mechanically observable.

Language and direction metadata. Natural-language strings need usable language/direction information when transported or embedded in interfaces. This is especially important for bidirectional text, but metadata is still not linguistic approval.

6. Structural checks: necessary evidence, deliberately insufficient

Automated QA earns its place by being exact about bounded properties. The lab extracts placeholders such as {amount} and {date} from source and candidate strings. If a required placeholder is missing, the checker can reject the record without asking whether the German sentence sounds natural. The same logic applies to forbidden terms in a known product scope, fixture-defined locale-format mismatches, and required RTL metadata.

The checker should not quietly grow into a fake linguistic judge. It does not assign a universal translation-quality score. It does not infer cultural appropriateness from a language tag. It does not accept a candidate because every mechanical check is green. Instead, it produces observations that the accountable reviewer uses alongside contextual evidence.

This boundary makes the system safer when AI assistance is added. A model may propose a candidate or summarize detected issues, but the evidence record must distinguish generated text from observed test results. NIST's Generative AI Profile remains a useful risk authority here because it treats confabulated content and inappropriate trust in generated output as risks that require governance rather than blind acceptance. [S-NIST-GAI] The lesson narrows that principle to one operational rule: model fluency or confidence never substitutes for the recorded review evidence.

7. Glossary limits and the AI-candidate rule

A glossary can prevent expensive inconsistency when a product term has an approved translation, a brand must remain unchanged, or an ambiguous word must use a specific sense. In the fixture, Product Alpha requires Projektbereich and forbids Arbeitsbereich for the current German UI. An AI candidate that uses the forbidden synonym is rejected by terminology QA even if the sentence is grammatically excellent.

But glossary compliance is only one dimension. Consider Projektbereich löschen. It uses the approved term, yet it may still be wrong if the source action is "archive workspace" rather than "delete workspace." The glossary cannot see the product intent unless that intent is part of the review context. This is why the evidence row contains both terminology and source intent.

The same principle applies to AI-assisted review. AI may suggest candidate checks, point out likely inconsistencies, or help summarize observations after the reviewer establishes the contract. It may not fabricate runtime evidence, citations, reviewer approval, or a product-owner decision. It may not mark its own output correct merely because another model response agrees. The accountable decision remains explicitly human in the artifact.

8. Worked example: eight messages, five injected failure classes

The deterministic release contains eight localized UI messages. M-001 drops {amount} from a German payment message. M-002 uses the forbidden old product term. M-003 preserves text meaning but uses the source-locale date/currency presentation. M-004 is fluent German but mistranslates the action because the candidate ignores the product context. M-005 is an Arabic candidate with missing direction metadata. M-006, M-007, and M-008 are clean comparison cases that still require recorded human review rather than automatic approval from the checker.

For M-001, the observation is a placeholder set mismatch. The correct decision is EDIT_REQUIRED, and the evidence must include the missing-token finding. For M-004, all structural checks may pass. The decisive observation is contextual: the source intent is to archive a workspace, while the candidate says to delete it. The correct decision is again edit/reject, but the authority is human context review rather than the structural checker.

For M-005, the candidate is Arabic and the transport fixture omits dir=rtl. The checker reports a metadata risk. The repair adds explicit language/direction metadata and reruns the check. The residual limitation states that this local fixture does not prove rendering behavior in every browser or host application.

9. Failure modes that survive attractive demos

Fluency-as-proof. A reviewer accepts M-004 because the German sounds natural. The product action is still wrong.

Green-check overreach. A dashboard shows all structural checks passing, so the system auto-accepts a string that has never received contextual review.

Glossary absolutism. Every approved term is present, so the sentence is declared correct even though the proposition is wrong.

Generated-evidence laundering. An AI assistant writes "tested in production" or invents a reviewer decision that never happened. The artifact becomes untrustworthy even if the wording is plausible.

Locale-format shortcut. The candidate copies $1,234.50 and 09/14/2026 into a German UI and the review records only "translation correct." Locale presentation evidence is missing.

RTL heuristic dependence. An Arabic string is shipped without explicit direction metadata because it looked fine in one screenshot. That observation does not establish portable handling.

Single-reviewer universalization. One German review is treated as proof for every target locale and audience. The lesson forbids that inference.

10. Experienced bridge and trade-offs

Teams often ask whether they should buy a localization QA platform, add model-based review, or build custom linting. This lesson does not prescribe a vendor. The invariant is the evidence boundary.

Static linters are excellent for tokens, message syntax, protected terms, and deterministic policy checks. They are cheap to run on every build. Their weakness is semantic blindness.

Model-assisted review can scale candidate triage and may surface suspicious wording or inconsistency. Its weakness is that the model can be confidently wrong, can miss product context, and can generate explanations that look like evidence. Therefore model output should be tagged as a candidate observation or suggestion until verified.

Human in-context review is strongest where meaning, tone, legal sensitivity, audience, and product behavior matter. It costs more and can vary between reviewers, so teams should provide source intent, screenshots or product access, terminology, issue taxonomy, and a structured decision record. Microsoft explicitly recommends in-context review with product context and reviewer resources. [S-MS-L10N-TEST]

A mature workflow combines these mechanisms rather than asking one to impersonate the others.

11. Guided practice, mutation, and independent transfer

Start by reading the fixture contract before the data. Predict which failure class should appear for each injected message. Run python3 run_checks.py. A passing baseline means the workflow correctly recognizes the known defects and records accountable decisions; it does not mean the raw candidate set is defect-free.

Then perform the deliberate mutations. Change the missing-token message to ACCEPT without restoring the variable. Accept the forbidden terminology. Mark the source-locale format as acceptable. Accept the context-wrong candidate. Remove RTL metadata and still approve the string. Each mutation must cause the audit to fail with the expected governance code. Restore the correct decision/evidence and rerun.

The independent transfer changes the product risk. A release-note notification contains legally sensitive wording and introduces fr-FR and es-MX. The learner must design a stricter path that adds a second accountable reviewer or legal/product owner where needed, preserves structural checks, and explicitly refuses the claim that automation proves legal or linguistic correctness. The decisive solution may not be copied from the baseline answer or supplied by AI.

12. Verification and the evidence record

A strong record allows a second reviewer to reconstruct why a decision was made. For each message, store the source intent, candidate, automated findings, terminology rule, locale-format observation, context note, language/direction evidence, human decision, reviewer role, and residual limitation.

Mutation evidence must contain prediction, observed issue code, diagnosis, repair, and rerun result. If the expected code does not appear, the learner has not proved the checker detects the failure. If the repair changes unrelated rows, the evidence is too broad to diagnose causality.

Separate facts from conclusions. "Candidate placeholder set is {date} while source requires {amount, date}" is an observation. "Amount substitution would be lost" is a bounded inference. EDIT_REQUIRED is the accountable decision. "The entire German localization is correct" would be an unsupported generalization.

The residual limitation is mandatory: this deterministic lab does not call Azure Translator, does not evaluate a live LLM, does not prove reviewer competence, does not establish legal approval, and does not prove browser rendering across all hosts. Its purpose is to prove the review method and failure boundaries locally.

13. Assessment, transfer, and freshness boundary

To close the lesson, you must state the core contract and one non-goal, implement the bounded review workflow, diagnose a deliberate mutation, defend the separation between automation and human judgment, and solve the legally sensitive changed-context transfer independently.

Blocking failures include accepting a structurally defective candidate; treating a glossary as whole-sentence proof; replacing contextual review with a model score; fabricating evidence or reviewer approval; or claiming one locale/reviewer proves universal correctness.

The source boundary remains exact. S-MS-L10N-TEST is the current Microsoft localization-testing guidance. S-AZ-GLOSSARY is the current Azure Translator glossary guidance; the page was updated in June 2026 and still emphasizes explicit source/target terms and regular review. S-NIST-GAI is the NIST Generative AI Profile used only for the bounded risk principle around generated-output trust. S-W3C-STRINGMETA is the W3C First Public Working Draft of 16 July 2026, so this lesson treats it as evolving guidance rather than a final Recommendation. [S-MS-L10N-TEST] [S-AZ-GLOSSARY] [S-NIST-GAI] [S-W3C-STRINGMETA]

The nearest review trigger is 2026-10-17 because the vendor glossary guidance is tool-volatile. A freshness update may narrow or update current claims; it may not expand WEB-022 into the data-model lesson before it or the relational-modeling lesson after it.