Vetting و Test Job

A hard test is not automatically a good test

Three candidates enter a paid-social creative test. Candidate A receives brief v3; Candidate B gets extra constraints but the system does not record a new instruction version; Candidate C submits a beautiful asset that misses the defined conversion constraint. If we store only “looks good” or one final score, we cannot reconstruct what capability was measured, whether conditions were comparable, or which evidence actually supports the next step. The owned artifact is Vetting Test-Job Evidence Matrix v1.

Owned boundary

COM-022 owns a bounded vetting/test-job process only. Talent-profile and prior capability evidence belong to COM-021; later assignment and responsibility workflow belong to COM-023. This lesson does not make a live hiring decision, determine worker status across jurisdictions, make payments, use credentials, or mutate production assignments.

Mental model: start from the role, not from an impressive test

Use this order:

role requirements -> bounded test design -> equivalent brief -> work sample -> rubric evidence -> reviewer conflict/adverse-signal check -> human decision or HOLD

Keep six questions separate: Which capability matters? Which task measures it? Were instructions, time and tools materially comparable? What rubric existed before scoring? What authority/conflict does the reviewer have? Is the evidence strong enough to send the case to a human decision without pretending the test is universal proof?

Separate FACT, INFERENCE, and DESIGN

FACT: U.S. EEOC guidance lists sample job tasks such as performance tests, simulations, work samples, and realistic job previews among selection procedures. It also discusses job-relatedness/validation, discrimination and adverse impact, and test accommodation. INFERENCE: auditability improves when capability targets, instruction version, rubric version, reviewer identity, score evidence, and negative signals are stored separately from the final human decision. DESIGN: READY_FOR_HUMAN_DECISION, HOLD, UNKNOWN, PARTIAL, DISPUTED, STALE, and BLOCKED are MAKNOW states. They are not EEOC terminology or law.

The 21-dimension Vetting Matrix contract

1) Identity: case, role, candidate, and test

vetting_case_id, role_or_assignment_id, candidate_id, and test_job_id must be explicit. Without role binding, a polished but irrelevant score can masquerade as selection evidence.

2) Scope and capability targets

task_scope should be bounded and connected to actual role capabilities. capability_targets state what the test is meant to measure. Every scored rubric dimension should map back to those targets; undocumented reviewer taste is not a capability target.

3) Instructions, timebox, tools, and adjustment

instructions_version, timebox, and allowed_tools make test conditions reconstructable. Equivalence means material differences are visible and governed; it does not mean refusing appropriate accessibility adjustments. accessibility_adjustments should preserve the adjustment needed for fair administration while avoiding unnecessary sensitive detail in broadly visible evidence.

4) Compensation and policy boundary

compensation_policy_ref identifies the governing compensation/contracting policy. If jurisdiction, worker status, or applicability is unclear, the workflow must not guess. Use HOLD and escalate to the appropriate human/legal authority.

5) The rubric comes before taste

rubric_version and rubric_dimensions are established before scoring. score_evidence traces each score to observable evidence. If a reviewer later dislikes a color but aesthetic preference was never a rubric dimension, that comment may be feedback; it cannot silently change the official score.

6) Reviewer, conflict, and adverse signal

reviewer_id and reviewer_role identify the evaluator. conflict_state preserves reviewer disagreement or conflict of interest. adverse_signal records a signal that requires human examination; its presence is not automatically proof of discrimination or unfairness, but deleting it is not an acceptable shortcut.

7) Decision and retention

decision_state must be no stronger than the evidence. READY_FOR_HUMAN_DECISION means the evidence package is ready for a human to review; it does not mean hire or accept. retention_rule and evidence_ref keep lifecycle and provenance visible.

Worked examples

1. Valid baseline -> READY_FOR_HUMAN_DECISION

Candidates receive a versioned brief tied to the same capability targets, with explicit timebox and tools. Needed adjustment is documented, the rubric predates submission, the reviewer has no unresolved conflict, and scores trace to rubric evidence. The case can proceed to human decision.

2. Material instruction difference without versioning -> HOLD

One candidate receives an extra constraint while every record still says instructions_version=v3. The comparison is not trustworthy. Version the difference, assess materiality, then re-test or normalize the review as appropriate.

3. Off-rubric taste affects score -> HOLD

The output meets the defined conversion constraint, but the reviewer lowers the official score because of a personal color preference absent from the rubric. Re-score against the declared rubric before a bounded decision is possible.

4. New role, old score -> STALE

A creative work sample may not be relevant to a different specialist role. Audit history can remain, but test, rubric, and score must be rebuilt around the new role.

5. Unresolved reviewer conflict -> DISPUTED

Two reviewers interpret one rubric dimension differently and support their scores with conflicting evidence. Keep the conflict visible and use a resolution rule; averaging is not a substitute for resolving the evidence dispute.

6. Compensation or jurisdiction unclear -> HOLD

A technical checkbox cannot determine whether a test-job arrangement is permissible in every worker-status or jurisdiction context. Applicability requires the appropriate human/legal authority.

Common failure modes

• Assuming the test is fair because everyone took “the same test” while material conditions differed.
• Confusing task difficulty with role relevance.
• Reusing one score as universal evidence of future performance.
• Changing the rubric after seeing the output.
• Injecting off-rubric preference into the official score.
• Erasing an adjustment record to make the data look uniform.
• Hiding reviewer conflict or adverse signals behind an average score.
• Guessing through compensation/jurisdiction uncertainty.
• Treating READY_FOR_HUMAN_DECISION as an automated hiring verdict.

Alternatives and trade-offs

Portfolio only: low friction, but it may not show performance under the new role's constraints. Bounded work sample + rubric: stronger direct evidence and auditability, with added design, compensation/policy review, and reviewer-calibration cost. Large speculative project: can feel realistic but increases scope, fairness, compensation, and value-leakage risk; it is a poor fit for this bounded outcome. Automated ranking: useful for deterministic triage, but it must not erase human authority or negative evidence.

Experienced bridge: 15 critical questions

Is the role explicit? Is the test mapped to role capabilities? Is scope bounded? Are instructions versioned? Are material candidate differences recorded? Is the timebox explicit? Are allowed tools explicit? Is only the necessary adjustment evidence retained? Is there a compensation policy reference? Was the rubric versioned before scoring? Does each score have evidence? Are reviewer identity and role known? Was an adverse signal preserved? Is conflict state explicit? Does the decision merely prepare the case for human review rather than turn one test into final truth?

Guided and independent practice

Complete all 21 fields for the baseline fixture. Then apply four mutations: an unversioned instruction difference, off-rubric preference, an erased adjustment record, and score reuse for a different role. For each mutation, state only three things: which evidence failed, the new typed state, and the smallest defensible repair. Then design a new test for a different role instead of copying the baseline score.

Acceptable evidence

Minimum evidence includes case/role/candidate/test identity, capability targets, instruction version, timebox, rubric version/dimensions, reviewer identity, score evidence, adverse signal, conflict state, decision state, and evidence reference. Compensation policy, allowed tools, adjustment, and retention must also be explicit enough for reconstruction. Hidden reasoning, undocumented taste, or provenance-free scores do not count.

Assessment and transfer

Passing requires four things: the COM-022 boundary, a complete baseline matrix, diagnosis of an instruction mismatch, and transfer to a different role. The baseline score is never the transfer oracle. Preserve only genuinely role-independent process evidence; rebuild relevance, test, rubric, and score for the new role. Success here means READY_FOR_HUMAN_DECISION, not an automated decision about a person.

References and freshness

• U.S. EEOC — Employment Tests and Selection Procedures. Rechecked on 2026-09-15. The current page still lists sample job tasks/work samples among selection procedures and covers job-relatedness/validation, discrimination/adverse impact, and accommodation: https://www.eeoc.gov/laws/guidance/employment-tests-and-selection-procedures
• The page is U.S. EEOC technical assistance and states that the document itself does not have independent force and effect of law. Other jurisdictions, contracts, and worker-status questions require the relevant legal authority.
• The state vocabulary and fail-closed rules here are MAKNOW DESIGN, not EEOC-defined states.
• Legal/policy-current claims are rechecked by 2026-11-02 or earlier after a material source change.