Tool Registry and Qualification

1. The real problem: discoverable is not qualified

Seeing docs.lookup in a tool list or registry says that a capability with that label is discoverable. It does not prove that it is the exact version previously reviewed, that its input/output schemas are unchanged, that authorization scopes stayed bounded, that it is still read-only, that its network destination is unchanged, or that qualification evidence is still fresh. This lesson builds an operational Tool Registry that separates discovery from admission.

The owned artifact is the “Qualified Tool Registry Entry & Admission Evidence Record v1.” Its decision must be traceable to exact identity and evidence, not to a familiar name or one successful invocation.

2. Separate three layers: discovery, qualification, and authorization

Discovery answers “what is being advertised?” Qualification answers “which exact identity, contract, and evidence were tested?” Authorization/workflow policy answers “may this specific request use that qualified capability?” Collapsing these layers is a common way to give an AI agent unintended authority.

A tool can remain QUALIFIED while the current workflow is not allowlisted; the invocation is then BLOCKED without rewriting qualification history. Conversely, authentication can succeed while the permission set has drifted away from the qualified boundary; network success is not permission to keep using the old evidence.

3. Current MCP facts and the inference boundary

MCP revision 2026-07-28 carries method/tool names in request metadata/headers, gives list responses cache hints, and lifts tool schemas to full JSON Schema 2020-12. These are useful for discovery, routing, caching, and validation, but none of them proves that a discovered tool is safe or approved for your product boundary.

Operational rule: do not turn a protocol capability into product evidence. Use protocol metadata to form candidate identity; use your own evidence to grant QUALIFIED.

4. What the Official MCP Registry gives—and does not give

At this lesson’s freshness check, the Official MCP Registry is still in preview and describes itself as the centralized metadata repository for publicly accessible MCP servers. Namespace authentication, standardized installation/configuration metadata, and discovery APIs are useful. Its own documentation also explains that it focuses on metadata hosting and namespace authenticity while security scanning of actual code is delegated to package registries and downstream aggregators.

Therefore registry presence is a provenance/discovery signal, not a MAKNOW qualification certificate. Record discovery_source, but derive QUALIFIED only from your qualification suite and evidence bundle.

5. Exact tool-entry identity

An operational entry key cannot be only tool_name. At minimum capture server identity, server version, tool name/version, protocol version, endpoint identity, and input/output schema digests. If implementation version differs from server version, version it separately as well.

The same name with a different schema is a different candidate. A new server version with the same tool name is also a new candidate unless your governed evidence explicitly proves compatibility. Preserve the old record; create a new entry and record the relationship instead of rewriting history.

6. Schema and argument contract

Compute input_schema_digest and output_schema_digest from a normalized, versioned representation. Comparing field labels is insufficient: required arguments, enums, conditionals, references, and output shape can change workflow behavior.

Record an argument_contract and required_arguments beside the digest. A schema change is a drift signal. If output schema changes while the name and endpoint stay familiar, the old qualification must not be inherited by name.

7. Permission, credential, and tenant boundary

The qualification record must capture expected auth scopes, permission set, credential mode, and tenant boundary. These fields describe the authority under which the tool was tested. “The server supports this scope” is different from “this invocation’s credential is constrained to this scope.”

If qualification covered docs:read and later configuration introduces docs:admin into the authority boundary, that is permission drift. Even when the current operation is read-only, the change must pass policy again; successful authentication or a previous call is not a substitute.

8. Side effects, data access, and network boundary

Every tool needs a side-effect class such as READ_ONLY, WRITE_CAPABLE, or DESTRUCTIVE. data_access_class can distinguish public, tenant-private, or sensitive data. network_scope and endpoint_identity define where execution may connect.

A tool that changes from read-only to write-capable is no longer the same qualified contract even when its visible schema looks similar. A change from docs.example to another network destination is also a requalification trigger. These are gates, not decorative metadata.

9. Admission is richer than “the call worked”

For tools with effects, idempotency class, approval requirements, rollback capability, and dry-run capability determine whether retries and mistakes can be contained. Timeout behavior, rate limits, error taxonomy, and observability should also be recorded where applicable.

One successful call is one sample. Qualification needs positive and negative fixtures so permission denial, timeout, malformed responses, and error semantics are bounded. Missing decisive evidence produces UNKNOWN or CANDIDATE—not an implicit pass.

10. Registry lifecycle

Start with CANDIDATE. After identity, schema, permissions, side effects, data/network boundaries, and qualification fixtures match the evidence bundle, the entry can become QUALIFIED. Expired evidence becomes STALE; decisive drift becomes SUSPENDED; workflow policy can make a specific invocation BLOCKED; withdrawn trust can become REVOKED.

Qualification state and request eligibility are distinct. A qualified entry may still be ineligible for a particular workflow.

11. Drift detection and requalification triggers

Use explicit drift labels: VERSION_DRIFT, INPUT_SCHEMA_DRIFT, OUTPUT_SCHEMA_DRIFT, PERMISSION_DRIFT, SIDE_EFFECT_DRIFT, DATA_ACCESS_DRIFT, NETWORK_DRIFT, or EVIDENCE_STALE. Typed drift keeps the reason for suspension auditable.

The lesson rule is simple: decisive drift is never closed by silent inheritance. Create a fresh candidate or suspend the current entry, then rerun the qualification slice that owns the changed dimension.

12. Lab: the docs.lookup scenario

The baseline fixture uses server com.example/docs version 2.4.1 and tool docs.lookup. Input digest is s-in-17, output digest s-out-9, scope docs:read, side effect READ_ONLY, data access PUBLIC, and network target docs.example. The suite passes and the entry becomes QUALIFIED.

Later fixtures change one decisive dimension at a time: server/schema, permission expansion, network, side effect, freshness, workflow allowlist, or discovery availability. Your job is to identify which changes break qualification and which only change discovery/operational availability.

13. Counterexample: same name, different capability

Suppose later discovery still returns docs.lookup, but server version becomes 2.5.0, output schema changes, and a write-capable permission appears. A registry keyed only by name would silently lend old qualification to new capability.

The bounded response is to preserve the 2.4.1 history, create a new candidate, stop eligibility for the changed capability until qualification is rerun, and record VERSION_DRIFT, OUTPUT_SCHEMA_DRIFT, plus side-effect/permission drift as applicable.

14. Transfer: broader scope, identical schema

In the transfer case, server version and schemas stay byte-identical while authorization configuration expands from docs:read to include docs:admin; the workflow only requests a read operation. Because the original qualification declared a narrower permission boundary, successful authentication cannot prove equivalence.

In this lesson’s policy, permission expansion is a requalification trigger. If architecture can demonstrably mint a credential restricted to docs:read, a new bounded/conditional qualification can be created with fresh evidence. The old record is not silently inherited.

15. Definition of Done and audit checklist

A complete record includes exact tool identity, schema digests, scopes/permissions, side effects, data/network boundaries, qualification suite and evidence reference, lifecycle state, drift signal, requalification trigger, workflow allowlist, and audit event. Each claim should distinguish protocol fact, discovery metadata, and product design/policy.

For final PASS, ask five questions: Is discovery separate from qualification? Is the entry bound to exact version/schema/permission? Is there a negative fixture? Are UNKNOWN/STALE/SUSPENDED/BLOCKED states preserved? Does changed context force a fresh decision? If any answer is no, the Tool Registry is not yet reliable for admission.