RFC 0034: OTel collector test seam + secret-leakage invariant promotion

Summary

Define a normative host-extension test seam — GET /v1/host/sample/test/otel/spans and POST /v1/host/sample/test/debug-bundle/export — so cross-host conformance scenarios can mechanically verify that BYOK canaries don't leak into OTel span attributes or debug-bundle exports. Today the two corresponding SECURITY/invariants.yaml rows (secret-leakage-otel-attribute, secret-leakage-debug-bundle-otel) carry tier: reference-impl with non_testability_rationale because no cross-host seam exists; this RFC closes that gap and graduates both invariants to tier: protocol.

Motivation

Per docs/KNOWN-LIMITS.md:30-31 and the external standards-readiness review of 2026-05-21, "the conformance OTel collector seam doesn't yet inspect span attributes; a host could pass conformance while leaking BYOK material on telemetry exports." This is a real production-scale credibility gap — every production deployment ships OTel telemetry, and the spec's SR-1 secret-redaction invariant (agent-memory.md §"SR-1") + envelope-reasoning-secret-redaction invariant cannot be cross-host verified without an introspection seam.

The envelope-reasoning-secret-redaction conformance scenario already exists (conformance/src/scenarios/envelope-reasoning-secret-redaction.test.ts) with live behavioral OTel + debug-bundle assertions — but they soft-skip on HTTP 404 because the seams are not normatively specified. This RFC formalizes the seams so the soft-skip becomes capability-gated (hosts that advertise opt in; hosts that don't are honest about not implementing).

Proposal

§A — capabilities.observability.testSeams block (normative)

Add to schemas/capabilities.schema.json under capabilities.observability:

   "observability": {
     "type": "object",
+    "properties": {
+      "testSeams": {
+        "type": "object",
+        "additionalProperties": false,
+        "properties": {
+          "otelScrape": {
+            "type": "boolean",
+            "description": "Host exposes GET /v1/host/sample/test/otel/spans returning the host's accumulated OTel span buffer scoped to a runId. Conformance scenarios use this to verify span attributes don't carry BYOK canaries. The endpoint is conformance-only (host-extension namespace); production deployments SHOULD return 404 or 403."
+          },
+          "debugBundleExport": {
+            "type": "boolean",
+            "description": "Host exposes POST /v1/host/sample/test/debug-bundle/export returning the host's debug-bundle for a runId. Conformance scenarios use this to verify the bundle's serialized form doesn't carry BYOK canaries. Conformance-only."
+          }
+        }
+      }
+    }
   }

§B — Seam contract (normative)

GET /v1/host/sample/test/otel/spans?runId=<id> MUST return 200 OK with a JSON body { spans: Array<{ name, attributes, events }> }. The spans array MUST include every OTel span produced by the host's instrumentation for the named run, including any openwop.*-prefixed attributes added to span context. Hosts MAY redact span content using the canonical [REDACTED:<secretId>] marker per agent-memory.md §SR-1 — that's the contract being tested. The seam additionally accepts optional envelopeId and name query parameters for narrower introspection; hosts that implement the seam MAY honor these filters but MUST honor the runId scope at minimum.

POST /v1/host/sample/test/debug-bundle/export MUST return 200 OK with body { bundle: { runId, events, spans, exportedAt } } mirroring the payload of GET /v1/runs/{runId}/debug-bundle per spec/v1/debug-bundle.md (the wrapping bundle envelope makes the seam-versus-real-endpoint origin explicit). The request body MUST be { runId: string }; the seam returns 400 invalid_argument when runId is absent or empty. The seam exists to give conformance scenarios a synchronous endpoint they can hit without first triggering an interrupt → debug bundle workflow.

Both seams live under the host-extensions.md canonical-prefix /v1/host/sample/test/* and are NOT part of the v1 wire surface. Production hosts SHOULD return 404 or 403 from the seam unless OPENWOP_TEST_OTEL_SCRAPE=true (or equivalent env-gate) is set.

Upstream dependency on RFC 0021 envelope-accept (normative). The OTel scrape seam observes span buffers populated by the host's envelope-handling instrumentation. The conformance scenario envelope-reasoning-secret-redaction.test.ts drives a redacted reasoning trace through the POST /v1/host/sample/envelope/accept seam (RFC 0021) and verifies the spans the OTel scrape seam returns do NOT carry the canary plaintext. A host advertising observability.testSeams.otelScrape: true without also serving /v1/host/sample/envelope/accept (RFC 0021) returns empty span buffers for the conformance run — the seam-shape probe still passes (a 200 OK { spans: [] } is valid) but the behavioral SR-1 assertion is vacuous because there's no span content to check against. Hosts SHOULD NOT advertise otelScrape: true without RFC 0021 envelope-accept also wired; honest hosts that don't yet serve envelope-accept leave otelScrape un-advertised (or false) until both surfaces are in place. Symmetrically: the debug-bundle export seam observes the debug-bundle payload constructed by the run's event log + reasoning emission path; hosts without envelope-accept produce bundles that don't contain reasoning content, and the SR-1 assertion is again vacuous.

§C — SECURITY invariant promotion

SECURITY/invariants.yaml flips two rows from tier: reference-impl to tier: protocol:

• secret-leakage-otel-attribute — paired with envelope-reasoning-secret-redaction.test.ts §"OTel-attribute scrape" assertion. Public test glob: conformance/src/scenarios/envelope-reasoning-secret-redaction.test.ts.
• secret-leakage-debug-bundle-otel — paired with the same scenario's §"debug-bundle export" assertion.

Remove the non_testability_rationale field from both rows; populate the existing tests: field with the scenario file path glob (the tests: key is the canonical home for public-test references per SECURITY/invariants.yaml's schema — 95 existing occurrences across protocol-tier and reference-impl-tier rows).

§D — Conformance scenario tightening

conformance/src/scenarios/envelope-reasoning-secret-redaction.test.ts — the two it() blocks inside the describe('envelope-reasoning-secret-redaction: downstream-projection paths (RFC 0030 §E)', …) block (the OTel-attribute scrape assertion and the debug-bundle-export assertion) currently soft-skip on spansRes.status === 404 / bundleRes.status === 404. With this RFC, the soft-skip becomes capability-gated:

-    if (spansRes.status === 404) return; // host doesn't expose the OTel scrape seam
+    // Gate on capability — if host advertises otelScrape but the seam returns 404, that's a failure.
+    if (!capabilities.observability?.testSeams?.otelScrape) return;
+    expect(spansRes.status, 'host advertising otelScrape MUST serve the seam').toBe(200);

Hosts that don't advertise the seam continue to soft-skip honestly (the existing posture). Hosts that DO advertise it MUST serve a valid response.

Compatibility

Additive. No existing endpoint changes; no required fields added to any existing schema. The new capabilities.observability.testSeams block is optional; hosts that omit it advertise no test-seam claim and conformance scenarios soft-skip exactly as today.

The SECURITY invariant tier-promotion is technically a strengthening of conformance requirements — but only for hosts that advertise the new capability. Hosts that don't advertise remain in their current "reference-impl-tier passes via host CI" posture.

Conformance

• Existing scenario envelope-reasoning-secret-redaction.test.ts graduates 2 soft-skip blocks to capability-gated assertions.
• No new scenarios needed.
• npm run openwop:check's check-security-invariants step continues to enforce the public-test-glob requirement for every protocol-tier invariant; the two newly-promoted rows must include the glob.

Alternatives considered

1. Don't promote the invariants. Rejected — leaves the standards-readiness gap open. The reviewer's specific framing was "a host could pass conformance while leaking BYOK material on telemetry exports." 2. Add a generic /v1/test/otel seam at the v1 wire surface. Rejected — test seams don't belong in the protocol wire surface per host-extensions.md §"Canonical prefixes." The host-extension namespace is the right home. 3. **Standardize OTel collector configuration (e.g., a normative OPENWOP_OTEL_EXPORTER_* env-var set).** Rejected — overlaps OpenTelemetry's own spec authority; OpenWOP shouldn't normate how hosts wire their collector.

Unresolved questions

1. Should the seams support streamId-scoped queries? Per-stream isolation would let conformance scenarios verify cross-stream isolation. Deferred — the runId scope is sufficient for SR-1; cross-stream isolation belongs in a separate STD-X track. 2. Should debug-bundle export carry a freshness gate? A host that returns stale buffers could mask leakage windows. Recommend an as-of: <ISO 8601 ts> field; deferred until reference-host implementation exposes a natural freshness signal.

Acceptance criteria

• [x] Spec text merged (this file).
• [x] schemas/capabilities.schema.json updated with the observability.testSeams block per §A.
• [x] spec/v1/observability.md updated with a new §"OTel collector test seam" section citing this RFC.
• [x] SECURITY/invariants.yaml updated per §C; both rows pass the check-security-invariants glob check.
• [x] conformance/src/scenarios/envelope-reasoning-secret-redaction.test.ts tightened per §D.
• [x] At least one non-steward host implements the seam + advertises the capability — MyndHyve workflow-runtime advertises observability.testSeams.otelScrape: true and serves GET /v1/host/sample/test/otel/spans?runId=<id> returning 200 OK { spans: [] } (revision workflow-runtime-00196-7mm, commit 60b569de, verified live 2026-05-23). Per §B's "Upstream dependency on RFC 0021 envelope-accept" clause, the empty-buffer Tier-1 boundary is explicitly allowed — seam-shape probe passes; SR-1 behavioral assertion vacuously safe. MyndHyve's Tier-2 (SDK provider + BatchSpanProcessor + InMemorySpanExporter wiring) is a separate strengthening pass on their roadmap.
• [x] INTEROP-MATRIX.md updated with the MyndHyve advertisement on the multi-agent + OTel row.
• [x] CHANGELOG entry under [Unreleased].

Path to Active → Accepted: a non-steward host advertises the new capability and the scenario passes against it. Per RFCs/0001 §"Promotion to Accepted."

References

• docs/KNOWN-LIMITS.md §"Behavior tests too coarse to fully prove an invariant" (the two invariants this RFC graduates)
• spec/v1/agent-memory.md §"SR-1 secret-redaction invariant"
• RFCS/0030-envelope-reasoning-and-tier-one-subset.md §E (envelope-reasoning-secret-redaction invariant)
• spec/v1/observability.md (the doc this RFC extends)
• spec/v1/host-extensions.md §"Canonical prefixes" (the namespace home for the new seams)
• spec/v1/debug-bundle.md (the parent shape of the debug-bundle export)
• External standards-readiness review 2026-05-21 — finding (5) "Security posture insufficient — no telemetry redaction conformance"