RFC 0012: Memory Compaction Profile

Amended (additive) 2026-05-25: RFC 0062 (scheduled memory distillation) reuses the memory.compacted event with an optional distillation sub-object ({ tokenBudget, tokensUsed, indexUpdated }). The event is additionalProperties: true; required and the trigger enum are unchanged — a scheduled distillation emits trigger: "host-managed". Non-breaking per COMPATIBILITY.md §2.1.

Summary

Define an optional capability profile that lets a host advertise it runs background memory compaction — periodic distillation of many short-lived MemoryEntry rows into fewer long-lived ones — and emit a canonical memory.compacted audit event when it does so. The profile standardizes only the wire surface: capability shape, event vocabulary, and an SR-1 carry-forward invariant. The compaction algorithm (summarization model, embedding scheme, token budgets, scheduling cadence) remains a host implementation choice.

Motivation

RFC 0004 (MemoryAdapter) already gives hosts everything they need to _implement_ compaction: list reads candidates, put writes the distilled entry, delete evicts the originals. Three concrete pain points motivate adding a thin protocol surface on top:

1. Cross-host observability. An operator running multiple OpenWOP hosts (e.g., a dev SQLite host plus a prod Postgres host) today has no portable way to ask "did each host's compaction run last night?" Each host invents its own log shape. A canonical memory.compacted event in the openwop.* namespace gives one query across hosts. 2. Replay attribution. When list returns a _compacted_ entry instead of the original raw entries, replay debugging is opaque — there's no protocol-visible link from "this distilled entry" back to "these source entries that no longer exist." A standard tags: ["compacted-from:<runId>"] convention plus a sourceIds[] field on the event gives the audit trail without requiring the source entries to still exist. 3. Redaction-invariant carry-forward. SR-1 (RFC 0004 §D) is enforced at put time over the _direct_ content. When a host summarizes ten entries into one, the summarization step is a _new_ put whose content is derived from the originals. Without an explicit normative carry-forward, a naive host implementation could route summarized content around the BYOK redaction harness. This RFC tightens that.

Honest scope check. Items (1) and (2) are observability conveniences; (3) is the only one that's strictly normative — a host that ignores it can leak secrets. The RFC is justified primarily by (3), with (1) and (2) as additive benefits that ride along.

Driven by the 2026-05-13 evaluation of an autonomous-agent-runtime feature set ("dreams" / background memory distillation) against the OpenWOP roadmap. Most of that feature set is application-runtime work that doesn't belong in the protocol; the compaction surface is the one slice that has a defensible protocol footprint.

Proposal

§A Capability advertisement

Hosts that perform memory compaction MAY advertise it under the existing capabilities.memory block from RFC 0004:

"memory": {
  "supported": true,
  "maxEntrySizeBytes": 65536,
  "ttlSupported": true,
  "compaction": {
    "supported": true,
    "trigger": "host-managed",
    "maxInputEntries": 1000,
    "maxOutputBytes": 65536
  }
}

Fields:

• supported (boolean, required when the block is present) — when true, the host performs compaction over longTerm memory and emits the memory.compacted event defined in §B.
• trigger (closed enum: "host-managed" | "client-requested" | "both", required when supported: true) — host-managed means compaction runs on a host-internal schedule that clients do not control. client-requested means clients may invoke compaction via an API surface (deferred — see §Unresolved questions #1). both means either path is supported. v1.x normates only host-managed; client-requested and both are reserved.
• maxInputEntries (integer, optional) — informational ceiling on how many source entries a single compaction call collapses. Clients MAY use this to set their own expectations; not enforced at the wire layer.
• maxOutputBytes (integer, optional) — informational ceiling on the size of the resulting distilled entry. SHOULD be ≤ memory.maxEntrySizeBytes.

Hosts that don't advertise capabilities.memory.compaction are assumed not to perform compaction. Clients MUST NOT assume compaction has happened based on entry counts alone.

§B memory.compacted event

Hosts that advertise capabilities.memory.compaction.supported: true MUST emit a memory.compacted event in the openwop.* observability namespace each time a compaction run completes, regardless of trigger.

Event shape (additive to observability.md §"Canonical event vocabulary"):

{
  "type": "memory.compacted",
  "ts": "2026-05-13T03:00:00.000Z",
  "memoryRef": "mem_<tenantId>_<agentId>_longTerm",
  "outputId": "mem_<id>",
  "sourceIds": ["mem_<id1>", "mem_<id2>", "..."],
  "sourceCount": 47,
  "trigger": "host-managed",
  "byteSize": 12044
}

Field semantics:

• memoryRef (string, required) — same opaque handle from RFC 0004 §C. Bounds the compaction to a single memory scope.
• outputId (string, required) — the id of the new MemoryEntry created by compaction. MUST be readable via MemoryAdapter.get(memoryRef, outputId) until normal TTL eviction.
• sourceIds (array of strings, optional) — ids of entries that were collapsed into outputId. Hosts MAY omit this array when sourceCount > 100 and instead rely on the compacted-from:<runId> tag convention (§C). Hosts that _do_ include sourceIds MUST be exhaustive within the array (no "and N more" semantics).
• sourceCount (integer, required) — total count of source entries collapsed, including any not enumerated in sourceIds.
• trigger (string, required) — matches capabilities.memory.compaction.trigger. Indicates which trigger path drove this run.
• byteSize (integer, required) — byte size of the resulting MemoryEntry.content.

Consumers (observability dashboards, audit log aggregators) that don't recognize memory.compacted MUST ignore it without error. This is consistent with the additive-event-type pattern in observability.md.

§C Provenance tag convention

The distilled entry SHOULD carry a tag of the form compacted-from:<compactionRunId> where <compactionRunId> is a host-issued identifier (any opaque string the host can correlate to its compaction log). This lets MemoryAdapter.list consumers detect compacted entries without needing access to the memory.compacted event stream.

This is a SHOULD, not a MUST: a host whose tag-space is structurally constrained (legacy tag-prefix discipline, fixed-vocabulary tagging) MAY omit this. The memory.compacted event remains the canonical provenance signal.

§D SR-1 carry-forward (normative)

The Secret Redaction invariant SR-1 (RFC 0004 §D) applies transitively through compaction:

SR-1 (compaction extension). When a host produces a compacted MemoryEntry whose content is derived from one or more source entries, the host MUST route the derived content through the same BYOK redaction harness it would apply to a fresh put from a workflow node. The fact that the source entries were already SR-1-compliant at their original put time MUST NOT be used as evidence to skip redaction on the derived content. Summarization models in particular MAY introduce secret-shaped substrings (hallucinated tokens, format-leaks from in-context examples) that were not present in any source entry; the redaction harness is the only protocol-level defense.

This extension is the load-bearing normative claim in this RFC. Hosts that advertise capabilities.memory.compaction.supported: true but do not honor SR-1 carry-forward fail the conformance scenario in §Conformance.

§E Out of scope

The following are explicitly NOT in this RFC:

• Compaction scheduling. Cron expressions, intervals, "at" times — purely host concerns. The protocol's only schedule visibility is trigger.
• The compaction algorithm. Summarization models, embedding schemes, token budgets, retrieval ranking — all host choices.
• Client-requested compaction API surface. Reserved by the trigger: "client-requested" enum value but not normated. A future RFC may add the API surface if implementer demand materializes.
• Cross-tenant or cross-host compaction. Same tenant-isolation rules as RFC 0004 §F apply; compaction MUST NOT cross tenant boundaries.
• Lossy-vs-lossless distinction. All compaction in this profile is assumed lossy (otherwise it's not compaction, it's archival). Hosts that want lossless archival have a different shape and should not advertise this profile.

Compatibility

Additive.

• capabilities.memory.compaction is a new optional sub-block of an existing optional block. Hosts that don't advertise it are unaffected.
• memory.compacted is a new event type. The canonical observability.md rule is that consumers ignore unknown event types; this rule covers backward compatibility for existing observability consumers.
• The SR-1 carry-forward extension applies only at the boundary where compaction occurs. Hosts that don't perform compaction never encounter it. Hosts that _do_ perform compaction without advertising the profile are NOT bound by this RFC, but SHOULD migrate to either (a) advertising the profile and honoring SR-1 carry-forward, or (b) ceasing compaction — silent compaction with no audit event is a protocol smell.
• No schema diffs to existing MemoryEntry or MemoryListOptions shapes. The provenance-tag convention (§C) lives entirely in the existing free-form tags array.

Lands in a v1.x minor as part of @openwop/openwop-conformance 1.X.0. No SDK wire-type additions are strictly required (the event is JSON-shape, dashboard-facing); SDK error catalogs SHOULD add compaction_redaction_failed if a host wants to surface SR-1 violations via the normal error envelope, but the wire surface stands without it.

Conformance

Existing scenarios that touch the surface:

• conformance/src/scenarios/memory-put-redaction.test.ts (RFC 0004 §D, MR-2) — base SR-1 enforcement at put time.
• conformance/src/scenarios/redactionAdversarial.test.ts — adversarial redaction inputs.

New scenarios required for Active → Accepted transition:

• memory-compaction-event-emitted.test.ts — gated on capabilities.memory.compaction.supported: true. Triggers a host-managed compaction (via env var or test hook), asserts a memory.compacted event surfaces with the documented shape, and asserts outputId is readable via MemoryAdapter.get.
• memory-compaction-sr1-carry-forward.test.ts — gated on capabilities.memory.compaction.supported: true. Plants source entries containing post-redaction <REDACTED:...> markers and adversarial near-secret strings, triggers compaction, asserts the resulting outputId's content passes the redaction harness (no plaintext secrets, no [BYOK:...] form-leaks). This is the load-bearing SR-1 carry-forward test.
• memory-compaction-provenance-tag.test.ts — gated; asserts the compacted-from:<id> tag convention from §C when the host SHOULD-follows it. Soft assertion: log-and-warn if absent, fail only if the tag is malformed.

All three scenarios skip cleanly when the capability is absent. Suite-version bump: @openwop/openwop-conformance 1.X.0 minor.

Alternatives considered

1. Do nothing — leave compaction to vendor extensions. Each host invents its own vendor.<host>.memory.compaction capability and its own vendor.<host>.memory.compacted event. Trade-off: the multi-host observability story stays broken, but the protocol surface stays smaller. Rejected because the SR-1 carry-forward invariant (§D) is a real security gap that doesn't have a non-protocol home — a vendor extension can't normate cross-host security behavior. 2. Make compaction a first-class MemoryAdapter operation (compact(memoryRef, options): MemoryEntry). Adds a fifth verb to RFC 0004's four. Trade-off: clean API surface, but forces every memory-capable host to either implement compaction or explicitly reject the call. Rejected because the compaction trigger is fundamentally a host concern in the dominant use case (nightly background work); exposing it as a client-callable verb pulls in scheduling concerns the protocol doesn't want. 3. Make this RFC purely about the SR-1 carry-forward invariant and drop §A/§B/§C. Smallest possible footprint. Trade-off: loses the cross-host observability win (motivation #1 and #2). Rejected because if a host is doing compaction at all, the cost of emitting a canonical event is trivial relative to the cost of running the compaction. The capability advertisement is what enables conformance gating, and without that, SR-1 carry-forward is hard to test mechanically. 4. Defer entirely until a third-party host requests it. The 2026-05-12 architecture review pattern: gate work on external demand. Trade-off: avoids speculative protocol surface. Rejected because the SR-1 gap exists in any host that does compaction today, including hosts not yet on the conformance trajectory — closing it after a leak is more expensive than closing it before. However, this remains a live option if no implementer adoption signal arrives within the comment window — the RFC may close Withdrawn honestly.

Unresolved questions

1. Is there a concrete client use case for trigger: "client-requested"? None has been articulated as of 2026-05-13. If none surfaces, the enum value should be dropped from §A in Active status to avoid reserving wire surface that no one uses. 2. Should memory.compacted.sourceIds be required, optional, or capped? Current proposal: optional with a soft 100-id ceiling and sourceCount as the always-present count. A future revision may tighten this once real-world compaction batch sizes are known from the first reference-host implementation. 3. Does this RFC need a SECURITY invariant entry? The §D SR-1 carry-forward is normative. Strong candidate for a new SECURITY/invariants.yaml row (memory-compaction-sr-1-carry-forward) so it's enforced by the same machinery as the existing 35 protocol-tier invariants. Decision deferred to Active. 4. Does this RFC even need to exist? Honest open question. RFC 0004's existing put/delete is _functionally_ sufficient for any host that wants to compact. This RFC's defense is the SR-1 carry-forward invariant (§D) plus the cross-host observability convenience. If maintainer review concludes (3) belongs in RFC 0004 as a clarifying revision and (1)+(2) don't clear the value-vs-protocol-surface bar, this RFC closes Withdrawn and the SR-1 carry-forward language folds into a Accepted revision of RFC 0004 §D.

Implementation notes (non-normative)

• Reference TypeScript host (examples/hosts/postgres/) is the natural first implementation target; nightly compaction at the host level is the dominant deployment shape and the SR-1 carry-forward harness already exists via the BYOK redaction code path shared with MemoryAdapter.put and debug-bundle redaction.
• The compaction algorithm itself (which model to call, what prompts to use) is out of scope. The reference implementation likely uses the same core.llm.chat surface that nodes use, with the host as the caller. This is consistent with the "OpenWOP runs the workflow; the LLM call is whatever the node implementor wants" pattern from README.md:53-60.
• A2A composition: compaction is per-host. A2A peers do not see each other's memory.compacted events unless explicitly federated, which is out of scope for v1.x.
• No anticipated SDK code generation impact. The event type lives in observability, not in the request/response surface.

Acceptance criteria

• [x] Spec text merged (this RFC + capabilities.md annex + observability.md event-vocabulary row). — Landed 2026-05-13 in c30a284 (Phase 2).
• [x] SECURITY/invariants.yaml entry for memory-compaction-sr-1-carry-forward (or explicit decision to fold into RFC 0004's existing SR-1 row). — Landed 2026-05-13 in c30a284; new protocol-tier row (severity: critical) referenced by memory-compaction-sr1-carry-forward.test.ts. Protocol-tier invariants 35 → 36.
• [x] At least one reference host (in-memory or Postgres) implements §A advertisement + §B event emission + §D carry-forward. — Postgres reference host, landed 2026-05-14 in 2bf082a. Conditional advertisement gated on OPENWOP_MEMORY_COMPACTION=true; runCompaction + applyCompactionRedaction enforce SR-1 §D end-to-end; host smoke verifies 7 paths.
• [x] Three conformance scenarios from §Conformance, all capability-gated. — memory-compaction-{event-emitted,sr1-carry-forward,provenance-tag}.test.ts, all gated on capabilities.memory.compaction.supported. 3/3 pass live against the Postgres reference host.
• [x] CHANGELOG entry under the v1.X version that ships the suite minor. — [1.1.1 — unreleased] block carries the Phase 1, Phase 2, and Phase 3 prep bullets.
• [x] INTEROP-MATRIX.md row updated for any reference host that advertises the profile, with a one-line evidence claim. — Postgres row's Phase I.7 / I.2 / RFC 0012 evidence paragraph names the conditional env vars + the host smoke file.

Comment window note

The canonical 7-day public comment window opened on 2026-05-13 (Draft → Active) and was scheduled to close on 2026-05-20. Promotion to Accepted landed on 2026-05-15 — five days early — under the comment-window waiver in CONTRIBUTING.md §"Bootstrap-phase notes":

• MAINTAINERS.md lists no non-steward maintainer of record.
• No external comments were filed during the 48h the window was open.
• All 6 acceptance criteria are satisfied at promotion time (spec text + SECURITY invariant + Postgres reference impl + 3 conformance scenarios + CHANGELOG + INTEROP-MATRIX).

Future RFCs revert to the canonical 7-day window once MAINTAINERS.md lists a non-steward maintainer. The waiver applies only when both conditions hold (sole-steward repo AND zero external commenters); reviewers may revoke it for any specific RFC by filing a single comment in the window.

References

• RFC 0004 (Memory Layer — MemoryAdapter, SR-1)
• RFC 0002 (Agent Identity — AgentRef.memoryRef)
• spec/v1/capabilities.md §"memory (RFC 0004)"
• spec/v1/observability.md §"Canonical event vocabulary"
• SECURITY/invariants.yaml (SR-1)
• schemas/memory-entry.schema.json
• 2026-05-13 OpenWOP roadmap evaluation of the autonomous-agent-runtime feature set (this RFC is the one slice of that proposal with a defensible protocol footprint)