commit fbfb7bf548df7df2b2d30e631bf3257eb665a3b0 from: Isaac Meerleo date: Fri Jul 17 23:59:21 2026 UTC Complete Compaction and System epoch tickets commit - babdd4d2b8a85b16001e26430738ef37618ebc3b commit + fbfb7bf548df7df2b2d30e631bf3257eb665a3b0 blob - f5a35261d507e6d228c2696687260ef55e26c2f1 blob + f10f6af073ebd926401db81471da6cf8b3776d3f --- .scratch/context-compaction/issues/02-cumulative-file-tracking-in-summaries.md +++ .scratch/context-compaction/issues/02-cumulative-file-tracking-in-summaries.md @@ -1,6 +1,7 @@ # Carry cumulative file tracking through compaction summaries Status: ready-for-agent +Resolution: completed Type: task ## What to build @@ -28,27 +29,27 @@ compaction can forget files that were important before ## Acceptance criteria -- [ ] A length-aware collector extracts `path` only from the supported +- [x] A length-aware collector extracts `path` only from the supported structured file-tool arguments; shell command text is never guessed as a path. -- [ ] Read-only paths appear under `` and any path written or +- [x] Read-only paths appear under `` and any path written or edited appears only under ``, regardless of operation order. -- [ ] Lists are bytewise deterministic, sorted, and deduplicated across the +- [x] Lists are bytewise deterministic, sorted, and deduplicated across the current span and every prior fugu-generated compaction. -- [ ] The final XML-like blocks are generated by fugu and appended once after +- [x] The final XML-like blocks are generated by fugu and appended once after model prose; the model is not trusted to preserve or generate the lists. -- [ ] Paths with newlines, controls, tag delimiters, invalid UTF-8, or excessive +- [x] Paths with newlines, controls, tag delimiters, invalid UTF-8, or excessive length cannot forge block structure; the encoding/rejection rule is pinned and tested. -- [ ] The compact journal record stores optional structured `read_files` and +- [x] The compact journal record stores optional structured `read_files` and `modified_files` details, remains backward-compatible with old summary-only records, and restores the sets on resume. -- [ ] `/clear` clears cumulative file state. A failed or cancelled compaction +- [x] `/clear` clears cumulative file state. A failed or cancelled compaction changes neither the active summary nor its file sets. -- [ ] Two successive `/compact` operations, resume-then-compact, read-then-edit +- [x] Two successive `/compact` operations, resume-then-compact, read-then-edit promotion, duplicates, malformed tool input, and empty lists have regression coverage. -- [ ] `docs/compaction.md` documents the blocks, cumulative semantics, and that +- [x] `docs/compaction.md` documents the blocks, cumulative semantics, and that they describe structured tool calls rather than an audited filesystem log. ## Code-cost analysis @@ -85,3 +86,10 @@ filesystem outcomes; documentation and tests must stat The remaining policy question is resolved above using the specified pi-compatible default, so this ticket is fully specified and ready for an agent. + +2026-07-17 completed in `5ba13c5`. The bounded collector, modified-wins +normalization, deterministic generated blocks, Journal compatibility, resume, +clear, repeated-compaction, failure, and adversarial-path cases are covered by +the Compaction, Journal, and Turn regressions. Final validation at `babdd4d`: +normal and hardened OpenBSD gates each passed 38 suites and 13,244 checks with +zero failures; the `fugu-0.3.tgz` package gate also passed. blob - dbad885fd0eda2385b60b61284eada022524fafb blob + 20ca98cd754fa0e4b795eb3c7c41d691052cce5a --- .scratch/context-compaction/issues/03-auto-compaction-with-overflow-recovery.md +++ .scratch/context-compaction/issues/03-auto-compaction-with-overflow-recovery.md @@ -1,6 +1,7 @@ # Add automatic compaction with one-shot overflow recovery Status: ready-for-agent +Resolution: completed Type: task ## What to build @@ -48,40 +49,40 @@ Generation, and privilege-separated API worker. ## Acceptance criteria -- [ ] Auto-compaction defaults to enabled and uses named defaults of 16,384 +- [x] Auto-compaction defaults to enabled and uses named defaults of 16,384 reserve tokens and 20,000 kept tokens; configuration syntax, validation, and disable behavior are documented and tested. -- [ ] With a known effective window, the strict `>` threshold is evaluated on +- [x] With a known effective window, the strict `>` threshold is evaluated on every prospective Lead Generation, including continuations after tool results. With an unknown window, threshold compaction is skipped but typed overflow recovery remains available. -- [ ] `contextTokens` uses one documented conservative estimator over the +- [x] `contextTokens` uses one documented conservative estimator over the actual prospective Projection. Provider usage may calibrate it but stale usage from a prior request is never treated as the exact next-request size. -- [ ] The cut planner normally retains whole recent Turns up to the 20,000-token +- [x] The cut planner normally retains whole recent Turns up to the 20,000-token target and never starts retained context with a tool-result message or leaves a tool call without its result. -- [ ] A Turn larger than the keep target is split only at a valid User or +- [x] A Turn larger than the keep target is split only at a valid User or Assistant boundary. Its early prefix receives a distinct Turn-context summary that is merged with the history summary before the valid suffix. -- [ ] Compaction can replace a prefix while the transaction is idle or active +- [x] Compaction can replace a prefix while the transaction is idle or active without committing, abandoning, duplicating, or losing pending Turn state. -- [ ] The compact journal record durably identifies or contains the retained +- [x] The compact journal record durably identifies or contains the retained suffix, and resume reconstructs exactly the same provider-valid Projection. -- [ ] Provider adapters/API IPC expose a bounded typed context-overflow +- [x] Provider adapters/API IPC expose a bounded typed context-overflow category based on HTTP status and provider error type/code—not broad English substring matching. Unknown errors retain today's failure behavior. -- [ ] An overflow-rejected Generation with no accepted content compacts and is +- [x] An overflow-rejected Generation with no accepted content compacts and is retried once. A second overflow, any post-content failure, cancellation, or compaction failure ends the Turn conspicuously with no further retry. -- [ ] Tool side effects occur at most once across recovery; tests cover an +- [x] Tool side effects occur at most once across recovery; tests cover an overflow after a completed tool-result batch and prove tools are not rerun. -- [ ] Prompt-cache receipts reset after a cut, usage remains additive and +- [x] Prompt-cache receipts reset after a cut, usage remains additive and nonduplicated, and the UI distinguishes summarization from the retry. -- [ ] Manual `/compact` remains available and shares the serializer, summary +- [x] Manual `/compact` remains available and shares the serializer, summary policy, durable state, and safety checks rather than maintaining a second implementation. -- [ ] `docs/compaction.md`, `fugu(1)`, `fugu.conf(5)`, behavior, design, and +- [x] `docs/compaction.md`, `fugu(1)`, `fugu.conf(5)`, behavior, design, and verification docs define threshold, cuts, split Turns, overflow recovery, unknown-window behavior, and the exactly-once limit. @@ -141,3 +142,14 @@ maintainer decisions before this high-risk Turn/Journa 2026-07-17 implementation kickoff: the Owner authorized the complete four-item dependency chain. The three decisions above are now pinned; issue 02 remains the only implementation dependency. + +2026-07-17 completed across `3f64e35`, `d092131`, `9c156e4`, `c46c1d5`, +`d8dc62c`, `bc1fdeb`, `2eaf4e2`, and `babdd4d`. The implementation covers +strict proactive thresholds, bounded valid cuts and oversized-Turn splits, +durable retained suffixes, active transaction replacement, typed provider +overflow classification, one-shot recovery, tool exactly-once behavior, +cache/usage accounting, shared manual serialization, UI notices, and docs. +Final validation at `babdd4d`: normal and hardened OpenBSD gates each passed +38 suites and 13,244 checks with zero failures; the OpenBSD package gate +produced `fugu-0.3.tgz` with SHA256 +`3cf884366c25de4b2ec3a1125dfec1619e84b38a82cccfd7e65c1a1840057e5d`. blob - c775ba0e9ab1ddec037a88890f909614176ccf11 blob + e37bd339710e9ad3737bdd94d8d58dfde53aa045 --- .scratch/context-epochs/issues/02-persist-atomic-system-epoch-records.md +++ .scratch/context-epochs/issues/02-persist-atomic-system-epoch-records.md @@ -1,6 +1,7 @@ # Persist atomic system-baseline epoch records Status: ready-for-agent +Resolution: completed Type: task ## What to build @@ -64,38 +65,38 @@ epoch record has been appended and synchronized. ## Acceptance criteria -- [ ] An ADR accepts immutable reset-boundary System epochs and explicitly +- [x] An ADR accepts immutable reset-boundary System epochs and explicitly rejects live observation and chronological canonical System messages. -- [ ] `CONTEXT.md` defines **System baseline** separately from Projection, +- [x] `CONTEXT.md` defines **System baseline** separately from Projection, Project context, Compaction, and Projection cursor state. -- [ ] One versioned Journal record stores exact base, personal, project, +- [x] One versioned Journal record stores exact base, personal, project, summary, full-baseline, and no-external-context-baseline byte spans plus its boundary reason. -- [ ] The codec checks every source span, JSON expansion, final NDJSON record, +- [x] The codec checks every source span, JSON expansion, final NDJSON record, and Provider request bound before publishing a boundary. -- [ ] Fresh Sessions synchronize an exact epoch before their first Provider +- [x] Fresh Sessions synchronize an exact epoch before their first Provider request, including a base-only full variant under `project_context no`. -- [ ] `/clear` appends and synchronizes one complete epoch before resetting the +- [x] `/clear` appends and synchronizes one complete epoch before resetting the Projection; it uses the cached personal bytes, re-reads project context under existing policy, drops Compaction, and adopts the invocation's current base. -- [ ] Successful `/compact` synchronizes one complete replacement epoch before +- [x] Successful `/compact` synchronizes one complete replacement epoch before resetting the Projection and retains the epoch's exact base/context bytes. -- [ ] Failed, oversized, malformed, Tool-producing, or cancelled Compaction +- [x] Failed, oversized, malformed, Tool-producing, or cancelled Compaction writes no epoch and leaves the old epoch and Projection byte-identical. -- [ ] Epoch-aware replay never exposes a clear/Compaction reset without its +- [x] Epoch-aware replay never exposes a clear/Compaction reset without its complete matching System epoch, including crash points before append, after append, before sync, after sync, and before live-state publication. -- [ ] Legacy Journals retain existing `context`/`clear`/`compact` semantics and +- [x] Legacy Journals retain existing `context`/`clear`/`compact` semantics and gain one additive synchronized migration epoch before their first request. -- [ ] A partial migration tail is ignored and retried; a complete migration is +- [x] A partial migration tail is ignored and retried; a complete migration is reused rather than duplicated on every resume. -- [ ] Recognized malformed, unsupported, or over-bound epoch records fail +- [x] Recognized malformed, unsupported, or over-bound epoch records fail conspicuously instead of silently falling back to current configuration. -- [ ] Journal session listing and ordinary User/Assistant Turn replay remain +- [x] Journal session listing and ordinary User/Assistant Turn replay remain unchanged, and no destructive format-wide migration is introduced. -- [ ] The Projection cursor remains cold at fresh start, resume, `/clear`, and +- [x] The Projection cursor remains cold at fresh start, resume, `/clear`, and successful `/compact` exactly as before. -- [ ] No Role, imsg type, worker privilege, dependency, file watcher, or +- [x] No Role, imsg type, worker privilege, dependency, file watcher, or per-Generation source read is added. ## Tests @@ -128,3 +129,12 @@ compaction so automatic cuts reuse the same atomic epo 2026-07-17 research follow-up: OpenCode Context Epochs were evaluated at commit `0df2f6245a9cd966c0912e12db2c9d809e0c589f`. Fugu is adopting only exact reset-boundary baseline persistence; chronological deltas were rejected. + +2026-07-17 completed across `8e964ca`, `f3f30b8`, and `18e7ebf`. System epoch +records are bounded, versioned, atomic, and transactional; replay/import/copy +preserve authoritative stored variants without recomposition, while summary +replacement preserves historical wrapper bytes. Journal and Turn-transaction +fixtures cover binary round trips, malformed records, legacy migration, +partial tails, ordering, and crash boundaries. Final validation at `babdd4d`: +normal and hardened OpenBSD gates each passed 38 suites and 13,244 checks with +zero failures; the `fugu-0.3.tgz` package gate also passed. blob - e5214655c7bca218666acc1609e5c8b1703bcd46 blob + f0b60417cf24d79a8d189e3340aa1a224cb41fb1 --- .scratch/context-epochs/issues/03-restore-policy-selected-system-epochs.md +++ .scratch/context-epochs/issues/03-restore-policy-selected-system-epochs.md @@ -1,6 +1,7 @@ # Restore policy-selected System epochs on resume Status: ready-for-agent +Resolution: completed Type: task ## What to build @@ -41,41 +42,41 @@ Journal or Provider Projection. ## Acceptance criteria -- [ ] Resume under `project_context yes` sends the stored full Lead baseline +- [x] Resume under `project_context yes` sends the stored full Lead baseline byte-for-byte through Anthropic, OpenAI, and Claude Subscription builders. -- [ ] Resume under `project_context no` sends the stored exact +- [x] Resume under `project_context no` sends the stored exact base-plus-Compaction variant and never sends stored personal/project bytes. -- [ ] Toggling the policy across repeated resumes selects the corresponding +- [x] Toggling the policy across repeated resumes selects the corresponding stored variant without rewriting or deleting either variant. -- [ ] A changed configured `system`, changed built-in prompt fixture, changed +- [x] A changed configured `system`, changed built-in prompt fixture, changed composition implementation, or changed context files cannot alter either selected stored baseline. -- [ ] `/clear` creates a fresh epoch from the current invocation's configured +- [x] `/clear` creates a fresh epoch from the current invocation's configured or built-in base, cached personal bytes, existing project re-read policy, and no Compaction summary. -- [ ] Successful `/compact` preserves the pinned epoch base/context and changes +- [x] Successful `/compact` preserves the pinned epoch base/context and changes only the summary-derived baseline bytes; failure preserves the old epoch. -- [ ] `#` capture and project edits leave the active epoch unchanged until +- [x] `#` capture and project edits leave the active epoch unchanged until `/clear` or a new Session; personal-file edits apply only to a new Session. -- [ ] Resumed Subagents use the pinned epoch base and selected context, retain +- [x] Resumed Subagents use the pinned epoch base and selected context, retain current read-only guidance, and never receive Compaction notes. -- [ ] `/context` counts the selected stored Lead baseline and reports stable +- [x] `/context` counts the selected stored Lead baseline and reports stable values when only current `conf->system` or external files differ. -- [ ] The restricted-variant notice appears exactly once per applicable resume +- [x] The restricted-variant notice appears exactly once per applicable resume in line, print, and curses presentation, and is absent from Journal replay and Provider requests. -- [ ] Current provider/model, Tool/skill catalogue, credential, endpoint, +- [x] Current provider/model, Tool/skill catalogue, credential, endpoint, protect, write, and network configuration remains authoritative and is never copied into an epoch record. -- [ ] Provider/model switching leaves the epoch bytes unchanged and keeps the +- [x] Provider/model switching leaves the epoch bytes unchanged and keeps the existing cold Projection-cursor behavior. -- [ ] Fresh, migrated-legacy, `/clear`, `/compact`, and repeated-resume paths +- [x] Fresh, migrated-legacy, `/clear`, `/compact`, and repeated-resume paths all produce Provider-valid requests with only canonical User/Assistant conversation roles. -- [ ] `handoff/behavior.md`, `handoff/invariants.md` I14, +- [x] `handoff/behavior.md`, `handoff/invariants.md` I14, `handoff/verification.md`, `docs/compaction.md`, `fugu(1)`, and `fugu.conf(5)` document selection, explicit admission, migration, and live-policy limits. -- [ ] Normal, hardened, Provider-codec, Subagent, Journal, Turn, package, and +- [x] Normal, hardened, Provider-codec, Subagent, Journal, Turn, package, and release gates remain green. ## Tests @@ -119,3 +120,11 @@ state and selecting one stored variant. 2026-07-17 research follow-up: this ticket preserves current authority-policy selection while making the selected privileged bytes exact. It deliberately does not persist live capabilities or introduce chronological context updates. + +2026-07-17 completed across `9e7c6a0`, `3f83e19`, `bc1fdeb`, `18e7ebf`, and +`4150f84`. Cross-process fixtures prove exact full/restricted selection, +policy toggling, composition and file drift resistance, migration reuse, +provider/model liveness, stable `/context`, one-shot line/curses notices, and +Lead/Subagent policy. Final validation at `babdd4d`: normal and hardened +OpenBSD gates each passed 38 suites and 13,244 checks with zero failures; the +`fugu-0.3.tgz` package gate also passed.