commit b39f351abea85b29012c34e4fdc8b97fb800260a from: Isaac Meerleo date: Tue Jul 21 21:26:24 2026 UTC Fix Codex subscription continuity replay Translate response state into typed request-safe items, add versioned continuity and legacy replay support, and enforce the boundary with codec, Journal, and TLS regressions. commit - c2901acefe02363a47b8ac841c82b8523d928de4 commit + b39f351abea85b29012c34e4fdc8b97fb800260a blob - 88453f0dacab0d7099d9e57b402be77ffc8a96e5 blob + 4df6087452ab907cba7e43f8c08b6b4f43c63ad3 --- .scratch/codex-subscription-adapter/issues/01-implement-pure-c-codex-subscription-adapter.md +++ .scratch/codex-subscription-adapter/issues/01-implement-pure-c-codex-subscription-adapter.md @@ -163,13 +163,22 @@ profile = chatgpt-codex-2026-07-17 model = kind = responses-continuity data = { - reasoning: [, ...], - calls: [, ...] + version: 2, + items: [ + , + , + , ... + ] } ``` -- Retain only final reasoning items with nonempty encrypted content and final - function-call identity records. +- Retain only typed, request-relevant fields from final reasoning, message, + and function-call items in exact contiguous output order. Never retain raw + response objects or synthesize a request-visible `status`. +- Bind every retained message id and phase to an exact offset/length/SHA-256 + span of canonical Assistant text. +- Dual-read legacy unversioned state, requiring its completed status as legacy + evidence but stripping that response-only field when rendering a request. - Cross-check `response.output_item.done` with any copy in `response.completed.output`; duplicate conflicts fail. - Bound the complete attachment to 2 MiB per Generation on every side of the @@ -185,7 +194,9 @@ data = { - Preserve it on committed resume; discard it with messages on abandonment, `/clear`, and `/compact`. - Keep Subagent state only in the Subagent's existing in-memory conversation. -- Replay only exact Provider/profile/model matches. Other Providers ignore it. +- Semantically validate only an exact Provider/profile/model/kind match. + Mismatched state remains attached as opaque inert history and is omitted + from the current Projection; matching corruption invalidates its Turn. - Join call metadata to canonical Tool blocks by exact `call_id`; provider state never supplies Tool names or arguments and never gains Tool authority. - Do not concatenate the two Responses ids into `block.tool_id`. @@ -458,3 +469,21 @@ was not run and no personal token or account id was re 17 remain unchecked, `Status` remains `needs-info`, and no Resolution is added until the Owner supplies scrubbed live HTTP SSE, Tool-continuation, continuity, and secret-absence evidence. + +2026-07-21 continuity correction: a live second Generation exposed that the +original stream Adapter synthesized reasoning `status:"completed"` into +Provider state and the request Adapter later raw-spliced that response object +into `input`, producing `Unknown parameter: 'input[1].status'`. The regression +suite had separately blessed both halves and never composed +stream -> Provider state -> next request; the local TLS Provider also accepted +every request body without a schema oracle. + +The continuity seam now writes strict typed version-2 state, reconstructs +reasoning/message/call request items field by field, preserves message +identity/phase and exact output order with text-span digests, and never emits +response status. Legacy version-1 Journal state remains readable and is +translated status-free. Matching corrupt state still fails closed, while +profile/model/kind mismatch remains attached but inert. Hermetic evidence is +2568/0 in the focused Codex suite and 430/0 in Journal on OpenBSD; the print TLS +stub builds with an independent request-side input-item schema oracle. The +Owner-only credentialed live criteria remain unchanged and `needs-info`. blob - da3fdea0e9b780c913bad0a001b23cd333553120 blob + 29799ac54a98c7cef146f75794aed27f1bde1f49 --- docs/adr/0010-experimental-codex-subscription-compatibility.md +++ docs/adr/0010-experimental-codex-subscription-compatibility.md @@ -55,14 +55,34 @@ until a separate design can atomically persist rotated moving them out of their Custodian. Fugu sends complete canonical history and never makes -`previous_response_id` authoritative. Completed encrypted reasoning and the -provider output-item ids paired with canonical function `call_id` values are -retained in a generic, bounded Provider-state attachment. The attachment is +`previous_response_id` authoritative. Completed encrypted reasoning, message +identities and phases, and provider output-item ids paired with canonical +function `call_id` values are retained in a generic, bounded Provider-state +attachment. The current data format is a strict version-2 tagged sequence of +reasoning, message, and call records in contiguous output-index order. Message +records bind their id and optional `commentary`/`final_answer` phase to exact +offset/length/SHA-256 spans of the canonical Assistant text. The attachment is atomic with its Assistant message, survives only a committed Lead Turn and resume, and is discarded with the Projection by abandonment, `/clear`, or `/compact`. Canonical Tool ids, names, arguments, and authority never come -from that opaque state. +from that state. +Continuity is directional translation, not raw JSON replay. The hostile stream +parser accepts only allowlisted response fields and writes a canonical +status-free representation. The request Adapter validates the attachment +against the canonical message and renders a fresh request item field by field; +inbound completion `status`, output-text metadata, terminal response metadata, +and unknown response members are never copied into `input`. Existing +unversioned version-1 Journal state remains readable: its required legacy +`status:"completed"` is validated and stripped during translation. New state +is written only as version 2. + +The active Provider/profile/model/kind owns semantic validation. Matching +corrupt state fails closed and invalidates its containing Journal Turn. +Mismatched state remains opaque, attached, and inert history so that profile +evolution does not delete otherwise valid conversation; the current request +builder omits it. + For this profile `max_tokens` remains a local Fugu ceiling; the request omits `max_output_tokens`, so it is not an exact provider-side output-token limit. The compiled Effective window uses the raw 272,000-token catalog value and @@ -104,12 +124,29 @@ phases, array-shaped output-text `annotations` / `logp terminal output array only after all streamed items are complete. The same pinned schema gives reasoning completion through the enclosing final-item event rather than an item `status`, and classifies `response.metadata` as advisory. -Fugu therefore accepts absent status on final reasoning items, normalizes its -retained copy to `status:"completed"`, and validates then discards only the -exact response-bound HTTP metadata envelope. Metadata never becomes text, Tool -authority, usage, or Provider state. These are decoder corrections within the -pinned event vocabulary, not a new request, route, model, or transport profile. +Fugu therefore accepts absent status on final reasoning items, validates an +optional status as exactly `completed`, and omits it from retained version-2 +state and every subsequent request. It validates then discards the exact +response-bound HTTP metadata envelope. Metadata never becomes text, Tool +authority, usage, Provider state, or request input. Exact message +identity/phase/text-span records preserve multiple message boundaries and their +position among reasoning and function calls. These are codec corrections +within the pinned event vocabulary, not a new request, route, model, or +transport profile. +Reasoning summary/content text remains length-bearing across both legacy and +version-2 translation, including embedded NUL. Nested reasoning-part vectors +are bounded by the parser token budget and Provider-state byte ceiling, not by +the separate top-level output-item limit. + +Regression coverage crosses the architectural seams: one fixture drives a +statusless response through stream decoding, version-2 state capture, and the +next request; interleaved phase/call fixtures prove exact order; Journal tests +cover legacy translation, version-2 replay, inert mismatch, and matching +corruption; and the hermetic TLS stub independently validates every outbound +input-item variant and rejects unknown or response-only request fields with an +`input[...]` parameter oracle. + The opt-in live gate never runs under `make check` and never asks a developer or agent to copy a credential. It proves one text response, one harmless `read` Tool continuation, continuity capture, honest identity, and absence of blob - 3afdcaa1250a614764bb45f2620f8e0a3d647d40 blob + 384e4cdb92cdd47fcebda0f71bd7444b8ef53365 --- docs/design/codex-subscription.md +++ docs/design/codex-subscription.md @@ -1,6 +1,7 @@ # Experimental pure-C Codex subscription adapter -Status: proposed design, 2026-07-17. No production code has been written. +Status: implemented experimental design, 2026-07-17; continuity contract +revised 2026-07-21. Release remains subject to the acceptance gates below. Protocol constants and the evidence behind them are pinned in [`docs/research/codex-subscription-native-clients.md`](../research/codex-subscription-native-clients.md). @@ -287,24 +288,26 @@ messages. The implementation owns all of these details reuses the key. Do not add `client_metadata`, a random request id, or a prompt, path, username, account identifier, or credential to request metadata. -10. A matching opaque continuity state attached to an assistant message - restores reasoning items and function-call item ids while rendering text, - Tool names, arguments, and results from canonical blocks. Mismatched - Provider, profile, or model state is omitted. +10. A matching typed continuity state attached to an assistant message + restores reasoning items, message identities/phases, and function-call + item ids in exact output-item order while rendering text, Tool names, + arguments, and results from canonical blocks. State with a mismatched + Provider, profile, model, or kind remains inert and is omitted. Tool names remain canonical unless the pinned fixture proves a reversible wire mapping is required. Never copy Codex's built-in tool definitions or system prompt: Fugu is the agent and its Palette is the authority. -### Opaque continuation state +### Versioned continuation state -Encrypted reasoning is required v1 state, not a discovery gate. OpenAI and -independent clients request, retain, and replay the completed reasoning item so -that a `store:false` function-call conversation can continue. Responses also -uses both a canonical `call_id` and a provider output-item id such as `fc_...`; -independent clients preserve the pair because reasoning/function-call pairing -can be validated by those item ids. Fugu carries both through a generic -envelope while keeping Codex item semantics inside the Adapter. +Encrypted reasoning is required continuity state, not a discovery gate. OpenAI +and independent clients request, retain, and replay completed reasoning so that +a `store:false` function-call conversation can continue. Responses also uses +message and function-call output-item identities, optional message phases, and +both a canonical `call_id` and a provider output-item id such as `fc_...`. +Fugu carries the minimum typed representation through a generic envelope while +keeping Codex item semantics inside the Adapter. The generic carrier treats the +data as opaque bytes; the Codex stream and request codecs do not. Extend `struct msg` with an optional provider-state attachment separate from its canonical content blocks. The attachment is one bounded byte string with @@ -316,39 +319,78 @@ profile = chatgpt-codex-2026-07-17 model = kind = responses-continuity data = { - reasoning: [, ...], - calls: [, ...] + version: 2, + items: [ + {index, kind: "reasoning", item: { + type: "reasoning", id, summary, [content], encrypted_content + }}, + {index, kind: "message", item_id, [phase], + text_offset, text_len, text_sha256}, + {index, kind: "call", call_id, item_id} + ] } ``` The envelope is generic infrastructure; `msg` and the Journal copy, free, and round-trip it but do not understand the data. It is not a text block, Tool block, UI event, summary, or token credential. Other Provider request builders -ignore it. `codex_sub_build_request()` is the only credential-free consumer and -revalidates the envelope and each raw JSON object before emitting state that -matches the active Provider, profile, and model. +ignore it. Version 2 is a strict tagged union: root and item fields are +allowlisted, indices are contiguous and equal array positions, identities are +bounded, prefix-checked, and unique, and only the exact optional +`commentary`/`final_answer` phase values are accepted. Reasoning summary and +content parts are likewise parsed through fixed typed allowlists. Their text +is length-bearing (including embedded NUL), and their nested array size is +bounded by the JSON-token and 2 MiB Provider-state limits rather than the +unrelated top-level output-item count. +Message records bind provider message identity and phase to the canonical +Assistant text without copying text into the state. Their contiguous +offset/length spans must exactly cover the flattened canonical text. Each span +carries a lowercase SHA-256 digest. This lets the request Adapter reconstruct +distinct message items around interleaved reasoning and calls while detecting +reordered, truncated, or substituted Journal text. Function names and +arguments are never trusted from state: call records join by exact `call_id` +to canonical Tool blocks. Version 2 therefore preserves the response's +complete item order without turning the attachment into a raw response replay +cache. + +`codex_sub_build_request()` is the only credential-free consumer. It +revalidates a matching envelope, reconstructs every request item field by +field, and never raw-splices a retained response object into `input`. Inbound +completion `status`, output-text annotations/logprobs, terminal response +metadata, and all other response-only fields are validated where applicable +and discarded; none can appear in a later request merely because the backend +sent it. + The request Adapter joins call metadata to canonical Tool blocks by exact -`call_id`; it never trusts state for the Tool name or arguments. Missing state -may omit an optional item id, but duplicate, conflicting, unknown, or -many-to-one mappings fail request construction. The completed output indices -preserve reasoning/function-call order. Do not concatenate ids into -`block.tool_id`: Tool execution and Tool results continue to use only the -canonical `call_id`. +`call_id`; it never trusts state for the Tool name or arguments. Duplicate, +conflicting, unknown, missing, or many-to-one version-2 mappings fail request +construction. Do not concatenate ids into `block.tool_id`: Tool execution and +Tool results continue to use only the canonical `call_id`. +Legacy Journal attachments may use an unversioned version-1 object with +separate `reasoning` and `calls` arrays. That form remains dual-read for +resume compatibility. Its reasoning item must carry the legacy +`status:"completed"`, but the Adapter parses it into the same typed reasoning +representation and emits a newly rendered, status-free request item. New +streams write only version 2. Neither version permits response metadata to be +replayed raw. + The hostile path is explicit: -1. `codex_sub_stream` accepts only final reasoning and function-call metadata; - partial items are never retained. A reasoning item requires nonempty - `encrypted_content`; its enclosing final-item event supplies completion, so - `status` may be absent but, when present, must be exactly `completed`. The - retained canonical copy always contains `status:"completed"`. A call record - requires bounded, distinct `call_id` and provider item id. It normally takes - `response.output_item.done`. If `response.completed.output` repeats an item, - identity and canonical content must agree; if terminal output is the only - final copy, it supplies the item. A terminal empty output array is accepted - only after every contiguous streamed item is already complete. Duplicate, - unfinished, missing, or conflicting copies fail. +1. `codex_sub_stream` accepts only final, typed reasoning, message, and + function-call metadata; partial items are never retained. A reasoning item + requires nonempty `encrypted_content`; its enclosing final-item event + supplies completion, so `status` may be absent but, when present, must be + exactly `completed`. The retained canonical copy is status-free. Message + records retain only identity, optional phase, and the text-span binding; + call records retain only bounded, distinct `call_id` and provider item id. + It normally takes `response.output_item.done`. If + `response.completed.output` repeats an item, identity and canonical content + must agree; if terminal output is the only final copy, it supplies the + item. A terminal empty output array is accepted only after every contiguous + streamed item is already complete. Duplicate, unfinished, missing, + out-of-order, or conflicting copies fail. 2. It emits `ASTREAM_PROVIDER_STATE` records to `fugu-api`. The Custodian validates each complete record, applies credential-reflection filtering, and builds one deterministic continuity object under a 2 MiB @@ -361,12 +403,15 @@ The hostile path is explicit: Projection visibility. Existing Turn commit/abandon replay rules therefore make the state atomic with the Tool call that needs it. -Journal `msg` records encode the opaque data as a JSON string, not as raw -record syntax. Replay requires the exact provider/profile/model/kind fields, a -single valid data object, unique output/call/item identities, and all -byte/count bounds. A malformed attachment -invalidates the containing Turn just as a malformed Tool block does; it is -never partially recovered. +Journal `msg` records encode the continuity data as a JSON string, not as raw +record syntax. Every attachment must have one bounded, syntactically valid +generic envelope. When Provider/profile/model/kind match the active Codex +profile, replay also validates the owned version-1 or version-2 schema against +the canonical Assistant message. Corruption in matching state invalidates the +containing Turn just as a malformed Tool block does; it is never partially +recovered. Compatibility-mismatched state remains attached as opaque, inert +history, because a newer Adapter does not own the schema of an older profile; +the current request builder ignores it. Lead state survives a committed resume and is discarded by abandonment, `/clear`, and `/compact` with the messages it describes. Subagent state lives @@ -396,7 +441,8 @@ The decoder must: - accept output-text deltas with explicit lengths; - accept absent or exact `commentary` / `final_answer` message phases while - preserving Fugu's existing canonical text ordering; + recording message identity, phase, and canonical text-span binding for exact + reconstruction around reasoning and Tool calls; - accept only array-shaped optional `annotations` and `logprobs` metadata on output-text parts; neither metadata field crosses the canonical Provider seam; @@ -415,9 +461,10 @@ The decoder must: input only as the `cache_read` detail, matching the current OpenAI accounting convention; - reject output Tool kinds Fugu did not offer; -- retain only completed reasoning items and function-call identity records, - require nonempty encrypted content and distinct bounded ids, and preserve - their response order in one bounded continuity object; +- retain only typed fields from completed reasoning, message, and function-call + items, require nonempty encrypted content and distinct bounded ids, and + preserve their exact contiguous response order in one bounded version-2 + continuity object; - accept `response.metadata` only as the exact observed HTTP SSE envelope: matching bounded `response_id`, nonnegative `INT_MAX`-bounded integer `sequence_number`, and a nonempty object-valued `metadata`, with no duplicate @@ -498,13 +545,13 @@ configuration copy, and never receives it again. `fugu inet dns`, cannot open or write files, cannot execute, and remains the sole Role linked with libtls and the hostile Codex stream parser. -The only new data crossing back from the Custodian is the bounded reasoning -envelope. It is not an OAuth token, cannot select a Tool or endpoint, is -credential-reflection filtered, and is never displayed. The Coordinator treats -live bytes opaquely; journal replay and the request Adapter independently -revalidate their exact JSON envelope before they can be sent back to the same -fixed Provider/profile. This adds data handling, not authority, to the -Coordinator. +The only new data crossing back from the Custodian is the bounded typed +continuity envelope. It is not an OAuth token, cannot select a Tool or +endpoint, is credential-reflection filtered, and is never displayed. The +Coordinator treats live bytes opaquely; Journal replay and the request Adapter +independently revalidate matching state before its typed fields can be +rendered for the same fixed Provider/profile/model/kind. Mismatched state +remains inert. This adds data handling, not authority, to the Coordinator. The profile does require normative wording changes before acceptance: @@ -527,8 +574,8 @@ Expected production changes: | Area | Files | Change | |---|---|---| | Profile interface | `src/common/codex_sub.h`, `codex_sub_profile.c` | Fixed profile, token/account validation, constants | -| Request codec | `src/common/codex_sub_req.c` | Canonical Projection to Responses input | -| Stream codec | `src/common/codex_sub_stream.c` | Responses SSE to `astream_event` | +| Request codec | `src/common/codex_sub_req.c`, `codex_sub_reasoning.c` | Canonical Projection and typed continuity to status-free Responses input | +| Stream codec | `src/common/codex_sub_stream.c`, `codex_sub_reasoning.c` | Responses SSE to canonical events and typed version-2 continuity | | Credentialed wire | `src/common/codex_sub_wire.c` | Bearer/account/profile headers and bounded errors | | State carrier | `src/common/msg.[ch]`, `generation.[ch]`, `provider_stream.h` | Generic bounded attachment and event span | | Durable state | `src/fugu/journal.c`, `turn_txn.c`, `proto.h` | Transactional encoding/replay and chunked imsg | @@ -544,23 +591,31 @@ compaction, or UI redesign. ## Verification -Add `regress/codex_subscription/` with fixture-driven tests for: +The fixture-driven regression contract in `regress/codex_subscription/` +covers: 1. exact request JSON for text, mixed text/Tool history, Tool-only assistant output, Tool errors, steers, compaction suffixes, empty optional fields, - matching continuity state, preserved function-call item ids, and omitted - mismatched profile/model state; + matching version-2 continuity state, exact reasoning/message/call order, + preserved message ids/phases/text spans and function-call item ids, + status-free outbound reasoning, legacy version-1 translation, and omitted + mismatched Provider/profile/model/kind state. Reasoning fixtures also cover + embedded-NUL text and nested part counts above the top-level output bound; 2. exact fixed headers with a synthetic JWT and account claim; wrong issuer, audience, expiry, claim type, NUL, CR/LF, oversize token, and ambiguous account fail closed; 3. every SSE byte split, multi-event reads, interleaved function calls, fragmented names/arguments, usage, normal completion, explicit failure, incomplete response, duplicate/conflicting items, output after terminal, - statusless final versus partial or non-completed reasoning items, canonical - completed-state normalization, multiple reasoning items, exact interleaved + statusless final versus partial or non-completed reasoning items, status-free + typed-state construction, multiple reasoning items, exact interleaved `response.metadata`, malformed metadata envelopes, `call_id`/item-id pairing and conflicts, unknown Tool types, malformed JSON, embedded NUL, and every - byte/count cap; + byte/count cap. A direct stream-to-state-to-next-request regression + reproduces the original multi-Turn failure and proves that no inbound + `status` reaches the next request; another round-trip interleaves reasoning, + commentary, calls, final answer, and another call to prove exact order and + phase fidelity; 4. credential reflection split across adjacent text, Tool, reasoning-state, error, and HTTP-body fragments; 5. config acceptance/redaction/wiping for implicit and named `codex` routes, @@ -570,11 +625,15 @@ Add `regress/codex_subscription/` with fixture-driven 6. full TLS worker tests proving the Coordinator sends credential-free JSON, only `fugu-api` adds auth, Lead and Subagent Tool rounds complete, 401/403 fail once, 429/5xx follow bounded retry, and cancellation preserves the Turn - transaction; -7. provider-state imsg chunking, normal-completion attachment, Journal - round-trip, fresh-process Tool continuation, abandoned-turn exclusion, - `/clear`, `/compact`, corrupted records, profile/model mismatch, and the 2 - MiB cap on both sides of each seam; + transaction. The fixture includes an independent provider-side request + oracle that strictly checks every emitted input-item variant and rejects + response-only or unknown members with the offending `input[...]` parameter, + rather than trusting Fugu's own expected byte string; +7. provider-state imsg chunking, normal-completion attachment, legacy and + version-2 Journal round-trips, fresh-process Tool continuation, + abandoned-turn exclusion, `/clear`, `/compact`, matching-state corruption, + inert profile/model/kind mismatch, and the 2 MiB cap on both sides of each + seam; 8. route switching, Context accounting without cache double-counting, cold cache state, and exact fixed-model/no-discovery behavior; 9. existing ktrace proofs that the Custodian cannot open, write, or execute. blob - 2847a11e8051afa413f4b8f5a50ef0e156daaf0d blob + 6313b809776cda76f67de26587ed898148db833a --- regress/codex_subscription/Makefile +++ regress/codex_subscription/Makefile @@ -1,6 +1,7 @@ PROG= codex_subscription_test SRCS= codex_subscription_test.c codex_sub_profile.c codex_sub_req.c \ - codex_sub_stream.c provider_state.c msg.c sse.c json.c buf.c log.c \ + codex_sub_stream.c codex_sub_reasoning.c provider_state.c msg.c sse.c \ + json.c buf.c log.c \ codex_sub_wire.c compaction.c model_window.c utf8.c xmalloc.c CFLAGS+= -I${.CURDIR}/../../src/fugu blob - 9495d2581e7d7a66b6af3b4d89e9b21a8f2ab6e1 blob + a80ea63848a568f4d5f31fa4d7b12924ed770032 --- regress/codex_subscription/codex_subscription_test.c +++ regress/codex_subscription/codex_subscription_test.c @@ -262,7 +262,7 @@ test_mixed_history_and_matching_continuity(void) "\"input\":[{\"type\":\"message\",\"role\":\"user\"," "\"content\":[{\"type\":\"input_text\",\"text\":\"question\"}]}," "{\"type\":\"reasoning\",\"id\":\"rs_1\",\"summary\":[]," - "\"encrypted_content\":\"cipher\",\"status\":\"completed\"}," + "\"encrypted_content\":\"cipher\"}," "{\"type\":\"message\"," "\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\"," "\"text\":\"working\"}]},{\"type\":\"function_call\",\"id\":" @@ -298,6 +298,8 @@ test_mixed_history_and_matching_continuity(void) buf_init(&out); CHECK(codex_sub_build_request(&out, &r) == 0); CHECK(memmem(out.data, out.len, want_input, sizeof(want_input) - 1) != NULL); + CHECK(memmem(out.data, out.len, "\"status\":\"completed\"", + sizeof("\"status\":\"completed\"") - 1) == NULL); CHECK(memmem(out.data, out.len, "call_1fc_1", 11) == NULL); buf_free(&out); msg_free(suffix); @@ -388,7 +390,7 @@ reasoning_state_rejected(const char *data) } static void -test_reasoning_state_requires_completed_status(void) +test_legacy_reasoning_state_requires_completed_status(void) { static const char missing[] = "{\"reasoning\":[{\"index\":0,\"item\":{\"type\":\"reasoning\"," @@ -409,6 +411,274 @@ test_reasoning_state_requires_completed_status(void) } static void +test_v2_continuity_rejects_corruption_and_unknown_fields(void) +{ + static const char status[] = + "{\"version\":2,\"items\":[{\"index\":0,\"kind\":\"reasoning\"," + "\"item\":{\"type\":\"reasoning\",\"id\":\"rs_status\"," + "\"summary\":[],\"encrypted_content\":\"opaque\",\"status\":" + "\"completed\"}},{\"index\":1,\"kind\":\"message\"," + "\"item_id\":\"msg_answer\",\"text_offset\":0,\"text_len\":6," + "\"text_sha256\":" + "\"0db52f4076c082518412afd3dd3576e2cb0c63703fd7fed5e23ade60efef31d9\"}]}"; + static const char bad_digest[] = + "{\"version\":2,\"items\":[{\"index\":0,\"kind\":\"message\"," + "\"item_id\":\"msg_answer\",\"text_offset\":0,\"text_len\":6," + "\"text_sha256\":" + "\"0db52f4076c082518412afd3dd3576e2cb0c63703fd7fed5e23ade60efef31d8\"}]}"; + static const char bad_offset[] = + "{\"version\":2,\"items\":[{\"index\":0,\"kind\":\"message\"," + "\"item_id\":\"msg_answer\",\"text_offset\":1,\"text_len\":5," + "\"text_sha256\":" + "\"4ef660738e0ecc285e2f5a93286bc226eb510d7b21a72e5a66a265bf4dfb69cb\"}]}"; + static const char bad_length[] = + "{\"version\":2,\"items\":[{\"index\":0,\"kind\":\"message\"," + "\"item_id\":\"msg_answer\",\"text_offset\":0,\"text_len\":5," + "\"text_sha256\":" + "\"4c54a73b2193e3490e99770b02a049cfdaa80a5358f439aa86939529b3813efc\"}]}"; + static const char unknown_summary_field[] = + "{\"version\":2,\"items\":[{\"index\":0,\"kind\":\"reasoning\"," + "\"item\":{\"type\":\"reasoning\",\"id\":\"rs_summary\"," + "\"summary\":[{\"type\":\"summary_text\",\"text\":\"brief\"," + "\"unexpected\":true}],\"encrypted_content\":\"opaque\"}},{" + "\"index\":1,\"kind\":\"message\",\"item_id\":\"msg_answer\"," + "\"text_offset\":0,\"text_len\":6,\"text_sha256\":" + "\"0db52f4076c082518412afd3dd3576e2cb0c63703fd7fed5e23ade60efef31d9\"}]}"; + static const char unknown_content_field[] = + "{\"version\":2,\"items\":[{\"index\":0,\"kind\":\"reasoning\"," + "\"item\":{\"type\":\"reasoning\",\"id\":\"rs_content\"," + "\"summary\":[],\"content\":[{\"type\":\"text\",\"text\":\"opaque\"," + "\"unexpected\":true}],\"encrypted_content\":\"cipher\"}},{" + "\"index\":1,\"kind\":\"message\",\"item_id\":\"msg_answer\"," + "\"text_offset\":0,\"text_len\":6,\"text_sha256\":" + "\"0db52f4076c082518412afd3dd3576e2cb0c63703fd7fed5e23ade60efef31d9\"}]}"; + + CHECK(reasoning_state_rejected(status)); + CHECK(reasoning_state_rejected(bad_digest)); + CHECK(reasoning_state_rejected(bad_offset)); + CHECK(reasoning_state_rejected(bad_length)); + CHECK(reasoning_state_rejected(unknown_summary_field)); + CHECK(reasoning_state_rejected(unknown_content_field)); +} + +static int +build_reasoning_content_request(struct buf *out, const char *state_data) +{ + struct codex_sub_req request; + struct msglist conv; + struct msg *message; + int ret; + + TAILQ_INIT(&conv); + message = text_message(ROLE_ASSISTANT, "answer"); + message->provider_state = provider_state_new(PROVIDER_CODEX, + "chatgpt-codex-2026-07-17", "gpt-5.5", "responses-continuity", + state_data, strlen(state_data)); + if (message->provider_state == NULL) { + msg_free(message); + return (-1); + } + append(&conv, message); + request_defaults(&request, &conv); + buf_init(out); + ret = codex_sub_build_request(out, &request); + msglist_free(&conv); + return (ret); +} + +static void +test_v2_reasoning_content_translation(void) +{ + static const char reasoning_text_state[] = + "{\"version\":2,\"items\":[{\"index\":0,\"kind\":\"reasoning\"," + "\"item\":{\"type\":\"reasoning\",\"id\":\"rs_reasoning_text\"," + "\"summary\":[],\"content\":[{\"type\":\"reasoning_text\"," + "\"text\":\"hidden\"}],\"encrypted_content\":\"cipher_hidden\"}},{" + "\"index\":1,\"kind\":\"message\",\"item_id\":\"msg_answer\"," + "\"text_offset\":0,\"text_len\":6,\"text_sha256\":" + "\"0db52f4076c082518412afd3dd3576e2cb0c63703fd7fed5e23ade60efef31d9\"}]}"; + static const char text_state[] = + "{\"version\":2,\"items\":[{\"index\":0,\"kind\":\"reasoning\"," + "\"item\":{\"type\":\"reasoning\",\"id\":\"rs_text\"," + "\"summary\":[],\"content\":[{\"type\":\"text\",\"text\":\"opaque\"}]," + "\"encrypted_content\":\"cipher_text\"}},{\"index\":1,\"kind\":" + "\"message\",\"item_id\":\"msg_answer\",\"text_offset\":0," + "\"text_len\":6,\"text_sha256\":" + "\"0db52f4076c082518412afd3dd3576e2cb0c63703fd7fed5e23ade60efef31d9\"}]}"; + static const char reasoning_without_content[] = + "{\"type\":\"reasoning\",\"id\":\"rs_reasoning_text\"," + "\"summary\":[],\"encrypted_content\":\"cipher_hidden\"}"; + static const char reasoning_with_text[] = + "{\"type\":\"reasoning\",\"id\":\"rs_text\",\"summary\":[]," + "\"content\":[{\"type\":\"text\",\"text\":\"opaque\"}]," + "\"encrypted_content\":\"cipher_text\"}"; + struct buf out; + + CHECK(build_reasoning_content_request(&out, reasoning_text_state) == 0); + CHECK(memmem(out.data, out.len, reasoning_without_content, + sizeof(reasoning_without_content) - 1) != NULL); + CHECK(memmem(out.data, out.len, "\"type\":\"reasoning_text\"", + sizeof("\"type\":\"reasoning_text\"") - 1) == NULL); + buf_free(&out); + + CHECK(build_reasoning_content_request(&out, text_state) == 0); + CHECK(memmem(out.data, out.len, reasoning_with_text, + sizeof(reasoning_with_text) - 1) != NULL); + buf_free(&out); +} + +static void +emit_reasoning_with_many_parts(struct json_out *out, const char *id, + int legacy_status) +{ + static const char embedded[] = { 'a', '\0', 'b' }; + int i; + + json_obj_begin(out); + json_key(out, "type"); + json_cstr(out, "reasoning"); + json_key(out, "id"); + json_cstr(out, id); + json_key(out, "summary"); + json_arr_begin(out); + for (i = 0; i < CODEX_SUB_OUTPUT_MAX + 1; i++) { + json_obj_begin(out); + json_key(out, "type"); + json_cstr(out, "summary_text"); + json_key(out, "text"); + if (i == 0) + json_str(out, embedded, sizeof(embedded)); + else + json_cstr(out, "s"); + json_obj_end(out); + } + json_arr_end(out); + json_key(out, "content"); + json_arr_begin(out); + json_obj_begin(out); + json_key(out, "type"); + json_cstr(out, "text"); + json_key(out, "text"); + json_str(out, embedded, sizeof(embedded)); + json_obj_end(out); + json_arr_end(out); + json_key(out, "encrypted_content"); + json_cstr(out, "cipher_many"); + if (legacy_status) { + json_key(out, "status"); + json_cstr(out, "completed"); + } + json_obj_end(out); +} + +static int +request_has_many_embedded_nul_parts(const struct buf *request, + const char *want_id) +{ + struct json json; + char *id = NULL, *text = NULL, *type = NULL; + size_t len; + int root, input, item, summary, content, part, token, i, n, ok = 0; + + if (json_parse(&json, request->data, request->len, 0) == -1) + return (0); + root = json_root(&json); + input = json_obj_get(&json, root, "input"); + if (!json_is_array(&json, input) || + (n = json_arr_len(&json, input)) < 0) + goto done; + item = -1; + for (i = 0; i < n; i++) { + int candidate = json_arr_get(&json, input, i); + + token = json_obj_get(&json, candidate, "type"); + type = json_get_str(&json, token, &len); + if (type != NULL && len == sizeof("reasoning") - 1 && + memcmp(type, "reasoning", len) == 0) { + item = candidate; + free(type); + type = NULL; + break; + } + free(type); + type = NULL; + } + if (item == -1 || json_obj_get(&json, item, "status") != -1 || + (token = json_obj_get(&json, item, "id")) == -1 || + (id = json_get_str(&json, token, &len)) == NULL || + strlen(want_id) != len || memcmp(id, want_id, len) != 0) + goto done; + summary = json_obj_get(&json, item, "summary"); + content = json_obj_get(&json, item, "content"); + if (!json_is_array(&json, summary) || + json_arr_len(&json, summary) != CODEX_SUB_OUTPUT_MAX + 1 || + !json_is_array(&json, content) || json_arr_len(&json, content) != 1) + goto done; + part = json_arr_get(&json, summary, 0); + token = json_obj_get(&json, part, "text"); + text = json_get_str(&json, token, &len); + if (text == NULL || len != 3 || memcmp(text, "a\0b", 3) != 0) + goto done; + free(text); + text = NULL; + part = json_arr_get(&json, content, 0); + token = json_obj_get(&json, part, "text"); + text = json_get_str(&json, token, &len); + if (text == NULL || len != 3 || memcmp(text, "a\0b", 3) != 0) + goto done; + ok = 1; + +done: + free(type); + free(text); + free(id); + json_done(&json); + return (ok); +} + +static void +test_legacy_reasoning_text_is_length_aware_and_parts_are_not_output_items(void) +{ + struct codex_sub_req request; + struct msglist conv; + struct json_out json_out; + struct msg *message; + struct buf state_data, out; + + /* Existing v1 Journals may contain length-bearing text and more nested + * parts than there are top-level response output items. */ + buf_init(&state_data); + json_out_init(&json_out, &state_data); + json_obj_begin(&json_out); + json_key(&json_out, "reasoning"); + json_arr_begin(&json_out); + json_obj_begin(&json_out); + json_key(&json_out, "index"); + json_num(&json_out, 0); + json_key(&json_out, "item"); + emit_reasoning_with_many_parts(&json_out, "rs_legacy_many", 1); + json_obj_end(&json_out); + json_arr_end(&json_out); + json_key(&json_out, "calls"); + json_arr_begin(&json_out); + json_arr_end(&json_out); + json_obj_end(&json_out); + TAILQ_INIT(&conv); + message = text_message(ROLE_ASSISTANT, "answer"); + message->provider_state = provider_state_new(PROVIDER_CODEX, + "chatgpt-codex-2026-07-17", "gpt-5.5", "responses-continuity", + state_data.data, state_data.len); + append(&conv, message); + request_defaults(&request, &conv); + buf_init(&out); + CHECK(codex_sub_build_request(&out, &request) == 0 && + request_has_many_embedded_nul_parts(&out, "rs_legacy_many")); + buf_free(&out); + msglist_free(&conv); + buf_free(&state_data); +} + +static void test_empty_instructions_and_partial_state_preserve_call_order(void) { static const char partial_state[] = @@ -572,6 +842,85 @@ capture_event(const struct astream_event *event, void } } +static void +test_stream_reasoning_text_is_length_aware(void) +{ + struct codex_sub_stream *stream; + struct provider_state *state = NULL; + struct codex_sub_req request; + struct msglist conv; + struct capture capture; + struct json_out json_out; + struct msg *message; + struct buf fixture, out; + + /* Response text is retained in v2 state and reconstructed without + * C-string truncation on the following request. */ + buf_init(&fixture); + buf_addstr(&fixture, "event: response.created\ndata: {\"type\":" + "\"response.created\",\"response\":{\"id\":\"resp_many\"}}\n\n" + "event: response.completed\ndata: "); + json_out_init(&json_out, &fixture); + json_obj_begin(&json_out); + json_key(&json_out, "type"); + json_cstr(&json_out, "response.completed"); + json_key(&json_out, "response"); + json_obj_begin(&json_out); + json_key(&json_out, "id"); + json_cstr(&json_out, "resp_many"); + json_key(&json_out, "status"); + json_cstr(&json_out, "completed"); + json_key(&json_out, "output"); + json_arr_begin(&json_out); + emit_reasoning_with_many_parts(&json_out, "rs_stream_many", 0); + json_obj_begin(&json_out); + json_key(&json_out, "type"); + json_cstr(&json_out, "message"); + json_key(&json_out, "id"); + json_cstr(&json_out, "msg_many"); + json_key(&json_out, "role"); + json_cstr(&json_out, "assistant"); + json_key(&json_out, "content"); + json_arr_begin(&json_out); + json_obj_begin(&json_out); + json_key(&json_out, "type"); + json_cstr(&json_out, "output_text"); + json_key(&json_out, "text"); + json_cstr(&json_out, "ok"); + json_obj_end(&json_out); + json_arr_end(&json_out); + json_key(&json_out, "status"); + json_cstr(&json_out, "completed"); + json_obj_end(&json_out); + json_arr_end(&json_out); + json_key(&json_out, "usage"); + json_null(&json_out); + json_obj_end(&json_out); + json_obj_end(&json_out); + buf_addstr(&fixture, "\n\n"); + capture_init(&capture); + stream = codex_sub_stream_new("gpt-5.5", capture_event, &capture); + CHECK(stream != NULL && codex_sub_stream_feed(stream, fixture.data, + fixture.len) == 0 && capture.nstate == 1 && + provider_state_decode(capture.state.data, capture.state.len, + &state) == 0 && state != NULL); + TAILQ_INIT(&conv); + message = text_message(ROLE_ASSISTANT, "ok"); + message->provider_state = state; + state = NULL; + append(&conv, message); + request_defaults(&request, &conv); + buf_init(&out); + CHECK(codex_sub_build_request(&out, &request) == 0 && + request_has_many_embedded_nul_parts(&out, "rs_stream_many")); + buf_free(&out); + msglist_free(&conv); + provider_state_free(state); + codex_sub_stream_free(stream); + capture_free(&capture); + buf_free(&fixture); +} + static const char complete_stream[] = "event: response.created\n" "data: {\"type\":\"response.created\",\"response\":{\"id\":\"resp_1\"}}\n\n" @@ -648,24 +997,33 @@ capture_is_complete(const struct capture *capture) } static void +check_state_data(const struct capture *capture, const char *want, size_t wantlen) +{ + struct provider_state *state = NULL; + + CHECK(provider_state_decode(capture->state.data, capture->state.len, + &state) == 0); + CHECK(state != NULL && state->provider_type == PROVIDER_CODEX && + strcmp(state->profile, "chatgpt-codex-2026-07-17") == 0 && + strcmp(state->model, "gpt-5.5") == 0 && + strcmp(state->kind, "responses-continuity") == 0 && + state->data_len == wantlen && memcmp(state->data, want, wantlen) == 0); + provider_state_free(state); +} + +static void check_captured_state(const struct capture *capture) { static const char want[] = - "{\"reasoning\":[{\"index\":0,\"item\":{\"type\":\"reasoning\"," - "\"id\":\"rs_1\",\"summary\":[],\"encrypted_content\":\"cipher\"," - "\"status\":\"completed\"}}],\"calls\":[{\"index\":2,\"call_id\":" - "\"call_1\",\"item_id\":\"fc_1\"}]}"; - struct provider_state *state = NULL; - - CHECK(provider_state_decode(capture->state.data, capture->state.len, - &state) == 0); - CHECK(state != NULL && state->provider_type == PROVIDER_CODEX && - strcmp(state->profile, "chatgpt-codex-2026-07-17") == 0 && - strcmp(state->model, "gpt-5.5") == 0 && - strcmp(state->kind, "responses-continuity") == 0 && - state->data_len == sizeof(want) - 1 && - memcmp(state->data, want, sizeof(want) - 1) == 0); - provider_state_free(state); + "{\"version\":2,\"items\":[{\"index\":0,\"kind\":\"reasoning\"," + "\"item\":{\"type\":\"reasoning\",\"id\":\"rs_1\",\"summary\":[]," + "\"encrypted_content\":\"cipher\"}},{\"index\":1,\"kind\":\"message\"," + "\"item_id\":\"msg_1\",\"phase\":\"commentary\",\"text_offset\":0," + "\"text_len\":5,\"text_sha256\":" + "\"2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824\"}," + "{\"index\":2,\"kind\":\"call\",\"call_id\":\"call_1\"," + "\"item_id\":\"fc_1\"}]}"; + check_state_data(capture, want, sizeof(want) - 1); } static void @@ -757,6 +1115,11 @@ static const char live_message_stream[] = static void test_live_message_stream_shape(void) { + static const char want[] = + "{\"version\":2,\"items\":[{\"index\":0,\"kind\":\"message\"," + "\"item_id\":\"msg_live\",\"phase\":\"final_answer\"," + "\"text_offset\":0,\"text_len\":5,\"text_sha256\":" + "\"2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824\"}]}"; struct codex_sub_stream *stream; struct capture capture; @@ -767,8 +1130,10 @@ test_live_message_stream_shape(void) codex_sub_stream_error(stream) == NULL && capture.text.len == 5 && memcmp(capture.text.data, "hello", 5) == 0 && capture.nusage == 1 && capture.input_tokens == 12 && capture.output_tokens == 3 && - capture.nstate == 0 && capture.ndone == 1 && capture.nerror == 0 && + capture.nstate == 1 && capture.ndone == 1 && capture.nerror == 0 && strcmp(capture.stop, "stop") == 0); + if (capture.nstate == 1) + check_state_data(&capture, want, sizeof(want) - 1); codex_sub_stream_free(stream); capture_free(&capture); } @@ -839,21 +1204,15 @@ static void check_live_reasoning_metadata_state(const struct capture *capture) { static const char want[] = - "{\"reasoning\":[{\"index\":0,\"item\":{\"type\":\"reasoning\"," - "\"id\":\"rs_reasoning_meta\",\"summary\":[],\"content\":[]," - "\"encrypted_content\":\"cipher_done\",\"status\":\"completed\"}}]," - "\"calls\":[]}"; - struct provider_state *state = NULL; + "{\"version\":2,\"items\":[{\"index\":0,\"kind\":\"reasoning\"," + "\"item\":{\"type\":\"reasoning\",\"id\":\"rs_reasoning_meta\"," + "\"summary\":[],\"content\":[],\"encrypted_content\":\"cipher_done\"}}," + "{\"index\":1,\"kind\":\"message\",\"item_id\":" + "\"msg_reasoning_meta\",\"phase\":\"final_answer\",\"text_offset\":0," + "\"text_len\":5,\"text_sha256\":" + "\"2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824\"}]}"; - CHECK(provider_state_decode(capture->state.data, capture->state.len, - &state) == 0); - CHECK(state != NULL && state->provider_type == PROVIDER_CODEX && - strcmp(state->profile, "chatgpt-codex-2026-07-17") == 0 && - strcmp(state->model, "gpt-5.5") == 0 && - strcmp(state->kind, "responses-continuity") == 0 && - state->data_len == sizeof(want) - 1 && - memcmp(state->data, want, sizeof(want) - 1) == 0); - provider_state_free(state); + check_state_data(capture, want, sizeof(want) - 1); } static void @@ -899,6 +1258,14 @@ static const char terminal_statusless_reasoning_stream static void test_terminal_statusless_reasoning_shape(void) { + static const char want[] = + "{\"version\":2,\"items\":[{\"index\":0,\"kind\":\"reasoning\"," + "\"item\":{\"type\":\"reasoning\",\"id\":\"rs_reasoning_meta\"," + "\"summary\":[],\"content\":[],\"encrypted_content\":\"cipher_done\"}}," + "{\"index\":1,\"kind\":\"message\",\"item_id\":" + "\"msg_terminal_reasoning\",\"text_offset\":0,\"text_len\":2," + "\"text_sha256\":" + "\"2689367b205c16ce32ed4200942b8b8b1e262dfc70d9bc9fbc77c49699a4f1df\"}]}"; struct codex_sub_stream *stream; struct capture capture; @@ -916,11 +1283,127 @@ test_terminal_statusless_reasoning_shape(void) capture.events[2] == ASTREAM_DONE && strcmp(capture.stop, "stop") == 0); if (capture.nstate == 1) - check_live_reasoning_metadata_state(&capture); + check_state_data(&capture, want, sizeof(want) - 1); codex_sub_stream_free(stream); capture_free(&capture); } +static void +test_statusless_stream_builds_statusless_followup(void) +{ + static const char want_input[] = + "\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{" + "\"type\":\"input_text\",\"text\":\"test\"}]},{\"type\":\"reasoning\"," + "\"id\":\"rs_reasoning_meta\",\"summary\":[],\"content\":[]," + "\"encrypted_content\":\"cipher_done\"},{\"type\":\"message\"," + "\"id\":\"msg_terminal_reasoning\",\"role\":\"assistant\"," + "\"content\":[{\"type\":\"output_text\",\"text\":\"ok\"}]},{" + "\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":" + "\"input_text\",\"text\":\"what model are you?\"}]}]"; + struct codex_sub_stream *stream; + struct provider_state *state = NULL; + struct codex_sub_req request; + struct msglist conv; + struct capture capture; + struct msg *message; + struct buf out; + + capture_init(&capture); + stream = codex_sub_stream_new("gpt-5.5", capture_event, &capture); + CHECK(stream != NULL && codex_sub_stream_feed(stream, + terminal_statusless_reasoning_stream, + sizeof(terminal_statusless_reasoning_stream) - 1) == 0 && + capture.nstate == 1 && provider_state_decode(capture.state.data, + capture.state.len, &state) == 0 && state != NULL); + TAILQ_INIT(&conv); + append(&conv, text_message(ROLE_USER, "test")); + message = text_message(ROLE_ASSISTANT, "ok"); + message->provider_state = state; + state = NULL; + append(&conv, message); + append(&conv, text_message(ROLE_USER, "what model are you?")); + request_defaults(&request, &conv); + buf_init(&out); + CHECK(codex_sub_build_request(&out, &request) == 0 && + memmem(out.data, out.len, want_input, sizeof(want_input) - 1) != NULL && + memmem(out.data, out.len, "\"status\"", 8) == NULL); + buf_free(&out); + msglist_free(&conv); + provider_state_free(state); + codex_sub_stream_free(stream); + capture_free(&capture); +} + +static const char interleaved_phase_stream[] = + "event: response.created\n" + "data: {\"type\":\"response.created\",\"response\":{\"id\":" + "\"resp_phases\"}}\n\n" + "event: response.completed\n" + "data: {\"type\":\"response.completed\",\"response\":{\"id\":" + "\"resp_phases\",\"status\":\"completed\",\"output\":[{\"type\":" + "\"reasoning\",\"id\":\"rs_phases\",\"summary\":[]," + "\"encrypted_content\":\"cipher_phases\"},{\"type\":\"message\"," + "\"id\":\"msg_one\",\"role\":\"assistant\",\"content\":[{\"type\":" + "\"output_text\",\"text\":\"one\"}],\"status\":\"completed\"," + "\"phase\":\"commentary\"},{\"type\":\"function_call\",\"id\":" + "\"fc_a\",\"name\":\"read\",\"arguments\":\"{}\",\"call_id\":" + "\"call_a\",\"status\":\"completed\"},{\"type\":\"message\",\"id\":" + "\"msg_two\",\"role\":\"assistant\",\"content\":[{\"type\":" + "\"output_text\",\"text\":\"two\"}],\"status\":\"completed\"," + "\"phase\":\"final_answer\"},{\"type\":\"function_call\",\"id\":" + "\"fc_b\",\"name\":\"read\",\"arguments\":\"{}\",\"call_id\":" + "\"call_b\",\"status\":\"completed\"}],\"usage\":null}}\n\n"; + +static void +test_interleaved_phase_and_call_order_round_trip(void) +{ + static const char want[] = + "{\"type\":\"reasoning\",\"id\":\"rs_phases\",\"summary\":[]," + "\"encrypted_content\":\"cipher_phases\"},{\"type\":\"message\"," + "\"id\":\"msg_one\",\"role\":\"assistant\",\"content\":[{\"type\":" + "\"output_text\",\"text\":\"one\"}],\"phase\":\"commentary\"},{" + "\"type\":\"function_call\",\"id\":\"fc_a\",\"name\":\"read\"," + "\"arguments\":\"{}\",\"call_id\":\"call_a\"},{\"type\":\"message\"," + "\"id\":\"msg_two\",\"role\":\"assistant\",\"content\":[{\"type\":" + "\"output_text\",\"text\":\"two\"}],\"phase\":\"final_answer\"},{" + "\"type\":\"function_call\",\"id\":\"fc_b\",\"name\":\"read\"," + "\"arguments\":\"{}\",\"call_id\":\"call_b\"}"; + struct codex_sub_stream *stream; + struct provider_state *state = NULL; + struct codex_sub_req request; + struct msglist conv; + struct capture capture; + struct msg *message; + struct buf out; + + capture_init(&capture); + stream = codex_sub_stream_new("gpt-5.5", capture_event, &capture); + CHECK(stream != NULL && codex_sub_stream_feed(stream, + interleaved_phase_stream, sizeof(interleaved_phase_stream) - 1) == 0 && + capture.text.len == 6 && memcmp(capture.text.data, "onetwo", 6) == 0 && + capture.ncalls == 2 && capture.calls[0].index == 2 && + capture.calls[1].index == 4 && capture.nstate == 1 && + provider_state_decode(capture.state.data, capture.state.len, &state) == 0); + TAILQ_INIT(&conv); + message = text_message(ROLE_ASSISTANT, "onetwo"); + msg_add_tool_use(message, "call_a", "read", "{}", 2); + msg_add_tool_use(message, "call_b", "read", "{}", 2); + message->provider_state = state; + state = NULL; + append(&conv, message); + append(&conv, text_message(ROLE_USER, "continue")); + request_defaults(&request, &conv); + buf_init(&out); + CHECK(codex_sub_build_request(&out, &request) == 0 && + memmem(out.data, out.len, want, sizeof(want) - 1) != NULL && + memmem(out.data, out.len, "\"status\"", 8) == NULL); + buf_free(&out); + msglist_free(&conv); + provider_state_free(state); + codex_sub_stream_free(stream); + capture_free(&capture); +} + static const char terminal_text_stream[] = "event: response.created\n" "data: {\"type\":\"response.created\",\"response\":{\"id\":\"resp_2\"}}\n\n" @@ -2097,12 +2580,18 @@ main(void) test_exact_compaction_history_request(); test_mixed_history_and_matching_continuity(); test_mismatch_omission_and_fail_closed_validation(); - test_reasoning_state_requires_completed_status(); + test_legacy_reasoning_state_requires_completed_status(); + test_v2_continuity_rejects_corruption_and_unknown_fields(); + test_v2_reasoning_content_translation(); + test_legacy_reasoning_text_is_length_aware_and_parts_are_not_output_items(); test_empty_instructions_and_partial_state_preserve_call_order(); + test_stream_reasoning_text_is_length_aware(); test_complete_stream_and_every_split(); test_live_message_stream_shape(); test_live_reasoning_metadata_stream_shape(); test_terminal_statusless_reasoning_shape(); + test_statusless_stream_builds_statusless_followup(); + test_interleaved_phase_and_call_order_round_trip(); test_call_only_stream(); test_malformed_event_bytes(); test_reasoning_and_metadata_guards(); blob - b68f60e8f648a5db2999001ef2c05cf4b51dbd4b blob + cc2d7aa72ce1c8f00f015c49740dddb28ddd5afd --- regress/journal/Makefile +++ regress/journal/Makefile @@ -1,6 +1,7 @@ PROG= journal_test SRCS= journal_test.c journal.c system_epoch.c compaction.c msg.c \ - codex_sub_req.c codex_sub_profile.c provider_state.c json.c \ + codex_sub_req.c codex_sub_profile.c codex_sub_reasoning.c \ + provider_state.c json.c \ buf.c utf8.c log.c xmalloc.c .PATH: ${.CURDIR}/../../src/fugu blob - ec864a3123dbff7336905ecbdae77c205e6a3c04 blob + 3cb750c6b750f21b81acf2b066104ac5d08a2dd3 --- regress/journal/journal_test.c +++ regress/journal/journal_test.c @@ -47,6 +47,7 @@ #include "xmalloc.h" #include "json.h" #include "msg.h" +#include "codex_sub.h" #include "provider_state.h" #include "compaction.h" #include "system_epoch.h" @@ -800,6 +801,8 @@ test_provider_state_is_transactional_and_escaped(void) struct msglist conv; struct msg *m; struct provider_state *state; + struct codex_sub_req request; + struct buf request_body; char id[SESSION_ID_MAX], path[1024], *raw; size_t rawlen; @@ -846,6 +849,21 @@ test_provider_state_is_transactional_and_escaped(void) "responses-continuity") && m->provider_state->data_len == sizeof(data) - 1 && memcmp(m->provider_state->data, data, sizeof(data) - 1) == 0); + + /* Legacy state may be read, but its response-only status is not sent + * back in a new provider request. */ + memset(&request, 0, sizeof(request)); + request.model = codex_sub_profile()->model; + request.max_tokens = 4096; + request.system = ""; + request.conv = &conv; + request.session_id = "journal-replay"; + buf_init(&request_body); + CHECK(codex_sub_build_request(&request_body, &request) == 0); + CHECK(strstr(buf_cstr(&request_body), "\"type\":\"reasoning\"") != + NULL); + CHECK(strstr(buf_cstr(&request_body), "\"status\"") == NULL); + buf_free(&request_body); msglist_free(&conv); } @@ -887,12 +905,14 @@ test_malformed_provider_state_discards_the_turn(void) } static void -check_semantically_invalid_codex_state_is_discarded(const char *profile, +check_mismatched_codex_state_is_retained(const char *profile, const char *model, const char *kind, const char *data) { struct journal *j; struct msglist conv; struct msg *m; + struct codex_sub_req request; + struct buf request_body; char id[SESSION_ID_MAX]; j = journal_create(dir, id, sizeof(id)); @@ -911,8 +931,28 @@ check_semantically_invalid_codex_state_is_discarded(co journal_close(j); TAILQ_INIT(&conv); - CHECK(journal_replay(dir, id, &conv) == 0 && count(&conv) == 2 && - strcmp(text_of(&conv, 0), "clean") == 0); + CHECK(journal_replay(dir, id, &conv) == 0 && count(&conv) == 3); + m = TAILQ_FIRST(&conv); + CHECK(m != NULL && m->role == ROLE_ASSISTANT && + m->provider_state != NULL && + m->provider_state->provider_type == 3 && + strcmp(m->provider_state->profile, profile) == 0 && + strcmp(m->provider_state->model, model) == 0 && + strcmp(m->provider_state->kind, kind) == 0 && + m->provider_state->data_len == strlen(data) && + memcmp(m->provider_state->data, data, strlen(data)) == 0); + CHECK(strcmp(text_of(&conv, 1), "clean") == 0); + + memset(&request, 0, sizeof(request)); + request.model = codex_sub_profile()->model; + request.max_tokens = 4096; + request.system = ""; + request.conv = &conv; + request.session_id = "journal-mismatch"; + buf_init(&request_body); + CHECK(codex_sub_build_request(&request_body, &request) == 0); + CHECK(strstr(buf_cstr(&request_body), "\"id\":\"rs_1\"") == NULL); + buf_free(&request_body); msglist_free(&conv); } @@ -968,19 +1008,84 @@ test_codex_provider_state_is_validated_during_replay(v "{\"reasoning\":[{\"index\":0,\"item\":{\"type\":\"reasoning\"," "\"id\":\"rs_progress\",\"summary\":[],\"encrypted_content\":" "\"opaque\",\"status\":\"in_progress\"}}],\"calls\":[]}"; + static const char bad_v2_digest[] = + "{\"version\":2,\"items\":[{\"index\":0,\"kind\":\"message\"," + "\"item_id\":\"msg_bad\",\"phase\":\"final_answer\"," + "\"text_offset\":0,\"text_len\":13,\"text_sha256\":" + "\"0000000000000000000000000000000000000000000000000000000000000000\"}]}"; - check_semantically_invalid_codex_state_is_discarded("old-profile", + /* State for another compatibility profile is inert history. Its + * current schema is deliberately not interpreted during replay. */ + check_mismatched_codex_state_is_retained("old-profile", "gpt-5.5", "responses-continuity", valid_data); - check_semantically_invalid_codex_state_is_discarded( + check_mismatched_codex_state_is_retained( + "chatgpt-codex-2026-07-17", "gpt-old", "responses-continuity", + valid_data); + check_mismatched_codex_state_is_retained( "chatgpt-codex-2026-07-17", "gpt-5.5", "wrong-kind", valid_data); - check_semantically_invalid_codex_state_is_discarded( - "chatgpt-codex-2026-07-17", "gpt-5.5", "responses-continuity", - duplicate_call_data); + check_invalid_codex_reasoning_state_is_discarded(duplicate_call_data); + check_invalid_codex_reasoning_state_is_discarded(bad_v2_digest); check_invalid_codex_reasoning_state_is_discarded(missing_status); check_invalid_codex_reasoning_state_is_discarded(null_status); check_invalid_codex_reasoning_state_is_discarded(in_progress_status); } +static void +test_v2_codex_provider_state_round_trip(void) +{ + static const char data[] = + "{\"version\":2,\"items\":[{\"index\":0,\"kind\":\"reasoning\"," + "\"item\":{\"type\":\"reasoning\",\"id\":\"rs_v2\"," + "\"summary\":[],\"encrypted_content\":\"opaque\"}},{\"index\":1," + "\"kind\":\"message\",\"item_id\":\"msg_v2\",\"phase\":" + "\"final_answer\",\"text_offset\":0,\"text_len\":6," + "\"text_sha256\":" + "\"0db52f4076c082518412afd3dd3576e2cb0c63703fd7fed5e23ade60efef31d9\"}]}"; + struct journal *j; + struct msglist conv; + struct msg *m; + struct codex_sub_req request; + struct buf request_body; + char id[SESSION_ID_MAX]; + + j = journal_create(dir, id, sizeof(id)); + CHECK(j != NULL); + journal_turn_begin(j, 1); + m = msg_new(ROLE_ASSISTANT); + msg_add_text(m, "answer", 6); + m->provider_state = provider_state_new(3, + "chatgpt-codex-2026-07-17", "gpt-5.5", + "responses-continuity", data, sizeof(data) - 1); + CHECK(m->provider_state != NULL); + journal_message(j, m); + msg_free(m); + journal_turn_end(j, 1, TURN_OK); + journal_close(j); + + TAILQ_INIT(&conv); + CHECK(journal_replay(dir, id, &conv) == 0 && count(&conv) == 1); + m = TAILQ_FIRST(&conv); + CHECK(m != NULL && m->provider_state != NULL && + m->provider_state->data_len == sizeof(data) - 1 && + memcmp(m->provider_state->data, data, sizeof(data) - 1) == 0); + + memset(&request, 0, sizeof(request)); + request.model = codex_sub_profile()->model; + request.max_tokens = 4096; + request.system = ""; + request.conv = &conv; + request.session_id = "journal-v2"; + buf_init(&request_body); + CHECK(codex_sub_build_request(&request_body, &request) == 0); + CHECK(strstr(buf_cstr(&request_body), + "\"type\":\"message\",\"id\":\"msg_v2\",\"role\":" + "\"assistant\",\"content\":[{\"type\":\"output_text\"," + "\"text\":\"answer\"}],\"phase\":\"final_answer\"") != NULL); + CHECK(strstr(buf_cstr(&request_body), "\"status\"") == NULL); + buf_free(&request_body); + msglist_free(&conv); +} + static int valid_records(const char *path, int want) { @@ -2105,6 +2210,7 @@ main(void) test_provider_state_is_transactional_and_escaped(); test_malformed_provider_state_discards_the_turn(); test_codex_provider_state_is_validated_during_replay(); + test_v2_codex_provider_state_round_trip(); test_hostile_tool_input(); test_legacy_raw_tool_input(); test_invalid_encoded_tool_input(); blob - fb3b79cfdaf3bdbfe137515bde8c7638bb185cac blob + 2b1b4464d92840ba07c4ada278581a8bb241b1d5 --- regress/print/Makefile +++ regress/print/Makefile @@ -1,10 +1,13 @@ PROG= stub -SRCS= stub.c +SRCS= stub.c json.c buf.c log.c xmalloc.c NOMAN= yes +CFLAGS+= -I${.CURDIR}/../../src/common LDADD= -ltls DPADD= ${LIBTLS} +.PATH: ${.CURDIR}/../../src/common + REGRESS_TARGETS= run-print CLEANFILES+= jsoncheck pglaunch blob - 373d92c0db5340b7cddd0c43bfeb52e343512679 blob + 907736476986770104868bfe56b76a965c7e5a51 --- regress/print/run.sh +++ regress/print/run.sh @@ -539,6 +539,12 @@ grep -q '"type":"reasoning","id":"rs_codex_1"' \ "$dir/codex-request-2" && grep -q 'CODEX-TOOL-RESULT' "$dir/codex-request-2" ok "codex subscription: canonical call ids and opaque reasoning continue" $? +codex_reasoning='{"type":"reasoning","id":"rs_codex_1","summary":[],"encrypted_content":"cipher_1"}' +for request in 2 3; do + grep -Fq "$codex_reasoning" "$dir/codex-request-body-$request" && + ! grep -q '"status"' "$dir/codex-request-body-$request" + ok "codex subscription: request $request replays only request-safe reasoning" $? +done grep -q '"type":"reasoning","id":"rs_codex_1"' \ "$dir/codex-request-3" && grep -q 'CODEX-TOOL-RESULT' "$dir/codex-request-3" && blob - 88d2cd5d53b3a1cf2da9c2df869944a4ae1819e8 blob + 0939ac578ad192b55d45fe3d5d72f64eafb1eb99 --- regress/print/stub.c +++ regress/print/stub.c @@ -22,12 +22,16 @@ #include #include #include +#include #include #include #include +#include #include #include +#include "json.h" + #define REQUEST_MAX (2 * 1024 * 1024) static ssize_t read_request(struct tls *, char *, size_t); @@ -35,6 +39,7 @@ static void record_request(const char *, const char *, static void record_request_log(const char *, const char *, size_t, const char *); static void record_count(const char *, int); +static int validate_codex_request(const char *, size_t, char *, size_t); static const char http_ok[] = "HTTP/1.1 200 OK\r\n" @@ -525,6 +530,44 @@ send_codex_http_error(struct tls *conn, int status) } static void +send_codex_schema_error(struct tls *conn, const char *parameter) +{ + struct buf body; + struct json_out out; + char *message; + char headers[256]; + int n; + + if (asprintf(&message, "Unknown parameter: '%s'.", parameter) == -1) + err(1, "asprintf Codex schema error"); + buf_init(&body); + json_out_init(&out, &body); + json_obj_begin(&out); + json_key(&out, "error"); + json_obj_begin(&out); + json_key(&out, "message"); + json_cstr(&out, message); + json_key(&out, "type"); + json_cstr(&out, "invalid_request_error"); + json_key(&out, "param"); + json_cstr(&out, parameter); + json_key(&out, "code"); + json_cstr(&out, "unknown_parameter"); + json_obj_end(&out); + json_obj_end(&out); + free(message); + n = snprintf(headers, sizeof(headers), + "HTTP/1.1 400 Bad Request\r\n" + "content-type: application/json\r\n" + "content-length: %zu\r\n\r\n", body.len); + if (n < 0 || (size_t)n >= sizeof(headers)) + errx(1, "Codex schema-error header too large"); + (void)tls_write_all(conn, headers, (size_t)n); + (void)tls_write_all(conn, (const char *)body.data, body.len); + buf_free(&body); +} + +static void send_sleep_tool(struct tls *conn, const char *name, const char *pidfile, int number) { @@ -558,13 +601,85 @@ send_sleep_tool(struct tls *conn, const char *name, co send_sse(conn, sse); } -/* Headers plus Content-Length bytes, bounded as hostile input. */ +static int +request_header_end(const char *request, size_t request_len, size_t *endp) +{ + size_t i; + + for (i = 0; i + 3 < request_len; i++) + if (memcmp(request + i, "\r\n\r\n", 4) == 0) { + *endp = i; + return (0); + } + return (-1); +} + +/* Parse one, and only one, syntactically valid Content-Length header. */ +static int +request_content_length(const char *request, size_t headers_end, size_t *lenp) +{ + size_t colon, i, line_end, line_start, value; + int seen = 0; + + for (i = 0; i + 1 < headers_end; i++) + if (request[i] == '\r' && request[i + 1] == '\n') + break; + if (i + 1 >= headers_end || i == 0) + return (-1); + line_start = i + 2; + while (line_start < headers_end) { + for (line_end = line_start; line_end <= headers_end; line_end++) + if (line_end == headers_end || (request[line_end] == '\r' && + line_end + 1 < headers_end && + request[line_end + 1] == '\n')) + break; + if (line_end == line_start) + return (-1); + for (colon = line_start; colon < line_end; colon++) + if (request[colon] == ':') + break; + if (colon == line_end) + return (-1); + if (colon - line_start == sizeof("Content-Length") - 1 && + strncasecmp(request + line_start, "Content-Length", + sizeof("Content-Length") - 1) == 0) { + if (seen) + return (-1); + seen = 1; + i = colon + 1; + while (i < line_end && + (request[i] == ' ' || request[i] == '\t')) + i++; + if (i == line_end || request[i] < '0' || request[i] > '9') + return (-1); + value = 0; + while (i < line_end && request[i] >= '0' && + request[i] <= '9') { + size_t digit = (size_t)(request[i] - '0'); + + if (value > (SIZE_MAX - digit) / 10) + return (-1); + value = value * 10 + digit; + i++; + } + while (i < line_end && + (request[i] == ' ' || request[i] == '\t')) + i++; + if (i != line_end) + return (-1); + *lenp = value; + } + line_start = line_end + 2; + } + return (seen ? 0 : -1); +} + +/* Headers plus exactly Content-Length bytes, bounded as hostile input. */ static ssize_t read_request(struct tls *tls, char *buf, size_t cap) { - size_t have = 0, want = 0; - char *e, *cl; - ssize_t n; + size_t body_offset, content_len, have = 0, headers_end, want = 0; + ssize_t n; for (;;) { if (have >= cap - 1) @@ -576,21 +691,464 @@ read_request(struct tls *tls, char *buf, size_t cap) return (-1); have += (size_t)n; buf[have] = '\0'; - if (want == 0 && (e = strstr(buf, "\r\n\r\n")) != NULL) { - size_t clen = 0; - - if ((cl = strcasestr(buf, "content-length:")) != NULL) - clen = (size_t)strtoll(cl + 15, NULL, 10); - if (clen > cap - (size_t)(e - buf) - 5) + if (want == 0 && request_header_end(buf, have, &headers_end) == 0) { + body_offset = headers_end + 4; + if (request_content_length(buf, headers_end, &content_len) == -1 || + body_offset >= cap || content_len > cap - body_offset - 1) return (-1); - want = (size_t)(e - buf) + 4 + clen; + want = body_offset + content_len; } if (want != 0 && have >= want) - return ((ssize_t)have); + return ((ssize_t)want); } } +static int +request_body(const char *request, size_t request_len, const char **body, + size_t *body_len) +{ + size_t content_len, headers_end, offset; + + if (request_header_end(request, request_len, &headers_end) == -1 || + request_content_length(request, headers_end, &content_len) == -1) + return (-1); + offset = headers_end + 4; + if (content_len == 0 || offset > request_len || + content_len > request_len - offset) + return (-1); + *body = request + offset; + *body_len = content_len; + return (0); +} + static void +parameter_member(char *parameter, size_t parametersz, const char *path, + const char *member) +{ + (void)snprintf(parameter, parametersz, "%s.%s", path, member); +} + +static int +strict_object_fields(const struct json *json, int object, + const char *const *names, int nname, int *fields, const char *path, + char *parameter, size_t parametersz) +{ + char *key; + size_t keylen; + int found, i, k, n, token; + + if (!json_is_object(json, object) || + (n = json_obj_len(json, object)) < 0) { + strlcpy(parameter, path, parametersz); + return (-1); + } + for (i = 0; i < nname; i++) + fields[i] = -1; + for (i = 0; i < n; i++) { + if (json_obj_item(json, object, i, &key, &keylen, &token) == -1) { + strlcpy(parameter, path, parametersz); + return (-1); + } + found = -1; + for (k = 0; k < nname; k++) + if (strlen(names[k]) == keylen && + memcmp(names[k], key, keylen) == 0) { + found = k; + break; + } + if (found == -1) { + (void)snprintf(parameter, parametersz, "%s.%.*s", path, + (int)keylen, key); + free(key); + return (-1); + } + free(key); + if (fields[found] != -1) { + parameter_member(parameter, parametersz, path, names[found]); + return (-1); + } + fields[found] = token; + } + return (0); +} + +static int +copy_json_string(const struct json *json, int token, char *value, + size_t valuesz) +{ + char *decoded; + size_t len; + + if (token < 0 || (decoded = json_get_str(json, token, &len)) == NULL) + return (-1); + if (len == 0 || len >= valuesz || strlen(decoded) != len || + memchr(decoded, '\r', len) != NULL || + memchr(decoded, '\n', len) != NULL) { + free(decoded); + return (-1); + } + memcpy(value, decoded, len + 1); + free(decoded); + return (0); +} + +static int +json_string_valid(const struct json *json, int token, int empty_ok, int nul_ok) +{ + char *decoded; + size_t len; + + if (token < 0 || (decoded = json_get_str(json, token, &len)) == NULL) + return (-1); + if ((!empty_ok && len == 0) || (!nul_ok && strlen(decoded) != len)) { + free(decoded); + return (-1); + } + free(decoded); + return (0); +} + +static int +typed_text_parts_valid(const struct json *json, int array, const char *kind, + int min_parts, int nul_ok, int input_index, const char *member, + char *parameter, size_t parametersz) +{ + static const char *const names[] = { "type", "text" }; + char path[128], type[32]; + int fields[2], i, n, object; + + (void)snprintf(path, sizeof(path), "input[%d].%s", input_index, member); + if (!json_is_array(json, array) || + (n = json_arr_len(json, array)) < min_parts) { + strlcpy(parameter, path, parametersz); + return (-1); + } + for (i = 0; i < n; i++) { + (void)snprintf(path, sizeof(path), "input[%d].%s[%d]", + input_index, member, i); + object = json_arr_get(json, array, i); + if (strict_object_fields(json, object, names, 2, fields, path, + parameter, parametersz) == -1) + return (-1); + if (fields[0] == -1 || copy_json_string(json, fields[0], type, + sizeof(type)) == -1 || strcmp(type, kind) != 0) { + parameter_member(parameter, parametersz, path, "type"); + return (-1); + } + if (fields[1] == -1 || + json_string_valid(json, fields[1], 1, nul_ok) == -1) { + parameter_member(parameter, parametersz, path, "text"); + return (-1); + } + } + return (0); +} + +static int +message_item_valid(const struct json *json, int object, int input_index, + char *parameter, size_t parametersz) +{ + static const char *const names[] = { + "type", "id", "role", "content", "phase" + }; + char id[257], path[64], phase[32], role[32], type[32]; + int fields[5], have_id, have_phase; + + (void)snprintf(path, sizeof(path), "input[%d]", input_index); + if (strict_object_fields(json, object, names, 5, fields, path, + parameter, parametersz) == -1) + return (-1); + if (fields[0] == -1 || copy_json_string(json, fields[0], type, + sizeof(type)) == -1 || strcmp(type, "message") != 0) { + parameter_member(parameter, parametersz, path, "type"); + return (-1); + } + if (fields[2] == -1 || copy_json_string(json, fields[2], role, + sizeof(role)) == -1 || (strcmp(role, "user") != 0 && + strcmp(role, "assistant") != 0)) { + parameter_member(parameter, parametersz, path, "role"); + return (-1); + } + have_id = fields[1] != -1; + if (have_id && (copy_json_string(json, fields[1], id, sizeof(id)) == -1 || + strncmp(id, "msg_", 4) != 0)) { + parameter_member(parameter, parametersz, path, "id"); + return (-1); + } + have_phase = fields[4] != -1; + if (have_phase && (copy_json_string(json, fields[4], phase, + sizeof(phase)) == -1 || (strcmp(phase, "commentary") != 0 && + strcmp(phase, "final_answer") != 0))) { + parameter_member(parameter, parametersz, path, "phase"); + return (-1); + } + if ((strcmp(role, "user") == 0 && (have_id || have_phase)) || + (have_phase && !have_id)) { + parameter_member(parameter, parametersz, path, + have_phase ? "phase" : "id"); + return (-1); + } + if (fields[3] == -1 || typed_text_parts_valid(json, fields[3], + strcmp(role, "user") == 0 ? "input_text" : "output_text", 1, 1, + input_index, "content", parameter, parametersz) == -1) { + if (fields[3] == -1) + parameter_member(parameter, parametersz, path, "content"); + return (-1); + } + return (0); +} + +static int +reasoning_item_valid(const struct json *json, int object, int input_index, + char *parameter, size_t parametersz) +{ + static const char *const names[] = { + "type", "id", "summary", "content", "encrypted_content" + }; + char id[257], path[64], type[32]; + int fields[5]; + + (void)snprintf(path, sizeof(path), "input[%d]", input_index); + if (strict_object_fields(json, object, names, 5, fields, path, + parameter, parametersz) == -1) + return (-1); + if (fields[0] == -1 || copy_json_string(json, fields[0], type, + sizeof(type)) == -1 || strcmp(type, "reasoning") != 0) { + parameter_member(parameter, parametersz, path, "type"); + return (-1); + } + if (fields[1] == -1 || copy_json_string(json, fields[1], id, + sizeof(id)) == -1 || strncmp(id, "rs_", 3) != 0) { + parameter_member(parameter, parametersz, path, "id"); + return (-1); + } + if (fields[2] == -1 || typed_text_parts_valid(json, fields[2], + "summary_text", 0, 1, input_index, "summary", parameter, + parametersz) == -1) { + if (fields[2] == -1) + parameter_member(parameter, parametersz, path, "summary"); + return (-1); + } + if (fields[3] != -1 && typed_text_parts_valid(json, fields[3], "text", + 0, 1, input_index, "content", parameter, parametersz) == -1) + return (-1); + if (fields[4] == -1 || + json_string_valid(json, fields[4], 0, 0) == -1) { + parameter_member(parameter, parametersz, path, "encrypted_content"); + return (-1); + } + return (0); +} + +static int +function_call_item_valid(const struct json *json, int object, int input_index, + char *parameter, size_t parametersz) +{ + static const char *const names[] = { + "type", "id", "name", "arguments", "call_id" + }; + char call_id[257], id[257], name[33], path[64], type[32], *arguments; + size_t arguments_len; + int fields[5]; + + (void)snprintf(path, sizeof(path), "input[%d]", input_index); + if (strict_object_fields(json, object, names, 5, fields, path, + parameter, parametersz) == -1) + return (-1); + if (fields[0] == -1 || copy_json_string(json, fields[0], type, + sizeof(type)) == -1 || strcmp(type, "function_call") != 0) { + parameter_member(parameter, parametersz, path, "type"); + return (-1); + } + if (fields[1] != -1 && (copy_json_string(json, fields[1], id, + sizeof(id)) == -1 || strncmp(id, "fc_", 3) != 0)) { + parameter_member(parameter, parametersz, path, "id"); + return (-1); + } + if (fields[2] == -1 || copy_json_string(json, fields[2], name, + sizeof(name)) == -1) { + parameter_member(parameter, parametersz, path, "name"); + return (-1); + } + if (fields[4] == -1 || copy_json_string(json, fields[4], call_id, + sizeof(call_id)) == -1) { + parameter_member(parameter, parametersz, path, "call_id"); + return (-1); + } + arguments = fields[3] == -1 ? NULL : + json_get_str(json, fields[3], &arguments_len); + if (arguments == NULL || arguments_len == 0 || + strlen(arguments) != arguments_len || + !json_valid_object(arguments, arguments_len)) { + free(arguments); + parameter_member(parameter, parametersz, path, "arguments"); + return (-1); + } + free(arguments); + return (0); +} + +static int +function_output_item_valid(const struct json *json, int object, + int input_index, char *parameter, size_t parametersz) +{ + static const char *const names[] = { "type", "call_id", "output" }; + char call_id[257], path[64], type[32]; + int fields[3]; + + (void)snprintf(path, sizeof(path), "input[%d]", input_index); + if (strict_object_fields(json, object, names, 3, fields, path, + parameter, parametersz) == -1) + return (-1); + if (fields[0] == -1 || copy_json_string(json, fields[0], type, + sizeof(type)) == -1 || strcmp(type, "function_call_output") != 0) { + parameter_member(parameter, parametersz, path, "type"); + return (-1); + } + if (fields[1] == -1 || copy_json_string(json, fields[1], call_id, + sizeof(call_id)) == -1) { + parameter_member(parameter, parametersz, path, "call_id"); + return (-1); + } + if (fields[2] == -1 || + json_string_valid(json, fields[2], 1, 1) == -1) { + parameter_member(parameter, parametersz, path, "output"); + return (-1); + } + return (0); +} + +static int +unique_object_member(const struct json *json, int object, const char *name, + int *tokenp) +{ + char *key; + size_t keylen, namelen = strlen(name); + int i, n, token; + + *tokenp = -1; + if (!json_is_object(json, object) || + (n = json_obj_len(json, object)) < 0) + return (-1); + for (i = 0; i < n; i++) { + if (json_obj_item(json, object, i, &key, &keylen, &token) == -1) + return (-1); + if (keylen == namelen && memcmp(key, name, namelen) == 0) { + free(key); + if (*tokenp != -1) + return (-1); + *tokenp = token; + } else + free(key); + } + return (*tokenp == -1 ? -1 : 0); +} + +/* Independent provider-side oracle for the response/request schema seam. */ +static int +validate_codex_request(const char *request, size_t request_len, + char *parameter, size_t parametersz) +{ + const char *body; + struct json json; + size_t body_len; + char path[64], type[32]; + int input, item, n, i, root, token; + + strlcpy(parameter, "input", parametersz); + if (request_body(request, request_len, &body, &body_len) == -1 || + !json_valid_object(body, body_len) || + json_parse(&json, body, body_len, 0) == -1) + return (-1); + root = json_root(&json); + if (unique_object_member(&json, root, "input", &input) == -1 || + !json_is_array(&json, input) || (n = json_arr_len(&json, input)) < 1) + goto fail; + for (i = 0; i < n; i++) { + (void)snprintf(path, sizeof(path), "input[%d]", i); + item = json_arr_get(&json, input, i); + if (!json_is_object(&json, item) || + (token = json_obj_get(&json, item, "type")) == -1 || + copy_json_string(&json, token, type, sizeof(type)) == -1) { + parameter_member(parameter, parametersz, path, "type"); + goto fail; + } + if (strcmp(type, "message") == 0) { + if (message_item_valid(&json, item, i, parameter, + parametersz) == -1) + goto fail; + } else if (strcmp(type, "reasoning") == 0) { + if (reasoning_item_valid(&json, item, i, parameter, + parametersz) == -1) + goto fail; + } else if (strcmp(type, "function_call") == 0) { + if (function_call_item_valid(&json, item, i, parameter, + parametersz) == -1) + goto fail; + } else if (strcmp(type, "function_call_output") == 0) { + if (function_output_item_valid(&json, item, i, parameter, + parametersz) == -1) + goto fail; + } else { + parameter_member(parameter, parametersz, path, "type"); + goto fail; + } + } + json_done(&json); + return (0); + +fail: + json_done(&json); + return (-1); +} + +static int +validate_codex_body(const char *body, char *parameter, size_t parametersz) +{ + char *request; + int result; + + if (asprintf(&request, "POST / HTTP/1.1\r\nContent-Length: %zu\r\n\r\n%s", + strlen(body), body) == -1) + err(1, "asprintf Codex oracle self-check"); + result = validate_codex_request(request, strlen(request), parameter, + parametersz); + free(request); + return (result); +} + +static void +codex_request_oracle_selfcheck(void) +{ + static const char valid[] = + "{\"input\":[" + "{\"type\":\"message\",\"role\":\"user\",\"content\":[{" + "\"type\":\"input_text\",\"text\":\"question\"}]}," + "{\"type\":\"message\",\"id\":\"msg_1\",\"role\":\"assistant\"," + "\"content\":[{\"type\":\"output_text\",\"text\":\"answer\"}]," + "\"phase\":\"final_answer\"}," + "{\"type\":\"reasoning\",\"id\":\"rs_1\",\"summary\":[{" + "\"type\":\"summary_text\",\"text\":\"brief\"}],\"content\":[{" + "\"type\":\"text\",\"text\":\"detail\"}]," + "\"encrypted_content\":\"cipher\"}," + "{\"type\":\"function_call\",\"id\":\"fc_1\",\"name\":\"shell\"," + "\"arguments\":\"{\\\"command\\\":\\\"true\\\"}\"," + "\"call_id\":\"call_1\"}," + "{\"type\":\"function_call_output\",\"call_id\":\"call_1\"," + "\"output\":\"ok\"}]}"; + static const char status[] = + "{\"input\":[{\"type\":\"reasoning\",\"id\":\"rs_1\"," + "\"summary\":[],\"encrypted_content\":\"cipher\"," + "\"status\":\"completed\"}]}"; + char parameter[512]; + + if (validate_codex_body(valid, parameter, sizeof(parameter)) == -1 || + validate_codex_body(status, parameter, sizeof(parameter)) != -1 || + strcmp(parameter, "input[0].status") != 0) + errx(1, "Codex request oracle self-check failed"); +} + +static void record_request(const char *path, const char *request, size_t len) { FILE *f; @@ -656,6 +1214,7 @@ main(int argc, char *argv[]) argv += optind; if (cert == NULL || key == NULL || argc == 0) errx(1, "usage: stub -c cert -k key scenario ..."); + codex_request_oracle_selfcheck(); reqout = getenv("REQOUT"); reqlog = getenv("REQLOG"); countfile = getenv("COUNTFILE"); @@ -706,6 +1265,15 @@ main(int argc, char *argv[]) record_request_log(reqlog, request, (size_t)reqlen, argv[i]); if (reqlen > 0) record_count(countfile, i + 1); + if (reqlen > 0 && strncmp(argv[i], "CODEX_", 6) == 0) { + char parameter[512]; + + if (validate_codex_request(request, (size_t)reqlen, + parameter, sizeof(parameter)) == -1) { + send_codex_schema_error(conn, parameter); + goto request_done; + } + } if (reqlen > 0 && strcmp(argv[i], "HOSTILE") == 0) send_sse(conn, sse_hostile); else if (reqlen > 0 && strcmp(argv[i], "BADUSAGE") == 0) @@ -797,6 +1365,7 @@ main(int argc, char *argv[]) for (;;) pause(); } +request_done: if (reflected_key != NULL) { explicit_bzero(reflected_key, strlen(reflected_key)); free(reflected_key); blob - a0ef980b8b60ea97e8e95889a8741154af129172 blob + df020c71557324f7aca1c6af6561446f734f4a6a --- regress/turn_txn/Makefile +++ regress/turn_txn/Makefile @@ -3,7 +3,8 @@ SRCS_turn_txn_test= turn_txn_test.c turn_txn.c msg.c p buf.c json.c log.c xmalloc.c SRCS_turn_txn_journal_test=turn_txn_journal_test.c turn_txn.c journal.c \ system_epoch.c compaction.c msg.c codex_sub_req.c \ - codex_sub_profile.c provider_state.c json.c buf.c utf8.c \ + codex_sub_profile.c codex_sub_reasoning.c provider_state.c \ + json.c buf.c utf8.c \ log.c xmalloc.c .PATH: ${.CURDIR}/../../src/fugu blob - f7675fa3e4d610523bcb5e696921e17e7201f451 blob + d1ac75dc265c82e0c4d80f861c4454f15a74cf98 --- src/common/codex_sub_req.c +++ src/common/codex_sub_req.c @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -29,17 +30,32 @@ #include "msg.h" #include "provider_state.h" #include "codex_sub.h" +#include "codex_sub_reasoning.h" #include "proto.h" #define CODEX_SUB_TOOL_NAME_MAX 32 struct state_reasoning { int index; - const u_char *item; - size_t item_len; + struct buf request_item; char item_id[CODEX_SUB_ITEM_ID_MAX + 1]; }; +enum state_message_phase { + STATE_PHASE_NONE, + STATE_PHASE_COMMENTARY, + STATE_PHASE_FINAL_ANSWER +}; + +struct state_message { + int index; + char item_id[CODEX_SUB_ITEM_ID_MAX + 1]; + enum state_message_phase phase; + size_t text_offset; + size_t text_len; + char text_sha256[SHA256_DIGEST_STRING_LENGTH]; +}; + struct state_call { int index; char call_id[CODEX_SUB_ITEM_ID_MAX + 1]; @@ -49,6 +65,7 @@ struct state_call { enum state_event_kind { STATE_REASONING, + STATE_MESSAGE, STATE_CALL }; @@ -61,25 +78,18 @@ struct state_event { struct state_view { struct json json; struct state_reasoning reasoning[CODEX_SUB_OUTPUT_MAX]; + struct state_message messages[CODEX_SUB_OUTPUT_MAX]; struct state_call calls[CODEX_SUB_OUTPUT_MAX]; struct state_event events[CODEX_SUB_OUTPUT_MAX]; int nreasoning; + int nmessages; int ncalls; int nevents; + int version; int parsed; }; static int -json_is_null_value(const struct json *j, int tok) -{ - const u_char *raw; - size_t len; - - return (json_raw_span(j, tok, &raw, &len) == 0 && len == 4 && - memcmp(raw, "null", 4) == 0); -} - -static int safe_session_id(const char *id) { size_t i, len; @@ -169,44 +179,14 @@ parse_index(const struct json *j, int tok, int *out) static int parse_reasoning_item(struct state_reasoning *reasoning, - const struct json *j, int item) + const struct json *j, int item, enum codex_sub_reasoning_mode mode) { - static const char *const names[] = { - "type", "id", "summary", "content", "encrypted_content", "status" - }; - int fields[6], t; - char type[32], status[32], *encrypted; - size_t encrypted_len; - - if (!json_is_object(j, item) || strict_fields(j, item, names, - (int)(sizeof(names) / sizeof(names[0])), fields) == -1 || - copy_json_ident(j, fields[0], type, sizeof(type)) == -1 || - strcmp(type, "reasoning") != 0 || - copy_json_ident(j, fields[1], reasoning->item_id, - sizeof(reasoning->item_id)) == -1 || - strncmp(reasoning->item_id, "rs_", 3) != 0 || fields[2] == -1 || - !json_is_array(j, fields[2]) || fields[4] == -1 || - (encrypted = json_get_str(j, fields[4], &encrypted_len)) == NULL) + buf_init(&reasoning->request_item); + if (codex_sub_reasoning_transform(&reasoning->request_item, + reasoning->item_id, sizeof(reasoning->item_id), j, item, mode) == -1) { + buf_free(&reasoning->request_item); return (-1); - if (encrypted_len == 0 || strlen(encrypted) != encrypted_len) { - free(encrypted); - return (-1); } - free(encrypted); - if (fields[3] != -1 && !json_is_null_value(j, fields[3]) && - !json_is_array(j, fields[3])) - return (-1); - if (fields[5] == -1 || copy_json_ident(j, fields[5], status, - sizeof(status)) == -1 || strcmp(status, "completed") != 0) - return (-1); - if (json_raw_span(j, item, &reasoning->item, - &reasoning->item_len) == -1 || reasoning->item_len == 0) - return (-1); - /* Ensure every required field really occurred even when object length was - * smaller than the allowed field vocabulary. */ - for (t = 0; t < 3; t++) - if (fields[t] == -1) - return (-1); return (0); } @@ -229,6 +209,9 @@ state_item_id_exists(const struct state_view *view, co for (i = 0; i < view->nreasoning; i++) if (strcmp(view->reasoning[i].item_id, item_id) == 0) return (1); + for (i = 0; i < view->nmessages; i++) + if (strcmp(view->messages[i].item_id, item_id) == 0) + return (1); for (i = 0; i < view->ncalls; i++) if (strcmp(view->calls[i].item_id, item_id) == 0) return (1); @@ -253,22 +236,70 @@ state_event_insert(struct state_view *view, int index, } static int -state_parse(struct state_view *view, const struct provider_state *state) +parse_size(const struct json *j, int token, size_t max, size_t *out) { + int64_t value; + + if (token < 0 || json_get_num(j, token, &value) == -1 || value < 0 || + (uint64_t)value > (uint64_t)max) + return (-1); + *out = (size_t)value; + return (0); +} + +static int +parse_message_phase(const struct json *j, int token, + enum state_message_phase *phase) +{ + char value[32]; + + *phase = STATE_PHASE_NONE; + if (token == -1) + return (0); + if (copy_json_ident(j, token, value, sizeof(value)) == -1) + return (-1); + if (strcmp(value, "commentary") == 0) + *phase = STATE_PHASE_COMMENTARY; + else if (strcmp(value, "final_answer") == 0) + *phase = STATE_PHASE_FINAL_ANSWER; + else + return (-1); + return (0); +} + +static int +copy_digest(const struct json *j, int token, char *digest, size_t digestsz) +{ + char *value; + size_t i, len; + + if (digestsz != SHA256_DIGEST_STRING_LENGTH || token < 0 || + (value = json_get_str(j, token, &len)) == NULL) + return (-1); + if (len != SHA256_DIGEST_LENGTH * 2 || strlen(value) != len) { + free(value); + return (-1); + } + for (i = 0; i < len; i++) + if (!((value[i] >= '0' && value[i] <= '9') || + (value[i] >= 'a' && value[i] <= 'f'))) { + free(value); + return (-1); + } + memcpy(digest, value, len + 1); + free(value); + return (0); +} + +static int +state_parse_legacy(struct state_view *view, int root) +{ static const char *const root_names[] = { "reasoning", "calls" }; static const char *const reasoning_names[] = { "index", "item" }; static const char *const call_names[] = { "index", "call_id", "item_id" }; - int root_fields[2], fields[3], root, reasoning, calls; + int root_fields[2], fields[3], reasoning, calls; int i, nreasoning, ncalls, item; - memset(view, 0, sizeof(*view)); - if (state == NULL || state->data == NULL || state->data_len == 0 || - state->data_len > PROVIDER_STATE_MAX || - !json_valid_object(state->data, state->data_len) || - json_parse(&view->json, state->data, state->data_len, 0) == -1) - return (-1); - view->parsed = 1; - root = json_root(&view->json); if (strict_fields(&view->json, root, root_names, 2, root_fields) == -1 || root_fields[0] == -1 || root_fields[1] == -1 || !json_is_array(&view->json, root_fields[0]) || @@ -289,10 +320,13 @@ state_parse(struct state_view *view, const struct prov parse_index(&view->json, fields[0], &view->reasoning[i].index) == -1 || state_index_exists(view, view->reasoning[i].index) || - parse_reasoning_item(&view->reasoning[i], &view->json, - fields[1]) == -1 || state_item_id_exists(view, - view->reasoning[i].item_id)) + parse_reasoning_item(&view->reasoning[i], &view->json, fields[1], + CODEX_SUB_REASONING_LEGACY_REQUEST) == -1) return (-1); + if (state_item_id_exists(view, view->reasoning[i].item_id)) { + buf_free(&view->reasoning[i].request_item); + return (-1); + } view->nreasoning++; state_event_insert(view, view->reasoning[i].index, STATE_REASONING, i); @@ -318,12 +352,148 @@ state_parse(struct state_view *view, const struct prov view->ncalls++; state_event_insert(view, view->calls[i].index, STATE_CALL, i); } + view->version = 1; return (0); } +static int +state_parse_v2(struct state_view *view, int root) +{ + static const char *const root_names[] = { "version", "items" }; + static const char *const item_names[] = { + "index", "kind", "item", "call_id", "item_id", "phase", + "text_offset", "text_len", "text_sha256" + }; + int root_fields[2], fields[9], items, object, i, n, index; + int64_t version; + size_t text_end = 0; + char kind[32]; + + if (strict_fields(&view->json, root, root_names, 2, root_fields) == -1 || + root_fields[0] == -1 || root_fields[1] == -1 || + json_get_num(&view->json, root_fields[0], &version) == -1 || + version != 2 || !json_is_array(&view->json, root_fields[1])) + return (-1); + items = root_fields[1]; + if ((n = json_arr_len(&view->json, items)) < 1 || + n > CODEX_SUB_OUTPUT_MAX) + return (-1); + for (i = 0; i < n; i++) { + object = json_arr_get(&view->json, items, i); + if (strict_fields(&view->json, object, item_names, 9, fields) == -1 || + fields[0] == -1 || fields[1] == -1 || + parse_index(&view->json, fields[0], &index) == -1 || index != i || + state_index_exists(view, index) || + copy_json_ident(&view->json, fields[1], kind, + sizeof(kind)) == -1) + return (-1); + if (strcmp(kind, "reasoning") == 0) { + struct state_reasoning *reasoning; + + if (fields[2] == -1 || fields[3] != -1 || fields[4] != -1 || + fields[5] != -1 || fields[6] != -1 || fields[7] != -1 || + fields[8] != -1 || view->nreasoning >= CODEX_SUB_OUTPUT_MAX) + return (-1); + reasoning = &view->reasoning[view->nreasoning]; + reasoning->index = index; + if (parse_reasoning_item(reasoning, &view->json, fields[2], + CODEX_SUB_REASONING_V2_REQUEST) == -1) + return (-1); + if (state_item_id_exists(view, reasoning->item_id)) { + buf_free(&reasoning->request_item); + return (-1); + } + state_event_insert(view, index, STATE_REASONING, + view->nreasoning++); + } else if (strcmp(kind, "message") == 0) { + struct state_message *message; + + if (fields[2] != -1 || fields[3] != -1 || fields[4] == -1 || + fields[6] == -1 || fields[7] == -1 || fields[8] == -1 || + view->nmessages >= CODEX_SUB_OUTPUT_MAX) + return (-1); + message = &view->messages[view->nmessages]; + message->index = index; + if (copy_json_ident(&view->json, fields[4], message->item_id, + sizeof(message->item_id)) == -1 || + strncmp(message->item_id, "msg_", 4) != 0 || + state_item_id_exists(view, message->item_id) || + parse_message_phase(&view->json, fields[5], + &message->phase) == -1 || + parse_size(&view->json, fields[6], FUGU_RESPONSE_MAX, + &message->text_offset) == -1 || + parse_size(&view->json, fields[7], FUGU_RESPONSE_MAX, + &message->text_len) == -1 || + message->text_offset != text_end || + message->text_len > FUGU_RESPONSE_MAX - text_end || + copy_digest(&view->json, fields[8], message->text_sha256, + sizeof(message->text_sha256)) == -1) + return (-1); + text_end += message->text_len; + state_event_insert(view, index, STATE_MESSAGE, + view->nmessages++); + } else if (strcmp(kind, "call") == 0) { + struct state_call *call; + int k; + + if (fields[2] != -1 || fields[3] == -1 || fields[4] == -1 || + fields[5] != -1 || fields[6] != -1 || fields[7] != -1 || + fields[8] != -1 || view->ncalls >= CODEX_SUB_OUTPUT_MAX) + return (-1); + call = &view->calls[view->ncalls]; + call->index = index; + if (copy_json_ident(&view->json, fields[3], call->call_id, + sizeof(call->call_id)) == -1 || + copy_json_ident(&view->json, fields[4], call->item_id, + sizeof(call->item_id)) == -1 || + strncmp(call->item_id, "fc_", 3) != 0 || + state_item_id_exists(view, call->item_id)) + return (-1); + for (k = 0; k < view->ncalls; k++) + if (strcmp(view->calls[k].call_id, call->call_id) == 0) + return (-1); + state_event_insert(view, index, STATE_CALL, view->ncalls++); + } else + return (-1); + } + view->version = 2; + return (0); +} + +static int +state_parse(struct state_view *view, const struct provider_state *state) +{ + int root; + + memset(view, 0, sizeof(*view)); + if (state == NULL || state->data == NULL || state->data_len == 0 || + state->data_len > PROVIDER_STATE_MAX || + !json_valid_object(state->data, state->data_len) || + json_parse(&view->json, state->data, state->data_len, 0) == -1) + return (-1); + view->parsed = 1; + root = json_root(&view->json); + if (state_parse_v2(view, root) == 0) + return (0); + /* A failed v2 parse may have allocated typed reasoning; start clean. */ + for (int i = 0; i < view->nreasoning; i++) + buf_free(&view->reasoning[i].request_item); + json_done(&view->json); + memset(view, 0, sizeof(*view)); + if (json_parse(&view->json, state->data, state->data_len, 0) == -1) + return (-1); + view->parsed = 1; + root = json_root(&view->json); + return (state_parse_legacy(view, root)); +} + static void state_done(struct state_view *view) { + int i; + + for (i = 0; i < view->nreasoning; i++) + buf_free(&view->reasoning[i].request_item); if (view->parsed) json_done(&view->json); memset(view, 0, sizeof(*view)); @@ -461,7 +631,69 @@ emit_text_message(struct json_out *out, const struct m json_obj_end(out); } +static const char * +state_phase_name(enum state_message_phase phase) +{ + switch (phase) { + case STATE_PHASE_COMMENTARY: + return ("commentary"); + case STATE_PHASE_FINAL_ANSWER: + return ("final_answer"); + case STATE_PHASE_NONE: + return (NULL); + } + return (NULL); +} + +static void +emit_state_message(struct json_out *out, const struct state_message *message, + const void *text, size_t textlen) +{ + const char *phase; + + json_obj_begin(out); + json_key(out, "type"); + json_cstr(out, "message"); + json_key(out, "id"); + json_cstr(out, message->item_id); + json_key(out, "role"); + json_cstr(out, "assistant"); + json_key(out, "content"); + json_arr_begin(out); + json_obj_begin(out); + json_key(out, "type"); + json_cstr(out, "output_text"); + json_key(out, "text"); + json_str(out, textlen == 0 ? "" : text, textlen); + json_obj_end(out); + json_arr_end(out); + if ((phase = state_phase_name(message->phase)) != NULL) { + json_key(out, "phase"); + json_cstr(out, phase); + } + json_obj_end(out); +} + static int +flatten_message_text(struct buf *text, const struct msg *message) +{ + const struct block *block; + + buf_init(text); + TAILQ_FOREACH(block, &message->blocks, entry) { + if (block->type != BLOCK_TEXT) + continue; + if (block->textlen > FUGU_RESPONSE_MAX - text->len) { + buf_free(text); + return (-1); + } + if (block->textlen > 0) + buf_add(text, block->text, block->textlen); + } + return (0); +} + +static int message_has_text(const struct msg *message) { const struct block *block; @@ -509,7 +741,7 @@ static int join_state_calls(struct state_view *view, const struct msg *message) { const struct block *block; - int i, matches, last_index = -1; + int i, matches, tool_count = 0, last_index = -1; for (i = 0; i < view->ncalls; i++) { matches = 0; @@ -526,6 +758,7 @@ join_state_calls(struct state_view *view, const struct TAILQ_FOREACH(block, &message->blocks, entry) { if (block->type != BLOCK_TOOL_USE) continue; + tool_count++; for (i = 0; i < view->ncalls; i++) if (view->calls[i].block == block) { if (view->calls[i].index <= last_index) @@ -534,27 +767,69 @@ join_state_calls(struct state_view *view, const struct break; } } + if (view->version == 2 && tool_count != view->ncalls) + return (-1); return (0); } +static int +validate_state_messages(const struct state_view *view, + const struct msg *message, struct buf *text) +{ + char digest[SHA256_DIGEST_STRING_LENGTH]; + size_t end = 0; + int i; + + buf_init(text); + if (view->version != 2) + return (0); + if (flatten_message_text(text, message) == -1) + return (-1); + for (i = 0; i < view->nmessages; i++) { + const struct state_message *state = &view->messages[i]; + const void *data; + + if (state->text_offset != end || end > text->len || + state->text_len > text->len - end) + goto fail; + data = state->text_len == 0 ? (const void *)"" : + text->data + state->text_offset; + SHA256Data(data, state->text_len, digest); + if (strcmp(digest, state->text_sha256) != 0) + goto fail; + end += state->text_len; + } + if (end != text->len) + goto fail; + return (0); + +fail: + buf_free(text); + return (-1); +} + int codex_sub_state_valid_for_message(const struct provider_state *state, const struct msg *message) { const struct codex_sub_profile *profile = codex_sub_profile(); struct state_view view; + struct buf text; int valid = -1; memset(&view, 0, sizeof(view)); + buf_init(&text); if (message == NULL || message->role != ROLE_ASSISTANT || !provider_state_matches(state, PROVIDER_CODEX, profile->id, profile->model, profile->state_kind) || state_parse(&view, state) == -1 || - join_state_calls(&view, message) == -1) + join_state_calls(&view, message) == -1 || + validate_state_messages(&view, message, &text) == -1) goto done; valid = 0; done: + buf_free(&text); state_done(&view); return (valid); } @@ -584,6 +859,40 @@ next_tool_block(const struct msg *message, const struc } static int +emit_v2_assistant(struct json_out *out, const struct msg *message, + const struct state_view *view, const struct buf *text) +{ + const struct block *next = next_tool_block(message, NULL); + int i; + + for (i = 0; i < view->nevents; i++) { + if (view->events[i].kind == STATE_REASONING) { + const struct state_reasoning *reasoning = + &view->reasoning[view->events[i].slot]; + + json_raw(out, (const char *)reasoning->request_item.data, + reasoning->request_item.len); + } else if (view->events[i].kind == STATE_MESSAGE) { + const struct state_message *state = + &view->messages[view->events[i].slot]; + const void *data = state->text_len == 0 ? (const void *)"" : + text->data + state->text_offset; + + emit_state_message(out, state, data, state->text_len); + } else { + const struct state_call *call = + &view->calls[view->events[i].slot]; + + if (next == NULL || next != call->block) + return (-1); + emit_function_call(out, next, call->item_id); + next = next_tool_block(message, next); + } + } + return (next == NULL ? 0 : -1); +} + +static int emit_assistant(struct json_out *out, const struct msg *message, const char *model) { @@ -591,24 +900,34 @@ emit_assistant(struct json_out *out, const struct msg const struct block *block, *next; const struct state_call *call; struct state_view view; - int i, have_state, text_emitted = 0; + struct buf text; + int i, result, have_state, text_emitted = 0; memset(&view, 0, sizeof(view)); + buf_init(&text); have_state = provider_state_matches(message->provider_state, PROVIDER_CODEX, profile->id, model, profile->state_kind); if (have_state && (state_parse(&view, message->provider_state) == -1 || - join_state_calls(&view, message) == -1)) { + join_state_calls(&view, message) == -1 || + validate_state_messages(&view, message, &text) == -1)) { + buf_free(&text); state_done(&view); return (-1); } + if (view.version == 2) { + result = emit_v2_assistant(out, message, &view, &text); + buf_free(&text); + state_done(&view); + return (result); + } next = next_tool_block(message, NULL); for (i = 0; i < view.nevents; i++) { if (view.events[i].kind == STATE_REASONING) { const struct state_reasoning *reasoning = &view.reasoning[view.events[i].slot]; - json_raw(out, (const char *)reasoning->item, - reasoning->item_len); + json_raw(out, (const char *)reasoning->request_item.data, + reasoning->request_item.len); } else { call = &view.calls[view.events[i].slot]; if (!text_emitted && message_has_text(message)) { @@ -626,6 +945,7 @@ emit_assistant(struct json_out *out, const struct msg break; } if (block != view.calls[view.events[i].slot].block) { + buf_free(&text); state_done(&view); return (-1); } @@ -640,6 +960,7 @@ emit_assistant(struct json_out *out, const struct msg emit_function_call(out, block, call != NULL ? call->item_id : NULL); } + buf_free(&text); state_done(&view); return (0); } blob - /dev/null blob + 38373556c7ba58b426e1cad7bc76899af5a9f0c3 (mode 644) --- /dev/null +++ src/common/codex_sub_reasoning.c @@ -0,0 +1,235 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* Typed reasoning translation shared by the response and request adapters. */ + +#include + +#include +#include + +#include "buf.h" +#include "json.h" +#include "provider_state.h" +#include "codex_sub_reasoning.h" + +static int +strict_fields(const struct json *json, int object, const char *const *names, + int nname, int *fields) +{ + char *key; + size_t keylen; + int i, k, n, token, found; + + if ((n = json_obj_len(json, object)) < 0) + return (-1); + for (i = 0; i < nname; i++) + fields[i] = -1; + for (i = 0; i < n; i++) { + if (json_obj_item(json, object, i, &key, &keylen, &token) == -1) + return (-1); + found = -1; + for (k = 0; k < nname; k++) + if (strlen(names[k]) == keylen && + memcmp(names[k], key, keylen) == 0) { + found = k; + break; + } + free(key); + if (found == -1 || fields[found] != -1) + return (-1); + fields[found] = token; + } + return (0); +} + +static int +json_is_null_value(const struct json *json, int token) +{ + const u_char *raw; + size_t len; + + return (json_raw_span(json, token, &raw, &len) == 0 && len == 4 && + memcmp(raw, "null", 4) == 0); +} + +static char * +get_string(const struct json *json, int token, size_t *lenp, int empty_ok) +{ + char *value; + size_t len; + + if (token < 0 || (value = json_get_str(json, token, &len)) == NULL) + return (NULL); + if ((!empty_ok && len == 0) || strlen(value) != len) { + free(value); + return (NULL); + } + if (lenp != NULL) + *lenp = len; + return (value); +} + +/* Text is length-bearing and may legitimately contain an embedded NUL. */ +static char * +get_text(const struct json *json, int token, size_t *lenp) +{ + char *value; + size_t len; + + if (token < 0 || (value = json_get_str(json, token, &len)) == NULL) + return (NULL); + if (lenp != NULL) + *lenp = len; + return (value); +} + +static int +copy_ident(const struct json *json, int token, char *output, size_t outsz) +{ + char *value; + size_t len; + + if ((value = get_string(json, token, &len, 0)) == NULL) + return (-1); + if (len >= outsz || memchr(value, '\r', len) != NULL || + memchr(value, '\n', len) != NULL) { + free(value); + return (-1); + } + memcpy(output, value, len + 1); + free(value); + return (0); +} + +/* Build one canonical array of typed {type,text} parts. */ +static int +transform_parts(struct buf *output, const struct json *json, int array, + int summary, int *has_reasoning_text) +{ + static const char *const names[] = { "type", "text" }; + struct json_out out; + char type[32], *text; + size_t textlen; + int fields[2], i, n, part; + + /* The JSON parser's token and Provider-state byte limits bound this + * nested vector. CODEX_SUB_OUTPUT_MAX applies only to output items. */ + if (!json_is_array(json, array) || (n = json_arr_len(json, array)) < 0) + return (-1); + json_out_init(&out, output); + json_arr_begin(&out); + for (i = 0; i < n; i++) { + part = json_arr_get(json, array, i); + if (strict_fields(json, part, names, 2, fields) == -1 || + fields[0] == -1 || fields[1] == -1 || + copy_ident(json, fields[0], type, sizeof(type)) == -1 || + (summary && strcmp(type, "summary_text") != 0) || + (!summary && strcmp(type, "reasoning_text") != 0 && + strcmp(type, "text") != 0) || + (text = get_text(json, fields[1], &textlen)) == NULL) + return (-1); + if (!summary && strcmp(type, "reasoning_text") == 0) + *has_reasoning_text = 1; + json_obj_begin(&out); + json_key(&out, "type"); + json_cstr(&out, type); + json_key(&out, "text"); + json_str(&out, text, textlen); + json_obj_end(&out); + free(text); + if (output->len > PROVIDER_STATE_MAX) + return (-1); + } + json_arr_end(&out); + return (output->len <= PROVIDER_STATE_MAX ? 0 : -1); +} + +int +codex_sub_reasoning_transform(struct buf *output, char *id, size_t idsz, + const struct json *json, int object, enum codex_sub_reasoning_mode mode) +{ + static const char *const names[] = { + "type", "id", "summary", "content", "encrypted_content", "status" + }; + struct buf canonical, summary, content; + struct json_out out; + char type[32], status[32], *encrypted = NULL; + size_t encrypted_len; + int fields[6], have_content = 0, has_reasoning_text = 0, error = -1; + + buf_init(&canonical); + buf_init(&summary); + buf_init(&content); + if (output == NULL || id == NULL || idsz == 0 || + mode < CODEX_SUB_REASONING_RESPONSE_STATE || + mode > CODEX_SUB_REASONING_V2_REQUEST || + !json_is_object(json, object) || + strict_fields(json, object, names, 6, fields) == -1 || + fields[0] == -1 || fields[1] == -1 || fields[2] == -1 || + fields[4] == -1 || + copy_ident(json, fields[0], type, sizeof(type)) == -1 || + strcmp(type, "reasoning") != 0 || + copy_ident(json, fields[1], id, idsz) == -1 || + strncmp(id, "rs_", 3) != 0 || + transform_parts(&summary, json, fields[2], 1, + &has_reasoning_text) == -1 || + (encrypted = get_string(json, fields[4], &encrypted_len, 0)) == NULL) + goto done; + + if (mode == CODEX_SUB_REASONING_LEGACY_REQUEST && fields[5] == -1) + goto done; + if (mode == CODEX_SUB_REASONING_V2_REQUEST && fields[5] != -1) + goto done; + if (fields[5] != -1 && + (copy_ident(json, fields[5], status, sizeof(status)) == -1 || + strcmp(status, "completed") != 0)) + goto done; + if (fields[3] != -1 && !json_is_null_value(json, fields[3])) { + if (transform_parts(&content, json, fields[3], 0, + &has_reasoning_text) == -1) + goto done; + have_content = 1; + } + + json_out_init(&out, &canonical); + json_obj_begin(&out); + json_key(&out, "type"); + json_cstr(&out, "reasoning"); + json_key(&out, "id"); + json_cstr(&out, id); + json_key(&out, "summary"); + json_raw(&out, (const char *)summary.data, summary.len); + if (have_content && (mode == CODEX_SUB_REASONING_RESPONSE_STATE || + !has_reasoning_text)) { + json_key(&out, "content"); + json_raw(&out, (const char *)content.data, content.len); + } + json_key(&out, "encrypted_content"); + json_str(&out, encrypted, encrypted_len); + json_obj_end(&out); + if (canonical.len == 0 || canonical.len > PROVIDER_STATE_MAX) + goto done; + buf_add(output, canonical.data, canonical.len); + error = 0; + +done: + free(encrypted); + buf_free(&content); + buf_free(&summary); + buf_free(&canonical); + return (error); +} blob - 110d1b32c843f11c6656bd7ec2e80be16bcee5cc blob + d0218b83ccfc9e3e1897c3ec4541c6c18896fa1d --- src/common/codex_sub_stream.c +++ src/common/codex_sub_stream.c @@ -19,6 +19,7 @@ #include #include +#include #include #include #include @@ -29,6 +30,7 @@ #include "sse.h" #include "provider_state.h" #include "codex_sub.h" +#include "codex_sub_reasoning.h" #include "proto.h" #define CODEX_EVENT_TYPE_MAX 128 @@ -83,6 +85,8 @@ struct codex_sub_stream { size_t args_total; size_t reasoning_total; size_t summary_total; + int last_text_index; + int have_text_index; int created; int in_progress; int finished; @@ -247,7 +251,8 @@ item_new(struct codex_sub_stream *stream, int index, e { struct output_item *item; - if (index < 0 || item_find(stream, index) != NULL || + if (index < 0 || index != stream->nitems || + item_find(stream, index) != NULL || stream->nitems >= CODEX_SUB_OUTPUT_MAX) return (NULL); item = &stream->items[stream->nitems++]; @@ -275,6 +280,16 @@ item_id_used(const struct codex_sub_stream *stream, } static int +text_index_accept(struct codex_sub_stream *stream, int index) +{ + if (stream->have_text_index && index < stream->last_text_index) + return (-1); + stream->last_text_index = index; + stream->have_text_index = 1; + return (0); +} + +static int call_id_used(const struct codex_sub_stream *stream, const struct output_item *self, const char *id) { @@ -326,62 +341,13 @@ static int parse_reasoning(const struct json *json, int object, char *id, size_t idsz, struct buf *canonical) { - static const char *const names[] = { - "type", "id", "summary", "content", "encrypted_content", "status" - }; - int fields[6]; - char type[32], *encrypted; - const u_char *raw; - size_t encrypted_len, rawlen; - struct json_out out; - buf_init(canonical); - if (strict_fields(json, object, names, 6, fields) == -1 || - copy_ident(json, fields[0], type, sizeof(type)) == -1 || - strcmp(type, "reasoning") != 0 || - copy_ident(json, fields[1], id, idsz) == -1 || - strncmp(id, "rs_", 3) != 0 || fields[2] == -1 || - !json_is_array(json, fields[2]) || fields[4] == -1 || - (encrypted = get_string(json, fields[4], &encrypted_len, 0)) == NULL) - return (-1); - if (fields[3] != -1 && !json_is_null_value(json, fields[3]) && - !json_is_array(json, fields[3])) - goto fail; - if (fields[5] != -1 && - parse_status(json, fields[5], "completed") == -1) - goto fail; - json_out_init(&out, canonical); - json_obj_begin(&out); - json_key(&out, "type"); - json_cstr(&out, "reasoning"); - json_key(&out, "id"); - json_cstr(&out, id); - json_key(&out, "summary"); - if (json_raw_span(json, fields[2], &raw, &rawlen) == -1) - goto fail; - json_raw(&out, (const char *)raw, rawlen); - if (fields[3] != -1) { - json_key(&out, "content"); - if (json_raw_span(json, fields[3], &raw, &rawlen) == -1) - goto fail; - json_raw(&out, (const char *)raw, rawlen); - } - json_key(&out, "encrypted_content"); - json_str(&out, encrypted, encrypted_len); - json_key(&out, "status"); - json_cstr(&out, "completed"); - json_obj_end(&out); - free(encrypted); - if (canonical->len == 0 || canonical->len > PROVIDER_STATE_MAX) { + if (codex_sub_reasoning_transform(canonical, id, idsz, json, object, + CODEX_SUB_REASONING_RESPONSE_STATE) == -1) { buf_free(canonical); return (-1); } return (0); - -fail: - free(encrypted); - buf_free(canonical); - return (-1); } struct parsed_call { @@ -447,6 +413,20 @@ parse_message_phase(const struct json *json, int token return (0); } +static const char * +message_phase_name(enum message_phase phase) +{ + switch (phase) { + case MESSAGE_PHASE_COMMENTARY: + return ("commentary"); + case MESSAGE_PHASE_FINAL_ANSWER: + return ("final_answer"); + case MESSAGE_PHASE_NONE: + return (NULL); + } + return (NULL); +} + static int parse_message_text(const struct json *json, int object, const char *status, char *id, size_t idsz, struct buf *text, enum message_phase *phase) @@ -619,6 +599,8 @@ message_apply(struct codex_sub_stream *stream, struct if (item->text.len == 0 && text->len > 0) { if (text->len > FUGU_RESPONSE_MAX - stream->text_total) return (-1); + if (text_index_accept(stream, item->index) == -1) + return (-1); buf_add(&item->text, text->data, text->len); stream->text_total += text->len; memset(&event, 0, sizeof(event)); @@ -782,6 +764,10 @@ handle_text_delta(struct codex_sub_stream *stream, con return (-1); } if (len > 0) { + if (text_index_accept(stream, index) == -1) { + free(delta); + return (-1); + } buf_add(&item->text, delta, len); stream->text_total += len; memset(&event, 0, sizeof(event)); @@ -895,51 +881,86 @@ build_state(struct codex_sub_stream *stream, struct bu struct json_out out; struct buf data; struct output_item *item; - int index, max_index = -1, have = 0; + const char *phase; + char digest[SHA256_DIGEST_STRING_LENGTH]; + size_t text_offset = 0; + int index, max_index = -1, messages = 0, need = 0; - for (int i = 0; i < stream->nitems; i++) + for (int i = 0; i < stream->nitems; i++) { if (stream->items[i].index > max_index) max_index = stream->items[i].index; + if (stream->items[i].kind == OUTPUT_MESSAGE) { + messages++; + if (stream->items[i].phase != MESSAGE_PHASE_NONE) + need = 1; + } else + need = 1; + } + if (messages > 1) + need = 1; + if (!need) + return (0); buf_init(&data); json_out_init(&out, &data); json_obj_begin(&out); - json_key(&out, "reasoning"); + json_key(&out, "version"); + json_num(&out, 2); + json_key(&out, "items"); json_arr_begin(&out); for (index = 0; index <= max_index; index++) { item = item_find(stream, index); - if (item == NULL || item->kind != OUTPUT_REASONING) - continue; - have = 1; + if (item == NULL) { + buf_free(&data); + return (-1); + } json_obj_begin(&out); json_key(&out, "index"); json_num(&out, index); - json_key(&out, "item"); - json_raw(&out, (const char *)item->reasoning.data, - item->reasoning.len); + json_key(&out, "kind"); + if (item->kind == OUTPUT_REASONING) { + json_cstr(&out, "reasoning"); + json_key(&out, "item"); + json_raw(&out, (const char *)item->reasoning.data, + item->reasoning.len); + } else if (item->kind == OUTPUT_MESSAGE) { + json_cstr(&out, "message"); + json_key(&out, "item_id"); + json_cstr(&out, item->id); + if ((phase = message_phase_name(item->phase)) != NULL) { + json_key(&out, "phase"); + json_cstr(&out, phase); + } + json_key(&out, "text_offset"); + json_num(&out, (int64_t)text_offset); + json_key(&out, "text_len"); + json_num(&out, (int64_t)item->text.len); + SHA256Data(item->text.len == 0 ? (const void *)"" : + item->text.data, + item->text.len, digest); + json_key(&out, "text_sha256"); + json_cstr(&out, digest); + if (item->text.len > SIZE_MAX - text_offset) { + buf_free(&data); + return (-1); + } + text_offset += item->text.len; + } else if (item->kind == OUTPUT_CALL) { + json_cstr(&out, "call"); + json_key(&out, "call_id"); + json_cstr(&out, item->call_id); + json_key(&out, "item_id"); + json_cstr(&out, item->id); + } else { + buf_free(&data); + return (-1); + } json_obj_end(&out); } json_arr_end(&out); - json_key(&out, "calls"); - json_arr_begin(&out); - for (index = 0; index <= max_index; index++) { - item = item_find(stream, index); - if (item == NULL || item->kind != OUTPUT_CALL) - continue; - have = 1; - json_obj_begin(&out); - json_key(&out, "index"); - json_num(&out, index); - json_key(&out, "call_id"); - json_cstr(&out, item->call_id); - json_key(&out, "item_id"); - json_cstr(&out, item->id); - json_obj_end(&out); - } - json_arr_end(&out); json_obj_end(&out); - if (!have) { + if (data.len == 0 || data.len > PROVIDER_STATE_MAX) { buf_free(&data); - return (0); + return (-1); } state = provider_state_new(PROVIDER_CODEX, profile->id, stream->model, profile->state_kind, data.data, data.len); blob - /dev/null blob + f6d1f60a9212bbfb7f543032b7c8c8ffd2bf9f13 (mode 644) --- /dev/null +++ src/common/codex_sub_reasoning.h @@ -0,0 +1,32 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + */ + +#ifndef CODEX_SUB_REASONING_H +#define CODEX_SUB_REASONING_H + +#include + +struct buf; +struct json; + +/* + * Reasoning crosses three dialects. Response items may carry completion + * metadata, legacy state must prove completion with that metadata, and v2 + * continuity state must not contain it. Every mode emits a fresh canonical + * object; provider bytes are never replayed directly into a request. + */ +enum codex_sub_reasoning_mode { + CODEX_SUB_REASONING_RESPONSE_STATE, + CODEX_SUB_REASONING_LEGACY_REQUEST, + CODEX_SUB_REASONING_V2_REQUEST +}; + +int codex_sub_reasoning_transform(struct buf *, char *, size_t, + const struct json *, int, enum codex_sub_reasoning_mode); + +#endif /* CODEX_SUB_REASONING_H */ blob - c805edb8d73ef9b945e5385a660518f1fa6fe68f blob + af0eef9fd2830f4783c27904c326d7ca876c372a --- src/fugu/Makefile +++ src/fugu/Makefile @@ -3,7 +3,8 @@ SRCS= main.c coord.c priv.c conf.c parse.y \ journal.c system_epoch.c sysprompt.c tooldefs.c tool_display.c agentcfg.c \ skills.c output.c \ turn_txn.c turn_mechanics.c anthropic_req.c claude_sub_req.c \ - claude_sub_profile.c codex_sub_req.c codex_sub_profile.c openai_req.c \ + claude_sub_profile.c codex_sub_req.c codex_sub_profile.c \ + codex_sub_reasoning.c openai_req.c \ generation.c compaction.c msg.c \ provider_state.c \ json.c utf8.c artifact_result.c artifact_page.c artifact_lock.c \ blob - 0816d59953135e860d9e36367209c01d6e890bbf blob + f17edb082d723aa1f7c3747268bb8e334c548607 --- src/fugu/journal.c +++ src/fugu/journal.c @@ -1165,6 +1165,7 @@ provider_state_token(const struct json *jd, int obj, i static struct msg * parse_message(const struct json *jd, int obj) { + const struct codex_sub_profile *profile; struct msg *m; struct provider_state *state = NULL; char *encoded = NULL; @@ -1304,7 +1305,12 @@ parse_message(const struct json *jd, int obj) msg_free(m); return (NULL); } + profile = codex_sub_profile(); + /* Compatibility-mismatched state is opaque, inert history. Interpret + * only the schema owned by the active profile. */ if (state != NULL && state->provider_type == PROVIDER_CODEX && + provider_state_matches(state, PROVIDER_CODEX, profile->id, + profile->model, profile->state_kind) && codex_sub_state_valid_for_message(state, m) == -1) { free(encoded); provider_state_free(state); blob - 408752cca3844ac95db86c4f7be9580cb8d0d649 blob + 7ffb214c018d987217ac1d4270818a0f45588bec --- src/fugu-api/Makefile +++ src/fugu-api/Makefile @@ -1,6 +1,7 @@ PROG= fugu-api SRCS= main.c worker.c imsgev.c claude_sub_profile.c claude_sub_wire.c \ codex_sub_profile.c codex_sub_wire.c codex_sub_stream.c \ + codex_sub_reasoning.c \ http.c anthropic_stream.c openai_stream.c sse.c json.c buf.c \ provider_state.c model_window.c log.c xmalloc.c BINDIR= ${LIBEXECDIR}/fugu blob - d20aac06cfa37b15618a2fb9c3355db0abf7cb9f blob + 073489ba7e4f25da15aaebaeec890f8af7013c52 --- src/fugu-tool/Makefile +++ src/fugu-tool/Makefile @@ -1,7 +1,7 @@ PROG= fugu-tool SRCS= main.c agent.c agent_boot.c tools.c job.c supervisor.c worker.c imsgev.c \ anthropic_req.c claude_sub_req.c claude_sub_profile.c openai_req.c \ - codex_sub_req.c codex_sub_profile.c \ + codex_sub_req.c codex_sub_profile.c codex_sub_reasoning.c \ turn_mechanics.c generation.c msg.c provider_state.c json.c \ artifact_capture.c artifact_pipe.c artifact_scope.c \ artifact_result.c effort.c buf.c diff.c log.c xmalloc.c