commit be34cde492d09f4d170c455cbc9f45744a760542 from: Isaac Meerleo date: Fri Jul 17 21:13:51 2026 UTC Pin provider context overflow signals commit - 92bb7d00361aafbc40523b8a74a22f41d828b937 commit + be34cde492d09f4d170c455cbc9f45744a760542 blob - /dev/null blob + 528574a4d7db15a5009afadff3eb818075c5eb21 (mode 644) --- /dev/null +++ docs/research/provider-context-overflow-classification.md @@ -0,0 +1,228 @@ +# Provider context-overflow classification + +Research date: 2026-07-17 + +## Decision + +Fugu can safely recognize one provider-reported input-context overflow signal +without interpreting prose: + +```text +OpenAI-compatible error.code == "context_length_exceeded" +``` + +For a rejection before streaming, require HTTP 400 as well. Treat a matching +code in an already-established stream as the same diagnostic category, but let +the Turn's independent "no accepted content" rule decide whether recovery is +still eligible. Do not require or interpret `error.message`, `error.type`, or +`error.param`; the code is the discriminator. + +Direct Anthropic Messages does not expose an equally specific structured error +field. Anthropic documents an overlong input as HTTP 400 with +`error.type: "invalid_request_error"` and the phrase "prompt is too long", but +the same type covers unrelated request-content and format failures. Under the +ticket's status-plus-type/code policy, direct Anthropic input overflow must +therefore remain **unknown**, not recoverable. HTTP 413 +`request_too_large` is a byte-size limit and must not be reclassified as a +token-context overflow. + +The experimental Claude Subscription route is distinguishable inside Fugu, +but its upstream overflow signal is not. It sends to Anthropic and consumes +Anthropic-shaped errors; Anthropic publishes no separate error contract for +this unofficial OAuth compatibility route. It must use the same fail-closed +classification as direct Anthropic unless a future versioned, first-party +fixture supplies a unique structured discriminator. + +## Scope and evidence standard + +This report covers Fugu's three current routes: direct Anthropic Messages, +OpenAI-compatible Chat Completions, and the experimental Claude Subscription +profile. Fugu documents those routes and the subscription adapter's reuse of +Anthropic-shaped SSE in [the provider contract](../../handoff/behavior.md#L690-L718). + +Only provider-owned documentation, provider-owned source, and fixtures checked +into provider-owned source are used for upstream claims. No community error +catalogue or English-message heuristic is treated as a protocol contract. + +## Classification matrix + +| Fugu route and response phase | HTTP status | Structured fields | Classification | Reason | +| --- | ---: | --- | --- | --- | +| OpenAI-compatible, rejected before SSE | `400` | `error.code == "context_length_exceeded"`; `type`, `param`, and `message` may be absent or arbitrary | `context_overflow` | Exact machine code; OpenAI's own Codex fixture pairs it with HTTP 400. | +| OpenAI-compatible, provider error after SSE began | successful stream status (normally `200`) | `error.code == "context_length_exceeded"` | Preserve `context_overflow`; recovery is allowed only if the Turn separately proves that no content was accepted | OpenAI's own client classifies the exact code, not the message. The response phase must remain separate from the category. | +| OpenAI-compatible, any other response | any | missing/different `error.code`, including only `type == "invalid_request_error"` | `unknown` | A compatible server is not necessarily OpenAI and broad 400/type/message rules create false positives. | +| Direct Anthropic, input alone exceeds the window | `400` | `error.type == "invalid_request_error"`; documentation describes "prompt is too long"; no error code | `unknown` | The type is shared by many validation failures and the English message is not a structured discriminator. | +| Direct Anthropic, request exceeds Messages byte ceiling | `413` | `error.type == "request_too_large"` | `request_too_large`, never `context_overflow` | Anthropic defines this as exceeding the 32 MiB request-byte ceiling, before API processing. | +| Direct Anthropic, generation reaches the context boundary | `200` | response `stop_reason == "model_context_window_exceeded"` | valid limited Generation, not a rejected-input overflow | The response is accepted and valid; it can contain output. It must not enter rejection recovery. | +| Direct Anthropic, older-model input plus `max_tokens` validation failure | not pinned by the cited context-window page | documented only as a validation error | `unknown` | No unique status/type/code tuple is documented. | +| Claude Subscription | same cases as Anthropic | same Anthropic-shaped fields | same classifications as Anthropic, tagged with route `claude` for diagnostics | The route is locally known, but no supported subscription-specific upstream error schema exists. | + +The OpenAI rule intentionally keys on `code` rather than requiring +`type == "invalid_request_error"`. OpenAI's current first-party HTTP 400 +fixture contains `code` and `message` but omits `type` and `param`; adding a +type requirement would reject that pinned client fixture. Conversely, +HTTP 400 alone is only a generic bad request. OpenAI's public error guide says +`BadRequestError` covers malformed, missing, incorrectly encoded, or oversized +request data and does not list `context_length_exceeded` as a public enum +([OpenAI error guide](https://developers.openai.com/api/docs/guides/error-codes)). +The current Chat Completions reference likewise specifies the success shape, +not a closed set of error codes +([Chat Completions reference](https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create)). +The rule is therefore an exact signal to recognize when present, not a promise +that every OpenAI-compatible server will emit it. + +## Anthropic evidence and limits + +Anthropic's context-window documentation explicitly distinguishes three +conditions: + +1. Input alone beyond the model window returns HTTP 400 + `invalid_request_error`, described as "prompt is too long". +2. On Claude 4.5 and newer, input plus `max_tokens` may be accepted; reaching + the boundary during generation produces + `stop_reason: "model_context_window_exceeded"`. +3. Earlier models instead return a validation error for the second condition. + +See [Anthropic's context-window overflow behavior](https://platform.claude.com/docs/en/build-with-claude/context-windows#context-window-overflow-behavior) +and [the stop-reason contract](https://platform.claude.com/docs/en/build-with-claude/handling-stop-reasons#model_context_window_exceeded). +Anthropic calls the latter a valid, limited response, so it is not evidence +that the input request was rejected and cannot authorize same-Generation +compaction/retry. + +The generic API error contract explains why HTTP 400 plus +`invalid_request_error` is insufficient. That type means any problem with the +request's format or content and may cover other unlisted 4xx statuses. The +same page gives unrelated 400 examples, including unsupported Assistant +prefill and modified thinking blocks. Error JSON guarantees `error.type` and +`error.message`, but no `error.code` +([Anthropic API errors](https://platform.claude.com/docs/en/api/errors)). +Anthropic also advises clients to use typed errors rather than string-match +messages. Fugu should not weaken that boundary with `contains("prompt is too +long")`, token-count regular expressions, or translated/case-folded variants. + +The same error page defines HTTP 413 `request_too_large` as exceeding the +Messages API's 32 MiB byte limit and notes that Cloudflare can return it before +the request reaches Anthropic. Compaction might incidentally shrink such a +request, but that does not turn a transport-byte limit into evidence of model +context overflow. + +## OpenAI evidence and limits + +OpenAI's first-party Codex source provides the strongest machine-readable +evidence currently available: + +- its response decoder defines an optional `error.code` and maps exactly + `"context_length_exceeded"` to a dedicated context-window error, without + inspecting the English message + ([classifier at commit `161748a`](https://github.com/openai/codex/blob/161748a68eb4a4aba5420c3a6f1739f098513178/codex-rs/codex-api/src/sse/responses.rs#L625-L627)); +- its decoder regression includes a `response.failed` event carrying that + exact code and proves the typed result + ([stream fixture](https://github.com/openai/codex/blob/161748a68eb4a4aba5420c3a6f1739f098513178/codex-rs/codex-api/src/sse/responses.rs#L1025-L1036)); and +- its remote-compaction regression returns HTTP 400 with an error object whose + `code` is `context_length_exceeded` + ([HTTP fixture](https://github.com/openai/codex/blob/161748a68eb4a4aba5420c3a6f1739f098513178/codex-rs/core/tests/suite/compact_remote.rs#L3479-L3488)). + +Those source fixtures exercise OpenAI's Responses/compact client rather than +Fugu's Chat Completions endpoint. They establish the meaning and current +first-party handling of the code, but they do not convert it into a documented +Chat Completions guarantee. Consequently: + +- classify the exact code when a Chat Completions-compatible endpoint emits + it; +- fail closed when it is absent; +- do not infer overflow from `error.type`, `error.param`, HTTP 400, or prose; + and +- add a provider-profile rule only after capturing a primary fixture from + that actual compatible server. + +This makes false negatives possible on servers that use another signal, but +keeps an ordinary invalid request from causing an automatic context rewrite +and retry. + +## Claude Subscription evidence and limits + +Fugu's `claude` route is locally distinct (`PROVIDER_CLAUDE`), but it uses the +same Anthropic error-body decoder as direct Messages +([HTTP dispatch](../../src/fugu-api/main.c#L1284-L1304)) and the same +Anthropic SSE decoder ([provider selection](../../src/fugu-api/main.c#L1341-L1347)). +Its profile changes credentials, identity headers, request placement, and tool +names; it does not define another response error schema. + +Anthropic does not document subscription OAuth as a supported third-party +Messages authentication contract. Its direct API authentication guide lists +API keys and Workload Identity Federation, while its Claude Code policy says +subscription OAuth is for Claude Code and native Anthropic applications +([API authentication](https://platform.claude.com/docs/en/manage-claude/authentication), +[Claude Code credential policy](https://code.claude.com/docs/en/legal-and-compliance#authentication-and-credential-use)). +Public Messages documentation therefore cannot guarantee that compatibility +rejection, quota enforcement, and input overflow will remain distinguishable +for this route. A direct Anthropic response that happens to carry the +documented 400/type/message shape is still not a unique machine code. +Preserve the route tag and raw bounded fields for diagnostics, but classify it +as unknown. + +A future Claude Subscription classifier needs a scrubbed, versioned live +fixture from the fixed Anthropic endpoint and current profile. It may enable +automatic recovery only if that fixture contains a unique structured type or +code. A captured message phrase alone is not enough. + +## Required Fugu error contract + +Fugu currently cannot apply even the safe OpenAI rule. Both non-2xx body +decoders flatten error fields into display strings: Anthropic retains only +`type` and `message` +([`anthropic_error_body`](../../src/common/anthropic_stream.c#L317-L350)), +while OpenAI also discards `code` +([`openai_error_body`](../../src/common/openai_stream.c#L537-L567)). +`fugu-api` then converts the response to a terminal text error +([`request_error_done`](../../src/fugu-api/main.c#L1284-L1310)). Stream error +events similarly expose only `type` and `message` +([OpenAI stream decoder](../../src/common/openai_stream.c#L364-L391)). + +Extend that seam with one bounded provider-error record, separate from its +human display text: + +```text +route/dialect +HTTP status (0 only when no HTTP response exists) +phase (pre-stream or stream) +error.type (optional, bounded) +error.code (optional, bounded) +error.message (optional, bounded diagnostics only) +category (unknown or context_overflow) +``` + +Classification belongs beside the hostile provider decoder, before fields are +flattened. The category and response phase then cross `fugu-api` -> +Generation -> Turn. The coordinator, not the codec, enforces the separate +transactional recovery conditions: no accepted content, not a summarization +request, and no overflow retry already consumed. + +The pure classifier should pin these positive cases and near misses: + +```text +openai, HTTP 400, code=context_length_exceeded -> context_overflow +openai, stream error, code=context_length_exceeded -> context_overflow +openai, HTTP 400, type=invalid_request_error only -> unknown +openai, HTTP 413, code absent -> unknown +anthropic, HTTP 400, type=invalid_request_error -> unknown +anthropic, HTTP 413, type=request_too_large -> request_too_large +anthropic, HTTP 200, stop=model_context_window_exceeded -> valid limit stop +claude, HTTP 400, type=invalid_request_error -> unknown +``` + +Also vary and omit every non-discriminator field, include embedded-NUL and +over-cap hostile values, and prove that an unrecognized code preserves today's +terminal failure behavior. Provider-wire fixtures should retain the complete +scrubbed error object so future profile-specific classifications can be added +as exact allow-list entries instead of broadening the defaults. + +## Ticket implication + +The provider-typing portion of automatic compaction can proceed with exact +OpenAI-code recovery and fail-closed Anthropic/Claude behavior. It must not +claim overflow-rejection recovery for all three routes. If parity across all +routes is an acceptance requirement, the work remains blocked on Anthropic +publishing a structured discriminator or on a maintainer explicitly accepting +message-based classification as a weaker policy. This report recommends +neither broad substring matching nor treating 413 as a substitute.