Unrolling the Codex agent loop

Key Actionable Insights

1

Structure your prompts with static content first and variable content last to maximize prompt cache hits. Place instructions, tool definitions, and system messages at the beginning, and put user-specific or changing content at the end. This ensures exact prefix matching works for caching.

The article demonstrates that prompt caching transforms quadratic sampling costs into linear costs. Codex specifically designs its prompt ordering (system → tools → instructions → developer messages → user context → user message) to maximize cache reuse across inference calls.

2

When building an agent loop, ensure new prompts are exact prefix extensions of previous prompts rather than modified versions. Append tool call results and new messages to the end of the input array rather than inserting or modifying items in the middle of the conversation history.

Codex explicitly maintains this prefix property across all conversation turns. When configuration changes occur mid-conversation (like working directory changes), Codex appends new messages with updated information rather than modifying the original messages, specifically to preserve cache hits.

3

Implement automatic conversation compaction to prevent context window exhaustion in long-running agent sessions. Set a token threshold that triggers compaction before hitting the hard context limit, and use the compacted representation as a drop-in replacement for the full conversation history.

Codex evolved from manual /compact commands to automatic compaction using the /responses/compact endpoint when the auto_compact_limit is exceeded. The compacted output includes encrypted_content that preserves the model's latent understanding of the original conversation.

4

Keep your Responses API requests fully stateless by including the complete conversation history in each request rather than relying on server-side state via previous_response_id. This simplifies the architecture, improves reliability, and supports Zero Data Retention configurations.

While this approach means quadratic JSON payload growth, prompt caching makes the actual model sampling cost linear. The trade-off favors statelessness because network traffic cost is dominated by sampling cost, and stateless requests are simpler to debug and operate.

5

When integrating MCP tools into an agent loop, enumerate tools in a consistent order across requests and be cautious about honoring tools/list_changed notifications mid-conversation, as they will cause expensive prompt cache misses.

Codex encountered a bug where MCP tools were listed in inconsistent order, causing cache misses. MCP servers can dynamically change their tool list, and honoring these changes mid-conversation invalidates the entire prompt cache for that session.

6

Design your agent's tool definitions to be modular, separating built-in tools, API-provided tools, and user-configured tools. Only sandbox the tools you control directly; external tools like MCP servers should enforce their own security guardrails.

Codex's sandboxing applies only to its built-in shell tool, not to MCP server tools. This separation of concerns allows flexibility while maintaining security for the tools the agent directly controls.