AWS Infinity lets AI agents wait without burning compute

A coding agent may start a build that runs twenty minutes, a triage agent may wait hours for the next pull request, and an assistant may wait until morning for an approval. Most agent harnesses run the agent loop as blocking steps: call the model, invoke a tool, wait for the result, repeat. While the agent waits, it holds a process and its memory, and it cannot respond to events that arrive in the meantime.

Infinity, released October 23, 2026 under the Apache 2.0 license as part of the Hydro project, a collaboration between AWS and UC Berkeley, is designed so that a waiting agent does not hold a process. It is an early research release shared for feedback, not a finished production product.

Infinity borrows the technique programming languages use for async functions. At a tool call, the agent's conversation history already contains everything needed to continue, so the runtime persists that history, dispatches the call, and suspends the agent. Each run is a bounded slice: load the history, run one model completion, persist the output, dispatch at most one tool call, and end. Nothing in a slice waits on an operation of unbounded duration.

Events that arrive while an agent waits are delivered as synthetic tool calls, named receive_event__injected, so the model can tell an injected event from a call it actually made. The same mechanism carries messages between parent and child agents: spawn_thread creates a child, and report_to_parent delivers its reports as events. By default each event is handled in a separate child thread, so a flood of events does not bloat the parent's context window.

AWS says 75,000 concurrent agents fit in the 8 GB of a Raspberry Pi 5. That figure is arithmetic, not a Pi test. The benchmark ran 80,000 agents, for 3.2 million completions, on a single thread of a server-class AMD EPYC 9R14 with in-memory stores and a simulated model. Each agent ran twenty turns built from streamed completions of about 1.2 KB, and resident memory grew about 100 KB per idle agent, mostly conversation history.

Dividing 8 GB by about 100 KB gives roughly 80,000; the project site gives the measured figure as about 103 KB per agent, with 80,000 agents using 8.4 GB, so about 75,000 fit within 8 GB. The measurement is the cost of remembering a conversation under those test conditions, not 75,000 simultaneous model inferences. An independent analysis (alfw.ai) notes that real agents accumulate tool outputs, file contents, diffs, and error logs, which would likely push the per-agent figure above 100 KB. The measured quantity is the rate: about 100 KB per idle agent in a simulated workload.

Because a slice never waits, it fits inside a serverless invocation that exits when the step is done. AWS's reference deployment uses an SQS FIFO queue with one message group per thread, a Lambda function per step, Aurora DSQL for conversation histories, and DynamoDB for deduplication. An agent waiting on a three-day CI pipeline uses no compute until the pipeline reports back.

Waiting is not free in every sense: alfw.ai's analysis notes that stored history, queue and timer infrastructure, and the model calls on resume can still incur costs, and AWS's post gives no dollar-denominated cost comparison. The authors also leave four questions open: how models reason about interleaved events, what scheduling priorities should be, how compaction should work across large trees of agents, and how a runtime should establish that an incoming event is authentic. AWS describes this as an early research release, and the post reports no production results.