Choosing a runtime for LLM agents

Frameworks help you start. Runtimes keep you alive. Choose with an exit in mind: your prompts, tools, and traces should outlive the library.

Must-haves

• Streaming tokens to the user without blocking tools
• Sandbox or allow-list for tool execution
• Structured logs you can ship to your own store
• Per-agent concurrency and spend caps

Nice-to-haves that become traps

A visual graph editor is fine until the graph cannot be reviewed in git. A vendor memory store is fine until you cannot export it. Prefer boring interfaces: HTTP, queues, files, and SQL.

If leaving the runtime requires rewriting the agent, you did not choose a runtime. You chose a host who holds the keys.