Sidecars

An agent gets tools into its loop, and an MCP server makes them discoverable. Neither answers the same question: who actually does the work? Who parses the PDF, who renders the video, who transcribes the audio? In almost every serious setup the answer has the same name - a sidecar.

How much rides on that question shows in a real session. The session viewer has one ready - press "Load example session"; the view is open to Pro members - and its growth curve has three steps next to which everything else looks small: three curl calls, each tipping a complete HTML document into the context. Together they account for 52,489 of the transcript's 120,547 characters, 43.5%. The single most expensive message holds 25,400 characters and consists of three complete error pages - "404: This page could not be found", three times over. So alongside the question of who does the work stands a second one, immediately: what comes back?

A sidecar is a service with one capability and no memory

A sidecar is a separate process - in practice a separate container - that runs next to the agent and does exactly one thing. It takes a request, does the work, returns the result, and then forgets that the request ever happened.

Request in, result out, no memory between two calls.

That is the whole definition, and the second half is the important part. Stateless means the second call knows nothing about the first. There is no session, no account, no progress lingering in memory between requests. The same input produces the same output - today, tomorrow, and on a freshly started container.

That single property buys you everything that follows:

• Restarting is a non-event. A service without state can die at any moment; it comes back up and is immediately itself again. There is nothing to migrate and nothing to restore.
• Scaling is trivial. Three instances of the same sidecar are interchangeable. Which one gets a given request does not matter - there is no "right" instance holding the context.
• Testing is honest. A curl against the endpoint proves exactly what the agent would prove. No warm-up, no preconditions, no "but on my machine something else had run first".

The boundaries: memory has state, MCP is the protocol

Two neighbours sit close enough that they get confused with sidecars all the time.

Versus memory (step 7): a memory service is the opposite design - it has state on purpose. Its entire point is that tomorrow's call knows something about today's. That is why it has a database, why it must be backed up, migrated and made deletable, why it is a genuine second system with operations of its own. A sidecar has none of that. If you are about to give your sidecar a database, either it stopped being a sidecar or you cut the problem in the wrong place.

Versus MCP (step 8): MCP is the socket, the sidecar is the appliance. An MCP server describes tools and forwards calls; it is a thin layer, often a few hundred lines. What actually happens behind the tool parse_pdf is not part of the protocol. Only together do they make a tool: MCP makes the capability discoverable and callable, the sidecar provides it. And the mapping is not one to one - one MCP server can front several sidecars, and a sidecar can be called directly over HTTP by an agent that has no MCP at all.

Why a container instead of just a function?

The obvious question: if the thing is stateless anyway - why not a library inside the agent process, import and done? Four reasons, and any one of them is enough.

1. A foreign runtime. Your agent is written in TypeScript, the best document extraction is Python, the best transcription is C++ with CUDA. A sidecar solves that by turning the language boundary into a network boundary. This project hits that case twice: the Docling sidecar (infra/sidecars/docling) is Python next to a Next.js application, and the render sidecar (video/render-server.mjs) runs on node:22-bookworm-slim because Remotion's headless Chrome shell is bound to glibc while the rest of the stack is Alpine. Not "would be nicer" - otherwise it simply does not run.

2. Foreign system dependencies. A browser, a GPU runtime, ffmpeg, fonts, a model with two gigabytes of weights. All of that inside the agent image means every agent deploy drags it along, every security update touches both, and the image that should have been 200 MB is 3 GB.

3. Independent scaling. The agent is mostly idle, waiting for model responses. The render service eats four cores for twelve minutes. Separated, you can multiply one and not the other - or, as in this project, push the expensive part onto a different machine entirely so the web server does not collapse under a video.

4. Independent failure. A memory leak in the PDF parser takes down the process it runs in. Inside the agent, that means the agent is gone - along with every conversation currently open. Inside a sidecar, the agent gets an error back and says "I could not read that document". That is the difference between a glitch and an outage.

What comes back: specialisation saves context

The four reasons above justify the process boundary. They say nothing about what travels back across it - and in practice that is where the second gain sits.

Back to the example session. The agent there has no tool for "read me this page"; it has a terminal, and with it it calls curl. What comes back is everything the server sent: doctype, meta tags, stylesheet links, the full Next.js RSC payload - and somewhere inside it the one line it was after. Two of those responses consist of nothing but error pages and together account for 33,151 characters, a good quarter of the entire transcript. You pay for that three times over: once on retrieval, once in every further model call that ships the transcript again, and once in attention - the more filler sits in the window, the harder the model finds what matters in it.

A sidecar for web access inverts that. The service fetches the page, processes it and returns the result: the extracted text, the three table rows, the price it read out - or simply "404". The raw material stays where it was processed; the agent's context never sees the 25,400 characters. That is the practical meaning of "does exactly one thing": whatever specialises in a capability also knows what is interesting about its result. And the saving is not cosmetic - it decides whether, after ten retrievals, the agent still knows what it was looking for.

The counter-check, stated honestly: on its own this argument does not justify a sidecar. A function in the same process could extract just as well and return just as little - saving context is a question of division of labour, not of the process boundary. It only becomes a sidecar once one of the four reasons above applies as well. For web access that is not a hypothetical condition.

The case where all four reasons land at once: web browsing

There is one sidecar for which not one of the four reasons applies but all four at the same time - and that is why it is the running example of this part: an agent that is supposed to act on the web, not just read it. Click, type, fill in a form, work its way through a multi-page flow.

That requires a real browser, and with it everything at once: a foreign runtime (a Chromium plus system fonts, roughly a gigabyte of image), foreign system dependencies, independent scaling (every open session occupies 300-500 MB of RAM while the agent next to it merely waits) and independent failure (a page that hangs the browser must not take the conversation with it).

On top of that comes something none of the other capabilities demands: the service needs a model of its own alongside yours, because "click through to the invoice" is itself a loop of looking, deciding and clicking. That makes browsing the heaviest sidecar there is - and the case in which every trade-off shows itself that the simple families let you skip. Who runs the loop and what that costs is in the next tier; follow-up questions, the security surface and the question of how you can even tell where a third-party package sends data are in the deep dive.

The price, stated honestly

A sidecar is not free. You take on: another container in the compose file, a network boundary with latency, serialisation in both directions, another service that has to start on deploy, and a second place where things fail without a stack trace connecting the two.

Hence the rule of thumb: a sidecar pays off as soon as one of the four reasons above applies - and not before. A function that turns Markdown into HTML stays a function. A function that would have to start a browser becomes a sidecar.

A sidecar is a stateless service next to the agent that offers exactly one capability: request in, result out, no memory between two calls.

That is precisely what separates it from its neighbours: memory has state on purpose, MCP is only the protocol in front. The sidecar is the appliance behind the socket.

Its own container beats a function when at least one of four reasons applies: foreign runtime, foreign system dependencies, independent scaling, independent failure. If none applies, write a function.