Skip to main content
Oct 23, 2026
Say you start 1,000 coding agents on the same codebase. In the first minute, every one of them clones the repo. Then they read files, run git status, search, edit and run tests, over and over. A few minutes in, the first results come back, and dozens of agents try to save their answer to the same file. The storage most of us use was built for people. A developer opens a few files, thinks for a while, and saves now and then. Agents do the same work much faster, from many machines at once, and you can’t fully trust any of those machines. The title looks far ahead. This post stays with what can be checked today: a swarm of about 1,000 agents. At that size, storage has to do several jobs at once. Each agent needs a private workspace. The swarm needs shared code and a way to pass results around. Work has to survive crashed sandboxes. And agents need to coordinate without blocking each other. Each of these has decent answers on its own. Covering all of them together is the hard part, and that combination is what we mean by an agent drive. We’ve looked at how teams handle this today. This post covers what we found: what people use, where it breaks, what it costs, and a few ideas that look promising.

​
How agents use storage differently

Agents differ from people in a few ways that matter for storage. They’re short-lived and you can’t trust them. A sandbox might run code the agent wrote a minute ago, and it can be paused or killed at any time. You don’t want to give it long-lived passwords or keys. They’re fast. Something a person does once, like listing a folder, an agent loop might do thousands of times an hour. They write at the same time. If one agent is slow or crashes, agents working on other things shouldn’t have to wait for it. You need to know who did what. When a swarm produces a result, you want to know which agent changed which file, and whether anything shared was tampered with. And they run on someone else’s computers. Most agent sandboxes are small virtual machines; E2B, for example, uses Firecracker. Firecracker can’t share a folder from the host machine with the VM, and that isn’t on its roadmap. So anything shared has to travel over the network. All that reading shows up on the bill. S3 and similar services charge for every request, and some file services charge for every byte read. A command that’s free on your laptop shows up on the bill when 1,000 agents run it every minute. Try the numbers below. The first preset assumes the repo sits in a bucket mounted as a folder, with nothing cached, so looking up each of 5,000 files costs two requests. Real setups cache some of those lookups, and real agents don’t run git status every minute. But notice that on both S3 and R2, listing a folder costs over ten times as much as reading a file. When you pay per request, chatty agents cost you money directly.

​
”Shared storage” is really three things

When people ask for shared storage for agents, they usually mean three different things, with a fourth hiding underneath. Each one gets used differently and costs differently, so we’ll take them one at a time.

​
Each agent’s workspace

Sandbox providers already handle much of this. Most can save a sandbox and start new sandboxes from it: These snapshots are often cheaper than they sound. Several products can capture just one directory instead of the whole machine, and many store only what changed since a base image or an earlier snapshot. Firecracker itself has diff snapshots in developer preview, which save only the memory that changed. What’s still missing is portability. A snapshot from one provider can’t be restored on another, because each depends on that provider’s setup; a Firecracker memory snapshot only restores on the same Firecracker version and similar hardware. How long snapshots are kept also depends on each provider, anywhere from three days to forever. Some teams now keep the machine and the work apart. They let the provider handle the machine, and save the agent’s work as a git commit on a private branch, which any sandbox on any provider can fetch. Others describe the environment once (a base image plus repos at fixed versions) and have each provider build its own template from that, so 1,000 agents don’t each download the same tools.

​
Shared folders

The obvious fix is one folder that every sandbox mounts. There are four kinds of options. Run your own shared file system. JuiceFS keeps the list of files in a database and the file contents in object storage, so checking whether a file exists doesn’t cost an S3 request. Fly.io built its Sprites on a modified version of it. CephFS behaves most like a local disk, but to do that it has to keep clients in step, and a stuck client can hold up other clients using the same files for about a minute before it’s flagged. Mount a bucket as a folder. Tools like Mountpoint for S3, s3fs and gcsfuse store each file as an object, so every file operation turns into requests. Mountpoint can’t rename files in a regular bucket or change part of a file, so git doesn’t work on it. With s3fs and similar tools, if two machines write the same file, one of them silently wins. Use a managed file service. EFS in its default Elastic mode charges $0.03 for every GB read and $0.06 for every GB written. It also rounds small operations up: each data read or write counts as at least 32 KiB, and each metadata operation, like looking up a file, as at least 4 KiB. Other services charge for size and speed instead of usage, and slow you down when you go over: EFS in Bursting or Provisioned mode, FSx for OpenZFS, Filestore and Azure Files. Two services launched this year with agents in mind. Amazon S3 Files (April) lets you use an S3 bucket like a network drive. Changes take about a minute to reach the bucket, and if the two disagree, the bucket wins. Google’s Filestore agent volumes (September) give you up to a million separate network drives in one region. Each one attaches to a sandboxed GKE pod in milliseconds, several pods can write to the same drive, and file locks work across machines. It’s only open to approved customers so far, but it shows the big clouds now treat agent storage as its own problem. Use the sandbox platform’s volumes. Their docs list the limits. E2B’s volumes are in private beta: file locks can hang, and a sandbox isn’t told when another one changes a file. Modal saves changes in the background every few seconds and again when a container shuts down, but other containers only see them after they reload the volume, and if two containers write the same file, the last write wins. Daytona says its volumes are “not transactional”. Vercel Drives became generally available in October, with forks that let parallel workers each change their own copy; Vercel’s guide describes one sandbox writing to a drive at a time, with others getting read-only snapshots. All of these run into the same old problem: two writers, one file. There are a few usual ways to handle it:
  • Let the last write win. Most volumes do this.
  • Take a lock. Only agents that need the same file wait, but if the holder gets stuck, they can wait a long time. How long depends on whether the system notices a dead holder and frees the lock.
  • Take a lease, which is a lock that expires. That caps the wait. But a holder that was paused can wake up after its lease ran out and still write, unless the storage checks a lease number and refuses old ones.
  • Save only if unchanged. The save goes through only if the file is still the version the agent read. If it isn’t, the save is refused, and the agent has to read the file again, redo its change and retry.
Step through each below. Locks haven’t gone well with agents. Cursor found that 20 agents sharing a locked task list got about as much done as 2 or 3. There’s more on that below. What seems to work is mostly reading from shared folders. Read-only copies of a saved version are fine, and so is a setup where each path has one owner who writes to it. Folders that anyone can write to, with no rule about who owns what, keep losing data without telling anyone. On the plus side, S3 added conditional writes in 2024, and R2 and Google Cloud Storage support them too, so “save only if unchanged” is cheap to build.

​
Shared code

For code, almost everyone uses git. Most coding agents hand their work back as a commit or a pull request. Codex, Cursor’s cloud agents and Devin each work in their own sandbox and return a branch. Claude Code agent teams share one working directory by default; the docs warn that two teammates editing the same file overwrite each other and suggest giving each teammate its own files, with git worktrees as a separate option. In Anthropic’s C compiler experiment, 16 agents shared one repo and claimed tasks by pushing small lock files. Whoever’s push landed first got the task. The two teams that have written in detail about running agents at this scale both ran into trouble with how the agents shared work. Anthropic’s agents did well while they worked on separate tests. When they all moved on to compiling the Linux kernel, every agent hit the same bug, fixed it, and overwrote the others’ fixes, so 16 agents did no better than one. The fix was to give each agent different files to work on. Later in the project, new commits often broke features that already worked, so every push had to pass a test suite first. Merge conflicts came up often, but the agents resolved them on their own. Cursor tried several setups:
  • Equal agents sharing a task file, with locks. Throughput dropped to that of two or three agents. Agents held locks too long, never released them, crashed while holding them, or edited the file without taking one.
  • Writes that fail if the file has changed. This was more reliable, but the agents started avoiding hard tasks and making small, safe changes.
  • A separate agent that merged everyone’s work. It became a bottleneck and was dropped, since workers could resolve conflicts themselves.
  • Planners that hand out tasks, and workers that each stick to one task. This worked. Hundreds of workers pushed to the same branch with few conflicts.
Git copes with parallel work fairly well. The server receives each push separately and only locks branch names briefly while it updates them. Pushes to different branches can still queue behind each other, though: git’s newer reftable format takes one lock per repository for every branch update, and the older format rewrites a shared file of branch names. Git 2.51 made updating many branches at once faster. Most of the strain lands on the git server.
  • A thousand agents cloning at once is a lot of work for the server, which has to package a copy for each one. Git’s bundle URIs let a clone download most of the history as a plain file from object storage or a CDN, then fetch only what’s new. GitLab supports this. Putting a recent copy of the repo inside the sandbox image avoids most of the download.
  • Swarms create many forks of the same repo. GitLab, GitHub and Bitbucket all store shared history once across forks. Cleaning up that shared history is hard: GitHub has written about how careful it has to be.
  • Sooner or later everyone wants to merge into main. Merge queues line those merges up one at a time. GitLab has merge trains, GitHub has a merge queue, and Gerrit users run Zuul.
  • An agent should only be able to push its own branches, and only while its run lasts. Git hosts give out tokens to users and bots, so this usually needs something extra on top.
Mainstream git hosts don’t publish numbers for this kind of load. Two companies that ran into it have written up what they built. Cursor’s Continuity. This is the storage behind Cursor’s Origin git hosting, described in Git at any scale.
  • It keeps plain git repos on local disks, but treats them as a cache. Each push is first saved to a log in S3-compatible object storage, and it isn’t acknowledged until that save completes.
  • Any machine can rebuild a repo from the log. So there’s no database tracking where each repo lives, and no copy that has to be nursed back to health after a failure.
  • Pushes to the same branch are ordered by a conditional write to the log. A push that loses fetches again, rebases and retries.
  • Cursor measured about 120 pushes per second with S3 Standard underneath, and over 300 with S3 Express One Zone. At that rate, git’s own repacking was the limit.
  • The post contrasts this with GitHub’s older Spokes design, which keeps three full copies of every repo and needs most of them to agree on each push. That gets slower as copies are added, and it’s wasteful for large numbers of small, mostly idle agent repos.
Anthropic’s Scion. Anthropic’s merge rate went from about 2,500 changes a day in May 2026 to over 30,000 by September. Its monorepo has almost 100 GB of history and gets about 25 million CI checkouts a day. Even with several hundred git mirrors, git couldn’t keep up, so a team of seven built its own version control system, Scion, in five months.
  • Commits, rebases and merges happen on the server. There’s nothing to push.
  • A commit lists the files that changed, each pointing to content stored by its hash. No full copy of the repo exists anywhere, not even on the server.
  • Clients fetch only the files they need, and many jobs never check out the repo at all.
  • Branch names are discouraged except for main and release branches. Moving one is a single conditional database update.
  • Merges don’t take a global lock. Tests run after merging and before deploying, and Claude resolves most conflicts.
  • Caches sit on each machine and in each region, and over 99% of reads hit the regional cache. Content never changes, so any cache can serve it.
Scion uses about 200 times less CPU and memory than the git mirrors it replaced, and checkouts take seconds instead of minutes. Anthropic says it only makes sense for very large repos or heavy agent output, that adopting it means replacing code review, code search and CI as well, and it isn’t releasing it. Other git hosting aimed at agents:
  • Cloudflare Artifacts puts each repo in its own Durable Object and charges per operation.
  • Entire was started by GitHub’s former CEO.
At the scale most teams run, the common pattern is still a branch per agent, a merge queue for shared branches, and big downloads served from object storage. Cursor and Anthropic show where it can go next. A log or a database becomes the source of truth, and git servers become caches in front of it. More of the work moves onto the server, so an agent doesn’t need a full clone to read history or commit a change. Google took a similar path years ago, running git on its own storage system with JGit.

​
Results and other outputs

Not everything an agent makes belongs in a repo. Test reports, built binaries, evaluation scores and data files need three things:
  • They shouldn’t change once saved, so nobody reads half a result.
  • You need a way to refer to one specific result.
  • You need to send them to one agent, a group, or everyone.
Most teams keep these in a bucket, or in the orchestrator’s database if they’re small. Two existing kinds of tools come close. Container registries can store any file, not just container images. The OCI standard and the ORAS tool give each upload an ID based on its contents, plus names (tags) that point at it. Harbor, Zot and the cloud registries all support this. But when two clients move the same tag at once, the last one wins. The reference registry updates the tag without checking anything, and the standard leaves the problem to clients. Build caches like Bazel’s are good at moving lots of small files. Their protocol can ask “which of these do you already have?” and upload or download many files in one request. But the servers that use it, like Buildbarn, bazel-remote and BuildBuddy, are caches. They delete old data on purpose, so they can’t hold your only copy. Small files cost money again here. If every small file is its own object, you pay for every upload. Backup tools like restic and kopia, and git itself, bundle many small files into bigger ones. Giving each file an ID based on its contents also means identical files get stored once, which helps when 1,000 agents produce nearly the same output. Git and these tools both name files by a hash of their contents, but they calculate it differently. Git adds a short header before hashing, so the same file gets one ID in git and another in a registry. Git LFS is the exception, since it uses a plain SHA-256. One approach keeps coming up. Save each result once and never change it, give it an ID based on its contents, and add names like best or latest that can only move with a conditional save. Sending a result to another agent then just means giving it permission and telling it the result is there. Nothing gets copied.

​
Coordination

Under all of this is a small amount of information that has to be right: who claimed which task, which result best points to, who can read what, and what changed since an agent last checked. Today it lives in lock files on shared storage, in the orchestrator’s database, or in a key-value store. There’s an old lesson here. A lease that expires doesn’t stop a paused agent from waking up later and writing anyway. The usual fix is to give each new holder a number that goes up every time, and have the storage reject writes that carry an old number. Agents make this more pressing, because their sandboxes get paused, killed and replaced all the time. The databases people reach for don’t all behave well when things fail. In Jepsen’s tests: Agents also need to find out what changed. If they keep listing a folder or bucket to check, you get the kind of traffic that per-request pricing charges for. Kubernetes has the same problem with etcd. It doesn’t let clients watch etcd directly; a cache in between sends changes out to them. A swarm needs something similar. Research points the same way. In the STORM paper, agents sharing one workspace, with each save checked against the version they had read, did better than agents working in separate git worktrees on closely connected code. In AgentRoom, giving agents ways to coordinate mattered more than merging their edits automatically. Both studies used eight agents or fewer. Nobody has published results for a thousand.

​
Where the money goes

Request charges get the attention, but with a swarm, moving data usually costs more. The calculator below compares storage options for a fleet of agents under three made-up workloads. Network charges are easy to miss, so the calculator lets you pick where the sandboxes run:
  • Next to the storage: no transfer charges.
  • Outside AWS: reading storage hosted in AWS is billed as data leaving for the internet.
  • In AWS, behind a NAT gateway: reaching storage outside AWS is charged for every GB that passes through the gateway, in both directions.
A separate switch covers sandboxes spread across availability zones while the storage sits in one. AWS charges $0.01 per GB in each direction for that. A few things show up whichever workload you pick. Paying per GB costs more than paying per request once agents read gigabytes an hour. EFS Elastic is the most expensive option in every scenario. Where things run can matter more than whose storage you buy. E2B’s managed sandboxes run on Google Cloud. If your storage is on AWS, every byte they read is billed as data leaving AWS. R2 doesn’t charge for data leaving it, but it still charges per request. Storage with “free requests” has fine print. Backblaze B2 charges for downloads beyond three times what you store. Wasabi expects you to download less than you store and bills deleted files for 90 days. Both suit backups, not files agents read all day. Paying for a fixed size and speed works like a flat fee, up to a point. Provisioned services and your own servers slow down rather than charge more, but each has a ceiling: one FSx for OpenZFS file system tops out at 10,240 MBps, and one Azure Files share at 102,400 IOPS. Past that, you split the data across several. Azure also caps a share’s speed relative to its size, so a fast share sometimes has to be bigger than the data in it. Caching is easiest on files that never change after they’re saved. Copies of those never go stale, so a cache can keep them indefinitely, and the number of requests reaching storage depends on how much unique data there is rather than on how often agents read it. Files that do change can be cached too, but then every cache has to check whether its copy is still current, or be told when it isn’t. The workloads are guesses, not measurements, so put in your own numbers. The overall shape of the bill matters more than any one figure.

​
What’s still unsolved

  • Merging. It’s easy to copy a sandbox, a volume or a bucket. Combining two copies back into one is harder. Vercel’s drives can be forked, but the forks are independent copies. Tools exist for some formats: git merges text, Dolt merges database tables, nbdime merges Jupyter notebooks, and CRDT libraries like Automerge merge JSON-like documents. Nothing covers the whole mix of files a swarm produces.
  • Cleaning up after crashes. Agents crash while holding locks, halfway through uploads, or before giving up a task. Most systems either wait for a timeout or need a person to step in.
  • Bad shared data. An agent fooled by a prompt injection can write instructions or bad data that other agents then trust. Anthropic’s docs for Claude Managed Agents memory warn about this for shared memory. Recording who wrote what, read-only access and review steps help, but there’s no standard answer yet.
  • Moving between providers. Snapshots, volumes and access keys are all tied to one sandbox provider. A swarm that runs on several has to keep its storage somewhere neutral.
  • Limits on sandbox count. E2B’s Pro plan allows 100 sandboxes at once by default, so 1,000 agents means a bigger plan. Other platforms have similar limits.
  • Real data at scale. Published studies of agents working together use 2 to 16 agents. Anything said about 1,000 agents, this post included, is an educated guess.

​
Products built for agents

A few products now sell storage made for agents. Two come up often. Archil offers disks backed by S3 or another bucket that several machines can mount. In shared mode, a machine “checks out” a folder to write to it, and everyone else can still read it. A machine that crashes keeps its folders checked out until someone forces a takeover, which throws away its unfinished writes. A third mode lets every machine write anywhere, but the docs don’t say what happens when two write the same file. Archil also charges for data sent to compute outside Archil, which matters when sandboxes run elsewhere. Mesa offers versioned folders that agents mount (on Linux, through FUSE) or open from an SDK. Each folder has a history of snapshots and named branches. Mounts on the same branch see each other’s edits within seconds, and when two agents edit the same file in the same second, Mesa keeps both versions, with conflict markers in the file. Mesa says it is built for thousands of agents reading and writing at once. Both cover parts of the combination from the start of this post: private workspaces and shared files. Their docs leave the rest unclear:
  • neither publishes tests with hundreds or thousands of agents writing at once;
  • neither describes task claiming or a feed of changes for coordination;
  • sending a result to one agent or a group comes down to access settings you configure yourself;
  • recovery after a crash is documented for Archil and not for Mesa;
  • both need FUSE inside the sandbox, or Mesa’s SDK, which rules out some sandboxes.
Some of these may be handled but not written down. We haven’t tested either product.

​
A field that’s changing fast

Most of the products in this post didn’t exist eighteen months ago. Sandbox companies are adding volumes and snapshots. AWS and Google are shipping file systems meant for agents. Git hosting is being rebuilt for machines instead of people. Startups are selling file systems for agents, and researchers have started measuring how agents share work. We haven’t found a product that does all of it together and shows it working at a thousand agents: private workspaces that move between providers, shared git, results you can hand to other agents, recovery from crashed sandboxes, and coordination that doesn’t block, at a price that doesn’t climb with every file read. Most of the parts exist, and they change month to month. If you run large groups of agents and have hit any of this, we’d like to hear how you dealt with it.