> ## Documentation Index > Fetch the complete documentation index at: https://tensorfuse.io/docs/llms.txt > Use this file to discover all available pages before exploring further. # Agent Drive: Storage for the next Trillion agents > What a swarm of 1,000 AI agents needs from storage (private workspaces, shared code, results, recovery and coordination), what teams use today, where it breaks, and what it costs. Oct 23, 2026![Saurabh Singh profile](https://ca.slack-edge.com/T028V4XRP-U8SP74Q66-f564c63ae63f-512)Saurabh SinghFounding EngineerSay you start 1,000 coding agents on the same codebase. In the first minute, every one of them clones the repo. Then they read files, run `git status`, search, edit and run tests, over and over. A few minutes in, the first results come back, and dozens of agents try to save their answer to the same file.The storage most of us use was built for people. A developer opens a few files, thinks for a while, and saves now and then. Agents do the same work much faster, from many machines at once, and you can't fully trust any of those machines.The title looks far ahead. This post stays with what can be checked today: a swarm of about 1,000 agents. At that size, storage has to do several jobs at once. Each agent needs a private workspace. The swarm needs shared code and a way to pass results around. Work has to survive crashed sandboxes. And agents need to coordinate without blocking each other. Each of these has decent answers on its own. Covering all of them together is the hard part, and that combination is what we mean by an **agent drive**.We've looked at how teams handle this today. This post covers what we found: what people use, where it breaks, what it costs, and a few ideas that look promising.```mermaid theme={null} flowchart LR subgraph H["A developer"] direction TB H1["One laptop"] --> H2["Opens a few dozen files an hour"] H2 --> H3["Pushes a few times a day"] end subgraph S["A swarm of agents"] direction TB S1["1,000 short-lived sandboxes"] --> S2["Each reads thousands of files an hour"] S2 --> S3["Hundreds try to save to the same place"] end H ~~~ S ```## How agents use storage differentlyAgents differ from people in a few ways that matter for storage.They're short-lived and you can't trust them. A sandbox might run code the agent wrote a minute ago, and it can be paused or killed at any time. You don't want to give it long-lived passwords or keys.They're fast. Something a person does once, like listing a folder, an agent loop might do thousands of times an hour.They write at the same time. If one agent is slow or crashes, agents working on other things shouldn't have to wait for it.You need to know who did what. When a swarm produces a result, you want to know which agent changed which file, and whether anything shared was tampered with.And they run on someone else's computers. Most agent sandboxes are small virtual machines; E2B, for example, uses Firecracker. Firecracker can't share a folder from the host machine with the VM, and [that isn't on its roadmap](https://github.com/firecracker-microvm/firecracker/issues/1180). So anything shared has to travel over the network.All that reading shows up on the bill. S3 and similar services charge for every request, and some file services charge for every byte read. A command that's free on your laptop shows up on the bill when 1,000 agents run it every minute. Try the numbers below. The first preset assumes the repo sits in a bucket mounted as a folder, with nothing cached, so looking up each of 5,000 files costs two requests.AgentsStorage calls per agent per minuteShare that are listings%The other calls are file lookups small file readsstorage calls a month ( , listings), running 730 hours.Request charges only, at list prices on October 22, 2026. Storage and data transfer are not included. On S3 and R2, a lookup is a HEAD request, a small read is a GET and a listing is a LIST. The git status preset assumes a 5,000-file repo in a bucket mounted as a folder with nothing cached, so each file costs one lookup and one listing. Real setups cache some of these.Real setups cache some of those lookups, and real agents don't run `git status` every minute. But notice that on both S3 and R2, listing a folder costs over ten times as much as reading a file. When you pay per request, chatty agents cost you money directly.## "Shared storage" is really three thingsWhen people ask for shared storage for agents, they usually mean three different things, with a fourth hiding underneath.```mermaid theme={null} flowchart TB W["Each agent's workspace
checked-out code, dependencies, build output"] C["Shared code
the repos the whole swarm works on"] A["Results
reports, binaries, scores, data handed to other agents"] K["Coordination
who's working on what, what changed, who can see what"] W -- "save work" --> C W -- "publish" --> A C -.- K A -.- K ```Each one gets used differently and costs differently, so we'll take them one at a time.## Each agent's workspaceSandbox providers already handle much of this. Most can save a sandbox and start new sandboxes from it:- E2B can [pause, snapshot and fork](https://docs.e2b.dev/sandbox/persistence) sandboxes, memory included. - Modal has [filesystem, directory and memory snapshots](https://modal.com/docs/guide/sandbox-snapshots). - Daytona has [snapshots and archiving](https://www.daytona.io/docs/en/snapshots). - Vercel made [saved sandboxes generally available](https://vercel.com/changelog/sandbox-persistence-is-now-ga) in May 2026. - Cloudflare's Sandbox SDK can [back up a single directory to R2](https://developers.cloudflare.com/sandbox/api/backups/). - Google's GKE Agent Sandbox has [Pod Snapshots](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/agent-sandbox-pod-snapshots) in preview. - AWS added [session storage to Bedrock AgentCore](https://aws.amazon.com/about-aws/whats-new/2026/03/bedrock-agentcore-runtime-session-storage) in March 2026.These snapshots are often cheaper than they sound. Several products can capture just one directory instead of the whole machine, and many store only what changed since a base image or an earlier snapshot. Firecracker itself has [diff snapshots](https://github.com/firecracker-microvm/firecracker/blob/main/docs/snapshotting/snapshot-support.md) in developer preview, which save only the memory that changed.What's still missing is portability. A snapshot from one provider can't be restored on another, because each depends on that provider's setup; a Firecracker memory snapshot only restores on the same Firecracker version and similar hardware. How long snapshots are kept also depends on each provider, anywhere from three days to forever.Some teams now keep the machine and the work apart. They let the provider handle the machine, and save the agent's work as a git commit on a private branch, which any sandbox on any provider can fetch. Others describe the environment once (a base image plus repos at fixed versions) and have each provider build its own template from that, so 1,000 agents don't each download the same tools.## Shared foldersThe obvious fix is one folder that every sandbox mounts. There are four kinds of options.```mermaid theme={null} flowchart TB subgraph P["Run your own"] P1["JuiceFS"] P2["CephFS"] P3["NFS server"] end subgraph F["Mount a bucket as a folder"] F1["Mountpoint for S3"] F2["s3fs, gcsfuse, rclone"] end subgraph M["Managed file services"] M1["EFS, FSx for OpenZFS"] M2["Filestore,
Filestore agent volumes"] M3["Azure Files"] M4["Amazon S3 Files"] end subgraph V["Sandbox platform volumes"] V1["E2B volumes (beta)"] V2["Modal Volumes"] V3["Daytona volumes"] V4["Vercel Drives"] end ```**Run your own shared file system.** [JuiceFS](https://juicefs.com/docs/community/internals/io_processing) keeps the list of files in a database and the file contents in object storage, so checking whether a file exists doesn't cost an S3 request. Fly.io built its [Sprites](https://fly.io/blog/design-and-implementation/) on a modified version of it. [CephFS](https://docs.ceph.com/en/latest/cephfs/) behaves most like a local disk, but to do that it has to keep clients in step, and a stuck client can [hold up other clients using the same files](https://docs.ceph.com/en/reef/cephfs/health-messages/) for about a minute before it's flagged.**Mount a bucket as a folder.** Tools like [Mountpoint for S3](https://github.com/awslabs/mountpoint-s3/blob/main/doc/SEMANTICS.md), s3fs and gcsfuse store each file as an object, so every file operation turns into requests. Mountpoint can't rename files in a regular bucket or change part of a file, so git doesn't work on it. With s3fs and similar tools, if two machines write the same file, one of them silently wins.**Use a managed file service.** [EFS](https://aws.amazon.com/efs/pricing/) in its default Elastic mode charges $0.03 for every GB read and $0.06 for every GB written. It also [rounds small operations up](https://docs.aws.amazon.com/efs/latest/ug/metered-sizes.html): each data read or write counts as at least 32 KiB, and each metadata operation, like looking up a file, as at least 4 KiB. Other services charge for size and speed instead of usage, and slow you down when you go over: EFS in Bursting or Provisioned mode, [FSx for OpenZFS](https://aws.amazon.com/fsx/openzfs/pricing/), [Filestore](https://cloud.google.com/filestore/pricing) and [Azure Files](https://learn.microsoft.com/en-us/azure/storage/files/understanding-billing).Two services launched this year with agents in mind. [Amazon S3 Files](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-files-best-practices.html) (April) lets you use an S3 bucket like a network drive. Changes take about a minute to reach the bucket, and if the two disagree, the bucket wins. [Google's Filestore agent volumes](https://cloud.google.com/blog/products/storage-data-transfer/filestore-agent-volumes) (September) give you up to a million separate network drives in one region. Each one attaches to a sandboxed GKE pod in milliseconds, several pods can write to the same drive, and file locks work across machines. It's [only open to approved customers](https://docs.cloud.google.com/filestore/docs/agent-volumes-overview) so far, but it shows the big clouds now treat agent storage as its own problem.**Use the sandbox platform's volumes.** Their docs list the limits. [E2B's volumes](https://docs.e2b.dev/faq/volumes-beta-limitations) are in private beta: file locks can hang, and a sandbox isn't told when another one changes a file. [Modal](https://modal.com/docs/guide/volumes) saves changes in the background every few seconds and again when a container shuts down, but other containers only see them after they reload the volume, and if two containers write the same file, the last write wins. [Daytona](https://www.daytona.io/docs/volumes) says its volumes are "not transactional". [Vercel Drives](https://vercel.com/changelog/vercel-drives-is-now-generally-available) became generally available in October, with forks that let parallel workers each change their own copy; Vercel's [guide](https://vercel.com/kb/guide/vercel-drives) describes one sandbox writing to a drive at a time, with others getting read-only snapshots.All of these run into the same old problem: two writers, one file.```mermaid theme={null} sequenceDiagram participant A as Agent A participant F as best.json participant B as Agent B A->>F: read version 1 B->>F: read version 1 A->>F: save A's result B->>F: save B's result, based on version 1 Note over F: A's result is gone.
Both saves reported success. ```There are a few usual ways to handle it:- **Let the last write win.** Most volumes do this. - **Take a lock.** Only agents that need the same file wait, but if the holder gets stuck, they can wait a long time. How long depends on whether the system notices a dead holder and frees the lock. - **Take a lease,** which is a lock that expires. That caps the wait. But a holder that was paused can wake up after its lease ran out and still write, unless the storage checks a lease number and refuses old ones. - **Save only if unchanged.** The save goes through only if the file is still the version the agent read. If it isn't, the save is refused, and the agent has to read the file again, redo its change and retry.Step through each below. \ \ \ \ \
tAgent AAgent Bbest.json
Back Next step Step of Locks haven't gone well with agents. Cursor found that 20 agents sharing a locked task list [got about as much done as 2 or 3](https://cursor.com/blog/scaling-agents). There's more on that [below](#shared-code).What seems to work is mostly reading from shared folders. Read-only copies of a saved version are fine, and so is a setup where each path has one owner who writes to it. Folders that anyone can write to, with no rule about who owns what, keep losing data without telling anyone. On the plus side, S3 [added conditional writes](https://aws.amazon.com/about-aws/whats-new/2024/11/amazon-s3-functionality-conditional-writes) in 2024, and R2 and Google Cloud Storage support them too, so "save only if unchanged" is cheap to build.## Shared codeFor code, almost everyone uses git. Most coding agents hand their work back as a commit or a pull request. Codex, Cursor's cloud agents and Devin each work in their own sandbox and return a branch. [Claude Code agent teams](https://code.claude.com/docs/en/agent-teams) share one working directory by default; the docs warn that two teammates editing the same file overwrite each other and suggest giving each teammate its own files, with [git worktrees](https://code.claude.com/docs/en/worktrees) as a separate option. In Anthropic's [C compiler experiment](https://anthropic.com/engineering/building-c-compiler), 16 agents shared one repo and claimed tasks by pushing small lock files. Whoever's push landed first got the task.The two teams that have written in detail about running agents at this scale both ran into trouble with how the agents shared work. Anthropic's agents did well while they worked on separate tests. When they all moved on to compiling the Linux kernel, every agent hit the same bug, fixed it, and overwrote the others' fixes, so 16 agents did no better than one. The fix was to give each agent different files to work on. Later in the project, new commits often broke features that already worked, so every push had to pass a test suite first. Merge conflicts came up often, but the agents resolved them on their own.[Cursor](https://cursor.com/blog/scaling-agents) tried several setups:- **Equal agents sharing a task file, with locks.** Throughput dropped to that of two or three agents. Agents held locks too long, never released them, crashed while holding them, or edited the file without taking one. - **Writes that fail if the file has changed.** This was more reliable, but the agents started avoiding hard tasks and making small, safe changes. - **A separate agent that merged everyone's work.** It became a bottleneck and was dropped, since workers could resolve conflicts themselves. - **Planners that hand out tasks, and workers that each stick to one task.** This worked. Hundreds of workers pushed to the same branch with few conflicts.Git copes with parallel work fairly well. The server receives each push separately and only locks branch names briefly while it updates them. Pushes to different branches can still queue behind each other, though: git's newer [reftable](https://about.gitlab.com/blog/a-beginners-guide-to-the-git-reftable-format/) format takes one lock per repository for every branch update, and the older format rewrites a shared file of branch names. Git [2.51](https://about.gitlab.com/blog/what-s-new-in-git-2-51-0/) made updating many branches at once faster. Most of the strain lands on the git server.```mermaid theme={null} flowchart LR subgraph Cold["1,000 agents start"] A1["agent"] & A2["agent"] & A3["… ×1,000"] end Cold -- "each asks for a full clone" --> G["Git server
packs a copy for every agent"] Cold -. "or downloads a ready-made bundle" .-> CDN["Bundle file on a CDN"] CDN -.-> G2["Git server only sends
what changed since"] ```- A thousand agents cloning at once is a lot of work for the server, which has to package a copy for each one. Git's [bundle URIs](https://git-scm.com/docs/bundle-uri) let a clone download most of the history as a plain file from object storage or a CDN, then fetch only what's new. GitLab [supports this](https://docs.gitlab.com/administration/gitaly/bundle_uris/). Putting a recent copy of the repo inside the sandbox image avoids most of the download. - Swarms create many forks of the same repo. GitLab, GitHub and Bitbucket all [store shared history once](https://docs.gitlab.com/development/git_object_deduplication/) across forks. Cleaning up that shared history is hard: GitHub has written about [how careful it has to be](https://github.blog/engineering/architecture-optimization/scaling-gits-garbage-collection/). - Sooner or later everyone wants to merge into `main`. Merge queues line those merges up one at a time. GitLab has merge trains, GitHub has a [merge queue](https://github.blog/engineering/engineering-principles/how-github-uses-merge-queue-to-ship-hundreds-of-changes-every-day/), and Gerrit users run Zuul. - An agent should only be able to push its own branches, and only while its run lasts. Git hosts give out tokens to users and bots, so this usually needs something extra on top.Mainstream git hosts don't publish numbers for this kind of load. Two companies that ran into it have written up what they built.**Cursor's Continuity.** This is the storage behind Cursor's Origin git hosting, described in [Git at any scale](https://cursor.com/blog/git-at-any-scale).- It keeps plain git repos on local disks, but treats them as a cache. Each push is first saved to a log in S3-compatible object storage, and it isn't acknowledged until that save completes. - Any machine can rebuild a repo from the log. So there's no database tracking where each repo lives, and no copy that has to be nursed back to health after a failure. - Pushes to the same branch are ordered by a conditional write to the log. A push that loses fetches again, rebases and retries. - Cursor measured about 120 pushes per second with S3 Standard underneath, and over 300 with S3 Express One Zone. At that rate, git's own repacking was the limit. - The post contrasts this with GitHub's older Spokes design, which keeps three full copies of every repo and needs most of them to agree on each push. That gets slower as copies are added, and it's wasteful for large numbers of small, mostly idle agent repos.**Anthropic's Scion.** Anthropic's merge rate went from about 2,500 changes a day in May 2026 to over 30,000 by September. Its monorepo has almost 100 GB of history and gets about 25 million CI checkouts a day. Even with several hundred git mirrors, [git couldn't keep up](https://www.anthropic.com/research/redesigning-source-control), so a team of seven built its own version control system, Scion, in five months.- Commits, rebases and merges happen on the server. There's nothing to push. - A commit lists the files that changed, each pointing to content stored by its hash. No full copy of the repo exists anywhere, not even on the server. - Clients fetch only the files they need, and many jobs never check out the repo at all. - Branch names are discouraged except for `main` and release branches. Moving one is a single conditional database update. - Merges don't take a global lock. Tests run after merging and before deploying, and Claude resolves most conflicts. - Caches sit on each machine and in each region, and over 99% of reads hit the regional cache. Content never changes, so any cache can serve it.Scion uses about 200 times less CPU and memory than the git mirrors it replaced, and checkouts take seconds instead of minutes. Anthropic says it only makes sense for very large repos or heavy agent output, that adopting it means replacing code review, code search and CI as well, and it isn't releasing it.Other git hosting aimed at agents:- [Cloudflare Artifacts](https://blog.cloudflare.com/artifacts-git-for-agents-beta/) puts each repo in its own Durable Object and charges per operation. - [Entire](https://www.theregister.com/a/5268694) was started by GitHub's former CEO.At the scale most teams run, the common pattern is still a branch per agent, a merge queue for shared branches, and big downloads served from object storage. Cursor and Anthropic show where it can go next. A log or a database becomes the source of truth, and git servers become caches in front of it. More of the work moves onto the server, so an agent doesn't need a full clone to read history or commit a change. Google took a similar path years ago, running git on its own storage system with [JGit](https://groups.google.com/d/msg/repo-discuss/IekVPmow0yE/BCCMXFy8CwAJ).## Results and other outputsNot everything an agent makes belongs in a repo. Test reports, built binaries, evaluation scores and data files need three things:- They shouldn't change once saved, so nobody reads half a result. - You need a way to refer to one specific result. - You need to send them to one agent, a group, or everyone.Most teams keep these in a bucket, or in the orchestrator's database if they're small. Two existing kinds of tools come close.Container registries can store any file, not just container images. The [OCI standard](https://github.com/opencontainers/distribution-spec/blob/main/spec.md) and the [ORAS](https://oras.land/docs/how_to_guides/pushing_and_pulling) tool give each upload an ID based on its contents, plus names (tags) that point at it. Harbor, Zot and the cloud registries all support this. But when two clients move the same tag at once, the last one wins. The reference registry [updates the tag without checking anything](https://github.com/distribution/distribution/blob/main/registry/handlers/manifests.go), and the standard leaves the problem to clients.Build caches like Bazel's are good at moving lots of small files. Their [protocol](https://github.com/bazelbuild/remote-apis) can ask "which of these do you already have?" and upload or download many files in one request. But the servers that use it, like Buildbarn, bazel-remote and BuildBuddy, are caches. They [delete old data on purpose](https://github.com/buildbarn/bb-adrs/blob/main/0002-storage.md), so they can't hold your only copy.Small files cost money again here. If every small file is its own object, you pay for every upload. Backup tools like restic and kopia, and git itself, bundle many small files into bigger ones. Giving each file an ID based on its contents also means identical files get stored once, which helps when 1,000 agents produce nearly the same output.Git and these tools both name files by a hash of their contents, but they calculate it differently. Git adds a short header before hashing, so the same file gets one ID in git and another in a registry. Git LFS is the exception, since it uses a plain SHA-256.One approach keeps coming up. Save each result once and never change it, give it an ID based on its contents, and add names like `best` or `latest` that can only move with a conditional save. Sending a result to another agent then just means giving it permission and telling it the result is there. Nothing gets copied.## CoordinationUnder all of this is a small amount of information that has to be right: who claimed which task, which result `best` points to, who can read what, and what changed since an agent last checked. Today it lives in lock files on shared storage, in the orchestrator's database, or in a key-value store.There's an old lesson here. A lease that expires doesn't stop a paused agent from waking up later and writing anyway. The [usual fix](https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html) is to give each new holder a number that goes up every time, and have the storage reject writes that carry an old number. Agents make this more pressing, because their sandboxes get paused, killed and replaced all the time.The databases people reach for don't all behave well when things fail. In Jepsen's tests:- etcd's [reads and writes behaved correctly](https://jepsen.io/analyses/etcd-3.4.3), but its locks could not guarantee that only one client held a lock at a time. - An Amazon RDS for PostgreSQL Multi-AZ cluster [gave inconsistent answers from its read replicas](https://jepsen.io/analyses/amazon-rds-for-postgresql-17.4), so anything that makes a decision should read from the primary. - NATS JetStream [lost data it had confirmed](https://jepsen.io/analyses/nats-2.12.1) under its default settings.```mermaid theme={null} flowchart LR subgraph Poll["Checking for changes"] P1["1,000 agents"] -- "list the folder every 5 s" --> P2["Storage"] end subgraph Feed["Being told about changes"] F1["Storage"] -- "changes, in order" --> F2["A service that forwards them"] F2 -- "one stream per agent" --> F3["1,000 agents"] end ```Agents also need to find out what changed. If they keep listing a folder or bucket to check, you get the kind of traffic that per-request pricing charges for. Kubernetes has the same problem with etcd. It doesn't let clients watch etcd directly; a [cache in between](https://www.kubernetes.dev/resources/keps/2340/) sends changes out to them. A swarm needs something similar.Research points the same way. In the [STORM](https://arxiv.org/html/2605.20563) paper, agents sharing one workspace, with each save checked against the version they had read, did better than agents working in separate git worktrees on closely connected code. In [AgentRoom](https://arxiv.org/abs/2608.23740), giving agents ways to coordinate mattered more than merging their edits automatically. Both studies used eight agents or fewer. Nobody has published results for a thousand.## Where the money goesRequest charges get the attention, but with a swarm, moving data usually costs more. The calculator below compares storage options for a fleet of agents under three made-up workloads. Network charges are easy to miss, so the calculator lets you pick where the sandboxes run:- **Next to the storage:** no transfer charges. - **Outside AWS:** reading storage hosted in AWS is billed as data leaving for the internet. - **In AWS, behind a NAT gateway:** reaching storage outside AWS is charged for every GB that passes through the gateway, in both directions.A separate switch covers sandboxes spread across availability zones while the storage sits in one. AWS charges \$0.01 per GB in each direction for that.Log scale LinearSandboxes run next to the storage outside AWS in AWS, behind a NAT gatewaySandboxes are spread over three zones and zonal storage sits in one (+$0.02/GB on AWS, +$0.01/GB on GCP, for two-thirds of traffic)reads · writes · listings · GB read · GB written per month · avg MB/s read■ metered ■ flat or provisioned ■ over a limitMonthly list prices collected October 22, 2026 (AWS us-east-1, GCP us-central1, Azure East US); 1 TB stored. One read is one GET, one write one PUT, one listing one LIST. Provisioned options are sized for twice the average rate. S3 is assumed to be reached through a free gateway endpoint, so no NAT charge applies to it. EFS has a mount target in every zone, so it has no cross-zone charge. EFS and FSx also need network access from the sandbox to the VPC. Workloads are assumptions, not measurements.A few things show up whichever workload you pick.Paying per GB costs more than paying per request once agents read gigabytes an hour. EFS Elastic is the most expensive option in every scenario.Where things run can matter more than whose storage you buy. E2B's managed sandboxes run on Google Cloud. If your storage is on AWS, every byte they read is billed as data leaving AWS. R2 doesn't charge for data leaving it, but it still charges per request.Storage with "free requests" has fine print. [Backblaze B2](https://www.backblaze.com/cloud-storage/pricing) charges for downloads beyond three times what you store. [Wasabi](https://wasabi.com/pricing/faq) expects you to download less than you store and bills deleted files for 90 days. Both suit backups, not files agents read all day.Paying for a fixed size and speed works like a flat fee, up to a point. Provisioned services and your own servers slow down rather than charge more, but each has a ceiling: one FSx for OpenZFS file system tops out at 10,240 MBps, and one Azure Files share at 102,400 IOPS. Past that, you split the data across several. Azure also caps a share's speed relative to its size, so a fast share sometimes has to be bigger than the data in it.Caching is easiest on files that never change after they're saved. Copies of those never go stale, so a cache can keep them indefinitely, and the number of requests reaching storage depends on how much unique data there is rather than on how often agents read it. Files that do change can be cached too, but then every cache has to check whether its copy is still current, or be told when it isn't.The workloads are guesses, not measurements, so put in your own numbers. The overall shape of the bill matters more than any one figure.## What's still unsolved- **Merging.** It's easy to copy a sandbox, a volume or a bucket. Combining two copies back into one is harder. Vercel's drives can be [forked](https://vercel.com/changelog/vercel-drives-is-now-generally-available), but the forks are independent copies. Tools exist for some formats: git merges text, [Dolt](https://www.dolthub.com/) merges database tables, [nbdime](https://github.com/jupyter/nbdime) merges Jupyter notebooks, and CRDT libraries like [Automerge](https://automerge.org/) merge JSON-like documents. Nothing covers the whole mix of files a swarm produces. - **Cleaning up after crashes.** Agents crash while holding locks, halfway through uploads, or before giving up a task. Most systems either wait for a timeout or need a person to step in. - **Bad shared data.** An agent fooled by a prompt injection can write instructions or bad data that other agents then trust. Anthropic's docs for [Claude Managed Agents memory](https://platform.claude.com/docs/en/managed-agents/memory) warn about this for shared memory. Recording who wrote what, read-only access and review steps help, but there's no standard answer yet. - **Moving between providers.** Snapshots, volumes and access keys are all tied to one sandbox provider. A swarm that runs on several has to keep its storage somewhere neutral. - **Limits on sandbox count.** E2B's Pro plan [allows 100 sandboxes at once](https://e2b.dev/pricing) by default, so 1,000 agents means a bigger plan. Other platforms have similar limits. - **Real data at scale.** Published studies of agents working together use 2 to 16 agents. Anything said about 1,000 agents, this post included, is an educated guess.## Products built for agentsA few products now sell storage made for agents. Two come up often.[Archil](https://docs.archil.com/concepts/sharing-disks) offers disks backed by S3 or another bucket that several machines can mount. In shared mode, a machine "checks out" a folder to write to it, and everyone else can still read it. A machine that crashes keeps its folders checked out until someone forces a takeover, which throws away its unfinished writes. A third mode lets every machine write anywhere, but the docs don't say what happens when two write the same file. Archil also [charges for data sent to compute outside Archil](https://docs.archil.com/administration/billing), which matters when sandboxes run elsewhere.[Mesa](https://docs.mesa.dev/content/mesa/overview) offers versioned folders that agents mount (on Linux, through FUSE) or open from an SDK. Each folder has a history of snapshots and named branches. Mounts on the same branch [see each other's edits within seconds](https://docs.mesa.dev/content/mesafs/advanced/realtime), and when two agents edit the same file in the same second, Mesa keeps both versions, with conflict markers in the file. Mesa says it is built for thousands of agents reading and writing at once.Both cover parts of the combination from the start of this post: private workspaces and shared files. Their docs leave the rest unclear:- neither publishes tests with hundreds or thousands of agents writing at once; - neither describes task claiming or a feed of changes for coordination; - sending a result to one agent or a group comes down to access settings you configure yourself; - recovery after a crash is documented for Archil and not for Mesa; - both need FUSE inside the sandbox, or Mesa's SDK, which rules out some sandboxes.Some of these may be handled but not written down. We haven't tested either product.## A field that's changing fastMost of the products in this post didn't exist eighteen months ago.| When | What changed | | - | - | | Aug–Nov 2024 | S3 adds conditional writes | | Nov 2025 | Cloudflare Containers can mount buckets as folders | | Jan 2026 | Fly.io publishes how Sprites store data; Cursor publishes what it learned running large agent swarms | | Feb 2026 | Anthropic has 16 agents build a C compiler in one shared git repo | | Mar 2026 | AWS AgentCore session storage (preview) | | Apr 2026 | Amazon S3 Files; Cloudflare Artifacts, git hosting for agents (beta) | | May 2026 | Saved sandboxes on Vercel become generally available | | Aug 2026 | Cursor describes Origin and Continuity, git hosting backed by a log in object storage | | Sep 2026 | Google Filestore agent volumes (approved customers only) | | Oct 2026 | Anthropic describes Scion, its own version control system; Vercel Drives becomes generally available, with forks |Sandbox companies are adding volumes and snapshots. AWS and Google are shipping file systems meant for agents. Git hosting is being rebuilt for machines instead of people. Startups are selling file systems for agents, and researchers have started measuring how agents share work.We haven't found a product that does all of it together and shows it working at a thousand agents: private workspaces that move between providers, shared git, results you can hand to other agents, recovery from crashed sandboxes, and coordination that doesn't block, at a price that doesn't climb with every file read. Most of the parts exist, and they change month to month.If you run large groups of agents and have hit any of this, we'd like to hear how you dealt with it. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.