BoxRouter

Fork

Builds on Common Methodology

Times how long a provider takes to create 4 sequential sandboxes from a single snapshot and run the first Python, Node.js, and git command in each fork.

Agent harnesses and reinforcement learning loops use snapshots to save a configured sandbox once and boot clean copies from that state without repeating installation or repository setup.

The Snapshots tab measures two patterns built on that primitive:

  • Snapshot and Restore times saving a snapshot of a configured sandbox, creating one new sandbox from that snapshot after the source sandbox is destroyed, and loading the Python, Node.js, and git runtimes inside it.
  • Fork takes one snapshot and creates four sandboxes from it one after another to measure restore speed and whether later restores accelerate as the provider caches the snapshot.
View the Procedure View Fork Results Download Raw Data View the Parameters

Metrics

Fork is the sum across all 4 sequential forks of two timed stages: Create from Snapshot (creating each sandbox from the shared trial snapshot) and Load Runtimes (running the chained Python, Node.js, and git command inside each fork).

Timed Part Starts Ends
Create fork The create-from-snapshot request The call returns with the fork
Load Runtimes The command Its result and sandbox clock readings return

Measurement

Each trial uses 5 sequential sandboxes (1 source sandbox and 4 sequential forks) created from BoxRouter's pinned standard image on providers that support the snapshots capability through BoxRouter.

  1. Create source sandbox, write marker file, take one snapshot, and destroy source sandbox (untimed on the Fork chart). Those setup steps are recorded in the trial data (Snapshot and Restore times them directly) and excluded from the Fork bar.
  2. Create From Snapshot Time For each of 4 sequential forks, restore a sandbox from the snapshot (Create from Snapshot). Timed
  3. Immediately run the Python, Node.js, and git command (Load Runtimes). Timed
  4. Verify the marker and destroy the fork before creating the next.
  5. Delete the snapshot. After Fork 4 is destroyed, the snapshot is deleted.

The marker file /workspace/snapshot/marker contains boxrouter-snapshot-marker/v1. In each fork, Load Runtimes runs without a prior readiness check, importing python3 (json, sqlite3, ssl, asyncio, urllib.request, email.parser, xml.etree.ElementTree) and node (crypto, zlib, http) modules and running git --version to fault in pages from the snapshot.

Scope Limits and Caveats

  • Forks are created one after another rather than in parallel, so the test isolates sequential snapshot caching (the first fork cold versus later forks warm) rather than concurrent fork contention.
  • Modal's API has no lookup to confirm that a snapshot is gone after deletion, so the benchmark script sends the delete call and records the confirmation as not checkable.

Failures and Exclusions

Outcome classifications, top-up rounds, and outlier rules follow Common Methodology; the first failed step ends the trial without in-trial retries.

A trial succeeds only when all 4 sequential restores and their runtime-load commands succeed within timeout, every fork verifies /workspace/snapshot/marker, and all 5 sandboxes and the snapshot are confirmed deleted.

Step Time Limit
Create source sandbox 5 min
Snapshot 10 min
Destroy source 5 min
Create fork 5 min
Load Runtimes 100 s
Verify and destroy fork 5 min
Delete snapshot 5 min

Statistics

Based On
Common Methodology

Per-unit percentiles, bootstrap confidence intervals, and tail rules follow Common Methodology.

A provider is ranked once it has at least 20 successful trials.

Sandbox and Trials

Image Digest
sha256
Image Version
0.3.11
Image Registry Page
View on the Registry
Pull the Image
docker pull ghcr.io/boxrouter/standard-image@sha256:89d4c97048dbdfb8b6d85ad91833962ff38d1c334a5f47e6de80a9762310191b Copy
Lifetime
15.0 min
Idle Connection Time
5.0 min

From Common Methodology

Idle Connection Exceptions
Provider Rule File Transfer Idle Time
Daytona The SDK has no idle-timeout setting. Its command channel keeps its connection open, and its file-transfer channel reconnects after 5 s of idle time. 5.0 s

From Common Methodology

Network Path Attempts
10

From Common Methodology

Mode
Sequential
Sandboxes per Trial
5
Parallel Trials
10
Concurrent Sandboxes
1

Provider Requirements

Required
Snapshots

At a Glance

Metrics
  • Create From Snapshot Time
  • First Command Time
Quantity
Time
Unit
Milliseconds (ms)
Direction
Lower Is Better
Statistic
Median
Clock
Job-host clock
Machine Size
2 vCPU / 4 GiB
Trials
40
Variants
  • One snapshot and four sequential forks

All Methods

Fork

Times how long a provider takes to create 4 sequential sandboxes from a single snapshot and run the first Python, Node.js, and git command in each fork.

Agent harnesses and reinforcement learning loops use snapshots to save a configured sandbox once and boot clean copies from that state without repeating installation or repository setup.

The Snapshots tab measures two patterns built on that primitive:

  • Snapshot and Restore times saving a snapshot of a configured sandbox, creating one new sandbox from that snapshot after the source sandbox is destroyed, and loading the Python, Node.js, and git runtimes inside it.
  • Fork takes one snapshot and creates four sandboxes from it one after another to measure restore speed and whether later restores accelerate as the provider caches the snapshot.

Metrics

Loading results…

Measurement

Loading results…

Scope Limits and Caveats

Loading results…

Failures and Exclusions

Loading results…

Statistics

Loading results…

Sandbox and Trials

Loading results…

Provider Requirements

Loading results…