Exec

Builds on Common Methodology

Times calls to a command with almost no runtime, one call alone and 100 calls in a row.

When an AI agent or test harness outside the sandbox dispatches commands over the provider's API, every step pays the round-trip overhead of the provider's execution endpoint and the container's internal daemon. The Exec tab isolates that round-trip floor by sending the shell builtin true, a command with nearly zero in-sandbox runtime, 100 times in a row inside each sandbox.

The view selector presents the same benchmark runs two ways: Latency per Call pools all 10,000 individual calls across 100 sandboxes to expose per-call time and tail spikes, and Latency per 100 Calls sums the 100 calls per sandbox to show session-level execution time.

View the Procedure View Exec Results Download Raw Data View the Parameters

Metrics

Latency per 100 Calls

Latency per 100 Calls is the sum of the 100 sequential true call durations inside a single sandbox trial, measuring the cumulative command-dispatch overhead a chatty external agent pays over a 100-step session.

Latency per Call

Latency per Call is the round-trip duration of a single true call (a command with almost no runtime) on a warm sandbox, pooled across all 10,000 calls from 100 sandboxes.

Latency per Call (2 vCPU / 8 GiB)

Latency per Call (2 vCPU / 8 GiB) is the same measurement as Latency per Call, taken in a 2 vCPU / 8 GiB sandbox.

Timed Part	Starts	Ends
true calls	The command call is dispatched.	The exit code and sandbox clock readings return to the job host.

T_i = \sum_{c=1}^{100} t_{i,c}

• T_i Time of the 100 calls in trial i.
• t_{i,c} Job-host round-trip time of call c in trial i, from the start of the request until the exit code and guest clock readings return.

\tilde{t} = \operatorname{median}_{i,c} \left( t_{i,c} \right), L_{i,c} = t_{i,c} - r_{i,c}

• \tilde{t} Median round-trip time of one call, taken over all calls of all successful trials.
• t_{i,c} Job-host round-trip time of call c in trial i, from the start of the request until the exit code and guest clock readings return.
• r_{i,c} Runtime of call c in trial i: the in-sandbox duration between the two $EPOCHREALTIME readings around true, close to zero because true is a shell builtin.
• L_{i,c} Latency is the total time minus the runtime of a call, capturing network transit, the provider's exec API, and the provider's process that starts the command inside the sandbox.

Measurement

Each trial uses a sandbox created from BoxRouter's pinned standard image.

1. Create sandbox and run the untimed readiness check.
2. Latency per 100 Calls Send 100 sequential true commands wrapped in $EPOCHREALTIME and /proc/uptime reads, timing each call on the job host from request start until the exit code and guest clock readings return. Timed
3. Destroy sandbox.

The shell builtin true exits immediately with status 0 without disk or network I/O, so each call's duration isolates API serialization, network transit, provider routing, and container daemon dispatch.

Scope Limits and Caveats

• Summing 100 calls per sandbox smooths out single-call time spikes that do not persist across the whole sandbox session; use Latency per Call to inspect per-call p95 tail spikes.
• Every call pays the network round-trip time between the job host and the provider's default US region.
• Calls run back to back over a connection kept open for up to the idle timeout set in Common Methodology while idle, measuring steady-state dispatch on a warm connection rather than a cold one.
• Each call sends the nine-line Bash clock-reading wrapper and receives the __SANDBOX_CLOCK trailer line on stderr, which adds a few hundred bytes to the request and response.

Failures and Exclusions

Outcome classifications, top-up rounds, and outlier rules follow Common Methodology; the first failed step ends the trial without in-trial retries.

A trial succeeds only when all 100 sequential true calls exit 0, every call passes the twenty-millisecond /proc/uptime clock gate, and the sandbox is confirmed destroyed.

Step	Time Limit
Create	5 min
true calls	30 s
Destroy	5 min

Statistics

Based On

Common Methodology

Per-unit percentiles, bootstrap confidence intervals, and tail rules follow Common Methodology.

For Latency per Call, the headline statistic (median, mean, or p95) is computed over all 10,000 pooled calls across the 100 trials, while bootstrap confidence intervals resample whole trials (100-call blocks) with replacement so intra-sandbox correlation does not artificially narrow the interval.

For Latency per 100 Calls, each sandbox trial yields one summed 100-call duration (up to 100 trial values per provider). A provider is ranked once it has at least 5 successful trials.

Sandbox and Trials

Image Digest

sha256

Image Version

0.3.11

Image Registry Page

View on the Registry

Pull the Image

docker pull ghcr.io/boxrouter/standard-image@sha256:89d4c97048dbdfb8b6d85ad91833962ff38d1c334a5f47e6de80a9762310191b Copy

Lifetime

10.0 min

Idle Connection Time

5.0 min

From Common Methodology

Idle Connection Exceptions

Provider	Rule	File Transfer Idle Time
Daytona	The SDK has no idle-timeout setting. Its command channel keeps its connection open, and its file-transfer channel reconnects after 5 s of idle time.	5.0 s

From Common Methodology

Network Path Attempts

10

From Common Methodology

Mode

Single

Sandboxes per Trial

1

Parallel Trials

10

Concurrent Sandboxes

1

At a Glance

Metrics

• Latency per 100 Calls
• Latency per Call
• Latency per Call (2 vCPU / 8 GiB)

Quantity

Latency

Unit

Milliseconds (ms)

Direction

Lower Is Better

Statistic

Median

Clock

Job-host clock

Machine Size

2 vCPU / 4 GiB

Trials

100

Variants

• true, 100 calls through the sandbox