Sequential TTI

Builds on Common Methodology

Times each sandbox from its create request until its first command returns.

Every interactive coding session, evaluation task, and reinforcement learning rollout begins by waiting for a sandbox to boot and accept its first shell command.

The Create tab measures that startup wait, Time to Interactive (TTI), under two launch patterns: starting sandboxes one after another in a single loop (Sequential TTI) and launching a parallel wave of sandboxes at the same moment (Burst TTI).

View the Procedure View Sequential TTI Results Download Raw Data View the Parameters

Metrics

Sequential TTI is the duration from the start of create until the first command inside the sandbox returns its expected output, computed as the sum of Create and First command.

Timed Part	Starts	Ends
Create	The create request	The create call returns
First command	The command	Its exit code is back at the job host

Measurement

Each trial uses 10 sequential sandboxes created from BoxRouter's pinned standard image plus a unique per-trial image layer (400 sandboxes per variant per provider).

1. Push a unique trial image layer and prepare it with the provider.
2. Time per Sandbox For each of 10 sequential sandboxes, run create (Create). Timed
3. Immediately run the selected first command (First command). Timed Sent once by the benchmark script, as soon as create returns, with no ready check before it and no retry. A not-ready answer (HTTP 502, 503 or 504, or a refused or reset connection) ends the trial like any other error.
4. Read /proc/uptime and probe size.
5. Destroy the sandbox and confirm deletion before starting the next.

Command
true
python3 -c 'import json, sqlite3, ssl, asyncio, urllib.request, email.parser, xml.etree.ElementTree'
node -e 'require("crypto"); require("zlib"); require("http")'
git --version

The four first-command variants compare bare container readiness (true) against the dynamic-linking and filesystem page-in time of loading a language runtime on first touch.

Scope Limits and Caveats

• The trial image layer is unique, but provider base-image caches and warm pools stay enabled; cold hosts and base layers are not guaranteed.
• Each trial adds one unique image layer; provider base-image caches and warm pools stay enabled, so later sandboxes may reuse provider state.
• Trials run in parallel per provider, up to 10 at once, so other sandboxes from the same batch can be in flight on the provider's infrastructure, and the in-flight count is recorded with each trial.
• Providers that lazily stream container filesystem blocks pay only for the blocks that /bin/bash, python3, node and git touch, not for the full pinned image.

Failures and Exclusions

Outcome classifications, top-up rounds, and outlier rules follow Common Methodology; the first failed step ends the trial without in-trial retries.

A trial succeeds only when image.prepare completes, all 10 sequential sandboxes complete create and their first command within 2 min with exit code 0, and every sandbox and the trial image are confirmed deleted.

Step	Time Limit
Image build	30 min
Create	5 min
First command	2 min
Uptime read	30 s
Destroy	5 min

Statistics

Based On

Common Methodology

Per-unit percentiles, bootstrap confidence intervals, and tail rules follow Common Methodology.

Across 40 trials of 10 sequential sandboxes, each variant pools up to 400 individual sandbox TTI observations, with confidence intervals resampled at the whole-trial level because sequential sandboxes inside one trial share the prepared trial image. A provider is ranked on Sequential TTI once it has at least 1 successful trial.

Sandbox and Trials

Image Digest

sha256

Image Version

0.3.11

Image Registry Page

View on the Registry

Pull the Image

docker pull ghcr.io/boxrouter/standard-image@sha256:89d4c97048dbdfb8b6d85ad91833962ff38d1c334a5f47e6de80a9762310191b Copy

Lifetime

5.0 min

Idle Connection Time

5.0 min

From Common Methodology

Idle Connection Exceptions

Provider	Rule	File Transfer Idle Time
Daytona	The SDK has no idle-timeout setting. Its command channel keeps its connection open, and its file-transfer channel reconnects after 5 s of idle time.	5.0 s

From Common Methodology

Network Path Attempts

10

From Common Methodology

Mode

Sequential

Sandboxes per Trial

10

Parallel Trials

10

Concurrent Sandboxes

1

At a Glance

Short Name

Create to first command

Quantity

Time

Per

Sandbox

Unit

Milliseconds (ms)

Direction

Lower Is Better

Statistic

Median

Clock

Job-host clock

From Common Methodology

Machine Size

2 vCPU / 4 GiB

Trials

40

Variants

• true
• python3 -c import
• node -e require
• git --version