Real-cloud E2E suite (Python SDK)

End-to-end tests of agent-sandbox (client-sdk) and agent-sandbox-server (server-sdk) against the real Alibaba Cloud FC agent sandbox platform: real sandboxes, real envd, real in-sandbox server. Templates are built and deployed with abx template build|deploy, which the CLI's own E2E suite covers. Mocks live in client-sdk/tests and server-sdk/tests; this suite exists because mocks cannot prove the product works (see the Definition of Done in ../.agents/notes/DEFINITION-OF-DONE.md).

中文说明见 README.zh-CN.md. Known product defects pinned by this suite: KNOWN-ISSUES.md.

Run

pip install -e ./server-sdk -e ./client-sdk pytest        # or: make dev
cp .env.example .env                                       # then fill in real values (gitignored)

make e2e          # sandbox tests only (fast path, ~8 min)
make e2e-all      # + slow tests
make e2e-cleanup  # dry-run janitor; add ARGS=--yes to delete leftovers

cd e2e && python -m pytest test_03_commands.py -k "timeout"   # a subset
cd e2e && python -m pytest -m "not e2e_slow" -ra               # what `make e2e` runs, plus summary

Required: ABX_SANDBOX_API_KEY (plus ABX_SANDBOX_BASE_URL/ABX_SANDBOX_DOMAIN for non-default regions). Values come from the environment or the gitignored .env files (e2e/.env, .env, client-sdk/.env); the environment wins.

Without credentials every test is skipped with the message E2E skipped: missing credentials [...]. The suite is never collected by the unit CI (client-sdk / server-sdk run pytest tests/ from their own directories).

Provider plugins

Third-party providers are not part of this suite's cloud run. They follow the same six tiers as the CLI, from plugin unit tests (T0) to a local OpenSandbox server (T4) and the Alibaba Cloud platform (T5). The table is cli/e2e/providers/PLAN.md (中文). OpenSandbox helpers for T4 live in opensandbox/README.md.

What is covered

File	Scope
`test_01_platform.py`	auth (valid/invalid key), sandbox list, error mapping
`test_02_lifecycle.py`	create / connect / info / list / kill, timeouts (`e2e_slow`: TTL expiry), metadata tagging
`test_03_commands.py`	`commands.run` / `run_code`, exit codes, stdout/stderr, cwd, background processes, process list / signals
`test_04_files_envd.py`	files through envd: write/read text and binary, list/stat, mkdir/move/remove, size boundaries
`test_05_network_and_code.py`	port URLs and in-sandbox HTTP services, code execution
`test_06_server_sdk.py`	`agent-sandbox-server` inside a real sandbox (template `python-hello`): `/commands`, `/files/`, `/process/`, `/capabilities`, `/system/info`, `/env`, `/ports`, custom routes
`test_07_declarative.py`	`@sandbox` declarative remote execution

Markers: e2e (everything), e2e_slow (> ~60 s), known_issue (strict xfail documenting a recorded bug, see KNOWN-ISSUES.md).

Guarantees

Tagging and cleanup. Every sandbox is created through fixtures that add the metadata e2e=<run id> and register it for deletion. The session finalizer deletes everything tracked and then sweeps by tag, even when tests fail or the run is interrupted. scripts/e2e-cleanup.py is the janitor for runs that were hard-killed (default is a dry run; --yes deletes; --run-id, --all-tagged, --older-than N). It never touches sandboxes without the tag.
Secrets are never printed. Failure reports pass through a masking hook that replaces every credential value (and derived forms); helpers never log request headers. Do not add print of environment values.
Determinism. Tests are independent (own sandbox or own directory under a per-run workdir), use generous timeouts and poll with eventually only for known-eventual operations (envd readiness, sandbox listing, server start-up).

Environment knobs

Variable	Default	Meaning
`E2E_SANDBOX_TTL`	300	sandbox lifetime in seconds
`E2E_SERVER_TEMPLATE`	`python-hello`	template with `agent-sandbox-server` preinstalled
`E2E_RUN_ID`	generated	tag value (`e2e=<id>`)

Adding a test

Use the fixtures (base_sandbox, code_sandbox, server_sandbox, sandbox_factory, workdir); never call Sandbox.create directly, or the sandbox will not be tracked.
Assert the intended behaviour. If it fails because of a product bug, mark the test @pytest.mark.known_issue + @pytest.mark.xfail(strict=True, reason="KI-0NN: ...") and add the entry to KNOWN-ISSUES.md (both languages).
Record the run in .agents/evidence/verify/<date>-<topic>.md (Definition of Done).