Agent Sandbox Registry

agent-sandbox

Known gaps

无中文版

parity/KNOWN-GAPS.md

Known gaps

Command surface redesign (CLI-only; deliberately diverges from the Python CLI)

The CLI has a fixed command surface that the Python CLI does not have. Python tests for the removed behavior were deleted (not skipped) and their entries removed from python-tests-to-port.txt; the replacements are named after the new behavior (tests/test_cli/test_command_surface.test.ts, test_run_command.test.ts, test_exec_command.test.ts). Differential cases under parity/diff/ that exercised removed spellings were updated or dropped; the oracle is still used for everything that did not change.

agent/mcp, agent/mcp_http, agent/tools (builder agent-B)

Behaviour was checked differentially against the Python oracle (python -m agent_sandbox.agent.mcp over stdio, and the Starlette/uvicorn create_mcp_app over raw sockets): 61 HTTP request cases and 41 auth cases are byte-identical (status line, header names/order, body); the 28-line stdio script is identical except for the Python defect below. The items here are the only known differences.

Tests not ported verbatim

Differential harnesses (parity/diff, need ORACLE_VENV)

api / protocol / session (builder core-B)

Everything in src/api, src/protocol, src/session is a line-by-line port; tests/test_api, tests/test_protocol and tests/test_session are ports of the client-sdk tests (438 tests). Differences and omissions:

Tests not ported (with reason)

Tests adapted (same name, different mechanism)

Behavioural differences

agent/qwen_code, clarify, codegen, adopt, coding_agent, template_guide, dockerfile_sources (builder agent-A)

Intentional changes

Behavioural differences

Test environment

models / utils / transport (builder core-A)

Integration verification (cross-slice, after all four slices existed)

parity/diff/mcp_toolerr.mjs (needs ORACLE_VENV) drives the Python oracle MCP stdio server and the Node one with 96 malformed / unauthorized / missing-sandbox / unreachable tool calls against 36 stub platform+envd behaviours (401/403/404/409/422/429/5xx, garbage/empty/array/null bodies, socket reset, closed port, DNS failure, envd failing in the same ways, undecodable file bytes): 3492 response lines and all stub request logs (method, path, body, auth headers) are byte-identical after port/multipart-boundary normalisation. Defects found and fixed by it: array-valued Host header rejected by node:http (every real request failed), connect failures worded [Errno 111] ... instead of anyio's All connection attempts failed, float(timeout) / .encode() / bytes.decode(encoding) / multipart-file type errors surfacing as Node errors, and json_decode using V8 error text instead of orjson.JSONDecodeError.

Remaining known differences of those areas:

cli command layer and output (src/cli/{main,kit,types,prompts,errors,io,output,formatters,table,region,repl,scaffold}.ts, src/bin/abx.ts)

Commands are parsed and rendered by commander (declared through src/cli/kit.ts), tables are plain text (src/cli/table.ts) and colours come from picocolors; none of this is byte-compatible with Click 8.5 or Rich any more. The scaffold templates remain byte-identical to render_scaffold. The items below are the known differences.

Sandbox sub-commands (sandbox_files, sandbox_process, sandbox_system)

Differential harness: ORACLE_VENV=$ORACLE_VENV node parity/diff/sandbox_subcmds.mjs (264 cases against stub control-plane/envd servers: help text, errors, exit codes, --json/--quiet, interactive confirm, request logs). Result: 0 differing cases; 2 cases marked KNOWN (first item below).

Main sandbox commands (sandbox.ts, _coding_agent.ts)

Differential harness: ORACLE_VENV=$ORACLE_VENV node parity/diff/sandbox_cmds.mjs [--only <substr>] [--verbose] [--show] (~560 cases against stub platform/envd servers incl. Connect-framed envd, an https stub for the in-sandbox SandboxServer (throw-away openssl cert, exact SANs), a scripted Qwen Code binary and a pty driver parity/diff/pty_run.py for tty prompts / Ctrl-C / Ctrl-D / connect: help text, errors, exit codes, --json/--quiet/--verbose combos, prompts, NL create pipeline, run (mechanism A and B), upload/download edge cases, region/timeout precedence, kill --all per HTTP failure class, request logs). Result: 538 cases, 5 differing (all the run mechanism-B SDK JSON-parse gap below); the timing-based request-timeout cases can flake when the machine is loaded (re-run --only request-timeout).

config_cmd (commands/config_cmd.ts, utils/_tomllib.ts)

Differential harness: ORACLE_VENV=$ORACLE_VENV node parity/diff/config_cmds.mjs [--only <substr>] [--verbose] [--jobs N] (~1450 cases incl. ~58 pty wizard / masked-input cases, 78 warn-transport cases that drive sandbox list to print the transport loader's WARNING: Failed to load <config.toml>: <tomllib message> line; isolated HOME and cwd, no real credentials, ABX_NO_KEYCHAIN=1). Result: 0 differing cases and no output normalisation other than the temp-dir path.

commands/mcp (builder mcp-cmd)

parity/diff/mcp_cmds.mjs (needs ORACLE_VENV; --only <substr>, --verbose) compares the oracle and the port in isolated HOME / cwd / run dirs against a stub platform server: 216 cases (install per target and edge inputs incl. unicode / floats / invalid UTF-8, status text+JSON, stop, start validation, deploy option matrix and generated artifact, STDIO over a stdin script, SIGTERM/SIGINT, second start refused, HTTP foreground with probes, port in use, unresolvable host, --background start/status/stop, background child dying early). All identical after normalising temp dirs, pids, ports, tokens, the launcher in IDE configs and the items below.

commands/template, _adopt, deploy (builder template-cmd)

parity/diff/template_cmds.mjs (needs ORACLE_VENV; --only <substr>, --verbose) compares the oracle and the port in an isolated HOME / work dir against a stub server (platform /templates, /index.yaml, GitHub tarballs via --registry-url) and a fake docker on PATH: 241 cases (help, list/info/delete incl. HTTP errors, create/push/build/deploy/top-level deploy option matrix, search/install from index / local / GitHub tarball / bare name, init cases / --from / natural language, adopt offline paths). Compared: exit code, stdout, stderr, stub request log, docker invocations, resulting work-dir tree. All identical after normalising temp names/ports/timings and the items below (cases with a known marker).

Unowned ledger entries: local install, evidence snapshots, template guide, integration (builder "unowned")

Follow-up fixes (JSON / YAML / int parsing) and what remains

Adversarial review fixes: agent / template (builder agent-template-fix)

Review: parity/review/agent-template.md (12 BUG, 4 GAP-UNJUSTIFIED). Every item is fixed except the two noted below; the ADV tests became tests/test_templates/adv_*.test.ts, tests/test_agent/adv_*.test.ts, tests/test_utils/adv_pyos_parity.test.ts.