browser-automation

中文版 | English

English

Sandbox environment with Playwright + Chromium preinstalled, suited for web scraping, UI automation testing, and browser interaction tasks.

This template enables the built-in BROWSER capability group, exposing 8 low-level browser endpoints while also registering 3 high-level business commands for AI agents.

Environment

Installation

Install from a local path:

abx template install ./browser-automation --registry-type local

Install from GitHub:

abx template install alibaba-agent-sandbox/awesome-templates//browser-automation

High-Level Business Commands

The template registers the following high-level commands for AI agent workflows:

browse(url, action="screenshot")

Smart browsing — navigates to a URL and performs different operations based on action.

scrape(url, selector)

Extract element text content by CSS selector. Returns an array of the matched elements' texts.

fill_form(url, fields_json)

Auto form filling — navigates to a URL and fills each form field as described by the JSON.

fields_json format:

[
  {"selector": "#email", "value": "test@example.com"},
  {"selector": "#password", "value": "123456"}
]

Built-in BROWSER Endpoints

After enabling CapabilityGroup.BROWSER, the following low-level HTTP endpoints are directly callable:

Usage

Create a sandbox instance:

abx create --template browser-automation

Take a screenshot with a high-level command:

abx exec <sandbox-id> browse --url https://example.com --action screenshot

Extract text with a high-level command:

abx exec <sandbox-id> browse --url https://example.com --action extract

Scrape a specific element with a high-level command:

abx exec <sandbox-id> scrape --url https://example.com --selector "h1"

Using the Python SDK:

from agent_sandbox import Sandbox

sandbox = Sandbox.create_sync(template="browser-automation")

# Take a webpage screenshot (registered command; returns the result value)
result = sandbox.run_sync("browse", url="https://example.com", action="screenshot")
print(result)  # {"action": "screenshot", "url": "...", "path": "/workspace/screenshot_xxx.png"}

# Extract page text
result = sandbox.run_sync("browse", url="https://example.com", action="extract")
print(result["text"])

# Scrape by selector
result = sandbox.run_sync("scrape", url="https://example.com", selector="h1")
print(result["texts"])

# Auto form filling
import json
fields = [
    {"selector": "#email", "value": "test@example.com"},
    {"selector": "#password", "value": "secret"},
]
result = sandbox.run_sync("fill_form", url="https://example.com/login", fields_json=json.dumps(fields))
print(result)

Configuration

Environment Variables

Headless vs. Headed Mode

Runs in headless mode by default (no GUI needed). For headed mode, start Xvfb first:

Xvfb :99 -screen 0 1920x1080x24 &

Notes

• Persistent browser instance: The template uses a module-level Playwright singleton, created on first call and reused by subsequent commands — far more performant than creating and destroying one each time.
• Memory usage: Browsers consume considerable memory at runtime; keeping at least 4096 MB is recommended.
• Chromium only: To keep the image small, only Chromium is installed by default. For Firefox or WebKit, run python3 -m playwright install firefox inside the sandbox.
• Network access: Web scraping tasks require outbound internet access; make sure the sandbox network policy allows outbound connections.
• Screenshots and files: Generated screenshots and files are saved under /workspace and can be downloaded via the file API.

中文

预装 Playwright + Chromium 的沙箱环境，适用于网页抓取、UI 自动化测试和浏览器交互任务。

本模板启用了 BROWSER 内置能力组，提供 8 个低层浏览器端点，同时注册了 3 个面向 AI Agent 的高层业务命令。

环境说明

安装方式

从本地安装：

abx template install ./browser-automation --registry-type local

从 GitHub 安装：

abx template install alibaba-agent-sandbox/awesome-templates//browser-automation

高层业务命令

模板注册了以下面向 AI Agent 工作流的高层命令：

browse(url, action="screenshot")

智能浏览 — 导航到 URL 并根据 action 执行不同操作。

scrape(url, selector)

按 CSS 选择器提取元素文本内容。返回匹配元素的文本数组。

fill_form(url, fields_json)

自动填表 — 导航到 URL，按 JSON 描述逐个填写表单字段。

fields_json 格式：

[
  {"selector": "#email", "value": "test@example.com"},
  {"selector": "#password", "value": "123456"}
]

内置 BROWSER 端点

启用 CapabilityGroup.BROWSER 后，以下低层 HTTP 端点可直接调用：

使用示例

创建沙箱实例：

abx create --template browser-automation

使用高层命令截图：

abx exec <sandbox-id> browse --url https://example.com --action screenshot

使用高层命令提取文本：

abx exec <sandbox-id> browse --url https://example.com --action extract

使用高层命令抓取指定元素：

abx exec <sandbox-id> scrape --url https://example.com --selector "h1"

使用 Python SDK：

from agent_sandbox import Sandbox

sandbox = Sandbox.create_sync(template="browser-automation")

# 截取网页截图（调用自定义命令，返回 CommandResult）
result = sandbox.run_sync("browse", url="https://example.com", action="screenshot")
print(result)  # {"action": "screenshot", "url": "...", "path": "/workspace/screenshot_xxx.png"}

# 提取页面文本
result = sandbox.run_sync("browse", url="https://example.com", action="extract")
print(result["text"])

# 按选择器抓取
result = sandbox.run_sync("scrape", url="https://example.com", selector="h1")
print(result["texts"])

# 自动填表
import json
fields = [
    {"selector": "#email", "value": "test@example.com"},
    {"selector": "#password", "value": "secret"},
]
result = sandbox.run_sync("fill_form", url="https://example.com/login", fields_json=json.dumps(fields))
print(result)

配置说明

环境变量

Headless 与 Headed 模式

默认以 headless 模式运行（无需 GUI）。如需 headed 模式，请先启动 Xvfb：

Xvfb :99 -screen 0 1920x1080x24 &

注意事项

• 持久化浏览器实例：模板使用模块级 Playwright 单例，首次调用时创建，后续命令复用同一实例，性能远优于每次创建/销毁。
• 内存消耗：浏览器运行时内存消耗较大，建议保持 4096 MB 以上配置。
• 仅含 Chromium：为减小镜像体积，默认只安装 Chromium。如需 Firefox 或 WebKit，可在沙箱内执行 python3 -m playwright install firefox。
• 网络访问：网页抓取任务需要外网访问权限，请确保沙箱网络策略允许出站连接。
• 截图与文件：生成的截图和文件保存在 /workspace 目录下，可通过文件 API 下载。

Deploy and local e2e

Install and register this template:

abx template install alibaba-agent-sandbox/awesome-templates//browser-automation

From this directory (build, push, and register):

abx deploy

From the repository root, with agent-sandbox-server and httpx installed:

python3 scripts/run_local_e2e.py

That starts python3 commands.py in this directory and runs e2e_test.py.