browser-automation

中文版 | English

English

Sandbox environment with Playwright + Chromium preinstalled, suited for web scraping, UI automation testing, and browser interaction tasks.

This template enables the built-in BROWSER capability group, exposing 8 low-level browser endpoints while also registering 3 high-level business commands for AI agents.

Environment

Category	Details
Base image	python:3.12-slim (Playwright and Chromium installed)
Browser	Chromium (managed by Playwright)
Python packages	`playwright`, `beautifulsoup4`, `lxml`, `httpx`, `Pillow`
System tools	curl, wget, xvfb, jq
Resources	2 CPU / 4096 MB memory
Enabled capability groups	`BROWSER`, `FILE_OPS`, `PROCESS`

Installation

Install from a local path:

abx template install ./browser-automation --registry-type local

Install from GitHub:

abx template install alibaba-agent-sandbox/awesome-templates//browser-automation

High-Level Business Commands

The template registers the following high-level commands for AI agent workflows:

`browse(url, action="screenshot")`

Smart browsing — navigates to a URL and performs different operations based on action.

action	Behavior
`screenshot`	Take a full-page screenshot and return the saved path
`extract`	Retrieve the visible text content of the page
`links`	Extract all hyperlinks on the page (href + text)
`full`	Screenshot + title + first 500 characters of text summary

`scrape(url, selector)`

Extract element text content by CSS selector. Returns an array of the matched elements' texts.

`fill_form(url, fields_json)`

Auto form filling — navigates to a URL and fills each form field as described by the JSON.

fields_json format:

[
  {"selector": "#email", "value": "test@example.com"},
  {"selector": "#password", "value": "123456"}
]

Built-in BROWSER Endpoints

After enabling CapabilityGroup.BROWSER, the following low-level HTTP endpoints are directly callable:

Method	Endpoint	Description
POST	`/browser/navigate`	Navigate to the specified URL
POST	`/browser/screenshot`	Capture a page screenshot (returns base64)
GET	`/browser/content`	Get page HTML or plain text
POST	`/browser/click`	Click the specified element
POST	`/browser/type`	Type text into the specified element
POST	`/browser/evaluate`	Execute JavaScript in the page
POST	`/browser/pdf`	Generate a PDF of the page (returns base64)
GET	`/browser/console`	Get the collected console logs

Usage

Create a sandbox instance:

abx create --template browser-automation

Take a screenshot with a high-level command:

abx exec <sandbox-id> browse --url https://example.com --action screenshot

Extract text with a high-level command:

abx exec <sandbox-id> browse --url https://example.com --action extract

Scrape a specific element with a high-level command:

abx exec <sandbox-id> scrape --url https://example.com --selector "h1"

Using the Python SDK:

from agent_sandbox import Sandbox

sandbox = Sandbox.create_sync(template="browser-automation")

# Take a webpage screenshot (registered command; returns the result value)
result = sandbox.run_sync("browse", url="https://example.com", action="screenshot")
print(result)  # {"action": "screenshot", "url": "...", "path": "/workspace/screenshot_xxx.png"}

# Extract page text
result = sandbox.run_sync("browse", url="https://example.com", action="extract")
print(result["text"])

# Scrape by selector
result = sandbox.run_sync("scrape", url="https://example.com", selector="h1")
print(result["texts"])

# Auto form filling
import json
fields = [
    {"selector": "#email", "value": "test@example.com"},
    {"selector": "#password", "value": "secret"},
]
result = sandbox.run_sync("fill_form", url="https://example.com/login", fields_json=json.dumps(fields))
print(result)

Configuration

Environment Variables

Variable	Description	Default
`DISPLAY`	X virtual framebuffer display number	`:99`
`PLAYWRIGHT_BROWSERS_PATH`	Browser installation path	`/ms-playwright`

Headless vs. Headed Mode

Runs in headless mode by default (no GUI needed). For headed mode, start Xvfb first:

Xvfb :99 -screen 0 1920x1080x24 &

Notes

Persistent browser instance: The template uses a module-level Playwright singleton, created on first call and reused by subsequent commands — far more performant than creating and destroying one each time.
Memory usage: Browsers consume considerable memory at runtime; keeping at least 4096 MB is recommended.
Chromium only: To keep the image small, only Chromium is installed by default. For Firefox or WebKit, run python3 -m playwright install firefox inside the sandbox.
Network access: Web scraping tasks require outbound internet access; make sure the sandbox network policy allows outbound connections.
Screenshots and files: Generated screenshots and files are saved under /workspace and can be downloaded via the file API.

中文

预装 Playwright + Chromium 的沙箱环境，适用于网页抓取、UI 自动化测试和浏览器交互任务。

本模板启用了 BROWSER 内置能力组，提供 8 个低层浏览器端点，同时注册了 3 个面向 AI Agent 的高层业务命令。

环境说明

类别	内容
基础镜像	python:3.12-slim（镜像内安装 Playwright 与 Chromium）
浏览器	Chromium（通过 Playwright 管理）
Python 包	`playwright`、`beautifulsoup4`、`lxml`、`httpx`、`Pillow`
系统工具	curl、wget、xvfb、jq
资源配置	2 CPU / 4096 MB 内存
启用能力组	`BROWSER`、`FILE_OPS`、`PROCESS`

安装方式

从本地安装：

abx template install ./browser-automation --registry-type local

从 GitHub 安装：

abx template install alibaba-agent-sandbox/awesome-templates//browser-automation

高层业务命令

模板注册了以下面向 AI Agent 工作流的高层命令：

`browse(url, action="screenshot")`

智能浏览 — 导航到 URL 并根据 action 执行不同操作。

action	行为
`screenshot`	截取整页截图，返回保存路径
`extract`	获取页面可见文本内容
`links`	提取页面中所有超链接（href + text）
`full`	截图 + 标题 + 前 500 字文本摘要

`scrape(url, selector)`

按 CSS 选择器提取元素文本内容。返回匹配元素的文本数组。

`fill_form(url, fields_json)`

自动填表 — 导航到 URL，按 JSON 描述逐个填写表单字段。

fields_json 格式：

[
  {"selector": "#email", "value": "test@example.com"},
  {"selector": "#password", "value": "123456"}
]

内置 BROWSER 端点

启用 CapabilityGroup.BROWSER 后，以下低层 HTTP 端点可直接调用：

方法	端点	说明
POST	`/browser/navigate`	导航到指定 URL
POST	`/browser/screenshot`	截取页面截图（返回 base64）
GET	`/browser/content`	获取页面 HTML 或纯文本
POST	`/browser/click`	点击指定元素
POST	`/browser/type`	向指定元素输入文本
POST	`/browser/evaluate`	在页面中执行 JavaScript
POST	`/browser/pdf`	生成页面 PDF（返回 base64）
GET	`/browser/console`	获取收集的控制台日志

使用示例

创建沙箱实例：

abx create --template browser-automation

使用高层命令截图：

abx exec <sandbox-id> browse --url https://example.com --action screenshot

使用高层命令提取文本：

abx exec <sandbox-id> browse --url https://example.com --action extract

使用高层命令抓取指定元素：

abx exec <sandbox-id> scrape --url https://example.com --selector "h1"

使用 Python SDK：

from agent_sandbox import Sandbox

sandbox = Sandbox.create_sync(template="browser-automation")

# 截取网页截图（调用自定义命令，返回 CommandResult）
result = sandbox.run_sync("browse", url="https://example.com", action="screenshot")
print(result)  # {"action": "screenshot", "url": "...", "path": "/workspace/screenshot_xxx.png"}

# 提取页面文本
result = sandbox.run_sync("browse", url="https://example.com", action="extract")
print(result["text"])

# 按选择器抓取
result = sandbox.run_sync("scrape", url="https://example.com", selector="h1")
print(result["texts"])

# 自动填表
import json
fields = [
    {"selector": "#email", "value": "test@example.com"},
    {"selector": "#password", "value": "secret"},
]
result = sandbox.run_sync("fill_form", url="https://example.com/login", fields_json=json.dumps(fields))
print(result)

配置说明

环境变量

变量名	说明	默认值
`DISPLAY`	X 虚拟帧缓冲显示编号	`:99`
`PLAYWRIGHT_BROWSERS_PATH`	浏览器安装路径	`/ms-playwright`

Headless 与 Headed 模式

默认以 headless 模式运行（无需 GUI）。如需 headed 模式，请先启动 Xvfb：

Xvfb :99 -screen 0 1920x1080x24 &

注意事项

持久化浏览器实例：模板使用模块级 Playwright 单例，首次调用时创建，后续命令复用同一实例，性能远优于每次创建/销毁。
内存消耗：浏览器运行时内存消耗较大，建议保持 4096 MB 以上配置。
仅含 Chromium：为减小镜像体积，默认只安装 Chromium。如需 Firefox 或 WebKit，可在沙箱内执行 python3 -m playwright install firefox。
网络访问：网页抓取任务需要外网访问权限，请确保沙箱网络策略允许出站连接。
截图与文件：生成的截图和文件保存在 /workspace 目录下，可通过文件 API 下载。

Deploy and local e2e

Install and register this template:

abx template install alibaba-agent-sandbox/awesome-templates//browser-automation

From this directory (build, push, and register):

abx deploy

From the repository root, with agent-sandbox-server and httpx installed:

python3 scripts/run_local_e2e.py

That starts python3 commands.py in this directory and runs e2e_test.py.

Sandbox environment with Playwright + Chr…

browser-automation

English

Environment

Installation

High-Level Business Commands

browse(url, action="screenshot")

scrape(url, selector)

fill_form(url, fields_json)

Built-in BROWSER Endpoints

Usage

Configuration

Environment Variables

Headless vs. Headed Mode

Notes

中文

环境说明

安装方式

高层业务命令

browse(url, action="screenshot")

scrape(url, selector)

fill_form(url, fields_json)

内置 BROWSER 端点

使用示例

配置说明

环境变量

Headless 与 Headed 模式

注意事项

Deploy and local e2e

`browse(url, action="screenshot")`

`scrape(url, selector)`

`fill_form(url, fields_json)`

`browse(url, action="screenshot")`

`scrape(url, selector)`

`fill_form(url, fields_json)`