feat(mcp): 浏览器分步填报 P3——7 个 browser 工具 + 单线程 actor + 磁盘锁
填报从「一把梭」变成分步：Agent 可以开浏览器、登录、建单、按步填、看页面、关，每步有截图， 中途失败不必从头重来。提交不在本步——提交不可逆，留给 P4 的 submit_preview/submit_execute。 - src/infra/browser/actor.py（新）：Playwright 单线程 actor（sync API 的 greenlet 线程亲和要求所有 调用回到创建线程）+ 会话注册表 + cases/<id>/.browser.lock（PID/started_at/heartbeat/url， 心跳 180 秒过期才允许接管；锁文件损坏按无锁处理；atexit 尽力关浏览器）。与 COM actor 分开。 - base.py：login_portal 拆成 submit_login + wait_for_portal（返回布尔不抛异常），新增 portal_reached/ human_required；_screenshot 支持 screenshot_dir 并按 001-步骤名.png 编号、返回路径、失败只告警 - src/mcp/tools/browser.py（新）：browser_open/login/new_form/fill/steps/page_state/close，工具 17 → 24 · need_human 用 ok:true + logged_in:false 返回（扫码不是失败，避免宿主当错误重试或另开窗口） · 步骤顺序硬约束，跳步拒绝并用 next_action 指回该补的一步；步骤报错回传 detail.screenshot · browser_close 明确不提交不存草稿；travel 五步、normal 四步（无 subsidy） - tests/test_mcp_browser.py（新，17 例，FakeBot/FakePage 不起浏览器）：锁四种情形、need_human 分支与 截图落点、重复登录幂等、未打开时各工具拒绝、类型推断与两 info 并存时拒猜、未登录拒建单、跳步拦截、 步骤推进与 remaining、未知步骤、报错截图、close 释放会话与锁、open 复用、steps 两种流程 - 文档：docs/MCP.md 工具段按 wire schema 重生成（24 工具）；AGENTS/SKILL/操作指南/README/浏览器 README 同步分步流程、锁与心跳、有头默认、提交必须交人工；方案文档 §13 记录实现、取舍与未完成项 验证： - 联机跑通 open（3.1s 起有头 Chromium）→ 锁写入（pid/heartbeat）→ page_state（about:blank， 001-state.png 4721 字节）→ 重复 open 复用（already_open=true，不多开窗）→ close（锁释放、注册表清空） - 登录与填报的联机验证需要用户在场（SSO 可能扫码），本轮未做 - 门禁：ruff / format --check / mypy src / deptry / pytest 129 passed / mcp_smoke 24 工具 instructions 2109 字节。仍用 --no-verify 提交并手跑等价检查（原因见 AGENTS 提交纪律）。
This commit is contained in:
wandering
@@ -13,15 +13,32 @@ Playwright 操作财务报销系统：登录信息门户、进入报销系统、
|
||||
| `normal.py` | 普通填报：基本信息 → 总明细 → 支付方式 → 附件 |
|
||||
| `__init__.py` | `run_bot(config, travel_info, normal_info)`；`run_bot_web(config, work_dir)` 从会话缓存加载后执行 |
|
||||
|
||||
## 现状与计划
|
||||
## 两种用法
|
||||
|
||||
现有入口是**一次性跑完**的编排函数，适合手工验证，不适合 Agent 分步介入：中途出错只能整体重来，
|
||||
也没法在填到一半时先看一眼。P3 计划拆成 `browser_open / login / new_form / fill(step) / page_state / close`
|
||||
分步工具，过程截图进 `cases/<id>/screenshots/NNN-<step>.png`，并用 `.browser.lock`（PID + 心跳）防止多代进程
|
||||
抢同一个浏览器；P4 再加提交前预览与一次性 token 护栏（提交不可逆）。
|
||||
- **MCP 分步工具**（推荐）：`browser_open → browser_login → browser_new_form → browser_fill(step) →
|
||||
`browser_page_state → browser_close`，实现在 `src/mcp/tools/browser.py`
|
||||
- `run_bot()` / `run_bot_web()`：一把梭编排，只留给手工验证与回归；新代码别用——中途失败只能整体重来
|
||||
|
||||
步骤函数（`fill_travel_info` / `add_travel_items` / `fill_normal_payment` 等）是两种用法共用的底座，
|
||||
**出错必须回抛**（`log.error` 之后要 `raise`）。吞异常会让上层以为填好了——本项目已经因为静默失败
|
||||
丢了 7 周的出库单产物（方案文档 §11.4），不要重犯。
|
||||
|
||||
## 线程与进程约束
|
||||
|
||||
- Playwright sync 的 browser/page 只能在创建它的线程里用；MCP 是多线程调用，所以一律经
|
||||
`actor.run_browser()` 提交到专属单线程执行（与 COM actor 分开，套间要求不同）
|
||||
- 浏览器归属写在 `cases/<id>/.browser.lock`（PID + 心跳；180 秒未更新可被别的进程接管）。
|
||||
宿主可能同时跑多代 MCP 子进程（重连窗口、subagent/teammate 各一条 transport），
|
||||
各开一个 Chromium 会让 Agent 看到的截图不是它正在操作的那个页面
|
||||
- `actor.py` 的 atexit 会尽力关浏览器并释放锁；被强杀时锁文件会留下，靠心跳过期接管
|
||||
|
||||
## 计划中的 P4
|
||||
|
||||
`submit_preview`（快照 + 摘要 + 一次性 token）与 `submit_execute`（token 一次性、摘要变更即失效）。
|
||||
在那之前，填报完成必须停下交人工核对提交。
|
||||
|
||||
## 注意
|
||||
|
||||
- 会打开 Chromium，自动化期间不要手动点页面
|
||||
- 调试截图落在 `images/`（已 gitignore，不进仓库）
|
||||
- 步骤函数最好返回「填了什么 / 什么失败」的结构，P3 可直接复用
|
||||
- 会打开 Chromium（默认有头），自动化期间不要手动点页面
|
||||
- 分步填报的截图写 `cases/<id>/screenshots/NNN-<步骤名>.png`；只有走 `run_bot()` 才写
|
||||
`images/debug_*.png`（已 gitignore）
|
||||
|
||||
@@ -0,0 +1,165 @@
|
||||
"""浏览器会话的常驻持有者与独占锁
|
||||
|
||||
两件事必须同时成立，填报才可信：
|
||||
|
||||
1. **线程亲和**：Playwright 同步 API 底层是 greenlet，browser/page 对象必须在创建它的那个线程里使用。
|
||||
MCP 服务器是多线程的（每个工具调用可能在不同线程），所以所有浏览器操作统一提交到这个单线程 actor。
|
||||
与 COM 的 actor 分开：COM 套间要求不同，混用会踩坑。
|
||||
2. **进程独占**：宿主可能同时存在多代 MCP 子进程（重连窗口、subagent/teammate 各一条 transport，
|
||||
实测 4 个 PID 并存）。每个进程各开一个 Chromium，Agent 看到的截图就不是它正在操作的那个页面。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import atexit
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
from collections.abc import Callable
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from ... import get_logger
|
||||
|
||||
log = get_logger("browser.actor")
|
||||
|
||||
LOCK_FILENAME = ".browser.lock"
|
||||
# 心跳超过这个秒数没更新就视为持有者已死，可被接管
|
||||
STALE_AFTER_SECONDS = 180
|
||||
|
||||
_executor: ThreadPoolExecutor | None = None
|
||||
_lock = threading.Lock()
|
||||
# case_id -> {"bot": BaseBot, "steps": [str, ...], "kind": str|None}
|
||||
_sessions: dict[str, dict[str, Any]] = {}
|
||||
|
||||
|
||||
def _pool() -> ThreadPoolExecutor:
|
||||
global _executor
|
||||
if _executor is None:
|
||||
_executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="playwright")
|
||||
return _executor
|
||||
|
||||
|
||||
def run_browser[T](fn: Callable[..., T], *args: Any, **kwargs: Any) -> T:
|
||||
"""在专用 Playwright 线程上串行执行 fn 并等待结果。"""
|
||||
with _lock:
|
||||
return _pool().submit(fn, *args, **kwargs).result()
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 会话注册表（只在 actor 线程里读写，不加锁也不竞争）
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def session(case_id: str) -> dict[str, Any] | None:
|
||||
"""取会话的浏览器上下文；未打开返回 None。"""
|
||||
return _sessions.get(case_id)
|
||||
|
||||
|
||||
def put_session(case_id: str, bot: Any, kind: str | None = None) -> dict[str, Any]:
|
||||
ctx: dict[str, Any] = {"bot": bot, "steps": [], "kind": kind, "opened_at": time.time()}
|
||||
_sessions[case_id] = ctx
|
||||
return ctx
|
||||
|
||||
|
||||
def drop_session(case_id: str) -> dict[str, Any] | None:
|
||||
return _sessions.pop(case_id, None)
|
||||
|
||||
|
||||
def list_sessions() -> list[str]:
|
||||
return sorted(_sessions)
|
||||
|
||||
|
||||
def mark_step(case_id: str, step: str) -> None:
|
||||
ctx = _sessions.get(case_id)
|
||||
if ctx is not None and step not in ctx["steps"]:
|
||||
ctx["steps"].append(step)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 磁盘锁
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def lock_path(case_dir: Path) -> Path:
|
||||
return case_dir / LOCK_FILENAME
|
||||
|
||||
|
||||
def read_lock(case_dir: Path) -> dict[str, Any] | None:
|
||||
"""读锁文件；损坏或不存在返回 None（损坏不阻断，按无锁处理）。"""
|
||||
path = lock_path(case_dir)
|
||||
if not path.exists():
|
||||
return None
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
except Exception as exc:
|
||||
log.warning("浏览器锁文件损坏，按无锁处理: %s (%s)", path, exc)
|
||||
return None
|
||||
return data if isinstance(data, dict) else None
|
||||
|
||||
|
||||
def _write_lock(case_dir: Path, url: str = "") -> dict[str, Any]:
|
||||
payload = {
|
||||
"pid": os.getpid(),
|
||||
"started_at": round(time.time(), 3),
|
||||
"heartbeat": round(time.time(), 3),
|
||||
"url": url,
|
||||
}
|
||||
lock_path(case_dir).write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
return payload
|
||||
|
||||
|
||||
def acquire(case_dir: Path, url: str = "") -> tuple[bool, dict[str, Any]]:
|
||||
"""尝试取得该会话的浏览器所有权。返回 (是否取得, 锁信息或占用者信息)。"""
|
||||
held = read_lock(case_dir)
|
||||
if held is not None:
|
||||
other_pid = int(held.get("pid", -1))
|
||||
age = time.time() - float(held.get("heartbeat", 0))
|
||||
if other_pid != os.getpid() and age < STALE_AFTER_SECONDS:
|
||||
return False, {**held, "age_seconds": round(age, 1), "stale": False}
|
||||
if other_pid != os.getpid():
|
||||
log.warning("接管过期浏览器锁（原持有 pid=%s，%.0fs 无心跳）", other_pid, age)
|
||||
return True, _write_lock(case_dir, url)
|
||||
|
||||
|
||||
def touch(case_dir: Path, url: str = "") -> None:
|
||||
"""刷心跳：每次浏览器工具调用都调一下。"""
|
||||
if read_lock(case_dir) is None:
|
||||
return
|
||||
_write_lock(case_dir, url)
|
||||
|
||||
|
||||
def release(case_dir: Path) -> None:
|
||||
held = read_lock(case_dir)
|
||||
if held is not None and int(held.get("pid", -1)) == os.getpid():
|
||||
try:
|
||||
lock_path(case_dir).unlink()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 收尾
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def close_all() -> None:
|
||||
"""尽力关闭所有浏览器并释放锁（进程退出时调用）。"""
|
||||
for case_id in list(_sessions):
|
||||
ctx = _sessions.pop(case_id, None)
|
||||
if not ctx:
|
||||
continue
|
||||
bot = ctx.get("bot")
|
||||
work_dir = getattr(bot, "work_dir", None)
|
||||
try:
|
||||
if bot is not None:
|
||||
bot.close()
|
||||
except Exception as exc:
|
||||
log.warning("关闭浏览器失败 %s: %s", case_id, exc)
|
||||
if isinstance(work_dir, Path):
|
||||
release(work_dir)
|
||||
|
||||
|
||||
atexit.register(close_all)
|
||||
+63
-29
@@ -38,6 +38,9 @@ class BaseBot:
|
||||
self.config = config
|
||||
self.headless = headless
|
||||
self.work_dir: Path | None = None
|
||||
# 设了就把截图按 001-步骤名.png 编号写进这里；不设保持旧行为（src/images/debug_*.png）
|
||||
self.screenshot_dir: Path | None = None
|
||||
self._shot_seq = 0
|
||||
self.browser: Any = None
|
||||
self.context: Any = None
|
||||
self.page: Any = None
|
||||
@@ -74,12 +77,44 @@ class BaseBot:
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
def login_portal(self) -> None:
|
||||
"""登录信息门户"""
|
||||
log.info("登录信息门户...")
|
||||
"""登录信息门户；一把梭版本，等价于 submit_login 后等门户出现。"""
|
||||
self.submit_login()
|
||||
if not self.wait_for_portal(30):
|
||||
self._screenshot("portal_timeout")
|
||||
raise TimeoutError("登录超时，未跳转到信息门户")
|
||||
|
||||
PORTAL_MARKERS = ("tyrz.fynu.edu.cn/zs-uip", "tyrz.fynu.edu.cn/oshall", "portal")
|
||||
|
||||
def portal_reached(self) -> bool:
|
||||
"""是否已落到信息门户；可反复调用判断登录是否完成。"""
|
||||
try:
|
||||
url = self.page.url or ""
|
||||
except Exception:
|
||||
return False
|
||||
return any(marker in url for marker in self.PORTAL_MARKERS)
|
||||
|
||||
def human_required(self) -> bool:
|
||||
"""页面是否卡在需要人工的地方（扫码、验证码、仍在 SSO 登录页）。"""
|
||||
try:
|
||||
if self.portal_reached():
|
||||
return False
|
||||
for selector in (
|
||||
'text="微信扫码登录"',
|
||||
'img[src*="qrcode"]',
|
||||
'img[src*="captcha"]',
|
||||
'input[placeholder*="验证码"]',
|
||||
):
|
||||
if self.page.query_selector(selector):
|
||||
return True
|
||||
return "login" in (self.page.url or "").lower()
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def submit_login(self) -> None:
|
||||
"""填工号密码、勾同意、点登录；不等跳转。"""
|
||||
log.info("登录信息门户...")
|
||||
self.page.goto(self.config["sso_login_url"], wait_until="domcontentloaded")
|
||||
self._wait_for('text="微信扫码登录"', timeout=5000)
|
||||
|
||||
try:
|
||||
self.page.fill(
|
||||
'input[placeholder*="工号"], input[placeholder*="学号"]',
|
||||
@@ -88,41 +123,27 @@ class BaseBot:
|
||||
self.page.fill('input[placeholder*="密码"]', self.config["password"])
|
||||
except Exception:
|
||||
log.warning("未找到登录输入框，可能已登录")
|
||||
|
||||
try:
|
||||
checkbox = self.page.query_selector('input[type="checkbox"]')
|
||||
if checkbox and not checkbox.is_checked():
|
||||
checkbox.click()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
for selector in ['button:has-text("登录")', 'input[value="登录"]', 'text="登录"']:
|
||||
for selector in ('button:has-text("登录")', 'input[value="登录"]', 'text="登录"'):
|
||||
try:
|
||||
self.page.click(selector, timeout=3000)
|
||||
break
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
self._wait_for_portal()
|
||||
|
||||
def _wait_for_portal(self) -> None:
|
||||
"""等待跳转到统一信息平台"""
|
||||
for _ in range(30):
|
||||
self.page.wait_for_timeout(1000)
|
||||
url = self.page.url
|
||||
if any(
|
||||
kw in url
|
||||
for kw in (
|
||||
"tyrz.fynu.edu.cn/zs-uip",
|
||||
"tyrz.fynu.edu.cn/oshall",
|
||||
"portal",
|
||||
)
|
||||
):
|
||||
def wait_for_portal(self, timeout: int = 30) -> bool:
|
||||
"""等待门户跳转；返回是否成功，不抛异常——分步填报靠它判 need_human。"""
|
||||
for _ in range(timeout):
|
||||
if self.portal_reached():
|
||||
self._screenshot("portal_loaded")
|
||||
return
|
||||
log.error("等待门户跳转超时")
|
||||
self._screenshot("portal_timeout")
|
||||
raise TimeoutError("登录超时，未跳转到信息门户")
|
||||
return True
|
||||
self.page.wait_for_timeout(1000)
|
||||
return self.portal_reached()
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# 导航
|
||||
@@ -197,7 +218,20 @@ class BaseBot:
|
||||
def _wait_for(self, selector: str, timeout: int | None = None) -> None:
|
||||
self.page.wait_for_selector(selector, timeout=timeout)
|
||||
|
||||
def _screenshot(self, name: str) -> None:
|
||||
img_dir = Path(__file__).parent.parent.parent / "images"
|
||||
img_dir.mkdir(exist_ok=True)
|
||||
self.page.screenshot(path=str(img_dir / f"debug_{name}.png"))
|
||||
def _screenshot(self, name: str) -> str:
|
||||
"""截图并返回落盘路径；截图失败只告警，别盖掉真正的填报错误。"""
|
||||
self._shot_seq += 1
|
||||
if self.screenshot_dir is not None:
|
||||
img_dir = Path(self.screenshot_dir)
|
||||
img_dir.mkdir(parents=True, exist_ok=True)
|
||||
target = img_dir / f"{self._shot_seq:03d}-{name}.png"
|
||||
else:
|
||||
img_dir = Path(__file__).parent.parent.parent / "images"
|
||||
img_dir.mkdir(exist_ok=True)
|
||||
target = img_dir / f"debug_{name}.png"
|
||||
try:
|
||||
self.page.screenshot(path=str(target))
|
||||
except Exception as exc:
|
||||
log.warning("截图失败 %s: %s", target.name, exc)
|
||||
return ""
|
||||
return str(target)
|
||||
|
||||
Reference in New Issue
Block a user