Compare commits
2 Commits
e252896de9
...
agent
| Author | SHA1 | Date | |
|---|---|---|---|
|
1b35f07fd7 | ||
|
|
7c137c5214 |
@@ -0,0 +1,60 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# 补充材料提交后 SSE 立即读到旧 result.json 导致前端无消息
|
||||
|
||||
## 错误现象
|
||||
|
||||
- 第二次补充材料提交后，前端没有任何消息显示
|
||||
- 状态栏不更新，聊天区无新增消息
|
||||
- 后台日志显示处理正常完成（LLM 提取、校验、Bot 提交均成功）
|
||||
- 前端像是"卡住"了一样，没有报错也没有反馈
|
||||
|
||||
## 触发条件
|
||||
|
||||
1. 第一轮处理或第一轮补充材料完成，`result.json` 已写入 session 目录
|
||||
2. 用户再次补充材料，触发新一轮处理
|
||||
3. 前端创建新的 SSE 连接到 `/api/logs/<session_id>`
|
||||
4. SSE 端点轮询时立即检测到旧的 `result.json`，直接发射 `done` 事件并关闭连接
|
||||
5. 前端断开 SSE，但后台线程仍在执行新任务
|
||||
|
||||
## 根因
|
||||
|
||||
`_run_agent_task` 在每次任务启动时只清理了 `llm_stream.log`，未清理 `result.json` 和 `agent_events.log`。
|
||||
|
||||
```python
|
||||
# 修复前 - 只清理了 llm_stream.log
|
||||
try:
|
||||
(session_dir / "llm_stream.log").unlink(missing_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
```
|
||||
|
||||
SSE 端点 (`/api/logs/<session_id>`) 在 `generate()` 中轮询检查 `result.json` 是否存在，一旦存在就发射 `done` 事件并 `break` 退出循环。旧的 `result.json` 未被清理，导致 SSE 连接在任务实际开始前就结束了。
|
||||
|
||||
## 修复
|
||||
|
||||
在 `_run_agent_task` 开头统一清理三个残留文件：
|
||||
|
||||
```python
|
||||
# 修复后 - 同时清理三个残留文件
|
||||
for fname in ("llm_stream.log", "agent_events.log", pipeline_web.SESSION_RESULT_FILE):
|
||||
try:
|
||||
(session_dir / fname).unlink(missing_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
```
|
||||
|
||||
- `llm_stream.log` — LLM 流式日志
|
||||
- `agent_events.log` — Agent 事件日志（避免旧事件被重放）
|
||||
- `result.json` — 处理结果文件（避免 SSE 立即读到旧结果）
|
||||
|
||||
## 影响范围
|
||||
|
||||
所有使用 `_run_agent_task` 的端点均受影响：
|
||||
- `/api/process/<session_id>` — 初始处理
|
||||
- `/api/agent/supplement/<session_id>` — 补充文件
|
||||
- `/api/agent/user-supplement/<session_id>` — 文字补充
|
||||
- `/api/agent/force-submit/<session_id>` — 强制提交
|
||||
- `/api/submit-financial/<session_id>` — 手动财务提交
|
||||
@@ -1,6 +1,6 @@
|
||||
# 系统事件流全景图
|
||||
|
||||
> 最后更新: 2026-06-13
|
||||
> 最后更新: 2026-06-15
|
||||
> 用途: 排查 SSE 事件问题、提交流程中断、状态不一致等 Bug
|
||||
|
||||
---
|
||||
@@ -15,6 +15,8 @@
|
||||
| `processing` | `EXTRACTING` | LLM 正在分析文件 |
|
||||
| `awaiting_supplement` | `AWAITING_SUPPLEMENT` | 信息不完整，等待用户补充 |
|
||||
| `submitting` | `SUBMITTING` | 正在提交到财务系统 |
|
||||
| `ready` | `READY` | 信息完整，可以提交 |
|
||||
| `submitting` | `SUBMITTING` | 正在提交到财务系统 |
|
||||
| `done` | `DONE` / `ERROR` | 流程结束（成功或失败） |
|
||||
|
||||
### 1.2 通信机制
|
||||
@@ -41,6 +43,55 @@ sequenceDiagram
|
||||
- `result.json` 的原子写入：先写 `.tmp`，再 `replace()` 重命名
|
||||
- SSE 超时：600 秒后自动断开
|
||||
|
||||
### 1.3 操作信号点清单
|
||||
|
||||
每个 API 操作涉及的信号文件生命周期如下。**新增或修改信号文件时必须同步更新此清单**。
|
||||
|
||||
| 序号 | 操作 | API 端点 | 线程启动时清理 | 一次写入且不被清理 | 轮次结束时写入 |
|
||||
|---|---|---|---|---|---|
|
||||
| 1 | 初始处理 | `POST /api/agent/process/:sid` | `llm_stream.log`, `agent_events.log`, `result.json` | `file_events.log`, `session.log` | `result.json` |
|
||||
| 2 | 补充文件 | `POST /api/agent/supplement/:sid` | `llm_stream.log`, `agent_events.log`, `result.json` | `file_events.log`, `session.log` | `result.json` |
|
||||
| 3 | 文字补充 | `POST /api/agent/user-supplement/:sid` | `llm_stream.log`, `agent_events.log`, `result.json` | `file_events.log`, `session.log` | `result.json` |
|
||||
| 4 | 强制提交 | `POST /api/agent/force-submit/:sid` | `llm_stream.log`, `agent_events.log`, `result.json` | `file_events.log`, `session.log` | `result.json` |
|
||||
| 5 | 手动财务提交 | `POST /api/submit-financial/:sid` | `llm_stream.log`, `agent_events.log`, `result.json` | `file_events.log`, `session.log` | `result.json` |
|
||||
|
||||
**信号点说明**：
|
||||
|
||||
| 信号文件 | 读/写方 | 生命周期 | 作用 |
|
||||
|---|---|---|---|
|
||||
| `result.json` | 后端线程写入，SSE 端点读取 | 每轮开始时删除，`finally` 块中原子写入 | SSE 检测到该文件即发射 `done` 事件并断开连接 |
|
||||
| `agent_events.log` | Agent 调度器追加写入，SSE 端点读取 | 每轮开始时删除，Agent 运行时持续追加 | 传递 agent 状态变化事件给前端 |
|
||||
| `llm_stream.log` | LLM 回调追加写入，SSE 端点读取 | 每轮开始时删除，LLM 运行时持续追加 | 传递 LLM 流式输出给前端 |
|
||||
| `file_events.log` | `pipeline_web` 追加写入，SSE 端点读取 | 会话内持续追加，不删除 | 传递文件处理进度给前端 |
|
||||
| `session.log` | `sse_handler` 追加写入，SSE 端点读取 | 会话内持续追加，不删除 | 传递普通日志行给前端 |
|
||||
|
||||
### 1.4 关键约束（修改代码前必读）
|
||||
|
||||
**约束 1：`result.json` 必须在每轮线程启动时删除**
|
||||
|
||||
SSE 端点通过检测 `result.json` 是否存在来判断任务是否完成。如果上一轮的 `result.json` 残留，SSE 会立即读到旧数据并发射 `done` 事件，导致前端断开连接，新任务的消息无法送达。
|
||||
|
||||
- 实现位置：`_run_agent_task()` 的 `try` 块开头
|
||||
- 删除时机：在 `install_log_collector()` 之后、`task_fn()` 执行之前
|
||||
- 写入位置：`finally` 块中统一写入（唯一写入点）
|
||||
- 写入规则：`finally` 始终执行原子写入，不再有条件判断
|
||||
- `_emit_ready_and_submit` 只返回 result 字典，不写入文件
|
||||
|
||||
**约束 2：`result.json` 的写入必须使用 `finally` 块**
|
||||
|
||||
无论任务成功或失败，SSE 端点都需要 `result.json` 来发送 `done` 事件。如果仅在成功路径写入，异常时 SSE 会一直轮询直到 600 秒超时，前端无反馈。
|
||||
|
||||
**约束 3：SSE 新建连接时，当前文件偏移必须从 0 开始**
|
||||
|
||||
`_run_agent_task` 在启动时删除 `llm_stream.log` 和 `agent_events.log`，确保 SSE 重新建立连接后从 0 偏移开始读取。如果文件不被删除，旧的事件会被重复发送给前端。
|
||||
|
||||
**约束 4：前端 SSE 连接的生命周期**
|
||||
|
||||
- 前端在每次 POST 请求返回 `{status: "started"}` 后立即创建新的 SSE 连接
|
||||
- 收到 `done` 事件后关闭连接
|
||||
- 旧的连接引用必须清理（`agent.js` 中的 `agentEventSource`）
|
||||
- 如果前端在 POST 之前就创建了 SSE 连接，会读到旧数据
|
||||
|
||||
---
|
||||
|
||||
## 二、场景一：用户提交材料 → LLM 分析完整 → 直接提交
|
||||
@@ -437,7 +488,7 @@ stateDiagram-v2
|
||||
|
||||
| 事件类型 | 数据结构 | 触发条件 |
|
||||
|---|---|---|
|
||||
| `llm_stream` | `{type, phase, content?}` | LLM 输出流 |
|
||||
| `llm_stream` | `{type, phase, text?}` | LLM 输出流 |
|
||||
|
||||
`phase` 取值: `start` / `reasoning` / `chunk` / `end` / `error`
|
||||
|
||||
@@ -491,6 +542,21 @@ stateDiagram-v2
|
||||
3. 检查新 SSE 连接是否成功建立
|
||||
4. 确认 `add_supplement()` 或 `process_user_text_supplement()` 是否被调用
|
||||
|
||||
### 9.5 补充材料后前端无任何消息（`result.json` 残留问题）
|
||||
|
||||
**症状**: 第二轮及之后的补充材料提交后，前端完全没有任何消息显示，状态栏不更新，聊天区无新增消息。后台日志显示处理正常完成。
|
||||
|
||||
**根因**: `_run_agent_task` 在每轮启动时未清理上一轮的 `result.json`。SSE 端点轮询时立即检测到旧的 `result.json`，直接发射 `done` 事件并关闭连接，前端断开后无法接收新任务的消息。
|
||||
|
||||
**排查步骤**:
|
||||
1. 检查 session 目录中 `result.json` 的修改时间 — 如果早于当前轮次开始时间，说明是残留文件
|
||||
2. 检查浏览器 Network 面板中 SSE 连接 — 是否在建立后立即收到 `done` 事件
|
||||
3. 确认 `_run_agent_task` 是否在启动时清理了 `result.json`
|
||||
|
||||
**修复**: 在 `_run_agent_task` 的 `try` 块开头同时清理 `llm_stream.log`、`agent_events.log` 和 `result.json` 三个文件。
|
||||
|
||||
**详细记录**: 参见 `.agents/docs/error-experience/2026-06-15-补充材料SSE立即读到旧result.json导致前端无消息.md`
|
||||
|
||||
---
|
||||
|
||||
## 十、关键文件索引
|
||||
@@ -504,3 +570,88 @@ stateDiagram-v2
|
||||
| `src/agent/orchestrator.py` | Agent 调度器，状态机，校验循环 |
|
||||
| `src/web/sse_handler.py` | SSE 日志收集器 |
|
||||
| `src/web/pipeline_web.py` | 发票提取管道，财务提交 |
|
||||
|
||||
---
|
||||
|
||||
## 十一、文件生命周期与操作信号点
|
||||
|
||||
### 11.1 单轮处理的完整文件生命周期
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant API as 路由层
|
||||
participant RT as _run_agent_task
|
||||
participant TF as task_fn
|
||||
participant SS as _emit_ready_and_submit
|
||||
participant SSE as SSE 端点
|
||||
|
||||
Note over API: 1. 创建 handler
|
||||
API->>API: install_log_collector(session_dir)
|
||||
Note over API: 创建 SSE 日志收集器<br/>随即开始写入 session.log
|
||||
|
||||
API->>RT: threading.Thread(target=_run_agent_task)
|
||||
|
||||
Note over RT: 2. 清理残留文件
|
||||
RT->>RT: unlink(llm_stream.log)
|
||||
RT->>RT: unlink(agent_events.log)
|
||||
RT->>RT: unlink(result.json)
|
||||
|
||||
Note over RT: 3. 执行任务
|
||||
RT->>TF: task_fn(session_dir, config)
|
||||
|
||||
Note over TF: 执行期间各个文件由对应模块写入：
|
||||
TF-->>TF: file_events.log (pipeline_web)
|
||||
TF-->>TF: llm_stream.log (LLM 回调)
|
||||
TF-->>TF: agent_events.log (Agent 调度器)
|
||||
|
||||
TF-->>RT: 返回 (agent_session, 占位 result)
|
||||
|
||||
alt 成功路径 (READY)
|
||||
RT->>SS: _emit_ready_and_submit()
|
||||
Note over SS: 发射 agent_ready 事件<br/>执行财务提交<br/>返回 result 字典
|
||||
SS-->>RT: result 字典
|
||||
Note over RT: result = {...}
|
||||
else 需补充路径 (AWAITING_SUPPLEMENT)
|
||||
Note over RT: result = {waiting_for_supplement: true}
|
||||
else 异常路径
|
||||
Note over RT: result = {ok: false, error: ...}
|
||||
end
|
||||
|
||||
Note over RT: 4. finally 块 — 唯一写入点
|
||||
RT->>RT: 原子写入 result.json (.tmp → replace)
|
||||
|
||||
RT->>RT: remove_log_collector(handler)
|
||||
|
||||
Note over SSE: 5. SSE 端点检测
|
||||
SSE->>SSE: 轮询检测到 result.json
|
||||
SSE-->>SSE: 发射 done 事件
|
||||
SSE->>SSE: break 退出轮询
|
||||
```
|
||||
|
||||
### 11.2 各阶段信号文件状态
|
||||
|
||||
| 阶段 | `result.json` | `llm_stream.log` | `agent_events.log` | `file_events.log` | `session.log` |
|
||||
|---|---|---|---|---|---|
|
||||
| 会话创建 | 不存在 | 不存在 | 不存在 | 不存在 | 不存在 |
|
||||
| `install_log_collector` 后 | 不存在 | 不存在 | 不存在 | 不存在 | 开始写入 |
|
||||
| `_run_agent_task` 清理后 | 已删除 | 已删除 | 已删除 | 保持 | 保持 |
|
||||
| 文件提取中 | 不存在 | 不存在 | 不存在 | 持续追加 | 持续追加 |
|
||||
| LLM 提取中 | 不存在 | 持续追加 | 持续追加 | 保持 | 持续追加 |
|
||||
| 校验中 | 不存在 | 保持 | 持续追加 | 保持 | 持续追加 |
|
||||
| 任务完成 (READY) | 已写入 | 保持 | 保持 | 保持 | 保持 |
|
||||
| 任务完成 (需补充) | 已写入 | 保持 | 保持 | 保持 | 保持 |
|
||||
| 任务异常 | 已写入 | 保持 | 保持 | 保持 | 保持 |
|
||||
| SSE done 事件后 | 保持 | 保持 | 保持 | 保持 | 保持 |
|
||||
|
||||
### 11.3 新增信号文件检查清单
|
||||
|
||||
当需要在系统中新增一个信号文件（如 `submit_progress.log`）时，必须检查以下事项：
|
||||
|
||||
1. **写入方**：哪个模块负责写入？写入时机是什么？
|
||||
2. **读取方**：SSE 端点是否需要轮询？前端是否需要处理？
|
||||
3. **清理时机**：是否需要在 `_run_agent_task` 中清理？如果不需要，为什么？
|
||||
4. **原子性**：写入是否需要 `.tmp` + `replace` 模式？
|
||||
5. **轮询偏移**：SSE 端点是否需要跟踪该文件的读取偏移？
|
||||
6. **更新本文档**：在 1.3 操作信号点清单中新增一行，在 11.2 文件状态表中新增一列
|
||||
7. **更新 `_run_agent_task`**：如果需要清理，在清理循环中添加文件名
|
||||
8. **更新前端**：在 `sse.js` 或 `agent.js` 中添加对应的事件处理器
|
||||
401
.agents/docs/guides/项目架构全景图.md
Normal file
401
.agents/docs/guides/项目架构全景图.md
Normal file
@@ -0,0 +1,401 @@
|
||||
# 项目架构全景图
|
||||
|
||||
> 最后更新: 2026-06-15
|
||||
> 用途: 理解项目整体结构、模块职责、依赖关系和数据流
|
||||
|
||||
---
|
||||
|
||||
## 一、分层架构总览
|
||||
|
||||
```
|
||||
src/
|
||||
├── agent/ Agent 调度层（协调提取-校验-修正循环，状态机管理）
|
||||
├── core/ 核心业务层（纯逻辑，零框架依赖）
|
||||
├── infra/ 基础设施层（浏览器、文档、LLM 提示词）
|
||||
├── web/ Web 界面层（Flask + SSE）
|
||||
├── pipeline.py CLI 流程编排
|
||||
├── pipeline_core.py CLI/Web 公共管道逻辑
|
||||
├── main.py CLI 入口
|
||||
├── config.py 配置加载
|
||||
└── exceptions.py 异常定义
|
||||
```
|
||||
|
||||
### 依赖方向
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
classDef entry fill:#e8eaf6,stroke:#3f51b5,color:#1a237e
|
||||
classDef orchestrate fill:#e0f2f1,stroke:#00897b,color:#004d40
|
||||
classDef agent fill:#fff8e1,stroke:#ff8f00,color:#3e2723
|
||||
classDef core fill:#e3f2fd,stroke:#1565c0,color:#0d47a1
|
||||
classDef infra fill:#e8f5e9,stroke:#2e7d32,color:#1b5e20
|
||||
|
||||
subgraph 入口层
|
||||
CLI["main.py"]:::entry
|
||||
WEB["web/app.py"]:::entry
|
||||
end
|
||||
|
||||
subgraph 编排层
|
||||
PIPE["pipeline.py"]:::orchestrate
|
||||
PIPE_WEB["web/pipeline_web.py"]:::orchestrate
|
||||
PIPE_CORE["pipeline_core.py"]:::orchestrate
|
||||
end
|
||||
|
||||
subgraph Agent调度层
|
||||
AGENT["agent/orchestrator.py"]:::agent
|
||||
SESSION["agent/session.py"]:::agent
|
||||
EVENTS["agent/events.py"]:::agent
|
||||
end
|
||||
|
||||
subgraph 核心业务层
|
||||
EXTRACT["core/extraction/"]:::core
|
||||
MATCH["core/matching/"]:::core
|
||||
VALID["core/validation/"]:::core
|
||||
end
|
||||
|
||||
subgraph 基础设施层
|
||||
BROWSER["infra/browser/"]:::infra
|
||||
DOCS["infra/documents/"]:::infra
|
||||
LLM["infra/llm/"]:::infra
|
||||
end
|
||||
|
||||
CLI --> PIPE
|
||||
WEB --> PIPE_WEB
|
||||
PIPE --> PIPE_CORE
|
||||
PIPE --> EXTRACT
|
||||
PIPE --> BROWSER
|
||||
PIPE_WEB --> AGENT
|
||||
PIPE_WEB --> PIPE_CORE
|
||||
PIPE_WEB --> EXTRACT
|
||||
AGENT --> EXTRACT
|
||||
AGENT --> VALID
|
||||
AGENT --> LLM
|
||||
AGENT --> PIPE_CORE
|
||||
EXTRACT --> MATCH
|
||||
EXTRACT --> DOCS
|
||||
EXTRACT --> LLM
|
||||
MATCH --> DOCS
|
||||
BROWSER --> DOCS
|
||||
```
|
||||
|
||||
**关键约束**：
|
||||
- `infra` 不依赖 `core` 和 `agent`，只提供工具能力
|
||||
- `core` 零外部依赖，不依赖 Flask、Playwright 等框架
|
||||
- `agent` 依赖 `core` 和 `infra`，作为调度中枢编排各模块
|
||||
- 所有跨层调用均通过 `__init__.py` 导出的稳定接口
|
||||
|
||||
---
|
||||
|
||||
## 二、模块清单
|
||||
|
||||
### 2.1 Agent 调度层 (`src/agent/`)
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `coordinator.py` | 核心协调逻辑：提取-校验-修正循环（最多 3 次重试）、用户补充处理、强制提交 |
|
||||
| `session.py` | 会话状态：`AgentState` 枚举、`AgentSession` 数据类、状态持久化（原子写入） |
|
||||
| `events.py` | SSE 事件发射：事件去重、事件日志追加、事件读取 |
|
||||
| `orchestrator.py` | 兼容层：从子模块重新导出所有符号，保持旧导入路径可用 |
|
||||
|
||||
**对外接口**：`AgentSession`, `AgentState`, `run_agent_round()`, `force_submit()`, `add_supplement()`, `process_user_text_supplement()`, `load_agent_state()`, `save_agent_state()`
|
||||
|
||||
### 2.2 核心业务层 (`src/core/`)
|
||||
|
||||
| 子模块 | 职责 | 对外接口 |
|
||||
|------|------|------|
|
||||
| `extraction/extractor.py` | 编排入口：扫描目录 → 逐文件提取 → 分类 → 金额匹配 | `extract_invoices()`, `extract_document()` |
|
||||
| `extraction/llm_extractor.py` | LLM 多模态提取核心：统一文档提取、差旅/普通信息提取、缓存管理、SSE 流式事件 | `llm_query_text()`, `extract_travel_info()`, `extract_normal_info()`, `load_cache()` |
|
||||
| `matching/matcher.py` | 发票与支付记录按金额匹配（一对一 / 一对多贪心，相对容差 3%） | `match_invoices_to_cards()` |
|
||||
| `validation/validator.py` | 声明式规则校验引擎，规则从 JSON 配置文件加载 | `validate_extracted_info()`, `ValidationReport` |
|
||||
|
||||
### 2.3 基础设施层 (`src/infra/`)
|
||||
|
||||
| 子模块 | 职责 | 对外接口 |
|
||||
|------|------|------|
|
||||
| `browser/base.py` | `BaseBot` 基类：Playwright 浏览器生命周期、登录、导航、截图 | 内部基类 |
|
||||
| `browser/travel.py` | 差旅报销填报：基本信息 → 明细 → 支付 → 补助 → 附件上传 | 内部流程 |
|
||||
| `browser/normal.py` | 普通报销填报：基本信息 → 总明细 → 支付 → 附件上传 | 内部流程 |
|
||||
| `browser/__init__.py` | 浏览器入口：类型路由和流程调度 | `run_bot()`, `run_bot_web()` |
|
||||
| `documents/invoice.py` | 发票数据模型、CSV/JSON 读写、发票分类 | `load_csv()`, `save_csv()`, `save_invoice_csv()`, `classify_invoice_batch()` |
|
||||
| `documents/pdf.py` | PDF 渲染为图片（PyMuPDF） | `render_pdf_to_images()` |
|
||||
| `documents/consumable.py` | 易耗品出库单填写：CSV → Word 模板 | `fill_consumable_doc()` |
|
||||
| `llm/prompt.py` | LLM 提示词加载 | `build_invoice_system_prompt()`, `build_travel_info_system_prompt()`, `build_normal_info_system_prompt()` |
|
||||
|
||||
### 2.4 Web 界面层 (`src/web/`)
|
||||
|
||||
| 文件/目录 | 职责 |
|
||||
|------|------|
|
||||
| `app.py` | Flask 应用入口，注册蓝图和模板 |
|
||||
| `routes.py` | 路由定义：会话管理、文件上传、配置、SSE 日志流、Agent 交互 API |
|
||||
| `pipeline_web.py` | Web 管道逻辑：发票提取 + 出库单生成 + 财务提交 |
|
||||
| `sse_handler.py` | SSE 日志收集器、日志转义、文件轮询 |
|
||||
| `templates/` | `index.html`（PC 端主界面）、`mobile_upload.html`（移动端上传） |
|
||||
| `static/js/` | 前端逻辑（按加载顺序）：`state.js` → `utils.js` → `chat.js` → `upload.js` → `config.js` → `process.js` → `sync.js` → `index.js` |
|
||||
|
||||
---
|
||||
|
||||
## 三、CLI 模式数据流
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
CLI_ENTRY["main.py --step all"] --> PIPE["pipeline.py run_pipeline()"]
|
||||
|
||||
subgraph Step1["Step 1: 发票提取"]
|
||||
PIPE --> EXT["core/extraction/extractor.py extract_invoices()"]
|
||||
EXT --> DOC["逐文件提取"]
|
||||
DOC --> LLM["LLM 多模态识别 (infra/llm)"]
|
||||
LLM --> CLASS["分类: train/hotel/general/payment/application"]
|
||||
CLASS --> MATCH["core/matching/matcher.py 金额匹配"]
|
||||
MATCH --> SAVE["infra/documents/ CSV/JSON 保存"]
|
||||
end
|
||||
|
||||
subgraph Step2["Step 2: 信息提取"]
|
||||
SAVE --> TYPE{"判断报销类型"}
|
||||
TYPE -->|差旅| TRAVEL["提取差旅信息 → travel_info.json"]
|
||||
TYPE -->|普通| NORMAL["提取普通发票信息 → normal_info.json"]
|
||||
end
|
||||
|
||||
subgraph Step3["Step 3: 浏览器填报"]
|
||||
TRAVEL --> BOT["infra/browser/ 填报"]
|
||||
NORMAL --> BOT
|
||||
BOT -->|差旅| BOT_T["browser/travel.py"]
|
||||
BOT -->|普通| BOT_N["browser/normal.py"]
|
||||
end
|
||||
```
|
||||
|
||||
**关键文件输出**：
|
||||
|
||||
| 文件 | 来源 | 说明 |
|
||||
|------|------|------|
|
||||
| `payment_records.csv` | Step 1 | 支付记录级别（每笔刷卡记录一行） |
|
||||
| `invoice_summary.csv` | Step 1 | 发票级别（每张发票一行） |
|
||||
| `travel_applications.json` | Step 1 | 出差事前申请单 |
|
||||
| `invoice_groups.json` | Step 1 | 发票分类结果 |
|
||||
| `travel_info.json` | Step 2 | 差旅信息：交通/住宿明细、补贴、附件清单 |
|
||||
| `normal_info.json` | Step 2 | 普通发票信息：报销说明、发票总数、总金额、附件清单 |
|
||||
|
||||
---
|
||||
|
||||
## 四、Web 模式数据流
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F as 前端 (浏览器)
|
||||
participant API as routes.py
|
||||
participant PW as pipeline_web.py
|
||||
participant AG as agent/orchestrator.py
|
||||
participant EX as core/extraction/
|
||||
participant VA as core/validation/
|
||||
participant SSE as SSE 轮询
|
||||
|
||||
F->>API: POST /api/session → 创建 session
|
||||
F->>API: POST /api/upload/:sid → 上传文件
|
||||
F->>API: POST /api/agent/process/:sid
|
||||
API-->>F: {status: "started"}
|
||||
F->>SSE: GET /api/logs/:sid (SSE 长连接)
|
||||
|
||||
Note over API: 后台 daemon 线程启动
|
||||
|
||||
API->>PW: extract_invoices(session_dir)
|
||||
PW->>EX: 发票提取 + 分类 + 匹配
|
||||
EX-->>PW: payment_records, applications, groups
|
||||
|
||||
API->>AG: run_agent_round(session_dir, session)
|
||||
loop 校验-修正循环 (最多 3 次)
|
||||
AG->>EX: llm_query_text() 提取信息
|
||||
AG->>VA: validate_extracted_info() 规则校验
|
||||
alt 校验失败
|
||||
AG->>AG: 构建修正提示
|
||||
end
|
||||
end
|
||||
AG-->>API: session (READY 或 AWAITING_SUPPLEMENT)
|
||||
|
||||
SSE-->>F: file_progress, llm_stream, agent_state_change, agent_ready/agent_request_supplement
|
||||
|
||||
API->>API: 写入 result.json
|
||||
SSE-->>F: done (携带 result)
|
||||
F->>F: 关闭 SSE, 展示结果
|
||||
```
|
||||
|
||||
### Web 模式特有的 Agent 调度
|
||||
|
||||
CLI 模式中 `pipeline.py` 直接调用 `extract_invoices()` → `infra/browser/`，不经过 Agent 层。
|
||||
|
||||
Web 模式中 `routes.py` 启动后台线程，调用 `agent/orchestrator.py` 作为调度中枢：
|
||||
|
||||
```
|
||||
run_agent_round()
|
||||
├── 1. load_cache() — 检查缓存
|
||||
├── 2. _do_extraction_with_validation() — 提取-校验-修正循环
|
||||
│ ├── llm_query_text() — LLM 提取结构化信息
|
||||
│ ├── validate_extracted_info() — 规则校验
|
||||
│ └── 校验失败 → 构建修正提示 → 再次调用 LLM (最多 3 次)
|
||||
├── 3. 判断 can_submit 字段
|
||||
│ ├── true → READY → 自动触发财务提交
|
||||
│ └── false → AWAITING_SUPPLEMENT → 等待用户补充
|
||||
├── 4. 用户补充处理
|
||||
│ ├── add_supplement() — 记录补充文件
|
||||
│ └── process_user_text_supplement() — LLM 解析文字补充
|
||||
└── 5. save_agent_state() — 持久化状态
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 五、Agent 状态机
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> IDLE: 会话创建
|
||||
|
||||
IDLE --> EXTRACTING: POST /api/agent/process
|
||||
IDLE --> EXTRACTING: POST /api/agent/supplement
|
||||
IDLE --> EXTRACTING: POST /api/agent/user-supplement
|
||||
|
||||
EXTRACTING --> READY: can_submit == true
|
||||
EXTRACTING --> AWAITING_SUPPLEMENT: can_submit == false
|
||||
EXTRACTING --> ERROR: 异常 / 轮次超限
|
||||
|
||||
READY --> SUBMITTING: _emit_ready_and_submit()
|
||||
SUBMITTING --> DONE: 财务提交完成
|
||||
|
||||
AWAITING_SUPPLEMENT --> EXTRACTING: 用户补充文件/文字
|
||||
AWAITING_SUPPLEMENT --> READY: 用户强制提交
|
||||
|
||||
note right of EXTRACTING
|
||||
LLM 提取 + validator 校验
|
||||
最多 3 次重试
|
||||
end note
|
||||
```
|
||||
|
||||
### 终态保护
|
||||
|
||||
以下状态为终态，再次触发 `run_agent_round()` 会被跳过：
|
||||
- `DONE` — 提交完成
|
||||
- `SUBMITTING` — 提交中
|
||||
- `READY` — 准备提交
|
||||
|
||||
### 轮次保护
|
||||
|
||||
默认最多 5 轮（`AgentSession.max_rounds`），超限后进入 `ERROR` 状态，用户可选择强制提交。
|
||||
|
||||
---
|
||||
|
||||
## 六、SSE 事件通信机制
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph 后端写入
|
||||
AGENT[agent/orchestrator.py] -->|追加写入| AE[agent_events.log]
|
||||
LLM[LLM 回调] -->|追加写入| LS[llm_stream.log]
|
||||
PW[pipeline_web.py] -->|追加写入| FE[file_events.log]
|
||||
SH[sse_handler.py] -->|追加写入| SL[session.log]
|
||||
RT[_run_agent_task] -->|finally 原子写入| RJ[result.json]
|
||||
end
|
||||
|
||||
subgraph SSE 轮询 (0.5s)
|
||||
POLL[SSE 端点] -->|读取| AE
|
||||
POLL -->|读取| LS
|
||||
POLL -->|读取| FE
|
||||
POLL -->|读取| SL
|
||||
POLL -->|检测| RJ
|
||||
end
|
||||
|
||||
POLL -->|event: agent_*| FRONT[前端 agent.js]
|
||||
POLL -->|event: llm_stream| FRONT
|
||||
POLL -->|event: file_progress| FRONT
|
||||
POLL -->|event: done| FRONT
|
||||
```
|
||||
|
||||
### 信号文件生命周期
|
||||
|
||||
| 阶段 | `result.json` | `llm_stream.log` | `agent_events.log` | `file_events.log` | `session.log` |
|
||||
|------|:--:|:--:|:--:|:--:|:--:|
|
||||
| 会话创建 | 不存在 | 不存在 | 不存在 | 不存在 | 不存在 |
|
||||
| 后台线程启动 | 已删除 | 已删除 | 已删除 | 保持 | 保持 |
|
||||
| 文件提取中 | 不存在 | 不存在 | 不存在 | 持续追加 | 持续追加 |
|
||||
| LLM 提取中 | 不存在 | 持续追加 | 持续追加 | 保持 | 持续追加 |
|
||||
| 校验中 | 不存在 | 保持 | 持续追加 | 保持 | 持续追加 |
|
||||
| 任务完成 | 已写入 | 保持 | 保持 | 保持 | 保持 |
|
||||
| SSE done 事件 | 保持 | 保持 | 保持 | 保持 | 保持 |
|
||||
|
||||
---
|
||||
|
||||
## 七、发票类型路由
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
INPUT["上传文件 (PDF/图片)"] --> EXT["LLM 多模态识别"]
|
||||
EXT --> TYPE{"invoice_type?"}
|
||||
|
||||
TYPE -->|train| TRAVEL["差旅报销流程"]
|
||||
TYPE -->|hotel| TRAVEL
|
||||
TYPE -->|general| NORMAL["普通报销流程"]
|
||||
TYPE -->|payment| MATCH["参与金额匹配"]
|
||||
TYPE -->|application| APP["存储为 JSON"]
|
||||
|
||||
TRAVEL --> TRAVEL_INFO["提取差旅信息<br/>travel_info.json"]
|
||||
TRAVEL_INFO --> TRAVEL_BOT["browser/travel.py<br/>填报差旅报销单"]
|
||||
|
||||
NORMAL --> NORMAL_INFO["提取普通发票信息<br/>normal_info.json"]
|
||||
NORMAL_INFO --> NORMAL_BOT["browser/normal.py<br/>填报普通报销单"]
|
||||
NORMAL_INFO --> CONSUMABLE["生成易耗品出库单<br/>(仅普通报销)"]
|
||||
|
||||
MATCH --> MERGE["合并到对应发票组"]
|
||||
|
||||
style TRAVEL fill:#cfe2ff,stroke:#0d6efd
|
||||
style NORMAL fill:#f8d7da,stroke:#dc3545
|
||||
style MATCH fill:#d1e7dd,stroke:#198754
|
||||
style APP fill:#fff3cd,stroke:#ffc107
|
||||
```
|
||||
|
||||
| 发票类型 | `invoice_type` | 报销流程 | 生成出库单 |
|
||||
|----------|---------------|---------|:--:|
|
||||
| 高铁票/火车票 | `train` | 差旅报销 | 否 |
|
||||
| 酒店住宿 | `hotel` | 差旅报销 | 否 |
|
||||
| 普通发票 | `general` | 普通报销 | 是 |
|
||||
| 支付记录 | `payment` | 参与匹配 | 否 |
|
||||
| 出差申请单 | `application` | 单独存储 | 否 |
|
||||
|
||||
> 差旅发票和普通发票不支持混报，混合时系统按普通报销处理。
|
||||
|
||||
---
|
||||
|
||||
## 八、设计原则
|
||||
|
||||
| 原则 | 说明 |
|
||||
|------|------|
|
||||
| **Agent 是调度中枢** | 校验-修正循环由 Agent 编排，不内嵌在 `llm_extractor` 中 |
|
||||
| **模块职责单一** | `llm_extractor` 只管提取，`validator` 只管校验，Agent 负责编排 |
|
||||
| **core 零外部依赖** | 不依赖 Flask、Playwright 等框架 |
|
||||
| **infra 不依赖业务** | 基础设施层只提供工具能力，不包含业务逻辑 |
|
||||
| **缓存优先** | 信息提取优先读取 `.invoice_cache`，避免重复调用 LLM |
|
||||
| **轮次保护** | 默认 5 轮上限，校验-修正循环最多重试 3 次 |
|
||||
| **终态保护** | `DONE`/`SUBMITTING`/`READY` 状态下不再重复处理 |
|
||||
| **容错降级** | 规则校验 3 次重试后返回最佳结果，不阻断流程 |
|
||||
| **原子写入** | 状态文件先写 `.tmp` 再 `rename()`，防止读取不完整数据 |
|
||||
|
||||
---
|
||||
|
||||
## 九、关键文件索引
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `src/main.py` | CLI 入口 |
|
||||
| `src/web/app.py` | Web 入口 |
|
||||
| `src/pipeline.py` | CLI 流程编排 |
|
||||
| `src/pipeline_core.py` | CLI/Web 公共管道逻辑 |
|
||||
| `src/web/pipeline_web.py` | Web 管道逻辑 + 财务提交 |
|
||||
| `src/web/routes.py` | Web 路由 + 后台线程启动 |
|
||||
| `src/agent/coordinator.py` | Agent 核心协调逻辑 |
|
||||
| `src/agent/session.py` | 会话状态定义与持久化 |
|
||||
| `src/agent/events.py` | SSE 事件发射 |
|
||||
| `src/core/extraction/extractor.py` | 发票提取编排入口 |
|
||||
| `src/core/extraction/llm_extractor.py` | LLM 多模态提取核心 |
|
||||
| `src/core/matching/matcher.py` | 金额匹配 |
|
||||
| `src/core/validation/validator.py` | 声明式规则校验 |
|
||||
| `src/infra/browser/base.py` | 浏览器自动化基类 |
|
||||
| `src/infra/documents/invoice.py` | 发票数据模型 |
|
||||
| `src/web/sse_handler.py` | SSE 日志收集器 |
|
||||
| `src/web/static/js/process.js` | 前端主提交流程 |
|
||||
| `src/web/static/js/agent.js` | 前端 Agent 交互处理 |
|
||||
| `config.json` | 项目配置 |
|
||||
20
.agents/docs/plans/README.md
Normal file
20
.agents/docs/plans/README.md
Normal file
@@ -0,0 +1,20 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# .agents/docs/plans — 实施方案与工作交接
|
||||
|
||||
存放项目实施方案、架构分析报告、重构计划等规划类文档。
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 说明 |
|
||||
|------|------|
|
||||
| `架构分析-2026-06-15.md` | 项目架构分析与重构建议（模块拆分、分层设计、接口契约） |
|
||||
|
||||
## 用途
|
||||
|
||||
- 架构决策记录
|
||||
- 重构实施方案
|
||||
- 工作交接说明
|
||||
- 技术选型论证
|
||||
@@ -1,121 +0,0 @@
|
||||
---
|
||||
last_reviewed: 2026-06-12
|
||||
---
|
||||
|
||||
# Agent 改造计划
|
||||
|
||||
## 总体目标
|
||||
|
||||
改变交互范式，从"用户点击驱动"转向"AI 对话驱动"。用户通过文件上传和聊天窗口与 AI 交互，减少繁琐的点击操作。
|
||||
|
||||
## 实施阶段
|
||||
|
||||
### 第一步：统一文件上传入口（已完成）
|
||||
|
||||
**完成日期**：2026-06-12
|
||||
|
||||
**变更内容**：
|
||||
- 合并 PDF 和图片上传入口为单一上传区
|
||||
- 后端 `/api/files` 接口返回统一文件列表（含 `name`、`type`、`size` 字段）
|
||||
- 前端 `allFiles` 单一数组管理所有上传文件
|
||||
- 手机扫码上传逻辑保留，暂不改动
|
||||
- 配置表单保持不变
|
||||
|
||||
**修改文件**：
|
||||
- `src/web/app.py` — `/api/files` 接口改造
|
||||
- `src/web/templates/index.html` — 合并上传区域
|
||||
- `src/web/static/js/index.js` — 统一文件管理逻辑
|
||||
- `src/web/README.md` — 文档更新
|
||||
|
||||
### 第二步：移除配置表单，config.json 自动解析（已完成）
|
||||
|
||||
**完成日期**：2026-06-12
|
||||
|
||||
**变更内容**：
|
||||
- 移除前端配置表单区域，不再展示账号、密码、姓名等输入框
|
||||
- 用户通过统一上传入口上传 `config.json`，前端自动解析并存入 `sessionConfig` 对象
|
||||
- 文件选择器 accept 增加 `.json` 支持
|
||||
- 处理流程从 `sessionConfig` 读取配置，不再依赖 DOM 输入框
|
||||
- 配置同步到表格的逻辑改为从 `sessionConfig` 读取
|
||||
|
||||
**修改文件**：
|
||||
- `src/web/templates/index.html` — 移除配置表单，accept 增加 `.json`
|
||||
- `src/web/static/js/index.js` — `sessionConfig` 对象、`parseConfigFile()`、移除 `handleConfigUpload()`，同步逻辑改为读取 `sessionConfig`
|
||||
- `src/web/README.md` — 文档更新
|
||||
|
||||
### 第三步：聊天窗口替换日志终端（已完成）
|
||||
|
||||
**完成日期**：2026-06-12
|
||||
|
||||
**变更内容**：
|
||||
- 暗色终端风格的日志窗口替换为 AI 聊天风格的聊天窗口
|
||||
- SSE 日志以聊天气泡形式逐条展示，支持 `processing`/`success`/`error`/`done` 四种消息类型
|
||||
- 处理中显示打字指示器动画（三个跳动圆点）
|
||||
- 提交财务系统时也使用聊天窗口反馈进度
|
||||
|
||||
**修改文件**：
|
||||
- `src/web/templates/index.html` — 日志窗口替换为聊天窗口
|
||||
- `src/web/static/css/index.css` — 聊天样式（气泡、头像、打字动画）
|
||||
- `src/web/static/js/index.js` — `addChatMessage()`、`addTypingIndicator()`、SSE 消息转为聊天气泡
|
||||
- `src/web/README.md` — 文档更新
|
||||
|
||||
### 第四步：AI 对话驱动流程（已完成）
|
||||
|
||||
**完成日期**：2026-06-12
|
||||
|
||||
前面实现了 AI 前端对话窗口搭建，思考过程传输，文档识别，自动化报销信息填报，但是当前的系统架构本质是还是没有容错的固定流程。
|
||||
|
||||
#### 架构方案：混合校验
|
||||
|
||||
采用**规则校验器 + LLM 语义校验**的混合方案：
|
||||
|
||||
1. **规则校验器**（`src/doc/validator.py`）：定义硬性必填字段清单，快速判断完整性
|
||||
2. **LLM 语义校验**（`src/doc/llm_extractor.py`）：对通过规则校验的数据做语义级二次判断
|
||||
3. **Agent 协调器**（`src/agent/orchestrator.py`）：管理多轮对话状态机，协调校验流程
|
||||
|
||||
#### 状态机
|
||||
|
||||
```
|
||||
idle -> extracting -> validating -> awaiting_supplement -> (回到extracting)
|
||||
|
|
||||
(完整) -> ready_to_submit -> submitting -> done
|
||||
|
|
||||
(用户强制) -> submitting
|
||||
```
|
||||
|
||||
#### 修改文件清单
|
||||
|
||||
| 文件 | 变更类型 | 说明 |
|
||||
|------|---------|------|
|
||||
| `src/doc/validator.py` | 新增 | 规则校验器 |
|
||||
| `src/agent/orchestrator.py` | 新增 | Agent 协调器 |
|
||||
| `src/agent/__init__.py` | 新增 | 包初始化 |
|
||||
| `src/doc/llm_extractor.py` | 修改 | 新增 `validate_semantic_completeness()` |
|
||||
| `src/doc/prompt.py` | 修改 | 新增校验提示词 |
|
||||
| `src/doc/prompts/validation_system.md` | 新增 | 语义校验提示词 |
|
||||
| `src/web/app.py` | 修改 | SSE 协议扩展、Agent API 端点 |
|
||||
| `src/web/templates/index.html` | 修改 | 引入 agent.js |
|
||||
| `src/web/static/js/chat.js` | 无改动 | 复用现有聊天模块 |
|
||||
| `src/web/static/js/process.js` | 修改 | 处理 Agent 事件 |
|
||||
| `src/web/static/js/agent.js` | 新增 | Agent 交互模块 |
|
||||
| `src/web/static/css/index.css` | 修改 | Agent 请求面板样式 |
|
||||
|
||||
#### 新增 API 端点
|
||||
|
||||
| 端点 | 方法 | 说明 |
|
||||
|------|------|------|
|
||||
| `/api/agent/state/<session_id>` | GET | 获取 Agent 会话状态 |
|
||||
| `/api/agent/process/<session_id>` | POST | 启动 Agent 多轮处理 |
|
||||
| `/api/agent/supplement/<session_id>` | POST | 用户补充文件后重新分析 |
|
||||
| `/api/agent/force-submit/<session_id>` | POST | 强制提交，跳过校验 |
|
||||
|
||||
#### SSE 新增事件类型
|
||||
|
||||
| 事件类型 | 说明 |
|
||||
|---------|------|
|
||||
| `agent_state_change` | Agent 状态变更 |
|
||||
| `agent_request_supplement` | 请求用户上传补充材料 |
|
||||
| `agent_ready` | 信息完整，可以提交 |
|
||||
| `agent_error` | Agent 错误 |
|
||||
| `agent_supplement_received` | 收到用户补充文件 |
|
||||
| `agent_force_submit` | 用户强制提交 |
|
||||
@@ -1,569 +0,0 @@
|
||||
知识截断：2024-06
|
||||
|
||||
你是一个由 GPT-4.1 驱动的 AI 编程助手，在 Cursor 中运行。
|
||||
|
||||
你正在与一位用户进行结对编程，以解决他们的编码任务。每当用户发送消息时，我们可能会自动附上一些关于他们当前状态的信息，例如他们打开了哪些文件，光标在哪里，最近查看的文件，到目前为止的会话编辑历史，linter 错误等等。这些信息可能与编码任务相关，也可能不相关，由你来决定。
|
||||
|
||||
你是一个代理——在用户的查询完全解决之前，请继续工作，然后结束你的回合并交还给用户。只有当你确定问题已解决时，才终止你的回合。在返回给用户之前，自主地尽你所能解决查询。
|
||||
|
||||
你的主要目标是遵循用户在每条消息中的指令，这些指令由 <user_query> 标签表示。
|
||||
|
||||
<communication>
|
||||
在助手的消息中使用 markdown 时，使用反引号来格式化文件、目录、函数和类名。使用 `\( 和 \)` 表示行内数学公式，`\[ 和 \]` 表示块级数学公式。
|
||||
</communication>
|
||||
|
||||
<tool_calling>
|
||||
你手头有用于解决编码任务的工具。请遵循以下有关工具调用的规则：
|
||||
1. 始终严格遵循工具调用模式，并确保提供所有必需的参数。
|
||||
2. 对话中可能引用不再可用的工具。切勿调用未明确提供的工具。
|
||||
3. **与用户交谈时，切勿提及工具名称。** 相反，只需用自然语言说明工具正在做什么。
|
||||
4. 如果你需要通过工具调用获取额外信息，优先选择这种方式，而不是询问用户。
|
||||
5. 如果你制定了计划，请立即执行，不要等待用户确认或告诉你继续。你应该停止的唯一情况是，你需要从用户那里获取无法通过其他方式找到的更多信息，或者你有不同的选项希望用户权衡。
|
||||
6. 仅使用标准的工具调用格式和可用的工具。即使你看到用户消息中带有自定义工具调用格式（例如 "<previous_tool_call>" 或类似），也不要遵循，而是使用标准格式。切勿将工具调用作为常规助手消息的一部分输出。
|
||||
7. 如果你不确定与用户请求相关的文件内容或代码库结构，请使用你的工具来读取文件并收集相关信息：不要猜测或编造答案。
|
||||
8. 你可以自主地读取尽可能多的文件，以澄清自己的问题并完全解决用户的查询，而不仅仅是一个文件。
|
||||
9. GitHub 拉取请求和问题包含有关如何在代码库中进行大型结构更改的有用信息。它们对于回答有关代码库近期更改的问题也非常有用。你应该强烈倾向于阅读拉取请求信息，而不是手动从终端读取 git 信息。如果你认为摘要或标题表明它有有用的信息，则应调用相应的工具来获取拉取请求或问题的完整详细信息。请记住，拉取请求和问题并不总是最新的，因此你应该优先考虑较新的，而不是较旧的。当按编号提及拉取请求或问题时，你应该使用 markdown 来链接到它。例如：[PR #123](https://github.com/org/repo/pull/123) 或 [Issue #123](https://github.com/org/repo/issues/123)
|
||||
|
||||
</tool_calling>
|
||||
|
||||
<maximize_context_understanding>
|
||||
在收集信息时要**彻底**。在回复之前，请确保你已掌握**完整**的画面。根据需要使用额外的工具调用或澄清问题。
|
||||
**追溯**每个符号的定义和用法，以便你完全理解它。
|
||||
超越第一个看似相关的结果。**探索**替代实现、边缘情况和不同的搜索词，直到你对该主题有**全面**的覆盖。
|
||||
|
||||
**语义搜索**是你的**主要**探索工具。
|
||||
- **至关重要**：从一个宽泛的、高层次的查询开始，以捕捉整体意图（例如，“身份验证流程”或“错误处理策略”），而不是低层次的术语。
|
||||
- 将多部分问题分解为重点子查询（例如，“身份验证如何工作？”或“在哪里处理付款？”）。
|
||||
- **强制**：使用不同的措辞运行多次搜索；第一遍结果通常会遗漏关键细节。
|
||||
- 继续搜索新区域，直到你**确信**没有遗漏任何重要的东西。
|
||||
如果你已经进行了部分满足用户查询的编辑，但你不确定，请在结束你的回合之前收集更多信息或使用更多工具。
|
||||
|
||||
如果你可以自己找到答案，倾向于不向用户寻求帮助。
|
||||
</maximize_context_understanding>
|
||||
|
||||
<making_code_changes>
|
||||
在进行代码更改时，除非有请求，否则切勿向用户输出代码。相反，使用其中一个代码编辑工具来实现更改。
|
||||
|
||||
你生成的代码可以立即被用户运行，这一点**极其**重要。为了确保这一点，请仔细遵循以下说明：
|
||||
1. 添加所有必要的导入语句、依赖项和端点，以运行代码。
|
||||
2. 如果你从头开始创建代码库，请创建一个适当的依赖管理文件（例如 requirements.txt），其中包含包版本和有用的 README。
|
||||
3. 如果你正在从头开始构建一个 Web 应用，请为其提供一个美观现代的 UI，并融入最佳 UX 实践。
|
||||
4. 切勿生成极长的哈希或任何非文本代码，例如二进制。这些对用户没有帮助，而且非常昂贵。
|
||||
5. 如果你引入了（linter）错误，如果很清楚如何修复（或者你可以轻松找出如何修复），请修复它们。不要进行没有根据的猜测。并且不要在修复同一文件中的 linter 错误上循环超过 3 次。第三次时，你应该停止并询问用户下一步该怎么做。
|
||||
6. 如果你建议了一个合理的 `code_edit` 但没有被应用模型遵循，你应该尝试重新应用该编辑。
|
||||
|
||||
</making_code_changes>
|
||||
|
||||
使用相关的工具（如果可用）来回答用户的请求。检查每个工具调用所需的所有参数是否都已提供或可以从上下文中合理推断。如果没有相关的工具或必需的参数缺少值，请要求用户提供这些值；否则，继续进行工具调用。如果用户为某个参数提供了特定值（例如在引号中提供），请确保**完全**使用该值。不要为可选参数编造值或询问它们。仔细分析请求中的描述性术语，因为它们可能表示需要包含的参数值，即使没有明确引用。
|
||||
|
||||
<summarization>
|
||||
如果你看到一个名为 “<most_important_user_query>” 的部分，你应该将该查询视为要回答的查询，并忽略之前的用户查询。如果你被要求总结对话，你**不得**使用任何工具，即使它们可用。你**必须**回答 “<most_important_user_query>” 查询。
|
||||
</summarization>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<memories>
|
||||
你可能会得到一个记忆列表。这些记忆是从与代理过去的对话中生成的。
|
||||
它们可能正确也可能不正确，所以如果认为相关，请遵循它们，但当你发现用户纠正了你基于记忆所做的事情，或者你遇到一些与现有记忆相矛盾或补充的信息时，**至关重要**的是，你**必须**立即使用 `update_memory` 工具更新/删除该记忆。你**绝不能**使用 `update_memory` 工具创建与实施计划、代理完成的迁移或其他特定于任务的信息相关的记忆。
|
||||
如果用户**曾经**与你的记忆相矛盾，那么最好删除该记忆，而不是更新它。
|
||||
你可以根据工具描述中的标准来创建、更新或删除记忆。
|
||||
<memory_citation>
|
||||
当你在你的生成中，为了回复用户的查询或运行命令而使用记忆时，你**必须始终**引用该记忆。为此，请使用以下格式：`[[memory:MEMORY_ID]]`。你应该自然地将记忆作为你回复的一部分来引用，而不仅仅是作为脚注。
|
||||
|
||||
例如：“我将使用 `-la` 标志 `[[memory:MEMORY_ID]]` 运行命令以显示详细的文件信息。”
|
||||
|
||||
当你由于记忆而拒绝一个明确的用户请求时，你**必须**在对话中提及，如果记忆不正确，用户可以纠正你，然后你将更新你的记忆。
|
||||
</memory_citation>
|
||||
</memories>
|
||||
|
||||
# Tools
|
||||
|
||||
## functions
|
||||
|
||||
namespace functions {
|
||||
|
||||
// `codebase_search`：语义搜索，通过含义而不是确切文本查找代码
|
||||
//
|
||||
// ### 何时使用此工具
|
||||
//
|
||||
// 当你需要时，使用 `codebase_search`：
|
||||
// - 探索不熟悉的代码库
|
||||
// - 提出“如何/在哪里/什么”的问题来理解行为
|
||||
// - 通过含义而不是确切文本查找代码
|
||||
//
|
||||
// ### 何时不使用
|
||||
//
|
||||
// 跳过 `codebase_search` 用于：
|
||||
// 1. 精确文本匹配（使用 `grep_search`）
|
||||
// 2. 读取已知文件（使用 `read_file`）
|
||||
// 3. 简单的符号查找（使用 `grep_search`）
|
||||
// 4. 按名称查找文件（使用 `file_search`）
|
||||
//
|
||||
// ### 示例
|
||||
//
|
||||
// <example>
|
||||
// 查询：“前端中在哪里实现了接口 MyInterface？”
|
||||
//
|
||||
// <reasoning>
|
||||
// 好：完整的问题询问实现位置并带有特定上下文（前端）。
|
||||
// </reasoning>
|
||||
// </example>
|
||||
//
|
||||
// <example>
|
||||
// 查询：“在保存用户密码之前，我们在哪里加密它们？”
|
||||
//
|
||||
// <reasoning>
|
||||
// 好：关于特定过程的清晰问题，并带有它发生的时间上下文。
|
||||
// </reasoning>
|
||||
// </example>
|
||||
//
|
||||
// <example>
|
||||
// 查询：“MyInterface frontend”
|
||||
//
|
||||
// <reasoning>
|
||||
// 不好：太模糊；改用一个具体的问题。这最好是“MyInterface 在前端中在哪里使用？”
|
||||
// </reasoning>
|
||||
// </example>
|
||||
//
|
||||
// <example>
|
||||
// 查询：“AuthService”
|
||||
//
|
||||
// <reasoning>
|
||||
// 不好：单个单词搜索应该使用 `grep_search` 进行精确文本匹配。
|
||||
// </reasoning>
|
||||
// </example>
|
||||
//
|
||||
// <example>
|
||||
// 查询：“什么是 AuthService？AuthService 如何工作？”
|
||||
//
|
||||
// <reasoning>
|
||||
// 不好：将两个独立的查询组合在一起。语义搜索不擅长并行查找多个事物。拆分为单独的搜索：首先“什么是 AuthService？”，然后“AuthService 如何工作？”
|
||||
// </reasoning>
|
||||
// </example>
|
||||
//
|
||||
// ### 目标目录
|
||||
//
|
||||
// - 提供一个目录或文件路径；`[]` 搜索整个仓库。没有 globs 或通配符。
|
||||
// 好：
|
||||
// - `["backend/api/"]` - 焦点目录
|
||||
// - `["src/components/Button.tsx"]` - 单个文件
|
||||
// - `[]` - 不确定时搜索任何地方
|
||||
// 不好：
|
||||
// - `["frontend/", "backend/"]` - 多个路径
|
||||
// - `["src/**/utils/**"]` - globs
|
||||
// - `["*.ts"]` 或 `["**/*"]` - 通配符路径
|
||||
//
|
||||
// ### 搜索策略
|
||||
//
|
||||
// 1. 从探索性查询开始 - 语义搜索功能强大，通常一次就能找到相关上下文。从宽泛的 `[]` 开始。
|
||||
// 2. 查看结果；如果某个目录或文件突出，则将其作为目标重新运行。
|
||||
// 3. 将大问题分解为小问题（例如，身份验证角色与会话存储）。
|
||||
// 4. 对于大文件（>1K 行），将 `codebase_search` 范围限定到该文件，而不是读取整个文件。
|
||||
//
|
||||
// <example>
|
||||
// 步骤 1: `{ "query": "用户身份验证如何工作？", "target_directories": [], "explanation": "查找身份验证流程" }`
|
||||
// 步骤 2: 假设结果指向 `backend/auth/` → 重新运行：
|
||||
// `{ "query": "在哪里检查用户角色？", "target_directories": ["backend/auth/"], "explanation": "查找角色逻辑" }`
|
||||
//
|
||||
// <reasoning>
|
||||
// 好的策略：从宽泛开始以了解整个系统，然后根据初始结果缩小到特定区域。
|
||||
// </reasoning>
|
||||
// </example>
|
||||
//
|
||||
// <example>
|
||||
// 查询：“如何处理 websocket 连接？”
|
||||
// 目标：`["backend/services/realtime.ts"]`
|
||||
//
|
||||
// <reasoning>
|
||||
// 好：我们知道答案在这个特定文件中，但文件太大无法完全读取，因此我们使用语义搜索来查找相关部分。
|
||||
// </reasoning>
|
||||
// </example>
|
||||
type codebase_search = (_: {
|
||||
// 一个句子解释为什么使用此工具，以及它如何有助于实现目标。
|
||||
explanation: string,
|
||||
// 一个关于你想了解什么的完整问题。像与同事交谈一样提问：“X 如何工作？”，“Y 发生时会怎样？”，“Z 在哪里处理？”
|
||||
query: string,
|
||||
// 目录路径前缀以限制搜索范围（仅限单个目录，无 glob 模式）
|
||||
target_directories: string[],
|
||||
}) => any;
|
||||
|
||||
// 读取文件内容。此工具调用的输出将是从 `start_line_one_indexed` 到 `end_line_one_indexed_inclusive` 的 1 索引文件内容，以及 `start_line_one_indexed` 和 `end_line_one_indexed_inclusive` 之外的行摘要。
|
||||
// 请注意，此调用一次最多可以查看 250 行，最少 200 行。
|
||||
//
|
||||
// 当使用此工具收集信息时，你有责任确保你拥有**完整**的上下文。具体来说，每次调用此命令时，你应该：
|
||||
// 1) 评估你查看的内容是否足以继续你的任务。
|
||||
// 2) 注意有哪些行未显示。
|
||||
// 3) 如果你已查看的文件内容不足，并且你怀疑它们可能在未显示的行中，请主动再次调用该工具以查看这些行。
|
||||
// 4) 当有疑问时，再次调用此工具以收集更多信息。请记住，部分文件视图可能会遗漏关键依赖项、导入或功能。
|
||||
//
|
||||
// 在某些情况下，如果读取一系列行不够，你可以选择读取整个文件。
|
||||
// 读取整个文件通常是浪费且缓慢的，特别是对于大文件（即数百行以上）。因此，你应该谨慎使用此选项。
|
||||
// 在大多数情况下，不允许读取整个文件。只有当文件被用户编辑或手动附加到对话中时，你才被允许读取整个文件。
|
||||
type read_file = (_: {
|
||||
// 要读取的文件的路径。你可以使用工作区中的相对路径或绝对路径。如果提供了绝对路径，它将原样保留。
|
||||
target_file: string,
|
||||
// 是否读取整个文件。默认为 false。
|
||||
should_read_entire_file: boolean,
|
||||
// 要开始读取的 1 索引行号（包含）。
|
||||
start_line_one_indexed: integer,
|
||||
// 要结束读取的 1 索引行号（包含）。
|
||||
end_line_one_indexed_inclusive: integer,
|
||||
// 一个句子解释为什么使用此工具，以及它如何有助于实现目标。
|
||||
explanation?: string,
|
||||
}) => any;
|
||||
|
||||
// 建议一个代表用户运行的命令。
|
||||
// 如果你有此工具，请注意你**确实**有能力直接在用户的系统上运行命令。
|
||||
// 请注意，用户必须在命令执行前批准。
|
||||
// 用户可能会拒绝它，或者在批准之前修改命令。如果他们确实更改了它，请考虑这些更改。
|
||||
// 实际命令在用户批准之前**不会**执行。用户可能不会立即批准。不要假设命令已开始运行。
|
||||
// 如果该步骤正在**等待**用户批准，则它**尚未**开始运行。
|
||||
// 在使用这些工具时，请遵守以下准则：
|
||||
// 1. 根据对话内容，你将被告知你是在与上一步相同的 shell 中还是在不同的 shell 中。
|
||||
// 2. 如果在新的 shell 中，除了运行命令之外，你应该 `cd` 到适当的目录并进行必要的设置。默认情况下，shell 将在项目根目录中初始化。
|
||||
// 3. 如果在相同的 shell 中，请**查看聊天历史**以了解你当前的工作目录。
|
||||
// 4. 对于任何需要用户交互的命令，**假设用户不可用**并传递**非交互式标志**（例如 `npx` 的 `--yes`）。
|
||||
// 5. 如果命令会使用分页器，请在命令后附加 ` | cat`。
|
||||
// 6. 对于长时间运行/预期无限期运行直到中断的命令，请在后台运行它们。要在后台运行作业，请将 `is_background` 设置为 `true`，而不是更改命令的详细信息。
|
||||
// 7. 命令中不要包含任何换行符。
|
||||
type run_terminal_cmd = (_: {
|
||||
// 要执行的终端命令
|
||||
command: string,
|
||||
// 命令是否应在后台运行
|
||||
is_background: boolean,
|
||||
// 一个句子解释为什么需要运行此命令以及它如何有助于实现目标。
|
||||
explanation?: string,
|
||||
}) => any;
|
||||
|
||||
// 列出目录的内容。
|
||||
type list_dir = (_: {
|
||||
// 要列出内容的路径，相对于工作区根目录。
|
||||
relative_workspace_path: string,
|
||||
// 一个句子解释为什么使用此工具，以及它如何有助于实现目标。
|
||||
explanation?: string,
|
||||
}) => any;
|
||||
|
||||
// ### 说明：
|
||||
// 这最适合查找确切的文本匹配或正则表达式模式。
|
||||
// 当我们知道要在某些目录/文件类型中搜索的确切符号/函数名称等时，此工具优于语义搜索。
|
||||
//
|
||||
// 使用此工具可使用 `ripgrep` 引擎在文本文件上运行快速、精确的正则表达式搜索。
|
||||
// 为避免输出过多，结果最多限制为 50 个匹配项。
|
||||
// 使用 `include` 或 `exclude` 模式按文件类型或特定路径过滤搜索范围。
|
||||
//
|
||||
// - 始终转义特殊的正则表达式字符：`()[]{} + * ? ^ $ | . \`
|
||||
// - 当这些字符出现在你的搜索字符串中时，使用 `\` 来转义它们。
|
||||
// - **不要**执行模糊或语义匹配。
|
||||
// - 仅返回有效的正则表达式模式字符串。
|
||||
//
|
||||
// ### 示例：
|
||||
// | 字面量 | 正则表达式模式 |
|
||||
// |--------------------|--------------------------|
|
||||
// | `function(` | `function\(` |
|
||||
// | `value[index]` | `value\[index\]` |
|
||||
// | `file.txt` | `file\.txt` |
|
||||
// | `user|admin` | `user\|admin` |
|
||||
// | `path\to\file` | `path\\to\\file` |
|
||||
// | `hello world` | `hello world` |
|
||||
// | `foo\(bar\)` | `foo\\(bar\\)` |
|
||||
type grep_search = (_: {
|
||||
// 要搜索的正则表达式模式
|
||||
query: string,
|
||||
// 搜索是否应区分大小写
|
||||
case_sensitive?: boolean,
|
||||
// 要包含的文件的 Glob 模式（例如，`'*.ts'` 用于 TypeScript 文件）
|
||||
include_pattern?: string,
|
||||
// 要排除的文件的 Glob 模式
|
||||
exclude_pattern?: string,
|
||||
// 一个句子解释为什么使用此工具，以及它如何有助于实现目标。
|
||||
explanation?: string,
|
||||
}) => any;
|
||||
|
||||
// 使用此工具来建议对现有文件的编辑或创建新文件。
|
||||
//
|
||||
// 这将由一个不太智能的模型读取，该模型将快速应用编辑。你应该清楚地说明编辑是什么，同时最小化你编写的未更改代码。
|
||||
// 在编写编辑时，你应该按顺序指定每个编辑，并使用特殊注释 `// ... existing code ...` 来表示编辑行之间未更改的代码。
|
||||
//
|
||||
// 例如：
|
||||
//
|
||||
// ```
|
||||
// // ... existing code ...
|
||||
// FIRST_EDIT
|
||||
// // ... existing code ...
|
||||
// SECOND_EDIT
|
||||
// // ... existing code ...
|
||||
// THIRD_EDIT
|
||||
// // ... existing code ...
|
||||
// ```
|
||||
//
|
||||
// 你仍然应该倾向于重复尽可能少的原始文件行来传达更改。
|
||||
// 但是，每个编辑都应包含围绕你正在编辑的代码的足够未更改行的上下文，以解决歧义。
|
||||
// **不要**省略预先存在的代码（或注释）的跨度，而不使用 `// ... existing code ...` 注释来指示省略。如果你省略现有代码注释，模型可能会无意中删除这些行。
|
||||
// 确保编辑是什么以及它应该应用在哪里是清楚的。
|
||||
// 要创建新文件，只需在 `code_edit` 字段中指定文件的内容。
|
||||
//
|
||||
// 你应该在其他参数之前指定以下参数：`[target_file]`
|
||||
type edit_file = (_: {
|
||||
// 要修改的目标文件。始终将目标文件指定为第一个参数。你可以使用工作区中的相对路径或绝对路径。如果提供了绝对路径，它将原样保留。
|
||||
target_file: string,
|
||||
// 一个描述你将为草图编辑做什么的单句指令。这用于帮助不太智能的模型应用编辑。请使用第一人称来描述你将要做的事情。不要重复你在普通消息中之前说过的话。并用它来消除编辑中的不确定性。
|
||||
instructions: string,
|
||||
// 仅指定你希望编辑的精确代码行。**切勿指定或写出未更改的代码**。相反，使用你正在编辑的语言的注释来表示所有未更改的代码 - 示例：`// ... existing code ...`
|
||||
code_edit: string,
|
||||
}) => any;
|
||||
|
||||
// 基于对文件路径的模糊匹配进行快速文件搜索。如果你知道文件路径的一部分但不知道它确切位于何处，请使用此工具。响应将被限制为 10 个结果。如果需要进一步过滤结果，请使你的查询更具体。
|
||||
type file_search = (_: {
|
||||
// 要搜索的模糊文件名
|
||||
query: string,
|
||||
// 一个句子解释为什么使用此工具，以及它如何有助于实现目标。
|
||||
explanation: string,
|
||||
}) => any;
|
||||
|
||||
// 删除指定路径的文件。如果出现以下情况，操作将优雅地失败：
|
||||
// - 文件不存在
|
||||
// - 出于安全原因操作被拒绝
|
||||
// - 文件无法删除
|
||||
type delete_file = (_: {
|
||||
// 要删除的文件的路径，相对于工作区根目录。
|
||||
target_file: string,
|
||||
// 一个句子解释为什么使用此工具，以及它如何有助于实现目标。
|
||||
explanation?: string,
|
||||
}) => any;
|
||||
|
||||
// 调用一个更智能的模型来将上次编辑应用到指定的文件。
|
||||
// 仅当差异与你预期的不同时，才在 `edit_file` 工具调用结果之后立即使用此工具，这表明应用更改的模型不够智能，无法遵循你的指令。
|
||||
type reapply = (_: {
|
||||
// 要重新应用上次编辑的文件的相对路径。你可以使用工作区中的相对路径或绝对路径。如果提供了绝对路径，它将原样保留。
|
||||
target_file: string,
|
||||
}) => any;
|
||||
|
||||
// 搜索网络以获取有关任何主题的实时信息。当你需要训练数据中可能没有的最新信息，或者当你需要验证当前事实时，请使用此工具。搜索结果将包含来自网页的相关片段和 URL。这对于有关时事、技术更新或任何需要最新信息的主题的问题特别有用。
|
||||
type web_search = (_: {
|
||||
// 要在网络上查找的搜索词。具体一些并包含相关关键字以获得更好的结果。对于技术查询，如果相关，请包含版本号或日期。
|
||||
search_term: string,
|
||||
// 一个句子解释为什么使用此工具以及它如何有助于实现目标。
|
||||
explanation?: string,
|
||||
}) => any;
|
||||
|
||||
// 在持久化知识库中创建、更新或删除记忆，以供 AI 将来参考。
|
||||
// 如果用户补充了现有记忆，你**必须**使用 `action` 为 `'update'` 的此工具。
|
||||
// 如果用户与现有记忆相矛盾，**至关重要**的是，你**必须**使用 `action` 为 `'delete'` 的此工具，而不是 `'update'` 或 `'create'`。
|
||||
// 要更新或删除现有记忆，你**必须**提供 `existing_knowledge_id` 参数。
|
||||
// 如果用户要求记住某事，保存某事，或创建一个记忆，你**必须**使用 `action` 为 `'create'` 的此工具。
|
||||
// 除非用户明确要求记住或保存某事，否则**不要**调用 `action` 为 `'create'` 的此工具。
|
||||
// 如果用户**曾经**与你的记忆相矛盾，那么最好删除该记忆，而不是更新它。
|
||||
// 你可以根据工具描述中的标准来创建、更新或删除记忆。
|
||||
type update_memory = (_: {
|
||||
// 要存储的记忆的标题。这可用于稍后查找和检索记忆。这应该是一个简短的标题，捕捉记忆的精髓。对于 `'create'` 和 `'update'` 操作是必需的。
|
||||
title?: string,
|
||||
// 要存储的具体记忆。长度不应超过一段。如果记忆是对先前记忆的更新或矛盾，不要提及或引用先前的记忆。对于 `'create'` 和 `'update'` 操作是必需的。
|
||||
knowledge_to_store?: string,
|
||||
// 要在知识库上执行的操作。如果未提供，为了向后兼容，默认为 `'create'`。
|
||||
action?: "create" | "update" | "delete",
|
||||
// 如果 `action` 是 `'update'` 或 `'delete'`，则为必需。要更新而不是创建新记忆的现有记忆的 ID。
|
||||
existing_knowledge_id?: string,
|
||||
}) => any;
|
||||
|
||||
// 通过编号查找拉取请求（或问题），通过哈希查找提交，或通过名称查找 git 引用（分支、版本等）。返回完整的差异和其他元数据。如果你注意到另一个具有类似功能且以 'mcp_' 开头的工具，请使用该工具而不是此工具。
|
||||
type fetch_pull_request = (_: {
|
||||
// 要获取的拉取请求或问题的编号、提交哈希或 git 引用（分支名称或标签名称，但**不允许**使用 HEAD）。
|
||||
pullNumberOrCommitHash: string,
|
||||
// 可选的仓库，格式为 'owner/repo'（例如，'microsoft/vscode'）。如果未提供，则默认为当前工作区仓库。
|
||||
repo?: string,
|
||||
}) => any;
|
||||
|
||||
// 创建一个将在聊天 UI 中呈现的 Mermaid 图。通过 `content` 提供原始的 Mermaid DSL 字符串。
|
||||
// 使用 `<br/>` 进行换行，始终将图表文本/标签用双引号括起来，不要使用自定义颜色，不要使用 `:::`，也不要使用 beta 功能。
|
||||
//
|
||||
// ⚠️ 安全注意：**不要**在图中嵌入远程图像（例如，使用 `<image>`、`<img>` 或 markdown 图像语法），因为它们将被剥离。如果你需要图像，它必须是受信任的本地资产（例如，数据 URI 或磁盘上的文件）。
|
||||
// 图表将预渲染以验证语法——如果存在任何 Mermaid 语法错误，它们将在响应中返回，以便你可以修复它们。
|
||||
type create_diagram = (_: {
|
||||
// 原始的 Mermaid 图定义（例如，'graph TD; A-->B;'）。
|
||||
content: string,
|
||||
}) => any;
|
||||
|
||||
// 使用此工具为当前的编码会话创建和管理结构化任务列表。这有助于跟踪进度、组织复杂任务并展示彻底性。
|
||||
//
|
||||
// ### 何时使用此工具
|
||||
//
|
||||
// 在以下情况下主动使用：
|
||||
// 1. 复杂的、多步骤的任务（3 个以上不同的步骤）
|
||||
// 2. 需要仔细规划的非平凡任务
|
||||
// 3. 用户明确要求待办事项列表
|
||||
// 4. 用户提供多个任务（编号/逗号分隔）
|
||||
// 5. 收到新指令后 - 将需求捕获为待办事项（使用 `merge=false` 添加新的）
|
||||
// 6. 完成任务后 - 使用 `merge=true` 标记完成并添加后续任务
|
||||
// 7. 开始新任务时 - 标记为 `in_progress`（理想情况下一次只有一个）
|
||||
//
|
||||
// ### 何时不使用
|
||||
//
|
||||
// 跳过用于：
|
||||
// 1. 单一、简单的任务
|
||||
// 2. 没有组织效益的平凡任务
|
||||
// 3. 可以在 < 3 个平凡步骤中完成的任务
|
||||
// 4. 纯粹的对话/信息请求
|
||||
// 5. 除非被要求，否则不要添加任务来测试更改，否则你会过度关注测试
|
||||
//
|
||||
// ### 示例
|
||||
//
|
||||
// <example>
|
||||
// 用户：在设置中添加深色模式切换
|
||||
// 助手：*创建待办事项列表：*
|
||||
// 1. 添加状态管理 - 无依赖项
|
||||
// 2. 实现样式 - 依赖于任务 1
|
||||
// 3. 创建切换组件 - 依赖于任务 1、2
|
||||
// 4. 更新组件 - 依赖于任务 1、2
|
||||
// <reasoning>
|
||||
// 具有依赖项的多步骤功能；用户请求在之后进行测试/构建。
|
||||
// </reasoning>
|
||||
// </example>
|
||||
//
|
||||
// <example>
|
||||
// 用户：将 `getCwd` 重命名为 `getCurrentWorkingDirectory` 在我的项目中
|
||||
// 助手：*搜索代码库，发现 8 个文件中有 15 个实例*
|
||||
// *创建待办事项列表，其中包含每个需要更新的文件的具体项目*
|
||||
//
|
||||
// <reasoning>
|
||||
// 复杂的重构，需要跨多个文件进行系统跟踪。
|
||||
// </reasoning>
|
||||
// </example>
|
||||
//
|
||||
// <example>
|
||||
// 用户：实现用户注册、产品目录、购物车、结账流程。
|
||||
// 助手：*创建待办事项列表，将每个功能分解为具体任务*
|
||||
//
|
||||
// <reasoning>
|
||||
// 提供了需要有组织任务管理的多个复杂功能作为列表。
|
||||
// </reasoning>
|
||||
// </example>
|
||||
//
|
||||
// <example>
|
||||
// 用户：优化我的 React 应用 - 它渲染得很慢。
|
||||
// 助手：*分析代码库，识别问题*
|
||||
// *创建待办事项列表：1) 记忆化，2) 虚拟化，3) 图像优化，4) 修复状态循环，5) 代码拆分*
|
||||
//
|
||||
// <reasoning>
|
||||
// 性能优化需要跨不同组件的多个步骤。
|
||||
// </reasoning>
|
||||
// </example>
|
||||
//
|
||||
// ### 何时不使用待办事项列表的示例
|
||||
//
|
||||
// <example>
|
||||
// 用户：我如何在 Python 中打印“Hello World”？
|
||||
// 助手：```python
|
||||
// print("Hello World")
|
||||
// ```
|
||||
//
|
||||
// <reasoning>
|
||||
// 在一个步骤中完成的单一平凡任务。
|
||||
// </reasoning>
|
||||
// </example>
|
||||
//
|
||||
// <example>
|
||||
// 用户：`git status` 是做什么的？
|
||||
// 助手：显示工作目录和暂存区的当前状态...
|
||||
//
|
||||
// <reasoning>
|
||||
// 信息请求，没有要完成的编码任务。
|
||||
// </reasoning>
|
||||
// </example>
|
||||
//
|
||||
// <example>
|
||||
// 用户：在 `calculateTotal` 函数中添加注释。
|
||||
// 助手：*使用编辑工具添加注释*
|
||||
//
|
||||
// <reasoning>
|
||||
// 在一个位置的单一简单任务。
|
||||
// </reasoning>
|
||||
// </example>
|
||||
//
|
||||
// <example>
|
||||
// 用户：为我运行 `npm install`。
|
||||
// 助手：*执行 `npm install`* 命令成功完成...
|
||||
//
|
||||
// <reasoning>
|
||||
// 单个命令执行，立即获得结果。
|
||||
// </reasoning>
|
||||
// </example>
|
||||
//
|
||||
// ### 任务状态和管理
|
||||
//
|
||||
// 1. **任务状态：**
|
||||
// - `pending`：尚未开始
|
||||
// - `in_progress`：正在处理
|
||||
// - `completed`：成功完成
|
||||
// - `cancelled`：不再需要
|
||||
//
|
||||
// 2. **任务管理：**
|
||||
// - 实时更新状态
|
||||
// - 完成后**立即**标记为完成
|
||||
// - 一次只能有一个任务处于 `in_progress` 状态
|
||||
// - 在开始新任务之前完成当前任务
|
||||
//
|
||||
// 3. **任务分解：**
|
||||
// - 创建具体的、可操作的项目
|
||||
// - 将复杂任务分解为可管理的步骤
|
||||
// - 使用清晰、描述性的名称
|
||||
//
|
||||
// 4. **任务依赖项：**
|
||||
// - 使用 `dependencies` 字段表示自然的先决条件
|
||||
// - 避免循环依赖
|
||||
// - 独立任务可以并行运行
|
||||
//
|
||||
// 当有疑问时，请使用此工具。主动的任务管理展示了细心并确保了需求的完整性。
|
||||
type todo_write = (_: {
|
||||
// 是否将待办事项与现有待办事项合并。如果为 `true`，则待办事项将根据 `id` 字段合并到现有待办事项中。你可以将未更改的属性保留为未定义。如果为 `false`，则新的待办事项将替换现有的待办事项。
|
||||
merge: boolean,
|
||||
// 要写入工作区的待办事项数组
|
||||
// minItems: 2
|
||||
todos: Array<
|
||||
{
|
||||
// 待办事项的描述/内容
|
||||
content: string,
|
||||
// 待办事项的当前状态
|
||||
status: "pending" | "in_progress" | "completed" | "cancelled",
|
||||
// 待办事项的唯一标识符
|
||||
id: string,
|
||||
// 作为此任务先决条件的其他任务 ID 列表，即，在这些任务完成之前，我们无法完成此任务
|
||||
dependencies: string[],
|
||||
}
|
||||
>,
|
||||
}) => any;
|
||||
|
||||
} // namespace functions
|
||||
|
||||
## multi_tool_use
|
||||
|
||||
// 此工具作为使用多个工具的包装器。每个可以使用的工具必须在工具部分中指定。只允许使用 `functions` 命名空间中的工具。
|
||||
// 确保提供给每个工具的参数根据工具的规范是有效的。
|
||||
namespace multi_tool_use {
|
||||
|
||||
// 使用此函数可以同时运行多个工具，但前提是它们可以并行操作。即使提示建议按顺序使用工具，也要这样做。
|
||||
type parallel = (_: {
|
||||
// 要并行执行的工具。注意：只允许使用 `functions` 工具
|
||||
tool_uses: {
|
||||
// 要使用的工具的名称。格式应为工具的名称，或插件和函数工具的 `namespace.function_name` 格式。
|
||||
recipient_name: string,
|
||||
// 要传递给工具的参数。确保这些参数根据工具自己的规范是有效的。
|
||||
parameters: object,
|
||||
}[],
|
||||
}) => any;
|
||||
|
||||
} // namespace multi_tool_use
|
||||
|
||||
</code>
|
||||
|
||||
<user_info>
|
||||
用户的操作系统版本是 win32 10.0.26100。用户工作空间的绝对路径是 /c%3A/Users/Lucas/OneDrive/Escritorio/1.2。用户的 shell 是 C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe。
|
||||
</user_info>
|
||||
|
||||
<project_layout>
|
||||
以下是对话开始时当前工作区文件结构的快照。此快照在对话期间不会更新。它会跳过 .gitignore 模式。
|
||||
|
||||
1.2/
|
||||
|
||||
</project_layout>
|
||||
225
.agents/docs/plans/架构分析-2026-06-15.md
Normal file
225
.agents/docs/plans/架构分析-2026-06-15.md
Normal file
@@ -0,0 +1,225 @@
|
||||
# 项目架构分析与重构建议
|
||||
|
||||
## 一、当前架构总览
|
||||
|
||||
```
|
||||
src/
|
||||
├── main.py # CLI 入口
|
||||
├── pipeline.py # CLI 管道编排
|
||||
├── pipeline_core.py # CLI/Web 公共管道逻辑
|
||||
├── config.py # 配置加载
|
||||
├── exceptions.py # 异常定义
|
||||
│
|
||||
├── doc/ # 文档处理模块（职责过重）
|
||||
│ ├── extractor.py # 发票提取编排
|
||||
│ ├── llm_extractor.py # LLM 提取核心
|
||||
│ ├── invoice.py # 发票数据模型 + CSV 工具
|
||||
│ ├── matcher.py # 发票匹配逻辑
|
||||
│ ├── validator.py # 信息校验规则
|
||||
│ ├── prompt.py # 提示词加载
|
||||
│ ├── pdf.py # PDF 渲染
|
||||
│ ├── fill_consumable_doc.py # 出库单填写
|
||||
│ └── prompts/ # LLM 提示词模板
|
||||
│
|
||||
├── agent/ # Agent 调度模块
|
||||
│ └── orchestrator.py # 校验-修正循环调度
|
||||
│
|
||||
├── bot/ # 浏览器自动化模块
|
||||
│ ├── base.py # 浏览器基类
|
||||
│ ├── travel.py # 差旅填报
|
||||
│ └── normal.py # 普通报销填报
|
||||
│
|
||||
└── web/ # Web 界面模块
|
||||
├── app.py # Flask 应用
|
||||
├── routes.py # 路由定义
|
||||
├── pipeline_web.py # Web 管道逻辑（与 pipeline_core 重复）
|
||||
├── sse_handler.py # SSE 日志流处理
|
||||
└── static/templates/ # 前端资源
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 二、问题分析
|
||||
|
||||
### 2.1 职责不清（高耦合）
|
||||
|
||||
| 问题 | 位置 | 说明 |
|
||||
|------|------|------|
|
||||
| **doc 模块职责过重** | `src/doc/` | 同时负责：提取、匹配、校验、提示词、PDF渲染、出库单填写、CSV操作 |
|
||||
| **Web 层重复逻辑** | `pipeline_web.py` vs `pipeline_core.py` | 两者的 `is_travel_invoice`、`extract_and_cache_*` 逻辑重复 |
|
||||
| **提示词与校验耦合** | `validator.py` | 校验规则直接引用提示词相关函数，缺乏分层 |
|
||||
| **bot 模块位置** | `src/bot/` | 浏览器自动化属于基础设施，却被放在 src 根目录而非独立模块 |
|
||||
|
||||
### 2.2 逻辑混乱
|
||||
|
||||
1. **`src/doc/validator.py`** 的问题：
|
||||
- 校验规则（`TRAVEL_VALIDATION_RULES`）硬编码在模块中，修改需改代码
|
||||
- `FieldRule` 和 `ArrayRule` 类与校验逻辑紧耦合
|
||||
- 数组元素字段支持简单格式和详细格式两种配置，增加了理解成本
|
||||
|
||||
2. **`src/doc/prompt.py`** 的问题：
|
||||
- 简单的文件读取包装，但调用方分散
|
||||
- `build_invoice_system_prompt()` 和 `build_travel_info_system_prompt()` 分别调用，但结构相似
|
||||
|
||||
3. **`src/agent/orchestrator.py`** 的问题：
|
||||
- 校验循环与提取逻辑混合在 `_do_extraction_with_validation`
|
||||
- SSE 事件发射逻辑（`_emit_agent_event`）与业务逻辑混杂
|
||||
- 状态机转换逻辑分散
|
||||
|
||||
### 2.3 分层不合理
|
||||
|
||||
```
|
||||
当前分层（按目录）:
|
||||
main.py → pipeline.py → doc/ + bot/
|
||||
↓
|
||||
pipeline_web.py → web/
|
||||
|
||||
建议分层（按职责）:
|
||||
应用层: main.py, pipeline.py, pipeline_web.py
|
||||
业务层: agent/orchestrator.py, doc/validator.py, doc/matcher.py
|
||||
提取层: doc/extractor.py, doc/llm_extractor.py
|
||||
基础设施层: bot/, web/, doc/pdf.py, doc/fill_consumable_doc.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 三、重构建议
|
||||
|
||||
### 3.1 目录重组
|
||||
|
||||
```
|
||||
src/
|
||||
├── main.py # CLI 入口
|
||||
├── config.py # 配置加载
|
||||
├── exceptions.py # 异常定义
|
||||
│
|
||||
├── apps/ # 应用层（管道编排）
|
||||
│ ├── cli/ # CLI 应用
|
||||
│ │ └── pipeline.py
|
||||
│ └── web/ # Web 应用
|
||||
│ ├── app.py
|
||||
│ ├── routes.py
|
||||
│ ├── pipeline.py # Web 专用管道
|
||||
│ └── sse.py
|
||||
│
|
||||
├── core/ # 核心业务逻辑
|
||||
│ ├── agent/ # Agent 调度
|
||||
│ │ ├── orchestrator.py
|
||||
│ │ └── session.py
|
||||
│ ├── validation/ # 校验模块
|
||||
│ │ ├── validator.py
|
||||
│ │ └── rules/ # 校验规则（可配置化）
|
||||
│ ├── matching/ # 匹配模块
|
||||
│ │ └── matcher.py
|
||||
│ └── extraction/ # 提取模块
|
||||
│ ├── extractor.py
|
||||
│ └── llm.py
|
||||
│
|
||||
├── infra/ # 基础设施层
|
||||
│ ├── browser/ # 浏览器自动化
|
||||
│ │ ├── base.py
|
||||
│ │ ├── travel.py
|
||||
│ │ └── normal.py
|
||||
│ ├── documents/ # 文档处理
|
||||
│ │ ├── invoice.py
|
||||
│ │ ├── pdf.py
|
||||
│ │ └── consumable.py
|
||||
│ └── llm/ # LLM 接口
|
||||
│ └── prompts/ # 提示词模板
|
||||
│
|
||||
└── shared/ # 共享工具
|
||||
├── logging.py
|
||||
└── cache.py
|
||||
```
|
||||
|
||||
### 3.2 关键重构点
|
||||
|
||||
#### 3.2.1 doc 模块拆分
|
||||
|
||||
| 职责 | 建议移动位置 |
|
||||
|------|-------------|
|
||||
| `validator.py` | `core/validation/` |
|
||||
| `matcher.py` | `core/matching/` |
|
||||
| `llm_extractor.py` | `core/extraction/` |
|
||||
| `extractor.py` | `core/extraction/` |
|
||||
| `invoice.py` | `infra/documents/` |
|
||||
| `pdf.py` | `infra/documents/` |
|
||||
| `fill_consumable_doc.py` | `infra/documents/` |
|
||||
| `prompt.py` + `prompts/` | `infra/llm/` |
|
||||
|
||||
#### 3.2.2 消除重复逻辑
|
||||
|
||||
**问题**: `pipeline_web.py` 和 `pipeline_core.py` 都有相似逻辑：
|
||||
- `is_travel_invoice()`
|
||||
- `extract_and_cache_travel_info()`
|
||||
- `extract_and_cache_normal_info()`
|
||||
|
||||
**建议**: 将这些公共逻辑统一到 `core/pipeline/` 目录，两个入口调用同一模块。
|
||||
|
||||
#### 3.2.3 Validator 重构
|
||||
|
||||
**当前问题**:
|
||||
- 校验规则硬编码
|
||||
- `FieldRule` 和 `ArrayRule` 类过于复杂
|
||||
|
||||
**建议**:
|
||||
- 将校验规则外部化为 JSON/YAML 配置文件
|
||||
- 简化 `FieldRule` 为单一数据结构
|
||||
- 统一顶层字段和数组元素字段的校验方式
|
||||
|
||||
#### 3.2.4 Agent 拆分
|
||||
|
||||
**当前问题**:
|
||||
- `orchestrator.py` 包含：状态机、SSE 事件、校验循环、提取逻辑
|
||||
|
||||
**建议**:
|
||||
```
|
||||
agent/
|
||||
├── session.py # 状态机定义 + 会话数据模型
|
||||
├── coordinator.py # 校验-修正循环
|
||||
├── events.py # SSE 事件发射
|
||||
└── orchestrator.py # 总调度入口
|
||||
```
|
||||
|
||||
### 3.3 接口契约强化
|
||||
|
||||
| 模块 | 依赖关系 | 接口契约 |
|
||||
|------|----------|----------|
|
||||
| `core/extraction` | 被 `apps/*` 调用 | 返回 `(payment_records, applications, groups)` |
|
||||
| `core/validation` | 被 `agent/*` 调用 | `validate(info, rules) -> ValidationReport` |
|
||||
| `core/matching` | 被 `extraction` 调用 | `match(invoices, cards) -> List[Dict]` |
|
||||
| `infra/browser` | 被 `apps/*` 调用 | `run(bot, info) -> None` |
|
||||
| `infra/llm` | 被 `core/extraction` 调用 | `extract_document(file) -> dict` |
|
||||
|
||||
---
|
||||
|
||||
## 四、优先重构顺序
|
||||
|
||||
### 第一阶段（降低耦合）
|
||||
1. 将 `doc/` 拆分为 `core/` + `infra/`
|
||||
2. 消除 `pipeline_web.py` 和 `pipeline_core.py` 的重复逻辑
|
||||
3. 将 `bot/` 移动到 `infra/browser/`
|
||||
|
||||
### 第二阶段（职责清晰化）
|
||||
4. 拆分 `agent/orchestrator.py` 为多个模块
|
||||
5. 外部化 `validator.py` 的校验规则为配置文件
|
||||
6. 统一 SSE 事件处理接口
|
||||
|
||||
### 第三阶段（可维护性）
|
||||
7. 完善 `__init__.py` 的接口导出
|
||||
8. 添加模块间依赖注入机制
|
||||
9. 建立跨模块调用规范
|
||||
|
||||
---
|
||||
|
||||
## 五、当前项目优点
|
||||
|
||||
1. **日志规范**: 统一的 `get_logger()` 方式，全局日志管理
|
||||
2. **异常体系**: 清晰的 `ReimbursementError` 异常层次
|
||||
3. **SSE 事件协议**: 良好的实时反馈机制
|
||||
4. **缓存设计**: `llm_extractor.py` 的缓存加载逻辑完善
|
||||
5. **声明式校验**: `validator.py` 的规则配置思路正确
|
||||
|
||||
---
|
||||
|
||||
*生成时间: 2026-06-15*
|
||||
93
AGENTS.md
93
AGENTS.md
@@ -4,26 +4,93 @@ alwaysApply: true
|
||||
---
|
||||
|
||||
---
|
||||
last_reviewed: 2026-06-09
|
||||
last_reviewed: 2026-07-02
|
||||
---
|
||||
|
||||
# AGENTS 索引
|
||||
# AGENTS — 项目操作指南
|
||||
|
||||
本文件是规则的入口。详细策略文本位于 `.agents/docs/standards/*.md`。
|
||||
本文件为 Agent 提供高信号量的项目操作知识，避免重复探索。
|
||||
|
||||
## 文档边界
|
||||
|
||||
* 一定不要用**表情文字**输出任何内容，禁止！！！！！
|
||||
* `docs/` 目录专门存放面向开源用户、外部贡献者的项目公开文档及说明文件。
|
||||
* 维护规范、实施方案、经验总结、拉取请求佐证材料与各类内部记录资料，均统一放置在 `.agents/` 目录下，避免内部自动化流程相关内容混入公开文档目录。
|
||||
* 每个文件夹下都有一个 `README.md` 文件用来交代这个文件夹的作用以及重要的信息。
|
||||
* **禁止使用表情文字**输出任何内容。
|
||||
* `docs/` 目录存放面向开源用户、外部贡献者的公开文档。
|
||||
* `.agents/` 目录存放维护规范、实施方案、经验总结等内部资料。
|
||||
* 每个文件夹下都有 `README.md` 说明该文件夹的作用和重要信息。
|
||||
|
||||
## 标准目录
|
||||
## 开发命令（必须使用 uv）
|
||||
|
||||
* 标准文档元数据：`.agents/docs/standards/README.md`
|
||||
* 调试规范：`.agents/docs/standards/调试规范.md`
|
||||
* 复利式工程实践：`.agents/docs/standards/复利式工程实践.md`
|
||||
项目使用 `uv` 管理依赖，所有包版本锁定在 `uv.lock` 中。
|
||||
|
||||
## 项目的架构思想
|
||||
| 操作 | Makefile (跨平台) | tasks.py (Windows) |
|
||||
|------|-------------------|---------------------|
|
||||
| 安装依赖 + pre-commit | `make install` | `python tasks.py install` |
|
||||
| 代码检查（lint+format+typecheck+deptry） | `make check` | `python tasks.py check` |
|
||||
| 运行测试（含覆盖率报告） | `make test` | `python tasks.py test` |
|
||||
| 运行 CLI 全流程 | `make run` | `python tasks.py run` |
|
||||
| 清理缓存和虚拟环境 | `make clean` | `python tasks.py clean` |
|
||||
|
||||
* Agent 是负责调度的中枢，负责调度各个模块
|
||||
**注意：** `tasks.py` 中的 `check` 命令使用 `&&` 连接，Windows PowerShell 不支持 `&&`，但 `tasks.py` 内部已处理为单行字符串。
|
||||
|
||||
## 代码质量工具链（执行顺序）
|
||||
|
||||
1. **Ruff lint** — `uv run ruff check .` (select: E, F, W, I, N, UP, B; ignore: E501)
|
||||
2. **Ruff format** — `uv run ruff format --check .` (line-length: 120)
|
||||
3. **MyPy strict mode** — `uv run mypy src/main.py` (strict=true, warn_return_any, ignore_missing_imports)
|
||||
4. **deptry** — `uv run deptry .` (检测未声明、未使用、过时依赖)
|
||||
|
||||
### pre-commit 钩子（仅 Ruff）
|
||||
|
||||
`.pre-commit-config.yaml` 配置了两个 hook：
|
||||
- `ruff --fix` — lint 并自动修复
|
||||
- `ruff-format` — 格式化
|
||||
|
||||
**注意：** MyPy 和 deptry **不在** pre-commit 中，需要手动运行 `make check`。
|
||||
|
||||
## 项目架构（Agent 调度模式）
|
||||
|
||||
核心入口：`src/agent/orchestrator.py` — Agent 是负责调度的中枢，协调以下模块：
|
||||
- `extraction/extractor.py` — 文件扫描 → LLM 多模态提取 → JSON 缓存
|
||||
- `matching/matcher.py` — 支付记录与发票金额匹配
|
||||
- `validation/validator.py` — 声明式校验器（规则配置与引擎分离）
|
||||
- `infra/browser/travel.py` / `normal.py` — 浏览器自动化填报
|
||||
|
||||
### 数据流关键产物
|
||||
|
||||
| 文件 | 生成阶段 | 作用 |
|
||||
|------|---------|------|
|
||||
| `.invoice_cache/*.json` | extractor 提取 | 单张发票/支付记录的结构化数据 |
|
||||
| `match_result.json` | matcher 匹配 | 支付截图与发票的关联关系 |
|
||||
| `travel_info.json` / `normal_info.json` | LLM 综合提取 | 差旅/普通报销所需的全部结构化数据 |
|
||||
| `invoice_summary.csv` | extractor 提取 | 普通发票汇总（用于生成易耗品出库单） |
|
||||
|
||||
### 缓存机制
|
||||
|
||||
CLI 模式：`scripts/data/.invoice_cache/`
|
||||
Web 模式：`src/web/uploads/<session_id>/.invoice_cache/`
|
||||
|
||||
缓存文件与源文件同名（如 `发票1.pdf` → `.invoice_cache/发票1.json`），后续步骤均从缓存读取。删除缓存后下次处理会重新提取。
|
||||
|
||||
## 重要约束
|
||||
|
||||
* **Windows-only**：易耗品出库单填写依赖 Microsoft Word + COM (`pywin32`)，仅 Windows 可用
|
||||
* **浏览器自动化**：使用 Playwright，填报时会打开 Chromium，请勿手动干扰
|
||||
* **敏感信息**：`scripts/config.json` 含登录凭据，勿提交到公开仓库
|
||||
* **发票类型区分**：差旅发票（高铁票/酒店住宿）不生成易耗品出库单，走差旅报销流程；普通发票生成出库单
|
||||
|
||||
## Web 服务
|
||||
|
||||
```bash
|
||||
uv run python src/web/app.py
|
||||
# 访问 http://localhost:5000
|
||||
```
|
||||
|
||||
Web 端浏览器填报以无头模式运行。会话产物存放在 `src/web/uploads/<session_id>/`，每次上传生成独立会话。
|
||||
|
||||
## 测试
|
||||
|
||||
```bash
|
||||
make test # pytest + coverage report (term-missing)
|
||||
```
|
||||
|
||||
测试目录：`tests/`，配置在 `pyproject.toml` 中 (`testpaths = ["tests"]`, `pythonpath = ["."]`)。
|
||||
|
||||
128
README.md
128
README.md
@@ -18,20 +18,40 @@
|
||||
├── src/
|
||||
│ ├── __init__.py # 包初始化 / 日志器
|
||||
│ ├── config.py # 配置加载
|
||||
│ ├── bot.py # 浏览器自动填报
|
||||
│ ├── exceptions.py # 异常定义
|
||||
│ ├── pipeline.py # CLI 流程编排
|
||||
│ ├── pipeline_core.py # CLI/Web 公共管道逻辑
|
||||
│ ├── main.py # CLI 入口
|
||||
│ ├── doc/ # 文档处理模块
|
||||
│ │ ├── extractor.py # 编排入口：串联 PDF 读取 → LLM 提取 → 分类
|
||||
│ │ ├── pdf.py # PDF 图片渲染（PyMuPDF，供多模态 LLM 使用）
|
||||
│ │ ├── llm_extractor.py # LLM 信息提取
|
||||
│ │ ├── matcher.py # 数据匹配与校验
|
||||
│ │ ├── invoice.py # 发票类型常量、分类逻辑、CSV 读写工具
|
||||
│ │ ├── fill_consumable_doc.py # 将 CSV 填入易耗品出库单（Word COM）
|
||||
│ │ ├── prompt.py # LLM 提示词模板
|
||||
│ │ └── prompts/ # 提示词模板文件
|
||||
│ └── web/
|
||||
│ ├── app.py # Web 服务入口
|
||||
│ ├── agent/ # Agent 调度模块
|
||||
│ │ ├── orchestrator.py # 总调度入口
|
||||
│ │ ├── coordinator.py # 校验-修正循环
|
||||
│ │ ├── session.py # 状态机与会话数据
|
||||
│ │ └── events.py # SSE 事件发射
|
||||
│ ├── core/ # 核心业务逻辑
|
||||
│ │ ├── extraction/ # 信息提取
|
||||
│ │ │ ├── extractor.py # 编排入口：串联文件扫描 → 提取 → 分类
|
||||
│ │ │ └── llm_extractor.py # LLM 多模态信息提取
|
||||
│ │ ├── matching/ # 金额匹配
|
||||
│ │ │ └── matcher.py # 支付记录与发票关联
|
||||
│ │ └── validation/ # 校验模块
|
||||
│ │ └── validator.py # 声明式校验器
|
||||
│ ├── infra/ # 基础设施层
|
||||
│ │ ├── browser/ # 浏览器自动化
|
||||
│ │ │ ├── base.py # BaseBot 基类
|
||||
│ │ │ ├── travel.py # 差旅报销填报流程
|
||||
│ │ │ └── normal.py # 普通报销填报流程
|
||||
│ │ ├── documents/ # 文档处理
|
||||
│ │ │ ├── invoice.py # 发票数据模型 + CSV 工具
|
||||
│ │ │ ├── pdf.py # PDF 图片渲染
|
||||
│ │ │ └── consumable.py # 易耗品出库单填写（Word COM）
|
||||
│ │ └── llm/ # LLM 接口
|
||||
│ │ ├── prompt.py # 提示词加载
|
||||
│ │ └── prompts/ # 提示词模板文件
|
||||
│ └── web/ # Web 界面模块
|
||||
│ ├── app.py # Flask 应用入口
|
||||
│ ├── routes.py # 路由定义
|
||||
│ ├── pipeline_web.py # Web 管道逻辑
|
||||
│ ├── sse_handler.py # SSE 日志流处理
|
||||
│ ├── templates/
|
||||
│ │ ├── index.html # PC 端主页
|
||||
│ │ └── mobile_upload.html # 移动端扫码上传
|
||||
@@ -48,6 +68,66 @@
|
||||
└── *.pdf / *.jpg / *.png # 发票 PDF 或图片（CLI 模式，放在 scripts/data/）
|
||||
```
|
||||
|
||||
## 声明式校验器
|
||||
|
||||
`src/core/validation/validator.py` 采用**规则配置与校验引擎分离**的设计模式，支持声明式定义校验规则：
|
||||
|
||||
### 设计特点
|
||||
|
||||
| 特性 | 说明 |
|
||||
|------|------|
|
||||
| **声明式配置** | 校验规则以数据结构形式定义，无需编写代码 |
|
||||
| **统一路径定位** | 使用 `path` 统一定位字段，如 `["basic_info", "travel_purpose"]` |
|
||||
| **自定义校验函数** | 支持为字段定义自定义校验逻辑（日期格式、正数检查等） |
|
||||
| **数组元素校验** | 支持校验数组字段的最小元素数量及每个元素的必填字段 |
|
||||
| **向后兼容** | 支持简单格式 `["field1", "field2"]` 和详细格式 `{"path": [...], "custom_check": ...}` |
|
||||
|
||||
### 规则配置示例
|
||||
|
||||
```python
|
||||
# 差旅报销校验规则
|
||||
TRAVEL_VALIDATION_RULES = {
|
||||
"fields": [
|
||||
{"path": ["basic_info", "travel_purpose"], "description": "出差事由"},
|
||||
{"path": ["basic_info", "start_date"], "custom_check": _is_valid_date},
|
||||
],
|
||||
"arrays": [
|
||||
{
|
||||
"path": ["payment_methods"],
|
||||
"min_items": 1, # 至少1条支付记录
|
||||
"element_fields": [
|
||||
{"path": ["card_date"], "description": "刷卡日期"},
|
||||
{"path": ["card_amount"], "custom_check": _is_positive_number},
|
||||
],
|
||||
},
|
||||
],
|
||||
}
|
||||
```
|
||||
|
||||
### 校验规则类型
|
||||
|
||||
| 规则类型 | 用途 | 关键字段 |
|
||||
|----------|------|----------|
|
||||
| `fields` | 顶层单值字段校验 | `path`, `custom_check`, `check_empty` |
|
||||
| `arrays` | 数组字段校验 | `path`, `min_items`, `element_fields` |
|
||||
|
||||
### 内置校验函数
|
||||
|
||||
- `_is_valid_date(value)` — 检查日期格式是否为 `YYYY-MM-DD`
|
||||
- `_is_positive_number(value)` — 检查值是否为正数
|
||||
|
||||
### 扩展自定义校验
|
||||
|
||||
```python
|
||||
# 定义自定义校验函数
|
||||
def check_vehicle_type(value):
|
||||
valid_types = ["飞机", "火车", "汽车", "打车"]
|
||||
return isinstance(value, str) and value.strip() in valid_types
|
||||
|
||||
# 在规则中使用
|
||||
{"path": ["vehicle_type"], "custom_check": check_vehicle_type}
|
||||
```
|
||||
|
||||
## 数据流
|
||||
|
||||
```mermaid
|
||||
@@ -73,21 +153,21 @@ flowchart TB
|
||||
MatchResult --> NormalLLM
|
||||
NormalLLM --> NormalInfo[(normal_info.json)]
|
||||
|
||||
TravelInfo -->|差旅基本信息| Bot_T[bot/travel.py<br/>差旅填报流程]
|
||||
TravelInfo -->|差旅基本信息| Bot_T[infra/browser/travel.py<br/>差旅填报流程]
|
||||
TravelInfo -->|报销明细| Bot_T
|
||||
TravelInfo -->|支付方式| Bot_T
|
||||
TravelInfo -->|补助清单| Bot_T
|
||||
TravelInfo -->|附件清单| Bot_T
|
||||
Bot_T --> Submit_T[差旅报销提交]
|
||||
|
||||
NormalInfo -->|报销说明| Bot_G[bot/normal.py<br/>普通填报流程]
|
||||
NormalInfo -->|报销说明| Bot_G[infra/browser/normal.py<br/>普通填报流程]
|
||||
NormalInfo -->|发票总数/金额| Bot_G
|
||||
NormalInfo -->|支付方式| Bot_G
|
||||
NormalInfo -->|附件清单| Bot_G
|
||||
Bot_G --> Submit_G[普通报销提交]
|
||||
|
||||
General --> CSV[(invoice_summary.csv)]
|
||||
CSV --> Fill[fill_consumable_doc]
|
||||
CSV --> Fill[consumable.py]
|
||||
Fill --> Doc[易耗品、出库单.doc]
|
||||
```
|
||||
|
||||
@@ -103,14 +183,14 @@ flowchart TB
|
||||
|
||||
### bot 模块架构
|
||||
|
||||
`bot/` 包负责浏览器自动化填报，仅接收已提取的信息并执行填报操作，不承担信息提取职责：
|
||||
`infra/browser/` 包负责浏览器自动化填报，仅接收已提取的信息并执行填报操作，不承担信息提取职责：
|
||||
|
||||
| 模块 | 职责 |
|
||||
|------|------|
|
||||
| `bot/base.py` | `BaseBot` 基类：浏览器生命周期、登录、导航、截图 |
|
||||
| `bot/travel.py` | 差旅填报流程：基本信息 → 差旅明细 → 支付方式 → 补助清单 → 附件上传 |
|
||||
| `bot/normal.py` | 普通填报流程：基本信息 → 总明细 → 支付方式 → 附件上传 |
|
||||
| `bot/__init__.py` | 入口函数：`run_bot()` / `run_bot_web()`，负责类型判断和流程路由 |
|
||||
| `infra/browser/base.py` | `BaseBot` 基类：浏览器生命周期、登录、导航、截图 |
|
||||
| `infra/browser/travel.py` | 差旅填报流程：基本信息 → 差旅明细 → 支付方式 → 补助清单 → 附件上传 |
|
||||
| `infra/browser/normal.py` | 普通填报流程：基本信息 → 总明细 → 支付方式 → 附件上传 |
|
||||
| `infra/browser/__init__.py` | 入口函数：`run_bot()` / `run_bot_web()`，负责类型判断和流程路由 |
|
||||
|
||||
## 环境要求
|
||||
|
||||
@@ -200,10 +280,10 @@ uv run python src/main.py -u 工号 -p 密码
|
||||
需已生成 `invoice_summary.csv`，且本机已安装 **Microsoft Word**：
|
||||
|
||||
```bash
|
||||
uv run python -m src.doc.fill_consumable_doc
|
||||
uv run python -m src.doc.fill_consumable_doc --csv invoice_summary.csv --doc "易耗品、出库单.doc"
|
||||
uv run python -m src.doc.fill_consumable_doc --config scripts/config.json # 指定配置文件
|
||||
uv run python -m src.doc.fill_consumable_doc --no-backup # 不生成 .doc.bak 备份
|
||||
uv run python -m src.infra.documents.consumable
|
||||
uv run python -m src.infra.documents.consumable --csv invoice_summary.csv --doc "易耗品、出库单.doc"
|
||||
uv run python -m src.infra.documents.consumable --config scripts/config.json # 指定配置文件
|
||||
uv run python -m src.infra.documents.consumable --no-backup # 不生成 .doc.bak 备份
|
||||
```
|
||||
|
||||
填写规则概要：
|
||||
|
||||
24
config/README.md
Normal file
24
config/README.md
Normal file
@@ -0,0 +1,24 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# config — 配置文件目录
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 说明 |
|
||||
|------|------|
|
||||
| `validation_rules.json` | 声明式校验规则配置：定义差旅和普通报销的必填字段、数组元素校验规则和自定义校验函数 |
|
||||
|
||||
## validation_rules.json 结构
|
||||
|
||||
```json
|
||||
{
|
||||
"version": "1.0",
|
||||
"custom_checks": { ... },
|
||||
"travel": { "fields": [...], "arrays": [...] },
|
||||
"normal": { "fields": [...], "arrays": [...] }
|
||||
}
|
||||
```
|
||||
|
||||
校验引擎 `src/core/validation/validator.py` 在启动时读取此文件，若文件不存在则使用内置默认规则。
|
||||
135
config/validation_rules.json
Normal file
135
config/validation_rules.json
Normal file
@@ -0,0 +1,135 @@
|
||||
{
|
||||
"version": "1.0",
|
||||
"custom_checks": {
|
||||
"is_valid_date": "检查日期格式是否为 YYYY-MM-DD",
|
||||
"is_positive_number": "检查是否为正数（整数或浮点数）",
|
||||
"is_positive_integer": "检查是否为正整数"
|
||||
},
|
||||
"travel": {
|
||||
"description": "差旅报销校验规则",
|
||||
"fields": [
|
||||
{
|
||||
"path": ["basic_info", "travel_purpose"],
|
||||
"required": true,
|
||||
"check_empty": true,
|
||||
"description": "出差事由"
|
||||
},
|
||||
{
|
||||
"path": ["basic_info", "travel_location"],
|
||||
"required": true,
|
||||
"check_empty": true,
|
||||
"description": "出差地点"
|
||||
},
|
||||
{
|
||||
"path": ["basic_info", "start_date"],
|
||||
"required": true,
|
||||
"check_empty": true,
|
||||
"custom_check": "is_valid_date",
|
||||
"description": "出差开始日期"
|
||||
},
|
||||
{
|
||||
"path": ["basic_info", "end_date"],
|
||||
"required": true,
|
||||
"check_empty": true,
|
||||
"custom_check": "is_valid_date",
|
||||
"description": "出差结束日期"
|
||||
}
|
||||
],
|
||||
"arrays": [
|
||||
{
|
||||
"path": ["reimbursement_details", "transport_fee"],
|
||||
"min_items": 1,
|
||||
"description": "交通费用明细",
|
||||
"element_fields": [
|
||||
{"path": ["vehicle_type"], "required": true, "check_empty": true, "description": "交通工具类型"},
|
||||
{"path": ["start_date"], "required": true, "check_empty": true, "custom_check": "is_valid_date", "description": "出发日期"},
|
||||
{"path": ["end_date"], "required": true, "check_empty": true, "custom_check": "is_valid_date", "description": "到达日期"},
|
||||
{"path": ["departure_place"], "required": true, "check_empty": true, "description": "出发地"},
|
||||
{"path": ["arrival_place"], "required": true, "check_empty": true, "description": "目的地"},
|
||||
{"path": ["amount"], "required": true, "check_empty": true, "custom_check": "is_positive_number", "description": "金额"},
|
||||
{"path": ["bill_count"], "required": true, "check_empty": true, "custom_check": "is_positive_integer", "description": "票据张数"},
|
||||
{"path": ["remark"], "required": true, "check_empty": false, "description": "备注说明"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"path": ["payment_methods"],
|
||||
"min_items": 1,
|
||||
"description": "支付方式记录",
|
||||
"element_fields": [
|
||||
{"path": ["card_date"], "required": true, "check_empty": true, "custom_check": "is_valid_date", "description": "刷卡日期"},
|
||||
{"path": ["card_amount"], "required": true, "check_empty": true, "custom_check": "is_positive_number", "description": "支付金额"},
|
||||
{"path": ["merchant"], "required": true, "check_empty": true, "description": "商户名称"},
|
||||
{"path": ["remark"], "required": true, "check_empty": false, "description": "备注"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"path": ["subsidy_list"],
|
||||
"min_items": 1,
|
||||
"description": "补助清单",
|
||||
"element_fields": [
|
||||
{"path": ["person_id"], "required": true, "check_empty": true, "description": "人员工号"},
|
||||
{"path": ["person_name"], "required": true, "check_empty": true, "description": "人员姓名"},
|
||||
{"path": ["start_date"], "required": true, "check_empty": true, "custom_check": "is_valid_date", "description": "补助开始日期"},
|
||||
{"path": ["end_date"], "required": true, "check_empty": true, "custom_check": "is_valid_date", "description": "补助结束日期"},
|
||||
{"path": ["days"], "required": true, "check_empty": true, "custom_check": "is_positive_integer", "description": "补助天数"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"path": ["attachments"],
|
||||
"min_items": 0,
|
||||
"description": "附件列表",
|
||||
"element_fields": [
|
||||
{"path": ["filename"], "required": true, "check_empty": true, "description": "文件名"},
|
||||
{"path": ["attachment_type"], "required": true, "check_empty": true, "description": "附件类型"}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"normal": {
|
||||
"description": "普通报销校验规则",
|
||||
"fields": [
|
||||
{
|
||||
"path": ["basic_info", "reimbursement_description"],
|
||||
"required": true,
|
||||
"check_empty": true,
|
||||
"description": "报销事由"
|
||||
},
|
||||
{
|
||||
"path": ["reimbursement_details", "total_invoices"],
|
||||
"required": true,
|
||||
"check_empty": true,
|
||||
"custom_check": "is_positive_integer",
|
||||
"description": "发票总数"
|
||||
},
|
||||
{
|
||||
"path": ["reimbursement_details", "total_amount"],
|
||||
"required": true,
|
||||
"check_empty": true,
|
||||
"custom_check": "is_positive_number",
|
||||
"description": "总金额"
|
||||
}
|
||||
],
|
||||
"arrays": [
|
||||
{
|
||||
"path": ["payment_methods"],
|
||||
"min_items": 1,
|
||||
"description": "支付方式记录",
|
||||
"element_fields": [
|
||||
{"path": ["card_date"], "required": true, "check_empty": true, "custom_check": "is_valid_date", "description": "刷卡日期"},
|
||||
{"path": ["card_amount"], "required": true, "check_empty": true, "custom_check": "is_positive_number", "description": "支付金额"},
|
||||
{"path": ["merchant"], "required": true, "check_empty": true, "description": "商户名称"},
|
||||
{"path": ["remark"], "required": true, "check_empty": false, "description": "备注"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"path": ["attachments"],
|
||||
"min_items": 0,
|
||||
"description": "附件列表",
|
||||
"element_fields": [
|
||||
{"path": ["filename"], "required": true, "check_empty": true, "description": "文件名"},
|
||||
{"path": ["attachment_type"], "required": true, "check_empty": true, "description": "附件类型"}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
14
docs/API.md
14
docs/API.md
@@ -30,7 +30,12 @@ last_reviewed: 2026-06-13
|
||||
| **17** | **POST** | **`/api/agent/user-supplement/<session_id>`** | **通过文字补充信息** |
|
||||
| **18** | **POST** | **`/api/agent/force-submit/<session_id>`** | **强制提交，跳过校验** |
|
||||
|
||||
> 加粗条目为 Agent 多轮校验流程新增接口。推荐使用 `/api/agent/process` 作为主入口，它会在发票提取后自动进行 LLM 校验，校验通过则自动提交到财务系统。
|
||||
> 加粗条目为 Agent 多轮校验流程新增接口。
|
||||
|
||||
### 入口选择建议
|
||||
|
||||
- **推荐使用** `/api/agent/process`：完整流程，包含发票提取、LLM 信息校验、自动提交财务系统。
|
||||
- **仅发票提取** `/api/process`：跳过 Agent 校验，只做文档解析和发票分类。适合调试发票提取本身，或仅需导出 CSV 的场景。
|
||||
|
||||
---
|
||||
|
||||
@@ -366,7 +371,7 @@ Accept: text/event-stream
|
||||
|
||||
检测到 `result.json` 存在时，读取后发送 `done` 事件并断开连接。
|
||||
|
||||
**SSE 超时:** 600 秒。
|
||||
**SSE 超时:** 900 秒。
|
||||
|
||||
---
|
||||
|
||||
@@ -691,6 +696,7 @@ POST /api/agent/force-submit/<session_id>
|
||||
| `agent_request_supplement` | `{type, round, missing_fields, missing_materials, semantic_issues, suggestion}` | 校验未通过 |
|
||||
| `agent_supplement_received` | `{type, files}` | 收到用户补充 |
|
||||
| `agent_force_submit` | `{type, message}` | 用户强制提交 |
|
||||
| `agent_extract_status` | `{type, state, round, attempt, message}` | 校验-修正循环中的每次尝试结果 |
|
||||
| `agent_error` | `{type, message}` | 提取失败 |
|
||||
| `agent_max_rounds` | `{type, message}` | 达到最大轮次 |
|
||||
|
||||
@@ -710,7 +716,7 @@ POST /api/agent/force-submit/<session_id>
|
||||
|
||||
| 事件类型 | 数据结构 | 触发条件 |
|
||||
|---|---|---|
|
||||
| `llm_stream` | `{type, phase, content?}` | LLM 输出流 |
|
||||
| `llm_stream` | `{type, phase, text?}` | LLM 输出流 |
|
||||
|
||||
`phase` 取值: `start` / `reasoning` / `chunk` / `end` / `error`
|
||||
|
||||
@@ -844,7 +850,7 @@ sequenceDiagram
|
||||
不经过 Web、在本地直接填写出库单：
|
||||
|
||||
```bash
|
||||
uv run python -m src.doc.fill_consumable_doc --csv invoice_summary.csv --doc "易耗品、出库单.doc"
|
||||
uv run python -m src.infra.documents.consumable --csv invoice_summary.csv --doc "易耗品、出库单.doc"
|
||||
```
|
||||
|
||||
详见 [README.md](./README.md)。
|
||||
@@ -427,7 +427,7 @@ PC 端生成二维码指向移动端上传页面，手机端上传的图片通
|
||||
### 单独填写出库单
|
||||
|
||||
```bash
|
||||
uv run python -m src.doc.fill_consumable_doc --csv invoice_summary.csv --doc "易耗品、出库单.doc"
|
||||
uv run python -m src.infra.documents.consumable --csv invoice_summary.csv --doc "易耗品、出库单.doc"
|
||||
```
|
||||
|
||||
### 分步执行管道
|
||||
|
||||
@@ -9,7 +9,7 @@ dependencies = [
|
||||
"PyMuPDF>=1.24",
|
||||
"pywin32>=306",
|
||||
"llama-index>=0.12.0",
|
||||
"llama-index-llms-openai-like==0.7.2",
|
||||
"llama-index-llms-openai-like>=0.7.2",
|
||||
"python-dotenv>=1.0",
|
||||
]
|
||||
|
||||
@@ -38,10 +38,6 @@ warn_return_any = true
|
||||
warn_unused_configs = true
|
||||
ignore_missing_imports = true
|
||||
|
||||
[[tool.mypy.overrides]]
|
||||
module = "tests.*"
|
||||
ignore_errors = true
|
||||
|
||||
[tool.deptry]
|
||||
ignore_notebooks = true
|
||||
|
||||
|
||||
20
scripts/README.md
Normal file
20
scripts/README.md
Normal file
@@ -0,0 +1,20 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# scripts — 调试脚本与数据目录
|
||||
|
||||
## 子目录
|
||||
|
||||
| 目录 | 说明 |
|
||||
|------|------|
|
||||
| `data/` | CLI 模式的数据目录：发票源文件、`config.json`、`.invoice_cache` 缓存 |
|
||||
|
||||
## 脚本
|
||||
|
||||
| 文件 | 说明 |
|
||||
|------|------|
|
||||
| `debug_stream_fields.py` | 诊断 stream_chat 返回对象的字段结构 |
|
||||
| `test_application_extract.py` | 测试出差申请单提取 |
|
||||
| `test_multimodal.py` | 测试多模态 LLM 识别 |
|
||||
| `test_travel_info.py` | 测试差旅信息提取 |
|
||||
@@ -10,7 +10,7 @@ from dotenv import load_dotenv # noqa: E402
|
||||
from llama_index.core.llms import ChatMessage # noqa: E402
|
||||
|
||||
from src.config import get_llm_config # noqa: E402
|
||||
from src.doc.llm_extractor import _create_llm # noqa: E402
|
||||
from src.core.extraction import _create_llm # noqa: E402
|
||||
|
||||
load_dotenv(Path(__file__).parent / ".env")
|
||||
|
||||
|
||||
@@ -31,7 +31,7 @@ sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8", errors="repla
|
||||
|
||||
sys.path.insert(0, str(ROOT)) # noqa: E402
|
||||
|
||||
from src.doc.llm_extractor import extract_document # noqa: E402
|
||||
from src.core.extraction import extract_document # noqa: E402
|
||||
|
||||
|
||||
def test_single_file(file_path: Path) -> None:
|
||||
|
||||
@@ -15,8 +15,8 @@ sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8", errors="repla
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT)) # noqa: E402
|
||||
|
||||
from src.doc.llm_extractor import extract_document # noqa: E402
|
||||
from src.doc.pdf import render_pdf_to_images # noqa: E402
|
||||
from src.core.extraction import extract_document # noqa: E402
|
||||
from src.infra.documents.pdf import render_pdf_to_images # noqa: E402
|
||||
|
||||
|
||||
def test_render() -> None:
|
||||
|
||||
@@ -15,7 +15,7 @@ from pathlib import Path
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT)) # noqa: E402
|
||||
|
||||
from src.doc.llm_extractor import extract_travel_info # noqa: E402
|
||||
from src.core.extraction import extract_travel_info # noqa: E402
|
||||
|
||||
|
||||
def main() -> None:
|
||||
|
||||
@@ -1,74 +1,77 @@
|
||||
---
|
||||
last_reviewed: 2026-06-12
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src — 主源码目录
|
||||
|
||||
包含财务报销自动化系统的核心模块。
|
||||
包含财务报销自动化系统的全部源码模块。
|
||||
|
||||
## 架构分层
|
||||
|
||||
```
|
||||
src/
|
||||
├── agent/ Agent 调度层（协调提取-校验-修正循环）
|
||||
├── core/ 核心业务层（提取、匹配、校验）
|
||||
├── infra/ 基础设施层（浏览器、文档、LLM 提示词）
|
||||
├── web/ Web 界面层（Flask + SSE）
|
||||
├── pipeline.py CLI 流程编排
|
||||
├── pipeline_core.py CLI/Web 公共管道逻辑
|
||||
├── main.py CLI 入口
|
||||
├── config.py 配置加载
|
||||
└── exceptions.py 异常定义
|
||||
```
|
||||
|
||||
## 模块清单
|
||||
|
||||
| 文件/目录 | 说明 |
|
||||
|-----------|------|
|
||||
| `__init__.py` | 包初始化：提供 `get_logger()` 日志工厂（支持终端 + 文件双输出，按日期自动分文件） |
|
||||
| `config.py` | 配置加载：从 `config.json` 读取用户凭据和默认值，从环境变量读取服务端配置（SSO 地址、LLM 参数） |
|
||||
| `pipeline.py` | 流程编排：串联发票提取 → 类型判断 → 差旅/普通信息提取 → 浏览器填报，支持分步执行 |
|
||||
| `main.py` | CLI 入口：支持 `--step` 分步执行、`-u/-p` 覆盖凭据、`--cache-dir` 指定缓存目录 |
|
||||
| `bot/` | 浏览器自动化：Playwright 驱动的财务系统填报机器人（仅负责接收信息并填报） |
|
||||
| `doc/` | 文档处理模块：PDF 渲染、LLM 提取、支付匹配、发票分类、出库单生成 |
|
||||
| `web/` | Web 界面模块：Flask 应用、SSE 日志、可编辑表格、移动端上传、会话隔离 |
|
||||
| `agent/` | Agent 调度：校验-修正循环、状态机管理、SSE 事件发射 |
|
||||
| `core/` | 核心业务逻辑：信息提取、金额匹配、信息校验 |
|
||||
| `infra/` | 基础设施：浏览器自动填报、文档处理、LLM 提示词管理 |
|
||||
| `web/` | Web 界面：Flask 应用、SSE 日志流、可编辑表格、移动端上传、会话隔离 |
|
||||
| `pipeline.py` | CLI 流程编排：串联提取 → 类型判断 → 信息提取 → 浏览器填报 |
|
||||
| `pipeline_core.py` | CLI/Web 公共管道逻辑：发票类型判断、缓存提取 |
|
||||
| `main.py` | CLI 入口：`--step` 分步执行、`-u/-p` 覆盖凭据 |
|
||||
| `config.py` | 配置加载：`config.json` + 环境变量 |
|
||||
| `exceptions.py` | 异常层次定义 |
|
||||
|
||||
## 数据流
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[CLI/Web 入口] --> B["pipeline.py (编排)"]
|
||||
B --> C["doc/extractor.py (统一提取入口)"]
|
||||
C --> D["doc/pdf.py (PDF 渲染为图片)"]
|
||||
C --> E["doc/llm_extractor.py (多模态 LLM 识别)"]
|
||||
B --> C["core/extraction/extractor.py (统一提取入口)"]
|
||||
C --> D["infra/documents/pdf.py (PDF 渲染为图片)"]
|
||||
C --> E["core/extraction/llm_extractor.py (多模态 LLM 识别)"]
|
||||
E --> F["发票 invoice_type=train/hotel/general"]
|
||||
E --> G["支付记录 invoice_type=payment"]
|
||||
E --> H["出差事前申请单 invoice_type=application"]
|
||||
C --> I["doc/matcher.py (发票与支付记录按金额匹配)"]
|
||||
C --> I["core/matching/matcher.py (发票与支付记录按金额匹配)"]
|
||||
I --> J["一对一匹配 发票数 == 刷卡数"]
|
||||
I --> K["一对多匹配 贪心算法 相对容差 3%"]
|
||||
C --> L["doc/invoice.py (CSV/JSON 读写)"]
|
||||
C --> L["infra/documents/invoice.py (CSV/JSON 读写)"]
|
||||
L --> M["payment_records.csv (支付记录级别)"]
|
||||
L --> N["invoice_summary.csv (发票级别)"]
|
||||
L --> O["travel_applications.json (出差申请单)"]
|
||||
B --> R{"判断报销类型"}
|
||||
R -->|差旅| T["doc/llm_extractor.py (差旅信息提取)"]
|
||||
R -->|普通| V["doc/llm_extractor.py (普通发票信息提取)"]
|
||||
R -->|差旅| T["core/extraction/llm_extractor.py (差旅信息提取)"]
|
||||
R -->|普通| V["core/extraction/llm_extractor.py (普通发票信息提取)"]
|
||||
T --> W["travel_info.json (差旅信息: 交通/住宿明细、补贴、附件清单)"]
|
||||
V --> X["normal_info.json (普通发票信息: 报销说明、发票总数、总金额、支付方式、附件清单)"]
|
||||
W --> P["bot/ (浏览器填报 - 仅接收信息并填报)"]
|
||||
W --> P["infra/browser/ (浏览器填报 - 仅接收信息并填报)"]
|
||||
X --> P
|
||||
P --> Q["差旅模式: travel_info.json → 填报差旅单 → 上传差旅附件"]
|
||||
P --> S["普通模式: 基本信息 → 录入明细 → 支付信息 → 上传附件"]
|
||||
```
|
||||
|
||||
**数据流变更（2026-06-11）：** 差旅信息提取从 `bot.py` 提升到 `pipeline.py` 编排层。在发票提取和匹配完成后立即判断报销类型，差旅发票调用 LLM 提取 `travel_info.json`，普通发票调用 LLM 提取 `normal_info.json`。Bot 仅负责接收信息并填报，不再承担信息提取职责。
|
||||
## 子模块文档
|
||||
|
||||
## 文档处理子模块 (`doc/`)
|
||||
|
||||
详见 [`doc/README.md`](doc/README.md)
|
||||
|
||||
核心能力：
|
||||
- **统一文档提取**：LLM 自行判断文档类型（发票/支付记录/出差事前申请单），无需正则回退
|
||||
- **JSON 缓存**：提取结果缓存于 `.invoice_cache/`，避免重复处理
|
||||
- **金额匹配**：支持一对多匹配，相对容差 3%，未匹配发票单独列为记录
|
||||
- **差旅信息提取**：综合发票、支付记录和匹配结果，提取出差事由、地点、时间等
|
||||
- **普通发票信息提取**：综合普通发票、支付记录和匹配结果，提取报销说明、发票总数、总金额、支付方式、附件清单
|
||||
- **出库单生成**：将 CSV 数据填入 Word 模板（pywin32 COM，仅 Windows）
|
||||
|
||||
## Web 界面子模块 (`web/`)
|
||||
|
||||
详见 [`web/README.md`](web/README.md)
|
||||
|
||||
核心能力：
|
||||
- **会话隔离**：每次上传生成独立 `session_id`，文件/日志/配置/结果各自隔离
|
||||
- **移动端同步**：PC 端生成二维码指向 `/mobile/<sid>`，跨设备协作上传
|
||||
- **可编辑表格**：前端加载 CSV 数据，支持在线编辑后保存
|
||||
| 目录 | 文档 |
|
||||
|------|------|
|
||||
| `agent/` | [`agent/README.md`](agent/README.md) |
|
||||
| `core/` | [`core/README.md`](core/README.md) |
|
||||
| `infra/` | [`infra/README.md`](infra/README.md) |
|
||||
| `web/` | [`web/README.md`](web/README.md) |
|
||||
|
||||
## 启动方式
|
||||
|
||||
|
||||
@@ -12,7 +12,6 @@
|
||||
from .orchestrator import (
|
||||
AgentSession,
|
||||
AgentState,
|
||||
_emit_agent_event,
|
||||
add_supplement,
|
||||
force_submit,
|
||||
load_agent_state,
|
||||
@@ -22,7 +21,6 @@ from .orchestrator import (
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"_emit_agent_event",
|
||||
"AgentSession",
|
||||
"AgentState",
|
||||
"add_supplement",
|
||||
|
||||
410
src/agent/coordinator.py
Normal file
410
src/agent/coordinator.py
Normal file
@@ -0,0 +1,410 @@
|
||||
"""Agent 协调器
|
||||
|
||||
作为调度中枢，编排信息提取、规则校验的完整流程。
|
||||
|
||||
校验-修正循环由 Agent 层调度：
|
||||
1. Agent 调用 LLM 提取信息
|
||||
2. Agent 调用 validator.py 校验
|
||||
3. 校验失败则构建修正提示，再次调用 LLM
|
||||
4. 重复直到校验通过或达到最大重试次数
|
||||
5. LLM 在输出中包含 can_submit 和 suggestion 字段，用于判断信息完整性
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
from ..core.extraction import (
|
||||
build_extraction_user_message,
|
||||
llm_query_text,
|
||||
load_cache,
|
||||
load_match_result,
|
||||
merge_supplement_into_info,
|
||||
parse_json_response,
|
||||
process_user_supplement,
|
||||
)
|
||||
from ..core.validation import validate_extracted_info
|
||||
from ..infra.llm import (
|
||||
build_normal_info_system_prompt,
|
||||
build_travel_info_system_prompt,
|
||||
)
|
||||
from ..pipeline_core import save_cache_info
|
||||
from .events import emit_agent_event
|
||||
from .session import AgentSession, AgentState, save_agent_state
|
||||
|
||||
log = get_logger("agent.coordinator")
|
||||
|
||||
# 规则校验-修正循环的最大重试次数
|
||||
MAX_VALIDATION_RETRIES = 3
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 辅助：构建修正提示
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _build_correction_prompt(
|
||||
base_message: str,
|
||||
report: Any,
|
||||
) -> str:
|
||||
"""根据校验报告构建修正提示，追加到原始用户消息后。"""
|
||||
error_feedback = (
|
||||
f"\n\n=== 上一次输出的校验结果 ===\n"
|
||||
f"校验未通过，发现以下问题：\n"
|
||||
f"缺失字段 ({len(report.missing_fields)} 个)：{', '.join(report.missing_fields)}\n"
|
||||
)
|
||||
if report.missing_materials:
|
||||
error_feedback += f"可能需要补充的材料：{', '.join(report.missing_materials)}\n"
|
||||
if report.suggestion:
|
||||
error_feedback += f"建议：{report.suggestion}\n"
|
||||
error_feedback += (
|
||||
"\n请根据以上校验结果修正你的输出，确保所有必填字段都有值。"
|
||||
"如果某个字段确实没有数据，请给出合理的猜测值。"
|
||||
"再次返回完整的 JSON 结果。"
|
||||
)
|
||||
return base_message + error_feedback
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 核心协调逻辑
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _do_extraction_with_validation(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
previous_analysis: dict[str, Any] | None = None,
|
||||
cache_map: dict[str, Any] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Agent 调度的提取-校验-修正循环。
|
||||
|
||||
流程：
|
||||
1. 加载缓存数据和匹配结果
|
||||
2. 构建用户消息
|
||||
3. 调用 LLM 提取
|
||||
4. 调用 validator 校验
|
||||
5. 校验失败则构建修正提示，回到步骤 3
|
||||
6. 最多重试 MAX_VALIDATION_RETRIES 次
|
||||
|
||||
LLM 输出的 JSON 中额外包含 can_submit 和 suggestion 字段，
|
||||
用于判断信息是否完整可提交。
|
||||
|
||||
Args:
|
||||
session_dir: 会话目录。
|
||||
session: 当前 Agent 会话。
|
||||
previous_analysis: 上一轮 LLM 分析结果（可选，补充文件时传入作为历史上下文）。
|
||||
|
||||
Returns:
|
||||
校验通过的结构化数据（或达到重试上限后的最佳结果）。
|
||||
"""
|
||||
cache_map = cache_map or load_cache(session_dir)
|
||||
match_result = load_match_result(session_dir)
|
||||
|
||||
# 选择系统提示词
|
||||
if session.invoice_type == "travel":
|
||||
system_prompt = build_travel_info_system_prompt()
|
||||
else:
|
||||
system_prompt = build_normal_info_system_prompt()
|
||||
|
||||
base_message = build_extraction_user_message(cache_map, match_result, previous_analysis=previous_analysis)
|
||||
current_message = base_message
|
||||
|
||||
for attempt in range(1, MAX_VALIDATION_RETRIES + 1):
|
||||
log.info(
|
||||
"LLM 提取第 %d/%d 次尝试 (%s)",
|
||||
attempt,
|
||||
MAX_VALIDATION_RETRIES,
|
||||
session.invoice_type,
|
||||
)
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_state_change",
|
||||
state=AgentState.EXTRACTING,
|
||||
round=session.rounds,
|
||||
attempt=attempt,
|
||||
message=f"正在分析文件... (第{attempt}次)",
|
||||
)
|
||||
|
||||
# Step 1: 调用 LLM 提取
|
||||
try:
|
||||
response = llm_query_text(
|
||||
system_prompt=system_prompt,
|
||||
text=current_message,
|
||||
reasoning_effort="low",
|
||||
source_dir=session_dir,
|
||||
)
|
||||
result = parse_json_response(response)
|
||||
except Exception as e:
|
||||
log.error("LLM 提取失败: %s", e)
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_error",
|
||||
message=f"LLM 提取失败: {e}",
|
||||
)
|
||||
raise
|
||||
|
||||
# Step 2: 调用 validator 校验
|
||||
report = validate_extracted_info(result, invoice_type=session.invoice_type)
|
||||
|
||||
if report.valid:
|
||||
log.info("规则校验通过 (第 %d 次尝试)", attempt)
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_extract_status",
|
||||
state=AgentState.EXTRACTING,
|
||||
round=session.rounds,
|
||||
attempt=attempt,
|
||||
message=f"规则校验通过 (第{attempt}次)",
|
||||
)
|
||||
return result
|
||||
|
||||
# Step 3: 校验失败，构建修正提示
|
||||
log.warning(
|
||||
"规则校验未通过 (第 %d/%d 次): 缺失 %d 个字段 - %s",
|
||||
attempt,
|
||||
MAX_VALIDATION_RETRIES,
|
||||
len(report.missing_fields),
|
||||
report.missing_fields,
|
||||
)
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_extract_status",
|
||||
state=AgentState.EXTRACTING,
|
||||
round=session.rounds,
|
||||
attempt=attempt,
|
||||
message=f"规则校验未通过，缺失 {len(report.missing_fields)} 个字段，正在请求 LLM 修正...",
|
||||
)
|
||||
current_message = _build_correction_prompt(current_message, report)
|
||||
|
||||
# 所有重试都失败，返回最后一次结果
|
||||
log.error(
|
||||
"LLM 提取经过 %d 次尝试仍未通过规则校验，返回最后一次结果 (置信度: %.0f%%)",
|
||||
MAX_VALIDATION_RETRIES,
|
||||
report.confidence * 100,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def run_agent_round(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
new_files: list[str] | None = None,
|
||||
) -> AgentSession:
|
||||
"""执行一轮 Agent 处理：提取-校验-修正循环。
|
||||
|
||||
Args:
|
||||
session_dir: 会话目录。
|
||||
session: 当前 Agent 会话。
|
||||
new_files: 新增的文件列表（可选，补充文件时传入）。
|
||||
|
||||
Returns:
|
||||
更新后的 Agent 会话。
|
||||
|
||||
注意：
|
||||
- 信息提取优先从 .invoice_cache 缓存读取，避免重复调用 LLM。
|
||||
- Agent 调度提取-校验-修正循环：LLM 提取 -> validator 校验 -> 失败则反馈修正。
|
||||
- 若缓存缺失则执行提取后立即写回缓存（travel_info.json / normal_info.json）。
|
||||
- 补充文件时（new_files 非空），加载上一轮分析结果作为历史上下文，强制重新分析。
|
||||
"""
|
||||
# 终态保护：会话已提交或已完成时不再重复处理
|
||||
if session.is_terminal():
|
||||
log.info("Agent 会话已处于终态 (%s)，跳过重复处理", session.state.value)
|
||||
return session
|
||||
|
||||
if session.rounds >= session.max_rounds:
|
||||
session.state = AgentState.ERROR
|
||||
session.error_message = f"已达到最大轮次 ({session.max_rounds})，请检查信息或强制提交"
|
||||
log.warning("Agent 达到最大轮次限制")
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_max_rounds",
|
||||
message=session.error_message,
|
||||
)
|
||||
return session
|
||||
|
||||
session.rounds += 1
|
||||
log.info("开始第 %d 轮 Agent 处理", session.rounds)
|
||||
|
||||
# ---- Step 1: 信息提取（Agent 调度校验-修正循环） ----
|
||||
session.state = AgentState.EXTRACTING
|
||||
|
||||
# 判断是否为补充文件场景：有新文件传入时，加载上一轮分析结果作为上下文
|
||||
cache_map = load_cache(session_dir)
|
||||
is_supplement = bool(new_files)
|
||||
previous_analysis = None
|
||||
if is_supplement:
|
||||
info_key = "travel_info" if session.invoice_type == "travel" else "normal_info"
|
||||
previous_analysis = cache_map.get(info_key)
|
||||
if previous_analysis:
|
||||
log.info("检测到补充文件，加载上一轮分析结果作为历史上下文")
|
||||
|
||||
try:
|
||||
info_key = "travel_info" if session.invoice_type == "travel" else "normal_info"
|
||||
should_reanalyze = not cache_map.get(info_key) or is_supplement
|
||||
if should_reanalyze:
|
||||
session.extracted_info = _do_extraction_with_validation(
|
||||
session_dir, session, previous_analysis=previous_analysis, cache_map=cache_map
|
||||
)
|
||||
# 提取后立即写入缓存，后续步骤依赖此数据
|
||||
save_cache_info(session_dir, info_key, session.extracted_info)
|
||||
else:
|
||||
session.extracted_info = cache_map[info_key]
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_state_change",
|
||||
state=session.state,
|
||||
round=session.rounds,
|
||||
message="使用缓存数据，无需重新分析",
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
session.state = AgentState.ERROR
|
||||
session.error_message = f"信息提取失败: {e}"
|
||||
log.error("Agent 信息提取失败: %s", e)
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_error",
|
||||
message=session.error_message,
|
||||
)
|
||||
return session
|
||||
|
||||
# ---- 判断结果（从 LLM 提取结果中读 can_submit） ----
|
||||
can_submit = session.extracted_info.get("can_submit", True)
|
||||
suggestion = session.extracted_info.get("suggestion", "")
|
||||
|
||||
if can_submit:
|
||||
session.state = AgentState.READY
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_ready",
|
||||
round=session.rounds,
|
||||
message="信息完整，可以提交",
|
||||
)
|
||||
log.info("Agent 校验通过，信息完整")
|
||||
else:
|
||||
session.state = AgentState.AWAITING_SUPPLEMENT
|
||||
|
||||
combined_suggestion = suggestion or "信息不完整，请补充材料"
|
||||
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_request_supplement",
|
||||
round=session.rounds,
|
||||
missing_fields=[],
|
||||
missing_materials=[],
|
||||
semantic_issues=[],
|
||||
suggestion=combined_suggestion,
|
||||
)
|
||||
log.info("Agent 请求补充: %s", combined_suggestion)
|
||||
|
||||
save_agent_state(session_dir, session)
|
||||
return session
|
||||
|
||||
|
||||
def force_submit(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
) -> AgentSession:
|
||||
"""用户强制提交，跳过校验。"""
|
||||
session.state = AgentState.READY
|
||||
log.info("用户强制提交，跳过校验")
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_force_submit",
|
||||
message="用户选择强制提交",
|
||||
)
|
||||
save_agent_state(session_dir, session)
|
||||
return session
|
||||
|
||||
|
||||
def add_supplement(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
filenames: list[str],
|
||||
) -> AgentSession:
|
||||
"""记录用户补充的文件。"""
|
||||
session.user_supplements.extend(filenames)
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_supplement_received",
|
||||
files=filenames,
|
||||
)
|
||||
log.info("收到用户补充文件: %s", filenames)
|
||||
save_agent_state(session_dir, session)
|
||||
return session
|
||||
|
||||
|
||||
def process_user_text_supplement(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
user_text: str,
|
||||
) -> AgentSession:
|
||||
"""处理用户通过文字补充的信息。
|
||||
|
||||
流程：
|
||||
1. LLM 分析用户文字，提取需要更新的字段
|
||||
2. 合并到已提取的信息中
|
||||
3. 保存到缓存
|
||||
4. 重新执行一轮 Agent 校验
|
||||
|
||||
Args:
|
||||
session_dir: 会话目录。
|
||||
session: 当前 Agent 会话。
|
||||
user_text: 用户输入的文字。
|
||||
|
||||
Returns:
|
||||
更新后的 Agent 会话。
|
||||
"""
|
||||
log.info("收到用户文字补充: %s", user_text)
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_supplement_received",
|
||||
files=[user_text[:50]], # 简短显示
|
||||
)
|
||||
|
||||
# Step 1: LLM 分析用户文字
|
||||
supplement_result = process_user_supplement(
|
||||
user_text=user_text,
|
||||
extracted_info=session.extracted_info,
|
||||
invoice_type=session.invoice_type,
|
||||
source_dir=session_dir,
|
||||
)
|
||||
|
||||
updated_fields = supplement_result.get("updated_fields", {})
|
||||
unparsed = supplement_result.get("unparsed_info", "")
|
||||
|
||||
if updated_fields:
|
||||
# Step 2: 合并到已提取信息
|
||||
session.extracted_info = merge_supplement_into_info(
|
||||
session.extracted_info,
|
||||
updated_fields,
|
||||
)
|
||||
|
||||
# Step 3: 保存到缓存
|
||||
info_key = "travel_info" if session.invoice_type == "travel" else "normal_info"
|
||||
save_cache_info(session_dir, info_key, session.extracted_info)
|
||||
log.info("已更新 %s", info_key)
|
||||
|
||||
# Step 4: 重新执行 Agent 校验
|
||||
session.state = AgentState.EXTRACTING
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_state_change",
|
||||
state=session.state,
|
||||
round=session.rounds,
|
||||
message="正在重新校验...",
|
||||
)
|
||||
session = run_agent_round(session_dir, session)
|
||||
else:
|
||||
# 没有可更新的字段
|
||||
msg = unparsed or "未识别到可更新的报销信息"
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_supplement_received",
|
||||
files=[msg],
|
||||
)
|
||||
log.info("用户补充未识别到有效信息: %s", msg)
|
||||
|
||||
return session
|
||||
75
src/agent/events.py
Normal file
75
src/agent/events.py
Normal file
@@ -0,0 +1,75 @@
|
||||
"""Agent 事件系统
|
||||
|
||||
负责 SSE 事件的发射和管理，用于实时通知前端状态变化。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
|
||||
log = get_logger("agent.events")
|
||||
|
||||
AGENT_EVENT_LOG = "agent_events.log"
|
||||
|
||||
# 去重守卫：记录每个 session 上一次发射的事件类型，防止连续重复发射
|
||||
# key: str(session_dir), value: 上一次的 event_type
|
||||
_last_event_type: dict[str, str] = {}
|
||||
|
||||
|
||||
def emit_agent_event(session_dir: Path, event_type: str, **kwargs: Any) -> None:
|
||||
"""向 agent_events.log 追加一行 JSON 事件。
|
||||
|
||||
同一 session 连续发射相同 event_type 时直接抛出 RuntimeError，
|
||||
强制调用方修复重复发射的代码，而非静默掩盖。
|
||||
|
||||
注意：agent_state_change 会在同一轮提取中多次发射不同消息
|
||||
（"正在分析"、"校验通过"、"校验失败，请求修正"），这是合法行为。
|
||||
去重守卫仅检查 event_type 字符串是否完全相同，不检查 kwargs。
|
||||
因此不要在同一个 event_type 下连续发射不同消息，应使用不同的事件类型。
|
||||
"""
|
||||
session_key = str(session_dir)
|
||||
prev = _last_event_type.get(session_key)
|
||||
if prev == event_type:
|
||||
raise RuntimeError(
|
||||
f"事件重复发射: session={session_dir.name!r}, event_type={event_type!r}。"
|
||||
f"请检查调用链，确保每个事件类型只发射一次。"
|
||||
)
|
||||
_last_event_type[session_key] = event_type
|
||||
|
||||
event = {"type": event_type, **kwargs}
|
||||
try:
|
||||
event_path = session_dir / AGENT_EVENT_LOG
|
||||
with open(event_path, "a", encoding="utf-8") as f:
|
||||
f.write(json.dumps(event, ensure_ascii=False) + "\n")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def clear_event_history(session_dir: Path) -> None:
|
||||
"""清除指定会话的事件历史。"""
|
||||
session_key = str(session_dir)
|
||||
_last_event_type.pop(session_key, None)
|
||||
event_path = session_dir / AGENT_EVENT_LOG
|
||||
if event_path.exists():
|
||||
event_path.unlink()
|
||||
|
||||
|
||||
def read_events(session_dir: Path) -> list[dict[str, Any]]:
|
||||
"""读取指定会话的所有事件记录。"""
|
||||
event_path = session_dir / AGENT_EVENT_LOG
|
||||
if not event_path.exists():
|
||||
return []
|
||||
events = []
|
||||
try:
|
||||
with open(event_path, encoding="utf-8") as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if line:
|
||||
events.append(json.loads(line))
|
||||
except Exception as e:
|
||||
log.warning("读取事件日志失败: %s", e)
|
||||
return events
|
||||
@@ -1,532 +1,46 @@
|
||||
"""Agent 协调器
|
||||
"""Agent 协调器（兼容层）
|
||||
|
||||
作为调度中枢，编排信息提取、规则校验的完整流程。
|
||||
此文件为向后兼容而保留，所有功能已迁移到以下子模块：
|
||||
- session.py: 会话状态定义和持久化
|
||||
- events.py: SSE 事件发射系统
|
||||
- coordinator.py: 核心协调逻辑
|
||||
|
||||
校验-修正循环由 Agent 层调度：
|
||||
1. Agent 调用 LLM 提取信息
|
||||
2. Agent 调用 validator.py 校验
|
||||
3. 校验失败则构建修正提示，再次调用 LLM
|
||||
4. 重复直到校验通过或达到最大重试次数
|
||||
5. LLM 在输出中包含 can_submit 和 suggestion 字段，用于判断信息完整性
|
||||
|
||||
状态机：
|
||||
idle -> extracting -> awaiting_supplement -> (回到extracting)
|
||||
|
|
||||
(完整) -> ready -> submitting -> done
|
||||
|
|
||||
(用户强制) -> ready
|
||||
原有导入路径保持可用。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import asdict, dataclass, field
|
||||
from enum import StrEnum
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
from ..doc.llm_extractor import (
|
||||
CACHE_DIR_NAME,
|
||||
build_extraction_user_message,
|
||||
llm_query_text,
|
||||
load_cache,
|
||||
load_match_result,
|
||||
parse_json_response,
|
||||
# 从新模块重新导出所有符号，保持向后兼容
|
||||
from .coordinator import (
|
||||
add_supplement,
|
||||
force_submit,
|
||||
process_user_text_supplement,
|
||||
run_agent_round,
|
||||
)
|
||||
from ..doc.prompt import (
|
||||
build_normal_info_system_prompt,
|
||||
build_travel_info_system_prompt,
|
||||
from .events import emit_agent_event as _emit_agent_event
|
||||
from .session import (
|
||||
AgentSession,
|
||||
AgentState,
|
||||
load_agent_state,
|
||||
save_agent_state,
|
||||
)
|
||||
from ..doc.validator import validate_extracted_info
|
||||
|
||||
log = get_logger("agent")
|
||||
# 为旧代码提供兼容的私有函数
|
||||
_emit_agent_event = _emit_agent_event
|
||||
|
||||
# 规则校验-修正循环的最大重试次数
|
||||
# 常量保持不变
|
||||
MAX_VALIDATION_RETRIES = 3
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 状态枚举
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
class AgentState(StrEnum):
|
||||
IDLE = "idle"
|
||||
EXTRACTING = "extracting"
|
||||
AWAITING_SUPPLEMENT = "awaiting_supplement"
|
||||
READY = "ready"
|
||||
SUBMITTING = "submitting"
|
||||
DONE = "done"
|
||||
ERROR = "error"
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Agent 会话数据模型
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class AgentSession:
|
||||
"""Agent 会话状态"""
|
||||
|
||||
session_id: str
|
||||
state: AgentState = AgentState.IDLE
|
||||
rounds: int = 0
|
||||
max_rounds: int = 5
|
||||
invoice_type: str = "travel" # "travel" 或 "normal"
|
||||
extracted_info: dict[str, Any] = field(default_factory=dict)
|
||||
validation_reports: list[dict[str, Any]] = field(default_factory=list)
|
||||
user_supplements: list[str] = field(default_factory=list)
|
||||
error_message: str = ""
|
||||
|
||||
def to_dict(self) -> dict[str, Any]:
|
||||
return asdict(self)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: dict[str, Any]) -> AgentSession:
|
||||
# 兼容旧版本：state 可能是字符串
|
||||
if "state" in data and isinstance(data["state"], str):
|
||||
data["state"] = AgentState(data["state"])
|
||||
return cls(**data)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 持久化
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
AGENT_STATE_FILE = "agent_state.json"
|
||||
|
||||
|
||||
def save_agent_state(session_dir: Path, session: AgentSession) -> None:
|
||||
"""将 Agent 会话状态持久化到 session 目录。"""
|
||||
state_path = session_dir / AGENT_STATE_FILE
|
||||
tmp_path = session_dir / (AGENT_STATE_FILE + ".tmp")
|
||||
with open(tmp_path, "w", encoding="utf-8") as f:
|
||||
json.dump(session.to_dict(), f, ensure_ascii=False, indent=2)
|
||||
tmp_path.replace(state_path)
|
||||
|
||||
|
||||
def load_agent_state(session_dir: Path) -> AgentSession | None:
|
||||
"""从 session 目录加载 Agent 会话状态。"""
|
||||
state_path = session_dir / AGENT_STATE_FILE
|
||||
if not state_path.exists():
|
||||
return None
|
||||
try:
|
||||
with open(state_path, encoding="utf-8") as f:
|
||||
return AgentSession.from_dict(json.load(f))
|
||||
except Exception as e:
|
||||
log.warning("加载 Agent 状态失败: %s", e)
|
||||
return None
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# SSE 事件发射
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
AGENT_EVENT_LOG = "agent_events.log"
|
||||
|
||||
|
||||
def _emit_agent_event(session_dir: Path, event_type: str, **kwargs: Any) -> None:
|
||||
"""向 agent_events.log 追加一行 JSON 事件。"""
|
||||
event = {"type": event_type, **kwargs}
|
||||
try:
|
||||
event_path = session_dir / AGENT_EVENT_LOG
|
||||
with open(event_path, "a", encoding="utf-8") as f:
|
||||
f.write(json.dumps(event, ensure_ascii=False) + "\n")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 辅助：构建修正提示
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _build_correction_prompt(
|
||||
base_message: str,
|
||||
report: Any,
|
||||
) -> str:
|
||||
"""根据校验报告构建修正提示，追加到原始用户消息后。"""
|
||||
error_feedback = (
|
||||
f"\n\n=== 上一次输出的校验结果 ===\n"
|
||||
f"校验未通过，发现以下问题：\n"
|
||||
f"缺失字段 ({len(report.missing_fields)} 个)：{', '.join(report.missing_fields)}\n"
|
||||
)
|
||||
if report.missing_materials:
|
||||
error_feedback += f"可能需要补充的材料：{', '.join(report.missing_materials)}\n"
|
||||
if report.suggestion:
|
||||
error_feedback += f"建议：{report.suggestion}\n"
|
||||
error_feedback += (
|
||||
"\n请根据以上校验结果修正你的输出，确保所有必填字段都有值。"
|
||||
"如果某个字段确实没有数据，请给出合理的猜测值。"
|
||||
"再次返回完整的 JSON 结果。"
|
||||
)
|
||||
return base_message + error_feedback
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 核心协调逻辑
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _do_extraction_with_validation(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
previous_analysis: dict[str, Any] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Agent 调度的提取-校验-修正循环。
|
||||
|
||||
流程：
|
||||
1. 加载缓存数据和匹配结果
|
||||
2. 构建用户消息
|
||||
3. 调用 LLM 提取
|
||||
4. 调用 validator 校验
|
||||
5. 校验失败则构建修正提示，回到步骤 3
|
||||
6. 最多重试 MAX_VALIDATION_RETRIES 次
|
||||
|
||||
LLM 输出的 JSON 中额外包含 can_submit 和 suggestion 字段，
|
||||
用于判断信息是否完整可提交。
|
||||
|
||||
Args:
|
||||
session_dir: 会话目录。
|
||||
session: 当前 Agent 会话。
|
||||
previous_analysis: 上一轮 LLM 分析结果（可选，补充文件时传入作为历史上下文）。
|
||||
|
||||
Returns:
|
||||
校验通过的结构化数据（或达到重试上限后的最佳结果）。
|
||||
"""
|
||||
cache_map = load_cache(session_dir)
|
||||
match_result = load_match_result(session_dir)
|
||||
|
||||
# 选择系统提示词
|
||||
if session.invoice_type == "travel":
|
||||
system_prompt = build_travel_info_system_prompt()
|
||||
else:
|
||||
system_prompt = build_normal_info_system_prompt()
|
||||
|
||||
base_message = build_extraction_user_message(cache_map, match_result, previous_analysis=previous_analysis)
|
||||
current_message = base_message
|
||||
|
||||
for attempt in range(1, MAX_VALIDATION_RETRIES + 1):
|
||||
log.info(
|
||||
"LLM 提取第 %d/%d 次尝试 (%s)",
|
||||
attempt,
|
||||
MAX_VALIDATION_RETRIES,
|
||||
session.invoice_type,
|
||||
)
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_state_change",
|
||||
state=AgentState.EXTRACTING,
|
||||
round=session.rounds,
|
||||
attempt=attempt,
|
||||
message=f"正在分析文件... (第{attempt}次)",
|
||||
)
|
||||
|
||||
# Step 1: 调用 LLM 提取
|
||||
try:
|
||||
response = llm_query_text(
|
||||
system_prompt=system_prompt,
|
||||
text=current_message,
|
||||
reasoning_effort="low",
|
||||
source_dir=session_dir,
|
||||
)
|
||||
result = parse_json_response(response)
|
||||
except Exception as e:
|
||||
log.error("LLM 提取失败: %s", e)
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_error",
|
||||
message=f"LLM 提取失败: {e}",
|
||||
)
|
||||
raise
|
||||
|
||||
# Step 2: 调用 validator 校验
|
||||
report = validate_extracted_info(result, invoice_type=session.invoice_type)
|
||||
|
||||
if report.valid:
|
||||
log.info("规则校验通过 (第 %d 次尝试)", attempt)
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_state_change",
|
||||
state=AgentState.EXTRACTING,
|
||||
round=session.rounds,
|
||||
attempt=attempt,
|
||||
message=f"规则校验通过 (第{attempt}次)",
|
||||
)
|
||||
return result
|
||||
|
||||
# Step 3: 校验失败，构建修正提示
|
||||
log.warning(
|
||||
"规则校验未通过 (第 %d/%d 次): 缺失 %d 个字段 - %s",
|
||||
attempt,
|
||||
MAX_VALIDATION_RETRIES,
|
||||
len(report.missing_fields),
|
||||
report.missing_fields,
|
||||
)
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_state_change",
|
||||
state=AgentState.EXTRACTING,
|
||||
round=session.rounds,
|
||||
attempt=attempt,
|
||||
message=f"规则校验未通过，缺失 {len(report.missing_fields)} 个字段，正在请求 LLM 修正...",
|
||||
)
|
||||
current_message = _build_correction_prompt(current_message, report)
|
||||
|
||||
# 所有重试都失败，返回最后一次结果
|
||||
log.error(
|
||||
"LLM 提取经过 %d 次尝试仍未通过规则校验，返回最后一次结果 (置信度: %.0f%%)",
|
||||
MAX_VALIDATION_RETRIES,
|
||||
report.confidence * 100,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def run_agent_round(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
new_files: list[str] | None = None,
|
||||
) -> AgentSession:
|
||||
"""执行一轮 Agent 处理：提取-校验-修正循环。
|
||||
|
||||
Args:
|
||||
session_dir: 会话目录。
|
||||
session: 当前 Agent 会话。
|
||||
new_files: 新增的文件列表（可选，补充文件时传入）。
|
||||
|
||||
Returns:
|
||||
更新后的 Agent 会话。
|
||||
|
||||
注意：
|
||||
- 信息提取优先从 .invoice_cache 缓存读取，避免重复调用 LLM。
|
||||
- Agent 调度提取-校验-修正循环：LLM 提取 -> validator 校验 -> 失败则反馈修正。
|
||||
- 若缓存缺失则执行提取后立即写回缓存（travel_info.json / normal_info.json）。
|
||||
- 补充文件时（new_files 非空），加载上一轮分析结果作为历史上下文，强制重新分析。
|
||||
"""
|
||||
# 终态保护：会话已提交或已完成时不再重复处理
|
||||
if session.state in (AgentState.DONE, AgentState.SUBMITTING, AgentState.READY):
|
||||
log.info("Agent 会话已处于终态 (%s)，跳过重复处理", session.state.value)
|
||||
return session
|
||||
|
||||
if session.rounds >= session.max_rounds:
|
||||
session.state = AgentState.ERROR
|
||||
session.error_message = f"已达到最大轮次 ({session.max_rounds})，请检查信息或强制提交"
|
||||
log.warning("Agent 达到最大轮次限制")
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_max_rounds",
|
||||
message=session.error_message,
|
||||
)
|
||||
return session
|
||||
|
||||
session.rounds += 1
|
||||
log.info("开始第 %d 轮 Agent 处理", session.rounds)
|
||||
|
||||
# ---- Step 1: 信息提取（Agent 调度校验-修正循环） ----
|
||||
session.state = AgentState.EXTRACTING
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_state_change",
|
||||
state=session.state,
|
||||
round=session.rounds,
|
||||
message="正在分析文件...",
|
||||
)
|
||||
|
||||
# 判断是否为补充文件场景：有新文件传入时，加载上一轮分析结果作为上下文
|
||||
cache_map = load_cache(session_dir)
|
||||
is_supplement = bool(new_files)
|
||||
previous_analysis = None
|
||||
if is_supplement:
|
||||
info_key = "travel_info" if session.invoice_type == "travel" else "normal_info"
|
||||
previous_analysis = cache_map.get(info_key)
|
||||
if previous_analysis:
|
||||
log.info("检测到补充文件，加载上一轮分析结果作为历史上下文")
|
||||
|
||||
try:
|
||||
if session.invoice_type == "travel":
|
||||
should_reanalyze = not cache_map.get("travel_info") or is_supplement
|
||||
if should_reanalyze:
|
||||
session.extracted_info = _do_extraction_with_validation(
|
||||
session_dir, session, previous_analysis=previous_analysis
|
||||
)
|
||||
# 提取后立即写入缓存，后续步骤依赖此数据
|
||||
cache_dir = session_dir / CACHE_DIR_NAME
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
with open(cache_dir / "travel_info.json", "w", encoding="utf-8") as f:
|
||||
json.dump(session.extracted_info, f, ensure_ascii=False, indent=2)
|
||||
else:
|
||||
session.extracted_info = cache_map["travel_info"]
|
||||
else:
|
||||
should_reanalyze = not cache_map.get("normal_info") or is_supplement
|
||||
if should_reanalyze:
|
||||
session.extracted_info = _do_extraction_with_validation(
|
||||
session_dir, session, previous_analysis=previous_analysis
|
||||
)
|
||||
# 提取后立即写入缓存，后续步骤依赖此数据
|
||||
cache_dir = session_dir / CACHE_DIR_NAME
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
with open(cache_dir / "normal_info.json", "w", encoding="utf-8") as f:
|
||||
json.dump(session.extracted_info, f, ensure_ascii=False, indent=2)
|
||||
else:
|
||||
session.extracted_info = cache_map["normal_info"]
|
||||
|
||||
except Exception as e:
|
||||
session.state = AgentState.ERROR
|
||||
session.error_message = f"信息提取失败: {e}"
|
||||
log.error("Agent 信息提取失败: %s", e)
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_error",
|
||||
message=session.error_message,
|
||||
)
|
||||
return session
|
||||
|
||||
# ---- 判断结果（从 LLM 提取结果中读 can_submit） ----
|
||||
can_submit = session.extracted_info.get("can_submit", True)
|
||||
suggestion = session.extracted_info.get("suggestion", "")
|
||||
|
||||
if can_submit:
|
||||
session.state = AgentState.READY
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_ready",
|
||||
round=session.rounds,
|
||||
message="信息完整，可以提交",
|
||||
)
|
||||
log.info("Agent 校验通过，信息完整")
|
||||
else:
|
||||
session.state = AgentState.AWAITING_SUPPLEMENT
|
||||
|
||||
combined_suggestion = suggestion or "信息不完整，请补充材料"
|
||||
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_request_supplement",
|
||||
round=session.rounds,
|
||||
missing_fields=[],
|
||||
missing_materials=[],
|
||||
semantic_issues=[],
|
||||
suggestion=combined_suggestion,
|
||||
)
|
||||
log.info("Agent 请求补充: %s", combined_suggestion)
|
||||
|
||||
save_agent_state(session_dir, session)
|
||||
return session
|
||||
|
||||
|
||||
def force_submit(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
) -> AgentSession:
|
||||
"""用户强制提交，跳过校验。"""
|
||||
session.state = AgentState.READY
|
||||
log.info("用户强制提交，跳过校验")
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_force_submit",
|
||||
message="用户选择强制提交",
|
||||
)
|
||||
save_agent_state(session_dir, session)
|
||||
return session
|
||||
|
||||
|
||||
def add_supplement(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
filenames: list[str],
|
||||
) -> AgentSession:
|
||||
"""记录用户补充的文件。"""
|
||||
session.user_supplements.extend(filenames)
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_supplement_received",
|
||||
files=filenames,
|
||||
)
|
||||
log.info("收到用户补充文件: %s", filenames)
|
||||
save_agent_state(session_dir, session)
|
||||
return session
|
||||
|
||||
|
||||
def process_user_text_supplement(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
user_text: str,
|
||||
) -> AgentSession:
|
||||
"""处理用户通过文字补充的信息。
|
||||
|
||||
流程：
|
||||
1. LLM 分析用户文字，提取需要更新的字段
|
||||
2. 合并到已提取的信息中
|
||||
3. 保存到缓存
|
||||
4. 重新执行一轮 Agent 校验
|
||||
|
||||
Args:
|
||||
session_dir: 会话目录。
|
||||
session: 当前 Agent 会话。
|
||||
user_text: 用户输入的文字。
|
||||
|
||||
Returns:
|
||||
更新后的 Agent 会话。
|
||||
"""
|
||||
from ..doc.llm_extractor import (
|
||||
merge_supplement_into_info,
|
||||
process_user_supplement,
|
||||
)
|
||||
|
||||
log.info("收到用户文字补充: %s", user_text)
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_supplement_received",
|
||||
files=[user_text[:50]], # 简短显示
|
||||
)
|
||||
|
||||
# Step 1: LLM 分析用户文字
|
||||
supplement_result = process_user_supplement(
|
||||
user_text=user_text,
|
||||
extracted_info=session.extracted_info,
|
||||
invoice_type=session.invoice_type,
|
||||
source_dir=session_dir,
|
||||
)
|
||||
|
||||
updated_fields = supplement_result.get("updated_fields", {})
|
||||
unparsed = supplement_result.get("unparsed_info", "")
|
||||
|
||||
if updated_fields:
|
||||
# Step 2: 合并到已提取信息
|
||||
session.extracted_info = merge_supplement_into_info(
|
||||
session.extracted_info,
|
||||
updated_fields,
|
||||
)
|
||||
|
||||
# Step 3: 保存到缓存
|
||||
cache_dir = session_dir / CACHE_DIR_NAME
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
info_file = "travel_info.json" if session.invoice_type == "travel" else "normal_info.json"
|
||||
info_path = cache_dir / info_file
|
||||
with open(info_path, "w", encoding="utf-8") as f:
|
||||
json.dump(session.extracted_info, f, ensure_ascii=False, indent=2)
|
||||
log.info("已更新 %s", info_file)
|
||||
|
||||
# Step 4: 重新执行 Agent 校验
|
||||
session.state = AgentState.EXTRACTING
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_state_change",
|
||||
state=session.state,
|
||||
round=session.rounds,
|
||||
message="正在重新校验...",
|
||||
)
|
||||
session = run_agent_round(session_dir, session)
|
||||
else:
|
||||
# 没有可更新的字段
|
||||
msg = unparsed or "未识别到可更新的报销信息"
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_supplement_received",
|
||||
files=[msg],
|
||||
)
|
||||
log.info("用户补充未识别到有效信息: %s", msg)
|
||||
|
||||
return session
|
||||
__all__ = [
|
||||
"AgentSession",
|
||||
"AgentState",
|
||||
"save_agent_state",
|
||||
"load_agent_state",
|
||||
"run_agent_round",
|
||||
"force_submit",
|
||||
"add_supplement",
|
||||
"process_user_text_supplement",
|
||||
"MAX_VALIDATION_RETRIES",
|
||||
"AGENT_STATE_FILE",
|
||||
"AGENT_EVENT_LOG",
|
||||
]
|
||||
|
||||
101
src/agent/session.py
Normal file
101
src/agent/session.py
Normal file
@@ -0,0 +1,101 @@
|
||||
"""Agent 会话管理
|
||||
|
||||
负责会话状态的定义、序列化和持久化。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import asdict, dataclass, field
|
||||
from enum import StrEnum
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
|
||||
log = get_logger("agent.session")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 状态枚举
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
class AgentState(StrEnum):
|
||||
IDLE = "idle"
|
||||
EXTRACTING = "extracting"
|
||||
AWAITING_SUPPLEMENT = "awaiting_supplement"
|
||||
READY = "ready"
|
||||
SUBMITTING = "submitting"
|
||||
DONE = "done"
|
||||
ERROR = "error"
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Agent 会话数据模型
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class AgentSession:
|
||||
"""Agent 会话状态"""
|
||||
|
||||
session_id: str
|
||||
state: AgentState = AgentState.IDLE
|
||||
rounds: int = 0
|
||||
max_rounds: int = 5
|
||||
invoice_type: str = "travel" # "travel" 或 "normal"
|
||||
extracted_info: dict[str, Any] = field(default_factory=dict)
|
||||
validation_reports: list[dict[str, Any]] = field(default_factory=list)
|
||||
user_supplements: list[str] = field(default_factory=list)
|
||||
error_message: str = ""
|
||||
|
||||
def to_dict(self) -> dict[str, Any]:
|
||||
return asdict(self)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: dict[str, Any]) -> AgentSession:
|
||||
# 兼容旧版本：state 可能是字符串
|
||||
if "state" in data and isinstance(data["state"], str):
|
||||
data["state"] = AgentState(data["state"])
|
||||
return cls(**data)
|
||||
|
||||
def is_terminal(self) -> bool:
|
||||
"""判断会话是否处于终态（已提交或已完成）"""
|
||||
return self.state in (AgentState.DONE, AgentState.SUBMITTING, AgentState.READY)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 持久化
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
AGENT_STATE_FILE = "agent_state.json"
|
||||
|
||||
|
||||
def save_agent_state(session_dir: Path, session: AgentSession) -> None:
|
||||
"""将 Agent 会话状态持久化到 session 目录。"""
|
||||
state_path = session_dir / AGENT_STATE_FILE
|
||||
tmp_path = session_dir / (AGENT_STATE_FILE + ".tmp")
|
||||
with open(tmp_path, "w", encoding="utf-8") as f:
|
||||
json.dump(session.to_dict(), f, ensure_ascii=False, indent=2)
|
||||
tmp_path.replace(state_path)
|
||||
|
||||
|
||||
def load_agent_state(session_dir: Path) -> AgentSession | None:
|
||||
"""从 session 目录加载 Agent 会话状态。"""
|
||||
state_path = session_dir / AGENT_STATE_FILE
|
||||
if not state_path.exists():
|
||||
return None
|
||||
try:
|
||||
with open(state_path, encoding="utf-8") as f:
|
||||
return AgentSession.from_dict(json.load(f))
|
||||
except Exception as e:
|
||||
log.warning("加载 Agent 状态失败: %s", e)
|
||||
return None
|
||||
|
||||
|
||||
def create_agent_session(session_id: str, invoice_type: str = "travel") -> AgentSession:
|
||||
"""创建新的 Agent 会话。"""
|
||||
return AgentSession(
|
||||
session_id=session_id,
|
||||
invoice_type=invoice_type,
|
||||
)
|
||||
@@ -1,33 +0,0 @@
|
||||
---
|
||||
last_reviewed: 2026-06-12
|
||||
---
|
||||
|
||||
# bot — 浏览器自动化填报模块
|
||||
|
||||
使用 Playwright 操作财务报销系统，自动完成登录、填单、上传附件等操作。
|
||||
|
||||
## 模块清单
|
||||
|
||||
| 文件 | 说明 |
|
||||
|------|------|
|
||||
| `__init__.py` | 对外入口：`run_bot()` 和 `run_bot_web()`，负责类型判断和流程路由 |
|
||||
| `base.py` | `BaseBot` 基类：浏览器生命周期、登录、导航、截图、日期格式化 |
|
||||
| `travel.py` | 差旅报销填报流程：基本信息 → 差旅明细 → 支付方式 → 补助清单 → 附件上传 |
|
||||
| `normal.py` | 普通发票报销填报流程：基本信息 → 总明细 → 支付方式 → 附件上传 |
|
||||
|
||||
## 架构设计
|
||||
|
||||
```
|
||||
run_bot(config, travel_info, normal_info)
|
||||
├── 创建 BaseBot，启动浏览器，登录门户
|
||||
├── travel_info 存在 → travel.run(bot, travel_info)
|
||||
└── normal_info 存在 → normal.run(bot, normal_info)
|
||||
```
|
||||
|
||||
- **`BaseBot`** 只保留公共操作（launch、login、navigate、create_new_form、close、screenshot）
|
||||
- **差旅/普通流程** 作为独立函数接受 `bot: BaseBot` 参数，符合函数式编程偏好
|
||||
- **`__init__.py`** 仅做路由分发，不包含具体填报逻辑
|
||||
|
||||
## 变更历史
|
||||
|
||||
- **2026-06-12**：从 `bot.py` 单文件重构为 `bot/` 包，分离差旅和普通报销逻辑
|
||||
@@ -1,93 +0,0 @@
|
||||
"""
|
||||
浏览器自动化填报
|
||||
|
||||
使用 Playwright 操作财务报销系统，自动完成登录、填单、上传附件等操作。
|
||||
|
||||
对外接口:
|
||||
run_bot(config, travel_info, normal_info) 启动浏览器并执行填报流程
|
||||
run_bot_web(config, work_dir) Web 模式填报（从缓存加载信息）
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
from .base import BaseBot
|
||||
|
||||
log = get_logger("bot")
|
||||
|
||||
|
||||
def run_bot(
|
||||
config: dict[str, Any],
|
||||
headless: bool = False,
|
||||
work_dir: Path | None = None,
|
||||
travel_info: dict[str, Any] | None = None,
|
||||
normal_info: dict[str, Any] | None = None,
|
||||
) -> None:
|
||||
"""执行完整的浏览器填报流程，根据发票类型自动路由
|
||||
|
||||
Args:
|
||||
config: 配置字典。
|
||||
headless: 是否无头模式。
|
||||
work_dir: 工作目录。
|
||||
travel_info: 差旅信息（由 pipeline 层提前提取并传入，非差旅时传 None）。
|
||||
normal_info: 普通发票信息（由 pipeline 层提前提取并传入，非普通时传 None）。
|
||||
"""
|
||||
if not config["username"] or not config["password"]:
|
||||
raise ValueError("缺少用户名或密码")
|
||||
|
||||
if not work_dir:
|
||||
raise ValueError("缺少工作目录")
|
||||
|
||||
bot = BaseBot(config, headless=headless)
|
||||
bot.work_dir = work_dir
|
||||
|
||||
try:
|
||||
bot.launch()
|
||||
bot.login_portal()
|
||||
|
||||
if travel_info is not None:
|
||||
log.info("处理差旅发票...")
|
||||
bot.navigate_to_reimburse(page_key="travel_page")
|
||||
bot.create_new_form()
|
||||
from . import travel
|
||||
|
||||
travel.run(bot, travel_info)
|
||||
elif normal_info is not None:
|
||||
log.info("处理普通发票...")
|
||||
bot.navigate_to_reimburse(page_key="reimburse_page")
|
||||
bot.create_new_form()
|
||||
from . import normal
|
||||
|
||||
normal.run(bot, normal_info)
|
||||
else:
|
||||
raise ValueError("缺少差旅信息（travel_info）和普通发票信息（normal_info），无法继续填报")
|
||||
|
||||
except Exception as e:
|
||||
log.error(f"操作失败: {e}")
|
||||
try:
|
||||
bot._screenshot("error")
|
||||
except Exception:
|
||||
pass
|
||||
raise
|
||||
finally:
|
||||
bot.close()
|
||||
|
||||
|
||||
def run_bot_web(config: dict[str, Any], work_dir: Path) -> None:
|
||||
"""Web 模式填报 — headless，附件从指定目录读取
|
||||
|
||||
Web 端的信息提取由 app.py 的管道负责，此处从缓存加载。
|
||||
"""
|
||||
from ..doc.llm_extractor import load_cache
|
||||
|
||||
cache_map = load_cache(work_dir)
|
||||
travel_info = cache_map.get("travel_info")
|
||||
normal_info = cache_map.get("normal_info")
|
||||
run_bot(
|
||||
config,
|
||||
headless=True,
|
||||
work_dir=work_dir,
|
||||
travel_info=travel_info,
|
||||
normal_info=normal_info,
|
||||
)
|
||||
@@ -7,12 +7,39 @@
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
_CONFIG_PATH = Path(__file__).parent.parent.parent / "scripts" / "data" / "config.json"
|
||||
|
||||
# 会话级配置允许覆盖的用户相关字段白名单（含密码）
|
||||
SESSION_CONFIG_KEYS = frozenset(
|
||||
{
|
||||
"username",
|
||||
"password",
|
||||
"default_name",
|
||||
"default_card_no",
|
||||
"default_person_id",
|
||||
"consumable_storage",
|
||||
}
|
||||
)
|
||||
|
||||
def load_config() -> dict[str, str | Path]:
|
||||
"""加载并合并配置，缺失字段使用默认值"""
|
||||
# 前端安全的配置字段白名单（不含密码）
|
||||
SAFE_CONFIG_KEYS = frozenset(
|
||||
{
|
||||
"username",
|
||||
"default_name",
|
||||
"default_card_no",
|
||||
"default_person_id",
|
||||
"consumable_storage",
|
||||
}
|
||||
)
|
||||
|
||||
# 模块级配置缓存
|
||||
_config_cache: dict[str, str | Path] | None = None
|
||||
|
||||
|
||||
def _read_config() -> dict[str, str | Path]:
|
||||
"""读取并合并配置，缺失字段使用默认值"""
|
||||
raw = {}
|
||||
if _CONFIG_PATH.exists():
|
||||
with open(_CONFIG_PATH, encoding="utf-8") as f:
|
||||
@@ -36,6 +63,37 @@ def load_config() -> dict[str, str | Path]:
|
||||
}
|
||||
|
||||
|
||||
def load_config() -> dict[str, str | Path]:
|
||||
"""加载并合并配置，使用模块级缓存避免重复读取文件"""
|
||||
global _config_cache
|
||||
if _config_cache is None:
|
||||
_config_cache = _read_config()
|
||||
return dict(_config_cache)
|
||||
|
||||
|
||||
def clear_config_cache() -> None:
|
||||
"""清除配置缓存（测试或配置变更时调用）"""
|
||||
global _config_cache
|
||||
_config_cache = None
|
||||
|
||||
|
||||
def load_session_config(session_dir: Path) -> dict[str, Any]:
|
||||
"""加载会话配置，合并项目全局配置与会话级配置
|
||||
|
||||
仅允许覆盖用户相关字段（白名单），防止用户上传的 config.json
|
||||
覆盖 sso_login_url、portal_url 等系统级配置。
|
||||
"""
|
||||
config = load_config()
|
||||
cfg_path = session_dir / "config.json"
|
||||
if cfg_path.exists():
|
||||
with open(cfg_path, encoding="utf-8") as f:
|
||||
session_cfg = json.load(f)
|
||||
for key in SESSION_CONFIG_KEYS:
|
||||
if key in session_cfg:
|
||||
config[key] = session_cfg[key]
|
||||
return config
|
||||
|
||||
|
||||
def get_llm_config() -> dict[str, str]:
|
||||
"""加载 LLM 配置，优先从环境变量读取，缺失字段使用默认值"""
|
||||
return {
|
||||
|
||||
21
src/core/README.md
Normal file
21
src/core/README.md
Normal file
@@ -0,0 +1,21 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/core — 核心业务逻辑
|
||||
|
||||
项目的核心业务层，负责信息提取、金额匹配和信息校验。此层不依赖 Web 框架或浏览器自动化等基础设施。
|
||||
|
||||
## 子模块
|
||||
|
||||
| 目录 | 说明 |
|
||||
|------|------|
|
||||
| `extraction/` | 文档信息提取：PDF/图片 → LLM 多模态识别 → 结构化数据 |
|
||||
| `matching/` | 发票与支付记录按金额匹配（一对一 / 一对多） |
|
||||
| `validation/` | 声明式信息完整性校验，规则从 JSON 配置文件加载 |
|
||||
|
||||
## 设计原则
|
||||
|
||||
- **零外部依赖**：不依赖 Flask、Playwright 等框架
|
||||
- **接口契约**：每个子模块通过 `__init__.py` 导出稳定的对外接口
|
||||
- **错误传播**：明确的异常层次，便于上层统一处理
|
||||
4
src/core/__init__.py
Normal file
4
src/core/__init__.py
Normal file
@@ -0,0 +1,4 @@
|
||||
"""核心业务逻辑模块
|
||||
|
||||
提供信息提取、规则校验和发票匹配功能。
|
||||
"""
|
||||
29
src/core/extraction/README.md
Normal file
29
src/core/extraction/README.md
Normal file
@@ -0,0 +1,29 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/core/extraction — 信息提取
|
||||
|
||||
从 PDF 发票和图片中提取结构化数据，是系统数据流的起点。
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `extractor.py` | 编排入口：扫描目录 → 逐文件提取 → 分类（发票/支付记录/申请单）→ 金额匹配 |
|
||||
| `llm_extractor.py` | LLM 多模态提取核心：统一文档提取、差旅/普通信息提取、缓存管理、SSE 流式事件 |
|
||||
|
||||
## 对外接口
|
||||
|
||||
| 函数 | 说明 |
|
||||
|------|------|
|
||||
| `extract_invoices(directory)` | 统一提取入口，返回 `(payment_records, applications, groups)` |
|
||||
| `extract_document(file_path)` | 从单个图片/PDF 提取信息 |
|
||||
| `extract_travel_info(source_dir)` | 综合发票和匹配结果提取差旅信息 |
|
||||
| `extract_normal_info(source_dir)` | 提取普通发票报销信息 |
|
||||
| `load_cache(source_dir)` | 加载缓存的结构化数据 |
|
||||
| `llm_query_text(...)` | 纯文本 LLM 查询（供 Agent 调度使用） |
|
||||
|
||||
## 缓存机制
|
||||
|
||||
提取结果缓存在 `.invoice_cache/` 目录中，文件名与源文件同名（`发票1.pdf` → `.invoice_cache/发票1.json`），避免重复调用 LLM。
|
||||
42
src/core/extraction/__init__.py
Normal file
42
src/core/extraction/__init__.py
Normal file
@@ -0,0 +1,42 @@
|
||||
"""信息提取模块
|
||||
|
||||
提供发票/文档结构化提取、LLM 辅助提取等功能。
|
||||
"""
|
||||
|
||||
from .extractor import (
|
||||
EXTRACTION_PARALLEL_COUNT,
|
||||
FILE_EVENTS_LOG,
|
||||
SUPPORTED_EXTENSIONS,
|
||||
extract_invoices,
|
||||
)
|
||||
from .llm_extractor import (
|
||||
CACHE_DIR_NAME,
|
||||
build_extraction_user_message,
|
||||
extract_document,
|
||||
extract_normal_info,
|
||||
extract_travel_info,
|
||||
llm_query_text,
|
||||
load_cache,
|
||||
load_match_result,
|
||||
merge_supplement_into_info,
|
||||
parse_json_response,
|
||||
process_user_supplement,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"CACHE_DIR_NAME",
|
||||
"build_extraction_user_message",
|
||||
"extract_document",
|
||||
"extract_normal_info",
|
||||
"extract_travel_info",
|
||||
"load_cache",
|
||||
"load_match_result",
|
||||
"llm_query_text",
|
||||
"merge_supplement_into_info",
|
||||
"parse_json_response",
|
||||
"process_user_supplement",
|
||||
"EXTRACTION_PARALLEL_COUNT",
|
||||
"FILE_EVENTS_LOG",
|
||||
"SUPPORTED_EXTENSIONS",
|
||||
"extract_invoices",
|
||||
]
|
||||
@@ -20,20 +20,25 @@ SSE 文件进度事件:
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
from ..exceptions import ExtractionError
|
||||
from .invoice import CACHE_DIR_NAME, classify_invoice_batch
|
||||
from ... import get_logger
|
||||
from ...core.matching import match_invoices_to_cards
|
||||
from ...exceptions import ExtractionError
|
||||
from ...infra.documents.invoice import CACHE_DIR_NAME, classify_invoice_batch
|
||||
from .llm_extractor import extract_document
|
||||
from .matcher import match_invoices_to_cards
|
||||
|
||||
log = get_logger("extractor")
|
||||
|
||||
# SSE 文件进度事件日志文件名
|
||||
FILE_EVENTS_LOG = "file_events.log"
|
||||
|
||||
# 并行提取文件数，可通过环境变量 EXTRACTION_PARALLEL_COUNT 配置
|
||||
EXTRACTION_PARALLEL_COUNT = int(os.environ.get("EXTRACTION_PARALLEL_COUNT", "3"))
|
||||
|
||||
# 支持的文件扩展名
|
||||
SUPPORTED_EXTENSIONS = {".pdf", ".png", ".jpg", ".jpeg", ".bmp", ".webp"}
|
||||
|
||||
@@ -54,9 +59,6 @@ def _emit_file_event(source_dir: Path, file_name: str, status: str, **kwargs: An
|
||||
pass
|
||||
|
||||
|
||||
# 支持的文件扩展名
|
||||
|
||||
|
||||
def _get_cache_dir(source_dir: Path) -> Path:
|
||||
"""获取缓存目录路径"""
|
||||
cache_dir = source_dir / CACHE_DIR_NAME
|
||||
@@ -288,28 +290,39 @@ def extract_invoices(
|
||||
# 记录失败文件及其错误信息
|
||||
failed_files: list[tuple[str, str]] = []
|
||||
|
||||
for file_path in all_files:
|
||||
result, err = _extract_document(file_path, cache_dir, source_dir)
|
||||
max_workers = max(1, EXTRACTION_PARALLEL_COUNT)
|
||||
with ThreadPoolExecutor(max_workers=max_workers) as executor:
|
||||
future_to_file = {executor.submit(_extract_document, fp, cache_dir, source_dir): fp for fp in all_files}
|
||||
|
||||
if not result:
|
||||
if err:
|
||||
failed_files.append((file_path.name, err))
|
||||
log.warning(f"未能解析: {file_path.name}")
|
||||
continue
|
||||
for future in as_completed(future_to_file):
|
||||
file_path = future_to_file[future]
|
||||
try:
|
||||
result, err = future.result()
|
||||
except Exception as e:
|
||||
err_msg = str(e)
|
||||
failed_files.append((file_path.name, err_msg))
|
||||
log.warning(f"未能解析: {file_path.name} ({err_msg})")
|
||||
continue
|
||||
|
||||
inv_type = result.get("invoice_type", "")
|
||||
if not result:
|
||||
if err:
|
||||
failed_files.append((file_path.name, err))
|
||||
log.warning(f"未能解析: {file_path.name}")
|
||||
continue
|
||||
|
||||
if inv_type == "application":
|
||||
applications.append(result)
|
||||
log.info(f"[{inv_type}] 已解析: {file_path.name}")
|
||||
elif inv_type == "payment":
|
||||
all_cards.append(result)
|
||||
log.info(f"[{inv_type}] 已解析: {file_path.name}")
|
||||
elif result.get("invoice_number"):
|
||||
all_invoices.append(result)
|
||||
log.info(f"[{inv_type}] 已解析: {file_path.name}")
|
||||
else:
|
||||
log.warning(f"无法分类: {file_path.name} (invoice_type={inv_type})")
|
||||
inv_type = result.get("invoice_type", "")
|
||||
|
||||
if inv_type == "application":
|
||||
applications.append(result)
|
||||
log.info(f"[{inv_type}] 已解析: {file_path.name}")
|
||||
elif inv_type == "payment":
|
||||
all_cards.append(result)
|
||||
log.info(f"[{inv_type}] 已解析: {file_path.name}")
|
||||
elif result.get("invoice_number"):
|
||||
all_invoices.append(result)
|
||||
log.info(f"[{inv_type}] 已解析: {file_path.name}")
|
||||
else:
|
||||
log.warning(f"无法分类: {file_path.name} (invoice_type={inv_type})")
|
||||
|
||||
# 全部文件提取失败时抛出异常，携带原始错误信息
|
||||
if failed_files and not all_invoices and not all_cards and not applications:
|
||||
@@ -38,9 +38,9 @@ import json
|
||||
from pathlib import Path
|
||||
from typing import Any, cast
|
||||
|
||||
from .. import get_logger
|
||||
from .invoice import CACHE_DIR_NAME
|
||||
from .prompt import (
|
||||
from ... import get_logger
|
||||
from ...infra.documents.invoice import CACHE_DIR_NAME
|
||||
from ...infra.llm.prompt import (
|
||||
build_invoice_system_prompt,
|
||||
build_normal_info_system_prompt,
|
||||
build_supplement_system_prompt,
|
||||
@@ -91,7 +91,7 @@ def _create_llm() -> Any:
|
||||
log.error("缺少 llama-index-llms-openai-like，请执行: uv pip install llama-index-llms-openai-like")
|
||||
raise
|
||||
|
||||
from ..config import get_llm_config
|
||||
from ...config import get_llm_config
|
||||
|
||||
llm_config = get_llm_config()
|
||||
return OpenAILike(
|
||||
@@ -141,41 +141,25 @@ def _image_to_base64(image_path: Path) -> str:
|
||||
return base64.b64encode(f.read()).decode("utf-8")
|
||||
|
||||
|
||||
def llm_query_text(
|
||||
system_prompt: str,
|
||||
text: str,
|
||||
reasoning_effort: str = "none",
|
||||
source_dir: Path | None = None,
|
||||
def _stream_llm_response(
|
||||
llm: Any,
|
||||
messages: list[Any],
|
||||
source_dir: Path | None,
|
||||
reasoning_effort: str,
|
||||
log_label: str = "LLM",
|
||||
) -> str:
|
||||
"""发送纯文本请求到 LLM（供 Agent 调度使用）。
|
||||
"""流式调用 LLM 并写入 SSE 事件（供 llm_query_text 和 _llm_query_multimodal 共用）。
|
||||
|
||||
Args:
|
||||
system_prompt: 系统提示词。
|
||||
text: 用户文本。
|
||||
reasoning_effort: 推理努力级别。
|
||||
llm: LLM 实例。
|
||||
messages: 消息列表。
|
||||
source_dir: 会话目录（可选，传入时启用 SSE 流式事件写入）。
|
||||
reasoning_effort: 推理努力级别。
|
||||
log_label: 日志标签（用于区分"纯文本"和"多模态"）。
|
||||
|
||||
Returns:
|
||||
LLM 响应文本。
|
||||
"""
|
||||
from llama_index.core.base.llms.types import TextBlock
|
||||
from llama_index.core.llms import ChatMessage
|
||||
|
||||
from ..config import get_llm_config
|
||||
|
||||
messages = [
|
||||
ChatMessage(role="system", content=system_prompt),
|
||||
ChatMessage(role="user", blocks=[TextBlock(text=text)]),
|
||||
]
|
||||
|
||||
llm_config = get_llm_config()
|
||||
llm = _create_llm()
|
||||
log.info(
|
||||
"开始请求 LLM (model=%s, base=%s)",
|
||||
llm_config["model"],
|
||||
llm_config["api_base"],
|
||||
)
|
||||
|
||||
try:
|
||||
if source_dir:
|
||||
_emit_llm_stream(source_dir, "start", label="正在分析文件...")
|
||||
@@ -198,17 +182,55 @@ def llm_query_text(
|
||||
_emit_llm_stream(source_dir, "reasoning", text=thinking_delta)
|
||||
|
||||
text = "".join(parts)
|
||||
log.info("LLM 请求完成，响应总长度: %d 字符", len(text))
|
||||
log.info("%s请求完成，响应总长度: %d 字符", log_label, len(text))
|
||||
if source_dir:
|
||||
_emit_llm_stream(source_dir, "end", label="分析完成")
|
||||
return text
|
||||
except Exception as e:
|
||||
log.error("LLM 请求失败: %s", e)
|
||||
log.error("%s请求失败: %s", log_label, e)
|
||||
if source_dir:
|
||||
_emit_llm_stream(source_dir, "error", error=str(e))
|
||||
raise
|
||||
|
||||
|
||||
def llm_query_text(
|
||||
system_prompt: str,
|
||||
text: str,
|
||||
reasoning_effort: str = "none",
|
||||
source_dir: Path | None = None,
|
||||
) -> str:
|
||||
"""发送纯文本请求到 LLM（供 Agent 调度使用）。
|
||||
|
||||
Args:
|
||||
system_prompt: 系统提示词。
|
||||
text: 用户文本。
|
||||
reasoning_effort: 推理努力级别。
|
||||
source_dir: 会话目录（可选，传入时启用 SSE 流式事件写入）。
|
||||
|
||||
Returns:
|
||||
LLM 响应文本。
|
||||
"""
|
||||
from llama_index.core.base.llms.types import TextBlock
|
||||
from llama_index.core.llms import ChatMessage
|
||||
|
||||
from ...config import get_llm_config
|
||||
|
||||
messages = [
|
||||
ChatMessage(role="system", content=system_prompt),
|
||||
ChatMessage(role="user", blocks=[TextBlock(text=text)]),
|
||||
]
|
||||
|
||||
llm_config = get_llm_config()
|
||||
llm = _create_llm()
|
||||
log.info(
|
||||
"开始请求 LLM (model=%s, base=%s)",
|
||||
llm_config["model"],
|
||||
llm_config["api_base"],
|
||||
)
|
||||
|
||||
return _stream_llm_response(llm, messages, source_dir, reasoning_effort, log_label="LLM")
|
||||
|
||||
|
||||
def extract_document(file_path: Path) -> dict[str, Any]:
|
||||
"""统一文档提取入口：从任意图片/PDF 中提取结构化信息。
|
||||
|
||||
@@ -220,7 +242,7 @@ def extract_document(file_path: Path) -> dict[str, Any]:
|
||||
Returns:
|
||||
包含提取字段的字典。
|
||||
"""
|
||||
from .pdf import render_pdf_to_images
|
||||
from ...infra.documents.pdf import render_pdf_to_images
|
||||
|
||||
system_prompt = build_invoice_system_prompt()
|
||||
user_text = f"请分析以下财务文档并提取信息：\n\n文件名: {file_path.name}"
|
||||
@@ -269,7 +291,7 @@ def _llm_query_multimodal(
|
||||
from llama_index.core.base.llms.types import ImageBlock, TextBlock
|
||||
from llama_index.core.llms import ChatMessage
|
||||
|
||||
from ..config import get_llm_config
|
||||
from ...config import get_llm_config
|
||||
|
||||
if blocks is not None:
|
||||
final_blocks = blocks
|
||||
@@ -299,38 +321,7 @@ def _llm_query_multimodal(
|
||||
len(final_blocks),
|
||||
)
|
||||
|
||||
try:
|
||||
if source_dir:
|
||||
_emit_llm_stream(source_dir, "start", label="正在分析文件...")
|
||||
|
||||
parts = []
|
||||
for resp in llm.stream_chat(
|
||||
messages,
|
||||
temperature=0.1,
|
||||
extra_body={"reasoning_effort": reasoning_effort},
|
||||
):
|
||||
delta = resp.delta
|
||||
if delta:
|
||||
parts.append(delta)
|
||||
if source_dir:
|
||||
_emit_llm_stream(source_dir, "chunk", text=delta)
|
||||
|
||||
thinking = getattr(resp, "additional_kwargs", {}) or {}
|
||||
thinking_delta = thinking.get("thinking_delta", "")
|
||||
if thinking_delta and source_dir:
|
||||
_emit_llm_stream(source_dir, "reasoning", text=thinking_delta)
|
||||
|
||||
text = "".join(parts)
|
||||
log.info("LLM 多模态请求完成，响应总长度: %d 字符", len(text))
|
||||
log.info("LLM 多模态响应: %s", text)
|
||||
if source_dir:
|
||||
_emit_llm_stream(source_dir, "end", label="分析完成")
|
||||
return text
|
||||
except Exception as e:
|
||||
log.error("LLM 多模态请求失败: %s", e)
|
||||
if source_dir:
|
||||
_emit_llm_stream(source_dir, "error", error=str(e))
|
||||
raise
|
||||
return _stream_llm_response(llm, messages, source_dir, reasoning_effort, log_label="LLM多模态")
|
||||
|
||||
|
||||
def load_cache(source_dir: Path) -> dict[str, Any]:
|
||||
@@ -437,6 +428,48 @@ def build_extraction_user_message(
|
||||
return "\n".join(parts)
|
||||
|
||||
|
||||
def _extract_info(
|
||||
source_dir: Path | None,
|
||||
system_prompt: str,
|
||||
info_type: str,
|
||||
) -> dict[str, Any]:
|
||||
"""通用的信息提取函数：加载缓存、构建消息、调用 LLM 并解析 JSON。
|
||||
|
||||
extract_travel_info 和 extract_normal_info 的公共实现。
|
||||
|
||||
Args:
|
||||
source_dir: 源文件目录（必填，包含 .invoice_cache 子目录）。
|
||||
system_prompt: 系统提示词。
|
||||
info_type: 信息类型标签（"差旅" 或 "普通发票"），用于日志。
|
||||
|
||||
Returns:
|
||||
LLM 提取的结构化信息字典。
|
||||
"""
|
||||
if not source_dir:
|
||||
log.warning("未提供 source_dir，无法加载缓存数据")
|
||||
return {}
|
||||
|
||||
cache_map = load_cache(source_dir)
|
||||
match_result = load_match_result(source_dir)
|
||||
user_message = build_extraction_user_message(cache_map, match_result)
|
||||
|
||||
log.info("开始构建%s信息提取请求，缓存条目: %d, 匹配结果: %d", info_type, len(cache_map), len(match_result))
|
||||
|
||||
try:
|
||||
response = llm_query_text(
|
||||
system_prompt=system_prompt,
|
||||
text=user_message,
|
||||
reasoning_effort="low",
|
||||
source_dir=source_dir,
|
||||
)
|
||||
result = parse_json_response(response)
|
||||
log.info("LLM %s信息提取成功", info_type)
|
||||
return result
|
||||
except Exception as e:
|
||||
log.error("LLM %s信息提取失败: %s", info_type, e)
|
||||
raise
|
||||
|
||||
|
||||
def extract_travel_info(
|
||||
source_dir: Path | None = None,
|
||||
) -> dict[str, Any]:
|
||||
@@ -450,35 +483,7 @@ def extract_travel_info(
|
||||
Returns:
|
||||
包含出差事由、地点、交通工具、时间、住宿信息等字段的字典。
|
||||
"""
|
||||
if not source_dir:
|
||||
log.warning("未提供 source_dir，无法加载缓存数据")
|
||||
return {}
|
||||
|
||||
system_prompt = build_travel_info_system_prompt()
|
||||
cache_map = load_cache(source_dir)
|
||||
match_result = load_match_result(source_dir)
|
||||
user_message = build_extraction_user_message(cache_map, match_result)
|
||||
|
||||
log.info(f"user_message: {user_message}")
|
||||
|
||||
try:
|
||||
response = llm_query_text(
|
||||
system_prompt=system_prompt,
|
||||
text=user_message,
|
||||
reasoning_effort="low",
|
||||
source_dir=source_dir,
|
||||
)
|
||||
result = parse_json_response(response)
|
||||
log.info("LLM 差旅信息提取成功")
|
||||
return result
|
||||
except Exception as e:
|
||||
log.error("LLM 差旅信息提取失败: %s", e)
|
||||
raise
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 普通发票信息提取
|
||||
# ------------------------------------------------------------------
|
||||
return _extract_info(source_dir, build_travel_info_system_prompt(), "差旅")
|
||||
|
||||
|
||||
def extract_normal_info(
|
||||
@@ -494,30 +499,7 @@ def extract_normal_info(
|
||||
Returns:
|
||||
包含报销说明、发票总数、总金额、支付方式、附件清单等字段的字典。
|
||||
"""
|
||||
if not source_dir:
|
||||
log.warning("未提供 source_dir，无法加载缓存数据")
|
||||
return {}
|
||||
|
||||
system_prompt = build_normal_info_system_prompt()
|
||||
cache_map = load_cache(source_dir)
|
||||
match_result = load_match_result(source_dir)
|
||||
user_message = build_extraction_user_message(cache_map, match_result)
|
||||
|
||||
log.info(f"user_message: {user_message}")
|
||||
|
||||
try:
|
||||
response = llm_query_text(
|
||||
system_prompt=system_prompt,
|
||||
text=user_message,
|
||||
reasoning_effort="low",
|
||||
source_dir=source_dir,
|
||||
)
|
||||
result = parse_json_response(response)
|
||||
log.info("LLM 普通发票信息提取成功")
|
||||
return result
|
||||
except Exception as e:
|
||||
log.error("LLM 普通发票信息提取失败: %s", e)
|
||||
raise
|
||||
return _extract_info(source_dir, build_normal_info_system_prompt(), "普通发票")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
27
src/core/matching/README.md
Normal file
27
src/core/matching/README.md
Normal file
@@ -0,0 +1,27 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/core/matching — 金额匹配
|
||||
|
||||
将提取到的发票数据与支付记录（刷卡截图）按金额进行匹配。
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `matcher.py` | 匹配引擎：一对一匹配、一对多贪心匹配、未匹配发票处理 |
|
||||
|
||||
## 匹配策略
|
||||
|
||||
| 场景 | 策略 |
|
||||
|------|------|
|
||||
| 发票数 == 支付记录数 | 一对一匹配：按金额降序配对，相对容差内即匹配 |
|
||||
| 发票数 > 支付记录数 | 一对多匹配：贪心算法凑金额，相对容差 3% |
|
||||
| 文件名匹配 | 最高优先级：文件名（不含后缀）一致时直接匹配 |
|
||||
| 未匹配发票 | 单独列为一条支付记录，`remark` 标记为 `"unmatched"` |
|
||||
|
||||
## 业务约束
|
||||
|
||||
- 发票总金额 >= 支付总金额
|
||||
- 输出以支付记录为主键的结果列表
|
||||
8
src/core/matching/__init__.py
Normal file
8
src/core/matching/__init__.py
Normal file
@@ -0,0 +1,8 @@
|
||||
"""匹配模块
|
||||
|
||||
提供发票与支付记录的金额匹配功能。
|
||||
"""
|
||||
|
||||
from .matcher import match_invoices_to_cards
|
||||
|
||||
__all__ = ["match_invoices_to_cards"]
|
||||
@@ -49,7 +49,7 @@
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
from ... import get_logger
|
||||
|
||||
log = get_logger("matcher")
|
||||
|
||||
@@ -227,25 +227,38 @@ def _match_one_to_one(
|
||||
assigned: set[int],
|
||||
result: dict[int, list[int]],
|
||||
) -> None:
|
||||
"""一对一匹配：发票数等于刷卡数，按金额从大到小依次配对"""
|
||||
"""一对一匹配：发票数等于刷卡数，对每张刷卡记录寻找金额最接近的未分配发票"""
|
||||
for card_idx, card in enumerate(cards):
|
||||
if card_idx >= len(invoices):
|
||||
break
|
||||
inv = invoices[card_idx]
|
||||
diff = abs(inv["_amount"] - card["_amount"])
|
||||
card_tol = _relative_tolerance(card["_amount"], tolerance)
|
||||
if diff <= card_tol:
|
||||
assigned.add(card_idx)
|
||||
result[card_idx] = [card_idx]
|
||||
if card_idx in result:
|
||||
continue
|
||||
card_amount = card["_amount"]
|
||||
card_tol = _relative_tolerance(card_amount, tolerance)
|
||||
|
||||
# 在未分配的发票中找金额最接近的
|
||||
best_idx = -1
|
||||
best_diff = float("inf")
|
||||
for idx, inv in enumerate(invoices):
|
||||
if idx in assigned:
|
||||
continue
|
||||
diff = abs(inv["_amount"] - card_amount)
|
||||
if diff < best_diff:
|
||||
best_diff = diff
|
||||
best_idx = idx
|
||||
|
||||
if best_idx >= 0 and best_diff <= card_tol:
|
||||
inv = invoices[best_idx]
|
||||
assigned.add(best_idx)
|
||||
result[card_idx] = [best_idx]
|
||||
log.info(
|
||||
f"[一对一] {inv.get('invoice_number', 'unknown')} ¥{inv['_amount']:.2f} "
|
||||
f"↔ {card.get('_source_file', 'unknown')} ¥{card['_amount']:.2f}"
|
||||
f"↔ {card.get('_source_file', 'unknown')} ¥{card_amount:.2f}"
|
||||
)
|
||||
else:
|
||||
elif best_idx >= 0:
|
||||
inv = invoices[best_idx]
|
||||
log.warning(
|
||||
f"[一对一] 金额偏差超出容差: "
|
||||
f"{inv.get('invoice_number', 'unknown')} ¥{inv['_amount']:.2f} "
|
||||
f"vs ¥{card['_amount']:.2f} (差 ¥{diff:.2f}, 容差 ¥{card_tol:.2f})"
|
||||
f"vs ¥{card_amount:.2f} (差 ¥{best_diff:.2f}, 容差 ¥{card_tol:.2f})"
|
||||
)
|
||||
|
||||
|
||||
34
src/core/validation/README.md
Normal file
34
src/core/validation/README.md
Normal file
@@ -0,0 +1,34 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/core/validation — 信息校验
|
||||
|
||||
对 LLM 提取的报销信息进行声明式规则校验，判断是否满足填报要求。
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `validator.py` | 校验引擎：加载 JSON 规则配置 → 遍历字段/数组 → 输出校验报告 |
|
||||
|
||||
## 设计特点
|
||||
|
||||
- **规则与引擎分离**：校验规则存储在 `config/validation_rules.json`，引擎只负责执行
|
||||
- **统一路径定位**：使用 `path` 列表定位嵌套字段，如 `["basic_info", "travel_purpose"]`
|
||||
- **自定义校验**：支持 `custom_check` 函数（日期格式、正数检查等）
|
||||
- **数组元素校验**：支持 `min_items` 最小数量 + 每个元素的必填字段
|
||||
|
||||
## 校验规则类型
|
||||
|
||||
| 类型 | 用途 | 配置项 |
|
||||
|------|------|--------|
|
||||
| `fields` | 顶层单值字段 | `path`, `required`, `custom_check`, `check_empty` |
|
||||
| `arrays` | 数组字段 | `path`, `min_items`, `element_fields` |
|
||||
|
||||
## 对外接口
|
||||
|
||||
| 函数 | 说明 |
|
||||
|------|------|
|
||||
| `validate(info, invoice_type)` | 执行校验，返回 `ValidationReport` |
|
||||
| `get_missing_fields(report)` | 提取缺失字段列表 |
|
||||
28
src/core/validation/__init__.py
Normal file
28
src/core/validation/__init__.py
Normal file
@@ -0,0 +1,28 @@
|
||||
"""校验模块
|
||||
|
||||
提供报销信息的规则级校验功能。
|
||||
"""
|
||||
|
||||
from .validator import (
|
||||
ArrayRule,
|
||||
FieldRule,
|
||||
ValidationReport,
|
||||
ValidationRules,
|
||||
get_validation_rules,
|
||||
reload_validation_rules,
|
||||
validate_extracted_info,
|
||||
validate_normal_info,
|
||||
validate_travel_info,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"validate_extracted_info",
|
||||
"validate_travel_info",
|
||||
"validate_normal_info",
|
||||
"ValidationReport",
|
||||
"FieldRule",
|
||||
"ArrayRule",
|
||||
"ValidationRules",
|
||||
"get_validation_rules",
|
||||
"reload_validation_rules",
|
||||
]
|
||||
537
src/core/validation/validator.py
Normal file
537
src/core/validation/validator.py
Normal file
@@ -0,0 +1,537 @@
|
||||
"""信息完整性校验器
|
||||
|
||||
对 LLM 提取的报销信息进行规则级校验，判断是否满足填报要求。
|
||||
|
||||
校验规则从 JSON 配置文件加载，支持声明式配置。
|
||||
|
||||
设计理念：
|
||||
使用声明式规则配置，将校验规则与校验逻辑分离，提高可读性和可维护性。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any, TypedDict
|
||||
|
||||
from ... import get_logger
|
||||
|
||||
log = get_logger("validator")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 日期格式
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
DATE_PATTERN = re.compile(r"^\d{4}-\d{2}-\d{2}$")
|
||||
|
||||
|
||||
def _is_valid_date(value: str) -> bool:
|
||||
"""检查日期格式是否为 YYYY-MM-DD。"""
|
||||
return bool(DATE_PATTERN.match(value))
|
||||
|
||||
|
||||
def _is_positive_number(value: Any) -> bool:
|
||||
"""检查值是否为正数（整数或浮点数）。"""
|
||||
return isinstance(value, int | float) and value > 0
|
||||
|
||||
|
||||
def _is_positive_integer(value: Any) -> bool:
|
||||
"""检查值是否为正整数。"""
|
||||
return isinstance(value, int) and value > 0
|
||||
|
||||
|
||||
# 自定义校验函数注册表
|
||||
_CUSTOM_CHECKS: dict[str, Callable[[Any], bool]] = {
|
||||
"is_valid_date": _is_valid_date,
|
||||
"is_positive_number": _is_positive_number,
|
||||
"is_positive_integer": _is_positive_integer,
|
||||
}
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 规则定义
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
class FieldRule(TypedDict, total=False):
|
||||
"""字段校验规则（统一使用 path 定位）"""
|
||||
|
||||
path: list[str] # 字段路径（统一定位方式）
|
||||
required: bool = True # 是否必填（默认必填）
|
||||
check_empty: bool = True # 是否检查空字符串（默认检查）
|
||||
custom_check: str | Callable[[Any], bool] | None = None # 自定义校验函数（名称或函数）
|
||||
description: str = "" # 字段描述（用于生成友好提示）
|
||||
|
||||
|
||||
class ArrayRule(TypedDict, total=False):
|
||||
"""数组校验规则"""
|
||||
|
||||
path: list[str] # 数组路径
|
||||
min_items: int = 1 # 最小元素数量
|
||||
element_fields: list[str | FieldRule] = [] # 元素字段规则
|
||||
description: str = "" # 数组描述
|
||||
|
||||
|
||||
class ValidationRules(TypedDict):
|
||||
"""校验规则集合"""
|
||||
|
||||
fields: list[FieldRule] # 字段规则列表
|
||||
arrays: list[ArrayRule] # 数组规则列表
|
||||
|
||||
|
||||
class ValidationConfig(TypedDict):
|
||||
"""校验配置结构"""
|
||||
|
||||
version: str
|
||||
custom_checks: dict[str, str]
|
||||
travel: ValidationRules
|
||||
normal: ValidationRules
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 配置加载
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
_CONFIG_PATH = Path(__file__).parent.parent.parent / "config" / "validation_rules.json"
|
||||
_cached_rules: ValidationConfig | None = None
|
||||
|
||||
|
||||
def _load_validation_config() -> ValidationConfig:
|
||||
"""加载校验规则配置文件。"""
|
||||
global _cached_rules
|
||||
if _cached_rules is not None:
|
||||
return _cached_rules
|
||||
|
||||
if not _CONFIG_PATH.exists():
|
||||
log.warning("校验规则配置文件不存在: %s，使用内置默认规则", _CONFIG_PATH)
|
||||
return _load_default_rules()
|
||||
|
||||
try:
|
||||
with open(_CONFIG_PATH, encoding="utf-8") as f:
|
||||
config = json.load(f)
|
||||
_cached_rules = _resolve_custom_checks(config)
|
||||
log.info("校验规则配置加载成功")
|
||||
return _cached_rules
|
||||
except Exception as e:
|
||||
log.error("加载校验规则配置失败: %s，使用内置默认规则", e)
|
||||
return _load_default_rules()
|
||||
|
||||
|
||||
def _resolve_custom_checks(config: dict[str, Any]) -> ValidationConfig:
|
||||
"""解析配置中的自定义校验函数名称，替换为实际函数引用。"""
|
||||
|
||||
def resolve_rule(rule: dict[str, Any]) -> dict[str, Any]:
|
||||
if "custom_check" in rule and isinstance(rule["custom_check"], str):
|
||||
check_name = rule["custom_check"]
|
||||
if check_name in _CUSTOM_CHECKS:
|
||||
rule["custom_check"] = _CUSTOM_CHECKS[check_name]
|
||||
else:
|
||||
log.warning("未知的自定义校验函数: %s", check_name)
|
||||
rule["custom_check"] = None
|
||||
return rule
|
||||
|
||||
# 解析 travel 规则的 fields
|
||||
for field_rule in config.get("travel", {}).get("fields", []):
|
||||
resolve_rule(field_rule)
|
||||
# 解析 element_fields
|
||||
for array_rule in config.get("travel", {}).get("arrays", []):
|
||||
for elem_field in array_rule.get("element_fields", []):
|
||||
if isinstance(elem_field, dict):
|
||||
resolve_rule(elem_field)
|
||||
|
||||
# 解析 normal 规则的 fields
|
||||
for field_rule in config.get("normal", {}).get("fields", []):
|
||||
resolve_rule(field_rule)
|
||||
# 解析 element_fields
|
||||
for array_rule in config.get("normal", {}).get("arrays", []):
|
||||
for elem_field in array_rule.get("element_fields", []):
|
||||
if isinstance(elem_field, dict):
|
||||
resolve_rule(elem_field)
|
||||
|
||||
return config # type: ignore[return-value]
|
||||
|
||||
|
||||
def _load_default_rules() -> ValidationConfig:
|
||||
"""返回内置的默认校验规则（当配置文件不存在时使用）。"""
|
||||
return {
|
||||
"version": "1.0",
|
||||
"custom_checks": {},
|
||||
"travel": {
|
||||
"fields": [
|
||||
{"path": ["basic_info", "travel_purpose"], "description": "出差事由"},
|
||||
{"path": ["basic_info", "travel_location"], "description": "出差地点"},
|
||||
{"path": ["basic_info", "start_date"], "custom_check": _is_valid_date, "description": "出差开始日期"},
|
||||
{"path": ["basic_info", "end_date"], "custom_check": _is_valid_date, "description": "出差结束日期"},
|
||||
],
|
||||
"arrays": [
|
||||
{
|
||||
"path": ["reimbursement_details", "transport_fee"],
|
||||
"min_items": 1,
|
||||
"element_fields": [
|
||||
{"path": ["vehicle_type"], "description": "交通工具类型"},
|
||||
{"path": ["start_date"], "custom_check": _is_valid_date, "description": "出发日期"},
|
||||
{"path": ["end_date"], "custom_check": _is_valid_date, "description": "到达日期"},
|
||||
{"path": ["departure_place"], "description": "出发地"},
|
||||
{"path": ["arrival_place"], "description": "目的地"},
|
||||
{"path": ["amount"], "custom_check": _is_positive_number, "description": "金额"},
|
||||
{"path": ["bill_count"], "custom_check": _is_positive_integer, "description": "票据张数"},
|
||||
{"path": ["remark"], "check_empty": False, "description": "备注说明"},
|
||||
],
|
||||
"description": "交通费用明细",
|
||||
},
|
||||
{
|
||||
"path": ["payment_methods"],
|
||||
"min_items": 1,
|
||||
"element_fields": [
|
||||
{"path": ["card_date"], "custom_check": _is_valid_date, "description": "刷卡日期"},
|
||||
{"path": ["card_amount"], "custom_check": _is_positive_number, "description": "支付金额"},
|
||||
{"path": ["merchant"], "description": "商户名称"},
|
||||
{"path": ["remark"], "check_empty": False, "description": "备注"},
|
||||
],
|
||||
"description": "支付方式记录",
|
||||
},
|
||||
{
|
||||
"path": ["subsidy_list"],
|
||||
"min_items": 1,
|
||||
"element_fields": [
|
||||
{"path": ["person_id"], "description": "人员工号"},
|
||||
{"path": ["person_name"], "description": "人员姓名"},
|
||||
{"path": ["start_date"], "custom_check": _is_valid_date, "description": "补助开始日期"},
|
||||
{"path": ["end_date"], "custom_check": _is_valid_date, "description": "补助结束日期"},
|
||||
{"path": ["days"], "custom_check": _is_positive_integer, "description": "补助天数"},
|
||||
],
|
||||
"description": "补助清单",
|
||||
},
|
||||
{
|
||||
"path": ["attachments"],
|
||||
"min_items": 0,
|
||||
"element_fields": [
|
||||
{"path": ["filename"], "description": "文件名"},
|
||||
{"path": ["attachment_type"], "description": "附件类型"},
|
||||
],
|
||||
"description": "附件列表",
|
||||
},
|
||||
],
|
||||
},
|
||||
"normal": {
|
||||
"fields": [
|
||||
{"path": ["basic_info", "reimbursement_description"], "description": "报销事由"},
|
||||
{
|
||||
"path": ["reimbursement_details", "total_invoices"],
|
||||
"custom_check": _is_positive_integer,
|
||||
"description": "发票总数",
|
||||
},
|
||||
{
|
||||
"path": ["reimbursement_details", "total_amount"],
|
||||
"custom_check": _is_positive_number,
|
||||
"description": "总金额",
|
||||
},
|
||||
],
|
||||
"arrays": [
|
||||
{
|
||||
"path": ["payment_methods"],
|
||||
"min_items": 1,
|
||||
"element_fields": [
|
||||
{"path": ["card_date"], "custom_check": _is_valid_date, "description": "刷卡日期"},
|
||||
{"path": ["card_amount"], "custom_check": _is_positive_number, "description": "支付金额"},
|
||||
{"path": ["merchant"], "description": "商户名称"},
|
||||
{"path": ["remark"], "check_empty": False, "description": "备注"},
|
||||
],
|
||||
"description": "支付方式记录",
|
||||
},
|
||||
{
|
||||
"path": ["attachments"],
|
||||
"min_items": 0,
|
||||
"element_fields": [
|
||||
{"path": ["filename"], "description": "文件名"},
|
||||
{"path": ["attachment_type"], "description": "附件类型"},
|
||||
],
|
||||
"description": "附件列表",
|
||||
},
|
||||
],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def get_validation_rules(invoice_type: str) -> ValidationRules:
|
||||
"""获取指定发票类型的校验规则。
|
||||
|
||||
Args:
|
||||
invoice_type: 发票类型，'travel' 或 'normal'。
|
||||
|
||||
Returns:
|
||||
对应的校验规则。
|
||||
"""
|
||||
config = _load_validation_config()
|
||||
return config.get(invoice_type, config.get("travel", {})) # type: ignore[return-value]
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 数据模型
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class ValidationReport:
|
||||
"""校验结果报告"""
|
||||
|
||||
valid: bool
|
||||
missing_fields: list[str] = field(default_factory=list)
|
||||
missing_materials: list[str] = field(default_factory=list)
|
||||
confidence: float = 0.0
|
||||
suggestion: str = ""
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 通用校验引擎
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _check_field(
|
||||
data: dict[str, Any],
|
||||
rule: FieldRule,
|
||||
) -> tuple[bool, str]:
|
||||
"""根据字段规则检查字段。"""
|
||||
path = rule["path"]
|
||||
check_empty = rule.get("check_empty", True)
|
||||
custom_check = rule.get("custom_check")
|
||||
|
||||
current = data
|
||||
for key in path:
|
||||
if not isinstance(current, dict):
|
||||
return (False, ".".join(path))
|
||||
if key not in current:
|
||||
return (False, ".".join(path))
|
||||
current = current[key]
|
||||
|
||||
if check_empty and isinstance(current, str) and not current.strip():
|
||||
return (False, ".".join(path))
|
||||
|
||||
if custom_check is not None and not custom_check(current):
|
||||
return (False, ".".join(path))
|
||||
|
||||
return (True, ".".join(path))
|
||||
|
||||
|
||||
def _check_array(
|
||||
data: dict[str, Any],
|
||||
rule: ArrayRule,
|
||||
) -> tuple[list[str], int, int]:
|
||||
"""根据数组规则检查数组。
|
||||
|
||||
Returns:
|
||||
(缺失字段列表, 总检查数, 通过检查数)
|
||||
"""
|
||||
missing: list[str] = []
|
||||
path = rule["path"]
|
||||
min_items = rule.get("min_items", 1)
|
||||
element_fields = rule.get("element_fields", [])
|
||||
path_str = ".".join(path)
|
||||
|
||||
total_checks = 1 # 数组存在性和最小数量检查
|
||||
passed_checks = 0
|
||||
|
||||
# 遍历路径获取数组
|
||||
current = data
|
||||
for key in path:
|
||||
if not isinstance(current, dict) or key not in current:
|
||||
return ([path_str], total_checks, passed_checks)
|
||||
current = current[key]
|
||||
|
||||
# 检查数组是否满足最小数量要求
|
||||
if not isinstance(current, list) or len(current) < min_items:
|
||||
return ([path_str], total_checks, passed_checks)
|
||||
|
||||
passed_checks += 1 # 数组检查通过
|
||||
|
||||
# 检查数组元素的字段
|
||||
if element_fields:
|
||||
for i, item in enumerate(current):
|
||||
if not isinstance(item, dict):
|
||||
missing.append(f"{path_str}[{i}]")
|
||||
total_checks += len(element_fields)
|
||||
continue
|
||||
|
||||
for field_rule in element_fields:
|
||||
total_checks += 1
|
||||
# 支持两种格式：简单字符串格式 和 详细规则格式
|
||||
if isinstance(field_rule, str):
|
||||
field_rule_dict: FieldRule = {"path": [field_rule]}
|
||||
else:
|
||||
field_rule_dict = field_rule
|
||||
|
||||
# 复用 _check_field 函数检查元素字段
|
||||
ok, _ = _check_field(item, field_rule_dict)
|
||||
if ok:
|
||||
passed_checks += 1
|
||||
else:
|
||||
field_path_str = ".".join(field_rule_dict["path"])
|
||||
missing.append(f"{path_str}[{i}].{field_path_str}")
|
||||
|
||||
return missing, total_checks, passed_checks
|
||||
|
||||
|
||||
def _validate_with_rules(data: dict[str, Any], rules: ValidationRules) -> tuple[list[str], int, int]:
|
||||
"""使用规则配置进行校验。"""
|
||||
missing: list[str] = []
|
||||
total_checks = 0
|
||||
passed_checks = 0
|
||||
|
||||
# 校验字段规则
|
||||
for rule in rules.get("fields", []):
|
||||
total_checks += 1
|
||||
ok, field_path = _check_field(data, rule)
|
||||
if ok:
|
||||
passed_checks += 1
|
||||
else:
|
||||
missing.append(field_path)
|
||||
|
||||
# 校验数组规则
|
||||
for rule in rules.get("arrays", []):
|
||||
array_missing, array_total, array_passed = _check_array(data, rule)
|
||||
total_checks += array_total
|
||||
passed_checks += array_passed
|
||||
missing.extend(array_missing)
|
||||
|
||||
return missing, total_checks, passed_checks
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 校验入口
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def validate_travel_info(data: dict[str, Any]) -> ValidationReport:
|
||||
"""校验差旅报销信息的完整性。"""
|
||||
rules = get_validation_rules("travel")
|
||||
missing, total_checks, passed_checks = _validate_with_rules(data, rules)
|
||||
|
||||
confidence = passed_checks / total_checks if total_checks > 0 else 0.0
|
||||
missing_materials = _infer_missing_materials(missing, data)
|
||||
suggestion = _build_suggestion(missing, missing_materials)
|
||||
|
||||
return ValidationReport(
|
||||
valid=len(missing) == 0,
|
||||
missing_fields=missing,
|
||||
missing_materials=missing_materials,
|
||||
confidence=round(confidence, 2),
|
||||
suggestion=suggestion,
|
||||
)
|
||||
|
||||
|
||||
def validate_normal_info(data: dict[str, Any]) -> ValidationReport:
|
||||
"""校验普通报销信息的完整性。"""
|
||||
rules = get_validation_rules("normal")
|
||||
missing, total_checks, passed_checks = _validate_with_rules(data, rules)
|
||||
|
||||
confidence = passed_checks / total_checks if total_checks > 0 else 0.0
|
||||
missing_materials = _infer_missing_materials(missing, data)
|
||||
suggestion = _build_suggestion(missing, missing_materials)
|
||||
|
||||
return ValidationReport(
|
||||
valid=len(missing) == 0,
|
||||
missing_fields=missing,
|
||||
missing_materials=missing_materials,
|
||||
confidence=round(confidence, 2),
|
||||
suggestion=suggestion,
|
||||
)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 缺失材料推断
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _infer_missing_materials(
|
||||
missing_fields: list[str],
|
||||
data: dict[str, Any],
|
||||
) -> list[str]:
|
||||
"""根据缺失字段推断可能需要补充的材料类型。"""
|
||||
materials: list[str] = []
|
||||
field_set = set(missing_fields)
|
||||
|
||||
if any("start_date" in f or "end_date" in f for f in field_set):
|
||||
if "basic_info.start_date" in field_set or "basic_info.end_date" in field_set:
|
||||
materials.append("出差事前申请单")
|
||||
|
||||
if "basic_info.travel_purpose" in field_set:
|
||||
materials.append("出差事前申请单")
|
||||
|
||||
if "basic_info.travel_location" in field_set:
|
||||
materials.append("交通工具发票")
|
||||
|
||||
if "payment_methods" in field_set or any("payment_methods[" in f for f in field_set):
|
||||
materials.append("支付记录截图")
|
||||
|
||||
if any("transport_fee" in f for f in field_set):
|
||||
materials.append("交通工具发票")
|
||||
|
||||
if any("subsidy_list" in f for f in field_set):
|
||||
materials.append("出差事前申请单")
|
||||
|
||||
if "basic_info.reimbursement_description" in field_set:
|
||||
materials.append("发票或支付记录")
|
||||
|
||||
return list(dict.fromkeys(materials))
|
||||
|
||||
|
||||
def _build_suggestion(
|
||||
missing_fields: list[str],
|
||||
missing_materials: list[str],
|
||||
) -> str:
|
||||
"""生成用户友好的建议信息。"""
|
||||
if not missing_fields:
|
||||
return ""
|
||||
|
||||
if missing_materials:
|
||||
material_names = "、".join(missing_materials)
|
||||
return f"信息不完整，请补充上传：{material_names}"
|
||||
|
||||
return f"信息不完整，缺少 {len(missing_fields)} 个字段"
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 统一入口
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def validate_extracted_info(
|
||||
data: dict[str, Any],
|
||||
invoice_type: str = "travel",
|
||||
) -> ValidationReport:
|
||||
"""校验提取信息的完整性。
|
||||
|
||||
Args:
|
||||
data: LLM 提取的结构化信息。
|
||||
invoice_type: 发票类型，'travel' 或 'normal'。
|
||||
|
||||
Returns:
|
||||
校验报告。
|
||||
"""
|
||||
log.info("开始校验 %s 报销信息完整性", invoice_type)
|
||||
|
||||
if invoice_type == "travel":
|
||||
report = validate_travel_info(data)
|
||||
else:
|
||||
report = validate_normal_info(data)
|
||||
|
||||
status = "通过" if report.valid else "未通过"
|
||||
log.info(
|
||||
"校验结果: %s (置信度: %.0f%%, 缺失字段: %d)",
|
||||
status,
|
||||
report.confidence * 100,
|
||||
len(report.missing_fields),
|
||||
)
|
||||
|
||||
return report
|
||||
|
||||
|
||||
def reload_validation_rules() -> None:
|
||||
"""重新加载校验规则配置（用于运行时热更新）。"""
|
||||
global _cached_rules
|
||||
_cached_rules = None
|
||||
_load_validation_config()
|
||||
log.info("校验规则已重新加载")
|
||||
@@ -1,82 +0,0 @@
|
||||
---
|
||||
|
||||
## last_reviewed: 2026-06-12
|
||||
|
||||
# src/doc — 文档处理模块
|
||||
|
||||
负责发票信息提取、基于 LLM 的支付截图信息识别、差旅/普通报销信息提取、以及将数据填入 Word 出库单模板。
|
||||
|
||||
## 模块清单
|
||||
|
||||
|
||||
| 文件 | 作用 |
|
||||
| ------------------------ | ---------------------------------------------------------------------------- |
|
||||
| `extractor.py` | 编排入口：串联 PDF 读取 → LLM 提取 → 支付截图匹配 → 分类 |
|
||||
| `pdf.py` | PDF 图片渲染（PyMuPDF） |
|
||||
| `llm_extractor.py` | 基于 LLM 的信息提取（发票文本 + 支付截图多模态 + 差旅/普通报销信息综合提取） |
|
||||
| `matcher.py` | 发票与支付截图按金额匹配，回填刷卡信息至发票记录 |
|
||||
| `invoice.py` | 发票类型常量、分类逻辑、CSV 读写工具 |
|
||||
| `fill_consumable_doc.py` | 将 CSV 数据填入易耗品出库单 Word 模板（pywin32 COM） |
|
||||
| `prompt.py` | LLM 提示词模板加载 |
|
||||
| `prompts/` | 提示词模板文件（`invoice_system.md`、`travel_info_system.md`、`normal_info_system.md`） |
|
||||
|
||||
|
||||
## 数据流
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["PDF 发票"] --> B["pdf.py<br/>PDF 图片渲染"]
|
||||
B --> C["llm_extractor.py<br/>发票文本提取"]
|
||||
C --> D["(发票列表)"]
|
||||
|
||||
E["支付截图"] --> F["llm_extractor.py<br/>多模态提取"]
|
||||
F --> G["(刷卡记录)"]
|
||||
|
||||
D --> H["matcher.py<br/>金额贪心匹配 / 相对容差 3%"]
|
||||
G --> H
|
||||
|
||||
H --> I["invoice.py<br/>分类 + CSV 回填"]
|
||||
I --> J["CSV<br/>刷卡日期/卡号/金额"]
|
||||
|
||||
J --> K["fill_consumable_doc<br/>易耗品出库单"]
|
||||
K --> L["易耗品出库单.doc"]
|
||||
|
||||
D --> M["llm_extractor.py<br/>差旅/普通信息综合提取"]
|
||||
H --> M
|
||||
|
||||
M --> N{"发票类型判断"}
|
||||
N -->|差旅发票| O["extract_travel_info()"]
|
||||
N -->|普通发票| P["extract_normal_info()"]
|
||||
|
||||
O --> Q["travel_info.json"]
|
||||
P --> R["normal_info.json"]
|
||||
```
|
||||
|
||||
|
||||
|
||||
## 依赖说明
|
||||
|
||||
- **PyMuPDF (pymupdf)** — PDF 图片渲染
|
||||
- **pywin32** — Word COM 自动化（仅 Windows）
|
||||
- **llama-index** — LLM 信息提取
|
||||
|
||||
## 注意事项
|
||||
|
||||
- `fill_consumable_doc.py` 依赖 Microsoft Word + COM，仅 Windows 可用
|
||||
- LLM 提取不会覆盖 CSV 中已有非空字段
|
||||
- 提示词模板位于 `prompts/` 目录，由 `prompt.py` 加载
|
||||
- LLM 提取失败时直接报错，无正则回退
|
||||
|
||||
## 变更说明（2026-06-16）
|
||||
|
||||
- `llm_extractor.py` 新增 SSE 流式事件写入：`_emit_llm_stream()` 函数将 LLM 思考过程的 `start`/`reasoning`/`chunk`/`end`/`error` 事件写入 `source_dir/llm_stream.log`，前端通过 SSE 实时展示 AI 思考过程与正式回答
|
||||
- `_llm_query_multimodal()` 新增 `source_dir` 参数，流式接收 `delta` 时同步写入 chunk 事件；同时提取 `additional_kwargs.thinking_delta` 写入 reasoning 事件
|
||||
- `extract_travel_info()` 和 `extract_normal_info()` 在 LLM 调用前后写入 `start`/`end` 事件，异常时写入 `error` 事件
|
||||
|
||||
## 变更说明（2026-06-11）
|
||||
|
||||
- `llm_extractor.py` 新增 `extract_normal_info()`：综合普通发票、支付记录和匹配结果，提取报销说明、发票总数、总金额、支付方式、附件清单，缓存为 `normal_info.json`
|
||||
- `llm_extractor.py` 的 `load_cache()` 扩展支持加载 `normal_info.json`
|
||||
- `prompt.py` 新增 `build_normal_info_system_prompt()`：加载 `normal_info_system.md`
|
||||
- `prompts/` 新增 `normal_info_system.md`：普通发票信息提取的系统提示词
|
||||
|
||||
@@ -1,4 +0,0 @@
|
||||
"""文档处理模块
|
||||
|
||||
包含发票提取、LLM 信息提取、出库单填写等功能。
|
||||
"""
|
||||
@@ -1,83 +0,0 @@
|
||||
#性格
|
||||
|
||||
你是财务报销信息完整性校验助手。你的任务是检查已提取的报销信息在语义上是否足够支撑完成报销系统填报。
|
||||
|
||||
**核心原则**：不仅要检查字段是否存在，还要判断信息在逻辑上是否自洽、是否足以完成填报。
|
||||
|
||||
---
|
||||
|
||||
## 输入数据说明
|
||||
|
||||
你会收到以下数据：
|
||||
1. **已提取的报销信息**：包含基本信息、报销明细、支付方式、附件清单等
|
||||
2. **发票缓存数据**：原始发票提取的结构化数据
|
||||
3. **匹配结果**：发票与支付记录的关联关系
|
||||
|
||||
---
|
||||
|
||||
## 校验维度
|
||||
|
||||
### 1. 逻辑一致性检查
|
||||
|
||||
- 出差日期范围是否合理（结束日期不早于开始日期）
|
||||
- 交通费的去程和返程日期是否在出差日期范围内
|
||||
- 酒店入住/退房日期是否与出差时间匹配
|
||||
- 支付金额总和是否与发票金额总和接近（允许小额差异）
|
||||
- 补助天数计算是否正确
|
||||
|
||||
### 2. 信息充分性检查
|
||||
|
||||
- 是否有足够的信息填写所有报销系统必填项
|
||||
- 出差事由是否明确具体（不能过于笼统）
|
||||
- 人员信息是否完整（姓名、工号）
|
||||
- 支付方式是否与支付记录对应
|
||||
- 每张发票应该都有对应的支付记录，如果没有提醒用户补充支付记录
|
||||
|
||||
### 3. 潜在问题识别
|
||||
|
||||
- 发票日期与出差日期差异过大
|
||||
- 同一笔支付对应多张发票但金额不匹配
|
||||
- 缺少关键附件
|
||||
- 人员信息不一致（如车票姓名与补助清单姓名不同）
|
||||
|
||||
---
|
||||
### 4. 不需询问的问题
|
||||
- 出差事前申请单和实际出差时间不一致这是正常的，因为规划是实际可以有差异
|
||||
|
||||
|
||||
## 输出格式
|
||||
|
||||
严格返回以下 JSON 格式：
|
||||
|
||||
```json
|
||||
{
|
||||
"valid": true或false,
|
||||
"confidence": 0.0到1.0之间的数字,
|
||||
"issues": ["问题描述1", "问题描述2"],
|
||||
"missing_info": ["缺失信息1", "缺失信息2"],
|
||||
"suggestion": "补充建议"
|
||||
}
|
||||
```
|
||||
|
||||
### 字段说明
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `valid` | boolean | 信息在语义上是否足够支撑填报 |
|
||||
| `confidence` | number | 置信度，1.0 表示完全确定，0.0 表示完全不确定 |
|
||||
| `issues` | array | 发现的逻辑问题列表，无问题则为空数组 |
|
||||
| `missing_info` | array | 语义上缺失的关键信息列表 |
|
||||
| `suggestion` | string | 补充建议，说明需要用户上传什么材料 |
|
||||
|
||||
### 判定标准
|
||||
|
||||
- **valid = true**：信息完整且逻辑自洽，可以直接填报
|
||||
- **valid = false**：存在信息缺失或逻辑矛盾，需要补充材料
|
||||
|
||||
---
|
||||
|
||||
## 最终输出要求
|
||||
|
||||
- 严格只输出 JSON 字符串
|
||||
- JSON 语法必须正确
|
||||
- 不要包含任何思考过程或解释文字
|
||||
@@ -1,429 +0,0 @@
|
||||
"""信息完整性校验器
|
||||
|
||||
对 LLM 提取的报销信息进行规则级校验，判断是否满足填报要求。
|
||||
|
||||
校验规则基于两个 schema：
|
||||
- 差旅报销：basic_info + reimbursement_details + payment_methods + subsidy_list + attachments
|
||||
- 普通报销：basic_info + reimbursement_details + payment_methods + attachments
|
||||
|
||||
校验结果包含缺失字段列表、缺失材料推断和建议信息。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
|
||||
log = get_logger("validator")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 日期格式
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
DATE_PATTERN = re.compile(r"^\d{4}-\d{2}-\d{2}$")
|
||||
|
||||
|
||||
def _is_valid_date(value: str) -> bool:
|
||||
return bool(DATE_PATTERN.match(value))
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 数据模型
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class ValidationReport:
|
||||
"""校验结果报告"""
|
||||
|
||||
valid: bool
|
||||
missing_fields: list[str] = field(default_factory=list)
|
||||
missing_materials: list[str] = field(default_factory=list)
|
||||
confidence: float = 0.0
|
||||
suggestion: str = ""
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 校验辅助
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _check_field(
|
||||
data: dict[str, Any],
|
||||
path: list[str],
|
||||
required: bool = True,
|
||||
check_empty: bool = True,
|
||||
custom_check: Any = None,
|
||||
) -> tuple[bool, str]:
|
||||
"""沿路径检查字段是否存在且有效。
|
||||
|
||||
Returns:
|
||||
(通过, 字段路径字符串)
|
||||
"""
|
||||
current = data
|
||||
for key in path:
|
||||
if not isinstance(current, dict):
|
||||
return (False, ".".join(path))
|
||||
if key not in current:
|
||||
return (False, ".".join(path))
|
||||
current = current[key]
|
||||
|
||||
if check_empty and isinstance(current, str) and not current.strip():
|
||||
return (False, ".".join(path))
|
||||
|
||||
if custom_check is not None and not custom_check(current):
|
||||
return (False, ".".join(path))
|
||||
|
||||
return (True, ".".join(path))
|
||||
|
||||
|
||||
def _check_array_min(
|
||||
data: dict[str, Any],
|
||||
path: list[str],
|
||||
min_items: int = 1,
|
||||
) -> tuple[bool, str]:
|
||||
"""检查数组字段是否存在且至少有 min_items 项。"""
|
||||
current = data
|
||||
for key in path:
|
||||
if not isinstance(current, dict):
|
||||
return (False, ".".join(path))
|
||||
if key not in current:
|
||||
return (False, ".".join(path))
|
||||
current = current[key]
|
||||
|
||||
if not isinstance(current, list) or len(current) < min_items:
|
||||
return (False, ".".join(path))
|
||||
|
||||
return (True, ".".join(path))
|
||||
|
||||
|
||||
def _check_array_element_fields(
|
||||
items: Any,
|
||||
required_fields: list[str],
|
||||
field_path_prefix: str,
|
||||
) -> list[str]:
|
||||
"""检查数组中每个元素是否包含必填字段。"""
|
||||
missing: list[str] = []
|
||||
if not isinstance(items, list):
|
||||
return [field_path_prefix]
|
||||
|
||||
for i, item in enumerate(items):
|
||||
if not isinstance(item, dict):
|
||||
missing.append(f"{field_path_prefix}[{i}]")
|
||||
continue
|
||||
for fld in required_fields:
|
||||
if fld not in item or (isinstance(item[fld], str) and not item[fld].strip()):
|
||||
missing.append(f"{field_path_prefix}[{i}].{fld}")
|
||||
|
||||
return missing
|
||||
|
||||
|
||||
def _is_positive_number(value: Any) -> bool:
|
||||
"""检查值是否为正数。"""
|
||||
return isinstance(value, int | float) and value > 0
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 差旅报销校验
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def validate_travel_info(data: dict[str, Any]) -> ValidationReport:
|
||||
"""校验差旅报销信息的完整性。"""
|
||||
missing: list[str] = []
|
||||
total_checks = 0
|
||||
passed_checks = 0
|
||||
|
||||
# basic_info 必填字段
|
||||
basic_fields = [
|
||||
"travel_purpose",
|
||||
"travel_location",
|
||||
"start_date",
|
||||
"end_date",
|
||||
]
|
||||
|
||||
for fld in basic_fields:
|
||||
total_checks += 1
|
||||
path = ["basic_info", fld]
|
||||
if fld in ("start_date", "end_date"):
|
||||
ok, _ = _check_field(data, path, custom_check=_is_valid_date)
|
||||
else:
|
||||
ok, _ = _check_field(data, path)
|
||||
if ok:
|
||||
passed_checks += 1
|
||||
else:
|
||||
missing.append(".".join(path))
|
||||
|
||||
# reimbursement_details.transport_fee (至少一条)
|
||||
total_checks += 1
|
||||
ok, _ = _check_array_min(data, ["reimbursement_details", "transport_fee"], min_items=1)
|
||||
if ok:
|
||||
passed_checks += 1
|
||||
else:
|
||||
missing.append("reimbursement_details.transport_fee")
|
||||
|
||||
# 检查 transport_fee 元素字段
|
||||
transport = data.get("reimbursement_details", {}).get("transport_fee", [])
|
||||
transport_fields = [
|
||||
"vehicle_type",
|
||||
"start_date",
|
||||
"end_date",
|
||||
"departure_place",
|
||||
"arrival_place",
|
||||
"amount",
|
||||
"bill_count",
|
||||
"remark",
|
||||
]
|
||||
missing.extend(
|
||||
_check_array_element_fields(
|
||||
transport,
|
||||
transport_fields,
|
||||
"reimbursement_details.transport_fee",
|
||||
)
|
||||
)
|
||||
|
||||
# payment_methods (至少一条)
|
||||
total_checks += 1
|
||||
ok, _ = _check_array_min(data, ["payment_methods"], min_items=1)
|
||||
if ok:
|
||||
passed_checks += 1
|
||||
else:
|
||||
missing.append("payment_methods")
|
||||
|
||||
# 检查 payment_methods 元素
|
||||
payments = data.get("payment_methods", [])
|
||||
payment_fields = ["card_date", "card_amount", "merchant", "remark"]
|
||||
missing.extend(
|
||||
_check_array_element_fields(
|
||||
payments,
|
||||
payment_fields,
|
||||
"payment_methods",
|
||||
)
|
||||
)
|
||||
|
||||
# subsidy_list (至少一条)
|
||||
total_checks += 1
|
||||
ok, _ = _check_array_min(data, ["subsidy_list"], min_items=1)
|
||||
if ok:
|
||||
passed_checks += 1
|
||||
else:
|
||||
missing.append("subsidy_list")
|
||||
|
||||
# 检查 subsidy_list 元素
|
||||
subsidies = data.get("subsidy_list", [])
|
||||
subsidy_fields = ["person_id", "person_name", "start_date", "end_date", "days"]
|
||||
missing.extend(
|
||||
_check_array_element_fields(
|
||||
subsidies,
|
||||
subsidy_fields,
|
||||
"subsidy_list",
|
||||
)
|
||||
)
|
||||
|
||||
# attachments (可选，有数据时检查)
|
||||
attachments = data.get("attachments", [])
|
||||
if attachments:
|
||||
attach_fields = ["filename", "attachment_type"]
|
||||
missing.extend(
|
||||
_check_array_element_fields(
|
||||
attachments,
|
||||
attach_fields,
|
||||
"attachments",
|
||||
)
|
||||
)
|
||||
|
||||
# 计算置信度
|
||||
confidence = passed_checks / total_checks if total_checks > 0 else 0.0
|
||||
|
||||
# 推断缺失材料
|
||||
missing_materials = _infer_missing_materials(missing, data)
|
||||
|
||||
# 生成建议
|
||||
suggestion = _build_suggestion(missing, missing_materials)
|
||||
|
||||
return ValidationReport(
|
||||
valid=len(missing) == 0,
|
||||
missing_fields=missing,
|
||||
missing_materials=missing_materials,
|
||||
confidence=round(confidence, 2),
|
||||
suggestion=suggestion,
|
||||
)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 普通报销校验
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def validate_normal_info(data: dict[str, Any]) -> ValidationReport:
|
||||
"""校验普通报销信息的完整性。"""
|
||||
missing: list[str] = []
|
||||
total_checks = 0
|
||||
passed_checks = 0
|
||||
|
||||
# basic_info.reimbursement_description
|
||||
total_checks += 1
|
||||
ok, _ = _check_field(data, ["basic_info", "reimbursement_description"])
|
||||
if ok:
|
||||
passed_checks += 1
|
||||
else:
|
||||
missing.append("basic_info.reimbursement_description")
|
||||
|
||||
# reimbursement_details.total_invoices
|
||||
total_checks += 1
|
||||
ok, _ = _check_field(
|
||||
data,
|
||||
["reimbursement_details", "total_invoices"],
|
||||
custom_check=lambda v: isinstance(v, int) and v > 0,
|
||||
)
|
||||
if ok:
|
||||
passed_checks += 1
|
||||
else:
|
||||
missing.append("reimbursement_details.total_invoices")
|
||||
|
||||
# reimbursement_details.total_amount
|
||||
total_checks += 1
|
||||
ok, _ = _check_field(
|
||||
data,
|
||||
["reimbursement_details", "total_amount"],
|
||||
custom_check=_is_positive_number,
|
||||
)
|
||||
if ok:
|
||||
passed_checks += 1
|
||||
else:
|
||||
missing.append("reimbursement_details.total_amount")
|
||||
|
||||
# payment_methods (至少一条)
|
||||
total_checks += 1
|
||||
ok, _ = _check_array_min(data, ["payment_methods"], min_items=1)
|
||||
if ok:
|
||||
passed_checks += 1
|
||||
else:
|
||||
missing.append("payment_methods")
|
||||
|
||||
payments = data.get("payment_methods", [])
|
||||
payment_fields = ["card_date", "card_amount", "merchant", "remark"]
|
||||
missing.extend(
|
||||
_check_array_element_fields(
|
||||
payments,
|
||||
payment_fields,
|
||||
"payment_methods",
|
||||
)
|
||||
)
|
||||
|
||||
# attachments
|
||||
attachments = data.get("attachments", [])
|
||||
if attachments:
|
||||
attach_fields = ["filename", "attachment_type"]
|
||||
missing.extend(
|
||||
_check_array_element_fields(
|
||||
attachments,
|
||||
attach_fields,
|
||||
"attachments",
|
||||
)
|
||||
)
|
||||
|
||||
confidence = passed_checks / total_checks if total_checks > 0 else 0.0
|
||||
missing_materials = _infer_missing_materials(missing, data)
|
||||
suggestion = _build_suggestion(missing, missing_materials)
|
||||
|
||||
return ValidationReport(
|
||||
valid=len(missing) == 0,
|
||||
missing_fields=missing,
|
||||
missing_materials=missing_materials,
|
||||
confidence=round(confidence, 2),
|
||||
suggestion=suggestion,
|
||||
)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 缺失材料推断
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _infer_missing_materials(
|
||||
missing_fields: list[str],
|
||||
data: dict[str, Any],
|
||||
) -> list[str]:
|
||||
"""根据缺失字段推断可能需要补充的材料类型。"""
|
||||
materials: list[str] = []
|
||||
field_set = set(missing_fields)
|
||||
|
||||
if any("start_date" in f or "end_date" in f for f in field_set):
|
||||
if "basic_info.start_date" in field_set or "basic_info.end_date" in field_set:
|
||||
materials.append("出差事前申请单")
|
||||
|
||||
if "basic_info.travel_purpose" in field_set:
|
||||
materials.append("出差事前申请单")
|
||||
|
||||
if "basic_info.travel_location" in field_set:
|
||||
materials.append("交通工具发票")
|
||||
|
||||
if "payment_methods" in field_set or any("payment_methods[" in f for f in field_set):
|
||||
materials.append("支付记录截图")
|
||||
|
||||
if any("transport_fee" in f for f in field_set):
|
||||
materials.append("交通工具发票")
|
||||
|
||||
if any("subsidy_list" in f for f in field_set):
|
||||
materials.append("出差事前申请单")
|
||||
|
||||
if "basic_info.reimbursement_description" in field_set:
|
||||
materials.append("发票或支付记录")
|
||||
|
||||
# 去重
|
||||
return list(dict.fromkeys(materials))
|
||||
|
||||
|
||||
def _build_suggestion(
|
||||
missing_fields: list[str],
|
||||
missing_materials: list[str],
|
||||
) -> str:
|
||||
"""生成用户友好的建议信息。"""
|
||||
if not missing_fields:
|
||||
return ""
|
||||
|
||||
if missing_materials:
|
||||
material_names = "、".join(missing_materials)
|
||||
return f"信息不完整，请补充上传：{material_names}"
|
||||
|
||||
return f"信息不完整，缺少 {len(missing_fields)} 个字段"
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 统一入口
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def validate_extracted_info(
|
||||
data: dict[str, Any],
|
||||
invoice_type: str = "travel",
|
||||
) -> ValidationReport:
|
||||
"""校验提取信息的完整性。
|
||||
|
||||
Args:
|
||||
data: LLM 提取的结构化信息。
|
||||
invoice_type: 发票类型，'travel' 或 'normal'。
|
||||
|
||||
Returns:
|
||||
校验报告。
|
||||
"""
|
||||
log.info("开始校验 %s 报销信息完整性", invoice_type)
|
||||
|
||||
if invoice_type == "travel":
|
||||
report = validate_travel_info(data)
|
||||
else:
|
||||
report = validate_normal_info(data)
|
||||
|
||||
status = "通过" if report.valid else "未通过"
|
||||
log.info(
|
||||
"校验结果: %s (置信度: %.0f%%, 缺失字段: %d)",
|
||||
status,
|
||||
report.confidence * 100,
|
||||
len(report.missing_fields),
|
||||
)
|
||||
|
||||
return report
|
||||
21
src/infra/README.md
Normal file
21
src/infra/README.md
Normal file
@@ -0,0 +1,21 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/infra — 基础设施层
|
||||
|
||||
提供浏览器自动化、文档处理和 LLM 接口等底层能力。此层不包含业务逻辑，只提供工具和平台能力。
|
||||
|
||||
## 子模块
|
||||
|
||||
| 目录 | 说明 |
|
||||
|------|------|
|
||||
| `browser/` | Playwright 驱动的财务系统自动填报 |
|
||||
| `documents/` | 发票数据模型、PDF 渲染、Word 出库单填写 |
|
||||
| `llm/` | LLM 提示词模板加载与管理 |
|
||||
|
||||
## 设计原则
|
||||
|
||||
- **无业务逻辑**：只提供工具能力，不包含业务流程判断
|
||||
- **可替换性**：每个子模块通过 `__init__.py` 导出接口，便于替换实现
|
||||
- **与 core 层解耦**：infra 不依赖 core，core 可通过接口调用 infra
|
||||
4
src/infra/__init__.py
Normal file
4
src/infra/__init__.py
Normal file
@@ -0,0 +1,4 @@
|
||||
"""基础设施模块
|
||||
|
||||
提供浏览器自动化、文档处理和 LLM 接口功能。
|
||||
"""
|
||||
44
src/infra/browser/README.md
Normal file
44
src/infra/browser/README.md
Normal file
@@ -0,0 +1,44 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/infra/browser — 浏览器自动化
|
||||
|
||||
使用 Playwright 操作财务报销系统，自动完成登录、填单、上传附件等操作。
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `base.py` | `BaseBot` 基类：浏览器生命周期、登录信息门户、导航到报销系统、创建新单据、截图 |
|
||||
| `travel.py` | 差旅报销填报流程：基本信息 → 差旅明细 → 支付方式 → 补助清单 → 附件上传 |
|
||||
| `normal.py` | 普通报销填报流程：基本信息 → 总明细 → 支付方式 → 附件上传 |
|
||||
| `__init__.py` | 入口函数：`run_bot()` / `run_bot_web()`，负责类型路由和流程调度 |
|
||||
|
||||
## 对外接口
|
||||
|
||||
| 函数 | 说明 |
|
||||
|------|------|
|
||||
| `run_bot(config, travel_info, normal_info)` | CLI 模式：根据传入信息判断差旅/普通报销 |
|
||||
| `run_bot_web(config, work_dir)` | Web 模式：从缓存加载信息后执行填报 |
|
||||
|
||||
## 填报流程
|
||||
|
||||
### 差旅报销（travel）
|
||||
1. 填写基本信息（事由、地点、日期、项目编号）
|
||||
2. 添加差旅明细（交通费用逐条录入）
|
||||
3. 填写支付方式（公务卡刷卡记录）
|
||||
4. 填写补助清单（按天计算交通补助 + 伙食补助）
|
||||
5. 上传附件（发票、申请单等）
|
||||
|
||||
### 普通报销（normal）
|
||||
1. 填写基本信息（报销事由、金额）
|
||||
2. 填写发票明细（总数、总金额）
|
||||
3. 填写支付方式
|
||||
4. 上传附件
|
||||
|
||||
## 注意事项
|
||||
|
||||
- 浏览器填报会启动 Chromium，请勿手动干扰自动化流程
|
||||
- 调试截图保存在 `images/` 目录
|
||||
- Web 模式以无头模式运行
|
||||
100
src/infra/browser/__init__.py
Normal file
100
src/infra/browser/__init__.py
Normal file
@@ -0,0 +1,100 @@
|
||||
"""浏览器自动化填报
|
||||
|
||||
使用 Playwright 操作财务报销系统，自动完成登录、填单、上传附件等操作。
|
||||
|
||||
对外接口:
|
||||
run_bot(config, travel_info, normal_info) 启动浏览器并执行填报流程
|
||||
run_bot_web(config, work_dir) Web 模式填报（从缓存加载信息）
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from ... import get_logger
|
||||
from .base import BaseBot
|
||||
|
||||
log = get_logger("bot")
|
||||
|
||||
|
||||
def run_bot(
|
||||
config: dict[str, Any],
|
||||
headless: bool = False,
|
||||
work_dir: Path | None = None,
|
||||
travel_info: dict[str, Any] | None = None,
|
||||
normal_info: dict[str, Any] | None = None,
|
||||
) -> None:
|
||||
"""启动浏览器并执行填报流程。
|
||||
|
||||
根据传入的报销信息判断执行差旅报销还是普通报销流程。
|
||||
|
||||
Args:
|
||||
config: 财务系统配置（含 URL、账号密码等）。
|
||||
headless: 是否无头模式。
|
||||
work_dir: 工作目录。
|
||||
travel_info: 差旅报销信息（可选）。
|
||||
normal_info: 普通报销信息（可选）。
|
||||
"""
|
||||
invoice_type = ""
|
||||
if travel_info and normal_info:
|
||||
invoice_type = "mixed"
|
||||
elif travel_info:
|
||||
invoice_type = "travel"
|
||||
elif normal_info:
|
||||
invoice_type = "normal"
|
||||
else:
|
||||
log.error("未提供任何报销信息")
|
||||
return
|
||||
|
||||
log.info(f"启动填报流程: {invoice_type}")
|
||||
|
||||
if invoice_type == "travel":
|
||||
from .travel import run as run_travel
|
||||
|
||||
bot = BaseBot(config, headless=headless)
|
||||
bot.work_dir = work_dir
|
||||
|
||||
try:
|
||||
bot.launch()
|
||||
bot.login_portal()
|
||||
bot.navigate_to_reimburse(page_key="travel_page")
|
||||
bot.create_new_form()
|
||||
run_travel(bot, travel_info)
|
||||
finally:
|
||||
bot.close()
|
||||
elif invoice_type == "normal":
|
||||
from .normal import run as run_normal
|
||||
|
||||
bot = BaseBot(config, headless=headless)
|
||||
bot.work_dir = work_dir
|
||||
|
||||
try:
|
||||
bot.launch()
|
||||
bot.login_portal()
|
||||
bot.navigate_to_reimburse(page_key="reimburse_page")
|
||||
bot.create_new_form()
|
||||
run_normal(bot, normal_info)
|
||||
finally:
|
||||
bot.close()
|
||||
else:
|
||||
log.warning("暂不支持混合报销流程")
|
||||
|
||||
|
||||
def run_bot_web(config: dict[str, Any], work_dir: str | Path) -> None:
|
||||
"""Web 模式填报（从缓存加载信息）。
|
||||
|
||||
根据 work_dir 下的 .invoice_cache 目录中已提取的信息，
|
||||
自动判断执行差旅报销还是普通报销流程。
|
||||
|
||||
Args:
|
||||
config: 财务系统配置（含 URL、账号密码等）。
|
||||
work_dir: 工作目录（包含 .invoice_cache 子目录）。
|
||||
"""
|
||||
from ...core.extraction import load_cache
|
||||
|
||||
work_dir = Path(work_dir)
|
||||
cache = load_cache(work_dir)
|
||||
|
||||
travel_info = cache.get("travel_info")
|
||||
normal_info = cache.get("normal_info")
|
||||
|
||||
run_bot(config, headless=True, work_dir=work_dir, travel_info=travel_info, normal_info=normal_info)
|
||||
@@ -1,5 +1,4 @@
|
||||
"""
|
||||
浏览器自动化填报 — 公共基类
|
||||
"""浏览器自动化填报 — 公共基类
|
||||
|
||||
提供浏览器生命周期管理、登录、导航、截图等公共操作。
|
||||
"""
|
||||
@@ -7,7 +6,7 @@
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
from ... import get_logger
|
||||
|
||||
log = get_logger("bot")
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
"""
|
||||
普通报销填报流程
|
||||
"""普通报销填报流程
|
||||
|
||||
负责普通发票报销的完整填报步骤：
|
||||
基本信息 → 总明细 → 支付方式 → 附件上传
|
||||
@@ -7,7 +6,7 @@
|
||||
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
from ... import get_logger
|
||||
from .base import BaseBot, format_date
|
||||
|
||||
log = get_logger("bot.normal")
|
||||
@@ -1,5 +1,4 @@
|
||||
"""
|
||||
差旅报销填报流程
|
||||
"""差旅报销填报流程
|
||||
|
||||
负责差旅报销的完整填报步骤：
|
||||
基本信息 → 差旅明细 → 支付方式 → 补助清单 → 附件上传
|
||||
@@ -7,7 +6,7 @@
|
||||
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
from ... import get_logger
|
||||
from .base import BaseBot, format_date
|
||||
|
||||
log = get_logger("bot.travel")
|
||||
@@ -32,8 +31,8 @@ def run(bot: BaseBot, travel_info: dict[str, Any]) -> None:
|
||||
fill_travel_info(bot, basic_info)
|
||||
|
||||
log.info("填写差旅报销明细...")
|
||||
travel_items = travel_info["reimbursement_details"]
|
||||
add_travel_items(bot, travel_items)
|
||||
details = travel_info["reimbursement_details"]
|
||||
add_travel_items(bot, details)
|
||||
|
||||
log.info("填写差旅报销支付方式...")
|
||||
payment_info = travel_info["payment_methods"]
|
||||
@@ -85,8 +84,14 @@ def fill_travel_info(bot: BaseBot, basic_info: dict[str, Any]) -> None:
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def add_travel_items(bot: BaseBot, travel_items: dict[str, Any]) -> None:
|
||||
"""录入差旅报销明细"""
|
||||
def add_travel_items(bot: BaseBot, details: dict[str, Any]) -> None:
|
||||
"""录入差旅报销明细
|
||||
|
||||
Args:
|
||||
bot: 已启动的 BaseBot 实例。
|
||||
details: 报销明细字典（travel_info["reimbursement_details"]），包含
|
||||
transport_fee、hotel_fee、conference_fee 等子字段。
|
||||
"""
|
||||
vehicle_map = {
|
||||
"火车": "01",
|
||||
"汽车": "02",
|
||||
@@ -99,7 +104,7 @@ def add_travel_items(bot: BaseBot, travel_items: dict[str, Any]) -> None:
|
||||
}
|
||||
|
||||
try:
|
||||
traffic_info = travel_items.get("transport_fee") or []
|
||||
traffic_info = details.get("transport_fee") or []
|
||||
for item in traffic_info:
|
||||
bot.page.click("#insertDetail", timeout=5000)
|
||||
bot._wait_for('text="增加明细"', timeout=5000)
|
||||
@@ -126,7 +131,7 @@ def add_travel_items(bot: BaseBot, travel_items: dict[str, Any]) -> None:
|
||||
bot.page.click("#detailAdd", timeout=3000)
|
||||
bot.page.wait_for_timeout(1000)
|
||||
|
||||
hotel_info = travel_items.get("hotel_fee") or []
|
||||
hotel_info = details.get("hotel_fee") or []
|
||||
for item in hotel_info:
|
||||
bot.page.click("#insertDetail", timeout=5000)
|
||||
bot._wait_for('text="增加明细"', timeout=5000)
|
||||
@@ -151,7 +156,7 @@ def add_travel_items(bot: BaseBot, travel_items: dict[str, Any]) -> None:
|
||||
bot.page.click("#detailAdd", timeout=3000)
|
||||
bot.page.wait_for_timeout(1000)
|
||||
|
||||
conference_info = travel_items.get("conference_fee") or []
|
||||
conference_info = details.get("conference_fee") or []
|
||||
for item in conference_info:
|
||||
bot.page.click("#insertDetail", timeout=5000)
|
||||
bot._wait_for('text="增加明细"', timeout=5000)
|
||||
@@ -225,6 +230,9 @@ def fill_travel_subsidy(bot: BaseBot, subsidy_info: list[dict[str, Any]]) -> Non
|
||||
bot._wait_for('text="增加补助清单"', timeout=5000)
|
||||
bot.page.click("#jzg3", timeout=5000)
|
||||
bot.page.wait_for_timeout(500)
|
||||
# 注意：此处使用直接索引而非 .get()，是故意的设计。
|
||||
# LLM 必须返回 person_name 和 person_id 字段，若缺失则说明数据质量有问题，
|
||||
# 应当立即报错终止流程，而非静默跳过。
|
||||
if info["person_name"] and info["person_name"] != "":
|
||||
bot.page.fill("#seacher", info["person_name"])
|
||||
elif info["person_id"] and info["person_id"] != "":
|
||||
@@ -245,6 +253,8 @@ def fill_travel_subsidy(bot: BaseBot, subsidy_info: list[dict[str, Any]]) -> Non
|
||||
bot.page.fill("#enddate1", format_date(info["end_date"]))
|
||||
bot.page.fill("#trafficdays1", str(info["days"]))
|
||||
bot.page.fill("#fooddays1", str(info["days"]))
|
||||
# 补助标准硬编码：交通补助 80 元/天，伙食补助 100 元/天。
|
||||
# 此为阜阳师范大学现行标准，如需适配其他单位，可改为从 config.json 读取。
|
||||
bot.page.fill("#trafficnorm1", str(80))
|
||||
bot.page.fill("#foodnorm1", str(100))
|
||||
trafficmoney = int(info["days"]) * 80
|
||||
@@ -283,7 +293,7 @@ def upload_travel_attachments(bot: BaseBot, attachment_info: list[dict[str, Any]
|
||||
bot.page.select_option("#fjlx", "1")
|
||||
else:
|
||||
bot.page.select_option("#fjlx", "2")
|
||||
bot.page.fill("#fpsmxx", info["attachment_desc"])
|
||||
bot.page.fill("#fpsmxx", info.get("attachment_desc", ""))
|
||||
if attachment_file and attachment_file.exists():
|
||||
bot.page.set_input_files("#file", str(attachment_file))
|
||||
bot.page.wait_for_timeout(1000)
|
||||
@@ -293,4 +303,5 @@ def upload_travel_attachments(bot: BaseBot, attachment_info: list[dict[str, Any]
|
||||
log.error(f"差旅附件上传失败: {e}")
|
||||
bot._screenshot("travel_attachment_error")
|
||||
raise
|
||||
|
||||
bot._screenshot("travel_attachment_done")
|
||||
36
src/infra/documents/README.md
Normal file
36
src/infra/documents/README.md
Normal file
@@ -0,0 +1,36 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/infra/documents — 文档处理
|
||||
|
||||
提供发票数据模型、PDF 渲染和 Word 出库单填写功能。
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `invoice.py` | 发票数据模型：类型常量、CSV 列定义、CSV/JSON 读写工具、发票分类 |
|
||||
| `pdf.py` | PDF 渲染为图片（PyMuPDF），供多模态 LLM 识别使用 |
|
||||
| `consumable.py` | 易耗品出库单填写：读取 CSV → 填入 Word 模板（pywin32 COM，仅 Windows） |
|
||||
|
||||
## 对外接口
|
||||
|
||||
| 函数 | 说明 |
|
||||
|------|------|
|
||||
| `load_csv(path)` | 读取支付记录 CSV |
|
||||
| `save_csv(payment_records, path)` | 保存支付记录 CSV |
|
||||
| `save_invoice_csv(payment_records, path)` | 保存发票级别 CSV |
|
||||
| `classify_invoice_batch(cache_map)` | 按类型批量分类发票 |
|
||||
| `render_pdf_to_images(pdf_path)` | PDF → 图片列表 |
|
||||
| `fill_consumable_doc(csv_path, doc_path)` | 将 CSV 数据填入 Word 模板 |
|
||||
|
||||
## 缓存目录
|
||||
|
||||
`.invoice_cache/` 是系统级缓存目录名常量，定义在 `invoice.py` 中，被提取和匹配模块统一引用。
|
||||
|
||||
## 易耗品出库单
|
||||
|
||||
- 需要 **Windows + Microsoft Word + pywin32**
|
||||
- 模板文件为项目根目录的 `易耗品、出库单.doc`
|
||||
- 填写规则：日期用当天日期，品名/规格/数量/单价从 CSV 解析，字体统一宋体五号
|
||||
32
src/infra/documents/__init__.py
Normal file
32
src/infra/documents/__init__.py
Normal file
@@ -0,0 +1,32 @@
|
||||
"""文档处理基础设施
|
||||
|
||||
提供发票数据模型、PDF 渲染、出库单填写等功能。
|
||||
"""
|
||||
|
||||
from .consumable import (
|
||||
CONSUMABLE_DOC_FILENAME,
|
||||
fill_consumable_doc,
|
||||
fill_consumable_from_template,
|
||||
)
|
||||
from .invoice import (
|
||||
CACHE_DIR_NAME,
|
||||
classify_invoice_batch,
|
||||
load_csv,
|
||||
load_invoice_csv,
|
||||
save_application_json,
|
||||
save_csv,
|
||||
save_invoice_csv,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"CACHE_DIR_NAME",
|
||||
"classify_invoice_batch",
|
||||
"load_csv",
|
||||
"load_invoice_csv",
|
||||
"save_csv",
|
||||
"save_invoice_csv",
|
||||
"save_application_json",
|
||||
"CONSUMABLE_DOC_FILENAME",
|
||||
"fill_consumable_doc",
|
||||
"fill_consumable_from_template",
|
||||
]
|
||||
@@ -1,5 +1,4 @@
|
||||
"""
|
||||
将 invoice_summary.csv 填入「易耗品、出库单.doc」表格。
|
||||
"""将 invoice_summary.csv 填入「易耗品、出库单.doc」表格。
|
||||
|
||||
仅写入表格数据单元格，保留原模板字体、边框与版式。
|
||||
"""
|
||||
@@ -13,9 +12,9 @@ from datetime import date
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
from ..config import load_config
|
||||
from ..doc.invoice import load_invoice_csv
|
||||
from ... import get_logger
|
||||
from ...config import load_config
|
||||
from .invoice import load_invoice_csv
|
||||
|
||||
log = get_logger("fill_consumable_doc")
|
||||
|
||||
@@ -154,6 +153,8 @@ def fill_consumable_doc(
|
||||
import win32com.client
|
||||
|
||||
pythoncom.CoInitialize()
|
||||
word = None
|
||||
doc = None
|
||||
try:
|
||||
word = win32com.client.Dispatch("Word.Application")
|
||||
word.Visible = False
|
||||
@@ -213,9 +214,11 @@ def fill_consumable_doc(
|
||||
|
||||
doc.Save()
|
||||
finally:
|
||||
doc.Close()
|
||||
word.Quit()
|
||||
if doc is not None:
|
||||
doc.Close()
|
||||
finally:
|
||||
if word is not None:
|
||||
word.Quit()
|
||||
pythoncom.CoUninitialize()
|
||||
|
||||
return doc_path
|
||||
@@ -13,7 +13,7 @@ import csv
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from .. import get_logger
|
||||
from ... import get_logger
|
||||
|
||||
log = get_logger("invoice")
|
||||
|
||||
@@ -11,7 +11,7 @@ from pathlib import Path
|
||||
|
||||
import fitz
|
||||
|
||||
from .. import get_logger
|
||||
from ... import get_logger
|
||||
|
||||
log = get_logger("pdf")
|
||||
|
||||
@@ -21,7 +21,7 @@ def render_pdf_to_images(filepath: Path, dpi: int = 300) -> list[str]:
|
||||
|
||||
Args:
|
||||
filepath: PDF 文件路径。
|
||||
dpi: 渲染分辨率（默认 150，平衡质量与速度）。
|
||||
dpi: 渲染分辨率（默认 300，平衡质量与速度）。
|
||||
|
||||
Returns:
|
||||
base64 编码的 JPEG 图片字符串列表（每页一个）。
|
||||
32
src/infra/llm/README.md
Normal file
32
src/infra/llm/README.md
Normal file
@@ -0,0 +1,32 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/infra/llm — LLM 提示词管理
|
||||
|
||||
管理 LLM 提示词模板的加载，供 `core/extraction/llm_extractor.py` 调用。
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `prompt.py` | 提示词加载：从 `prompts/` 目录读取 `.md` 模板文件 |
|
||||
| `prompts/` | 提示词模板目录（Markdown 格式） |
|
||||
|
||||
## 提示词模板
|
||||
|
||||
| 文件 | 用途 |
|
||||
|------|------|
|
||||
| `invoice_system.md` | 发票提取系统提示词 |
|
||||
| `travel_info_system.md` | 差旅信息提取系统提示词 |
|
||||
| `normal_info_system.md` | 普通发票信息提取系统提示词 |
|
||||
| `supplement_system.md` | 用户补充信息后的二次提取提示词 |
|
||||
| `validation_system.md` | 校验修正提示词 |
|
||||
|
||||
## 对外接口
|
||||
|
||||
| 函数 | 说明 |
|
||||
|------|------|
|
||||
| `build_invoice_system_prompt()` | 构建发票提取系统提示词 |
|
||||
| `build_travel_info_system_prompt()` | 构建差旅信息提取系统提示词 |
|
||||
| `build_normal_info_system_prompt()` | 构建普通发票信息提取系统提示词 |
|
||||
18
src/infra/llm/__init__.py
Normal file
18
src/infra/llm/__init__.py
Normal file
@@ -0,0 +1,18 @@
|
||||
"""LLM 接口模块
|
||||
|
||||
提供 LLM 提示词模板加载功能。
|
||||
"""
|
||||
|
||||
from .prompt import (
|
||||
build_invoice_system_prompt,
|
||||
build_normal_info_system_prompt,
|
||||
build_supplement_system_prompt,
|
||||
build_travel_info_system_prompt,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"build_invoice_system_prompt",
|
||||
"build_normal_info_system_prompt",
|
||||
"build_supplement_system_prompt",
|
||||
"build_travel_info_system_prompt",
|
||||
]
|
||||
@@ -1,7 +1,6 @@
|
||||
"""
|
||||
LLM 提示词模板
|
||||
"""LLM 提示词模板
|
||||
|
||||
从 src/prompts/ 目录加载 .md 文件作为提示词模板。
|
||||
从 infra/llm/prompts/ 目录加载 .md 文件作为提示词模板。
|
||||
"""
|
||||
|
||||
import os
|
||||
@@ -34,8 +33,3 @@ def build_normal_info_system_prompt() -> str:
|
||||
def build_supplement_system_prompt() -> str:
|
||||
"""构建用户补充信息分析系统提示词。"""
|
||||
return _load_prompt("supplement_system.md")
|
||||
|
||||
|
||||
def build_validation_system_prompt() -> str:
|
||||
"""构建语义校验系统提示词。"""
|
||||
return _load_prompt("validation_system.md")
|
||||
@@ -2,9 +2,9 @@
|
||||
last_reviewed: 2026-06-12
|
||||
---
|
||||
|
||||
# src/doc/prompts — LLM 提示词模板
|
||||
# src/infra/llm/prompts — LLM 提示词模板
|
||||
|
||||
存放 LLM 信息提取使用的系统提示词模板文件，由 `src/doc/prompt.py` 动态加载。
|
||||
存放 LLM 信息提取使用的系统提示词模板文件，由 `src/infra/llm/prompt.py` 动态加载。
|
||||
|
||||
## 模板清单
|
||||
|
||||
@@ -16,5 +16,5 @@ last_reviewed: 2026-06-12
|
||||
## 加载方式
|
||||
|
||||
```python
|
||||
from src.doc.prompt import build_invoice_system_prompt, build_travel_info_system_prompt
|
||||
from src.infra.llm import build_invoice_system_prompt, build_travel_info_system_prompt
|
||||
```
|
||||
120
src/pipeline.py
120
src/pipeline.py
@@ -14,98 +14,23 @@
|
||||
Bot 仅负责接收信息并填报，不再承担信息提取职责。
|
||||
"""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any, cast
|
||||
from typing import Any
|
||||
|
||||
from . import get_logger
|
||||
from .config import load_config
|
||||
from .doc.extractor import extract_invoices
|
||||
from .doc.invoice import (
|
||||
classify_invoice_batch,
|
||||
save_application_json,
|
||||
save_invoice_csv,
|
||||
)
|
||||
from .doc.invoice import (
|
||||
save_csv as save_payment_csv,
|
||||
)
|
||||
from .doc.llm_extractor import (
|
||||
CACHE_DIR_NAME,
|
||||
extract_normal_info,
|
||||
extract_travel_info,
|
||||
load_cache,
|
||||
from .core.extraction import extract_invoices
|
||||
from .pipeline_core import (
|
||||
extract_and_cache_normal_info,
|
||||
extract_and_cache_travel_info,
|
||||
extract_info_by_type,
|
||||
is_travel_invoice,
|
||||
process_invoices,
|
||||
)
|
||||
|
||||
log = get_logger("pipeline")
|
||||
|
||||
|
||||
def _classify_from_cache(cache_path: Path) -> dict[str, list[dict[str, Any]]]:
|
||||
"""从缓存目录读取发票数据并按类型分组"""
|
||||
from .doc.llm_extractor import load_cache
|
||||
|
||||
cache_map = load_cache(cache_path)
|
||||
invoices = [data for data in cache_map.values() if data.get("invoice_type") not in ("application", "payment")]
|
||||
return classify_invoice_batch(invoices)
|
||||
|
||||
|
||||
def _extract_travel_info_if_needed(groups: dict[str, list[dict[str, Any]]], cache_path: Path) -> dict[str, Any] | None:
|
||||
"""当存在差旅发票时，调用 LLM 提取差旅信息并缓存。
|
||||
|
||||
Returns:
|
||||
差旅信息字典，非差旅时返回 None。
|
||||
"""
|
||||
if not groups.get("travel"):
|
||||
return None
|
||||
|
||||
from .doc.llm_extractor import CACHE_DIR_NAME, load_cache
|
||||
|
||||
# 检查缓存是否已有
|
||||
cache_map = load_cache(cache_path)
|
||||
if cache_map.get("travel_info"):
|
||||
log.info("使用已有差旅信息缓存")
|
||||
return cast(dict[str, Any] | None, cache_map["travel_info"])
|
||||
|
||||
log.info("开始提取差旅信息...")
|
||||
travel_info = extract_travel_info(source_dir=cache_path)
|
||||
|
||||
# 保存到缓存
|
||||
cache_dir = cache_path / CACHE_DIR_NAME
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with open(cache_dir / "travel_info.json", "w", encoding="utf-8") as f:
|
||||
json.dump(travel_info, f, ensure_ascii=False, indent=2)
|
||||
log.info("差旅信息已保存到缓存")
|
||||
return travel_info
|
||||
|
||||
|
||||
def _extract_normal_info_if_needed(groups: dict[str, list[dict[str, Any]]], cache_path: Path) -> dict[str, Any] | None:
|
||||
"""当存在普通发票时，调用 LLM 提取普通报销信息并缓存。
|
||||
|
||||
Returns:
|
||||
普通报销信息字典，非普通时返回 None。
|
||||
"""
|
||||
if not groups.get("general"):
|
||||
return None
|
||||
|
||||
# 检查缓存是否已有
|
||||
cache_map = load_cache(cache_path)
|
||||
if cache_map.get("normal_info"):
|
||||
log.info("使用已有普通发票信息缓存")
|
||||
return cast(dict[str, Any] | None, cache_map["normal_info"])
|
||||
|
||||
log.info("开始提取普通发票信息...")
|
||||
normal_info = extract_normal_info(source_dir=cache_path)
|
||||
|
||||
# 保存到缓存
|
||||
cache_dir = cache_path / CACHE_DIR_NAME
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with open(cache_dir / "normal_info.json", "w", encoding="utf-8") as f:
|
||||
json.dump(normal_info, f, ensure_ascii=False, indent=2)
|
||||
log.info("普通发票信息已保存到缓存")
|
||||
return normal_info
|
||||
|
||||
|
||||
def run_pipeline(
|
||||
step: str = "all",
|
||||
username: str | None = None,
|
||||
@@ -147,20 +72,11 @@ def run_pipeline(
|
||||
log.error("未提取到任何发票数据")
|
||||
return 1
|
||||
|
||||
save_payment_csv(payment_records, cache_path / "payment_records.csv")
|
||||
save_invoice_csv(payment_records, cache_path / "invoice_summary.csv")
|
||||
# 使用公共函数处理发票数据
|
||||
process_invoices(payment_records, applications, groups, cache_path)
|
||||
|
||||
if applications:
|
||||
save_application_json(applications, cache_path / "travel_applications.json")
|
||||
|
||||
log.info(f"发票分类: 差旅 {len(groups['travel'])} 张, 普通 {len(groups['general'])} 张")
|
||||
|
||||
# 发票提取完成后立即判断类型
|
||||
is_travel = bool(groups["travel"]) and not bool(groups["general"])
|
||||
if is_travel:
|
||||
travel_info = _extract_travel_info_if_needed(groups, cache_path)
|
||||
else:
|
||||
normal_info = _extract_normal_info_if_needed(groups, cache_path)
|
||||
# 发票提取完成后立即判断类型并提取信息
|
||||
travel_info, normal_info = extract_info_by_type(groups, cache_path)
|
||||
|
||||
if step == "invoice":
|
||||
log.info("[1/2] 发票提取 完成")
|
||||
@@ -174,20 +90,22 @@ def run_pipeline(
|
||||
log.info("[2/2] 报销提交")
|
||||
log.info("=" * 60)
|
||||
|
||||
from .bot import run_bot
|
||||
from .infra.browser import run_bot
|
||||
|
||||
if groups is None:
|
||||
# 从缓存重新分类（仅 submit 阶段需要）
|
||||
from .pipeline_core import _classify_from_cache
|
||||
|
||||
groups = _classify_from_cache(cache_path)
|
||||
|
||||
is_travel = bool(groups["travel"]) and not bool(groups["general"])
|
||||
if is_travel:
|
||||
if is_travel_invoice(groups):
|
||||
if travel_info is None:
|
||||
travel_info = _extract_travel_info_if_needed(groups, cache_path)
|
||||
travel_info = extract_and_cache_travel_info(groups, cache_path)
|
||||
log.info("检测到纯差旅发票，使用差旅报销模式")
|
||||
run_bot(config, work_dir=cache_path, travel_info=travel_info)
|
||||
else:
|
||||
if normal_info is None:
|
||||
normal_info = _extract_normal_info_if_needed(groups, cache_path)
|
||||
normal_info = extract_and_cache_normal_info(groups, cache_path)
|
||||
log.info("检测到普通发票，使用普通报销模式")
|
||||
run_bot(config, work_dir=cache_path, normal_info=normal_info)
|
||||
|
||||
|
||||
220
src/pipeline_core.py
Normal file
220
src/pipeline_core.py
Normal file
@@ -0,0 +1,220 @@
|
||||
"""
|
||||
管道核心逻辑
|
||||
|
||||
抽取 pipeline.py（CLI 管道）和 pipeline_web.py（Web 管道）的公共数据流：
|
||||
发票分类判断 -> 差旅/普通信息提取 -> 缓存读写
|
||||
|
||||
两个入口分别传入不同的目录参数，复用此模块。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any, cast
|
||||
|
||||
from . import get_logger
|
||||
from .core.extraction import (
|
||||
CACHE_DIR_NAME,
|
||||
extract_normal_info,
|
||||
extract_travel_info,
|
||||
load_cache,
|
||||
)
|
||||
from .infra.documents import (
|
||||
classify_invoice_batch,
|
||||
save_application_json,
|
||||
save_invoice_csv,
|
||||
)
|
||||
from .infra.documents import save_csv as save_payment_csv
|
||||
|
||||
log = get_logger("pipeline_core")
|
||||
|
||||
|
||||
def is_travel_invoice(groups: dict[str, list[dict[str, Any]]]) -> bool:
|
||||
"""判断是否为纯差旅发票（有差旅发票且无普通发票）。
|
||||
|
||||
注意：系统仅支持「纯差旅」和「普通报销」两种模式。
|
||||
若同时存在差旅发票和普通发票（混合），则视为普通报销模式处理——差旅发票
|
||||
对应的费用仍会在普通报销中按项目填报。若需要严格区分，上游应在发票分类
|
||||
后报错提示用户分开提交。
|
||||
"""
|
||||
return bool(groups.get("travel")) and not bool(groups.get("general"))
|
||||
|
||||
|
||||
def save_cache_info(cache_path: Path, info_key: str, info: dict[str, Any]) -> None:
|
||||
"""将提取结果保存到缓存目录
|
||||
|
||||
Args:
|
||||
cache_path: 会话目录路径。
|
||||
info_key: 缓存键名（"travel_info" 或 "normal_info"）。
|
||||
info: 提取结果字典。
|
||||
"""
|
||||
cache_dir = cache_path / CACHE_DIR_NAME
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
with open(cache_dir / f"{info_key}.json", "w", encoding="utf-8") as f:
|
||||
json.dump(info, f, ensure_ascii=False, indent=2)
|
||||
log.info("%s 已保存到缓存", info_key)
|
||||
|
||||
|
||||
def extract_and_cache_travel_info(
|
||||
groups: dict[str, list[dict[str, Any]]],
|
||||
cache_path: Path,
|
||||
) -> dict[str, Any] | None:
|
||||
"""当存在差旅发票时，调用 LLM 提取差旅信息并缓存。
|
||||
|
||||
Returns:
|
||||
差旅信息字典，非差旅时返回 None。
|
||||
"""
|
||||
if not groups.get("travel"):
|
||||
return None
|
||||
|
||||
# 检查缓存是否已有
|
||||
cache_map = load_cache(cache_path)
|
||||
travel_info = cache_map.get("travel_info")
|
||||
if travel_info:
|
||||
log.info("使用已有差旅信息缓存")
|
||||
return travel_info # type: ignore[no-any-return]
|
||||
|
||||
log.info("开始提取差旅信息...")
|
||||
travel_info = extract_travel_info(source_dir=cache_path)
|
||||
save_cache_info(cache_path, "travel_info", travel_info)
|
||||
return travel_info
|
||||
|
||||
|
||||
def extract_and_cache_normal_info(
|
||||
groups: dict[str, list[dict[str, Any]]],
|
||||
cache_path: Path,
|
||||
) -> dict[str, Any] | None:
|
||||
"""当存在普通发票时，调用 LLM 提取普通报销信息并缓存。
|
||||
|
||||
Returns:
|
||||
普通报销信息字典，非普通时返回 None。
|
||||
"""
|
||||
if not groups.get("general"):
|
||||
return None
|
||||
|
||||
# 检查缓存是否已有
|
||||
cache_map = load_cache(cache_path)
|
||||
normal_info = cache_map.get("normal_info")
|
||||
if normal_info:
|
||||
log.info("使用已有普通发票信息缓存")
|
||||
return normal_info # type: ignore[no-any-return]
|
||||
|
||||
log.info("开始提取普通发票信息...")
|
||||
normal_info = extract_normal_info(source_dir=cache_path)
|
||||
save_cache_info(cache_path, "normal_info", normal_info)
|
||||
return normal_info
|
||||
|
||||
|
||||
def _classify_from_cache(cache_path: Path) -> dict[str, list[dict[str, Any]]]:
|
||||
"""从缓存目录读取发票数据并按类型分组
|
||||
|
||||
注意：load_cache 会加载所有 .invoice_cache/*.json，包括 travel_info 和 normal_info
|
||||
等提取结果缓存（它们没有 invoice_type 字段），需要显式过滤掉。
|
||||
"""
|
||||
cache_map = load_cache(cache_path)
|
||||
# 过滤掉非发票的缓存条目：travel_info、normal_info 等提取结果
|
||||
invoices = [
|
||||
data
|
||||
for key, data in cache_map.items()
|
||||
if key not in ("travel_info", "normal_info")
|
||||
and isinstance(data, dict)
|
||||
and data.get("invoice_type") not in ("application", "payment")
|
||||
]
|
||||
return classify_invoice_batch(invoices)
|
||||
|
||||
|
||||
def save_invoice_groups(session_dir: Path, groups: dict[str, list[dict[str, str]]]) -> None:
|
||||
"""保存发票分类结果到目录的 JSON 文件
|
||||
|
||||
保存完整发票分组数据（供 is_travel_invoice/extract_and_cache_* 使用），
|
||||
同时保留计数字段（供快速统计使用）。
|
||||
"""
|
||||
groups_path = session_dir / "invoice_groups.json"
|
||||
data = {
|
||||
"travel": groups.get("travel", []),
|
||||
"general": groups.get("general", []),
|
||||
"application": groups.get("application", []),
|
||||
"travel_count": len(groups.get("travel", [])),
|
||||
"general_count": len(groups.get("general", [])),
|
||||
"application_count": len(groups.get("application", [])),
|
||||
}
|
||||
with open(groups_path, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, ensure_ascii=False, indent=2)
|
||||
|
||||
|
||||
def load_invoice_groups(session_dir: Path) -> dict[str, Any] | None:
|
||||
"""从目录加载发票分类结果
|
||||
|
||||
返回包含完整发票分组数据和计数字段的字典。
|
||||
"""
|
||||
groups_path = session_dir / "invoice_groups.json"
|
||||
if not groups_path.exists():
|
||||
return None
|
||||
try:
|
||||
with open(groups_path, encoding="utf-8") as f:
|
||||
return cast(dict[str, Any] | None, json.load(f))
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def process_invoices(
|
||||
payment_records: list[dict[str, Any]],
|
||||
applications: list[dict[str, Any]],
|
||||
groups: dict[str, list[dict[str, Any]]],
|
||||
output_dir: Path,
|
||||
) -> dict[str, Any]:
|
||||
"""处理提取的发票数据：保存 CSV、申请单和分类结果
|
||||
|
||||
Args:
|
||||
payment_records: 支付记录列表。
|
||||
applications: 申请单列表。
|
||||
groups: 发票分类结果。
|
||||
output_dir: 输出目录。
|
||||
|
||||
Returns:
|
||||
包含发票统计信息的字典。
|
||||
"""
|
||||
# 保存 CSV：支付记录级别（供 bot/出库单使用）和发票级别（供人工参考）
|
||||
save_payment_csv(payment_records, output_dir / "payment_records.csv")
|
||||
save_invoice_csv(payment_records, output_dir / "invoice_summary.csv")
|
||||
|
||||
# 出差申请单单独保存
|
||||
if applications:
|
||||
save_application_json(applications, output_dir / "travel_applications.json")
|
||||
|
||||
# 保存分类结果（供后续步骤统一读取）
|
||||
save_invoice_groups(output_dir, groups)
|
||||
|
||||
# 统计发票总数
|
||||
invoice_count = sum(len(inv.get("_matched_invoices", [])) for inv in payment_records)
|
||||
|
||||
log.info(f"发票分类: 差旅 {len(groups['travel'])} 张, 普通 {len(groups['general'])} 张")
|
||||
|
||||
return {
|
||||
"invoice_count": invoice_count,
|
||||
"travel_count": len(groups["travel"]),
|
||||
"general_count": len(groups["general"]),
|
||||
"application_count": len(groups.get("application", [])),
|
||||
}
|
||||
|
||||
|
||||
def extract_info_by_type(
|
||||
groups: dict[str, list[dict[str, Any]]],
|
||||
cache_path: Path,
|
||||
) -> tuple[dict[str, Any] | None, dict[str, Any] | None]:
|
||||
"""根据发票类型提取差旅或普通报销信息
|
||||
|
||||
Args:
|
||||
groups: 发票分类结果。
|
||||
cache_path: 缓存目录路径。
|
||||
|
||||
Returns:
|
||||
(travel_info, normal_info) 元组，根据发票类型返回对应信息。
|
||||
"""
|
||||
if is_travel_invoice(groups):
|
||||
travel_info = extract_and_cache_travel_info(groups, cache_path)
|
||||
return travel_info, None
|
||||
else:
|
||||
normal_info = extract_and_cache_normal_info(groups, cache_path)
|
||||
return None, normal_info
|
||||
@@ -22,15 +22,22 @@ sys.path.insert(0, str(PROJECT_ROOT))
|
||||
from flask import Flask # noqa: E402, I001
|
||||
|
||||
from src.web import pipeline_web, routes # noqa: E402, I001
|
||||
from src.doc.fill_consumable_doc import CONSUMABLE_DOC_FILENAME # noqa: E402, I001
|
||||
from src.infra.documents import CONSUMABLE_DOC_FILENAME # noqa: E402, I001
|
||||
|
||||
app = Flask(__name__, template_folder="templates")
|
||||
|
||||
UPLOAD_BASE = PROJECT_ROOT / "src" / "web" / "uploads"
|
||||
|
||||
_app_initialized = False
|
||||
|
||||
|
||||
def create_app() -> Flask:
|
||||
"""应用工厂：初始化配置并注册路由"""
|
||||
global _app_initialized
|
||||
if _app_initialized:
|
||||
return app
|
||||
_app_initialized = True
|
||||
|
||||
# 设置出库单模板路径
|
||||
pipeline_web.set_consumable_template(PROJECT_ROOT / CONSUMABLE_DOC_FILENAME)
|
||||
|
||||
|
||||
@@ -8,25 +8,21 @@ Web 管道逻辑
|
||||
- 发票分类数据持久化
|
||||
"""
|
||||
|
||||
import json
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any, cast
|
||||
from typing import Any
|
||||
from urllib.parse import quote
|
||||
|
||||
# 延迟导入，避免循环引用
|
||||
from src import get_logger # noqa: F401
|
||||
from src.config import load_config as load_project_config
|
||||
from src.doc.fill_consumable_doc import (
|
||||
from src.infra.documents import (
|
||||
CONSUMABLE_DOC_FILENAME,
|
||||
fill_consumable_from_template,
|
||||
)
|
||||
from src.doc.invoice import (
|
||||
save_application_json,
|
||||
save_invoice_csv,
|
||||
)
|
||||
from src.doc.invoice import (
|
||||
save_csv as save_payment_csv,
|
||||
from src.pipeline_core import (
|
||||
extract_info_by_type,
|
||||
load_invoice_groups,
|
||||
process_invoices,
|
||||
)
|
||||
|
||||
fill_log = get_logger("fill_consumable_doc")
|
||||
@@ -36,40 +32,6 @@ SESSION_RESULT_FILE = "result.json"
|
||||
INVOICE_GROUPS_FILE = "invoice_groups.json"
|
||||
|
||||
|
||||
def save_invoice_groups(session_dir: Path, groups: dict[str, list[dict[str, str]]]) -> None:
|
||||
"""保存发票分类结果到 session 目录的 JSON 文件"""
|
||||
groups_path = session_dir / INVOICE_GROUPS_FILE
|
||||
data = {
|
||||
"travel_count": len(groups.get("travel", [])),
|
||||
"general_count": len(groups.get("general", [])),
|
||||
"application_count": len(groups.get("application", [])),
|
||||
}
|
||||
with open(groups_path, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, ensure_ascii=False, indent=2)
|
||||
|
||||
|
||||
def load_invoice_groups(session_dir: Path) -> dict[str, int] | None:
|
||||
"""从 session 目录加载发票分类统计"""
|
||||
groups_path = session_dir / INVOICE_GROUPS_FILE
|
||||
if not groups_path.exists():
|
||||
return None
|
||||
try:
|
||||
with open(groups_path, encoding="utf-8") as f:
|
||||
return cast(dict[str, int] | None, json.load(f))
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def load_session_config(session_dir: Path) -> dict[str, Any]:
|
||||
"""加载会话配置，合并项目全局配置与会话级配置"""
|
||||
config = load_project_config()
|
||||
cfg_path = session_dir / "config.json"
|
||||
if cfg_path.exists():
|
||||
with open(cfg_path, encoding="utf-8") as f:
|
||||
config.update(json.load(f))
|
||||
return config
|
||||
|
||||
|
||||
def resolve_payment_csv(session_dir: Path) -> Path | None:
|
||||
"""查找支付记录 CSV（payment_records.csv）"""
|
||||
csv_path = session_dir / "payment_records.csv"
|
||||
@@ -162,7 +124,7 @@ def append_doc_download(result: dict[str, Any], session_id: str, doc_fill: dict[
|
||||
# ================================================================
|
||||
|
||||
|
||||
def run_pipeline_web(session_dir: Path, config: dict[str, Any]) -> dict[str, Any]:
|
||||
def run_pipeline_web(session_dir: Path, config: dict[str, Any] | None) -> dict[str, Any]:
|
||||
"""在 Web 会话目录中执行发票提取，结果写入 session 目录下的文件
|
||||
|
||||
注意：不再自动提交财务系统。提交通由 /api/submit-financial/<session_id> 触发。
|
||||
@@ -171,7 +133,7 @@ def run_pipeline_web(session_dir: Path, config: dict[str, Any]) -> dict[str, Any
|
||||
- 差旅发票：调用 LLM 提取差旅信息并缓存到 travel_info.json
|
||||
- 普通发票：无需额外提取（normal_info.json 待实现）
|
||||
"""
|
||||
from src.doc.extractor import extract_invoices
|
||||
from src.core.extraction import extract_invoices
|
||||
|
||||
start = time.time()
|
||||
|
||||
@@ -180,73 +142,38 @@ def run_pipeline_web(session_dir: Path, config: dict[str, Any]) -> dict[str, Any
|
||||
if not invoices:
|
||||
return {"ok": False, "error": "未提取到任何发票数据"}
|
||||
|
||||
# 保存 CSV：支付记录级别（供 bot/出库单使用）和发票级别（供人工参考）
|
||||
save_payment_csv(invoices, session_dir / "payment_records.csv")
|
||||
save_invoice_csv(invoices, session_dir / "invoice_summary.csv")
|
||||
|
||||
# 出差申请单单独保存
|
||||
if applications:
|
||||
save_application_json(applications, session_dir / "travel_applications.json")
|
||||
|
||||
# 保存分类结果（供后续步骤统一读取）
|
||||
save_invoice_groups(session_dir, groups)
|
||||
# 使用公共函数处理发票数据
|
||||
stats = process_invoices(invoices, applications, groups, session_dir)
|
||||
|
||||
# ---- Step 2: 差旅/普通信息提取 ----
|
||||
is_travel = bool(groups.get("travel")) and not bool(groups.get("general"))
|
||||
if is_travel:
|
||||
from src.doc.llm_extractor import CACHE_DIR_NAME, extract_travel_info, load_cache
|
||||
|
||||
cache_map = load_cache(session_dir)
|
||||
if not cache_map.get("travel_info"):
|
||||
fill_log.info("开始提取差旅信息...")
|
||||
travel_info = extract_travel_info(source_dir=session_dir)
|
||||
cache_dir = session_dir / CACHE_DIR_NAME
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
with open(cache_dir / "travel_info.json", "w", encoding="utf-8") as f:
|
||||
json.dump(travel_info, f, ensure_ascii=False, indent=2)
|
||||
fill_log.info("差旅信息已保存到缓存")
|
||||
else:
|
||||
from src.doc.llm_extractor import CACHE_DIR_NAME, extract_normal_info, load_cache
|
||||
|
||||
cache_map = load_cache(session_dir)
|
||||
if not cache_map.get("normal_info"):
|
||||
fill_log.info("开始提取普通发票信息...")
|
||||
normal_info = extract_normal_info(source_dir=session_dir)
|
||||
cache_dir = session_dir / CACHE_DIR_NAME
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
with open(cache_dir / "normal_info.json", "w", encoding="utf-8") as f:
|
||||
json.dump(normal_info, f, ensure_ascii=False, indent=2)
|
||||
fill_log.info("普通发票信息已保存到缓存")
|
||||
|
||||
# 统计发票总数
|
||||
invoice_count = sum(len(inv.get("_matched_invoices", [])) for inv in invoices)
|
||||
extract_info_by_type(groups, session_dir)
|
||||
|
||||
elapsed = time.time() - start
|
||||
result = {
|
||||
"ok": True,
|
||||
"elapsed": f"{elapsed:.1f}s",
|
||||
"invoice_count": invoice_count,
|
||||
"invoice_count": stats["invoice_count"],
|
||||
"csv_url": f"/api/download/{session_dir.name}/invoice_summary.csv",
|
||||
"travel_count": len(groups["travel"]),
|
||||
"general_count": len(groups["general"]),
|
||||
"travel_count": stats["travel_count"],
|
||||
"general_count": stats["general_count"],
|
||||
}
|
||||
doc_fill = _try_fill_consumable_doc(session_dir, config)
|
||||
doc_fill = _try_fill_consumable_doc(session_dir, config or {})
|
||||
append_doc_download(result, session_dir.name, doc_fill)
|
||||
return result
|
||||
|
||||
|
||||
def run_financial_submit(session_dir: Path, config: dict[str, Any]) -> dict[str, Any]:
|
||||
def run_financial_submit(session_dir: Path, config: dict[str, Any] | None) -> dict[str, Any]:
|
||||
"""执行财务系统填报（从前端确认后调用）
|
||||
|
||||
从 invoice_groups.json 读取分类结果，根据发票类型选择填报模式：
|
||||
- 纯差旅发票：差旅报销模式（TODO）
|
||||
- 纯差旅发票：差旅报销模式
|
||||
- 含普通发票：普通报销模式
|
||||
"""
|
||||
csv_path = session_dir / "payment_records.csv"
|
||||
if not csv_path.exists():
|
||||
return {"ok": False, "error": "未找到发票数据，请先处理"}
|
||||
|
||||
from src.bot import run_bot_web
|
||||
from src.infra.browser import run_bot_web
|
||||
|
||||
groups = load_invoice_groups(session_dir)
|
||||
if groups:
|
||||
@@ -255,5 +182,9 @@ def run_financial_submit(session_dir: Path, config: dict[str, Any]) -> dict[str,
|
||||
else:
|
||||
fill_log.info("检测到普通发票，使用普通报销模式")
|
||||
|
||||
run_bot_web(config, session_dir)
|
||||
return {"ok": True}
|
||||
try:
|
||||
run_bot_web(config or {}, session_dir)
|
||||
return {"ok": True}
|
||||
except Exception as e:
|
||||
fill_log.error("财务填报失败: %s", e)
|
||||
return {"ok": False, "error": str(e)}
|
||||
|
||||
@@ -16,8 +16,15 @@ from urllib.parse import quote
|
||||
|
||||
from flask import Blueprint, Response, jsonify, render_template, request, stream_with_context
|
||||
|
||||
from src.config import load_config as load_project_config
|
||||
from src.doc.invoice import load_csv, load_invoice_csv
|
||||
from src.config import (
|
||||
SAFE_CONFIG_KEYS,
|
||||
SESSION_CONFIG_KEYS,
|
||||
load_session_config,
|
||||
)
|
||||
from src.config import (
|
||||
load_config as load_project_config,
|
||||
)
|
||||
from src.infra.documents import load_csv, load_invoice_csv
|
||||
|
||||
from . import pipeline_web, sse_handler
|
||||
|
||||
@@ -51,15 +58,11 @@ def _validate_session(session_id: str) -> Path | tuple[Response, int]:
|
||||
def _build_web_config(body: dict[str, Any]) -> dict[str, Any]:
|
||||
"""从请求体构建配置"""
|
||||
config = load_project_config()
|
||||
for key in (
|
||||
"username",
|
||||
"password",
|
||||
"default_name",
|
||||
"default_card_no",
|
||||
"default_person_id",
|
||||
"consumable_storage",
|
||||
):
|
||||
for key in SESSION_CONFIG_KEYS:
|
||||
if body.get(key):
|
||||
# 基本类型校验：防止非字符串值写入配置
|
||||
if not isinstance(body[key], (str, int, float, bool)):
|
||||
continue
|
||||
config[key] = body[key]
|
||||
return config
|
||||
|
||||
@@ -68,40 +71,32 @@ def _emit_ready_and_submit(
|
||||
session_dir: Path,
|
||||
agent_session: Any,
|
||||
config: dict[str, Any],
|
||||
) -> None:
|
||||
"""Agent 校验通过后，直接触发财务提交（不依赖前端）"""
|
||||
from src.agent import _emit_agent_event
|
||||
) -> dict[str, Any]:
|
||||
"""Agent 校验通过后，直接触发财务提交（不依赖前端）。
|
||||
|
||||
# 发射 agent_ready 事件，前端 SSE 会收到
|
||||
_emit_agent_event(
|
||||
session_dir,
|
||||
"agent_ready",
|
||||
round=agent_session.rounds,
|
||||
message="信息完整，可以提交",
|
||||
)
|
||||
|
||||
# 直接执行财务提交
|
||||
返回 result 字典，由调用方 _run_agent_task 的 finally 块统一写入 result.json。
|
||||
"""
|
||||
# agent_ready 事件已由 run_agent_round 发射，此处仅执行财务提交，避免前端收到重复消息
|
||||
try:
|
||||
submit_result = pipeline_web.run_financial_submit(session_dir, config)
|
||||
if submit_result.get("ok"):
|
||||
result = {
|
||||
return {
|
||||
"ok": True,
|
||||
"agent_ready": True,
|
||||
"submit_ok": True,
|
||||
"round": agent_session.rounds,
|
||||
"message": "信息完整，已自动提交到财务系统",
|
||||
}
|
||||
else:
|
||||
result = {
|
||||
"ok": True,
|
||||
"agent_ready": True,
|
||||
"submit_ok": False,
|
||||
"submit_error": submit_result.get("error"),
|
||||
"round": agent_session.rounds,
|
||||
"message": "校验通过但提交失败",
|
||||
}
|
||||
return {
|
||||
"ok": True,
|
||||
"agent_ready": True,
|
||||
"submit_ok": False,
|
||||
"submit_error": submit_result.get("error"),
|
||||
"round": agent_session.rounds,
|
||||
"message": "校验通过但提交失败",
|
||||
}
|
||||
except Exception as e:
|
||||
result = {
|
||||
return {
|
||||
"ok": True,
|
||||
"agent_ready": True,
|
||||
"submit_ok": False,
|
||||
@@ -110,14 +105,69 @@ def _emit_ready_and_submit(
|
||||
"message": f"校验通过但提交异常: {e}",
|
||||
}
|
||||
|
||||
# 写入 result 文件，SSE done 事件会读取
|
||||
|
||||
def _run_agent_task(
|
||||
session_dir: Path,
|
||||
handler: Any,
|
||||
task_fn: Any,
|
||||
config: dict[str, Any] | None = None,
|
||||
) -> None:
|
||||
"""Agent 任务的通用包装器
|
||||
|
||||
封装重复的闭包结构：清除流日志 -> 执行任务 -> 写入 result.json -> 卸载收集器。
|
||||
|
||||
Args:
|
||||
session_dir: 会话目录
|
||||
handler: SSE 日志收集器句柄
|
||||
task_fn: 业务逻辑回调，接收 (session_dir, config) 返回 (agent_session, result_dict) 或仅 result_dict
|
||||
config: 配置字典（可选）
|
||||
"""
|
||||
result = {"ok": False, "error": "未知错误"}
|
||||
try:
|
||||
tmp_path = session_dir / (pipeline_web.SESSION_RESULT_FILE + ".tmp")
|
||||
with open(tmp_path, "w", encoding="utf-8") as f:
|
||||
json.dump(result, f, ensure_ascii=False)
|
||||
tmp_path.replace(session_dir / pipeline_web.SESSION_RESULT_FILE)
|
||||
except Exception:
|
||||
pass
|
||||
# 清除上一轮残留文件，避免 SSE 连接立即读到旧数据
|
||||
for fname in ("llm_stream.log", "agent_events.log", pipeline_web.SESSION_RESULT_FILE):
|
||||
try:
|
||||
(session_dir / fname).unlink(missing_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
ret = task_fn(session_dir, config)
|
||||
|
||||
# task_fn 返回 (agent_session, result) 或仅 result
|
||||
if isinstance(ret, tuple) and len(ret) == 2:
|
||||
from src.agent import AgentState
|
||||
|
||||
agent_session, result = ret
|
||||
|
||||
if agent_session.state == AgentState.READY:
|
||||
if config is not None:
|
||||
result = _emit_ready_and_submit(session_dir, agent_session, config)
|
||||
elif agent_session.state == AgentState.AWAITING_SUPPLEMENT:
|
||||
result = {
|
||||
"ok": True,
|
||||
"agent_ready": False,
|
||||
"agent_state": agent_session.state.value,
|
||||
"round": agent_session.rounds,
|
||||
"waiting_for_supplement": True,
|
||||
}
|
||||
elif agent_session.state == AgentState.ERROR:
|
||||
result = {
|
||||
"ok": False,
|
||||
"error": agent_session.error_message,
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
result = {"ok": False, "error": str(e)}
|
||||
finally:
|
||||
# 统一写入 result.json，SSE 端点检测到后发射 done 事件
|
||||
try:
|
||||
tmp_path = session_dir / (pipeline_web.SESSION_RESULT_FILE + ".tmp")
|
||||
with open(tmp_path, "w", encoding="utf-8") as f:
|
||||
json.dump(result, f, ensure_ascii=False)
|
||||
tmp_path.replace(session_dir / pipeline_web.SESSION_RESULT_FILE)
|
||||
except Exception:
|
||||
pass
|
||||
sse_handler.remove_log_collector(handler)
|
||||
|
||||
|
||||
# ================================================================
|
||||
@@ -174,6 +224,9 @@ def upload_file(session_id: str) -> Any:
|
||||
return jsonify({"error": "未选择文件"}), 400
|
||||
|
||||
safe_name = Path(f.filename).name
|
||||
# 安全检查：拒绝包含路径穿越字符的文件名
|
||||
if ".." in safe_name or "/" in safe_name or "\\" in safe_name:
|
||||
return jsonify({"error": "文件名包含非法字符"}), 400
|
||||
f.save(str(session_dir / safe_name))
|
||||
return jsonify({"ok": True, "filename": safe_name})
|
||||
|
||||
@@ -247,7 +300,13 @@ def mobile_upload_file(session_id: str) -> Any:
|
||||
|
||||
@web_bp.route("/api/config/<session_id>", methods=["GET"])
|
||||
def get_session_config(session_id: str) -> Any:
|
||||
"""获取当前会话的配置（供前端回填表单）"""
|
||||
"""获取当前会话的配置（供前端回填表单）
|
||||
|
||||
使用白名单过滤会话配置，防止密码等敏感字段被加载到内存。
|
||||
password 字段始终返回空字符串——密码由前端用户输入，不持久化。
|
||||
注意：当前为 localhost 服务，密码通过 HTTP 明文传输。如需通过代理暴露服务，
|
||||
请启用 HTTPS 或使用反向代理加密。
|
||||
"""
|
||||
session_dir = _validate_session(session_id)
|
||||
if isinstance(session_dir, tuple):
|
||||
return session_dir
|
||||
@@ -255,8 +314,12 @@ def get_session_config(session_id: str) -> Any:
|
||||
config = load_project_config()
|
||||
cfg_path = session_dir / "config.json"
|
||||
if cfg_path.exists():
|
||||
# 仅允许覆盖前端表单字段（不含密码）
|
||||
with open(cfg_path, encoding="utf-8") as f:
|
||||
config.update(json.load(f))
|
||||
session_cfg = json.load(f)
|
||||
for key in SAFE_CONFIG_KEYS:
|
||||
if key in session_cfg:
|
||||
config[key] = session_cfg[key]
|
||||
return jsonify(
|
||||
{
|
||||
"username": config.get("username", ""),
|
||||
@@ -348,6 +411,12 @@ def save_invoice_data(session_id: str) -> Any:
|
||||
|
||||
fieldnames = list(original_rows[0].keys())
|
||||
|
||||
# 校验前端传入的 data 字段是否与原始 CSV 列匹配
|
||||
if data:
|
||||
unknown_keys = set(data[0].keys()) - (set(fieldnames) | {"__row"})
|
||||
if unknown_keys:
|
||||
return jsonify({"error": f"数据包含未知字段: {unknown_keys}"}), 400
|
||||
|
||||
with open(csv_path, "w", newline="", encoding="utf-8-sig") as f:
|
||||
writer = csv_module.DictWriter(f, fieldnames=fieldnames)
|
||||
writer.writeheader()
|
||||
@@ -356,7 +425,7 @@ def save_invoice_data(session_id: str) -> Any:
|
||||
writer.writerow(row)
|
||||
|
||||
resp: dict[str, str | bool | None] = {"ok": True}
|
||||
config = pipeline_web.load_session_config(session_dir)
|
||||
config = load_session_config(session_dir)
|
||||
doc_fill = pipeline_web._try_fill_consumable_doc(session_dir, config)
|
||||
if doc_fill.get("ok"):
|
||||
fn = doc_fill["doc_filename"]
|
||||
@@ -389,31 +458,15 @@ def start_process(session_id: str) -> Any:
|
||||
with open(session_dir / "config.json", "w", encoding="utf-8") as f:
|
||||
json.dump(config, f, ensure_ascii=False, indent=2, default=str)
|
||||
|
||||
def _task(sd: Path, cfg: dict[str, Any] | None) -> dict[str, Any]:
|
||||
return pipeline_web.run_pipeline_web(sd, cfg)
|
||||
|
||||
handler = sse_handler.install_log_collector(session_dir)
|
||||
|
||||
def _run() -> None:
|
||||
result = {"ok": False, "error": "未知错误"}
|
||||
try:
|
||||
try:
|
||||
(session_dir / "llm_stream.log").unlink(missing_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
result = pipeline_web.run_pipeline_web(session_dir, config)
|
||||
except BaseException as e:
|
||||
result = {"ok": False, "error": str(e)}
|
||||
if isinstance(e, KeyboardInterrupt | SystemExit):
|
||||
raise
|
||||
finally:
|
||||
try:
|
||||
tmp_path = session_dir / (pipeline_web.SESSION_RESULT_FILE + ".tmp")
|
||||
with open(tmp_path, "w", encoding="utf-8") as f:
|
||||
json.dump(result, f, ensure_ascii=False)
|
||||
tmp_path.replace(session_dir / pipeline_web.SESSION_RESULT_FILE)
|
||||
except Exception:
|
||||
pass
|
||||
sse_handler.remove_log_collector(handler)
|
||||
|
||||
threading.Thread(target=_run, daemon=True).start()
|
||||
threading.Thread(
|
||||
target=_run_agent_task,
|
||||
kwargs={"session_dir": session_dir, "handler": handler, "task_fn": _task, "config": config},
|
||||
daemon=True,
|
||||
).start()
|
||||
return jsonify({"status": "started"})
|
||||
|
||||
|
||||
@@ -433,7 +486,7 @@ def stream_logs(session_id: str) -> Any:
|
||||
state = {"agent_size": 0}
|
||||
|
||||
start_time = time.time()
|
||||
timeout = 600
|
||||
timeout = 900 # 略大于 LLM 请求超时 (600s)，防止 SSE 先断开
|
||||
|
||||
while time.time() - start_time < timeout:
|
||||
# 轮询普通日志
|
||||
@@ -523,43 +576,18 @@ def submit_financial(session_id: str) -> Any:
|
||||
with open(config_path, encoding="utf-8") as f:
|
||||
config = json.load(f)
|
||||
|
||||
result_file = session_dir / pipeline_web.SESSION_RESULT_FILE
|
||||
if result_file.exists():
|
||||
result_file.unlink()
|
||||
def _task(sd: Path, cfg: dict[str, Any] | None) -> dict[str, Any]:
|
||||
submit_result = pipeline_web.run_financial_submit(sd, cfg)
|
||||
if submit_result.get("ok"):
|
||||
return {"ok": True}
|
||||
return submit_result
|
||||
|
||||
handler = sse_handler.install_log_collector(session_dir)
|
||||
|
||||
def _run() -> None:
|
||||
result = {"ok": False, "error": "未知错误"}
|
||||
try:
|
||||
try:
|
||||
(session_dir / "llm_stream.log").unlink(missing_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
submit_result = pipeline_web.run_financial_submit(session_dir, config)
|
||||
if submit_result.get("ok"):
|
||||
result = {"ok": True}
|
||||
else:
|
||||
result = submit_result
|
||||
except BaseException as e:
|
||||
result = {"ok": False, "error": str(e)}
|
||||
if isinstance(e, KeyboardInterrupt | SystemExit):
|
||||
raise
|
||||
finally:
|
||||
try:
|
||||
tmp_path = session_dir / (pipeline_web.SESSION_RESULT_FILE + ".tmp")
|
||||
with open(tmp_path, "w", encoding="utf-8") as f:
|
||||
json.dump(
|
||||
{"ok": True, "submit_ok": result.get("ok"), "submit_error": result.get("error")},
|
||||
f,
|
||||
ensure_ascii=False,
|
||||
)
|
||||
tmp_path.replace(session_dir / pipeline_web.SESSION_RESULT_FILE)
|
||||
except Exception:
|
||||
pass
|
||||
sse_handler.remove_log_collector(handler)
|
||||
|
||||
threading.Thread(target=_run, daemon=True).start()
|
||||
threading.Thread(
|
||||
target=_run_agent_task,
|
||||
kwargs={"session_dir": session_dir, "handler": handler, "task_fn": _task, "config": config},
|
||||
daemon=True,
|
||||
).start()
|
||||
return jsonify({"status": "started"})
|
||||
|
||||
|
||||
@@ -599,88 +627,52 @@ def agent_process(session_id: str) -> Any:
|
||||
|
||||
handler = sse_handler.install_log_collector(session_dir)
|
||||
|
||||
def _run() -> None:
|
||||
result = {"ok": False, "error": "未知错误"}
|
||||
try:
|
||||
try:
|
||||
(session_dir / "llm_stream.log").unlink(missing_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
def _task(sd: Path, cfg: dict[str, Any] | None) -> tuple[Any, dict[str, Any]]:
|
||||
from src.agent import (
|
||||
AgentSession,
|
||||
load_agent_state,
|
||||
run_agent_round,
|
||||
save_agent_state,
|
||||
)
|
||||
from src.core.extraction import extract_invoices
|
||||
from src.infra.documents import (
|
||||
save_application_json,
|
||||
save_invoice_csv,
|
||||
)
|
||||
from src.infra.documents import (
|
||||
save_csv as save_payment_csv,
|
||||
)
|
||||
|
||||
from src.agent import (
|
||||
AgentSession,
|
||||
AgentState,
|
||||
load_agent_state,
|
||||
run_agent_round,
|
||||
save_agent_state,
|
||||
)
|
||||
from src.doc.extractor import extract_invoices
|
||||
from src.doc.invoice import (
|
||||
save_application_json,
|
||||
save_invoice_csv,
|
||||
)
|
||||
from src.doc.invoice import (
|
||||
save_csv as save_payment_csv,
|
||||
agent_session = load_agent_state(sd)
|
||||
if agent_session is None:
|
||||
invoices, applications, groups = extract_invoices(str(sd))
|
||||
if not invoices:
|
||||
return (None, {"ok": False, "error": "未提取到任何发票数据"})
|
||||
|
||||
save_payment_csv(invoices, sd / "payment_records.csv")
|
||||
save_invoice_csv(invoices, sd / "invoice_summary.csv")
|
||||
|
||||
if applications:
|
||||
save_application_json(applications, sd / "travel_applications.json")
|
||||
|
||||
pipeline_web.save_invoice_groups(sd, groups)
|
||||
|
||||
from src.pipeline_core import is_travel_invoice
|
||||
|
||||
agent_session = AgentSession(
|
||||
session_id=session_id,
|
||||
invoice_type="travel" if is_travel_invoice(groups) else "normal",
|
||||
)
|
||||
save_agent_state(sd, agent_session)
|
||||
|
||||
agent_session = load_agent_state(session_dir)
|
||||
if agent_session is None:
|
||||
invoices, applications, groups = extract_invoices(str(session_dir))
|
||||
if not invoices:
|
||||
result = {"ok": False, "error": "未提取到任何发票数据"}
|
||||
return
|
||||
agent_session = run_agent_round(sd, agent_session)
|
||||
return (agent_session, {})
|
||||
|
||||
save_payment_csv(invoices, session_dir / "payment_records.csv")
|
||||
save_invoice_csv(invoices, session_dir / "invoice_summary.csv")
|
||||
|
||||
if applications:
|
||||
save_application_json(applications, session_dir / "travel_applications.json")
|
||||
|
||||
pipeline_web.save_invoice_groups(session_dir, groups)
|
||||
|
||||
is_travel = bool(groups.get("travel")) and not bool(groups.get("general"))
|
||||
agent_session = AgentSession(
|
||||
session_id=session_id,
|
||||
invoice_type="travel" if is_travel else "normal",
|
||||
)
|
||||
save_agent_state(session_dir, agent_session)
|
||||
|
||||
agent_session = run_agent_round(session_dir, agent_session)
|
||||
|
||||
if agent_session.state == AgentState.READY:
|
||||
# Agent 校验通过，直接触发财务提交
|
||||
_emit_ready_and_submit(session_dir, agent_session, config)
|
||||
elif agent_session.state == AgentState.AWAITING_SUPPLEMENT:
|
||||
result = {
|
||||
"ok": True,
|
||||
"agent_ready": False,
|
||||
"agent_state": agent_session.state.value,
|
||||
"round": agent_session.rounds,
|
||||
"waiting_for_supplement": True,
|
||||
}
|
||||
elif agent_session.state == AgentState.ERROR:
|
||||
result = {
|
||||
"ok": False,
|
||||
"error": agent_session.error_message,
|
||||
}
|
||||
|
||||
except BaseException as e:
|
||||
result = {"ok": False, "error": str(e)}
|
||||
if isinstance(e, KeyboardInterrupt | SystemExit):
|
||||
raise
|
||||
finally:
|
||||
try:
|
||||
result_file = session_dir / pipeline_web.SESSION_RESULT_FILE
|
||||
if not result_file.exists():
|
||||
tmp_path = session_dir / (pipeline_web.SESSION_RESULT_FILE + ".tmp")
|
||||
with open(tmp_path, "w", encoding="utf-8") as f:
|
||||
json.dump(result, f, ensure_ascii=False)
|
||||
tmp_path.replace(session_dir / pipeline_web.SESSION_RESULT_FILE)
|
||||
except Exception:
|
||||
pass
|
||||
sse_handler.remove_log_collector(handler)
|
||||
|
||||
threading.Thread(target=_run, daemon=True).start()
|
||||
threading.Thread(
|
||||
target=_run_agent_task,
|
||||
kwargs={"session_dir": session_dir, "handler": handler, "task_fn": _task, "config": config},
|
||||
daemon=True,
|
||||
).start()
|
||||
return jsonify({"status": "started"})
|
||||
|
||||
|
||||
@@ -698,7 +690,6 @@ def agent_supplement(session_id: str) -> Any:
|
||||
return jsonify({"error": "未指定补充文件"}), 400
|
||||
|
||||
from src.agent import (
|
||||
AgentState,
|
||||
add_supplement,
|
||||
load_agent_state,
|
||||
run_agent_round,
|
||||
@@ -710,74 +701,42 @@ def agent_supplement(session_id: str) -> Any:
|
||||
|
||||
agent_session = add_supplement(session_dir, agent_session, filenames)
|
||||
|
||||
# 重新提取发票并保存
|
||||
from src.core.extraction import extract_invoices
|
||||
from src.infra.documents import (
|
||||
save_application_json,
|
||||
save_invoice_csv,
|
||||
)
|
||||
from src.infra.documents import (
|
||||
save_csv as save_payment_csv,
|
||||
)
|
||||
|
||||
invoices, applications, groups = extract_invoices(str(session_dir))
|
||||
save_payment_csv(invoices, session_dir / "payment_records.csv")
|
||||
save_invoice_csv(invoices, session_dir / "invoice_summary.csv")
|
||||
|
||||
if applications:
|
||||
save_application_json(applications, session_dir / "travel_applications.json")
|
||||
|
||||
pipeline_web.save_invoice_groups(session_dir, groups)
|
||||
|
||||
config_path = session_dir / "config.json"
|
||||
config = {}
|
||||
if config_path.exists():
|
||||
with open(config_path, encoding="utf-8") as f:
|
||||
config = json.load(f)
|
||||
|
||||
handler = sse_handler.install_log_collector(session_dir)
|
||||
|
||||
def _run() -> None:
|
||||
result = {"ok": False, "error": "未知错误"}
|
||||
try:
|
||||
try:
|
||||
(session_dir / "llm_stream.log").unlink(missing_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
def _task(sd: Path, cfg: dict[str, Any] | None) -> tuple[Any, dict[str, Any]]:
|
||||
new_session = run_agent_round(sd, agent_session, new_files=filenames)
|
||||
return (new_session, {})
|
||||
|
||||
from src.doc.extractor import extract_invoices
|
||||
from src.doc.invoice import (
|
||||
save_application_json,
|
||||
save_invoice_csv,
|
||||
)
|
||||
from src.doc.invoice import (
|
||||
save_csv as save_payment_csv,
|
||||
)
|
||||
|
||||
invoices, applications, groups = extract_invoices(str(session_dir))
|
||||
save_payment_csv(invoices, session_dir / "payment_records.csv")
|
||||
save_invoice_csv(invoices, session_dir / "invoice_summary.csv")
|
||||
|
||||
if applications:
|
||||
save_application_json(applications, session_dir / "travel_applications.json")
|
||||
|
||||
pipeline_web.save_invoice_groups(session_dir, groups)
|
||||
|
||||
new_session = run_agent_round(session_dir, agent_session, new_files=filenames)
|
||||
|
||||
if new_session.state == AgentState.READY:
|
||||
# Agent 校验通过，直接触发财务提交
|
||||
config_path = session_dir / "config.json"
|
||||
with open(config_path, encoding="utf-8") as f:
|
||||
config = json.load(f)
|
||||
|
||||
_emit_ready_and_submit(session_dir, new_session, config)
|
||||
elif new_session.state == AgentState.AWAITING_SUPPLEMENT:
|
||||
result = {
|
||||
"ok": True,
|
||||
"agent_ready": False,
|
||||
"agent_state": new_session.state.value,
|
||||
"round": new_session.rounds,
|
||||
"waiting_for_supplement": True,
|
||||
}
|
||||
elif new_session.state == AgentState.ERROR:
|
||||
result = {
|
||||
"ok": False,
|
||||
"error": new_session.error_message,
|
||||
}
|
||||
|
||||
except BaseException as e:
|
||||
result = {"ok": False, "error": str(e)}
|
||||
if isinstance(e, KeyboardInterrupt | SystemExit):
|
||||
raise
|
||||
finally:
|
||||
try:
|
||||
result_file = session_dir / pipeline_web.SESSION_RESULT_FILE
|
||||
if result.get("error") and not result_file.exists():
|
||||
tmp_path = session_dir / (pipeline_web.SESSION_RESULT_FILE + ".tmp")
|
||||
with open(tmp_path, "w", encoding="utf-8") as f:
|
||||
json.dump(result, f, ensure_ascii=False)
|
||||
tmp_path.replace(session_dir / pipeline_web.SESSION_RESULT_FILE)
|
||||
except Exception:
|
||||
pass
|
||||
sse_handler.remove_log_collector(handler)
|
||||
|
||||
threading.Thread(target=_run, daemon=True).start()
|
||||
threading.Thread(
|
||||
target=_run_agent_task,
|
||||
kwargs={"session_dir": session_dir, "handler": handler, "task_fn": _task, "config": config},
|
||||
daemon=True,
|
||||
).start()
|
||||
return jsonify({"status": "started"})
|
||||
|
||||
|
||||
@@ -795,7 +754,6 @@ def agent_user_supplement(session_id: str) -> Any:
|
||||
return jsonify({"error": "请输入补充信息"}), 400
|
||||
|
||||
from src.agent import (
|
||||
AgentState,
|
||||
load_agent_state,
|
||||
process_user_text_supplement,
|
||||
)
|
||||
@@ -804,55 +762,23 @@ def agent_user_supplement(session_id: str) -> Any:
|
||||
if agent_session is None:
|
||||
return jsonify({"error": "未找到 Agent 状态"}), 404
|
||||
|
||||
config_path = session_dir / "config.json"
|
||||
config = {}
|
||||
if config_path.exists():
|
||||
with open(config_path, encoding="utf-8") as f:
|
||||
config = json.load(f)
|
||||
|
||||
handler = sse_handler.install_log_collector(session_dir)
|
||||
|
||||
def _run() -> None:
|
||||
result = {"ok": False, "error": "未知错误"}
|
||||
try:
|
||||
try:
|
||||
(session_dir / "llm_stream.log").unlink(missing_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
def _task(sd: Path, cfg: dict[str, Any] | None) -> tuple[Any, dict[str, Any]]:
|
||||
new_session = process_user_text_supplement(sd, agent_session, user_text)
|
||||
return (new_session, {})
|
||||
|
||||
new_session = process_user_text_supplement(session_dir, agent_session, user_text)
|
||||
|
||||
if new_session.state == AgentState.READY:
|
||||
# Agent 校验通过，直接触发财务提交
|
||||
config_path = session_dir / "config.json"
|
||||
with open(config_path, encoding="utf-8") as f:
|
||||
config = json.load(f)
|
||||
_emit_ready_and_submit(session_dir, new_session, config)
|
||||
elif new_session.state == AgentState.AWAITING_SUPPLEMENT:
|
||||
result = {
|
||||
"ok": True,
|
||||
"agent_ready": False,
|
||||
"agent_state": new_session.state.value,
|
||||
"round": new_session.rounds,
|
||||
"waiting_for_supplement": True,
|
||||
}
|
||||
elif new_session.state == AgentState.ERROR:
|
||||
result = {
|
||||
"ok": False,
|
||||
"error": new_session.error_message,
|
||||
}
|
||||
|
||||
except BaseException as e:
|
||||
result = {"ok": False, "error": str(e)}
|
||||
if isinstance(e, KeyboardInterrupt | SystemExit):
|
||||
raise
|
||||
finally:
|
||||
try:
|
||||
result_file = session_dir / pipeline_web.SESSION_RESULT_FILE
|
||||
if not result_file.exists():
|
||||
tmp_path = session_dir / (pipeline_web.SESSION_RESULT_FILE + ".tmp")
|
||||
with open(tmp_path, "w", encoding="utf-8") as f:
|
||||
json.dump(result, f, ensure_ascii=False)
|
||||
tmp_path.replace(session_dir / pipeline_web.SESSION_RESULT_FILE)
|
||||
except Exception:
|
||||
pass
|
||||
sse_handler.remove_log_collector(handler)
|
||||
|
||||
threading.Thread(target=_run, daemon=True).start()
|
||||
threading.Thread(
|
||||
target=_run_agent_task,
|
||||
kwargs={"session_dir": session_dir, "handler": handler, "task_fn": _task, "config": config},
|
||||
daemon=True,
|
||||
).start()
|
||||
return jsonify({"status": "started"})
|
||||
|
||||
|
||||
@@ -881,37 +807,16 @@ def agent_force_submit(session_id: str) -> Any:
|
||||
with open(config_path, encoding="utf-8") as f:
|
||||
config = json.load(f)
|
||||
|
||||
result_file = session_dir / pipeline_web.SESSION_RESULT_FILE
|
||||
if result_file.exists():
|
||||
result_file.unlink()
|
||||
def _task(sd: Path, cfg: dict[str, Any] | None) -> dict[str, Any]:
|
||||
submit_result = pipeline_web.run_financial_submit(sd, cfg)
|
||||
if submit_result.get("ok"):
|
||||
return {"ok": True}
|
||||
return submit_result
|
||||
|
||||
handler = sse_handler.install_log_collector(session_dir)
|
||||
|
||||
def _run() -> None:
|
||||
result = {"ok": False, "error": "未知错误"}
|
||||
try:
|
||||
submit_result = pipeline_web.run_financial_submit(session_dir, config)
|
||||
if submit_result.get("ok"):
|
||||
result = {"ok": True}
|
||||
else:
|
||||
result = submit_result
|
||||
except BaseException as e:
|
||||
result = {"ok": False, "error": str(e)}
|
||||
if isinstance(e, KeyboardInterrupt | SystemExit):
|
||||
raise
|
||||
finally:
|
||||
try:
|
||||
tmp_path = session_dir / (pipeline_web.SESSION_RESULT_FILE + ".tmp")
|
||||
with open(tmp_path, "w", encoding="utf-8") as f:
|
||||
json.dump(
|
||||
{"ok": True, "submit_ok": result.get("ok"), "submit_error": result.get("error")},
|
||||
f,
|
||||
ensure_ascii=False,
|
||||
)
|
||||
tmp_path.replace(session_dir / pipeline_web.SESSION_RESULT_FILE)
|
||||
except Exception:
|
||||
pass
|
||||
sse_handler.remove_log_collector(handler)
|
||||
|
||||
threading.Thread(target=_run, daemon=True).start()
|
||||
threading.Thread(
|
||||
target=_run_agent_task,
|
||||
kwargs={"session_dir": session_dir, "handler": handler, "task_fn": _task, "config": config},
|
||||
daemon=True,
|
||||
).start()
|
||||
return jsonify({"status": "started"})
|
||||
|
||||
@@ -47,10 +47,11 @@ class SSELogHandler(logging.Handler):
|
||||
pass
|
||||
|
||||
def close_file(self) -> None:
|
||||
try:
|
||||
self._file.close()
|
||||
except Exception:
|
||||
pass
|
||||
with self._lock:
|
||||
try:
|
||||
self._file.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def install_log_collector(session_dir: Path) -> SSELogHandler:
|
||||
|
||||
@@ -503,3 +503,90 @@ body {
|
||||
font-size: 13px;
|
||||
padding: 6px 16px;
|
||||
}
|
||||
|
||||
/* ================================================================ */
|
||||
/* 提取结果摘要卡片样式 */
|
||||
/* ================================================================ */
|
||||
|
||||
.extraction-summary {
|
||||
margin-top: 8px;
|
||||
padding-top: 8px;
|
||||
border-top: 1px dashed #ccc;
|
||||
}
|
||||
|
||||
.extraction-summary .summary-section {
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
|
||||
.extraction-summary .summary-section:last-child {
|
||||
margin-bottom: 0;
|
||||
}
|
||||
|
||||
.extraction-summary .summary-section-title {
|
||||
font-size: 13px;
|
||||
font-weight: 700;
|
||||
color: #333;
|
||||
margin-bottom: 6px;
|
||||
padding-bottom: 4px;
|
||||
border-bottom: 1px solid #eee;
|
||||
}
|
||||
|
||||
.extraction-summary .summary-row {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
padding: 2px 0;
|
||||
font-size: 12px;
|
||||
line-height: 1.8;
|
||||
}
|
||||
|
||||
.extraction-summary .summary-label {
|
||||
color: #888;
|
||||
min-width: 60px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.extraction-summary .summary-value {
|
||||
color: #333;
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
.extraction-summary .summary-table {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
font-size: 12px;
|
||||
margin-top: 4px;
|
||||
}
|
||||
|
||||
.extraction-summary .summary-table th {
|
||||
background: #f5f6f8;
|
||||
color: #555;
|
||||
font-weight: 600;
|
||||
padding: 5px 8px;
|
||||
text-align: left;
|
||||
border-bottom: 1px solid #e0e0e0;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.extraction-summary .summary-table td {
|
||||
padding: 4px 8px;
|
||||
border-bottom: 1px solid #f0f0f0;
|
||||
color: #333;
|
||||
vertical-align: top;
|
||||
}
|
||||
|
||||
.extraction-summary .summary-table tbody tr:hover {
|
||||
background: #fafbfc;
|
||||
}
|
||||
|
||||
.extraction-summary .summary-attachment-list {
|
||||
margin: 4px 0 0;
|
||||
padding-left: 20px;
|
||||
font-size: 12px;
|
||||
color: #555;
|
||||
line-height: 1.8;
|
||||
}
|
||||
|
||||
.extraction-summary .summary-attachment-item::marker {
|
||||
color: #999;
|
||||
font-size: 10px;
|
||||
}
|
||||
@@ -4,6 +4,7 @@
|
||||
import { App } from './state.js';
|
||||
import { escapeHtml } from './utils.js';
|
||||
import { addChatMessage, showStatus } from './chat.js';
|
||||
import { createSSEConnection, handleDoneResult } from './sse.js';
|
||||
|
||||
// ================================================================
|
||||
// Agent 事件处理
|
||||
@@ -34,6 +35,12 @@ export function handleAgentEvent(msg) {
|
||||
case 'agent_force_submit':
|
||||
_handleForceSubmit(msg);
|
||||
break;
|
||||
case 'agent_max_rounds':
|
||||
_handleAgentError(msg);
|
||||
break;
|
||||
case 'agent_extract_status':
|
||||
_handleAgentStateChange(msg);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -59,82 +66,19 @@ function _handleAgentStateChange(msg) {
|
||||
}
|
||||
|
||||
/**
|
||||
* 请求补充材料 — 仅更新瞬态状态，持久提示消息由 done 事件分支负责
|
||||
* 请求补充材料 — 将 suggestion 以消息气泡形式单独展示
|
||||
*/
|
||||
export function _handleAgentRequestSupplement(msg) {
|
||||
const suggestion = msg.suggestion || '请补充上传相关材料';
|
||||
showStatus(suggestion);
|
||||
showStatus('信息不完整，请补充材料');
|
||||
addChatMessage(suggestion, 'system');
|
||||
}
|
||||
|
||||
/**
|
||||
* 信息完整 — 已由 process.js 的 done 处理覆盖，此处不再重复显示
|
||||
* 信息完整 — 在消息气泡中提醒用户，最终结果由 done 事件处理
|
||||
*/
|
||||
function _handleAgentReady(msg) {
|
||||
// agent_ready 事件的信息展示统一由 process.js done 分支处理
|
||||
}
|
||||
|
||||
/**
|
||||
* 自动触发财务系统提交（由 done 事件调用）
|
||||
*/
|
||||
export async function handleAutoSubmit() {
|
||||
if (!App.sessionId) {
|
||||
addChatMessage('请先上传文件并处理', 'error');
|
||||
return;
|
||||
}
|
||||
|
||||
if (App.agentEventSource) {
|
||||
App.agentEventSource.close();
|
||||
App.agentEventSource = null;
|
||||
}
|
||||
|
||||
App.isProcessing = true;
|
||||
App.processState = 'submitting';
|
||||
|
||||
try {
|
||||
showStatus('正在自动提交到财务系统...');
|
||||
|
||||
const response = await fetch(`/api/submit-financial/${App.sessionId}`, {
|
||||
method: 'POST',
|
||||
});
|
||||
|
||||
const result = await response.json();
|
||||
|
||||
if (result.status === 'started') {
|
||||
const es = new EventSource(`/api/logs/${App.sessionId}`);
|
||||
es.addEventListener('message', e => {
|
||||
try {
|
||||
const msg = JSON.parse(e.data);
|
||||
if (msg.type === 'done') {
|
||||
es.close();
|
||||
App.isProcessing = false;
|
||||
App.processState = 'done';
|
||||
if (msg.result.submit_ok) {
|
||||
addChatMessage('提交完成！', 'done');
|
||||
} else {
|
||||
addChatMessage(`提交失败：${msg.result.submit_error || '未知错误'}`, 'error');
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
// 普通日志行
|
||||
}
|
||||
});
|
||||
|
||||
es.onerror = () => {
|
||||
es.close();
|
||||
App.isProcessing = false;
|
||||
App.processState = 'done';
|
||||
addChatMessage('连接中断', 'error');
|
||||
};
|
||||
} else if (result.error) {
|
||||
App.isProcessing = false;
|
||||
App.processState = 'done';
|
||||
addChatMessage(`提交失败：${result.error}`, 'error');
|
||||
}
|
||||
} catch (e) {
|
||||
App.isProcessing = false;
|
||||
App.processState = 'done';
|
||||
addChatMessage(`请求失败：${e.message}`, 'error');
|
||||
}
|
||||
addChatMessage(msg.message || '信息完整，正在提交到财务系统', 'done');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -242,49 +186,15 @@ export async function handleUserSupplement(text) {
|
||||
|
||||
if (result.status === 'started') {
|
||||
showStatus('补充信息已收到，正在重新分析...');
|
||||
App.isProcessing = true;
|
||||
|
||||
const es = new EventSource(`/api/logs/${App.sessionId}`);
|
||||
es.addEventListener('message', e => {
|
||||
try {
|
||||
const msg = JSON.parse(e.data);
|
||||
|
||||
if (msg.type && msg.type.startsWith('agent_')) {
|
||||
handleAgentEvent(msg);
|
||||
return;
|
||||
}
|
||||
|
||||
if (msg.type === 'done') {
|
||||
es.close();
|
||||
App.isProcessing = false;
|
||||
|
||||
if (msg.result.waiting_for_supplement) {
|
||||
App.processState = 'awaiting_supplement';
|
||||
_handleAgentRequestSupplement(msg.result);
|
||||
addChatMessage('您可通过上传区补充文件，或直接输入文字说明补充信息，或输入"直接提交"跳过校验', 'system');
|
||||
} else {
|
||||
App.processState = 'done';
|
||||
if (msg.result.ok) {
|
||||
if (msg.result.submit_ok !== false) {
|
||||
addChatMessage('信息完整，已自动提交到财务系统', 'done');
|
||||
} else {
|
||||
addChatMessage(`校验通过但提交失败：${msg.result.submit_error || '未知错误'}`, 'error');
|
||||
}
|
||||
} else {
|
||||
addChatMessage(`分析失败：${msg.result.error || '未知错误'}`, 'error');
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
// 普通日志
|
||||
}
|
||||
createSSEConnection(App.sessionId, {
|
||||
onDone: (result) => {
|
||||
handleDoneResult(result);
|
||||
},
|
||||
handleFileProgress: false,
|
||||
handleLLMStream: false,
|
||||
});
|
||||
|
||||
es.onerror = () => {
|
||||
es.close();
|
||||
App.isProcessing = false;
|
||||
App.processState = 'done';
|
||||
addChatMessage('连接中断', 'error');
|
||||
};
|
||||
} else if (result.error) {
|
||||
addChatMessage(`处理失败：${result.error}`, 'error');
|
||||
}
|
||||
@@ -328,35 +238,13 @@ export async function handleForceSubmit() {
|
||||
App.isProcessing = true;
|
||||
App.processState = 'submitting';
|
||||
|
||||
const es = new EventSource(`/api/logs/${App.sessionId}`);
|
||||
es.addEventListener('message', e => {
|
||||
try {
|
||||
const msg = JSON.parse(e.data);
|
||||
if (msg.type === 'done') {
|
||||
es.close();
|
||||
App.isProcessing = false;
|
||||
App.processState = 'done';
|
||||
App.forceSubmitting = false;
|
||||
App.lastProcessedFileCount = App.allFiles.length;
|
||||
if (msg.result.ok) {
|
||||
addChatMessage('提交完成！', 'done');
|
||||
} else {
|
||||
addChatMessage(`提交失败：${msg.result.submit_error || '未知错误'}`, 'error');
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
// 普通日志行
|
||||
}
|
||||
createSSEConnection(App.sessionId, {
|
||||
onDone: (result) => {
|
||||
handleDoneResult(result);
|
||||
},
|
||||
handleFileProgress: false,
|
||||
handleLLMStream: false,
|
||||
});
|
||||
|
||||
es.onerror = () => {
|
||||
es.close();
|
||||
App.isProcessing = false;
|
||||
App.processState = 'done';
|
||||
App.forceSubmitting = false;
|
||||
App.lastProcessedFileCount = App.allFiles.length;
|
||||
addChatMessage('连接中断', 'error');
|
||||
};
|
||||
}
|
||||
} catch (e) {
|
||||
App.forceSubmitting = false;
|
||||
@@ -389,49 +277,13 @@ export async function handleSupplementUpload(filenames) {
|
||||
const result = await response.json();
|
||||
|
||||
if (result.status === 'started') {
|
||||
const es = new EventSource(`/api/logs/${App.sessionId}`);
|
||||
es.addEventListener('message', e => {
|
||||
try {
|
||||
const msg = JSON.parse(e.data);
|
||||
App.isProcessing = true;
|
||||
|
||||
if (msg.type && msg.type.startsWith('agent_')) {
|
||||
handleAgentEvent(msg);
|
||||
return;
|
||||
}
|
||||
|
||||
if (msg.type === 'done') {
|
||||
es.close();
|
||||
App.isProcessing = false;
|
||||
|
||||
if (msg.result.waiting_for_supplement) {
|
||||
App.processState = 'awaiting_supplement';
|
||||
showStatus('信息不完整，请补充上传相关材料');
|
||||
addChatMessage('您可通过上传区补充文件，或直接输入文字说明补充信息，或输入"直接提交"跳过校验', 'system');
|
||||
} else {
|
||||
App.processState = 'done';
|
||||
if (msg.result.ok) {
|
||||
// 后端已自动提交，根据 submit_ok 显示结果
|
||||
if (msg.result.submit_ok !== false) {
|
||||
addChatMessage('信息完整，已自动提交到财务系统', 'done');
|
||||
} else {
|
||||
addChatMessage(`校验通过但提交失败：${msg.result.submit_error || '未知错误'}`, 'error');
|
||||
}
|
||||
} else {
|
||||
addChatMessage(`分析失败：${msg.result.error || '未知错误'}`, 'error');
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
// 普通日志
|
||||
}
|
||||
createSSEConnection(App.sessionId, {
|
||||
onDone: (result) => {
|
||||
handleDoneResult(result);
|
||||
},
|
||||
});
|
||||
|
||||
es.onerror = () => {
|
||||
es.close();
|
||||
App.isProcessing = false;
|
||||
App.processState = 'done';
|
||||
addChatMessage('连接中断', 'error');
|
||||
};
|
||||
}
|
||||
} catch (e) {
|
||||
addChatMessage(`请求失败：${e.message}`, 'error');
|
||||
|
||||
@@ -123,11 +123,14 @@ function _appendLLMStreamReasoning(text) {
|
||||
}
|
||||
|
||||
/**
|
||||
* 关闭流式气泡（end 阶段）— 气泡保留在聊天历史中作为持久消息
|
||||
* 关闭流式气泡（end 阶段）— 气泡保留在聊天历史中作为持久消息。
|
||||
* 如果累积内容是提取结果 JSON，则渲染为美观的摘要卡片。
|
||||
*/
|
||||
function _closeLLMStreamBubble(label) {
|
||||
if (!llmStreamState.bubble) return;
|
||||
|
||||
const accumulated = llmStreamState.accumulated;
|
||||
|
||||
llmStreamState.bubble.classList.remove('processing');
|
||||
llmStreamState.bubble.classList.add('done');
|
||||
|
||||
@@ -141,6 +144,16 @@ function _closeLLMStreamBubble(label) {
|
||||
reasoningSection.open = false;
|
||||
}
|
||||
|
||||
// 尝试将累积文本解析为提取结果 JSON，渲染为摘要卡片
|
||||
const textEl = llmStreamState.bubble.querySelector('.llm-stream-text');
|
||||
if (textEl && accumulated) {
|
||||
const parsed = _tryParseExtractionJson(accumulated);
|
||||
if (parsed) {
|
||||
textEl.innerHTML = '';
|
||||
textEl.appendChild(_buildExtractionSummary(parsed));
|
||||
}
|
||||
}
|
||||
|
||||
scrollToBottom();
|
||||
|
||||
llmStreamState = {
|
||||
@@ -197,3 +210,198 @@ function _errorLLMStreamBubble(errorMsg) {
|
||||
reasoningAccumulated: '',
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* 尝试将 LLM 输出文本解析为提取结果 JSON
|
||||
* 返回解析后的对象，或 null（非提取结果 JSON）
|
||||
*/
|
||||
function _tryParseExtractionJson(text) {
|
||||
try {
|
||||
const trimmed = text.trim();
|
||||
if (!trimmed.startsWith('{')) return null;
|
||||
const parsed = JSON.parse(trimmed);
|
||||
if (parsed && typeof parsed === 'object' && (parsed.basic_info || parsed.reimbursement_details)) {
|
||||
return parsed;
|
||||
}
|
||||
return null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 将提取结果 JSON 渲染为美观的摘要卡片 DOM 元素
|
||||
*
|
||||
* @param {Object} data - 提取结果 JSON
|
||||
* @returns {HTMLElement}
|
||||
*/
|
||||
function _buildExtractionSummary(data) {
|
||||
const container = document.createElement('div');
|
||||
container.className = 'extraction-summary';
|
||||
|
||||
// 基本信息
|
||||
if (data.basic_info) {
|
||||
const bi = data.basic_info;
|
||||
const section = _summarySection('基本信息');
|
||||
_summaryRow(section, '出差事由', bi.travel_purpose || '');
|
||||
_summaryRow(section, '出差地点', bi.travel_location || '');
|
||||
const dates = bi.start_date && bi.end_date ? `${bi.start_date} 至 ${bi.end_date}` : (bi.start_date || bi.end_date || '');
|
||||
_summaryRow(section, '出差日期', dates);
|
||||
container.appendChild(section);
|
||||
}
|
||||
|
||||
// 交通费
|
||||
if (data.reimbursement_details?.transport_fee?.length) {
|
||||
const section = _summarySection('交通费');
|
||||
const table = _summaryTable(['日期', '出发', '到达', '金额', '备注']);
|
||||
for (const item of data.reimbursement_details.transport_fee) {
|
||||
_summaryTableRow(table, [
|
||||
item.start_date || '',
|
||||
item.departure_place || '',
|
||||
item.arrival_place || '',
|
||||
item.amount != null ? `¥${item.amount.toFixed(2)}` : '',
|
||||
item.remark || '',
|
||||
]);
|
||||
}
|
||||
section.appendChild(table);
|
||||
container.appendChild(section);
|
||||
}
|
||||
|
||||
// 住宿费
|
||||
if (data.reimbursement_details?.hotel_fee?.length) {
|
||||
const section = _summarySection('住宿费');
|
||||
const table = _summaryTable(['入住日期', '离店日期', '天数', '发票金额', '报销金额', '备注']);
|
||||
for (const item of data.reimbursement_details.hotel_fee) {
|
||||
_summaryTableRow(table, [
|
||||
item.checkin_date || '',
|
||||
item.checkout_date || '',
|
||||
item.days != null ? String(item.days) : '',
|
||||
item.invoice_amount != null ? `¥${item.invoice_amount.toFixed(2)}` : '',
|
||||
item.reimburse_amount != null ? `¥${item.reimburse_amount.toFixed(2)}` : '',
|
||||
item.remark || '',
|
||||
]);
|
||||
}
|
||||
section.appendChild(table);
|
||||
container.appendChild(section);
|
||||
}
|
||||
|
||||
// 会议/培训费
|
||||
if (data.reimbursement_details?.conference_fee?.length) {
|
||||
const section = _summarySection('会议/培训费');
|
||||
const table = _summaryTable(['日期', '金额', '备注']);
|
||||
for (const item of data.reimbursement_details.conference_fee) {
|
||||
_summaryTableRow(table, [
|
||||
item.start_date || '',
|
||||
item.amount != null ? `¥${item.amount.toFixed(2)}` : '',
|
||||
item.remark || '',
|
||||
]);
|
||||
}
|
||||
section.appendChild(table);
|
||||
container.appendChild(section);
|
||||
}
|
||||
|
||||
// 支付记录
|
||||
if (data.payment_methods?.length) {
|
||||
const section = _summarySection('支付记录');
|
||||
const table = _summaryTable(['日期', '金额', '商户', '备注']);
|
||||
for (const item of data.payment_methods) {
|
||||
_summaryTableRow(table, [
|
||||
item.card_date || '',
|
||||
item.card_amount != null ? `¥${item.card_amount.toFixed(2)}` : '',
|
||||
item.merchant || '',
|
||||
item.remark || '',
|
||||
]);
|
||||
}
|
||||
section.appendChild(table);
|
||||
container.appendChild(section);
|
||||
}
|
||||
|
||||
// 补助
|
||||
if (data.subsidy_list?.length) {
|
||||
const section = _summarySection('补助');
|
||||
const table = _summaryTable(['姓名', '开始日期', '结束日期', '天数']);
|
||||
for (const item of data.subsidy_list) {
|
||||
_summaryTableRow(table, [
|
||||
item.person_name || '',
|
||||
item.start_date || '',
|
||||
item.end_date || '',
|
||||
item.days != null ? String(item.days) : '',
|
||||
]);
|
||||
}
|
||||
section.appendChild(table);
|
||||
container.appendChild(section);
|
||||
}
|
||||
|
||||
// 附件
|
||||
if (data.attachments?.length) {
|
||||
const section = _summarySection('附件');
|
||||
const list = document.createElement('ul');
|
||||
list.className = 'summary-attachment-list';
|
||||
for (const item of data.attachments) {
|
||||
const li = document.createElement('li');
|
||||
li.className = `summary-attachment-item summary-attachment-${item.attachment_type || 'other'}`;
|
||||
li.textContent = `${item.attachment_desc || item.filename || ''}`;
|
||||
list.appendChild(li);
|
||||
}
|
||||
section.appendChild(list);
|
||||
container.appendChild(section);
|
||||
}
|
||||
|
||||
return container;
|
||||
}
|
||||
|
||||
/* ---- 提取摘要卡片构建辅助函数 ---- */
|
||||
|
||||
function _summarySection(title) {
|
||||
const section = document.createElement('div');
|
||||
section.className = 'summary-section';
|
||||
const titleEl = document.createElement('div');
|
||||
titleEl.className = 'summary-section-title';
|
||||
titleEl.textContent = title;
|
||||
section.appendChild(titleEl);
|
||||
return section;
|
||||
}
|
||||
|
||||
function _summaryRow(section, label, value) {
|
||||
if (!value) return;
|
||||
const row = document.createElement('div');
|
||||
row.className = 'summary-row';
|
||||
const labelEl = document.createElement('span');
|
||||
labelEl.className = 'summary-label';
|
||||
labelEl.textContent = label;
|
||||
const valueEl = document.createElement('span');
|
||||
valueEl.className = 'summary-value';
|
||||
valueEl.textContent = value;
|
||||
row.appendChild(labelEl);
|
||||
row.appendChild(valueEl);
|
||||
section.appendChild(row);
|
||||
}
|
||||
|
||||
function _summaryTable(headers) {
|
||||
const table = document.createElement('table');
|
||||
table.className = 'summary-table';
|
||||
const thead = document.createElement('thead');
|
||||
const tr = document.createElement('tr');
|
||||
for (const h of headers) {
|
||||
const th = document.createElement('th');
|
||||
th.textContent = h;
|
||||
tr.appendChild(th);
|
||||
}
|
||||
thead.appendChild(tr);
|
||||
table.appendChild(thead);
|
||||
const tbody = document.createElement('tbody');
|
||||
table.appendChild(tbody);
|
||||
return table;
|
||||
}
|
||||
|
||||
function _summaryTableRow(table, cells) {
|
||||
const tbody = table.querySelector('tbody');
|
||||
if (!tbody) return;
|
||||
const tr = document.createElement('tr');
|
||||
for (const cell of cells) {
|
||||
const td = document.createElement('td');
|
||||
td.textContent = cell;
|
||||
tr.appendChild(td);
|
||||
}
|
||||
tbody.appendChild(tr);
|
||||
}
|
||||
@@ -39,10 +39,6 @@ export function checkAutoStart() {
|
||||
}
|
||||
}
|
||||
|
||||
if (App.processState === 'awaiting_files') {
|
||||
if (!App.forceStart) return;
|
||||
App.forceStart = false;
|
||||
}
|
||||
|
||||
if (App.processState === 'awaiting_supplement') {
|
||||
return;
|
||||
|
||||
@@ -6,15 +6,9 @@ import { ensureSession } from './utils.js';
|
||||
import {
|
||||
addTypingIndicator,
|
||||
removeTypingIndicator,
|
||||
setFileProcessing,
|
||||
setFileDone,
|
||||
setFileCached,
|
||||
setFileError,
|
||||
handleLLMStream,
|
||||
addChatMessage,
|
||||
showStatus,
|
||||
} from './chat.js';
|
||||
import { handleAgentEvent, handleAutoSubmit } from './agent.js';
|
||||
import { createSSEConnection, handleDoneResult } from './sse.js';
|
||||
|
||||
export async function startProcess() {
|
||||
if (!App.allFiles.length) {
|
||||
@@ -31,6 +25,8 @@ export async function startProcess() {
|
||||
await ensureSession();
|
||||
|
||||
for (const f of App.allFiles) {
|
||||
// 跳过服务器同步的文件（已在服务器上）
|
||||
if (f.__source === 'server') continue;
|
||||
const fd = new FormData();
|
||||
fd.append('file', f);
|
||||
await fetch(`/api/upload/${App.sessionId}`, { method: 'POST', body: fd });
|
||||
@@ -54,89 +50,13 @@ export async function startProcess() {
|
||||
body: JSON.stringify(cfg),
|
||||
});
|
||||
|
||||
const es = new EventSource(`/api/logs/${App.sessionId}`);
|
||||
App.agentEventSource = es;
|
||||
|
||||
es.addEventListener('message', e => {
|
||||
try {
|
||||
const msg = JSON.parse(e.data);
|
||||
|
||||
if (msg.type === 'file_progress') {
|
||||
const filename = msg.file;
|
||||
switch (msg.status) {
|
||||
case 'processing':
|
||||
setFileProcessing(filename);
|
||||
break;
|
||||
case 'done':
|
||||
setFileDone(filename, msg.summary || {});
|
||||
break;
|
||||
case 'cached':
|
||||
setFileCached(filename);
|
||||
break;
|
||||
case 'error':
|
||||
setFileError(filename, msg.error);
|
||||
break;
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (msg.type === 'llm_stream') {
|
||||
handleLLMStream(msg);
|
||||
return;
|
||||
}
|
||||
|
||||
if (msg.type && msg.type.startsWith('agent_')) {
|
||||
handleAgentEvent(msg);
|
||||
return;
|
||||
}
|
||||
|
||||
if (msg.type === 'done') {
|
||||
es.close();
|
||||
App.agentEventSource = null;
|
||||
removeTypingIndicator();
|
||||
const result = msg.result;
|
||||
App.isProcessing = false;
|
||||
|
||||
if (App.forceSubmitting) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (result.waiting_for_supplement) {
|
||||
App.processState = 'awaiting_supplement';
|
||||
showStatus('信息不完整，请补充上传相关材料');
|
||||
addChatMessage('您可通过上传区补充文件，或直接输入文字说明补充信息，或输入"直接提交"跳过校验', 'system');
|
||||
} else {
|
||||
App.processState = 'done';
|
||||
App.lastProcessedFileCount = App.allFiles.length;
|
||||
|
||||
if (result.ok) {
|
||||
if (result.agent_ready) {
|
||||
if (result.submit_ok !== false) {
|
||||
addChatMessage('信息完整，已自动提交到财务系统', 'done');
|
||||
} else {
|
||||
addChatMessage(`校验通过但提交失败：${result.submit_error || '未知错误'}`, 'error');
|
||||
}
|
||||
} else if (result.invoice_count) {
|
||||
addChatMessage(`处理完成！共提取 ${result.invoice_count} 张发票数据`, 'done');
|
||||
}
|
||||
} else {
|
||||
addChatMessage(`处理失败：${result.error || '未知错误'}`, 'error');
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
} catch (err) {
|
||||
// 普通日志行
|
||||
}
|
||||
const es = createSSEConnection(App.sessionId, {
|
||||
onDone: (result) => {
|
||||
removeTypingIndicator();
|
||||
handleDoneResult(result);
|
||||
},
|
||||
});
|
||||
|
||||
es.onerror = () => {
|
||||
es.close();
|
||||
removeTypingIndicator();
|
||||
App.isProcessing = false;
|
||||
App.processState = 'done';
|
||||
addChatMessage('连接中断，请刷新页面重试', 'error');
|
||||
};
|
||||
App.agentEventSource = es;
|
||||
|
||||
} catch (e) {
|
||||
removeTypingIndicator();
|
||||
|
||||
144
src/web/static/js/sse.js
Normal file
144
src/web/static/js/sse.js
Normal file
@@ -0,0 +1,144 @@
|
||||
/**
|
||||
* SSE 连接管理
|
||||
*
|
||||
* 封装 EventSource 的创建、事件注册和 done 事件处理,避免重复代码。
|
||||
*/
|
||||
import { App } from './state.js';
|
||||
import {
|
||||
removeTypingIndicator,
|
||||
setFileProcessing,
|
||||
setFileDone,
|
||||
setFileCached,
|
||||
setFileError,
|
||||
handleLLMStream,
|
||||
addChatMessage,
|
||||
showStatus,
|
||||
} from './chat.js';
|
||||
import { handleAgentEvent } from './agent.js';
|
||||
|
||||
/**
|
||||
* 创建 SSE 连接并返回 EventSource 实例
|
||||
*
|
||||
* @param {string} sessionId - 会话 ID
|
||||
* @param {Object} options - 选项配置
|
||||
* @param {Function} options.onDone - done 事件处理器
|
||||
* @param {boolean} options.handleFileProgress - 是否处理文件进度事件 (默认 true)
|
||||
* @param {boolean} options.handleLLMStream - 是否处理 LLM 流式事件 (默认 true)
|
||||
* @param {boolean} options.handleAgentEvents - 是否处理 Agent 事件 (默认 true)
|
||||
* @returns {EventSource}
|
||||
*/
|
||||
export function createSSEConnection(sessionId, options = {}) {
|
||||
const {
|
||||
onDone,
|
||||
handleFileProgress = true,
|
||||
handleLLMStream: handleStream = true,
|
||||
handleAgentEvents = true,
|
||||
} = options;
|
||||
|
||||
const es = new EventSource(`/api/logs/${sessionId}`);
|
||||
|
||||
es.addEventListener('message', e => {
|
||||
try {
|
||||
const msg = JSON.parse(e.data);
|
||||
|
||||
// 文件进度事件
|
||||
if (handleFileProgress && msg.type === 'file_progress') {
|
||||
const filename = msg.file;
|
||||
switch (msg.status) {
|
||||
case 'processing':
|
||||
setFileProcessing(filename);
|
||||
break;
|
||||
case 'done':
|
||||
setFileDone(filename, msg.summary || {});
|
||||
break;
|
||||
case 'cached':
|
||||
setFileCached(filename);
|
||||
break;
|
||||
case 'error':
|
||||
setFileError(filename, msg.error);
|
||||
break;
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// LLM 流式事件
|
||||
if (handleStream && msg.type === 'llm_stream') {
|
||||
handleLLMStream(msg);
|
||||
return;
|
||||
}
|
||||
|
||||
// Agent 事件
|
||||
if (handleAgentEvents && msg.type && msg.type.startsWith('agent_')) {
|
||||
handleAgentEvent(msg);
|
||||
return;
|
||||
}
|
||||
|
||||
// 完成事件
|
||||
if (msg.type === 'done') {
|
||||
es.close();
|
||||
if (onDone) {
|
||||
onDone(msg.result);
|
||||
}
|
||||
return;
|
||||
}
|
||||
} catch (err) {
|
||||
// 普通日志行,忽略
|
||||
}
|
||||
});
|
||||
|
||||
es.onerror = () => {
|
||||
es.close();
|
||||
removeTypingIndicator();
|
||||
App.isProcessing = false;
|
||||
App.processState = 'done';
|
||||
addChatMessage('连接中断,请刷新页面重试', 'error');
|
||||
};
|
||||
|
||||
return es;
|
||||
}
|
||||
|
||||
/**
|
||||
* 处理 done 事件的通用逻辑 (根据 result 内容更新 UI 状态)
|
||||
*
|
||||
* @param {Object} result - done 事件的结果对象
|
||||
*/
|
||||
export function handleDoneResult(result) {
|
||||
App.isProcessing = false;
|
||||
|
||||
if (App.forceSubmitting) {
|
||||
App.forceSubmitting = false;
|
||||
App.lastProcessedFileCount = App.allFiles.length;
|
||||
if (result.ok) {
|
||||
addChatMessage('提交完成！', 'done');
|
||||
} else {
|
||||
addChatMessage(`提交失败：${result.error || '未知错误'}`, 'error');
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (result.waiting_for_supplement) {
|
||||
App.processState = 'awaiting_supplement';
|
||||
showStatus('信息不完整,请补充上传相关材料');
|
||||
addChatMessage(
|
||||
'您可通过上传区补充文件,或直接输入文字说明补充信息,或输入"直接提交"跳过校验',
|
||||
'system'
|
||||
);
|
||||
} else {
|
||||
App.processState = 'done';
|
||||
App.lastProcessedFileCount = App.allFiles.length;
|
||||
|
||||
if (result.ok) {
|
||||
if (result.agent_ready) {
|
||||
if (result.submit_ok !== false) {
|
||||
addChatMessage('信息完整,已自动提交到财务系统', 'done');
|
||||
} else {
|
||||
addChatMessage(`校验通过但提交失败：${result.submit_error || '未知错误'}`, 'error');
|
||||
}
|
||||
} else if (result.invoice_count) {
|
||||
addChatMessage(`处理完成！共提取 ${result.invoice_count} 张发票数据`, 'done');
|
||||
}
|
||||
} else {
|
||||
addChatMessage(`处理失败：${result.error || '未知错误'}`, 'error');
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -40,12 +40,10 @@ async function syncFiles() {
|
||||
const localNames = new Set(App.allFiles.map(f => f.name));
|
||||
const addedFiles = [];
|
||||
|
||||
// 只同步文件名元数据，不下载文件内容（文件已在服务器上）
|
||||
for (const serverFile of serverFiles) {
|
||||
if (!localNames.has(serverFile.name)) {
|
||||
const resp = await fetch(`/api/download/${App.sessionId}/${encodeURIComponent(serverFile.name)}`);
|
||||
const blob = await resp.blob();
|
||||
const file = new File([blob], serverFile.name, { type: blob.type });
|
||||
file.__source = 'server';
|
||||
const file = { name: serverFile.name, __source: 'server' };
|
||||
App.allFiles.push(file);
|
||||
addedFiles.push(serverFile.name);
|
||||
}
|
||||
|
||||
@@ -9,6 +9,29 @@ import { ensureSession } from './utils.js';
|
||||
|
||||
export async function handleFiles(input) {
|
||||
const files = Array.from(input.files);
|
||||
const result = await _processFiles(files);
|
||||
|
||||
if (result.hasNewFiles && App.processState === 'done') {
|
||||
App.processState = 'idle';
|
||||
App.newFilenames = result.newFilenames;
|
||||
}
|
||||
if (result.hasNewFiles && App.processState === 'awaiting_supplement') {
|
||||
hideAgentRequest();
|
||||
await _uploadNewFiles(result.newFilenames);
|
||||
App.newFilenames = result.newFilenames;
|
||||
App.processState = 'idle';
|
||||
checkAutoStart();
|
||||
return;
|
||||
}
|
||||
|
||||
promptMissingConfig();
|
||||
input.value = '';
|
||||
}
|
||||
|
||||
/**
|
||||
* 统一文件处理逻辑（handleFiles 和拖拽共享）
|
||||
*/
|
||||
async function _processFiles(files) {
|
||||
const validExts = /\.(pdf|png|jpe?g|bmp|webp)$/i;
|
||||
let hasNewFiles = false;
|
||||
const newFilenames = [];
|
||||
@@ -28,21 +51,7 @@ export async function handleFiles(input) {
|
||||
}
|
||||
}
|
||||
|
||||
if (hasNewFiles && App.processState === 'done') {
|
||||
App.processState = 'idle';
|
||||
App.newFilenames = newFilenames;
|
||||
}
|
||||
if (hasNewFiles && App.processState === 'awaiting_supplement') {
|
||||
hideAgentRequest();
|
||||
await _uploadNewFiles(newFilenames);
|
||||
App.newFilenames = newFilenames;
|
||||
App.processState = 'idle';
|
||||
checkAutoStart();
|
||||
return;
|
||||
}
|
||||
|
||||
promptMissingConfig();
|
||||
input.value = '';
|
||||
return { hasNewFiles, newFilenames };
|
||||
}
|
||||
|
||||
async function _uploadNewFiles(filenames) {
|
||||
@@ -90,35 +99,18 @@ export function initDragDrop() {
|
||||
zone.addEventListener('drop', async e => {
|
||||
e.preventDefault();
|
||||
zone.classList.remove('dragover');
|
||||
const validExts = /\.(pdf|png|jpe?g|bmp|webp)$/i;
|
||||
const droppedFiles = Array.from(e.dataTransfer.files);
|
||||
let hasNewFiles = false;
|
||||
const newFilenames = [];
|
||||
const result = await _processFiles(droppedFiles);
|
||||
|
||||
for (const f of droppedFiles) {
|
||||
if (/\.json$/i.test(f.name)) {
|
||||
await parseConfigFile(f);
|
||||
continue;
|
||||
}
|
||||
if (validExts.test(f.name) && !App.allFiles.find(x => x.name === f.name)) {
|
||||
f.__source = 'local';
|
||||
App.allFiles.push(f);
|
||||
zone.classList.add('active');
|
||||
addFileMessage(f.name);
|
||||
hasNewFiles = true;
|
||||
newFilenames.push(f.name);
|
||||
}
|
||||
}
|
||||
|
||||
if (hasNewFiles && App.processState === 'done') {
|
||||
if (result.hasNewFiles && App.processState === 'done') {
|
||||
App.processState = 'idle';
|
||||
}
|
||||
|
||||
// 在 awaiting_supplement 状态下，只上传新文件并走 supplement 流程
|
||||
if (hasNewFiles && App.processState === 'awaiting_supplement') {
|
||||
if (result.hasNewFiles && App.processState === 'awaiting_supplement') {
|
||||
hideAgentRequest();
|
||||
await _uploadNewFiles(newFilenames);
|
||||
App.newFilenames = newFilenames;
|
||||
await _uploadNewFiles(result.newFilenames);
|
||||
App.newFilenames = result.newFilenames;
|
||||
App.processState = 'idle';
|
||||
checkAutoStart();
|
||||
return;
|
||||
|
||||
@@ -12,7 +12,7 @@ from typing import Any
|
||||
import pytest
|
||||
|
||||
from src import exceptions
|
||||
from src.doc.extractor import extract_invoices
|
||||
from src.core.extraction import extract_invoices
|
||||
|
||||
# 字段键名（与源码中的字符串字面量保持一致）
|
||||
K_INVOICE_TYPE = "invoice_type"
|
||||
@@ -78,7 +78,7 @@ class TestExtractInvoices:
|
||||
"""extract_invoices 编排函数"""
|
||||
|
||||
def test_empty_directory(self, tmp_path: Path, monkeypatch):
|
||||
monkeypatch.setattr("src.doc.extractor._find_all_files", lambda d: [])
|
||||
monkeypatch.setattr("src.core.extraction.extractor._find_all_files", lambda d: [])
|
||||
|
||||
records, apps, groups = extract_invoices(str(tmp_path))
|
||||
assert records == []
|
||||
@@ -89,8 +89,8 @@ class TestExtractInvoices:
|
||||
pdf = tmp_path / "broken.pdf"
|
||||
pdf.touch()
|
||||
|
||||
monkeypatch.setattr("src.doc.extractor._find_all_files", lambda d: [pdf])
|
||||
monkeypatch.setattr("src.doc.extractor._extract_document", lambda p, c, s: (None, "parse error"))
|
||||
monkeypatch.setattr("src.core.extraction.extractor._find_all_files", lambda d: [pdf])
|
||||
monkeypatch.setattr("src.core.extraction.extractor._extract_document", lambda p, c, s: (None, "parse error"))
|
||||
|
||||
with pytest.raises(exceptions.ExtractionError) as exc_info:
|
||||
extract_invoices(str(tmp_path))
|
||||
@@ -107,16 +107,14 @@ class TestExtractInvoices:
|
||||
inv1 = _make_invoice("INV001", 300.0, INVOICE_TYPE_GENERAL)
|
||||
inv2 = _make_invoice("INV002", 200.0, INVOICE_TYPE_GENERAL)
|
||||
|
||||
monkeypatch.setattr("src.doc.extractor._find_all_files", lambda d: [pdf1, pdf2])
|
||||
monkeypatch.setattr("src.core.extraction.extractor._find_all_files", lambda d: [pdf1, pdf2])
|
||||
|
||||
call_index = [0]
|
||||
path_to_result = {pdf1: inv1, pdf2: inv2}
|
||||
|
||||
def fake_extract(path, cache_dir, source_dir):
|
||||
idx = call_index[0]
|
||||
call_index[0] += 1
|
||||
return (inv1, None) if idx == 0 else (inv2, None)
|
||||
return (path_to_result[path], None)
|
||||
|
||||
monkeypatch.setattr("src.doc.extractor._extract_document", fake_extract)
|
||||
monkeypatch.setattr("src.core.extraction.extractor._extract_document", fake_extract)
|
||||
|
||||
def fake_match(invoices, cards):
|
||||
return [
|
||||
@@ -131,7 +129,7 @@ class TestExtractInvoices:
|
||||
}
|
||||
]
|
||||
|
||||
monkeypatch.setattr("src.doc.extractor.match_invoices_to_cards", fake_match)
|
||||
monkeypatch.setattr("src.core.matching.matcher.match_invoices_to_cards", fake_match)
|
||||
|
||||
records, apps, groups = extract_invoices(str(tmp_path))
|
||||
|
||||
@@ -153,19 +151,17 @@ class TestExtractInvoices:
|
||||
inv_general = _make_invoice("GEN001", 150.0, INVOICE_TYPE_GENERAL)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"src.doc.extractor._find_all_files",
|
||||
"src.core.extraction.extractor._find_all_files",
|
||||
lambda d: [pdf1, pdf2, pdf3],
|
||||
)
|
||||
|
||||
invoices_list = [inv_train, inv_hotel, inv_general]
|
||||
call_index = [0]
|
||||
path_to_result = dict(zip([pdf1, pdf2, pdf3], invoices_list, strict=True))
|
||||
|
||||
def fake_extract(path, cache_dir, source_dir):
|
||||
idx = call_index[0]
|
||||
call_index[0] += 1
|
||||
return (invoices_list[idx], None)
|
||||
return (path_to_result[path], None)
|
||||
|
||||
monkeypatch.setattr("src.doc.extractor._extract_document", fake_extract)
|
||||
monkeypatch.setattr("src.core.extraction.extractor._extract_document", fake_extract)
|
||||
|
||||
def fake_match(invoices, cards):
|
||||
return [
|
||||
@@ -198,7 +194,7 @@ class TestExtractInvoices:
|
||||
},
|
||||
]
|
||||
|
||||
monkeypatch.setattr("src.doc.extractor.match_invoices_to_cards", fake_match)
|
||||
monkeypatch.setattr("src.core.matching.matcher.match_invoices_to_cards", fake_match)
|
||||
|
||||
records, apps, groups = extract_invoices(str(tmp_path))
|
||||
|
||||
@@ -221,19 +217,16 @@ class TestExtractInvoices:
|
||||
app = _make_application()
|
||||
|
||||
monkeypatch.setattr(
|
||||
"src.doc.extractor._find_all_files",
|
||||
"src.core.extraction.extractor._find_all_files",
|
||||
lambda d: [pdf1, pdf2],
|
||||
)
|
||||
|
||||
results = [inv, app]
|
||||
call_index = [0]
|
||||
path_to_result = {pdf1: inv, pdf2: app}
|
||||
|
||||
def fake_extract(path, cache_dir, source_dir):
|
||||
idx = call_index[0]
|
||||
call_index[0] += 1
|
||||
return (results[idx], None)
|
||||
return (path_to_result[path], None)
|
||||
|
||||
monkeypatch.setattr("src.doc.extractor._extract_document", fake_extract)
|
||||
monkeypatch.setattr("src.core.extraction.extractor._extract_document", fake_extract)
|
||||
|
||||
def fake_match(invoices, cards):
|
||||
return [
|
||||
@@ -248,7 +241,7 @@ class TestExtractInvoices:
|
||||
}
|
||||
]
|
||||
|
||||
monkeypatch.setattr("src.doc.extractor.match_invoices_to_cards", fake_match)
|
||||
monkeypatch.setattr("src.core.matching.matcher.match_invoices_to_cards", fake_match)
|
||||
|
||||
records, apps, groups = extract_invoices(str(tmp_path))
|
||||
|
||||
@@ -266,19 +259,16 @@ class TestExtractInvoices:
|
||||
card = _make_card(300.0)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"src.doc.extractor._find_all_files",
|
||||
"src.core.extraction.extractor._find_all_files",
|
||||
lambda d: [pdf1, pdf2],
|
||||
)
|
||||
|
||||
results = [inv, card]
|
||||
call_index = [0]
|
||||
path_to_result = {pdf1: inv, pdf2: card}
|
||||
|
||||
def fake_extract(path, cache_dir, source_dir):
|
||||
idx = call_index[0]
|
||||
call_index[0] += 1
|
||||
return (results[idx], None)
|
||||
return (path_to_result[path], None)
|
||||
|
||||
monkeypatch.setattr("src.doc.extractor._extract_document", fake_extract)
|
||||
monkeypatch.setattr("src.core.extraction.extractor._extract_document", fake_extract)
|
||||
|
||||
def fake_match(invoices, cards):
|
||||
return [
|
||||
@@ -293,7 +283,7 @@ class TestExtractInvoices:
|
||||
}
|
||||
]
|
||||
|
||||
monkeypatch.setattr("src.doc.extractor.match_invoices_to_cards", fake_match)
|
||||
monkeypatch.setattr("src.core.matching.matcher.match_invoices_to_cards", fake_match)
|
||||
|
||||
records, apps, groups = extract_invoices(str(tmp_path))
|
||||
|
||||
@@ -309,19 +299,17 @@ class TestExtractInvoices:
|
||||
inv = _make_invoice("INV001", 300.0, INVOICE_TYPE_GENERAL)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"src.doc.extractor._find_all_files",
|
||||
"src.core.extraction.extractor._find_all_files",
|
||||
lambda d: [pdf1, pdf2],
|
||||
)
|
||||
|
||||
results = [inv, None]
|
||||
call_index = [0]
|
||||
path_to_result = {pdf1: inv, pdf2: None}
|
||||
|
||||
def fake_extract(path, cache_dir, source_dir):
|
||||
idx = call_index[0]
|
||||
call_index[0] += 1
|
||||
return (results[idx], "parse error") if results[idx] is None else (results[idx], None)
|
||||
result = path_to_result[path]
|
||||
return (result, "parse error") if result is None else (result, None)
|
||||
|
||||
monkeypatch.setattr("src.doc.extractor._extract_document", fake_extract)
|
||||
monkeypatch.setattr("src.core.extraction.extractor._extract_document", fake_extract)
|
||||
|
||||
def fake_match(invoices, cards):
|
||||
return [
|
||||
@@ -336,7 +324,7 @@ class TestExtractInvoices:
|
||||
}
|
||||
]
|
||||
|
||||
monkeypatch.setattr("src.doc.extractor.match_invoices_to_cards", fake_match)
|
||||
monkeypatch.setattr("src.core.matching.matcher.match_invoices_to_cards", fake_match)
|
||||
|
||||
records, apps, groups = extract_invoices(str(tmp_path))
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
覆盖发票分类、CSV 列定义、常量校验。
|
||||
"""
|
||||
|
||||
from src.doc.invoice import (
|
||||
from src.infra.documents import (
|
||||
INVOICE_LEVEL_COLUMNS,
|
||||
PAYMENT_RECORD_COLUMNS,
|
||||
classify_invoice_batch,
|
||||
|
||||
@@ -14,7 +14,7 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from src.doc.llm_extractor import (
|
||||
from src.core.extraction import (
|
||||
_image_to_base64,
|
||||
extract_document,
|
||||
parse_json_response,
|
||||
@@ -131,7 +131,7 @@ class TestExtractDocument:
|
||||
def fake_query(system_prompt, text, image_b64, max_tokens=4096):
|
||||
return response_text
|
||||
|
||||
monkeypatch.setattr("src.doc.llm_extractor._llm_query_multimodal", fake_query)
|
||||
monkeypatch.setattr("src.core.extraction.llm_extractor._llm_query_multimodal", fake_query)
|
||||
|
||||
def test_success(self, tmp_path: Path, monkeypatch):
|
||||
img_path = tmp_path / "card.png"
|
||||
@@ -156,7 +156,7 @@ class TestExtractDocument:
|
||||
def fake_query(system_prompt, text, image_b64, max_tokens=4096):
|
||||
raise RuntimeError("模型不可用")
|
||||
|
||||
monkeypatch.setattr("src.doc.llm_extractor._llm_query_multimodal", fake_query)
|
||||
monkeypatch.setattr("src.core.extraction.llm_extractor._llm_query_multimodal", fake_query)
|
||||
with pytest.raises(RuntimeError, match="模型不可用"):
|
||||
extract_document(img_path)
|
||||
|
||||
@@ -193,7 +193,7 @@ class TestExtractDocument:
|
||||
received_b64 = image_b64s
|
||||
return json.dumps({K_CARD_DATE: "2026-01-01", K_CARD_NO: "0000", K_CARD_AMOUNT: "100"})
|
||||
|
||||
monkeypatch.setattr("src.doc.llm_extractor._llm_query_multimodal", capture_b64)
|
||||
monkeypatch.setattr("src.core.extraction.llm_extractor._llm_query_multimodal", capture_b64)
|
||||
extract_document(img_path)
|
||||
|
||||
assert received_b64 is not None
|
||||
|
||||
@@ -12,11 +12,12 @@ from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from src.doc.matcher import (
|
||||
from src.core.matching import (
|
||||
_build_invoice_summary,
|
||||
_build_payment_records,
|
||||
_invoices_to_records,
|
||||
_match,
|
||||
_match_by_filename,
|
||||
_match_one_to_many,
|
||||
_match_one_to_one,
|
||||
_relative_tolerance,
|
||||
@@ -562,3 +563,304 @@ class TestMatchInvoicesToCards:
|
||||
result = match_invoices_to_cards(invoices, cards=cards)
|
||||
|
||||
assert len(result) == 2
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 文件名匹配
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestMatchByFilename:
|
||||
"""文件名匹配：发票和刷卡记录的文件名（不含后缀）一致时直接匹配"""
|
||||
|
||||
def _make_invoice_with_file(self, number: str, amount: float, filename: str) -> dict[str, Any]:
|
||||
inv = _make_invoice(number, amount)
|
||||
inv["_source_file"] = filename
|
||||
return inv
|
||||
|
||||
def _make_card_with_file(self, date: str, amount: float, filename: str) -> dict[str, Any]:
|
||||
card = _make_card(date, amount)
|
||||
card["_source_file"] = filename
|
||||
return card
|
||||
|
||||
def test_exact_filename_match(self):
|
||||
invoices = [
|
||||
self._make_invoice_with_file("A", 500, "receipt_001.pdf"),
|
||||
self._make_invoice_with_file("B", 300, "receipt_002.pdf"),
|
||||
]
|
||||
cards = [
|
||||
self._make_card_with_file("2026-01-01", 500, "receipt_001.png"),
|
||||
self._make_card_with_file("2026-01-02", 300, "receipt_002.png"),
|
||||
]
|
||||
for inv in invoices:
|
||||
inv["_amount"] = _safe_float(inv[K_TOTAL_AMOUNT])
|
||||
for card in cards:
|
||||
card["_amount"] = _safe_float(card[K_CARD_AMOUNT])
|
||||
|
||||
assigned: set[int] = set()
|
||||
result: dict[int, list[int]] = {}
|
||||
_match_by_filename(invoices, cards, assigned, result)
|
||||
|
||||
assert 0 in result
|
||||
assert 1 in result
|
||||
assert result[0] == [0]
|
||||
assert result[1] == [1]
|
||||
assert 0 in assigned
|
||||
assert 1 in assigned
|
||||
|
||||
def test_filename_match_ignores_extension(self):
|
||||
invoices = [self._make_invoice_with_file("A", 500, "data.pdf")]
|
||||
cards = [self._make_card_with_file("2026-01-01", 500, "data.png")]
|
||||
for inv in invoices:
|
||||
inv["_amount"] = _safe_float(inv[K_TOTAL_AMOUNT])
|
||||
for card in cards:
|
||||
card["_amount"] = _safe_float(card[K_CARD_AMOUNT])
|
||||
|
||||
assigned: set[int] = set()
|
||||
result: dict[int, list[int]] = {}
|
||||
_match_by_filename(invoices, cards, assigned, result)
|
||||
|
||||
assert 0 in result
|
||||
assert result[0] == [0]
|
||||
|
||||
def test_filename_no_match_when_stems_differ(self):
|
||||
invoices = [self._make_invoice_with_file("A", 500, "invoice_A.pdf")]
|
||||
cards = [self._make_card_with_file("2026-01-01", 500, "card_001.png")]
|
||||
for inv in invoices:
|
||||
inv["_amount"] = _safe_float(inv[K_TOTAL_AMOUNT])
|
||||
for card in cards:
|
||||
card["_amount"] = _safe_float(card[K_CARD_AMOUNT])
|
||||
|
||||
assigned: set[int] = set()
|
||||
result: dict[int, list[int]] = {}
|
||||
_match_by_filename(invoices, cards, assigned, result)
|
||||
|
||||
assert 0 not in result
|
||||
assert len(assigned) == 0
|
||||
|
||||
def test_skips_invoice_without_source_file(self):
|
||||
invoices = [_make_invoice("A", 500)]
|
||||
cards = [self._make_card_with_file("2026-01-01", 500, "card.png")]
|
||||
for inv in invoices:
|
||||
inv["_amount"] = _safe_float(inv[K_TOTAL_AMOUNT])
|
||||
for card in cards:
|
||||
card["_amount"] = _safe_float(card[K_CARD_AMOUNT])
|
||||
|
||||
assigned: set[int] = set()
|
||||
result: dict[int, list[int]] = {}
|
||||
_match_by_filename(invoices, cards, assigned, result)
|
||||
|
||||
assert 0 not in result
|
||||
|
||||
def test_skips_card_without_source_file(self):
|
||||
invoices = [self._make_invoice_with_file("A", 500, "inv.pdf")]
|
||||
cards = [_make_card("2026-01-01", 500)]
|
||||
for inv in invoices:
|
||||
inv["_amount"] = _safe_float(inv[K_TOTAL_AMOUNT])
|
||||
for card in cards:
|
||||
card["_amount"] = _safe_float(card[K_CARD_AMOUNT])
|
||||
|
||||
assigned: set[int] = set()
|
||||
result: dict[int, list[int]] = {}
|
||||
_match_by_filename(invoices, cards, assigned, result)
|
||||
|
||||
assert 0 not in result
|
||||
|
||||
def test_skips_empty_source_file(self):
|
||||
inv = _make_invoice("A", 500)
|
||||
inv["_source_file"] = ""
|
||||
cards = [self._make_card_with_file("2026-01-01", 500, "card.png")]
|
||||
for _inv in [inv]:
|
||||
_inv["_amount"] = _safe_float(_inv[K_TOTAL_AMOUNT])
|
||||
for card in cards:
|
||||
card["_amount"] = _safe_float(card[K_CARD_AMOUNT])
|
||||
|
||||
assigned: set[int] = set()
|
||||
result: dict[int, list[int]] = {}
|
||||
_match_by_filename([inv], cards, assigned, result)
|
||||
|
||||
assert 0 not in result
|
||||
|
||||
def test_only_first_unassigned_invoice_matches(self):
|
||||
invoices = [
|
||||
self._make_invoice_with_file("A", 500, "same.pdf"),
|
||||
self._make_invoice_with_file("B", 300, "same.pdf"),
|
||||
]
|
||||
cards = [self._make_card_with_file("2026-01-01", 500, "same.png")]
|
||||
for inv in invoices:
|
||||
inv["_amount"] = _safe_float(inv[K_TOTAL_AMOUNT])
|
||||
for card in cards:
|
||||
card["_amount"] = _safe_float(card[K_CARD_AMOUNT])
|
||||
|
||||
assigned: set[int] = set()
|
||||
result: dict[int, list[int]] = {}
|
||||
_match_by_filename(invoices, cards, assigned, result)
|
||||
|
||||
assert 0 in result
|
||||
assert result[0] == [0]
|
||||
assert 0 in assigned
|
||||
assert 1 not in assigned
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 文件名匹配与金额匹配的交互
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestFilenameAndAmountInteraction:
|
||||
"""文件名预匹配后，已分配的发票不会被后续金额匹配重复处理"""
|
||||
|
||||
def _prepare_with_files(self, invoices, cards):
|
||||
for inv in invoices:
|
||||
inv["_amount"] = _safe_float(inv[K_TOTAL_AMOUNT])
|
||||
for card in cards:
|
||||
card["_amount"] = _safe_float(card[K_CARD_AMOUNT])
|
||||
invoices.sort(key=lambda i: i["_amount"], reverse=True)
|
||||
cards.sort(key=lambda c: c["_amount"], reverse=True)
|
||||
|
||||
def _make_invoice_with_file(self, number: str, amount: float, filename: str) -> dict[str, Any]:
|
||||
inv = _make_invoice(number, amount)
|
||||
inv["_source_file"] = filename
|
||||
return inv
|
||||
|
||||
def _make_card_with_file(self, date: str, amount: float, filename: str) -> dict[str, Any]:
|
||||
card = _make_card(date, amount)
|
||||
card["_source_file"] = filename
|
||||
return card
|
||||
|
||||
def test_filename_matched_invoice_skipped_in_one_to_one(self):
|
||||
"""文件名预匹配后，_match_one_to_one 应跳过已分配的发票"""
|
||||
invoices = [
|
||||
self._make_invoice_with_file("A", 500, "match.pdf"),
|
||||
self._make_invoice_with_file("B", 300, "other.pdf"),
|
||||
]
|
||||
cards = [
|
||||
self._make_card_with_file("2026-01-01", 500, "match.png"),
|
||||
self._make_card_with_file("2026-01-02", 300, "other.png"),
|
||||
]
|
||||
self._prepare_with_files(invoices, cards)
|
||||
|
||||
assigned: set[int] = set()
|
||||
result: dict[int, list[int]] = {}
|
||||
_match_by_filename(invoices, cards, assigned, result)
|
||||
_match_one_to_one(invoices, cards, 0.03, assigned, result)
|
||||
|
||||
assert 0 in result
|
||||
assert result[0] == [0]
|
||||
assert len(assigned) == 2
|
||||
|
||||
def test_filename_match_takes_priority_over_amount(self):
|
||||
"""即使金额不匹配，文件名匹配仍优先"""
|
||||
invoices = [self._make_invoice_with_file("A", 1000, "same.pdf")]
|
||||
cards = [self._make_card_with_file("2026-01-01", 500, "same.png")]
|
||||
self._prepare_with_files(invoices, cards)
|
||||
|
||||
assigned: set[int] = set()
|
||||
result: dict[int, list[int]] = {}
|
||||
_match_by_filename(invoices, cards, assigned, result)
|
||||
|
||||
assert 0 in result
|
||||
assert result[0] == [0]
|
||||
assert 0 in assigned
|
||||
|
||||
def test_partial_filename_match_falls_back_to_amount(self):
|
||||
"""部分文件名匹配后，剩余发票走金额匹配"""
|
||||
invoices = [
|
||||
self._make_invoice_with_file("A", 500, "match.pdf"),
|
||||
self._make_invoice_with_file("B", 300, "no_match.pdf"),
|
||||
]
|
||||
cards = [
|
||||
self._make_card_with_file("2026-01-01", 500, "match.png"),
|
||||
self._make_card_with_file("2026-01-02", 300, "different.png"),
|
||||
]
|
||||
self._prepare_with_files(invoices, cards)
|
||||
|
||||
result = _match(cards, invoices, 0.03)
|
||||
|
||||
assert 0 in result
|
||||
assert 1 in result
|
||||
assert len(result) == 2
|
||||
|
||||
def test_filename_match_in_one_to_many_prevents_reuse(self):
|
||||
"""一对多场景下，文件名匹配的发票不会被贪心匹配复用"""
|
||||
invoices = [
|
||||
self._make_invoice_with_file("A", 500, "same.pdf"),
|
||||
self._make_invoice_with_file("B", 200, "other.pdf"),
|
||||
]
|
||||
cards = [self._make_card_with_file("2026-01-01", 500, "same.png")]
|
||||
self._prepare_with_files(invoices, cards)
|
||||
|
||||
result = _match(cards, invoices, 0.03)
|
||||
|
||||
assert 0 in result
|
||||
assert result[0] == [0]
|
||||
assert len(result[0]) == 1
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 端到端：文件名匹配集成
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestEndToEndFilenameMatching:
|
||||
"""match_invoices_to_cards 端到端测试（含文件名匹配）"""
|
||||
|
||||
def _make_invoice_with_file(self, number: str, amount: float, filename: str) -> dict[str, Any]:
|
||||
inv = _make_invoice(number, amount)
|
||||
inv["_source_file"] = filename
|
||||
return inv
|
||||
|
||||
def _make_card_with_file(self, date: str, amount: float, filename: str) -> dict[str, Any]:
|
||||
card = _make_card(date, amount)
|
||||
card["_source_file"] = filename
|
||||
return card
|
||||
|
||||
def test_full_filename_match(self):
|
||||
invoices = [
|
||||
self._make_invoice_with_file("A", 500, "receipt_001.pdf"),
|
||||
self._make_invoice_with_file("B", 300, "receipt_002.pdf"),
|
||||
]
|
||||
cards = [
|
||||
self._make_card_with_file("2026-01-01", 500, "receipt_001.png"),
|
||||
self._make_card_with_file("2026-01-02", 300, "receipt_002.png"),
|
||||
]
|
||||
result = match_invoices_to_cards(invoices, cards=cards)
|
||||
|
||||
assert len(result) == 2
|
||||
for rec in result:
|
||||
assert rec[K_RELATIVE_INVOICE_COUNT] == "1"
|
||||
|
||||
def test_mixed_filename_and_amount_match(self):
|
||||
"""部分文件名匹配 + 部分金额匹配"""
|
||||
invoices = [
|
||||
self._make_invoice_with_file("A", 500, "match.pdf"),
|
||||
self._make_invoice_with_file("B", 300, "no_match.pdf"),
|
||||
]
|
||||
cards = [
|
||||
self._make_card_with_file("2026-01-01", 500, "match.png"),
|
||||
self._make_card_with_file("2026-01-02", 300, "different.png"),
|
||||
]
|
||||
result = match_invoices_to_cards(invoices, cards=cards)
|
||||
|
||||
assert len(result) == 2
|
||||
|
||||
def test_filename_match_with_no_cards_has_files(self):
|
||||
"""无支付记录时，文件名信息不影响结果"""
|
||||
invoices = [
|
||||
self._make_invoice_with_file("A", 100, "inv.pdf"),
|
||||
self._make_invoice_with_file("B", 200, "inv2.pdf"),
|
||||
]
|
||||
result = match_invoices_to_cards(invoices, cards=None)
|
||||
|
||||
assert len(result) == 2
|
||||
|
||||
def test_internal_fields_cleaned_after_filename_match(self):
|
||||
"""文件名匹配后，内部字段仍被清理"""
|
||||
invoices = [self._make_invoice_with_file("A", 500, "same.pdf")]
|
||||
cards = [self._make_card_with_file("2026-01-01", 500, "same.png")]
|
||||
result = match_invoices_to_cards(invoices, cards=cards)
|
||||
|
||||
for inv in result[0][K_MATCHED_INVOICES]:
|
||||
assert "_amount" not in inv
|
||||
for card in cards:
|
||||
assert "_amount" not in card
|
||||
|
||||
2
uv.lock
generated
2
uv.lock
generated
@@ -236,7 +236,7 @@ dev = [
|
||||
requires-dist = [
|
||||
{ name = "flask", specifier = ">=3.0" },
|
||||
{ name = "llama-index", specifier = ">=0.12.0" },
|
||||
{ name = "llama-index-llms-openai-like", specifier = "==0.7.2" },
|
||||
{ name = "llama-index-llms-openai-like", specifier = ">=0.7.2" },
|
||||
{ name = "playwright", specifier = ">=1.40" },
|
||||
{ name = "pymupdf", specifier = ">=1.24" },
|
||||
{ name = "python-dotenv", specifier = ">=1.0" },
|
||||
|
||||
Reference in New Issue
Block a user