Compare commits
13 Commits
f754386695
...
agent
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1b35f07fd7 | ||
|
|
7c137c5214 | ||
|
|
e252896de9 | ||
|
|
46305fdebb | ||
|
|
8dd91df3b9 | ||
|
|
6d66a27aab | ||
|
|
10115214aa | ||
|
|
cf567c22f2 | ||
|
|
98e3d21c83 | ||
|
|
0074975591 | ||
|
|
a72c4ffaab | ||
|
|
70f61be3aa | ||
|
|
02e2ca90a7 |
19
.agents/README.md
Normal file
19
.agents/README.md
Normal file
@@ -0,0 +1,19 @@
|
||||
---
|
||||
last_reviewed: 2026-06-11
|
||||
---
|
||||
|
||||
# .agents — 项目内部维护目录
|
||||
|
||||
本目录存放维护人员与自动化代理相关资料,不属于对外公开的用户文档。
|
||||
|
||||
## 目录结构
|
||||
|
||||
| 子目录 | 说明 |
|
||||
|--------|------|
|
||||
| `docs/` | 维护文档:规范、经验总结、实施方案、操作指南 |
|
||||
| `skills/` | Cursor Agent 技能:定义自动化工作流和代码检查流程 |
|
||||
|
||||
## 文档边界
|
||||
|
||||
- 面向开源用户、外部贡献者的公开文档统一放置在项目根目录 `docs/` 下。
|
||||
- 维护规范、实施方案、经验总结、拉取请求佐证材料与各类内部记录资料,均统一放置在本目录下。
|
||||
12
.agents/docs/README.md
Normal file
12
.agents/docs/README.md
Normal file
@@ -0,0 +1,12 @@
|
||||
# 项目维护人员文档
|
||||
|
||||
本目录存放维护人员与自动化代理相关资料,主要用于项目运维,不属于对外公开的用户文档。
|
||||
|
||||
- standards/:存放维护人员需遵守的规范制度与校验规则
|
||||
- plans/:存放实施方案与工作交接说明
|
||||
- error-experience/、good-experience/:存放内部经验总结文档
|
||||
- guides/:面向维护人员的工作流程及集成实操手册
|
||||
- architecture/manifest.yaml:记录可读性检查所覆盖的文件路径
|
||||
|
||||
对外用户文档及说明文件请统一放置在 docs/ 目录下。
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
last_reviewed: 2026-06-09
|
||||
---
|
||||
|
||||
# LlamaIndex 多模态消息格式错误
|
||||
|
||||
## 错误现象
|
||||
|
||||
```
|
||||
pydantic_core._pydantic_core.ValidationError: 2 validation errors for ChatMessage
|
||||
blocks.0
|
||||
Unable to extract tag using discriminator 'block_type' [type=union_tag_not_found, ...
|
||||
blocks.1
|
||||
Unable to extract tag using discriminator 'block_type' [type=union_tag_not_found, ...
|
||||
```
|
||||
|
||||
## 触发条件
|
||||
|
||||
- llama-index-core >= 0.14.x
|
||||
- 使用 `ChatMessage` 构造多模态消息(文本 + 图片)
|
||||
- 传入 OpenAI 格式的 `content` 列表:`[{"type": "text", ...}, {"type": "image_url", ...}]`
|
||||
|
||||
## 原因
|
||||
|
||||
`llama-index-core 0.14.x` 重构了 `ChatMessage` 的内部结构:
|
||||
|
||||
| 版本 | 字段 | 多模态 content 格式 |
|
||||
|------|------|-------------------|
|
||||
| 0.14.x | `role`, `blocks`, `additional_kwargs` | `TextBlock` / `ImageBlock` 实例 |
|
||||
| 旧版本 | `role`, `content` | OpenAI 风格字典列表 |
|
||||
|
||||
底层 Pydantic 模型使用 `block_type` 作为 union discriminator,OpenAI 格式的 `{"type": "text", ...}` 字典不包含该字段,导致验证失败。
|
||||
|
||||
## 修复方法
|
||||
|
||||
**正确写法(llama-index 原生 blocks 格式):**
|
||||
|
||||
```python
|
||||
from llama_index.core.llms import ChatMessage
|
||||
from llama_index.core.base.llms.types import ImageBlock, TextBlock
|
||||
|
||||
messages = [
|
||||
ChatMessage(role="system", content=system_prompt),
|
||||
ChatMessage(
|
||||
role="user",
|
||||
blocks=[
|
||||
TextBlock(text="请分析这张图片"),
|
||||
ImageBlock(
|
||||
url=f"data:image/jpeg;base64,{image_b64}",
|
||||
detail="high",
|
||||
),
|
||||
],
|
||||
),
|
||||
]
|
||||
```
|
||||
## 适用版本
|
||||
|
||||
- llama-index-core: 0.14.22
|
||||
- llama-index-llms-openai-like: 0.7.2
|
||||
@@ -0,0 +1,74 @@
|
||||
---
|
||||
last_reviewed: 2026-06-13
|
||||
---
|
||||
|
||||
# llm_query_text 缺少 start/end 事件导致前端不显示思考过程
|
||||
|
||||
## 错误现象
|
||||
|
||||
- 前端只显示 "正在分析文件..." 的文字提示(来自 agent 事件的 `agent_state_change`)
|
||||
- LLM 返回的思考过程气泡和正式回答气泡都不显示
|
||||
- 校验流程的气泡正常显示,但提取流程的气泡缺失
|
||||
- 后端日志正常,LLM 请求成功返回
|
||||
|
||||
## 触发条件
|
||||
|
||||
- `llm_query_text()` 或 `_llm_query_multimodal()` 被 Agent 调度调用
|
||||
- `source_dir` 参数已传入(启用 SSE 流式事件)
|
||||
- 函数内部只发了 `chunk` 和 `reasoning` 事件,缺少 `start` 和 `end`
|
||||
|
||||
## 原因
|
||||
|
||||
前端 `chat.js` 的 `handleLLMStream` 状态机:
|
||||
|
||||
```
|
||||
start -> _createLLMStreamBubble() // 创建聊天气泡 DOM
|
||||
reasoning -> _appendLLMStreamReasoning() // 往气泡追加思考内容
|
||||
chunk -> _appendLLMStreamChunk() // 往气泡追加正式回答
|
||||
end -> _closeLLMStreamBubble() // 关闭气泡,切换完成样式
|
||||
```
|
||||
|
||||
`_appendLLMStreamReasoning` 和 `_appendLLMStreamChunk` 的入口守卫:
|
||||
|
||||
```js
|
||||
if (!llmStreamState.reasoningContent) return;
|
||||
if (!llmStreamState.textContent) return;
|
||||
```
|
||||
|
||||
没有 `start` 事件,`llmStreamState` 就一直是初始空值,后续所有 `reasoning` 和 `chunk` 事件都会被静默丢弃。
|
||||
|
||||
**为什么校验流程正常?** 因为 `validate_semantic_completeness()` 在调用 `llm_query_text` 之前,自己手动发了 `start` 和 `end` 事件,绕过了这个问题。
|
||||
|
||||
## 修复方法
|
||||
|
||||
`llm_query_text` 和 `_llm_query_multimodal` 在 `source_dir` 非空时,必须在流式循环前后发送完整的事件序列:
|
||||
|
||||
```python
|
||||
# 循环前
|
||||
if source_dir:
|
||||
_emit_llm_stream(source_dir, "start", label="正在分析文件...")
|
||||
|
||||
try:
|
||||
for resp in llm.stream_chat(...):
|
||||
# ... chunk / reasoning ...
|
||||
# 循环后
|
||||
if source_dir:
|
||||
_emit_llm_stream(source_dir, "end", label="分析完成")
|
||||
except Exception as e:
|
||||
if source_dir:
|
||||
_emit_llm_stream(source_dir, "error", error=str(e))
|
||||
```
|
||||
|
||||
## 防回归要点
|
||||
|
||||
修改 `llm_query_text` 或 `_llm_query_multimodal` 时,检查事件发送是否完整:
|
||||
|
||||
| 阶段 | 事件 | 必需性 |
|
||||
|------|------|--------|
|
||||
| 循环前 | `start` | 必需(前端创建气泡) |
|
||||
| 循环中 | `chunk` | 可选(无内容时不发) |
|
||||
| 循环中 | `reasoning` | 可选(模型不支持时不发) |
|
||||
| 成功 | `end` | 必需(前端切换完成样式) |
|
||||
| 失败 | `error` | 必需 |
|
||||
|
||||
删除 `start` 或 `end` 会导致前端气泡丢失,是高频回归点。
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
last_reviewed: 2026-06-13
|
||||
---
|
||||
|
||||
# 闭包内复用外层变量名导致 UnboundLocalError
|
||||
|
||||
## 错误现象
|
||||
|
||||
- 补充材料提交后,提取函数正常执行完成
|
||||
- 日志停在 `travel_applications.json` 保存处,后续没有任何 Agent 处理日志
|
||||
- 没有报错、没有异常堆栈,看起来像"停止"
|
||||
- `result.json` 里实际记录了 `{"ok": false, "error": "local variable 'agent_session' referenced before assignment"}`
|
||||
|
||||
## 触发条件
|
||||
|
||||
- 在 Flask 路由中用 `threading.Thread` 启动后台任务
|
||||
- 外层作用域已有一个变量(如 `agent_session`)
|
||||
- 闭包 `_run()` 内对同名变量既读又写:`agent_session = run_agent_round(session_dir, agent_session, ...)`
|
||||
|
||||
## 原因
|
||||
|
||||
Python 变量作用域规则:**只要函数体内有任何对某标识符的赋值,该标识符在整个函数内都被视为局部变量**。
|
||||
|
||||
```python
|
||||
agent_session = add_supplement(session_dir, agent_session, filenames) # 外层变量
|
||||
|
||||
def _run() -> None:
|
||||
# ...
|
||||
agent_session = run_agent_round( # 赋值 -> 整个 _run 内 agent_session 是局部变量
|
||||
session_dir, agent_session, # 读局部变量,但此时还未赋值 -> UnboundLocalError
|
||||
new_files=filenames,
|
||||
)
|
||||
```
|
||||
|
||||
`run_agent_round` 调用时,`agent_session` 作为参数被求值,但此时它还是未初始化的局部变量,触发 `UnboundLocalError`。该异常被 `except BaseException` 捕获后写入 result.json,没有在日志中输出,所以表现为"静默停止"。
|
||||
|
||||
## 修复方法
|
||||
|
||||
闭包内使用不同名称接收返回值:
|
||||
|
||||
```python
|
||||
# 修复前
|
||||
agent_session = run_agent_round(session_dir, agent_session, new_files=filenames)
|
||||
|
||||
# 修复后
|
||||
new_session = run_agent_round(session_dir, agent_session, new_files=filenames)
|
||||
```
|
||||
|
||||
后续对返回值的引用统一改为 `new_session`。
|
||||
|
||||
## 防回归要点
|
||||
|
||||
| 场景 | 风险 | 检查方法 |
|
||||
|------|------|----------|
|
||||
| 在闭包/嵌套函数内赋值与外层同名的变量 | UnboundLocalError | ruff F823 规则 |
|
||||
| `except BaseException` 吞掉异常且不打日志 | 静默失败,难以排查 | 至少记录 `log.exception` |
|
||||
| 用 `# noqa: F823` 压制警告而不修复 | 问题持续存在 | noqa 只应用于确认安全的场景 |
|
||||
|
||||
**核心原则**:在闭包内需要接收外层变量的返回值时,始终使用不同的变量名。不要依赖 `nonlocal` 来修复合法性问题——换名字更简单、更安全。
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# 补充材料提交后 SSE 立即读到旧 result.json 导致前端无消息
|
||||
|
||||
## 错误现象
|
||||
|
||||
- 第二次补充材料提交后,前端没有任何消息显示
|
||||
- 状态栏不更新,聊天区无新增消息
|
||||
- 后台日志显示处理正常完成(LLM 提取、校验、Bot 提交均成功)
|
||||
- 前端像是"卡住"了一样,没有报错也没有反馈
|
||||
|
||||
## 触发条件
|
||||
|
||||
1. 第一轮处理或第一轮补充材料完成,`result.json` 已写入 session 目录
|
||||
2. 用户再次补充材料,触发新一轮处理
|
||||
3. 前端创建新的 SSE 连接到 `/api/logs/<session_id>`
|
||||
4. SSE 端点轮询时立即检测到旧的 `result.json`,直接发射 `done` 事件并关闭连接
|
||||
5. 前端断开 SSE,但后台线程仍在执行新任务
|
||||
|
||||
## 根因
|
||||
|
||||
`_run_agent_task` 在每次任务启动时只清理了 `llm_stream.log`,未清理 `result.json` 和 `agent_events.log`。
|
||||
|
||||
```python
|
||||
# 修复前 - 只清理了 llm_stream.log
|
||||
try:
|
||||
(session_dir / "llm_stream.log").unlink(missing_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
```
|
||||
|
||||
SSE 端点 (`/api/logs/<session_id>`) 在 `generate()` 中轮询检查 `result.json` 是否存在,一旦存在就发射 `done` 事件并 `break` 退出循环。旧的 `result.json` 未被清理,导致 SSE 连接在任务实际开始前就结束了。
|
||||
|
||||
## 修复
|
||||
|
||||
在 `_run_agent_task` 开头统一清理三个残留文件:
|
||||
|
||||
```python
|
||||
# 修复后 - 同时清理三个残留文件
|
||||
for fname in ("llm_stream.log", "agent_events.log", pipeline_web.SESSION_RESULT_FILE):
|
||||
try:
|
||||
(session_dir / fname).unlink(missing_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
```
|
||||
|
||||
- `llm_stream.log` — LLM 流式日志
|
||||
- `agent_events.log` — Agent 事件日志(避免旧事件被重放)
|
||||
- `result.json` — 处理结果文件(避免 SSE 立即读到旧结果)
|
||||
|
||||
## 影响范围
|
||||
|
||||
所有使用 `_run_agent_task` 的端点均受影响:
|
||||
- `/api/process/<session_id>` — 初始处理
|
||||
- `/api/agent/supplement/<session_id>` — 补充文件
|
||||
- `/api/agent/user-supplement/<session_id>` — 文字补充
|
||||
- `/api/agent/force-submit/<session_id>` — 强制提交
|
||||
- `/api/submit-financial/<session_id>` — 手动财务提交
|
||||
36
.agents/docs/good-experience/2026-06-11-出现问题好的排查流程.md
Normal file
36
.agents/docs/good-experience/2026-06-11-出现问题好的排查流程.md
Normal file
@@ -0,0 +1,36 @@
|
||||
---
|
||||
last_reviewed: 2026-06-11
|
||||
---
|
||||
|
||||
# 出现问题好的排查流程
|
||||
|
||||
## 出现问题后的排查流程
|
||||
|
||||
```
|
||||
日志报错
|
||||
│
|
||||
├─ 1. 定位错误源码
|
||||
│ 根据日志标签 + 错误信息 → 找到报错函数和校验条件
|
||||
│
|
||||
├─ 2. 分析日志时间线
|
||||
│ 对比错误时间与前后事件 → 判断是主流程还是试探性调用
|
||||
│
|
||||
├─ 3. 编写诊断脚本(隔离测试)
|
||||
│ │
|
||||
│ ├─ 正常 save/load 循环 → 确认基础路径是否健康
|
||||
│ ├─ 编码异常检测 → BOM、GBK 混入等
|
||||
│ ├─ 数据流变更检测 → 前端编辑/外部写入后列结构变化
|
||||
│ └─ 回退链检测 → glob 扫描到非预期文件
|
||||
│
|
||||
├─ 4. 最小化复现
|
||||
│ 针对失败的测试用例,提取最简输入证明根因
|
||||
│
|
||||
└─ 5. 修复 + 验证
|
||||
最小改动修复 → 确认不影响正常输入
|
||||
```
|
||||
|
||||
## 关键判断点
|
||||
|
||||
- 错误不影响主流程 → 优先排查试探性调用和回退链
|
||||
- 列名校验失败但文件肉眼正常 → 优先检查 BOM 和不可见字符
|
||||
- 错误出现在外部数据入口 → 优先检查编码兼容性和数据清洗
|
||||
261
.agents/docs/guides/工程实践指南.md
Normal file
261
.agents/docs/guides/工程实践指南.md
Normal file
@@ -0,0 +1,261 @@
|
||||
---
|
||||
last_reviewed: 2026-06-09
|
||||
---
|
||||
|
||||
# 工程实践指南
|
||||
|
||||
创建或修改任何 Python 项目时,必须严格遵守以下工程规范。
|
||||
|
||||
---
|
||||
|
||||
## 1. 包管理与虚拟环境
|
||||
|
||||
- **唯一包管理器**:使用 `uv`,不使用 `pip`、`pipenv`、`poetry`。
|
||||
- **依赖声明**:所有依赖统一在 `pyproject.toml` 中管理,遵循 PEP 621 + PEP 735。
|
||||
- `[project].dependencies` 仅放运行时依赖。
|
||||
- `[dependency-groups].dev` 放开发依赖(测试、lint、类型检查等)。
|
||||
- **版本锁定**:使用 `uv.lock` 锁定依赖版本,提交到版本控制。
|
||||
- **安装命令**:`uv sync` 创建虚拟环境并安装所有依赖。
|
||||
- **运行命令**:所有 Python 命令通过 `uv run` 前缀执行,确保使用项目虚拟环境。
|
||||
- **禁止**:全局安装 Python 包、手动 `python -m venv`、使用 `requirements.txt` 作为主要依赖文件。
|
||||
|
||||
### `pyproject.toml` 必填字段模板
|
||||
|
||||
```toml
|
||||
[project]
|
||||
name = "<project-name>"
|
||||
version = "0.1.0"
|
||||
description = "<项目描述>"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
# 运行时依赖
|
||||
]
|
||||
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
"pytest>=8.0",
|
||||
"pytest-cov>=5.0",
|
||||
"ruff>=0.9",
|
||||
"mypy>=1.14",
|
||||
"deptry>=0.22",
|
||||
"pre-commit>=4.0",
|
||||
]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. 目录结构
|
||||
|
||||
```
|
||||
<project-root>/
|
||||
├── pyproject.toml # 项目配置(依赖 + 工具配置)
|
||||
├── uv.lock # 依赖锁定文件
|
||||
├── .pre-commit-config.yaml # 提交前检查配置
|
||||
├── Makefile # Unix 任务脚本
|
||||
├── tasks.py # Windows/跨平台任务脚本
|
||||
├── .cursorignore # IDE 忽略配置
|
||||
├── src/ # 源代码目录
|
||||
│ └── main.py
|
||||
├── tests/ # 测试目录
|
||||
│ └── test_main.py
|
||||
└── logs/ # 运行时产物(不提交)
|
||||
```
|
||||
|
||||
- 源码统一放在 `src/` 下,不直接在根目录放业务代码。
|
||||
- 测试统一放在 `tests/` 下。
|
||||
- 运行时产物(日志、缓存、临时文件)不提交到版本控制。
|
||||
|
||||
---
|
||||
|
||||
## 3. 代码质量工具链
|
||||
|
||||
### 3.1 Ruff(Lint + Format)
|
||||
|
||||
在 `pyproject.toml` 中配置:
|
||||
|
||||
```toml
|
||||
[tool.ruff]
|
||||
target-version = "py312"
|
||||
line-length = 120
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "W", "I", "N", "UP", "B"]
|
||||
ignore = ["E501"]
|
||||
```
|
||||
|
||||
- **规则覆盖**:格式错误(E/F/W)、导入排序(I)、命名规范(N)、语法升级(UP)、常见 Bug(B)。
|
||||
- **格式化**:使用 `ruff format` 替代 Black。
|
||||
- **运行方式**:
|
||||
- `uv run ruff check .` — 检查问题
|
||||
- `uv run ruff check --fix .` — 自动修复
|
||||
- `uv run ruff format .` — 格式化代码
|
||||
- `uv run ruff format --check .` — 仅检查格式
|
||||
|
||||
### 3.2 MyPy(静态类型检查)
|
||||
|
||||
```toml
|
||||
[tool.mypy]
|
||||
python_version = "3.12"
|
||||
strict = true
|
||||
warn_return_any = true
|
||||
warn_unused_configs = true
|
||||
ignore_missing_imports = true
|
||||
|
||||
[[tool.mypy.overrides]]
|
||||
module = "tests.*"
|
||||
ignore_errors = true
|
||||
```
|
||||
|
||||
- 启用严格模式,测试文件豁免。
|
||||
- **运行方式**:`uv run mypy src/main.py`
|
||||
|
||||
### 3.3 Deptry(依赖审计)
|
||||
|
||||
```toml
|
||||
[tool.deptry]
|
||||
ignore_notebooks = true
|
||||
```
|
||||
|
||||
- 检测未使用、缺失、重复的依赖。
|
||||
- **运行方式**:`uv run deptry .`
|
||||
|
||||
---
|
||||
|
||||
## 4. 测试规范
|
||||
|
||||
### 4.1 Pytest 配置
|
||||
|
||||
```toml
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
pythonpath = ["."]
|
||||
```
|
||||
|
||||
### 4.2 测试要求
|
||||
|
||||
- 测试文件命名:`test_*.py`,放在 `tests/` 目录。
|
||||
- 测试函数/类命名:以 `test_` 或 `Test` 开头。
|
||||
- 使用 `pytest.fixture` 管理测试资源。
|
||||
- 必须使用 `pytest-cov` 生成覆盖率报告。
|
||||
- **运行方式**:
|
||||
```bash
|
||||
uv run python -m pytest --cov --cov-config=pyproject.toml --cov-report=term-missing
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Pre-commit Hooks
|
||||
|
||||
`.pre-commit-config.yaml` 必须包含:
|
||||
|
||||
```yaml
|
||||
repos:
|
||||
- repo: https://github.com/astral-sh/ruff-pre-commit
|
||||
rev: v0.9.6
|
||||
hooks:
|
||||
- id: ruff
|
||||
args: [--fix]
|
||||
- id: ruff-format
|
||||
```
|
||||
|
||||
- 安装:`uv run pre-commit install`
|
||||
- 手动运行:`uv run pre-commit run --all-files`
|
||||
|
||||
---
|
||||
|
||||
## 6. 任务运行器
|
||||
|
||||
### 6.1 Makefile(Unix)
|
||||
|
||||
```makefile
|
||||
.PHONY: install check test run clean
|
||||
|
||||
install:
|
||||
@uv sync
|
||||
@uv run pre-commit install
|
||||
|
||||
check:
|
||||
@uv lock --locked
|
||||
@uv run ruff check .
|
||||
@uv run ruff format --check .
|
||||
@uv run mypy src/main.py
|
||||
@uv run deptry .
|
||||
|
||||
test:
|
||||
@uv run python -m pytest --cov --cov-config=pyproject.toml --cov-report=term-missing
|
||||
|
||||
run:
|
||||
@uv run python src/main.py
|
||||
|
||||
clean:
|
||||
@rm -rf .venv __pycache__ .pytest_cache .mypy_cache .ruff_cache
|
||||
```
|
||||
|
||||
### 6.2 tasks.py(跨平台)
|
||||
|
||||
提供 `tasks.py` 作为 Windows 兼容的任务运行器,支持相同任务名:`install`、`check`、`test`、`run`、`clean`。
|
||||
|
||||
---
|
||||
|
||||
## 7. 版本控制忽略
|
||||
|
||||
`.cursorignore` / `.gitignore` 必须排除:
|
||||
|
||||
```
|
||||
.venv/
|
||||
__pycache__/
|
||||
.pytest_cache/
|
||||
.mypy_cache/
|
||||
.ruff_cache/
|
||||
*.pyc
|
||||
logs/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. 开发工作流
|
||||
|
||||
新项目初始化顺序:
|
||||
|
||||
1. 创建 `pyproject.toml`,声明项目元数据和依赖。
|
||||
2. 运行 `uv sync` 创建虚拟环境。
|
||||
3. 创建 `.pre-commit-config.yaml`,运行 `uv run pre-commit install`。
|
||||
4. 创建 `src/` 目录和入口文件。
|
||||
5. 创建 `tests/` 目录和基础测试。
|
||||
6. 创建 `Makefile` + `tasks.py`。
|
||||
7. 运行 `make check` 或 `python tasks.py check` 验证代码质量。
|
||||
8. 运行 `make test` 或 `python tasks.py test` 验证测试通过。
|
||||
|
||||
日常开发顺序:
|
||||
|
||||
1. `uv run ruff check --fix .` — 先修复 lint 问题。
|
||||
2. `uv run ruff format .` — 格式化代码。
|
||||
3. `uv run mypy src/` — 类型检查。
|
||||
4. `uv run python -m pytest` — 运行测试。
|
||||
5. 提交前 pre-commit 会自动执行检查和格式化。
|
||||
|
||||
---
|
||||
|
||||
## 9. 编码风格
|
||||
|
||||
- Python 3.12+ 语法,使用现代特性(如 `match/case`、类型合并 `X | Y`)。
|
||||
- 函数和模块必须有 docstring。
|
||||
- 优先使用类型注解,返回值类型必须标注。
|
||||
- 行长度限制 120 字符。
|
||||
- 导入按标准库 → 第三方 → 本地模块分组排序。
|
||||
- 偏好函数式编程风格,避免不必要的面向对象封装。
|
||||
- 配置与代码分离,使用常量或配置模块管理可变参数。
|
||||
|
||||
---
|
||||
|
||||
## 10. 强制检查清单
|
||||
|
||||
在提交代码或声明任务完成前,必须确认:
|
||||
|
||||
- [ ] `uv lock --locked` 通过(锁定文件与 pyproject.toml 一致)
|
||||
- [ ] `uv run ruff check .` 无错误
|
||||
- [ ] `uv run ruff format --check .` 无差异
|
||||
- [ ] `uv run mypy src/` 无类型错误
|
||||
- [ ] `uv run deptry .` 无依赖问题
|
||||
- [ ] `uv run python -m pytest --cov` 全部通过且覆盖率合理
|
||||
- [ ] 所有命令使用 `uv run` 前缀,无全局 pip 操作
|
||||
657
.agents/docs/guides/系统事件流全景图.md
Normal file
657
.agents/docs/guides/系统事件流全景图.md
Normal file
@@ -0,0 +1,657 @@
|
||||
# 系统事件流全景图
|
||||
|
||||
> 最后更新: 2026-06-15
|
||||
> 用途: 排查 SSE 事件问题、提交流程中断、状态不一致等 Bug
|
||||
|
||||
---
|
||||
|
||||
## 一、核心概念
|
||||
|
||||
### 1.1 前后端状态映射
|
||||
|
||||
| 前端 `App.processState` | 后端 `AgentState` | 含义 |
|
||||
|---|---|---|
|
||||
| `idle` | `IDLE` | 初始状态,等待用户操作 |
|
||||
| `processing` | `EXTRACTING` | LLM 正在分析文件 |
|
||||
| `awaiting_supplement` | `AWAITING_SUPPLEMENT` | 信息不完整,等待用户补充 |
|
||||
| `submitting` | `SUBMITTING` | 正在提交到财务系统 |
|
||||
| `ready` | `READY` | 信息完整,可以提交 |
|
||||
| `submitting` | `SUBMITTING` | 正在提交到财务系统 |
|
||||
| `done` | `DONE` / `ERROR` | 流程结束(成功或失败) |
|
||||
|
||||
### 1.2 通信机制
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F as 前端
|
||||
participant S as SSE连接
|
||||
participant B as 后端线程
|
||||
|
||||
F->>B: POST /api/agent/process/:sid
|
||||
B-->>F: {status: "started"}
|
||||
F->>S: GET /api/logs/:sid (SSE长连接)
|
||||
S-->>F: message: file_progress (轮询 file_events.log)
|
||||
S-->>F: message: llm_stream (轮询 llm_stream.log)
|
||||
S-->>F: message: agent_* (轮询 agent_events.log)
|
||||
S-->>F: message: done (检测到 result.json)
|
||||
S->>S: 连接关闭
|
||||
```
|
||||
|
||||
**关键约束**:
|
||||
- 后端所有处理接口均返回 `{status: "started"}`,实际工作在 daemon 线程中执行
|
||||
- SSE 通过每 0.5 秒轮询 4 个日志文件实现(非原生 SSE,是长轮询模拟)
|
||||
- `result.json` 的原子写入:先写 `.tmp`,再 `replace()` 重命名
|
||||
- SSE 超时:600 秒后自动断开
|
||||
|
||||
### 1.3 操作信号点清单
|
||||
|
||||
每个 API 操作涉及的信号文件生命周期如下。**新增或修改信号文件时必须同步更新此清单**。
|
||||
|
||||
| 序号 | 操作 | API 端点 | 线程启动时清理 | 一次写入且不被清理 | 轮次结束时写入 |
|
||||
|---|---|---|---|---|---|
|
||||
| 1 | 初始处理 | `POST /api/agent/process/:sid` | `llm_stream.log`, `agent_events.log`, `result.json` | `file_events.log`, `session.log` | `result.json` |
|
||||
| 2 | 补充文件 | `POST /api/agent/supplement/:sid` | `llm_stream.log`, `agent_events.log`, `result.json` | `file_events.log`, `session.log` | `result.json` |
|
||||
| 3 | 文字补充 | `POST /api/agent/user-supplement/:sid` | `llm_stream.log`, `agent_events.log`, `result.json` | `file_events.log`, `session.log` | `result.json` |
|
||||
| 4 | 强制提交 | `POST /api/agent/force-submit/:sid` | `llm_stream.log`, `agent_events.log`, `result.json` | `file_events.log`, `session.log` | `result.json` |
|
||||
| 5 | 手动财务提交 | `POST /api/submit-financial/:sid` | `llm_stream.log`, `agent_events.log`, `result.json` | `file_events.log`, `session.log` | `result.json` |
|
||||
|
||||
**信号点说明**:
|
||||
|
||||
| 信号文件 | 读/写方 | 生命周期 | 作用 |
|
||||
|---|---|---|---|
|
||||
| `result.json` | 后端线程写入,SSE 端点读取 | 每轮开始时删除,`finally` 块中原子写入 | SSE 检测到该文件即发射 `done` 事件并断开连接 |
|
||||
| `agent_events.log` | Agent 调度器追加写入,SSE 端点读取 | 每轮开始时删除,Agent 运行时持续追加 | 传递 agent 状态变化事件给前端 |
|
||||
| `llm_stream.log` | LLM 回调追加写入,SSE 端点读取 | 每轮开始时删除,LLM 运行时持续追加 | 传递 LLM 流式输出给前端 |
|
||||
| `file_events.log` | `pipeline_web` 追加写入,SSE 端点读取 | 会话内持续追加,不删除 | 传递文件处理进度给前端 |
|
||||
| `session.log` | `sse_handler` 追加写入,SSE 端点读取 | 会话内持续追加,不删除 | 传递普通日志行给前端 |
|
||||
|
||||
### 1.4 关键约束(修改代码前必读)
|
||||
|
||||
**约束 1:`result.json` 必须在每轮线程启动时删除**
|
||||
|
||||
SSE 端点通过检测 `result.json` 是否存在来判断任务是否完成。如果上一轮的 `result.json` 残留,SSE 会立即读到旧数据并发射 `done` 事件,导致前端断开连接,新任务的消息无法送达。
|
||||
|
||||
- 实现位置:`_run_agent_task()` 的 `try` 块开头
|
||||
- 删除时机:在 `install_log_collector()` 之后、`task_fn()` 执行之前
|
||||
- 写入位置:`finally` 块中统一写入(唯一写入点)
|
||||
- 写入规则:`finally` 始终执行原子写入,不再有条件判断
|
||||
- `_emit_ready_and_submit` 只返回 result 字典,不写入文件
|
||||
|
||||
**约束 2:`result.json` 的写入必须使用 `finally` 块**
|
||||
|
||||
无论任务成功或失败,SSE 端点都需要 `result.json` 来发送 `done` 事件。如果仅在成功路径写入,异常时 SSE 会一直轮询直到 600 秒超时,前端无反馈。
|
||||
|
||||
**约束 3:SSE 新建连接时,当前文件偏移必须从 0 开始**
|
||||
|
||||
`_run_agent_task` 在启动时删除 `llm_stream.log` 和 `agent_events.log`,确保 SSE 重新建立连接后从 0 偏移开始读取。如果文件不被删除,旧的事件会被重复发送给前端。
|
||||
|
||||
**约束 4:前端 SSE 连接的生命周期**
|
||||
|
||||
- 前端在每次 POST 请求返回 `{status: "started"}` 后立即创建新的 SSE 连接
|
||||
- 收到 `done` 事件后关闭连接
|
||||
- 旧的连接引用必须清理(`agent.js` 中的 `agentEventSource`)
|
||||
- 如果前端在 POST 之前就创建了 SSE 连接,会读到旧数据
|
||||
|
||||
---
|
||||
|
||||
## 二、场景一:用户提交材料 → LLM 分析完整 → 直接提交
|
||||
|
||||
### 2.1 时序图
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F as 前端
|
||||
participant S as SSE连接
|
||||
participant B as 后端线程
|
||||
participant A as Agent调度器
|
||||
|
||||
F->>F: startProcess()
|
||||
F->>B: POST /api/agent/process/:sid
|
||||
B-->>F: {status: "started"}
|
||||
F->>S: GET /api/logs/:sid
|
||||
|
||||
Note over B,A: 后台线程启动
|
||||
B->>A: extract_invoices()
|
||||
S-->>F: file_progress (processing/done)
|
||||
Note over S: 轮询 file_events.log
|
||||
|
||||
S-->>F: llm_stream (start/chunk/end)
|
||||
Note over S: 轮询 llm_stream.log
|
||||
|
||||
S-->>F: agent_state_change (state=extracting)
|
||||
Note over S: 轮询 agent_events.log
|
||||
|
||||
Note over A: _do_extraction_with_validation()<br/>LLM提取 → validator校验<br/>最多3次重试
|
||||
|
||||
S-->>F: agent_state_change (校验通过/未通过)
|
||||
|
||||
Note over A: can_submit == true
|
||||
|
||||
A->>A: state → READY
|
||||
A->>A: _emit_agent_event (agent_ready)
|
||||
A->>A: _emit_ready_and_submit()
|
||||
A->>A: run_financial_submit()
|
||||
|
||||
S-->>F: agent_ready
|
||||
|
||||
Note over B: 写入 result.json
|
||||
|
||||
S-->>F: done (携带 result)
|
||||
Note over S: 检测到 result.json
|
||||
|
||||
F->>F: es.close()
|
||||
F->>F: App.processState = 'done'
|
||||
F->>F: addChatMessage(成功)
|
||||
Note over B: remove_log_collector
|
||||
```
|
||||
|
||||
### 2.2 事件流清单
|
||||
|
||||
| 序号 | 事件类型 | 来源文件 | 触发时机 | 前端处理 |
|
||||
|---|---|---|---|---|
|
||||
| 1 | `file_progress` | `file_events.log` | 每个文件处理开始/完成 | 更新文件状态 UI |
|
||||
| 2 | `llm_stream` | `llm_stream.log` | LLM 流式输出 | 显示聊天气泡 |
|
||||
| 3 | `agent_state_change` | `agent_events.log` | 状态变为 `extracting` | 显示瞬态状态提示 |
|
||||
| 4 | `agent_state_change` | `agent_events.log` | 校验通过/未通过 | 更新瞬态状态 |
|
||||
| 5 | `agent_ready` | `agent_events.log` | 双重校验通过 | 由 `done` 事件统一处理 |
|
||||
| 6 | `done` | SSE 检测到 `result.json` | 流程结束 | 根据 `result` 判断终态 |
|
||||
|
||||
### 2.3 result.json 结构(成功路径)
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"agent_ready": true,
|
||||
"submit_ok": true,
|
||||
"round": 1,
|
||||
"message": "信息完整,已自动提交到财务系统"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 三、场景二:用户提交材料 → 需补充 → 用户上传文件
|
||||
|
||||
### 3.1 时序图
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F as 前端
|
||||
participant S as SSE连接
|
||||
participant B as 后端线程
|
||||
participant A as Agent调度器
|
||||
|
||||
Note over F,A: 阶段1: 初次分析
|
||||
F->>B: POST /api/agent/process/:sid
|
||||
B-->>F: {status: "started"}
|
||||
F->>S: GET /api/logs/:sid
|
||||
|
||||
S-->>F: agent_state_change (state=extracting)
|
||||
|
||||
Note over A: can_submit == false
|
||||
|
||||
A->>A: state → AWAITING_SUPPLEMENT
|
||||
|
||||
S-->>F: agent_request_supplement
|
||||
|
||||
Note over B: 写入 result.json<br/>(waiting_for_supplement=true)
|
||||
|
||||
S-->>F: done
|
||||
F->>F: es.close()
|
||||
F->>F: App.processState = 'awaiting_supplement'
|
||||
F->>F: showStatus('请补充')
|
||||
F->>F: showAgentRequest()
|
||||
Note over B: remove_log_collector
|
||||
|
||||
Note over F,A: 阶段2: 用户上传补充文件
|
||||
F->>F: 用户点击"上传补充材料"
|
||||
F->>F: 文件上传完成
|
||||
F->>F: handleSupplementUpload(newFilenames)
|
||||
F->>B: POST /api/agent/supplement/:sid<br/>{files: [...]}
|
||||
B-->>F: {status: "started"}
|
||||
F->>S: GET /api/logs/:sid
|
||||
|
||||
Note over B: 新后台线程启动
|
||||
|
||||
A->>A: add_supplement()<br/>(记录文件名, 发射收到事件)
|
||||
|
||||
S-->>F: agent_supplement_received
|
||||
|
||||
A->>A: extract_invoices()<br/>(重新提取所有文件)
|
||||
A->>A: run_agent_round(new_files=[...])
|
||||
Note over A: 加载上一轮结果作为<br/>previous_analysis
|
||||
|
||||
S-->>F: agent_state_change (state=extracting)
|
||||
|
||||
Note over A: LLM提取 → 校验循环
|
||||
|
||||
alt 分支A: 补充后仍不完整
|
||||
S-->>F: agent_request_supplement
|
||||
S-->>F: done (waiting=true)
|
||||
F->>F: es.close()
|
||||
F->>F: App.processState = 'awaiting_supplement'
|
||||
else 分支B: 补充后完整
|
||||
Note over A: can_submit == true
|
||||
A->>A: state → READY
|
||||
A->>A: _emit_ready_and_submit()
|
||||
S-->>F: agent_ready
|
||||
S-->>F: done (submit_ok=true)
|
||||
F->>F: es.close()
|
||||
F->>F: App.processState = 'done'
|
||||
F->>F: addChatMessage(成功)
|
||||
end
|
||||
```
|
||||
|
||||
### 3.2 事件流清单(补充文件路径)
|
||||
|
||||
| 序号 | 事件类型 | 来源文件 | 触发时机 | 前端处理 |
|
||||
|---|---|---|---|---|
|
||||
| 1 | `agent_supplement_received` | `agent_events.log` | 收到补充文件列表 | 显示"已收到补充文件" |
|
||||
| 2 | `agent_state_change` | `agent_events.log` | 开始重新分析 | 显示瞬态状态 |
|
||||
| 3 | `agent_request_supplement` | `agent_events.log` | 仍不完整 | 更新补充请求面板 |
|
||||
| 4 | `agent_ready` | `agent_events.log` | 校验通过 | 由 `done` 统一处理 |
|
||||
| 5 | `done` | SSE 检测到 `result.json` | 流程结束 | 判断终态 |
|
||||
|
||||
### 3.3 result.json 结构(需补充)
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"agent_ready": false,
|
||||
"agent_state": "awaiting_supplement",
|
||||
"round": 1,
|
||||
"waiting_for_supplement": true
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 四、场景三:用户提交材料 → 需补充 → 用户通过对话提供信息
|
||||
|
||||
### 4.1 时序图
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F as 前端
|
||||
participant S as SSE连接
|
||||
participant B as 后端线程
|
||||
participant A as Agent调度器
|
||||
|
||||
Note over F,A: 阶段1: 初次分析
|
||||
F->>B: POST /api/agent/process/:sid
|
||||
B-->>F: {status: "started"}
|
||||
F->>S: GET /api/logs/:sid
|
||||
|
||||
S-->>F: agent_request_supplement
|
||||
S-->>F: done (waiting=true)
|
||||
F->>F: es.close()
|
||||
F->>F: App.processState = 'awaiting_supplement'
|
||||
Note over B: remove_log_collector
|
||||
|
||||
Note over F,A: 阶段2: 用户输入文字
|
||||
F->>F: 用户在输入框输入文字
|
||||
F->>F: handleUserSupplement()
|
||||
F->>B: POST /api/agent/user-supplement/:sid<br/>{text: "..."}
|
||||
B-->>F: {status: "started"}
|
||||
F->>S: GET /api/logs/:sid
|
||||
|
||||
Note over B: 新后台线程启动
|
||||
|
||||
S-->>F: agent_supplement_received
|
||||
|
||||
A->>A: process_user_text_supplement()
|
||||
A->>A: process_user_supplement()<br/>(LLM解析用户文字)
|
||||
|
||||
S-->>F: llm_stream (解析过程)
|
||||
|
||||
A->>A: merge_supplement_into_info()<br/>(合并到 extracted_info)
|
||||
Note over A: 保存到缓存文件
|
||||
|
||||
A->>A: run_agent_round()<br/>(重新校验)
|
||||
|
||||
S-->>F: agent_state_change<br/>(state=extracting, 正在重新校验)
|
||||
|
||||
alt 分支A: 补充后仍不完整
|
||||
S-->>F: agent_request_supplement
|
||||
S-->>F: done (waiting=true)
|
||||
F->>F: es.close()
|
||||
F->>F: App.processState = 'awaiting_supplement'
|
||||
else 分支B: 补充后完整
|
||||
Note over A: can_submit == true
|
||||
A->>A: state → READY
|
||||
A->>A: _emit_ready_and_submit()
|
||||
S-->>F: agent_ready
|
||||
S-->>F: done (submit_ok=true)
|
||||
F->>F: es.close()
|
||||
F->>F: App.processState = 'done'
|
||||
F->>F: addChatMessage(成功)
|
||||
end
|
||||
```
|
||||
|
||||
### 4.2 事件流清单(文字补充路径)
|
||||
|
||||
| 序号 | 事件类型 | 来源文件 | 触发时机 | 前端处理 |
|
||||
|---|---|---|---|---|
|
||||
| 1 | `agent_supplement_received` | `agent_events.log` | 收到用户文字 | 显示"已收到补充" |
|
||||
| 2 | `llm_stream` | `llm_stream.log` | LLM 解析用户文字 | 显示解析过程 |
|
||||
| 3 | `agent_state_change` | `agent_events.log` | 开始重新校验 | 显示"正在重新校验" |
|
||||
| 4 | `agent_request_supplement` | `agent_events.log` | 仍不完整 | 更新补充请求 |
|
||||
| 5 | `agent_ready` | `agent_events.log` | 校验通过 | 由 `done` 统一处理 |
|
||||
| 6 | `done` | SSE 检测到 `result.json` | 流程结束 | 判断终态 |
|
||||
|
||||
---
|
||||
|
||||
## 五、强制提交流程
|
||||
|
||||
### 5.1 时序图
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F as 前端
|
||||
participant S as SSE连接
|
||||
participant B as 后端线程
|
||||
|
||||
F->>F: handleForceSubmit()
|
||||
F->>F: App.forceSubmitting = true
|
||||
F->>B: POST /api/agent/force-submit/:sid
|
||||
B-->>F: {status: "started"}
|
||||
F->>S: GET /api/logs/:sid
|
||||
|
||||
Note over B: 后台线程启动
|
||||
|
||||
B->>B: force_submit()<br/>(state → READY)
|
||||
|
||||
S-->>F: agent_force_submit
|
||||
|
||||
B->>B: run_financial_submit()
|
||||
|
||||
Note over B: 写入 result.json
|
||||
|
||||
S-->>F: done
|
||||
F->>F: es.close()
|
||||
F->>F: App.forceSubmitting = false
|
||||
F->>F: App.processState = 'done'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 六、错误处理路径
|
||||
|
||||
### 6.1 错误场景和事件
|
||||
|
||||
| 错误场景 | 后端行为 | 发射事件 | 前端表现 |
|
||||
|---|---|---|---|
|
||||
| LLM 提取异常 | `state → ERROR` | `agent_error` | 聊天显示错误,`processState → 'done'` |
|
||||
| 规则校验 3 次失败 | 返回最后一次结果,继续语义判断 | `agent_state_change` | 依赖 `can_submit` 字段决定 |
|
||||
| 轮次超限 (5 轮) | `state → ERROR` | `agent_max_rounds` | 聊天显示错误,可强制提交 |
|
||||
| 财务提交失败 | `result.submit_ok = false` | 无独立事件 | `done` 事件携带错误信息 |
|
||||
| SSE 连接中断 | 无 | `es.onerror` 触发 | 显示"连接中断" |
|
||||
| 超时 (600s) | SSE 轮询循环退出 | 连接自然断开 | 连接断开 |
|
||||
|
||||
### 6.2 agent_error 事件结构
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "agent_error",
|
||||
"message": "LLM 提取失败: ..."
|
||||
}
|
||||
```
|
||||
|
||||
### 6.3 agent_max_rounds 事件结构
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "agent_max_rounds",
|
||||
"message": "已达到最大轮次 (5),请检查信息或强制提交"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 七、状态机完整图
|
||||
|
||||
### 7.1 后端 AgentState 状态机
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> IDLE
|
||||
|
||||
IDLE --> EXTRACTING: POST /api/agent/process\nPOST /api/agent/supplement\nPOST /api/agent/user-supplement
|
||||
|
||||
EXTRACTING --> READY: can_submit == true
|
||||
EXTRACTING --> AWAITING_SUPPLEMENT: can_submit == false
|
||||
EXTRACTING --> ERROR: 异常 / 轮次超限
|
||||
|
||||
READY --> SUBMITTING: _emit_ready_and_submit()
|
||||
SUBMITTING --> DONE: 财务提交完成
|
||||
|
||||
AWAITING_SUPPLEMENT --> EXTRACTING: 用户补充文件/文字
|
||||
|
||||
READY: 准备提交\n(终态保护)
|
||||
SUBMITTING: 财务提交中\n(终态保护)
|
||||
DONE: 终态\n(终态保护)
|
||||
ERROR: 错误状态\n(可强制提交)
|
||||
|
||||
note right of EXTRACTING
|
||||
LLM 提取 + validator 校验\n最多 3 次重试
|
||||
end note
|
||||
```
|
||||
|
||||
### 7.2 前端 processState 状态机
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> idle
|
||||
|
||||
idle --> processing: startProcess()
|
||||
|
||||
processing --> awaiting_supplement: done事件\nresult.waiting_for_supplement
|
||||
processing --> done: done事件\nresult.ok
|
||||
|
||||
awaiting_supplement --> processing: 补充文件或文字
|
||||
awaiting_supplement --> submitting: 强制提交
|
||||
|
||||
submitting --> done: done事件
|
||||
|
||||
done: 流程结束
|
||||
idle: 初始状态
|
||||
processing: 处理中
|
||||
awaiting_supplement: 等待补充
|
||||
submitting: 提交中
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 八、SSE 事件类型完整参考
|
||||
|
||||
### 8.1 Agent 事件 (agent_events.log)
|
||||
|
||||
| 事件类型 | 数据结构 | 触发条件 |
|
||||
|---|---|---|
|
||||
| `agent_state_change` | `{type, state, round, attempt, message}` | 状态切换 |
|
||||
| `agent_ready` | `{type, round, message}` | 双重校验通过 |
|
||||
| `agent_request_supplement` | `{type, round, missing_fields, missing_materials, semantic_issues, suggestion}` | 校验未通过 |
|
||||
| `agent_supplement_received` | `{type, files}` | 收到用户补充 |
|
||||
| `agent_force_submit` | `{type, message}` | 用户强制提交 |
|
||||
| `agent_error` | `{type, message}` | 提取失败 |
|
||||
| `agent_max_rounds` | `{type, message}` | 达到最大轮次 |
|
||||
|
||||
### 8.2 文件进度事件 (file_events.log)
|
||||
|
||||
| 事件类型 | 数据结构 | 触发条件 |
|
||||
|---|---|---|
|
||||
| `file_progress` | `{type, file, status, summary?, error?}` | 文件处理状态变更 |
|
||||
|
||||
`status` 取值: `processing` / `done` / `cached` / `error`
|
||||
|
||||
### 8.3 LLM 流式事件 (llm_stream.log)
|
||||
|
||||
| 事件类型 | 数据结构 | 触发条件 |
|
||||
|---|---|---|
|
||||
| `llm_stream` | `{type, phase, text?}` | LLM 输出流 |
|
||||
|
||||
`phase` 取值: `start` / `reasoning` / `chunk` / `end` / `error`
|
||||
|
||||
### 8.4 完成事件 (SSE 直接发送)
|
||||
|
||||
| 事件类型 | 数据结构 | 触发条件 |
|
||||
|---|---|---|
|
||||
| `done` | `{type, result: {...}}` | `result.json` 出现 |
|
||||
|
||||
---
|
||||
|
||||
## 九、常见问题排查清单
|
||||
|
||||
### 9.1 SSE 事件丢失
|
||||
|
||||
**症状**: 前端没有收到预期的 agent 事件
|
||||
|
||||
**排查步骤**:
|
||||
1. 检查 `agent_events.log` 是否存在、是否有内容
|
||||
2. 检查 SSE 连接是否建立成功(浏览器 Network 面板)
|
||||
3. 确认 `sse_handler.install_log_collector()` 是否被调用
|
||||
4. 确认 `remove_log_collector()` 是否过早调用
|
||||
|
||||
### 9.2 提交流程中断
|
||||
|
||||
**症状**: 流程在某个中间状态卡住,没有 `done` 事件
|
||||
|
||||
**排查步骤**:
|
||||
1. 检查 `result.json` 是否被写入
|
||||
2. 检查后台线程是否异常退出(查看 `session.log`)
|
||||
3. 确认 `finally` 块中的 `result.json` 写入逻辑是否执行
|
||||
4. 检查是否触发了 600 秒超时
|
||||
|
||||
### 9.3 状态不一致
|
||||
|
||||
**症状**: 前端 `processState` 和后端 `AgentState` 不匹配
|
||||
|
||||
**排查步骤**:
|
||||
1. 对比 `agent_events.log` 中的状态变更序列
|
||||
2. 检查前端是否正确处理了 `done` 事件
|
||||
3. 确认 SSE 连接是否在适当时机关闭和重建
|
||||
4. 检查 `App.agentEventSource` 引用是否正确清理
|
||||
|
||||
### 9.4 补充流程不触发
|
||||
|
||||
**症状**: 用户上传补充文件或输入文字后,没有重新分析
|
||||
|
||||
**排查步骤**:
|
||||
1. 确认 `processState` 是否为 `awaiting_supplement`
|
||||
2. 检查补充 API 是否返回 `{status: "started"}`
|
||||
3. 检查新 SSE 连接是否成功建立
|
||||
4. 确认 `add_supplement()` 或 `process_user_text_supplement()` 是否被调用
|
||||
|
||||
### 9.5 补充材料后前端无任何消息(`result.json` 残留问题)
|
||||
|
||||
**症状**: 第二轮及之后的补充材料提交后,前端完全没有任何消息显示,状态栏不更新,聊天区无新增消息。后台日志显示处理正常完成。
|
||||
|
||||
**根因**: `_run_agent_task` 在每轮启动时未清理上一轮的 `result.json`。SSE 端点轮询时立即检测到旧的 `result.json`,直接发射 `done` 事件并关闭连接,前端断开后无法接收新任务的消息。
|
||||
|
||||
**排查步骤**:
|
||||
1. 检查 session 目录中 `result.json` 的修改时间 — 如果早于当前轮次开始时间,说明是残留文件
|
||||
2. 检查浏览器 Network 面板中 SSE 连接 — 是否在建立后立即收到 `done` 事件
|
||||
3. 确认 `_run_agent_task` 是否在启动时清理了 `result.json`
|
||||
|
||||
**修复**: 在 `_run_agent_task` 的 `try` 块开头同时清理 `llm_stream.log`、`agent_events.log` 和 `result.json` 三个文件。
|
||||
|
||||
**详细记录**: 参见 `.agents/docs/error-experience/2026-06-15-补充材料SSE立即读到旧result.json导致前端无消息.md`
|
||||
|
||||
---
|
||||
|
||||
## 十、关键文件索引
|
||||
|
||||
| 文件 | 职责 |
|
||||
|---|---|
|
||||
| `src/web/static/js/process.js` | 主提交流程入口,SSE 事件分发 |
|
||||
| `src/web/static/js/agent.js` | Agent 事件处理,补充/强制提交逻辑 |
|
||||
| `src/web/static/js/state.js` | 全局状态管理 |
|
||||
| `src/web/routes.py` | 后端路由,后台线程启动 |
|
||||
| `src/agent/orchestrator.py` | Agent 调度器,状态机,校验循环 |
|
||||
| `src/web/sse_handler.py` | SSE 日志收集器 |
|
||||
| `src/web/pipeline_web.py` | 发票提取管道,财务提交 |
|
||||
|
||||
---
|
||||
|
||||
## 十一、文件生命周期与操作信号点
|
||||
|
||||
### 11.1 单轮处理的完整文件生命周期
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant API as 路由层
|
||||
participant RT as _run_agent_task
|
||||
participant TF as task_fn
|
||||
participant SS as _emit_ready_and_submit
|
||||
participant SSE as SSE 端点
|
||||
|
||||
Note over API: 1. 创建 handler
|
||||
API->>API: install_log_collector(session_dir)
|
||||
Note over API: 创建 SSE 日志收集器<br/>随即开始写入 session.log
|
||||
|
||||
API->>RT: threading.Thread(target=_run_agent_task)
|
||||
|
||||
Note over RT: 2. 清理残留文件
|
||||
RT->>RT: unlink(llm_stream.log)
|
||||
RT->>RT: unlink(agent_events.log)
|
||||
RT->>RT: unlink(result.json)
|
||||
|
||||
Note over RT: 3. 执行任务
|
||||
RT->>TF: task_fn(session_dir, config)
|
||||
|
||||
Note over TF: 执行期间各个文件由对应模块写入:
|
||||
TF-->>TF: file_events.log (pipeline_web)
|
||||
TF-->>TF: llm_stream.log (LLM 回调)
|
||||
TF-->>TF: agent_events.log (Agent 调度器)
|
||||
|
||||
TF-->>RT: 返回 (agent_session, 占位 result)
|
||||
|
||||
alt 成功路径 (READY)
|
||||
RT->>SS: _emit_ready_and_submit()
|
||||
Note over SS: 发射 agent_ready 事件<br/>执行财务提交<br/>返回 result 字典
|
||||
SS-->>RT: result 字典
|
||||
Note over RT: result = {...}
|
||||
else 需补充路径 (AWAITING_SUPPLEMENT)
|
||||
Note over RT: result = {waiting_for_supplement: true}
|
||||
else 异常路径
|
||||
Note over RT: result = {ok: false, error: ...}
|
||||
end
|
||||
|
||||
Note over RT: 4. finally 块 — 唯一写入点
|
||||
RT->>RT: 原子写入 result.json (.tmp → replace)
|
||||
|
||||
RT->>RT: remove_log_collector(handler)
|
||||
|
||||
Note over SSE: 5. SSE 端点检测
|
||||
SSE->>SSE: 轮询检测到 result.json
|
||||
SSE-->>SSE: 发射 done 事件
|
||||
SSE->>SSE: break 退出轮询
|
||||
```
|
||||
|
||||
### 11.2 各阶段信号文件状态
|
||||
|
||||
| 阶段 | `result.json` | `llm_stream.log` | `agent_events.log` | `file_events.log` | `session.log` |
|
||||
|---|---|---|---|---|---|
|
||||
| 会话创建 | 不存在 | 不存在 | 不存在 | 不存在 | 不存在 |
|
||||
| `install_log_collector` 后 | 不存在 | 不存在 | 不存在 | 不存在 | 开始写入 |
|
||||
| `_run_agent_task` 清理后 | 已删除 | 已删除 | 已删除 | 保持 | 保持 |
|
||||
| 文件提取中 | 不存在 | 不存在 | 不存在 | 持续追加 | 持续追加 |
|
||||
| LLM 提取中 | 不存在 | 持续追加 | 持续追加 | 保持 | 持续追加 |
|
||||
| 校验中 | 不存在 | 保持 | 持续追加 | 保持 | 持续追加 |
|
||||
| 任务完成 (READY) | 已写入 | 保持 | 保持 | 保持 | 保持 |
|
||||
| 任务完成 (需补充) | 已写入 | 保持 | 保持 | 保持 | 保持 |
|
||||
| 任务异常 | 已写入 | 保持 | 保持 | 保持 | 保持 |
|
||||
| SSE done 事件后 | 保持 | 保持 | 保持 | 保持 | 保持 |
|
||||
|
||||
### 11.3 新增信号文件检查清单
|
||||
|
||||
当需要在系统中新增一个信号文件(如 `submit_progress.log`)时,必须检查以下事项:
|
||||
|
||||
1. **写入方**:哪个模块负责写入?写入时机是什么?
|
||||
2. **读取方**:SSE 端点是否需要轮询?前端是否需要处理?
|
||||
3. **清理时机**:是否需要在 `_run_agent_task` 中清理?如果不需要,为什么?
|
||||
4. **原子性**:写入是否需要 `.tmp` + `replace` 模式?
|
||||
5. **轮询偏移**:SSE 端点是否需要跟踪该文件的读取偏移?
|
||||
6. **更新本文档**:在 1.3 操作信号点清单中新增一行,在 11.2 文件状态表中新增一列
|
||||
7. **更新 `_run_agent_task`**:如果需要清理,在清理循环中添加文件名
|
||||
8. **更新前端**:在 `sse.js` 或 `agent.js` 中添加对应的事件处理器
|
||||
401
.agents/docs/guides/项目架构全景图.md
Normal file
401
.agents/docs/guides/项目架构全景图.md
Normal file
@@ -0,0 +1,401 @@
|
||||
# 项目架构全景图
|
||||
|
||||
> 最后更新: 2026-06-15
|
||||
> 用途: 理解项目整体结构、模块职责、依赖关系和数据流
|
||||
|
||||
---
|
||||
|
||||
## 一、分层架构总览
|
||||
|
||||
```
|
||||
src/
|
||||
├── agent/ Agent 调度层(协调提取-校验-修正循环,状态机管理)
|
||||
├── core/ 核心业务层(纯逻辑,零框架依赖)
|
||||
├── infra/ 基础设施层(浏览器、文档、LLM 提示词)
|
||||
├── web/ Web 界面层(Flask + SSE)
|
||||
├── pipeline.py CLI 流程编排
|
||||
├── pipeline_core.py CLI/Web 公共管道逻辑
|
||||
├── main.py CLI 入口
|
||||
├── config.py 配置加载
|
||||
└── exceptions.py 异常定义
|
||||
```
|
||||
|
||||
### 依赖方向
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
classDef entry fill:#e8eaf6,stroke:#3f51b5,color:#1a237e
|
||||
classDef orchestrate fill:#e0f2f1,stroke:#00897b,color:#004d40
|
||||
classDef agent fill:#fff8e1,stroke:#ff8f00,color:#3e2723
|
||||
classDef core fill:#e3f2fd,stroke:#1565c0,color:#0d47a1
|
||||
classDef infra fill:#e8f5e9,stroke:#2e7d32,color:#1b5e20
|
||||
|
||||
subgraph 入口层
|
||||
CLI["main.py"]:::entry
|
||||
WEB["web/app.py"]:::entry
|
||||
end
|
||||
|
||||
subgraph 编排层
|
||||
PIPE["pipeline.py"]:::orchestrate
|
||||
PIPE_WEB["web/pipeline_web.py"]:::orchestrate
|
||||
PIPE_CORE["pipeline_core.py"]:::orchestrate
|
||||
end
|
||||
|
||||
subgraph Agent调度层
|
||||
AGENT["agent/orchestrator.py"]:::agent
|
||||
SESSION["agent/session.py"]:::agent
|
||||
EVENTS["agent/events.py"]:::agent
|
||||
end
|
||||
|
||||
subgraph 核心业务层
|
||||
EXTRACT["core/extraction/"]:::core
|
||||
MATCH["core/matching/"]:::core
|
||||
VALID["core/validation/"]:::core
|
||||
end
|
||||
|
||||
subgraph 基础设施层
|
||||
BROWSER["infra/browser/"]:::infra
|
||||
DOCS["infra/documents/"]:::infra
|
||||
LLM["infra/llm/"]:::infra
|
||||
end
|
||||
|
||||
CLI --> PIPE
|
||||
WEB --> PIPE_WEB
|
||||
PIPE --> PIPE_CORE
|
||||
PIPE --> EXTRACT
|
||||
PIPE --> BROWSER
|
||||
PIPE_WEB --> AGENT
|
||||
PIPE_WEB --> PIPE_CORE
|
||||
PIPE_WEB --> EXTRACT
|
||||
AGENT --> EXTRACT
|
||||
AGENT --> VALID
|
||||
AGENT --> LLM
|
||||
AGENT --> PIPE_CORE
|
||||
EXTRACT --> MATCH
|
||||
EXTRACT --> DOCS
|
||||
EXTRACT --> LLM
|
||||
MATCH --> DOCS
|
||||
BROWSER --> DOCS
|
||||
```
|
||||
|
||||
**关键约束**:
|
||||
- `infra` 不依赖 `core` 和 `agent`,只提供工具能力
|
||||
- `core` 零外部依赖,不依赖 Flask、Playwright 等框架
|
||||
- `agent` 依赖 `core` 和 `infra`,作为调度中枢编排各模块
|
||||
- 所有跨层调用均通过 `__init__.py` 导出的稳定接口
|
||||
|
||||
---
|
||||
|
||||
## 二、模块清单
|
||||
|
||||
### 2.1 Agent 调度层 (`src/agent/`)
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `coordinator.py` | 核心协调逻辑:提取-校验-修正循环(最多 3 次重试)、用户补充处理、强制提交 |
|
||||
| `session.py` | 会话状态:`AgentState` 枚举、`AgentSession` 数据类、状态持久化(原子写入) |
|
||||
| `events.py` | SSE 事件发射:事件去重、事件日志追加、事件读取 |
|
||||
| `orchestrator.py` | 兼容层:从子模块重新导出所有符号,保持旧导入路径可用 |
|
||||
|
||||
**对外接口**:`AgentSession`, `AgentState`, `run_agent_round()`, `force_submit()`, `add_supplement()`, `process_user_text_supplement()`, `load_agent_state()`, `save_agent_state()`
|
||||
|
||||
### 2.2 核心业务层 (`src/core/`)
|
||||
|
||||
| 子模块 | 职责 | 对外接口 |
|
||||
|------|------|------|
|
||||
| `extraction/extractor.py` | 编排入口:扫描目录 → 逐文件提取 → 分类 → 金额匹配 | `extract_invoices()`, `extract_document()` |
|
||||
| `extraction/llm_extractor.py` | LLM 多模态提取核心:统一文档提取、差旅/普通信息提取、缓存管理、SSE 流式事件 | `llm_query_text()`, `extract_travel_info()`, `extract_normal_info()`, `load_cache()` |
|
||||
| `matching/matcher.py` | 发票与支付记录按金额匹配(一对一 / 一对多贪心,相对容差 3%) | `match_invoices_to_cards()` |
|
||||
| `validation/validator.py` | 声明式规则校验引擎,规则从 JSON 配置文件加载 | `validate_extracted_info()`, `ValidationReport` |
|
||||
|
||||
### 2.3 基础设施层 (`src/infra/`)
|
||||
|
||||
| 子模块 | 职责 | 对外接口 |
|
||||
|------|------|------|
|
||||
| `browser/base.py` | `BaseBot` 基类:Playwright 浏览器生命周期、登录、导航、截图 | 内部基类 |
|
||||
| `browser/travel.py` | 差旅报销填报:基本信息 → 明细 → 支付 → 补助 → 附件上传 | 内部流程 |
|
||||
| `browser/normal.py` | 普通报销填报:基本信息 → 总明细 → 支付 → 附件上传 | 内部流程 |
|
||||
| `browser/__init__.py` | 浏览器入口:类型路由和流程调度 | `run_bot()`, `run_bot_web()` |
|
||||
| `documents/invoice.py` | 发票数据模型、CSV/JSON 读写、发票分类 | `load_csv()`, `save_csv()`, `save_invoice_csv()`, `classify_invoice_batch()` |
|
||||
| `documents/pdf.py` | PDF 渲染为图片(PyMuPDF) | `render_pdf_to_images()` |
|
||||
| `documents/consumable.py` | 易耗品出库单填写:CSV → Word 模板 | `fill_consumable_doc()` |
|
||||
| `llm/prompt.py` | LLM 提示词加载 | `build_invoice_system_prompt()`, `build_travel_info_system_prompt()`, `build_normal_info_system_prompt()` |
|
||||
|
||||
### 2.4 Web 界面层 (`src/web/`)
|
||||
|
||||
| 文件/目录 | 职责 |
|
||||
|------|------|
|
||||
| `app.py` | Flask 应用入口,注册蓝图和模板 |
|
||||
| `routes.py` | 路由定义:会话管理、文件上传、配置、SSE 日志流、Agent 交互 API |
|
||||
| `pipeline_web.py` | Web 管道逻辑:发票提取 + 出库单生成 + 财务提交 |
|
||||
| `sse_handler.py` | SSE 日志收集器、日志转义、文件轮询 |
|
||||
| `templates/` | `index.html`(PC 端主界面)、`mobile_upload.html`(移动端上传) |
|
||||
| `static/js/` | 前端逻辑(按加载顺序):`state.js` → `utils.js` → `chat.js` → `upload.js` → `config.js` → `process.js` → `sync.js` → `index.js` |
|
||||
|
||||
---
|
||||
|
||||
## 三、CLI 模式数据流
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
CLI_ENTRY["main.py --step all"] --> PIPE["pipeline.py run_pipeline()"]
|
||||
|
||||
subgraph Step1["Step 1: 发票提取"]
|
||||
PIPE --> EXT["core/extraction/extractor.py extract_invoices()"]
|
||||
EXT --> DOC["逐文件提取"]
|
||||
DOC --> LLM["LLM 多模态识别 (infra/llm)"]
|
||||
LLM --> CLASS["分类: train/hotel/general/payment/application"]
|
||||
CLASS --> MATCH["core/matching/matcher.py 金额匹配"]
|
||||
MATCH --> SAVE["infra/documents/ CSV/JSON 保存"]
|
||||
end
|
||||
|
||||
subgraph Step2["Step 2: 信息提取"]
|
||||
SAVE --> TYPE{"判断报销类型"}
|
||||
TYPE -->|差旅| TRAVEL["提取差旅信息 → travel_info.json"]
|
||||
TYPE -->|普通| NORMAL["提取普通发票信息 → normal_info.json"]
|
||||
end
|
||||
|
||||
subgraph Step3["Step 3: 浏览器填报"]
|
||||
TRAVEL --> BOT["infra/browser/ 填报"]
|
||||
NORMAL --> BOT
|
||||
BOT -->|差旅| BOT_T["browser/travel.py"]
|
||||
BOT -->|普通| BOT_N["browser/normal.py"]
|
||||
end
|
||||
```
|
||||
|
||||
**关键文件输出**:
|
||||
|
||||
| 文件 | 来源 | 说明 |
|
||||
|------|------|------|
|
||||
| `payment_records.csv` | Step 1 | 支付记录级别(每笔刷卡记录一行) |
|
||||
| `invoice_summary.csv` | Step 1 | 发票级别(每张发票一行) |
|
||||
| `travel_applications.json` | Step 1 | 出差事前申请单 |
|
||||
| `invoice_groups.json` | Step 1 | 发票分类结果 |
|
||||
| `travel_info.json` | Step 2 | 差旅信息:交通/住宿明细、补贴、附件清单 |
|
||||
| `normal_info.json` | Step 2 | 普通发票信息:报销说明、发票总数、总金额、附件清单 |
|
||||
|
||||
---
|
||||
|
||||
## 四、Web 模式数据流
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F as 前端 (浏览器)
|
||||
participant API as routes.py
|
||||
participant PW as pipeline_web.py
|
||||
participant AG as agent/orchestrator.py
|
||||
participant EX as core/extraction/
|
||||
participant VA as core/validation/
|
||||
participant SSE as SSE 轮询
|
||||
|
||||
F->>API: POST /api/session → 创建 session
|
||||
F->>API: POST /api/upload/:sid → 上传文件
|
||||
F->>API: POST /api/agent/process/:sid
|
||||
API-->>F: {status: "started"}
|
||||
F->>SSE: GET /api/logs/:sid (SSE 长连接)
|
||||
|
||||
Note over API: 后台 daemon 线程启动
|
||||
|
||||
API->>PW: extract_invoices(session_dir)
|
||||
PW->>EX: 发票提取 + 分类 + 匹配
|
||||
EX-->>PW: payment_records, applications, groups
|
||||
|
||||
API->>AG: run_agent_round(session_dir, session)
|
||||
loop 校验-修正循环 (最多 3 次)
|
||||
AG->>EX: llm_query_text() 提取信息
|
||||
AG->>VA: validate_extracted_info() 规则校验
|
||||
alt 校验失败
|
||||
AG->>AG: 构建修正提示
|
||||
end
|
||||
end
|
||||
AG-->>API: session (READY 或 AWAITING_SUPPLEMENT)
|
||||
|
||||
SSE-->>F: file_progress, llm_stream, agent_state_change, agent_ready/agent_request_supplement
|
||||
|
||||
API->>API: 写入 result.json
|
||||
SSE-->>F: done (携带 result)
|
||||
F->>F: 关闭 SSE, 展示结果
|
||||
```
|
||||
|
||||
### Web 模式特有的 Agent 调度
|
||||
|
||||
CLI 模式中 `pipeline.py` 直接调用 `extract_invoices()` → `infra/browser/`,不经过 Agent 层。
|
||||
|
||||
Web 模式中 `routes.py` 启动后台线程,调用 `agent/orchestrator.py` 作为调度中枢:
|
||||
|
||||
```
|
||||
run_agent_round()
|
||||
├── 1. load_cache() — 检查缓存
|
||||
├── 2. _do_extraction_with_validation() — 提取-校验-修正循环
|
||||
│ ├── llm_query_text() — LLM 提取结构化信息
|
||||
│ ├── validate_extracted_info() — 规则校验
|
||||
│ └── 校验失败 → 构建修正提示 → 再次调用 LLM (最多 3 次)
|
||||
├── 3. 判断 can_submit 字段
|
||||
│ ├── true → READY → 自动触发财务提交
|
||||
│ └── false → AWAITING_SUPPLEMENT → 等待用户补充
|
||||
├── 4. 用户补充处理
|
||||
│ ├── add_supplement() — 记录补充文件
|
||||
│ └── process_user_text_supplement() — LLM 解析文字补充
|
||||
└── 5. save_agent_state() — 持久化状态
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 五、Agent 状态机
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> IDLE: 会话创建
|
||||
|
||||
IDLE --> EXTRACTING: POST /api/agent/process
|
||||
IDLE --> EXTRACTING: POST /api/agent/supplement
|
||||
IDLE --> EXTRACTING: POST /api/agent/user-supplement
|
||||
|
||||
EXTRACTING --> READY: can_submit == true
|
||||
EXTRACTING --> AWAITING_SUPPLEMENT: can_submit == false
|
||||
EXTRACTING --> ERROR: 异常 / 轮次超限
|
||||
|
||||
READY --> SUBMITTING: _emit_ready_and_submit()
|
||||
SUBMITTING --> DONE: 财务提交完成
|
||||
|
||||
AWAITING_SUPPLEMENT --> EXTRACTING: 用户补充文件/文字
|
||||
AWAITING_SUPPLEMENT --> READY: 用户强制提交
|
||||
|
||||
note right of EXTRACTING
|
||||
LLM 提取 + validator 校验
|
||||
最多 3 次重试
|
||||
end note
|
||||
```
|
||||
|
||||
### 终态保护
|
||||
|
||||
以下状态为终态,再次触发 `run_agent_round()` 会被跳过:
|
||||
- `DONE` — 提交完成
|
||||
- `SUBMITTING` — 提交中
|
||||
- `READY` — 准备提交
|
||||
|
||||
### 轮次保护
|
||||
|
||||
默认最多 5 轮(`AgentSession.max_rounds`),超限后进入 `ERROR` 状态,用户可选择强制提交。
|
||||
|
||||
---
|
||||
|
||||
## 六、SSE 事件通信机制
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph 后端写入
|
||||
AGENT[agent/orchestrator.py] -->|追加写入| AE[agent_events.log]
|
||||
LLM[LLM 回调] -->|追加写入| LS[llm_stream.log]
|
||||
PW[pipeline_web.py] -->|追加写入| FE[file_events.log]
|
||||
SH[sse_handler.py] -->|追加写入| SL[session.log]
|
||||
RT[_run_agent_task] -->|finally 原子写入| RJ[result.json]
|
||||
end
|
||||
|
||||
subgraph SSE 轮询 (0.5s)
|
||||
POLL[SSE 端点] -->|读取| AE
|
||||
POLL -->|读取| LS
|
||||
POLL -->|读取| FE
|
||||
POLL -->|读取| SL
|
||||
POLL -->|检测| RJ
|
||||
end
|
||||
|
||||
POLL -->|event: agent_*| FRONT[前端 agent.js]
|
||||
POLL -->|event: llm_stream| FRONT
|
||||
POLL -->|event: file_progress| FRONT
|
||||
POLL -->|event: done| FRONT
|
||||
```
|
||||
|
||||
### 信号文件生命周期
|
||||
|
||||
| 阶段 | `result.json` | `llm_stream.log` | `agent_events.log` | `file_events.log` | `session.log` |
|
||||
|------|:--:|:--:|:--:|:--:|:--:|
|
||||
| 会话创建 | 不存在 | 不存在 | 不存在 | 不存在 | 不存在 |
|
||||
| 后台线程启动 | 已删除 | 已删除 | 已删除 | 保持 | 保持 |
|
||||
| 文件提取中 | 不存在 | 不存在 | 不存在 | 持续追加 | 持续追加 |
|
||||
| LLM 提取中 | 不存在 | 持续追加 | 持续追加 | 保持 | 持续追加 |
|
||||
| 校验中 | 不存在 | 保持 | 持续追加 | 保持 | 持续追加 |
|
||||
| 任务完成 | 已写入 | 保持 | 保持 | 保持 | 保持 |
|
||||
| SSE done 事件 | 保持 | 保持 | 保持 | 保持 | 保持 |
|
||||
|
||||
---
|
||||
|
||||
## 七、发票类型路由
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
INPUT["上传文件 (PDF/图片)"] --> EXT["LLM 多模态识别"]
|
||||
EXT --> TYPE{"invoice_type?"}
|
||||
|
||||
TYPE -->|train| TRAVEL["差旅报销流程"]
|
||||
TYPE -->|hotel| TRAVEL
|
||||
TYPE -->|general| NORMAL["普通报销流程"]
|
||||
TYPE -->|payment| MATCH["参与金额匹配"]
|
||||
TYPE -->|application| APP["存储为 JSON"]
|
||||
|
||||
TRAVEL --> TRAVEL_INFO["提取差旅信息<br/>travel_info.json"]
|
||||
TRAVEL_INFO --> TRAVEL_BOT["browser/travel.py<br/>填报差旅报销单"]
|
||||
|
||||
NORMAL --> NORMAL_INFO["提取普通发票信息<br/>normal_info.json"]
|
||||
NORMAL_INFO --> NORMAL_BOT["browser/normal.py<br/>填报普通报销单"]
|
||||
NORMAL_INFO --> CONSUMABLE["生成易耗品出库单<br/>(仅普通报销)"]
|
||||
|
||||
MATCH --> MERGE["合并到对应发票组"]
|
||||
|
||||
style TRAVEL fill:#cfe2ff,stroke:#0d6efd
|
||||
style NORMAL fill:#f8d7da,stroke:#dc3545
|
||||
style MATCH fill:#d1e7dd,stroke:#198754
|
||||
style APP fill:#fff3cd,stroke:#ffc107
|
||||
```
|
||||
|
||||
| 发票类型 | `invoice_type` | 报销流程 | 生成出库单 |
|
||||
|----------|---------------|---------|:--:|
|
||||
| 高铁票/火车票 | `train` | 差旅报销 | 否 |
|
||||
| 酒店住宿 | `hotel` | 差旅报销 | 否 |
|
||||
| 普通发票 | `general` | 普通报销 | 是 |
|
||||
| 支付记录 | `payment` | 参与匹配 | 否 |
|
||||
| 出差申请单 | `application` | 单独存储 | 否 |
|
||||
|
||||
> 差旅发票和普通发票不支持混报,混合时系统按普通报销处理。
|
||||
|
||||
---
|
||||
|
||||
## 八、设计原则
|
||||
|
||||
| 原则 | 说明 |
|
||||
|------|------|
|
||||
| **Agent 是调度中枢** | 校验-修正循环由 Agent 编排,不内嵌在 `llm_extractor` 中 |
|
||||
| **模块职责单一** | `llm_extractor` 只管提取,`validator` 只管校验,Agent 负责编排 |
|
||||
| **core 零外部依赖** | 不依赖 Flask、Playwright 等框架 |
|
||||
| **infra 不依赖业务** | 基础设施层只提供工具能力,不包含业务逻辑 |
|
||||
| **缓存优先** | 信息提取优先读取 `.invoice_cache`,避免重复调用 LLM |
|
||||
| **轮次保护** | 默认 5 轮上限,校验-修正循环最多重试 3 次 |
|
||||
| **终态保护** | `DONE`/`SUBMITTING`/`READY` 状态下不再重复处理 |
|
||||
| **容错降级** | 规则校验 3 次重试后返回最佳结果,不阻断流程 |
|
||||
| **原子写入** | 状态文件先写 `.tmp` 再 `rename()`,防止读取不完整数据 |
|
||||
|
||||
---
|
||||
|
||||
## 九、关键文件索引
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `src/main.py` | CLI 入口 |
|
||||
| `src/web/app.py` | Web 入口 |
|
||||
| `src/pipeline.py` | CLI 流程编排 |
|
||||
| `src/pipeline_core.py` | CLI/Web 公共管道逻辑 |
|
||||
| `src/web/pipeline_web.py` | Web 管道逻辑 + 财务提交 |
|
||||
| `src/web/routes.py` | Web 路由 + 后台线程启动 |
|
||||
| `src/agent/coordinator.py` | Agent 核心协调逻辑 |
|
||||
| `src/agent/session.py` | 会话状态定义与持久化 |
|
||||
| `src/agent/events.py` | SSE 事件发射 |
|
||||
| `src/core/extraction/extractor.py` | 发票提取编排入口 |
|
||||
| `src/core/extraction/llm_extractor.py` | LLM 多模态提取核心 |
|
||||
| `src/core/matching/matcher.py` | 金额匹配 |
|
||||
| `src/core/validation/validator.py` | 声明式规则校验 |
|
||||
| `src/infra/browser/base.py` | 浏览器自动化基类 |
|
||||
| `src/infra/documents/invoice.py` | 发票数据模型 |
|
||||
| `src/web/sse_handler.py` | SSE 日志收集器 |
|
||||
| `src/web/static/js/process.js` | 前端主提交流程 |
|
||||
| `src/web/static/js/agent.js` | 前端 Agent 交互处理 |
|
||||
| `config.json` | 项目配置 |
|
||||
20
.agents/docs/plans/README.md
Normal file
20
.agents/docs/plans/README.md
Normal file
@@ -0,0 +1,20 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# .agents/docs/plans — 实施方案与工作交接
|
||||
|
||||
存放项目实施方案、架构分析报告、重构计划等规划类文档。
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 说明 |
|
||||
|------|------|
|
||||
| `架构分析-2026-06-15.md` | 项目架构分析与重构建议(模块拆分、分层设计、接口契约) |
|
||||
|
||||
## 用途
|
||||
|
||||
- 架构决策记录
|
||||
- 重构实施方案
|
||||
- 工作交接说明
|
||||
- 技术选型论证
|
||||
225
.agents/docs/plans/架构分析-2026-06-15.md
Normal file
225
.agents/docs/plans/架构分析-2026-06-15.md
Normal file
@@ -0,0 +1,225 @@
|
||||
# 项目架构分析与重构建议
|
||||
|
||||
## 一、当前架构总览
|
||||
|
||||
```
|
||||
src/
|
||||
├── main.py # CLI 入口
|
||||
├── pipeline.py # CLI 管道编排
|
||||
├── pipeline_core.py # CLI/Web 公共管道逻辑
|
||||
├── config.py # 配置加载
|
||||
├── exceptions.py # 异常定义
|
||||
│
|
||||
├── doc/ # 文档处理模块(职责过重)
|
||||
│ ├── extractor.py # 发票提取编排
|
||||
│ ├── llm_extractor.py # LLM 提取核心
|
||||
│ ├── invoice.py # 发票数据模型 + CSV 工具
|
||||
│ ├── matcher.py # 发票匹配逻辑
|
||||
│ ├── validator.py # 信息校验规则
|
||||
│ ├── prompt.py # 提示词加载
|
||||
│ ├── pdf.py # PDF 渲染
|
||||
│ ├── fill_consumable_doc.py # 出库单填写
|
||||
│ └── prompts/ # LLM 提示词模板
|
||||
│
|
||||
├── agent/ # Agent 调度模块
|
||||
│ └── orchestrator.py # 校验-修正循环调度
|
||||
│
|
||||
├── bot/ # 浏览器自动化模块
|
||||
│ ├── base.py # 浏览器基类
|
||||
│ ├── travel.py # 差旅填报
|
||||
│ └── normal.py # 普通报销填报
|
||||
│
|
||||
└── web/ # Web 界面模块
|
||||
├── app.py # Flask 应用
|
||||
├── routes.py # 路由定义
|
||||
├── pipeline_web.py # Web 管道逻辑(与 pipeline_core 重复)
|
||||
├── sse_handler.py # SSE 日志流处理
|
||||
└── static/templates/ # 前端资源
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 二、问题分析
|
||||
|
||||
### 2.1 职责不清(高耦合)
|
||||
|
||||
| 问题 | 位置 | 说明 |
|
||||
|------|------|------|
|
||||
| **doc 模块职责过重** | `src/doc/` | 同时负责:提取、匹配、校验、提示词、PDF渲染、出库单填写、CSV操作 |
|
||||
| **Web 层重复逻辑** | `pipeline_web.py` vs `pipeline_core.py` | 两者的 `is_travel_invoice`、`extract_and_cache_*` 逻辑重复 |
|
||||
| **提示词与校验耦合** | `validator.py` | 校验规则直接引用提示词相关函数,缺乏分层 |
|
||||
| **bot 模块位置** | `src/bot/` | 浏览器自动化属于基础设施,却被放在 src 根目录而非独立模块 |
|
||||
|
||||
### 2.2 逻辑混乱
|
||||
|
||||
1. **`src/doc/validator.py`** 的问题:
|
||||
- 校验规则(`TRAVEL_VALIDATION_RULES`)硬编码在模块中,修改需改代码
|
||||
- `FieldRule` 和 `ArrayRule` 类与校验逻辑紧耦合
|
||||
- 数组元素字段支持简单格式和详细格式两种配置,增加了理解成本
|
||||
|
||||
2. **`src/doc/prompt.py`** 的问题:
|
||||
- 简单的文件读取包装,但调用方分散
|
||||
- `build_invoice_system_prompt()` 和 `build_travel_info_system_prompt()` 分别调用,但结构相似
|
||||
|
||||
3. **`src/agent/orchestrator.py`** 的问题:
|
||||
- 校验循环与提取逻辑混合在 `_do_extraction_with_validation`
|
||||
- SSE 事件发射逻辑(`_emit_agent_event`)与业务逻辑混杂
|
||||
- 状态机转换逻辑分散
|
||||
|
||||
### 2.3 分层不合理
|
||||
|
||||
```
|
||||
当前分层(按目录):
|
||||
main.py → pipeline.py → doc/ + bot/
|
||||
↓
|
||||
pipeline_web.py → web/
|
||||
|
||||
建议分层(按职责):
|
||||
应用层: main.py, pipeline.py, pipeline_web.py
|
||||
业务层: agent/orchestrator.py, doc/validator.py, doc/matcher.py
|
||||
提取层: doc/extractor.py, doc/llm_extractor.py
|
||||
基础设施层: bot/, web/, doc/pdf.py, doc/fill_consumable_doc.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 三、重构建议
|
||||
|
||||
### 3.1 目录重组
|
||||
|
||||
```
|
||||
src/
|
||||
├── main.py # CLI 入口
|
||||
├── config.py # 配置加载
|
||||
├── exceptions.py # 异常定义
|
||||
│
|
||||
├── apps/ # 应用层(管道编排)
|
||||
│ ├── cli/ # CLI 应用
|
||||
│ │ └── pipeline.py
|
||||
│ └── web/ # Web 应用
|
||||
│ ├── app.py
|
||||
│ ├── routes.py
|
||||
│ ├── pipeline.py # Web 专用管道
|
||||
│ └── sse.py
|
||||
│
|
||||
├── core/ # 核心业务逻辑
|
||||
│ ├── agent/ # Agent 调度
|
||||
│ │ ├── orchestrator.py
|
||||
│ │ └── session.py
|
||||
│ ├── validation/ # 校验模块
|
||||
│ │ ├── validator.py
|
||||
│ │ └── rules/ # 校验规则(可配置化)
|
||||
│ ├── matching/ # 匹配模块
|
||||
│ │ └── matcher.py
|
||||
│ └── extraction/ # 提取模块
|
||||
│ ├── extractor.py
|
||||
│ └── llm.py
|
||||
│
|
||||
├── infra/ # 基础设施层
|
||||
│ ├── browser/ # 浏览器自动化
|
||||
│ │ ├── base.py
|
||||
│ │ ├── travel.py
|
||||
│ │ └── normal.py
|
||||
│ ├── documents/ # 文档处理
|
||||
│ │ ├── invoice.py
|
||||
│ │ ├── pdf.py
|
||||
│ │ └── consumable.py
|
||||
│ └── llm/ # LLM 接口
|
||||
│ └── prompts/ # 提示词模板
|
||||
│
|
||||
└── shared/ # 共享工具
|
||||
├── logging.py
|
||||
└── cache.py
|
||||
```
|
||||
|
||||
### 3.2 关键重构点
|
||||
|
||||
#### 3.2.1 doc 模块拆分
|
||||
|
||||
| 职责 | 建议移动位置 |
|
||||
|------|-------------|
|
||||
| `validator.py` | `core/validation/` |
|
||||
| `matcher.py` | `core/matching/` |
|
||||
| `llm_extractor.py` | `core/extraction/` |
|
||||
| `extractor.py` | `core/extraction/` |
|
||||
| `invoice.py` | `infra/documents/` |
|
||||
| `pdf.py` | `infra/documents/` |
|
||||
| `fill_consumable_doc.py` | `infra/documents/` |
|
||||
| `prompt.py` + `prompts/` | `infra/llm/` |
|
||||
|
||||
#### 3.2.2 消除重复逻辑
|
||||
|
||||
**问题**: `pipeline_web.py` 和 `pipeline_core.py` 都有相似逻辑:
|
||||
- `is_travel_invoice()`
|
||||
- `extract_and_cache_travel_info()`
|
||||
- `extract_and_cache_normal_info()`
|
||||
|
||||
**建议**: 将这些公共逻辑统一到 `core/pipeline/` 目录,两个入口调用同一模块。
|
||||
|
||||
#### 3.2.3 Validator 重构
|
||||
|
||||
**当前问题**:
|
||||
- 校验规则硬编码
|
||||
- `FieldRule` 和 `ArrayRule` 类过于复杂
|
||||
|
||||
**建议**:
|
||||
- 将校验规则外部化为 JSON/YAML 配置文件
|
||||
- 简化 `FieldRule` 为单一数据结构
|
||||
- 统一顶层字段和数组元素字段的校验方式
|
||||
|
||||
#### 3.2.4 Agent 拆分
|
||||
|
||||
**当前问题**:
|
||||
- `orchestrator.py` 包含:状态机、SSE 事件、校验循环、提取逻辑
|
||||
|
||||
**建议**:
|
||||
```
|
||||
agent/
|
||||
├── session.py # 状态机定义 + 会话数据模型
|
||||
├── coordinator.py # 校验-修正循环
|
||||
├── events.py # SSE 事件发射
|
||||
└── orchestrator.py # 总调度入口
|
||||
```
|
||||
|
||||
### 3.3 接口契约强化
|
||||
|
||||
| 模块 | 依赖关系 | 接口契约 |
|
||||
|------|----------|----------|
|
||||
| `core/extraction` | 被 `apps/*` 调用 | 返回 `(payment_records, applications, groups)` |
|
||||
| `core/validation` | 被 `agent/*` 调用 | `validate(info, rules) -> ValidationReport` |
|
||||
| `core/matching` | 被 `extraction` 调用 | `match(invoices, cards) -> List[Dict]` |
|
||||
| `infra/browser` | 被 `apps/*` 调用 | `run(bot, info) -> None` |
|
||||
| `infra/llm` | 被 `core/extraction` 调用 | `extract_document(file) -> dict` |
|
||||
|
||||
---
|
||||
|
||||
## 四、优先重构顺序
|
||||
|
||||
### 第一阶段(降低耦合)
|
||||
1. 将 `doc/` 拆分为 `core/` + `infra/`
|
||||
2. 消除 `pipeline_web.py` 和 `pipeline_core.py` 的重复逻辑
|
||||
3. 将 `bot/` 移动到 `infra/browser/`
|
||||
|
||||
### 第二阶段(职责清晰化)
|
||||
4. 拆分 `agent/orchestrator.py` 为多个模块
|
||||
5. 外部化 `validator.py` 的校验规则为配置文件
|
||||
6. 统一 SSE 事件处理接口
|
||||
|
||||
### 第三阶段(可维护性)
|
||||
7. 完善 `__init__.py` 的接口导出
|
||||
8. 添加模块间依赖注入机制
|
||||
9. 建立跨模块调用规范
|
||||
|
||||
---
|
||||
|
||||
## 五、当前项目优点
|
||||
|
||||
1. **日志规范**: 统一的 `get_logger()` 方式,全局日志管理
|
||||
2. **异常体系**: 清晰的 `ReimbursementError` 异常层次
|
||||
3. **SSE 事件协议**: 良好的实时反馈机制
|
||||
4. **缓存设计**: `llm_extractor.py` 的缓存加载逻辑完善
|
||||
5. **声明式校验**: `validator.py` 的规则配置思路正确
|
||||
|
||||
---
|
||||
|
||||
*生成时间: 2026-06-15*
|
||||
9
.agents/docs/standards/README.md
Normal file
9
.agents/docs/standards/README.md
Normal file
@@ -0,0 +1,9 @@
|
||||
---
|
||||
last_reviewed: 2026-06-09
|
||||
---
|
||||
|
||||
# 标准元数据
|
||||
|
||||
`.agents/docs/standards/*.md` 下所有文件都必须包含 frontmatter,字段包括:
|
||||
|
||||
- `last_reviewed`:最近一次策略审查的 ISO 日期 `YYYY-MM-DD`。
|
||||
13
.agents/docs/standards/复利式工程实践.md
Normal file
13
.agents/docs/standards/复利式工程实践.md
Normal file
@@ -0,0 +1,13 @@
|
||||
---
|
||||
last_reviewed: 2026-06-09
|
||||
---
|
||||
|
||||
# 复利式工程实践
|
||||
|
||||
记录经验教训:
|
||||
* 错误经验:`.agents/docs/error-experience/YYYY-MM-DD-<slug>.md`
|
||||
* 正向经验:`.agents/docs/good-experience/YYYY-MM-DD-<slug>.md`
|
||||
* 计划:`.agents/docs/plans/`
|
||||
* 指南:`agents/docs/guides/`
|
||||
|
||||
在出现重大 bug、CI 失败或发现有价值模式后,创建一条条目并记录根因与经验。
|
||||
34
.agents/docs/standards/调试规范.md
Normal file
34
.agents/docs/standards/调试规范.md
Normal file
@@ -0,0 +1,34 @@
|
||||
---
|
||||
last_reviewed: 2026-06-09
|
||||
---
|
||||
# 调试规范
|
||||
|
||||
## 调试前检查清单
|
||||
|
||||
在循环调试任务前,需完成以下检查:
|
||||
|
||||
1. 梳理代码链路(最长耗时 5 分钟)。从入口函数追踪至异常执行环节,排查硬编码值、参数缺失或分支逻辑异常等问题。
|
||||
2. 对比正常与异常场景。若功能 A 运行正常、功能 B 出现故障,梳理二者代码链路的差异,问题通常就出在差异部分。
|
||||
3. 排查基础配置项。检查代理设置、环境变量、端口号、功能开关等。多数故障由配置问题导致,而非代码逻辑错误。
|
||||
|
||||
## 调试过程要求
|
||||
|
||||
1. 两次尝试原则。若同一排查方式(重跑测试、调整参数等)连续失败两次,立即停止,更换排查思路:
|
||||
* 增加针对性日志或打印语句
|
||||
* 阅读异常依赖库的源码
|
||||
* 精简代码,复现最小故障案例
|
||||
* 反思:自身哪些预设判断可能存在偏差
|
||||
2. 禁止无限循环调试。定时任务仅用于监控正常运行的进程,不可作为调试工具。若定时循环连续两轮无进展,关闭循环,转为人工调试。
|
||||
3. 记录排查思路。每开始一次尝试前,做好记录:
|
||||
* 初步判断的问题原因
|
||||
* 用于验证猜想的依据
|
||||
* 本次准备执行的操作
|
||||
* 避免重复无效尝试与逻辑死循环
|
||||
|
||||
## 调试收尾工作
|
||||
|
||||
1. 编写经验文档。所有非简单故障的调试工作,均需在`.agents/docs/error-experience/` 目录下新建记录文档,内容包含:
|
||||
* 故障现象
|
||||
* 历次排查操作及失败原因
|
||||
* 最终解决方案
|
||||
* 后续可借鉴的调试经验
|
||||
17
.agents/skills/README.md
Normal file
17
.agents/skills/README.md
Normal file
@@ -0,0 +1,17 @@
|
||||
---
|
||||
last_reviewed: 2026-06-11
|
||||
---
|
||||
|
||||
# .agents/skills — Cursor Agent 技能目录
|
||||
|
||||
存放 Cursor Agent 可调用的自动化技能定义。
|
||||
|
||||
## 技能清单
|
||||
|
||||
| 技能 | 说明 |
|
||||
|------|------|
|
||||
| `pre-commit-check/` | 提交前代码质量检查:运行 ruff lint/format、mypy 类型检查、deptry 依赖审计,自动修复可修复问题 |
|
||||
|
||||
## 使用方式
|
||||
|
||||
Agent 在用户请求提交代码或检查代码质量时自动触发对应技能,无需手动调用。
|
||||
107
.agents/skills/clean-git-history/SKILL.md
Normal file
107
.agents/skills/clean-git-history/SKILL.md
Normal file
@@ -0,0 +1,107 @@
|
||||
---
|
||||
name: clean-git-history
|
||||
description: >-
|
||||
Remove sensitive files and directories from Git commit history using git-filter-repo.
|
||||
Use when the user wants to remove secrets, credentials, uploaded files, or any sensitive data
|
||||
that was accidentally committed to Git history. Also use when the user mentions cleaning
|
||||
Git history, removing leaked files, or scrubbing sensitive information from repositories.
|
||||
---
|
||||
|
||||
# Clean Git History
|
||||
|
||||
Remove sensitive files from Git history using `git-filter-repo`. This is a destructive operation that rewrites commit history.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Install `git-filter-repo` if not already available:
|
||||
|
||||
```powershell
|
||||
python -m pip install git-filter-repo
|
||||
```
|
||||
|
||||
## Safety Checklist
|
||||
|
||||
Before proceeding, verify:
|
||||
|
||||
- [ ] Local source code is intact (`git log --oneline` shows expected commits)
|
||||
- [ ] Remote repository is accessible (`git fetch origin` succeeds)
|
||||
- [ ] Sensitive files are identified in history (`git log --all --pretty=format: --name-only | Select-String "pattern"`)
|
||||
|
||||
## Step-by-Step Workflow
|
||||
|
||||
### 1. Identify Sensitive Files
|
||||
|
||||
Check what sensitive paths exist in history:
|
||||
|
||||
```powershell
|
||||
git log --all --pretty=format: --name-only | Select-String "\.env|uploads/|images/|scripts/data/|logs/" | Sort-Object -Unique
|
||||
```
|
||||
|
||||
### 2. Clean One Path at a Time
|
||||
|
||||
Remove each sensitive path separately, verifying after each step:
|
||||
|
||||
```powershell
|
||||
# Remove .env from history
|
||||
python -m git_filter_repo --path .env --invert-paths --force
|
||||
|
||||
# Remove uploads directory from history
|
||||
python -m git_filter_repo --path src/web/uploads/ --invert-paths --force
|
||||
|
||||
# Remove images directory from history
|
||||
python -m git_filter_repo --path images/ --invert-paths --force
|
||||
```
|
||||
|
||||
**Critical**: Always use `--invert-paths` to exclude files. Without it, `--path` keeps only those files and deletes everything else.
|
||||
|
||||
### 3. Verify Cleanup
|
||||
|
||||
Confirm sensitive files are gone:
|
||||
|
||||
```powershell
|
||||
git log --all --pretty=format: --name-only | Select-String "\.env|uploads/|images/" | Sort-Object -Unique
|
||||
```
|
||||
|
||||
Result should be empty.
|
||||
|
||||
### 4. Restore Remote and Push
|
||||
|
||||
`git-filter-repo` removes the origin remote. Re-add and force push:
|
||||
|
||||
```powershell
|
||||
# Re-add remote (replace with actual URL)
|
||||
git remote add origin <remote-url>
|
||||
|
||||
# Force push cleaned history
|
||||
git push --force origin <branch-name>
|
||||
```
|
||||
|
||||
If multiple branches exist, push each one:
|
||||
|
||||
```powershell
|
||||
git push --force origin master
|
||||
git push --force origin feature/table
|
||||
```
|
||||
|
||||
### 5. Final Verification
|
||||
|
||||
Verify remote history is clean:
|
||||
|
||||
```powershell
|
||||
git fetch origin
|
||||
git log --all --pretty=format: --name-only | Select-String "\.env|uploads/|images/" | Sort-Object -Unique
|
||||
```
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
| Mistake | Consequence | Fix |
|
||||
|---------|-------------|-----|
|
||||
| Missing `--invert-paths` | Deletes all files except the listed ones | Restore from remote: `git reset --hard origin/<branch>` |
|
||||
| Wrong Python environment | `No module named git_filter_repo` | Use `python -m pip install git-filter-repo` in current environment |
|
||||
| Forgetting to restore remote | Cannot push changes | Re-add remote with `git remote add origin <url>` |
|
||||
|
||||
## Post-Cleanup Actions
|
||||
|
||||
- Rotate any secrets that were exposed in history
|
||||
- Update `.gitignore` to prevent re-committing sensitive files
|
||||
- Notify team members to re-clone the repository (old clones still contain sensitive history)
|
||||
68
.agents/skills/pre-commit-check/SKILL.md
Normal file
68
.agents/skills/pre-commit-check/SKILL.md
Normal file
@@ -0,0 +1,68 @@
|
||||
---
|
||||
name: pre-commit-check
|
||||
description: >-
|
||||
Run pre-commit code quality checks (ruff lint/format, mypy type check, deptry dependency audit)
|
||||
and fix any issues before committing. Use when the user wants to commit code, asks to check code
|
||||
quality, or mentions pre-commit validation. Also use when preparing code for submission or
|
||||
when the user says "check before commit" or "run checks".
|
||||
---
|
||||
|
||||
# Pre-Commit Code Quality Check
|
||||
|
||||
## Workflow
|
||||
|
||||
Run all checks in parallel first, then fix any issues iteratively:
|
||||
|
||||
```
|
||||
Step 1: Run checks (parallel)
|
||||
- uv run ruff check .
|
||||
- uv run ruff format --check .
|
||||
- uv run mypy src/
|
||||
- uv run deptry .
|
||||
|
||||
Step 2: If any check fails, fix issues
|
||||
- ruff check --fix . (auto-fix lint issues)
|
||||
- ruff format . (auto-format)
|
||||
- Fix type annotation errors manually
|
||||
|
||||
Step 3: Re-run all checks to confirm
|
||||
Step 4: Report results with table
|
||||
```
|
||||
|
||||
## Common Fixes
|
||||
|
||||
| Error | Fix |
|
||||
|-------|-----|
|
||||
| `UP038` | Replace `isinstance(e, (X, Y))` with `isinstance(e, X \| Y)` |
|
||||
| `no-untyped-def` | Add return type annotation (`-> None` for void functions) |
|
||||
| `I001` | Run `uv run ruff check --fix .` |
|
||||
| `no-any-return` | Use `cast(Type, expression)` from `typing` |
|
||||
| `type-arg` missing | Add type arguments: `dict[str, Any]` instead of `dict` |
|
||||
| `unused-ignore` | Remove stale `# type: ignore` comments |
|
||||
|
||||
## Type Annotation Rules
|
||||
|
||||
- Functions that modify data in-place and return nothing: `-> None`
|
||||
- Functions that call untyped external APIs: add `-> Any` return type
|
||||
- Dicts that hold mixed types (str + float): use `dict[str, Any]`
|
||||
- Import `Any` from `typing` when needed
|
||||
- Import `cast` from `typing` when `no-any-return` triggers
|
||||
|
||||
## Verification Report
|
||||
|
||||
After all checks pass, report:
|
||||
|
||||
| 检查项 | 状态 |
|
||||
|--------|------|
|
||||
| `ruff check .` | ✅ |
|
||||
| `ruff format --check .` | ✅ |
|
||||
| `mypy src/` | ✅ |
|
||||
| `deptry .` | ✅ |
|
||||
|
||||
## Notes
|
||||
|
||||
- All commands use `uv run` prefix (project virtual environment)
|
||||
- `mypy` has `strict = true` in `pyproject.toml` - expect strict type checking
|
||||
- `ignore_missing_imports = true` is set, so missing third-party stubs are OK
|
||||
- If `tests.*` override shows "unused section" note, it's normal (no tests dir yet)
|
||||
- Never skip a check - all four must pass before committing
|
||||
7
.cursorignore
Normal file
7
.cursorignore
Normal file
@@ -0,0 +1,7 @@
|
||||
.venv/
|
||||
__pycache__/
|
||||
.pytest_cache/
|
||||
.mypy_cache/
|
||||
.ruff_cache/
|
||||
.playwright-mcp/
|
||||
*.pyc
|
||||
13
.env.example
Normal file
13
.env.example
Normal file
@@ -0,0 +1,13 @@
|
||||
# 系统 URL 配置(服务端专用,Web 用户无需配置)
|
||||
# 复制此文件为 .env 并填入实际值
|
||||
|
||||
SSO_LOGIN_URL=https://your-sso-url/login
|
||||
PORTAL_URL=https://your-portal-url/oshall
|
||||
REIMBURSE_URL=http://your-reimburse-host:8081
|
||||
REIMBURSE_PAGE=/expen/common/common?v=4.0
|
||||
TRAVEL_PAGE=/expen/travel/travel?v=4.0
|
||||
|
||||
# LLM 配置
|
||||
LLM_MODEL=your-model-name
|
||||
LLM_API_BASE=http://your-llm-host:port/v1
|
||||
LLM_API_KEY=your-api-key
|
||||
37
.gitignore
vendored
37
.gitignore
vendored
@@ -1,29 +1,14 @@
|
||||
# Python
|
||||
.venv/
|
||||
.cursor/
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.pyo
|
||||
*.egg-info/
|
||||
dist/
|
||||
build/
|
||||
.eggs/
|
||||
|
||||
# Playwright MCP snapshots
|
||||
.pytest_cache/
|
||||
.mypy_cache/
|
||||
.ruff_cache/
|
||||
.playwright-mcp/
|
||||
|
||||
# Uploads (user data)
|
||||
web/uploads/
|
||||
|
||||
# Logs
|
||||
pipeline.log
|
||||
*.log
|
||||
|
||||
# Debug images
|
||||
images/
|
||||
|
||||
# IDE
|
||||
*.pyc
|
||||
logs/
|
||||
.vscode/
|
||||
.idea/
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
uploads/
|
||||
.env
|
||||
images/
|
||||
scripts/data/
|
||||
7
.pre-commit-config.yaml
Normal file
7
.pre-commit-config.yaml
Normal file
@@ -0,0 +1,7 @@
|
||||
repos:
|
||||
- repo: https://github.com/astral-sh/ruff-pre-commit
|
||||
rev: v0.14.6
|
||||
hooks:
|
||||
- id: ruff
|
||||
args: [--fix]
|
||||
- id: ruff-format
|
||||
BIN
1. 电容一批.pdf
BIN
1. 电容一批.pdf
Binary file not shown.
BIN
1. 电容一批.png
BIN
1. 电容一批.png
Binary file not shown.
|
Before Width: | Height: | Size: 204 KiB |
BIN
2. 电阻一批.pdf
BIN
2. 电阻一批.pdf
Binary file not shown.
BIN
2. 电阻一批.png
BIN
2. 电阻一批.png
Binary file not shown.
|
Before Width: | Height: | Size: 201 KiB |
BIN
3. LED一批.pdf
BIN
3. LED一批.pdf
Binary file not shown.
BIN
3. LED一批.png
BIN
3. LED一批.png
Binary file not shown.
|
Before Width: | Height: | Size: 197 KiB |
BIN
4. 元器件盒.pdf
BIN
4. 元器件盒.pdf
Binary file not shown.
BIN
4. 元器件盒.png
BIN
4. 元器件盒.png
Binary file not shown.
|
Before Width: | Height: | Size: 200 KiB |
96
AGENTS.md
Normal file
96
AGENTS.md
Normal file
@@ -0,0 +1,96 @@
|
||||
---
|
||||
description:
|
||||
alwaysApply: true
|
||||
---
|
||||
|
||||
---
|
||||
last_reviewed: 2026-07-02
|
||||
---
|
||||
|
||||
# AGENTS — 项目操作指南
|
||||
|
||||
本文件为 Agent 提供高信号量的项目操作知识,避免重复探索。
|
||||
|
||||
## 文档边界
|
||||
|
||||
* **禁止使用表情文字**输出任何内容。
|
||||
* `docs/` 目录存放面向开源用户、外部贡献者的公开文档。
|
||||
* `.agents/` 目录存放维护规范、实施方案、经验总结等内部资料。
|
||||
* 每个文件夹下都有 `README.md` 说明该文件夹的作用和重要信息。
|
||||
|
||||
## 开发命令(必须使用 uv)
|
||||
|
||||
项目使用 `uv` 管理依赖,所有包版本锁定在 `uv.lock` 中。
|
||||
|
||||
| 操作 | Makefile (跨平台) | tasks.py (Windows) |
|
||||
|------|-------------------|---------------------|
|
||||
| 安装依赖 + pre-commit | `make install` | `python tasks.py install` |
|
||||
| 代码检查(lint+format+typecheck+deptry) | `make check` | `python tasks.py check` |
|
||||
| 运行测试(含覆盖率报告) | `make test` | `python tasks.py test` |
|
||||
| 运行 CLI 全流程 | `make run` | `python tasks.py run` |
|
||||
| 清理缓存和虚拟环境 | `make clean` | `python tasks.py clean` |
|
||||
|
||||
**注意:** `tasks.py` 中的 `check` 命令使用 `&&` 连接,Windows PowerShell 不支持 `&&`,但 `tasks.py` 内部已处理为单行字符串。
|
||||
|
||||
## 代码质量工具链(执行顺序)
|
||||
|
||||
1. **Ruff lint** — `uv run ruff check .` (select: E, F, W, I, N, UP, B; ignore: E501)
|
||||
2. **Ruff format** — `uv run ruff format --check .` (line-length: 120)
|
||||
3. **MyPy strict mode** — `uv run mypy src/main.py` (strict=true, warn_return_any, ignore_missing_imports)
|
||||
4. **deptry** — `uv run deptry .` (检测未声明、未使用、过时依赖)
|
||||
|
||||
### pre-commit 钩子(仅 Ruff)
|
||||
|
||||
`.pre-commit-config.yaml` 配置了两个 hook:
|
||||
- `ruff --fix` — lint 并自动修复
|
||||
- `ruff-format` — 格式化
|
||||
|
||||
**注意:** MyPy 和 deptry **不在** pre-commit 中,需要手动运行 `make check`。
|
||||
|
||||
## 项目架构(Agent 调度模式)
|
||||
|
||||
核心入口:`src/agent/orchestrator.py` — Agent 是负责调度的中枢,协调以下模块:
|
||||
- `extraction/extractor.py` — 文件扫描 → LLM 多模态提取 → JSON 缓存
|
||||
- `matching/matcher.py` — 支付记录与发票金额匹配
|
||||
- `validation/validator.py` — 声明式校验器(规则配置与引擎分离)
|
||||
- `infra/browser/travel.py` / `normal.py` — 浏览器自动化填报
|
||||
|
||||
### 数据流关键产物
|
||||
|
||||
| 文件 | 生成阶段 | 作用 |
|
||||
|------|---------|------|
|
||||
| `.invoice_cache/*.json` | extractor 提取 | 单张发票/支付记录的结构化数据 |
|
||||
| `match_result.json` | matcher 匹配 | 支付截图与发票的关联关系 |
|
||||
| `travel_info.json` / `normal_info.json` | LLM 综合提取 | 差旅/普通报销所需的全部结构化数据 |
|
||||
| `invoice_summary.csv` | extractor 提取 | 普通发票汇总(用于生成易耗品出库单) |
|
||||
|
||||
### 缓存机制
|
||||
|
||||
CLI 模式:`scripts/data/.invoice_cache/`
|
||||
Web 模式:`src/web/uploads/<session_id>/.invoice_cache/`
|
||||
|
||||
缓存文件与源文件同名(如 `发票1.pdf` → `.invoice_cache/发票1.json`),后续步骤均从缓存读取。删除缓存后下次处理会重新提取。
|
||||
|
||||
## 重要约束
|
||||
|
||||
* **Windows-only**:易耗品出库单填写依赖 Microsoft Word + COM (`pywin32`),仅 Windows 可用
|
||||
* **浏览器自动化**:使用 Playwright,填报时会打开 Chromium,请勿手动干扰
|
||||
* **敏感信息**:`scripts/config.json` 含登录凭据,勿提交到公开仓库
|
||||
* **发票类型区分**:差旅发票(高铁票/酒店住宿)不生成易耗品出库单,走差旅报销流程;普通发票生成出库单
|
||||
|
||||
## Web 服务
|
||||
|
||||
```bash
|
||||
uv run python src/web/app.py
|
||||
# 访问 http://localhost:5000
|
||||
```
|
||||
|
||||
Web 端浏览器填报以无头模式运行。会话产物存放在 `src/web/uploads/<session_id>/`,每次上传生成独立会话。
|
||||
|
||||
## 测试
|
||||
|
||||
```bash
|
||||
make test # pytest + coverage report (term-missing)
|
||||
```
|
||||
|
||||
测试目录:`tests/`,配置在 `pyproject.toml` 中 (`testpaths = ["tests"]`, `pythonpath = ["."]`)。
|
||||
21
Makefile
Normal file
21
Makefile
Normal file
@@ -0,0 +1,21 @@
|
||||
.PHONY: install check test run clean
|
||||
|
||||
install:
|
||||
@uv sync
|
||||
@uv run pre-commit install
|
||||
|
||||
check:
|
||||
@uv lock --locked
|
||||
@uv run ruff check .
|
||||
@uv run ruff format --check .
|
||||
@uv run mypy src/main.py
|
||||
@uv run deptry .
|
||||
|
||||
test:
|
||||
@uv run python -m pytest --cov --cov-config=pyproject.toml --cov-report=term-missing
|
||||
|
||||
run:
|
||||
@uv run python src/main.py
|
||||
|
||||
clean:
|
||||
@rm -rf .venv __pycache__ .pytest_cache .mypy_cache .ruff_cache
|
||||
447
README.md
447
README.md
@@ -1,73 +1,231 @@
|
||||
# 财务报销自动化
|
||||
|
||||
自动从 PDF 发票提取信息,OCR 识别支付记录截图,然后在财务系统中自动填报报销单。
|
||||
自动从 PDF 发票或图片中提取信息,生成发票汇总表与易耗品出库单,并可选在财务系统中自动填报报销单。
|
||||
|
||||
**支持发票类型区分**:系统自动识别高铁票、酒店住宿等差旅发票与普通发票。差旅发票不生成易耗品出库单,走差旅报销流程;普通发票生成出库单,走普通报销流程。
|
||||
|
||||
## 项目结构
|
||||
|
||||
```
|
||||
├── run.py # CLI 入口
|
||||
├── config.json # 配置文件(登录凭据、系统 URL 等)
|
||||
├── app/
|
||||
│ ├── config.py # 配置加载
|
||||
│ ├── extractor.py # PDF 发票信息提取
|
||||
│ ├── ocr.py # OCR 刷卡信息识别
|
||||
│ ├── bot.py # 浏览器自动填报
|
||||
│ └── pipeline.py # 流程编排(数据在内存中流转)
|
||||
├── web/
|
||||
│ ├── app.py # Web 服务入口
|
||||
│ ├── templates/
|
||||
│ │ └── index.html # Web 前端页面
|
||||
│ └── uploads/ # 用户上传文件目录
|
||||
├── *.pdf # 发票 PDF(按需放置)
|
||||
├── *.png / *.jpg # 与 PDF 同名的支付截图
|
||||
├── invoice_summary.csv # 中间产物 — 发票汇总表
|
||||
└── images/ # 调试截图
|
||||
├── pyproject.toml # 项目配置(依赖、工具链)
|
||||
├── uv.lock # 依赖锁定文件
|
||||
├── Makefile # 任务脚本(跨平台)
|
||||
├── tasks.py # 任务脚本(Windows 兼容)
|
||||
├── .pre-commit-config.yaml # pre-commit 钩子配置
|
||||
├── .env.example # 环境变量示例(SSO 地址、LLM 配置等)
|
||||
├── config.example.json # 用户配置示例
|
||||
├── 易耗品、出库单.doc # 易耗品出库单 Word 模板
|
||||
├── src/
|
||||
│ ├── __init__.py # 包初始化 / 日志器
|
||||
│ ├── config.py # 配置加载
|
||||
│ ├── exceptions.py # 异常定义
|
||||
│ ├── pipeline.py # CLI 流程编排
|
||||
│ ├── pipeline_core.py # CLI/Web 公共管道逻辑
|
||||
│ ├── main.py # CLI 入口
|
||||
│ ├── agent/ # Agent 调度模块
|
||||
│ │ ├── orchestrator.py # 总调度入口
|
||||
│ │ ├── coordinator.py # 校验-修正循环
|
||||
│ │ ├── session.py # 状态机与会话数据
|
||||
│ │ └── events.py # SSE 事件发射
|
||||
│ ├── core/ # 核心业务逻辑
|
||||
│ │ ├── extraction/ # 信息提取
|
||||
│ │ │ ├── extractor.py # 编排入口:串联文件扫描 → 提取 → 分类
|
||||
│ │ │ └── llm_extractor.py # LLM 多模态信息提取
|
||||
│ │ ├── matching/ # 金额匹配
|
||||
│ │ │ └── matcher.py # 支付记录与发票关联
|
||||
│ │ └── validation/ # 校验模块
|
||||
│ │ └── validator.py # 声明式校验器
|
||||
│ ├── infra/ # 基础设施层
|
||||
│ │ ├── browser/ # 浏览器自动化
|
||||
│ │ │ ├── base.py # BaseBot 基类
|
||||
│ │ │ ├── travel.py # 差旅报销填报流程
|
||||
│ │ │ └── normal.py # 普通报销填报流程
|
||||
│ │ ├── documents/ # 文档处理
|
||||
│ │ │ ├── invoice.py # 发票数据模型 + CSV 工具
|
||||
│ │ │ ├── pdf.py # PDF 图片渲染
|
||||
│ │ │ └── consumable.py # 易耗品出库单填写(Word COM)
|
||||
│ │ └── llm/ # LLM 接口
|
||||
│ │ ├── prompt.py # 提示词加载
|
||||
│ │ └── prompts/ # 提示词模板文件
|
||||
│ └── web/ # Web 界面模块
|
||||
│ ├── app.py # Flask 应用入口
|
||||
│ ├── routes.py # 路由定义
|
||||
│ ├── pipeline_web.py # Web 管道逻辑
|
||||
│ ├── sse_handler.py # SSE 日志流处理
|
||||
│ ├── templates/
|
||||
│ │ ├── index.html # PC 端主页
|
||||
│ │ └── mobile_upload.html # 移动端扫码上传
|
||||
│ ├── static/
|
||||
│ │ ├── css/ # 样式文件
|
||||
│ │ └── js/ # 前端脚本
|
||||
│ └── uploads/ # 按会话隔离的上传与产物目录
|
||||
├── scripts/ # CLI 数据目录
|
||||
│ ├── data/ # 发票源文件、config.json 与 .invoice_cache 缓存
|
||||
│ └── test_*.py # 测试脚本
|
||||
├── tests/ # 测试目录
|
||||
├── docs/ # 用户文档(API 说明、操作指南等)
|
||||
├── images/ # 浏览器调试截图
|
||||
└── *.pdf / *.jpg / *.png # 发票 PDF 或图片(CLI 模式,放在 scripts/data/)
|
||||
```
|
||||
|
||||
## 声明式校验器
|
||||
|
||||
`src/core/validation/validator.py` 采用**规则配置与校验引擎分离**的设计模式,支持声明式定义校验规则:
|
||||
|
||||
### 设计特点
|
||||
|
||||
| 特性 | 说明 |
|
||||
|------|------|
|
||||
| **声明式配置** | 校验规则以数据结构形式定义,无需编写代码 |
|
||||
| **统一路径定位** | 使用 `path` 统一定位字段,如 `["basic_info", "travel_purpose"]` |
|
||||
| **自定义校验函数** | 支持为字段定义自定义校验逻辑(日期格式、正数检查等) |
|
||||
| **数组元素校验** | 支持校验数组字段的最小元素数量及每个元素的必填字段 |
|
||||
| **向后兼容** | 支持简单格式 `["field1", "field2"]` 和详细格式 `{"path": [...], "custom_check": ...}` |
|
||||
|
||||
### 规则配置示例
|
||||
|
||||
```python
|
||||
# 差旅报销校验规则
|
||||
TRAVEL_VALIDATION_RULES = {
|
||||
"fields": [
|
||||
{"path": ["basic_info", "travel_purpose"], "description": "出差事由"},
|
||||
{"path": ["basic_info", "start_date"], "custom_check": _is_valid_date},
|
||||
],
|
||||
"arrays": [
|
||||
{
|
||||
"path": ["payment_methods"],
|
||||
"min_items": 1, # 至少1条支付记录
|
||||
"element_fields": [
|
||||
{"path": ["card_date"], "description": "刷卡日期"},
|
||||
{"path": ["card_amount"], "custom_check": _is_positive_number},
|
||||
],
|
||||
},
|
||||
],
|
||||
}
|
||||
```
|
||||
|
||||
### 校验规则类型
|
||||
|
||||
| 规则类型 | 用途 | 关键字段 |
|
||||
|----------|------|----------|
|
||||
| `fields` | 顶层单值字段校验 | `path`, `custom_check`, `check_empty` |
|
||||
| `arrays` | 数组字段校验 | `path`, `min_items`, `element_fields` |
|
||||
|
||||
### 内置校验函数
|
||||
|
||||
- `_is_valid_date(value)` — 检查日期格式是否为 `YYYY-MM-DD`
|
||||
- `_is_positive_number(value)` — 检查值是否为正数
|
||||
|
||||
### 扩展自定义校验
|
||||
|
||||
```python
|
||||
# 定义自定义校验函数
|
||||
def check_vehicle_type(value):
|
||||
valid_types = ["飞机", "火车", "汽车", "打车"]
|
||||
return isinstance(value, str) and value.strip() in valid_types
|
||||
|
||||
# 在规则中使用
|
||||
{"path": ["vehicle_type"], "custom_check": check_vehicle_type}
|
||||
```
|
||||
|
||||
## 数据流
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
PDF[PDF 发票 / 图片] --> Extract[extractor 多模态提取]
|
||||
Extract --> Cache[(.invoice_cache/*.json)]
|
||||
|
||||
Cache --> Classify{发票类型分类}
|
||||
Classify -->|差旅发票| Travel[高铁票 / 酒店住宿]
|
||||
Classify -->|普通发票| General[普通发票]
|
||||
Classify -->|支付记录| Payment[支付截图]
|
||||
Classify -->|申请单| Application[出差事前申请单]
|
||||
|
||||
Travel --> Matcher[matcher 金额匹配]
|
||||
Payment --> Matcher
|
||||
Matcher --> MatchResult[(match_result.json)]
|
||||
|
||||
Cache --> TravelLLM[LLM 差旅信息提取]
|
||||
MatchResult --> TravelLLM
|
||||
TravelLLM --> TravelInfo[(travel_info.json)]
|
||||
|
||||
Cache --> NormalLLM[LLM 普通发票信息提取]
|
||||
MatchResult --> NormalLLM
|
||||
NormalLLM --> NormalInfo[(normal_info.json)]
|
||||
|
||||
TravelInfo -->|差旅基本信息| Bot_T[infra/browser/travel.py<br/>差旅填报流程]
|
||||
TravelInfo -->|报销明细| Bot_T
|
||||
TravelInfo -->|支付方式| Bot_T
|
||||
TravelInfo -->|补助清单| Bot_T
|
||||
TravelInfo -->|附件清单| Bot_T
|
||||
Bot_T --> Submit_T[差旅报销提交]
|
||||
|
||||
NormalInfo -->|报销说明| Bot_G[infra/browser/normal.py<br/>普通填报流程]
|
||||
NormalInfo -->|发票总数/金额| Bot_G
|
||||
NormalInfo -->|支付方式| Bot_G
|
||||
NormalInfo -->|附件清单| Bot_G
|
||||
Bot_G --> Submit_G[普通报销提交]
|
||||
|
||||
General --> CSV[(invoice_summary.csv)]
|
||||
CSV --> Fill[consumable.py]
|
||||
Fill --> Doc[易耗品、出库单.doc]
|
||||
```
|
||||
PDF 文件 ──► extractor 提取 ──► 发票列表
|
||||
│
|
||||
支付截图 ──► OCR 识别 ────────────► 回填刷卡信息
|
||||
│
|
||||
invoice_summary.csv
|
||||
│
|
||||
bot 打开浏览器 ──► 自动填报
|
||||
```
|
||||
|
||||
### 关键中间产物
|
||||
|
||||
| 文件 | 生成阶段 | 作用 |
|
||||
|------|---------|------|
|
||||
| `.invoice_cache/*.json` | extractor 提取 | 单张发票/支付记录/申请单的结构化数据 |
|
||||
| `match_result.json` | matcher 匹配 | 支付截图与发票的关联关系(按金额匹配) |
|
||||
| `travel_info.json` | LLM 差旅信息提取 | 综合发票缓存 + 匹配结果,生成差旅报销所需的全部结构化数据 |
|
||||
| `normal_info.json` | LLM 普通发票信息提取 | 综合普通发票 + 匹配结果,生成普通报销所需的全部结构化数据 |
|
||||
| `invoice_summary.csv` | extractor 提取 | 普通发票汇总(用于生成易耗品出库单) |
|
||||
|
||||
### bot 模块架构
|
||||
|
||||
`infra/browser/` 包负责浏览器自动化填报,仅接收已提取的信息并执行填报操作,不承担信息提取职责:
|
||||
|
||||
| 模块 | 职责 |
|
||||
|------|------|
|
||||
| `infra/browser/base.py` | `BaseBot` 基类:浏览器生命周期、登录、导航、截图 |
|
||||
| `infra/browser/travel.py` | 差旅填报流程:基本信息 → 差旅明细 → 支付方式 → 补助清单 → 附件上传 |
|
||||
| `infra/browser/normal.py` | 普通填报流程:基本信息 → 总明细 → 支付方式 → 附件上传 |
|
||||
| `infra/browser/__init__.py` | 入口函数:`run_bot()` / `run_bot_web()`,负责类型判断和流程路由 |
|
||||
|
||||
## 环境要求
|
||||
|
||||
- Python 3.10+
|
||||
- 依赖见下方安装步骤
|
||||
- **Python 3.12+**
|
||||
- **uv** 包管理器([安装指南](https://docs.astral.sh/uv/getting-started/installation/))
|
||||
- Windows(易耗品出库单填写依赖 Microsoft Word + COM,仅 Windows 可用)
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 1. 安装依赖
|
||||
|
||||
> 以下依赖需在 MinerU 虚拟环境中安装:
|
||||
> ```bash
|
||||
> conda activate MinerU
|
||||
> ```
|
||||
|
||||
```bash
|
||||
pip install pdfplumber==0.11.9 paddleocr==2.8.1 playwright==1.60.0 flask==3.0.3
|
||||
playwright install chromium
|
||||
# 同步所有依赖(运行时 + 开发工具)
|
||||
make install
|
||||
# Windows 上等效命令:
|
||||
python tasks.py install
|
||||
```
|
||||
|
||||
### 2. 准备数据
|
||||
项目使用 `uv` 管理依赖,所有包版本锁定在 `uv.lock` 中,确保可复现。
|
||||
|
||||
将发票 PDF 和对应的支付截图放在项目根目录下。脚本会自动匹配 PDF 与截图:
|
||||
| 依赖 | 用途 |
|
||||
|------|------|
|
||||
| PyMuPDF | PDF 图片渲染(供多模态 LLM 使用) |
|
||||
| llama-index | LLM 信息提取(发票识别、差旅信息提取) |
|
||||
| playwright | 财务系统浏览器自动化 |
|
||||
| flask | Web 服务 |
|
||||
| pywin32 | 填写 Word 出库单(`fill_consumable_doc`) |
|
||||
|
||||
1. **优先文件名匹配** — PDF 与截图同名(如 `发票.pdf` ↔ `发票.png`)
|
||||
2. **金额近邻匹配** — 文件名不同时,自动提取 PDF 的价税合计和截图的刷卡金额进行配对
|
||||
### 2. 准备数据(CLI 模式)
|
||||
|
||||
截图支持格式:`.png`、`.jpg`、`.jpeg`、`.bmp`、`.webp`。
|
||||
```
|
||||
将发票 PDF 或图片(`.jpg`、`.png`、`.webp`、`.bmp`)放在 `scripts/data/` 目录下。
|
||||
|
||||
### 3. 配置
|
||||
|
||||
编辑 `config.json`,填写登录凭据和默认值:
|
||||
编辑 `scripts/config.json`(参考 `config.example.json`):
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -75,72 +233,207 @@ playwright install chromium
|
||||
"password": "你的密码",
|
||||
"default_name": "默认报销人姓名",
|
||||
"default_card_no": "默认公务卡号",
|
||||
"default_person_id": "默认人员编号"
|
||||
"default_person_id": "默认人员编号",
|
||||
"consumable_storage": "物料存储地"
|
||||
}
|
||||
```
|
||||
|
||||
未填写的字段将使用默认值,URL 类配置一般无需修改。
|
||||
| 字段 | 说明 |
|
||||
|------|------|
|
||||
| `username` / `password` | 信息门户登录凭据 |
|
||||
| `default_name` | 默认报销人姓名 |
|
||||
| `default_card_no` | 默认公务卡号 |
|
||||
| `default_person_id` | 默认人员编号(工号) |
|
||||
| `consumable_storage` | 出库单「存放地点」列默认值(默认: `躬行楼 C205`) |
|
||||
|
||||
### 4. 运行
|
||||
**服务端配置**(SSO 地址、报销系统 URL、LLM 参数)通过环境变量提供,有默认值,一般无需修改:
|
||||
|
||||
| 环境变量 | 默认值 | 说明 |
|
||||
|----------|--------|------|
|
||||
| `SSO_LOGIN_URL` | `https://tyrz.fynu.edu.cn/sso/login` | SSO 登录地址 |
|
||||
| `PORTAL_URL` | `https://tyrz.fynu.edu.cn/oshall` | 统一信息平台地址 |
|
||||
| `REIMBURSE_URL` | `http://210.45.32.214:8081` | 报销系统地址 |
|
||||
| `REIMBURSE_PAGE` | `/expen/common/common?v=4.0` | 普通报销页面路径 |
|
||||
| `TRAVEL_PAGE` | `/expen/travel/travel?v=4.0` | 差旅报销页面路径 |
|
||||
| `LLM_MODEL` | `qwen-vl-max` | LLM 模型名称 |
|
||||
| `LLM_API_BASE` | `http://localhost:8080/v1` | LLM API 地址 |
|
||||
| `LLM_API_KEY` | `lm-studio` | LLM API 密钥 |
|
||||
|
||||
### 4. 运行(CLI)
|
||||
|
||||
```bash
|
||||
# 全流程(发票提取 → OCR 识别 → 浏览器填报)
|
||||
python run.py
|
||||
# 全流程(发票提取 → 浏览器填报)
|
||||
make run
|
||||
# Windows 等效:
|
||||
python tasks.py run
|
||||
|
||||
# 仅执行某一步
|
||||
python run.py --step invoice # 仅发票提取
|
||||
python run.py --step ocr # 仅 OCR 识别
|
||||
python run.py --step submit # 仅浏览器填报
|
||||
uv run python src/main.py --step invoice # 仅发票提取
|
||||
uv run python src/main.py --step submit # 仅浏览器填报
|
||||
|
||||
# 覆盖配置中的登录凭据
|
||||
python run.py -u 工号 -p 密码
|
||||
uv run python src/main.py -u 工号 -p 密码
|
||||
```
|
||||
|
||||
### 5. 填写易耗品出库单(CLI)
|
||||
|
||||
需已生成 `invoice_summary.csv`,且本机已安装 **Microsoft Word**:
|
||||
|
||||
```bash
|
||||
uv run python -m src.infra.documents.consumable
|
||||
uv run python -m src.infra.documents.consumable --csv invoice_summary.csv --doc "易耗品、出库单.doc"
|
||||
uv run python -m src.infra.documents.consumable --config scripts/config.json # 指定配置文件
|
||||
uv run python -m src.infra.documents.consumable --no-backup # 不生成 .doc.bak 备份
|
||||
```
|
||||
|
||||
填写规则概要:
|
||||
|
||||
- 表头「日期」使用**填写当天**的日期(非发票开票日期)
|
||||
- 从 `spec_model` 解析品名、规格、单位、数量、单价;`card_amount` 写入金额列
|
||||
- 单价/金额保留两位小数;表格内统一为 **宋体五号(10.5 磅)**
|
||||
- 存放地点取自 `consumable_storage`(默认: `躬行楼 C205`);购货人/领用人签字、备注保持空白
|
||||
|
||||
## 执行步骤说明
|
||||
|
||||
| 步骤 | 命令 | 说明 |
|
||||
|------|------|------|
|
||||
| 发票提取 | `--step invoice` | 扫描根目录 PDF,提取发票号码、金额、销售方等信息,生成 `invoice_summary.csv` 和 `.md` |
|
||||
| OCR 识别 | `--step ocr` | 对支付截图执行 OCR,识别刷卡日期、刷卡金额、持卡人姓名,回填到 CSV |
|
||||
| 浏览器填报 | `--step submit` | 打开浏览器,登录信息门户 → 进入报销系统 → 自动填单、录入明细、上传附件 |
|
||||
| 发票提取 | `--step invoice` | 扫描 `scripts/data/` 目录的 PDF 和图片,生成 `invoice_summary.csv` |
|
||||
| 浏览器填报 | `--step submit` | 登录信息门户 → 报销系统 → 自动填单、上传附件 |
|
||||
|
||||
> 分步执行时,上一步的 CSV 产物会自动成为下一步的输入。
|
||||
> 全流程执行时数据在内存中流转,CSV 为参考产物。分步执行时,缓存数据会自动成为下一步的输入。
|
||||
|
||||
### 缓存机制
|
||||
|
||||
系统使用 JSON 缓存作为数据中转站,串联整个处理流程:
|
||||
|
||||
```
|
||||
源文件 (PDF/图片) → LLM 多模态提取 → JSON 缓存 → 匹配/分类/填报
|
||||
```
|
||||
|
||||
| 缓存文件 | 说明 |
|
||||
|----------|------|
|
||||
| `<文件名>.json` | 每个源文件的 LLM 提取结果(发票信息、支付记录等) |
|
||||
| `match_result.json` | 发票与支付记录的匹配结果 |
|
||||
| `travel_info.json` | 差旅信息(事由、地点、时间等)提取结果 |
|
||||
|
||||
缓存位置:CLI 模式为 `scripts/data/.invoice_cache/`,Web 模式为 `src/web/uploads/<session_id>/.invoice_cache/`。缓存文件与源文件同名(如 `发票1.pdf` 对应 `.invoice_cache/发票1.json`),后续步骤(金额匹配、发票分类、浏览器填报)均从缓存读取结构化数据。删除缓存后下次处理会重新提取。
|
||||
|
||||
### CSV 字段说明
|
||||
|
||||
发票级别 CSV(`invoice_summary.csv`)使用英文列名:
|
||||
|
||||
| 列名 | 说明 |
|
||||
|------|------|
|
||||
| `index` | 行号 |
|
||||
| `invoice_type` | 发票类型:`train` / `hotel` / `general` |
|
||||
| `invoice_number` | 电子发票号码 |
|
||||
| `invoice_date` | 开票日期 |
|
||||
| `item_name` | 货物或应税劳务名称 |
|
||||
| `spec_model` | 规格型号(差旅发票为出发站→到达站) |
|
||||
| `total_amount` | 发票含税金额 |
|
||||
| `seller_name` | 销方名称 |
|
||||
| `departure` / `arrival` | 出发站 / 到达站(高铁票专用) |
|
||||
| `train_no` / `ride_date` / `seat_class` | 车次 / 乘车日期 / 座位等级(高铁票专用) |
|
||||
| `person_name` | 人员姓名 |
|
||||
| `card_date` / `card_no` / `card_amount` | 刷卡日期 / 公务卡号 / 刷卡金额 |
|
||||
| `remark` | 备注 |
|
||||
| `person_id` | 工号 |
|
||||
|
||||
## Web 服务
|
||||
|
||||
提供浏览器界面,上传文件即可自动处理:
|
||||
提供浏览器界面:上传文件 → 自动处理 → 在线编辑 → 下载产物 → 可选提交财务系统。
|
||||
|
||||
```bash
|
||||
python web/app.py
|
||||
uv run python src/web/app.py
|
||||
```
|
||||
|
||||
访问 `http://localhost:5000`,上传文件并填写配置后点击「开始处理」。
|
||||
访问 `http://localhost:5000`。
|
||||
|
||||
### 两种处理模式
|
||||
### 推荐使用流程
|
||||
|
||||
| 模式 | 入口 | 说明 |
|
||||
|------|------|------|
|
||||
| **PDF 模式** | 上传 PDF + 截图 | 自动提取发票信息 → OCR 识别刷卡记录 → 生成 CSV → 可选浏览器填报 |
|
||||
| **CSV 快捷模式** | 上传已有 CSV 文件 | 跳过提取和 OCR,直接使用 CSV 数据进行浏览器填报 |
|
||||
1. 上传 PDF 或图片(或上传已有 CSV)
|
||||
2. 填写配置(账号、密码、姓名、公务卡号、存放地点等),可上传 `config.json` 一键填充
|
||||
3. 点击 **开始处理** — 完成发票提取、生成 CSV,系统自动识别发票类型并分类统计
|
||||
4. **普通发票**:自动生成 **易耗品、出库单.doc**,可下载
|
||||
5. **差旅发票**(高铁票/酒店住宿):跳过出库单生成,直接进入差旅报销流程
|
||||
6. 在表格中核对、修改发票数据(提交财务系统前会自动保存)
|
||||
7. 确认无误后点击 **提交到财务系统**
|
||||
|
||||
### 功能说明
|
||||
### 处理流程
|
||||
|
||||
上传 PDF 或图片 → LLM 识别文档类型 → 结构化提取 → 分类处理
|
||||
|
||||
系统通过 LLM 多模态识别自动判断每张文档的类型,无需手动指定:
|
||||
|
||||
| 文档类型 | 处理方式 | 状态 |
|
||||
|----------|----------|------|
|
||||
| 发票(高铁票/酒店住宿/普通发票) | 提取发票信息 → 金额匹配 → 分类 | 已实现 |
|
||||
| 支付记录(刷卡截图) | 提取刷卡信息 → 与发票匹配 | 已实现 |
|
||||
| 出差事前申请单 | 提取出差事由、地点、时间 | 已实现 |
|
||||
| 飞机票 | 同高铁票处理流程 | 计划中 |
|
||||
|
||||
### 功能一览
|
||||
|
||||
| 功能 | 说明 |
|
||||
|------|------|
|
||||
| 发票提取 + OCR | 上传 PDF 后自动完成,无需手动操作 |
|
||||
| CSV 快捷上传 | 已有发票数据 CSV 可直接上传,跳过前面的步骤 |
|
||||
| 浏览器填报 | 勾选「同时提交到财务系统」后自动运行 |
|
||||
| 实时日志 | 处理进度通过 SSE 实时推送 |
|
||||
| 下载 CSV | 处理后下载发票汇总表 |
|
||||
| 配置上传 | 可上传 `config.json` 自动填充表单 |
|
||||
| 发票提取 | 上传 PDF 或图片后自动完成 |
|
||||
| 发票类型自动分类 | 高铁票/酒店住宿/普通发票,自动分流处理 |
|
||||
| 易耗品出库单 | 仅普通发票自动生成 Word,差旅发票跳过 |
|
||||
| 表格在线编辑 | 处理完成后可修改 CSV 各字段;保存后重新生成出库单 |
|
||||
| 财务系统填报 | 单独按钮触发,处理阶段不会自动提交 |
|
||||
| 实时日志 | SSE 推送处理进度 |
|
||||
| 配置上传 | 支持上传 `config.json` 填充表单 |
|
||||
| 手机扫码上传 | 二维码打开移动端页面,拍照上传,PC 端轮询同步 |
|
||||
|
||||
> Web 模式下浏览器以无头模式运行,不会弹出窗口。
|
||||
> 若未上传 PDF 附件,浏览器填报阶段将自动跳过附件上传步骤。
|
||||
> Web 端浏览器填报以无头模式运行。未上传 PDF 或图片时,填报阶段会跳过附件上传。
|
||||
> 出库单生成需要 **Windows + Word + pywin32**;若失败,页面会显示具体原因,CSV 等其它产物仍可正常使用。
|
||||
|
||||
### 会话产物
|
||||
|
||||
每次上传生成独立会话,产物存放在 `src/web/uploads/<session_id>/`:
|
||||
|
||||
| 产物 | 说明 |
|
||||
|------|------|
|
||||
| `invoice_summary.csv` | 发票汇总数据(含 LLM 识别结果) |
|
||||
| `payment_records.csv` | 支付记录级别数据(含匹配结果) |
|
||||
| `travel_applications.json` | 出差事前申请单数据(JSON 格式) |
|
||||
| `易耗品、出库单.doc` | 自动填写的出库单(仅普通发票) |
|
||||
| `config.json` | 当次会话配置 |
|
||||
| `session.log` | 处理日志 |
|
||||
| `result.json` | 处理结果 |
|
||||
| `.invoice_cache/` | LLM 提取结果缓存(JSON 格式,避免重复处理) |
|
||||
|
||||
会话缓存机制与 CLI 模式相同,缓存目录中的 JSON 数据是后续匹配、分类和填报的唯一数据来源。
|
||||
|
||||
接口说明见 [API.md](./API.md)。
|
||||
|
||||
## 开发任务
|
||||
|
||||
项目提供统一的任务脚本,支持跨平台使用:
|
||||
|
||||
| 任务 | Makefile | tasks.py | 说明 |
|
||||
|------|----------|----------|------|
|
||||
| 安装依赖 | `make install` | `python tasks.py install` | 同步依赖 + 安装 pre-commit |
|
||||
| 代码检查 | `make check` | `python tasks.py check` | Ruff lint + 格式化 + MyPy 类型检查 + deptry 依赖检查 |
|
||||
| 运行测试 | `make test` | `python tasks.py test` | pytest + 覆盖率报告 |
|
||||
| 运行 CLI | `make run` | `python tasks.py run` | 执行全流程 |
|
||||
| 清理缓存 | `make clean` | `python tasks.py clean` | 删除虚拟环境和缓存 |
|
||||
|
||||
## 代码质量
|
||||
|
||||
项目配置了完整的代码质量工具链:
|
||||
|
||||
- **Ruff** — 快速 lint 检查和代码格式化(替代 flake8 + isort + black)
|
||||
- **MyPy** — 严格模式类型检查(`strict = true`)
|
||||
- **deptry** — 检测未声明、未使用、过时依赖
|
||||
- **pre-commit** — 提交前自动运行 Ruff 检查和格式化
|
||||
|
||||
所有检查通过后方可提交代码。
|
||||
|
||||
## 注意事项
|
||||
|
||||
- 第三步会打开浏览器窗口,请勿关闭或切换标签页
|
||||
- 首次运行可能需要手动处理 SSO 登录(如已保存会话则跳过)
|
||||
- 调试截图保存在 `images/` 目录,出错时可查看
|
||||
- `invoice_summary.csv` 中空白的字段会在 OCR 步骤自动回填,不会覆盖已有数据
|
||||
- 提交按钮默认未启用,确认数据无误后可在 `app/bot.py` 中取消注释 `bot.submit()`
|
||||
- 浏览器填报时会打开或使用 Chromium,请勿手动干扰自动化流程
|
||||
- 调试截图保存在 `images/` 目录
|
||||
- 项目根目录需保留 `易耗品、出库单.doc` 模板;Web 每次从模板复制到会话目录再填写,不修改原模板
|
||||
- `scripts/config.json` 含敏感信息,请勿提交到公开仓库
|
||||
- **发票类型区分**:差旅发票(高铁票/酒店住宿)不会生成易耗品出库单,差旅报销填报流程已完整实现(含差旅信息提取、明细录入、支付方式、补助清单、附件上传)
|
||||
@@ -1,35 +0,0 @@
|
||||
"""财务报销自动化工具包"""
|
||||
|
||||
import io
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_LOG_FMT = "%(asctime)s [%(levelname)-5s] %(name)s: %(message)s"
|
||||
_LOG_DATE_FMT = "%Y-%m-%d %H:%M:%S"
|
||||
_LOG_FILE = Path(__file__).resolve().parent.parent / "pipeline.log"
|
||||
|
||||
|
||||
def get_logger(name: str) -> logging.Logger:
|
||||
"""获取带时间戳的日志记录器
|
||||
|
||||
输出格式: 2026-05-24 12:34:56 [INFO ] extractor: 扫描目录: ...
|
||||
日志同时输出到终端和项目根目录的 pipeline.log
|
||||
"""
|
||||
logger = logging.getLogger(name)
|
||||
if not logger.handlers:
|
||||
logger.setLevel(logging.INFO)
|
||||
formatter = logging.Formatter(_LOG_FMT, _LOG_DATE_FMT)
|
||||
|
||||
# 终端输出
|
||||
utf8_stream = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8", errors="replace")
|
||||
stream_handler = logging.StreamHandler(utf8_stream)
|
||||
stream_handler.setFormatter(formatter)
|
||||
logger.addHandler(stream_handler)
|
||||
|
||||
# 文件输出
|
||||
file_handler = logging.FileHandler(str(_LOG_FILE), encoding="utf-8")
|
||||
file_handler.setFormatter(formatter)
|
||||
logger.addHandler(file_handler)
|
||||
|
||||
return logger
|
||||
435
app/bot.py
435
app/bot.py
@@ -1,435 +0,0 @@
|
||||
"""
|
||||
浏览器自动化填报
|
||||
|
||||
使用 Playwright 操作财务报销系统,自动完成登录、填单、上传附件等操作。
|
||||
|
||||
对外接口:
|
||||
load_invoice_data(csv_path, config) -> list[dict] 从 CSV 加载并补全默认值
|
||||
run_bot(config, invoices) 启动浏览器并执行填报流程
|
||||
"""
|
||||
|
||||
import csv
|
||||
from pathlib import Path
|
||||
|
||||
from . import get_logger
|
||||
|
||||
log = get_logger("bot")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 日期格式化
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _format_date(date_str: str) -> str:
|
||||
"""将 '2026/5/13' 或 '2026-5-13' 转为 '2026-05-13'"""
|
||||
if not date_str:
|
||||
return ""
|
||||
parts = date_str.replace("-", "/").split("/")
|
||||
if len(parts) == 3:
|
||||
return f"{parts[0].zfill(4)}-{parts[1].zfill(2)}-{parts[2].zfill(2)}"
|
||||
return date_str
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# CSV 数据加载
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def load_invoice_data(csv_path: str, config: dict) -> list[dict]:
|
||||
"""从 CSV 加载发票数据,自动补全空白字段的默认值"""
|
||||
invoices = []
|
||||
with open(csv_path, encoding="utf-8") as f:
|
||||
reader = csv.DictReader(f)
|
||||
for row in reader:
|
||||
invoices.append({
|
||||
"seq": row.get("序号", ""),
|
||||
"invoice_no": row.get("发票号码", ""),
|
||||
"invoice_date": row.get("开票日期", ""),
|
||||
"item_name": row.get("项目名称", ""),
|
||||
"spec_model": row.get("规格型号", ""),
|
||||
"total_amount": float(row.get("价税合计", 0)),
|
||||
"seller_name": row.get("销售方名称", ""),
|
||||
"person_name": row.get("人员姓名") or config.get("default_name", ""),
|
||||
"card_date": _format_date(row.get("刷卡日期") or ""),
|
||||
"card_no": row.get("公务卡号") or config.get("default_card_no", ""),
|
||||
"card_amount": float(row.get("刷卡金额") or "0"),
|
||||
"remark": row.get("备注") or "",
|
||||
"person_id": row.get("工号") or config.get("default_person_id", ""),
|
||||
})
|
||||
return invoices
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 报销机器人
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
class ReimburseBot:
|
||||
"""财务报销自动化机器人"""
|
||||
|
||||
def __init__(self, config: dict, headless: bool = False):
|
||||
self.config = config
|
||||
self.headless = headless
|
||||
self.work_dir: Path | None = None
|
||||
self.browser = None
|
||||
self.context = None
|
||||
self.page = None
|
||||
|
||||
from playwright.sync_api import sync_playwright
|
||||
self._pw_ctx = sync_playwright()
|
||||
self.pw = self._pw_ctx.__enter__()
|
||||
|
||||
def launch(self):
|
||||
"""启动浏览器"""
|
||||
self.browser = self.pw.chromium.launch(headless=self.headless)
|
||||
self.context = self.browser.new_context(viewport={"width": 1920, "height": 1080})
|
||||
self.page = self.context.new_page()
|
||||
self.page.set_default_timeout(30000)
|
||||
|
||||
def login_portal(self):
|
||||
"""登录信息门户"""
|
||||
log.info("登录信息门户...")
|
||||
|
||||
self.page.goto(self.config["sso_login_url"], wait_until="domcontentloaded")
|
||||
self._wait_for('text="微信扫码登录"', timeout=5000)
|
||||
|
||||
try:
|
||||
self.page.fill('input[placeholder*="工号"], input[placeholder*="学号"]', self.config["username"])
|
||||
self.page.fill('input[placeholder*="密码"]', self.config["password"])
|
||||
except Exception:
|
||||
log.warning("未找到登录输入框,可能已登录")
|
||||
|
||||
try:
|
||||
checkbox = self.page.query_selector('input[type="checkbox"]')
|
||||
if checkbox and not checkbox.is_checked():
|
||||
checkbox.click()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
for selector in ['button:has-text("登录")', 'input[value="登录"]', 'text="登录"']:
|
||||
try:
|
||||
self.page.click(selector, timeout=3000)
|
||||
break
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
self._wait_for_portal()
|
||||
|
||||
def _wait_for_portal(self):
|
||||
"""等待跳转到统一信息平台"""
|
||||
for _ in range(30):
|
||||
self.page.wait_for_timeout(1000)
|
||||
url = self.page.url
|
||||
if any(kw in url for kw in ("tyrz.fynu.edu.cn/zs-uip", "tyrz.fynu.edu.cn/oshall", "portal")):
|
||||
self._screenshot("portal_loaded")
|
||||
return
|
||||
log.error("等待门户跳转超时")
|
||||
self._screenshot("portal_timeout")
|
||||
raise TimeoutError("登录超时,未跳转到信息门户")
|
||||
|
||||
def navigate_to_reimburse(self):
|
||||
"""从统一信息平台进入报销系统"""
|
||||
log.info("进入报销系统...")
|
||||
self._wait_for('text="快捷入口"', timeout=5000)
|
||||
|
||||
try:
|
||||
self.page.click('text="财务系统"', timeout=5000)
|
||||
except Exception:
|
||||
log.warning("未找到财务系统入口")
|
||||
|
||||
new_tab = None
|
||||
for _ in range(15):
|
||||
self.page.wait_for_timeout(1000)
|
||||
for p in self.context.pages:
|
||||
if "dddl" in p.url or "210.45.32.214" in p.url:
|
||||
new_tab = p
|
||||
break
|
||||
if new_tab:
|
||||
break
|
||||
|
||||
if new_tab:
|
||||
self.page = new_tab
|
||||
self._wait_for('text="网络报销"', timeout=5000)
|
||||
else:
|
||||
log.warning(f"未找到单点登录页面,当前 URL: {self.page.url}")
|
||||
|
||||
for p in self.context.pages[:-1]:
|
||||
try:
|
||||
p.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
link = self.page.query_selector('a:has(img[src*="wlbx"])')
|
||||
if link:
|
||||
reimburse_url = link.get_attribute("href")
|
||||
self.page.goto(reimburse_url, wait_until="domcontentloaded", timeout=15000)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
self._wait_for('text="报销录入"', timeout=5000)
|
||||
common_url = self.config["reimburse_url"] + self.config["reimburse_page"]
|
||||
self.page.goto(common_url, wait_until="domcontentloaded", timeout=15000)
|
||||
self._wait_for('text="单据状态:"', timeout=5000)
|
||||
|
||||
def open_reimburse_menu(self):
|
||||
"""点击「新增」创建新报销单"""
|
||||
log.info("创建新报销单...")
|
||||
self.page.wait_for_timeout(2000)
|
||||
|
||||
try:
|
||||
self.page.click('button:has-text("新增")', timeout=5000)
|
||||
except Exception:
|
||||
try:
|
||||
self.page.click("text=新增", timeout=3000)
|
||||
except Exception:
|
||||
self._screenshot("no_add_button")
|
||||
raise RuntimeError("无法点击新增按钮")
|
||||
|
||||
self.page.wait_for_timeout(3000)
|
||||
self._screenshot("after_add_click")
|
||||
|
||||
def fill_basic_info(self, description: str = "元器件采购报销"):
|
||||
"""填写基本信息"""
|
||||
log.info("填写基本信息...")
|
||||
|
||||
try:
|
||||
self.page.fill("#EXPENEXPLAIN", description)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
self.page.click("#PROJECTCODE", timeout=10000)
|
||||
self.page.wait_for_timeout(1000)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
self._screenshot("step3_project_modal")
|
||||
|
||||
try:
|
||||
self.page.wait_for_selector("#promodal .fixed-table-body tbody tr", timeout=10000)
|
||||
first_row = self.page.query_selector("#promodal .fixed-table-body tbody tr")
|
||||
if first_row:
|
||||
first_row.click()
|
||||
self.page.wait_for_timeout(1000)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
self._screenshot("step3_project_selected")
|
||||
|
||||
try:
|
||||
self.page.click("#saveAndNext", timeout=5000)
|
||||
self.page.wait_for_timeout(2000)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
self._screenshot("step3_done")
|
||||
|
||||
def add_reimburse_items(self, invoices: list[dict]):
|
||||
"""录入报销明细(一条总明细)"""
|
||||
card_amount = sum(inv["card_amount"] for inv in invoices)
|
||||
log.info(f"录入报销明细 (合计 ¥{card_amount:.2f})...")
|
||||
|
||||
try:
|
||||
self.page.click("#insertDetail", timeout=5000)
|
||||
self.page.wait_for_timeout(1000)
|
||||
self._wait_for('text="经济事项名称"', timeout=5000)
|
||||
self.page.click("#economicscode2")
|
||||
self.page.wait_for_timeout(1000)
|
||||
|
||||
try:
|
||||
self.page.wait_for_selector("#econmodal .fixed-table-body tbody tr", timeout=10000)
|
||||
rows = self.page.query_selector_all("#econmodal .fixed-table-body tbody tr")
|
||||
if len(rows) >= 3:
|
||||
rows[2].click()
|
||||
self.page.wait_for_timeout(1000)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
self.page.fill('input[name="expenPwCommondetail.HOWBILLS"]', f"{len(invoices)}")
|
||||
self.page.fill("#je_zwzcdz", f"{card_amount:.2f}")
|
||||
self.page.click("#detailAdd", timeout=3000)
|
||||
self.page.wait_for_timeout(1000)
|
||||
|
||||
self._screenshot("item_total")
|
||||
except Exception as e:
|
||||
log.error(f"录入总明细失败: {e}")
|
||||
self._screenshot("item_total_error")
|
||||
raise
|
||||
|
||||
def fill_payment(self, invoices: list[dict]):
|
||||
"""录入支付信息"""
|
||||
log.info("录入支付信息...")
|
||||
try:
|
||||
self.page.click('text="下一步(支付方式)"', timeout=5000)
|
||||
self._wait_for('text="下一步(附件清单)"', timeout=5000)
|
||||
|
||||
for inv in invoices:
|
||||
self.page.click("#insertPay", timeout=5000)
|
||||
self.page.wait_for_timeout(1000)
|
||||
self.page.fill("#personid2", inv["person_id"])
|
||||
self.page.fill("#accountname2", inv["person_name"])
|
||||
self.page.fill("#receiptdate2", inv["card_date"])
|
||||
self.page.fill("#localaccount2", inv["card_no"])
|
||||
self.page.fill("#receiptmoney2", str(inv["card_amount"]))
|
||||
self.page.fill("#money2", str(inv["card_amount"]))
|
||||
self.page.fill("#merchant2", inv["seller_name"])
|
||||
self.page.fill("#smark2", inv["remark"])
|
||||
self.page.click("#payAdd", timeout=3000)
|
||||
self.page.wait_for_timeout(1000)
|
||||
except Exception as e:
|
||||
log.error(f"支付方式录入失败: {e}")
|
||||
self._screenshot("step5_error")
|
||||
raise
|
||||
|
||||
self._screenshot("step5_done")
|
||||
|
||||
def upload_attachments(self, invoices: list[dict]):
|
||||
"""上传发票附件"""
|
||||
log.info("上传附件...")
|
||||
|
||||
try:
|
||||
self.page.click("#next3", timeout=5000)
|
||||
self._wait_for("#submit2", timeout=5000)
|
||||
|
||||
attachment_files = sorted((self.work_dir or Path(__file__).parent.parent).glob("*.pdf"))
|
||||
if not attachment_files:
|
||||
log.warning("未找到附件 PDF,跳过附件上传")
|
||||
return
|
||||
|
||||
for i, inv in enumerate(invoices):
|
||||
file_path = attachment_files[i] if i < len(attachment_files) else None
|
||||
|
||||
self.page.click("#insertAcc", timeout=5000)
|
||||
self.page.wait_for_timeout(1000)
|
||||
|
||||
try:
|
||||
self._wait_for("#fjlx", timeout=5000)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
self.page.select_option("#fjlx", "1")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
explanation = f"{inv['item_name']} - {inv['invoice_no']}"
|
||||
self.page.fill("#fpsmxx", explanation)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
if file_path and file_path.exists():
|
||||
try:
|
||||
self.page.set_input_files("#file", str(file_path))
|
||||
self.page.wait_for_timeout(1000)
|
||||
except Exception as e:
|
||||
log.error(f"文件上传失败: {e}")
|
||||
|
||||
try:
|
||||
self.page.click("#cjtj", timeout=5000)
|
||||
self.page.wait_for_timeout(1500)
|
||||
except Exception:
|
||||
try:
|
||||
self.page.press("body", "Escape")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
except Exception as e:
|
||||
log.error(f"附件上传失败: {e}")
|
||||
self._screenshot("step6_error")
|
||||
raise
|
||||
|
||||
self._screenshot("step6_done")
|
||||
|
||||
def submit(self):
|
||||
"""提交报销单"""
|
||||
log.info("提交报销单...")
|
||||
try:
|
||||
self.page.click("#submit", timeout=5000)
|
||||
self.page.wait_for_timeout(1000)
|
||||
self._screenshot("submitted")
|
||||
except Exception as e:
|
||||
log.error(f"提交失败: {e}")
|
||||
self._screenshot("submit_error")
|
||||
raise
|
||||
|
||||
def close(self):
|
||||
"""关闭浏览器"""
|
||||
if self.context:
|
||||
self.context.close()
|
||||
if self.browser:
|
||||
self.browser.close()
|
||||
try:
|
||||
self._pw_ctx.__exit__(None, None, None)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# --------------------------------------------------------
|
||||
# 辅助方法
|
||||
# --------------------------------------------------------
|
||||
|
||||
def _wait_for(self, selector: str, timeout: int = None):
|
||||
self.page.wait_for_selector(selector, timeout=timeout)
|
||||
|
||||
def _screenshot(self, name: str):
|
||||
img_dir = Path(__file__).parent.parent / "images"
|
||||
img_dir.mkdir(exist_ok=True)
|
||||
self.page.screenshot(path=str(img_dir / f"debug_{name}.png"))
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 对外入口
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def run_bot(config: dict, invoices: list[dict], headless: bool = False, work_dir: Path | None = None):
|
||||
"""执行完整的浏览器填报流程"""
|
||||
if not config["username"] or not config["password"]:
|
||||
raise ValueError("缺少用户名或密码")
|
||||
|
||||
bot = ReimburseBot(config, headless=headless)
|
||||
bot.work_dir = work_dir
|
||||
try:
|
||||
bot.launch()
|
||||
bot.login_portal()
|
||||
bot.navigate_to_reimburse()
|
||||
bot.open_reimburse_menu()
|
||||
bot.fill_basic_info()
|
||||
bot.add_reimburse_items(invoices)
|
||||
bot.fill_payment(invoices)
|
||||
bot.upload_attachments(invoices)
|
||||
# bot.submit() # 确认无误后再取消注释
|
||||
except Exception as e:
|
||||
log.error(f"操作失败: {e}")
|
||||
try:
|
||||
bot._screenshot("error")
|
||||
except Exception:
|
||||
pass
|
||||
raise
|
||||
finally:
|
||||
bot.close()
|
||||
|
||||
|
||||
def run_bot_web(config: dict, invoices: list[dict], work_dir: Path):
|
||||
"""Web 模式填报 — headless,附件从指定目录读取"""
|
||||
if not config["username"] or not config["password"]:
|
||||
raise ValueError("缺少用户名或密码")
|
||||
|
||||
bot = ReimburseBot(config, headless=True)
|
||||
bot.work_dir = work_dir
|
||||
try:
|
||||
bot.launch()
|
||||
bot.login_portal()
|
||||
bot.navigate_to_reimburse()
|
||||
bot.open_reimburse_menu()
|
||||
bot.fill_basic_info()
|
||||
bot.add_reimburse_items(invoices)
|
||||
bot.fill_payment(invoices)
|
||||
bot.upload_attachments(invoices)
|
||||
except Exception as e:
|
||||
log.error(f"操作失败: {e}")
|
||||
try:
|
||||
bot._screenshot("error")
|
||||
except Exception:
|
||||
pass
|
||||
raise
|
||||
finally:
|
||||
bot.close()
|
||||
@@ -1,33 +0,0 @@
|
||||
"""
|
||||
配置加载
|
||||
|
||||
从项目根目录的 config.json 读取配置,返回结构化的配置字典。
|
||||
"""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
_CONFIG_PATH = Path(__file__).parent.parent / "config.json"
|
||||
|
||||
|
||||
def load_config() -> dict:
|
||||
"""加载并合并配置,缺失字段使用默认值"""
|
||||
raw = {}
|
||||
if _CONFIG_PATH.exists():
|
||||
with open(_CONFIG_PATH, encoding="utf-8") as f:
|
||||
raw = json.load(f)
|
||||
|
||||
project_root = _CONFIG_PATH.parent
|
||||
|
||||
return {
|
||||
"sso_login_url": raw.get("sso_login_url", "https://tyrz.fynu.edu.cn/sso/login"),
|
||||
"portal_url": raw.get("portal_url", "https://tyrz.fynu.edu.cn/oshall"),
|
||||
"reimburse_url": raw.get("reimburse_url", "http://210.45.32.214:8081"),
|
||||
"reimburse_page": raw.get("reimburse_page", "/expen/common/common?v=4.0"),
|
||||
"username": raw.get("username", ""),
|
||||
"password": raw.get("password", ""),
|
||||
"default_name": raw.get("default_name", ""),
|
||||
"default_card_no": raw.get("default_card_no", ""),
|
||||
"default_person_id": raw.get("default_person_id", ""),
|
||||
"attachment_dir": project_root / "attachments",
|
||||
}
|
||||
256
app/extractor.py
256
app/extractor.py
@@ -1,256 +0,0 @@
|
||||
"""
|
||||
PDF 发票信息提取
|
||||
|
||||
从 PDF 发票文件中提取关键字段,输出为标准化的发票数据列表。
|
||||
|
||||
对外接口:
|
||||
extract_invoices(directory) -> list[dict] 扫描目录下所有 PDF 并提取
|
||||
save_csv(invoices, path) 保存为 CSV
|
||||
save_markdown(invoices, path) 保存为 Markdown 汇总
|
||||
"""
|
||||
|
||||
import csv
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
from . import get_logger
|
||||
|
||||
log = get_logger("extractor")
|
||||
|
||||
CSV_COLUMNS = [
|
||||
"序号", "发票号码", "开票日期", "项目名称", "规格型号",
|
||||
"价税合计", "销售方名称", "人员姓名", "刷卡日期",
|
||||
"公务卡号", "刷卡金额", "备注", "工号",
|
||||
]
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# PDF 文件发现与文本提取
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def find_pdf_files(directory: str = ".") -> list[Path]:
|
||||
"""查找目录下所有 PDF 文件(非递归)"""
|
||||
pdf_dir = Path(directory)
|
||||
if not pdf_dir.exists():
|
||||
return []
|
||||
return sorted(pdf_dir.glob("*.pdf"))
|
||||
|
||||
|
||||
def extract_text_from_pdf(filepath: Path) -> str:
|
||||
"""从单个 PDF 中提取全部文本"""
|
||||
try:
|
||||
import pdfplumber
|
||||
except ImportError:
|
||||
raise ImportError("缺少 pdfplumber,请执行: pip install pdfplumber")
|
||||
|
||||
try:
|
||||
parts = []
|
||||
with pdfplumber.open(filepath) as pdf:
|
||||
for page in pdf.pages:
|
||||
text = page.extract_text()
|
||||
if text:
|
||||
parts.append(text)
|
||||
return "\n".join(parts)
|
||||
except Exception as e:
|
||||
log.error(f"无法读取 {filepath.name}: {e}")
|
||||
return ""
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 字段解析
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _first(regexes: list[str], text: str) -> str | None:
|
||||
"""尝试多个正则,返回第一个匹配组的文本"""
|
||||
for pattern in regexes:
|
||||
m = re.search(pattern, text)
|
||||
if m:
|
||||
return m.group(1).strip()
|
||||
return None
|
||||
|
||||
|
||||
def _parse_line_item(line: str) -> dict | None:
|
||||
"""解析单行明细(*分类*具体名称 格式)"""
|
||||
m = re.match(r"\*([^*]+)\*\s*(.+)", line)
|
||||
if m:
|
||||
return {
|
||||
"项目名称": f"*{m.group(1).strip()}*{m.group(2).strip()}",
|
||||
"规格型号": m.group(2).strip(),
|
||||
}
|
||||
return None
|
||||
|
||||
|
||||
def _extract_line_items(text: str) -> list[dict]:
|
||||
"""从发票文本中提取所有明细行"""
|
||||
items = []
|
||||
skip_keywords = ["项目名称", "合 计", "价税合计", "备注", "开票人"]
|
||||
|
||||
for line in text.split("\n"):
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
if any(kw in line for kw in skip_keywords):
|
||||
continue
|
||||
if "*" in line:
|
||||
item = _parse_line_item(line)
|
||||
if item:
|
||||
items.append(item)
|
||||
|
||||
return items
|
||||
|
||||
|
||||
def _format_date(date_raw: str) -> str:
|
||||
"""将「2026年5月18日」转为「2026/5/18」"""
|
||||
m = re.match(r"(\d{4})年(\d{1,2})月(\d{1,2})日", date_raw)
|
||||
if m:
|
||||
return f"{m.group(1)}/{m.group(2)}/{m.group(3)}"
|
||||
return date_raw
|
||||
|
||||
|
||||
def parse_invoice(text: str) -> dict:
|
||||
"""从发票文本中提取关键字段,返回 dict
|
||||
|
||||
返回字段:
|
||||
发票号码, 开票日期, 销售方名称, 价税合计, _items (明细列表)
|
||||
其他字段(人员姓名等)留空,后续由 OCR 步骤填充
|
||||
"""
|
||||
invoice: dict[str, str] = {}
|
||||
|
||||
invoice["发票号码"] = _first([r"发票号码[::]?\s*(\d+)"], text) or ""
|
||||
|
||||
date_raw = _first([r"开票日期[::]?\s*(\d{4}年\d{1,2}月\d{1,2}日)"], text) or ""
|
||||
invoice["开票日期"] = _format_date(date_raw) if date_raw else ""
|
||||
|
||||
invoice["销售方名称"] = _first(
|
||||
[
|
||||
r"销\s*售?\s*方?\s*名称[::]?\s*(.+?)(?:\n|$)",
|
||||
r"销\s*名称[::]?\s*(.+?)(?:\n|$)",
|
||||
],
|
||||
text,
|
||||
) or ""
|
||||
|
||||
invoice["价税合计"] = _first(
|
||||
[r"价税合计.*?(小写)[¥¥]?\s*(\d+\.?\d*)"], text
|
||||
) or ""
|
||||
|
||||
invoice["_items"] = _extract_line_items(text)
|
||||
|
||||
# 以下字段无法从 PDF 提取,留空由 OCR 步骤填充
|
||||
for key in ("项目名称", "规格型号", "人员姓名", "刷卡日期",
|
||||
"公务卡号", "刷卡金额", "备注", "工号"):
|
||||
if key not in invoice:
|
||||
invoice[key] = ""
|
||||
|
||||
return invoice
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# CSV / Markdown 输出
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def save_csv(invoices: list[dict], output_path: str | Path = "invoice_summary.csv"):
|
||||
"""将发票列表保存为 CSV"""
|
||||
csv_path = Path(output_path)
|
||||
|
||||
with open(csv_path, "w", encoding="utf-8", newline="") as f:
|
||||
writer = csv.writer(f)
|
||||
writer.writerow(CSV_COLUMNS)
|
||||
|
||||
for idx, inv in enumerate(invoices, 1):
|
||||
items = inv.get("_items", [])
|
||||
first_item = items[0] if items else {}
|
||||
writer.writerow([
|
||||
idx,
|
||||
inv.get("发票号码", ""),
|
||||
inv.get("开票日期", ""),
|
||||
first_item.get("项目名称", inv.get("项目名称", "")),
|
||||
first_item.get("规格型号", inv.get("规格型号", "")),
|
||||
inv.get("价税合计", ""),
|
||||
inv.get("销售方名称", ""),
|
||||
inv.get("人员姓名", ""),
|
||||
inv.get("刷卡日期", ""),
|
||||
inv.get("公务卡号", ""),
|
||||
inv.get("刷卡金额", ""),
|
||||
inv.get("备注", ""),
|
||||
inv.get("工号", ""),
|
||||
])
|
||||
|
||||
log.info(f"CSV 已保存: {csv_path.name}")
|
||||
|
||||
|
||||
def save_markdown(invoices: list[dict], output_path: str | Path = "invoice_summary.md"):
|
||||
"""将发票列表保存为 Markdown 汇总表"""
|
||||
md_path = Path(output_path)
|
||||
lines = [
|
||||
"# 发票信息汇总表",
|
||||
"",
|
||||
"| 序号 | 发票号码 | 开票日期 | 项目名称 | 规格型号 | 价税合计 | 销售方名称 |",
|
||||
"|------|---------|---------|---------|---------|---------|-----------|",
|
||||
]
|
||||
|
||||
total = 0.0
|
||||
for idx, inv in enumerate(invoices, 1):
|
||||
amount = 0.0
|
||||
try:
|
||||
amount = float(inv.get("价税合计", "0"))
|
||||
except (ValueError, TypeError):
|
||||
pass
|
||||
total += amount
|
||||
|
||||
items = inv.get("_items", [])
|
||||
first_item = items[0] if items else {}
|
||||
project = first_item.get("项目名称", inv.get("项目名称", "-"))
|
||||
spec = first_item.get("规格型号", inv.get("规格型号", "-"))
|
||||
|
||||
lines.append(
|
||||
f"| {idx} "
|
||||
f"| {inv.get('发票号码', '')} "
|
||||
f"| {inv.get('开票日期', '')} "
|
||||
f"| {project} | {spec} "
|
||||
f"| ¥{amount:,.2f} "
|
||||
f"| {inv.get('销售方名称', '')} |"
|
||||
)
|
||||
|
||||
lines.append("")
|
||||
lines.append(f"**总计: ¥{total:,.2f}**")
|
||||
lines.append("")
|
||||
|
||||
with open(md_path, "w", encoding="utf-8") as f:
|
||||
f.write("\n".join(lines))
|
||||
|
||||
log.info(f"Markdown 已保存: {md_path.name}")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 主入口
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def extract_invoices(directory: str = ".") -> list[dict]:
|
||||
"""扫描目录下所有 PDF,提取发票信息并返回列表"""
|
||||
target_dir = Path(directory).absolute()
|
||||
|
||||
pdf_files = find_pdf_files(directory)
|
||||
if not pdf_files:
|
||||
log.warning("未找到 PDF 文件")
|
||||
return []
|
||||
|
||||
log.info(f"发现 {len(pdf_files)} 个 PDF 文件")
|
||||
|
||||
all_invoices = []
|
||||
for pdf_path in pdf_files:
|
||||
text = extract_text_from_pdf(pdf_path)
|
||||
if text:
|
||||
invoice = parse_invoice(text)
|
||||
if invoice:
|
||||
all_invoices.append(invoice)
|
||||
else:
|
||||
log.warning(f"未能解析: {pdf_path.name}")
|
||||
else:
|
||||
log.warning(f"未能提取文本: {pdf_path.name}")
|
||||
|
||||
if all_invoices:
|
||||
log.info(f"共处理 {len(all_invoices)} 张发票")
|
||||
else:
|
||||
log.warning("未成功解析任何发票")
|
||||
|
||||
return all_invoices
|
||||
454
app/ocr.py
454
app/ocr.py
@@ -1,454 +0,0 @@
|
||||
"""
|
||||
OCR 刷卡信息提取
|
||||
|
||||
从支付截图中识别刷卡记录(姓名、日期、金额),回填到发票数据中。
|
||||
|
||||
匹配策略:
|
||||
1. 先按文件名匹配(PDF 和图片同名)
|
||||
2. 未匹配的通过金额近邻匹配
|
||||
|
||||
对外接口:
|
||||
enrich_with_ocr(rows, directory) -> list[dict] 用 OCR 识别结果丰富发票数据
|
||||
"""
|
||||
|
||||
import csv
|
||||
import os
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
from . import get_logger
|
||||
|
||||
log = get_logger("ocr")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 懒加载 OCR
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
_ocr_instance = None
|
||||
|
||||
|
||||
def _get_ocr():
|
||||
"""懒加载 PaddleOCR 实例(兼容 2.x / 3.x)"""
|
||||
global _ocr_instance
|
||||
if _ocr_instance is not None:
|
||||
return _ocr_instance
|
||||
|
||||
os.environ.setdefault("FLAGS_use_mkldnn", "0")
|
||||
os.environ.setdefault("FLAGS_mkldnn_cache_enabled", "0")
|
||||
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
try:
|
||||
_ocr_instance = PaddleOCR(use_textline_orientation=True, lang="ch")
|
||||
except TypeError:
|
||||
try:
|
||||
_ocr_instance = PaddleOCR(lang="ch")
|
||||
except TypeError:
|
||||
_ocr_instance = PaddleOCR()
|
||||
|
||||
return _ocr_instance
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# OCR 识别
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def ocr_image(image_path: Path) -> list[dict]:
|
||||
"""对单张图片执行 OCR,返回 [{"text": str, "confidence": float}, ...]"""
|
||||
ocr = _get_ocr()
|
||||
texts = []
|
||||
|
||||
try:
|
||||
results = ocr.ocr(str(image_path), cls=True)
|
||||
if results and isinstance(results, list):
|
||||
for page_result in results:
|
||||
if not page_result:
|
||||
continue
|
||||
for line in page_result:
|
||||
if isinstance(line, (list, tuple)) and len(line) >= 2:
|
||||
_, text_info = line[0], line[1]
|
||||
if isinstance(text_info, (list, tuple)) and len(text_info) >= 2:
|
||||
texts.append({
|
||||
"text": str(text_info[0]),
|
||||
"confidence": float(text_info[1]),
|
||||
})
|
||||
except Exception:
|
||||
try:
|
||||
if hasattr(ocr, "predict"):
|
||||
results = ocr.predict(str(image_path))
|
||||
if results:
|
||||
for result in results:
|
||||
if hasattr(result, "rec_result_list"):
|
||||
for line in result.rec_result_list:
|
||||
t = getattr(line, "text", "") or ""
|
||||
s = getattr(line, "score", 0.0) or 0.0
|
||||
texts.append({"text": str(t), "confidence": float(s)})
|
||||
elif isinstance(result, list):
|
||||
for line in result:
|
||||
if isinstance(line, (list, tuple)) and len(line) >= 2:
|
||||
t = line[1][0] if isinstance(line[1], (list, tuple)) else str(line[1])
|
||||
s = line[1][1] if isinstance(line[1], (list, tuple)) and len(line[1]) > 1 else 0.0
|
||||
texts.append({"text": str(t), "confidence": float(s)})
|
||||
except Exception as e:
|
||||
log.error(f"OCR 识别失败: {e}")
|
||||
|
||||
return texts
|
||||
|
||||
|
||||
def extract_card_info(texts: list[dict]) -> dict:
|
||||
"""从 OCR 文本中提取刷卡信息(日期 / 金额 / 姓名)"""
|
||||
info = {"刷卡日期": "", "刷卡金额": "", "人员姓名": ""}
|
||||
|
||||
valid = [t for t in texts if t["confidence"] > 0.5]
|
||||
full_text = " ".join(t["text"] for t in valid)
|
||||
if not full_text:
|
||||
return info
|
||||
|
||||
# 日期(优先级匹配,避免误抓发票开票日期)
|
||||
date_candidates = []
|
||||
for pattern, priority in [
|
||||
(r"记账时间[::\s]*(\d{4}[-/]\d{1,2}[-/]\d{1,2})", 10),
|
||||
(r"交易时间[::\s]*(\d{4}[-/]\d{1,2}[-/]\d{1,2})", 9),
|
||||
(r"刷卡日期[::\s]*(\d{4}[-/]\d{1,2}[-/]\d{1,2})", 9),
|
||||
(r"日期[::\s]*(\d{4}[-/]\d{1,2}[-/]\d{1,2})", 5),
|
||||
]:
|
||||
for m in re.finditer(pattern, full_text):
|
||||
date_candidates.append((priority, m.start(), m.group(1).replace("-", "/")))
|
||||
if date_candidates:
|
||||
date_candidates.sort(key=lambda x: (-x[0], x[1]))
|
||||
info["刷卡日期"] = date_candidates[0][2]
|
||||
|
||||
# 金额
|
||||
amount_candidates = []
|
||||
for pattern, priority in [
|
||||
(r"交易金额[::\s]*([+-]?[\d,]+\.?\d*)", 10),
|
||||
(r"刷卡金额[::\s]*([+-]?[\d,]+\.?\d*)", 10),
|
||||
(r"金额[::\s]*([+-]?[\d,]+\.?\d*)", 5),
|
||||
]:
|
||||
for m in re.finditer(pattern, full_text):
|
||||
amt_str = m.group(1).replace(",", "").replace("+", "")
|
||||
try:
|
||||
val = float(amt_str)
|
||||
if 0 < val < 999999:
|
||||
amount_candidates.append((priority, m.start(), amt_str))
|
||||
except ValueError:
|
||||
continue
|
||||
if amount_candidates:
|
||||
amount_candidates.sort(key=lambda x: (-x[0], x[1]))
|
||||
info["刷卡金额"] = amount_candidates[0][2]
|
||||
|
||||
# 姓名(排除公司/机构后缀)
|
||||
EXCLUDE_SUFFIXES = ("公司", "银行", "中心", "支行", "商户", "网点", "有限", "责任")
|
||||
name_candidates = []
|
||||
for pattern, priority in [
|
||||
(r"交易户名[::\s]*([\u4e00-\u9fff]{2,6})", 10),
|
||||
(r"户名[::\s]*([\u4e00-\u9fff]{2,6})", 8),
|
||||
(r"持卡人[::\s]*([\u4e00-\u9fff]{2,6})", 8),
|
||||
(r"姓名[::\s]*([\u4e00-\u9fff]{2,6})", 8),
|
||||
]:
|
||||
for m in re.finditer(pattern, full_text):
|
||||
name = m.group(1)
|
||||
if not any(name.endswith(s) for s in EXCLUDE_SUFFIXES):
|
||||
name_candidates.append((priority, m.start(), name))
|
||||
if name_candidates:
|
||||
name_candidates.sort(key=lambda x: (-x[0], x[1]))
|
||||
info["人员姓名"] = name_candidates[0][2]
|
||||
|
||||
return info
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# PDF 发票号提取
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def extract_invoice_number(pdf_path: Path) -> str:
|
||||
"""从 PDF 中提取发票号码"""
|
||||
try:
|
||||
import pdfplumber
|
||||
except ImportError:
|
||||
log.warning("缺少 pdfplumber,跳过发票号提取")
|
||||
return ""
|
||||
|
||||
try:
|
||||
with pdfplumber.open(str(pdf_path)) as pdf_file:
|
||||
page_text = ""
|
||||
for page in pdf_file.pages:
|
||||
page_text += page.extract_text() or ""
|
||||
|
||||
for pattern in [
|
||||
r"发票号码[::\s]*([A-Za-z0-9]{8,20})",
|
||||
r"发票代码[::\s]*([A-Za-z0-9]{10,12})",
|
||||
r"号码[::\s]*([A-Za-z0-9]{8,20})",
|
||||
]:
|
||||
m = re.search(pattern, page_text)
|
||||
if m:
|
||||
return m.group(1)
|
||||
except Exception as e:
|
||||
log.warning(f"PDF 读取失败 ({pdf_path.name}): {e}")
|
||||
|
||||
return ""
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 图片配对
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _extract_amount_from_pdf(pdf_path: Path) -> float | None:
|
||||
"""从 PDF 中提取价税合计金额"""
|
||||
try:
|
||||
import pdfplumber
|
||||
with pdfplumber.open(str(pdf_path)) as pdf:
|
||||
text = ""
|
||||
for page in pdf.pages:
|
||||
t = page.extract_text()
|
||||
if t:
|
||||
text += t + "\n"
|
||||
m = re.search(r"价税合计.*?(小写)[¥¥]?\s*(\d+\.?\d*)", text)
|
||||
if m:
|
||||
return float(m.group(1))
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def _extract_amount_from_image(img_path: Path) -> float | None:
|
||||
"""从图片 OCR 中提取刷卡金额"""
|
||||
texts = ocr_image(img_path)
|
||||
if not texts:
|
||||
return None
|
||||
info = extract_card_info(texts)
|
||||
amt_str = info.get("刷卡金额", "")
|
||||
if amt_str:
|
||||
try:
|
||||
return float(amt_str)
|
||||
except ValueError:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def find_image_pairs(directory: str = ".") -> list[tuple[Path, Path]]:
|
||||
"""查找 PDF 和对应图片的配对
|
||||
|
||||
1. 先按文件名匹配(PDF 和图片同名)
|
||||
2. 未匹配的通过金额近邻匹配
|
||||
"""
|
||||
base = Path(directory)
|
||||
pdfs = sorted(base.glob("*.pdf"))
|
||||
image_exts = {".png", ".jpg", ".jpeg", ".bmp", ".webp"}
|
||||
|
||||
all_images = sorted(
|
||||
f for ext in image_exts for f in base.glob(f"*{ext}")
|
||||
)
|
||||
|
||||
# ---- Phase 1: 文件名匹配 ----
|
||||
pairs: list[tuple[Path, Path]] = []
|
||||
matched_pdfs: set[Path] = set()
|
||||
matched_imgs: set[Path] = set()
|
||||
|
||||
for pdf in pdfs:
|
||||
for ext in image_exts:
|
||||
img = base / f"{pdf.stem}{ext}"
|
||||
if img.exists():
|
||||
pairs.append((pdf, img))
|
||||
matched_pdfs.add(pdf)
|
||||
matched_imgs.add(img)
|
||||
break
|
||||
|
||||
unmatched_pdfs = [p for p in pdfs if p not in matched_pdfs]
|
||||
unmatched_imgs = [i for i in all_images if i not in matched_imgs]
|
||||
|
||||
if not unmatched_pdfs or not unmatched_imgs:
|
||||
return pairs
|
||||
|
||||
# ---- Phase 2: 金额近邻匹配 ----
|
||||
if len(unmatched_pdfs) > 0 and len(unmatched_imgs) > 0:
|
||||
log.info(f"文件名匹配 {len(pairs)} 组,剩余 {len(unmatched_pdfs)} 个 PDF、{len(unmatched_imgs)} 张图片,尝试金额匹配...")
|
||||
|
||||
pdf_amounts: dict[Path, float] = {}
|
||||
for pdf in unmatched_pdfs:
|
||||
amt = _extract_amount_from_pdf(pdf)
|
||||
if amt is not None:
|
||||
pdf_amounts[pdf] = amt
|
||||
|
||||
img_amounts: dict[Path, float] = {}
|
||||
for img in unmatched_imgs:
|
||||
amt = _extract_amount_from_image(img)
|
||||
if amt is not None:
|
||||
img_amounts[img] = amt
|
||||
|
||||
# 贪婪匹配:每张图片找金额差最小的 PDF
|
||||
used_pdfs: set[Path] = set()
|
||||
for img, img_amt in sorted(img_amounts.items(), key=lambda x: x[0].name):
|
||||
best_pdf: Path | None = None
|
||||
best_diff: float = float("inf")
|
||||
|
||||
for pdf, pdf_amt in pdf_amounts.items():
|
||||
if pdf in used_pdfs:
|
||||
continue
|
||||
diff = abs(pdf_amt - img_amt)
|
||||
if diff < best_diff:
|
||||
best_diff = diff
|
||||
best_pdf = pdf
|
||||
|
||||
if best_pdf is not None:
|
||||
pairs.append((best_pdf, img))
|
||||
used_pdfs.add(best_pdf)
|
||||
|
||||
log.info(f"金额匹配完成,共 {len(pairs)} 组配对")
|
||||
|
||||
return pairs
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# CSV 读写
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
from .extractor import CSV_COLUMNS
|
||||
|
||||
|
||||
def _load_csv(csv_path: Path) -> list[dict] | None:
|
||||
"""读取现有 CSV 为 dict 列表,失败返回 None"""
|
||||
try:
|
||||
with open(csv_path, encoding="utf-8", newline="") as f:
|
||||
reader = csv.DictReader(f)
|
||||
fieldnames = reader.fieldnames or []
|
||||
missing = [c for c in CSV_COLUMNS if c not in fieldnames]
|
||||
if missing:
|
||||
log.error(f"CSV 缺少必要列: {missing}")
|
||||
return None
|
||||
return [row for row in reader]
|
||||
except FileNotFoundError:
|
||||
log.error(f"CSV 文件不存在: {csv_path.name}")
|
||||
return None
|
||||
except Exception as e:
|
||||
log.error(f"CSV 读取失败: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def _save_csv(csv_path: Path, rows: list[dict]):
|
||||
"""保存 CSV"""
|
||||
with open(csv_path, "w", encoding="utf-8", newline="") as f:
|
||||
writer = csv.DictWriter(f, fieldnames=CSV_COLUMNS)
|
||||
writer.writeheader()
|
||||
writer.writerows(rows)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Markdown 同步
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def save_markdown_from_csv(csv_path: Path, rows: list[dict]):
|
||||
"""根据最新 CSV 数据生成 Markdown 汇总表"""
|
||||
md_path = csv_path.with_suffix(".md")
|
||||
columns = [
|
||||
("序号", "序号"), ("发票号码", "发票号码"), ("开票日期", "开票日期"),
|
||||
("项目名称", "项目名称"), ("规格型号", "规格型号"), ("价税合计", "价税合计"),
|
||||
("销售方名称", "销售方名称"), ("人员姓名", "人员姓名"),
|
||||
("刷卡日期", "刷卡日期"), ("公务卡号", "公务卡号"),
|
||||
("刷卡金额", "刷卡金额"), ("备注", "备注"), ("工号", "工号"),
|
||||
]
|
||||
|
||||
lines = ["# 发票信息汇总表", ""]
|
||||
header = " | ".join(col[1] for col in columns)
|
||||
separator = "|".join(["------" for _ in columns])
|
||||
lines.append(f"| {header} |")
|
||||
lines.append(f"|{separator}|")
|
||||
|
||||
total_price = 0.0
|
||||
total_card = 0.0
|
||||
|
||||
for row in rows:
|
||||
cells = []
|
||||
for key, _ in columns:
|
||||
value = row.get(key, "").strip()
|
||||
|
||||
if key == "价税合计" and value:
|
||||
try:
|
||||
total_price += float(value.replace(",", ""))
|
||||
cells.append(f"¥{float(value.replace(',', '')):,.2f}")
|
||||
except (ValueError, TypeError):
|
||||
cells.append(value)
|
||||
elif key == "刷卡金额" and value:
|
||||
try:
|
||||
total_card += float(value.replace(",", ""))
|
||||
cells.append(f"¥{float(value.replace(',', '')):,.2f}")
|
||||
except (ValueError, TypeError):
|
||||
cells.append(value)
|
||||
else:
|
||||
cells.append(value if value else "")
|
||||
|
||||
lines.append("| " + " | ".join(cells) + " |")
|
||||
|
||||
lines.append("")
|
||||
lines.append(f"**价税合计总计: ¥{total_price:,.2f}**")
|
||||
lines.append(f"**刷卡金额总计: ¥{total_card:,.2f}**")
|
||||
lines.append("")
|
||||
|
||||
with open(md_path, "w", encoding="utf-8") as f:
|
||||
f.write("\n".join(lines))
|
||||
|
||||
log.info(f"Markdown 已同步: {md_path.name}")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 主入口
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def enrich_with_ocr(rows: list[dict], directory: str = ".") -> list[dict]:
|
||||
"""用 OCR 识别结果丰富发票数据,返回更新后的行列表
|
||||
|
||||
rows 应包含「发票号码」列,已存在的字段不会覆盖。
|
||||
"""
|
||||
pairs = find_image_pairs(directory)
|
||||
if not pairs:
|
||||
log.warning("未找到 PDF-图片配对文件,跳过 OCR")
|
||||
return rows
|
||||
|
||||
log.info(f"找到 {len(pairs)} 组 PDF-图片配对")
|
||||
|
||||
ocr_by_invoice: dict[str, dict] = {}
|
||||
|
||||
for idx, (pdf, img) in enumerate(pairs, 1):
|
||||
inv_num = extract_invoice_number(pdf)
|
||||
if not inv_num:
|
||||
inv_num = pdf.stem
|
||||
|
||||
texts = ocr_image(img)
|
||||
if not texts:
|
||||
log.warning(f"OCR 未识别到文本: {img.name}")
|
||||
continue
|
||||
|
||||
info = extract_card_info(texts)
|
||||
ocr_by_invoice[inv_num] = info
|
||||
|
||||
# 更新行数据
|
||||
updated = 0
|
||||
matched = 0
|
||||
|
||||
for i, row in enumerate(rows):
|
||||
inv_num = row.get("发票号码", "").strip()
|
||||
ocr_info = ocr_by_invoice.get(inv_num)
|
||||
|
||||
if not ocr_info:
|
||||
for key, val in ocr_by_invoice.items():
|
||||
if inv_num in key or key in inv_num:
|
||||
ocr_info = val
|
||||
break
|
||||
|
||||
if ocr_info:
|
||||
matched += 1
|
||||
|
||||
if not row.get("人员姓名", "").strip() and ocr_info["人员姓名"]:
|
||||
row["人员姓名"] = ocr_info["人员姓名"]
|
||||
updated += 1
|
||||
if not row.get("刷卡日期", "").strip() and ocr_info["刷卡日期"]:
|
||||
row["刷卡日期"] = ocr_info["刷卡日期"]
|
||||
updated += 1
|
||||
if not row.get("刷卡金额", "").strip() and ocr_info["刷卡金额"]:
|
||||
row["刷卡金额"] = ocr_info["刷卡金额"]
|
||||
updated += 1
|
||||
|
||||
log.info(f"OCR 完成: 匹配 {matched}/{len(rows)} 行,更新 {updated} 个字段")
|
||||
|
||||
return rows
|
||||
128
app/pipeline.py
128
app/pipeline.py
@@ -1,128 +0,0 @@
|
||||
"""
|
||||
报销全流程编排
|
||||
|
||||
将发票提取 → OCR 识别 → 浏览器填报串联为一条管道,
|
||||
数据在内存中流转,同时生成 CSV / Markdown 中间产物。
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from . import get_logger
|
||||
from .config import load_config
|
||||
from .extractor import extract_invoices, save_csv as save_invoice_csv, save_markdown as save_invoice_md
|
||||
from .ocr import enrich_with_ocr, _save_csv as save_ocr_csv, save_markdown_from_csv, _load_csv
|
||||
|
||||
log = get_logger("pipeline")
|
||||
|
||||
|
||||
def run_pipeline(step: str = "all", username: str = None, password: str = None):
|
||||
"""执行报销流程
|
||||
|
||||
Args:
|
||||
step: all | invoice | ocr | submit
|
||||
username: 覆盖 config.json 中的用户名
|
||||
password: 覆盖 config.json 中的密码
|
||||
"""
|
||||
config = load_config()
|
||||
if username:
|
||||
config["username"] = username
|
||||
if password:
|
||||
config["password"] = password
|
||||
|
||||
# 工作目录(项目根目录)
|
||||
project_dir = Path(__file__).parent.parent
|
||||
|
||||
# --------------------------------------------------
|
||||
# Step 1: 发票提取
|
||||
# --------------------------------------------------
|
||||
invoices = None
|
||||
|
||||
if step in ("all", "invoice"):
|
||||
log.info("=" * 60)
|
||||
log.info("[1/3] 发票提取")
|
||||
log.info("=" * 60)
|
||||
|
||||
invoices = extract_invoices(str(project_dir))
|
||||
if not invoices:
|
||||
log.error("未提取到任何发票数据")
|
||||
return 1
|
||||
|
||||
save_invoice_csv(invoices, project_dir / "invoice_summary.csv")
|
||||
save_invoice_md(invoices, project_dir / "invoice_summary.md")
|
||||
|
||||
if step == "invoice":
|
||||
log.info("[1/3] 发票提取 完成")
|
||||
return 0
|
||||
|
||||
# --------------------------------------------------
|
||||
# Step 2: OCR 识别
|
||||
# --------------------------------------------------
|
||||
if step in ("all", "ocr"):
|
||||
log.info("=" * 60)
|
||||
log.info("[2/3] OCR 识别")
|
||||
log.info("=" * 60)
|
||||
|
||||
csv_path = project_dir / "invoice_summary.csv"
|
||||
|
||||
if invoices is None:
|
||||
rows = _load_csv(csv_path)
|
||||
if rows is None:
|
||||
return 1
|
||||
else:
|
||||
# 将 dict 列表转为 CSV 风格的 dict(对齐列名)
|
||||
from .extractor import CSV_COLUMNS
|
||||
rows = []
|
||||
for idx, inv in enumerate(invoices, 1):
|
||||
items = inv.get("_items", [])
|
||||
first_item = items[0] if items else {}
|
||||
rows.append({
|
||||
"序号": str(idx),
|
||||
"发票号码": inv.get("发票号码", ""),
|
||||
"开票日期": inv.get("开票日期", ""),
|
||||
"项目名称": first_item.get("项目名称", inv.get("项目名称", "")),
|
||||
"规格型号": first_item.get("规格型号", inv.get("规格型号", "")),
|
||||
"价税合计": inv.get("价税合计", ""),
|
||||
"销售方名称": inv.get("销售方名称", ""),
|
||||
"人员姓名": inv.get("人员姓名", ""),
|
||||
"刷卡日期": inv.get("刷卡日期", ""),
|
||||
"公务卡号": inv.get("公务卡号", ""),
|
||||
"刷卡金额": inv.get("刷卡金额", ""),
|
||||
"备注": inv.get("备注", ""),
|
||||
"工号": inv.get("工号", ""),
|
||||
})
|
||||
|
||||
rows = enrich_with_ocr(rows, str(project_dir))
|
||||
save_ocr_csv(csv_path, rows)
|
||||
save_markdown_from_csv(csv_path, rows)
|
||||
invoices = rows
|
||||
|
||||
if step == "ocr":
|
||||
log.info("[2/3] OCR 识别 完成")
|
||||
return 0
|
||||
|
||||
# --------------------------------------------------
|
||||
# Step 3: 浏览器填报
|
||||
# --------------------------------------------------
|
||||
if step in ("all", "submit"):
|
||||
log.info("=" * 60)
|
||||
log.info("[3/3] 报销提交")
|
||||
log.info("=" * 60)
|
||||
|
||||
from .bot import load_invoice_data, run_bot
|
||||
|
||||
csv_path = project_dir / "invoice_summary.csv"
|
||||
bot_invoices = load_invoice_data(str(csv_path), config)
|
||||
run_bot(config, bot_invoices)
|
||||
|
||||
if step == "submit":
|
||||
log.info("[3/3] 报销提交 完成")
|
||||
return 0
|
||||
|
||||
# --------------------------------------------------
|
||||
# 全流程完成
|
||||
# --------------------------------------------------
|
||||
log.info("=" * 60)
|
||||
log.info("全流程执行完毕")
|
||||
log.info("=" * 60)
|
||||
return 0
|
||||
8
config.example.json
Normal file
8
config.example.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"username": "你的工号",
|
||||
"password": "你的密码",
|
||||
"default_name": "默认报销人姓名",
|
||||
"default_card_no": "默认公务卡号",
|
||||
"default_person_id": "默认人员编号",
|
||||
"consumable_storage": "躬行楼 C205"
|
||||
}
|
||||
11
config.json
11
config.json
@@ -1,11 +0,0 @@
|
||||
{
|
||||
"username": "202407021",
|
||||
"password": "wang!1624155937",
|
||||
"sso_login_url": "https://tyrz.fynu.edu.cn/sso/login",
|
||||
"portal_url": "https://tyrz.fynu.edu.cn/oshall",
|
||||
"reimburse_url": "http://210.45.32.214:8081",
|
||||
"reimburse_page": "/expen/common/common?v=4.0",
|
||||
"default_name": "王建锋",
|
||||
"default_card_no": "6282880139161682",
|
||||
"default_person_id": "202407021"
|
||||
}
|
||||
24
config/README.md
Normal file
24
config/README.md
Normal file
@@ -0,0 +1,24 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# config — 配置文件目录
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 说明 |
|
||||
|------|------|
|
||||
| `validation_rules.json` | 声明式校验规则配置:定义差旅和普通报销的必填字段、数组元素校验规则和自定义校验函数 |
|
||||
|
||||
## validation_rules.json 结构
|
||||
|
||||
```json
|
||||
{
|
||||
"version": "1.0",
|
||||
"custom_checks": { ... },
|
||||
"travel": { "fields": [...], "arrays": [...] },
|
||||
"normal": { "fields": [...], "arrays": [...] }
|
||||
}
|
||||
```
|
||||
|
||||
校验引擎 `src/core/validation/validator.py` 在启动时读取此文件,若文件不存在则使用内置默认规则。
|
||||
135
config/validation_rules.json
Normal file
135
config/validation_rules.json
Normal file
@@ -0,0 +1,135 @@
|
||||
{
|
||||
"version": "1.0",
|
||||
"custom_checks": {
|
||||
"is_valid_date": "检查日期格式是否为 YYYY-MM-DD",
|
||||
"is_positive_number": "检查是否为正数(整数或浮点数)",
|
||||
"is_positive_integer": "检查是否为正整数"
|
||||
},
|
||||
"travel": {
|
||||
"description": "差旅报销校验规则",
|
||||
"fields": [
|
||||
{
|
||||
"path": ["basic_info", "travel_purpose"],
|
||||
"required": true,
|
||||
"check_empty": true,
|
||||
"description": "出差事由"
|
||||
},
|
||||
{
|
||||
"path": ["basic_info", "travel_location"],
|
||||
"required": true,
|
||||
"check_empty": true,
|
||||
"description": "出差地点"
|
||||
},
|
||||
{
|
||||
"path": ["basic_info", "start_date"],
|
||||
"required": true,
|
||||
"check_empty": true,
|
||||
"custom_check": "is_valid_date",
|
||||
"description": "出差开始日期"
|
||||
},
|
||||
{
|
||||
"path": ["basic_info", "end_date"],
|
||||
"required": true,
|
||||
"check_empty": true,
|
||||
"custom_check": "is_valid_date",
|
||||
"description": "出差结束日期"
|
||||
}
|
||||
],
|
||||
"arrays": [
|
||||
{
|
||||
"path": ["reimbursement_details", "transport_fee"],
|
||||
"min_items": 1,
|
||||
"description": "交通费用明细",
|
||||
"element_fields": [
|
||||
{"path": ["vehicle_type"], "required": true, "check_empty": true, "description": "交通工具类型"},
|
||||
{"path": ["start_date"], "required": true, "check_empty": true, "custom_check": "is_valid_date", "description": "出发日期"},
|
||||
{"path": ["end_date"], "required": true, "check_empty": true, "custom_check": "is_valid_date", "description": "到达日期"},
|
||||
{"path": ["departure_place"], "required": true, "check_empty": true, "description": "出发地"},
|
||||
{"path": ["arrival_place"], "required": true, "check_empty": true, "description": "目的地"},
|
||||
{"path": ["amount"], "required": true, "check_empty": true, "custom_check": "is_positive_number", "description": "金额"},
|
||||
{"path": ["bill_count"], "required": true, "check_empty": true, "custom_check": "is_positive_integer", "description": "票据张数"},
|
||||
{"path": ["remark"], "required": true, "check_empty": false, "description": "备注说明"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"path": ["payment_methods"],
|
||||
"min_items": 1,
|
||||
"description": "支付方式记录",
|
||||
"element_fields": [
|
||||
{"path": ["card_date"], "required": true, "check_empty": true, "custom_check": "is_valid_date", "description": "刷卡日期"},
|
||||
{"path": ["card_amount"], "required": true, "check_empty": true, "custom_check": "is_positive_number", "description": "支付金额"},
|
||||
{"path": ["merchant"], "required": true, "check_empty": true, "description": "商户名称"},
|
||||
{"path": ["remark"], "required": true, "check_empty": false, "description": "备注"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"path": ["subsidy_list"],
|
||||
"min_items": 1,
|
||||
"description": "补助清单",
|
||||
"element_fields": [
|
||||
{"path": ["person_id"], "required": true, "check_empty": true, "description": "人员工号"},
|
||||
{"path": ["person_name"], "required": true, "check_empty": true, "description": "人员姓名"},
|
||||
{"path": ["start_date"], "required": true, "check_empty": true, "custom_check": "is_valid_date", "description": "补助开始日期"},
|
||||
{"path": ["end_date"], "required": true, "check_empty": true, "custom_check": "is_valid_date", "description": "补助结束日期"},
|
||||
{"path": ["days"], "required": true, "check_empty": true, "custom_check": "is_positive_integer", "description": "补助天数"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"path": ["attachments"],
|
||||
"min_items": 0,
|
||||
"description": "附件列表",
|
||||
"element_fields": [
|
||||
{"path": ["filename"], "required": true, "check_empty": true, "description": "文件名"},
|
||||
{"path": ["attachment_type"], "required": true, "check_empty": true, "description": "附件类型"}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"normal": {
|
||||
"description": "普通报销校验规则",
|
||||
"fields": [
|
||||
{
|
||||
"path": ["basic_info", "reimbursement_description"],
|
||||
"required": true,
|
||||
"check_empty": true,
|
||||
"description": "报销事由"
|
||||
},
|
||||
{
|
||||
"path": ["reimbursement_details", "total_invoices"],
|
||||
"required": true,
|
||||
"check_empty": true,
|
||||
"custom_check": "is_positive_integer",
|
||||
"description": "发票总数"
|
||||
},
|
||||
{
|
||||
"path": ["reimbursement_details", "total_amount"],
|
||||
"required": true,
|
||||
"check_empty": true,
|
||||
"custom_check": "is_positive_number",
|
||||
"description": "总金额"
|
||||
}
|
||||
],
|
||||
"arrays": [
|
||||
{
|
||||
"path": ["payment_methods"],
|
||||
"min_items": 1,
|
||||
"description": "支付方式记录",
|
||||
"element_fields": [
|
||||
{"path": ["card_date"], "required": true, "check_empty": true, "custom_check": "is_valid_date", "description": "刷卡日期"},
|
||||
{"path": ["card_amount"], "required": true, "check_empty": true, "custom_check": "is_positive_number", "description": "支付金额"},
|
||||
{"path": ["merchant"], "required": true, "check_empty": true, "description": "商户名称"},
|
||||
{"path": ["remark"], "required": true, "check_empty": false, "description": "备注"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"path": ["attachments"],
|
||||
"min_items": 0,
|
||||
"description": "附件列表",
|
||||
"element_fields": [
|
||||
{"path": ["filename"], "required": true, "check_empty": true, "description": "文件名"},
|
||||
{"path": ["attachment_type"], "required": true, "check_empty": true, "description": "附件类型"}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
856
docs/API.md
Normal file
856
docs/API.md
Normal file
@@ -0,0 +1,856 @@
|
||||
---
|
||||
last_reviewed: 2026-06-13
|
||||
---
|
||||
|
||||
# 财务报销自动化 — API 文档
|
||||
|
||||
> 基础地址: `http://localhost:5000`
|
||||
> 启动: `uv run python src/web/app.py`
|
||||
|
||||
## 总览
|
||||
|
||||
| # | 方法 | 路径 | 说明 |
|
||||
|---|------|------|------|
|
||||
| 1 | GET | `/` | PC 端主页 |
|
||||
| 2 | GET | `/mobile/<session_id>` | 移动端上传页面 |
|
||||
| 3 | POST | `/api/session` | 创建会话 |
|
||||
| 4 | POST | `/api/upload/<session_id>` | 上传文件(PDF/图片) |
|
||||
| 5 | GET | `/api/files/<session_id>` | 列出会话目录中的文件 |
|
||||
| 6 | GET | `/api/download/<session_id>/<filename>` | 下载生成的文件 |
|
||||
| 7 | POST | `/api/mobile-upload/<session_id>` | 移动端上传图片 |
|
||||
| 8 | GET | `/api/config/<session_id>` | 获取会话配置 |
|
||||
| 9 | GET | `/api/data/<session_id>` | 获取发票数据(JSON) |
|
||||
| 10 | POST | `/api/save/<session_id>` | 保存编辑后的发票数据 |
|
||||
| 11 | POST | `/api/process/<session_id>` | 启动管道处理(仅发票提取,不自动提交) |
|
||||
| 12 | GET | `/api/logs/<session_id>` | SSE 日志流 |
|
||||
| 13 | POST | `/api/submit-financial/<session_id>` | 提交到财务系统 |
|
||||
| **14** | **GET** | **`/api/agent/state/<session_id>`** | **获取 Agent 会话状态** |
|
||||
| **15** | **POST** | **`/api/agent/process/<session_id>`** | **启动 Agent 多轮处理(主入口)** |
|
||||
| **16** | **POST** | **`/api/agent/supplement/<session_id>`** | **补充文件后重新分析** |
|
||||
| **17** | **POST** | **`/api/agent/user-supplement/<session_id>`** | **通过文字补充信息** |
|
||||
| **18** | **POST** | **`/api/agent/force-submit/<session_id>`** | **强制提交,跳过校验** |
|
||||
|
||||
> 加粗条目为 Agent 多轮校验流程新增接口。
|
||||
|
||||
### 入口选择建议
|
||||
|
||||
- **推荐使用** `/api/agent/process`:完整流程,包含发票提取、LLM 信息校验、自动提交财务系统。
|
||||
- **仅发票提取** `/api/process`:跳过 Agent 校验,只做文档解析和发票分类。适合调试发票提取本身,或仅需导出 CSV 的场景。
|
||||
|
||||
---
|
||||
|
||||
## 会话与目录
|
||||
|
||||
- 调用 `POST /api/session` 获得 `session_id`
|
||||
- 该会话下所有文件存放在 `src/web/uploads/<session_id>/`
|
||||
- 典型产物:`invoice_summary.csv`、`payment_records.csv`、`易耗品、出库单.doc`、`config.json`、`session.log`、`result.json`、`agent_state.json`、`agent_events.log`、`file_events.log`、`llm_stream.log`
|
||||
|
||||
---
|
||||
|
||||
## 接口详情
|
||||
|
||||
### 1. 创建会话
|
||||
|
||||
```
|
||||
POST /api/session
|
||||
```
|
||||
|
||||
**响应:**
|
||||
|
||||
```json
|
||||
{ "session_id": "a1b2c3d4e5f6" }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. 上传文件(PDF/图片)
|
||||
|
||||
```
|
||||
POST /api/upload/<session_id>
|
||||
Content-Type: multipart/form-data
|
||||
```
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| file | File | PDF 发票或支付截图 |
|
||||
|
||||
**响应(成功):**
|
||||
|
||||
```json
|
||||
{ "ok": true, "filename": "1. 电容一批.pdf" }
|
||||
```
|
||||
|
||||
**响应(失败):**
|
||||
|
||||
```json
|
||||
{ "error": "未选择文件" }
|
||||
```
|
||||
|
||||
HTTP `400`
|
||||
|
||||
---
|
||||
|
||||
### 3. 列出会话文件
|
||||
|
||||
```
|
||||
GET /api/files/<session_id>
|
||||
```
|
||||
|
||||
**响应:**
|
||||
|
||||
```json
|
||||
{
|
||||
"files": [
|
||||
{ "name": "1. 电容一批.pdf", "type": "pdf", "size": 12345 },
|
||||
{ "name": "payment_01.jpg", "type": "image", "size": 67890 }
|
||||
],
|
||||
"pdfs": ["1. 电容一批.pdf"],
|
||||
"images": ["payment_01.jpg"]
|
||||
}
|
||||
```
|
||||
|
||||
`images` 包含扩展名:`.png`、`.jpg`、`.jpeg`、`.bmp`、`.webp`。
|
||||
|
||||
---
|
||||
|
||||
### 4. 下载文件
|
||||
|
||||
```
|
||||
GET /api/download/<session_id>/<filename>
|
||||
```
|
||||
|
||||
**响应:** 文件二进制流,带 `Content-Disposition: attachment` 与 UTF-8 文件名。
|
||||
|
||||
| 扩展名 | Content-Type |
|
||||
|--------|----------------|
|
||||
| `.csv` | `text/csv; charset=utf-8` |
|
||||
| `.doc` | `application/msword` |
|
||||
| 其它 | `application/octet-stream` |
|
||||
|
||||
**常见文件名:**
|
||||
|
||||
| 文件名 | 说明 |
|
||||
|--------|------|
|
||||
| `invoice_summary.csv` | 发票汇总 |
|
||||
| `payment_records.csv` | 支付记录 |
|
||||
| `易耗品、出库单.doc` | 自动填写的出库单 |
|
||||
| `travel_applications.json` | 差旅申请信息 |
|
||||
| `result.json` | 处理结果 |
|
||||
| `agent_state.json` | Agent 会话状态 |
|
||||
|
||||
**错误:**
|
||||
|
||||
```json
|
||||
{ "error": "文件不存在" }
|
||||
```
|
||||
|
||||
HTTP `404`。`filename` 仅允许会话目录内的文件名(防止路径穿越)。
|
||||
|
||||
---
|
||||
|
||||
### 5. 移动端上传页面
|
||||
|
||||
```
|
||||
GET /mobile/<session_id>
|
||||
```
|
||||
|
||||
返回移动端 HTML 页面。
|
||||
|
||||
---
|
||||
|
||||
### 6. 移动端上传图片
|
||||
|
||||
```
|
||||
POST /api/mobile-upload/<session_id>
|
||||
Content-Type: multipart/form-data
|
||||
```
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| file | File | 图片文件 |
|
||||
|
||||
逻辑与 `POST /api/upload/<session_id>` 相同。
|
||||
|
||||
---
|
||||
|
||||
### 7. 获取会话配置
|
||||
|
||||
```
|
||||
GET /api/config/<session_id>
|
||||
```
|
||||
|
||||
获取当前会话的配置,供前端回填表单。优先读取会话目录下的 `config.json`,未找到则使用项目全局配置。
|
||||
|
||||
**响应:**
|
||||
|
||||
```json
|
||||
{
|
||||
"username": "",
|
||||
"password": "",
|
||||
"default_name": "",
|
||||
"default_card_no": "",
|
||||
"default_person_id": "",
|
||||
"consumable_storage": ""
|
||||
}
|
||||
```
|
||||
|
||||
注意:`password` 字段始终返回空字符串。
|
||||
|
||||
---
|
||||
|
||||
### 8. 获取发票数据
|
||||
|
||||
```
|
||||
GET /api/data/<session_id>
|
||||
```
|
||||
|
||||
**响应:**
|
||||
|
||||
```json
|
||||
{
|
||||
"csv_filename": "payment_records.csv",
|
||||
"fields": [
|
||||
"序号", "发票号码", "开票日期", "项目名称", "规格型号",
|
||||
"价税合计", "销售方名称", "人员姓名", "刷卡日期",
|
||||
"公务卡号", "刷卡金额", "备注", "工号"
|
||||
],
|
||||
"data": [
|
||||
{
|
||||
"__row": 0,
|
||||
"序号": "1",
|
||||
"发票号码": "26442000005432755951",
|
||||
"价税合计": "2900.00"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
读取优先级:`payment_records.csv` → `invoice_summary.csv` → 任意 `.csv` 文件。
|
||||
|
||||
- `fields`:列顺序
|
||||
- `data[].__row`:内部行索引(保存时不需要提交,服务端按数组顺序写回)
|
||||
|
||||
**错误:** `404` 未找到 CSV;`500` 读取失败。
|
||||
|
||||
---
|
||||
|
||||
### 9. 保存编辑后的发票数据
|
||||
|
||||
```
|
||||
POST /api/save/<session_id>
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
**请求体:**
|
||||
|
||||
```json
|
||||
{
|
||||
"csv_filename": "invoice_summary.csv",
|
||||
"data": [
|
||||
{
|
||||
"序号": "1",
|
||||
"发票号码": "26442000005432755951",
|
||||
"开票日期": "2026/05/18",
|
||||
"项目名称": "...",
|
||||
"规格型号": "...",
|
||||
"价税合计": "2900.00",
|
||||
"销售方名称": "...",
|
||||
"人员姓名": "",
|
||||
"刷卡日期": "2026/04/28",
|
||||
"公务卡号": "",
|
||||
"刷卡金额": "2850.00",
|
||||
"备注": "",
|
||||
"工号": "202407021"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**响应(成功):**
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"doc_ok": true,
|
||||
"doc_url": "/api/download/<session_id>/%E6%98%93%E8%80%97%E5%93%81%E3%80%81%E5%87%BA%E5%BA%93%E5%8D%95.doc"
|
||||
}
|
||||
```
|
||||
|
||||
保存后会根据最新 CSV **重新生成** 出库单 Word(与会话 `config.json` 中的 `consumable_storage` 等配置一致)。
|
||||
|
||||
**纯差旅发票跳过出库单:**
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"doc_ok": null,
|
||||
"doc_skipped": true
|
||||
}
|
||||
```
|
||||
|
||||
**出库单生成失败:**
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"doc_ok": false,
|
||||
"doc_error": "出库单模板不存在,请将模板放在项目根目录"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 10. 启动管道处理(仅发票提取)
|
||||
|
||||
```
|
||||
POST /api/process/<session_id>
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
> 此接口仅执行发票提取和 LLM 识别,**不会**触发 Agent 多轮校验,也**不会**自动提交到财务系统。如需完整的 Agent 校验流程,请使用 `/api/agent/process`。
|
||||
|
||||
**请求体:**
|
||||
|
||||
| 字段 | 类型 | 必填 | 说明 |
|
||||
|------|------|------|------|
|
||||
| username | string | 否 | 财务系统工号 |
|
||||
| password | string | 否 | 登录密码 |
|
||||
| default_name | string | 否 | 默认报销人姓名 |
|
||||
| default_card_no | string | 否 | 默认公务卡号 |
|
||||
| default_person_id | string | 否 | 默认人员编号 |
|
||||
| consumable_storage | string | 否 | 出库单存放地点 |
|
||||
|
||||
**处理内容:**
|
||||
|
||||
1. 从会话目录 PDF 提取发票信息 → `invoice_summary.csv`
|
||||
2. 对支付截图多模态 LLM 识别,回填刷卡字段
|
||||
3. 根据发票类型自动分类:差旅发票(高铁票/酒店住宿)不生成出库单;普通发票从模板复制并自动填写
|
||||
|
||||
配置会写入 `src/web/uploads/<session_id>/config.json`。
|
||||
|
||||
**响应(立即):**
|
||||
|
||||
```json
|
||||
{ "status": "started" }
|
||||
```
|
||||
|
||||
处理在后台线程执行,进度与结果通过 `GET /api/logs/<session_id>`(SSE)获取。
|
||||
|
||||
**SSE 完成时 `result` 示例(成功):**
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"elapsed": "45.2s",
|
||||
"invoice_count": 4,
|
||||
"csv_url": "/api/download/<session_id>/invoice_summary.csv",
|
||||
"travel_count": 2,
|
||||
"general_count": 2,
|
||||
"doc_url": "/api/download/<session_id>/%E6%98%93%E8%80%97%E5%93%81%E3%80%81%E5%87%BA%E5%BA%93%E5%8D%95.doc",
|
||||
"doc_ok": true
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 11. SSE 日志流
|
||||
|
||||
```
|
||||
GET /api/logs/<session_id>
|
||||
Accept: text/event-stream
|
||||
```
|
||||
|
||||
每 0.5 秒轮询 4 个日志文件,通过文件 size 增量检测新内容:
|
||||
|
||||
| 文件 | 内容 |
|
||||
|------|------|
|
||||
| `session.log` | 普通日志(extractor、llm_extractor、matcher、pipeline、bot、agent、validator 等模块) |
|
||||
| `file_events.log` | 文件处理进度事件 |
|
||||
| `llm_stream.log` | LLM 流式输出 |
|
||||
| `agent_events.log` | Agent 调度事件 |
|
||||
|
||||
检测到 `result.json` 存在时,读取后发送 `done` 事件并断开连接。
|
||||
|
||||
**SSE 超时:** 900 秒。
|
||||
|
||||
---
|
||||
|
||||
### 12. 提交到财务系统
|
||||
|
||||
```
|
||||
POST /api/submit-financial/<session_id>
|
||||
```
|
||||
|
||||
**前置条件:**
|
||||
|
||||
- 会话目录存在 `config.json`,否则返回 `400`
|
||||
- 存在可用的发票 CSV(通常为 `invoice_summary.csv` 或 `payment_records.csv`)
|
||||
|
||||
**说明:**
|
||||
|
||||
- 前端一般在提交前调用 `/api/save` 保存表格修改
|
||||
- 根据发票类型选择填报模式:纯差旅发票走差旅报销流程,含普通发票走普通报销流程
|
||||
|
||||
**响应(立即):**
|
||||
|
||||
```json
|
||||
{ "status": "started" }
|
||||
```
|
||||
|
||||
**SSE 完成示例:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "done",
|
||||
"result": {
|
||||
"ok": true,
|
||||
"submit_ok": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
失败时 `submit_ok: false`,`submit_error` 为错误描述。
|
||||
|
||||
---
|
||||
|
||||
## Agent 多轮校验流程
|
||||
|
||||
Agent 是系统的调度中枢,负责编排信息提取、规则校验、补充材料请求的完整流程。推荐使用 `/api/agent/process` 作为主入口。
|
||||
|
||||
### Agent 状态机
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> IDLE
|
||||
IDLE --> EXTRACTING: 启动处理
|
||||
EXTRACTING --> READY: can_submit == true
|
||||
EXTRACTING --> AWAITING_SUPPLEMENT: can_submit == false
|
||||
EXTRACTING --> ERROR: 异常 / 轮次超限
|
||||
READY --> SUBMITTING: _emit_ready_and_submit()
|
||||
SUBMITTING --> DONE: 财务提交完成
|
||||
AWAITING_SUPPLEMENT --> EXTRACTING: 用户补充文件/文字
|
||||
AWAITING_SUPPLEMENT --> READY: 用户强制提交
|
||||
|
||||
note right of EXTRACTING
|
||||
LLM 提取 + validator 校验\n最多 3 次重试
|
||||
end note
|
||||
```
|
||||
|
||||
### 13. 获取 Agent 会话状态
|
||||
|
||||
```
|
||||
GET /api/agent/state/<session_id>
|
||||
```
|
||||
|
||||
**响应:**
|
||||
|
||||
```json
|
||||
{
|
||||
"session_id": "a1b2c3d4e5f6",
|
||||
"state": "extracting",
|
||||
"rounds": 1,
|
||||
"max_rounds": 5,
|
||||
"invoice_type": "travel",
|
||||
"extracted_info": { ... },
|
||||
"validation_reports": [ ... ],
|
||||
"user_supplements": [ ... ],
|
||||
"error_message": ""
|
||||
}
|
||||
```
|
||||
|
||||
**状态值:**
|
||||
|
||||
| 状态 | 含义 |
|
||||
|------|------|
|
||||
| `idle` | 初始状态 |
|
||||
| `extracting` | LLM 正在分析文件 |
|
||||
| `awaiting_supplement` | 信息不完整,等待用户补充 |
|
||||
| `ready` | 信息完整,可以提交 |
|
||||
| `submitting` | 正在提交到财务系统 |
|
||||
| `done` | 流程结束 |
|
||||
| `error` | 出错 |
|
||||
|
||||
---
|
||||
|
||||
### 14. 启动 Agent 多轮处理(主入口)
|
||||
|
||||
```
|
||||
POST /api/agent/process/<session_id>
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
**请求体:**
|
||||
|
||||
| 字段 | 类型 | 必填 | 说明 |
|
||||
|------|------|------|------|
|
||||
| username | string | 否 | 财务系统工号 |
|
||||
| password | string | 否 | 登录密码 |
|
||||
| default_name | string | 否 | 默认报销人姓名 |
|
||||
| default_card_no | string | 否 | 默认公务卡号 |
|
||||
| default_person_id | string | 否 | 默认人员编号 |
|
||||
| consumable_storage | string | 否 | 出库单存放地点 |
|
||||
|
||||
**处理流程:**
|
||||
|
||||
1. 发票提取(同 `/api/process`)
|
||||
2. Agent 调度 LLM 分析提取结果
|
||||
3. validator 规则校验(最多 3 次校验-修正循环)
|
||||
4. LLM 语义判断信息完整性(`can_submit` 字段)
|
||||
5. 校验通过 → 自动提交到财务系统
|
||||
6. 校验未通过 → 等待用户补充材料
|
||||
|
||||
**响应(立即):**
|
||||
|
||||
```json
|
||||
{ "status": "started" }
|
||||
```
|
||||
|
||||
**SSE done 事件 - 信息完整(成功提交):**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "done",
|
||||
"result": {
|
||||
"ok": true,
|
||||
"agent_ready": true,
|
||||
"submit_ok": true,
|
||||
"round": 1,
|
||||
"message": "信息完整,已自动提交到财务系统"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**SSE done 事件 - 信息完整但提交失败:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "done",
|
||||
"result": {
|
||||
"ok": true,
|
||||
"agent_ready": true,
|
||||
"submit_ok": false,
|
||||
"submit_error": "提交失败原因",
|
||||
"round": 1,
|
||||
"message": "校验通过但提交失败"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**SSE done 事件 - 需补充材料:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "done",
|
||||
"result": {
|
||||
"ok": true,
|
||||
"agent_ready": false,
|
||||
"agent_state": "awaiting_supplement",
|
||||
"round": 1,
|
||||
"waiting_for_supplement": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**SSE done 事件 - 处理失败:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "done",
|
||||
"result": {
|
||||
"ok": false,
|
||||
"error": "未提取到任何发票数据"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 15. 补充文件后重新分析
|
||||
|
||||
```
|
||||
POST /api/agent/supplement/<session_id>
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
在 Agent 请求补充材料后,用户上传新文件并调用此接口触发重新分析。
|
||||
|
||||
**请求体:**
|
||||
|
||||
```json
|
||||
{
|
||||
"files": ["补充材料1.pdf", "补充材料2.jpg"]
|
||||
}
|
||||
```
|
||||
|
||||
**处理流程:**
|
||||
|
||||
1. 记录补充文件
|
||||
2. 重新提取所有发票(包含新文件)
|
||||
3. 加载上一轮分析结果作为历史上下文
|
||||
4. 重新执行 Agent 校验
|
||||
|
||||
**响应(立即):**
|
||||
|
||||
```json
|
||||
{ "status": "started" }
|
||||
```
|
||||
|
||||
**SSE done 事件:** 同上(可能仍需补充或校验通过自动提交)。
|
||||
|
||||
**错误:**
|
||||
|
||||
```json
|
||||
{ "error": "未找到 Agent 状态" }
|
||||
```
|
||||
|
||||
HTTP `404`(未先调用 `/api/agent/process` 或 Agent 状态已丢失)。
|
||||
|
||||
---
|
||||
|
||||
### 16. 通过文字补充信息
|
||||
|
||||
```
|
||||
POST /api/agent/user-supplement/<session_id>
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
用户通过对话方式提供补充信息,LLM 解析后更新已提取的信息并重新校验。
|
||||
|
||||
**请求体:**
|
||||
|
||||
```json
|
||||
{
|
||||
"text": "报销人是张三,公务卡号是 6228480402564890001"
|
||||
}
|
||||
```
|
||||
|
||||
**处理流程:**
|
||||
|
||||
1. LLM 分析用户文字,提取需要更新的字段
|
||||
2. 合并到已提取的信息中
|
||||
3. 保存到缓存
|
||||
4. 重新执行 Agent 校验
|
||||
|
||||
**响应(立即):**
|
||||
|
||||
```json
|
||||
{ "status": "started" }
|
||||
```
|
||||
|
||||
**SSE done 事件:** 同上(可能仍需补充或校验通过自动提交)。
|
||||
|
||||
**错误:**
|
||||
|
||||
```json
|
||||
{ "error": "请输入补充信息" }
|
||||
```
|
||||
|
||||
HTTP `400`(文本为空)。
|
||||
|
||||
---
|
||||
|
||||
### 17. 强制提交,跳过校验
|
||||
|
||||
```
|
||||
POST /api/agent/force-submit/<session_id>
|
||||
```
|
||||
|
||||
当 Agent 校验未通过或出错时,用户可选择强制提交,跳过所有校验直接提交到财务系统。
|
||||
|
||||
**处理流程:**
|
||||
|
||||
1. 将 Agent 状态设为 `READY`
|
||||
2. 执行财务提交
|
||||
|
||||
**响应(立即):**
|
||||
|
||||
```json
|
||||
{ "status": "started" }
|
||||
```
|
||||
|
||||
**SSE done 事件:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "done",
|
||||
"result": {
|
||||
"ok": true,
|
||||
"submit_ok": true,
|
||||
"submit_error": null
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SSE 事件类型
|
||||
|
||||
### Agent 事件
|
||||
|
||||
通过 `agent_events.log` 轮询推送:
|
||||
|
||||
| 事件类型 | 数据结构 | 触发条件 |
|
||||
|---|---|---|
|
||||
| `agent_state_change` | `{type, state, round, attempt, message}` | 状态切换 |
|
||||
| `agent_ready` | `{type, round, message}` | 双重校验通过 |
|
||||
| `agent_request_supplement` | `{type, round, missing_fields, missing_materials, semantic_issues, suggestion}` | 校验未通过 |
|
||||
| `agent_supplement_received` | `{type, files}` | 收到用户补充 |
|
||||
| `agent_force_submit` | `{type, message}` | 用户强制提交 |
|
||||
| `agent_extract_status` | `{type, state, round, attempt, message}` | 校验-修正循环中的每次尝试结果 |
|
||||
| `agent_error` | `{type, message}` | 提取失败 |
|
||||
| `agent_max_rounds` | `{type, message}` | 达到最大轮次 |
|
||||
|
||||
### 文件进度事件
|
||||
|
||||
通过 `file_events.log` 轮询推送:
|
||||
|
||||
| 事件类型 | 数据结构 | 触发条件 |
|
||||
|---|---|---|
|
||||
| `file_progress` | `{type, file, status, summary?, error?}` | 文件处理状态变更 |
|
||||
|
||||
`status` 取值: `processing` / `done` / `cached` / `error`
|
||||
|
||||
### LLM 流式事件
|
||||
|
||||
通过 `llm_stream.log` 轮询推送:
|
||||
|
||||
| 事件类型 | 数据结构 | 触发条件 |
|
||||
|---|---|---|
|
||||
| `llm_stream` | `{type, phase, text?}` | LLM 输出流 |
|
||||
|
||||
`phase` 取值: `start` / `reasoning` / `chunk` / `end` / `error`
|
||||
|
||||
### 完成事件
|
||||
|
||||
SSE 检测到 `result.json` 后直接发送:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "done",
|
||||
"result": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 错误码汇总
|
||||
|
||||
| HTTP | 场景 |
|
||||
|------|------|
|
||||
| 400 | 参数缺失、未找到配置、文本为空等 |
|
||||
| 404 | `session_id` 不存在、文件不存在、Agent 状态丢失 |
|
||||
| 500 | CSV 读取失败、服务未初始化等内部错误 |
|
||||
|
||||
统一错误体:
|
||||
|
||||
```json
|
||||
{ "error": "错误描述" }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 发票类型分类
|
||||
|
||||
系统自动将发票分为两类,影响出库单生成和后续报销流程:
|
||||
|
||||
| 类型 | 判断依据 | 出库单 | 报销流程 |
|
||||
|------|----------|--------|----------|
|
||||
| 差旅发票 | 高铁票、酒店住宿等 | 不生成 | 差旅报销 |
|
||||
| 普通发票 | 其他(办公用品、耗材等) | 自动生成 | 普通报销 |
|
||||
|
||||
`/api/process` 和 `/api/save` 的响应中 `travel_count` / `general_count` 即为分类统计。
|
||||
|
||||
---
|
||||
|
||||
## 端到端流程
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F as 前端
|
||||
participant S as SSE连接
|
||||
participant B as 后端线程
|
||||
participant A as Agent调度器
|
||||
|
||||
F->>B: POST /api/session
|
||||
B-->>F: session_id
|
||||
|
||||
F->>B: POST /api/upload/{sid} (多次)
|
||||
F->>B: POST /api/agent/process/{sid}
|
||||
B-->>F: {status: "started"}
|
||||
|
||||
F->>S: GET /api/logs/{sid}
|
||||
|
||||
Note over B,A: 后台线程启动
|
||||
B->>A: extract_invoices()
|
||||
S-->>F: file_progress (processing/done)
|
||||
S-->>F: llm_stream (start/chunk/end)
|
||||
S-->>F: agent_state_change (extracting)
|
||||
|
||||
Note over A: LLM 提取 + validator 校验<br/>最多 3 次重试
|
||||
|
||||
alt 信息完整
|
||||
A->>A: state → READY
|
||||
A->>A: _emit_ready_and_submit()
|
||||
S-->>F: agent_ready
|
||||
S-->>F: done (submit_ok=true)
|
||||
F->>F: es.close()
|
||||
else 信息不完整
|
||||
A->>A: state → AWAITING_SUPPLEMENT
|
||||
S-->>F: agent_request_supplement
|
||||
S-->>F: done (waiting_for_supplement=true)
|
||||
F->>F: es.close()
|
||||
|
||||
F->>B: 上传补充文件或输入文字
|
||||
alt 文件补充
|
||||
F->>B: POST /api/agent/supplement/{sid}
|
||||
else 文字补充
|
||||
F->>B: POST /api/agent/user-supplement/{sid}
|
||||
end
|
||||
B-->>F: {status: "started"}
|
||||
F->>S: GET /api/logs/{sid}
|
||||
|
||||
Note over A: 重新分析 + 校验
|
||||
|
||||
alt 仍不完整
|
||||
S-->>F: agent_request_supplement
|
||||
S-->>F: done (waiting=true)
|
||||
F->>F: es.close()
|
||||
Note over F: 可继续补充或强制提交
|
||||
else 完整
|
||||
A->>A: state → READY
|
||||
A->>A: _emit_ready_and_submit()
|
||||
S-->>F: agent_ready
|
||||
S-->>F: done (submit_ok=true)
|
||||
F->>F: es.close()
|
||||
end
|
||||
|
||||
alt 强制提交
|
||||
F->>B: POST /api/agent/force-submit/{sid}
|
||||
B-->>F: {status: "started"}
|
||||
F->>S: GET /api/logs/{sid}
|
||||
S-->>F: agent_force_submit
|
||||
S-->>F: done
|
||||
F->>F: es.close()
|
||||
end
|
||||
end
|
||||
|
||||
F->>B: GET /api/data/{sid}
|
||||
B-->>F: fields + data
|
||||
F->>B: POST /api/save/{sid}
|
||||
Note over B: 更新 CSV,重新生成出库单
|
||||
B-->>F: doc_url
|
||||
|
||||
F->>B: GET /api/download/{sid}/易耗品、出库单.doc
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 相关 CLI
|
||||
|
||||
不经过 Web、在本地直接填写出库单:
|
||||
|
||||
```bash
|
||||
uv run python -m src.infra.documents.consumable --csv invoice_summary.csv --doc "易耗品、出库单.doc"
|
||||
```
|
||||
|
||||
详见 [README.md](./README.md)。
|
||||
18
docs/README.md
Normal file
18
docs/README.md
Normal file
@@ -0,0 +1,18 @@
|
||||
---
|
||||
last_reviewed: 2026-06-11
|
||||
---
|
||||
|
||||
# docs — 公开文档目录
|
||||
|
||||
本目录存放面向项目用户和外部贡献者的公开文档及说明文件。
|
||||
|
||||
## 文档清单
|
||||
|
||||
| 文件 | 说明 |
|
||||
|------|------|
|
||||
| `API.md` | Web API 完整接口文档:路由、请求/响应格式、SSE 日志流、错误码、端到端流程 |
|
||||
| `报销操作指南.md` | 面向最终用户的操作步骤说明 |
|
||||
|
||||
## 文档边界
|
||||
|
||||
维护规范、实施方案、经验总结等内部资料统一放置在 `.agents/` 目录下,不混入本目录。
|
||||
454
docs/报销操作指南.md
Normal file
454
docs/报销操作指南.md
Normal file
@@ -0,0 +1,454 @@
|
||||
---
|
||||
last_reviewed: 2026-06-09
|
||||
---
|
||||
|
||||
# 阜阳师范大学财务报销系统 - 自动化操作指南
|
||||
|
||||
> 适用场景:日常报销录入(支持 Web 界面和 CLI 两种方式)
|
||||
> 最后更新:2026-06-09
|
||||
|
||||
---
|
||||
|
||||
## 一、系统概览
|
||||
|
||||
```
|
||||
整体流程:
|
||||
|
||||
信息门户(SSO登录) → 财务系统入口 → 单点登录页 → 网络报销 → 日常报销录入
|
||||
(tyrz.fynu.edu.cn) (点击"财务系统") (新标签页) (a:has(img)) (/expen/common/common)
|
||||
|
||||
目标系统: http://210.45.32.214:8081
|
||||
用户: 张三 (工号: xxxxxxx)
|
||||
```
|
||||
|
||||
### 关键 URL
|
||||
|
||||
| 系统 | URL | 说明 |
|
||||
|------|-----|------|
|
||||
| SSO 登录 | `https://tyrz.fynu.edu.cn/sso/login` | 统一认证入口 |
|
||||
| 信息门户 | `https://tyrz.fynu.edu.cn/oshall` | 登录后跳转目标 |
|
||||
| 报销系统 | `http://210.45.32.214:8081` | 网络报销主系统 |
|
||||
| 日常报销录入 | `/expen/common/common?v=4.0` | 目标录入页面 |
|
||||
|
||||
---
|
||||
|
||||
## 二、两种使用方式
|
||||
|
||||
### 方式一:Web 界面(推荐)
|
||||
|
||||
适合日常使用,可视化操作,支持手机端拍照上传。
|
||||
|
||||
```bash
|
||||
uv run python src/web/app.py
|
||||
# 访问: http://localhost:5000
|
||||
```
|
||||
|
||||
### 方式二:CLI 命令行
|
||||
|
||||
适合脚本化、批量处理。
|
||||
|
||||
```bash
|
||||
# 全流程(发票提取 → LLM 识别 → 浏览器填报)
|
||||
uv run python src/main.py
|
||||
|
||||
# 分步执行
|
||||
uv run python src/main.py --step invoice # 仅发票提取
|
||||
uv run python src/main.py --step submit # 仅浏览器填报
|
||||
|
||||
# 覆盖登录凭据
|
||||
uv run python src/main.py -u 202407021 -p "your_password"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 三、数据准备
|
||||
|
||||
### 3.1 发票数据 CSV
|
||||
|
||||
上传 PDF 发票文件和支付截图,系统自动完成:
|
||||
1. 从 PDF 提取发票信息
|
||||
2. 多模态 LLM 识别支付截图中的刷卡信息
|
||||
3. 生成 `invoice_summary.csv`
|
||||
|
||||
### 3.2 发票类型分类
|
||||
|
||||
系统自动将发票分为两类,影响后续处理流程:
|
||||
|
||||
| 类型 | 判断依据 | 出库单 | 报销流程 |
|
||||
|------|----------|--------|----------|
|
||||
| 差旅发票 | 高铁票、酒店住宿等 | 不生成 | 差旅报销 |
|
||||
| 普通发票 | 其他(办公用品、耗材等) | 自动生成 | 普通报销 |
|
||||
|
||||
### 3.3 附件文件
|
||||
|
||||
PDF 发票文件需与支付截图配对上传。系统通过金额匹配自动关联发票和支付记录。
|
||||
|
||||
---
|
||||
|
||||
## 四、Web 界面操作流程
|
||||
|
||||
### 4.1 启动服务
|
||||
|
||||
```bash
|
||||
uv run python src/web/app.py
|
||||
# 访问: http://localhost:5000
|
||||
```
|
||||
|
||||
### 4.2 上传文件
|
||||
|
||||
1. **发票 PDF**:点击或拖拽上传 PDF 文件(支持多选)
|
||||
2. **支付截图**:点击或拖拽上传图片文件(支持多选)
|
||||
3. **手机扫码上传**:扫描页面二维码,通过手机拍照上传支付截图
|
||||
|
||||
### 4.3 配置信息
|
||||
|
||||
填写以下配置项(也可通过上传 `config.json` 快速填充):
|
||||
|
||||
| 字段 | 说明 |
|
||||
|------|------|
|
||||
| 账号 | 财务系统工号 |
|
||||
| 密码 | 登录密码 |
|
||||
| 默认姓名 | 报销人姓名 |
|
||||
| 公务卡号 | 公务卡卡号 |
|
||||
| 人员编号 | 人员编号 |
|
||||
| 存放地点 | 出库单存放地点 |
|
||||
|
||||
### 4.4 开始处理
|
||||
|
||||
点击"开始处理"按钮,系统自动执行:
|
||||
|
||||
1. **发票提取**:从 PDF 提取发票信息
|
||||
2. **LLM 信息提取**:多模态 LLM 识别支付截图中的刷卡信息
|
||||
3. **发票分类**:自动区分差旅发票和普通发票
|
||||
4. **出库单生成**:普通发票自动生成易耗品出库单(差旅发票跳过)
|
||||
|
||||
处理过程中可在日志面板实时查看进度。
|
||||
|
||||
### 4.5 编辑发票数据
|
||||
|
||||
处理完成后,发票数据以可编辑表格形式展示:
|
||||
|
||||
- 直接点击单元格即可编辑
|
||||
- 修改后点击"提交到财务系统"时自动保存
|
||||
- 保存后会自动重新生成出库单
|
||||
|
||||
### 4.6 下载文件
|
||||
|
||||
处理完成后下载:
|
||||
|
||||
| 文件 | 说明 |
|
||||
|------|------|
|
||||
| `invoice_summary.csv` | 发票汇总数据 |
|
||||
| `易耗品、出库单.doc` | 自动填写的出库单(仅普通发票) |
|
||||
|
||||
### 4.7 提交到财务系统
|
||||
|
||||
确认数据无误后,点击"提交到财务系统"按钮,系统自动:
|
||||
|
||||
1. 登录信息门户
|
||||
2. 进入财务系统
|
||||
3. 创建报销单
|
||||
4. 填写基本信息
|
||||
5. 录入报销明细
|
||||
6. 录入支付方式
|
||||
7. 上传附件
|
||||
|
||||
---
|
||||
|
||||
## 五、CLI 执行流程
|
||||
|
||||
### 5.1 全流程执行
|
||||
|
||||
```bash
|
||||
uv run python src/main.py
|
||||
```
|
||||
|
||||
执行步骤:
|
||||
|
||||
```
|
||||
Step 1: 发票提取
|
||||
├── 扫描 PDF 发票文件
|
||||
├── 提取发票信息
|
||||
├── 生成 invoice_summary.csv
|
||||
└── 自动分类(差旅/普通)
|
||||
|
||||
Step 2: LLM 信息提取
|
||||
├── 扫描支付截图
|
||||
├── 多模态 LLM 识别刷卡信息
|
||||
├── 回填 CSV 刷卡字段
|
||||
└── 更新 invoice_summary.csv
|
||||
|
||||
Step 3: 报销提交
|
||||
├── 登录信息门户
|
||||
├── 进入财务系统
|
||||
├── 创建报销单
|
||||
├── 填写基本信息
|
||||
├── 录入报销明细
|
||||
├── 录入支付方式
|
||||
├── 上传附件
|
||||
└── 提交(需手动确认)
|
||||
```
|
||||
|
||||
### 5.2 分步执行
|
||||
|
||||
```bash
|
||||
# 仅发票提取
|
||||
uv run python src/main.py --step invoice
|
||||
|
||||
# 仅浏览器填报
|
||||
uv run python src/main.py --step submit
|
||||
```
|
||||
|
||||
### 5.3 上传目录
|
||||
|
||||
CLI 模式自动发现 `src/web/uploads/` 下最新的会话文件夹,该文件夹包含上传的 PDF 和图片。
|
||||
|
||||
---
|
||||
|
||||
## 六、页面元素速查
|
||||
|
||||
### 基本信息页
|
||||
|
||||
| 字段 | 选择器 | 操作 |
|
||||
|------|--------|------|
|
||||
| 报销说明 | `#EXPENEXPLAIN` | fill |
|
||||
| 项目代码 | `#PROJECTCODE` | click → 弹窗选择 |
|
||||
| 项目弹窗 | `#promodal .fixed-table-body tbody tr` | 点击第一行 |
|
||||
| 下一步按钮 | `#saveAndNext` | click |
|
||||
|
||||
### 报销明细页
|
||||
|
||||
| 字段 | 选择器 | 操作 |
|
||||
|------|--------|------|
|
||||
| 增加按钮 | `#insertDetail` | click |
|
||||
| 经济事项代码 | `#economicscode2` | click → 弹窗选择 |
|
||||
| 经济科目弹窗 | `#econmodal .fixed-table-body tbody tr` | 点击第 3 行 |
|
||||
| 单据数 | `input[name="expenPwCommondetail.HOWBILLS"]` | fill |
|
||||
| 报销总金额 | `#je_zwzcdz` | fill |
|
||||
| 确定按钮 | `#detailAdd` | click |
|
||||
|
||||
### 支付方式页
|
||||
|
||||
| 字段 | 选择器 | 操作 |
|
||||
|------|--------|------|
|
||||
| 增加按钮 | `#insertPay` | click |
|
||||
| 人员编号 | `#personid2` | fill |
|
||||
| 人员姓名 | `#accountname2` | fill |
|
||||
| 刷卡日期 | `#receiptdate2` | fill |
|
||||
| 公务卡号 | `#localaccount2` | fill |
|
||||
| 刷卡金额 | `#receiptmoney2` | fill |
|
||||
| 实报金额 | `#money2` | fill |
|
||||
| 商户 | `#merchant2` | fill |
|
||||
| 备注 | `#smark2` | fill |
|
||||
| 确定按钮 | `#payAdd` | click |
|
||||
|
||||
### 附件清单页
|
||||
|
||||
| 字段 | 选择器 | 操作 |
|
||||
|------|--------|------|
|
||||
| 增加按钮 | `#insertAcc` | click |
|
||||
| 附件类型 | `#fjlx` | select_option → '1'(发票) |
|
||||
| 附件说明 | `#fpsmxx` | fill |
|
||||
| 文件上传 | `#file` | set_input_files |
|
||||
| 确定按钮 | `#cjtj` | click |
|
||||
|
||||
### 提交
|
||||
|
||||
| 操作 | 选择器 |
|
||||
|------|--------|
|
||||
| 提交按钮 | `#submit` |
|
||||
| 提交按钮(备用) | `#submit2` |
|
||||
|
||||
---
|
||||
|
||||
## 七、数据流向
|
||||
|
||||
```
|
||||
PDF 发票 + 支付截图
|
||||
│
|
||||
▼ extract_invoices()
|
||||
│
|
||||
├── 读取 PDF 发票
|
||||
├── 提取发票信息
|
||||
├── 自动分类(差旅/普通)
|
||||
└── 输出: invoice_summary.csv
|
||||
│
|
||||
▼ enrich_with_llm()
|
||||
├── 多模态 LLM 识别支付截图
|
||||
├── 回填刷卡字段
|
||||
└── 更新 invoice_summary.csv
|
||||
│
|
||||
▼ fill_consumable_from_template()
|
||||
├── 仅普通发票
|
||||
├── 从模板复制出库单
|
||||
└── 自动填写出库单
|
||||
│
|
||||
▼ run_bot()
|
||||
├── 登录信息门户
|
||||
├── 进入财务系统
|
||||
├── 创建报销单
|
||||
├── 填写基本信息
|
||||
├── 录入报销明细
|
||||
├── 录入支付方式
|
||||
├── 上传附件
|
||||
└── 提交
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 八、关键设计说明
|
||||
|
||||
### 8.1 浏览器启动
|
||||
|
||||
系统使用 Playwright 的 `sync_playwright()` 启动 Chromium 浏览器,支持 `headless` 模式。每次运行均启动新浏览器实例,需重新登录。可通过 `headless` 参数控制是否显示浏览器窗口。
|
||||
|
||||
### 8.2 明细录入策略
|
||||
|
||||
系统采用"一条总明细"策略:将所有发票合并为一条报销明细,报销总金额为所有发票刷卡金额之和,单据数为发票总张数。支付方式则逐张发票分别录入,每张发票对应一条支付记录。
|
||||
|
||||
### 8.3 经济科目选择
|
||||
|
||||
系统在经济科目弹窗中固定选择第 3 行。如需更改科目,修改 `rows[2]` 的索引即可。
|
||||
|
||||
### 8.4 项目选择
|
||||
|
||||
系统在项目选择弹窗中固定选择第 1 行。如需更改项目,修改 `first_row` 的选择逻辑即可。
|
||||
|
||||
### 8.5 网络报销链接动态获取
|
||||
|
||||
单点登录页的"网络报销"链接参数每次不同,系统通过 `a:has(img[src*="wlbx"])` 精确定位链接,动态提取 `href` 属性后导航,不硬编码 URL。
|
||||
|
||||
### 8.6 发票类型分流
|
||||
|
||||
- **差旅发票**(高铁票/酒店住宿):不生成易耗品出库单,走差旅报销流程
|
||||
- **普通发票**:生成易耗品出库单,走普通报销流程
|
||||
|
||||
### 8.7 移动端同步
|
||||
|
||||
PC 端生成二维码指向移动端上传页面,手机端上传的图片通过轮询同步到 PC 端,实现跨设备协作。
|
||||
|
||||
---
|
||||
|
||||
## 九、配置说明
|
||||
|
||||
### config.json
|
||||
|
||||
项目根目录 `config.json` 包含默认配置:
|
||||
|
||||
```json
|
||||
{
|
||||
"username": "202407021",
|
||||
"password": "your_password",
|
||||
"sso_login_url": "https://tyrz.fynu.edu.cn/sso/login",
|
||||
"portal_url": "https://tyrz.fynu.edu.cn/oshall",
|
||||
"reimburse_url": "http://210.45.32.214:8081",
|
||||
"reimburse_page": "/expen/common/common?v=4.0",
|
||||
"default_name": "王建锋",
|
||||
"default_card_no": "6282880139161682",
|
||||
"default_person_id": "202407021",
|
||||
"consumable_storage": "躬行楼 C205",
|
||||
"llm": {
|
||||
"model": "qwen3.5-9b",
|
||||
"api_base": "http://100.123.83.115:1234/v1",
|
||||
"api_key": "123456"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| 字段 | 说明 |
|
||||
|------|------|
|
||||
| `username` | 财务系统工号 |
|
||||
| `password` | 登录密码 |
|
||||
| `sso_login_url` | SSO 登录地址 |
|
||||
| `portal_url` | 信息门户地址 |
|
||||
| `reimburse_url` | 报销系统地址 |
|
||||
| `reimburse_page` | 报销录入页面路径 |
|
||||
| `default_name` | 默认报销人姓名 |
|
||||
| `default_card_no` | 默认公务卡号 |
|
||||
| `default_person_id` | 默认人员编号 |
|
||||
| `consumable_storage` | 出库单存放地点 |
|
||||
| `llm.model` | LLM 模型名称 |
|
||||
| `llm.api_base` | LLM API 地址 |
|
||||
| `llm.api_key` | LLM API 密钥 |
|
||||
|
||||
---
|
||||
|
||||
## 十、日志与调试
|
||||
|
||||
### 日志输出
|
||||
|
||||
- 控制台实时输出(INFO 级别)
|
||||
- 文件日志:`logs/<日期>.log`(UTF-8 编码)
|
||||
- Web 界面:实时 SSE 日志流
|
||||
|
||||
### 截图保存
|
||||
|
||||
每个关键步骤自动截图到 `images/` 目录:
|
||||
|
||||
| 截图文件 | 对应步骤 |
|
||||
|----------|----------|
|
||||
| `debug_portal_loaded.png` | 登录成功 |
|
||||
| `debug_step3_project_modal.png` | 项目弹窗打开 |
|
||||
| `debug_step3_project_selected.png` | 项目选择完成 |
|
||||
| `debug_step3_done.png` | 基本信息完成 |
|
||||
| `debug_after_add_click.png` | 点击新增后 |
|
||||
| `debug_item_total.png` | 总明细录入完成 |
|
||||
| `debug_step5_done.png` | 支付方式完成 |
|
||||
| `debug_step6_done.png` | 附件上传完成 |
|
||||
| `debug_error.png` | 异常状态 |
|
||||
|
||||
### 超时设置
|
||||
|
||||
- 页面默认超时:30 秒
|
||||
- 登录门户等待:最多 30 秒
|
||||
- 单点登录页等待:最多 15 秒
|
||||
- SSE 日志流超时:10 分钟(600 秒)
|
||||
|
||||
---
|
||||
|
||||
## 十一、常见问题
|
||||
|
||||
| 问题 | 原因 | 解决方案 |
|
||||
|------|------|----------|
|
||||
| 登录超时 | SSO 需要手动验证码/微信扫码 | 手动完成验证后脚本继续 |
|
||||
| 未找到财务系统入口 | 门户页面结构变化 | 检查 `images/debug_*` 截图定位 |
|
||||
| 经济科目选择失败 | 弹窗加载延迟 | 检查超时设置,增加等待时间 |
|
||||
| 附件上传失败 | PDF 文件不存在或路径错误 | 确认 PDF 在当前工作目录 |
|
||||
| 金额不匹配 | 明细合计 ≠ 支付合计 | 检查 CSV 数据中刷卡金额 |
|
||||
| 提交被拦截 | 必填项为空 | 检查 `logs/<日期>.log` 定位失败步骤 |
|
||||
| LLM 多模态提取失败 | llama-index 版本不兼容 | 确保使用 llama-index-core >= 0.14.x,多模态消息使用 `blocks` 格式 |
|
||||
| 出库单生成失败 | 缺少 pywin32 或模板文件 | 安装 `pywin32`,确保项目根目录有 `易耗品、出库单.doc` 模板 |
|
||||
| 差旅发票生成了出库单 | 分类不准确 | 检查发票内容是否包含"高铁票""酒店住宿"等关键词 |
|
||||
|
||||
---
|
||||
|
||||
## 十二、相关 CLI 命令
|
||||
|
||||
### 单独填写出库单
|
||||
|
||||
```bash
|
||||
uv run python -m src.infra.documents.consumable --csv invoice_summary.csv --doc "易耗品、出库单.doc"
|
||||
```
|
||||
|
||||
### 分步执行管道
|
||||
|
||||
```bash
|
||||
# 仅发票提取
|
||||
uv run python src/main.py --step invoice
|
||||
|
||||
# 仅浏览器填报
|
||||
uv run python src/main.py --step submit
|
||||
```
|
||||
|
||||
### 启动 Web 服务
|
||||
|
||||
```bash
|
||||
uv run python src/web/app.py
|
||||
# 访问: http://localhost:5000
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 附录:API 文档
|
||||
|
||||
详细的 API 接口文档见 [API.md](./API.md)。
|
||||
@@ -1,5 +0,0 @@
|
||||
序号,发票号码,开票日期,项目名称,规格型号,价税合计,销售方名称,人员姓名,刷卡日期,公务卡号,刷卡金额,备注,工号
|
||||
1,26442000005432755951,2026/05/18,*电子元件*电容器 电容一批 个 500000 0.0057425742574 2871.29 1% 28.71,电容器 电容一批 个 500000 0.0057425742574 2871.29 1% 28.71,2900.00,佛山市泓宇芯科技有限公司,陈陈,2026/04/28,,2850.00,,
|
||||
2,26442000005432652421,2026/05/18,*电子元件*电阻 电阻一批 个 500000 0.0057425742574 2871.29 1% 28.71,电阻 电阻一批 个 500000 0.0057425742574 2871.29 1% 28.71,2900.00,佛山市泓宇芯科技有限公司,陈陈,2026/04/28,,2880.00,,
|
||||
3,26442000005432937661,2026/05/18,*集成电路*集成电路 LED一批 个 2638.92 1% 26.39,集成电路 LED一批 个 2638.92 1% 26.39,2665.31,佛山市泓宇芯科技有限公司,陈陈,2026/04/28,,2661.61,,
|
||||
4,26442000005468940571,2026/05/18,*电子工业设备*元件盒 1# 个 1 47.5247524752475 47.52 1% 0.48,元件盒 1# 个 1 47.5247524752475 47.52 1% 0.48,96.00,东莞市长安顺淘电子工具经营部,陈陈,2026/05/07,,96.00,,
|
||||
|
@@ -1,11 +0,0 @@
|
||||
# 发票信息汇总表
|
||||
|
||||
| 序号 | 发票号码 | 开票日期 | 项目名称 | 规格型号 | 价税合计 | 销售方名称 | 人员姓名 | 刷卡日期 | 公务卡号 | 刷卡金额 | 备注 | 工号 |
|
||||
|------|------|------|------|------|------|------|------|------|------|------|------|------|
|
||||
| 1 | 26442000005432755951 | 2026/05/18 | *电子元件*电容器 电容一批 个 500000 0.0057425742574 2871.29 1% 28.71 | 电容器 电容一批 个 500000 0.0057425742574 2871.29 1% 28.71 | ¥2,900.00 | 佛山市泓宇芯科技有限公司 | 陈陈 | 2026/04/28 | | ¥2,850.00 | | |
|
||||
| 2 | 26442000005432652421 | 2026/05/18 | *电子元件*电阻 电阻一批 个 500000 0.0057425742574 2871.29 1% 28.71 | 电阻 电阻一批 个 500000 0.0057425742574 2871.29 1% 28.71 | ¥2,900.00 | 佛山市泓宇芯科技有限公司 | 陈陈 | 2026/04/28 | | ¥2,880.00 | | |
|
||||
| 3 | 26442000005432937661 | 2026/05/18 | *集成电路*集成电路 LED一批 个 2638.92 1% 26.39 | 集成电路 LED一批 个 2638.92 1% 26.39 | ¥2,665.31 | 佛山市泓宇芯科技有限公司 | 陈陈 | 2026/04/28 | | ¥2,661.61 | | |
|
||||
| 4 | 26442000005468940571 | 2026/05/18 | *电子工业设备*元件盒 1# 个 1 47.5247524752475 47.52 1% 0.48 | 元件盒 1# 个 1 47.5247524752475 47.52 1% 0.48 | ¥96.00 | 东莞市长安顺淘电子工具经营部 | 陈陈 | 2026/05/07 | | ¥96.00 | | |
|
||||
|
||||
**价税合计总计: ¥8,561.31**
|
||||
**刷卡金额总计: ¥8,487.61**
|
||||
46
pyproject.toml
Normal file
46
pyproject.toml
Normal file
@@ -0,0 +1,46 @@
|
||||
[project]
|
||||
name = "auto-reimbursement-system"
|
||||
version = "0.1.0"
|
||||
description = "财务报销自动化系统"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
"flask>=3.0",
|
||||
"playwright>=1.40",
|
||||
"PyMuPDF>=1.24",
|
||||
"pywin32>=306",
|
||||
"llama-index>=0.12.0",
|
||||
"llama-index-llms-openai-like>=0.7.2",
|
||||
"python-dotenv>=1.0",
|
||||
]
|
||||
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
"pytest>=8.0",
|
||||
"pytest-cov>=5.0",
|
||||
"ruff>=0.9",
|
||||
"mypy>=1.14",
|
||||
"deptry>=0.22",
|
||||
"pre-commit>=4.0",
|
||||
]
|
||||
|
||||
[tool.ruff]
|
||||
target-version = "py312"
|
||||
line-length = 120
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "W", "I", "N", "UP", "B"]
|
||||
ignore = ["E501"]
|
||||
|
||||
[tool.mypy]
|
||||
python_version = "3.12"
|
||||
strict = true
|
||||
warn_return_any = true
|
||||
warn_unused_configs = true
|
||||
ignore_missing_imports = true
|
||||
|
||||
[tool.deptry]
|
||||
ignore_notebooks = true
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
pythonpath = ["."]
|
||||
84
run_all.py
84
run_all.py
@@ -1,84 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
财务报销全流程编排脚本
|
||||
|
||||
依次执行:
|
||||
1. extract_invoice.py — 从 PDF 发票提取信息,生成 invoice_summary.csv
|
||||
2. extract_image_ocr.py — 从支付截图 OCR 识别刷卡信息,更新 CSV
|
||||
3. reimburse.py — 打开浏览器登录财务系统并自动填报
|
||||
|
||||
用法:
|
||||
python run_all.py # 默认执行全部三步
|
||||
python run_all.py --step invoice # 仅执行第 1 步
|
||||
python run_all.py --step ocr # 仅执行第 2 步
|
||||
python run_all.py --step submit # 仅执行第 3 步
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
PROJECT_DIR = Path(__file__).parent.resolve()
|
||||
PYTHON = sys.executable
|
||||
|
||||
|
||||
def step(name: str, module: str, args: list[str]) -> bool:
|
||||
"""运行单个步骤,返回是否成功"""
|
||||
cmd = [PYTHON, str(PROJECT_DIR / module), *args]
|
||||
print(f"\n{'=' * 60}")
|
||||
print(f" [{name}] {module}")
|
||||
print(f"{'=' * 60}")
|
||||
|
||||
result = subprocess.run(cmd, cwd=str(PROJECT_DIR))
|
||||
ok = result.returncode == 0
|
||||
|
||||
if ok:
|
||||
print(f"\n[{name}] 完成")
|
||||
else:
|
||||
print(f"\n[错误] {name} 失败 (exit code: {result.returncode})")
|
||||
|
||||
return ok
|
||||
|
||||
|
||||
def run_all() -> int:
|
||||
print("=" * 60)
|
||||
print(" 财务报销自动化流程")
|
||||
print(f" 工作目录: {PROJECT_DIR}")
|
||||
print(f" Python: {PYTHON}")
|
||||
print("=" * 60)
|
||||
|
||||
if not step("发票提取", "extract_invoice.py", []):
|
||||
return 1
|
||||
|
||||
if not step("OCR 识别", "extract_image_ocr.py", []):
|
||||
return 1
|
||||
|
||||
if not step("报销提交", "reimburse.py", ["--data", str(PROJECT_DIR / "invoice_summary.csv")]):
|
||||
return 1
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print(" 全流程执行完毕")
|
||||
print("=" * 60)
|
||||
return 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if len(sys.argv) >= 3 and sys.argv[1] == "--step":
|
||||
name = sys.argv[2]
|
||||
steps = {
|
||||
"invoice": ("发票提取", "extract_invoice.py", []),
|
||||
"ocr": ("OCR 识别", "extract_image_ocr.py", []),
|
||||
"submit": ("报销提交", "reimburse.py", ["--data", str(PROJECT_DIR / "invoice_summary.csv")]),
|
||||
}
|
||||
if name not in steps:
|
||||
print(f"未知步骤: {name},可选: {', '.join(steps)}")
|
||||
return 1
|
||||
label, module, args = steps[name]
|
||||
return 0 if step(label, module, args) else 1
|
||||
|
||||
return run_all()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
20
scripts/README.md
Normal file
20
scripts/README.md
Normal file
@@ -0,0 +1,20 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# scripts — 调试脚本与数据目录
|
||||
|
||||
## 子目录
|
||||
|
||||
| 目录 | 说明 |
|
||||
|------|------|
|
||||
| `data/` | CLI 模式的数据目录:发票源文件、`config.json`、`.invoice_cache` 缓存 |
|
||||
|
||||
## 脚本
|
||||
|
||||
| 文件 | 说明 |
|
||||
|------|------|
|
||||
| `debug_stream_fields.py` | 诊断 stream_chat 返回对象的字段结构 |
|
||||
| `test_application_extract.py` | 测试出差申请单提取 |
|
||||
| `test_multimodal.py` | 测试多模态 LLM 识别 |
|
||||
| `test_travel_info.py` | 测试差旅信息提取 |
|
||||
69
scripts/debug_stream_fields.py
Normal file
69
scripts/debug_stream_fields.py
Normal file
@@ -0,0 +1,69 @@
|
||||
"""诊断脚本:检查 stream_chat 返回对象的字段结构"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
|
||||
# 加载 .env
|
||||
from dotenv import load_dotenv # noqa: E402
|
||||
from llama_index.core.llms import ChatMessage # noqa: E402
|
||||
|
||||
from src.config import get_llm_config # noqa: E402
|
||||
from src.core.extraction import _create_llm # noqa: E402
|
||||
|
||||
load_dotenv(Path(__file__).parent / ".env")
|
||||
|
||||
llm_config = get_llm_config()
|
||||
print(f"LLM config: model={llm_config['model']}, api_base={llm_config['api_base']}")
|
||||
|
||||
llm = _create_llm()
|
||||
messages = [
|
||||
ChatMessage(role="system", content="你是一个助手"),
|
||||
ChatMessage(role="user", content="1+1 等于几?"),
|
||||
]
|
||||
|
||||
print("\n=== 检查 stream_chat 返回对象的字段 ===")
|
||||
count = 0
|
||||
try:
|
||||
for resp in llm.stream_chat(messages, temperature=0.1):
|
||||
count += 1
|
||||
if count <= 3:
|
||||
print(f"\n--- chunk #{count} ---")
|
||||
print(f" type: {type(resp).__name__}")
|
||||
print(f" delta: {repr(resp.delta)[:200]}")
|
||||
if hasattr(resp, "additional_kwargs") and resp.additional_kwargs:
|
||||
print(f" additional_kwargs keys: {list(resp.additional_kwargs.keys())}")
|
||||
for k, v in resp.additional_kwargs.items():
|
||||
val_preview = str(v)[:200]
|
||||
print(f" additional_kwargs['{k}']: {val_preview}")
|
||||
if hasattr(resp, "raw"):
|
||||
raw = resp.raw
|
||||
if isinstance(raw, dict):
|
||||
print(f" raw keys: {list(raw.keys())}")
|
||||
for k, v in raw.items():
|
||||
val_preview = str(v)[:200]
|
||||
print(f" raw['{k}']: {val_preview}")
|
||||
else:
|
||||
print(f" raw type: {type(raw)}")
|
||||
if hasattr(resp, "message") and resp.message:
|
||||
msg = resp.message
|
||||
print(f" message type: {type(msg).__name__}")
|
||||
if hasattr(msg, "additional_kwargs") and msg.additional_kwargs:
|
||||
print(f" message.additional_kwargs keys: {list(msg.additional_kwargs.keys())}")
|
||||
for k, v in msg.additional_kwargs.items():
|
||||
val_preview = str(v)[:200]
|
||||
print(f" message.additional_kwargs['{k}']: {val_preview}")
|
||||
if hasattr(msg, "reasoning_content"):
|
||||
rc = msg.reasoning_content
|
||||
print(f" message.reasoning_content: {repr(rc)[:200]}")
|
||||
elif count == 4:
|
||||
print("\n... (更多 chunk 省略)")
|
||||
if count >= 10:
|
||||
break
|
||||
print(f"\n=== 共收到 {count} 个 chunk ===")
|
||||
except Exception as e:
|
||||
print(f"\n错误: {e}")
|
||||
import traceback
|
||||
|
||||
traceback.print_exc()
|
||||
208
scripts/test_application_extract.py
Normal file
208
scripts/test_application_extract.py
Normal file
@@ -0,0 +1,208 @@
|
||||
#!/usr/bin/env python3
|
||||
"""单独测试事前申请单的信息提取功能
|
||||
|
||||
用于调试 LLM 对事前申请单的提取准确率。
|
||||
|
||||
用法:
|
||||
# 测试项目根目录的事前申请单.pdf
|
||||
python scripts/test_application_extract.py
|
||||
|
||||
# 测试指定文件
|
||||
python scripts/test_application_extract.py --file path/to/file.pdf
|
||||
|
||||
# 测试 scripts/data 目录下的事前申请单
|
||||
python scripts/test_application_extract.py --dir scripts/data
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import io
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# 加载 .env 环境变量(在导入 src 模块之前)
|
||||
from dotenv import load_dotenv
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
load_dotenv(ROOT / ".env")
|
||||
|
||||
# Windows 终端强制 UTF-8
|
||||
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8", errors="replace")
|
||||
|
||||
sys.path.insert(0, str(ROOT)) # noqa: E402
|
||||
|
||||
from src.core.extraction import extract_document # noqa: E402
|
||||
|
||||
|
||||
def test_single_file(file_path: Path) -> None:
|
||||
"""测试单个文件的提取效果"""
|
||||
print("=" * 60)
|
||||
print(f"测试文件: {file_path.name}")
|
||||
print("=" * 60)
|
||||
|
||||
if not file_path.exists():
|
||||
print(f"文件不存在: {file_path}")
|
||||
return
|
||||
|
||||
try:
|
||||
# 调用统一提取接口
|
||||
result = extract_document(file_path)
|
||||
|
||||
if not result:
|
||||
print("[ERROR] 提取返回空结果")
|
||||
return
|
||||
|
||||
# 检查类型判断
|
||||
inv_type = result.get("invoice_type", "")
|
||||
print(f"\n[类型判断] invoice_type = {inv_type}")
|
||||
|
||||
if inv_type != "application":
|
||||
print(f"[WARNING] 类型判断错误!期望 'application',实际得到 '{inv_type}'")
|
||||
else:
|
||||
print("[OK] 类型判断正确")
|
||||
|
||||
# 展示提取结果
|
||||
print("\n[提取结果]")
|
||||
print(json.dumps(result, ensure_ascii=False, indent=2))
|
||||
|
||||
# 字段完整性检查
|
||||
print("\n[字段检查]")
|
||||
expected_fields = {
|
||||
"invoice_type": "类型标识",
|
||||
"project_name": "项目名称",
|
||||
"purpose": "出差事由",
|
||||
"start_date": "开始日期",
|
||||
"end_date": "结束日期",
|
||||
"person_info": "人员信息",
|
||||
}
|
||||
|
||||
for field, desc in expected_fields.items():
|
||||
value = result.get(field)
|
||||
if value is None:
|
||||
print(f" [MISSING] {desc} ({field}) - 字段缺失")
|
||||
elif value == "" or value == []:
|
||||
print(f" [EMPTY] {desc} ({field}) - 字段为空")
|
||||
else:
|
||||
print(f" [OK] {desc} ({field})")
|
||||
|
||||
# 日期格式检查
|
||||
for date_field in ["start_date", "end_date"]:
|
||||
date_value = result.get(date_field, "")
|
||||
if date_value and len(date_value) == 10:
|
||||
try:
|
||||
parts = date_value.split("-")
|
||||
if len(parts) == 3:
|
||||
int(parts[0]) # year
|
||||
int(parts[1]) # month
|
||||
int(parts[2]) # day
|
||||
print(f" [OK] {date_field} 格式正确 (YYYY-MM-DD)")
|
||||
except (ValueError, IndexError):
|
||||
print(f" [ERROR] {date_field} 格式错误: {date_value}")
|
||||
elif date_value:
|
||||
print(f" [ERROR] {date_field} 格式错误: {date_value}")
|
||||
|
||||
# 人员信息结构检查
|
||||
person_info = result.get("person_info")
|
||||
if person_info:
|
||||
if isinstance(person_info, list):
|
||||
print(f"\n[人员信息] 共 {len(person_info)} 人")
|
||||
for i, person in enumerate(person_info, 1):
|
||||
pid = person.get("person_id", "")
|
||||
pname = person.get("person_name", "")
|
||||
print(f" 人员 {i}: {pname} ({pid})")
|
||||
elif isinstance(person_info, dict):
|
||||
print("\n[人员信息] 单人格式")
|
||||
print(f" 姓名: {person_info.get('person_name', '')}")
|
||||
print(f" 编号: {person_info.get('person_id', '')}")
|
||||
|
||||
except Exception as e:
|
||||
print(f"\n[EXCEPTION] 提取失败: {e}")
|
||||
import traceback
|
||||
|
||||
traceback.print_exc()
|
||||
|
||||
|
||||
def test_cache_comparison(file_path: Path) -> None:
|
||||
"""对比原始提取和缓存结果"""
|
||||
cache_dir = file_path.parent / ".invoice_cache"
|
||||
cache_file = cache_dir / f"{file_path.stem}{file_path.suffix}.json"
|
||||
|
||||
if not cache_file.exists():
|
||||
print(f"\n[INFO] 无缓存文件对比: {cache_file}")
|
||||
return
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("[缓存对比]")
|
||||
print("=" * 60)
|
||||
|
||||
try:
|
||||
with open(cache_file, encoding="utf-8") as f:
|
||||
cache_data = json.load(f)
|
||||
|
||||
cached_result = cache_data.get("extracted_data", {})
|
||||
print("\n[缓存数据]")
|
||||
print(json.dumps(cached_result, ensure_ascii=False, indent=2))
|
||||
|
||||
# 对比关键字段
|
||||
print("\n[字段对比]")
|
||||
for key in ["invoice_type", "project_name", "purpose", "start_date", "end_date"]:
|
||||
cached_value = cached_result.get(key, "<缺失>")
|
||||
print(f" {key}: {cached_value}")
|
||||
|
||||
except Exception as e:
|
||||
print(f"[ERROR] 读取缓存失败: {e}")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="测试事前申请单信息提取")
|
||||
parser.add_argument(
|
||||
"--file",
|
||||
type=str,
|
||||
default=None,
|
||||
help="要测试的文件路径 (默认: 扫描 scripts/data 目录)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--dir",
|
||||
type=str,
|
||||
default=str(ROOT / "scripts" / "data"),
|
||||
help="扫描目录下所有事前申请单文件 (默认: scripts/data)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--no-cache-compare",
|
||||
action="store_true",
|
||||
help="不执行缓存对比",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.dir:
|
||||
# 目录模式:扫描所有PDF文件
|
||||
dir_path = Path(args.dir)
|
||||
if not dir_path.exists():
|
||||
print(f"目录不存在: {dir_path}")
|
||||
sys.exit(1)
|
||||
|
||||
pdf_files = sorted(dir_path.glob("*.pdf"))
|
||||
if not pdf_files:
|
||||
print(f"目录下没有找到PDF文件: {dir_path}")
|
||||
sys.exit(1)
|
||||
|
||||
print(f"发现 {len(pdf_files)} 个PDF文件,开始逐个测试...\n")
|
||||
for pdf in pdf_files:
|
||||
if "事前申请" in pdf.name or "申请单" in pdf.name:
|
||||
test_single_file(pdf)
|
||||
if not args.no_cache_compare:
|
||||
test_cache_comparison(pdf)
|
||||
print()
|
||||
else:
|
||||
# 单文件模式
|
||||
file_path = Path(args.file)
|
||||
test_single_file(file_path)
|
||||
if not args.no_cache_compare:
|
||||
test_cache_comparison(file_path)
|
||||
|
||||
print("\n测试完成!")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
81
scripts/test_multimodal.py
Normal file
81
scripts/test_multimodal.py
Normal file
@@ -0,0 +1,81 @@
|
||||
#!/usr/bin/env python3
|
||||
"""测试 PDF 多模态提取完整链路
|
||||
|
||||
用法:
|
||||
python scripts/test_multimodal.py
|
||||
"""
|
||||
|
||||
import io
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Windows 终端强制 UTF-8
|
||||
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8", errors="replace")
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT)) # noqa: E402
|
||||
|
||||
from src.core.extraction import extract_document # noqa: E402
|
||||
from src.infra.documents.pdf import render_pdf_to_images # noqa: E402
|
||||
|
||||
|
||||
def test_render() -> None:
|
||||
"""测试 PDF 渲染"""
|
||||
pdf_path = ROOT / "事前申请单.pdf"
|
||||
if not pdf_path.exists():
|
||||
print(f"跳过: {pdf_path.name} 不存在")
|
||||
return
|
||||
|
||||
print("=" * 60)
|
||||
print("测试 PDF 渲染")
|
||||
print("=" * 60)
|
||||
try:
|
||||
images = render_pdf_to_images(pdf_path, dpi=150)
|
||||
if images:
|
||||
print(f"成功渲染 {len(images)} 页")
|
||||
for i, img_b64 in enumerate(images):
|
||||
print(f" 第 {i + 1} 页: base64 长度 {len(img_b64)} 字符")
|
||||
else:
|
||||
print("渲染返回空列表!")
|
||||
except ImportError as e:
|
||||
print(f"导入失败: {e}")
|
||||
except Exception as e:
|
||||
print(f"渲染异常: {e}")
|
||||
|
||||
|
||||
def test_multimodal_extract() -> None:
|
||||
"""测试完整的多模态提取链路"""
|
||||
pdf_path = ROOT / "事前申请单.pdf"
|
||||
if not pdf_path.exists():
|
||||
print(f"跳过: {pdf_path.name} 不存在")
|
||||
return
|
||||
|
||||
print()
|
||||
print("=" * 60)
|
||||
print("测试多模态提取")
|
||||
print("=" * 60)
|
||||
try:
|
||||
result = extract_document(pdf_path)
|
||||
if result:
|
||||
print("提取成功:")
|
||||
import json
|
||||
|
||||
print(json.dumps(result, ensure_ascii=False, indent=2))
|
||||
else:
|
||||
print("提取返回空字典!")
|
||||
except Exception as e:
|
||||
print(f"提取异常: {e}")
|
||||
import traceback
|
||||
|
||||
traceback.print_exc()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
test_render()
|
||||
test_multimodal_extract()
|
||||
print()
|
||||
print("测试完成!")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
54
scripts/test_travel_info.py
Normal file
54
scripts/test_travel_info.py
Normal file
@@ -0,0 +1,54 @@
|
||||
#!/usr/bin/env python3
|
||||
"""直接测试 extract_travel_info 函数
|
||||
|
||||
所有数据从 .invoice_cache 缓存中自动加载,无需手动构造样本数据。
|
||||
|
||||
用法:
|
||||
python scripts/test_travel_info.py
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# 项目根目录
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT)) # noqa: E402
|
||||
|
||||
from src.core.extraction import extract_travel_info # noqa: E402
|
||||
|
||||
|
||||
def main() -> None:
|
||||
source_dir = ROOT / "scripts" / "data"
|
||||
|
||||
if not source_dir.exists():
|
||||
print(f"源文件目录不存在: {source_dir}")
|
||||
sys.exit(1)
|
||||
|
||||
print("=" * 60)
|
||||
print("测试 extract_travel_info(数据来自缓存)")
|
||||
print("=" * 60)
|
||||
print(f"源文件目录: {source_dir}")
|
||||
print()
|
||||
|
||||
try:
|
||||
result = extract_travel_info(
|
||||
source_dir=source_dir,
|
||||
)
|
||||
print()
|
||||
print("=" * 60)
|
||||
print("提取结果:")
|
||||
print("=" * 60)
|
||||
print(json.dumps(result, ensure_ascii=False, indent=2))
|
||||
print()
|
||||
print("测试通过!")
|
||||
except Exception as e:
|
||||
print(f"测试失败: {e}")
|
||||
import traceback
|
||||
|
||||
traceback.print_exc()
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
97
src/README.md
Normal file
97
src/README.md
Normal file
@@ -0,0 +1,97 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src — 主源码目录
|
||||
|
||||
包含财务报销自动化系统的全部源码模块。
|
||||
|
||||
## 架构分层
|
||||
|
||||
```
|
||||
src/
|
||||
├── agent/ Agent 调度层(协调提取-校验-修正循环)
|
||||
├── core/ 核心业务层(提取、匹配、校验)
|
||||
├── infra/ 基础设施层(浏览器、文档、LLM 提示词)
|
||||
├── web/ Web 界面层(Flask + SSE)
|
||||
├── pipeline.py CLI 流程编排
|
||||
├── pipeline_core.py CLI/Web 公共管道逻辑
|
||||
├── main.py CLI 入口
|
||||
├── config.py 配置加载
|
||||
└── exceptions.py 异常定义
|
||||
```
|
||||
|
||||
## 模块清单
|
||||
|
||||
| 文件/目录 | 说明 |
|
||||
|-----------|------|
|
||||
| `agent/` | Agent 调度:校验-修正循环、状态机管理、SSE 事件发射 |
|
||||
| `core/` | 核心业务逻辑:信息提取、金额匹配、信息校验 |
|
||||
| `infra/` | 基础设施:浏览器自动填报、文档处理、LLM 提示词管理 |
|
||||
| `web/` | Web 界面:Flask 应用、SSE 日志流、可编辑表格、移动端上传、会话隔离 |
|
||||
| `pipeline.py` | CLI 流程编排:串联提取 → 类型判断 → 信息提取 → 浏览器填报 |
|
||||
| `pipeline_core.py` | CLI/Web 公共管道逻辑:发票类型判断、缓存提取 |
|
||||
| `main.py` | CLI 入口:`--step` 分步执行、`-u/-p` 覆盖凭据 |
|
||||
| `config.py` | 配置加载:`config.json` + 环境变量 |
|
||||
| `exceptions.py` | 异常层次定义 |
|
||||
|
||||
## 数据流
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[CLI/Web 入口] --> B["pipeline.py (编排)"]
|
||||
B --> C["core/extraction/extractor.py (统一提取入口)"]
|
||||
C --> D["infra/documents/pdf.py (PDF 渲染为图片)"]
|
||||
C --> E["core/extraction/llm_extractor.py (多模态 LLM 识别)"]
|
||||
E --> F["发票 invoice_type=train/hotel/general"]
|
||||
E --> G["支付记录 invoice_type=payment"]
|
||||
E --> H["出差事前申请单 invoice_type=application"]
|
||||
C --> I["core/matching/matcher.py (发票与支付记录按金额匹配)"]
|
||||
I --> J["一对一匹配 发票数 == 刷卡数"]
|
||||
I --> K["一对多匹配 贪心算法 相对容差 3%"]
|
||||
C --> L["infra/documents/invoice.py (CSV/JSON 读写)"]
|
||||
L --> M["payment_records.csv (支付记录级别)"]
|
||||
L --> N["invoice_summary.csv (发票级别)"]
|
||||
L --> O["travel_applications.json (出差申请单)"]
|
||||
B --> R{"判断报销类型"}
|
||||
R -->|差旅| T["core/extraction/llm_extractor.py (差旅信息提取)"]
|
||||
R -->|普通| V["core/extraction/llm_extractor.py (普通发票信息提取)"]
|
||||
T --> W["travel_info.json (差旅信息: 交通/住宿明细、补贴、附件清单)"]
|
||||
V --> X["normal_info.json (普通发票信息: 报销说明、发票总数、总金额、支付方式、附件清单)"]
|
||||
W --> P["infra/browser/ (浏览器填报 - 仅接收信息并填报)"]
|
||||
X --> P
|
||||
P --> Q["差旅模式: travel_info.json → 填报差旅单 → 上传差旅附件"]
|
||||
P --> S["普通模式: 基本信息 → 录入明细 → 支付信息 → 上传附件"]
|
||||
```
|
||||
|
||||
## 子模块文档
|
||||
|
||||
| 目录 | 文档 |
|
||||
|------|------|
|
||||
| `agent/` | [`agent/README.md`](agent/README.md) |
|
||||
| `core/` | [`core/README.md`](core/README.md) |
|
||||
| `infra/` | [`infra/README.md`](infra/README.md) |
|
||||
| `web/` | [`web/README.md`](web/README.md) |
|
||||
|
||||
## 启动方式
|
||||
|
||||
```bash
|
||||
# CLI 模式
|
||||
uv run python src/main.py --step all
|
||||
|
||||
# Web 模式
|
||||
uv run python src/web/app.py
|
||||
# 访问: http://localhost:5000
|
||||
```
|
||||
|
||||
## 发票类型与路由
|
||||
|
||||
系统根据 `invoice_type` 字段自动分流:
|
||||
|
||||
| 发票类型 | 走什么流程 | 是否生成出库单 |
|
||||
|----------|-----------|--------------|
|
||||
| `train` / `hotel` | 差旅报销 | 否 |
|
||||
| `general` | 普通报销 | 是(易耗品出库单) |
|
||||
| `application` | 出差事前申请单 | 否(单独存储为 JSON) |
|
||||
|
||||
**注意**:差旅发票和普通发票不支持混报,混合时会报错。
|
||||
58
src/__init__.py
Normal file
58
src/__init__.py
Normal file
@@ -0,0 +1,58 @@
|
||||
"""财务报销自动化工具包"""
|
||||
|
||||
import datetime
|
||||
import io
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_LOG_FMT = "%(asctime)s [%(levelname)-5s] %(name)s: %(message)s"
|
||||
_LOG_DATE_FMT = "%Y-%m-%d %H:%M:%S"
|
||||
_LOG_DIR = Path(__file__).resolve().parent.parent / "logs"
|
||||
|
||||
|
||||
def _get_log_file() -> Path:
|
||||
"""返回当日日志文件路径,如 logs/2026-06-09.log"""
|
||||
_LOG_DIR.mkdir(parents=True, exist_ok=True)
|
||||
return _LOG_DIR / f"{datetime.date.today():%Y-%m-%d}.log"
|
||||
|
||||
|
||||
def get_logger(name: str) -> logging.Logger:
|
||||
"""获取带时间戳的日志记录器
|
||||
|
||||
输出格式: 2026-05-24 12:34:56 [INFO ] extractor: 扫描目录: ...
|
||||
日志同时输出到终端和 logs/<日期>.log
|
||||
|
||||
只在最顶层的 logger 上添加标准 handler(stream + file),
|
||||
子 logger 通过 propagate 将日志传递给父 logger 统一处理。
|
||||
这样 _SSELogHandler 只需挂在父 logger 上即可捕获所有子日志。
|
||||
"""
|
||||
logger = logging.getLogger(name)
|
||||
logger.setLevel(logging.INFO)
|
||||
|
||||
# 检查是否有父 logger 已初始化标准 handler
|
||||
# 如果有,子 logger 不重复添加,靠 propagate 传递即可
|
||||
parent_name = name.rsplit(".", 1)[0] if "." in name else None
|
||||
parent_has_handlers = False
|
||||
if parent_name:
|
||||
parent = logging.getLogger(parent_name)
|
||||
parent_has_handlers = getattr(parent, "_standard_handlers_initialized", False)
|
||||
|
||||
if not getattr(logger, "_standard_handlers_initialized", False) and not parent_has_handlers:
|
||||
formatter = logging.Formatter(_LOG_FMT, _LOG_DATE_FMT)
|
||||
|
||||
# 只有没有已初始化的父 logger 时,才添加标准 handler
|
||||
# 终端输出
|
||||
utf8_stream = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8", errors="replace")
|
||||
stream_handler = logging.StreamHandler(utf8_stream)
|
||||
stream_handler.setFormatter(formatter)
|
||||
logger.addHandler(stream_handler)
|
||||
|
||||
# 文件输出
|
||||
file_handler = logging.FileHandler(str(_get_log_file()), encoding="utf-8")
|
||||
file_handler.setFormatter(formatter)
|
||||
logger.addHandler(file_handler)
|
||||
|
||||
logger._standard_handlers_initialized = True # type: ignore[attr-defined]
|
||||
|
||||
return logger
|
||||
242
src/agent/README.md
Normal file
242
src/agent/README.md
Normal file
@@ -0,0 +1,242 @@
|
||||
---
|
||||
|
||||
## last_reviewed: 2026-06-13
|
||||
|
||||
# src/agent — Agent 协调模块
|
||||
|
||||
作为调度中枢,编排信息提取、规则校验和语义校验的完整流程,驱动用户完成材料补充直到信息完整可提交。
|
||||
|
||||
## 模块清单
|
||||
|
||||
|
||||
| 文件 | 作用 |
|
||||
| ----------------- | ------------------------------------------ |
|
||||
| `orchestrator.py` | 调度中枢:状态机管理、轮次调度、校验-修正循环编排、语义校验、用户补充处理、事件发射 |
|
||||
|
||||
|
||||
## 架构概览
|
||||
|
||||
Agent 是报销系统的调度中枢,负责编排各模块完成信息提取和校验。它的核心职责:
|
||||
|
||||
1. **调度 LLM 提取** — 调用 `doc.llm_extractor` 的纯提取接口
|
||||
2. **调度规则校验** — 调用 `doc.validator` 按报销规范检查字段完整性
|
||||
3. **编排校验-修正循环** — 校验失败时构建修正提示,再次调度 LLM 修正
|
||||
4. **调度语义校验** — 调用 LLM 判断提取信息在语义层面是否自洽、充分
|
||||
5. **状态持久化** — 每轮结束后将会话状态写入磁盘,支持中断恢复
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph agent["src/agent (调度中枢)"]
|
||||
O[orchestrator.py]
|
||||
end
|
||||
|
||||
subgraph doc["src/doc"]
|
||||
LE[llm_extractor.py]
|
||||
VA[validator.py]
|
||||
end
|
||||
|
||||
O -->|"1. 调度 LLM 提取"| LE
|
||||
O -->|"2. 调度规则校验"| VA
|
||||
VA -->|"校验失败"| O
|
||||
O -->|"3. 构建修正提示"| LE
|
||||
O -->|"4. 调度语义校验"| LE
|
||||
LE -->|"缓存读写"| C[.invoice_cache/]
|
||||
O -->|"SSE 事件"| E[agent_events.log]
|
||||
O -->|"状态持久化"| S[agent_state.json]
|
||||
```
|
||||
|
||||
|
||||
|
||||
### 校验-修正循环
|
||||
|
||||
Agent 编排的校验-修正循环流程:
|
||||
|
||||
1. Agent 调用 `llm_extractor.llm_query_text()` 让 LLM 提取信息
|
||||
2. Agent 调用 `validator.validate_extracted_info()` 规则校验
|
||||
3. 校验失败则 Agent 构建修正提示(包含缺失字段列表),再次调用 LLM
|
||||
4. 最多重试 3 次(`MAX_VALIDATION_RETRIES`),3 次后返回最佳结果
|
||||
|
||||
**关键点**:校验逻辑不内嵌在 `llm_extractor` 中,而是由 Agent 层调度。`llm_extractor` 只提供纯提取能力,`validator` 只提供纯校验能力,Agent 负责编排。
|
||||
|
||||
## 状态机
|
||||
|
||||
Agent 会话通过 `AgentState` 枚举管理生命周期:
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> IDLE
|
||||
IDLE --> EXTRACTING
|
||||
EXTRACTING --> AWAITING_SUPPLEMENT
|
||||
EXTRACTING --> READY
|
||||
AWAITING_SUPPLEMENT --> EXTRACTING: 用户补充后重新校验
|
||||
READY --> [*]
|
||||
EXTRACTING --> ERROR: 提取失败
|
||||
AWAITING_SUPPLEMENT --> ERROR: 超出最大轮次
|
||||
READY --> SUBMITTING: 用户提交
|
||||
SUBMITTING --> DONE
|
||||
DONE --> [*]
|
||||
AWAITING_SUPPLEMENT --> READY: 用户强制提交
|
||||
```
|
||||
|
||||
|
||||
|
||||
### 状态说明
|
||||
|
||||
|
||||
| 状态 | 含义 | 触发条件 |
|
||||
| --------------------- | ------------------------ | ------------------------ |
|
||||
| `IDLE` | 初始状态,等待启动 | 会话创建时 |
|
||||
| `EXTRACTING` | 正在执行提取-校验-修正循环 | `run_agent_round()` 开始执行 |
|
||||
| `AWAITING_SUPPLEMENT` | 语义校验未通过,等待用户补充材料或文字说明 | 语义校验失败 |
|
||||
| `READY` | 规则校验 + 语义校验均通过,信息完整,可以提交 | 双重校验通过 |
|
||||
| `SUBMITTING` | 用户确认提交,进入提交流程 | 调用提交接口 |
|
||||
| `DONE` | 提交完成,终态 | 提交成功后 |
|
||||
| `ERROR` | 提取失败或超出最大轮次(默认 5 轮) | 异常或轮次耗尽 |
|
||||
|
||||
|
||||
## 数据流
|
||||
|
||||
### 单轮处理流程
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[开始第 N 轮] --> B{终态保护?}
|
||||
B -->|是| Z[跳过返回]
|
||||
B -->|否| C{轮次超限?}
|
||||
C -->|是| E[转入 ERROR]
|
||||
C -->|否| D[EXTRACTING]
|
||||
|
||||
D --> F{缓存命中?}
|
||||
F -->|是| G[读取缓存数据]
|
||||
F -->|否| H[Agent 调度校验-修正循环]
|
||||
|
||||
H --> I[调用 LLM 提取]
|
||||
I --> J[调用 validator 校验]
|
||||
J --> K{校验通过?}
|
||||
K -->|是| L[写入缓存]
|
||||
K -->|否| M{重试次数<3?}
|
||||
M -->|是| N[构建修正提示]
|
||||
N --> I
|
||||
M -->|否| L
|
||||
|
||||
G --> O[语义校验 validate_semantic_completeness]
|
||||
L --> O
|
||||
|
||||
O --> P{语义通过?}
|
||||
P -->|是| Q[READY]
|
||||
P -->|否| R[发射 agent_request_supplement 事件]
|
||||
R --> S[AWAITING_SUPPLEMENT]
|
||||
Q --> T[发射 agent_ready 事件]
|
||||
|
||||
S --> U[持久化状态]
|
||||
T --> U
|
||||
U --> V[返回 session]
|
||||
E --> U
|
||||
```
|
||||
|
||||
|
||||
|
||||
### 用户补充流程
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[用户补充] --> B{补充方式}
|
||||
B -->|文件上传| C[add_supplement 记录文件名]
|
||||
B -->|文字输入| D[process_user_text_supplement]
|
||||
B -->|强制提交| E[force_submit 跳过校验]
|
||||
|
||||
D --> F[LLM 解析文字提取字段]
|
||||
F --> G{有有效字段?}
|
||||
G -->|是| H[合并到 extracted_info]
|
||||
H --> I[更新缓存]
|
||||
I --> J[触发新一轮 run_agent_round]
|
||||
G -->|否| K[返回无变化提示]
|
||||
E --> L[直接转入 READY]
|
||||
|
||||
C --> M[等待下一轮 run_agent_round]
|
||||
J --> M
|
||||
L --> M
|
||||
K --> M
|
||||
```
|
||||
|
||||
|
||||
|
||||
## 核心数据结构
|
||||
|
||||
### AgentSession
|
||||
|
||||
会话状态的完整载体,包含:
|
||||
|
||||
```python
|
||||
{
|
||||
"session_id": "str", # 会话唯一标识
|
||||
"state": "AgentState", # 当前状态
|
||||
"rounds": 0, # 已执行的轮次数
|
||||
"max_rounds": 5, # 最大轮次限制
|
||||
"invoice_type": "travel", # 发票类型: "travel" | "normal"
|
||||
"extracted_info": {}, # 提取的结构化报销信息
|
||||
"validation_reports": [], # 历次语义校验报告列表
|
||||
"user_supplements": [], # 用户补充的文件名列表
|
||||
"error_message": "" # 错误信息
|
||||
}
|
||||
```
|
||||
|
||||
### 校验报告
|
||||
|
||||
语义校验生成的报告条目追加到 `validation_reports`:
|
||||
|
||||
```python
|
||||
{
|
||||
"round": 1,
|
||||
"type": "semantic",
|
||||
"valid": False,
|
||||
"issues": ["金额不一致"],
|
||||
"missing_info": ["报销说明"],
|
||||
"suggestion": "请确认金额一致性并补充报销说明",
|
||||
"confidence": 0.6
|
||||
}
|
||||
```
|
||||
|
||||
### SSE 事件
|
||||
|
||||
通过 `agent_events.log` 向外部发射实时事件,每行一条 JSON:
|
||||
|
||||
|
||||
| 事件类型 | 触发时机 |
|
||||
| --------------------------- | ------------- |
|
||||
| `agent_state_change` | 状态切换时 |
|
||||
| `agent_ready` | 双重校验通过,信息完整 |
|
||||
| `agent_request_supplement` | 校验未通过,请求用户补充 |
|
||||
| `agent_supplement_received` | 收到用户补充(文件或文字) |
|
||||
| `agent_force_submit` | 用户选择强制提交 |
|
||||
| `agent_error` | 信息提取失败 |
|
||||
| `agent_max_rounds` | 达到最大轮次限制 |
|
||||
|
||||
|
||||
## 持久化机制
|
||||
|
||||
- **状态文件**:`session_dir/agent_state.json` — 原子写入(先写 `.tmp` 再 rename)
|
||||
- **事件日志**:`session_dir/agent_events.log` — 追加写入,支持前端 SSE 轮询
|
||||
- **缓存目录**:`session_dir/.invoice_cache/` — 存放 `travel_info.json` / `normal_info.json`
|
||||
|
||||
## 依赖说明
|
||||
|
||||
|
||||
| 上游依赖 | 用途 |
|
||||
| -------------------------- | ------------------------- |
|
||||
| `src/doc/llm_extractor.py` | 纯 LLM 提取、语义校验、缓存加载、用户补充解析 |
|
||||
| `src/doc/validator.py` | 纯规则校验 |
|
||||
| `src/doc/prompt.py` | 系统提示词加载 |
|
||||
|
||||
|
||||
Agent 是调度中枢,`llm_extractor` 提供纯提取能力,`validator` 提供纯校验能力,Agent 负责编排校验-修正循环。各模块职责清晰,不互相嵌套。
|
||||
|
||||
## 设计原则
|
||||
|
||||
- **Agent 是调度中枢**:校验-修正循环由 Agent 编排,不内嵌在 `llm_extractor` 中
|
||||
- **模块职责单一**:`llm_extractor` 只管提取,`validator` 只管校验,Agent 负责编排
|
||||
- **缓存优先**:信息提取优先读取 `.invoice_cache`,避免重复调用 LLM
|
||||
- **轮次保护**:默认 5 轮上限,防止无限循环;校验-修正循环最多重试 3 次
|
||||
- **终态保护**:`DONE` / `SUBMITTING` / `READY` 状态下不再重复处理
|
||||
- **容错降级**:语义校验失败不阻断流程,仅记录警告日志;规则校验 3 次重试后返回最佳结果
|
||||
|
||||
32
src/agent/__init__.py
Normal file
32
src/agent/__init__.py
Normal file
@@ -0,0 +1,32 @@
|
||||
"""Agent 模块
|
||||
|
||||
提供多轮对话协调、规则校验和语义校验能力。
|
||||
|
||||
入口:
|
||||
- `orchestrator.run_agent_round()` — 执行一轮 Agent 处理
|
||||
- `orchestrator.force_submit()` — 用户强制提交
|
||||
- `orchestrator.add_supplement()` — 记录用户补充的文件
|
||||
- `orchestrator.load_agent_state()` / `save_agent_state()` — 状态持久化
|
||||
"""
|
||||
|
||||
from .orchestrator import (
|
||||
AgentSession,
|
||||
AgentState,
|
||||
add_supplement,
|
||||
force_submit,
|
||||
load_agent_state,
|
||||
process_user_text_supplement,
|
||||
run_agent_round,
|
||||
save_agent_state,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"AgentSession",
|
||||
"AgentState",
|
||||
"add_supplement",
|
||||
"force_submit",
|
||||
"load_agent_state",
|
||||
"process_user_text_supplement",
|
||||
"run_agent_round",
|
||||
"save_agent_state",
|
||||
]
|
||||
410
src/agent/coordinator.py
Normal file
410
src/agent/coordinator.py
Normal file
@@ -0,0 +1,410 @@
|
||||
"""Agent 协调器
|
||||
|
||||
作为调度中枢,编排信息提取、规则校验的完整流程。
|
||||
|
||||
校验-修正循环由 Agent 层调度:
|
||||
1. Agent 调用 LLM 提取信息
|
||||
2. Agent 调用 validator.py 校验
|
||||
3. 校验失败则构建修正提示,再次调用 LLM
|
||||
4. 重复直到校验通过或达到最大重试次数
|
||||
5. LLM 在输出中包含 can_submit 和 suggestion 字段,用于判断信息完整性
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
from ..core.extraction import (
|
||||
build_extraction_user_message,
|
||||
llm_query_text,
|
||||
load_cache,
|
||||
load_match_result,
|
||||
merge_supplement_into_info,
|
||||
parse_json_response,
|
||||
process_user_supplement,
|
||||
)
|
||||
from ..core.validation import validate_extracted_info
|
||||
from ..infra.llm import (
|
||||
build_normal_info_system_prompt,
|
||||
build_travel_info_system_prompt,
|
||||
)
|
||||
from ..pipeline_core import save_cache_info
|
||||
from .events import emit_agent_event
|
||||
from .session import AgentSession, AgentState, save_agent_state
|
||||
|
||||
log = get_logger("agent.coordinator")
|
||||
|
||||
# 规则校验-修正循环的最大重试次数
|
||||
MAX_VALIDATION_RETRIES = 3
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 辅助:构建修正提示
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _build_correction_prompt(
|
||||
base_message: str,
|
||||
report: Any,
|
||||
) -> str:
|
||||
"""根据校验报告构建修正提示,追加到原始用户消息后。"""
|
||||
error_feedback = (
|
||||
f"\n\n=== 上一次输出的校验结果 ===\n"
|
||||
f"校验未通过,发现以下问题:\n"
|
||||
f"缺失字段 ({len(report.missing_fields)} 个):{', '.join(report.missing_fields)}\n"
|
||||
)
|
||||
if report.missing_materials:
|
||||
error_feedback += f"可能需要补充的材料:{', '.join(report.missing_materials)}\n"
|
||||
if report.suggestion:
|
||||
error_feedback += f"建议:{report.suggestion}\n"
|
||||
error_feedback += (
|
||||
"\n请根据以上校验结果修正你的输出,确保所有必填字段都有值。"
|
||||
"如果某个字段确实没有数据,请给出合理的猜测值。"
|
||||
"再次返回完整的 JSON 结果。"
|
||||
)
|
||||
return base_message + error_feedback
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 核心协调逻辑
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _do_extraction_with_validation(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
previous_analysis: dict[str, Any] | None = None,
|
||||
cache_map: dict[str, Any] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Agent 调度的提取-校验-修正循环。
|
||||
|
||||
流程:
|
||||
1. 加载缓存数据和匹配结果
|
||||
2. 构建用户消息
|
||||
3. 调用 LLM 提取
|
||||
4. 调用 validator 校验
|
||||
5. 校验失败则构建修正提示,回到步骤 3
|
||||
6. 最多重试 MAX_VALIDATION_RETRIES 次
|
||||
|
||||
LLM 输出的 JSON 中额外包含 can_submit 和 suggestion 字段,
|
||||
用于判断信息是否完整可提交。
|
||||
|
||||
Args:
|
||||
session_dir: 会话目录。
|
||||
session: 当前 Agent 会话。
|
||||
previous_analysis: 上一轮 LLM 分析结果(可选,补充文件时传入作为历史上下文)。
|
||||
|
||||
Returns:
|
||||
校验通过的结构化数据(或达到重试上限后的最佳结果)。
|
||||
"""
|
||||
cache_map = cache_map or load_cache(session_dir)
|
||||
match_result = load_match_result(session_dir)
|
||||
|
||||
# 选择系统提示词
|
||||
if session.invoice_type == "travel":
|
||||
system_prompt = build_travel_info_system_prompt()
|
||||
else:
|
||||
system_prompt = build_normal_info_system_prompt()
|
||||
|
||||
base_message = build_extraction_user_message(cache_map, match_result, previous_analysis=previous_analysis)
|
||||
current_message = base_message
|
||||
|
||||
for attempt in range(1, MAX_VALIDATION_RETRIES + 1):
|
||||
log.info(
|
||||
"LLM 提取第 %d/%d 次尝试 (%s)",
|
||||
attempt,
|
||||
MAX_VALIDATION_RETRIES,
|
||||
session.invoice_type,
|
||||
)
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_state_change",
|
||||
state=AgentState.EXTRACTING,
|
||||
round=session.rounds,
|
||||
attempt=attempt,
|
||||
message=f"正在分析文件... (第{attempt}次)",
|
||||
)
|
||||
|
||||
# Step 1: 调用 LLM 提取
|
||||
try:
|
||||
response = llm_query_text(
|
||||
system_prompt=system_prompt,
|
||||
text=current_message,
|
||||
reasoning_effort="low",
|
||||
source_dir=session_dir,
|
||||
)
|
||||
result = parse_json_response(response)
|
||||
except Exception as e:
|
||||
log.error("LLM 提取失败: %s", e)
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_error",
|
||||
message=f"LLM 提取失败: {e}",
|
||||
)
|
||||
raise
|
||||
|
||||
# Step 2: 调用 validator 校验
|
||||
report = validate_extracted_info(result, invoice_type=session.invoice_type)
|
||||
|
||||
if report.valid:
|
||||
log.info("规则校验通过 (第 %d 次尝试)", attempt)
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_extract_status",
|
||||
state=AgentState.EXTRACTING,
|
||||
round=session.rounds,
|
||||
attempt=attempt,
|
||||
message=f"规则校验通过 (第{attempt}次)",
|
||||
)
|
||||
return result
|
||||
|
||||
# Step 3: 校验失败,构建修正提示
|
||||
log.warning(
|
||||
"规则校验未通过 (第 %d/%d 次): 缺失 %d 个字段 - %s",
|
||||
attempt,
|
||||
MAX_VALIDATION_RETRIES,
|
||||
len(report.missing_fields),
|
||||
report.missing_fields,
|
||||
)
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_extract_status",
|
||||
state=AgentState.EXTRACTING,
|
||||
round=session.rounds,
|
||||
attempt=attempt,
|
||||
message=f"规则校验未通过,缺失 {len(report.missing_fields)} 个字段,正在请求 LLM 修正...",
|
||||
)
|
||||
current_message = _build_correction_prompt(current_message, report)
|
||||
|
||||
# 所有重试都失败,返回最后一次结果
|
||||
log.error(
|
||||
"LLM 提取经过 %d 次尝试仍未通过规则校验,返回最后一次结果 (置信度: %.0f%%)",
|
||||
MAX_VALIDATION_RETRIES,
|
||||
report.confidence * 100,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def run_agent_round(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
new_files: list[str] | None = None,
|
||||
) -> AgentSession:
|
||||
"""执行一轮 Agent 处理:提取-校验-修正循环。
|
||||
|
||||
Args:
|
||||
session_dir: 会话目录。
|
||||
session: 当前 Agent 会话。
|
||||
new_files: 新增的文件列表(可选,补充文件时传入)。
|
||||
|
||||
Returns:
|
||||
更新后的 Agent 会话。
|
||||
|
||||
注意:
|
||||
- 信息提取优先从 .invoice_cache 缓存读取,避免重复调用 LLM。
|
||||
- Agent 调度提取-校验-修正循环:LLM 提取 -> validator 校验 -> 失败则反馈修正。
|
||||
- 若缓存缺失则执行提取后立即写回缓存(travel_info.json / normal_info.json)。
|
||||
- 补充文件时(new_files 非空),加载上一轮分析结果作为历史上下文,强制重新分析。
|
||||
"""
|
||||
# 终态保护:会话已提交或已完成时不再重复处理
|
||||
if session.is_terminal():
|
||||
log.info("Agent 会话已处于终态 (%s),跳过重复处理", session.state.value)
|
||||
return session
|
||||
|
||||
if session.rounds >= session.max_rounds:
|
||||
session.state = AgentState.ERROR
|
||||
session.error_message = f"已达到最大轮次 ({session.max_rounds}),请检查信息或强制提交"
|
||||
log.warning("Agent 达到最大轮次限制")
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_max_rounds",
|
||||
message=session.error_message,
|
||||
)
|
||||
return session
|
||||
|
||||
session.rounds += 1
|
||||
log.info("开始第 %d 轮 Agent 处理", session.rounds)
|
||||
|
||||
# ---- Step 1: 信息提取(Agent 调度校验-修正循环) ----
|
||||
session.state = AgentState.EXTRACTING
|
||||
|
||||
# 判断是否为补充文件场景:有新文件传入时,加载上一轮分析结果作为上下文
|
||||
cache_map = load_cache(session_dir)
|
||||
is_supplement = bool(new_files)
|
||||
previous_analysis = None
|
||||
if is_supplement:
|
||||
info_key = "travel_info" if session.invoice_type == "travel" else "normal_info"
|
||||
previous_analysis = cache_map.get(info_key)
|
||||
if previous_analysis:
|
||||
log.info("检测到补充文件,加载上一轮分析结果作为历史上下文")
|
||||
|
||||
try:
|
||||
info_key = "travel_info" if session.invoice_type == "travel" else "normal_info"
|
||||
should_reanalyze = not cache_map.get(info_key) or is_supplement
|
||||
if should_reanalyze:
|
||||
session.extracted_info = _do_extraction_with_validation(
|
||||
session_dir, session, previous_analysis=previous_analysis, cache_map=cache_map
|
||||
)
|
||||
# 提取后立即写入缓存,后续步骤依赖此数据
|
||||
save_cache_info(session_dir, info_key, session.extracted_info)
|
||||
else:
|
||||
session.extracted_info = cache_map[info_key]
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_state_change",
|
||||
state=session.state,
|
||||
round=session.rounds,
|
||||
message="使用缓存数据,无需重新分析",
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
session.state = AgentState.ERROR
|
||||
session.error_message = f"信息提取失败: {e}"
|
||||
log.error("Agent 信息提取失败: %s", e)
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_error",
|
||||
message=session.error_message,
|
||||
)
|
||||
return session
|
||||
|
||||
# ---- 判断结果(从 LLM 提取结果中读 can_submit) ----
|
||||
can_submit = session.extracted_info.get("can_submit", True)
|
||||
suggestion = session.extracted_info.get("suggestion", "")
|
||||
|
||||
if can_submit:
|
||||
session.state = AgentState.READY
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_ready",
|
||||
round=session.rounds,
|
||||
message="信息完整,可以提交",
|
||||
)
|
||||
log.info("Agent 校验通过,信息完整")
|
||||
else:
|
||||
session.state = AgentState.AWAITING_SUPPLEMENT
|
||||
|
||||
combined_suggestion = suggestion or "信息不完整,请补充材料"
|
||||
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_request_supplement",
|
||||
round=session.rounds,
|
||||
missing_fields=[],
|
||||
missing_materials=[],
|
||||
semantic_issues=[],
|
||||
suggestion=combined_suggestion,
|
||||
)
|
||||
log.info("Agent 请求补充: %s", combined_suggestion)
|
||||
|
||||
save_agent_state(session_dir, session)
|
||||
return session
|
||||
|
||||
|
||||
def force_submit(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
) -> AgentSession:
|
||||
"""用户强制提交,跳过校验。"""
|
||||
session.state = AgentState.READY
|
||||
log.info("用户强制提交,跳过校验")
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_force_submit",
|
||||
message="用户选择强制提交",
|
||||
)
|
||||
save_agent_state(session_dir, session)
|
||||
return session
|
||||
|
||||
|
||||
def add_supplement(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
filenames: list[str],
|
||||
) -> AgentSession:
|
||||
"""记录用户补充的文件。"""
|
||||
session.user_supplements.extend(filenames)
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_supplement_received",
|
||||
files=filenames,
|
||||
)
|
||||
log.info("收到用户补充文件: %s", filenames)
|
||||
save_agent_state(session_dir, session)
|
||||
return session
|
||||
|
||||
|
||||
def process_user_text_supplement(
|
||||
session_dir: Path,
|
||||
session: AgentSession,
|
||||
user_text: str,
|
||||
) -> AgentSession:
|
||||
"""处理用户通过文字补充的信息。
|
||||
|
||||
流程:
|
||||
1. LLM 分析用户文字,提取需要更新的字段
|
||||
2. 合并到已提取的信息中
|
||||
3. 保存到缓存
|
||||
4. 重新执行一轮 Agent 校验
|
||||
|
||||
Args:
|
||||
session_dir: 会话目录。
|
||||
session: 当前 Agent 会话。
|
||||
user_text: 用户输入的文字。
|
||||
|
||||
Returns:
|
||||
更新后的 Agent 会话。
|
||||
"""
|
||||
log.info("收到用户文字补充: %s", user_text)
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_supplement_received",
|
||||
files=[user_text[:50]], # 简短显示
|
||||
)
|
||||
|
||||
# Step 1: LLM 分析用户文字
|
||||
supplement_result = process_user_supplement(
|
||||
user_text=user_text,
|
||||
extracted_info=session.extracted_info,
|
||||
invoice_type=session.invoice_type,
|
||||
source_dir=session_dir,
|
||||
)
|
||||
|
||||
updated_fields = supplement_result.get("updated_fields", {})
|
||||
unparsed = supplement_result.get("unparsed_info", "")
|
||||
|
||||
if updated_fields:
|
||||
# Step 2: 合并到已提取信息
|
||||
session.extracted_info = merge_supplement_into_info(
|
||||
session.extracted_info,
|
||||
updated_fields,
|
||||
)
|
||||
|
||||
# Step 3: 保存到缓存
|
||||
info_key = "travel_info" if session.invoice_type == "travel" else "normal_info"
|
||||
save_cache_info(session_dir, info_key, session.extracted_info)
|
||||
log.info("已更新 %s", info_key)
|
||||
|
||||
# Step 4: 重新执行 Agent 校验
|
||||
session.state = AgentState.EXTRACTING
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_state_change",
|
||||
state=session.state,
|
||||
round=session.rounds,
|
||||
message="正在重新校验...",
|
||||
)
|
||||
session = run_agent_round(session_dir, session)
|
||||
else:
|
||||
# 没有可更新的字段
|
||||
msg = unparsed or "未识别到可更新的报销信息"
|
||||
emit_agent_event(
|
||||
session_dir,
|
||||
"agent_supplement_received",
|
||||
files=[msg],
|
||||
)
|
||||
log.info("用户补充未识别到有效信息: %s", msg)
|
||||
|
||||
return session
|
||||
75
src/agent/events.py
Normal file
75
src/agent/events.py
Normal file
@@ -0,0 +1,75 @@
|
||||
"""Agent 事件系统
|
||||
|
||||
负责 SSE 事件的发射和管理,用于实时通知前端状态变化。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
|
||||
log = get_logger("agent.events")
|
||||
|
||||
AGENT_EVENT_LOG = "agent_events.log"
|
||||
|
||||
# 去重守卫:记录每个 session 上一次发射的事件类型,防止连续重复发射
|
||||
# key: str(session_dir), value: 上一次的 event_type
|
||||
_last_event_type: dict[str, str] = {}
|
||||
|
||||
|
||||
def emit_agent_event(session_dir: Path, event_type: str, **kwargs: Any) -> None:
|
||||
"""向 agent_events.log 追加一行 JSON 事件。
|
||||
|
||||
同一 session 连续发射相同 event_type 时直接抛出 RuntimeError,
|
||||
强制调用方修复重复发射的代码,而非静默掩盖。
|
||||
|
||||
注意:agent_state_change 会在同一轮提取中多次发射不同消息
|
||||
("正在分析"、"校验通过"、"校验失败,请求修正"),这是合法行为。
|
||||
去重守卫仅检查 event_type 字符串是否完全相同,不检查 kwargs。
|
||||
因此不要在同一个 event_type 下连续发射不同消息,应使用不同的事件类型。
|
||||
"""
|
||||
session_key = str(session_dir)
|
||||
prev = _last_event_type.get(session_key)
|
||||
if prev == event_type:
|
||||
raise RuntimeError(
|
||||
f"事件重复发射: session={session_dir.name!r}, event_type={event_type!r}。"
|
||||
f"请检查调用链,确保每个事件类型只发射一次。"
|
||||
)
|
||||
_last_event_type[session_key] = event_type
|
||||
|
||||
event = {"type": event_type, **kwargs}
|
||||
try:
|
||||
event_path = session_dir / AGENT_EVENT_LOG
|
||||
with open(event_path, "a", encoding="utf-8") as f:
|
||||
f.write(json.dumps(event, ensure_ascii=False) + "\n")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def clear_event_history(session_dir: Path) -> None:
|
||||
"""清除指定会话的事件历史。"""
|
||||
session_key = str(session_dir)
|
||||
_last_event_type.pop(session_key, None)
|
||||
event_path = session_dir / AGENT_EVENT_LOG
|
||||
if event_path.exists():
|
||||
event_path.unlink()
|
||||
|
||||
|
||||
def read_events(session_dir: Path) -> list[dict[str, Any]]:
|
||||
"""读取指定会话的所有事件记录。"""
|
||||
event_path = session_dir / AGENT_EVENT_LOG
|
||||
if not event_path.exists():
|
||||
return []
|
||||
events = []
|
||||
try:
|
||||
with open(event_path, encoding="utf-8") as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if line:
|
||||
events.append(json.loads(line))
|
||||
except Exception as e:
|
||||
log.warning("读取事件日志失败: %s", e)
|
||||
return events
|
||||
46
src/agent/orchestrator.py
Normal file
46
src/agent/orchestrator.py
Normal file
@@ -0,0 +1,46 @@
|
||||
"""Agent 协调器(兼容层)
|
||||
|
||||
此文件为向后兼容而保留,所有功能已迁移到以下子模块:
|
||||
- session.py: 会话状态定义和持久化
|
||||
- events.py: SSE 事件发射系统
|
||||
- coordinator.py: 核心协调逻辑
|
||||
|
||||
原有导入路径保持可用。
|
||||
"""
|
||||
|
||||
# 从新模块重新导出所有符号,保持向后兼容
|
||||
from .coordinator import (
|
||||
add_supplement,
|
||||
force_submit,
|
||||
process_user_text_supplement,
|
||||
run_agent_round,
|
||||
)
|
||||
from .events import emit_agent_event as _emit_agent_event
|
||||
from .session import (
|
||||
AgentSession,
|
||||
AgentState,
|
||||
load_agent_state,
|
||||
save_agent_state,
|
||||
)
|
||||
|
||||
# 为旧代码提供兼容的私有函数
|
||||
_emit_agent_event = _emit_agent_event
|
||||
|
||||
# 常量保持不变
|
||||
MAX_VALIDATION_RETRIES = 3
|
||||
AGENT_STATE_FILE = "agent_state.json"
|
||||
AGENT_EVENT_LOG = "agent_events.log"
|
||||
|
||||
__all__ = [
|
||||
"AgentSession",
|
||||
"AgentState",
|
||||
"save_agent_state",
|
||||
"load_agent_state",
|
||||
"run_agent_round",
|
||||
"force_submit",
|
||||
"add_supplement",
|
||||
"process_user_text_supplement",
|
||||
"MAX_VALIDATION_RETRIES",
|
||||
"AGENT_STATE_FILE",
|
||||
"AGENT_EVENT_LOG",
|
||||
]
|
||||
101
src/agent/session.py
Normal file
101
src/agent/session.py
Normal file
@@ -0,0 +1,101 @@
|
||||
"""Agent 会话管理
|
||||
|
||||
负责会话状态的定义、序列化和持久化。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import asdict, dataclass, field
|
||||
from enum import StrEnum
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .. import get_logger
|
||||
|
||||
log = get_logger("agent.session")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 状态枚举
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
class AgentState(StrEnum):
|
||||
IDLE = "idle"
|
||||
EXTRACTING = "extracting"
|
||||
AWAITING_SUPPLEMENT = "awaiting_supplement"
|
||||
READY = "ready"
|
||||
SUBMITTING = "submitting"
|
||||
DONE = "done"
|
||||
ERROR = "error"
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Agent 会话数据模型
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class AgentSession:
|
||||
"""Agent 会话状态"""
|
||||
|
||||
session_id: str
|
||||
state: AgentState = AgentState.IDLE
|
||||
rounds: int = 0
|
||||
max_rounds: int = 5
|
||||
invoice_type: str = "travel" # "travel" 或 "normal"
|
||||
extracted_info: dict[str, Any] = field(default_factory=dict)
|
||||
validation_reports: list[dict[str, Any]] = field(default_factory=list)
|
||||
user_supplements: list[str] = field(default_factory=list)
|
||||
error_message: str = ""
|
||||
|
||||
def to_dict(self) -> dict[str, Any]:
|
||||
return asdict(self)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: dict[str, Any]) -> AgentSession:
|
||||
# 兼容旧版本:state 可能是字符串
|
||||
if "state" in data and isinstance(data["state"], str):
|
||||
data["state"] = AgentState(data["state"])
|
||||
return cls(**data)
|
||||
|
||||
def is_terminal(self) -> bool:
|
||||
"""判断会话是否处于终态(已提交或已完成)"""
|
||||
return self.state in (AgentState.DONE, AgentState.SUBMITTING, AgentState.READY)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 持久化
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
AGENT_STATE_FILE = "agent_state.json"
|
||||
|
||||
|
||||
def save_agent_state(session_dir: Path, session: AgentSession) -> None:
|
||||
"""将 Agent 会话状态持久化到 session 目录。"""
|
||||
state_path = session_dir / AGENT_STATE_FILE
|
||||
tmp_path = session_dir / (AGENT_STATE_FILE + ".tmp")
|
||||
with open(tmp_path, "w", encoding="utf-8") as f:
|
||||
json.dump(session.to_dict(), f, ensure_ascii=False, indent=2)
|
||||
tmp_path.replace(state_path)
|
||||
|
||||
|
||||
def load_agent_state(session_dir: Path) -> AgentSession | None:
|
||||
"""从 session 目录加载 Agent 会话状态。"""
|
||||
state_path = session_dir / AGENT_STATE_FILE
|
||||
if not state_path.exists():
|
||||
return None
|
||||
try:
|
||||
with open(state_path, encoding="utf-8") as f:
|
||||
return AgentSession.from_dict(json.load(f))
|
||||
except Exception as e:
|
||||
log.warning("加载 Agent 状态失败: %s", e)
|
||||
return None
|
||||
|
||||
|
||||
def create_agent_session(session_id: str, invoice_type: str = "travel") -> AgentSession:
|
||||
"""创建新的 Agent 会话。"""
|
||||
return AgentSession(
|
||||
session_id=session_id,
|
||||
invoice_type=invoice_type,
|
||||
)
|
||||
103
src/config.py
Normal file
103
src/config.py
Normal file
@@ -0,0 +1,103 @@
|
||||
"""
|
||||
配置加载
|
||||
|
||||
从 scripts/data/config.json 读取用户配置,从环境变量读取服务端配置(LLM + 系统 URL)。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
_CONFIG_PATH = Path(__file__).parent.parent.parent / "scripts" / "data" / "config.json"
|
||||
|
||||
# 会话级配置允许覆盖的用户相关字段白名单(含密码)
|
||||
SESSION_CONFIG_KEYS = frozenset(
|
||||
{
|
||||
"username",
|
||||
"password",
|
||||
"default_name",
|
||||
"default_card_no",
|
||||
"default_person_id",
|
||||
"consumable_storage",
|
||||
}
|
||||
)
|
||||
|
||||
# 前端安全的配置字段白名单(不含密码)
|
||||
SAFE_CONFIG_KEYS = frozenset(
|
||||
{
|
||||
"username",
|
||||
"default_name",
|
||||
"default_card_no",
|
||||
"default_person_id",
|
||||
"consumable_storage",
|
||||
}
|
||||
)
|
||||
|
||||
# 模块级配置缓存
|
||||
_config_cache: dict[str, str | Path] | None = None
|
||||
|
||||
|
||||
def _read_config() -> dict[str, str | Path]:
|
||||
"""读取并合并配置,缺失字段使用默认值"""
|
||||
raw = {}
|
||||
if _CONFIG_PATH.exists():
|
||||
with open(_CONFIG_PATH, encoding="utf-8") as f:
|
||||
raw = json.load(f)
|
||||
|
||||
project_root = _CONFIG_PATH.parent
|
||||
|
||||
return {
|
||||
"sso_login_url": os.environ.get("SSO_LOGIN_URL", "https://tyrz.fynu.edu.cn/sso/login"),
|
||||
"portal_url": os.environ.get("PORTAL_URL", "https://tyrz.fynu.edu.cn/oshall"),
|
||||
"reimburse_url": os.environ.get("REIMBURSE_URL", "http://210.45.32.214:8081"),
|
||||
"reimburse_page": os.environ.get("REIMBURSE_PAGE", "/expen/common/common?v=4.0"),
|
||||
"travel_page": os.environ.get("TRAVEL_PAGE", "/expen/travel/travel?v=4.0"),
|
||||
"username": raw.get("username", ""),
|
||||
"password": raw.get("password", ""),
|
||||
"default_name": raw.get("default_name", ""),
|
||||
"default_card_no": raw.get("default_card_no", ""),
|
||||
"default_person_id": raw.get("default_person_id", ""),
|
||||
"consumable_storage": raw.get("consumable_storage", "躬行楼 C205"),
|
||||
"attachment_dir": project_root / "attachments",
|
||||
}
|
||||
|
||||
|
||||
def load_config() -> dict[str, str | Path]:
|
||||
"""加载并合并配置,使用模块级缓存避免重复读取文件"""
|
||||
global _config_cache
|
||||
if _config_cache is None:
|
||||
_config_cache = _read_config()
|
||||
return dict(_config_cache)
|
||||
|
||||
|
||||
def clear_config_cache() -> None:
|
||||
"""清除配置缓存(测试或配置变更时调用)"""
|
||||
global _config_cache
|
||||
_config_cache = None
|
||||
|
||||
|
||||
def load_session_config(session_dir: Path) -> dict[str, Any]:
|
||||
"""加载会话配置,合并项目全局配置与会话级配置
|
||||
|
||||
仅允许覆盖用户相关字段(白名单),防止用户上传的 config.json
|
||||
覆盖 sso_login_url、portal_url 等系统级配置。
|
||||
"""
|
||||
config = load_config()
|
||||
cfg_path = session_dir / "config.json"
|
||||
if cfg_path.exists():
|
||||
with open(cfg_path, encoding="utf-8") as f:
|
||||
session_cfg = json.load(f)
|
||||
for key in SESSION_CONFIG_KEYS:
|
||||
if key in session_cfg:
|
||||
config[key] = session_cfg[key]
|
||||
return config
|
||||
|
||||
|
||||
def get_llm_config() -> dict[str, str]:
|
||||
"""加载 LLM 配置,优先从环境变量读取,缺失字段使用默认值"""
|
||||
return {
|
||||
"model": os.environ.get("LLM_MODEL", "qwen-vl-max"),
|
||||
"api_base": os.environ.get("LLM_API_BASE", "http://localhost:8080/v1"),
|
||||
"api_key": os.environ.get("LLM_API_KEY", "lm-studio"),
|
||||
}
|
||||
21
src/core/README.md
Normal file
21
src/core/README.md
Normal file
@@ -0,0 +1,21 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/core — 核心业务逻辑
|
||||
|
||||
项目的核心业务层,负责信息提取、金额匹配和信息校验。此层不依赖 Web 框架或浏览器自动化等基础设施。
|
||||
|
||||
## 子模块
|
||||
|
||||
| 目录 | 说明 |
|
||||
|------|------|
|
||||
| `extraction/` | 文档信息提取:PDF/图片 → LLM 多模态识别 → 结构化数据 |
|
||||
| `matching/` | 发票与支付记录按金额匹配(一对一 / 一对多) |
|
||||
| `validation/` | 声明式信息完整性校验,规则从 JSON 配置文件加载 |
|
||||
|
||||
## 设计原则
|
||||
|
||||
- **零外部依赖**:不依赖 Flask、Playwright 等框架
|
||||
- **接口契约**:每个子模块通过 `__init__.py` 导出稳定的对外接口
|
||||
- **错误传播**:明确的异常层次,便于上层统一处理
|
||||
4
src/core/__init__.py
Normal file
4
src/core/__init__.py
Normal file
@@ -0,0 +1,4 @@
|
||||
"""核心业务逻辑模块
|
||||
|
||||
提供信息提取、规则校验和发票匹配功能。
|
||||
"""
|
||||
29
src/core/extraction/README.md
Normal file
29
src/core/extraction/README.md
Normal file
@@ -0,0 +1,29 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/core/extraction — 信息提取
|
||||
|
||||
从 PDF 发票和图片中提取结构化数据,是系统数据流的起点。
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `extractor.py` | 编排入口:扫描目录 → 逐文件提取 → 分类(发票/支付记录/申请单)→ 金额匹配 |
|
||||
| `llm_extractor.py` | LLM 多模态提取核心:统一文档提取、差旅/普通信息提取、缓存管理、SSE 流式事件 |
|
||||
|
||||
## 对外接口
|
||||
|
||||
| 函数 | 说明 |
|
||||
|------|------|
|
||||
| `extract_invoices(directory)` | 统一提取入口,返回 `(payment_records, applications, groups)` |
|
||||
| `extract_document(file_path)` | 从单个图片/PDF 提取信息 |
|
||||
| `extract_travel_info(source_dir)` | 综合发票和匹配结果提取差旅信息 |
|
||||
| `extract_normal_info(source_dir)` | 提取普通发票报销信息 |
|
||||
| `load_cache(source_dir)` | 加载缓存的结构化数据 |
|
||||
| `llm_query_text(...)` | 纯文本 LLM 查询(供 Agent 调度使用) |
|
||||
|
||||
## 缓存机制
|
||||
|
||||
提取结果缓存在 `.invoice_cache/` 目录中,文件名与源文件同名(`发票1.pdf` → `.invoice_cache/发票1.json`),避免重复调用 LLM。
|
||||
42
src/core/extraction/__init__.py
Normal file
42
src/core/extraction/__init__.py
Normal file
@@ -0,0 +1,42 @@
|
||||
"""信息提取模块
|
||||
|
||||
提供发票/文档结构化提取、LLM 辅助提取等功能。
|
||||
"""
|
||||
|
||||
from .extractor import (
|
||||
EXTRACTION_PARALLEL_COUNT,
|
||||
FILE_EVENTS_LOG,
|
||||
SUPPORTED_EXTENSIONS,
|
||||
extract_invoices,
|
||||
)
|
||||
from .llm_extractor import (
|
||||
CACHE_DIR_NAME,
|
||||
build_extraction_user_message,
|
||||
extract_document,
|
||||
extract_normal_info,
|
||||
extract_travel_info,
|
||||
llm_query_text,
|
||||
load_cache,
|
||||
load_match_result,
|
||||
merge_supplement_into_info,
|
||||
parse_json_response,
|
||||
process_user_supplement,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"CACHE_DIR_NAME",
|
||||
"build_extraction_user_message",
|
||||
"extract_document",
|
||||
"extract_normal_info",
|
||||
"extract_travel_info",
|
||||
"load_cache",
|
||||
"load_match_result",
|
||||
"llm_query_text",
|
||||
"merge_supplement_into_info",
|
||||
"parse_json_response",
|
||||
"process_user_supplement",
|
||||
"EXTRACTION_PARALLEL_COUNT",
|
||||
"FILE_EVENTS_LOG",
|
||||
"SUPPORTED_EXTENSIONS",
|
||||
"extract_invoices",
|
||||
]
|
||||
358
src/core/extraction/extractor.py
Normal file
358
src/core/extraction/extractor.py
Normal file
@@ -0,0 +1,358 @@
|
||||
"""发票提取编排
|
||||
|
||||
统一扫描目录下所有文件(PDF + 图片),通过 LLM 提取结构化数据,
|
||||
根据 LLM 返回的「invoice_type」字段自动分类为发票/支付记录/出差事前申请单。
|
||||
|
||||
对外接口:
|
||||
extract_invoices(directory) -> tuple[list[dict], list[dict], dict]
|
||||
|
||||
错误传播规则:
|
||||
- 单个文件提取失败: 记录日志 + SSE error 事件,继续处理下一个文件
|
||||
- 全部文件提取失败: raise ExtractionError,携带失败文件列表和原始错误
|
||||
- 缓存读取失败: 静默降级,尝试重新提取
|
||||
|
||||
SSE 文件进度事件:
|
||||
在 source_dir 下写入 file_events.log,每行一个 JSON 对象:
|
||||
- {"type": "file_progress", "file": "...", "status": "processing"}
|
||||
- {"type": "file_progress", "file": "...", "status": "done", "summary": {...}}
|
||||
- {"type": "file_progress", "file": "...", "status": "cached"}
|
||||
- {"type": "file_progress", "file": "...", "status": "error", "error": "..."}
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from ... import get_logger
|
||||
from ...core.matching import match_invoices_to_cards
|
||||
from ...exceptions import ExtractionError
|
||||
from ...infra.documents.invoice import CACHE_DIR_NAME, classify_invoice_batch
|
||||
from .llm_extractor import extract_document
|
||||
|
||||
log = get_logger("extractor")
|
||||
|
||||
# SSE 文件进度事件日志文件名
|
||||
FILE_EVENTS_LOG = "file_events.log"
|
||||
|
||||
# 并行提取文件数,可通过环境变量 EXTRACTION_PARALLEL_COUNT 配置
|
||||
EXTRACTION_PARALLEL_COUNT = int(os.environ.get("EXTRACTION_PARALLEL_COUNT", "3"))
|
||||
|
||||
# 支持的文件扩展名
|
||||
SUPPORTED_EXTENSIONS = {".pdf", ".png", ".jpg", ".jpeg", ".bmp", ".webp"}
|
||||
|
||||
|
||||
def _emit_file_event(source_dir: Path, file_name: str, status: str, **kwargs: Any) -> None:
|
||||
"""向 file_events.log 追加一行 JSON 事件(线程安全,失败时静默忽略)"""
|
||||
event = {
|
||||
"type": "file_progress",
|
||||
"file": file_name,
|
||||
"status": status,
|
||||
**kwargs,
|
||||
}
|
||||
try:
|
||||
event_path = source_dir / FILE_EVENTS_LOG
|
||||
with open(event_path, "a", encoding="utf-8") as f:
|
||||
f.write(json.dumps(event, ensure_ascii=False) + "\n")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _get_cache_dir(source_dir: Path) -> Path:
|
||||
"""获取缓存目录路径"""
|
||||
cache_dir = source_dir / CACHE_DIR_NAME
|
||||
cache_dir.mkdir(exist_ok=True)
|
||||
return cache_dir
|
||||
|
||||
|
||||
def _get_json_path(file_path: Path, cache_dir: Path) -> Path:
|
||||
"""根据文件路径生成对应的 JSON 缓存路径(包含后缀名以区分同名的 PDF/图片)"""
|
||||
return cache_dir / f"{file_path.stem}{file_path.suffix}.json"
|
||||
|
||||
|
||||
def _save_to_cache(file_path: Path, extracted_data: dict[str, Any], cache_dir: Path) -> Path:
|
||||
"""将提取结果保存到 JSON 缓存文件,并记录源文件路径和后缀名"""
|
||||
json_path = _get_json_path(file_path, cache_dir)
|
||||
cache_data = {
|
||||
"source_file": str(file_path),
|
||||
"source_filename": file_path.name,
|
||||
"source_extension": file_path.suffix.lower(),
|
||||
"extracted_data": extracted_data,
|
||||
}
|
||||
with open(json_path, "w", encoding="utf-8") as f:
|
||||
json.dump(cache_data, f, ensure_ascii=False, indent=2)
|
||||
log.info(f"提取结果已缓存: {json_path.name}")
|
||||
return json_path
|
||||
|
||||
|
||||
def _load_from_cache(json_path: Path, expected_extension: str | None = None) -> dict[str, Any] | None:
|
||||
"""从 JSON 缓存文件加载提取结果,可选校验后缀名一致性"""
|
||||
if not json_path.exists():
|
||||
return None
|
||||
try:
|
||||
with open(json_path, encoding="utf-8") as f:
|
||||
cache_data: dict[str, Any] = json.load(f)
|
||||
# 校验后缀名是否一致,防止同名不同后缀的文件误命中缓存
|
||||
if expected_extension and cache_data.get("source_extension", "").lower() != expected_extension.lower():
|
||||
return None
|
||||
result: dict[str, Any] | None = cache_data.get("extracted_data")
|
||||
return result
|
||||
except Exception as e:
|
||||
log.warning(f"读取缓存失败 {json_path.name}: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def _build_summary(data: dict[str, Any]) -> dict[str, str]:
|
||||
"""从提取结果构建前端展示摘要(人类可读格式)。
|
||||
|
||||
返回所有非内部字段(排除 _source_file 等下划线前缀字段),
|
||||
并将 invoice_type 转换为中文标签。列表/字典类型的值会展开为可读文本。
|
||||
"""
|
||||
invoice_type_map = {
|
||||
"train": "高铁票",
|
||||
"hotel": "酒店住宿",
|
||||
"general": "普通发票",
|
||||
"payment": "支付记录",
|
||||
"application": "出差申请单",
|
||||
}
|
||||
|
||||
summary = {}
|
||||
for key, value in data.items():
|
||||
# 跳过内部字段
|
||||
if key.startswith("_"):
|
||||
continue
|
||||
# 跳过空值
|
||||
if value is None or value == "":
|
||||
continue
|
||||
# invoice_type 转为中文标签
|
||||
if key == "invoice_type":
|
||||
summary["invoice_type_label"] = invoice_type_map.get(str(value), str(value))
|
||||
elif isinstance(value, list):
|
||||
# 列表展开为多行可读文本
|
||||
if len(value) == 0:
|
||||
continue
|
||||
parts = []
|
||||
for item in value:
|
||||
if isinstance(item, dict):
|
||||
# 字典项用 "key: value" 格式拼接
|
||||
pair_parts = [f"{k}: {v}" for k, v in item.items()]
|
||||
parts.append(" | ".join(pair_parts))
|
||||
else:
|
||||
parts.append(str(item))
|
||||
summary[key] = "\n".join(parts)
|
||||
elif isinstance(value, dict):
|
||||
# 字典展开为 "key: value" 格式
|
||||
pair_parts = [f"{k}: {v}" for k, v in value.items()]
|
||||
summary[key] = " | ".join(pair_parts)
|
||||
else:
|
||||
summary[key] = str(value)
|
||||
|
||||
return summary
|
||||
|
||||
|
||||
def _extract_document(
|
||||
file_path: Path,
|
||||
cache_dir: Path,
|
||||
source_dir: Path,
|
||||
) -> tuple[dict[str, str] | None, str | None]:
|
||||
"""提取单个文件的结构化信息,优先使用缓存。
|
||||
|
||||
根据 LLM 返回的「invoice_type」字段自动分类:
|
||||
- "payment" -> 支付记录
|
||||
- "application" -> 申请单
|
||||
- 有 "invoice_number" -> 发票
|
||||
- 其他 -> 无法识别
|
||||
|
||||
Args:
|
||||
file_path: 文件路径(PDF 或图片)。
|
||||
cache_dir: JSON 缓存目录。
|
||||
source_dir: 源目录(用于写入 SSE 进度事件)。
|
||||
|
||||
Returns:
|
||||
(提取结果字典或 None, 错误信息或 None)。
|
||||
"""
|
||||
file_name = file_path.name
|
||||
|
||||
json_path = _get_json_path(file_path, cache_dir)
|
||||
cached = _load_from_cache(json_path, expected_extension=file_path.suffix.lower())
|
||||
if cached:
|
||||
cached["_source_file"] = file_name
|
||||
log.info(f"使用缓存: {file_name}")
|
||||
_emit_file_event(source_dir, file_name, "cached")
|
||||
return cached, None
|
||||
|
||||
# 发送处理中事件
|
||||
_emit_file_event(source_dir, file_name, "processing")
|
||||
|
||||
log.info(f"使用多模态提取: {file_name}")
|
||||
try:
|
||||
result = extract_document(file_path)
|
||||
if result:
|
||||
result["_source_file"] = file_name
|
||||
_save_to_cache(file_path, result, cache_dir)
|
||||
|
||||
# 发送完成事件(含摘要)
|
||||
summary = _build_summary(result)
|
||||
_emit_file_event(source_dir, file_name, "done", summary=summary)
|
||||
|
||||
return result, None
|
||||
except Exception as e:
|
||||
err_msg = str(e)
|
||||
log.warning(f"多模态提取失败: {file_name} ({err_msg})")
|
||||
_emit_file_event(source_dir, file_name, "error", error=err_msg)
|
||||
return None, err_msg
|
||||
|
||||
return None, None
|
||||
|
||||
|
||||
def _find_all_files(directory: str) -> list[Path]:
|
||||
"""扫描目录下所有支持的文件(PDF + 图片)"""
|
||||
dir_path = Path(directory)
|
||||
files = [f for f in dir_path.iterdir() if f.is_file() and f.suffix.lower() in SUPPORTED_EXTENSIONS]
|
||||
return sorted(files)
|
||||
|
||||
|
||||
def _save_match_result(payment_records: list[dict[str, Any]], cache_dir: Path) -> None:
|
||||
"""将发票与支付记录的匹配结果保存到缓存。"""
|
||||
match_data: dict[str, list[dict[str, Any]]] = {}
|
||||
for record in payment_records:
|
||||
card_source = record.get("_source_file", "")
|
||||
matched = record.get("_matched_invoices", [])
|
||||
card_amount = record.get("card_amount", "")
|
||||
|
||||
if matched:
|
||||
invoice_list = [
|
||||
{
|
||||
"file": inv.get("_source_file", ""),
|
||||
"type": inv.get("invoice_type", ""),
|
||||
"amount": inv.get("total_amount", inv.get("total_amount", "")),
|
||||
}
|
||||
for inv in matched
|
||||
]
|
||||
|
||||
if card_source:
|
||||
match_data[f"{card_source} (¥{card_amount})"] = invoice_list
|
||||
else:
|
||||
key = "__unmatched__"
|
||||
if key not in match_data:
|
||||
match_data[key] = []
|
||||
match_data[key].extend(invoice_list)
|
||||
|
||||
if not match_data:
|
||||
return
|
||||
|
||||
match_path = cache_dir / "match_result.json"
|
||||
with open(match_path, "w", encoding="utf-8") as f:
|
||||
json.dump(match_data, f, ensure_ascii=False, indent=2)
|
||||
log.info(f"匹配结果已缓存: {match_path.name}")
|
||||
|
||||
|
||||
def extract_invoices(
|
||||
directory: str = ".",
|
||||
) -> tuple[list[dict[str, str]], list[dict[str, str]], dict[str, list[dict[str, str]]]]:
|
||||
"""扫描目录下所有文件,提取信息并匹配支付记录
|
||||
|
||||
统一使用 LLM 提取,根据返回的「invoice_type」自动分类:
|
||||
- 发票(有 invoice_number)-> 参与金额匹配
|
||||
- 支付记录(invoice_type="payment")-> 参与金额匹配
|
||||
- 出差事前申请单 -> 单独存储,不参与匹配
|
||||
|
||||
Returns:
|
||||
(payment_records, applications, groups):
|
||||
- payment_records: 支付记录列表(仅包含真实发票,不含申请单)
|
||||
- applications: 出差事前申请单列表(单独存储,不参与支付匹配)
|
||||
- groups: 按文档类型分组的字典
|
||||
{'travel': [差旅发票], 'general': [普通发票], 'application': [出差事前申请单]}
|
||||
|
||||
Raises:
|
||||
ExtractionError: 当所有文件提取均失败时抛出,携带失败文件列表和原始错误。
|
||||
"""
|
||||
source_dir = Path(directory)
|
||||
cache_dir = _get_cache_dir(source_dir)
|
||||
|
||||
# 清空上次的文件进度事件
|
||||
try:
|
||||
(source_dir / FILE_EVENTS_LOG).unlink(missing_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
all_files = _find_all_files(directory)
|
||||
if not all_files:
|
||||
log.warning("未找到支持的文件")
|
||||
return [], [], {"travel": [], "general": [], "application": []}
|
||||
|
||||
log.info(f"发现 {len(all_files)} 个文件")
|
||||
|
||||
all_invoices = []
|
||||
all_cards = []
|
||||
applications = []
|
||||
# 记录失败文件及其错误信息
|
||||
failed_files: list[tuple[str, str]] = []
|
||||
|
||||
max_workers = max(1, EXTRACTION_PARALLEL_COUNT)
|
||||
with ThreadPoolExecutor(max_workers=max_workers) as executor:
|
||||
future_to_file = {executor.submit(_extract_document, fp, cache_dir, source_dir): fp for fp in all_files}
|
||||
|
||||
for future in as_completed(future_to_file):
|
||||
file_path = future_to_file[future]
|
||||
try:
|
||||
result, err = future.result()
|
||||
except Exception as e:
|
||||
err_msg = str(e)
|
||||
failed_files.append((file_path.name, err_msg))
|
||||
log.warning(f"未能解析: {file_path.name} ({err_msg})")
|
||||
continue
|
||||
|
||||
if not result:
|
||||
if err:
|
||||
failed_files.append((file_path.name, err))
|
||||
log.warning(f"未能解析: {file_path.name}")
|
||||
continue
|
||||
|
||||
inv_type = result.get("invoice_type", "")
|
||||
|
||||
if inv_type == "application":
|
||||
applications.append(result)
|
||||
log.info(f"[{inv_type}] 已解析: {file_path.name}")
|
||||
elif inv_type == "payment":
|
||||
all_cards.append(result)
|
||||
log.info(f"[{inv_type}] 已解析: {file_path.name}")
|
||||
elif result.get("invoice_number"):
|
||||
all_invoices.append(result)
|
||||
log.info(f"[{inv_type}] 已解析: {file_path.name}")
|
||||
else:
|
||||
log.warning(f"无法分类: {file_path.name} (invoice_type={inv_type})")
|
||||
|
||||
# 全部文件提取失败时抛出异常,携带原始错误信息
|
||||
if failed_files and not all_invoices and not all_cards and not applications:
|
||||
failed_names = [name for name, _ in failed_files]
|
||||
error_details = {name: err for name, err in failed_files}
|
||||
raise ExtractionError(
|
||||
f"所有 {len(all_files)} 个文件提取均失败",
|
||||
failed_files=failed_names,
|
||||
details=error_details,
|
||||
)
|
||||
|
||||
log.info(f"分类结果: 发票 {len(all_invoices)} 张, 支付记录 {len(all_cards)} 条, 申请单 {len(applications)} 份")
|
||||
|
||||
if not all_invoices:
|
||||
log.warning("未成功解析任何发票")
|
||||
|
||||
payment_records = match_invoices_to_cards(all_invoices, all_cards)
|
||||
_save_match_result(payment_records, cache_dir)
|
||||
|
||||
# 构建分类
|
||||
all_documents: list[dict[str, str]] = []
|
||||
for record in payment_records:
|
||||
all_documents.extend(record.get("_matched_invoices", []))
|
||||
all_documents.extend(applications)
|
||||
|
||||
groups = classify_invoice_batch(all_documents)
|
||||
log.info(
|
||||
f"文档分类: 差旅发票 {len(groups['travel'])} 张, "
|
||||
f"普通发票 {len(groups['general'])} 张, "
|
||||
f"出差事前申请单 {len(groups['application'])} 份"
|
||||
)
|
||||
|
||||
return payment_records, applications, groups
|
||||
591
src/core/extraction/llm_extractor.py
Normal file
591
src/core/extraction/llm_extractor.py
Normal file
@@ -0,0 +1,591 @@
|
||||
"""LLM 信息提取
|
||||
|
||||
使用 LLM 从 PDF 文本/图片、支付截图中提取结构化数据。
|
||||
|
||||
## 功能模块
|
||||
|
||||
- **统一文档提取**:使用一套提示词,LLM 自行判断文档类型(发票/支付记录/出差事前申请单等),支持 JSON 格式输出。
|
||||
- **差旅信息提取**:综合多张发票、支付记录和匹配结果,提取出差事由、地点、时间等差旅相关信息。
|
||||
- **缓存管理**:支持从 `.invoice_cache/` 目录加载已提取的结构化数据和匹配结果,避免重复处理。
|
||||
- **SSE 流式事件**:`extract_travel_info` 和 `extract_normal_info` 在调用 LLM 时,向 `source_dir/llm_stream.log` 写入流式事件(start/reasoning/chunk/end/error),前端通过 SSE 实时展示 AI 思考过程与正式回答。
|
||||
|
||||
## 对外接口
|
||||
|
||||
- `extract_document(file_path) -> dict` — 统一入口:从任意图片/PDF 提取信息
|
||||
- `extract_travel_info(source_dir) -> dict` — 综合发票和匹配结果提取差旅信息
|
||||
- `extract_normal_info(source_dir) -> dict` — 提取普通发票报销信息
|
||||
- `load_cache(source_dir) -> dict` — 加载缓存的结构化数据
|
||||
- `load_match_result(source_dir) -> dict` — 加载发票与支付记录的匹配结果
|
||||
- `llm_query_text(system_prompt, text, source_dir) -> str` — 纯文本 LLM 查询(供 Agent 调度使用)
|
||||
- `parse_json_response(text) -> dict` — 从 LLM 响应中提取 JSON(供 Agent 调度使用)
|
||||
- `build_extraction_user_message(cache_map, match_result) -> str` — 构建提取请求的用户消息(供 Agent 调度使用)
|
||||
|
||||
## SSE 流式事件协议
|
||||
|
||||
`llm_stream.log` 每行一个 JSON 对象:
|
||||
|
||||
- `{"type": "llm_stream", "phase": "start", "label": "..."}` — LLM 调用开始
|
||||
- `{"type": "llm_stream", "phase": "reasoning", "text": "..."}` — 模型原生推理/思考片段(来自 `thinking_delta`)
|
||||
- `{"type": "llm_stream", "phase": "chunk", "text": "..."}` — 流式文本片段(正式回答)
|
||||
- `{"type": "llm_stream", "phase": "end", "label": "..."}` — LLM 调用结束
|
||||
- `{"type": "llm_stream", "phase": "error", "error": "..."}` — LLM 调用失败
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any, cast
|
||||
|
||||
from ... import get_logger
|
||||
from ...infra.documents.invoice import CACHE_DIR_NAME
|
||||
from ...infra.llm.prompt import (
|
||||
build_invoice_system_prompt,
|
||||
build_normal_info_system_prompt,
|
||||
build_supplement_system_prompt,
|
||||
build_travel_info_system_prompt,
|
||||
)
|
||||
|
||||
log = get_logger("llm_extractor")
|
||||
|
||||
# Re-export CACHE_DIR_NAME for convenience
|
||||
__all__ = [
|
||||
"CACHE_DIR_NAME",
|
||||
"extract_document",
|
||||
"extract_travel_info",
|
||||
"extract_normal_info",
|
||||
"load_cache",
|
||||
"load_match_result",
|
||||
"llm_query_text",
|
||||
"parse_json_response",
|
||||
"build_extraction_user_message",
|
||||
]
|
||||
|
||||
# SSE LLM 流式事件日志文件名
|
||||
LLM_STREAM_LOG = "llm_stream.log"
|
||||
|
||||
|
||||
def _emit_llm_stream(source_dir: Path, phase: str, **kwargs: Any) -> None:
|
||||
"""向 llm_stream.log 追加一行 JSON 事件(线程安全,失败时静默忽略)
|
||||
|
||||
Args:
|
||||
source_dir: 会话目录路径。
|
||||
phase: 事件阶段 ("start" / "chunk" / "end" / "error")。
|
||||
**kwargs: 额外字段 (text, label, error 等)。
|
||||
"""
|
||||
event = {"type": "llm_stream", "phase": phase, **kwargs}
|
||||
try:
|
||||
event_path = source_dir / LLM_STREAM_LOG
|
||||
with open(event_path, "a", encoding="utf-8") as f:
|
||||
f.write(json.dumps(event, ensure_ascii=False) + "\n")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _create_llm() -> Any:
|
||||
"""根据配置文件创建 LLM 实例。"""
|
||||
try:
|
||||
from llama_index.llms.openai_like import OpenAILike
|
||||
except ImportError:
|
||||
log.error("缺少 llama-index-llms-openai-like,请执行: uv pip install llama-index-llms-openai-like")
|
||||
raise
|
||||
|
||||
from ...config import get_llm_config
|
||||
|
||||
llm_config = get_llm_config()
|
||||
return OpenAILike(
|
||||
model=llm_config["model"],
|
||||
api_base=llm_config["api_base"],
|
||||
api_key=llm_config.get("api_key", "lm-studio"),
|
||||
temperature=0.1,
|
||||
max_tokens=65535,
|
||||
request_timeout=600.0,
|
||||
is_chat_model=True,
|
||||
)
|
||||
|
||||
|
||||
def parse_json_response(text: str) -> dict[str, Any]:
|
||||
"""从 LLM 响应中提取 JSON,处理可能的 Markdown 包裹。
|
||||
|
||||
Args:
|
||||
text: LLM 响应文本。
|
||||
|
||||
Returns:
|
||||
解析后的字典。
|
||||
"""
|
||||
text = text.strip()
|
||||
|
||||
# 处理 ```json ... ``` 包裹
|
||||
if "```" in text:
|
||||
start = text.find("```") + 3
|
||||
end = text.find("```", start)
|
||||
if end > start:
|
||||
text = text[start:end].strip()
|
||||
|
||||
# 去掉可能的前缀 (如 "json")
|
||||
if text.lower().startswith("json"):
|
||||
text = text[4:].strip()
|
||||
|
||||
return cast(dict[str, Any], json.loads(text))
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 统一文档提取(多模态,直接传图片给 LLM)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _image_to_base64(image_path: Path) -> str:
|
||||
"""将图片文件读取为 base64 字符串。"""
|
||||
with open(image_path, "rb") as f:
|
||||
return base64.b64encode(f.read()).decode("utf-8")
|
||||
|
||||
|
||||
def _stream_llm_response(
|
||||
llm: Any,
|
||||
messages: list[Any],
|
||||
source_dir: Path | None,
|
||||
reasoning_effort: str,
|
||||
log_label: str = "LLM",
|
||||
) -> str:
|
||||
"""流式调用 LLM 并写入 SSE 事件(供 llm_query_text 和 _llm_query_multimodal 共用)。
|
||||
|
||||
Args:
|
||||
llm: LLM 实例。
|
||||
messages: 消息列表。
|
||||
source_dir: 会话目录(可选,传入时启用 SSE 流式事件写入)。
|
||||
reasoning_effort: 推理努力级别。
|
||||
log_label: 日志标签(用于区分"纯文本"和"多模态")。
|
||||
|
||||
Returns:
|
||||
LLM 响应文本。
|
||||
"""
|
||||
try:
|
||||
if source_dir:
|
||||
_emit_llm_stream(source_dir, "start", label="正在分析文件...")
|
||||
|
||||
parts = []
|
||||
for resp in llm.stream_chat(
|
||||
messages,
|
||||
temperature=0.1,
|
||||
extra_body={"reasoning_effort": reasoning_effort},
|
||||
):
|
||||
delta = resp.delta
|
||||
if delta:
|
||||
parts.append(delta)
|
||||
if source_dir:
|
||||
_emit_llm_stream(source_dir, "chunk", text=delta)
|
||||
|
||||
thinking = getattr(resp, "additional_kwargs", {}) or {}
|
||||
thinking_delta = thinking.get("thinking_delta", "")
|
||||
if thinking_delta and source_dir:
|
||||
_emit_llm_stream(source_dir, "reasoning", text=thinking_delta)
|
||||
|
||||
text = "".join(parts)
|
||||
log.info("%s请求完成,响应总长度: %d 字符", log_label, len(text))
|
||||
if source_dir:
|
||||
_emit_llm_stream(source_dir, "end", label="分析完成")
|
||||
return text
|
||||
except Exception as e:
|
||||
log.error("%s请求失败: %s", log_label, e)
|
||||
if source_dir:
|
||||
_emit_llm_stream(source_dir, "error", error=str(e))
|
||||
raise
|
||||
|
||||
|
||||
def llm_query_text(
|
||||
system_prompt: str,
|
||||
text: str,
|
||||
reasoning_effort: str = "none",
|
||||
source_dir: Path | None = None,
|
||||
) -> str:
|
||||
"""发送纯文本请求到 LLM(供 Agent 调度使用)。
|
||||
|
||||
Args:
|
||||
system_prompt: 系统提示词。
|
||||
text: 用户文本。
|
||||
reasoning_effort: 推理努力级别。
|
||||
source_dir: 会话目录(可选,传入时启用 SSE 流式事件写入)。
|
||||
|
||||
Returns:
|
||||
LLM 响应文本。
|
||||
"""
|
||||
from llama_index.core.base.llms.types import TextBlock
|
||||
from llama_index.core.llms import ChatMessage
|
||||
|
||||
from ...config import get_llm_config
|
||||
|
||||
messages = [
|
||||
ChatMessage(role="system", content=system_prompt),
|
||||
ChatMessage(role="user", blocks=[TextBlock(text=text)]),
|
||||
]
|
||||
|
||||
llm_config = get_llm_config()
|
||||
llm = _create_llm()
|
||||
log.info(
|
||||
"开始请求 LLM (model=%s, base=%s)",
|
||||
llm_config["model"],
|
||||
llm_config["api_base"],
|
||||
)
|
||||
|
||||
return _stream_llm_response(llm, messages, source_dir, reasoning_effort, log_label="LLM")
|
||||
|
||||
|
||||
def extract_document(file_path: Path) -> dict[str, Any]:
|
||||
"""统一文档提取入口:从任意图片/PDF 中提取结构化信息。
|
||||
|
||||
LLM 会根据统一提示词自行判断文档类型(发票/支付记录/出差事前申请单等)。
|
||||
|
||||
Args:
|
||||
file_path: 文件路径(支持 PDF 和图片格式)。
|
||||
|
||||
Returns:
|
||||
包含提取字段的字典。
|
||||
"""
|
||||
from ...infra.documents.pdf import render_pdf_to_images
|
||||
|
||||
system_prompt = build_invoice_system_prompt()
|
||||
user_text = f"请分析以下财务文档并提取信息:\n\n文件名: {file_path.name}"
|
||||
|
||||
# PDF 先渲染为图片
|
||||
suffix = file_path.suffix.lower()
|
||||
if suffix == ".pdf":
|
||||
image_b64s = render_pdf_to_images(file_path)
|
||||
else:
|
||||
image_b64s = [_image_to_base64(file_path)]
|
||||
|
||||
if not image_b64s:
|
||||
log.warning(f"文件渲染为空: {file_path.name}")
|
||||
return {}
|
||||
|
||||
try:
|
||||
response = _llm_query_multimodal(system_prompt, user_text, image_b64s)
|
||||
result = parse_json_response(response)
|
||||
log.info("LLM 文档提取成功: %s", file_path.name)
|
||||
return result
|
||||
except Exception as e:
|
||||
log.error("LLM 文档提取失败: %s (%s)", file_path.name, e)
|
||||
raise
|
||||
|
||||
|
||||
def _llm_query_multimodal(
|
||||
system_prompt: str,
|
||||
text: str | None = None,
|
||||
image_b64s: list[str] | None = None,
|
||||
blocks: list[Any] | None = None,
|
||||
reasoning_effort: str = "none",
|
||||
source_dir: Path | None = None,
|
||||
) -> str:
|
||||
"""发送多模态请求到 LLM(内部使用)。
|
||||
|
||||
Args:
|
||||
system_prompt: 系统提示词。
|
||||
text: 用户文本(与 image_b64s 配合使用,文本在前、图片在后)。
|
||||
image_b64s: base64 编码的图片列表。
|
||||
blocks: 预构建的内容块列表(TextBlock/ImageBlock),传入时忽略 text 和 image_b64s。
|
||||
source_dir: 会话目录(可选,传入时启用 SSE 流式事件写入)。
|
||||
|
||||
Returns:
|
||||
LLM 响应文本。
|
||||
"""
|
||||
from llama_index.core.base.llms.types import ImageBlock, TextBlock
|
||||
from llama_index.core.llms import ChatMessage
|
||||
|
||||
from ...config import get_llm_config
|
||||
|
||||
if blocks is not None:
|
||||
final_blocks = blocks
|
||||
else:
|
||||
text = text or ""
|
||||
image_b64s = image_b64s or []
|
||||
final_blocks = [TextBlock(text=text)]
|
||||
for img_b64 in image_b64s:
|
||||
final_blocks.append(
|
||||
ImageBlock(
|
||||
url=f"data:image/jpeg;base64,{img_b64}",
|
||||
detail="high",
|
||||
)
|
||||
)
|
||||
|
||||
messages = [
|
||||
ChatMessage(role="system", content=system_prompt),
|
||||
ChatMessage(role="user", blocks=final_blocks),
|
||||
]
|
||||
|
||||
llm_config = get_llm_config()
|
||||
llm = _create_llm()
|
||||
log.info(
|
||||
"开始请求 LLM 多模态 (model=%s, base=%s, blocks=%d)",
|
||||
llm_config["model"],
|
||||
llm_config["api_base"],
|
||||
len(final_blocks),
|
||||
)
|
||||
|
||||
return _stream_llm_response(llm, messages, source_dir, reasoning_effort, log_label="LLM多模态")
|
||||
|
||||
|
||||
def load_cache(source_dir: Path) -> dict[str, Any]:
|
||||
"""从 JSON 缓存目录加载结构化数据,构建 source filename -> 缓存数据的映射。
|
||||
|
||||
Args:
|
||||
source_dir: 源文件目录(包含 .invoice_cache 子目录)。
|
||||
|
||||
Returns:
|
||||
{source_filename: extracted_data} 字典。
|
||||
额外包含 "travel_info" 键(如果 travel_info.json 存在)。
|
||||
"""
|
||||
cache_map: dict[str, Any] = {}
|
||||
cache_dir = source_dir / CACHE_DIR_NAME
|
||||
if not cache_dir.exists():
|
||||
return cache_map
|
||||
|
||||
for json_path in sorted(cache_dir.glob("*.json")):
|
||||
try:
|
||||
with open(json_path, encoding="utf-8") as f:
|
||||
cache_data = json.load(f)
|
||||
|
||||
if json_path.name in ("travel_info.json", "normal_info.json"):
|
||||
cache_map[json_path.name.replace(".json", "")] = cache_data
|
||||
continue
|
||||
|
||||
extracted = cache_data.get("extracted_data", {})
|
||||
src_file = extracted.get("_source_file", "")
|
||||
if src_file:
|
||||
cache_map[src_file] = extracted
|
||||
except Exception as e:
|
||||
log.warning(f"读取缓存失败 {json_path.name}: {e}")
|
||||
|
||||
return cache_map
|
||||
|
||||
|
||||
def load_match_result(source_dir: Path) -> dict[str, list[dict[str, Any]]]:
|
||||
"""从 JSON 缓存目录加载发票与支付记录的匹配结果。
|
||||
|
||||
Args:
|
||||
source_dir: 源文件目录(包含 .invoice_cache 子目录)。
|
||||
|
||||
Returns:
|
||||
{支付记录源文件 (含金额): [发票信息列表]} 字典。
|
||||
每个发票信息包含 file, type, amount 字段。
|
||||
"""
|
||||
cache_dir = source_dir / CACHE_DIR_NAME
|
||||
match_path = cache_dir / "match_result.json"
|
||||
if not match_path.exists():
|
||||
return {}
|
||||
|
||||
try:
|
||||
with open(match_path, encoding="utf-8") as f:
|
||||
result: dict[str, list[dict[str, Any]]] = json.load(f)
|
||||
return result
|
||||
except Exception as e:
|
||||
log.warning(f"读取匹配结果缓存失败: {e}")
|
||||
return {}
|
||||
|
||||
|
||||
def build_extraction_user_message(
|
||||
cache_map: dict[str, Any],
|
||||
match_result: dict[str, list[dict[str, Any]]],
|
||||
previous_analysis: dict[str, Any] | None = None,
|
||||
) -> str:
|
||||
"""构建提取请求的用户消息(供 Agent 调度使用)。
|
||||
|
||||
Args:
|
||||
cache_map: 缓存数据映射。
|
||||
match_result: 匹配结果。
|
||||
previous_analysis: 上一轮 LLM 分析结果(可选,补充文件时传入作为历史上下文)。
|
||||
|
||||
Returns:
|
||||
拼接好的用户消息字符串。
|
||||
"""
|
||||
parts = [
|
||||
"以下是本次报销的所有源文件及其提取出的结构化数据。"
|
||||
"每个源文件的数据来自 OCR 识别和发票信息提取,已按文件名分组展示。"
|
||||
]
|
||||
|
||||
if previous_analysis:
|
||||
parts.append(
|
||||
"【上一轮分析结果】"
|
||||
"以下是上一轮 LLM 对已有文件的分析结果。"
|
||||
"注意:用户可能已补充新文件,请综合所有数据(含新文件)重新分析。"
|
||||
"如果新文件填补了之前的信息缺失,请相应更新分析结果。\n"
|
||||
+ json.dumps(previous_analysis, ensure_ascii=False, indent=2)
|
||||
)
|
||||
|
||||
if match_result:
|
||||
parts.append(
|
||||
"【发票与支付记录匹配结果】"
|
||||
"以下数据已将发票信息与对应的支付记录进行关联匹配,"
|
||||
"用于判断每笔支付对应的发票和商户信息。\n" + json.dumps(match_result, ensure_ascii=False, indent=2)
|
||||
)
|
||||
|
||||
for filename, extracted in cache_map.items():
|
||||
parts.append(
|
||||
f"【源文件: {filename}】"
|
||||
"以下为从该文件提取的结构化发票/支付/申请单数据。\n" + json.dumps(extracted, ensure_ascii=False, indent=2)
|
||||
)
|
||||
|
||||
parts.append("\n=== 请返回 JSON 格式结果 ===")
|
||||
return "\n".join(parts)
|
||||
|
||||
|
||||
def _extract_info(
|
||||
source_dir: Path | None,
|
||||
system_prompt: str,
|
||||
info_type: str,
|
||||
) -> dict[str, Any]:
|
||||
"""通用的信息提取函数:加载缓存、构建消息、调用 LLM 并解析 JSON。
|
||||
|
||||
extract_travel_info 和 extract_normal_info 的公共实现。
|
||||
|
||||
Args:
|
||||
source_dir: 源文件目录(必填,包含 .invoice_cache 子目录)。
|
||||
system_prompt: 系统提示词。
|
||||
info_type: 信息类型标签("差旅" 或 "普通发票"),用于日志。
|
||||
|
||||
Returns:
|
||||
LLM 提取的结构化信息字典。
|
||||
"""
|
||||
if not source_dir:
|
||||
log.warning("未提供 source_dir,无法加载缓存数据")
|
||||
return {}
|
||||
|
||||
cache_map = load_cache(source_dir)
|
||||
match_result = load_match_result(source_dir)
|
||||
user_message = build_extraction_user_message(cache_map, match_result)
|
||||
|
||||
log.info("开始构建%s信息提取请求,缓存条目: %d, 匹配结果: %d", info_type, len(cache_map), len(match_result))
|
||||
|
||||
try:
|
||||
response = llm_query_text(
|
||||
system_prompt=system_prompt,
|
||||
text=user_message,
|
||||
reasoning_effort="low",
|
||||
source_dir=source_dir,
|
||||
)
|
||||
result = parse_json_response(response)
|
||||
log.info("LLM %s信息提取成功", info_type)
|
||||
return result
|
||||
except Exception as e:
|
||||
log.error("LLM %s信息提取失败: %s", info_type, e)
|
||||
raise
|
||||
|
||||
|
||||
def extract_travel_info(
|
||||
source_dir: Path | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""根据差旅发票(bot 格式),让 LLM 提取出差相关信息。
|
||||
|
||||
纯提取,不包含校验逻辑。校验由 Agent 层调度。
|
||||
|
||||
Args:
|
||||
source_dir: 源文件目录(必填,包含 .invoice_cache 子目录)。
|
||||
|
||||
Returns:
|
||||
包含出差事由、地点、交通工具、时间、住宿信息等字段的字典。
|
||||
"""
|
||||
return _extract_info(source_dir, build_travel_info_system_prompt(), "差旅")
|
||||
|
||||
|
||||
def extract_normal_info(
|
||||
source_dir: Path | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""根据普通发票(非差旅),让 LLM 提取报销相关信息。
|
||||
|
||||
纯提取,不包含校验逻辑。校验由 Agent 层调度。
|
||||
|
||||
Args:
|
||||
source_dir: 源文件目录(必填,包含 .invoice_cache 子目录)。
|
||||
|
||||
Returns:
|
||||
包含报销说明、发票总数、总金额、支付方式、附件清单等字段的字典。
|
||||
"""
|
||||
return _extract_info(source_dir, build_normal_info_system_prompt(), "普通发票")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 用户补充信息处理
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def process_user_supplement(
|
||||
user_text: str,
|
||||
extracted_info: dict[str, Any],
|
||||
invoice_type: str,
|
||||
source_dir: Path | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""让用户补充的文字信息通过 LLM 分析,返回需要更新的字段。
|
||||
|
||||
Args:
|
||||
user_text: 用户输入的文字。
|
||||
extracted_info: 当前已提取的报销信息。
|
||||
invoice_type: "travel" 或 "normal"。
|
||||
source_dir: 会话目录(可选,传入时启用 SSE 流式事件写入)。
|
||||
|
||||
Returns:
|
||||
包含 updated_fields, changes, confidence, unparsed_info 的字典。
|
||||
"""
|
||||
system_prompt = build_supplement_system_prompt()
|
||||
|
||||
parts = [
|
||||
f"发票类型: {'差旅报销' if invoice_type == 'travel' else '普通报销'}",
|
||||
"",
|
||||
"以下是当前已提取的报销信息:",
|
||||
json.dumps(extracted_info, ensure_ascii=False, indent=2),
|
||||
"",
|
||||
f"用户补充信息:{user_text}",
|
||||
"",
|
||||
"=== 请分析用户输入并返回需要更新的字段 ===",
|
||||
]
|
||||
user_message = "\n".join(parts)
|
||||
|
||||
try:
|
||||
response = llm_query_text(
|
||||
system_prompt=system_prompt,
|
||||
text=user_message,
|
||||
reasoning_effort="low",
|
||||
source_dir=source_dir,
|
||||
)
|
||||
result = parse_json_response(response)
|
||||
log.info("LLM 补充信息分析完成")
|
||||
|
||||
return result
|
||||
except Exception as e:
|
||||
log.error("LLM 补充信息分析失败: %s", e)
|
||||
return {
|
||||
"updated_fields": {},
|
||||
"changes": [],
|
||||
"confidence": 0.0,
|
||||
"unparsed_info": f"分析失败: {e}",
|
||||
}
|
||||
|
||||
|
||||
def merge_supplement_into_info(
|
||||
extracted_info: dict[str, Any],
|
||||
updated_fields: dict[str, Any],
|
||||
) -> dict[str, Any]:
|
||||
"""将 LLM 返回的更新字段合并到已提取的信息中。
|
||||
|
||||
支持点号路径(如 basic_info.travel_purpose)表示嵌套更新。
|
||||
|
||||
Args:
|
||||
extracted_info: 当前已提取的报销信息。
|
||||
updated_fields: LLM 返回的需要更新的字段。
|
||||
|
||||
Returns:
|
||||
更新后的报销信息。
|
||||
"""
|
||||
import copy
|
||||
|
||||
result = copy.deepcopy(extracted_info)
|
||||
|
||||
for field_path, value in updated_fields.items():
|
||||
parts = field_path.split(".")
|
||||
current = result
|
||||
for part in parts[:-1]:
|
||||
if part not in current:
|
||||
current[part] = {}
|
||||
current = current[part]
|
||||
current[parts[-1]] = value
|
||||
log.info("更新字段 %s = %s", field_path, value)
|
||||
|
||||
return result
|
||||
27
src/core/matching/README.md
Normal file
27
src/core/matching/README.md
Normal file
@@ -0,0 +1,27 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/core/matching — 金额匹配
|
||||
|
||||
将提取到的发票数据与支付记录(刷卡截图)按金额进行匹配。
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `matcher.py` | 匹配引擎:一对一匹配、一对多贪心匹配、未匹配发票处理 |
|
||||
|
||||
## 匹配策略
|
||||
|
||||
| 场景 | 策略 |
|
||||
|------|------|
|
||||
| 发票数 == 支付记录数 | 一对一匹配:按金额降序配对,相对容差内即匹配 |
|
||||
| 发票数 > 支付记录数 | 一对多匹配:贪心算法凑金额,相对容差 3% |
|
||||
| 文件名匹配 | 最高优先级:文件名(不含后缀)一致时直接匹配 |
|
||||
| 未匹配发票 | 单独列为一条支付记录,`remark` 标记为 `"unmatched"` |
|
||||
|
||||
## 业务约束
|
||||
|
||||
- 发票总金额 >= 支付总金额
|
||||
- 输出以支付记录为主键的结果列表
|
||||
8
src/core/matching/__init__.py
Normal file
8
src/core/matching/__init__.py
Normal file
@@ -0,0 +1,8 @@
|
||||
"""匹配模块
|
||||
|
||||
提供发票与支付记录的金额匹配功能。
|
||||
"""
|
||||
|
||||
from .matcher import match_invoices_to_cards
|
||||
|
||||
__all__ = ["match_invoices_to_cards"]
|
||||
409
src/core/matching/matcher.py
Normal file
409
src/core/matching/matcher.py
Normal file
@@ -0,0 +1,409 @@
|
||||
"""发票与支付记录匹配
|
||||
|
||||
将提取到的发票数据与支付记录进行金额匹配,
|
||||
输出以支付记录为主键的结果列表。
|
||||
|
||||
支付记录由统一提取模块(extractor)根据 LLM 返回的「invoice_type」字段分类而来,
|
||||
不再按文件类型假设文档类型。
|
||||
|
||||
## 业务约束
|
||||
|
||||
- 发票数 >= 付款记录数(最少一张发票对应一张付款记录)
|
||||
- 发票总金额 >= 付款总金额(发票只能比付款多,不能少)
|
||||
- 若发票数 == 付款数,走一对一匹配,无需一对多
|
||||
|
||||
## 匹配流程
|
||||
|
||||
1. 接收分类好的发票和支付记录列表
|
||||
2. 解析发票和刷卡记录的金额,进行总额校验
|
||||
- 发票总额 < 刷卡总额时发出 warning
|
||||
3. 按金额降序排序
|
||||
4. 文件名匹配(最高优先级):发票和刷卡记录的文件名(不含后缀)一致时直接匹配
|
||||
5. 根据数量关系选择匹配策略:
|
||||
- 数量相等 → 一对一匹配:按金额从大到小依次配对,相对容差内即匹配
|
||||
- 发票更多 → 一对多匹配:对每张刷卡记录贪心凑金额,相对容差内结束
|
||||
6. 构建以支付记录为主键的结果列表
|
||||
7. 未匹配的发票单独作为一条记录(无刷卡信息)
|
||||
8. 清理内部字段,输出支付记录列表
|
||||
|
||||
## 容差计算
|
||||
|
||||
使用相对容差(默认 3%),以刷卡金额为基准:
|
||||
- ¥2900 发票 vs ¥2850 刷卡 → 差 ¥50,容差 ¥85.5 → 匹配成功
|
||||
- ¥100 发票 vs ¥95 刷卡 → 差 ¥5,容差 ¥3.0 → 不匹配(需精确匹配或调整)
|
||||
|
||||
## 一对多匹配细节
|
||||
|
||||
- 对每张刷卡记录,维护 remaining(剩余待匹配金额)
|
||||
- 遍历未分配的发票(按金额降序):
|
||||
- 若发票金额 + 容差 >= remaining,视为最后一张,匹配后退出
|
||||
- 否则发票金额不超过 remaining + 容差即可匹配
|
||||
- 匹配后 remaining 为负且超出容差时回滚最后一张发票
|
||||
- 每张发票只会被分配一次
|
||||
|
||||
## 对外接口
|
||||
|
||||
match_invoices_to_cards(invoices, cards, tolerance) -> list[dict]
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from ... import get_logger
|
||||
|
||||
log = get_logger("matcher")
|
||||
|
||||
|
||||
def _safe_float(value: str | None, default: float = 0.0) -> float:
|
||||
"""安全转换为浮点数"""
|
||||
if value is None or str(value).strip() == "":
|
||||
return default
|
||||
try:
|
||||
return float(str(value).replace(",", ""))
|
||||
except (ValueError, TypeError):
|
||||
return default
|
||||
|
||||
|
||||
def _build_invoice_summary(invoices: list[dict[str, Any]]) -> str:
|
||||
"""将多张发票信息汇总为备注字符串"""
|
||||
parts = []
|
||||
for inv in invoices:
|
||||
inv_type = inv.get("invoice_type", "unknown")
|
||||
amount = inv.get("total_amount", "unknown")
|
||||
|
||||
if inv_type == "train":
|
||||
label = inv.get("person_name") or inv.get("invoice_number", "unknown")
|
||||
elif inv_type == "hotel":
|
||||
label = "hotel"
|
||||
else:
|
||||
label = inv.get("item_name") or inv.get("invoice_number", "unknown")
|
||||
|
||||
parts.append(f"{inv_type}[{label}]¥{amount}")
|
||||
return " | ".join(parts)
|
||||
|
||||
|
||||
def _relative_tolerance(base: float, rate: float = 0.03) -> float:
|
||||
"""根据基准金额计算相对容差(默认 3%)"""
|
||||
return abs(base) * rate
|
||||
|
||||
|
||||
def match_invoices_to_cards(
|
||||
invoices: list[dict[str, Any]],
|
||||
cards: list[dict[str, Any]] | None = None,
|
||||
tolerance: float = 0.03,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""将发票与支付记录按金额匹配,输出以支付记录为主键的结果列表
|
||||
|
||||
支付记录由统一提取模块(extractor)根据 LLM 返回的「invoice_type」字段分类而来。
|
||||
|
||||
业务约束:
|
||||
- 发票数 >= 付款记录数
|
||||
- 发票总金额 >= 付款总金额(发票只能比付款多,不能少)
|
||||
- 若发票数 == 付款数,一对一匹配,无需一对多
|
||||
|
||||
Args:
|
||||
invoices: 发票列表,需包含 "total_amount" 字段
|
||||
cards: 支付记录列表,需包含 "card_amount" 字段(由 extractor 分类提供)
|
||||
tolerance: 金额匹配容差比例(默认 0.03 = 3%)
|
||||
|
||||
Returns:
|
||||
以支付记录为主键的结果列表,每条记录包含:
|
||||
- card_date, card_no, card_amount(支付信息)
|
||||
- 关联发票列表(_matched_invoices)
|
||||
- 发票详情备注
|
||||
- 未匹配发票单独作为一条无刷卡信息的记录
|
||||
"""
|
||||
if not cards:
|
||||
log.warning("无支付记录可供匹配,发票将保持原状")
|
||||
return _invoices_to_records(invoices)
|
||||
|
||||
# 解析金额
|
||||
for card in cards:
|
||||
card["_amount"] = _safe_float(card.get("card_amount"))
|
||||
for inv in invoices:
|
||||
inv["_amount"] = _safe_float(inv.get("total_amount"))
|
||||
|
||||
# 数据校验
|
||||
total_invoices = sum(inv["_amount"] for inv in invoices)
|
||||
total_cards = sum(card["_amount"] for card in cards)
|
||||
log.info(
|
||||
f"金额校验: 发票总额 ¥{total_invoices:.2f}, 刷卡总额 ¥{total_cards:.2f}, "
|
||||
f"发票数 {len(invoices)}, 刷卡数 {len(cards)}"
|
||||
)
|
||||
|
||||
total_tolerance = _relative_tolerance(max(total_invoices, total_cards), tolerance)
|
||||
if total_invoices < total_cards - total_tolerance:
|
||||
log.warning(
|
||||
f"发票总额 (¥{total_invoices:.2f}) 小于刷卡总额 (¥{total_cards:.2f}), "
|
||||
f"超出容差 {tolerance * 100:.0f}%, 匹配结果可能有偏差"
|
||||
)
|
||||
|
||||
# 按金额降序排序
|
||||
cards.sort(key=lambda c: c["_amount"], reverse=True)
|
||||
invoices.sort(key=lambda i: i["_amount"], reverse=True)
|
||||
|
||||
# 执行匹配,返回 {card_index: [invoice_indices]} 的映射
|
||||
card_to_invoices = _match(cards, invoices, tolerance)
|
||||
|
||||
# 构建支付记录列表
|
||||
records = _build_payment_records(cards, invoices, card_to_invoices)
|
||||
|
||||
# 清理内部字段
|
||||
for inv in invoices:
|
||||
inv.pop("_amount", None)
|
||||
for card in cards:
|
||||
card.pop("_amount", None)
|
||||
|
||||
# 统计
|
||||
matched_invoices = sum(len(inv_list) for inv_list in card_to_invoices.values())
|
||||
unmatched_count = len(invoices) - matched_invoices
|
||||
log.info(f"匹配完成: {len(records)} 条支付记录, {matched_invoices}/{len(invoices)} 张发票已关联")
|
||||
if unmatched_count:
|
||||
log.info(f"未匹配发票: {unmatched_count} 张(已单独列为记录)")
|
||||
|
||||
return records
|
||||
|
||||
|
||||
def _match(
|
||||
cards: list[dict[str, Any]],
|
||||
invoices: list[dict[str, Any]],
|
||||
tolerance: float,
|
||||
) -> dict[int, list[int]]:
|
||||
"""执行匹配,返回 {card_index: [invoice_indices]} 的映射
|
||||
|
||||
tolerance 为相对容差比例(如 0.03 表示 3%)
|
||||
|
||||
匹配优先级(从高到低):
|
||||
1. 文件名匹配:发票和刷卡记录的文件名(不含后缀)一致时直接匹配
|
||||
2. 精确匹配:金额差 <= 0.01 元
|
||||
3. 一对一 / 一对多贪心匹配:按金额容差匹配
|
||||
"""
|
||||
result: dict[int, list[int]] = {}
|
||||
assigned: set[int] = set()
|
||||
|
||||
# ---- 阶段 0:文件名匹配(最高优先级)----
|
||||
_match_by_filename(invoices, cards, assigned, result)
|
||||
|
||||
if len(invoices) == len(cards):
|
||||
_match_one_to_one(invoices, cards, tolerance, assigned, result)
|
||||
else:
|
||||
_match_one_to_many(invoices, cards, tolerance, assigned, result)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def _match_by_filename(
|
||||
invoices: list[dict[str, Any]],
|
||||
cards: list[dict[str, Any]],
|
||||
assigned: set[int],
|
||||
result: dict[int, list[int]],
|
||||
) -> None:
|
||||
"""文件名匹配:发票和刷卡记录的文件名(不含后缀)一致时直接匹配"""
|
||||
for card_idx, card in enumerate(cards):
|
||||
card_name = card.get("_source_file", "")
|
||||
if not card_name:
|
||||
continue
|
||||
card_stem = Path(card_name).stem
|
||||
|
||||
for idx, inv in enumerate(invoices):
|
||||
if idx in assigned:
|
||||
continue
|
||||
inv_name = inv.get("_source_file", "")
|
||||
if not inv_name:
|
||||
continue
|
||||
inv_stem = Path(inv_name).stem
|
||||
|
||||
if inv_stem == card_stem:
|
||||
assigned.add(idx)
|
||||
result[card_idx] = [idx]
|
||||
log.info(f"[文件名匹配] {inv_name} ↔ {card_name}")
|
||||
break
|
||||
|
||||
|
||||
def _match_one_to_one(
|
||||
invoices: list[dict[str, Any]],
|
||||
cards: list[dict[str, Any]],
|
||||
tolerance: float,
|
||||
assigned: set[int],
|
||||
result: dict[int, list[int]],
|
||||
) -> None:
|
||||
"""一对一匹配:发票数等于刷卡数,对每张刷卡记录寻找金额最接近的未分配发票"""
|
||||
for card_idx, card in enumerate(cards):
|
||||
if card_idx in result:
|
||||
continue
|
||||
card_amount = card["_amount"]
|
||||
card_tol = _relative_tolerance(card_amount, tolerance)
|
||||
|
||||
# 在未分配的发票中找金额最接近的
|
||||
best_idx = -1
|
||||
best_diff = float("inf")
|
||||
for idx, inv in enumerate(invoices):
|
||||
if idx in assigned:
|
||||
continue
|
||||
diff = abs(inv["_amount"] - card_amount)
|
||||
if diff < best_diff:
|
||||
best_diff = diff
|
||||
best_idx = idx
|
||||
|
||||
if best_idx >= 0 and best_diff <= card_tol:
|
||||
inv = invoices[best_idx]
|
||||
assigned.add(best_idx)
|
||||
result[card_idx] = [best_idx]
|
||||
log.info(
|
||||
f"[一对一] {inv.get('invoice_number', 'unknown')} ¥{inv['_amount']:.2f} "
|
||||
f"↔ {card.get('_source_file', 'unknown')} ¥{card_amount:.2f}"
|
||||
)
|
||||
elif best_idx >= 0:
|
||||
inv = invoices[best_idx]
|
||||
log.warning(
|
||||
f"[一对一] 金额偏差超出容差: "
|
||||
f"{inv.get('invoice_number', 'unknown')} ¥{inv['_amount']:.2f} "
|
||||
f"vs ¥{card_amount:.2f} (差 ¥{best_diff:.2f}, 容差 ¥{card_tol:.2f})"
|
||||
)
|
||||
|
||||
|
||||
def _match_one_to_many(
|
||||
invoices: list[dict[str, Any]],
|
||||
cards: list[dict[str, Any]],
|
||||
tolerance: float,
|
||||
assigned: set[int],
|
||||
result: dict[int, list[int]],
|
||||
) -> None:
|
||||
"""一对多匹配:一张刷卡可能对应多张发票,按金额从大到小贪心匹配"""
|
||||
|
||||
# ---- 阶段 1:精确匹配(金额差 <= 0.01 元视为相等)----
|
||||
exact_tolerance = 0.01
|
||||
for card_idx, card in enumerate(cards):
|
||||
card_amount = card["_amount"]
|
||||
if card_amount <= 0:
|
||||
continue
|
||||
|
||||
for idx, inv in enumerate(invoices):
|
||||
if idx in assigned:
|
||||
continue
|
||||
inv_amount = inv["_amount"]
|
||||
if inv_amount <= 0:
|
||||
continue
|
||||
|
||||
if abs(inv_amount - card_amount) <= exact_tolerance:
|
||||
assigned.add(idx)
|
||||
result[card_idx] = [idx]
|
||||
log.info(
|
||||
f"[一对多-精确] {inv.get('invoice_number', 'unknown')} ¥{inv_amount:.2f} "
|
||||
f"↔ {card.get('_source_file', 'unknown')} ¥{card_amount:.2f}"
|
||||
)
|
||||
break
|
||||
|
||||
# ---- 阶段 2:贪心匹配(仅处理未精确匹配的刷卡记录)----
|
||||
for card_idx, card in enumerate(cards):
|
||||
if card_idx in result:
|
||||
continue
|
||||
|
||||
card_amount = card["_amount"]
|
||||
if card_amount <= 0:
|
||||
continue
|
||||
|
||||
card_tol = _relative_tolerance(card_amount, tolerance)
|
||||
|
||||
remaining = card_amount
|
||||
matched_indices: list[int] = []
|
||||
|
||||
for idx, inv in enumerate(invoices):
|
||||
if idx in assigned:
|
||||
continue
|
||||
if remaining <= card_tol:
|
||||
break
|
||||
|
||||
inv_amount = inv["_amount"]
|
||||
if inv_amount <= 0:
|
||||
continue
|
||||
|
||||
if inv_amount + card_tol >= remaining:
|
||||
is_match = True
|
||||
else:
|
||||
is_match = inv_amount <= remaining + card_tol
|
||||
|
||||
if is_match:
|
||||
assigned.add(idx)
|
||||
matched_indices.append(idx)
|
||||
remaining -= inv_amount
|
||||
if remaining <= card_tol:
|
||||
break
|
||||
|
||||
# 回滚:如果匹配后 remaining 为负且超出容差
|
||||
if remaining < -card_tol and matched_indices:
|
||||
last_idx = matched_indices.pop()
|
||||
assigned.discard(last_idx)
|
||||
remaining += invoices[last_idx]["_amount"]
|
||||
|
||||
# 记录匹配结果
|
||||
if matched_indices:
|
||||
result[card_idx] = matched_indices
|
||||
for idx in matched_indices:
|
||||
inv = invoices[idx]
|
||||
log.info(
|
||||
f"[一对多-贪心] {inv.get('invoice_number', 'unknown')} ¥{inv['_amount']:.2f} "
|
||||
f"→ {card.get('_source_file', 'unknown')} ¥{card['_amount']:.2f}"
|
||||
)
|
||||
|
||||
|
||||
def _build_payment_records(
|
||||
cards: list[dict[str, Any]],
|
||||
invoices: list[dict[str, Any]],
|
||||
card_to_invoices: dict[int, list[int]],
|
||||
) -> list[dict[str, Any]]:
|
||||
"""构建以支付记录为主键的结果列表"""
|
||||
records: list[dict[str, Any]] = []
|
||||
|
||||
for card_idx, inv_indices in card_to_invoices.items():
|
||||
card = cards[card_idx]
|
||||
matched_invs = [invoices[idx] for idx in inv_indices]
|
||||
|
||||
record = {
|
||||
"card_date": card.get("card_date", ""),
|
||||
"card_no": card.get("card_no", ""),
|
||||
"card_amount": str(card["_amount"]),
|
||||
"relative_invoice_count": str(len(matched_invs)),
|
||||
"invoice_detail": _build_invoice_summary(matched_invs),
|
||||
"remark": "",
|
||||
"_source_file": card.get("_source_file", ""),
|
||||
"_matched_invoices": matched_invs,
|
||||
}
|
||||
records.append(record)
|
||||
|
||||
# 未匹配的发票,单独作为记录
|
||||
matched_indices = set()
|
||||
for inv_indices in card_to_invoices.values():
|
||||
matched_indices.update(inv_indices)
|
||||
|
||||
unmatched = [inv for idx, inv in enumerate(invoices) if idx not in matched_indices]
|
||||
for inv in unmatched:
|
||||
record = {
|
||||
"card_date": "",
|
||||
"card_no": "",
|
||||
"card_amount": "",
|
||||
"relative_invoice_count": "1",
|
||||
"invoice_detail": _build_invoice_summary([inv]),
|
||||
"remark": "unmatched",
|
||||
"_matched_invoices": [inv],
|
||||
}
|
||||
records.append(record)
|
||||
|
||||
return records
|
||||
|
||||
|
||||
def _invoices_to_records(invoices: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
"""无刷卡记录时,将每张发票转为独立记录"""
|
||||
records = []
|
||||
for inv in invoices:
|
||||
record = {
|
||||
"card_date": "",
|
||||
"card_no": "",
|
||||
"card_amount": "",
|
||||
"relative_invoice_count": "1",
|
||||
"invoice_detail": _build_invoice_summary([inv]),
|
||||
"remark": "",
|
||||
"_matched_invoices": [inv],
|
||||
}
|
||||
records.append(record)
|
||||
return records
|
||||
34
src/core/validation/README.md
Normal file
34
src/core/validation/README.md
Normal file
@@ -0,0 +1,34 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/core/validation — 信息校验
|
||||
|
||||
对 LLM 提取的报销信息进行声明式规则校验,判断是否满足填报要求。
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `validator.py` | 校验引擎:加载 JSON 规则配置 → 遍历字段/数组 → 输出校验报告 |
|
||||
|
||||
## 设计特点
|
||||
|
||||
- **规则与引擎分离**:校验规则存储在 `config/validation_rules.json`,引擎只负责执行
|
||||
- **统一路径定位**:使用 `path` 列表定位嵌套字段,如 `["basic_info", "travel_purpose"]`
|
||||
- **自定义校验**:支持 `custom_check` 函数(日期格式、正数检查等)
|
||||
- **数组元素校验**:支持 `min_items` 最小数量 + 每个元素的必填字段
|
||||
|
||||
## 校验规则类型
|
||||
|
||||
| 类型 | 用途 | 配置项 |
|
||||
|------|------|--------|
|
||||
| `fields` | 顶层单值字段 | `path`, `required`, `custom_check`, `check_empty` |
|
||||
| `arrays` | 数组字段 | `path`, `min_items`, `element_fields` |
|
||||
|
||||
## 对外接口
|
||||
|
||||
| 函数 | 说明 |
|
||||
|------|------|
|
||||
| `validate(info, invoice_type)` | 执行校验,返回 `ValidationReport` |
|
||||
| `get_missing_fields(report)` | 提取缺失字段列表 |
|
||||
28
src/core/validation/__init__.py
Normal file
28
src/core/validation/__init__.py
Normal file
@@ -0,0 +1,28 @@
|
||||
"""校验模块
|
||||
|
||||
提供报销信息的规则级校验功能。
|
||||
"""
|
||||
|
||||
from .validator import (
|
||||
ArrayRule,
|
||||
FieldRule,
|
||||
ValidationReport,
|
||||
ValidationRules,
|
||||
get_validation_rules,
|
||||
reload_validation_rules,
|
||||
validate_extracted_info,
|
||||
validate_normal_info,
|
||||
validate_travel_info,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"validate_extracted_info",
|
||||
"validate_travel_info",
|
||||
"validate_normal_info",
|
||||
"ValidationReport",
|
||||
"FieldRule",
|
||||
"ArrayRule",
|
||||
"ValidationRules",
|
||||
"get_validation_rules",
|
||||
"reload_validation_rules",
|
||||
]
|
||||
537
src/core/validation/validator.py
Normal file
537
src/core/validation/validator.py
Normal file
@@ -0,0 +1,537 @@
|
||||
"""信息完整性校验器
|
||||
|
||||
对 LLM 提取的报销信息进行规则级校验,判断是否满足填报要求。
|
||||
|
||||
校验规则从 JSON 配置文件加载,支持声明式配置。
|
||||
|
||||
设计理念:
|
||||
使用声明式规则配置,将校验规则与校验逻辑分离,提高可读性和可维护性。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any, TypedDict
|
||||
|
||||
from ... import get_logger
|
||||
|
||||
log = get_logger("validator")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 日期格式
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
DATE_PATTERN = re.compile(r"^\d{4}-\d{2}-\d{2}$")
|
||||
|
||||
|
||||
def _is_valid_date(value: str) -> bool:
|
||||
"""检查日期格式是否为 YYYY-MM-DD。"""
|
||||
return bool(DATE_PATTERN.match(value))
|
||||
|
||||
|
||||
def _is_positive_number(value: Any) -> bool:
|
||||
"""检查值是否为正数(整数或浮点数)。"""
|
||||
return isinstance(value, int | float) and value > 0
|
||||
|
||||
|
||||
def _is_positive_integer(value: Any) -> bool:
|
||||
"""检查值是否为正整数。"""
|
||||
return isinstance(value, int) and value > 0
|
||||
|
||||
|
||||
# 自定义校验函数注册表
|
||||
_CUSTOM_CHECKS: dict[str, Callable[[Any], bool]] = {
|
||||
"is_valid_date": _is_valid_date,
|
||||
"is_positive_number": _is_positive_number,
|
||||
"is_positive_integer": _is_positive_integer,
|
||||
}
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 规则定义
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
class FieldRule(TypedDict, total=False):
|
||||
"""字段校验规则(统一使用 path 定位)"""
|
||||
|
||||
path: list[str] # 字段路径(统一定位方式)
|
||||
required: bool = True # 是否必填(默认必填)
|
||||
check_empty: bool = True # 是否检查空字符串(默认检查)
|
||||
custom_check: str | Callable[[Any], bool] | None = None # 自定义校验函数(名称或函数)
|
||||
description: str = "" # 字段描述(用于生成友好提示)
|
||||
|
||||
|
||||
class ArrayRule(TypedDict, total=False):
|
||||
"""数组校验规则"""
|
||||
|
||||
path: list[str] # 数组路径
|
||||
min_items: int = 1 # 最小元素数量
|
||||
element_fields: list[str | FieldRule] = [] # 元素字段规则
|
||||
description: str = "" # 数组描述
|
||||
|
||||
|
||||
class ValidationRules(TypedDict):
|
||||
"""校验规则集合"""
|
||||
|
||||
fields: list[FieldRule] # 字段规则列表
|
||||
arrays: list[ArrayRule] # 数组规则列表
|
||||
|
||||
|
||||
class ValidationConfig(TypedDict):
|
||||
"""校验配置结构"""
|
||||
|
||||
version: str
|
||||
custom_checks: dict[str, str]
|
||||
travel: ValidationRules
|
||||
normal: ValidationRules
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 配置加载
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
_CONFIG_PATH = Path(__file__).parent.parent.parent / "config" / "validation_rules.json"
|
||||
_cached_rules: ValidationConfig | None = None
|
||||
|
||||
|
||||
def _load_validation_config() -> ValidationConfig:
|
||||
"""加载校验规则配置文件。"""
|
||||
global _cached_rules
|
||||
if _cached_rules is not None:
|
||||
return _cached_rules
|
||||
|
||||
if not _CONFIG_PATH.exists():
|
||||
log.warning("校验规则配置文件不存在: %s,使用内置默认规则", _CONFIG_PATH)
|
||||
return _load_default_rules()
|
||||
|
||||
try:
|
||||
with open(_CONFIG_PATH, encoding="utf-8") as f:
|
||||
config = json.load(f)
|
||||
_cached_rules = _resolve_custom_checks(config)
|
||||
log.info("校验规则配置加载成功")
|
||||
return _cached_rules
|
||||
except Exception as e:
|
||||
log.error("加载校验规则配置失败: %s,使用内置默认规则", e)
|
||||
return _load_default_rules()
|
||||
|
||||
|
||||
def _resolve_custom_checks(config: dict[str, Any]) -> ValidationConfig:
|
||||
"""解析配置中的自定义校验函数名称,替换为实际函数引用。"""
|
||||
|
||||
def resolve_rule(rule: dict[str, Any]) -> dict[str, Any]:
|
||||
if "custom_check" in rule and isinstance(rule["custom_check"], str):
|
||||
check_name = rule["custom_check"]
|
||||
if check_name in _CUSTOM_CHECKS:
|
||||
rule["custom_check"] = _CUSTOM_CHECKS[check_name]
|
||||
else:
|
||||
log.warning("未知的自定义校验函数: %s", check_name)
|
||||
rule["custom_check"] = None
|
||||
return rule
|
||||
|
||||
# 解析 travel 规则的 fields
|
||||
for field_rule in config.get("travel", {}).get("fields", []):
|
||||
resolve_rule(field_rule)
|
||||
# 解析 element_fields
|
||||
for array_rule in config.get("travel", {}).get("arrays", []):
|
||||
for elem_field in array_rule.get("element_fields", []):
|
||||
if isinstance(elem_field, dict):
|
||||
resolve_rule(elem_field)
|
||||
|
||||
# 解析 normal 规则的 fields
|
||||
for field_rule in config.get("normal", {}).get("fields", []):
|
||||
resolve_rule(field_rule)
|
||||
# 解析 element_fields
|
||||
for array_rule in config.get("normal", {}).get("arrays", []):
|
||||
for elem_field in array_rule.get("element_fields", []):
|
||||
if isinstance(elem_field, dict):
|
||||
resolve_rule(elem_field)
|
||||
|
||||
return config # type: ignore[return-value]
|
||||
|
||||
|
||||
def _load_default_rules() -> ValidationConfig:
|
||||
"""返回内置的默认校验规则(当配置文件不存在时使用)。"""
|
||||
return {
|
||||
"version": "1.0",
|
||||
"custom_checks": {},
|
||||
"travel": {
|
||||
"fields": [
|
||||
{"path": ["basic_info", "travel_purpose"], "description": "出差事由"},
|
||||
{"path": ["basic_info", "travel_location"], "description": "出差地点"},
|
||||
{"path": ["basic_info", "start_date"], "custom_check": _is_valid_date, "description": "出差开始日期"},
|
||||
{"path": ["basic_info", "end_date"], "custom_check": _is_valid_date, "description": "出差结束日期"},
|
||||
],
|
||||
"arrays": [
|
||||
{
|
||||
"path": ["reimbursement_details", "transport_fee"],
|
||||
"min_items": 1,
|
||||
"element_fields": [
|
||||
{"path": ["vehicle_type"], "description": "交通工具类型"},
|
||||
{"path": ["start_date"], "custom_check": _is_valid_date, "description": "出发日期"},
|
||||
{"path": ["end_date"], "custom_check": _is_valid_date, "description": "到达日期"},
|
||||
{"path": ["departure_place"], "description": "出发地"},
|
||||
{"path": ["arrival_place"], "description": "目的地"},
|
||||
{"path": ["amount"], "custom_check": _is_positive_number, "description": "金额"},
|
||||
{"path": ["bill_count"], "custom_check": _is_positive_integer, "description": "票据张数"},
|
||||
{"path": ["remark"], "check_empty": False, "description": "备注说明"},
|
||||
],
|
||||
"description": "交通费用明细",
|
||||
},
|
||||
{
|
||||
"path": ["payment_methods"],
|
||||
"min_items": 1,
|
||||
"element_fields": [
|
||||
{"path": ["card_date"], "custom_check": _is_valid_date, "description": "刷卡日期"},
|
||||
{"path": ["card_amount"], "custom_check": _is_positive_number, "description": "支付金额"},
|
||||
{"path": ["merchant"], "description": "商户名称"},
|
||||
{"path": ["remark"], "check_empty": False, "description": "备注"},
|
||||
],
|
||||
"description": "支付方式记录",
|
||||
},
|
||||
{
|
||||
"path": ["subsidy_list"],
|
||||
"min_items": 1,
|
||||
"element_fields": [
|
||||
{"path": ["person_id"], "description": "人员工号"},
|
||||
{"path": ["person_name"], "description": "人员姓名"},
|
||||
{"path": ["start_date"], "custom_check": _is_valid_date, "description": "补助开始日期"},
|
||||
{"path": ["end_date"], "custom_check": _is_valid_date, "description": "补助结束日期"},
|
||||
{"path": ["days"], "custom_check": _is_positive_integer, "description": "补助天数"},
|
||||
],
|
||||
"description": "补助清单",
|
||||
},
|
||||
{
|
||||
"path": ["attachments"],
|
||||
"min_items": 0,
|
||||
"element_fields": [
|
||||
{"path": ["filename"], "description": "文件名"},
|
||||
{"path": ["attachment_type"], "description": "附件类型"},
|
||||
],
|
||||
"description": "附件列表",
|
||||
},
|
||||
],
|
||||
},
|
||||
"normal": {
|
||||
"fields": [
|
||||
{"path": ["basic_info", "reimbursement_description"], "description": "报销事由"},
|
||||
{
|
||||
"path": ["reimbursement_details", "total_invoices"],
|
||||
"custom_check": _is_positive_integer,
|
||||
"description": "发票总数",
|
||||
},
|
||||
{
|
||||
"path": ["reimbursement_details", "total_amount"],
|
||||
"custom_check": _is_positive_number,
|
||||
"description": "总金额",
|
||||
},
|
||||
],
|
||||
"arrays": [
|
||||
{
|
||||
"path": ["payment_methods"],
|
||||
"min_items": 1,
|
||||
"element_fields": [
|
||||
{"path": ["card_date"], "custom_check": _is_valid_date, "description": "刷卡日期"},
|
||||
{"path": ["card_amount"], "custom_check": _is_positive_number, "description": "支付金额"},
|
||||
{"path": ["merchant"], "description": "商户名称"},
|
||||
{"path": ["remark"], "check_empty": False, "description": "备注"},
|
||||
],
|
||||
"description": "支付方式记录",
|
||||
},
|
||||
{
|
||||
"path": ["attachments"],
|
||||
"min_items": 0,
|
||||
"element_fields": [
|
||||
{"path": ["filename"], "description": "文件名"},
|
||||
{"path": ["attachment_type"], "description": "附件类型"},
|
||||
],
|
||||
"description": "附件列表",
|
||||
},
|
||||
],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def get_validation_rules(invoice_type: str) -> ValidationRules:
|
||||
"""获取指定发票类型的校验规则。
|
||||
|
||||
Args:
|
||||
invoice_type: 发票类型,'travel' 或 'normal'。
|
||||
|
||||
Returns:
|
||||
对应的校验规则。
|
||||
"""
|
||||
config = _load_validation_config()
|
||||
return config.get(invoice_type, config.get("travel", {})) # type: ignore[return-value]
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 数据模型
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class ValidationReport:
|
||||
"""校验结果报告"""
|
||||
|
||||
valid: bool
|
||||
missing_fields: list[str] = field(default_factory=list)
|
||||
missing_materials: list[str] = field(default_factory=list)
|
||||
confidence: float = 0.0
|
||||
suggestion: str = ""
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 通用校验引擎
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _check_field(
|
||||
data: dict[str, Any],
|
||||
rule: FieldRule,
|
||||
) -> tuple[bool, str]:
|
||||
"""根据字段规则检查字段。"""
|
||||
path = rule["path"]
|
||||
check_empty = rule.get("check_empty", True)
|
||||
custom_check = rule.get("custom_check")
|
||||
|
||||
current = data
|
||||
for key in path:
|
||||
if not isinstance(current, dict):
|
||||
return (False, ".".join(path))
|
||||
if key not in current:
|
||||
return (False, ".".join(path))
|
||||
current = current[key]
|
||||
|
||||
if check_empty and isinstance(current, str) and not current.strip():
|
||||
return (False, ".".join(path))
|
||||
|
||||
if custom_check is not None and not custom_check(current):
|
||||
return (False, ".".join(path))
|
||||
|
||||
return (True, ".".join(path))
|
||||
|
||||
|
||||
def _check_array(
|
||||
data: dict[str, Any],
|
||||
rule: ArrayRule,
|
||||
) -> tuple[list[str], int, int]:
|
||||
"""根据数组规则检查数组。
|
||||
|
||||
Returns:
|
||||
(缺失字段列表, 总检查数, 通过检查数)
|
||||
"""
|
||||
missing: list[str] = []
|
||||
path = rule["path"]
|
||||
min_items = rule.get("min_items", 1)
|
||||
element_fields = rule.get("element_fields", [])
|
||||
path_str = ".".join(path)
|
||||
|
||||
total_checks = 1 # 数组存在性和最小数量检查
|
||||
passed_checks = 0
|
||||
|
||||
# 遍历路径获取数组
|
||||
current = data
|
||||
for key in path:
|
||||
if not isinstance(current, dict) or key not in current:
|
||||
return ([path_str], total_checks, passed_checks)
|
||||
current = current[key]
|
||||
|
||||
# 检查数组是否满足最小数量要求
|
||||
if not isinstance(current, list) or len(current) < min_items:
|
||||
return ([path_str], total_checks, passed_checks)
|
||||
|
||||
passed_checks += 1 # 数组检查通过
|
||||
|
||||
# 检查数组元素的字段
|
||||
if element_fields:
|
||||
for i, item in enumerate(current):
|
||||
if not isinstance(item, dict):
|
||||
missing.append(f"{path_str}[{i}]")
|
||||
total_checks += len(element_fields)
|
||||
continue
|
||||
|
||||
for field_rule in element_fields:
|
||||
total_checks += 1
|
||||
# 支持两种格式:简单字符串格式 和 详细规则格式
|
||||
if isinstance(field_rule, str):
|
||||
field_rule_dict: FieldRule = {"path": [field_rule]}
|
||||
else:
|
||||
field_rule_dict = field_rule
|
||||
|
||||
# 复用 _check_field 函数检查元素字段
|
||||
ok, _ = _check_field(item, field_rule_dict)
|
||||
if ok:
|
||||
passed_checks += 1
|
||||
else:
|
||||
field_path_str = ".".join(field_rule_dict["path"])
|
||||
missing.append(f"{path_str}[{i}].{field_path_str}")
|
||||
|
||||
return missing, total_checks, passed_checks
|
||||
|
||||
|
||||
def _validate_with_rules(data: dict[str, Any], rules: ValidationRules) -> tuple[list[str], int, int]:
|
||||
"""使用规则配置进行校验。"""
|
||||
missing: list[str] = []
|
||||
total_checks = 0
|
||||
passed_checks = 0
|
||||
|
||||
# 校验字段规则
|
||||
for rule in rules.get("fields", []):
|
||||
total_checks += 1
|
||||
ok, field_path = _check_field(data, rule)
|
||||
if ok:
|
||||
passed_checks += 1
|
||||
else:
|
||||
missing.append(field_path)
|
||||
|
||||
# 校验数组规则
|
||||
for rule in rules.get("arrays", []):
|
||||
array_missing, array_total, array_passed = _check_array(data, rule)
|
||||
total_checks += array_total
|
||||
passed_checks += array_passed
|
||||
missing.extend(array_missing)
|
||||
|
||||
return missing, total_checks, passed_checks
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 校验入口
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def validate_travel_info(data: dict[str, Any]) -> ValidationReport:
|
||||
"""校验差旅报销信息的完整性。"""
|
||||
rules = get_validation_rules("travel")
|
||||
missing, total_checks, passed_checks = _validate_with_rules(data, rules)
|
||||
|
||||
confidence = passed_checks / total_checks if total_checks > 0 else 0.0
|
||||
missing_materials = _infer_missing_materials(missing, data)
|
||||
suggestion = _build_suggestion(missing, missing_materials)
|
||||
|
||||
return ValidationReport(
|
||||
valid=len(missing) == 0,
|
||||
missing_fields=missing,
|
||||
missing_materials=missing_materials,
|
||||
confidence=round(confidence, 2),
|
||||
suggestion=suggestion,
|
||||
)
|
||||
|
||||
|
||||
def validate_normal_info(data: dict[str, Any]) -> ValidationReport:
|
||||
"""校验普通报销信息的完整性。"""
|
||||
rules = get_validation_rules("normal")
|
||||
missing, total_checks, passed_checks = _validate_with_rules(data, rules)
|
||||
|
||||
confidence = passed_checks / total_checks if total_checks > 0 else 0.0
|
||||
missing_materials = _infer_missing_materials(missing, data)
|
||||
suggestion = _build_suggestion(missing, missing_materials)
|
||||
|
||||
return ValidationReport(
|
||||
valid=len(missing) == 0,
|
||||
missing_fields=missing,
|
||||
missing_materials=missing_materials,
|
||||
confidence=round(confidence, 2),
|
||||
suggestion=suggestion,
|
||||
)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 缺失材料推断
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _infer_missing_materials(
|
||||
missing_fields: list[str],
|
||||
data: dict[str, Any],
|
||||
) -> list[str]:
|
||||
"""根据缺失字段推断可能需要补充的材料类型。"""
|
||||
materials: list[str] = []
|
||||
field_set = set(missing_fields)
|
||||
|
||||
if any("start_date" in f or "end_date" in f for f in field_set):
|
||||
if "basic_info.start_date" in field_set or "basic_info.end_date" in field_set:
|
||||
materials.append("出差事前申请单")
|
||||
|
||||
if "basic_info.travel_purpose" in field_set:
|
||||
materials.append("出差事前申请单")
|
||||
|
||||
if "basic_info.travel_location" in field_set:
|
||||
materials.append("交通工具发票")
|
||||
|
||||
if "payment_methods" in field_set or any("payment_methods[" in f for f in field_set):
|
||||
materials.append("支付记录截图")
|
||||
|
||||
if any("transport_fee" in f for f in field_set):
|
||||
materials.append("交通工具发票")
|
||||
|
||||
if any("subsidy_list" in f for f in field_set):
|
||||
materials.append("出差事前申请单")
|
||||
|
||||
if "basic_info.reimbursement_description" in field_set:
|
||||
materials.append("发票或支付记录")
|
||||
|
||||
return list(dict.fromkeys(materials))
|
||||
|
||||
|
||||
def _build_suggestion(
|
||||
missing_fields: list[str],
|
||||
missing_materials: list[str],
|
||||
) -> str:
|
||||
"""生成用户友好的建议信息。"""
|
||||
if not missing_fields:
|
||||
return ""
|
||||
|
||||
if missing_materials:
|
||||
material_names = "、".join(missing_materials)
|
||||
return f"信息不完整,请补充上传:{material_names}"
|
||||
|
||||
return f"信息不完整,缺少 {len(missing_fields)} 个字段"
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 统一入口
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def validate_extracted_info(
|
||||
data: dict[str, Any],
|
||||
invoice_type: str = "travel",
|
||||
) -> ValidationReport:
|
||||
"""校验提取信息的完整性。
|
||||
|
||||
Args:
|
||||
data: LLM 提取的结构化信息。
|
||||
invoice_type: 发票类型,'travel' 或 'normal'。
|
||||
|
||||
Returns:
|
||||
校验报告。
|
||||
"""
|
||||
log.info("开始校验 %s 报销信息完整性", invoice_type)
|
||||
|
||||
if invoice_type == "travel":
|
||||
report = validate_travel_info(data)
|
||||
else:
|
||||
report = validate_normal_info(data)
|
||||
|
||||
status = "通过" if report.valid else "未通过"
|
||||
log.info(
|
||||
"校验结果: %s (置信度: %.0f%%, 缺失字段: %d)",
|
||||
status,
|
||||
report.confidence * 100,
|
||||
len(report.missing_fields),
|
||||
)
|
||||
|
||||
return report
|
||||
|
||||
|
||||
def reload_validation_rules() -> None:
|
||||
"""重新加载校验规则配置(用于运行时热更新)。"""
|
||||
global _cached_rules
|
||||
_cached_rules = None
|
||||
_load_validation_config()
|
||||
log.info("校验规则已重新加载")
|
||||
58
src/exceptions.py
Normal file
58
src/exceptions.py
Normal file
@@ -0,0 +1,58 @@
|
||||
"""项目级异常定义
|
||||
|
||||
异常层次:
|
||||
ReimbursementError — 所有业务异常的基类
|
||||
├── ExtractionError — 文档提取失败(单个文件/批量全失败)
|
||||
├── BrowserError — 浏览器自动化失败
|
||||
└── ValidationError — 校验失败(规则校验/语义校验)
|
||||
|
||||
使用规则:
|
||||
- 模块内部: 捕获具体异常 → 记录日志 → 截图(如适用) → re-raise
|
||||
- 模块边界: 不吞异常,向上传播
|
||||
- 顶层 (routes.py / orchestrator.py): 统一捕获 ReimbursementError
|
||||
- 可恢复场景: 返回结构化结果而非 raise
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
|
||||
class ReimbursementError(Exception):
|
||||
"""业务异常基类"""
|
||||
|
||||
def __init__(self, message: str, details: dict[str, Any] | None = None) -> None:
|
||||
super().__init__(message)
|
||||
self.details = details or {}
|
||||
|
||||
|
||||
class ExtractionError(ReimbursementError):
|
||||
"""文档提取失败"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str,
|
||||
failed_files: list[str] | None = None,
|
||||
details: dict[str, Any] | None = None,
|
||||
) -> None:
|
||||
super().__init__(message, details)
|
||||
self.failed_files = failed_files or []
|
||||
|
||||
|
||||
class BrowserError(ReimbursementError):
|
||||
"""浏览器自动化操作失败"""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
class ValidationError(ReimbursementError):
|
||||
"""校验失败"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str,
|
||||
missing_fields: list[str] | None = None,
|
||||
details: dict[str, Any] | None = None,
|
||||
) -> None:
|
||||
super().__init__(message, details)
|
||||
self.missing_fields = missing_fields or []
|
||||
21
src/infra/README.md
Normal file
21
src/infra/README.md
Normal file
@@ -0,0 +1,21 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/infra — 基础设施层
|
||||
|
||||
提供浏览器自动化、文档处理和 LLM 接口等底层能力。此层不包含业务逻辑,只提供工具和平台能力。
|
||||
|
||||
## 子模块
|
||||
|
||||
| 目录 | 说明 |
|
||||
|------|------|
|
||||
| `browser/` | Playwright 驱动的财务系统自动填报 |
|
||||
| `documents/` | 发票数据模型、PDF 渲染、Word 出库单填写 |
|
||||
| `llm/` | LLM 提示词模板加载与管理 |
|
||||
|
||||
## 设计原则
|
||||
|
||||
- **无业务逻辑**:只提供工具能力,不包含业务流程判断
|
||||
- **可替换性**:每个子模块通过 `__init__.py` 导出接口,便于替换实现
|
||||
- **与 core 层解耦**:infra 不依赖 core,core 可通过接口调用 infra
|
||||
4
src/infra/__init__.py
Normal file
4
src/infra/__init__.py
Normal file
@@ -0,0 +1,4 @@
|
||||
"""基础设施模块
|
||||
|
||||
提供浏览器自动化、文档处理和 LLM 接口功能。
|
||||
"""
|
||||
44
src/infra/browser/README.md
Normal file
44
src/infra/browser/README.md
Normal file
@@ -0,0 +1,44 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/infra/browser — 浏览器自动化
|
||||
|
||||
使用 Playwright 操作财务报销系统,自动完成登录、填单、上传附件等操作。
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `base.py` | `BaseBot` 基类:浏览器生命周期、登录信息门户、导航到报销系统、创建新单据、截图 |
|
||||
| `travel.py` | 差旅报销填报流程:基本信息 → 差旅明细 → 支付方式 → 补助清单 → 附件上传 |
|
||||
| `normal.py` | 普通报销填报流程:基本信息 → 总明细 → 支付方式 → 附件上传 |
|
||||
| `__init__.py` | 入口函数:`run_bot()` / `run_bot_web()`,负责类型路由和流程调度 |
|
||||
|
||||
## 对外接口
|
||||
|
||||
| 函数 | 说明 |
|
||||
|------|------|
|
||||
| `run_bot(config, travel_info, normal_info)` | CLI 模式:根据传入信息判断差旅/普通报销 |
|
||||
| `run_bot_web(config, work_dir)` | Web 模式:从缓存加载信息后执行填报 |
|
||||
|
||||
## 填报流程
|
||||
|
||||
### 差旅报销(travel)
|
||||
1. 填写基本信息(事由、地点、日期、项目编号)
|
||||
2. 添加差旅明细(交通费用逐条录入)
|
||||
3. 填写支付方式(公务卡刷卡记录)
|
||||
4. 填写补助清单(按天计算交通补助 + 伙食补助)
|
||||
5. 上传附件(发票、申请单等)
|
||||
|
||||
### 普通报销(normal)
|
||||
1. 填写基本信息(报销事由、金额)
|
||||
2. 填写发票明细(总数、总金额)
|
||||
3. 填写支付方式
|
||||
4. 上传附件
|
||||
|
||||
## 注意事项
|
||||
|
||||
- 浏览器填报会启动 Chromium,请勿手动干扰自动化流程
|
||||
- 调试截图保存在 `images/` 目录
|
||||
- Web 模式以无头模式运行
|
||||
100
src/infra/browser/__init__.py
Normal file
100
src/infra/browser/__init__.py
Normal file
@@ -0,0 +1,100 @@
|
||||
"""浏览器自动化填报
|
||||
|
||||
使用 Playwright 操作财务报销系统,自动完成登录、填单、上传附件等操作。
|
||||
|
||||
对外接口:
|
||||
run_bot(config, travel_info, normal_info) 启动浏览器并执行填报流程
|
||||
run_bot_web(config, work_dir) Web 模式填报(从缓存加载信息)
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from ... import get_logger
|
||||
from .base import BaseBot
|
||||
|
||||
log = get_logger("bot")
|
||||
|
||||
|
||||
def run_bot(
|
||||
config: dict[str, Any],
|
||||
headless: bool = False,
|
||||
work_dir: Path | None = None,
|
||||
travel_info: dict[str, Any] | None = None,
|
||||
normal_info: dict[str, Any] | None = None,
|
||||
) -> None:
|
||||
"""启动浏览器并执行填报流程。
|
||||
|
||||
根据传入的报销信息判断执行差旅报销还是普通报销流程。
|
||||
|
||||
Args:
|
||||
config: 财务系统配置(含 URL、账号密码等)。
|
||||
headless: 是否无头模式。
|
||||
work_dir: 工作目录。
|
||||
travel_info: 差旅报销信息(可选)。
|
||||
normal_info: 普通报销信息(可选)。
|
||||
"""
|
||||
invoice_type = ""
|
||||
if travel_info and normal_info:
|
||||
invoice_type = "mixed"
|
||||
elif travel_info:
|
||||
invoice_type = "travel"
|
||||
elif normal_info:
|
||||
invoice_type = "normal"
|
||||
else:
|
||||
log.error("未提供任何报销信息")
|
||||
return
|
||||
|
||||
log.info(f"启动填报流程: {invoice_type}")
|
||||
|
||||
if invoice_type == "travel":
|
||||
from .travel import run as run_travel
|
||||
|
||||
bot = BaseBot(config, headless=headless)
|
||||
bot.work_dir = work_dir
|
||||
|
||||
try:
|
||||
bot.launch()
|
||||
bot.login_portal()
|
||||
bot.navigate_to_reimburse(page_key="travel_page")
|
||||
bot.create_new_form()
|
||||
run_travel(bot, travel_info)
|
||||
finally:
|
||||
bot.close()
|
||||
elif invoice_type == "normal":
|
||||
from .normal import run as run_normal
|
||||
|
||||
bot = BaseBot(config, headless=headless)
|
||||
bot.work_dir = work_dir
|
||||
|
||||
try:
|
||||
bot.launch()
|
||||
bot.login_portal()
|
||||
bot.navigate_to_reimburse(page_key="reimburse_page")
|
||||
bot.create_new_form()
|
||||
run_normal(bot, normal_info)
|
||||
finally:
|
||||
bot.close()
|
||||
else:
|
||||
log.warning("暂不支持混合报销流程")
|
||||
|
||||
|
||||
def run_bot_web(config: dict[str, Any], work_dir: str | Path) -> None:
|
||||
"""Web 模式填报(从缓存加载信息)。
|
||||
|
||||
根据 work_dir 下的 .invoice_cache 目录中已提取的信息,
|
||||
自动判断执行差旅报销还是普通报销流程。
|
||||
|
||||
Args:
|
||||
config: 财务系统配置(含 URL、账号密码等)。
|
||||
work_dir: 工作目录(包含 .invoice_cache 子目录)。
|
||||
"""
|
||||
from ...core.extraction import load_cache
|
||||
|
||||
work_dir = Path(work_dir)
|
||||
cache = load_cache(work_dir)
|
||||
|
||||
travel_info = cache.get("travel_info")
|
||||
normal_info = cache.get("normal_info")
|
||||
|
||||
run_bot(config, headless=True, work_dir=work_dir, travel_info=travel_info, normal_info=normal_info)
|
||||
203
src/infra/browser/base.py
Normal file
203
src/infra/browser/base.py
Normal file
@@ -0,0 +1,203 @@
|
||||
"""浏览器自动化填报 — 公共基类
|
||||
|
||||
提供浏览器生命周期管理、登录、导航、截图等公共操作。
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from ... import get_logger
|
||||
|
||||
log = get_logger("bot")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 工具函数
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def format_date(date_str: str) -> str:
|
||||
"""将 '2026/5/13' 或 '2026-5-13' 转为 '2026-05-13'"""
|
||||
if not date_str:
|
||||
return ""
|
||||
parts = date_str.replace("-", "/").split("/")
|
||||
if len(parts) == 3:
|
||||
return f"{parts[0].zfill(4)}-{parts[1].zfill(2)}-{parts[2].zfill(2)}"
|
||||
return date_str
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 基类
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
class BaseBot:
|
||||
"""浏览器自动化基类 — 管理浏览器生命周期与公共操作"""
|
||||
|
||||
def __init__(self, config: dict[str, Any], headless: bool = False) -> None:
|
||||
self.config = config
|
||||
self.headless = headless
|
||||
self.work_dir: Path | None = None
|
||||
self.browser: Any = None
|
||||
self.context: Any = None
|
||||
self.page: Any = None
|
||||
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
self._pw_ctx = sync_playwright()
|
||||
self.pw = self._pw_ctx.__enter__()
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# 浏览器生命周期
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
def launch(self) -> None:
|
||||
"""启动浏览器"""
|
||||
self.browser = self.pw.chromium.launch(headless=self.headless)
|
||||
self.context = self.browser.new_context(viewport={"width": 1360, "height": 768})
|
||||
self.page = self.context.new_page()
|
||||
self.page.set_default_timeout(30000)
|
||||
|
||||
def close(self) -> None:
|
||||
"""关闭浏览器"""
|
||||
if self.context:
|
||||
self.context.close()
|
||||
if self.browser:
|
||||
self.browser.close()
|
||||
try:
|
||||
self._pw_ctx.__exit__(None, None, None)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# 登录
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
def login_portal(self) -> None:
|
||||
"""登录信息门户"""
|
||||
log.info("登录信息门户...")
|
||||
|
||||
self.page.goto(self.config["sso_login_url"], wait_until="domcontentloaded")
|
||||
self._wait_for('text="微信扫码登录"', timeout=5000)
|
||||
|
||||
try:
|
||||
self.page.fill(
|
||||
'input[placeholder*="工号"], input[placeholder*="学号"]',
|
||||
self.config["username"],
|
||||
)
|
||||
self.page.fill('input[placeholder*="密码"]', self.config["password"])
|
||||
except Exception:
|
||||
log.warning("未找到登录输入框,可能已登录")
|
||||
|
||||
try:
|
||||
checkbox = self.page.query_selector('input[type="checkbox"]')
|
||||
if checkbox and not checkbox.is_checked():
|
||||
checkbox.click()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
for selector in ['button:has-text("登录")', 'input[value="登录"]', 'text="登录"']:
|
||||
try:
|
||||
self.page.click(selector, timeout=3000)
|
||||
break
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
self._wait_for_portal()
|
||||
|
||||
def _wait_for_portal(self) -> None:
|
||||
"""等待跳转到统一信息平台"""
|
||||
for _ in range(30):
|
||||
self.page.wait_for_timeout(1000)
|
||||
url = self.page.url
|
||||
if any(
|
||||
kw in url
|
||||
for kw in (
|
||||
"tyrz.fynu.edu.cn/zs-uip",
|
||||
"tyrz.fynu.edu.cn/oshall",
|
||||
"portal",
|
||||
)
|
||||
):
|
||||
self._screenshot("portal_loaded")
|
||||
return
|
||||
log.error("等待门户跳转超时")
|
||||
self._screenshot("portal_timeout")
|
||||
raise TimeoutError("登录超时,未跳转到信息门户")
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# 导航
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
def navigate_to_reimburse(self, page_key: str = "reimburse_page") -> None:
|
||||
"""从统一信息平台进入报销系统"""
|
||||
log.info("进入报销系统...")
|
||||
self._wait_for('text="快捷入口"', timeout=5000)
|
||||
|
||||
try:
|
||||
self.page.click('text="财务系统"', timeout=5000)
|
||||
except Exception:
|
||||
log.warning("未找到财务系统入口")
|
||||
|
||||
new_tab = None
|
||||
for _ in range(15):
|
||||
self.page.wait_for_timeout(1000)
|
||||
for p in self.context.pages:
|
||||
if "dddl" in p.url or "210.45.32.214" in p.url:
|
||||
new_tab = p
|
||||
break
|
||||
if new_tab:
|
||||
break
|
||||
|
||||
if new_tab:
|
||||
self.page = new_tab
|
||||
self._wait_for('text="网络报销"', timeout=5000)
|
||||
else:
|
||||
log.warning(f"未找到单点登录页面,当前 URL: {self.page.url}")
|
||||
|
||||
for p in self.context.pages[:-1]:
|
||||
try:
|
||||
p.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
link = self.page.query_selector('a:has(img[src*="wlbx"])')
|
||||
if link:
|
||||
reimburse_url = link.get_attribute("href")
|
||||
self.page.goto(reimburse_url, wait_until="domcontentloaded", timeout=15000)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
self._wait_for('text="报销录入"', timeout=5000)
|
||||
common_url = self.config["reimburse_url"] + self.config[page_key]
|
||||
self.page.goto(common_url, wait_until="domcontentloaded", timeout=15000)
|
||||
self._wait_for('text="单据状态:"', timeout=5000)
|
||||
|
||||
def create_new_form(self) -> None:
|
||||
"""点击「新增」创建新单据"""
|
||||
log.info("创建新单据...")
|
||||
self.page.wait_for_timeout(2000)
|
||||
|
||||
try:
|
||||
self.page.click("#insert", timeout=5000)
|
||||
except Exception:
|
||||
try:
|
||||
self.page.click("text=新增", timeout=3000)
|
||||
except Exception as err:
|
||||
self._screenshot("no_add_button")
|
||||
raise RuntimeError("无法点击新增按钮") from err
|
||||
|
||||
self.page.wait_for_timeout(3000)
|
||||
self._screenshot("after_add_click")
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# 辅助方法
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
def _wait_for(self, selector: str, timeout: int | None = None) -> None:
|
||||
self.page.wait_for_selector(selector, timeout=timeout)
|
||||
|
||||
def _screenshot(self, name: str) -> None:
|
||||
img_dir = Path(__file__).parent.parent.parent / "images"
|
||||
img_dir.mkdir(exist_ok=True)
|
||||
self.page.screenshot(path=str(img_dir / f"debug_{name}.png"))
|
||||
179
src/infra/browser/normal.py
Normal file
179
src/infra/browser/normal.py
Normal file
@@ -0,0 +1,179 @@
|
||||
"""普通报销填报流程
|
||||
|
||||
负责普通发票报销的完整填报步骤:
|
||||
基本信息 → 总明细 → 支付方式 → 附件上传
|
||||
"""
|
||||
|
||||
from typing import Any
|
||||
|
||||
from ... import get_logger
|
||||
from .base import BaseBot, format_date
|
||||
|
||||
log = get_logger("bot.normal")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 普通报销流程
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def run(
|
||||
bot: BaseBot,
|
||||
normal_info: dict[str, Any],
|
||||
) -> None:
|
||||
"""执行普通发票报销填报流程
|
||||
|
||||
Args:
|
||||
bot: 已启动并登录的 BaseBot 实例。
|
||||
normal_info: 普通发票信息字典。
|
||||
"""
|
||||
log.info("开始普通发票报销填报...")
|
||||
|
||||
log.info("填写基本信息...")
|
||||
description = normal_info.get("basic_info", {}).get("reimbursement_description", "元器件采购报销")
|
||||
fill_basic_info(bot, description)
|
||||
|
||||
total_invoices = normal_info.get("reimbursement_details", {}).get("total_invoices", 0)
|
||||
total_amount = normal_info.get("reimbursement_details", {}).get("total_amount", 0)
|
||||
log.info(f"录入普通发票总明细 (共 {total_invoices} 张, 合计 ¥{total_amount:.2f})...")
|
||||
add_normal_item(bot, total_invoices, total_amount)
|
||||
|
||||
payment_info = normal_info.get("payment_methods", [])
|
||||
log.info(f"录入普通发票支付信息 (共 {len(payment_info)} 笔)...")
|
||||
fill_normal_payment(bot, payment_info)
|
||||
|
||||
log.info("上传普通发票附件...")
|
||||
attachment_info = normal_info.get("attachments", [])
|
||||
upload_normal_attachments(bot, attachment_info)
|
||||
|
||||
log.info("普通发票报销填报完成")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 基本信息
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def fill_basic_info(bot: BaseBot, description: str = "元器件采购报销") -> None:
|
||||
"""填写基本信息"""
|
||||
try:
|
||||
bot.page.fill("#EXPENEXPLAIN", description)
|
||||
bot.page.click("#PROJECTCODE", timeout=10000)
|
||||
bot.page.wait_for_timeout(1000)
|
||||
|
||||
bot.page.wait_for_selector("#promodal .fixed-table-body tbody tr", timeout=10000)
|
||||
first_row = bot.page.query_selector("#promodal .fixed-table-body tbody tr")
|
||||
|
||||
if first_row:
|
||||
first_row.click()
|
||||
bot.page.wait_for_timeout(1000)
|
||||
|
||||
bot.page.click("#saveAndNext", timeout=5000)
|
||||
bot.page.wait_for_timeout(2000)
|
||||
|
||||
bot._screenshot("step3_done")
|
||||
except Exception as e:
|
||||
log.error(f"填写基本信息失败: {e}")
|
||||
bot._screenshot("basic_info_error")
|
||||
raise
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 总明细
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def add_normal_item(bot: BaseBot, total_invoices: int, total_amount: float) -> None:
|
||||
"""录入普通发票总明细"""
|
||||
try:
|
||||
bot.page.click("#insertDetail", timeout=5000)
|
||||
bot._wait_for('text="经济事项名称"', timeout=5000)
|
||||
bot.page.click("#economicscode2")
|
||||
bot.page.wait_for_timeout(1000)
|
||||
|
||||
try:
|
||||
bot.page.wait_for_selector("#econmodal .fixed-table-body tbody tr", timeout=10000)
|
||||
rows = bot.page.query_selector_all("#econmodal .fixed-table-body tbody tr")
|
||||
if len(rows) >= 3:
|
||||
rows[2].click()
|
||||
bot.page.wait_for_timeout(1000)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
bot.page.fill('input[name="expenPwCommondetail.HOWBILLS"]', str(total_invoices))
|
||||
bot.page.fill("#je_zwzcdz", f"{total_amount:.2f}")
|
||||
bot.page.click("#detailAdd", timeout=3000)
|
||||
bot.page.wait_for_timeout(1000)
|
||||
|
||||
bot._screenshot("normal_item_done")
|
||||
except Exception as e:
|
||||
log.error(f"录入总明细失败: {e}")
|
||||
bot._screenshot("normal_item_error")
|
||||
raise
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 支付方式
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def fill_normal_payment(bot: BaseBot, payment_info: list[dict[str, Any]]) -> None:
|
||||
"""录入普通发票支付信息"""
|
||||
try:
|
||||
bot.page.click('text="下一步(支付方式)"', timeout=5000)
|
||||
bot._wait_for('text="下一步(附件清单)"', timeout=5000)
|
||||
|
||||
for info in payment_info:
|
||||
bot.page.click("#insertPay", timeout=5000)
|
||||
bot.page.wait_for_timeout(1000)
|
||||
bot.page.fill("#personid2", bot.config["default_person_id"])
|
||||
bot.page.fill("#accountname2", bot.config["default_name"])
|
||||
bot.page.fill("#receiptdate2", format_date(str(info.get("card_date", ""))))
|
||||
bot.page.fill("#localaccount2", bot.config["default_card_no"])
|
||||
bot.page.fill("#receiptmoney2", str(info.get("card_amount", 0)))
|
||||
bot.page.fill("#money2", str(info.get("card_amount", 0)))
|
||||
bot.page.fill("#merchant2", str(info.get("merchant", "")))
|
||||
bot.page.fill("#smark2", str(info.get("remark", "")))
|
||||
bot.page.click("#payAdd", timeout=3000)
|
||||
bot.page.wait_for_timeout(1000)
|
||||
except Exception as e:
|
||||
log.error(f"支付方式录入失败: {e}")
|
||||
bot._screenshot("normal_payment_error")
|
||||
raise
|
||||
|
||||
bot._screenshot("normal_payment_done")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 附件上传
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def upload_normal_attachments(bot: BaseBot, attachment_info: list[dict[str, Any]]) -> None:
|
||||
"""上传普通发票附件"""
|
||||
log.info(f"上传普通发票附件 (共 {len(attachment_info)} 个)...")
|
||||
try:
|
||||
bot.page.click("#next3", timeout=5000)
|
||||
bot._wait_for("#submit2", timeout=5000)
|
||||
|
||||
for info in attachment_info:
|
||||
attachment_file = bot.work_dir / info["filename"]
|
||||
bot._wait_for("#insertAcc", timeout=20000)
|
||||
bot.page.click("#insertAcc", timeout=5000)
|
||||
bot._wait_for("#fjlx", timeout=5000)
|
||||
if info.get("attachment_type") == "invoice":
|
||||
bot.page.select_option("#fjlx", "1")
|
||||
else:
|
||||
bot.page.select_option("#fjlx", "2")
|
||||
bot.page.fill("#fpsmxx", info.get("attachment_desc", ""))
|
||||
if attachment_file and attachment_file.exists():
|
||||
bot.page.set_input_files("#file", str(attachment_file))
|
||||
bot.page.wait_for_timeout(1000)
|
||||
bot.page.click("#cjtj", timeout=5000)
|
||||
log.info("上传普通发票附件完成")
|
||||
except Exception as e:
|
||||
log.error(f"附件上传失败: {e}")
|
||||
bot._screenshot("normal_attachment_error")
|
||||
raise
|
||||
|
||||
bot._screenshot("normal_attachment_done")
|
||||
307
src/infra/browser/travel.py
Normal file
307
src/infra/browser/travel.py
Normal file
@@ -0,0 +1,307 @@
|
||||
"""差旅报销填报流程
|
||||
|
||||
负责差旅报销的完整填报步骤:
|
||||
基本信息 → 差旅明细 → 支付方式 → 补助清单 → 附件上传
|
||||
"""
|
||||
|
||||
from typing import Any
|
||||
|
||||
from ... import get_logger
|
||||
from .base import BaseBot, format_date
|
||||
|
||||
log = get_logger("bot.travel")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 差旅填报流程
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def run(bot: BaseBot, travel_info: dict[str, Any]) -> None:
|
||||
"""执行差旅报销填报流程
|
||||
|
||||
Args:
|
||||
bot: 已启动并登录的 BaseBot 实例。
|
||||
travel_info: 差旅信息字典(包含 basic_info、reimbursement_details 等)。
|
||||
"""
|
||||
log.info("开始差旅报销填报...")
|
||||
|
||||
log.info("填写差旅报销信息...")
|
||||
basic_info = travel_info["basic_info"]
|
||||
fill_travel_info(bot, basic_info)
|
||||
|
||||
log.info("填写差旅报销明细...")
|
||||
details = travel_info["reimbursement_details"]
|
||||
add_travel_items(bot, details)
|
||||
|
||||
log.info("填写差旅报销支付方式...")
|
||||
payment_info = travel_info["payment_methods"]
|
||||
fill_travel_payment(bot, payment_info)
|
||||
|
||||
log.info("填写差旅报销补助清单...")
|
||||
subsidy_info = travel_info["subsidy_list"]
|
||||
fill_travel_subsidy(bot, subsidy_info)
|
||||
|
||||
log.info("上传差旅报销附件...")
|
||||
attachment_info = travel_info["attachments"]
|
||||
upload_travel_attachments(bot, attachment_info)
|
||||
|
||||
log.info("差旅报销填报完成")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 基本信息
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def fill_travel_info(bot: BaseBot, basic_info: dict[str, Any]) -> None:
|
||||
"""填写差旅报销基本信息"""
|
||||
try:
|
||||
bot.page.fill("#CAUSE", basic_info.get("travel_purpose", ""))
|
||||
bot.page.fill("#SITE", basic_info.get("travel_location", ""))
|
||||
|
||||
bot.page.click("#PROJECTCODE", timeout=10000)
|
||||
bot.page.wait_for_timeout(1000)
|
||||
bot.page.wait_for_selector("#promodal .fixed-table-body tbody tr", timeout=10000)
|
||||
first_row = bot.page.query_selector("#promodal .fixed-table-body tbody tr")
|
||||
if first_row:
|
||||
first_row.click()
|
||||
bot.page.wait_for_timeout(1000)
|
||||
|
||||
bot.page.fill("#THEKSRQ", format_date(basic_info.get("start_date", "")))
|
||||
bot.page.fill("#THEJSRQ", format_date(basic_info.get("end_date", "")))
|
||||
bot.page.click("#saveAndNext", timeout=5000)
|
||||
bot.page.wait_for_timeout(2000)
|
||||
bot._screenshot("travel_basic_done")
|
||||
except Exception as e:
|
||||
log.error(f"填写基本信息失败: {e}")
|
||||
bot._screenshot("travel_basic_error")
|
||||
raise
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 差旅明细
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def add_travel_items(bot: BaseBot, details: dict[str, Any]) -> None:
|
||||
"""录入差旅报销明细
|
||||
|
||||
Args:
|
||||
bot: 已启动的 BaseBot 实例。
|
||||
details: 报销明细字典(travel_info["reimbursement_details"]),包含
|
||||
transport_fee、hotel_fee、conference_fee 等子字段。
|
||||
"""
|
||||
vehicle_map = {
|
||||
"火车": "01",
|
||||
"汽车": "02",
|
||||
"轮船": "03",
|
||||
"自带车": "04",
|
||||
"公务车": "05",
|
||||
"飞机": "06",
|
||||
"租车": "07",
|
||||
"自驾车": "08",
|
||||
}
|
||||
|
||||
try:
|
||||
traffic_info = details.get("transport_fee") or []
|
||||
for item in traffic_info:
|
||||
bot.page.click("#insertDetail", timeout=5000)
|
||||
bot._wait_for('text="增加明细"', timeout=5000)
|
||||
bot.page.select_option("#cost", "1")
|
||||
bot.page.wait_for_timeout(500)
|
||||
vehicle = item.get("vehicle_type", "")
|
||||
if vehicle in vehicle_map:
|
||||
bot.page.select_option("#jtgj", vehicle_map[vehicle])
|
||||
|
||||
bot.page.fill("#ksdd", item.get("departure_place", ""))
|
||||
bot.page.fill("#jsdd", item.get("arrival_place", ""))
|
||||
bot.page.fill(
|
||||
'#t1 input[name="expenPwTraveldetail.MONEY"]',
|
||||
str(item.get("amount", "")),
|
||||
)
|
||||
bot.page.fill(
|
||||
'#t1 input[name="expenPwTraveldetail.HOWBILL"]',
|
||||
str(item.get("bill_count", "")),
|
||||
)
|
||||
bot.page.fill(
|
||||
'#t1 input[name="expenPwTraveldetail.SMARK"]',
|
||||
str(item.get("remark", "")),
|
||||
)
|
||||
bot.page.click("#detailAdd", timeout=3000)
|
||||
bot.page.wait_for_timeout(1000)
|
||||
|
||||
hotel_info = details.get("hotel_fee") or []
|
||||
for item in hotel_info:
|
||||
bot.page.click("#insertDetail", timeout=5000)
|
||||
bot._wait_for('text="增加明细"', timeout=5000)
|
||||
bot.page.select_option("#cost", "2")
|
||||
bot.page.wait_for_timeout(500)
|
||||
bot.page.fill("#ksrq2", format_date(str(item.get("checkin_date", ""))))
|
||||
bot.page.fill("#jsrq2", format_date(str(item.get("checkout_date", ""))))
|
||||
bot.page.fill("#ts2", str(item.get("days", "")))
|
||||
bot.page.fill("#rs2", str(item.get("person_count", "")))
|
||||
bot.page.fill(
|
||||
'#t2 input[name="expenPwTraveldetail.FPMONEY"]',
|
||||
str(item.get("invoice_amount", "")),
|
||||
)
|
||||
bot.page.fill(
|
||||
'#t2 input[name="expenPwTraveldetail.MONEY"]',
|
||||
str(item.get("reimburse_amount", "")),
|
||||
)
|
||||
bot.page.fill(
|
||||
'#t2 input[name="expenPwTraveldetail.SMARK"]',
|
||||
str(item.get("remark", "")),
|
||||
)
|
||||
bot.page.click("#detailAdd", timeout=3000)
|
||||
bot.page.wait_for_timeout(1000)
|
||||
|
||||
conference_info = details.get("conference_fee") or []
|
||||
for item in conference_info:
|
||||
bot.page.click("#insertDetail", timeout=5000)
|
||||
bot._wait_for('text="增加明细"', timeout=5000)
|
||||
bot.page.select_option("#cost", "3")
|
||||
bot.page.wait_for_timeout(500)
|
||||
bot.page.fill(
|
||||
'#t3 input[name="expenPwTraveldetail.HOWBILL"]',
|
||||
str(item.get("bill_count", "")),
|
||||
)
|
||||
bot.page.fill(
|
||||
'#t3 input[name="expenPwTraveldetail.MONEY"]',
|
||||
str(item.get("amount", "")),
|
||||
)
|
||||
bot.page.fill(
|
||||
'#t3 input[name="expenPwTraveldetail.SMARK"]',
|
||||
str(item.get("remark", "")),
|
||||
)
|
||||
bot.page.click("#detailAdd", timeout=3000)
|
||||
bot.page.wait_for_timeout(1000)
|
||||
except Exception as e:
|
||||
log.error(f"录入总明细失败: {e}")
|
||||
bot._screenshot("item_total_error")
|
||||
raise
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 支付方式
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def fill_travel_payment(bot: BaseBot, payment_info: list[dict[str, Any]]) -> None:
|
||||
"""录入差旅支付信息"""
|
||||
try:
|
||||
bot.page.click('text="下一步(支付方式)"', timeout=5000)
|
||||
bot._wait_for('text="下一步(补助清单)"', timeout=5000)
|
||||
|
||||
for info in payment_info:
|
||||
bot.page.click("#insertPay", timeout=5000)
|
||||
bot.page.wait_for_timeout(1000)
|
||||
bot.page.fill("#personid2", bot.config["default_person_id"])
|
||||
bot.page.fill("#accountname2", bot.config["default_name"])
|
||||
bot.page.fill("#receiptdate2", format_date(info["card_date"]))
|
||||
bot.page.fill("#localaccount2", bot.config["default_card_no"])
|
||||
bot.page.fill("#receiptmoney2", str(info["card_amount"]))
|
||||
bot.page.fill("#money2", str(info["card_amount"]))
|
||||
bot.page.fill("#merchant2", info.get("merchant", ""))
|
||||
bot.page.fill("#smark2", info.get("remark", ""))
|
||||
bot.page.click("#payAdd", timeout=3000)
|
||||
bot.page.wait_for_timeout(1000)
|
||||
except Exception as e:
|
||||
log.error(f"支付方式录入失败: {e}")
|
||||
bot._screenshot("step5_error")
|
||||
raise
|
||||
|
||||
bot._screenshot("step5_done")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 补助清单
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def fill_travel_subsidy(bot: BaseBot, subsidy_info: list[dict[str, Any]]) -> None:
|
||||
"""录入差旅补助清单"""
|
||||
try:
|
||||
bot.page.click("#next3", timeout=5000)
|
||||
bot._wait_for("#next4", timeout=5000)
|
||||
|
||||
for info in subsidy_info:
|
||||
bot.page.click("#insertSubsidy", timeout=5000)
|
||||
bot._wait_for('text="增加补助清单"', timeout=5000)
|
||||
bot.page.click("#jzg3", timeout=5000)
|
||||
bot.page.wait_for_timeout(500)
|
||||
# 注意:此处使用直接索引而非 .get(),是故意的设计。
|
||||
# LLM 必须返回 person_name 和 person_id 字段,若缺失则说明数据质量有问题,
|
||||
# 应当立即报错终止流程,而非静默跳过。
|
||||
if info["person_name"] and info["person_name"] != "":
|
||||
bot.page.fill("#seacher", info["person_name"])
|
||||
elif info["person_id"] and info["person_id"] != "":
|
||||
bot.page.fill("#seacher", info["person_id"])
|
||||
else:
|
||||
raise ValueError(f"人员编号和人员姓名不能同时为空: {info}")
|
||||
bot.page.click("#cx", timeout=5000)
|
||||
bot.page.wait_for_selector("div.fixed-table-loading", state="hidden", timeout=10000)
|
||||
bot.page.click("#tableEmp tbody tr", timeout=10000)
|
||||
bot.page.wait_for_timeout(1000)
|
||||
|
||||
open_bank = bot.page.input_value("#openbank1")
|
||||
if not open_bank:
|
||||
log.info("员工开户行未填写,默认填写中国工商银行")
|
||||
bot.page.fill("#openbank1", "中国工商银行")
|
||||
|
||||
bot.page.fill("#startdate1", format_date(info["start_date"]))
|
||||
bot.page.fill("#enddate1", format_date(info["end_date"]))
|
||||
bot.page.fill("#trafficdays1", str(info["days"]))
|
||||
bot.page.fill("#fooddays1", str(info["days"]))
|
||||
# 补助标准硬编码:交通补助 80 元/天,伙食补助 100 元/天。
|
||||
# 此为阜阳师范大学现行标准,如需适配其他单位,可改为从 config.json 读取。
|
||||
bot.page.fill("#trafficnorm1", str(80))
|
||||
bot.page.fill("#foodnorm1", str(100))
|
||||
trafficmoney = int(info["days"]) * 80
|
||||
foodmoney = int(info["days"]) * 100
|
||||
subsidymoney = trafficmoney + foodmoney
|
||||
bot.page.fill("#trafficmoney1", str(trafficmoney))
|
||||
bot.page.fill("#foodmoney1", str(foodmoney))
|
||||
bot.page.fill("#subsidymoney1", str(subsidymoney))
|
||||
bot.page.click("#add", timeout=3000)
|
||||
bot.page.wait_for_timeout(1000)
|
||||
except Exception as e:
|
||||
log.error(f"差旅补助清单录入失败: {e}")
|
||||
bot._screenshot("subsidy_error")
|
||||
raise
|
||||
|
||||
bot._screenshot("subsidy_done")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 附件上传
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def upload_travel_attachments(bot: BaseBot, attachment_info: list[dict[str, Any]]) -> None:
|
||||
"""上传差旅附件"""
|
||||
try:
|
||||
bot.page.click("#next4", timeout=5000)
|
||||
bot._wait_for("#submit2", timeout=5000)
|
||||
|
||||
for info in attachment_info:
|
||||
attachment_file = bot.work_dir / info["filename"]
|
||||
bot._wait_for("#insertAcc", timeout=20000)
|
||||
bot.page.click("#insertAcc", timeout=5000)
|
||||
bot._wait_for("#fjlx", timeout=5000)
|
||||
if info["attachment_type"] == "invoice":
|
||||
bot.page.select_option("#fjlx", "1")
|
||||
else:
|
||||
bot.page.select_option("#fjlx", "2")
|
||||
bot.page.fill("#fpsmxx", info.get("attachment_desc", ""))
|
||||
if attachment_file and attachment_file.exists():
|
||||
bot.page.set_input_files("#file", str(attachment_file))
|
||||
bot.page.wait_for_timeout(1000)
|
||||
bot.page.click("#cjtj", timeout=5000)
|
||||
log.info("上传差旅附件完成")
|
||||
except Exception as e:
|
||||
log.error(f"差旅附件上传失败: {e}")
|
||||
bot._screenshot("travel_attachment_error")
|
||||
raise
|
||||
|
||||
bot._screenshot("travel_attachment_done")
|
||||
36
src/infra/documents/README.md
Normal file
36
src/infra/documents/README.md
Normal file
@@ -0,0 +1,36 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/infra/documents — 文档处理
|
||||
|
||||
提供发票数据模型、PDF 渲染和 Word 出库单填写功能。
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `invoice.py` | 发票数据模型:类型常量、CSV 列定义、CSV/JSON 读写工具、发票分类 |
|
||||
| `pdf.py` | PDF 渲染为图片(PyMuPDF),供多模态 LLM 识别使用 |
|
||||
| `consumable.py` | 易耗品出库单填写:读取 CSV → 填入 Word 模板(pywin32 COM,仅 Windows) |
|
||||
|
||||
## 对外接口
|
||||
|
||||
| 函数 | 说明 |
|
||||
|------|------|
|
||||
| `load_csv(path)` | 读取支付记录 CSV |
|
||||
| `save_csv(payment_records, path)` | 保存支付记录 CSV |
|
||||
| `save_invoice_csv(payment_records, path)` | 保存发票级别 CSV |
|
||||
| `classify_invoice_batch(cache_map)` | 按类型批量分类发票 |
|
||||
| `render_pdf_to_images(pdf_path)` | PDF → 图片列表 |
|
||||
| `fill_consumable_doc(csv_path, doc_path)` | 将 CSV 数据填入 Word 模板 |
|
||||
|
||||
## 缓存目录
|
||||
|
||||
`.invoice_cache/` 是系统级缓存目录名常量,定义在 `invoice.py` 中,被提取和匹配模块统一引用。
|
||||
|
||||
## 易耗品出库单
|
||||
|
||||
- 需要 **Windows + Microsoft Word + pywin32**
|
||||
- 模板文件为项目根目录的 `易耗品、出库单.doc`
|
||||
- 填写规则:日期用当天日期,品名/规格/数量/单价从 CSV 解析,字体统一宋体五号
|
||||
32
src/infra/documents/__init__.py
Normal file
32
src/infra/documents/__init__.py
Normal file
@@ -0,0 +1,32 @@
|
||||
"""文档处理基础设施
|
||||
|
||||
提供发票数据模型、PDF 渲染、出库单填写等功能。
|
||||
"""
|
||||
|
||||
from .consumable import (
|
||||
CONSUMABLE_DOC_FILENAME,
|
||||
fill_consumable_doc,
|
||||
fill_consumable_from_template,
|
||||
)
|
||||
from .invoice import (
|
||||
CACHE_DIR_NAME,
|
||||
classify_invoice_batch,
|
||||
load_csv,
|
||||
load_invoice_csv,
|
||||
save_application_json,
|
||||
save_csv,
|
||||
save_invoice_csv,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"CACHE_DIR_NAME",
|
||||
"classify_invoice_batch",
|
||||
"load_csv",
|
||||
"load_invoice_csv",
|
||||
"save_csv",
|
||||
"save_invoice_csv",
|
||||
"save_application_json",
|
||||
"CONSUMABLE_DOC_FILENAME",
|
||||
"fill_consumable_doc",
|
||||
"fill_consumable_from_template",
|
||||
]
|
||||
262
src/infra/documents/consumable.py
Normal file
262
src/infra/documents/consumable.py
Normal file
@@ -0,0 +1,262 @@
|
||||
"""将 invoice_summary.csv 填入「易耗品、出库单.doc」表格。
|
||||
|
||||
仅写入表格数据单元格,保留原模板字体、边框与版式。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import re
|
||||
import shutil
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from ... import get_logger
|
||||
from ...config import load_config
|
||||
from .invoice import load_invoice_csv
|
||||
|
||||
log = get_logger("fill_consumable_doc")
|
||||
|
||||
CONSUMABLE_DOC_FILENAME = "易耗品、出库单.doc"
|
||||
|
||||
# Word COM 常量
|
||||
WD_CHARACTER = 1
|
||||
|
||||
# 表格统一字体:宋体、五号(10.5 磅)
|
||||
TABLE_FONT_NAME = "宋体"
|
||||
TABLE_FONT_SIZE = 10.5
|
||||
|
||||
|
||||
def _split_name_spec(left: str) -> tuple[str, str]:
|
||||
m = re.search(r"(\S+一批)\s*$", left)
|
||||
if m:
|
||||
return m.group(1), left[: m.start()].strip()
|
||||
parts = left.split(" ", 1)
|
||||
if len(parts) == 2:
|
||||
return parts[0], parts[1]
|
||||
return left, ""
|
||||
|
||||
|
||||
def parse_spec_model(spec: str) -> dict[str, str]:
|
||||
spec = (spec or "").strip()
|
||||
if " 个 " not in spec:
|
||||
return {
|
||||
"product_name": spec,
|
||||
"spec": "",
|
||||
"unit": "",
|
||||
"qty": "",
|
||||
"unit_price": "",
|
||||
}
|
||||
|
||||
left, right = spec.split(" 个 ", 1)
|
||||
product_name, model_spec = _split_name_spec(left.strip())
|
||||
tokens = right.split()
|
||||
|
||||
qty = ""
|
||||
unit_price = ""
|
||||
if len(tokens) >= 4 and re.fullmatch(r"\d+(?:\.\d+)?", tokens[0]):
|
||||
qty, unit_price = tokens[0], tokens[1]
|
||||
elif tokens and re.fullmatch(r"\d+(?:\.\d+)?", tokens[0]):
|
||||
qty, unit_price = "1", tokens[0]
|
||||
|
||||
return {
|
||||
"product_name": product_name,
|
||||
"spec": model_spec,
|
||||
"unit": "个",
|
||||
"qty": qty,
|
||||
"unit_price": unit_price,
|
||||
}
|
||||
|
||||
|
||||
def _format_money(value: str | float) -> str:
|
||||
"""单价、金额:固定保留两位小数。"""
|
||||
if value is None or value == "":
|
||||
return ""
|
||||
try:
|
||||
num = float(value)
|
||||
except (TypeError, ValueError):
|
||||
return str(value)
|
||||
return f"{num:.2f}"
|
||||
|
||||
|
||||
def _today_cn_date() -> str:
|
||||
"""当前日期,格式:2026年5月26日"""
|
||||
today = date.today()
|
||||
return f"{today.year}年{today.month}月{today.day}日"
|
||||
|
||||
|
||||
def _apply_font(rng: Any) -> None:
|
||||
"""将范围字体设为宋体五号(含数字与英文)。"""
|
||||
font = rng.Font
|
||||
font.Name = TABLE_FONT_NAME
|
||||
font.NameFarEast = TABLE_FONT_NAME
|
||||
font.NameAscii = TABLE_FONT_NAME
|
||||
font.NameOther = TABLE_FONT_NAME
|
||||
font.NameBi = TABLE_FONT_NAME
|
||||
font.Size = TABLE_FONT_SIZE
|
||||
|
||||
|
||||
def _set_cell_value(cell: Any, text: str) -> None:
|
||||
"""写入单元格正文(不含末尾单元格标记)。"""
|
||||
rng = cell.Range
|
||||
rng.MoveEnd(WD_CHARACTER, -1)
|
||||
rng.Text = "" if text is None else str(text)
|
||||
_apply_font(rng)
|
||||
|
||||
|
||||
def _normalize_table_font(tbl: Any) -> None:
|
||||
"""填写完成后统一整张表的字体。"""
|
||||
for row in tbl.Rows:
|
||||
for cell in row.Cells:
|
||||
rng = cell.Range
|
||||
rng.MoveEnd(WD_CHARACTER, -1)
|
||||
_apply_font(rng)
|
||||
|
||||
|
||||
def _replace_date_in_doc(doc: Any, new_date: str) -> None:
|
||||
"""仅替换表头段落中的日期文字,不改动段落其余部分。"""
|
||||
if not new_date:
|
||||
return
|
||||
try:
|
||||
para = doc.Paragraphs(3)
|
||||
except Exception:
|
||||
return
|
||||
rng = para.Range
|
||||
text = rng.Text.replace("\r", "").replace("\x07", "")
|
||||
m = re.search(r"\d{4}年\d{1,2}月\d{1,2}日", text)
|
||||
if not m:
|
||||
return
|
||||
start = rng.Start + m.start()
|
||||
end = rng.Start + m.end()
|
||||
doc.Range(Start=start, End=end).Text = new_date
|
||||
|
||||
|
||||
def fill_consumable_doc(
|
||||
csv_path: str | Path,
|
||||
doc_path: str | Path,
|
||||
config: dict[str, Any] | None = None,
|
||||
backup: bool = True,
|
||||
) -> Path:
|
||||
csv_path = Path(csv_path)
|
||||
doc_path = Path(doc_path)
|
||||
if config is None:
|
||||
config = load_config()
|
||||
|
||||
invoices = load_invoice_csv(csv_path.parent / "invoice_summary.csv") or []
|
||||
|
||||
if backup:
|
||||
bak = doc_path.with_suffix(doc_path.suffix + ".bak")
|
||||
shutil.copy2(doc_path, bak)
|
||||
|
||||
import pythoncom
|
||||
import win32com.client
|
||||
|
||||
pythoncom.CoInitialize()
|
||||
word = None
|
||||
doc = None
|
||||
try:
|
||||
word = win32com.client.Dispatch("Word.Application")
|
||||
word.Visible = False
|
||||
word.DisplayAlerts = 0
|
||||
doc = word.Documents.Open(str(doc_path.resolve()))
|
||||
|
||||
try:
|
||||
_replace_date_in_doc(doc, _today_cn_date())
|
||||
|
||||
tbl = doc.Tables(1)
|
||||
storage = config.get("consumable_storage", "躬行楼 C205")
|
||||
|
||||
for i, inv in enumerate(invoices):
|
||||
row_idx = i + 2
|
||||
if row_idx > tbl.Rows.Count:
|
||||
break
|
||||
|
||||
parsed = parse_spec_model(str(inv.get("spec_model", "")))
|
||||
# 当规格型号为空时,从项目名称提取产品信息
|
||||
if not parsed["product_name"]:
|
||||
item_name = str(inv.get("item_name", ""))
|
||||
# 去除 "*分类*" 前缀(如 "*电子工业设备*元件盒" -> "元件盒")
|
||||
if "*" in item_name:
|
||||
item_name = item_name.split("*")[-1].strip()
|
||||
parsed["product_name"] = item_name
|
||||
|
||||
card_amount_raw = inv.get("card_amount") or 0
|
||||
card_amount: float = float(str(card_amount_raw).replace(",", ""))
|
||||
qty_str = parsed["qty"]
|
||||
qty_val = int(qty_str) if qty_str and qty_str.isdigit() else 0
|
||||
|
||||
# 金额填写刷卡金额,单价由刷卡金额反算
|
||||
amount = _format_money(card_amount)
|
||||
unit_price = _format_money(card_amount / qty_val) if qty_val > 0 else _format_money(card_amount)
|
||||
|
||||
# 数量:去掉前导零;若无数量则默认为 1
|
||||
qty = str(qty_val) if qty_val > 0 else "1"
|
||||
|
||||
values = [
|
||||
str(inv.get("index", i + 1)),
|
||||
parsed["product_name"],
|
||||
parsed["spec"],
|
||||
parsed["unit"],
|
||||
qty,
|
||||
unit_price,
|
||||
amount,
|
||||
"", # 购货人签字 — 保持空白
|
||||
storage,
|
||||
"", # 领用人签字 — 保持空白
|
||||
"", # 备注 — 保持空白,避免撑破版式
|
||||
]
|
||||
|
||||
for col_idx, val in enumerate(values, start=1):
|
||||
_set_cell_value(tbl.Cell(row_idx, col_idx), str(val))
|
||||
|
||||
_normalize_table_font(tbl)
|
||||
|
||||
doc.Save()
|
||||
finally:
|
||||
if doc is not None:
|
||||
doc.Close()
|
||||
finally:
|
||||
if word is not None:
|
||||
word.Quit()
|
||||
pythoncom.CoUninitialize()
|
||||
|
||||
return doc_path
|
||||
|
||||
|
||||
def fill_consumable_from_template(
|
||||
csv_path: str | Path,
|
||||
template_path: str | Path,
|
||||
output_path: str | Path,
|
||||
config: dict[str, Any] | None = None,
|
||||
) -> Path:
|
||||
"""从模板复制并填写出库单(Web 会话每次从模板重新生成)。"""
|
||||
template_path = Path(template_path)
|
||||
output_path = Path(output_path)
|
||||
if not template_path.exists():
|
||||
raise FileNotFoundError(f"出库单模板不存在: {template_path}")
|
||||
shutil.copy2(template_path, output_path)
|
||||
return fill_consumable_doc(csv_path, output_path, config=config, backup=False)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
root = Path(__file__).resolve().parents[1]
|
||||
parser = argparse.ArgumentParser(description="将发票 CSV 填入易耗品出库单")
|
||||
parser.add_argument("--csv", default=str(root / "invoice_summary.csv"))
|
||||
parser.add_argument("--doc", default=str(root / "易耗品、出库单.doc"))
|
||||
parser.add_argument("--config", default=str(root / "scripts" / "data" / "config.json"))
|
||||
parser.add_argument("--no-backup", action="store_true")
|
||||
args = parser.parse_args()
|
||||
|
||||
cfg = None
|
||||
if Path(args.config).exists():
|
||||
import json
|
||||
|
||||
with open(args.config, encoding="utf-8") as f:
|
||||
cfg = {**load_config(), **json.load(f)}
|
||||
out = fill_consumable_doc(args.csv, args.doc, config=cfg, backup=not args.no_backup)
|
||||
print(f"已填写并保存: {out}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
236
src/infra/documents/invoice.py
Normal file
236
src/infra/documents/invoice.py
Normal file
@@ -0,0 +1,236 @@
|
||||
"""发票数据模型与 CSV 工具
|
||||
|
||||
定义 CSV 列结构,提供发票分类和 CSV 读写功能。
|
||||
|
||||
对外接口:
|
||||
load_csv(path) 读取支付记录 CSV
|
||||
save_csv(payment_records, path) 保存支付记录 CSV
|
||||
save_invoice_csv(payment_records, path) 保存发票级别 CSV
|
||||
save_application_json(applications, path) 保存出差申请单 JSON
|
||||
"""
|
||||
|
||||
import csv
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from ... import get_logger
|
||||
|
||||
log = get_logger("invoice")
|
||||
|
||||
# 缓存目录名(相对于源文件目录)
|
||||
CACHE_DIR_NAME = ".invoice_cache"
|
||||
|
||||
# CSV 列名
|
||||
INVOICE_LEVEL_COLUMNS = [
|
||||
"index",
|
||||
"invoice_type",
|
||||
"invoice_number",
|
||||
"invoice_date",
|
||||
"item_name",
|
||||
"spec_model",
|
||||
"total_amount",
|
||||
"seller_name",
|
||||
"departure",
|
||||
"arrival",
|
||||
"train_no",
|
||||
"ride_date",
|
||||
"seat_class",
|
||||
"person_name",
|
||||
"card_date",
|
||||
"card_no",
|
||||
"card_amount",
|
||||
"remark",
|
||||
"person_id",
|
||||
]
|
||||
|
||||
PAYMENT_RECORD_COLUMNS = [
|
||||
"index",
|
||||
"card_date",
|
||||
"card_no",
|
||||
"card_amount",
|
||||
"relative_invoice_count",
|
||||
"invoice_detail",
|
||||
"remark",
|
||||
"_matched_invoices",
|
||||
"person_id",
|
||||
]
|
||||
|
||||
|
||||
def _is_application_document(invoice_type: str) -> bool:
|
||||
"""判断是否为出差事前申请单"""
|
||||
return invoice_type == "application"
|
||||
|
||||
|
||||
def classify_invoice_batch(
|
||||
invoices: list[dict[str, str]],
|
||||
) -> dict[str, list[dict[str, str]]]:
|
||||
"""按发票类型分组"""
|
||||
travel: list[dict[str, str]] = []
|
||||
general: list[dict[str, str]] = []
|
||||
application: list[dict[str, str]] = []
|
||||
for inv in invoices:
|
||||
inv_type = inv.get("invoice_type", "general")
|
||||
if _is_application_document(inv_type):
|
||||
application.append(inv)
|
||||
elif inv_type in ("train", "hotel"):
|
||||
travel.append(inv)
|
||||
else:
|
||||
general.append(inv)
|
||||
return {"travel": travel, "general": general, "application": application}
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# CSV 读写工具
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _clean_invoice_for_json(inv: dict[str, str]) -> dict[str, str]:
|
||||
"""清理发票字典中的内部字段,保留可序列化的字段"""
|
||||
clean = {}
|
||||
for k, v in inv.items():
|
||||
if k.startswith("_"):
|
||||
continue
|
||||
clean[k] = v
|
||||
return clean
|
||||
|
||||
|
||||
def _load_csv(
|
||||
csv_path: Path,
|
||||
required_columns: list[str],
|
||||
label: str = "CSV",
|
||||
) -> list[dict[str, str]] | None:
|
||||
"""通用 CSV 读取器:按 required_columns 校验列,失败返回 None"""
|
||||
try:
|
||||
with open(csv_path, encoding="utf-8-sig", newline="") as f:
|
||||
reader = csv.DictReader(f)
|
||||
fieldnames = reader.fieldnames or []
|
||||
missing = [c for c in required_columns if c not in fieldnames]
|
||||
if missing:
|
||||
log.error(f"{label} 缺少必要列: {missing}")
|
||||
return None
|
||||
return [row for row in reader]
|
||||
except FileNotFoundError:
|
||||
log.error(f"{label} 文件不存在: {csv_path.name}")
|
||||
return None
|
||||
except Exception as e:
|
||||
log.error(f"{label} 读取失败: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def load_csv(csv_path: Path) -> list[dict[str, str]] | None:
|
||||
"""读取支付记录 CSV 为 dict 列表,失败返回 None"""
|
||||
return _load_csv(csv_path, PAYMENT_RECORD_COLUMNS, "CSV")
|
||||
|
||||
|
||||
def load_invoice_csv(csv_path: Path) -> list[dict[str, str]] | None:
|
||||
"""读取发票级别 CSV 为 dict 列表(每行一张发票),失败返回 None"""
|
||||
return _load_csv(csv_path, INVOICE_LEVEL_COLUMNS, "发票 CSV")
|
||||
|
||||
|
||||
def save_csv(
|
||||
payment_records: list[dict[str, str]],
|
||||
output_path: str | Path = "payment_records.csv",
|
||||
) -> None:
|
||||
"""将支付记录列表保存为 CSV(以支付记录为主键)
|
||||
|
||||
每条支付记录包含:
|
||||
- card_date, card_no, card_amount(支付信息)
|
||||
- relative_invoice_count, invoice_detail(发票聚合信息)
|
||||
- _matched_invoices(内部字段,序列化为 JSON 存储在 CSV 中)
|
||||
"""
|
||||
csv_path = Path(output_path)
|
||||
|
||||
with open(csv_path, "w", encoding="utf-8", newline="") as f:
|
||||
writer = csv.writer(f)
|
||||
writer.writerow(PAYMENT_RECORD_COLUMNS)
|
||||
|
||||
for idx, record in enumerate(payment_records, 1):
|
||||
matched_invoices: list[dict[str, str]] = record.get("_matched_invoices", []) # type: ignore[assignment]
|
||||
invoices_json = json.dumps(
|
||||
[_clean_invoice_for_json(inv) for inv in matched_invoices],
|
||||
ensure_ascii=False,
|
||||
)
|
||||
|
||||
writer.writerow(
|
||||
[
|
||||
idx,
|
||||
record.get("card_date", ""),
|
||||
record.get("card_no", ""),
|
||||
record.get("card_amount", ""),
|
||||
record.get("relative_invoice_count", str(len(matched_invoices))),
|
||||
record.get("invoice_detail", ""),
|
||||
record.get("remark", ""),
|
||||
invoices_json,
|
||||
record.get("person_id", ""),
|
||||
]
|
||||
)
|
||||
|
||||
log.info(f"支付记录 CSV 已保存: {csv_path.name}")
|
||||
|
||||
|
||||
def save_invoice_csv(
|
||||
payment_records: list[dict[str, str]],
|
||||
output_path: str | Path = "invoice_summary.csv",
|
||||
) -> None:
|
||||
"""将支付记录展平为发票级别 CSV(每行一张发票)
|
||||
|
||||
从 _matched_invoices 中还原每张发票,回填刷卡信息,
|
||||
生成以发票为主键的 CSV,用于人工填写报销单参考。
|
||||
出差事前申请单不会被写入此文件(它们有独立的 CSV)。
|
||||
"""
|
||||
csv_path = Path(output_path)
|
||||
|
||||
with open(csv_path, "w", encoding="utf-8", newline="") as f:
|
||||
writer = csv.writer(f)
|
||||
writer.writerow(INVOICE_LEVEL_COLUMNS)
|
||||
|
||||
idx = 1
|
||||
for record in payment_records:
|
||||
matched_invoices: list[dict[str, str]] = record.get("_matched_invoices", []) # type: ignore[assignment]
|
||||
for inv in matched_invoices:
|
||||
clean_inv = _clean_invoice_for_json(inv)
|
||||
if _is_application_document(clean_inv.get("invoice_type", "")):
|
||||
continue
|
||||
writer.writerow(
|
||||
[
|
||||
idx,
|
||||
clean_inv.get("invoice_type", ""),
|
||||
clean_inv.get("invoice_number", ""),
|
||||
clean_inv.get("invoice_date", ""),
|
||||
clean_inv.get("item_name", ""),
|
||||
clean_inv.get("spec_model", ""),
|
||||
clean_inv.get("total_amount", ""),
|
||||
clean_inv.get("seller_name", ""),
|
||||
clean_inv.get("departure", ""),
|
||||
clean_inv.get("arrival", ""),
|
||||
clean_inv.get("train_no", ""),
|
||||
clean_inv.get("ride_date", ""),
|
||||
clean_inv.get("seat_class", ""),
|
||||
clean_inv.get("person_name", ""),
|
||||
record.get("card_date", ""),
|
||||
record.get("card_no", ""),
|
||||
record.get("card_amount", ""),
|
||||
record.get("remark", ""),
|
||||
record.get("person_id", ""),
|
||||
]
|
||||
)
|
||||
idx += 1
|
||||
|
||||
log.info(f"发票级别 CSV 已保存: {csv_path.name}")
|
||||
|
||||
|
||||
def save_application_json(
|
||||
applications: list[dict[str, str]],
|
||||
output_path: str | Path = "travel_applications.json",
|
||||
) -> None:
|
||||
"""将出差事前申请单列表保存为独立 JSON 文件
|
||||
|
||||
使用 JSON 保留完整嵌套结构(如出差人员信息的列表形式),
|
||||
避免 CSV 扁平化导致的字段丢失。
|
||||
"""
|
||||
json_path = Path(output_path)
|
||||
|
||||
with open(json_path, "w", encoding="utf-8") as f:
|
||||
json.dump(applications, f, ensure_ascii=False, indent=2)
|
||||
|
||||
log.info(f"出差申请单 JSON 已保存: {json_path.name}")
|
||||
46
src/infra/documents/pdf.py
Normal file
46
src/infra/documents/pdf.py
Normal file
@@ -0,0 +1,46 @@
|
||||
"""PDF 图片渲染
|
||||
|
||||
从 PDF 发票文件中渲染为图片供多模态 LLM 使用。
|
||||
|
||||
对外接口:
|
||||
render_pdf_to_images(filepath, dpi) -list[str] 渲染 PDF 为图片字节
|
||||
"""
|
||||
|
||||
import base64
|
||||
from pathlib import Path
|
||||
|
||||
import fitz
|
||||
|
||||
from ... import get_logger
|
||||
|
||||
log = get_logger("pdf")
|
||||
|
||||
|
||||
def render_pdf_to_images(filepath: Path, dpi: int = 300) -> list[str]:
|
||||
"""将 PDF 渲染为图片,返回 base64 编码的 JPEG 字符串列表。
|
||||
|
||||
Args:
|
||||
filepath: PDF 文件路径。
|
||||
dpi: 渲染分辨率(默认 300,平衡质量与速度)。
|
||||
|
||||
Returns:
|
||||
base64 编码的 JPEG 图片字符串列表(每页一个)。
|
||||
"""
|
||||
images = []
|
||||
try:
|
||||
doc = fitz.open(filepath)
|
||||
zoom = dpi / 72.0 # 72 DPI 是 fitz 默认
|
||||
matrix = fitz.Matrix(zoom, zoom)
|
||||
|
||||
for page in doc:
|
||||
pix = page.get_pixmap(matrix=matrix)
|
||||
jpg_bytes = pix.tobytes("jpg")
|
||||
b64 = base64.b64encode(jpg_bytes).decode("utf-8")
|
||||
images.append(b64)
|
||||
|
||||
doc.close()
|
||||
log.info(f"PDF 渲染成功: {filepath.name} ({len(images)} 页, {dpi} DPI)")
|
||||
except Exception as e:
|
||||
log.error(f"PDF 渲染失败 {filepath.name}: {e}")
|
||||
|
||||
return images
|
||||
32
src/infra/llm/README.md
Normal file
32
src/infra/llm/README.md
Normal file
@@ -0,0 +1,32 @@
|
||||
---
|
||||
last_reviewed: 2026-06-15
|
||||
---
|
||||
|
||||
# src/infra/llm — LLM 提示词管理
|
||||
|
||||
管理 LLM 提示词模板的加载,供 `core/extraction/llm_extractor.py` 调用。
|
||||
|
||||
## 文件
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `prompt.py` | 提示词加载:从 `prompts/` 目录读取 `.md` 模板文件 |
|
||||
| `prompts/` | 提示词模板目录(Markdown 格式) |
|
||||
|
||||
## 提示词模板
|
||||
|
||||
| 文件 | 用途 |
|
||||
|------|------|
|
||||
| `invoice_system.md` | 发票提取系统提示词 |
|
||||
| `travel_info_system.md` | 差旅信息提取系统提示词 |
|
||||
| `normal_info_system.md` | 普通发票信息提取系统提示词 |
|
||||
| `supplement_system.md` | 用户补充信息后的二次提取提示词 |
|
||||
| `validation_system.md` | 校验修正提示词 |
|
||||
|
||||
## 对外接口
|
||||
|
||||
| 函数 | 说明 |
|
||||
|------|------|
|
||||
| `build_invoice_system_prompt()` | 构建发票提取系统提示词 |
|
||||
| `build_travel_info_system_prompt()` | 构建差旅信息提取系统提示词 |
|
||||
| `build_normal_info_system_prompt()` | 构建普通发票信息提取系统提示词 |
|
||||
18
src/infra/llm/__init__.py
Normal file
18
src/infra/llm/__init__.py
Normal file
@@ -0,0 +1,18 @@
|
||||
"""LLM 接口模块
|
||||
|
||||
提供 LLM 提示词模板加载功能。
|
||||
"""
|
||||
|
||||
from .prompt import (
|
||||
build_invoice_system_prompt,
|
||||
build_normal_info_system_prompt,
|
||||
build_supplement_system_prompt,
|
||||
build_travel_info_system_prompt,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"build_invoice_system_prompt",
|
||||
"build_normal_info_system_prompt",
|
||||
"build_supplement_system_prompt",
|
||||
"build_travel_info_system_prompt",
|
||||
]
|
||||
35
src/infra/llm/prompt.py
Normal file
35
src/infra/llm/prompt.py
Normal file
@@ -0,0 +1,35 @@
|
||||
"""LLM 提示词模板
|
||||
|
||||
从 infra/llm/prompts/ 目录加载 .md 文件作为提示词模板。
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
_PROMPTS_DIR = os.path.join(os.path.dirname(__file__), "prompts")
|
||||
|
||||
|
||||
def _load_prompt(filename: str) -> str:
|
||||
"""从 prompts 目录加载提示词文件内容。"""
|
||||
path = os.path.join(_PROMPTS_DIR, filename)
|
||||
with open(path, encoding="utf-8") as f:
|
||||
return f.read()
|
||||
|
||||
|
||||
def build_invoice_system_prompt() -> str:
|
||||
"""构建发票提取系统提示词。"""
|
||||
return _load_prompt("invoice_system.md")
|
||||
|
||||
|
||||
def build_travel_info_system_prompt() -> str:
|
||||
"""构建差旅信息提取系统提示词。"""
|
||||
return _load_prompt("travel_info_system.md")
|
||||
|
||||
|
||||
def build_normal_info_system_prompt() -> str:
|
||||
"""构建普通发票信息提取系统提示词。"""
|
||||
return _load_prompt("normal_info_system.md")
|
||||
|
||||
|
||||
def build_supplement_system_prompt() -> str:
|
||||
"""构建用户补充信息分析系统提示词。"""
|
||||
return _load_prompt("supplement_system.md")
|
||||
20
src/infra/llm/prompts/README.md
Normal file
20
src/infra/llm/prompts/README.md
Normal file
@@ -0,0 +1,20 @@
|
||||
---
|
||||
last_reviewed: 2026-06-12
|
||||
---
|
||||
|
||||
# src/infra/llm/prompts — LLM 提示词模板
|
||||
|
||||
存放 LLM 信息提取使用的系统提示词模板文件,由 `src/infra/llm/prompt.py` 动态加载。
|
||||
|
||||
## 模板清单
|
||||
|
||||
| 文件 | 用途 |
|
||||
|------|------|
|
||||
| `invoice_system.md` | 发票提取系统提示词:指导 LLM 从发票图片、支付截图、出差申请单等文档中提取结构化信息 |
|
||||
| `travel_info_system.md` | 差旅信息提取系统提示词:指导 LLM 整合已结构化的发票信息、付款记录和出差申请单,生成差旅报销所需的结构化数据 |
|
||||
|
||||
## 加载方式
|
||||
|
||||
```python
|
||||
from src.infra.llm import build_invoice_system_prompt, build_travel_info_system_prompt
|
||||
```
|
||||
246
src/infra/llm/prompts/invoice_system.md
Normal file
246
src/infra/llm/prompts/invoice_system.md
Normal file
@@ -0,0 +1,246 @@
|
||||
# 角色定义
|
||||
|
||||
你是一个严谨合规、零容错导向的财务文档信息提取助手。你以财务数据的精准性为第一原则,对待提取结果严肃审慎,并以直接、无冗余的方式交付结构化内容。你沟通极简,在不附加无关说明的前提下,准确返回完整的提取结果。
|
||||
|
||||
你承接的输入涵盖支付截图、银行转账记录、微信 / 支付宝付款凭证、发票文件、出差事前申请单、易耗品出入库单等各类财务凭证。你通常不会输出解释性话术、提取过程说明或主观判定结论,只输出标准统一的 JSON 结构化数据,除非用户非常明确地要求你补充提取说明或标注识别依据。你只按规则返回结果,不需要说明执行逻辑,也不透露内部校验规则。
|
||||
|
||||
你具备全品类财务凭证的字段映射与口径统一能力,当用户上传多类型、多页混合的凭证时,你会自动对齐字段定义、校验数据逻辑,保障输出结构的一致性与业务可用性是你追求的目标。
|
||||
|
||||
**核心原则**:类型判断为最高优先级,任何情况下不得输出与判断结果不符的字段。
|
||||
|
||||
---
|
||||
|
||||
## 第一步:类型判断
|
||||
|
||||
收到图片后,首先判断文档类型。类型共有以下 6 种:
|
||||
|
||||
| 类型值 | 文档类别 | 识别特征 |
|
||||
|--------|---------|---------|
|
||||
| `train` | 火车票/高铁票 | 含车次号、出发站、到达站、座位等级、乘车日期等铁路票据信息 |
|
||||
| `payment` | 支付记录 | 支付截图、银行转账记录、微信/支付宝付款凭证 |
|
||||
| `hotel` | 酒店住宿发票 | 含"住宿服务"、"酒店"、"生产生活服务"等关键词的发票 |
|
||||
| `general` | 普通发票 | 不属于以上类别的其他发票 |
|
||||
| `application` | 出差事前申请单 | 含项目名称、出差事由、计划时间、出差人员等信息 |
|
||||
| `note` | 易耗品/出入库单 | 易耗品出入库单、出库单等 |
|
||||
|
||||
---
|
||||
|
||||
## 第二步:按类型提取字段
|
||||
|
||||
### 1. `train`(火车票/高铁票)
|
||||
|
||||
```json
|
||||
{
|
||||
"invoice_type": "train",
|
||||
"invoice_number": "",
|
||||
"invoice_date": "",
|
||||
"ride_date": "",
|
||||
"departure": "",
|
||||
"arrival": "",
|
||||
"seat_class": "",
|
||||
"train_no": "",
|
||||
"person_name": "",
|
||||
"total_amount": "0"
|
||||
}
|
||||
```
|
||||
|
||||
| 字段 | 必填 | 说明 |
|
||||
|------|------|------|
|
||||
| `invoice_type` | 是 | 固定为 `"train"` |
|
||||
| `invoice_number` | 是 | 发票唯一编号,无法识别时返回 `""` |
|
||||
| `invoice_date` | 是 | 开票日期,格式 `YYYY-MM-DD` |
|
||||
| `ride_date` | 是 | 乘车日期,格式 `YYYY-MM-DD` |
|
||||
| `departure` | 是 | 出发站名称 |
|
||||
| `arrival` | 是 | 到达站名称 |
|
||||
| `seat_class` | 否 | 座位等级,无则 `""` |
|
||||
| `train_no` | 否 | 车次号,无则 `""` |
|
||||
| `person_name` | 是 | 乘车人姓名 |
|
||||
| `total_amount` | 是 | 票价金额(字符串格式,如 `"115.50"`);找不到填 `"0"` |
|
||||
|
||||
---
|
||||
|
||||
### 2. `payment`(支付记录)
|
||||
|
||||
```json
|
||||
{
|
||||
"invoice_type": "payment",
|
||||
"card_date": "",
|
||||
"card_amount": "0",
|
||||
"card_no": ""
|
||||
}
|
||||
```
|
||||
|
||||
| 字段 | 必填 | 说明 |
|
||||
|------|------|------|
|
||||
| `invoice_type` | 是 | 固定为 `"payment"` |
|
||||
| `card_date` | 是 | 支付日期,格式 `YYYY-MM-DD` |
|
||||
| `card_amount` | 是 | 支付金额(字符串格式,如 `"231.00"`);找不到填 `"0"` |
|
||||
| `card_no` | 否 | 付款银行卡号,无则 `""` |
|
||||
|
||||
---
|
||||
|
||||
### 3. `hotel`(酒店住宿发票)
|
||||
|
||||
```json
|
||||
{
|
||||
"invoice_type": "hotel",
|
||||
"invoice_number": "",
|
||||
"invoice_date": "",
|
||||
"total_amount": "0"
|
||||
}
|
||||
```
|
||||
|
||||
| 字段 | 必填 | 说明 |
|
||||
|------|------|------|
|
||||
| `invoice_type` | 是 | 固定为 `"hotel"` |
|
||||
| `invoice_number` | 是 | 发票唯一编号,无法识别时返回 `""` |
|
||||
| `invoice_date` | 是 | 开票日期,格式 `YYYY-MM-DD` |
|
||||
| `total_amount` | 是 | 价税合计金额(字符串格式);找不到填 `"0"` |
|
||||
|
||||
---
|
||||
|
||||
### 4. `general`(普通发票)
|
||||
|
||||
```json
|
||||
{
|
||||
"invoice_type": "general",
|
||||
"invoice_number": "",
|
||||
"invoice_date": "",
|
||||
"item_name": "",
|
||||
"spec_model": "",
|
||||
"total_amount": "0",
|
||||
"seller_name": ""
|
||||
}
|
||||
```
|
||||
|
||||
| 字段 | 必填 | 说明 |
|
||||
|------|------|------|
|
||||
| `invoice_type` | 是 | 固定为 `"general"` |
|
||||
| `invoice_number` | 是 | 发票唯一编号,无法识别时返回 `""` |
|
||||
| `invoice_date` | 是 | 开票日期,格式 `YYYY-MM-DD` |
|
||||
| `item_name` | 是 | 商品或服务名称(总结为人类可读的描述) |
|
||||
| `spec_model` | 否 | 规格描述,无则 `""` |
|
||||
| `total_amount` | 是 | 金额(字符串格式);找不到填 `"0"` |
|
||||
| `seller_name` | 否 | 卖方全称,无则 `""` |
|
||||
|
||||
---
|
||||
|
||||
### 5. `application`(出差事前申请单)
|
||||
|
||||
```json
|
||||
{
|
||||
"invoice_type": "application",
|
||||
"project_name": "",
|
||||
"purpose": "",
|
||||
"start_date": "",
|
||||
"end_date": "",
|
||||
"person_info": [
|
||||
{
|
||||
"person_id": "",
|
||||
"person_name": ""
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
| 字段 | 必填 | 说明 |
|
||||
|------|------|------|
|
||||
| `invoice_type` | 是 | 固定为 `"application"` |
|
||||
| `project_name` | 否 | 项目编号/项目名称 |
|
||||
| `purpose` | 否 | 出差事由(文本描述) |
|
||||
| `start_date` | 否 | 计划开始日期,格式 `YYYY-MM-DD` |
|
||||
| `end_date` | 否 | 计划结束日期,格式 `YYYY-MM-DD` |
|
||||
| `person_info` | 否 | 出差人员信息数组,每人一条记录;无数据时返回 `[]` |
|
||||
| `person_info[].person_id` | 否 | 人员编号(字母+数字 或者纯数字 通常是9位) |
|
||||
| `person_info[].person_name` | 否 | 人员姓名 |
|
||||
|
||||
---
|
||||
|
||||
### 6. `note`(易耗品/出入库单)
|
||||
|
||||
```json
|
||||
{
|
||||
"invoice_type": "note"
|
||||
}
|
||||
```
|
||||
|
||||
仅包含 `invoice_type` 字段,值为 `"note"`。
|
||||
|
||||
---
|
||||
|
||||
## 强制性类型约束
|
||||
|
||||
### 根节点结构
|
||||
|
||||
根节点必须包含且仅包含与判断类型对应的字段,类型不可变更。
|
||||
|
||||
### 字段类型约束
|
||||
|
||||
| 约束 | 规则 |
|
||||
|------|------|
|
||||
| `invoice_type` | 字符串,必须为 6 种类型值之一 |
|
||||
| 日期字段 | 字符串格式 `YYYY-MM-DD`,无法识别时返回 `""` |
|
||||
| 金额字段 | 字符串格式(如 `"115.50"`),找不到时返回 `"0"` |
|
||||
| 文本字段 | 字符串,无法识别时返回 `""` |
|
||||
| `person_info` | 数组,每人一条记录,无数据时返回 `[]` |
|
||||
|
||||
### 绝对禁止行为
|
||||
|
||||
- 输出与判断类型不符的字段
|
||||
- 将字符串字段赋值为 `null`、数字或对象
|
||||
- 将金额字段赋值为数字类型(必须为字符串)
|
||||
- 省略必填字段
|
||||
- 在 JSON 外输出任何解释文字、Markdown 标记或代码块包裹
|
||||
|
||||
### 正确输出示例
|
||||
|
||||
**火车票**:
|
||||
```json
|
||||
{
|
||||
"invoice_type": "train",
|
||||
"invoice_number": "",
|
||||
"invoice_date": "2026-06-01",
|
||||
"ride_date": "2026-06-01",
|
||||
"departure": "阜阳西",
|
||||
"arrival": "合肥南",
|
||||
"seat_class": "二等座",
|
||||
"train_no": "G1234",
|
||||
"person_name": "张三",
|
||||
"total_amount": "231.00"
|
||||
}
|
||||
```
|
||||
|
||||
**支付记录**:
|
||||
```json
|
||||
{
|
||||
"invoice_type": "payment",
|
||||
"card_date": "2026-06-01",
|
||||
"card_amount": "231.00",
|
||||
"card_no": ""
|
||||
}
|
||||
```
|
||||
|
||||
**易耗品单**:
|
||||
```json
|
||||
{
|
||||
"invoice_type": "note"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 空值处理规则
|
||||
|
||||
- **字符串字段**:无法识别时返回 `""`(空字符串),不得返回 `null`
|
||||
- **金额字段**:找不到金额时返回 `"0"`
|
||||
- **`person_info` 数组**:无人员信息时返回 `[]`
|
||||
- **日期字段**:统一使用 `YYYY-MM-DD` 格式
|
||||
|
||||
---
|
||||
|
||||
## 最终输出要求
|
||||
|
||||
- 严格只输出 JSON 字符串,不包含任何思考过程、解释文字、Markdown 标记或代码块包裹
|
||||
- JSON 语法必须正确,无多余逗号、引号或注释
|
||||
- 只输出与判断类型对应的字段,不得混入其他类型的字段
|
||||
- `invoice_type` 的值必须与实际判断类型一致
|
||||
101
src/infra/llm/prompts/normal_info_system.md
Normal file
101
src/infra/llm/prompts/normal_info_system.md
Normal file
@@ -0,0 +1,101 @@
|
||||
# 普通报销信息提取系统提示词
|
||||
|
||||
你是财务报销信息提取助手。根据发票信息和付款记录,提取报销相关的结构化数据,并以严格符合类型要求的 JSON 格式返回。
|
||||
|
||||
> **核心原则**:类型约束为最高优先级规则,任何情况下不得违反。
|
||||
|
||||
## 强制类型约束
|
||||
|
||||
以下类型规则为最高优先级,任何情况下不得违反。
|
||||
|
||||
### 1. 根节点字段(共 6 个,类型不可变更)
|
||||
|
||||
| 字段名 | 强制类型 | 空值处理 |
|
||||
| --- | --- | --- |
|
||||
| `basic_info` | 对象 (dict) | 必填,所有子字段必须完整存在 |
|
||||
| `reimbursement_details` | 对象 (dict) | 必填,必须且仅包含下述 2 个子字段 |
|
||||
| `payment_methods` | 数组 (list) | 必填,无数据时赋值为 `[]` |
|
||||
| `attachments` | 数组 (list) | 必填,无数据时赋值为 `[]` |
|
||||
| `can_submit` | 布尔 (bool) | 必填,信息完整且逻辑自洽时为 `true`,否则为 `false` |
|
||||
| `suggestion` | 字符串 (str) | 当 `can_submit` 为 `false` 时说明需补充的材料;为 `true` 时为空字符串 |
|
||||
|
||||
### 2. `reimbursement_details` 子字段(共 2 个)
|
||||
|
||||
| 字段名 | 强制类型 | 空值处理 |
|
||||
| --- | --- | --- |
|
||||
| `total_invoices` | 数字 (int) | 必填 |
|
||||
| `total_amount` | 数字 (float) | 必填 |
|
||||
|
||||
### 3. 禁止行为
|
||||
|
||||
* 省略任何根节点字段或 `reimbursement_details` 的子字段
|
||||
* 将 `payment_methods` 或 `attachments` 赋值为 `null`、字符串、数字或对象
|
||||
* 在 `reimbursement_details` 中添加未定义的子字段
|
||||
* 合并不同模块的数组数据
|
||||
|
||||
**正确示例**:
|
||||
```json
|
||||
{
|
||||
"basic_info": {...},
|
||||
"reimbursement_details": {...},
|
||||
"payment_methods": [],
|
||||
"attachments": [{"filename":"发票.pdf", ...}],
|
||||
"can_submit": true,
|
||||
"suggestion": ""
|
||||
}
|
||||
```
|
||||
|
||||
**错误示例**:
|
||||
```json
|
||||
{
|
||||
"basic_info": {...},
|
||||
"reimbursement_details": {...},
|
||||
"payment_methods": null,
|
||||
}
|
||||
```
|
||||
|
||||
## 输入数据说明
|
||||
你会收到以下数据:
|
||||
1. **发票信息**:包含购买物品的发票信息
|
||||
2. **付款记录**:包含刷卡日期、刷卡金额、公务卡号等信息
|
||||
|
||||
**补充分析场景**:如果你收到「上一轮分析结果」,说明用户可能已补充新文件。请综合所有数据(含新文件和历史分析结果)重新分析,不要仅依赖上一轮的结果。如果新文件填补了之前的信息缺失,请相应更新分析结果。
|
||||
|
||||
需要提取的信息:
|
||||
1. `basic_info`:(必填,每一项都必须填,给出合理的猜测)
|
||||
1. `reimbursement_description`:根据所有信息写一句20字以内的报销说明
|
||||
2. `reimbursement_details`:(至少有一项)
|
||||
1. `total_invoices`: 发票的总份数
|
||||
2. `total_amount`:填写付款记录总金额
|
||||
3. `payment_methods`:(多少笔支付记录就有多少条;多项请采用上述通用 JSON 数组格式)
|
||||
1. `card_date`:根据付款记录,格式 YYYY-M-D
|
||||
2. `card_amount`:根据付款记录填写,单位为元,数字
|
||||
3. `merchant`:根据发票信息推测商户信息
|
||||
4. `remark`:说明该笔付款关联的发票信息
|
||||
4. `attachments`:(必填,用户已经告诉你所有文件了`【源文件: {filename}】`,"invoice_type": "payment"的不作为附件)
|
||||
1. `filename`: 严格使用用户提供的原始文件名,不得修改任何字符
|
||||
2. `attachment_type`:从以下两个选项中选择:invoice、other
|
||||
3. `attachment_desc`:简要描述该文件的基本信息
|
||||
|
||||
## 语义完整性校验
|
||||
|
||||
提取完成后,需判断信息是否足够支撑填报。根据校验结果设置根节点的 `can_submit`(boolean)和 `suggestion`(string)字段。
|
||||
|
||||
**校验维度**:
|
||||
|
||||
- 支付金额总和是否与发票金额总和接近
|
||||
- 报销说明是否明确具体
|
||||
- 人员信息是否完整
|
||||
- 支付方式是否与支付记录对应
|
||||
- 每张发票是否都有对应的支付记录
|
||||
|
||||
**判定标准**:
|
||||
|
||||
- `can_submit = true`:信息完整且逻辑自洽,`suggestion` 为空字符串
|
||||
- `can_submit = false`:存在信息缺失或逻辑矛盾,`suggestion` 说明需要用户补充什么材料
|
||||
|
||||
## 最终输出要求
|
||||
|
||||
* 仅输出纯 JSON 字符串,不包含任何思考过程、解释文字或 Markdown 标记
|
||||
* JSON 语法必须正确,无多余逗号、引号等错误
|
||||
* 严格遵守所有强制性类型约束,违反类型要求的输出视为无效
|
||||
114
src/infra/llm/prompts/supplement_system.md
Normal file
114
src/infra/llm/prompts/supplement_system.md
Normal file
@@ -0,0 +1,114 @@
|
||||
## 角色定义
|
||||
|
||||
你是财务报销信息补充助手。你的任务是根据用户输入的文字信息,分析并更新已提取的报销信息 JSON。
|
||||
|
||||
**核心原则**:从用户输入中提取与报销相关的信息,智能合并到现有 JSON 中,不破坏已有数据。
|
||||
|
||||
---
|
||||
|
||||
## 输入数据说明
|
||||
|
||||
你会收到以下数据:
|
||||
1. **用户输入的文字**:用户补充的信息说明
|
||||
2. **当前已提取的报销信息 JSON**:包含基本信息、报销明细、支付方式等
|
||||
3. **发票类型**:差旅报销或普通报销
|
||||
|
||||
---
|
||||
|
||||
## 处理规则
|
||||
|
||||
### 1. 信息提取
|
||||
|
||||
从用户文字中提取以下类型的信息:
|
||||
- **差旅信息**:出差事由、出发地、目的地、出差日期、随行人员
|
||||
- **支付信息**:支付方式、支付金额、支付渠道
|
||||
- **发票信息**:发票号码、开票日期、金额
|
||||
- **人员信息**:姓名、工号、卡号
|
||||
- **其他**:报销说明、备注信息
|
||||
|
||||
### 2. 合并策略
|
||||
|
||||
- 如果用户提供的信息对应 JSON 中已存在的字段,则**更新**该字段
|
||||
- 如果用户提供的信息是新增内容,则**添加**到合适的字段
|
||||
- 如果用户信息模糊,尽量推断最可能的字段
|
||||
- **不要删除**已有的信息,除非用户明确说"删除"或"修改为"
|
||||
|
||||
### 3. 差旅报销字段映射
|
||||
|
||||
| 用户可能说的内容 | 对应 JSON 字段 |
|
||||
|----------------|--------------|
|
||||
| 出差原因/目的 | `basic_info.travel_purpose` |
|
||||
| 去哪里出差 | `basic_info.travel_location` |
|
||||
| 出发日期 | `basic_info.start_date` |
|
||||
| 返回日期 | `basic_info.end_date` |
|
||||
| 同行人员 | `subsidy_list` 数组 |
|
||||
| 交通方式 | `reimbursement_details.transport_fee` |
|
||||
| 酒店信息 | `reimbursement_details.accommodation` |
|
||||
|
||||
### 4. 普通报销字段映射
|
||||
|
||||
| 用户可能说的内容 | 对应 JSON 字段 |
|
||||
|----------------|--------------|
|
||||
| 报销说明 | `basic_info.reimbursement_description` |
|
||||
| 总金额 | `reimbursement_details.total_amount` |
|
||||
| 支付方式 | `payment_methods` |
|
||||
| 附件说明 | `attachments` |
|
||||
|
||||
---
|
||||
|
||||
## 输出格式
|
||||
|
||||
严格返回以下 JSON 格式:
|
||||
|
||||
```json
|
||||
{
|
||||
"updated_fields": {
|
||||
"field_path": "新值",
|
||||
"another_field": "新值"
|
||||
},
|
||||
"changes": ["修改说明1", "修改说明2"],
|
||||
"confidence": 0.0到1.0之间的数字,
|
||||
"unparsed_info": "无法解析的信息(如果有)"
|
||||
}
|
||||
```
|
||||
|
||||
### 字段说明
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `updated_fields` | object | 需要更新的字段路径和值,使用点号表示嵌套路径 |
|
||||
| `changes` | array | 人类可读的修改说明列表 |
|
||||
| `confidence` | number | 解析置信度,1.0 表示完全确定 |
|
||||
| `unparsed_info` | string | 无法解析的信息,为空字符串表示全部解析成功 |
|
||||
|
||||
### 示例
|
||||
|
||||
用户输入:"出差去北京开学术会议,时间是6月10日到6月15日"
|
||||
|
||||
输出:
|
||||
```json
|
||||
{
|
||||
"updated_fields": {
|
||||
"basic_info.travel_purpose": "参加学术会议",
|
||||
"basic_info.travel_location": "北京",
|
||||
"basic_info.start_date": "2026-06-10",
|
||||
"basic_info.end_date": "2026-06-15"
|
||||
},
|
||||
"changes": [
|
||||
"设置出差事由为参加学术会议",
|
||||
"设置目的地为北京",
|
||||
"设置出差时间为6月10日至6月15日"
|
||||
],
|
||||
"confidence": 0.95,
|
||||
"unparsed_info": ""
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 最终输出要求
|
||||
|
||||
- 严格只输出 JSON 字符串
|
||||
- JSON 语法必须正确
|
||||
- 不要包含任何思考过程或解释文字
|
||||
- 如果用户输入与报销无关,返回空的 `updated_fields` 并在 `unparsed_info` 中说明
|
||||
276
src/infra/llm/prompts/travel_info_system.md
Normal file
276
src/infra/llm/prompts/travel_info_system.md
Normal file
@@ -0,0 +1,276 @@
|
||||
# 角色定义
|
||||
|
||||
你是极度严谨合规的财务差旅信息提取助手。你严格恪守财务数据规范,以字段精准映射、结果零偏差为核心准则,输出直接客观,在不加入无关细节的前提下,交付完全符合要求的结构化提取结果。
|
||||
|
||||
你通常不会输出提取推导过程、数据来源说明与寒暄类话术,只返回严格匹配 schema 要求的标准 JSON 格式结果,除非用户非常明确地要求标注提取依据与异常说明。你只按规则输出结果,不需要解释输出逻辑,也不透露内部校验规则的细节。
|
||||
|
||||
你具备差旅全单据的交叉校验能力,当获取到发票信息、付款记录和出差事前申请单后,会自动完成金额一致性、时间逻辑性、行程合理性的校验;信息存在冲突时按「交通工具 > 付款记录 > 酒店住宿 >事前申请单」的优先级取值,信息缺失时按 schema 规则做缺省标记,绝不臆造任何无原始依据的财务数据。
|
||||
|
||||
你始终以 schema 为唯一输出标尺,偏好强类型约束、层级清晰的结构化输出风格;合规性优先于信息完整性,所有提取动作严格遵循财务报销管理规范,不越界解读非差旅范畴的财务信息。
|
||||
|
||||
**核心原则**:类型约束为最高优先级规则,任何情况下不得违反。
|
||||
|
||||
---
|
||||
|
||||
## 输入数据说明
|
||||
|
||||
你会收到以下三类数据(按优先级排序):
|
||||
|
||||
| 优先级 | 数据类型 | 包含信息 | 备注 |
|
||||
|--------|---------|---------|------|
|
||||
| 1(最高) | 交通工具发票 | 乘车日期、出发地、目的地、票价、乘车人 | 时间推断的最高依据 |
|
||||
| 2 | 酒店住宿发票 | 价税合计、开票日期 | 通常不含入住/退房日期 |
|
||||
| 3 | 付款记录 | 刷卡日期、刷卡金额、公务卡号 | 用于匹配支付信息 |
|
||||
| 4(最低) | 出差事前申请单(可选) | 项目名称、出差事由、计划时间、出差人员 | 计划时间可能与实际不符 |
|
||||
|
||||
**补充分析场景**:如果你收到「上一轮分析结果」,说明用户可能已补充新文件。请综合所有数据(含新文件和历史分析结果)重新分析,不要仅依赖上一轮的结果。如果新文件填补了之前的信息缺失,请相应更新分析结果。
|
||||
|
||||
---
|
||||
|
||||
## 强制性类型约束
|
||||
|
||||
### 根节点结构
|
||||
|
||||
根节点必须包含且仅包含以下 7 个字段,类型不可变更:
|
||||
|
||||
| 字段名 | 类型 | 空值处理 |
|
||||
|--------|------|---------|
|
||||
| `basic_info` | object | 必填,所有子字段必须存在 |
|
||||
| `reimbursement_details` | object | 必填,必须且仅含 3 个子字段 |
|
||||
| `payment_methods` | array | 无数据时返回 `[]` |
|
||||
| `subsidy_list` | array | 无数据时返回 `[]` |
|
||||
| `attachments` | array | 无数据时返回 `[]` |
|
||||
| `can_submit` | boolean | 必填,信息完整且逻辑自洽时为 `true`,否则为 `false` |
|
||||
| `suggestion` | string | 当 `can_submit` 为 `false` 时说明需补充的材料;为 `true` 时为空字符串 |
|
||||
|
||||
### 报销明细节点结构
|
||||
|
||||
`reimbursement_details` 必须包含且仅包含以下 3 个子字段,均为数组类型:
|
||||
|
||||
| 子字段 | 类型 | 空值处理 |
|
||||
|--------|------|---------|
|
||||
| `transport_fee` | array | 无数据时返回 `[]` |
|
||||
| `hotel_fee` | array | 无数据时返回 `[]` |
|
||||
| `conference_fee` | array | 无数据时返回 `[]` |
|
||||
|
||||
### 绝对禁止行为
|
||||
|
||||
- 省略任何根节点字段或报销明细节点
|
||||
- 将数组类型赋值为 `null`、字符串、数字或对象
|
||||
- 在 `reimbursement_details` 中添加未定义的子字段
|
||||
- 合并不同模块的数组数据(如将去程和返程交通费合并为一条)
|
||||
|
||||
### 正确输出骨架
|
||||
|
||||
```json
|
||||
{
|
||||
"basic_info": { /* 所有子字段完整存在 */ },
|
||||
"reimbursement_details": {
|
||||
"transport_fee": [ /* 去程一条,返程一条,可以一个人单独一条,也可以多人合并一条 */ ],
|
||||
"hotel_fee": [],
|
||||
"conference_fee": []
|
||||
},
|
||||
"payment_methods": [],
|
||||
"subsidy_list": [],
|
||||
"attachments": [],
|
||||
"can_submit": true,
|
||||
"suggestion": ""
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 字段 Schema
|
||||
|
||||
### 1. `basic_info`(全部必填,无直接信息时给出合理猜测)
|
||||
|
||||
| 字段 | 类型 | 说明 | 推断优先级 |
|
||||
|------|------|------|-----------|
|
||||
| `travel_purpose` | string | 出差事由 | ①申请单事由 → ②根据发票信息总结 |
|
||||
| `travel_location` | string | 出差目的地 | ①交通工具目的地 → ②申请单说明 |
|
||||
| `start_date` | string | 出差开始日期,格式 `YYYY-MM-DD` | ①最早乘车日期 → ②申请单时间 → ③开票/付款日期 |
|
||||
| `end_date` | string | 出差结束日期,格式 `YYYY-MM-DD` | ①最晚乘车日期 → ②申请单时间 → ③开票/付款日期 |
|
||||
|
||||
**注意**:出差一定从阜阳出发。
|
||||
|
||||
---
|
||||
|
||||
### 2. `transport_fee` 数组元素(去程和返程分开,各为一条记录)
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `vehicle_type` | string | 枚举:`train` / `car` / `ship` / `personal_car` / `official_car` / `plane` / `rental_car` / `self_drive` |
|
||||
| `start_date` | string | 乘车日期,格式 `YYYY-MM-DD` |
|
||||
| `end_date` | string | 乘车日期,格式 `YYYY-MM-DD` |
|
||||
| `departure_place` | string | 出发地(通常为城市名称) |
|
||||
| `arrival_place` | string | 目的地(通常为城市名称) |
|
||||
| `amount` | number | 票价金额 |
|
||||
| `bill_count` | integer | 发票张数 |
|
||||
| `remark` | string | 基本信息,例:`王建锋和张国庆高铁票` |
|
||||
|
||||
---
|
||||
|
||||
### 3. `hotel_fee` 数组元素
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `checkin_date` | string | 入住日期,格式 `YYYY-MM-DD` |
|
||||
| `checkout_date` | string | 退房日期,格式 `YYYY-MM-DD` |
|
||||
| `days` | integer | `checkout_date - checkin_date`,结果 ≥ 0 |
|
||||
| `person_count` | integer | 住宿人数 |
|
||||
| `invoice_amount` | number | 酒店发票价税合计总额 |
|
||||
| `reimburse_amount` | number | 酒店付款记录合计总额 |
|
||||
| `remark` | string | 住宿人员姓名,例:`王建锋、张国庆住宿` |
|
||||
|
||||
**日期推断优先级**:①交通工具发票日期(最高)→ ②酒店发票信息 → ③申请单时间(可能不准)
|
||||
|
||||
**默认值**:单张酒店发票且无明确信息时,`days` 和 `person_count` 均默认为 1。
|
||||
|
||||
---
|
||||
|
||||
### 4. `conference_fee` 数组元素
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `bill_count` | integer | 会务费/培训费发票张数 |
|
||||
| `amount` | number | 会务费/培训费总金额 |
|
||||
| `remark` | string | 会务培训基本信息 |
|
||||
|
||||
---
|
||||
|
||||
### 5. `payment_methods` 数组元素(多少笔支付就多少条)
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `card_date` | string | 刷卡日期,格式 `YYYY-MM-DD` |
|
||||
| `card_amount` | number | 刷卡金额(元) |
|
||||
| `merchant` | string | 商户信息(高铁票统一为`中国铁路`) |
|
||||
| `remark` | string | 关联的发票信息,例:`王建锋和张国庆从阜阳西-合肥南高铁票` |
|
||||
|
||||
---
|
||||
|
||||
### 6. `subsidy_list` 数组元素(按出差人员数量决定条目数)
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `person_id` | string | 人员编号(无直接信息时给出合理编号) |
|
||||
| `person_name` | string | 出差人员姓名 |
|
||||
| `start_date` | string | 该人员出差开始日期,格式 `YYYY-MM-DD` |
|
||||
| `end_date` | string | 该人员出差结束日期,格式 `YYYY-MM-DD` |
|
||||
| `days` | integer | `end_date - start_date + 1` |
|
||||
|
||||
**日期推断**:优先取该人员个人的来回交通工具发票日期;无个人数据时取 `basic_info` 中的日期。
|
||||
|
||||
---
|
||||
|
||||
### 7. `attachments` 数组元素
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `filename` | string | 严格使用原始文件名,不得修改任何字符 |
|
||||
| `attachment_type` | string | 枚举:`invoice` / `other` |
|
||||
| `attachment_desc` | string | 文件基本信息描述 |
|
||||
|
||||
**排除规则**:`invoice_type` 为 `payment` 的记录不作为附件。
|
||||
|
||||
---
|
||||
|
||||
## 推理规则
|
||||
|
||||
### 数据推断优先级链
|
||||
|
||||
```
|
||||
日期推断:交通工具发票 > 申请单时间 > 开票/付款日期
|
||||
事由推断:申请单事由 > 发票信息总结
|
||||
地点推断:交通工具出发/目的地 > 申请单说明
|
||||
人员推断:车票姓名 > 住宿发票信息 > 申请单人员
|
||||
```
|
||||
|
||||
### 关键规则
|
||||
|
||||
1. **去回分开**:交通费的去程和返程必须分两条记录,禁止合并
|
||||
2. **住宿天数**:`days = checkout_date - checkin_date`,结果必须 ≥ 0
|
||||
3. **补助天数**:`days = end_date - start_date + 1`
|
||||
4. **支付记录排除**:付款记录不放入 `attachments`
|
||||
5. **合理猜测**:无直接信息时给出合理猜测,不得留空或返回 `null`
|
||||
|
||||
### 语义完整性校验
|
||||
|
||||
提取完成后,需判断信息是否足够支撑填报。根据校验结果设置根节点的 `can_submit`(boolean)和 `suggestion`(string)字段。
|
||||
|
||||
**校验维度**:
|
||||
|
||||
- 出差日期范围是否合理(结束日期不早于开始日期)
|
||||
- 交通费的去程和返程日期是否在出差日期范围内
|
||||
- 支付金额总和是否与发票金额总和接近
|
||||
- 通常每张发票都要有对应的支付记录
|
||||
- 是否缺少发票
|
||||
- 是否缺少支付记录
|
||||
- 人员信息是否完整
|
||||
|
||||
**不需要关注的**
|
||||
- 非必填项没有填写信息,不要提醒补充
|
||||
- 酒店住宿有发票就行,不需要别的证明
|
||||
|
||||
|
||||
**一定要关注的**
|
||||
- `reimbursement_details` 和 `payment_methods` 两个的总金额应该一样,如果不一样,要么是缺发票,要么是缺支付记录,需要提醒用户
|
||||
- 用户一定要提供出差事情申请单
|
||||
|
||||
**判定标准**:
|
||||
|
||||
- `can_submit = true`:信息完整且逻辑自洽,`suggestion` 为空字符串
|
||||
- `can_submit = false`:存在信息缺失或逻辑矛盾,`suggestion` 说明需要用户补充什么材料
|
||||
|
||||
### 示例
|
||||
|
||||
**补助清单**(2 人出差,6月1日至6月3日):
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"person_id": "xxxxxxx",
|
||||
"person_name": "张三",
|
||||
"start_date": "2026-06-01",
|
||||
"end_date": "2026-06-03",
|
||||
"days": 3
|
||||
},
|
||||
{
|
||||
"person_id": "2024xxxxx",
|
||||
"person_name": "李四",
|
||||
"start_date": "2026-06-01",
|
||||
"end_date": "2026-06-03",
|
||||
"days": 3
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
**支付方式**(高铁票付款):
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"card_date": "2026-06-01",
|
||||
"card_amount": 231.0,
|
||||
"merchant": "中国铁路网络有限公司",
|
||||
"remark": "张国庆和王建锋从阜阳西-合肥南高铁票"
|
||||
},
|
||||
{
|
||||
"card_date": "2026-06-01",
|
||||
"card_amount": 167.0,
|
||||
"merchant": "中国铁路网络有限公司",
|
||||
"remark": "陈曙光从阜阳西-合肥南高铁票"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 最终输出要求
|
||||
|
||||
- 严格只输出 JSON 字符串,不包含任何思考过程、解释文字、Markdown 标记或其他内容
|
||||
- JSON 语法必须正确,无多余逗号、引号等错误
|
||||
- 严格遵守所有类型约束,任何违反均视为无效输出
|
||||
- 日期统一使用 `YYYY-MM-DD` 格式
|
||||
- 金额使用数字类型(非字符串)
|
||||
- 计数使用整数类型
|
||||
@@ -1,17 +1,14 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
财务报销自动化
|
||||
|
||||
依次执行:
|
||||
1. 发票提取 — 从 PDF 发票提取信息,生成 invoice_summary.csv
|
||||
2. OCR 识别 — 从支付截图识别刷卡信息,回填 CSV
|
||||
3. 报销提交 — 打开浏览器登录财务系统并自动填报
|
||||
2. 报销提交 — 打开浏览器登录财务系统并自动填报
|
||||
|
||||
用法:
|
||||
python run.py # 全流程
|
||||
python run.py --step invoice # 仅发票提取
|
||||
python run.py --step ocr # 仅 OCR 识别
|
||||
python run.py --step submit # 仅浏览器填报
|
||||
python run.py -u 工号 -p 密码 # 覆盖登录凭据
|
||||
"""
|
||||
@@ -21,30 +18,37 @@ import sys
|
||||
from pathlib import Path
|
||||
|
||||
# 确保项目根目录在 sys.path 中
|
||||
sys.path.insert(0, str(Path(__file__).parent.resolve()))
|
||||
sys.path.insert(0, str(Path(__file__).parent.resolve().parent))
|
||||
|
||||
from app.pipeline import run_pipeline
|
||||
from src.pipeline import run_pipeline
|
||||
|
||||
|
||||
def main():
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="财务报销自动化 - 发票提取 → OCR 识别 → 浏览器填报",
|
||||
description="财务报销自动化 - 发票提取 → 浏览器填报",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--step",
|
||||
choices=["all", "invoice", "ocr", "submit"],
|
||||
choices=["all", "invoice", "submit"],
|
||||
default="all",
|
||||
help="执行步骤 (默认: all)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-u", "--username",
|
||||
"-u",
|
||||
"--username",
|
||||
default=None,
|
||||
help="信息门户登录账号(覆盖 config.json)",
|
||||
help="信息门户登录账号(覆盖 scripts/config.json)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-p", "--password",
|
||||
"-p",
|
||||
"--password",
|
||||
default=None,
|
||||
help="信息门户登录密码(覆盖 config.json)",
|
||||
help="信息门户登录密码(覆盖 scripts/config.json)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--cache-dir",
|
||||
default=None,
|
||||
help="发票缓存目录(包含 .invoice_cache 子目录,默认: scripts/data)",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
@@ -52,9 +56,10 @@ def main():
|
||||
step=args.step,
|
||||
username=args.username,
|
||||
password=args.password,
|
||||
cache_dir=args.cache_dir,
|
||||
)
|
||||
sys.exit(exit_code)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
main()
|
||||
119
src/pipeline.py
Normal file
119
src/pipeline.py
Normal file
@@ -0,0 +1,119 @@
|
||||
"""
|
||||
报销全流程编排
|
||||
|
||||
将发票提取 → 差旅/普通信息提取 → 浏览器填报串联为一条管道,
|
||||
数据在内存中流转,同时生成 CSV 中间产物。
|
||||
|
||||
发票类型区分:
|
||||
- 差旅发票(train/hotel):不生成易耗品出库单,走差旅报销流程
|
||||
- 普通发票(general):生成易耗品出库单,走普通报销流程
|
||||
|
||||
数据流变更(2026-06-11):
|
||||
在发票提取和匹配完成后立即判断报销类型(差旅/普通),
|
||||
差旅调用 LLM 提取 travel_info.json,普通预留 normal_info.json。
|
||||
Bot 仅负责接收信息并填报,不再承担信息提取职责。
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from . import get_logger
|
||||
from .config import load_config
|
||||
from .core.extraction import extract_invoices
|
||||
from .pipeline_core import (
|
||||
extract_and_cache_normal_info,
|
||||
extract_and_cache_travel_info,
|
||||
extract_info_by_type,
|
||||
is_travel_invoice,
|
||||
process_invoices,
|
||||
)
|
||||
|
||||
log = get_logger("pipeline")
|
||||
|
||||
|
||||
def run_pipeline(
|
||||
step: str = "all",
|
||||
username: str | None = None,
|
||||
password: str | None = None,
|
||||
cache_dir: str | None = None,
|
||||
) -> int:
|
||||
"""执行报销流程
|
||||
|
||||
Args:
|
||||
step: all | invoice | submit
|
||||
username: 覆盖 config.json 中的用户名
|
||||
password: 覆盖 config.json 中的密码
|
||||
cache_dir: 发票缓存目录(包含 .invoice_cache 子目录)
|
||||
"""
|
||||
config = load_config()
|
||||
if username:
|
||||
config["username"] = username
|
||||
if password:
|
||||
config["password"] = password
|
||||
|
||||
project_dir = Path(__file__).parent.parent
|
||||
cache_path = Path(cache_dir) if cache_dir else project_dir / "scripts" / "data"
|
||||
|
||||
# --------------------------------------------------
|
||||
# Step 1: 发票提取 + 类型判断 + 信息提取
|
||||
# --------------------------------------------------
|
||||
payment_records: list[dict[str, str]] | None = None
|
||||
groups: dict[str, list[dict[str, str]]] | None = None
|
||||
travel_info: dict[str, Any] | None = None
|
||||
normal_info: dict[str, Any] | None = None
|
||||
|
||||
if step in ("all", "invoice"):
|
||||
log.info("=" * 60)
|
||||
log.info("[1/2] 发票提取")
|
||||
log.info("=" * 60)
|
||||
|
||||
payment_records, applications, groups = extract_invoices(str(cache_path))
|
||||
if not payment_records:
|
||||
log.error("未提取到任何发票数据")
|
||||
return 1
|
||||
|
||||
# 使用公共函数处理发票数据
|
||||
process_invoices(payment_records, applications, groups, cache_path)
|
||||
|
||||
# 发票提取完成后立即判断类型并提取信息
|
||||
travel_info, normal_info = extract_info_by_type(groups, cache_path)
|
||||
|
||||
if step == "invoice":
|
||||
log.info("[1/2] 发票提取 完成")
|
||||
return 0
|
||||
|
||||
# --------------------------------------------------
|
||||
# Step 2: 浏览器填报
|
||||
# --------------------------------------------------
|
||||
if step in ("all", "submit"):
|
||||
log.info("=" * 60)
|
||||
log.info("[2/2] 报销提交")
|
||||
log.info("=" * 60)
|
||||
|
||||
from .infra.browser import run_bot
|
||||
|
||||
if groups is None:
|
||||
# 从缓存重新分类(仅 submit 阶段需要)
|
||||
from .pipeline_core import _classify_from_cache
|
||||
|
||||
groups = _classify_from_cache(cache_path)
|
||||
|
||||
if is_travel_invoice(groups):
|
||||
if travel_info is None:
|
||||
travel_info = extract_and_cache_travel_info(groups, cache_path)
|
||||
log.info("检测到纯差旅发票,使用差旅报销模式")
|
||||
run_bot(config, work_dir=cache_path, travel_info=travel_info)
|
||||
else:
|
||||
if normal_info is None:
|
||||
normal_info = extract_and_cache_normal_info(groups, cache_path)
|
||||
log.info("检测到普通发票,使用普通报销模式")
|
||||
run_bot(config, work_dir=cache_path, normal_info=normal_info)
|
||||
|
||||
if step == "submit":
|
||||
log.info("[2/2] 报销提交 完成")
|
||||
return 0
|
||||
|
||||
log.info("=" * 60)
|
||||
log.info("全流程执行完毕")
|
||||
log.info("=" * 60)
|
||||
return 0
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user