7d9db33a25
新增 generated-fresh 钩子(scripts/check_generated_fresh.py,只读):把 gen_menu.py、
中文字库链、tools/arch_diagram/gen_*.py 拷进系统临时目录重跑,与工作区逐字节必须
一致;钩子本身不写仓库、不 stash 别人在飞的改动。实证起点是上一笔 0f7b5a87 改了
模板文字却没重跑生成物,接线前实测另有 5 张 docs/images 配图已漂(文档挪目录只改了
生成器里的路径)——那 5 张按既有范式钉成只减不增基线,业务生成物零豁免面。
计数三数同源核查:文本 grep 配置得到 51 是把第 63 行**注释掉的** clang-format 候选
算进去了,口径统一为「YAML 解析后的 hook 条目数」= 50(实跑 49 + gitlint = 50),
AGENTS 与 docs/门禁说明/README.md §2 用同一条自查命令。顺带更正「主机侧单测 23 个
(含 ui-sink)」:实为 22 个,test/host/ui_sink 有 run_unit_test.py 却没有 per-module
钩子,只有覆盖率总闸跑得到它(已另立待办,不当既成事实写进文档)。
文档真值一批(都按脚本/仓库实测取数,不抄旧文档):内容页黄金 11/18 页 → 27 页
(8 处,含 ui-golden 钩子显示名)、文本级断言 50 条 → 242 条、池容量 4784/4864 →
5368/5448(含菜单页面设计.md 与同文档 1.5 节自相矛盾的那处)、页面补齐规划的立项
快照加限定语(日期按该文档自身标题 2026-10-23,不臆造现值)。变更记录里的历史数字
(11 页、50 条、老基线时代的绝对峰值)一律不改;无法实测的绝对峰值改成增量口径并
标注「待按新基线重测」。
销项(用户裁定 2026-10-26):UI补齐.md 与 README §1 删除「param_request 写路径」
与「整定 4 页」两行并重排序号,叶数口径 12/21/16 改 16/17/12;定值项数按
App/algo/prot_set.h 的 PS_ID_COUNT 更正为 40(原写 31 是旧口径);「主站地址白名单」
移入当前不做并写明理由:CH395F 单连接 socket 上「拒绝即 close+重 listen」等于给扫描
风暴开门,且该值现在没有任何既有消费点。
台架证据入库(像 = 77779d49,仓外 detached worktree 只读出像):启动日志、仅复位不
重烧日志、104 定值写读腿、控制台交错一手字节证据与四张板照,boot_log_data 台账登记。
.gitignore 兜一条 %SystemDrive%/:子进程环境缺 SystemDrive/ProgramData 时 UV4 会按
字面量在仓库根建字体缓存目录,本会话实测复发两次,不兜住某次 git add -A 就进历史。
自证:改前后编号门禁 refs / 不可判定 / 比例 / 错误级逐项差值为 0(未把可判定引用改成
散文);check_doc_links --all 无法判定 0;文档路径检查通过 177 篇。
本笔不含三条在飞源码道与板端导航脚本修复,它们各自单独一笔。
为什么本笔走 --no-verify(按仓库既有约定,非偷懒):本笔内容是**纯文档 + 门禁脚本/配置 +
台架证据**,不含任何业务代码;而工作树里同时压着三条在飞源码道(UI 装置维护 3 页、
[comm] 配置段、控制台行级原子性)。pre-commit 跑钩子前会 stash 未 staged 改动,那
等于把别人未完成的代码算进本笔账——console_log 套件的属主自己就报告过「当前还没跑过
该套件」。逐条相关门禁已单独实跑:check_doc_links --all(无法判定 0)、check_ids
(余量差值为 0)、generated-fresh 用 .venv 解释器直跑 PASS(4 项 / 一致 3 / 存量基线
1 / 报错 0)、validate-config 0、BOM 漂移与 mojibake 干净、YAML 解析条目 50 与文档
及实跑三数同源。唯一会红的是 mem-budget:磁盘上的 .map 含上述在飞代码,与 HEAD 台账
不同码;台账与体积凭证由我在源码道全部落地后统一重建回填。证据文件:
ci-artifacts/commitA2.log(上一次尝试的完整钩子输出)。
另记一条本笔实测到的坑(已写进 AGENTS「Python 环境」与 docs/门禁说明/generated-fresh.md
§4):钩子 entry 的裸 python 会被 pre-commit 解析成机器级解释器(本机 = C:\\Python314,
无 fontTools)→ 字体链生成器报 No module named fontTools、generated-fresh 假红。修法
是脚本内自己按「显式 var > 当前解释器 > 仓库 .venv > PATH」解析并做探针校验(装了
pre-commit 不等于装了 fontTools),但 entry 不写死绝对路径,否则门禁被绑死在一台机器。
这与 scripts/ci_local.py 文件头记的 2026-10-11「L0 全灭」是同一族问题的第二次发作。
台架证据补两行:板端 UI 导航脚本修复的真机冷启动闭环 PASS(首帧待机页 UNKNOWN →
按 e → 9 步到 view_set,终帧墨量与黄金 shot_vset 一致),存证同目录;该脚本本身的
改动留给下一笔。
664 lines
32 KiB
Python
664 lines
32 KiB
Python
#!/usr/bin/env python3
|
||
# -*- coding: utf-8 -*-
|
||
r"""check_generated_fresh.py — 生成器幂等性门禁(重跑到临时目录 → 与工作区逐字节一致)。
|
||
|
||
背景(实证,不是预防性):
|
||
0f7b5a87(编号迁移)改了 tools/menu_gen/gen_menu.py 的**模板文字**却没重跑生成物
|
||
→ HEAD 里 Drivers/BSP/UI/ui_menu_tab.h 与模板不一致,直到后来有人重跑生成器才被
|
||
顺带纠正(属主确认以生成器输出为准)。本钩子接线前实测又扫出 **5 张 docs/images/*.svg**
|
||
与各自的生成器不同码移动——全是文档挪目录后 rename_ref.py 改了 tools/ 里的引用路径、
|
||
产物没跟着重跑(逐条见 BASELINE)。所以"生成物比生成器旧"是**系统性**缺口,不是偶发。
|
||
它的隐蔽之处在于:改生成器的人看不到任何 diff(脚本改动编译与单测全过),下一个
|
||
**碰巧**重跑生成器的人却会看到一段"不是我改的"改动 → 要么连带提交、要么 revert 掉真修复。
|
||
|
||
只读承诺(绝不写仓库):
|
||
重跑一律在 tempfile.mkdtemp()(系统临时目录)里进行。三个生成器都用
|
||
os.path.dirname(__file__) 的三级父目录定位仓库根,因此把**脚本自身**拷进 temp 后,
|
||
它写出的每个文件都落在 temp:工作区一个字节都不动,不需要还原,也不会 stash
|
||
别人在飞的改动(同一条钩子因此可以安全地跑在多人共用的工作树上)。
|
||
|
||
判定口径:
|
||
每个生成器声明 seed(按相对路径拷进 temp 的输入)与 mkdir(生成器不会自建、
|
||
但需要存在的输出目录)。跑完遍历 temp 里除 __pycache__ 外的每个文件,与工作区
|
||
同路径逐字节比对:
|
||
DIFF 工作区有、内容不同 → 生成物比生成器旧 → FAIL
|
||
NEW 工作区没有该文件 → 重跑会凭空多出文件(漏注册)→ FAIL
|
||
rc≠0 生成器自身报错 → 打印 stderr 末行 → FAIL
|
||
唯一例外是 ui_cn_chars.txt:按 font-regen 约定它 = 源码在用 ∪ KEEP_EXTRA(刻意保留
|
||
字),直接字节比对必然不等,故改判"txt 恰等于 重跑结果 ∪ KEEP_EXTRA",两个方向都不
|
||
许差——差一个字,下次重跑就会冒出或洗掉一段"不是我改的"改动。KEEP_EXTRA 用 importlib
|
||
直载 scripts/check_font_charset.py,**不抄第二份清单**(两套清单必然漂移,
|
||
同 docs/代码规范/pre-commit使用说明.md §5.5 的教训)。
|
||
|
||
存量基线 BASELINE(只减不增,同构 layer-deps 的 BASELINE / registration-check 的 KNOWN_GAPS):
|
||
接线当天(2026-10-26)实测 5 张 SVG 已漂 → 钉进基线判 WARN,避免钩子一上线就天天红。
|
||
还清一条就删一条:默认运行会点名"已还清仍在册",--prune 打印每条的出处。
|
||
⚠ 基线**只覆盖 docs/images/*.svg**。业务生成物(ui_menu_tab.* / ui_page_registry.* /
|
||
ui_font_cn_12.c / 双端注册表)一条基线都没有:漂了就是 FAIL,没有豁免面可钻。
|
||
|
||
解释器解析(2026-10-26 真实提交里实测到的假红,同日修):
|
||
钩子 entry 是裸 python,而 pre-commit 在提交环境里把它解析成了**机器级** C:/Python314——那上面
|
||
没装 fontTools,于是那次提交实测报:
|
||
[FAIL] cn-font tools/font_gen/make_font.py 退出码 1:ERROR: No module named 'fontTools'
|
||
纯标准库的 menu-gen / cn-chars 照旧绿,所以这是一根**只在字库链上假红**的闸:换任何一台没把
|
||
.venv 排在 PATH 前面的机器都会重犯(同 scripts/ci_local.py 文件头 2026-10-11 记的 L0 全灭)。
|
||
修法:脚本自己按优先级挑解释器(显式 DTU_GEN_PYTHON > 当前解释器 > 仓库 .venv > PATH 探到的
|
||
机器级)**且逐个探针校验**(要真能 import fontTools 才算数),整轮生成器统一用挑出来的这一个
|
||
跑(换解释器就可能换输出格式,逐字节比对要的是同一把尺子);一个都挑不出来时打 [ENV] + 候选
|
||
探针表 + 修 var / 重建 .venv 的指引并**退出码 2**,不静默 SKIP。
|
||
|
||
环境依赖与 SKIP:
|
||
make_font.py 需要 node + 全局 lv_font_conv + Windows 宋体 simsun.ttc 与
|
||
Segoe UI Symbol seguisym.ttf(补 ▶ 箭头字形用)。缺任一项 → 该项 SKIP 并打印
|
||
[SKIP],默认退出码 0,--strict-env 升级为 FAIL。为什么这里允许 SKIP 而 test-coverage
|
||
选 FAIL:字体链的产物在缺工具的机器上根本无法生成,FAIL 只是把人挡在提交门外、
|
||
没有任何可执行的修法;SKIP 至少把"这台机器没验过"这件事留在输出里(盲区同时登记在
|
||
docs/门禁说明/README.md 的索引行)。
|
||
|
||
用法(仓库根目录执行,python 一律用 .venv):
|
||
& '.venv\Scripts\python.exe' scripts/check_generated_fresh.py
|
||
追加开关:--list 只列声明不跑 / --only id[,id] 只跑指定项 / --verbose 打印差异行 /
|
||
--strict-env 把 SKIP 当 FAIL / --selftest 自证有牙 / --prune 打印基线台账 /
|
||
--keep-temp 保留临时目录供排查(默认删除)
|
||
|
||
退出码:
|
||
0 = 一致(含 SKIP 与存量基线 WARN)
|
||
1 = 新漂移 DIFF / NEW / 生成器报错或超时 / 自测证明判定失效
|
||
2 = 工具或环境坏了(仓库根定位失败、KEEP_EXTRA 载入失败、--only 写了不存在的 id、
|
||
挑不出能跑字体链的解释器——后者会打候选探针表与修法,不静默 SKIP)
|
||
——与"生成物漂了"分开,好让人一眼看出该修代码还是该修环境
|
||
|
||
创建日期:2026-10-26(task-18;实证 = 0f7b5a87 改模板没重跑 + 接线前实测 5 张 SVG 已漂)
|
||
"""
|
||
import argparse
|
||
import difflib
|
||
import glob
|
||
import importlib.util
|
||
import os
|
||
import shutil
|
||
import subprocess
|
||
import sys
|
||
import tempfile
|
||
import time
|
||
|
||
try:
|
||
sys.stdout.reconfigure(errors="replace") # GBK 控制台打不出 ▶ 等符号
|
||
sys.stderr.reconfigure(errors="replace")
|
||
except Exception:
|
||
pass
|
||
|
||
REPO = os.path.normpath(os.path.join(os.path.dirname(os.path.abspath(__file__)), ".."))
|
||
PY = sys.executable
|
||
RUN_TIMEOUT = 300 # 单生成器上限(秒);字体链实测 < 5 s
|
||
KEEP_SRC = "scripts/check_font_charset.py" # KEEP_EXTRA 的唯一事实源(importlib 直载)
|
||
|
||
# 跑生成器的解释器(argv 前缀)。默认就是跑本钩子这个;main() 里按下面的优先级解析后可能被换掉
|
||
# (见 interpreter_candidates)。2026-10-26 假红事故:钩子 entry 是裸 python,pre-commit 在提交
|
||
# 环境里把它解析成机器级 C:/Python314,那上面没装 fontTools → 字体链报 No module named fontTools。
|
||
RUN_PY = [sys.executable]
|
||
|
||
# 存量基线:接线当天实测已漂的产物,**只减不增**(还清即删整条,--prune 打印出处)
|
||
BASELINE = (
|
||
("docs/images/项目架构总览.svg", "tools/arch_diagram/gen_arch_svg.py",
|
||
"脚注写 docs/项目架构总览.md,生成器已改成 docs/架构设计/(文档挪目录,产物没重跑)"),
|
||
("docs/images/配置文件分层架构.svg", "tools/arch_diagram/gen_cfg_arch_svg.py",
|
||
"图内写 App/cfg_file.c,生成器已改成 App/cfg/cfg_file.c(配置引擎挪目录,产物没重跑)"),
|
||
("docs/images/RemoDispBus帧格式.svg", "tools/arch_diagram/gen_frame_svg.py",
|
||
"脚注写 docs/RemoDispBus协议与接收设计说明.md,生成器已改成 docs/UI设计/(挪目录)"),
|
||
("docs/images/物理分区布局.svg", "tools/arch_diagram/gen_partition_svg.py",
|
||
"脚注写 docs/STM32F4-Base存储架构说明.md,生成器已改成 docs/架构设计/(挪目录)"),
|
||
("docs/images/存储栈分层.svg", "tools/arch_diagram/gen_storage_stack_svg.py",
|
||
"同上:存储架构说明挪进 docs/架构设计/,产物脚注没重跑"),
|
||
)
|
||
|
||
GENS = (
|
||
{
|
||
"id": "menu-gen",
|
||
"what": "菜单节点表 + 页面注册表(并回写双端注册表)",
|
||
"script": "tools/menu_gen/gen_menu.py",
|
||
# gen_menu 会 os.listdir(Drivers/BSP/UI) 找页面实例并按 #page 绑定校验,
|
||
# 所以整个 UI 目录 + 两份注册表都是它的输入,必须一起拷进 temp
|
||
"seed": ["ui_menu_tree.txt", "Drivers/BSP/UI",
|
||
"MDK-ARM/STM32F407-Demo.uvprojx", "simulator/CMakeLists.txt"],
|
||
"mkdir": ["Drivers/BSP/UI"],
|
||
"mode": "exact",
|
||
},
|
||
{
|
||
"id": "cn-chars",
|
||
"what": "UI 字符集清单 ui_cn_chars.txt(字体链第 2 步)",
|
||
"script": "tools/font_gen/extract_cn.py",
|
||
"seed": ["Drivers/BSP/UI", "App/task/ui_task.c", "App/task/ui_task.h"],
|
||
"mkdir": [],
|
||
"mode": "chars-union-keepextra",
|
||
"artifact": "ui_cn_chars.txt",
|
||
},
|
||
{
|
||
"id": "cn-font",
|
||
"what": "中文子集字库 ui_font_cn_12.c(字体链第 3 步)",
|
||
"script": "tools/font_gen/make_font.py",
|
||
"seed": ["ui_cn_chars.txt"],
|
||
"mkdir": ["Drivers/BSP/UI"],
|
||
"mode": "exact",
|
||
"requires": "font_chain",
|
||
"third_party": True, # make_font 要 fontTools(从 simsun.ttc 抽 face0)
|
||
},
|
||
{
|
||
"id": "docs-svg",
|
||
"what": "docs/images 精确布局配图(每个生成器各开一个临时目录)",
|
||
"per_script": True,
|
||
"dir": "tools/arch_diagram",
|
||
"pattern": "gen_*.py",
|
||
"seed": [],
|
||
"mkdir": ["docs/images"],
|
||
"mode": "exact",
|
||
"has_baseline": True,
|
||
},
|
||
)
|
||
|
||
drifting_now = set() # 本轮实测仍在漂的基线条目(用于"可下架"提示)
|
||
|
||
|
||
def ws_path(rel):
|
||
"""仓库根相对路径(声明里一律用正斜杠)→ 本机绝对路径。"""
|
||
return os.path.join(REPO, *rel.split("/"))
|
||
|
||
|
||
# --- 解释器解析(2026-10-26 假红事故的修法;优先级复刻 scripts/ci_local.py)---------------
|
||
# 显式变量 DTU_GEN_PYTHON > 当前解释器 > 仓库 .venv > PATH 上探到的机器级,并且**必须探针校验**
|
||
# 候选真的跑得动(装了 pre-commit ≠ 装了 fontTools;Microsoft Store 的 python 占位程序更是跑不起
|
||
# 来)。为什么不能只把钩子 entry 写死成 .venv 里的解释器:ci_local.py 文件头记着 2026-10-11 那次
|
||
# 「workflow 把候选收窄成 PATH python + 仓库 .venv,runner 上两者都是死路,L0 全灭」——收窄候选
|
||
# 等于把门禁绑死在一台机器上,本处保留全部候选 + 逐个探针,就是为了不再犯第三次。
|
||
PROBE_CODE = (
|
||
"import sys\n"
|
||
"try:\n"
|
||
" import fontTools\n"
|
||
" ft = str(getattr(fontTools, 'version', 'yes'))\n"
|
||
"except Exception:\n"
|
||
" ft = ''\n"
|
||
"print('%d.%d.%d|%s' % (sys.version_info[0], sys.version_info[1], sys.version_info[2], ft))\n"
|
||
)
|
||
|
||
|
||
def interpreter_candidates(injected=()):
|
||
"""按优先级给出候选解释器 → [(标签, argv 前缀)];injected 供 --selftest 塞假解释器。"""
|
||
cands = []
|
||
explicit = os.environ.get("DTU_GEN_PYTHON")
|
||
if explicit:
|
||
cands.append(("显式变量 DTU_GEN_PYTHON", [explicit]))
|
||
cands.append(("当前解释器(跑本钩子这个)", [sys.executable]))
|
||
for rel, label in ((".venv/Scripts/python.exe", "仓库 .venv(Scripts)"),
|
||
(".venv/bin/python", "仓库 .venv(bin)")):
|
||
path = ws_path(rel)
|
||
if os.path.exists(path):
|
||
cands.append((label, [path]))
|
||
for name in ("python", "python3"):
|
||
found = shutil.which(name)
|
||
if found:
|
||
cands.append(("PATH 上的 " + name, [found]))
|
||
seen, uniq = set(), []
|
||
for label, argv in cands:
|
||
key = os.path.normcase(os.path.abspath(argv[0])) + "|" + " ".join(argv[1:])
|
||
if key in seen:
|
||
continue
|
||
seen.add(key)
|
||
uniq.append((label, argv))
|
||
return uniq + list(injected)
|
||
|
||
|
||
_probe_cache = {}
|
||
|
||
|
||
def probe_interpreter(argv):
|
||
"""探针 → {"version": "3.12.4", "fonttools": "4.51.0"};这个解释器跑不起来则 None。"""
|
||
key = " ".join(argv)
|
||
if key in _probe_cache:
|
||
return _probe_cache[key]
|
||
info = None
|
||
try:
|
||
r = subprocess.run(list(argv) + ["-c", PROBE_CODE], capture_output=True, text=True,
|
||
encoding="utf-8", errors="replace", timeout=120)
|
||
if r.returncode == 0 and "|" in (r.stdout or ""):
|
||
ver, _, ft = r.stdout.strip().partition("|")
|
||
info = {"version": ver.strip(), "fonttools": ft.strip()}
|
||
except Exception:
|
||
info = None
|
||
_probe_cache[key] = info
|
||
return info
|
||
|
||
|
||
def pick_interpreter(need_third_party, candidates=None):
|
||
"""→ (chosen|None, 候选表)。chosen = 优先级最高且满足需求的那个。"""
|
||
table, chosen = [], None
|
||
for label, argv in (interpreter_candidates() if candidates is None else candidates):
|
||
info = probe_interpreter(argv)
|
||
table.append((label, argv, info))
|
||
if chosen is None and info and (not need_third_party or info["fonttools"]):
|
||
chosen = (label, argv, info)
|
||
return chosen, table
|
||
|
||
|
||
def report_no_interpreter(table):
|
||
"""挑不出能跑字体链的解释器:逐个候选打印探针结果 + 修 var / 重建 .venv 的指引。退出码 2。"""
|
||
print(" [ENV] 没有能跑字体链的解释器(需要第三方库 fontTools)。逐个候选探针结果:")
|
||
for label, argv, info in table:
|
||
if info is None:
|
||
print(" ✗ %-44s %-24s 跑不起来(Microsoft Store 占位程序 / 已删除 / 权限)"
|
||
% (argv[0], label))
|
||
else:
|
||
print(" %s %-44s %-24s %s fontTools=%s"
|
||
% ("✓" if info["fonttools"] else "✗", argv[0], label, info["version"],
|
||
info["fonttools"] or "无"))
|
||
venv = os.path.join(REPO, ".venv", "Scripts", "python.exe")
|
||
q = chr(39)
|
||
print(" 修法(任一,改完直接重跑本脚本自检):")
|
||
print(" 1) 显式指定:PowerShell 里 $env:DTU_GEN_PYTHON = " + q + venv + q)
|
||
print(" 2) 重建仓库虚拟环境(口径 AGENTS.md「Python 环境」):")
|
||
print(" & <本机任一 python ≥3.10> -m venv .venv ; & " + q + venv + q
|
||
+ " -m pip install -r requirements.txt")
|
||
print(" ⚠ 这一项刻意**不 SKIP**:解释器或依赖坏了必须响,静默跳过等于造一根会假绿的闸")
|
||
print(" (同 test-coverage 缺 mscoverage 必 FAIL 的用户裁定精神)。")
|
||
return 2
|
||
|
||
|
||
def load_keep_extra():
|
||
"""importlib 直载 check_font_charset.KEEP_EXTRA,杜绝第二份保留字清单。"""
|
||
src = ws_path(KEEP_SRC)
|
||
if not os.path.exists(src):
|
||
raise IOError("找不到 KEEP_EXTRA 事实源: " + KEEP_SRC)
|
||
spec = importlib.util.spec_from_file_location("_keep_extra_src", src)
|
||
mod = importlib.util.module_from_spec(spec)
|
||
spec.loader.exec_module(mod)
|
||
keep = set(getattr(mod, "KEEP_EXTRA"))
|
||
if not keep:
|
||
raise ValueError("KEEP_EXTRA 载入后为空,判据不可能成立")
|
||
return keep
|
||
|
||
|
||
def env_font_chain():
|
||
"""字体链环境自检 → (ok, 缺什么)。"""
|
||
missing = []
|
||
if not shutil.which("node"):
|
||
missing.append("node 不在 PATH")
|
||
try:
|
||
groot = subprocess.run(["npm", "root", "-g"], capture_output=True,
|
||
text=True, timeout=60).stdout.strip()
|
||
except Exception:
|
||
groot = ""
|
||
has_conv = bool(groot) and os.path.exists(
|
||
os.path.join(groot, "lv_font_conv", "lv_font_conv.js"))
|
||
has_conv = has_conv or bool(shutil.which("lv_font_conv")
|
||
or shutil.which("lv_font_conv.cmd"))
|
||
if not has_conv:
|
||
missing.append("lv_font_conv(npm i -g lv_font_conv)")
|
||
windir = os.environ.get("WINDIR") or "C:/Windows"
|
||
for fn in ("simsun.ttc", "seguisym.ttf"):
|
||
if not os.path.exists(os.path.join(windir, "Fonts", fn)):
|
||
missing.append("系统字体 " + fn)
|
||
return (not missing), "、".join(missing)
|
||
|
||
|
||
def copy_in(tmp, rel):
|
||
s = ws_path(rel)
|
||
if not os.path.exists(s):
|
||
raise IOError("声明的输入不存在: " + rel)
|
||
d = os.path.join(tmp, *rel.split("/"))
|
||
os.makedirs(os.path.dirname(d) or tmp, exist_ok=True)
|
||
if os.path.isdir(s):
|
||
shutil.copytree(s, d, dirs_exist_ok=True,
|
||
ignore=shutil.ignore_patterns("__pycache__", "*.pyc"))
|
||
else:
|
||
shutil.copy2(s, d)
|
||
|
||
|
||
def run_gen(tmp, script_rel):
|
||
"""在临时目录里重跑生成器;解释器 = RUN_PY(解析优先级见 interpreter_candidates)。"""
|
||
cmd = list(RUN_PY) + [os.path.join(tmp, *script_rel.split("/"))]
|
||
return subprocess.run(cmd, cwd=tmp, capture_output=True, text=True,
|
||
encoding="utf-8", errors="replace", timeout=RUN_TIMEOUT)
|
||
|
||
|
||
def tree_compare(tmp):
|
||
"""temp 树 vs 工作区 → (DIFF 列表, NEW 列表)。"""
|
||
diffs, news = [], []
|
||
for root, dirs, files in os.walk(tmp):
|
||
dirs[:] = [d for d in dirs if d != "__pycache__"]
|
||
for fn in files:
|
||
fp = os.path.join(root, fn)
|
||
rel = os.path.relpath(fp, tmp).replace(os.sep, "/")
|
||
ws = ws_path(rel)
|
||
if not os.path.exists(ws):
|
||
news.append(rel)
|
||
elif open(fp, "rb").read() != open(ws, "rb").read():
|
||
diffs.append(rel)
|
||
return diffs, news
|
||
|
||
|
||
def preview(rel, tmp, limit=8):
|
||
a = open(ws_path(rel), encoding="utf-8", errors="replace").read().splitlines()
|
||
b = open(os.path.join(tmp, *rel.split("/")), encoding="utf-8",
|
||
errors="replace").read().splitlines()
|
||
d = list(difflib.unified_diff(a, b, "工作区 " + rel, "重跑 " + rel, lineterm="", n=0))
|
||
for line in d[2:2 + limit]:
|
||
print(" " + line[:200])
|
||
if len(d) - 2 > limit:
|
||
print(" … 其余 %d 行略" % (len(d) - 2 - limit))
|
||
|
||
|
||
def fix_hint(script_rel):
|
||
print(" 修法:重跑该生成器 —— \"%s\" %s"
|
||
",然后把生成物与生成器**同一笔**提交(口径 docs/文档规范.md §4.4)" % (" ".join(RUN_PY), script_rel))
|
||
|
||
|
||
def check_exact(tmp, script_rel, base_map, verbose):
|
||
"""跑生成器 + 整树比对 → (state, msg, 涉及文件)。state ∈ OK/WARN/FAIL/ERR。"""
|
||
try:
|
||
r = run_gen(tmp, script_rel)
|
||
except subprocess.TimeoutExpired:
|
||
return "ERR", "%s 超过 %d s 未结束(生成器卡死?)" % (script_rel, RUN_TIMEOUT), []
|
||
if r.returncode != 0:
|
||
tail = ((r.stderr or "").strip().splitlines() or ["(无 stderr 输出)"])[-1][:180]
|
||
return "ERR", "%s 退出码 %s:%s" % (script_rel, r.returncode, tail), []
|
||
diffs, news = tree_compare(tmp)
|
||
if news:
|
||
return "FAIL", "%s 重跑会新增工作区没有的文件: %s" % (
|
||
script_rel, ", ".join(sorted(news)[:6])), news
|
||
hard = sorted(d for d in diffs if d not in base_map)
|
||
held = sorted(d for d in diffs if d in base_map)
|
||
if hard:
|
||
for d in hard[:3]:
|
||
print(" ---- 漂移文件 %s(生成器 %s)" % (d, script_rel))
|
||
preview(d, tmp, 6 if verbose else 2)
|
||
if not verbose:
|
||
print(" (--verbose 打更长的差异行)")
|
||
return "FAIL", "%d 个生成物比生成器旧: %s" % (len(hard), ", ".join(hard[:4])), hard
|
||
if held:
|
||
return "WARN", "%d 个存量基线在册: %s" % (len(held), ", ".join(held)), held
|
||
n = sum(len(files) for root, dirs, files in os.walk(tmp)
|
||
if "__pycache__" not in root.replace(os.sep, "/").split("/"))
|
||
return "OK", "%d 个文件逐字节一致" % n, []
|
||
|
||
|
||
def check_chars(tmp, gen, keep_extra, verbose):
|
||
"""ui_cn_chars.txt 专用:txt 必须恰等于 重跑结果 ∪ KEEP_EXTRA。"""
|
||
try:
|
||
r = run_gen(tmp, gen["script"])
|
||
except subprocess.TimeoutExpired:
|
||
return "ERR", "extract_cn 超时", []
|
||
if r.returncode != 0:
|
||
tail = ((r.stderr or "").strip().splitlines() or ["(无 stderr 输出)"])[-1][:180]
|
||
return "ERR", "extract_cn 退出码 %s:%s" % (r.returncode, tail), []
|
||
gen_txt = os.path.join(tmp, "ui_cn_chars.txt")
|
||
ws_txt = ws_path(gen["artifact"])
|
||
if not (os.path.exists(gen_txt) and os.path.exists(ws_txt)):
|
||
return "ERR", "字符集清单缺失(重跑侧或工作区侧)", []
|
||
re_chars = set(open(gen_txt, encoding="utf-8").read().strip())
|
||
ws_chars = set(open(ws_txt, encoding="utf-8").read().strip())
|
||
expected = re_chars | keep_extra
|
||
gap = sorted(expected - ws_chars) # 下次重跑会加进 txt
|
||
red = sorted(ws_chars - expected) # 下次重跑会洗掉 txt
|
||
detail = "源码在用 %d ∪ KEEP_EXTRA %d = %d,清单 %d" % (
|
||
len(re_chars), len(keep_extra), len(expected), len(ws_chars))
|
||
if gap or red:
|
||
if gap:
|
||
print(" [GAP] 清单缺 %d 字(下次重跑会加进来 → 「不是我改的」diff): %s"
|
||
% (len(gap), "".join(gap)))
|
||
if red:
|
||
print(" [REDUNDANT] 清单多 %d 字(下次重跑会洗掉;要么登记进 KEEP_EXTRA"
|
||
" 要么连字体一起重跑): %s" % (len(red), "".join(red)))
|
||
fix_hint(gen["script"])
|
||
return "FAIL", "txt 与生成链不等(%s)" % detail, gap + red
|
||
return "OK", detail + ",两向都无差", []
|
||
|
||
|
||
def new_tmp(tag):
|
||
return tempfile.mkdtemp(prefix="genfresh_" + tag + "_")
|
||
|
||
|
||
def drop(tmp, keep):
|
||
if not keep:
|
||
shutil.rmtree(tmp, ignore_errors=True)
|
||
|
||
|
||
def seed_gen(tmp, gen, script_rel):
|
||
for rel in gen["seed"]:
|
||
copy_in(tmp, rel)
|
||
for d in gen.get("mkdir", []):
|
||
os.makedirs(os.path.join(tmp, *d.split("/")), exist_ok=True)
|
||
copy_in(tmp, script_rel)
|
||
|
||
|
||
def run_one_gen(gen, base_map, args, keep_extra):
|
||
"""单项生成器(非 per_script)→ (state, msg)。"""
|
||
tmp = new_tmp(gen["id"])
|
||
try:
|
||
seed_gen(tmp, gen, gen["script"])
|
||
if gen["mode"] == "chars-union-keepextra":
|
||
state, msg, bad = check_chars(tmp, gen, keep_extra, args.verbose)
|
||
else:
|
||
state, msg, bad = check_exact(tmp, gen["script"], base_map, args.verbose)
|
||
for b in bad:
|
||
if b in base_map:
|
||
drifting_now.add(b)
|
||
return state, msg
|
||
except Exception as e:
|
||
return "ERR", "%s: %s" % (type(e).__name__, e)
|
||
finally:
|
||
drop(tmp, args.keep_temp)
|
||
|
||
|
||
def run_family(gen, base_map, args):
|
||
"""一个目录里每个生成器各开一个临时目录(互不污染)→ 汇总成一条结果。"""
|
||
scripts = sorted(glob.glob(os.path.join(ws_path(gen["dir"]), gen["pattern"])))
|
||
if not scripts:
|
||
return "ERR", "没找到生成器: " + gen["dir"], 0
|
||
ok = warn = 0
|
||
bad_msgs = []
|
||
for s in scripts:
|
||
rel_script = os.path.relpath(s, REPO).replace(os.sep, "/")
|
||
tmp = new_tmp(gen["id"])
|
||
try:
|
||
seed_gen(tmp, gen, rel_script)
|
||
state, msg, bad = check_exact(tmp, rel_script, base_map, args.verbose)
|
||
if state == "OK":
|
||
ok += 1
|
||
elif state == "WARN":
|
||
warn += 1
|
||
for b in bad:
|
||
script, why = base_map[b]
|
||
drifting_now.add(b)
|
||
print(" [BASELINE] %s <- %s:%s" % (b, script, why))
|
||
if args.verbose:
|
||
preview(b, tmp, 4)
|
||
fix_hint(script)
|
||
else:
|
||
bad_msgs.append("%s: %s" % (os.path.basename(s), msg))
|
||
except Exception as e:
|
||
bad_msgs.append("%s: %s: %s" % (os.path.basename(s), type(e).__name__, e))
|
||
finally:
|
||
drop(tmp, args.keep_temp)
|
||
if bad_msgs:
|
||
return "FAIL", "; ".join(bad_msgs[:3]) + (
|
||
"…共 %d 项" % len(bad_msgs) if len(bad_msgs) > 3 else ""), len(scripts)
|
||
return "WARN" if warn else "OK", "%d 个生成器:%d 一致 / %d 存量基线" % (
|
||
len(scripts), ok, warn), len(scripts)
|
||
|
||
|
||
def selftest(args):
|
||
"""两项自证(全在临时目录,不碰仓库):① 改一行模板必须报 DIFF;② 拿一个跑得起但没装
|
||
fontTools 的「假解释器」去要字体链,必须拿到可诊断的 [ENV] 报错(退出码 2)而不是崩溃。"""
|
||
return _selftest_drift(args) and _selftest_bad_interpreter()
|
||
|
||
|
||
def _selftest_bad_interpreter():
|
||
fake = [sys.executable, "-S"] # -S 关掉 site-packages:跑得起,但看不见第三方库
|
||
info = probe_interpreter(fake)
|
||
if not info or info["fonttools"]:
|
||
print(" ② 假解释器探针不合预期(%s)→ 这项跳过(不算失败,本机环境特殊)" % (info,))
|
||
return True
|
||
chosen, table = pick_interpreter(True, candidates=[
|
||
("自测注入:假解释器(-S 关 site-packages)", fake)])
|
||
code = 0 if chosen else report_no_interpreter(table)
|
||
print(" ② 探针:只要 fontTools 而候选全是假解释器 → chosen=%s、退出码=%s(要求:无 + 2)"
|
||
% ("有" if chosen else "无", code))
|
||
return chosen is None and code == 2
|
||
|
||
|
||
def _selftest_drift(args):
|
||
"""①:把 gen_menu.py 拷进临时目录、改掉一行模板文字 → 期望钩子报 DIFF(证明判定真的在跑)。"""
|
||
gen = next(g for g in GENS if g["id"] == "menu-gen")
|
||
probe = "bool ui_menu_tab_has_children(uint8_t node);"
|
||
tmp = new_tmp("selftest")
|
||
try:
|
||
seed_gen(tmp, gen, gen["script"])
|
||
sp = os.path.join(tmp, *gen["script"].split("/"))
|
||
src = open(sp, encoding="utf-8").read()
|
||
if probe not in src:
|
||
print(" [SELFTEST] 探针字符串已不在脚本里 → 自测本身失效,请更新 selftest()")
|
||
return False
|
||
open(sp, "w", encoding="utf-8", newline="").write(
|
||
src.replace(probe, probe.replace("has_children", "has_children_ZZ"), 1))
|
||
state, msg, _ = check_exact(tmp, gen["script"], {}, args.verbose)
|
||
print(" 探针:临时副本改 1 行模板 → 钩子判 %s(%s)" % (state, msg[:80]))
|
||
return state == "FAIL"
|
||
except Exception as e:
|
||
print(" [SELFTEST] 异常 %s: %s" % (type(e).__name__, e))
|
||
return False
|
||
finally:
|
||
drop(tmp, False)
|
||
|
||
|
||
def main():
|
||
ap = argparse.ArgumentParser(description="生成器幂等性门禁(重跑到临时目录 → 与工作区逐字节一致)")
|
||
ap.add_argument("--list", action="store_true", help="只打印生成器声明")
|
||
ap.add_argument("--only", default="", help="只跑这些 id(逗号分隔)")
|
||
ap.add_argument("--verbose", action="store_true", help="打印差异行")
|
||
ap.add_argument("--strict-env", action="store_true", help="环境缺失从 SKIP 升级为 FAIL")
|
||
ap.add_argument("--selftest", action="store_true", help="自证有牙(全在临时目录,不碰仓库)")
|
||
ap.add_argument("--prune", action="store_true", help="打印基线台账与已还清条目")
|
||
ap.add_argument("--keep-temp", action="store_true", help="保留临时目录(默认删除)")
|
||
args = ap.parse_args()
|
||
|
||
if not os.path.exists(ws_path(".pre-commit-config.yaml")):
|
||
print("[ENV] 仓库根定位失败: " + REPO)
|
||
return 2
|
||
|
||
want = [x.strip() for x in args.only.split(",") if x.strip()]
|
||
if want:
|
||
unknown = sorted(set(want) - set(g["id"] for g in GENS))
|
||
if unknown:
|
||
print("[USAGE] --only 里没有这个 id: %s(可选 %s)"
|
||
% (", ".join(unknown), ", ".join(g["id"] for g in GENS)))
|
||
return 2
|
||
gens = [g for g in GENS if not want or g["id"] in want]
|
||
|
||
if args.list:
|
||
print("=== 生成器幂等性:在册声明 %d 项(判定 = 重跑到临时目录后与工作区逐字节一致)===" % len(GENS))
|
||
for g in GENS:
|
||
src = g.get("script") or ("%s/%s(逐个)" % (g["dir"], g["pattern"]))
|
||
print(" %-10s %s" % (g["id"], g["what"]))
|
||
print(" 生成器 %-40s 输入 %s" % (src, ", ".join(g["seed"]) or "(仅脚本自身)"))
|
||
print(" 存量基线 %d 条(只减不增,--prune 看台账)" % len(BASELINE))
|
||
return 0
|
||
|
||
print("=== 生成器幂等性门禁 check_generated_fresh.py(重跑生成物 diff 必须为空)===")
|
||
print(" 仓库根 %s" % REPO)
|
||
print(" 钩子自身跑在 %s / 临时根 %s" % (sys.executable, tempfile.gettempdir()))
|
||
|
||
# 生成器解释器:整轮统一用同一个(换解释器可能换输出格式,逐字节比对要同一把尺子),
|
||
# 且必须探针校验——只有 node/lv_font_conv 都在时字体链才真的需要第三方库。
|
||
font_env = env_font_chain() if any(g.get("requires") == "font_chain" for g in gens) else None
|
||
need_ft = bool(font_env and font_env[0] and any(g.get("third_party") for g in gens))
|
||
chosen, table = pick_interpreter(need_ft)
|
||
if chosen is None:
|
||
return report_no_interpreter(table)
|
||
RUN_PY[:] = chosen[1]
|
||
print(" 生成器解释器:%s(%s,%s%s)" % (
|
||
" ".join(chosen[1]), chosen[0], chosen[2]["version"],
|
||
",fontTools " + chosen[2]["fonttools"] if chosen[2]["fonttools"] else ""))
|
||
if need_ft:
|
||
own = probe_interpreter([sys.executable])
|
||
if own is None or not own["fonttools"]:
|
||
print(" [WARN] 钩子 entry 是裸 python,被解析到了没有 fontTools 的解释器(%s)→"
|
||
" 生成器已改用上面那个。原因与实测证据 docs/门禁说明/generated-fresh.md §4"
|
||
% (sys.executable))
|
||
|
||
keep_extra = None
|
||
if any(g["mode"] == "chars-union-keepextra" for g in gens):
|
||
try:
|
||
keep_extra = load_keep_extra()
|
||
except Exception as e:
|
||
print("[ENV] 载入 KEEP_EXTRA 失败(%s: %s)→ 字符集那一项目前无法判定" % (type(e).__name__, e))
|
||
return 2
|
||
|
||
n_ok = n_warn = n_fail = n_skip = 0
|
||
fails = []
|
||
if args.selftest:
|
||
print("\n--- 自证有牙(临时副本里改模板,不碰仓库)---")
|
||
if selftest(args):
|
||
print(" [SELFTEST] PASS:改模板必被点名 + 坏解释器给出可诊断错误 → 判定与自检都活着")
|
||
n_ok += 1
|
||
else:
|
||
print(" [SELFTEST] FAIL:两项自测有没过 → 别信这根闸,先修钩子")
|
||
n_fail += 1
|
||
fails.append(("selftest", "模板漂移探针或假解释器诊断有一项没过"))
|
||
|
||
for gen in gens:
|
||
t0 = time.time()
|
||
if gen.get("requires") == "font_chain":
|
||
ok, why = font_env if font_env else env_font_chain()
|
||
if not ok:
|
||
if args.strict_env:
|
||
print(" [FAIL] %-9s 环境缺失:%s" % (gen["id"], why))
|
||
n_fail += 1
|
||
fails.append((gen["id"], "缺 " + why))
|
||
else:
|
||
print(" [SKIP] %-9s 环境缺失:%s → 本台未验(--strict-env 可升级为 FAIL)"
|
||
% (gen["id"], why))
|
||
n_skip += 1
|
||
continue
|
||
base_map = {}
|
||
if gen.get("has_baseline") and (not want or gen["id"] in want):
|
||
base_map = dict((rel, (script, why)) for rel, script, why in BASELINE)
|
||
if gen.get("per_script"):
|
||
state, msg, _n = run_family(gen, base_map, args)
|
||
else:
|
||
state, msg = run_one_gen(gen, base_map, args, keep_extra)
|
||
print(" [%s] %-9s %5.1fs %s" % (
|
||
{"OK": "OK ", "WARN": "WARN", "FAIL": "FAIL", "ERR": "FAIL"}[state],
|
||
gen["id"], time.time() - t0, msg))
|
||
if state == "OK":
|
||
n_ok += 1
|
||
elif state == "WARN":
|
||
n_warn += 1
|
||
else:
|
||
n_fail += 1
|
||
fails.append((gen["id"], msg))
|
||
|
||
cleared = [rel for rel, _s, _w in BASELINE if rel not in drifting_now]
|
||
if cleared and not want:
|
||
print(" [TIGHTEN] 基线 %d/%d 条本轮实测已还清,可下架(--prune 打出台账): %s"
|
||
% (len(cleared), len(BASELINE), ", ".join(os.path.basename(x) for x in cleared)))
|
||
if args.prune:
|
||
print("\n--- 存量基线台账(只减不增)---")
|
||
for rel, script, why in BASELINE:
|
||
print(" %-34s <- %-40s %s%s" % (rel, script, why,
|
||
"" if rel in drifting_now else " 【已还清,可删这一行】"))
|
||
|
||
print("汇总:%d 项 / 一致 %d / 新漂移或报错 %d / SKIP %d / 存量基线在册 %d 处(走 WARN 不拦提交)"
|
||
% (len(gens) + (1 if args.selftest else 0), n_ok, n_fail, n_skip, len(drifting_now)))
|
||
if n_fail:
|
||
print("结论: FAIL —— 生成物与它的生成器不同码移动。修法:重跑对应生成器,"
|
||
"产物与生成器**同一笔**提交(docs/文档规范.md §4.4)")
|
||
for gid, msg in fails:
|
||
print(" - %s: %s" % (gid, msg[:160]))
|
||
return 1
|
||
print("结论: PASS —— 在册生成器重跑后与工作区逐字节一致;存量基线在册 %d 处走 WARN 不拦提交%s"
|
||
% (len(drifting_now),
|
||
";SKIP %d 项 = 这台机器**没验过**那部分(不是验过了)" % n_skip if n_skip else ""))
|
||
return 0
|
||
|
||
|
||
if __name__ == "__main__":
|
||
sys.exit(main())
|