f5d524fdde
task-33 诊断结论的落地(道G 执行,环境**一个字节没动**:不装包、不重建、不改 requirements 约束)。
**要防的是什么**:`.git/hooks/pre-commit` 把 `INSTALL_PYTHON` 钉在 `.venv`,而 43 条
`language: system` 用
裸 `python` ⇒ pre-commit 本体走 `.venv`(3.12.4)、钩子 entry 走 PATH 首位(3.14.7,零第
三方包)。今天
不炸是**巧合**(那 43 条模块级 import 全是标准库,唯一要 fontTools 的那条自己解析解释器)。真正的危
险形态是:某条钩子在**跑到它那一刻**才报 `No module named ...`——一次提交里前 14 条已经改写完工作区、
第 15 条才崩,现场最难看,也最容易被误当成代码问题。L0a 把这件事提前到门禁第一段。
**FAIL 判据刻意收窄成三事齐备**:模块级第三方 import + 该脚本不自解析解释器 + PATH 首位那个解释器
真缺包。惰性/条件导入(`make_font` 的 fontTools 在函数体内)只列为**候选**、不判决——否则就是假阳性
(见下)。退出码 2 = 环境错误,已写进文件头退出码段与用法。
**道G 自己推翻过自己两处,都留在文档里没删**:
① 上一版诊断说「42 条里唯一需要第三方包的那条自带解析」是**宽松正则**审出来的(把注释与字符串里的包
名也算命中),机械口径下按模块级 import 判是 **0 条**,条目数也因 nested-comment 落地从 42 变 43;
② 第一版 L0a 把 font-charset 判成 FAIL,他拿真解释器实跑反证(`C:\Python314\python.exe
scripts/check_font_charset.py` = exit 0,因为 fontTools 的导入在函数体内、加载期不执行)
⇒ 判据改窄。
**红-绿自证**:临时假仓库三条钩子(要 fontTools 且不自解析 / 同样要但自解析 / 纯标准库)→ 坏的那条判
FAIL、好的判 OK、整体 rc=2;真实仓 → 43 条、forced 缺包 0、候选 fontTools、exit 0;同一预检在 .venv 与
PATH 首位两个解释器下都 exit 0。
文档 §5 重排成**依赖表(§5.2) → 动之前死顺序(§5.3) → 遗留账(§5.4)**(原文按追加顺序把失效顺序插成了
§5.4,读起来先结论后依据;重排时同步修掉块内那条会指错的「见下面 §5.3 第三条」,以及 ci_local 里指向
死顺序却写着 §5.2 的指针——这两处 doc-links 都**静默放过**,因为本文恰好有同号小节,见 §3.5 盲区)。
四种现场(删/换 `.venv`、只换包、换 PATH 首位、改 requirements 约束)逐行钉死「第一现场 / 症状 / 还能
动什么」,回滚按现场分条不混用:freeze 备份放仓外、只 `--force-reinstall --no-deps` 单包不 `-U`、
`pre-commit clean` 会重建别人的钩子环境所以必须先要窗口、最后手段才删 `.venv`(删前记住基底是 Anaconda,
换机复制不出来)。
⚠ 一条与本笔无关的既有红灯(道G 报出,我核过成因):`check_size_gate.py --local` 现在 exit 1 —— 它按
**git 索引 blob** 生成代码指纹,而工作区正有 comm 道在飞的 `App/protocol/iec104/iec104_comm
.{c,h}` 未入
索引,「同码」必然不成立;**该脚本不是 pre-commit 钩子**(只挂在 ci_local),所以不拦提交,但别人跑
ci_local 会看到它,别误判成 L0a 新加的东西。台账回填等这批在飞代码提交后统一做一次。
自查:py_compile 0、`--preflight-only` exit 0(两个解释器下都是)、`check_doc_links --a
ll` exit 0(198 篇)、
本篇 `--sections-strict` 通过、`id-check --strict` exit 0 且不可判定绝对数 **978 未变**
(写文档没吃余量)、
mojibake / bom(81) / nested-comment(409) 全 0、尾随空白 0。
803 lines
38 KiB
Python
803 lines
38 KiB
Python
#!/usr/bin/env python3
|
||
# -*- coding: utf-8 -*-
|
||
"""ci_local.py - 本地一键跑"与 CI 等价"的门禁(纯标准库)。
|
||
|
||
为什么需要它(2026-10-01):
|
||
CI(`.gitea/workflows/ci.yml`)只在 push / PR 时跑,且需要 Gitea Actions +
|
||
Windows runner 就绪。**提交前想自查、或 runner 还没配好时**,用本脚本在本地
|
||
跑同一套判定,命令与退出码与 CI 一一对应。
|
||
|
||
覆盖范围(与 CI 完全一致):
|
||
L0 静态门禁 = `pre-commit run --all-files` + 提交信息门禁(gitlint)
|
||
L0c 体积门禁(强制路径,2026-10-25)= `scripts/check_size_gate.py --local`
|
||
**fail-closed**:没有体积同码凭证、或代码与测量不同码即红——不再"没跑=通过"。
|
||
CI 侧的「体积门禁」步跑的是同一条实现(那条无需 Keil,见 .gitea/workflows/ci.yml)。
|
||
L0.5 主机侧纯逻辑单测 = `test/host/**/run_unit_test.py`
|
||
L0.5c 覆盖率(**opt-in:`--coverage`**,2026-10-23)= `scripts/check_test_coverage.py`
|
||
默认**不跑**、CI 也**不跑**:它依赖 VS 可选的「代码覆盖率工具」组件,且与 L0.5 那 16 个
|
||
钩子跑的是同一批测试(覆盖率只是换一种看法)。口径与盲区 → docs/测试说明/主机单测覆盖率口径.md
|
||
|
||
⚠️ gitlint 必须**单独跑**(2026-10-11 审计):`stages: [commit-msg]` 写在 gitlint **上游**
|
||
的 `.pre-commit-hooks.yaml`(项目 `.pre-commit-config.yaml` 里查不到这一行),而
|
||
`pre-commit run --all-files` 只跑 pre-commit stage —— 它既不在 --all-files 里,
|
||
CI 也一度漏掉(18 个钩子只报 17 个,且表现为"全绿")。本脚本与 CI 的
|
||
`scripts/ci_gitlint_range.ps1` 同语义:逐条把最近 N 条非 merge 提交的
|
||
message 喂给钩子。
|
||
|
||
刻意**不含**(与 CI 同理由,见 workflow 文件头):
|
||
- `@build`(需 Keil 授权;CI 也不跑,真编译是开发者本地红线)
|
||
- `mem-numbers` / `mem-budget` 钩子(事实源 `.map` 是 Keil 产物——**本地若存在 .map
|
||
就照常跑**,与 CI 的自适应逻辑一致)。⚠ 这两根闸在"无 .map"时是 **SKIP 而不是
|
||
PASS**(等于这一趟什么都没查);那个缺口由上面的 L0c 体积门禁补住:同码凭证
|
||
+ 92% 红线复核,无需 Keil 也能判,且 fail-closed。
|
||
|
||
退出码:
|
||
0 = 全部通过(或不适用项已按规则跳过)
|
||
1 = 有门禁/单测失败
|
||
2 = 环境错误(缺 pre-commit / python 模块等;L0a 钩子解释器预检未过也是 2)
|
||
|
||
汇总行(2026-10-02 起,对齐 CI 的 ::notice::):
|
||
[SUMMARY] L0: pre-commit hooks -- passed=N failed=M skipped=K
|
||
[SUMMARY] gitlint: N commit message(s) checked, M failed
|
||
[SUMMARY] L0.5: K runner(s), A/B assertions passed, fail=F
|
||
|
||
用法(仓库根目录):
|
||
& '.venv\\Scripts\\python.exe' scripts/ci_local.py
|
||
& '.venv\\Scripts\\python.exe' scripts/ci_local.py --l0-only
|
||
& '.venv\\Scripts\\python.exe' scripts/ci_local.py --size-only
|
||
& '.venv\\Scripts\\python.exe' scripts/ci_local.py --l05-only
|
||
& '.venv\\Scripts\\python.exe' scripts/ci_local.py --preflight-only # 只跑 L0a 钩子解释器预检
|
||
& '.venv\\Scripts\\python.exe' scripts/ci_local.py --keep-going # 失败也继续跑完
|
||
& '.venv\\Scripts\\python.exe' scripts/ci_local.py --commits 20 # 提交信息校验条数
|
||
|
||
设计依据:`.gitea/workflows/ci.yml`;docs/测试说明/测试开发标准流程.md §9
|
||
创建日期:2026-10-01
|
||
"""
|
||
|
||
import argparse
|
||
import os
|
||
import re
|
||
import shutil
|
||
import subprocess
|
||
import sys
|
||
from pathlib import Path
|
||
|
||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||
|
||
|
||
def _global_python_candidates():
|
||
"""常见**机器级** Python 安装位置(必须与 scripts/ci_find_python.ps1 同序)。
|
||
|
||
两处探测一旦不同步,就会出现"本地能跑、CI 挂"的老毛病:2026-10-11 的
|
||
L0 全灭就是 workflow 把候选收窄成 PATH python + 仓库 .venv(runner 上两者
|
||
都是死路),而真正装了 pre-commit 的机器级解释器(如
|
||
C:\\ProgramData\\anaconda3\\python.exe)不再被探测。
|
||
|
||
本机实测矩阵(哪个解释器装了哪些包、钩子实际跑在谁身上)见
|
||
docs/环境与部署/本地工具路径环境变量方案.md §5:提交期钩子由 .git/hooks/pre-commit
|
||
里的 INSTALL_PYTHON 钉死在仓库 .venv,而 language: system 的 entry 用裸 python 时
|
||
**不继承**它——所以依赖第三方包的脚本必须像本函数这样自己解析解释器。
|
||
"""
|
||
out = []
|
||
pf = os.environ.get("ProgramFiles", r"C:\Program Files")
|
||
pf86 = os.environ.get("ProgramFiles(x86)", r"C:\Program Files (x86)")
|
||
local = os.environ.get("LOCALAPPDATA")
|
||
pd = os.environ.get("ProgramData", r"C:\ProgramData")
|
||
# 新版本优先:新装的解释器通常才是想要的。
|
||
for ver in ("Python313", "Python312", "Python311", "Python310"):
|
||
out.append(os.path.join(pf, ver, "python.exe"))
|
||
out.append(os.path.join(pf86, ver, "python.exe"))
|
||
if local:
|
||
out.append(os.path.join(local, "Programs", "Python", ver, "python.exe"))
|
||
for dist in ("anaconda3", "miniconda3"):
|
||
out.append(os.path.join(pd, dist, "python.exe"))
|
||
return out
|
||
|
||
|
||
def _resolve_python():
|
||
r"""挑一个真正装了 pre-commit 的解释器。
|
||
|
||
本机可能有多个 python(.venv / 发行版 / 系统),而 **pre-commit 只装在其中一个**。
|
||
按优先级探测(见 docs/环境与部署/本地工具路径环境变量方案.md §1.1):
|
||
1. DTU_PRECOMMIT_PYTHON(显式配置:文件缺失或无 pre-commit 模块 → 立即报
|
||
配置错误,**不静默回退**到别的解释器)
|
||
2. 当前解释器(sys.executable)
|
||
3. 仓库 `.venv\Scripts\python.exe`
|
||
4. PATH 上的 `python`
|
||
5. 常见机器级安装位置(见 _global_python_candidates)
|
||
找到即返回 (python_path, note);都找不到则返回 (当前解释器, None) 交由调用方报错。
|
||
"""
|
||
explicit = os.environ.get("DTU_PRECOMMIT_PYTHON", "").strip()
|
||
if explicit:
|
||
if not os.path.isfile(explicit):
|
||
print("[ERROR] DTU_PRECOMMIT_PYTHON 已设置但文件不存在: {}".format(explicit))
|
||
print(" 请修正该变量(python.exe 完整路径),或清除该变量改用自动探测。")
|
||
sys.exit(2)
|
||
try:
|
||
cp = subprocess.run([explicit, "-m", "pre_commit", "--version"],
|
||
capture_output=True, timeout=60)
|
||
except (OSError, subprocess.TimeoutExpired) as e:
|
||
print("[ERROR] DTU_PRECOMMIT_PYTHON 指定的解释器无法执行: {}({})".format(explicit, e))
|
||
sys.exit(2)
|
||
if cp.returncode != 0:
|
||
print("[ERROR] DTU_PRECOMMIT_PYTHON 指向的解释器没有 pre-commit 模块: {}".format(explicit))
|
||
print(" 请修正该变量(指向装有 pre-commit 的 python),或清除该变量改用自动探测。")
|
||
sys.exit(2)
|
||
return explicit, "(显式配置 DTU_PRECOMMIT_PYTHON)"
|
||
|
||
candidates = [
|
||
sys.executable,
|
||
str(REPO_ROOT / ".venv" / "Scripts" / "python.exe"),
|
||
shutil.which("python"),
|
||
] + _global_python_candidates()
|
||
seen = set()
|
||
tried = []
|
||
for py in candidates:
|
||
if not py or py in seen or not os.path.isfile(py):
|
||
continue
|
||
seen.add(py)
|
||
tried.append(py)
|
||
try:
|
||
cp = subprocess.run([py, "-m", "pre_commit", "--version"],
|
||
capture_output=True, timeout=60)
|
||
except (OSError, subprocess.TimeoutExpired):
|
||
continue
|
||
if cp.returncode == 0:
|
||
note = None if py == sys.executable else "(自动选用,本解释器无 pre-commit)"
|
||
return py, note
|
||
print("[WARN] 未找到装有 pre-commit 的解释器,已尝试:")
|
||
for py in tried:
|
||
print(" - {}".format(py))
|
||
return sys.executable, None
|
||
|
||
|
||
PY, PY_NOTE = _resolve_python()
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 钩子解释器预检(道G task-33,2026-10-27)
|
||
#
|
||
# 为什么要这一道:提交期"钩子跑在谁身上"有两个互不相干的来源——
|
||
# · pre-commit 本体跑在 .git/hooks/pre-commit 里写死的 INSTALL_PYTHON 上(本机 = 仓库 .venv);
|
||
# · 而 language: system 的 entry 只要写裸 python,pre-commit **不改 PATH**,那条脚本就
|
||
# 落到 PATH 首位的解释器(本机实测 3.14.7,一个第三方包都没装)。
|
||
# 今天没炸是两件事同时成立的结果:那批裸 python 条目全只用标准库,且唯一需要第三方包的那条
|
||
# 脚本自己解析解释器。矩阵、成因与新增 entry 的硬规则见
|
||
# docs/环境与部署/本地工具路径环境变量方案.md §5(含 §5.1 硬规则、§5.2 修复死顺序与回滚)。
|
||
# 本预检把那条隐性依赖变成显式闸:一旦凑齐「entry 用裸 python + 脚本需要第三方包 + 脚本不
|
||
# 自带解释器解析 + PATH 首位没装那个包」四件事,就当场 exit 2 指名是哪根钩子缺什么,
|
||
# 而不是等跑到那根钩子才崩——半路崩会被误读成"代码有问题",白查一小时。
|
||
#
|
||
# 只读:不改环境、不装包、不写仓库。
|
||
# ---------------------------------------------------------------------------
|
||
|
||
#: 常见第三方包名 -> 导入名(不在表里的按 sys.stdlib_module_names 反判)
|
||
_THIRD_PARTY_HINTS = ("fontTools", "PIL", "serial", "yaml", "numpy", "pandas", "requests")
|
||
|
||
|
||
def _hook_bare_python_entries():
|
||
"""扫 .pre-commit-config.yaml,返回 [(hook_id, script_path)]。
|
||
|
||
只收 language: system 且 entry 以裸 python / python3 开头的条目——这类条目的解释器
|
||
由 PATH 决定,不受 pre-commit 本体那个 INSTALL_PYTHON 影响。配置文件是手写的规整
|
||
格式,这里按行扫即可(不依赖 PyYAML:那正是本预检要防的"依赖解释器"问题)。
|
||
"""
|
||
cfg = REPO_ROOT / ".pre-commit-config.yaml"
|
||
if not cfg.is_file():
|
||
return []
|
||
out = []
|
||
cur_id = None
|
||
cur_entry = None
|
||
cur_lang = None
|
||
for raw in cfg.read_text(encoding="utf-8", errors="replace").splitlines():
|
||
s = raw.strip()
|
||
if s.startswith("#"):
|
||
continue
|
||
m = re.match(r"^-\s+id:\s*(\S+)", s)
|
||
if m:
|
||
cur_id, cur_entry, cur_lang = m.group(1), None, None
|
||
continue
|
||
m = re.match(r"^entry:\s*(.+)$", s)
|
||
if m:
|
||
cur_entry = m.group(1).strip()
|
||
continue
|
||
m = re.match(r"^language:\s*(\S+)", s)
|
||
if m:
|
||
cur_lang = m.group(1)
|
||
if cur_lang == "system" and cur_entry and cur_id:
|
||
toks = cur_entry.split()
|
||
if toks and re.match(r"^python(3)?$|^py$", toks[0]) and len(toks) > 1:
|
||
out.append((cur_id, toks[1].strip("'\"")))
|
||
continue
|
||
return out
|
||
|
||
|
||
def _script_third_party(script_rel):
|
||
"""返回 {第三方包名: 由哪个脚本引入};纯标准库且下游也不需要的脚本返回空字典。
|
||
|
||
判据:顶层导入名既不在 sys.stdlib_module_names 里、也不在仓库内解析得到(同目录/
|
||
scripts/tools 下的兄弟模块算仓库内),就算第三方。老解释器没有
|
||
stdlib_module_names 时退到 _THIRD_PARTY_HINTS 白名单,宁可少报不误报。
|
||
顺带沿"脚本里以字符串出现的仓库内 .py"下探一层(见 _referenced_scripts),
|
||
否则漏掉 2026-10-26 那类"钩子干净、被它拉起的生成器缺包"的真事故。
|
||
"""
|
||
forced = _direct_third_party(script_rel)
|
||
hinted = {}
|
||
for name, src in _collect_third_party(script_rel, 2, set()).items():
|
||
if name not in forced:
|
||
hinted[name] = src
|
||
lazy = {}
|
||
for name, src in _collect_lazy_third_party(script_rel, 2, set()).items():
|
||
if name not in forced and name not in hinted:
|
||
lazy[name] = src
|
||
return forced, hinted, lazy
|
||
|
||
|
||
def _is_repo_internal(script_dir, name):
|
||
"""该导入名能否在仓库内解析到(兄弟模块 / scripts / tools / 主机单测公共件)。"""
|
||
search_dirs = [script_dir, REPO_ROOT / "scripts", REPO_ROOT / "tools",
|
||
REPO_ROOT / "test" / "host" / "common", REPO_ROOT]
|
||
return any((d / (name + ".py")).is_file() or (d / name).is_dir() for d in search_dirs)
|
||
|
||
|
||
def _direct_third_party(script_rel):
|
||
"""本脚本直接 import 的第三方顶层模块 -> 该脚本路径(判据见 _script_third_party 的头注释)。"""
|
||
path = REPO_ROOT / script_rel
|
||
if not path.is_file():
|
||
return {}
|
||
text = path.read_text(encoding="utf-8", errors="replace")
|
||
mods = set()
|
||
# 只认**模块级**(第 0 列)的 import:函数体里的 try/惰性导入不是"加载必崩",
|
||
# 按它报会把好钩子判成坏钩子(实测 check_font_charset 就在 3.14 上跑得过,
|
||
# 因为 make_font 的 fontTools 是函数内惰性导入)。
|
||
for m in re.finditer(r"^(?:import|from)[ \t]+([A-Za-z_][\w.]*)", text, re.M):
|
||
mods.add(m.group(1).split(".")[0])
|
||
std = getattr(sys, "stdlib_module_names", None)
|
||
out = {}
|
||
for name in sorted(mods):
|
||
if std is not None:
|
||
if name in std:
|
||
continue
|
||
elif name not in _THIRD_PARTY_HINTS:
|
||
continue
|
||
if _is_repo_internal(path.parent, name):
|
||
continue
|
||
out[name] = script_rel
|
||
return out
|
||
|
||
|
||
def _collect_lazy_third_party(script_rel, depth, seen):
|
||
"""惰性 / 条件导入的第三方包(函数体内的 import)——**只列候选,不作判决**。
|
||
|
||
静态判不了它到底会不会被走到,所以不当失败依据;但值得摊开看:2026-10-26 那次
|
||
"No module named fontTools" 正是这种形态(钩子脚本自己只 import 标准库,被它拉起的
|
||
生成器在函数里导 fontTools)。判 FAIL 用 _direct_third_party,本函数只负责"看得见"。
|
||
"""
|
||
if script_rel in seen or not (REPO_ROOT / script_rel).is_file():
|
||
return {}
|
||
seen.add(script_rel)
|
||
text = (REPO_ROOT / script_rel).read_text(encoding="utf-8", errors="replace")
|
||
mods = set()
|
||
for m in re.finditer(r"^[ \t]+(?:import|from)[ \t]+([A-Za-z_][\w.]*)", text, re.M):
|
||
mods.add(m.group(1).split(".")[0])
|
||
std = getattr(sys, "stdlib_module_names", None)
|
||
path = REPO_ROOT / script_rel
|
||
found = {}
|
||
for name in sorted(mods):
|
||
if (name in std) if std is not None else (name not in _THIRD_PARTY_HINTS):
|
||
continue
|
||
if _is_repo_internal(path.parent, name):
|
||
continue
|
||
found[name] = script_rel
|
||
if depth > 0:
|
||
for ref in _referenced_scripts(script_rel, text):
|
||
for name, src in _collect_lazy_third_party(ref, depth - 1, seen).items():
|
||
found.setdefault(name, src)
|
||
return found
|
||
|
||
|
||
def _referenced_scripts(script_rel, text):
|
||
"""脚本里以字符串出现的仓库内 .py —— subprocess / importlib 的下游目标。
|
||
|
||
为什么要顺着找一层:2026-10-26 那次真事故就是"钩子本身只 import 标准库、被它拉起的
|
||
字体链脚本要 fontTools",只看直接 import 会漏判(详见 docs/环境与部署/
|
||
本地工具路径环境变量方案.md §5.1)。
|
||
"""
|
||
out = []
|
||
for m in re.finditer(r"([A-Za-z0-9_./\\-]+\.py)", text):
|
||
rel = m.group(1).lstrip("./\\").replace("\\", "/")
|
||
if (REPO_ROOT / rel).is_file() and rel != script_rel:
|
||
out.append(rel)
|
||
return sorted(set(out))
|
||
|
||
|
||
def _collect_third_party(script_rel, depth, seen):
|
||
"""顺着下游目标合并第三方包,返回 {包名: 由哪个脚本引入}(深度上限 2,防环)。"""
|
||
if script_rel in seen or not (REPO_ROOT / script_rel).is_file():
|
||
return {}
|
||
seen.add(script_rel)
|
||
text = (REPO_ROOT / script_rel).read_text(encoding="utf-8", errors="replace")
|
||
found = dict(_direct_third_party(script_rel))
|
||
if depth <= 0:
|
||
return found
|
||
for ref in _referenced_scripts(script_rel, text):
|
||
for name, src in _collect_third_party(ref, depth - 1, seen).items():
|
||
found.setdefault(name, src)
|
||
return found
|
||
|
||
|
||
def _script_self_resolves(script_rel):
|
||
"""脚本是否自带解释器解析(先例:scripts/check_generated_fresh.py)。"""
|
||
path = REPO_ROOT / script_rel
|
||
if not path.is_file():
|
||
return False
|
||
text = path.read_text(encoding="utf-8", errors="replace")
|
||
return bool(re.search(r"DTU_[A-Z0-9_]*PYTHON|_resolve_python|_pick_python|_find_python|"
|
||
r"python_discovery|_global_python_candidates|Scripts[\/][^\n]{0,16}python\.exe|"
|
||
r"re-?exec|ensure_python", text))
|
||
|
||
|
||
def _missing_modules(python_path, modules):
|
||
"""用指定解释器一次性试导入这批模块,返回它缺的那些(解释器本身跑不起来返回 None)。"""
|
||
if not modules:
|
||
return []
|
||
code = "import importlib\nmiss = []\nfor m in %r:\n try:\n importlib.import_module(m)\n except Exception:\n miss.append(m)\nprint(','.join(miss))\n" % (sorted(modules),)
|
||
try:
|
||
cp = subprocess.run([python_path, "-c", code], capture_output=True, timeout=90)
|
||
except (OSError, subprocess.TimeoutExpired):
|
||
return None
|
||
if cp.returncode != 0:
|
||
return None
|
||
tail = cp.stdout.decode("utf-8", "replace").strip().splitlines()
|
||
if not tail:
|
||
return []
|
||
return [x for x in tail[-1].split(",") if x]
|
||
|
||
|
||
def run_hook_preflight():
|
||
"""钩子解释器预检:报清"钩子跑在谁身上、裸 python 落到谁、缺哪些包"。
|
||
|
||
返回值:0 = 健康(含"确实缺包,但那些脚本自带解释器解析"这种合规形态);
|
||
2 = 环境/配置错误——已指名到钩子与包名,修完再跑门禁。
|
||
"""
|
||
print("")
|
||
print("=" * 64)
|
||
print(" L0a 钩子解释器预检(只读,不改环境)")
|
||
print("=" * 64)
|
||
|
||
hook = REPO_ROOT / ".git" / "hooks" / "pre-commit"
|
||
venv = str(REPO_ROOT / ".venv" / "Scripts" / "python.exe").lower()
|
||
if hook.is_file():
|
||
m = re.search(r"INSTALL_PYTHON='([^']+)'", hook.read_text(encoding="utf-8", errors="replace"))
|
||
if not m:
|
||
print("[WARN] .git/hooks/pre-commit 里读不到 INSTALL_PYTHON(pre-commit 版本变更?)")
|
||
elif m.group(1).lower() != venv:
|
||
print("[WARN] 钩子的 INSTALL_PYTHON = {}".format(m.group(1)))
|
||
print(" 而仓库 .venv = {} —— 两者不一致:提交期钩子跑的**不是**这个 .venv".format(venv))
|
||
print(" 先确认哪个才是你要的环境(详见 docs/环境与部署/本地工具路径环境变量方案.md §5)")
|
||
else:
|
||
print("[INFO] 提交期钩子跑在:{}(与仓库 .venv 一致)".format(m.group(1)))
|
||
else:
|
||
print("[INFO] 本工作树没有 .git/hooks/pre-commit(worktree 或未安装钩子),跳过一致性核对")
|
||
|
||
path_first = shutil.which("python")
|
||
if not path_first:
|
||
print("[WARN] PATH 上没有 python —— language: system 的裸 python 条目会直接找不到解释器")
|
||
return 2
|
||
try:
|
||
ver = subprocess.run([path_first, "-c", "import sys;print(sys.version.split()[0])"],
|
||
capture_output=True, timeout=60).stdout.decode("utf-8", "replace").strip()
|
||
except (OSError, subprocess.TimeoutExpired):
|
||
ver = "?"
|
||
entries = _hook_bare_python_entries()
|
||
need_all = set()
|
||
rows = []
|
||
lazy_all = {}
|
||
for hook_id, script in entries:
|
||
forced, hinted, lazy = _script_third_party(script)
|
||
if forced or hinted:
|
||
need_all.update(forced.keys())
|
||
for name, src in lazy.items():
|
||
lazy_all.setdefault(name, []).append(hook_id)
|
||
rows.append((hook_id, script, forced, hinted,
|
||
_script_self_resolves(script) if (forced or hinted) else True))
|
||
missing = _missing_modules(path_first, need_all)
|
||
if missing is None:
|
||
print("[ERROR] 探测解释器失败:{}(跑不起来,预检无法完成)".format(path_first))
|
||
return 2
|
||
print("[INFO] language: system 用裸 python 的条目:{} 条;其 entry 解释器 = {} {}(PATH 首位)".format(
|
||
len(entries), path_first, ver))
|
||
print("[INFO] 这些条目累计需要的第三方包:{};该解释器缺:{}".format(
|
||
", ".join(sorted(need_all)) or "(无,全是纯标准库)", ", ".join(sorted(missing)) or "(无)"))
|
||
bad = [(h, s, f) for (h, s, f, _h2, resolved) in rows
|
||
if f and not resolved and any(x in missing for x in f)]
|
||
for hook_id, script, forced, hinted, resolved in rows:
|
||
if not forced and not hinted:
|
||
continue
|
||
names = sorted(set(list(forced) + list(hinted)))
|
||
lack = [x for x in names if x in missing]
|
||
tag = "OK " if (not lack or resolved or not forced) else "FAIL"
|
||
if lack and hinted and not forced:
|
||
tag = "WARN"
|
||
via = sorted({src for name, src in hinted.items() if src and src != script})
|
||
print(" [{}] {} <- {} 需 {}{};脚本自带解释器解析:{}{}".format(
|
||
tag, hook_id, script, ", ".join(names),
|
||
"(该解释器缺 " + ", ".join(lack) + ")" if lack else "", "是" if resolved else "否",
|
||
";其中下游引入:" + ", ".join(via) if via else ""))
|
||
if bad:
|
||
print("")
|
||
print("[ERROR] 以下钩子会**在跑到它时**才崩(No module named ...),现在提前拦下:")
|
||
for hook_id, script, mods in bad:
|
||
print(" - {}:{} 需要 {},而裸 python({})没装".format(hook_id, script, ", ".join(mods), path_first))
|
||
print(" 三选一(别改 requirements.txt 的版本约束,那是别人环境的输入):")
|
||
print(" 1) 把 entry 改成能自解析解释器的写法,或在脚本内按「"
|
||
"显式 var > 当前解释器 > 仓库 .venv > PATH」解析 + 探针校验(先例 scripts/check_generated_fresh.py);")
|
||
print(" 2) 把 DTU_PRECOMMIT_PYTHON 显式指向装有这些包的解释器(不会改变裸 python 条目的落点,"
|
||
"但能让 pre-commit 本体确定);")
|
||
print(" 3) 在该解释器里装缺的包——动环境前必须拿到静默窗口,"
|
||
"顺序与回滚见 docs/环境与部署/本地工具路径环境变量方案.md §5.3。")
|
||
return 2
|
||
if lazy_all:
|
||
print("[INFO] 候选(下游惰性 / 条件导入的第三方包,静态判不了会不会被走到,故不作判决):")
|
||
for name in sorted(lazy_all):
|
||
probe = "" if name not in missing else "(裸 python 那个解释器**确实没装**)"
|
||
print(" - {}:{}{}".format(name, ", ".join(sorted(set(lazy_all[name]))[:6]), probe))
|
||
print("[OK] 预检通过:没有「裸 python + 缺包 + 不自解析」三事齐备的钩子")
|
||
return 0
|
||
|
||
|
||
def _find_map():
|
||
"""返回 Keil 构建产物 .map 的路径(不存在则 None)。"""
|
||
hits = sorted((REPO_ROOT / "MDK-ARM").rglob("*.map"))
|
||
return hits[0] if hits else None
|
||
|
||
|
||
def _find_unit_runners():
|
||
"""test/host 下所有模块运行器,按路径排序。"""
|
||
return sorted((REPO_ROOT / "test" / "host").rglob("run_unit_test.py"))
|
||
|
||
|
||
def _run(cmd, env=None):
|
||
"""跑子进程,输出直通终端。返回退出码。"""
|
||
print("$ " + " ".join(str(c) for c in cmd))
|
||
cp = subprocess.run(cmd, cwd=str(REPO_ROOT), env=env)
|
||
return cp.returncode
|
||
|
||
|
||
def _run_capture(cmd, env=None):
|
||
"""跑子进程,输出**既直通终端又捕获**(对齐 CI 的 Tee-Object)。
|
||
|
||
返回 (退出码, 全部输出文本)。用于 L0.5 解析各模块汇总行。
|
||
"""
|
||
print("$ " + " ".join(str(c) for c in cmd))
|
||
cp = subprocess.run(cmd, cwd=str(REPO_ROOT), env=env,
|
||
capture_output=True)
|
||
try:
|
||
text = cp.stdout.decode("utf-8")
|
||
except UnicodeDecodeError:
|
||
text = cp.stdout.decode("utf-8", errors="replace")
|
||
sys.stdout.write(text)
|
||
if cp.stderr:
|
||
try:
|
||
sys.stderr.write(cp.stderr.decode("utf-8"))
|
||
except UnicodeDecodeError:
|
||
sys.stderr.write(cp.stderr.decode("utf-8", errors="replace"))
|
||
return cp.returncode, text
|
||
|
||
|
||
def run_l0():
|
||
"""L0:pre-commit 全部钩子。返回退出码(0/1/2)。"""
|
||
print("")
|
||
print("=" * 64)
|
||
print(" L0 静态门禁(pre-commit --all-files)")
|
||
print("=" * 64)
|
||
|
||
env = dict(os.environ)
|
||
# 与 CI 相同的自适应:无 .map(未做 Keil 构建)=> mem-numbers / mem-budget
|
||
# 不适用 => 跳过(两者事实源都是 Keil 产物 .map)
|
||
map_path = _find_map()
|
||
if map_path:
|
||
print("[INFO] 找到 .map:{}".format(map_path.relative_to(REPO_ROOT)))
|
||
print("[INFO] mem-numbers / mem-budget 照常运行")
|
||
else:
|
||
print("[INFO] 未找到 .map(未做 Keil 构建)-> 跳过 mem-numbers / "
|
||
"mem-budget(与 CI 一致)")
|
||
print("[INFO] :warning: 体积门禁未执行——改代码后请本地 @build + "
|
||
"check_mem_numbers.py --scan + ram_baseline.py --gate 兜底")
|
||
skip = env.get("SKIP", "")
|
||
add = "mem-numbers,mem-budget"
|
||
env["SKIP"] = (skip + "," if skip else "") + add
|
||
|
||
rc, text = _run_capture([PY, "-m", "pre_commit", "run", "--all-files"], env=env)
|
||
# 汇总行(对齐 CI 的 ::notice::):统计钩子 Passed/Failed/Skipped
|
||
lines = text.splitlines()
|
||
passed = sum(1 for l in lines if l.rstrip().endswith("Passed"))
|
||
failed = sum(1 for l in lines if l.rstrip().endswith("Failed"))
|
||
skipd = sum(1 for l in lines if l.rstrip().endswith("Skipped"))
|
||
print("[SUMMARY] L0: pre-commit hooks -- passed={} failed={} skipped={}".format(
|
||
passed, failed, skipd))
|
||
return rc
|
||
|
||
|
||
def run_gitlint(count):
|
||
"""提交信息门禁:把最近 N 条非 merge 提交的 message 逐条喂给 gitlint 钩子。
|
||
|
||
与 CI 的 scripts/ci_gitlint_range.ps1 同语义(gitlint 声明
|
||
`stages: [commit-msg]`,`pre-commit run --all-files` 永远跑不到它)。
|
||
返回退出码(0/1/2)。
|
||
"""
|
||
print("")
|
||
print("=" * 64)
|
||
print(" L0b 提交信息门禁(gitlint / commit-msg stage,最近 {} 条)".format(count))
|
||
print("=" * 64)
|
||
|
||
def _git(*args):
|
||
return subprocess.run(["git"] + list(args), cwd=str(REPO_ROOT),
|
||
capture_output=True, timeout=60)
|
||
|
||
try:
|
||
out = _git("log", "--no-merges", "--format=%H", "-n", str(count), "HEAD")
|
||
except (OSError, subprocess.TimeoutExpired) as e:
|
||
print("[ERROR] 无法调用 git:{}".format(e))
|
||
return 2
|
||
if out.returncode != 0:
|
||
print("[ERROR] git log 失败:{}".format(
|
||
out.stderr.decode("utf-8", errors="replace").strip()))
|
||
return 2
|
||
|
||
shas = [s.strip() for s in out.stdout.decode("utf-8", errors="replace").splitlines()
|
||
if s.strip()]
|
||
if not shas:
|
||
print("[INFO] 无可校验提交 —— 跳过")
|
||
print("[SUMMARY] gitlint: 0 commit message(s) checked, 0 failed")
|
||
return 0
|
||
|
||
# 消息文件必须写在【仓库内】且【UTF-8 无 BOM】:
|
||
# - 跨盘符会被 pre-commit 直接拒绝(path is on mount 'C:', start on mount 'D:')
|
||
# - BOM 会占掉标题 1 个字符,扭曲 .gitlint 的 100 字符上限
|
||
msg_dir = REPO_ROOT / "ci-artifacts" / "gitlint-local"
|
||
msg_dir.mkdir(parents=True, exist_ok=True)
|
||
|
||
checked = 0
|
||
failed = 0
|
||
try:
|
||
for sha in shas:
|
||
checked += 1
|
||
body = _git("log", "-1", "--format=%B", sha)
|
||
title = _git("log", "-1", "--format=%s", sha)
|
||
text = body.stdout.decode("utf-8", errors="replace").rstrip("\r\n")
|
||
title_line = title.stdout.decode("utf-8", errors="replace").strip()
|
||
|
||
msg_path = msg_dir / (sha + ".txt")
|
||
msg_path.write_text(text, encoding="utf-8", newline="\n")
|
||
|
||
print("")
|
||
print("--- {} {}".format(sha[:8], title_line))
|
||
rc, _ = _run_capture([PY, "-m", "pre_commit", "run",
|
||
"--hook-stage", "commit-msg",
|
||
"--commit-msg-filename", str(msg_path),
|
||
"gitlint"])
|
||
if rc != 0:
|
||
failed += 1
|
||
print("[FAIL] gitlint 拒绝该提交信息:{}".format(title_line))
|
||
finally:
|
||
shutil.rmtree(str(msg_dir), ignore_errors=True)
|
||
|
||
print("[SUMMARY] gitlint: {} commit message(s) checked, {} failed".format(
|
||
checked, failed))
|
||
return 1 if failed else 0
|
||
|
||
|
||
def run_size_gate():
|
||
"""L0c:体积门禁(强制路径)。返回退出码(0/1/2)。
|
||
|
||
为什么必须单列一步(2026-10-25):`mem-numbers` / `mem-budget` 的事实源是 Keil
|
||
`.map`,无 `.map` 时两者**自适应 SKIP 并返回 0**——"改了代码但根本没编译"
|
||
这条最常见的漏测路径因此在本地也是绿的,CI 更是整根钩子都不跑。它要拦的正是
|
||
这个**"没跑 = 通过"**的盲区。
|
||
|
||
本步跑 `scripts/check_size_gate.py --local`,不需要 Keil:它校验**体积同码凭证**
|
||
(`scripts/size_attestation.json`,只能由"新鲜 .map + 未超 92% 线"的测量写出),
|
||
于是"代码变了但体积从没测过"在这里必然拦红,并给出可照做的修复命令。
|
||
CI 的「体积门禁」步用的是同一条实现,两边不会各说一套口径。
|
||
"""
|
||
print("")
|
||
print("=" * 64)
|
||
print(" L0c 体积门禁(同码凭证 + 92% 红线,无需 Keil)")
|
||
print("=" * 64)
|
||
script = REPO_ROOT / "scripts" / "check_size_gate.py"
|
||
if not script.is_file():
|
||
# 缺脚本 = 这一层没装上,按失败处理(fail-closed,与"没跑=通过"相反)
|
||
print("[ERROR] 缺 scripts/check_size_gate.py —— 体积门禁无法执行,按失败处理")
|
||
print("[SUMMARY] size-gate: rc=2 (gate script missing)")
|
||
return 2
|
||
rc = _run([PY, str(script), "--local"])
|
||
if rc == 1:
|
||
print(" (1 = 没有凭证,或代码与测量不同码:@build 0/0 之后跑 "
|
||
"python scripts/ram_baseline.py --update)")
|
||
print("[SUMMARY] size-gate: rc={}".format(rc))
|
||
return rc
|
||
|
||
|
||
def run_l05(args):
|
||
"""L0.5:主机侧单测。返回退出码(0/1/2)。"""
|
||
print("")
|
||
print("=" * 64)
|
||
print(" L0.5 主机侧纯逻辑单测(MSVC,零硬件)")
|
||
print("=" * 64)
|
||
|
||
runners = _find_unit_runners()
|
||
if not runners:
|
||
print("[WARN] 未找到 test/host/**/run_unit_test.py —— 跳过")
|
||
return 0
|
||
|
||
rc_all = 0
|
||
runner_count = 0
|
||
sum_pass = 0
|
||
sum_total = 0
|
||
for r in runners:
|
||
runner_count += 1
|
||
rel = r.relative_to(REPO_ROOT)
|
||
print("")
|
||
print("--- 运行:{} ---".format(rel))
|
||
rc, text = _run_capture([PY, str(r)])
|
||
# 解析模块自身汇总行,如 "=== System Clock Tests: 100/100 PASSED (failed=0) ==="
|
||
m = None
|
||
for line in text.splitlines():
|
||
mm = re.search(r":\s*(\d+)/(\d+)\s+PASSED\s+\(failed=(\d+)\)", line)
|
||
if mm:
|
||
m = mm
|
||
if m:
|
||
sum_pass += int(m.group(1))
|
||
sum_total += int(m.group(2))
|
||
print("[SUMMARY] {}: {}/{} PASSED (failed={})".format(
|
||
r.parent.name, m.group(1), m.group(2), m.group(3)))
|
||
else:
|
||
print("[WARN] {} 未输出 '=== ... N/M PASSED ===' 汇总行".format(rel))
|
||
if rc != 0:
|
||
print("[FAIL] {}(退出码 {})".format(rel, rc))
|
||
if rc == 2:
|
||
print(" (退出码 2 = 环境错误:多半是本机未装 MSVC,见 "
|
||
"docs/测试说明/测试开发标准流程.md §4.2)")
|
||
rc_all = 1
|
||
if not args.keep_going:
|
||
break
|
||
else:
|
||
print("[OK] {}".format(rel))
|
||
|
||
print("[SUMMARY] L0.5: {} runner(s), {}/{} assertions passed, fail={}".format(
|
||
runner_count, sum_pass, sum_total, rc_all))
|
||
return rc_all
|
||
|
||
|
||
def run_coverage():
|
||
"""覆盖率附加步骤(opt-in,CI 不跑):主机单测覆盖率 + 基线增量门禁。
|
||
|
||
单独一步而不并进 L0.5:L0.5 是**必须绿**的门禁,覆盖率是**趋势视图**,两者失败含义不同
|
||
(前者=测试挂了,后者=覆盖回退了 / 缺工具链)。退出码原样透传,由 main 汇总。
|
||
"""
|
||
print("")
|
||
print("=" * 64)
|
||
print(" L0.5c 主机单测覆盖率(mscoverage 动态插桩 + 基线增量门禁)")
|
||
print("=" * 64)
|
||
script = REPO_ROOT / "scripts" / "check_test_coverage.py"
|
||
if not script.is_file():
|
||
print("[WARN] 缺 scripts/check_test_coverage.py —— 跳过该步")
|
||
return 0
|
||
rc = _run([PY, str(script)])
|
||
if rc == 2:
|
||
print(" (退出码 2 = 环境错误:多半没装 VS 的「代码覆盖率工具」组件,或需设")
|
||
print(" DTU_MS_COVERAGE。口径与排障 → docs/测试说明/主机单测覆盖率口径.md)")
|
||
elif rc == 1:
|
||
print(" (退出码 1 = 有单测失败,或已覆盖行/函数相对基线回退;确认无误后")
|
||
print(" 跑 scripts/check_test_coverage.py --update-baseline 重登基线)")
|
||
return rc
|
||
|
||
|
||
def main(argv=None):
|
||
for stream in (sys.stdout, sys.stderr):
|
||
try:
|
||
stream.reconfigure(encoding="utf-8", errors="replace")
|
||
except Exception:
|
||
pass
|
||
|
||
ap = argparse.ArgumentParser(
|
||
description="本地跑与 CI 等价的门禁(L0 + L0.5,纯标准库)")
|
||
g = ap.add_mutually_exclusive_group()
|
||
g.add_argument("--l0-only", action="store_true", help="只跑 L0 静态门禁")
|
||
g.add_argument("--l05-only", action="store_true", help="只跑 L0.5 主机侧单测")
|
||
g.add_argument("--size-only", action="store_true",
|
||
help="只跑 L0c 体积门禁(强制路径;约 0.3 s,无需 Keil)")
|
||
ap.add_argument("--coverage", action="store_true",
|
||
help="额外跑主机单测覆盖率(约 25 s;需 VS 代码覆盖率组件;默认不跑,CI 也不跑)")
|
||
ap.add_argument("--preflight-only", action="store_true",
|
||
help="只跑 L0a 钩子解释器预检(只读;缺包时当场指名钩子)")
|
||
ap.add_argument("--keep-going", action="store_true",
|
||
help="某步失败也继续跑后续步骤(默认首错即停)")
|
||
ap.add_argument("--commits", type=int, default=10, metavar="N",
|
||
help="提交信息门禁(gitlint)校验最近 N 条非 merge 提交(默认 10)")
|
||
args = ap.parse_args(argv)
|
||
|
||
print("=== 本地 CI(等价 .gitea/workflows/ci.yml)===")
|
||
print("仓库:{}".format(REPO_ROOT))
|
||
print("Python:{}".format(PY))
|
||
if PY_NOTE:
|
||
print(" {}".format(PY_NOTE))
|
||
|
||
# 环境预检:pre-commit 必须可用(CI 里由"环境自检"步骤拦)
|
||
try:
|
||
probe = subprocess.run([PY, "-m", "pre_commit", "--version"],
|
||
capture_output=True, timeout=60)
|
||
except (OSError, subprocess.TimeoutExpired) as e:
|
||
print("[ERROR] 无法调用 pre-commit:{}".format(e))
|
||
return 2
|
||
if probe.returncode != 0:
|
||
print("[ERROR] 该解释器没有 pre-commit 模块。请任选其一:")
|
||
print(" - 用装了 pre-commit 的解释器跑本脚本(如 "
|
||
"& '.venv\\Scripts\\python.exe' scripts/ci_local.py)")
|
||
print(" - 或设置 DTU_PRECOMMIT_PYTHON 指向该解释器")
|
||
print(" (见 docs/环境与部署/本地工具路径环境变量方案.md)")
|
||
print(" - 或在当前环境安装:pip install pre-commit")
|
||
return 2
|
||
|
||
# L0a 钩子解释器预检:把"裸 python 条目 + 缺第三方包 + 脚本不自解析"这条隐性依赖
|
||
# 提到所有钩子之前判掉(道G task-33)。--preflight-only 时它是唯一一段。
|
||
rc_pre = run_hook_preflight()
|
||
if args.preflight_only:
|
||
return rc_pre
|
||
if rc_pre == 2:
|
||
print("")
|
||
print("[FAIL] L0a 预检未过——先按上面指名的钩子/包修配置或环境,再跑门禁")
|
||
if not args.keep_going:
|
||
return 2
|
||
|
||
rc = 0
|
||
# 实际跑过哪些段——收尾那行只报这些。理由与那个盲区同源且是同一类错误:
|
||
# "--l05-only 却宣布 L0/L0b/L0c 全部通过"就是把没跑的说成跑过了(SKIP 冒充 PASS
|
||
# 的镜像版)。2026-10-25 做 task-10 时从 --l05-only 的输出里发现,当场改掉。
|
||
ran = ["L0a"] # 钩子解释器预检已在本脚本开头无条件跑过,收尾行照实列出
|
||
try:
|
||
if args.size_only:
|
||
return run_size_gate()
|
||
if not args.l05_only:
|
||
rc = run_l0()
|
||
ran.append("L0")
|
||
# 提交信息门禁独立于 --all-files(stage 不同),故单独跑一段;
|
||
# 即使 L0 已失败也照跑,一次看全所有违规项。
|
||
rc_msg = run_gitlint(args.commits)
|
||
ran.append("L0b")
|
||
# L0c 体积门禁同样**无条件照跑**:它回答的是"这份代码测过体积没有",
|
||
# 与 L0/L0b 的结论正交,且只要 0.3 s——少跑一次就是那个盲区。
|
||
rc_size = run_size_gate()
|
||
ran.append("L0c")
|
||
if rc != 0 or rc_msg != 0 or rc_size != 0:
|
||
if not args.keep_going:
|
||
print("")
|
||
print("[FAIL] L0 / L0b / L0c 有未通过项 —— 中止"
|
||
"(--keep-going 可继续跑 L0.5)")
|
||
return 1
|
||
rc = rc or rc_msg or rc_size
|
||
if not args.l0_only:
|
||
rc05 = run_l05(args)
|
||
ran.append("L0.5")
|
||
rc = rc or rc05
|
||
if args.coverage:
|
||
rc = rc or run_coverage()
|
||
ran.append("L0.5c")
|
||
except FileNotFoundError as e:
|
||
print("[ERROR] 环境错误:{}".format(e))
|
||
return 2
|
||
|
||
print("")
|
||
print("=" * 64)
|
||
if rc == 0:
|
||
print(" [OK] 本地 CI 全部通过({})".format(" + ".join(ran)))
|
||
else:
|
||
print(" [FAIL] 本地 CI 有未通过项(见上方输出)")
|
||
print("=" * 64)
|
||
return rc
|
||
|
||
|
||
if __name__ == "__main__":
|
||
sys.exit(main())
|