Files
STM32F4-Base/test/run_all_suites.py
T
wandering1 10323c4c22 feat(ioboard): board test suite for DI/DO, fix default_task conditional compile
Bench verification of the IO board needs an observation window: production
build has no consumer of ioboard_di_snap_read(), so the DI values were
invisible. Add TEST_SUITE_IOBOARD as a semi-automatic bench sequence and fix
two real bugs found while wiring it in.

New files:
- test/board/ioboard_test.h, test/board/ioboard_test_task.c

Test suite:
- Owns its own thread (iobTest, 4096 words) instead of running in testTask:
  testTask is osPriorityBelowNormal, below defaultTask whose 1ms console
  drain loop can starve a blocking wait in a lower priority context.
- Phases: bit-order self-check, DI eight-point bijection, DO point-by-point
  toggle with operator confirmation, release-all on every exit path.
- Only exit is ioboard_release_all(); it is called from the entry guard,
  every phase failure, the write-failure branch and the thread tail.
- Sample window is 20 reads x 2ms = 40ms and does overlap the 50ms
  production poll. That is stated as harmless in the header: the driver
  wraps the whole read in taskENTER_CRITICAL (26us, not the whole 40ms),
  both sides only read, and a torn cross-reader sample is caught by the
  debounce layer rather than acted upon.

default_task.c fix:
- DEFAULT_TASK_DI_PUBLISH_EN was defined twice: once at file scope and once
  inside default_task_func(). #define has no scope, so the second was a
  redefinition of the first. Collapsed to a single authoritative site.
- Add TEST_SUITE_STORAGE to the disable list: the STORAGE suite owns the
  file system via testTask, and default_task_di_poll() referenced four
  statics that live inside an !TEST_SUITE_STORAGE block, which broke the
  STORAGE build with 11 errors.
- Shrink the !TEST_SUITE_STORAGE block to cover only
  default_task_net_cfg_save_notify() and move the three declarations out;
  move edges/fails into the #else branch so the empty branch references
  none of the four statics.

Registration and gates:
- MDK-ARM/STM32F407-Demo.uvprojx and simulator/CMakeLists.txt entries.
- scripts/check_test_suite.py: whitelist TEST_SUITE_IOBOARD.
- scripts/stack_report.py: let a self-started thread that is not declared in
  Src/freertos.c osThreadAttr_t still be budget checked.
- test/run_all_suites.py: bring the suite into the orchestrator view.

Verified: six-build matrix all 0 errors 0 warnings. CH395F/STORAGE/GD5F/RTC
and the product build are bit-identical at Code=257936; the IOBOARD build is
Code=264640 (+6704), which is the evidence that the suite really enters the
image. Host unit tests 100/100, stack report 1760/4096 words (57%).
2026-11-16 19:35:15 +08:00

309 lines
16 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""run_all_suites.py - 多套件回归编排(纯标准库)。
做什么(2026-10-01 测试架构评审):
套件互斥(`test_config.h` 的 `#error`)+ 编译期开关 → 跑 4 个套件的回归
本来要手工循环 4 轮(改宏 → 构建 → 烧录 → 采集 → 判读)。本脚本把它编排成一条命令:
对每个套件:
1. 改 test_config.h 启用该套件(其它全注释)
2. /keil-build(0 错 0 警,否则跳过该套件)
3. 烧录(stlink-flash skill:stlink_flash.py flash,Reset & Run)
4. /serial-boot-test 采集 + 判读(.dsh/skills/serial-boot-test 的 verify_log.py,含套件汇总行校验)
结束后:
5. **无条件还原 test_config.h**(try/finally,含异常/KeyboardInterrupt 路径)
为什么要还原(这是本脚本存在的核心理由):
跑完一轮忘了还原 = **测试固件进库**(与提交期门禁同源风险)。提交期门禁是兜底,
本脚本是"流程自动化"——两条防线各自独立。
⚠️ 代价提醒:
每轮要真机烧录 + 采集(8s 静默 + boot),4 套件 ≈ 数分钟起;破坏性套件还会改设备
持久状态。**仅回归窗口用,绝不挂 pre-commit。**
退出码:
0 = 全部套件 PASS
1 = 有套件 FAIL(构建失败 / 采集错误 / 标记判定失败)
2 = 环境/用法错误(缺文件、无法还原配置等)
用法(仓库根目录):
& '.venv\\Scripts\\python.exe' test/run_all_suites.py
& '.venv\\Scripts\\python.exe' test/run_all_suites.py --suites STORAGE,GD5F
& '.venv\\Scripts\\python.exe' test/run_all_suites.py --dry-run # 只打印计划
& '.venv\\Scripts\\python.exe' test/run_all_suites.py --check-build # 只编译不烧录(无需 MCU)
& '.venv\\Scripts\\python.exe' test/run_all_suites.py --keep-config # 不还原(调试用,慎)
--check-build:
只做"改宏 + /keil-build",不烧录不采集,故**不需要 MCU / 串口 / ST-LINK**。
用途:① 改动测试固件或 test_config.h 后自检 4 套件仍可编译;
② 防止"某套件长期编不过而无人知"(2026-08~09 CH395F 套件实例)。
注意 RAM 溢出会自然暴露为链接错误 L6406E——这正是想要的。
设计依据:docs/测试说明/流程与规范/测试开发标准流程.md §3.1
创建日期:2026-10-01
"""
import argparse
import re
import subprocess
import sys
import time
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CONFIG = REPO_ROOT / "test" / "board" / "test_config.h"
KEIL_BUILD = REPO_ROOT / ".dsh" / "skills" / "keil-build" / "scripts" / "keil_build.py"
STLINK_FLASH = REPO_ROOT / ".dsh" / "skills" / "stlink-flash" / "scripts" / "stlink_flash.py"
RUN_BOOT = REPO_ROOT / ".dsh" / "skills" / "serial-boot-test" / "scripts" / "run_boot_test.py"
# 套件态的 forbidden 覆盖(判据边界见下方注释块,正文只这一处)。
FORBIDDEN_SUITE = "HardFault;MEMFAULT;BUSFAULT;USAGEFAULT;assert;bring-up FAIL"
# 套件 -> (宏名, verify_log 需要的汇总行名称, 该套件的 Required 标记, --forbidden 覆盖)
# 汇总行名称与套件 `XXX_TEST_REPORT("...")` 里的字符串一致(见各模块 Test_Guide)。
# forbidden 覆盖:套件含"预期 [ERR]"的守卫负路径时须覆盖(空 = 用默认
# `HardFault;assert;[ERR]`,与 .serialtest/config.json 的产品判据同源)。
#
# 为什么不能只是"把 [ERR] 删掉"(2026-10-25 台架实测 + verify_log 源码核对):
# 四种 Fault 的崩溃行长这样:`[ERR] HARDFAULT|MEMFAULT|BUSFAULT|USAGEFAULT …`
# (Src/stm32f4xx_it.c 的 hard_fault_report),而 verify_log 的 forbidden 是**大小写
# 不敏感的子串匹配**(fl in ln.lower())→ 默认集合里的 `HardFault` 能命中硬Fault,
# 但 MEM/BUS/USAGE 三种**平时完全靠 [ERR] 兜**。所以旧 GD5F 只写 `HardFault;assert` 其实
# 放过了后三种崩溃。覆盖集合一律显式列出四个 Fault 标签 + assert + 起网失败字样:
# 豁免的是**设计内的守卫 [ERR]**,崩溃照旧判失败。
# 产品 /serial-boot-test 判据(.serialtest/config.json 的 forbidden 含 [ERR])**不在这里动**——
# 豁免只活在"套件回归"这条路上,渗不回产品路径。
# GD5F:TC-GD5F-704 故意用非 DMA 可达缓冲触发 `[ERR][NAND] RX DMA buffer not
# DMA-reachable` 回落轮询(用例 PASS)→ 需豁免设计内 [ERR]。
# CH395F:Phase 15 的 sn 越界负向用例命中驱动守卫 `[ERR][CH395] ch395f P2
# boundary fault @get_state sn`(Drivers/BSP/CH395F/ch395f.c 的
# ch395f_p2_fault_note():首次留痕、之后只计数)→ 同属设计内。
# 另外 `[ERR][WD] task stall` 那类**不是设计内**,是构建配置缺陷(HW 层
# 测试构建下必需心跳掩码仍含 netTask),已单独提修,不用豁免掩盖它。
SUITES = {
"CH395F": ("TEST_SUITE_CH395F", None, "", FORBIDDEN_SUITE),
"STORAGE": ("TEST_SUITE_STORAGE", "Storage Tests", "=== Storage Tests:", ""),
"GD5F": ("TEST_SUITE_GD5F", "GD5F2GQ5UE Tests", "=== GD5F2GQ5UE Tests:", FORBIDDEN_SUITE),
"RTC": ("TEST_SUITE_RTC", "RTC Tests", "=== RTC Tests:", ""),
# IOBOARD:半自动台架套件(人工按提示短接 DI / 听继电器动作)。--check-build 只编译;
# 完整回归要求操作员在场跟着日志节拍动手,汇总行 `=== IOBOARD Tests:` 是判定标记。
"IOBOARD": ("TEST_SUITE_IOBOARD", "IOBOARD Tests", "=== IOBOARD Tests:", ""),
}
# 套件名 -> 汇总行里的名称可能因模块而异,故上面的 summary 值仅为默认;实际以
# `--required` 给出的前缀做 Required 匹配,`--require-suite-summary` 做严格解析。
_ALL_MACROS = [m for m, _, _, _ in SUITES.values()]
def _rewrite_config(enable_macro):
"""把 test_config.h 里所有 TEST_SUITE_* 开关设为只启用 enable_macro(None = 全注释)。
只动"选择开关"行(`#define TEST_SUITE_<NAME>` / `//#define TEST_SUITE_<NAME>`),
不碰派生的 TEST_SUITE_SELECTED / TEST_SUITE_NAME。
按原换行符与 BOM 状态回写(仓库编码铁律)。
"""
raw = CONFIG.read_bytes()
bom = raw.startswith(b"\xef\xbb\xbf")
text = raw.decode("utf-8-sig") if bom else raw.decode("utf-8")
newline = "\r\n" if "\r\n" in text else "\n"
lines = text.split(newline)
switch_re = re.compile(r"^\s*(//)?\s*#\s*define\s+(TEST_SUITE_\w+)\s*$")
out = []
for ln in lines:
m = switch_re.match(ln)
if m and m.group(2) in _ALL_MACROS:
name = m.group(2)
if enable_macro is not None and name == enable_macro:
out.append("#define {}".format(name))
else:
out.append("//#define {}".format(name))
else:
out.append(ln)
data = newline.join(out).encode("utf-8")
if bom:
data = b"\xef\xbb\xbf" + data
CONFIG.write_bytes(data)
def _run(cmd, timeout=None):
"""跑子进程,返回 (rc, 输出文本)。输出按字节捕获后 UTF-8 兜底解码。"""
try:
cp = subprocess.run(cmd, capture_output=True, timeout=timeout,
cwd=str(REPO_ROOT), shell=isinstance(cmd, str))
except subprocess.TimeoutExpired as e:
blob = (e.stdout or b"") + (e.stderr or b"")
return -1, blob.decode("utf-8", errors="replace")
blob = (cp.stdout or b"") + (cp.stderr or b"")
return cp.returncode, blob.decode("utf-8", errors="replace")
def main(argv=None):
for stream in (sys.stdout, sys.stderr):
try:
stream.reconfigure(encoding="utf-8", errors="replace")
except Exception:
pass
ap = argparse.ArgumentParser(description="多套件回归编排(纯标准库)")
ap.add_argument("--suites", default=",".join(SUITES.keys()),
help="要跑的套件,逗号分隔(默认全部:{})".format(",".join(SUITES)))
ap.add_argument("--build-timeout", type=int, default=300)
ap.add_argument("--flash-timeout", type=int, default=300)
ap.add_argument("--dry-run", action="store_true", help="只打印计划,不改配置不构建不烧录")
ap.add_argument("--check-build", action="store_true",
help="只编译不烧录:逐套件改宏 + /keil-build,验证 4 套件可编译性;"
"不需要 MCU / 串口 / ST-LINK,可用于 CI 或改动测试固件后自检")
ap.add_argument("--keep-config", action="store_true",
help="结束后不还原 test_config.h(调试用;⚠️ 会留下测试固件配置)")
args = ap.parse_args(argv)
py = sys.executable
picked = [s.strip().upper() for s in args.suites.split(",") if s.strip()]
bad = [s for s in picked if s not in SUITES]
if bad:
print("[ERROR] 未知套件:{}(可选:{})".format(
",".join(bad), ",".join(SUITES)))
return 2
# --check-build 不需要串口/烧录工具,故不强求 run_boot_test.py 存在
required = (CONFIG, KEIL_BUILD) if args.check_build else (CONFIG, KEIL_BUILD, STLINK_FLASH, RUN_BOOT)
for p in required:
if not p.is_file():
print("[ERROR] 缺文件:{}".format(p))
return 2
mode = "只编译(--check-build,不烧录)" if args.check_build else "完整回归(编译+烧录+采集)"
print("=== 多套件回归编排 ===")
print("模式:{}".format(mode))
print("套件:{}".format(", ".join(picked)))
print("构建:{}".format(KEIL_BUILD))
if not args.check_build:
print("烧录:{}".format(STLINK_FLASH))
print("判读:{}".format(RUN_BOOT))
if args.dry_run:
print("[DRY-RUN] 仅打印计划,未改动任何文件。")
return 0
results = {} # 套件 -> "PASS"/"FAIL"/"BUILD_FAIL"
restored = False
baseline = None # 进入时的 test_config.h 基线(try 内赋值;finally 依赖它,故先置 None)
try:
# 先确保从产品构建(干净起点)开始,避免上一轮的残留干扰
_rewrite_config(None)
baseline = CONFIG.read_bytes()
for suite in picked:
macro, summary_name, required, forbid = SUITES[suite]
print("")
print("################ {} ################".format(suite))
_rewrite_config(macro)
# 1) 构建(keil-build skill 执行体;脚本内超时先于 _run 超时触发,
# 超时会返回 rc=1 + BUILD_FAIL 文案,而不是被 _run 杀掉)
rc, out = _run([py, str(KEIL_BUILD), "--timeout", str(args.build_timeout)],
timeout=args.build_timeout + 60)
tail = "\n".join(out.strip().splitlines()[-6:])
print("[BUILD] rc={}".format(rc))
if tail:
print(tail)
if rc != 0:
print("[{}] BUILD_FAIL -> 跳过本套件".format(suite))
results[suite] = "BUILD_FAIL"
continue
# --check-build:编译通过即达标,不烧录不采集
if args.check_build:
results[suite] = "PASS"
print("[{}] BUILD_OK -> PASS(--check-build,未烧录)".format(suite))
continue
# 2) 烧录
t0 = time.monotonic()
frc, fout = _run([py, str(STLINK_FLASH), "flash"], timeout=args.flash_timeout)
print("[FLASH] rc={} elapsed={:.1f}s".format(frc, time.monotonic() - t0))
if frc != 0:
print("\n".join(fout.strip().splitlines()[-6:]))
# 烧录失败不直接判死:板子可能已复位,交由采集+判读决定(与 /serial-boot-test 一致)
print("[WARN] 烧录 rc={}(继续:板子可能已复位)".format(frc))
# 3) 采集 + 判读(含套件汇总行校验)
# 烧录统一走 stlink-flash skill(与 .serialtest/config.json 的 flash_cmd 同源)
flash_cmd = '"{}" "{}" flash'.format(py, STLINK_FLASH)
cmd = [py, str(RUN_BOOT), "--flash-cmd", flash_cmd]
if required:
cmd += ["--required", required]
if summary_name:
cmd += ["--require-suite-summary", summary_name]
# 套件构建预期可能出现 [ERR](守卫负路径)——由套件自身的 forbid 覆盖决定
# (空 = 默认 HardFault;assert;[ERR];GD5F = HardFault;assert,见 SUITES 注释)
cmd += ["--forbidden", forbid or "HardFault;assert;[ERR]"]
rc, out = _run(cmd, timeout=args.flash_timeout + 120)
print("\n".join(out.strip().splitlines()[-8:]))
results[suite] = "PASS" if rc == 0 else "FAIL"
print("[{}] -> {}".format(suite, results[suite]))
finally:
# 4) 无条件还原(本脚本存在的核心理由)
if args.keep_config:
print("")
print("[WARN] --keep-config:未还原 test_config.h(⚠️ 可能留下测试固件配置)")
else:
try:
_rewrite_config(None)
# 与进入时的基线一致性核对(防止还原逻辑本身出错)
now = CONFIG.read_bytes()
if baseline is not None and now != baseline:
print("[WARN] 还原后的配置与进入时基线不一致,请人工核对 {}".format(CONFIG))
else:
restored = True
print("")
print("[OK] 已还原 test_config.h 为产品构建(全部 TEST_SUITE_* 注释)")
except Exception as e: # noqa: BLE001 - 还原失败必须显式报出
print("[ERROR] 还原 test_config.h 失败:{}".format(e), file=sys.stderr)
# 5) 复位构建产物为产品态(2026-10-02):最后一轮构建留下的是**测试套件**的
# .map/.axf,会污染 mem-numbers 体积门禁(下次提交即 FAIL)。仅还原源码不够
# ——必须再编一次产品固件,使 .map 与 test_config.h 一起回到产品态。
# --check-build 也要做:它每轮都 /keil-build,同样会留下测试态 .map。
if restored and not args.keep_config:
print("")
print("[..] 复位构建产物为产品态(重建一次产品固件,避免 .map 停留在测试套件态)...")
rc, out = _run([py, str(KEIL_BUILD), "--timeout", str(args.build_timeout)],
timeout=args.build_timeout + 60)
tail = "\n".join(out.strip().splitlines()[-4:])
if rc == 0:
print("[OK] 产品固件已重建(.map/.axf 复位)")
else:
# 不判死:配置已还原,仅提示手工重建
print("[WARN] 产品固件重建失败(rc={}),请手工跑 /keil-build 复位 .map".format(rc))
if tail:
print(tail)
# 汇总
print("")
print("=== 编排结果{} ===".format("(--check-build)" if args.check_build else ""))
for s in picked:
print(" {:<8} {}".format(s, results.get(s, "SKIPPED")))
if not restored and not args.keep_config:
return 2
failed = [s for s, r in results.items() if r != "PASS"]
if failed:
print("[FAIL] 未通过:{}".format(", ".join(failed)))
if args.check_build:
print(" (--check-build:失败 = 该套件编译不过,需修测试固件代码)")
return 1
if args.check_build:
print("[OK] 全部套件编译通过(--check-build:未烧录、未采集)")
else:
print("[OK] 全部套件 PASS")
return 0
if __name__ == "__main__":
sys.exit(main())