feat(台架工具): 原子性复验能力并进既有 verify_log 并接进 @test——旧口径判 PASS 的缺陷新口径判 FAIL

关键决定是**不造第二只校验器**：卡片原建议把校验器放进 scripts/，但「标记校验」这个能力已归 .dsh/skills/serial-boot-test/scripts/verify_log.py，再造一只就是本仓反对的实现漂移（reset.bat 折进技能是同一类判断）。改为扩展它 + 把开关接进 run_boot_test.py ——**不接就等于吸收了没人用得到**；默认路径逐字节不变（不传开关时 exact_check 根本不进）。新增：--exact 整行锚定（并把「含最长字面串却未整行命中」的邻居原样打出来，那就是行级交错的特征）、--pair 跨复位配对、--allow-residual 与 --head-char 服务崩溃注入腿、 --include-comments 复现旧行为、--selftest 判别自检。钉长度改成显式 @@LEN=<n> 逃生口， **不再是默认**。最硬的一条实证（进文档表格，也钉成一条 selftest 用例）：同一份缺陷捕获——行首被吃、尾巴完好的那行 NAND_FTL——旧 --required 子串口径判 **PASS（假绿）**，--exact 判 **FAIL 并打出缺行首的邻居**。口径据此改写：凡是这一行可能丢行首或被插字节的标记，一律 --exact，不许只用子串。实测不是声称：--selftest 16/16；真日志三向（修复后 PASS / 修复前 FAIL+邻居 / 带 ASCII 溯源头不再被头里的 [ERR] 误报）；被动烟测（--flash-cmd 置空 = 不烧录、不动板上镜像、只采 60 行空闲流量，bench 锁取了即还）证明四个开关在 @test 闭环里**真的执行了**；空闲流量里 PROF 行长 124/78/123 交替，顺手再证一次「钉长度当默认」会天天假红。作者自报四个自己的问题（都被自测抓到并修掉，卡里也写了）：残留行被重复计成交错邻居（会假红）、一条断言写反、**CLI 参数忘了接进 verify()——最危险那种：功能静默空跑而外面看着全绿**、以及误报 run_boot_test.py 变 CRLF（实为自己的检查把换行符写成了字面量，两个文件复核均为纯 LF，别按那句去查）。 boot_log_data/README.md 里同时有两个道的内容：bench23 的台账行，与 task-25 那条道写的「证据可达性」规则与 force-add 定案——autostash 往返后两侧都逐一确认仍在。三份 bench23 证据 txt 首次入库（可再生的采集文本；两张非再生 .bin 已随前一笔入库）。
2026-10-26 23:42:34 +08:00
parent 496b4dc5e2
commit b1280e3cd1
9 changed files with 1283 additions and 8 deletions
@@ -14,7 +14,9 @@ whenToUse: 需要向目标板烧录或复位后，用串口日志自动判定运
 | 脚本 | 作用 |
 |---|---|
 | `.dsh/skills/serial-boot-test/scripts/run_boot_test.py` | 一键闭环：open+clear → 烧录 → wait-idle → 提取 boot log → 烧录回读校验 → 标记校验。串口助手按 `$DSH_SERIAL_TOOL` > 同目录 > `../serial-assistant/scripts/serial_tool.py` 解析 |
-| `.dsh/skills/serial-boot-test/scripts/verify_log.py` | 对任意日志文件做标记校验（Required/Forbidden/锚点计数/顺序/耗时），可单独用 |
+| `.dsh/skills/serial-boot-test/scripts/verify_log.py` | 对任意日志文件做标记校验（Required/Forbidden/锚点计数/顺序/耗时），可单独用。`--exact` 做**整行 `^…$` 锚定**（子串口径照不出"行还在但行首被吃掉/行内被插字节"那一类损坏）、`--pair` 做跨复位配对、`--allow-residual/--head-char` 服务崩溃注入腿、`--selftest` 跑内置判别用例（**证明这套判据不是空跑**）。四个行级开关（`--exact` / `--pair` / `--allow-residual` / `--head-char`）同时接进了 `run_boot_test.py`，@test 闭环直接可用 |
+
+> **判据取向（重要，会复发）**：`--exact` 默认**只锚文本、不钉长度**——字节长度属于某一份构建，钉死它会在无辜改动上假红（`@@LEN=<n>` 是显式逃生口，不是默认）。子串 `--required` 不能用于"可能丢行首/被插字节"的行。来龙去脉与固定动作见本仓 `docs/测试说明/启动日志原子性复验流程.md`。

 交互式串口操作（open/send/log/wait-idle/status 等）不属于本 skill——
 **用 `serial-assistant` skill**。
@@ -392,6 +392,23 @@ def main(argv=None):
    ap.add_argument("--require-order", default=None,
                    help="'>'-separated markers that must appear in that line order "
                         "(catches out-of-order boot steps); empty = off")
+    ap.add_argument("--exact", action="append", default=[],
+                    help="repeatable regex that must match a WHOLE firmware line "
+                         "(forwarded to verify_log.py --exact). Use it instead of --required "
+                         "for any marker whose line could lose its head or gain spliced "
+                         "bytes -- substring matching cannot see that class of damage. "
+                         "Suffix '@@LEN=<n>' pins byte length: opt-in only, lengths belong "
+                         "to one image")
+    ap.add_argument("--pair", action="append", nargs=2, metavar=("FIRST", "THEN"), default=[],
+                    help="repeatable cross-reset pairing (exactly one FIRST, then exactly "
+                         "one THEN after it), e.g. --pair MEMFAULT 'reset cause:'")
+    ap.add_argument("--allow-residual", type=int, default=0,
+                    help="accept up to N stray bytes before the line head on a --exact hit "
+                         "(crash-dump legs: 1 leftover from a reset mid-byte is known; "
+                         "recorded, not a failure)")
+    ap.add_argument("--head-char", default="",
+                    help="every --exact hit must start with this char after the optional "
+                         "residual prefix (crash-dump legs: '[')")
    ap.add_argument("--done-marker", default=None,
                    help="marker whose line marks boot completion (for --max-boot-ms); "
                         "empty = don't measure")
@@ -743,7 +760,9 @@ def main(argv=None):
                expect_reset_cause=args.expect_reset_cause,
                require_order=args.require_order,
                done_marker=args.done_marker,
-                max_boot_ms=args.max_boot_ms)
+                max_boot_ms=args.max_boot_ms,
+                exact=args.exact, pairs=args.pair,
+                allow_residual=args.allow_residual, head_char=args.head_char)
    if selftest_forced_fail and rc == 0:
        # 标记校验全过，但自检台账判定根本没能执行（无数据源 / 抓取失败 / 无法证明无新增）
        # —— 这类"判不了"必须以 FAIL 收口，否则就又回到假绿。
@@ -114,6 +114,106 @@ def _read_lines(path):
    return raw.decode("utf-8", errors="replace").splitlines()


+# --- whole-line exact matching (2026-10-26, task-36; absorbed from bench-runner scratch) ---
+# Why this exists (measured, not theoretical): -Required is a SUBSTRING check, and the
+# console-interleave defect kept the substring while destroying the line, e.g. the
+# headless victim line "26-10-26 08:37:07][NAND_FTL] Initializing NAND FTL..." still
+# contains the marker "Initializing NAND FTL" -> the old check called that boot a PASS.
+# Any marker whose line can lose its head is therefore NOT verifiable by substring alone.
+# --exact anchors the WHOLE payload (^ ... $) so one spliced byte breaks the match, and it
+# also reports lines containing the pattern's longest literal run that do NOT match fully:
+# that is the splicing signature and it must never stay silent. Length pinning is OPT-IN
+# ("@@LEN=<n>"): byte lengths belong to one image, pinning them by default gives a false
+# red on innocent commits (measured uiTask stack swing 4116..4188 on ONE image).
+EXACT_LEN_SEP = "@@LEN="
+_DAEMON_RX = re.compile(r"^\[RX\] \S+ (.*)$")
+_RUNBT_RX = re.compile(r"^\[\+\d+ms\] (.*)$")
+
+
+def _parse_exact(spec):
+    """'REGEX@@LEN=42' -> (regex, 42); no suffix -> (regex, None)."""
+    if EXACT_LEN_SEP in spec:
+        pat, _, ln = spec.rpartition(EXACT_LEN_SEP)
+        return pat, int(ln.strip())
+    return spec, None
+
+
+def _payload_of(line):
+    """Drop the collector prefix so ^...$ applies to the firmware line itself.
+
+    Handles serial-assistant daemon lines ('[RX] <iso> ...'), run_boot_test boot logs
+    ('[+Nms] ...') and bare payload lines. Archived evidence files may carry a leading
+    '#' provenance header; those are dropped unless include_comments is set (otherwise a
+    header that merely MENTIONS [ERR] would trip Forbidden / pollute exact counts).
+    """
+    for rx in (_DAEMON_RX, _RUNBT_RX):
+        m = rx.match(line)
+        if m:
+            return m.group(1)
+    return line
+
+
+def _longest_literal(pattern):
+    """Longest run of non-metacharacters in a regex -> a greppable neighbour probe."""
+    parts = re.split(r"\\.|\\\[|\]|\(|\)|\{|\}|\||\*|\+|\?|\^|\$|\.|\[:", pattern)
+    parts = [x for x in parts if x and not x.startswith("\\")]
+    return max(parts, key=len) if parts else ""
+
+
+def exact_check(payloads, spec, allow_residual=0, head_char=""):
+    """(ok, detail, neighbours, residuals) for one --exact spec. See block note above."""
+    pat, want_len = _parse_exact(spec)
+    rx = re.compile("^" + pat + "$")
+    hits, residuals, consumed = [], [], set()
+    for idx, pl in enumerate(payloads):
+        if rx.match(pl):
+            hits.append(pl)
+            consumed.add(idx)
+            continue
+        if allow_residual > 0:
+            for k in range(1, allow_residual + 1):
+                if len(pl) > k and pl[k] == "[" and rx.match(pl[k:]):
+                    hits.append(pl[k:])
+                    residuals.append((k, pl[:k]))
+                    consumed.add(idx)
+                    break
+    key = _longest_literal(pat)
+    # a line counted (directly or after residual realignment) is never also a neighbour
+    neigh = [pl for i2, pl in enumerate(payloads) if key and key in pl and i2 not in consumed]
+    detail = "hits={} lens={} loose_neighbours={}".format(
+        len(hits), [len(h) for h in hits], len(neigh))
+    if residuals:
+        detail += " residual_head_bytes={}".format([(k, repr(v)) for k, v in residuals])
+    ok = len(hits) >= 1 and not neigh
+    if want_len is not None:
+        ok = ok and all(len(h) == want_len for h in hits)
+        detail += " want_len={}".format(want_len)
+    if head_char:
+        bad = [h for h in hits if not h.startswith(head_char)]
+        ok = ok and not bad
+        detail += " head_char={!r} bad={}".format(head_char, len(bad))
+    return ok, detail, neigh[:3], residuals[:3]
+
+
+def pair_check(payloads, first_pat, then_pat):
+    """Exactly one FIRST line with exactly one THEN line after it (cross-reset pairing).
+
+    Crash-dump legs need this: the fault line must pair with the NEXT boot's reset cause,
+    and a second following cause would mean an unlogged or repeated reset.
+    """
+    frx, trx = re.compile(first_pat), re.compile(then_pat)
+    fi = [i for i, pl in enumerate(payloads) if frx.search(pl)]
+    ti = [i for i, pl in enumerate(payloads) if trx.search(pl)]
+    if len(fi) != 1:
+        return False, "first matches={} (want exactly 1), then matches={}".format(len(fi), len(ti))
+    after = [i for i in ti if i > fi[0]]
+    if len(after) != 1:
+        return False, "first at L{} but following THEN count={} at {} (want exactly 1)".format(
+            fi[0] + 1, len(after), [x + 1 for x in after])
+    return True, "first L{} -> only following THEN at L{} (gap {} line(s))".format(
+        fi[0] + 1, after[0] + 1, after[0] - fi[0])
+
+
 def _reset_cause_of(line, marker):
    """从 `reset cause: software (NVIC_SystemReset) (CSR=0x14000000)` 取原因文本。"""
    i = line.find(marker)
@@ -129,7 +229,8 @@ def verify(log_path, required=REQUIRED_DEFAULT, forbidden=FORBIDDEN_DEFAULT,
           selftest_pass=False, selftest_log="",
           boot_marker=BOOT_MARKER_DEFAULT, max_boot_count=0,
           expect_reset_cause="", require_order="", done_marker="",
-           max_boot_ms=0, as_json=False):
+           max_boot_ms=0, as_json=False, exact=None, pairs=None,
+           allow_residual=0, head_char="", include_comments=False):
    """标记校验核心（可被 run_boot_test.py 复用）。返回退出码 0/1/2。

    suite_summary: ';' 分隔的套件名列表。非空时，要求日志中每个套件都出现
@@ -151,7 +252,8 @@ def verify(log_path, required=REQUIRED_DEFAULT, forbidden=FORBIDDEN_DEFAULT,
    """
    res = {"log_path": str(log_path), "lines": 0, "bytes": 0, "required_missing": [],
           "forbidden_hits": [], "boot": None, "suites_bad": [], "selftest": None,
-           "order_bad": [], "timing": None, "tail": [], "verdict": None, "exit": None}
+           "order_bad": [], "timing": None, "tail": [], "verdict": None, "exit": None,
+           "exact_bad": [], "exact_ok": [], "pair_bad": [], "pair_ok": []}
    emit = out

    def finish(code):
@@ -173,6 +275,10 @@ def verify(log_path, required=REQUIRED_DEFAULT, forbidden=FORBIDDEN_DEFAULT,
    except OSError:
        out("VERIFY_ERROR: cannot read log file: {}".format(log_path))
        return finish(2)
+    if not include_comments:
+        # archived evidence files carry an ASCII provenance header; a header that merely
+        # MENTIONS [ERR] must not trip Forbidden (tool-level false-red guard, task-36)
+        lines = [l for l in lines if not l.lstrip().startswith("#")]

    full = "\n".join(lines)
    req_list = _split_list(required)
@@ -213,6 +319,37 @@ def verify(log_path, required=REQUIRED_DEFAULT, forbidden=FORBIDDEN_DEFAULT,
        else:
            out("[OK]   forbidden pattern absent: {}".format(f))

+    # ---- whole-line exact checks + cross-reset pairing (task-36) ----
+    payloads = [_payload_of(l) for l in lines]
+    res["payload_lines"] = len(payloads)
+    exact_bad = []
+    for spec in (exact or []):
+        pat, _want = _parse_exact(spec)
+        ok_x, detail, neigh, residuals = exact_check(payloads, spec, allow_residual, head_char)
+        if ok_x:
+            out("[OK]   exact line: {}  {}".format(pat[:64], detail))
+            res["exact_ok"].append({"pattern": pat, "detail": detail})
+        else:
+            out("[FAIL] exact line: {}  {}".format(pat[:64], detail))
+            for nb in neigh:
+                out("       inexact neighbour: {}".format(nb.strip()[:170]))
+            exact_bad.append(pat)
+            res["exact_bad"].append({"pattern": pat, "detail": detail,
+                                     "neighbours": [n.strip()[:170] for n in neigh]})
+        for k, v in residuals:
+            out("[NOTE] {} residual byte(s) before the line head, realigned and NOT a "
+                "failure: {}".format(k, repr(v)))
+    pair_bad = []
+    for f_pat, t_pat in (pairs or []):
+        ok_p, detail = pair_check(payloads, f_pat, t_pat)
+        if ok_p:
+            out("[OK]   pair: {} -> {}".format(f_pat[:44], t_pat[:44]))
+            res["pair_ok"].append({"first": f_pat, "then": t_pat, "detail": detail})
+        else:
+            out("[FAIL] pair: {} -> {}  {}".format(f_pat[:44], t_pat[:44], detail))
+            pair_bad.append({"first": f_pat, "then": t_pat, "detail": detail})
+            res["pair_bad"].append(detail)
+
    # 启动锚点计数（2026-10-10）：抓"跑完但中途复位过"和"采集没从复位开始"
    boot_issue = False
    if max_boot_count > 0:
@@ -384,13 +521,75 @@ def verify(log_path, required=REQUIRED_DEFAULT, forbidden=FORBIDDEN_DEFAULT,

    out("")
    if (missing or hits or bad_summary or selftest_fail or boot_issue
-            or order_issue or timing_issue):
+            or order_issue or timing_issue or exact_bad or pair_bad):
        out("VERDICT: FAIL")
        return finish(1)
    out("VERDICT: PASS")
    return finish(0)


+def selftest(out=print):
+    """Prove the exact/pair machinery actually discriminates (exit 0 = all behaviours ok).
+
+    Written because a PASS that cannot fail is worthless: case 2/3 are the real captured
+    shapes of the 2026-10-26 console-interleave defect, and case 9 shows that the legacy
+    substring check calls the SAME defective line a PASS -- that is the blind spot --exact
+    exists to close.
+    """
+    T = "[2026-10-26 08:00:00]"
+    prot_re = (r"\[\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\]\[PROT\] uncalibrated -> protection "
+               r"blocked \(UA/UN/IA VALID not all set\)")
+    nand_re = r"\[\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\]\[NAND_FTL\] Initializing NAND FTL\.\.\."
+    prot_ok = T + "[PROT] uncalibrated -> protection blocked (UA/UN/IA VALID not all set)"
+    prot_spliced = T + "[PROT] uncalibrated -> protection [blocked (UA/UN/IA VALI2D not all s0et)"
+    nand_ok = T + "[NAND_FTL] Initializing NAND FTL..."
+    nand_headless = "26-10-26 08:37:07][NAND_FTL] Initializing NAND FTL..."
+    mem_re = r"\[ERR\] MEMFAULT PC=\S+ LR=\S+ CFSR=\S+ HFSR=\S+"
+    mem_ok = "[ERR] MEMFAULT PC=0x08004a12 LR=0x08004a3f CFSR=0x00008200 HFSR=0x40000000"
+    rc_ok = T + "[APP] reset cause: software (NVIC_SystemReset) (CSR=0x14000000)"
+    cases = []
+
+    def chk(name, got, want):
+        cases.append((name, got == want, "got={!r} want={!r}".format(got, want)))
+
+    chk("clean exact", exact_check([prot_ok], prot_re)[0], True)
+    ok_s, _d_s, neigh_s, _r_s = exact_check([prot_spliced], prot_re)
+    chk("spliced line rejected", ok_s, False)
+    chk("spliced line fails even with broken probe key", len(neigh_s), 0)
+    ok_n, _d_n, neigh_n, _r_n = exact_check([nand_headless], nand_re)
+    chk("headless line rejected", ok_n, False)
+    chk("headless line reported as neighbour", len(neigh_n) >= 1, True)
+    chk("residual 1 rejected when not allowed", exact_check(["#" + nand_ok], nand_re)[0], False)
+    ok_r, det_r, _n_r, res_r = exact_check(["#" + nand_ok], nand_re, allow_residual=1)
+    chk("residual 1 accepted when allowed", ok_r and len(res_r) == 1, True)
+    chk("residual 2 still rejected", exact_check(["##" + nand_ok], nand_re, allow_residual=1)[0], False)
+    chk("head_char enforced", exact_check([nand_ok], nand_re, 0, "[")[0], True)
+    chk("head_char catches wrong head", exact_check([nand_headless], r"26-10-26.*NAND_FTL.*", 0, "[")[0], False)
+    chk("len pin ok", exact_check([nand_ok], nand_re + EXACT_LEN_SEP + str(len(nand_ok)))[0], True)
+    chk("len pin mismatch fails", exact_check([nand_ok], nand_re + EXACT_LEN_SEP + "9")[0], False)
+    chk("pair good", pair_check([mem_ok, rc_ok], "MEMFAULT", "reset cause:")[0], True)
+    chk("pair two following causes fails",
+        pair_check([mem_ok, rc_ok, rc_ok], "MEMFAULT", "reset cause:")[0], False)
+    chk("pair wrong order fails",
+        pair_check([rc_ok, mem_ok], "MEMFAULT", "reset cause:")[0], False)
+    # blind spot: legacy substring would PASS on this defective line
+    full = "\n".join([prot_ok, nand_headless])
+    legacy_present = "Initializing NAND FTL" in full          # what --required sees
+    exact_ok_here = exact_check([prot_ok, nand_headless], nand_re)[0]
+    chk("blind spot proven: legacy substring PASSES where --exact FAILS",
+        legacy_present and not exact_ok_here, True)
+
+    bad = 0
+    for name, ok, detail in cases:
+        out("{} {}".format("[OK]  " if ok else "[FAIL]", name))
+        if not ok:
+            out("       {}".format(detail))
+            bad += 1
+    out("SELFTEST: {} case(s), {} failure(s) -> {}".format(
+        len(cases), bad, "PASS" if bad == 0 else "FAIL"))
+    return 0 if bad == 0 else 1
+
+
 def main(argv=None):
    # 控制台输出固定 UTF-8：Windows 代码页（GBK/UTF-8 终端）均不会让本脚本崩溃
    for stream in (sys.stdout, sys.stderr):
@@ -431,10 +630,36 @@ def main(argv=None):
    ap.add_argument("--max-boot-ms", type=int, default=0,
                    help="fail if time from first line to --done-marker exceeds N ms "
                         "(needs '[+Nms]' line prefix; 0 = off)")
+    ap.add_argument("--exact", action="append", default=[],
+                    help="repeatable regex that must match a WHOLE firmware line "
+                         "(collector prefix stripped, anchored ^...$); a spliced byte or a "
+                         "missing line head fails. Optional suffix '@@LEN=<n>' pins the byte "
+                         "length -- opt-in only, lengths belong to one image. Use --exact "
+                         "instead of --required for any marker whose line can lose its head.")
+    ap.add_argument("--allow-residual", type=int, default=0,
+                    help="accept up to N stray bytes before the line head on an --exact hit "
+                         "(crash-dump legs: 1 shift-register leftover is known; recorded, "
+                         "not a failure). 0 = strict (default)")
+    ap.add_argument("--head-char", default="",
+                    help="every --exact hit must start with this char right after the "
+                         "optional residual prefix (crash-dump legs: '[')")
+    ap.add_argument("--pair", action="append", nargs=2, metavar=("FIRST", "THEN"), default=[],
+                    help="repeatable pair: exactly one FIRST line, and exactly one THEN line "
+                         "after it (cross-reset pairing, e.g. 'MEMFAULT' 'reset cause:')")
+    ap.add_argument("--include-comments", action="store_true",
+                    help="do NOT skip '#' lines (archived evidence files carry an ASCII "
+                         "provenance header; skipping it is the default so a header that "
+                         "merely mentions [ERR] cannot trip --forbidden)")
+    ap.add_argument("--selftest", action="store_true",
+                    help="run the built-in discrimination cases and exit (0 = all behave "
+                         "as specified); needs no log file and no hardware")
    ap.add_argument("--json", action="store_true",
                    help="emit one JSON object instead of the text verdict (exit code unchanged)")
    args = ap.parse_args(argv)

+    if args.selftest:
+        return selftest()
+
    return verify(args.log_path, args.required, args.forbidden,
                  args.show_tail_lines, suite_summary=args.require_suite_summary,
                  selftest_pass=args.require_selftest_pass,
@@ -445,7 +670,10 @@ def main(argv=None):
                  require_order=args.require_order,
                  done_marker=args.done_marker,
                  max_boot_ms=args.max_boot_ms,
-                  as_json=args.json)
+                  as_json=args.json,
+                  exact=args.exact, pairs=args.pair,
+                  allow_residual=args.allow_residual, head_char=args.head_char,
+                  include_comments=args.include_comments)


 if __name__ == "__main__":