wandering1/STM32F4-Base
1
0
Fork 0
Code Issues 30 Pull Requests Actions Packages Projects 2 Releases Wiki Activity

CI runner 修日志噪声与 -Status 假警：PS5.1 把原生 stderr 包成 ErrorRecord

Browse Source 
双看门狗缺陷已实测收敛：管理员 -StartNow 清掉 3 个残留（孤儿看门狗 26900/19716 +
占锁 daemon 53848）→ "daemon up: pid 40360 (single instance)"，监管链恢复为
svchost(任务) → 看门狗 → daemon，看门狗日志 25 秒零增长，runner 回 online。
切换瞬间新看门狗先失败 1~2 次（旧 daemon 刚死、锁未释放）后自愈，属预期。

顺带查出并修掉两个由验证动作暴露的问题：

1) 日志噪声：daemon 把日志写到 stderr，PS 5.1 的 2>&1 会把每行原生 stderr 包成
   ErrorRecord，Out-File 渲染出 "gitea-runner.exe : " 前缀 + 4 行 CategoryInfo 块。
   实测对比（用无效子命令产生真实 stderr）：& exe 2>&1 | Out-File 把 2 行真日志写成
   8 行——每 4 行有 3 行是 PowerShell 噪声，而且就在文档让大家第一个打开的文件里。
   改为 2>&1 | ForEach-Object { "$_" } | Out-File -Encoding utf8：实测与 cmd 包装等价
   但不用操心 cmd 对含空格路径的引号剥离，ErrorRecord.ToString() 即原始行，
   $LASTEXITCODE 不受影响（看门狗靠它打印退出码）。§7.2 方式 B 的示例原本正是
   "> log 2>&1"（UTF-16LE + ErrorRecord 双坑），已改；新增 §7.3 汇总这两条陷阱。

2) -Status 两处不诚实：① 非管理员读不到 SYSTEM 进程的 CommandLine（返回 null），
   过滤器静默匹配 0 个 → 会谎报"没有看门狗"（今天两次被这个假象误导），改为明说
   "需提权"；② 重启循环判据原本靠"日志尾部 40 行里有没有 lock 字样"，而日志不轮转，
   修好之后旧失败仍留在尾部 → 对健康 runner 报假警。改看 daemon 进程存活时长
   （活过 10 秒重拉间隔就不可能在循环里）；用 CIM CreationDate，因为
   Process.StartTime 对 SYSTEM 进程不可读。

验证：bootstrap parse 0 错 / 纯 ASCII / -Status 与 -DryRun 均正常；
check_doc_links.py --all 125 篇 0 问题；pre-commit run --all-files 17 项全 Passed。

注：磁盘上的 run_runner_watchdog.ps1 仍是旧模板，噪声修复要下次管理员 -StartNow
（或开机自启）时才生效，纯观感问题，不必专门再重启一次 runner。
This commit is contained in:
Wanderingss
2026-10-12 17:59:45 +08:00
parent 49f3e2dac5
commit 3557b60ceb
3 changed files with 84 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/ci_runner_bootstrap.ps1
+47 -11
View File
@@ -229,18 +229,44 @@ if ($Status) {
# A watchdog that cannot get the lock fails every 10s while the runner still
# reports ONLINE, so the loop is invisible from the web UI. Count watchdogs and
# grep the daemon log for the lock error (measured 2026-10-12).
$wd = @(Get-CimInstance Win32_Process -Filter "Name='powershell.exe'" -ErrorAction SilentlyContinue |
Where-Object { $_.CommandLine -match 'run_runner_watchdog\.ps1' })
if ($wd.Count -gt 1) {
Warn (" {0} watchdog processes (pids {1}) -- an old one survived a task re-registration" -f `
$wd.Count, (($wd | ForEach-Object { $_.ProcessId }) -join ','))
Warn ' and is racing the new one for .runner.lock; re-run elevated with -StartNow'
# MEASURED TRAP #3 (2026-10-12): a NON-ELEVATED session cannot read SYSTEM
# process command lines -- CommandLine comes back $null, so the filter below
# silently matches NOTHING and reporting "no watchdog" would be a lie (this
# bit us twice while diagnosing). Say what we cannot see instead.
if (-not (Test-Admin)) {
Info ' watchdog count skipped: needs elevation (SYSTEM command lines are hidden)'
} else {
$wd = @(Get-CimInstance Win32_Process -Filter "Name='powershell.exe'" -ErrorAction SilentlyContinue |
Where-Object { $_.CommandLine -match 'run_runner_watchdog\.ps1' })
if ($wd.Count -gt 1) {
Warn (" {0} watchdog processes (pids {1}) -- an old one survived a task re-registration" -f `
$wd.Count, (($wd | ForEach-Object { $_.ProcessId }) -join ','))
Warn ' and is racing the new one for .runner.lock; re-run elevated with -StartNow'
} elseif ($wd.Count -eq 1) {
Ok ("watchdog running: pid {0}" -f $wd[0].ProcessId)
} else {
Warn 'no watchdog process found (task not running?)'
}
}
$dlog = Join-Path $RunnerDir 'log\runner-daemon.log'
if (Test-Path -LiteralPath $dlog) {
$tail = Get-Content -LiteralPath $dlog -Tail 40 -Encoding UTF8 -ErrorAction SilentlyContinue
if ($tail -match 'already locked by another process') {
Warn ' daemon log says ".runner is already locked by another process" -- stale instance present'
# Restart-loop detection. Do NOT just grep the log tail: the file is never
# rotated, so old failures stay in the last lines forever and a HEALTHY runner
# keeps "reporting" a stale error (measured false positive 2026-10-12). The
# reliable signal is a daemon that keeps dying young -- a process alive longer
# than the 10s restart interval cannot be inside that loop. CIM CreationDate is
# readable without elevation; Process.StartTime is NOT for SYSTEM processes.
$dmon = Get-CimInstance Win32_Process -Filter "Name='gitea-runner.exe'" -ErrorAction SilentlyContinue |
Select-Object -First 1
$upSec = if ($dmon) { [int]((Get-Date) - $dmon.CreationDate).TotalSeconds } else { -1 }
if ($upSec -ge 90) {
Ok ("daemon up for {0} min -- no restart loop" -f [int]($upSec / 60))
} else {
$dlog = Join-Path $RunnerDir 'log\runner-daemon.log'
if (Test-Path -LiteralPath $dlog) {
$tail = Get-Content -LiteralPath $dlog -Tail 40 -Encoding UTF8 -ErrorAction SilentlyContinue
if ($tail -match 'already locked by another process') {
Warn ' daemon keeps dying young and the log shows ".runner is already locked'
Warn ' by another process" -- a stale instance holds the lock; re-run with -StartNow'
}
}
}
@@ -402,7 +428,17 @@ while ($true) {
# daemon lines read back as "t i m e = 2 0 2 6 ..." (a NUL after every byte).
# Never rely on a redirection's default encoding -- state it explicitly.
# utf8 (not ascii) because job/commit text can legitimately be non-ASCII.
# MEASURED TRAP #2 (2026-10-12): the daemon writes its log to STDERR, and in
# PS 5.1 a native `2>&1` merge wraps every stderr line into an ErrorRecord.
# Out-File then renders "gitea-runner.exe : <line>" plus a 4-line CategoryInfo
# block -- measured: a 2-line message became 8 lines, so 3 of every 4 lines in
# the daemon log were PowerShell noise, in the very file the docs tell people
# to read first. Stringifying each pipeline item keeps the text and drops the
# decoration (ErrorRecord.ToString() IS the original stderr line) and leaves
# $LASTEXITCODE intact. Verified against a cmd.exe wrapper; same result, and
# this form avoids cmd's quote-stripping rules for spaced paths.
& $runnerExe daemon --config $runnerCfg 2>&1 |
ForEach-Object { "$_" } |
Out-File -FilePath $daemonLog -Append -Encoding utf8
Log ("daemon exited, exit code {0}; restarting in 10s" -f $LASTEXITCODE)
Start-Sleep -Seconds 10