Compare commits

..

7 Commits

Author SHA1 Message Date
Xavier Roche
eee18cd2fc Name the gatherer, not a chain, and the source it holds
Review pass: the comment called the same mechanism by a second name and
described the binary package rather than the source that inherits the
removal clock.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-08-13 11:31:01 +02:00
Xavier Roche
47a688c861 webhttrack's browser dependency drags httrack toward autoremoval
The autoremoval gatherer follows only the first alternative of a
disjunction and never looks at Recommends. With "chromium | firefox-esr
| www-browser" in Depends, the httrack source therefore inherited
chromium's RC bug clock: #1128867 has 3.49.14-1 marked for removal from
testing on 9 September.

We already played this the other way round in #436, where the list led
with firefox-esr until its own RC bugs did the same thing. Leading with
any real browser just picks the next victim, so the disjunction moves to
Recommends instead, out of the gatherer's reach. Apt installs Recommends
by default, so nothing changes for a desktop install, and webhttrack
serves its UI over HTTP on localhost, so a headless machine browsed from
elsewhere needs no browser package at all.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-08-13 11:27:00 +02:00
Xavier Roche
1483289eec debian: record the 3.49.21-2 upload (#1234)
The revision carried the hppa FTBFS fix to the buildds as a quilt patch against
the frozen orig. The patch itself stays out of git, where the fix is already in
the source; only the changelog entry belongs here, so the file stays continuous
for the next upload.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 09:14:28 +00:00
Xavier Roche
1751c4b593 mkdeb.sh reaches debsign before it discovers the key is wrong (#1233)
The key is only resolved when debsign runs, which on a release is after the
tarball has been built. Ask gpg for it at startup instead, and name the
0x-prefixed fingerprint form in the error, since a short id is what people reach
for first and it is the ambiguous one.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 06:21:17 +00:00
Xavier Roche
bad28b629c A lost Windows runner leaves almost no telemetry behind (#1229)
The suite watchdog's commit statuses are the only evidence that survives a lost runner, and most of it was being thrown away: they went to the PR merge commit, whose SHA GitHub collects, leaving 3 of 146 recorded kills with any telemetry at all. They now go to the PR head, which costs us a line in the PR's checks list.

Each status carries more too. Loop lag and the number of posts that failed since one landed separate a box that stopped from a network that broke; `n=`/`f=`/`e=` come from one `GetTcpIPv4Statistics` call; `m=`/`c=`/`a=` are summed from the process array already enumerated for `p=` and `h=`. The cadence halves to 15s, and the in-flight test name is clipped to 30 characters to pay for the new fields. The widest real line measures 103 of the 140 characters GitHub keeps.

The rest closes two unsound spots in the Windows kill paths, neither of which explains the deaths. `stop_server` re-read `/proc/<pid>/winpid` after signalling its target, so the number it handed to `taskkill /F /T` could already belong to a stranger, and a hosted runner reissues a freed PID within a second rather than in theory. Nothing checked the image before firing either. Both the winpid and the image are now read while the target is alive and checked against `tasklist`, with a warning annotation when they disagree, which is also the meter for how often a tree kill would have left our own trees.

Refs #1228.
2026-08-13 08:08:03 +02:00
Xavier Roche
6584fded15 mkdeb.sh regenerates an orig that debian/patches cannot apply to (#1232)
A quilt patch is written against the tarball it will be applied to. Once the fix
it backports is upstream, HEAD carries it too, so an orig regenerated from HEAD
makes the patch fail or apply with fuzz. Require --orig whenever the series file
is non-empty, unsigned builds included: this one breaks the build, not policy.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 05:43:13 +00:00
Xavier Roche
d8f0cec17d Three suite tests fail on an emulated buildd for being slow, not broken (#1231)
* Three suite tests fail on an emulated buildd for being slow, not broken

hppa reported 3.49.21-1 as Build-Attempted with 105, 151 and 269 red, all three
on wall clock under qemu-user: the guard's own diagnostics dump took 31s of a
30s bound, one configure run passed a hard 300s cap, and the pairwise header
sweep outran the 600s harness budget mid-batch.

Each now measures the property instead of the host. 105 times the guard to its
DUMP announcement, leaving the dump (minutes, emulated) out of the bound. 151
watches configure's output for silence rather than capping its total, and skips
when the budget runs out while it is still making progress. The sweep runs in
slices so it can be given up on, paces itself against what 269 hands it, and 269
declares the larger budget its n^2 compiles need, so the emulated leg still runs
it to completion instead of pacing out.

The emulated leg is the CI counterpart of that buildd and was green on the same
commit: it prints test-suite.log now, so a test that skipped rather than ran
stops reading as coverage.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Fix the review's two skip-masking holes and the unbounded budget raise

A hung configure could reach the budget skip before the silence window closed, so
a wedge reported SKIP; the skip now needs a run that is still writing and enough
budget left for the detector to speak. 151 also paced inside run(), before
accept/reject read the verdict, so a configure that answered wrongly could exit 77
instead of 1: pacing moved to the callers, which judge first.

The per-test raise is bounded and normalized through one budget parser, since bash
test errors rather than compares past intmax and would have left the guard unarmed,
and a leading zero read as octal in arithmetic and decimal in test. Renamed
TEST_TIMEOUT_AT_LEAST so each use site carries the upwards-only rule, documented in
AGENTS.md, and the sweep takes an explicit --budget rather than sniffing the
environment, so the MSVC job cannot report a paced skip as a header break.

Tests for each: 151 drives run() through a configure_cmd seam with a child that
hangs and one that only crawls, 105 requires the DUMP announcement to be seen while
the guard runs (its fallback made the latency bound vacuous) and pins the hostile
budget values, and the sweep counts the units that reached the compiler.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Two of the new assertions passed a mutant; make them bite

Mutation-testing the added tests found two that could not see the bug they were
written for. The verdict check stubbed run() out, so a pacer left inside the real
one stayed invisible: it now drives the real run() through a child that answers
wrongly with the budget spent. The announcement check only asked that the marker
appear while the guard ran, which a driver announcing after the dump still
satisfies: a slow ps widens the dump, and the marker must now precede it.

Writing the first exposed a third: reject takes one argument fewer than accept, so
the extra one reached run() as an env assignment and the child failed to exec,
which made the probe pass on the wrong answer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* The verdict probe left the pacer disarmed

It numbered its run past the case count, so the pacer it exists to catch declined
to fire on a negative "steps left", and the mutant that puts pacing back inside
run() survived. Keep cases ahead of the run number.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* A killed configure left its child running and hung the macOS job

The macOS leg reported no test failure and then sat until its 20-minute step
timeout: bash 3.2 does not replace the subshell around the child with the child
itself, so killing that subshell left the run alive, and it outlived make check.
Give each run its own process group and kill the group.

The probe that exposed it now covers it: a child that spawns a child of its own,
and nothing of it left running afterwards.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 01:01:27 +02:00
19 changed files with 607 additions and 450 deletions

View File

@@ -270,8 +270,9 @@ jobs:
# Through the environment, never argv, which the process list exposes.
WATCHDOG_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WATCHDOG_REPO: ${{ github.repository }}
# github.sha here is the PR's merge commit, so statuses posted against it stay out of the PR's checks UI.
WATCHDOG_SHA: ${{ github.sha }}
# The PR head, not github.sha: a merge commit is garbage-collected, and
# these statuses are the only trace a lost runner leaves (#1228).
WATCHDOG_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
WATCHDOG_CONTEXT: windows-suite (${{ matrix.platform }}, ${{ matrix.configuration }})
WATCHDOG_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |

View File

@@ -1,364 +0,0 @@
# Scratch probe: does taskkill /F /T follow a recorded parent PID whose owner is
# long dead? Not part of the build gate; branch-scoped, delete when answered.
name: zz pid-reuse probe
on:
push:
branches: [zz-pidreuse-probe]
permissions:
contents: read
jobs:
probe:
runs-on: windows-2022
timeout-minutes: 15
steps:
- name: Probe
shell: pwsh
run: |
$ErrorActionPreference = 'Continue'
$ProgressPreference = 'SilentlyContinue'
$sum = $env:GITHUB_STEP_SUMMARY
function Say([string]$m) {
Write-Host $m
if ($sum) { try { Add-Content -LiteralPath $sum -Value $m -ErrorAction SilentlyContinue } catch { } }
}
Say "# PID-reuse probe on $env:COMPUTERNAME $(Get-Date -Format o)"
Say ""
Say '```'
# ---------------------------------------------------------------
# Part 1: is the hazard even present?
# ---------------------------------------------------------------
Say "== PART 1: recorded-parent state of the runner's own processes =="
$snap = @(Get-CimInstance Win32_Process)
$byPid = @{}
foreach ($p in $snap) { $byPid[[int]$p.ProcessId] = $p }
Say ("snapshot: {0} live processes" -f $snap.Count)
function Stamp($d) { if ($null -eq $d) { return '?' } else { return ([datetime]$d).ToString('HH:mm:ss.fff') } }
function Classify([int]$q) {
$c = $byPid[$q]
if ($null -eq $c) { return 'NOPROC' }
$par = $byPid[[int]$c.ParentProcessId]
if ($null -eq $par) { return 'DEAD-PARENT' }
if ($null -eq $par.CreationDate -or $null -eq $c.CreationDate) { return 'OK' }
if ($par.CreationDate -gt $c.CreationDate) { return 'RECYCLED-PARENT' }
return 'OK'
}
function Chain([int]$q) {
$out = @()
$seen = @{}
$cur = $q
for ($i = 0; $i -lt 24; $i++) {
$c = $byPid[$cur]
if ($null -eq $c) { $out += ("{0}=<no such process>" -f $cur); break }
$out += ("{0}/{1}@{2}" -f $c.ProcessId, $c.Name, (Stamp $c.CreationDate))
if ($seen.ContainsKey($cur)) { $out += '<cycle>'; break }
$seen[$cur] = $true
$st = Classify $cur
if ($st -ne 'OK') { $out += ("<<{0}>>" -f $st); break }
$cur = [int]$c.ParentProcessId
if ($cur -eq 0) { break }
}
return ($out -join ' <- ')
}
$interesting = @($snap | Where-Object {
$_.Name -like 'Runner.*' -or $_.Name -like 'Agent.*' -or $_.Name -like 'hosted-*' -or
$_.Name -eq 'bash.exe' -or $_.Name -eq 'pwsh.exe' -or
$_.Name -eq 'powershell.exe' -or $_.Name -eq 'dotnet.exe' -or
$_.Name -eq 'node.exe' -or $_.Name -eq 'cmd.exe' -or
$_.Name -eq 'conhost.exe' -or $_.Name -eq 'sh.exe'
} | Sort-Object Name, ProcessId)
$armed = 0
foreach ($p in $interesting) {
$st = Classify ([int]$p.ProcessId)
Say (" {0,-24} pid={1,-7} ppid={2,-7} {3,-16} {4}" -f $p.Name, $p.ProcessId, $p.ParentProcessId, $st, (Chain ([int]$p.ProcessId)))
if ($st -ne 'OK' -and ($p.Name -like 'Runner.*' -or $p.Name -like 'Agent.*' -or $p.Name -like 'hosted-*')) { $armed++ }
}
$dead = 0; $rec = 0; $ok = 0
$boxOrphan = @{}
foreach ($p in $snap) {
$st = Classify ([int]$p.ProcessId)
if ($st -eq 'DEAD-PARENT') {
$dead++
$k = [int]$p.ParentProcessId
if (-not $boxOrphan.ContainsKey($k)) { $boxOrphan[$k] = @() }
$boxOrphan[$k] += ("{0}/{1}" -f $p.ProcessId, $p.Name)
} elseif ($st -eq 'RECYCLED-PARENT') { $rec++ } else { $ok++ }
}
Say ""
Say ("box-wide: {0} processes, {1} with a DEAD recorded parent, {2} with a RECYCLED one, {3} sound" -f $snap.Count, $dead, $rec, $ok)
foreach ($k in ($boxOrphan.Keys | Sort-Object)) {
Say (" dead ppid {0,-7} still claimed as parent by: {1}" -f $k, ($boxOrphan[$k] -join ', '))
}
Say ""
if ($armed -gt 0) {
Say "PART 1 VERDICT: HAZARD PRESENT - $armed runner-agent process(es) record a parent PID that is dead or already recycled."
} elseif ($dead -gt 0) {
Say "PART 1 VERDICT: no Runner.*/Agent.* process has a stale recorded parent, but $dead other live processes do."
} else {
Say "PART 1 VERDICT: NOT ARMED - every live process's recorded parent is alive and older."
}
Say ""
# ---------------------------------------------------------------
# Part 2. Five sets of freed PIDs, released at the same instant, so
# that a null on the set the theory needs can be told apart from a
# probe that never recycles anything:
# A plain ping, killed, .NET handle disposed (baseline: does reuse happen?)
# C plain ping, killed, handle deliberately HELD (does a handle pin the PID?)
# D cmd orphan-maker, child killed (is the construction sound?)
# B cmd orphan-maker, ping child left ALIVE (the question)
# E cmd orphan-maker, wscript child left ALIVE (same, with a child that
# attaches to no console)
# Every process inherits this shell's console, so no console host survives
# to hold a handle on the dead cmd. Run 2 created 500 of those.
# ---------------------------------------------------------------
Say "== PART 2: can a dead parent's PID be handed out again, and does taskkill /T follow it? =="
$vbs = Join-Path $env:TEMP 'zzsleep.vbs'
Set-Content -LiteralPath $vbs -Value 'WScript.Sleep 600000' -Encoding ascii
function New-Ping {
$psi = New-Object System.Diagnostics.ProcessStartInfo
$psi.FileName = "$env:SystemRoot\System32\PING.EXE"
$psi.Arguments = '-n 600 127.0.0.1'
$psi.UseShellExecute = $false
$psi.CreateNoWindow = $false
$psi.RedirectStandardOutput = $true
$psi.RedirectStandardError = $true
return [System.Diagnostics.Process]::Start($psi)
}
function New-Orphan([string]$childArgs) {
# cmd exits at once, leaving the child alive with cmd's now-free PID recorded as parent.
$psi = New-Object System.Diagnostics.ProcessStartInfo
$psi.FileName = "$env:ComSpec"
$psi.Arguments = $childArgs
$psi.UseShellExecute = $false
$psi.CreateNoWindow = $false
return [System.Diagnostics.Process]::Start($psi)
}
function Get-Proc([int]$q) {
return (Get-CimInstance Win32_Process -Filter "ProcessId = $q" -ErrorAction SilentlyContinue | Select-Object -First 1)
}
# --- control: a tree kill of a LIVE parent must kill its child, or the probe is blind.
Say ""
Say "-- control: taskkill /F /T on a live cmd whose child ping is genuine"
$ctl = New-Orphan '/c ping -n 600 127.0.0.1 > NUL'
Start-Sleep -Milliseconds 1500
$ctlKid = @(Get-CimInstance Win32_Process -Filter "ParentProcessId = $($ctl.Id)")
if ($ctlKid.Count -eq 0) {
Say " control INCONCLUSIVE: no child found under cmd $($ctl.Id)"
try { $ctl.Kill() } catch { }
} else {
$kp = [int]$ctlKid[0].ProcessId
Say (" cmd {0} -> child {1}/{2}" -f $ctl.Id, $kp, $ctlKid[0].Name)
$co = (& taskkill.exe /F /T /PID $ctl.Id 2>&1 | Out-String).Trim()
Say (" taskkill: " + ($co -replace "`r?`n", ' | '))
Start-Sleep -Milliseconds 800
$still = Get-Proc $kp
Say (" control result: child {0} {1}" -f $kp, $(if ($null -eq $still) { 'GONE - the probe can observe a tree kill' } else { 'STILL ALIVE - probe is blind, part 2 is inconclusive' }))
}
try { $ctl.Dispose() } catch { }
$conBefore = @(Get-Process conhost -ErrorAction SilentlyContinue).Count
# --- build the sets.
$NB = 300; $ND = 200; $NE = 200; $NA = 200; $NC = 200
Say ""
Say "-- building sets: B=$NB (ping child alive) E=$NE (wscript child alive) D=$ND (child killed) A=$NA (baseline) C=$NC (handle held)"
$bCmd = New-Object 'System.Collections.Generic.List[System.Diagnostics.Process]'
for ($i = 0; $i -lt $NB; $i++) { try { $bCmd.Add((New-Orphan '/c start /b ping -n 600 127.0.0.1 > NUL')) } catch { } }
$eCmd = New-Object 'System.Collections.Generic.List[System.Diagnostics.Process]'
for ($i = 0; $i -lt $NE; $i++) { try { $eCmd.Add((New-Orphan ('/c start /b wscript.exe //B //Nologo "' + $vbs + '"'))) } catch { } }
$dCmd = New-Object 'System.Collections.Generic.List[System.Diagnostics.Process]'
for ($i = 0; $i -lt $ND; $i++) { try { $dCmd.Add((New-Orphan '/c start /b ping -n 600 127.0.0.1 > NUL')) } catch { } }
$aProc = New-Object 'System.Collections.Generic.List[System.Diagnostics.Process]'
for ($i = 0; $i -lt $NA; $i++) { try { $aProc.Add((New-Ping)) } catch { } }
$pinned = New-Object 'System.Collections.Generic.List[System.Diagnostics.Process]'
for ($i = 0; $i -lt $NC; $i++) { try { $pinned.Add((New-Ping)) } catch { } }
$bPid = New-Object 'System.Collections.Generic.HashSet[int]'
foreach ($p in $bCmd) { [void]$bPid.Add([int]$p.Id) }
$ePid = New-Object 'System.Collections.Generic.HashSet[int]'
foreach ($p in $eCmd) { [void]$ePid.Add([int]$p.Id) }
$dPid = New-Object 'System.Collections.Generic.HashSet[int]'
foreach ($p in $dCmd) { [void]$dPid.Add([int]$p.Id) }
$aPid = New-Object 'System.Collections.Generic.HashSet[int]'
foreach ($p in $aProc) { [void]$aPid.Add([int]$p.Id) }
$cPid = New-Object 'System.Collections.Generic.HashSet[int]'
foreach ($p in $pinned) { [void]$cPid.Add([int]$p.Id) }
Say (" distinct PIDs held: B={0} E={1} D={2} A={3} C={4}" -f $bPid.Count, $ePid.Count, $dPid.Count, $aPid.Count, $cPid.Count)
if ($bPid.Count -eq 0 -or $aPid.Count -eq 0) { Say " set construction failed, aborting"; Say '```'; exit 1 }
Start-Sleep -Seconds 3
# Map the surviving children back to the cmd that made them.
$orphan = @{}
$dKids = New-Object 'System.Collections.Generic.List[int]'
$nB = 0; $nE = 0
foreach ($p in @(Get-CimInstance Win32_Process | Where-Object { $_.Name -eq 'ping.exe' -or $_.Name -eq 'wscript.exe' })) {
$pp = [int]$p.ParentProcessId
if ($bPid.Contains($pp) -or $ePid.Contains($pp)) {
if (-not $orphan.ContainsKey($pp)) { $orphan[$pp] = @() }
$orphan[$pp] += [pscustomobject]@{ Pid = [int]$p.ProcessId; Created = $p.CreationDate }
if ($bPid.Contains($pp)) { $nB++ } else { $nE++ }
} elseif ($dPid.Contains($pp)) {
$dKids.Add([int]$p.ProcessId)
}
}
Say (" surviving orphan children: {0} under B parents, {1} under E parents, {2} under D parents (to be killed)" -f $nB, $nE, $dKids.Count)
# Free every set at the same instant. Only C keeps its .NET handle open.
foreach ($p in $bCmd) { try { [void]$p.WaitForExit(2000) } catch { }; try { $p.Dispose() } catch { } }
foreach ($p in $eCmd) { try { [void]$p.WaitForExit(2000) } catch { }; try { $p.Dispose() } catch { } }
foreach ($p in $dCmd) { try { [void]$p.WaitForExit(2000) } catch { }; try { $p.Dispose() } catch { } }
foreach ($p in $aProc) { try { $p.Kill() } catch { } }
foreach ($p in $pinned) { try { $p.Kill() } catch { } }
Start-Sleep -Milliseconds 1000
foreach ($p in $aProc) { try { $p.Dispose() } catch { } }
$aProc.Clear(); $bCmd.Clear(); $eCmd.Clear(); $dCmd.Clear()
if ($dKids.Count -gt 0) { Stop-Process -Id $dKids.ToArray() -Force -ErrorAction SilentlyContinue }
[GC]::Collect(); [GC]::WaitForPendingFinalizers(); [GC]::Collect()
Start-Sleep -Seconds 2
$liveNow = New-Object 'System.Collections.Generic.HashSet[int]'
foreach ($p in @(Get-CimInstance Win32_Process)) { [void]$liveNow.Add([int]$p.ProcessId) }
function CountLive($s) { $n = 0; foreach ($x in $s) { if ($liveNow.Contains($x)) { $n++ } }; return $n }
Say (" set members still live (must be 0): B={0} E={1} D={2} A={3} C={4}" -f (CountLive $bPid), (CountLive $ePid), (CountLive $dPid), (CountLive $aPid), (CountLive $cPid))
$conAfter = @(Get-Process conhost -ErrorAction SilentlyContinue).Count
Say (" conhost.exe: {0} before, {1} after (a surviving console host would hold a handle on its dead creator and pin its PID)" -f $conBefore, $conAfter)
$kidsAlive = 0
foreach ($k in $orphan.Keys) { foreach ($o in $orphan[$k]) { if ($liveNow.Contains($o.Pid)) { $kidsAlive++ } } }
Say (" orphan children still alive under a dead B/E parent: {0}" -f $kidsAlive)
# --- hunt.
$hitA = New-Object 'System.Collections.Generic.HashSet[int]'
$hitB = New-Object 'System.Collections.Generic.HashSet[int]'
$hitC = New-Object 'System.Collections.Generic.HashSet[int]'
$hitD = New-Object 'System.Collections.Generic.HashSet[int]'
$hitE = New-Object 'System.Collections.Generic.HashSet[int]'
$boxHits = 0
$spawns = 0; $tested = 0; $followed = 0; $survived = 0
$KILLCAP = 25
$distinct = New-Object 'System.Collections.Generic.HashSet[int]'
$window = New-Object 'System.Collections.Generic.Queue[System.Diagnostics.Process]'
$minPid = [int]::MaxValue; $maxPid = 0
function Test-Hit([int]$np) {
if ($script:aPid.Contains($np)) { [void]$script:hitA.Add($np) }
if ($script:dPid.Contains($np)) { [void]$script:hitD.Add($np) }
if ($script:cPid.Contains($np)) {
if ($script:hitC.Add($np) -and $script:hitC.Count -le 5) { Say ("!! set C hit: PID $np was handed out although an open process handle on it is still held") }
}
if ($script:boxOrphan.ContainsKey($np)) {
$script:boxHits++
Say ""
Say ("!! REAL COLLISION: our spawn got PID $np, which live box process(es) still record as their parent: $($script:boxOrphan[$np] -join ', ')")
Say " (not tree-killing this one: it would reap a process we did not start)"
Say ""
}
$isB = $script:bPid.Contains($np)
$isE = $script:ePid.Contains($np)
if (-not ($isB -or $isE)) { return $false }
if ($isB) { [void]$script:hitB.Add($np) } else { [void]$script:hitE.Add($np) }
if ($script:tested -ge $script:KILLCAP) { return $false }
$live = @()
foreach ($o in $script:orphan[$np]) {
$cur = Get-Proc $o.Pid
if ($null -ne $cur -and $cur.CreationDate -eq $o.Created) { $live += $o }
}
if ($live.Count -eq 0) { return $false }
$script:tested++
Say ""
Say ("### HIT #$($script:tested) on set $(if ($isB) { 'B' } else { 'E' }): spawn #$($script:spawns) at t=$([int]$script:sw.Elapsed.TotalSeconds)s was handed PID $np, the PID of a dead parent whose child is still running")
foreach ($k in @(Get-CimInstance Win32_Process -Filter "ParentProcessId = $np")) {
Say (" the process table calls this a child of $np : {0}/{1} created {2}" -f $k.ProcessId, $k.Name, $k.CreationDate)
}
foreach ($o in $live) { Say (" stale-PPID orphan {0} is alive before the kill" -f $o.Pid) }
Say (" issuing: taskkill /F /T /PID $np")
$out = (& taskkill.exe /F /T /PID $np 2>&1 | Out-String).Trim()
Say (" taskkill said: " + ($out -replace "`r?`n", ' | '))
Start-Sleep -Milliseconds 1000
$anyDied = $false
foreach ($o in $live) {
$cur = Get-Proc $o.Pid
$alive = ($null -ne $cur -and $cur.CreationDate -eq $o.Created)
if (-not $alive) { $anyDied = $true }
Say (" stale-PPID orphan {0}: alive after the kill = {1}" -f $o.Pid, $alive)
}
if (-not $anyDied) { $script:survived++ }
if ($anyDied) {
$script:followed++
Say " >>> THE TREE WALK FOLLOWED A STALE PPID. taskkill /T does not validate that the recorded parent is the real one."
} else {
Say " >>> the orphan survived: taskkill /T did not follow the stale PPID."
}
Say ""
return $false
}
$sw = [Diagnostics.Stopwatch]::StartNew()
Say ""
Say "-- hunting: rolling window of 200 live spawns, 300s cap"
$done = $false
while (-not $done -and $sw.Elapsed.TotalSeconds -lt 300) {
try { $np = (New-Ping) } catch { Start-Sleep -Milliseconds 50; continue }
$window.Enqueue($np)
$spawns++
$id = [int]$np.Id
[void]$distinct.Add($id)
if ($id -lt $minPid) { $minPid = $id }
if ($id -gt $maxPid) { $maxPid = $id }
try { $done = Test-Hit $id } catch { Say ("Test-Hit failed on $id : " + $_.Exception.Message) }
while ($window.Count -gt 200) {
$old = $window.Dequeue()
try { $old.Kill() } catch { }
try { $old.Dispose() } catch { }
}
if (($spawns % 250) -eq 0) { Write-Host (" ... spawn $spawns, distinct $($distinct.Count), A=$($hitA.Count) B=$($hitB.Count) C=$($hitC.Count) D=$($hitD.Count) E=$($hitE.Count), t=$([int]$sw.Elapsed.TotalSeconds)s") }
}
Say ""
Say "== RESULT =="
Say ("spawns {0}, distinct PIDs {1}, range {2}..{3}, {4:N0}s" -f $spawns, $distinct.Count, $minPid, $maxPid, $sw.Elapsed.TotalSeconds)
Say ("distinct set PIDs reissued, by set:")
Say (" A killed, handle disposed, no child : {0} of {1}" -f $hitA.Count, $aPid.Count)
Say (" C killed, handle still OPEN : {0} of {1}" -f $hitC.Count, $cPid.Count)
Say (" D dead orphan-maker, child killed : {0} of {1}" -f $hitD.Count, $dPid.Count)
Say (" B dead orphan-maker, ping child ALIVE : {0} of {1}" -f $hitB.Count, $bPid.Count)
Say (" E dead orphan-maker, wscript ALIVE : {0} of {1}" -f $hitE.Count, $ePid.Count)
Say ("tree kills issued on a reissued stale-parent PID: {0}; reached the orphan: {1}; orphan survived: {2}" -f $tested, $followed, $survived)
Say ("collisions with a dead PPID recorded by a process we did NOT start: {0}" -f $boxHits)
if ($tested -gt 0 -and $followed -eq 0) {
Say "INTERPRETATION: a dead parent's PID IS reissued while a live process still records it as its parent, in $tested measured cases, and taskkill /F /T left that process alone every time. The tree walk does not follow a stale PPID: it skips a claimed child that predates the process now holding the PID."
} elseif ($hitA.Count -eq 0 -and $hitB.Count -eq 0 -and $hitE.Count -eq 0) {
Say "INTERPRETATION: nothing came back, so this run recycled nothing and says nothing either way."
} elseif ($tested -eq 0) {
Say "INTERPRETATION: PIDs recycle (A>0) but no PID with a live process recording it as parent ever came back. On this kernel such a PID stays reserved, so a tree kill cannot land on one and the stale-PPID mechanism cannot fire."
} elseif ($followed -gt 0) {
Say "INTERPRETATION: a dead parent's PID IS reissued while a live process records it, and taskkill /F /T reaped that process. The mechanism is real on this image."
} else {
Say "INTERPRETATION: the PID was reissued, but taskkill /F /T left the stale-PPID orphan alone: the tree walk validates parentage."
}
Say '```'
while ($window.Count -gt 0) { $o = $window.Dequeue(); try { $o.Kill() } catch { } }
try { Get-Process ping, wscript -ErrorAction SilentlyContinue | Stop-Process -Force -ErrorAction SilentlyContinue } catch { }
Write-Host "probe done"

View File

@@ -30,6 +30,11 @@ the operational checklist: toolchain, invariants, and how to ship a change.
check`, or `PATH="<bld>/src:$PATH"` for a manual run.
- Give new `.test` scripts `set -e`: the older ones predate the rule, so several
`local-crawl.sh` calls with no `set -e` report PASS on any non-last failure.
- Each test runs under a 600s wall-clock guard that reports a wedge as 124. A test
whose own work outlasts it raises the budget with a `# TEST_TIMEOUT_AT_LEAST: N`
line, at column 0 within its first 40 lines, and paces itself with
`skip_if_out_of_budget` so a host too slow to finish skips instead. The value only
ever raises the budget: nothing can disarm the guard.
- Run teardown with errexit off: `trap 'set +e; cleanup' EXIT`. Under `set -e` a
failing cleanup command becomes the test's exit status (#773). Keep the other
signals on their own `trap` line, or errexit stays off for the rest of the run.

10
debian/changelog vendored
View File

@@ -1,3 +1,13 @@
httrack (3.49.21-2) unstable; urgency=medium
* Fix the FTBFS on hppa: three suite tests measured the build host rather
than the property they cover, and failed on the qemu-user buildd purely
for being slower there. Patched from upstream, which now skips a step a
host is too slow to finish instead of failing the build
(skip-emulated-host-test-failures.patch).
-- Xavier Roche <xavier@debian.org> Thu, 13 Aug 2026 07:26:38 +0200
httrack (3.49.21-1) unstable; urgency=medium
* New upstream release: a site answering to several hostnames can now be

5
debian/control vendored
View File

@@ -29,7 +29,10 @@ Description: Copy websites to your computer (Offline browser)
Package: webhttrack
Architecture: any
Multi-Arch: foreign
Depends: ${misc:Depends}, ${shlibs:Depends}, webhttrack-common, sensible-utils, chromium | firefox-esr | www-browser
Depends: ${misc:Depends}, ${shlibs:Depends}, webhttrack-common, sensible-utils
# Recommends, not Depends: the autoremoval gatherer follows only a disjunction's
# first alternative, which ties the httrack source to whichever browser leads it.
Recommends: firefox-esr | chromium | www-browser
Replaces: webhttrack-common (<< 3.43.9-2)
Breaks: webhttrack-common (<< 3.43.9-2)
Suggests: httrack, httrack-doc

View File

@@ -36,11 +36,43 @@ echo "marker: the wedged test started"
sleep 300
EOF
start=$SECONDS
# Time the guard to its DUMP announcement, not to the driver's exit: the dump that
# follows runs for minutes on an emulated host, and that is not what is under test.
rc=0
HTTRACK_PROGRESS_LOG="$tmp/progress" HTTRACK_TEST_TIMEOUT=5 \
bash "$driver" "$tmp/90_wedged.test" >"$out" 2>&1 || rc=$?
elapsed=$((SECONDS - start))
fired=
marked=
total=
run_wedged() { # run_wedged <budget> [VAR=VAL...]
local budget=$1 start=$SECONDS pid
shift
: >"$tmp/progress"
rc=0
fired=
marked=
env "$@" HTTRACK_PROGRESS_LOG="$tmp/progress" HTTRACK_TEST_TIMEOUT="$budget" \
bash "$driver" "$tmp/90_wedged.test" >"$out" 2>&1 &
pid=$!
while kill -0 "$pid" 2>/dev/null; do
# Read with the shell: a fork per poll would blur the latency being measured.
if read -r line <"$tmp/progress" 2>/dev/null && test "$line" = "DUMP 90_wedged.test"; then
fired=$((SECONDS - start))
marked=1
break
fi
poll_wait 0.1 || sleep 1
done
wait "$pid" || rc=$?
total=$((SECONDS - start))
# Missed between two polls: the whole run then bounds the latency from above.
test -n "$fired" || fired=$total
}
# Retried once, because a dump short enough to fall between two polls is a race, not a
# regression; announcing after the dump loses both attempts.
run_wedged 5
test -n "$marked" || run_wedged 5
test -n "$marked" ||
fail "the announcement was never seen while the guard ran, so its latency is unknown"
# Announced when the dump starts, which runs for minutes: a suite watchdog
# reading that log would take the silence for a wedge and kill the step.
@@ -49,8 +81,8 @@ grep -qx 'DUMP 90_wedged.test' "$tmp/progress" ||
test "$rc" -eq 124 || fail "wedged test reported $rc, want 124"
# Never before the budget, or a slow-but-healthy test would be killed too.
test "$elapsed" -ge 5 || fail "the guard fired early (${elapsed}s of a 5s budget)"
test "$elapsed" -lt 30 || fail "the guard fired late (${elapsed}s)"
test "$fired" -ge 5 || fail "the guard fired early (${fired}s of a 5s budget)"
test "$fired" -lt 30 || fail "the guard fired late (${fired}s)"
grep -q 'marker: the wedged test started' "$out" || fail "the test's own output was lost"
# The header, not a bare name: the process list quotes the test's path too, so a
# wrapper that named nothing would still match that.
@@ -58,6 +90,20 @@ grep -q '^===== TIMEOUT: 90_wedged.test exceeded' "$out" ||
fail "the diagnostics do not name the test"
grep -q "own process tree" "$out" || fail "no process list in the diagnostics"
# Announced BEFORE the dump, not merely at some point during it: the watchdog reading
# that log takes the silence of a dump for a wedge. Only a slow dump tells the two
# orderings apart, and the dump's own ps is what this makes slow.
if ! is_windows; then
printf '#!/bin/sh\nsleep 3\nexec %s "$@"\n' "$(command -v ps)" >"$shim/ps"
chmod +x "$shim/ps"
run_wedged 5 "PATH=$shim:$PATH"
rm -f "$shim/ps" # before the starve shim shares this directory
test "$rc" -eq 124 || fail "the guard reported $rc under a slow dump, want 124"
test -n "$marked" || fail "no announcement under a slow dump"
test "$((total - fired))" -ge 2 ||
fail "announced with the dump (${fired}s of ${total}s), not before it"
fi
# The budget is read, not hard-coded: well under it, the same shape survives.
printf 'sleep 3\necho "slow but healthy"\n' >"$tmp/92_slow.test"
rc=0
@@ -78,15 +124,11 @@ sleep 1
# the loop it stretches is the same one poll_wait's fd tick drives.
(
starve_sleep "$shim" 4 || fail "could not install the slow sleep"
start=$SECONDS
rc=0
HTTRACK_POLL_SLEEP=1 HTTRACK_TEST_TIMEOUT=1 \
bash "$driver" "$tmp/90_wedged.test" >"$out" 2>&1 || rc=$?
elapsed=$((SECONDS - start))
run_wedged 1 HTTRACK_POLL_SLEEP=1
test "$rc" -eq 124 || fail "starved guard reported $rc, want 124"
# Generous: the diagnostics dump runs inside this window too.
test "$elapsed" -lt 25 ||
fail "budget counted polls, not seconds: ${elapsed}s for a 1s budget"
# 10 stretched polls would be 40s; a handful of them is the whole margin here.
test "$fired" -lt 25 ||
fail "budget counted polls, not seconds: ${fired}s for a 1s budget"
)
# --- exit status and output of a healthy test pass straight through ----------
@@ -127,6 +169,84 @@ saw_budget 45 "an explicit budget"
HTTRACK_TEST_TIMEOUT=0 bash "$driver" "$tmp/95_budget.test" >"$out" 2>&1
saw_budget 0 "a disabled guard"
# --- a test may raise the budget, never lower it ----------------------------
# 269's header sweep is n^2 compiles, real work that outlasts the wedge budget on an
# emulated host; anything else asking would be disarming the guard.
raiser() { # raiser <asked for>
# shellcheck disable=SC2016 # the fixture has to read the variable, not us
printf '# TEST_TIMEOUT_AT_LEAST: %s\necho "budget=${HTTRACK_TEST_TIMEOUT-unset}"\n' \
"$1" >"$tmp/97_raise.test"
}
# 0900 must not read as octal, and a value past intmax must not reach `test`, which
# errors on it rather than comparing and would leave the guard unarmed.
for want in 900:900 5:600 garbage:600 0900:900 99999999999999999999:600 ' 900':600; do
raiser "${want%%:*}"
HTTRACK_TEST_TIMEOUT=600 bash "$driver" "$tmp/97_raise.test" >"$out" 2>&1
saw_budget "${want##*:}" "a test asking for '${want%%:*}'"
done
raiser 900
HTTRACK_TEST_TIMEOUT=0 bash "$driver" "$tmp/97_raise.test" >"$out" 2>&1
saw_budget 0 "a raise under a disabled guard"
# Read from the header only, or a test's own data would be one: 151 writes this line.
window() { # window <lines of padding> <budget wanted>
{
i=0
while test "$i" -lt "$1"; do
i=$((i + 1))
echo "# pad $i"
done
cat "$tmp/97_raise.test"
} >"$tmp/97_deep.test"
HTTRACK_TEST_TIMEOUT=600 bash "$driver" "$tmp/97_deep.test" >"$out" 2>&1
saw_budget "$2" "a raise on line $(($1 + 1))"
}
raiser 900
window 39 900 # the last line the header reaches
window 40 600
# Enforced, not merely exported: the number the guard uses is the one it kills on.
printf '# TEST_TIMEOUT_AT_LEAST: 900\nsleep 4\necho "outlived the default"\n' \
>"$tmp/97_raise.test"
rc=0
HTTRACK_TEST_TIMEOUT=2 bash "$driver" "$tmp/97_raise.test" >"$out" 2>&1 || rc=$?
test "$rc" -eq 0 || fail "a 4s test that raised the budget to 900 reported $rc"
grep -q 'outlived the default' "$out" || fail "the raised budget killed the test anyway"
printf '# TEST_TIMEOUT_AT_LEAST: 1\nsleep 3\necho "not shrunk"\n' >"$tmp/97_raise.test"
rc=0
HTTRACK_TEST_TIMEOUT=600 bash "$driver" "$tmp/97_raise.test" >"$out" 2>&1 || rc=$?
test "$rc" -eq 0 || fail "a 3s test asking for a 1s budget reported $rc"
grep -q 'not shrunk' "$out" || fail "the header shrank the budget and killed the test"
# --- one budget parser, and what is left of it ------------------------------
secs() { # secs <value in the environment> <seconds it must read as>
local got
got=$(HTTRACK_TEST_TIMEOUT=$1 bash -c '. "$1"; budget_secs' _ "${testdir}/testlib.sh" 2>&1)
test "$got" = "$2" || fail "budget_secs read '$1' as '$got', want $2"
}
secs 45 45
secs 0900 900 # decimal, or $((...)) and test disagree on the same string
secs garbage 600
secs 99999999999999999999 600 # past intmax, where test errors instead of comparing
secs 0 0
# budget_left hands a child what is left of it, and keeps 0 meaning "no guard".
# shellcheck disable=SC2016 # the fixture has to call the helper, not us
printf '. "%s"\nsleep 2\necho "left=$(budget_left) at=$SECONDS"\n' "${testdir}/testlib.sh" \
>"$tmp/98_left.test"
# Exact, against the clock the child itself read: a tolerance would pass a wrong epoch.
HTTRACK_TEST_TIMEOUT=60 bash "$driver" "$tmp/98_left.test" >"$out" 2>&1
got=$(sed -n 's/^left=\([0-9][0-9]*\) .*/\1/p' "$out")
at=$(sed -n 's/^left=[0-9][0-9]* at=\([0-9][0-9]*\)$/\1/p' "$out")
case "$got$at" in '' | *[!0-9]*) fail "a 60s budget printed '$(cat "$out")'" ;; esac
test "$got" -eq "$((60 - at))" ||
fail "a 60s budget left $got with $at gone, want $((60 - at))"
HTTRACK_TEST_TIMEOUT=0 bash "$driver" "$tmp/98_left.test" >"$out" 2>&1
grep -q '^left=0 ' "$out" || fail "a disabled guard left '$(cat "$out")', want 0"
# Never 0 on an exhausted budget: a child would read that as the guard being off.
HTTRACK_TEST_TIMEOUT=1 bash -c '. "$1"; sleep 2; echo "left=$(budget_left)"' \
_ "${testdir}/testlib.sh" >"$out" 2>&1
grep -qx 'left=1' "$out" || fail "an exhausted budget left '$(cat "$out")', want 1"
# --- a test too slow to finish skips instead of being killed ----------------
# hppa spends ~150s on one configure run, and 124 takes the build down where 77
# does not.

View File

@@ -8,6 +8,8 @@ set -euo pipefail
# shellcheck source=tests/testlib.sh
. "$(dirname "$0")/testlib.sh"
# shellcheck source=tests/proclib.sh
. "$(dirname "$0")/proclib.sh"
sh=${BASH_SHELL:-}
test -n "$sh" || {
@@ -51,11 +53,21 @@ chmod 755 "$tmp/fakebin/bash"
mkfifo "$tmp/fifo"
chmod 755 "$tmp/fifo"
# Sampled rather than polled per second: the size read is a fork, and an emulated
# host pays for it. SILENCE clears the slowest single configure probe there.
SAMPLE=5
SILENCE=${HTTRACK_CONFIGURE_SILENCE:-120}
RESERVE=15 # what killing the run and skipping still needs of the budget
n=0
cases=16 # reject/accept calls below; pinned again once they have all run
status=0
log=
rundir=
took=0
# What run() launches, so the checks below can hand it a child that hangs or one that
# only crawls; nothing else may override it.
configure_cmd=(bash "$tmp/src/configure" --disable-https)
run() { # run <label> <env argument>...
local label=$1 began=$SECONDS
shift
@@ -65,23 +77,56 @@ run() { # run <label> <env argument>...
status=0
# Capped: configure executes the candidate, and a hang wedges "make check" with no output
# at all. Polled, not a backgrounded "sleep" watchdog, which outlives the run it guards.
(cd "$rundir" && env "$@" bash "$tmp/src/configure" --disable-https) \
local had_m=
case "$-" in *m*) had_m=1 ;; esac
# Own process group, so the kills below reach what configure spawned: bash 3.2 keeps
# the subshell it runs in, and killing that alone leaves the child running (macOS).
set -m
(cd "$rundir" && env "$@" "${configure_cmd[@]}") \
>"$rundir/log" 2>&1 &
local pid=$! waited=0
while test "$waited" -lt 300 && kill -0 "$pid" 2>/dev/null; do
local pid=$! waited=0 quiet=0 size=0 now left
test -n "$had_m" || set +m
# A hang is silence, not slowness: configure writes a line per probe,
# but hppa's emulated run can take longer overall than a runner's whole budget (#1146).
while kill -0 "$pid" 2>/dev/null; do
sleep 1
waited=$((waited + 1))
test "$((waited % SAMPLE))" -eq 0 || continue
now=$(wc -c <"$rundir/log")
if test "$now" -gt "$size"; then
size=$now
quiet=0
else
quiet=$((quiet + SAMPLE))
fi
test "$quiet" -lt "$SILENCE" || {
kill_tree "$pid"
echo "configure wrote nothing for ${quiet}s of ${waited}s for $label" >&2
tail -5 "$rundir/log" >&2
exit 1
}
# Still writing but out of time: skip, where the harness would kill the whole test
# and take the build down with it. Only while writing, and only with a full silence
# window still affordable, or a hang would reach this before the check above fires
# and a wedge would report a skip. 0 is the guard off.
left=$(budget_left)
if test "$quiet" -eq 0 && test "$left" -ne 0 &&
test "$left" -le "$((RESERVE + SILENCE))"; then
kill_tree "$pid"
echo "$label was still configuring ${waited}s in and the budget is out; skipping" >&2
exit 77
fi
done
if kill -0 "$pid" 2>/dev/null; then
kill -9 "$pid" 2>/dev/null
echo "configure did not return within ${waited}s for $label" >&2
tail -5 "$rundir/log" >&2
exit 1
fi
wait "$pid" || status=$?
log=$(cat "$rundir/log")
took=$((SECONDS - began))
echo "run $n ($label): exit $status"
skip_if_out_of_budget "$((cases - n))" "$((SECONDS - began))"
}
# Pace here rather than in run(), which returns with the answer still unjudged: a
# skip between the two would bury a configure that answered wrongly.
paced() {
skip_if_out_of_budget "$((cases - n))" "$took"
}
reject() { # reject <label> <expected message> <env argument>...
@@ -97,6 +142,7 @@ reject() { # reject <label> <expected message> <env argument>...
tail -5 <<<"$log" >&2
exit 1
}
paced
}
# accept <label> <expected $(BASH_SHELL), "" for any> <expected message, "" for none> <env argument>...
@@ -125,8 +171,69 @@ accept() {
exit 1
}
fi
paced
}
# --- what run() does to a child that hangs, and to one that is merely slow -------
# Driven through configure_cmd, since the real configure can do neither on demand.
probe() { # probe <run number> <seconds of budget left> <command>...
local want_n=$1 left=$2 rc=0
shift 2
(
# shellcheck disable=SC2030 # the isolation is the point: the real count is next door
n=$want_n SAMPLE=1 SILENCE=2
# shellcheck disable=SC2030,SC2031 # likewise: the budget here is the probe's own
export HTTRACK_TEST_TIMEOUT=$((SECONDS + left))
configure_cmd=("$@")
run probe
) >"$tmp/probe.log" 2>&1 || rc=$?
echo "$rc"
}
# A wedge must fail even with the budget gone, or #922 comes back as a skip.
rc=$(probe 90 6 sleep 999)
test "$rc" -eq 1 || fail "a silent configure with 6s of budget reported $rc, want 1"
grep -q 'wrote nothing' "$tmp/probe.log" || fail "the hang was not named: $(cat "$tmp/probe.log")"
# Slow but talking is the emulated buildd, and a skip there beats the harness kill.
rc=$(probe 91 6 bash -c 'while :; do echo tick; sleep 1; done')
test "$rc" -eq 77 || fail "a slow but writing configure with 6s of budget reported $rc, want 77"
# The kill has to reach what configure spawned. bash 3.2 keeps the subshell around the
# child, so killing that alone leaves a live configure behind: it outlives "make check"
# and holds the CI step open to its own timeout, with the suite reporting no failure.
rc=$(probe 92 6 bash -c 'sleep 987 & wait')
test "$rc" -eq 1 || fail "a silent configure with a child of its own reported $rc, want 1"
sleep 1
! ps_snapshot | grep -q '[s]leep 987' || fail "the killed run left its child running"
# The pacer must not fire before the case is judged: run() returns with the verdict
# still unread, and a skip there would bury a configure that answered wrongly. Through
# the real run(), since a stub cannot see a pacer left inside the one it replaced.
verdict() { # verdict <accept|reject> <run number> <status the child exits with>
local rc=0
(
# shellcheck disable=SC2030,SC2031 # the isolation is the point: the real run is next door
# Cases still to come, or the pacer this is looking for would decline to fire.
n=$2 cases=$(($2 + 5)) SAMPLE=1
# Spent by the time the run ends, so a pacer anywhere after it would fire.
# shellcheck disable=SC2030,SC2031 # likewise: the budget here is the probe's own
export HTTRACK_TEST_TIMEOUT=$((SECONDS + 4))
configure_cmd=(bash -c "sleep 3; exit $3")
# Their arities differ, and an extra argument would reach run() as an env
# assignment: the child would then fail to exec and answer the wrong question.
case "$1" in
accept) accept probe-verdict '' '' ;;
*) reject probe-verdict '' ;;
esac
) >/dev/null 2>&1 || rc=$?
test "$rc" -eq 1 || fail "$1 of a wrong answer with the budget spent reported $rc, want 1"
}
verdict accept 80 1 # configure rejected what it must accept
verdict reject 81 0 # configure accepted what it must reject
# The probes ran in subshells, so the real cases below start from a clean count.
cases=16
n=0
status=0
log=
took=0
# The four that configure to completion run first. A reject stops at the
# BASH_SHELL check and costs a fraction of one, and the pacer projects the step it
# just timed: behind the cheap ones it read far too low and 151 met the harness

View File

@@ -188,20 +188,29 @@ rc=0
kill_pid() { echo "DIRECT $1" >>"$rec"; }
# shellcheck disable=SC2317
kill_tree() {
echo "TREE $1" >>"$rec"
echo "TREE $*" >>"$rec"
exit 9
}
# The suite's own pid has no /proc entry here, so the capture answers what a
# POSIX box answers: empty, and the kill goes on unguarded.
# shellcheck disable=SC2317
win_capture() {
echo "CAPTURE $1" >>"$rec"
WIN_PID=4242 WIN_IMAGE=bash.exe
}
hb_depth=$BASH_SUBSHELL
ci_suite_heartbeat 960 360 "$progress" 900 4242 >"$tmp/hedge" 2>&1
) || rc=$?
test "$rc" -eq 9 || fail "the tree kill never fired: watchdog returned $rc"
test "$(sed -n 1p "$rec")" = "DIRECT 777" ||
fail "the reporter was not killed ahead of the suite: $(tr '\n' '/' <"$rec")"
test "$(sed -n 2p "$rec")" = "DIRECT 4242" ||
test "$(sed -n 2p "$rec")" = "CAPTURE 4242" ||
fail "the winpid was not read before the target was signalled: $(tr '\n' '/' <"$rec")"
test "$(sed -n 3p "$rec")" = "DIRECT 4242" ||
fail "the target was not signalled directly ahead of the tree walk: $(tr '\n' '/' <"$rec")"
test "$(sed -n 3p "$rec")" = "TREE 4242" ||
fail "the tree was not killed after the direct signal: $(tr '\n' '/' <"$rec")"
test "$(sed -n '$=' "$rec")" -eq 3 || fail "extra kills: $(tr '\n' '/' <"$rec")"
test "$(sed -n 4p "$rec")" = "TREE 4242 4242 bash.exe" ||
fail "the tree kill did not carry what was captured: $(tr '\n' '/' <"$rec")"
test "$(sed -n '$=' "$rec")" -eq 4 || fail "extra kills: $(tr '\n' '/' <"$rec")"
test ! -e "$tmp/forked" || fail "the clock was read through a subshell, a fork a starved box cannot spare"

View File

@@ -214,8 +214,9 @@ def perms_of(wf, job):
WANT_ENV = {
"WATCHDOG_TOKEN": "${{ secrets.GITHUB_TOKEN }}",
"WATCHDOG_REPO": "${{ github.repository }}",
# The merge commit, which no PR checks UI reads.
"WATCHDOG_SHA": "${{ github.sha }}",
# The PR head: statuses on the merge commit are GC'd, and they are the only
# trace a lost runner leaves (#1228).
"WATCHDOG_SHA": "${{ github.event.pull_request.head.sha || github.sha }}",
}
def audit(wf):
@@ -264,7 +265,7 @@ def mutate(wf, kind):
elif kind == "token":
suite_steps(wf)[0]["env"]["WATCHDOG_TOKEN"] = "${{ secrets.WATCHDOG_PAT }}"
elif kind == "sha":
suite_steps(wf)[0]["env"]["WATCHDOG_SHA"] = "${{ github.event.pull_request.head.sha }}"
suite_steps(wf)[0]["env"]["WATCHDOG_SHA"] = "${{ github.sha }}"
elif kind == "context":
suite_steps(wf)[0]["env"]["WATCHDOG_CONTEXT"] = "windows-suite"
elif kind == "url":
@@ -555,6 +556,12 @@ backoff_leg() {
echo "$calls calls and $lines log lines against an API rejecting every one: nothing backs off"
return 1
fi
# Every attempt here is refused, so each one after the first must say how many
# went missing: x= is what separates a stopped box from a network that healed.
grep -q ' x=[1-9]' "$posts" || {
echo "no posted status counted the failures before it: $(cat "$posts")"
return 1
}
}
fullstdout_leg() {
@@ -609,9 +616,18 @@ for psrun in "${psruns[@]}"; do
# shellcheck disable=SC2016
mutate tail-always-ok 's/\$r = @{ Ok = \$false;/$r = @{ Ok = $true;/' 'reads as one that was read'
mutate counters-empty "s/return (\$f -join ' ')/return ''/" 'counters are not key=value'
# shellcheck disable=SC2016
mutate lag-not-measured "s/'l={0}' -f \$LagMs/'l={0}' -f 0/" 'loop lag is not what the caller measured'
# shellcheck disable=SC2016
mutate failed-not-reported "s/'x={0}' -f \$Failed/'x={0}' -f 0/" 'failed-post count is not what the caller'
# shellcheck disable=SC2016
mutate tcp-total-not-delta 's/(\$Cur\[0\] - \$Prev\[0\])/($Cur[0])/' 'total was reported where the delta'
# shellcheck disable=SC2016
mutate lag-not-a-peak 's/if (\$lag -gt \$Peak) { return \$lag }/if ($false) { return $lag }/' \
'overshot by 200ms'
# The production cadence: no leg below runs without a schedule of its own.
# shellcheck disable=SC2016
mutate default-cadence 's/\[int\]\$IntervalSeconds = 30/[int]$IntervalSeconds = 3000/' \
mutate default-cadence 's/\[int\]\$IntervalSeconds = 15/[int]$IntervalSeconds = 1500/' \
'default status cadence'
# shellcheck disable=SC2016
mutate default-poll 's/\[int\]\$PollSeconds = 5/[int]$PollSeconds = 50/' 'default poll'
@@ -714,6 +730,10 @@ for psrun in "${psruns[@]}"; do
throttle 'a landed post throttles the next'
# shellcheck disable=SC2016
mutate_leg backoff-never-skips 's/if (\$skip -gt 0)/if ($false)/' backoff 'nothing backs off'
# shellcheck disable=SC2016
mutate_leg failures-not-counted \
's/if (\$ok) { \$failed = 0; \$lagMax = 0 } else { \$failed++ }/$failed = 0/' \
backoff 'counted the failures before it'
if test "$devfull" -eq 1; then
mutate_leg log-write-fatal 's/try { \(Write-Host .*\) } catch { }/\1/' \
fullstdout 'took the loop with it'

View File

@@ -72,6 +72,45 @@ kill_tree 99
test "$(cat "$tmp/killed")" = '/F /T /PID 4242' || fail "kill_tree with a winpid ran: $(cat "$tmp/killed")"
win_pid() { :; }
# The image read while the target was alive: a freed winpid can already be a
# stranger's, and /T would take its children too (#1228).
: >"$tmp/killed"
kill_tree 99 4242 PROXYTRACK.EXE
test "$(cat "$tmp/killed")" = '/F /T /PID 4242' ||
fail "a verified tree kill did not run: $(cat "$tmp/killed")"
: >"$tmp/killed"
out=$(kill_tree 99 4242 python.exe)
test ! -s "$tmp/killed" || fail "pid 4242 was killed as a python.exe: $(cat "$tmp/killed")"
grep -q '::warning::pid 4242 no longer runs python.exe' <<<"$out" ||
fail "the skipped kill was not reported: $out"
# Gone from the table entirely, which is what a freed winpid usually looks like.
: >"$tmp/killed"
kill_tree 99 4343 proxytrack.exe >/dev/null
test ! -s "$tmp/killed" || fail "a pid tasklist does not list was killed: $(cat "$tmp/killed")"
# A stranger's pid leaves the caller with no target, so the serial runner's last
# resort still applies: skipping it too would leave the engines running.
: >"$tmp/killed"
HTTRACK_EXCLUSIVE_HOST=1 kill_tree 99 4242 python.exe >/dev/null
got=$(sort "$tmp/killed")
test "$got" = "$want" || fail "an unverified pid skipped the last-resort sweep: $got"
# Graded on the order, since only a winpid read while the target lived names it.
: >"$tmp/killed"
: >"$tmp/order"
kill() { echo "kill $*" >>"$tmp/order"; }
win_capture() {
echo "capture $*" >>"$tmp/order"
WIN_PID=4242 WIN_IMAGE=proxytrack.exe
}
# The pid is fictional, and reap_bounded polls it through the kill stub above.
reap_bounded() { :; }
stop_server 99
got=$(tr '\n' ' ' <"$tmp/order")
test "$got" = 'capture 99 kill 99 ' || fail "stop_server did not capture before signalling: $got"
test "$(cat "$tmp/killed")" = '/F /T /PID 4242' ||
fail "stop_server did not tree-kill what it captured: $(cat "$tmp/killed")"
unset -f kill win_capture reap_bounded
: >"$tmp/killed"
out=$(reap_leftover_processes 99_probe.test)
grep -q '99_probe.test left processes behind' <<<"$out" || fail "the leak was not attributed: $out"

View File

@@ -6,6 +6,10 @@
# a break needing three headers, or a macro the consumer defined first, is out
# of reach here. The sweep is shared with the MSVC job, which has no automake to
# install with and so stages the same list out of DevIncludes_DATA (#1153).
#
# n^2 compiles is real work, not a wedge: emulated, it needs more than the suite's
# default budget, and the sweep paces itself against whatever is left of this one.
# TEST_TIMEOUT_AT_LEAST: 900
set -euo pipefail
@@ -69,7 +73,10 @@ done
sweep_argv=(--headers-dir "$tmp/include/httrack" --cc "${CC:-cc}" --cxx "$cxx")
[ "${#cpp_argv[@]}" -eq 0 ] || sweep_argv+=(-- "${cpp_argv[@]}")
bash "$testdir/install-headers-sweep.sh" "${sweep_argv[@]}" ||
fail "installed headers do not survive every include order"
rc=0
bash "$testdir/install-headers-sweep.sh" --budget "$(budget_left)" "${sweep_argv[@]}" || rc=$?
# 77 is the sweep giving up on a host too slow to finish it, not a broken header.
[ "$rc" -ne 77 ] || exit 77
[ "$rc" -eq 0 ] || fail "installed headers do not survive every include order"
exit 0

View File

@@ -39,15 +39,8 @@ test "$((SECONDS - start))" -lt 15 || fail "watchdog fired late"
rc=0
if is_windows; then
# Existence by exact Windows PID, not a global ping.exe count: the timing
# sub-test above leaves a still-dying ping that a count would race. Plain
# tasklist, no switches (the workflow's MSYS2_ARG_CONV_EXCL='*' mangles a
# //FI filter arg into a silent no-match); $2 is the PID, and $1 must be
# ping.exe too, since Windows hands a freed PID straight back out. Folded
# case, as the tasklist matchers in proclib.sh already are.
alive() {
tasklist 2>/dev/null |
awk -v p="$1" 'tolower($1) == "ping.exe" && $2 == p {f = 1} END {exit !f}'
}
# sub-test above leaves a still-dying ping that a count would race.
alive() { win_pid_runs "$1" ping.exe; }
# alive() is a conjunction now, and one that never matches would call every
# survivor reaped. Prove it fires on a live ping, reached the same way.
ping -n 20 127.0.0.1 >/dev/null 2>&1 &
@@ -55,6 +48,12 @@ if is_windows; then
cw=$(cat "/proc/$cpid/winpid" 2>/dev/null)
test -n "$cw" || fail "could not read a live ping's Windows PID"
alive "$cw" || fail "alive() cannot see a running ping.exe (pid $cw)"
# What kill_tree checks before firing (#1228), on a live process: /proc must
# name the image tasklist answers with, or the check passes nothing on.
win_capture "$cpid"
test "$WIN_PID" = "$cw" || fail "win_capture read winpid '$WIN_PID', /proc says $cw"
win_pid_runs "$cw" "$WIN_IMAGE" || fail "tasklist does not call pid $cw a '$WIN_IMAGE'"
! win_pid_runs "$cw" no-such-image.exe || fail "win_pid_runs accepts any image at all"
kill_tree "$cpid" "$cw"
wait "$cpid" 2>/dev/null || true
# The grandchild ping records its own Windows PID: non-empty proves it ran

View File

@@ -73,14 +73,18 @@ ci_start_native_watchdog() {
# End the step, announcing $2 first: the kill runs no EXIT trap, so an unexplained
# death is all the log would otherwise hold.
ci_heartbeat_kill() {
local main=$1
local main=$1 winpid winimage
ci_annotate error "suite watchdog" "$2"
# Ahead of the kill, which runs no EXIT trap: an orphan would outlive the
# step and overwrite its last status with a frozen tail.
test -z "${watchdog:-}" || kill_pid "$watchdog"
# Read before the two kills below, which would leave the winpid naming
# whoever Windows hands the number to next (#1228).
win_capture "$main"
winpid=$WIN_PID winimage=$WIN_IMAGE
# Direct first: kill_tree may reap this watchdog before its own root (#953).
kill_pid "$main"
kill_tree "$main"
kill_tree "$main" "$winpid" "$winimage"
}
ci_suite_heartbeat() {

View File

@@ -3,7 +3,8 @@
# channel that outlives a dead runner. Every status carries the same state.
param(
[string]$ProgressLog = '',
[int]$IntervalSeconds = 30,
# 15s, not 30: a lost runner dies inside a single status period (#1228).
[int]$IntervalSeconds = 15,
[int]$PollSeconds = 5,
# Cannot outlive the step, whatever the caller forgets to kill.
[int]$MaxSeconds = 2700,
@@ -48,19 +49,40 @@ function Format-WatchdogStatus {
$q = '?'
if ($Static -ge 0) { $q = [string]$Static }
$t = ($InFlight -replace '\s+', ' ').Trim()
if ($t.Length -gt 46) { $t = $t.Substring(0, 46) }
if ($t.Length -gt 30) { $t = $t.Substring(0, 30) }
$s = 't={0}s q={1}s {2} | {3}' -f $Elapsed, $q, $t, $Counters
if ($s.Length -gt 140) { $s = $s.Substring(0, 140) }
return $s
}
# The worse of the running peak and how much longer the last iteration took than
# the poll it asked for. Peak, not last: a status covers several iterations.
function Get-MaxLag {
param([int]$Peak, [double]$Elapsed, [double]$Since, [int]$Poll)
$lag = [int]($Elapsed - $Since - $Poll * 1000)
if ($lag -gt $Peak) { return $lag }
return $Peak
}
# The TCP counters are cumulative since boot, so only the change over a status
# period says what the suite did; $Prev is $null on the first sample.
function Get-TcpDelta {
param($Prev, $Cur)
if ($null -eq $Prev) { return 'n=? f=?' }
return 'n={0} f={1}' -f ($Cur[0] - $Prev[0]), ($Cur[1] - $Prev[1])
}
# --- probes ------------------------------------------------------------------
$script:LastTcp = $null
# One try/catch per counter: a probe that fails costs its own field, not the loop.
# In-process only. A CIM query is richer, but its connect to a wedged WMI service
# is unbounded, and would hang the one reporter still standing.
function Get-WatchdogCounters {
param([int]$LagMs = 0, [int]$Failed = 0)
$f = New-Object System.Collections.ArrayList
$ps = @()
try {
$ps = @(Get-Process)
[void]$f.Add('p={0}' -f $ps.Count)
@@ -68,8 +90,31 @@ function Get-WatchdogCounters {
} catch { [void]$f.Add('p=? h=?') }
try {
$drive = New-Object System.IO.DriveInfo($env:SystemDrive + '\')
[void]$f.Add('d={0}' -f [int]($drive.AvailableFreeSpace / 1MB))
[void]$f.Add('d={0}' -f [int]($drive.AvailableFreeSpace / 1GB))
} catch { [void]$f.Add('d=?') }
# The ramp detector: starvation is what makes a poll overshoot.
[void]$f.Add('l={0}' -f $LagMs)
# Box-stop against network-break: the status that lands after an outage says
# how many it swallowed, and a box that stopped never lands one.
[void]$f.Add('x={0}' -f $Failed)
try {
# One GetTcpStatisticsEx; GetActiveTcpConnections() would allocate per socket.
$t = [System.Net.NetworkInformation.IPGlobalProperties]::GetIPGlobalProperties().GetTcpIPv4Statistics()
$cur = @($t.ConnectionsInitiated, ($t.FailedConnectionAttempts + $t.ResetConnections))
[void]$f.Add((Get-TcpDelta $script:LastTcp $cur))
$script:LastTcp = $cur
[void]$f.Add('e={0}' -f $t.CurrentConnections)
} catch { [void]$f.Add('n=? f=? e=?') }
try {
if ($ps.Count -lt 1) { throw 'no process list' }
[void]$f.Add('m={0}' -f [int]((($ps | Measure-Object -Property WorkingSet64 -Sum).Sum) / 1MB))
[void]$f.Add('c={0}' -f [int]((($ps | Measure-Object -Property PagedMemorySize64 -Sum).Sum) / 1MB))
# Its own field: the agent is what stops reporting, and the box total hides it.
$agent = @($ps | Where-Object { $_.Name -eq 'Runner.Worker' })
$ws = 0
if ($agent.Count -gt 0) { $ws = [int]((($agent | Measure-Object -Property WorkingSet64 -Sum).Sum) / 1MB) }
[void]$f.Add('a={0}' -f $ws)
} catch { [void]$f.Add('m=? c=? a=?') }
return ($f -join ' ')
}
@@ -156,14 +201,15 @@ function Invoke-WatchdogSelfTest {
Assert-That ($ko[0] -eq 8 -and $ko[1] -eq 8) 'a repeat rejection does not widen the gap'
$long = '43_local-update-truncate-with-a-very-long-name-indeed.test'
$line = Format-WatchdogStatus 812 41 $long 'p=118 h=41230 d=13210'
# The widest real counter line, so a status that fits here fits on the runner.
$line = Format-WatchdogStatus 2700 2700 $long 'p=201 h=54598 d=85 l=120 x=0 n=412 f=0 e=180 m=3100 c=4200 a=210'
Assert-That ($line.Length -le 140) ('status description is {0} characters' -f $line.Length)
Assert-That ($line -like 't=812s q=41s 43_local-update-truncate*') ('status leads with the wrong fields: {0}' -f $line)
Assert-That ($line -like '*d=13210') 'the counters did not survive a long test name'
Assert-That ($line -like 't=2700s q=2700s 43_local-update-truncate*') ('status leads with the wrong fields: {0}' -f $line)
Assert-That ($line -like '*a=210') 'the counters did not survive a long test name'
# -match, not -like: '?' is a wildcard there, so q=0s would satisfy it too.
Assert-That ((Format-WatchdogStatus 8 -1 'x' 'y') -match '^t=8s q=\?s x \| y$') 'an unknown staticness reads as a number'
$clip = Format-WatchdogStatus 1 2 ('x' * 80) 'c'
Assert-That ($clip -match '^t=1s q=2s x{46} \| c$') ('the in-flight name was not clipped to 46: {0}' -f $clip)
Assert-That ($clip -match '^t=1s q=2s x{30} \| c$') ('the in-flight name was not clipped to 30: {0}' -f $clip)
$wide = Format-WatchdogStatus 1 2 ('x' * 300) ('y' * 300)
Assert-That ($wide.Length -le 140) ('an oversized status was not clipped: {0}' -f $wide.Length)
# Cut from the tail: the head carries the fields a wedge is read for.
@@ -181,15 +227,23 @@ function Invoke-WatchdogSelfTest {
# Space-separated key=value: the counters share the 140-char description with
# the fields a wedge is read for, and '?' from a failed probe is a value.
$c = Get-WatchdogCounters
$c = Get-WatchdogCounters 120 3
Assert-That ($c -match '^[a-z]+=\S+( [a-z]+=\S+)*$') ('the counters are not key=value pairs: {0}' -f $c)
foreach ($k in 'p', 'h', 'd') {
foreach ($k in 'p', 'h', 'd', 'l', 'x', 'n', 'f', 'e', 'm', 'c', 'a') {
Assert-That ($c -match ('(^| ){0}=' -f $k)) ('the counters dropped {0}=: {1}' -f $k, $c)
}
Assert-That ($c.Length -le 60) ('the counters take {0} of the 140 characters' -f $c.Length)
Assert-That ($c -match '(^| )l=120( |$)') ('the loop lag is not what the caller measured: {0}' -f $c)
Assert-That ($c -match '(^| )x=3( |$)') ('the failed-post count is not what the caller passed: {0}' -f $c)
Assert-That ((Get-MaxLag 0 6200 1000 5) -eq 200) 'a poll that overshot by 200ms was not measured'
Assert-That ((Get-MaxLag 500 6200 1000 5) -eq 500) 'a smaller lag replaced the peak'
Assert-That ((Get-MaxLag 0 5900 1000 5) -eq 0) 'a poll that returned early reported a lag'
Assert-That ((Get-TcpDelta $null @(70, 9)) -eq 'n=? f=?') 'a first sample with no predecessor reported a delta'
Assert-That ((Get-TcpDelta @(64, 7) @(70, 9)) -eq 'n=6 f=2') 'a total was reported where the delta was asked for'
# 140 less the 16 of t=/q= and the 33 a clipped test name and its separator take.
Assert-That ($c.Length -le 91) ('the counters take {0} of the 140 characters' -f $c.Length)
# Nothing else reads these: every other leg passes its own schedule.
Assert-That ($IntervalSeconds -eq 30) ('the default status cadence is {0}s' -f $IntervalSeconds)
Assert-That ($IntervalSeconds -eq 15) ('the default status cadence is {0}s' -f $IntervalSeconds)
Assert-That ($PollSeconds -eq 5) ('the default poll is {0}s' -f $PollSeconds)
Assert-That (-not (Send-WatchdogStatus 'self-test')) 'the self-test can reach the API'
@@ -216,6 +270,9 @@ $movedAt = 0
$postedAt = -$IntervalSeconds
$backoff = 0
$skip = 0
$lagMax = 0
$failed = 0
$tickAt = $sw.Elapsed.TotalMilliseconds
# Guarded like the rest; the launcher waits for this exact line.
try { Write-Host 'watchdog ready' } catch { }
@@ -223,7 +280,12 @@ Write-WatchdogLog ('watching {0} every {1}s' -f $ProgressLog, $IntervalSeconds)
while ($sw.Elapsed.TotalSeconds -lt $MaxSeconds) {
# Measured, never accumulated: starvation is what makes a sleep overshoot.
$now = [int]$sw.Elapsed.TotalSeconds
$ms = $sw.Elapsed.TotalMilliseconds
$now = [int]($ms / 1000)
# Measured at the top, so the lag covers the probes and the post as well as
# the sleep: starvation stretches all three.
$lagMax = Get-MaxLag $lagMax $ms $tickAt $PollSeconds
$tickAt = $ms
try {
$tail = Get-ProgressTail -Path $ProgressLog
if ($tail.Ok -and $tail.Signature -ne $lastSig) {
@@ -234,14 +296,18 @@ while ($sw.Elapsed.TotalSeconds -lt $MaxSeconds) {
$postedAt = $now
$static = -1
if ($tail.Ok) { $static = $now - $movedAt }
$desc = Format-WatchdogStatus $now $static $tail.Line (Get-WatchdogCounters)
$desc = Format-WatchdogStatus $now $static $tail.Line (Get-WatchdogCounters $lagMax $failed)
# Logged whatever the backoff decides: it throttles the API, not the
# artifact, which is all a run whose token cannot post will leave.
Write-WatchdogLog $desc
if ($skip -gt 0) {
$skip--
} else {
$next = Get-NextThrottle (Send-WatchdogStatus $desc) $backoff
$ok = Send-WatchdogStatus $desc
# Cleared together, and only by a status that landed: a peak reached
# while nothing was getting through is what the next one has to carry.
if ($ok) { $failed = 0; $lagMax = 0 } else { $failed++ }
$next = Get-NextThrottle $ok $backoff
$skip = $next[0]
$backoff = $next[1]
}

View File

@@ -14,7 +14,8 @@ set -euo pipefail
usage() {
echo "usage: ${0##*/} {--srcdir DIR [--builddir DIR] | --headers-dir DIR}" \
"[--backend cl|cc] [--cc CMD] [--cxx CMD] [--self-test] [-- CPPFLAGS...]" >&2
"[--backend cl|cc] [--cc CMD] [--cxx CMD] [--budget SECONDS] [--self-test]" \
"[-- CPPFLAGS...]" >&2
exit 1
}
@@ -26,6 +27,7 @@ cc_cmd=""
cxx_cmd=""
cxx_set=0
selftest=0
budget=
extra=()
while [ $# -gt 0 ]; do
case $1 in
@@ -33,6 +35,10 @@ while [ $# -gt 0 ]; do
selftest=1
shift
;;
--budget)
budget=${2-}
shift 2 || usage
;;
--srcdir)
srcdir=${2-}
shift 2 || usage
@@ -237,15 +243,45 @@ fi
began=$SECONDS
bad=0
# Sliced only for a caller that gave a budget: one call per batch cannot be given up on,
# and an emulated compiler needs more time for it than the harness allows a test (#1146).
# Unsliced elsewhere, so the Windows job keeps paying one compiler spawn per batch.
if [ -n "$budget" ] && [ "$budget" -gt 0 ]; then
export HTTRACK_TEST_TIMEOUT=$budget
slices=8
else
slices=1
fi
slice=$(((${#units[@]} + slices - 1) / slices))
# From the slice size, not from $slices: they differ whenever the units do not divide
# evenly, and a step count that outlives the loop leaves the pacer projecting forever.
per=$(((${#units[@]} + slice - 1) / slice))
left=$((${#langs[@]} * ${#modes[@]} * per))
swept=0
for lang in "${langs[@]}"; do
for mode in "${modes[@]}"; do
compile "$lang" "$mode" "${units[@]}" || {
head -40 "$sweep_log" >&2
echo "the headers do not compile as $lang standalone and pairwise ($mode)" >&2
bad=1
}
i=0
while [ "$i" -lt "${#units[@]}" ]; do
step=$SECONDS
chunk=("${units[@]:i:slice}")
swept=$((swept + ${#chunk[@]}))
compile "$lang" "$mode" "${chunk[@]}" || {
head -40 "$sweep_log" >&2
echo "the headers do not compile as $lang standalone and pairwise ($mode)" >&2
bad=1
}
i=$((i + slice))
left=$((left - 1))
# Only while nothing has failed: a skip past a real break would bury it.
[ "$bad" -ne 0 ] || [ -z "$budget" ] ||
skip_if_out_of_budget "$left" "$((SECONDS - step))"
done
done
done
# What reached the compiler, not what was generated: a slice loop that steps past a unit
# would otherwise report the full set and pass.
want=$((${#langs[@]} * ${#modes[@]} * ${#units[@]}))
[ "$swept" -eq "$want" ] || fail "compiled $swept units of $want, the slicing lost some"
echo "swept $n headers standalone and pairwise x ${#modes[@]} bytecode modes x ${langs[*]}" \
"= $((${#modes[@]} * ${#langs[@]} * ${#units[@]})) units in $((SECONDS - began))s with $backend"
[ "$bad" -eq 0 ] || exit 1

View File

@@ -22,19 +22,39 @@ testdir=$(cd "$(dirname "$0")" && pwd)
# (CRAWL_DEADLINE, 180s a pass) -- budget below that and a slow-but-legitimate
# run would be killed. The slowest healthy test measures 39s. A non-numeric or
# absurd value falls back; 0 disables the guard, for use under a debugger.
budget=${HTTRACK_TEST_TIMEOUT:-600}
case "$budget" in
'' | *[!0-9]*) budget=600 ;;
esac
budget=$(budget_secs)
# The test script is the last argument; automake passes no others today.
for path in "$@"; do :; done
name=$(basename "$path")
# A test whose work legitimately outlasts the wedge budget says so in its header
# (269 sweeps n^2 compiles and paces itself inside it). The name carries the rule the
# reader cannot see: it raises the budget, so no test can disarm the guard. Read with
# the shell to keep it off the per-test fork bill, and bounded, since bash's `test`
# errors rather than compares past intmax and would leave the guard unarmed.
if test "$budget" -gt 0 && test -r "$path"; then
read_lines=0
while test "$read_lines" -lt 40 && IFS= read -r line; do
read_lines=$((read_lines + 1))
case "$line" in
'# TEST_TIMEOUT_AT_LEAST: '*)
want=${line#'# TEST_TIMEOUT_AT_LEAST: '}
case "$want" in
'' | *[!0-9]* | ???????*) ;;
*) test "$((10#$want))" -le "$budget" || budget=$((10#$want)) ;;
esac
break
;;
esac
done <"$path"
fi
# Exported so a test can pace itself against the same number (skip_if_out_of_budget)
# instead of being killed halfway.
export HTTRACK_TEST_TIMEOUT="$budget"
test "$budget" -gt 0 || exec "$BASH" "$@"
# The test script is the last argument; automake passes no others today.
for name in "$@"; do :; done
name=$(basename "$name")
# Give the test its own TMPDIR, so the hang dump can salvage exactly this test's
# crawl logs instead of racing (and deleting) a sibling's under "make check -j".
tmproot=${TMPDIR:-/tmp}

View File

@@ -286,8 +286,12 @@ poll_wait() {
# trap, where a survivor would turn a passing test into a harness timeout.
stop_server() {
test -n "${1:-}" || return 0
local winpid winimage
# Before the signal: a winpid read after it can already name a stranger.
win_capture "$1"
winpid=$WIN_PID winimage=$WIN_IMAGE
kill "$1" 2>/dev/null || true
if is_windows; then kill_tree "$1"; fi
if is_windows; then kill_tree "$1" "$winpid" "$winimage"; fi
reap_bounded "$1" || true
return 0
}
@@ -454,6 +458,29 @@ win_pid() {
fi
}
# WIN_PID and WIN_IMAGE for MSYS pid $1, read while it is alive: /proc keeps the
# entry once the process is gone and Windows reissues the number at once, so a
# later read can name a stranger (#1228). Not for a job just backgrounded: until
# its exec lands, tens of milliseconds later, both still name the forking shell.
# Assigned rather than echoed, a command substitution being a fork (#795).
win_capture() { # win_capture <pid>
WIN_PID='' WIN_IMAGE=''
is_windows || return 0
# Unguarded reads: a missing file leaves the empty value set above, and read
# reports EOF on an unterminated line having already assigned it.
{ read -r WIN_PID <"/proc/$1/winpid"; } 2>/dev/null || true
{ read -r WIN_IMAGE <"/proc/$1/winexename"; } 2>/dev/null || true
WIN_IMAGE=${WIN_IMAGE##*[\\/]}
return 0
}
# Whether Windows PID $1 runs image $2. Both columns at once, since either alone
# answers for a recycled PID, and case-folded as the proclib.sh matchers are.
win_pid_runs() { # win_pid_runs <winpid> <image>
tasklist 2>/dev/null |
awk -v p="$1" -v i="$2" 'tolower($1) == tolower(i) && $2 == p { f = 1 } END { exit !f }'
}
# Signal one process, never its descendants: a caller inside the target's own
# tree cannot rely on kill_tree, whose taskkill is then a grandchild of it (#953).
kill_pid() {
@@ -477,11 +504,17 @@ kill_pid() {
# so args pass verbatim and a //T would reach taskkill unfolded and be rejected.
# $2 is that Windows PID when the caller read it while the job was certainly
# alive: /proc/<pid>/winpid is already gone for a job that has just died, and
# without it the only route left is the host-wide sweep below.
# without it the only route left is the host-wide sweep below. $3 is the image it
# ran then: a number that no longer runs it was reissued while we were not
# looking, and naming a stranger is as good as naming nobody (#1228).
kill_tree() {
local pid=$1 winpid=${2:-}
local pid=$1 winpid=${2:-} image=${3:-}
if is_windows; then
test -n "$winpid" || winpid=$(win_pid "$pid")
if test -n "$winpid" && test -n "$image" && ! win_pid_runs "$winpid" "$image"; then
printf '::warning::pid %s no longer runs %s, not killing it\n' "$winpid" "$image"
winpid=
fi
if test -n "$winpid"; then
taskkill /F /T /PID "$winpid" >/dev/null 2>&1 || true
# Last resort, so it is opt-in: it kills every engine and every python on
@@ -539,16 +572,39 @@ EOF
# one step is slower than its neighbours. It asks an ordering of the callers
# instead, expensive steps first, so no step left can outrun the reserve the one
# before it set (#1146).
skip_if_out_of_budget() { # skip_if_out_of_budget <steps left> <seconds the last took>
local budget=${HTTRACK_TEST_TIMEOUT:-600} need=$(($2 + $2 / 2))
# The budget test-timeout.sh enforces, in seconds, 0 being the guard off. The one
# parser: a value bash arithmetic or test would choke on falls back to the default,
# and a leading zero would otherwise read as octal in one place and decimal in the next.
budget_secs() {
local budget=${HTTRACK_TEST_TIMEOUT:-600}
case "$budget" in '' | *[!0-9]* | ???????*) budget=600 ;; esac
echo "$((10#$budget))"
}
case "$budget" in '' | *[!0-9]*) budget=600 ;; esac
skip_if_out_of_budget() { # skip_if_out_of_budget <steps left> <seconds the last took>
local budget need=$(($2 + $2 / 2))
budget=$(budget_secs)
test "$1" -gt 0 && test "$budget" -gt 0 || return 0
test "$((SECONDS + need))" -ge "$budget" || return 0
echo "$1 steps left, the last took ${2}s and the budget is ${budget}s; skipping" >&2
exit 77
}
# Seconds left of the budget, for a child pacing itself against it (269 hands it to
# the sweep). Never below 1 unless the guard is off, when it stays 0.
budget_left() {
local budget left
budget=$(budget_secs)
test "$budget" -gt 0 || {
echo 0
return 0
}
left=$((budget - SECONDS))
test "$left" -ge 1 || left=1
echo "$left"
}
# Collect a killed job, giving up after REAP_GRACE seconds. kill_tree can fail to
# reap a native Windows descendant -- the very case these watchdogs exist for --
# and a bare `wait` then blocks the watchdog itself forever, so the timeout it was

View File

@@ -48,7 +48,12 @@ cd /bld
bash "${GITHUB_WORKSPACE:-/src}/configure"
make -j"$(nproc)"
# The buildd's own invocation, so a failure here is the one it would report.
make check -j"$(nproc)"
rc=0
make check -j"$(nproc)" || rc=$?
# Always, not only where automake prints it: this leg exists to say what an emulated
# host does. A paced-out skip must not read as coverage with no reason given.
cat tests/test-suite.log || true
test "$rc" -eq 0 || exit "$rc"
# make check exits 0 for an all-SKIP run, and this leg skips a lot by design, so
# a container that quietly lost a dependency would report a green covering

View File

@@ -22,7 +22,8 @@
# -o, --outdir DIR output directory (default: <repo>/dist)
# --orig FILE reuse this upstream orig tarball instead of
# regenerating it (required for a Debian revision
# >= 2, whose orig is frozen in the archive)
# >= 2, whose orig is frozen in the archive, and
# whenever debian/patches carries a patch)
# -s, --source-only build only the source package
# -u, --unsigned do not sign anything (implies no release sigs)
# --no-release-artifacts skip the orig tarball .asc/.md5/.sha1
@@ -39,7 +40,9 @@
#
# The Debian revision in debian/changelog decides the orig: revision 1 builds a
# fresh upstream tarball; revision >= 2 must reuse the orig frozen at revision 1
# (the .dsc references it by checksum), so pass it with --orig.
# (the .dsc references it by checksum), so pass it with --orig. debian/patches
# needs the same tarball for a different reason: a patch backported from upstream
# no longer applies to a tree that has the fix, which is what HEAD would give.
#
# SOURCE_DATE_EPOCH is honored for reproducible output.
@@ -127,6 +130,10 @@ main() {
if [[ $unsigned -eq 0 ]]; then
need gpg
[[ -n $key ]] || die "no signing key (pass --key or set DEBSIGN_KEYID, or use --unsigned)"
# Here rather than at debsign, which runs once the tarball is built: a key id gpg
# cannot resolve to a secret key would otherwise cost the whole build first.
gpg --list-secret-keys -- "$key" >/dev/null 2>&1 ||
die "gpg has no secret key for '$key' (an 0x-prefixed full fingerprint is unambiguous)"
fi
local repo
@@ -168,6 +175,13 @@ main() {
die "Debian revision $rev needs --orig FILE (the orig is frozen from revision 1)"
fi
# A quilt patch is written against the orig it is applied to. Once the fix is
# upstream, HEAD already carries it, so a regenerated orig makes the patch fail
# or, worse, apply with fuzz. Unsigned too: this one breaks the build, not policy.
if [[ -z $orig_in && -s $export_dir/debian/patches/series ]]; then
die "debian/patches is not empty, so --orig FILE is required: the orig built from HEAD already carries the patches"
fi
if [[ -n $orig_in ]]; then
info "reusing upstream tarball $orig_in"
cp -- "$orig_in" "$scratch/$orig"