Compare commits

...

3 Commits

Author SHA1 Message Date
Xavier Roche
4c93947f44 Allowlist the Windows skip of the footer-overflow test
The Windows CI pins expected skips so an all-skipped suite can't pass green;
01_engine-footer-overflow.test now skips there (MAX_PATH), so add it to the list.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-22 19:14:49 +02:00
Xavier Roche
680ece6518 Skip the footer-overflow test on Windows (MAX_PATH)
The test triggers the overflow with a path longer than Windows MAX_PATH (260),
which the source tree and mirror output both exceed there, so httrack can't
create it and the crawl fails for an unrelated reason. Restrict it to POSIX; the
fix is platform-independent and the formatter's overflow-return contract is
still covered cross-platform by the footerfmt self-test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-22 18:49:42 +02:00
Xavier Roche
76c989abaf Skip an oversized -%F footer instead of aborting the crawl
The per-page footer is expanded into a fixed ~3KB stack buffer. On overflow
hts_footer_format returns <0 and leaves the buffer unterminated, but the call
site ignored the return and ran strcatbuff on it regardless; strcatbuff's
bounded strlen finds no terminator within capacity and abort()s, killing the
crawl with SIGABRT. Reachable whenever the footer expansion exceeds the buffer
(a field referenced several times with a long URL/path, or a URL that triples
under the HTML-comment escaping); the pre-existing default footer has the same
exposure.

Guard the emit on the formatter's return: on overflow, drop the footer rather
than crash. Regression test drives a file:// crawl of a deep path with a footer
repeating {path} and asserts the crawl completes; it aborts on the pre-fix
binary.

Closes #669

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-22 18:31:42 +02:00
4 changed files with 70 additions and 7 deletions

View File

@@ -202,7 +202,8 @@ jobs:
# Every gate here exits 77, so an all-skipped suite would report green having
# tested nothing: pin the skips, and floor the passes in case the glob empties.
expected_skips=" 48_local-crange-memresume.test 71_local-crange-repaircache.test" # pending #581
# footer-overflow skips on Windows (needs a path past MAX_PATH); crange pending #581.
expected_skips=" 01_engine-footer-overflow.test 48_local-crange-memresume.test 71_local-crange-repaircache.test"
[ "$pass" -ge 90 ] || { echo "::error::only $pass tests passed ($skip skipped)"; exit 1; }
[ "$skipped" = "$expected_skips" ] || { echo "::error::unexpected skips:$skipped"; exit 1; }
[ "$fail" -eq 0 ] || { echo "::error::failing:$failed"; exit 1; }

View File

@@ -846,12 +846,17 @@ int htsparse(htsmoduleStruct * str, htsmoduleStructExtended * stre) {
tempo[0] = '\0';
strcatbuff(tempo, eol);
hts_footer_format(tempo + strlen(tempo),
sizeof(tempo) - strlen(tempo),
StringBuff(opt->footer), fields,
sizeof(fields) / sizeof(fields[0]));
strcatbuff(tempo, eol);
HT_ADD(tempo);
// hts_footer_format returns <0 on overflow, leaving tempo
// unterminated; emitting it would abort in strcatbuff
// below.
if (hts_footer_format(tempo + strlen(tempo),
sizeof(tempo) - strlen(tempo),
StringBuff(opt->footer), fields,
sizeof(fields) / sizeof(fields[0])) >=
0) {
strcatbuff(tempo, eol);
HT_ADD(tempo);
}
}
}
// Emit charset ?

View File

@@ -0,0 +1,56 @@
#!/bin/bash
#
# Keep this POSIX-portable: the harness runs it via $(BASH), which is a plain
# POSIX /bin/sh on some platforms (e.g. macOS), so avoid bashisms despite the
# #!/bin/bash above.
# A -%F footer whose expansion overflows the on-page buffer must be dropped, not
# crash the crawl. Before the fix the unchecked hts_footer_format return left the
# buffer unterminated and the next strcatbuff aborted (SIGABRT).
set -eu
# The overflow needs a path longer than Windows MAX_PATH (260), so both the
# source tree and the mirror output blow past it there; run on POSIX only. The
# fix itself is platform-independent and covered on Linux/macOS.
case "$(uname -s 2>/dev/null)" in
MINGW* | MSYS* | CYGWIN*) exit 77 ;;
esac
dir=$(mktemp -d)
trap 'rm -rf "$dir"' EXIT
# A few-hundred-char file path (kept well under the URL length limit) makes the
# {path} field long.
seg=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
deep="$dir"
i=0
while [ "$i" -lt 6 ]; do
deep="$deep/$seg"
i=$((i + 1))
done
mkdir -p "$deep"
printf '<html><body>hi</body></html>' >"$deep/index.html"
# 40 {path} references (~240 chars, under the 254-char -%F cap); expanded against
# a few-hundred-char path this far exceeds the per-page footer buffer.
footer=""
i=0
while [ "$i" -lt 40 ]; do
footer="$footer{path}"
i=$((i + 1))
done
mir="$dir/mir"
httrack "file://$deep/index.html" -O "$mir" -%F "$footer" -q -s0 -%v0 \
>/dev/null 2>&1 || {
echo "crawl failed/aborted (exit $?) on an oversized footer" >&2
exit 1
}
# The crawled page must exist (proves the URL wasn't rejected for length, so the
# footer path ran). Look under file/, not $mir, to skip the makeindex top index.
find "$mir/file" -name index.html | grep -q . || {
echo "page not mirrored; the oversized-footer path was not exercised" >&2
exit 1
}

View File

@@ -39,6 +39,7 @@ TESTS = \
01_engine-doitlog.test \
01_engine-entities.test \
01_engine-footerfmt.test \
01_engine-footer-overflow.test \
01_engine-filelist.test \
01_engine-filter.test \
01_engine-filterdual.test \