Compare commits

...

2 Commits

Author SHA1 Message Date
Xavier Roche
ee550c3972 Bound test 19's dead-address connects so it is fast on macOS
19_local-connect-fallback pins dead loopback IPs (127.0.0.2/.3) before the live
one to exercise address fallback. On Linux those refuse instantly (all of
127.0.0.0/8 is loopback); macOS has only 127.0.0.1, so the connects block to
--timeout, and default retries replay the whole crawl, so the exhaustion case
took ~2 minutes. It passed but dominated the macOS suite once the rest runs in
parallel. Drop --timeout to 5s (bounds the last candidate's stall; the earlier
ones already fall back at min(timeout,10)) and set --retries=0 (no ×3 replay).
Linux is unchanged: the refused connects never reach the timeout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-11 16:54:25 +02:00
Xavier Roche
145ea4769f Make test 31 fail loudly when the java plugin doesn't load (#529)
31_local-javaclass verifies the java plugin parses a .class constant pool
by asserting a resource named only inside Foo.class gets crawled. When the
plugin fails to load, that assertion just reports hello.gif "not found",
with no hint the plugin was the cause -- which is why an intermittent
local failure sat uninvestigated.

The failure mode is a shadowing system libhtsjava: on a box with httrack
installed, if the soname httrack dlopens is absent from the fresh build
but present in /usr/lib, the loader picks up the system plugin (a
mismatched, non-instrumented build) and it silently parses nothing. It
looked ASan-specific only because that was the build in use; clean CI
runners have no system plugin, so they never hit it. #475 aligned the
dlopen name with the build's own soname, closing the gap.

Assert the plugin's launched-banner suffix ("+libhtsjava") first, so a
load failure trips there with a clear message instead of the opaque
hello.gif miss. The "+libhtsjava" prefix survives future soname bumps.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 16:33:45 +02:00
2 changed files with 10 additions and 4 deletions

View File

@@ -66,9 +66,12 @@ test -n "$port" || {
}
out="$tmpdir/crawl"
# macOS has only 127.0.0.1, so the dead 127.0.0.x connects block to the timeout
# instead of refusing instantly (as on Linux); a small --timeout + --retries=0
# bound the stall without changing what the fallback exercises.
HTTRACK_DEBUG_RESOLVE="deadhost:127.0.0.2,127.0.0.1" \
httrack "http://deadhost:$port/simple/basic.html" -O "$out" \
-c1 --robots=0 --timeout=30 --quiet -Z >"$tmpdir/log" 2>&1
-c1 --robots=0 --timeout=5 --retries=0 --quiet -Z >"$tmpdir/log" 2>&1
log="$out/hts-log.txt"
@@ -92,12 +95,12 @@ test -f "$out/deadhost_$port/simple/basic.html" || {
exit 1
}
# every address refused: the slot exhausts the list, then errors out (the
# harness timeout would catch a hang/loop; refused connects are instant)
# every address dead: the slot exhausts the list, then errors out (the harness
# timeout would catch a hang/loop)
out2="$tmpdir/crawl2"
HTTRACK_DEBUG_RESOLVE="alldead:127.0.0.2,127.0.0.3" \
httrack "http://alldead:$port/simple/basic.html" -O "$out2" \
-c1 --robots=0 --timeout=30 --quiet -Z >"$tmpdir/log2" 2>&1
-c1 --robots=0 --timeout=5 --retries=0 --quiet -Z >"$tmpdir/log2" 2>&1
log2="$out2/hts-log.txt"
grep -q "trying next address" "$log2" || {

View File

@@ -17,7 +17,10 @@ printf 'GIF89a' >"$tmproot/javaclass/hello.gif"
printf '\xCA\xFE\xBA\xBE\x00\x00\x00\x32\x00\x02\x01\x00\x09hello.gif\x00\x00\x00\x00' \
>"$tmproot/javaclass/Foo.class"
# --log-found first: a plugin-load failure (wrong soname, shadowing system
# libhtsjava) trips here on the banner suffix, not as an opaque "not found".
bash "$top_srcdir/tests/local-crawl.sh" --root "$tmproot" --errors 0 \
--log-found '\+libhtsjava' \
--found 'javaclass/Foo.class' \
--found 'javaclass/hello.gif' \
httrack 'BASEURL/javaclass/index.html'