mirror of
https://github.com/xroche/httrack.git
synced 2026-07-23 09:09:05 +03:00
484fc47eab
* Reword the "bogus state" cache-skip warnings Users read "file not stored in cache due to bogus state" as a crash or cache corruption; it is a benign skip when the transfer is shorter than the Content-Length. Say what happened, the consequence, and the -%B override instead, and give the delayed-type variant a plain wording. Test pins updated; 22_local-broken-size gains set -e (its first crawl's audit failure was masked by the second crawl's exit status). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Make one bad cache-entry write skippable instead of ending the mirror Since #426 any new.zip write failure cleanly aborts the whole mirror. Keep that for storage-level trouble (fatal errno such as ENOSPC, or every entry failing), but let an isolated failure drop only the current entry: abandon it, warn with the URL, and keep the mirror and the cache stream going. A streak of CACHE_MAX_WRITE_FAILURES consecutive failures still aborts. Also degrade the >2GB assertf crash in cache_add: an oversized on-disk body is stored headers-only (X-In-Cache: 0), an in-memory one drops the entry. The cache-writefail self-test now pins all four regimes (fatal errno, persistent streak, isolated skip with sibling round-trip, oversize); on the previous code it fails the new assertions and hits the oversize assertf (SIGABRT). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * review: pin streak semantics, assert the skip warning, add EDQUOT Adversarial test audit found the policy's core semantics unpinned: no test distinguished a consecutive-failure cap from a total count, or proved a stored entry resets the streak. Phase 1 now asserts the abort lands exactly on the 8th consecutive failure, and a new phase drives 10 failures interleaved with successes and asserts no abort. The .test now greps a URL-bearing skip warning. check_fatal_io_errno gains EDQUOT (quota exhaustion is disk-full for our purposes). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * review: trim comments to the one-line default Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> --------- Signed-off-by: Xavier Roche <roche@httrack.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
30 lines
976 B
Bash
30 lines
976 B
Bash
#!/bin/bash
|
|
#
|
|
# Keep this POSIX-portable: the harness runs it via $(BASH), which is a plain
|
|
# POSIX /bin/sh on some platforms (e.g. macOS), so avoid bashisms and GNU-only
|
|
# tool flags despite the #!/bin/bash above.
|
|
|
|
# Cache write-failure policy (-#test=cache-writefail <dir>). #174/#219: disk
|
|
# full or a failure streak aborts cleanly; an isolated failure or an oversized
|
|
# entry is only dropped.
|
|
|
|
set -eu
|
|
|
|
dir=$(mktemp -d)
|
|
trap 'rm -rf "$dir"' EXIT
|
|
|
|
out=$(httrack -#test=cache-writefail "$dir")
|
|
|
|
# Match the exact success line (error logs also go to stdout); a renamed/removed
|
|
# test prints the registry to stderr, which exits non-zero but never prints this.
|
|
printf '%s\n' "$out" | grep -qx "cache-writefail: OK" || {
|
|
echo "expected 'cache-writefail: OK', got: $out" >&2
|
|
exit 1
|
|
}
|
|
|
|
# A skipped entry must be warned about with its URL.
|
|
printf '%s\n' "$out" | grep -q "entry not cached: example.com/" || {
|
|
echo "expected a URL-bearing skip warning" >&2
|
|
exit 1
|
|
}
|