Compare commits

..

1 Commits

Author SHA1 Message Date
Xavier Roche
d5b05c163c Keep the good local copy when an update fetch returns an HTTP error (#176)
On --update, a URL that returned 200 on the first crawl but now answers a
transient error (403/404/5xx) had its good local copy destroyed: the error
body overwrote it (errpage is on by default) and/or the delete_old purge
removed it. The guard that masks such an error as a 304 to keep the cached
copy was gated on !opt->delete_old, but delete_old is on by default, so it
never ran for a normal update.

Drop that outer gate so the masking runs by default, but restrict it to the
complete-cached-copy case (range_req_size == 0): an error on a resume/range
fetch must still fall through, else a stale-partial 416 would be masked to
304 and the partial never re-fetched (regressing #206). The masked error
routes through the not-modified handler, which reloads and re-registers the
cached file, blocking both the overwrite and the purge. Unlinked-page purging
is unaffected: it targets pages not fetched this run, not a fetched page that
errored.

Test 44_local-update-errormask drives it over a local server whose keep.dat
200s on the first crawl and 403s on the conditional update fetch.

Closes #176

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-11 13:22:33 +02:00
2 changed files with 1 additions and 10 deletions

View File

@@ -355,11 +355,8 @@ typedef __int64 LLint;
typedef __int64 TStamp;
#define LLintP "%I64d"
/* x32/ILP32 sets __x86_64__ but long is 32-bit; exclude it so LLint stays
* 64-bit. */
#elif (defined(_LP64) || defined(__x86_64__) || defined(__powerpc64__) || \
defined(__64BIT__)) && \
!defined(__ILP32__)
defined(__64BIT__))
typedef long int LLint;
@@ -376,10 +373,6 @@ typedef long long int TStamp;
#endif /* HTS_LONGLONG */
/* Claiming long-long support must yield a real 64-bit LLint (x32 regressed:
__x86_64__ set but long is 32-bit). Compile-time trip, portable to C90. */
typedef char hts_assert_llint_is_64bit[sizeof(LLint) == 8 ? 1 : -1];
#else
typedef int LLint;

View File

@@ -8,6 +8,4 @@ set -euo pipefail
bash "$top_srcdir/tests/local-crawl.sh" --rerun \
--found 'errmask/keep.dat' \
--file-min-bytes 'errmask/keep.dat' 1024 \
--file-matches 'errmask/keep.dat' '^KEEP' \
--file-not-matches 'errmask/keep.dat' 'error 403' \
httrack 'BASEURL/errmask/index.html'