Compare commits

..

3 Commits

Author SHA1 Message Date
Xavier Roche
36b4f4733d Assert kept content in the update-error test, not just size (#176 follow-up)
44_local-update-errormask checked only keep.dat's size (>= 1024). The good
body carries a distinctive KEEP marker that nothing asserted, so a same-size
wrong-content overwrite would still pass. Also assert the content matches
^KEEP and lacks the 403 error body, so the test discriminates the fix from
an equal-size clobber the size check alone misses.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-11 14:39:36 +02:00
Xavier Roche
167ede32c0 Keep the good local copy when an update fetch returns an HTTP error (#176) (#523)
On --update, a URL that returned 200 on the first crawl but now answers a
transient error (403/404/5xx) had its good local copy destroyed: the error
body overwrote it (errpage is on by default) and/or the delete_old purge
removed it. The guard that masks such an error as a 304 to keep the cached
copy was gated on !opt->delete_old, but delete_old is on by default, so it
never ran for a normal update.

Drop that outer gate so the masking runs by default, but restrict it to the
complete-cached-copy case (range_req_size == 0): an error on a resume/range
fetch must still fall through, else a stale-partial 416 would be masked to
304 and the partial never re-fetched (regressing #206). The masked error
routes through the not-modified handler, which reloads and re-registers the
cached file, blocking both the overwrite and the purge. Unlinked-page purging
is unaffected: it targets pages not fetched this run, not a fetched page that
errored.

Test 44_local-update-errormask drives it over a local server whose keep.dat
200s on the first crawl and 403s on the conditional update fetch.

Closes #176

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:30:43 +02:00
Xavier Roche
c8b7c63c9b Keep LLint 64-bit on the x32 ABI (#524)
The wide-integer typedef keyed on __x86_64__, which the x32 ABI also
defines even though its long is 32-bit. That silently made LLint -- the
signed 64-bit type for byte counts and file sizes -- 32-bit on x32, so
any value past INT_MAX overflowed (a >2GB size wrapped negative and the
cache dropped the entry). Exclude __ILP32__ so x32 takes the long-long
path, plus a compile-time guard that trips wherever 64-bit support is
claimed but LLint isn't actually 64-bit.

Surfaced by the cache selftests on Debian x32: cache-writefail (oversize)
and cache-corrupt both exercise >2GB entries and failed.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:21:27 +02:00
2 changed files with 10 additions and 1 deletions

View File

@@ -355,8 +355,11 @@ typedef __int64 LLint;
typedef __int64 TStamp;
#define LLintP "%I64d"
/* x32/ILP32 sets __x86_64__ but long is 32-bit; exclude it so LLint stays
* 64-bit. */
#elif (defined(_LP64) || defined(__x86_64__) || defined(__powerpc64__) || \
defined(__64BIT__))
defined(__64BIT__)) && \
!defined(__ILP32__)
typedef long int LLint;
@@ -373,6 +376,10 @@ typedef long long int TStamp;
#endif /* HTS_LONGLONG */
/* Claiming long-long support must yield a real 64-bit LLint (x32 regressed:
__x86_64__ set but long is 32-bit). Compile-time trip, portable to C90. */
typedef char hts_assert_llint_is_64bit[sizeof(LLint) == 8 ? 1 : -1];
#else
typedef int LLint;

View File

@@ -8,4 +8,6 @@ set -euo pipefail
bash "$top_srcdir/tests/local-crawl.sh" --rerun \
--found 'errmask/keep.dat' \
--file-min-bytes 'errmask/keep.dat' 1024 \
--file-matches 'errmask/keep.dat' '^KEEP' \
--file-not-matches 'errmask/keep.dat' 'error 403' \
httrack 'BASEURL/errmask/index.html'