strjoker_rec charged its work budget only after the depth-cap early-return, so depth-capped recursive calls cost nothing. A class pattern like *[a]*[a]... drives recursion to STRJOKER_MAXDEPTH, sets the sticky cut flag (disabling the failure memo), and the band below the cap then re-explores exponentially without ever tripping the budget — a ~2KB filter ran ~80s under ASan. Moving the increment ahead of the depth-cap return makes every call count, bounding total work; the match fails safe. Behavior is unchanged for real filters. Adds a fuzzer corpus regression seed and a class-branch lower-bound pin to the filterbounds self-test.
Adds fuzz-htsparse, a libFuzzer harness driving the real htsparse() over a
mocked engine: the tag/attribute/JS-inscript scanner, the link rewriter, and
the accept -> url_savename -> hts_record_link path that runs on every crawled
page, previously reached only by slow full-crawl tests. The parser's coupling
is state, not network, so the harness builds the minimal crawl state
httpmirror() sets up (opt, cache, hash, filters, robots, backing, a seeded
link heap with ptr=1) and walks a NUL-terminated page through the parser, then
discards it. Four seeds cover links, srcset, base/meta/usemap, a
document.write body, and malformed tags. Clean under ASan+UBSan+LeakSanitizer.
Bundles the src/coucal bump to ab59c6a (coucal#13): this harness is the first
fuzzer to hash keys through coucal under UBSan, which tripped a pre-existing
getblock32 pointer-overflow.
Merged past a transient arm64 apt-mirror outage; every code-exercising check
(fuzz, sanitize, msan, distcheck, all non-arm64 builds) passed.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix catastrophic backtracking in the strjoker filter matcher
The recursive wildcard matcher re-explored the same (subject, pattern)
positions exponentially on *-heavy patterns against a non-matching
subject: 9 stars against 50 characters already took 149 seconds, about
7x per added star. Subjects come from crawled URLs, so a hostile site
could stall matching against a user's filters (ReDoS).
Memoize failures instead: one bit per (subject, pattern) offset pair,
checked before and recorded after each recursion, which bounds the
work polynomially. A pair that failed once fails forever (*size is
only written on the success path), so the memo is sound and the
matcher's semantics are unchanged; a differential run of 9000 random
pattern/subject/size cases old vs new is byte-identical. The bitmap
lives on the stack for common sizes, on the heap above 2 KB.
The engine test adds the pathological pattern (instant now, hung
before) plus a case where a later star's dead-end must re-extend an
earlier star, which guards against a single-backtrack-point rewrite.
A corpus seed lets the fuzz replay job cover the same shape.
Closes#501
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Broaden matching-system test coverage
Adds near-miss negative cases for the arms the filter tests skipped
(word order, exact tails, range boundaries, class case sensitivity,
empty star runs, descending ranges, *[param], rule ordering, failed
size tests), first coverage of mime: filters via a new
-#test=filtermime self-test, and a -#test=filtermemo differential:
20000 seeded random pattern/subject/size cases where the memoized
matcher must agree with a new no-memo oracle (strjoker_nomemo) on
result, *size and *size_flag, asserting both polarities occur.
Corrupting the memo indexing by hand makes the differential fail on
case 8, so it genuinely guards the memo.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The zip cache replaced this format in 3.31 (2003); what remained was an
import-only parser for hostile input, compiled unconditionally, plus a
tail of dead code keyed to it: the CACHE-1.5 writer behind if(1), the
cross-session HTML-MD5 dedup (its store was never written in the zip
era), the legacy legs of the startup rotation (which silently renamed
or deleted a user's .dat/.ndx pair) and of hts_cache_reconcile.
cache_init now detects a legacy pair, logs a clear refusal, leaves the
files untouched, and re-crawls. The cache-legacy self-test pins refusal
for both layouts (--update old.* and ro new.*), no is_update flag, and
untouched files; proven end-to-end against the local server. The
cacheindex self-test and fuzz-cachendx stay: cache_brstr/cache_binput
still back the -#C cache listing.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The engine's header receive loop reads via cache_binput, which drops
every \r; the harness fed raw lines, so corpus bytes with \r exercised
a shape production code never sees. Strip them per line.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Add libFuzzer harnesses for the HTTP header and cache-index parsers
P3-5 fuzz Tier-2. Two more harnesses over hostile-input parsers that read
structured bytes into fixed buffers: fuzz-header drives treatfirstline plus
treathead on each response-header line (the Content-Type/-Encoding path hardened
in #506, and the cookie/Location/Content-Range fields); fuzz-cachendx drives the
hts-cache/*.ndx length-prefixed scan that cache_readex_new loads on --update.
fuzz-cachendx found the over-advance fixed in the parent commit; its seed corpus
carries the two crash reproducers as replay regressions. The cache harness stops
at the scan rather than the trailing coucal insert, whose murmur hash trips a
separate pointer-overflow the .ndx parser does not own.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Correct copyright year on the new fuzz harnesses (2026)
fuzz-header.c and fuzz-cachendx.c are new in the 2026 audit cycle;
match the sibling *_selftest.c files rather than the boilerplate 1998.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
New fuzz/ tree with a libFuzzer harness per pure (buf,size) parser that ingests
network-sourced bytes: the charset/UTF-8/IDNA codecs, the HTML <meta> charset
scan, HTML entity and URL percent decoders, the wildcard filter matcher, and
the URL splitter. Gated behind --enable-fuzzers (requires clang, forces
--disable-shared so clang's static sanitizer runtimes link); a normal build is
untouched.
A CI job builds the harnesses under ASan+UBSan+LeakSanitizer and replays each
seed corpus. Replay, not open-ended mutation, keeps the job deterministic and
avoids strjoker's catastrophic backtracking (a data-dependent exponential in
the wildcard matcher, tracked separately). Maintainers run the timed mutation
mode locally for discovery; OSS-Fuzz is the next step.
The corpora carry the reproducers for the four parser bugs these harnesses
found, so the job also pins those regressions. It depends on the fixes in the
preceding commit; merge that first.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
