mirror of
https://github.com/xroche/httrack.git
synced 2026-07-23 17:19:17 +03:00
* Cap the wildcard matcher's recursion depth (#574) strjoker() recursed once per pattern segment, bounded only by the length cap: a hostile filter of 1023 stars reached 2046 frames, ~900KB of stack. That fits Linux's 8MB but not the 1MB a Windows thread gets, so -#test=filterbounds died silently on MSVC x64 and Win32 instead of rejecting the pattern it is meant to reject. Cap the depth at 256 (real filters use fewer than ten). A cut branch does not memoize its failure: the same pair may still match when reached at a shallower depth. The self-test now asserts the depth reached equals the cap, and the .test re-runs it under a 512K stack, which segfaults without the cap. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Take argv as UTF-8 on Windows, not the ANSI codepage (#573) httrack.exe is an MBCS build, so the CRT transcodes the UTF-16 command line down to the machine's ANSI codepage before handing us char **argv. Non-ASCII arguments are lossy, and anything outside that codepage is destroyed: a Cyrillic or CJK URL on a Western-codepage box becomes '?'. Everything else in the engine already treats char* as UTF-8 on Windows - FOPEN, STAT, UNLINK are hts_*_utf8 wrappers converting to UTF-16 at the syscall boundary. argv is the one thing that never got the memo. Decode the real UTF-16 command line at the entry point instead. The manifest's activeCodePage would fix it in one line, but it needs Windows 10 1903+ and is silently ignored below that, which is precisely the population running the codepages that mangle non-Latin URLs today. Windows-only, so no POSIX symbol and no soname change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Make the engine tests pass on Windows Three of them could never pass there, and none of the three was an engine bug: - rcfile wrote .httrackrc, but HTS_HTTRACKRC is "httrackrc" on Windows, so the engine never found it. Write both names. - filelist made a file unreadable with chmod 000, which Windows ignores. Probe whether the mode is enforced instead of assuming, which also subsumes the root special-case. - ftp-line was #ifndef _WIN32 for its socketpair(), though get_ftp_line() itself is portable. Pair over loopback TCP and register it everywhere. The charset self-tests fed raw non-UTF-8 bytes through argv to probe the decoders. Those bytes cannot survive any Windows command line, whatever argv does, so take them as hex - the convention the sniff self-test already uses - and keep the coverage on every platform. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Run the engine test suite on Windows CI The ~90 tests only ever ran on Linux and macOS, so a Windows-only regression was invisible until a user hit it - and the first run of the offline ones found a third of them failing. Drive them from Git Bash against the native MSVC httrack.exe. Two things that silently ruin the run: MSYS rewrites any argument shaped like a POSIX path, and a URL path is shaped exactly like one, so "-#test=mime /a/b.html" reached the engine as "C:/Program Files/Git/a/b.html"; and a suite that degrades to all-skipped would report green having tested nothing, hence the floor on tests actually passed. This subsumes the codec and cache self-tests the workflow ran inline. The .gitattributes pins the test scripts to LF: a converting checkout rewrites them to CRLF and bash then dies on $'\r' on every line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Log the launch banner argv without re-encoding it The banner ran hts_convertStringSystemToUTF8() over each argv element on Windows, which was right when argv came in the ANSI codepage. argv is UTF-8 on every platform now, so that pass double-encoded a non-ASCII argument into mojibake in hts-log.txt. Log the bytes as they are. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Fix the Win32 CLI build: include htscharset.h after winsock2.h htscharset.h pulls in <windows.h>, which drags in the legacy <winsock.h> unless <winsock2.h> was included first. In httrack.c it sat ahead of the net headers, so the x86 build redefined every sockaddr/winsock symbol (C2011). Move it below htslib.h, where winsock2.h is already in. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Fix two more tests the Win32 CI leg exposed Both are test bugs surfaced by running on 32-bit Windows for the first time, not engine bugs: - strsafe drove its overflow through a char[4]. On a 32-bit build that equals sizeof(char*), so htssafe's array-vs-pointer heuristic (sizeof(A) != sizeof(char*), the MSVC path) reads the array as a pointer and skips the bound: the copy then genuinely overflows and crashes instead of aborting cleanly. Size the buffer off the pointer width so the checked path runs everywhere. - idna's malformed-UTF-8 rejection case passed the bad bytes through argv. They cannot survive a Windows command line: the UTF-16 round-trip turns them into valid U+FFFD, which the encoder then accepts. Feed them as hex so the exact bytes reach the bundled punycode encoder, which rejects them identically on every platform. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> --------- Signed-off-by: Xavier Roche <roche@httrack.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
50 lines
1.7 KiB
Bash
Executable File
50 lines
1.7 KiB
Bash
Executable File
#!/bin/bash
|
|
#
|
|
|
|
set -euo pipefail
|
|
|
|
# htssafe.h bounded string operations (driven by 'httrack -#test=strsafe').
|
|
|
|
# Success path: every bounded op (strcpybuff/strcatbuff/strncatbuff/strlcpybuff)
|
|
# must behave correctly. 'run' selects the success path (vs the overflow modes).
|
|
rc=0
|
|
out=$(httrack -#test=strsafe run) || rc=$?
|
|
test "$rc" -eq 0 || exit 1
|
|
test "$out" == "strsafe: OK" || exit 1
|
|
|
|
# Overflow path: an over-capacity write into a sized buffer must be caught by
|
|
# the bounded macro and abort the process, not be silently truncated/completed.
|
|
# Assert the htssafe abort signature specifically, so the test cannot pass for
|
|
# an unrelated reason (e.g. the strsafe test being gone, which prints the
|
|
# registry to stderr and also exits non-zero).
|
|
# the bounded macro aborts (non-zero exit), so don't let set -e trip on it
|
|
err=$(httrack -#test=strsafe overflow "this string is far too long for the buffer" 2>&1) || true
|
|
case "$err" in
|
|
*"strsafe: NOT aborted"*)
|
|
echo "over-capacity write was NOT caught" >&2
|
|
exit 1
|
|
;;
|
|
*"overflow while copying"*) ;;
|
|
*)
|
|
echo "expected htssafe overflow abort, got: $err" >&2
|
|
exit 1
|
|
;;
|
|
esac
|
|
|
|
# Same guarantee for the htsbuff builder. The source is exactly the buffer
|
|
# capacity (6 bytes into a 6-byte buffer), so this also pins the boundary: a
|
|
# '<=' off-by-one in the capacity check would let it through (and print "NOT
|
|
# aborted"). Match the specific htsbuff abort message, not just any assert.
|
|
err=$(httrack -#test=strsafe overflow-buff "abcdef" 2>&1) || true
|
|
case "$err" in
|
|
*"strsafe: NOT aborted"*)
|
|
echo "htsbuff over-capacity write was NOT caught" >&2
|
|
exit 1
|
|
;;
|
|
*"htsbuff append overflow"*) ;;
|
|
*)
|
|
echo "expected htsbuff overflow abort, got: $err" >&2
|
|
exit 1
|
|
;;
|
|
esac
|