mirror of
https://github.com/xroche/httrack.git
synced 2026-07-24 09:39:30 +03:00
b026c80bb6
* Cap the wildcard matcher's recursion depth (#574) strjoker() recursed once per pattern segment, bounded only by the length cap: a hostile filter of 1023 stars reached 2046 frames, ~900KB of stack. That fits Linux's 8MB but not the 1MB a Windows thread gets, so -#test=filterbounds died silently on MSVC x64 and Win32 instead of rejecting the pattern it is meant to reject. Cap the depth at 256 (real filters use fewer than ten). A cut branch does not memoize its failure: the same pair may still match when reached at a shallower depth. The self-test now asserts the depth reached equals the cap, and the .test re-runs it under a 512K stack, which segfaults without the cap. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Take argv as UTF-8 on Windows, not the ANSI codepage (#573) httrack.exe is an MBCS build, so the CRT transcodes the UTF-16 command line down to the machine's ANSI codepage before handing us char **argv. Non-ASCII arguments are lossy, and anything outside that codepage is destroyed: a Cyrillic or CJK URL on a Western-codepage box becomes '?'. Everything else in the engine already treats char* as UTF-8 on Windows - FOPEN, STAT, UNLINK are hts_*_utf8 wrappers converting to UTF-16 at the syscall boundary. argv is the one thing that never got the memo. Decode the real UTF-16 command line at the entry point instead. The manifest's activeCodePage would fix it in one line, but it needs Windows 10 1903+ and is silently ignored below that, which is precisely the population running the codepages that mangle non-Latin URLs today. Windows-only, so no POSIX symbol and no soname change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Make the engine tests pass on Windows Three of them could never pass there, and none of the three was an engine bug: - rcfile wrote .httrackrc, but HTS_HTTRACKRC is "httrackrc" on Windows, so the engine never found it. Write both names. - filelist made a file unreadable with chmod 000, which Windows ignores. Probe whether the mode is enforced instead of assuming, which also subsumes the root special-case. - ftp-line was #ifndef _WIN32 for its socketpair(), though get_ftp_line() itself is portable. Pair over loopback TCP and register it everywhere. The charset self-tests fed raw non-UTF-8 bytes through argv to probe the decoders. Those bytes cannot survive any Windows command line, whatever argv does, so take them as hex - the convention the sniff self-test already uses - and keep the coverage on every platform. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Run the engine test suite on Windows CI The ~90 tests only ever ran on Linux and macOS, so a Windows-only regression was invisible until a user hit it - and the first run of the offline ones found a third of them failing. Drive them from Git Bash against the native MSVC httrack.exe. Two things that silently ruin the run: MSYS rewrites any argument shaped like a POSIX path, and a URL path is shaped exactly like one, so "-#test=mime /a/b.html" reached the engine as "C:/Program Files/Git/a/b.html"; and a suite that degrades to all-skipped would report green having tested nothing, hence the floor on tests actually passed. This subsumes the codec and cache self-tests the workflow ran inline. The .gitattributes pins the test scripts to LF: a converting checkout rewrites them to CRLF and bash then dies on $'\r' on every line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Log the launch banner argv without re-encoding it The banner ran hts_convertStringSystemToUTF8() over each argv element on Windows, which was right when argv came in the ANSI codepage. argv is UTF-8 on every platform now, so that pass double-encoded a non-ASCII argument into mojibake in hts-log.txt. Log the bytes as they are. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Fix the Win32 CLI build: include htscharset.h after winsock2.h htscharset.h pulls in <windows.h>, which drags in the legacy <winsock.h> unless <winsock2.h> was included first. In httrack.c it sat ahead of the net headers, so the x86 build redefined every sockaddr/winsock symbol (C2011). Move it below htslib.h, where winsock2.h is already in. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Fix two more tests the Win32 CI leg exposed Both are test bugs surfaced by running on 32-bit Windows for the first time, not engine bugs: - strsafe drove its overflow through a char[4]. On a 32-bit build that equals sizeof(char*), so htssafe's array-vs-pointer heuristic (sizeof(A) != sizeof(char*), the MSVC path) reads the array as a pointer and skips the bound: the copy then genuinely overflows and crashes instead of aborting cleanly. Size the buffer off the pointer width so the checked path runs everywhere. - idna's malformed-UTF-8 rejection case passed the bad bytes through argv. They cannot survive a Windows command line: the UTF-16 round-trip turns them into valid U+FFFD, which the encoder then accepts. Feed them as hex so the exact bytes reach the bundled punycode encoder, which rejects them identically on every platform. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> --------- Signed-off-by: Xavier Roche <roche@httrack.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
9 lines
342 B
Plaintext
9 lines
342 B
Plaintext
# Resource scripts are Windows tooling input and must stay CRLF: the engine has no
|
|
# encoding guard, so an autocrlf checkout could otherwise flatten them silently.
|
|
*.rc text eol=crlf
|
|
|
|
# The test scripts run under Git Bash on the Windows CI; a CRLF checkout makes
|
|
# bash die on $'\r' on every line of them.
|
|
*.test text eol=lf
|
|
*.sh text eol=lf
|