Files
httrack/tests/01_engine-strsafe.test
Xavier Roche b026c80bb6 Run the engine test suite on Windows (#577)
* Cap the wildcard matcher's recursion depth (#574)

strjoker() recursed once per pattern segment, bounded only by the length
cap: a hostile filter of 1023 stars reached 2046 frames, ~900KB of stack.
That fits Linux's 8MB but not the 1MB a Windows thread gets, so
-#test=filterbounds died silently on MSVC x64 and Win32 instead of
rejecting the pattern it is meant to reject.

Cap the depth at 256 (real filters use fewer than ten). A cut branch does
not memoize its failure: the same pair may still match when reached at a
shallower depth.

The self-test now asserts the depth reached equals the cap, and the .test
re-runs it under a 512K stack, which segfaults without the cap.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Take argv as UTF-8 on Windows, not the ANSI codepage (#573)

httrack.exe is an MBCS build, so the CRT transcodes the UTF-16 command
line down to the machine's ANSI codepage before handing us char **argv.
Non-ASCII arguments are lossy, and anything outside that codepage is
destroyed: a Cyrillic or CJK URL on a Western-codepage box becomes '?'.

Everything else in the engine already treats char* as UTF-8 on Windows -
FOPEN, STAT, UNLINK are hts_*_utf8 wrappers converting to UTF-16 at the
syscall boundary. argv is the one thing that never got the memo.

Decode the real UTF-16 command line at the entry point instead. The
manifest's activeCodePage would fix it in one line, but it needs Windows
10 1903+ and is silently ignored below that, which is precisely the
population running the codepages that mangle non-Latin URLs today.

Windows-only, so no POSIX symbol and no soname change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Make the engine tests pass on Windows

Three of them could never pass there, and none of the three was an engine
bug:

- rcfile wrote .httrackrc, but HTS_HTTRACKRC is "httrackrc" on Windows,
  so the engine never found it. Write both names.
- filelist made a file unreadable with chmod 000, which Windows ignores.
  Probe whether the mode is enforced instead of assuming, which also
  subsumes the root special-case.
- ftp-line was #ifndef _WIN32 for its socketpair(), though get_ftp_line()
  itself is portable. Pair over loopback TCP and register it everywhere.

The charset self-tests fed raw non-UTF-8 bytes through argv to probe the
decoders. Those bytes cannot survive any Windows command line, whatever
argv does, so take them as hex - the convention the sniff self-test
already uses - and keep the coverage on every platform.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Run the engine test suite on Windows CI

The ~90 tests only ever ran on Linux and macOS, so a Windows-only
regression was invisible until a user hit it - and the first run of the
offline ones found a third of them failing.

Drive them from Git Bash against the native MSVC httrack.exe. Two things
that silently ruin the run: MSYS rewrites any argument shaped like a POSIX
path, and a URL path is shaped exactly like one, so "-#test=mime /a/b.html"
reached the engine as "C:/Program Files/Git/a/b.html"; and a suite that
degrades to all-skipped would report green having tested nothing, hence
the floor on tests actually passed.

This subsumes the codec and cache self-tests the workflow ran inline. The
.gitattributes pins the test scripts to LF: a converting checkout rewrites
them to CRLF and bash then dies on $'\r' on every line.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Log the launch banner argv without re-encoding it

The banner ran hts_convertStringSystemToUTF8() over each argv element on
Windows, which was right when argv came in the ANSI codepage. argv is
UTF-8 on every platform now, so that pass double-encoded a non-ASCII
argument into mojibake in hts-log.txt. Log the bytes as they are.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Fix the Win32 CLI build: include htscharset.h after winsock2.h

htscharset.h pulls in <windows.h>, which drags in the legacy <winsock.h>
unless <winsock2.h> was included first. In httrack.c it sat ahead of the
net headers, so the x86 build redefined every sockaddr/winsock symbol
(C2011). Move it below htslib.h, where winsock2.h is already in.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Fix two more tests the Win32 CI leg exposed

Both are test bugs surfaced by running on 32-bit Windows for the first
time, not engine bugs:

- strsafe drove its overflow through a char[4]. On a 32-bit build that
  equals sizeof(char*), so htssafe's array-vs-pointer heuristic
  (sizeof(A) != sizeof(char*), the MSVC path) reads the array as a
  pointer and skips the bound: the copy then genuinely overflows and
  crashes instead of aborting cleanly. Size the buffer off the pointer
  width so the checked path runs everywhere.

- idna's malformed-UTF-8 rejection case passed the bad bytes through
  argv. They cannot survive a Windows command line: the UTF-16 round-trip
  turns them into valid U+FFFD, which the encoder then accepts. Feed them
  as hex so the exact bytes reach the bundled punycode encoder, which
  rejects them identically on every platform.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:03:40 +02:00

50 lines
1.7 KiB
Bash
Executable File

#!/bin/bash
#
set -euo pipefail
# htssafe.h bounded string operations (driven by 'httrack -#test=strsafe').
# Success path: every bounded op (strcpybuff/strcatbuff/strncatbuff/strlcpybuff)
# must behave correctly. 'run' selects the success path (vs the overflow modes).
rc=0
out=$(httrack -#test=strsafe run) || rc=$?
test "$rc" -eq 0 || exit 1
test "$out" == "strsafe: OK" || exit 1
# Overflow path: an over-capacity write into a sized buffer must be caught by
# the bounded macro and abort the process, not be silently truncated/completed.
# Assert the htssafe abort signature specifically, so the test cannot pass for
# an unrelated reason (e.g. the strsafe test being gone, which prints the
# registry to stderr and also exits non-zero).
# the bounded macro aborts (non-zero exit), so don't let set -e trip on it
err=$(httrack -#test=strsafe overflow "this string is far too long for the buffer" 2>&1) || true
case "$err" in
*"strsafe: NOT aborted"*)
echo "over-capacity write was NOT caught" >&2
exit 1
;;
*"overflow while copying"*) ;;
*)
echo "expected htssafe overflow abort, got: $err" >&2
exit 1
;;
esac
# Same guarantee for the htsbuff builder. The source is exactly the buffer
# capacity (6 bytes into a 6-byte buffer), so this also pins the boundary: a
# '<=' off-by-one in the capacity check would let it through (and print "NOT
# aborted"). Match the specific htsbuff abort message, not just any assert.
err=$(httrack -#test=strsafe overflow-buff "abcdef" 2>&1) || true
case "$err" in
*"strsafe: NOT aborted"*)
echo "htsbuff over-capacity write was NOT caught" >&2
exit 1
;;
*"htsbuff append overflow"*) ;;
*)
echo "expected htsbuff overflow abort, got: $err" >&2
exit 1
;;
esac