The Windows suite driver is 140 lines of shell inlined in YAML (#951)

* The Windows suite driver is 140 lines of shell inlined in YAML

Move the "Run the engine test suite" step body to tests/ci-windows-suite.sh,
where shellcheck and shfmt reach it and it can be run by hand. The two
deliberate word-splits in the skip-set compare needed a directive; nothing
else changed, verified by diffing the shfmt-normalized old body against the
new file.

ci_annotate and ci_suite_heartbeat move with it, out of the test library
every test sources on every platform. 171_watchdog-heartbeat.test sources
the driver, which returns early unless run directly.

Closes #948

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Test the driver on this platform, and ask the shell whether it was sourced

The guard read "${BASH_SOURCE[0]}" = "$0", which is only false by accident of
what the caller put in $0: bash -c '. "$0"' <driver> makes the two equal and
falls through into the suite. Ask the shell instead.

172_ci-windows-driver.test pins that case, the bindir contract and the loop's
accounting against a stub bindir, none of which a Windows-only leg proves
before merge.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Silence two shellcheck findings in the new driver test

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Make the driver test survive distcheck's read-only srcdir

cp carries the source's mode over, so under distcheck the neutered testlib
copy came out read-only and the append failed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

.github/workflows/ci.yml

@@ -143,7 +143,7 @@ jobs:
   # so point configure at it; everything else is in the SDK or default paths.
   macos:
     name: build (macOS arm64, clang)
-    runs-on: macos-14
+    runs-on: macos-15
     steps:
       - uses: actions/checkout@v7
         with:
@@ -191,7 +191,7 @@ jobs:
   # temp prefix, then check webhttrack brings up htsserver and serves the UI.
   webhttrack-macos:
     name: webhttrack smoke (macOS arm64)
-    runs-on: macos-14
+    runs-on: macos-15
     steps:
       - uses: actions/checkout@v7
         with:
@@ -247,6 +247,88 @@ jobs:
       - name: Smoke-test webhttrack
         run: bash tests/webhttrack-smoke.sh "$RUNNER_TEMP/inst"
 
+  # The macOS app bundle (#886): assemble it, then prove it still works after being
+  # moved, which is the only thing a .app has to survive that a prefix install does not.
+  macos-app:
+    name: macOS app bundle (arm64)
+    runs-on: macos-15
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          submodules: recursive
+
+      - name: Install build dependencies
+        run: |
+          set -euo pipefail
+          brew install autoconf automake libtool autoconf-archive brotli zstd
+
+      # --disable-shared keeps libhttrack inside the binaries, so the bundle carries no
+      # absolute dylib path back to the staging prefix.
+      - name: Build and install into a temp prefix
+        run: |
+          set -euo pipefail
+          ssl="$(brew --prefix openssl@3)"
+          brewp="$(brew --prefix)"
+          ./bootstrap
+          ./configure CPPFLAGS="-I${ssl}/include -I${brewp}/include" \
+            LDFLAGS="-L${ssl}/lib -L${brewp}/lib" \
+            --prefix="$RUNNER_TEMP/inst" --disable-shared
+          make -j"$(sysctl -n hw.ncpu)"
+          make install
+
+      - name: Assemble HTTrack.app
+        run: |
+          set -euo pipefail
+          make macos-app APP_FLAGS="--prefix $RUNNER_TEMP/inst --out $RUNNER_TEMP"
+          plutil -lint "$RUNNER_TEMP/HTTrack.app/Contents/Info.plist"
+
+      - name: Run it through the bundle stub
+        run: |
+          set -euo pipefail
+          app="$RUNNER_TEMP/HTTrack.app"
+          bash tests/webhttrack-smoke.sh "$app/Contents/Resources" "$app/Contents/MacOS/HTTrack"
+
+      - name: Run it again after moving it
+        run: |
+          set -euo pipefail
+          mkdir -p "$RUNNER_TEMP/moved"
+          mv "$RUNNER_TEMP/HTTrack.app" "$RUNNER_TEMP/moved/"
+          rm -rf "$RUNNER_TEMP/inst"
+          app="$RUNNER_TEMP/moved/HTTrack.app"
+          bash tests/webhttrack-smoke.sh "$app/Contents/Resources" "$app/Contents/MacOS/HTTrack"
+
+      # The load-command check cannot fail while the loader can still reach Homebrew (#901).
+      - name: Run it with Homebrew's libraries out of reach
+        run: |
+          set -euo pipefail
+          app="$RUNNER_TEMP/moved/HTTrack.app"
+          # Without this the hiding below proves nothing: an empty Frameworks passes it.
+          ls "$app/Contents/Frameworks"/libssl.*.dylib >/dev/null
+          # The kegs, not just the opt symlinks: a source build records the Cellar path.
+          hidden=""
+          for keg in openssl@3 brotli zstd; do
+            for d in "$(brew --prefix)/opt/$keg" "$(brew --prefix)/Cellar/$keg"; do
+              if [ -e "$d" ]; then sudo mv "$d" "$d.hidden"; hidden="$hidden $d"; fi
+            done
+          done
+          trap 'for d in $hidden; do sudo mv "$d.hidden" "$d"; done' EXIT
+          test -n "$hidden"
+          bash tests/webhttrack-smoke.sh "$app/Contents/Resources" "$app/Contents/MacOS/HTTrack"
+
+      # No Developer ID here, so Gatekeeper cannot be exercised; an ad-hoc signature
+      # still proves the bundle is well formed enough to sign.
+      - name: Check the bundle is signable
+        run: |
+          set -euo pipefail
+          app="$RUNNER_TEMP/moved/HTTrack.app"
+          # Before --force, which would re-sign away the damage this is meant to catch.
+          for f in "$app/Contents/Resources/bin/"* "$app/Contents/Frameworks/"*.dylib; do
+            file "$f" | grep -q Mach-O || continue
+            codesign --verify --strict "$f"
+          done
+          codesign -s - --force --deep "$app"
+          codesign --verify --strict --verbose=2 "$app"
+
   # Portability/hardening: 32-bit (i386) build on the x86-64 runner via multilib
   # -- no extra hardware. Exercises the 32-bit size_t/pointer ABI, where size
   # and bounds math can truncate or wrap in ways 64-bit never reveals (the axis
@@ -623,9 +705,10 @@ jobs:
         man/makeman.sh
         src/htsbasiccharsets.sh
         src/htsentities.sh
-        src/webhttrack
+        src/webhttrack.in
         tests/*.sh
         tests/*.test
+        tools/macos-app.sh
         tools/mkdeb.sh
     steps:
       - uses: actions/checkout@v7

.github/workflows/windows-build.yml

@@ -167,132 +167,8 @@ jobs:
         working-directory: tests
         timeout-minutes: 45
         run: |
-          set -u
-          bin="$(cygpath -u "$GITHUB_WORKSPACE")/src/${{ matrix.platform }}/${{ matrix.configuration }}"
-          export PATH="$bin:$PATH"
-          command -v httrack >/dev/null || { echo "::error::no httrack.exe in $bin"; exit 1; }
-
-          # httrack.exe is native, so MSYS rewrites any argument shaped like a
-          # POSIX path, and a URL path is shaped exactly like one: "/a/b.html"
-          # reached the engine as "C:/Program Files/Git/a/b.html". Switch that
-          # off, and hand the tests a TMPDIR that is already a Windows path.
-          export MSYS_NO_PATHCONV=1
-          export MSYS2_ARG_CONV_EXCL='*'
-          TMPDIR="$(cygpath -m "$RUNNER_TEMP")"
-          export TMPDIR
-
-          # Mirror what configure hands the suite. LC_ALL sets the codeset MSYS maps
-          # a UTF-8 mirror name onto UTF-16 with, which the intl crawls "test -f".
-          export HTTPS_SUPPORT=yes BROTLI_ENABLED=yes ZSTD_ENABLED=yes
-          export LC_ALL=C.UTF-8
-
-          # A wedged crawl must not eat the job's timeout budget. timeout(1)'s
-          # signals can't reap a native httrack.exe (MSYS signals don't reach it),
-          # so a hang orphaned processes that starved the runner; run_with_timeout
-          # TerminateProcess-es the whole tree. 600s is unchanged: it clears the
-          # 540s a three-pass crawl may legitimately take under local-crawl.sh's
-          # own watchdogs, against a slowest healthy test here of 39s.
-          . ./testlib.sh
-          per_test=600
-
-          # The whole suite must give up before the step timeout above. A cancelled
-          # step keeps neither its log nor the artifacts the later if:always()
-          # steps would upload, so an overrun that ends in a cancel tells us
-          # nothing; failing on our own terms keeps both. Healthy runs take 8-9
-          # min. The check sits between tests, so the step can still reach 25 min
-          # plus one per-test budget, and that worst case stays inside the 45.
-          suite_deadline=1500
-          started=$SECONDS
-
-          # Survives into the artifact even if the tail of the step log does not.
-          progress=suite-progress.log
-          : >"$progress"
-
-          pass=0 fail=0 skip=0 failed="" skipped="" deadline=0
-          # Globbed, not enumerated: a new NNN_engine-*.test or NNN_local-*.test
-          # is picked up automatically instead of silently getting zero coverage.
-          for t in 00_runnable.test *_engine-*.test *_zlib-*.test \
-                   *_local-*.test *_watchdog*.test *_crawl_proxy_https.test \
-                   *_crawl-log-salvage.test; do
-            elapsed=$((SECONDS - started))
-            if [ "$elapsed" -ge "$suite_deadline" ]; then
-              echo "::error::suite deadline: ${elapsed}s elapsed, stopping before $t"
-              echo "DEADLINE before $t after ${elapsed}s" >>"$progress"
-              # Per-test start times, so the slow ones are named rather than guessed.
-              sed 's/^/      /' "$progress"
-              deadline=1
-              break
-            fi
-            echo "RUN $t at ${elapsed}s" >>"$progress"
-            rc=0
-            # Same guard "make check" uses on POSIX, so a wedge is diagnosed the
-            # same way on every platform. It dumps before it kills, which a bare
-            # run_with_timeout cannot: by the time that returns, the tree whose
-            # stack we wanted is already gone.
-            HTTRACK_TEST_TIMEOUT=$per_test bash ./test-timeout.sh "$t" >"$t.log" 2>&1 || rc=$?
-            case "$rc" in
-              0) pass=$((pass + 1)); echo "PASS $t" ;;
-              77) skip=$((skip + 1)) skipped="$skipped $t"; echo "SKIP $t" ;;
-              124)
-                fail=$((fail + 1)) failed="$failed $t"
-                # test-timeout.sh has already written the process list, the stacks
-                # and the killed crawl's own logs into $t.log.
-                echo "FAIL $t (timed out, tree killed)"
-                tail -n 25 "$t.log" | sed 's/^/      /'
-                ;;
-              *)
-                fail=$((fail + 1)) failed="$failed $t"
-                echo "FAIL $t (exit $rc)"
-                # These assert with `test "$(...)" == "..." || exit 1`, which
-                # says nothing at all on failure. Re-run traced, still bounded.
-                run_with_timeout "$per_test" bash -x "$t" >>"$t.log" 2>&1 || true
-                tail -n 25 "$t.log" | sed 's/^/      /'
-                ;;
-            esac
-            echo "$rc $t" >>"$progress"
-            # An orphaned native httrack.exe spins and starves the runner, which
-            # is how this job dies with "lost communication" rather than a plain
-            # timeout. Clear them between tests and name whoever leaked them.
-            reap_leftover_processes "$t" | tee -a "$progress"
-          done
-          echo "ran=$((pass + fail + skip)) pass=$pass fail=$fail skip=$skip" |
-            tee -a "$GITHUB_STEP_SUMMARY"
-
-          # Every gate here exits 77, so an all-skipped suite would report green having
-          # tested nothing: pin the skips, and floor the passes in case the glob empties.
-          # One name per line, so two branches each appending one don't collide on the
-          # same line; compared as a sorted set below, so glob discovery order can't
-          # cause a false mismatch either.
-          # footer-overflow and purge-longpath skip on Windows (need a path past MAX_PATH);
-          # crange pending #581;
-          # webdav-default and webdav-mime need a reapable background listener, which MSYS cannot give them;
-          # badmtime needs a filesystem that stores an mtime past gmtime's range;
-          # single-file ends on a GUI half needing htsserver, which this job does not build;
-          # update-304-leak needs a LeakSanitizer build, which MSVC has no equivalent of;
-          # crash-symbolize needs backtrace(), which Windows has no equivalent of.
-          expected_skips="01_engine-footer-overflow.test
-          100_local-purge-longpath.test
-          114_local-update-304-leak.test
-          120_local-proxytrack-webdav-default.test
-          48_local-crange-memresume.test
-          71_local-crange-repaircache.test
-          79_local-proxytrack-webdav-mime.test
-          80_engine-crash-symbolize.test
-          88_local-proxytrack-badmtime.test
-          94_local-single-file.test"
-          # First, or the deadline reads as an unexplained shortfall in the gates below.
-          [ "$deadline" -eq 0 ] || { echo "::error::suite did not finish within ${suite_deadline}s"; exit 1; }
-          [ "$pass" -ge 90 ] || { echo "::error::only $pass tests passed ($skip skipped)"; exit 1; }
-          # Word-split on whitespace (space-joined $skipped, newline-joined
-          # expected_skips both work) and sort, so the compare is a set, not a string.
-          got=$(printf '%s\n' $skipped | sort)
-          want=$(printf '%s\n' $expected_skips | sort)
-          if [ "$got" != "$want" ]; then
-            echo "::error::skip set changed from expected; - missing, + newly skipped"
-            diff -u <(echo "$want") <(echo "$got") | tail -n +3 | sed 's/^/      /'
-            exit 1
-          fi
-          [ "$fail" -eq 0 ] || { echo "::error::failing:$failed"; exit 1; }
+          bash ./ci-windows-suite.sh \
+            "$(cygpath -u "$GITHUB_WORKSPACE")/src/${{ matrix.platform }}/${{ matrix.configuration }}"
 
       - name: Upload the test logs
         if: always()

.gitignore

@@ -24,6 +24,8 @@ Makefile.in
 /config.log
 /config.status
 /stamp-h1
+# src/webhttrack.in is the source; an in-tree build generates this one (#887).
+/src/webhttrack
 Makefile
 .deps/
 .libs/

Makefile.am

@@ -6,13 +6,22 @@ ACLOCAL_AMFLAGS = -I m4
 EXTRA_DIST = INSTALL.Linux \
 		gpl-fr.txt license.txt greetings.txt history.txt \
 		httrack-doc.html lang.def README.md tools/mkdeb.sh \
+		tools/macos-app.sh tools/Info.plist.in tools/HTTrack.icns \
 		bootstrap build.sh
 
 # Build the signed Debian packages from a clean source export. Pass the signing
 # key and other options through DEB_FLAGS, e.g.:
 #   make deb DEB_FLAGS="--key BB71C7E6CB1AD8FAF53FE42A60C3AA7180598EFB"
-# See tools/mkdeb.sh --help for all options.
+# See tools/mkdeb.sh --help for all options. Not $(SHELL): mkdeb.sh is bash, and dash chokes (#891).
 DEB_FLAGS =
 deb:
-	$(SHELL) $(top_srcdir)/tools/mkdeb.sh $(DEB_FLAGS)
+	$(BASH_SHELL) $(top_srcdir)/tools/mkdeb.sh $(DEB_FLAGS)
 .PHONY: deb
+
+# Assemble the macOS application bundle from an installed prefix, e.g.
+#   make macos-app APP_FLAGS="--prefix /tmp/inst --out /tmp"
+APP_FLAGS =
+macos-app:
+	$(SHELL) $(top_srcdir)/tools/macos-app.sh --plist tools/Info.plist \
+		--icon $(top_srcdir)/tools/HTTrack.icns $(APP_FLAGS)
+.PHONY: macos-app

configure.ac

@@ -38,6 +38,70 @@ VERSION_INFO="3:7:0"
 AM_MAINTAINER_MODE
 AC_USE_SYSTEM_EXTENSIONS
 
+# A real bash, for "make deb" and the test harness. Not searched into BASH: bash presets
+# that to its own path, and macOS /bin/sh is a bash, so the macro never searches (#895).
+# BASH_SHELL isn't preset the way BASH is, so AC_ARG_VAR needs no guard. Kept ahead of the
+# compiler probes so a bad override dies before them.
+AC_ARG_VAR([BASH_SHELL], [path to a real (non-POSIX-mode) bash])
+# AC_PATH_PROGS drops a relative override and searches instead, which loses the user's intent.
+# Nothing quotes $(BASH_SHELL) in the Makefiles, and quoting could not save it anyway: make
+# splits on whitespace, expands '$' and treats '#' as a comment before any shell sees it.
+case $BASH_SHELL in
+*[[[:space:]]]* | *'#'* | *'$'* | *'`'* | *'\'* | *'"'* | *"'"* | *';'* | *'&'* | *'|'* | \
+*'<'* | *'>'* | *'('* | *')'* | *'*'* | *'?'* | *'@<:@'* | *'@:>@'* | *'{'* | *'}'*)
+  AC_MSG_ERROR([BASH_SHELL must not contain shell or make metacharacters, got: $BASH_SHELL]) ;;
+'' | [[\\/]]* | ?:[[\\/]]*) ;;
+*) AC_MSG_ERROR([BASH_SHELL must be an absolute path, got: $BASH_SHELL]) ;;
+esac
+hts_bash_override=$BASH_SHELL
+AC_PATH_PROGS([BASH_SHELL], [bash], [/bin/bash])
+
+# An absolute override is taken verbatim, so BASH_SHELL=/bin/sh would put #895 back and only
+# surface at "make check" or "make deb" (#908). What we found ourselves is only a warning:
+# a box with no bash still builds, it just cannot run those two.
+AC_MSG_CHECKING([whether $BASH_SHELL is a bash outside POSIX mode])
+hts_bash_why=
+hts_bash_env=no
+# AS_EXECUTABLE_P, not "test -x": the PATH search above already demands a regular file, and
+# bash blocks forever reading a FIFO it failed to exec, so -x alone hangs configure (#922).
+if ! AS_EXECUTABLE_P(["$BASH_SHELL"]); then
+  hts_bash_why="not an executable regular file"
+elif test -z "$("$BASH_SHELL" -c 'echo "${BASH_VERSINFO[[0]]}"' 2>/dev/null)"; then
+  # Not BASH_VERSION: that is an ordinary variable, so any shell echoes back a spoofed one.
+  hts_bash_why="not a bash: it reports no BASH_VERSINFO"
+else
+  # sh-mode bash reports a version too, so only SHELLOPTS tells the two apart.
+  case $("$BASH_SHELL" -c 'echo ":$SHELLOPTS:"' 2>/dev/null) in
+  *:posix:*)
+    hts_bash_why="a bash in POSIX sh-mode"
+    # POSIXLY_CORRECT and an exported SHELLOPTS do that to every bash on the box, so no path
+    # can pass and blaming this one would send the user hunting for another. Reading them
+    # here would not do: configure puts its own shell in posix mode, which sets both.
+    case $(env -u POSIXLY_CORRECT -u SHELLOPTS "$BASH_SHELL" -c 'echo ":$SHELLOPTS:"' 2>/dev/null) in
+    '' | *:posix:*) ;; # no "env -u", or posix whatever the environment: blame the path
+    *) hts_bash_env=yes ;;
+    esac
+    ;;
+  esac
+fi
+if test -z "$hts_bash_why"; then
+  AC_MSG_RESULT([yes])
+else
+  AC_MSG_RESULT([no])
+  hts_bash_msg="POSIXLY_CORRECT or SHELLOPTS forces every bash into POSIX sh-mode, $BASH_SHELL included. Clear them for configure and for make, which hands them to make check and make deb: env -u POSIXLY_CORRECT -u SHELLOPTS ..."
+  if test "$hts_bash_env" = yes; then
+    if test -n "$hts_bash_override"; then
+      AC_MSG_ERROR([$hts_bash_msg])
+    fi
+    AC_MSG_WARN([$hts_bash_msg])
+  else
+    if test -n "$hts_bash_override"; then
+      AC_MSG_ERROR([BASH_SHELL=$BASH_SHELL is $hts_bash_why])
+    fi
+    AC_MSG_WARN([no usable bash found: $BASH_SHELL is $hts_bash_why. "make check" and "make deb" need one; pass BASH_SHELL=/path/to/bash])
+  fi
+fi
+
 AC_PROG_CC
 AM_PROG_CC_C_O
 m4_warn([obsolete],
@@ -48,14 +112,13 @@ m4_warn([obsolete],
 # script's behavior did not change.  They are probably safe to remove.
 AC_CHECK_INCLUDES_DEFAULT
 AC_PROG_EGREP
+# $(SED) substitutes $(datadir) into src/webhttrack
+AC_PROG_SED
 
 LT_INIT
 AC_PROG_LN_S
 LT_INIT
 
-# bash, used to run the test scripts (see tests/Makefile.am TEST_LOG_COMPILER)
-AC_PATH_PROGS([BASH], [bash], [/bin/bash])
-
 # Export LD_LIBRARY_PATH name or equivalent.
 AC_SUBST(SHLIBPATH_VAR,$shlibpath_var)
 
@@ -100,6 +163,9 @@ AX_CHECK_COMPILE_FLAG([-fstack-protector-strong], [DEFAULT_CFLAGS="$DEFAULT_CFLA
 	[AX_CHECK_COMPILE_FLAG([-fstack-protector], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstack-protector"], [], [-Werror])], [-Werror])
 AX_CHECK_COMPILE_FLAG([-fstack-clash-protection], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstack-clash-protection"], [], [-Werror])
 AX_CHECK_COMPILE_FLAG([-fcf-protection], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fcf-protection"], [], [-Werror])
+# backtrace() unwinds through these; armhf, unlike amd64/arm64, defaults them off
+# and printed frameless crash reports.
+AX_CHECK_COMPILE_FLAG([-fasynchronous-unwind-tables], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fasynchronous-unwind-tables"], [], [-Werror])
 # No --discard-all: it drops the local symbols naming every static function, so
 # a trace misattributes them to the nearest surviving global. Costs 0.6% size.
 AX_CHECK_LINK_FLAG([-Wl,--no-undefined], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,--no-undefined"])
@@ -131,6 +197,11 @@ AC_SUBST([LDFLAGS_PIE])
 # Ties a crash trace from a stripped build back to its separate debug symbols.
 AX_CHECK_LINK_FLAG([-Wl,--build-id], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,--build-id"])
 
+# tools/macos-app.sh rewrites load commands to @rpath, which needs header room ld64 does
+# not leave by default; the probe fails on GNU ld, so this self-gates on Darwin.
+AX_CHECK_LINK_FLAG([-Wl,-headerpad_max_install_names],
+	[DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,-headerpad_max_install_names"])
+
 ### Check for -fvisibility=hidden support
 gl_VISIBILITY
 AM_CFLAGS="$AM_CFLAGS $CFLAG_VISIBILITY"
@@ -360,5 +431,6 @@ html/Makefile
 libtest/Makefile
 tests/Makefile
 fuzz/Makefile
+tools/Info.plist
 ])
 AC_OUTPUT

fuzz/Makefile.am

@@ -2,7 +2,7 @@
 if FUZZERS
 noinst_PROGRAMS = fuzz-charset fuzz-meta fuzz-idna fuzz-entities \
 	fuzz-unescape fuzz-filters fuzz-url fuzz-header fuzz-cachendx \
-	fuzz-htsparse fuzz-singlefile fuzz-sitemap
+	fuzz-htsparse fuzz-singlefile fuzz-sitemap fuzz-arc
 endif
 
 AM_CPPFLAGS = \
@@ -30,6 +30,11 @@ fuzz_htsparse_SOURCES = fuzz-htsparse.c fuzz.h
 fuzz_singlefile_SOURCES = fuzz-singlefile.c fuzz.h
 fuzz_sitemap_SOURCES = fuzz-sitemap.c fuzz.h
 
+# proxytrack does not link libhttrack, so its store compiles into the harness;
+# coucal, minizip and md5 still come from the static libhttrack above.
+fuzz_arc_SOURCES = fuzz-arc.c fuzz.h $(top_srcdir)/src/proxy/store.c
+fuzz_arc_CPPFLAGS = $(AM_CPPFLAGS) -DZLIB_CONST
+
 # List corpus files explicitly: automake does not expand EXTRA_DIST globs.
 EXTRA_DIST = README.md run-fuzzers.sh \
 	corpus/charset/utf8.txt corpus/charset/latin1.txt corpus/charset/sjis.txt \
@@ -55,4 +60,6 @@ EXTRA_DIST = README.md run-fuzzers.sh \
 	corpus/singlefile/srcset.html corpus/singlefile/rawtext.html \
 	corpus/singlefile/malformed.html corpus/singlefile/many-attrs.html \
 	corpus/sitemap/urlset.xml corpus/sitemap/sitemapindex.xml \
-	corpus/sitemap/truncated.xml corpus/sitemap/urlset.xml.gz
+	corpus/sitemap/truncated.xml corpus/sitemap/urlset.xml.gz \
+	corpus/arc/roundtrip.arc corpus/arc/truncated.arc \
+	corpus/arc/regress-null-body.arc

fuzz/README.md

@@ -13,3 +13,5 @@ bash /path/to/httrack/fuzz/run-fuzzers.sh fuzz 60   # 60s per target
 ```
 
 Run one target by hand: `fuzz/fuzz-url -max_total_time=300 corpusdir fuzz/corpus/url`. Seed corpora live in `corpus/<target>/`; a crash reproducer is replayed with `fuzz/fuzz-url crash-file`.
+
+`fuzz-arc` is the odd one out: it drives proxytrack's `.arc` reader the way `--convert` does, through a temp file rather than a buffer, and it compiles `src/proxy/store.c` into the harness because proxytrack does not link libhttrack. Both readers and the writer print to stderr on malformed input, so pass `-close_fd_mask=2` for anything longer than a corpus replay.

fuzz/corpus/arc/regress-null-body.arc

@@ -0,0 +1,9 @@
+filedesc://t.arc 0.0.0.0 20250101000000 text/plain 200 - - 0 t.arc 9
+2 0 test
+
+
+http://example.com/p.html 0.0.0.0 20250101000000 text/html -1 - - 0 t.arc 77
+HTTP/1.1 -1 Broken
+Content-Type: text/html
+Content-Length: 10
+

fuzz/corpus/arc/roundtrip.arc

@@ -0,0 +1,11 @@
+filedesc://t.arc 0.0.0.0 20250101000000 text/plain 200 - - 0 t.arc 9
+2 0 test
+
+
+http://example.com/p.html 0.0.0.0 20250101000000 text/html 200 - - 0 t.arc 120
+HTTP/1.1 200 OK
+Content-Type: text/html
+Last-Modified: Wed, 01 Jan 2025 00:00:00 GMT
+Content-Length: 10
+
+BODYMARKER

fuzz/corpus/arc/truncated.arc

@@ -0,0 +1,9 @@
+filedesc://t.arc 0.0.0.0 20250101000000 text/plain 200 - - 0 t.arc 9
+2 0 test
+
+
+http://example.com/q.html 0.0.0.0 20250101000000 text/html 200 - - 0 t.arc 2000000000
+HTTP/1.1 200 OK
+Content-Type: text/html
+
+HI

fuzz/fuzz-arc.c

@@ -0,0 +1,106 @@
+/* ------------------------------------------------------------ */
+/*
+HTTrack Website Copier, Offline Browser for Windows and Unix
+Copyright (C) 2026 Xavier Roche and other contributors
+
+SPDX-License-Identifier: GPL-3.0-or-later
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+Ethical use: we kindly ask that you NOT use this software to harvest email
+addresses or to collect any other private information about people. Doing so
+would dishonor our work and waste the many hours we have spent on it.
+
+Please visit our Website: http://www.httrack.com
+*/
+
+/* Fuzz proxytrack's .arc reader the way `--convert` drives it: the record loop
+   seeks on lengths read from the file, and every entry reaches a writer. */
+#include "fuzz.h"
+
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+#include "coucal.h"
+#include "proxy/store.h"
+
+/* An .arc past this size says nothing a smaller one cannot. */
+#define FUZZ_ARC_MAXSIZE (1024 * 1024)
+
+static char arc_dir[256];
+static char arc_in[sizeof(arc_dir) + sizeof("/in.arc")];
+static char arc_out[sizeof(arc_dir) + sizeof("/out.arc")];
+
+static void fuzz_arc_cleanup(void) {
+  (void) unlink(arc_in);
+  (void) unlink(arc_out);
+  (void) rmdir(arc_dir);
+}
+
+/* proxytrack's main() installs one; without it coucal logs stats per free */
+static void fuzz_arc_coucal_log(coucal_opaque arg, coucal_loglevel level,
+                                const char *format, va_list args) {
+  (void) arg;
+  (void) level;
+  (void) format;
+  (void) args;
+}
+
+/* PT_GetType() picks the format from the extension, so the names end in .arc */
+static int fuzz_arc_setup(void) {
+  if (arc_in[0] == '\0') {
+    const char *const tmp = getenv("TMPDIR");
+
+    coucal_set_global_assert_handler(fuzz_arc_coucal_log, NULL);
+
+    snprintf(arc_dir, sizeof(arc_dir), "%s/fuzz-arc-XXXXXX",
+             tmp != NULL && *tmp != '\0' ? tmp : "/tmp");
+    if (mkdtemp(arc_dir) == NULL) {
+      return -1;
+    }
+    snprintf(arc_out, sizeof(arc_out), "%s/out.arc", arc_dir);
+    snprintf(arc_in, sizeof(arc_in), "%s/in.arc", arc_dir);
+    atexit(fuzz_arc_cleanup);
+  }
+  return 0;
+}
+
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
+  PT_Indexes indexes;
+  FILE *fp;
+  hts_boolean written;
+
+  if (size > FUZZ_ARC_MAXSIZE || fuzz_arc_setup() != 0) {
+    return 0;
+  }
+  if ((fp = fopen(arc_in, "wb")) == NULL) {
+    return 0;
+  }
+  written = fwrite(data, 1, size, fp) == size ? HTS_TRUE : HTS_FALSE;
+  if (fclose(fp) != 0 || !written) {
+    return 0;
+  }
+
+  indexes = PT_New();
+  if (indexes != NULL) {
+    if (PT_AddIndex(indexes, arc_in) > 0) {
+      /* the writer reads back every entry the loader indexed */
+      (void) PT_SaveCache(indexes, arc_out);
+    }
+    PT_Delete(indexes);
+  }
+  return 0;
+}

fuzz/fuzz.h

@@ -38,7 +38,7 @@ Please visit our Website: http://www.httrack.com
 #include "htsbase.h"
 
 /* Heap NUL-terminated copy of the fuzzer input, so ASan bounds every read. */
-static char *fuzz_strdup(const uint8_t *data, size_t size) {
+HTS_UNUSED static char *fuzz_strdup(const uint8_t *data, size_t size) {
   char *s = malloct(size + 1);
 
   memcpy(s, data, size);

history.txt

@@ -128,7 +128,7 @@ This file lists all changes and fixes that have been made for HTTrack
 + Fixed: report why a -%L URL list could not be loaded (#49)
 + Changed: multiple internal hardening, build and CI improvements
 
-.49-9
+3.49-9
 + Fixed: file-type detection from the Content-Type header: trust a declared type over a binary URL extension, honor --assume under the delayed type check, and keep a known extension against a bogus or empty Content-Type (#267, #29, #56)
 + Fixed: an uninitialized-buffer read when the Content-Type is empty (#411)
 + Fixed: restored C++ source-compatibility of the installed headers so reverse dependencies (httraqt) build again (#413)

html/Makefile.am

@@ -11,6 +11,10 @@ WebPixmapdir = $(datadir)/pixmaps
 WebIcon16x16dir = $(datadir)/icons/hicolor/16x16/apps
 WebIcon32x32dir = $(datadir)/icons/hicolor/32x32/apps
 WebIcon48x48dir = $(datadir)/icons/hicolor/48x48/apps
+WebIcon64x64dir = $(datadir)/icons/hicolor/64x64/apps
+WebIcon128x128dir = $(datadir)/icons/hicolor/128x128/apps
+WebIcon256x256dir = $(datadir)/icons/hicolor/256x256/apps
+WebIconScalabledir = $(datadir)/icons/hicolor/scalable/apps
 VFolderEntrydir = $(prefix)/share/applications
 MetaInfodir = $(datadir)/metainfo
 
@@ -26,17 +30,20 @@ HelpHtmlimages_DATA = $(srcdir)/images/*
 HelpHtmlTxt_DATA = ../greetings.txt ../history.txt ../license.txt
 WebHtml_DATA = $(srcdir)/server/*.html $(srcdir)/server/*.js $(srcdir)/server/*.css
 WebHtmlimages_DATA = $(srcdir)/server/images/*
-# note: converted & normalized by
-# ico2xpm favicon.ico -o httrack.xpm
-# mogrify -format xpm -map /usr/share/doc/menu/examples/cmap.xpm httrack.xpm
+# Generated from the brand master by gen/export.py. Asserted by tests/160.
 WebPixmap_DATA = $(srcdir)/server/div/*.xpm
 WebIcon16x16_DATA = $(srcdir)/server/div/16x16/*.png
 WebIcon32x32_DATA = $(srcdir)/server/div/32x32/*.png
 WebIcon48x48_DATA = $(srcdir)/server/div/48x48/*.png
+WebIcon64x64_DATA = $(srcdir)/server/div/64x64/*.png
+WebIcon128x128_DATA = $(srcdir)/server/div/128x128/*.png
+WebIcon256x256_DATA = $(srcdir)/server/div/256x256/*.png
+WebIconScalable_DATA = $(srcdir)/server/div/scalable/*.svg
 VFolderEntry_DATA = $(srcdir)/server/div/*.desktop
 MetaInfo_DATA = $(srcdir)/server/div/*.metainfo.xml
 
 EXTRA_DIST = $(HelpHtml_DATA) $(HelpHtmlimg_DATA) $(HelpHtmlimages_DATA) \
 	$(HelpHtmldiv_DATA) $(WebHtml_DATA) $(WebHtmlimages_DATA) \
 	$(WebPixmap_DATA) $(WebIcon16x16_DATA) $(WebIcon32x32_DATA) $(WebIcon48x48_DATA) \
-	$(VFolderEntry_DATA) $(MetaInfo_DATA)
+	$(WebIcon64x64_DATA) $(WebIcon128x128_DATA) $(WebIcon256x256_DATA) \
+	$(WebIconScalable_DATA) $(VFolderEntry_DATA) $(MetaInfo_DATA)

html/abuse.html

@@ -16,7 +16,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 

html/cache.html

@@ -16,7 +16,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 

html/changes.html

@@ -16,7 +16,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 

html/cmdguide.html

@@ -16,7 +16,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 

html/contact.html

@@ -26,7 +26,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 

html/dev.html

@@ -16,7 +16,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 

html/doc.css

@@ -57,13 +57,21 @@ body {
 	padding: .6rem 0 0;
 }
 
-.masthead img { display: block; max-width: 100%; height: auto; }
+/* Take back the whitespace the SVG carries around the letters, restoring the old bitmap's spacing. */
+.masthead img {
+	display: block;
+	max-width: 100%;
+	height: auto;
+	margin: -1.7px 0 -1.1px;
+}
 
 .masthead .tagline {
 	background: #000;
 	color: #fff;
 	font-weight: bold;
-	padding: .25rem .4rem;
+	/* A one-line label has no use for the body's prose leading. */
+	line-height: 1;
+	padding: .3rem .4rem;
 	margin-top: .4rem;
 }
 
@@ -72,14 +80,14 @@ body {
 	background: var(--panel);
 	border-bottom: 6px solid #000;
 	padding: 1.5rem;
-	/* The rings the 2007 pages carried. The image has the light panel colour baked
-	   into it, so it is dropped rather than inverted in dark mode. */
-	background-image: url(images/bg_rings.gif);
+	/* The rings the 2007 pages carried. Transparent, so it rides on --panel. */
+	background-image: url(images/bg_rings.svg);
 	background-repeat: no-repeat;
 	background-position: top right;
 }
 
 @media (prefers-color-scheme: dark) {
+	/* Rings dropped, not inverted: their lavender is a light-panel tone. */
 	.wrap { background-image: none; }
 
 	/* The wordmark is black on transparent, and all but vanishes on the dark field. */

html/faq.html

@@ -16,7 +16,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 

html/fcguide.html

@@ -16,7 +16,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 

html/filters.html

@@ -16,7 +16,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 

html/guide.html

@@ -16,7 +16,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 
@@ -66,7 +66,7 @@ the screenshots and notes follow along.</p>
 
 <div class="platforms" role="group" aria-label="Choose your version">
 	<button type="button" data-platform="win" aria-pressed="false">WinHTTrack <small>Windows</small></button>
-	<button type="button" data-platform="web" aria-pressed="false">WebHTTrack <small>Linux and Unix</small></button>
+	<button type="button" data-platform="web" aria-pressed="false">WebHTTrack <small>Linux, macOS and Unix</small></button>
 	<button type="button" data-platform="droid" aria-pressed="false">HTTrack <small>Android</small></button>
 </div>
 
@@ -125,6 +125,11 @@ data-for="droid">Tap <b>Next</b></span> to create a project, or open one you alr
 <p class="note" data-for="web">The language dropdown starts blank on purpose: its first entry
 means "leave the interface as it is". Pick a language only if you want to change it.</p>
 
+<p class="note" data-for="web">On macOS, <code>brew install httrack</code> installs WebHTTrack
+alongside the command line tool. Open <b>HTTrack.app</b> if you have it, or run
+<code>webhttrack</code> in a terminal; either way the interface opens in your browser and
+behaves exactly as described here.</p>
+
 <h2 id="step-project">2. Name the project</h2>
 
 <p>A project is one mirror: its name becomes the folder your files land in, so give it something

html/httrack.css

@@ -37,7 +37,7 @@ a:active  { text-decoration: underline; }
 	border-bottom: 6px solid #000;
 	padding: 10px;  padding-top: 20px;
 	line-height: 1.65em;
-	background-image: url(images/bg_rings.gif);
+	background-image: url(images/bg_rings.svg);
 	background-repeat: no-repeat;
 	background-position: top right;
 	}

html/images/bg_rings.svg

@@ -0,0 +1,8 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="501" height="456" viewBox="0 0 501 456">
+<title>HTTrack rings</title>
+<!-- Transparent, unlike the GIF this replaces: the panel's own #ccd shows through. -->
+<g fill="#b9b9d0" fill-rule="evenodd">
+<path d="M614.59 346.3A323.36 197.55 21.16 1 0 11.46 112.89A323.36 197.55 21.16 1 0 614.59 346.3 ZM590.57 315.66A262.58 160.99 22.18 1 0 104.26 117.43A262.58 160.99 22.18 1 0 590.57 315.66 Z"/>
+<path d="M578.08 292.16A185.6 113.57 21.23 1 0 232.08 157.73A185.6 113.57 21.23 1 0 578.08 292.16 ZM564.4 274.56A150.77 92.55 22.26 1 0 285.34 160.32A150.77 92.55 22.26 1 0 564.4 274.56 Z"/>
+</g>
+</svg>

html/index.html

@@ -13,7 +13,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 

html/library.html

@@ -16,7 +16,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 

html/plug.html

@@ -16,7 +16,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 

html/plug_330.html

@@ -16,7 +16,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 

html/scripting.html

@@ -16,7 +16,7 @@
 <a class="skip" href="#main">Skip to content</a>
 
 <header class="masthead">
-	<img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier">
+	<img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier">
 	<div class="tagline">Open Source offline browser</div>
 </header>
 

html/server/about.html

@@ -28,7 +28,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/addurl.html

@@ -64,7 +64,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="3" class="tableWidth">

html/server/div/com.httrack.WebHTTrack.metainfo.xml

@@ -49,7 +49,72 @@
     </screenshot>
   </screenshots>
   <content_rating type="oars-1.1"/>
+  <!-- Newest first; tests/01_engine-version-macros.test enforces it. -->
   <releases>
-    <release version="3.49.8" date="2026-06-07"/>
+    <release version="3.49.15" date="2026-07-30">
+      <description>
+        <ul>
+          <li>Save a page and everything it needs as one self-contained file</li>
+          <li>Read a site's sitemap, so pages nothing links to are still found</li>
+          <li>See what a new pass added, updated or removed</li>
+          <li>Updating a mirror no longer destroys a good local copy when a download fails or is interrupted</li>
+          <li>In the web interface, options ticked by default can be un-ticked again, the size limit applies to the whole site, and the finished mirror opens from its link</li>
+        </ul>
+      </description>
+    </release>
+    <release version="3.49.14" date="2026-07-24">
+      <description>
+        <ul>
+          <li>Save a mirror as a web archive, indexed and packaged for replay</li>
+          <li>Choose what the page footer says</li>
+          <li>A mirror saved to a folder with accented characters in its name lands in the right place</li>
+        </ul>
+      </description>
+    </release>
+    <release version="3.49.13" date="2026-07-18">
+      <description>
+        <ul>
+          <li>Mirror through a SOCKS5 proxy</li>
+          <li>Files of 2 GB and over are handled correctly on Windows and on 32-bit systems</li>
+          <li>The web interface offers options that used to be command-line only, among them a cookies file and a pause between downloads</li>
+        </ul>
+      </description>
+    </release>
+    <release version="3.49.12" date="2026-07-10">
+      <description>
+        <p>Links carrying accented characters are followed again, a page wrongly labelled as compressed is no longer lost, and a new option explains why a given address is kept or skipped.</p>
+      </description>
+    </release>
+    <release version="3.49.11" date="2026-07-05">
+      <description>
+        <ul>
+          <li>Audio and video sources are mirrored along with the page</li>
+          <li>A site's robots.txt is followed more closely, including its Allow rules and wildcards</li>
+          <li>A time limit now stops a slow download instead of waiting for it to end</li>
+        </ul>
+      </description>
+    </release>
+    <release version="3.49.10" date="2026-06-28">
+      <description>
+        <ul>
+          <li>Start from a cookies file, and space downloads out with a random pause</li>
+          <li>Ignore chosen query parameters when naming saved files</li>
+          <li>Resuming a partial download no longer duplicates bytes</li>
+        </ul>
+      </description>
+    </release>
+    <release version="3.49.9" date="2026-06-21">
+      <description>
+        <p>Saved files get the right type when the server and the address disagree about it.</p>
+      </description>
+    </release>
+    <release version="3.49.8" date="2026-06-20">
+      <description>
+        <ul>
+          <li>Secure downloads can go through a plain web proxy</li>
+          <li>Every size of a responsive image is fetched</li>
+        </ul>
+      </description>
+    </release>
   </releases>
 </component>

html/server/div/httrack16x16.xpm

@@ -1,25 +1,37 @@
 /* XPM */
-static char *httrack__[] = {
+static char *httrack16x16[] = {
 /* columns rows colors chars-per-pixel */
-"16 16 3 1",
-"  c #000000000000",
-". c #000000008080",
-"X c #808080808080",
+"16 16 15 1 ",
+"  c #0D0D10",
+". c #191921",
+"X c #2A2A38",
+"o c #363647",
+"O c #424257",
+"+ c #53536E",
+"@ c #717197",
+"# c #7E7EA8",
+"$ c #8585B1",
+"% c #9898CB",
+"& c #9999CC",
+"* c #9A9ACE",
+"= c #9696C9",
+"- c #9C9CD0",
+"; c #9999CC",
 /* pixels */
-"X.   XXXXXX   .X",
-"X.   XXXXXX   .X",
-"X.   XXXXXX   .X",
-"X.   XXXXXX   .X",
-"X.   XXXXXX   .X",
-"X.   XXXXXX   .X",
-"X.            .X",
-"X.            .X",
-"X.            .X",
-"X.   XXXXXX   .X",
-"X.   XXXXXX   .X",
-"X.   XXXXXX   .X",
-"X.   XXXXXX   .X",
-"X.   XXXXXX   .X",
-"X.   XXXXXX   .X",
-"X.   XXXXXX   .X"
+"&&&&&&&&&%%&&&&&",
+"&&&&&&&&%%%%&&&&",
+"&&&&&&&&%%%%&&&&",
+"&%%&&&%%%%%%%%%&",
+"%+o$&%+o@ooooo+%",
+"%o $-%X @X.  .O%",
+"%o @$$X $=@ o$=&",
+"%o  .   $-# o&&&",
+"%o oOo. $-# o%&&",
+"%o $-%X $-# o%&&",
+"%X $&%X $-# o%&&",
+"%+o$&%+o$&$o+%&&",
+"%%%&&&%%&&&%%&&&",
+"%%&&&&&&&&&&&&&&",
+"%&&&&&&&&&&&&&&&",
+"&&&&&&&&&&&&&&&&"
 };

html/server/div/httrack32x32.xpm

@@ -1,45 +1,55 @@
 /* XPM */
-static char *httrack__[] = {
+static char *httrack32x32[] = {
 /* columns rows colors chars-per-pixel */
-"32 32 7 1",
-"  c #040404040404",
-". c #0C0C0C0C0C0C",
-"X c #161616161616",
-"o c #333333333333",
-"O c #333333336666",
-"+ c #666666669999",
-"@ c #99999999CCCC",
+"32 32 17 1 ",
+"  c #060606",
+". c #1C1C25",
+"X c #292935",
+"o c #3D3D50",
+"O c #46465D",
+"+ c #595976",
+"@ c #727297",
+"# c #7D7DA6",
+"$ c #8E8EBD",
+"% c #8686B3",
+"& c #8686B3",
+"* c #9999CC",
+"= c #9A9ACD",
+"- c #9A9ACE",
+"; c #9B9BCF",
+": c #9C9CD0",
+"> c #9999CD",
 /* pixels */
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       OOOOOOOOOOOo       @@@",
-"@@+                          @@@",
-"@@+                          @@@",
-"@@+                          @@@",
-"@@+                          @@@",
-"@@+                          @@@",
-"@@+                          @@@",
-"@@+       XXXXXXXXXXX.       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@",
-"@@+       @@@@@@@@@@@+       @@@"
+"********************************",
+"********************************",
+"********************************",
+"********************************",
+"********************************",
+"********************************",
+"********************************",
+"***:::*******::***::::*;:::::***",
+"**#+++$*****#+++$&+++++++++++#**",
+"*:+   #*****O   %@           +:*",
+"*:+   #*****+   %@           +:*",
+"*:+   #*****+   %#XXX.   .XXX@**",
+"*:+   #:***:+   #****@   +******",
+"**+   @&&&&&O   #***:@   +******",
+"*:+    .....    #****@   +******",
+"*:+             #****@   +******",
+"*;+             #***:@   +:*****",
+"**+   +@@@@@o   #***:@   +:*****",
+"**+   #:***:+   #:***@   +:*****",
+"**+   #*****+   #****@   +:*****",
+"*:+   #:****+   %****@   +:*****",
+"*:+   #:****+   %:***@   +:*****",
+"*:+   #****:O   #***:@   +:*****",
+"**#+++$*****#+++$****%+++%******",
+"***;::*******::*******:::*******",
+"********************************",
+"********************************",
+"********************************",
+"********************************",
+"********************************",
+"********************************",
+"********************************"
 };

html/server/div/httrack48x48.xpm

@@ -1,88 +1,70 @@
 /* XPM */
-static char *httrack__[] = {
+static char *httrack48x48[] = {
 /* columns rows colors chars-per-pixel */
-"48 48 34 1",
-"  c #040404040404",
-". c #080808080808",
-"X c #0C0C0C0C0C0C",
-"o c #111111111111",
-"O c #161616161616",
-"+ c #1C1C1C1C1C1C",
-"@ c #222222222222",
-"# c #292929292929",
-"$ c #333333333333",
-"% c #393939393939",
-"& c #333333336666",
-"* c #424242424242",
-"= c #4D4D4D4D4D4D",
-"- c #555555555555",
-"; c #5F5F5F5F5F5F",
-": c #666666666666",
-"> c #777777777777",
-", c #666666669999",
-"< c #66666666CCCC",
-"1 c #808080808080",
-"2 c #868686868686",
-"3 c #969696969696",
-"4 c #999999999999",
-"5 c #A0A0A0A0A4A4",
-"6 c #B2B2B2B2B2B2",
-"7 c #99999999CCCC",
-"8 c #C0C0C0C0C0C0",
-"9 c #CCCCCCCCCCCC",
-"0 c #D7D7D7D7D7D7",
-"q c #DDDDDDDDDDDD",
-"w c #E3E3E3E3E3E3",
-"e c #EAEAEAEAEAEA",
-"r c #F1F1F1F1F1F1",
-"t c None",
+"48 48 16 1 ",
+"  c #060607",
+". c #1D1D25",
+"X c #282835",
+"o c #343445",
+"O c #5A5A77",
+"+ c #6E6E92",
+"@ c #7B7BA4",
+"# c #8484B0",
+"$ c #9999CC",
+"% c #9999CD",
+"& c #9A9ACD",
+"* c #9B9BCF",
+"= c #9292C4",
+"- c #9D9DD2",
+"; c #9E9ED2",
+": c #9999CC",
 /* pixels */
-"77777777777777777777777777777777777777777<,,<777",
-";&,77,&-<-&&&&&,-&&&&&,777777777777777777<$$<777",
-"# ,77, +<o     -+     *777777777777777777<oo<777",
-"# ,77, +<o     -+     *777777777777777777<oo<777",
-"# ,77, +<o     -+     *777777777777777777<oo<777",
-"# ,77, +<o     -+     *777777777777777777<oo<777",
-"# ,77, +<@O  oO;#O. oO&777777777777777777<oo<777",
-"# ,77, +777+ ,7777% ;77777777777777777777<oo<777",
-"# ,77, +777+ ,7777% ;77777777777777777777<oo<777",
-"# ,77, +777+ ,7777% ;77<<<<<7<,<<<<77<,<<<oo<<<<",
-"# ,77, +777+ ,7777% ;77,-,&<7- -,-,77; O,<oo<,--",
-"# #&&$ +777+ ,7777% ;77& & <<o O$ ,7,o  $<oo<& #",
-"#      +777+ ,7777% ;77& + <,   X ,7-   #<oo<@ &",
-"#      +777+ ,7777% ;77&   <%     ,7@   #<oo,  ,",
-"#      +777+ ,7777% ;77&   <+     ,<. +$#<oo; O<",
-"#      +777+ ,7777% ;77&  .,o &&  ,,  ,,-<oo% $7",
-"# Xooo +777+ ,7777% ;77& O<, .<<o ,, O777<oXX ,7",
-"# &,,- +777+ ,7777% ;77& &7, O<7# ,, #777<o   ,7",
-"# ,77, +777+ ,7777% ;77& -7, #77$ ,, $777<o   <7",
-"# ,77, +777+ ,7777% ;77& -7, #77$ ,, $777<ooO ,7",
-"# ,77, +777+ ,7777% ;77& -7, +77# ,, +777<oo% &7",
-"# ,77, +777+ ,7777% ;77& -7, o<<o ,,  ,7<<oo- @7",
-"# ,77, +777+ ,7777% ;77& -7<o ;;  ,,  $-%<oo, o<",
-"# ,77, +777+ ,7777% ;77& -7<@     ,7o   #<oo<O ,",
-"# ,77, +777+ ,7777% ;77& -77%     ,7$   #<oo<$ -",
-"# ,77, +777+ ,7777% ;77& -77,   @ ,7;   #<oo<& #",
-"# ,77, +777+ ,7777% ;77& -77<O #$ ,7<+  $<oo<, o",
-"o #&&$ X&&&o $&&&&O #&&+ #&&&+ @O #&&+  @&. %%  ",
-"                                                ",
-"                                                ",
-"                                                ",
-"                                                ",
-"*@ @O **44#3= +w1+=242:41   =r4 #q8O 2>X;+34%21X",
-";; -$ 2;tt*qqO:00%1qtq6t8   99q 40e:Xqw=4$rt:9w-",
-"$1 1- 4*4=@66$5%:$1%9%5-$  $8#=O9$:8X6:24$6=+6;4",
-"O4 51 4*> O4>%5O $1 6 4O   >-  *2 X0X5O54$3  6O5",
-" 6 64O2*4-@42+3: $1 6 5:*  3@  :-  5#5O44$6-@6O4",
-" 6O46$>*re*08 =w*$1 6 6e6  5X  2$  2%6>14$ee;6:2",
-" 2=15:=*96$ee% 48$1 6 561  5X  2#  1%wq@4$06*0w$",
-" :4-24@*> O4:2 X9*1 6 4O   5O  2%  2$8* 4$3  88 ",
-" %w#:0X*> O4#3  4;1 6 4O   3$  :;  6+5  4$3  68X",
-" OrO%0 *> O4#24X3;1 6 4+   >1  *6 X8X5  4$4  64#",
-"  6 +5 *61#62:6:8*1 6 52:  #w=:Xq:13X5  4$81$6;:",
-"  >  : *tt*e0+:r8$1 6 6t8   8eq 3rr%X5  4$rt:6#4",
-"  @  @ +::O;# O5*O# * *:*   $6: +61  %  %O;:@% %",
-"                                                ",
-"                                                ",
-"                                                "
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$-$$$$$$$$$$$$$$$-$$$$$$$$-$$$$$$$$$$$$$$$",
+"$$$=@@@@@=$$$$$$$$=@@@@@=$#@@@@@@@@@@@@@@@@@=$$$",
+"$$$@.   .@$$$$$$$$+.   .@-O                .@$$$",
+"$$$@     +$$$$$$$$+     @-O                 @$$$",
+"$$$@     +$$$$$$$$+     @-O                 @$$$",
+"$$$@     +$$$$$$$$+     @-O                 @$$$",
+"$$$@     +$$$$$$$$+     @*+ooooo.     Xooooo#$$$",
+"$$$@     +$$$$$$$-+     @$$$$$$$+    .#$$$$$$$$$",
+"$$$@     +-$$$$$$-+     @$$$$$$$+     #$$$$$$$$$",
+"$$$@     +########+     @$$$$$$$+    .#$$$$$$$$$",
+"$$$@     ..........     @$$$$$$-+    .#$$$$$$$$$",
+"$$$@                    @$$$$$$-+    .#$$$$$$$$$",
+"$$$@                    @$$$$$$-+    .#$$$$$$$$$",
+"$$$@                    @$$$$$$-+     #$$$$$$$$$",
+"$$$@     .XXXXXXXX.     @$$$$$$-+     #$$$$$$$$$",
+"$$$@     +$=======+     @$$$$$$-+     #$$$$$$$$$",
+"$$$@     +-$$$$$$$+     @$$$$$$-+     #$$$$$$$$$",
+"$$$@     +$$$$$$$$+     @$$$$$$-+    .#$$$$$$$$$",
+"$$$@     +-$$$$$$$+     @$$$$$$-+    .#$$$$$$$$$",
+"$$$@     +$$$$$$$-+     @$$$$$$-+    .#$$$$$$$$$",
+"$$$@     +$$$$$$$-+     @$$$$$$-+    .#$$$$$$$$$",
+"$$$@     +$$$$$$$$+     @-$$$$$-+    .#$$$$$$$$$",
+"$$$@     +$$$$$$$$+     @$$$$$$-+    .#$$$$$$$$$",
+"$$$@.   .@$$$$$$$$+.   .@$$$$$$-+    .#$$$$$$$$$",
+"$$$=@@@@@=$$$$$$$$=@@@@@=$$$$$$$#@@@@@=$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$-$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$",
+"$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$"
 };

html/server/div/scalable/httrack.svg

@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1024 1024" width="1024" height="1024"><title>HTTrack</title><rect width="1024" height="1024" fill="#9999cc"/><g fill="#8a8ac0" fill-rule="evenodd"><path d="M1105.92 409.6A634.88 409.6 21 1 0 -163.84 409.6A634.88 409.6 21 1 0 1105.92 409.6 ZM1049.6 409.6A578.56 353.28 21 1 0 -107.52 409.6A578.56 353.28 21 1 0 1049.6 409.6 Z"/><path d="M942.08 471.04A409.6 256 21 1 0 122.88 471.04A409.6 256 21 1 0 942.08 471.04 ZM896 471.04A363.52 209.92 21 1 0 168.96 471.04A363.52 209.92 21 1 0 896 471.04 Z"/></g><g fill="#040404"><path d="M189.6 543H480.1V460H189.6ZM421.6 320.1V703.9H514.2V320.1ZM163.8 320.1V703.9H256.5V320.1Z"/><path d="M556.5 403.1H662V703.9H754.6V403.1H860.2V320.1H556.5Z"/></g></svg>

html/server/error.html

@@ -27,7 +27,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/file.html

@@ -27,7 +27,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/finished.html

@@ -27,7 +27,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="3" class="tableWidth">

html/server/help.html

@@ -27,7 +27,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/images/bg_rings.svg

@@ -0,0 +1,8 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="501" height="456" viewBox="0 0 501 456">
+<title>HTTrack rings</title>
+<!-- Transparent, unlike the GIF this replaces: the panel's own #ccd shows through. -->
+<g fill="#b9b9d0" fill-rule="evenodd">
+<path d="M614.59 346.3A323.36 197.55 21.16 1 0 11.46 112.89A323.36 197.55 21.16 1 0 614.59 346.3 ZM590.57 315.66A262.58 160.99 22.18 1 0 104.26 117.43A262.58 160.99 22.18 1 0 590.57 315.66 Z"/>
+<path d="M578.08 292.16A185.6 113.57 21.23 1 0 232.08 157.73A185.6 113.57 21.23 1 0 578.08 292.16 ZM564.4 274.56A150.77 92.55 22.26 1 0 285.34 160.32A150.77 92.55 22.26 1 0 564.4 274.56 Z"/>
+</g>
+</svg>

html/server/index.html

@@ -34,7 +34,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="3" class="tableWidth">

html/server/option1.html

@@ -31,7 +31,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/option10.html

@@ -31,7 +31,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/option11.html

@@ -31,7 +31,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/option2.html

@@ -31,7 +31,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/option2b.html

@@ -41,7 +41,7 @@ function str_replace(str,from,to) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/option3.html

@@ -31,7 +31,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/option4.html

@@ -31,7 +31,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/option5.html

@@ -31,7 +31,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/option6.html

@@ -31,7 +31,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/option7.html

@@ -31,7 +31,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/option8.html

@@ -31,7 +31,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/option9.html

@@ -31,7 +31,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 

html/server/refresh.html

@@ -64,7 +64,7 @@ function no_refresh() {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="3" class="tableWidth">

html/server/step2.html

@@ -43,7 +43,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="3" class="tableWidth">

html/server/step3.html

@@ -52,7 +52,7 @@ function do_check_child() {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="3" class="tableWidth">

html/server/step4.html

@@ -34,7 +34,7 @@ function info(str) {
 
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
 	<tr>
-	<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
+	<td><img src="images/wordmark.svg" width="400" height="36" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
 	</tr>
 </table>
 <table width="76%" border="0" align="center" cellspacing="0" cellpadding="3" class="tableWidth">

html/server/style.css

@@ -36,7 +36,7 @@ a:active  { text-decoration: underline; }
 	border-bottom: 6px solid #000;
 	padding: 10px;  padding-top: 20px;
 	line-height: 1.65em;
-	background-image: url(images/bg_rings.gif);
+	background-image: url(images/bg_rings.svg);
 	background-repeat: no-repeat;
 	background-position: top right;
 	}

src/Makefile.am

@@ -60,6 +60,13 @@ proxytrack_SOURCES = proxy/main.c \
 whttrackrundir = $(bindir)
 whttrackrun_SCRIPTS = webhttrack
 
+# $(datadir) only expands at make time, so substitute here rather than through
+# AC_CONFIG_FILES (#887).
+webhttrack: webhttrack.in Makefile
+	$(AM_V_GEN)$(SED) -e 's|@datadir[@]|$(datadir)|g' $(srcdir)/webhttrack.in >$@.tmp \
+		&& chmod +x $@.tmp && mv -f $@.tmp $@
+CLEANFILES = webhttrack
+
 libhttrack_la_SOURCES =  htscore.c htsparse.c htsback.c htscache.c \
 	htscache_selftest.c htsdns_selftest.c htsselftest.c \
 	htscatchurl.c htsfilters.c htsftp.c htshash.c coucal/coucal.c \
@@ -88,7 +95,7 @@ libhttrack_la_LIBADD = $(THREADS_LIBS) $(ZLIB_LIBS) $(BROTLI_LIBS) $(ZSTD_LIBS)
 libhttrack_la_CFLAGS = $(AM_CFLAGS) -DLIBHTTRACK_EXPORTS -DZLIB_CONST
 libhttrack_la_LDFLAGS = $(AM_LDFLAGS) -version-info $(VERSION_INFO)
 
-EXTRA_DIST = httrack.h htsstats.h webhttrack \
+EXTRA_DIST = httrack.h htsstats.h webhttrack.in \
 		version.rc \
 		libhttrack.rc \
 		httrack.rc \

src/htsback.c

@@ -62,7 +62,30 @@ Please visit our Website: http://www.httrack.com
 
 #define VT_CLREOL       "\33[K"
 
+/* Subdirectory holding a mirrored file's temporaries, beside it. url_savename()
+   maps '~' to '_', so no URL can ever be mirrored inside it (#774, #842). */
+#define HTS_TMPDIR "~hts-tmp"
+
 /* Slot operations */
+static hts_boolean back_tmpname(char *dest, size_t size, const char *save,
+                                const char *ext);
+
+hts_boolean back_spoolname(httrackp *opt, const char *save, char *dest,
+                           size_t size) {
+  /* -p0 keeps no save name to derive from, so it counts instead. No separator:
+     path_html_utf8 brings its own, and is "" with no -O, where an added one
+     would make this absolute and spool into the filesystem root. */
+  if (opt->getmode == 0) {
+    if (!slprintfbuff(dest, size, "%s" HTS_TMPDIR "/tmpfile%d.tmp",
+                      StringBuff(opt->path_html_utf8),
+                      opt->state.tmpnameid++)) {
+      dest[0] = '\0';
+      return HTS_FALSE;
+    }
+    return HTS_TRUE;
+  }
+  return back_tmpname(dest, size, save, "tmp");
+}
 static int slot_can_be_cached_on_disk(const lien_back * back);
 static int slot_can_be_cleaned(const lien_back * back);
 static int slot_can_be_finalized(httrackp * opt, const lien_back * back);
@@ -217,6 +240,7 @@ void back_delete_all(httrackp * opt, cache_back * cache, struct_back * sback) {
 
         if (filename != NULL) {
           (void) UNLINK(filename);
+          back_tmpdir_drop(filename);
         }
 #else
         /* clear entry content (but not yet the entry) */
@@ -311,6 +335,7 @@ static int back_index_ready(httrackp * opt, struct_back * sback, const char *adr
                       adr, fil, sav);
       }
       (void) UNLINK(fileback);
+      back_tmpdir_drop(fileback);
 #else
       itemback = (lien_back *) ptr;
 #endif
@@ -483,22 +508,11 @@ int back_cleanup_background(httrackp * opt, cache_back * cache,
 #ifndef HTS_NO_BACK_ON_DISK
       /* temporarily serialize the entry on disk */
       {
-        /* +16: room for the ".tmp" the url_sav form appends to a full-length
-           save name, so one buffer holds both shapes */
-        char BIGSTK tmpname[HTS_URLMAXSIZE * 2 + 16];
+        /* +32: room for the directory and extension back_spoolname() inserts */
+        char BIGSTK tmpname[HTS_URLMAXSIZE * 2 + 32];
         char *filename;
-        hts_boolean named;
-
-        /* the -p0 name is not derived from url_sav, so it needs a buffer of
-           its own size rather than the save name's */
-        if (opt->getmode != 0) {
-          named =
-              slprintfbuff(tmpname, sizeof(tmpname), "%s.tmp", back[i].url_sav);
-        } else {
-          named = slprintfbuff(tmpname, sizeof(tmpname), "%stmpfile%d.tmp",
-                               StringBuff(opt->path_html_utf8),
-                               opt->state.tmpnameid++);
-        }
+        const hts_boolean named =
+            back_spoolname(opt, back[i].url_sav, tmpname, sizeof(tmpname));
         filename = named ? strdupt(tmpname) : NULL;
 
         if (filename != NULL) {
@@ -656,10 +670,6 @@ int back_nsoc_overall(const struct_back * sback) {
   return n;
 }
 
-/* Subdirectory holding a mirrored file's temporaries, beside it. url_savename()
-   maps '~' to '_', so no URL can ever be mirrored inside it (#774, #842). */
-#define HTS_TMPDIR "~hts-tmp"
-
 /* Build save's temporary as <dir>/<HTS_TMPDIR>/<name>.<ext>. Appending the
    extension to save instead put it in the mirror namespace, so a site serving
    <path>.bak had its copy taken as the backup and then unlinked (#774).
@@ -717,7 +727,7 @@ static int create_back_tmpfile(httrackp *opt, lien_back *const back,
     /* same directory as the named case, so back_tmpdir_drop() only removes one
        the engine made (#842) */
     /* truncation here would collide distinct tmpnameid's onto one name */
-    if (!sprintfbuff(back->tmpfile_buffer, "%s/" HTS_TMPDIR "/tmp%d.%s",
+    if (!sprintfbuff(back->tmpfile_buffer, "%s" HTS_TMPDIR "/tmp%d.%s",
                      StringBuff(opt->path_html_utf8), opt->state.tmpnameid++,
                      ext)) {
       hts_log_print(opt, LOG_WARNING, "temporary filename too long in %s",
@@ -819,6 +829,14 @@ static hts_boolean back_chunked_unterminated(const lien_back *const back) {
   return back->is_chunk && back->chunk_blocksize != -1 ? HTS_TRUE : HTS_FALSE;
 }
 
+/* Past the terminating chunk, the line still owed is the optional trailer
+   section (RFC 9112 7.1.2), read and discarded like a header block. */
+static hts_boolean back_in_chunk_trailers(const lien_back *const back) {
+  return back->status == STATUS_CHUNK_CR && back->chunk_blocksize == -1
+             ? HTS_TRUE
+             : HTS_FALSE;
+}
+
 // objet (lien) téléchargé ou transféré depuis le cache
 //
 // fermer les paramètres de transfert,
@@ -2053,11 +2071,13 @@ int back_add(struct_back *sback, httrackp *opt, cache_back *cache,
       int hash_pos_return = 0;
 
       if (cache->hashtable) {
-        char BIGSTK buff[HTS_URLMAXSIZE * 4];
+        char BIGSTK buff[CACHE_KEY_SIZE];
+        size_t used = 0;
 
-        strcpybuff(buff, adr);
-        strcatbuff(buff, fil);
-        hash_pos_return = coucal_read(cache->hashtable, buff, &hash_pos);
+        /* a key too long to be in the table is a miss, not a fatal error */
+        if (slcatprintfbuff(buff, sizeof(buff), &used, "%s%s", adr, fil)) {
+          hash_pos_return = coucal_read(cache->hashtable, buff, &hash_pos);
+        }
 
         // negative values when data is not in cache
         if (hash_pos_return < 0) {
@@ -3475,11 +3495,16 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
 
           // réception de données depuis socket ou fichier
           if (back[i].status) {
-            if (back[i].status == STATUS_WAIT_HEADERS)  // recevoir par bloc de lignes
-              retour_fread = http_xfread1(&(back[i].r), 0);
+            if (back[i].status == STATUS_WAIT_HEADERS)
+              retour_fread = http_xfread1(&(back[i].r), HTS_XFREAD_LINE_BLOCK);
             else if (back[i].status == STATUS_CHUNK_WAIT || back[i].status == STATUS_CHUNK_CR) {        // recevoir longueur chunk en hexa caractère par caractère
               // backuper pour lire dans le buffer chunk
               htsblk r;
+              /* Block mode bounds the trailer section, which declares no length
+                 of its own, by HTS_LINE_BLOCK_SIZE. */
+              const int chunk_read_mode = back_in_chunk_trailers(&back[i])
+                                              ? HTS_XFREAD_LINE_BLOCK
+                                              : HTS_XFREAD_LINE;
 
               memcpy(&r, &(back[i].r), sizeof(htsblk));
               back[i].r.is_write = 0;   // mémoire
@@ -3489,8 +3514,8 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
               back[i].r.out = NULL;
               back[i].r.is_file = 0;
               //
-              // ligne par ligne
-              retour_fread = http_xfread1(&(back[i].r), -1);
+              // one line, or the whole trailer block
+              retour_fread = http_xfread1(&(back[i].r), chunk_read_mode);
               // modifier et restaurer
               back[i].chunk_adr = back[i].r.adr;        // adresse
               back[i].chunk_size = back[i].r.size;      // taille taille chunk
@@ -3628,12 +3653,22 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
             }
             // Traitement des en têtes chunks ou en têtes
             if (back[i].status == STATUS_CHUNK_WAIT || back[i].status == STATUS_CHUNK_CR) {     // réception taille chunk en hexa (  après les en têtes, peut ne pas
-              if (back[i].chunk_size > 0
-                  && back[i].chunk_adr[back[i].chunk_size - 1] == 10) {
+              const hts_boolean in_trailers = back_in_chunk_trailers(&back[i]);
+
+              /* A chunk-size or chunk-CRLF line closes on its first LF, the
+                 trailer section on the blank line ending it. Two LFs mean a
+                 blank line only because the reader drops every CR. */
+              if (back[i].chunk_size > 0 &&
+                  back[i].chunk_adr[back[i].chunk_size - 1] == 10 &&
+                  (!in_trailers || back[i].chunk_size == 1 ||
+                   back[i].chunk_adr[back[i].chunk_size - 2] == 10)) {
                 int chunk_size = -1;
                 char chunk_data[64];
 
-                if (back[i].chunk_size < 32) {  // pas trop gros
+                if (in_trailers) {
+                  chunk_size =
+                      0; /* fields discarded, the blank line ends the body */
+                } else if (back[i].chunk_size < 32) { // not too big
                   char *chstrip = back[i].chunk_adr;
 
                   back[i].chunk_adr[back[i].chunk_size - 1] = '\0';     // octet nul 
@@ -3817,12 +3852,6 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
                           }
                         }
                       }
-
-                      /* Oops, trailers! */
-                      if (back[i].r.keep_alive_trailers) {
-                        /* fixme (not yet supported) */
-                      }
-
                     }
 
                   }
@@ -3836,7 +3865,7 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
                   // NO! xxback[i].chunk_blocksize = 0;
                 }
 
-              }                 // taille buffer chunk > 1 && LF
+              } // chunk buffer holds a complete line
               //
             } else if (back[i].status == STATUS_WAIT_HEADERS) { // en têtes (avant le chunk si il est présent)
               //

src/htsback.h

@@ -155,6 +155,11 @@ hts_boolean back_finalize_backup(httrackp *opt, lien_back *const back,
 /* Remove the reserved directory a temporary sat in, once the last slot sharing
    it is done; a non-empty one just refuses. No-op outside that directory. */
 void back_tmpdir_drop(const char *tmp);
+/* Name the spool file of a frozen backlog slot, inside the reserved directory
+   no save name can spell. HTS_FALSE (dest emptied) if it would not fit.
+   Consumes an opt->state.tmpnameid under -p0. Note: utf-8. */
+hts_boolean back_spoolname(httrackp *opt, const char *save, char *dest,
+                           size_t size);
 /* -#test=backswap: slots eligible for the on-disk ready table. */
 int back_selftest_slot_swap(void);
 void back_info(struct_back * sback, int i, int j, FILE * fp);

src/htsbacktrace.c

@@ -211,6 +211,13 @@ void hts_backtrace_init(void) {
 #endif
 }
 
+/* Why the report has no frames: a silent gap reads as a handler that died. */
+static void print_no_trace(int fd, const char *msg, size_t len) {
+  if (write(fd, msg, len) != len) { /* no ssize_t: this is built on MSVC too */
+    /* sorry GCC */
+  }
+}
+
 void hts_print_backtrace(int fd) {
 #ifdef USES_BACKTRACE
   void *stack[256];
@@ -227,12 +234,15 @@ void hts_print_backtrace(int fd) {
       symbolize_backtrace(stack, size, fd);
       entered = 0;
     }
+  } else {
+    /* An empty trace means the build carries no unwind tables. */
+    const char msg[] = "No stack trace available: unwinding failed\n";
+
+    print_no_trace(fd, msg, sizeof(msg) - 1);
   }
 #else
   const char msg[] = "No stack trace available on this OS :(\n";
 
-  if (write(fd, msg, sizeof(msg) - 1) != sizeof(msg) - 1) {
-    /* sorry GCC */
-  }
+  print_no_trace(fd, msg, sizeof(msg) - 1);
 #endif
 }

src/htscache.c

@@ -119,22 +119,32 @@ void cache_mayadd(httrackp * opt, cache_back * cache, htsblk * r,
   // ---fin stockage en cache---
 }
 
-#define ZIP_FIELD_STRING(headers, headersSize, field, value) do { \
-  if ( (value != NULL) && (value)[0] != '\0') { \
-    sprintf(headers + headersSize, "%s: %s\r\n", field, (value != NULL) ? (value) : ""); \
-    (headersSize) += (int) strlen(headers + headersSize); \
-  } \
-} while(0)
-#define ZIP_FIELD_INT(headers, headersSize, field, value) do { \
-  if ( (value != 0) ) { \
-    sprintf(headers + headersSize, "%s: "LLintP"\r\n", field, (LLint)(value)); \
-    (headersSize) += (int) strlen(headers + headersSize); \
-  } \
-} while(0)
-#define ZIP_FIELD_INT_FORCE(headers, headersSize, field, value) do { \
-  sprintf(headers + headersSize, "%s: "LLintP"\r\n", field, (LLint)(value)); \
-  (headersSize) += (int) strlen(headers + headersSize); \
-} while(0)
+/* Remote-controlled values (ETag, Location, the URL) can together outgrow the
+   block, and a clipped field reads back as a valid shorter one, so one that
+   does not fit is dropped whole and counted. `headers` must be an array. */
+#define ZIP_FIELD_STRING(headers, headersSize, dropped, field, value)          \
+  do {                                                                         \
+    if ((value) != NULL && (value)[0] != '\0' &&                               \
+        !slcatprintfbuff(headers, sizeof(headers), &(headersSize),             \
+                         "%s: %s\r\n", field, value)) {                        \
+      (dropped)++;                                                             \
+    }                                                                          \
+  } while (0)
+#define ZIP_FIELD_INT(headers, headersSize, dropped, field, value)             \
+  do {                                                                         \
+    if ((value) != 0 &&                                                        \
+        !slcatprintfbuff(headers, sizeof(headers), &(headersSize),             \
+                         "%s: " LLintP "\r\n", field, (LLint) (value))) {      \
+      (dropped)++;                                                             \
+    }                                                                          \
+  } while (0)
+#define ZIP_FIELD_INT_FORCE(headers, headersSize, dropped, field, value)       \
+  do {                                                                         \
+    if (!slcatprintfbuff(headers, sizeof(headers), &(headersSize),             \
+                         "%s: " LLintP "\r\n", field, (LLint) (value))) {      \
+      (dropped)++;                                                             \
+    }                                                                          \
+  } while (0)
 
 struct cache_back_zip_entry {
   unsigned long int hdrPos;
@@ -206,11 +216,12 @@ static void cache_zip_write_failed(httrackp *opt, cache_back *cache,
 void cache_add(httrackp * opt, cache_back * cache, const htsblk * r,
                const char *url_adr, const char *url_fil, const char *url_save,
                int all_in_cache, const char *path_prefix) {
-  char BIGSTK filename[HTS_URLMAXSIZE * 4];
+  char BIGSTK filename[CACHE_ENTRYNAME_SIZE];
   char catbuff[CATBUFF_SIZE];
   int dataincache = 0;          // put data in cache ?
-  char BIGSTK headers[8192];
-  int headersSize = 0;
+  char BIGSTK headers[CACHE_HEADERS_SIZE];
+  size_t headersSize = 0;
+  int headersDropped = 0;
 
   zip_fileinfo fi;
   const char *url_save_suffix = url_save;
@@ -275,10 +286,11 @@ void cache_add(httrackp * opt, cache_back * cache, const htsblk * r,
       message = "(See X-StatusMessage)";
     }
     /* 64 characters MAX for first line */
-    sprintf(headers + headersSize, "HTTP/1.%c %d %s\r\n", '1', r->statuscode,
-            message);
+    if (!slcatprintfbuff(headers, sizeof(headers), &headersSize,
+                         "HTTP/1.%c %d %s\r\n", '1', r->statuscode, message)) {
+      headersDropped++;
+    }
   }
-  headersSize += (int) strlen(headers + headersSize);
 
   if (path_prefix != NULL && path_prefix[0] != '\0' && url_save != NULL
       && url_save[0] != '\0') {
@@ -290,28 +302,55 @@ void cache_add(httrackp * opt, cache_back * cache, const htsblk * r,
   }
 
   /* Second line MUST ALWAYS be X-In-Cache */
-  ZIP_FIELD_INT_FORCE(headers, headersSize, "X-In-Cache", dataincache);
-  ZIP_FIELD_INT(headers, headersSize, "X-StatusCode", r->statuscode);
-  ZIP_FIELD_STRING(headers, headersSize, "X-StatusMessage", r->msg);
-  ZIP_FIELD_INT(headers, headersSize, "X-Size", r->size);       // size
-  ZIP_FIELD_STRING(headers, headersSize, "Content-Type", r->contenttype);       // contenttype
-  ZIP_FIELD_STRING(headers, headersSize, "X-Charset", r->charset);      // contenttype
-  ZIP_FIELD_STRING(headers, headersSize, "Last-Modified", r->lastmodified);     // last-modified
-  ZIP_FIELD_STRING(headers, headersSize, "Etag", r->etag);      // Etag
-  ZIP_FIELD_STRING(headers, headersSize, "Location", r->location);      // 'location' pour moved
-  ZIP_FIELD_STRING(headers, headersSize, "Content-Disposition", r->cdispo);     // Content-disposition
-  ZIP_FIELD_STRING(headers, headersSize, "X-Addr", url_adr);    // Original address
-  ZIP_FIELD_STRING(headers, headersSize, "X-Fil", url_fil);     // Original URI filename
-  ZIP_FIELD_STRING(headers, headersSize, "X-Save", url_save_suffix);    // Original save filename
+  ZIP_FIELD_INT_FORCE(headers, headersSize, headersDropped, "X-In-Cache",
+                      dataincache);
+  ZIP_FIELD_INT(headers, headersSize, headersDropped, "X-StatusCode",
+                r->statuscode);
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "X-StatusMessage",
+                   r->msg);
+  ZIP_FIELD_INT(headers, headersSize, headersDropped, "X-Size",
+                r->size); // size
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "Content-Type",
+                   r->contenttype); // contenttype
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "X-Charset",
+                   r->charset); // contenttype
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "Last-Modified",
+                   r->lastmodified); // last-modified
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "Etag",
+                   r->etag); // Etag
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "Location",
+                   r->location); // 'location' pour moved
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "Content-Disposition",
+                   r->cdispo); // Content-disposition
+  /* X-Save first of the three: the only one a reader acts on, so a full block
+     must not drop it (X-Addr/X-Fil are pass-through metadata) */
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "X-Save",
+                   url_save_suffix); // Original save filename
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "X-Addr",
+                   url_adr); // Original address
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "X-Fil",
+                   url_fil); // Original URI filename
+  if (headersDropped != 0) {
+    hts_log_print(opt, LOG_WARNING,
+                  "cached headers too large, %d field(s) dropped: %s%s",
+                  headersDropped, url_adr, url_fil);
+  }
 
   /* Filename */
-  if (!link_has_authority(url_adr)) {
-    strcpybuff(filename, "http://");
-  } else {
-    strcpybuff(filename, "");
+  {
+    size_t used = 0;
+
+    /* the URL comes off the wire: drop the entry rather than abort the mirror,
+       and stop where the index load does so nothing written is unreadable */
+    if (!slcatprintfbuff(filename, sizeof(filename) - 2, &used, "%s%s%s",
+                         link_has_authority(url_adr) ? "" : "http://", url_adr,
+                         url_fil)) {
+      hts_log_print(opt, LOG_WARNING,
+                    "URL too long to be cached, entry not cached: %s%s",
+                    url_adr, url_fil);
+      return;
+    }
   }
-  strcatbuff(filename, url_adr);
-  strcatbuff(filename, url_fil);
 
   /* Time */
   memset(&fi, 0, sizeof(fi));
@@ -474,7 +513,7 @@ static htsblk cache_readex_new(httrackp * opt, cache_back * cache,
                                const char *target_save, char *location,
                                char *return_save, int readonly) {
   char BIGSTK location_default[HTS_URLMAXSIZE * 2];
-  char BIGSTK buff[HTS_URLMAXSIZE * 2];
+  char BIGSTK buff[CACHE_KEY_SIZE];
   char BIGSTK previous_save[HTS_URLMAXSIZE * 2];
   char BIGSTK previous_save_[HTS_URLMAXSIZE * 2];
   char catbuff[CATBUFF_SIZE];
@@ -493,9 +532,16 @@ static htsblk cache_readex_new(httrackp * opt, cache_back * cache,
     r.location = location_default;
   }
   r.location[0] = '\0';
-  strcpybuff(buff, adr);
-  strcatbuff(buff, fil);
-  hash_pos_return = coucal_read(cache->hashtable, buff, &hash_pos);
+  {
+    size_t used = 0;
+
+    /* a key too long to be in the table is a miss, not a fatal error */
+    if (!slcatprintfbuff(buff, sizeof(buff), &used, "%s%s", adr, fil)) {
+      hash_pos_return = 0;
+    } else {
+      hash_pos_return = coucal_read(cache->hashtable, buff, &hash_pos);
+    }
+  }
   /* avoid errors on data entries */
   if (adr[0] == '/' && adr[1] == '/' && adr[2] == '[') {
     hash_pos_return = 0;
@@ -1094,7 +1140,8 @@ void cache_init(cache_back * cache, httrackp * opt) {
         /* Ready directory entries */
         if ((zErr = unzGoToFirstFile((unzFile) cache->zipInput)) == Z_OK) {
           char comment[128];
-          char BIGSTK filename[HTS_URLMAXSIZE * 4];
+          char BIGSTK filename[CACHE_ENTRYNAME_SIZE];
+          unz_file_info zfi;
           int entries = 0;
 
           memset(comment, 0, sizeof(comment));  // for truncated reads
@@ -1103,13 +1150,17 @@ void cache_init(cache_back * cache, httrackp * opt) {
 
             filename[0] = '\0';
             comment[0] = '\0';
+            memset(&zfi, 0, sizeof(zfi));
             if (unzOpenCurrentFile((unzFile) cache->zipInput) == Z_OK) {
               if ((readSizeHeader =
-                   unzGetLocalExtrafield((unzFile) cache->zipInput, comment,
-                                         sizeof(comment) - 2)) > 0
-                  && unzGetCurrentFileInfo((unzFile) cache->zipInput, NULL,
-                                           filename, sizeof(filename) - 2, NULL,
-                                           0, NULL, 0) == Z_OK) {
+                       unzGetLocalExtrafield((unzFile) cache->zipInput, comment,
+                                             sizeof(comment) - 2)) > 0 &&
+                  unzGetCurrentFileInfo((unzFile) cache->zipInput, &zfi,
+                                        filename, sizeof(filename) - 2, NULL, 0,
+                                        NULL, 0) == Z_OK
+                  /* minizip leaves a name this long unterminated, and a clipped
+                     one would index as another URL's key */
+                  && zfi.size_filename < sizeof(filename) - 2) {
                 long int pos =
                   (long int) unzGetOffset((unzFile) cache->zipInput);
                 assertf(readSizeHeader < sizeof(comment));

src/htscache.h

@@ -99,6 +99,16 @@ typedef enum {
    the involved files are absent. */
 void hts_cache_reconcile(httrackp *opt, hts_cache_reconcile_mode mode);
 
+/* Capacity of the per-entry header block cache_add builds; the self-test
+   asserts the writer stays inside it, so both must move together. */
+#define CACHE_HEADERS_SIZE 8192
+
+/* Cache key: url_adr followed by url_fil, each lien_back-sized. Write, index
+   load and lookup must each hold one whole -- a clipped key aliases another. */
+#define CACHE_KEY_SIZE (HTS_URLMAXSIZE * 4)
+/* The ZIP entry name is the key behind a "http://" the index load strips. */
+#define CACHE_ENTRYNAME_SIZE (CACHE_KEY_SIZE + 8)
+
 void cache_rstr(FILE *fp, char *s, size_t s_size);
 char *cache_rstr_addr(FILE * fp);
 int cache_brstr(char *adr, char *s, size_t s_size);

src/htscache_selftest.c

@@ -921,7 +921,7 @@ static const golden_entry golden_entries[] = {
 
 #define GOLDEN_COUNT (sizeof(golden_entries) / sizeof(golden_entries[0]))
 
-static void golden_setup(httrackp *opt, const char *dir) {
+static void selftest_setup_dir(httrackp *opt, const char *dir) {
   char base[HTS_URLMAXSIZE];
 
   strcpybuff(base, dir);
@@ -940,7 +940,7 @@ int cache_golden_selftest(httrackp *opt, const char *dir, int regen) {
   cache_back cache;
 
   selftest_tag = "cache-golden";
-  golden_setup(opt, dir);
+  selftest_setup_dir(opt, dir);
 
   /* regen rewrites the fixture from the table; the test never passes it, so the
      read pass verifies bytes a previous build froze */
@@ -1025,7 +1025,7 @@ int cache_reconcile_selftest(httrackp *opt, const char *dir) {
   /* around the interrupted-run thresholds (new < 32768, old > 65536) */
   static const LLint TINY = 1024, MID = 40000, SOLID = 131072;
 
-  golden_setup(opt, dir);
+  selftest_setup_dir(opt, dir);
 #ifdef _WIN32
   mkdir(reconcile_st_path(opt, "hts-cache"));
 #else
@@ -1174,7 +1174,7 @@ int cache_legacy_refused_selftest(httrackp *opt, const char *dir) {
   int variant;
 
   selftest_tag = "cache-legacy";
-  golden_setup(opt, dir);
+  selftest_setup_dir(opt, dir);
 #ifdef _WIN32
   mkdir(reconcile_st_path(opt, "hts-cache"));
 #else
@@ -1552,7 +1552,7 @@ int cache_corruption_selftest(httrackp *opt, const char *dir) {
   int failures = 0;
 
   selftest_tag = "cache-corrupt";
-  golden_setup(opt, dir);
+  selftest_setup_dir(opt, dir);
 
   failures +=
       corrupt_case_zip(opt, "X-Size: 44", "X-Size: 99", 1, 1,
@@ -1641,3 +1641,344 @@ int cache_corruption_selftest(httrackp *opt, const char *dir) {
 
   return failures;
 }
+
+/* Header block of the ZIP's first entry, from its local extra field. By
+   position, not name: unzLocateFile compares through a 256-byte buffer and
+   cannot find the multi-KB entry name this test writes. */
+static int read_first_entry_extra(const char *path, char *extra,
+                                  size_t extralen) {
+  unzFile z = unzOpen(path);
+  int elen = -1;
+
+  if (z == NULL)
+    return -1;
+  if (unzGoToFirstFile(z) == UNZ_OK && unzOpenCurrentFile(z) == UNZ_OK) {
+    elen = unzGetLocalExtrafield(z, extra, (unsigned) (extralen - 1));
+    if (elen >= 0) {
+      extra[elen] = '\0';
+    }
+    unzCloseCurrentFile(z);
+  }
+  unzClose(z);
+  return elen;
+}
+
+static void fill_str(char *s, size_t size, size_t len, char c) {
+  assertf(len < size);
+  memset(s, c, len);
+  s[len] = '\0';
+}
+
+/* A value read back must be a prefix of what was written, and no shorter than
+   `least`: the reader clips a line to its parse buffer, but never splices,
+   reorders, or drops more than that. */
+static int check_prefix(const char *what, const char *got, const char *want,
+                        size_t least) {
+  const size_t len = strlen(got);
+
+  if (len < least || strncmp(got, want, len) != 0) {
+    fprintf(stderr,
+            "cache-hdrbounds: %s reads back '%.32s...' (%d bytes), expected a "
+            "prefix of the %d written, at least %d long\n",
+            what, got, (int) len, (int) strlen(want), (int) least);
+    return 1;
+  }
+  return 0;
+}
+
+int cache_header_bounds_selftest(httrackp *opt, const char *dir) {
+  int failures = 0;
+  cache_back cache;
+  /* every htsblk field at the cap its declaration allows */
+  static char msg[80], ctype[HTS_MIMETYPE_SIZE], charset[HTS_MIMETYPE_SIZE];
+  static char etag[256], cdispo[256], location[HTS_URLMAXSIZE * 2];
+  /* url_adr/url_fil/save are all lien_back-sized, so the three alone reach
+     6 KB, past the header block the writer builds them in */
+  static char big_adr[HTS_URLMAXSIZE * 2], big_fil[HTS_URLMAXSIZE * 2];
+  static char big_save[HTS_URLMAXSIZE * 2];
+  /* the control: same maxed fields, but a URL short enough to leave the header
+     block room, so nothing is dropped and the round-trip must be exact */
+  static char ok_fil[HTS_URLMAXSIZE];
+  const char *const ok_adr = "example.com";
+  const char *const ok_save = "example.com/big.html";
+  const char *const body = "<html><body>maxed</body></html>";
+
+  fill_str(msg, sizeof(msg), sizeof(msg) - 1, 'M');
+  fill_str(ctype, sizeof(ctype), sizeof(ctype) - 1, 'C');
+  fill_str(charset, sizeof(charset), sizeof(charset) - 1, 'H');
+  fill_str(etag, sizeof(etag), sizeof(etag) - 1, 'E');
+  fill_str(cdispo, sizeof(cdispo), sizeof(cdispo) - 1, 'D');
+  fill_str(location, sizeof(location), sizeof(location) - 1, 'L');
+  fill_str(big_adr, sizeof(big_adr), sizeof(big_adr) - 1, 'a');
+  big_fil[0] = '/';
+  fill_str(big_fil + 1, sizeof(big_fil) - 1, 2046, 'f');
+  fill_str(big_save, sizeof(big_save), sizeof(big_save) - 1, 'S');
+  ok_fil[0] = '/';
+  fill_str(ok_fil + 1, sizeof(ok_fil) - 1, sizeof(ok_fil) - 2, 'k');
+
+  {
+    char base[HTS_URLMAXSIZE];
+
+    strcpybuff(base, dir);
+    if (base[0] != '\0' && hts_lastchar(base) != '/') {
+      strcatbuff(base, "/");
+    }
+    StringCopy(opt->path_log, base);
+    StringCopy(opt->path_html, base);
+    StringCopy(opt->path_html_utf8, base);
+  }
+  opt->cache = HTS_CACHE_PRIORITY;
+
+  selftest_open_for_write(&cache, opt);
+  store_entry(opt, &cache, big_adr, big_fil, big_save, 200, msg, ctype, charset,
+              "Mon, 01 Jan 2024 00:00:00 GMT", etag, location, cdispo, body,
+              strlen(body));
+  store_entry(opt, &cache, ok_adr, ok_fil, ok_save, 200, msg, ctype, charset,
+              "Mon, 01 Jan 2024 00:00:00 GMT", etag, location, cdispo, body,
+              strlen(body));
+  selftest_close(&cache);
+
+  /* Read the header straight from the ZIP, not through cache_readex: a runtime
+     that does not trap the overrun would store the block and still pass. */
+  {
+    char zippath[HTS_URLMAXSIZE];
+    static char extra[CACHE_HEADERS_SIZE * 4];
+    size_t elen;
+
+    slprintfbuff_clip(zippath, sizeof(zippath), "%shts-cache/new.zip",
+                      StringBuff(opt->path_log));
+    if (read_first_entry_extra(zippath, extra, sizeof(extra)) < 0) {
+      fprintf(stderr, "cache-hdrbounds: maxed entry missing from the ZIP\n");
+      failures++;
+    } else if (strstr(extra, cdispo) == NULL) {
+      /* an early-out writer would leave the control entry first, and every
+         check below would then pass on the wrong block */
+      fprintf(stderr,
+              "cache-hdrbounds: first ZIP entry is not the maxed one\n");
+      failures++;
+    } else {
+      elen = strlen(extra);
+      if (elen >= CACHE_HEADERS_SIZE) {
+        fprintf(stderr,
+                "cache-hdrbounds: header block is %d bytes, past the writer's "
+                "%d-byte buffer\n",
+                (int) elen, (int) CACHE_HEADERS_SIZE);
+        failures++;
+      }
+      if (elen < 2 || strcmp(extra + elen - 2, "\r\n") != 0) {
+        fprintf(stderr, "cache-hdrbounds: header block ends mid-line\n");
+        failures++;
+      }
+      /* X-Fil is written last, so its absence proves the block filled; the URL
+         is already at its lien_back cap, so a surviving X-Fil means the case
+         needs rebuilding, not silently weakening. */
+      if (strstr(extra, "X-Fil: ") != NULL) {
+        fprintf(stderr,
+                "cache-hdrbounds: the maxed entry no longer fills the %d-byte "
+                "block (%d bytes); this case has stopped testing the bound\n",
+                (int) CACHE_HEADERS_SIZE, (int) elen);
+        failures++;
+      }
+      /* X-Save is the only one of the three a reader acts on, so a full block
+         must not be what drops it */
+      if (strstr(extra, "X-Save: ") == NULL) {
+        fprintf(stderr, "cache-hdrbounds: X-Save was dropped from a full "
+                        "block; it must be written before X-Addr/X-Fil\n");
+        failures++;
+      }
+    }
+  }
+
+  selftest_open_for_read(&cache, opt);
+  {
+    char *locbuf = malloct(HTS_URLMAXSIZE * 2);
+    htsblk r;
+
+    locbuf[0] = '\0';
+    r = cache_readex(opt, &cache, ok_adr, ok_fil, "", locbuf, NULL, 1);
+    if (r.statuscode != 200) {
+      fprintf(stderr,
+              "cache-hdrbounds: entry after the overflowing one reads "
+              "statuscode %d, expected 200\n",
+              r.statuscode);
+      failures++;
+    }
+    /* a header block that fits loses nothing */
+    if (strcmp(r.msg, msg) != 0 || strcmp(r.contenttype, ctype) != 0 ||
+        strcmp(r.charset, charset) != 0 || strcmp(r.etag, etag) != 0 ||
+        strcmp(r.cdispo, cdispo) != 0) {
+      fprintf(stderr, "cache-hdrbounds: a maxed field did not round-trip\n");
+      failures++;
+    }
+    /* the reader clips this to its line buffer (1014 of the 2047 bytes at the
+       current HTS_URLMAXSIZE); the floor only has to rule out a real loss */
+    failures += check_prefix("location", locbuf, location, 1000);
+    if (r.adr == NULL || r.size != (LLint) strlen(body) ||
+        memcmp(r.adr, body, strlen(body)) != 0) {
+      fprintf(stderr, "cache-hdrbounds: body did not round-trip\n");
+      failures++;
+    }
+    if (r.adr != NULL) {
+      freet(r.adr);
+    }
+    freet(locbuf);
+  }
+  selftest_close(&cache);
+
+  return failures;
+}
+
+/* A URL the cache cannot hold must read as a miss, never abort. */
+static int expect_miss(httrackp *opt, cache_back *cache, const char *adr,
+                       const char *fil) {
+  htsblk r = cache_readex(opt, cache, adr, fil, "", NULL, NULL, 1);
+  int failures = 0;
+
+  if (r.statuscode != STATUSCODE_INVALID) {
+    fprintf(stderr,
+            "%s: an unstorable URL reads statuscode %d, expected a "
+            "miss\n",
+            selftest_tag, r.statuscode);
+    failures++;
+  }
+  if (r.adr != NULL) {
+    freet(r.adr);
+  }
+  return failures;
+}
+
+/* A cache ZIP whose first entry name is longer than the writer can emit,
+   followed by a sane one; minizip hands such a name back unterminated. */
+static int urlbounds_write_foreign_zip(const char *zippath, size_t namelen) {
+  static const char hdr[] = "HTTP/1.1 200 OK\r\nX-In-Cache: 1\r\n"
+                            "X-StatusCode: 200\r\nX-Size: 4\r\n"
+                            "X-StatusMessage: OK\r\n"
+                            "Content-Type: text/html\r\nX-Charset: utf-8\r\n";
+  static char longname[CACHE_ENTRYNAME_SIZE * 2];
+  const char *const body = "okay";
+  zip_fileinfo fi;
+  zipFile z = zipOpen(zippath, APPEND_STATUS_CREATE);
+  int err = 0;
+  int i;
+
+  if (z == NULL) {
+    return -1;
+  }
+  memset(&fi, 0, sizeof(fi));
+  fill_str(longname, sizeof(longname), namelen, 'z');
+  memcpy(longname, "http://", 7);
+  for (i = 0; i < 2; i++) {
+    const char *const name = i == 0 ? longname : "http://example.com/sane.html";
+
+    err |= zipOpenNewFileInZip(z, name, &fi, hdr, (uInt) strlen(hdr), NULL, 0,
+                               NULL, Z_DEFLATED, Z_DEFAULT_COMPRESSION);
+    err |= zipWriteInFileInZip(z, body, (int) strlen(body));
+    err |= zipCloseFileInZip(z);
+  }
+  err |= zipClose(z, NULL);
+  return err;
+}
+
+int cache_url_bounds_selftest(httrackp *opt, const char *dir) {
+  int failures = 0;
+  cache_back cache;
+  /* the largest pair lien_back can hold, and a twin differing only in its last
+     byte: a key clipped anywhere on the path aliases the two together */
+  static char big_adr[HTS_URLMAXSIZE * 2], big_fil[HTS_URLMAXSIZE * 2];
+  static char twin_fil[HTS_URLMAXSIZE * 2];
+  /* past every buffer on the path: no lien_back holds these, but the cache API
+     takes plain pointers, and dropping the entry beats aborting the mirror.
+     One overshoots url_fil, the other url_adr, so a bound on one destination
+     only is not enough to pass */
+  static char huge_fil[CACHE_ENTRYNAME_SIZE * 2];
+  static char huge_adr[CACHE_ENTRYNAME_SIZE * 2];
+  /* the key a writer that clipped huge_fil would have stored it under */
+  static char decoy_fil[HTS_URLMAXSIZE * 2];
+  const char *const short_adr = "example.com";
+  const char *const short_fil = "/after.html";
+  const char *const big_body = "<html><body>maxed</body></html>";
+  const char *const twin_body = "<html><body>twin</body></html>";
+  const char *const short_body = "<html><body>after</body></html>";
+  const char *const decoy_body = "<html><body>decoy</body></html>";
+  const char *const lm = "Mon, 01 Jan 2024 00:00:00 GMT";
+
+  fill_str(big_adr, sizeof(big_adr), sizeof(big_adr) - 1, 'a');
+  big_fil[0] = '/';
+  fill_str(big_fil + 1, sizeof(big_fil) - 1, sizeof(big_fil) - 2, 'f');
+  memcpy(twin_fil, big_fil, sizeof(twin_fil));
+  twin_fil[sizeof(twin_fil) - 2] = 'g';
+  huge_fil[0] = '/';
+  fill_str(huge_fil + 1, sizeof(huge_fil) - 1, sizeof(huge_fil) - 2, 'h');
+  fill_str(huge_adr, sizeof(huge_adr), sizeof(huge_adr) - 1, 'A');
+  decoy_fil[0] = '/';
+  fill_str(decoy_fil + 1, sizeof(decoy_fil) - 1, sizeof(decoy_fil) - 2, 'h');
+
+  selftest_tag = "cache-urlbounds";
+  selftest_setup_dir(opt, dir);
+
+  selftest_open_for_write(&cache, opt);
+  store_entry(opt, &cache, big_adr, big_fil, "example.com/big.html", 200, "OK",
+              "text/html", "utf-8", lm, "", "", "", big_body, strlen(big_body));
+  store_entry(opt, &cache, big_adr, twin_fil, "example.com/twin.html", 200,
+              "OK", "text/html", "utf-8", lm, "", "", "", twin_body,
+              strlen(twin_body));
+  /* stored first, so a writer that clipped rather than dropped would overwrite
+     it under its own key */
+  store_entry(opt, &cache, big_adr, decoy_fil, "example.com/decoy.html", 200,
+              "OK", "text/html", "utf-8", lm, "", "", "", decoy_body,
+              strlen(decoy_body));
+  store_entry(opt, &cache, big_adr, huge_fil, "example.com/huge.html", 200,
+              "OK", "text/html", "utf-8", lm, "", "", "", "dropped", 7);
+  store_entry(opt, &cache, huge_adr, "/short.html", "example.com/wide.html",
+              200, "OK", "text/html", "utf-8", lm, "", "", "", "dropped", 7);
+  /* written after the dropped one: the drop must cost nothing but itself */
+  store_entry(opt, &cache, short_adr, short_fil, "example.com/after.html", 200,
+              "OK", "text/html", "utf-8", lm, "", "", "", short_body,
+              strlen(short_body));
+  selftest_close(&cache);
+
+  selftest_open_for_read(&cache, opt);
+  /* the twin differs from the maxed entry only in its last byte: a key clipped
+     anywhere on the path serves one for the other */
+  failures += check_entry(opt, &cache, big_adr, big_fil, 200, "OK", "text/html",
+                          "utf-8", lm, "", "", "", big_body, strlen(big_body));
+  failures +=
+      check_entry(opt, &cache, big_adr, twin_fil, 200, "OK", "text/html",
+                  "utf-8", lm, "", "", "", twin_body, strlen(twin_body));
+  failures +=
+      check_entry(opt, &cache, short_adr, short_fil, 200, "OK", "text/html",
+                  "utf-8", lm, "", "", "", short_body, strlen(short_body));
+  failures +=
+      check_entry(opt, &cache, big_adr, decoy_fil, 200, "OK", "text/html",
+                  "utf-8", lm, "", "", "", decoy_body, strlen(decoy_body));
+  failures += expect_miss(opt, &cache, big_adr, huge_fil);
+  failures += expect_miss(opt, &cache, huge_adr, "/short.html");
+  selftest_close(&cache);
+
+  /* An entry name past every buffer on the path must be skipped, not indexed
+     from an unterminated read -- which only the sanitizer legs see. */
+  {
+    char zippath[HTS_URLMAXSIZE];
+    /* no pair of lien_back fields reaches this key, so look it up as one */
+    static char foreign_fil[CACHE_KEY_SIZE];
+
+    slprintfbuff_clip(zippath, sizeof(zippath), "%shts-cache/new.zip",
+                      StringBuff(opt->path_log));
+    if (urlbounds_write_foreign_zip(zippath, CACHE_ENTRYNAME_SIZE - 2) != 0) {
+      fprintf(stderr, "cache-urlbounds: could not write the foreign cache\n");
+      failures++;
+    } else {
+      /* the name the loader saw, minus the "http://" the index load strips */
+      fill_str(foreign_fil, sizeof(foreign_fil), CACHE_ENTRYNAME_SIZE - 2 - 7,
+               'z');
+      selftest_open_for_read(&cache, opt);
+      failures +=
+          check_entry(opt, &cache, "example.com", "/sane.html", 200, "OK",
+                      "text/html", "utf-8", "", "", "", "", "okay", 4);
+      failures += expect_miss(opt, &cache, "", foreign_fil);
+      selftest_close(&cache);
+    }
+  }
+
+  return failures;
+}

src/htscache_selftest.h

@@ -64,6 +64,17 @@ int cache_reconcile_selftest(httrackp *opt, const char *dir);
    Returns the number of failed checks (0 = pass). */
 int cache_legacy_refused_selftest(httrackp *opt, const char *dir);
 
+/* A cache entry whose every field sits at its declared cap: the header block
+   the writer builds outgrows its fixed buffer, so this pins that the writer
+   bounds it, drops whole fields rather than halves, and still round-trips
+   everything written before the block filled. Returns the failed-check count.
+ */
+int cache_header_bounds_selftest(httrackp *opt, const char *dir);
+
+/* URLs at (and past) the cache API's length cap: store and lookup must neither
+   abort nor alias two keys. Returns the failed-check count. */
+int cache_url_bounds_selftest(httrackp *opt, const char *dir);
+
 /* Inject read-side corruption (zip byte surgery: bad size, header, deflate)
    under <dir> and assert every case degrades to STATUSCODE_INVALID without
    tainting a sibling entry. */

src/htscoremain.c

@@ -52,6 +52,13 @@ Please visit our Website: http://www.httrack.com
 #include "htsmd5.h"
 
 #include <ctype.h>
+/* hts_self_path() */
+#ifdef __APPLE__
+#include <mach-o/dyld.h>
+#endif
+#ifdef HAVE_UNISTD_H
+#include <unistd.h>
+#endif
 #if USE_BEGINTHREAD
 #ifdef _WIN32
 #include <process.h>
@@ -71,6 +78,95 @@ Please visit our Website: http://www.httrack.com
 /* Resolver */
 extern int IPV6_resolver;
 
+/* A data directory is one that carries the templates path_bin is read for. */
+static int datadir_has_templates(const char *dir) {
+  char catbuff[CATBUFF_SIZE];
+
+  return dir != NULL && *dir != '\0' &&
+         fexist(fconcat(catbuff, sizeof(catbuff), dir,
+                        "templates/index-header.html"));
+}
+
+const char *hts_self_path(char *dst, size_t dstsize) {
+#if defined(_WIN32)
+  const DWORD n = GetModuleFileNameA(NULL, dst, (DWORD) dstsize);
+
+  /* Pre-Win8 returns nSize on truncation without terminating: a full buffer
+     is a failure, not a path. */
+  return (n > 0 && (size_t) n < dstsize) ? dst : NULL;
+#elif defined(__APPLE__)
+  uint32_t n = (uint32_t) dstsize;
+
+  return _NSGetExecutablePath(dst, &n) == 0 ? dst : NULL;
+#else
+  /* Linux; anywhere else this is simply absent and argv[0] has to do. */
+  const ssize_t n = readlink("/proc/self/exe", dst, dstsize - 1);
+
+  if (n <= 0 || (size_t) n >= dstsize - 1)
+    return NULL;
+  dst[n] = '\0';
+  return dst;
+#endif
+}
+
+/* Directory part of path, trailing '/' kept, or NULL when it carries none: a
+   bare name came from a PATH lookup and locates nothing. */
+static const char *dirname_of(char *dst, size_t dstsize, const char *path) {
+  char catbuff[CATBUFF_SIZE];
+  const char *slashed, *sep;
+  size_t len;
+
+  if (path == NULL)
+    return NULL;
+  slashed = fslash(catbuff, sizeof(catbuff), path);
+  if ((sep = strrchr(slashed, '/')) == NULL)
+    return NULL;
+  len = (size_t) (sep - slashed) + 1;
+  if (len >= dstsize)
+    return NULL;
+  memcpy(dst, slashed, len);
+  dst[len] = '\0';
+  return dst;
+}
+
+void hts_resolve_datadir(char *dst, size_t dstsize, const char *selfpath,
+                         const char *builtin) {
+  /* An installed tree that was moved, then a flat one with templates/ beside
+     the binary. */
+  static const char *const layout[] = {"../share/httrack/", ""};
+  char exedir[HTS_URLMAXSIZE * 2];
+  const char *base = NULL;
+  const char *fallback;
+
+  if (!datadir_has_templates(builtin)) {
+    base = dirname_of(exedir, sizeof(exedir), selfpath);
+  }
+  if (base != NULL) {
+    size_t i;
+
+    for (i = 0; i < sizeof(layout) / sizeof(layout[0]); i++) {
+      char cand[HTS_URLMAXSIZE * 2];
+      /* snprintf, not the strlncatbuff idiom: appending to a non-empty buffer
+         aborts on overflow, and a long enough argv[0] reaches it. */
+      const int n = snprintf(cand, sizeof(cand), "%s%s", base, layout[i]);
+
+      if (n < 0 || (size_t) n >= sizeof(cand)) {
+        continue; /* truncated, so not the path we meant to probe */
+      }
+      if (datadir_has_templates(cand)) {
+        snprintf(dst, dstsize, "%s", cand);
+        return;
+      }
+    }
+  }
+  /* Windows has no compiled-in data directory, so there the executable's own
+     is all we have. */
+  fallback = (builtin != NULL && *builtin != '\0') ? builtin
+             : (base != NULL)                      ? base
+                                                   : "";
+  snprintf(dst, dstsize, "%s", fallback);
+}
+
 #define htsmain_free() do { \
   if (url != NULL) { \
     free(url); \
@@ -174,21 +270,25 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
   }
 #endif
 
-  // Binary program path?
-#ifndef HTS_HTTRACKDIR
+  // Data directory holding the HTML templates
   {
-    char catbuff[CATBUFF_SIZE];
-    char *path = fslash(catbuff, sizeof(catbuff), argv[0]);
-    char *a;
+    char datadir[HTS_URLMAXSIZE * 2];
+    char selfbuff[HTS_URLMAXSIZE * 2];
+    const char *self = hts_self_path(selfbuff, sizeof(selfbuff));
 
-    if ((a = strrchr(path, '/'))) {
-      StringCopyN(opt->path_bin, argv[0], a - path);
-    }
-  }
+#ifdef HTS_HTTRACKDIR
+    const char *const builtin = HTS_HTTRACKDIR;
 #else
-  StringCopy(opt->path_bin, HTS_HTTRACKDIR);
+    const char *const builtin = "";
 #endif
 
+    if (self == NULL && argc > 0) {
+      self = argv[0];
+    }
+    hts_resolve_datadir(datadir, sizeof(datadir), self, builtin);
+    StringCopy(opt->path_bin, datadir);
+  }
+
   /* filter CR, LF, TAB.. */
   {
     int na;

src/htscoremain.h

@@ -47,6 +47,16 @@ Please visit our Website: http://www.httrack.com
 int cmdl_opt(char *s);
 int check_path(String * s, char *defaultname);
 
+/* Absolute path of the running executable, or NULL where the OS will not say
+   and argv[0] is the only clue left. Fills dst (dstsize bytes). */
+const char *hts_self_path(char *dst, size_t dstsize);
+
+/* Write the data directory holding the HTML templates into dst (dstsize bytes,
+   NUL-terminated, trailing '/' included): builtin when it is there, else a
+   layout under selfpath's directory. selfpath may be NULL, builtin empty. */
+void hts_resolve_datadir(char *dst, size_t dstsize, const char *selfpath,
+                         const char *builtin);
+
 #endif
 
 #endif

src/htslib.c

@@ -1834,12 +1834,8 @@ int check_writeinput_t(T_SOC soc, int timeout) {
     return 0;
 }
 
-// idem, sauf qu'ici on peut choisir la taille max de données à recevoir
-// SI bufl==0 alors le buffer est censé être de 8kos, et on recoit par bloc de lignes
-// en éliminant les cr (ex: header), arrêt si double-lf
-// SI bufl==-1 alors le buffer est censé être de 8kos, et on recoit ligne par ligne
-// en éliminant les cr (ex: header), arrêt si double-lf
-// Note: les +1 dans les malloc sont dûs à l'octet nul rajouté en fin de fichier
+// Read one block: bufl is a byte count, or one of the HTS_XFREAD_* line modes.
+// Note: the +1 in the mallocs is the trailing NUL appended to the data.
 LLint http_xfread1(htsblk * r, int bufl) {
   int nl = -1;
 
@@ -1952,14 +1948,7 @@ LLint http_xfread1(htsblk * r, int bufl) {
 
     }                           // stockage disque ou mémoire
 
-  } else if (bufl == -2) {      // force reserve
-    if (r->adr == NULL) {
-      r->adr = (char *) malloct(8192);
-      r->size = 0;
-      return 0;
-    }
-    return -1;
-  } else {                      // réception d'un en-tête octet par octet
+  } else { // line modes: byte by byte, CR dropped
     int count = 256;
     int tot_nl = 0;
     int lf_detected = 0;
@@ -1969,18 +1958,16 @@ LLint http_xfread1(htsblk * r, int bufl) {
       nl = READ_INTERNAL_ERROR;
       count--;
       if (r->adr == NULL) {
-        r->adr = (char *) malloct(8192);
+        r->adr = (char *) malloct(HTS_LINE_BLOCK_SIZE);
         r->size = 0;
       }
       if (r->adr != NULL) {
-        if (r->size < 8190) {
+        if (r->size < HTS_LINE_BLOCK_SIZE - 2) {
           // lecture
           nl = hts_read(r, r->adr + r->size, 1);
           if (nl > 0) {
-            // exit if:
-            // lf detected AND already detected before
-            // or
-            // lf detected AND first character read
+            // exit on a blank line (LF seen twice, or LF as the first byte),
+            // or on the first LF in HTS_XFREAD_LINE mode
             if (*(r->adr + r->size) == 10) {
               if (lf_detected || (at_beginning) || (bufl < 0))
                 count = -1;
@@ -2045,7 +2032,7 @@ htsblk http_test(httrackp * opt, const char *adr, const char *fil, char *loc) {
 
     // tant qu'on a des données, et qu'on ne recoit pas deux LF, et que le timeout n'arrie pas
     do {
-      if (http_xfread1(&retour, 0) < 0)
+      if (http_xfread1(&retour, HTS_XFREAD_LINE_BLOCK) < 0)
         e = 1;
       else {
         if (retour.adr != NULL) {

src/htslib.h

@@ -248,6 +248,14 @@ void treathead(t_cookie * cookie, const char *adr, const char *fil, htsblk * ret
 void treatfirstline(htsblk * retour, const char *rcvd);
 
 // sous-fonctions
+/* Buffer http_xfread1() fills in its line modes, and so the ceiling on any
+   blank-line-terminated block it reads: a header section or a chunk trailer
+   section. Overrunning it fails the transfer. */
+#define HTS_LINE_BLOCK_SIZE 8192
+/* http_xfread1() read modes: a positive bufl reads at most that many raw bytes,
+   anything else reads CR-stripped lines into the HTS_LINE_BLOCK_SIZE buffer. */
+#define HTS_XFREAD_LINE_BLOCK 0 /* lines up to a blank one (header/trailer) */
+#define HTS_XFREAD_LINE (-1)    /* one line, stopping at the first LF */
 LLint http_xfread1(htsblk * r, int bufl);
 /* Cached resolver: fill out[0..count-1] with up to max addresses for iadr (in
    resolver order), returning the count (0 = does not resolve, negative-cached).

src/htssafe.h

@@ -514,6 +514,30 @@ static HTS_INLINE HTS_UNUSED HTS_CHECK_RESULT HTS_PRINTF_FUN(3, 4) hts_boolean
   return ret;
 }
 
+/**
+ * Append formatted text at dest[*used] (dest capacity size, NUL included),
+ * advancing *used past it. All-or-nothing: on overflow dest is left as it was
+ * and HTS_FALSE returned, so a record parsed back field by field never carries
+ * a half-written one.
+ */
+static HTS_INLINE HTS_UNUSED HTS_CHECK_RESULT HTS_PRINTF_FUN(4, 5) hts_boolean
+    slcatprintfbuff(char *dest, size_t size, size_t *used, const char *fmt,
+                    ...) {
+  va_list args;
+  hts_boolean fit;
+
+  assertf(dest != NULL && used != NULL && *used < size);
+  va_start(args, fmt);
+  fit = vslprintfbuff(dest + *used, size - *used, fmt, args);
+  va_end(args);
+  if (fit) {
+    *used += strlen(dest + *used);
+  } else {
+    dest[*used] = '\0';
+  }
+  return fit;
+}
+
 /**
  * slprintfbuff() for diagnostics quoting remote or client text, which are
  * meant to be clipped: nothing to act on, hence not HTS_CHECK_RESULT. A (void)

src/htsselftest.c

@@ -54,6 +54,7 @@ Please visit our Website: http://www.httrack.com
 #include "htsdns_selftest.h"
 #include "htscharset.h"
 #include "htscmdline.h"
+#include "htscoremain.h"
 #include "htsencoding.h"
 #include "htsftp.h"
 #include "htsmd5.h"
@@ -79,6 +80,7 @@ Please visit our Website: http://www.httrack.com
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
+#include <wchar.h>
 #ifndef _WIN32
 #include <sys/socket.h>
 #include <unistd.h>
@@ -2861,6 +2863,30 @@ static int st_cache_writefail(httrackp *opt, int argc, char **argv) {
   return err;
 }
 
+static int st_cache_hdrbounds(httrackp *opt, int argc, char **argv) {
+  int err;
+
+  if (argc < 1) {
+    fprintf(stderr, "cache-hdrbounds: needs a directory\n");
+    return 1;
+  }
+  err = cache_header_bounds_selftest(opt, argv[0]);
+  printf("cache-hdrbounds: %s\n", err ? "FAIL" : "OK");
+  return err;
+}
+
+static int st_cache_urlbounds(httrackp *opt, int argc, char **argv) {
+  int err;
+
+  if (argc < 1) {
+    fprintf(stderr, "cache-urlbounds: needs a directory\n");
+    return 1;
+  }
+  err = cache_url_bounds_selftest(opt, argv[0]);
+  printf("cache-urlbounds: %s\n", err ? "FAIL" : "OK");
+  return err;
+}
+
 static int st_cache_corrupt(httrackp *opt, int argc, char **argv) {
   int err;
 
@@ -3314,6 +3340,104 @@ static int st_makeindex(httrackp *opt, int argc, char **argv) {
   return 0;
 }
 
+static void datadir_expect(const char *selfpath, const char *builtin,
+                           const char *expect) {
+  char got[HTS_URLMAXSIZE * 2];
+
+  hts_resolve_datadir(got, sizeof(got), selfpath, builtin);
+  if (strcmp(got, expect) != 0) {
+    fprintf(stderr,
+            "datadir: self=%s builtin=\"%s\" gave \"%s\", expected \"%s\"\n",
+            selfpath != NULL ? selfpath : "(null)", builtin, got, expect);
+  }
+  assertf(strcmp(got, expect) == 0);
+}
+
+// -#test=datadir <dir>: a relocated tree must find its own templates instead of
+// silently falling back to the built-in ones (#894). argv[0] is writable.
+static int st_datadir(httrackp *opt, int argc, char **argv) {
+  char path[HTS_URLMAXSIZE];
+  char self[HTS_URLMAXSIZE];
+  char expect[HTS_URLMAXSIZE * 2];
+  char installed[HTS_URLMAXSIZE];
+  char gone[HTS_URLMAXSIZE];
+  /* Each holds a templates/index-header.html, the file path_bin is read for.
+     nest/ keeps the flat case away from the installed share/httrack above. */
+  static const char *const dirs[] = {"share/httrack", "bin", "nest/flat"};
+  size_t i;
+
+  (void) opt;
+  assertf(argc >= 1);
+
+  /* argv[0] is a fallback: what the engine actually resolves from is this. */
+  assertf(hts_self_path(path, sizeof(path)) != NULL);
+  assertf(fexist(path));
+  /* Too small for any real path, so the truncation guard must refuse. */
+  assertf(hts_self_path(path, 2) == NULL);
+
+  for (i = 0; i < sizeof(dirs) / sizeof(dirs[0]); i++) {
+    FILE *fp;
+
+    snprintf(path, sizeof(path), "%s/%s/templates/", argv[0], dirs[i]);
+    assertf(structcheck(path) == 0);
+    snprintf(path, sizeof(path), "%s/%s/templates/index-header.html", argv[0],
+             dirs[i]);
+    fp = fopen(path, "wb");
+    assertf(fp != NULL);
+    fclose(fp);
+  }
+  snprintf(installed, sizeof(installed), "%s/share/httrack/", argv[0]);
+  snprintf(gone, sizeof(gone), "%s/gone/", argv[0]);
+
+  /* A moved install: bin/ carries templates too, so this pins the order. */
+  snprintf(self, sizeof(self), "%s/bin/httrack", argv[0]);
+  snprintf(expect, sizeof(expect), "%s/bin/../share/httrack/", argv[0]);
+  datadir_expect(self, gone, expect);
+
+  /* The compiled-in path still wins when it exists. */
+  datadir_expect(self, installed, installed);
+
+  /* Flat layout: templates/ sits beside the binary. */
+  snprintf(self, sizeof(self), "%s/nest/flat/httrack", argv[0]);
+  snprintf(expect, sizeof(expect), "%s/nest/flat/", argv[0]);
+  datadir_expect(self, gone, expect);
+
+  /* Nothing to derive from, or nothing found: the compiled-in path stands. */
+  datadir_expect("httrack", gone, gone);
+  datadir_expect(NULL, gone, gone);
+  snprintf(self, sizeof(self), "%s/nowhere/deep/httrack", argv[0]);
+  datadir_expect(self, gone, gone);
+
+  /* No compiled-in path, as on Windows: the executable's own directory. */
+  snprintf(self, sizeof(self), "%s/nowhere/deep/httrack", argv[0]);
+  snprintf(expect, sizeof(expect), "%s/nowhere/deep/", argv[0]);
+  datadir_expect(self, "", expect);
+  datadir_expect("httrack", "", "");
+
+  /* A directory part too long for the layout suffix to be appended must clip,
+     not abort: appending to a non-empty buffer is the *_safe_ abort path. */
+  {
+    /* Long enough that dirname + "../share/httrack/" overflows the candidate
+       buffer, short enough that the dirname itself still fits. */
+    const size_t dirlen = HTS_URLMAXSIZE * 2 - 8;
+    char huge[HTS_URLMAXSIZE * 3];
+    char got[HTS_URLMAXSIZE * 2];
+    size_t n;
+
+    huge[0] = '/';
+    for (n = 1; n < dirlen - 1; n++) {
+      huge[n] = 'a';
+    }
+    huge[dirlen - 1] = '/';
+    memcpy(huge + dirlen, "httrack", sizeof("httrack"));
+    hts_resolve_datadir(got, sizeof(got), huge, gone);
+    assertf(strcmp(got, gone) == 0);
+  }
+
+  printf("datadir self-test OK\n");
+  return 0;
+}
+
 // hts_buildtopindex() writes a system-charset name into a charset=utf-8 doc: on
 // Windows the gifs land in a mangled twin dir (#217) and a listed name renders
 // as mojibake (#216). Both must come out utf-8. argv[0] is writable.
@@ -6468,6 +6592,88 @@ static int st_refetchbackup(httrackp *opt, int argc, char **argv) {
   return err;
 }
 
+// -#test=spoolname <dir>: a frozen backlog slot must spool inside ~hts-tmp, not
+// beside the mirrored file where a site serving <path>.tmp collides (#859).
+static int st_spoolname(httrackp *opt, int argc, char **argv) {
+  char BIGSTK got[HTS_URLMAXSIZE * 2 + 32];
+  char BIGSTK want[HTS_URLMAXSIZE * 2 + 32];
+  char BIGSTK save[HTS_URLMAXSIZE * 2];
+  int err = 0;
+
+  if (argc < 1) {
+    fprintf(stderr, "spoolname: needs a writable base dir\n");
+    return 1;
+  }
+
+  /* named: the spool lands in the save name's own ~hts-tmp, which no URL can
+     spell since url_savename() maps '~' to '_' */
+  snprintf(save, sizeof(save), "%s/sub/page.html", argv[0]);
+  snprintf(want, sizeof(want), "%s/sub/~hts-tmp/page.html.tmp", argv[0]);
+  opt->getmode = 1;
+  if (!back_spoolname(opt, save, got, sizeof(got))) {
+    fprintf(stderr, "spoolname: naming failed for %s\n", save);
+    err++;
+  } else if (strcmp(got, want) != 0) {
+    fprintf(stderr, "spoolname: got %s, want %s\n", got, want);
+    err++;
+  }
+
+  /* pin the pre-#859 name as forbidden too: a site serving sub/page.html.tmp
+     was mirrored straight onto it */
+  snprintf(want, sizeof(want), "%s.tmp", save);
+  if (strcmp(got, want) == 0) {
+    fprintf(stderr, "spoolname: still spooling into the mirror namespace\n");
+    err++;
+  }
+
+  /* -p0 keeps no save name, so the spool counts inside path_html's ~hts-tmp */
+  {
+    char BIGSTK base[HTS_URLMAXSIZE * 2];
+
+    snprintf(base, sizeof(base), "%s/", argv[0]);
+    StringCopy(opt->path_html_utf8, base);
+    opt->getmode = 0;
+    opt->state.tmpnameid = 7;
+    snprintf(want, sizeof(want), "%s/~hts-tmp/tmpfile7.tmp", argv[0]);
+    if (!back_spoolname(opt, "", got, sizeof(got))) {
+      fprintf(stderr, "spoolname: naming failed under -p0\n");
+      err++;
+    } else if (strcmp(got, want) != 0) {
+      fprintf(stderr, "spoolname: -p0 got %s, want %s\n", got, want);
+      err++;
+    }
+    if (opt->state.tmpnameid != 8) {
+      fprintf(stderr, "spoolname: -p0 did not consume a tmpnameid\n");
+      err++;
+    }
+  }
+
+  /* with no -O, path_html_utf8 is empty and the spool must stay relative to
+     the working directory; a separator of our own would put it in / */
+  StringCopy(opt->path_html_utf8, "");
+  opt->getmode = 0;
+  opt->state.tmpnameid = 0;
+  if (!back_spoolname(opt, "", got, sizeof(got))) {
+    fprintf(stderr, "spoolname: naming failed with no output directory\n");
+    err++;
+  } else if (strcmp(got, "~hts-tmp/tmpfile0.tmp") != 0) {
+    fprintf(stderr, "spoolname: no -O gave %s, want ~hts-tmp/tmpfile0.tmp\n",
+            got);
+    err++;
+  }
+
+  /* too long must empty dest, not hand back a truncated name landing
+     somewhere real */
+  opt->getmode = 1;
+  if (back_spoolname(opt, save, got, 8) || got[0] != '\0') {
+    fprintf(stderr, "spoolname: an overlong name was not rejected\n");
+    err++;
+  }
+
+  printf("spoolname: %s\n", err ? "FAIL" : "OK");
+  return err;
+}
+
 // -#test=direnum <dir>: enumerate a long+non-ASCII directory via the
 // opendir/readdir wrappers; children must round-trip as UTF-8 (#133,#630).
 static int st_direnum(httrackp *opt, int argc, char **argv) {
@@ -7450,6 +7656,149 @@ static int st_rtrim(httrackp *opt, int argc, char **argv) {
   return err;
 }
 
+/* Format LEN bytes of EXPECTED into S as two arguments, and check what came
+   back. HEAD and TAIL are scratch buffers of at least LEN+1 bytes. */
+static int strsprintf_case(String *s, const char *expected, size_t len,
+                           char *head, char *tail) {
+  const size_t half = len / 2;
+
+  memcpy(head, expected, half);
+  head[half] = '\0';
+  memcpy(tail, expected + half, len - half);
+  tail[len - half] = '\0';
+  StringSprintf(*s, "%s%s", head, tail);
+  return StringLength(*s) == len &&
+         memcmp(StringBuff(*s), expected, len) == 0 &&
+         StringBuff(*s)[len] == '\0';
+}
+
+/* StringSprintf_ stores the terminator at buffer[ret], so its `ret < capacity`
+   guard is off by one byte at the exact fill: an output whose length equals the
+   capacity writes past the allocation (#836). The lengths that reach it are the
+   capacities themselves, floored at 256 and doubling from there. */
+static int st_strsprintf(httrackp *opt, int argc, char **argv) {
+  static const size_t caps[] = {256, 512, 1024, 2048};
+
+  enum { maxLen = 2100 };
+
+  char *expected = malloct(maxLen + 1);
+  char *head = malloct(maxLen + 1);
+  char *tail = malloct(maxLen + 1);
+  String reused = STRING_EMPTY;
+  size_t i, len;
+  int err = 0;
+
+  (void) opt;
+  (void) argc;
+  (void) argv;
+
+  if (expected == NULL || head == NULL || tail == NULL) {
+    printf("strsprintf self-test: FAIL (out of memory)\n");
+    return 1;
+  }
+  for (i = 0; i < maxLen; i++)
+    expected[i] = (char) ('a' + (i % 26));
+  expected[maxLen] = '\0';
+
+  /* one call on a String whose capacity is pinned to the boundary, so len ==
+     capacity is reached exactly once per boundary */
+  for (i = 0; !err && i < sizeof(caps) / sizeof(caps[0]); i++) {
+    for (len = caps[i] - 3; !err && len <= caps[i] + 3; len++) {
+      String s = STRING_EMPTY;
+
+      StringRoomTotal(s, caps[i]);
+      if (StringCapacity(s) != caps[i]) {
+        printf("  FAIL: capacity %u pinned to %u\n", (unsigned) caps[i],
+               (unsigned) StringCapacity(s));
+        err = 1;
+      } else if (!strsprintf_case(&s, expected, len, head, tail)) {
+        printf("  FAIL: length %u at capacity %u\n", (unsigned) len,
+               (unsigned) caps[i]);
+        err = 1;
+      }
+      StringFree(s);
+    }
+  }
+
+  /* the same String reused: its capacity grows under it between calls, and a
+     shorter output must not leave the previous one behind */
+  for (len = 0; !err && len <= maxLen; len++) {
+    if (!strsprintf_case(&reused, expected, len, head, tail)) {
+      printf("  FAIL: growing length %u\n", (unsigned) len);
+      err = 1;
+    }
+  }
+  for (len = maxLen + 1; !err && len-- > 0;) {
+    if (!strsprintf_case(&reused, expected, len, head, tail)) {
+      printf("  FAIL: shrinking length %u\n", (unsigned) len);
+      err = 1;
+    }
+  }
+  StringFree(reused);
+
+  /* The give-up path: an argument libc cannot convert fails at every capacity,
+     so the retry loop climbs to STRING_SPRINTF_MAX and then empties the
+     String. Probe libc first -- a platform that formats an unpaired surrogate
+     without faulting never reaches the path. */
+  {
+    static const wchar_t bad[] = {(wchar_t) 0xd800, 0};
+    char probe[32];
+
+    if (snprintf(probe, sizeof(probe), "%ls", bad) < 0) {
+      String s = STRING_EMPTY;
+
+      StringCopy(s, "leftover");
+      StringSprintf(s, "%ls", bad);
+      if (StringNotEmpty(s) || StringBuff(s) == NULL ||
+          StringBuff(s)[0] != '\0') {
+        printf("  FAIL: a failed conversion left %u bytes behind\n",
+               (unsigned) StringLength(s));
+        err = 1;
+      }
+      StringFree(s);
+    } else { /* stderr: test 150 pins stdout to the one-line verdict */
+      fprintf(stderr, "  (skipped: this libc formats an unconvertible wide "
+                      "string)\n");
+    }
+  }
+
+  /* StringSprintf empties the String when it gives up, and the WebDAV
+     enumeration pops the trailing '/' right after: on an empty String an
+     unguarded pop would wrap the unsigned length and write off the end. */
+  {
+    String never = STRING_EMPTY;
+    String cleared = STRING_EMPTY;
+
+    StringPopRight(never); /* never written to: buffer_ is still NULL */
+    if (StringLength(never) != 0 || StringBuff(never) != NULL) {
+      printf("  FAIL: pop on an unallocated String\n");
+      err = 1;
+    }
+    StringClear(cleared);
+    StringPopRight(cleared);
+    if (StringLength(cleared) != 0 || StringBuff(cleared)[0] != '\0') {
+      printf("  FAIL: pop on an emptied String\n");
+      err = 1;
+    }
+    /* control: the guard must not swallow a pop that has a byte to drop */
+    StringSprintf(cleared, "ab");
+    StringPopRight(cleared);
+    if (StringLength(cleared) != 1 || strcmp(StringBuff(cleared), "a") != 0) {
+      printf("  FAIL: pop on a non-empty String\n");
+      err = 1;
+    }
+    StringFree(never);
+    StringFree(cleared);
+  }
+
+  freet(expected);
+  freet(head);
+  freet(tail);
+
+  printf("strsprintf self-test: %s\n", err ? "FAIL" : "OK");
+  return err;
+}
+
 /* ------------------------------------------------------------ */
 /* Registry: name -> handler, with a usage hint and a one-line description. */
 /* ------------------------------------------------------------ */
@@ -7504,6 +7853,8 @@ static const struct selftest_entry {
     {"hashtable", "<count|file>", "coucal hashtable stress test", st_hashtable},
     {"strsafe", "[overflow|overflow-buff|overflow-src [str]]",
      "bounded string-op self-test", st_strsafe},
+    {"strsprintf", "", "StringSprintf grows to fit at every capacity boundary",
+     st_strsprintf},
     {"copyopt", "", "copy_htsopt option-copy self-test", st_copyopt},
     {"lastchar", "",
      "last-char helpers never index before the buffer (#770, #781, #821)",
@@ -7569,6 +7920,12 @@ static const struct selftest_entry {
      st_cache_legacy},
     {"cache-corrupt", "<dir>", "cache read-side corruption self-test",
      st_cache_corrupt},
+    {"cache-hdrbounds", "<dir>",
+     "cache header block must stay bounded at max-length fields",
+     st_cache_hdrbounds},
+    {"cache-urlbounds", "<dir>",
+     "cache store and lookup at max-length URLs must not abort or alias",
+     st_cache_urlbounds},
     {"zip-repair-shift", "<dir>",
      "cache zip-repair header read must not overflow a signed shift",
      st_zip_repair_shift},
@@ -7585,6 +7942,9 @@ static const struct selftest_entry {
     {"topindex", "[dir]",
      "hts_buildtopindex charset handling of a non-ASCII project dir",
      st_topindex},
+    {"datadir", "<dir>",
+     "data directory resolution: compiled-in path, then the executable's tree",
+     st_datadir},
     {"inplace-escape", "", "inplace_escape_* vs escape_* equivalence self-test",
      st_inplace_escape},
     {"escape-room", "", "HT_ADD_HTMLESCAPED* reservation-factor self-test",
@@ -7631,6 +7991,8 @@ static const struct selftest_entry {
     {"refetchbackup", "<dir>",
      "the re-fetch backup always leaves a copy, and stays out of the mirror",
      st_refetchbackup},
+    {"spoolname", "<dir>",
+     "a frozen backlog slot spools outside the mirror namespace", st_spoolname},
     {"direnum", "<dir>",
      "enumerate a long+non-ASCII directory through opendir/readdir",
      st_direnum},

src/htsstrings.h

@@ -36,6 +36,9 @@ Please visit our Website: http://www.httrack.com
 #define HTS_STRINGS_DEFSTATIC
 
 /* System definitions. */
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
 #include <string.h>
 
 /* GCC extension */
@@ -91,10 +94,6 @@ struct String {
 
 #define STRING_FREE(BUFF) free(BUFF)
 #endif
-#ifndef STRING_ASSERT
-#include <assert.h>
-#define STRING_ASSERT(EXP) assert(EXP)
-#endif
 
 /** Initializer for an empty String (NULL buffer). Use to declare or reset. **/
 #define STRING_EMPTY {(char *) NULL, 0, 0}
@@ -127,27 +126,44 @@ struct String {
 /** Byte POS positions from the end (read/write). POS==1 is the last byte. **/
 #define StringRightRW(BLK, POS) (StringBuffRW(BLK)[StringLength(BLK) - POS])
 
-/** Drop the last byte and re-terminate. Undefined if the String is empty
-    (no length check; would underflow). **/
+/** Drop the last byte and re-terminate. No-op on an empty String: the length
+    is unsigned, so an unguarded pop would wrap it and write off the end. **/
 #define StringPopRight(BLK)                                                    \
   do {                                                                         \
-    StringBuffRW(BLK)[--StringLength(BLK)] = '\0';                             \
+    if (StringLength(BLK) > 0) {                                               \
+      StringBuffRW(BLK)[--StringLength(BLK)] = '\0';                           \
+    }                                                                          \
   } while (0)
 
+/** Terminate on allocation failure. The String API returns void throughout, so
+    there is no channel to report it on, and continuing would hand the caller a
+    NULL buffer to write into. **/
+HTS_STATIC void StringOom_(size_t size) {
+  fprintf(stderr, "String: out of memory allocating %lu bytes\n",
+          (unsigned long) size);
+  fflush(stderr); /* abort() flushes nothing; Windows buffers a piped stderr */
+  abort();
+}
+
+/** What to do when an allocation of SIZE bytes fails; overridable. **/
+#ifndef STRING_OOM
+#define STRING_OOM(SIZE) StringOom_(SIZE)
+#endif
+
 /** Grow so capacity_ >= CAPACITY (total bytes, including the NUL). May realloc
-    (invalidating prior buffer pointers); aborts via STRING_ASSERT on OOM.
-    Never shrinks. **/
+    (invalidating prior buffer pointers); aborts on OOM. Never shrinks. **/
 #define StringRoomTotal(BLK, CAPACITY)                                         \
   do {                                                                         \
     const size_t capacity_ = (size_t) (CAPACITY);                              \
     while ((BLK).capacity_ < capacity_) {                                      \
-      if ((BLK).capacity_ < 16) {                                              \
-        (BLK).capacity_ = 16;                                                  \
-      } else {                                                                 \
-        (BLK).capacity_ *= 2;                                                  \
+      const size_t newcap_ = (BLK).capacity_ < 16 ? 16 : (BLK).capacity_ * 2;  \
+      char *const buff_ = STRING_REALLOC((BLK).buffer_, newcap_);              \
+                                                                               \
+      if (buff_ == NULL) {                                                     \
+        STRING_OOM(newcap_);                                                   \
       }                                                                        \
-      (BLK).buffer_ = STRING_REALLOC((BLK).buffer_, (BLK).capacity_);          \
-      STRING_ASSERT((BLK).buffer_ != NULL);                                    \
+      (BLK).buffer_ = buff_;                                                   \
+      (BLK).capacity_ = newcap_;                                               \
     }                                                                          \
   } while (0)
 
@@ -165,6 +181,46 @@ HTS_STATIC char *StringBuffN_(String *blk, int size) {
   return StringBuffRW(*blk);
 }
 
+/** Ceiling on the pre-C99 doubling search below; BLK is emptied past it. **/
+#define STRING_SPRINTF_MAX ((size_t) 16 * 1024 * 1024)
+
+/** Replace BLK's contents with the formatted output, growing to fit, so no
+    fixed reserve has to bound an argument carrying remote input. An argument
+    may not point into BLK's own buffer, which is reallocated here.
+    Leaves BLK empty if the output cannot be produced (a conversion error, or
+    past STRING_SPRINTF_MAX): callers must not assume a non-empty result. **/
+#define StringSprintf(BLK, ...) StringSprintf_(&(BLK), __VA_ARGS__)
+
+HTS_STATIC HTS_PRINTF_FUN(2, 3) void StringSprintf_(String *blk,
+                                                    const char *fmt, ...) {
+  size_t capacity = StringCapacity(*blk) > 256 ? StringCapacity(*blk) : 256;
+
+  for (;;) {
+    va_list args;
+    int ret;
+
+    StringRoomTotal(*blk, capacity);
+    va_start(args, fmt);
+    ret = vsnprintf(StringBuffRW(*blk), capacity, fmt, args);
+    va_end(args);
+    if (ret >= 0 && (size_t) ret < capacity) {
+      StringBuffRW(*blk)[ret] = '\0';
+      StringLength(*blk) = (size_t) ret;
+      return;
+    }
+    if (ret >= 0) {
+      capacity = (size_t) ret + 1; /* C99 said what it needs */
+    } else if (capacity < STRING_SPRINTF_MAX) {
+      capacity *= 2; /* pre-C99 msvcrt only says "too small" */
+    } else {
+      /* a conversion error returns -1 too, and no capacity ever fixes that */
+      StringBuffRW(*blk)[0] = '\0';
+      StringLength(*blk) = 0;
+      return;
+    }
+  }
+}
+
 /** Zero the fields (NULL buffer, no allocation). Use on an uninitialized
     String only; does NOT free an existing buffer (use StringFree to reset
     an owned one), so calling it on a live String leaks. **/

src/proxy/main.c

@@ -37,9 +37,33 @@ Please visit our Website: http://www.httrack.com
 #include "htsbase.h"
 #include "htsnet.h"
 #include "htslib.h"
+#include "coucal.h"
 #include "store.h"
 #include "proxytrack.h"
 
+/* HTS_LOG, the engine's own debug switch. */
+static hts_boolean proxytrack_verbose = HTS_FALSE;
+
+/* Without a handler coucal writes to stderr itself, prefixing the table
+   address; the per-enumeration statistics summary then lands on the operator's
+   console once per WebDAV request (#918). */
+static void proxytrack_coucal_loghandler(coucal_opaque arg,
+                                         coucal_loglevel level,
+                                         const char *format, va_list args) {
+  const char *severity;
+
+  (void) arg;
+  if (level <= coucal_log_critical) {
+    severity = CRITICAL;
+  } else if (level <= coucal_log_warning) {
+    severity = WARNING;
+  } else {
+    /* not the DEBUG macro: it is NULL outside a debug build, voiding HTS_LOG */
+    severity = proxytrack_verbose ? "debug" : NULL;
+  }
+  proxytrack_vprint_log(severity, format, args);
+}
+
 #ifndef _WIN32
 #include <signal.h>
 static void sig_brpipe(int code) {
@@ -90,6 +114,17 @@ int main(int argc, char *argv[]) {
   }
 #endif
 
+  /* Before the first table is built. */
+  {
+    const char *const dbg_env = getenv("HTS_LOG");
+    int level = 0;
+
+    if (dbg_env != NULL && sscanf(dbg_env, "%d", &level) == 1 && level > 0) {
+      proxytrack_verbose = HTS_TRUE;
+    }
+  }
+  coucal_set_global_assert_handler(proxytrack_coucal_loghandler, NULL);
+
   /* Args */
   printf("ProxyTrack %s, build proxies upon HTTrack Website Copier Archives\n",
          PROXYTRACK_VERSION);

src/proxy/proxytrack.c

@@ -548,29 +548,37 @@ static void proxytrack_add_DAV_Item(String * item, String * buff,
         name = "Default Document for the Folder";
     }
 
-    StringRoom(*item, 1024);
-    sprintf(StringBuffRW(*item),
-            "<response xmlns=\"DAV:\">\r\n" "<href>/webdav%s%s</href>\r\n"
-            "<propstat>\r\n" "<prop>\r\n" "<displayname>%s</displayname>\r\n"
-            "<iscollection>%d</iscollection>\r\n"
-            "<haschildren>%d</haschildren>\r\n" "<isfolder>%d</isfolder>\r\n"
-            "<resourcetype>%s</resourcetype>\r\n"
-            "<creationdate>%d-%02d-%02dT%02d:%02d:%02dZ</creationdate>\r\n"
-            "<getlastmodified>%s</getlastmodified>\r\n"
-            "<supportedlock></supportedlock>\r\n" "<lockdiscovery/>\r\n"
-            "<getcontenttype>%s</getcontenttype>\r\n"
-            "<getcontentlength>%d</getcontentlength>\r\n"
-            "<isroot>%d</isroot>\r\n" "</prop>\r\n"
-            "<status>HTTP/1.1 200 OK</status>\r\n" "</propstat>\r\n"
-            "</response>\r\n",
-            /* */
-            (StringBuff(*buff)[0] == '/') ? "" : "/", StringBuff(*buff), name,
-            isDir ? 1 : 0, isDir ? 1 : 0, isDir ? 1 : 0,
-            isDir ? "<collection/>" : "", timetm->tm_year + 1900,
-            timetm->tm_mon + 1, timetm->tm_mday, timetm->tm_hour,
-            timetm->tm_min, timetm->tm_sec, tms,
-            isDir ? "httpd/unix-directory" : mime, (int) size, isRoot ? 1 : 0);
-    StringLength(*item) = (int) strlen(StringBuff(*item));
+    /* The path lands here twice and escapexml() expands '&' fivefold, so no
+       fixed reserve bounds it (#836). */
+    StringSprintf(
+        *item,
+        "<response xmlns=\"DAV:\">\r\n"
+        "<href>/webdav%s%s</href>\r\n"
+        "<propstat>\r\n"
+        "<prop>\r\n"
+        "<displayname>%s</displayname>\r\n"
+        "<iscollection>%d</iscollection>\r\n"
+        "<haschildren>%d</haschildren>\r\n"
+        "<isfolder>%d</isfolder>\r\n"
+        "<resourcetype>%s</resourcetype>\r\n"
+        "<creationdate>%d-%02d-%02dT%02d:%02d:%02dZ</creationdate>\r\n"
+        "<getlastmodified>%s</getlastmodified>\r\n"
+        "<supportedlock></supportedlock>\r\n"
+        "<lockdiscovery/>\r\n"
+        "<getcontenttype>%s</getcontenttype>\r\n"
+        "<getcontentlength>%d</getcontentlength>\r\n"
+        "<isroot>%d</isroot>\r\n"
+        "</prop>\r\n"
+        "<status>HTTP/1.1 200 OK</status>\r\n"
+        "</propstat>\r\n"
+        "</response>\r\n",
+        /* */
+        (StringBuff(*buff)[0] == '/') ? "" : "/", StringBuff(*buff), name,
+        isDir ? 1 : 0, isDir ? 1 : 0, isDir ? 1 : 0,
+        isDir ? "<collection/>" : "", timetm->tm_year + 1900,
+        timetm->tm_mon + 1, timetm->tm_mday, timetm->tm_hour, timetm->tm_min,
+        timetm->tm_sec, tms, isDir ? "httpd/unix-directory" : mime, (int) size,
+        isRoot ? 1 : 0);
   }
 }
 
@@ -721,11 +729,8 @@ static PT_Element proxytrack_process_DAV_Request(PT_Indexes indexes,
     }
 
     /* Form response */
-    StringRoom(response, 1024);
-    sprintf(StringBuffRW(response),
-            "<?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n"
-            "<multistatus xmlns=\"DAV:\">\r\n");
-    StringLength(response) = (int) strlen(StringBuff(response));
+    StringSprintf(response, "<?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n"
+                            "<multistatus xmlns=\"DAV:\">\r\n");
     /* */
 
     /* Root */
@@ -739,7 +744,6 @@ static PT_Element proxytrack_process_DAV_Request(PT_Indexes indexes,
     if (depth > 0) {
       time_t timestampRep = (time_t) - 1;
       const char *prefix = StringBuff(url);
-      unsigned int prefixLen = (unsigned int) strlen(prefix);
       char **list = PT_Enumerate(indexes, prefix, 0);
 
       if (list != NULL) {
@@ -752,15 +756,13 @@ static PT_Element proxytrack_process_DAV_Request(PT_Indexes indexes,
             int thisIsDir = (hts_lastchar(thisUrl) == '/') ? 1 : 0;
 
             /* Item URL */
-            StringRoom(itemUrl,
-                       thisUrlLen + prefixLen + sizeof("/webdav/") + 1);
-            StringClear(itemUrl);
-            sprintf(StringBuffRW(itemUrl), "/%s/%s", prefix, thisUrl);
-            if (!thisIsDir)
-              StringLength(itemUrl) = (int) strlen(StringBuff(itemUrl));
-            else
-              StringLength(itemUrl) = (int) strlen(StringBuff(itemUrl)) - 1;
-            StringBuffRW(itemUrl)[StringLength(itemUrl)] = '\0';
+            StringSprintf(itemUrl, "/%s/%s", prefix, thisUrl);
+            if (!StringNotEmpty(itemUrl)) { /* formatting gave up: unnameable */
+              continue;
+            }
+            if (thisIsDir) { /* drop the trailing '/' */
+              StringPopRight(itemUrl);
+            }
 
             if (thisIsDir == isDir) {
               size_t size = 0;
@@ -827,8 +829,6 @@ static PT_Element proxytrack_process_DAV_Request(PT_Indexes indexes,
     strcpybuff(elt->msg, "Multi-Status");
     StringFree(response);
 
-    fprintf(stderr, "RESPONSE:\n%s\n", elt->adr);
-
     return elt;
   }
   return NULL;
@@ -1037,18 +1037,18 @@ static void proxytrack_process_HTTP(PT_Indexes indexes, T_SOC soc_c) {
         const char *options = "GET, HEAD, OPTIONS, POST, PROPFIND, TRACE" ", MKCOL, DELETE, PUT";       /* Not supported */
 
         msgCode = HTTP_OK;
-        StringRoom(headers, 8192);
-        sprintf(StringBuffRW(headers),
-                "HTTP/1.1 %d %s\r\n" "DAV: 1, 2\r\n" "MS-Author-Via: DAV\r\n"
-                "Cache-Control: private\r\n" "Allow: %s\r\n", msgCode,
-                GetHttpMessage(msgCode), options);
-        StringLength(headers) = (int) strlen(StringBuff(headers));
+        StringSprintf(headers,
+                      "HTTP/1.1 %d %s\r\n"
+                      "DAV: 1, 2\r\n"
+                      "MS-Author-Via: DAV\r\n"
+                      "Cache-Control: private\r\n"
+                      "Allow: %s\r\n",
+                      msgCode, GetHttpMessage(msgCode), options);
       } else if (strcasecmp(command, "propfind") == 0) {
         if (davDepth > 1) {
           msgCode = 403;
           msgError = "DAV Depth Limit Forbidden";
         } else {
-          fprintf(stderr, "DEBUG: DAV-DATA=<%s>\n", StringBuff(davRequest));
           listRequest = 2;      /* propfind */
         }
       } else if (strcasecmp(command, "mkcol") == 0
@@ -1141,11 +1141,9 @@ static void proxytrack_process_HTTP(PT_Indexes indexes, T_SOC soc_c) {
                proxytrack_process_DAV_Request(indexes, StringBuff(url),
                                               davDepth)) != NULL) {
             msgCode = element->statuscode;
-            StringRoom(davHeaders, 1024);
-            sprintf(StringBuffRW(davHeaders),
-                    "DAV: 1, 2\r\n" "MS-Author-Via: DAV\r\n"
-                    "Cache-Control: private\r\n");
-            StringLength(davHeaders) = (int) strlen(StringBuff(davHeaders));
+            StringSprintf(davHeaders, "DAV: 1, 2\r\n"
+                                      "MS-Author-Via: DAV\r\n"
+                                      "Cache-Control: private\r\n");
           }
         }
 #endif
@@ -1164,45 +1162,46 @@ static void proxytrack_process_HTTP(PT_Indexes indexes, T_SOC soc_c) {
           }
         }
         if (element != NULL) {
+          /* lifted out of the format: a directive inside a macro argument list
+             is undefined, and MSVC rejects it */
+#ifndef NO_WEBDAV
+          const char *const davPart = StringBuff(davHeaders);
+#else
+          const char *const davPart = "";
+#endif
+
           msgCode = element->statuscode;
-          StringRoom(headers, 8192);
-          sprintf(StringBuffRW(headers),
-                  "HTTP/1.1 %d %s\r\n"
-#ifndef NO_WEBDAV
-                  "%s"
-#endif
-                  "Content-Type: %s%s%s%s\r\n"
-                  "%s%s%s"
-                  "%s%s%s"
-                  "%s%s%s",
-                  /* */
-                  msgCode, element->msg,
-#ifndef NO_WEBDAV
-                  /* DAV */
-                  StringBuff(davHeaders),
-#endif
-                  /* Content-type: foo; [ charset=bar ] */
-                  hts_effective_mime(element->contenttype),
-                  ((element->charset[0]) ? "; charset=\"" : ""),
-                  element->charset, ((element->charset[0]) ? "\"" : ""),
-                  /* location */
-                  ((element->location != NULL && element->location[0])
-                       ? "Location: "
-                       : ""),
-                  ((element->location != NULL && element->location[0])
-                       ? element->location
-                       : ""),
-                  ((element->location != NULL && element->location[0]) ? "\r\n"
-                                                                       : ""),
-                  /* last-modified */
-                  ((element->lastmodified[0]) ? "Last-Modified: " : ""),
-                  ((element->lastmodified[0]) ? element->lastmodified : ""),
-                  ((element->lastmodified[0]) ? "\r\n" : ""),
-                  /* etag */
-                  ((element->etag[0]) ? "ETag: " : ""),
-                  ((element->etag[0]) ? element->etag : ""),
-                  ((element->etag[0]) ? "\r\n" : ""));
-          StringLength(headers) = (int) strlen(StringBuff(headers));
+          StringSprintf(
+              headers,
+              "HTTP/1.1 %d %s\r\n"
+              "%s"
+              "Content-Type: %s%s%s%s\r\n"
+              "%s%s%s"
+              "%s%s%s"
+              "%s%s%s",
+              /* */
+              msgCode, element->msg, davPart,
+              /* Content-type: foo; [ charset=bar ] */
+              hts_effective_mime(element->contenttype),
+              ((element->charset[0]) ? "; charset=\"" : ""), element->charset,
+              ((element->charset[0]) ? "\"" : ""),
+              /* location */
+              ((element->location != NULL && element->location[0])
+                   ? "Location: "
+                   : ""),
+              ((element->location != NULL && element->location[0])
+                   ? element->location
+                   : ""),
+              ((element->location != NULL && element->location[0]) ? "\r\n"
+                                                                   : ""),
+              /* last-modified */
+              ((element->lastmodified[0]) ? "Last-Modified: " : ""),
+              ((element->lastmodified[0]) ? element->lastmodified : ""),
+              ((element->lastmodified[0]) ? "\r\n" : ""),
+              /* etag */
+              ((element->etag[0]) ? "ETag: " : ""),
+              ((element->etag[0]) ? element->etag : ""),
+              ((element->etag[0]) ? "\r\n" : ""));
         } else {
           /* No query string, no ending / : check the the <url>/ page */
           if (StringLength(url) > 0
@@ -1212,29 +1211,32 @@ static void proxytrack_process_HTTP(PT_Indexes indexes, T_SOC soc_c) {
             StringCat(urlRedirect, "/");
             if (PT_LookupIndex(indexes, StringBuff(urlRedirect))) {
               msgCode = 301;    /* Moved Permanently */
-              StringRoom(headers, 8192);
-              sprintf(StringBuffRW(headers),
-                      "HTTP/1.1 %d %s\r\n" "Content-Type: text/html\r\n"
-                      "Location: %s\r\n",
-                      /* */
-                      msgCode, GetHttpMessage(msgCode), StringBuff(urlRedirect)
-                );
-              StringLength(headers) = (int) strlen(StringBuff(headers));
+              StringSprintf(headers,
+                            "HTTP/1.1 %d %s\r\n"
+                            "Content-Type: text/html\r\n"
+                            "Location: %s\r\n",
+                            /* */
+                            msgCode, GetHttpMessage(msgCode),
+                            StringBuff(urlRedirect));
               /* */
-              StringRoom(output,
-                         1024 + sizeof(PROXYTRACK_COMMENT_HEADER) +
-                         sizeof(DISABLE_IE_FRIENDLY_HTTP_ERROR_MESSAGES));
-              sprintf(StringBuffRW(output),
-                      "<html>" PROXYTRACK_COMMENT_HEADER
-                      DISABLE_IE_FRIENDLY_HTTP_ERROR_MESSAGES "<head>"
-                      "<title>ProxyTrack - Page has moved</title>" "</head>\r\n"
-                      "<body>" "<h3>The correct location is:</h3><br />"
-                      "<b><a href=\"%s\">%s</a></b><br />" "<br />" "<br />\r\n"
-                      "<i>Generated by ProxyTrack " PROXYTRACK_VERSION
-                      ", (C) Xavier Roche and other contributors</i>" "\r\n"
-                      "</body>" "</header>", StringBuff(urlRedirect),
-                      StringBuff(urlRedirect));
-              StringLength(output) = (int) strlen(StringBuff(output));
+              StringSprintf(
+                  output,
+                  "<html"
+                  ">" PROXYTRACK_COMMENT_HEADER DISABLE_IE_FRIENDLY_HTTP_ERROR_MESSAGES
+                  "<head>"
+                  "<title>ProxyTrack - Page has moved</title>"
+                  "</head>\r\n"
+                  "<body>"
+                  "<h3>The correct location is:</h3><br />"
+                  "<b><a href=\"%s\">%s</a></b><br />"
+                  "<br />"
+                  "<br />\r\n"
+                  "<i>Generated by ProxyTrack " PROXYTRACK_VERSION
+                  ", (C) Xavier Roche and other contributors</i>"
+                  "\r\n"
+                  "</body>"
+                  "</header>",
+                  StringBuff(urlRedirect), StringBuff(urlRedirect));
             }
           }
           if (msgCode == 0) {
@@ -1255,25 +1257,29 @@ static void proxytrack_process_HTTP(PT_Indexes indexes, T_SOC soc_c) {
       } else if (msgError == NULL) {
         msgError = GetHttpMessage(msgCode);
       }
-      StringRoom(headers, 256);
-      sprintf(StringBuffRW(headers),
-              "HTTP/1.1 %d %s\r\n" "Content-type: text/html\r\n", msgCode,
-              msgError);
-      StringLength(headers) = (int) strlen(StringBuff(headers));
-      StringRoom(output,
-                 1024 + sizeof(PROXYTRACK_COMMENT_HEADER) +
-                 sizeof(DISABLE_IE_FRIENDLY_HTTP_ERROR_MESSAGES));
-      sprintf(StringBuffRW(output),
-              "<html>" PROXYTRACK_COMMENT_HEADER
-              DISABLE_IE_FRIENDLY_HTTP_ERROR_MESSAGES "<head>"
-              "<title>ProxyTrack - HTTP Proxy Error %d</title>" "</head>\r\n"
-              "<body>"
-              "<h3>A proxy error has occurred while processing the request.</h3><br />"
-              "<b>Error HTTP %d: <i>%s</i></b><br />" "<br />" "<br />\r\n"
-              "<i>Generated by ProxyTrack " PROXYTRACK_VERSION
-              ", (C) Xavier Roche and other contributors</i>" "\r\n" "</body>"
-              "</html>", msgCode, msgCode, msgError);
-      StringLength(output) = (int) strlen(StringBuff(output));
+      StringSprintf(headers,
+                    "HTTP/1.1 %d %s\r\n"
+                    "Content-type: text/html\r\n",
+                    msgCode, msgError);
+      StringSprintf(
+          output,
+          "<html"
+          ">" PROXYTRACK_COMMENT_HEADER DISABLE_IE_FRIENDLY_HTTP_ERROR_MESSAGES
+          "<head>"
+          "<title>ProxyTrack - HTTP Proxy Error %d</title>"
+          "</head>\r\n"
+          "<body>"
+          "<h3>A proxy error has occurred while processing the "
+          "request.</h3><br />"
+          "<b>Error HTTP %d: <i>%s</i></b><br />"
+          "<br />"
+          "<br />\r\n"
+          "<i>Generated by ProxyTrack " PROXYTRACK_VERSION
+          ", (C) Xavier Roche and other contributors</i>"
+          "\r\n"
+          "</body>"
+          "</html>",
+          msgCode, msgCode, msgError);
     }
     {
       char tmp[20 + 1];         /* 2^64 = 18446744073709551616 */

src/proxy/proxytrack.h

@@ -62,22 +62,32 @@ int proxytrack_main(char *proxyAddr, int proxyPort, char *icpAddr, int icpPort,
 
 /* Static definitions */
 
-HTS_UNUSED static void proxytrack_print_log(const char *severity, const char *format, ...) {
+/* Log one line; a NULL severity discards it. */
+HTS_UNUSED static void proxytrack_vprint_log(const char *severity,
+                                             const char *format, va_list args) {
   if (severity != NULL) {
     const int error = errno;
     FILE *const fp = stderr;
-    va_list args;
 
     fprintf(fp, " * %s: ", severity);
-    va_start(args, format);
     (void) vfprintf(fp, format, args);
-    va_end(args);
     fputs("\n", fp);
     fflush(fp);
     errno = error;
   }
 }
 
+HTS_UNUSED static void proxytrack_print_log(const char *severity,
+                                            const char *format, ...) {
+  if (severity != NULL) {
+    va_list args;
+
+    va_start(args, format);
+    proxytrack_vprint_log(severity, format, args);
+    va_end(args);
+  }
+}
+
 #define CRITICAL "critical"
 #define WARNING "warning"
 #define LOG "log"

src/proxy/store.c

@@ -28,6 +28,7 @@ Please visit our Website: http://www.httrack.com
 /* Author: Xavier Roche                                         */
 /* ------------------------------------------------------------ */
 
+#include <limits.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
@@ -196,6 +197,7 @@ struct _PT_Index__Old {
 struct _PT_Index__Arc {
   PT_INDEX_COMMON_STRUCTURE;
   FILE *file;
+  long int fileSize; /* bound for the lengths read out of the file */
   PT_Mutex fileLock;
   int version;
   char lastmodified[1024];
@@ -246,6 +248,13 @@ PT_Indexes PT_New(void) {
 
 void PT_Delete(PT_Indexes index) {
   if (index != NULL) {
+    int i;
+
+    /* PT_IndexMerge took ownership of each index, file handle included */
+    for (i = 0; i < index->index_size; i++) {
+      PT_Index_Delete(&index->index[i]);
+    }
+    free(index->index);
     coucal_delete(&index->cil);
     free(index);
   }
@@ -353,6 +362,11 @@ static void PT_Index_Delete__Arc(PT_Index * pindex) {
 
     if (index->file != NULL) {
       fclose(index->file);
+      index->file = NULL;
+    }
+    if (index->hash != NULL) {
+      coucal_delete(&index->hash);
+      index->hash = NULL;
     }
     MutexFree(&index->fileLock);
   }
@@ -559,8 +573,8 @@ PT_Index PT_LoadCache(const char *filename) {
         proxytrack_print_log(DEBUG,
                              "reading httrack cache (format #%d) %s : error",
                              type, filename);
-        free(index);
-        index = NULL;
+        /* the loader may already hold a file handle and entries */
+        PT_Index_Delete(&index);
         return NULL;
       } else {
         proxytrack_print_log(DEBUG,
@@ -694,12 +708,16 @@ int PT_IndexMerge(PT_Indexes indexes, PT_Index * pindex) {
     PT_Index index = *pindex;
     struct_coucal_enum en = coucal_enum_new(index->slots.common.hash);
     coucal_item *chain;
-    int index_id = indexes->index_size++;
+    /* index_size counts a slot only once the array holds it, so a failed
+       realloc leaves neither a phantom entry nor a dropped array */
+    PT_Index *const grown = realloc(
+        indexes->index, sizeof(*indexes->index) * (indexes->index_size + 1));
+    int index_id;
     int nMerged = 0;
 
-    if ((indexes->index =
-         realloc(indexes->index,
-                 sizeof(struct _PT_Index) * indexes->index_size)) != NULL) {
+    if (grown != NULL) {
+      indexes->index = grown;
+      index_id = indexes->index_size++;
       indexes->index[index_id] = index;
       *pindex = NULL;
       while((chain = coucal_enum_next(&en)) != NULL) {
@@ -854,22 +872,32 @@ static PT_Element PT_ReadCache__New(PT_Index index, const char *url, int flags)
 /* New HTTrack cache (new.zip) format                           */
 /* ------------------------------------------------------------ */
 
-#define ZIP_FIELD_STRING(headers, headersSize, field, value) do { \
-  if ( (value != NULL) && (value)[0] != '\0') { \
-    sprintf(headers + headersSize, "%s: %s\r\n", field, (value != NULL) ? (value) : ""); \
-    (headersSize) += (int) strlen(headers + headersSize); \
-  } \
-} while(0)
-#define ZIP_FIELD_INT(headers, headersSize, field, value) do { \
-  if ( (value != 0) ) { \
-    sprintf(headers + headersSize, "%s: "LLintP"\r\n", field, (LLint)(value)); \
-    (headersSize) += (int) strlen(headers + headersSize); \
-  } \
-} while(0)
-#define ZIP_FIELD_INT_FORCE(headers, headersSize, field, value) do { \
-  sprintf(headers + headersSize, "%s: "LLintP"\r\n", field, (LLint)(value)); \
-  (headersSize) += (int) strlen(headers + headersSize); \
-} while(0)
+/* Values read back off a cache, so the block is bounded rather than trusted to
+   the element caps; a field that does not fit is dropped whole and counted,
+   since a clipped one reads back as valid. `headers` must be an array. */
+#define ZIP_FIELD_STRING(headers, headersSize, dropped, field, value)          \
+  do {                                                                         \
+    if ((value) != NULL && (value)[0] != '\0' &&                               \
+        !slcatprintfbuff(headers, sizeof(headers), &(headersSize),             \
+                         "%s: %s\r\n", field, value)) {                        \
+      (dropped)++;                                                             \
+    }                                                                          \
+  } while (0)
+#define ZIP_FIELD_INT(headers, headersSize, dropped, field, value)             \
+  do {                                                                         \
+    if ((value) != 0 &&                                                        \
+        !slcatprintfbuff(headers, sizeof(headers), &(headersSize),             \
+                         "%s: " LLintP "\r\n", field, (LLint) (value))) {      \
+      (dropped)++;                                                             \
+    }                                                                          \
+  } while (0)
+#define ZIP_FIELD_INT_FORCE(headers, headersSize, dropped, field, value)       \
+  do {                                                                         \
+    if (!slcatprintfbuff(headers, sizeof(headers), &(headersSize),             \
+                         "%s: " LLintP "\r\n", field, (LLint) (value))) {      \
+      (dropped)++;                                                             \
+    }                                                                          \
+  } while (0)
 /* refvalue_size is mandatory: the cache line is bounded only by the line
    buffer, not by the destination. */
 #define ZIP_READFIELD_STRING(line, value, refline, refvalue, refvalue_size)    \
@@ -1269,21 +1297,25 @@ static PT_Element PT_ReadCache__New_u(PT_Index index_, const char *url,
   return r;
 }
 
+/* Bytes a writer may take from an element: a reader that could not fetch the
+   body still hands back the declared size (#931). */
+static size_t PT_Element_BodySize(const PT_Element element) {
+  return element->adr != NULL ? element->size : 0;
+}
+
 static int PT_SaveCache__New_Fun(void *arg, const char *url, PT_Element element) {
   zipFile zFileOut = (zipFile) arg;
+  const size_t body_size = PT_Element_BodySize(element);
   char headers[8192];
-  int headersSize;
+  size_t headersSize = 0;
+  int headersDropped = 0;
   zip_fileinfo fi;
   int zErr;
   const char *url_adr = "";
   const char *url_fil = "";
 
-  headers[0] = '\0';
-  headersSize = 0;
-
   /* Fields */
   headers[0] = '\0';
-  headersSize = 0;
   /* */
   {
     const char *message;
@@ -1294,25 +1326,43 @@ static int PT_SaveCache__New_Fun(void *arg, const char *url, PT_Element element)
       message = "(See X-StatusMessage)";
     }
     /* 64 characters MAX for first line */
-    sprintf(headers + headersSize, "HTTP/1.%c %d %s\r\n", '1',
-            element->statuscode, message);
+    if (!slcatprintfbuff(headers, sizeof(headers), &headersSize,
+                         "HTTP/1.%c %d %s\r\n", '1', element->statuscode,
+                         message)) {
+      headersDropped++;
+    }
   }
-  headersSize += (int) strlen(headers + headersSize);
 
   /* Second line MUST ALWAYS be X-In-Cache */
-  ZIP_FIELD_INT_FORCE(headers, headersSize, "X-In-Cache", 1);
-  ZIP_FIELD_INT(headers, headersSize, "X-StatusCode", element->statuscode);
-  ZIP_FIELD_STRING(headers, headersSize, "X-StatusMessage", element->msg);
-  ZIP_FIELD_INT(headers, headersSize, "X-Size", element->size); // size
-  ZIP_FIELD_STRING(headers, headersSize, "Content-Type", element->contenttype); // contenttype
-  ZIP_FIELD_STRING(headers, headersSize, "X-Charset", element->charset);        // contenttype
-  ZIP_FIELD_STRING(headers, headersSize, "Last-Modified", element->lastmodified);       // last-modified
-  ZIP_FIELD_STRING(headers, headersSize, "Etag", element->etag);        // Etag
-  ZIP_FIELD_STRING(headers, headersSize, "Location", element->location);        // 'location' pour moved
-  ZIP_FIELD_STRING(headers, headersSize, "Content-Disposition", element->cdispo);       // Content-disposition
-  ZIP_FIELD_STRING(headers, headersSize, "X-Addr", url_adr);    // Original address
-  ZIP_FIELD_STRING(headers, headersSize, "X-Fil", url_fil);     // Original URI filename
-  ZIP_FIELD_STRING(headers, headersSize, "X-Save", ""); // Original save filename
+  ZIP_FIELD_INT_FORCE(headers, headersSize, headersDropped, "X-In-Cache", 1);
+  ZIP_FIELD_INT(headers, headersSize, headersDropped, "X-StatusCode",
+                element->statuscode);
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "X-StatusMessage",
+                   element->msg);
+  ZIP_FIELD_INT(headers, headersSize, headersDropped, "X-Size",
+                (int) body_size); // size
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "Content-Type",
+                   element->contenttype); // contenttype
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "X-Charset",
+                   element->charset); // contenttype
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "Last-Modified",
+                   element->lastmodified); // last-modified
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "Etag",
+                   element->etag); // Etag
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "Location",
+                   element->location); // 'location' pour moved
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "Content-Disposition",
+                   element->cdispo); // Content-disposition
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "X-Addr",
+                   url_adr); // Original address
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "X-Fil",
+                   url_fil); // Original URI filename
+  ZIP_FIELD_STRING(headers, headersSize, headersDropped, "X-Save",
+                   ""); // Original save filename
+  if (headersDropped != 0) {
+    fprintf(stderr, "Headers of %s: %d field(s) dropped" LF, url,
+            headersDropped);
+  }
 
   /* Time */
   memset(&fi, 0, sizeof(fi));
@@ -1342,10 +1392,9 @@ static int PT_SaveCache__New_Fun(void *arg, const char *url, PT_Element element)
   }
 
   /* Write data in cache */
-  if (element->size > 0 && element->adr != NULL) {
-    if ((zErr =
-         zipWriteInFileInZip(zFileOut, element->adr,
-                             (int) element->size)) != Z_OK) {
+  if (body_size != 0) {
+    if ((zErr = zipWriteInFileInZip(zFileOut, element->adr, (int) body_size)) !=
+        Z_OK) {
       assertf(! "zip_zipWriteInFileInZip_failed");
     }
   }
@@ -1497,6 +1546,9 @@ static int PT_LoadCache__Old(PT_Index index_, const char *filename) {
 
     cache->filenameDat[0] = '\0';
     cache->filenameNdx[0] = '\0';
+    /* before the first early return: the readers lock it and the destructor
+       frees it, whether or not we get as far as opening a file */
+    MutexInit(&cache->fileLock);
 
     index_base_path(cache->path, sizeof(cache->path), filename);
 
@@ -1983,6 +2035,20 @@ static int skipArcNl(FILE * file) {
   return -1;
 }
 
+/* Stop on the newline opening the first record: archives whose version block
+   length swallowed the closing blank line leave only that one. */
+static int skipArcVersionNl(FILE *file) {
+  long int pos = ftell(file);
+
+  if (pos < 0 || fgetc(file) != 0x0a) {
+    return -1;
+  }
+  if (fgetc(file) == 0x0a) { /* the blank line, when outside the length */
+    pos++;
+  }
+  return fseek(file, pos, SEEK_SET);
+}
+
 static int skipArcData(FILE * file, const char *line) {
   int jump = getArcLength(line);
 
@@ -2083,6 +2149,13 @@ int PT_LoadCache__Arc(PT_Index index_, const char *filename) {
     if (index->file != NULL) {
       coucal hashtable = index->hash;
 
+      /* past LONG_MAX a 32-bit ftell cannot answer; keep serving the records
+         it can still reach, with the bound no longer constraining */
+      if (fseek(index->file, 0, SEEK_END) != 0 ||
+          (index->fileSize = ftell(index->file)) < 0) {
+        index->fileSize = LONG_MAX;
+      }
+      rewind(index->file);
       if (readArcURLRecord(index) == 0) {
         int entries = 0;
 
@@ -2095,9 +2168,9 @@ int PT_LoadCache__Arc(PT_Index index_, const char *filename) {
         }
         /* Timestamp */
         index->timestamp = getArcTimestamp(index->line);
-        /* Skip first entry */
-        if (skipArcData(index->file, index->line) != 0
-            || skipArcNl(index->file) != 0) {
+        /* Skip the version block, leaving the record loop its own separator */
+        if (skipArcData(index->file, index->line) != 0 ||
+            skipArcVersionNl(index->file) != 0) {
           fprintf(stderr, "Unexpected bad data offset size first entry" LF);
           fclose(index->file);
           index->file = NULL;
@@ -2281,7 +2354,10 @@ static PT_Element PT_ReadCache__Arc_u(PT_Index index_, const char *url,
 
               if (fetchSize <= 0) {
                 fetchSize = dataLength - metaSize;
-              } else if (fetchSize > dataLength - metaSize) {
+              }
+              /* the declared body may exceed the archive we allocate it from */
+              if (fetchSize < 0 || fetchSize > dataLength - metaSize ||
+                  fetchSize > index->fileSize - fposCurrent) {
                 r->statuscode = STATUSCODE_INVALID;
                 strcpybuff(r->msg, "Cache Read Error : Truncated Data");
               }
@@ -2386,6 +2462,7 @@ static hts_boolean arc_headers_cat(char *headers, size_t size,
 static int PT_SaveCache__Arc_Fun(void *arg, const char *url, PT_Element element) {
   PT_SaveCache__Arc_t *st = (PT_SaveCache__Arc_t *) arg;
   FILE *const fp = st->fp;
+  const size_t body_size = PT_Element_BodySize(element);
   struct tm *tm = convert_time_rfc822(&st->buff, element->lastmodified);
   struct tm unknown_date;
   /* the two strcatbuff calls closing the block rely on these 4 bytes */
@@ -2393,6 +2470,11 @@ static int PT_SaveCache__Arc_Fun(void *arg, const char *url, PT_Element element)
   hts_boolean fit;
   int size_headers;
 
+  if (body_size != element->size) {
+    fprintf(stderr, "Entry %s stored without its %lu-byte body" LF, url,
+            (unsigned long) element->size);
+  }
+
   /* a cached entry with no parseable Last-Modified must not take the writer
      down; the epoch is the conventional "date unknown" */
   if (tm == NULL) {
@@ -2417,7 +2499,7 @@ static int PT_SaveCache__Arc_Fun(void *arg, const char *url, PT_Element element)
                      (element->charset[0] ? "; charset=\"" : ""),
                      (element->charset[0] ? element->charset : ""),
                      (element->charset[0] ? "\"" : ""),
-                     /**/ element->lastmodified, (int) element->size);
+                     /**/ element->lastmodified, (int) body_size);
   if (element->location != NULL && element->location[0] != '\0') {
     if (!arc_headers_cat(st->headers, room, "Location: ") ||
         !arc_headers_cat(st->headers, room, element->location) ||
@@ -2442,8 +2524,8 @@ static int PT_SaveCache__Arc_Fun(void *arg, const char *url, PT_Element element)
   /* doc == <nl><URL-record><nl><network_doc> */
 
   /* Format: URL IP date mime result checksum location offset filename length */
-  if (element->adr != NULL) {
-    domd5mem(element->adr, element->size, st->md5, 1);
+  if (body_size != 0) {
+    domd5mem(element->adr, body_size, st->md5, 1);
   } else {
     strcpybuff(st->md5, "-");
   }
@@ -2460,12 +2542,10 @@ static int PT_SaveCache__Arc_Fun(void *arg, const char *url, PT_Element element)
           tm->tm_min, tm->tm_sec, hts_effective_mime(element->contenttype),
           element->statuscode, st->md5,
           (element->location ? element->location : "-"), (long int) ftell(fp),
-          st->filename, (long int) (size_headers + element->size));
+          st->filename, (long int) (size_headers + body_size));
   /* network_doc */
-  if (fwrite(st->headers, 1, size_headers, fp) != size_headers
-      || (element->size > 0
-          && fwrite(element->adr, 1, element->size, fp) != element->size)
-    ) {
+  if (fwrite(st->headers, 1, size_headers, fp) != size_headers ||
+      (body_size != 0 && fwrite(element->adr, 1, body_size, fp) != body_size)) {
     return 1;                   /* Error */
   }
 
@@ -2486,16 +2566,20 @@ static int PT_SaveCache__Arc(PT_Indexes indexes, const char *filename) {
        2<sp><reserved><sp><origin-code><nl>
        URL<sp>IP-address<sp>Archive-date<sp>Content-type<sp>Result-code<sp>Checksum<sp>Location<sp> Offset<sp>Filename<sp>Archive-length<nl>
        <nl> */
-    const char *prefix =
-      "2 0 HTTrack Website Copier" "\n"
-      "URL IP-address Archive-Date Content-Type Result-code Checksum Location Offset Filename Archive-length"
-      "\n" "\n";
+    const char *prefix = "2 0 HTTrack Website Copier"
+                         "\n"
+                         "URL IP-address Archive-Date Content-Type Result-code "
+                         "Checksum Location Offset Filename Archive-length"
+                         "\n";
     sprintf(st.filename, "httrack_%d.arc", (int) t);
     fprintf(fp,
             "filedesc://%s 0.0.0.0 %04d%02d%02d%02d%02d%02d text/plain 200 - - 0 %s %d"
             "\n" "%s", st.filename, tm.tm_year + 1900, tm.tm_mon + 1,
             tm.tm_mday, tm.tm_hour, tm.tm_min, tm.tm_sec, st.filename,
             (int) strlen(prefix), prefix);
+    /* the blank line closing the version block is a separator, outside the
+       declared length */
+    fputc('\n', fp);
     st.fp = fp;
     st.indexes = indexes;
     st.t = t;

src/webhttrack.in

@@ -23,7 +23,9 @@ for d in "${pathdirs[@]}"; do
     # drop empty PATH fields, matching the old echo|tr word-split
     test -n "$d" && SRCHPATH+=("$d")
 done
-SRCHDISTPATH=("$BINWD/../share" "$BINWD/.." /usr/share /usr/local /usr /local /usr/local/share "${HOME}/usr" "${HOME}/usr/share" /opt/local/share /sw "${HOME}/usr/local" "${HOME}/usr/share")
+# The substituted datadir goes after the relative entries so a moved tree wins, and
+# before the guesses so --datadir works (#887).
+SRCHDISTPATH=("$BINWD/../share" "$BINWD/.." "@datadir@" /usr/share /usr/local /usr /local /usr/local/share "${HOME}/usr" "${HOME}/usr/share" /opt/local/share /sw "${HOME}/usr/local" "${HOME}/usr/share")
 
 ###
 # And now some famous cuisine

tests/01_engine-copyopt.test

@@ -3,7 +3,6 @@
 # Regression guard for the unsigned-enum sentinel trap: copy_htsopt's
 # `if (from->X > -1)` guard is always false for unsigned hts_boolean fields, so
 # they silently stop being copied. Driven by the in-process 'httrack -#test=copyopt' test.
-# Keep POSIX-portable (harness runs it via $(BASH), a plain /bin/sh on macOS).
 
 set -eu
 

tests/01_engine-footer-overflow.test

@@ -1,9 +1,5 @@
 #!/bin/bash
 #
-# Keep this POSIX-portable: the harness runs it via $(BASH), which is a plain
-# POSIX /bin/sh on some platforms (e.g. macOS), so avoid bashisms despite the
-# #!/bin/bash above.
-
 # A -%F footer whose expansion overflows the on-page buffer must be dropped, not
 # crash the crawl. Before the fix the unchecked hts_footer_format return left the
 # buffer unterminated and the next strcatbuff aborted (SIGABRT).

tests/01_engine-pause.test

@@ -2,7 +2,7 @@
 #
 # --pause (#185): the inter-file pause target must stay in [min,max] and spread
 # across it (a per-call rand() would collapse it toward min). Driven by the
-# in-process 'httrack -#test=pause' test. POSIX-portable ($(BASH) is /bin/sh on macOS).
+# in-process 'httrack -#test=pause' test.
 
 set -eu
 

tests/01_engine-version-macros.test

@@ -1,15 +1,17 @@
 #!/bin/bash
 #
-# version.rc repeats the version that htsglobal.h declares. Signing enforces that
-# every binary in a release reports the same one, so a drift fails the signing
-# request on release day rather than the build. Assert the two agree.
+# htsglobal.h declares the version; version.rc, configure.ac and the metainfo
+# repeat it. Miss one and a release ships a mismatched binary or a stale store entry.
 
 set -euo pipefail
 
-src="${top_srcdir:-..}/src"
+top="${top_srcdir:-..}"
+src="$top/src"
 h="$src/htsglobal.h"
 rc="$src/version.rc"
-for f in "$h" "$rc"; do
+ac="$top/configure.ac"
+metainfo="$top/html/server/div/com.httrack.WebHTTrack.metainfo.xml"
+for f in "$h" "$rc" "$ac" "$metainfo"; do
     [ -f "$f" ] || {
         echo "cannot find $f"
         exit 1
@@ -31,23 +33,49 @@ rc_fileversion=$(sed -n 's/.*VALUE "FileVersion",[[:space:]]*"\([^"]*\)".*/\1/p'
 rc_productversion=$(sed -n 's/.*VALUE "ProductVersion",[[:space:]]*"\([^"]*\)".*/\1/p' "$rc")
 rc_productname=$(sed -n 's/.*VALUE "ProductName",[[:space:]]*"\([^"]*\)".*/\1/p' "$rc")
 
+# And again in configure.ac and the AppStream metainfo.
+ac_version=$(sed -n 's/^AC_INIT(\[[^]]*\],[[:space:]]*\[\([^]]*\)\].*/\1/p' "$ac")
+# Comments dropped, one tag per line: a parked <release> or a version= on the
+# <releases> wrapper would otherwise pose as the newest entry.
+releases=$(sed 's/<!--.*-->//g' "$metainfo" | tr '<' '\n' |
+    sed -n 's/^release[[:space:]][^>]*version="\([^"]*\)".*/\1/p')
+if [ -z "$ac_version" ]; then
+    echo "could not read the version from $ac"
+    exit 1
+fi
+if [ -z "$releases" ]; then
+    echo "could not read any release version from $metainfo"
+    exit 1
+fi
+
 # 3.49.12 -> 3,49,12,0
 expected_numeric="$(echo "$versionid" | tr '.' ','),0"
 
 fail=0
 check() { # what expected actual
     if [ "$2" != "$3" ]; then
-        echo "version.rc $1 is \"$3\", but htsglobal.h says it should be \"$2\""
+        echo "$1 is \"$3\", but htsglobal.h says it should be \"$2\""
         fail=1
     fi
 }
-check FILEVERSION "$expected_numeric" "$fileversion"
-check PRODUCTVERSION "$expected_numeric" "$productversion"
-check FileVersion "$versionid" "$rc_fileversion"
-check ProductVersion "$version" "$rc_productversion"
+check "version.rc FILEVERSION" "$expected_numeric" "$fileversion"
+check "version.rc PRODUCTVERSION" "$expected_numeric" "$productversion"
+check "version.rc FileVersion" "$versionid" "$rc_fileversion"
+check "version.rc ProductVersion" "$version" "$rc_productversion"
 
 # Signing pins the product name too.
-check ProductName "HTTrack Website Copier" "$rc_productname"
+check "version.rc ProductName" "HTTrack Website Copier" "$rc_productname"
+
+check "the configure.ac AC_INIT version" "$versionid" "$ac_version"
+check "the newest metainfo release" "$versionid" "$(echo "$releases" | head -n 1)"
+
+# The check above trusts head -1, so require the list to actually descend.
+echo "$releases" | awk -F'[.]' '
+    !/^[0-9]+\.[0-9]+\.[0-9]+$/ { print "unparsable metainfo release version: " $0; exit 1 }
+    { n = $1 * 1000000000 + $2 * 1000000 + $3 }
+    NR > 1 && n >= prev { print "metainfo releases are not newest-first: " $0; exit 1 }
+    { prev = n }
+' || fail=1
 
 [ "$fail" -eq 0 ] || exit 1
 
@@ -61,4 +89,4 @@ case "$out" in
     ;;
 esac
 
-echo "version resource agrees with htsglobal.h: $version ($expected_numeric)"
+echo "version.rc, configure.ac and the metainfo agree with htsglobal.h: $version ($expected_numeric)"

tests/01_zlib-cache-golden.test

@@ -1,9 +1,5 @@
 #!/bin/bash
 #
-# Keep this POSIX-portable: the harness runs it via $(BASH), which is a plain
-# POSIX /bin/sh on some platforms (e.g. macOS), so avoid bashisms and GNU-only
-# tool flags despite the #!/bin/bash above.
-
 # Golden cache-format regression test (driven by 'httrack -#test=cache-golden <dir>').
 #
 # 01_zlib-cache.test writes the cache with the same build it reads back (a