tests/12_crawl_https.test

#!/bin/bash
#

set -euo pipefail

testdir=$(cd "$(dirname "$0")" && pwd)

bash "$testdir/check-network.sh" || ! echo "skipping online unit tests" || exit 77

if test "${HTTPS_SUPPORT:-}" == "no"; then
    echo "no https support compiled, skipping"
    exit 77
fi

bash "$testdir/crawl-test.sh" --errors 0 --files 5 httrack https://ut.httrack.com/simple/basic.html
Added a basic https unit test. 2013-06-05 19:00:53 +00:00			`#!/bin/bash`
			`#`

tests: run the *.test scripts under set -euo pipefail The test scripts mostly ran with no error flags, so a failing command in the middle would be ignored and the script would limp on to a misleading result. Turn on strict mode everywhere, guarding the spots that legitimately expect a non-zero exit: - the htssafe overflow probes (-#8) deliberately abort, and the strsafe/ cmdline crawls capture an exit code to assert on, so those are run with `\|\| true` / `\|\| rc=$?` rather than letting set -e kill the script first; - the parser fixture crawl ignores httrack's own exit (it checks the mirrored files), so it keeps `\|\| true`; - 02_update-cache replaced `find ... \| grep -q .` with a `-print -quit` command substitution: under pipefail grep -q can close the pipe early and leave find killed by SIGPIPE, which would spuriously fail an existing file; - 12_crawl_https guards $HTTPS_SUPPORT with `${...:-}` for set -u. 02_manpage-regen and 01_engine-cache stay on `set -eu` (no pipefail): both are run via $(BASH), which can be a plain POSIX /bin/sh where `set -o pipefail` does not exist. shellcheck clean; make check: 15 PASS, 7 SKIP (offline). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> 2026-06-17 06:17:38 +02:00			`set -euo pipefail`

Out-of-tree builds skip the online crawl tests with a misleading reason (#1047) * tests: resolve online-gate helper scripts against the script's own directory The seven online crawl tests invoked check-network.sh and crawl-test.sh by bare relative name, which the shell resolves against the caller's cwd. make check runs from the build tree's tests/, so an out-of-tree build finds no copy: the gate failed to open, the \|\| branch reported "skipping online unit tests", and the tests skipped whatever --enable-online-unit-tests asked for. check-network.sh had the same defect in its own auto-probe call to crawl-test.sh. Resolve all of them against $(dirname "$0"), the idiom local-crawl.sh already uses. 232 pins the skip reason from an unrelated cwd and sweeps the tree for the same form. The option's help string advertised default=yes; the code has always defaulted to no. Signed-off-by: Xavier Roche <roche@httrack.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * tests: keep 232's comment clear of 226's credential-scrub gate 226 flags any .test naming the Windows suite driver without the scrub line, comment or not, so the exemption note tripped it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * tests: tighten 232's sweep, which is the only check below the gate With the flag off check-network.sh exits before a test reaches its own crawl-test.sh line, so the runtime probe cannot see that class at all and the static sweep carries it alone. Four spellings of #1016 were getting through: a leading ./, a quoted bare name, eval, and a variable holding the bare name. Quoting and ./ change nothing about what the shell opens, so the shape exemption is gone and the two call sites that mean the cwd are named instead. A variable operand is resolved back to its assignment, and eval sites are pinned as a set rather than passed over. Each form is planted as a fixture, with anchored and commented fixtures for the other direction. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * tests: 172 never checked that it ran its own staged suite copy Pointing it at the srcdir copy passed, dropping the reap_leftover_processes neuter it appends to the staged testlib.sh and leaving the test to reap sibling engines under "make check -j", a flake whose symptom points nowhere near its cause. test-timeout.sh sources testlib.sh from the staging directory too, so a plain marker file would still be written under the mutant; record the sourcing script instead and require the driver. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> --------- Signed-off-by: Xavier Roche <roche@httrack.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> 2026-08-07 18:38:48 +02:00			`testdir=$(cd "$(dirname "$0")" && pwd)`

			`bash "$testdir/check-network.sh" \|\| ! echo "skipping online unit tests" \|\| exit 77`
Added a basic https unit test. 2013-06-05 19:00:53 +00:00
tests: run the *.test scripts under set -euo pipefail The test scripts mostly ran with no error flags, so a failing command in the middle would be ignored and the script would limp on to a misleading result. Turn on strict mode everywhere, guarding the spots that legitimately expect a non-zero exit: - the htssafe overflow probes (-#8) deliberately abort, and the strsafe/ cmdline crawls capture an exit code to assert on, so those are run with `\|\| true` / `\|\| rc=$?` rather than letting set -e kill the script first; - the parser fixture crawl ignores httrack's own exit (it checks the mirrored files), so it keeps `\|\| true`; - 02_update-cache replaced `find ... \| grep -q .` with a `-print -quit` command substitution: under pipefail grep -q can close the pipe early and leave find killed by SIGPIPE, which would spuriously fail an existing file; - 12_crawl_https guards $HTTPS_SUPPORT with `${...:-}` for set -u. 02_manpage-regen and 01_engine-cache stay on `set -eu` (no pipefail): both are run via $(BASH), which can be a plain POSIX /bin/sh where `set -o pipefail` does not exist. shellcheck clean; make check: 15 PASS, 7 SKIP (offline). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> 2026-06-17 06:17:38 +02:00			`if test "${HTTPS_SUPPORT:-}" == "no"; then`
Reformat every shell script with shfmt -i 4 Mechanical pass: run shfmt -i 4 over the whole tracked shell tree (the test harness .test files, the regen generators, webhttrack, the CGI search helper, and the build/dist scripts) so they share one style. shfmt also normalised backticks to $(...) and $[..] to $((..)). No behaviour change: arithmetic is preserved exactly, non-ASCII bytes are untouched, and the full make check suite still passes. The tab indented .test files become 4-space indented, hence the wide diff. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> 2026-06-20 11:24:01 +02:00			`echo "no https support compiled, skipping"`
			`exit 77`
Added a basic https unit test. 2013-06-05 19:00:53 +00:00			`fi`

Out-of-tree builds skip the online crawl tests with a misleading reason (#1047) * tests: resolve online-gate helper scripts against the script's own directory The seven online crawl tests invoked check-network.sh and crawl-test.sh by bare relative name, which the shell resolves against the caller's cwd. make check runs from the build tree's tests/, so an out-of-tree build finds no copy: the gate failed to open, the \|\| branch reported "skipping online unit tests", and the tests skipped whatever --enable-online-unit-tests asked for. check-network.sh had the same defect in its own auto-probe call to crawl-test.sh. Resolve all of them against $(dirname "$0"), the idiom local-crawl.sh already uses. 232 pins the skip reason from an unrelated cwd and sweeps the tree for the same form. The option's help string advertised default=yes; the code has always defaulted to no. Signed-off-by: Xavier Roche <roche@httrack.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * tests: keep 232's comment clear of 226's credential-scrub gate 226 flags any .test naming the Windows suite driver without the scrub line, comment or not, so the exemption note tripped it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * tests: tighten 232's sweep, which is the only check below the gate With the flag off check-network.sh exits before a test reaches its own crawl-test.sh line, so the runtime probe cannot see that class at all and the static sweep carries it alone. Four spellings of #1016 were getting through: a leading ./, a quoted bare name, eval, and a variable holding the bare name. Quoting and ./ change nothing about what the shell opens, so the shape exemption is gone and the two call sites that mean the cwd are named instead. A variable operand is resolved back to its assignment, and eval sites are pinned as a set rather than passed over. Each form is planted as a fixture, with anchored and commented fixtures for the other direction. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * tests: 172 never checked that it ran its own staged suite copy Pointing it at the srcdir copy passed, dropping the reap_leftover_processes neuter it appends to the staged testlib.sh and leaving the test to reap sibling engines under "make check -j", a flake whose symptom points nowhere near its cause. test-timeout.sh sources testlib.sh from the staging directory too, so a plain marker file would still be written under the mutant; record the sourcing script instead and require the driver. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> --------- Signed-off-by: Xavier Roche <roche@httrack.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> 2026-08-07 18:38:48 +02:00			`bash "$testdir/crawl-test.sh" --errors 0 --files 5 httrack https://ut.httrack.com/simple/basic.html`