tests: add offline local test server prototype (cookies + HTTPS)

Replace the network dependency for crawl tests with a self-contained Python
stdlib server (http.server + ssl) that httrack crawls over loopback. The server
binds an ephemeral port and prints it on stdout; local-crawl.sh discovers the
port, substitutes the BASEURL token into the httrack arguments, runs the crawl,
and audits the mirror under the discovered host-root directory.

This prototype migrates two cases off ut.httrack.com:

- 13_local-cookies.test drives the cookie chain (entrance/second/third)
  reimplemented as Python handlers from the old ut/cookies/*.php fixtures. A
  missing or wrong cookie answers 500, so a clean 3-files/0-errors run proves
  the cookie jar is replayed across links.
- 14_local-https.test crawls over HTTPS using a shipped long-dated self-signed
  cert. httrack does not verify certs, so the cert is accepted as-is and the
  real TLS path runs offline.

The group skips (exit 77) when python3 is missing, mirroring check-network.sh.
Fixtures and the cert are listed explicitly in EXTRA_DIST (automake does not
expand globs); make distcheck passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

.flake8

@@ -0,0 +1,5 @@
+[flake8]
+# Match black's formatting so the two tools don't fight.
+max-line-length = 88
+# E203/W503 conflict with black's slice and line-break style.
+extend-ignore = E203, W503

.github/workflows/ci.yml

@@ -320,37 +320,20 @@ jobs:
   lint:
     name: lint (shellcheck, shfmt)
     runs-on: ubuntu-24.04
-    env:
-      SHFMT_VERSION: v3.8.0
     steps:
       - uses: actions/checkout@v6
 
-      # shfmt is a pinned release binary, so it never changes: cache it keyed on
-      # the version. Same rationale as the git-clang-format driver below -- avoid
-      # re-downloading an unchanging file from github.com on every run.
-      - name: Cache shfmt binary
-        uses: actions/cache@v4
-        with:
-          path: ~/.cache/shfmt/shfmt
-          key: shfmt-${{ env.SHFMT_VERSION }}-${{ runner.arch }}
-
       - name: Install linters
+        env:
+          SHFMT_VERSION: v3.8.0
         run: |
           set -euo pipefail
           sudo apt-get update
           sudo apt-get install -y --no-install-recommends shellcheck
-          # shfmt is not packaged in apt; fetch a pinned release binary (cold
-          # cache only), retrying through transient errors.
-          shfmt="$HOME/.cache/shfmt/shfmt"
-          if [ ! -s "$shfmt" ]; then
-            echo "shfmt cache MISS: fetching ${SHFMT_VERSION} from github.com"
-            mkdir -p "$(dirname "$shfmt")"
-            curl --retry 5 --retry-all-errors -fsSL -o "$shfmt" \
-              "https://github.com/mvdan/sh/releases/download/${SHFMT_VERSION}/shfmt_${SHFMT_VERSION}_linux_$(dpkg --print-architecture)"
-          else
-            echo "shfmt cache HIT: using cached ${SHFMT_VERSION}"
-          fi
-          sudo install -m 0755 "$shfmt" /usr/local/bin/shfmt
+          # shfmt is not packaged in apt; fetch a pinned release binary.
+          curl -fsSL -o /tmp/shfmt \
+            "https://github.com/mvdan/sh/releases/download/${SHFMT_VERSION}/shfmt_${SHFMT_VERSION}_linux_$(dpkg --print-architecture)"
+          sudo install -m 0755 /tmp/shfmt /usr/local/bin/shfmt
 
       # Lint the scripts we maintain; the legacy scripts are a separate cleanup.
       - name: shellcheck
@@ -366,24 +349,11 @@ jobs:
     name: format (clang-format-19, changed lines)
     if: github.event_name == 'pull_request'
     runs-on: ubuntu-24.04
-    env:
-      # Single-source the tag so the cache key and the fetch URL can never drift.
-      LLVM_TAG: llvmorg-19.1.7
     steps:
       - uses: actions/checkout@v6
         with:
           fetch-depth: 0
 
-      # The git-clang-format driver is pinned to an immutable release tag, so the
-      # fetched file never changes: cache it keyed on the tag. raw.githubusercontent.com
-      # 429-rate-limits the shared runner egress IPs, and re-downloading an unchanging
-      # file every run was the only thing that could (and did) hit that limit.
-      - name: Cache git-clang-format driver
-        uses: actions/cache@v4
-        with:
-          path: ~/.cache/git-clang-format/git-clang-format
-          key: git-clang-format-${{ env.LLVM_TAG }}
-
       - name: Install clang-format 19 (pinned, from apt.llvm.org)
         run: |
           set -euo pipefail
@@ -394,17 +364,11 @@ jobs:
             | sudo tee /etc/apt/sources.list.d/llvm-19.list >/dev/null
           sudo apt-get update
           sudo apt-get install -y --no-install-recommends clang-format-19
-          # Cold cache only: fetch the driver, retrying through transient 429s.
-          driver="$HOME/.cache/git-clang-format/git-clang-format"
-          if [ ! -s "$driver" ]; then
-            echo "git-clang-format cache MISS: fetching ${LLVM_TAG} from raw.githubusercontent.com"
-            mkdir -p "$(dirname "$driver")"
-            curl --retry 5 --retry-all-errors -fsSL -o "$driver" \
-              "https://raw.githubusercontent.com/llvm/llvm-project/${LLVM_TAG}/clang/tools/clang-format/git-clang-format"
-          else
-            echo "git-clang-format cache HIT: using cached ${LLVM_TAG}"
-          fi
-          sudo install -m 0755 "$driver" /usr/local/bin/git-clang-format
+          # git-clang-format driver, pinned to an immutable release tag (not a
+          # moving branch) since we curl and then execute it.
+          sudo curl -fsSL -o /usr/local/bin/git-clang-format \
+            https://raw.githubusercontent.com/llvm/llvm-project/llvmorg-19.1.7/clang/tools/clang-format/git-clang-format
+          sudo chmod 0755 /usr/local/bin/git-clang-format
           clang-format-19 --version
 
       - name: Check formatting of changed lines

AGENTS.md

@@ -1,67 +0,0 @@
-# AGENTS.md — working in the HTTrack tree
-
-Policy and PR etiquette live in [CONTRIBUTING.md](CONTRIBUTING.md). This file is
-the operational checklist: toolchain, invariants, and how to ship a change.
-
-## Build & test
-- Fresh clone first: `git submodule update --init src/coucal`
-- `bash configure && make && make check`
-
-## Hard invariants
-- **Toolchain edit** (`configure.ac`, any `Makefile.am`, `m4/`) → run
-  `autoreconf -fi` and commit the regenerated tracked files. The repo ships the
-  generated `configure`/`Makefile.in` so users build without autotools; CI does
-  **not** catch staleness.
-- **Format only changed lines** with `git clang-format` (clang-format 19). Never
-  reformat untouched code: the engine was formatted by an old tool and won't
-  round-trip.
-- **Byte-safe edits.** Files with raw high bytes are ISO-8859-1 (French
-  comments). Edit them byte-wise (`perl -0pi`, `sed`), not through a tool that
-  re-encodes to UTF-8 and corrupts them.
-
-## Security (HTTrack parses hostile input off the network)
-- Bounds-check every copy. Overflow-safe form: put the untrusted value alone,
-  `untrusted < limit - controlled` — never `controlled + untrusted < limit`,
-  which can wrap and pass.
-
-## Code & prose
-- Be terse. Comment the why, in English; translate French comments you touch.
-- Strip AI tells from prose (em-dash overuse, rule-of-three, filler, vague
-  attributions). Ref: Wikipedia "Signs of AI writing". Claude Code: `/humanizer`.
-- Behavior change → add a test. Fast path: a hidden `httrack -#N` debug
-  subcommand (`htscoremain.c`) driven by a `tests/NN_*.test`, over a slow crawl.
-
-## Review your change adversarially (strongly suggested)
-Before pushing, and when reviewing others, don't skim for bugs:
-- **One invariant at a time.** Name a property the diff must preserve (bounds
-  hold, cache/wire format unchanged, no use-after-free, ABI stable), then
-  construct inputs that would break it. "General correctness" is not a charter.
-- **Audit tests against the spec, not the code.** For each new test ask: "what
-  buggy path would still pass this?" If you can build one, the test is
-  confirmation-biased: assertions copied from observed output lock bugs in.
-- **Risk areas need runtime probes.** Touching hostile-input parsing, struct
-  layout/ABI, cache/wire format, or a security path? A static or unit check
-  isn't enough; exercise the wrong behavior at runtime. Claude Code:
-  `/review-recipe`.
-
-## Commits
-- **Sign-off is mandatory.** Every commit carries a `Signed-off-by` trailer:
-  `git commit -s` (DCO, CI-enforced — unsigned commits are rejected).
-- **Co-Authored-By is mandatory for AI-assisted commits.** Carry a
-  `Co-Authored-By:` trailer naming the assistant. Attribute there, never in a
-  PR-body footer.
-- PRs land as a merge commit; every commit on the branch goes onto master, so
-  keep each commit message clean and meaningful.
-
-## PR descriptions
-- Plain concise prose; lead with what changed and why. No What/Why/How template.
-- Title names the problem, not the implementation.
-- Don't restate the diff — give what it can't show: motivation, context,
-  tradeoffs, risk.
-- Length tracks the change: a typo is one sentence; a security fix earns a writeup.
-- Verify claims against the code before you write them; flag drift, don't repeat it.
-- Don't hard-wrap (GitHub reflows). No "Generated with Claude" footer. Run the
-  prose through `/humanizer`.
-
-## Toolchain
-C · clang-format-19 · autoreconf · shfmt + shellcheck (shell) · black + flake8 (Python)

CLAUDE.md

@@ -1 +0,0 @@
-@AGENTS.md

CONTRIBUTING.md

@@ -1,15 +1,12 @@
 # Contributing to HTTrack
 
-HTTrack is small and old. Keep changes easy to review and safe to merge. Working
-with an AI assistant? The operational checklist is [AGENTS.md](AGENTS.md).
+HTTrack is small and old. Keep changes easy to review and safe to merge.
 
 ## Pull requests
 
 - One change per PR. Small diffs merge fast.
-- PRs land as a merge commit, so the branch's commits go onto master as-is: keep
-  each commit message clean and explain *why*.
-- Be terse in the PR title and description: name the problem, not the fix, don't
-  restate the diff, and calibrate length to the change.
+- PRs are squash-merged: the title and description become the commit message, so
+  explain *why*.
 - Add or update tests for engine changes (`tests/`), and keep CI green.
 
 ## Style
@@ -33,9 +30,6 @@ Welcome, and nothing to disclose. Two rules:
 - **Own every line** as if you wrote it. Can't explain it in review? Not ready.
 - **Don't push your work onto reviewers.** A raw generated patch a maintainer has
   to vet from scratch will be closed.
-- **Attribution is mandatory.** AI-assisted commits must carry a
-  `Co-Authored-By:` trailer naming the assistant, not a footer in the PR
-  description.
 
 The sign-off covers AI-assisted code too.
 

configure

@@ -3685,9 +3685,7 @@ fi
 
 
 
-# 3:0:0: htsblk layout changed (contenttype/charset/contentencoding widened to
-# 128), an incompatible ABI break, so bump current and reset revision/age.
-VERSION_INFO="3:0:0"
+VERSION_INFO="2:49:0"
 
 { printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking whether to enable maintainer-specific portions of Makefiles" >&5
 printf %s "checking whether to enable maintainer-specific portions of Makefiles... " >&6; }

configure.ac

@@ -29,9 +29,7 @@ AC_CONFIG_SRCDIR(src/httrack.c)
 AC_CONFIG_MACRO_DIR([m4])
 AC_CONFIG_HEADERS(config.h)
 AM_INIT_AUTOMAKE([subdir-objects])
-# 3:0:0: htsblk layout changed (contenttype/charset/contentencoding widened to
-# 128), an incompatible ABI break, so bump current and reset revision/age.
-VERSION_INFO="3:0:0"
+VERSION_INFO="2:49:0"
 AM_MAINTAINER_MODE
 AC_USE_SYSTEM_EXTENSIONS
 

src/htsalias.c

@@ -41,24 +41,19 @@ Please visit our Website: http://www.httrack.com
 
 #define _NOT_NULL(a) ( (a!=NULL) ? (a) : "" )
 
-// COPY OF cmdl_ins in htscoremain.c
-/* Bytes left in x_argvblk from offset ptr. The offset can in principle outrun
-   the block (alias/doit.log expansion), so the copy aborts cleanly instead of
-   the subtraction wrapping to a huge unbounded size. */
-#define cmdl_room(bufsize, ptr)                                                \
-  ((ptr) < (size_t) (bufsize) ? (size_t) (bufsize) - (ptr) : 0)
-// Insert a command in the argc/argv (buff has total capacity bufsize)
-#define cmdl_ins(token, argc, argv, buff, bufsize, ptr)                        \
-  {                                                                            \
-    int i;                                                                     \
-    for (i = argc; i > 0; i--)                                                 \
-      argv[i] = argv[i - 1];                                                   \
-  }                                                                            \
-  argv[0] = (buff + ptr);                                                      \
-  strlcpybuff(argv[0], token, cmdl_room(bufsize, ptr));                        \
-  ptr += (int) (strlen(argv[0]) + 1);                                          \
+// COPY OF cmdl_ins in htsmain.c
+// Insert a command in the argc/argv
+#define cmdl_ins(token,argc,argv,buff,ptr) \
+  { \
+  int i; \
+  for(i=argc;i>0;i--)\
+  argv[i]=argv[i-1];\
+  } \
+  argv[0]=(buff+ptr); \
+  strcpybuff(argv[0],token); \
+  ptr += (int) (strlen(argv[0])+1); \
   argc++
-// END OF COPY OF cmdl_ins in htscoremain.c
+// END OF COPY OF cmdl_ins in htsmain.c
 
 /*
   Aliases for command-line and config file definitions
@@ -473,7 +468,7 @@ const char *optalias_help(const char *token) {
 */
 /* Note: NOT utf-8 */
 int optinclude_file(const char *name, int *argc, char **argv, char *x_argvblk,
-                    size_t x_argvblk_size, int *x_ptr) {
+                    int *x_ptr) {
   FILE *fp;
 
   fp = fopen(name, "rb");
@@ -547,15 +542,14 @@ int optinclude_file(const char *name, int *argc, char **argv, char *x_argvblk,
               /* temporary argc: Number of parameters after minus insert_after_argc */
               insert_after_argc = (*argc) - insert_after;
               cmdl_ins((tmp_argv[2]), insert_after_argc, (argv + insert_after),
-                       x_argvblk, x_argvblk_size, (*x_ptr));
+                       x_argvblk, (*x_ptr));
               *argc = insert_after_argc + insert_after;
               insert_after++;
               /* Second one */
               if (return_argc > 1) {
                 insert_after_argc = (*argc) - insert_after;
                 cmdl_ins((tmp_argv[3]), insert_after_argc,
-                         (argv + insert_after), x_argvblk, x_argvblk_size,
-                         (*x_ptr));
+                         (argv + insert_after), x_argvblk, (*x_ptr));
                 *argc = insert_after_argc + insert_after;
                 insert_after++;
               }

src/htsalias.h

@@ -45,7 +45,7 @@ int optalias_find(const char *token);
 const char *optalias_help(const char *token);
 int optreal_find(const char *token);
 int optinclude_file(const char *name, int *argc, char **argv, char *x_argvblk,
-                    size_t x_argvblk_size, int *x_ptr);
+                    int *x_ptr);
 const char *optreal_value(int p);
 const char *optalias_value(int p);
 const char *opttype_value(int p);

src/htsback.c

@@ -3584,9 +3584,8 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
                         back[i].r.is_file = 1;
                         back[i].r.totalsize = back[i].r.size =
                           fsize_utf8(back[i].url_sav);
-                        get_httptype_sized(opt, back[i].r.contenttype,
-                                           sizeof(back[i].r.contenttype),
-                                           back[i].url_sav, 1);
+                        get_httptype(opt, back[i].r.contenttype,
+                                     back[i].url_sav, 1);
                         hts_log_print(opt, LOG_DEBUG,
                                       "Not-modified status without cache guessed: %s%s",
                                       back[i].url_adr, back[i].url_fil);

src/htscore.c

@@ -1734,7 +1734,7 @@ int httpmirror(char *url1, httrackp * opt) {
                 {
                   char buff[256];
 
-                  guess_httptype_sized(opt, buff, sizeof(buff), urlfil());
+                  guess_httptype(opt, buff, urlfil());
                   if (strcmp(buff, "image/gif") == 0)
                     create_gif_warning = 1;
                 }
@@ -3150,7 +3150,7 @@ static void postprocess_file(httrackp * opt, const char *save, const char *adr,
           /* CID */
           make_content_id(adr, fil, cid, sizeof(cid));
 
-          guess_httptype_sized(opt, mimebuff, sizeof(mimebuff), save);
+          guess_httptype(opt, mimebuff, save);
           fprintf(opt->state.mimefp, "--%s\r\n",
                   StringBuff(opt->state.mimemid));
           /*if (first)
@@ -3862,8 +3862,7 @@ int htsAddLink(htsmoduleStruct * str, char *link) {
             opt->savename_83 = b;
             if (r != -1 && !forbidden_url) {
               if (savename()) {
-                if (lienrelatif(tempo, sizeof(tempo), afs.save, savename()) ==
-                    0) {
+                if (lienrelatif(tempo, afs.save, savename()) == 0) {
                   hts_log_print(opt, LOG_DEBUG,
                                 "(module): relative link at %s build with %s and %s: %s",
                                 afs.af.adr, afs.save, savename(), tempo);

src/htscoremain.c

@@ -69,29 +69,23 @@ Please visit our Website: http://www.httrack.com
 /* Resolver */
 extern int IPV6_resolver;
 
-/* Remaining room in the argv block; 0 once it is exhausted (alias expansion or
-   doit.log insertion can outrun the +32768 slack), so the copy aborts cleanly
-   instead of the subtraction wrapping to a huge unbounded size. */
-#define cmdl_room(bufsize, ptr)                                                \
-  ((ptr) < (size_t) (bufsize) ? (size_t) (bufsize) - (ptr) : 0)
-
-// Add a command in the argc/argv (buff has total capacity bufsize)
-#define cmdl_add(token, argc, argv, buff, bufsize, ptr)                        \
-  argv[argc] = (buff + ptr);                                                   \
-  strlcpybuff(argv[argc], token, cmdl_room(bufsize, ptr));                     \
-  ptr += (int) (strlen(argv[argc]) + 2);                                       \
+// Add a command in the argc/argv
+#define cmdl_add(token,argc,argv,buff,ptr) \
+  argv[argc]=(buff+ptr); \
+  strcpybuff(argv[argc],token); \
+  ptr += (int) (strlen(argv[argc])+2); \
   argc++
 
-// Insert a command in the argc/argv (buff has total capacity bufsize)
-#define cmdl_ins(token, argc, argv, buff, bufsize, ptr)                        \
-  {                                                                            \
-    int i;                                                                     \
-    for (i = argc; i > 0; i--)                                                 \
-      argv[i] = argv[i - 1];                                                   \
-  }                                                                            \
-  argv[0] = (buff + ptr);                                                      \
-  strlcpybuff(argv[0], token, cmdl_room(bufsize, ptr));                        \
-  ptr += (int) (strlen(argv[0]) + 2);                                          \
+// Insert a command in the argc/argv
+#define cmdl_ins(token,argc,argv,buff,ptr) \
+  { \
+  int i; \
+  for(i=argc;i>0;i--)\
+  argv[i]=argv[i-1];\
+  } \
+  argv[0]=(buff+ptr); \
+  strcpybuff(argv[0],token); \
+  ptr += (int) (strlen(argv[0])+2); \
   argc++
 
 #define htsmain_free() do { \
@@ -291,196 +285,6 @@ static void basic_selftests(void) {
       assertf(end == NULL && strcmp(tok, "a\\") == 0);
     }
   }
-  // fil_normalized(): canonicalizes a URL path. Query arguments are sorted
-  // alphabetically (by the text after each '?'/'&') and the query is rebuilt
-  // through a bounded builder; outside the query, "//" collapses to "/".
-  // Regression for that builder.
-  {
-    char norm[256];
-
-    assertf(strcmp(fil_normalized("/p?b=2&a=1&c=3", norm), "/p?a=1&b=2&c=3") ==
-            0);
-    assertf(strcmp(fil_normalized("/a//b", norm), "/a/b") == 0);
-    // "//" is collapsed only before the query; inside the query it is kept
-    assertf(strcmp(fil_normalized("/a//b?x=c//d", norm), "/a/b?x=c//d") == 0);
-  }
-  // give_mimext(): mime type -> file extension, bounded into the caller buffer.
-  // Returns 1 when an extension was written, 0 otherwise.
-  {
-    char ext[16];
-
-    assertf(give_mimext(ext, sizeof(ext), "image/gif") == 1);
-    assertf(strcmp(ext, "gif") == 0);
-    assertf(give_mimext(ext, sizeof(ext), "text/html") == 1);
-    assertf(strcmp(ext, "html") == 0);
-    assertf(give_mimext(ext, sizeof(ext), "no/such-mime-type") == 0);
-    assertf(ext[0] == '\0');
-  }
-  // convtolower(): lower-cases into the caller buffer (bounded by its size).
-  {
-    char low[64];
-
-    assertf(strcmp(convtolower(low, sizeof(low), "ABC/Def.HTML"),
-                   "abc/def.html") == 0);
-  }
-  // cut_path(): splits a path into directory (with trailing '/') and basename,
-  // each bounded by its buffer size.
-  {
-    char path[256];
-    char pname[256];
-
-    {
-      char full[] = "/dir/sub/file.html";
-
-      cut_path(full, path, sizeof(path), pname, sizeof(pname));
-      assertf(strcmp(path, "/dir/sub/") == 0);
-      assertf(strcmp(pname, "file.html") == 0);
-    }
-    { // a trailing slash is trimmed before the split
-      char full[] = "/dir/sub/";
-
-      cut_path(full, path, sizeof(path), pname, sizeof(pname));
-      assertf(strcmp(path, "/dir/") == 0);
-      assertf(strcmp(pname, "sub") == 0);
-    }
-    { // a path of length <= 1 yields empty results
-      char full[] = "/";
-
-      cut_path(full, path, sizeof(path), pname, sizeof(pname));
-      assertf(path[0] == '\0' && pname[0] == '\0');
-    }
-  }
-  // get_httptype_sized(): a long MIME type (Office OOXML reaches 73 chars) is
-  // written whole into a contenttype-sized buffer; returns 1 on a match, 0 when
-  // flag==0 and nothing matched. Regression for the old contenttype[64]
-  // overflow.
-  {
-    httrackp *opt = hts_create_opt();
-    htsblk r; // write into the real struct field, not a stand-in
-
-    assertf(opt != NULL);
-    // a long MIME (Office OOXML reaches 73 chars) must fit htsblk.contenttype
-    // whole: a [64] field would make this bounded copy abort.
-    assertf(get_httptype_sized(opt, r.contenttype, sizeof(r.contenttype),
-                               "deck.pptx", 0) == 1);
-    assertf(strcmp(r.contenttype,
-                   "application/vnd.openxmlformats-officedocument."
-                   "presentationml.presentation") == 0);
-    assertf(get_httptype_sized(opt, r.contenttype, sizeof(r.contenttype),
-                               "x.gif", 0) == 1);
-    assertf(strcmp(r.contenttype, "image/gif") == 0);
-    // no extension and flag==0: nothing written, returns 0
-    assertf(get_httptype_sized(opt, r.contenttype, sizeof(r.contenttype),
-                               "noextfile", 0) == 0);
-    assertf(r.contenttype[0] == '\0');
-    // no extension and flag==1: octet-stream fallback, returns 1
-    assertf(get_httptype_sized(opt, r.contenttype, sizeof(r.contenttype),
-                               "noextfile", 1) == 1);
-    assertf(strcmp(r.contenttype, "application/octet-stream") == 0);
-    // a user --assume rule with an empty value matches but writes nothing:
-    // get_userhttptype returns 1 with the buffer empty, so get_httptype_sized
-    // must still report 0 (callers test the return like the old
-    // strnotempty(s)).
-    StringCopy(opt->mimedefs, "\ncgi=\n");
-    assertf(get_httptype_sized(opt, r.contenttype, sizeof(r.contenttype),
-                               "/x.cgi", 0) == 0);
-    assertf(r.contenttype[0] == '\0');
-    StringCopy(opt->mimedefs, "\ncgi=text/html\n");
-    assertf(get_httptype_sized(opt, r.contenttype, sizeof(r.contenttype),
-                               "/x.cgi", 0) == 1);
-    assertf(strcmp(r.contenttype, "text/html") == 0);
-    hts_free_opt(opt);
-  }
-  // adr_normalized_sized(): bounded host normalization (passthrough when
-  // already normal).
-  {
-    char n[HTS_URLMAXSIZE];
-
-    assertf(strcmp(adr_normalized_sized("example.com", n, sizeof(n)),
-                   "example.com") == 0);
-  }
-  // standard_name(): builds "<name><md5?>.<ext>" into a bounded buffer. The md5
-  // is appended (4 chars) only when the URL has a query string (see url_md5),
-  // so test both; pin the structure (name + ext, lengths), not the md5 chars.
-  {
-    char b[HTS_URLMAXSIZE * 2];
-    const char *nom = "index.html"; // name part
-    const char *dot = nom + 5;      // points at ".html"
-    size_t len;
-
-    // no query -> no md5: "index" + ".html"
-    standard_name(b, sizeof(b), dot, nom, "http://example.com/index.html", 0);
-    assertf(strcmp(b, "index.html") == 0);
-    // query -> 4 md5 chars between name and ext: "index" + md5(4) + ".html"
-    standard_name(b, sizeof(b), dot, nom, "http://example.com/index.html?v=1",
-                  0);
-    len = strlen(b);
-    assertf(len == 5 + 4 + 5);
-    assertf(strncmp(b, "index", 5) == 0);
-    assertf(strcmp(b + len - 5, ".html") == 0);
-    // short names: name kept (<=8), the extension is clamped to 3 -> ".htm"
-    standard_name(b, sizeof(b), dot, nom, "http://example.com/index.html?v=1",
-                  1);
-    len = strlen(b);
-    assertf(len == 5 + 4 + 4);
-    assertf(strcmp(b + len - 4, ".htm") == 0);
-    // short names with a >8-char name: the name is clamped to 8 ("indexpag")
-    {
-      const char *lnom = "indexpage.html";
-      const char *ldot = lnom + 9; // points at ".html"
-
-      standard_name(b, sizeof(b), ldot, lnom,
-                    "http://example.com/indexpage.html?v=1", 1);
-      len = strlen(b);
-      assertf(len == 8 + 4 + 4);
-      assertf(strncmp(b, "indexpag", 8) == 0);
-      assertf(strcmp(b + len - 4, ".htm") == 0);
-    }
-  }
-  // longfile_to_83(): single-name 8-3 (mode 1) / ISO9660 (mode 2) conversion;
-  // uppercases, clamps the name (8 / 31) and the extension (3). It rewrites
-  // 'save' in place, so pass a mutable array.
-  {
-    char n83[256];
-
-    {
-      char save[] = "longfilename.html";
-
-      longfile_to_83(1, n83, sizeof(n83), save); // 8-3: name->8, ext->3
-      assertf(strcmp(n83, "LONGFILE.HTM") == 0);
-    }
-    {
-      char save[] = "longfilename.html";
-
-      longfile_to_83(2, n83, sizeof(n83), save); // ISO9660: name->31, ext->3
-      assertf(strcmp(n83, "LONGFILENAME.HTM") == 0);
-    }
-    { // sanitization: leading '.'->'_', interior dots
-      char save[] = ".a b.c.d e"; // collapse to '_', spaces/specials -> '_'
-                                  // (only the last dot stays as the separator)
-      longfile_to_83(1, n83, sizeof(n83), save);
-      assertf(strcmp(n83, "_A_B_C.D_E") == 0);
-    }
-  }
-  // long_to_83(): per-segment 8-3 conversion of a whole path.
-  {
-    char n83[HTS_URLMAXSIZE * 2];
-    char save[] = "dir/longfilename.html";
-
-    long_to_83(1, n83, sizeof(n83), save);
-    assertf(strcmp(n83, "DIR/LONGFILE.HTM") == 0);
-  }
-  // lienrelatif(): relative path from the directory of curr_fil to link.
-  {
-    char s[HTS_URLMAXSIZE * 2];
-
-    // same directory -> just the basename
-    assertf(lienrelatif(s, sizeof(s), "dir/page.html", "dir/index.html") == 0);
-    assertf(strcmp(s, "page.html") == 0);
-    // link one level up -> a "../" prefix
-    assertf(lienrelatif(s, sizeof(s), "a.html", "dir/index.html") == 0);
-    assertf(strcmp(s, "../a.html") == 0);
-  }
 }
 
 /* Self-tests for the htssafe.h bounded string ops (driven by httrack -#8).
@@ -598,7 +402,6 @@ HTSEXT_API int hts_main2(int argc, char **argv, httrackp * opt) {
 static int hts_main_internal(int argc, char **argv, httrackp * opt) {
   char **x_argv = NULL;         // Patch pour argv et argc: en cas de récupération de ligne de commande
   char *x_argvblk = NULL;       // (reprise ou update)
-  size_t x_argvblk_size = 0;    // total capacity of x_argvblk
   int x_ptr = 0;                // offset
 
   //
@@ -676,8 +479,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
         *a = ' ';
       /* equivalent to "empty parameter" */
       if ((strcmp(argv[na], HTS_NOPARAM) == 0) || (strcmp(argv[na], HTS_NOPARAM2) == 0))        // (none)
-        /* replacing "(none)"/"\"(none)\"" with "\"\"" always fits in place */
-        strlcpybuff(argv[na], "\"\"", strlen(argv[na]) + 1);
+        strcpybuff(argv[na], "\"\"");
       if (strncmp(argv[na], "-&", 2) == 0)
         argv[na][1] = '%';
     }
@@ -699,7 +501,6 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
       htsmain_free();
       return -1;
     }
-    x_argvblk_size = (size_t) (current_size + 32768);
     x_argvblk[0] = '\0';
     x_ptr = 0;
 
@@ -721,7 +522,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
     //
     argv_url = 0;               /* pour comptage */
     //
-    cmdl_add(argv[0], x_argc, x_argv, x_argvblk, x_argvblk_size, x_ptr);
+    cmdl_add(argv[0], x_argc, x_argv, x_argvblk, x_ptr);
     na = 1;                     /* commencer après nom_prg */
     while(na < argc) {
       int result = 1;
@@ -742,10 +543,9 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
         }
 
         /* Copier */
-        cmdl_add(tmp_argv[0], x_argc, x_argv, x_argvblk, x_argvblk_size, x_ptr);
+        cmdl_add(tmp_argv[0], x_argc, x_argv, x_argvblk, x_ptr);
         if (tmp_argc > 1) {
-          cmdl_add(tmp_argv[1], x_argc, x_argv, x_argvblk, x_argvblk_size,
-                   x_ptr);
+          cmdl_add(tmp_argv[1], x_argc, x_argv, x_argvblk, x_ptr);
         }
 
         /* Compter URLs et détecter -i,-q.. */
@@ -817,7 +617,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
           char BIGSTK tempo[HTS_CDLMAXSIZE];
 
           strcpybuff(tempo, argv[na] + 1);
-          if (tempo[0] == '\0' || tempo[strlen(tempo) - 1] != '"') {
+          if (tempo[strlen(tempo) - 1] != '"') {
             char BIGSTK s[HTS_CDLMAXSIZE];
 
             sprintf(s, "Missing quote in %s", argv[na]);
@@ -826,9 +626,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
             return -1;
           }
           tempo[strlen(tempo) - 1] = '\0';
-          /* tempo is argv[na] minus its surrounding quotes, so it fits in place
-           */
-          strlcpybuff(argv[na], tempo, strlen(argv[na]) + 1);
+          strcpybuff(argv[na], tempo);
         }
 
         if (cmdl_opt(argv[na])) {       // option
@@ -929,19 +727,18 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
               (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
               StringBuff(opt->path_log),
                "hts-cache/doit.log"))) || (argv_url > 0)) {
-          if (!optinclude_file(
-                  fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
-                          StringBuff(opt->path_log), HTS_HTTRACKRC),
-                  &argc, argv, x_argvblk, x_argvblk_size, &x_ptr))
-            if (!optinclude_file(HTS_HTTRACKRC, &argc, argv, x_argvblk,
-                                 x_argvblk_size, &x_ptr)) {
-              if (!optinclude_file(
-                      fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
-                              hts_gethome(), "/" HTS_HTTRACKRC),
-                      &argc, argv, x_argvblk, x_argvblk_size, &x_ptr)) {
+          if (!optinclude_file
+              (fconcat
+               (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+               StringBuff(opt->path_log), HTS_HTTRACKRC),
+               &argc, argv, x_argvblk, &x_ptr))
+            if (!optinclude_file(HTS_HTTRACKRC, &argc, argv, x_argvblk, &x_ptr)) {
+              if (!optinclude_file
+                  (fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                  hts_gethome(), "/" HTS_HTTRACKRC),
+                   &argc, argv, x_argvblk, &x_ptr)) {
 #ifdef HTS_HTTRACKCNF
-                optinclude_file(HTS_HTTRACKCNF, &argc, argv, x_argvblk,
-                                x_argvblk_size, &x_ptr);
+                optinclude_file(HTS_HTTRACKCNF, &argc, argv, x_argvblk, &x_ptr);
 #endif
               }
             }
@@ -994,7 +791,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
           if (strnotempty(lastp)) {
             insert_after_argc = argc - insert_after;
             cmdl_ins(lastp, insert_after_argc, (argv + insert_after), x_argvblk,
-                     x_argvblk_size, x_ptr);
+                     x_ptr);
             argc = insert_after_argc + insert_after;
             insert_after++;
           }
@@ -1114,7 +911,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
       if (argv[i][0] == '-') {
         if (argv[i][1] == '-') {        // --xxx
           if ((strfield2(argv[i] + 2, "clean")) || (strfield2(argv[i] + 2, "tide"))) {  // nettoyer
-            argv[i][1] = '\0';
+            strcpybuff(argv[i] + 1, "");
             if (fexist
                 (fconcat
                  (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-log.txt")))
@@ -1223,8 +1020,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
             //
           } else if (strfield2(argv[i] + 2, "catchurl")) {      // capture d'URL via proxy temporaire!
             argv_url = 1;       // forcer a passer les parametres
-            /* argv[i] is "--catchurl"; "#P" fits after its first char */
-            strlcpybuff(argv[i] + 1, "#P", strlen(argv[i] + 1) + 1);
+            strcpybuff(argv[i] + 1, "#P");
             //
           } else if (strfield2(argv[i] + 2, "updatehttrack")) {
 #ifdef _WIN32
@@ -1552,7 +1348,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
         char BIGSTK tempo[HTS_CDLMAXSIZE + 256];
 
         strcpybuff(tempo, argv[na] + 1);
-        if (tempo[0] == '\0' || tempo[strlen(tempo) - 1] != '"') {
+        if (tempo[strlen(tempo) - 1] != '"') {
           char s[HTS_CDLMAXSIZE + 256];
 
           sprintf(s, "Missing quote in %s", argv[na]);
@@ -1561,9 +1357,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
           return -1;
         }
         tempo[strlen(tempo) - 1] = '\0';
-        /* tempo is argv[na] minus its surrounding quotes, so it fits in place
-         */
-        strlcpybuff(argv[na], tempo, strlen(argv[na]) + 1);
+        strcpybuff(argv[na], tempo);
       }
 
       if (cmdl_opt(argv[na])) { // option
@@ -2804,12 +2598,15 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
                   // initialiser mimedefs
                   //get_userhttptype(opt,1,opt->mimedefs,NULL);
                   // check
-                  if (get_httptype_sized(opt, mime, sizeof(mime), argv[na + 1],
-                                         0)) {
+                  mime[0] = '\0';
+                  get_httptype(opt, mime, argv[na + 1], 0);
+                  if (mime[0] != '\0') {
                     char ext[256];
 
                     printf("%s is '%s'\n", argv[na + 1], mime);
-                    if (give_mimext(ext, sizeof(ext), mime)) {
+                    ext[0] = '\0';
+                    give_mimext(ext, mime);
+                    if (ext[0]) {
                       printf("and its local type is '.%s'\n", ext);
                     }
                   } else {
@@ -3222,7 +3019,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
         if (urlSize < HTS_URLMAXSIZE) {
           ensureUrlCapacity(url, url_sz, capa);
           if (strnotempty(url))
-            strlcatbuff(url, " ", url_sz); // separator space
+            strcatbuff(url, " ");       // espace de séparation
           append_escape_spc_url(unescape_http_unharm(catbuff, sizeof(catbuff), argv[na], 1), url, url_sz);
         }
       }                         // if argv=- etc. 

src/htsglobal.h

@@ -197,13 +197,10 @@ Please visit our Website: http://www.httrack.com
 
 #endif
 
-/* Max URL length */
+/* Taille max d'une URL */
 #define HTS_URLMAXSIZE 1024
-/* Max command-line length (>=HTS_URLMAXSIZE*2) */
+/* Taille max ligne de commande (>=HTS_URLMAXSIZE*2) */
 #define HTS_CDLMAXSIZE 1024
-/* MIME-type buffer contract (htsblk.contenttype/charset/contentencoding); holds
-   the longest registered MIME type, the Office OOXML ones reaching 73 chars */
-#define HTS_MIMETYPE_SIZE 128
 
 /* Copyright (C) 1998 Xavier Roche and other contributors */
 #define HTTRACK_AFF_AUTHORS "[XR&CO'2014]"
@@ -253,22 +250,6 @@ Please visit our Website: http://www.httrack.com
 #endif
 #endif
 
-/**
- * Mark a function deprecated, with a message pointing at the replacement.
- * Placed before the declaration so both the GCC/Clang attribute and the MSVC
- * __declspec sit in a position both accept. Degrades to nothing elsewhere.
- */
-#if defined(__GNUC__) &&                                                       \
-    (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 5))
-#define HTS_DEPRECATED(msg) __attribute__((deprecated(msg)))
-#elif defined(__GNUC__)
-#define HTS_DEPRECATED(msg) __attribute__((deprecated))
-#elif defined(_MSC_VER) && (_MSC_VER >= 1400)
-#define HTS_DEPRECATED(msg) __declspec(deprecated(msg))
-#else
-#define HTS_DEPRECATED(msg)
-#endif
-
 #ifndef HTS_LONGLONG
 #ifdef HTS_NO_64_BIT
 #define HTS_LONGLONG 0

src/htshash.c

@@ -76,7 +76,7 @@ static coucal_key key_duphandler(void *arg, coucal_key_const name) {
 /* Key sav hashes are using case-insensitive version */
 static coucal_hashkeys key_sav_hashes(void *arg, coucal_key_const key) {
   hash_struct *const hash = (hash_struct*) arg;
-  convtolower(hash->catbuff, sizeof(hash->catbuff), (const char *) key);
+  convtolower(hash->catbuff, (const char*) key);
   return coucal_hash_string(hash->catbuff);
 }
 

src/htsjava.c

@@ -472,8 +472,9 @@ static int tris(httrackp * opt, char *buffer) {
   {
     char type[256];
 
-    if (get_httptype_sized(opt, type, sizeof(type), buffer,
-                           0)) // recognized type
+    type[0] = '\0';
+    get_httptype(opt, type, buffer, 0);
+    if (strnotempty(type))      // type reconnu!
       return 1;
     // ajout RX 05/2001
     else if (is_dyntype(get_ext(catbuff, sizeof(catbuff), buffer)))      // asp,cgi...

src/htslib.c

@@ -754,8 +754,7 @@ T_SOC http_xfopen(httrackp * opt, int mode, int treat, int waitconnect,
       if (soc != INVALID_SOCKET) {
         retour->statuscode = HTTP_OK;   // OK
         strcpybuff(retour->msg, "OK");
-        guess_httptype_sized(opt, retour->contenttype,
-                             sizeof(retour->contenttype), fil);
+        guess_httptype(opt, retour->contenttype, fil);
       } else if (strnotempty(retour->msg) == 0)
         strcpybuff(retour->msg, "Unable to open local file");
       return soc;               // renvoyer
@@ -1531,9 +1530,8 @@ void treathead(t_cookie * cookie, const char *adr, const char *fil, htsblk * ret
       if (retour->location) {
         while(is_realspace(*(rcvd + p)))
           p++;                  // sauter espaces
-        if ((int) strlen(rcvd + p) < HTS_URLMAXSIZE) // not too long?
-          /* location aliases location_buffer[HTS_URLMAXSIZE * 2] */
-          strlcpybuff(retour->location, rcvd + p, HTS_URLMAXSIZE * 2);
+        if ((int) strlen(rcvd + p) < HTS_URLMAXSIZE)    // pas trop long?
+          strcpybuff(retour->location, rcvd + p);
         else                    // erreur.. ignorer
           retour->location[0] = '\0';
       }
@@ -3446,17 +3444,16 @@ HTSEXT_API char *fil_normalized(const char *source, char *dest) {
     /* Replace query by sorted query */
     copyBuff = malloct(qLen + 1);
     assertf(copyBuff != NULL);
-    {
-      htsbuff cb = htsbuff_ptr(copyBuff, qLen + 1);
-
-      for (i = 0; i < ampargs; i++) {
-        htsbuff_cat(&cb, i == 0 ? "?" : "&");
-        htsbuff_cat(&cb, amps[i] + 1);
-      }
-      assertf(cb.len == qLen);
+    copyBuff[0] = '\0';
+    for(i = 0; i < ampargs; i++) {
+      if (i == 0)
+        strcatbuff(copyBuff, "?");
+      else
+        strcatbuff(copyBuff, "&");
+      strcatbuff(copyBuff, amps[i] + 1);
     }
-    /* query points into dest where the original qLen-byte query was */
-    strlcpybuff(query, copyBuff, qLen + 1);
+    assertf(strlen(copyBuff) == qLen);
+    strcpybuff(query, copyBuff);
 
     /* Cleanup */
     freet(amps);
@@ -3467,17 +3464,10 @@ HTSEXT_API char *fil_normalized(const char *source, char *dest) {
 }
 
 #define endwith(a) ( (len >= (sizeof(a)-1)) ? ( strncmp(dest, a+len-(sizeof(a)-1), sizeof(a)-1) == 0 ) : 0 );
-HTSEXT_API char *adr_normalized_sized(const char *source, char *dest,
-                                      size_t destsize) {
-  /* not yet too aggressive (no com<->net<->org checkings) */
-  strlcpybuff(dest, jump_normalized_const(source), destsize);
-  return dest;
-}
-
-// deprecated variant; kept for ABI compatibility. Bounds to the implicit
-// contract the old callers relied on (an HTS_URLMAXSIZE*2 URL buffer).
 HTSEXT_API char *adr_normalized(const char *source, char *dest) {
-  return adr_normalized_sized(source, dest, HTS_URLMAXSIZE * 2);
+  /* not yet too aggressive (no com<->net<->org checkings) */
+  strcpybuff(dest, jump_normalized_const(source));
+  return dest;
 }
 
 #undef endwith
@@ -3904,9 +3894,9 @@ HTSEXT_API size_t escape_for_html_print_full(const char *const s, char *const de
 
 #undef ADD_CHAR
 
-// lower-case conversion into caller buffer (capacity catbuffsize)
-char *convtolower(char *catbuff, size_t catbuffsize, const char *a) {
-  strlcpybuff(catbuff, a, catbuffsize);
+// conversion minuscules, avec buffer
+char *convtolower(char *catbuff, const char *a) {
+  strcpybuff(catbuff, a);
   hts_lowcase(catbuff);         // lower case
   return catbuff;
 }
@@ -3929,34 +3919,22 @@ void hts_replace(char *s, char from, char to) {
   }
 }
 
-// guess a local file's mime type (e.g. fil="toto.gif" -> s="image/gif")
-// returns 1 if a type was written to s, 0 otherwise
-int guess_httptype_sized(httrackp *opt, char *s, size_t ssize,
-                         const char *fil) {
-  return get_httptype_sized(opt, s, ssize, fil, 1);
-}
-
-// deprecated variant; kept for ABI compatibility. Bounds to the implicit
-// contract the old callers relied on (a contenttype-sized buffer).
+// deviner type d'un fichier local..
+// ex: fil="toto.gif" -> s="image/gif"
 void guess_httptype(httrackp * opt, char *s, const char *fil) {
-  (void) get_httptype_sized(opt, s, HTS_MIMETYPE_SIZE, fil, 1);
+  get_httptype(opt, s, fil, 1);
 }
 
-// write the mime type for fil into s (capacity ssize)
-// flag: 1 to always return a type (the "application/..." / octet-stream
-// fallback) returns 1 if a type was written to s, 0 otherwise
-HTSEXT_API int get_httptype_sized(httrackp *opt, char *s, size_t ssize,
-                                  const char *fil, int flag) {
-  // userdef overrides get_httptype (a rule with an empty value, e.g. "--assume
-  // cgi=", matches but writes nothing: report it as "no type" like the old
-  // code, whose callers tested strnotempty(s))
+// idem
+// flag: 1 si toujours renvoyer un type
+HTSEXT_API void get_httptype(httrackp * opt, char *s, const char *fil, int flag) {
+  // userdef overrides get_httptype
   if (get_userhttptype(opt, s, fil)) {
-    return s[0] != '\0';
+    return;
   }
   // regular tests
   if (ishtml(opt, fil) == 1) {
-    strlcpybuff(s, "text/html", ssize);
-    return 1;
+    strcpybuff(s, "text/html");
   } else {
     /* Check html -> text/html */
     const char *a = fil + strlen(fil) - 1;
@@ -3969,33 +3947,21 @@ HTSEXT_API int get_httptype_sized(httrackp *opt, char *s, size_t ssize,
       a++;
       while(strnotempty(hts_mime[j][1])) {
         if (strfield2(hts_mime[j][1], a)) {
-          if (hts_mime[j][0][0] != '*') { // a match exists
-            strlcpybuff(s, hts_mime[j][0], ssize);
-            return 1;
+          if (hts_mime[j][0][0] != '*') {       // Une correspondance existe
+            strcpybuff(s, hts_mime[j][0]);
+            return;
           }
         }
         j++;
       }
 
-      if (flag) {
-        snprintf(s, ssize, "application/%s", a);
-        return 1;
-      }
+      if (flag)
+        sprintf(s, "application/%s", a);
     } else {
-      if (flag) {
-        strlcpybuff(s, "application/octet-stream", ssize);
-        return 1;
-      }
+      if (flag)
+        strcpybuff(s, "application/octet-stream");
     }
   }
-  return 0;
-}
-
-// deprecated variant; kept for ABI compatibility. Bounds to the implicit
-// contract the old callers relied on (a contenttype-sized buffer).
-HTSEXT_API void get_httptype(httrackp *opt, char *s, const char *fil,
-                             int flag) {
-  (void) get_httptype_sized(opt, s, HTS_MIMETYPE_SIZE, fil, flag);
 }
 
 // get type of fil (php)
@@ -4105,17 +4071,17 @@ int get_userhttptype(httrackp * opt, char *s, const char *fil) {
   return 0;
 }
 
-// give the file extension for a mime type (e.g. "image/gif" -> "gif")
-// returns 1 if an extension was found (and written to s), 0 otherwise
-int give_mimext(char *s, size_t ssize, const char *st) {
+// renvoyer extesion d'un type mime..
+// ex: "image/gif" -> gif
+void give_mimext(char *s, const char *st) {
   int ok = 0;
   int j = 0;
 
   s[0] = '\0';
   while((!ok) && (strnotempty(hts_mime[j][1]))) {
     if (strfield2(hts_mime[j][0], st)) {
-      if (hts_mime[j][1][0] != '*') { // a match exists
-        strlcpybuff(s, hts_mime[j][1], ssize);
+      if (hts_mime[j][1][0] != '*') {   // Une correspondance existe
+        strcpybuff(s, hts_mime[j][1]);
         ok = 1;
       }
     }
@@ -4136,13 +4102,12 @@ int give_mimext(char *s, size_t ssize, const char *st) {
     if (a) {
       if ((int) strlen(a) >= 1) {
         if ((int) strlen(a) <= 4) {
-          strlcpybuff(s, a, ssize);
+          strcpybuff(s, a);
           ok = 1;
         }
       }
     }
   }
-  return ok;
 }
 
 // extension connue?..
@@ -4240,8 +4205,9 @@ int may_bogus_multiple(httrackp * opt, const char *mime, const char *filename) {
     if (strfield2(hts_mime_bogus_multiple[j], mime)) {  /* found mime type in suspicious list */
       char ext[64];
 
-      if (give_mimext(ext, sizeof(ext),
-                      mime)) { /* we have an extension for that */
+      ext[0] = '\0';
+      give_mimext(ext, mime);
+      if (ext[0] != 0) {        /* we have an extension for that */
         const size_t ext_size = strlen(ext);
         const char *file = strrchr(filename, '/');      /* fetch terminal filename */
 
@@ -4964,8 +4930,7 @@ void hts_freeall(void) {
 
 // cut path and project name
 // patch also initial path
-void cut_path(char *fullpath, char *path, size_t path_size, char *pname,
-              size_t pname_size) {
+void cut_path(char *fullpath, char *path, char *pname) {
   path[0] = pname[0] = '\0';
   if (strnotempty(fullpath)) {
     if ((fullpath[strlen(fullpath) - 1] == '/')
@@ -4981,8 +4946,8 @@ void cut_path(char *fullpath, char *path, size_t path_size, char *pname,
         a--;
       if (*a == '/')
         a++;
-      strlcpybuff(pname, a, pname_size);
-      strlncatbuff(path, fullpath, path_size, (size_t) (a - fullpath));
+      strcpybuff(pname, a);
+      strncatbuff(path, fullpath, (int) (a - fullpath));
     }
   }
 }

src/htslib.h

@@ -252,7 +252,7 @@ int ishtml_ext(const char *a);
 int ishttperror(int err);
 
 int get_userhttptype(httrackp * opt, char *s, const char *fil);
-int give_mimext(char *s, size_t ssize, const char *st);
+void give_mimext(char *s, const char *st);
 
 int may_bogus_multiple(httrackp * opt, const char *mime, const char *filename);
 int may_unknown2(httrackp * opt, const char *mime, const char *filename);
@@ -264,7 +264,7 @@ void code64(unsigned char *a, int size_a, unsigned char *b, int crlf);
 
 #define copychar(catbuff,a) concat(catbuff,(a),NULL)
 
-char *convtolower(char *catbuff, size_t catbuffsize, const char *a);
+char *convtolower(char *catbuff, const char *a);
 void hts_lowcase(char *s);
 void hts_replace(char *s, char from, char to);
 int multipleStringMatch(const char *s, const char *match);
@@ -276,8 +276,7 @@ void fprintfio(FILE * fp, const char *buff, const char *prefix);
 int sig_ignore_flag(int setflag);  // flag ignore
 #endif
 
-void cut_path(char *fullpath, char *path, size_t path_size, char *pname,
-              size_t pname_size);
+void cut_path(char *fullpath, char *path, char *pname);
 int fexist(const char *s);
 int fexist_utf8(const char *s);
 
@@ -500,8 +499,7 @@ HTS_STATIC int is_hypertext_mime(httrackp * opt, const char *mime,
     char guessed[256];
 
     guessed[0] = '\0';
-    if (!guess_httptype_sized(opt, guessed, sizeof(guessed), file))
-      return 0;
+    guess_httptype(opt, guessed, file);
     return is_hypertext_mime__(guessed);
   }
   return 0;
@@ -516,8 +514,7 @@ HTS_STATIC int may_be_hypertext_mime(httrackp * opt, const char *mime,
     char guessed[256];
 
     guessed[0] = '\0';
-    if (!guess_httptype_sized(opt, guessed, sizeof(guessed), file))
-      return 0;
+    guess_httptype(opt, guessed, file);
     return may_be_hypertext_mime__(guessed);
   }
   return 0;
@@ -532,8 +529,7 @@ HTS_STATIC int compare_mime(httrackp * opt, const char *mime, const char *file,
     char guessed[256];
 
     guessed[0] = '\0';
-    if (!guess_httptype_sized(opt, guessed, sizeof(guessed), file))
-      return 0;
+    guess_httptype(opt, guessed, file);
     return strfield2(guessed, reference);
   }
   return 0;

src/htsname.c

@@ -51,13 +51,12 @@ Please visit our Website: http://www.httrack.com
       url_savename_addstr(afs->save, buff);\
     }
 
-#define ADD_STANDARD_NAME(shortname)                                           \
-  { /* add name */                                                             \
-    char BIGSTK buff[HTS_URLMAXSIZE * 2];                                      \
-    standard_name(buff, sizeof(buff), dot_pos, nom_pos, fil_complete,          \
-                  (shortname));                                                \
-    url_savename_addstr(afs->save, buff);                                      \
-  }
+#define ADD_STANDARD_NAME(shortname) \
+    {  /* ajout nom */\
+      char BIGSTK buff[HTS_URLMAXSIZE*2];\
+      standard_name(buff,dot_pos,nom_pos,fil_complete,(shortname));\
+      url_savename_addstr(afs->save, buff);\
+    }
 
 /* Avoid stupid DOS system folders/file such as 'nul' */
 /* Based on linux/fs/umsdos/mangle.c */
@@ -201,7 +200,7 @@ int url_savename(lien_adrfilsave *const afs,
   // foo.com/bar//foobar -> foo.com/bar/foobar
   if (opt->urlhack) {
     // copy of adr (without protocol), used for lookups (see urlhack)
-    normadr = adr_normalized_sized(adr, normadr_, sizeof(normadr_));
+    normadr = adr_normalized(adr, normadr_);
     normfil = fil_normalized(fil_complete, normfil_);
   } else {
     if (link_has_authority(adr_complete)) {     // https or other protocols : in "http/" subfolder
@@ -345,7 +344,8 @@ int url_savename(lien_adrfilsave *const afs,
         mime[0] = ext[0] = '\0';
         get_userhttptype(opt, mime, fil);
         if (strnotempty(mime)) {
-          if (give_mimext(ext, sizeof(ext), mime)) {
+          give_mimext(ext, mime);
+          if (strnotempty(ext)) {
             ext_chg = 1;
           }
         }
@@ -378,8 +378,8 @@ int url_savename(lien_adrfilsave *const afs,
                 ext_chg = 2;      /* change filename */
                 strcpybuff(ext, r.cdispo);
               } else if (!may_unknown2(opt, r.contenttype, fil)) {        // on peut patcher à priori?
-                if (give_mimext(s, sizeof(s),
-                                r.contenttype)) { // recognized extension
+                give_mimext(s, r.contenttype);    // obtenir extension
+                if (strnotempty(s) > 0) { // on a reconnu l'extension
                   ext_chg = 1;
                   strcpybuff(ext, s);
                 }
@@ -403,7 +403,8 @@ int url_savename(lien_adrfilsave *const afs,
             mime[0] = ext[0] = '\0';
             get_userhttptype(opt, mime, fil);
             if (strnotempty(mime)) {
-              if (give_mimext(ext, sizeof(ext), mime)) {
+              give_mimext(ext, mime);
+              if (strnotempty(ext)) {
                 ext_chg = 1;
               }
             }
@@ -419,9 +420,9 @@ int url_savename(lien_adrfilsave *const afs,
                 strcpybuff(ext, headers->r.cdispo);
               } else if (!may_unknown2(opt, headers->r.contenttype, headers->url_fil)) {    // on peut patcher à priori? (pas interdit ou pas de type)
                 char s[16];
-                if (give_mimext(
-                        s, sizeof(s),
-                        headers->r.contenttype)) { // recognized extension
+                s[0] = '\0';
+                give_mimext(s, headers->r.contenttype);    // obtenir extension
+                if (strnotempty(s) > 0) { // on a reconnu l'extension
                   ext_chg = 1;
                   strcpybuff(ext, s);
                 }
@@ -430,14 +431,13 @@ int url_savename(lien_adrfilsave *const afs,
             else if (mime_type != NULL) {
               ext[0] = '\0';
               if (*mime_type) {
-                give_mimext(ext, sizeof(ext), mime_type);
+                give_mimext(ext, mime_type);
               }
               if (strnotempty(ext)) {
                 char mime_from_file[128];
 
                 mime_from_file[0] = 0;
-                get_httptype_sized(opt, mime_from_file, sizeof(mime_from_file),
-                                   fil, 1);
+                get_httptype(opt, mime_from_file, fil, 1);
                 if (!strnotempty(mime_from_file) || strcasecmp(mime_type, mime_from_file) != 0) {       /* different mime for this type */
                   /* type change not forbidden (or no extension at all) */
                   if (!may_unknown2(opt, mime_type, fil)) {
@@ -646,9 +646,8 @@ int url_savename(lien_adrfilsave *const afs,
                       ext_chg = 2;      /* change filename */
                       strcpybuff(ext, back[b].r.cdispo);
                     } else if (!may_unknown2(opt, back[b].r.contenttype, back[b].url_fil)) {    // on peut patcher à priori? (pas interdit ou pas de type)
-                      if (give_mimext(
-                              s, sizeof(s),
-                              back[b].r.contenttype)) { // recognized extension
+                      give_mimext(s, back[b].r.contenttype);    // obtenir extension
+                      if (strnotempty(s) > 0) { // on a reconnu l'extension
                         ext_chg = 1;
                         strcpybuff(ext, s);
                       }
@@ -925,7 +924,7 @@ int url_savename(lien_adrfilsave *const afs,
 
               pth[0] = n83[0] = '\0';
               strncatbuff(pth, fil, (int) (nom_pos - fil) - 1);
-              long_to_83(opt->savename_83, n83, sizeof(n83), pth);
+              long_to_83(opt->savename_83, n83, pth);
               htsbuff_cat(&sb, n83);
             }
           }
@@ -1307,7 +1306,7 @@ int url_savename(lien_adrfilsave *const afs,
   if (opt->savename_83) {
     char BIGSTK n83[HTS_URLMAXSIZE * 2];
 
-    long_to_83(opt->savename_83, n83, sizeof(n83), afs->save);
+    long_to_83(opt->savename_83, n83, afs->save);
     strcpybuff(afs->save, n83);
   }
   // enforce stricter ISO9660 compliance (bug reported by Steffo Carlsson)
@@ -1378,9 +1377,7 @@ int url_savename(lien_adrfilsave *const afs,
     if (lastDot == NULL) {
       strcatbuff(afs->save, "." DELAYED_EXT);
     } else if (!IS_DELAYED_EXT(afs->save)) {
-      /* lastDot points within afs->save; bound by the remaining capacity */
-      strlcatbuff(lastDot, "." DELAYED_EXT,
-                  sizeof(afs->save) - (size_t) (lastDot - afs->save));
+      strcatbuff(lastDot, "." DELAYED_EXT);
     }
   }
   // enforce 260-character path limit before inserting destination path
@@ -1585,41 +1582,41 @@ int url_savename(lien_adrfilsave *const afs,
   return 0;
 }
 
-/* md5-based name used everywhere; builds into b (capacity bsize) */
-void standard_name(char *b, size_t bsize, const char *dot_pos,
-                   const char *nom_pos, const char *fil, int short_ver) {
+/* nom avec md5 urilisé partout */
+void standard_name(char *b, const char *dot_pos, const char *nom_pos, const char *fil,
+                   int short_ver) {
   char md5[32 + 2];
-  htsbuff bb = htsbuff_ptr(b, bsize);
 
-  /* Name */
+  b[0] = '\0';
+  /* Nom */
   if (dot_pos) {
-    if (!short_ver) // long names
-      htsbuff_catn(&bb, nom_pos, (size_t) (dot_pos - nom_pos));
+    if (!short_ver)             // Noms longs
+      strncatbuff(b, nom_pos, (dot_pos - nom_pos));
     else
-      htsbuff_catn(&bb, nom_pos, (size_t) min(dot_pos - nom_pos, 8));
+      strncatbuff(b, nom_pos, min(dot_pos - nom_pos, 8));
   } else {
-    if (!short_ver) // long names
-      htsbuff_cat(&bb, nom_pos);
+    if (!short_ver)             // Noms longs
+      strcatbuff(b, nom_pos);
     else
-      htsbuff_catn(&bb, nom_pos, 8);
+      strncatbuff(b, nom_pos, 8);
   }
   /* MD5 - 16 bits */
-  htsbuff_catn(&bb, url_md5(md5, fil), 4);
+  strncatbuff(b, url_md5(md5, fil), 4);
   /* Ext */
   if (dot_pos) {
-    htsbuff_catc(&bb, '.');
-    if (!short_ver) // long names
-      htsbuff_cat(&bb, dot_pos + 1);
+    strcatbuff(b, ".");
+    if (!short_ver)             // Noms longs
+      strcatbuff(b, dot_pos + 1);
     else
-      htsbuff_catn(&bb, dot_pos + 1, 3);
+      strncatbuff(b, dot_pos + 1, 3);
   }
   // Allow extensionless
 #ifdef DO_NOT_ALLOW_EXTENSIONLESS
   else {
-    if (!short_ver) // long names
-      htsbuff_cat(&bb, DEFAULT_EXT);
+    if (!short_ver)             // Noms longs
+      strcatbuff(b, DEFAULT_EXT);
     else
-      htsbuff_cat(&bb, DEFAULT_EXT_SHORT);
+      strcatbuff(b, DEFAULT_EXT_SHORT);
   }
 #endif
 }

src/htsname.h

@@ -96,8 +96,8 @@ int url_savename(lien_adrfilsave *const afs,
                  httrackp * opt, struct_back * sback, cache_back * cache,
                  hash_struct * hash, int ptr, int numero_passe,
                  const lien_back * headers);
-void standard_name(char *b, size_t bsize, const char *dot_pos,
-                   const char *nom_pos, const char *fil_complete,
+void standard_name(char *b, const char *dot_pos, const char *nom_pos,
+                   const char *fil_complete,
                    int short_ver);
 void url_savename_addstr(char *d, const char *s);
 char *url_md5(char *digest_buffer, const char *fil_complete);

src/htsopt.h

@@ -499,9 +499,9 @@ struct htsblk {
   FILE *out;                    // écriture directe sur disque (si is_write=1)
   LLint size;                   // taille fichier
   char msg[80];                 // message éventuel si échec ("\0"=non précisé)
-  char contenttype[HTS_MIMETYPE_SIZE];     // content-type (e.g. "text/html")
-  char charset[HTS_MIMETYPE_SIZE];         // charset (e.g. "iso-8859-1")
-  char contentencoding[HTS_MIMETYPE_SIZE]; // content-encoding (e.g. "gzip")
+  char contenttype[64];         // content-type ("text/html" par exemple)
+  char charset[64];             // charset ("iso-8859-1" par exemple)
+  char contentencoding[64];     // content-encoding ("gzip" par exemple)
   char *location;               // on copie dedans éventuellement la véritable 'location'
   LLint totalsize;              // taille totale à télécharger (-1=inconnue)
   short int is_file;            // ce n'est pas une socket mais un descripteur de fichier si 1

src/htsparse.c

@@ -610,22 +610,20 @@ int htsparse(htsmoduleStruct * str, htsmoduleStructExtended * stre) {
                           b = strchr(a, '<');   // prochain tag
                         }
                       }
-                      if (lienrelatif(tempo, sizeof(tempo), heap(ptr)->sav,
-                                      concat(OPT_GET_BUFF(opt),
-                                             OPT_GET_BUFF_SIZE(opt),
-                                             StringBuff(opt->path_html_utf8),
-                                             "index.html")) == 0) {
+                      if (lienrelatif
+                          (tempo, heap(ptr)->sav,
+                           concat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                                  StringBuff(opt->path_html_utf8),
+                                  "index.html")) == 0) {
                         detect_title = 1;       // ok détecté pour cette page!
                         makeindex_links++;      // un de plus
-                        strlcpybuff(makeindex_firstlink, tempo,
-                                    HTS_URLMAXSIZE * 2);
+                        strcpybuff(makeindex_firstlink, tempo);
                         //
 
                         /* Hack */
                         if (opt->mimehtml) {
-                          strlcpybuff(makeindex_firstlink,
-                                      "cid:primary/primary",
-                                      HTS_URLMAXSIZE * 2);
+                          strcpybuff(makeindex_firstlink,
+                                     "cid:primary/primary");
                         }
 
                         if ((b == a) || (a == NULL) || (b == NULL)) {   // pas de titre
@@ -1651,9 +1649,8 @@ int htsparse(htsmoduleStruct * str, htsmoduleStructExtended * stre) {
                                 }
                                 // Prendre si extension reconnue
                                 if (!url_ok) {
-                                  if (get_httptype_sized(opt, type,
-                                                         sizeof(type), tempo,
-                                                         0)) // recognized type
+                                  get_httptype(opt, type, tempo, 0);
+                                  if (strnotempty(type))        // type reconnu!
                                     url_ok = 1;
                                   else if (is_dyntype(get_ext(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), tempo)))       // reconnu php,cgi,asp..
                                     url_ok = 1;
@@ -2321,12 +2318,12 @@ int htsparse(htsmoduleStruct * str, htsmoduleStructExtended * stre) {
                       switch (p_type) {
                       case 2:{
                           //if (*lien!='/') strcatbuff(base,"/");
-                          strlcpybuff(base, lien, HTS_URLMAXSIZE * 2);
+                          strcpybuff(base, lien);
                         }
                         break;  // base
                       case -2:{
                           //if (*lien!='/') strcatbuff(codebase,"/");
-                          strlcpybuff(codebase, lien, HTS_URLMAXSIZE * 2);
+                          strcpybuff(codebase, lien);
                         }
                         break;  // base
                       }
@@ -2722,8 +2719,7 @@ int htsparse(htsmoduleStruct * str, htsmoduleStructExtended * stre) {
 
                               strcpybuff(save, StringBuff(opt->path_html_utf8));
                               strcatbuff(save, cat_name);
-                              if (lienrelatif(tempo, sizeof(tempo), save,
-                                              relativesavename()) == 0) {
+                              if (lienrelatif(tempo, save, relativesavename()) == 0) {
                                 /* Never escape high-chars (we don't know the encoding!!) */
                                 inplace_escape_uri_utf(tempo, sizeof(tempo));  // escape with %xx
                                 //if (!no_esc_utf)
@@ -2953,8 +2949,7 @@ int htsparse(htsmoduleStruct * str, htsmoduleStructExtended * stre) {
                       tempo[0] = '\0';
                       // calculer le lien relatif
 
-                      if (lienrelatif(tempo, sizeof(tempo), afs.save,
-                                      relativesavename()) == 0) {
+                      if (lienrelatif(tempo, afs.save, relativesavename()) == 0) {
                         if (!in_media) {        // In media (such as real audio): don't patch
                           /* Never escape high-chars (we don't know the encoding!!) */
                           inplace_escape_uri_utf(tempo, sizeof(tempo));
@@ -3512,9 +3507,9 @@ int hts_mirror_check_moved(htsmoduleStruct * str,
             char BIGSTK pn_adr[HTS_URLMAXSIZE * 2], pn_fil[HTS_URLMAXSIZE * 2];
 
             n_adr[0] = n_fil[0] = '\0';
-            (void) adr_normalized_sized(moved->adr, n_adr, sizeof(n_adr));
+            (void) adr_normalized(moved->adr, n_adr);
             (void) fil_normalized(moved->fil, n_fil);
-            (void) adr_normalized_sized(urladr(), pn_adr, sizeof(pn_adr));
+            (void) adr_normalized(urladr(), pn_adr);
             (void) fil_normalized(urlfil(), pn_fil);
             if (strcasecmp(n_adr, pn_adr) == 0
                 && strcasecmp(n_fil, pn_fil) == 0) {
@@ -4399,7 +4394,7 @@ int hts_mirror_wait_for_next_file(htsmoduleStruct * str,
       memcpy(r, &(back[b].r), sizeof(htsblk));
       r->location = stre->loc_; // ne PAS copier location!! adresse, pas de buffer
       if (back[b].r.location)
-        strlcpybuff(r->location, back[b].r.location, HTS_URLMAXSIZE * 2);
+        strcpybuff(r->location, back[b].r.location);
       back[b].r.adr = NULL;     // ne pas faire de desalloc ensuite
 
       // libérer emplacement backing

src/htssafe.h

@@ -237,15 +237,6 @@ static char *strncatbuff_ptr_(char *dest, const char *src, size_t n) {
   HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), (size_t) -1, \
   "overflow while appending '" #B "' to '"#A"'", __FILE__, __LINE__)
 
-/**
- * Append at most "N" characters of "B" to "A", "A" having a maximum capacity
- * of "S".
- */
-#define strlncatbuff(A, B, S, N)                                               \
-  strncat_safe_(A, S, B, HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B),  \
-                N, "overflow while appending '" #B "' to '" #A "'", __FILE__,  \
-                __LINE__)
-
 /**
  * Copy characters of "B" to "A", "A" having a maximum capacity of "S".
  */

src/htstools.c

@@ -274,9 +274,7 @@ int ident_url_relatif(const char *lien, const char *origin_adr,
       char *const idna = hts_convertStringUTF8ToIDNA(a, strlen(a));
       if (idna != NULL) {
         if (strlen(idna) < HTS_URLMAXSIZE) {
-          /* a points within adrfil->adr; bound by the remaining capacity */
-          strlcpybuff(a, idna,
-                      sizeof(adrfil->adr) - (size_t) (a - adrfil->adr));
+          strcpybuff(a, idna);
         }
         free(idna);
       }
@@ -288,7 +286,7 @@ int ident_url_relatif(const char *lien, const char *origin_adr,
 
 // créer dans s, à partir du chemin courant curr_fil, le lien vers link (absolu)
 // un ident_url_relatif a déja été fait avant, pour que link ne soit pas un chemin relatif
-int lienrelatif(char *s, size_t ssize, const char *link, const char *curr_fil) {
+int lienrelatif(char *s, const char *link, const char *curr_fil) {
   char BIGSTK _curr[HTS_URLMAXSIZE * 2];
   char BIGSTK newcurr_fil[HTS_URLMAXSIZE * 2], newlink[HTS_URLMAXSIZE * 2];
   char *curr;
@@ -316,9 +314,9 @@ int lienrelatif(char *s, size_t ssize, const char *link, const char *curr_fil) {
     }
   }
 
-  // copy only the current path
+  // recopier uniquement le chemin courant
   curr = _curr;
-  strlcpybuff(curr, curr_fil, sizeof(_curr));
+  strcpybuff(curr, curr_fil);
   if ((a = strchr(curr, '?')) == NULL)  // couper au ? (params)
     a = curr + strlen(curr) - 1;        // pas de params: aller à la fin
   while((*a != '/') && (a > curr))
@@ -361,14 +359,14 @@ int lienrelatif(char *s, size_t ssize, const char *link, const char *curr_fil) {
     a++;
   while(*a)
     if (*(a++) == '/')
-      strlcatbuff(s, "../", ssize);
+      strcatbuff(s, "../");
   //if (strlen(s)==0) strcatbuff(s,"/");
 
   if (slash)
-    strlcatbuff(s, "/", ssize); // keep it absolute!
+    strcatbuff(s, "/");         // garder absolu!!
 
-  // we are in the starting directory, copy
-  strlcatbuff(s, link + ((*link == '/') ? 1 : 0), ssize);
+  // on est dans le répertoire de départ, copier
+  strcatbuff(s, link + ((*link == '/') ? 1 : 0));
 
   /* Security check */
   if (strlen(s) >= HTS_URLMAXSIZE)
@@ -412,7 +410,7 @@ int link_has_authorization(const char *lien) {
 }
 
 // conversion chemin de fichier/dossier vers 8-3 ou ISO9660
-void long_to_83(int mode, char *n83, size_t n83size, char *save) {
+void long_to_83(int mode, char *n83, char *save) {
   n83[0] = '\0';
 
   while(*save) {
@@ -427,19 +425,19 @@ void long_to_83(int mode, char *n83, size_t n83size, char *save) {
     }
     fnl[j] = '\0';
     // conversion
-    longfile_to_83(mode, fn83, sizeof(fn83), fnl);
-    strlcatbuff(n83, fn83, n83size);
+    longfile_to_83(mode, fn83, fnl);
+    strcatbuff(n83, fn83);
 
     save += i;
     if (*save == '/') {
-      strlcatbuff(n83, "/", n83size);
+      strcatbuff(n83, "/");
       save++;
     }
   }
 }
 
 // conversion nom de fichier/dossier isolé vers 8-3 ou ISO9660
-void longfile_to_83(int mode, char *n83, size_t n83size, char *save) {
+void longfile_to_83(int mode, char *n83, char *save) {
   int j = 0, max = 0;
   int i = 0;
   char nom[256];
@@ -528,10 +526,10 @@ void longfile_to_83(int mode, char *n83, size_t n83size, char *save) {
   }
   // corriger vers 8-3
   n83[0] = '\0';
-  strlncatbuff(n83, nom, n83size, max);
+  strncatbuff(n83, nom, max);
   if (strnotempty(ext)) {
-    strlcatbuff(n83, ".", n83size);
-    strlncatbuff(n83, ext, n83size, 3);
+    strcatbuff(n83, ".");
+    strncatbuff(n83, ext, 3);
   }
 }
 

src/htstools.h

@@ -61,11 +61,11 @@ typedef struct lien_adrfilsave lien_adrfilsave;
 int ident_url_relatif(const char *lien, const char *origin_adr,
                       const char *origin_fil,
                       lien_adrfil* const adrfil);
-int lienrelatif(char *s, size_t ssize, const char *link, const char *curr);
+int lienrelatif(char *s, const char *link, const char *curr);
 int link_has_authority(const char *lien);
 int link_has_authorization(const char *lien);
-void long_to_83(int mode, char *n83, size_t n83size, char *save);
-void longfile_to_83(int mode, char *n83, size_t n83size, char *save);
+void long_to_83(int mode, char *n83, char *save);
+void longfile_to_83(int mode, char *n83, char *save);
 HTS_INLINE int __rech_tageq(const char *adr, const char *s);
 HTS_INLINE int __rech_tageqbegdigits(const char *adr, const char *s);
 HTS_INLINE int rech_tageq_all(const char *adr, const char *s);

src/htswizard.c

@@ -223,9 +223,8 @@ static int hts_acceptlink_(httrackp * opt, int ptr,
       // note (up/down): on calcule à partir du lien primaire, ET du lien précédent.
       // ex: si on descend 2 fois on peut remonter 1 fois
 
-      if (lienrelatif(tempo, sizeof(tempo), fil,
-                      heap(heap(ptr)->premier)->fil) == 0) {
-        if (lienrelatif(tempo2, sizeof(tempo2), fil, heap(ptr)->fil) == 0) {
+      if (lienrelatif(tempo, fil, heap(heap(ptr)->premier)->fil) == 0) {
+        if (lienrelatif(tempo2, fil, heap(ptr)->fil) == 0) {
           hts_log_print(opt, LOG_DEBUG,
                         "build relative links to test: %s %s (with %s and %s)",
                         tempo, tempo2, heap(heap(ptr)->premier)->fil,
@@ -327,9 +326,8 @@ static int hts_acceptlink_(httrackp * opt, int ptr,
       char BIGSTK tempo[HTS_URLMAXSIZE * 2];
       char BIGSTK tempo2[HTS_URLMAXSIZE * 2];
 
-      if (lienrelatif(tempo, sizeof(tempo), fil,
-                      heap(heap(ptr)->premier)->fil) == 0) {
-        if (lienrelatif(tempo2, sizeof(tempo2), fil, heap(ptr)->fil) == 0) {
+      if (lienrelatif(tempo, fil, heap(heap(ptr)->premier)->fil) == 0) {
+        if (lienrelatif(tempo2, fil, heap(ptr)->fil) == 0) {
         } else {
           hts_log_print(opt, LOG_ERROR,
                         "Error building relative link %s and %s", fil,
@@ -338,6 +336,7 @@ static int hts_acceptlink_(httrackp * opt, int ptr,
       } else {
         hts_log_print(opt, LOG_ERROR, "Error building relative link %s and %s",
                       fil, heap(heap(ptr)->premier)->fil);
+
       }
     }                           // fin tester interdiction de monter
 

src/httrack-library.h

@@ -207,9 +207,6 @@ HTSEXT_API const char *jump_normalized_const(const char *);
 HTSEXT_API char *jump_toport(char *);
 HTSEXT_API const char *jump_toport_const(const char *);
 HTSEXT_API char *fil_normalized(const char *source, char *dest);
-HTSEXT_API char *adr_normalized_sized(const char *source, char *dest,
-                                      size_t destsize);
-HTS_DEPRECATED("use adr_normalized_sized(source, dest, destsize)")
 HTSEXT_API char *adr_normalized(const char *source, char *dest);
 HTSEXT_API const char *hts_rootdir(char *file);
 
@@ -247,9 +244,6 @@ HTSEXT_API char *unescape_http_unharm(char *const catbuff, const size_t size, co
 HTSEXT_API char *antislash_unescaped(char *catbuff, const char *s);
 
 HTSEXT_API void escape_remove_control(char *s);
-HTSEXT_API int get_httptype_sized(httrackp *opt, char *s, size_t ssize,
-                                  const char *fil, int flag);
-HTS_DEPRECATED("use get_httptype_sized(opt, s, ssize, fil, flag)")
 HTSEXT_API void get_httptype(httrackp * opt, char *s, const char *fil,
                              int flag);
 HTSEXT_API int is_knowntype(httrackp * opt, const char *fil);
@@ -257,9 +251,6 @@ HTSEXT_API int is_userknowntype(httrackp * opt, const char *fil);
 HTSEXT_API int is_dyntype(const char *fil);
 HTSEXT_API const char *get_ext(char *catbuff, size_t size, const char *fil);
 HTSEXT_API int may_unknown(httrackp * opt, const char *st);
-HTSEXT_API int guess_httptype_sized(httrackp *opt, char *s, size_t ssize,
-                                    const char *fil);
-HTS_DEPRECATED("use guess_httptype_sized(opt, s, ssize, fil)")
 HTSEXT_API void guess_httptype(httrackp * opt, char *s, const char *fil);
 
 /* Ugly string tools */

tests/01_engine-cmdline.test

@@ -30,17 +30,6 @@ run() {
     RC=$?
 }
 
-# crawl using exactly the given args as the only URL(s), no implicit primary URL;
-# leaves the exit status in RC
-run_only() {
-    local out="$1"
-    shift
-    rm -rf "$out"
-    mkdir -p "$out"
-    httrack -O "$out" --quiet -n "$@" >"$out/.log" 2>&1
-    RC=$?
-}
-
 # assert the value was accepted: clean exit and the fixture was mirrored
 accepted() {
     { test "$RC" -eq 0 && test -n "$(find "$1" -type f -path '*/index.html' -print -quit)"; } ||
@@ -79,15 +68,4 @@ refused "#152: over-cap -F not refused cleanly"
 run "$tmp/ov-l" --user-agent "$over"
 refused "#152: over-cap --user-agent not refused cleanly"
 
-# Quote handling on the sole URL (run_only, so the quoted arg is the only URL and
-# can't be masked by an implicit one). A fully "-quoted URL has its surrounding
-# quotes stripped in place and is mirrored; a dangling opening quote, and a lone
-# quote (empty after the opening "), are refused cleanly and never crash.
-run_only "$tmp/q-ok" "\"file://$tmp/index.html\""
-accepted "$tmp/q-ok" "quoted URL not stripped/mirrored"
-run_only "$tmp/q-bad" '"foo'
-refused "dangling-quote argument not refused cleanly"
-run_only "$tmp/q-lone" '"'
-refused "lone-quote argument not refused cleanly"
-
 exit 0

tests/01_engine-rcfile.test

@@ -1,91 +0,0 @@
-#!/bin/bash
-#
-
-# Config-file alias loading (no network). A .httrackrc in the working directory
-# is read by optinclude_file(), whose cmdl_ins macro inserts each alias-expanded
-# token into the x_argvblk block. That macro used to copy with an unbounded
-# strcpy on a bare char*; it is now bounded (strlcpybuff + cmdl_room over the
-# block capacity). Two properties are checked:
-#   1. The bound does not truncate: a long user-agent alias reaches doit.log
-#      intact. user-agent expands to two tokens (-F <value>), so it exercises
-#      both cmdl_ins insertions.
-#   2. The bound holds under exhaustion: a pathological .httrackrc whose alias
-#      expansions overflow the block aborts cleanly through the htssafe bounds
-#      check (a message naming htsalias.c) instead of overrunning the heap. The
-#      unbounded version segfaulted here.
-
-# set -e with the intentional-nonzero httrack runs guarded explicitly (the
-# crawls below are expected to fail/abort and their status is inspected by hand).
-set -euo pipefail
-
-# Resolve httrack to an absolute path before we cd: PATH may hold a build-relative
-# entry that would not resolve from the temp directory.
-bin=$(command -v httrack) || {
-    echo "FAIL: httrack not found on PATH"
-    exit 1
-}
-case "$bin" in
-/*) ;;
-*) bin="$(cd "$(dirname "$bin")" && pwd)/$(basename "$bin")" ;;
-esac
-
-tmp=$(mktemp -d "${TMPDIR:-/tmp}/httrack_rcfile.XXXXXX") || exit 1
-trap 'rm -rf "$tmp"' EXIT HUP INT QUIT PIPE TERM
-
-# --- 1. alias token survives the bound intact -------------------------------
-d1="$tmp/intact"
-mkdir -p "$d1"
-echo '<html><body>hello</body></html>' >"$d1/index.html"
-
-# optinclude_file() lowercases each config line, so the marker is lowercase to
-# survive the comparison verbatim.
-marker='zzz_rcfile_marker_0123456789_abcdefghijklmnopqrstuvwxyz_intact'
-printf 'user-agent=%s\n' "$marker" >"$d1/.httrackrc"
-
-# Run with no -O so the working-directory .httrackrc is loaded (an -O path makes
-# the engine skip the rc files). Output lands in the temp dir. Guard the run so a
-# nonzero exit is captured for the assertion instead of tripping set -e.
-rc=0
-(cd "$d1" && "$bin" "file://$d1/index.html" --quiet -n >.log 2>&1) || rc=$?
-
-test "$rc" -eq 0 || {
-    echo "FAIL: rc-file crawl exited $rc"
-    exit 1
-}
-test -f "$d1/hts-cache/doit.log" || {
-    echo "FAIL: doit.log not written (rc file not processed)"
-    exit 1
-}
-# A truncated copy would cut the token; require the full -F value.
-grep -q -- "-F $marker" "$d1/hts-cache/doit.log" || {
-    echo "FAIL: user-agent alias missing or truncated in doit.log"
-    head -1 "$d1/hts-cache/doit.log"
-    exit 1
-}
-
-# --- 2. block exhaustion aborts through the bound, not the heap -------------
-d2="$tmp/exhaust"
-mkdir -p "$d2"
-echo '<html><body>hi</body></html>' >"$d2/index.html"
-
-# Each line inserts ~two tokens of ~200 bytes; 400 lines overflow the block's
-# fixed slack (current_size + 32768) many times over, deterministically.
-val=$(printf 'a%.0s' $(seq 1 200))
-for _ in $(seq 1 400); do
-    printf 'user-agent=%s\n' "$val"
-done >"$d2/.httrackrc"
-
-# The process aborts (httrack turns the fatal signal into exit 134 either way),
-# so the exit code does not distinguish the bounded abort from a heap overflow;
-# the stderr diagnostic does. The htssafe bounds check names the offending file.
-# Expected to fail, so the nonzero exit is swallowed; only the log is inspected.
-(cd "$d2" && "$bin" "file://$d2/index.html" --quiet -n >.log 2>&1) || true
-
-grep -Eq "overflow while copying.*htsalias\.c" "$d2/.log" || {
-    echo "FAIL: exhausted rc file did not abort through the htsalias.c bound"
-    echo "(an unbounded copy would overrun the heap here)"
-    tail -3 "$d2/.log"
-    exit 1
-}
-
-exit 0

tests/13_local-cookies.test

@@ -0,0 +1,15 @@
+#!/bin/bash
+#
+# Cookie chain against the local test server (replaces the old online
+# ut/cookies/*.php fixtures). entrance.php sets cat/cake; second.php checks
+# them and sets badger; third.php checks all three. A missing or wrong cookie
+# returns 500, which would surface as an httrack error and a missing file, so a
+# clean 3-files/0-errors run proves the cookie jar is replayed across links.
+
+: "${top_srcdir:=..}"
+
+bash "$top_srcdir/tests/local-crawl.sh" --errors 0 --files 3 \
+    --found 'cookies/entrance.html' \
+    --found 'cookies/second.html' \
+    --found 'cookies/third.html' \
+    httrack 'BASEURL/cookies/entrance.php'

tests/14_local-https.test

@@ -0,0 +1,18 @@
+#!/bin/bash
+#
+# HTTPS crawl against the local test server, using the shipped self-signed
+# cert. httrack does not verify certs (htslib.c: SSL_CTX_new with no
+# SSL_CTX_set_verify), so the self-signed cert is accepted as-is and this
+# exercises the real TLS path offline. basic.html links to link.html with four
+# distinct query strings, each saved under a hashed name -> 5 files.
+
+: "${top_srcdir:=..}"
+
+if test "$HTTPS_SUPPORT" == "no"; then
+    echo "no https support compiled, skipping"
+    exit 77
+fi
+
+bash "$top_srcdir/tests/local-crawl.sh" --tls --errors 0 --files 5 \
+    --found 'simple/basic.html' \
+    httrack 'BASEURL/simple/basic.html'

tests/Makefile.am

@@ -1,4 +1,7 @@
-EXTRA_DIST = $(TESTS) crawl-test.sh run-all-tests.sh check-network.sh
+# Note: EXTRA_DIST globs are NOT expanded by automake; list fixtures explicitly.
+EXTRA_DIST = $(TESTS) crawl-test.sh run-all-tests.sh check-network.sh \
+	local-crawl.sh local-server.py server.crt server.key \
+	server-root/simple/basic.html server-root/simple/link.html
 
 TESTS_ENVIRONMENT =
 TESTS_ENVIRONMENT += PATH=$(top_builddir)/src$(PATH_SEPARATOR)$$PATH
@@ -25,7 +28,6 @@ TESTS = \
 	01_engine-idna.test \
 	01_engine-mime.test \
 	01_engine-parse.test \
-	01_engine-rcfile.test \
 	01_engine-simplify.test \
 	01_engine-strsafe.test \
 	02_manpage-regen.test \
@@ -36,6 +38,8 @@ TESTS = \
 	11_crawl-international.test \
 	11_crawl-longurl.test \
 	11_crawl-parsing.test \
-	12_crawl_https.test
+	12_crawl_https.test \
+	13_local-cookies.test \
+	14_local-https.test
 
 CLEANFILES = check-network_sh.cache

tests/Makefile.in

@@ -476,7 +476,12 @@ target_alias = @target_alias@
 top_build_prefix = @top_build_prefix@
 top_builddir = @top_builddir@
 top_srcdir = @top_srcdir@
-EXTRA_DIST = $(TESTS) crawl-test.sh run-all-tests.sh check-network.sh
+
+# Note: EXTRA_DIST globs are NOT expanded by automake; list fixtures explicitly.
+EXTRA_DIST = $(TESTS) crawl-test.sh run-all-tests.sh check-network.sh \
+	local-crawl.sh local-server.py server.crt server.key \
+	server-root/simple/basic.html server-root/simple/link.html
+
 # note: libtool should handle that
 ### TESTS_ENVIRONMENT += $(SHLIBPATH_VAR)="$(top_builddir)/src/$(LT_CV_OBJDIR)$${$(SHLIBPATH_VAR):+$(PATH_SEPARATOR)}$$$(SHLIBPATH_VAR)"
 TESTS_ENVIRONMENT = PATH=$(top_builddir)/src$(PATH_SEPARATOR)$$PATH \
@@ -499,7 +504,6 @@ TESTS = \
 	01_engine-idna.test \
 	01_engine-mime.test \
 	01_engine-parse.test \
-	01_engine-rcfile.test \
 	01_engine-simplify.test \
 	01_engine-strsafe.test \
 	02_manpage-regen.test \
@@ -510,7 +514,9 @@ TESTS = \
 	11_crawl-international.test \
 	11_crawl-longurl.test \
 	11_crawl-parsing.test \
-	12_crawl_https.test
+	12_crawl_https.test \
+	13_local-cookies.test \
+	14_local-https.test
 
 CLEANFILES = check-network_sh.cache
 all: all-am

tests/local-crawl.sh

@@ -0,0 +1,235 @@
+#!/bin/bash
+#
+# Launcher for httrack crawl tests against the local Python test server.
+#
+# Starts tests/local-server.py on an ephemeral port, discovers the port from
+# the server's stdout, then runs httrack against http(s)://127.0.0.1:$PORT and
+# audits the mirror. The server is always killed and the tmpdir removed on exit.
+#
+# The token BASEURL in any httrack argument is replaced with the discovered
+# http(s)://127.0.0.1:$PORT base. --found/--directory paths are relative to the
+# discovered host root (127.0.0.1_<port>/), since the random port leaks into
+# the mirror directory name.
+#
+# Usage:
+#   bash local-crawl.sh [--tls] [--root DIR] \
+#       --errors N --files N --found PATH ... --directory PATH ... \
+#       httrack BASEURL/some/path [httrack-args...]
+
+set -u
+
+testdir=$(cd "$(dirname "$0")" && pwd)
+server="${testdir}/local-server.py"
+root="${LOCAL_SERVER_ROOT:-${testdir}/server-root}"
+cert="${testdir}/server.crt"
+key="${testdir}/server.key"
+
+tls=
+verbose=
+tmpdir=
+serverpid=
+crawlpid=
+
+function warning {
+    echo "** $*" >&2
+    return 0
+}
+function die {
+    warning "$*"
+    exit 1
+}
+function debug {
+    test -n "$verbose" && echo "$*" >&2
+    return 0
+}
+function info { printf "[%s] ..\t" "$*" >&2; }
+function result { echo "$*" >&2; }
+
+function cleanup {
+    if test -n "$crawlpid"; then
+        kill -9 "$crawlpid" 2>/dev/null
+        crawlpid=
+    fi
+    if test -n "$serverpid"; then
+        kill "$serverpid" 2>/dev/null
+        # Reap it so the port is released before we rm the tmpdir/log.
+        wait "$serverpid" 2>/dev/null
+        serverpid=
+    fi
+    if test -n "$tmpdir" && test -d "$tmpdir"; then
+        test -n "$nopurge" || rm -rf "$tmpdir"
+    fi
+}
+
+function assert_equals {
+    info "$1"
+    if test ! "$2" == "$3"; then
+        result "expected '$2', got '$3'"
+        exit 1
+    fi
+    result "OK ($2)"
+}
+
+nopurge=
+trap cleanup EXIT HUP INT QUIT PIPE TERM
+
+# python3 is required; mirror check-network.sh's skip-with-77 convention.
+command -v python3 >/dev/null || ! echo "python3 not found; skipping local crawl tests" || exit 77
+
+tmptopdir=${TMPDIR:-/tmp}
+test -d "$tmptopdir" || mkdir -p "$tmptopdir" || die "no temporary directory; set TMPDIR"
+tmpdir=$(mktemp -d "${tmptopdir}/httrack_local.XXXXXX") || die "could not create tmpdir"
+
+# --- parse leading control flags --------------------------------------------
+declare -a audit=()
+scheme=http
+pos=0
+args=("$@")
+nargs=$#
+while test "$pos" -lt "$nargs"; do
+    case "${args[$pos]}" in
+    --debug) verbose=1 ;;
+    --no-purge)
+        nopurge=1
+        audit+=("--no-purge")
+        ;;
+    --tls)
+        tls=1
+        scheme=https
+        ;;
+    --root)
+        pos=$((pos + 1))
+        root="${args[$pos]}"
+        ;;
+    --errors | --files)
+        audit+=("${args[$pos]}" "${args[$((pos + 1))]}")
+        pos=$((pos + 1))
+        ;;
+    --found | --not-found | --directory)
+        audit+=("${args[$pos]}" "${args[$((pos + 1))]}")
+        pos=$((pos + 1))
+        ;;
+    httrack)
+        pos=$((pos + 1))
+        break
+        ;;
+    *) die "unrecognized option ${args[$pos]}" ;;
+    esac
+    pos=$((pos + 1))
+done
+
+# --- start the server --------------------------------------------------------
+test -r "$server" || die "cannot read $server"
+serverlog="${tmpdir}/server.log"
+serverargs=(--root "$root")
+if test -n "$tls"; then
+    serverargs+=(--tls --cert "$cert" --key "$key")
+fi
+debug "starting python3 $server ${serverargs[*]}"
+python3 "$server" "${serverargs[@]}" >"$serverlog" 2>&1 &
+serverpid=$!
+
+# Wait for the "PORT <n>" line (server prints it once bound).
+port=
+for _ in $(seq 1 50); do
+    if test -s "$serverlog"; then
+        line=$(head -n1 "$serverlog")
+        if test "${line%% *}" == "PORT"; then
+            port="${line#PORT }"
+            break
+        fi
+    fi
+    kill -0 "$serverpid" 2>/dev/null || die "server exited early: $(cat "$serverlog")"
+    sleep 0.1
+done
+test -n "$port" || die "could not discover server port: $(cat "$serverlog")"
+debug "server listening on ${scheme}://127.0.0.1:${port}"
+
+baseurl="${scheme}://127.0.0.1:${port}"
+
+# --- substitute BASEURL in the remaining (httrack) args ----------------------
+declare -a hts=()
+while test "$pos" -lt "$nargs"; do
+    hts+=("${args[$pos]//BASEURL/$baseurl}")
+    pos=$((pos + 1))
+done
+
+# --- run httrack -------------------------------------------------------------
+which httrack >/dev/null || die "could not find httrack"
+ver=$(httrack -O /dev/null --version | sed -e 's/HTTrack version //')
+test -n "$ver" || die "could not run httrack"
+
+out="${tmpdir}/crawl"
+mkdir "$out" || die "could not create $out"
+# Localhost is fast; disable the rate/bandwidth safety limits but keep a
+# max-time backstop so a hang cannot wedge the suite.
+declare -a moreargs=(--quiet --max-time=120 --timeout=30 --disable-security-limits --robots=0)
+log="${tmpdir}/log"
+info "running httrack ${hts[*]}"
+httrack -O "$out" --user-agent="httrack $ver local ($(uname -omrs))" "${moreargs[@]}" "${hts[@]}" >"$log" 2>&1 &
+crawlpid=$!
+wait "$crawlpid"
+crawlres=$?
+crawlpid=
+# httrack exits 0 even on hard connect/DNS errors, so this is a backstop only;
+# the real guard is the audit below (--errors 0 plus the host-root existence check).
+test "$crawlres" -eq 0 || ! result "httrack exited $crawlres" || {
+    cat "$log" >&2
+    exit 1
+}
+result "OK"
+grep -iE "^[0-9:]*[[:space:]]Error:" "${out}/hts-log.txt" >&2
+
+# --- discover the single host root (127.0.0.1_<port> or 127.0.0.1) -----------
+hostroot=
+for cand in "${out}/127.0.0.1_${port}" "${out}/127.0.0.1"; do
+    if test -d "$cand"; then
+        hostroot="$cand"
+        break
+    fi
+done
+test -n "$hostroot" || die "could not find host root under $out"
+debug "host root: $hostroot"
+
+# --- audit -------------------------------------------------------------------
+i=0
+while test "$i" -lt "${#audit[@]}"; do
+    case "${audit[$i]}" in
+    --errors)
+        i=$((i + 1))
+        assert_equals "checking errors" "${audit[$i]}" \
+            "$(grep -iEc "^[0-9:]*[[:space:]]Error:" "${out}/hts-log.txt")"
+        ;;
+    --files)
+        i=$((i + 1))
+        nFiles=$(grep -E "^HTTrack Website Copier/[^ ]* mirror complete in " "${out}/hts-log.txt" |
+            sed -e 's/.*[[:space:]]\([^ ]*\)[[:space:]]files written.*/\1/g')
+        assert_equals "checking files" "${audit[$i]}" "$nFiles"
+        ;;
+    --found)
+        i=$((i + 1))
+        info "checking for ${audit[$i]}"
+        if test -f "${hostroot}/${audit[$i]}"; then result "OK"; else
+            result "not found"
+            exit 1
+        fi
+        ;;
+    --not-found)
+        i=$((i + 1))
+        info "checking absence of ${audit[$i]}"
+        if test ! -f "${hostroot}/${audit[$i]}"; then result "OK"; else
+            result "present"
+            exit 1
+        fi
+        ;;
+    --directory)
+        i=$((i + 1))
+        info "checking for dir ${audit[$i]}"
+        if test -d "${hostroot}/${audit[$i]}"; then result "OK"; else
+            result "not found"
+            exit 1
+        fi
+        ;;
+    esac
+    i=$((i + 1))
+done

tests/local-server.py

@@ -0,0 +1,182 @@
+#!/usr/bin/env python3
+"""Self-contained local web server for httrack's crawl tests.
+
+Serves static fixtures from a docroot plus a handful of dynamic endpoints
+(cookies, ...) so httrack can be exercised over loopback, deterministically and
+offline, instead of crawling the live ut.httrack.com.
+
+Binds to an ephemeral port (port 0) and prints the chosen port to stdout as
+"PORT <n>\n" so a launcher can discover it. Pass --tls to wrap the socket with
+the shipped self-signed test cert; httrack does not verify certs, so no CA
+trust plumbing is needed.
+
+stdlib only (http.server + ssl) -- no new build or runtime dependency.
+"""
+
+import argparse
+import os
+from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
+from urllib.parse import quote, unquote, urlsplit
+
+# Cookie chain replicated from the old ut/cookies/*.php fixtures.
+COOKIE_PATH = "/cookies/"
+COOKIES = {
+    "cat": "dog",
+    "cake": "is a lie!",
+    "badger": "mushroom, with 'ants'",
+}
+
+PAGE = """<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+\t"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
+<head>
+\t<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+\t<title>Sample test</title>
+</head>
+<body>
+{body}
+</body>
+</html>
+"""
+
+
+class Handler(SimpleHTTPRequestHandler):
+    # Quieter logging; the launcher captures httrack's own log anyway.
+    def log_message(self, fmt, *args):
+        if os.environ.get("LOCAL_SERVER_VERBOSE"):
+            super().log_message(fmt, *args)
+
+    # --- helpers -----------------------------------------------------------
+
+    def request_cookies(self):
+        """Parse the Cookie header into {name: decoded-value}.
+
+        Mirrors PHP's $_COOKIE: values are url-decoded, matching the encoding
+        applied when the cookie was set (see set_cookie)."""
+        jar = {}
+        raw = self.headers.get("Cookie", "")
+        for pair in raw.split(";"):
+            pair = pair.strip()
+            if "=" in pair:
+                name, value = pair.split("=", 1)
+                jar[name.strip()] = unquote(value.strip())
+        return jar
+
+    def set_cookie(self, name, value):
+        """Queue a Set-Cookie header, url-encoding the value like PHP's
+        setcookie() so spaces/quotes/commas stay a single token that httrack
+        can store and replay verbatim."""
+        self._set_cookies.append(f"{name}={quote(value)}; Path={COOKIE_PATH}")
+
+    def send_html(self, body, status=200, extra_status=None):
+        encoded = PAGE.format(body=body).encode("utf-8")
+        self.send_response(status, extra_status)
+        self.send_header("Content-Type", "text/html; charset=utf-8")
+        self.send_header("Content-Length", str(len(encoded)))
+        for cookie in self._set_cookies:
+            self.send_header("Set-Cookie", cookie)
+        self.end_headers()
+        if self.command != "HEAD":
+            self.wfile.write(encoded)
+
+    def fail_cookie(self, what):
+        # The old PHPs answered 500 with the reason in the status line.
+        self.send_html("", status=500, extra_status=f"The {what} is missing or invalid")
+
+    # --- dynamic routes ----------------------------------------------------
+
+    def route_entrance(self):
+        self.set_cookie("cat", COOKIES["cat"])
+        self.set_cookie("cake", COOKIES["cake"])
+        self.send_html('\tThis is a <a href="second.php">link</a>')
+
+    def route_second(self):
+        jar = self.request_cookies()
+        if jar.get("cat") != COOKIES["cat"]:
+            return self.fail_cookie("cat")
+        if jar.get("cake") != COOKIES["cake"]:
+            return self.fail_cookie("cake")
+        self.set_cookie("badger", COOKIES["badger"])
+        self.send_html('\tThis is a <a href="third.php">link</a>')
+
+    def route_third(self):
+        jar = self.request_cookies()
+        if jar.get("cat") != COOKIES["cat"]:
+            return self.fail_cookie("cat")
+        if jar.get("cake") != COOKIES["cake"]:
+            return self.fail_cookie("cake")
+        if jar.get("badger") != COOKIES["badger"]:
+            return self.fail_cookie("badger")
+        self.send_html("\tThis is a test.")
+
+    def route_robots(self):
+        body = b"User-agent: *\nDisallow:\n"
+        self.send_response(200)
+        self.send_header("Content-Type", "text/plain")
+        self.send_header("Content-Length", str(len(body)))
+        self.end_headers()
+        if self.command != "HEAD":
+            self.wfile.write(body)
+
+    ROUTES = {
+        "/cookies/entrance.php": route_entrance,
+        "/cookies/second.php": route_second,
+        "/cookies/third.php": route_third,
+        "/robots.txt": route_robots,
+    }
+
+    # --- dispatch ----------------------------------------------------------
+
+    def dispatch(self):
+        self._set_cookies = []
+        path = urlsplit(self.path).path
+        handler = self.ROUTES.get(path)
+        if handler is not None:
+            handler(self)
+            return True
+        return False
+
+    def do_GET(self):
+        if not self.dispatch():
+            super().do_GET()
+
+    def do_HEAD(self):
+        if not self.dispatch():
+            super().do_HEAD()
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--root", required=True, help="docroot for static files")
+    parser.add_argument("--bind", default="127.0.0.1", help="bind address")
+    parser.add_argument("--tls", action="store_true", help="serve HTTPS")
+    parser.add_argument("--cert", help="TLS certificate (PEM)")
+    parser.add_argument("--key", help="TLS private key (PEM)")
+    args = parser.parse_args()
+
+    root = os.path.abspath(args.root)
+
+    def factory(*a, **kw):
+        return Handler(*a, directory=root, **kw)
+
+    httpd = ThreadingHTTPServer((args.bind, 0), factory)
+
+    if args.tls:
+        import ssl
+
+        ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
+        ctx.load_cert_chain(certfile=args.cert, keyfile=args.key)
+        httpd.socket = ctx.wrap_socket(httpd.socket, server_side=True)
+
+    port = httpd.socket.getsockname()[1]
+    # The launcher reads this line to discover the ephemeral port.
+    print(f"PORT {port}", flush=True)
+
+    try:
+        httpd.serve_forever()
+    except KeyboardInterrupt:
+        pass
+
+
+if __name__ == "__main__":
+    main()

tests/server-root/simple/basic.html

@@ -0,0 +1,18 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+
+<html xmlns="http://www.w3.org/1999/xhtml" lang="fr">
+
+<head>
+	<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+	<title>Sample test</title>
+</head>
+
+<body>
+
+	This is a <a href="link.html?v=1">link</a>
+	This is a <a href='link.html?v=2'>link</a>
+	This is a <a href="./link.html?v=3">link</a>
+	This is a <a href=link.html?v=4>link</a>
+
+</body>

tests/server-root/simple/link.html

@@ -0,0 +1,3 @@
+This is a link.
+
+Go back to <a href="basic.html">home</a>.

tests/server.crt

@@ -0,0 +1,21 @@
+-----BEGIN CERTIFICATE-----
+MIIDbzCCAlegAwIBAgIUdWkDDomnY3WW95UqJ+UOASuR/i0wDQYJKoZIhvcNAQEL
+BQAwODESMBAGA1UEAwwJMTI3LjAuMC4xMSIwIAYDVQQKDBlIVFRyYWNrIGxvY2Fs
+IHRlc3Qgc2VydmVyMCAXDTI2MDYxNTE0NDQxMFoYDzIwNTYwNjA3MTQ0NDEwWjA4
+MRIwEAYDVQQDDAkxMjcuMC4wLjExIjAgBgNVBAoMGUhUVHJhY2sgbG9jYWwgdGVz
+dCBzZXJ2ZXIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDx78mogNhT
+noWwRa51NeGtapQ1PfTYLlIMUzuloFXOsR1/ozRkFucqHNftF22wf0gg4VQJSBSf
+3rwj79vsnt3nyaD03bTAafpHXkd+IJxQowiG8TfOJF0R/Qg9g7DCE66R9agQpMJC
+SGxIin9p/4ld4Hn6869d4hNq4fHxNf/qkj2cnf8DYxrldz2FGsi6yMed4tzz2Am4
+ZbPgwep+fy843ZdYrVIms9vJluNa9E+6Vpw9FwdjzQ/IBBMLvGaC2pDkc95YelaE
+nQrAlTO/0l5vjc8XuTQFlo3DbUg+WEld/pxvCqsd/q1mqjL0WbxtXl2zCwGzAoJx
+rjVEPfA8QSbtAgMBAAGjbzBtMB0GA1UdDgQWBBTHE0KKW8REV4HxajzVsIBxz3iL
+9zAfBgNVHSMEGDAWgBTHE0KKW8REV4HxajzVsIBxz3iL9zAPBgNVHRMBAf8EBTAD
+AQH/MBoGA1UdEQQTMBGHBH8AAAGCCWxvY2FsaG9zdDANBgkqhkiG9w0BAQsFAAOC
+AQEAYlTEftrwGJBXuPmtxhmtw2HO/VTC4TGnq67hH5H+ptwgZJuuxCQ5KW6flTyp
+FTyMhha33WD4EBL3wqqJsWr9Y4BXqi4G0lRqXBcC1oIUa2VYIDMER7kaY1qTSqE8
+ARpwdB2BhvngAzDLc+4Jt4jQMRGr8fHAwxpDBoIZ1knbyzYNP73Bajse6/8YtxUu
+nB2BsldjZnLvyHvRxUpWp92OyQih4jYSrlN6olDFlKDg7++kMhkHtJQW9a1t54VN
+0ZXrB1ZRuHUUvGBq26x71riTWor7HNOSQaGeCMQjZNQkh5tfshNygUGSZVXTEwhG
+xSrOL7NqBt2+EkVwf7LjGzjmBw==
+-----END CERTIFICATE-----

tests/server.key

@@ -0,0 +1,28 @@
+-----BEGIN PRIVATE KEY-----
+MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDx78mogNhTnoWw
+Ra51NeGtapQ1PfTYLlIMUzuloFXOsR1/ozRkFucqHNftF22wf0gg4VQJSBSf3rwj
+79vsnt3nyaD03bTAafpHXkd+IJxQowiG8TfOJF0R/Qg9g7DCE66R9agQpMJCSGxI
+in9p/4ld4Hn6869d4hNq4fHxNf/qkj2cnf8DYxrldz2FGsi6yMed4tzz2Am4ZbPg
+wep+fy843ZdYrVIms9vJluNa9E+6Vpw9FwdjzQ/IBBMLvGaC2pDkc95YelaEnQrA
+lTO/0l5vjc8XuTQFlo3DbUg+WEld/pxvCqsd/q1mqjL0WbxtXl2zCwGzAoJxrjVE
+PfA8QSbtAgMBAAECggEACgNK4klq1T3IpKdNoBY5yoE7CbUQZBNkBpSPRxHgBezj
+SVFfgrZGnOySrIJSt4JHtuynG2Hl+0ku74HRep/ck+eOsh5W3mZvGvMLnGxhwR3u
+Or99osTIgU0VQTkpC0SLQ16FCnih0uJycNIikdLR7uuya1tt1OyIBzK7XlNGIywT
+p85zJc7/6TfTC9eM7lqh7JGR7KplBxSvgZL1pUr7y4rNpKms6uzOvPND79CcKnbU
+BBA9Tu4qdOkoOljsZKkvh3pihxyG9X6d8QTZ/uX3pkvliwSFBc+Sz9EootA3/4r5
+gVWpQ2t/AY7fY4hqzLIX/HivVaPj3cWk1G+SHm0XNQKBgQD5I9rijqFvV/p6FmUl
+FbnjJFFHHgZLivlGxAC5vOyJNQQaqdeDzg7yMotNmQTggVGjT6sjdosQb3n+ctuk
+EhQnZSU5VkNKv1+PTR35WrRkaECCaqz3Pv79pV9GVcX3it7UuYjNiOeSPqINWe+X
+49JwnJFz+qQ1BchAwOis4zkENwKBgQD4mShDaYLOO97VpgZj4cGxHHWyEK9CRQvp
+I7HxRmfaWS3JHwb88lOmALEU6pAj5cYJPAznv8BnUWcVHalZbkQ1JWYtUJRqj6OI
+Ym7rw/nm4Ay5ijbdEism173dSk3IjOe+PdAlxzsOuVzYdBTqElmeQWtBzhY9aHvX
+r+A02C2j+wKBgHHDo6Gsi57yR5gUPd9vSlCkNtEIrss0DJv5yHMIB+KnaNZcE+NF
+5qFF30Jxyz5RDtxJ9tXcvaeln8lG3XDQKI/MqfDCqTuqo5ImHrfMaW8oA70JxS2p
+gHqGVzkg1aMxsIrmpcdk6olnPExocvWivGdbtzeEjhMALu8Sp6y6nUCFAoGBAK5h
+KLgYw/OMVaQCIMthaa+l6f0s7PMMYe1453H6VBD6qz4/8HPwO7LfG1gzrUYxADgs
+ElVh0UHn/On383nS+i9Ze5Hfyyvwc+LQQURKJPrJQMPJavCptPE7NmiKnYNHK6vr
+yh0l4oxShAklbCJBGvICq4zuVfVfXDeQnDIVTfaPAoGBAMCrZqYdOUhUu+aUqxZq
+qO/TTQxrxftU63jGUg+o042TdgI4KWLn07wvHJ8/E2OqF35eXenvcuKbNLI1l72J
+4cp+3cUv8iAXThTRYEztr5CS/wta4o4CNN8zfjn5dV9AI4Hmt4V7EaGWpBcViGbj
+n0Mhag+dO8DHuenqi1yfMrAt
+-----END PRIVATE KEY-----