Use *[path]/ so the PDF recipe follows nested directory indexes

*[file] cannot cross a slash, so *[file]/ only admits a single directory level and
silently drops PDFs linked under nested paths (dated archives, doc trees). *[path]/
spans slashes and follows directory-index pages at any depth, with no over-admission
(only trailing-slash URLs match, never files).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

.github/workflows/codeql.yml

@@ -55,6 +55,9 @@ jobs:
             query-filters:
               - exclude:
                   id: cpp/world-writable-file-creation
+              # Models auth-bypass-by-spoofing; httrack has no auth surface, its +/- crawl filter is a mirror boundary, not a security one.
+              - exclude:
+                  id: cpp/user-controlled-bypass
 
       # Manual build: CodeQL traces the compiler, so build exactly what ships.
       - name: Build

html/cmdguide.html

@@ -234,7 +234,7 @@ below let you go faster when you own the target, and slower when you do not.</p>
   <tr><td><tt>--max-time (-E)</tt></td><td>Stop after N seconds of wall-clock time.</td></tr>
   <tr><td><tt>--max-files (-m)</tt></td><td>Per-file size caps.</td></tr>
   <tr><td><tt>--timeout (-T), --retries (-R), --min-rate (-J), --host-control (-H)</tt></td><td>Idle timeout, retry count, minimum acceptable rate, and host-ban behavior for slow or dead hosts.</td></tr>
-  <tr><td><tt>--max-pause (-G), -%G</tt></td><td>Pause the mirror at N bytes, or pause between files, to spread the load.</td></tr>
+  <tr><td><tt>--max-pause (-G), --pause (-%G)</tt></td><td>Pause the mirror at N bytes, or pause between files, to spread the load.</td></tr>
 </table>
 
 <p><b>The security clamps.</b> To keep an accidental typo from turning into a flood,
@@ -257,8 +257,8 @@ really HTML.</p>
   <tr><td><tt>--structure (-N)</tt></td><td>The local path and name layout. Presets are numeric, and you can also give a template such as <tt>--structure "%h%p/%n%q.%t"</tt>.</td></tr>
   <tr><td><tt>--long-names (-L)</tt></td><td>Long names, 8.3 names, or ISO9660 for CD masters.</td></tr>
   <tr><td><tt>--assume (-%A)</tt></td><td>Assume a MIME type for an extension, for example <tt>--assume php=text/html</tt>. This also skips the extra HEAD probe HTTrack would otherwise send to learn the type.</td></tr>
-  <tr><td><tt>-%N, --cached-delayed-type-check (-%D), --check-type (-u), -%t</tt></td><td>When and how the content type is checked, and whether the original extension is kept.</td></tr>
-  <tr><td><tt>--include-query-string (-%q), -%g</tt></td><td>Whether the query string appears in the local filename, and whether query keys are stripped when deciding if two URLs are the same file.</td></tr>
+  <tr><td><tt>--delayed-type-check (-%N), --cached-delayed-type-check (-%D), --check-type (-u), -%t</tt></td><td>When and how the content type is checked, and whether the original extension is kept.</td></tr>
+  <tr><td><tt>--include-query-string (-%q), --strip-query (-%g)</tt></td><td>Whether the query string appears in the local filename, and whether query keys are stripped when deciding if two URLs are the same file.</td></tr>
 </table>
 
 <p>The <tt>-N</tt> presets are built from modular arithmetic on the name fields, so
@@ -329,7 +329,7 @@ across links at the same time.</p>
   <tr><td><tt>--proxy (-P)</tt></td><td>Route through a proxy. HTTP, SOCKS5 and CONNECT are supported: <tt>-P host:8080</tt>, <tt>-P socks5://host:1080</tt>, <tt>-P connect://host:443</tt>, with optional <tt>user:pass@</tt>.</td></tr>
   <tr><td><tt>--httpproxy-ftp (-%f)</tt></td><td>Send FTP requests through the HTTP proxy.</td></tr>
   <tr><td><tt>--protocol (-@i)</tt></td><td>Prefer IPv4 or IPv6.</td></tr>
-  <tr><td><tt>--http-10 (-%h), --keep-alive (-%k), -%z</tt></td><td>Force HTTP/1.0 (drops keep-alive and compression, useful for fragile CGI), toggle keep-alive, and toggle compression.</td></tr>
+  <tr><td><tt>--http-10 (-%h), --keep-alive (-%k), --disable-compression (-%z)</tt></td><td>Force HTTP/1.0 (drops keep-alive and compression, useful for fragile CGI), toggle keep-alive, and toggle compression.</td></tr>
   <tr><td><tt>--bind (-%b), --tolerant (-%B)</tt></td><td>Bind to a local address, and accept technically-bogus responses some servers send.</td></tr>
 </table>
 
@@ -353,7 +353,7 @@ away and you lose the ability to continue or update the mirror.</p>
   <tr><td><tt>--update</tt></td><td>Re-run the mirror, revalidating each page with the server (If-Modified-Since / If-None-Match) and downloading only what changed.</td></tr>
   <tr><td><tt>--purge-old=0 (-X0)</tt></td><td>Do not purge. By default an update deletes local files that are no longer part of the mirror; <tt>--purge-old=0</tt> keeps them.</td></tr>
   <tr><td><tt>--cache (-C)</tt></td><td>Cache mode. The default already does the right thing and switches to update-checking when it detects an existing mirror.</td></tr>
-  <tr><td><tt>--urlhack (-%u), -%j, -%o, -%y, --do-not-recatch (-%n), --updatehack (-%s), --store-all-in-cache (-k)</tt></td><td>URL-deduplication behavior and cache storage details.</td></tr>
+  <tr><td><tt>--urlhack (-%u), --keep-www-prefix (-%j), --keep-double-slashes (-%o), --keep-query-order (-%y), --do-not-recatch (-%n), --updatehack (-%s), --store-all-in-cache (-k)</tt></td><td>URL-deduplication behavior and cache storage details.</td></tr>
   <tr><td><tt>--debug-cache (-#C), --repair-cache (-#R), --clean</tt></td><td>Inspect the cache, repair its ZIP, and erase cache plus logs.</td></tr>
 </table>
 
@@ -404,14 +404,16 @@ host and stops the crawl; start from the final URL, or add
 rule wins.</small></p>
 
 <h4>Download the PDFs on a site</h4>
-<p><tt>httrack https://example.com/ --path mydir</tt><br>
-<small>There is no PDF-only crawl. HTTrack discovers PDF links by parsing the
-site's HTML pages, so a <tt>"-*" "+example.com/*.pdf"</tt> filter blocks the very
-pages that carry the links and grabs only the PDFs linked from the front page. Let
-it crawl the site: the HTML pages come along as the scaffolding, and every reachable
-PDF is saved with them. If some PDFs live on another host (a CDN or a docs
-subdomain), allow that host too, for example
-<tt>"+docs.example.com/*.pdf"</tt>.</small></p>
+<p><tt>httrack https://example.com/ "-*" "+https://example.com/*.html" "+https://example.com/*[path]/" "+https://example.com/*.pdf" --path mydir</tt><br>
+<small>HTTrack finds PDFs by parsing the site's HTML, so a plain
+<tt>"-*" "+example.com/*.pdf"</tt> is wrong: it prunes the pages that carry the
+links and keeps only PDFs reachable from the front page. Instead admit the HTML as
+scaffolding (<tt>*.html</tt> and <tt>*[path]/</tt> for directory-index pages at any
+depth, e.g. <tt>docs/</tt> or <tt>a/b/deep/</tt>; <tt>*[file]/</tt> would stop at one
+level), keep the PDFs, and let <tt>-*</tt> drop everything else (images,
+archives, off-site assets). PDFs on another host (a CDN or docs subdomain) are not
+included by default; allow that host too, e.g. <tt>"+docs.example.com/*.pdf"</tt>,
+or widen to <tt>"+*.pdf"</tt> for PDFs anywhere.</small></p>
 
 <h4>Keep page requisites, including off-host images</h4>
 <p><tt>httrack https://example.com/blog/ --near --path mydir</tt><br>

man/httrack.1

@@ -3,7 +3,7 @@
 .\"
 .\" This file is generated by man/makeman.sh; do not edit by hand.
 .\" SPDX-License-Identifier: GPL-3.0-or-later
-.TH httrack 1 "21 July 2026" "httrack website copier"
+.TH httrack 1 "22 July 2026" "httrack website copier"
 .SH NAME
 httrack \- offline browser : copy websites to a local directory
 .SH SYNOPSIS
@@ -37,6 +37,7 @@ httrack \- offline browser : copy websites to a local directory
 [ \fB\-%L, \-\-list\fR ]
 [ \fB\-%S, \-\-urllist\fR ]
 [ \fB\-NN, \-\-structure[=N]\fR ]
+[ \fB\-%N, \-\-delayed\-type\-check\fR ]
 [ \fB\-%D, \-\-cached\-delayed\-type\-check\fR ]
 [ \fB\-%M, \-\-mime\-html\fR ]
 [ \fB\-LN, \-\-long\-names[=N]\fR ]
@@ -189,7 +190,7 @@ structure type (0 *original structure, 1+: see below) (\-\-structure[=N])
 .br
 or user defined structure (\-N "%h%p/%n%q.%t")
 .IP \-%N
-delayed type check, don't make any link test but wait for files download to start instead (experimental) (%N0 don't use, %N1 use for unknown extensions, * %N2 always use)
+delayed type check, don't make any link test but wait for files download to start instead (experimental) (%N0 don't use, %N1 use for unknown extensions, * %N2 always use) (\-\-delayed\-type\-check)
 .IP \-%D
 cached delayed type check, don't wait for remote type during updates, to speedup them (%D0 wait, * %D1 don't wait) (\-\-cached\-delayed\-type\-check)
 .IP \-%M

src/htscache.c

@@ -856,7 +856,41 @@ static htsblk cache_readex_new(httrackp * opt, cache_back * cache,
 // si save==null alors test unqiquement
 static int hts_rename(httrackp * opt, const char *a, const char *b) {
   hts_log_print(opt, LOG_DEBUG, "Cache: rename %s -> %s (%p %p)", a, b, a, b);
-  return rename(a, b);
+  return RENAME(a, b);
+}
+
+/* Open the cache ZIP via hts_fopen_utf8 so a non-ASCII path_log isn't mangled
+   to ANSI (#630); 64-bit funcs keep multi-GB caches whole on Windows LLP64. */
+static voidpf ZCALLBACK hts_zip_fopen_utf8(voidpf opaque, const void *filename,
+                                           int mode) {
+  const char *mode_fopen = NULL;
+
+  (void) opaque;
+  if ((mode & ZLIB_FILEFUNC_MODE_READWRITEFILTER) == ZLIB_FILEFUNC_MODE_READ)
+    mode_fopen = "rb";
+  else if (mode & ZLIB_FILEFUNC_MODE_EXISTING)
+    mode_fopen = "r+b";
+  else if (mode & ZLIB_FILEFUNC_MODE_CREATE)
+    mode_fopen = "wb";
+  if (filename == NULL || mode_fopen == NULL)
+    return NULL;
+  return (voidpf) FOPEN((const char *) filename, mode_fopen);
+}
+
+static unzFile hts_unzOpen_utf8(const char *path) {
+  zlib_filefunc64_def ff;
+
+  fill_fopen64_filefunc(&ff);
+  ff.zopen64_file = hts_zip_fopen_utf8;
+  return unzOpen2_64(path, &ff);
+}
+
+static zipFile hts_zipOpen_utf8(const char *path, int append) {
+  zlib_filefunc64_def ff;
+
+  fill_fopen64_filefunc(&ff);
+  ff.zopen64_file = hts_zip_fopen_utf8;
+  return zipOpen2_64(path, append, NULL, &ff);
 }
 
 /* Pathname of a file inside the mirror dir (rotating concat buffer). */
@@ -874,9 +908,9 @@ static char *reconcile_path(httrackp *opt, const char *name) {
 /* Replace the new-generation file by the old one, when the old one exists. */
 static void reconcile_promote(httrackp *opt, const char *oldname,
                               const char *newname) {
-  if (fexist(reconcile_path(opt, oldname))) {
-    remove(reconcile_path(opt, newname));
-    rename(reconcile_path(opt, oldname), reconcile_path(opt, newname));
+  if (fexist_utf8(reconcile_path(opt, oldname))) {
+    UNLINK(reconcile_path(opt, newname));
+    RENAME(reconcile_path(opt, oldname), reconcile_path(opt, newname));
   }
 }
 
@@ -885,7 +919,7 @@ void hts_cache_reconcile(httrackp *opt, hts_cache_reconcile_mode mode) {
   case CACHE_RECONCILE_PROMOTE:
     /* Previous run rotated new.* to old.* then died before writing: promote
        the old generation back, whichever format it uses. */
-    if (!fexist(reconcile_path(opt, "hts-cache/new.zip")))
+    if (!fexist_utf8(reconcile_path(opt, "hts-cache/new.zip")))
       reconcile_promote(opt, "hts-cache/old.zip", "hts-cache/new.zip");
     break;
   case CACHE_RECONCILE_INTERRUPTED:
@@ -894,16 +928,17 @@ void hts_cache_reconcile(httrackp *opt, hts_cache_reconcile_mode mode) {
        is -1 for a missing file, which would spuriously pass the "< TINY" test
        and overwrite a solid old generation that PROMOTE/ROLLBACK should keep.
      */
-    if (!opt->cache || !fexist(reconcile_path(opt, "hts-in_progress.lock")))
+    if (!opt->cache ||
+        !fexist_utf8(reconcile_path(opt, "hts-in_progress.lock")))
       break;
-    if (fexist(reconcile_path(opt, "hts-cache/new.zip")) &&
-        fexist(reconcile_path(opt, "hts-cache/old.zip")) &&
-        fsize(reconcile_path(opt, "hts-cache/new.zip")) <
+    if (fexist_utf8(reconcile_path(opt, "hts-cache/new.zip")) &&
+        fexist_utf8(reconcile_path(opt, "hts-cache/old.zip")) &&
+        fsize_utf8(reconcile_path(opt, "hts-cache/new.zip")) <
             CACHE_RECONCILE_NEW_TINY &&
-        fsize(reconcile_path(opt, "hts-cache/old.zip")) >
+        fsize_utf8(reconcile_path(opt, "hts-cache/old.zip")) >
             CACHE_RECONCILE_OLD_SOLID &&
-        fsize(reconcile_path(opt, "hts-cache/old.zip")) >
-            fsize(reconcile_path(opt, "hts-cache/new.zip")))
+        fsize_utf8(reconcile_path(opt, "hts-cache/old.zip")) >
+            fsize_utf8(reconcile_path(opt, "hts-cache/new.zip")))
       reconcile_promote(opt, "hts-cache/old.zip", "hts-cache/new.zip");
     break;
   case CACHE_RECONCILE_ROLLBACK:
@@ -940,24 +975,26 @@ void cache_init(cache_back * cache, httrackp * opt) {
 #endif
     if (!cache->ro) {
 #ifdef _WIN32
-      mkdir(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-cache"));
+      /* Windows mkdir takes no mode; use the UTF-8 wrapper for #630. */
+      MKDIR(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                    StringBuff(opt->path_log), "hts-cache"));
 #else
-      mkdir(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-cache"),
+      /* keep the cache dir 0700, not MKDIR's 0755. */
+      mkdir(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                    StringBuff(opt->path_log), "hts-cache"),
             HTS_PROTECT_FOLDER);
 #endif
-      if ((fexist(fconcat(
+      if ((fexist_utf8(fconcat(
               OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
               StringBuff(opt->path_log),
               "hts-cache/new.zip")))) { // a previous cache exists.. rename it
         /* Remove OLD cache */
-        if (fexist
-            (fconcat
-             (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-              "hts-cache/old.zip"))) {
-          if (remove
-              (fconcat
-               (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                "hts-cache/old.zip")) != 0) {
+        if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                                StringBuff(opt->path_log),
+                                "hts-cache/old.zip"))) {
+          if (UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                             StringBuff(opt->path_log), "hts-cache/old.zip")) !=
+              0) {
             hts_log_print(opt, LOG_WARNING | LOG_ERRNO,
                           "Cache: error while moving previous cache");
           }
@@ -980,33 +1017,27 @@ void cache_init(cache_back * cache, httrackp * opt) {
       hts_log_print(opt, LOG_DEBUG, "Cache: no cache found");
     }
     hts_log_print(opt, LOG_DEBUG, "Cache: size %d",
-                  (int)
-                  fsize(fconcat
-                        (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                         "hts-cache/old.zip")));
+                  (int) fsize_utf8(
+                      fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                              StringBuff(opt->path_log), "hts-cache/old.zip")));
 
     // charger index cache précédent
-    if ((!cache->ro
-         &&
-         fsize(fconcat
-               (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                "hts-cache/old.zip")) > 0)
-        || (cache->ro
-            &&
-            fsize(fconcat
-                  (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                   "hts-cache/new.zip")) > 0)
-      ) {
+    if ((!cache->ro &&
+         fsize_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                            StringBuff(opt->path_log), "hts-cache/old.zip")) >
+             0) ||
+        (cache->ro &&
+         fsize_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                            StringBuff(opt->path_log), "hts-cache/new.zip")) >
+             0)) {
       if (!cache->ro) {
-        cache->zipInput =
-          unzOpen(fconcat
-                  (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                   "hts-cache/old.zip"));
+        cache->zipInput = hts_unzOpen_utf8(
+            fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                    StringBuff(opt->path_log), "hts-cache/old.zip"));
       } else {
-        cache->zipInput =
-          unzOpen(fconcat
-                  (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                   "hts-cache/new.zip"));
+        cache->zipInput = hts_unzOpen_utf8(
+            fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                    StringBuff(opt->path_log), "hts-cache/new.zip"));
       }
 
       // Corrupted ZIP file ? Try to repair!
@@ -1026,6 +1057,8 @@ void cache_init(cache_back * cache, httrackp * opt) {
         }
         hts_log_print(opt, LOG_WARNING,
                       "Cache: damaged cache, trying to repair");
+        /* mztools has no UTF-8 hook, so repairing a corrupt cache under a
+           non-ASCII path_log fails cleanly (re-crawl), never forks a twin. */
         if (unzRepair
             (name,
              fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
@@ -1033,11 +1066,11 @@ void cache_init(cache_back * cache, httrackp * opt) {
                                                       StringBuff(opt->path_log),
                                                       "hts-cache/repair.tmp"),
              &repaired, &repairedBytes) == Z_OK) {
-          unlink(name);
-          rename(fconcat
-                 (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                  "hts-cache/repair.zip"), name);
-          cache->zipInput = unzOpen(name);
+          UNLINK(name);
+          RENAME(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                         StringBuff(opt->path_log), "hts-cache/repair.zip"),
+                 name);
+          cache->zipInput = hts_unzOpen_utf8(name);
           hts_log_print(opt, LOG_WARNING,
                         "Cache: %d bytes successfully recovered in %d entries",
                         (int) repairedBytes, (int) repaired);
@@ -1133,8 +1166,8 @@ void cache_init(cache_back * cache, httrackp * opt) {
                       "Cache: error trying to open the cache");
       }
 
-    } else if (fsize(reconcile_path(opt, "hts-cache/old.ndx")) > 0 ||
-               fsize(reconcile_path(opt, "hts-cache/new.ndx")) > 0) {
+    } else if (fsize_utf8(reconcile_path(opt, "hts-cache/old.ndx")) > 0 ||
+               fsize_utf8(reconcile_path(opt, "hts-cache/new.ndx")) > 0) {
       /* pre-3.31 (2003) .dat/.ndx cache: import support removed */
       hts_log_print(opt, LOG_ERROR,
                     "Cache: the pre-3.31 .dat/.ndx cache format is no longer "
@@ -1150,67 +1183,58 @@ void cache_init(cache_back * cache, httrackp * opt) {
 #endif
     if (!cache->ro) {
       // ouvrir caches actuels
-      structcheck(fconcat
-                  (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-cache/"));
+      structcheck_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                               StringBuff(opt->path_log), "hts-cache/"));
 
       {
         /* Create ZIP file cache */
-        cache->zipOutput =
-          (void *)
-          zipOpen(fconcat
-                  (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                   "hts-cache/new.zip"), 0);
+        cache->zipOutput = (void *) hts_zipOpen_utf8(
+            fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                    StringBuff(opt->path_log), "hts-cache/new.zip"),
+            0);
 
         if (cache->zipOutput != NULL) {
           // supprimer old.lst
-          if (fexist
-              (fconcat
-               (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                "hts-cache/old.lst")))
-            remove(fconcat
-                   (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                    "hts-cache/old.lst"));
+          if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                                  StringBuff(opt->path_log),
+                                  "hts-cache/old.lst")))
+            UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                           StringBuff(opt->path_log), "hts-cache/old.lst"));
           // renommer
-          if (fexist
-              (fconcat
-               (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                "hts-cache/new.lst")))
-            rename(fconcat
-                   (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                    "hts-cache/new.lst"), fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
-                                                  StringBuff(opt->path_log),
-                                                  "hts-cache/old.lst"));
+          if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                                  StringBuff(opt->path_log),
+                                  "hts-cache/new.lst")))
+            RENAME(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                           StringBuff(opt->path_log), "hts-cache/new.lst"),
+                   fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                           StringBuff(opt->path_log), "hts-cache/old.lst"));
           // ouvrir
           cache->lst =
-            fopen(fconcat
-                  (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                   "hts-cache/new.lst"), "wb");
+              FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                            StringBuff(opt->path_log), "hts-cache/new.lst"),
+                    "wb");
           strcpybuff(opt->state.strc.path, StringBuff(opt->path_html));
           opt->state.strc.lst = cache->lst;
 
           // supprimer old.txt
-          if (fexist
-              (fconcat
-               (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                "hts-cache/old.txt")))
-            remove(fconcat
-                   (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                    "hts-cache/old.txt"));
+          if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                                  StringBuff(opt->path_log),
+                                  "hts-cache/old.txt")))
+            UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                           StringBuff(opt->path_log), "hts-cache/old.txt"));
           // renommer
-          if (fexist
-              (fconcat
-               (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                "hts-cache/new.txt")))
-            rename(fconcat
-                   (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                    "hts-cache/new.txt"), fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
-                                                  StringBuff(opt->path_log),
-                                                  "hts-cache/old.txt"));
+          if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                                  StringBuff(opt->path_log),
+                                  "hts-cache/new.txt")))
+            RENAME(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                           StringBuff(opt->path_log), "hts-cache/new.txt"),
+                   fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                           StringBuff(opt->path_log), "hts-cache/old.txt"));
           // ouvrir
           cache->txt =
-            fopen(fconcat
-                  (OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
-                   "hts-cache/new.txt"), "wb");
+              FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
+                            StringBuff(opt->path_log), "hts-cache/new.txt"),
+                    "wb");
           if (cache->txt) {
             fprintf(cache->txt,
                     "date\tsize'/'remotesize\tflags(request:Update,Range state:File response:Modified,Chunked,gZipped)\t");

src/htshelp.c

@@ -77,17 +77,17 @@ void infomsg(const char *msg) {
           if (msg[2] != ' ') {
             if ((msg[3] == ' ') || (msg[4] == ' ')) {
               char cmd[32] = "-";
-              int p = 0;
+              int p;
 
-              while(cmd[p] == ' ')
-                p++;
-              sscanf(msg + p, "%s", cmd + strlen(cmd));
-              /* clears cN -> c */
-              if ((p = (int) strlen(cmd)) > 2)
-                if (cmd[p - 1] == 'N')
-                  cmd[p - 1] = '\0';
-              /* finds alias (if any) */
+              sscanf(msg, "%30s", cmd + strlen(cmd));
+              /* try the flag as-is, then strip a trailing N as the numeric-arg
+                 placeholder (cN -> c); this order keeps -%N from becoming -% */
               p = optreal_find(cmd);
+              if (p < 0 && (int) strlen(cmd) > 2 &&
+                  cmd[strlen(cmd) - 1] == 'N') {
+                cmd[strlen(cmd) - 1] = '\0';
+                p = optreal_find(cmd);
+              }
               if (p >= 0) {
                 /* fings type of parameter: number,param,param concatenated,single cmd */
                 if (strcmp(opttype_value(p), "param") == 0)
@@ -648,9 +648,8 @@ void help(const char *app, int more) {
   infomsg("Command-line specific options:");
   infomsg
     ("  V execute system command after each files ($0 is the filename: -V \"rm \\$0\")");
-  infomsg
-    (" %W use an external library function as a wrapper (-%W myfoo.so[,myparameters])");
-  /* infomsg(" %O do a chroot before setuid"); */
+  infomsg(" %W use an external library function as a wrapper (-%W "
+          "myfoo.so[,myparameters])");
   infomsg("");
   infomsg("Details: Option N");
   infomsg("  N0 Site-structure (default)");

src/htsparse.c

@@ -315,9 +315,18 @@ static void escape_url_parens(char *const s, const size_t size) {
   strlcpybuff(s, buff, size);
 }
 
-/* Strip a default ":80" from lien's authority in place. Any spelling that
-   range-parses to 80 (":80", ":080") is dropped by its matched length, not a
-   hardcoded 3 chars; a value that merely wraps to 80 as an int (#614) stays. */
+/* Default port for lien's scheme (case-insensitive), 80 if absent/unknown; so
+   schemeless and protocol-relative //host links default to 80 (known gap). */
+static int scheme_default_port(const char *lien) {
+  if (strfield(lien, "https:"))
+    return 443;
+  if (strfield(lien, "ftp:"))
+    return 21;
+  return 80;
+}
+
+/* Strip the scheme's own default port (80 http, 443 https, 21 ftp) from lien's
+   authority in place; :80 on https/ftp stays as a real port (#638, #614). */
 void hts_strip_default_port(char *lien, size_t size) {
   char *a;
 
@@ -339,7 +348,8 @@ void hts_strip_default_port(char *lien, size_t size) {
       b++;
     saved = *b;
     *b = '\0';
-    is_default = hts_parse_url_port(a + 1, &port) && port == 80;
+    is_default =
+        hts_parse_url_port(a + 1, &port) && port == scheme_default_port(lien);
     *b = saved;
     if (is_default) { // default port, strip it
       char BIGSTK tempo[HTS_URLMAXSIZE * 2];

src/htsselftest.c

@@ -1589,10 +1589,8 @@ static int st_identabs(httrackp *opt, int argc, char **argv) {
   return 0;
 }
 
-/* Default-port strip (#627): a genuine 80 (any spelling) is removed by its
-   matched length, host preserved; a non-80 port or one that only wraps to 80 as
-   a 32-bit int (#614) is left intact. Guards the old bug where ":080"/":0080"
-   dropped a hardcoded 3 chars and glued the leftover digits onto the host. */
+/* Default-port strip is scheme-aware (#638), overflow-safe (#614): a scheme's
+   own default (any spelling) is dropped, a real port stays; guards #627. */
 static int st_stripport(httrackp *opt, int argc, char **argv) {
   static const struct {
     const char *in, *out;
@@ -1606,6 +1604,13 @@ static int st_stripport(httrackp *opt, int argc, char **argv) {
       {"http://127.0.0.1:8080/x", "http://127.0.0.1:8080/x"},
       {"http://127.0.0.1:4294967376/x", "http://127.0.0.1:4294967376/x"},
       {"http://127.0.0.1/x", "http://127.0.0.1/x"},
+      {"https://127.0.0.1:443/x", "https://127.0.0.1/x"},
+      {"https://127.0.0.1:80/x", "https://127.0.0.1:80/x"},
+      // Scheme match is case-insensitive: HTTPS' default is 443, so :80 stays.
+      {"HTTPS://127.0.0.1:80/x", "HTTPS://127.0.0.1:80/x"},
+      {"ftp://127.0.0.1:21/x", "ftp://127.0.0.1/x"},
+      {"ftp://127.0.0.1:80/x", "ftp://127.0.0.1:80/x"},
+      {"http://127.0.0.1:443/x", "http://127.0.0.1:443/x"},
   };
 
   size_t k;

src/minizip/mztools.c.diff

@@ -1,6 +1,6 @@
 --- mztools.c.orig	2024-01-27 14:07:18.636193212 +0100
 +++ mztools.c	2024-01-27 14:09:55.356620093 +0100
-@@ -10,6 +10,7 @@
+@@ -10,10 +10,11 @@
  #include <string.h>
  #include "zlib.h"
  #include "unzip.h"
@@ -8,6 +8,11 @@
  
  #define READ_8(adr)  ((unsigned char)*(adr))
  #define READ_16(adr) ( READ_8(adr) | (READ_8(adr+1) << 8) )
+-#define READ_32(adr) ( READ_16(adr) | (READ_16((adr)+2) << 16) )
++#define READ_32(adr) ((uLong) READ_16(adr) | ((uLong) READ_16((adr) + 2) << 16))
+ 
+ #define WRITE_8(buff, n) do { \
+   *((unsigned char*)(buff)) = (unsigned char) ((n) & 0xff); \
 @@ -141,8 +142,8 @@
          /* Central directory entry */
          {

tests/66_engine-port80-strip.test

@@ -6,6 +6,7 @@ set -euo pipefail
 # Stripping a default :80 from a crawled link used to skip a hardcoded 3 chars
 # (#627): ":080"/":0080" lost only ":80" and glued the rest onto the host
 # (127.0.0.1:080/x -> 127.0.0.10/x), and a value wrapping to 80 as a 32-bit int
-# (#614) was stripped as if it were the default. All assertions live in the
-# engine self-test.
+# (#614) was stripped as if it were the default. The default is scheme-aware
+# (#638): an explicit :80 on https/ftp stays, :443/:21 strip under their own
+# scheme. All assertions live in the engine self-test.
 httrack -O /dev/null -#test=stripport | grep -q "stripport self-test OK"

tests/69_local-intl-logdir.test

@@ -1,15 +1,17 @@
 #!/bin/bash
 #
 # A single non-ASCII -O sets both path_html and path_log to "café" (#630). The
-# logs (hts-log.txt/hts-err.txt) must land there, not in an ANSI-mangled twin:
-# on Windows path_log holds UTF-8 bytes the raw file calls read as the codepage.
-# The --errors/--files/--log-found audits all grep logroot=café/hts-log.txt, so
-# a log written to the twin fails them. POSIX has no twin, so this bites on the
-# Windows CI leg (like test 64). The cache twin is a separate, larger fix.
+# logs (hts-log.txt/hts-err.txt) and the hts-cache must land there, not in an
+# ANSI-mangled twin: on Windows path_log holds UTF-8 bytes the raw file calls
+# read as the codepage. The --log-found audit greps logroot=café/hts-log.txt and
+# --cache-under-logroot checks café/hts-cache/new.zip, so anything written to the
+# twin fails them. POSIX has no twin, so this bites on the Windows CI leg (test
+# 64), exercising only the code path elsewhere.
 
 : "${top_srcdir:=..}"
 
 bash "$top_srcdir/tests/local-crawl.sh" --outdir-intl 'café' --errors 0 --files 5 \
     --found 'simple/basic.html' \
     --log-found 'mirror complete in' \
+    --cache-under-logroot \
     httrack 'BASEURL/simple/basic.html'

tests/local-crawl.sh

@@ -118,6 +118,9 @@ while test "$pos" -lt "$nargs"; do
         nopurge=1
         audit+=("--no-purge")
         ;;
+    --cache-under-logroot)
+        audit+=("--cache-under-logroot")
+        ;;
     --tls)
         tls=1
         scheme=https
@@ -379,6 +382,15 @@ while test "$i" -lt "${#audit[@]}"; do
             sed -e 's/.*[[:space:]]\([^ ]*\)[[:space:]]files written.*/\1/g')
         assert_equals "checking files" "${audit[$i]}" "$nFiles"
         ;;
+    --cache-under-logroot)
+        # The cache must sit under path_log (logroot), not an ANSI-mangled twin
+        # of a non-ASCII -O dir (#630 cache half). Bites only on the Windows leg.
+        info "checking cache under logroot"
+        if test -e "${logroot}/hts-cache/new.zip"; then result "OK"; else
+            result "cache not under logroot (mangled twin?)"
+            exit 1
+        fi
+        ;;
     --found)
         i=$((i + 1))
         info "checking for ${audit[$i]}"