ZIP_READFIELD_STRING in proxy/store.c took no destination size and used a
raw strcpy, while the engine's namesake in htscache.c has taken a
refvalue_size for years. The source is a cache-entry header line bounded
only by line[HTS_URLMAXSIZE + 2].
contenttype[64] sits just before the location pointer in the calloc'd
_PT_Element, so an over-long Content-Type walks over charset and then
over location, which the next Location: line copies through.
Clip rather than reject: the engine stores Content-Type in 128 bytes, so
a valid cache legitimately carries fields wider than ours.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* htsserver builds the redirect Location header in a 256-byte stack buffer
The POST redirect path checks strlen(file) but sprintf's newfile, which comes
straight from the client's "redirect" POST field with no length cap. A 300-byte
value overflows tmp[256]. The same value reached the Location header with no
CR/LF check, so it could split the response and inject headers.
Append into the dynamic String the other headers already use, and drop the
header entirely when the value carries a CR or LF.
The listen socket was SOCaddr_initany, so the server answered the LAN and not
just the local browser it exists to serve. Bind 127.0.0.1 by default, with
--bind <addr> to widen it again, resolved through the existing gethost() helper
the way proxytrack already does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: satisfy shellcheck and shfmt in the new server test
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: drop the pre-fix narration from the oversized-value comment
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: assert the bound socket, not the announced URL
The listen-address assertions only compared the URL= banner, which is a
literal echo of argv: a build that announced 127.0.0.1 while binding the
wildcard passed. Probe 127.0.0.2 on the same port instead, which a wildcard
listener takes and a loopback-only one leaves free.
Also refuse an empty --bind, which fell through to every interface and
silently undid the new default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: bound the response read and the empty --bind run
The recv() loop had no timeout and read until EOF; htsserver need not close
the connection after responding, which wedged the macOS runner for over an
hour. Stop at the end of the header block, which is all the test reads.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* htsserver: the session id must gate the request body, not the reply
Every field of a POST body is written straight into the one global key store
the templates and the command dispatcher both read, and "command" from there
reaches the engine. The gate ran after that write and compared "sid" against
"_sid" -- but "_sid" is copied into "sid" beforehand so the templates can
render it, so a request that simply omitted the field compared equal to
itself. Only a wrong id was refused; an absent one passed. Clearing the reply
afterwards does not help either, because the dispatcher sits outside the reply
guard.
Authenticate before parsing instead: scan the raw body for "sid", require at
least one occurrence and reject if any of them differs, and drop the body
untouched when it does not match. That leaves the shared template key alone,
and it closes the dispatcher for free.
A refused request also emitted only a Content-length line, since the status
line for that branch was behind _DEBUG. Any client reads that as a protocol
error, which is how test 68 failed rather than reporting the refusal. Send a
403 instead.
Tests 68 and 77 posted without an id, which is what the engine used to accept,
so both now fetch the one the server renders into the form. Test 78 covers
accept, missing, empty and wrong, asserts the 403, and probes the key store
through ${projname} rather than the suppressed reply -- a reply-only assertion
passes even when the write goes through.
Also fix a leak that hung macOS CI: start() runs inside a command
substitution, so its $! never reached the parent and stop() guarded on an
empty variable, leaving one htsserver per call. Test 77 starts four, which is
exactly the four orphans the runner reported while sitting for half an hour
behind a green test log. Take the pid from the PID= line the server already
announces.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* htsserver: serve the crawled mirror verbatim, never through the expander
WebHTTrack serves the mirror under /website/ from the same small server as its
own GUI, and the decision to run a response through the ${...} template
expander was a substring test for ".htm" on the request path. A mirrored page
therefore had its directives evaluated: ${_sid} rendered the live session id,
handing the crawled site the token that authenticates commands on the local
GUI, and ${do:...} gave it the rest of the template verbs.
The /website/ prefix was already detected, but only to keep the crawl-state
override from hijacking a mirror request. Reuse it as the expansion gate, so
expansion is limited to files under the GUI's html root, and re-evaluate it
after that override, which can substitute a GUI page for a mirror path.
Mirrored pages keep their text/html type: verbatim must not turn browsing the
mirror into a download.
Note that mirrored content still shares the control origin, so a script in it
can read the session id from a GUI page itself; that is a separate fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* htsserver: stop letting a posted projpath name the /website/ root
The static-file path in smallserver() was composed with a length guard that
measured only the server root and the request path, then wrote the posted
"projpath" field into a 1024-byte stack buffer: a 2000-byte projpath followed
by any /website/ request smashed the stack (buffer overflow detected, server
gone). The guard also summed two untrusted lengths before comparing, the shape
that can wrap and pass.
Composition now goes through a bounded, non-aborting append that keeps the
untrusted length alone on one side, and /website/ is served from the project
directory the server itself set up, rejecting a ".." in it, rather than from
whatever root the request body claimed. Without that, projpath=/etc/ plus
GET /website/passwd read an arbitrary file, since the ".." check looked at the
request path only. fsfile is also cleared before the error-redirect branch,
which could otherwise reach fopen() uninitialized.
tests/80_webhttrack-projpath.test drives the three cases against a live server
and keeps a legitimate project browsable as the control.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: establish test 79's mirror root through the server
/website/ no longer serves the posted projpath, so injecting one as a
fixture stopped working and test 79 got a 404 instead of the mirrored
page. Save a profile first (no command_do=start, so nothing crawls) to
make the server record the root, then plant the file under it.
Re-checked against a reverted expander gate: still fails there, so the
fixture change did not cost the assertion its teeth.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: cover the running-crawl half of the /website/ override
Test 84 only exercised the idle server, where the override never fires and
the recomputed virtualpath is indistinguishable from the stale one. Drive a
crawl through the server so /website/*.html is rewritten to the GUI refresh
page, which 404s without the recompute.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: cover the overflow, the '..' rejection and the fsfile hoist
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: fix the merged test 84 duplicate post() helper
The rename-side and the branch-side each defined post(), with different
argument shapes; the last one won and silently mangled the save body.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: trim the new comments
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: drop the stale start() arg comments in test 84
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* WebHTTrack's mirror links are dead file:// URLs
The GUI is served from http://127.0.0.1:PORT/, and browsers refuse to navigate
from an http: page to a file: URL, so the "browse the mirror" links in
finished.html and file.html did nothing when clicked. They now point at the
server's own /website/ route, which already serves the project directory. The
desktop entry's browse mode still works, because there the shell hands the
file: URL to the browser instead of navigating from a page.
The per-project picker in file.html goes too: /website/ is bound to the running
project, and reaching the other projects over HTTP would mean serving the whole
mirror tree from the control origin.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* finished.html: link the served mirror, not a dead file:// URL
The page is served over http:, so its file:// link is a cross-scheme
navigation that Chrome and Firefox refuse. Nothing happened on click. The
mirror is already reachable at /website/, which the two list entries just
below were using all along.
file.html keeps its file:// links. It is the fresh-session entry point for
sites mirrored earlier, and /website/ is bound to one project, so pointing
it there would trade a dead link for a 404. Serving an arbitrary past
project needs a route that does not exist yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: tighten the browse-link assertion and trim its comments
The runtime check for href="/website/index.html" also matched the list
entry below the anchor, so it passed on the unfixed page; match the
anchor by its mirror-path label instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* htsserver builds the redirect Location header in a 256-byte stack buffer
The POST redirect path checks strlen(file) but sprintf's newfile, which comes
straight from the client's "redirect" POST field with no length cap. A 300-byte
value overflows tmp[256]. The same value reached the Location header with no
CR/LF check, so it could split the response and inject headers.
Append into the dynamic String the other headers already use, and drop the
header entirely when the value carries a CR or LF.
The listen socket was SOCaddr_initany, so the server answered the LAN and not
just the local browser it exists to serve. Bind 127.0.0.1 by default, with
--bind <addr> to widen it again, resolved through the existing gethost() helper
the way proxytrack already does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: satisfy shellcheck and shfmt in the new server test
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: drop the pre-fix narration from the oversized-value comment
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: assert the bound socket, not the announced URL
The listen-address assertions only compared the URL= banner, which is a
literal echo of argv: a build that announced 127.0.0.1 while binding the
wildcard passed. Probe 127.0.0.2 on the same port instead, which a wildcard
listener takes and a loopback-only one leaves free.
Also refuse an empty --bind, which fell through to every interface and
silently undid the new default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: bound the response read and the empty --bind run
The recv() loop had no timeout and read until EOF; htsserver need not close
the connection after responding, which wedged the macOS runner for over an
hour. Stop at the end of the header block, which is all the test reads.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* htsserver: the session id must gate the request body, not the reply
Every field of a POST body is written straight into the one global key store
the templates and the command dispatcher both read, and "command" from there
reaches the engine. The gate ran after that write and compared "sid" against
"_sid" -- but "_sid" is copied into "sid" beforehand so the templates can
render it, so a request that simply omitted the field compared equal to
itself. Only a wrong id was refused; an absent one passed. Clearing the reply
afterwards does not help either, because the dispatcher sits outside the reply
guard.
Authenticate before parsing instead: scan the raw body for "sid", require at
least one occurrence and reject if any of them differs, and drop the body
untouched when it does not match. That leaves the shared template key alone,
and it closes the dispatcher for free.
A refused request also emitted only a Content-length line, since the status
line for that branch was behind _DEBUG. Any client reads that as a protocol
error, which is how test 68 failed rather than reporting the refusal. Send a
403 instead.
Tests 68 and 77 posted without an id, which is what the engine used to accept,
so both now fetch the one the server renders into the form. Test 78 covers
accept, missing, empty and wrong, asserts the 403, and probes the key store
through ${projname} rather than the suppressed reply -- a reply-only assertion
passes even when the write goes through.
Also fix a leak that hung macOS CI: start() runs inside a command
substitution, so its $! never reached the parent and stop() guarded on an
empty variable, leaving one htsserver per call. Test 77 starts four, which is
exactly the four orphans the runner reported while sitting for half an hour
behind a green test log. Take the pid from the PID= line the server already
announces.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* htsserver: serve the crawled mirror verbatim, never through the expander
WebHTTrack serves the mirror under /website/ from the same small server as its
own GUI, and the decision to run a response through the ${...} template
expander was a substring test for ".htm" on the request path. A mirrored page
therefore had its directives evaluated: ${_sid} rendered the live session id,
handing the crawled site the token that authenticates commands on the local
GUI, and ${do:...} gave it the rest of the template verbs.
The /website/ prefix was already detected, but only to keep the crawl-state
override from hijacking a mirror request. Reuse it as the expansion gate, so
expansion is limited to files under the GUI's html root, and re-evaluate it
after that override, which can substitute a GUI page for a mirror path.
Mirrored pages keep their text/html type: verbatim must not turn browsing the
mirror into a download.
Note that mirrored content still shares the control origin, so a script in it
can read the session id from a GUI page itself; that is a separate fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: cover the running-crawl half of the /website/ override
Test 84 only exercised the idle server, where the override never fires and
the recomputed virtualpath is indistinguishable from the stale one. Drive a
crawl through the server so /website/*.html is rewritten to the GUI refresh
page, which 404s without the recompute.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* webhttrack: "max site size" set a per-file cap instead of the overall one
step4.html mapped all three size fields of the wizard onto --max-files (-m),
so "Max site size" emitted a per-file limit rather than --max-size (-M), and
landed a second -m on the command line. That second -m also clobbered the HTML
per-file limit: a bare -m<n> resets maxfile_html, so whichever of the two came
last won. Point sizemax at --max-size and emit the bare -m before the -m,<n>
form so both per-file caps survive.
Two template typos in the same family, where a malformed ${...} renders as
nothing or as its own key instead of erroring: the winprofile.ini writer's
Dos=${dos was missing its closing brace, and option2b.html's OK button read
${LANG_OK] with a bracket.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* tests: carry the session id in test 81's POST
The gate that landed with #700 refuses a body without one, so the wizard
POST came back refused and the option audit had nothing to read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: close the confirmation-biased gaps in test 81
Post the size fields empty too, so a step4.html that lost its ${test:} guard
and rendered a valueless --max-files= is caught; assert the winprofile.ini
MaxHtml/MaxOther/MaxAll keys the header claimed to audit; and pin the OK button
label, since an unknown ${LANG_} key renders empty and passed the absence check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds `sprintfbuff()`/`slprintfbuff()` to `htssafe.h`: a formatted print that truncates to fit and returns whether it had to, marked `warn_unused_result` so the answer cannot be dropped. It fills the gap between the `strcpybuff` family, which aborts on overflow, and `String`, which grows without bound. Abort is the wrong contract wherever the text is built from a remote peer's reply.
Four `-Wformat-truncation=` sites used the result as if `snprintf` had never truncated. Two only needed a bigger destination, so they get one: `hts_finish_makeindex`'s `tempo` was a flat 1024 against a 2048-byte escaped URL and is now sized off it, and the wizard's `cmd[4096]` could not hold the answers it concatenates. The other two cannot grow. `create_back_tmpfile` formats `<url_sav>.bak` into a buffer the same size as `url_sav`, and that struct is installed, so a dropped extension would alias the backup onto the live file that `back_finalize_backup()` unlinks. ProxyTrack's `startUrl[1024]` is fed by cache content. Both take the error path they already had, and ProxyTrack moves to the next cache entry rather than publishing a clipped one.
On what the wrapper buys, since it is not what I first assumed: checking a raw `snprintf` return inline silences the warning just as well. The wrapper's value is that the capacity comes from `sizeof`, the check is the default rather than the exception, and `warn_unused_result` makes skipping it visible.
`-#test=strsafe` covers the primitive (exact fit, one over, 4 KB source, `size == 1`, trailing canary, destination repoisoned between cases), and `-#test=makeindex` gains a first link whose escaped form overruns the old buffer. Both were mutation-checked. The ProxyTrack change ships without a direct test: its only observable is the catalog page, which renders solely as a PROPFIND fallback I could not drive from curl. 25 gcc warnings down to 21; the rest of the cluster is diagnostic-only and follows separately.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* htsserver builds the redirect Location header in a 256-byte stack buffer
The POST redirect path checks strlen(file) but sprintf's newfile, which comes
straight from the client's "redirect" POST field with no length cap. A 300-byte
value overflows tmp[256]. The same value reached the Location header with no
CR/LF check, so it could split the response and inject headers.
Append into the dynamic String the other headers already use, and drop the
header entirely when the value carries a CR or LF.
The listen socket was SOCaddr_initany, so the server answered the LAN and not
just the local browser it exists to serve. Bind 127.0.0.1 by default, with
--bind <addr> to widen it again, resolved through the existing gethost() helper
the way proxytrack already does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: satisfy shellcheck and shfmt in the new server test
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: drop the pre-fix narration from the oversized-value comment
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: assert the bound socket, not the announced URL
The listen-address assertions only compared the URL= banner, which is a
literal echo of argv: a build that announced 127.0.0.1 while binding the
wildcard passed. Probe 127.0.0.2 on the same port instead, which a wildcard
listener takes and a loopback-only one leaves free.
Also refuse an empty --bind, which fell through to every interface and
silently undid the new default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: bound the response read and the empty --bind run
The recv() loop had no timeout and read until EOF; htsserver need not close
the connection after responding, which wedged the macOS runner for over an
hour. Stop at the end of the header block, which is all the test reads.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* htsserver: the session id must gate the request body, not the reply
Every field of a POST body is written straight into the one global key store
the templates and the command dispatcher both read, and "command" from there
reaches the engine. The gate ran after that write and compared "sid" against
"_sid" -- but "_sid" is copied into "sid" beforehand so the templates can
render it, so a request that simply omitted the field compared equal to
itself. Only a wrong id was refused; an absent one passed. Clearing the reply
afterwards does not help either, because the dispatcher sits outside the reply
guard.
Authenticate before parsing instead: scan the raw body for "sid", require at
least one occurrence and reject if any of them differs, and drop the body
untouched when it does not match. That leaves the shared template key alone,
and it closes the dispatcher for free.
A refused request also emitted only a Content-length line, since the status
line for that branch was behind _DEBUG. Any client reads that as a protocol
error, which is how test 68 failed rather than reporting the refusal. Send a
403 instead.
Tests 68 and 77 posted without an id, which is what the engine used to accept,
so both now fetch the one the server renders into the form. Test 78 covers
accept, missing, empty and wrong, asserts the 403, and probes the key store
through ${projname} rather than the suppressed reply -- a reply-only assertion
passes even when the write goes through.
Also fix a leak that hung macOS CI: start() runs inside a command
substitution, so its $! never reached the parent and stop() guarded on an
empty variable, leaving one htsserver per call. Test 77 starts four, which is
exactly the four orphans the runner reported while sitting for half an hour
behind a green test log. Take the pid from the PID= line the server already
announces.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* webhttrack: bound the argv vector and escape quotes in the wizard command line
The wizard hands its httrack command line to the engine as one string, which
back_launch_cmd() split back into argv. Two things were wrong with that split.
It wrote into a fixed 1024-pointer vector with no bound, and every unquoted
space in the posted string yields an entry, so an ordinary mirror with a few
hundred URLs walked off the allocation. The split now lives in htscmdline.c as
hts_split_cmdline(), which sizes the vector from the separator count before
filling it, and the engine self-tests can reach it.
Quotes were also purely advisory: they toggled the "inside an argument" state
but nothing escaped them, so a double quote typed into a wizard field (user
agent, footer, path, project name) closed the argument early and the rest of
the value was parsed as fresh options -- among them -V, which reaches system().
Escaping has to happen where the argument boundary is known, so the template
gets its own ${arg:} filter for that context; ${html:} keeps its meaning for
the HTML attributes it is used in everywhere else, and HTML escaping would not
help anyway since the browser undoes it when it posts the command line back.
${arg:} backslash-escapes a quote and a backslash, and the splitter reads those
inside a quoted run, the same convention next_token() already implements for
doit.log. A value containing a quote now survives it intact instead of turning
into options.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* webhttrack: keep a stray quote in an unquoted field out of the split
The url and wildcard-filter fields go into the command line outside quotes,
where no backslash can escape anything: a single quote there flips the parity
of every quote after it, so a later escaped value ends up split as flags and
the escaping buys nothing. Emit %22 for those fields instead.
NULL-terminate the argv vector while here, matching the convention the tree
documents in htscharset.c, and fold the third copy of the entity table into
one helper.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: do not feed snprintf's return value back as its size argument
snprintf returns the length it wanted to write, so accumulating it blind
lets the next size argument wrap. The buffer is sized well past what the
loop needs, but the pattern is the one the project forbids.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Symbolize fatal-signal backtraces through addr2line
backtrace_symbols_fd() resolves names from .dynsym only, and
-fvisibility=hidden keeps every engine frame out of it, so a crash report
arrived as a column of bare module+offset. The handler now emits the raw trace
first and unconditionally, then groups the frames per module and runs addr2line
(or llvm-symbolizer) over the offsets, which reads DWARF and names the static
frames plus their inline chain.
-rdynamic is dropped: it only populated .dynsym and bought exactly one named
frame. -Wl,--build-id replaces it, so a trace from a stripped build can be
matched to its debug symbols.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Move the crash backtrace printer into src/htsbacktrace.c
httrack.c keeps only the two call sites. The symbolizer needs _GNU_SOURCE for
dladdr(), which is now confined to its own translation unit instead of being
forced on the whole CLI front-end.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Test: match glibc's backtrace format with or without the space
backtrace_symbols_fd() prints the trailing "[0xADDR]" with a leading space on
some glibc versions and without on others (Ubuntu 24.04), so the raw-frame
assertion failed everywhere but the dev box. Match only up to the offset.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Skip pseudo-modules with no file on disk
A frame in linux-vdso.so.1 made addr2line complain instead of resolving, so
the arm64 leg lost its symbolized output entirely. Renumber the test too:
77 landed on master with #700.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Stop discarding local symbols, which made the names wrong
--discard-all drops the local symbol of every static function, so addr2line
attributes the frame to the nearest surviving global: the abort frame read
dns_timeout_selftests instead of abortf_, with the file and line still right.
A wrong name is worse than none, and because the symbols go at link time no
-dbgsym package can recover them. Costs 21784 bytes on libhttrack.so.3, 0.6%.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Symbolize once when the handler itself faults
A fault inside the handler re-enters the printer, which interleaved a second
symbolized trace on the same fd and spent a second budget: 3.04s and two
overlapping reports, measured.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <xroche@gmail.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Drop NULL tests on inline array members
`lien_back::url_sav`, `htsblk::msg` and POSIX `dirent::d_name` are arrays,
so testing their address folds to a constant and gcc/clang report it
(-Waddress, -Wpointer-bool-conversion). Every site keeps whatever real
condition sat beside the dead one, so behavior is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* help_wizard: check the allocation, not the arrays it contains
The out-of-memory guard has been constant-false since d593418 folded the
nine separate wizard buffers into one struct: the names it tests are now
inline arrays, so `malloct()`'s result is never checked and an exhausted
heap gets a NULL-page write instead of the intended message. Also switch
the raw free() to freet() and release the struct on the two early returns
that leaked it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* proxytrack: test the WebDAV header fields, not their addresses
`PT_Element::lastmodified` and `::contenttype` are inline arrays, so both
`if`s were constant-true (-Waddress); use the `[0]` form the same file
already uses when it emits the GET headers. Neither is observable:
get_time_rfc822("") returns 0 and falls through to the index timestamp,
and proxytrack_add_DAV_Item already substitutes application/octet-stream
for an empty mime, which a PROPFIND probe against a cache entry carrying
no Content-Type confirms both before and after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Fix the discarded const qualifiers rather than casting them away
`binput` and `cache_binput` only ever read through their source pointer,
so they take `const char *` now; that alone clears the cast in
htsrobots.c, and neither is exported nor declared in an installed header,
so no ABI question arises. `treathead` keeps `char *rcvd` because it does
NUL-cut the header in place, and the two selftest calls that fed it a
string literal get a mutable buffer instead, matching their three
siblings and removing a latent write to .rodata. The remaining two are
one-liners: zlib's `next_in` is already `const` under -DZLIB_CONST, and
htsback can call the non-const `jump_protocol` twin on its mutable
`url_adr`. libhttrack.vcxproj gains ZLIB_CONST so the MSVC build agrees
with autotools, as webhttrack and proxytrack already do.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: prove the WebDAV mime/timestamp fallback survives an empty field
proxytrack's DAV PROPFIND response computes a fallback content-type and
timestamp when a cache entry has no Content-Type/Last-Modified; that
fallback already existed before commit eae1dd0 changed the surrounding
always-true array-address checks, so this test guards the equivalence
rather than a bug. Verified it fails when the fallback default is
disabled, and passes unmodified against the pre-eae1dd0 code too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: bound the PROPFIND request and cut the header comment
curl had no --max-time; an unbounded read wedges the runner instead of
failing it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: skip the WebDAV mime test on Windows
It is the first test to run proxytrack as a live listener, and MSYS cannot
reap a native one: the orphan wedged the whole Windows suite past its
45-minute budget, twice, destroying the log upload with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* htsserver builds the redirect Location header in a 256-byte stack buffer
The POST redirect path checks strlen(file) but sprintf's newfile, which comes
straight from the client's "redirect" POST field with no length cap. A 300-byte
value overflows tmp[256]. The same value reached the Location header with no
CR/LF check, so it could split the response and inject headers.
Append into the dynamic String the other headers already use, and drop the
header entirely when the value carries a CR or LF.
The listen socket was SOCaddr_initany, so the server answered the LAN and not
just the local browser it exists to serve. Bind 127.0.0.1 by default, with
--bind <addr> to widen it again, resolved through the existing gethost() helper
the way proxytrack already does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: satisfy shellcheck and shfmt in the new server test
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: drop the pre-fix narration from the oversized-value comment
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: assert the bound socket, not the announced URL
The listen-address assertions only compared the URL= banner, which is a
literal echo of argv: a build that announced 127.0.0.1 while binding the
wildcard passed. Probe 127.0.0.2 on the same port instead, which a wildcard
listener takes and a loopback-only one leaves free.
Also refuse an empty --bind, which fell through to every interface and
silently undid the new default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: bound the response read and the empty --bind run
The recv() loop had no timeout and read until EOF; htsserver need not close
the connection after responding, which wedged the macOS runner for over an
hour. Stop at the end of the header block, which is all the test reads.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* htsserver: the session id must gate the request body, not the reply
Every field of a POST body is written straight into the one global key store
the templates and the command dispatcher both read, and "command" from there
reaches the engine. The gate ran after that write and compared "sid" against
"_sid" -- but "_sid" is copied into "sid" beforehand so the templates can
render it, so a request that simply omitted the field compared equal to
itself. Only a wrong id was refused; an absent one passed. Clearing the reply
afterwards does not help either, because the dispatcher sits outside the reply
guard.
Authenticate before parsing instead: scan the raw body for "sid", require at
least one occurrence and reject if any of them differs, and drop the body
untouched when it does not match. That leaves the shared template key alone,
and it closes the dispatcher for free.
A refused request also emitted only a Content-length line, since the status
line for that branch was behind _DEBUG. Any client reads that as a protocol
error, which is how test 68 failed rather than reporting the refusal. Send a
403 instead.
Tests 68 and 77 posted without an id, which is what the engine used to accept,
so both now fetch the one the server renders into the form. Test 78 covers
accept, missing, empty and wrong, asserts the 403, and probes the key store
through ${projname} rather than the suppressed reply -- a reply-only assertion
passes even when the write goes through.
Also fix a leak that hung macOS CI: start() runs inside a command
substitution, so its $! never reached the parent and stop() guarded on an
empty variable, leaving one htsserver per call. Test 77 starts four, which is
exactly the four orphans the runner reported while sitting for half an hour
behind a green test log. Take the pid from the PID= line the server already
announces.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* -%S list file over 4GB overflows the heap
hts_main_internal() sized the -%S buffer as `cl + fz + 8192` and stored
the sum in an int url_sz, then fread() the untruncated 64-bit fz into it.
A 4GB+100KB rules file wraps the capacity to 110602 bytes on x64, the
realloct() succeeds, and the read walks off the heap. Intermediate sizes
land on a negative int and fail the allocation, which is luck, not design.
Route the file-size arithmetic through llint_grow_size_t(), a saturating
sibling of llint_to_size_t() that refuses a total it cannot represent, and
widen url_sz and the filelist offsets to size_t. The "config" sizing in the
same function and htscore.c's primary_len had the same shape: a file size
accumulated into an int before reaching an allocator. htscache.c's two
mirrored-file comparisons held a 64-bit fsize_utf8() in a size_t, which
truncates on Win32 and re-downloads a >4GB file already on disk.
Found by MSVC C4244 on x64; invisible to gcc/clang because int64_t to
size_t is width-preserving on LP64.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Print T_SOC with the format matching its width, not a bare %d
T_SOC is unsigned __int64 on Win64 (htsglobal.h): passing it to fprintf's
%d is undefined behavior, flagged by MSVC C4477. Add T_SOCP beside the
typedef, following the existing LLintP/INTsysP precedent, and use it at
both deletesoc() call sites (htslib.c:2601, :2607).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Fix MSVC C2099 in the new growsize self-test
A static const object used inside another object's static initializer is a
GNU/clang extension, not standard C: MSVC's /TC C mode rejects it ("initializer
is not a constant"). Replace the over32 local with a macro.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: cover the slack-only overrun and the largest capacity
A helper dropping the slack bound passed the table; -1 as extra also refused
either way, since llint_to_size_t() maps it to SIZE_MAX regardless.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ARC cache replays a truncated HTTP reason phrase
proxytrack bounded the reason-phrase copy out of an ARC index by
sizeof(pos) - 1 where pos is a const char *, so a stored "404 Not Found"
replays as "404 Not Fou" (and "404 Not" on 32-bit). Use strncatbuff with
the destination's own size.
Fold the nine copies of the buff() family's source-capacity expression
into HTS_SIZEOF_SRC_, applying sizeof to the type so a decayed operand no
longer trips -Wsizeof-array-decay at five call sites; MSVC keeps the old
expression behind the guard HTS_IS_CHAR_BUFFER already uses. The two
other raw strncat calls become strncatbuff, htsbuff_catn stops handing
strnlen the (size_t)-1 sentinel, and htsweb.c no longer compares ep
against a NULL eps.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Size the new strsafe buffers away from sizeof(char*)
char[8] equals a pointer on LP64, so MSVC's array-vs-pointer heuristic
read the unterminated source as a pointer, skipped the bound and never
aborted; the x64 build failed while Win32 passed. Same trap the existing
comment in that function already warns about.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* configure: fix flag probes, move -rdynamic to the link line
AX_CHECK_COMPILE_FLAG only checks the exit status, and clang merely warns on
an unknown -W name, so -Wmissing-parameter-type reached every clang build and
warned on all 66 TUs. Probe with -Werror; -Wformat-nonliteral also needs
-Wformat there or gcc rejects it and the flag would be lost.
-rdynamic lived in DEFAULT_CFLAGS, i.e. AM_CPPFLAGS, so no link line ever saw
it; make it a link check. Drop -pie from CFLAGS_PIE (LDFLAGS_PIE has it), and
drop -Wdeclaration-after-statement, a C90 rule the gnu17 build does not follow
anywhere else.
Distinct build warnings: gcc 82 -> 54, clang 54 -> 41.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* configure: tighten the two new flag-block comments
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
WinHTTrack is an MBCS build, so it still lands on the ANSI-only Win32 entry points (TTN_NEEDTEXTA and the like) while the engine hands it UTF-8. The other direction is already exported as `hts_convertStringSystemToUTF8`, so this adds the mirror, `hts_convertStringUTF8ToSystem`, a one-liner over the existing `hts_convertStringCPFromUTF8`. The GUI can then delegate instead of keeping its own MultiByteToWideChar/WideCharToMultiByte copy (`CopyTextUTF8ToCP` in newlang.cpp, added while fixing #114). `hts_convertStringFromUTF8` would also have done the job, but it is declared plain `extern` and never reaches the DLL export table; left alone here. Windows-only addition, so no POSIX ABI change and no soname move.
The new `-#test=syscharset` self-test round-trips against the raw Win32 two-step it replaces, and skips off Windows.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* CLI: redraw the whole screen when the terminal is resized
The animated -%v display assumed a fixed 80x24 layout: it cleared the screen
once at start, and afterwards cleared only to end of line on the rows it
wrote. A resize left stale wrapped text around the frame, and on a terminal
shorter than the 20-row frame the stats block scrolled off the top at every
refresh.
Poll the terminal geometry at each refresh (TIOCGWINSZ, or the console screen
buffer info on Windows) and repaint in full when it moved. The in-progress
list is capped to the rows that fit, and the URL column follows the width; it
stays at the historical 40 characters on an 80-column terminal.
Closes#97
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* tests: assert the resize repaint, the row clamp and the URL width
The first pass only checked that a clear-screen followed one resize, which a
display clearing on every frame passes just as well, and neither the row clamp
nor the URL budget was exercised at all: the single 24x80 to 40x100 resize
leaves both at their 80x24 values.
The harness now waits for the display instead of sleeping, resizes width and
height separately, and asserts a repaint after each one, none in between, a
frame that fits a 10-row terminal, and a long URL that stops being truncated at
200 columns. The long path is a new /trickle/deep route, so the shared
/trickle/ index other tests assert on stays untouched. Mutating any of the four
behaviors away fails the test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Dependabot only watched vcpkg, so the action pins drifted by hand:
windows-build.yml sat on actions/checkout@v4 while everything else moved to
v6. Add the github-actions ecosystem and bump that stray checkout to v6.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
vcpkg builds openssl/brotli/zlib/zstd from source on every Windows run.
Redirect its files binary cache into the workspace and persist it with
actions/cache, keyed on the manifest (which carries the builtin-baseline).
x-gha, which used to do this, was dropped from vcpkg-tool (#1662) after
GitHub changed the Actions cache API; actions/cache over the archive dir is
the endorsed replacement and stays within the GitHub-owned-actions policy.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Drop the removed Java .class parser from the credits
The Java binary .class parser was removed in #552, but its credit lingered:
"JavaParserClasses: Yann Philippot" in the WinHTTrack About box (lang.def plus
every lang/*.txt), and "for the java binary .class parser" in greetings.txt and
the contact page. Remove the About-box line and the now-defunct descriptor,
keeping Yann Philippot's name in the developed-by acknowledgments.
lang.def and lang/*.txt are read as strict line pairs, so the edit deletes the
substring inside the single credits line rather than removing any line, keeping
the pairing intact and each translated file's English msgid in lockstep with
lang.def. Verified by tests/62_lang-integrity.test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Also drop the localized java-parser credit from the translations
The About box shows the translation (msgstr), not the English msgid. The first
pass only removed the literal English "JavaParserClasses: Yann Philippot", so the
12 files whose translators localized the credit (Chinese, French, Russian,
Turkish, ...) still displayed it. Remove the localized credit line from each of
those msgstrs too; the pairing and every English msgid are untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Reword Yann Philippot's credit as a past contributor
Dropping the descriptor left a bare name in the developed-by list. Mark the
java binary .class parser as a past contribution instead, keeping the
acknowledgment meaningful now that the code is gone.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* contact.html: use a literal dot in .class
The " dot " spelling is the page's email anti-harvest obfuscation; applied to a
filename extension it just left a stray double space. The page already writes
literal dots elsewhere (v2.0, v3.0).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bump the version in configure.ac, htsglobal.h and version.rc, move VERSION_INFO to 3:6:0, and add the history.txt and debian/changelog entries.
The WARC fields landed at the tail of httrackp and htsblk, but the embedded htsoptstate and htsblk members sit mid-struct, so the fields after them shifted. Installed layouts moved while the soname stays .so.3; the VERSION_INFO comment records that rather than claiming the layouts are unchanged.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Windows: import IE cookies via FindFirstFileW so long/non-ASCII jars load
cookie_load's IE-cookie scan and its cookies.txt read used the narrow
ANSI file API (FindFirstFileA, fopen, remove), so a cookie folder with a
non-ASCII or >MAX_PATH path was silently skipped, and any matched IE
cookie name was fed back through the mirror path as CP_ACP bytes. Route
the glob through hts_pathToUCS2 + FindFirstFileW, convert cFileName to
UTF-8 before rebuilding the path, and use the FOPEN/UNLINK wrappers.
The A->W find and the sink swaps land together on purpose: swapping only
the sink would feed a CP_ACP name into hts_fopen_utf8's UTF-8 decode and
mojibake the accidentally-consistent ANSI path. hts_pathToUCS2 is
un-static'd (internal, still -fvisibility=hidden / not HTSEXT_API) so the
glob gets the same \\?\ long-path treatment as the other wrappers.
Last piece of the #133 Windows long-path series. Self-test
-#test=cookieimport drives cookie_load against a long, non-ASCII folder;
POSIX is a positive control (IE block compiled out), the wide glob and
IE import run on the Windows legs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Trim review-flagged comments to one line
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Windows: enumerate long or non-ASCII directories via FindFirstFileW
The opendir/readdir emulation was fully ANSI (FindFirstFileA, CP_ACP
cFileName), so it capped listing at MAX_PATH and mis-decoded a non-ASCII
path — the odd wrapper out in an engine that feeds UTF-8 paths and reads
d_name as UTF-8. The one live Windows caller, the end-of-crawl
hts-cache/ref cleanup, silently no-op'd at a long or non-ASCII project
root, leaking the temp ref/ directory into the finished mirror.
Route both through hts_pathToUCS2 (\\?\ prefixing, #684) and the wide
APIs, converting cFileName back to UTF-8. d_name now holds UTF-8, so
HTS_DIRENT_SIZE grows to 1024 to fit MAX_PATH's worst-case expansion; the
struct is _WIN32-only, no POSIX ABI impact. Self-test direnum drives it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Trim comments to one line each (review follow-up)
Condense the opendir/readdir and direnum-selftest comments per the
review's comment-conciseness pass; no behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* topindex: convert the winprofile.ini category to UTF-8 on Windows (#216)
The project category read from hts-cache/winprofile.ini comes back in the
Windows ANSI codepage and is written into the charset=utf-8 topindex
template, so a non-ASCII category renders as mojibake. Convert it the same
way #681 did the project name. The topindex self-test now writes a
winprofile.ini with a non-ASCII category and asserts the generated index
carries the UTF-8 form.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* topindex: drop the redundant category-assert comment
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 01_engine-fsize self-test builds a 5GB sparse file to exercise the 32-bit size wrap. On GNU/Hurd i386 the ext2fs translator rejects the extend with EFBIG, failing the Debian build on that port. Return 77 (automake skip) instead of 1 when the extend fails with EFBIG, and map that to a skip in the .test wrapper. Any other errno, or a wrong size report, still fails, so real regressions are not masked.
* Windows: route mirror-tree file ops through the UTF-8/long-path wrappers
The engine's raw fopen/remove/rename/rmdir/mkdir/fexist/fsize calls on
mirror-tree paths bypassed the UTF-8 (_w*) wrappers, so on Windows a mirror
under a non-ASCII or >MAX_PATH directory silently broke: a long/non-ASCII
file read as absent (its unlink/reget skipped), or its bytes went to a
mojibaked, truncated path.
Swap those call sites in htscoremain/htscore/htsparse/htsindex/htshelp to
the FOPEN/UNLINK/RENAME/MKDIR/fexist_utf8/fsize_utf8 twins, and add
hts_rmdir_utf8 (RMDIR) for the three rmdir sites that lacked a wrapper. The
arguments are already UTF-8 (Windows argv is transcoded at startup, fconcat
is byte-transparent), so there is no double-conversion. Raw calls on
genuinely system-charset or ASCII paths (getenv/$HOME, structcheck's ASCII
twin, "config") are left as-is.
-#test=mirrorio drives a long AND non-ASCII path through the wrapped guards
and the new rmdir wrapper; on POSIX it stands as a byte-transparent control.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* test: cover RENAME on the long+non-ASCII mirror path
The mirrorio self-test drove FOPEN/UNLINK/RMDIR but not RENAME, which the
sweep newly routes to. Rename the leaf to a non-ASCII sibling and verify
the move before teardown.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Windows: reach past MAX_PATH via \\?\ at the file-op choke-point (#133)
The _w* file wrappers on Windows all funnel their path through one
converter. Route it through a new hts_pathToUCS2 that, for a path near
MAX_PATH, absolutizes and normalizes it with GetFullPathNameW and
prepends the "\\?\" verbatim prefix ("\\?\UNC\" for UNC shares) so the
wide file APIs accept paths past 260 chars. Shorter paths keep today's
exact behavior, so nothing changes until the naming ceiling in htsname.c
is raised (a follow-up); a -#test=longpath self-test drives a >260-char
path through the wrappers to exercise the new branch on Windows CI now.
Partially addresses #133.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Run the long-path self-test on the Windows CI leg; tighten comments
The Windows job iterates an explicit test glob (01_engine-*, 01_zlib-*,
*_local-*, ...), so 76_engine-longpath-io.test matched nothing and the
\\?\ path never ran on the one platform that needs it. Rename it to
01_engine-longpath-io.test so the glob picks it up and the prefixing is
actually exercised on Windows. Also trim the review comments to the why.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Use the platform path limit off Windows, not the Windows MAX_PATH (#133)
url_savename capped every saved path at 236 chars (MAX_PATH minus 8.3
headroom minus the ".delayed" marker) on all platforms, so Linux, macOS
and Android hashed long names to fit a limit only Windows has. Off
Windows, derive the ceiling from the platform's own PATH_MAX/NAME_MAX,
clamped to the fixed save buffer so an oversized output dir can never
push the final path past it and abort. Windows keeps the MAX_PATH
ceiling until the engine can prefix its paths with \\?\.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Cut the save name in bytes, not codepoints, so multibyte paths can't overflow
The path-length guard measured length in UTF-8 codepoints (hts_stringLengthUTF8)
while the buffer that receives parent+name is a fixed byte array whose overflow
aborts() rather than truncates. With the ceiling raised to ~1984 on POSIX, a
multibyte name (e.g. a long CJK path) can stay under the codepoint cap yet run
its byte length past the 2048-byte buffer, crashing the crawl on the final
prepend. Add an unconditional byte-boundary cut before the parent is prepended,
and drop the earlier codepoint-unit clamp it supersedes. Covers the oversized
-O parent case in the same measure. Regression test uses a 339-char CJK name
under a ~1020-byte output dir, which aborted before the cut.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Leave an oversized parent to the existing abort, not an empty colliding name
When the output dir alone fills the save buffer, forcing the name to empty made
every URL under it collapse to the same path, feeding the unbounded collision
sprintf and turning a clean abort into a 1-2 byte overflow. Only shrink the
name; a buffer-filling parent aborts on the prepend exactly as before (that
oversized-parent path is pre-existing and unrelated to the #133 ceiling).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* topindex: non-ASCII project names render as mojibake on Windows (#216)
hts_buildtopindex() lists each sub-project by the name FindFirstFileA
returns in the ANSI codepage, then writes it into a document declaring
charset=utf-8, so a non-ASCII name shows up as mojibake. Convert the name
to UTF-8 on Windows, matching the gif-path fix already in this function
for #217. The topindex self-test now builds a non-ASCII sub-project and
asserts the generated index carries the UTF-8 name.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* topindex: drop const so freet() can null the converted name (MSVC)
The #ifdef _WIN32 conversion block only compiles on Windows, where MSVC
rejected freet() nulling a char *const (C2166). Make the pointer mutable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* topindex: tighten the added comments
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The command-line guide's recipe section had no entry for WARC output, so
the five --warc* flags were discoverable only through --help or the man
page. Add a §11 recipe leading with the gotcha that trips people: the
archive is written alongside the browsable mirror, not instead of it.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
WARC is a transaction-level archive of what was fetched (a sibling of the
hts-cache), not a browsable-mirror build format like MHTML, so it belongs in
the Log/index/cache group next to the cache options. Help text only; the
webhttrack GUI already places it on the Log/Index/Cache tab (option9).
Signed-off-by: Xavier Roche <xroche@gmail.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Verbatim compressed bodies (Content-Encoding kept, only hop-by-hop
Transfer-Encoding dropped, Content-Length rewritten to the stored length)
are now the sole behavior, matching wget --warc and Heritrix. The former
default that decoded the body and stripped Content-Encoding is gone, along
with the --warc-verbatim switch (added the same day, never released), the
warc_verbatim option field, and the dual-mode plumbing in
normalize_http_headers.
This also fixes a cap-truncated compressed response: it previously stored
the raw compressed partial while stripping Content-Encoding, mislabeling
gzip bytes as identity. Keeping Content-Encoding makes the record's label
match its body; the truncation still carries WARC-Truncated: length/time.
header_is now tolerates whitespace before the ':' so a non-compliant
"Content-Encoding : gzip" is still recognized.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add --warc-cdx: sorted CDXJ index alongside the WARC archive
WARC v2 PR B. --warc-cdx (alias --warc-cdxj, sub-flag -%rc) writes a sorted
CDXJ index next to the .warc.gz, one line per response/revisit/resource record
(warcinfo/request are not indexed): the SURT sort key, a 14-digit timestamp,
and a JSON object with url, mime, status, payload digest, and the record's byte
offset and length in the gzip stream so a replay tool can seek and inflate a
single member. Rotation-aware (the filename field tracks the current segment).
SURT canonicalization is new (surt_canon in htswarc.c): scheme/userinfo
dropped, host lowercased with a leading www[digits] label and the scheme
default port stripped, labels reversed and comma-joined, a non-default port
kept, IPv4 and [IPv6] literals left verbatim. Lines are accumulated in the
writer and qsort'd in LC_ALL=C byte order at close.
Tested by -#test=warc-surt (SURT vectors, 01_engine, MSan-instrumented) and
-#test=warc-cdx (end-to-end: sorted, one line per record, each offset/length
independently inflates to the matching WARC-Target-URI; 01_zlib).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Add --wacz: package the WARC archive, CDXJ index and pages as a WACZ file
--wacz bundles the crawl's WARC segment(s), the sorted CDXJ index and a
generated pages.jsonl into a single WACZ 1.1.1 package at crawl end, using
the in-tree minizip. It implies --warc and --warc-cdx. Every ZIP entry is
stored (no re-deflate of the already-gzipped WARC), as the WACZ spec
requires. datapackage.json lists each file with its SHA-256 and size, and
datapackage-digest.json chains the digest of datapackage.json.
SHA-256 comes from OpenSSL; a build without OpenSSL cannot emit conformant
digests, so --wacz is refused there with a clear log line while the WARC and
CDXJ are still written.
pages.jsonl captures the top-level 200 text/html responses (URL + WARC-Date),
bounded like the CDX accumulator; the first seeds datapackage mainPageUrl.
Tests: an in-process self-test (-#test=warc-wacz) unzips the package and
asserts the layout, STORE mode on every entry, each recomputed SHA-256, the
digest chain, and the pages header; a crawl test validates a real .wacz with
a stdlib validator (py-wacz/pywb when importable). Both skip cleanly without
OpenSSL.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Package WACZ atomically so a failed re-run can't destroy a good archive
warc_wacz_package opened <base>.wacz directly with a create/truncate zip
handle, wiping the previous archive before it checked its inputs existed. A
re-run that produced zero indexable records (empty crawl, or every fetch
erroring so warc_cdx_flush writes no .cdx) truncated the good .wacz, failed to
find indexes/index.cdx, then unlinked the now-empty file with no log. Same
data-loss class as the hard-abort file truncation (#522).
Build the package into <base>.wacz.tmp and only rename it over <base>.wacz on
full success (RENAME, with an unlink+rename fallback so Windows can clobber).
On any error unlink the temp, leave the existing archive untouched, and warn
(the old error path was silent); the writer keeps opt for close-time logging.
The warc-wacz self-test now proves it: after a good package it drops the .cdx,
re-runs empty, and asserts the .wacz is byte-unchanged. Also tightens both the
C self-test and the Python validator against the WACZ spec (profile,
wacz_version, digest path, and >= 1 pages.jsonl body row with url + ts) so a
writer emitting an empty package can't pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* WARC: add --warc-verbatim to store compressed bodies as received
--warc-verbatim (internal -%rv, implies --warc) captures a compressed
response in its as-received content-coded form instead of the decoded
strategy-B body. back_finalize already materializes the whole de-chunked
compressed body as a temp file just before decoding it, so the tee is a
save-before-unlink: adopt that spool onto the new htsblk.warc_rawpath /
warc_rawsize instead of unlinking it, and emit the response record with
Content-Encoding kept and Content-Length set to the compressed length
(normalize_http_headers gains a keep_ce mode). WARC-Payload-Digest is
then over the coded payload, which is what the record carries.
Non-compressed responses, the never-spooled is_write archive-ext case,
and the cap-truncated second emit site all leave warc_rawpath NULL and
fall back to strategy B unchanged. The spool is owned by the entry:
freed and unlinked in warc_free_request, NULLed in back_copy_static and
back_unserialize so a shallow copy never double-unlinks.
Output is now byte-replay-identical to what the origin sent, matching
wget --warc and Heritrix.
Tests: a warc-verbatim engine self-test asserts the response keeps
Content-Encoding: gzip, Content-Length == the compressed length, the
stored bytes equal the gzip input and inflate to the known plaintext
(bite-checked); tests/74_local-warc-verbatim crawls a gzip-served
fixture and runs the strategy-A/B differential through
warc-validate.py --verbatim (inflate(stored) == the decoded body).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* WARC verbatim: cover the direct-to-disk adoption path, harden the validator
Review follow-up to --warc-verbatim (strategy A).
The spool adoption in back_finalize runs on both the in-memory and the
direct-to-disk (is_write) branch, but only the in-memory path was crawl-tested.
Add a non-hypertext gzip fixture (application/octet-stream) so the crawler
streams it to disk and the is_write adoption is exercised; the warc-validate
--verbatim gate now asserts Content-Encoding kept, Content-Length == compressed
length, and inflate(stored) == the served plaintext for both assets. Confirmed
at runtime that page.html takes is_write=0 and data.bin is_write=1, and
bite-checked that dropping the is_write adoption fails the new assertion.
warc-validate --verbatim now requires WARC-Payload-Digest on a verbatim record
when the file emits digests at all (an OpenSSL build), so a regression that
drops the digest is caught without breaking the no-OpenSSL leg.
Extract warc_adopt_rawspool() so the WARC spool bookkeeping lives behind the
htswarc boundary, mirroring warc_stash_response. Comment trims throughout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
WARC v2 PR B. --warc-cdx (alias --warc-cdxj, sub-flag -%rc) writes a sorted
CDXJ index next to the .warc.gz, one line per response/revisit/resource record
(warcinfo/request are not indexed): the SURT sort key, a 14-digit timestamp,
and a JSON object with url, mime, status, payload digest, and the record's byte
offset and length in the gzip stream so a replay tool can seek and inflate a
single member. Rotation-aware (the filename field tracks the current segment).
SURT canonicalization is new (surt_canon in htswarc.c): scheme/userinfo
dropped, host lowercased with a leading www[digits] label and the scheme
default port stripped, labels reversed and comma-joined, a non-default port
kept, IPv4 and [IPv6] literals left verbatim. Lines are accumulated in the
writer and qsort'd in LC_ALL=C byte order at close.
Tested by -#test=warc-surt (SURT vectors, 01_engine, MSan-instrumented) and
-#test=warc-cdx (end-to-end: sorted, one line per record, each offset/length
independently inflates to the matching WARC-Target-URI; 01_zlib).
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Expose --warc / --warc-file in the webhttrack GUI
Add a WARC output checkbox and an optional archive-name field to the
"Log, Index, Cache" option tab, beside store-all-in-cache. The checkbox
emits --warc (auto-named archive); the text field emits --warc-file NAME.
Wiring mirrors how #589 added cookies-file and strip-query: the option9.html
form fields, the generated httrack command and winprofile.ini in step4.html,
and the reload remap in step2.html.
New LANG_WARC / LANG_WARCFILE strings and tooltips land in lang.def,
English.txt and Francais.txt; the remaining 28 language files fall back to
French for now and are a follow-up.
The webhttrack smoke test now also fetches option9.html and requires the
WARC control to render.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Translate the webhttrack WARC strings into the remaining languages
PR #672 added a WARC toggle to the webhttrack GUI with 4 new LANG keys,
translated only in English and French. Append the same 4 msgid/translation
pairs to the other 28 lang/*.txt so those locales stop falling back to
English. Each translation is encoded in the file's declared LANGUAGE_CHARSET
(the charset the server serves the page and parses the form as) with a strict
lossless round-trip, matching each file's existing CRLF/LF line ending.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add WARC/1.1 v1.1: --warc-max-size rotation, WARC-Truncated, FTP resource records
Extends the merged WARC writer with the three v1.1 items. All logic stays in
htswarc.c; the engine hooks stay thin.
--warc-max-size N (-%rs) rotates the archive into NAME-00000.warc.gz, -00001,
... once a segment passes N bytes (wget naming), each segment led by its own
warcinfo and never splitting a record. N<=0 keeps the single-file behavior.
WARC-Truncated tags a body cut short by a cap. The mirror size (-M) and time
(-E) caps abort in-flight transfers and overwrite the slot's status to a
negative TIMEOUT, after which back_finalize and the WARC hook both bail; so the
partial is archived at the abort site, before the clobber, with WARC-Truncated:
length/time. HTTrack's own incomplete/retry bookkeeping is untouched, and a
genuine broken transfer (not a cap) is still not archived. The per-file cap (-m)
rejects on Content-Length rather than truncating, so it has no truncated body.
Standard ISO 28500 tokens only (length/time/disconnect); the task's non-standard
"disk" is left out.
FTP transfers have no HTTP envelope, so an ftp:// capture becomes one resource
record: WARC-Type: resource, the payload's own Content-Type, block = payload,
no request/response. FTP back_finalize runs on the main loop (the worker thread
only flips status), so the single-writer no-lock design holds.
Self-tests warc-trunc / warc-ftp / warc-rotate added to the -#test=warc family
and 01_zlib-warc.test; each was confirmed to fail without the corresponding
code. man/httrack.1 + html regenerated for the %r help line.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Range-check --warc-max-size and regenerate the man page
Replace the unchecked sscanf on the --warc-max-size argument with a
strtoll parse that rejects non-numeric, negative, and overflowing values,
leaving the default 0 (single archive) intact. The man page and its HTML
render were already regenerated for the option's help line in the feature
commit, so this only tightens the parse.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds `--warc` / `--warc-file NAME`, writing an ISO-28500 WARC/1.1 archive of a crawl (warcinfo + request + response records, gzip per record, SHA-1 digests under OpenSSL) so HTTrack output replays in Wayback-style tools (ReplayWeb.page, pywb). Under `--update`, unchanged resources become `revisit` records instead of duplicate copies. No new dependency.
Fidelity note: HTTrack decodes gzip/br/zstd inline, so v1 stores the decoded body and normalizes the encoding headers (drops `Content-Encoding`/`Transfer-Encoding`, recomputes `Content-Length`). Valid and replayable; verbatim-compressed capture and WACZ are v2. Logic is isolated in a new `src/warc.c`.
First phase (v1) of #668.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Skip an oversized -%F footer instead of aborting the crawl
The per-page footer is expanded into a fixed ~3KB stack buffer. On overflow
hts_footer_format returns <0 and leaves the buffer unterminated, but the call
site ignored the return and ran strcatbuff on it regardless; strcatbuff's
bounded strlen finds no terminator within capacity and abort()s, killing the
crawl with SIGABRT. Reachable whenever the footer expansion exceeds the buffer
(a field referenced several times with a long URL/path, or a URL that triples
under the HTML-comment escaping); the pre-existing default footer has the same
exposure.
Guard the emit on the formatter's return: on overflow, drop the footer rather
than crash. Regression test drives a file:// crawl of a deep path with a footer
repeating {path} and asserts the crawl completes; it aborts on the pre-fix
binary.
Closes#669
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Skip the footer-overflow test on Windows (MAX_PATH)
The test triggers the overflow with a path longer than Windows MAX_PATH (260),
which the source tree and mirror output both exceed there, so httrack can't
create it and the crawl fails for an unrelated reason. Restrict it to POSIX; the
fix is platform-independent and the formatter's overflow-return contract is
still covered cross-platform by the footerfmt self-test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Allowlist the Windows skip of the footer-overflow test
The Windows CI pins expected skips so an all-skipped suite can't pass green;
01_engine-footer-overflow.test now skips there (MAX_PATH), so add it to the list.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add named {addr}{path}{date}{version} footer fields alongside the legacy %s form
The -%F footer was a bare positional printf: each %s consumed the next of a
fixed addr/path/date/version arg list. It is order-coupled and inexpressive
(you cannot reach {date} without emitting addr and path first, cannot reorder,
and a wrong %s count silently yields "???" or shifted values), and the help
text advertised a cryptic bracket syntax.
hts_footer_format now dispatches on content: a footer containing %s keeps the
legacy positional model byte-for-byte (the default footer and every existing
-%F string are unaffected), while a footer without %s uses named fields
{addr} {path} {date} {version}, with "{{"/"}}" for a literal brace and any
unrecognized {...} left verbatim so typos stay visible. Sanitization is
unchanged: addr/path still pass through html_inline_safe() at the call site,
so the comment-injection guard (#165) still holds.
Driven by a new -#test=footerfmt self-test (tests/01_engine-footerfmt.test)
covering both models, brace escaping, the mixed-mode dispatch boundary, and
the overflow/zero-size return paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Regenerate html/httrack.man.html for the -%F help change
The man/html sync CI guard requires html/httrack.man.html to change whenever
man/httrack.1 does; the -%F help-text update regenerated the man page but not
its html rendering.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Add {url}{lastmodified}{mime}{charset}{status}{size} footer fields
Extends the named footer set beyond addr/path/date/version. hts_footer_format
now takes a {name,value} table instead of four fixed parameters, so the field
set is data rather than signature and the legacy %s path looks addr/path/date/
version up by name (order-independent, still byte-for-byte). The call site
supplies the new fields from the current response: {url} (scheme + host + path,
credentials stripped like {addr}), {lastmodified}, {mime}, {charset}, {status}
and {size}.
Every network-derived string is html_inline_safe()'d, since the footer sits
inside an HTML comment and a value holding "-->" would otherwise close it and
inject markup (#165); {status}/{size} are formatted integers and need none.
Documented in --help, the man pages and the Command-Line Guide, and covered by
the -#test=footerfmt self-test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The offline crawl harness reads the server's ephemeral port from the "PORT <n>"
line it prints once bound, but only waited 5s (50 x 0.1s) and matched head -n1.
Under `make check -jN` up to 16 Python servers cold-start at once; on a loaded
Windows runner a cold MSYS Python start can lag past 5s, so discovery timed out
with an empty log ("could not discover server port:") and failed 13_local-cookies
spuriously. Give it a 30s deadline, and match the PORT line anywhere so a stray
startup warning merged via 2>&1 can't wedge discovery until timeout.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#R was listed twice: "cache repair" and "old FTP routines". The FTP handler
is commented out in the parser, so the only live meaning is cache repair;
drop the bogus line. #X is a no-op (the parser prints "option has no effect"),
so remove its help line and misleading default marker.
Add help lines for three real but undocumented options: -%z
(--disable-compression), -%t (keep the original file extension), and -y
(--background-on-suspend).
Regenerate man/httrack.1 and html/httrack.man.html from the updated --help.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The offline suite waited on each httrack crawl with a bare wait bounded only by the engine's --max-time; a fetch that wedges past that (a Windows socket stall the engine misses) blocked wait forever and ran the test step to its 45-minute cap. wait_bounded attaches the #595 kill_tree reaper to the crawl pid so an overrun is reaped in seconds, stop_server now reaps the server's native tree, and 72_watchdog-crawl proves the fail-fast against an always-stall endpoint.
The "Select URLs" and "Start" section headings on the webhttrack server
wizard were literal English, bypassing the ${LANG_...} substitution, so
they stayed English in all 30 locales. Reuse existing, already-translated
keys: step3 -> ${LANG_G44} ("Web Addresses: (URL)"), step4 -> ${LANG_J9}
("Start"). No lang/*.txt changes needed.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Correct spelling errors (beginning, dishonest, redistributing, forbid,
personal, chosen, address, occurrences) and refresh stale content: collapse
the obsolete per-Windows-version questions into a single current statement,
drop dead OS keywords, describe the cookies.txt / --cookies-file workflow
instead of the Netscape/IE folder steps, and reword the rtsp note to say
streaming protocols are out of scope rather than "not supported yet".
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Correct several spelling typos in html/abuse.html and html/filters.html,
and point filters.html readers at the --why (-%Y) option, which reports
which filter rule accepted or blocked a given URL.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 2007 cmddoc.html beginner walkthrough is fully subsumed by cmdguide.html.
Repoint index.html's "Command-line version" bullet to httrack.man.html (the
generated reference, previously unlinked) and mark the now-orphaned legacy
options.html as possibly-stale, pointing at the man page and the guide. Also
fixes two long-standing index.html typos (relese, Developper).
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix the download-PDFs example in the command-line guide
The old example ran httrack with no filters, so it mirrored the whole
site rather than downloading its PDFs. The corrected command restricts
to the site, admits the HTML pages as scaffolding for link discovery
(including directory-index pages via *[file]/ so sub-directory PDFs are
found), keeps the PDFs, and drops everything else.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Use *[path]/ so the PDF recipe follows nested directory indexes
*[file] cannot cross a slash, so *[file]/ only admits a single directory level and
silently drops PDFs linked under nested paths (dated archives, doc trees). *[path]/
spans slashes and follows directory-index pages at any depth, with no over-admission
(only trailing-slash URLs match, never files).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
html/httrack.man.html is groff-rendered from man/httrack.1 and committed, but
had drifted far behind the roff page (missing the option table-of-contents and
many options such as --pause and --delayed-type-check). Regenerate it, and make
regen-man-html strip groff's version-stamp and creation-date comments so the
committed file no longer churns across groff versions or rebuilds.
Add a lightweight CI guard: a PR that changes man/httrack.1 must also change
html/httrack.man.html, catching the "regenerated the roff page, forgot the HTML"
case that caused this drift. Rendering needs the full groff html device, so CI
verifies the two move together rather than regenerating in-CI.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a "Filter wildcards" subsection listing the bracket wildcard forms
the matcher (src/htsfilters.c) implements: *, *[file]/*[name], *[path],
*[param], character sets and ranges, literal escapes, size rules, and
the *[] end anchor. It links to the full filters page for the rest.
Added as an h4 under the filters section so section numbering and the
page ToC are untouched.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
src/minizip/ vendors patched minizip as X.orig + X.diff pairs; X.diff is
replayed onto fresh upstream on the next re-sync, so a live edit that skips
X.diff silently reverts then. PR #640 fixed a signed-shift UB in the live
mztools.c (READ_32 now casts each recovered 16-bit half to uLong before the
shift) but never regenerated mztools.c.diff, so a re-sync would drop it.
Regenerate mztools.c.diff from the live file; it reproduces mztools.c
byte-for-byte again. Header date labels keep the existing convention so the
change is content-only. The other four snapshots already match and are untouched.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Non-ASCII single -O drops the hts-cache into a mangled twin directory on Windows
PR #636 fixed the log half of #630 and left the cache for a follow-up. With a
single -O café, path_log holds UTF-8 bytes, but cache_init created hts-cache and
opened new.zip through ANSI calls, so on Windows the cache landed in a second
mangled café-twin directory beside the mirror, and a later --update read the
cache from the wrong place.
This routes every path_log filesystem op in cache_init and the reconcile helpers
through the UTF-8 wrappers (structcheck_utf8, MKDIR, FOPEN, UNLINK, RENAME,
fexist_utf8, fsize_utf8), and opens the cache ZIP through minizip's filefunc
entry points (zipOpen2/unzOpen2) backed by hts_fopen_utf8. On POSIX the wrappers
resolve to the same libc calls, so only Windows changes. The conversion is
all-or-nothing: a directory created UTF-8 but a ZIP opened ANSI would split the
cache, so the whole cache-open path flips together.
The corrupt-cache repair path stays ANSI. mztools' unzRepair takes no filefunc,
so a corrupt cache under a non-ASCII path fails to repair (the site is
re-crawled) rather than forking a twin.
Test 69 now also asserts the cache lands under the café directory through a new
--cache-under-logroot audit. Like the log assertion it only bites on the Windows
CI leg; on POSIX it exercises the code path without reproducing the mojibake.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Cache UTF-8 filefunc must keep 64-bit offsets and the cache dir 0700
The #630 cache-UTF-8 hook built a 32-bit zlib_filefunc_def via
fill_fopen_filefunc and called unzOpen2/zipOpen2. Minizip runs a 32-bit
filefunc through fill_zlib_filefunc64_32_def_from_filefunc32, which NULLs the
64-bit seek so every seek truncates the offset to uLong: on Windows LLP64 a
cache >=4GB hard-fails and >=2GB corrupts. The ANSI code this replaced went
through fill_fopen64_filefunc and never truncated. Keep 64-bit: fill the
zlib_filefunc64_def and override only zopen64_file with the UTF-8 opener, then
call unzOpen2_64/zipOpen2_64.
Also restore the POSIX cache-dir mode: the MKDIR macro creates with
HTS_ACCESS_FOLDER (0755), loosening the 0700 the explicit mkdir used. There is
no UTF-8 mkdir-with-mode, so keep the platform split - UTF-8 MKDIR on Windows
(which ignores the mode), mkdir(HTS_PROTECT_FOLDER) on POSIX.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Tighten the #630 comments
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Strip only the scheme's own default port, not :80 on every scheme
hts_strip_default_port treated 80 as the default for every scheme, so an
explicit :80 on a non-http URL was dropped and the fetch silently moved to that
scheme's real default: https://h:80/x became https://h/x (fetched on 443), ftp
likewise. The reverse gap: a scheme's own default (443 https, 21 ftp) was never
stripped, so https://h:443/x kept a redundant port and would not dedup against
the portless form.
Derive the default from lien's scheme (80 http, 443 https, 21 ftp; 80 when
absent or unknown) and strip only when the port equals that. Same family as
#627/#614.
Closes#638
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Exclude the auth-bypass CodeQL query and pin case-insensitive scheme matching
cpp/user-controlled-bypass models auth-bypass-by-spoofing, but httrack has no
auth or access-control surface; its nearest downstream decision, the +/- crawl
filter, is a mirror boundary, not a security one. Exclude it like the
world-writable query.
The stripport self-test's scheme detection is case-insensitive (strfield/streql)
but nothing pinned it: a case-sensitive rewrite would strip :80 from HTTPS://
unnoticed. Replace the non-discriminating :8443 case (no scheme defaults to 8443,
so it passes for any impl) with HTTPS://h:80, which must keep its port. Note at
scheme_default_port that schemeless/protocol-relative links default to 80.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Tighten the #638 comments
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Force a whole-file refetch when a rejected 206 resume can loop (#581)
Resuming an interrupted download, the engine sends a Range and, if the
server answers an unusable 206 (Content-Range not matching the partial on
disk), drops the partial and retries the whole file. On Windows a hard-killed
first pass leaves a truncated cache that the next pass must repair; the retry
can then re-derive a Range from a partial or temp-ref that outlived the
restart, meet the same unusable 206, and loop until retries run out. The
partial is removed but never refetched, so the file is lost from the mirror.
Carry a refetch-whole signal from the restart-whole decision onto the
requeued link, so the retry's back_add drops any stale temp-ref and skips the
partial/temp-ref resume branches: it sends no Range and GETs the whole file,
regardless of whether a partial survived the restart. On POSIX the restart
already removes both sources, so the retry was already Range-less and behavior
is unchanged; test 71 still recovers the file whole.
back_add gains an internal (hidden, non-exported) parameter; htsblk and
lien_url gain one trailing field each. No exported symbol changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Use hts_boolean and HTS_TRUE/HTS_FALSE for the refetch-whole flag
The #581 fix carried the whole-file refetch flag under three inconsistent
types (short int, char, int) set with bare 1/0. Normalize all three to the
house hts_boolean type and use the HTS_TRUE/HTS_FALSE macros for the literal
sets. No behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Advertise -%N's long option and document the real long forms
The help display appends each option's long alias by looking it up in the
alias table, but it first stripped a trailing N to turn placeholders like
cN into c. That also turned -%N into -%, so --delayed-type-check was never
advertised in --help (nor in the generated man page). Try the flag as-is
first and only strip a trailing N on a miss; -%N now shows its long form,
and cN still resolves to --sockets.
The command-line guide had several rows marked short-only that in fact
have long aliases (I had read them off a stale installed binary): fill in
--pause, --strip-query, --disable-compression, the three --keep-* dedup
opts, and --delayed-type-check. Regenerate man/httrack.1 to match, and
drop the long-dead commented-out -%O chroot help line.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Bound the option-token sscanf to the buffer size
CodeQL flagged the %s read into cmd[32] as an unbounded copy on the line
the previous commit touched. The tokens come from HTTrack's own help
strings, not hostile input, so it was not reachable in practice, but the
copy should be bounded regardless. Limit it to %30s (the buffer holds the
leading '-' plus 30 chars and a NUL).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add a task-oriented command-line guide to the offline docs
The command-line docs so far are the option list and the generated manual
page: exhaustive, but organized by flag, not by task. Newcomers arrive
expecting wget/curl syntax and hit the same walls (only the index came
down, filters that do the opposite of what they read, an update that
deletes files), because the reference answers "what does -X do", not "how
do I do the thing I want".
cmdguide.html is a task-oriented layer on top of the manual page: quick
start, scope, filters, limits, naming, links, identity/login, proxy,
update/cache, scripting, then eleven copy-ready recipes with the one
gotcha each. It foregrounds the defaults that actually surprise people
(the ~100 KB/s rate cap, the -c/-A/-%c security clamps, --update purge
semantics, the depth off-by-one, mime filters running after headers) and
links the manual page for per-option detail. Linked from cmddoc.html and
the index. Every documented default and behavior was checked against the
engine source.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Correct four scope/option descriptions in the guide
Adversarial review against the engine source caught four mislabels:
-d is "same principal domain" and -l is "same TLD" (not "same directory"
and "same domain"); -U goes up only and -B goes both ways (the guide had
-U as up-and-down and -B as "anywhere"); -%M archives the whole mirror
into one index.mht, not each page; and -t HEAD-tests links outside scope
rather than reporting "what a scope would reach". The rest of the guide's
documented behavior verified against code.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Fix the PDF recipe, prefer long options, drop the section rules
Review feedback from Xavier:
- The "grab every PDF" recipe was wrong. `-* +*.pdf` blocks the HTML pages
that carry the PDF links, so the crawl only keeps PDFs linked from the
entry page (verified on a local fixture). There is no PDF-only crawl:
HTTrack finds PDFs by parsing HTML. Reworded to let the site traverse,
with the off-host case handled by a `+host/*.pdf` rule.
- Recipes now use long options throughout. `-%!` in particular is replaced
by `--disable-security-limits`: the bare `!` triggers shell history
expansion and is easy to fumble, and the long name says what it does.
- Dropped the per-section `<hr>` rules; the sibling doc pages don't use
them and the section headings already separate the content.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Key the reference tables by long option name, short in parens
Follows the recipe conversion: every table row now leads with the long
option (--depth, --stay-on-same-domain, ...) and carries the short flag in
parentheses. The guru %-flags that have no long form (-%g, -%j, -%o, -%y,
-%z, -%G, -%t, -%N) stay short. Also switches the remaining inline command
snippets in descriptions to long form (--assume, --structure,
--user-agent "").
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
--help (and the manual page generated from it) claimed the default socket
count is 8 and the default new-connections-per-second is 10. The engine
actually defaults to 4 sockets (htslib.c) and 5 connections/second, which
is also the -%c ceiling. The 8 in the old text is the -c clamp, not the
default, so the line conflated the two.
Correct the two markers to (*c4) and (*%c5) in htshelp.c and regenerate
man/httrack.1 via man/makeman.sh. html/httrack.man.html carries the same
two-token fix applied directly, since regenerating it needs groff's html
device (Debian: full groff, not groff-base); a later full regen on such a
host reproduces the identical output.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The option parser's `case 'K':` had no `break`, so a bare -K (no trailing
digit) fell through into `case 'c':`, hit its no-digit branch, and forced
the socket count back to the default of 4. `-c8 -K` ended up with 4
connections, not 8; the -K rewrite mode quietly overrode an earlier -c.
Only a trailing bare -K triggered it (`-K -c8` and `-Kc8` were fine),
which is why it went unnoticed.
Add the missing break. The regression test drives the observable that
maxsoc has: a socket count above 8 trips the "limited to 8" security
warning in hts-log.txt, so `-c16` warns and `-c16 -K` warns only if the
16 survives. It fails on the pre-fix build and passes after.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The command-line documentation (cmddoc.html, options.html, and the generated httrack.man.html) was unreachable from the documentation index; they only linked each other. Link cmddoc.html into the "How to Use" list so the whole set is reachable, and replace options.html's stale ~2007 option list with a pointer to the generated man page.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Adds html/android.html, a step-by-step guide for the HTTrack Android app (install, project, address, options, run, browse, storage), with screenshots of the shipped app and an options section covering all eleven tabs and the settings that are missing or fixed on Android. Linked from the documentation index between the WinHTTrack/WebHTTrack and Fred Cohen entries.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Bumps the copyright year in the html/ documentation footers (and the inline notice in contact.html) from 2007 to 1998-2026. Footer text only, no content changes; fcguide.html (Fred Cohen, upstream) is left untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Four factual bugs in the shipped html/ help pages: the FAQ's stale "SOCKS? Not yet!" answer (SOCKS5 and HTTP CONNECT proxies have shipped), cache.html's false ">4GiB ZIP64 not supported" claim, two -mime:video/* example rows in filters.html mislabeled "application/", and a compile-breaking fprintf(stder, ...) in plug.html's sample module.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
src/vcpkg.json had no builtin-baseline, so vcpkg resolved OpenSSL/zlib/brotli/zstd in Classic mode from whatever ports tree the build runner happened to sit at: the crypto shipped next to libhttrack.dll was a property of the runner image, not of any commit. Pin the baseline (resolves OpenSSL 3.6.3, zlib 1.3.2#1, what the Windows build already ships) so it becomes deterministic, and add a Dependabot vcpkg entry so the baseline advances on a schedule instead of freezing into a stale, CVE-bearing OpenSSL. The Windows workflow now fetches the pinned baseline before building, read back from the manifest so Dependabot bumps need no workflow edit.
Closes#642
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fold repo-specific conventions from private notes into the tracked
guide: a C conventions section (htssafe.h *t allocators, HTSEXT_API as
the exported ABI surface, Windows-breakable/POSIX-stable ABI split),
the concrete Latin-1 file list behind the byte-safe-edits rule, and
three test gotchas (register NN_*.test in TESTS, installed-binary PATH
shadowing, set -e in new .test scripts).
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#581 reports a Windows-only data loss: an interrupted mirror that resumes into an unusable Content-Range 206 drops its partial and never refetches, where the same sequence recovers the file whole on Linux. The trigger is the damaged cache a hard TerminateProcess leaves behind (MSYS can't signal a native exe), which pass 2 has to repair before resuming.
This does not fix the engine. I couldn't reproduce the failure on Linux, so an engine change would be guesswork. What it adds is a deterministic test that drives the damaged-cache regime, plus an analysis of where the two platforms part.
Test 71 leaves a partial and a temp-ref in pass 1, truncates `new.zip` past its last local entry so the central directory a hard kill never wrote is gone and the repair path runs, then resumes into the hostile 206. On Linux that fires the repair, takes the "unusable range -> restart whole" branch, and recovers the file whole every time, across every damage severity I tried, including a repair that recovers zero entries or fails outright. So the cache repair and the restart-whole logic are not themselves where Linux and Windows differ.
Root cause, as far as I can pin it from Linux: restart-whole doesn't refetch. It removes the partial and the temp-ref, flags `STATUSCODE_NON_FATAL`, and leans on the ordinary retry to requeue the URL. The requeued attempt rebuilds the request from the cache, then the temp-ref, then the on-disk partial. On Linux the removals leave none of those, so the retry is a clean whole-file GET and it succeeds. For the file to be lost, the retried attempt has to send a Range again: a second unusable 206, a second -5, and once the retry budget is spent the partial is already gone. That second Range can only come from a temp-ref or partial that outlived the restart-whole removal, which points at a Windows-specific removal or path effect I can't confirm from here.
For the maintainer: the fragile hinge is that restart-whole depends on a budget-consuming retry that re-derives its Range state from disk. A sturdier fix would make the retried attempt refuse to resume, via a per-link "refetch whole, no Range" flag the request builder honors, so a leftover temp-ref or partial can't re-enter the 206 loop whatever the removal quirk turns out to be. Checking that `UNLINK` and `url_savename_refname_remove` actually succeed on Windows would confirm the mechanism first.
Test 71 carries the same Windows skip as test 48. Lifting that skip should reproduce the failure, and turn the test into the fix's verification.
Refs #581.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
webhttrack splits the raw HTTP POST body into argv and calls hts_main2 directly, but that body is in the web form's declared charset (LANGUAGE_CHARSET: ISO-8859-1, windows-125x, BIG5, gb2312, shift-jis, depending on the language), not UTF-8. The httrack CLI and WinHTTrack both hand the engine UTF-8, which htsname's path budget and htscache's format detection now assume, so a non-ASCII output path or URL from the web UI reached the engine as raw form-charset bytes and put the mirror in a mojibake directory instead of the one the user named.
The command line is now converted from the current LANGUAGE_CHARSET to UTF-8 before the crawl starts, in htsserver.c right before webhttrack_main() where that charset is known via LANGSEL. ASCII and already-UTF-8 input pass through untouched. webhttrack's own argv also gets the Windows hts_argv_utf8 treatment the CLI already has.
This exports hts_convertStringToUTF8 so htsserver (which links the shared library) can reach it: an additive ABI change, new symbol, soname unchanged, alongside the already-exported hts_convertStringSystemToUTF8. Flagging it since it touches the public export set.
Test 68 drives the real htsserver over HTTP, posts a start command whose -O dir is café in ISO-8859-1, and checks the mirror lands under the UTF-8 café directory, not the ISO-8859-1 twin. It fails on master and passes with the fix.
Closes#629
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
unzRepair parses the local file header of a damaged cache zip during
repair. READ_32 combined two int-typed READ_16 halves as
READ_16(adr) | (READ_16((adr)+2) << 16); when the high half has bit 15
set (>= 0x8000), shifting it left by 16 exceeds INT_MAX, which is signed
overflow. UBSan aborts on the CRC/size fields of a header whose high
16-bit word has that bit set. Repair runs on hostile input (a corrupt or
foreign new.zip). Cast the halves to uLong before the shift, matching the
sibling minizip readers in unzip.c and zip.c.
Closes#639
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A single `-O` sets both `path_html` and `path_log`, so `httrack -O café url` sends the logs through `path_log` too. argv is UTF-8, so `path_log` holds UTF-8 bytes, but the two-file log branch in `hts_main_internal()` still created its directory and opened `hts-log.txt`/`hts-err.txt` through the ANSI `structcheck()`/`fopen()`. On Windows those read the bytes as the codepage and dropped the logs into a second, mangled `caf<mojibake>/` directory beside the mirror. This routes that branch through the same UTF-8 wrappers #628 used for the mirror root (`structcheck_utf8`/`FOPEN`/`UNLINK`/`fexist_utf8`), so the logs land under `café/` with the mirror. On a UTF-8 filesystem the wrappers resolve to the same calls, so only Windows changes.
This is the log half of #630. I left the cache half out on purpose: relocating `hts-cache` is a much larger, coupled change, because the cache is written through minizip's own ANSI `fopen` and cleaned up through an ANSI `opendir`, so half-converting it would split or break the cache rather than move it. That wants its own PR, and #630 should stay open for it.
Test 69 mirrors into a single non-ASCII `-O` and asserts the audits read `hts-log.txt` from that directory. Like test 64 it only bites on the Windows CI leg, where the two encodings differ; on Linux the change is a no-op and the whole suite stays green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
HTTrack gives a file a temporary `.delayed` placeholder name while it can't resolve the file type yet, then renames the placeholder once the type is known. The problem is that url_savename appends that marker before it enforces the 236-char path ceiling, and the ceiling cuts the tail of the last path segment, which is exactly where the marker lives. Once the trailing `.delayed` is gone the name no longer matches `IS_DELAYED_EXT`, so back_delayed_rename bails out ("nothing bound to the placeholder name") and the downloaded file is never moved to its final name. It is reachable through #133-style deep paths whose final segment is a long hashed filename.
The fix reserves the trailing `.<id>.delayed` in the last-segment copy loop and trims the head of the segment instead, so the result still fits under the ceiling. The hex-id scan stops at its dot separator, so a wholly-hex hashed base is never mistaken for the collision tag and pulled into the marker.
Test 67 drives the naming path over a deep, over-long delayed URL and checks that the marker survives the cut. It fails on master and passes with the fix.
Closes#623
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The test matched engine output with `echo "$out" | grep -q PATTERN`. Under
`set -o pipefail`, grep -q exits the moment it matches, so echo takes a SIGPIPE
writing the rest and the pipeline reports failure even though the pattern was
found. The `|| { echo FAIL; exit 1; }` guard then fired on a passing case,
turning it into an intermittent, output-size-dependent failure (seen on the
Debian buildd leg of #632).
Match with here-strings instead, dropping the pipe and the race entirely.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The link-rewrite step that drops a redundant :80 read the port with a
hand-rolled digit accumulator, then skipped a hardcoded 3 chars (":80")
when the value equaled 80. Any longer spelling that still evaluates to
80 lost only its first 3 chars and glued the rest onto the host:
http://127.0.0.1:080/x became 127.0.0.10/x, :0080 became 127.0.0.180.
The accumulator was also unchecked, so :4294967376 wrapped to 80 and
took the same path (#614 shape).
Extract the strip into hts_strip_default_port() and parse the matched
digits with the range-checked hts_parse_url_port(): a wrapped or
out-of-range value no longer aliases 80, and a genuine default is
dropped by its full matched length instead of 3. Covered by the
"stripport" engine self-test (tests/66_engine-port80-strip.test).
Closes#627
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bump the version in configure.ac, htsglobal.h and the Windows version
resource, and move VERSION_INFO to 3:5:0. The installed struct layouts and
the exported symbol set are unchanged since 3.49.12, so the soname stays
libhttrack.so.3 and no Debian package rename is needed.
The release is a large one: SOCKS5 and CONNECT proxy support, brotli and
zstd content decoding, a family of --update data-loss fixes, 2 GB+ file
handling on Windows and 32-bit builds, and the Windows toolchain port to
VS2022 with its own CI test suite.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Range-check the listen and ftp:// ports, not just the crawled-URL one
The --port listen port, proxytrack's host:port arguments and an ftp:// URL
port all took their value from sscanf("%d"), which range-checks nothing and
is undefined past INT_MAX (glibc wraps). "--port 4295009395" listened on
42099, and proxytrack, which had no range check at all, listened on port 80
for ":65616".
Reuse hts_parse_url_port rather than range-check after the fact. It moves to
its own file: htsserver cannot see a hidden libhttrack symbol and proxytrack
does not link the library at all, so neither could reach it in htslib.c.
htsweb's --port and proxytrack's arguments are operator input for a listen
socket, so both refuse a bad value at startup through the error path each
already had, rather than falling back to a default: silently listening
somewhere the operator did not name is the bug, and a default would keep the
same wrong behavior. The ftp:// port is a URL port and follows the
crawled-URL policy, refusing the link.
--port 0 (which asked for the auto-pick) and 65535 (refused by an off-by-one
">= 65535") both now mean what they say.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Wire htsurlport into the MSVC projects and make test 64 portable
The new htsurlport.c was added to the autotools targets but not the MSVC
projects, so libhttrack.dll, proxytrack.exe and webhttrack.exe failed to
link on Windows with hts_parse_url_port unresolved. Add it to the three
.vcxproj that compile a caller, mirroring the autotools split.
Test 64 drove the listen servers with timeout(1), which is absent on macOS,
so every assertion tripped over a missing command. Source testlib.sh and use
its run_with_timeout instead, the same helper the other blocking-server
tests use. The htsserver and proxytrack listeners block once a port is
accepted, so both need the bound regardless.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Since argv became UTF-8 through hts_argv_utf8(), opt->path_html is already
UTF-8, but hts_main_internal() still re-encoded it with
hts_convertStringSystemToUTF8(), reinterpreting the bytes as the ANSI codepage.
path_html_utf8 prefixes every saved file, so a non-ASCII -O path sent the whole
mirror into a double-encoded directory. Collapse the Windows branch to the
plain UTF-8 copy the POSIX side already used, and create the mirror root with
structcheck_utf8() so the pre-created directory matches where files are written.
On a UTF-8 filesystem both resolve to the same mkdir, so only Windows changes.
This fixes the CLI and the WinHTTrack GUI, which both hand the engine UTF-8 argv.
A local-crawl test mirrors into a non-ASCII -O directory and asserts the files
land there; it runs on the Windows CI leg where the bug is reachable.
Two adjacent gaps stay open with their own issues: a single-directory -O still
leaves a mangled path_log twin holding logs and cache (#630), and webhttrack
never converts its argv to UTF-8 (#629).
Closes#621
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PR #619 fixed hts_gethome() for an exported-but-empty $HOME, where getenv()
returns "" rather than NULL and the old NULL-only check let "~/foo" collapse
into the absolute "/foo". The same shape lives in two static clones that the
fix missed; htsserver's is live.
With HOME= exported, htsserver renders /websites as the default base path,
reads /.httrack.ini on the first request of every session, and writes
/.httrack.ini when a mirror starts. Running as root that last one lands a file
at the filesystem root; otherwise it silently fails on EACCES.
The clones cannot just call hts_gethome(): it is hidden by -fvisibility=hidden
(nm reports it as a local "t" in libhttrack.so), so htsserver cannot reach it
even though it links the library, and proxytrack does not link libhttrack at
all. Exporting it would grow the public ABI for no other caller, so each copy
keeps its own body. proxytrack's is dead today and fixed anyway: this bug got a
second round precisely because one copy kept the old shape.
Test 63 drives htsserver and reads the base path back out of the rendered UI,
since a static in htsserver.h is out of reach of the -#test registry. It pins a
real HOME too, or the empty case would also pass against a gethomedir() that
ignored the environment and always answered ".".
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
hts_buildtopindex() works in the system charset, but verif_backblue(), which
writes backblue.gif and fade.gif, expects UTF-8, and every other caller already
hands it the UTF-8 path. Only this one passed a system-charset path, so on
Windows the gifs were written into a mangled twin directory instead of the
project. Convert at the call site, guarded by _WIN32 since the converter is
Windows-only and the system charset is UTF-8 everywhere else.
Adds a topindex self-test (driven from the offline suite, so it runs on the
Windows leg where the bug is reachable) that asserts the gifs land in the
project directory. It failed on both Windows legs before the fix.
Sibling #216, the mojibake project name in the generated index, is a separate
leak in the same function and is not addressed here.
Closes#217
Refs #216
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Refuse a URL whose port is outside 1..65535
newhttp_addr() parsed a crawled URL's port with sscanf("%d") and validated it
with "i != -1", then cast it to unsigned short. An out-of-range port therefore
folded into 0..65535 and httrack connected there: http://host:105579/ mirrored
off port 40043, storing the result under host_105579 while the bytes came from
somewhere else. A value past INT_MAX is also UB, and wraps on glibc.
What carries the weight is that a user's port exclusion filter stops describing
the port httrack opens, since the filter sees the literal spelling and the
connect uses the folded one.
Parse it with hts_parse_url_port() instead, factored out of #610's
parse_proxy_port(), and refuse the link through newhttp_addr()'s existing
"msg + INVALID_SOCKET" path. A proxy port can fall back to the scheme default;
a URL port cannot, because the URL named one and named a nonsense one.
socks5_handshake_stream() had the same shape, range-checking only after the
sscanf had already wrapped a huge value into a plausible port; it now shares
the parser and its existing socks5_fail() rejection.
Covered by the dns self-test, which drives newhttp_addr() through the mocked
resolver: *addr_count separates a refused port (0, no resolve) from one merely
truncated or defaulted (2), so a "fall back to 80" fix would not pass.
Closes#614
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Keep accepting an empty "host:" port, which means the default
WHATWG and curl both treat an empty port as valid and meaning the scheme
default, and httrack's URL parser hands "http://host:/x" to newhttp_addr() as
adr="host:", so refusing it as an unparseable port would drop a legal link that
master crawls today.
Guard the parse on a non-empty port text, and move the case to the accepted
side of the self-test. Removing the guard fails that test, so the case is
covered rather than merely asserted.
The SOCKS5 origin path already refused an empty port before this series and
still does; that predates #614 and is left alone.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Cover the SOCKS5 origin port in the self-test, and trim comments
Reverting htsproxy.c alone to the wrap-then-check version passed the whole
suite, so nothing pinned that half of the fix and a later refactor could have
dropped it silently. The scripted SOCKS5 harness now asserts a bad origin port
is refused with no byte sent.
65616 would not have tested anything: it fits an int, so the old range check
already caught it. The case the old check could not see is 4294967376, which
overflowed the sscanf("%d") into a plausible 80 and passed; the trailing-junk
and signed spellings sscanf also accepted are covered alongside it. Against
master's htsproxy.c the new assertion fails.
Also note at the SOCKS5 call site that an empty "host:" stays refused there
while the direct path accepts it, so the asymmetry reads as intended rather
than as an oversight, and halve two comments.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The SOCKS5 test server read the sub-negotiation version byte and dropped it. That byte must be 0x01, RFC 1929's own version, not the 0x05 of the SOCKS5 greeting. `htsproxy.c` gets it right and even comments the trap, but since the server never looked, a client that regressed to 0x05 would have kept `52_local-socks5.test` green forever while every real proxy rejected the handshake.
The server now rejects a mismatch the way a real proxy does, and the test names it, so a bad version byte reads as itself rather than as the vaguer "origin not downloaded".
This turned up while checking the proxy client against implementations we didn't write (microsocks, squid, tinyproxy, tor) over the live site. The client is correct on every leg, but this test could not have told us that. Verified by mutation: with `htsproxy.c` patched to send 0x05, master's test passes and the fixed one fails.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Restore ~/ expansion in the -O base path
expand_home() has never fired since 660b569 (3.41.2) refactored it from char*
to String: the correct str[0] == '~' became StringSub(*str, 1) == '~', but the
body still skips a single leading character. "~/foo"[1] is '/', so the guard
never matches, while any path whose *second* char is '~' silently lost its
first character and gained $HOME ("a~/x" -> "/home/smith~/x").
Index from 0 and gate on a following '/' so ~user/ is left alone rather than
mangled into $HOME + "user/" -- resolving it needs getpwnam, which this does
not do. An empty $HOME now falls back to '.' like an unset one, so ~/foo can
no longer expand to the absolute /foo, and an oversized $HOME leaves the path
untouched instead of aborting inside strcatbuff.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Pin the expandhome test to a $HOME that MSYS leaves alone
The Win32 leg spawns a native httrack.exe, and MSYS rewrites a POSIX absolute
$HOME into a Windows path on the way in, so the engine read
C:\Program Files\Git\home\smith and the pinned /home/smith never matched. A
$HOME with no leading '/' is passed through verbatim; expand_home only
concatenates, so the sentinel exercises the same code.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Restore the clang-format run over the gate condition
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Ask the engine for $HOME rather than pinning it in the expandhome test
MSYS rewrites $HOME into a cwd-relative Windows path before the native
httrack.exe reads it (HTSHOME became D:\a\httrack\httrack\tests\HTSHOME), so
no pinned value survives the Win32 leg. Take the expansion of a bare '~' as
the reference instead, and assert it moved off '~' so the derived cases cannot
go vacuous. The ~user/ and a~/ negatives are literals and stay exact.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Pin what the expandhome test could not distinguish
A hts_gethome() that ignored $HOME and always returned the fallback passed
the whole suite: the empty-HOME cases expect ./foo, which a constant "."
satisfies. That is the same silent no-op the fix exists to remove. Also cover
the oversized-$HOME bail, which had no coverage at all.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Keep the oversized-$HOME case inside Windows' environment limit
70000 chars cannot be set as an environment variable on Windows, so the case
failed the MSVC legs, and $(seq 1 70000) flooded the -x log. The buffer is
2 * HTS_URLMAXSIZE, so 4096 clears it and still kills the no-guard mutant.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Run the oversized-$HOME case only where HOME survives the shell
MSYS rewrites HOME into a path of its own, so the engine never saw the long
value: it expanded ~/foo against the build directory instead of bailing. Gate
the case on the engine reporting the HOME we set; it still runs, and still
fails without the guard, on every platform that passes HOME through.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <xroche@gmail.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix three broken msgids in lang/ and assert the pairing in a test
lang/*.txt are consecutive line pairs, an English msgid then its translation,
and nothing validates that shape. Finnish.txt had ended with a blank line since
2012, which was harmless while it was the last line; #588 appended new strings
after it, so the blank became an interior line and knocked every pair past it
out of phase. All 14 strings #588 added to Finnish, plus the CONNECT proxy one
from 938a873, were keyed off a translation instead of an msgid and rendered as
English. Drop the blank.
Italiano and Portugues-Brasil each carry one msgid that matches nothing in
English.txt ("ISO 9660" for "ISO9660", "Relative URL" for "Relative URI"), so
those two entries never resolved either.
The test is the point: it rebuilds the msgid set from English.txt and checks
every file pairs against it, byte-wise, since each file is in its own legacy
charset. It also checks lang.def's English values, which are the join key into
the msgids. It fails on all three defects before this change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Renumber the lang integrity test to 62
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Note the test's physical-line assumption
The engine continues a msgid ending in \ onto the next line; the test does
not. Latent today: no lang line ends in one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 07:44:19 +02:00
221 changed files with 12203 additions and 2540 deletions
This file lists all changes and fixes that have been made for HTTrack
3.49-14
+ New: WARC/1.1 archive output (--warc), with a sorted CDXJ index (--warc-cdx) and WACZ packaging (--wacz), also available from webhttrack (#668)
+ New: -%F takes named footer fields such as {url}, {lastmodified}, {mime}, {charset} and {status} instead of a fixed layout (#667)
+ New: a command-line guide organized by task ships with the offline documentation (#649)
+ Fixed: on Windows, paths beyond MAX_PATH truncated files, and mirroring into a long or non-ASCII directory silently failed (#133)
+ Fixed: the top index showed mojibake for non-ASCII project names and categories on Windows (#216)
+ Fixed: webhttrack handed the engine the web form's charset rather than UTF-8, so a non-ASCII path mirrored into a mojibake directory (#629)
+ Fixed: a non-ASCII single -O left the logs and the cache in a mangled twin directory on Windows (#630)
+ Fixed: a path-ceiling truncation dropped the .delayed marker, losing the file (#623)
+ Fixed: an oversized -%F footer aborted the crawl instead of being skipped (#669)
+ Fixed: a rejected 206 resume could loop and lose the file rather than refetch it whole (#581)
+ Fixed: default-port stripping was scheme-blind and dropped explicit ports from https and ftp URLs, and a :80 written with leading zeros mangled the host (#627, #638)
+ Fixed: -K silently reset the -c socket count (#650)
+ Fixed: signed-shift undefined behaviour in the zip-repair local-header read (#639)
+ Changed: the offline documentation drops stale facts, gains an Android help page, and documents the filter wildcards and the real long option forms
+ Changed: multiple internal hardening, test and CI improvements
3.49-13
+ New: SOCKS5 proxy support, with scheme-aware -P URLs (socks5://, socks5h://, connect://) and plain HTTP tunneled through a CONNECT-only proxy (#563, #564)
+ New: decode brotli and zstd content codings, advertised over TLS only as browsers do (#556)
+ New: webhttrack exposes the engine options added since 3.49-2, among them --cookies-file, --pause and --strip-query (#587)
+ Fixed: files of 2 GB or more were mishandled on Windows and on every 32-bit build (#569)
+ Fixed: --update destroyed a good local copy when the re-fetch returned an HTTP error, was aborted by -M/-E, failed to decode, or came in short (#176, #521, #557, #562)
+ Fixed: a self-redirect cookie wall was dropped instead of being re-fetched with the cookie (#15)
+ Fixed: a stalled TLS handshake ignored --timeout, and synchronous DNS resolution could wedge a crawl past --max-time (#607, #613)
+ Fixed: -M metered saved bytes rather than received volume, and overshot its cap under a slow server (#77, #520)
+ Fixed: several network-facing overflows and denial-of-service paths in the Content-Range, chunked-transfer, cookie, filter and ProxyTrack cache parsers
+ Fixed: a failed connect did not fall back to the next address on Windows (#579)
+ Fixed: -P took an out-of-range port as a garbage port, and scanned past an IPv6 literal's closing bracket (#598, #602)
+ Fixed: reject a port outside 1..65535 wherever one is parsed (a crawled URL, the htsserver and proxytrack listen arguments, an ftp:// URL), instead of letting a bare sscanf wrap a huge value into a plausible port and silently use it (#614, #626)
+ Fixed: a configured proxy still resolved and dialed the origin itself (#592)
+ Fixed: ~/ in the -O base path was never expanded (#270)
+ Fixed: a non-ASCII -O output path was double-encoded on Windows once argv became UTF-8 (#621)
+ Fixed: files under a non-ASCII project path were saved to a mangled directory on Windows (#217)
+ Fixed: --build-top-index (-%i) and --protocol (-@i) were taken for the -i continue flag, wiping the URL list and exiting on the usage screen (#615)
+ Fixed: webhttrack ignored LC_ALL/LC_MESSAGES and picked the wrong Chinese and Portuguese (#95)
+ Fixed: webhttrack wrote its base path and httrack.ini to the filesystem root when $HOME was empty (#625)
+ Fixed: crawls on a non-default port were slowed by a per-request pre-resolve (#181)
+ Changed: Windows builds moved to Visual Studio 2022 and OpenSSL 3.x, the VS2008 project files are retired, and the binaries carry a version resource
+ Changed: removed the obsolete Java-applet (.class) parser and the dead SWF module remnants
+ Changed: multiple internal hardening, test and CI improvements (Windows and macOS crawl suites, HTML-parser fuzzing, parallel make check)
3.49-12
+ New: --why explains which filter rule accepts or rejects a given URL, then exits (#505)
+ Fixed: links carrying raw UTF-8 bytes were fetched double-encoded and 404'd (#516)
shows. They might do that because they are connected through expensive modem connection,
or because they would like to consult pages while travelling, or archive sites that may be
removed
one day, make some data mining, comiling information ("if only I could find this
one day, make some data mining, compiling information ("if only I could find this
website I saw one day.."). <br>
There are many good reasons to mirror websites, and this helps many good people.<br>
As a webmaster, you might be interested to use such tools, too: test broken links, move a
@@ -229,7 +229,7 @@ test the webserver response and performances, index it..<br>
Anyway, bandwidth abuse can be a problem. If your site is regularly "clobbered"
by evil downloaders, you have <br>
various solutions. You have radical solutions, and intermediate solutions. I strongly
recomment not to use<br>
recommend not to use<br>
radical solutions, because of the previous remarks (good people often mirror websites).<br>
<br>
In general, for all solutions,<br>
@@ -244,7 +244,7 @@ or, to be extreme: if you unplug the wire, there will be no bandwidth abuse<br>
Good: Will work with good people. Many good people just don't KNOW that they can slow down
a network.<br>
Bad: Will **only** work with good people<br>
How to do: Obvious - place a note, a warning, an article, a draw, a poeme or whatever you
How to do: Obvious - place a note, a warning, an article, a draw, a poem or whatever you
want<br>
<br>
</li><li>Use "robots.txt" file<br>
@@ -266,7 +266,7 @@ Good: Efficient<br>
Bad: Multiple users behind proxies will be slow down, not really easy to setup<br>
How to do: Depends on webserver. Might be done with low-level IP rules (QoS)<br>
<br>
</li><li>Priorize small files, against large files<br>
</li><li>Prioritize small files, against large files<br>
Good: Efficient if large files are the cause of abuse<br>
Bad: Not always efficient<br>
How to do: Depends on the webserver<br>
@@ -283,7 +283,7 @@ How to do: Use routine QoS (fair queuing), or webserver options<br>
<br>
</li><li>Use technical tricks (like javascript) to hide URLs<br>
Good: Efficient<br>
Bad: The most efficient tricks will also cause your website to he heavy, and not
Bad: The most efficient tricks will also cause your website to be heavy, and not
user-friendly (and therefore less attractive, even for surfing users). Remember: clients
or visitors might want to consult offline your website. Advanced users will also be still
able to note the URLs and catch them. Will not work on non-javascript browsers. It will
@@ -335,7 +335,7 @@ Example: Use things like
</li><li>Use technical tricks to temporarily ban IPs<br>
Good: Efficient<br>
Bad: Radical (your site will only be available online for all users), not easy to setup<br>
How to to: Create fake links with "killing" targets<br>
How to do: Create fake links with "killing" targets<br>
Example: Use things like <a href="killme.cgi"><nothing></a>
(again an example in php:)<br>
<tt>
@@ -417,7 +417,7 @@ Example:<br>
</li><li>Another one is to create images of emails<br>
Good: Efficient, does not require javascript<br>
Bad: There is still the problem of the link (mailto:), images are bigger than text, and it can cause problems for blind people (a good solution is use an ALT attribute with the email written like "smith at mycompany dot com")<br>
How to do: Not so obvious of you do not want to create images by yourself<br>
How to do: Not so obvious if you do not want to create images by yourself<br>
Example: (php, Unix)<br>
<tt>
@@ -491,7 +491,7 @@ echo <br>
</li><li>You can also create temporary email aliases, each week, for all users<br>
Good: Efficient, and you can give your real email in your reply-to address<br>
Bad: Temporary emails<br>
How to do: Not so hard todo<br>
How to do: Not so hard todo<br>
Example: (script & php, Unix)<br>
<tt>
@@ -579,7 +579,7 @@ And then, put the email address in your pages through:
<metaname="description"content="How to mirror a website with HTTrack on Android: install the app, create a project, enter the address, run the mirror, and browse the result on your device."/>
<metaname="keywords"content="httrack, HTTrack, android, offline browser, web mirror utility, website mirroring, mobile, Google Play"/>
@@ -173,7 +173,7 @@ There are also specific issues regarding this format:
<ul>
<li>The data in the central directory (such as CD extra field, and CD comments) are not used</li>
<li>The ZIP archive is allowed to contains more than 2^16 files (65535) ; in such case the total number of entries in the 32-bit central directory is 65536 (0xffff), but the presence of the 64-bit central directory is not mandatory</li>
<li>The ZIP archive is allowed to contains more than 2^32 bytes (4GiB) ; in such case the 64-bit central directory must be present <b>(not currently supported)</b></li>
<li>The ZIP archive is allowed to contains more than 2^32 bytes (4GiB) ; in such case the 64-bit central directory is emitted automatically (a single stored entry of 4GiB or more is not supported)</li>
</ul>
<br/>
@@ -282,7 +282,7 @@ Libraries should generally handle this peculiar format, however.
<metaname="description"content="HTTrack is an easy-to-use website mirror utility. It allows you to download a World Wide website from the Internet to a local directory,building recursively all structures, getting html, images, and other files from the server to your computer. Links are rebuiltrelatively so that you can freely browse to the local site (works with any browser). You can mirror several sites together so that you can jump from one toanother. You can, also, update an existing mirror site, or resume an interrupted download. The robot is fully configurable, with an integrated help"/>
<metaname="keywords"content="httrack, HTTRACK, HTTrack, winhttrack, WINHTTRACK, WinHTTrack, offline browser, web mirror utility, aspirateur web, surf offline, web capture, www mirror utility, browse offline, local site builder, website mirroring, aspirateur www, internet grabber, capture de site web, internet tool, hors connexion, unix, dos, windows 95, windows 98, solaris, ibm580, AIX 4.0, HTS, HTGet, web aspirator, web aspirateur, libre, GPL, GNU, free software"/>
<metaname="description"content="HTTrack is an easy-to-use website mirror utility. It allows you to download a World Wide website from the Internet to a local directory,building recursively all structures, getting html, images, and other files from the server to your computer. Links are rebuiltrelatively so that you can freely browse to the local site (works with any browser). You can mirror several sites together so that you can jump from one toanother. You can, also, update an existing mirror site, or resume an interrupted download. The robot is fully configurable, with an integrated help"/>
<metaname="keywords"content="httrack, HTTRACK, HTTrack, winhttrack, WINHTTRACK, WinHTTrack, offline browser, web mirror utility, aspirateur web, surf offline, web capture, www mirror utility, browse offline, local site builder, website mirroring, aspirateur www, internet grabber, capture de site web, internet tool, hors connexion, unix, dos, windows 95, windows 98, solaris, ibm580, AIX 4.0, HTS, HTGet, web aspirator, web aspirateur, libre, GPL, GNU, free software"/>
<p>With no other options HTTrack mirrors that site, stays on the same host, follows
links to any depth, rebuilds them to browse offline, and stores everything under
<tt>mydir</tt>. The same directory also holds the log files and the
<tt>hts-cache/</tt> folder that makes a later update or resume possible.</p>
<p>Two defaults are worth knowing up front, because both catch people out:</p>
<ul>
<li>HTTrack throttles itself to about <b>100 KB/s</b> even when you pass no rate
option. If a mirror feels slow, that is why. See
<ahref="#limits">Limits</a> for how to lift it.</li>
<li>The download proceeds as a well-behaved robot: it identifies itself as
<tt>HTTrack</tt>, obeys <tt>robots.txt</tt>, and sends a Referer with each
request. A site that blocks that behavior needs the levers in
<ahref="#identity">Identity</a>, not brute force.</li>
</ul>
<h3id="scope">2. Scope: how far the crawl reaches</h3>
<p>Scope decides which links HTTrack is even willing to follow, before any filter
you write. Get this right and most "it downloaded too much" or "it only grabbed
the index" problems disappear.</p>
<tableclass="tblRegular tableWidth"border="0">
<trclass="tblHeaderColor"><td><b>Option</b></td><td><b>What it controls</b></td></tr>
<tr><td><tt>--depth (-r)</tt></td><td>Maximum link depth. <b>The start page is level 1</b>, so one level of links out is <tt>-r2</tt>, not <tt>-r1</tt>.</td></tr>
<tr><td><tt>--stay-on-same-address (-a), --stay-on-same-domain (-d), --stay-on-same-tld (-l), --go-everywhere (-e)</tt></td><td>How far off the starting host the crawl may travel: same address (host), same principal domain, same top-level domain (for example .com), or everywhere. The default keeps you on the starting host.</td></tr>
<tr><td><tt>--can-go-down (-D), --can-go-up (-U), --stay-on-same-dir (-S), --can-go-up-and-down (-B)</tt></td><td>Directory travel: down into subdirectories only, up to parent directories only, stay in the same directory, or both up and down.</td></tr>
<tr><td><tt>--near (-n)</tt></td><td>Also fetch non-HTML files "near" a followed link, such as an image linked from a page you kept but hosted elsewhere.</td></tr>
<tr><td><tt>--ext-depth (-%e)</tt></td><td>How many levels of external links to follow once the crawl leaves your scope (default 0).</td></tr>
<tr><td><tt>--test (-t)</tt></td><td>Also HEAD-test links that fall outside the scope, which are normally refused, without downloading them: a way to see what scope is excluding.</td></tr>
</table>
<p>The single most common surprise is "only the home page came down." That is
usually not a scope option at all: it is an off-host redirect. A start URL of
<tt>http://example.com/</tt> that redirects to <tt>https://www.example.com/</tt>
lands you on a different host, and same-host scope stops the crawl there. Start
from the final URL, or add a filter that re-admits the real host (see
<ahref="#filters">Filters</a>). The log will show the redirect.</p>
<p><tt>-n</tt> is the fix for pages that render locally without their images or
stylesheets: it lets HTTrack pull in requisites that sit just outside scope. Note
that its embedded-asset handling (following <tt>img</tt>, <tt>link</tt>,
<tt>script</tt>, <tt>style</tt> and HTML5 <tt>source</tt>/<tt>track</tt> targets
past the normal depth and filter limits) applies only when <tt>-n</tt> is on; it
is not automatic. It can also over-fetch by dragging in a whole external host from
a single link, in which case name the assets you want with a filter instead.</p>
<h3id="filters">3. Filters and scan rules</h3>
<p>Filters are the number-one source of confusion, and also the tool that solves
most scope problems once you understand them. A filter is a rule that accepts
(<tt>+</tt>) or rejects (<tt>-</tt>) URLs by pattern. The sign is mandatory:
<tt>+pattern</tt> adds, <tt>-pattern</tt> removes, and a bare pattern is an error.</p>
<p>The rules that matter:</p>
<ul>
<li><b>Last match wins.</b> Rules are applied in order and the last one that
matches a URL decides its fate. Order your rules from general to specific.</li>
<li><b>Wildcards.</b><tt>*</tt> matches any run of characters;
<tt>*[a-z]</tt>, <tt>*[0-9]</tt> and similar classes match sets. So
<tt>+*.pdf</tt> means "any URL ending in .pdf".</li>
<li><b>Whitelisting.</b> To keep one site and nothing else, deny everything then
re-admit the host: <tt>"-*" "+example.com/*"</tt>. A lone <tt>+</tt> rule only
adds to the default scope; it never restricts.</li>
<li><b>Size rules.</b><tt>*[>100000]</tt> and <tt>*[<1000]</tt> filter by
byte size. Because size is only known once the transfer starts, an oversize file
is fetched partway and then aborted, not skipped for free.</li>
<li><b>mime: rules.</b> A rule like <tt>-mime:video/*</tt> matches the
<tt>Content-Type</tt>. That type is only known after the response headers arrive,
so a mime rule <b>cannot stop a request</b>; it can only abort the body. Use a
URL pattern when you want to avoid the fetch entirely.</li>
</ul>
<p>Quote your filters. Shells treat <tt>*</tt>, <tt>[</tt> and sometimes <tt>+</tt>
specially, so wrap each rule in quotes as shown above. The full pattern language,
with tables for wildcards, size and mime, is in
<ahref="filters.html">the filters page</a>, and the
<ahref="faq.html">FAQ</a> has a worked tutorial.</p>
<p><b>robots.txt.</b> By default HTTrack obeys <tt>robots.txt</tt> (<tt>-s2</tt>).
<tt>-s0</tt> ignores it entirely, <tt>-s1</tt> obeys it but lets one of your
<tt>+</tt> filters override a disallow for a URL you explicitly asked for. Note
that a <tt>403 Forbidden</tt> is a server refusal, not a robots rule: robots
options will not help there. That is an
<ahref="#identity">identity</a> problem.</p>
<h4>Filter wildcards</h4>
<p>Inside a filter pattern, <tt>*</tt> matches any run of characters; a few
bracket forms match narrower sets. The full table, with size and mime rules, is on
<tr><td><tt>*</tt></td><td>any run of characters</td><td><tt>+*.pdf</tt>— any URL ending <tt>.pdf</tt></td></tr>
<tr><td><tt>*[file]</tt>, <tt>*[name]</tt></td><td>one path segment (any char but <tt>/</tt> and <tt>?</tt>)</td><td><tt>example.com/*[file]/</tt>— a directory-index page</td></tr>
<tr><td><tt>*[path]</tt></td><td>a path, slashes allowed (any char but <tt>?</tt>)</td><td><tt>example.com/*[path].zip</tt></td></tr>
<tr><td><tt>*[param]</tt></td><td>an optional query string</td><td><tt>page.html*[param]</tt> matches with or without <tt>?...</tt></td></tr>
<tr><td><tt>*[a,b,c]</tt></td><td>any one character in the set</td><td><tt>*[a,b,c].txt</tt></td></tr>
<tr><td><tt>*[a-z]</tt></td><td>any one character in the range</td><td><tt>img*[0-9].gif</tt></td></tr>
<tr><td><tt>*[\x]</tt></td><td>the literal character x (escapes <tt>* [ ] \</tt>)</td><td><tt>*[\*]</tt> matches a real <tt>*</tt></td></tr>
<tr><td><tt>*[<NN]</tt>, <tt>*[>NN]</tt></td><td>file size in KB below / above NN</td><td><tt>-*.gif*[<5]</tt> skips GIFs under 5 KB</td></tr>
<tr><td><tt>*[]</tt></td><td>end anchor: nothing may follow</td><td><tt>*.html*[]</tt> rejects <tt>i.html?p=1</tt></td></tr>
</table>
<h3id="limits">4. Limits and politeness</h3>
<p>HTTrack ships cautious on purpose: it is easy to hammer a small site by
accident, and the <ahref="abuse.html">abuse page</a> is worth a read. The limits
below let you go faster when you own the target, and slower when you do not.</p>
<tableclass="tblRegular tableWidth"border="0">
<trclass="tblHeaderColor"><td><b>Option</b></td><td><b>What it controls</b></td></tr>
<tr><td><tt>--max-rate (-A)</tt></td><td>Maximum transfer rate in bytes/sec. <b>The default is about 100 KB/s even without this flag.</b> Raise it to go faster.</td></tr>
<tr><td><tt>--sockets (-c)</tt></td><td>Number of parallel connections (default 4). <tt>--tiny</tt>, <tt>--wide</tt> and <tt>--ultrawide</tt> are presets.</td></tr>
<tr><td><tt>--connection-per-second (-%c)</tt></td><td>New connections opened per second (default 5).</td></tr>
<tr><td><tt>--max-size (-M)</tt></td><td>Stop after N bytes <b>received from the network</b> across the whole mirror (this counts what was transferred, not what was saved).</td></tr>
<tr><td><tt>--max-time (-E)</tt></td><td>Stop after N seconds of wall-clock time.</td></tr>
<tr><td><tt>--timeout (-T), --retries (-R), --min-rate (-J), --host-control (-H)</tt></td><td>Idle timeout, retry count, minimum acceptable rate, and host-ban behavior for slow or dead hosts.</td></tr>
<tr><td><tt>--max-pause (-G), --pause (-%G)</tt></td><td>Pause the mirror at N bytes, or pause between files, to spread the load.</td></tr>
</table>
<p><b>The security clamps.</b> To keep an accidental typo from turning into a flood,
HTTrack silently caps a few values: at most 8 connections (<tt>-c</tt>), at most
10 MB/s (<tt>-A</tt>), and at most 5 new connections per second (<tt>-%c</tt>).
Ask for more and you get the ceiling, quietly. The single flag
<tt>--disable-security-limits</tt> lifts all three (the short form <tt>-%!</tt>
also works, but the bare <tt>!</tt> is awkward to type safely in a shell). Use it
only against infrastructure you are allowed to load that hard.</p>
<h3id="names">5. File names and types</h3>
<p>Where local files land, and what they are called, is controlled by the naming
options. This is the second-biggest source of "why did it do that" questions,
usually about a URL like <tt>/article?id=42</tt> or a <tt>.php</tt> page that is
really HTML.</p>
<tableclass="tblRegular tableWidth"border="0">
<trclass="tblHeaderColor"><td><b>Option</b></td><td><b>What it controls</b></td></tr>
<tr><td><tt>--structure (-N)</tt></td><td>The local path and name layout. Presets are numeric, and you can also give a template such as <tt>--structure "%h%p/%n%q.%t"</tt>.</td></tr>
<tr><td><tt>--long-names (-L)</tt></td><td>Long names, 8.3 names, or ISO9660 for CD masters.</td></tr>
<tr><td><tt>--assume (-%A)</tt></td><td>Assume a MIME type for an extension, for example <tt>--assume php=text/html</tt>. This also skips the extra HEAD probe HTTrack would otherwise send to learn the type.</td></tr>
<tr><td><tt>--delayed-type-check (-%N), --cached-delayed-type-check (-%D), --check-type (-u), -%t</tt></td><td>When and how the content type is checked, and whether the original extension is kept.</td></tr>
<tr><td><tt>--include-query-string (-%q), --strip-query (-%g)</tt></td><td>Whether the query string appears in the local filename, and whether query keys are stripped when deciding if two URLs are the same file.</td></tr>
</table>
<p>The <tt>-N</tt> presets are built from modular arithmetic on the name fields, so
undocumented number combinations often "work" by accident. If you care about the
exact layout, use an explicit template (the <tt>%h %p %n %q %t</tt> placeholders)
rather than a magic number, and check the result on a small crawl first.</p>
<p>A dynamic page served as <tt>.php</tt> or <tt>.asp</tt> that is actually HTML is
the classic case: without help it can be saved with an extension a browser will
not open locally. <tt>--assume php=text/html</tt> fixes both the extension and the
naming.</p>
<h3id="links">6. Links and page building</h3>
<p>After a page is downloaded, HTTrack parses it for more links and rewrites the
ones it kept so the local copy browses offline. These options tune both halves.</p>
<tableclass="tblRegular tableWidth"border="0">
<trclass="tblHeaderColor"><td><b>Option</b></td><td><b>What it controls</b></td></tr>
<tr><td><tt>--keep-links (-K)</tt></td><td>How links are rewritten in saved pages. The numbering is inverted from what you might guess: bare <tt>-K</tt> keeps <b>absolute</b> URLs, and <tt>-K0</tt> is the <b>relative</b> default. <tt>-K3</tt> keeps absolute URIs, <tt>-K4</tt> keeps the original links.</td></tr>
<tr><td><tt>--replace-external (-x), --generate-errors (-o)</tt></td><td>Replace external links with an error page, and generate an error page for links that failed.</td></tr>
<tr><td><tt>--preserve (-%p), --disable-passwords (-%x)</tt></td><td>Leave HTML untouched (no rewriting), and strip passwords out of saved links.</td></tr>
<tr><td><tt>--extended-parsing (-%P), --parse-java (-j)</tt></td><td>Aggressive link discovery, and how much script content is parsed for links.</td></tr>
<tr><td><tt>--mime-html (-%M)</tt></td><td>Save the whole mirror as a single MIME-encapsulated <tt>.mht</tt> archive (<tt>index.mht</tt>).</td></tr>
<tr><td><tt>--index (-I), --build-top-index (-%i), --search-index (-%I)</tt></td><td>Build a per-mirror index, a top index across projects, and a searchable keyword index.</td></tr>
</table>
<p>HTTrack finds links by parsing HTML and CSS. It does not run JavaScript, so any
URL a page builds at runtime in script (a lazy-loaded image, a
JavaScript-assembled path) is invisible to the crawler and will be missing from
the mirror. There is no flag that fixes this; the asset has to appear in the
static HTML or CSS to be found. <tt>-%P</tt> widens discovery for links that are
present but awkwardly formatted, not for links that do not exist until script
runs.</p>
<h3id="identity">7. Identity, cookies and login</h3>
<p>By default HTTrack is an honest robot: it sends a <tt>User-Agent</tt> of
<tt>HTTrack</tt>, a Referer with each link (which reveals the crawl path to the
server), and obeys robots. Plenty of sites filter exactly that profile. These
options control what HTTrack says about itself.</p>
<tableclass="tblRegular tableWidth"border="0">
<trclass="tblHeaderColor"><td><b>Option</b></td><td><b>What it controls</b></td></tr>
<tr><td><tt>--user-agent (-F)</tt></td><td>The <tt>User-Agent</tt>. Set a browser string to get past crawler blocks; <tt>--user-agent ""</tt> sends none.</td></tr>
<tt>{lastmodified}</tt> (the page's Last-Modified), <tt>{version}</tt>,
<tt>{mime}</tt>, <tt>{charset}</tt>, <tt>{status}</tt> and <tt>{size}</tt>; write
<tt>{{</tt> or <tt>}}</tt> for a literal brace. A footer that contains <tt>%s</tt>
keeps the older positional form (host, path, date in that order). Example:
<tt>-%F "<!-- Mirrored from {url} on {date} -->"</tt>.</p>
<p><b>Login.</b> For HTTP Basic auth, put the credentials in the URL:
<tt>http://user:pass@host/</tt>. An <tt>@</tt> inside the username must be written
<tt>%40</tt>. Only Basic is supported, not Digest.</p>
<p>For cookie or form logins, the simplest path is to log in with a browser, export
its <tt>cookies.txt</tt>, and drop that file in the project directory so HTTrack
sends the session cookie. For a form that needs a POST, <tt>--catchurl</tt> can
capture the exact request your browser sends and replay it. A few cookie caveats
to know: expiry is ignored, there is a silent cap of about 8 cookies sent per
request, and <tt>-b0</tt> disables cookies and the reuse of Basic credentials
across links at the same time.</p>
<h3id="proxy">8. Proxy and network</h3>
<tableclass="tblRegular tableWidth"border="0">
<trclass="tblHeaderColor"><td><b>Option</b></td><td><b>What it controls</b></td></tr>
<tr><td><tt>--proxy (-P)</tt></td><td>Route through a proxy. HTTP, SOCKS5 and CONNECT are supported: <tt>-P host:8080</tt>, <tt>-P socks5://host:1080</tt>, <tt>-P connect://host:443</tt>, with optional <tt>user:pass@</tt>.</td></tr>
<tr><td><tt>--httpproxy-ftp (-%f)</tt></td><td>Send FTP requests through the HTTP proxy.</td></tr>
<tr><td><tt>--protocol (-@i)</tt></td><td>Prefer IPv4 or IPv6.</td></tr>
<tr><td><tt>--http-10 (-%h), --keep-alive (-%k), --disable-compression (-%z)</tt></td><td>Force HTTP/1.0 (drops keep-alive and compression, useful for fragile CGI), toggle keep-alive, and toggle compression.</td></tr>
<tr><td><tt>--bind (-%b), --tolerant (-%B)</tt></td><td>Bind to a local address, and accept technically-bogus responses some servers send.</td></tr>
host names at the proxy (remote DNS) for both <tt>socks5://</tt> and
<tt>socks5h://</tt>, so your local resolver is never consulted. And HTTrack does
not verify TLS certificates: HTTPS gives you an encrypted transport, but not an
authenticated one. That is a deliberate choice for a mirroring tool, not a bug,
but it is worth knowing if you are relying on it for trust.</p>
<h3id="update">9. Update and cache</h3>
<p>Every project keeps a cache under <tt>hts-cache/</tt>. It records every URL that
was fetched, together with the options you used, and it is what makes resuming and
updating possible. It is not a size-limited scratch area you can delete: throw it
away and you lose the ability to continue or update the mirror.</p>
<tableclass="tblRegular tableWidth"border="0">
<trclass="tblHeaderColor"><td><b>Option</b></td><td><b>What it controls</b></td></tr>
<tr><td><tt>--continue</tt></td><td>Carry on an interrupted mirror, trusting the cache: it does not re-check pages already stored.</td></tr>
<tr><td><tt>--update</tt></td><td>Re-run the mirror, revalidating each page with the server (If-Modified-Since / If-None-Match) and downloading only what changed.</td></tr>
<tr><td><tt>--purge-old=0 (-X0)</tt></td><td>Do not purge. By default an update deletes local files that are no longer part of the mirror; <tt>--purge-old=0</tt> keeps them.</td></tr>
<tr><td><tt>--cache (-C)</tt></td><td>Cache mode. The default already does the right thing and switches to update-checking when it detects an existing mirror.</td></tr>
<tr><td><tt>--debug-cache (-#C), --repair-cache (-#R), --clean</tt></td><td>Inspect the cache, repair its ZIP, and erase cache plus logs.</td></tr>
</table>
<p><b>The purge trap.</b> An <tt>--update</tt> run rebuilds the list of files the
mirror should contain, then deletes any previously-mirrored file that is not on the
new list. This is what keeps a mirror in sync with a shrinking site, but it means a
partial or interrupted update can delete files you meant to keep. If an update might
not complete cleanly, add <tt>-X0</tt> to protect the existing tree, and expect
dynamic pages to look "changed" on every run. See the
<ahref="cache.html">cache page</a> for the details.</p>
<h3id="experts">10. Experts and scripting</h3>
<p>HTTrack is also a scriptable fetch-and-scan tool. These options turn off the
mirror behavior and expose the engine.</p>
<tableclass="tblRegular tableWidth"border="0">
<trclass="tblHeaderColor"><td><b>Option</b></td><td><b>What it controls</b></td></tr>
<tr><td><tt>--get URL</tt></td><td>Fetch a single file and stop. Cache, index, depth, cookies and robots are all off for this mode.</td></tr>
<tr><td><tt>--spider --testlinks --skeleton</tt></td><td>Scan without saving, test links at depth 1, or keep HTML only. Handy for checking a site before a real crawl.</td></tr>
<tr><td><tt>--userdef-cmd (-V)</tt></td><td>Run a shell command on each downloaded file; <tt>$0</tt> is the file path. Good for on-the-fly processing.</td></tr>
<tr><td><tt>--callback (-%W)</tt></td><td>Load an external callback module to hook the engine.</td></tr>
<tr><td><tt>--do-not-log (-Q), --quiet (-q), --verbose (-v), --file-log (-f), --extra-log (-z), --debug-log (-Z)</tt></td><td>Logging: quiet, no questions, verbose on screen, and the various log-to-file levels.</td></tr>
</table>
<p>The <ahref="dev.html">developer page</a> covers the callback API and batch use
in more depth.</p>
<h3id="recipes">11. Recipes</h3>
<p>Copy-ready command lines for the tasks people ask about most. Each has the one
<metaname="description"content="HTTrack is an easy-to-use website mirror utility. It allows you to download a World Wide website from the Internet to a local directory,building recursively all structures, getting html, images, and other files from the server to your computer. Links are rebuiltrelatively so that you can freely browse to the local site (works with any browser). You can mirror several sites together so that you can jump from one toanother. You can, also, update an existing mirror site, or resume an interrupted download. The robot is fully configurable, with an integrated help"/>
<metaname="keywords"content="httrack, HTTRACK, HTTrack, winhttrack, WINHTTRACK, WinHTTrack, offline browser, web mirror utility, aspirateur web, surf offline, web capture, www mirror utility, browse offline, local site builder, website mirroring, aspirateur www, internet grabber, capture de site web, internet tool, hors connexion, unix, dos, windows vista, windows seven, windows 8, solaris, ibm580, AIX 4.0, HTS, HTGet, web aspirator, web aspirateur, libre, GPL, GNU, free software"/>
<metaname="keywords"content="httrack, HTTRACK, HTTrack, winhttrack, WINHTTRACK, WinHTTrack, offline browser, web mirror utility, aspirateur web, surf offline, web capture, www mirror utility, browse offline, local site builder, website mirroring, aspirateur www, internet grabber, capture de site web, internet tool, hors connexion, unix, linux, windows, macos, HTS, HTGet, web aspirator, web aspirateur, libre, GPL, GNU, free software"/>
You have noticed the <tt>-</tt> in the begining of the third rule: this means "refuse links matching the rule"
; and the rule is "any files begining with <tt>www.example.com/gallery/trees/hugetrees/</tt><br>
You have noticed the <tt>-</tt> in the beginning of the third rule: this means "refuse links matching the rule"
; and the rule is "any files beginning with <tt>www.example.com/gallery/trees/hugetrees/</tt><br>
Voila! With these three rules, you have precisely defined what you wanted to capture.<br>
<br>
@@ -361,12 +359,12 @@ You can freely download it, without paying any fees, copy it to your friends, an
There are NO official/authorized resellers, because HTTrack is <b>NOT</b> a commercial product.
But you can be charged for duplication fees, or any other services (example: software CDroms or shareware collections, or fees for maintenance),
but you should have been informed that the software was free software/GPL, and you <b><u>MUST</u></b> have received a copy of the GNU General Public License.
Otherwise this is dishonnest and unfair (ie. selling httrack on ebay without telling that it was a free software is a scam).
Otherwise this is dishonest and unfair (ie. selling httrack on ebay without telling that it was a free software is a scam).
</em>
<br><br><aNAME="QG0b">Q: <strong>Are there any risks of viruses with this software?</strong></a><br>
A: <em>For the software itself:
All official releases (at httrack.com) are checked against all known viruses, and the packaging process is also checked. Archives are stored on Un*x servers, not really concerned by viruses. It has been reported, however, that some rogue freeware sites are embedding free softwares and freewares inside badware installers. Always download httrack from the main site (www.httrack.com), and never from an untrusted source!<br>
All official releases (at httrack.com) are checked against all known viruses, and the packaging process is also checked. Archives are stored on Un*x servers, not really concerned by viruses. It has been reported, however, that some rogue freeware sites are embedding free software and freeware inside badware installers. Always download httrack from the main site (www.httrack.com), and never from an untrusted source!<br>
For files you are downloading on the WWW using HTTrack: You may encounter websites which were corrupted by viruses, and downloading data on these websites might be dangerous if you execute downloaded executables, or if embedded pages contain infected material (as dangerous as if using a regular Browser). Always ensure that websites you are crawling are safe.
(Note: remember that using an antivirus software is a good idea once you are connected to the Internet)</em>
@@ -376,11 +374,8 @@ A: <em>That's right. You can, however, install WinHTTrack on your own machine, a
<br><br><aNAME="QG2">Q: <strong>Where can I find French/other languages documentation?</strong></a><br>
A: <em>Windows interface is available on several languages, but not yet the documentation!</em>
<br><br><aNAME="QG3b">Q: <strong>Is HTTrack working on Windows Vista/Windows Seven/Windows 8 ?</strong></a><br>
A: <em>Yes, it does</em>
<br><br><aNAME="QG3">Q: <strong>Is HTTrack working on Windows 95/98 ?</strong></a><br>
A: <em>No, not anymore. You may try to pick an older release (such as 3.33)</em>
<br><br><aNAME="QG3">Q: <strong>Which systems does HTTrack run on?</strong></a><br>
A: <em>HTTrack runs on current Windows, Linux and other Unix-like systems, and macOS. Very old platforms such as Windows 95/98 are no longer supported; you may try an older release (such as 3.33) on those.</em>
<br><br><aNAME="QG4">Q: <strong>What's the difference between HTTrack, WinHTTrack and WebHTTrack?</strong></a><br>
A: <em>WinHTTrack is the Windows GUI release of HTTrack (with a native graphic shell) and WebHTTrack is the Linux/Posix release of HTTrack (with an html graphic shell)</em>
@@ -394,7 +389,7 @@ A: <em>It should. The <tt>configure.ac</tt> may be modified in some cases, howev
<br><br><aNAME="QG7">Q: <strong>I use HTTrack for professional purpose. What about restrictions/license fee?</strong></a><br>
A: <em>HTTrack is covered by the GNU General Public License (GPL). There is no restrictions using HTTrack for professional purpose,
except if you develop a software which uses HTTrack components (parts of the source, or any other component).
See the <tt>license.txt</tt> file for more information</em>. See also the next question regarding copyright issues when reditributing downloaded material.
See the <tt>license.txt</tt> file for more information</em>. See also the next question regarding copyright issues when redistributing downloaded material.
<br><br><aNAME="QG7b">Q: <strong>Is there any license royalties for distributing a mirror made with HTTrack?</strong></a><br>
A: <em>On the HTTrack side, no. However, sharing, publishing or reusing copyrighted material downloaded from a site requires the authorization of the copyright holders, and possibly paying royalty fees. Always ask the authorization before creating a mirror of a site, even if the site appears to be royalty-free and/or without copyright notice.</em>
@@ -415,7 +410,7 @@ There are several reasons (and solutions) for a mirror to fail. Reading the log
<ul>
<li>Links within the site refers to external links, or links located in another (or upper) directories, not captured by default - the use of filters is generally THE solution, as this is one of the powerful option in HTTrack. <u>See the above questions/answers</u>.</li>
<li>Website <ahref="#Q1b1">'robots.txt' rules</a> forbide access to several website parts - you can disable them, but only with great care!</li>
<li>Website <ahref="#Q1b1">'robots.txt' rules</a> forbid access to several website parts - you can disable them, but only with great care!</li>
<li>HTTrack is filtered (by its default User-agent IDentity) - you can change the Browser User-Agent identity to an anonymous one (MSIE, Netscape..) - here again, use this option with care, as this measure might have been put to avoid some bandwidth abuse (see also the <ahref="abuse.html">abuse faq</a>!)</li>
</ul>
@@ -457,14 +452,14 @@ A: <em>Yes, HTTrack does support (since 3.20 release) ipv6 sites, using A/AAAA e
A: <em>Check the build options (you may have selected user-defined structure with wrong parameters!)</em>
<br><br><aNAME="QT5">Q: <strong>When capturing real audio/video links (.ram), I only get a shortcut!</a></strong></a></br>
A: <em>Yes, but .ra/.rm associated file should be captured together - except if rtsp:// protocol is used (not supported by HTTrack yet), or if proper filters are needed</em>
A: <em>The .ra/.rm associated file can be captured together with the shortcut, if proper filters are set. Streaming protocols such as rtsp:// are out of scope: HTTrack is an HTTP/FTP mirror and does not capture rtsp streams.</em>
<br><br><aNAME="QT6">Q: <strong>Using user:password@address is not working!</a></strong></a></br>
A: <em>Again, first check the <tt>hts-log.txt</tt> and <tt>hts-err.txt</tt> error log files - this can give you precious information<br>
The site may have a different authentication scheme - form based authentication, for example.
In this case, use the URL capture features of HTTrack, it might work.
<br>Note: If your username and/or password contains a '<tt>@</tt>' character, you may have to replace all '<tt>@</tt>'
occurences by '<tt>%40</tt>' so that it can work, such as in <tt>user%40domain.com:foobar@www.foo.com/auth/.
occurrences by '<tt>%40</tt>' so that it can work, such as in <tt>user%40domain.com:foobar@www.foo.com/auth/.
You may have to do the same for all "special" characters like spaces (%20), quotes (%22)..</tt>
</em>
<br><br>
@@ -520,7 +515,7 @@ These rules, stored in a file called robots.txt, are given by the website, to sp
- for example, /cgi-bin or large images files.
They are followed by default by HTTrack, as it is advised. Therefore, you may miss some files that would have been downloaded without
these rules - check in your logs if it is the case:<br>
<tt>Info: Note: due to www.foobar.com remote robots.txt rules, links begining with these path will be forbidden: /cgi-bin/,/images/ (see in the options to disable this)
<tt>Info: Note: due to www.foobar.com remote robots.txt rules, links beginning with these path will be forbidden: /cgi-bin/,/images/ (see in the options to disable this)
</tt>
<br>
If you want to disable them, just change the corresponding option in the option list! (but only disable this option with great care,
@@ -540,7 +535,7 @@ HTTrack must find one. Therefore, two index.html will be produced, one with the
<br>
It might be a good idea to consider that http://www.foobar.com/ and http://www.foobar.com/index.html are the same links, to avoid
duplicate files, isn't it?
NO, because the top index (/) can refer to ANY filename, and if index.html is generally the default name, index.htm can be choosen,
NO, because the top index (/) can refer to ANY filename, and if index.html is generally the default name, index.htm can be chosen,
or index.php3, mydog.jpg, or anything you may imagine. (some webmasters are really crazy)
<br>
<br>
@@ -595,8 +590,7 @@ A: <em>Simply use the <tt>--assume dat=application/x-zip</tt> option
A: <em>You may need cookies! Cookies are specific data (for example, your username or password) that are sent to your browser once
you have logged in certain sites so that you only have to log-in once. For example, after having entered your username in a website, you can
view pages and articles, and the next time you will go to this site, you will not have to re-enter your username/password.<br>
To "merge" your personnal cookies to an HTTrack project, just copy the cookies.txt file from your Netscape folder (or the cookies located into the Temporary Internet Files folder for IE)
into your project folder (or even the HTTrack folder)
To supply your own cookies to an HTTrack project, put a Netscape-format <tt>cookies.txt</tt> file in your project folder, or point HTTrack at one with the <tt>--cookies-file</tt> option. You can export your browser's session cookies to that format with a browser extension.
</a><aNAME="Q3b">Q: <strong>I want to update a site, but it's taking too much time! What's happening?</strong><br>
A: <em>First, HTTrack always tries to minimize the download flow by interrogating the server about the
file changes. But, because HTTrack has to rescan all files from the begining to rebuild the local site structure,
file changes. But, because HTTrack has to rescan all files from the beginning to rebuild the local site structure,
it can take some time.
Besides, some servers are not very smart and always consider that they get newer files, forcing HTTrack to reload them,
even if no changes have been made!
@@ -740,7 +734,7 @@ retransferred.</em><br>
</a><aNAME="Q7">Q: <strong>I just want to retrieve all ZIP files or other files in a web
site/in a page. How do I do it?</strong><br>
A: <em>You can use different methods. You can use the 'get files near a link' option if
files are in a foreign domain. You can use, too, a filter adress: adding <tt>+*.zip</tt>
files are in a foreign domain. You can use, too, a filter address: adding <tt>+*.zip</tt>
in the URL list (or in the filter list) will accept all ZIP files, even if these files are
outside the address. <br>
Example : <tt>httrack www.example.com/someaddress.html +*.zip</tt> will allow
@@ -835,8 +829,8 @@ A: <em>Yes. Use user:password@your_proxy_name as your proxy name (example: <tt>s
<br><br><aNAME="QM8">Q: <strong>Can HTTrack generates HP-UX or ISO9660 compatible files?</strong></a><br>
A: <em>Yes. See the build options (-N, or see the WinHTTrack options)</em>
<br><br><aNAME="QM9">Q: <strong>If there any SOCKS support?</strong></a><br>
A: <em>Not yet!</em>
<br><br><aNAME="QM9">Q: <strong>Is there any SOCKS support?</strong></a><br>
A: <em>Yes. HTTrack supports SOCKS5 and HTTP CONNECT proxies: give the proxy with a scheme prefix, e.g. <tt>-P socks5://host:port</tt> or <tt>-P connect://host:port</tt> (prefix <tt>user:pass@</tt> before the host for authenticated proxies).</em>
<br><br><aNAME="QM10">Q: <strong>What's this hts-cache directory? Can I remove it?</strong></a><br>
A: <em>NO if you want to update the site, because this directory is used by HTTrack for this purpose.
@@ -111,7 +111,7 @@ See also: The <a href="faq.html#VF1">FAQ</a><br>
starts links, the default mode is to mirror these links - i.e. if one of your start page is
www.example.com/test/index.html, all links starting with www.example.com/test/ will be
accepted. But links directly in www.example.com/.. will not be accepted, however, because
they are in a higher strcuture. This prevent HTTrack from mirroring the whole site. (All
they are in a higher structure. This prevent HTTrack from mirroring the whole site. (All
files in structure levels equal or lower than the primary links will be retrieved.)<br>
</i>
<br>
@@ -127,13 +127,15 @@ See also: The <a href="faq.html#VF1">FAQ</a><br>
an authorization filter, like <b><tt>+*.gif</tt></b>. The pattern is a plus (this one: <b><tt>+</tt></b>),
followed by a pattern composed of letters and wildcards (this one: <b><tt>*</tt></b>).
<br><br>
To forbide a family of links, define
To forbid a family of links, define
an authorization filter, like <b><tt>-*.gif</tt></b>. The pattern is a dash (this one: <b><tt>-</tt></b>),
followed by a the same kind of pattern as for the authorization filter.
<br><br>
Example: +*.gif will accept all files finished by .gif<br>
Example: -*.gif will refuse all files finished by .gif<br>
<br>
To see which rule accepted or blocked a given URL, run HTTrack with the <b><tt>--why</tt></b> (<b><tt>-%Y</tt></b>) option, described in <ahref="httrack.man.html#OPTIONS">the manual page</a>.<br>
<br>
<p>
<h4>Scan rules based on size (e.g. accept or refuse files bigger/smaller than a certain size)</h4>
@@ -143,7 +145,7 @@ See also: The <a href="faq.html#VF1">FAQ</a><br>
size to ensure that you won't reach a defined limit.
Example: You may want to accept all files on the domain www.example.com, using '+www.example.com/*',
including gif files inside this domain and outside (eternal images), but not take to large images,
including gif files inside this domain and outside (external images), but not take to large images,
or too small ones (thumbnails)<br>
Excluding gif images smaller than 5KB and images larger than 100KB is therefore a good option;
+www.example.com +*.gif -*.gif*[<5]-*.gif*[>100]
@@ -189,7 +191,7 @@ See also: The <a href="faq.html#VF1">FAQ</a><br>
<br>
Filters are analyzed by HTTrack from the first filter to the last one. The complete URL
name is compared to filters defined by the user or added automatically by HTTrack. <br><br>
A scan rule has an higher priority is it is declared later - hierarchy is important: <br>
A scan rule has a higher priority if it is declared later - hierarchy is important: <br>
<br>
<tableBORDER="1"CELLPADDING="2">
@@ -307,7 +309,7 @@ See also: The <a href="faq.html#VF1">FAQ</a><br>
<br>
Filters are analyzed by HTTrack from the first filter to the last one. The sizes
are compared against scan rules defined by the user.<br><br>
A scan rule has an higher priority is it is declared later - hierarchy is important.<br>
A scan rule has a higher priority if it is declared later - hierarchy is important.<br>
Note: scan rules based on size can be mixed with regular URL patterns<br>
@@ -361,7 +363,7 @@ See also: The <a href="faq.html#VF1">FAQ</a><br>
<br>
Filters are analyzed by HTTrack from the first filter to the last one. The complete MIME
type is compared against scan rules defined by the user.<br><br>
A scan rule has an higher priority is it is declared later - hierarchy is important<br>
A scan rule has a higher priority if it is declared later - hierarchy is important<br>
Note: scan rules based on MIME types can <b>NOT</b> be mixed with regular URL patterns or size patterns within the same rule, but you can use both of them in distinct ones<br>
@@ -387,12 +389,12 @@ See also: The <a href="faq.html#VF1">FAQ</a><br>
<tr>
<tdnowrap><tt>-mime:video/*</tt></td>
<td>This will refuse all video links that were already scheduled for download
(i.e. all other 'application/' link download will be aborted)</td>
<p><strong>Note:</strong> this is a legacy option list and may be out of date.
For the authoritative, always-current reference see <ahref="httrack.man.html">the httrack manual page</a>,
and for a task-oriented walkthrough see the <ahref="cmdguide.html">Command-line Guide</a>.</p>
<ul>
<li>Filters: <ahref="filters.html">how to use them</a></li>
<br><small>Here you can find informations on filters: how to accept all gif files in a mirror, for example</small>
<br><br>
<li>List of options</li>
</ul>
</ul>
<tt>
<pre>
<p>The complete, current list of command-line options is generated from the
program itself, in <ahref="httrack.man.html">the httrack manual page</a>.</p>
w mirror with automatic wizard
This is the default scanning option, the engine automatically scans links according to the default options, and filters defined. It does not prompt a message when a "foreign" link is reached.
<p>WinHTTrack and WebHTTrack offer the same options through their graphical option
panels, described in the <ahref="step9.html">step-by-step guide</a>.</p>
W semi-automatic mirror with help-wizard (asks questions)
This option lets the engine ask the user if a link must be mirrored or not, when a new web has been found.
g just get files (saved in the current directory)
This option forces the engine not to scan the files indicated - i.e. the engine only gets the files indicated.
i continue an interrupted mirror using the cache
This option indicates to the engine that a mirror must be updated or continued.
rN recurse get with limited link depth of N
This option sets the maximum recurse level. Default is infinite (the engine "knows" that it should not go out of current domain)
a stay on the same address
This is the default primary scanning option, the engine does not go out of domains without permissions (filters, for example)
d stay on the same principal domain
This option lets the engine go on all sites that exist on the same principal domain.
Example: a link located at www.example.com that goes to members.example.com will be followed.
l stay on the same location (.com, etc.)
This option lets the engine go on all sites that exist on the same location.
Example: a link located at www.example.com that goes to www.anyotherweb.com will be followed.
Warning: this is a potentially dangerous option, limit the recurse depth with r option.
e go everywhere on the web
This option lets the engine go on any sites.
Example: a link located at www.example.com that goes to www.anyotherweb.org will be followed.
Warning: this is a potentially dangerous option, limit the recurse depth with r option.
n get non-html files 'near' an html file (ex: an image located outside)
This option lets the engine catch all files that have references on a page, but that exist outside the web site.
Example: List of ZIP files links on a page.
t test all URLs (even forbidden ones)
This option lets the engine test all links that are not caught.
Example: to test broken links in a site
x replace external html links by error pages
This option tells the engine to rewrite all links not taken into warning pages.
Example: to browse offline a site, and to warn people that they must be online if they click to external links.
sN follow robots.txt and meta robots tags
This option sets the way the engine treats "robots.txt" files. This file is often set by webmasters to avoir cgi-bin directories, or other irrevelant pages.
Values:
s0 Do not take robots.txt rules
s1 Follow rules, if compatible with internal filters
s2 Always follow site's rules
bN accept cookies in cookies.txt
This option activates or unactivates the cookie
b0 do not accept cookies
b1 accept cookies
S stay on the same directory
This option asks the engine to stay on the same folder level.
Example: A link in /index.html that points to /sub/other.html will not be followed
D can only go down into subdirs
This is the default option, the engine can go everywhere on the same directoy, or in lower structures
U can only go to upper directories
This option asks the engine to stay on the same folder level or in upper structures
B can both go up&down into the directory structure
This option lets the engine to go in any directory level
Y mirror ALL links located in the first level pages (mirror links)
This option is activated for the links typed in the command line
Example: if you have a list of web sites in www.asitelist.com/index.html, then all these sites will be mirrored
NN name conversion type (0 *original structure 1,2,3 html/data in one directory)
N0 Site-structure (default)
N1 Html in web/, images/other files in web/images/
N2 Html in web/html, images/other in web/images
N3 Html in web/, images/other in web/
N4 Html in web/, images/other in web/xxx, where xxx is the file extension (all gif will be placed onto web/gif, for example)
N5 Images/other in web/xxx and Html in web/html
N99 All files in web/, with random names (gadget !)
N100 Site-structure, without www.domain.xxx/
N101 Identical to N1 except that "web" is replaced by the site's name
N102 Identical to N2 except that "web" is replaced by the site's name
N103 Identical to N3 except that "web" is replaced by the site's name
N104 Identical to N4 except that "web" is replaced by the site's name
N105 Identical to N5 except that "web" is replaced by the site's name
N199 Identical to N99 except that "web" is replaced by the site's name
N1001 Identical to N1 except that there is no "web" directory
N1002 Identical to N2 except that there is no "web" directory
N1003 Identical to N3 except that there is no "web" directory (option set for g option)
N1004 Identical to N4 except that there is no "web" directory
N1005 Identical to N5 except that there is no "web" directory
N1099 Identical to N99 except that there is no "web" directory
LN long names
L0 Filenames and directory names are limited to 8 characters + 3 for extension
L1 No restrictions (default)
K keep original links (e.g. http://www.adr/link) (K0 *relative link)
This option has only been kept for compatibility reasons
pN priority mode:
p0 just scan, don't save anything (for checking links)
p1 save only html files
p2 save only non html files
p3 save all files
p7 get html files before, then treat other files
cN number of multiple connections (*c8)
Set the numer of multiple simultaneous connections
O path for mirror/logfiles+cache (-O path_mirror[,path_cache_and_logfiles])
This option define the path for mirror and log files
Example: -P "/user/webs","/user/logs"
P proxy use (-P proxy:port or -P user:pass@proxy:port)
This option define the proxy used in this mirror
Example: -P proxy.myhost.com:8080
F user-agent field (-F \"user-agent name\
This option define the user-agent field
Example: -F "Mozilla/4.5 (compatible; HTTrack 1.2x; Windows 98)"
mN maximum file length for a non-html file
This option define the maximum size for non-html files
Example: -m100000
mN,N' for non html (N) and html (N')
This option define the maximum size for non-html files and html-files
Example: -m100000,250000
MN maximum overall size that can be uploaded/scanned
This option define the maximum amount of bytes that can be downloaded
Example: -M1000000
EN maximum mirror time in seconds (60=1 minute, 3600=1 hour)
This option define the maximum time that the mirror can last
Example: -E3600
AN maximum transfer rate in bytes/seconds (1000=1kb/s max)
This option define the maximum transfer rate
Example: -A2000
GN pause transfer if N bytes reached, and wait until lock file is deleted
This option asks the engine to pause every time N bytes have been transferred, and restarts when the lock file "hts-pause.lock" is being deleted
Example: -G20000000
u check document type if unknown (cgi,asp..)
This option define the way the engine checks the file type
u0 do not check
u1 check but /
u2 check always
RN number of retries, in case of timeout or non-fatal errors (*R0)
This option sets the maximum number of tries that can be processed for a file
o *generate output html file in case of error (404..) (o0 don't generate)
This option define whether the engine has to generate html output file or not if an error occurred
TN timeout, number of seconds after a non-responding link is shutdown
This option define the timeout
Example: -T120
JN traffic jam control, minimum transfert rate (bytes/seconds) tolerated for a link
This option define the minimum transfer rate
Example: -J200
HN host is abandoned if: 0=never, 1=timeout, 2=slow, 3=timeout or slow
This option define whether the engine has to abandon a host if a timeout/"too slow" error occurred
&P extended parsing, attempt to parse all links (even in unknown tags or Javascript)
This option activates the extended parsing, that attempt to find links in unknown Html code/javascript
j *parse scripts (j0 don't parse)
This option defines whether the engine has to parse scripts or not to catch included files
I *make an index (I0 don't make)
This option define whether the engine has to generate an index.html on the top directory
X *delete old files after update (X0 keep delete)
This option define whether the engine has to delete locally, after an update, files that have been deleted in the remote mirror, or that have been excluded
C *create/use a cache for updates and retries (C0 no cache)
This option define whether the engine has to generate a cache for retries and updates or not
k store all files in cache (not useful if files on disk)
This option define whether the engine has to store all files in cache or not
V execute system command after each files ($0 is the filename: -V \"rm \\$0\
This option lets the engine execute a command for each file saved on disk
q quiet mode (no questions)
Do not ask questions (for example, for confirm an option)
Q log quiet mode (no log)
Do not generate log files
v verbose screen mode
Log files are printed in the screen
f *log file mode
Log files are generated into two log files
z extra infos log
Add more informations on log files
Z debug log
Add debug informations on log files
--mirror <URLs> *make a mirror of site(s)
--get <URLs> get the files indicated, do not seek other URLs
--mirrorlinks <URLs> test links in pages (identical to -Y)
--testlinks <URLs> test links in pages
--spider <URLs> spider site(s), to test links (reports Errors & Warnings)
--update <URLs> update a mirror, without confirmation
--skeleton<URLs> make a mirror, but gets only html files
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(notifíquenos fallos o problemas)\r\n\r\nDesarrollo:\r\nInterface (Windows): Xavier Roche\r\nMotor: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Spanish translations to:\r\nJuan Pablo Barrio Lera (Universidad de León)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(notifíquenos fallos o problemas)\r\n\r\nDesarrollo:\r\nInterface (Windows): Xavier Roche\r\nMotor: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Spanish translations to:\r\nJuan Pablo Barrio Lera (Universidad de León)
About WinHTTrack Website Copier
Acerca de...
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Eliminar claves de query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Claves de query string, separadas por comas, que se eliminarán del nombre de los archivos guardados (p. ej. sid,utm_source).
Write a WARC archive of the crawl
Escribir un archivo WARC del rastreo
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Guardar también cada respuesta descargada en un archivo WARC/1.1 ISO-28500, junto a la réplica.
WARC archive name:
Nombre del archivo WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nombre base opcional para el archivo WARC; déjelo en blanco para nombrarlo automáticamente en el directorio de salida.
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Prosíme o podání hlášení o jakýchkoliv chybách)\r\n\r\nVývoj:\r\nRozhraní (Windows): Xavier Roche\r\nPavouk: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche a ostatní, kdo pøispìli\r\nÈeský pøeklad:\r\nAntonín Matìjèík (matejcik@volny.cz)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Prosíme o podání hlášení o jakýchkoliv chybách)\r\n\r\nVývoj:\r\nRozhraní (Windows): Xavier Roche\r\nPavouk: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche a ostatní, kdo pøispìli\r\nÈeský pøeklad:\r\nAntonín Matìjèík (matejcik@volny.cz)
About WinHTTrack Website Copier
O programu WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Odebrat klíèe dotazu:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Klíèe dotazu oddìlené èárkami, které se vynechají z pojmenování ukládaných souborù (napø. sid,utm_source).
Write a WARC archive of the crawl
Zapsat archiv WARC z procházení
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Uložit také každou staženou odpovìï do archivu WARC/1.1 podle ISO-28500 vedle zrcadla.
WARC archive name:
Název archivu WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Volitelný základní název archivu WARC; ponechte prázdné pro automatické pojmenování ve výstupním adresáøi.
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(有任何程序錯誤或問題請聯絡我們)\r\n\r\n開發:\r\n界面設計 (Windows): Xavier Roche\r\n分析引擎: Xavier Roche\r\nJava分析引擎: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\n感謝:\r\nDavid Hing Cheong Hung (DAVEHUNG@mtr.com.hk) 提供翻譯
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(有任何程序錯誤或問題請聯絡我們)\r\n\r\n開發:\r\n界面設計 (Windows): Xavier Roche\r\n分析引擎: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\n感謝:\r\nDavid Hing Cheong Hung (DAVEHUNG@mtr.com.hk) 提供翻譯
About WinHTTrack Website Copier
關於WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
移除查詢鍵:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
以逗號分隔的查詢鍵，將其從儲存檔案的命名中移除 (例如 sid,utm_source)。
Write a WARC archive of the crawl
寫入此次抓取的 WARC 封存檔
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
同時將每個已擷取的回應儲存為 ISO-28500 WARC/1.1 封存檔，置於鏡像網站旁。
WARC archive name:
WARC 封存檔名稱：
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(有任何程序错误或问题请联系我们)\r\n\r\n开发:\r\n界面设计 (Windows): Xavier Roche\r\n解析引擎: Xavier Roche\r\nJava解析引擎: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\n感谢:\r\nRobert Lagadec (rlagadec@yahoo.fr) 提供翻译技巧
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(有任何程序错误或问题请联系我们)\r\n\r\n开发:\r\n界面设计 (Windows): Xavier Roche\r\n解析引擎: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\n感谢:\r\nRobert Lagadec (rlagadec@yahoo.fr) 提供翻译技巧
About WinHTTrack Website Copier
关于WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
剥离查询键:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
用逗号分隔的查询键，将其从保存文件的命名中删除 (例如 sid,utm_source)。
Write a WARC archive of the crawl
写入本次抓取的 WARC 归档
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
同时将每个已获取的响应保存为 ISO-28500 WARC/1.1 归档，置于镜像站点旁边。
WARC archive name:
WARC 归档名称：
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Molimo da nas obavijestite o svim pogreškama i poteškoæama)\r\n\r\nRazvoj:\r\nSuèelje (Windows): Xavier Roche\r\nPauk: Xavier Roche\r\nRazrediRašèlanjivaèaJave: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche i drugi suradnici\r\nPUNO HVALA za prijevodne preporuke upuæujemo:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Molimo da nas obavijestite o svim pogreškama i poteškoæama)\r\n\r\nRazvoj:\r\nSuèelje (Windows): Xavier Roche\r\nPauk: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche i drugi suradnici\r\nPUNO HVALA za prijevodne preporuke upuæujemo:\r\nRobert Lagadec (rlagadec@yahoo.fr)
About WinHTTrack Website Copier
O programu WinHTTrack Website Copier
Please visit our Web page
@@ -958,3 +958,11 @@ Strip query keys:
Ukloniti kljuèeve upita:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Zarezom odvojeni kljuèevi upita koji se izostavljaju iz naziva spremljenih datoteka (npr. sid,utm_source).
Write a WARC archive of the crawl
Zapi¹i WARC arhivu obilaska
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Spremi i svaki preuzeti odgovor u ISO-28500 WARC/1.1 arhivu, uz zrcalo.
WARC archive name:
Naziv WARC arhive:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Neobavezni osnovni naziv WARC arhive; ostavite prazno za automatsko imenovanje u izlaznom direktoriju.
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(finder du fejl eller opstår der problemer så kontakt os venligst)\r\n\r\nUdvikling:\r\nBrugerflade (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche og andre bidragydere\r\nMange tak for dansk oversættelse til:\r\nJesper Bramm (bramm@get2net.dk)\r\nscootergrisen
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(finder du fejl eller opstår der problemer så kontakt os venligst)\r\n\r\nUdvikling:\r\nBrugerflade (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche og andre bidragydere\r\nMange tak for dansk oversættelse til:\r\nJesper Bramm (bramm@get2net.dk)\r\nscootergrisen
About WinHTTrack Website Copier
Om WinHTTrack Website Copier
Please visit our Web page
@@ -1004,3 +1004,11 @@ Strip query keys:
Fjern forespørgselsnøgler:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommaseparerede forespørgselsnøgler, der udelades i navngivningen af gemte filer (f.eks. sid,utm_source).
Write a WARC archive of the crawl
Skriv et WARC-arkiv af gennemsøgningen
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Gem også hvert hentet svar i et ISO-28500 WARC/1.1-arkiv ved siden af spejlet.
WARC archive name:
Navn på WARC-arkiv:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Valgfrit basisnavn til WARC-arkivet; lad feltet stå tomt for automatisk navngivning i outputmappen.
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Bitte benachrichtigen Sie uns über Fehler und Probleme)\r\n\r\nEntwicklung:\r\nOberfläche (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for German translations to:\r\nRainer Klueting (rainer@klueting.de)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Bitte benachrichtigen Sie uns über Fehler und Probleme)\r\n\r\nEntwicklung:\r\nOberfläche (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for German translations to:\r\nRainer Klueting (rainer@klueting.de)
About WinHTTrack Website Copier
Über WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Query-Schlüssel entfernen:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommagetrennte Query-Schlüssel, die bei der Benennung gespeicherter Dateien entfallen (z. B. sid,utm_source).
Write a WARC archive of the crawl
WARC-Archiv des Crawls schreiben
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Jede heruntergeladene Antwort zusätzlich in einem ISO-28500-WARC/1.1-Archiv neben dem Spiegel speichern.
WARC archive name:
Name des WARC-Archivs:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Optionaler Basisname für das WARC-Archiv; leer lassen, um es automatisch im Ausgabeverzeichnis zu benennen.
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Palun teata meile igast veast või probleemist)\r\n\r\nArendajad:\r\nKasutajaliides (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nEestikeelne tõlge: Tõnu Virma
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Palun teata meile igast veast või probleemist)\r\n\r\nArendajad:\r\nKasutajaliides (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nEestikeelne tõlge: Tõnu Virma
About WinHTTrack Website Copier
Info WinHTTrack Website Copier'i kohta
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Eemalda päringuvõtmed:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Komadega eraldatud päringuvõtmed, mis jäetakse salvestatud faili nimest välja (nt sid,utm_source).
Write a WARC archive of the crawl
Kirjuta läbimise WARC-arhiiv
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Salvesta iga alla laaditud vastus ka ISO-28500 WARC/1.1 arhiivi peegli kõrvale.
WARC archive name:
WARC-arhiivi nimi:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
WARC-arhiivi valikuline põhinimi; jäta tühjaks, et see väljundkataloogis automaatselt nimetada.
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
About WinHTTrack Website Copier
About WinHTTrack Website Copier
Please visit our Web page
@@ -1004,3 +1004,11 @@ Strip query keys:
Strip query keys:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Write a WARC archive of the crawl
Write a WARC archive of the crawl
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
WARC archive name:
WARC archive name:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Kerro meille bugeista ja ongelmista)\r\n\r\nKehitys:\r\nKäyttöliittymä (Windows): Xavier Roche\r\nNettirobotti: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C) 1998-2003 Xavier Roche ja muut avustajat\r\nSuomentanut Mika Kähkönen 22.-24.7.2005\r\nmika.kahkonen@mbnet.fi\r\nhttp://koti.mbnet.fi/kahoset
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Kerro meille bugeista ja ongelmista)\r\n\r\nKehitys:\r\nKäyttöliittymä (Windows): Xavier Roche\r\nNettirobotti: Xavier Roche\r\n\r\n(C) 1998-2003 Xavier Roche ja muut avustajat\r\nSuomentanut Mika Kähkönen 22.-24.7.2005\r\nmika.kahkonen@mbnet.fi\r\nhttp://koti.mbnet.fi/kahoset
About WinHTTrack Website Copier
Tietoja WinHTTrack Website Copier
Please visit our Web page
@@ -930,7 +930,6 @@ Server terminated
Palvelin lopetettu
A fatal error has occurred during this mirror
Tällä peilillä tapahtui vakava virhe
Proxy type:
Välityspalvelimen tyyppi:
Proxy protocol. HTTP: standard proxy. HTTP (CONNECT tunnel): sends every request through a CONNECT tunnel, for CONNECT-only proxies like Tor's HTTPTunnelPort. SOCKS5: default port 1080.
@@ -959,3 +958,11 @@ Strip query keys:
Poista kyselyavaimet:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Pilkuin erotellut kyselyavaimet, jotka jätetään pois tallennettujen tiedostojen nimeämisestä (esim. sid,utm_source).
Write a WARC archive of the crawl
Kirjoita imuroinnin WARC-arkisto
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Tallenna myös jokainen noudettu vastaus ISO-28500 WARC/1.1 -arkistoon peilin viereen.
WARC archive name:
WARC-arkiston nimi:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Valinnainen WARC-arkiston perusnimi; jätä tyhjäksi, jotta se nimetään automaattisesti tulostehakemistoon.
Déconnecter le modem lorsque l'opération sera finie
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(signalez-nous tout bug ou problème)\r\n\r\nDéveloppement:\r\nInterface (Windows): Xavier Roche\r\nMoteur: Xavier Roche\r\nParseurClassesJava: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMERCI pour la relecture à:\r\nRobert Lagadec
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(signalez-nous tout bug ou problème)\r\n\r\nDéveloppement:\r\nInterface (Windows): Xavier Roche\r\nMoteur: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMERCI pour la relecture à:\r\nRobert Lagadec
About WinHTTrack Website Copier
A propos de WinHTTrack Website Copier
Please visit our Web page
@@ -1004,3 +1004,11 @@ Strip query keys:
Supprimer les clés de query string :
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Clés de query string à retirer du nommage des fichiers enregistrés, séparées par des virgules (par ex. sid,utm_source).
Write a WARC archive of the crawl
Écrire une archive WARC du crawl
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Enregistrer aussi chaque réponse téléchargée dans une archive WARC/1.1 (ISO-28500), à côté du miroir.
WARC archive name:
Nom de l'archive WARC :
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nom de base optionnel pour l'archive WARC ; laissez vide pour le générer automatiquement dans le répertoire de sortie.
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Παρακαλώ να μας ενημερώσετε για κάθε πρόβλημα ή bug)\r\n\r\nΑνάπτυξη:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Greek translations to:\r\nMichael Papadakis (mikepap at freemail dot gr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Παρακαλώ να μας ενημερώσετε για κάθε πρόβλημα ή bug)\r\n\r\nΑνάπτυξη:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Greek translations to:\r\nMichael Papadakis (mikepap at freemail dot gr)
About WinHTTrack Website Copier
Σχετικά με το WinHTTrack Website Copier
Please visit our Web page
@@ -958,3 +958,11 @@ Strip query keys:
Αφαίρεση κλειδιών ερωτήματος:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Κλειδιά ερωτήματος χωρισμένα με κόμμα, που θα αφαιρεθούν από την ονομασία των αποθηκευμένων αρχείων (π.χ. sid,utm_source).
Write a WARC archive of the crawl
Εγγραφή αρχείου WARC της ανίχνευσης
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Αποθήκευση κάθε ληφθείσας απόκρισης και σε αρχείο WARC/1.1 ISO-28500, δίπλα στο είδωλο.
WARC archive name:
Όνομα αρχείου WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Προαιρετικό βασικό όνομα για το αρχείο WARC. Αφήστε το κενό για αυτόματη ονομασία στον κατάλογο εξόδου.
@@ -374,7 +374,7 @@ Create error logging and report files
Crea file di log per segnalare errori e informazioni
Generate DOS 8-3 filenames ONLY
Genera solo nomi di file in formato 8.3
Generate ISO9660 filenames ONLY for CDROM medias
Generate ISO9660 filenames ONLY for CDROM medias
Genera nomi di file in formato ISO9660 per i CDROM
Do not create HTML error pages
Non generare pagine d'errore HTML
@@ -634,8 +634,8 @@ Disconnect when finished
Disconnetti alla fine
Disconnect modem on completion
Disconnetti il modem quando il mirror è finito
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(informateci degli errori o problemi)\r\n\rSviluppo:\r\nInterfacccia: Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Italian translations to:\r\nWitold Krakowski (wkrakowski@libero.it)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(informateci degli errori o problemi)\r\n\rSviluppo:\r\nInterfacccia: Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Italian translations to:\r\nWitold Krakowski (wkrakowski@libero.it)
About WinHTTrack Website Copier
Informazioni su WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Rimuovi chiavi della query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Chiavi della query string, separate da virgole, da rimuovere dai nomi dei file salvati (ad es. sid,utm_source).
Write a WARC archive of the crawl
Scrivi un archivio WARC della scansione
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Salva anche ogni risposta scaricata in un archivio WARC/1.1 ISO-28500, accanto al mirror.
WARC archive name:
Nome dell'archivio WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nome di base facoltativo per l'archivio WARC; lascia vuoto per assegnarlo automaticamente nella directory di output.
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(バグまたは問題についてわれわれに知らせてください)\r\n\r\nDevelopment:\r\nInterface(Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nJapanese translation :TAPKAL(nakataka@mars.dti.ne.jp)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(バグまたは問題についてわれわれに知らせてください)\r\n\r\nDevelopment:\r\nInterface(Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nJapanese translation :TAPKAL(nakataka@mars.dti.ne.jp)
About WinHTTrack Website Copier
WinHTTrackについて
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
削除するクエリキー:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Kérjük, jelezzen nekünk bármilyen hibát vagy problémát)\r\n\r\nFejlesztés:\r\nKezelõfelület (Windows): Xavier Roche\r\nIndexelés: Xavier Roche\r\nJavaElemzõOsztályok: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nEZER KÖSZÖNET a magyar fordításért:\r\nHerczeg József Tamásnak (hdodi@freemail.hu)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Kérjük, jelezzen nekünk bármilyen hibát vagy problémát)\r\n\r\nFejlesztés:\r\nKezelõfelület (Windows): Xavier Roche\r\nIndexelés: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nEZER KÖSZÖNET a magyar fordításért:\r\nHerczeg József Tamásnak (hdodi@freemail.hu)
About WinHTTrack Website Copier
WinHTTrack webhely másoló névjegye
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Lekérdezési kulcsok eltávolítása:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Vesszõvel elválasztott lekérdezési kulcsok, amelyeket el kell hagyni a mentett fájl elnevezésébõl (pl. sid,utm_source).
Write a WARC archive of the crawl
A bejárás WARC archívumának írása
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Minden letöltött válasz mentése ISO-28500 WARC/1.1 archívumba is, a tükör mellé.
WARC archive name:
WARC archívum neve:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
A WARC archívum opcionális alapneve; hagyja üresen az automatikus elnevezéshez a kimeneti könyvtárban.
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ons fouten en problemen mede te delen)\r\n\r\nOntwikkeling:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Dutch translations to:\r\nRudi Ferrari (Wyando@netcologne.de)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ons fouten en problemen mede te delen)\r\n\r\nOntwikkeling:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Dutch translations to:\r\nRudi Ferrari (Wyando@netcologne.de)
About WinHTTrack Website Copier
Over WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Query-sleutels verwijderen:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Door komma's gescheiden query-sleutels die bij het benoemen van opgeslagen bestanden worden weggelaten (bijv. sid,utm_source).
Write a WARC archive of the crawl
Schrijf een WARC-archief van de crawl
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Sla ook elke opgehaalde respons op in een ISO-28500 WARC/1.1-archief, naast de mirror.
WARC archive name:
Naam van WARC-archief:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Optionele basisnaam voor het WARC-archief; laat leeg om het automatisch een naam te geven in de uitvoermap.
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Vennligst fortell oss om feil eller problemer med programmet)\r\n\r\nUtvikling (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche og andre bidragsytere\r\n\r\nNorwegian translation:\r\nTobias "Spug" Langhoff ( spug_enigma@hotmail.com )
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Vennligst fortell oss om feil eller problemer med programmet)\r\n\r\nUtvikling (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche og andre bidragsytere\r\n\r\nNorwegian translation:\r\nTobias "Spug" Langhoff ( spug_enigma@hotmail.com )
About WinHTTrack Website Copier
Om WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Fjern spørrenøkler:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommaseparerte spørrenøkler som utelates i navngivingen av lagrede filer (f.eks. sid,utm_source).
Write a WARC archive of the crawl
Skriv et WARC-arkiv av gjennomgangen
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Lagre også hvert nedlastet svar i et ISO-28500 WARC/1.1-arkiv ved siden av speilet.
WARC archive name:
Navn på WARC-arkiv:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Valgfritt basisnavn for WARC-arkivet; la feltet stå tomt for automatisk navngivning i utdatamappen.
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(proszê poinformowaæ nas o jakichkolwiek b³êdach w dzia³aniu programu)\r\n\r\nTwórcy:\r\nInterfejs(Windows): Xavier Roche\r\nMotor: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nPolska wersja jêzykowa: £ukasz Jokiel (Opole University of Technology, Lukasz.Jokiel@po.opole.pl)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(proszê poinformowaæ nas o jakichkolwiek b³êdach w dzia³aniu programu)\r\n\r\nTwórcy:\r\nInterfejs(Windows): Xavier Roche\r\nMotor: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nPolska wersja jêzykowa: £ukasz Jokiel (Opole University of Technology, Lukasz.Jokiel@po.opole.pl)
About WinHTTrack Website Copier
O... WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Usuñ klucze zapytania:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Rozdzielone przecinkami klucze zapytania pomijane przy nazywaniu zapisanych plików (np. sid,utm_source).
Write a WARC archive of the crawl
Zapisz archiwum WARC z indeksowania
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Zapisz te¿ ka¿d± pobran± odpowied¼ do archiwum WARC/1.1 ISO-28500, obok kopii lustrzanej.
WARC archive name:
Nazwa archiwum WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Opcjonalna nazwa bazowa archiwum WARC; pozostaw puste, aby nazwaæ je automatycznie w katalogu wyj¶ciowym.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
