Compare commits

...

36 Commits

Author SHA1 Message Date
Xavier Roche
b90d699d2e Single-file output is MHT, which browsers no longer open
Add --single-file (-%Z): after the mirror completes, rewrite every saved
page in place with its stylesheets, scripts, images and fonts embedded as
data: URIs, while links between pages stay relative. The mirror remains a
browsable tree and each page also stands alone.

This cannot reuse the -%M path: MHT streams a MIME part per file as it is
saved, but a data: URI needs the asset's bytes when the page is written,
and pages are normally saved before their assets are fetched. The new pass
runs over the finished tree at the tail of httpmirror(), after the update
purge.

Audio, video, page-to-page links and anything over --single-file-max-size
(10 MB default) keep an ordinary link. References carrying a scheme are
skipped, which covers data: and makes a second --update run a no-op.
Resolution is clamped to the mirror root: the HTML is hostile input.

htsopt.h gains two tail-appended fields; VERSION_INFO is untouched.

Closes #713

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
2026-07-26 17:45:48 +02:00
Xavier Roche
1027a9f392 htsserver: crawled pages run through the template expander and leak the session id (#706)
* htsserver builds the redirect Location header in a 256-byte stack buffer

The POST redirect path checks strlen(file) but sprintf's newfile, which comes
straight from the client's "redirect" POST field with no length cap. A 300-byte
value overflows tmp[256]. The same value reached the Location header with no
CR/LF check, so it could split the response and inject headers.

Append into the dynamic String the other headers already use, and drop the
header entirely when the value carries a CR or LF.

The listen socket was SOCaddr_initany, so the server answered the LAN and not
just the local browser it exists to serve. Bind 127.0.0.1 by default, with
--bind <addr> to widen it again, resolved through the existing gethost() helper
the way proxytrack already does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: satisfy shellcheck and shfmt in the new server test

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: drop the pre-fix narration from the oversized-value comment

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: assert the bound socket, not the announced URL

The listen-address assertions only compared the URL= banner, which is a
literal echo of argv: a build that announced 127.0.0.1 while binding the
wildcard passed. Probe 127.0.0.2 on the same port instead, which a wildcard
listener takes and a loopback-only one leaves free.

Also refuse an empty --bind, which fell through to every interface and
silently undid the new default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: bound the response read and the empty --bind run

The recv() loop had no timeout and read until EOF; htsserver need not close
the connection after responding, which wedged the macOS runner for over an
hour. Stop at the end of the header block, which is all the test reads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* htsserver: the session id must gate the request body, not the reply

Every field of a POST body is written straight into the one global key store
the templates and the command dispatcher both read, and "command" from there
reaches the engine. The gate ran after that write and compared "sid" against
"_sid" -- but "_sid" is copied into "sid" beforehand so the templates can
render it, so a request that simply omitted the field compared equal to
itself. Only a wrong id was refused; an absent one passed. Clearing the reply
afterwards does not help either, because the dispatcher sits outside the reply
guard.

Authenticate before parsing instead: scan the raw body for "sid", require at
least one occurrence and reject if any of them differs, and drop the body
untouched when it does not match. That leaves the shared template key alone,
and it closes the dispatcher for free.

A refused request also emitted only a Content-length line, since the status
line for that branch was behind _DEBUG. Any client reads that as a protocol
error, which is how test 68 failed rather than reporting the refusal. Send a
403 instead.

Tests 68 and 77 posted without an id, which is what the engine used to accept,
so both now fetch the one the server renders into the form. Test 78 covers
accept, missing, empty and wrong, asserts the 403, and probes the key store
through ${projname} rather than the suppressed reply -- a reply-only assertion
passes even when the write goes through.

Also fix a leak that hung macOS CI: start() runs inside a command
substitution, so its $! never reached the parent and stop() guarded on an
empty variable, leaving one htsserver per call. Test 77 starts four, which is
exactly the four orphans the runner reported while sitting for half an hour
behind a green test log. Take the pid from the PID= line the server already
announces.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* htsserver: serve the crawled mirror verbatim, never through the expander

WebHTTrack serves the mirror under /website/ from the same small server as its
own GUI, and the decision to run a response through the ${...} template
expander was a substring test for ".htm" on the request path. A mirrored page
therefore had its directives evaluated: ${_sid} rendered the live session id,
handing the crawled site the token that authenticates commands on the local
GUI, and ${do:...} gave it the rest of the template verbs.

The /website/ prefix was already detected, but only to keep the crawl-state
override from hijacking a mirror request. Reuse it as the expansion gate, so
expansion is limited to files under the GUI's html root, and re-evaluate it
after that override, which can substitute a GUI page for a mirror path.
Mirrored pages keep their text/html type: verbatim must not turn browsing the
mirror into a download.

Note that mirrored content still shares the control origin, so a script in it
can read the session id from a GUI page itself; that is a separate fix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: cover the running-crawl half of the /website/ override

Test 84 only exercised the idle server, where the override never fires and
the recomputed virtualpath is indistinguishable from the stale one. Drive a
crawl through the server so /website/*.html is rewritten to the GUI refresh
page, which 404s without the recompute.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 16:57:59 +02:00
Xavier Roche
d0a1573a04 webhttrack: "max site size" sets a per-file cap and discards the HTML one (#708)
* webhttrack: "max site size" set a per-file cap instead of the overall one

step4.html mapped all three size fields of the wizard onto --max-files (-m),
so "Max site size" emitted a per-file limit rather than --max-size (-M), and
landed a second -m on the command line. That second -m also clobbered the HTML
per-file limit: a bare -m<n> resets maxfile_html, so whichever of the two came
last won. Point sizemax at --max-size and emit the bare -m before the -m,<n>
form so both per-file caps survive.

Two template typos in the same family, where a malformed ${...} renders as
nothing or as its own key instead of erroring: the winprofile.ini writer's
Dos=${dos was missing its closing brace, and option2b.html's OK button read
${LANG_OK] with a bracket.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* tests: carry the session id in test 81's POST

The gate that landed with #700 refuses a body without one, so the wizard
POST came back refused and the option audit had nothing to read.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: close the confirmation-biased gaps in test 81

Post the size fields empty too, so a step4.html that lost its ${test:} guard
and rendered a valueless --max-files= is caught; assert the winprofile.ini
MaxHtml/MaxOther/MaxAll keys the header claimed to audit; and pin the OK button
label, since an unknown ${LANG_} key renders empty and passed the absence check.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 16:40:40 +02:00
Xavier Roche
4362ac34cb Check snprintf truncation where the result must be complete (#711)
Adds `sprintfbuff()`/`slprintfbuff()` to `htssafe.h`: a formatted print that truncates to fit and returns whether it had to, marked `warn_unused_result` so the answer cannot be dropped. It fills the gap between the `strcpybuff` family, which aborts on overflow, and `String`, which grows without bound. Abort is the wrong contract wherever the text is built from a remote peer's reply.

Four `-Wformat-truncation=` sites used the result as if `snprintf` had never truncated. Two only needed a bigger destination, so they get one: `hts_finish_makeindex`'s `tempo` was a flat 1024 against a 2048-byte escaped URL and is now sized off it, and the wizard's `cmd[4096]` could not hold the answers it concatenates. The other two cannot grow. `create_back_tmpfile` formats `<url_sav>.bak` into a buffer the same size as `url_sav`, and that struct is installed, so a dropped extension would alias the backup onto the live file that `back_finalize_backup()` unlinks. ProxyTrack's `startUrl[1024]` is fed by cache content. Both take the error path they already had, and ProxyTrack moves to the next cache entry rather than publishing a clipped one.

On what the wrapper buys, since it is not what I first assumed: checking a raw `snprintf` return inline silences the warning just as well. The wrapper's value is that the capacity comes from `sizeof`, the check is the default rather than the exception, and `warn_unused_result` makes skipping it visible.

`-#test=strsafe` covers the primitive (exact fit, one over, 4 KB source, `size == 1`, trailing canary, destination repoisoned between cases), and `-#test=makeindex` gains a first link whose escaped form overruns the old buffer. Both were mutation-checked. The ProxyTrack change ships without a direct test: its only observable is the catalog page, which renders solely as a PROPFIND fallback I could not drive from curl. 25 gcc warnings down to 21; the rest of the cluster is diagnostic-only and follows separately.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 16:37:48 +02:00
Xavier Roche
6579436607 webhttrack: the command line overflows its argv vector and a quoted value can inject flags (#710)
* htsserver builds the redirect Location header in a 256-byte stack buffer

The POST redirect path checks strlen(file) but sprintf's newfile, which comes
straight from the client's "redirect" POST field with no length cap. A 300-byte
value overflows tmp[256]. The same value reached the Location header with no
CR/LF check, so it could split the response and inject headers.

Append into the dynamic String the other headers already use, and drop the
header entirely when the value carries a CR or LF.

The listen socket was SOCaddr_initany, so the server answered the LAN and not
just the local browser it exists to serve. Bind 127.0.0.1 by default, with
--bind <addr> to widen it again, resolved through the existing gethost() helper
the way proxytrack already does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: satisfy shellcheck and shfmt in the new server test

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: drop the pre-fix narration from the oversized-value comment

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: assert the bound socket, not the announced URL

The listen-address assertions only compared the URL= banner, which is a
literal echo of argv: a build that announced 127.0.0.1 while binding the
wildcard passed. Probe 127.0.0.2 on the same port instead, which a wildcard
listener takes and a loopback-only one leaves free.

Also refuse an empty --bind, which fell through to every interface and
silently undid the new default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: bound the response read and the empty --bind run

The recv() loop had no timeout and read until EOF; htsserver need not close
the connection after responding, which wedged the macOS runner for over an
hour. Stop at the end of the header block, which is all the test reads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* htsserver: the session id must gate the request body, not the reply

Every field of a POST body is written straight into the one global key store
the templates and the command dispatcher both read, and "command" from there
reaches the engine. The gate ran after that write and compared "sid" against
"_sid" -- but "_sid" is copied into "sid" beforehand so the templates can
render it, so a request that simply omitted the field compared equal to
itself. Only a wrong id was refused; an absent one passed. Clearing the reply
afterwards does not help either, because the dispatcher sits outside the reply
guard.

Authenticate before parsing instead: scan the raw body for "sid", require at
least one occurrence and reject if any of them differs, and drop the body
untouched when it does not match. That leaves the shared template key alone,
and it closes the dispatcher for free.

A refused request also emitted only a Content-length line, since the status
line for that branch was behind _DEBUG. Any client reads that as a protocol
error, which is how test 68 failed rather than reporting the refusal. Send a
403 instead.

Tests 68 and 77 posted without an id, which is what the engine used to accept,
so both now fetch the one the server renders into the form. Test 78 covers
accept, missing, empty and wrong, asserts the 403, and probes the key store
through ${projname} rather than the suppressed reply -- a reply-only assertion
passes even when the write goes through.

Also fix a leak that hung macOS CI: start() runs inside a command
substitution, so its $! never reached the parent and stop() guarded on an
empty variable, leaving one htsserver per call. Test 77 starts four, which is
exactly the four orphans the runner reported while sitting for half an hour
behind a green test log. Take the pid from the PID= line the server already
announces.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* webhttrack: bound the argv vector and escape quotes in the wizard command line

The wizard hands its httrack command line to the engine as one string, which
back_launch_cmd() split back into argv. Two things were wrong with that split.

It wrote into a fixed 1024-pointer vector with no bound, and every unquoted
space in the posted string yields an entry, so an ordinary mirror with a few
hundred URLs walked off the allocation. The split now lives in htscmdline.c as
hts_split_cmdline(), which sizes the vector from the separator count before
filling it, and the engine self-tests can reach it.

Quotes were also purely advisory: they toggled the "inside an argument" state
but nothing escaped them, so a double quote typed into a wizard field (user
agent, footer, path, project name) closed the argument early and the rest of
the value was parsed as fresh options -- among them -V, which reaches system().
Escaping has to happen where the argument boundary is known, so the template
gets its own ${arg:} filter for that context; ${html:} keeps its meaning for
the HTML attributes it is used in everywhere else, and HTML escaping would not
help anyway since the browser undoes it when it posts the command line back.
${arg:} backslash-escapes a quote and a backslash, and the splitter reads those
inside a quoted run, the same convention next_token() already implements for
doit.log. A value containing a quote now survives it intact instead of turning
into options.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* webhttrack: keep a stray quote in an unquoted field out of the split

The url and wildcard-filter fields go into the command line outside quotes,
where no backslash can escape anything: a single quote there flips the parity
of every quote after it, so a later escaped value ends up split as flags and
the escaping buys nothing. Emit %22 for those fields instead.

NULL-terminate the argv vector while here, matching the convention the tree
documents in htscharset.c, and fold the third copy of the entity table into
one helper.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: do not feed snprintf's return value back as its size argument

snprintf returns the length it wanted to write, so accumulating it blind
lets the next size argument wrap. The buffer is sized well past what the
loop needs, but the pattern is the one the project forbids.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 16:26:55 +02:00
Xavier Roche
cb98b1b197 Fatal-signal backtraces are unreadable: every engine frame is a bare module+offset (#705)
* Symbolize fatal-signal backtraces through addr2line

backtrace_symbols_fd() resolves names from .dynsym only, and
-fvisibility=hidden keeps every engine frame out of it, so a crash report
arrived as a column of bare module+offset. The handler now emits the raw trace
first and unconditionally, then groups the frames per module and runs addr2line
(or llvm-symbolizer) over the offsets, which reads DWARF and names the static
frames plus their inline chain.

-rdynamic is dropped: it only populated .dynsym and bought exactly one named
frame. -Wl,--build-id replaces it, so a trace from a stripped build can be
matched to its debug symbols.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Move the crash backtrace printer into src/htsbacktrace.c

httrack.c keeps only the two call sites. The symbolizer needs _GNU_SOURCE for
dladdr(), which is now confined to its own translation unit instead of being
forced on the whole CLI front-end.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Test: match glibc's backtrace format with or without the space

backtrace_symbols_fd() prints the trailing "[0xADDR]" with a leading space on
some glibc versions and without on others (Ubuntu 24.04), so the raw-frame
assertion failed everywhere but the dev box. Match only up to the offset.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Skip pseudo-modules with no file on disk

A frame in linux-vdso.so.1 made addr2line complain instead of resolving, so
the arm64 leg lost its symbolized output entirely. Renumber the test too:
77 landed on master with #700.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Stop discarding local symbols, which made the names wrong

--discard-all drops the local symbol of every static function, so addr2line
attributes the frame to the nearest surviving global: the abort frame read
dns_timeout_selftests instead of abortf_, with the file and line still right.
A wrong name is worse than none, and because the symbols go at link time no
-dbgsym package can recover them. Costs 21784 bytes on libhttrack.so.3, 0.6%.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Symbolize once when the handler itself faults

A fault inside the handler re-enters the printer, which interleaved a second
symbolized trace on the same fd and spent a second budget: 3.04s and two
overlapping reports, measured.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <xroche@gmail.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 15:43:17 +02:00
Xavier Roche
da8fbfff49 Fix dead array-address checks and discarded-qualifier casts (-Waddress, -Wcast-qual) (#703)
* Drop NULL tests on inline array members

`lien_back::url_sav`, `htsblk::msg` and POSIX `dirent::d_name` are arrays,
so testing their address folds to a constant and gcc/clang report it
(-Waddress, -Wpointer-bool-conversion). Every site keeps whatever real
condition sat beside the dead one, so behavior is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* help_wizard: check the allocation, not the arrays it contains

The out-of-memory guard has been constant-false since d593418 folded the
nine separate wizard buffers into one struct: the names it tests are now
inline arrays, so `malloct()`'s result is never checked and an exhausted
heap gets a NULL-page write instead of the intended message. Also switch
the raw free() to freet() and release the struct on the two early returns
that leaked it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* proxytrack: test the WebDAV header fields, not their addresses

`PT_Element::lastmodified` and `::contenttype` are inline arrays, so both
`if`s were constant-true (-Waddress); use the `[0]` form the same file
already uses when it emits the GET headers. Neither is observable:
get_time_rfc822("") returns 0 and falls through to the index timestamp,
and proxytrack_add_DAV_Item already substitutes application/octet-stream
for an empty mime, which a PROPFIND probe against a cache entry carrying
no Content-Type confirms both before and after.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Fix the discarded const qualifiers rather than casting them away

`binput` and `cache_binput` only ever read through their source pointer,
so they take `const char *` now; that alone clears the cast in
htsrobots.c, and neither is exported nor declared in an installed header,
so no ABI question arises. `treathead` keeps `char *rcvd` because it does
NUL-cut the header in place, and the two selftest calls that fed it a
string literal get a mutable buffer instead, matching their three
siblings and removing a latent write to .rodata. The remaining two are
one-liners: zlib's `next_in` is already `const` under -DZLIB_CONST, and
htsback can call the non-const `jump_protocol` twin on its mutable
`url_adr`. libhttrack.vcxproj gains ZLIB_CONST so the MSVC build agrees
with autotools, as webhttrack and proxytrack already do.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: prove the WebDAV mime/timestamp fallback survives an empty field

proxytrack's DAV PROPFIND response computes a fallback content-type and
timestamp when a cache entry has no Content-Type/Last-Modified; that
fallback already existed before commit eae1dd0 changed the surrounding
always-true array-address checks, so this test guards the equivalence
rather than a bug. Verified it fails when the fallback default is
disabled, and passes unmodified against the pre-eae1dd0 code too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: bound the PROPFIND request and cut the header comment

curl had no --max-time; an unbounded read wedges the runner instead of
failing it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: skip the WebDAV mime test on Windows

It is the first test to run proxytrack as a live listener, and MSYS cannot
reap a native one: the orphan wedged the whole Windows suite past its
45-minute budget, twice, destroying the log upload with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 14:49:29 +02:00
Xavier Roche
a7fbd3f739 htsserver builds the redirect Location header in a 256-byte stack buffer (#700)
* htsserver builds the redirect Location header in a 256-byte stack buffer

The POST redirect path checks strlen(file) but sprintf's newfile, which comes
straight from the client's "redirect" POST field with no length cap. A 300-byte
value overflows tmp[256]. The same value reached the Location header with no
CR/LF check, so it could split the response and inject headers.

Append into the dynamic String the other headers already use, and drop the
header entirely when the value carries a CR or LF.

The listen socket was SOCaddr_initany, so the server answered the LAN and not
just the local browser it exists to serve. Bind 127.0.0.1 by default, with
--bind <addr> to widen it again, resolved through the existing gethost() helper
the way proxytrack already does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: satisfy shellcheck and shfmt in the new server test

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: drop the pre-fix narration from the oversized-value comment

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: assert the bound socket, not the announced URL

The listen-address assertions only compared the URL= banner, which is a
literal echo of argv: a build that announced 127.0.0.1 while binding the
wildcard passed. Probe 127.0.0.2 on the same port instead, which a wildcard
listener takes and a loopback-only one leaves free.

Also refuse an empty --bind, which fell through to every interface and
silently undid the new default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: bound the response read and the empty --bind run

The recv() loop had no timeout and read until EOF; htsserver need not close
the connection after responding, which wedged the macOS runner for over an
hour. Stop at the end of the header block, which is all the test reads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* htsserver: the session id must gate the request body, not the reply

Every field of a POST body is written straight into the one global key store
the templates and the command dispatcher both read, and "command" from there
reaches the engine. The gate ran after that write and compared "sid" against
"_sid" -- but "_sid" is copied into "sid" beforehand so the templates can
render it, so a request that simply omitted the field compared equal to
itself. Only a wrong id was refused; an absent one passed. Clearing the reply
afterwards does not help either, because the dispatcher sits outside the reply
guard.

Authenticate before parsing instead: scan the raw body for "sid", require at
least one occurrence and reject if any of them differs, and drop the body
untouched when it does not match. That leaves the shared template key alone,
and it closes the dispatcher for free.

A refused request also emitted only a Content-length line, since the status
line for that branch was behind _DEBUG. Any client reads that as a protocol
error, which is how test 68 failed rather than reporting the refusal. Send a
403 instead.

Tests 68 and 77 posted without an id, which is what the engine used to accept,
so both now fetch the one the server renders into the form. Test 78 covers
accept, missing, empty and wrong, asserts the 403, and probes the key store
through ${projname} rather than the suppressed reply -- a reply-only assertion
passes even when the write goes through.

Also fix a leak that hung macOS CI: start() runs inside a command
substitution, so its $! never reached the parent and stop() guarded on an
empty variable, leaving one htsserver per call. Test 77 starts four, which is
exactly the four orphans the runner reported while sitting for half an hour
behind a green test log. Take the pid from the PID= line the server already
announces.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 13:59:15 +02:00
Xavier Roche
c4b803eb33 -%S list file over 4GB overflows the heap (#702)
* -%S list file over 4GB overflows the heap

hts_main_internal() sized the -%S buffer as `cl + fz + 8192` and stored
the sum in an int url_sz, then fread() the untruncated 64-bit fz into it.
A 4GB+100KB rules file wraps the capacity to 110602 bytes on x64, the
realloct() succeeds, and the read walks off the heap. Intermediate sizes
land on a negative int and fail the allocation, which is luck, not design.

Route the file-size arithmetic through llint_grow_size_t(), a saturating
sibling of llint_to_size_t() that refuses a total it cannot represent, and
widen url_sz and the filelist offsets to size_t. The "config" sizing in the
same function and htscore.c's primary_len had the same shape: a file size
accumulated into an int before reaching an allocator. htscache.c's two
mirrored-file comparisons held a 64-bit fsize_utf8() in a size_t, which
truncates on Win32 and re-downloads a >4GB file already on disk.

Found by MSVC C4244 on x64; invisible to gcc/clang because int64_t to
size_t is width-preserving on LP64.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Print T_SOC with the format matching its width, not a bare %d

T_SOC is unsigned __int64 on Win64 (htsglobal.h): passing it to fprintf's
%d is undefined behavior, flagged by MSVC C4477. Add T_SOCP beside the
typedef, following the existing LLintP/INTsysP precedent, and use it at
both deletesoc() call sites (htslib.c:2601, :2607).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Fix MSVC C2099 in the new growsize self-test

A static const object used inside another object's static initializer is a
GNU/clang extension, not standard C: MSVC's /TC C mode rejects it ("initializer
is not a constant"). Replace the over32 local with a macro.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: cover the slack-only overrun and the largest capacity

A helper dropping the slack bound passed the table; -1 as extra also refused
either way, since llint_to_size_t() maps it to SIZE_MAX regardless.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 13:23:03 +02:00
Xavier Roche
408316db99 ARC cache replays a truncated HTTP reason phrase (#701)
* ARC cache replays a truncated HTTP reason phrase

proxytrack bounded the reason-phrase copy out of an ARC index by
sizeof(pos) - 1 where pos is a const char *, so a stored "404 Not Found"
replays as "404 Not Fou" (and "404 Not" on 32-bit). Use strncatbuff with
the destination's own size.

Fold the nine copies of the buff() family's source-capacity expression
into HTS_SIZEOF_SRC_, applying sizeof to the type so a decayed operand no
longer trips -Wsizeof-array-decay at five call sites; MSVC keeps the old
expression behind the guard HTS_IS_CHAR_BUFFER already uses. The two
other raw strncat calls become strncatbuff, htsbuff_catn stops handing
strnlen the (size_t)-1 sentinel, and htsweb.c no longer compares ep
against a NULL eps.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Size the new strsafe buffers away from sizeof(char*)

char[8] equals a pointer on LP64, so MSVC's array-vs-pointer heuristic
read the unterminated source as a pointer, skipped the bound and never
aborted; the x64 build failed while Win32 passed. Same trap the existing
comment in that function already warns about.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 11:31:45 +02:00
Xavier Roche
450db10d92 Compiler-flag probes accept flags clang only warns about, and -rdynamic never reaches the linker (#699)
* configure: fix flag probes, move -rdynamic to the link line

AX_CHECK_COMPILE_FLAG only checks the exit status, and clang merely warns on
an unknown -W name, so -Wmissing-parameter-type reached every clang build and
warned on all 66 TUs. Probe with -Werror; -Wformat-nonliteral also needs
-Wformat there or gcc rejects it and the flag would be lost.

-rdynamic lived in DEFAULT_CFLAGS, i.e. AM_CPPFLAGS, so no link line ever saw
it; make it a link check. Drop -pie from CFLAGS_PIE (LDFLAGS_PIE has it), and
drop -Wdeclaration-after-statement, a C90 rule the gnu17 build does not follow
anywhere else.

Distinct build warnings: gcc 82 -> 54, clang 54 -> 41.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* configure: tighten the two new flag-block comments

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 11:24:36 +02:00
Xavier Roche
dfdf10e7fd No UTF-8 to system-codepage conversion is exported to the Windows GUI (#698)
WinHTTrack is an MBCS build, so it still lands on the ANSI-only Win32 entry points (TTN_NEEDTEXTA and the like) while the engine hands it UTF-8. The other direction is already exported as `hts_convertStringSystemToUTF8`, so this adds the mirror, `hts_convertStringUTF8ToSystem`, a one-liner over the existing `hts_convertStringCPFromUTF8`. The GUI can then delegate instead of keeping its own MultiByteToWideChar/WideCharToMultiByte copy (`CopyTextUTF8ToCP` in newlang.cpp, added while fixing #114). `hts_convertStringFromUTF8` would also have done the job, but it is declared plain `extern` and never reaches the DLL export table; left alone here. Windows-only addition, so no POSIX ABI change and no soname move.

The new `-#test=syscharset` self-test round-trips against the raw Win32 two-step it replaces, and skips off Windows.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 09:11:12 +02:00
Xavier Roche
7fe073e9fb CLI display does not repaint when the terminal is resized (#697)
* CLI: redraw the whole screen when the terminal is resized

The animated -%v display assumed a fixed 80x24 layout: it cleared the screen
once at start, and afterwards cleared only to end of line on the rows it
wrote. A resize left stale wrapped text around the frame, and on a terminal
shorter than the 20-row frame the stats block scrolled off the top at every
refresh.

Poll the terminal geometry at each refresh (TIOCGWINSZ, or the console screen
buffer info on Windows) and repaint in full when it moved. The in-progress
list is capped to the rows that fit, and the URL column follows the width; it
stays at the historical 40 characters on an 80-column terminal.

Closes #97

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: assert the resize repaint, the row clamp and the URL width

The first pass only checked that a clear-screen followed one resize, which a
display clearing on every frame passes just as well, and neither the row clamp
nor the URL budget was exercised at all: the single 24x80 to 40x100 resize
leaves both at their 80x24 values.

The harness now waits for the display instead of sleeping, resizes width and
height separately, and asserts a repaint after each one, none in between, a
frame that fits a 10-row terminal, and a long URL that stops being truncated at
200 columns. The long path is a new /trickle/deep route, so the shared
/trickle/ index other tests assert on stays untouched. Mutating any of the four
behaviors away fails the test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 08:56:26 +02:00
dependabot[bot]
f47359247d Bump actions/checkout from 6 to 7 (#696)
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-26 08:37:41 +02:00
dependabot[bot]
dfc72fdbc6 Bump actions/cache from 4 to 6 (#695)
Bumps [actions/cache](https://github.com/actions/cache) from 4 to 6.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v4...v6)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-26 08:04:17 +02:00
dependabot[bot]
8ed607b078 Bump actions/upload-artifact from 4 to 7 (#694)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4 to 7.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4...v7)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-26 08:04:14 +02:00
Xavier Roche
0708bf9f98 CI: let Dependabot track GitHub Actions versions (#693)
Dependabot only watched vcpkg, so the action pins drifted by hand:
windows-build.yml sat on actions/checkout@v4 while everything else moved to
v6. Add the github-actions ecosystem and bump that stray checkout to v6.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 10:17:54 +02:00
Xavier Roche
301d7717be Windows CI: cache vcpkg binary archives across runs (#692)
vcpkg builds openssl/brotli/zlib/zstd from source on every Windows run.
Redirect its files binary cache into the workspace and persist it with
actions/cache, keyed on the manifest (which carries the builtin-baseline).

x-gha, which used to do this, was dropped from vcpkg-tool (#1662) after
GitHub changed the Actions cache API; actions/cache over the archive dir is
the endorsed replacement and stays within the GitHub-owned-actions policy.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 10:17:14 +02:00
Xavier Roche
e13f0f10c9 Drop the removed Java .class parser from the credits (#691)
* Drop the removed Java .class parser from the credits

The Java binary .class parser was removed in #552, but its credit lingered:
"JavaParserClasses: Yann Philippot" in the WinHTTrack About box (lang.def plus
every lang/*.txt), and "for the java binary .class parser" in greetings.txt and
the contact page. Remove the About-box line and the now-defunct descriptor,
keeping Yann Philippot's name in the developed-by acknowledgments.

lang.def and lang/*.txt are read as strict line pairs, so the edit deletes the
substring inside the single credits line rather than removing any line, keeping
the pairing intact and each translated file's English msgid in lockstep with
lang.def. Verified by tests/62_lang-integrity.test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Also drop the localized java-parser credit from the translations

The About box shows the translation (msgstr), not the English msgid. The first
pass only removed the literal English "JavaParserClasses: Yann Philippot", so the
12 files whose translators localized the credit (Chinese, French, Russian,
Turkish, ...) still displayed it. Remove the localized credit line from each of
those msgstrs too; the pairing and every English msgid are untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Reword Yann Philippot's credit as a past contributor

Dropping the descriptor left a bare name in the developed-by list. Mark the
java binary .class parser as a past contribution instead, keeping the
acknowledgment meaningful now that the code is gone.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* contact.html: use a literal dot in .class

The " dot " spelling is the page's email anti-harvest obfuscation; applied to a
filename extension it just left a stray double space. The page already writes
literal dots elsewhere (v2.0, v3.0).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 15:31:29 +02:00
Xavier Roche
3265a5de00 Release 3.49.14 (#690)
Bump the version in configure.ac, htsglobal.h and version.rc, move VERSION_INFO to 3:6:0, and add the history.txt and debian/changelog entries.

The WARC fields landed at the tail of httrackp and htsblk, but the embedded htsoptstate and htsblk members sit mid-struct, so the fields after them shifted. Installed layouts moved while the soname stays .so.3; the VERSION_INFO comment records that rather than claiming the layouts are unchanged.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 08:23:28 +02:00
Xavier Roche
d33f666b38 Windows: load IE cookies from long or non-ASCII folders (FindFirstFileW) (#689)
* Windows: import IE cookies via FindFirstFileW so long/non-ASCII jars load

cookie_load's IE-cookie scan and its cookies.txt read used the narrow
ANSI file API (FindFirstFileA, fopen, remove), so a cookie folder with a
non-ASCII or >MAX_PATH path was silently skipped, and any matched IE
cookie name was fed back through the mirror path as CP_ACP bytes. Route
the glob through hts_pathToUCS2 + FindFirstFileW, convert cFileName to
UTF-8 before rebuilding the path, and use the FOPEN/UNLINK wrappers.

The A->W find and the sink swaps land together on purpose: swapping only
the sink would feed a CP_ACP name into hts_fopen_utf8's UTF-8 decode and
mojibake the accidentally-consistent ANSI path. hts_pathToUCS2 is
un-static'd (internal, still -fvisibility=hidden / not HTSEXT_API) so the
glob gets the same \\?\ long-path treatment as the other wrappers.

Last piece of the #133 Windows long-path series. Self-test
-#test=cookieimport drives cookie_load against a long, non-ASCII folder;
POSIX is a positive control (IE block compiled out), the wide glob and
IE import run on the Windows legs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Trim review-flagged comments to one line

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 17:18:18 +02:00
Xavier Roche
6223739cba Windows: enumerate long or non-ASCII directories via FindFirstFileW (#688)
* Windows: enumerate long or non-ASCII directories via FindFirstFileW

The opendir/readdir emulation was fully ANSI (FindFirstFileA, CP_ACP
cFileName), so it capped listing at MAX_PATH and mis-decoded a non-ASCII
path — the odd wrapper out in an engine that feeds UTF-8 paths and reads
d_name as UTF-8. The one live Windows caller, the end-of-crawl
hts-cache/ref cleanup, silently no-op'd at a long or non-ASCII project
root, leaking the temp ref/ directory into the finished mirror.

Route both through hts_pathToUCS2 (\\?\ prefixing, #684) and the wide
APIs, converting cFileName back to UTF-8. d_name now holds UTF-8, so
HTS_DIRENT_SIZE grows to 1024 to fit MAX_PATH's worst-case expansion; the
struct is _WIN32-only, no POSIX ABI impact. Self-test direnum drives it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Trim comments to one line each (review follow-up)

Condense the opendir/readdir and direnum-selftest comments per the
review's comment-conciseness pass; no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 14:59:45 +02:00
Xavier Roche
301f5c2f2f topindex: fix mojibake for non-ASCII project categories on Windows (#216) (#683)
* topindex: convert the winprofile.ini category to UTF-8 on Windows (#216)

The project category read from hts-cache/winprofile.ini comes back in the
Windows ANSI codepage and is written into the charset=utf-8 topindex
template, so a non-ASCII category renders as mojibake. Convert it the same
way #681 did the project name. The topindex self-test now writes a
winprofile.ini with a non-ASCII category and asserts the generated index
carries the UTF-8 form.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* topindex: drop the redundant category-assert comment

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 13:49:21 +02:00
Xavier Roche
dd321171b4 tests/fsize: skip when the filesystem caps file size below 5GB (#687)
The 01_engine-fsize self-test builds a 5GB sparse file to exercise the 32-bit size wrap. On GNU/Hurd i386 the ext2fs translator rejects the extend with EFBIG, failing the Debian build on that port. Return 77 (automake skip) instead of 1 when the extend fails with EFBIG, and map that to a skip in the .test wrapper. Any other errno, or a wrong size report, still fails, so real regressions are not masked.
2026-07-23 13:47:58 +02:00
Xavier Roche
2f158c05d0 Windows: mirror files under non-ASCII or long directories silently break (#686)
* Windows: route mirror-tree file ops through the UTF-8/long-path wrappers

The engine's raw fopen/remove/rename/rmdir/mkdir/fexist/fsize calls on
mirror-tree paths bypassed the UTF-8 (_w*) wrappers, so on Windows a mirror
under a non-ASCII or >MAX_PATH directory silently broke: a long/non-ASCII
file read as absent (its unlink/reget skipped), or its bytes went to a
mojibaked, truncated path.

Swap those call sites in htscoremain/htscore/htsparse/htsindex/htshelp to
the FOPEN/UNLINK/RENAME/MKDIR/fexist_utf8/fsize_utf8 twins, and add
hts_rmdir_utf8 (RMDIR) for the three rmdir sites that lacked a wrapper. The
arguments are already UTF-8 (Windows argv is transcoded at startup, fconcat
is byte-transparent), so there is no double-conversion. Raw calls on
genuinely system-charset or ASCII paths (getenv/$HOME, structcheck's ASCII
twin, "config") are left as-is.

-#test=mirrorio drives a long AND non-ASCII path through the wrapped guards
and the new rmdir wrapper; on POSIX it stands as a byte-transparent control.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* test: cover RENAME on the long+non-ASCII mirror path

The mirrorio self-test drove FOPEN/UNLINK/RMDIR but not RENAME, which the
sweep newly routes to. Rename the leaf to a non-ASCII sibling and verify
the move before teardown.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 13:47:38 +02:00
Xavier Roche
6c74d94802 Windows: reach past MAX_PATH via \\?\ at the file-op choke-point (#133) (#684)
* Windows: reach past MAX_PATH via \\?\ at the file-op choke-point (#133)

The _w* file wrappers on Windows all funnel their path through one
converter. Route it through a new hts_pathToUCS2 that, for a path near
MAX_PATH, absolutizes and normalizes it with GetFullPathNameW and
prepends the "\\?\" verbatim prefix ("\\?\UNC\" for UNC shares) so the
wide file APIs accept paths past 260 chars. Shorter paths keep today's
exact behavior, so nothing changes until the naming ceiling in htsname.c
is raised (a follow-up); a -#test=longpath self-test drives a >260-char
path through the wrappers to exercise the new branch on Windows CI now.

Partially addresses #133.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Run the long-path self-test on the Windows CI leg; tighten comments

The Windows job iterates an explicit test glob (01_engine-*, 01_zlib-*,
*_local-*, ...), so 76_engine-longpath-io.test matched nothing and the
\\?\ path never ran on the one platform that needs it. Rename it to
01_engine-longpath-io.test so the glob picks it up and the prefixing is
actually exercised on Windows. Also trim the review comments to the why.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 12:27:44 +02:00
Xavier Roche
00b0f5728c Use the platform path limit off Windows, not the Windows MAX_PATH (#133)
* Use the platform path limit off Windows, not the Windows MAX_PATH (#133)

url_savename capped every saved path at 236 chars (MAX_PATH minus 8.3
headroom minus the ".delayed" marker) on all platforms, so Linux, macOS
and Android hashed long names to fit a limit only Windows has. Off
Windows, derive the ceiling from the platform's own PATH_MAX/NAME_MAX,
clamped to the fixed save buffer so an oversized output dir can never
push the final path past it and abort. Windows keeps the MAX_PATH
ceiling until the engine can prefix its paths with \\?\.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Cut the save name in bytes, not codepoints, so multibyte paths can't overflow

The path-length guard measured length in UTF-8 codepoints (hts_stringLengthUTF8)
while the buffer that receives parent+name is a fixed byte array whose overflow
aborts() rather than truncates. With the ceiling raised to ~1984 on POSIX, a
multibyte name (e.g. a long CJK path) can stay under the codepoint cap yet run
its byte length past the 2048-byte buffer, crashing the crawl on the final
prepend. Add an unconditional byte-boundary cut before the parent is prepended,
and drop the earlier codepoint-unit clamp it supersedes. Covers the oversized
-O parent case in the same measure. Regression test uses a 339-char CJK name
under a ~1020-byte output dir, which aborted before the cut.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Leave an oversized parent to the existing abort, not an empty colliding name

When the output dir alone fills the save buffer, forcing the name to empty made
every URL under it collapse to the same path, feeding the unbounded collision
sprintf and turning a clean abort into a 1-2 byte overflow. Only shrink the
name; a buffer-filling parent aborts on the prepend exactly as before (that
oversized-parent path is pre-existing and unrelated to the #133 ceiling).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 09:36:06 +02:00
Xavier Roche
93314bcef9 topindex: fix mojibake for non-ASCII project names on Windows (#216) (#681)
* topindex: non-ASCII project names render as mojibake on Windows (#216)

hts_buildtopindex() lists each sub-project by the name FindFirstFileA
returns in the ANSI codepage, then writes it into a document declaring
charset=utf-8, so a non-ASCII name shows up as mojibake. Convert the name
to UTF-8 on Windows, matching the gif-path fix already in this function
for #217. The topindex self-test now builds a non-ASCII sub-project and
asserts the generated index carries the UTF-8 name.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* topindex: drop const so freet() can null the converted name (MSVC)

The #ifdef _WIN32 conversion block only compiles on Windows, where MSVC
rejected freet() nulling a char *const (C2166). Make the pointer mutable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* topindex: tighten the added comments

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 09:34:27 +02:00
Xavier Roche
d3d3bce8af cmdguide: add a WARC/WACZ archive recipe (#680)
The command-line guide's recipe section had no entry for WARC output, so
the five --warc* flags were discoverable only through --help or the man
page. Add a §11 recipe leading with the gotcha that trips people: the
archive is written alongside the browsable mirror, not instead of it.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 08:09:16 +02:00
Xavier Roche
b3e51d753b WARC: file the --warc help under Log/index/cache, not Build (#679)
WARC is a transaction-level archive of what was fetched (a sibling of the
hts-cache), not a browsable-mirror build format like MHTML, so it belongs in
the Log/index/cache group next to the cache options. Help text only; the
webhttrack GUI already places it on the Log/Index/Cache tab (option9).

Signed-off-by: Xavier Roche <xroche@gmail.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 07:07:05 +02:00
Xavier Roche
5099efc1cf WARC: store response bodies verbatim by default, drop --warc-verbatim (#678)
Verbatim compressed bodies (Content-Encoding kept, only hop-by-hop
Transfer-Encoding dropped, Content-Length rewritten to the stored length)
are now the sole behavior, matching wget --warc and Heritrix. The former
default that decoded the body and stripped Content-Encoding is gone, along
with the --warc-verbatim switch (added the same day, never released), the
warc_verbatim option field, and the dual-mode plumbing in
normalize_http_headers.

This also fixes a cap-truncated compressed response: it previously stored
the raw compressed partial while stripping Content-Encoding, mislabeling
gzip bytes as identity. Keeping Content-Encoding makes the record's label
match its body; the truncation still carries WARC-Truncated: length/time.

header_is now tolerates whitespace before the ':' so a non-compliant
"Content-Encoding : gzip" is still recognized.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 03:35:53 +02:00
Xavier Roche
e1d4c35ee6 WARC: WACZ packaging (--wacz) (#677)
* Add --warc-cdx: sorted CDXJ index alongside the WARC archive

WARC v2 PR B. --warc-cdx (alias --warc-cdxj, sub-flag -%rc) writes a sorted
CDXJ index next to the .warc.gz, one line per response/revisit/resource record
(warcinfo/request are not indexed): the SURT sort key, a 14-digit timestamp,
and a JSON object with url, mime, status, payload digest, and the record's byte
offset and length in the gzip stream so a replay tool can seek and inflate a
single member. Rotation-aware (the filename field tracks the current segment).

SURT canonicalization is new (surt_canon in htswarc.c): scheme/userinfo
dropped, host lowercased with a leading www[digits] label and the scheme
default port stripped, labels reversed and comma-joined, a non-default port
kept, IPv4 and [IPv6] literals left verbatim. Lines are accumulated in the
writer and qsort'd in LC_ALL=C byte order at close.

Tested by -#test=warc-surt (SURT vectors, 01_engine, MSan-instrumented) and
-#test=warc-cdx (end-to-end: sorted, one line per record, each offset/length
independently inflates to the matching WARC-Target-URI; 01_zlib).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Add --wacz: package the WARC archive, CDXJ index and pages as a WACZ file

--wacz bundles the crawl's WARC segment(s), the sorted CDXJ index and a
generated pages.jsonl into a single WACZ 1.1.1 package at crawl end, using
the in-tree minizip. It implies --warc and --warc-cdx. Every ZIP entry is
stored (no re-deflate of the already-gzipped WARC), as the WACZ spec
requires. datapackage.json lists each file with its SHA-256 and size, and
datapackage-digest.json chains the digest of datapackage.json.

SHA-256 comes from OpenSSL; a build without OpenSSL cannot emit conformant
digests, so --wacz is refused there with a clear log line while the WARC and
CDXJ are still written.

pages.jsonl captures the top-level 200 text/html responses (URL + WARC-Date),
bounded like the CDX accumulator; the first seeds datapackage mainPageUrl.

Tests: an in-process self-test (-#test=warc-wacz) unzips the package and
asserts the layout, STORE mode on every entry, each recomputed SHA-256, the
digest chain, and the pages header; a crawl test validates a real .wacz with
a stdlib validator (py-wacz/pywb when importable). Both skip cleanly without
OpenSSL.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Package WACZ atomically so a failed re-run can't destroy a good archive

warc_wacz_package opened <base>.wacz directly with a create/truncate zip
handle, wiping the previous archive before it checked its inputs existed. A
re-run that produced zero indexable records (empty crawl, or every fetch
erroring so warc_cdx_flush writes no .cdx) truncated the good .wacz, failed to
find indexes/index.cdx, then unlinked the now-empty file with no log. Same
data-loss class as the hard-abort file truncation (#522).

Build the package into <base>.wacz.tmp and only rename it over <base>.wacz on
full success (RENAME, with an unlink+rename fallback so Windows can clobber).
On any error unlink the temp, leave the existing archive untouched, and warn
(the old error path was silent); the writer keeps opt for close-time logging.

The warc-wacz self-test now proves it: after a good package it drops the .cdx,
re-runs empty, and asserts the .wacz is byte-unchanged. Also tightens both the
C self-test and the Python validator against the WACZ spec (profile,
wacz_version, digest path, and >= 1 pages.jsonl body row with url + ts) so a
writer emitting an empty package can't pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 01:32:34 +02:00
Xavier Roche
69c562bf0c WARC: verbatim compressed response bodies (--warc-verbatim) (#675)
* WARC: add --warc-verbatim to store compressed bodies as received

--warc-verbatim (internal -%rv, implies --warc) captures a compressed
response in its as-received content-coded form instead of the decoded
strategy-B body. back_finalize already materializes the whole de-chunked
compressed body as a temp file just before decoding it, so the tee is a
save-before-unlink: adopt that spool onto the new htsblk.warc_rawpath /
warc_rawsize instead of unlinking it, and emit the response record with
Content-Encoding kept and Content-Length set to the compressed length
(normalize_http_headers gains a keep_ce mode). WARC-Payload-Digest is
then over the coded payload, which is what the record carries.

Non-compressed responses, the never-spooled is_write archive-ext case,
and the cap-truncated second emit site all leave warc_rawpath NULL and
fall back to strategy B unchanged. The spool is owned by the entry:
freed and unlinked in warc_free_request, NULLed in back_copy_static and
back_unserialize so a shallow copy never double-unlinks.

Output is now byte-replay-identical to what the origin sent, matching
wget --warc and Heritrix.

Tests: a warc-verbatim engine self-test asserts the response keeps
Content-Encoding: gzip, Content-Length == the compressed length, the
stored bytes equal the gzip input and inflate to the known plaintext
(bite-checked); tests/74_local-warc-verbatim crawls a gzip-served
fixture and runs the strategy-A/B differential through
warc-validate.py --verbatim (inflate(stored) == the decoded body).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* WARC verbatim: cover the direct-to-disk adoption path, harden the validator

Review follow-up to --warc-verbatim (strategy A).

The spool adoption in back_finalize runs on both the in-memory and the
direct-to-disk (is_write) branch, but only the in-memory path was crawl-tested.
Add a non-hypertext gzip fixture (application/octet-stream) so the crawler
streams it to disk and the is_write adoption is exercised; the warc-validate
--verbatim gate now asserts Content-Encoding kept, Content-Length == compressed
length, and inflate(stored) == the served plaintext for both assets. Confirmed
at runtime that page.html takes is_write=0 and data.bin is_write=1, and
bite-checked that dropping the is_write adoption fails the new assertion.

warc-validate --verbatim now requires WARC-Payload-Digest on a verbatim record
when the file emits digests at all (an OpenSSL build), so a regression that
drops the digest is caught without breaking the no-OpenSSL leg.

Extract warc_adopt_rawspool() so the WARC spool bookkeeping lives behind the
htswarc boundary, mirroring warc_stash_response. Comment trims throughout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 01:02:45 +02:00
Xavier Roche
d83ed3fdee Add --warc-cdx: sorted CDXJ index alongside the WARC archive (#674)
WARC v2 PR B. --warc-cdx (alias --warc-cdxj, sub-flag -%rc) writes a sorted
CDXJ index next to the .warc.gz, one line per response/revisit/resource record
(warcinfo/request are not indexed): the SURT sort key, a 14-digit timestamp,
and a JSON object with url, mime, status, payload digest, and the record's byte
offset and length in the gzip stream so a replay tool can seek and inflate a
single member. Rotation-aware (the filename field tracks the current segment).

SURT canonicalization is new (surt_canon in htswarc.c): scheme/userinfo
dropped, host lowercased with a leading www[digits] label and the scheme
default port stripped, labels reversed and comma-joined, a non-default port
kept, IPv4 and [IPv6] literals left verbatim. Lines are accumulated in the
writer and qsort'd in LC_ALL=C byte order at close.

Tested by -#test=warc-surt (SURT vectors, 01_engine, MSan-instrumented) and
-#test=warc-cdx (end-to-end: sorted, one line per record, each offset/length
independently inflates to the matching WARC-Target-URI; 01_zlib).

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 00:36:30 +02:00
Xavier Roche
ce278a4184 Expose WARC output in the webhttrack GUI (#672)
* Expose --warc / --warc-file in the webhttrack GUI

Add a WARC output checkbox and an optional archive-name field to the
"Log, Index, Cache" option tab, beside store-all-in-cache. The checkbox
emits --warc (auto-named archive); the text field emits --warc-file NAME.
Wiring mirrors how #589 added cookies-file and strip-query: the option9.html
form fields, the generated httrack command and winprofile.ini in step4.html,
and the reload remap in step2.html.

New LANG_WARC / LANG_WARCFILE strings and tooltips land in lang.def,
English.txt and Francais.txt; the remaining 28 language files fall back to
French for now and are a follow-up.

The webhttrack smoke test now also fetches option9.html and requires the
WARC control to render.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Translate the webhttrack WARC strings into the remaining languages

PR #672 added a WARC toggle to the webhttrack GUI with 4 new LANG keys,
translated only in English and French. Append the same 4 msgid/translation
pairs to the other 28 lang/*.txt so those locales stop falling back to
English. Each translation is encoded in the file's declared LANGUAGE_CHARSET
(the charset the server serves the page and parses the form as) with a strict
lossless round-trip, matching each file's existing CRLF/LF line ending.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 22:49:36 +02:00
Xavier Roche
f6f46e88b2 WARC segment rotation, truncation tagging, and FTP resource records (#673)
* Add WARC/1.1 v1.1: --warc-max-size rotation, WARC-Truncated, FTP resource records

Extends the merged WARC writer with the three v1.1 items. All logic stays in
htswarc.c; the engine hooks stay thin.

--warc-max-size N (-%rs) rotates the archive into NAME-00000.warc.gz, -00001,
... once a segment passes N bytes (wget naming), each segment led by its own
warcinfo and never splitting a record. N<=0 keeps the single-file behavior.

WARC-Truncated tags a body cut short by a cap. The mirror size (-M) and time
(-E) caps abort in-flight transfers and overwrite the slot's status to a
negative TIMEOUT, after which back_finalize and the WARC hook both bail; so the
partial is archived at the abort site, before the clobber, with WARC-Truncated:
length/time. HTTrack's own incomplete/retry bookkeeping is untouched, and a
genuine broken transfer (not a cap) is still not archived. The per-file cap (-m)
rejects on Content-Length rather than truncating, so it has no truncated body.
Standard ISO 28500 tokens only (length/time/disconnect); the task's non-standard
"disk" is left out.

FTP transfers have no HTTP envelope, so an ftp:// capture becomes one resource
record: WARC-Type: resource, the payload's own Content-Type, block = payload,
no request/response. FTP back_finalize runs on the main loop (the worker thread
only flips status), so the single-writer no-lock design holds.

Self-tests warc-trunc / warc-ftp / warc-rotate added to the -#test=warc family
and 01_zlib-warc.test; each was confirmed to fail without the corresponding
code. man/httrack.1 + html regenerated for the %r help line.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Range-check --warc-max-size and regenerate the man page

Replace the unchecked sscanf on the --warc-max-size argument with a
strtoll parse that rejects non-numeric, negative, and overflowing values,
leaving the default 0 (single archive) intact. The man page and its HTML
render were already regenerated for the option's help line in the feature
commit, so this only tightens the parse.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 22:39:32 +02:00
124 changed files with 8317 additions and 948 deletions

View File

@@ -6,3 +6,9 @@ updates:
directory: /src
schedule:
interval: weekly
# Keep the workflow action pins current (they only rot manually otherwise).
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly

View File

@@ -31,7 +31,7 @@ jobs:
env:
CC: ${{ matrix.cc }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -75,7 +75,7 @@ jobs:
name: build (no python3, Debian buildd)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -119,7 +119,7 @@ jobs:
name: build (macOS arm64, clang)
runs-on: macos-14
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -166,7 +166,7 @@ jobs:
name: webhttrack smoke (macOS arm64)
runs-on: macos-14
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -199,7 +199,7 @@ jobs:
name: build (linux i386, gcc -m32)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -242,7 +242,7 @@ jobs:
name: sanitize (ASan+UBSan, gcc)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -296,7 +296,7 @@ jobs:
name: msan (MemorySanitizer, clang)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -344,7 +344,7 @@ jobs:
name: fuzz (libFuzzer corpus replay, clang)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -383,7 +383,7 @@ jobs:
name: build (no openssl, --disable-https)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -437,7 +437,7 @@ jobs:
zlib1g-dev libssl-dev libbrotli-dev libzstd-dev \
debhelper devscripts lintian fakeroot
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -468,7 +468,7 @@ jobs:
name: distcheck (release tarball)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -493,7 +493,7 @@ jobs:
if: github.event_name == 'pull_request'
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
fetch-depth: 0
@@ -537,7 +537,7 @@ jobs:
tests/*.test
tools/mkdeb.sh
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: Install linters
run: |
@@ -572,7 +572,7 @@ jobs:
if: github.event_name == 'pull_request'
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
fetch-depth: 0
@@ -624,7 +624,7 @@ jobs:
if: github.event_name == 'pull_request'
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
fetch-depth: 0

View File

@@ -26,7 +26,7 @@ jobs:
# Upload findings to the repo's code-scanning dashboard.
security-events: write
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive

View File

@@ -23,8 +23,13 @@ jobs:
matrix:
platform: [x64, Win32]
configuration: [Release]
# Redirect vcpkg's default `files` binary cache into the workspace so
# actions/cache can persist it. vcpkg builds openssl/brotli/zlib/zstd from
# source otherwise, several minutes every run.
env:
VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}\vcpkg_cache
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
with:
submodules: recursive # coucal lives in src/coucal
@@ -44,6 +49,24 @@ jobs:
shell: pwsh
run: vcpkg integrate install
# vcpkg errors if VCPKG_DEFAULT_BINARY_CACHE points at a missing dir, and
# actions/cache does not create it on a miss.
- name: Create the vcpkg binary cache directory
shell: pwsh
run: New-Item -ItemType Directory -Force -Path $env:VCPKG_DEFAULT_BINARY_CACHE | Out-Null
# x-gha is gone (vcpkg-tool #1662 dropped it after GitHub changed the cache
# API), so cache the binary archives directly. Keyed on the manifest, which
# carries the builtin-baseline, so a Dependabot bump busts it; restore-keys
# still seeds the unchanged ports' archives, so only the bumped one rebuilds.
- name: Cache vcpkg binary archives
uses: actions/cache@v6
with:
path: ${{ github.workspace }}\vcpkg_cache
key: vcpkg-${{ matrix.platform }}-${{ hashFiles('src/vcpkg.json') }}
restore-keys: |
vcpkg-${{ matrix.platform }}-
# The runner image's vcpkg checkout is pinned to some commit; our manifest's
# builtin-baseline is usually newer, so `git show <baseline>:versions/...`
# fails until that commit is local. Fetch exactly the pinned baseline (read
@@ -202,15 +225,16 @@ jobs:
# Every gate here exits 77, so an all-skipped suite would report green having
# tested nothing: pin the skips, and floor the passes in case the glob empties.
# footer-overflow skips on Windows (needs a path past MAX_PATH); crange pending #581.
expected_skips=" 01_engine-footer-overflow.test 48_local-crange-memresume.test 71_local-crange-repaircache.test"
# footer-overflow skips on Windows (needs a path past MAX_PATH); crange pending #581;
# webdav-mime needs a reapable background listener, which MSYS cannot give it.
expected_skips=" 01_engine-footer-overflow.test 48_local-crange-memresume.test 71_local-crange-repaircache.test 79_local-proxytrack-webdav-mime.test"
[ "$pass" -ge 90 ] || { echo "::error::only $pass tests passed ($skip skipped)"; exit 1; }
[ "$skipped" = "$expected_skips" ] || { echo "::error::unexpected skips:$skipped"; exit 1; }
[ "$fail" -eq 0 ] || { echo "::error::failing:$failed"; exit 1; }
- name: Upload the test logs
if: always()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: engine-tests-${{ matrix.platform }}-${{ matrix.configuration }}
path: tests/*.log
@@ -218,7 +242,7 @@ jobs:
- name: Upload MSBuild logs
if: always()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: msbuild-${{ matrix.platform }}-${{ matrix.configuration }}
path: msbuild-*.log

View File

@@ -1,6 +1,6 @@
AC_PREREQ([2.71])
AC_INIT([httrack], [3.49.13], [roche+packaging@httrack.com], [httrack], [http://www.httrack.com/])
AC_INIT([httrack], [3.49.14], [roche+packaging@httrack.com], [httrack], [http://www.httrack.com/])
AC_COPYRIGHT([
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998-2015 Xavier Roche and other contributors
@@ -29,13 +29,12 @@ AC_CONFIG_SRCDIR(src/httrack.c)
AC_CONFIG_MACRO_DIR([m4])
AC_CONFIG_HEADERS(config.h)
AM_INIT_AUTOMAKE([subdir-objects])
# 3:5:0: 3.49.13 leaves every installed struct layout and the exported symbol set
# untouched vs 3.49.12; the 2 GB (_stat64) and UTF-8 export widenings are _WIN32-only,
# and INTsys widened on POSIX (#600) but sits in no installed struct. Stays soname
# .so.3; bump revision. x32 alone changes layout (LLint was 4 bytes there, #524), and
# is not a release architecture.
# 3:6:0: 3.49.14 grows two installed structs from the inside (htsoptstate sits mid-
# httrackp, htsblk mid-lien_back), so the fields after them shift: httrackp.cookies_file
# +8, lien_back.http11 +48. Soname deliberately stays .so.3: the shifted fields are
# 3.49-era additions no external consumer reaches, and a libhttrack3 rename isn't worth it.
# (3:0:0 was the htsblk mime-buffer widening, the ABI break that moved .so.2 -> .so.3.)
VERSION_INFO="3:5:0"
VERSION_INFO="3:6:0"
AM_MAINTAINER_MODE
AC_USE_SYSTEM_EXTENSIONS
@@ -67,10 +66,11 @@ AC_SUBST(LT_CV_OBJDIR,$lt_cv_objdir)
AC_SUBST(VERSION_INFO)
### Default CFLAGS
# No -Wdeclaration-after-statement: nothing sets -std=, so this builds as gnu17.
DEFAULT_CFLAGS="-Wall -Wformat -Wformat-security \
-Wmultichar -Wwrite-strings -Wcast-qual -Wcast-align \
-Wstrict-prototypes -Wmissing-prototypes \
-Wmissing-declarations -Wdeclaration-after-statement \
-Wmissing-declarations \
-Wpointer-arith -Wsequence-point -Wnested-externs \
-D_REENTRANT"
AC_SUBST(DEFAULT_CFLAGS)
@@ -78,27 +78,30 @@ DEFAULT_LDFLAGS=""
AC_SUBST(DEFAULT_LDFLAGS)
### Additional flags (if supported)
AX_CHECK_COMPILE_FLAG([-Wparentheses], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wparentheses"])
AX_CHECK_COMPILE_FLAG([-Winit-self], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Winit-self"])
AX_CHECK_COMPILE_FLAG([-Wunused-but-set-parameter], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wunused-but-set-parameter"])
AX_CHECK_COMPILE_FLAG([-Waddress], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Waddress"])
AX_CHECK_COMPILE_FLAG([-Wuninitialized], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wuninitialized"])
AX_CHECK_COMPILE_FLAG([-Wformat=2], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wformat=2"])
AX_CHECK_COMPILE_FLAG([-Wformat-nonliteral], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wformat-nonliteral"])
AX_CHECK_COMPILE_FLAG([-Wmissing-parameter-type], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wmissing-parameter-type"])
AX_CHECK_COMPILE_FLAG([-Wold-style-definition], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wold-style-definition"])
AX_CHECK_COMPILE_FLAG([-Wignored-qualifiers], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wignored-qualifiers"])
# -Werror on probes: exit-status-only checks let clang's warn-on-unknown-flag through.
AX_CHECK_COMPILE_FLAG([-Wparentheses], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wparentheses"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Winit-self], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Winit-self"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Wunused-but-set-parameter], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wunused-but-set-parameter"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Waddress], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Waddress"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Wuninitialized], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wuninitialized"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Wformat=2], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wformat=2"], [], [-Werror])
# -Wformat-nonliteral needs -Wformat in the probe or gcc rejects it as ignored.
AX_CHECK_COMPILE_FLAG([-Wformat-nonliteral], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wformat-nonliteral"], [], [-Werror -Wformat])
AX_CHECK_COMPILE_FLAG([-Wmissing-parameter-type], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wmissing-parameter-type"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Wold-style-definition], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wold-style-definition"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Wignored-qualifiers], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wignored-qualifiers"], [], [-Werror])
# Make htssafe.h's pointer-dest 'warning' attribute a hard error in our build
# (migration is at zero; a new char* dest is a regression). gcc/clang each take
# only their own spelling; downstream keeps the plain warning, not a build break.
AX_CHECK_COMPILE_FLAG([-Werror=attribute-warning], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Werror=attribute-warning"])
AX_CHECK_COMPILE_FLAG([-Werror=user-defined-warnings], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Werror=user-defined-warnings"])
AX_CHECK_COMPILE_FLAG([-fstrict-aliasing -Wstrict-aliasing], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstrict-aliasing -Wstrict-aliasing"])
AX_CHECK_COMPILE_FLAG([-Werror=attribute-warning], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Werror=attribute-warning"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Werror=user-defined-warnings], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Werror=user-defined-warnings"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-fstrict-aliasing -Wstrict-aliasing], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstrict-aliasing -Wstrict-aliasing"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-fstack-protector-strong], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstack-protector-strong"],
[AX_CHECK_COMPILE_FLAG([-fstack-protector], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstack-protector"])])
AX_CHECK_COMPILE_FLAG([-fstack-clash-protection], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstack-clash-protection"])
AX_CHECK_COMPILE_FLAG([-fcf-protection], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fcf-protection"])
AX_CHECK_LINK_FLAG([-Wl,--discard-all], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,--discard-all"])
[AX_CHECK_COMPILE_FLAG([-fstack-protector], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstack-protector"], [], [-Werror])], [-Werror])
AX_CHECK_COMPILE_FLAG([-fstack-clash-protection], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstack-clash-protection"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-fcf-protection], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fcf-protection"], [], [-Werror])
# No --discard-all: it drops the local symbols naming every static function, so
# a trace misattributes them to the nearest surviving global. Costs 0.6% size.
AX_CHECK_LINK_FLAG([-Wl,--no-undefined], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,--no-undefined"])
AX_CHECK_LINK_FLAG([-Wl,-z,relro,-z,now], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,-z,relro,-z,now"])
AX_CHECK_LINK_FLAG([-Wl,-z,noexecstack], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,-z,noexecstack"])
@@ -120,13 +123,13 @@ AC_SUBST([LIBC_FORCE_LINK])
### PIE
CFLAGS_PIE=""
LDFLAGS_PIE=""
AX_CHECK_COMPILE_FLAG([-fpie -pie], [CFLAGS_PIE="-fpie -pie"])
AX_CHECK_COMPILE_FLAG([-fpie], [CFLAGS_PIE="-fpie"], [], [-Werror])
AX_CHECK_LINK_FLAG([-pie], [LDFLAGS_PIE="-pie"])
AC_SUBST([CFLAGS_PIE])
AC_SUBST([LDFLAGS_PIE])
## Export all symbols for backtraces
AX_CHECK_COMPILE_FLAG([-rdynamic], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -rdynamic"])
# Ties a crash trace from a stripped build back to its separate debug symbols.
AX_CHECK_LINK_FLAG([-Wl,--build-id], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,--build-id"])
### Check for -fvisibility=hidden support
gl_VISIBILITY
@@ -170,7 +173,7 @@ AC_CHECK_TYPE(sa_family_t, [], [AC_DEFINE([sa_family_t], [uint16_t], [sa_family_
AX_CHECK_ALIGNED_ACCESS_REQUIRED
# check for various headers
AC_CHECK_HEADERS([execinfo.h])
AC_CHECK_HEADERS([execinfo.h sys/ioctl.h])
### zlib
CHECK_ZLIB()

8
debian/changelog vendored
View File

@@ -1,3 +1,11 @@
httrack (3.49.14-1) unstable; urgency=medium
* New upstream release: WARC/WACZ archive output, Windows long-path support,
named -%F footer fields, and encoding fixes for non-ASCII project paths;
full list in history.txt.
-- Xavier Roche <xavier@debian.org> Fri, 24 Jul 2026 08:01:43 +0200
httrack (3.49.13-1) unstable; urgency=medium
* New upstream release: SOCKS5 and CONNECT proxy support, brotli and zstd

View File

@@ -5,7 +5,7 @@ Xavier Roche (xroche at httrack.com)
project leader
core engine, Windows/Linux GUI
Yann Philippot (yphilippot at lemel.fr)
for the java binary .class parser
past contributor (java binary .class parser)
With the help of:
Leto Kauler (molotov at tasmail.com)

View File

@@ -4,6 +4,23 @@ HTTrack Website Copier release history:
This file lists all changes and fixes that have been made for HTTrack
3.49-14
+ New: WARC/1.1 archive output (--warc), with a sorted CDXJ index (--warc-cdx) and WACZ packaging (--wacz), also available from webhttrack (#668)
+ New: -%F takes named footer fields such as {url}, {lastmodified}, {mime}, {charset} and {status} instead of a fixed layout (#667)
+ New: a command-line guide organized by task ships with the offline documentation (#649)
+ Fixed: on Windows, paths beyond MAX_PATH truncated files, and mirroring into a long or non-ASCII directory silently failed (#133)
+ Fixed: the top index showed mojibake for non-ASCII project names and categories on Windows (#216)
+ Fixed: webhttrack handed the engine the web form's charset rather than UTF-8, so a non-ASCII path mirrored into a mojibake directory (#629)
+ Fixed: a non-ASCII single -O left the logs and the cache in a mangled twin directory on Windows (#630)
+ Fixed: a path-ceiling truncation dropped the .delayed marker, losing the file (#623)
+ Fixed: an oversized -%F footer aborted the crawl instead of being skipped (#669)
+ Fixed: a rejected 206 resume could loop and lose the file rather than refetch it whole (#581)
+ Fixed: default-port stripping was scheme-blind and dropped explicit ports from https and ftp URLs, and a :80 written with leading zeros mangled the host (#627, #638)
+ Fixed: -K silently reset the -c socket count (#650)
+ Fixed: signed-shift undefined behaviour in the zip-repair local-header read (#639)
+ Changed: the offline documentation drops stale facts, gains an Android help page, and documents the filter wildcards and the real long option forms
+ Changed: multiple internal hardening, test and CI improvements
3.49-13
+ New: SOCKS5 proxy support, with scheme-aware -P URLs (socks5://, socks5h://, connect://) and plain HTTP tunneled through a CONNECT-only proxy (#563, #564)
+ New: decode brotli and zstd content codings, advertised over TLS only as browsers do (#556)

View File

@@ -300,6 +300,7 @@ ones it kept so the local copy browses offline. These options tune both halves.<
<tr><td><tt>--preserve (-%p), --disable-passwords (-%x)</tt></td><td>Leave HTML untouched (no rewriting), and strip passwords out of saved links.</td></tr>
<tr><td><tt>--extended-parsing (-%P), --parse-java (-j)</tt></td><td>Aggressive link discovery, and how much script content is parsed for links.</td></tr>
<tr><td><tt>--mime-html (-%M)</tt></td><td>Save the whole mirror as a single MIME-encapsulated <tt>.mht</tt> archive (<tt>index.mht</tt>).</td></tr>
<tr><td><tt>--single-file (-%Z), --single-file-max-size N</tt></td><td>Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded as <tt>data:</tt> URIs. Assets over the cap (10&nbsp;MB by default) keep their link, as do audio, video, and the links from one page to another.</td></tr>
<tr><td><tt>--index (-I), --build-top-index (-%i), --search-index (-%I)</tt></td><td>Build a per-mirror index, a top index across projects, and a searchable keyword index.</td></tr>
</table>
@@ -473,6 +474,27 @@ then:</small><br>
lifts the built-in caps; use it only against infrastructure you are allowed to
load.</small></p>
<h4>Save a WARC archive of the crawl</h4>
<p><tt>httrack https://example.com/ --warc --path mydir</tt><br>
<small>Writes a standard WARC/1.1 file (<tt>httrack-&lt;timestamp&gt;.warc.gz</tt>) in
the project folder alongside the browsable mirror, not instead of it. Set the name
with <tt>--warc-file NAME</tt> and split a large crawl with <tt>--warc-max-size N</tt>;
add <tt>--warc-cdx</tt> for a sorted CDXJ index, or <tt>--wacz</tt> to bundle the
archive, index and pages into one WACZ for replay tools such as
replayweb.page.</small></p>
<h4>Pages you can hand to someone as one file</h4>
<p><tt>httrack https://example.com/ --single-file --path mydir</tt><br>
<small>Rewrites each saved page after the crawl so its stylesheets, scripts,
images and fonts are embedded as <tt>data:</tt> URIs. The mirror stays a normal
browsable tree &mdash; links between pages remain relative, and the assets are
still on disk &mdash; but any single <tt>.html</tt> file can now be mailed or
archived on its own. Unlike <tt>--mime-html</tt>, which produces one
<tt>.mht</tt> archive that current browsers no longer open, the output is
ordinary HTML. Raise or lower the 10&nbsp;MB per-asset limit with
<tt>--single-file-max-size N</tt>; anything over it, plus audio and video, keeps
its link.</small></p>
<h4>HTTrack as a fetch tool</h4>
<p><tt>httrack --get https://host/file.bin --path tmp</tt><br>
<small><tt>--get</tt> fetches one file with cache, index, depth, cookies and robots

View File

@@ -132,7 +132,7 @@ Xavier Roche (xroche at httrack dot com)
for the main engine and Windows interface
and maintainer for v2.0 and v3.0
Yann Philippot (yphilippot at lemel dot fr)
for the java binary dot class parser
past contributor (java binary .class parser)
David Lawrie (dalawrie at lineone dot net)
Robert Lagadec (rlagadec at yahoo dot fr)
for checking both English & French translations

View File

@@ -90,35 +90,35 @@ offline browser : copy websites to a local directory</p>
] [ <b>-NN, --structure[=N]</b> ] [ <b>-%N,
--delayed-type-check</b> ] [ <b>-%D,
--cached-delayed-type-check</b> ] [ <b>-%M, --mime-html</b>
] [ <b>-%r, --warc</b> ] [ <b>-LN, --long-names[=N]</b> ] [
<b>-KN, --keep-links[=N]</b> ] [ <b>-x,
--replace-external</b> ] [ <b>-%x, --disable-passwords</b> ]
[ <b>-%q, --include-query-string</b> ] [ <b>-%g,
--strip-query</b> ] [ <b>-o, --generate-errors</b> ] [
<b>-X, --purge-old[=N]</b> ] [ <b>-%p, --preserve</b> ] [
<b>-%T, --utf8-conversion</b> ] [ <b>-bN, --cookies[=N]</b>
] [ <b>-%K, --cookies-file</b> ] [ <b>-%Y, --why</b> ] [
<b>-u, --check-type[=N]</b> ] [ <b>-j, --parse-java[=N]</b>
] [ <b>-sN, --robots[=N]</b> ] [ <b>-%h, --http-10</b> ] [
<b>-%k, --keep-alive</b> ] [ <b>-%z,
--disable-compression</b> ] [ <b>-%B, --tolerant</b> ] [
<b>-%s, --updatehack</b> ] [ <b>-%u, --urlhack</b> ] [
] [ <b>-%Z, --single-file</b> ] [ <b>-LN,
--long-names[=N]</b> ] [ <b>-KN, --keep-links[=N]</b> ] [
<b>-x, --replace-external</b> ] [ <b>-%x,
--disable-passwords</b> ] [ <b>-%q,
--include-query-string</b> ] [ <b>-%g, --strip-query</b> ] [
<b>-o, --generate-errors</b> ] [ <b>-X, --purge-old[=N]</b>
] [ <b>-%p, --preserve</b> ] [ <b>-%T, --utf8-conversion</b>
] [ <b>-bN, --cookies[=N]</b> ] [ <b>-%K, --cookies-file</b>
] [ <b>-%Y, --why</b> ] [ <b>-u, --check-type[=N]</b> ] [
<b>-j, --parse-java[=N]</b> ] [ <b>-sN, --robots[=N]</b> ] [
<b>-%h, --http-10</b> ] [ <b>-%k, --keep-alive</b> ] [
<b>-%z, --disable-compression</b> ] [ <b>-%B, --tolerant</b>
] [ <b>-%s, --updatehack</b> ] [ <b>-%u, --urlhack</b> ] [
<b>-%A, --assume</b> ] [ <b>-@iN, --protocol[=N]</b> ] [
<b>-%w, --disable-module</b> ] [ <b>-F, --user-agent</b> ] [
<b>-%R, --referer</b> ] [ <b>-%E, --from</b> ] [ <b>-%F,
--footer</b> ] [ <b>-%l, --language</b> ] [ <b>-%a,
--accept</b> ] [ <b>-%X, --headers</b> ] [ <b>-C,
--cache[=N]</b> ] [ <b>-k, --store-all-in-cache</b> ] [
<b>-%n, --do-not-recatch</b> ] [ <b>-%v, --display</b> ] [
<b>-Q, --do-not-log</b> ] [ <b>-q, --quiet</b> ] [ <b>-z,
--extra-log</b> ] [ <b>-Z, --debug-log</b> ] [ <b>-v,
--verbose</b> ] [ <b>-f, --file-log</b> ] [ <b>-f2,
--single-log</b> ] [ <b>-I, --index</b> ] [ <b>-%i,
--build-top-index</b> ] [ <b>-%I, --search-index</b> ] [
<b>-pN, --priority[=N]</b> ] [ <b>-S, --stay-on-same-dir</b>
] [ <b>-D, --can-go-down</b> ] [ <b>-U, --can-go-up</b> ] [
<b>-B, --can-go-up-and-down</b> ] [ <b>-a,
--stay-on-same-address</b> ] [ <b>-d,
<b>-%r, --warc</b> ] [ <b>-%n, --do-not-recatch</b> ] [
<b>-%v, --display</b> ] [ <b>-Q, --do-not-log</b> ] [ <b>-q,
--quiet</b> ] [ <b>-z, --extra-log</b> ] [ <b>-Z,
--debug-log</b> ] [ <b>-v, --verbose</b> ] [ <b>-f,
--file-log</b> ] [ <b>-f2, --single-log</b> ] [ <b>-I,
--index</b> ] [ <b>-%i, --build-top-index</b> ] [ <b>-%I,
--search-index</b> ] [ <b>-pN, --priority[=N]</b> ] [ <b>-S,
--stay-on-same-dir</b> ] [ <b>-D, --can-go-down</b> ] [
<b>-U, --can-go-up</b> ] [ <b>-B, --can-go-up-and-down</b> ]
[ <b>-a, --stay-on-same-address</b> ] [ <b>-d,
--stay-on-same-domain</b> ] [ <b>-l, --stay-on-same-tld</b>
] [ <b>-e, --go-everywhere</b> ] [ <b>-%H,
--debug-headers</b> ] [ <b>-%!,
@@ -649,13 +649,17 @@ don&rsquo;t wait) (--cached-delayed-type-check)</p></td></tr>
<td width="4%">
<p>-%r</p></td>
<p>-%Z</p></td>
<td width="5%"></td>
<td width="82%">
<p>write an ISO-28500 WARC/1.1 archive; --warc-file NAME
sets the output name (--warc)</p></td></tr>
<p>after the mirror, rewrite each saved page with its
stylesheets, scripts, images and fonts inlined as data:
URIs, so any page also opens on its own (links between pages
stay relative; audio and video stay links);
--single-file-max-size N caps each asset (default 10485760
bytes) (--single-file)</p></td></tr>
<tr valign="top" align="left">
<td width="9%"></td>
<td width="4%">
@@ -1142,6 +1146,20 @@ update before) (--cache[=N])</p></td></tr>
<td width="4%">
<p>-%r</p></td>
<td width="5%"></td>
<td width="82%">
<p>write an ISO-28500 WARC/1.1 archive; --warc-file NAME
sets the output name, --warc-max-size N rotates segments
past N bytes, --warc-cdx also writes a sorted CDXJ index,
--wacz packages it all as a WACZ file (--warc)</p></td></tr>
<tr valign="top" align="left">
<td width="9%"></td>
<td width="4%">
<p>-%n</p></td>
<td width="5%"></td>
<td width="82%">

View File

@@ -136,6 +136,13 @@ ${listid:build:LISTDEF_3}
<tr><td><input type="checkbox" name="nopurge" ${checked:nopurge}
title='${html:LANG_I1a}' onMouseOver="info('${html:LANG_I1a}'); return true" onMouseOut="info('&nbsp;'); return true"
> ${LANG_I57}</td></tr>
<tr><td><input type="checkbox" name="singlefile" ${checked:singlefile}
title='${html:LANG_SINGLEFILETIP}' onMouseOver="info('${html:LANG_SINGLEFILETIP}'); return true" onMouseOut="info('&nbsp;'); return true"
> ${LANG_SINGLEFILE}</td></tr>
<tr><td>${LANG_SINGLEFILEMAX}
<input name="singlefilemax" value="${singlefilemax}" size="12"
title='${html:LANG_SINGLEFILEMAXTIP}' onMouseOver="info('${html:LANG_SINGLEFILEMAXTIP}'); return true" onMouseOut="info('&nbsp;'); return true"
></td></tr>
</table>
<tr><td>

View File

@@ -103,7 +103,7 @@ ${LANG_Q3}
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK]"
<input type="submit" value="${LANG_OK}"
${do:output-mode:html-urlescaped}
onClick="if (confirm(str_replace(str_replace('${LANG_DIAL7}', '%20', ' '), '%0a', ' '))) { form.closeme.value=1; form.submit(); } return false;"
${do:output-mode:}

View File

@@ -103,6 +103,17 @@ ${do:end-if}
> ${LANG_I61}
<br><br>
<input type="checkbox" name="warc" ${checked:warc}
title='${html:LANG_WARCTIP}' onMouseOver="info('${html:LANG_WARCTIP}'); return true" onMouseOut="info('&nbsp;'); return true"
> ${LANG_WARC}
<br><br>
${LANG_WARCFILE}
<input name="warcfile" value="${warcfile}" size="40"
title='${html:LANG_WARCFILETIP}' onMouseOver="info('${html:LANG_WARCFILETIP}'); return true" onMouseOut="info('&nbsp;'); return true"
>
<br><br>
<input type="checkbox" name="norecatch" ${checked:norecatch}
title='${html:LANG_I5b}' onMouseOver="info('${html:LANG_I5b}'); return true" onMouseOut="info('&nbsp;'); return true"
> ${LANG_I34b}

View File

@@ -141,6 +141,10 @@ ${do:copy:KeepSlashes:keepslashes}
${do:copy:KeepQueryOrder:keepqueryorder}
${do:copy:StripQuery:stripquery}
${do:copy:StoreAllInCache:cache2}
${do:copy:Warc:warc}
${do:copy:WarcFile:warcfile}
${do:copy:SingleFile:singlefile}
${do:copy:SingleFileMaxSize:singlefilemax}
${do:copy:LogType:logtype}
${do:copy:UseHTTPProxyForFTP:ftpprox}
${do:copy:ProxyType:proxytype}

View File

@@ -121,9 +121,9 @@ httrack \
--quiet \
--build-top-index \
${test:todo:--mirror:--mirror:--mirror-wizard:--get:--mirrorlinks:--testlinks:--continue:--update}
${urls}
${test:filelist:-%L "}${filelist}${test:filelist:"}
--path "${html:path}/${html:projname}"
${unquoted:urls}
${test:filelist:-%L "}${arg:filelist}${test:filelist:"}
--path "${arg:path}/${arg:projname}"
\
${test:parseall:--near}
${test:link:--test}
@@ -131,7 +131,7 @@ httrack \
${test:htmlfirst::--priority=7}
\
${do:if-not-empty:BuildString}
--structure "${BuildString}"
--structure "${arg:BuildString}"
${do:end-if}
${test:build:-N0:-N0:-N1:-N2:-N3:-N4:-N5:-N100:-N101:-N102:-N103:-N104:-N105:-N99:-N199:}
\
@@ -150,29 +150,30 @@ ${do:end-if}
${test:travel3::--keep-links=0:--keep-links:--keep-links=3:--keep-links=4}
${test:windebug:--debug-headers}
\
${test:connexion:--sockets=}${connexion}
${test:connexion:--sockets=}${unquoted:connexion}
${test:ka:--keep-alive}
${test:timeout:--timeout=}${timeout}
${test:timeout:--timeout=}${unquoted:timeout}
${test:remt:--host-control=1}
${test:retry:--retries=}${retry}
${test:rate:--min-rate=}${rate}
${test:retry:--retries=}${unquoted:retry}
${test:rate:--min-rate=}${unquoted:rate}
${test:rems:--host-control=2}
\
${test:depth:--depth=}${depth}
${test:depth2:--ext-depth=}${depth2}
${test:maxhtml:--max-files=,}${maxhtml}
${test:othermax:--max-files=}${othermax}
${test:sizemax:--max-files=}${sizemax}
${test:pausebytes:--max-pause=}${pausebytes}
${test:maxtime:--max-time=}${maxtime}
${test:maxrate:--max-rate=}${maxrate}
${test:maxconn:--connection-per-second=}${maxconn}
${test:maxlinks:--advanced-maxlinks=}${maxlinks}
${test:depth:--depth=}${unquoted:depth}
${test:depth2:--ext-depth=}${unquoted:depth2}
${/* -m<n> resets the html limit, so the bare form must precede the -m,<n> one */}
${test:othermax:--max-files=}${unquoted:othermax}
${test:maxhtml:--max-files=,}${unquoted:maxhtml}
${test:sizemax:--max-size=}${unquoted:sizemax}
${test:pausebytes:--max-pause=}${unquoted:pausebytes}
${test:maxtime:--max-time=}${unquoted:maxtime}
${test:maxrate:--max-rate=}${unquoted:maxrate}
${test:maxconn:--connection-per-second=}${unquoted:maxconn}
${test:maxlinks:--advanced-maxlinks=}${unquoted:maxlinks}
\
--user-agent "${html:user}"
--footer "${html:footer}"
--user-agent "${arg:user}"
--footer "${arg:footer}"
\
${url2}
${unquoted:url2}
\
${test:cookies:--cookies=0:}
${test:parsejava:--parse-java=0:}
@@ -181,18 +182,22 @@ ${do:end-if}
${test:keepwww:--keep-www-prefix}
${test:keepslashes:--keep-double-slashes}
${test:keepqueryorder:--keep-query-order}
${test:cookiesfile:--cookies-file "}${html:cookiesfile}${test:cookiesfile:"}
${test:pausefiles:--pause "}${pausefiles}${test:pausefiles:"}
${test:stripquery:--strip-query "}${html:stripquery}${test:stripquery:"}
${test:cookiesfile:--cookies-file "}${arg:cookiesfile}${test:cookiesfile:"}
${test:pausefiles:--pause "}${arg:pausefiles}${test:pausefiles:"}
${test:stripquery:--strip-query "}${arg:stripquery}${test:stripquery:"}
${test:toler:--tolerant}
${test:http10:--http-10}
${test:cache2:--store-all-in-cache}
${test:warc:--warc}
${test:warcfile:--warc-file "}${arg:warcfile}${test:warcfile:"}
${test:singlefile:--single-file}
${test:singlefilemax:--single-file-max-size=}${unquoted:singlefilemax}
${test:norecatch:--do-not-recatch}
${test:logf:--single-log}
${test:logtype:::--extra-log:--debug-log}
${test:index:--index=0:}
${test:index2:--search-index=0:--search-index}
${test:prox:--proxy "}${do:if-not-empty:prox}${test:proxytype::socks5:connect}${test:proxytype:\3A//}${do:end-if}${do:output-mode:html}${prox}${test:prox:\3A}${portprox}${test:prox:"}
${test:prox:--proxy "}${do:if-not-empty:prox}${test:proxytype::socks5:connect}${test:proxytype:\3A//}${do:end-if}${do:output-mode:html}${arg:prox}${test:prox:\3A}${arg:portprox}${test:prox:"}
${test:ftpprox:--httpproxy-ftp=0:--httpproxy-ftp}
</textarea>
@@ -211,7 +216,7 @@ ParseAll=${ztest:parseall:0:1}
HTMLFirst=${ztest:htmlfirst:0:1}
Cache=${ztest:cache:0:1}
NoRecatch=${ztest:norecatch:0:1}
Dos=${dos
Dos=${dos}
Index=${ztest:index:0:1}
WordIndex=${ztest:index2:0:1}
Log=${ztest:logf:0:1:2}
@@ -237,6 +242,10 @@ KeepSlashes=${ztest:keepslashes:0:1}
KeepQueryOrder=${ztest:keepqueryorder:0:1}
StripQuery=${stripquery}
StoreAllInCache=${ztest:cache2:0:1}
Warc=${ztest:warc:0:1}
WarcFile=${warcfile}
SingleFile=${ztest:singlefile:0:1}
SingleFileMaxSize=${singlefilemax}
LogType=${logtype}
UseHTTPProxyForFTP=${ztest:ftpprox:0:1}
ProxyType=${proxytype}

View File

@@ -713,7 +713,7 @@ Disconnect when finished
LANG_J17
Disconnect modem on completion
LANG_K1
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
LANG_K2
About WinHTTrack Website Copier
LANG_K3
@@ -1034,3 +1034,19 @@ LANG_STRIPQUERY
Strip query keys:
LANG_STRIPQUERYTIP
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
LANG_WARC
Write a WARC archive of the crawl
LANG_WARCTIP
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
LANG_WARCFILE
WARC archive name:
LANG_WARCFILETIP
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
LANG_SINGLEFILE
Inline assets as data: URIs (self-contained pages)
LANG_SINGLEFILETIP
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
LANG_SINGLEFILEMAX
Largest inlined asset (bytes):
LANG_SINGLEFILEMAXTIP
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Ïðåêðàòè âðúçêàòà ñëåä êðàÿ íà îïåðàöèÿòà
Disconnect modem on completion
Ñëåä êðàÿ íà îïåðàöèÿòà ðàçêà÷è ìîäåìà
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ìîëÿ óâåäîìåòå íè çà âñÿêà ãðåøêà èëè ïðîáëåì)\r\n\r\nÐàçðàáîòêà:\r\nÈíòåðôåéñ (Windows): Xavier Roche\r\nÏàÿê: Xavier Roche\r\nJava Ðàçáîðíè Êëàñîâå: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche è äðóãè ñúòðóäíèöè\r\nÌÍÎÃÎ ÁËÀÃÎÄÀÐÍÎÑÒÈ çà Áúëãàðñêèÿò ïðåâîä íà:\r\nÈëèÿ Ëèíäîâ (ilia@infomat-bg.com)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ìîëÿ óâåäîìåòå íè çà âñÿêà ãðåøêà èëè ïðîáëåì)\r\n\r\nÐàçðàáîòêà:\r\nÈíòåðôåéñ (Windows): Xavier Roche\r\nÏàÿê: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche è äðóãè ñúòðóäíèöè\r\nÌÍÎÃÎ ÁËÀÃÎÄÀÐÍÎÑÒÈ çà Áúëãàðñêèÿò ïðåâîä íà:\r\nÈëèÿ Ëèíäîâ (ilia@infomat-bg.com)
About WinHTTrack Website Copier
Çà WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Ïðåìàõâàíå íà êëþ÷îâå îò çàÿâêàòà:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Êëþ÷îâå îò çàÿâêàòà, ðàçäåëåíè ñúñ çàïåòàÿ, êîèòî äà ñå ïðåìàõíàò îò èìåòî íà çàïèñàíèÿ ôàéë (íàïðèìåð sid,utm_source).
Write a WARC archive of the crawl
Çàïèñâàíå íà WARC àðõèâ íà îáõîæäàíåòî
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Çàïèñâàíå íà âñåêè èçòåãëåí îòãîâîð è â WARC/1.1 àðõèâ ïî ISO-28500, äî îãëåäàëîòî.
WARC archive name:
Èìå íà WARC àðõèâà:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Íåçàäúëæèòåëíî áàçîâî èìå çà WARC àðõèâà; îñòàâåòå ïðàçíî çà àâòîìàòè÷íî èìåíóâàíå â èçõîäíàòà äèðåêòîðèÿ.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Desconectar al terminar la operación
Disconnect modem on completion
Desconectar el modem al terminar
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(notifíquenos fallos o problemas)\r\n\r\nDesarrollo:\r\nInterface (Windows): Xavier Roche\r\nMotor: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Spanish translations to:\r\nJuan Pablo Barrio Lera (Universidad de León)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(notifíquenos fallos o problemas)\r\n\r\nDesarrollo:\r\nInterface (Windows): Xavier Roche\r\nMotor: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Spanish translations to:\r\nJuan Pablo Barrio Lera (Universidad de León)
About WinHTTrack Website Copier
Acerca de...
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Eliminar claves de query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Claves de query string, separadas por comas, que se eliminarán del nombre de los archivos guardados (p. ej. sid,utm_source).
Write a WARC archive of the crawl
Escribir un archivo WARC del rastreo
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Guardar también cada respuesta descargada en un archivo WARC/1.1 ISO-28500, junto a la réplica.
WARC archive name:
Nombre del archivo WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nombre base opcional para el archivo WARC; déjelo en blanco para nombrarlo automáticamente en el directorio de salida.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Odpojit po dokonèení
Disconnect modem on completion
Odpojit modem po dokonèení
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Prosíme o podání hlášení o jakýchkoliv chybách)\r\n\r\nVývoj:\r\nRozhraní (Windows): Xavier Roche\r\nPavouk: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche a ostatní, kdo pøispìli\r\nÈeský pøeklad:\r\nAntonín Matìjèík (matejcik@volny.cz)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Prosíme o podání hlášení o jakýchkoliv chybách)\r\n\r\nVývoj:\r\nRozhraní (Windows): Xavier Roche\r\nPavouk: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche a ostatní, kdo pøispìli\r\nÈeský pøeklad:\r\nAntonín Matìjèík (matejcik@volny.cz)
About WinHTTrack Website Copier
O programu WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Odebrat klíèe dotazu:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Klíèe dotazu oddìlené èárkami, které se vynechají z pojmenování ukládaných souborù (napø. sid,utm_source).
Write a WARC archive of the crawl
Zapsat archiv WARC z procházení
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Uložit také každou staženou odpovìï do archivu WARC/1.1 podle ISO-28500 vedle zrcadla.
WARC archive name:
Název archivu WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Volitelný základní název archivu WARC; ponechte prázdné pro automatické pojmenování ve výstupním adresáøi.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
完成時切斷連線
Disconnect modem on completion
完成時切斷數據機
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(有任何程序錯誤或問題請聯絡我們)\r\n\r\n開發:\r\n界面設計 (Windows): Xavier Roche\r\n分析引擎: Xavier Roche\r\nJava分析引擎: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\n感謝:\r\nDavid Hing Cheong Hung (DAVEHUNG@mtr.com.hk) 提供翻譯
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(有任何程序錯誤或問題請聯絡我們)\r\n\r\n開發:\r\n界面設計 (Windows): Xavier Roche\r\n分析引擎: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\n感謝:\r\nDavid Hing Cheong Hung (DAVEHUNG@mtr.com.hk) 提供翻譯
About WinHTTrack Website Copier
關於WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
移除查詢鍵:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
以逗號分隔的查詢鍵,將其從儲存檔案的命名中移除 (例如 sid,utm_source)。
Write a WARC archive of the crawl
寫入此次抓取的 WARC 封存檔
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
同時將每個已擷取的回應儲存為 ISO-28500 WARC/1.1 封存檔,置於鏡像網站旁。
WARC archive name:
WARC 封存檔名稱:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
WARC 封存檔的選用基本名稱;留空則於輸出目錄中自動命名。

View File

@@ -634,8 +634,8 @@ Disconnect when finished
完成时断掉连接
Disconnect modem on completion
完成时断掉调制解调器
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(有任何程序错误或问题请联系我们)\r\n\r\n开发:\r\n界面设计 (Windows): Xavier Roche\r\n解析引擎: Xavier Roche\r\nJava解析引擎: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\n感谢:\r\nRobert Lagadec (rlagadec@yahoo.fr) 提供翻译技巧
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(有任何程序错误或问题请联系我们)\r\n\r\n开发:\r\n界面设计 (Windows): Xavier Roche\r\n解析引擎: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\n感谢:\r\nRobert Lagadec (rlagadec@yahoo.fr) 提供翻译技巧
About WinHTTrack Website Copier
关于WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
剥离查询键:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
用逗号分隔的查询键,将其从保存文件的命名中删除 (例如 sid,utm_source)。
Write a WARC archive of the crawl
写入本次抓取的 WARC 归档
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
同时将每个已获取的响应保存为 ISO-28500 WARC/1.1 归档,置于镜像站点旁边。
WARC archive name:
WARC 归档名称:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
WARC 归档的可选基本名称;留空则在输出目录中自动命名。

View File

@@ -636,8 +636,8 @@ Disconnect when finished
Prekinuti vezu kada bude gotovo
Disconnect modem on completion
Po dovršetku odvojiti modemsku vezu
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Molimo da nas obavijestite o svim pogreškama i poteškoæama)\r\n\r\nRazvoj:\r\nSuèelje (Windows): Xavier Roche\r\nPauk: Xavier Roche\r\nRazrediRašèlanjivaèaJave: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche i drugi suradnici\r\nPUNO HVALA za prijevodne preporuke upuæujemo:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Molimo da nas obavijestite o svim pogreškama i poteškoæama)\r\n\r\nRazvoj:\r\nSuèelje (Windows): Xavier Roche\r\nPauk: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche i drugi suradnici\r\nPUNO HVALA za prijevodne preporuke upuæujemo:\r\nRobert Lagadec (rlagadec@yahoo.fr)
About WinHTTrack Website Copier
O programu WinHTTrack Website Copier
Please visit our Web page
@@ -958,3 +958,11 @@ Strip query keys:
Ukloniti kljuèeve upita:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Zarezom odvojeni kljuèevi upita koji se izostavljaju iz naziva spremljenih datoteka (npr. sid,utm_source).
Write a WARC archive of the crawl
Zapi¹i WARC arhivu obilaska
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Spremi i svaki preuzeti odgovor u ISO-28500 WARC/1.1 arhivu, uz zrcalo.
WARC archive name:
Naziv WARC arhive:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Neobavezni osnovni naziv WARC arhive; ostavite prazno za automatsko imenovanje u izlaznom direktoriju.

View File

@@ -652,8 +652,8 @@ Disconnect when finished
Afbryd forbindelsen når overførslen er færdig
Disconnect modem on completion
Afbryd modem når overførslen er færdig
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(finder du fejl eller opstår der problemer så kontakt os venligst)\r\n\r\nUdvikling:\r\nBrugerflade (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche og andre bidragydere\r\nMange tak for dansk oversættelse til:\r\nJesper Bramm (bramm@get2net.dk)\r\nscootergrisen
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(finder du fejl eller opstår der problemer så kontakt os venligst)\r\n\r\nUdvikling:\r\nBrugerflade (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche og andre bidragydere\r\nMange tak for dansk oversættelse til:\r\nJesper Bramm (bramm@get2net.dk)\r\nscootergrisen
About WinHTTrack Website Copier
Om WinHTTrack Website Copier
Please visit our Web page
@@ -1004,3 +1004,11 @@ Strip query keys:
Fjern forespørgselsnøgler:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommaseparerede forespørgselsnøgler, der udelades i navngivningen af gemte filer (f.eks. sid,utm_source).
Write a WARC archive of the crawl
Skriv et WARC-arkiv af gennemsøgningen
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Gem også hvert hentet svar i et ISO-28500 WARC/1.1-arkiv ved siden af spejlet.
WARC archive name:
Navn på WARC-arkiv:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Valgfrit basisnavn til WARC-arkivet; lad feltet stå tomt for automatisk navngivning i outputmappen.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Nach der Aktion Verbindung trennen
Disconnect modem on completion
Modemverbindung trennen, wenn fertig
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Bitte benachrichtigen Sie uns über Fehler und Probleme)\r\n\r\nEntwicklung:\r\nOberfläche (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for German translations to:\r\nRainer Klueting (rainer@klueting.de)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Bitte benachrichtigen Sie uns über Fehler und Probleme)\r\n\r\nEntwicklung:\r\nOberfläche (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for German translations to:\r\nRainer Klueting (rainer@klueting.de)
About WinHTTrack Website Copier
Über WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Query-Schlüssel entfernen:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommagetrennte Query-Schlüssel, die bei der Benennung gespeicherter Dateien entfallen (z. B. sid,utm_source).
Write a WARC archive of the crawl
WARC-Archiv des Crawls schreiben
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Jede heruntergeladene Antwort zusätzlich in einem ISO-28500-WARC/1.1-Archiv neben dem Spiegel speichern.
WARC archive name:
Name des WARC-Archivs:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Optionaler Basisname für das WARC-Archiv; leer lassen, um es automatisch im Ausgabeverzeichnis zu benennen.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Lahuta ühendus, kui on lõpetatud
Disconnect modem on completion
Lahuta modem lõpetamisel
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Palun teata meile igast veast või probleemist)\r\n\r\nArendajad:\r\nKasutajaliides (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nEestikeelne tõlge: Tõnu Virma
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Palun teata meile igast veast või probleemist)\r\n\r\nArendajad:\r\nKasutajaliides (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nEestikeelne tõlge: Tõnu Virma
About WinHTTrack Website Copier
Info WinHTTrack Website Copier'i kohta
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Eemalda päringuvõtmed:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Komadega eraldatud päringuvõtmed, mis jäetakse salvestatud faili nimest välja (nt sid,utm_source).
Write a WARC archive of the crawl
Kirjuta läbimise WARC-arhiiv
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Salvesta iga alla laaditud vastus ka ISO-28500 WARC/1.1 arhiivi peegli kõrvale.
WARC archive name:
WARC-arhiivi nimi:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
WARC-arhiivi valikuline põhinimi; jäta tühjaks, et see väljundkataloogis automaatselt nimetada.

View File

@@ -652,8 +652,8 @@ Disconnect when finished
Disconnect when finished
Disconnect modem on completion
Disconnect modem on completion
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
About WinHTTrack Website Copier
About WinHTTrack Website Copier
Please visit our Web page
@@ -1004,3 +1004,19 @@ Strip query keys:
Strip query keys:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Write a WARC archive of the crawl
Write a WARC archive of the crawl
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
WARC archive name:
WARC archive name:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Inline assets as data: URIs (self-contained pages)
Inline assets as data: URIs (self-contained pages)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Largest inlined asset (bytes):
Largest inlined asset (bytes):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.

View File

@@ -636,8 +636,8 @@ Disconnect when finished
Katkaise yhteys, kun on valmista
Disconnect modem on completion
Katkaise modeemiyhteys, kun on valmista
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Kerro meille bugeista ja ongelmista)\r\n\r\nKehitys:\r\nKäyttöliittymä (Windows): Xavier Roche\r\nNettirobotti: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C) 1998-2003 Xavier Roche ja muut avustajat\r\nSuomentanut Mika Kähkönen 22.-24.7.2005\r\nmika.kahkonen@mbnet.fi\r\nhttp://koti.mbnet.fi/kahoset
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Kerro meille bugeista ja ongelmista)\r\n\r\nKehitys:\r\nKäyttöliittymä (Windows): Xavier Roche\r\nNettirobotti: Xavier Roche\r\n\r\n(C) 1998-2003 Xavier Roche ja muut avustajat\r\nSuomentanut Mika Kähkönen 22.-24.7.2005\r\nmika.kahkonen@mbnet.fi\r\nhttp://koti.mbnet.fi/kahoset
About WinHTTrack Website Copier
Tietoja WinHTTrack Website Copier
Please visit our Web page
@@ -958,3 +958,11 @@ Strip query keys:
Poista kyselyavaimet:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Pilkuin erotellut kyselyavaimet, jotka jätetään pois tallennettujen tiedostojen nimeämisestä (esim. sid,utm_source).
Write a WARC archive of the crawl
Kirjoita imuroinnin WARC-arkisto
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Tallenna myös jokainen noudettu vastaus ISO-28500 WARC/1.1 -arkistoon peilin viereen.
WARC archive name:
WARC-arkiston nimi:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Valinnainen WARC-arkiston perusnimi; jätä tyhjäksi, jotta se nimetään automaattisesti tulostehakemistoon.

View File

@@ -644,8 +644,8 @@ Disconnect when finished
Déconnecter à la fin de l'opération
Disconnect modem on completion
Déconnecter le modem lorsque l'opération sera finie
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(signalez-nous tout bug ou problème)\r\n\r\nDéveloppement:\r\nInterface (Windows): Xavier Roche\r\nMoteur: Xavier Roche\r\nParseurClassesJava: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMERCI pour la relecture à:\r\nRobert Lagadec
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(signalez-nous tout bug ou problème)\r\n\r\nDéveloppement:\r\nInterface (Windows): Xavier Roche\r\nMoteur: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMERCI pour la relecture à:\r\nRobert Lagadec
About WinHTTrack Website Copier
A propos de WinHTTrack Website Copier
Please visit our Web page
@@ -1004,3 +1004,19 @@ Strip query keys:
Supprimer les clés de query string :
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Clés de query string à retirer du nommage des fichiers enregistrés, séparées par des virgules (par ex. sid,utm_source).
Write a WARC archive of the crawl
Écrire une archive WARC du crawl
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Enregistrer aussi chaque réponse téléchargée dans une archive WARC/1.1 (ISO-28500), à côté du miroir.
WARC archive name:
Nom de l'archive WARC :
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nom de base optionnel pour l'archive WARC ; laissez vide pour le générer automatiquement dans le répertoire de sortie.
Inline assets as data: URIs (self-contained pages)
Intégrer les ressources en URI data: (pages autonomes)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Une fois le miroir terminé, réécrire chaque page enregistrée avec ses feuilles de style, scripts, images et polices intégrés, afin qu'une page puisse aussi être ouverte seule.
Largest inlined asset (bytes):
Taille maximale d'une ressource intégrée (octets) :
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Au-delà de cette taille, la ressource reste un lien ordinaire ; laissez vide pour la valeur par défaut de 10485760 octets.

View File

@@ -636,8 +636,8 @@ Disconnect when finished
Αποσύνδεση στο τέλος
Disconnect modem on completion
Αποσύνδεση του modem στο τέλος
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Παρακαλώ να μας ενημερώσετε για κάθε πρόβλημα ή bug)\r\n\r\nΑνάπτυξη:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Greek translations to:\r\nMichael Papadakis (mikepap at freemail dot gr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Παρακαλώ να μας ενημερώσετε για κάθε πρόβλημα ή bug)\r\n\r\nΑνάπτυξη:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Greek translations to:\r\nMichael Papadakis (mikepap at freemail dot gr)
About WinHTTrack Website Copier
Σχετικά με το WinHTTrack Website Copier
Please visit our Web page
@@ -958,3 +958,11 @@ Strip query keys:
Αφαίρεση κλειδιών ερωτήματος:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Κλειδιά ερωτήματος χωρισμένα με κόμμα, που θα αφαιρεθούν από την ονομασία των αποθηκευμένων αρχείων (π.χ. sid,utm_source).
Write a WARC archive of the crawl
Εγγραφή αρχείου WARC της ανίχνευσης
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Αποθήκευση κάθε ληφθείσας απόκρισης και σε αρχείο WARC/1.1 ISO-28500, δίπλα στο είδωλο.
WARC archive name:
Όνομα αρχείου WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Προαιρετικό βασικό όνομα για το αρχείο WARC. Αφήστε το κενό για αυτόματη ονομασία στον κατάλογο εξόδου.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Disconnetti alla fine
Disconnect modem on completion
Disconnetti il modem quando il mirror è finito
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(informateci degli errori o problemi)\r\n\rSviluppo:\r\nInterfacccia: Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Italian translations to:\r\nWitold Krakowski (wkrakowski@libero.it)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(informateci degli errori o problemi)\r\n\rSviluppo:\r\nInterfacccia: Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Italian translations to:\r\nWitold Krakowski (wkrakowski@libero.it)
About WinHTTrack Website Copier
Informazioni su WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Rimuovi chiavi della query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Chiavi della query string, separate da virgole, da rimuovere dai nomi dei file salvati (ad es. sid,utm_source).
Write a WARC archive of the crawl
Scrivi un archivio WARC della scansione
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Salva anche ogni risposta scaricata in un archivio WARC/1.1 ISO-28500, accanto al mirror.
WARC archive name:
Nome dell'archivio WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nome di base facoltativo per l'archivio WARC; lascia vuoto per assegnarlo automaticamente nella directory di output.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
終了したら接続を切断する
Disconnect modem on completion
完了したらモデムとの接続を切断する
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(バグまたは問題についてわれわれに知らせてください)\r\n\r\nDevelopment:\r\nInterface(Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nJapanese translation :TAPKAL(nakataka@mars.dti.ne.jp)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(バグまたは問題についてわれわれに知らせてください)\r\n\r\nDevelopment:\r\nInterface(Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nJapanese translation :TAPKAL(nakataka@mars.dti.ne.jp)
About WinHTTrack Website Copier
WinHTTrackについて
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
削除するクエリキー:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
保存ファイル名の生成から除外するクエリキーをカンマ区切りで指定します (例: sid,utm_source)。
Write a WARC archive of the crawl
クロールの WARC アーカイブを書き出す
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
取得した各レスポンスを ISO-28500 WARC/1.1 アーカイブとしてミラーの隣にも保存します。
WARC archive name:
WARC アーカイブ名:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
WARC アーカイブの任意のベース名。空欄にすると出力ディレクトリ内で自動的に名前が付けられます。

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Äèñêîíåêòèð༠ñå êîãà <20>å çàâðøè
Disconnect modem on completion
Äèñêîíåêòèð༠ãî ìîäåìîò íà êðà¼
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\Âå ìîëèìå èçâåñòåòå íå çà áèëî êàêâà ãðåøêà èëè ïðîáëåì)\r\n\r\nDevelopment:\r\nÈíòåðôå¼ñ (Windows):Xavier Roche\r\nSpider:Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche è äðóãè\r\nÌÍÎÃÓ ÁËÀÃÎÄÀÐÍÎÑÒ çà ìàêåäîíñêèîò ïðåâîä íà:\r\nÀëåêñàíäàð Ñàâè<C3A2> (aleks@macedonia.eu.org)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\Âå ìîëèìå èçâåñòåòå íå çà áèëî êàêâà ãðåøêà èëè ïðîáëåì)\r\n\r\nDevelopment:\r\nÈíòåðôå¼ñ (Windows):Xavier Roche\r\nSpider:Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche è äðóãè\r\nÌÍÎÃÓ ÁËÀÃÎÄÀÐÍÎÑÒ çà ìàêåäîíñêèîò ïðåâîä íà:\r\nÀëåêñàíäàð Ñàâè<C3A2> (aleks@macedonia.eu.org)
About WinHTTrack Website Copier
Çà WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
¾âáâàÐÝØ ÚÛãçÕÒØ ÞÔ ÑÐàÐúÕâÞ:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
ºÛãçÕÒØ ÞÔ ÑÐàÐúÕâÞ, ÞÔÔÕÛÕÝØ áÞ ×ÐߨàÚÐ, èâÞ áÕ ÞâáâàÐÝãÒÐÐâ ÞÔ ØÜÕâÞ ÝÐ ×ÐçãÒÐÝÐâÐ ÔÐâÞâÕÚÐ (ÝÐ ßàØÜÕà sid,utm_source).
Write a WARC archive of the crawl
·ÐßØèØ WARC ÐàåØÒÐ ÝÐ ßàÕÑÐàãÒÐúÕâÞ
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
·ÐçãÒÐø ÓÞ áÕÚÞø ßàÕ×ÕÜÕÝ ÞÔÓÞÒÞà Ø ÒÞ ISO-28500 WARC/1.1 ÐàåØÒÐ, ßÞÚàÐø ÞÓÛÕÔÐÛÞâÞ.
WARC archive name:
¸ÜÕ ÝÐ WARC ÐàåØÒÐâÐ:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
¸×ÑÞàÝÞ ÞáÝÞÒÝÞ ØÜÕ ×Ð WARC ÐàåØÒÐâÐ; ÞáâÐÒÕâÕ ßàÐ×ÝÞ ×Ð ÐÒâÞÜÐâáÚÞ ØÜÕÝãÒÐúÕ ÒÞ Ø×ÛÕ×ÝØÞâ ÔØàÕÚâÞàØãÜ.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Vonalbontás, ha kész
Disconnect modem on completion
Modem leválasztása, ha kész
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Kérjük, jelezzen nekünk bármilyen hibát vagy problémát)\r\n\r\nFejlesztés:\r\nKezelõfelület (Windows): Xavier Roche\r\nIndexelés: Xavier Roche\r\nJavaElemzõOsztályok: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nEZER KÖSZÖNET a magyar fordításért:\r\nHerczeg József Tamásnak (hdodi@freemail.hu)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Kérjük, jelezzen nekünk bármilyen hibát vagy problémát)\r\n\r\nFejlesztés:\r\nKezelõfelület (Windows): Xavier Roche\r\nIndexelés: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nEZER KÖSZÖNET a magyar fordításért:\r\nHerczeg József Tamásnak (hdodi@freemail.hu)
About WinHTTrack Website Copier
WinHTTrack webhely másoló névjegye
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Lekérdezési kulcsok eltávolítása:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Vesszõvel elválasztott lekérdezési kulcsok, amelyeket el kell hagyni a mentett fájl elnevezésébõl (pl. sid,utm_source).
Write a WARC archive of the crawl
A bejárás WARC archívumának írása
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Minden letöltött válasz mentése ISO-28500 WARC/1.1 archívumba is, a tükör mellé.
WARC archive name:
WARC archívum neve:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
A WARC archívum opcionális alapneve; hagyja üresen az automatikus elnevezéshez a kimeneti könyvtárban.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Indien gedaan verbinding verbreken
Disconnect modem on completion
Indien gedaan modemverbinding verbreken
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ons fouten en problemen mede te delen)\r\n\r\nOntwikkeling:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Dutch translations to:\r\nRudi Ferrari (Wyando@netcologne.de)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ons fouten en problemen mede te delen)\r\n\r\nOntwikkeling:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Dutch translations to:\r\nRudi Ferrari (Wyando@netcologne.de)
About WinHTTrack Website Copier
Over WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Query-sleutels verwijderen:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Door komma's gescheiden query-sleutels die bij het benoemen van opgeslagen bestanden worden weggelaten (bijv. sid,utm_source).
Write a WARC archive of the crawl
Schrijf een WARC-archief van de crawl
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Sla ook elke opgehaalde respons op in een ISO-28500 WARC/1.1-archief, naast de mirror.
WARC archive name:
Naam van WARC-archief:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Optionele basisnaam voor het WARC-archief; laat leeg om het automatisch een naam te geven in de uitvoermap.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Koble fra når fullført
Disconnect modem on completion
Koble fra modem når fullført
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Vennligst fortell oss om feil eller problemer med programmet)\r\n\r\nUtvikling (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche og andre bidragsytere\r\n\r\nNorwegian translation:\r\nTobias "Spug" Langhoff ( spug_enigma@hotmail.com )
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Vennligst fortell oss om feil eller problemer med programmet)\r\n\r\nUtvikling (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche og andre bidragsytere\r\n\r\nNorwegian translation:\r\nTobias "Spug" Langhoff ( spug_enigma@hotmail.com )
About WinHTTrack Website Copier
Om WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Fjern spørrenøkler:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommaseparerte spørrenøkler som utelates i navngivingen av lagrede filer (f.eks. sid,utm_source).
Write a WARC archive of the crawl
Skriv et WARC-arkiv av gjennomgangen
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Lagre også hvert nedlastet svar i et ISO-28500 WARC/1.1-arkiv ved siden av speilet.
WARC archive name:
Navn på WARC-arkiv:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Valgfritt basisnavn for WARC-arkivet; la feltet stå tomt for automatisk navngivning i utdatamappen.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Zakoñcz po³¹czenie z us³ugodawc¹ po pobraniu
Disconnect modem on completion
Po pobraniu roz³¹cz modem
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(proszê poinformowaæ nas o jakichkolwiek b³êdach w dzia³aniu programu)\r\n\r\nTwórcy:\r\nInterfejs(Windows): Xavier Roche\r\nMotor: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nPolska wersja jêzykowa: £ukasz Jokiel (Opole University of Technology, Lukasz.Jokiel@po.opole.pl)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(proszê poinformowaæ nas o jakichkolwiek b³êdach w dzia³aniu programu)\r\n\r\nTwórcy:\r\nInterfejs(Windows): Xavier Roche\r\nMotor: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nPolska wersja jêzykowa: £ukasz Jokiel (Opole University of Technology, Lukasz.Jokiel@po.opole.pl)
About WinHTTrack Website Copier
O... WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Usuñ klucze zapytania:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Rozdzielone przecinkami klucze zapytania pomijane przy nazywaniu zapisanych plików (np. sid,utm_source).
Write a WARC archive of the crawl
Zapisz archiwum WARC z indeksowania
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Zapisz te¿ ka¿d± pobran± odpowied¼ do archiwum WARC/1.1 ISO-28500, obok kopii lustrzanej.
WARC archive name:
Nazwa archiwum WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Opcjonalna nazwa bazowa archiwum WARC; pozostaw puste, aby nazwaæ je automatycznie w katalogu wyj¶ciowym.

View File

@@ -652,8 +652,8 @@ Disconnect when finished
Desconectar ao finalizar
Disconnect modem on completion
Desconectar o modem ao finalizar
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(por favor nos comunique sobre erros ou problemas)\r\n\r\nDesenvolvimento:\r\nInterface (Windows): Xavier Roche\r\nMotor: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche e outros colaboradores\r\nTraduzido para o Português-Brasil por :\r\nPaulo Neto (layoutbr@lexxa.com.br)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(por favor nos comunique sobre erros ou problemas)\r\n\r\nDesenvolvimento:\r\nInterface (Windows): Xavier Roche\r\nMotor: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche e outros colaboradores\r\nTraduzido para o Português-Brasil por :\r\nPaulo Neto (layoutbr@lexxa.com.br)
About WinHTTrack Website Copier
Sobre o WinHTTrack Website Copier
Please visit our Web page
@@ -1004,3 +1004,11 @@ Strip query keys:
Remover chaves da query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Chaves da query string, separadas por vírgulas, a serem removidas da nomeação dos arquivos salvos (ex.: sid,utm_source).
Write a WARC archive of the crawl
Gravar um arquivo WARC do rastreamento
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Salvar também cada resposta baixada em um arquivo WARC/1.1 ISO-28500, ao lado do espelho.
WARC archive name:
Nome do arquivo WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nome base opcional para o arquivo WARC; deixe em branco para nomeá-lo automaticamente no diretório de saída.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Desligar no fim da operação
Disconnect modem on completion
Desconectar o modem no final
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Por favor avise-nos acerca de erros ou problemas)\r\n\r\nDesenvolvimento:\r\nInterface (Windows): Xavier Roche\r\nMotor: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nAgradecimentos pela tradução para o Português para:\r\nRui Fernandes (CANTIC, ruiefe@mail.malhatlantica.pt)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Por favor avise-nos acerca de erros ou problemas)\r\n\r\nDesenvolvimento:\r\nInterface (Windows): Xavier Roche\r\nMotor: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nAgradecimentos pela tradução para o Português para:\r\nRui Fernandes (CANTIC, ruiefe@mail.malhatlantica.pt)
About WinHTTrack Website Copier
Acerca do WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Remover chaves da query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Chaves da query string, separadas por vírgulas, a remover da nomeação dos ficheiros guardados (por ex. sid,utm_source).
Write a WARC archive of the crawl
Escrever um arquivo WARC do rastreio
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Guardar também cada resposta transferida num arquivo WARC/1.1 ISO-28500, ao lado do espelho.
WARC archive name:
Nome do arquivo WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nome base opcional para o arquivo WARC; deixe em branco para o nomear automaticamente no diretório de saída.

View File

@@ -634,7 +634,7 @@ Disconnect when finished
Deconectează şa terminare
Disconnect modem on completion
Deconectează modemul la terminare
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
About WinHTTrack Website Copier
Despre WinHTTrack Website Copier
@@ -956,3 +956,11 @@ Strip query keys:
Elimina cheile din query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Chei din query string, separate prin virgula, de eliminat din denumirea fisierelor salvate (de ex. sid,utm_source).
Write a WARC archive of the crawl
Scrie o arhiva WARC a parcurgerii
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Salveaza si fiecare raspuns descarcat intr-o arhiva WARC/1.1 ISO-28500, langa oglinda.
WARC archive name:
Numele arhivei WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nume de baza optional pentru arhiva WARC; lasati gol pentru a-l denumi automat in directorul de iesire.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Îòñîåäèíèòüñÿ ïðè çàâåðøåíèè
Disconnect modem on completion
Îòñîåäèíèòü ïðè çàâåðøåíèè
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ñîîáùèòå íàì, ïîæàëóéñòà, î çàìå÷åííûõ ïðîáëåìàõ è îøèáêàõ)\r\n\r\nÐàçðàáîòêà:\r\nÈíòåðôåéñ (Windows): Xavier Roche\r\nÊà÷àëêà (spider): Xavier Roche\r\nÏàðñåð ÿâà-êëàññîâ: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Russian translations to:\r\nAndrei Iliev (andreiiliev@mail.ru)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ñîîáùèòå íàì, ïîæàëóéñòà, î çàìå÷åííûõ ïðîáëåìàõ è îøèáêàõ)\r\n\r\nÐàçðàáîòêà:\r\nÈíòåðôåéñ (Windows): Xavier Roche\r\nÊà÷àëêà (spider): Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Russian translations to:\r\nAndrei Iliev (andreiiliev@mail.ru)
About WinHTTrack Website Copier
Î ïðîãðàììå WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Óäàëÿòü êëþ÷è çàïðîñà:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Êëþ÷è çàïðîñà ÷åðåç çàïÿòóþ, óäàëÿåìûå èç èìåíè ñîõðàíÿåìîãî ôàéëà (íàïðèìåð, sid,utm_source).
Write a WARC archive of the crawl
Çàïèñàòü WARC-àðõèâ îáõîäà
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Ñîõðàíÿòü êàæäûé çàãðóæåííûé îòâåò òàêæå â àðõèâ WARC/1.1 ISO-28500 ðÿäîì ñ çåðêàëîì.
WARC archive name:
Èìÿ WARC-àðõèâà:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Íåîáÿçàòåëüíîå áàçîâîå èìÿ WARC-àðõèâà; îñòàâüòå ïóñòûì äëÿ àâòîìàòè÷åñêîãî èìåíîâàíèÿ â âûõîäíîì êàòàëîãå.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Odpoji<EFBFBD>, ak je kopírovanie ukonèené
Disconnect modem on completion
Odpoji<EFBFBD> modem po dokonèení
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Oznám nám akúko¾vek chybu alebo problém)\r\n\r\nVývoj:\r\nProstredie (Windows): Xavier Roche\r\nPavúk: Xavier Roche\r\nKontrolór Javy: Yann Philippot\r\n\r\n(C)1998-2001 Xavier Roche a ostatní pomocníci\r\nVE¼KÉ POÏAKOVANIE za preklady:\r\nDr. Martin Sereday (sereday@slovanet.sk)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Oznám nám akúko¾vek chybu alebo problém)\r\n\r\nVývoj:\r\nProstredie (Windows): Xavier Roche\r\nPavúk: Xavier Roche\r\n\r\n(C)1998-2001 Xavier Roche a ostatní pomocníci\r\nVE¼KÉ POÏAKOVANIE za preklady:\r\nDr. Martin Sereday (sereday@slovanet.sk)
About WinHTTrack Website Copier
O WinHTTrack Website Copier...
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Odstráni» kµúèe dotazu:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kµúèe dotazu oddelené èiarkami, ktoré sa vynechajú z pomenovania ukladaných súborov (napr. sid,utm_source).
Write a WARC archive of the crawl
Zapísa» archív WARC z prehµadávania
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Ulo¾i» aj ka¾dú stiahnutú odpoveï do archívu WARC/1.1 ISO-28500 vedµa zrkadla.
WARC archive name:
Názov archívu WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Voliteµný základný názov archívu WARC; ponechajte prázdne pre automatické pomenovanie vo výstupnom adresári.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Prekini povezavo, ko bo konèano
Disconnect modem on completion
Izkljuèi modem ob dokonèanju
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Prosimo Vas, da nas obvestite o kakršnih koli težavah ali napakah)\r\n\r\nRazvoj:\r\nVmesnik (Okna): Xavier Roche\r\nHitrost: Xavier Roche\r\nJavaParserRazredi: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche in drugi sodelujoèi\r\nVELIKA ZAHVALA za namige prevodov gre:\r\nRobertu Lagadecu (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Prosimo Vas, da nas obvestite o kakršnih koli težavah ali napakah)\r\n\r\nRazvoj:\r\nVmesnik (Okna): Xavier Roche\r\nHitrost: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche in drugi sodelujoèi\r\nVELIKA ZAHVALA za namige prevodov gre:\r\nRobertu Lagadecu (rlagadec@yahoo.fr)
About WinHTTrack Website Copier
O programu WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Odstrani kljuce poizvedbe:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Z vejicami loceni kljuci poizvedbe, ki se izpustijo pri poimenovanju shranjenih datotek (npr. sid,utm_source).
Write a WARC archive of the crawl
Zapisi arhiv WARC iz pregledovanja
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Shrani tudi vsak preneseni odgovor v arhiv WARC/1.1 ISO-28500 poleg zrcala.
WARC archive name:
Ime arhiva WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Neobvezno osnovno ime arhiva WARC; pustite prazno za samodejno poimenovanje v izhodni mapi.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Koppla ner förbindelsen när överföringen är klar
Disconnect modem on completion
Koppla ned modemet efter avslutning
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Hittar du fel eller uppstår det problem, kontakta oss)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Swedish translations to:\r\nStaffan Ström (staffan@fam-strom.org)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Hittar du fel eller uppstår det problem, kontakta oss)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Swedish translations to:\r\nStaffan Ström (staffan@fam-strom.org)
About WinHTTrack Website Copier
Om WinHTTrack Website Copier...
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Ta bort frågenycklar:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommaseparerade frågenycklar som utelämnas vid namngivningen av sparade filer (t.ex. sid,utm_source).
Write a WARC archive of the crawl
Skriv ett WARC-arkiv av genomsökningen
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Spara även varje hämtat svar i ett ISO-28500 WARC/1.1-arkiv, bredvid spegeln.
WARC archive name:
WARC-arkivets namn:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Valfritt basnamn för WARC-arkivet; lämna tomt för att namnge det automatiskt i utdatakatalogen.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Bittiðinde baðlantýyý kes
Disconnect modem on completion
Ýþlem tamamlandýðýnda modemden baðlantýyý kopar
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Lütfen bize her türlü hatayý ve problemi aktarýnýz)\r\n\r\nGeliþtirme:\r\nArayüz (Windows): Xavier Roche\r\nAð: Xavier Roche\r\nJava Ayýklama Sýnýflarý: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche ve diðerleri\r\nÇeviri ipuçlarý için teþekkürler: \r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Lütfen bize her türlü hatayý ve problemi aktarýnýz)\r\n\r\nGeliþtirme:\r\nArayüz (Windows): Xavier Roche\r\nAð: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche ve diðerleri\r\nÇeviri ipuçlarý için teþekkürler: \r\nRobert Lagadec (rlagadec@yahoo.fr)
About WinHTTrack Website Copier
WinHTTrack Website Copier Hakkýnda
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Sorgu anahtarlarýný çýkar:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kaydedilen dosya adlandýrmasýndan çýkarýlacak, virgülle ayrýlmýþ sorgu anahtarlarý (örn. sid,utm_source).
Write a WARC archive of the crawl
Taramanýn WARC arþivini yaz
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Ýndirilen her yanýtý ayrýca aynanýn yanýna bir ISO-28500 WARC/1.1 arþivine kaydet.
WARC archive name:
WARC arþivi adý:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
WARC arþivi için isteðe baðlý temel ad; çýktý dizininde otomatik adlandýrma için boþ býrakýn.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
³ä'ºäíàòèñü ïðè çàâåðøåíí³
Disconnect modem on completion
³ä'ºäíàòè ïðè çàâåðøåíí³
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ïîâ³äîìèòå íàì áóäü ëàñêà ïðî ïîì³÷åí³ ïðîáëåìè ³ ïîìèëêè)\r\n\r\nðàçðàáîòêà:\r\nèíòåðôåéñ (Windows): Xavier Roche\r\nêà÷àëêà (spider): Xavier Roche\r\nïàðñåð java-êëàñ³â: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Ukrainian translations to:\r\nAndrij Shevchuk (andrijsh@mail.lviv.ua) http://programy.com.ua, http://vic-info.com.ua
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ïîâ³äîìèòå íàì áóäü ëàñêà ïðî ïîì³÷åí³ ïðîáëåìè ³ ïîìèëêè)\r\n\r\nðàçðàáîòêà:\r\nèíòåðôåéñ (Windows): Xavier Roche\r\nêà÷àëêà (spider): Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Ukrainian translations to:\r\nAndrij Shevchuk (andrijsh@mail.lviv.ua) http://programy.com.ua, http://vic-info.com.ua
About WinHTTrack Website Copier
Ïðî ïðîãðàìó WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Âèëó÷àòè êëþ÷³ çàïèòó:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Êëþ÷³ çàïèòó ÷åðåç êîìó, ÿê³ âèëó÷àþòüñÿ ç ³ìåí³ çáåðåæåíîãî ôàéëó (íàïðèêëàä, sid,utm_source).
Write a WARC archive of the crawl
Çàïèñàòè WARC-àðõ³â îáõîäó
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Çáåð³ãàòè êîæíó çàâàíòàæåíó â³äïîâ³äü òàêîæ ó àðõ³â WARC/1.1 ISO-28500 ïîðÿä ³ç äçåðêàëîì.
WARC archive name:
²ì'ÿ WARC-àðõ³âó:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Íåîáîâ'ÿçêîâà áàçîâà íàçâà WARC-àðõ³âó; çàëèøòå ïîðîæí³ì äëÿ àâòîìàòè÷íîãî íàéìåíóâàííÿ ó âèõ³äíîìó êàòàëîç³.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Îòñîåäèíèòüñÿ ïðè çàâåðøåíèè
Disconnect modem on completion
Îòñîåäåíèòü ïðè çàâåðøåíèè
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ñîîáùèòå íàì ïîæàëóéñòà î çàìå÷åííûõ ïðîáëåìàõ è îøèáêàõ)\r\n\r\nÐàçðàáîòêà:\r\nÈíòåðôåéñ (Windows): Xavier Roche\r\nÊà÷àëêà (spider): Xavier Roche\r\nÏàðñåð ÿâà-êëàññîâ: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Russian translations to:\r\nAndrei Iliev (andreiiliev@mail.ru)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ñîîáùèòå íàì ïîæàëóéñòà î çàìå÷åííûõ ïðîáëåìàõ è îøèáêàõ)\r\n\r\nÐàçðàáîòêà:\r\nÈíòåðôåéñ (Windows): Xavier Roche\r\nÊà÷àëêà (spider): Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Russian translations to:\r\nAndrei Iliev (andreiiliev@mail.ru)
About WinHTTrack Website Copier
Î ïðîãðàììå WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,11 @@ Strip query keys:
Olib tashlanadigan sorov kalitlari:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Saqlangan fayl nomidan olib tashlanadigan, vergul bilan ajratilgan sorov kalitlari (masalan, sid,utm_source).
Write a WARC archive of the crawl
Qidiruvning WARC arxivini yozish
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Har bir yuklab olingan javobni ISO-28500 WARC/1.1 arxiviga ham, ko'zgu yonida saqlash.
WARC archive name:
WARC arxivi nomi:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
WARC arxivi uchun ixtiyoriy asosiy nom; chiqish katalogida avtomatik nomlash uchun bo'sh qoldiring.

View File

@@ -3,7 +3,7 @@
.\"
.\" This file is generated by man/makeman.sh; do not edit by hand.
.\" SPDX-License-Identifier: GPL-3.0-or-later
.TH httrack 1 "22 July 2026" "httrack website copier"
.TH httrack 1 "26 July 2026" "httrack website copier"
.SH NAME
httrack \- offline browser : copy websites to a local directory
.SH SYNOPSIS
@@ -40,7 +40,7 @@ httrack \- offline browser : copy websites to a local directory
[ \fB\-%N, \-\-delayed\-type\-check\fR ]
[ \fB\-%D, \-\-cached\-delayed\-type\-check\fR ]
[ \fB\-%M, \-\-mime\-html\fR ]
[ \fB\-%r, \-\-warc\fR ]
[ \fB\-%Z, \-\-single\-file\fR ]
[ \fB\-LN, \-\-long\-names[=N]\fR ]
[ \fB\-KN, \-\-keep\-links[=N]\fR ]
[ \fB\-x, \-\-replace\-external\fR ]
@@ -75,6 +75,7 @@ httrack \- offline browser : copy websites to a local directory
[ \fB\-%X, \-\-headers\fR ]
[ \fB\-C, \-\-cache[=N]\fR ]
[ \fB\-k, \-\-store\-all\-in\-cache\fR ]
[ \fB\-%r, \-\-warc\fR ]
[ \fB\-%n, \-\-do\-not\-recatch\fR ]
[ \fB\-%v, \-\-display\fR ]
[ \fB\-Q, \-\-do\-not\-log\fR ]
@@ -198,8 +199,8 @@ delayed type check, don't make any link test but wait for files download to star
cached delayed type check, don't wait for remote type during updates, to speedup them (%D0 wait, * %D1 don't wait) (\-\-cached\-delayed\-type\-check)
.IP \-%M
generate a RFC MIME\-encapsulated full\-archive (.mht) (\-\-mime\-html)
.IP \-%r
write an ISO\-28500 WARC/1.1 archive; \-\-warc\-file NAME sets the output name (\-\-warc)
.IP \-%Z
after the mirror, rewrite each saved page with its stylesheets, scripts, images and fonts inlined as data: URIs, so any page also opens on its own (links between pages stay relative; audio and video stay links); \-\-single\-file\-max\-size N caps each asset (default 10485760 bytes) (\-\-single\-file)
.IP \-%t
keep the original file extension, don't rewrite it from the MIME type (%t0 rewrite)
.IP \-LN
@@ -279,6 +280,8 @@ additional HTTP header line (\-%X "X\-Magic: 42" (\-\-headers <param>)
create/use a cache for updates and retries (C0 no cache,C1 cache is prioritary,* C2 test update before) (\-\-cache[=N])
.IP \-k
store all files in cache (not useful if files on disk) (\-\-store\-all\-in\-cache)
.IP \-%r
write an ISO\-28500 WARC/1.1 archive; \-\-warc\-file NAME sets the output name, \-\-warc\-max\-size N rotates segments past N bytes, \-\-warc\-cdx also writes a sorted CDXJ index, \-\-wacz packages it all as a WACZ file (\-\-warc)
.IP \-%n
do not re\-download locally erased files (\-\-do\-not\-recatch)
.IP \-%v

View File

@@ -32,7 +32,9 @@ AM_LDFLAGS = \
bin_PROGRAMS = proxytrack httrack htsserver
httrack_LDADD = $(THREADS_LIBS) libhttrack.la
httrack_SOURCES = httrack.c htsbacktrace.c htsbacktrace.h
# $(DL_LIBS): dladdr() in the crash handler, still in libdl on pre-2.34 glibc.
httrack_LDADD = $(THREADS_LIBS) $(DL_LIBS) libhttrack.la
htsserver_LDADD = $(THREADS_LIBS) $(SOCKET_LIBS) libhttrack.la
proxytrack_LDADD = $(THREADS_LIBS) $(SOCKET_LIBS)
@@ -47,6 +49,7 @@ htsserver_LDFLAGS = $(AM_LDFLAGS) $(LDFLAGS_PIE)
lib_LTLIBRARIES = libhttrack.la
htsserver_SOURCES = htsserver.c htsserver.h htsweb.c htsweb.h \
htscmdline.c htscmdline.h \
htsurlport.c htsurlport.h
proxytrack_SOURCES = proxy/main.c \
proxy/proxytrack.c proxy/store.c \
@@ -60,21 +63,21 @@ whttrackrun_SCRIPTS = webhttrack
libhttrack_la_SOURCES = htscore.c htsparse.c htsback.c htscache.c \
htscache_selftest.c htsdns_selftest.c htsselftest.c \
htscatchurl.c htsfilters.c htsftp.c htshash.c coucal/coucal.c \
htshelp.c htslib.c htsurlport.c htscoremain.c \
htscmdline.c htshelp.c htslib.c htsurlport.c htscoremain.c \
htsname.c htsrobots.c htstools.c htswizard.c \
htsalias.c htsthread.c htsindex.c htsbauth.c \
htsmd5.c htscodec.c htswarc.c htsproxy.c htszlib.c htswrap.c htsconcat.c \
htsmd5.c htscodec.c htswarc.c htssinglefile.c htsproxy.c htszlib.c htswrap.c htsconcat.c \
htsmodules.c htscharset.c punycode.c htsencoding.c htssniff.c \
md5.c \
minizip/ioapi.c minizip/mztools.c minizip/unzip.c minizip/zip.c \
hts-indextmpl.h htsalias.h htsback.h htsbase.h htssafe.h \
htsbasenet.h htsbauth.h htscache.h htscache_selftest.h htsdns_selftest.h htsselftest.h htscatchurl.h \
htsconfig.h htscore.h htsparse.h htscoremain.h htsdefines.h \
htscmdline.h htsconfig.h htscore.h htsparse.h htscoremain.h htsdefines.h \
htsfilters.h htsftp.h htsglobal.h htshash.h coucal/coucal.h \
htshelp.h htsindex.h htslib.h htsurlport.h htsmd5.h \
htsmodules.h htsname.h htsnet.h htssniff.h \
htsopt.h htsrobots.h htsthread.h \
htstools.h htswizard.h htswrap.h htscodec.h htswarc.h htsproxy.h htszlib.h \
htstools.h htswizard.h htswrap.h htscodec.h htswarc.h htssinglefile.h htsproxy.h htszlib.h \
htsstrings.h htsarrays.h httrack-library.h \
htscharset.h punycode.h htsencoding.h \
htsentities.h htsentities.sh htsbasiccharsets.sh htscodepages.h \

View File

@@ -116,6 +116,17 @@ const char *hts_optalias[][4] = {
"load extra cookies from a Netscape cookies.txt"},
{"warc", "-%r", "single", "write an ISO-28500 WARC/1.1 archive of the crawl"},
{"warc-file", "-%rf", "param1", "write a WARC archive to the given base name"},
{"warc-max-size", "-%rs", "param1",
"rotate the WARC archive once a segment passes N bytes (0: single file)"},
{"warc-cdx", "-%rc", "single",
"write a sorted CDXJ index next to the WARC archive"},
{"warc-cdxj", "-%rc", "single", ""},
{"wacz", "-%rz", "single",
"package the WARC archive, CDXJ index and pages as a WACZ file"},
{"single-file", "-%Z", "single",
"after the mirror, inline each page's assets as data: URIs"},
{"single-file-max-size", "-%Zs", "param1",
"per-asset cap for --single-file, in bytes (implies it; default 10485760)"},
{"why", "-%Y", "param1",
"explain which filter rule accepts or rejects a URL, then exit"},
{"pause", "-%G", "param1",

View File

@@ -541,9 +541,15 @@ static int create_back_tmpfile(httrackp *opt, lien_back *const back,
const char *ext) {
// do not use tempnam() but a regular filename
back->tmpfile_buffer[0] = '\0';
if (back->url_sav != NULL && back->url_sav[0] != '\0') {
snprintf(back->tmpfile_buffer, sizeof(back->tmpfile_buffer), "%s.%s",
back->url_sav, ext);
if (back->url_sav[0] != '\0') {
/* same capacity as url_sav, so truncation drops the extension and aliases
the temp name onto the live file that back_finalize_backup() UNLINKs */
if (!sprintfbuff(back->tmpfile_buffer, "%s.%s", back->url_sav, ext)) {
hts_log_print(opt, LOG_WARNING, "temporary filename too long for %s",
back->url_sav);
back->tmpfile_buffer[0] = '\0';
return -1;
}
back->tmpfile = back->tmpfile_buffer;
if (structcheck(back->tmpfile) != 0) {
hts_log_print(opt, LOG_WARNING, "can not create directory to %s",
@@ -551,8 +557,15 @@ static int create_back_tmpfile(httrackp *opt, lien_back *const back,
return -1;
}
} else {
snprintf(back->tmpfile_buffer, sizeof(back->tmpfile_buffer), "%s/tmp%d.%s",
StringBuff(opt->path_html_utf8), opt->state.tmpnameid++, ext);
/* truncation here would collide distinct tmpnameid's onto one name */
if (!sprintfbuff(back->tmpfile_buffer, "%s/tmp%d.%s",
StringBuff(opt->path_html_utf8), opt->state.tmpnameid++,
ext)) {
hts_log_print(opt, LOG_WARNING, "temporary filename too long in %s",
StringBuff(opt->path_html_utf8));
back->tmpfile_buffer[0] = '\0';
return -1;
}
back->tmpfile = back->tmpfile_buffer;
}
/* OK */
@@ -750,9 +763,20 @@ int back_finalize(httrackp * opt, cache_back * cache, struct_back * sback,
fexist_utf8(back[p].url_sav))
filenote(&opt->state.strc, back[p].url_sav, NULL);
}
/* Keep the compressed spool so the WARC record stores the body
verbatim (Content-Encoding preserved) instead of unlinking it.
*/
if (StringNotEmpty(opt->warc_file)) {
warc_adopt_rawspool(&back[p].r, back[p].tmpfile);
if (back[p].r.warc_rawpath != NULL)
back[p].tmpfile =
NULL; /* adopted: freed via warc_free_request */
}
/* ensure that no remaining temporary file exists */
unlink(back[p].tmpfile);
back[p].tmpfile = NULL;
if (back[p].tmpfile != NULL) {
unlink(back[p].tmpfile);
back[p].tmpfile = NULL;
}
}
// stats
HTS_STAT.total_packed += back[p].compressed_size;
@@ -876,8 +900,7 @@ int back_finalize(httrackp * opt, cache_back * cache, struct_back * sback,
HTS_STAT.stat_bytes += back[p].r.size;
HTS_STAT.stat_files++;
hts_log_print(opt, LOG_TRACE, "added file %s%s => %s",
back[p].url_adr, back[p].url_fil,
back[p].url_sav != NULL ? back[p].url_sav : "");
back[p].url_adr, back[p].url_fil, back[p].url_sav);
}
if ((!back[p].r.notmodified) && (opt->is_update)) {
HTS_STAT.stat_updated_files++; // page modifiée
@@ -1062,6 +1085,9 @@ void back_copy_static(const lien_back * src, lien_back * dst) {
dst->r.headers = NULL;
dst->r.warc_reqhdr = NULL;
dst->r.warc_resphdr = NULL;
dst->r.warc_rawpath =
NULL; /* the spool stays owned by src (no double-unlink) */
dst->r.warc_truncated = 0;
dst->r.out = NULL;
dst->r.location = dst->location_buffer;
dst->r.fp = NULL;
@@ -1127,6 +1153,8 @@ int back_unserialize(FILE * fp, lien_back ** dst) {
(*dst)->r.out = NULL;
(*dst)->r.warc_reqhdr = NULL;
(*dst)->r.warc_resphdr = NULL;
(*dst)->r.warc_rawpath = NULL;
(*dst)->r.warc_truncated = 0;
(*dst)->r.location = (*dst)->location_buffer;
(*dst)->r.fp = NULL;
(*dst)->r.soc = INVALID_SOCKET;
@@ -2286,26 +2314,24 @@ int back_add(struct_back *sback, httrackp *opt, cache_back *cache,
&& slot_can_be_finalized(opt, &back[i]);
int may_serialize = slot_can_be_cached_on_disk(&back[i]);
hts_log_print(opt, LOG_DEBUG,
"back[%03d]: may_clean=%d, may_finalize_disk=%d, may_serialize=%d:"
LF "\t"
"finalized(%d), status(%d), locked(%d), delayed(%d), test(%d), "
LF "\t"
"statuscode(%d), size(%d), is_write(%d), may_hypertext(%d), "
LF "\t" "contenttype(%s), url(%s%s), save(%s)", i,
may_clean, may_finalize, may_serialize,
back[i].finalized, back[i].status, back[i].locked,
IS_DELAYED_EXT(back[i].url_sav), back[i].testmode,
back[i].r.statuscode, (int) back[i].r.size,
back[i].r.is_write, may_be_hypertext_mime(opt,
back[i].r.
contenttype,
back[i].
url_fil),
/* */
back[i].r.contenttype, back[i].url_adr,
back[i].url_fil,
back[i].url_sav ? back[i].url_sav : "<null>");
hts_log_print(
opt, LOG_DEBUG,
"back[%03d]: may_clean=%d, may_finalize_disk=%d, "
"may_serialize=%d:" LF "\t"
"finalized(%d), status(%d), locked(%d), delayed(%d), "
"test(%d), " LF "\t"
"statuscode(%d), size(%d), is_write(%d), may_hypertext(%d), " LF
"\t"
"contenttype(%s), url(%s%s), save(%s)",
i, may_clean, may_finalize, may_serialize, back[i].finalized,
back[i].status, back[i].locked, IS_DELAYED_EXT(back[i].url_sav),
back[i].testmode, back[i].r.statuscode, (int) back[i].r.size,
back[i].r.is_write,
may_be_hypertext_mime(opt, back[i].r.contenttype,
back[i].url_fil),
/* */
back[i].r.contenttype, back[i].url_adr, back[i].url_fil,
back[i].url_sav);
}
}
}
@@ -2519,6 +2545,19 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
for (i = 0; i < (unsigned int) back_max; i++) {
if (back[i].status > 0 && back[i].status < STATUS_FTP_TRANSFER) {
/* A cap-truncated body is deliberate, not broken: archive what arrived
with WARC-Truncated before the abort overwrites the slot's real 2xx
status. HTTrack still treats the slot as incomplete afterwards. */
if (StringNotEmpty(opt->warc_file) && back[i].r.statuscode > 0 &&
back[i].r.warc_resphdr != NULL && back[i].r.size > 0 &&
!(back[i].r.is_write && IS_DELAYED_EXT(back[i].url_sav))) {
if (back[i].r.is_write && back[i].r.out != NULL)
fflush(back[i].r.out);
back[i].r.warc_truncated = (limit == HTS_MIRROR_LIMIT_SIZE)
? WARC_TRUNC_LENGTH
: WARC_TRUNC_TIME;
warc_write_backtransaction(opt, &back[i]);
}
if (back[i].r.soc != INVALID_SOCKET) {
deletehttp(&back[i].r);
}
@@ -2828,7 +2867,8 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
// new session
back[i].r.ssl_con = SSL_new(openssl_ctx);
if (back[i].r.ssl_con) {
const char* hostname = jump_protocol_const(back[i].url_adr);
/* non-const twin: the OpenSSL macro casts the qualifier away */
char *hostname = jump_protocol(back[i].url_adr);
// some servers expect the hostname on the clienthello (SNI TLS extension)
SSL_set_tlsext_host_name(back[i].r.ssl_con, hostname);
SSL_clear(back[i].r.ssl_con);

238
src/htsbacktrace.c Normal file
View File

@@ -0,0 +1,238 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* File: crash backtrace printer */
/* Author: Xavier Roche */
/* ------------------------------------------------------------ */
/* Before every header: glibc gates dladdr() on it. */
#if defined(__linux) && !defined(_GNU_SOURCE)
#define _GNU_SOURCE
#endif
#include "htsbacktrace.h"
#include "htsglobal.h"
#include <stdlib.h>
#include <string.h>
#ifdef _WIN32
#include <io.h> /* write */
#endif
#ifdef HAVE_UNISTD_H
#include <unistd.h>
#endif
#if (defined(__linux) && defined(HAVE_EXECINFO_H))
#include <dlfcn.h>
#include <errno.h>
#include <execinfo.h>
#include <signal.h>
#include <time.h>
#include <sys/wait.h>
#define USES_BACKTRACE
#endif
#ifdef USES_BACKTRACE
#define BT_MAX_FRAMES 64 /* frames we try to name */
#define BT_MAX_MODULES 8 /* distinct modules, one child each */
#define BT_HEX_SIZE 19 /* "0x" + 16 nibbles + NUL */
#define BT_PATH_SIZE 1024 /* module path; longer is skipped */
#define BT_WAIT_TICKS 300 /* 10ms ticks, shared: cap a slow child */
#define BT_NO_SYMBOLIZER 127 /* child exit: execvp() found none */
static hts_boolean symbolize_crash = HTS_TRUE;
/* "0x"-prefixed hex: the handler must stay stdio-free. */
static void print_hex(char *buffer, uintptr_t value) {
static const char digits[] = "0123456789abcdef";
size_t i = 2, a, b;
buffer[0] = '0';
buffer[1] = 'x';
do {
buffer[i++] = digits[value & 0xf];
value >>= 4;
} while (value != 0);
buffer[i] = '\0';
for (a = 2, b = i - 1; a < b; a++, b--) {
const char c = buffer[a];
buffer[a] = buffer[b];
buffer[b] = c;
}
}
/* HTS_FALSE if src does not fit: a truncated module path would point the
symbolizer at the wrong file. */
static hts_boolean copy_bounded(char *dest, size_t size, const char *src) {
size_t i;
for (i = 0; i < size - 1 && src[i] != '\0'; i++) {
dest[i] = src[i];
}
dest[i] = '\0';
return src[i] == '\0' ? HTS_TRUE : HTS_FALSE;
}
/* Run the symbolizer on argv, output on fd, within *budget ticks. HTS_FALSE
only if none could be run at all; otherwise silent, the raw trace stands. */
static hts_boolean spawn_symbolizer(char **argv, int fd, int *budget) {
const pid_t pid = fork();
int status = 0;
if (pid == -1)
return HTS_FALSE;
if (pid == 0) {
static char llvm_prog[] = "llvm-symbolizer";
static char llvm_opts[] = "-p";
dup2(fd, 1); /* both symbolizers write on stdout */
execvp(argv[0], argv);
argv[0] = llvm_prog; /* an LLVM-only install ships no addr2line */
argv[1] = llvm_opts;
execvp(argv[0], argv);
_exit(BT_NO_SYMBOLIZER);
}
for (; *budget > 0; (*budget)--) {
const struct timespec tick = {0, 10 * 1000 * 1000};
const pid_t reaped = waitpid(pid, &status, WNOHANG);
if (reaped == pid)
return WIFEXITED(status) && WEXITSTATUS(status) == BT_NO_SYMBOLIZER
? HTS_FALSE
: HTS_TRUE;
if (reaped == -1 && errno != EINTR)
return HTS_TRUE;
nanosleep(&tick, NULL);
}
kill(pid, SIGKILL);
waitpid(pid, NULL, 0);
return HTS_TRUE;
}
/* Name the frames backtrace_symbols_fd() leaves as module+offset:
-fvisibility=hidden keeps them out of .dynsym, but DWARF has them. dladdr()
is not formally async-signal-safe; accepted, this path is already fatal. */
static void symbolize_backtrace(void *const *stack, int size, int fd) {
static char prog[] = "addr2line";
static char opts[] = "-Cfipa";
static char dashe[] = "-e";
char hex[BT_MAX_FRAMES][BT_HEX_SIZE];
const void *base[BT_MAX_FRAMES];
const char *name[BT_MAX_FRAMES];
hts_boolean grouped[BT_MAX_FRAMES];
char module[BT_PATH_SIZE];
char *argv[4 + BT_MAX_FRAMES + 1];
int budget = BT_WAIT_TICKS;
int i, spawned;
if (size > BT_MAX_FRAMES)
size = BT_MAX_FRAMES;
for (i = 0; i < size; i++) {
Dl_info info;
grouped[i] = HTS_TRUE; /* skipped unless dladdr() places the frame */
if (dladdr(stack[i], &info) == 0 || info.dli_fname == NULL ||
info.dli_fname[0] == '\0')
continue;
base[i] = info.dli_fbase;
name[i] = info.dli_fname;
print_hex(hex[i], (uintptr_t) ((const char *) stack[i] -
(const char *) info.dli_fbase));
grouped[i] = HTS_FALSE;
}
/* One child per module: addr2line takes a single -e. Each frame is claimed
once, so argc cannot exceed argv[]. */
for (spawned = 0; spawned < BT_MAX_MODULES; spawned++) {
int first, j, argc = 0;
for (first = 0; first < size && grouped[first]; first++)
;
if (first >= size)
break;
argv[argc++] = prog;
argv[argc++] = opts;
argv[argc++] = dashe;
argv[argc++] = module;
for (j = first; j < size; j++) {
if (grouped[j] || base[j] != base[first])
continue;
grouped[j] = HTS_TRUE;
argv[argc++] = hex[j];
}
argv[argc] = NULL;
/* access(): skip pseudo-modules like linux-vdso, which have no file and
would draw nothing but an addr2line complaint. */
if (copy_bounded(module, sizeof(module), name[first]) &&
access(module, R_OK) == 0) {
const size_t len = strlen(module);
/* addr2line -a prints offsets only: say which module they are in. */
(void) (write(fd, module, len) == (ssize_t) len);
(void) (write(fd, ":\n", 2) == 2);
if (!spawn_symbolizer(argv, fd, &budget))
break; /* no symbolizer: stop at one header */
}
}
}
#endif
void hts_backtrace_init(void) {
#ifdef USES_BACKTRACE
symbolize_crash =
getenv("HTTRACK_NO_SYMBOLIZE") == NULL ? HTS_TRUE : HTS_FALSE;
#endif
}
void hts_print_backtrace(int fd) {
#ifdef USES_BACKTRACE
void *stack[256];
const int size = backtrace(stack, sizeof(stack) / sizeof(stack[0]));
/* A fault inside the handler lands back here: symbolizing twice interleaves
two traces on fd and spends a second budget. */
static volatile sig_atomic_t entered = 0;
if (size != 0) {
backtrace_symbols_fd(stack, size, fd);
if (symbolize_crash && entered == 0) {
entered = 1;
symbolize_backtrace(stack, size, fd);
entered = 0;
}
}
#else
const char msg[] = "No stack trace available on this OS :(\n";
if (write(fd, msg, sizeof(msg) - 1) != sizeof(msg) - 1) {
/* sorry GCC */
}
#endif
}

45
src/htsbacktrace.h Normal file
View File

@@ -0,0 +1,45 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* File: crash backtrace printer */
/* Author: Xavier Roche */
/* ------------------------------------------------------------ */
#ifndef HTSBACKTRACE_DEFH
#define HTSBACKTRACE_DEFH
/* Sample HTTRACK_NO_SYMBOLIZE before any crash: getenv() is not signal-safe.
Call once, from the process that installs the fatal-signal handlers. */
void hts_backtrace_init(void);
/* Write the calling thread's stack to fd, callable from a fatal signal handler:
raw frames first, then whatever an external symbolizer can name. Allocates
nothing; prints a one-line notice where the OS has no backtrace(). */
void hts_print_backtrace(int fd);
#endif

View File

@@ -39,6 +39,9 @@ Please visit our Website: http://www.httrack.com
#include "htsglobal.h"
#include "htslib.h"
#include "htscore.h"
#ifdef _WIN32
#include "htscharset.h" /* hts_pathToUCS2, hts_convertUCS2StringToUTF8 */
#endif
/* END specific definitions */
@@ -200,28 +203,36 @@ char *cookie_nextfield(char *a) {
return a;
}
// lire cookies.txt
// lire également (Windows seulement) les *@*.txt (cookies IE copiés)
// !=0 : erreur
// Read cookies.txt (+ copied IE cookies *@*.txt on Windows); !=0 on error.
int cookie_load(t_cookie * cookie, const char *fpath, const char *name) {
char catbuff[CATBUFF_SIZE];
char buffer[8192];
// Fusionner d'abord les éventuels cookies IE
// Merge any IE cookies first
#ifdef _WIN32
{
WIN32_FIND_DATAA find;
WIN32_FIND_DATAW find;
HANDLE h;
char pth[MAX_PATH + 32];
char pth[HTS_URLMAXSIZE];
LPWSTR wpth;
strcpybuff(pth, fpath);
strcatbuff(pth, "*@*.txt");
h = FindFirstFileA((char *) pth, &find);
// Wide glob so a long or non-ASCII IE-cookie folder is scanned (#133).
wpth = hts_pathToUCS2(pth);
h = wpth != NULL ? FindFirstFileW(wpth, &find) : INVALID_HANDLE_VALUE;
freet(wpth);
if (h != INVALID_HANDLE_VALUE) {
do {
if (!(find.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY))
if (!(find.dwFileAttributes & FILE_ATTRIBUTE_SYSTEM)) {
FILE *fp = fopen(fconcat(catbuff, sizeof(catbuff), fpath, find.cFileName), "rb");
// cFileName is UTF-16: convert to UTF-8 so the mirror path and
// file wrappers stay UTF-8 (no CP_ACP mojibake).
char *u = hts_convertUCS2StringToUTF8(find.cFileName, -1);
FILE *fp =
u != NULL
? FOPEN(fconcat(catbuff, sizeof(catbuff), fpath, u), "rb")
: NULL;
if (fp) {
char cook_name[256];
@@ -229,11 +240,9 @@ int cookie_load(t_cookie * cookie, const char *fpath, const char *name) {
char domainpathpath[512];
char dummy[512];
//
lien_adrfil af; // chemin (/)
lien_adrfil af; // host + path (/)
int cookie_merged = 0;
//
// Read all cookies
while(!feof(fp)) {
cook_name[0] = cook_value[0] = domainpathpath[0]
@@ -262,10 +271,11 @@ int cookie_load(t_cookie * cookie, const char *fpath, const char *name) {
}
fclose(fp);
if (cookie_merged)
remove(fconcat(catbuff, sizeof(catbuff), fpath, find.cFileName));
UNLINK(fconcat(catbuff, sizeof(catbuff), fpath, u));
} // if fp
freet(u);
}
} while(FindNextFileA(h, &find));
} while (FindNextFileW(h, &find));
FindClose(h);
}
}
@@ -273,7 +283,7 @@ int cookie_load(t_cookie * cookie, const char *fpath, const char *name) {
// Ensuite, cookies.txt
{
FILE *fp = fopen(fconcat(catbuff, sizeof(catbuff), fpath, name), "rb");
FILE *fp = FOPEN(fconcat(catbuff, sizeof(catbuff), fpath, name), "rb");
if (fp) {
char BIGSTK line[8192];
@@ -315,7 +325,7 @@ int cookie_save(t_cookie * cookie, const char *name) {
if (strnotempty(cookie->data)) {
char BIGSTK line[8192];
#ifdef _WIN32
FILE *fp = fopen(fconv(catbuff, sizeof(catbuff), name), "wb");
FILE *fp = FOPEN(fconv(catbuff, sizeof(catbuff), name), "wb");
#else
const int fd = open(fconv(catbuff, sizeof(catbuff), name),
O_WRONLY | O_CREAT | O_TRUNC, HTS_PROTECT_FILE);

View File

@@ -629,8 +629,8 @@ static htsblk cache_readex_new(httrackp * opt, cache_back * cache,
// File exists on disk with declared cache name (this is expected!)
if (fexist_utf8(fconv(catbuff, sizeof(catbuff), previous_save))) { // un fichier existe déja
// Expected size ?
const size_t fsize =
fsize_utf8(fconv(catbuff, sizeof(catbuff), previous_save));
const LLint fsize = fsize_utf8(
fconv(catbuff, sizeof(catbuff), previous_save));
if (fsize == r.size) {
// Target name is the previous name, and the file looks good: nothing to do!
if (strcmp(previous_save, target_save) == 0) {
@@ -666,7 +666,8 @@ static htsblk cache_readex_new(httrackp * opt, cache_back * cache,
// Suppose a broken mirror, with a file being renamed: OK
else if (fexist_utf8(fconv(catbuff, sizeof(catbuff), target_save))) {
// Expected size ?
const size_t fsize = fsize_utf8(fconv(catbuff, sizeof(catbuff), target_save));
const LLint fsize =
fsize_utf8(fconv(catbuff, sizeof(catbuff), target_save));
if (fsize == r.size) {
// So far so good
@@ -1440,7 +1441,7 @@ int cache_brstr(char *adr, char *s, size_t s_size) {
/* binput bounded to a NUL-terminated buffer: refuse to start a read at or
past `end`, so a prior over-advance can't walk a cache-index parse OOB. */
int cache_binput(char *adr, const char *end, char *s, int max) {
int cache_binput(const char *adr, const char *end, char *s, int max) {
if (adr >= end) {
s[0] = '\0';
return 0;

View File

@@ -93,7 +93,7 @@ void cache_rstr(FILE *fp, char *s, size_t s_size);
char *cache_rstr_addr(FILE * fp);
int cache_brstr(char *adr, char *s, size_t s_size);
/* binput over a NUL-terminated buffer, bounded: no read starts at/past end. */
int cache_binput(char *adr, const char *end, char *s, int max);
int cache_binput(const char *adr, const char *end, char *s, int max);
int cache_brint(char *adr, int *i);
void cache_rint(FILE * fp, int *i);
void cache_rLLint(FILE * fp, LLint * i);

View File

@@ -390,6 +390,10 @@ HTSEXT_API char *hts_convertStringSystemToUTF8(const char *s, size_t size) {
return hts_convertStringCPToUTF8(s, size, GetACP());
}
HTSEXT_API char *hts_convertStringUTF8ToSystem(const char *s, size_t size) {
return hts_convertStringCPFromUTF8(s, size, GetACP());
}
HTSEXT_API void hts_argv_utf8(int *pargc, char ***pargv) {
int wargc = 0;
LPWSTR *const wargv = CommandLineToArgvW(GetCommandLineW(), &wargc);

View File

@@ -171,12 +171,25 @@ extern LPWSTR hts_convertUTF8StringToUCS2(const char *s, int size, int *pwsize);
**/
extern char *hts_convertUCS2StringToUTF8(LPWSTR woutput, int wsize);
/**
* UTF-8 path to UCS-2 for the wide file/FindFirst APIs, \\?\-prefixed above
* MAX_PATH (#133). Internal, not exported; caller frees.
* This function is WIN32 specific.
**/
extern LPWSTR hts_pathToUCS2(const char *path);
/**
* Convert current system codepage to UTF-8.
* This function is WIN32 specific.
**/
HTSEXT_API char *hts_convertStringSystemToUTF8(const char *s, size_t size);
/**
* Convert UTF-8 to the current system codepage. Caller frees; NULL upon error.
* This function is WIN32 specific.
**/
HTSEXT_API char *hts_convertStringUTF8ToSystem(const char *s, size_t size);
/**
* Replace the CRT's ANSI argv by a UTF-8 one decoded from the real UTF-16
* command line: every char* is UTF-8 on Windows (FOPEN, STAT, ... convert at

95
src/htscmdline.c Normal file
View File

@@ -0,0 +1,95 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* File: command line splitter, shared by the engine and */
/* htsserver */
/* Author: Xavier Roche */
/* ------------------------------------------------------------ */
#include "htscmdline.h"
#include "htssafe.h"
#include <limits.h>
#include <stdint.h>
char **hts_split_cmdline(char *cmd, int *nargs) {
size_t nsep = 0;
size_t capacity;
size_t r;
size_t w;
int argc = 0;
hts_boolean quoted = HTS_FALSE;
char **argv;
*nargs = 0;
/* fold the other separators, so counting them sizes the vector exactly */
for (r = 0; cmd[r] != '\0'; r++) {
if (cmd[r] == '\t' || cmd[r] == '\r' || cmd[r] == '\n') {
cmd[r] = ' ';
}
if (cmd[r] == ' ') {
nsep++;
}
}
/* at most one argument per separator, plus the leading one and the NULL */
if (nsep > (size_t) INT_MAX - 1 || nsep > SIZE_MAX / sizeof(char *) - 2) {
return NULL;
}
capacity = nsep + 2;
argv = (char **) malloct(capacity * sizeof(char *));
if (argv == NULL) {
return NULL;
}
argv[argc++] = cmd;
for (r = 0, w = 0; cmd[r] != '\0';) {
if (quoted && cmd[r] == '\\' &&
(cmd[r + 1] == '\\' || cmd[r + 1] == '\"')) {
r++;
cmd[w++] = cmd[r++];
} else if (cmd[r] == '\"') {
quoted = !quoted;
cmd[w++] = cmd[r++];
} else if (cmd[r] == ' ' && !quoted) {
cmd[w++] = '\0';
assertf((size_t) argc < capacity - 1); /* the last slot holds the NULL */
argv[argc++] = cmd + w;
r++;
} else {
cmd[w++] = cmd[r++];
}
}
cmd[w] = '\0';
argv[argc] = NULL; /* callers may rely on argv[argc] == NULL */
*nargs = argc;
return argv;
}

45
src/htscmdline.h Normal file
View File

@@ -0,0 +1,45 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* File: command line splitter, shared by the engine and */
/* htsserver */
/* Author: Xavier Roche */
/* ------------------------------------------------------------ */
#ifndef HTSCMDLINE_DEFH
#define HTSCMDLINE_DEFH
#include "htsglobal.h"
/* Split "cmd" in place into a NULL-terminated argv vector of *nargs entries,
argv[0] being the program name and quotes left for the engine to strip.
Returns a malloct'ed vector of pointers into cmd (freet the vector, never its
entries), or NULL when it cannot be sized or allocated. */
char **hts_split_cmdline(char *cmd, int *nargs);
#endif

View File

@@ -40,6 +40,7 @@ Please visit our Website: http://www.httrack.com
/* File defs */
#include "htscore.h"
#include "htswarc.h"
#include "htssinglefile.h"
/* specific definitions */
#include "htsbase.h"
@@ -447,13 +448,22 @@ void hts_finish_makeindex(httrackp *opt, int *makeindex_done,
const char *fil) {
if (!*makeindex_done) {
if (*makeindex_fp) {
char BIGSTK tempo[1024];
/* sized off link_escaped below: at the old flat 1024 a long first link
produced a redirect to a clipped URL */
char BIGSTK tempo[HTS_URLMAXSIZE * 2 + 64];
if (makeindex_links == 1) {
char BIGSTK link_escaped[HTS_URLMAXSIZE * 2];
escape_uri_utf(makeindex_firstlink, link_escaped, sizeof(link_escaped));
snprintf(tempo, sizeof(tempo),
"<meta HTTP-EQUIV=\"Refresh\" CONTENT=\"0; URL=%s\">" CRLF,
link_escaped);
/* no redirect beats one pointing at a clipped URL */
if (!sprintfbuff(
tempo,
"<meta HTTP-EQUIV=\"Refresh\" CONTENT=\"0; URL=%s\">" CRLF,
link_escaped)) {
hts_log_print(opt, LOG_WARNING,
"index redirect omitted: first link too long (%s)",
makeindex_firstlink);
tempo[0] = '\0';
}
} else
tempo[0] = '\0';
hts_template_format(*makeindex_fp, template_footer,
@@ -696,17 +706,19 @@ int httpmirror(char *url1, httrackp * opt) {
// copier adresse(s) dans liste des adresses
{
char *a = url1;
int primary_len = 8192;
if (StringNotEmpty(opt->filelist)) {
primary_len += max(0, fsize(StringBuff(opt->filelist)) * 2);
}
primary_len += (int) strlen(url1) * 2;
const LLint list_sz = StringNotEmpty(opt->filelist)
? fsize_utf8(StringBuff(opt->filelist))
: 0;
/* two bytes reserved per list byte; -1 makes an undoublable size refused */
const LLint list_room =
list_sz > 0 ? (list_sz <= INT64_MAX / 2 ? list_sz * 2 : -1) : 0;
const size_t primary_len =
llint_grow_size_t(8192 + strlen(url1) * 2, list_room, 0);
// création de la première page, qui contient les liens de base à scanner
// c'est plus propre et plus logique que d'entrer à la main les liens dans la pile
// on bénéficie ainsi des vérifications et des tests du robot pour les liens "primaires"
primary = (char *) malloct(primary_len);
primary = primary_len != (size_t) -1 ? (char *) malloct(primary_len) : NULL;
if (!primary) {
printf("PANIC! : Not enough memory [%d]\n", __LINE__);
XH_extuninit;
@@ -856,19 +868,19 @@ int httpmirror(char *url1, httrackp * opt) {
char *filelist_buff = NULL;
size_t filelist_sz = 0;
const char *filelist_err = NULL; /* failure reason, NULL on success */
const LLint fs = fsize(StringBuff(opt->filelist));
const LLint fs = fsize_utf8(StringBuff(opt->filelist));
if (fs < 0) {
/* fsize() hides the cause; redo stat() for a precise errno (#49) */
struct stat st;
filelist_err = stat(StringBuff(opt->filelist), &st) != 0
STRUCT_STAT st;
filelist_err = STAT(StringBuff(opt->filelist), &st) != 0
? strerror(errno)
: "not a regular file";
} else if ((filelist_sz = llint_to_size_t(fs)) == (size_t) -1) {
filelist_err = "file too large";
filelist_sz = 0;
} else {
FILE *fp = fopen(StringBuff(opt->filelist), "rb");
FILE *fp = FOPEN(StringBuff(opt->filelist), "rb");
if (fp == NULL) {
filelist_err = strerror(errno);
@@ -887,7 +899,7 @@ int httpmirror(char *url1, httrackp * opt) {
}
if (filelist_buff != NULL) {
int filelist_ptr = 0;
size_t filelist_ptr = 0;
int n = 0;
char BIGSTK line[HTS_URLMAXSIZE * 2];
@@ -976,10 +988,9 @@ int httpmirror(char *url1, httrackp * opt) {
}
// statistiques
if (opt->makestat) {
makestat_fp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-stats.txt"),
"wb");
makestat_fp = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-stats.txt"),
"wb");
if (makestat_fp != NULL) {
fprintf(makestat_fp, "HTTrack statistics report, every minutes" LF LF);
fflush(makestat_fp);
@@ -987,10 +998,9 @@ int httpmirror(char *url1, httrackp * opt) {
}
// tracking -- débuggage
if (opt->maketrack) {
maketrack_fp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-track.txt"),
"wb");
maketrack_fp = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-track.txt"),
"wb");
if (maketrack_fp != NULL) {
fprintf(maketrack_fp, "HTTrack tracking report, every minutes" LF LF);
fflush(maketrack_fp);
@@ -1639,7 +1649,8 @@ int httpmirror(char *url1, httrackp * opt) {
/* Remove file if being processed */
if (is_loaded_from_file) {
(void) unlink(fconv(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), savename()));
(void) UNLINK(
fconv(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), savename()));
is_loaded_from_file = 0;
}
@@ -1952,8 +1963,8 @@ int httpmirror(char *url1, httrackp * opt) {
} else */
/* External modules */
if (opt->parsejava && (opt->parsejava & HTSPARSE_NO_CLASS) == 0
&& fexist(savename())) {
if (opt->parsejava && (opt->parsejava & HTSPARSE_NO_CLASS) == 0 &&
fexist_utf8(savename())) {
char BIGSTK buff_err_msg[1024];
htsmoduleStruct BIGSTK str;
@@ -1994,7 +2005,7 @@ int httpmirror(char *url1, httrackp * opt) {
} // text/html ou autre
/* Post-processing */
if (fexist(savename())) {
if (fexist_utf8(savename())) {
usercommand(opt, 0, NULL, savename(), urladr(), urlfil());
}
@@ -2085,18 +2096,16 @@ int httpmirror(char *url1, httrackp * opt) {
//
opt->state._hts_in_html_parsing = 3;
//
old_lst =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.lst"), "rb");
old_lst = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/old.lst"),
"rb");
if (old_lst) {
const size_t sz = llint_to_size_t(
fsize(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.lst")));
new_lst =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.lst"), "rb");
const size_t sz = llint_to_size_t(fsize_utf8(
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.lst")));
new_lst = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.lst"),
"rb");
if (new_lst != NULL && sz != (size_t) -1) {
/* +1 for the NUL below: new.lst is read raw, and the strstr()
that follows needs a terminated C string. */
@@ -2116,9 +2125,9 @@ int httpmirror(char *url1, httrackp * opt) {
strcpybuff(file, StringBuff(opt->path_html));
strcatbuff(file, line + 1);
file[strlen(file) - 1] = '\0';
if (fexist(file)) { // toujours sur disque: virer
if (fexist_utf8(file)) { // toujours sur disque: virer
hts_log_print(opt, LOG_INFO, "Purging %s", file);
remove(file);
UNLINK(file);
purge = 1;
}
}
@@ -2139,7 +2148,8 @@ int httpmirror(char *url1, httrackp * opt) {
strcpybuff(file, StringBuff(opt->path_html));
strcatbuff(file, line + 1);
while((strnotempty(file)) && (rmdir(file) == 0)) { // ok, éliminé (existait)
while ((strnotempty(file)) &&
(RMDIR(file) == 0)) { // ok, éliminé (existait)
purge = 1;
if (opt->log) {
hts_log_print(opt, LOG_INFO, "Purging directory %s/",
@@ -2173,6 +2183,10 @@ int httpmirror(char *url1, httrackp * opt) {
}
// fin purge!
/* --single-file: inline each page's assets now that the tree is final (after
the purge, so we never rewrite a file that is about to be deleted). */
singlefile_process_mirror(opt);
// Indexation
if (opt->kindex)
index_finish(StringBuff(opt->path_html), opt->kindex);
@@ -2962,8 +2976,8 @@ static void postprocess_file(httrackp *opt, const char *save, const char *adr,
if (adr != NULL && strcmp(adr, "primary") == 0) {
adr = NULL;
}
if (save != NULL && opt != NULL && adr != NULL && adr[0]
&& strnotempty(save) && fexist(save)) {
if (save != NULL && opt != NULL && adr != NULL && adr[0] &&
strnotempty(save) && fexist_utf8(save)) {
const char *rsc_save = save;
const char *rsc_fil = strrchr(fil, '/');
int n;
@@ -2985,13 +2999,11 @@ static void postprocess_file(httrackp *opt, const char *save, const char *adr,
if (!opt->state.mimehtml_created) {
opt->state.mimefp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html), "index.mht"),
"wb");
(void) unlink(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html),
"index.eml"));
FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html), "index.mht"),
"wb");
(void) UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html), "index.eml"));
#ifndef _WIN32
if (symlink("index.mht",
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_html),
@@ -3632,6 +3644,13 @@ HTSEXT_API int copy_htsopt(const httrackp * from, httrackp * to) {
if (StringNotEmpty(from->warc_file))
StringCopyS(to->warc_file, from->warc_file);
to->warc_max_size = from->warc_max_size;
to->warc_cdx = from->warc_cdx;
to->warc_wacz = from->warc_wacz;
to->single_file = from->single_file;
if (from->single_file_max_size > 0)
to->single_file_max_size = from->single_file_max_size;
if (from->pause_max_ms > 0) {
to->pause_min_ms = from->pause_min_ms;

View File

@@ -145,7 +145,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
int argv_url = -1; // ==0 : utiliser cache et doit.log
char *argv_firsturl = NULL; // utilisé pour nommage par défaut
char *url = NULL; // URLS séparées par un espace
int url_sz = 65535;
size_t url_sz = 65535;
// the parametres
int httrack_logmode = 3; // ONE log file
@@ -224,21 +224,22 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
/* create x_argvblk buffer for transformed command line */
{
int current_size = 0;
int size;
size_t current_size = 0;
const LLint size = fsize("config");
size_t blk_size;
int na;
for(na = 0; na < argc; na++)
current_size += (int) (strlen(argv[na]) + 1);
if ((size = fsize("config")) > 0)
current_size += size;
x_argvblk = (char *) malloct(current_size + 32768);
current_size += strlen(argv[na]) + 1;
/* a huge file named "config" must saturate, not wrap, the capacity */
blk_size = llint_grow_size_t(current_size, size > 0 ? size : 0, 32768);
x_argvblk = blk_size != (size_t) -1 ? (char *) malloct(blk_size) : NULL;
if (x_argvblk == NULL) {
HTS_PANIC_PRINTF("Error, not enough memory");
htsmain_free();
return -1;
}
x_argvblk_size = (size_t) (current_size + 32768);
x_argvblk_size = blk_size;
x_argvblk[0] = '\0';
x_ptr = 0;
@@ -442,11 +443,10 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
|| (strnotempty(StringBuff(opt->path_html))))
loops++; // do not loop once again and do not include rc file (O option exists)
else {
if ((!fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/doit.log"))) || (argv_url > 0)) {
if ((!fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/doit.log"))) ||
(argv_url > 0)) {
if (!optinclude_file(
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), HTS_HTTRACKRC),
@@ -473,16 +473,12 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
} // traiter -O
/* load doit.log and insert in current command line */
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/doit.log"))
&& (argv_url <= 0)) {
FILE *fp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/doit.log"), "rb");
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/doit.log")) &&
(argv_url <= 0)) {
FILE *fp = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/doit.log"),
"rb");
if (fp) {
int insert_after = 1; /* insérer après nom au début */
@@ -531,23 +527,16 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
/* Interrupted mirror detected */
if (!opt->quiet) {
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-in_progress.lock"))) {
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-in_progress.lock"))) {
/* Old cache */
if ((fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.dat")))
&&
(fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.ndx")))) {
if ((fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.dat"))) &&
(fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.ndx")))) {
if (opt->log != NULL) {
fprintf(opt->log, "Warning!\n");
fprintf(opt->log,
@@ -576,111 +565,82 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
if (argv[i][1] == '-') { // --xxx
if ((strfield2(argv[i] + 2, "clean")) || (strfield2(argv[i] + 2, "tide"))) { // nettoyer
argv[i][1] = '\0';
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-log.txt")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-log.txt"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-err.txt")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-err.txt"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_html), "index.html")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_html),
"index.html"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-log.txt")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-log.txt"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-err.txt")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-err.txt"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html), "index.html")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html), "index.html"));
/* */
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.zip")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.zip"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.zip")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.zip"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.dat")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.dat"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.ndx")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.ndx"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.dat")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.dat"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.ndx")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.ndx"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.lst")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.lst"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.lst")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.lst"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.txt")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.txt"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.txt")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.txt"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/doit.log")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/doit.log"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-in_progress.lock")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-in_progress.lock"));
rmdir(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-cache"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.zip")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.zip"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.zip")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/old.zip"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.dat")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.dat"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.ndx")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.ndx"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.dat")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/old.dat"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.ndx")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/old.ndx"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.lst")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.lst"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.lst")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/old.lst"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.txt")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.txt"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.txt")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/old.txt"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/doit.log")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/doit.log"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-in_progress.lock")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-in_progress.lock"));
RMDIR(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache"));
//
} else if (strfield2(argv[i] + 2, "catchurl")) { // capture d'URL via proxy temporaire!
argv_url = 1; // forcer a passer les parametres
@@ -737,29 +697,27 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
#endif
if (argv_url == 0) {
// Présence d'un cache, que faire?..
if ((fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.zip")))
||
(fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.dat"))
&&
fexist(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.ndx")))
) { // il existe déja un cache précédent.. renommer
if (fexist(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/doit.log"))) { // un cache est présent
if ((fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.zip"))) ||
(fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.dat")) &&
fexist_utf8(
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.ndx")))) { // il existe déja un cache
// précédent.. renommer
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/doit.log"))) { // un cache est présent
if (x_argvblk != NULL) {
int m;
// établir mode - mode cache: 1 (cache valide) 2 (cache à vérifier)
if (fexist(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-in_progress.lock"))) { // cache prioritaire
if (fexist_utf8(
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-in_progress.lock"))) { // cache prioritaire
m = 1;
} else {
m = 2;
@@ -794,7 +752,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
htsmain_free();
return -1;
}
} else { // log existe pas
} else { // log existe pas
HTS_PANIC_PRINTF("A cache has been found, but no command line");
printf
("Please launch httrack with proper parameters to reuse the cache\n");
@@ -802,7 +760,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
return -1;
}
} else { // aucune URL définie et pas de cache
} else { // aucune URL définie et pas de cache
if (opt->quiet) {
help(argv[0], !opt->quiet);
htsmain_free();
@@ -817,26 +775,21 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
}
} else { // plus de 2 paramètres
// un fichier log existe?
if (fexist(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-in_progress.lock"))) { // fichier lock?
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-in_progress.lock"))) { // fichier lock?
opt->cache = HTS_CACHE_PRIORITY; // cache prioritaire
if (opt->quiet == 0) {
if ((fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.zip")))
||
(fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.dat"))
&&
fexist(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.ndx")))
) {
if ((fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.zip"))) ||
(fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.dat")) &&
fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.ndx")))) {
HT_REQUEST_START;
HT_PRINT("There is a lock-file in the directory ");
HT_PRINT(StringBuff(opt->path_log));
@@ -850,24 +803,19 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
}
}
}
} else if (fexist(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html), "index.html"))) {
} else if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html), "index.html"))) {
opt->cache = HTS_CACHE_TEST_UPDATE;
if (opt->quiet == 0) {
if ((fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.zip")))
||
(fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.dat"))
&&
fexist(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.ndx")))
) {
if ((fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.zip"))) ||
(fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.dat")) &&
fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.ndx")))) {
HT_REQUEST_START;
HT_PRINT
("There is an index.html and a hts-cache folder in the directory ");
@@ -1500,29 +1448,38 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
LLint fz;
na++;
fz = fsize(argv[na]);
fz = fsize_utf8(argv[na]);
if (fz < 0) {
HTS_PANIC_PRINTF("File url list could not be opened");
htsmain_free();
return -1;
} else {
FILE *fp = fopen(argv[na], "rb");
FILE *fp = FOPEN(argv[na], "rb");
if (fp != NULL) {
int cl = (int) strlen(url);
size_t cl = strlen(url);
const size_t fzs = llint_to_size_t(fz);
const size_t capa = llint_grow_size_t(cl, fz, 8192);
ensureUrlCapacity(url, url_sz, cl + fz + 8192);
if (capa == (size_t) -1) {
fclose(fp);
HTS_PANIC_PRINTF("File url list too large");
htsmain_free();
return -1;
}
ensureUrlCapacity(url, url_sz, capa);
if (cl > 0) { /* don't stick! (3.43) */
url[cl] = ' ';
cl++;
}
if (fread(url + cl, 1, fz, fp) != fz) {
if (fread(url + cl, 1, fzs, fp) != fzs) {
fclose(fp);
HTS_PANIC_PRINTF("File url list could not be read");
htsmain_free();
return -1;
}
fclose(fp);
*(url + cl + fz) = '\0';
*(url + cl + fzs) = '\0';
}
}
}
@@ -1806,10 +1763,68 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
return -1;
}
StringCopy(opt->warc_file, argv[na]);
} else if (*(com + 1) == 's') { // --warc-max-size N: rotation
com++;
if ((na + 1 >= argc) || (argv[na + 1][0] == '-')) {
HTS_PANIC_PRINTF(
"Option warc-max-size needs a blank space and a size");
htsmain_free();
return -1;
}
na++;
{ // reject non-numeric/negative/overflow; keep default 0
// (single file)
char *end;
LLint v;
errno = 0;
v = strtoll(argv[na], &end, 10);
if (isdigit((unsigned char) argv[na][0]) && *end == '\0' &&
errno != ERANGE)
opt->warc_max_size = v;
}
} else if (*(com + 1) == 'c') { // --warc-cdx: sorted CDXJ index
com++;
opt->warc_cdx = 1;
} else if (*(com + 1) == 'z') { // --wacz: WACZ package
com++;
opt->warc_wacz = 1;
opt->warc_cdx = 1; // WACZ embeds the CDXJ index
if (!StringNotEmpty(opt->warc_file))
StringCopy(opt->warc_file, WARC_AUTONAME);
} else { // --warc: auto-named archive under the output dir
StringCopy(opt->warc_file, WARC_AUTONAME);
}
break;
case 'Z': // single-file: inline each page's assets as data: URIs
if (*(com + 1) == 's') { // --single-file-max-size N: per-asset
com++;
if ((na + 1 >= argc) || (argv[na + 1][0] == '-')) {
HTS_PANIC_PRINTF(
"Option single-file-max-size needs a blank "
"space and a size");
htsmain_free();
return -1;
}
na++;
{ // reject non-numeric/negative/overflow; keep the default
char *end;
LLint v;
errno = 0;
v = strtoll(argv[na], &end, 10);
if (isdigit((unsigned char) argv[na][0]) && *end == '\0' &&
errno != ERANGE && v > 0)
opt->single_file_max_size = v;
}
opt->single_file = HTS_TRUE;
} else {
opt->single_file = HTS_TRUE;
if (*(com + 1) == '0') {
opt->single_file = HTS_FALSE;
com++;
}
}
break;
case 'Y': // why: explain the filter verdict for a URL, no crawl
if ((na + 1 >= argc) || (argv[na + 1][0] == '-')) {
HTS_PANIC_PRINTF("Option why needs a blank space and a URL");
@@ -2022,7 +2037,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
/*cache */ &cache, /*hash */ NULL, /*ptr */
0, /*numero_passe */ 0, /*mime_type */
NULL) != -1) {
if (fexist(afs.save)) {
if (fexist_utf8(afs.save)) {
fprintf(stdout, "Content-location: %s\r\n",
afs.save);
}
@@ -2120,18 +2135,16 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
uLong repaired = 0;
uLong repairedBytes = 0;
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.zip"))) {
if (fexist_utf8(fconcat(
OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.zip"))) {
name =
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.zip");
} else
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.zip"))) {
} else if (fexist_utf8(fconcat(OPT_GET_BUFF(opt),
OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.zip"))) {
name =
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.zip");
@@ -2150,10 +2163,11 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/repair.tmp"), &repaired,
&repairedBytes) == Z_OK) {
unlink(name);
rename(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/repair.zip"), name);
UNLINK(name);
RENAME(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/repair.zip"),
name);
fprintf(stderr,
"Cache: %d bytes successfully recovered in %d entries\n",
(int) repairedBytes, (int) repaired);
@@ -2322,8 +2336,8 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
} else { // URL/filters
char catbuff[CATBUFF_SIZE];
const int urlSize = (int) strlen(argv[na]);
const int capa = (int) (strlen(url) + urlSize + 32);
const size_t urlSize = strlen(argv[na]);
const size_t capa = strlen(url) + urlSize + 32;
assertf(urlSize < HTS_URLMAXSIZE);
if (urlSize < HTS_URLMAXSIZE) {
@@ -2371,10 +2385,9 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
hts_cache_reconcile(opt, CACHE_RECONCILE_INTERRUPTED);
// Débuggage des en têtes
if (_DEBUG_HEAD) {
ioinfo =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-ioinfo.txt"),
"wb");
ioinfo = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-ioinfo.txt"),
"wb");
}
{
@@ -2449,9 +2462,9 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
/* readme for information purpose */
{
FILE *fp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/readme.txt"), "wb");
FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/readme.txt"),
"wb");
if (fp) {
fprintf(fp, "What's in this folder?" LF);
fprintf(fp, "" LF);
@@ -2503,17 +2516,16 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
int i;
#ifdef _WIN32
mkdir(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-cache"));
hts_mkdir_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache"));
#else
mkdir(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-cache"),
HTS_PROTECT_FOLDER);
#endif
fp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/doit.log"), "wb");
fp = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/doit.log"),
"wb");
if (fp) {
for(i = 0 + 1; i < argc; i++) {
if (((strchr(argv[i], ' ') != NULL)
@@ -2558,7 +2570,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
}
}
// petit message dans le lock
if ((fp = fopen(n_lock, "wb")) != NULL) {
if ((fp = FOPEN(n_lock, "wb")) != NULL) {
int i;
fprintf(fp, "Mirror in progress since %s .. please wait!" LF, t);
@@ -2718,16 +2730,15 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
char *f = OPT_GET_BUFF(opt);
sprintf(f, "%s/%s", CACHE_REFNAME, entry->d_name);
(void)
unlink(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), f));
(void) UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), f));
}
}
if (dir != NULL) {
(void) closedir(dir);
}
(void)
rmdir(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), CACHE_REFNAME));
(void) RMDIR(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), CACHE_REFNAME));
}
/* Info for wrappers */
@@ -2755,7 +2766,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
}
}
// supprimer lock
remove(n_lock);
UNLINK(n_lock);
}
if (x_argvblk)

View File

@@ -43,8 +43,8 @@ Please visit our Website: http://www.httrack.com
configure.ac, decoupled from these). VERSION is the display form, VERSIONID
the dotted numeric form, AFF_VERSION the short form shown in footers,
LIB_VERSION the data/cache format generation. */
#define HTTRACK_VERSION "3.49-13"
#define HTTRACK_VERSIONID "3.49.13"
#define HTTRACK_VERSION "3.49-14"
#define HTTRACK_VERSIONID "3.49.14"
#define HTTRACK_AFF_VERSION "3.x"
#define HTTRACK_LIB_VERSION "2.0"
@@ -72,7 +72,8 @@ Please visit our Website: http://www.httrack.com
HTS_UNUSED: suppress unused-symbol warnings. HTS_STATIC: an unused-safe
static. HTS_PRINTF_FUN(fmt, arg): mark a printf-like function so the
compiler type-checks the format string at argument index fmt against the
varargs starting at arg. */
varargs starting at arg. HTS_CHECK_RESULT: the return value carries the only
error signal, so dropping it is a bug; a (void) cast does not silence it. */
#ifndef HTS_UNUSED
#ifdef __GNUC__
#define HTS_UNUSED __attribute__((unused))
@@ -80,10 +81,13 @@ Please visit our Website: http://www.httrack.com
#define HTS_STATIC static __attribute__((unused))
#define HTS_PRINTF_FUN(fmt, arg) __attribute__((format(printf, fmt, arg)))
#define HTS_CHECK_RESULT __attribute__((warn_unused_result))
#else
#define HTS_UNUSED
#define HTS_STATIC static
#define HTS_PRINTF_FUN(fmt, arg)
#define HTS_CHECK_RESULT
#endif
#endif
@@ -336,16 +340,20 @@ typedef int INTsys;
#endif
/* Socket-handle type. An unsigned integer wide enough for a Windows SOCKET;
a plain int file descriptor on POSIX. */
a plain int file descriptor on POSIX. T_SOCP is its printf conversion,
'%' included: unsigned __int64 on Win64 must not be printed with "%d". */
#ifdef _WIN32
#if defined(_WIN64)
typedef unsigned __int64 T_SOC;
#define T_SOCP "%" PRIu64
#else
typedef unsigned __int32 T_SOC;
#define T_SOCP "%" PRIu32
#endif
#else
typedef int T_SOC;
#define T_SOCP "%d"
#endif
/* Buffer size for a printed network address (IPv4 or IPv6, NUL included). */

View File

@@ -124,7 +124,9 @@ typedef struct help_wizard_buffers {
char stropt[2048]; // options
char stropt2[2048]; // options longues
char strwild[2048]; // wildcards
char cmd[4096];
/* holds all four of the above plus separators: at 4096 a long answer set
clipped the filters off the command line */
char cmd[HTS_URLMAXSIZE * 2 + 3 * 2048 + 4];
char str[256];
char *argv[256];
} help_wizard_buffers;
@@ -156,9 +158,7 @@ void help_wizard(httrackp * opt) {
char *a;
//
if (urls == NULL || mainpath == NULL || projname == NULL || stropt == NULL
|| stropt2 == NULL || strwild == NULL || cmd == NULL || str == NULL
|| argv == NULL) {
if (buffers == NULL) {
fprintf(stderr, "* memory exhausted in %s, line %d\n", __FILE__, __LINE__);
return;
}
@@ -251,6 +251,7 @@ void help_wizard(httrackp * opt) {
strcatbuff(stropt2, "--update ");
break;
case 0:
freet(buffers);
return;
break;
}
@@ -309,14 +310,23 @@ void help_wizard(httrackp * opt) {
printf("\n");
if (strlen(stropt) == 1)
stropt[0] = '\0'; // aucune
snprintf(cmd, sizeof(cmd), "%s %s %s %s", urls, stropt, stropt2, strwild);
/* the tail is the filter list, and cmd is split into the argv handed to
hts_main() below: a clipped line would silently widen the crawl */
if (!sprintfbuff(cmd, "%s %s %s %s", urls, stropt, stropt2, strwild)) {
printf("* command line too long (%d bytes max)\n",
(int) sizeof(cmd) - 1);
freet(buffers);
return;
}
printf("---> Wizard command line: httrack %s\n\n", cmd);
printf("Ready to launch the mirror? (Y/n) :");
fflush(stdout);
linput(stdin, str, 250);
if (strnotempty(str)) {
if (!((str[0] == 'y') || (str[0] == 'Y')))
if (!((str[0] == 'y') || (str[0] == 'Y'))) {
freet(buffers);
return;
}
}
printf("\n");
@@ -340,7 +350,7 @@ void help_wizard(httrackp * opt) {
}
/* Free buffers */
free(buffers);
freet(buffers);
#undef urls
#undef mainpath
#undef projname
@@ -412,9 +422,9 @@ void help_catchurl(const char *dest_path) {
do {
snprintf(dest, sizeof(dest), "%s%s%d", dest_path, "hts-post", i);
i++;
} while(fexist(dest));
} while (fexist_utf8(dest));
{
FILE *fp = fopen(dest, "wb");
FILE *fp = FOPEN(dest, "wb");
if (fp) {
fwrite(data, strlen(data), 1, fp);
@@ -524,8 +534,11 @@ void help(const char *app, int more) {
infomsg
(" %D cached delayed type check, don't wait for remote type during updates, to speedup them (%D0 wait, * %D1 don't wait)");
infomsg(" %M generate a RFC MIME-encapsulated full-archive (.mht)");
infomsg(" %r write an ISO-28500 WARC/1.1 archive; --warc-file NAME sets the "
"output name");
infomsg(" %Z after the mirror, rewrite each saved page with its "
"stylesheets, scripts, images and fonts inlined as data: URIs, so "
"any page also opens on its own (links between pages stay relative; "
"audio and video stay links); --single-file-max-size N caps each "
"asset (default 10485760 bytes)");
infomsg(" %t keep the original file extension, don't rewrite it from the "
"MIME type (%t0 rewrite)");
infomsg
@@ -595,6 +608,10 @@ void help(const char *app, int more) {
infomsg
(" C create/use a cache for updates and retries (C0 no cache,C1 cache is prioritary,* C2 test update before)");
infomsg(" k store all files in cache (not useful if files on disk)");
infomsg(" %r write an ISO-28500 WARC/1.1 archive; --warc-file NAME sets the "
"output name, --warc-max-size N rotates segments past N bytes, "
"--warc-cdx also writes a sorted CDXJ index, --wacz packages it all "
"as a WACZ file");
infomsg(" %n do not re-download locally erased files");
infomsg
(" %v display on screen filenames downloaded (in realtime) - * %v1 short version - %v2 full animation");

View File

@@ -349,9 +349,13 @@ void index_finish(const char *indexpath, int mode) {
// Write new file
if (mode == 1) // TEXT
fp = fopen(concat(catbuff, sizeof(catbuff), indexpath, "index.txt"), "wb");
fp = FOPEN(
concat(catbuff, sizeof(catbuff), indexpath, "index.txt"),
"wb");
else // HTML
fp = fopen(concat(catbuff, sizeof(catbuff), indexpath, "sindex.html"), "wb");
fp = FOPEN(
concat(catbuff, sizeof(catbuff), indexpath, "sindex.html"),
"wb");
if (fp) {
char current_word[KEYW_LEN + 32];
char word[KEYW_LEN + 32];

View File

@@ -37,6 +37,7 @@ Please visit our Website: http://www.httrack.com
#include "htscore.h"
#include "htswarc.h"
#include "htssinglefile.h"
/* specific definitions */
#include "htsbase.h"
@@ -704,18 +705,16 @@ T_SOC http_xfopen(httrackp *opt, int mode, int treat, int waitconnect,
/* Check for errors */
if (soc == INVALID_SOCKET) {
if (retour) {
if (retour->msg) {
if (!strnotempty(retour->msg)) {
if (!strnotempty(retour->msg)) {
#ifdef _WIN32
int last_errno = WSAGetLastError();
int last_errno = WSAGetLastError();
sprintf(retour->msg, "Connect error: %s", strerror(last_errno));
sprintf(retour->msg, "Connect error: %s", strerror(last_errno));
#else
int last_errno = errno;
int last_errno = errno;
sprintf(retour->msg, "Connect error: %s", strerror(last_errno));
sprintf(retour->msg, "Connect error: %s", strerror(last_errno));
#endif
}
}
}
}
@@ -2207,7 +2206,7 @@ T_SOC newhttp_addr(httrackp *opt, const char *_iadr, htsblk *retour, int port,
#if DEBUG
printf("erreur gethostbyname\n");
#endif
if (retour && retour->msg) {
if (retour != NULL) {
#ifdef _WIN32
snprintf(retour->msg, sizeof(retour->msg),
"Unable to get server's address: %s", error);
@@ -2238,7 +2237,7 @@ T_SOC newhttp_addr(httrackp *opt, const char *_iadr, htsblk *retour, int port,
DEBUG_W("socket()=%d\n" _(int) soc);
#endif
if (soc == INVALID_SOCKET) {
if (retour && retour->msg) {
if (retour != NULL) {
#ifdef _WIN32
int last_errno = WSAGetLastError();
@@ -2262,17 +2261,8 @@ T_SOC newhttp_addr(httrackp *opt, const char *_iadr, htsblk *retour, int port,
&bind_addr, &error) == NULL
|| bind(soc, &SOCaddr_sockaddr(bind_addr),
SOCaddr_size(bind_addr)) != 0) {
if (retour && retour->msg) {
#ifdef _WIN32
snprintf(retour->msg, sizeof(retour->msg),
"Unable to bind the specificied server address: %s",
error);
#else
snprintf(retour->msg, sizeof(retour->msg),
"Unable to bind the specificied server address: %s",
error);
#endif
}
snprintf(retour->msg, sizeof(retour->msg),
"Unable to bind the specificied server address: %s", error);
deletesoc(soc);
return INVALID_SOCKET;
}
@@ -2319,7 +2309,7 @@ T_SOC newhttp_addr(httrackp *opt, const char *_iadr, htsblk *retour, int port,
#if HDEBUG
printf("unable to connect!\n");
#endif
if (retour != NULL && retour->msg) {
if (retour != NULL) {
#ifdef _WIN32
const int last_errno = WSAGetLastError();
@@ -2598,13 +2588,15 @@ void deletesoc(T_SOC soc) {
if (closesocket(soc) != 0) {
int err = WSAGetLastError();
fprintf(stderr, "* error closing socket %d: %s\n", soc, strerror(err));
fprintf(stderr, "* error closing socket " T_SOCP ": %s\n", soc,
strerror(err));
}
#else
if (close(soc) != 0) {
const int err = errno;
fprintf(stderr, "* error closing socket %d: %s\n", soc, strerror(err));
fprintf(stderr, "* error closing socket " T_SOCP ": %s\n", soc,
strerror(err));
}
#endif
#if HTS_WIDE_DEBUG
@@ -3017,7 +3009,7 @@ int finput(T_SOC fd, char *s, int max) {
}
// Like linput, but in memory (optimized)
int binput(char *buff, char *s, int max) {
int binput(const char *buff, char *s, int max) {
int count = 0;
int destCount = 0;
@@ -6019,6 +6011,9 @@ HTSEXT_API httrackp *hts_create_opt(void) {
StringCopy(opt->strip_query, "");
StringCopy(opt->cookies_file, "");
StringCopy(opt->warc_file, "");
opt->warc_max_size = 0; /* no rotation unless --warc-max-size sets it */
opt->single_file = HTS_FALSE;
opt->single_file_max_size = SINGLEFILE_DEFAULT_MAX_SIZE;
StringCopy(opt->why_url, "");
opt->pause_min_ms = 0;
opt->pause_max_ms = 0;
@@ -6430,21 +6425,28 @@ HTSEXT_API int hts_resetvar(void) {
#ifdef _WIN32
typedef struct dirent dirent;
DIR *opendir(const char *name) {
WIN32_FILE_ATTRIBUTE_DATA st;
DIR *dir;
size_t len;
int i;
LPWSTR wname;
if (name == NULL || *name == '\0') {
errno = ENOENT;
return NULL;
}
if (!GetFileAttributesEx(name, GetFileExInfoStandard, &st)
|| (st.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) == 0) {
// Wide \\?\ path: no MAX_PATH cap, no CP_ACP mis-decode (#133,#630).
wname = hts_pathToUCS2(name);
if (wname == NULL ||
!GetFileAttributesExW(wname, GetFileExInfoStandard, &st) ||
(st.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) == 0) {
freet(wname);
errno = ENOENT;
return NULL;
}
freet(wname);
dir = calloc(sizeof(DIR), 1);
if (dir == NULL) {
errno = ENOMEM;
@@ -6464,19 +6466,29 @@ DIR *opendir(const char *name) {
}
struct dirent *readdir(DIR * dir) {
WIN32_FIND_DATAA find;
WIN32_FIND_DATAW find;
if (dir->h == INVALID_HANDLE_VALUE) {
dir->h = FindFirstFileA(dir->name, &find);
// \\?\-prefix so a long/non-ASCII directory enumerates instead of ENOENT.
LPWSTR wname = hts_pathToUCS2(dir->name);
dir->h =
wname != NULL ? FindFirstFileW(wname, &find) : INVALID_HANDLE_VALUE;
freet(wname);
} else {
if (!FindNextFile(dir->h, &find)) {
if (!FindNextFileW(dir->h, &find)) {
FindClose(dir->h);
dir->h = INVALID_HANDLE_VALUE;
}
}
if (dir->h != INVALID_HANDLE_VALUE) {
char *u = hts_convertUCS2StringToUTF8(find.cFileName, -1);
dir->entry.d_name[0] = 0;
strncat(dir->entry.d_name, find.cFileName, HTS_DIRENT_SIZE - 1);
if (u != NULL) {
strncat(dir->entry.d_name, u, HTS_DIRENT_SIZE - 1);
freet(u);
}
return &dir->entry;
}
errno = ENOENT;
@@ -6509,9 +6521,61 @@ static void copyWchar(LPWSTR dest, const char *src) {
dest[i] = '\0';
}
/* UTF-8 path -> UCS-2 for the _w* file APIs. At/above HTS_WIN_LONGPATH_MIN,
\\?\-prefix it via GetFullPathNameW to clear MAX_PATH (#133); else unchanged.
Any prefixing failure falls back to the plain converted path. */
#define HTS_WIN_LONGPATH_MIN 240 /* stay clear of MAX_PATH (260) */
LPWSTR hts_pathToUCS2(const char *path) {
LPWSTR wpath = hts_convertUTF8StringToUCS2(path, (int) strlen(path), NULL);
if (wpath == NULL) {
return NULL;
}
const size_t len = wcslen(wpath);
// Already "\\?\" or "\\.\": don't re-prefix.
const int verbatim = len >= 4 && wpath[0] == L'\\' && wpath[1] == L'\\' &&
(wpath[2] == L'?' || wpath[2] == L'.') &&
wpath[3] == L'\\';
if (len < HTS_WIN_LONGPATH_MIN || verbatim) {
return wpath;
}
const DWORD need = GetFullPathNameW(wpath, 0, NULL, NULL); /* incl NUL */
LPWSTR full = need != 0 ? malloct((size_t) need * sizeof(WCHAR)) : NULL;
if (full == NULL) {
return wpath; /* fall back to the plain path */
}
const DWORD written = GetFullPathNameW(wpath, need, full, NULL);
if (written == 0 || written >= need || full[0] == L'\0') {
freet(full);
return wpath;
}
const int isUNC = full[0] == L'\\' && full[1] == L'\\';
// UNC "\\srv\share" -> "\\?\UNC\srv\share": the prefix subsumes the "\\".
const WCHAR *const pfx = isUNC ? L"\\\\?\\UNC\\" : L"\\\\?\\";
const WCHAR *const body = isUNC ? full + 2 : full;
const size_t pfxLen = wcslen(pfx), bodyLen = wcslen(body);
LPWSTR out = malloct((pfxLen + bodyLen + 1) * sizeof(WCHAR));
if (out == NULL) {
freet(full);
return wpath;
}
memcpybuff(out, pfx, pfxLen * sizeof(WCHAR));
memcpybuff(out + pfxLen, body, (bodyLen + 1) * sizeof(WCHAR));
freet(full);
freet(wpath);
return out;
}
FILE *hts_fopen_utf8(const char *path, const char *mode) {
WCHAR wmode[32];
LPWSTR wpath = hts_convertUTF8StringToUCS2(path, (int) strlen(path), NULL);
LPWSTR wpath = hts_pathToUCS2(path);
assertf(strlen(mode) < sizeof(wmode) / sizeof(WCHAR));
copyWchar(wmode, mode);
@@ -6527,7 +6591,7 @@ FILE *hts_fopen_utf8(const char *path, const char *mode) {
}
int hts_stat_utf8(const char *path, STRUCT_STAT * buf) {
LPWSTR wpath = hts_convertUTF8StringToUCS2(path, (int) strlen(path), NULL);
LPWSTR wpath = hts_pathToUCS2(path);
if (wpath != NULL) {
const int result = _wstat64(wpath, buf);
@@ -6541,7 +6605,7 @@ int hts_stat_utf8(const char *path, STRUCT_STAT * buf) {
}
int hts_unlink_utf8(const char *path) {
LPWSTR wpath = hts_convertUTF8StringToUCS2(path, (int) strlen(path), NULL);
LPWSTR wpath = hts_pathToUCS2(path);
if (wpath != NULL) {
const int result = _wunlink(wpath);
@@ -6555,10 +6619,8 @@ int hts_unlink_utf8(const char *path) {
}
int hts_rename_utf8(const char *oldpath, const char *newpath) {
LPWSTR woldpath =
hts_convertUTF8StringToUCS2(oldpath, (int) strlen(oldpath), NULL);
LPWSTR wnewpath =
hts_convertUTF8StringToUCS2(newpath, (int) strlen(newpath), NULL);
LPWSTR woldpath = hts_pathToUCS2(oldpath);
LPWSTR wnewpath = hts_pathToUCS2(newpath);
if (woldpath != NULL && wnewpath != NULL) {
const int result = _wrename(woldpath, wnewpath);
@@ -6575,8 +6637,22 @@ int hts_rename_utf8(const char *oldpath, const char *newpath) {
}
}
int hts_rmdir_utf8(const char *path) {
LPWSTR wpath = hts_pathToUCS2(path);
if (wpath != NULL) {
const int result = _wrmdir(wpath);
free(wpath);
return result;
} else {
// Fallback on conversion error.
return _rmdir(path);
}
}
int hts_mkdir_utf8(const char *path) {
LPWSTR wpath = hts_convertUTF8StringToUCS2(path, (int) strlen(path), NULL);
LPWSTR wpath = hts_pathToUCS2(path);
if (wpath != NULL) {
const int result = _wmkdir(wpath);
@@ -6591,7 +6667,7 @@ int hts_mkdir_utf8(const char *path) {
HTSEXT_API int hts_utime_utf8(const char *path, const STRUCT_UTIMBUF * times) {
STRUCT_UTIMBUF mtimes = *times;
LPWSTR wpath = hts_convertUTF8StringToUCS2(path, (int) strlen(path), NULL);
LPWSTR wpath = hts_pathToUCS2(path);
if (wpath != NULL) {
const int result = _wutime(wpath, &mtimes);

View File

@@ -262,7 +262,7 @@ HTS_INLINE void time_rfc822_local(char *s, struct tm *A);
HTS_INLINE int sendc(htsblk * r, const char *s);
int finput(T_SOC fd, char *s, int max);
int binput(char *buff, char *s, int max);
int binput(const char *buff, char *s, int max);
int linput(FILE * fp, char *s, int max);
int linputsoc(T_SOC soc, char *s, int max);
int linputsoc_t(T_SOC soc, char *s, int max, int timeout);
@@ -607,9 +607,26 @@ static HTS_UNUSED size_t llint_to_size_t(LLint o) {
}
}
/* Capacity for @p used bytes plus @p extra more plus @p slack spare;
(size_t) -1 if the total exceeds (size_t) -2 or @p extra is negative
(llint_to_size_t() would map that to a huge valid-looking size). */
static HTS_UNUSED size_t llint_grow_size_t(size_t used, LLint extra,
size_t slack) {
const size_t max = (size_t) -2; /* (size_t) -1 is the error value */
const size_t e = extra >= 0 ? llint_to_size_t(extra) : (size_t) -1;
if (e == (size_t) -1 || used > max || slack > max - used ||
e > max - used - slack) {
return (size_t) -1;
}
return used + e + slack;
}
/* dirent() compatibility */
#ifdef _WIN32
#define HTS_DIRENT_SIZE 256
/* Holds a UTF-8 d_name: MAX_PATH (260) UTF-16 units expand to <=3 bytes each.
Windows-only struct, ABI free to break. */
#define HTS_DIRENT_SIZE 1024
struct dirent {
ino_t d_ino; /* ignored */
off_t d_off; /* ignored */

View File

@@ -47,6 +47,7 @@ Please visit our Website: http://www.httrack.com
#include "htszlib.h"
#endif
#include <ctype.h>
#include <limits.h>
#define ADD_STANDARD_PATH \
{ /* ajout nom */\
@@ -1475,15 +1476,37 @@ int url_savename(lien_adrfilsave *const afs,
sizeof(afs->save) - (size_t) (lastDot - afs->save));
}
}
// enforce 260-character path limit before inserting destination path
// note: 12 characters at least for WIN32, and 12 for ".99.delayed"
// (MSDN) "When using an API to create a directory, the specified path
// cannot be so long that you cannot append an 8.3 file name
// (that is, the directory name cannot exceed MAX_PATH minus 12)."
#define HTS_MAX_PATH_LEN ( 260 - 12 - 12 )
// Cap the save path: the final parent+name is copied into a fixed buffer that
// aborts() on overflow (htssafe.h), so clamp every ceiling to fit it.
#define HTS_SAVE_BUFSIZE (HTS_URLMAXSIZE * 2) /* sizeof(afs->save) */
#define HTS_PATH_TAIL_RESERVE 64 /* collision suffix + ".delayed" + NUL */
#ifdef _WIN32
// MAX_PATH minus 8.3 headroom (MSDN) minus the ".delayed" marker; raising it
// needs the engine to "\\?\"-prefix its paths, which is separate work.
#define HTS_MAX_PATH_LEN (260 - 12 - 12)
#define MAX_SEG_LEN 48
#else
// #133: use the platform's own PATH_MAX/NAME_MAX (Linux/Android 4096, macOS
// 1024) rather than the far smaller Windows MAX_PATH.
#ifdef PATH_MAX
#define HTS_PATH_MAX_ PATH_MAX
#else
#define HTS_PATH_MAX_ 1024
#endif
#ifdef NAME_MAX
#define HTS_NAME_MAX_ NAME_MAX
#else
#define HTS_NAME_MAX_ 255
#endif
#define HTS_MAX_PATH_LEN \
((HTS_PATH_MAX_ - HTS_PATH_TAIL_RESERVE) < \
(HTS_SAVE_BUFSIZE - HTS_PATH_TAIL_RESERVE) \
? (HTS_PATH_MAX_ - HTS_PATH_TAIL_RESERVE) \
: (HTS_SAVE_BUFSIZE - HTS_PATH_TAIL_RESERVE))
#define MAX_SEG_LEN (HTS_NAME_MAX_ > 64 ? HTS_NAME_MAX_ - 16 : HTS_NAME_MAX_)
#endif
#define MIN_LAST_SEG_RESERVE 12
#define MAX_LAST_SEG_RESERVE 24
#define MAX_SEG_LEN 48
if (hts_stringLengthUTF8(afs->save) +
hts_stringLengthUTF8(StringBuff(opt->path_html_utf8)) >=
HTS_MAX_PATH_LEN) {
@@ -1493,7 +1516,7 @@ int url_savename(lien_adrfilsave *const afs,
if (wsave != NULL) {
const size_t parentLen =
hts_stringLengthUTF8(StringBuff(opt->path_html_utf8));
// parent path length is not insane (otherwise, ignore and pick 200 as
// parent path length is not insane (otherwise, ignore and pick 200 as
// suffix length)
const size_t maxLen =
parentLen <
@@ -1584,9 +1607,37 @@ int url_savename(lien_adrfilsave *const afs,
// Re-check again ending space or dot after cut (see bug #5)
cleanEndingSpaceOrDot(afs->save);
}
// The cut above counts UTF-8 codepoints, but parent+name lands in a fixed
// byte buffer that aborts() on overflow (htssafe.h). A multibyte name can
// pass the codepoint cap yet overflow in bytes, so hard-cut on a codepoint
// boundary to keep parent+name inside the buffer regardless (#133).
{
const size_t parentBytes = strlen(StringBuff(opt->path_html_utf8));
const size_t cap = HTS_SAVE_BUFSIZE - HTS_PATH_TAIL_RESERVE;
// Shrink only the name. A parent that alone fills the buffer is left to the
// existing prepend abort, not collapsed to an empty name that would collide
// across URLs and overrun the unbounded collision-suffix sprintf.
if (parentBytes < cap) {
size_t budget = cap - parentBytes;
if (strlen(afs->save) > budget) {
while (budget > 0 && ((unsigned char) afs->save[budget] & 0xC0) == 0x80)
budget--; // back off a continuation byte, never split a char
afs->save[budget] = '\0';
cleanEndingSpaceOrDot(afs->save);
}
}
}
#undef MAX_UTF8_SEQ_CHARS
#undef MIN_LAST_SEG_RESERVE
#undef MAX_LAST_SEG_RESERVE
#undef MAX_SEG_LEN
#undef HTS_MAX_PATH_LEN
#undef HTS_PATH_TAIL_RESERVE
#undef HTS_SAVE_BUFSIZE
#ifndef _WIN32
#undef HTS_PATH_MAX_
#undef HTS_NAME_MAX_
#endif
// chemin primaire éventuel A METTRE AVANT
if (strnotempty(StringBuff(opt->path_html_utf8))) {

View File

@@ -541,6 +541,18 @@ struct httrackp {
and exit without crawling */
String warc_file; /**< WARC output: WARC_AUTONAME for --warc, or the
--warc-file basename (appended at the tail: ABI) */
LLint warc_max_size; /**< --warc-max-size: rotate the archive past this many
bytes (<=0: single file). Tail: ABI */
hts_boolean warc_cdx; /**< --warc-cdx: write a sorted CDXJ index next to the
archive. Tail: ABI */
hts_boolean warc_wacz; /**< --wacz: package archive+index+pages as a WACZ zip
(implies --warc + --warc-cdx). Tail: ABI */
hts_boolean single_file; /**< --single-file: once the mirror is done, rewrite
each saved page with its assets inlined as
data: URIs. Tail: ABI */
LLint single_file_max_size; /**< --single-file-max-size: per-asset cap in
bytes; a bigger asset stays a link.
Tail: ABI */
};
/* Running statistics for a mirror. */
@@ -667,6 +679,11 @@ struct htsblk {
char *warc_reqhdr; /**< stashed raw request header block for WARC (or NULL) */
char *
warc_resphdr; /**< stashed raw response header block for WARC (or NULL) */
int warc_truncated; /**< WARC-Truncated reason for a cap-truncated body
(WARC_TRUNC_*, 0=none). Tail: ABI */
char *warc_rawpath; /**< verbatim WARC: spooled compressed body path, or NULL
(owns the file; unlinked on free). Tail: ABI */
LLint warc_rawsize; /**< byte length of warc_rawpath. Tail: ABI */
/*char digest[32+2]; // md5 digest generated by the engine ("" if none) */
};

View File

@@ -3656,7 +3656,7 @@ int hts_mirror_check_moved(htsmoduleStruct * str,
!ref_existed;
if (fexist_utf8(heap(ptr)->sav)) {
had_partial = 1;
remove(heap(ptr)->sav);
UNLINK(heap(ptr)->sav);
}
// Re-get once, only if a partial existed and both Range triggers are
@@ -3862,18 +3862,13 @@ void hts_mirror_process_user_interaction(htsmoduleStruct * str,
int do_pause = 0;
// user pause lockfile : create hts-paused.lock --> HTTrack will be paused
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-stop.lock"))) {
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-stop.lock"))) {
// remove lockfile
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-stop.lock"));
if (!fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-stop.lock"))) {
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-stop.lock"));
if (!fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-stop.lock"))) {
do_pause = 1;
}
}
@@ -3902,11 +3897,9 @@ void hts_mirror_process_user_interaction(htsmoduleStruct * str,
}
}
{
FILE *fp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-paused.lock"), "wb");
FILE *fp = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-paused.lock"),
"wb");
if (fp) {
fspc(NULL, fp, "info"); // dater
fprintf(fp,
@@ -4222,18 +4215,17 @@ int hts_mirror_wait_for_next_file(htsmoduleStruct * str,
int a = 0;
*stre->last_info_shell_ = tl;
if (fexist(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-autopsy"))) { // débuggage: teste si le robot est vivant
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-autopsy"))) { // débuggage: teste si le
// robot est vivant
// (oui je sais un robot vivant.. mais bon.. il a le droit de vivre lui aussi)
// (libérons les robots esclaves de l'internet!)
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-autopsy"));
fp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-isalive"), "wb");
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-autopsy"));
fp = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-isalive"),
"wb");
a = 1;
}
if ((*stre->info_shell_) || a) {
@@ -4591,7 +4583,7 @@ int hts_wait_delayed(htsmoduleStruct * str, lien_adrfilsave *afs,
/* seen as in error */
in_error = back[b].r.statuscode;
in_error_msg[0] = 0;
strncat(in_error_msg, back[b].r.msg, sizeof(in_error_msg) - 1);
strncatbuff(in_error_msg, back[b].r.msg, sizeof(in_error_msg) - 1);
in_error_size = back[b].r.totalsize;
/* don't break, even with "don't take error pages" switch, because we need to process the slot anyway (and cache the error) */
}

View File

@@ -33,6 +33,7 @@ Please visit our Website: http://www.httrack.com
#ifndef HTSSAFE_DEFH
#define HTSSAFE_DEFH
#include <stdarg.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
@@ -116,6 +117,15 @@ static HTS_UNUSED void abortf_(const char *exp, const char *file, int line) {
#endif
#define HTS_IS_NOT_CHAR_BUFFER(VAR) (!HTS_IS_CHAR_BUFFER(VAR))
/* Source capacity for the buff() family, (size_t)-1 when unknown; sizeof of the
TYPE keeps a decayed operand ("buf + 1") off -Wsizeof-array-decay. */
#if (defined(__GNUC__) && !defined(__cplusplus))
#define HTS_SIZEOF_SRC_(B) \
(HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(__typeof__(B)))
#else
#define HTS_SIZEOF_SRC_(B) (HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B))
#endif
/* Compile-time checks. */
static HTS_UNUSED void htssafe_compile_time_check_(void) {
char array[32];
@@ -205,19 +215,17 @@ static char *strncatbuff_ptr_(char *dest, const char *src, size_t n) {
#if (defined(__GNUC__) && !defined(__cplusplus))
#define strncatbuff(A, B, N) \
__builtin_choose_expr( \
HTS_IS_CHAR_BUFFER(A), \
strncat_safe_(A, sizeof(A), B, \
HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), N, \
"overflow while appending '" #B "' to '" #A "'", __FILE__, \
__LINE__), \
strncatbuff_ptr_((A), (B), (N)))
__builtin_choose_expr(HTS_IS_CHAR_BUFFER(A), \
strncat_safe_(A, sizeof(A), B, HTS_SIZEOF_SRC_(B), N, \
"overflow while appending '" #B \
"' to '" #A "'", \
__FILE__, __LINE__), \
strncatbuff_ptr_((A), (B), (N)))
#else
#define strncatbuff(A, B, N) \
(HTS_IS_NOT_CHAR_BUFFER(A) \
? strncat(A, B, N) \
: strncat_safe_(A, sizeof(A), B, \
HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), N, \
: strncat_safe_(A, sizeof(A), B, HTS_SIZEOF_SRC_(B), N, \
"overflow while appending '" #B "' to '" #A "'", \
__FILE__, __LINE__))
#endif
@@ -232,9 +240,7 @@ static char *strncatbuff_ptr_(char *dest, const char *src, size_t n) {
#define strcatbuff(A, B) \
__builtin_choose_expr( \
HTS_IS_CHAR_BUFFER(A), \
strncat_safe_(A, sizeof(A), B, \
HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), \
(size_t) -1, \
strncat_safe_(A, sizeof(A), B, HTS_SIZEOF_SRC_(B), (size_t) -1, \
"overflow while appending '" #B "' to '" #A "'", __FILE__, \
__LINE__), \
strcatbuff_ptr_((A), (B)))
@@ -242,9 +248,7 @@ static char *strncatbuff_ptr_(char *dest, const char *src, size_t n) {
#define strcatbuff(A, B) \
(HTS_IS_NOT_CHAR_BUFFER(A) \
? strcat(A, B) \
: strncat_safe_(A, sizeof(A), B, \
HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), \
(size_t) -1, \
: strncat_safe_(A, sizeof(A), B, HTS_SIZEOF_SRC_(B), (size_t) -1, \
"overflow while appending '" #B "' to '" #A "'", \
__FILE__, __LINE__))
#endif
@@ -257,19 +261,17 @@ static char *strncatbuff_ptr_(char *dest, const char *src, size_t n) {
#if (defined(__GNUC__) && !defined(__cplusplus))
#define strcpybuff(A, B) \
__builtin_choose_expr( \
HTS_IS_CHAR_BUFFER(A), \
strcpy_safe_(A, sizeof(A), B, \
HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), \
"overflow while copying '" #B "' to '" #A "'", __FILE__, \
__LINE__), \
strcpybuff_ptr_((A), (B)))
__builtin_choose_expr(HTS_IS_CHAR_BUFFER(A), \
strcpy_safe_(A, sizeof(A), B, HTS_SIZEOF_SRC_(B), \
"overflow while copying '" #B "' to '" #A \
"'", \
__FILE__, __LINE__), \
strcpybuff_ptr_((A), (B)))
#else
#define strcpybuff(A, B) \
(HTS_IS_NOT_CHAR_BUFFER(A) \
? strcpy(A, B) \
: strcpy_safe_(A, sizeof(A), B, \
HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), \
: strcpy_safe_(A, sizeof(A), B, HTS_SIZEOF_SRC_(B), \
"overflow while copying '" #B "' to '" #A "'", __FILE__, \
__LINE__))
#endif
@@ -286,24 +288,24 @@ static char *strncatbuff_ptr_(char *dest, const char *src, size_t n) {
* Append characters of "B" to "A", "A" having a maximum capacity of "S".
*/
#define strlcatbuff(A, B, S) \
strncat_safe_(A, S, B, HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), \
(size_t) -1, "overflow while appending '" #B "' to '" #A "'", \
__FILE__, __LINE__)
strncat_safe_(A, S, B, HTS_SIZEOF_SRC_(B), (size_t) -1, \
"overflow while appending '" #B "' to '" #A "'", __FILE__, \
__LINE__)
/**
* Append at most "N" characters of "B" to "A", "A" having a maximum capacity
* of "S".
*/
#define strlncatbuff(A, B, S, N) \
strncat_safe_(A, S, B, HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), \
N, "overflow while appending '" #B "' to '" #A "'", __FILE__, \
strncat_safe_(A, S, B, HTS_SIZEOF_SRC_(B), N, \
"overflow while appending '" #B "' to '" #A "'", __FILE__, \
__LINE__)
/**
* Copy characters of "B" to "A", "A" having a maximum capacity of "S".
*/
#define strlcpybuff(A, B, S) \
strcpy_safe_(A, S, B, HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), \
strcpy_safe_(A, S, B, HTS_SIZEOF_SRC_(B), \
"overflow while copying '" #B "' to '" #A "'", __FILE__, \
__LINE__)
@@ -422,7 +424,9 @@ static HTS_INLINE HTS_UNUSED htsbuff htsbuff_ptr_(char *buf, size_t cap) {
*/
static HTS_INLINE HTS_UNUSED void htsbuff_catn(htsbuff *b, const char *s,
size_t n) {
const size_t add = strnlen(s, n);
/* the (size_t)-1 "no limit" sentinel would reach strnlen as a bound past
PTRDIFF_MAX */
const size_t add = n != (size_t) -1 ? strnlen(s, n) : strlen(s);
/* Overflow-safe: keep the (potentially huge) 'add' alone on one side. The
maintained invariant len < cap makes 'cap - len' >= 1 (no underflow), so
'add < cap - len' cannot wrap the way 'len + add < cap' could. */
@@ -456,6 +460,37 @@ static HTS_INLINE HTS_UNUSED const char *htsbuff_str(const htsbuff *b) {
return b->buf;
}
/**
* Formatted print into dest (capacity size, NUL included), truncating to fit
* and always NUL-terminating. Returns HTS_TRUE if the whole output fit; the
* result is the only truncation signal, so it must be acted on. Unlike
* strcpybuff() it never aborts, so it suits text built from remote input.
*/
static HTS_INLINE HTS_UNUSED HTS_CHECK_RESULT HTS_PRINTF_FUN(3, 4) hts_boolean
slprintfbuff(char *dest, size_t size, const char *fmt, ...) {
va_list args;
int ret;
assertf(dest != NULL && size != 0);
va_start(args, fmt);
ret = vsnprintf(dest, size, fmt, args);
va_end(args);
/* pre-C99 runtimes (msvcrt _vsnprintf) return -1 and do not terminate */
dest[size - 1] = '\0';
return ret >= 0 && (size_t) ret < size ? HTS_TRUE : HTS_FALSE;
}
/**
* slprintfbuff() over the in-scope array ARR (capacity = sizeof(ARR)).
* On GCC/Clang a pointer is a compile error; use slprintfbuff() for those.
*/
#if (defined(__GNUC__) && !defined(__cplusplus))
#define sprintfbuff(ARR, ...) \
slprintfbuff((ARR), sizeof(ARR) + htsbuff_must_be_array_(ARR), __VA_ARGS__)
#else
#define sprintfbuff(ARR, ...) slprintfbuff((ARR), sizeof(ARR), __VA_ARGS__)
#endif
/* Thin aliases over the libc allocator/memcpy (historical "t" suffix); no
added bounds checking. freet() also NULLs the freed pointer and tolerates
NULL. memcpybuff() despite the name is a raw memcpy: the caller owns the

File diff suppressed because it is too large Load Diff

View File

@@ -147,7 +147,8 @@ HTS_UNUSED static int LANG_LIST(const char *path, char *buffer, size_t size);
// 0- Init the URL catcher with standard port
// smallserver_init(&port,&return_host);
T_SOC smallserver_init_std(int *port_prox, char *adr_prox, int defaultPort) {
T_SOC smallserver_init_std(int *port_prox, char *adr_prox, int defaultPort,
const char *bindAddr) {
T_SOC soc;
if (defaultPort <= 0) {
@@ -160,12 +161,12 @@ T_SOC smallserver_init_std(int *port_prox, char *adr_prox, int defaultPort) {
int i = 0;
do {
soc = smallserver_init(&try_to_listen_to[i], adr_prox);
soc = smallserver_init(&try_to_listen_to[i], adr_prox, bindAddr);
*port_prox = try_to_listen_to[i];
i++;
} while((soc == INVALID_SOCKET) && (try_to_listen_to[i] >= 0));
} else {
soc = smallserver_init(&defaultPort, adr_prox);
soc = smallserver_init(&defaultPort, adr_prox, bindAddr);
*port_prox = defaultPort;
}
return soc;
@@ -243,9 +244,10 @@ static int my_gethostname(char *h_loc, size_t size) {
}
// smallserver_init(&port,&return_host);
T_SOC smallserver_init(int *port, char *adr) {
T_SOC smallserver_init(int *port, char *adr, const char *bindAddr) {
T_SOC soc = INVALID_SOCKET;
char h_loc[256 + 2];
SOCaddr server;
commandRunning = commandEnd = commandReturn = commandReturnSet =
commandEndRequested = 0;
@@ -256,25 +258,23 @@ T_SOC smallserver_init(int *port, char *adr) {
free(commandReturnCmdl);
commandReturnCmdl = NULL;
if (my_gethostname(h_loc, 256) == 0) { // host name
SOCaddr server;
SOCaddr_initany(server);
if (bindAddr != NULL && *bindAddr != '\0') {
/* advertise the bound address, else the URL we print is unreachable */
if (strlen(bindAddr) >= sizeof(h_loc) || !gethost(bindAddr, &server)) {
return INVALID_SOCKET;
}
strcpybuff(h_loc, bindAddr);
} else if (my_gethostname(h_loc, 256) != 0) { // host name
return INVALID_SOCKET;
}
SOCaddr_initany(server);
if ((soc =
(T_SOC) socket(SOCaddr_sinfamily(server), SOCK_STREAM,
0)) != INVALID_SOCKET) {
SOCaddr_initport(server, *port);
if (bind(soc, &SOCaddr_sockaddr(server), SOCaddr_size(server)) == 0) {
if (listen(soc, 10) >= 0) {
strcpy(adr, h_loc);
} else {
#ifdef _WIN32
closesocket(soc);
#else
close(soc);
#endif
soc = INVALID_SOCKET;
}
if ((soc = (T_SOC) socket(SOCaddr_sinfamily(server), SOCK_STREAM, 0)) !=
INVALID_SOCKET) {
SOCaddr_initport(server, *port);
if (bind(soc, &SOCaddr_sockaddr(server), SOCaddr_size(server)) == 0) {
if (listen(soc, 10) >= 0) {
strcpy(adr, h_loc);
} else {
#ifdef _WIN32
closesocket(soc);
@@ -283,6 +283,13 @@ T_SOC smallserver_init(int *port, char *adr) {
#endif
soc = INVALID_SOCKET;
}
} else {
#ifdef _WIN32
closesocket(soc);
#else
close(soc);
#endif
soc = INVALID_SOCKET;
}
}
return soc;
@@ -311,6 +318,88 @@ typedef struct {
error_redirect = "/server/error.html"; \
} while(0)
/* Longest "sid" value worth unescaping: the expected one is an md5 hex digest,
so anything near this is already invalid and is rejected unread. */
#define SID_VALUE_MAX 64
/** Does the urlencoded request body present the expected session id?
True only if at least one "sid" field is present and every occurrence
matches, so it holds whichever one a later last-write-wins parse keeps.
Non-destructive: it runs before the body is tokenized in place. */
static hts_boolean body_sid_is_valid(const char *body, const char *expected) {
const char *s = body;
hts_boolean seen = HTS_FALSE;
while (s != NULL && *s != '\0') {
const char *const amp = strchr(s, '&');
const char *const eq = strchr(s, '=');
if (eq != NULL && (amp == NULL || eq < amp) && (size_t) (eq - s) == 3 &&
strncmp(s, "sid", 3) == 0) {
const size_t len = amp != NULL ? (size_t) (amp - eq - 1) : strlen(eq + 1);
hts_boolean match = HTS_FALSE;
if (len < SID_VALUE_MAX) {
char raw[SID_VALUE_MAX];
String value = STRING_EMPTY;
memcpy(raw, eq + 1, len);
raw[len] = '\0';
unescapehttp(raw, &value);
/* StringBuff is NULL until written, so an empty value lands here. */
if (StringBuff(value) != NULL &&
strcmp(StringBuff(value), expected) == 0) {
match = HTS_TRUE;
}
StringFree(value);
}
if (!match) {
return HTS_FALSE;
}
seen = HTS_TRUE;
}
s = amp != NULL ? amp + 1 : NULL;
}
return seen;
}
/* Append c to dst as an HTML entity, or return HTS_FALSE if it needs none. */
static hts_boolean cat_html_escaped(String *dst, char c) {
switch (c) {
case '<':
StringCat(*dst, "&lt;");
break;
case '>':
StringCat(*dst, "&gt;");
break;
case '&':
StringCat(*dst, "&amp;");
break;
case '\'':
StringCat(*dst, "&#39;");
break;
default:
return HTS_FALSE;
}
return HTS_TRUE;
}
/* Append the value of a double-quoted command-line argument: escaped for HTML,
which the browser undoes when it posts the command line back, and for the
argv splitter, which does not. */
static void cat_cmdline_arg(String *output, const char *value) {
const char *a;
for (a = value; *a != '\0'; a++) {
if (*a == '\\' || *a == '\"') {
StringCat(*output, "\\");
}
if (!cat_html_escaped(output, *a)) {
StringMemcat(*output, a, 1);
}
}
}
int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
int timeout = 30;
int retour = 0;
@@ -395,6 +484,7 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
T_SOC soc_c;
LLint length = 0;
const char *error_redirect = NULL;
hts_boolean denied = HTS_FALSE;
line[0] = '\0';
buffer[0] = '\0';
@@ -506,6 +596,22 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
}
/* Authenticate the body before parsing it: every field it carries is
written straight into the global key store below, "command" included,
and that one reaches the engine. Checking afterwards cannot work — the
damage is already done, and the pre-seeded "sid" above would compare
equal to itself for a request that simply omits the field. */
if (meth && buffer[0]) {
intptr_t expected = 0;
if (!coucal_readptr(NewLangList, "_sid", &expected) ||
!body_sid_is_valid(buffer, (const char *) expected)) {
buffer[0] = '\0';
meth = 0;
denied = HTS_TRUE;
}
}
/* check variables */
if (meth && buffer[0]) {
char *s = buffer;
@@ -526,20 +632,6 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
}
/* Error check */
{
intptr_t adr = 0;
intptr_t adr2 = 0;
if (coucal_readptr(NewLangList, "sid", &adr)) {
if (coucal_readptr(NewLangList, "_sid", &adr2)) {
if (strcmp((char *) adr, (char *) adr2) != 0) {
meth = 0;
}
}
}
}
/* Check variables (internal) */
if (meth) {
int doLoad = 0;
@@ -802,7 +894,7 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
/* Response */
if (meth) {
int virtualpath = 0;
hts_boolean virtualpath = HTS_FALSE;
char *pos;
char *url = strchr(line1, ' ');
@@ -830,7 +922,7 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
if (strncmp(file, "/website/", 9) == 0) {
virtualpath = 1;
virtualpath = HTS_TRUE;
}
/* override */
@@ -844,8 +936,11 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
}
/* the override above may have swapped a mirror path for a GUI page */
virtualpath = strncmp(file, "/website/", 9) == 0;
if (strlen(path) + strlen(file) + 32 < sizeof(fsfile)) {
if (strncmp(file, "/website/", 9) != 0) {
if (!virtualpath) {
sprintf(fsfile, "%shtml%s", path, file);
} else {
intptr_t adr = 0;
@@ -903,16 +998,16 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
}
StringMemcat(headers, redir, strlen(redir));
{
char tmp[256];
if (strlen(file) < sizeof(tmp) - 32) {
sprintf(tmp, "Location: %s\r\n", newfile);
StringMemcat(headers, tmp, strlen(tmp));
}
/* client-supplied: a CR/LF here would split the response */
if (newfile[strcspn(newfile, "\r\n")] == '\0') {
StringCat(headers, "Location: ");
StringCat(headers, newfile);
StringCat(headers, "\r\n");
}
coucal_write(NewLangList, "redirect", (intptr_t) NULL);
} else if (is_html(file)) {
} else if (!virtualpath && is_html(file)) {
/* GUI templates only: ${_sid} in a mirrored page would hand the
crawled site the session id that authenticates commands */
int outputmode = 0;
StringMemcat(headers, ok, sizeof(ok) - 1);
@@ -940,6 +1035,7 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
int p;
int format = 0;
int listDefault = 0;
hts_boolean unquoted = HTS_FALSE;
name[0] = '\0';
strlncatbuff(name, str, sizeof(name_), n);
@@ -949,6 +1045,12 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
} else if ((p = strfield(name, "html:"))) {
name += p;
format = 1;
} else if ((p = strfield(name, "unquoted:"))) {
name += p;
unquoted = HTS_TRUE;
} else if ((p = strfield(name, "arg:"))) {
name += p;
format = 5;
} else if ((p = strfield(name, "list:"))) {
name += p;
format = 2;
@@ -1085,8 +1187,8 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
test:<if ==0>:<if ==1>:<if == 2>..
ztest:<if == 0 || !exist>:<if == 1>:<if == 2>..
*/
else if ((p = strfield(name, "test:"))
|| (p = strfield(name, "ztest:"))) {
else if ((p = strfield(name, "test:")) ||
(p = strfield(name, "ztest:"))) {
intptr_t adr = 0;
char *pos2;
int ztest = (name[0] == 'z');
@@ -1189,6 +1291,12 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
}
}
/* consumed here: it shares nothing with the list and
option formats below */
if (format == 5 && langstr != NULL && outputmode != -1) {
cat_cmdline_arg(&output, langstr);
langstr = NULL;
}
if (langstr && outputmode != -1) {
switch (format) {
case 0:
@@ -1206,18 +1314,18 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
StringMemcat(output, &c, 1);
}
a += 2;
} else if (outputmode && a[0] == '<') {
StringCat(output, "&lt;");
} else if (outputmode && a[0] == '>') {
StringCat(output, "&gt;");
} else if (outputmode && a[0] == '&') {
StringCat(output, "&amp;");
} else if (outputmode && a[0] == '\'') {
StringCat(output, "&#39;");
} else if (unquoted && a[0] == '\"') {
/* the browser posts an entity back as a raw
quote, which would open a quoted run in the
argv splitter; a URI cannot hold one anyway */
StringCat(output, "%22");
} else if (outputmode &&
cat_html_escaped(&output, a[0])) {
/* appended as an entity */
} else if (outputmode == 3 && a[0] == ' ') {
StringCat(output, "%20");
} else if (outputmode >= 2
&& ((unsigned char) a[0]) < 32) {
} else if (outputmode >= 2 &&
((unsigned char) a[0]) < 32) {
char tmp[32];
sprintf(tmp, "%%%02x", (unsigned char) a[0]);
@@ -1278,20 +1386,10 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
StringClear(tmpbuff);
break;
case '<':
StringCat(tmpbuff, "&lt;");
break;
case '>':
StringCat(tmpbuff, "&gt;");
break;
case '&':
StringCat(tmpbuff, "&amp;");
break;
case '\'':
StringCat(tmpbuff, "&#39;");
break;
default:
StringMemcat(tmpbuff, fstr, 1);
if (!cat_html_escaped(&tmpbuff, *fstr)) {
StringMemcat(tmpbuff, fstr, 1);
}
break;
}
fstr++;
@@ -1331,7 +1429,9 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
#endif
} else {
if (is_text(file)) {
if (is_html(file)) {
StringMemcat(headers, ok, sizeof(ok) - 1);
} else if (is_text(file)) {
StringMemcat(headers, ok_text, sizeof(ok_text) - 1);
} else if (is_js(file)) {
StringMemcat(headers, ok_js, sizeof(ok_js) - 1);
@@ -1368,6 +1468,11 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
StringCat(output, error);
}
}
} else if (denied) {
StringCat(headers, "HTTP/1.0 403 Forbidden\r\n"
"Server: httrack small server\r\n"
"Content-type: text/html\r\n");
StringCat(output, "Missing or invalid session id.\r\n");
} else {
#ifdef _DEBUG
char error_hdr[] =

View File

@@ -43,8 +43,11 @@ Please visit our Website: http://www.httrack.com
// Fonctions
void socinput(T_SOC soc, char *s, int max);
T_SOC smallserver_init_std(int *port_prox, char *adr_prox, int defaultPort);
T_SOC smallserver_init(int *port, char *adr);
/* Listen on bindAddr, or every interface if NULL/empty; adr (>= 258 bytes) gets
the address to advertise. INVALID_SOCKET on error. */
T_SOC smallserver_init_std(int *port_prox, char *adr_prox, int defaultPort,
const char *bindAddr);
T_SOC smallserver_init(int *port, char *adr, const char *bindAddr);
int smallserver(T_SOC soc, char *url, char *method, char *data, char *path);
#define CATCH_RESPONSE \

1076
src/htssinglefile.c Normal file

File diff suppressed because it is too large Load Diff

72
src/htssinglefile.h Normal file
View File

@@ -0,0 +1,72 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 2026 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* --single-file: post-mirror pass embedding each page's assets as data: URIs.
Internal, not installed. Unlike -%M (one MHT archive for the whole mirror)
this rewrites the saved pages in place and keeps page-to-page links
relative, so the mirror stays browsable and every page also stands alone. */
/* ------------------------------------------------------------ */
#ifndef HTS_SINGLEFILE_DEFH
#define HTS_SINGLEFILE_DEFH
#include "htsopt.h"
#include "htsstrings.h"
#ifdef __cplusplus
extern "C" {
#endif
/* --single-file-max-size default: a bigger asset stays an ordinary link. */
#define SINGLEFILE_DEFAULT_MAX_SIZE (10 * 1024 * 1024)
/* Never base64 more than this, whatever the cap: code64() sizes in int. */
#define SINGLEFILE_HARD_MAX_SIZE (256 * 1024 * 1024)
/* Rewrite every HTML page the mirror produced. No-op unless opt->single_file;
call once the tree is final (after the update purge and the index). */
void singlefile_process_mirror(httrackp *opt);
/* Rewrite one HTML document held in memory, appending the result to out.
root is the mirror directory that references may not escape; page_path is
the document's own path under it (both UTF-8, '/' or native separators).
Returns HTS_TRUE if at least one reference was replaced. */
hts_boolean singlefile_rewrite_html(httrackp *opt, const char *root,
const char *page_path, const char *html,
size_t html_len, String *out);
/* singlefile_rewrite_html() over the file at page_path, rewritten in place.
Returns HTS_TRUE if the file changed. */
hts_boolean singlefile_rewrite_file(httrackp *opt, const char *root,
const char *page_path);
#ifdef __cplusplus
}
#endif
#endif

View File

@@ -1027,6 +1027,17 @@ HTSEXT_API int hts_buildtopindex(httrackp * opt, const char *path,
freet(category);
category = NULL;
}
#ifdef _WIN32
/* category is ANSI-codepage, doc is utf-8: convert (#216) */
else {
char *cat_utf8 = hts_convertStringSystemToUTF8(
category, strlen(category));
if (cat_utf8 != NULL) {
freet(category);
category = cat_utf8;
}
}
#endif
}
}
if (category == NULL) {
@@ -1044,7 +1055,18 @@ HTSEXT_API int hts_buildtopindex(httrackp * opt, const char *path,
oldchain->next = chain;
}
chain->next = NULL;
#ifdef _WIN32
/* name is ANSI-codepage, doc is utf-8: convert (#216) */
{
const char *const name = hts_findgetname(h);
char *name_utf8 =
hts_convertStringSystemToUTF8(name, strlen(name));
strcpybuff(chain->name, name_utf8 != NULL ? name_utf8 : name);
freet(name_utf8);
}
#else
strcpybuff(chain->name, hts_findgetname(h));
#endif
chain->category = category;
chain->level = level;
}
@@ -1296,19 +1318,19 @@ HTSEXT_API hts_boolean hts_findnext(find_handle find) {
if (find) {
#ifdef _WIN32
if ((FindNextFileA(find->handle, &find->hdata)))
return 1;
return HTS_TRUE;
#else
char catbuff[CATBUFF_SIZE];
memset(&(find->filestat), 0, sizeof(find->filestat));
if ((find->dirp = readdir(find->hdir)))
if (find->dirp->d_name)
if (!STAT
(concat(catbuff, sizeof(catbuff), find->path, find->dirp->d_name), &find->filestat))
return 1;
if (!STAT(
concat(catbuff, sizeof(catbuff), find->path, find->dirp->d_name),
&find->filestat))
return HTS_TRUE;
#endif
}
return 0;
return HTS_FALSE;
}
HTSEXT_API int hts_findclose(find_handle find) {

File diff suppressed because it is too large Load Diff

View File

@@ -45,6 +45,16 @@ extern "C" {
under the project's output directory at open time. */
#define WARC_AUTONAME "\001auto"
/* htsblk.warc_truncated / WARC-Truncated reason tokens (ISO 28500 sec 5.13).
A cap-truncated body is still archived, tagged with why it was cut short. */
#define WARC_TRUNC_NONE 0
#define WARC_TRUNC_LENGTH 1 /* hit a size cap (-M mirror / -m per-file) */
#define WARC_TRUNC_TIME 2 /* hit the mirror time cap (-E/--max-time) */
#define WARC_TRUNC_DISCONNECT 3 /* connection dropped mid-body */
/* WARC-Truncated token for a warc_truncated code, or NULL for none. */
const char *warc_truncated_reason(int code);
typedef struct warc_writer warc_writer;
/* Stash the raw request header block (bstr.buffer) on r for the later WARC
@@ -58,6 +68,13 @@ void warc_stash_response(htsblk *r, const char *resphdr);
/* Free both stashed header blocks (idempotent, NULL-safe). */
void warc_free_request(htsblk *r);
/* Adopt the de-chunked compressed spool at tmpfile_path onto
r->warc_rawpath/warc_rawsize (strdupt; frees any prior) so the WARC record
stores the body verbatim. No-op leaving warc_rawpath NULL when tmpfile_path
is empty or the spool is missing/empty (the record then stores the decoded
in-memory/on-disk body instead). */
void warc_adopt_rawspool(htsblk *r, const char *tmpfile_path);
/* Emit the request + response (or revisit) records for one finished
transaction. Lazily opens the writer into opt->state.warc; a no-op (logged
once) if the archive cannot be created. */
@@ -76,6 +93,10 @@ warc_writer *warc_open(httrackp *opt, const char *path);
/* Flush, close and free the writer (NULL-safe). */
void warc_close(warc_writer *w);
/* SURT-canonicalize url into out[outsz] (the CDXJ sort key). Returns 0 on
success, -1 on error or truncation. Exposed for the -#test=warc-surt test. */
int warc_surt(const char *url, char *out, size_t outsz);
/* Write one transaction's request + response (or revisit) records.
target_uri: absolute URL fetched.
ip: numeric peer IP, or NULL/"" to omit.
@@ -84,12 +105,24 @@ void warc_close(warc_writer *w);
body/body_len: decoded in-memory body, or NULL when on disk.
body_path: file re-read for the body when body==NULL (may be NULL).
is_update_unchanged: nonzero for a 304 server-not-modified revisit.
truncated: a WARC_TRUNC_* reason to tag a cap-truncated body, else 0.
The body is stored verbatim: Content-Encoding is kept and Content-Length set
to body_len, so body/body_len must be the as-received (coded) bytes.
Returns 0 on success, -1 on error. */
int warc_write_transaction(warc_writer *w, const char *target_uri,
const char *ip, const char *req_hdr,
const char *resp_hdr, const char *body,
size_t body_len, const char *body_path,
int statuscode, int is_update_unchanged);
int statuscode, int is_update_unchanged,
int truncated);
/* Write one non-HTTP capture as a single WARC 'resource' record: the block is
the raw payload (no HTTP envelope), Content-Type is the payload's own MIME.
Used for ftp:// transfers. truncated is a WARC_TRUNC_* reason or 0.
Returns 0 on success, -1 on error. */
int warc_write_resource(warc_writer *w, const char *target_uri, const char *ip,
const char *content_type, const char *body,
size_t body_len, const char *body_path, int truncated);
#ifdef __cplusplus
}

View File

@@ -62,6 +62,7 @@ Please visit our Website: http://www.httrack.com
#include "htsmd5.c"
#include "md5.c"
#include "htscmdline.h"
#include "htsserver.h"
#include "htsurlport.h"
#include "htsweb.h"
@@ -88,7 +89,7 @@ Please visit our Website: http://www.httrack.com
static htsmutex refreshMutex = HTSMUTEX_INIT;
static int help_server(char *dest_path, int defaultPort);
static int help_server(char *dest_path, int defaultPort, const char *bindAddr);
extern int commandRunning;
extern int commandEnd;
extern int commandReturn;
@@ -153,6 +154,8 @@ int main(int argc, char *argv[]) {
int ret = 0;
int defaultPort = 0;
int parentPid = 0;
/* loopback by default: the handler trusts its input; --bind widens it */
const char *bindAddr = "127.0.0.1";
printf("Initializing the server..\n");
@@ -179,7 +182,8 @@ int main(int argc, char *argv[]) {
if (argc < 2 || (argc % 2) != 0) {
fprintf(stderr, "** Warning: use the webhttrack frontend if available\n");
fprintf(stderr,
"usage: %s [--port <port>] [--ppid parent-pid] <path-to-html-root-dir> [key value [key value]..]\n",
"usage: %s [--port <port>] [--bind <address>] [--ppid parent-pid] "
"<path-to-html-root-dir> [key value [key value]..]\n",
argv[0]);
fprintf(stderr, "example: %s /usr/share/httrack/\n", argv[0]);
return 1;
@@ -267,6 +271,14 @@ int main(int argc, char *argv[]) {
fprintf(stderr, "couldn't set the port number to %s\n", argv[i + 1]);
return -1;
}
} else if (strcmp(argv[i], "--bind") == 0 && i + 1 < argc) {
/* empty would fall back to every interface, silently undoing the default
*/
if (!strnotempty(argv[i + 1])) {
fprintf(stderr, "--bind needs an address\n");
return -1;
}
bindAddr = argv[i + 1];
} else if (strcmp(argv[i], "--ppid") == 0 && i + 1 < argc) {
if (sscanf(argv[i + 1], "%u", &parentPid) != 1) {
fprintf(stderr, "couldn't set the parent PID to %s\n", argv[i + 1]);
@@ -293,7 +305,7 @@ int main(int argc, char *argv[]) {
}
/* launch */
ret = help_server(argv[1], defaultPort);
ret = help_server(argv[1], defaultPort, bindAddr);
htsthread_wait_n(background_threads - 1);
hts_uninit();
@@ -308,10 +320,8 @@ int main(int argc, char *argv[]) {
static int webhttrack_runmain(httrackp * opt, int argc, char **argv);
static void back_launch_cmd(void *pP) {
char *cmd = (char *) pP;
char **argv = (char **) malloct(1024 * sizeof(char *));
char **argv;
int argc = 0;
int i = 0;
int g = 0;
//
httrackp *opt;
@@ -322,28 +332,19 @@ static void back_launch_cmd(void *pP) {
commandReturnCmdl = strdup(cmd);
/* split */
argv[0] = strdup("webhttrack");
argv[1] = cmd;
argc++;
i = 0;
while(cmd[i]) {
if (cmd[i] == '\t' || cmd[i] == '\r' || cmd[i] == '\n') {
cmd[i] = ' ';
}
i++;
}
i = 0;
while(cmd[i]) {
if (cmd[i] == '\"')
g = !g;
if (cmd[i] == ' ') {
if (!g) {
cmd[i] = '\0';
argv[argc++] = cmd + i + 1;
}
}
i++;
argv = hts_split_cmdline(cmd, &argc);
if (argv == NULL) {
if (commandReturnMsg)
free(commandReturnMsg);
commandReturnMsg = strdup("could not parse the command line");
commandReturn = -1;
commandRunning = 0;
commandEnd = 1;
free(cmd);
return;
}
/* drop the program name the posted command line carries */
argv[0] = strdupt("webhttrack");
/* init */
hts_init();
@@ -372,6 +373,7 @@ static void back_launch_cmd(void *pP) {
/* free */
free(cmd);
freet(argv[0]);
freet(argv);
return;
}
@@ -427,11 +429,11 @@ static int webhttrack_runmain(httrackp * opt, int argc, char **argv) {
return ret;
}
static int help_server(char *dest_path, int defaultPort) {
static int help_server(char *dest_path, int defaultPort, const char *bindAddr) {
int returncode = 0;
char adr_prox[HTS_URLMAXSIZE * 2];
int port_prox;
T_SOC soc = smallserver_init_std(&port_prox, adr_prox, defaultPort);
T_SOC soc = smallserver_init_std(&port_prox, adr_prox, defaultPort, bindAddr);
if (soc != INVALID_SOCKET) {
char url[HTS_URLMAXSIZE * 2];
@@ -671,10 +673,10 @@ int __cdecl htsshow_loop(t_hts_callbackarg * carg, httrackp * opt, lien_back * b
char *eps = strchr(back[i].url_adr, '/');
int count;
if (ep != NULL && ep < eps
&& (count = (int) (ep - back[i].url_adr)) < 4) {
if (ep != NULL && eps != NULL && ep < eps &&
(count = (int) (ep - back[i].url_adr)) < 4) {
proto[0] = '\0';
strncat(proto, back[i].url_adr, count);
strncatbuff(proto, back[i].url_adr, count);
}
}
snprintf(StatsBuffer[index].state, sizeof(StatsBuffer[index].state),

View File

@@ -758,6 +758,9 @@ HTSEXT_API int hts_rename_utf8(const char *oldpath, const char *newpath);
HTSEXT_API int hts_mkdir_utf8(const char *pathname);
#define RMDIR hts_rmdir_utf8
HTSEXT_API int hts_rmdir_utf8(const char *pathname);
#define UTIME(A, B) hts_utime_utf8(A, B)
typedef struct _utimbuf STRUCT_UTIMBUF;
@@ -772,6 +775,7 @@ typedef struct stat STRUCT_STAT;
#define UNLINK unlink
#define RENAME rename
#define MKDIR(F) mkdir(F, HTS_ACCESS_FOLDER)
#define RMDIR rmdir
typedef struct utimbuf STRUCT_UTIMBUF;

View File

@@ -46,6 +46,7 @@ Please visit our Website: http://www.httrack.com
#include "httrack.h"
#include "htslib.h"
#include "htscharset.h" // after htslib.h: winsock2.h must precede windows.h
#include "htsbacktrace.h"
/* Static definitions */
static int fexist(const char *s);
@@ -68,11 +69,10 @@ static int linput(FILE * fp, char *s, int max);
#ifdef HAVE_UNISTD_H
#include <unistd.h>
#endif
#include <ctype.h>
#if (defined(__linux) && defined(HAVE_EXECINFO_H))
#include <execinfo.h>
#define USES_BACKTRACE
#ifdef HAVE_SYS_IOCTL_H
#include <sys/ioctl.h>
#endif
#include <ctype.h>
/* END specific definitions */
static void __cdecl htsshow_init(t_hts_callbackarg * carg);
@@ -185,6 +185,43 @@ static void vt_home(void) {
printf("%s%s", VT_RESET, VT_GOTOXY("1", "0"));
}
/* Last known terminal geometry; the defaults are the classic VT100 size. */
static int term_cols = 80;
static int term_rows = 24;
/* Returns HTS_TRUE if the terminal size changed since the last call. */
static hts_boolean vt_size_refresh(void) {
int cols = 0;
int rows = 0;
#ifdef _WIN32
CONSOLE_SCREEN_BUFFER_INFO info;
const HANDLE console = GetStdHandle(STD_OUTPUT_HANDLE);
if (console != INVALID_HANDLE_VALUE &&
GetConsoleScreenBufferInfo(console, &info)) {
cols = info.srWindow.Right - info.srWindow.Left + 1;
rows = info.srWindow.Bottom - info.srWindow.Top + 1;
}
#elif defined(TIOCGWINSZ)
struct winsize ws;
if (ioctl(fileno(stdout), TIOCGWINSZ, &ws) == 0) {
cols = ws.ws_col;
rows = ws.ws_row;
}
#endif
if (cols <= 0)
cols = 80;
if (rows <= 0)
rows = 24;
if (cols == term_cols && rows == term_rows)
return HTS_FALSE;
term_cols = cols;
term_rows = rows;
return HTS_TRUE;
}
//
/*
@@ -195,7 +232,8 @@ static void vt_home(void) {
#define STYLE_STATTEXT VT_UNBOLD
#define STYLE_STATRESET VT_UNBOLD
#define NStatsBuffer 14
#define MAX_LEN_INPROGRESS 40
/* Rows the stats block and "Current job" take above the in-progress list. */
#define NStatsHeaderRows 7
static int use_show;
static httrackp *global_opt = NULL;
@@ -291,6 +329,7 @@ static int __cdecl htsshow_start(t_hts_callbackarg * carg, httrackp * opt) {
use_show = 0;
if (opt->verbosedisplay == HTS_VERBOSE_FULL) {
use_show = 1;
(void) vt_size_refresh();
vt_clear();
}
return 1;
@@ -396,6 +435,10 @@ static int __cdecl htsshow_loop(t_hts_callbackarg * carg, httrackp * opt, lien_b
prev_mytime = mytime;
/* A resize leaves stale wrapped text on screen: repaint everything. */
if (vt_size_refresh())
vt_clear();
st[0] = '\0';
qsec2str(st, stat_time);
vt_home();
@@ -435,6 +478,13 @@ static int __cdecl htsshow_loop(t_hts_callbackarg * carg, httrackp * opt, lien_b
//
t_StatsBuffer StatsBuffer[NStatsBuffer];
/* Keep the frame within the terminal, or it scrolls the stats away. */
const int nstats =
min(NStatsBuffer, max(0, term_rows - NStatsHeaderRows));
/* URL budget: half the width, i.e. the historical 40 at 80 columns. */
const int maxurl =
min((int) sizeof(StatsBuffer[0].name) - 1, max(16, term_cols / 2));
{
int i;
@@ -449,10 +499,11 @@ static int __cdecl htsshow_loop(t_hts_callbackarg * carg, httrackp * opt, lien_b
}
}
for(k = 0; k < 2; k++) { // 0: lien en cours 1: autres liens
for(j = 0; (j < 3) && (index < NStatsBuffer); j++) { // passe de priorité
for (j = 0; (j < 3) && (index < nstats); j++) { // passe de priorité
int _i;
for(_i = 0 + k; (_i < max(back_max * k, 1)) && (index < NStatsBuffer); _i++) { // no lien
for (_i = 0 + k; (_i < max(back_max * k, 1)) && (index < nstats);
_i++) { // no lien
int i = (back_index + _i) % back_max; // commencer par le "premier" (l'actuel)
if (back[i].status >= 0) { // signifie "lien actif"
@@ -527,16 +578,14 @@ static int __cdecl htsshow_loop(t_hts_callbackarg * carg, httrackp * opt, lien_b
}
}
if ((l = (int) strlen(s)) < MAX_LEN_INPROGRESS)
if ((l = (int) strlen(s)) < maxurl)
strcpybuff(StatsBuffer[index].name, s);
else {
// couper
StatsBuffer[index].name[0] = '\0';
strncatbuff(StatsBuffer[index].name, s,
MAX_LEN_INPROGRESS / 2 - 2);
strncatbuff(StatsBuffer[index].name, s, maxurl / 2 - 2);
strcatbuff(StatsBuffer[index].name, "...");
strcatbuff(StatsBuffer[index].name,
s + l - MAX_LEN_INPROGRESS / 2 + 2);
strcatbuff(StatsBuffer[index].name, s + l - maxurl / 2 + 2);
}
if (back[i].r.totalsize >= 0) { // taille prédéfinie
@@ -597,7 +646,7 @@ static int __cdecl htsshow_loop(t_hts_callbackarg * carg, httrackp * opt, lien_b
{
int i;
for(i = 0; i < NStatsBuffer; i++) {
for (i = 0; i < nstats; i++) {
if (strnotempty(StatsBuffer[i].state)) {
printf(VT_CLREOL " %s - \t%s%s \t%s / \t%s", StatsBuffer[i].state,
StatsBuffer[i].name, StatsBuffer[i].file, int2bytes(&strc,
@@ -828,21 +877,6 @@ static void sig_doback(int blind) { // mettre en backing
#undef FD_ERR
#define FD_ERR 2
static void print_backtrace(void) {
#ifdef USES_BACKTRACE
void *stack[256];
const int size = backtrace(stack, sizeof(stack)/sizeof(stack[0]));
if (size != 0) {
backtrace_symbols_fd(stack, size, FD_ERR);
}
#else
const char msg[] = "No stack trace available on this OS :(\n";
if (write(FD_ERR, msg, sizeof(msg) - 1) != sizeof(msg) - 1) {
/* sorry GCC */
}
#endif
}
static size_t print_num(char *buffer, int num) {
size_t i, j;
if (num < 0) {
@@ -876,7 +910,7 @@ static void sig_fatal(int code) {
size += print_num(&buffer[size], code);
buffer[size++] = '\n';
(void) (write(FD_ERR, buffer, size) == size);
print_backtrace();
hts_print_backtrace(FD_ERR);
(void) (write(FD_ERR, msgreport, sizeof(msgreport) - 1)
== sizeof(msgreport) - 1);
abort();
@@ -899,6 +933,7 @@ static void sig_leave(int code) {
}
static void signal_handlers(void) {
hts_backtrace_init();
#ifdef _WIN32
signal(SIGINT, sig_leave); // ^C
signal(SIGTERM, sig_finish); // kill <process>

View File

@@ -100,6 +100,7 @@
<ItemGroup>
<ClCompile Include="httrack.c" />
<ClCompile Include="htsbacktrace.c" />
</ItemGroup>
<!-- Pulls in libhttrack.lib and builds it first. -->

View File

@@ -59,9 +59,10 @@
<ItemDefinitionGroup>
<ClCompile>
<!-- LIBHTTRACK_EXPORTS turns HTSEXT_API into __declspec(dllexport); ZLIB_DLL
imports zlib from its DLL. HTS_USEBROTLI/HTS_USEZSTD enable the br and
zstd content codings. Windows 7 floor, matching WinHTTrack. -->
<PreprocessorDefinitions>WIN32;_WINDOWS;_MBCS;_USRDLL;LIBHTTRACK_EXPORTS;ZLIB_DLL;HTS_USEBROTLI=1;HTS_USEZSTD=1;WINVER=0x0601;_WIN32_WINNT=0x0601;_CRT_SECURE_NO_WARNINGS;_CRT_NONSTDC_NO_DEPRECATE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
imports zlib from its DLL, ZLIB_CONST makes its next_in const as the
autotools build does. HTS_USEBROTLI/HTS_USEZSTD enable the br and zstd
content codings. Windows 7 floor, matching WinHTTrack. -->
<PreprocessorDefinitions>WIN32;_WINDOWS;_MBCS;_USRDLL;LIBHTTRACK_EXPORTS;ZLIB_DLL;ZLIB_CONST;HTS_USEBROTLI=1;HTS_USEZSTD=1;WINVER=0x0601;_WIN32_WINNT=0x0601;_CRT_SECURE_NO_WARNINGS;_CRT_NONSTDC_NO_DEPRECATE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(MSBuildThisFileDirectory);$(MSBuildThisFileDirectory)coucal;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<WarningLevel>Level3</WarningLevel>
<MultiProcessorCompilation>true</MultiProcessorCompilation>
@@ -121,6 +122,7 @@
<ClCompile Include="htshelp.c" />
<ClCompile Include="htsindex.c" />
<ClCompile Include="htslib.c" />
<ClCompile Include="htscmdline.c" />
<ClCompile Include="htsurlport.c" />
<ClCompile Include="htsmd5.c" />
<ClCompile Include="htsmodules.c" />
@@ -129,6 +131,7 @@
<ClCompile Include="htsproxy.c" />
<ClCompile Include="htsrobots.c" />
<ClCompile Include="htsselftest.c" />
<ClCompile Include="htssinglefile.c" />
<ClCompile Include="htssniff.c" />
<ClCompile Include="htsthread.c" />
<ClCompile Include="htstools.c" />

View File

@@ -771,7 +771,7 @@ static PT_Element proxytrack_process_DAV_Request(PT_Indexes indexes,
PT_ReadIndex(indexes, StringBuff(itemUrl) + 1, FETCH_HEADERS);
if (file != NULL && file->statuscode == HTTP_OK) {
size = file->size;
if (file->lastmodified) {
if (file->lastmodified[0] != '\0') {
timestamp = get_time_rfc822(file->lastmodified);
}
if (timestamp == (time_t) 0) {
@@ -785,7 +785,7 @@ static PT_Element proxytrack_process_DAV_Request(PT_Indexes indexes,
}
timestamp = timestampRep;
}
if (file->contenttype) {
if (file->contenttype[0] != '\0') {
mimeType = file->contenttype;
}
}

View File

@@ -579,9 +579,10 @@ PT_Index PT_LoadCache(const char *filename) {
if (chain != NULL) {
const char *scheme = link_has_authority(chain->name) ? "" : "http://";
snprintf(index->slots.common.startUrl,
sizeof(index->slots.common.startUrl), "%s%s", scheme,
(const char *) chain->name);
/* dropped rather than truncated: empty already reads as "unset" */
if (!sprintfbuff(index->slots.common.startUrl, "%s%s", scheme,
(const char *) chain->name))
index->slots.common.startUrl[0] = '\0';
}
}
}
@@ -972,9 +973,12 @@ int PT_LoadCache__New(PT_Index index_, const char *filename) {
const char *scheme =
link_has_authority(filenameIndex) ? "" : "http://";
firstSeen = 1;
snprintf(index->startUrl, sizeof(index->startUrl), "%s%s",
scheme, filenameIndex);
/* dropped rather than truncated; try the next entry */
if (sprintfbuff(index->startUrl, "%s%s", scheme,
filenameIndex))
firstSeen = 1;
else
index->startUrl[0] = '\0';
}
}
} else {
@@ -1570,9 +1574,11 @@ static int PT_LoadCache__Old(PT_Index index_, const char *filename) {
const char *scheme =
link_has_authority(line) ? "" : "http://";
firstSeen = 1;
snprintf(index->startUrl, sizeof(index->startUrl), "%s%s",
scheme, line);
/* dropped rather than truncated; try the next entry */
if (sprintfbuff(index->startUrl, "%s%s", scheme, line))
firstSeen = 1;
else
index->startUrl[0] = '\0';
}
}
@@ -2191,7 +2197,7 @@ static PT_Element PT_ReadCache__Arc_u(PT_Index index_, const char *url,
}
if ((pos = getArcField(index->line, 2)) != NULL) {
r->msg[0] = '\0';
strncat(r->msg, pos, sizeof(pos) - 1);
strncatbuff(r->msg, pos, sizeof(r->msg) - 1);
}
while(linput(index->file, index->line, sizeof(index->line) - 1)
&& index->line[0] != '\0') {

View File

@@ -17,8 +17,8 @@
#endif
VS_VERSION_INFO VERSIONINFO
FILEVERSION 3, 49, 13, 0
PRODUCTVERSION 3, 49, 13, 0
FILEVERSION 3, 49, 14, 0
PRODUCTVERSION 3, 49, 14, 0
FILEFLAGSMASK VS_FFI_FILEFLAGSMASK
#ifdef _DEBUG
FILEFLAGS VS_FF_DEBUG
@@ -35,12 +35,12 @@ BEGIN
BEGIN
VALUE "CompanyName", "Xavier Roche"
VALUE "FileDescription", VER_FILE_DESCRIPTION
VALUE "FileVersion", "3.49.13"
VALUE "FileVersion", "3.49.14"
VALUE "InternalName", VER_ORIGINAL_FILENAME
VALUE "LegalCopyright", "Copyright (C) 1998-2026 Xavier Roche and other contributors. GNU GPL v3 or later."
VALUE "OriginalFilename", VER_ORIGINAL_FILENAME
VALUE "ProductName", "HTTrack Website Copier"
VALUE "ProductVersion", "3.49-13"
VALUE "ProductVersion", "3.49-14"
END
END
BLOCK "VarFileInfo"

View File

@@ -97,6 +97,7 @@
<ItemGroup>
<ClCompile Include="htsserver.c" />
<ClCompile Include="htsweb.c" />
<ClCompile Include="htscmdline.c" />
<ClCompile Include="htsurlport.c" />
</ItemGroup>

View File

@@ -0,0 +1,8 @@
#!/bin/bash
#
set -euo pipefail
# webhttrack posts its command line as one string: the argv split must grow past
# 1024 arguments and keep a quote inside a value out of the option parser.
httrack -O /dev/null -#test=cmdline-split run | grep -q "cmdline-split self-test OK"

View File

@@ -0,0 +1,12 @@
#!/bin/bash
#
set -euo pipefail
# Drives -#test=cookieimport: load a cookie jar (and, on Windows, copied IE
# cookies *@*.txt) from a long, non-ASCII folder through the UTF-8/long-path
# file wrappers (#133,#630).
dir=$(mktemp -d)
trap 'rm -rf "$dir"' EXIT
httrack -O /dev/null -#test=cookieimport "$dir" | grep -q "cookieimport:.*OK"

11
tests/01_engine-direnum.test Executable file
View File

@@ -0,0 +1,11 @@
#!/bin/bash
#
set -euo pipefail
# Drives -#test=direnum: enumerate a long+non-ASCII directory through the
# opendir/readdir wrappers, checking each child round-trips as UTF-8 (#133,#630).
dir=$(mktemp -d)
trap 'rm -rf "$dir"' EXIT
httrack -O /dev/null -#test=direnum "$dir" | grep -q "direnum:.*OK"

View File

@@ -8,9 +8,14 @@ set -euo pipefail
dir=$(mktemp -d)
trap 'rm -rf "$dir"' EXIT
out=$(httrack -#test=fsize "$dir")
rc=0
out=$(httrack -#test=fsize "$dir") || rc=$?
echo "$out"
# 77 = platform can't host the 5GB probe (EFBIG); skip, don't fail.
[ "$rc" = 77 ] && exit 77
[ "$rc" = 0 ] || exit "$rc"
want="fsize: width=8,8 size=5368709120,5368709120 psize=5368709120 absent=-1"
test "$out" == "$want" || {
echo "FAIL: unexpected size report (want '$want')"

View File

@@ -0,0 +1,25 @@
#!/bin/bash
#
# Buffer capacity for a 64-bit file size ('httrack -#test=growsize'): a -%S list
# file past 4GB must size its buffer exactly or be refused, never wrap.
set -euo pipefail
out=$(httrack -#test=growsize)
echo "$out"
test "$out" == "growsize self-test OK"
tmp=$(mktemp -d "${TMPDIR:-/tmp}/httrack_growsize.XXXXXX")
trap 'rm -rf "$tmp"' EXIT HUP INT QUIT PIPE TERM
echo '<html><body>hi</body></html>' >"$tmp/index.html"
printf -- '-*/zzmarker*\n' >"$tmp/rules.txt"
# the rules file lands in the URL/filter string, echoed back by the banner
run=$(httrack -O "$tmp/out" --quiet -n "-%S" "$tmp/rules.txt" \
"file://$tmp/index.html" 2>&1) || true
printf '%s\n' "$run" | grep -q 'zzmarker' || {
echo "FAIL: -%S rules file was not loaded"
printf '%s\n' "$run"
exit 1
}

View File

@@ -0,0 +1,11 @@
#!/bin/bash
#
set -euo pipefail
# Drives -#test=longpath: a >MAX_PATH round trip exercising hts_pathToUCS2's
# \\?\ prefixing on Windows (#133); a positive control on POSIX.
dir=$(mktemp -d)
trap 'rm -rf "$dir"' EXIT
httrack -O /dev/null -#test=longpath "$dir" | grep -q "longpath:.*OK"

12
tests/01_engine-mirror-io.test Executable file
View File

@@ -0,0 +1,12 @@
#!/bin/bash
#
set -euo pipefail
# Drives -#test=mirrorio: rounds a file through a path that is both long
# (>MAX_PATH) and non-ASCII, exercising the mirror I/O wrappers the engine's
# raw file ops now route to on Windows (#133, #630). Positive control on POSIX.
dir=$(mktemp -d)
trap 'rm -rf "$dir"' EXIT
httrack -O /dev/null -#test=mirrorio "$dir" | grep -q "mirrorio:.*OK"

View File

@@ -149,10 +149,19 @@ full '/page.php?id=3&sid=42' 'text/html' '/dev/null/www.example.com/page475b.htm
strip=sid 'prior=www.example.com|/page.php?id=4|/dev/null/www.example.com/PAGE-PRIOR.html'
# Hostile fils stay rooted under the mirror: ../ (raw or %2e-encoded) drops out,
# control characters become spaces, oversized names cap at 210 chars (the cap
# can chop the extension off entirely).
# control characters become spaces.
full '/../../etc/passwd' 'text/html' '/dev/null/www.example.com///etc/passwd.html'
full '/%2e%2e/%2e%2e/etc/passwd' 'text/html' '/dev/null/www.example.com///etc/passwd.html'
full '/x.php' 'application/pdf' '/dev/null/www.example.com///evil.exe' 'cdispo=../../evil.exe'
name $'/evil\rname\t.php' 'text/html' 'evil name .html'
name "/$(printf 'a%.0s' {1..300}).php" 'text/html' "$(printf 'a%.0s' {1..210})"
# #133: oversized names are capped only on Windows (MAX_PATH, chopping the
# extension); elsewhere the platform PATH_MAX leaves a 300-char name whole.
case "$(uname -s 2>/dev/null)" in
MINGW* | MSYS* | CYGWIN*)
name "/$(printf 'a%.0s' {1..300}).php" 'text/html' "$(printf 'a%.0s' {1..210})"
;;
*)
name "/$(printf 'a%.0s' {1..300}).php" 'text/html' "$(printf 'a%.0s' {1..300}).html"
;;
esac

View File

@@ -47,3 +47,18 @@ case "$err" in
exit 1
;;
esac
# An array source with no NUL must abort on the source bound rather than run
# off the array: the array half of the source-capacity selection.
err=$(httrack -#test=strsafe overflow-src "x" 2>&1) || true
case "$err" in
*"strsafe: NOT aborted"*)
echo "unterminated source array was NOT caught" >&2
exit 1
;;
*"size < sizeof_source"*) ;;
*)
echo "expected htssafe source-bound abort, got: $err" >&2
exit 1
;;
esac

View File

@@ -0,0 +1,21 @@
#!/bin/bash
#
# UTF-8 <-> system codepage ('httrack -#test=syscharset'), the pair an MBCS GUI
# uses at the Win32 ANSI entry points. WIN32 only, skipped elsewhere.
set -euo pipefail
rc=0
out=$(httrack -O /dev/null -#test=syscharset) || rc=$?
echo "$out"
[ "$rc" = 77 ] && exit 77
[ "$rc" = 0 ] || exit "$rc"
case "$out" in
"syscharset: acp="*": OK") ;;
*)
echo "FAIL: unexpected report"
exit 1
;;
esac

8
tests/01_engine-warc-surt.test Executable file
View File

@@ -0,0 +1,8 @@
#!/bin/bash
#
set -euo pipefail
# SURT canonicalization of the CDXJ sort key (--warc-cdx). Pure string work,
# so it runs under the MSan-instrumented 01_engine glob.
httrack -O /dev/null -#test=warc-surt | grep -q "warc-surt: OK"

19
tests/01_zlib-warc-cdx.test Executable file
View File

@@ -0,0 +1,19 @@
#!/bin/bash
#
set -euo pipefail
# --warc-cdx CDXJ index over synthetic transactions: sorted, one line per
# response/revisit/resource, each offset/length inflates to the right member.
httrack_bin=$(cd "$(dirname "$(command -v httrack)")" && pwd)/httrack
scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
out=$("$httrack_bin" -O /dev/null -#test=warc-cdx "$scratch/")
echo "$out"
case "$out" in
*": OK") ;;
*) exit 1 ;;
esac

Some files were not shown because too many files have changed in this diff Show More