Compare commits

..

61 Commits

Author SHA1 Message Date
Xavier Roche
250d811781 single-file: translate the new GUI strings into the remaining 28 locales
The feature PR added the four LANG_SINGLEFILE* entries to lang.def with English
and Francais only; every other locale fell back to English in the WebHTTrack
form. Each file is written in its own declared charset.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-26 21:33:18 +02:00
Xavier Roche
42072f54b0 tests: check the sprintfbuff master just made warn_unused_result
#722 gave slprintfbuff the attribute, so the over-wide-tag fixture's call
became the one warning this branch adds over master's baseline. Handle it the
way that PR's own self-test code does; the buffer cannot truncate here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
2026-07-26 18:59:38 +02:00
Xavier Roche
5f3787eb28 Merge origin/master into feat/single-file
Master took test number 86 for the proxytrack cache-longfields test; both
that and 87_local-single-file.test now sit in TESTS. The only conflict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
2026-07-26 18:56:04 +02:00
Xavier Roche
ca533abefd Unbounded writes into the 80-byte htsblk failure message (#722)
* Bound the remaining writes into htsblk.msg[80]

Nine writers still filled the 80-byte msg with no bound. Six sprintf the
result of strerror(), whose longest glibc string is 49 bytes in the C
locale and 72 in fr_FR against 46 bytes of room after the longest prefix.
Realistic connect() errnos still fit, so this was latent rather than live.

The FTP helper's .ok result file had no excuse: it was copied byte by byte
until EOF into the same 80 bytes. Split that parse out as
back_read_ftp_result() so a self-test can drive it, and stop at capacity.

These sites never appeared in the -Wformat-truncation cluster that added
htsblk_failf: the diagnostic only fires on a bounded snprintf whose return
is discarded, so a raw sprintf is invisible to it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Close the self-test's blind spots around msg[]

Six of eight mutants survived the first version. The neighbour canary
compared against zero, so it saw a stray 'X' but not the stray NUL an
off-by-one terminator actually writes; poison it instead. Only the
over-capacity case existed, so padding every message to 79 bytes or eating
its last character both passed, as did a sign-extended 0xff reading as EOF
and the new unparseable-status branch, which had no coverage at all.

Same zero-comparison weakness applied to the htsblk_failf canary inherited
from #715, so that one is poisoned too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 18:51:41 +02:00
Xavier Roche
dc6819b7b7 Cached headers longer than ProxyTrack's fields overflow into the location pointer (#717)
* Bound ProxyTrack's cache header copies

ZIP_READFIELD_STRING in proxy/store.c took no destination size and used a
raw strcpy, while the engine's namesake in htscache.c has taken a
refvalue_size for years. The source is a cache-entry header line bounded
only by line[HTS_URLMAXSIZE + 2].

contenttype[64] sits just before the location pointer in the calloc'd
_PT_Element, so an over-long Content-Type walks over charset and then
over location, which the next Location: line copies through.

Clip rather than reject: the engine stores Content-Type in 128 bytes, so
a valid cache legitimately carries fields wider than ours.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Bound the ARC reader's header copies too

Review found store.c carries a second copy of the same macro:
HTTP_READFIELD_STRING feeds the ARC reader from index->line[2048], twice
the ZIP path's reach, into the same contenttype[64] and its neighbouring
location pointer. proxytrack --convert on a plain-text ARC segfaults.

Test 86 covered one of eight fields, so a per-field clip and a
one-size-fits-all clip were indistinguishable. It now overshoots every
destination and asserts each surviving length against its own capacity,
and exercises the ARC reader without needing python.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 18:51:13 +02:00
Xavier Roche
4a62cfb75b tests: stand the single-file GUI check down where htsserver is absent
The Windows job builds httrack.exe only, and reaches this file through its
*_local-*.test glob, so requiring htsserver failed the whole test there even
though every crawl assertion had passed. Skip that half instead, and run the
engine self-test before it so the skip cannot swallow it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
2026-07-26 18:34:38 +02:00
Xavier Roche
906906efa7 tests: cover the option plumbing review found untested
copy_htsopt's two new fields, including the >0 guard that must not let an
unset source clear the target's default; -%Z0; and a rejected
--single-file-max-size argument falling back to the built-in cap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
2026-07-26 18:17:54 +02:00
Xavier Roche
f50307d878 singlefile: tighten two comments
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
2026-07-26 18:14:53 +02:00
Xavier Roche
c6b7c407ed Fix the parser and path bugs review found, and cover them
sf_relative_from() indexed one byte past from_dir's terminator whenever the
page's own directory was a prefix of the asset path, which is the ordinary
mirror layout: an out-of-bounds read that also miscounted the ../ prefix and
emitted a broken link. Two review agents hit the same ASan trace; the branch
had no test because every nested asset in the fixture was under the cap.

Also from review: an escaped quote no longer ends a CSS string early (which
exposed its contents to the url()/@import scanner), @import url(...) now
inlines like the quoted form, a raw-text element ends only on a real end tag
rather than any prefix of one, an over-wide tag is copied through by the same
quote-aware scan instead of a second quote-blind one, <!--> is an empty
comment, whitespace before a tag's > survives, and a page cannot inline more
than SINGLEFILE_MAX_PAGE_SIZE, which bounds the multiplicative @import
fan-out. A failed encode no longer leaves a payload-less data: prefix behind,
and base_dir is matched against the root on a component boundary.

sf_readfile now wraps a new readfile2_utf8() rather than being a fifth copy
of the readfile family. Docs place --single-file beside -%M instead of
implying it supersedes it: MHT stays the better container, this wins on
opening anywhere.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
2026-07-26 18:10:48 +02:00
Xavier Roche
3247e5b183 Diagnostic messages built from remote replies discard snprintf's truncation signal (#715)
* Fold htsblk/PT_Element failure messages into a clipping helper

The 18 remaining -Wformat-truncation warnings all came from building a
diagnostic string out of a remote server's reply and dropping snprintf's
return. Add htsblk_failf() for htsblk.msg[80] and PT_Element_failf() for
ProxyTrack's msg[1024]: both clip to fit and absorb the discard once, so
the obligation is not silently laundered at each call site.

msg[80] lives in the installed htsopt.h, so growing it would break the
ABI; a clipped FTP banner is the intended outcome anyway. The display
StatsBuffer.state is not installed, so it grows to fit back->info instead.

Also bounds two unbounded sprintf(r->msg, ...) in ProxyTrack's cache
reader and moves the raw strcpy neighbours to strcpybuff.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Catch a one-past-the-end write into msg[]'s neighbour

Review found the new self-test blind to a store at msg[sizeof(msg)]: it
lands in the adjacent contenttype field, which no assertion read, and an
intra-struct overflow is invisible to ASan and _FORTIFY_SOURCE. Check the
neighbour after every call, and add the exact-fit case the block lacked.

Correct the contract comment too: msg is not purely diagnostic, it
round-trips through the cache as X-StatusMessage.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 18:00:21 +02:00
Xavier Roche
a1226401e0 tests: renumber the single-file test to 87, run the self-test last
Master took 82 through 85 while this branch was open. The engine self-test
also moves to the end of the script: it and the crawl assertions cover
different ground, and failing first hid the crawl half.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
2026-07-26 17:52:28 +02:00
Xavier Roche
71ca31e2ec Merge origin/master into feat/single-file
Master's #709 took test number 82, so the single-file crawl test moves to 86.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-26 17:48:43 +02:00
Xavier Roche
b90d699d2e Single-file output is MHT, which browsers no longer open
Add --single-file (-%Z): after the mirror completes, rewrite every saved
page in place with its stylesheets, scripts, images and fonts embedded as
data: URIs, while links between pages stay relative. The mirror remains a
browsable tree and each page also stands alone.

This cannot reuse the -%M path: MHT streams a MIME part per file as it is
saved, but a data: URI needs the asset's bytes when the page is written,
and pages are normally saved before their assets are fetched. The new pass
runs over the finished tree at the tail of httpmirror(), after the update
purge.

Audio, video, page-to-page links and anything over --single-file-max-size
(10 MB default) keep an ordinary link. References carrying a scheme are
skipped, which covers data: and makes a second --update run a no-op.
Resolution is clamped to the mirror root: the HTML is hostile input.

htsopt.h gains two tail-appended fields; VERSION_INFO is untouched.

Closes #713

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
2026-07-26 17:45:48 +02:00
Xavier Roche
bef7120423 tests: test 82 fails on master after #707 and #709 landed together (#716)
* tests: arm test 82's mirror root through a profile save

#707 made /website/ serve only the root htsserver recorded at structcheck
success, so the posted projpath test 82 relied on no longer names anything.
Each PR was green alone; the pair was not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: drop the now-unused project-path argument

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 17:44:55 +02:00
Xavier Roche
7cf54af485 htsserver: a posted projpath repoints the served root and overflows the composed path (#707)
* htsserver builds the redirect Location header in a 256-byte stack buffer

The POST redirect path checks strlen(file) but sprintf's newfile, which comes
straight from the client's "redirect" POST field with no length cap. A 300-byte
value overflows tmp[256]. The same value reached the Location header with no
CR/LF check, so it could split the response and inject headers.

Append into the dynamic String the other headers already use, and drop the
header entirely when the value carries a CR or LF.

The listen socket was SOCaddr_initany, so the server answered the LAN and not
just the local browser it exists to serve. Bind 127.0.0.1 by default, with
--bind <addr> to widen it again, resolved through the existing gethost() helper
the way proxytrack already does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: satisfy shellcheck and shfmt in the new server test

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: drop the pre-fix narration from the oversized-value comment

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: assert the bound socket, not the announced URL

The listen-address assertions only compared the URL= banner, which is a
literal echo of argv: a build that announced 127.0.0.1 while binding the
wildcard passed. Probe 127.0.0.2 on the same port instead, which a wildcard
listener takes and a loopback-only one leaves free.

Also refuse an empty --bind, which fell through to every interface and
silently undid the new default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: bound the response read and the empty --bind run

The recv() loop had no timeout and read until EOF; htsserver need not close
the connection after responding, which wedged the macOS runner for over an
hour. Stop at the end of the header block, which is all the test reads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* htsserver: the session id must gate the request body, not the reply

Every field of a POST body is written straight into the one global key store
the templates and the command dispatcher both read, and "command" from there
reaches the engine. The gate ran after that write and compared "sid" against
"_sid" -- but "_sid" is copied into "sid" beforehand so the templates can
render it, so a request that simply omitted the field compared equal to
itself. Only a wrong id was refused; an absent one passed. Clearing the reply
afterwards does not help either, because the dispatcher sits outside the reply
guard.

Authenticate before parsing instead: scan the raw body for "sid", require at
least one occurrence and reject if any of them differs, and drop the body
untouched when it does not match. That leaves the shared template key alone,
and it closes the dispatcher for free.

A refused request also emitted only a Content-length line, since the status
line for that branch was behind _DEBUG. Any client reads that as a protocol
error, which is how test 68 failed rather than reporting the refusal. Send a
403 instead.

Tests 68 and 77 posted without an id, which is what the engine used to accept,
so both now fetch the one the server renders into the form. Test 78 covers
accept, missing, empty and wrong, asserts the 403, and probes the key store
through ${projname} rather than the suppressed reply -- a reply-only assertion
passes even when the write goes through.

Also fix a leak that hung macOS CI: start() runs inside a command
substitution, so its $! never reached the parent and stop() guarded on an
empty variable, leaving one htsserver per call. Test 77 starts four, which is
exactly the four orphans the runner reported while sitting for half an hour
behind a green test log. Take the pid from the PID= line the server already
announces.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* htsserver: serve the crawled mirror verbatim, never through the expander

WebHTTrack serves the mirror under /website/ from the same small server as its
own GUI, and the decision to run a response through the ${...} template
expander was a substring test for ".htm" on the request path. A mirrored page
therefore had its directives evaluated: ${_sid} rendered the live session id,
handing the crawled site the token that authenticates commands on the local
GUI, and ${do:...} gave it the rest of the template verbs.

The /website/ prefix was already detected, but only to keep the crawl-state
override from hijacking a mirror request. Reuse it as the expansion gate, so
expansion is limited to files under the GUI's html root, and re-evaluate it
after that override, which can substitute a GUI page for a mirror path.
Mirrored pages keep their text/html type: verbatim must not turn browsing the
mirror into a download.

Note that mirrored content still shares the control origin, so a script in it
can read the session id from a GUI page itself; that is a separate fix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* htsserver: stop letting a posted projpath name the /website/ root

The static-file path in smallserver() was composed with a length guard that
measured only the server root and the request path, then wrote the posted
"projpath" field into a 1024-byte stack buffer: a 2000-byte projpath followed
by any /website/ request smashed the stack (buffer overflow detected, server
gone). The guard also summed two untrusted lengths before comparing, the shape
that can wrap and pass.

Composition now goes through a bounded, non-aborting append that keeps the
untrusted length alone on one side, and /website/ is served from the project
directory the server itself set up, rejecting a ".." in it, rather than from
whatever root the request body claimed. Without that, projpath=/etc/ plus
GET /website/passwd read an arbitrary file, since the ".." check looked at the
request path only. fsfile is also cleared before the error-redirect branch,
which could otherwise reach fopen() uninitialized.

tests/80_webhttrack-projpath.test drives the three cases against a live server
and keeps a legitimate project browsable as the control.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: establish test 79's mirror root through the server

/website/ no longer serves the posted projpath, so injecting one as a
fixture stopped working and test 79 got a 404 instead of the mirrored
page. Save a profile first (no command_do=start, so nothing crawls) to
make the server record the root, then plant the file under it.

Re-checked against a reverted expander gate: still fails there, so the
fixture change did not cost the assertion its teeth.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: cover the running-crawl half of the /website/ override

Test 84 only exercised the idle server, where the override never fires and
the recomputed virtualpath is indistinguishable from the stale one. Drive a
crawl through the server so /website/*.html is rewritten to the GUI refresh
page, which 404s without the recompute.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: cover the overflow, the '..' rejection and the fsfile hoist

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: fix the merged test 84 duplicate post() helper

The rename-side and the branch-side each defined post(), with different
argument shapes; the last one won and silently mangled the save body.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: trim the new comments

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: drop the stale start() arg comments in test 84

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 17:18:43 +02:00
Xavier Roche
c32a47110e webhttrack: the mirror link on the finished page cannot be followed (#709)
* WebHTTrack's mirror links are dead file:// URLs

The GUI is served from http://127.0.0.1:PORT/, and browsers refuse to navigate
from an http: page to a file: URL, so the "browse the mirror" links in
finished.html and file.html did nothing when clicked. They now point at the
server's own /website/ route, which already serves the project directory. The
desktop entry's browse mode still works, because there the shell hands the
file: URL to the browser instead of navigating from a page.

The per-project picker in file.html goes too: /website/ is bound to the running
project, and reaching the other projects over HTTP would mean serving the whole
mirror tree from the control origin.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* finished.html: link the served mirror, not a dead file:// URL

The page is served over http:, so its file:// link is a cross-scheme
navigation that Chrome and Firefox refuse. Nothing happened on click. The
mirror is already reachable at /website/, which the two list entries just
below were using all along.

file.html keeps its file:// links. It is the fresh-session entry point for
sites mirrored earlier, and /website/ is bound to one project, so pointing
it there would trade a dead link for a 404. Serving an arbitrary past
project needs a route that does not exist yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: tighten the browse-link assertion and trim its comments

The runtime check for href="/website/index.html" also matched the list
entry below the anchor, so it passed on the unfixed page; match the
anchor by its mirror-path label instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 17:18:17 +02:00
Xavier Roche
1027a9f392 htsserver: crawled pages run through the template expander and leak the session id (#706)
* htsserver builds the redirect Location header in a 256-byte stack buffer

The POST redirect path checks strlen(file) but sprintf's newfile, which comes
straight from the client's "redirect" POST field with no length cap. A 300-byte
value overflows tmp[256]. The same value reached the Location header with no
CR/LF check, so it could split the response and inject headers.

Append into the dynamic String the other headers already use, and drop the
header entirely when the value carries a CR or LF.

The listen socket was SOCaddr_initany, so the server answered the LAN and not
just the local browser it exists to serve. Bind 127.0.0.1 by default, with
--bind <addr> to widen it again, resolved through the existing gethost() helper
the way proxytrack already does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: satisfy shellcheck and shfmt in the new server test

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: drop the pre-fix narration from the oversized-value comment

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: assert the bound socket, not the announced URL

The listen-address assertions only compared the URL= banner, which is a
literal echo of argv: a build that announced 127.0.0.1 while binding the
wildcard passed. Probe 127.0.0.2 on the same port instead, which a wildcard
listener takes and a loopback-only one leaves free.

Also refuse an empty --bind, which fell through to every interface and
silently undid the new default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: bound the response read and the empty --bind run

The recv() loop had no timeout and read until EOF; htsserver need not close
the connection after responding, which wedged the macOS runner for over an
hour. Stop at the end of the header block, which is all the test reads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* htsserver: the session id must gate the request body, not the reply

Every field of a POST body is written straight into the one global key store
the templates and the command dispatcher both read, and "command" from there
reaches the engine. The gate ran after that write and compared "sid" against
"_sid" -- but "_sid" is copied into "sid" beforehand so the templates can
render it, so a request that simply omitted the field compared equal to
itself. Only a wrong id was refused; an absent one passed. Clearing the reply
afterwards does not help either, because the dispatcher sits outside the reply
guard.

Authenticate before parsing instead: scan the raw body for "sid", require at
least one occurrence and reject if any of them differs, and drop the body
untouched when it does not match. That leaves the shared template key alone,
and it closes the dispatcher for free.

A refused request also emitted only a Content-length line, since the status
line for that branch was behind _DEBUG. Any client reads that as a protocol
error, which is how test 68 failed rather than reporting the refusal. Send a
403 instead.

Tests 68 and 77 posted without an id, which is what the engine used to accept,
so both now fetch the one the server renders into the form. Test 78 covers
accept, missing, empty and wrong, asserts the 403, and probes the key store
through ${projname} rather than the suppressed reply -- a reply-only assertion
passes even when the write goes through.

Also fix a leak that hung macOS CI: start() runs inside a command
substitution, so its $! never reached the parent and stop() guarded on an
empty variable, leaving one htsserver per call. Test 77 starts four, which is
exactly the four orphans the runner reported while sitting for half an hour
behind a green test log. Take the pid from the PID= line the server already
announces.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* htsserver: serve the crawled mirror verbatim, never through the expander

WebHTTrack serves the mirror under /website/ from the same small server as its
own GUI, and the decision to run a response through the ${...} template
expander was a substring test for ".htm" on the request path. A mirrored page
therefore had its directives evaluated: ${_sid} rendered the live session id,
handing the crawled site the token that authenticates commands on the local
GUI, and ${do:...} gave it the rest of the template verbs.

The /website/ prefix was already detected, but only to keep the crawl-state
override from hijacking a mirror request. Reuse it as the expansion gate, so
expansion is limited to files under the GUI's html root, and re-evaluate it
after that override, which can substitute a GUI page for a mirror path.
Mirrored pages keep their text/html type: verbatim must not turn browsing the
mirror into a download.

Note that mirrored content still shares the control origin, so a script in it
can read the session id from a GUI page itself; that is a separate fix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: cover the running-crawl half of the /website/ override

Test 84 only exercised the idle server, where the override never fires and
the recomputed virtualpath is indistinguishable from the stale one. Drive a
crawl through the server so /website/*.html is rewritten to the GUI refresh
page, which 404s without the recompute.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 16:57:59 +02:00
Xavier Roche
d0a1573a04 webhttrack: "max site size" sets a per-file cap and discards the HTML one (#708)
* webhttrack: "max site size" set a per-file cap instead of the overall one

step4.html mapped all three size fields of the wizard onto --max-files (-m),
so "Max site size" emitted a per-file limit rather than --max-size (-M), and
landed a second -m on the command line. That second -m also clobbered the HTML
per-file limit: a bare -m<n> resets maxfile_html, so whichever of the two came
last won. Point sizemax at --max-size and emit the bare -m before the -m,<n>
form so both per-file caps survive.

Two template typos in the same family, where a malformed ${...} renders as
nothing or as its own key instead of erroring: the winprofile.ini writer's
Dos=${dos was missing its closing brace, and option2b.html's OK button read
${LANG_OK] with a bracket.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* tests: carry the session id in test 81's POST

The gate that landed with #700 refuses a body without one, so the wizard
POST came back refused and the option audit had nothing to read.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: close the confirmation-biased gaps in test 81

Post the size fields empty too, so a step4.html that lost its ${test:} guard
and rendered a valueless --max-files= is caught; assert the winprofile.ini
MaxHtml/MaxOther/MaxAll keys the header claimed to audit; and pin the OK button
label, since an unknown ${LANG_} key renders empty and passed the absence check.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 16:40:40 +02:00
Xavier Roche
4362ac34cb Check snprintf truncation where the result must be complete (#711)
Adds `sprintfbuff()`/`slprintfbuff()` to `htssafe.h`: a formatted print that truncates to fit and returns whether it had to, marked `warn_unused_result` so the answer cannot be dropped. It fills the gap between the `strcpybuff` family, which aborts on overflow, and `String`, which grows without bound. Abort is the wrong contract wherever the text is built from a remote peer's reply.

Four `-Wformat-truncation=` sites used the result as if `snprintf` had never truncated. Two only needed a bigger destination, so they get one: `hts_finish_makeindex`'s `tempo` was a flat 1024 against a 2048-byte escaped URL and is now sized off it, and the wizard's `cmd[4096]` could not hold the answers it concatenates. The other two cannot grow. `create_back_tmpfile` formats `<url_sav>.bak` into a buffer the same size as `url_sav`, and that struct is installed, so a dropped extension would alias the backup onto the live file that `back_finalize_backup()` unlinks. ProxyTrack's `startUrl[1024]` is fed by cache content. Both take the error path they already had, and ProxyTrack moves to the next cache entry rather than publishing a clipped one.

On what the wrapper buys, since it is not what I first assumed: checking a raw `snprintf` return inline silences the warning just as well. The wrapper's value is that the capacity comes from `sizeof`, the check is the default rather than the exception, and `warn_unused_result` makes skipping it visible.

`-#test=strsafe` covers the primitive (exact fit, one over, 4 KB source, `size == 1`, trailing canary, destination repoisoned between cases), and `-#test=makeindex` gains a first link whose escaped form overruns the old buffer. Both were mutation-checked. The ProxyTrack change ships without a direct test: its only observable is the catalog page, which renders solely as a PROPFIND fallback I could not drive from curl. 25 gcc warnings down to 21; the rest of the cluster is diagnostic-only and follows separately.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 16:37:48 +02:00
Xavier Roche
6579436607 webhttrack: the command line overflows its argv vector and a quoted value can inject flags (#710)
* htsserver builds the redirect Location header in a 256-byte stack buffer

The POST redirect path checks strlen(file) but sprintf's newfile, which comes
straight from the client's "redirect" POST field with no length cap. A 300-byte
value overflows tmp[256]. The same value reached the Location header with no
CR/LF check, so it could split the response and inject headers.

Append into the dynamic String the other headers already use, and drop the
header entirely when the value carries a CR or LF.

The listen socket was SOCaddr_initany, so the server answered the LAN and not
just the local browser it exists to serve. Bind 127.0.0.1 by default, with
--bind <addr> to widen it again, resolved through the existing gethost() helper
the way proxytrack already does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: satisfy shellcheck and shfmt in the new server test

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: drop the pre-fix narration from the oversized-value comment

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: assert the bound socket, not the announced URL

The listen-address assertions only compared the URL= banner, which is a
literal echo of argv: a build that announced 127.0.0.1 while binding the
wildcard passed. Probe 127.0.0.2 on the same port instead, which a wildcard
listener takes and a loopback-only one leaves free.

Also refuse an empty --bind, which fell through to every interface and
silently undid the new default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: bound the response read and the empty --bind run

The recv() loop had no timeout and read until EOF; htsserver need not close
the connection after responding, which wedged the macOS runner for over an
hour. Stop at the end of the header block, which is all the test reads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* htsserver: the session id must gate the request body, not the reply

Every field of a POST body is written straight into the one global key store
the templates and the command dispatcher both read, and "command" from there
reaches the engine. The gate ran after that write and compared "sid" against
"_sid" -- but "_sid" is copied into "sid" beforehand so the templates can
render it, so a request that simply omitted the field compared equal to
itself. Only a wrong id was refused; an absent one passed. Clearing the reply
afterwards does not help either, because the dispatcher sits outside the reply
guard.

Authenticate before parsing instead: scan the raw body for "sid", require at
least one occurrence and reject if any of them differs, and drop the body
untouched when it does not match. That leaves the shared template key alone,
and it closes the dispatcher for free.

A refused request also emitted only a Content-length line, since the status
line for that branch was behind _DEBUG. Any client reads that as a protocol
error, which is how test 68 failed rather than reporting the refusal. Send a
403 instead.

Tests 68 and 77 posted without an id, which is what the engine used to accept,
so both now fetch the one the server renders into the form. Test 78 covers
accept, missing, empty and wrong, asserts the 403, and probes the key store
through ${projname} rather than the suppressed reply -- a reply-only assertion
passes even when the write goes through.

Also fix a leak that hung macOS CI: start() runs inside a command
substitution, so its $! never reached the parent and stop() guarded on an
empty variable, leaving one htsserver per call. Test 77 starts four, which is
exactly the four orphans the runner reported while sitting for half an hour
behind a green test log. Take the pid from the PID= line the server already
announces.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* webhttrack: bound the argv vector and escape quotes in the wizard command line

The wizard hands its httrack command line to the engine as one string, which
back_launch_cmd() split back into argv. Two things were wrong with that split.

It wrote into a fixed 1024-pointer vector with no bound, and every unquoted
space in the posted string yields an entry, so an ordinary mirror with a few
hundred URLs walked off the allocation. The split now lives in htscmdline.c as
hts_split_cmdline(), which sizes the vector from the separator count before
filling it, and the engine self-tests can reach it.

Quotes were also purely advisory: they toggled the "inside an argument" state
but nothing escaped them, so a double quote typed into a wizard field (user
agent, footer, path, project name) closed the argument early and the rest of
the value was parsed as fresh options -- among them -V, which reaches system().
Escaping has to happen where the argument boundary is known, so the template
gets its own ${arg:} filter for that context; ${html:} keeps its meaning for
the HTML attributes it is used in everywhere else, and HTML escaping would not
help anyway since the browser undoes it when it posts the command line back.
${arg:} backslash-escapes a quote and a backslash, and the splitter reads those
inside a quoted run, the same convention next_token() already implements for
doit.log. A value containing a quote now survives it intact instead of turning
into options.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* webhttrack: keep a stray quote in an unquoted field out of the split

The url and wildcard-filter fields go into the command line outside quotes,
where no backslash can escape anything: a single quote there flips the parity
of every quote after it, so a later escaped value ends up split as flags and
the escaping buys nothing. Emit %22 for those fields instead.

NULL-terminate the argv vector while here, matching the convention the tree
documents in htscharset.c, and fold the third copy of the entity table into
one helper.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: do not feed snprintf's return value back as its size argument

snprintf returns the length it wanted to write, so accumulating it blind
lets the next size argument wrap. The buffer is sized well past what the
loop needs, but the pattern is the one the project forbids.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 16:26:55 +02:00
Xavier Roche
cb98b1b197 Fatal-signal backtraces are unreadable: every engine frame is a bare module+offset (#705)
* Symbolize fatal-signal backtraces through addr2line

backtrace_symbols_fd() resolves names from .dynsym only, and
-fvisibility=hidden keeps every engine frame out of it, so a crash report
arrived as a column of bare module+offset. The handler now emits the raw trace
first and unconditionally, then groups the frames per module and runs addr2line
(or llvm-symbolizer) over the offsets, which reads DWARF and names the static
frames plus their inline chain.

-rdynamic is dropped: it only populated .dynsym and bought exactly one named
frame. -Wl,--build-id replaces it, so a trace from a stripped build can be
matched to its debug symbols.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Move the crash backtrace printer into src/htsbacktrace.c

httrack.c keeps only the two call sites. The symbolizer needs _GNU_SOURCE for
dladdr(), which is now confined to its own translation unit instead of being
forced on the whole CLI front-end.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Test: match glibc's backtrace format with or without the space

backtrace_symbols_fd() prints the trailing "[0xADDR]" with a leading space on
some glibc versions and without on others (Ubuntu 24.04), so the raw-frame
assertion failed everywhere but the dev box. Match only up to the offset.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Skip pseudo-modules with no file on disk

A frame in linux-vdso.so.1 made addr2line complain instead of resolving, so
the arm64 leg lost its symbolized output entirely. Renumber the test too:
77 landed on master with #700.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Stop discarding local symbols, which made the names wrong

--discard-all drops the local symbol of every static function, so addr2line
attributes the frame to the nearest surviving global: the abort frame read
dns_timeout_selftests instead of abortf_, with the file and line still right.
A wrong name is worse than none, and because the symbols go at link time no
-dbgsym package can recover them. Costs 21784 bytes on libhttrack.so.3, 0.6%.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Symbolize once when the handler itself faults

A fault inside the handler re-enters the printer, which interleaved a second
symbolized trace on the same fd and spent a second budget: 3.04s and two
overlapping reports, measured.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <xroche@gmail.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 15:43:17 +02:00
Xavier Roche
da8fbfff49 Fix dead array-address checks and discarded-qualifier casts (-Waddress, -Wcast-qual) (#703)
* Drop NULL tests on inline array members

`lien_back::url_sav`, `htsblk::msg` and POSIX `dirent::d_name` are arrays,
so testing their address folds to a constant and gcc/clang report it
(-Waddress, -Wpointer-bool-conversion). Every site keeps whatever real
condition sat beside the dead one, so behavior is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* help_wizard: check the allocation, not the arrays it contains

The out-of-memory guard has been constant-false since d593418 folded the
nine separate wizard buffers into one struct: the names it tests are now
inline arrays, so `malloct()`'s result is never checked and an exhausted
heap gets a NULL-page write instead of the intended message. Also switch
the raw free() to freet() and release the struct on the two early returns
that leaked it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* proxytrack: test the WebDAV header fields, not their addresses

`PT_Element::lastmodified` and `::contenttype` are inline arrays, so both
`if`s were constant-true (-Waddress); use the `[0]` form the same file
already uses when it emits the GET headers. Neither is observable:
get_time_rfc822("") returns 0 and falls through to the index timestamp,
and proxytrack_add_DAV_Item already substitutes application/octet-stream
for an empty mime, which a PROPFIND probe against a cache entry carrying
no Content-Type confirms both before and after.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Fix the discarded const qualifiers rather than casting them away

`binput` and `cache_binput` only ever read through their source pointer,
so they take `const char *` now; that alone clears the cast in
htsrobots.c, and neither is exported nor declared in an installed header,
so no ABI question arises. `treathead` keeps `char *rcvd` because it does
NUL-cut the header in place, and the two selftest calls that fed it a
string literal get a mutable buffer instead, matching their three
siblings and removing a latent write to .rodata. The remaining two are
one-liners: zlib's `next_in` is already `const` under -DZLIB_CONST, and
htsback can call the non-const `jump_protocol` twin on its mutable
`url_adr`. libhttrack.vcxproj gains ZLIB_CONST so the MSVC build agrees
with autotools, as webhttrack and proxytrack already do.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: prove the WebDAV mime/timestamp fallback survives an empty field

proxytrack's DAV PROPFIND response computes a fallback content-type and
timestamp when a cache entry has no Content-Type/Last-Modified; that
fallback already existed before commit eae1dd0 changed the surrounding
always-true array-address checks, so this test guards the equivalence
rather than a bug. Verified it fails when the fallback default is
disabled, and passes unmodified against the pre-eae1dd0 code too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: bound the PROPFIND request and cut the header comment

curl had no --max-time; an unbounded read wedges the runner instead of
failing it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: skip the WebDAV mime test on Windows

It is the first test to run proxytrack as a live listener, and MSYS cannot
reap a native one: the orphan wedged the whole Windows suite past its
45-minute budget, twice, destroying the log upload with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 14:49:29 +02:00
Xavier Roche
a7fbd3f739 htsserver builds the redirect Location header in a 256-byte stack buffer (#700)
* htsserver builds the redirect Location header in a 256-byte stack buffer

The POST redirect path checks strlen(file) but sprintf's newfile, which comes
straight from the client's "redirect" POST field with no length cap. A 300-byte
value overflows tmp[256]. The same value reached the Location header with no
CR/LF check, so it could split the response and inject headers.

Append into the dynamic String the other headers already use, and drop the
header entirely when the value carries a CR or LF.

The listen socket was SOCaddr_initany, so the server answered the LAN and not
just the local browser it exists to serve. Bind 127.0.0.1 by default, with
--bind <addr> to widen it again, resolved through the existing gethost() helper
the way proxytrack already does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: satisfy shellcheck and shfmt in the new server test

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: drop the pre-fix narration from the oversized-value comment

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: assert the bound socket, not the announced URL

The listen-address assertions only compared the URL= banner, which is a
literal echo of argv: a build that announced 127.0.0.1 while binding the
wildcard passed. Probe 127.0.0.2 on the same port instead, which a wildcard
listener takes and a loopback-only one leaves free.

Also refuse an empty --bind, which fell through to every interface and
silently undid the new default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: bound the response read and the empty --bind run

The recv() loop had no timeout and read until EOF; htsserver need not close
the connection after responding, which wedged the macOS runner for over an
hour. Stop at the end of the header block, which is all the test reads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* htsserver: the session id must gate the request body, not the reply

Every field of a POST body is written straight into the one global key store
the templates and the command dispatcher both read, and "command" from there
reaches the engine. The gate ran after that write and compared "sid" against
"_sid" -- but "_sid" is copied into "sid" beforehand so the templates can
render it, so a request that simply omitted the field compared equal to
itself. Only a wrong id was refused; an absent one passed. Clearing the reply
afterwards does not help either, because the dispatcher sits outside the reply
guard.

Authenticate before parsing instead: scan the raw body for "sid", require at
least one occurrence and reject if any of them differs, and drop the body
untouched when it does not match. That leaves the shared template key alone,
and it closes the dispatcher for free.

A refused request also emitted only a Content-length line, since the status
line for that branch was behind _DEBUG. Any client reads that as a protocol
error, which is how test 68 failed rather than reporting the refusal. Send a
403 instead.

Tests 68 and 77 posted without an id, which is what the engine used to accept,
so both now fetch the one the server renders into the form. Test 78 covers
accept, missing, empty and wrong, asserts the 403, and probes the key store
through ${projname} rather than the suppressed reply -- a reply-only assertion
passes even when the write goes through.

Also fix a leak that hung macOS CI: start() runs inside a command
substitution, so its $! never reached the parent and stop() guarded on an
empty variable, leaving one htsserver per call. Test 77 starts four, which is
exactly the four orphans the runner reported while sitting for half an hour
behind a green test log. Take the pid from the PID= line the server already
announces.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 13:59:15 +02:00
Xavier Roche
c4b803eb33 -%S list file over 4GB overflows the heap (#702)
* -%S list file over 4GB overflows the heap

hts_main_internal() sized the -%S buffer as `cl + fz + 8192` and stored
the sum in an int url_sz, then fread() the untruncated 64-bit fz into it.
A 4GB+100KB rules file wraps the capacity to 110602 bytes on x64, the
realloct() succeeds, and the read walks off the heap. Intermediate sizes
land on a negative int and fail the allocation, which is luck, not design.

Route the file-size arithmetic through llint_grow_size_t(), a saturating
sibling of llint_to_size_t() that refuses a total it cannot represent, and
widen url_sz and the filelist offsets to size_t. The "config" sizing in the
same function and htscore.c's primary_len had the same shape: a file size
accumulated into an int before reaching an allocator. htscache.c's two
mirrored-file comparisons held a 64-bit fsize_utf8() in a size_t, which
truncates on Win32 and re-downloads a >4GB file already on disk.

Found by MSVC C4244 on x64; invisible to gcc/clang because int64_t to
size_t is width-preserving on LP64.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Print T_SOC with the format matching its width, not a bare %d

T_SOC is unsigned __int64 on Win64 (htsglobal.h): passing it to fprintf's
%d is undefined behavior, flagged by MSVC C4477. Add T_SOCP beside the
typedef, following the existing LLintP/INTsysP precedent, and use it at
both deletesoc() call sites (htslib.c:2601, :2607).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Fix MSVC C2099 in the new growsize self-test

A static const object used inside another object's static initializer is a
GNU/clang extension, not standard C: MSVC's /TC C mode rejects it ("initializer
is not a constant"). Replace the over32 local with a macro.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: cover the slack-only overrun and the largest capacity

A helper dropping the slack bound passed the table; -1 as extra also refused
either way, since llint_to_size_t() maps it to SIZE_MAX regardless.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 13:23:03 +02:00
Xavier Roche
408316db99 ARC cache replays a truncated HTTP reason phrase (#701)
* ARC cache replays a truncated HTTP reason phrase

proxytrack bounded the reason-phrase copy out of an ARC index by
sizeof(pos) - 1 where pos is a const char *, so a stored "404 Not Found"
replays as "404 Not Fou" (and "404 Not" on 32-bit). Use strncatbuff with
the destination's own size.

Fold the nine copies of the buff() family's source-capacity expression
into HTS_SIZEOF_SRC_, applying sizeof to the type so a decayed operand no
longer trips -Wsizeof-array-decay at five call sites; MSVC keeps the old
expression behind the guard HTS_IS_CHAR_BUFFER already uses. The two
other raw strncat calls become strncatbuff, htsbuff_catn stops handing
strnlen the (size_t)-1 sentinel, and htsweb.c no longer compares ep
against a NULL eps.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Size the new strsafe buffers away from sizeof(char*)

char[8] equals a pointer on LP64, so MSVC's array-vs-pointer heuristic
read the unterminated source as a pointer, skipped the bound and never
aborted; the x64 build failed while Win32 passed. Same trap the existing
comment in that function already warns about.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 11:31:45 +02:00
Xavier Roche
450db10d92 Compiler-flag probes accept flags clang only warns about, and -rdynamic never reaches the linker (#699)
* configure: fix flag probes, move -rdynamic to the link line

AX_CHECK_COMPILE_FLAG only checks the exit status, and clang merely warns on
an unknown -W name, so -Wmissing-parameter-type reached every clang build and
warned on all 66 TUs. Probe with -Werror; -Wformat-nonliteral also needs
-Wformat there or gcc rejects it and the flag would be lost.

-rdynamic lived in DEFAULT_CFLAGS, i.e. AM_CPPFLAGS, so no link line ever saw
it; make it a link check. Drop -pie from CFLAGS_PIE (LDFLAGS_PIE has it), and
drop -Wdeclaration-after-statement, a C90 rule the gnu17 build does not follow
anywhere else.

Distinct build warnings: gcc 82 -> 54, clang 54 -> 41.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* configure: tighten the two new flag-block comments

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 11:24:36 +02:00
Xavier Roche
dfdf10e7fd No UTF-8 to system-codepage conversion is exported to the Windows GUI (#698)
WinHTTrack is an MBCS build, so it still lands on the ANSI-only Win32 entry points (TTN_NEEDTEXTA and the like) while the engine hands it UTF-8. The other direction is already exported as `hts_convertStringSystemToUTF8`, so this adds the mirror, `hts_convertStringUTF8ToSystem`, a one-liner over the existing `hts_convertStringCPFromUTF8`. The GUI can then delegate instead of keeping its own MultiByteToWideChar/WideCharToMultiByte copy (`CopyTextUTF8ToCP` in newlang.cpp, added while fixing #114). `hts_convertStringFromUTF8` would also have done the job, but it is declared plain `extern` and never reaches the DLL export table; left alone here. Windows-only addition, so no POSIX ABI change and no soname move.

The new `-#test=syscharset` self-test round-trips against the raw Win32 two-step it replaces, and skips off Windows.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 09:11:12 +02:00
Xavier Roche
7fe073e9fb CLI display does not repaint when the terminal is resized (#697)
* CLI: redraw the whole screen when the terminal is resized

The animated -%v display assumed a fixed 80x24 layout: it cleared the screen
once at start, and afterwards cleared only to end of line on the rows it
wrote. A resize left stale wrapped text around the frame, and on a terminal
shorter than the 20-row frame the stats block scrolled off the top at every
refresh.

Poll the terminal geometry at each refresh (TIOCGWINSZ, or the console screen
buffer info on Windows) and repaint in full when it moved. The in-progress
list is capped to the rows that fit, and the URL column follows the width; it
stays at the historical 40 characters on an 80-column terminal.

Closes #97

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: assert the resize repaint, the row clamp and the URL width

The first pass only checked that a clear-screen followed one resize, which a
display clearing on every frame passes just as well, and neither the row clamp
nor the URL budget was exercised at all: the single 24x80 to 40x100 resize
leaves both at their 80x24 values.

The harness now waits for the display instead of sleeping, resizes width and
height separately, and asserts a repaint after each one, none in between, a
frame that fits a 10-row terminal, and a long URL that stops being truncated at
200 columns. The long path is a new /trickle/deep route, so the shared
/trickle/ index other tests assert on stays untouched. Mutating any of the four
behaviors away fails the test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 08:56:26 +02:00
dependabot[bot]
f47359247d Bump actions/checkout from 6 to 7 (#696)
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-26 08:37:41 +02:00
dependabot[bot]
dfc72fdbc6 Bump actions/cache from 4 to 6 (#695)
Bumps [actions/cache](https://github.com/actions/cache) from 4 to 6.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v4...v6)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-26 08:04:17 +02:00
dependabot[bot]
8ed607b078 Bump actions/upload-artifact from 4 to 7 (#694)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4 to 7.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4...v7)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-26 08:04:14 +02:00
Xavier Roche
0708bf9f98 CI: let Dependabot track GitHub Actions versions (#693)
Dependabot only watched vcpkg, so the action pins drifted by hand:
windows-build.yml sat on actions/checkout@v4 while everything else moved to
v6. Add the github-actions ecosystem and bump that stray checkout to v6.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 10:17:54 +02:00
Xavier Roche
301d7717be Windows CI: cache vcpkg binary archives across runs (#692)
vcpkg builds openssl/brotli/zlib/zstd from source on every Windows run.
Redirect its files binary cache into the workspace and persist it with
actions/cache, keyed on the manifest (which carries the builtin-baseline).

x-gha, which used to do this, was dropped from vcpkg-tool (#1662) after
GitHub changed the Actions cache API; actions/cache over the archive dir is
the endorsed replacement and stays within the GitHub-owned-actions policy.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 10:17:14 +02:00
Xavier Roche
e13f0f10c9 Drop the removed Java .class parser from the credits (#691)
* Drop the removed Java .class parser from the credits

The Java binary .class parser was removed in #552, but its credit lingered:
"JavaParserClasses: Yann Philippot" in the WinHTTrack About box (lang.def plus
every lang/*.txt), and "for the java binary .class parser" in greetings.txt and
the contact page. Remove the About-box line and the now-defunct descriptor,
keeping Yann Philippot's name in the developed-by acknowledgments.

lang.def and lang/*.txt are read as strict line pairs, so the edit deletes the
substring inside the single credits line rather than removing any line, keeping
the pairing intact and each translated file's English msgid in lockstep with
lang.def. Verified by tests/62_lang-integrity.test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Also drop the localized java-parser credit from the translations

The About box shows the translation (msgstr), not the English msgid. The first
pass only removed the literal English "JavaParserClasses: Yann Philippot", so the
12 files whose translators localized the credit (Chinese, French, Russian,
Turkish, ...) still displayed it. Remove the localized credit line from each of
those msgstrs too; the pairing and every English msgid are untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Reword Yann Philippot's credit as a past contributor

Dropping the descriptor left a bare name in the developed-by list. Mark the
java binary .class parser as a past contribution instead, keeping the
acknowledgment meaningful now that the code is gone.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* contact.html: use a literal dot in .class

The " dot " spelling is the page's email anti-harvest obfuscation; applied to a
filename extension it just left a stray double space. The page already writes
literal dots elsewhere (v2.0, v3.0).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 15:31:29 +02:00
Xavier Roche
3265a5de00 Release 3.49.14 (#690)
Bump the version in configure.ac, htsglobal.h and version.rc, move VERSION_INFO to 3:6:0, and add the history.txt and debian/changelog entries.

The WARC fields landed at the tail of httrackp and htsblk, but the embedded htsoptstate and htsblk members sit mid-struct, so the fields after them shifted. Installed layouts moved while the soname stays .so.3; the VERSION_INFO comment records that rather than claiming the layouts are unchanged.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 08:23:28 +02:00
Xavier Roche
d33f666b38 Windows: load IE cookies from long or non-ASCII folders (FindFirstFileW) (#689)
* Windows: import IE cookies via FindFirstFileW so long/non-ASCII jars load

cookie_load's IE-cookie scan and its cookies.txt read used the narrow
ANSI file API (FindFirstFileA, fopen, remove), so a cookie folder with a
non-ASCII or >MAX_PATH path was silently skipped, and any matched IE
cookie name was fed back through the mirror path as CP_ACP bytes. Route
the glob through hts_pathToUCS2 + FindFirstFileW, convert cFileName to
UTF-8 before rebuilding the path, and use the FOPEN/UNLINK wrappers.

The A->W find and the sink swaps land together on purpose: swapping only
the sink would feed a CP_ACP name into hts_fopen_utf8's UTF-8 decode and
mojibake the accidentally-consistent ANSI path. hts_pathToUCS2 is
un-static'd (internal, still -fvisibility=hidden / not HTSEXT_API) so the
glob gets the same \\?\ long-path treatment as the other wrappers.

Last piece of the #133 Windows long-path series. Self-test
-#test=cookieimport drives cookie_load against a long, non-ASCII folder;
POSIX is a positive control (IE block compiled out), the wide glob and
IE import run on the Windows legs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Trim review-flagged comments to one line

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 17:18:18 +02:00
Xavier Roche
6223739cba Windows: enumerate long or non-ASCII directories via FindFirstFileW (#688)
* Windows: enumerate long or non-ASCII directories via FindFirstFileW

The opendir/readdir emulation was fully ANSI (FindFirstFileA, CP_ACP
cFileName), so it capped listing at MAX_PATH and mis-decoded a non-ASCII
path — the odd wrapper out in an engine that feeds UTF-8 paths and reads
d_name as UTF-8. The one live Windows caller, the end-of-crawl
hts-cache/ref cleanup, silently no-op'd at a long or non-ASCII project
root, leaking the temp ref/ directory into the finished mirror.

Route both through hts_pathToUCS2 (\\?\ prefixing, #684) and the wide
APIs, converting cFileName back to UTF-8. d_name now holds UTF-8, so
HTS_DIRENT_SIZE grows to 1024 to fit MAX_PATH's worst-case expansion; the
struct is _WIN32-only, no POSIX ABI impact. Self-test direnum drives it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Trim comments to one line each (review follow-up)

Condense the opendir/readdir and direnum-selftest comments per the
review's comment-conciseness pass; no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 14:59:45 +02:00
Xavier Roche
301f5c2f2f topindex: fix mojibake for non-ASCII project categories on Windows (#216) (#683)
* topindex: convert the winprofile.ini category to UTF-8 on Windows (#216)

The project category read from hts-cache/winprofile.ini comes back in the
Windows ANSI codepage and is written into the charset=utf-8 topindex
template, so a non-ASCII category renders as mojibake. Convert it the same
way #681 did the project name. The topindex self-test now writes a
winprofile.ini with a non-ASCII category and asserts the generated index
carries the UTF-8 form.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* topindex: drop the redundant category-assert comment

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 13:49:21 +02:00
Xavier Roche
dd321171b4 tests/fsize: skip when the filesystem caps file size below 5GB (#687)
The 01_engine-fsize self-test builds a 5GB sparse file to exercise the 32-bit size wrap. On GNU/Hurd i386 the ext2fs translator rejects the extend with EFBIG, failing the Debian build on that port. Return 77 (automake skip) instead of 1 when the extend fails with EFBIG, and map that to a skip in the .test wrapper. Any other errno, or a wrong size report, still fails, so real regressions are not masked.
2026-07-23 13:47:58 +02:00
Xavier Roche
2f158c05d0 Windows: mirror files under non-ASCII or long directories silently break (#686)
* Windows: route mirror-tree file ops through the UTF-8/long-path wrappers

The engine's raw fopen/remove/rename/rmdir/mkdir/fexist/fsize calls on
mirror-tree paths bypassed the UTF-8 (_w*) wrappers, so on Windows a mirror
under a non-ASCII or >MAX_PATH directory silently broke: a long/non-ASCII
file read as absent (its unlink/reget skipped), or its bytes went to a
mojibaked, truncated path.

Swap those call sites in htscoremain/htscore/htsparse/htsindex/htshelp to
the FOPEN/UNLINK/RENAME/MKDIR/fexist_utf8/fsize_utf8 twins, and add
hts_rmdir_utf8 (RMDIR) for the three rmdir sites that lacked a wrapper. The
arguments are already UTF-8 (Windows argv is transcoded at startup, fconcat
is byte-transparent), so there is no double-conversion. Raw calls on
genuinely system-charset or ASCII paths (getenv/$HOME, structcheck's ASCII
twin, "config") are left as-is.

-#test=mirrorio drives a long AND non-ASCII path through the wrapped guards
and the new rmdir wrapper; on POSIX it stands as a byte-transparent control.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* test: cover RENAME on the long+non-ASCII mirror path

The mirrorio self-test drove FOPEN/UNLINK/RMDIR but not RENAME, which the
sweep newly routes to. Rename the leaf to a non-ASCII sibling and verify
the move before teardown.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 13:47:38 +02:00
Xavier Roche
6c74d94802 Windows: reach past MAX_PATH via \\?\ at the file-op choke-point (#133) (#684)
* Windows: reach past MAX_PATH via \\?\ at the file-op choke-point (#133)

The _w* file wrappers on Windows all funnel their path through one
converter. Route it through a new hts_pathToUCS2 that, for a path near
MAX_PATH, absolutizes and normalizes it with GetFullPathNameW and
prepends the "\\?\" verbatim prefix ("\\?\UNC\" for UNC shares) so the
wide file APIs accept paths past 260 chars. Shorter paths keep today's
exact behavior, so nothing changes until the naming ceiling in htsname.c
is raised (a follow-up); a -#test=longpath self-test drives a >260-char
path through the wrappers to exercise the new branch on Windows CI now.

Partially addresses #133.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Run the long-path self-test on the Windows CI leg; tighten comments

The Windows job iterates an explicit test glob (01_engine-*, 01_zlib-*,
*_local-*, ...), so 76_engine-longpath-io.test matched nothing and the
\\?\ path never ran on the one platform that needs it. Rename it to
01_engine-longpath-io.test so the glob picks it up and the prefixing is
actually exercised on Windows. Also trim the review comments to the why.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 12:27:44 +02:00
Xavier Roche
00b0f5728c Use the platform path limit off Windows, not the Windows MAX_PATH (#133)
* Use the platform path limit off Windows, not the Windows MAX_PATH (#133)

url_savename capped every saved path at 236 chars (MAX_PATH minus 8.3
headroom minus the ".delayed" marker) on all platforms, so Linux, macOS
and Android hashed long names to fit a limit only Windows has. Off
Windows, derive the ceiling from the platform's own PATH_MAX/NAME_MAX,
clamped to the fixed save buffer so an oversized output dir can never
push the final path past it and abort. Windows keeps the MAX_PATH
ceiling until the engine can prefix its paths with \\?\.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Cut the save name in bytes, not codepoints, so multibyte paths can't overflow

The path-length guard measured length in UTF-8 codepoints (hts_stringLengthUTF8)
while the buffer that receives parent+name is a fixed byte array whose overflow
aborts() rather than truncates. With the ceiling raised to ~1984 on POSIX, a
multibyte name (e.g. a long CJK path) can stay under the codepoint cap yet run
its byte length past the 2048-byte buffer, crashing the crawl on the final
prepend. Add an unconditional byte-boundary cut before the parent is prepended,
and drop the earlier codepoint-unit clamp it supersedes. Covers the oversized
-O parent case in the same measure. Regression test uses a 339-char CJK name
under a ~1020-byte output dir, which aborted before the cut.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Leave an oversized parent to the existing abort, not an empty colliding name

When the output dir alone fills the save buffer, forcing the name to empty made
every URL under it collapse to the same path, feeding the unbounded collision
sprintf and turning a clean abort into a 1-2 byte overflow. Only shrink the
name; a buffer-filling parent aborts on the prepend exactly as before (that
oversized-parent path is pre-existing and unrelated to the #133 ceiling).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 09:36:06 +02:00
Xavier Roche
93314bcef9 topindex: fix mojibake for non-ASCII project names on Windows (#216) (#681)
* topindex: non-ASCII project names render as mojibake on Windows (#216)

hts_buildtopindex() lists each sub-project by the name FindFirstFileA
returns in the ANSI codepage, then writes it into a document declaring
charset=utf-8, so a non-ASCII name shows up as mojibake. Convert the name
to UTF-8 on Windows, matching the gif-path fix already in this function
for #217. The topindex self-test now builds a non-ASCII sub-project and
asserts the generated index carries the UTF-8 name.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* topindex: drop const so freet() can null the converted name (MSVC)

The #ifdef _WIN32 conversion block only compiles on Windows, where MSVC
rejected freet() nulling a char *const (C2166). Make the pointer mutable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* topindex: tighten the added comments

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 09:34:27 +02:00
Xavier Roche
d3d3bce8af cmdguide: add a WARC/WACZ archive recipe (#680)
The command-line guide's recipe section had no entry for WARC output, so
the five --warc* flags were discoverable only through --help or the man
page. Add a §11 recipe leading with the gotcha that trips people: the
archive is written alongside the browsable mirror, not instead of it.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 08:09:16 +02:00
Xavier Roche
b3e51d753b WARC: file the --warc help under Log/index/cache, not Build (#679)
WARC is a transaction-level archive of what was fetched (a sibling of the
hts-cache), not a browsable-mirror build format like MHTML, so it belongs in
the Log/index/cache group next to the cache options. Help text only; the
webhttrack GUI already places it on the Log/Index/Cache tab (option9).

Signed-off-by: Xavier Roche <xroche@gmail.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 07:07:05 +02:00
Xavier Roche
5099efc1cf WARC: store response bodies verbatim by default, drop --warc-verbatim (#678)
Verbatim compressed bodies (Content-Encoding kept, only hop-by-hop
Transfer-Encoding dropped, Content-Length rewritten to the stored length)
are now the sole behavior, matching wget --warc and Heritrix. The former
default that decoded the body and stripped Content-Encoding is gone, along
with the --warc-verbatim switch (added the same day, never released), the
warc_verbatim option field, and the dual-mode plumbing in
normalize_http_headers.

This also fixes a cap-truncated compressed response: it previously stored
the raw compressed partial while stripping Content-Encoding, mislabeling
gzip bytes as identity. Keeping Content-Encoding makes the record's label
match its body; the truncation still carries WARC-Truncated: length/time.

header_is now tolerates whitespace before the ':' so a non-compliant
"Content-Encoding : gzip" is still recognized.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 03:35:53 +02:00
Xavier Roche
e1d4c35ee6 WARC: WACZ packaging (--wacz) (#677)
* Add --warc-cdx: sorted CDXJ index alongside the WARC archive

WARC v2 PR B. --warc-cdx (alias --warc-cdxj, sub-flag -%rc) writes a sorted
CDXJ index next to the .warc.gz, one line per response/revisit/resource record
(warcinfo/request are not indexed): the SURT sort key, a 14-digit timestamp,
and a JSON object with url, mime, status, payload digest, and the record's byte
offset and length in the gzip stream so a replay tool can seek and inflate a
single member. Rotation-aware (the filename field tracks the current segment).

SURT canonicalization is new (surt_canon in htswarc.c): scheme/userinfo
dropped, host lowercased with a leading www[digits] label and the scheme
default port stripped, labels reversed and comma-joined, a non-default port
kept, IPv4 and [IPv6] literals left verbatim. Lines are accumulated in the
writer and qsort'd in LC_ALL=C byte order at close.

Tested by -#test=warc-surt (SURT vectors, 01_engine, MSan-instrumented) and
-#test=warc-cdx (end-to-end: sorted, one line per record, each offset/length
independently inflates to the matching WARC-Target-URI; 01_zlib).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Add --wacz: package the WARC archive, CDXJ index and pages as a WACZ file

--wacz bundles the crawl's WARC segment(s), the sorted CDXJ index and a
generated pages.jsonl into a single WACZ 1.1.1 package at crawl end, using
the in-tree minizip. It implies --warc and --warc-cdx. Every ZIP entry is
stored (no re-deflate of the already-gzipped WARC), as the WACZ spec
requires. datapackage.json lists each file with its SHA-256 and size, and
datapackage-digest.json chains the digest of datapackage.json.

SHA-256 comes from OpenSSL; a build without OpenSSL cannot emit conformant
digests, so --wacz is refused there with a clear log line while the WARC and
CDXJ are still written.

pages.jsonl captures the top-level 200 text/html responses (URL + WARC-Date),
bounded like the CDX accumulator; the first seeds datapackage mainPageUrl.

Tests: an in-process self-test (-#test=warc-wacz) unzips the package and
asserts the layout, STORE mode on every entry, each recomputed SHA-256, the
digest chain, and the pages header; a crawl test validates a real .wacz with
a stdlib validator (py-wacz/pywb when importable). Both skip cleanly without
OpenSSL.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Package WACZ atomically so a failed re-run can't destroy a good archive

warc_wacz_package opened <base>.wacz directly with a create/truncate zip
handle, wiping the previous archive before it checked its inputs existed. A
re-run that produced zero indexable records (empty crawl, or every fetch
erroring so warc_cdx_flush writes no .cdx) truncated the good .wacz, failed to
find indexes/index.cdx, then unlinked the now-empty file with no log. Same
data-loss class as the hard-abort file truncation (#522).

Build the package into <base>.wacz.tmp and only rename it over <base>.wacz on
full success (RENAME, with an unlink+rename fallback so Windows can clobber).
On any error unlink the temp, leave the existing archive untouched, and warn
(the old error path was silent); the writer keeps opt for close-time logging.

The warc-wacz self-test now proves it: after a good package it drops the .cdx,
re-runs empty, and asserts the .wacz is byte-unchanged. Also tightens both the
C self-test and the Python validator against the WACZ spec (profile,
wacz_version, digest path, and >= 1 pages.jsonl body row with url + ts) so a
writer emitting an empty package can't pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 01:32:34 +02:00
Xavier Roche
69c562bf0c WARC: verbatim compressed response bodies (--warc-verbatim) (#675)
* WARC: add --warc-verbatim to store compressed bodies as received

--warc-verbatim (internal -%rv, implies --warc) captures a compressed
response in its as-received content-coded form instead of the decoded
strategy-B body. back_finalize already materializes the whole de-chunked
compressed body as a temp file just before decoding it, so the tee is a
save-before-unlink: adopt that spool onto the new htsblk.warc_rawpath /
warc_rawsize instead of unlinking it, and emit the response record with
Content-Encoding kept and Content-Length set to the compressed length
(normalize_http_headers gains a keep_ce mode). WARC-Payload-Digest is
then over the coded payload, which is what the record carries.

Non-compressed responses, the never-spooled is_write archive-ext case,
and the cap-truncated second emit site all leave warc_rawpath NULL and
fall back to strategy B unchanged. The spool is owned by the entry:
freed and unlinked in warc_free_request, NULLed in back_copy_static and
back_unserialize so a shallow copy never double-unlinks.

Output is now byte-replay-identical to what the origin sent, matching
wget --warc and Heritrix.

Tests: a warc-verbatim engine self-test asserts the response keeps
Content-Encoding: gzip, Content-Length == the compressed length, the
stored bytes equal the gzip input and inflate to the known plaintext
(bite-checked); tests/74_local-warc-verbatim crawls a gzip-served
fixture and runs the strategy-A/B differential through
warc-validate.py --verbatim (inflate(stored) == the decoded body).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* WARC verbatim: cover the direct-to-disk adoption path, harden the validator

Review follow-up to --warc-verbatim (strategy A).

The spool adoption in back_finalize runs on both the in-memory and the
direct-to-disk (is_write) branch, but only the in-memory path was crawl-tested.
Add a non-hypertext gzip fixture (application/octet-stream) so the crawler
streams it to disk and the is_write adoption is exercised; the warc-validate
--verbatim gate now asserts Content-Encoding kept, Content-Length == compressed
length, and inflate(stored) == the served plaintext for both assets. Confirmed
at runtime that page.html takes is_write=0 and data.bin is_write=1, and
bite-checked that dropping the is_write adoption fails the new assertion.

warc-validate --verbatim now requires WARC-Payload-Digest on a verbatim record
when the file emits digests at all (an OpenSSL build), so a regression that
drops the digest is caught without breaking the no-OpenSSL leg.

Extract warc_adopt_rawspool() so the WARC spool bookkeeping lives behind the
htswarc boundary, mirroring warc_stash_response. Comment trims throughout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 01:02:45 +02:00
Xavier Roche
d83ed3fdee Add --warc-cdx: sorted CDXJ index alongside the WARC archive (#674)
WARC v2 PR B. --warc-cdx (alias --warc-cdxj, sub-flag -%rc) writes a sorted
CDXJ index next to the .warc.gz, one line per response/revisit/resource record
(warcinfo/request are not indexed): the SURT sort key, a 14-digit timestamp,
and a JSON object with url, mime, status, payload digest, and the record's byte
offset and length in the gzip stream so a replay tool can seek and inflate a
single member. Rotation-aware (the filename field tracks the current segment).

SURT canonicalization is new (surt_canon in htswarc.c): scheme/userinfo
dropped, host lowercased with a leading www[digits] label and the scheme
default port stripped, labels reversed and comma-joined, a non-default port
kept, IPv4 and [IPv6] literals left verbatim. Lines are accumulated in the
writer and qsort'd in LC_ALL=C byte order at close.

Tested by -#test=warc-surt (SURT vectors, 01_engine, MSan-instrumented) and
-#test=warc-cdx (end-to-end: sorted, one line per record, each offset/length
independently inflates to the matching WARC-Target-URI; 01_zlib).

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 00:36:30 +02:00
Xavier Roche
ce278a4184 Expose WARC output in the webhttrack GUI (#672)
* Expose --warc / --warc-file in the webhttrack GUI

Add a WARC output checkbox and an optional archive-name field to the
"Log, Index, Cache" option tab, beside store-all-in-cache. The checkbox
emits --warc (auto-named archive); the text field emits --warc-file NAME.
Wiring mirrors how #589 added cookies-file and strip-query: the option9.html
form fields, the generated httrack command and winprofile.ini in step4.html,
and the reload remap in step2.html.

New LANG_WARC / LANG_WARCFILE strings and tooltips land in lang.def,
English.txt and Francais.txt; the remaining 28 language files fall back to
French for now and are a follow-up.

The webhttrack smoke test now also fetches option9.html and requires the
WARC control to render.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Translate the webhttrack WARC strings into the remaining languages

PR #672 added a WARC toggle to the webhttrack GUI with 4 new LANG keys,
translated only in English and French. Append the same 4 msgid/translation
pairs to the other 28 lang/*.txt so those locales stop falling back to
English. Each translation is encoded in the file's declared LANGUAGE_CHARSET
(the charset the server serves the page and parses the form as) with a strict
lossless round-trip, matching each file's existing CRLF/LF line ending.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 22:49:36 +02:00
Xavier Roche
f6f46e88b2 WARC segment rotation, truncation tagging, and FTP resource records (#673)
* Add WARC/1.1 v1.1: --warc-max-size rotation, WARC-Truncated, FTP resource records

Extends the merged WARC writer with the three v1.1 items. All logic stays in
htswarc.c; the engine hooks stay thin.

--warc-max-size N (-%rs) rotates the archive into NAME-00000.warc.gz, -00001,
... once a segment passes N bytes (wget naming), each segment led by its own
warcinfo and never splitting a record. N<=0 keeps the single-file behavior.

WARC-Truncated tags a body cut short by a cap. The mirror size (-M) and time
(-E) caps abort in-flight transfers and overwrite the slot's status to a
negative TIMEOUT, after which back_finalize and the WARC hook both bail; so the
partial is archived at the abort site, before the clobber, with WARC-Truncated:
length/time. HTTrack's own incomplete/retry bookkeeping is untouched, and a
genuine broken transfer (not a cap) is still not archived. The per-file cap (-m)
rejects on Content-Length rather than truncating, so it has no truncated body.
Standard ISO 28500 tokens only (length/time/disconnect); the task's non-standard
"disk" is left out.

FTP transfers have no HTTP envelope, so an ftp:// capture becomes one resource
record: WARC-Type: resource, the payload's own Content-Type, block = payload,
no request/response. FTP back_finalize runs on the main loop (the worker thread
only flips status), so the single-writer no-lock design holds.

Self-tests warc-trunc / warc-ftp / warc-rotate added to the -#test=warc family
and 01_zlib-warc.test; each was confirmed to fail without the corresponding
code. man/httrack.1 + html regenerated for the %r help line.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Range-check --warc-max-size and regenerate the man page

Replace the unchecked sscanf on the --warc-max-size argument with a
strtoll parse that rejects non-numeric, negative, and overflowing values,
leaving the default 0 (single archive) intact. The man page and its HTML
render were already regenerated for the option's help line in the feature
commit, so this only tightens the parse.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 22:39:32 +02:00
Xavier Roche
1e0c009273 Add WARC/1.1 archive output (--warc) (#671)
Adds `--warc` / `--warc-file NAME`, writing an ISO-28500 WARC/1.1 archive of a crawl (warcinfo + request + response records, gzip per record, SHA-1 digests under OpenSSL) so HTTrack output replays in Wayback-style tools (ReplayWeb.page, pywb). Under `--update`, unchanged resources become `revisit` records instead of duplicate copies. No new dependency.

Fidelity note: HTTrack decodes gzip/br/zstd inline, so v1 stores the decoded body and normalizes the encoding headers (drops `Content-Encoding`/`Transfer-Encoding`, recomputes `Content-Length`). Valid and replayable; verbatim-compressed capture and WACZ are v2. Logic is isolated in a new `src/warc.c`.

First phase (v1) of #668.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 20:58:01 +02:00
Xavier Roche
f894a64ff8 Skip an oversized -%F footer instead of aborting the crawl (#670)
* Skip an oversized -%F footer instead of aborting the crawl

The per-page footer is expanded into a fixed ~3KB stack buffer. On overflow
hts_footer_format returns <0 and leaves the buffer unterminated, but the call
site ignored the return and ran strcatbuff on it regardless; strcatbuff's
bounded strlen finds no terminator within capacity and abort()s, killing the
crawl with SIGABRT. Reachable whenever the footer expansion exceeds the buffer
(a field referenced several times with a long URL/path, or a URL that triples
under the HTML-comment escaping); the pre-existing default footer has the same
exposure.

Guard the emit on the formatter's return: on overflow, drop the footer rather
than crash. Regression test drives a file:// crawl of a deep path with a footer
repeating {path} and asserts the crawl completes; it aborts on the pre-fix
binary.

Closes #669

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Skip the footer-overflow test on Windows (MAX_PATH)

The test triggers the overflow with a path longer than Windows MAX_PATH (260),
which the source tree and mirror output both exceed there, so httrack can't
create it and the crawl fails for an unrelated reason. Restrict it to POSIX; the
fix is platform-independent and the formatter's overflow-return contract is
still covered cross-platform by the footerfmt self-test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Allowlist the Windows skip of the footer-overflow test

The Windows CI pins expected skips so an all-skipped suite can't pass green;
01_engine-footer-overflow.test now skips there (MAX_PATH), so add it to the list.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 19:33:19 +02:00
Xavier Roche
febdd08cae Named footer fields for -%F (#667)
* Add named {addr}{path}{date}{version} footer fields alongside the legacy %s form

The -%F footer was a bare positional printf: each %s consumed the next of a
fixed addr/path/date/version arg list. It is order-coupled and inexpressive
(you cannot reach {date} without emitting addr and path first, cannot reorder,
and a wrong %s count silently yields "???" or shifted values), and the help
text advertised a cryptic bracket syntax.

hts_footer_format now dispatches on content: a footer containing %s keeps the
legacy positional model byte-for-byte (the default footer and every existing
-%F string are unaffected), while a footer without %s uses named fields
{addr} {path} {date} {version}, with "{{"/"}}" for a literal brace and any
unrecognized {...} left verbatim so typos stay visible. Sanitization is
unchanged: addr/path still pass through html_inline_safe() at the call site,
so the comment-injection guard (#165) still holds.

Driven by a new -#test=footerfmt self-test (tests/01_engine-footerfmt.test)
covering both models, brace escaping, the mixed-mode dispatch boundary, and
the overflow/zero-size return paths.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Regenerate html/httrack.man.html for the -%F help change

The man/html sync CI guard requires html/httrack.man.html to change whenever
man/httrack.1 does; the -%F help-text update regenerated the man page but not
its html rendering.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Add {url}{lastmodified}{mime}{charset}{status}{size} footer fields

Extends the named footer set beyond addr/path/date/version. hts_footer_format
now takes a {name,value} table instead of four fixed parameters, so the field
set is data rather than signature and the legacy %s path looks addr/path/date/
version up by name (order-independent, still byte-for-byte). The call site
supplies the new fields from the current response: {url} (scheme + host + path,
credentials stripped like {addr}), {lastmodified}, {mime}, {charset}, {status}
and {size}.

Every network-derived string is html_inline_safe()'d, since the footer sits
inside an HTML comment and a value holding "-->" would otherwise close it and
inject markup (#165); {status}/{size} are formatted integers and need none.
Documented in --help, the man pages and the Command-Line Guide, and covered by
the -#test=footerfmt self-test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 18:15:12 +02:00
Xavier Roche
3e1d1abdab Poll longer for the test server's port so a cold Windows CI start can't flake (#666)
The offline crawl harness reads the server's ephemeral port from the "PORT <n>"
line it prints once bound, but only waited 5s (50 x 0.1s) and matched head -n1.
Under `make check -jN` up to 16 Python servers cold-start at once; on a loaded
Windows runner a cold MSYS Python start can lag past 5s, so discovery timed out
with an empty log ("could not discover server port:") and failed 13_local-cookies
spuriously. Give it a 30s deadline, and match the PORT line anywhere so a stray
startup warning merged via 2>&1 can't wedge discovery until timeout.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 17:51:01 +02:00
Xavier Roche
8f7bfbb2f6 Fix duplicate and dead --help option letters, document -%z, -%t, -y (#665)
#R was listed twice: "cache repair" and "old FTP routines". The FTP handler
is commented out in the parser, so the only live meaning is cache repair;
drop the bogus line. #X is a no-op (the parser prints "option has no effect"),
so remove its help line and misleading default marker.

Add help lines for three real but undocumented options: -%z
(--disable-compression), -%t (keep the original file extension), and -y
(--background-on-suspend).

Regenerate man/httrack.1 and html/httrack.man.html from the updated --help.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:52:08 +02:00
Xavier Roche
2ba353eb6f Bound offline crawl tests with a watchdog so a wedged fetch can't hang the Windows CI step (#660)
The offline suite waited on each httrack crawl with a bare wait bounded only by the engine's --max-time; a fetch that wedges past that (a Windows socket stall the engine misses) blocked wait forever and ran the test step to its 45-minute cap. wait_bounded attaches the #595 kill_tree reaper to the crawl pid so an overrun is reaped in seconds, stop_server now reaps the server's native tree, and 72_watchdog-crawl proves the fail-fast against an always-stall endpoint.
2026-07-22 13:07:26 +02:00
Xavier Roche
3dd8a97611 Localize the hardcoded step3/step4 wizard headings in webhttrack (#664)
The "Select URLs" and "Start" section headings on the webhttrack server
wizard were literal English, bypassing the ${LANG_...} substitution, so
they stayed English in all 30 locales. Reuse existing, already-translated
keys: step3 -> ${LANG_G44} ("Web Addresses: (URL)"), step4 -> ${LANG_J9}
("Start"). No lang/*.txt changes needed.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:04:18 +02:00
Xavier Roche
84656760be Fix typos and modernize dated content in the FAQ (#663)
Correct spelling errors (beginning, dishonest, redistributing, forbid,
personal, chosen, address, occurrences) and refresh stale content: collapse
the obsolete per-Windows-version questions into a single current statement,
drop dead OS keywords, describe the cookies.txt / --cookies-file workflow
instead of the Netscape/IE folder steps, and reword the rtsp note to say
streaming protocols are out of scope rather than "not supported yet".

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:04:13 +02:00
Xavier Roche
f26d11d3aa Fix prose typos in abuse/filters docs; cross-ref --why filter debugging (#662)
Correct several spelling typos in html/abuse.html and html/filters.html,
and point filters.html readers at the --why (-%Y) option, which reports
which filter rule accepted or blocked a given URL.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:04:08 +02:00
Xavier Roche
46d65c6610 Retire obsolete cmddoc.html; give the man page a nav home (#661)
The 2007 cmddoc.html beginner walkthrough is fully subsumed by cmdguide.html.
Repoint index.html's "Command-line version" bullet to httrack.man.html (the
generated reference, previously unlinked) and mark the now-orphaned legacy
options.html as possibly-stale, pointing at the man page and the guide. Also
fixes two long-standing index.html typos (relese, Developper).

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:04:03 +02:00
146 changed files with 11491 additions and 1244 deletions

View File

@@ -6,3 +6,9 @@ updates:
directory: /src
schedule:
interval: weekly
# Keep the workflow action pins current (they only rot manually otherwise).
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly

View File

@@ -31,7 +31,7 @@ jobs:
env:
CC: ${{ matrix.cc }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -75,7 +75,7 @@ jobs:
name: build (no python3, Debian buildd)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -119,7 +119,7 @@ jobs:
name: build (macOS arm64, clang)
runs-on: macos-14
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -166,7 +166,7 @@ jobs:
name: webhttrack smoke (macOS arm64)
runs-on: macos-14
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -199,7 +199,7 @@ jobs:
name: build (linux i386, gcc -m32)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -242,7 +242,7 @@ jobs:
name: sanitize (ASan+UBSan, gcc)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -296,7 +296,7 @@ jobs:
name: msan (MemorySanitizer, clang)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -344,7 +344,7 @@ jobs:
name: fuzz (libFuzzer corpus replay, clang)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -383,7 +383,7 @@ jobs:
name: build (no openssl, --disable-https)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -437,7 +437,7 @@ jobs:
zlib1g-dev libssl-dev libbrotli-dev libzstd-dev \
debhelper devscripts lintian fakeroot
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -468,7 +468,7 @@ jobs:
name: distcheck (release tarball)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive
@@ -493,7 +493,7 @@ jobs:
if: github.event_name == 'pull_request'
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
fetch-depth: 0
@@ -537,7 +537,7 @@ jobs:
tests/*.test
tools/mkdeb.sh
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: Install linters
run: |
@@ -572,7 +572,7 @@ jobs:
if: github.event_name == 'pull_request'
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
fetch-depth: 0
@@ -624,7 +624,7 @@ jobs:
if: github.event_name == 'pull_request'
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
fetch-depth: 0

View File

@@ -26,7 +26,7 @@ jobs:
# Upload findings to the repo's code-scanning dashboard.
security-events: write
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
submodules: recursive

View File

@@ -23,8 +23,13 @@ jobs:
matrix:
platform: [x64, Win32]
configuration: [Release]
# Redirect vcpkg's default `files` binary cache into the workspace so
# actions/cache can persist it. vcpkg builds openssl/brotli/zlib/zstd from
# source otherwise, several minutes every run.
env:
VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}\vcpkg_cache
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
with:
submodules: recursive # coucal lives in src/coucal
@@ -44,6 +49,24 @@ jobs:
shell: pwsh
run: vcpkg integrate install
# vcpkg errors if VCPKG_DEFAULT_BINARY_CACHE points at a missing dir, and
# actions/cache does not create it on a miss.
- name: Create the vcpkg binary cache directory
shell: pwsh
run: New-Item -ItemType Directory -Force -Path $env:VCPKG_DEFAULT_BINARY_CACHE | Out-Null
# x-gha is gone (vcpkg-tool #1662 dropped it after GitHub changed the cache
# API), so cache the binary archives directly. Keyed on the manifest, which
# carries the builtin-baseline, so a Dependabot bump busts it; restore-keys
# still seeds the unchanged ports' archives, so only the bumped one rebuilds.
- name: Cache vcpkg binary archives
uses: actions/cache@v6
with:
path: ${{ github.workspace }}\vcpkg_cache
key: vcpkg-${{ matrix.platform }}-${{ hashFiles('src/vcpkg.json') }}
restore-keys: |
vcpkg-${{ matrix.platform }}-
# The runner image's vcpkg checkout is pinned to some commit; our manifest's
# builtin-baseline is usually newer, so `git show <baseline>:versions/...`
# fails until that commit is local. Fetch exactly the pinned baseline (read
@@ -202,14 +225,16 @@ jobs:
# Every gate here exits 77, so an all-skipped suite would report green having
# tested nothing: pin the skips, and floor the passes in case the glob empties.
expected_skips=" 48_local-crange-memresume.test 71_local-crange-repaircache.test" # pending #581
# footer-overflow skips on Windows (needs a path past MAX_PATH); crange pending #581;
# webdav-mime needs a reapable background listener, which MSYS cannot give it.
expected_skips=" 01_engine-footer-overflow.test 48_local-crange-memresume.test 71_local-crange-repaircache.test 79_local-proxytrack-webdav-mime.test"
[ "$pass" -ge 90 ] || { echo "::error::only $pass tests passed ($skip skipped)"; exit 1; }
[ "$skipped" = "$expected_skips" ] || { echo "::error::unexpected skips:$skipped"; exit 1; }
[ "$fail" -eq 0 ] || { echo "::error::failing:$failed"; exit 1; }
- name: Upload the test logs
if: always()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: engine-tests-${{ matrix.platform }}-${{ matrix.configuration }}
path: tests/*.log
@@ -217,7 +242,7 @@ jobs:
- name: Upload MSBuild logs
if: always()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: msbuild-${{ matrix.platform }}-${{ matrix.configuration }}
path: msbuild-*.log

View File

@@ -1,6 +1,6 @@
AC_PREREQ([2.71])
AC_INIT([httrack], [3.49.13], [roche+packaging@httrack.com], [httrack], [http://www.httrack.com/])
AC_INIT([httrack], [3.49.14], [roche+packaging@httrack.com], [httrack], [http://www.httrack.com/])
AC_COPYRIGHT([
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998-2015 Xavier Roche and other contributors
@@ -29,13 +29,12 @@ AC_CONFIG_SRCDIR(src/httrack.c)
AC_CONFIG_MACRO_DIR([m4])
AC_CONFIG_HEADERS(config.h)
AM_INIT_AUTOMAKE([subdir-objects])
# 3:5:0: 3.49.13 leaves every installed struct layout and the exported symbol set
# untouched vs 3.49.12; the 2 GB (_stat64) and UTF-8 export widenings are _WIN32-only,
# and INTsys widened on POSIX (#600) but sits in no installed struct. Stays soname
# .so.3; bump revision. x32 alone changes layout (LLint was 4 bytes there, #524), and
# is not a release architecture.
# 3:6:0: 3.49.14 grows two installed structs from the inside (htsoptstate sits mid-
# httrackp, htsblk mid-lien_back), so the fields after them shift: httrackp.cookies_file
# +8, lien_back.http11 +48. Soname deliberately stays .so.3: the shifted fields are
# 3.49-era additions no external consumer reaches, and a libhttrack3 rename isn't worth it.
# (3:0:0 was the htsblk mime-buffer widening, the ABI break that moved .so.2 -> .so.3.)
VERSION_INFO="3:5:0"
VERSION_INFO="3:6:0"
AM_MAINTAINER_MODE
AC_USE_SYSTEM_EXTENSIONS
@@ -67,10 +66,11 @@ AC_SUBST(LT_CV_OBJDIR,$lt_cv_objdir)
AC_SUBST(VERSION_INFO)
### Default CFLAGS
# No -Wdeclaration-after-statement: nothing sets -std=, so this builds as gnu17.
DEFAULT_CFLAGS="-Wall -Wformat -Wformat-security \
-Wmultichar -Wwrite-strings -Wcast-qual -Wcast-align \
-Wstrict-prototypes -Wmissing-prototypes \
-Wmissing-declarations -Wdeclaration-after-statement \
-Wmissing-declarations \
-Wpointer-arith -Wsequence-point -Wnested-externs \
-D_REENTRANT"
AC_SUBST(DEFAULT_CFLAGS)
@@ -78,27 +78,30 @@ DEFAULT_LDFLAGS=""
AC_SUBST(DEFAULT_LDFLAGS)
### Additional flags (if supported)
AX_CHECK_COMPILE_FLAG([-Wparentheses], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wparentheses"])
AX_CHECK_COMPILE_FLAG([-Winit-self], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Winit-self"])
AX_CHECK_COMPILE_FLAG([-Wunused-but-set-parameter], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wunused-but-set-parameter"])
AX_CHECK_COMPILE_FLAG([-Waddress], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Waddress"])
AX_CHECK_COMPILE_FLAG([-Wuninitialized], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wuninitialized"])
AX_CHECK_COMPILE_FLAG([-Wformat=2], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wformat=2"])
AX_CHECK_COMPILE_FLAG([-Wformat-nonliteral], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wformat-nonliteral"])
AX_CHECK_COMPILE_FLAG([-Wmissing-parameter-type], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wmissing-parameter-type"])
AX_CHECK_COMPILE_FLAG([-Wold-style-definition], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wold-style-definition"])
AX_CHECK_COMPILE_FLAG([-Wignored-qualifiers], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wignored-qualifiers"])
# -Werror on probes: exit-status-only checks let clang's warn-on-unknown-flag through.
AX_CHECK_COMPILE_FLAG([-Wparentheses], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wparentheses"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Winit-self], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Winit-self"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Wunused-but-set-parameter], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wunused-but-set-parameter"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Waddress], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Waddress"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Wuninitialized], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wuninitialized"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Wformat=2], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wformat=2"], [], [-Werror])
# -Wformat-nonliteral needs -Wformat in the probe or gcc rejects it as ignored.
AX_CHECK_COMPILE_FLAG([-Wformat-nonliteral], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wformat-nonliteral"], [], [-Werror -Wformat])
AX_CHECK_COMPILE_FLAG([-Wmissing-parameter-type], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wmissing-parameter-type"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Wold-style-definition], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wold-style-definition"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Wignored-qualifiers], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Wignored-qualifiers"], [], [-Werror])
# Make htssafe.h's pointer-dest 'warning' attribute a hard error in our build
# (migration is at zero; a new char* dest is a regression). gcc/clang each take
# only their own spelling; downstream keeps the plain warning, not a build break.
AX_CHECK_COMPILE_FLAG([-Werror=attribute-warning], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Werror=attribute-warning"])
AX_CHECK_COMPILE_FLAG([-Werror=user-defined-warnings], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Werror=user-defined-warnings"])
AX_CHECK_COMPILE_FLAG([-fstrict-aliasing -Wstrict-aliasing], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstrict-aliasing -Wstrict-aliasing"])
AX_CHECK_COMPILE_FLAG([-Werror=attribute-warning], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Werror=attribute-warning"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-Werror=user-defined-warnings], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -Werror=user-defined-warnings"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-fstrict-aliasing -Wstrict-aliasing], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstrict-aliasing -Wstrict-aliasing"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-fstack-protector-strong], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstack-protector-strong"],
[AX_CHECK_COMPILE_FLAG([-fstack-protector], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstack-protector"])])
AX_CHECK_COMPILE_FLAG([-fstack-clash-protection], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstack-clash-protection"])
AX_CHECK_COMPILE_FLAG([-fcf-protection], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fcf-protection"])
AX_CHECK_LINK_FLAG([-Wl,--discard-all], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,--discard-all"])
[AX_CHECK_COMPILE_FLAG([-fstack-protector], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstack-protector"], [], [-Werror])], [-Werror])
AX_CHECK_COMPILE_FLAG([-fstack-clash-protection], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstack-clash-protection"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-fcf-protection], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fcf-protection"], [], [-Werror])
# No --discard-all: it drops the local symbols naming every static function, so
# a trace misattributes them to the nearest surviving global. Costs 0.6% size.
AX_CHECK_LINK_FLAG([-Wl,--no-undefined], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,--no-undefined"])
AX_CHECK_LINK_FLAG([-Wl,-z,relro,-z,now], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,-z,relro,-z,now"])
AX_CHECK_LINK_FLAG([-Wl,-z,noexecstack], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,-z,noexecstack"])
@@ -120,13 +123,13 @@ AC_SUBST([LIBC_FORCE_LINK])
### PIE
CFLAGS_PIE=""
LDFLAGS_PIE=""
AX_CHECK_COMPILE_FLAG([-fpie -pie], [CFLAGS_PIE="-fpie -pie"])
AX_CHECK_COMPILE_FLAG([-fpie], [CFLAGS_PIE="-fpie"], [], [-Werror])
AX_CHECK_LINK_FLAG([-pie], [LDFLAGS_PIE="-pie"])
AC_SUBST([CFLAGS_PIE])
AC_SUBST([LDFLAGS_PIE])
## Export all symbols for backtraces
AX_CHECK_COMPILE_FLAG([-rdynamic], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -rdynamic"])
# Ties a crash trace from a stripped build back to its separate debug symbols.
AX_CHECK_LINK_FLAG([-Wl,--build-id], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,--build-id"])
### Check for -fvisibility=hidden support
gl_VISIBILITY
@@ -170,7 +173,7 @@ AC_CHECK_TYPE(sa_family_t, [], [AC_DEFINE([sa_family_t], [uint16_t], [sa_family_
AX_CHECK_ALIGNED_ACCESS_REQUIRED
# check for various headers
AC_CHECK_HEADERS([execinfo.h])
AC_CHECK_HEADERS([execinfo.h sys/ioctl.h])
### zlib
CHECK_ZLIB()

8
debian/changelog vendored
View File

@@ -1,3 +1,11 @@
httrack (3.49.14-1) unstable; urgency=medium
* New upstream release: WARC/WACZ archive output, Windows long-path support,
named -%F footer fields, and encoding fixes for non-ASCII project paths;
full list in history.txt.
-- Xavier Roche <xavier@debian.org> Fri, 24 Jul 2026 08:01:43 +0200
httrack (3.49.13-1) unstable; urgency=medium
* New upstream release: SOCKS5 and CONNECT proxy support, brotli and zstd

View File

@@ -5,7 +5,7 @@ Xavier Roche (xroche at httrack.com)
project leader
core engine, Windows/Linux GUI
Yann Philippot (yphilippot at lemel.fr)
for the java binary .class parser
past contributor (java binary .class parser)
With the help of:
Leto Kauler (molotov at tasmail.com)

View File

@@ -4,6 +4,23 @@ HTTrack Website Copier release history:
This file lists all changes and fixes that have been made for HTTrack
3.49-14
+ New: WARC/1.1 archive output (--warc), with a sorted CDXJ index (--warc-cdx) and WACZ packaging (--wacz), also available from webhttrack (#668)
+ New: -%F takes named footer fields such as {url}, {lastmodified}, {mime}, {charset} and {status} instead of a fixed layout (#667)
+ New: a command-line guide organized by task ships with the offline documentation (#649)
+ Fixed: on Windows, paths beyond MAX_PATH truncated files, and mirroring into a long or non-ASCII directory silently failed (#133)
+ Fixed: the top index showed mojibake for non-ASCII project names and categories on Windows (#216)
+ Fixed: webhttrack handed the engine the web form's charset rather than UTF-8, so a non-ASCII path mirrored into a mojibake directory (#629)
+ Fixed: a non-ASCII single -O left the logs and the cache in a mangled twin directory on Windows (#630)
+ Fixed: a path-ceiling truncation dropped the .delayed marker, losing the file (#623)
+ Fixed: an oversized -%F footer aborted the crawl instead of being skipped (#669)
+ Fixed: a rejected 206 resume could loop and lose the file rather than refetch it whole (#581)
+ Fixed: default-port stripping was scheme-blind and dropped explicit ports from https and ftp URLs, and a :80 written with leading zeros mangled the host (#627, #638)
+ Fixed: -K silently reset the -c socket count (#650)
+ Fixed: signed-shift undefined behaviour in the zip-repair local-header read (#639)
+ Changed: the offline documentation drops stale facts, gains an Android help page, and documents the filter wildcards and the real long option forms
+ Changed: multiple internal hardening, test and CI improvements
3.49-13
+ New: SOCKS5 proxy support, with scheme-aware -P URLs (socks5://, socks5h://, connect://) and plain HTTP tunneled through a CONNECT-only proxy (#563, #564)
+ New: decode brotli and zstd content codings, advertised over TLS only as browsers do (#556)

View File

@@ -217,7 +217,7 @@ school or
shows. They might do that because they are connected through expensive modem connection,
or because they would like to consult pages while travelling, or archive sites that may be
removed
one day, make some data mining, comiling information (&quot;if only I could find this
one day, make some data mining, compiling information (&quot;if only I could find this
website I saw one day..&quot;). <br>
There are many good reasons to mirror websites, and this helps many good people.<br>
As a webmaster, you might be interested to use such tools, too: test broken links, move a
@@ -229,7 +229,7 @@ test the webserver response and performances, index it..<br>
Anyway, bandwidth abuse can be a problem. If your site is regularly &quot;clobbered&quot;
by evil downloaders, you have <br>
various solutions. You have radical solutions, and intermediate solutions. I strongly
recomment not to use<br>
recommend not to use<br>
radical solutions, because of the previous remarks (good people often mirror websites).<br>
<br>
In general, for all solutions,<br>
@@ -244,7 +244,7 @@ or, to be extreme: if you unplug the wire, there will be no bandwidth abuse<br>
Good: Will work with good people. Many good people just don't KNOW that they can slow down
a network.<br>
Bad: Will **only** work with good people<br>
How to do: Obvious - place a note, a warning, an article, a draw, a poeme or whatever you
How to do: Obvious - place a note, a warning, an article, a draw, a poem or whatever you
want<br>
<br>
</li><li>Use &quot;robots.txt&quot; file<br>
@@ -266,7 +266,7 @@ Good: Efficient<br>
Bad: Multiple users behind proxies will be slow down, not really easy to setup<br>
How to do: Depends on webserver. Might be done with low-level IP rules (QoS)<br>
<br>
</li><li>Priorize small files, against large files<br>
</li><li>Prioritize small files, against large files<br>
Good: Efficient if large files are the cause of abuse<br>
Bad: Not always efficient<br>
How to do: Depends on the webserver<br>
@@ -283,7 +283,7 @@ How to do: Use routine QoS (fair queuing), or webserver options<br>
<br>
</li><li>Use technical tricks (like javascript) to hide URLs<br>
Good: Efficient<br>
Bad: The most efficient tricks will also cause your website to he heavy, and not
Bad: The most efficient tricks will also cause your website to be heavy, and not
user-friendly (and therefore less attractive, even for surfing users). Remember: clients
or visitors might want to consult offline your website. Advanced users will also be still
able to note the URLs and catch them. Will not work on non-javascript browsers. It will
@@ -335,7 +335,7 @@ Example: Use things like
</li><li>Use technical tricks to temporarily ban IPs<br>
Good: Efficient<br>
Bad: Radical (your site will only be available online for all users), not easy to setup<br>
How to to: Create fake links with &quot;killing&quot; targets<br>
How to do: Create fake links with &quot;killing&quot; targets<br>
Example: Use things like &lt;a href=&quot;killme.cgi&quot;&gt;&lt;nothing&gt;&lt;/a&gt;
(again an example in php:)<br>
<tt>
@@ -417,7 +417,7 @@ Example:<br>
</li><li>Another one is to create images of emails<br>
Good: Efficient, does not require javascript<br>
Bad: There is still the problem of the link (mailto:), images are bigger than text, and it can cause problems for blind people (a good solution is use an ALT attribute with the email written like "smith at mycompany dot com")<br>
How to do: Not so obvious of you do not want to create images by yourself<br>
How to do: Not so obvious if you do not want to create images by yourself<br>
Example: (php, Unix)<br>
<tt>
@@ -491,7 +491,7 @@ echo <br>
</li><li>You can also create temporary email aliases, each week, for all users<br>
Good: Efficient, and you can give your real email in your reply-to address<br>
Bad: Temporary emails<br>
How to do: Not so hard todo<br>
How to do: Not so hard to do<br>
Example: (script &amp; php, Unix)<br>
<tt>

View File

@@ -1,157 +0,0 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<meta name="description" content="HTTrack is an easy-to-use website mirror utility. It allows you to download a World Wide website from the Internet to a local directory,building recursively all structures, getting html, images, and other files from the server to your computer. Links are rebuiltrelatively so that you can freely browse to the local site (works with any browser). You can mirror several sites together so that you can jump from one toanother. You can, also, update an existing mirror site, or resume an interrupted download. The robot is fully configurable, with an integrated help" />
<meta name="keywords" content="httrack, HTTRACK, HTTrack, winhttrack, WINHTTRACK, WinHTTrack, offline browser, web mirror utility, aspirateur web, surf offline, web capture, www mirror utility, browse offline, local site builder, website mirroring, aspirateur www, internet grabber, capture de site web, internet tool, hors connexion, unix, dos, windows 95, windows 98, solaris, ibm580, AIX 4.0, HTS, HTGet, web aspirator, web aspirateur, libre, GPL, GNU, free software" />
<title>HTTrack Website Copier - Offline Browser</title>
<style type="text/css">
<!--
body {
margin: 0; padding: 0; margin-bottom: 15px; margin-top: 8px;
background: #77b;
}
body, td {
font: 14px "Trebuchet MS", Verdana, Arial, Helvetica, sans-serif;
}
#subTitle {
background: #000; color: #fff; padding: 4px; font-weight: bold;
}
#siteNavigation a, #siteNavigation .current {
font-weight: bold; color: #448;
}
#siteNavigation a:link { text-decoration: none; }
#siteNavigation a:visited { text-decoration: none; }
#siteNavigation .current { background-color: #ccd; }
#siteNavigation a:hover { text-decoration: none; background-color: #fff; color: #000; }
#siteNavigation a:active { text-decoration: none; background-color: #ccc; }
a:link { text-decoration: underline; color: #00f; }
a:visited { text-decoration: underline; color: #000; }
a:hover { text-decoration: underline; color: #c00; }
a:active { text-decoration: underline; }
#pageContent {
clear: both;
border-bottom: 6px solid #000;
padding: 10px; padding-top: 20px;
line-height: 1.65em;
background-image: url(images/bg_rings.gif);
background-repeat: no-repeat;
background-position: top right;
}
#pageContent, #siteNavigation {
background-color: #ccd;
}
.imgLeft { float: left; margin-right: 10px; margin-bottom: 10px; }
.imgRight { float: right; margin-left: 10px; margin-bottom: 10px; }
hr { height: 1px; color: #000; background-color: #000; margin-bottom: 15px; }
h1 { margin: 0; font-weight: bold; font-size: 2em; }
h2 { margin: 0; font-weight: bold; font-size: 1.6em; }
h3 { margin: 0; font-weight: bold; font-size: 1.3em; }
h4 { margin: 0; font-weight: bold; font-size: 1.18em; }
.blak { background-color: #000; }
.hide { display: none; }
.tableWidth { min-width: 400px; }
.tblRegular { border-collapse: collapse; }
.tblRegular td { padding: 6px; background-image: url(fade.gif); border: 2px solid #99c; }
.tblHeaderColor, .tblHeaderColor td { background: #99c; }
.tblNoBorder td { border: 0; }
// -->
</style>
</head>
<table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
<tr>
<td><img src="images/header_title_4.gif" width="400" height="34" alt="HTTrack Website Copier" title="" border="0" id="title" /></td>
</tr>
</table>
<table width="76%" border="0" align="center" cellspacing="0" cellpadding="3" class="tableWidth">
<tr>
<td id="subTitle">Open Source offline browser</td>
</tr>
</table>
<table width="76%" border="0" align="center" cellspacing="0" cellpadding="0" class="tableWidth">
<tr class="blak">
<td>
<table width="100%" border="0" align="center" cellspacing="1" cellpadding="0">
<tr>
<td colspan="6">
<table width="100%" border="0" align="center" cellspacing="0" cellpadding="10">
<tr>
<td id="pageContent">
<!-- ==================== End prologue ==================== -->
<h2 align="center"><em>Command-Line Documentation</em></h2>
<br>
The command-line version
<ul>
<li><a href="cmdguide.html">Command-line Guide</a></li>
<br>A task-oriented guide: how to do the common things, and the defaults that surprise newcomers<br><br>
<li><a href="options.html">Command line Options</a></li>
<br>List of all powerful command line options<br><br>
<li>How to use httrack command-line version:</li>
<ul>
<li>Open a shell window</li>
<br>
<br>
<li>Type in <tt>httrack</tt> (or the complete path to the httrack executable)</li>
<br><small><tt>httrack</tt></small>
<br>
<br>
<li>Add the URLs, separated by a blank space</li>
<br><small><tt>httrack www.example.com/foo/</tt></small>
<br>
<br>
<li>If you need, add some options (see the <a href="options.html">option list</a>)</li>
<br><small><tt>httrack www.example.com/foo/ -O "/webs" -N4 -P proxy.myhost.com:3128</tt></small>
<br>
<br>
<li>Launch the command line, and wait until the mirror is finishing</li>
<br><small>You can (especially on the Unix release) press ^C to stop the mirror or put httrack in background</small>
<br>
<br>
</ul>
</ul>
<!-- ==================== Start epilogue ==================== -->
</td>
</tr>
</table>
</td>
</tr>
</table>
</td>
</tr>
</table>
<table width="76%" border="0" align="center" valign="bottom" cellspacing="0" cellpadding="0">
<tr>
<td id="footer"><small>&copy; 1998-2026 Xavier Roche & other contributors - Web Design: Leto Kauler.</small></td>
</tr>
</table>
</body>
</html>

View File

@@ -300,6 +300,7 @@ ones it kept so the local copy browses offline. These options tune both halves.<
<tr><td><tt>--preserve (-%p), --disable-passwords (-%x)</tt></td><td>Leave HTML untouched (no rewriting), and strip passwords out of saved links.</td></tr>
<tr><td><tt>--extended-parsing (-%P), --parse-java (-j)</tt></td><td>Aggressive link discovery, and how much script content is parsed for links.</td></tr>
<tr><td><tt>--mime-html (-%M)</tt></td><td>Save the whole mirror as a single MIME-encapsulated <tt>.mht</tt> archive (<tt>index.mht</tt>).</td></tr>
<tr><td><tt>--single-file (-%Z), --single-file-max-size N</tt></td><td>Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded as <tt>data:</tt> URIs. Assets over the cap (10&nbsp;MB by default) keep their link, as do audio, video, and the links from one page to another. A sibling of <tt>-%M</tt>, not a replacement: see the recipe below for which to pick.</td></tr>
<tr><td><tt>--index (-I), --build-top-index (-%i), --search-index (-%I)</tt></td><td>Build a per-mirror index, a top index across projects, and a searchable keyword index.</td></tr>
</table>
@@ -323,10 +324,18 @@ options control what HTTrack says about itself.</p>
<tr><td><tt>--user-agent (-F)</tt></td><td>The <tt>User-Agent</tt>. Set a browser string to get past crawler blocks; <tt>--user-agent ""</tt> sends none.</td></tr>
<tr><td><tt>--referer (-%R), --from (-%E), --language (-%l), --accept (-%a)</tt></td><td>Referer, From, Accept-Language and Accept headers.</td></tr>
<tr><td><tt>--headers (-%X)</tt></td><td>Add raw header lines to every request.</td></tr>
<tr><td><tt>--footer (-%F)</tt></td><td>A footer written into saved pages (on disk, not a network header).</td></tr>
<tr><td><tt>--footer (-%F)</tt></td><td>A footer written into saved pages (on disk, not a network header). See <b>Footer fields</b> below.</td></tr>
<tr><td><tt>--cookies (-b), --cookies-file (-%K)</tt></td><td>Accept cookies, and preload a Netscape <tt>cookies.txt</tt>.</td></tr>
</table>
<p><b>Footer fields.</b> A footer with no <tt>%s</tt> may reference named fields:
<tt>{addr}</tt>, <tt>{path}</tt>, <tt>{url}</tt>, <tt>{date}</tt> (mirror time),
<tt>{lastmodified}</tt> (the page's Last-Modified), <tt>{version}</tt>,
<tt>{mime}</tt>, <tt>{charset}</tt>, <tt>{status}</tt> and <tt>{size}</tt>; write
<tt>{{</tt> or <tt>}}</tt> for a literal brace. A footer that contains <tt>%s</tt>
keeps the older positional form (host, path, date in that order). Example:
<tt>-%F "&lt;!-- Mirrored from {url} on {date} --&gt;"</tt>.</p>
<p><b>Login.</b> For HTTP Basic auth, put the credentials in the URL:
<tt>http://user:pass@host/</tt>. An <tt>@</tt> inside the username must be written
<tt>%40</tt>. Only Basic is supported, not Digest.</p>
@@ -465,6 +474,31 @@ then:</small><br>
lifts the built-in caps; use it only against infrastructure you are allowed to
load.</small></p>
<h4>Save a WARC archive of the crawl</h4>
<p><tt>httrack https://example.com/ --warc --path mydir</tt><br>
<small>Writes a standard WARC/1.1 file (<tt>httrack-&lt;timestamp&gt;.warc.gz</tt>) in
the project folder alongside the browsable mirror, not instead of it. Set the name
with <tt>--warc-file NAME</tt> and split a large crawl with <tt>--warc-max-size N</tt>;
add <tt>--warc-cdx</tt> for a sorted CDXJ index, or <tt>--wacz</tt> to bundle the
archive, index and pages into one WACZ for replay tools such as
replayweb.page.</small></p>
<h4>Pages you can hand to someone as one file</h4>
<p><tt>httrack https://example.com/ --single-file --path mydir</tt><br>
<small>Rewrites each saved page after the crawl so its stylesheets, scripts,
images and fonts are embedded as <tt>data:</tt> URIs. The mirror stays a normal
browsable tree, with links between pages relative and the assets still on disk,
but any single <tt>.html</tt> file can now be mailed or archived on its own.
Raise or lower the 10&nbsp;MB per-asset limit with
<tt>--single-file-max-size N</tt>; anything over it, plus audio and video, keeps
its link.<br>
Reach for this when the file has to open for someone you cannot make assumptions
about: it is plain HTML and needs no add-on. Reach for <tt>--mime-html</tt>
instead when a Chromium-family browser is a given and the mirror is large: MIME
carries text parts without the base64 tax, keeps each resource's original URL,
and stores a shared asset once rather than re-embedding it in every page that
uses it.</small></p>
<h4>HTTrack as a fetch tool</h4>
<p><tt>httrack --get https://host/file.bin --path tmp</tt><br>
<small><tt>--get</tt> fetches one file with cache, index, depth, cookies and robots

View File

@@ -132,7 +132,7 @@ Xavier Roche (xroche at httrack dot com)
for the main engine and Windows interface
and maintainer for v2.0 and v3.0
Yann Philippot (yphilippot at lemel dot fr)
for the java binary dot class parser
past contributor (java binary .class parser)
David Lawrie (dalawrie at lineone dot net)
Robert Lagadec (rlagadec at yahoo dot fr)
for checking both English & French translations

View File

@@ -111,7 +111,7 @@ See also: The <a href="faq.html#VF1">FAQ</a><br>
starts links, the default mode is to mirror these links - i.e. if one of your start page is
www.example.com/test/index.html, all links starting with www.example.com/test/ will be
accepted. But links directly in www.example.com/.. will not be accepted, however, because
they are in a higher strcuture. This prevent HTTrack from mirroring the whole site. (All
they are in a higher structure. This prevent HTTrack from mirroring the whole site. (All
files in structure levels equal or lower than the primary links will be retrieved.)<br>
</i>
<br>
@@ -127,13 +127,15 @@ See also: The <a href="faq.html#VF1">FAQ</a><br>
an authorization filter, like <b><tt>+*.gif</tt></b>. The pattern is a plus (this one: <b><tt>+</tt></b>),
followed by a pattern composed of letters and wildcards (this one: <b><tt>*</tt></b>).
<br><br>
To forbide a family of links, define
To forbid a family of links, define
an authorization filter, like <b><tt>-*.gif</tt></b>. The pattern is a dash (this one: <b><tt>-</tt></b>),
followed by a the same kind of pattern as for the authorization filter.
<br><br>
Example: +*.gif will accept all files finished by .gif<br>
Example: -*.gif will refuse all files finished by .gif<br>
<br>
To see which rule accepted or blocked a given URL, run HTTrack with the <b><tt>--why</tt></b> (<b><tt>-%Y</tt></b>) option, described in <a href="httrack.man.html#OPTIONS">the manual page</a>.<br>
<br>
<p>
<h4>Scan rules based on size (e.g. accept or refuse files bigger/smaller than a certain size)</h4>
@@ -143,7 +145,7 @@ See also: The <a href="faq.html#VF1">FAQ</a><br>
size to ensure that you won't reach a defined limit.
Example: You may want to accept all files on the domain www.example.com, using '+www.example.com/*',
including gif files inside this domain and outside (eternal images), but not take to large images,
including gif files inside this domain and outside (external images), but not take to large images,
or too small ones (thumbnails)<br>
Excluding gif images smaller than 5KB and images larger than 100KB is therefore a good option;
+www.example.com +*.gif -*.gif*[<5] -*.gif*[>100]

View File

@@ -90,9 +90,10 @@ offline browser : copy websites to a local directory</p>
] [ <b>-NN, --structure[=N]</b> ] [ <b>-%N,
--delayed-type-check</b> ] [ <b>-%D,
--cached-delayed-type-check</b> ] [ <b>-%M, --mime-html</b>
] [ <b>-LN, --long-names[=N]</b> ] [ <b>-KN,
--keep-links[=N]</b> ] [ <b>-x, --replace-external</b> ] [
<b>-%x, --disable-passwords</b> ] [ <b>-%q,
] [ <b>-%Z, --single-file</b> ] [ <b>-LN,
--long-names[=N]</b> ] [ <b>-KN, --keep-links[=N]</b> ] [
<b>-x, --replace-external</b> ] [ <b>-%x,
--disable-passwords</b> ] [ <b>-%q,
--include-query-string</b> ] [ <b>-%g, --strip-query</b> ] [
<b>-o, --generate-errors</b> ] [ <b>-X, --purge-old[=N]</b>
] [ <b>-%p, --preserve</b> ] [ <b>-%T, --utf8-conversion</b>
@@ -100,16 +101,17 @@ offline browser : copy websites to a local directory</p>
] [ <b>-%Y, --why</b> ] [ <b>-u, --check-type[=N]</b> ] [
<b>-j, --parse-java[=N]</b> ] [ <b>-sN, --robots[=N]</b> ] [
<b>-%h, --http-10</b> ] [ <b>-%k, --keep-alive</b> ] [
<b>-%B, --tolerant</b> ] [ <b>-%s, --updatehack</b> ] [
<b>-%u, --urlhack</b> ] [ <b>-%A, --assume</b> ] [ <b>-@iN,
--protocol[=N]</b> ] [ <b>-%w, --disable-module</b> ] [
<b>-F, --user-agent</b> ] [ <b>-%R, --referer</b> ] [
<b>-%E, --from</b> ] [ <b>-%F, --footer</b> ] [ <b>-%l,
--language</b> ] [ <b>-%a, --accept</b> ] [ <b>-%X,
--headers</b> ] [ <b>-C, --cache[=N]</b> ] [ <b>-k,
--store-all-in-cache</b> ] [ <b>-%n, --do-not-recatch</b> ]
[ <b>-%v, --display</b> ] [ <b>-Q, --do-not-log</b> ] [
<b>-q, --quiet</b> ] [ <b>-z, --extra-log</b> ] [ <b>-Z,
<b>-%z, --disable-compression</b> ] [ <b>-%B, --tolerant</b>
] [ <b>-%s, --updatehack</b> ] [ <b>-%u, --urlhack</b> ] [
<b>-%A, --assume</b> ] [ <b>-@iN, --protocol[=N]</b> ] [
<b>-%w, --disable-module</b> ] [ <b>-F, --user-agent</b> ] [
<b>-%R, --referer</b> ] [ <b>-%E, --from</b> ] [ <b>-%F,
--footer</b> ] [ <b>-%l, --language</b> ] [ <b>-%a,
--accept</b> ] [ <b>-%X, --headers</b> ] [ <b>-C,
--cache[=N]</b> ] [ <b>-k, --store-all-in-cache</b> ] [
<b>-%r, --warc</b> ] [ <b>-%n, --do-not-recatch</b> ] [
<b>-%v, --display</b> ] [ <b>-Q, --do-not-log</b> ] [ <b>-q,
--quiet</b> ] [ <b>-z, --extra-log</b> ] [ <b>-Z,
--debug-log</b> ] [ <b>-v, --verbose</b> ] [ <b>-f,
--file-log</b> ] [ <b>-f2, --single-log</b> ] [ <b>-I,
--index</b> ] [ <b>-%i, --build-top-index</b> ] [ <b>-%I,
@@ -121,7 +123,8 @@ offline browser : copy websites to a local directory</p>
] [ <b>-e, --go-everywhere</b> ] [ <b>-%H,
--debug-headers</b> ] [ <b>-%!,
--disable-security-limits</b> ] [ <b>-V, --userdef-cmd</b> ]
[ <b>-%W, --callback</b> ] [ <b>-K, --keep-links[=N]</b>
[ <b>-%W, --callback</b> ] [ <b>-y,
--background-on-suspend</b> ] [ <b>-K, --keep-links[=N]</b>
]</p>
<h2>DESCRIPTION
@@ -646,6 +649,36 @@ don&rsquo;t wait) (--cached-delayed-type-check)</p></td></tr>
<td width="4%">
<p>-%Z</p></td>
<td width="5%"></td>
<td width="82%">
<p>after the mirror, rewrite each saved page with its
stylesheets, scripts, images and fonts inlined as data:
URIs, so any page opens by double-click anywhere (links
between pages stay relative; audio and video stay links);
--single-file-max-size N caps each asset (default 10485760
bytes). %M is the better container where a Chromium-family
browser is a given: one archive, no base64 tax on text, a
shared asset stored once (--single-file)</p></td></tr>
<tr valign="top" align="left">
<td width="9%"></td>
<td width="4%">
<p>-%t</p></td>
<td width="5%"></td>
<td width="82%">
<p>keep the original file extension, don&rsquo;t rewrite it
from the MIME type (%t0 rewrite)</p></td></tr>
<tr valign="top" align="left">
<td width="9%"></td>
<td width="4%">
<p>-LN</p></td>
<td width="5%"></td>
<td width="82%">
@@ -878,6 +911,18 @@ for small files and test requests (%k0 don&rsquo;t use)
<td width="4%">
<p>-%z</p></td>
<td width="5%"></td>
<td width="82%">
<p>do not request compressed content (%z0 request)
(--disable-compression)</p> </td></tr>
<tr valign="top" align="left">
<td width="9%"></td>
<td width="4%">
<p>-%B</p></td>
<td width="5%"></td>
<td width="82%">
@@ -1023,9 +1068,10 @@ headers (-F &quot;user-agent name&quot;) (--user-agent
<td width="82%">
<p>footer string in Html code (-%F &quot;Mirrored [from
host %s [file %s [at %s]]]&quot; (--footer
&lt;param&gt;)</p> </td></tr>
<p>footer string in Html code (-%F &quot;Mirrored from
{url} on {date}&quot;; fields {addr} {path} {url} {date}
{lastmodified} {version} {mime} {charset} {status} {size},
or legacy %s) (--footer &lt;param&gt;)</p></td></tr>
<tr valign="top" align="left">
<td width="9%"></td>
<td width="4%">
@@ -1102,6 +1148,20 @@ update before) (--cache[=N])</p></td></tr>
<td width="4%">
<p>-%r</p></td>
<td width="5%"></td>
<td width="82%">
<p>write an ISO-28500 WARC/1.1 archive; --warc-file NAME
sets the output name, --warc-max-size N rotates segments
past N bytes, --warc-cdx also writes a sorted CDXJ index,
--wacz packages it all as a WACZ file (--warc)</p></td></tr>
<tr valign="top" align="left">
<td width="9%"></td>
<td width="4%">
<p>-%n</p></td>
<td width="5%"></td>
<td width="82%">
@@ -1383,25 +1443,13 @@ p7 get html files before, then treat other files</p>
<td width="8%">
<p style="margin-top: 1em">-#X</p></td>
<p style="margin-top: 1em">-#test</p></td>
<td width="1%"></td>
<td width="82%">
<p style="margin-top: 1em">*use optimized engine (limited
memory boundary checks) (--fast-engine)</p></td></tr>
<tr valign="top" align="left">
<td width="9%"></td>
<td width="8%">
<p>-#test</p></td>
<td width="1%"></td>
<td width="82%">
<p>list engine self-tests (run one with -#test=NAME
[args])</p> </td></tr>
<p style="margin-top: 1em">list engine self-tests (run one
with -#test=NAME [args])</p></td></tr>
<tr valign="top" align="left">
<td width="9%"></td>
<td width="8%">
@@ -1532,17 +1580,6 @@ memory boundary checks) (--fast-engine)</p></td></tr>
<td width="8%">
<p>-#R</p></td>
<td width="1%"></td>
<td width="82%">
<p>old FTP routines (debug) (--repair-cache)</p></td></tr>
<tr valign="top" align="left">
<td width="9%"></td>
<td width="8%">
<p>-#T</p></td>
<td width="1%"></td>
<td width="82%">
@@ -1633,6 +1670,18 @@ each files ($0 is the filename: -V &quot;rm \$0&quot;)
<p>use an external library function as a wrapper (-%W
myfoo.so[,myparameters]) (--callback &lt;param&gt;)</p></td></tr>
<tr valign="top" align="left">
<td width="9%"></td>
<td width="4%">
<p>-y</p></td>
<td width="5%"></td>
<td width="82%">
<p>go to background when suspended (y0 don&rsquo;t)
(--background-on-suspend)</p> </td></tr>
</table>
<h3>Details: Option N

View File

@@ -109,12 +109,12 @@ h4 { margin: 0; font-weight: bold; font-size: 1.18em; }
<li>How to Use</li>
<ul>
<li><a href="shelldoc.html">WinHTTrack/WebHTTrack (GUI version for Windows or Linux/Unix)</a></li>
<br>HTTrack GUI documentation, with step-by-step example, for the Windows release (WinHTTrack) and the Linux/Unix relese (WebHTTrack)<br>
<br>HTTrack GUI documentation, with step-by-step example, for the Windows release (WinHTTrack) and the Linux/Unix release (WebHTTrack)<br>
<br>
<li><a href="android.html">HTTrack on Android</a></li>
<br>Step-by-step guide for the Android app<br>
<br>
<li><a href="cmddoc.html">Command-line version</a></li>
<li><a href="httrack.man.html">Command-line version</a></li>
<br>How to run HTTrack from the command line, and where to find every option<br>
<br>
<li><a href="cmdguide.html">Command-line Guide</a></li>
@@ -123,7 +123,7 @@ h4 { margin: 0; font-weight: bold; font-size: 1.18em; }
<li><a href="fcguide.html">HTTrack Users Guide By Fred Cohen</a></li>
<br>A tutorial that describes all command-line options, for Linux and Windows users<br>
<br>
<li><a href="dev.html">Developper/Programming</a></li>
<li><a href="dev.html">Developer/Programming</a></li>
<br>How to use HTTrack in batch files, and how to use the library<br>
</ul>
<br>

View File

@@ -101,6 +101,10 @@ h4 { margin: 0; font-weight: bold; font-size: 1.18em; }
<h2 align="center"><em>Options</em></h2>
<p><strong>Note:</strong> this is a legacy option list and may be out of date.
For the authoritative, always-current reference see <a href="httrack.man.html">the httrack manual page</a>,
and for a task-oriented walkthrough see the <a href="cmdguide.html">Command-line Guide</a>.</p>
<ul>
<li>Filters: <a href="filters.html">how to use them</a></li>
<br><small>Here you can find informations on filters: how to accept all gif files in a mirror, for example</small>

View File

@@ -97,9 +97,8 @@ ${do:end-if}
</pre>
${LANG_G8} :
${do:output-mode:html-urlescaped}
<a href="file://${path}/${projname}/" target="_new">
${do:output-mode:}
${/* an http: page cannot navigate to file:, so the mirror is reached through the server */}
<a href="/website/index.html" target="_new">
${path}/${projname}
</a></li>
<ul>

View File

@@ -136,6 +136,13 @@ ${listid:build:LISTDEF_3}
<tr><td><input type="checkbox" name="nopurge" ${checked:nopurge}
title='${html:LANG_I1a}' onMouseOver="info('${html:LANG_I1a}'); return true" onMouseOut="info('&nbsp;'); return true"
> ${LANG_I57}</td></tr>
<tr><td><input type="checkbox" name="singlefile" ${checked:singlefile}
title='${html:LANG_SINGLEFILETIP}' onMouseOver="info('${html:LANG_SINGLEFILETIP}'); return true" onMouseOut="info('&nbsp;'); return true"
> ${LANG_SINGLEFILE}</td></tr>
<tr><td>${LANG_SINGLEFILEMAX}
<input name="singlefilemax" value="${singlefilemax}" size="12"
title='${html:LANG_SINGLEFILEMAXTIP}' onMouseOver="info('${html:LANG_SINGLEFILEMAXTIP}'); return true" onMouseOut="info('&nbsp;'); return true"
></td></tr>
</table>
<tr><td>

View File

@@ -103,7 +103,7 @@ ${LANG_Q3}
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK]"
<input type="submit" value="${LANG_OK}"
${do:output-mode:html-urlescaped}
onClick="if (confirm(str_replace(str_replace('${LANG_DIAL7}', '%20', ' '), '%0a', ' '))) { form.closeme.value=1; form.submit(); } return false;"
${do:output-mode:}

View File

@@ -103,6 +103,17 @@ ${do:end-if}
> ${LANG_I61}
<br><br>
<input type="checkbox" name="warc" ${checked:warc}
title='${html:LANG_WARCTIP}' onMouseOver="info('${html:LANG_WARCTIP}'); return true" onMouseOut="info('&nbsp;'); return true"
> ${LANG_WARC}
<br><br>
${LANG_WARCFILE}
<input name="warcfile" value="${warcfile}" size="40"
title='${html:LANG_WARCFILETIP}' onMouseOver="info('${html:LANG_WARCFILETIP}'); return true" onMouseOut="info('&nbsp;'); return true"
>
<br><br>
<input type="checkbox" name="norecatch" ${checked:norecatch}
title='${html:LANG_I5b}' onMouseOver="info('${html:LANG_I5b}'); return true" onMouseOut="info('&nbsp;'); return true"
> ${LANG_I34b}

View File

@@ -141,6 +141,10 @@ ${do:copy:KeepSlashes:keepslashes}
${do:copy:KeepQueryOrder:keepqueryorder}
${do:copy:StripQuery:stripquery}
${do:copy:StoreAllInCache:cache2}
${do:copy:Warc:warc}
${do:copy:WarcFile:warcfile}
${do:copy:SingleFile:singlefile}
${do:copy:SingleFileMaxSize:singlefilemax}
${do:copy:LogType:logtype}
${do:copy:UseHTTPProxyForFTP:ftpprox}
${do:copy:ProxyType:proxytype}

View File

@@ -95,7 +95,7 @@ ${do:end-if}
<table border="0" width="100%">
<tr><td width="90%">
<h2 align="center"><em>Select URLs</em></h2>
<h2 align="center"><em>${LANG_G44}</em></h2>
</td>
${/* show help only if available */}
${do:if-file-exists:html/index.html}

View File

@@ -77,7 +77,7 @@ ${do:end-if}
<table border="0" width="100%">
<tr><td width="90%">
<h2 align="center"><em>Start</em></h2>
<h2 align="center"><em>${LANG_J9}</em></h2>
</td>
${/* show help only if available */}
${do:if-file-exists:html/index.html}
@@ -121,9 +121,9 @@ httrack \
--quiet \
--build-top-index \
${test:todo:--mirror:--mirror:--mirror-wizard:--get:--mirrorlinks:--testlinks:--continue:--update}
${urls}
${test:filelist:-%L "}${filelist}${test:filelist:"}
--path "${html:path}/${html:projname}"
${unquoted:urls}
${test:filelist:-%L "}${arg:filelist}${test:filelist:"}
--path "${arg:path}/${arg:projname}"
\
${test:parseall:--near}
${test:link:--test}
@@ -131,7 +131,7 @@ httrack \
${test:htmlfirst::--priority=7}
\
${do:if-not-empty:BuildString}
--structure "${BuildString}"
--structure "${arg:BuildString}"
${do:end-if}
${test:build:-N0:-N0:-N1:-N2:-N3:-N4:-N5:-N100:-N101:-N102:-N103:-N104:-N105:-N99:-N199:}
\
@@ -150,29 +150,30 @@ ${do:end-if}
${test:travel3::--keep-links=0:--keep-links:--keep-links=3:--keep-links=4}
${test:windebug:--debug-headers}
\
${test:connexion:--sockets=}${connexion}
${test:connexion:--sockets=}${unquoted:connexion}
${test:ka:--keep-alive}
${test:timeout:--timeout=}${timeout}
${test:timeout:--timeout=}${unquoted:timeout}
${test:remt:--host-control=1}
${test:retry:--retries=}${retry}
${test:rate:--min-rate=}${rate}
${test:retry:--retries=}${unquoted:retry}
${test:rate:--min-rate=}${unquoted:rate}
${test:rems:--host-control=2}
\
${test:depth:--depth=}${depth}
${test:depth2:--ext-depth=}${depth2}
${test:maxhtml:--max-files=,}${maxhtml}
${test:othermax:--max-files=}${othermax}
${test:sizemax:--max-files=}${sizemax}
${test:pausebytes:--max-pause=}${pausebytes}
${test:maxtime:--max-time=}${maxtime}
${test:maxrate:--max-rate=}${maxrate}
${test:maxconn:--connection-per-second=}${maxconn}
${test:maxlinks:--advanced-maxlinks=}${maxlinks}
${test:depth:--depth=}${unquoted:depth}
${test:depth2:--ext-depth=}${unquoted:depth2}
${/* -m<n> resets the html limit, so the bare form must precede the -m,<n> one */}
${test:othermax:--max-files=}${unquoted:othermax}
${test:maxhtml:--max-files=,}${unquoted:maxhtml}
${test:sizemax:--max-size=}${unquoted:sizemax}
${test:pausebytes:--max-pause=}${unquoted:pausebytes}
${test:maxtime:--max-time=}${unquoted:maxtime}
${test:maxrate:--max-rate=}${unquoted:maxrate}
${test:maxconn:--connection-per-second=}${unquoted:maxconn}
${test:maxlinks:--advanced-maxlinks=}${unquoted:maxlinks}
\
--user-agent "${html:user}"
--footer "${html:footer}"
--user-agent "${arg:user}"
--footer "${arg:footer}"
\
${url2}
${unquoted:url2}
\
${test:cookies:--cookies=0:}
${test:parsejava:--parse-java=0:}
@@ -181,18 +182,22 @@ ${do:end-if}
${test:keepwww:--keep-www-prefix}
${test:keepslashes:--keep-double-slashes}
${test:keepqueryorder:--keep-query-order}
${test:cookiesfile:--cookies-file "}${html:cookiesfile}${test:cookiesfile:"}
${test:pausefiles:--pause "}${pausefiles}${test:pausefiles:"}
${test:stripquery:--strip-query "}${html:stripquery}${test:stripquery:"}
${test:cookiesfile:--cookies-file "}${arg:cookiesfile}${test:cookiesfile:"}
${test:pausefiles:--pause "}${arg:pausefiles}${test:pausefiles:"}
${test:stripquery:--strip-query "}${arg:stripquery}${test:stripquery:"}
${test:toler:--tolerant}
${test:http10:--http-10}
${test:cache2:--store-all-in-cache}
${test:warc:--warc}
${test:warcfile:--warc-file "}${arg:warcfile}${test:warcfile:"}
${test:singlefile:--single-file}
${test:singlefilemax:--single-file-max-size=}${unquoted:singlefilemax}
${test:norecatch:--do-not-recatch}
${test:logf:--single-log}
${test:logtype:::--extra-log:--debug-log}
${test:index:--index=0:}
${test:index2:--search-index=0:--search-index}
${test:prox:--proxy "}${do:if-not-empty:prox}${test:proxytype::socks5:connect}${test:proxytype:\3A//}${do:end-if}${do:output-mode:html}${prox}${test:prox:\3A}${portprox}${test:prox:"}
${test:prox:--proxy "}${do:if-not-empty:prox}${test:proxytype::socks5:connect}${test:proxytype:\3A//}${do:end-if}${do:output-mode:html}${arg:prox}${test:prox:\3A}${arg:portprox}${test:prox:"}
${test:ftpprox:--httpproxy-ftp=0:--httpproxy-ftp}
</textarea>
@@ -211,7 +216,7 @@ ParseAll=${ztest:parseall:0:1}
HTMLFirst=${ztest:htmlfirst:0:1}
Cache=${ztest:cache:0:1}
NoRecatch=${ztest:norecatch:0:1}
Dos=${dos
Dos=${dos}
Index=${ztest:index:0:1}
WordIndex=${ztest:index2:0:1}
Log=${ztest:logf:0:1:2}
@@ -237,6 +242,10 @@ KeepSlashes=${ztest:keepslashes:0:1}
KeepQueryOrder=${ztest:keepqueryorder:0:1}
StripQuery=${stripquery}
StoreAllInCache=${ztest:cache2:0:1}
Warc=${ztest:warc:0:1}
WarcFile=${warcfile}
SingleFile=${ztest:singlefile:0:1}
SingleFileMaxSize=${singlefilemax}
LogType=${logtype}
UseHTTPProxyForFTP=${ztest:ftpprox:0:1}
ProxyType=${proxytype}

View File

@@ -713,7 +713,7 @@ Disconnect when finished
LANG_J17
Disconnect modem on completion
LANG_K1
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
LANG_K2
About WinHTTrack Website Copier
LANG_K3
@@ -1034,3 +1034,19 @@ LANG_STRIPQUERY
Strip query keys:
LANG_STRIPQUERYTIP
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
LANG_WARC
Write a WARC archive of the crawl
LANG_WARCTIP
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
LANG_WARCFILE
WARC archive name:
LANG_WARCFILETIP
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
LANG_SINGLEFILE
Inline assets as data: URIs (self-contained pages)
LANG_SINGLEFILETIP
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
LANG_SINGLEFILEMAX
Largest inlined asset (bytes):
LANG_SINGLEFILEMAXTIP
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Ïðåêðàòè âðúçêàòà ñëåä êðàÿ íà îïåðàöèÿòà
Disconnect modem on completion
Ñëåä êðàÿ íà îïåðàöèÿòà ðàçêà÷è ìîäåìà
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ìîëÿ óâåäîìåòå íè çà âñÿêà ãðåøêà èëè ïðîáëåì)\r\n\r\nÐàçðàáîòêà:\r\nÈíòåðôåéñ (Windows): Xavier Roche\r\nÏàÿê: Xavier Roche\r\nJava Ðàçáîðíè Êëàñîâå: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche è äðóãè ñúòðóäíèöè\r\nÌÍÎÃÎ ÁËÀÃÎÄÀÐÍÎÑÒÈ çà Áúëãàðñêèÿò ïðåâîä íà:\r\nÈëèÿ Ëèíäîâ (ilia@infomat-bg.com)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ìîëÿ óâåäîìåòå íè çà âñÿêà ãðåøêà èëè ïðîáëåì)\r\n\r\nÐàçðàáîòêà:\r\nÈíòåðôåéñ (Windows): Xavier Roche\r\nÏàÿê: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche è äðóãè ñúòðóäíèöè\r\nÌÍÎÃÎ ÁËÀÃÎÄÀÐÍÎÑÒÈ çà Áúëãàðñêèÿò ïðåâîä íà:\r\nÈëèÿ Ëèíäîâ (ilia@infomat-bg.com)
About WinHTTrack Website Copier
Çà WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Ïðåìàõâàíå íà êëþ÷îâå îò çàÿâêàòà:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Êëþ÷îâå îò çàÿâêàòà, ðàçäåëåíè ñúñ çàïåòàÿ, êîèòî äà ñå ïðåìàõíàò îò èìåòî íà çàïèñàíèÿ ôàéë (íàïðèìåð sid,utm_source).
Write a WARC archive of the crawl
Çàïèñâàíå íà WARC àðõèâ íà îáõîæäàíåòî
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Çàïèñâàíå íà âñåêè èçòåãëåí îòãîâîð è â WARC/1.1 àðõèâ ïî ISO-28500, äî îãëåäàëîòî.
WARC archive name:
Èìå íà WARC àðõèâà:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Íåçàäúëæèòåëíî áàçîâî èìå çà WARC àðõèâà; îñòàâåòå ïðàçíî çà àâòîìàòè÷íî èìåíóâàíå â èçõîäíàòà äèðåêòîðèÿ.
Inline assets as data: URIs (self-contained pages)
Âãðàæäàíå íà ðåñóðñèòå êàòî data: URI (ñàìîñòîÿòåëíè ñòðàíèöè)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Ñëåä çàâúðøâàíå íà îãëåäàëîòî âñÿêà çàïàçåíà ñòðàíèöà ñå ïðåçàïèñâà ñ âãðàäåíè ñòèëîâå, ñêðèïòîâå, èçîáðàæåíèÿ è øðèôòîâå, òàêà ÷å ñòðàíèöàòà äà ìîæå äà ñå îòâîðè è ñàìîñòîÿòåëíî.
Largest inlined asset (bytes):
Íàé-ãîëÿì âãðàäåí ðåñóðñ (áàéòîâå):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Ðåñóðñ íàä òîçè ðàçìåð çàïàçâà îáèêíîâåíà âðúçêà; îñòàâåòå ïðàçíî çà ñòîéíîñòòà ïî ïîäðàçáèðàíå îò 10485760 áàéòà.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Desconectar al terminar la operación
Disconnect modem on completion
Desconectar el modem al terminar
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(notifíquenos fallos o problemas)\r\n\r\nDesarrollo:\r\nInterface (Windows): Xavier Roche\r\nMotor: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Spanish translations to:\r\nJuan Pablo Barrio Lera (Universidad de León)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(notifíquenos fallos o problemas)\r\n\r\nDesarrollo:\r\nInterface (Windows): Xavier Roche\r\nMotor: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Spanish translations to:\r\nJuan Pablo Barrio Lera (Universidad de León)
About WinHTTrack Website Copier
Acerca de...
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Eliminar claves de query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Claves de query string, separadas por comas, que se eliminarán del nombre de los archivos guardados (p. ej. sid,utm_source).
Write a WARC archive of the crawl
Escribir un archivo WARC del rastreo
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Guardar también cada respuesta descargada en un archivo WARC/1.1 ISO-28500, junto a la réplica.
WARC archive name:
Nombre del archivo WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nombre base opcional para el archivo WARC; déjelo en blanco para nombrarlo automáticamente en el directorio de salida.
Inline assets as data: URIs (self-contained pages)
Incrustar los recursos como URI data: (páginas autónomas)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Una vez terminada la copia, reescribir cada página guardada con sus hojas de estilo, scripts, imágenes y fuentes incrustadas, de modo que una página también pueda abrirse por sí sola.
Largest inlined asset (bytes):
Tamaño máximo del recurso incrustado (bytes):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Un recurso mayor que este tamaño conserva un enlace normal; déjelo vacío para el valor predeterminado de 10485760 bytes.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Odpojit po dokonèení
Disconnect modem on completion
Odpojit modem po dokonèení
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Prosíme o podání hlášení o jakýchkoliv chybách)\r\n\r\nVývoj:\r\nRozhraní (Windows): Xavier Roche\r\nPavouk: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche a ostatní, kdo pøispìli\r\nÈeský pøeklad:\r\nAntonín Matìjèík (matejcik@volny.cz)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Prosíme o podání hlášení o jakýchkoliv chybách)\r\n\r\nVývoj:\r\nRozhraní (Windows): Xavier Roche\r\nPavouk: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche a ostatní, kdo pøispìli\r\nÈeský pøeklad:\r\nAntonín Matìjèík (matejcik@volny.cz)
About WinHTTrack Website Copier
O programu WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Odebrat klíèe dotazu:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Klíèe dotazu oddìlené èárkami, které se vynechají z pojmenování ukládaných souborù (napø. sid,utm_source).
Write a WARC archive of the crawl
Zapsat archiv WARC z procházení
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Uložit také každou staženou odpovìï do archivu WARC/1.1 podle ISO-28500 vedle zrcadla.
WARC archive name:
Název archivu WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Volitelný základní název archivu WARC; ponechte prázdné pro automatické pojmenování ve výstupním adresáøi.
Inline assets as data: URIs (self-contained pages)
Vložit zdroje jako URI data: (samostatné stránky)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Po dokonèení zrcadlení pøepsat každou uloženou stránku s vloženými styly, skripty, obrázky a písmy, aby ji bylo možné otevøít i samostatnì.
Largest inlined asset (bytes):
Nejvìtší vložený zdroj (bajty):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Zdroj vìtší než tato velikost si ponechá bìžný odkaz; ponechte prázdné pro výchozí hodnotu 10485760 bajtù.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
完成時切斷連線
Disconnect modem on completion
完成時切斷數據機
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(有任何程序錯誤或問題請聯絡我們)\r\n\r\n開發:\r\n界面設計 (Windows): Xavier Roche\r\n分析引擎: Xavier Roche\r\nJava分析引擎: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\n感謝:\r\nDavid Hing Cheong Hung (DAVEHUNG@mtr.com.hk) 提供翻譯
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(有任何程序錯誤或問題請聯絡我們)\r\n\r\n開發:\r\n界面設計 (Windows): Xavier Roche\r\n分析引擎: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\n感謝:\r\nDavid Hing Cheong Hung (DAVEHUNG@mtr.com.hk) 提供翻譯
About WinHTTrack Website Copier
關於WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
移除查詢鍵:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
以逗號分隔的查詢鍵,將其從儲存檔案的命名中移除 (例如 sid,utm_source)。
Write a WARC archive of the crawl
寫入此次抓取的 WARC 封存檔
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
同時將每個已擷取的回應儲存為 ISO-28500 WARC/1.1 封存檔,置於鏡像網站旁。
WARC archive name:
WARC 封存檔名稱:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
WARC 封存檔的選用基本名稱;留空則於輸出目錄中自動命名。
Inline assets as data: URIs (self-contained pages)
將資源內嵌為 data: URI獨立網頁
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
鏡射完成後,重寫每個已儲存的網頁,將樣式表、指令碼、圖片與字型內嵌其中,讓網頁也能單獨開啟。
Largest inlined asset (bytes):
內嵌資源大小上限(位元組):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
超過此大小的資源會保留一般連結;留空則使用預設的 10485760 位元組。

View File

@@ -634,8 +634,8 @@ Disconnect when finished
完成时断掉连接
Disconnect modem on completion
完成时断掉调制解调器
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(有任何程序错误或问题请联系我们)\r\n\r\n开发:\r\n界面设计 (Windows): Xavier Roche\r\n解析引擎: Xavier Roche\r\nJava解析引擎: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\n感谢:\r\nRobert Lagadec (rlagadec@yahoo.fr) 提供翻译技巧
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(有任何程序错误或问题请联系我们)\r\n\r\n开发:\r\n界面设计 (Windows): Xavier Roche\r\n解析引擎: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\n感谢:\r\nRobert Lagadec (rlagadec@yahoo.fr) 提供翻译技巧
About WinHTTrack Website Copier
关于WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
剥离查询键:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
用逗号分隔的查询键,将其从保存文件的命名中删除 (例如 sid,utm_source)。
Write a WARC archive of the crawl
写入本次抓取的 WARC 归档
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
同时将每个已获取的响应保存为 ISO-28500 WARC/1.1 归档,置于镜像站点旁边。
WARC archive name:
WARC 归档名称:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
WARC 归档的可选基本名称;留空则在输出目录中自动命名。
Inline assets as data: URIs (self-contained pages)
将资源内嵌为 data: URI独立网页
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
镜像完成后,重写每个已保存的网页,将样式表、脚本、图片和字体内嵌其中,使网页也能单独打开。
Largest inlined asset (bytes):
内嵌资源大小上限(字节):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
超过此大小的资源会保留普通链接;留空则使用默认的 10485760 字节。

View File

@@ -636,8 +636,8 @@ Disconnect when finished
Prekinuti vezu kada bude gotovo
Disconnect modem on completion
Po dovršetku odvojiti modemsku vezu
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Molimo da nas obavijestite o svim pogreškama i poteškoæama)\r\n\r\nRazvoj:\r\nSuèelje (Windows): Xavier Roche\r\nPauk: Xavier Roche\r\nRazrediRašèlanjivaèaJave: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche i drugi suradnici\r\nPUNO HVALA za prijevodne preporuke upuæujemo:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Molimo da nas obavijestite o svim pogreškama i poteškoæama)\r\n\r\nRazvoj:\r\nSuèelje (Windows): Xavier Roche\r\nPauk: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche i drugi suradnici\r\nPUNO HVALA za prijevodne preporuke upuæujemo:\r\nRobert Lagadec (rlagadec@yahoo.fr)
About WinHTTrack Website Copier
O programu WinHTTrack Website Copier
Please visit our Web page
@@ -958,3 +958,19 @@ Strip query keys:
Ukloniti kljuèeve upita:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Zarezom odvojeni kljuèevi upita koji se izostavljaju iz naziva spremljenih datoteka (npr. sid,utm_source).
Write a WARC archive of the crawl
Zapi¹i WARC arhivu obilaska
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Spremi i svaki preuzeti odgovor u ISO-28500 WARC/1.1 arhivu, uz zrcalo.
WARC archive name:
Naziv WARC arhive:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Neobavezni osnovni naziv WARC arhive; ostavite prazno za automatsko imenovanje u izlaznom direktoriju.
Inline assets as data: URIs (self-contained pages)
Ugradi resurse kao data: URI-je (samostalne stranice)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Nakon dovr¹etka zrcaljenja ponovno zapi¹i svaku spremljenu stranicu s ugraðenim stilskim listovima, skriptama, slikama i fontovima, tako da se stranica mo¾e otvoriti i zasebno.
Largest inlined asset (bytes):
Najveæi ugraðeni resurs (bajtovi):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Resurs veæi od ove velièine zadr¾ava obiènu poveznicu; ostavite prazno za zadanih 10485760 bajtova.

View File

@@ -652,8 +652,8 @@ Disconnect when finished
Afbryd forbindelsen når overførslen er færdig
Disconnect modem on completion
Afbryd modem når overførslen er færdig
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(finder du fejl eller opstår der problemer så kontakt os venligst)\r\n\r\nUdvikling:\r\nBrugerflade (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche og andre bidragydere\r\nMange tak for dansk oversættelse til:\r\nJesper Bramm (bramm@get2net.dk)\r\nscootergrisen
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(finder du fejl eller opstår der problemer så kontakt os venligst)\r\n\r\nUdvikling:\r\nBrugerflade (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche og andre bidragydere\r\nMange tak for dansk oversættelse til:\r\nJesper Bramm (bramm@get2net.dk)\r\nscootergrisen
About WinHTTrack Website Copier
Om WinHTTrack Website Copier
Please visit our Web page
@@ -1004,3 +1004,19 @@ Strip query keys:
Fjern forespørgselsnøgler:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommaseparerede forespørgselsnøgler, der udelades i navngivningen af gemte filer (f.eks. sid,utm_source).
Write a WARC archive of the crawl
Skriv et WARC-arkiv af gennemsøgningen
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Gem også hvert hentet svar i et ISO-28500 WARC/1.1-arkiv ved siden af spejlet.
WARC archive name:
Navn på WARC-arkiv:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Valgfrit basisnavn til WARC-arkivet; lad feltet stå tomt for automatisk navngivning i outputmappen.
Inline assets as data: URIs (self-contained pages)
Indlejr ressourcer som data:-URI'er (selvstændige sider)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Når spejlingen er færdig, omskrives hver gemt side med dens typografiark, scripts, billeder og skrifttyper indlejret, så en side også kan åbnes alene.
Largest inlined asset (bytes):
Største indlejrede ressource (byte):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
En ressource over denne størrelse beholder et almindeligt link; lad feltet stå tomt for standardværdien på 10485760 byte.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Nach der Aktion Verbindung trennen
Disconnect modem on completion
Modemverbindung trennen, wenn fertig
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Bitte benachrichtigen Sie uns über Fehler und Probleme)\r\n\r\nEntwicklung:\r\nOberfläche (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for German translations to:\r\nRainer Klueting (rainer@klueting.de)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Bitte benachrichtigen Sie uns über Fehler und Probleme)\r\n\r\nEntwicklung:\r\nOberfläche (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for German translations to:\r\nRainer Klueting (rainer@klueting.de)
About WinHTTrack Website Copier
Über WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Query-Schlüssel entfernen:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommagetrennte Query-Schlüssel, die bei der Benennung gespeicherter Dateien entfallen (z. B. sid,utm_source).
Write a WARC archive of the crawl
WARC-Archiv des Crawls schreiben
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Jede heruntergeladene Antwort zusätzlich in einem ISO-28500-WARC/1.1-Archiv neben dem Spiegel speichern.
WARC archive name:
Name des WARC-Archivs:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Optionaler Basisname für das WARC-Archiv; leer lassen, um es automatisch im Ausgabeverzeichnis zu benennen.
Inline assets as data: URIs (self-contained pages)
Ressourcen als data:-URIs einbetten (eigenständige Seiten)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Nach Abschluss der Spiegelung jede gespeicherte Seite mit eingebetteten Stylesheets, Skripten, Bildern und Schriftarten neu schreiben, sodass eine Seite auch für sich allein geöffnet werden kann.
Largest inlined asset (bytes):
Größte eingebettete Ressource (Bytes):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Eine Ressource über dieser Größe behält einen gewöhnlichen Link; leer lassen für den Standardwert von 10485760 Bytes.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Lahuta ühendus, kui on lõpetatud
Disconnect modem on completion
Lahuta modem lõpetamisel
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Palun teata meile igast veast või probleemist)\r\n\r\nArendajad:\r\nKasutajaliides (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nEestikeelne tõlge: Tõnu Virma
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Palun teata meile igast veast või probleemist)\r\n\r\nArendajad:\r\nKasutajaliides (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nEestikeelne tõlge: Tõnu Virma
About WinHTTrack Website Copier
Info WinHTTrack Website Copier'i kohta
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Eemalda päringuvõtmed:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Komadega eraldatud päringuvõtmed, mis jäetakse salvestatud faili nimest välja (nt sid,utm_source).
Write a WARC archive of the crawl
Kirjuta läbimise WARC-arhiiv
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Salvesta iga alla laaditud vastus ka ISO-28500 WARC/1.1 arhiivi peegli kõrvale.
WARC archive name:
WARC-arhiivi nimi:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
WARC-arhiivi valikuline põhinimi; jäta tühjaks, et see väljundkataloogis automaatselt nimetada.
Inline assets as data: URIs (self-contained pages)
Manusta ressursid data: URI-dena (iseseisvad lehed)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Pärast peegelduse lõppu kirjuta iga salvestatud leht ümber, manustades selle laaditabelid, skriptid, pildid ja fondid, nii et lehte saab avada ka eraldi.
Largest inlined asset (bytes):
Suurim manustatud ressurss (baiti):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Sellest suurem ressurss säilitab tavalise lingi; jäta tühjaks vaikeväärtuse 10485760 baiti jaoks.

View File

@@ -652,8 +652,8 @@ Disconnect when finished
Disconnect when finished
Disconnect modem on completion
Disconnect modem on completion
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
About WinHTTrack Website Copier
About WinHTTrack Website Copier
Please visit our Web page
@@ -1004,3 +1004,19 @@ Strip query keys:
Strip query keys:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Write a WARC archive of the crawl
Write a WARC archive of the crawl
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
WARC archive name:
WARC archive name:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Inline assets as data: URIs (self-contained pages)
Inline assets as data: URIs (self-contained pages)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Largest inlined asset (bytes):
Largest inlined asset (bytes):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.

View File

@@ -636,8 +636,8 @@ Disconnect when finished
Katkaise yhteys, kun on valmista
Disconnect modem on completion
Katkaise modeemiyhteys, kun on valmista
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Kerro meille bugeista ja ongelmista)\r\n\r\nKehitys:\r\nKäyttöliittymä (Windows): Xavier Roche\r\nNettirobotti: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C) 1998-2003 Xavier Roche ja muut avustajat\r\nSuomentanut Mika Kähkönen 22.-24.7.2005\r\nmika.kahkonen@mbnet.fi\r\nhttp://koti.mbnet.fi/kahoset
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Kerro meille bugeista ja ongelmista)\r\n\r\nKehitys:\r\nKäyttöliittymä (Windows): Xavier Roche\r\nNettirobotti: Xavier Roche\r\n\r\n(C) 1998-2003 Xavier Roche ja muut avustajat\r\nSuomentanut Mika Kähkönen 22.-24.7.2005\r\nmika.kahkonen@mbnet.fi\r\nhttp://koti.mbnet.fi/kahoset
About WinHTTrack Website Copier
Tietoja WinHTTrack Website Copier
Please visit our Web page
@@ -958,3 +958,19 @@ Strip query keys:
Poista kyselyavaimet:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Pilkuin erotellut kyselyavaimet, jotka jätetään pois tallennettujen tiedostojen nimeämisestä (esim. sid,utm_source).
Write a WARC archive of the crawl
Kirjoita imuroinnin WARC-arkisto
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Tallenna myös jokainen noudettu vastaus ISO-28500 WARC/1.1 -arkistoon peilin viereen.
WARC archive name:
WARC-arkiston nimi:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Valinnainen WARC-arkiston perusnimi; jätä tyhjäksi, jotta se nimetään automaattisesti tulostehakemistoon.
Inline assets as data: URIs (self-contained pages)
Upota resurssit data:-URI-osoitteina (itsenäiset sivut)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Kun peilaus on valmis, kirjoita jokainen tallennettu sivu uudelleen niin, että sen tyylitiedostot, komentosarjat, kuvat ja fontit upotetaan, jolloin sivun voi avata myös yksinään.
Largest inlined asset (bytes):
Suurin upotettu resurssi (tavua):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Tätä suurempi resurssi säilyttää tavallisen linkin; jätä tyhjäksi, jolloin käytetään oletusarvoa 10485760 tavua.

View File

@@ -644,8 +644,8 @@ Disconnect when finished
Déconnecter à la fin de l'opération
Disconnect modem on completion
Déconnecter le modem lorsque l'opération sera finie
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(signalez-nous tout bug ou problème)\r\n\r\nDéveloppement:\r\nInterface (Windows): Xavier Roche\r\nMoteur: Xavier Roche\r\nParseurClassesJava: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMERCI pour la relecture à:\r\nRobert Lagadec
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(signalez-nous tout bug ou problème)\r\n\r\nDéveloppement:\r\nInterface (Windows): Xavier Roche\r\nMoteur: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMERCI pour la relecture à:\r\nRobert Lagadec
About WinHTTrack Website Copier
A propos de WinHTTrack Website Copier
Please visit our Web page
@@ -1004,3 +1004,19 @@ Strip query keys:
Supprimer les clés de query string :
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Clés de query string à retirer du nommage des fichiers enregistrés, séparées par des virgules (par ex. sid,utm_source).
Write a WARC archive of the crawl
Écrire une archive WARC du crawl
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Enregistrer aussi chaque réponse téléchargée dans une archive WARC/1.1 (ISO-28500), à côté du miroir.
WARC archive name:
Nom de l'archive WARC :
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nom de base optionnel pour l'archive WARC ; laissez vide pour le générer automatiquement dans le répertoire de sortie.
Inline assets as data: URIs (self-contained pages)
Intégrer les ressources en URI data: (pages autonomes)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Une fois le miroir terminé, réécrire chaque page enregistrée avec ses feuilles de style, scripts, images et polices intégrés, afin qu'une page puisse aussi être ouverte seule.
Largest inlined asset (bytes):
Taille maximale d'une ressource intégrée (octets) :
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Au-delà de cette taille, la ressource reste un lien ordinaire ; laissez vide pour la valeur par défaut de 10485760 octets.

View File

@@ -636,8 +636,8 @@ Disconnect when finished
Αποσύνδεση στο τέλος
Disconnect modem on completion
Αποσύνδεση του modem στο τέλος
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Παρακαλώ να μας ενημερώσετε για κάθε πρόβλημα ή bug)\r\n\r\nΑνάπτυξη:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Greek translations to:\r\nMichael Papadakis (mikepap at freemail dot gr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Παρακαλώ να μας ενημερώσετε για κάθε πρόβλημα ή bug)\r\n\r\nΑνάπτυξη:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Greek translations to:\r\nMichael Papadakis (mikepap at freemail dot gr)
About WinHTTrack Website Copier
Σχετικά με το WinHTTrack Website Copier
Please visit our Web page
@@ -958,3 +958,19 @@ Strip query keys:
Αφαίρεση κλειδιών ερωτήματος:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Κλειδιά ερωτήματος χωρισμένα με κόμμα, που θα αφαιρεθούν από την ονομασία των αποθηκευμένων αρχείων (π.χ. sid,utm_source).
Write a WARC archive of the crawl
Εγγραφή αρχείου WARC της ανίχνευσης
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Αποθήκευση κάθε ληφθείσας απόκρισης και σε αρχείο WARC/1.1 ISO-28500, δίπλα στο είδωλο.
WARC archive name:
Όνομα αρχείου WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Προαιρετικό βασικό όνομα για το αρχείο WARC. Αφήστε το κενό για αυτόματη ονομασία στον κατάλογο εξόδου.
Inline assets as data: URIs (self-contained pages)
Ενσωμάτωση των πόρων ως URI data: (αυτόνομες σελίδες)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Μόλις ολοκληρωθεί το αντίγραφο, κάθε αποθηκευμένη σελίδα ξαναγράφεται με ενσωματωμένα τα φύλλα στυλ, τα σενάρια, τις εικόνες και τις γραμματοσειρές της, ώστε η σελίδα να μπορεί να ανοίξει και μόνη της.
Largest inlined asset (bytes):
Μέγιστος ενσωματωμένος πόρος (byte):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Πόρος μεγαλύτερος από αυτό το μέγεθος διατηρεί κανονικό σύνδεσμο. Αφήστε το κενό για την προεπιλογή των 10485760 byte.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Disconnetti alla fine
Disconnect modem on completion
Disconnetti il modem quando il mirror è finito
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(informateci degli errori o problemi)\r\n\rSviluppo:\r\nInterfacccia: Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Italian translations to:\r\nWitold Krakowski (wkrakowski@libero.it)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(informateci degli errori o problemi)\r\n\rSviluppo:\r\nInterfacccia: Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Italian translations to:\r\nWitold Krakowski (wkrakowski@libero.it)
About WinHTTrack Website Copier
Informazioni su WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Rimuovi chiavi della query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Chiavi della query string, separate da virgole, da rimuovere dai nomi dei file salvati (ad es. sid,utm_source).
Write a WARC archive of the crawl
Scrivi un archivio WARC della scansione
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Salva anche ogni risposta scaricata in un archivio WARC/1.1 ISO-28500, accanto al mirror.
WARC archive name:
Nome dell'archivio WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nome di base facoltativo per l'archivio WARC; lascia vuoto per assegnarlo automaticamente nella directory di output.
Inline assets as data: URIs (self-contained pages)
Incorpora le risorse come URI data: (pagine autonome)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Al termine della copia, riscrive ogni pagina salvata incorporando fogli di stile, script, immagini e caratteri, in modo che una pagina possa essere aperta anche da sola.
Largest inlined asset (bytes):
Dimensione massima della risorsa incorporata (byte):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Una risorsa oltre questa dimensione mantiene un collegamento normale; lasciare vuoto per il valore predefinito di 10485760 byte.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
終了したら接続を切断する
Disconnect modem on completion
完了したらモデムとの接続を切断する
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(バグまたは問題についてわれわれに知らせてください)\r\n\r\nDevelopment:\r\nInterface(Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nJapanese translation :TAPKAL(nakataka@mars.dti.ne.jp)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(バグまたは問題についてわれわれに知らせてください)\r\n\r\nDevelopment:\r\nInterface(Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nJapanese translation :TAPKAL(nakataka@mars.dti.ne.jp)
About WinHTTrack Website Copier
WinHTTrackについて
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
削除するクエリキー:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
保存ファイル名の生成から除外するクエリキーをカンマ区切りで指定します (例: sid,utm_source)。
Write a WARC archive of the crawl
クロールの WARC アーカイブを書き出す
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
取得した各レスポンスを ISO-28500 WARC/1.1 アーカイブとしてミラーの隣にも保存します。
WARC archive name:
WARC アーカイブ名:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
WARC アーカイブの任意のベース名。空欄にすると出力ディレクトリ内で自動的に名前が付けられます。
Inline assets as data: URIs (self-contained pages)
リソースを data: URI として埋め込む (単独で開けるページ)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
ミラーの完了後、保存された各ページを、スタイルシート、スクリプト、画像、フォントを埋め込んだ形で書き直します。ページ単体でも開けるようになります。
Largest inlined asset (bytes):
埋め込む最大サイズ (バイト):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
このサイズを超えるリソースは通常のリンクのままになります。空欄にすると既定値の 10485760 バイトになります。

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Äèñêîíåêòèð༠ñå êîãà <20>å çàâðøè
Disconnect modem on completion
Äèñêîíåêòèð༠ãî ìîäåìîò íà êðà¼
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\Âå ìîëèìå èçâåñòåòå íå çà áèëî êàêâà ãðåøêà èëè ïðîáëåì)\r\n\r\nDevelopment:\r\nÈíòåðôå¼ñ (Windows):Xavier Roche\r\nSpider:Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche è äðóãè\r\nÌÍÎÃÓ ÁËÀÃÎÄÀÐÍÎÑÒ çà ìàêåäîíñêèîò ïðåâîä íà:\r\nÀëåêñàíäàð Ñàâè<C3A2> (aleks@macedonia.eu.org)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\Âå ìîëèìå èçâåñòåòå íå çà áèëî êàêâà ãðåøêà èëè ïðîáëåì)\r\n\r\nDevelopment:\r\nÈíòåðôå¼ñ (Windows):Xavier Roche\r\nSpider:Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche è äðóãè\r\nÌÍÎÃÓ ÁËÀÃÎÄÀÐÍÎÑÒ çà ìàêåäîíñêèîò ïðåâîä íà:\r\nÀëåêñàíäàð Ñàâè<C3A2> (aleks@macedonia.eu.org)
About WinHTTrack Website Copier
Çà WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
¾âáâàÐÝØ ÚÛãçÕÒØ ÞÔ ÑÐàÐúÕâÞ:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
ºÛãçÕÒØ ÞÔ ÑÐàÐúÕâÞ, ÞÔÔÕÛÕÝØ áÞ ×ÐߨàÚÐ, èâÞ áÕ ÞâáâàÐÝãÒÐÐâ ÞÔ ØÜÕâÞ ÝÐ ×ÐçãÒÐÝÐâÐ ÔÐâÞâÕÚÐ (ÝÐ ßàØÜÕà sid,utm_source).
Write a WARC archive of the crawl
·ÐßØèØ WARC ÐàåØÒÐ ÝÐ ßàÕÑÐàãÒÐúÕâÞ
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
·ÐçãÒÐø ÓÞ áÕÚÞø ßàÕ×ÕÜÕÝ ÞÔÓÞÒÞà Ø ÒÞ ISO-28500 WARC/1.1 ÐàåØÒÐ, ßÞÚàÐø ÞÓÛÕÔÐÛÞâÞ.
WARC archive name:
¸ÜÕ ÝÐ WARC ÐàåØÒÐâÐ:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
¸×ÑÞàÝÞ ÞáÝÞÒÝÞ ØÜÕ ×Ð WARC ÐàåØÒÐâÐ; ÞáâÐÒÕâÕ ßàÐ×ÝÞ ×Ð ÐÒâÞÜÐâáÚÞ ØÜÕÝãÒÐúÕ ÒÞ Ø×ÛÕ×ÝØÞâ ÔØàÕÚâÞàØãÜ.
Inline assets as data: URIs (self-contained pages)
²ÓàÐÔØ ÓØ àÕáãàáØâÕ ÚÐÚÞ data: URI (áÐÜÞáâÞøÝØ áâàÐÝØæØ)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
¿Þ ×ÐÒàèãÒÐúÕ ÝÐ ÞÓÛÕÔÐÛÞâÞ, áÕÚÞøÐ ×ÐçãÒÐÝÐ áâàÐÝØæÐ áÕ ßàÕߨèãÒÐ áÞ ÒÓàÐÔÕÝØ áâØÛáÚØ ÛØáâÞÒØ, áÚàØßâØ, áÛØÚØ Ø äÞÝâÞÒØ, ×Ð ÔÐ ÜÞÖÕ áâàÐÝØæÐâÐ ÔÐ áÕ ÞâÒÞàØ Ø áÐÜÞáâÞøÝÞ.
Largest inlined asset (bytes):
½ÐøÓÞÛÕÜ ÒÓàÐÔÕÝ àÕáãàá (ÑÐøâØ):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
ÀÕáãàá ßÞÓÞÛÕÜ ÞÔ ÞÒÐÐ ÓÞÛÕÜØÝÐ ×ÐÔàÖãÒÐ ÞÑØçÝÐ ÒàáÚÐ; ÞáâÐÒÕâÕ ßàÐ×ÝÞ ×Ð áâÐÝÔÐàÔÝØâÕ 10485760 ÑÐøâØ.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Vonalbontás, ha kész
Disconnect modem on completion
Modem leválasztása, ha kész
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Kérjük, jelezzen nekünk bármilyen hibát vagy problémát)\r\n\r\nFejlesztés:\r\nKezelõfelület (Windows): Xavier Roche\r\nIndexelés: Xavier Roche\r\nJavaElemzõOsztályok: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nEZER KÖSZÖNET a magyar fordításért:\r\nHerczeg József Tamásnak (hdodi@freemail.hu)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Kérjük, jelezzen nekünk bármilyen hibát vagy problémát)\r\n\r\nFejlesztés:\r\nKezelõfelület (Windows): Xavier Roche\r\nIndexelés: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nEZER KÖSZÖNET a magyar fordításért:\r\nHerczeg József Tamásnak (hdodi@freemail.hu)
About WinHTTrack Website Copier
WinHTTrack webhely másoló névjegye
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Lekérdezési kulcsok eltávolítása:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Vesszõvel elválasztott lekérdezési kulcsok, amelyeket el kell hagyni a mentett fájl elnevezésébõl (pl. sid,utm_source).
Write a WARC archive of the crawl
A bejárás WARC archívumának írása
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Minden letöltött válasz mentése ISO-28500 WARC/1.1 archívumba is, a tükör mellé.
WARC archive name:
WARC archívum neve:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
A WARC archívum opcionális alapneve; hagyja üresen az automatikus elnevezéshez a kimeneti könyvtárban.
Inline assets as data: URIs (self-contained pages)
Erõforrások beágyazása data: URI-ként (önálló oldalak)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
A tükrözés befejezése után minden mentett oldal újraírása a stíluslapok, parancsfájlok, képek és betûtípusok beágyazásával, így az oldal önmagában is megnyitható.
Largest inlined asset (bytes):
Legnagyobb beágyazott erõforrás (bájt):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Az ennél nagyobb erõforrás közönséges hivatkozás marad; hagyja üresen a 10485760 bájtos alapértelmezéshez.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Indien gedaan verbinding verbreken
Disconnect modem on completion
Indien gedaan modemverbinding verbreken
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ons fouten en problemen mede te delen)\r\n\r\nOntwikkeling:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Dutch translations to:\r\nRudi Ferrari (Wyando@netcologne.de)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ons fouten en problemen mede te delen)\r\n\r\nOntwikkeling:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Dutch translations to:\r\nRudi Ferrari (Wyando@netcologne.de)
About WinHTTrack Website Copier
Over WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Query-sleutels verwijderen:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Door komma's gescheiden query-sleutels die bij het benoemen van opgeslagen bestanden worden weggelaten (bijv. sid,utm_source).
Write a WARC archive of the crawl
Schrijf een WARC-archief van de crawl
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Sla ook elke opgehaalde respons op in een ISO-28500 WARC/1.1-archief, naast de mirror.
WARC archive name:
Naam van WARC-archief:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Optionele basisnaam voor het WARC-archief; laat leeg om het automatisch een naam te geven in de uitvoermap.
Inline assets as data: URIs (self-contained pages)
Bronnen insluiten als data:-URI's (op zichzelf staande pagina's)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Zodra de spiegeling klaar is, elke opgeslagen pagina herschrijven met ingesloten stijlbladen, scripts, afbeeldingen en lettertypen, zodat een pagina ook los geopend kan worden.
Largest inlined asset (bytes):
Grootste ingesloten bron (bytes):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Een bron boven deze grootte houdt een gewone koppeling; laat leeg voor de standaardwaarde van 10485760 bytes.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Koble fra når fullført
Disconnect modem on completion
Koble fra modem når fullført
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Vennligst fortell oss om feil eller problemer med programmet)\r\n\r\nUtvikling (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche og andre bidragsytere\r\n\r\nNorwegian translation:\r\nTobias "Spug" Langhoff ( spug_enigma@hotmail.com )
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Vennligst fortell oss om feil eller problemer med programmet)\r\n\r\nUtvikling (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche og andre bidragsytere\r\n\r\nNorwegian translation:\r\nTobias "Spug" Langhoff ( spug_enigma@hotmail.com )
About WinHTTrack Website Copier
Om WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Fjern spørrenøkler:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommaseparerte spørrenøkler som utelates i navngivingen av lagrede filer (f.eks. sid,utm_source).
Write a WARC archive of the crawl
Skriv et WARC-arkiv av gjennomgangen
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Lagre også hvert nedlastet svar i et ISO-28500 WARC/1.1-arkiv ved siden av speilet.
WARC archive name:
Navn på WARC-arkiv:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Valgfritt basisnavn for WARC-arkivet; la feltet stå tomt for automatisk navngivning i utdatamappen.
Inline assets as data: URIs (self-contained pages)
Bygg inn ressurser som data:-URI-er (selvstendige sider)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Når speilingen er ferdig, skrives hver lagrede side om med stilark, skript, bilder og skrifter innebygd, slik at en side også kan åpnes alene.
Largest inlined asset (bytes):
Største innebygde ressurs (byte):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
En ressurs over denne størrelsen beholder en vanlig lenke; la feltet stå tomt for standardverdien på 10485760 byte.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Zakoñcz po³¹czenie z us³ugodawc¹ po pobraniu
Disconnect modem on completion
Po pobraniu roz³¹cz modem
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(proszê poinformowaæ nas o jakichkolwiek b³êdach w dzia³aniu programu)\r\n\r\nTwórcy:\r\nInterfejs(Windows): Xavier Roche\r\nMotor: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nPolska wersja jêzykowa: £ukasz Jokiel (Opole University of Technology, Lukasz.Jokiel@po.opole.pl)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(proszê poinformowaæ nas o jakichkolwiek b³êdach w dzia³aniu programu)\r\n\r\nTwórcy:\r\nInterfejs(Windows): Xavier Roche\r\nMotor: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nPolska wersja jêzykowa: £ukasz Jokiel (Opole University of Technology, Lukasz.Jokiel@po.opole.pl)
About WinHTTrack Website Copier
O... WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Usuñ klucze zapytania:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Rozdzielone przecinkami klucze zapytania pomijane przy nazywaniu zapisanych plików (np. sid,utm_source).
Write a WARC archive of the crawl
Zapisz archiwum WARC z indeksowania
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Zapisz te¿ ka¿d± pobran± odpowied¼ do archiwum WARC/1.1 ISO-28500, obok kopii lustrzanej.
WARC archive name:
Nazwa archiwum WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Opcjonalna nazwa bazowa archiwum WARC; pozostaw puste, aby nazwaæ je automatycznie w katalogu wyj¶ciowym.
Inline assets as data: URIs (self-contained pages)
Osad¼ zasoby jako identyfikatory data: URI (samodzielne strony)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Po zakoñczeniu tworzenia kopii przepisz ka¿d± zapisan± stronê z osadzonymi arkuszami stylów, skryptami, obrazami i czcionkami, aby stronê mo¿na by³o otworzyæ równie¿ samodzielnie.
Largest inlined asset (bytes):
Najwiêkszy osadzony zasób (bajty):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Zasób wiêkszy ni¿ ten rozmiar zachowuje zwyk³y odno¶nik; pozostaw puste, aby u¿yæ domy¶lnych 10485760 bajtów.

View File

@@ -652,8 +652,8 @@ Disconnect when finished
Desconectar ao finalizar
Disconnect modem on completion
Desconectar o modem ao finalizar
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(por favor nos comunique sobre erros ou problemas)\r\n\r\nDesenvolvimento:\r\nInterface (Windows): Xavier Roche\r\nMotor: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche e outros colaboradores\r\nTraduzido para o Português-Brasil por :\r\nPaulo Neto (layoutbr@lexxa.com.br)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(por favor nos comunique sobre erros ou problemas)\r\n\r\nDesenvolvimento:\r\nInterface (Windows): Xavier Roche\r\nMotor: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche e outros colaboradores\r\nTraduzido para o Português-Brasil por :\r\nPaulo Neto (layoutbr@lexxa.com.br)
About WinHTTrack Website Copier
Sobre o WinHTTrack Website Copier
Please visit our Web page
@@ -1004,3 +1004,19 @@ Strip query keys:
Remover chaves da query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Chaves da query string, separadas por vírgulas, a serem removidas da nomeação dos arquivos salvos (ex.: sid,utm_source).
Write a WARC archive of the crawl
Gravar um arquivo WARC do rastreamento
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Salvar também cada resposta baixada em um arquivo WARC/1.1 ISO-28500, ao lado do espelho.
WARC archive name:
Nome do arquivo WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nome base opcional para o arquivo WARC; deixe em branco para nomeá-lo automaticamente no diretório de saída.
Inline assets as data: URIs (self-contained pages)
Incorporar os recursos como URIs data: (páginas autônomas)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Ao terminar o espelhamento, reescrever cada página salva com suas folhas de estilo, scripts, imagens e fontes incorporadas, para que a página também possa ser aberta sozinha.
Largest inlined asset (bytes):
Maior recurso incorporado (bytes):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Um recurso acima desse tamanho mantém um link comum; deixe em branco para o padrão de 10485760 bytes.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Desligar no fim da operação
Disconnect modem on completion
Desconectar o modem no final
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Por favor avise-nos acerca de erros ou problemas)\r\n\r\nDesenvolvimento:\r\nInterface (Windows): Xavier Roche\r\nMotor: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nAgradecimentos pela tradução para o Português para:\r\nRui Fernandes (CANTIC, ruiefe@mail.malhatlantica.pt)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Por favor avise-nos acerca de erros ou problemas)\r\n\r\nDesenvolvimento:\r\nInterface (Windows): Xavier Roche\r\nMotor: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nAgradecimentos pela tradução para o Português para:\r\nRui Fernandes (CANTIC, ruiefe@mail.malhatlantica.pt)
About WinHTTrack Website Copier
Acerca do WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Remover chaves da query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Chaves da query string, separadas por vírgulas, a remover da nomeação dos ficheiros guardados (por ex. sid,utm_source).
Write a WARC archive of the crawl
Escrever um arquivo WARC do rastreio
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Guardar também cada resposta transferida num arquivo WARC/1.1 ISO-28500, ao lado do espelho.
WARC archive name:
Nome do arquivo WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nome base opcional para o arquivo WARC; deixe em branco para o nomear automaticamente no diretório de saída.
Inline assets as data: URIs (self-contained pages)
Incorporar os recursos como URI data: (páginas autónomas)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Depois de terminado o espelho, reescrever cada página guardada com as suas folhas de estilo, scripts, imagens e tipos de letra incorporados, para que a página também possa ser aberta sozinha.
Largest inlined asset (bytes):
Maior recurso incorporado (bytes):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Um recurso acima deste tamanho mantém uma ligação normal; deixe em branco para o valor predefinido de 10485760 bytes.

View File

@@ -634,7 +634,7 @@ Disconnect when finished
Deconectează şa terminare
Disconnect modem on completion
Deconectează modemul la terminare
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
About WinHTTrack Website Copier
Despre WinHTTrack Website Copier
@@ -956,3 +956,19 @@ Strip query keys:
Elimina cheile din query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Chei din query string, separate prin virgula, de eliminat din denumirea fisierelor salvate (de ex. sid,utm_source).
Write a WARC archive of the crawl
Scrie o arhiva WARC a parcurgerii
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Salveaza si fiecare raspuns descarcat intr-o arhiva WARC/1.1 ISO-28500, langa oglinda.
WARC archive name:
Numele arhivei WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Nume de baza optional pentru arhiva WARC; lasati gol pentru a-l denumi automat in directorul de iesire.
Inline assets as data: URIs (self-contained pages)
Încorporeaza resursele ca URI data: (pagini de sine statatoare)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Dupa terminarea copiei, fiecare pagina salvata este rescrisa cu foile de stil, scripturile, imaginile si fonturile încorporate, astfel încât pagina sa poata fi deschisa si singura.
Largest inlined asset (bytes):
Cea mai mare resursa încorporata (octeti):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
O resursa mai mare decât aceasta dimensiune pastreaza o legatura obisnuita; lasati gol pentru valoarea implicita de 10485760 de octeti.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Îòñîåäèíèòüñÿ ïðè çàâåðøåíèè
Disconnect modem on completion
Îòñîåäèíèòü ïðè çàâåðøåíèè
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ñîîáùèòå íàì, ïîæàëóéñòà, î çàìå÷åííûõ ïðîáëåìàõ è îøèáêàõ)\r\n\r\nÐàçðàáîòêà:\r\nÈíòåðôåéñ (Windows): Xavier Roche\r\nÊà÷àëêà (spider): Xavier Roche\r\nÏàðñåð ÿâà-êëàññîâ: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Russian translations to:\r\nAndrei Iliev (andreiiliev@mail.ru)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ñîîáùèòå íàì, ïîæàëóéñòà, î çàìå÷åííûõ ïðîáëåìàõ è îøèáêàõ)\r\n\r\nÐàçðàáîòêà:\r\nÈíòåðôåéñ (Windows): Xavier Roche\r\nÊà÷àëêà (spider): Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Russian translations to:\r\nAndrei Iliev (andreiiliev@mail.ru)
About WinHTTrack Website Copier
Î ïðîãðàììå WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Óäàëÿòü êëþ÷è çàïðîñà:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Êëþ÷è çàïðîñà ÷åðåç çàïÿòóþ, óäàëÿåìûå èç èìåíè ñîõðàíÿåìîãî ôàéëà (íàïðèìåð, sid,utm_source).
Write a WARC archive of the crawl
Çàïèñàòü WARC-àðõèâ îáõîäà
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Ñîõðàíÿòü êàæäûé çàãðóæåííûé îòâåò òàêæå â àðõèâ WARC/1.1 ISO-28500 ðÿäîì ñ çåðêàëîì.
WARC archive name:
Èìÿ WARC-àðõèâà:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Íåîáÿçàòåëüíîå áàçîâîå èìÿ WARC-àðõèâà; îñòàâüòå ïóñòûì äëÿ àâòîìàòè÷åñêîãî èìåíîâàíèÿ â âûõîäíîì êàòàëîãå.
Inline assets as data: URIs (self-contained pages)
Âñòðàèâàòü ðåñóðñû êàê data: URI (ñàìîñòîÿòåëüíûå ñòðàíèöû)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Ïîñëå çàâåðøåíèÿ çåðêàëèðîâàíèÿ êàæäàÿ ñîõðàí¸ííàÿ ñòðàíèöà ïåðåçàïèñûâàåòñÿ ñî âñòðîåííûìè òàáëèöàìè ñòèëåé, ñöåíàðèÿìè, èçîáðàæåíèÿìè è øðèôòàìè, ÷òîáû ñòðàíèöó ìîæíî áûëî îòêðûòü îòäåëüíî.
Largest inlined asset (bytes):
Íàèáîëüøèé âñòðàèâàåìûé ðåñóðñ (áàéòû):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Ðåñóðñ áîëüøå ýòîãî ðàçìåðà ñîõðàíÿåò îáû÷íóþ ññûëêó; îñòàâüòå ïóñòûì äëÿ çíà÷åíèÿ ïî óìîë÷àíèþ 10485760 áàéò.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Odpoji<EFBFBD>, ak je kopírovanie ukonèené
Disconnect modem on completion
Odpoji<EFBFBD> modem po dokonèení
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Oznám nám akúko¾vek chybu alebo problém)\r\n\r\nVývoj:\r\nProstredie (Windows): Xavier Roche\r\nPavúk: Xavier Roche\r\nKontrolór Javy: Yann Philippot\r\n\r\n(C)1998-2001 Xavier Roche a ostatní pomocníci\r\nVE¼KÉ POÏAKOVANIE za preklady:\r\nDr. Martin Sereday (sereday@slovanet.sk)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Oznám nám akúko¾vek chybu alebo problém)\r\n\r\nVývoj:\r\nProstredie (Windows): Xavier Roche\r\nPavúk: Xavier Roche\r\n\r\n(C)1998-2001 Xavier Roche a ostatní pomocníci\r\nVE¼KÉ POÏAKOVANIE za preklady:\r\nDr. Martin Sereday (sereday@slovanet.sk)
About WinHTTrack Website Copier
O WinHTTrack Website Copier...
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Odstráni» kµúèe dotazu:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kµúèe dotazu oddelené èiarkami, ktoré sa vynechajú z pomenovania ukladaných súborov (napr. sid,utm_source).
Write a WARC archive of the crawl
Zapísa» archív WARC z prehµadávania
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Ulo¾i» aj ka¾dú stiahnutú odpoveï do archívu WARC/1.1 ISO-28500 vedµa zrkadla.
WARC archive name:
Názov archívu WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Voliteµný základný názov archívu WARC; ponechajte prázdne pre automatické pomenovanie vo výstupnom adresári.
Inline assets as data: URIs (self-contained pages)
Vlo¾i» zdroje ako URI data: (samostatné stránky)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Po dokonèení zrkadlenia prepísa» ka¾dú ulo¾enú stránku s vlo¾enými ¹týlmi, skriptmi, obrázkami a písmami, aby sa stránka dala otvori» aj samostatne.
Largest inlined asset (bytes):
Najväè¹í vlo¾ený zdroj (bajty):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Zdroj väè¹í ne¾ táto veµkos» si ponechá be¾ný odkaz; ponechajte prázdne pre predvolených 10485760 bajtov.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Prekini povezavo, ko bo konèano
Disconnect modem on completion
Izkljuèi modem ob dokonèanju
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Prosimo Vas, da nas obvestite o kakršnih koli težavah ali napakah)\r\n\r\nRazvoj:\r\nVmesnik (Okna): Xavier Roche\r\nHitrost: Xavier Roche\r\nJavaParserRazredi: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche in drugi sodelujoèi\r\nVELIKA ZAHVALA za namige prevodov gre:\r\nRobertu Lagadecu (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Prosimo Vas, da nas obvestite o kakršnih koli težavah ali napakah)\r\n\r\nRazvoj:\r\nVmesnik (Okna): Xavier Roche\r\nHitrost: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche in drugi sodelujoèi\r\nVELIKA ZAHVALA za namige prevodov gre:\r\nRobertu Lagadecu (rlagadec@yahoo.fr)
About WinHTTrack Website Copier
O programu WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Odstrani kljuce poizvedbe:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Z vejicami loceni kljuci poizvedbe, ki se izpustijo pri poimenovanju shranjenih datotek (npr. sid,utm_source).
Write a WARC archive of the crawl
Zapisi arhiv WARC iz pregledovanja
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Shrani tudi vsak preneseni odgovor v arhiv WARC/1.1 ISO-28500 poleg zrcala.
WARC archive name:
Ime arhiva WARC:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Neobvezno osnovno ime arhiva WARC; pustite prazno za samodejno poimenovanje v izhodni mapi.
Inline assets as data: URIs (self-contained pages)
Vgradi vire kot data: URI (samostojne strani)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Ko je zrcaljenje koncano, se vsaka shranjena stran prepise z vgrajenimi slogovnimi listi, skripti, slikami in pisavami, tako da jo je mogoce odpreti tudi samostojno.
Largest inlined asset (bytes):
Najvecji vgrajeni vir (bajti):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Vir, vecji od te velikosti, ohrani obicajno povezavo; pustite prazno za privzetih 10485760 bajtov.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Koppla ner förbindelsen när överföringen är klar
Disconnect modem on completion
Koppla ned modemet efter avslutning
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Hittar du fel eller uppstår det problem, kontakta oss)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Swedish translations to:\r\nStaffan Ström (staffan@fam-strom.org)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Hittar du fel eller uppstår det problem, kontakta oss)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Swedish translations to:\r\nStaffan Ström (staffan@fam-strom.org)
About WinHTTrack Website Copier
Om WinHTTrack Website Copier...
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Ta bort frågenycklar:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommaseparerade frågenycklar som utelämnas vid namngivningen av sparade filer (t.ex. sid,utm_source).
Write a WARC archive of the crawl
Skriv ett WARC-arkiv av genomsökningen
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Spara även varje hämtat svar i ett ISO-28500 WARC/1.1-arkiv, bredvid spegeln.
WARC archive name:
WARC-arkivets namn:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Valfritt basnamn för WARC-arkivet; lämna tomt för att namnge det automatiskt i utdatakatalogen.
Inline assets as data: URIs (self-contained pages)
Bädda in resurser som data:-URI:er (fristående sidor)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
När speglingen är klar skrivs varje sparad sida om med sina formatmallar, skript, bilder och teckensnitt inbäddade, så att en sida också kan öppnas för sig.
Largest inlined asset (bytes):
Största inbäddade resurs (byte):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
En resurs över den här storleken behåller en vanlig länk; lämna tomt för standardvärdet 10485760 byte.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Bittiðinde baðlantýyý kes
Disconnect modem on completion
Ýþlem tamamlandýðýnda modemden baðlantýyý kopar
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Lütfen bize her türlü hatayý ve problemi aktarýnýz)\r\n\r\nGeliþtirme:\r\nArayüz (Windows): Xavier Roche\r\nAð: Xavier Roche\r\nJava Ayýklama Sýnýflarý: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche ve diðerleri\r\nÇeviri ipuçlarý için teþekkürler: \r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Lütfen bize her türlü hatayý ve problemi aktarýnýz)\r\n\r\nGeliþtirme:\r\nArayüz (Windows): Xavier Roche\r\nAð: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche ve diðerleri\r\nÇeviri ipuçlarý için teþekkürler: \r\nRobert Lagadec (rlagadec@yahoo.fr)
About WinHTTrack Website Copier
WinHTTrack Website Copier Hakkýnda
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Sorgu anahtarlarýný çýkar:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kaydedilen dosya adlandýrmasýndan çýkarýlacak, virgülle ayrýlmýþ sorgu anahtarlarý (örn. sid,utm_source).
Write a WARC archive of the crawl
Taramanýn WARC arþivini yaz
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Ýndirilen her yanýtý ayrýca aynanýn yanýna bir ISO-28500 WARC/1.1 arþivine kaydet.
WARC archive name:
WARC arþivi adý:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
WARC arþivi için isteðe baðlý temel ad; çýktý dizininde otomatik adlandýrma için boþ býrakýn.
Inline assets as data: URIs (self-contained pages)
Kaynaklarý data: URI olarak göm (kendi kendine yeten sayfalar)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Yansýlama bittiðinde, kaydedilen her sayfa stil sayfalarý, betikleri, görselleri ve yazý tipleri gömülü olarak yeniden yazýlýr; böylece sayfa tek baþýna da açýlabilir.
Largest inlined asset (bytes):
En büyük gömülü kaynak (bayt):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Bu boyutun üzerindeki bir kaynak sýradan baðlantýsýný korur; 10485760 baytlýk varsayýlan için boþ býrakýn.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
³ä'ºäíàòèñü ïðè çàâåðøåíí³
Disconnect modem on completion
³ä'ºäíàòè ïðè çàâåðøåíí³
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ïîâ³äîìèòå íàì áóäü ëàñêà ïðî ïîì³÷åí³ ïðîáëåìè ³ ïîìèëêè)\r\n\r\nðàçðàáîòêà:\r\nèíòåðôåéñ (Windows): Xavier Roche\r\nêà÷àëêà (spider): Xavier Roche\r\nïàðñåð java-êëàñ³â: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Ukrainian translations to:\r\nAndrij Shevchuk (andrijsh@mail.lviv.ua) http://programy.com.ua, http://vic-info.com.ua
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ïîâ³äîìèòå íàì áóäü ëàñêà ïðî ïîì³÷åí³ ïðîáëåìè ³ ïîìèëêè)\r\n\r\nðàçðàáîòêà:\r\nèíòåðôåéñ (Windows): Xavier Roche\r\nêà÷àëêà (spider): Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Ukrainian translations to:\r\nAndrij Shevchuk (andrijsh@mail.lviv.ua) http://programy.com.ua, http://vic-info.com.ua
About WinHTTrack Website Copier
Ïðî ïðîãðàìó WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Âèëó÷àòè êëþ÷³ çàïèòó:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Êëþ÷³ çàïèòó ÷åðåç êîìó, ÿê³ âèëó÷àþòüñÿ ç ³ìåí³ çáåðåæåíîãî ôàéëó (íàïðèêëàä, sid,utm_source).
Write a WARC archive of the crawl
Çàïèñàòè WARC-àðõ³â îáõîäó
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Çáåð³ãàòè êîæíó çàâàíòàæåíó â³äïîâ³äü òàêîæ ó àðõ³â WARC/1.1 ISO-28500 ïîðÿä ³ç äçåðêàëîì.
WARC archive name:
²ì'ÿ WARC-àðõ³âó:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
Íåîáîâ'ÿçêîâà áàçîâà íàçâà WARC-àðõ³âó; çàëèøòå ïîðîæí³ì äëÿ àâòîìàòè÷íîãî íàéìåíóâàííÿ ó âèõ³äíîìó êàòàëîç³.
Inline assets as data: URIs (self-contained pages)
Âáóäîâóâàòè ðåñóðñè ÿê data: URI (ñàìîñò³éí³ ñòîð³íêè)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
ϳñëÿ çàâåðøåííÿ äçåðêàëþâàííÿ êîæíà çáåðåæåíà ñòîð³íêà ïåðåçàïèñóºòüñÿ ç âáóäîâàíèìè òàáëèöÿìè ñòèë³â, ñöåíàð³ÿìè, çîáðàæåííÿìè òà øðèôòàìè, ùîá ñòîð³íêó ìîæíà áóëî â³äêðèòè îêðåìî.
Largest inlined asset (bytes):
Íàéá³ëüøèé âáóäîâàíèé ðåñóðñ (áàéòè):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Ðåñóðñ, á³ëüøèé çà öåé ðîçì³ð, çáåð³ãຠçâè÷àéíå ïîñèëàííÿ; çàëèøòå ïîðîæí³ì äëÿ òèïîâîãî çíà÷åííÿ 10485760 áàéò³â.

View File

@@ -634,8 +634,8 @@ Disconnect when finished
Îòñîåäèíèòüñÿ ïðè çàâåðøåíèè
Disconnect modem on completion
Îòñîåäåíèòü ïðè çàâåðøåíèè
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\nJavaParserClasses: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ñîîáùèòå íàì ïîæàëóéñòà î çàìå÷åííûõ ïðîáëåìàõ è îøèáêàõ)\r\n\r\nÐàçðàáîòêà:\r\nÈíòåðôåéñ (Windows): Xavier Roche\r\nÊà÷àëêà (spider): Xavier Roche\r\nÏàðñåð ÿâà-êëàññîâ: Yann Philippot\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Russian translations to:\r\nAndrei Iliev (andreiiliev@mail.ru)
\r\n(Please notify us of any bug or problem)\r\n\r\nDevelopment:\r\nInterface (Windows): Xavier Roche\r\nSpider: Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for translation tips to:\r\nRobert Lagadec (rlagadec@yahoo.fr)
\r\n(Ñîîáùèòå íàì ïîæàëóéñòà î çàìå÷åííûõ ïðîáëåìàõ è îøèáêàõ)\r\n\r\nÐàçðàáîòêà:\r\nÈíòåðôåéñ (Windows): Xavier Roche\r\nÊà÷àëêà (spider): Xavier Roche\r\n\r\n(C)1998-2003 Xavier Roche and other contributors\r\nMANY THANKS for Russian translations to:\r\nAndrei Iliev (andreiiliev@mail.ru)
About WinHTTrack Website Copier
Î ïðîãðàììå WinHTTrack Website Copier
Please visit our Web page
@@ -956,3 +956,19 @@ Strip query keys:
Olib tashlanadigan sorov kalitlari:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Saqlangan fayl nomidan olib tashlanadigan, vergul bilan ajratilgan sorov kalitlari (masalan, sid,utm_source).
Write a WARC archive of the crawl
Qidiruvning WARC arxivini yozish
Also save every fetched response into an ISO-28500 WARC/1.1 archive, next to the mirror.
Har bir yuklab olingan javobni ISO-28500 WARC/1.1 arxiviga ham, ko'zgu yonida saqlash.
WARC archive name:
WARC arxivi nomi:
Optional base name for the WARC archive; leave blank to auto-name it under the output directory.
WARC arxivi uchun ixtiyoriy asosiy nom; chiqish katalogida avtomatik nomlash uchun bo'sh qoldiring.
Inline assets as data: URIs (self-contained pages)
Resurslarni data: URI sifatida joylash (mustaqil sahifalar)
Once the mirror is finished, rewrite every saved page with its stylesheets, scripts, images and fonts embedded, so a page can also be opened on its own.
Nusxalash tugagach, har bir saqlangan sahifa uslublar jadvallari, skriptlar, rasmlar va shriftlar joylangan holda qayta yoziladi, shunda sahifani alohida ham ochish mumkin.
Largest inlined asset (bytes):
Eng katta joylangan resurs (bayt):
An asset above this size keeps an ordinary link; leave blank for the 10485760-byte default.
Bu olchamdan katta resurs oddiy havolani saqlab qoladi; standart 10485760 bayt uchun bosh qoldiring.

View File

@@ -3,7 +3,7 @@
.\"
.\" This file is generated by man/makeman.sh; do not edit by hand.
.\" SPDX-License-Identifier: GPL-3.0-or-later
.TH httrack 1 "22 July 2026" "httrack website copier"
.TH httrack 1 "26 July 2026" "httrack website copier"
.SH NAME
httrack \- offline browser : copy websites to a local directory
.SH SYNOPSIS
@@ -40,6 +40,7 @@ httrack \- offline browser : copy websites to a local directory
[ \fB\-%N, \-\-delayed\-type\-check\fR ]
[ \fB\-%D, \-\-cached\-delayed\-type\-check\fR ]
[ \fB\-%M, \-\-mime\-html\fR ]
[ \fB\-%Z, \-\-single\-file\fR ]
[ \fB\-LN, \-\-long\-names[=N]\fR ]
[ \fB\-KN, \-\-keep\-links[=N]\fR ]
[ \fB\-x, \-\-replace\-external\fR ]
@@ -58,6 +59,7 @@ httrack \- offline browser : copy websites to a local directory
[ \fB\-sN, \-\-robots[=N]\fR ]
[ \fB\-%h, \-\-http\-10\fR ]
[ \fB\-%k, \-\-keep\-alive\fR ]
[ \fB\-%z, \-\-disable\-compression\fR ]
[ \fB\-%B, \-\-tolerant\fR ]
[ \fB\-%s, \-\-updatehack\fR ]
[ \fB\-%u, \-\-urlhack\fR ]
@@ -73,6 +75,7 @@ httrack \- offline browser : copy websites to a local directory
[ \fB\-%X, \-\-headers\fR ]
[ \fB\-C, \-\-cache[=N]\fR ]
[ \fB\-k, \-\-store\-all\-in\-cache\fR ]
[ \fB\-%r, \-\-warc\fR ]
[ \fB\-%n, \-\-do\-not\-recatch\fR ]
[ \fB\-%v, \-\-display\fR ]
[ \fB\-Q, \-\-do\-not\-log\fR ]
@@ -98,6 +101,7 @@ httrack \- offline browser : copy websites to a local directory
[ \fB\-%!, \-\-disable\-security\-limits\fR ]
[ \fB\-V, \-\-userdef\-cmd\fR ]
[ \fB\-%W, \-\-callback\fR ]
[ \fB\-y, \-\-background\-on\-suspend\fR ]
[ \fB\-K, \-\-keep\-links[=N]\fR ]
.SH DESCRIPTION
.B httrack
@@ -195,6 +199,10 @@ delayed type check, don't make any link test but wait for files download to star
cached delayed type check, don't wait for remote type during updates, to speedup them (%D0 wait, * %D1 don't wait) (\-\-cached\-delayed\-type\-check)
.IP \-%M
generate a RFC MIME\-encapsulated full\-archive (.mht) (\-\-mime\-html)
.IP \-%Z
after the mirror, rewrite each saved page with its stylesheets, scripts, images and fonts inlined as data: URIs, so any page opens by double\-click anywhere (links between pages stay relative; audio and video stay links); \-\-single\-file\-max\-size N caps each asset (default 10485760 bytes). %M is the better container where a Chromium\-family browser is a given: one archive, no base64 tax on text, a shared asset stored once (\-\-single\-file)
.IP \-%t
keep the original file extension, don't rewrite it from the MIME type (%t0 rewrite)
.IP \-LN
long names (L1 *long names / L0 8\-3 conversion / L2 ISO9660 compatible) (\-\-long\-names[=N])
.IP \-KN
@@ -232,6 +240,8 @@ follow robots.txt and meta robots tags (0=never,1=sometimes,* 2=always, 3=always
force HTTP/1.0 requests (reduce update features, only for old servers or proxies) (\-\-http\-10)
.IP \-%k
use keep\-alive if possible, greately reducing latency for small files and test requests (%k0 don't use) (\-\-keep\-alive)
.IP \-%z
do not request compressed content (%z0 request) (\-\-disable\-compression)
.IP \-%B
tolerant requests (accept bogus responses on some servers, but not standard!) (\-\-tolerant)
.IP \-%s
@@ -258,7 +268,7 @@ default referer field sent in HTTP headers (\-\-referer <param>)
.IP \-%E
from email address sent in HTTP headers (\-\-from <param>)
.IP \-%F
footer string in Html code (\-%F "Mirrored [from host %s [file %s [at %s]]]" (\-\-footer <param>)
footer string in Html code (\-%F "Mirrored from {url} on {date}"; fields {addr} {path} {url} {date} {lastmodified} {version} {mime} {charset} {status} {size}, or legacy %s) (\-\-footer <param>)
.IP \-%l
preferred language (\-%l "fr, en, jp, *" (\-\-language <param>)
.IP \-%a
@@ -270,6 +280,8 @@ additional HTTP header line (\-%X "X\-Magic: 42" (\-\-headers <param>)
create/use a cache for updates and retries (C0 no cache,C1 cache is prioritary,* C2 test update before) (\-\-cache[=N])
.IP \-k
store all files in cache (not useful if files on disk) (\-\-store\-all\-in\-cache)
.IP \-%r
write an ISO\-28500 WARC/1.1 archive; \-\-warc\-file NAME sets the output name, \-\-warc\-max\-size N rotates segments past N bytes, \-\-warc\-cdx also writes a sorted CDXJ index, \-\-wacz packages it all as a WACZ file (\-\-warc)
.IP \-%n
do not re\-download locally erased files (\-\-do\-not\-recatch)
.IP \-%v
@@ -326,8 +338,6 @@ go everywhere on the web (\-\-go\-everywhere)
.IP \-%H
debug HTTP headers in logfile (\-\-debug\-headers)
.SS Guru options: (do NOT use if possible)
.IP \-#X
*use optimized engine (limited memory boundary checks) (\-\-fast\-engine)
.IP \-#test
list engine self\-tests (run one with \-#test=NAME [args])
.IP \-#C
@@ -352,8 +362,6 @@ maximum number of links (\-#L1000000) (\-\-advanced\-maxlinks[=N])
display ugly progress information (\-\-advanced\-progressinfo)
.IP \-#P
catch URL (\-\-catch\-url)
.IP \-#R
old FTP routines (debug) (\-\-repair\-cache)
.IP \-#T
generate transfer ops. log every minutes (\-\-debug\-xfrstats)
.IP \-#u
@@ -372,6 +380,8 @@ USE IT WITH EXTREME CARE
execute system command after each files ($0 is the filename: \-V "rm \\$0") (\-\-userdef\-cmd <param>)
.IP \-%W
use an external library function as a wrapper (\-%W myfoo.so[,myparameters]) (\-\-callback <param>)
.IP \-y
go to background when suspended (y0 don't) (\-\-background\-on\-suspend)
.SS Details: Option N
.IP \-N0
Site\-structure (default)

View File

@@ -32,7 +32,9 @@ AM_LDFLAGS = \
bin_PROGRAMS = proxytrack httrack htsserver
httrack_LDADD = $(THREADS_LIBS) libhttrack.la
httrack_SOURCES = httrack.c htsbacktrace.c htsbacktrace.h
# $(DL_LIBS): dladdr() in the crash handler, still in libdl on pre-2.34 glibc.
httrack_LDADD = $(THREADS_LIBS) $(DL_LIBS) libhttrack.la
htsserver_LDADD = $(THREADS_LIBS) $(SOCKET_LIBS) libhttrack.la
proxytrack_LDADD = $(THREADS_LIBS) $(SOCKET_LIBS)
@@ -47,6 +49,7 @@ htsserver_LDFLAGS = $(AM_LDFLAGS) $(LDFLAGS_PIE)
lib_LTLIBRARIES = libhttrack.la
htsserver_SOURCES = htsserver.c htsserver.h htsweb.c htsweb.h \
htscmdline.c htscmdline.h \
htsurlport.c htsurlport.h
proxytrack_SOURCES = proxy/main.c \
proxy/proxytrack.c proxy/store.c \
@@ -60,21 +63,21 @@ whttrackrun_SCRIPTS = webhttrack
libhttrack_la_SOURCES = htscore.c htsparse.c htsback.c htscache.c \
htscache_selftest.c htsdns_selftest.c htsselftest.c \
htscatchurl.c htsfilters.c htsftp.c htshash.c coucal/coucal.c \
htshelp.c htslib.c htsurlport.c htscoremain.c \
htscmdline.c htshelp.c htslib.c htsurlport.c htscoremain.c \
htsname.c htsrobots.c htstools.c htswizard.c \
htsalias.c htsthread.c htsindex.c htsbauth.c \
htsmd5.c htscodec.c htsproxy.c htszlib.c htswrap.c htsconcat.c \
htsmd5.c htscodec.c htswarc.c htssinglefile.c htsproxy.c htszlib.c htswrap.c htsconcat.c \
htsmodules.c htscharset.c punycode.c htsencoding.c htssniff.c \
md5.c \
minizip/ioapi.c minizip/mztools.c minizip/unzip.c minizip/zip.c \
hts-indextmpl.h htsalias.h htsback.h htsbase.h htssafe.h \
htsbasenet.h htsbauth.h htscache.h htscache_selftest.h htsdns_selftest.h htsselftest.h htscatchurl.h \
htsconfig.h htscore.h htsparse.h htscoremain.h htsdefines.h \
htscmdline.h htsconfig.h htscore.h htsparse.h htscoremain.h htsdefines.h \
htsfilters.h htsftp.h htsglobal.h htshash.h coucal/coucal.h \
htshelp.h htsindex.h htslib.h htsurlport.h htsmd5.h \
htsmodules.h htsname.h htsnet.h htssniff.h \
htsopt.h htsrobots.h htsthread.h \
htstools.h htswizard.h htswrap.h htscodec.h htsproxy.h htszlib.h \
htstools.h htswizard.h htswrap.h htscodec.h htswarc.h htssinglefile.h htsproxy.h htszlib.h \
htsstrings.h htsarrays.h httrack-library.h \
htscharset.h punycode.h htsencoding.h \
htsentities.h htsentities.sh htsbasiccharsets.sh htscodepages.h \

View File

@@ -114,6 +114,19 @@ const char *hts_optalias[][4] = {
"strip [host/pattern=]key1,key2,... from URLs"},
{"cookies-file", "-%K", "param1",
"load extra cookies from a Netscape cookies.txt"},
{"warc", "-%r", "single", "write an ISO-28500 WARC/1.1 archive of the crawl"},
{"warc-file", "-%rf", "param1", "write a WARC archive to the given base name"},
{"warc-max-size", "-%rs", "param1",
"rotate the WARC archive once a segment passes N bytes (0: single file)"},
{"warc-cdx", "-%rc", "single",
"write a sorted CDXJ index next to the WARC archive"},
{"warc-cdxj", "-%rc", "single", ""},
{"wacz", "-%rz", "single",
"package the WARC archive, CDXJ index and pages as a WACZ file"},
{"single-file", "-%Z", "single",
"after the mirror, inline each page's assets as data: URIs"},
{"single-file-max-size", "-%Zs", "param1",
"per-asset cap for --single-file, in bytes (implies it; default 10485760)"},
{"why", "-%Y", "param1",
"explain which filter rule accepts or rejects a URL, then exit"},
{"pause", "-%G", "param1",

View File

@@ -37,6 +37,7 @@ Please visit our Website: http://www.httrack.com
/* specific definitions */
#include "htsnet.h"
#include "htscore.h"
#include "htswarc.h"
#include "htsthread.h"
#include <time.h>
/* END specific definitions */
@@ -124,6 +125,22 @@ void back_free(struct_back ** sback) {
above a normal handshake. The last candidate still gets the full timeout. */
#define HTS_CONNECT_FALLBACK_TIMEOUT 10
void back_read_ftp_result(FILE *fp, htsblk *r) {
size_t j = 0;
if (fscanf(fp, "%d ", &r->statuscode) != 1)
r->statuscode = STATUSCODE_INVALID;
// an external helper writes this file: stop at capacity, not at EOF
while (j + 1 < sizeof(r->msg)) {
const int c = fgetc(fp);
if (c == EOF)
break;
r->msg[j++] = (char) c;
}
r->msg[j] = '\0';
}
int back_connect_fallback_due(int addr_index, int addr_count, int elapsed,
int timeout) {
int deadline;
@@ -540,9 +557,15 @@ static int create_back_tmpfile(httrackp *opt, lien_back *const back,
const char *ext) {
// do not use tempnam() but a regular filename
back->tmpfile_buffer[0] = '\0';
if (back->url_sav != NULL && back->url_sav[0] != '\0') {
snprintf(back->tmpfile_buffer, sizeof(back->tmpfile_buffer), "%s.%s",
back->url_sav, ext);
if (back->url_sav[0] != '\0') {
/* same capacity as url_sav, so truncation drops the extension and aliases
the temp name onto the live file that back_finalize_backup() UNLINKs */
if (!sprintfbuff(back->tmpfile_buffer, "%s.%s", back->url_sav, ext)) {
hts_log_print(opt, LOG_WARNING, "temporary filename too long for %s",
back->url_sav);
back->tmpfile_buffer[0] = '\0';
return -1;
}
back->tmpfile = back->tmpfile_buffer;
if (structcheck(back->tmpfile) != 0) {
hts_log_print(opt, LOG_WARNING, "can not create directory to %s",
@@ -550,8 +573,15 @@ static int create_back_tmpfile(httrackp *opt, lien_back *const back,
return -1;
}
} else {
snprintf(back->tmpfile_buffer, sizeof(back->tmpfile_buffer), "%s/tmp%d.%s",
StringBuff(opt->path_html_utf8), opt->state.tmpnameid++, ext);
/* truncation here would collide distinct tmpnameid's onto one name */
if (!sprintfbuff(back->tmpfile_buffer, "%s/tmp%d.%s",
StringBuff(opt->path_html_utf8), opt->state.tmpnameid++,
ext)) {
hts_log_print(opt, LOG_WARNING, "temporary filename too long in %s",
StringBuff(opt->path_html_utf8));
back->tmpfile_buffer[0] = '\0';
return -1;
}
back->tmpfile = back->tmpfile_buffer;
}
/* OK */
@@ -749,9 +779,20 @@ int back_finalize(httrackp * opt, cache_back * cache, struct_back * sback,
fexist_utf8(back[p].url_sav))
filenote(&opt->state.strc, back[p].url_sav, NULL);
}
/* Keep the compressed spool so the WARC record stores the body
verbatim (Content-Encoding preserved) instead of unlinking it.
*/
if (StringNotEmpty(opt->warc_file)) {
warc_adopt_rawspool(&back[p].r, back[p].tmpfile);
if (back[p].r.warc_rawpath != NULL)
back[p].tmpfile =
NULL; /* adopted: freed via warc_free_request */
}
/* ensure that no remaining temporary file exists */
unlink(back[p].tmpfile);
back[p].tmpfile = NULL;
if (back[p].tmpfile != NULL) {
unlink(back[p].tmpfile);
back[p].tmpfile = NULL;
}
}
// stats
HTS_STAT.total_packed += back[p].compressed_size;
@@ -875,8 +916,7 @@ int back_finalize(httrackp * opt, cache_back * cache, struct_back * sback,
HTS_STAT.stat_bytes += back[p].r.size;
HTS_STAT.stat_files++;
hts_log_print(opt, LOG_TRACE, "added file %s%s => %s",
back[p].url_adr, back[p].url_fil,
back[p].url_sav != NULL ? back[p].url_sav : "");
back[p].url_adr, back[p].url_fil, back[p].url_sav);
}
if ((!back[p].r.notmodified) && (opt->is_update)) {
HTS_STAT.stat_updated_files++; // page modifiée
@@ -979,6 +1019,10 @@ int back_finalize(httrackp * opt, cache_back * cache, struct_back * sback,
// status finished callback
RUN_CALLBACK1(opt, xfrstatus, &back[p]);
// WARC archive of the transaction (request + response/revisit)
if (StringNotEmpty(opt->warc_file))
warc_write_backtransaction(opt, &back[p]);
return 0;
} else { // testmode
if (back[p].r.statuscode / 100 >= 3) { /* Store 3XX, 4XX, 5XX test response codes, but NOT 2XX */
@@ -1055,6 +1099,11 @@ void back_copy_static(const lien_back * src, lien_back * dst) {
dst->r.soc = INVALID_SOCKET;
dst->r.adr = NULL;
dst->r.headers = NULL;
dst->r.warc_reqhdr = NULL;
dst->r.warc_resphdr = NULL;
dst->r.warc_rawpath =
NULL; /* the spool stays owned by src (no double-unlink) */
dst->r.warc_truncated = 0;
dst->r.out = NULL;
dst->r.location = dst->location_buffer;
dst->r.fp = NULL;
@@ -1118,6 +1167,10 @@ int back_unserialize(FILE * fp, lien_back ** dst) {
(*dst)->chunk_adr = NULL;
(*dst)->r.adr = NULL;
(*dst)->r.out = NULL;
(*dst)->r.warc_reqhdr = NULL;
(*dst)->r.warc_resphdr = NULL;
(*dst)->r.warc_rawpath = NULL;
(*dst)->r.warc_truncated = 0;
(*dst)->r.location = (*dst)->location_buffer;
(*dst)->r.fp = NULL;
(*dst)->r.soc = INVALID_SOCKET;
@@ -1585,6 +1638,7 @@ int back_clear_entry(lien_back * back) {
freet(back->r.headers);
back->r.headers = NULL;
}
warc_free_request(&back->r);
// Tout nettoyer
memset(back, 0, sizeof(lien_back));
back->r.soc = INVALID_SOCKET;
@@ -2276,26 +2330,24 @@ int back_add(struct_back *sback, httrackp *opt, cache_back *cache,
&& slot_can_be_finalized(opt, &back[i]);
int may_serialize = slot_can_be_cached_on_disk(&back[i]);
hts_log_print(opt, LOG_DEBUG,
"back[%03d]: may_clean=%d, may_finalize_disk=%d, may_serialize=%d:"
LF "\t"
"finalized(%d), status(%d), locked(%d), delayed(%d), test(%d), "
LF "\t"
"statuscode(%d), size(%d), is_write(%d), may_hypertext(%d), "
LF "\t" "contenttype(%s), url(%s%s), save(%s)", i,
may_clean, may_finalize, may_serialize,
back[i].finalized, back[i].status, back[i].locked,
IS_DELAYED_EXT(back[i].url_sav), back[i].testmode,
back[i].r.statuscode, (int) back[i].r.size,
back[i].r.is_write, may_be_hypertext_mime(opt,
back[i].r.
contenttype,
back[i].
url_fil),
/* */
back[i].r.contenttype, back[i].url_adr,
back[i].url_fil,
back[i].url_sav ? back[i].url_sav : "<null>");
hts_log_print(
opt, LOG_DEBUG,
"back[%03d]: may_clean=%d, may_finalize_disk=%d, "
"may_serialize=%d:" LF "\t"
"finalized(%d), status(%d), locked(%d), delayed(%d), "
"test(%d), " LF "\t"
"statuscode(%d), size(%d), is_write(%d), may_hypertext(%d), " LF
"\t"
"contenttype(%s), url(%s%s), save(%s)",
i, may_clean, may_finalize, may_serialize, back[i].finalized,
back[i].status, back[i].locked, IS_DELAYED_EXT(back[i].url_sav),
back[i].testmode, back[i].r.statuscode, (int) back[i].r.size,
back[i].r.is_write,
may_be_hypertext_mime(opt, back[i].r.contenttype,
back[i].url_fil),
/* */
back[i].r.contenttype, back[i].url_adr, back[i].url_fil,
back[i].url_sav);
}
}
}
@@ -2509,6 +2561,19 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
for (i = 0; i < (unsigned int) back_max; i++) {
if (back[i].status > 0 && back[i].status < STATUS_FTP_TRANSFER) {
/* A cap-truncated body is deliberate, not broken: archive what arrived
with WARC-Truncated before the abort overwrites the slot's real 2xx
status. HTTrack still treats the slot as incomplete afterwards. */
if (StringNotEmpty(opt->warc_file) && back[i].r.statuscode > 0 &&
back[i].r.warc_resphdr != NULL && back[i].r.size > 0 &&
!(back[i].r.is_write && IS_DELAYED_EXT(back[i].url_sav))) {
if (back[i].r.is_write && back[i].r.out != NULL)
fflush(back[i].r.out);
back[i].r.warc_truncated = (limit == HTS_MIRROR_LIMIT_SIZE)
? WARC_TRUNC_LENGTH
: WARC_TRUNC_TIME;
warc_write_backtransaction(opt, &back[i]);
}
if (back[i].r.soc != INVALID_SOCKET) {
deletehttp(&back[i].r);
}
@@ -2818,7 +2883,8 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
// new session
back[i].r.ssl_con = SSL_new(openssl_ctx);
if (back[i].r.ssl_con) {
const char* hostname = jump_protocol_const(back[i].url_adr);
/* non-const twin: the OpenSSL macro casts the qualifier away */
char *hostname = jump_protocol(back[i].url_adr);
// some servers expect the hostname on the clienthello (SNI TLS extension)
SSL_set_tlsext_host_name(back[i].r.ssl_con, hostname);
SSL_clear(back[i].r.ssl_con);
@@ -2910,7 +2976,7 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
back[i].r.msg[0] = '\0';
strncatbuff(back[i].r.msg, tmp, sizeof(back[i].r.msg) - 2);
if (!strnotempty(back[i].r.msg)) {
sprintf(back[i].r.msg, "SSL/TLS error %d", err_code);
htsblk_failf(&back[i].r, "SSL/TLS error %d", err_code);
}
deletehttp(&back[i].r);
back[i].r.soc = INVALID_SOCKET;
@@ -2983,16 +3049,7 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
FOPEN(fconcat(OPT_GET_BUFF(opt), back[i].location_buffer, ".ok"),
"rb");
if (fp) {
int j = 0;
fscanf(fp, "%d ", &(back[i].r.statuscode));
while(!feof(fp)) {
int c = fgetc(fp);
if (c != EOF)
back[i].r.msg[j++] = c;
}
back[i].r.msg[j++] = '\0';
back_read_ftp_result(fp, &back[i].r);
fclose(fp);
UNLINK(fconcat(OPT_GET_BUFF(opt), back[i].location_buffer, ".ok"));
strcpybuff(fconcat
@@ -3293,10 +3350,11 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
deleteaddr(&back[i].r);
if (back[i].r.size < back[i].r.totalsize)
back[i].r.statuscode = STATUSCODE_CONNERROR; // recatch
sprintf(back[i].r.msg,
"Incorrect length (" LLintP " Bytes, " LLintP
" expected)", (LLint) back[i].r.size,
(LLint) back[i].r.totalsize);
htsblk_failf(&back[i].r,
"Incorrect length (" LLintP " Bytes, " LLintP
" expected)",
(LLint) back[i].r.size,
(LLint) back[i].r.totalsize);
} else {
// Un warning suffira..
hts_log_print(opt, LOG_WARNING,
@@ -3631,6 +3689,10 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
deleteaddr(&back[i].r);
back[i].r.headers = block;
}
// Stash the raw response headers for WARC (deletehttp frees
// r.headers when the socket closes, before back_finalize)
if (StringNotEmpty(opt->warc_file))
warc_stash_response(&back[i].r, back[i].r.headers);
/*
Status code and header-response hacks

View File

@@ -74,6 +74,10 @@ void back_free(struct_back ** sback);
// backing
#define BACK_ADD_TEST "(dummy)"
#define BACK_ADD_TEST2 "(dummy2)"
/* Parse an external FTP helper's "<statuscode> <message>" result file into r,
clipping the message to r->msg. */
void back_read_ftp_result(FILE *fp, htsblk *r);
int back_index(httrackp * opt, struct_back * sback, const char *adr, const char *fil,
const char *sav);
int back_available(const struct_back * sback);

238
src/htsbacktrace.c Normal file
View File

@@ -0,0 +1,238 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* File: crash backtrace printer */
/* Author: Xavier Roche */
/* ------------------------------------------------------------ */
/* Before every header: glibc gates dladdr() on it. */
#if defined(__linux) && !defined(_GNU_SOURCE)
#define _GNU_SOURCE
#endif
#include "htsbacktrace.h"
#include "htsglobal.h"
#include <stdlib.h>
#include <string.h>
#ifdef _WIN32
#include <io.h> /* write */
#endif
#ifdef HAVE_UNISTD_H
#include <unistd.h>
#endif
#if (defined(__linux) && defined(HAVE_EXECINFO_H))
#include <dlfcn.h>
#include <errno.h>
#include <execinfo.h>
#include <signal.h>
#include <time.h>
#include <sys/wait.h>
#define USES_BACKTRACE
#endif
#ifdef USES_BACKTRACE
#define BT_MAX_FRAMES 64 /* frames we try to name */
#define BT_MAX_MODULES 8 /* distinct modules, one child each */
#define BT_HEX_SIZE 19 /* "0x" + 16 nibbles + NUL */
#define BT_PATH_SIZE 1024 /* module path; longer is skipped */
#define BT_WAIT_TICKS 300 /* 10ms ticks, shared: cap a slow child */
#define BT_NO_SYMBOLIZER 127 /* child exit: execvp() found none */
static hts_boolean symbolize_crash = HTS_TRUE;
/* "0x"-prefixed hex: the handler must stay stdio-free. */
static void print_hex(char *buffer, uintptr_t value) {
static const char digits[] = "0123456789abcdef";
size_t i = 2, a, b;
buffer[0] = '0';
buffer[1] = 'x';
do {
buffer[i++] = digits[value & 0xf];
value >>= 4;
} while (value != 0);
buffer[i] = '\0';
for (a = 2, b = i - 1; a < b; a++, b--) {
const char c = buffer[a];
buffer[a] = buffer[b];
buffer[b] = c;
}
}
/* HTS_FALSE if src does not fit: a truncated module path would point the
symbolizer at the wrong file. */
static hts_boolean copy_bounded(char *dest, size_t size, const char *src) {
size_t i;
for (i = 0; i < size - 1 && src[i] != '\0'; i++) {
dest[i] = src[i];
}
dest[i] = '\0';
return src[i] == '\0' ? HTS_TRUE : HTS_FALSE;
}
/* Run the symbolizer on argv, output on fd, within *budget ticks. HTS_FALSE
only if none could be run at all; otherwise silent, the raw trace stands. */
static hts_boolean spawn_symbolizer(char **argv, int fd, int *budget) {
const pid_t pid = fork();
int status = 0;
if (pid == -1)
return HTS_FALSE;
if (pid == 0) {
static char llvm_prog[] = "llvm-symbolizer";
static char llvm_opts[] = "-p";
dup2(fd, 1); /* both symbolizers write on stdout */
execvp(argv[0], argv);
argv[0] = llvm_prog; /* an LLVM-only install ships no addr2line */
argv[1] = llvm_opts;
execvp(argv[0], argv);
_exit(BT_NO_SYMBOLIZER);
}
for (; *budget > 0; (*budget)--) {
const struct timespec tick = {0, 10 * 1000 * 1000};
const pid_t reaped = waitpid(pid, &status, WNOHANG);
if (reaped == pid)
return WIFEXITED(status) && WEXITSTATUS(status) == BT_NO_SYMBOLIZER
? HTS_FALSE
: HTS_TRUE;
if (reaped == -1 && errno != EINTR)
return HTS_TRUE;
nanosleep(&tick, NULL);
}
kill(pid, SIGKILL);
waitpid(pid, NULL, 0);
return HTS_TRUE;
}
/* Name the frames backtrace_symbols_fd() leaves as module+offset:
-fvisibility=hidden keeps them out of .dynsym, but DWARF has them. dladdr()
is not formally async-signal-safe; accepted, this path is already fatal. */
static void symbolize_backtrace(void *const *stack, int size, int fd) {
static char prog[] = "addr2line";
static char opts[] = "-Cfipa";
static char dashe[] = "-e";
char hex[BT_MAX_FRAMES][BT_HEX_SIZE];
const void *base[BT_MAX_FRAMES];
const char *name[BT_MAX_FRAMES];
hts_boolean grouped[BT_MAX_FRAMES];
char module[BT_PATH_SIZE];
char *argv[4 + BT_MAX_FRAMES + 1];
int budget = BT_WAIT_TICKS;
int i, spawned;
if (size > BT_MAX_FRAMES)
size = BT_MAX_FRAMES;
for (i = 0; i < size; i++) {
Dl_info info;
grouped[i] = HTS_TRUE; /* skipped unless dladdr() places the frame */
if (dladdr(stack[i], &info) == 0 || info.dli_fname == NULL ||
info.dli_fname[0] == '\0')
continue;
base[i] = info.dli_fbase;
name[i] = info.dli_fname;
print_hex(hex[i], (uintptr_t) ((const char *) stack[i] -
(const char *) info.dli_fbase));
grouped[i] = HTS_FALSE;
}
/* One child per module: addr2line takes a single -e. Each frame is claimed
once, so argc cannot exceed argv[]. */
for (spawned = 0; spawned < BT_MAX_MODULES; spawned++) {
int first, j, argc = 0;
for (first = 0; first < size && grouped[first]; first++)
;
if (first >= size)
break;
argv[argc++] = prog;
argv[argc++] = opts;
argv[argc++] = dashe;
argv[argc++] = module;
for (j = first; j < size; j++) {
if (grouped[j] || base[j] != base[first])
continue;
grouped[j] = HTS_TRUE;
argv[argc++] = hex[j];
}
argv[argc] = NULL;
/* access(): skip pseudo-modules like linux-vdso, which have no file and
would draw nothing but an addr2line complaint. */
if (copy_bounded(module, sizeof(module), name[first]) &&
access(module, R_OK) == 0) {
const size_t len = strlen(module);
/* addr2line -a prints offsets only: say which module they are in. */
(void) (write(fd, module, len) == (ssize_t) len);
(void) (write(fd, ":\n", 2) == 2);
if (!spawn_symbolizer(argv, fd, &budget))
break; /* no symbolizer: stop at one header */
}
}
}
#endif
void hts_backtrace_init(void) {
#ifdef USES_BACKTRACE
symbolize_crash =
getenv("HTTRACK_NO_SYMBOLIZE") == NULL ? HTS_TRUE : HTS_FALSE;
#endif
}
void hts_print_backtrace(int fd) {
#ifdef USES_BACKTRACE
void *stack[256];
const int size = backtrace(stack, sizeof(stack) / sizeof(stack[0]));
/* A fault inside the handler lands back here: symbolizing twice interleaves
two traces on fd and spends a second budget. */
static volatile sig_atomic_t entered = 0;
if (size != 0) {
backtrace_symbols_fd(stack, size, fd);
if (symbolize_crash && entered == 0) {
entered = 1;
symbolize_backtrace(stack, size, fd);
entered = 0;
}
}
#else
const char msg[] = "No stack trace available on this OS :(\n";
if (write(fd, msg, sizeof(msg) - 1) != sizeof(msg) - 1) {
/* sorry GCC */
}
#endif
}

45
src/htsbacktrace.h Normal file
View File

@@ -0,0 +1,45 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* File: crash backtrace printer */
/* Author: Xavier Roche */
/* ------------------------------------------------------------ */
#ifndef HTSBACKTRACE_DEFH
#define HTSBACKTRACE_DEFH
/* Sample HTTRACK_NO_SYMBOLIZE before any crash: getenv() is not signal-safe.
Call once, from the process that installs the fatal-signal handlers. */
void hts_backtrace_init(void);
/* Write the calling thread's stack to fd, callable from a fatal signal handler:
raw frames first, then whatever an external symbolizer can name. Allocates
nothing; prints a one-line notice where the OS has no backtrace(). */
void hts_print_backtrace(int fd);
#endif

View File

@@ -39,6 +39,9 @@ Please visit our Website: http://www.httrack.com
#include "htsglobal.h"
#include "htslib.h"
#include "htscore.h"
#ifdef _WIN32
#include "htscharset.h" /* hts_pathToUCS2, hts_convertUCS2StringToUTF8 */
#endif
/* END specific definitions */
@@ -200,28 +203,36 @@ char *cookie_nextfield(char *a) {
return a;
}
// lire cookies.txt
// lire également (Windows seulement) les *@*.txt (cookies IE copiés)
// !=0 : erreur
// Read cookies.txt (+ copied IE cookies *@*.txt on Windows); !=0 on error.
int cookie_load(t_cookie * cookie, const char *fpath, const char *name) {
char catbuff[CATBUFF_SIZE];
char buffer[8192];
// Fusionner d'abord les éventuels cookies IE
// Merge any IE cookies first
#ifdef _WIN32
{
WIN32_FIND_DATAA find;
WIN32_FIND_DATAW find;
HANDLE h;
char pth[MAX_PATH + 32];
char pth[HTS_URLMAXSIZE];
LPWSTR wpth;
strcpybuff(pth, fpath);
strcatbuff(pth, "*@*.txt");
h = FindFirstFileA((char *) pth, &find);
// Wide glob so a long or non-ASCII IE-cookie folder is scanned (#133).
wpth = hts_pathToUCS2(pth);
h = wpth != NULL ? FindFirstFileW(wpth, &find) : INVALID_HANDLE_VALUE;
freet(wpth);
if (h != INVALID_HANDLE_VALUE) {
do {
if (!(find.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY))
if (!(find.dwFileAttributes & FILE_ATTRIBUTE_SYSTEM)) {
FILE *fp = fopen(fconcat(catbuff, sizeof(catbuff), fpath, find.cFileName), "rb");
// cFileName is UTF-16: convert to UTF-8 so the mirror path and
// file wrappers stay UTF-8 (no CP_ACP mojibake).
char *u = hts_convertUCS2StringToUTF8(find.cFileName, -1);
FILE *fp =
u != NULL
? FOPEN(fconcat(catbuff, sizeof(catbuff), fpath, u), "rb")
: NULL;
if (fp) {
char cook_name[256];
@@ -229,11 +240,9 @@ int cookie_load(t_cookie * cookie, const char *fpath, const char *name) {
char domainpathpath[512];
char dummy[512];
//
lien_adrfil af; // chemin (/)
lien_adrfil af; // host + path (/)
int cookie_merged = 0;
//
// Read all cookies
while(!feof(fp)) {
cook_name[0] = cook_value[0] = domainpathpath[0]
@@ -262,10 +271,11 @@ int cookie_load(t_cookie * cookie, const char *fpath, const char *name) {
}
fclose(fp);
if (cookie_merged)
remove(fconcat(catbuff, sizeof(catbuff), fpath, find.cFileName));
UNLINK(fconcat(catbuff, sizeof(catbuff), fpath, u));
} // if fp
freet(u);
}
} while(FindNextFileA(h, &find));
} while (FindNextFileW(h, &find));
FindClose(h);
}
}
@@ -273,7 +283,7 @@ int cookie_load(t_cookie * cookie, const char *fpath, const char *name) {
// Ensuite, cookies.txt
{
FILE *fp = fopen(fconcat(catbuff, sizeof(catbuff), fpath, name), "rb");
FILE *fp = FOPEN(fconcat(catbuff, sizeof(catbuff), fpath, name), "rb");
if (fp) {
char BIGSTK line[8192];
@@ -315,7 +325,7 @@ int cookie_save(t_cookie * cookie, const char *name) {
if (strnotempty(cookie->data)) {
char BIGSTK line[8192];
#ifdef _WIN32
FILE *fp = fopen(fconv(catbuff, sizeof(catbuff), name), "wb");
FILE *fp = FOPEN(fconv(catbuff, sizeof(catbuff), name), "wb");
#else
const int fd = open(fconv(catbuff, sizeof(catbuff), name),
O_WRONLY | O_CREAT | O_TRUNC, HTS_PROTECT_FILE);

View File

@@ -629,8 +629,8 @@ static htsblk cache_readex_new(httrackp * opt, cache_back * cache,
// File exists on disk with declared cache name (this is expected!)
if (fexist_utf8(fconv(catbuff, sizeof(catbuff), previous_save))) { // un fichier existe déja
// Expected size ?
const size_t fsize =
fsize_utf8(fconv(catbuff, sizeof(catbuff), previous_save));
const LLint fsize = fsize_utf8(
fconv(catbuff, sizeof(catbuff), previous_save));
if (fsize == r.size) {
// Target name is the previous name, and the file looks good: nothing to do!
if (strcmp(previous_save, target_save) == 0) {
@@ -666,7 +666,8 @@ static htsblk cache_readex_new(httrackp * opt, cache_back * cache,
// Suppose a broken mirror, with a file being renamed: OK
else if (fexist_utf8(fconv(catbuff, sizeof(catbuff), target_save))) {
// Expected size ?
const size_t fsize = fsize_utf8(fconv(catbuff, sizeof(catbuff), target_save));
const LLint fsize =
fsize_utf8(fconv(catbuff, sizeof(catbuff), target_save));
if (fsize == r.size) {
// So far so good
@@ -1294,12 +1295,17 @@ char *readfile2(const char *fil, LLint * size) {
}
/* Note: utf-8 */
char *readfile_utf8(const char *fil) {
char *readfile_utf8(const char *fil) { return readfile2_utf8(fil, NULL); }
/* Note: utf-8 */
char *readfile2_utf8(const char *fil, LLint *size) {
char *adr = NULL;
char catbuff[CATBUFF_SIZE];
const LLint len = fsize_utf8(fil);
const size_t buflen = len >= 0 ? llint_to_size_t(len) : (size_t) -1;
if (size != NULL)
*size = len;
if (buflen != (size_t) -1) { // exists, and is addressable (see readfile2)
FILE *const fp = FOPEN(fconv(catbuff, sizeof(catbuff), fil), "rb");
@@ -1440,7 +1446,7 @@ int cache_brstr(char *adr, char *s, size_t s_size) {
/* binput bounded to a NUL-terminated buffer: refuse to start a read at or
past `end`, so a prior over-advance can't walk a cache-index parse OOB. */
int cache_binput(char *adr, const char *end, char *s, int max) {
int cache_binput(const char *adr, const char *end, char *s, int max) {
if (adr >= end) {
s[0] = '\0';
return 0;

View File

@@ -93,7 +93,7 @@ void cache_rstr(FILE *fp, char *s, size_t s_size);
char *cache_rstr_addr(FILE * fp);
int cache_brstr(char *adr, char *s, size_t s_size);
/* binput over a NUL-terminated buffer, bounded: no read starts at/past end. */
int cache_binput(char *adr, const char *end, char *s, int max);
int cache_binput(const char *adr, const char *end, char *s, int max);
int cache_brint(char *adr, int *i);
void cache_rint(FILE * fp, int *i);
void cache_rLLint(FILE * fp, LLint * i);

View File

@@ -390,6 +390,10 @@ HTSEXT_API char *hts_convertStringSystemToUTF8(const char *s, size_t size) {
return hts_convertStringCPToUTF8(s, size, GetACP());
}
HTSEXT_API char *hts_convertStringUTF8ToSystem(const char *s, size_t size) {
return hts_convertStringCPFromUTF8(s, size, GetACP());
}
HTSEXT_API void hts_argv_utf8(int *pargc, char ***pargv) {
int wargc = 0;
LPWSTR *const wargv = CommandLineToArgvW(GetCommandLineW(), &wargc);

View File

@@ -171,12 +171,25 @@ extern LPWSTR hts_convertUTF8StringToUCS2(const char *s, int size, int *pwsize);
**/
extern char *hts_convertUCS2StringToUTF8(LPWSTR woutput, int wsize);
/**
* UTF-8 path to UCS-2 for the wide file/FindFirst APIs, \\?\-prefixed above
* MAX_PATH (#133). Internal, not exported; caller frees.
* This function is WIN32 specific.
**/
extern LPWSTR hts_pathToUCS2(const char *path);
/**
* Convert current system codepage to UTF-8.
* This function is WIN32 specific.
**/
HTSEXT_API char *hts_convertStringSystemToUTF8(const char *s, size_t size);
/**
* Convert UTF-8 to the current system codepage. Caller frees; NULL upon error.
* This function is WIN32 specific.
**/
HTSEXT_API char *hts_convertStringUTF8ToSystem(const char *s, size_t size);
/**
* Replace the CRT's ANSI argv by a UTF-8 one decoded from the real UTF-16
* command line: every char* is UTF-8 on Windows (FOPEN, STAT, ... convert at

95
src/htscmdline.c Normal file
View File

@@ -0,0 +1,95 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* File: command line splitter, shared by the engine and */
/* htsserver */
/* Author: Xavier Roche */
/* ------------------------------------------------------------ */
#include "htscmdline.h"
#include "htssafe.h"
#include <limits.h>
#include <stdint.h>
char **hts_split_cmdline(char *cmd, int *nargs) {
size_t nsep = 0;
size_t capacity;
size_t r;
size_t w;
int argc = 0;
hts_boolean quoted = HTS_FALSE;
char **argv;
*nargs = 0;
/* fold the other separators, so counting them sizes the vector exactly */
for (r = 0; cmd[r] != '\0'; r++) {
if (cmd[r] == '\t' || cmd[r] == '\r' || cmd[r] == '\n') {
cmd[r] = ' ';
}
if (cmd[r] == ' ') {
nsep++;
}
}
/* at most one argument per separator, plus the leading one and the NULL */
if (nsep > (size_t) INT_MAX - 1 || nsep > SIZE_MAX / sizeof(char *) - 2) {
return NULL;
}
capacity = nsep + 2;
argv = (char **) malloct(capacity * sizeof(char *));
if (argv == NULL) {
return NULL;
}
argv[argc++] = cmd;
for (r = 0, w = 0; cmd[r] != '\0';) {
if (quoted && cmd[r] == '\\' &&
(cmd[r + 1] == '\\' || cmd[r + 1] == '\"')) {
r++;
cmd[w++] = cmd[r++];
} else if (cmd[r] == '\"') {
quoted = !quoted;
cmd[w++] = cmd[r++];
} else if (cmd[r] == ' ' && !quoted) {
cmd[w++] = '\0';
assertf((size_t) argc < capacity - 1); /* the last slot holds the NULL */
argv[argc++] = cmd + w;
r++;
} else {
cmd[w++] = cmd[r++];
}
}
cmd[w] = '\0';
argv[argc] = NULL; /* callers may rely on argv[argc] == NULL */
*nargs = argc;
return argv;
}

45
src/htscmdline.h Normal file
View File

@@ -0,0 +1,45 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* File: command line splitter, shared by the engine and */
/* htsserver */
/* Author: Xavier Roche */
/* ------------------------------------------------------------ */
#ifndef HTSCMDLINE_DEFH
#define HTSCMDLINE_DEFH
#include "htsglobal.h"
/* Split "cmd" in place into a NULL-terminated argv vector of *nargs entries,
argv[0] being the program name and quotes left for the engine to strip.
Returns a malloct'ed vector of pointers into cmd (freet the vector, never its
entries), or NULL when it cannot be sized or allocated. */
char **hts_split_cmdline(char *cmd, int *nargs);
#endif

View File

@@ -39,6 +39,8 @@ Please visit our Website: http://www.httrack.com
/* File defs */
#include "htscore.h"
#include "htswarc.h"
#include "htssinglefile.h"
/* specific definitions */
#include "htsbase.h"
@@ -446,13 +448,22 @@ void hts_finish_makeindex(httrackp *opt, int *makeindex_done,
const char *fil) {
if (!*makeindex_done) {
if (*makeindex_fp) {
char BIGSTK tempo[1024];
/* sized off link_escaped below: at the old flat 1024 a long first link
produced a redirect to a clipped URL */
char BIGSTK tempo[HTS_URLMAXSIZE * 2 + 64];
if (makeindex_links == 1) {
char BIGSTK link_escaped[HTS_URLMAXSIZE * 2];
escape_uri_utf(makeindex_firstlink, link_escaped, sizeof(link_escaped));
snprintf(tempo, sizeof(tempo),
"<meta HTTP-EQUIV=\"Refresh\" CONTENT=\"0; URL=%s\">" CRLF,
link_escaped);
/* no redirect beats one pointing at a clipped URL */
if (!sprintfbuff(
tempo,
"<meta HTTP-EQUIV=\"Refresh\" CONTENT=\"0; URL=%s\">" CRLF,
link_escaped)) {
hts_log_print(opt, LOG_WARNING,
"index redirect omitted: first link too long (%s)",
makeindex_firstlink);
tempo[0] = '\0';
}
} else
tempo[0] = '\0';
hts_template_format(*makeindex_fp, template_footer,
@@ -695,17 +706,19 @@ int httpmirror(char *url1, httrackp * opt) {
// copier adresse(s) dans liste des adresses
{
char *a = url1;
int primary_len = 8192;
if (StringNotEmpty(opt->filelist)) {
primary_len += max(0, fsize(StringBuff(opt->filelist)) * 2);
}
primary_len += (int) strlen(url1) * 2;
const LLint list_sz = StringNotEmpty(opt->filelist)
? fsize_utf8(StringBuff(opt->filelist))
: 0;
/* two bytes reserved per list byte; -1 makes an undoublable size refused */
const LLint list_room =
list_sz > 0 ? (list_sz <= INT64_MAX / 2 ? list_sz * 2 : -1) : 0;
const size_t primary_len =
llint_grow_size_t(8192 + strlen(url1) * 2, list_room, 0);
// création de la première page, qui contient les liens de base à scanner
// c'est plus propre et plus logique que d'entrer à la main les liens dans la pile
// on bénéficie ainsi des vérifications et des tests du robot pour les liens "primaires"
primary = (char *) malloct(primary_len);
primary = primary_len != (size_t) -1 ? (char *) malloct(primary_len) : NULL;
if (!primary) {
printf("PANIC! : Not enough memory [%d]\n", __LINE__);
XH_extuninit;
@@ -855,19 +868,19 @@ int httpmirror(char *url1, httrackp * opt) {
char *filelist_buff = NULL;
size_t filelist_sz = 0;
const char *filelist_err = NULL; /* failure reason, NULL on success */
const LLint fs = fsize(StringBuff(opt->filelist));
const LLint fs = fsize_utf8(StringBuff(opt->filelist));
if (fs < 0) {
/* fsize() hides the cause; redo stat() for a precise errno (#49) */
struct stat st;
filelist_err = stat(StringBuff(opt->filelist), &st) != 0
STRUCT_STAT st;
filelist_err = STAT(StringBuff(opt->filelist), &st) != 0
? strerror(errno)
: "not a regular file";
} else if ((filelist_sz = llint_to_size_t(fs)) == (size_t) -1) {
filelist_err = "file too large";
filelist_sz = 0;
} else {
FILE *fp = fopen(StringBuff(opt->filelist), "rb");
FILE *fp = FOPEN(StringBuff(opt->filelist), "rb");
if (fp == NULL) {
filelist_err = strerror(errno);
@@ -886,7 +899,7 @@ int httpmirror(char *url1, httrackp * opt) {
}
if (filelist_buff != NULL) {
int filelist_ptr = 0;
size_t filelist_ptr = 0;
int n = 0;
char BIGSTK line[HTS_URLMAXSIZE * 2];
@@ -975,10 +988,9 @@ int httpmirror(char *url1, httrackp * opt) {
}
// statistiques
if (opt->makestat) {
makestat_fp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-stats.txt"),
"wb");
makestat_fp = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-stats.txt"),
"wb");
if (makestat_fp != NULL) {
fprintf(makestat_fp, "HTTrack statistics report, every minutes" LF LF);
fflush(makestat_fp);
@@ -986,10 +998,9 @@ int httpmirror(char *url1, httrackp * opt) {
}
// tracking -- débuggage
if (opt->maketrack) {
maketrack_fp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-track.txt"),
"wb");
maketrack_fp = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-track.txt"),
"wb");
if (maketrack_fp != NULL) {
fprintf(maketrack_fp, "HTTrack tracking report, every minutes" LF LF);
fflush(maketrack_fp);
@@ -1638,7 +1649,8 @@ int httpmirror(char *url1, httrackp * opt) {
/* Remove file if being processed */
if (is_loaded_from_file) {
(void) unlink(fconv(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), savename()));
(void) UNLINK(
fconv(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), savename()));
is_loaded_from_file = 0;
}
@@ -1951,8 +1963,8 @@ int httpmirror(char *url1, httrackp * opt) {
} else */
/* External modules */
if (opt->parsejava && (opt->parsejava & HTSPARSE_NO_CLASS) == 0
&& fexist(savename())) {
if (opt->parsejava && (opt->parsejava & HTSPARSE_NO_CLASS) == 0 &&
fexist_utf8(savename())) {
char BIGSTK buff_err_msg[1024];
htsmoduleStruct BIGSTK str;
@@ -1993,7 +2005,7 @@ int httpmirror(char *url1, httrackp * opt) {
} // text/html ou autre
/* Post-processing */
if (fexist(savename())) {
if (fexist_utf8(savename())) {
usercommand(opt, 0, NULL, savename(), urladr(), urlfil());
}
@@ -2084,18 +2096,16 @@ int httpmirror(char *url1, httrackp * opt) {
//
opt->state._hts_in_html_parsing = 3;
//
old_lst =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.lst"), "rb");
old_lst = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/old.lst"),
"rb");
if (old_lst) {
const size_t sz = llint_to_size_t(
fsize(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.lst")));
new_lst =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.lst"), "rb");
const size_t sz = llint_to_size_t(fsize_utf8(
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.lst")));
new_lst = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.lst"),
"rb");
if (new_lst != NULL && sz != (size_t) -1) {
/* +1 for the NUL below: new.lst is read raw, and the strstr()
that follows needs a terminated C string. */
@@ -2115,9 +2125,9 @@ int httpmirror(char *url1, httrackp * opt) {
strcpybuff(file, StringBuff(opt->path_html));
strcatbuff(file, line + 1);
file[strlen(file) - 1] = '\0';
if (fexist(file)) { // toujours sur disque: virer
if (fexist_utf8(file)) { // toujours sur disque: virer
hts_log_print(opt, LOG_INFO, "Purging %s", file);
remove(file);
UNLINK(file);
purge = 1;
}
}
@@ -2138,7 +2148,8 @@ int httpmirror(char *url1, httrackp * opt) {
strcpybuff(file, StringBuff(opt->path_html));
strcatbuff(file, line + 1);
while((strnotempty(file)) && (rmdir(file) == 0)) { // ok, éliminé (existait)
while ((strnotempty(file)) &&
(RMDIR(file) == 0)) { // ok, éliminé (existait)
purge = 1;
if (opt->log) {
hts_log_print(opt, LOG_INFO, "Purging directory %s/",
@@ -2172,6 +2183,10 @@ int httpmirror(char *url1, httrackp * opt) {
}
// fin purge!
/* --single-file: inline each page's assets now the tree is final, after the
purge has deleted whatever this run dropped. */
singlefile_process_mirror(opt);
// Indexation
if (opt->kindex)
index_finish(StringBuff(opt->path_html), opt->kindex);
@@ -2245,6 +2260,7 @@ int httpmirror(char *url1, httrackp * opt) {
// ending
usercommand(opt, 0, NULL, NULL, NULL, NULL);
warc_close_opt(opt);
// désallocation mémoire & buffers
XH_uninit;
@@ -2960,8 +2976,8 @@ static void postprocess_file(httrackp *opt, const char *save, const char *adr,
if (adr != NULL && strcmp(adr, "primary") == 0) {
adr = NULL;
}
if (save != NULL && opt != NULL && adr != NULL && adr[0]
&& strnotempty(save) && fexist(save)) {
if (save != NULL && opt != NULL && adr != NULL && adr[0] &&
strnotempty(save) && fexist_utf8(save)) {
const char *rsc_save = save;
const char *rsc_fil = strrchr(fil, '/');
int n;
@@ -2983,13 +2999,11 @@ static void postprocess_file(httrackp *opt, const char *save, const char *adr,
if (!opt->state.mimehtml_created) {
opt->state.mimefp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html), "index.mht"),
"wb");
(void) unlink(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html),
"index.eml"));
FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html), "index.mht"),
"wb");
(void) UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html), "index.eml"));
#ifndef _WIN32
if (symlink("index.mht",
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_html),
@@ -3628,6 +3642,16 @@ HTSEXT_API int copy_htsopt(const httrackp * from, httrackp * to) {
if (StringNotEmpty(from->cookies_file))
StringCopyS(to->cookies_file, from->cookies_file);
if (StringNotEmpty(from->warc_file))
StringCopyS(to->warc_file, from->warc_file);
to->warc_max_size = from->warc_max_size;
to->warc_cdx = from->warc_cdx;
to->warc_wacz = from->warc_wacz;
to->single_file = from->single_file;
if (from->single_file_max_size > 0)
to->single_file_max_size = from->single_file_max_size;
if (from->pause_max_ms > 0) {
to->pause_min_ms = from->pause_min_ms;
to->pause_max_ms = from->pause_max_ms;

View File

@@ -409,6 +409,8 @@ char *readfile2(const char *fil, LLint * size);
char *readfile_utf8(const char *fil);
char *readfile2_utf8(const char *fil, LLint *size);
char *readfile_or(const char *fil, const char *defaultdata);
/* Backing (download-slot) scheduler. Operate on the back[] ring (struct_back).

View File

@@ -40,6 +40,7 @@ Please visit our Website: http://www.httrack.com
#include "htscore.h"
#include "htsdefines.h"
#include "htsalias.h"
#include "htswarc.h"
#include "htsbauth.h"
#include "htswrap.h"
#include "htsmodules.h"
@@ -144,7 +145,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
int argv_url = -1; // ==0 : utiliser cache et doit.log
char *argv_firsturl = NULL; // utilisé pour nommage par défaut
char *url = NULL; // URLS séparées par un espace
int url_sz = 65535;
size_t url_sz = 65535;
// the parametres
int httrack_logmode = 3; // ONE log file
@@ -223,21 +224,22 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
/* create x_argvblk buffer for transformed command line */
{
int current_size = 0;
int size;
size_t current_size = 0;
const LLint size = fsize("config");
size_t blk_size;
int na;
for(na = 0; na < argc; na++)
current_size += (int) (strlen(argv[na]) + 1);
if ((size = fsize("config")) > 0)
current_size += size;
x_argvblk = (char *) malloct(current_size + 32768);
current_size += strlen(argv[na]) + 1;
/* a huge file named "config" must saturate, not wrap, the capacity */
blk_size = llint_grow_size_t(current_size, size > 0 ? size : 0, 32768);
x_argvblk = blk_size != (size_t) -1 ? (char *) malloct(blk_size) : NULL;
if (x_argvblk == NULL) {
HTS_PANIC_PRINTF("Error, not enough memory");
htsmain_free();
return -1;
}
x_argvblk_size = (size_t) (current_size + 32768);
x_argvblk_size = blk_size;
x_argvblk[0] = '\0';
x_ptr = 0;
@@ -441,11 +443,10 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
|| (strnotempty(StringBuff(opt->path_html))))
loops++; // do not loop once again and do not include rc file (O option exists)
else {
if ((!fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/doit.log"))) || (argv_url > 0)) {
if ((!fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/doit.log"))) ||
(argv_url > 0)) {
if (!optinclude_file(
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), HTS_HTTRACKRC),
@@ -472,16 +473,12 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
} // traiter -O
/* load doit.log and insert in current command line */
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/doit.log"))
&& (argv_url <= 0)) {
FILE *fp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/doit.log"), "rb");
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/doit.log")) &&
(argv_url <= 0)) {
FILE *fp = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/doit.log"),
"rb");
if (fp) {
int insert_after = 1; /* insérer après nom au début */
@@ -530,23 +527,16 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
/* Interrupted mirror detected */
if (!opt->quiet) {
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-in_progress.lock"))) {
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-in_progress.lock"))) {
/* Old cache */
if ((fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.dat")))
&&
(fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.ndx")))) {
if ((fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.dat"))) &&
(fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.ndx")))) {
if (opt->log != NULL) {
fprintf(opt->log, "Warning!\n");
fprintf(opt->log,
@@ -575,111 +565,82 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
if (argv[i][1] == '-') { // --xxx
if ((strfield2(argv[i] + 2, "clean")) || (strfield2(argv[i] + 2, "tide"))) { // nettoyer
argv[i][1] = '\0';
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-log.txt")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-log.txt"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-err.txt")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-err.txt"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_html), "index.html")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_html),
"index.html"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-log.txt")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-log.txt"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-err.txt")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-err.txt"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html), "index.html")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html), "index.html"));
/* */
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.zip")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.zip"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.zip")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.zip"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.dat")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.dat"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.ndx")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.ndx"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.dat")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.dat"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.ndx")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.ndx"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.lst")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.lst"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.lst")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.lst"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.txt")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.txt"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.txt")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.txt"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/doit.log")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/doit.log"));
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-in_progress.lock")))
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-in_progress.lock"));
rmdir(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-cache"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.zip")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.zip"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.zip")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/old.zip"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.dat")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.dat"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.ndx")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.ndx"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.dat")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/old.dat"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.ndx")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/old.ndx"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.lst")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.lst"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.lst")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/old.lst"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.txt")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.txt"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.txt")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/old.txt"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/doit.log")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/doit.log"));
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-in_progress.lock")))
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-in_progress.lock"));
RMDIR(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache"));
//
} else if (strfield2(argv[i] + 2, "catchurl")) { // capture d'URL via proxy temporaire!
argv_url = 1; // forcer a passer les parametres
@@ -736,29 +697,27 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
#endif
if (argv_url == 0) {
// Présence d'un cache, que faire?..
if ((fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.zip")))
||
(fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.dat"))
&&
fexist(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.ndx")))
) { // il existe déja un cache précédent.. renommer
if (fexist(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/doit.log"))) { // un cache est présent
if ((fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.zip"))) ||
(fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.dat")) &&
fexist_utf8(
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.ndx")))) { // il existe déja un cache
// précédent.. renommer
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/doit.log"))) { // un cache est présent
if (x_argvblk != NULL) {
int m;
// établir mode - mode cache: 1 (cache valide) 2 (cache à vérifier)
if (fexist(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-in_progress.lock"))) { // cache prioritaire
if (fexist_utf8(
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-in_progress.lock"))) { // cache prioritaire
m = 1;
} else {
m = 2;
@@ -793,7 +752,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
htsmain_free();
return -1;
}
} else { // log existe pas
} else { // log existe pas
HTS_PANIC_PRINTF("A cache has been found, but no command line");
printf
("Please launch httrack with proper parameters to reuse the cache\n");
@@ -801,7 +760,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
return -1;
}
} else { // aucune URL définie et pas de cache
} else { // aucune URL définie et pas de cache
if (opt->quiet) {
help(argv[0], !opt->quiet);
htsmain_free();
@@ -816,26 +775,21 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
}
} else { // plus de 2 paramètres
// un fichier log existe?
if (fexist(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-in_progress.lock"))) { // fichier lock?
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-in_progress.lock"))) { // fichier lock?
opt->cache = HTS_CACHE_PRIORITY; // cache prioritaire
if (opt->quiet == 0) {
if ((fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.zip")))
||
(fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.dat"))
&&
fexist(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.ndx")))
) {
if ((fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.zip"))) ||
(fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.dat")) &&
fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.ndx")))) {
HT_REQUEST_START;
HT_PRINT("There is a lock-file in the directory ");
HT_PRINT(StringBuff(opt->path_log));
@@ -849,24 +803,19 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
}
}
}
} else if (fexist(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html), "index.html"))) {
} else if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_html), "index.html"))) {
opt->cache = HTS_CACHE_TEST_UPDATE;
if (opt->quiet == 0) {
if ((fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.zip")))
||
(fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.dat"))
&&
fexist(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.ndx")))
) {
if ((fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.zip"))) ||
(fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.dat")) &&
fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/new.ndx")))) {
HT_REQUEST_START;
HT_PRINT
("There is an index.html and a hts-cache folder in the directory ");
@@ -1499,29 +1448,38 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
LLint fz;
na++;
fz = fsize(argv[na]);
fz = fsize_utf8(argv[na]);
if (fz < 0) {
HTS_PANIC_PRINTF("File url list could not be opened");
htsmain_free();
return -1;
} else {
FILE *fp = fopen(argv[na], "rb");
FILE *fp = FOPEN(argv[na], "rb");
if (fp != NULL) {
int cl = (int) strlen(url);
size_t cl = strlen(url);
const size_t fzs = llint_to_size_t(fz);
const size_t capa = llint_grow_size_t(cl, fz, 8192);
ensureUrlCapacity(url, url_sz, cl + fz + 8192);
if (capa == (size_t) -1) {
fclose(fp);
HTS_PANIC_PRINTF("File url list too large");
htsmain_free();
return -1;
}
ensureUrlCapacity(url, url_sz, capa);
if (cl > 0) { /* don't stick! (3.43) */
url[cl] = ' ';
cl++;
}
if (fread(url + cl, 1, fz, fp) != fz) {
if (fread(url + cl, 1, fzs, fp) != fzs) {
fclose(fp);
HTS_PANIC_PRINTF("File url list could not be read");
htsmain_free();
return -1;
}
fclose(fp);
*(url + cl + fz) = '\0';
*(url + cl + fzs) = '\0';
}
}
}
@@ -1622,10 +1580,9 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
if ((na + 1 >= argc) || (argv[na + 1][0] == '-')) {
HTS_PANIC_PRINTF
("Option %F needs to be followed by a blank space, and a footer string");
printf
("Example: -%%F \"<!-- Mirrored from %%s by HTTrack Website Copier/"
HTTRACK_AFF_VERSION " " HTTRACK_AFF_AUTHORS
", %%s -->\"\n");
printf("Example: -%%F \"<!-- Mirrored from {addr}{path} by "
"HTTrack Website Copier/"
"{version} " HTTRACK_AFF_AUTHORS ", {date} -->\"\n");
htsmain_free();
return -1;
} else {
@@ -1790,6 +1747,84 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
StringCopy(opt->cookies_file, argv[na]);
}
break;
case 'r': // warc / warc-file: write an ISO-28500 WARC archive
if (*(com + 1) == 'f') { // --warc-file NAME: explicit basename
com++;
if ((na + 1 >= argc) || (argv[na + 1][0] == '-')) {
HTS_PANIC_PRINTF(
"Option warc-file needs a blank space and a WARC name");
htsmain_free();
return -1;
}
na++;
if (strlen(argv[na]) >= 1024) {
HTS_PANIC_PRINTF("WARC file name too long");
htsmain_free();
return -1;
}
StringCopy(opt->warc_file, argv[na]);
} else if (*(com + 1) == 's') { // --warc-max-size N: rotation
com++;
if ((na + 1 >= argc) || (argv[na + 1][0] == '-')) {
HTS_PANIC_PRINTF(
"Option warc-max-size needs a blank space and a size");
htsmain_free();
return -1;
}
na++;
{ // reject non-numeric/negative/overflow; keep default 0
// (single file)
char *end;
LLint v;
errno = 0;
v = strtoll(argv[na], &end, 10);
if (isdigit((unsigned char) argv[na][0]) && *end == '\0' &&
errno != ERANGE)
opt->warc_max_size = v;
}
} else if (*(com + 1) == 'c') { // --warc-cdx: sorted CDXJ index
com++;
opt->warc_cdx = 1;
} else if (*(com + 1) == 'z') { // --wacz: WACZ package
com++;
opt->warc_wacz = 1;
opt->warc_cdx = 1; // WACZ embeds the CDXJ index
if (!StringNotEmpty(opt->warc_file))
StringCopy(opt->warc_file, WARC_AUTONAME);
} else { // --warc: auto-named archive under the output dir
StringCopy(opt->warc_file, WARC_AUTONAME);
}
break;
case 'Z': // single-file: inline each page's assets as data: URIs
if (*(com + 1) == 's') { // --single-file-max-size N
com++;
if ((na + 1 >= argc) || (argv[na + 1][0] == '-')) {
HTS_PANIC_PRINTF(
"Option single-file-max-size needs a blank "
"space and a size");
htsmain_free();
return -1;
}
na++;
{ // reject non-numeric/negative/overflow; keep the default
char *end;
LLint v;
errno = 0;
v = strtoll(argv[na], &end, 10);
if (isdigit((unsigned char) argv[na][0]) && *end == '\0' &&
errno != ERANGE && v > 0)
opt->single_file_max_size = v;
}
opt->single_file = HTS_TRUE;
} else {
opt->single_file = HTS_TRUE;
if (*(com + 1) == '0') {
opt->single_file = HTS_FALSE;
com++;
}
}
break;
case 'Y': // why: explain the filter verdict for a URL, no crawl
if ((na + 1 >= argc) || (argv[na + 1][0] == '-')) {
HTS_PANIC_PRINTF("Option why needs a blank space and a URL");
@@ -2002,7 +2037,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
/*cache */ &cache, /*hash */ NULL, /*ptr */
0, /*numero_passe */ 0, /*mime_type */
NULL) != -1) {
if (fexist(afs.save)) {
if (fexist_utf8(afs.save)) {
fprintf(stdout, "Content-location: %s\r\n",
afs.save);
}
@@ -2100,18 +2135,16 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
uLong repaired = 0;
uLong repairedBytes = 0;
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.zip"))) {
if (fexist_utf8(fconcat(
OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/new.zip"))) {
name =
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/new.zip");
} else
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.zip"))) {
} else if (fexist_utf8(fconcat(OPT_GET_BUFF(opt),
OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/old.zip"))) {
name =
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/old.zip");
@@ -2130,10 +2163,11 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/repair.tmp"), &repaired,
&repairedBytes) == Z_OK) {
unlink(name);
rename(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/repair.zip"), name);
UNLINK(name);
RENAME(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-cache/repair.zip"),
name);
fprintf(stderr,
"Cache: %d bytes successfully recovered in %d entries\n",
(int) repairedBytes, (int) repaired);
@@ -2302,8 +2336,8 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
} else { // URL/filters
char catbuff[CATBUFF_SIZE];
const int urlSize = (int) strlen(argv[na]);
const int capa = (int) (strlen(url) + urlSize + 32);
const size_t urlSize = strlen(argv[na]);
const size_t capa = strlen(url) + urlSize + 32;
assertf(urlSize < HTS_URLMAXSIZE);
if (urlSize < HTS_URLMAXSIZE) {
@@ -2351,10 +2385,9 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
hts_cache_reconcile(opt, CACHE_RECONCILE_INTERRUPTED);
// Débuggage des en têtes
if (_DEBUG_HEAD) {
ioinfo =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-ioinfo.txt"),
"wb");
ioinfo = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-ioinfo.txt"),
"wb");
}
{
@@ -2429,9 +2462,9 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
/* readme for information purpose */
{
FILE *fp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/readme.txt"), "wb");
FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/readme.txt"),
"wb");
if (fp) {
fprintf(fp, "What's in this folder?" LF);
fprintf(fp, "" LF);
@@ -2483,17 +2516,16 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
int i;
#ifdef _WIN32
mkdir(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-cache"));
hts_mkdir_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache"));
#else
mkdir(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-cache"),
HTS_PROTECT_FOLDER);
#endif
fp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log),
"hts-cache/doit.log"), "wb");
fp = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-cache/doit.log"),
"wb");
if (fp) {
for(i = 0 + 1; i < argc; i++) {
if (((strchr(argv[i], ' ') != NULL)
@@ -2538,7 +2570,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
}
}
// petit message dans le lock
if ((fp = fopen(n_lock, "wb")) != NULL) {
if ((fp = FOPEN(n_lock, "wb")) != NULL) {
int i;
fprintf(fp, "Mirror in progress since %s .. please wait!" LF, t);
@@ -2698,16 +2730,15 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
char *f = OPT_GET_BUFF(opt);
sprintf(f, "%s/%s", CACHE_REFNAME, entry->d_name);
(void)
unlink(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), f));
(void) UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), f));
}
}
if (dir != NULL) {
(void) closedir(dir);
}
(void)
rmdir(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), CACHE_REFNAME));
(void) RMDIR(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), CACHE_REFNAME));
}
/* Info for wrappers */
@@ -2735,7 +2766,7 @@ static int hts_main_internal(int argc, char **argv, httrackp * opt) {
}
}
// supprimer lock
remove(n_lock);
UNLINK(n_lock);
}
if (x_argvblk)

View File

@@ -251,7 +251,7 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
// folding a nonsense port into 1..65535 fetches one the link never named;
// an empty "host:" just means the default (#614)
if (a[1] != '\0' && !hts_parse_url_port(a + 1, &port)) {
snprintf(back->r.msg, sizeof(back->r.msg), "Invalid port: %s", a + 1);
htsblk_failf(&back->r, "Invalid port: %s", a + 1);
back->r.statuscode = STATUSCODE_INVALID; // permanent, unlike a DNS miss
_HALT_FTP return 0;
}
@@ -262,8 +262,7 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
// récupérer adresse résolue
strcpybuff(back->info, "host name");
if (hts_dns_resolve2(opt, _adr, &server, &error) == NULL) {
snprintf(back->r.msg, sizeof(back->r.msg),
"Unable to get server's address: %s", error);
htsblk_failf(&back->r, "Unable to get server's address: %s", error);
back->r.statuscode = STATUSCODE_NON_FATAL;
_HALT_FTP return 0;
}
@@ -332,18 +331,15 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
}
} else {
snprintf(back->r.msg, sizeof(back->r.msg), "Bad password: %s",
linejmp(line));
htsblk_failf(&back->r, "Bad password: %s", linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
}
} else {
snprintf(back->r.msg, sizeof(back->r.msg), "Bad user name: %s",
linejmp(line));
htsblk_failf(&back->r, "Bad user name: %s", linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
}
} else {
snprintf(back->r.msg, sizeof(back->r.msg), "Connection refused: %s",
linejmp(line));
htsblk_failf(&back->r, "Connection refused: %s", linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
}
@@ -410,8 +406,7 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
}
// -- fin analyse de l'adresse IP et du port --
} else {
snprintf(back->r.msg, sizeof(back->r.msg), "PASV incorrect: %s",
linejmp(line));
htsblk_failf(&back->r, "PASV incorrect: %s", linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
} // sinon on est prêts
} else {
@@ -442,13 +437,11 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
}
}
} else {
snprintf(back->r.msg, sizeof(back->r.msg), "EPSV incorrect: %s",
linejmp(line));
htsblk_failf(&back->r, "EPSV incorrect: %s", linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
}
} else {
snprintf(back->r.msg, sizeof(back->r.msg), "PASV/EPSV error: %s",
linejmp(line));
htsblk_failf(&back->r, "PASV/EPSV error: %s", linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
} // sinon on est prêts
}
@@ -554,8 +547,8 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
deletesoc(soc_dat);
soc_dat = INVALID_SOCKET;
//
snprintf(back->r.msg, sizeof(back->r.msg),
"RETR command error: %s", linejmp(line));
htsblk_failf(&back->r, "RETR command error: %s",
linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
} // sinon on est prêts
} else {
@@ -573,13 +566,12 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
back->r.statuscode = STATUSCODE_INVALID;
} // sinon on est prêts
} else {
snprintf(back->r.msg, sizeof(back->r.msg),
"Unable to resolve IP %s: %s", adr_ip, error);
htsblk_failf(&back->r, "Unable to resolve IP %s: %s", adr_ip,
error);
back->r.statuscode = STATUSCODE_INVALID;
} // sinon on est prêts
} else {
snprintf(back->r.msg, sizeof(back->r.msg), "PASV incorrect: %s",
linejmp(line));
htsblk_failf(&back->r, "PASV incorrect: %s", linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
} // sinon on est prêts
#else
@@ -603,13 +595,11 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
back->r.statuscode = STATUSCODE_INVALID;
}
} else {
snprintf(back->r.msg, sizeof(back->r.msg),
"RETR command error: %s", linejmp(line));
htsblk_failf(&back->r, "RETR command error: %s", linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
}
} else {
snprintf(back->r.msg, sizeof(back->r.msg), "PORT command error: %s",
linejmp(line));
htsblk_failf(&back->r, "PORT command error: %s", linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
}
#ifdef _WIN32
@@ -652,8 +642,7 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
len = 0; // fin
break;
case 0:
snprintf(back->r.msg, sizeof(back->r.msg), "Time out (%d)",
timeout);
htsblk_failf(&back->r, "Time out (%d)", timeout);
back->r.statuscode = STATUSCODE_INVALID;
len = 0; // fin
break;
@@ -716,8 +705,7 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
strcpybuff(back->r.msg, "OK");
back->r.statuscode = HTTP_OK;
} else {
snprintf(back->r.msg, sizeof(back->r.msg), "RETR incorrect: %s",
linejmp(line));
htsblk_failf(&back->r, "RETR incorrect: %s", linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
}
} else {

View File

@@ -43,8 +43,8 @@ Please visit our Website: http://www.httrack.com
configure.ac, decoupled from these). VERSION is the display form, VERSIONID
the dotted numeric form, AFF_VERSION the short form shown in footers,
LIB_VERSION the data/cache format generation. */
#define HTTRACK_VERSION "3.49-13"
#define HTTRACK_VERSIONID "3.49.13"
#define HTTRACK_VERSION "3.49-14"
#define HTTRACK_VERSIONID "3.49.14"
#define HTTRACK_AFF_VERSION "3.x"
#define HTTRACK_LIB_VERSION "2.0"
@@ -72,7 +72,8 @@ Please visit our Website: http://www.httrack.com
HTS_UNUSED: suppress unused-symbol warnings. HTS_STATIC: an unused-safe
static. HTS_PRINTF_FUN(fmt, arg): mark a printf-like function so the
compiler type-checks the format string at argument index fmt against the
varargs starting at arg. */
varargs starting at arg. HTS_CHECK_RESULT: the return value carries the only
error signal, so dropping it is a bug; a (void) cast does not silence it. */
#ifndef HTS_UNUSED
#ifdef __GNUC__
#define HTS_UNUSED __attribute__((unused))
@@ -80,10 +81,13 @@ Please visit our Website: http://www.httrack.com
#define HTS_STATIC static __attribute__((unused))
#define HTS_PRINTF_FUN(fmt, arg) __attribute__((format(printf, fmt, arg)))
#define HTS_CHECK_RESULT __attribute__((warn_unused_result))
#else
#define HTS_UNUSED
#define HTS_STATIC static
#define HTS_PRINTF_FUN(fmt, arg)
#define HTS_CHECK_RESULT
#endif
#endif
@@ -336,16 +340,20 @@ typedef int INTsys;
#endif
/* Socket-handle type. An unsigned integer wide enough for a Windows SOCKET;
a plain int file descriptor on POSIX. */
a plain int file descriptor on POSIX. T_SOCP is its printf conversion,
'%' included: unsigned __int64 on Win64 must not be printed with "%d". */
#ifdef _WIN32
#if defined(_WIN64)
typedef unsigned __int64 T_SOC;
#define T_SOCP "%" PRIu64
#else
typedef unsigned __int32 T_SOC;
#define T_SOCP "%" PRIu32
#endif
#else
typedef int T_SOC;
#define T_SOCP "%d"
#endif
/* Buffer size for a printed network address (IPv4 or IPv6, NUL included). */

View File

@@ -124,7 +124,9 @@ typedef struct help_wizard_buffers {
char stropt[2048]; // options
char stropt2[2048]; // options longues
char strwild[2048]; // wildcards
char cmd[4096];
/* holds all four of the above plus separators: at 4096 a long answer set
clipped the filters off the command line */
char cmd[HTS_URLMAXSIZE * 2 + 3 * 2048 + 4];
char str[256];
char *argv[256];
} help_wizard_buffers;
@@ -156,9 +158,7 @@ void help_wizard(httrackp * opt) {
char *a;
//
if (urls == NULL || mainpath == NULL || projname == NULL || stropt == NULL
|| stropt2 == NULL || strwild == NULL || cmd == NULL || str == NULL
|| argv == NULL) {
if (buffers == NULL) {
fprintf(stderr, "* memory exhausted in %s, line %d\n", __FILE__, __LINE__);
return;
}
@@ -251,6 +251,7 @@ void help_wizard(httrackp * opt) {
strcatbuff(stropt2, "--update ");
break;
case 0:
freet(buffers);
return;
break;
}
@@ -309,14 +310,23 @@ void help_wizard(httrackp * opt) {
printf("\n");
if (strlen(stropt) == 1)
stropt[0] = '\0'; // aucune
snprintf(cmd, sizeof(cmd), "%s %s %s %s", urls, stropt, stropt2, strwild);
/* the tail is the filter list, and cmd is split into the argv handed to
hts_main() below: a clipped line would silently widen the crawl */
if (!sprintfbuff(cmd, "%s %s %s %s", urls, stropt, stropt2, strwild)) {
printf("* command line too long (%d bytes max)\n",
(int) sizeof(cmd) - 1);
freet(buffers);
return;
}
printf("---> Wizard command line: httrack %s\n\n", cmd);
printf("Ready to launch the mirror? (Y/n) :");
fflush(stdout);
linput(stdin, str, 250);
if (strnotempty(str)) {
if (!((str[0] == 'y') || (str[0] == 'Y')))
if (!((str[0] == 'y') || (str[0] == 'Y'))) {
freet(buffers);
return;
}
}
printf("\n");
@@ -340,7 +350,7 @@ void help_wizard(httrackp * opt) {
}
/* Free buffers */
free(buffers);
freet(buffers);
#undef urls
#undef mainpath
#undef projname
@@ -412,9 +422,9 @@ void help_catchurl(const char *dest_path) {
do {
snprintf(dest, sizeof(dest), "%s%s%d", dest_path, "hts-post", i);
i++;
} while(fexist(dest));
} while (fexist_utf8(dest));
{
FILE *fp = fopen(dest, "wb");
FILE *fp = FOPEN(dest, "wb");
if (fp) {
fwrite(data, strlen(data), 1, fp);
@@ -524,6 +534,15 @@ void help(const char *app, int more) {
infomsg
(" %D cached delayed type check, don't wait for remote type during updates, to speedup them (%D0 wait, * %D1 don't wait)");
infomsg(" %M generate a RFC MIME-encapsulated full-archive (.mht)");
infomsg(" %Z after the mirror, rewrite each saved page with its "
"stylesheets, scripts, images and fonts inlined as data: URIs, so "
"any page opens by double-click anywhere (links between pages stay "
"relative; audio and video stay links); --single-file-max-size N "
"caps each asset (default 10485760 bytes). %M is the better "
"container where a Chromium-family browser is a given: one archive, "
"no base64 tax on text, a shared asset stored once");
infomsg(" %t keep the original file extension, don't rewrite it from the "
"MIME type (%t0 rewrite)");
infomsg
(" LN long names (L1 *long names / L0 8-3 conversion / L2 ISO9660 compatible)");
infomsg
@@ -554,6 +573,7 @@ void help(const char *app, int more) {
(" %h force HTTP/1.0 requests (reduce update features, only for old servers or proxies)");
infomsg
(" %k use keep-alive if possible, greately reducing latency for small files and test requests (%k0 don't use)");
infomsg(" %z do not request compressed content (%z0 request)");
infomsg
(" %B tolerant requests (accept bogus responses on some servers, but not standard!)");
infomsg
@@ -578,8 +598,10 @@ void help(const char *app, int more) {
(" F user-agent field sent in HTTP headers (-F \"user-agent name\")");
infomsg(" %R default referer field sent in HTTP headers");
infomsg(" %E from email address sent in HTTP headers");
infomsg
(" %F footer string in Html code (-%F \"Mirrored [from host %s [file %s [at %s]]]\"");
infomsg(
" %F footer string in Html code (-%F \"Mirrored from {url} on "
"{date}\"; fields {addr} {path} {url} {date} {lastmodified} {version} "
"{mime} {charset} {status} {size}, or legacy %s)");
infomsg(" %l preferred language (-%l \"fr, en, jp, *\"");
infomsg(" %a accepted formats (-%a \"text/html,image/png;q=0.9,*/*;q=0.1\"");
infomsg(" %X additional HTTP header line (-%X \"X-Magic: 42\"");
@@ -588,6 +610,10 @@ void help(const char *app, int more) {
infomsg
(" C create/use a cache for updates and retries (C0 no cache,C1 cache is prioritary,* C2 test update before)");
infomsg(" k store all files in cache (not useful if files on disk)");
infomsg(" %r write an ISO-28500 WARC/1.1 archive; --warc-file NAME sets the "
"output name, --warc-max-size N rotates segments past N bytes, "
"--warc-cdx also writes a sorted CDXJ index, --wacz packages it all "
"as a WACZ file");
infomsg(" %n do not re-download locally erased files");
infomsg
(" %v display on screen filenames downloaded (in realtime) - * %v1 short version - %v2 full animation");
@@ -620,7 +646,6 @@ void help(const char *app, int more) {
infomsg(" %H debug HTTP headers in logfile");
infomsg("");
infomsg("Guru options: (do NOT use if possible)");
infomsg(" #X *use optimized engine (limited memory boundary checks)");
infomsg(" #test list engine self-tests (run one with -#test=NAME [args])");
infomsg(" #C cache list (-#C '*.com/spider*.gif'");
infomsg(" #R cache repair (damaged cache)");
@@ -633,7 +658,6 @@ void help(const char *app, int more) {
infomsg(" #L maximum number of links (-#L1000000)");
infomsg(" #p display ugly progress information");
infomsg(" #P catch URL");
infomsg(" #R old FTP routines (debug)");
infomsg(" #T generate transfer ops. log every minutes");
infomsg(" #u wait time");
infomsg(" #Z generate transfer rate statistics every minutes");
@@ -650,6 +674,7 @@ void help(const char *app, int more) {
(" V execute system command after each files ($0 is the filename: -V \"rm \\$0\")");
infomsg(" %W use an external library function as a wrapper (-%W "
"myfoo.so[,myparameters])");
infomsg(" y go to background when suspended (y0 don't)");
infomsg("");
infomsg("Details: Option N");
infomsg(" N0 Site-structure (default)");

View File

@@ -349,9 +349,13 @@ void index_finish(const char *indexpath, int mode) {
// Write new file
if (mode == 1) // TEXT
fp = fopen(concat(catbuff, sizeof(catbuff), indexpath, "index.txt"), "wb");
fp = FOPEN(
concat(catbuff, sizeof(catbuff), indexpath, "index.txt"),
"wb");
else // HTML
fp = fopen(concat(catbuff, sizeof(catbuff), indexpath, "sindex.html"), "wb");
fp = FOPEN(
concat(catbuff, sizeof(catbuff), indexpath, "sindex.html"),
"wb");
if (fp) {
char current_word[KEYW_LEN + 32];
char word[KEYW_LEN + 32];

View File

@@ -36,6 +36,8 @@ Please visit our Website: http://www.httrack.com
// Fichier librairie .c
#include "htscore.h"
#include "htswarc.h"
#include "htssinglefile.h"
/* specific definitions */
#include "htsbase.h"
@@ -703,18 +705,16 @@ T_SOC http_xfopen(httrackp *opt, int mode, int treat, int waitconnect,
/* Check for errors */
if (soc == INVALID_SOCKET) {
if (retour) {
if (retour->msg) {
if (!strnotempty(retour->msg)) {
if (!strnotempty(retour->msg)) {
#ifdef _WIN32
int last_errno = WSAGetLastError();
int last_errno = WSAGetLastError();
sprintf(retour->msg, "Connect error: %s", strerror(last_errno));
htsblk_failf(retour, "Connect error: %s", strerror(last_errno));
#else
int last_errno = errno;
int last_errno = errno;
sprintf(retour->msg, "Connect error: %s", strerror(last_errno));
htsblk_failf(retour, "Connect error: %s", strerror(last_errno));
#endif
}
}
}
}
@@ -1201,6 +1201,11 @@ int http_sendhead(httrackp * opt, t_cookie * cookie, int mode,
} // Fin test pas postfile
//
// Stash the raw request for the WARC request record (freed at
// back_clear_entry)
if (StringNotEmpty(opt->warc_file))
warc_stash_request(retour, bstr.buffer);
// Callback
{
int test_head =
@@ -2114,6 +2119,8 @@ htsblk http_test(httrackp * opt, const char *adr, const char *fil, char *loc) {
#if HTS_DEBUG_CLOSESOCK
DEBUG_W("http_test: deletehttp\n");
#endif
// this probe's htsblk is discarded by callers, so free any WARC stash here
warc_free_request(&retour);
deletehttp(&retour);
retour.soc = INVALID_SOCKET;
}
@@ -2199,7 +2206,7 @@ T_SOC newhttp_addr(httrackp *opt, const char *_iadr, htsblk *retour, int port,
#if DEBUG
printf("erreur gethostbyname\n");
#endif
if (retour && retour->msg) {
if (retour != NULL) {
#ifdef _WIN32
snprintf(retour->msg, sizeof(retour->msg),
"Unable to get server's address: %s", error);
@@ -2230,17 +2237,17 @@ T_SOC newhttp_addr(httrackp *opt, const char *_iadr, htsblk *retour, int port,
DEBUG_W("socket()=%d\n" _(int) soc);
#endif
if (soc == INVALID_SOCKET) {
if (retour && retour->msg) {
if (retour != NULL) {
#ifdef _WIN32
int last_errno = WSAGetLastError();
sprintf(retour->msg, "Unable to create a socket: %s",
strerror(last_errno));
htsblk_failf(retour, "Unable to create a socket: %s",
strerror(last_errno));
#else
int last_errno = errno;
sprintf(retour->msg, "Unable to create a socket: %s",
strerror(last_errno));
htsblk_failf(retour, "Unable to create a socket: %s",
strerror(last_errno));
#endif
}
return INVALID_SOCKET; // erreur création socket impossible
@@ -2254,17 +2261,8 @@ T_SOC newhttp_addr(httrackp *opt, const char *_iadr, htsblk *retour, int port,
&bind_addr, &error) == NULL
|| bind(soc, &SOCaddr_sockaddr(bind_addr),
SOCaddr_size(bind_addr)) != 0) {
if (retour && retour->msg) {
#ifdef _WIN32
snprintf(retour->msg, sizeof(retour->msg),
"Unable to bind the specificied server address: %s",
error);
#else
snprintf(retour->msg, sizeof(retour->msg),
"Unable to bind the specificied server address: %s",
error);
#endif
}
snprintf(retour->msg, sizeof(retour->msg),
"Unable to bind the specificied server address: %s", error);
deletesoc(soc);
return INVALID_SOCKET;
}
@@ -2311,17 +2309,17 @@ T_SOC newhttp_addr(httrackp *opt, const char *_iadr, htsblk *retour, int port,
#if HDEBUG
printf("unable to connect!\n");
#endif
if (retour != NULL && retour->msg) {
if (retour != NULL) {
#ifdef _WIN32
const int last_errno = WSAGetLastError();
sprintf(retour->msg, "Unable to connect to the server: %s",
strerror(last_errno));
htsblk_failf(retour, "Unable to connect to the server: %s",
strerror(last_errno));
#else
const int last_errno = errno;
sprintf(retour->msg, "Unable to connect to the server: %s",
strerror(last_errno));
htsblk_failf(retour, "Unable to connect to the server: %s",
strerror(last_errno));
#endif
}
/* Close the socket and notify the error!!! */
@@ -2540,6 +2538,15 @@ void fil_simplifie(char *f) {
}
}
void htsblk_failf(htsblk *r, const char *fmt, ...) {
va_list args;
va_start(args, fmt);
// deliberate clip: the reason is quoted from a remote reply
(void) vslprintfbuff(r->msg, sizeof(r->msg), fmt, args);
va_end(args);
}
// fermer liaison fichier ou socket
void deletehttp(htsblk * r) {
#if HTS_DEBUG_CLOSESOCK
@@ -2590,13 +2597,15 @@ void deletesoc(T_SOC soc) {
if (closesocket(soc) != 0) {
int err = WSAGetLastError();
fprintf(stderr, "* error closing socket %d: %s\n", soc, strerror(err));
fprintf(stderr, "* error closing socket " T_SOCP ": %s\n", soc,
strerror(err));
}
#else
if (close(soc) != 0) {
const int err = errno;
fprintf(stderr, "* error closing socket %d: %s\n", soc, strerror(err));
fprintf(stderr, "* error closing socket " T_SOCP ": %s\n", soc,
strerror(err));
}
#endif
#if HTS_WIDE_DEBUG
@@ -3009,7 +3018,7 @@ int finput(T_SOC fd, char *s, int max) {
}
// Like linput, but in memory (optimized)
int binput(char *buff, char *s, int max) {
int binput(const char *buff, char *s, int max) {
int count = 0;
int destCount = 0;
@@ -6010,6 +6019,10 @@ HTSEXT_API httrackp *hts_create_opt(void) {
StringCopy(opt->footer, HTS_DEFAULT_FOOTER);
StringCopy(opt->strip_query, "");
StringCopy(opt->cookies_file, "");
StringCopy(opt->warc_file, "");
opt->warc_max_size = 0; /* no rotation unless --warc-max-size sets it */
opt->single_file = HTS_FALSE;
opt->single_file_max_size = SINGLEFILE_DEFAULT_MAX_SIZE;
StringCopy(opt->why_url, "");
opt->pause_min_ms = 0;
opt->pause_max_ms = 0;
@@ -6161,6 +6174,7 @@ HTSEXT_API void hts_free_opt(httrackp * opt) {
StringFree(opt->strip_query);
StringFree(opt->cookies_file);
StringFree(opt->why_url);
StringFree(opt->warc_file);
StringFree(opt->path_html);
StringFree(opt->path_html_utf8);
@@ -6420,21 +6434,28 @@ HTSEXT_API int hts_resetvar(void) {
#ifdef _WIN32
typedef struct dirent dirent;
DIR *opendir(const char *name) {
WIN32_FILE_ATTRIBUTE_DATA st;
DIR *dir;
size_t len;
int i;
LPWSTR wname;
if (name == NULL || *name == '\0') {
errno = ENOENT;
return NULL;
}
if (!GetFileAttributesEx(name, GetFileExInfoStandard, &st)
|| (st.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) == 0) {
// Wide \\?\ path: no MAX_PATH cap, no CP_ACP mis-decode (#133,#630).
wname = hts_pathToUCS2(name);
if (wname == NULL ||
!GetFileAttributesExW(wname, GetFileExInfoStandard, &st) ||
(st.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) == 0) {
freet(wname);
errno = ENOENT;
return NULL;
}
freet(wname);
dir = calloc(sizeof(DIR), 1);
if (dir == NULL) {
errno = ENOMEM;
@@ -6454,19 +6475,29 @@ DIR *opendir(const char *name) {
}
struct dirent *readdir(DIR * dir) {
WIN32_FIND_DATAA find;
WIN32_FIND_DATAW find;
if (dir->h == INVALID_HANDLE_VALUE) {
dir->h = FindFirstFileA(dir->name, &find);
// \\?\-prefix so a long/non-ASCII directory enumerates instead of ENOENT.
LPWSTR wname = hts_pathToUCS2(dir->name);
dir->h =
wname != NULL ? FindFirstFileW(wname, &find) : INVALID_HANDLE_VALUE;
freet(wname);
} else {
if (!FindNextFile(dir->h, &find)) {
if (!FindNextFileW(dir->h, &find)) {
FindClose(dir->h);
dir->h = INVALID_HANDLE_VALUE;
}
}
if (dir->h != INVALID_HANDLE_VALUE) {
char *u = hts_convertUCS2StringToUTF8(find.cFileName, -1);
dir->entry.d_name[0] = 0;
strncat(dir->entry.d_name, find.cFileName, HTS_DIRENT_SIZE - 1);
if (u != NULL) {
strncat(dir->entry.d_name, u, HTS_DIRENT_SIZE - 1);
freet(u);
}
return &dir->entry;
}
errno = ENOENT;
@@ -6499,9 +6530,61 @@ static void copyWchar(LPWSTR dest, const char *src) {
dest[i] = '\0';
}
/* UTF-8 path -> UCS-2 for the _w* file APIs. At/above HTS_WIN_LONGPATH_MIN,
\\?\-prefix it via GetFullPathNameW to clear MAX_PATH (#133); else unchanged.
Any prefixing failure falls back to the plain converted path. */
#define HTS_WIN_LONGPATH_MIN 240 /* stay clear of MAX_PATH (260) */
LPWSTR hts_pathToUCS2(const char *path) {
LPWSTR wpath = hts_convertUTF8StringToUCS2(path, (int) strlen(path), NULL);
if (wpath == NULL) {
return NULL;
}
const size_t len = wcslen(wpath);
// Already "\\?\" or "\\.\": don't re-prefix.
const int verbatim = len >= 4 && wpath[0] == L'\\' && wpath[1] == L'\\' &&
(wpath[2] == L'?' || wpath[2] == L'.') &&
wpath[3] == L'\\';
if (len < HTS_WIN_LONGPATH_MIN || verbatim) {
return wpath;
}
const DWORD need = GetFullPathNameW(wpath, 0, NULL, NULL); /* incl NUL */
LPWSTR full = need != 0 ? malloct((size_t) need * sizeof(WCHAR)) : NULL;
if (full == NULL) {
return wpath; /* fall back to the plain path */
}
const DWORD written = GetFullPathNameW(wpath, need, full, NULL);
if (written == 0 || written >= need || full[0] == L'\0') {
freet(full);
return wpath;
}
const int isUNC = full[0] == L'\\' && full[1] == L'\\';
// UNC "\\srv\share" -> "\\?\UNC\srv\share": the prefix subsumes the "\\".
const WCHAR *const pfx = isUNC ? L"\\\\?\\UNC\\" : L"\\\\?\\";
const WCHAR *const body = isUNC ? full + 2 : full;
const size_t pfxLen = wcslen(pfx), bodyLen = wcslen(body);
LPWSTR out = malloct((pfxLen + bodyLen + 1) * sizeof(WCHAR));
if (out == NULL) {
freet(full);
return wpath;
}
memcpybuff(out, pfx, pfxLen * sizeof(WCHAR));
memcpybuff(out + pfxLen, body, (bodyLen + 1) * sizeof(WCHAR));
freet(full);
freet(wpath);
return out;
}
FILE *hts_fopen_utf8(const char *path, const char *mode) {
WCHAR wmode[32];
LPWSTR wpath = hts_convertUTF8StringToUCS2(path, (int) strlen(path), NULL);
LPWSTR wpath = hts_pathToUCS2(path);
assertf(strlen(mode) < sizeof(wmode) / sizeof(WCHAR));
copyWchar(wmode, mode);
@@ -6517,7 +6600,7 @@ FILE *hts_fopen_utf8(const char *path, const char *mode) {
}
int hts_stat_utf8(const char *path, STRUCT_STAT * buf) {
LPWSTR wpath = hts_convertUTF8StringToUCS2(path, (int) strlen(path), NULL);
LPWSTR wpath = hts_pathToUCS2(path);
if (wpath != NULL) {
const int result = _wstat64(wpath, buf);
@@ -6531,7 +6614,7 @@ int hts_stat_utf8(const char *path, STRUCT_STAT * buf) {
}
int hts_unlink_utf8(const char *path) {
LPWSTR wpath = hts_convertUTF8StringToUCS2(path, (int) strlen(path), NULL);
LPWSTR wpath = hts_pathToUCS2(path);
if (wpath != NULL) {
const int result = _wunlink(wpath);
@@ -6545,10 +6628,8 @@ int hts_unlink_utf8(const char *path) {
}
int hts_rename_utf8(const char *oldpath, const char *newpath) {
LPWSTR woldpath =
hts_convertUTF8StringToUCS2(oldpath, (int) strlen(oldpath), NULL);
LPWSTR wnewpath =
hts_convertUTF8StringToUCS2(newpath, (int) strlen(newpath), NULL);
LPWSTR woldpath = hts_pathToUCS2(oldpath);
LPWSTR wnewpath = hts_pathToUCS2(newpath);
if (woldpath != NULL && wnewpath != NULL) {
const int result = _wrename(woldpath, wnewpath);
@@ -6565,8 +6646,22 @@ int hts_rename_utf8(const char *oldpath, const char *newpath) {
}
}
int hts_rmdir_utf8(const char *path) {
LPWSTR wpath = hts_pathToUCS2(path);
if (wpath != NULL) {
const int result = _wrmdir(wpath);
free(wpath);
return result;
} else {
// Fallback on conversion error.
return _rmdir(path);
}
}
int hts_mkdir_utf8(const char *path) {
LPWSTR wpath = hts_convertUTF8StringToUCS2(path, (int) strlen(path), NULL);
LPWSTR wpath = hts_pathToUCS2(path);
if (wpath != NULL) {
const int result = _wmkdir(wpath);
@@ -6581,7 +6676,7 @@ int hts_mkdir_utf8(const char *path) {
HTSEXT_API int hts_utime_utf8(const char *path, const STRUCT_UTIMBUF * times) {
STRUCT_UTIMBUF mtimes = *times;
LPWSTR wpath = hts_convertUTF8StringToUCS2(path, (int) strlen(path), NULL);
LPWSTR wpath = hts_pathToUCS2(path);
if (wpath != NULL) {
const int result = _wutime(wpath, &mtimes);

View File

@@ -199,6 +199,10 @@ T_SOC newhttp(httrackp * opt, const char *iadr, htsblk * retour, int port,
etc.). */
T_SOC newhttp_addr(httrackp *opt, const char *iadr, htsblk *retour, int port,
int waitconnect, int addr_index, int *addr_count);
/* Clips the formatted failure reason into r->msg, which also round-trips
through the cache as X-StatusMessage. Leaves r->statuscode to the caller. */
void htsblk_failf(htsblk *r, const char *fmt, ...) HTS_PRINTF_FUN(2, 3);
HTS_INLINE void deletehttp(htsblk * r);
HTS_INLINE int deleteaddr(htsblk * r);
HTS_INLINE void deletesoc(T_SOC soc);
@@ -262,7 +266,7 @@ HTS_INLINE void time_rfc822_local(char *s, struct tm *A);
HTS_INLINE int sendc(htsblk * r, const char *s);
int finput(T_SOC fd, char *s, int max);
int binput(char *buff, char *s, int max);
int binput(const char *buff, char *s, int max);
int linput(FILE * fp, char *s, int max);
int linputsoc(T_SOC soc, char *s, int max);
int linputsoc_t(T_SOC soc, char *s, int max, int timeout);
@@ -607,9 +611,26 @@ static HTS_UNUSED size_t llint_to_size_t(LLint o) {
}
}
/* Capacity for @p used bytes plus @p extra more plus @p slack spare;
(size_t) -1 if the total exceeds (size_t) -2 or @p extra is negative
(llint_to_size_t() would map that to a huge valid-looking size). */
static HTS_UNUSED size_t llint_grow_size_t(size_t used, LLint extra,
size_t slack) {
const size_t max = (size_t) -2; /* (size_t) -1 is the error value */
const size_t e = extra >= 0 ? llint_to_size_t(extra) : (size_t) -1;
if (e == (size_t) -1 || used > max || slack > max - used ||
e > max - used - slack) {
return (size_t) -1;
}
return used + e + slack;
}
/* dirent() compatibility */
#ifdef _WIN32
#define HTS_DIRENT_SIZE 256
/* Holds a UTF-8 d_name: MAX_PATH (260) UTF-16 units expand to <=3 bytes each.
Windows-only struct, ABI free to break. */
#define HTS_DIRENT_SIZE 1024
struct dirent {
ino_t d_ino; /* ignored */
off_t d_off; /* ignored */

View File

@@ -47,6 +47,7 @@ Please visit our Website: http://www.httrack.com
#include "htszlib.h"
#endif
#include <ctype.h>
#include <limits.h>
#define ADD_STANDARD_PATH \
{ /* ajout nom */\
@@ -1475,15 +1476,37 @@ int url_savename(lien_adrfilsave *const afs,
sizeof(afs->save) - (size_t) (lastDot - afs->save));
}
}
// enforce 260-character path limit before inserting destination path
// note: 12 characters at least for WIN32, and 12 for ".99.delayed"
// (MSDN) "When using an API to create a directory, the specified path
// cannot be so long that you cannot append an 8.3 file name
// (that is, the directory name cannot exceed MAX_PATH minus 12)."
#define HTS_MAX_PATH_LEN ( 260 - 12 - 12 )
// Cap the save path: the final parent+name is copied into a fixed buffer that
// aborts() on overflow (htssafe.h), so clamp every ceiling to fit it.
#define HTS_SAVE_BUFSIZE (HTS_URLMAXSIZE * 2) /* sizeof(afs->save) */
#define HTS_PATH_TAIL_RESERVE 64 /* collision suffix + ".delayed" + NUL */
#ifdef _WIN32
// MAX_PATH minus 8.3 headroom (MSDN) minus the ".delayed" marker; raising it
// needs the engine to "\\?\"-prefix its paths, which is separate work.
#define HTS_MAX_PATH_LEN (260 - 12 - 12)
#define MAX_SEG_LEN 48
#else
// #133: use the platform's own PATH_MAX/NAME_MAX (Linux/Android 4096, macOS
// 1024) rather than the far smaller Windows MAX_PATH.
#ifdef PATH_MAX
#define HTS_PATH_MAX_ PATH_MAX
#else
#define HTS_PATH_MAX_ 1024
#endif
#ifdef NAME_MAX
#define HTS_NAME_MAX_ NAME_MAX
#else
#define HTS_NAME_MAX_ 255
#endif
#define HTS_MAX_PATH_LEN \
((HTS_PATH_MAX_ - HTS_PATH_TAIL_RESERVE) < \
(HTS_SAVE_BUFSIZE - HTS_PATH_TAIL_RESERVE) \
? (HTS_PATH_MAX_ - HTS_PATH_TAIL_RESERVE) \
: (HTS_SAVE_BUFSIZE - HTS_PATH_TAIL_RESERVE))
#define MAX_SEG_LEN (HTS_NAME_MAX_ > 64 ? HTS_NAME_MAX_ - 16 : HTS_NAME_MAX_)
#endif
#define MIN_LAST_SEG_RESERVE 12
#define MAX_LAST_SEG_RESERVE 24
#define MAX_SEG_LEN 48
if (hts_stringLengthUTF8(afs->save) +
hts_stringLengthUTF8(StringBuff(opt->path_html_utf8)) >=
HTS_MAX_PATH_LEN) {
@@ -1493,7 +1516,7 @@ int url_savename(lien_adrfilsave *const afs,
if (wsave != NULL) {
const size_t parentLen =
hts_stringLengthUTF8(StringBuff(opt->path_html_utf8));
// parent path length is not insane (otherwise, ignore and pick 200 as
// parent path length is not insane (otherwise, ignore and pick 200 as
// suffix length)
const size_t maxLen =
parentLen <
@@ -1584,9 +1607,37 @@ int url_savename(lien_adrfilsave *const afs,
// Re-check again ending space or dot after cut (see bug #5)
cleanEndingSpaceOrDot(afs->save);
}
// The cut above counts UTF-8 codepoints, but parent+name lands in a fixed
// byte buffer that aborts() on overflow (htssafe.h). A multibyte name can
// pass the codepoint cap yet overflow in bytes, so hard-cut on a codepoint
// boundary to keep parent+name inside the buffer regardless (#133).
{
const size_t parentBytes = strlen(StringBuff(opt->path_html_utf8));
const size_t cap = HTS_SAVE_BUFSIZE - HTS_PATH_TAIL_RESERVE;
// Shrink only the name. A parent that alone fills the buffer is left to the
// existing prepend abort, not collapsed to an empty name that would collide
// across URLs and overrun the unbounded collision-suffix sprintf.
if (parentBytes < cap) {
size_t budget = cap - parentBytes;
if (strlen(afs->save) > budget) {
while (budget > 0 && ((unsigned char) afs->save[budget] & 0xC0) == 0x80)
budget--; // back off a continuation byte, never split a char
afs->save[budget] = '\0';
cleanEndingSpaceOrDot(afs->save);
}
}
}
#undef MAX_UTF8_SEQ_CHARS
#undef MIN_LAST_SEG_RESERVE
#undef MAX_LAST_SEG_RESERVE
#undef MAX_SEG_LEN
#undef HTS_MAX_PATH_LEN
#undef HTS_PATH_TAIL_RESERVE
#undef HTS_SAVE_BUFSIZE
#ifndef _WIN32
#undef HTS_PATH_MAX_
#undef HTS_NAME_MAX_
#endif
// chemin primaire éventuel A METTRE AVANT
if (strnotempty(StringBuff(opt->path_html_utf8))) {

View File

@@ -256,6 +256,7 @@ struct htsoptstate {
unsigned int debug_state;
unsigned int tmpnameid; /**< counter for temporary file names */
int is_ended; /**< mirror has finished */
void *warc; /**< open WARC writer (warc_writer*), or NULL */
};
/* Library handles */
@@ -538,6 +539,20 @@ struct httrackp {
int pause_max_ms; /**< inter-file pause upper bound, ms */
String why_url; /**< URL to diagnose (--why): print the deciding filter rule
and exit without crawling */
String warc_file; /**< WARC output: WARC_AUTONAME for --warc, or the
--warc-file basename (appended at the tail: ABI) */
LLint warc_max_size; /**< --warc-max-size: rotate the archive past this many
bytes (<=0: single file). Tail: ABI */
hts_boolean warc_cdx; /**< --warc-cdx: write a sorted CDXJ index next to the
archive. Tail: ABI */
hts_boolean warc_wacz; /**< --wacz: package archive+index+pages as a WACZ zip
(implies --warc + --warc-cdx). Tail: ABI */
hts_boolean single_file; /**< --single-file: once the mirror is done, rewrite
each saved page with its assets inlined as
data: URIs. Tail: ABI */
LLint single_file_max_size; /**< --single-file-max-size: per-asset cap in
bytes; a bigger asset stays a link.
Tail: ABI */
};
/* Running statistics for a mirror. */
@@ -661,6 +676,14 @@ struct htsblk {
/* Restart-whole signal: a resume this response rejected (unusable 206) must
retry with no Range, else a surviving partial/temp-ref loops (#581). */
hts_boolean refetch_wholefile;
char *warc_reqhdr; /**< stashed raw request header block for WARC (or NULL) */
char *
warc_resphdr; /**< stashed raw response header block for WARC (or NULL) */
int warc_truncated; /**< WARC-Truncated reason for a cap-truncated body
(WARC_TRUNC_*, 0=none). Tail: ABI */
char *warc_rawpath; /**< verbatim WARC: spooled compressed body path, or NULL
(owns the file; unlinked on free). Tail: ABI */
LLint warc_rawsize; /**< byte length of warc_rawpath. Tail: ABI */
/*char digest[32+2]; // md5 digest generated by the engine ("" if none) */
};

View File

@@ -792,17 +792,72 @@ int htsparse(htsmoduleStruct * str, htsmoduleStructExtended * stre) {
char gmttime[256];
char BIGSTK safe_adr[HTS_URLMAXSIZE * 3 + 4];
char BIGSTK safe_fil[HTS_URLMAXSIZE * 3 + 4];
char BIGSTK safe_url[HTS_URLMAXSIZE * 6 + 16];
char safe_lastmod[sizeof(r->lastmodified) * 3 + 4];
char safe_ctype[sizeof(r->contenttype) * 3 + 4];
char safe_charset[sizeof(r->charset) * 3 + 4];
char status_str[16];
char size_str[32];
// {url} scheme: jump_identification_const strips it for
// http/https/ftp, so re-add it (bare host is http); for any
// other scheme the host keeps its "scheme://" and we add
// none.
const char *const url_host =
jump_identification_const(urladr());
const char *const url_scheme =
strstr(url_host, "://") != NULL ? ""
: strfield(urladr(), "https://") ? "https://"
: strfield(urladr(), "ftp://") ? "ftp://"
: "http://";
tempo[0] = '\0';
// {addr}/{path} are the escaped host and remote path; {url}
// prepends the scheme to them (credentials already stripped
// by jump_identification_const, and the scheme is a safe
// literal, so no re-escaping is needed).
html_inline_safe(url_host, safe_adr, sizeof(safe_adr));
html_inline_safe(urlfil(), safe_fil, sizeof(safe_fil));
snprintf(safe_url, sizeof(safe_url), "%s%s%s", url_scheme,
safe_adr, safe_fil);
snprintf(status_str, sizeof(status_str), "%d", r->statuscode);
snprintf(size_str, sizeof(size_str), LLintP, (LLint) r->size);
time_gmt_rfc822(gmttime);
strcatbuff(tempo, eol);
hts_template_format_str(tempo + strlen(tempo), sizeof(tempo) - strlen(tempo),
StringBuff(opt->footer),
html_inline_safe(jump_identification_const(urladr()), safe_adr, sizeof(safe_adr)),
html_inline_safe(urlfil(), safe_fil, sizeof(safe_fil)), gmttime,
HTTRACK_VERSIONID, /* EOF */ NULL);
strcatbuff(tempo, eol);
HT_ADD(tempo);
{
// Every network-derived string is html_inline_safe()'d: the
// footer sits inside an HTML comment, so a value holding
// "-->" would otherwise close it and inject markup (#165).
// status/size are formatted integers and need no escaping.
const hts_footer_field fields[] = {
{"addr", safe_adr},
{"path", safe_fil},
{"url", safe_url},
{"date", gmttime},
{"lastmodified",
html_inline_safe(r->lastmodified, safe_lastmod,
sizeof(safe_lastmod))},
{"version", HTTRACK_VERSIONID},
{"mime", html_inline_safe(r->contenttype, safe_ctype,
sizeof(safe_ctype))},
{"charset", html_inline_safe(r->charset, safe_charset,
sizeof(safe_charset))},
{"status", status_str},
{"size", size_str},
};
tempo[0] = '\0';
strcatbuff(tempo, eol);
// hts_footer_format returns <0 on overflow, leaving tempo
// unterminated; emitting it would abort in strcatbuff
// below.
if (hts_footer_format(tempo + strlen(tempo),
sizeof(tempo) - strlen(tempo),
StringBuff(opt->footer), fields,
sizeof(fields) / sizeof(fields[0])) >=
0) {
strcatbuff(tempo, eol);
HT_ADD(tempo);
}
}
}
// Emit charset ?
if (emited_footer == 1 && strnotempty(r->charset)) {
@@ -3601,7 +3656,7 @@ int hts_mirror_check_moved(htsmoduleStruct * str,
!ref_existed;
if (fexist_utf8(heap(ptr)->sav)) {
had_partial = 1;
remove(heap(ptr)->sav);
UNLINK(heap(ptr)->sav);
}
// Re-get once, only if a partial existed and both Range triggers are
@@ -3807,18 +3862,13 @@ void hts_mirror_process_user_interaction(htsmoduleStruct * str,
int do_pause = 0;
// user pause lockfile : create hts-paused.lock --> HTTrack will be paused
if (fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-stop.lock"))) {
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-stop.lock"))) {
// remove lockfile
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-stop.lock"));
if (!fexist
(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-stop.lock"))) {
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-stop.lock"));
if (!fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-stop.lock"))) {
do_pause = 1;
}
}
@@ -3847,11 +3897,9 @@ void hts_mirror_process_user_interaction(htsmoduleStruct * str,
}
}
{
FILE *fp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-paused.lock"), "wb");
FILE *fp = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-paused.lock"),
"wb");
if (fp) {
fspc(NULL, fp, "info"); // dater
fprintf(fp,
@@ -4167,18 +4215,17 @@ int hts_mirror_wait_for_next_file(htsmoduleStruct * str,
int a = 0;
*stre->last_info_shell_ = tl;
if (fexist(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt), StringBuff(opt->path_log), "hts-autopsy"))) { // débuggage: teste si le robot est vivant
if (fexist_utf8(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-autopsy"))) { // débuggage: teste si le
// robot est vivant
// (oui je sais un robot vivant.. mais bon.. il a le droit de vivre lui aussi)
// (libérons les robots esclaves de l'internet!)
remove(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-autopsy"));
fp =
fopen(fconcat
(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log),
"hts-isalive"), "wb");
UNLINK(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-autopsy"));
fp = FOPEN(fconcat(OPT_GET_BUFF(opt), OPT_GET_BUFF_SIZE(opt),
StringBuff(opt->path_log), "hts-isalive"),
"wb");
a = 1;
}
if ((*stre->info_shell_) || a) {
@@ -4536,7 +4583,7 @@ int hts_wait_delayed(htsmoduleStruct * str, lien_adrfilsave *afs,
/* seen as in error */
in_error = back[b].r.statuscode;
in_error_msg[0] = 0;
strncat(in_error_msg, back[b].r.msg, sizeof(in_error_msg) - 1);
strncatbuff(in_error_msg, back[b].r.msg, sizeof(in_error_msg) - 1);
in_error_size = back[b].r.totalsize;
/* don't break, even with "don't take error pages" switch, because we need to process the slot anyway (and cache the error) */
}

View File

@@ -173,8 +173,8 @@ int http_proxy_tunnel(httrackp *opt, htsblk *retour, const char *adr,
if (sscanf(line, "HTTP/%*d.%*d %d", &code) < 1)
code = 0;
if (code < 200 || code >= 300) {
snprintf(retour->msg, sizeof(retour->msg), "proxy CONNECT refused: %s",
strnotempty(line) ? line : "(no status)");
htsblk_failf(retour, "proxy CONNECT refused: %s",
strnotempty(line) ? line : "(no status)");
return 0;
}

View File

@@ -33,6 +33,7 @@ Please visit our Website: http://www.httrack.com
#ifndef HTSSAFE_DEFH
#define HTSSAFE_DEFH
#include <stdarg.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
@@ -116,6 +117,15 @@ static HTS_UNUSED void abortf_(const char *exp, const char *file, int line) {
#endif
#define HTS_IS_NOT_CHAR_BUFFER(VAR) (!HTS_IS_CHAR_BUFFER(VAR))
/* Source capacity for the buff() family, (size_t)-1 when unknown; sizeof of the
TYPE keeps a decayed operand ("buf + 1") off -Wsizeof-array-decay. */
#if (defined(__GNUC__) && !defined(__cplusplus))
#define HTS_SIZEOF_SRC_(B) \
(HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(__typeof__(B)))
#else
#define HTS_SIZEOF_SRC_(B) (HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B))
#endif
/* Compile-time checks. */
static HTS_UNUSED void htssafe_compile_time_check_(void) {
char array[32];
@@ -205,19 +215,17 @@ static char *strncatbuff_ptr_(char *dest, const char *src, size_t n) {
#if (defined(__GNUC__) && !defined(__cplusplus))
#define strncatbuff(A, B, N) \
__builtin_choose_expr( \
HTS_IS_CHAR_BUFFER(A), \
strncat_safe_(A, sizeof(A), B, \
HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), N, \
"overflow while appending '" #B "' to '" #A "'", __FILE__, \
__LINE__), \
strncatbuff_ptr_((A), (B), (N)))
__builtin_choose_expr(HTS_IS_CHAR_BUFFER(A), \
strncat_safe_(A, sizeof(A), B, HTS_SIZEOF_SRC_(B), N, \
"overflow while appending '" #B \
"' to '" #A "'", \
__FILE__, __LINE__), \
strncatbuff_ptr_((A), (B), (N)))
#else
#define strncatbuff(A, B, N) \
(HTS_IS_NOT_CHAR_BUFFER(A) \
? strncat(A, B, N) \
: strncat_safe_(A, sizeof(A), B, \
HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), N, \
: strncat_safe_(A, sizeof(A), B, HTS_SIZEOF_SRC_(B), N, \
"overflow while appending '" #B "' to '" #A "'", \
__FILE__, __LINE__))
#endif
@@ -232,9 +240,7 @@ static char *strncatbuff_ptr_(char *dest, const char *src, size_t n) {
#define strcatbuff(A, B) \
__builtin_choose_expr( \
HTS_IS_CHAR_BUFFER(A), \
strncat_safe_(A, sizeof(A), B, \
HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), \
(size_t) -1, \
strncat_safe_(A, sizeof(A), B, HTS_SIZEOF_SRC_(B), (size_t) -1, \
"overflow while appending '" #B "' to '" #A "'", __FILE__, \
__LINE__), \
strcatbuff_ptr_((A), (B)))
@@ -242,9 +248,7 @@ static char *strncatbuff_ptr_(char *dest, const char *src, size_t n) {
#define strcatbuff(A, B) \
(HTS_IS_NOT_CHAR_BUFFER(A) \
? strcat(A, B) \
: strncat_safe_(A, sizeof(A), B, \
HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), \
(size_t) -1, \
: strncat_safe_(A, sizeof(A), B, HTS_SIZEOF_SRC_(B), (size_t) -1, \
"overflow while appending '" #B "' to '" #A "'", \
__FILE__, __LINE__))
#endif
@@ -257,19 +261,17 @@ static char *strncatbuff_ptr_(char *dest, const char *src, size_t n) {
#if (defined(__GNUC__) && !defined(__cplusplus))
#define strcpybuff(A, B) \
__builtin_choose_expr( \
HTS_IS_CHAR_BUFFER(A), \
strcpy_safe_(A, sizeof(A), B, \
HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), \
"overflow while copying '" #B "' to '" #A "'", __FILE__, \
__LINE__), \
strcpybuff_ptr_((A), (B)))
__builtin_choose_expr(HTS_IS_CHAR_BUFFER(A), \
strcpy_safe_(A, sizeof(A), B, HTS_SIZEOF_SRC_(B), \
"overflow while copying '" #B "' to '" #A \
"'", \
__FILE__, __LINE__), \
strcpybuff_ptr_((A), (B)))
#else
#define strcpybuff(A, B) \
(HTS_IS_NOT_CHAR_BUFFER(A) \
? strcpy(A, B) \
: strcpy_safe_(A, sizeof(A), B, \
HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), \
: strcpy_safe_(A, sizeof(A), B, HTS_SIZEOF_SRC_(B), \
"overflow while copying '" #B "' to '" #A "'", __FILE__, \
__LINE__))
#endif
@@ -286,24 +288,24 @@ static char *strncatbuff_ptr_(char *dest, const char *src, size_t n) {
* Append characters of "B" to "A", "A" having a maximum capacity of "S".
*/
#define strlcatbuff(A, B, S) \
strncat_safe_(A, S, B, HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), \
(size_t) -1, "overflow while appending '" #B "' to '" #A "'", \
__FILE__, __LINE__)
strncat_safe_(A, S, B, HTS_SIZEOF_SRC_(B), (size_t) -1, \
"overflow while appending '" #B "' to '" #A "'", __FILE__, \
__LINE__)
/**
* Append at most "N" characters of "B" to "A", "A" having a maximum capacity
* of "S".
*/
#define strlncatbuff(A, B, S, N) \
strncat_safe_(A, S, B, HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), \
N, "overflow while appending '" #B "' to '" #A "'", __FILE__, \
strncat_safe_(A, S, B, HTS_SIZEOF_SRC_(B), N, \
"overflow while appending '" #B "' to '" #A "'", __FILE__, \
__LINE__)
/**
* Copy characters of "B" to "A", "A" having a maximum capacity of "S".
*/
#define strlcpybuff(A, B, S) \
strcpy_safe_(A, S, B, HTS_IS_NOT_CHAR_BUFFER(B) ? (size_t) -1 : sizeof(B), \
strcpy_safe_(A, S, B, HTS_SIZEOF_SRC_(B), \
"overflow while copying '" #B "' to '" #A "'", __FILE__, \
__LINE__)
@@ -422,7 +424,9 @@ static HTS_INLINE HTS_UNUSED htsbuff htsbuff_ptr_(char *buf, size_t cap) {
*/
static HTS_INLINE HTS_UNUSED void htsbuff_catn(htsbuff *b, const char *s,
size_t n) {
const size_t add = strnlen(s, n);
/* the (size_t)-1 "no limit" sentinel would reach strnlen as a bound past
PTRDIFF_MAX */
const size_t add = n != (size_t) -1 ? strnlen(s, n) : strlen(s);
/* Overflow-safe: keep the (potentially huge) 'add' alone on one side. The
maintained invariant len < cap makes 'cap - len' >= 1 (no underflow), so
'add < cap - len' cannot wrap the way 'len + add < cap' could. */
@@ -456,6 +460,49 @@ static HTS_INLINE HTS_UNUSED const char *htsbuff_str(const htsbuff *b) {
return b->buf;
}
/**
* Callers that deliberately ignore truncation use this instead of
* slprintfbuff(), so it is not HTS_CHECK_RESULT.
*/
static HTS_INLINE HTS_UNUSED HTS_PRINTF_FUN(3, 0) hts_boolean
vslprintfbuff(char *dest, size_t size, const char *fmt, va_list args) {
int ret;
assertf(dest != NULL && size != 0);
ret = vsnprintf(dest, size, fmt, args);
/* pre-C99 runtimes (msvcrt _vsnprintf) return -1 and do not terminate */
dest[size - 1] = '\0';
return ret >= 0 && (size_t) ret < size ? HTS_TRUE : HTS_FALSE;
}
/**
* Formatted print into dest (capacity size, NUL included), truncating to fit
* and always NUL-terminating. Returns HTS_TRUE if the whole output fit; the
* result is the only truncation signal, so it must be acted on. Unlike
* strcpybuff() it never aborts, so it suits text built from remote input.
*/
static HTS_INLINE HTS_UNUSED HTS_CHECK_RESULT HTS_PRINTF_FUN(3, 4) hts_boolean
slprintfbuff(char *dest, size_t size, const char *fmt, ...) {
va_list args;
hts_boolean ret;
va_start(args, fmt);
ret = vslprintfbuff(dest, size, fmt, args);
va_end(args);
return ret;
}
/**
* slprintfbuff() over the in-scope array ARR (capacity = sizeof(ARR)).
* On GCC/Clang a pointer is a compile error; use slprintfbuff() for those.
*/
#if (defined(__GNUC__) && !defined(__cplusplus))
#define sprintfbuff(ARR, ...) \
slprintfbuff((ARR), sizeof(ARR) + htsbuff_must_be_array_(ARR), __VA_ARGS__)
#else
#define sprintfbuff(ARR, ...) slprintfbuff((ARR), sizeof(ARR), __VA_ARGS__)
#endif
/* Thin aliases over the libc allocator/memcpy (historical "t" suffix); no
added bounds checking. freet() also NULLs the freed pointer and tolerates
NULL. memcpybuff() despite the name is a raw memcpy: the caller owns the

File diff suppressed because it is too large Load Diff

View File

@@ -147,7 +147,8 @@ HTS_UNUSED static int LANG_LIST(const char *path, char *buffer, size_t size);
// 0- Init the URL catcher with standard port
// smallserver_init(&port,&return_host);
T_SOC smallserver_init_std(int *port_prox, char *adr_prox, int defaultPort) {
T_SOC smallserver_init_std(int *port_prox, char *adr_prox, int defaultPort,
const char *bindAddr) {
T_SOC soc;
if (defaultPort <= 0) {
@@ -160,12 +161,12 @@ T_SOC smallserver_init_std(int *port_prox, char *adr_prox, int defaultPort) {
int i = 0;
do {
soc = smallserver_init(&try_to_listen_to[i], adr_prox);
soc = smallserver_init(&try_to_listen_to[i], adr_prox, bindAddr);
*port_prox = try_to_listen_to[i];
i++;
} while((soc == INVALID_SOCKET) && (try_to_listen_to[i] >= 0));
} else {
soc = smallserver_init(&defaultPort, adr_prox);
soc = smallserver_init(&defaultPort, adr_prox, bindAddr);
*port_prox = defaultPort;
}
return soc;
@@ -243,9 +244,10 @@ static int my_gethostname(char *h_loc, size_t size) {
}
// smallserver_init(&port,&return_host);
T_SOC smallserver_init(int *port, char *adr) {
T_SOC smallserver_init(int *port, char *adr, const char *bindAddr) {
T_SOC soc = INVALID_SOCKET;
char h_loc[256 + 2];
SOCaddr server;
commandRunning = commandEnd = commandReturn = commandReturnSet =
commandEndRequested = 0;
@@ -256,25 +258,23 @@ T_SOC smallserver_init(int *port, char *adr) {
free(commandReturnCmdl);
commandReturnCmdl = NULL;
if (my_gethostname(h_loc, 256) == 0) { // host name
SOCaddr server;
SOCaddr_initany(server);
if (bindAddr != NULL && *bindAddr != '\0') {
/* advertise the bound address, else the URL we print is unreachable */
if (strlen(bindAddr) >= sizeof(h_loc) || !gethost(bindAddr, &server)) {
return INVALID_SOCKET;
}
strcpybuff(h_loc, bindAddr);
} else if (my_gethostname(h_loc, 256) != 0) { // host name
return INVALID_SOCKET;
}
SOCaddr_initany(server);
if ((soc =
(T_SOC) socket(SOCaddr_sinfamily(server), SOCK_STREAM,
0)) != INVALID_SOCKET) {
SOCaddr_initport(server, *port);
if (bind(soc, &SOCaddr_sockaddr(server), SOCaddr_size(server)) == 0) {
if (listen(soc, 10) >= 0) {
strcpy(adr, h_loc);
} else {
#ifdef _WIN32
closesocket(soc);
#else
close(soc);
#endif
soc = INVALID_SOCKET;
}
if ((soc = (T_SOC) socket(SOCaddr_sinfamily(server), SOCK_STREAM, 0)) !=
INVALID_SOCKET) {
SOCaddr_initport(server, *port);
if (bind(soc, &SOCaddr_sockaddr(server), SOCaddr_size(server)) == 0) {
if (listen(soc, 10) >= 0) {
strcpy(adr, h_loc);
} else {
#ifdef _WIN32
closesocket(soc);
@@ -283,6 +283,13 @@ T_SOC smallserver_init(int *port, char *adr) {
#endif
soc = INVALID_SOCKET;
}
} else {
#ifdef _WIN32
closesocket(soc);
#else
close(soc);
#endif
soc = INVALID_SOCKET;
}
}
return soc;
@@ -311,6 +318,104 @@ typedef struct {
error_redirect = "/server/error.html"; \
} while(0)
/* Longest "sid" value worth unescaping: the expected one is an md5 hex digest,
so anything near this is already invalid and is rejected unread. */
#define SID_VALUE_MAX 64
/** Does the urlencoded request body present the expected session id?
True only if at least one "sid" field is present and every occurrence
matches, so it holds whichever one a later last-write-wins parse keeps.
Non-destructive: it runs before the body is tokenized in place. */
static hts_boolean body_sid_is_valid(const char *body, const char *expected) {
const char *s = body;
hts_boolean seen = HTS_FALSE;
while (s != NULL && *s != '\0') {
const char *const amp = strchr(s, '&');
const char *const eq = strchr(s, '=');
if (eq != NULL && (amp == NULL || eq < amp) && (size_t) (eq - s) == 3 &&
strncmp(s, "sid", 3) == 0) {
const size_t len = amp != NULL ? (size_t) (amp - eq - 1) : strlen(eq + 1);
hts_boolean match = HTS_FALSE;
if (len < SID_VALUE_MAX) {
char raw[SID_VALUE_MAX];
String value = STRING_EMPTY;
memcpy(raw, eq + 1, len);
raw[len] = '\0';
unescapehttp(raw, &value);
/* StringBuff is NULL until written, so an empty value lands here. */
if (StringBuff(value) != NULL &&
strcmp(StringBuff(value), expected) == 0) {
match = HTS_TRUE;
}
StringFree(value);
}
if (!match) {
return HTS_FALSE;
}
seen = HTS_TRUE;
}
s = amp != NULL ? amp + 1 : NULL;
}
return seen;
}
/** Append src to the NUL-terminated dst of capacity size (NUL included).
False, leaving dst untouched, if it would not fit: unlike strcatbuff() this
never aborts, because every piece appended here is client-supplied. */
static hts_boolean path_append(char *dst, size_t size, const char *src) {
const size_t used = strlen(dst);
const size_t len = strlen(src);
/* dst holds at most size-1 bytes, so "size - used" is >= 1 and the untrusted
len stays alone: "used + len < size" could wrap and pass. */
if (len >= size - used) {
return HTS_FALSE;
}
memcpy(dst + used, src, len + 1);
return HTS_TRUE;
}
/* Append c to dst as an HTML entity, or return HTS_FALSE if it needs none. */
static hts_boolean cat_html_escaped(String *dst, char c) {
switch (c) {
case '<':
StringCat(*dst, "&lt;");
break;
case '>':
StringCat(*dst, "&gt;");
break;
case '&':
StringCat(*dst, "&amp;");
break;
case '\'':
StringCat(*dst, "&#39;");
break;
default:
return HTS_FALSE;
}
return HTS_TRUE;
}
/* Append the value of a double-quoted command-line argument: escaped for HTML,
which the browser undoes when it posts the command line back, and for the
argv splitter, which does not. */
static void cat_cmdline_arg(String *output, const char *value) {
const char *a;
for (a = value; *a != '\0'; a++) {
if (*a == '\\' || *a == '\"') {
StringCat(*output, "\\");
}
if (!cat_html_escaped(output, *a)) {
StringMemcat(*output, a, 1);
}
}
}
int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
int timeout = 30;
int retour = 0;
@@ -322,6 +427,9 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
String tmpbuff = STRING_EMPTY;
String tmpbuff2 = STRING_EMPTY;
String fspath = STRING_EMPTY;
/* Project directory this server set up; the only root /website/ serves from,
and deliberately not cleared between requests. */
String website = STRING_EMPTY;
char catbuff[CATBUFF_SIZE];
/* Load strings */
@@ -395,6 +503,7 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
T_SOC soc_c;
LLint length = 0;
const char *error_redirect = NULL;
hts_boolean denied = HTS_FALSE;
line[0] = '\0';
buffer[0] = '\0';
@@ -506,6 +615,22 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
}
/* Authenticate the body before parsing it: every field it carries is
written straight into the global key store below, "command" included,
and that one reaches the engine. Checking afterwards cannot work — the
damage is already done, and the pre-seeded "sid" above would compare
equal to itself for a request that simply omits the field. */
if (meth && buffer[0]) {
intptr_t expected = 0;
if (!coucal_readptr(NewLangList, "_sid", &expected) ||
!body_sid_is_valid(buffer, (const char *) expected)) {
buffer[0] = '\0';
meth = 0;
denied = HTS_TRUE;
}
}
/* check variables */
if (meth && buffer[0]) {
char *s = buffer;
@@ -526,20 +651,6 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
}
/* Error check */
{
intptr_t adr = 0;
intptr_t adr2 = 0;
if (coucal_readptr(NewLangList, "sid", &adr)) {
if (coucal_readptr(NewLangList, "_sid", &adr2)) {
if (strcmp((char *) adr, (char *) adr2) != 0) {
meth = 0;
}
}
}
}
/* Check variables (internal) */
if (meth) {
int doLoad = 0;
@@ -730,6 +841,11 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
if (!structcheck(StringBuff(tmpbuff))) {
FILE *fp;
/* Both halves of fspath come from posted fields, so a ".."
in them would escape the mirror once served. */
if (strstr(StringBuff(fspath), "..") == NULL) {
StringCopy(website, StringBuff(fspath));
}
StringCat(tmpbuff, "winprofile.ini");
fp = fopen(StringBuff(tmpbuff), "wb");
if (fp != NULL) {
@@ -802,7 +918,7 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
/* Response */
if (meth) {
int virtualpath = 0;
hts_boolean virtualpath = HTS_FALSE;
char *pos;
char *url = strchr(line1, ' ');
@@ -813,11 +929,11 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
char *qpos;
/* get the URL */
fsfile[0] = '\0';
if (error_redirect == NULL) {
if ((qpos = strchr(url, '?'))) {
*qpos = '\0';
}
fsfile[0] = '\0';
if (strcmp(url, "/") == 0) {
file = "/server/index.html";
meth = 2;
@@ -830,7 +946,7 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
if (strncmp(file, "/website/", 9) == 0) {
virtualpath = 1;
virtualpath = HTS_TRUE;
}
/* override */
@@ -844,18 +960,26 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
}
if (strlen(path) + strlen(file) + 32 < sizeof(fsfile)) {
if (strncmp(file, "/website/", 9) != 0) {
sprintf(fsfile, "%shtml%s", path, file);
} else {
intptr_t adr = 0;
/* the override above may have swapped a mirror path for a GUI page */
virtualpath = strncmp(file, "/website/", 9) == 0;
if (coucal_readptr(NewLangList, "projpath", &adr)) {
sprintf(fsfile, "%s%s", (char *) adr, file + 9);
}
if (!virtualpath) {
if (!path_append(fsfile, sizeof(fsfile), path) ||
!path_append(fsfile, sizeof(fsfile), "html") ||
!path_append(fsfile, sizeof(fsfile), file)) {
fsfile[0] = '\0';
}
} else if (StringNotEmpty(website)) {
/* Never the posted "projpath": a client root reads any file. */
if (!path_append(fsfile, sizeof(fsfile), StringBuff(website)) ||
!path_append(fsfile, sizeof(fsfile), "/") ||
!path_append(fsfile, sizeof(fsfile), file + 9)) {
fsfile[0] = '\0';
}
}
/* path itself may hold ".." (webhttrack passes "<bin>/../share"), so
only the untrusted halves are checked: file here, website above. */
if (fsfile[0] && strstr(file, "..") == NULL
&& (fp = fopen(fsfile, "rb"))) {
char ok[] =
@@ -903,16 +1027,16 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
}
StringMemcat(headers, redir, strlen(redir));
{
char tmp[256];
if (strlen(file) < sizeof(tmp) - 32) {
sprintf(tmp, "Location: %s\r\n", newfile);
StringMemcat(headers, tmp, strlen(tmp));
}
/* client-supplied: a CR/LF here would split the response */
if (newfile[strcspn(newfile, "\r\n")] == '\0') {
StringCat(headers, "Location: ");
StringCat(headers, newfile);
StringCat(headers, "\r\n");
}
coucal_write(NewLangList, "redirect", (intptr_t) NULL);
} else if (is_html(file)) {
} else if (!virtualpath && is_html(file)) {
/* GUI templates only: ${_sid} in a mirrored page would hand the
crawled site the session id that authenticates commands */
int outputmode = 0;
StringMemcat(headers, ok, sizeof(ok) - 1);
@@ -940,6 +1064,7 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
int p;
int format = 0;
int listDefault = 0;
hts_boolean unquoted = HTS_FALSE;
name[0] = '\0';
strlncatbuff(name, str, sizeof(name_), n);
@@ -949,6 +1074,12 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
} else if ((p = strfield(name, "html:"))) {
name += p;
format = 1;
} else if ((p = strfield(name, "unquoted:"))) {
name += p;
unquoted = HTS_TRUE;
} else if ((p = strfield(name, "arg:"))) {
name += p;
format = 5;
} else if ((p = strfield(name, "list:"))) {
name += p;
format = 2;
@@ -1085,8 +1216,8 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
test:<if ==0>:<if ==1>:<if == 2>..
ztest:<if == 0 || !exist>:<if == 1>:<if == 2>..
*/
else if ((p = strfield(name, "test:"))
|| (p = strfield(name, "ztest:"))) {
else if ((p = strfield(name, "test:")) ||
(p = strfield(name, "ztest:"))) {
intptr_t adr = 0;
char *pos2;
int ztest = (name[0] == 'z');
@@ -1189,6 +1320,12 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
}
}
/* consumed here: it shares nothing with the list and
option formats below */
if (format == 5 && langstr != NULL && outputmode != -1) {
cat_cmdline_arg(&output, langstr);
langstr = NULL;
}
if (langstr && outputmode != -1) {
switch (format) {
case 0:
@@ -1206,18 +1343,18 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
StringMemcat(output, &c, 1);
}
a += 2;
} else if (outputmode && a[0] == '<') {
StringCat(output, "&lt;");
} else if (outputmode && a[0] == '>') {
StringCat(output, "&gt;");
} else if (outputmode && a[0] == '&') {
StringCat(output, "&amp;");
} else if (outputmode && a[0] == '\'') {
StringCat(output, "&#39;");
} else if (unquoted && a[0] == '\"') {
/* the browser posts an entity back as a raw
quote, which would open a quoted run in the
argv splitter; a URI cannot hold one anyway */
StringCat(output, "%22");
} else if (outputmode &&
cat_html_escaped(&output, a[0])) {
/* appended as an entity */
} else if (outputmode == 3 && a[0] == ' ') {
StringCat(output, "%20");
} else if (outputmode >= 2
&& ((unsigned char) a[0]) < 32) {
} else if (outputmode >= 2 &&
((unsigned char) a[0]) < 32) {
char tmp[32];
sprintf(tmp, "%%%02x", (unsigned char) a[0]);
@@ -1278,20 +1415,10 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
StringClear(tmpbuff);
break;
case '<':
StringCat(tmpbuff, "&lt;");
break;
case '>':
StringCat(tmpbuff, "&gt;");
break;
case '&':
StringCat(tmpbuff, "&amp;");
break;
case '\'':
StringCat(tmpbuff, "&#39;");
break;
default:
StringMemcat(tmpbuff, fstr, 1);
if (!cat_html_escaped(&tmpbuff, *fstr)) {
StringMemcat(tmpbuff, fstr, 1);
}
break;
}
fstr++;
@@ -1331,7 +1458,9 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
#endif
} else {
if (is_text(file)) {
if (is_html(file)) {
StringMemcat(headers, ok, sizeof(ok) - 1);
} else if (is_text(file)) {
StringMemcat(headers, ok_text, sizeof(ok_text) - 1);
} else if (is_js(file)) {
StringMemcat(headers, ok_js, sizeof(ok_js) - 1);
@@ -1368,6 +1497,11 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
StringCat(output, error);
}
}
} else if (denied) {
StringCat(headers, "HTTP/1.0 403 Forbidden\r\n"
"Server: httrack small server\r\n"
"Content-type: text/html\r\n");
StringCat(output, "Missing or invalid session id.\r\n");
} else {
#ifdef _DEBUG
char error_hdr[] =
@@ -1433,6 +1567,7 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
StringFree(tmpbuff);
StringFree(tmpbuff2);
StringFree(fspath);
StringFree(website);
if (buffer)
free(buffer);

View File

@@ -43,8 +43,11 @@ Please visit our Website: http://www.httrack.com
// Fonctions
void socinput(T_SOC soc, char *s, int max);
T_SOC smallserver_init_std(int *port_prox, char *adr_prox, int defaultPort);
T_SOC smallserver_init(int *port, char *adr);
/* Listen on bindAddr, or every interface if NULL/empty; adr (>= 258 bytes) gets
the address to advertise. INVALID_SOCKET on error. */
T_SOC smallserver_init_std(int *port_prox, char *adr_prox, int defaultPort,
const char *bindAddr);
T_SOC smallserver_init(int *port, char *adr, const char *bindAddr);
int smallserver(T_SOC soc, char *url, char *method, char *data, char *path);
#define CATCH_RESPONSE \

1140
src/htssinglefile.c Normal file

File diff suppressed because it is too large Load Diff

78
src/htssinglefile.h Normal file
View File

@@ -0,0 +1,78 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 2026 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* --single-file: post-mirror pass embedding each page's assets as data: URIs.
Internal, not installed. Unlike -%M (one MHT archive for the whole mirror)
this rewrites the saved pages in place and keeps page-to-page links
relative, so the mirror stays browsable and every page also stands alone. */
/* ------------------------------------------------------------ */
#ifndef HTS_SINGLEFILE_DEFH
#define HTS_SINGLEFILE_DEFH
#include "htsopt.h"
#include "htsstrings.h"
#ifdef __cplusplus
extern "C" {
#endif
/* --single-file-max-size default: a bigger asset stays an ordinary link. */
#define SINGLEFILE_DEFAULT_MAX_SIZE (10 * 1024 * 1024)
/* Never base64 more than this, whatever the cap: code64() sizes in int. */
#define SINGLEFILE_HARD_MAX_SIZE (256 * 1024 * 1024)
/* Total bytes one page may inline. Nested @import fans out multiplicatively,
so a few hundred bytes of hostile CSS can otherwise ask for gigabytes. */
#define SINGLEFILE_MAX_PAGE_SIZE (64 * 1024 * 1024)
/* Rewrite every HTML page the mirror produced. No-op unless opt->single_file;
call once the tree is final, after the update purge. */
void singlefile_process_mirror(httrackp *opt);
/* Rewrite one HTML document held in memory, appending the result to out.
root is the mirror directory that references may not escape; page_path is
the document's own path under it (both UTF-8, '/' or native separators).
Returns HTS_TRUE if at least one reference was replaced; out may still
differ from the input when that is HTS_FALSE, since a style or srcset value
is re-serialized in place. */
hts_boolean singlefile_rewrite_html(httrackp *opt, const char *root,
const char *page_path, const char *html,
size_t html_len, String *out);
/* singlefile_rewrite_html() over the file at page_path, rewritten in place.
Returns HTS_TRUE if the file changed. */
hts_boolean singlefile_rewrite_file(httrackp *opt, const char *root,
const char *page_path);
#ifdef __cplusplus
}
#endif
#endif

View File

@@ -754,35 +754,48 @@ typedef struct hts_template_format_buf {
size_t offset;
} hts_template_format_buf;
// Bounded append to a template buffer (or FILE); returns -1 on overflow/error.
static int htsfmt_putc(hts_template_format_buf *buf, char c) {
if (buf->fp != NULL) {
assertf(buf->buffer == NULL);
if (fputc(c, buf->fp) < 0)
return -1;
} else {
assertf(buf->buffer != NULL);
if (buf->offset + 1 < buf->size)
buf->buffer[buf->offset++] = c;
else
return -1;
}
return 0;
}
static int htsfmt_puts(hts_template_format_buf *buf, const char *s) {
size_t i;
assertf(s != NULL);
for (i = 0; s[i] != '\0'; i++) {
if (htsfmt_putc(buf, s[i]) < 0)
return -1;
}
return 0;
}
// note: upstream arg list MUST be NULL-terminated for safety
// returns a negative value upon error
static int hts_template_formatv(hts_template_format_buf *buf,
static int hts_template_formatv(hts_template_format_buf *buf,
const char *format, va_list args) {
#undef FPUTC
#undef FPUTS
#define FPUTC(C) do { \
if (buf->fp != NULL) { \
assertf(buf->buffer == NULL); \
if (fputc(C, buf->fp) < 0) { \
return -1; \
} \
} else { \
assertf(buf->buffer != NULL); \
if (buf->offset + 1 < buf->size) { \
buf->buffer[buf->offset++] = (C); \
} else { \
return -1; \
} \
} \
} while(0)
#define FPUTS(S) do { \
size_t i; \
const char *const str_ = (S); \
assertf(str_ != NULL); \
for(i = 0 ; str_[i] != '\0' ; i++) { \
FPUTC(str_[i]); \
} \
} while(0)
#define FPUTC(C) \
do { \
if (htsfmt_putc(buf, (C)) < 0) \
return -1; \
} while (0)
#define FPUTS(S) \
do { \
if (htsfmt_puts(buf, (S)) < 0) \
return -1; \
} while (0)
if (buf != NULL && format != NULL) {
const char *arg_expanded[32];
@@ -859,6 +872,74 @@ int hts_template_format_str(char *buffer, size_t size, const char *format, ...)
return success;
}
// Value of the named field, or "" if absent (never NULL, so callers can pass it
// straight to a formatter).
static const char *footer_field_value(const hts_footer_field *fields,
size_t nfields, const char *name) {
size_t j;
for (j = 0; j < nfields; j++) {
if (strcmp(fields[j].name, name) == 0)
return fields[j].value != NULL ? fields[j].value : "";
}
return "";
}
int hts_footer_format(char *buffer, size_t size, const char *footer,
const hts_footer_field *fields, size_t nfields) {
hts_template_format_buf buf = {NULL, buffer, size, 0};
size_t i;
if (footer == NULL || buffer == NULL || size == 0)
return -1;
// %s keeps the legacy positional model, byte-for-byte for existing -%F
// strings: addr, path, date, version, looked up by name and
// order-independent.
if (strstr(footer, "%s") != NULL)
return hts_template_format_str(
buffer, size, footer, footer_field_value(fields, nfields, "addr"),
footer_field_value(fields, nfields, "path"),
footer_field_value(fields, nfields, "date"),
footer_field_value(fields, nfields, "version"), /* EOF */ NULL);
// "{{"/"}}" emit a literal brace; an unknown "{...}" is left verbatim so
// typos stay visible.
for (i = 0; footer[i] != '\0'; i++) {
const char c = footer[i];
if (c == '{' && footer[i + 1] == '{') {
if (htsfmt_putc(&buf, '{') < 0)
return -1;
i++;
} else if (c == '}' && footer[i + 1] == '}') {
if (htsfmt_putc(&buf, '}') < 0)
return -1;
i++;
} else if (c == '{') {
const char *const end = strchr(footer + i + 1, '}');
int matched = 0;
if (end != NULL) {
const size_t namelen = (size_t) (end - (footer + i + 1));
size_t j;
for (j = 0; j < nfields; j++) {
if (strlen(fields[j].name) == namelen &&
strncmp(fields[j].name, footer + i + 1, namelen) == 0) {
if (htsfmt_puts(&buf,
fields[j].value != NULL ? fields[j].value : "") < 0)
return -1;
i += namelen + 1; // consume the name and its closing '}'
matched = 1;
break;
}
}
}
if (!matched && htsfmt_putc(&buf, '{') < 0)
return -1;
} else if (htsfmt_putc(&buf, c) < 0) {
return -1;
}
}
buffer[buf.offset] = '\0';
return 1;
}
/* Note: NOT utf-8 */
HTSEXT_API int hts_buildtopindex(httrackp * opt, const char *path,
const char *binpath) {
@@ -946,6 +1027,17 @@ HTSEXT_API int hts_buildtopindex(httrackp * opt, const char *path,
freet(category);
category = NULL;
}
#ifdef _WIN32
/* category is ANSI-codepage, doc is utf-8: convert (#216) */
else {
char *cat_utf8 = hts_convertStringSystemToUTF8(
category, strlen(category));
if (cat_utf8 != NULL) {
freet(category);
category = cat_utf8;
}
}
#endif
}
}
if (category == NULL) {
@@ -963,7 +1055,18 @@ HTSEXT_API int hts_buildtopindex(httrackp * opt, const char *path,
oldchain->next = chain;
}
chain->next = NULL;
#ifdef _WIN32
/* name is ANSI-codepage, doc is utf-8: convert (#216) */
{
const char *const name = hts_findgetname(h);
char *name_utf8 =
hts_convertStringSystemToUTF8(name, strlen(name));
strcpybuff(chain->name, name_utf8 != NULL ? name_utf8 : name);
freet(name_utf8);
}
#else
strcpybuff(chain->name, hts_findgetname(h));
#endif
chain->category = category;
chain->level = level;
}
@@ -1215,19 +1318,19 @@ HTSEXT_API hts_boolean hts_findnext(find_handle find) {
if (find) {
#ifdef _WIN32
if ((FindNextFileA(find->handle, &find->hdata)))
return 1;
return HTS_TRUE;
#else
char catbuff[CATBUFF_SIZE];
memset(&(find->filestat), 0, sizeof(find->filestat));
if ((find->dirp = readdir(find->hdir)))
if (find->dirp->d_name)
if (!STAT
(concat(catbuff, sizeof(catbuff), find->path, find->dirp->d_name), &find->filestat))
return 1;
if (!STAT(
concat(catbuff, sizeof(catbuff), find->path, find->dirp->d_name),
&find->filestat))
return HTS_TRUE;
#endif
}
return 0;
return HTS_FALSE;
}
HTSEXT_API int hts_findclose(find_handle find) {

View File

@@ -73,6 +73,20 @@ HTS_INLINE int rech_tageq_all(const char *adr, const char *s);
int hts_template_format(FILE *const out, const char *format, ...);
int hts_template_format_str(char *buffer, size_t size, const char *format, ...);
// A footer named field and its already-context-escaped value.
typedef struct hts_footer_field {
const char *name;
const char *value;
} hts_footer_field;
// Expand a footer template. A "%s" in it selects the legacy positional model,
// consuming the "addr"/"path"/"date"/"version" fields in that order; otherwise
// "{name}" is substituted from fields by name ("{{"/"}}" emit a literal brace,
// an unknown "{...}" is left verbatim). Values must already be escaped for the
// target context by the caller. Returns <0 on overflow.
int hts_footer_format(char *buffer, size_t size, const char *footer,
const hts_footer_field *fields, size_t nfields);
#define rech_tageq(adr,s) \
( \
( (*((adr)-1)=='<') || (is_space(*((adr)-1))) ) ? \

1799
src/htswarc.c Normal file

File diff suppressed because it is too large Load Diff

131
src/htswarc.h Normal file
View File

@@ -0,0 +1,131 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 2026 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* HTTrack WARC/1.1 output writer (ISO 28500). Internal, not installed.
All WARC record serialization, gzip-member framing, digests, UUID and
revisit dedup live here; the engine only stashes the request, frees it,
and calls one entry point per finished transaction. */
/* ------------------------------------------------------------ */
#ifndef HTS_WARC_DEFH
#define HTS_WARC_DEFH
#include "htsopt.h"
#ifdef __cplusplus
extern "C" {
#endif
/* opt->warc_file sentinel: --warc with no argument => auto-name the archive
under the project's output directory at open time. */
#define WARC_AUTONAME "\001auto"
/* htsblk.warc_truncated / WARC-Truncated reason tokens (ISO 28500 sec 5.13).
A cap-truncated body is still archived, tagged with why it was cut short. */
#define WARC_TRUNC_NONE 0
#define WARC_TRUNC_LENGTH 1 /* hit a size cap (-M mirror / -m per-file) */
#define WARC_TRUNC_TIME 2 /* hit the mirror time cap (-E/--max-time) */
#define WARC_TRUNC_DISCONNECT 3 /* connection dropped mid-body */
/* WARC-Truncated token for a warc_truncated code, or NULL for none. */
const char *warc_truncated_reason(int code);
typedef struct warc_writer warc_writer;
/* Stash the raw request header block (bstr.buffer) on r for the later WARC
request record; frees any prior stash. No-op when reqhdr is NULL. */
void warc_stash_request(htsblk *r, const char *reqhdr);
/* Stash the raw response header block: deletehttp frees r.headers when the
socket closes, before back_finalize, so keep a WARC-owned copy. */
void warc_stash_response(htsblk *r, const char *resphdr);
/* Free both stashed header blocks (idempotent, NULL-safe). */
void warc_free_request(htsblk *r);
/* Adopt the de-chunked compressed spool at tmpfile_path onto
r->warc_rawpath/warc_rawsize (strdupt; frees any prior) so the WARC record
stores the body verbatim. No-op leaving warc_rawpath NULL when tmpfile_path
is empty or the spool is missing/empty (the record then stores the decoded
in-memory/on-disk body instead). */
void warc_adopt_rawspool(htsblk *r, const char *tmpfile_path);
/* Emit the request + response (or revisit) records for one finished
transaction. Lazily opens the writer into opt->state.warc; a no-op (logged
once) if the archive cannot be created. */
void warc_write_backtransaction(httrackp *opt, lien_back *back);
/* Close and free the writer held in opt->state.warc, if any. */
void warc_close_opt(httrackp *opt);
/* --- Direct writer API (used by the hooks above and the self-test). --- */
/* Create the archive at path (auto-named when path is WARC_AUTONAME), writing
the warcinfo record. .warc.gz => one gzip member per record; .warc => raw.
Returns NULL on failure. */
warc_writer *warc_open(httrackp *opt, const char *path);
/* Flush, close and free the writer (NULL-safe). */
void warc_close(warc_writer *w);
/* SURT-canonicalize url into out[outsz] (the CDXJ sort key). Returns 0 on
success, -1 on error or truncation. Exposed for the -#test=warc-surt test. */
int warc_surt(const char *url, char *out, size_t outsz);
/* Write one transaction's request + response (or revisit) records.
target_uri: absolute URL fetched.
ip: numeric peer IP, or NULL/"" to omit.
req_hdr: exact request header block sent, or NULL to skip the request.
resp_hdr: raw received response header block (status line + headers).
body/body_len: decoded in-memory body, or NULL when on disk.
body_path: file re-read for the body when body==NULL (may be NULL).
is_update_unchanged: nonzero for a 304 server-not-modified revisit.
truncated: a WARC_TRUNC_* reason to tag a cap-truncated body, else 0.
The body is stored verbatim: Content-Encoding is kept and Content-Length set
to body_len, so body/body_len must be the as-received (coded) bytes.
Returns 0 on success, -1 on error. */
int warc_write_transaction(warc_writer *w, const char *target_uri,
const char *ip, const char *req_hdr,
const char *resp_hdr, const char *body,
size_t body_len, const char *body_path,
int statuscode, int is_update_unchanged,
int truncated);
/* Write one non-HTTP capture as a single WARC 'resource' record: the block is
the raw payload (no HTTP envelope), Content-Type is the payload's own MIME.
Used for ftp:// transfers. truncated is a WARC_TRUNC_* reason or 0.
Returns 0 on success, -1 on error. */
int warc_write_resource(warc_writer *w, const char *target_uri, const char *ip,
const char *content_type, const char *body,
size_t body_len, const char *body_path, int truncated);
#ifdef __cplusplus
}
#endif
#endif

View File

@@ -62,6 +62,7 @@ Please visit our Website: http://www.httrack.com
#include "htsmd5.c"
#include "md5.c"
#include "htscmdline.h"
#include "htsserver.h"
#include "htsurlport.h"
#include "htsweb.h"
@@ -88,7 +89,7 @@ Please visit our Website: http://www.httrack.com
static htsmutex refreshMutex = HTSMUTEX_INIT;
static int help_server(char *dest_path, int defaultPort);
static int help_server(char *dest_path, int defaultPort, const char *bindAddr);
extern int commandRunning;
extern int commandEnd;
extern int commandReturn;
@@ -153,6 +154,8 @@ int main(int argc, char *argv[]) {
int ret = 0;
int defaultPort = 0;
int parentPid = 0;
/* loopback by default: the handler trusts its input; --bind widens it */
const char *bindAddr = "127.0.0.1";
printf("Initializing the server..\n");
@@ -179,7 +182,8 @@ int main(int argc, char *argv[]) {
if (argc < 2 || (argc % 2) != 0) {
fprintf(stderr, "** Warning: use the webhttrack frontend if available\n");
fprintf(stderr,
"usage: %s [--port <port>] [--ppid parent-pid] <path-to-html-root-dir> [key value [key value]..]\n",
"usage: %s [--port <port>] [--bind <address>] [--ppid parent-pid] "
"<path-to-html-root-dir> [key value [key value]..]\n",
argv[0]);
fprintf(stderr, "example: %s /usr/share/httrack/\n", argv[0]);
return 1;
@@ -267,6 +271,14 @@ int main(int argc, char *argv[]) {
fprintf(stderr, "couldn't set the port number to %s\n", argv[i + 1]);
return -1;
}
} else if (strcmp(argv[i], "--bind") == 0 && i + 1 < argc) {
/* empty would fall back to every interface, silently undoing the default
*/
if (!strnotempty(argv[i + 1])) {
fprintf(stderr, "--bind needs an address\n");
return -1;
}
bindAddr = argv[i + 1];
} else if (strcmp(argv[i], "--ppid") == 0 && i + 1 < argc) {
if (sscanf(argv[i + 1], "%u", &parentPid) != 1) {
fprintf(stderr, "couldn't set the parent PID to %s\n", argv[i + 1]);
@@ -293,7 +305,7 @@ int main(int argc, char *argv[]) {
}
/* launch */
ret = help_server(argv[1], defaultPort);
ret = help_server(argv[1], defaultPort, bindAddr);
htsthread_wait_n(background_threads - 1);
hts_uninit();
@@ -308,10 +320,8 @@ int main(int argc, char *argv[]) {
static int webhttrack_runmain(httrackp * opt, int argc, char **argv);
static void back_launch_cmd(void *pP) {
char *cmd = (char *) pP;
char **argv = (char **) malloct(1024 * sizeof(char *));
char **argv;
int argc = 0;
int i = 0;
int g = 0;
//
httrackp *opt;
@@ -322,28 +332,19 @@ static void back_launch_cmd(void *pP) {
commandReturnCmdl = strdup(cmd);
/* split */
argv[0] = strdup("webhttrack");
argv[1] = cmd;
argc++;
i = 0;
while(cmd[i]) {
if (cmd[i] == '\t' || cmd[i] == '\r' || cmd[i] == '\n') {
cmd[i] = ' ';
}
i++;
}
i = 0;
while(cmd[i]) {
if (cmd[i] == '\"')
g = !g;
if (cmd[i] == ' ') {
if (!g) {
cmd[i] = '\0';
argv[argc++] = cmd + i + 1;
}
}
i++;
argv = hts_split_cmdline(cmd, &argc);
if (argv == NULL) {
if (commandReturnMsg)
free(commandReturnMsg);
commandReturnMsg = strdup("could not parse the command line");
commandReturn = -1;
commandRunning = 0;
commandEnd = 1;
free(cmd);
return;
}
/* drop the program name the posted command line carries */
argv[0] = strdupt("webhttrack");
/* init */
hts_init();
@@ -372,6 +373,7 @@ static void back_launch_cmd(void *pP) {
/* free */
free(cmd);
freet(argv[0]);
freet(argv);
return;
}
@@ -427,11 +429,11 @@ static int webhttrack_runmain(httrackp * opt, int argc, char **argv) {
return ret;
}
static int help_server(char *dest_path, int defaultPort) {
static int help_server(char *dest_path, int defaultPort, const char *bindAddr) {
int returncode = 0;
char adr_prox[HTS_URLMAXSIZE * 2];
int port_prox;
T_SOC soc = smallserver_init_std(&port_prox, adr_prox, defaultPort);
T_SOC soc = smallserver_init_std(&port_prox, adr_prox, defaultPort, bindAddr);
if (soc != INVALID_SOCKET) {
char url[HTS_URLMAXSIZE * 2];
@@ -671,10 +673,10 @@ int __cdecl htsshow_loop(t_hts_callbackarg * carg, httrackp * opt, lien_back * b
char *eps = strchr(back[i].url_adr, '/');
int count;
if (ep != NULL && ep < eps
&& (count = (int) (ep - back[i].url_adr)) < 4) {
if (ep != NULL && eps != NULL && ep < eps &&
(count = (int) (ep - back[i].url_adr)) < 4) {
proto[0] = '\0';
strncat(proto, back[i].url_adr, count);
strncatbuff(proto, back[i].url_adr, count);
}
}
snprintf(StatsBuffer[index].state, sizeof(StatsBuffer[index].state),

View File

@@ -42,7 +42,7 @@ Please visit our Website: http://www.httrack.com
typedef struct t_StatsBuffer {
char name[1024];
char file[1024];
char state[256];
char state[288]; // a short label plus back->info[256]
char url_sav[HTS_URLMAXSIZE * 2]; // pour cancel
char url_adr[HTS_URLMAXSIZE * 2];
char url_fil[HTS_URLMAXSIZE * 2];

View File

@@ -758,6 +758,9 @@ HTSEXT_API int hts_rename_utf8(const char *oldpath, const char *newpath);
HTSEXT_API int hts_mkdir_utf8(const char *pathname);
#define RMDIR hts_rmdir_utf8
HTSEXT_API int hts_rmdir_utf8(const char *pathname);
#define UTIME(A, B) hts_utime_utf8(A, B)
typedef struct _utimbuf STRUCT_UTIMBUF;
@@ -772,6 +775,7 @@ typedef struct stat STRUCT_STAT;
#define UNLINK unlink
#define RENAME rename
#define MKDIR(F) mkdir(F, HTS_ACCESS_FOLDER)
#define RMDIR rmdir
typedef struct utimbuf STRUCT_UTIMBUF;

View File

@@ -46,6 +46,7 @@ Please visit our Website: http://www.httrack.com
#include "httrack.h"
#include "htslib.h"
#include "htscharset.h" // after htslib.h: winsock2.h must precede windows.h
#include "htsbacktrace.h"
/* Static definitions */
static int fexist(const char *s);
@@ -68,11 +69,10 @@ static int linput(FILE * fp, char *s, int max);
#ifdef HAVE_UNISTD_H
#include <unistd.h>
#endif
#include <ctype.h>
#if (defined(__linux) && defined(HAVE_EXECINFO_H))
#include <execinfo.h>
#define USES_BACKTRACE
#ifdef HAVE_SYS_IOCTL_H
#include <sys/ioctl.h>
#endif
#include <ctype.h>
/* END specific definitions */
static void __cdecl htsshow_init(t_hts_callbackarg * carg);
@@ -185,6 +185,43 @@ static void vt_home(void) {
printf("%s%s", VT_RESET, VT_GOTOXY("1", "0"));
}
/* Last known terminal geometry; the defaults are the classic VT100 size. */
static int term_cols = 80;
static int term_rows = 24;
/* Returns HTS_TRUE if the terminal size changed since the last call. */
static hts_boolean vt_size_refresh(void) {
int cols = 0;
int rows = 0;
#ifdef _WIN32
CONSOLE_SCREEN_BUFFER_INFO info;
const HANDLE console = GetStdHandle(STD_OUTPUT_HANDLE);
if (console != INVALID_HANDLE_VALUE &&
GetConsoleScreenBufferInfo(console, &info)) {
cols = info.srWindow.Right - info.srWindow.Left + 1;
rows = info.srWindow.Bottom - info.srWindow.Top + 1;
}
#elif defined(TIOCGWINSZ)
struct winsize ws;
if (ioctl(fileno(stdout), TIOCGWINSZ, &ws) == 0) {
cols = ws.ws_col;
rows = ws.ws_row;
}
#endif
if (cols <= 0)
cols = 80;
if (rows <= 0)
rows = 24;
if (cols == term_cols && rows == term_rows)
return HTS_FALSE;
term_cols = cols;
term_rows = rows;
return HTS_TRUE;
}
//
/*
@@ -195,7 +232,8 @@ static void vt_home(void) {
#define STYLE_STATTEXT VT_UNBOLD
#define STYLE_STATRESET VT_UNBOLD
#define NStatsBuffer 14
#define MAX_LEN_INPROGRESS 40
/* Rows the stats block and "Current job" take above the in-progress list. */
#define NStatsHeaderRows 7
static int use_show;
static httrackp *global_opt = NULL;
@@ -291,6 +329,7 @@ static int __cdecl htsshow_start(t_hts_callbackarg * carg, httrackp * opt) {
use_show = 0;
if (opt->verbosedisplay == HTS_VERBOSE_FULL) {
use_show = 1;
(void) vt_size_refresh();
vt_clear();
}
return 1;
@@ -396,6 +435,10 @@ static int __cdecl htsshow_loop(t_hts_callbackarg * carg, httrackp * opt, lien_b
prev_mytime = mytime;
/* A resize leaves stale wrapped text on screen: repaint everything. */
if (vt_size_refresh())
vt_clear();
st[0] = '\0';
qsec2str(st, stat_time);
vt_home();
@@ -435,6 +478,13 @@ static int __cdecl htsshow_loop(t_hts_callbackarg * carg, httrackp * opt, lien_b
//
t_StatsBuffer StatsBuffer[NStatsBuffer];
/* Keep the frame within the terminal, or it scrolls the stats away. */
const int nstats =
min(NStatsBuffer, max(0, term_rows - NStatsHeaderRows));
/* URL budget: half the width, i.e. the historical 40 at 80 columns. */
const int maxurl =
min((int) sizeof(StatsBuffer[0].name) - 1, max(16, term_cols / 2));
{
int i;
@@ -449,10 +499,11 @@ static int __cdecl htsshow_loop(t_hts_callbackarg * carg, httrackp * opt, lien_b
}
}
for(k = 0; k < 2; k++) { // 0: lien en cours 1: autres liens
for(j = 0; (j < 3) && (index < NStatsBuffer); j++) { // passe de priorité
for (j = 0; (j < 3) && (index < nstats); j++) { // passe de priorité
int _i;
for(_i = 0 + k; (_i < max(back_max * k, 1)) && (index < NStatsBuffer); _i++) { // no lien
for (_i = 0 + k; (_i < max(back_max * k, 1)) && (index < nstats);
_i++) { // no lien
int i = (back_index + _i) % back_max; // commencer par le "premier" (l'actuel)
if (back[i].status >= 0) { // signifie "lien actif"
@@ -527,16 +578,14 @@ static int __cdecl htsshow_loop(t_hts_callbackarg * carg, httrackp * opt, lien_b
}
}
if ((l = (int) strlen(s)) < MAX_LEN_INPROGRESS)
if ((l = (int) strlen(s)) < maxurl)
strcpybuff(StatsBuffer[index].name, s);
else {
// couper
StatsBuffer[index].name[0] = '\0';
strncatbuff(StatsBuffer[index].name, s,
MAX_LEN_INPROGRESS / 2 - 2);
strncatbuff(StatsBuffer[index].name, s, maxurl / 2 - 2);
strcatbuff(StatsBuffer[index].name, "...");
strcatbuff(StatsBuffer[index].name,
s + l - MAX_LEN_INPROGRESS / 2 + 2);
strcatbuff(StatsBuffer[index].name, s + l - maxurl / 2 + 2);
}
if (back[i].r.totalsize >= 0) { // taille prédéfinie
@@ -597,7 +646,7 @@ static int __cdecl htsshow_loop(t_hts_callbackarg * carg, httrackp * opt, lien_b
{
int i;
for(i = 0; i < NStatsBuffer; i++) {
for (i = 0; i < nstats; i++) {
if (strnotempty(StatsBuffer[i].state)) {
printf(VT_CLREOL " %s - \t%s%s \t%s / \t%s", StatsBuffer[i].state,
StatsBuffer[i].name, StatsBuffer[i].file, int2bytes(&strc,
@@ -828,21 +877,6 @@ static void sig_doback(int blind) { // mettre en backing
#undef FD_ERR
#define FD_ERR 2
static void print_backtrace(void) {
#ifdef USES_BACKTRACE
void *stack[256];
const int size = backtrace(stack, sizeof(stack)/sizeof(stack[0]));
if (size != 0) {
backtrace_symbols_fd(stack, size, FD_ERR);
}
#else
const char msg[] = "No stack trace available on this OS :(\n";
if (write(FD_ERR, msg, sizeof(msg) - 1) != sizeof(msg) - 1) {
/* sorry GCC */
}
#endif
}
static size_t print_num(char *buffer, int num) {
size_t i, j;
if (num < 0) {
@@ -876,7 +910,7 @@ static void sig_fatal(int code) {
size += print_num(&buffer[size], code);
buffer[size++] = '\n';
(void) (write(FD_ERR, buffer, size) == size);
print_backtrace();
hts_print_backtrace(FD_ERR);
(void) (write(FD_ERR, msgreport, sizeof(msgreport) - 1)
== sizeof(msgreport) - 1);
abort();
@@ -899,6 +933,7 @@ static void sig_leave(int code) {
}
static void signal_handlers(void) {
hts_backtrace_init();
#ifdef _WIN32
signal(SIGINT, sig_leave); // ^C
signal(SIGTERM, sig_finish); // kill <process>

View File

@@ -44,7 +44,7 @@ typedef struct t_StatsBuffer t_StatsBuffer;
struct t_StatsBuffer {
char name[1024];
char file[1024];
char state[256];
char state[288]; // a short label plus back->info[256]
char BIGSTK url_sav[HTS_URLMAXSIZE * 2]; // pour cancel
char BIGSTK url_adr[HTS_URLMAXSIZE * 2];
char BIGSTK url_fil[HTS_URLMAXSIZE * 2];

View File

@@ -100,6 +100,7 @@
<ItemGroup>
<ClCompile Include="httrack.c" />
<ClCompile Include="htsbacktrace.c" />
</ItemGroup>
<!-- Pulls in libhttrack.lib and builds it first. -->

View File

@@ -59,9 +59,10 @@
<ItemDefinitionGroup>
<ClCompile>
<!-- LIBHTTRACK_EXPORTS turns HTSEXT_API into __declspec(dllexport); ZLIB_DLL
imports zlib from its DLL. HTS_USEBROTLI/HTS_USEZSTD enable the br and
zstd content codings. Windows 7 floor, matching WinHTTrack. -->
<PreprocessorDefinitions>WIN32;_WINDOWS;_MBCS;_USRDLL;LIBHTTRACK_EXPORTS;ZLIB_DLL;HTS_USEBROTLI=1;HTS_USEZSTD=1;WINVER=0x0601;_WIN32_WINNT=0x0601;_CRT_SECURE_NO_WARNINGS;_CRT_NONSTDC_NO_DEPRECATE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
imports zlib from its DLL, ZLIB_CONST makes its next_in const as the
autotools build does. HTS_USEBROTLI/HTS_USEZSTD enable the br and zstd
content codings. Windows 7 floor, matching WinHTTrack. -->
<PreprocessorDefinitions>WIN32;_WINDOWS;_MBCS;_USRDLL;LIBHTTRACK_EXPORTS;ZLIB_DLL;ZLIB_CONST;HTS_USEBROTLI=1;HTS_USEZSTD=1;WINVER=0x0601;_WIN32_WINNT=0x0601;_CRT_SECURE_NO_WARNINGS;_CRT_NONSTDC_NO_DEPRECATE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(MSBuildThisFileDirectory);$(MSBuildThisFileDirectory)coucal;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<WarningLevel>Level3</WarningLevel>
<MultiProcessorCompilation>true</MultiProcessorCompilation>
@@ -121,6 +122,7 @@
<ClCompile Include="htshelp.c" />
<ClCompile Include="htsindex.c" />
<ClCompile Include="htslib.c" />
<ClCompile Include="htscmdline.c" />
<ClCompile Include="htsurlport.c" />
<ClCompile Include="htsmd5.c" />
<ClCompile Include="htsmodules.c" />
@@ -129,12 +131,14 @@
<ClCompile Include="htsproxy.c" />
<ClCompile Include="htsrobots.c" />
<ClCompile Include="htsselftest.c" />
<ClCompile Include="htssinglefile.c" />
<ClCompile Include="htssniff.c" />
<ClCompile Include="htsthread.c" />
<ClCompile Include="htstools.c" />
<ClCompile Include="htswizard.c" />
<ClCompile Include="htswrap.c" />
<ClCompile Include="htszlib.c" />
<ClCompile Include="htswarc.c" />
<ClCompile Include="md5.c" />
<ClCompile Include="minizip\ioapi.c" />
<ClCompile Include="minizip\iowin32.c" />

View File

@@ -688,7 +688,7 @@ static PT_Element proxytrack_process_DAV_Request(PT_Indexes indexes,
PT_Element elt = PT_ElementNew();
elt->statuscode = 405;
strcpy(elt->msg, "Method Not Allowed");
strcpybuff(elt->msg, "Method Not Allowed");
return elt;
}
@@ -771,7 +771,7 @@ static PT_Element proxytrack_process_DAV_Request(PT_Indexes indexes,
PT_ReadIndex(indexes, StringBuff(itemUrl) + 1, FETCH_HEADERS);
if (file != NULL && file->statuscode == HTTP_OK) {
size = file->size;
if (file->lastmodified) {
if (file->lastmodified[0] != '\0') {
timestamp = get_time_rfc822(file->lastmodified);
}
if (timestamp == (time_t) 0) {
@@ -785,7 +785,7 @@ static PT_Element proxytrack_process_DAV_Request(PT_Indexes indexes,
}
timestamp = timestampRep;
}
if (file->contenttype) {
if (file->contenttype[0] != '\0') {
mimeType = file->contenttype;
}
}
@@ -822,7 +822,7 @@ static PT_Element proxytrack_process_DAV_Request(PT_Indexes indexes,
elt->statuscode = 207; /* Multi-Status */
strcpy(elt->charset, "utf-8");
strcpy(elt->contenttype, "text/xml");
strcpy(elt->msg, "Multi-Status");
strcpybuff(elt->msg, "Multi-Status");
StringFree(response);
fprintf(stderr, "RESPONSE:\n%s\n", elt->adr);
@@ -879,7 +879,7 @@ static PT_Element proxytrack_process_HTTP_List(PT_Indexes indexes,
elt->statuscode = HTTP_OK;
strcpy(elt->charset, "iso-8859-1");
strcpy(elt->contenttype, "text/html");
strcpy(elt->msg, "OK");
strcpybuff(elt->msg, "OK");
StringFree(html);
return elt;
}

View File

@@ -407,7 +407,7 @@ PT_Element PT_Index_HTML_BuildRootInfo(PT_Indexes indexes) {
elt->statuscode = HTTP_OK;
strcpy(elt->charset, "iso-8859-1");
strcpy(elt->contenttype, "text/html");
strcpy(elt->msg, "OK");
strcpybuff(elt->msg, "OK");
StringFree(html);
return elt;
}
@@ -579,9 +579,10 @@ PT_Index PT_LoadCache(const char *filename) {
if (chain != NULL) {
const char *scheme = link_has_authority(chain->name) ? "" : "http://";
snprintf(index->slots.common.startUrl,
sizeof(index->slots.common.startUrl), "%s%s", scheme,
(const char *) chain->name);
/* dropped rather than truncated: empty already reads as "unset" */
if (!sprintfbuff(index->slots.common.startUrl, "%s%s", scheme,
(const char *) chain->name))
index->slots.common.startUrl[0] = '\0';
}
}
}
@@ -827,6 +828,18 @@ PT_Element PT_ElementNew(void) {
return r;
}
/* ProxyTrack's htsblk_failf(): a clipped, diagnostic-only failure reason. */
static void PT_Element_failf(PT_Element r, const char *fmt, ...)
HTS_PRINTF_FUN(2, 3);
static void PT_Element_failf(PT_Element r, const char *fmt, ...) {
va_list args;
va_start(args, fmt);
(void) vslprintfbuff(r->msg, sizeof(r->msg), fmt, args);
va_end(args);
}
PT_Element PT_ReadCache(PT_Index index, const char *url, int flags) {
if (index != NULL && SAFE_INDEX(index)) {
return _IndexFuncts[index->type].PT_ReadCache(index, url, flags);
@@ -865,12 +878,17 @@ static PT_Element PT_ReadCache__New(PT_Index index, const char *url, int flags)
sprintf(headers + headersSize, "%s: "LLintP"\r\n", field, (LLint)(value)); \
(headersSize) += (int) strlen(headers + headersSize); \
} while(0)
#define ZIP_READFIELD_STRING(line, value, refline, refvalue) do { \
if (line[0] != '\0' && strfield2(line, refline)) { \
strcpy(refvalue, value); \
line[0] = '\0'; \
} \
} while(0)
/* refvalue_size is mandatory: the cache line is bounded only by the line
buffer, not by the destination. Clip rather than reject, since an
engine-written field can be wider than ours. */
#define ZIP_READFIELD_STRING(line, value, refline, refvalue, refvalue_size) \
do { \
if (line[0] != '\0' && strfield2(line, refline)) { \
(refvalue)[0] = '\0'; \
strlncatbuff(refvalue, value, refvalue_size, (refvalue_size) - 1); \
line[0] = '\0'; \
} \
} while (0)
#define ZIP_READFIELD_INT(line, value, refline, refvalue) do { \
if (line[0] != '\0' && strfield2(line, refline)) { \
int intval = 0; \
@@ -972,9 +990,12 @@ int PT_LoadCache__New(PT_Index index_, const char *filename) {
const char *scheme =
link_has_authority(filenameIndex) ? "" : "http://";
firstSeen = 1;
snprintf(index->startUrl, sizeof(index->startUrl), "%s%s",
scheme, filenameIndex);
/* dropped rather than truncated; try the next entry */
if (sprintfbuff(index->startUrl, "%s%s", scheme,
filenameIndex))
firstSeen = 1;
else
index->startUrl[0] = '\0';
}
}
} else {
@@ -1070,16 +1091,23 @@ static PT_Element PT_ReadCache__New_u(PT_Index index_, const char *url,
value++;
ZIP_READFIELD_INT(line, value, "X-In-Cache", dataincache);
ZIP_READFIELD_INT(line, value, "X-Statuscode", r->statuscode);
ZIP_READFIELD_STRING(line, value, "X-StatusMessage", r->msg); // msg
ZIP_READFIELD_STRING(line, value, "X-StatusMessage", r->msg,
sizeof(r->msg));
ZIP_READFIELD_INT(line, value, "X-Size", r->size); // size
ZIP_READFIELD_STRING(line, value, "Content-Type", r->contenttype); // contenttype
ZIP_READFIELD_STRING(line, value, "X-Charset", r->charset); // contenttype
ZIP_READFIELD_STRING(line, value, "Last-Modified", r->lastmodified); // last-modified
ZIP_READFIELD_STRING(line, value, "Etag", r->etag); // Etag
ZIP_READFIELD_STRING(line, value, "Location", r->location); // 'location' pour moved
ZIP_READFIELD_STRING(line, value, "Content-Type", r->contenttype,
sizeof(r->contenttype));
ZIP_READFIELD_STRING(line, value, "X-Charset", r->charset,
sizeof(r->charset));
ZIP_READFIELD_STRING(line, value, "Last-Modified",
r->lastmodified, sizeof(r->lastmodified));
ZIP_READFIELD_STRING(line, value, "Etag", r->etag,
sizeof(r->etag));
ZIP_READFIELD_STRING(line, value, "Location", r->location,
sizeof(location_default));
ZIP_READFIELD_STRING(line, value, "Content-Disposition",
r->cdispo); // Content-disposition
ZIP_READFIELD_STRING(line, value, "X-Save", previous_save_); // Original save filename
r->cdispo, sizeof(r->cdispo));
ZIP_READFIELD_STRING(line, value, "X-Save", previous_save_,
sizeof(previous_save_));
if (line[0] != '\0') {
int len = r->headers ? ((int) strlen(r->headers)) : 0;
int nlen =
@@ -1135,8 +1163,9 @@ static PT_Element PT_ReadCache__New_u(PT_Index index_, const char *url,
snprintf(previous_save, sizeof(previous_save), "%s%s",
index->path, previous_save_ + index->fixedPath);
} else {
snprintf(r->msg, sizeof(r->msg), "Bogus fixePath prefix for %s (prefixLen=%d)",
previous_save_, (int) index->fixedPath);
PT_Element_failf(
r, "Bogus fixePath prefix for %s (prefixLen=%d)",
previous_save_, (int) index->fixedPath);
r->statuscode = STATUSCODE_INVALID;
}
} else {
@@ -1155,7 +1184,7 @@ static PT_Element PT_ReadCache__New_u(PT_Index index_, const char *url,
// Peut-on stocker le fichier directement sur disque?
if (ok) {
if (r->msg[0] == '\0') {
strcpy(r->msg, "Cache Read Error : Unexpected error");
strcpybuff(r->msg, "Cache Read Error : Unexpected error");
}
} else { // lire en mémoire
@@ -1173,24 +1202,27 @@ static PT_Element PT_ReadCache__New_u(PT_Index index_, const char *url,
int last_errno = errno;
r->statuscode = STATUSCODE_INVALID;
sprintf(r->msg, "Read error in cache disk data: %s",
strerror(last_errno));
PT_Element_failf(r,
"Read error in cache disk data: %s",
strerror(last_errno));
}
r->adr[r->size] = '\0';
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg,
"Read error (memory exhausted) from cache");
strcpybuff(r->msg,
"Read error (memory exhausted) from cache");
}
fclose(fp);
} else {
r->statuscode = STATUSCODE_INVALID;
snprintf(r->msg, sizeof(r->msg), "Read error (can't open '%s') from cache",
file_convert(catbuff, sizeof(catbuff), previous_save));
PT_Element_failf(
r, "Read error (can't open '%s') from cache",
file_convert(catbuff, sizeof(catbuff),
previous_save));
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cached file name is invalid");
strcpybuff(r->msg, "Cached file name is invalid");
}
}
} else {
@@ -1202,12 +1234,12 @@ static PT_Element PT_ReadCache__New_u(PT_Index index_, const char *url,
free(r->adr);
r->adr = NULL;
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cache Read Error : Read Data");
strcpybuff(r->msg, "Cache Read Error : Read Data");
} else
*(r->adr + r->size) = '\0';
} else { // erreur
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cache Memory Error");
strcpybuff(r->msg, "Cache Memory Error");
}
}
}
@@ -1215,21 +1247,21 @@ static PT_Element PT_ReadCache__New_u(PT_Index index_, const char *url,
} // si save==null, ne rien charger (juste en tête)
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cache Read Error : Read Header Data");
strcpybuff(r->msg, "Cache Read Error : Read Header Data");
}
unzCloseCurrentFile(index->zFile);
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cache Read Error : Open File");
strcpybuff(r->msg, "Cache Read Error : Open File");
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cache Read Error : Bad Offset");
strcpybuff(r->msg, "Cache Read Error : Bad Offset");
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "File Cache Entry Not Found");
strcpybuff(r->msg, "File Cache Entry Not Found");
}
if (r->location[0] != '\0') {
r->location = strdup(r->location);
@@ -1570,9 +1602,11 @@ static int PT_LoadCache__Old(PT_Index index_, const char *filename) {
const char *scheme =
link_has_authority(line) ? "" : "http://";
firstSeen = 1;
snprintf(index->startUrl, sizeof(index->startUrl), "%s%s",
scheme, line);
/* dropped rather than truncated; try the next entry */
if (sprintfbuff(index->startUrl, "%s%s", scheme, line))
firstSeen = 1;
else
index->startUrl[0] = '\0';
}
}
@@ -1793,8 +1827,9 @@ static PT_Element PT_ReadCache__Old_u(PT_Index index_, const char *url,
snprintf(previous_save, sizeof(previous_save), "%s%s",
index->path, previous_save_ + index->fixedPath);
} else {
snprintf(r->msg, sizeof(r->msg), "Bogus fixePath prefix for %s (prefixLen=%d)",
previous_save_, (int) index->fixedPath);
PT_Element_failf(r,
"Bogus fixePath prefix for %s (prefixLen=%d)",
previous_save_, (int) index->fixedPath);
r->statuscode = STATUSCODE_INVALID;
}
} else {
@@ -1820,17 +1855,18 @@ static PT_Element PT_ReadCache__Old_u(PT_Index index_, const char *url,
if (r->adr != NULL) {
if (r->size > 0 && fread(r->adr, 1, r->size, fp) != r->size) {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Read error in cache disk data");
strcpybuff(r->msg, "Read error in cache disk data");
}
r->adr[r->size] = '\0';
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Read error (memory exhausted) from cache");
strcpybuff(r->msg,
"Read error (memory exhausted) from cache");
}
fclose(fp);
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Previous cache file not found (2)");
strcpybuff(r->msg, "Previous cache file not found (2)");
}
}
} else {
@@ -1842,30 +1878,30 @@ static PT_Element PT_ReadCache__Old_u(PT_Index index_, const char *url,
free(r->adr);
r->adr = NULL;
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cache Read Error : Read Data");
strcpybuff(r->msg, "Cache Read Error : Read Data");
} else
r->adr[r->size] = '\0';
} else { // erreur
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cache Memory Error");
strcpybuff(r->msg, "Cache Memory Error");
}
}
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cache Read Error : Bad Data");
strcpybuff(r->msg, "Cache Read Error : Bad Data");
}
} else { // erreur
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cache Read Error : Read Header");
strcpybuff(r->msg, "Cache Read Error : Read Header");
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cache Read Error : Seek Failed");
strcpybuff(r->msg, "Cache Read Error : Seek Failed");
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "File Cache Entry Not Found");
strcpybuff(r->msg, "File Cache Entry Not Found");
}
if (r->location[0] != '\0') {
r->location = strdup(r->location);
@@ -2128,12 +2164,15 @@ int PT_LoadCache__Arc(PT_Index index_, const char *filename) {
return 0;
}
#define HTTP_READFIELD_STRING(line, value, refline, refvalue) do { \
if (line[0] != '\0' && strfield2(line, refline)) { \
strcpy(refvalue, value); \
line[0] = '\0'; \
} \
} while(0)
/* Same contract as ZIP_READFIELD_STRING, for the ARC reader's header lines. */
#define HTTP_READFIELD_STRING(line, value, refline, refvalue, refvalue_size) \
do { \
if (line[0] != '\0' && strfield2(line, refline)) { \
(refvalue)[0] = '\0'; \
strlncatbuff(refvalue, value, refvalue_size, (refvalue_size) - 1); \
line[0] = '\0'; \
} \
} while (0)
#define HTTP_READFIELD_INT(line, value, refline, refvalue) do { \
if (line[0] != '\0' && strfield2(line, refline)) { \
int intval = 0; \
@@ -2191,7 +2230,7 @@ static PT_Element PT_ReadCache__Arc_u(PT_Index index_, const char *url,
}
if ((pos = getArcField(index->line, 2)) != NULL) {
r->msg[0] = '\0';
strncat(r->msg, pos, sizeof(pos) - 1);
strncatbuff(r->msg, pos, sizeof(r->msg) - 1);
}
while(linput(index->file, index->line, sizeof(index->line) - 1)
&& index->line[0] != '\0') {
@@ -2202,11 +2241,16 @@ static PT_Element PT_ReadCache__Arc_u(PT_Index index_, const char *url,
*value = '\0';
for(value++; *value == ' ' || *value == '\t'; value++) ;
HTTP_READFIELD_INT(line, value, "Content-Length", r->size); // size
HTTP_READFIELD_STRING(line, value, "Content-Type", r->contenttype); // contenttype
HTTP_READFIELD_STRING(line, value, "Last-Modified", r->lastmodified); // last-modified
HTTP_READFIELD_STRING(line, value, "Etag", r->etag); // Etag
HTTP_READFIELD_STRING(line, value, "Location", r->location); // 'location' pour moved
HTTP_READFIELD_STRING(line, value, "Content-Disposition", r->cdispo); // Content-disposition
HTTP_READFIELD_STRING(line, value, "Content-Type", r->contenttype,
sizeof(r->contenttype));
HTTP_READFIELD_STRING(line, value, "Last-Modified",
r->lastmodified, sizeof(r->lastmodified));
HTTP_READFIELD_STRING(line, value, "Etag", r->etag,
sizeof(r->etag));
HTTP_READFIELD_STRING(line, value, "Location", r->location,
sizeof(location_default));
HTTP_READFIELD_STRING(line, value, "Content-Disposition",
r->cdispo, sizeof(r->cdispo));
if (line[0] != '\0') {
int len = r->headers ? ((int) strlen(r->headers)) : 0;
int nlen =
@@ -2246,7 +2290,7 @@ static PT_Element PT_ReadCache__Arc_u(PT_Index index_, const char *url,
fetchSize = dataLength - metaSize;
} else if (fetchSize > dataLength - metaSize) {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cache Read Error : Truncated Data");
strcpybuff(r->msg, "Cache Read Error : Truncated Data");
}
r->size = 0;
if (r->statuscode != STATUSCODE_INVALID) {
@@ -2259,12 +2303,13 @@ static PT_Element PT_ReadCache__Arc_u(PT_Index index_, const char *url,
int last_errno = errno;
r->statuscode = STATUSCODE_INVALID;
sprintf(r->msg, "Read error in cache disk data: %s",
strerror(last_errno));
PT_Element_failf(r, "Read error in cache disk data: %s",
strerror(last_errno));
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Read error (memory exhausted) from cache");
strcpybuff(r->msg,
"Read error (memory exhausted) from cache");
}
}
}
@@ -2272,21 +2317,21 @@ static PT_Element PT_ReadCache__Arc_u(PT_Index index_, const char *url,
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cache Read Error : Read Header Error");
strcpybuff(r->msg, "Cache Read Error : Read Header Error");
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cache Read Error : Read Header Error");
strcpybuff(r->msg, "Cache Read Error : Read Header Error");
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "Cache Read Error : Seek Error");
strcpybuff(r->msg, "Cache Read Error : Seek Error");
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpy(r->msg, "File Cache Entry Not Found");
strcpybuff(r->msg, "File Cache Entry Not Found");
}
if (r->location[0] != '\0') {
r->location = strdup(r->location);

Some files were not shown because too many files have changed in this diff Show More