Compare commits

..

8 Commits

Author SHA1 Message Date
Xavier Roche
64cdbaaf3d Expose post-3.49-2 engine options in webhttrack
The web GUI was frozen before the engine gained SOCKS5 proxying, --cookies-file,
--pause, --strip-query, and the three --keep-* URL-hack opt-outs. Wire each into
the option tabs, the generated httrack command, the winprofile.ini persistence,
and the reload remap so they reach parity with WinHTTrack (xroche/httrack-windows#31).

The SOCKS5 selector reuses the single proxy field and prepends socks5:// to -P;
it is gated on a non-empty host so a SOCKS5-selected-but-blank config can't emit a
stray scheme token. winprofile.ini key names match WinHTTrack so projects round-trip
across both GUIs. Labels/tooltips reuse the LANG_ strings already added in #588.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-15 19:45:16 +02:00
Xavier Roche
89b1e485d7 Add and translate the new WinHTTrack GUI option strings (#588)
Add 14 new LANG_ keys (proxy type, cookies-file, pause-between-files, keep-www / keep-double-slashes / keep-query-order, strip-query, each with a tooltip) to lang.def and English.txt, and translate them across all 30 language files. Purely additive; each file keeps its own legacy on-disk charset and line endings (Svenska stores Latin-1 despite its ISO-8859-2 header; Chinese-BIG5 is cp950; Uzbek is Latin-script and LF-terminated). Windows libhttrack checks were red on an unrelated vcpkg download.qt.io network flake.
2026-07-15 19:19:03 +02:00
Xavier Roche
08481464bd Fuzz the real HTML parser (htsparse) with a mocked engine (#586)
Adds fuzz-htsparse, a libFuzzer harness driving the real htsparse() over a
mocked engine: the tag/attribute/JS-inscript scanner, the link rewriter, and
the accept -> url_savename -> hts_record_link path that runs on every crawled
page, previously reached only by slow full-crawl tests. The parser's coupling
is state, not network, so the harness builds the minimal crawl state
httpmirror() sets up (opt, cache, hash, filters, robots, backing, a seeded
link heap with ptr=1) and walks a NUL-terminated page through the parser, then
discards it. Four seeds cover links, srcset, base/meta/usemap, a
document.write body, and malformed tags. Clean under ASan+UBSan+LeakSanitizer.

Bundles the src/coucal bump to ab59c6a (coucal#13): this harness is the first
fuzzer to hash keys through coucal under UBSan, which tripped a pre-existing
getblock32 pointer-overflow.

Merged past a transient arm64 apt-mirror outage; every code-exercising check
(fuzz, sanitize, msan, distcheck, all non-arm64 builds) passed.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 19:18:59 +02:00
Xavier Roche
3e7b9f91ee Free opt->accept and opt->headers in hts_free_opt (#585)
hts_free_opt frees about twenty of the option struct's String fields but omits
opt->accept and opt->headers, both allocated in hts_create_opt. Every opt
create/free cycle leaks the two. Harmless for the CLI but real, and it makes
LeakSanitizer noisy: the fuzz job runs detect_leaks=1, so any harness that
builds an opt trips on it.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 18:53:14 +02:00
Xavier Roche
800d498893 Keep the mirrored file when an --update re-fetch comes in short (#584)
* Keep the mirrored file when an --update re-fetch comes in short (#562)

An --update re-fetch that arrives incomplete (server declares a Content-Length
it does not deliver, or the connection drops mid-body) is correctly refused:
httrack logs "incomplete transfer ... will be retried on the next update" and
does not store the partial body. But the incomplete-transfer branch in
back_finalize() returned without noting the surviving copy, so the URL never
reached new.lst and the end-of-update purge deleted the good file it had just
declined to overwrite.

filenote() the previous copy when it still exists on disk, mirroring the
decode-failure path added in #560. Not compression-specific and needs no abort:
a plain HTML page reproduces it, so it is broader than #521 (abort) and #557
(decode failure).

Closes #562

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Reflow the #562 comment to satisfy clang-format

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: share one fetch-pass counter across the update-refetch routes

The #562 /uptrunc/ routes copied upcodec's per-path body-fetch counter verbatim. Fold both onto one refetch_pass() helper and one dict.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 17:26:30 +02:00
Xavier Roche
991d259981 tests: run the connect-fallback crawl on Windows now that #580 landed (#583)
#580 fixed the Winsock connect fallback (closed #579), so 19_local-connect-fallback
can exercise the real path on Windows. Drop its Windows skip and remove it from
the workflow's expected-skips.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 08:43:58 +02:00
Xavier Roche
d1bf4f7a25 Export hts_convertStringSystemToUTF8 from the DLL (#582)
Mark the CP_ACP->UTF-8 converter HTSEXT_API, matching hts_argv_utf8 next
to it, so WinHTTrack can call the engine's converter instead of keeping a
private copy. Windows-only (inside the _WIN32 block), so no POSIX ABI
change and no soname bump.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 08:15:07 +02:00
Xavier Roche
86e145fc1c Run the loopback crawl tests on Windows (#578)
* tests: run the loopback crawl suite on Windows

The ~40 *_local-* tests crawl the bundled Python server: the real TLS
handshake, cache, and file writer, none of which any Windows check covered.
Three things stopped them running there. python3 is python.exe on Windows;
MSYS hands out /d/a/... paths a native python.exe cannot resolve (and arg
rewriting is off, so nothing fixes them up); and Python's text layer turns the
"PORT <n>" discovery line into CRLF, so the \r landed in the parsed port.

Factor the python lookup and the path conversion into tests/testlib.sh, fix the
newline at the source in the three servers, and skip the --file-mode assertion
on Windows, where the engine does not chmod. Behaviour on POSIX is unchanged:
the full suite still passes 97/0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: share the server shutdown, and make a skip fail the Windows run

stop_server moves to testlib.sh: the MSYS "a signal cannot reach a native
python.exe, only -9 lands" knowledge was in one of the nine places that kill a
python server. Every step is "|| true" because the callers run under set -e and
reaping a server we just signalled makes wait return 143.

Nothing is expected to skip on Windows, so treat a skip as a failure: the pass
floor alone left slack for exactly the tests that can silently gate themselves
off (TLS, the content codings, socks5, connect-fallback). Add the local proxy
crawl, which the glob was missing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: resolve the server path before backgrounding the server

The port-discovery poll raced the server it had just started: the native path
conversion sat inside the backgrounded command, so the child ran two forks
(command -v cygpath, cygpath) before applying its ">server.log" redirect, and
on Windows the parent's first "head" reached the file first. head then failed,
and under set -e that killed the test silently.

Resolve the paths in the parent, and create the log before the launch so the
first poll cannot lose the race. nativepath keys off the platform, not off
cygpath happening to be on PATH.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: skip the connect-fallback crawl on Windows

19_local-connect-fallback fails there because the engine never falls back to
the next address: Winsock reports a failed connect in select()'s exception set
rather than as writable, and the exception loop fails the slot before
back_connect_next() is reached. Skip it pending the engine fix (#579).

Pin the expected skips instead of counting them, so a gate that silently turns
some other test off still fails the run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: trim the comments added by the Windows port

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: say why the crange resume lost its file

48_local-crange-memresume fails on Windows with a bare "blob.bin missing", and
the crawl log it would have to explain that lives in the tmpdir the test wipes.
Dump the engine's errors and the mirror on that branch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: dump the crawl log when the crange resume loses its file

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: check the resume pass survived, don't assume it

48_local-crange-memresume ran pass 2 and printed "terminated" whatever came
back, so a crashed engine read as a clean run. It fails on Windows with an
empty log and an empty mirror, which is what that blind spot looks like.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: dump both passes' state for the Windows crange failure

Temporary: pass-1 mirror + log and pass-2 log/mirror, to see why the resume
mirrors and logs nothing on Windows.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* tests: skip the crange memory-resume crawl on Windows

48_local-crange-memresume needs a graceful pass-1 interrupt so the cache is
clean when pass 2 resumes; MSYS can only hard-kill a native exe, and the
engine's restart-whole path after an unusable 206 then fails on the repaired
cache (#581). Skip on Windows pending that fix.

Keep the pass-2 exit-code check the investigation added: the test printed
"terminated" whatever came back, so a crashed engine read as a clean run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 07:41:41 +02:00
62 changed files with 1445 additions and 103 deletions

View File

@@ -123,9 +123,12 @@ jobs:
# is Linux/macOS only. These are the offline ones, driven from Git Bash
# against the native httrack.exe. They subsume the self-tests this step
# used to run inline (codecs, cache, fsize).
# The *_local-* ones crawl the bundled Python server over loopback: the real
# TLS handshake, cache and file writer, which nothing else on Windows covers.
- name: Run the engine test suite (offline tests)
shell: bash
working-directory: tests
timeout-minutes: 45
run: |
set -u
bin="$(cygpath -u "$GITHUB_WORKSPACE")/src/${{ matrix.platform }}/${{ matrix.configuration }}"
@@ -141,13 +144,24 @@ jobs:
TMPDIR="$(cygpath -m "$RUNNER_TEMP")"
export TMPDIR
pass=0 fail=0 skip=0 failed=""
for t in 00_runnable.test 01_engine-*.test 01_zlib-*.test; do
# Mirror what configure hands the suite. LC_ALL sets the codeset MSYS maps
# a UTF-8 mirror name onto UTF-16 with, which the intl crawls "test -f".
export HTTPS_SUPPORT=yes BROTLI_ENABLED=yes ZSTD_ENABLED=yes
export LC_ALL=C.UTF-8
# A wedged crawl must not eat the job's whole timeout budget.
watchdog=""
command -v timeout >/dev/null && watchdog="timeout 600"
pass=0 fail=0 skip=0 failed="" skipped=""
for t in 00_runnable.test 01_engine-*.test 01_zlib-*.test \
*_local-*.test 13_crawl_proxy_https.test; do
rc=0
bash "$t" >"$t.log" 2>&1 || rc=$?
# shellcheck disable=SC2086
$watchdog bash "$t" >"$t.log" 2>&1 || rc=$?
case "$rc" in
0) pass=$((pass + 1)); echo "PASS $t" ;;
77) skip=$((skip + 1)); echo "SKIP $t" ;;
77) skip=$((skip + 1)) skipped="$skipped $t"; echo "SKIP $t" ;;
*)
fail=$((fail + 1)) failed="$failed $t"
echo "FAIL $t (exit $rc)"
@@ -161,10 +175,11 @@ jobs:
echo "ran=$((pass + fail + skip)) pass=$pass fail=$fail skip=$skip" |
tee -a "$GITHUB_STEP_SUMMARY"
# Every gate in these scripts exits 77, so a suite that degraded to
# all-skipped would report green having tested nothing: assert a floor
# on what actually ran, not just the absence of failures.
[ "$pass" -ge 45 ] || { echo "::error::only $pass tests passed ($skip skipped)"; exit 1; }
# Every gate here exits 77, so an all-skipped suite would report green having
# tested nothing: pin the skips, and floor the passes in case the glob empties.
expected_skips=" 48_local-crange-memresume.test" # pending #581
[ "$pass" -ge 90 ] || { echo "::error::only $pass tests passed ($skip skipped)"; exit 1; }
[ "$skipped" = "$expected_skips" ] || { echo "::error::unexpected skips:$skipped"; exit 1; }
[ "$fail" -eq 0 ] || { echo "::error::failing:$failed"; exit 1; }
- name: Upload the test logs

View File

@@ -1,7 +1,8 @@
# libFuzzer harnesses; built only with --enable-fuzzers (requires clang).
if FUZZERS
noinst_PROGRAMS = fuzz-charset fuzz-meta fuzz-idna fuzz-entities \
fuzz-unescape fuzz-filters fuzz-url fuzz-header fuzz-cachendx
fuzz-unescape fuzz-filters fuzz-url fuzz-header fuzz-cachendx \
fuzz-htsparse
endif
AM_CPPFLAGS = \
@@ -25,6 +26,7 @@ fuzz_filters_SOURCES = fuzz-filters.c fuzz.h
fuzz_url_SOURCES = fuzz-url.c fuzz.h
fuzz_header_SOURCES = fuzz-header.c fuzz.h
fuzz_cachendx_SOURCES = fuzz-cachendx.c fuzz.h
fuzz_htsparse_SOURCES = fuzz-htsparse.c fuzz.h
# List corpus files explicitly: automake does not expand EXTRA_DIST globs.
EXTRA_DIST = README.md run-fuzzers.sh \
@@ -42,4 +44,6 @@ EXTRA_DIST = README.md run-fuzzers.sh \
corpus/header/full-response.txt corpus/header/redirect.txt \
corpus/cachendx/new-format.txt corpus/cachendx/old-format.txt \
corpus/cachendx/regress-overadvance.bin \
corpus/cachendx/regress-truncated-entry.bin
corpus/cachendx/regress-truncated-entry.bin \
corpus/htsparse/basic.html corpus/htsparse/script-inscript.html \
corpus/htsparse/meta-usemap.html corpus/htsparse/malformed.html

View File

@@ -0,0 +1,15 @@
<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<base href="http://example.com/dir/">
<title>Seed</title>
<link rel="stylesheet" href="style.css">
<script src="app.js"></script>
</head><body>
<h1>Hi</h1>
<a href="page2.html">next</a>
<a href="http://other.example.org/x?y=1#frag">abs</a>
<img src="pic.png" srcset="a.png 1x, b.png 2x">
<script>var u="inline.html"; document.write('<a href="gen.html">g</a>');</script>
<form action="/cgi/submit"><input name="q"></form>
</body></html>

View File

@@ -0,0 +1,8 @@
<a href="unclosed.html
<img src='mix"ed.png>
<a href=noquote.html >bare</a>
<!-- comment <a href="incomment.html"> -->
<a href="tab newline
.html">wsp</a>
<script>unterminated "string and <a href=
<a href=

View File

@@ -0,0 +1,11 @@
<html><head>
<meta http-equiv="refresh" content="0; url=redir.html">
<base href="http://h/b/">
<link rel="alternate" href="feed.xml">
</head><body background="page-bg.jpg">
<img src="map.png" usemap="#m">
<map name="m"><area href="area1.html" coords="0,0,10,10"></map>
<applet code="A.class" codebase="applets/"><param name="src" value="p.dat"></applet>
<object data="o.swf"><embed src="e.svg"></object>
<a href="&#104;ttp://ent/e.html">entity</a>
</body></html>

View File

@@ -0,0 +1,13 @@
<html><body>
<script type="text/javascript">
var a = "http://x/1.html", b = 'q\'uote', c = "/*not*/comment";
// line "with" quotes and http://y/2.html
/* block 'with' <a href="notparsed"> */
var re = /a\/b\//g;
document.write('<img src="w1.png">');
document.writeln("<a href='w2.html'>k</a>");
element.onclick = "location='onh.html'";
</script>
<a href="after.html" onmouseover="go('ev.html')">x</a>
<div style="background:url(bg.png)">z</div>
</body></html>

175
fuzz/fuzz-htsparse.c Normal file
View File

@@ -0,0 +1,175 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 2026 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Please visit our Website: http://www.httrack.com
*/
/* Fuzz the real htsparse() over a mocked engine: the minimal crawl state
httpmirror() builds, then a page walked through the parser and discarded. The
str/stre wiring below mirrors htsparse()'s call site in htscore.c; update it
in lockstep if those structs gain a field the parser reads. */
#include "fuzz.h"
#include "httrack-library.h"
#include "htscore.h"
#include "htsback.h"
#include "htshash.h"
#include "htsrobots.h"
#include "htsparse.h"
#include "htsmodules.h"
#include "coucal.h"
/* htsparse ignores str.addLink on the internal parse; stub it. */
static int fuzz_addlink(htsmoduleStruct *str, char *link) {
(void) str;
(void) link;
return 0;
}
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
static int inited = 0;
httrackp *opt;
cache_back cache;
hash_struct hash;
robots_wizard robots;
struct_back *sback;
char **filters = NULL;
int filptr = 0;
htsblk r;
htsmoduleStruct str;
htsmoduleStructExtended stre;
int ptr, error = 0, store_errpage = 0;
int makeindex_done = 0, makeindex_links = 0;
FILE *makeindex_fp = NULL;
char makeindex_firstlink[HTS_URLMAXSIZE * 2] = "";
LLint stat_fragment = 0, makestat_total = 0;
int makestat_lnk = 0;
char base[HTS_URLMAXSIZE * 2] = "";
char codebase[HTS_URLMAXSIZE * 2] = "";
char err_msg[1024] = "";
if (!inited) {
hts_init();
inited = 1;
}
opt = hts_create_opt();
opt->log = opt->errlog = NULL;
opt->robots = 0;
memset(&cache, 0, sizeof(cache));
cache.type = 0; /* no on-disk cache */
cache.hashtable = coucal_new(0);
cache.cached_tests = coucal_new(0);
coucal_value_is_malloc(cache.cached_tests, 1);
memset(&robots, 0, sizeof(robots));
strcpybuff(robots.adr, "!");
opt->robotsptr = &robots;
opt->maxfilter = maximum(opt->maxfilter, 128);
filters_init(&filters, opt->maxfilter, 0);
opt->filters.filters = &filters;
opt->filters.filptr = &filptr;
opt->hash = &hash;
hts_record_init(opt);
hash_init(opt, &hash, opt->urlhack);
hash.liens = (const lien_url *const *const *) &opt->liens;
sback = back_new(opt, opt->maxsoc * 32 + 1024);
/* ptr=1 (index 1 is the parsed page) selects full HTML parsing + the
rewriter; urladr()/urlfil()/savename() alias heap(ptr), save=/dev/null. */
hts_record_link(opt, "example.com", "/", "/dev/null", "", "", NULL);
hts_record_link(opt, "example.com", "/index.html", "/dev/null", "", "", NULL);
ptr = 1;
/* NUL-terminated in a size+1 alloc: htsparse one-past-reads onto the NUL. */
hts_init_htsblk(&r);
r.statuscode = 200;
r.size = (LLint) size;
r.adr = malloct(size + 1);
if (size)
memcpy(r.adr, data, size);
r.adr[size] = '\0';
strcpybuff(r.contenttype, "text/html");
memset(&str, 0, sizeof(str));
memset(&stre, 0, sizeof(stre));
str.err_msg = err_msg;
str.filename = heap(ptr)->sav;
str.mime = r.contenttype;
str.url_host = heap(ptr)->adr;
str.url_file = heap(ptr)->fil;
str.size = (int) r.size;
str.addLink = fuzz_addlink;
str.opt = opt;
str.sback = sback;
str.cache = &cache;
str.hashptr = &hash;
str.numero_passe = 0;
str.ptr_ = &ptr;
str.page_charset_ = NULL;
stre.r_ = &r;
stre.error_ = &error;
stre.exit_xh_ = &opt->state.exit_xh;
stre.store_errpage_ = &store_errpage;
stre.base = base;
stre.codebase = codebase;
stre.filters_ = &filters;
stre.filptr_ = &filptr;
stre.robots_ = &robots;
stre.hash_ = &hash;
stre.makeindex_done_ = &makeindex_done;
stre.makeindex_fp_ = &makeindex_fp;
stre.makeindex_links_ = &makeindex_links;
stre.makeindex_firstlink_ = makeindex_firstlink;
stre.template_header_ = "";
stre.template_body_ = "";
stre.template_footer_ = "";
stre.stat_fragment_ = &stat_fragment;
stre.makestat_time = 0;
stre.makestat_fp = NULL;
stre.makestat_total_ = &makestat_total;
stre.makestat_lnk_ = &makestat_lnk;
stre.maketrack_fp = NULL;
(void) htsparse(&str, &stre);
freet(r.adr);
back_delete_all(opt, &cache, sback);
back_free(&sback);
hash_free(&hash);
coucal_delete(&cache.hashtable);
coucal_delete(&cache.cached_tests);
checkrobots_free(&robots);
if (filters != NULL) {
if (filters[0] != NULL)
freet(filters[0]);
freet(filters);
}
hts_record_free(opt);
hts_free_opt(opt);
return 0;
}

View File

@@ -98,7 +98,16 @@ ${do:end-if}
<input type="hidden" name="redirect" value="">
<input type="hidden" name="closeme" value="">
${LANG_IOPT10}:
${LANG_PROXYTYPE}
<select name="proxytype"
onMouseOver="info('${html:LANG_PROXYTYPETIP}'); return true" onMouseOut="info('&nbsp;'); return true"
>
<option value=""${ztest:proxytype: selected:}>HTTP</option>
<option value="1"${test:proxytype:: selected}>SOCKS5</option>
</select>
<br><br>
${LANG_IOPT10}:
<input name="prox" value="${prox}" size="32"
onMouseOver="info('${html:LANG_G14}'); return true" onMouseOut="info('&nbsp;'); return true"
>:

View File

@@ -148,6 +148,14 @@ ${LANG_I65}
>
</td></tr>
<tr><td>
${LANG_PAUSEFILES}
</td><td>
<input name="pausefiles" value="${pausefiles}" size="8"
onMouseOver="info('${html:LANG_PAUSEFILESTIP}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
<tr><td>
${LANG_I52}
</td><td>

View File

@@ -103,6 +103,12 @@ ${do:end-if}
> ${LANG_I58}
<br><br>
${LANG_COOKIEFILE}
<input name="cookiesfile" value="${cookiesfile}" size="40"
onMouseOver="info('${html:LANG_COOKIEFILETIP}'); return true" onMouseOut="info('&nbsp;'); return true"
>
<br><br>
${LANG_I59}
<br>
<select name="checktype"
@@ -136,6 +142,27 @@ ${listid:robots:LISTDEF_8}
> ${LANG_I62b2}
<br><br>
<input type="checkbox" name="keepwww" ${checked:keepwww}
onMouseOver="info('${html:LANG_KEEPWWWTIP}'); return true" onMouseOut="info('&nbsp;'); return true"
> ${LANG_KEEPWWW}
<br><br>
<input type="checkbox" name="keepslashes" ${checked:keepslashes}
onMouseOver="info('${html:LANG_KEEPSLASHESTIP}'); return true" onMouseOut="info('&nbsp;'); return true"
> ${LANG_KEEPSLASHES}
<br><br>
<input type="checkbox" name="keepqueryorder" ${checked:keepqueryorder}
onMouseOver="info('${html:LANG_KEEPQUERYORDERTIP}'); return true" onMouseOut="info('&nbsp;'); return true"
> ${LANG_KEEPQUERYORDER}
<br><br>
${LANG_STRIPQUERY}
<input name="stripquery" value="${stripquery}" size="40"
onMouseOver="info('${html:LANG_STRIPQUERYTIP}'); return true" onMouseOut="info('&nbsp;'); return true"
>
<br><br>
<input type="checkbox" name="toler" ${checked:toler}
onMouseOver="info('${html:LANG_I1i}'); return true" onMouseOut="info('&nbsp;'); return true"
> ${LANG_I62}

View File

@@ -129,15 +129,21 @@ ${do:copy:NoPwdInPages:hidepwd}
${do:copy:NoQueryStrings:hidequery}
${do:copy:NoPurgeOldFiles:nopurge}
${do:copy:Cookies:cookies}
${do:copy:CookiesFile:cookiesfile}
${do:copy:CheckType:checktype}
${do:copy:ParseJava:parsejava}
${do:copy:HTTP10:http10}
${do:copy:TolerantRequests:toler}
${do:copy:UpdateHack:updhack}
${do:copy:URLHack:urlhack}
${do:copy:KeepWww:keepwww}
${do:copy:KeepSlashes:keepslashes}
${do:copy:KeepQueryOrder:keepqueryorder}
${do:copy:StripQuery:stripquery}
${do:copy:StoreAllInCache:cache2}
${do:copy:LogType:logtype}
${do:copy:UseHTTPProxyForFTP:ftpprox}
${do:copy:ProxyType:proxytype}
${do:copy:Build:build}
${do:copy:PrimaryScan:filter}
${do:copy:Travel:travel}
@@ -148,6 +154,7 @@ ${do:copy:MaxHtml:maxhtml}
${do:copy:MaxOther:othermax}
${do:copy:MaxAll:sizemax}
${do:copy:MaxWait:pausebytes}
${do:copy:PauseFiles:pausefiles}
${do:copy:Sockets:connexion}
${do:copy:Retry:retry}
${do:copy:MaxTime:maxtime}

View File

@@ -178,6 +178,12 @@ ${do:end-if}
${test:parsejava:--parse-java=0:}
${test:updhack:--updatehack}
${test:urlhack:--urlhack=0:--urlhack}
${test:keepwww:--keep-www-prefix}
${test:keepslashes:--keep-double-slashes}
${test:keepqueryorder:--keep-query-order}
${test:cookiesfile:--cookies-file "}${html:cookiesfile}${test:cookiesfile:"}
${test:pausefiles:--pause "}${pausefiles}${test:pausefiles:"}
${test:stripquery:--strip-query "}${html:stripquery}${test:stripquery:"}
${test:toler:--tolerant}
${test:http10:--http-10}
${test:cache2:--store-all-in-cache}
@@ -186,7 +192,7 @@ ${do:end-if}
${test:logtype:::--extra-log:--debug-log}
${test:index:--index=0:}
${test:index2:--search-index=0:--search-index}
${test:prox:--proxy "}${prox}${test:prox:\3A}${portprox}${test:prox:"}
${test:prox:--proxy "}${do:if-not-empty:prox}${test:proxytype:socks5\3A//}${do:end-if}${do:output-mode:html}${prox}${test:prox:\3A}${portprox}${test:prox:"}
${test:ftpprox:--httpproxy-ftp=0:--httpproxy-ftp}
</textarea>
@@ -219,15 +225,21 @@ NoPwdInPages=${ztest:hidepwd:0:1}
NoQueryStrings=${ztest:hidequery:0:1}
NoPurgeOldFiles=${ztest:nopurge:0:1}
Cookies=${ztest:cookies:0:1}
CookiesFile=${cookiesfile}
CheckType=${ztest:checktype:0:1:2}
ParseJava=${ztest:parsejava:0:1}
HTTP10=${ztest:http10:0:1}
TolerantRequests=${ztest:toler:0:1}
UpdateHack=${ztest:updhack:0:1}
URLHack=${ztest:urlhack:0:1}
KeepWww=${ztest:keepwww:0:1}
KeepSlashes=${ztest:keepslashes:0:1}
KeepQueryOrder=${ztest:keepqueryorder:0:1}
StripQuery=${stripquery}
StoreAllInCache=${ztest:cache2:0:1}
LogType=${logtype}
UseHTTPProxyForFTP=${ztest:ftpprox:0:1}
ProxyType=${proxytype}
Build=${build}
PrimaryScan=${filter}
Travel=${travel}
@@ -238,6 +250,7 @@ MaxHtml=${maxhtml}
MaxOther=${othermax}
MaxAll=${sizemax}
MaxWait=${pausebytes}
PauseFiles=${pausefiles}
Sockets=${connexion}
Retry=${retry}
MaxTime=${maxtime}

View File

@@ -1006,3 +1006,31 @@ LANG_SERVEND
Server terminated
LANG_FATALERR
A fatal error has occurred during this mirror
LANG_PROXYTYPE
Proxy type:
LANG_PROXYTYPETIP
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
LANG_COOKIEFILE
Load cookies from file:
LANG_COOKIEFILETIP
Preload cookies from a Netscape cookies.txt file before crawling.
LANG_PAUSEFILES
Pause between files:
LANG_PAUSEFILESTIP
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
LANG_KEEPWWW
Keep the www. prefix (do not merge www.host with host)
LANG_KEEPWWWTIP
Do not treat www.host and host as the same site.
LANG_KEEPSLASHES
Keep double slashes in URLs
LANG_KEEPSLASHESTIP
Do not collapse duplicate slashes in URLs.
LANG_KEEPQUERYORDER
Keep the original query-string order
LANG_KEEPQUERYORDERTIP
Do not reorder query-string parameters when deduplicating URLs.
LANG_STRIPQUERY
Strip query keys:
LANG_STRIPQUERYTIP
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
Ñúðâúðúò íå îòãîâàðÿ
A fatal error has occurred during this mirror
Ôàòàëíà ãðåøêà ïðè ñúçäàâàíåòî íà òîçè îãëåäàëåí ñàéò
Proxy type:
Òèï íà ïðîêñè:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Ïðîòîêîë íà ïðîêñè: HTTP èëè SOCKS5 (ñòàíäàðòíèÿò ïîðò íà SOCKS5 å 1080).
Load cookies from file:
Çàðåæäàíå íà 'áèñêâèòêè' îò ôàéë:
Preload cookies from a Netscape cookies.txt file before crawling.
Ïðåäâàðèòåëíî çàðåæäàíå íà 'áèñêâèòêè' îò ôàéë Netscape cookies.txt ïðåäè îáõîæäàíåòî.
Pause between files:
Ïàóçà ìåæäó ôàéëîâåòå:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Ñëó÷àéíî çàáàâÿíå ìåæäó èçòåãëÿíèÿòà íà ôàéëîâå, â ñåêóíäè. Èçïîëçâàéòå MIN:MAX çà ñëó÷àåí äèàïàçîí (íàïðèìåð 2:8).
Keep the www. prefix (do not merge www.host with host)
Çàïàçâàíå íà ïðåôèêñà www. (áåç ñëèâàíå íà www.host ñ host)
Do not treat www.host and host as the same site.
www.host è host äà íå ñå òðåòèðàò êàòî åäèí è ñúù ñàéò.
Keep double slashes in URLs
Çàïàçâàíå íà äâîéíèòå íàêëîíåíè ÷åðòè â URL
Do not collapse duplicate slashes in URLs.
Áåç ïðåìàõâàíå íà ïîâòàðÿùèòå ñå íàêëîíåíè ÷åðòè â URL.
Keep the original query-string order
Çàïàçâàíå íà îðèãèíàëíèÿ ðåä íà ïàðàìåòðèòå â çàÿâêàòà
Do not reorder query-string parameters when deduplicating URLs.
Áåç ïðåíàðåæäàíå íà ïàðàìåòðèòå íà çàÿâêàòà ïðè ïðåìàõâàíå íà äóáëèðàíè URL.
Strip query keys:
Ïðåìàõâàíå íà êëþ÷îâå îò çàÿâêàòà:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Êëþ÷îâå îò çàÿâêàòà, ðàçäåëåíè ñúñ çàïåòàÿ, êîèòî äà ñå ïðåìàõíàò îò èìåòî íà çàïèñàíèÿ ôàéë (íàïðèìåð sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
Servidor desconectado
A fatal error has occurred during this mirror
Ha ocurrido un error fatal durante esta copia
Proxy type:
Tipo de proxy:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Protocolo del proxy: HTTP o SOCKS5 (el puerto SOCKS5 predeterminado es 1080).
Load cookies from file:
Cargar cookies desde un archivo:
Preload cookies from a Netscape cookies.txt file before crawling.
Precargar cookies desde un archivo Netscape cookies.txt antes de comenzar la descarga.
Pause between files:
Pausa entre archivos:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Retardo aleatorio entre descargas de archivos, en segundos. Use MIN:MAX para un rango aleatorio (p. ej. 2:8).
Keep the www. prefix (do not merge www.host with host)
Mantener el prefijo www. (no combinar www.host con host)
Do not treat www.host and host as the same site.
No tratar www.host y host como el mismo sitio.
Keep double slashes in URLs
Mantener las barras dobles en las URL
Do not collapse duplicate slashes in URLs.
No colapsar las barras duplicadas en las URL.
Keep the original query-string order
Mantener el orden original de la query string
Do not reorder query-string parameters when deduplicating URLs.
No reordenar los parámetros de la query string al deduplicar las URL.
Strip query keys:
Eliminar claves de query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Claves de query string, separadas por comas, que se eliminarán del nombre de los archivos guardados (p. ej. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
A fatal error has occurred during this mirror
Proxy type:
Typ proxy:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Protokol proxy: HTTP nebo SOCKS5 (výchozí port SOCKS5 je 1080).
Load cookies from file:
Naèíst cookies ze souboru:
Preload cookies from a Netscape cookies.txt file before crawling.
Naèíst cookies ze souboru Netscape cookies.txt pøed zahájením stahování.
Pause between files:
Prodleva mezi soubory:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Náhodná prodleva mezi stahováním souborù, v sekundách. Pro náhodný rozsah použijte MIN:MAX (napø. 2:8).
Keep the www. prefix (do not merge www.host with host)
Zachovat pøedponu www. (nesluèovat www.host s host)
Do not treat www.host and host as the same site.
Nepovažovat www.host a host za stejný web.
Keep double slashes in URLs
Zachovat dvojitá lomítka v URL
Do not collapse duplicate slashes in URLs.
Nesluèovat opakovaná lomítka v URL.
Keep the original query-string order
Zachovat pùvodní poøadí øetìzce dotazu
Do not reorder query-string parameters when deduplicating URLs.
Nemìnit poøadí parametrù øetìzce dotazu pøi odstraòování duplicitních URL.
Strip query keys:
Odebrat klíèe dotazu:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Klíèe dotazu oddìlené èárkami, které se vynechají z pojmenování ukládaných souborù (napø. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
伺服器已終止
A fatal error has occurred during this mirror
這鏡像發生了不可回復的錯誤
Proxy type:
proxy 類型:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
proxy 協定: HTTP 或 SOCKS5 (SOCKS5 的預設連接埠為 1080)。
Load cookies from file:
從檔案載入 cookies:
Preload cookies from a Netscape cookies.txt file before crawling.
在抓取前從 Netscape 格式的 cookies.txt 檔案預先載入 cookies。
Pause between files:
檔案之間暫停:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
檔案下載之間的隨機延遲(秒)。使用 MIN:MAX 指定隨機範圍 (例如 2:8)。
Keep the www. prefix (do not merge www.host with host)
保留 www. 前綴 (不將 www.host 與 host 合併)
Do not treat www.host and host as the same site.
不將 www.host 與 host 視為同一站點。
Keep double slashes in URLs
保留 URL 中的雙斜線
Do not collapse duplicate slashes in URLs.
不合併 URL 中重複的斜線。
Keep the original query-string order
保留查詢字串的原始順序
Do not reorder query-string parameters when deduplicating URLs.
在對 URL 去重時不重新排列查詢字串參數。
Strip query keys:
移除查詢鍵:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
以逗號分隔的查詢鍵,將其從儲存檔案的命名中移除 (例如 sid,utm_source)。

View File

@@ -928,3 +928,31 @@ Server terminated
A fatal error has occurred during this mirror
Proxy type:
代理类型:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
代理协议: HTTP 或 SOCKS5 (SOCKS5 的默认端口为 1080)。
Load cookies from file:
从文件加载 cookies:
Preload cookies from a Netscape cookies.txt file before crawling.
在抓取前从 Netscape 格式的 cookies.txt 文件预加载 cookies。
Pause between files:
文件间暂停:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
文件下载之间的随机延迟(秒)。使用 MIN:MAX 指定随机范围 (例如 2:8)。
Keep the www. prefix (do not merge www.host with host)
保留 www. 前缀 (不将 www.host 与 host 合并)
Do not treat www.host and host as the same site.
不把 www.host 和 host 视为同一站点。
Keep double slashes in URLs
保留 URL 中的双斜杠
Do not collapse duplicate slashes in URLs.
不合并 URL 中重复的斜杠。
Keep the original query-string order
保留查询字符串的原始顺序
Do not reorder query-string parameters when deduplicating URLs.
在对 URL 去重时不重新排列查询字符串参数。
Strip query keys:
剥离查询键:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
用逗号分隔的查询键,将其从保存文件的命名中删除 (例如 sid,utm_source)。

View File

@@ -930,3 +930,31 @@ Server terminated
Poslužitelj je razriješen
A fatal error has occurred during this mirror
Tijekom ovog zrcaljenja je nastala fatalna pogreška
Proxy type:
Vrsta posrednika:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Protokol posrednika: HTTP ili SOCKS5 (zadani port za SOCKS5 je 1080).
Load cookies from file:
Uèitati kolaèiæe iz datoteke:
Preload cookies from a Netscape cookies.txt file before crawling.
Prije zrcaljenja uèitati kolaèiæe iz datoteke Netscape cookies.txt.
Pause between files:
Stanka izmeðu datoteka:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Nasumièna odgoda izmeðu preuzimanja datoteka, u sekundama. Za nasumièni raspon koristiti MIN:MAX (npr. 2:8).
Keep the www. prefix (do not merge www.host with host)
Zadr¾ati www. predmetak (ne spajati www.host s host)
Do not treat www.host and host as the same site.
www.host i host ne smatrati istim web-mjestom.
Keep double slashes in URLs
Zadr¾ati dvostruke kose crte u URL-ovima
Do not collapse duplicate slashes in URLs.
Ne sa¾imati ponovljene kose crte u URL-ovima.
Keep the original query-string order
Zadr¾ati izvorni redoslijed teksta za upite
Do not reorder query-string parameters when deduplicating URLs.
Ne mijenjati redoslijed parametara teksta za upite pri uklanjanju dvostrukih URL-ova.
Strip query keys:
Ukloniti kljuèeve upita:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Zarezom odvojeni kljuèevi upita koji se izostavljaju iz naziva spremljenih datoteka (npr. sid,utm_source).

View File

@@ -976,3 +976,31 @@ Click on this notification to restart the interrupted mirror
Klik på denne notifikation for at genstarte den afbrudte spejlkopiering
HTTrack: could not save profile for '%s'!
HTTrack: kunne ikke gemme profil for '%s'!
Proxy type:
Proxytype:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Proxyprotokol: HTTP eller SOCKS5 (SOCKS5 bruger som standard port 1080).
Load cookies from file:
Indlæs cookies fra fil:
Preload cookies from a Netscape cookies.txt file before crawling.
Indlæs cookies fra en Netscape cookies.txt-fil, før crawlingen starter.
Pause between files:
Pause mellem filer:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Tilfældig forsinkelse mellem filoverførsler, i sekunder. Brug MIN:MAX for et tilfældigt interval (f.eks. 2:8).
Keep the www. prefix (do not merge www.host with host)
Behold www.-præfikset (sammenlæg ikke www.host med host)
Do not treat www.host and host as the same site.
Behandl ikke www.host og host som det samme websted.
Keep double slashes in URLs
Behold dobbelte skråstreger i URL'er
Do not collapse duplicate slashes in URLs.
Sammenfold ikke gentagne skråstreger i URL'er.
Keep the original query-string order
Behold den oprindelige rækkefølge i forespørgselsstrengen
Do not reorder query-string parameters when deduplicating URLs.
Omorden ikke forespørgselsstrengens parametre, når dublerede URL'er fjernes.
Strip query keys:
Fjern forespørgselsnøgler:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommaseparerede forespørgselsnøgler, der udelades i navngivningen af gemte filer (f.eks. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
Der Server wurde beendet
A fatal error has occurred during this mirror
Fataler Fehler während der Webseiten-Kopie
Proxy type:
Proxy-Typ:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Proxy-Protokoll: HTTP oder SOCKS5 (Standardport von SOCKS5 ist 1080).
Load cookies from file:
Cookies aus Datei laden:
Preload cookies from a Netscape cookies.txt file before crawling.
Cookies aus einer Netscape-cookies.txt-Datei laden, bevor das Crawlen beginnt.
Pause between files:
Pause zwischen Dateien:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Zufällige Verzögerung zwischen Datei-Downloads, in Sekunden. MIN:MAX für einen Zufallsbereich verwenden (z. B. 2:8).
Keep the www. prefix (do not merge www.host with host)
www.-Präfix beibehalten (www.host nicht mit host zusammenführen)
Do not treat www.host and host as the same site.
www.host und host nicht als dieselbe Website behandeln.
Keep double slashes in URLs
Doppelte Schrägstriche in URLs beibehalten
Do not collapse duplicate slashes in URLs.
Doppelte Schrägstriche in URLs nicht zusammenführen.
Keep the original query-string order
Ursprüngliche Reihenfolge des Query-Strings beibehalten
Do not reorder query-string parameters when deduplicating URLs.
Query-String-Parameter beim Entfernen doppelter URLs nicht neu anordnen.
Strip query keys:
Query-Schlüssel entfernen:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommagetrennte Query-Schlüssel, die bei der Benennung gespeicherter Dateien entfallen (z. B. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
A fatal error has occurred during this mirror
Proxy type:
Proxy tüüp:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Proxy protokoll: HTTP või SOCKS5 (SOCKS5 vaikeport on 1080).
Load cookies from file:
Lae küpsised failist:
Preload cookies from a Netscape cookies.txt file before crawling.
Lae küpsised eelnevalt Netscape cookies.txt failist enne kopeerimist.
Pause between files:
Paus failide vahel:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Juhuslik viivitus failide allalaadimiste vahel, sekundites. Juhusliku vahemiku jaoks kasuta MIN:MAX (nt 2:8).
Keep the www. prefix (do not merge www.host with host)
Säilita www. eesliide (ära ühenda www.host ja host)
Do not treat www.host and host as the same site.
Ära käsitle www.host ja host sama saidina.
Keep double slashes in URLs
Säilita topeltkaldkriipsud URL-ides
Do not collapse duplicate slashes in URLs.
Ära ühenda korduvaid kaldkriipse URL-ides.
Keep the original query-string order
Säilita päringustringi algne järjekord
Do not reorder query-string parameters when deduplicating URLs.
Ära järjesta päringustringi parameetreid ümber URL-ide korduste eemaldamisel.
Strip query keys:
Eemalda päringuvõtmed:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Komadega eraldatud päringuvõtmed, mis jäetakse salvestatud faili nimest välja (nt sid,utm_source).

View File

@@ -976,3 +976,31 @@ Click on this notification to restart the interrupted mirror
Click on this notification to restart the interrupted mirror
HTTrack: could not save profile for '%s'!
HTTrack: could not save profile for '%s'!
Proxy type:
Proxy type:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Load cookies from file:
Load cookies from file:
Preload cookies from a Netscape cookies.txt file before crawling.
Preload cookies from a Netscape cookies.txt file before crawling.
Pause between files:
Pause between files:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Keep the www. prefix (do not merge www.host with host)
Keep the www. prefix (do not merge www.host with host)
Do not treat www.host and host as the same site.
Do not treat www.host and host as the same site.
Keep double slashes in URLs
Keep double slashes in URLs
Do not collapse duplicate slashes in URLs.
Do not collapse duplicate slashes in URLs.
Keep the original query-string order
Keep the original query-string order
Do not reorder query-string parameters when deduplicating URLs.
Do not reorder query-string parameters when deduplicating URLs.
Strip query keys:
Strip query keys:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).

View File

@@ -931,3 +931,31 @@ Palvelin lopetettu
A fatal error has occurred during this mirror
Tällä peilillä tapahtui vakava virhe
Proxy type:
Välityspalvelimen tyyppi:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Välityspalvelimen protokolla: HTTP tai SOCKS5 (SOCKS5:n oletusportti on 1080).
Load cookies from file:
Lataa evästeet tiedostosta:
Preload cookies from a Netscape cookies.txt file before crawling.
Lataa evästeet Netscape cookies.txt -tiedostosta ennen sivuston läpikäyntiä.
Pause between files:
Tauko tiedostojen välillä:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Satunnainen viive tiedostolatausten välillä, sekunteina. Käytä muotoa MIN:MAX satunnaista väliä varten (esim. 2:8).
Keep the www. prefix (do not merge www.host with host)
Säilytä www.-etuliite (älä yhdistä www.host- ja host-osoitteita)
Do not treat www.host and host as the same site.
Älä käsittele www.host- ja host-osoitteita samana sivustona.
Keep double slashes in URLs
Säilytä kaksoiskenoviivat URL-osoitteissa
Do not collapse duplicate slashes in URLs.
Älä yhdistä toistuvia kenoviivoja URL-osoitteissa.
Keep the original query-string order
Säilytä kyselymerkkijonon alkuperäinen järjestys
Do not reorder query-string parameters when deduplicating URLs.
Älä järjestä kyselymerkkijonon parametreja uudelleen, kun URL-osoitteiden kaksoiskappaleet poistetaan.
Strip query keys:
Poista kyselyavaimet:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Pilkuin erotellut kyselyavaimet, jotka jätetään pois tallennettujen tiedostojen nimeämisestä (esim. sid,utm_source).

View File

@@ -976,3 +976,31 @@ Build a mail archive
Construire une archive mail
Default referer URL
Champ referer par défaut
Proxy type:
Type de proxy :
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Protocole du proxy : HTTP ou SOCKS5 (le port SOCKS5 par défaut est 1080).
Load cookies from file:
Charger les cookies depuis un fichier :
Preload cookies from a Netscape cookies.txt file before crawling.
Précharger les cookies depuis un fichier Netscape cookies.txt avant la copie du site.
Pause between files:
Pause entre les fichiers :
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Délai aléatoire entre les téléchargements de fichiers, en secondes. Utilisez MIN:MAX pour une plage aléatoire (par ex. 2:8).
Keep the www. prefix (do not merge www.host with host)
Conserver le préfixe www. (ne pas fusionner www.host avec host)
Do not treat www.host and host as the same site.
Ne pas traiter www.host et host comme le même site.
Keep double slashes in URLs
Conserver les doubles barres obliques dans les URL
Do not collapse duplicate slashes in URLs.
Ne pas réduire les barres obliques en double dans les URL.
Keep the original query-string order
Conserver l'ordre d'origine de la query string
Do not reorder query-string parameters when deduplicating URLs.
Ne pas réordonner les paramètres de la query string lors de la déduplication des URL.
Strip query keys:
Supprimer les clés de query string :
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Clés de query string à retirer du nommage des fichiers enregistrés, séparées par des virgules (par ex. sid,utm_source).

View File

@@ -930,3 +930,31 @@ Server terminated
Ακυρώθηκε από τον εξυπηρετητή
A fatal error has occurred during this mirror
Ένα καταστροφικό σφάλμα προκλήθηκε κατά την αντιγραφή αυτού του τόπου
Proxy type:
Τύπος proxy:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Πρωτόκολλο proxy: HTTP ή SOCKS5 (η προεπιλεγμένη θύρα του SOCKS5 είναι 1080).
Load cookies from file:
Φόρτωση cookies από αρχείο:
Preload cookies from a Netscape cookies.txt file before crawling.
Προφόρτωση των cookies από ένα αρχείο Netscape cookies.txt πριν την ανίχνευση.
Pause between files:
Παύση μεταξύ αρχείων:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Τυχαία καθυστέρηση μεταξύ των λήψεων αρχείων, σε δευτερόλεπτα. Χρησιμοποιήστε MIN:MAX για τυχαίο εύρος (π.χ. 2:8).
Keep the www. prefix (do not merge www.host with host)
Διατήρηση του προθέματος www. (χωρίς συγχώνευση του www.host με το host)
Do not treat www.host and host as the same site.
Να μην θεωρούνται τα www.host και host ως ο ίδιος ιστότοπος.
Keep double slashes in URLs
Διατήρηση των διπλών καθέτων στα URL
Do not collapse duplicate slashes in URLs.
Να μην συμπτύσσονται οι διπλές κάθετοι στα URL.
Keep the original query-string order
Διατήρηση της αρχικής σειράς του ερωτήματος αναζήτησης
Do not reorder query-string parameters when deduplicating URLs.
Να μην αναδιατάσσονται οι παράμετροι του ερωτήματος αναζήτησης κατά την αφαίρεση διπλότυπων URL.
Strip query keys:
Αφαίρεση κλειδιών ερωτήματος:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Κλειδιά ερωτήματος χωρισμένα με κόμμα, που θα αφαιρεθούν από την ονομασία των αποθηκευμένων αρχείων (π.χ. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
Server disconnesso
A fatal error has occurred during this mirror
Si è verificato un errore fatale durante la copia
Proxy type:
Tipo di proxy:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Protocollo del proxy: HTTP o SOCKS5 (la porta SOCKS5 predefinita è 1080).
Load cookies from file:
Carica i cookie da un file:
Preload cookies from a Netscape cookies.txt file before crawling.
Precarica i cookie da un file Netscape cookies.txt prima della copia.
Pause between files:
Pausa tra i file:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Ritardo casuale tra i download dei file, in secondi. Usa MIN:MAX per un intervallo casuale (ad es. 2:8).
Keep the www. prefix (do not merge www.host with host)
Mantieni il prefisso www. (non unire www.host con host)
Do not treat www.host and host as the same site.
Non trattare www.host e host come lo stesso sito.
Keep double slashes in URLs
Mantieni le doppie barre negli URL
Do not collapse duplicate slashes in URLs.
Non ridurre le barre duplicate negli URL.
Keep the original query-string order
Mantieni l'ordine originale della query string
Do not reorder query-string parameters when deduplicating URLs.
Non riordinare i parametri della query string durante la deduplicazione degli URL.
Strip query keys:
Rimuovi chiavi della query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Chiavi della query string, separate da virgole, da rimuovere dai nomi dei file salvati (ad es. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
A fatal error has occurred during this mirror
Proxy type:
プロキシの種類:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
プロキシのプロトコル: HTTP または SOCKS5 (SOCKS5 の既定ポートは 1080)。
Load cookies from file:
クッキーをファイルから読み込む:
Preload cookies from a Netscape cookies.txt file before crawling.
クロールを開始する前に Netscape の cookies.txt ファイルからクッキーを読み込みます。
Pause between files:
ファイル間の待機:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
ファイルのダウンロード間のランダムな遅延(秒)。範囲を指定するには MIN:MAX を使用します (例: 2:8)。
Keep the www. prefix (do not merge www.host with host)
www. プレフィックスを保持する (www.host と host を統合しない)
Do not treat www.host and host as the same site.
www.host と host を同じサイトとして扱いません。
Keep double slashes in URLs
URL 内の二重スラッシュを保持する
Do not collapse duplicate slashes in URLs.
URL 内の連続したスラッシュをまとめません。
Keep the original query-string order
クエリ文字列の元の順序を保持する
Do not reorder query-string parameters when deduplicating URLs.
URL の重複排除時にクエリ文字列のパラメータを並べ替えません。
Strip query keys:
削除するクエリキー:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
保存ファイル名の生成から除外するクエリキーをカンマ区切りで指定します (例: sid,utm_source)。

View File

@@ -928,3 +928,31 @@ Server terminated
Ñåðâåðîò å ïðåêèíàò
A fatal error has occurred during this mirror
Íàñòàíà ôàòàëíà ãðåøêà ïðè îâî¼ mirror
Proxy type:
ÂØß ÝÐ ßàÞÚáØ:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
¿àÞâÞÚÞÛ ÝÐ ßàÞÚáØ: HTTP ØÛØ SOCKS5 (áâÐÝÔÐàÔÝÐâÐ ßÞàâÐ ÝÐ SOCKS5 Õ 1080).
Load cookies from file:
²çØâÐø ÚÞÛÐçØúÐ ÞÔ ÔÐâÞâÕÚÐ:
Preload cookies from a Netscape cookies.txt file before crawling.
¾ÔÝÐßàÕÔ ÒçØâÐø ÚÞÛÐçØúÐ ÞÔ ÔÐâÞâÕÚÐ Netscape cookies.txt ßàÕÔ ßàÕ×ÕÜÐúÕâÞ.
Pause between files:
¿Ðã×Ð ÜÕóã ÔÐâÞâÕÚØ:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
ÁÛãçÐøÝÞ ÔÞæÝÕúÕ ÜÕóã ßàÕ×ÕÜÐúÐâÐ ÝÐ ÔÐâÞâÕÚØ, ÒÞ áÕÚãÝÔØ. ºÞàØáâØ MIN:MAX ×Ð áÛãçÐÕÝ ÞßáÕÓ (ÝÐ ßàØÜÕà 2:8).
Keep the www. prefix (do not merge www.host with host)
·ÐÔàÖØ ÓÞ ßàÕäØÚáÞâ www. (ÝÕ áßÞøãÒÐø www.host áÞ host)
Do not treat www.host and host as the same site.
½Õ âàÕâØàÐø ÓØ www.host Ø host ÚÐÚÞ Øáâ áÐøâ.
Keep double slashes in URLs
·ÐÔàÖØ ÓØ ÔÒÞøÝØâÕ ÚÞáØ æàâØ ÒÞ URL
Do not collapse duplicate slashes in URLs.
½Õ ÞâáâàÐÝãÒÐø ÓØ ßÞÒâÞàÕÝØâÕ ÚÞáØ æàâØ ÒÞ URL.
Keep the original query-string order
·ÐÔàÖØ ÓÞ Ø×ÒÞàÝØÞâ àÕÔÞáÛÕÔ ÝÐ ßÐàÐÜÕâàØâÕ ÒÞ ÑÐàÐúÕâÞ
Do not reorder query-string parameters when deduplicating URLs.
½Õ ÜÕÝãÒÐø ÓÞ àÕÔÞáÛÕÔÞâ ÝÐ ßÐàÐÜÕâàØâÕ ÝÐ ÑÐàÐúÕâÞ ßàØ ÞâáâàÐÝãÒÐúÕ ÔãßÛØÚÐâØ URL.
Strip query keys:
¾âáâàÐÝØ ÚÛãçÕÒØ ÞÔ ÑÐàÐúÕâÞ:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
ºÛãçÕÒØ ÞÔ ÑÐàÐúÕâÞ, ÞÔÔÕÛÕÝØ áÞ ×ÐߨàÚÐ, èâÞ áÕ ÞâáâàÐÝãÒÐÐâ ÞÔ ØÜÕâÞ ÝÐ ×ÐçãÒÐÝÐâÐ ÔÐâÞâÕÚÐ (ÝÐ ßàØÜÕà sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
A kiszolgáló befejezte a kapcsolatot
A fatal error has occurred during this mirror
Végzetes hiba történt a tükrözés közben
Proxy type:
Proxy típusa:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Proxy protokoll: HTTP vagy SOCKS5 (a SOCKS5 alapértelmezett portja 1080).
Load cookies from file:
Sütik betöltése fájlból:
Preload cookies from a Netscape cookies.txt file before crawling.
Sütik elõzetes betöltése egy Netscape cookies.txt fájlból a tükrözés elõtt.
Pause between files:
Szünet a fájlok között:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Véletlenszerû késleltetés a fájlletöltések között, másodpercben. Véletlen tartományhoz használja a MIN:MAX formát (pl. 2:8).
Keep the www. prefix (do not merge www.host with host)
A www. elõtag megtartása (ne vonja össze a www.host és a host címet)
Do not treat www.host and host as the same site.
Ne kezelje a www.host és a host címet ugyanazon webhelyként.
Keep double slashes in URLs
Dupla perjelek megtartása az URL-ekben
Do not collapse duplicate slashes in URLs.
Ne vonja össze az ismétlõdõ perjeleket az URL-ekben.
Keep the original query-string order
Az eredeti lekérdezési szöveg sorrendjének megtartása
Do not reorder query-string parameters when deduplicating URLs.
Ne rendezze át a lekérdezési szöveg paramétereit az ismétlõdõ URL-ek kiszûrésekor.
Strip query keys:
Lekérdezési kulcsok eltávolítása:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Vesszõvel elválasztott lekérdezési kulcsok, amelyeket el kell hagyni a mentett fájl elnevezésébõl (pl. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
Server beeindigd
A fatal error has occurred during this mirror
Een fatale fout is opgetreden tijdens deze spiegeling
Proxy type:
Proxytype:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Proxyprotocol: HTTP of SOCKS5 (standaardpoort van SOCKS5 is 1080).
Load cookies from file:
Cookies uit bestand laden:
Preload cookies from a Netscape cookies.txt file before crawling.
Cookies vooraf laden uit een Netscape cookies.txt-bestand voordat het crawlen begint.
Pause between files:
Pauze tussen bestanden:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Willekeurige vertraging tussen bestandsdownloads, in seconden. Gebruik MIN:MAX voor een willekeurig bereik (bijv. 2:8).
Keep the www. prefix (do not merge www.host with host)
www.-voorvoegsel behouden (www.host niet samenvoegen met host)
Do not treat www.host and host as the same site.
Behandel www.host en host niet als dezelfde site.
Keep double slashes in URLs
Dubbele schuine strepen in URL's behouden
Do not collapse duplicate slashes in URLs.
Dubbele schuine strepen in URL's niet samenvoegen.
Keep the original query-string order
Oorspronkelijke volgorde van de query string behouden
Do not reorder query-string parameters when deduplicating URLs.
Query string-parameters niet herordenen bij het ontdubbelen van URL's.
Strip query keys:
Query-sleutels verwijderen:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Door komma's gescheiden query-sleutels die bij het benoemen van opgeslagen bestanden worden weggelaten (bijv. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
A fatal error has occurred during this mirror
Proxy type:
Proxytype:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Proxyprotokoll: HTTP eller SOCKS5 (SOCKS5 bruker som standard port 1080).
Load cookies from file:
Last inn cookies fra fil:
Preload cookies from a Netscape cookies.txt file before crawling.
Last inn cookies fra en Netscape cookies.txt-fil før crawlingen starter.
Pause between files:
Pause mellom filer:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Tilfeldig forsinkelse mellom filnedlastinger, i sekunder. Bruk MIN:MAX for et tilfeldig intervall (f.eks. 2:8).
Keep the www. prefix (do not merge www.host with host)
Behold www.-prefikset (slå ikke sammen www.host med host)
Do not treat www.host and host as the same site.
Behandle ikke www.host og host som samme nettsted.
Keep double slashes in URLs
Behold doble skråstreker i URL-er
Do not collapse duplicate slashes in URLs.
Slå ikke sammen gjentatte skråstreker i URL-er.
Keep the original query-string order
Behold den opprinnelige rekkefølgen i spørrestrengen
Do not reorder query-string parameters when deduplicating URLs.
Endre ikke rekkefølgen på spørrestrengens parametere når dupliserte URL-er fjernes.
Strip query keys:
Fjern spørrenøkler:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommaseparerte spørrenøkler som utelates i navngivingen av lagrede filer (f.eks. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
Serwer zakonczyl prace
A fatal error has occurred during this mirror
Podczas tworzenia lustra wydarzyl sie fatalny blad.
Proxy type:
Typ proxy:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Protokó³ proxy: HTTP lub SOCKS5 (domy¶lny port SOCKS5 to 1080).
Load cookies from file:
Wczytaj ciasteczka z pliku:
Preload cookies from a Netscape cookies.txt file before crawling.
Wczytaj ciasteczka z pliku Netscape cookies.txt przed rozpoczêciem pobierania.
Pause between files:
Pauza miêdzy plikami:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Losowe opó¼nienie miêdzy pobieraniem plików, w sekundach. U¿yj MIN:MAX dla losowego zakresu (np. 2:8).
Keep the www. prefix (do not merge www.host with host)
Zachowaj przedrostek www. (nie ³±cz www.host z host)
Do not treat www.host and host as the same site.
Nie traktuj www.host i host jako tej samej witryny.
Keep double slashes in URLs
Zachowaj podwójne uko¶niki w adresach URL
Do not collapse duplicate slashes in URLs.
Nie scalaj powtórzonych uko¶ników w adresach URL.
Keep the original query-string order
Zachowaj oryginaln± kolejno¶æ ci±gu zapytania
Do not reorder query-string parameters when deduplicating URLs.
Nie zmieniaj kolejno¶ci parametrów zapytania podczas usuwania duplikatów adresów URL.
Strip query keys:
Usuñ klucze zapytania:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Rozdzielone przecinkami klucze zapytania pomijane przy nazywaniu zapisanych plików (np. sid,utm_source).

View File

@@ -976,3 +976,31 @@ Click on this notification to restart the interrupted mirror
Clique nesta notificação para reiniciar o espelho interrompido
HTTrack: could not save profile for '%s'!
HTTrack: não foi possível salvar o perfil para '%s'!
Proxy type:
Tipo de proxy:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Protocolo do proxy: HTTP ou SOCKS5 (a porta SOCKS5 padrão é 1080).
Load cookies from file:
Carregar cookies de um arquivo:
Preload cookies from a Netscape cookies.txt file before crawling.
Pré-carregar cookies de um arquivo Netscape cookies.txt antes de iniciar a cópia.
Pause between files:
Pausa entre arquivos:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Atraso aleatório entre downloads de arquivos, em segundos. Use MIN:MAX para um intervalo aleatório (ex.: 2:8).
Keep the www. prefix (do not merge www.host with host)
Manter o prefixo www. (não mesclar www.host com host)
Do not treat www.host and host as the same site.
Não tratar www.host e host como o mesmo site.
Keep double slashes in URLs
Manter as barras duplas nas URLs
Do not collapse duplicate slashes in URLs.
Não reduzir as barras duplicadas nas URLs.
Keep the original query-string order
Manter a ordem original da query string
Do not reorder query-string parameters when deduplicating URLs.
Não reordenar os parâmetros da query string ao remover URLs duplicadas.
Strip query keys:
Remover chaves da query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Chaves da query string, separadas por vírgulas, a serem removidas da nomeação dos arquivos salvos (ex.: sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
A fatal error has occurred during this mirror
Proxy type:
Tipo de proxy:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Protocolo do proxy: HTTP ou SOCKS5 (a porta SOCKS5 predefinida é 1080).
Load cookies from file:
Carregar cookies de um ficheiro:
Preload cookies from a Netscape cookies.txt file before crawling.
Pré-carregar cookies de um ficheiro Netscape cookies.txt antes de iniciar a cópia.
Pause between files:
Pausa entre ficheiros:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Atraso aleatório entre transferências de ficheiros, em segundos. Utilize MIN:MAX para um intervalo aleatório (por ex. 2:8).
Keep the www. prefix (do not merge www.host with host)
Manter o prefixo www. (não combinar www.host com host)
Do not treat www.host and host as the same site.
Não tratar www.host e host como o mesmo site.
Keep double slashes in URLs
Manter as barras duplas nos URL
Do not collapse duplicate slashes in URLs.
Não reduzir as barras duplicadas nos URL.
Keep the original query-string order
Manter a ordem original da query string
Do not reorder query-string parameters when deduplicating URLs.
Não reordenar os parâmetros da query string ao eliminar URL duplicados.
Strip query keys:
Remover chaves da query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Chaves da query string, separadas por vírgulas, a remover da nomeação dos ficheiros guardados (por ex. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
Server terminat
A fatal error has occurred during this mirror
A survenit o eroare fatală în timpul acestei clonări.
Proxy type:
Tip proxy:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Protocol proxy: HTTP sau SOCKS5 (portul SOCKS5 implicit este 1080).
Load cookies from file:
Încarca cookies dintr-un fisier:
Preload cookies from a Netscape cookies.txt file before crawling.
Preîncarca cookies dintr-un fisier Netscape cookies.txt înainte de copiere.
Pause between files:
Pauza între fisiere:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Întârziere aleatoare între descarcari de fisiere, în secunde. Folositi MIN:MAX pentru un interval aleatoriu (de ex. 2:8).
Keep the www. prefix (do not merge www.host with host)
Pastreaza prefixul www. (nu combina www.host cu host)
Do not treat www.host and host as the same site.
Nu trata www.host si host ca fiind acelasi site.
Keep double slashes in URLs
Pastreaza barele duble din URL-uri
Do not collapse duplicate slashes in URLs.
Nu comprima barele duplicate din URL-uri.
Keep the original query-string order
Pastreaza ordinea originala din query string
Do not reorder query-string parameters when deduplicating URLs.
Nu reordona parametrii din query string la eliminarea URL-urilor duplicate.
Strip query keys:
Elimina cheile din query string:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Chei din query string, separate prin virgula, de eliminat din denumirea fisierelor salvate (de ex. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
Server terminated
A fatal error has occurred during this mirror
Âî âðåìÿ òåêóùåé çàêà÷êè ïðîèçîøëà ôàòàëüíàÿ îøèáêà
Proxy type:
Òèï ïðîêñè:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Ïðîòîêîë ïðîêñè: HTTP èëè SOCKS5 (ïî óìîë÷àíèþ SOCKS5 èñïîëüçóåò ïîðò 1080).
Load cookies from file:
Çàãðóçèòü cookies èç ôàéëà:
Preload cookies from a Netscape cookies.txt file before crawling.
Ïðåäâàðèòåëüíî çàãðóçèòü cookies èç ôàéëà Netscape cookies.txt ïåðåä çàêà÷êîé.
Pause between files:
Ïàóçà ìåæäó ôàéëàìè:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Ñëó÷àéíàÿ çàäåðæêà ìåæäó çàêà÷êàìè ôàéëîâ, â ñåêóíäàõ. Èñïîëüçóéòå MIN:MAX äëÿ ñëó÷àéíîãî äèàïàçîíà (íàïðèìåð, 2:8).
Keep the www. prefix (do not merge www.host with host)
Ñîõðàíÿòü ïðåôèêñ www. (íå îáúåäèíÿòü www.host ñ host)
Do not treat www.host and host as the same site.
Íå ñ÷èòàòü www.host è host îäíèì è òåì æå ñàéòîì.
Keep double slashes in URLs
Ñîõðàíÿòü äâîéíûå ñëýøè â URL
Do not collapse duplicate slashes in URLs.
Íå óäàëÿòü ïîâòîðÿþùèåñÿ ñëýøè â URL.
Keep the original query-string order
Ñîõðàíÿòü èñõîäíûé ïîðÿäîê ïàðàìåòðîâ çàïðîñà
Do not reorder query-string parameters when deduplicating URLs.
Íå ìåíÿòü ïîðÿäîê ïàðàìåòðîâ ñòðîêè çàïðîñà ïðè óäàëåíèè äóáëèêàòîâ URL.
Strip query keys:
Óäàëÿòü êëþ÷è çàïðîñà:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Êëþ÷è çàïðîñà ÷åðåç çàïÿòóþ, óäàëÿåìûå èç èìåíè ñîõðàíÿåìîãî ôàéëà (íàïðèìåð, sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
A fatal error has occurred during this mirror
Proxy type:
Typ proxy:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Protokol proxy: HTTP alebo SOCKS5 (predvolený port SOCKS5 je 1080).
Load cookies from file:
Naèíta» cookies zo súboru:
Preload cookies from a Netscape cookies.txt file before crawling.
Naèíta» cookies zo súboru Netscape cookies.txt pred zaèatím s»ahovania.
Pause between files:
Prestávka medzi súbormi:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Náhodné oneskorenie medzi s»ahovaním súborov, v sekundách. Pre náhodný rozsah pou¾ite MIN:MAX (napr. 2:8).
Keep the www. prefix (do not merge www.host with host)
Zachova» predponu www. (nezluèova» www.host s host)
Do not treat www.host and host as the same site.
Nepova¾ova» www.host a host za tú istú lokalitu.
Keep double slashes in URLs
Zachova» dvojité lomky v URL
Do not collapse duplicate slashes in URLs.
Nezluèova» opakované lomky v URL.
Keep the original query-string order
Zachova» pôvodné poradie re»azca dotazu
Do not reorder query-string parameters when deduplicating URLs.
Nemeni» poradie parametrov re»azca dotazu pri odstraòovaní duplicitných URL.
Strip query keys:
Odstráni» kµúèe dotazu:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kµúèe dotazu oddelené èiarkami, ktoré sa vynechajú z pomenovania ukladaných súborov (napr. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
A fatal error has occurred during this mirror
Proxy type:
Vrsta proxyja:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Protokol proxy: HTTP ali SOCKS5 (privzeta vrata SOCKS5 so 1080).
Load cookies from file:
Nalozi piskotke iz datoteke:
Preload cookies from a Netscape cookies.txt file before crawling.
Predhodno nalozi piskotke iz datoteke Netscape cookies.txt pred zajemanjem.
Pause between files:
Premor med datotekami:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Nakljucni zamik med prenosi datotek, v sekundah. Za nakljucni obseg uporabite MIN:MAX (npr. 2:8).
Keep the www. prefix (do not merge www.host with host)
Ohrani predpono www. (ne zdruzi www.host z host)
Do not treat www.host and host as the same site.
www.host in host ne obravnavaj kot isto spletno mesto.
Keep double slashes in URLs
Ohrani dvojne posevnice v URL-jih
Do not collapse duplicate slashes in URLs.
Ne zdruzuj podvojenih posevnic v URL-jih.
Keep the original query-string order
Ohrani izvirni vrstni red poizvedovalnega niza
Do not reorder query-string parameters when deduplicating URLs.
Pri odstranjevanju podvojenih URL-jev ne spreminjaj vrstnega reda parametrov poizvedbe.
Strip query keys:
Odstrani kljuce poizvedbe:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Z vejicami loceni kljuci poizvedbe, ki se izpustijo pri poimenovanju shranjenih datotek (npr. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
A fatal error has occurred during this mirror
Proxy type:
Proxytyp:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Proxyprotokoll: HTTP eller SOCKS5 (SOCKS5 använder som standard port 1080).
Load cookies from file:
Läs in cookies från fil:
Preload cookies from a Netscape cookies.txt file before crawling.
Läs in cookies från en Netscape cookies.txt-fil innan crawlningen börjar.
Pause between files:
Paus mellan filer:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Slumpmässig fördröjning mellan filnedladdningar, i sekunder. Använd MIN:MAX för ett slumpmässigt intervall (t.ex. 2:8).
Keep the www. prefix (do not merge www.host with host)
Behåll www.-prefixet (slå inte ihop www.host med host)
Do not treat www.host and host as the same site.
Behandla inte www.host och host som samma webbplats.
Keep double slashes in URLs
Behåll dubbla snedstreck i URL:er
Do not collapse duplicate slashes in URLs.
Slå inte ihop upprepade snedstreck i URL:er.
Keep the original query-string order
Behåll frågesträngens ursprungliga ordning
Do not reorder query-string parameters when deduplicating URLs.
Ändra inte ordningen på frågesträngens parametrar när dubbletter av URL:er tas bort.
Strip query keys:
Ta bort frågenycklar:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kommaseparerade frågenycklar som utelämnas vid namngivningen av sparade filer (t.ex. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
Sunucu sonlandýrdý
A fatal error has occurred during this mirror
Bu yansýlama iþlemi sýrasýnda ölümcül bir hata oluþtu
Proxy type:
Proxy türü:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Proxy protokolü: HTTP veya SOCKS5 (SOCKS5 varsayýlan portu 1080).
Load cookies from file:
Çerezleri dosyadan yükle:
Preload cookies from a Netscape cookies.txt file before crawling.
Taramadan önce bir Netscape cookies.txt dosyasýndan çerezleri önceden yükle.
Pause between files:
Dosyalar arasýnda bekleme:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Dosya indirmeleri arasýnda saniye cinsinden rastgele gecikme. Rastgele bir aralýk için MIN:MAX kullanýn (örn. 2:8).
Keep the www. prefix (do not merge www.host with host)
www. önekini koru (www.host ile host'u birleþtirme)
Do not treat www.host and host as the same site.
www.host ve host'u ayný site olarak deðerlendirme.
Keep double slashes in URLs
URL'lerdeki çift eðik çizgileri koru
Do not collapse duplicate slashes in URLs.
URL'lerdeki yinelenen eðik çizgileri birleþtirme.
Keep the original query-string order
Özgün sorgu dizesi sýrasýný koru
Do not reorder query-string parameters when deduplicating URLs.
URL'lerin yinelenenlerini ayýklarken sorgu dizesi parametrelerini yeniden sýralama.
Strip query keys:
Sorgu anahtarlarýný çýkar:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Kaydedilen dosya adlandýrmasýndan çýkarýlacak, virgülle ayrýlmýþ sorgu anahtarlarý (örn. sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
A fatal error has occurred during this mirror
Proxy type:
Òèï ïðîêñ³:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Ïðîòîêîë ïðîêñ³: HTTP àáî SOCKS5 (çà çàìîâ÷óâàííÿì SOCKS5 âèêîðèñòîâóº ïîðò 1080).
Load cookies from file:
Çàâàíòàæèòè cookies ç ôàéëó:
Preload cookies from a Netscape cookies.txt file before crawling.
Ïîïåðåäíüî çàâàíòàæèòè cookies ç ôàéëó Netscape cookies.txt ïåðåä çàâàíòàæåííÿì.
Pause between files:
Ïàóçà ì³æ ôàéëàìè:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Âèïàäêîâà çàòðèìêà ì³æ çàâàíòàæåííÿìè ôàéë³â, ó ñåêóíäàõ. Âèêîðèñòîâóéòå MIN:MAX äëÿ âèïàäêîâîãî ä³àïàçîíó (íàïðèêëàä, 2:8).
Keep the www. prefix (do not merge www.host with host)
Çáåð³ãàòè ïðåô³êñ www. (íå îá'ºäíóâàòè www.host ç host)
Do not treat www.host and host as the same site.
Íå ââàæàòè www.host ³ host îäíèì ³ òèì ñàìèì ñàéòîì.
Keep double slashes in URLs
Çáåð³ãàòè ïîäâ³éí³ ñëåø³ â URL
Do not collapse duplicate slashes in URLs.
Íå âèäàëÿòè ïîâòîðþâàí³ ñëåø³ â URL.
Keep the original query-string order
Çáåð³ãàòè ïî÷àòêîâèé ïîðÿäîê ïàðàìåòð³â çàïèòó
Do not reorder query-string parameters when deduplicating URLs.
Íå çì³íþâàòè ïîðÿäîê ïàðàìåòð³â ðÿäêà çàïèòó ï³ä ÷àñ óñóíåííÿ äóáë³êàò³â URL.
Strip query keys:
Âèëó÷àòè êëþ÷³ çàïèòó:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Êëþ÷³ çàïèòó ÷åðåç êîìó, ÿê³ âèëó÷àþòüñÿ ç ³ìåí³ çáåðåæåíîãî ôàéëó (íàïðèêëàä, sid,utm_source).

View File

@@ -928,3 +928,31 @@ Server terminated
Server terminated
A fatal error has occurred during this mirror
Joriy kochirish vaqtida jiddiy xatolik yuz berdi
Proxy type:
Proksi turi:
Proxy protocol: HTTP, or SOCKS5 (SOCKS5 default port is 1080).
Proksi protokoli: HTTP yoki SOCKS5 (SOCKS5 ning standart porti 1080).
Load cookies from file:
Fayldan cookies yuklamoq:
Preload cookies from a Netscape cookies.txt file before crawling.
Kochirib olishdan oldin Netscape cookies.txt faylidan cookiesni oldindan yuklamoq.
Pause between files:
Fayllar orasidagi pauza:
Random delay between file downloads, in seconds. Use MIN:MAX for a random range (e.g. 2:8).
Fayllarni kochirib olish orasidagi tasodifiy kechikish, soniyalarda. Tasodifiy diapazon uchun MIN:MAX dan foydalaning (masalan, 2:8).
Keep the www. prefix (do not merge www.host with host)
www. prefiksini saqlash (www.host ni host bilan birlashtirmaslik)
Do not treat www.host and host as the same site.
www.host va host ni bir xil sayt deb hisoblamaslik.
Keep double slashes in URLs
URL lardagi qosh sleshlarni saqlash
Do not collapse duplicate slashes in URLs.
URL lardagi takroriy sleshlarni olib tashlamaslik.
Keep the original query-string order
Sorov satrining asl tartibini saqlash
Do not reorder query-string parameters when deduplicating URLs.
URL dublikatlarini olib tashlashda sorov satri parametrlari tartibini ozgartirmaslik.
Strip query keys:
Olib tashlanadigan sorov kalitlari:
Comma-separated query keys to drop from the saved-file naming (e.g. sid,utm_source).
Saqlangan fayl nomidan olib tashlanadigan, vergul bilan ajratilgan sorov kalitlari (masalan, sid,utm_source).

View File

@@ -628,6 +628,10 @@ int back_finalize(httrackp * opt, cache_back * cache, struct_back * sback,
back[p].r.size, back[p].url_adr, back[p].url_fil);
}
back_finalize_backup(opt, &back[p], HTS_FALSE);
/* Keep the surviving copy in new.lst, else the update purge drops the
file we refused to overwrite with the partial body (#562). */
if (fexist_utf8(back[p].url_sav))
filenote(&opt->state.strc, back[p].url_sav, NULL);
return -1;
}

View File

@@ -6030,6 +6030,8 @@ HTSEXT_API void hts_free_opt(httrackp * opt) {
StringFree(opt->referer);
StringFree(opt->from);
StringFree(opt->lang_iso);
StringFree(opt->accept);
StringFree(opt->headers);
StringFree(opt->sys_com);
StringFree(opt->mimedefs);
StringFree(opt->filelist);

View File

@@ -8,22 +8,26 @@ set -euo pipefail
: "${top_srcdir:=..}"
# shellcheck source=tests/testlib.sh
. "$top_srcdir/tests/testlib.sh"
if test "${HTTPS_SUPPORT:-}" == "no"; then
echo "no https support compiled, skipping"
exit 77
fi
if ! command -v python3 >/dev/null 2>&1 || ! command -v openssl >/dev/null 2>&1; then
python=$(find_python) || python=
if test -z "$python" || ! command -v openssl >/dev/null 2>&1; then
echo "python3/openssl missing, skipping"
exit 77
fi
server="$top_srcdir/tests/proxy-https-server.py"
server=$(nativepath "$top_srcdir/tests/proxy-https-server.py")
tmpdir=$(mktemp -d)
pids=
cleanup() {
for pid in $pids; do
kill "$pid" 2>/dev/null || true
stop_server "$pid"
done
rm -rf "$tmpdir"
}
@@ -38,10 +42,13 @@ cat "$tmpdir/key.pem" "$tmpdir/cert.pem" >"$tmpdir/both.pem"
# start_server <logdir> <mode>: launches a proxy+origin pair, sets $origin_port
# and $proxy_port from its announced ephemeral ports.
start_server() {
local dir="$1" mode="$2" ports
local dir="$1" mode="$2" ports pem
mkdir -p "$dir"
ports="$dir/ports.txt"
python3 "$server" "$tmpdir/both.pem" "$dir" "$mode" \
pem=$(nativepath "$tmpdir/both.pem")
dir_native=$(nativepath "$dir")
: >"$ports"
"$python" "$server" "$pem" "$dir_native" "$mode" \
>"$ports" 2>"$dir/server.err" &
pids="$pids $!"
for _ in $(seq 1 100); do

View File

@@ -12,14 +12,17 @@ set -euo pipefail
: "${top_srcdir:=..}"
# shellcheck source=tests/testlib.sh
. "$top_srcdir/tests/testlib.sh"
if test "${V6_SUPPORT:-}" == "no"; then
echo "no IPv6 support (resolver list/override is IPv6-only), skipping"
exit 77
fi
if ! command -v python3 >/dev/null 2>&1; then
python=$(find_python) || {
echo "python3 missing, skipping"
exit 77
fi
}
# The fixture needs a second loopback IP (dead 127.0.0.2 + live 127.0.0.1) for
# the fallback to have a target; GNU/Hurd has only 127.0.0.1, so skip there.
case "$(uname -s)" in
@@ -29,23 +32,20 @@ GNU | GNU/*)
;;
esac
server="$top_srcdir/tests/local-server.py"
root="$top_srcdir/tests/server-root"
server=$(nativepath "$top_srcdir/tests/local-server.py")
root=$(nativepath "$top_srcdir/tests/server-root")
tmpdir=$(mktemp -d)
serverpid=
cleanup() {
if test -n "$serverpid"; then
kill "$serverpid" 2>/dev/null || true
wait "$serverpid" 2>/dev/null || true
fi
stop_server "$serverpid"
rm -rf "$tmpdir"
return 0
}
trap cleanup EXIT
# bind the live server to 127.0.0.1 only, so 127.0.0.2 refuses the connect
python3 "$server" --root "$root" --bind 127.0.0.1 >"$tmpdir/srv.out" 2>"$tmpdir/srv.err" &
"$python" "$server" --root "$root" --bind 127.0.0.1 >"$tmpdir/srv.out" 2>"$tmpdir/srv.err" &
serverpid=$!
port=
for _ in $(seq 1 50); do

View File

@@ -5,19 +5,19 @@ set -u
: "${top_srcdir:=..}"
testdir=$(cd "$(dirname "$0")" && pwd)
server="${testdir}/local-server.py"
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
server=$(nativepath "${testdir}/local-server.py")
root=$(nativepath "${testdir}/server-root")
command -v python3 >/dev/null || ! echo "python3 not found; skipping" || exit 77
python=$(find_python) || ! echo "python3 not found; skipping" >&2 || exit 77
tmpdir=$(mktemp -d "${TMPDIR:-/tmp}/httrack_206.XXXXXX") || exit 1
serverpid=
crawlpid=
cleanup() {
test -n "$crawlpid" && kill -9 "$crawlpid" 2>/dev/null
if test -n "$serverpid"; then
kill "$serverpid" 2>/dev/null
wait "$serverpid" 2>/dev/null
fi
stop_server "$serverpid"
rm -rf "$tmpdir"
}
trap cleanup EXIT HUP INT QUIT PIPE TERM
@@ -25,8 +25,9 @@ trap cleanup EXIT HUP INT QUIT PIPE TERM
# --- start the server, discover its ephemeral port --------------------------
# RESUME_COUNTER gets a byte per /resume/blob.txt request (pass-2 delta bounds re-gets).
serverlog="${tmpdir}/server.log"
: >"$serverlog"
counter="${tmpdir}/blobcount"
RESUME_COUNTER="$counter" python3 "$server" --root "${testdir}/server-root" >"$serverlog" 2>&1 &
RESUME_COUNTER="$counter" "$python" "$server" --root "$root" >"$serverlog" 2>&1 &
serverpid=$!
port=
for _ in $(seq 1 50); do

View File

@@ -9,29 +9,30 @@ set -eu
: "${top_srcdir:=..}"
testdir=$(cd "$(dirname "$0")" && pwd)
server="${testdir}/local-server.py"
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
server=$(nativepath "${testdir}/local-server.py")
root=$(nativepath "${testdir}/server-root")
command -v python3 >/dev/null || ! echo "python3 not found; skipping" || exit 77
python=$(find_python) || ! echo "python3 not found; skipping" >&2 || exit 77
tmpdir=$(mktemp -d "${TMPDIR:-/tmp}/httrack_198.XXXXXX") || exit 1
serverpid=
crawlpid=
cleanup() {
if test -n "$crawlpid"; then kill -9 "$crawlpid" 2>/dev/null || true; fi
if test -n "$serverpid"; then
kill "$serverpid" 2>/dev/null || true
wait "$serverpid" 2>/dev/null || true
fi
stop_server "$serverpid"
rm -rf "$tmpdir"
}
trap cleanup EXIT HUP INT QUIT PIPE TERM
# OVERLAP_COUNTER gets a byte per flaky.bin request so pass 1 knows when to interrupt.
serverlog="${tmpdir}/server.log"
: >"$serverlog"
counter="${tmpdir}/hits"
resumed="${tmpdir}/resumed" # gets a byte when the server serves a resume 206
OVERLAP_COUNTER="$counter" OVERLAP_RESUMED="$resumed" \
python3 "$server" --root "${testdir}/server-root" \
"$python" "$server" --root "$root" \
>"$serverlog" 2>&1 &
serverpid=$!
port=

View File

@@ -9,9 +9,12 @@ set -e
: "${top_srcdir:=..}"
# shellcheck source=tests/testlib.sh
. "$top_srcdir/tests/testlib.sh"
# python3 runs the local server (mirror local-crawl.sh); skip when absent, else
# run() swallows its exit-77 and the serverless 0s/0s crawl looks like a fail.
command -v python3 >/dev/null || {
find_python >/dev/null || {
echo "python3 not found; skipping local crawl tests"
exit 77
}

View File

@@ -7,27 +7,28 @@ set -u
: "${top_srcdir:=..}"
testdir=$(cd "$(dirname "$0")" && pwd)
server="${testdir}/local-server.py"
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
server=$(nativepath "${testdir}/local-server.py")
root=$(nativepath "${testdir}/server-root")
command -v python3 >/dev/null || ! echo "python3 not found; skipping" || exit 77
python=$(find_python) || ! echo "python3 not found; skipping" >&2 || exit 77
tmpdir=$(mktemp -d "${TMPDIR:-/tmp}/httrack_c7.XXXXXX") || exit 1
serverpid=
crawlpid=
cleanup() {
test -n "$crawlpid" && kill -9 "$crawlpid" 2>/dev/null
if test -n "$serverpid"; then
kill "$serverpid" 2>/dev/null
wait "$serverpid" 2>/dev/null
fi
stop_server "$serverpid"
rm -rf "$tmpdir"
}
trap cleanup EXIT HUP INT QUIT PIPE TERM
serverlog="${tmpdir}/server.log"
: >"$serverlog"
counter="${tmpdir}/reqcount"
mark="${tmpdir}/got304"
RESUME304_COUNTER="$counter" RESUME304_MARK="$mark" python3 "$server" --root "${testdir}/server-root" >"$serverlog" 2>&1 &
RESUME304_COUNTER="$counter" RESUME304_MARK="$mark" "$python" "$server" --root "$root" >"$serverlog" 2>&1 &
serverpid=$!
port=
for _ in $(seq 1 50); do

View File

@@ -8,25 +8,26 @@ set -u
: "${top_srcdir:=..}"
testdir=$(cd "$(dirname "$0")" && pwd)
server="${testdir}/local-server.py"
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
server=$(nativepath "${testdir}/local-server.py")
root=$(nativepath "${testdir}/server-root")
command -v python3 >/dev/null || ! echo "python3 not found; skipping" || exit 77
python=$(find_python) || ! echo "python3 not found; skipping" >&2 || exit 77
tmpdir=$(mktemp -d "${TMPDIR:-/tmp}/httrack_crange.XXXXXX") || exit 1
serverpid=
crawlpid=
cleanup() {
test -n "$crawlpid" && kill -9 "$crawlpid" 2>/dev/null
if test -n "$serverpid"; then
kill "$serverpid" 2>/dev/null
wait "$serverpid" 2>/dev/null
fi
stop_server "$serverpid"
rm -rf "$tmpdir"
}
trap cleanup EXIT HUP INT QUIT PIPE TERM
serverlog="${tmpdir}/server.log"
python3 "$server" --root "${testdir}/server-root" >"$serverlog" 2>&1 &
: >"$serverlog"
"$python" "$server" --root "$root" >"$serverlog" 2>&1 &
serverpid=$!
port=
for _ in $(seq 1 50); do

View File

@@ -8,25 +8,34 @@ set -u
: "${top_srcdir:=..}"
testdir=$(cd "$(dirname "$0")" && pwd)
server="${testdir}/local-server.py"
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
server=$(nativepath "${testdir}/local-server.py")
root=$(nativepath "${testdir}/server-root")
command -v python3 >/dev/null || ! echo "python3 not found; skipping" || exit 77
python=$(find_python) || ! echo "python3 not found; skipping" >&2 || exit 77
# The resume needs a graceful pass-1 interrupt (a clean cache), which MSYS can't
# deliver to a native exe, and the restart-whole path fails on a repaired cache
# (#581); TODO: drop once that lands.
if is_windows; then
echo "Windows: interrupted-resume restart fails on a repaired cache (#581), skipping"
exit 77
fi
tmpdir=$(mktemp -d "${TMPDIR:-/tmp}/httrack_crangemem.XXXXXX") || exit 1
serverpid=
crawlpid=
cleanup() {
test -n "$crawlpid" && kill -9 "$crawlpid" 2>/dev/null
if test -n "$serverpid"; then
kill "$serverpid" 2>/dev/null
wait "$serverpid" 2>/dev/null
fi
stop_server "$serverpid"
rm -rf "$tmpdir"
}
trap cleanup EXIT HUP INT QUIT PIPE TERM
serverlog="${tmpdir}/server.log"
python3 "$server" --root "${testdir}/server-root" >"$serverlog" 2>&1 &
: >"$serverlog"
"$python" "$server" --root "$root" >"$serverlog" 2>&1 &
serverpid=$!
port=
for _ in $(seq 1 50); do
@@ -77,7 +86,14 @@ echo "OK (temp-ref present)"
# --- pass 2: --continue -> resume -> hostile INT64_MAX Content-Range ----------
printf '[pass 2: hostile 206, no overflow, refetch] ..\t'
httrack "${common[@]}" --continue "${base}/crange206mem/index.html" >"${tmpdir}/log2" 2>&1
rc=0
httrack "${common[@]}" --continue "${base}/crange206mem/index.html" >"${tmpdir}/log2" 2>&1 || rc=$?
# a crash here would otherwise read as a clean "terminated"
test "$rc" -eq 0 || {
echo "FAIL: httrack exited $rc"
cat "${tmpdir}/log2" >&2
exit 1
}
echo "OK (terminated)"
blob=$(find "$out" -name blob.bin 2>/dev/null | head -1)

View File

@@ -8,11 +8,15 @@ set -euo pipefail
: "${top_srcdir:=..}"
# shellcheck source=tests/testlib.sh
. "$top_srcdir/tests/testlib.sh"
if test "${HTTPS_SUPPORT:-}" == "no"; then
echo "no https support compiled, skipping"
exit 77
fi
if ! command -v python3 >/dev/null 2>&1 || ! command -v openssl >/dev/null 2>&1; then
python=$(find_python) || python=
if test -z "$python" || ! command -v openssl >/dev/null 2>&1; then
echo "python3/openssl missing, skipping"
exit 77
fi
@@ -20,13 +24,13 @@ fi
# a .invalid name never resolves (RFC 6761): reaching the page at all proves the
# proxy did the DNS
host="socks-origin.invalid"
server="$top_srcdir/tests/socks5-server.py"
server=$(nativepath "$top_srcdir/tests/socks5-server.py")
tmpdir=$(mktemp -d)
pids=
cleanup() {
for pid in $pids; do
kill "$pid" 2>/dev/null || true
stop_server "$pid"
done
rm -rf "$tmpdir"
}
@@ -39,10 +43,13 @@ cat "$tmpdir/key.pem" "$tmpdir/cert.pem" >"$tmpdir/both.pem"
# start_server <logdir> <mode>: sets $tls_port/$http_port/$socks_port
start_server() {
local dir="$1" mode="$2" ports
local dir="$1" mode="$2" ports pem
mkdir -p "$dir"
ports="$dir/ports.txt"
python3 "$server" "$tmpdir/both.pem" "$dir" "$mode" \
pem=$(nativepath "$tmpdir/both.pem")
dir_native=$(nativepath "$dir")
: >"$ports"
"$python" "$server" "$pem" "$dir_native" "$mode" \
>"$ports" 2>"$dir/server.err" &
pids="$pids $!"
for _ in $(seq 1 100); do

View File

@@ -0,0 +1,23 @@
#!/bin/bash
#
# An --update re-fetch that comes in short must not delete the mirrored file (#562).
# Pass 1 mirrors page.html (in-memory) and file.bin (direct-to-disk); pass 2 serves
# both with the full declared Content-Length but half the body, then closes. httrack
# refuses the partial ("will be retried") — and unfixed, the end-of-update purge then
# deletes the good pass-1 copy, which was never overwritten. stay.html is the control:
# fully served both passes, it must update to V2.
set -euo pipefail
: "${top_srcdir:=..}"
bash "$top_srcdir/tests/local-crawl.sh" \
--rerun-args '--update' \
--log-found 'incomplete transfer.*uptrunc/page\.html' \
--log-found 'incomplete transfer.*uptrunc/file\.bin' \
--file-matches 'uptrunc/page.html' 'MIRRORED-PAGE-V1' \
--file-matches 'uptrunc/file.bin' 'MIRRORED-BIN-V1' \
--file-min-bytes 'uptrunc/file.bin' 32768 \
--file-matches 'uptrunc/stay.html' 'STAY-V2' \
--file-not-matches 'uptrunc/stay.html' 'STAY-V1' \
httrack 'BASEURL/uptrunc/index.html'

View File

@@ -3,7 +3,7 @@
# silently drop it from the dist tarball and break "make distcheck".
EXTRA_DIST = $(TESTS) crawl-test.sh run-all-tests.sh check-network.sh \
proxy-https-server.py socks5-server.py \
local-crawl.sh local-server.py server.crt server.key \
local-crawl.sh local-server.py testlib.sh server.crt server.key \
server-root/simple/basic.html server-root/simple/link.html \
server-root/stripquery/index.html server-root/stripquery/a.html \
server-root/fraglink/index.html server-root/fraglink/target.html \
@@ -130,6 +130,7 @@ TESTS = \
50_local-contentcodings.test \
51_local-update-codec.test \
52_local-socks5.test \
53_local-proxytrack-cache-corrupt.test
53_local-proxytrack-cache-corrupt.test \
54_local-update-truncate-purge.test
CLEANFILES = check-network_sh.cache

View File

@@ -38,6 +38,8 @@
set -u
testdir=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
server="${testdir}/local-server.py"
root="${LOCAL_SERVER_ROOT:-${testdir}/server-root}"
cert="${testdir}/server.crt"
@@ -72,12 +74,8 @@ function cleanup {
kill -9 "$crawlpid" 2>/dev/null
crawlpid=
fi
if test -n "$serverpid"; then
kill "$serverpid" 2>/dev/null
# Reap it so the port is released before we rm the tmpdir/log.
wait "$serverpid" 2>/dev/null
serverpid=
fi
stop_server "$serverpid"
serverpid=
if test -n "$tmpdir" && test -d "$tmpdir"; then
test -n "$nopurge" || rm -rf "$tmpdir"
fi
@@ -96,7 +94,7 @@ nopurge=
trap cleanup EXIT HUP INT QUIT PIPE TERM
# python3 is required; mirror check-network.sh's skip-with-77 convention.
command -v python3 >/dev/null || ! echo "python3 not found; skipping local crawl tests" || exit 77
python=$(find_python) || ! echo "python3 not found; skipping local crawl tests" >&2 || exit 77
tmptopdir=${TMPDIR:-/tmp}
test -d "$tmptopdir" || mkdir -p "$tmptopdir" || die "no temporary directory; set TMPDIR"
@@ -158,12 +156,12 @@ done
# --- start the server --------------------------------------------------------
test -r "$server" || die "cannot read $server"
serverlog="${tmpdir}/server.log"
serverargs=(--root "$root")
serverargs=(--root "$(nativepath "$root")")
if test -n "$tls"; then
serverargs+=(--tls --cert "$cert" --key "$key")
serverargs+=(--tls --cert "$(nativepath "$cert")" --key "$(nativepath "$key")")
fi
debug "starting python3 $server ${serverargs[*]}"
python3 "$server" "${serverargs[@]}" >"$serverlog" 2>&1 &
debug "starting $python $server ${serverargs[*]}"
"$python" "$(nativepath "$server")" "${serverargs[@]}" >"$serverlog" 2>&1 &
serverpid=$!
# Wait for the "PORT <n>" line (server prints it once bound).
@@ -203,7 +201,7 @@ if test "${#cookies[@]}" -gt 0; then
fi
# --- run httrack -------------------------------------------------------------
which httrack >/dev/null || die "could not find httrack"
command -v httrack >/dev/null || die "could not find httrack"
ver=$(httrack -O /dev/null --version | sed -e 's/HTTrack version //')
test -n "$ver" || die "could not run httrack"
@@ -278,8 +276,7 @@ if test -n "$rerun_dead"; then
test -s "$zip" || die "no cache was written by the first pass"
cp "$zip" "${tmpdir}/cache-before.zip"
cp "${out}/hts-log.txt" "${tmpdir}/log-before.txt"
kill "$serverpid" 2>/dev/null
wait "$serverpid" 2>/dev/null
stop_server "$serverpid"
serverpid=
info "re-running httrack against the stopped server"
httrack -O "$out" --user-agent="httrack $ver local ($(uname -mrs))" \
@@ -441,12 +438,18 @@ while test "$i" -lt "${#audit[@]}"; do
--file-mode)
path="${audit[$((i + 1))]}"
i=$((i + 2))
mode=$(stat -c '%a' "${hostroot}/${path}" 2>/dev/null ||
stat -f '%Lp' "${hostroot}/${path}" 2>/dev/null)
info "checking ${path} mode ${mode:-none} is ${audit[$i]}"
if test "$mode" = "${audit[$i]}"; then result "OK"; else
result "wrong mode"
exit 1
if is_windows; then
# No POSIX modes, and the engine only chmods #ifndef _WIN32.
info "checking ${path} mode"
result "SKIP (no POSIX modes)"
else
mode=$(stat -c '%a' "${hostroot}/${path}" 2>/dev/null ||
stat -f '%Lp' "${hostroot}/${path}" 2>/dev/null)
info "checking ${path} mode ${mode:-none} is ${audit[$i]}"
if test "$mode" = "${audit[$i]}"; then result "OK"; else
result "wrong mode"
exit 1
fi
fi
;;
esac

View File

@@ -18,6 +18,7 @@ import base64
import gzip
import hashlib
import os
import sys
import time
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import quote, unquote, urlsplit
@@ -699,15 +700,17 @@ class Handler(SimpleHTTPRequestHandler):
# Pass 1 mirrors each file from a valid gzip body; pass 2 (--update) serves
# a body that cannot be decoded. The previously-mirrored copy must survive.
# fresh.html is the control: its pass-2 body decodes, so it must be updated.
UPCODEC_SEEN = {}
# Per-path body-fetch counter shared by the update-refetch routes; paths are
# distinct so one dict serves all of them.
REFETCH_SEEN = {}
def upcodec_pass(self):
"""1 on the first body fetch of this path, 2 on the next ones. HEADs
don't count, so a stray one can't shift which pass gets the bad body."""
def refetch_pass(self):
"""1 on the first body fetch of this path, N on the Nth. HEADs don't
count, so a stray one can't shift which pass gets the special body."""
if self.command == "HEAD":
return 1
seen = Handler.UPCODEC_SEEN.get(self.path, 0) + 1
Handler.UPCODEC_SEEN[self.path] = seen
seen = Handler.REFETCH_SEEN.get(self.path, 0) + 1
Handler.REFETCH_SEEN[self.path] = seen
return seen
@staticmethod
@@ -739,36 +742,78 @@ class Handler(SimpleHTTPRequestHandler):
UNSUP_V1 = b"<html><body><p>MIRRORED-UNSUP-V1</p></body></html>"
def route_upcodec_mem(self):
if self.upcodec_pass() == 1:
if self.refetch_pass() == 1:
self.send_coded(self.gzipped(self.MEM_V1), "text/html")
else:
self.send_coded(self.bad_gzip(self.MEM_V1), "text/html")
def route_upcodec_disk(self):
if self.upcodec_pass() == 1:
if self.refetch_pass() == 1:
self.send_coded(self.gzipped(self.DISK_V1), "application/octet-stream")
else:
self.send_coded(self.bad_gzip(self.DISK_V1), "application/octet-stream")
# Pass 2 switches to a coding we have no decoder for.
def route_upcodec_unsup(self):
if self.upcodec_pass() == 1:
if self.refetch_pass() == 1:
self.send_coded(self.gzipped(self.UNSUP_V1), "text/html")
else:
self.send_coded(self.UNSUP_V1, "text/html", coding="compress")
def route_upcodec_fresh(self):
pass1 = self.upcodec_pass() == 1
pass1 = self.refetch_pass() == 1
body = b"<html><body><p>FRESH-V%d</p></body></html>" % (1 if pass1 else 2)
self.send_coded(self.gzipped(body), "text/html")
# Same, direct-to-disk: the update pass decodes, so the temp is renamed over
# an existing mirror file.
def route_upcodec_freshdisk(self):
pass1 = self.upcodec_pass() == 1
pass1 = self.refetch_pass() == 1
body = b"FRESHDISK-V%d\n" % (1 if pass1 else 2) + b"\x03\x02\x01\xfe" * 8192
self.send_coded(self.gzipped(body), "application/octet-stream")
# #562: pass 1 mirrors fully; pass 2 (--update) declares the full
# Content-Length but delivers half then closes, so httrack refuses the partial.
PAGE_V1 = b"<html><body><p>MIRRORED-PAGE-V1</p></body></html>"
BIN_V1 = b"MIRRORED-BIN-V1\n" + b"\x00\x01\x02\xff" * 8192
def route_uptrunc_index(self):
self.send_html(
'\t<a href="page.html">page</a>\n'
'\t<a href="file.bin">file</a>\n'
'\t<a href="stay.html">stay</a>\n'
)
def send_truncated(self, body, content_type):
self.send_response(200)
self.send_header("Content-Type", content_type)
self.send_header("Content-Length", str(len(body)))
self.end_headers()
if self.command == "HEAD":
return
try:
self.wfile.write(body[: len(body) // 2]) # short, then close
self.wfile.flush()
except OSError:
pass
def route_uptrunc_page(self):
if self.refetch_pass() == 1:
self.send_raw(self.PAGE_V1, "text/html")
else:
self.send_truncated(self.PAGE_V1, "text/html")
def route_uptrunc_file(self):
if self.refetch_pass() == 1:
self.send_raw(self.BIN_V1, "application/octet-stream")
else:
self.send_truncated(self.BIN_V1, "application/octet-stream")
# Control: fully served both passes, so a normal --update still lands.
def route_uptrunc_stay(self):
v = 1 if self.refetch_pass() == 1 else 2
self.send_raw(b"<html><body><p>STAY-V%d</p></body></html>" % v, "text/html")
# Echo what httrack advertised, so a crawl can assert the header.
def route_codec_ae(self):
self.send_raw(
@@ -1472,6 +1517,10 @@ class Handler(SimpleHTTPRequestHandler):
"/upcodec/unsup.html": route_upcodec_unsup,
"/upcodec/fresh.html": route_upcodec_fresh,
"/upcodec/freshdisk.bin": route_upcodec_freshdisk,
"/uptrunc/index.html": route_uptrunc_index,
"/uptrunc/page.html": route_uptrunc_page,
"/uptrunc/file.bin": route_uptrunc_file,
"/uptrunc/stay.html": route_uptrunc_stay,
"/types/index.html": route_types_index,
"/types/control.php": route_types,
"/types/photo.png": route_types,
@@ -1817,7 +1866,8 @@ def main():
httpd.socket = ctx.wrap_socket(httpd.socket, server_side=True)
port = httpd.socket.getsockname()[1]
# The launcher reads this line to discover the ephemeral port.
# Keep the port line the launcher parses LF: Windows would emit \r\n.
sys.stdout.reconfigure(newline="\n")
print(f"PORT {port}", flush=True)
try:

View File

@@ -141,6 +141,8 @@ def main():
open(os.path.join(logdir, name), "w").close()
origin_port = start_origin(certfile, logdir)
proxy_port = start_proxy(logdir, mode)
# Keep the port lines the caller parses LF: Windows would emit \r\n.
sys.stdout.reconfigure(newline="\n")
print("ORIGIN %d" % origin_port, flush=True)
print("PROXY %d" % proxy_port, flush=True)
print("ready", flush=True)

View File

@@ -217,6 +217,8 @@ def main():
tls_port = start_origin(logdir, certfile)
http_port = start_origin(logdir, None)
socks_port = start_socks(logdir, mode)
# Keep the port lines the caller parses LF: Windows would emit \r\n.
sys.stdout.reconfigure(newline="\n")
print("TLS %d" % tls_port, flush=True)
print("HTTP %d" % http_port, flush=True)
print("SOCKS %d" % socks_port, flush=True)

45
tests/testlib.sh Normal file
View File

@@ -0,0 +1,45 @@
#!/bin/bash
#
# Helpers shared by the crawl tests. Sourced, not run.
# Python 3 interpreter, or empty: Windows only installs python.exe, and a bare
# "python" may be 2.x or the Store stub.
find_python() {
local py
for py in "${PYTHON:-}" python3 python; do
test -n "$py" || continue
"$py" -c 'import sys; sys.exit(sys.version_info[0] != 3)' 2>/dev/null || continue
printf '%s\n' "$py"
return 0
done
return 1
}
# Native form of a path: a non-MSYS binary cannot resolve Git Bash's /d/a/... ones.
nativepath() {
if is_windows && command -v cygpath >/dev/null 2>&1; then
cygpath -m "$1"
else
printf '%s\n' "$1"
fi
}
is_windows() {
case "$(uname -s)" in
MINGW* | MSYS* | CYGWIN*) return 0 ;;
*) return 1 ;;
esac
}
# Stop a backgrounded server and reap it; MSYS cannot signal a native python.exe,
# so only -9 lands. Every step is "|| true": callers run under set -e, and reaping
# a server we just signalled makes wait return 143.
stop_server() {
test -n "${1:-}" || return 0
kill "$1" 2>/dev/null || true
if is_windows; then
kill -9 "$1" 2>/dev/null || true
fi
wait "$1" 2>/dev/null || true
return 0
}