mirror of
https://github.com/xroche/httrack.git
synced 2026-07-29 03:52:13 +03:00
bc6c53598c
* The -o help text promises a generated error page the engine never builds `-o` only decides whether the error page the server sent survives: `store_errpage` keeps `r.adr` alive so the normal save path writes it, and the `-o0` arm frees it. Nothing anywhere builds a stand-in body. The one block that would have was dead since the 3.20.2 import and was removed in #783. Reword the help line, the man page and fcguide's two `-o` prose blocks to say the server's error page is saved rather than generated, and extend 23_local-errpage so the mirrored 404 has to carry the server's own body. Closes #787 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Condense the -o1 control comment to one line Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <xroche@gmail.com> --------- Signed-off-by: Xavier Roche <roche@httrack.com> Signed-off-by: Xavier Roche <xroche@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2423 lines
51 KiB
HTML
2423 lines
51 KiB
HTML
<!-- Creator : groff -->
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
|
|
"http://www.w3.org/TR/html4/loose.dtd">
|
|
<html>
|
|
<head>
|
|
<meta name="generator" content="groff -Thtml, see www.gnu.org">
|
|
<meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
|
|
<meta name="Content-Style" content="text/css">
|
|
<style type="text/css">
|
|
p { margin-top: 0; margin-bottom: 0; vertical-align: top }
|
|
pre { margin-top: 0; margin-bottom: 0; vertical-align: top }
|
|
table { margin-top: 0; margin-bottom: 0; vertical-align: top }
|
|
h1 { text-align: center }
|
|
</style>
|
|
<title>httrack</title>
|
|
|
|
</head>
|
|
<body>
|
|
|
|
<h1 align="center">httrack</h1>
|
|
|
|
<a href="#NAME">NAME</a><br>
|
|
<a href="#SYNOPSIS">SYNOPSIS</a><br>
|
|
<a href="#DESCRIPTION">DESCRIPTION</a><br>
|
|
<a href="#EXAMPLES">EXAMPLES</a><br>
|
|
<a href="#OPTIONS">OPTIONS</a><br>
|
|
<a href="#General options:">General options:</a><br>
|
|
<a href="#Action options:">Action options:</a><br>
|
|
<a href="#Proxy options:">Proxy options:</a><br>
|
|
<a href="#Limits options:">Limits options:</a><br>
|
|
<a href="#Flow control:">Flow control:</a><br>
|
|
<a href="#Links options:">Links options:</a><br>
|
|
<a href="#Build options:">Build options:</a><br>
|
|
<a href="#Spider options:">Spider options:</a><br>
|
|
<a href="#Browser ID:">Browser ID:</a><br>
|
|
<a href="#Log, index, cache">Log, index, cache</a><br>
|
|
<a href="#Expert options:">Expert options:</a><br>
|
|
<a href="#Guru options: (do NOT use if possible)">Guru options: (do NOT use if possible)</a><br>
|
|
<a href="#Dangerous options: (do NOT use unless you exactly know what you are doing)">Dangerous options: (do NOT use unless you exactly know what you are doing)</a><br>
|
|
<a href="#Command-line specific options:">Command-line specific options:</a><br>
|
|
<a href="#Details: Option N">Details: Option N</a><br>
|
|
<a href="#Details: User-defined option N">Details: User-defined option N</a><br>
|
|
<a href="#Details: User-defined option N and advanced variable extraction">Details: User-defined option N and advanced variable extraction</a><br>
|
|
<a href="#Details: Option K">Details: Option K</a><br>
|
|
<a href="#Shortcuts:">Shortcuts:</a><br>
|
|
<a href="#Details: Option %W: External callbacks prototypes">Details: Option %W: External callbacks prototypes</a><br>
|
|
<a href="#see htsdefines.h">see htsdefines.h</a><br>
|
|
<a href="#FILES">FILES</a><br>
|
|
<a href="#ENVIRONMENT">ENVIRONMENT</a><br>
|
|
<a href="#DIAGNOSTICS">DIAGNOSTICS</a><br>
|
|
<a href="#LIMITS">LIMITS</a><br>
|
|
<a href="#BUGS">BUGS</a><br>
|
|
<a href="#COPYRIGHT">COPYRIGHT</a><br>
|
|
<a href="#AVAILABILITY">AVAILABILITY</a><br>
|
|
<a href="#AUTHOR">AUTHOR</a><br>
|
|
<a href="#SEE ALSO">SEE ALSO</a><br>
|
|
|
|
<hr>
|
|
|
|
|
|
<h2>NAME
|
|
<a name="NAME"></a>
|
|
</h2>
|
|
|
|
|
|
<p style="margin-left:9%; margin-top: 1em">httrack -
|
|
offline browser : copy websites to a local directory</p>
|
|
|
|
<h2>SYNOPSIS
|
|
<a name="SYNOPSIS"></a>
|
|
</h2>
|
|
|
|
|
|
<p style="margin-left:9%; margin-top: 1em"><b>httrack [ url
|
|
]... [ -filter ]... [ +filter ]...</b> [ <b>-O, --path</b> ]
|
|
[ <b>-w, --mirror</b> ] [ <b>-W, --mirror-wizard</b> ] [
|
|
<b>-g, --get-files</b> ] [ <b>-i, --continue</b> ] [ <b>-Y,
|
|
--mirrorlinks</b> ] [ <b>-P, --proxy</b> ] [ <b>-%f,
|
|
--httpproxy-ftp[=N]</b> ] [ <b>-%b, --bind</b> ] [ <b>-rN,
|
|
--depth[=N]</b> ] [ <b>-%eN, --ext-depth[=N]</b> ] [ <b>-mN,
|
|
--max-files[=N]</b> ] [ <b>-MN, --max-size[=N]</b> ] [
|
|
<b>-EN, --max-time[=N]</b> ] [ <b>-AN, --max-rate[=N]</b> ]
|
|
[ <b>-%cN, --connection-per-second[=N]</b> ] [ <b>-%G,
|
|
--pause</b> ] [ <b>-GN, --max-pause[=N]</b> ] [ <b>-cN,
|
|
--sockets[=N]</b> ] [ <b>-TN, --timeout[=N]</b> ] [ <b>-RN,
|
|
--retries[=N]</b> ] [ <b>-JN, --min-rate[=N]</b> ] [ <b>-HN,
|
|
--host-control[=N]</b> ] [ <b>-%P,
|
|
--extended-parsing[=N]</b> ] [ <b>-n, --near</b> ] [ <b>-t,
|
|
--test</b> ] [ <b>-%L, --list</b> ] [ <b>-%S, --urllist</b>
|
|
] [ <b>-%m, --sitemap</b> ] [ <b>-NN, --structure[=N]</b> ]
|
|
[ <b>-%N, --delayed-type-check</b> ] [ <b>-%D,
|
|
--cached-delayed-type-check</b> ] [ <b>-%M, --mime-html</b>
|
|
] [ <b>-%Z, --single-file</b> ] [ <b>-LN,
|
|
--long-names[=N]</b> ] [ <b>-KN, --keep-links[=N]</b> ] [
|
|
<b>-x, --replace-external</b> ] [ <b>-%x,
|
|
--disable-passwords</b> ] [ <b>-%q,
|
|
--include-query-string</b> ] [ <b>-%g, --strip-query</b> ] [
|
|
<b>-o, --generate-errors</b> ] [ <b>-X, --purge-old[=N]</b>
|
|
] [ <b>-%p, --preserve</b> ] [ <b>-%T, --utf8-conversion</b>
|
|
] [ <b>-bN, --cookies[=N]</b> ] [ <b>-%K, --cookies-file</b>
|
|
] [ <b>-%Y, --why</b> ] [ <b>-u, --check-type[=N]</b> ] [
|
|
<b>-j, --parse-java[=N]</b> ] [ <b>-sN, --robots[=N]</b> ] [
|
|
<b>-%h, --http-10</b> ] [ <b>-%k, --keep-alive</b> ] [
|
|
<b>-%z, --disable-compression</b> ] [ <b>-%B, --tolerant</b>
|
|
] [ <b>-%s, --updatehack</b> ] [ <b>-%u, --urlhack</b> ] [
|
|
<b>-%A, --assume</b> ] [ <b>-@iN, --protocol[=N]</b> ] [
|
|
<b>-%w, --disable-module</b> ] [ <b>-F, --user-agent</b> ] [
|
|
<b>-%R, --referer</b> ] [ <b>-%E, --from</b> ] [ <b>-%F,
|
|
--footer</b> ] [ <b>-%l, --language</b> ] [ <b>-%a,
|
|
--accept</b> ] [ <b>-%X, --headers</b> ] [ <b>-C,
|
|
--cache[=N]</b> ] [ <b>-k, --store-all-in-cache</b> ] [
|
|
<b>-%r, --warc</b> ] [ <b>-%d, --changes</b> ] [ <b>-%n,
|
|
--do-not-recatch</b> ] [ <b>-%v, --display</b> ] [ <b>-Q,
|
|
--do-not-log</b> ] [ <b>-q, --quiet</b> ] [ <b>-z,
|
|
--extra-log</b> ] [ <b>-Z, --debug-log</b> ] [ <b>-v,
|
|
--verbose</b> ] [ <b>-f, --file-log</b> ] [ <b>-f2,
|
|
--single-log</b> ] [ <b>-I, --index</b> ] [ <b>-%i,
|
|
--build-top-index</b> ] [ <b>-%I, --search-index</b> ] [
|
|
<b>-pN, --priority[=N]</b> ] [ <b>-S, --stay-on-same-dir</b>
|
|
] [ <b>-D, --can-go-down</b> ] [ <b>-U, --can-go-up</b> ] [
|
|
<b>-B, --can-go-up-and-down</b> ] [ <b>-a,
|
|
--stay-on-same-address</b> ] [ <b>-d,
|
|
--stay-on-same-domain</b> ] [ <b>-l, --stay-on-same-tld</b>
|
|
] [ <b>-e, --go-everywhere</b> ] [ <b>-%H,
|
|
--debug-headers</b> ] [ <b>-%!,
|
|
--disable-security-limits</b> ] [ <b>-V, --userdef-cmd</b> ]
|
|
[ <b>-%W, --callback</b> ] [ <b>-y,
|
|
--background-on-suspend</b> ] [ <b>-K, --keep-links[=N]</b>
|
|
]</p>
|
|
|
|
<h2>DESCRIPTION
|
|
<a name="DESCRIPTION"></a>
|
|
</h2>
|
|
|
|
|
|
<p style="margin-left:9%; margin-top: 1em"><b>httrack</b>
|
|
allows you to download a World Wide Web site from the
|
|
Internet to a local directory, building recursively all
|
|
directories, getting HTML, images, and other files from the
|
|
server to your computer. HTTrack arranges the original
|
|
site’s relative link-structure. Simply open a page of
|
|
the "mirrored" website in your browser, and you
|
|
can browse the site from link to link, as if you were
|
|
viewing it online. HTTrack can also update an existing
|
|
mirrored site, and resume interrupted downloads.</p>
|
|
|
|
<h2>EXAMPLES
|
|
<a name="EXAMPLES"></a>
|
|
</h2>
|
|
|
|
|
|
<p style="margin-left:9%; margin-top: 1em"><b>httrack
|
|
www.example.com/bob/</b></p>
|
|
|
|
<p style="margin-left:18%;">mirror site
|
|
www.example.com/bob/ and only this site</p>
|
|
|
|
<p style="margin-left:9%;"><b>httrack www.example.com/bob/
|
|
www.anothertest.com/mike/ +*.com/*.jpg <br>
|
|
-mime:application/*</b></p>
|
|
|
|
<p style="margin-left:18%;">mirror the two sites together
|
|
(with shared links) and accept any .jpg files on .com
|
|
sites</p>
|
|
|
|
<p style="margin-left:9%;"><b>httrack
|
|
www.example.com/bob/bobby.html +* -r6 <br>
|
|
httrack www.example.com/bob/bobby.html --spider -P <br>
|
|
proxy.myhost.com:8080 <br>
|
|
httrack --update <br>
|
|
httrack <br>
|
|
httrack --continue</b></p>
|
|
|
|
<h2>OPTIONS
|
|
<a name="OPTIONS"></a>
|
|
</h2>
|
|
|
|
|
|
<h3>General options:
|
|
<a name="General options:"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="3%">
|
|
|
|
|
|
<p style="margin-top: 1em">-O</p></td>
|
|
<td width="6%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">path for mirror/logfiles+cache
|
|
(-O path_mirror[,path_cache_and_logfiles]) (--path
|
|
<param>)</p> </td></tr>
|
|
</table>
|
|
|
|
<h3>Action options:
|
|
<a name="Action options:"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="3%">
|
|
|
|
|
|
<p style="margin-top: 1em">-w</p></td>
|
|
<td width="6%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">*mirror web sites (--mirror)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="3%">
|
|
|
|
|
|
<p>-W</p></td>
|
|
<td width="6%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>mirror web sites, semi-automatic (asks questions)
|
|
(--mirror-wizard)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="3%">
|
|
|
|
|
|
<p>-g</p></td>
|
|
<td width="6%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>just get files (saved in the current directory)
|
|
(--get-files)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="3%">
|
|
|
|
|
|
<p>-i</p></td>
|
|
<td width="6%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>continue an interrupted mirror using the cache
|
|
(--continue)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="3%">
|
|
|
|
|
|
<p>-Y</p></td>
|
|
<td width="6%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>mirror ALL links located in the first level pages
|
|
(mirror links) (--mirrorlinks)</p></td></tr>
|
|
</table>
|
|
|
|
<h3>Proxy options:
|
|
<a name="Proxy options:"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-P</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">proxy use (-P
|
|
[socks5://|connect://][user:pass@]proxy:port) (--proxy
|
|
<param>)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%f</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>*use proxy for ftp (f0 don’t use)
|
|
(--httpproxy-ftp[=N])</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%b</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>use this local hostname to make/send requests (-%b
|
|
hostname) (--bind <param>)</p></td></tr>
|
|
</table>
|
|
|
|
<h3>Limits options:
|
|
<a name="Limits options:"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p style="margin-top: 1em">-rN</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">set the mirror depth to N (*
|
|
r9999) (--depth[=N])</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-%eN</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>set the external links depth to N (* %e0)
|
|
(--ext-depth[=N])</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-mN</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>maximum file length for a non-html file
|
|
(--max-files[=N])</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-mN,N2</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>maximum file length for non html (N) and html (N2)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-MN</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>maximum overall size that can be uploaded/scanned
|
|
(--max-size[=N])</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-EN</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>maximum mirror time in seconds (60=1 minute, 3600=1
|
|
hour) (--max-time[=N])</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-AN</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>maximum transfer rate in bytes/seconds (1000=1KB/s max)
|
|
(--max-rate[=N])</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-%cN</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>maximum number of connections/seconds (*%c5)
|
|
(--connection-per-second[=N])</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-%G</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>random pause of MIN[:MAX] seconds between files (e.g.
|
|
%G5:10) (--pause <param>)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-GN</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>pause transfer if N bytes reached, and wait until lock
|
|
file is deleted (--max-pause[=N])</p></td></tr>
|
|
</table>
|
|
|
|
<h3>Flow control:
|
|
<a name="Flow control:"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-cN</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">number of multiple connections
|
|
(*c4) (--sockets[=N])</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-TN</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>timeout, number of seconds after a non-responding link
|
|
is shutdown; also bounds host name resolution
|
|
(--timeout[=N])</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-RN</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>number of retries, in case of timeout or non-fatal
|
|
errors (*R1) (--retries[=N])</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-JN</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>traffic jam control, minimum transfert rate
|
|
(bytes/seconds) tolerated for a link (--min-rate[=N])</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-HN</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>host is abandoned if: 0=never, 1=timeout, 2=slow,
|
|
3=timeout or slow (--host-control[=N])</p></td></tr>
|
|
</table>
|
|
|
|
<h3>Links options:
|
|
<a name="Links options:"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-%P</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">*extended parsing, attempt to
|
|
parse all links, even in unknown tags or Javascript (%P0
|
|
don’t use) (--extended-parsing[=N])</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-n</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>get non-html files ’near’ an html file (ex:
|
|
an image located outside) (--near)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-t</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>test all URLs (even forbidden ones) (--test)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%L</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p><file> add all URL located in this text file (one
|
|
URL per line) (--list <param>)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%S</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p><file> add all scan rules located in this text
|
|
file (one scan rule per line) (--urllist <param>)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%m</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>seed the crawl from the site’s sitemap (robots.txt
|
|
Sitemap:, then /sitemap.xml); --sitemap-url URL names one
|
|
explicitly. A sitemap you name, or one the site declares, is
|
|
fetched even under robots.txt Disallow; only the guessed
|
|
/sitemap.xml obeys it. The URLs found still pass every
|
|
filter and scope rule (--sitemap)</p></td></tr>
|
|
</table>
|
|
|
|
<h3>Build options:
|
|
<a name="Build options:"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-NN</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">structure type (0 *original
|
|
structure, 1+: see below) (--structure[=N])</p></td></tr>
|
|
</table>
|
|
|
|
<p style="margin-left:18%;">or user defined structure (-N
|
|
"%h%p/%n%q.%t")</p>
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-%N</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">delayed type check, don’t
|
|
make any link test but wait for files download to start
|
|
instead (experimental) (%N0 don’t use, %N1 use for
|
|
unknown extensions, * %N2 always use)
|
|
(--delayed-type-check)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%D</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>cached delayed type check, don’t wait for remote
|
|
type during updates, to speedup them (%D0 wait, * %D1
|
|
don’t wait) (--cached-delayed-type-check)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%M</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>generate a RFC MIME-encapsulated full-archive (.mht)
|
|
(--mime-html)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%Z</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>after the mirror, rewrite each saved page with its
|
|
stylesheets, scripts, images and fonts inlined as data:
|
|
URIs, so any page opens by double-click anywhere (links
|
|
between pages stay relative; audio and video stay links);
|
|
--single-file-max-size N caps each asset (default 10485760
|
|
bytes). %M is the better container where a Chromium-family
|
|
browser is a given: one archive, no base64 tax on text, a
|
|
shared asset stored once (--single-file)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%t</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>keep the original file extension, don’t rewrite it
|
|
from the MIME type (%t0 rewrite)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-LN</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>long names (L1 *long names / L0 8-3 conversion / L2
|
|
ISO9660 compatible) (--long-names[=N])</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-KN</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>keep original links (e.g. http://www.adr/link) (K0
|
|
*relative link, K absolute links, K4 original links, K3
|
|
absolute URI links, K5 transparent proxy link)
|
|
(--keep-links[=N])</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-x</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>replace external html links by error pages
|
|
(--replace-external)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%x</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>do not include any password for external password
|
|
protected websites (%x0 include) (--disable-passwords)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%q</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>*include query string for local files (useless, for
|
|
information purpose only) (%q0 don’t include)
|
|
(--include-query-string)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%g</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>strip query keys for dedup
|
|
([host/pattern=]key1,key2,...) (--strip-query
|
|
<param>)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-o</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>*save the server’s error pages (404..) (o0 discard
|
|
them) (--generate-errors)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-X</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>*purge old files after update (X0 keep delete)
|
|
(--purge-old[=N])</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%p</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>preserve html files ’as is’ (identical to
|
|
’-K4 -%F ""’) (--preserve)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%T</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>links conversion to UTF-8 (--utf8-conversion)</p></td></tr>
|
|
</table>
|
|
|
|
<h3>Spider options:
|
|
<a name="Spider options:"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-bN</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">accept cookies in cookies.txt
|
|
(0=do not accept,* 1=accept) (--cookies[=N])</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%K</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>load extra cookies from a Netscape cookies.txt
|
|
(--cookies-file <param>)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%Y</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>explain which filter rule accepts or rejects a URL, then
|
|
exit (--why <param>)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-u</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>check document type if unknown (cgi,asp..) (u0
|
|
don’t check, * u1 check but /, u2 check always)
|
|
(--check-type[=N])</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-j</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>*parse scripts (j0 don’t parse, bitmask: |1 parse
|
|
default, |4 don’t parse .js |8 don’t be
|
|
aggressive) (--parse-java[=N])</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-sN</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>follow robots.txt and meta robots tags
|
|
(0=never,1=sometimes,* 2=always, 3=always (even strict
|
|
rules)) (--robots[=N])</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%h</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>force HTTP/1.0 requests (reduce update features, only
|
|
for old servers or proxies) (--http-10)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%k</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>use keep-alive if possible, greately reducing latency
|
|
for small files and test requests (%k0 don’t use)
|
|
(--keep-alive)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%z</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>do not request compressed content (%z0 request)
|
|
(--disable-compression)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%B</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>tolerant requests (accept bogus responses on some
|
|
servers, but not standard!) (--tolerant)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%s</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>update hacks: various hacks to limit re-transfers when
|
|
updating (identical size, bogus response..)
|
|
(--updatehack)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%u</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>url hacks: various hacks to limit duplicate URLs (strip
|
|
//, www.foo.com==foo.com..) (--urlhack)</p></td></tr>
|
|
</table>
|
|
|
|
<p style="margin-left:18%;">opt out of one url-hack part:
|
|
--keep-www-prefix (www.foo.com<>foo.com),
|
|
--keep-double-slashes (//), --keep-query-order
|
|
(?b&a)</p>
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-%A</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">assume that a type (cgi,asp..)
|
|
is always linked with a mime type (-%A
|
|
php3,cgi=text/html;dat,bin=application/x-zip) (--assume
|
|
<param>)</p> </td></tr>
|
|
</table>
|
|
|
|
<p style="margin-left:18%;">shortcut: ’--assume
|
|
standard’ is equivalent to -%A php2 php3 php4 php cgi
|
|
asp jsp pl cfm nsf=text/html <br>
|
|
can also be used to force a specific file type: --assume
|
|
foo.cgi=text/html</p>
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="5%">
|
|
|
|
|
|
<p style="margin-top: 1em">-@iN</p></td>
|
|
<td width="4%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">internet protocol (0=both
|
|
ipv6+ipv4, 4=ipv4 only, 6=ipv6 only) (--protocol[=N])</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="5%">
|
|
|
|
|
|
<p>-%w</p></td>
|
|
<td width="4%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>disable a specific external mime module (-%w
|
|
httrack-plugin) (--disable-module <param>)</p></td></tr>
|
|
</table>
|
|
|
|
<h3>Browser ID:
|
|
<a name="Browser ID:"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-F</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">user-agent field sent in HTTP
|
|
headers (-F "user-agent name") (--user-agent
|
|
<param>)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%R</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>default referer field sent in HTTP headers (--referer
|
|
<param>)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%E</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>from email address sent in HTTP headers (--from
|
|
<param>)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%F</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>footer string in Html code (-%F "Mirrored from
|
|
{url} on {date}"; fields {addr} {path} {url} {date}
|
|
{lastmodified} {version} {mime} {charset} {status} {size},
|
|
or legacy %s) (--footer <param>)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%l</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>preferred language (-%l "fr, en, jp, *"
|
|
(--language <param>)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%a</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>accepted formats (-%a
|
|
"text/html,image/png;q=0.9,*/*;q=0.1" (--accept
|
|
<param>)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%X</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>additional HTTP header line (-%X "X-Magic: 42"
|
|
(--headers <param>)</p></td></tr>
|
|
</table>
|
|
|
|
<h3>Log, index, cache
|
|
<a name="Log, index, cache"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-C</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">create/use a cache for updates
|
|
and retries (C0 no cache,C1 cache is prioritary,* C2 test
|
|
update before) (--cache[=N])</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-k</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>store all files in cache (not useful if files on disk)
|
|
(--store-all-in-cache)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%r</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>write an ISO-28500 WARC/1.1 archive; --warc-file NAME
|
|
sets the output name, --warc-max-size N rotates segments
|
|
past N bytes, --warc-cdx also writes a sorted CDXJ index,
|
|
--wacz packages it all as a WACZ file (--warc)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%d</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>write hts-changes.json listing what this crawl left new,
|
|
changed, unchanged and gone compared to the previous mirror
|
|
(--changes)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%n</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>do not re-download locally erased files
|
|
(--do-not-recatch)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%v</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>display on screen filenames downloaded (in realtime) - *
|
|
%v1 short version - %v2 full animation (--display)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-Q</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>no log - quiet mode (--do-not-log)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-q</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>no questions - quiet mode (--quiet)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-z</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>log - extra infos (--extra-log)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-Z</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>log - debug (--debug-log)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-v</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>log on screen (--verbose)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-f</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>*log in files (--file-log)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-f2</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>one single log file (--single-log)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-I</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>*make an index (I0 don’t make) (--index)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%i</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>make a top index for a project folder (* %i0 don’t
|
|
make) (--build-top-index)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%I</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>make an searchable index for this mirror (* %I0
|
|
don’t make) (--search-index)</p></td></tr>
|
|
</table>
|
|
|
|
<h3>Expert options:
|
|
<a name="Expert options:"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-pN</p></td>
|
|
<td width="5%"></td>
|
|
<td width="49%">
|
|
|
|
|
|
<p style="margin-top: 1em">priority mode: (* p3)
|
|
(--priority[=N])</p> </td>
|
|
<td width="33%">
|
|
</td></tr>
|
|
</table>
|
|
|
|
<p style="margin-left:18%;">p0 just scan, don’t save
|
|
anything (for checking links) <br>
|
|
p1 save only html files <br>
|
|
p2 save only non html files <br>
|
|
*p3 save all files <br>
|
|
p7 get html files before, then treat other files</p>
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-S</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">stay on the same directory
|
|
(--stay-on-same-dir)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-D</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>*can only go down into subdirs (--can-go-down)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-U</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>can only go to upper directories (--can-go-up)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-B</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>can both go up&down into the directory structure
|
|
(--can-go-up-and-down)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-a</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>*stay on the same address (--stay-on-same-address)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-d</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>stay on the same principal domain
|
|
(--stay-on-same-domain)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-l</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>stay on the same TLD (eg: .com) (--stay-on-same-tld)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-e</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>go everywhere on the web (--go-everywhere)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%H</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>debug HTTP headers in logfile (--debug-headers)</p></td></tr>
|
|
</table>
|
|
|
|
<h3>Guru options: (do NOT use if possible)
|
|
<a name="Guru options: (do NOT use if possible)"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p style="margin-top: 1em">-#test</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">list engine self-tests (run one
|
|
with -#test=NAME [args])</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-#C</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>cache list (-#C ’*.com/spider*.gif’
|
|
(--debug-cache <param>)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-#R</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>cache repair (damaged cache) (--repair-cache)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-#d</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>debug parser (--debug-parsing)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-#E</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>extract new.zip cache meta-data in meta.zip</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-#f</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>always flush log files (--advanced-flushlogs)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-#FN</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>maximum number of filters
|
|
(--advanced-maxfilters[=N])</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-#h</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>version info (--version)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-#K</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>scan stdin (debug) (--debug-scanstdin)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-#L</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>maximum number of links (-#L1000000)
|
|
(--advanced-maxlinks[=N])</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-#p</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>display ugly progress information
|
|
(--advanced-progressinfo)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-#P</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>catch URL (--catch-url)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-#T</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>generate transfer ops. log every minutes
|
|
(--debug-xfrstats)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-#u</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>wait time (--advanced-wait)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-#Z</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>generate transfer rate statistics every minutes
|
|
(--debug-ratestats)</p> </td></tr>
|
|
</table>
|
|
|
|
<h3>Dangerous options: (do NOT use unless you exactly know what you are doing)
|
|
<a name="Dangerous options: (do NOT use unless you exactly know what you are doing)"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-%!</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">bypass built-in security limits
|
|
aimed to avoid bandwidth abuses (bandwidth, simultaneous
|
|
connections) (--disable-security-limits)</p></td></tr>
|
|
</table>
|
|
|
|
<p style="margin-left:18%;">IMPORTANT NOTE: DANGEROUS
|
|
OPTION, ONLY SUITABLE FOR EXPERTS <br>
|
|
USE IT WITH EXTREME CARE</p>
|
|
|
|
<h3>Command-line specific options:
|
|
<a name="Command-line specific options:"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-V</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">execute system command after
|
|
each files ($0 is the filename: -V "rm \$0")
|
|
(--userdef-cmd <param>)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%W</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>use an external library function as a wrapper (-%W
|
|
myfoo.so[,myparameters]) (--callback <param>)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-y</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>go to background when suspended (y0 don’t)
|
|
(--background-on-suspend)</p> </td></tr>
|
|
</table>
|
|
|
|
<h3>Details: Option N
|
|
<a name="Details: Option N"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p style="margin-top: 1em">-N0</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">Site-structure (default)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N1</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>HTML in web/, images/other files in web/images/</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N2</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>HTML in web/HTML, images/other in web/images</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N3</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>HTML in web/, images/other in web/</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N4</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>HTML in web/, images/other in web/xxx, where xxx is the
|
|
file extension (all gif will be placed onto web/gif, for
|
|
example)</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N5</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>Images/other in web/xxx and HTML in web/HTML</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N99</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>All files in web/, with random names (gadget !)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N100</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>Site-structure, without www.domain.xxx/</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N101</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>Identical to N1 except that "web" is replaced
|
|
by the site’s name</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N102</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>Identical to N2 except that "web" is replaced
|
|
by the site’s name</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N103</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>Identical to N3 except that "web" is replaced
|
|
by the site’s name</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N104</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>Identical to N4 except that "web" is replaced
|
|
by the site’s name</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N105</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>Identical to N5 except that "web" is replaced
|
|
by the site’s name</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N199</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>Identical to N99 except that "web" is replaced
|
|
by the site’s name</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N1001</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>Identical to N1 except that there is no "web"
|
|
directory</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N1002</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>Identical to N2 except that there is no "web"
|
|
directory</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N1003</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>Identical to N3 except that there is no "web"
|
|
directory (option set for g option)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N1004</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>Identical to N4 except that there is no "web"
|
|
directory</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N1005</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>Identical to N5 except that there is no "web"
|
|
directory</p> </td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>-N1099</p></td>
|
|
<td width="1%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>Identical to N99 except that there is no "web"
|
|
directory</p> </td></tr>
|
|
</table>
|
|
|
|
<h3>Details: User-defined option N
|
|
<a name="Details: User-defined option N"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-%n</p></td>
|
|
<td width="5%"></td>
|
|
<td width="63%">
|
|
|
|
|
|
<p style="margin-top: 1em">Name of file without file type
|
|
(ex: image)</p></td>
|
|
<td width="19%">
|
|
</td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%N</p></td>
|
|
<td width="5%"></td>
|
|
<td width="63%">
|
|
|
|
|
|
<p>Name of file, including file type (ex: image.gif)</p></td>
|
|
<td width="19%">
|
|
</td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%t</p></td>
|
|
<td width="5%"></td>
|
|
<td width="63%">
|
|
|
|
|
|
<p>File type (ex: gif)</p></td>
|
|
<td width="19%">
|
|
</td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%p</p></td>
|
|
<td width="5%"></td>
|
|
<td width="63%">
|
|
|
|
|
|
<p>Path [without ending /] (ex: /someimages)</p></td>
|
|
<td width="19%">
|
|
</td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%h</p></td>
|
|
<td width="5%"></td>
|
|
<td width="63%">
|
|
|
|
|
|
<p>Host name (ex: www.example.com)</p></td>
|
|
<td width="19%">
|
|
</td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%M</p></td>
|
|
<td width="5%"></td>
|
|
<td width="63%">
|
|
|
|
|
|
<p>URL MD5 (128 bits, 32 ascii bytes)</p></td>
|
|
<td width="19%">
|
|
</td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%Q</p></td>
|
|
<td width="5%"></td>
|
|
<td width="63%">
|
|
|
|
|
|
<p>query string MD5 (128 bits, 32 ascii bytes)</p></td>
|
|
<td width="19%">
|
|
</td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%k</p></td>
|
|
<td width="5%"></td>
|
|
<td width="63%">
|
|
|
|
|
|
<p>full query string</p></td>
|
|
<td width="19%">
|
|
</td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%r</p></td>
|
|
<td width="5%"></td>
|
|
<td width="63%">
|
|
|
|
|
|
<p>protocol name (ex: http)</p></td>
|
|
<td width="19%">
|
|
</td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-%q</p></td>
|
|
<td width="5%"></td>
|
|
<td width="63%">
|
|
|
|
|
|
<p>small query string MD5 (16 bits, 4 ascii bytes)</p></td>
|
|
<td width="19%">
|
|
</td></tr>
|
|
</table>
|
|
|
|
<p style="margin-left:18%;">’%s?’ Short name
|
|
version (ex: %sN)</p>
|
|
|
|
<p style="margin-left:9%;">-%[param]</p>
|
|
|
|
<p style="margin-left:18%;">param variable in query
|
|
string</p>
|
|
|
|
|
|
<p style="margin-left:9%;">-%[param:before:after:empty:notfound]</p>
|
|
|
|
<p style="margin-left:18%;">advanced variable
|
|
extraction</p>
|
|
|
|
<h3>Details: User-defined option N and advanced variable extraction
|
|
<a name="Details: User-defined option N and advanced variable extraction"></a>
|
|
</h3>
|
|
|
|
|
|
|
|
<p style="margin-left:9%; margin-top: 1em">%[param:before:after:empty:notfound]
|
|
<br>
|
|
param : parameter name <br>
|
|
before : string to prepend if the parameter was found <br>
|
|
after : string to append if the parameter was found <br>
|
|
notfound : string replacement if the parameter could not be
|
|
found <br>
|
|
empty : string replacement if the parameter was empty <br>
|
|
all fields, except the first one (the parameter name), can
|
|
be empty</p>
|
|
|
|
<h3>Details: Option K
|
|
<a name="Details: Option K"></a>
|
|
</h3>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p style="margin-top: 1em">-K0</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">foo.cgi?q=45 ->
|
|
foo4B54.html?q=45 (relative URI, default)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-K</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>-> http://www.foobar.com/folder/foo.cgi?q=45
|
|
(absolute URL) (--keep-links[=N])</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-K3</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>-> /folder/foo.cgi?q=45 (absolute URI)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-K4</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>-> foo.cgi?q=45 (original URL)</p></td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="4%">
|
|
|
|
|
|
<p>-K5</p></td>
|
|
<td width="5%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p>-> http://www.foobar.com/folder/foo4B54.html?q=45
|
|
(transparent proxy URL)</p></td></tr>
|
|
</table>
|
|
|
|
<h3>Shortcuts:
|
|
<a name="Shortcuts:"></a>
|
|
</h3>
|
|
|
|
|
|
<p style="margin-left:9%; margin-top: 1em">--mirror</p>
|
|
|
|
<p style="margin-left:18%;"><URLs> *make a mirror of
|
|
site(s) (default)</p>
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>--get</p></td>
|
|
<td width="1%"></td>
|
|
<td width="78%">
|
|
|
|
|
|
<p><URLs> get the files indicated, do not seek other
|
|
URLs (-qg)</p></td>
|
|
<td width="4%">
|
|
</td></tr>
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="8%">
|
|
|
|
|
|
<p>--list</p></td>
|
|
<td width="1%"></td>
|
|
<td width="78%">
|
|
|
|
|
|
<p><text file> add all URL located in this text file
|
|
(-%L)</p> </td>
|
|
<td width="4%">
|
|
</td></tr>
|
|
</table>
|
|
|
|
<p style="margin-left:9%;">--mirrorlinks</p>
|
|
|
|
<p style="margin-left:18%;"><URLs> mirror all links
|
|
in 1st level pages (-Y)</p>
|
|
|
|
<p style="margin-left:9%;">--testlinks</p>
|
|
|
|
<p style="margin-left:18%;"><URLs> test links in
|
|
pages (-r1p0C0I0t)</p>
|
|
|
|
<p style="margin-left:9%;">--spider</p>
|
|
|
|
<p style="margin-left:18%;"><URLs> spider site(s), to
|
|
test links: reports Errors & Warnings (-p0C0I0t)</p>
|
|
|
|
<p style="margin-left:9%;">--testsite</p>
|
|
|
|
<p style="margin-left:18%;"><URLs> identical to
|
|
--spider</p>
|
|
|
|
<p style="margin-left:9%;">--skeleton</p>
|
|
|
|
<p style="margin-left:18%;"><URLs> make a mirror, but
|
|
gets only html files (-p1)</p>
|
|
|
|
<p style="margin-left:9%;">--update</p>
|
|
|
|
<p style="margin-left:18%;">update a mirror, without
|
|
confirmation (-iC2)</p>
|
|
|
|
<p style="margin-left:9%;">--continue</p>
|
|
|
|
<p style="margin-left:18%;">continue a mirror, without
|
|
confirmation (-iC1)</p>
|
|
|
|
<p style="margin-left:9%;">--catchurl</p>
|
|
|
|
<p style="margin-left:18%;">create a temporary proxy to
|
|
capture an URL or a form post URL</p>
|
|
|
|
<p style="margin-left:9%;">--clean</p>
|
|
|
|
<p style="margin-left:18%;">erase cache & log files</p>
|
|
|
|
<p style="margin-left:9%;">--http10</p>
|
|
|
|
<p style="margin-left:18%;">force http/1.0 requests
|
|
(-%h)</p>
|
|
|
|
<h3>Details: Option %W: External callbacks prototypes
|
|
<a name="Details: Option %W: External callbacks prototypes"></a>
|
|
</h3>
|
|
|
|
|
|
<h3>see htsdefines.h
|
|
<a name="see htsdefines.h"></a>
|
|
</h3>
|
|
|
|
|
|
<h2>FILES
|
|
<a name="FILES"></a>
|
|
</h2>
|
|
|
|
|
|
|
|
<p style="margin-left:9%; margin-top: 1em"><i>/etc/httrack.conf</i></p>
|
|
|
|
<p style="margin-left:18%;">The system wide configuration
|
|
file.</p>
|
|
|
|
<h2>ENVIRONMENT
|
|
<a name="ENVIRONMENT"></a>
|
|
</h2>
|
|
|
|
|
|
<table width="100%" border="0" rules="none" frame="void"
|
|
cellspacing="0" cellpadding="0">
|
|
<tr valign="top" align="left">
|
|
<td width="9%"></td>
|
|
<td width="5%">
|
|
|
|
|
|
<p style="margin-top: 1em">HOME</p></td>
|
|
<td width="4%"></td>
|
|
<td width="82%">
|
|
|
|
|
|
<p style="margin-top: 1em">Is being used if you defined in
|
|
/etc/httrack.conf the line <i>path ˜/websites/#</i></p></td></tr>
|
|
</table>
|
|
|
|
<h2>DIAGNOSTICS
|
|
<a name="DIAGNOSTICS"></a>
|
|
</h2>
|
|
|
|
|
|
<p style="margin-left:9%; margin-top: 1em">Errors/Warnings
|
|
are reported to <i>hts-log.txt</i> by default, or to stderr
|
|
if the <i>-v</i> option was specified.</p>
|
|
|
|
<h2>LIMITS
|
|
<a name="LIMITS"></a>
|
|
</h2>
|
|
|
|
|
|
<p style="margin-left:9%; margin-top: 1em">These are the
|
|
principals limits of HTTrack for that moment. Note that we
|
|
did not heard about any other utility that would have solved
|
|
them. <small>- Several scripts generating complex filenames
|
|
may not find them (ex:
|
|
img.src=’image’+a+Mobj.dst+’.gif’) -
|
|
Cgi-bin links may not work properly in some cases
|
|
(parameters needed). To avoid them: use filters like
|
|
-*cgi-bin*</small></p>
|
|
|
|
<h2>BUGS
|
|
<a name="BUGS"></a>
|
|
</h2>
|
|
|
|
|
|
<p style="margin-left:9%; margin-top: 1em">Please reports
|
|
bugs to <b><bugs@httrack.com>.</b> Include a complete,
|
|
self-contained example that will allow the bug to be
|
|
reproduced, and say which version of httrack you are using.
|
|
Do not forget to detail options used, OS version, and any
|
|
other information you deem necessary.</p>
|
|
|
|
<h2>COPYRIGHT
|
|
<a name="COPYRIGHT"></a>
|
|
</h2>
|
|
|
|
|
|
<p style="margin-left:9%; margin-top: 1em">Copyright (C)
|
|
1998-2026 Xavier Roche and other contributors</p>
|
|
|
|
<p style="margin-left:9%; margin-top: 1em">This program is
|
|
free software: you can redistribute it and/or modify it
|
|
under the terms of the GNU General Public License as
|
|
published by the Free Software Foundation, either version 3
|
|
of the License, or (at your option) any later version.</p>
|
|
|
|
<p style="margin-left:9%; margin-top: 1em">This program is
|
|
distributed in the hope that it will be useful, but WITHOUT
|
|
ANY WARRANTY; without even the implied warranty of
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
GNU General Public License for more details.</p>
|
|
|
|
<p style="margin-left:9%; margin-top: 1em">You should have
|
|
received a copy of the GNU General Public License along with
|
|
this program. If not, see
|
|
<http://www.gnu.org/licenses/>.</p>
|
|
|
|
<h2>AVAILABILITY
|
|
<a name="AVAILABILITY"></a>
|
|
</h2>
|
|
|
|
|
|
<p style="margin-left:9%; margin-top: 1em">The most recent
|
|
released version of httrack can be found at:
|
|
<b>http://www.httrack.com</b></p>
|
|
|
|
<h2>AUTHOR
|
|
<a name="AUTHOR"></a>
|
|
</h2>
|
|
|
|
|
|
<p style="margin-left:9%; margin-top: 1em">Xavier Roche
|
|
<roche@httrack.com></p>
|
|
|
|
<h2>SEE ALSO
|
|
<a name="SEE ALSO"></a>
|
|
</h2>
|
|
|
|
|
|
<p style="margin-left:9%; margin-top: 1em">The <b>HTML</b>
|
|
documentation (available online at
|
|
<b>http://www.httrack.com/html/</b> ) contains more detailed
|
|
information. Please also refer to the <b>httrack FAQ</b>
|
|
(available online at
|
|
<b>http://www.httrack.com/html/faq.html</b> )</p>
|
|
<hr>
|
|
</body>
|
|
</html>
|