15 Commits

Author SHA1 Message Date
Xavier Roche
8d7c9caf61 Release 3.49.17 (#1020)
* Release 3.49.17

Bumps the four version spots, the AppStream release block and VERSION_INFO
(3:8:0 -> 3:9:0, revision only: #991 added an exported symbol and #1018 changed
no POSIX ABI, so the soname stands and there is no Debian package rename).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Cite issues rather than PRs in the release notes, and fill three gaps

House style cites the issue a change closes; five lines named the PR instead.
The pkg-config line cited #978, which is the macOS relocation report and says
nothing about pkg-config.

Adds the command-box escaping fix (#998), the struct addrinfo scoping (#987)
and the four-architecture build failure (#1015), all user-facing and none
covered by a neighbouring line. The crash-report line now names the loader
case it was citing #996 for, and drops #997, which has no user-visible symptom.
AppStream said C:\ab was mangled as C:\folder, but the decode needs two hex
digits after the backslash, so \fo never matched.

VERSION_INFO's rationale named only #991; #1005 exported escape_control_url
too. The bump stays revision-only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 13:05:18 +02:00
Xavier Roche
a7201b5d22 A relocated install still cannot find libhttrack on macOS (#1018)
* A relocated install still cannot find libhttrack on macOS

#964 gave httrack and htsserver a loader-relative rpath but opted Darwin
out: dyld expands @rpath only for a load path spelled that way, and
libtool stamps the dylib with the absolute libdir, so the entry would
never be consulted.

The id comes out of libtool's own archive_cmds template rather than from
LDFLAGS, so configure rewrites that template when the gate is on, and the
dylib links as @rpath/libhttrack.3.dylib from the start. The Darwin probe
now builds a library carrying such an id and runs a program that needs
it, so a dead LC_RPATH cannot pass for a live one; an unrecognised
template leaves the whole thing off.

Anything linked against libhttrack after this records the @rpath id and
must supply an rpath of its own. That is not an soname break, so
VERSION_INFO and the Debian package name are untouched.

Closes #978

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Tighten the Darwin rpath gate and what 219 proves

Arm the install_name rewrite only when both libtool templates take it, so a
future divergence between archive_cmds and archive_expsym_cmds disarms the gate
rather than letting the -export-symbols path stamp an absolute id.

219 now reads the executables' LC_RPATH too, which catches an id moved to @rpath
while the binaries carry no entry answering it, and counts the libhttrack
dependency per binary instead of totalling across both.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Ship a pkg-config file for libhttrack

The @rpath install_name is consumer-visible: anything linking against
libhttrack afterwards records @rpath/libhttrack.3.dylib and needs its own
LC_RPATH. libhttrack.pc is the standard pairing, handing out the include path
and, for a non-system libdir, -Wl,-rpath,${libdir}. The gate is
configure's existing hts_origin_sysdir, so an rpath into /usr/lib stays out.

222_pkgconfig-consumer.test stages an install, queries the staged .pc, then
builds and runs a consumer off it; the run is what proves the id and the rpath
resolve.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Keep the pkg-config file out of the macOS bundle

An .app ships runtime, not development files, and the .pc names the staging
prefix the bundle check rejects.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Trim the comments added with libhttrack.pc

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Let 222 run against a sanitized libhttrack

The consumer is built from the .pc alone, so ASan's runtime is not first in its
library list; the interposer tests already turn that check off.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Make libhttrack.pc link statically, and 222 able to see it

--static --libs handed out -lcrypto before -lssl, with -lz and -lzstd ahead
of the libcrypto that needs them, so an archive link left 76 symbols
undefined. Name the dependencies as pkg-config modules in Requires.private,
which orders them and carries any -L an out-of-the-way prefix needs, falling
back to the probed link line where a dependency ships no .pc. libhttrack-dev
gains the matching -dev dependencies: pkg-config fails outright on a module
whose own .pc is missing.

Test 222 could not see any of this. --define-variable overrides an
assignment the .pc already carries, so its relocation overwrote prefix,
libdir and includedir before anything was asserted: a hard-coded
libdir=${exec_prefix}/lib, which breaks Debian multiarch, passed unchanged.
It also only grepped --static --libs and then linked with the dynamic ones.
Assert the un-relocated values, and build and run a real static consumer.

The rpath the .pc hands out now answers to the same gates the executables'
own rpath respects, so --disable-origin-rpath and an archive-only build stay
rpath-free; 196 covers all six cases. On Darwin 222 skips where configure
left the dylib an absolute install name, and 219 skips on an unset
abs_top_builddir rather than erroring, as 222 already did.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Trim test 222 to the assertions that carry a mutant

The pkg-config consumer test had grown to 225 lines, much of it belt and
braces around checks that already fire elsewhere: the -I/-L extraction and
its stage-relocation loop, the -lz grep on --static --libs, and the rpath
gate check that test 196 now reads straight out of the generated .pc in all
six of its configure cases.

Each of the eight mutants the test was written for still dies: a wrong
libdir, includedir or prefix in the .pc, a multiarch build hard-coding
libdir=${exec_prefix}/lib, the rpath flag dropped from Libs:, the old
Libs.private order with no Requires.private, Cflags one level too high, and
a .pc generated but never installed. The dropped rpath is now caught by the
run itself, since the consumer is launched with LD_LIBRARY_PATH unset
wherever the .pc promises an rpath.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 10:25:32 +00:00
Xavier Roche
77182813a5 A crawled page can read the WebHTTrack session id and drive the control panel (#1011)
* Sandbox mirrored pages so their script cannot reach the control panel

htsserver serves the browsed mirror under /website/ on the same origin as
the panel under /server/, where an unauthenticated GET renders the session
id that gates commands. Send a CSP sandbox without allow-same-origin on
mirrored responses only, so a crawled page's script lands in an opaque
origin.

Closes #877

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Close the blind-POST channel and make the session id unguessable

The CSP sandbox added in this branch narrows #877 but does not close it: a
sandboxed page can still fire a CORS-safelisted no-cors POST at /server/, and
the command runs even though the reply is opaque. Refuse any POST whose Origin
names something other than the panel itself; a missing Origin stays allowed, as
non-browser clients send none.

The session id that authenticates those commands was an md5 of time(NULL) and
rand() seeded from the same second, guessable from the "Mirrored from" stamp
every mirrored page carries. Seed it from /dev/urandom, or RtlGenRandom on
Windows, and refuse to start when neither answers.

Also add allow-downloads to the sandbox token list, without which Chrome blocks
a mirrored .zip or .pdf link.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Cover the origin refusal, the unguessable id and a subdirectory mirror

Test 220 grew a mirrored page one level down and a post-save panel probe: both
a gate keyed on server state and one that only sandboxes mirror-root files
passed the earlier version. It now also drives the Origin refusal end to end,
asserting the refused save binds no mirror. Test 221 starts two servers inside
one second and requires their ids to differ, which the clock-seeded version
cannot do.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Give the two servers distinct ports up front

Two sequential ephemeral-port picks can return the same number; binding both
sockets at once cannot.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Renumber the session-id test clear of 221

221 landed on master as the FTP control-byte crawl test (#1012).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 09:33:55 +00:00
Xavier Roche
e2a57d6c07 WebHTTrack settings pages interpolate values into double-quoted attributes unescaped (#1013)
* WebHTTrack settings pages interpolate values into double-quoted attributes unescaped

#988 escaped the templates' element bodies and event handlers but left
attribute values alone, because cat_html_escaped() keeps the double quote
raw on purpose: the tooltips are written title='${html:LANG_x}'. The
settings pages put about seventy values inside double-quoted attributes,
where a " opens a new attribute.

Add a fourth interpolation mode, attr:, escaping < > & ' " as entities and
nothing else, and point the 230 double-quoted attribute positions at it.
file.html's file:// href stays on the html-urlescaped mode, which now emits
%22 for the quote the way unquoted: already did, and step2.html's in-script
literal moves to js:. Test 185 now requires attr: in a double-quoted
attribute and js: inside a script block; 217 drives hostile values through
three templates and a translation.

Closes #989

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Close the two attribute sites the template sweep could not reach

The liststr: format builds its own <option value="..."> in C, so no
template edit covers it, and it escaped with the helper that leaves the
double quote raw. Its values are the project directory names and the
Category= line of each project's winprofile.ini, both of which can hold
a quote.

The save side kept hex-decoding a backslash plus two hex digits, so a
path typed as C:\ab displayed correctly but was written to ~/.httrack.ini
as C: plus byte 0xAB. The ini writer has no inverse for that escape; the
one caller that needs it is a template literal (${test:proxytype:\3A//}),
so the decode now applies only there.

Test 185 classified attributes with a regex that missed a spaced =, an
unquoted or single-quoted value, a template outside html/server/*.html,
and an output-mode region the file never closes; the counting floors it
asserted also survived a count-preserving rewrite. It now walks tags,
asserts the property on every site it classifies, and proves it can see
each of those forms on a synthetic template first.

Test 217 gains a Latin-1 byte in a translation, the POST-render-POST
round trip, and the two fixes above.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Trim the comments the last commit added

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 08:20:14 +00:00
Xavier Roche
502ce78413 FTP URL bytes reach the control channel unchecked (#1012)
* FTP commands are built from percent-decoded URL bytes with no control-character check

A path or userinfo carrying %0d%0a decodes back to raw CR and LF that reach the
FTP control channel unchecked, so a crawled ftp:// link can append commands of
its own to the ones HTTrack sends.

Screen the command in send_line(), the choke point every command goes through,
the way htsproxy.c already screens the CONNECT authority, and fail the transfer
up front in run_launch_ftp() so the caller reports the error instead of waiting
out a reply to a command that was never sent.

Closes #1010

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Share the control-byte predicate, quote the LIST argument, cover the driver

The check landed as a third copy of a rule htsproxy.c already implements twice
(CONNECT authority, SOCKS5 host); move it to hts_is_control_free() in htslib.c
and call it from all three. The FTP path is now decoded once into ftp_path and
screened there, so SIZE and MDTM stop relying on their argument happening to be
a prefix of the screened RETR line.

"LIST -A %s" pasted the path in bare, so a space in it gave the server a second
token: ftp://host/x%20-la/ went out as LIST -A /x -la/. Both listing and
retrieval now go through ftp_command(), which also quotes a leading '-'.

The self-test drove send_line() directly and never entered run_launch_ftp(),
so deleting the pre-screen passed every assertion while a real crawl stalled on
a 300s read. 221_local-ftp-ctrlchars.test crawls the injected URL against
tests/ftp-server.py and requires a prompt, logged failure; the self-test now
builds each command from its URL path instead of asserting strings by hand.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Dump the mirror when the quoted listing assertion fails

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 07:32:26 +00:00
Xavier Roche
a3e1d3dd40 3.49.16-1 fails to build on four Debian architectures (#1017)
* Fix the four 3.49.16-1 Debian buildd failures

armhf and loong64 cannot unwind out of the frame that faulted on the guard
page, so 180_crash-stack-overflow saw no repeated address and read that as
"never recursed". Judge the recursion only where the report is deeper than
the ordinary-fault control, which keeps the mutant that stopped recursing
failing on x86-64.

hppa needs ~150s per configure run, so 151_bash-shell-validate and
196_install-rpath-gates blew the 600s per-test budget meant to catch a
wedge. They now pace themselves against that budget and skip out rather
than take the build down with them.

hurd-i386 has a network, so the online crawl probe said yes and the crawls
then failed on DNS. Debian Policy 4.9 forbids network access during a build
anyway, so stop asking for it. While there, install crawl-test.sh's traps
one at a time: SIGSTKFLT is Linux-only and its absence made the whole trap
command complain on every Hurd run.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Close two vacuity holes the review found in the new gates

Test 180 compared the stack trace against the segv control's depth, but the
two call chains differ by a frame that only -O2 tail-calls away: at -O0, -O1
and -Og the control is 12 frames and a crash_stack that faults in its own
body is 11, so that mutant slipped through the gate. Use an absolute floor
instead. The handler contributes 3 frames and any non-recursing crash_stack
still unwinds 11, so 6 sits clear of both.

skip_if_out_of_budget projected the step that just ran across every step
left. Test 196 shares one config.cache, so its first step costs several
times the rest and the projection over-estimated by ~2.5x, skipping runs
that fit -- including the hppa case this is for. Look one step ahead
instead, at 1.5x the last step's cost.

Both gates now have coverage in 105_suite-timeout.test: dropping the export
or the skip fails it.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <xroche@gmail.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 07:19:09 +00:00
Xavier Roche
8fcc9d8cae A non-PIE build's crash report still names no frame of the executable (#1014)
* Crash reports from a non-PIE build name no frame of the executable

dladdr() fills dli_fbase with the mapping start and dl_iterate_phdr() fills
dlpi_addr with the relocation bias; the two agree only for a PIE. On an
ET_EXEC the /proc/self/exe override added by #984 never fires, and the
offsets handed to addr2line are the load address too low. Match the main
program by load range, and subtract the bias rather than the base.

Also reject /proc/self/exe when it names the loader rather than the program
(#996), and point the two copies of that resolution at each other (#997).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Reject /proc/self/exe by identity, not by argv[0]'s basename

dladdr() returns argv[0] verbatim for the main program, so the #996 guard
compared /proc/self/exe against a caller-controlled string. Whenever the two
basenames differ and argv[0] carries no slash -- update-alternatives, a Nix or
snap wrapper, a busybox-style dispatcher, a cosmetic argv[0] -- main_path was
dropped, access() then failed on the bare name, and the executable's whole
module vanished from the report.

Reject /proc/self/exe when it stats to a NAMED link-map entry instead: the main
program's entry is the unnamed one, so that catches the loader and nothing else,
and keeps every symlink and rename. Test 218 gains the renamed-symlink launch.

Also reset main_hi with main_lo, and match the mktemp path literally rather than
as an ERE.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 07:00:43 +00:00
Xavier Roche
38eb901cea A worker thread that exhausts its stack still dies with no diagnostic (#991)
* A worker thread that exhausts its stack still dies with no diagnostic

sigaltstack() state is per-thread, so the alternate stack #866 gave the main
thread does nothing for the engine's workers, and the crawl recurses there.
A worker running out of stack leaves the kernel no room for a signal frame,
and it is killed outright with no report.

htsthread gains a pair of hooks the thread start routine runs around the
worker body, and httrack registers the altstack installer and its release.
Releasing matters now that it is per-thread: 64kB of mapping per worker,
never given back, adds up over a crawl.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Cover the release half of the worker alternate-stack hooks

183 could only see the install: crash_stack_thread never returns, so
hts_entry_point never reaches the leave hook and every release-side bug was
structurally invisible. altstackprobe grows an LD_PRELOAD tracer for
sigaltstack(), mmap() and munmap(), and a second leg drives -#test=threadwait,
whose workers do return: 25 stacks installed, 24 handed back, each unmap right
behind its own SS_DISABLE, and the main thread's kept.

Also: hts_set_thread_hooks refuses half a pair, crash_threadstack aborts on a
spawn failure instead of leaving the test to blame the handler, and the -#c
kinds list clips rather than aborts now the table has outgrown its 64 bytes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Gate the alt-stack tracer on Linux: mmap64 and off64_t are glibc-only

The macOS leg builds tests/altstackprobe.c like every other platform, and it
has neither off64_t nor mmap64. SYS_gettid is Linux-only too, so gating just
the glibc half would still not compile there.

The whole tracer now lives behind __linux__, with mmap64 behind __GLIBC__
inside it: musl has no LFS split and its plain mmap() is already the one the
engine calls. Nothing outside the gate changed, so on macOS the file is the
one that was already building, plus four headers it has. The trace is what
183's second leg reads, and that leg is Linux-only by the same reasoning.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 06:12:20 +00:00
Xavier Roche
fbd825d4dd A crawled link can inject header lines into the requests HTTrack sends (#1008)
* Percent-escape network-derived URL pieces in outgoing request headers

A crawled href carrying &#13;&#10; keeps raw CR and LF in the engine's link
buffers: htsparse.c escapes control bytes before hts_unescapeEntities() decodes
the character references, so the escape runs too early to see them. The request
line survived that because it re-escapes 'fil' at emission, but four other
interpolations in http_sendhead() emitted the same data raw, letting a crawled
page add header lines of its choosing to the requests HTTrack then sends.

Escape at emission with escape_check_url(), the same escaper the request line
already uses, so a poisoned URL now goes out percent-encoded in every field
that carries it. The scratch buffer is sized like the url_adr/url_fil buffers
it mirrors, so escaping can only shrink what reaches the request buffer.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>

* Cover the Referer host escape, and state the buffer bound accurately

The proxied leaf pages had no outbound links, so a poisoned host never became
anyone's referer and the Referer host escape went ungraded: reverting it alone
still passed. Give those pages a link so the host reaches a Referer, which also
adds a positive control on the plain proxied host.

The scratch buffer comment claimed escaping only shrinks its input, which is
backwards: percent-escaping expands up to 3x. What holds is that the result is
capped at the source buffer's own size, so the worst case emitted is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>

* Grade the two escapes the probe could not reach, and the bare-LF split

The checker split each captured request on CRLF only, so an injection
that used a bare LF stayed glued inside one line. It reported "none
carrying an injected header" on a wire that carried one; real servers
and proxies do terminate a header line on a bare LF. Split on LF and
strip the CR, the way a tolerant parser reads it.

The proxy leg asserted nothing about the Referer, and no leg reached
the ftp-through-proxy request line at all, so those two escapes were
graded only by the catch-all. A crawl change that stopped following
either link would have un-covered them silently, which is how the
referer host went ungraded the first time round. Give the proxied root
an ftp:// link, and assert that both requests happened and carry the
escaped authority.

The crawl status was discarded, so an assertf abort would have been
graded on whatever bytes it managed to send first. Fail on the deadline
and on death by signal, and add the missing signal trap so the probe
server cannot outlive an interrupted run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Do not name a signal for an exit status that may be a panic

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <xroche@gmail.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 20:09:55 +00:00
Xavier Roche
c36a5428d7 Release notes understate the #841 cache-header overflow (#1009)
The 3.49.16 entries call it an unbounded sprintf and count it as one of "two
ProxyTrack buffer overflows", but #934 bounded that block in the engine's own
cache_add too, where the field caps sum past the 8192-byte stack buffer and the
values come off the wire. Say so, so a reader does not conclude only proxytrack
was exposed.

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 22:04:59 +02:00
Xavier Roche
2945f0a5e4 The WebHTTrack progress panel renders crawled URLs into HTML and JavaScript unescaped (#988)
* Escape crawled URLs in the WebHTTrack progress panel

refresh.html interpolated ${info.state}, ${info.name} and ${info.file} as
raw HTML and ${info.url_sav} into a single-quoted JS literal, so a link
path holding < > or ' ran as script in the panel origin. Point them at
htsserver.c's html: and js: modes, and sweep the other templates for the
same shape. Widen 185's scan to any unescaped handler or element-body
interpolation, and add 186 to probe the rendered panel during a crawl.

Closes #973

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Pin the panel's name column in the escaping test

186 asserted only on the leaf of the crawled path, so info.name could go
back to raw and still pass: 185 cannot see it either, because ${unquoted:}
sits in the directive regex it skips. The link now carries metacharacters
in a directory component too, short enough to survive the engine's 40-char
elision of that column, and the poll breaks on either form so the positive
control stays honest.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Make test 186 executable, like its siblings

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Re-arm the output mode that ${do:end-if} clears

${do:end-if} sets outputmode to 0 rather than restoring what it replaced
(htsserver.c), so the if-not-empty block partway down step4.html's command
textarea drops the ${do:output-mode:html} armed above it, and the 17
${unquoted:} settings below render raw. Posting maxtime or url2 as
A</textarea><img src=x onerror=alert(1)>B breaks out of the textarea.
finished.html's <pre> has the same shape; those two are the only ones.

--sitemap-url moves from html: to arg: like its quoted siblings, since
cat_html_escaped() escapes neither " nor \.

Test 185 had unquoted: on its directive skip list, which is how all 17
sailed past it, and now also rejects an html: value on the command line.
Test 83 reads back the escaped forms at runtime.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 19:42:37 +00:00
Xavier Roche
2312d711e6 A control byte in a link makes the crawler fetch a URL the page never wrote (#1005)
* A vertical tab or form feed in a link is deleted, and the crawler fetches a URL the page never wrote

The link scan treated any byte below 32 as the end of a URL unless is_space()
accepted it, and is_space() counts VT and FF. Neither is among the CR, LF and
TAB stripped from the URL further down, so both survived into
escape_remove_control(), which deletes them: href="/a<VT>bc" resolved to /abc.
The other 28 control bytes ended the scan where they stood, so the crawler
fetched the truncated prefix instead.

A control byte the existing end-of-URL tests did not already consume now costs
the link. CR, LF and TAB keep their exemption because the parser strips them a
few lines later, and the check sits below that end-of-URL logic rather than
above it, so a VT or FF ending an unquoted attribute value still ends it.

Closes #982

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Percent-encode a link's control bytes instead of truncating or deleting them

A control byte inside a link ended the URL where it stood, so
href="/s<SOH>oh.html" fetched /s. VT and FF came out worse: is_space()
counts them, so the scan carried them into the URL, and neither is among
the CR, LF and TAB stripped further down, so escape_remove_control()
deleted them and href="/a<VT>bc" resolved to /abc.

Both now percent-encode as a browser does. The encoding runs on the
extracted link, where the destination capacity is visible, because a byte
grows to three; a link whose encoded form no longer fits is dropped rather
than clipped to a URL nobody wrote.

Closes #982

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Tighten the wording of the encode-site comment

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Assert that a link's CR, LF and TAB are still stripped

Test 158 covered the encoding half of the fix but not the stripping half,
on a line this branch rewrote. Making the strip unconditional, so CR, LF
and TAB percent-encode like every other control byte, left the whole suite
green. The new href carries all three and only its stripped target exists
on the server, so that mutant now fails on the 404 it provokes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Strip a link's end control bytes, and bound the query's growth too

Review of the encoding change found two holes.

A control byte at either end of a link used to end the scan there, which
threw away nothing and gave the right URL; encoding it instead turned
href="page.html<SOH>" into a 404. Both ends now strip every C0 control or
space, as the WHATWG URL parser does, so only interior bytes are encoded.
The leading loop stops on the terminator the trailing loop just wrote,
since '\0' is <= ' ' and it would otherwise read on into the previous
link's leftovers and fetch those.

The path bound missed the query, which is escaped separately and appended
back by a helper that clips silently. An absolute link whose path and
query together outgrow the buffer was landing just under the URL-length
gate and being fetched truncated; it is now dropped like the path case.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 19:37:14 +00:00
Xavier Roche
7bff8133bb Two installed headers declare symbols the library does not export (#990)
* Two installed headers declare symbols the library does not export

htsbasenet.h declares openssl_ctx and htsnet.h declares
hts_dns_set_resolver_backend, both hidden by -fvisibility=hidden and both
installed into $(includedir)/httrack. A consumer that includes either header
and uses the name compiles, then fails to link.

Move both behind HTS_INTERNAL_BYTECODE, the guard the other internal
declarations in the installed set already use. Every in-tree caller defines
it, so the engine build is unaffected.

Test 207 installs the headers, derives the hidden set from the library's
symbol table minus its dynamic table, and links a probe for every hidden name
a consumer can reach through an installed header.

Closes #977

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Test 207's floor against a vacuous run counts only header-static symbols

Every one of the 31 symbols that reaches the link probe is a static function
defined in the installed header itself (abortf_ and the rest of htssafe.h,
StringOom_ from htsstrings.h). The probe object compiles its own copy, so the
link cannot fail whatever the library exports, and "probed >= 5" holds without
the harvest ever seeing an extern declaration. Cutting the header loop down to
htssafe.h alone leaves probed at 7, and the test still passes with 13 of the 14
headers unchecked.

So plant a leak: a canary header declaring the hidden symbol the negative
control has already shown cannot link. The loop has to report it, which puts
the install, the harvest, the intersection and the link on the same path a real
leak takes. The cut-down loop now fails.

The HTTRACK_SHLIB comment also blamed the skip on static-only builds. macOS
skips as well, where libtool names the library .dylib.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Trim test 207's comments to the house one-line default

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Declare struct addrinfo at file scope in htsnet.h

Without it the tag in the resolver-backend prototypes is a fresh type scoped
to its own declaration wherever <netdb.h> has not already declared it.

Closes #987

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 19:10:38 +00:00
Xavier Roche
e5985903e3 The crash report can hang: the symbolizer forks from the signal handler (#985)
* Crash report can hang: the symbolizer forks from the signal handler

fork() runs the pthread_atfork prepare handlers before it forks, and glibc's
malloc registers one that takes every arena lock. A signal raised inside
malloc (glibc's own heap-corruption detector aborts from exactly there, and
SIGABRT is wired to sig_fatal) then leaves the handler blocked on a lock its
own thread holds, and the report is lost.

vfork() bypasses the atfork handlers, at the price of a child that may only
issue syscalls: the PATH search execvp() performs allocates, so the symbolizer
is resolved once at startup and the child execv()s an absolute path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Test 182 pins a 64-bit address width, so it fails the i386 leg

addr2line -a zero-pads the address to the target's pointer size: 8 nibbles
against a 32-bit ELF, 16 against a 64-bit one. The gate is "is addr2line
installed", not the architecture, so the {16} match fails a correct build on
the gcc -m32 leg and on Debian's 32-bit buildds. Test 80 already matches the
same line unanchored.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Spawn the symbolizer with posix_spawnp() instead of fork()

posix_spawnp() reaches the same child through clone(CLONE_VM|CLONE_VFORK),
which runs no pthread_atfork handler, so the arena lock a signal raised inside
malloc already holds cannot deadlock the report away.

It also does the PATH search itself and reports an exec failure through its
return value, so the hand-rolled find_on_path(), the two statics caching its
answer and the BT_NO_SYMBOLIZER exit protocol all go. Building the file actions
in hts_backtrace_init() keeps the only allocation off the crash path, and with
the rewrite happening in the parent the addr2line to llvm-symbolizer fallback
comes back.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Assert the symbolizer output lands on the report fd

The child's stdout redirect is now a file actions object built at startup, so a
wrong target sends symbolized frames to the crawler's own stdout instead. The
existing checks merge both streams and cannot see that.

Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Take the report fd out of hts_print_backtrace()'s signature

The child's stdout redirect is prebuilt at init against stderr, so passing any
other fd quietly skipped symbolization on a runtime check. The one caller
passed stderr, and htsbacktrace.h ships with the program and not with the
installed dev headers, so there is no ABI cost to making the wrong value
impossible to write.

Test 182 also picks up test 80's ARM gate. A build with no unwind tables
prints "unwinding failed" instead of the OS notice, which 182's skip check
misses, so it would have gone red on armhf rather than skipping.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Assert a symbol name in test 182, not just the frame shape

An all-"??" symbolization regression still emits the address lines.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 18:52:04 +00:00
Xavier Roche
ed8a376278 escape_remove_control() leaves the original tail glued to the result (#981)
* escape_remove_control() leaves the original tail glued to the result

It compacted the non-control bytes downward in place but never wrote the
new terminator, so any input it actually shortened came back as the
compacted head plus whatever the old tail left behind: "/a\013bc" came
out "/abcc". The function is HTSEXT_API and declared in the installed
httrack-library.h, so the broken contract is a public one.

The parser can reach it: VT and FF pass the link scan as is_space()
members and the later strip only removes CR, LF and TAB, so a page with
href="/a<VT>bc" fetched /abcc before this.

Closes #974

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* escape-control self-test lets a shifted cut and a stray write through

Grading mutants against the new test found two survivors. No vector held
a space, so moving the loop's cut to `c > 32` stripped it and the suite
stayed green. Nothing reached above 0x0c either, so `c >= 13` passed too.
The added vectors pin both sides of the cut, and the second drops two
bytes instead of one.

The canary read one byte at `inlen + 1`, so a stray write two or more
past the compacted end went unseen. The buffer is already poisoned in
full, so scanning the rest of it costs nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 18:19:30 +00:00
91 changed files with 3675 additions and 513 deletions

1
.gitignore vendored
View File

@@ -26,6 +26,7 @@ Makefile.in
/stamp-h1
# src/webhttrack.in is the source; an in-tree build generates this one (#887).
/src/webhttrack
/src/libhttrack.pc
Makefile
.deps/
.dirstamp

View File

@@ -1,6 +1,6 @@
AC_PREREQ([2.71])
AC_INIT([httrack], [3.49.16], [roche+packaging@httrack.com], [httrack], [http://www.httrack.com/])
AC_INIT([httrack], [3.49.17], [roche+packaging@httrack.com], [httrack], [http://www.httrack.com/])
AC_COPYRIGHT([
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998-2015 Xavier Roche and other contributors
@@ -29,13 +29,15 @@ AC_CONFIG_SRCDIR(src/httrack.c)
AC_CONFIG_MACRO_DIR([m4])
AC_CONFIG_HEADERS(config.h)
AM_INIT_AUTOMAKE([subdir-objects])
# 3:9:0: revision-only bump. #991 and #1005 each added an export
# (hts_set_thread_hooks, escape_control_url); nothing changed or went away.
# 3:8:0: revision-only bump, no ABI change.
# 3:7:0: htsblk gained a tail field and lien_back embeds it by value, so
# lien_back.is_update and everything after it shift +8 (httrackp's own tail growth
# moves nothing). Soname stays .so.3: HTTrackQt is the only consumer of the installed
# headers, so a libhttrack4 rename isn't worth it.
# (3:0:0 was the htsblk mime-buffer widening, the ABI break that moved .so.2 -> .so.3.)
VERSION_INFO="3:8:0"
VERSION_INFO="3:9:0"
AM_MAINTAINER_MODE
AC_USE_SYSTEM_EXTENSIONS
@@ -203,7 +205,7 @@ AX_CHECK_LINK_FLAG([-Wl,--build-id], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,--bu
AX_CHECK_LINK_FLAG([-Wl,-headerpad_max_install_names],
[DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,-headerpad_max_install_names"])
### Relocatable install (#906)
### Relocatable install (#906, #978)
# libtool records the configured libdir as an absolute rpath, so a copied tree
# cannot find libhttrack. A binary-relative rpath fixes that where the loader
# expands the token, which the BSDs disagree on: probe by running one.
@@ -212,11 +214,15 @@ AC_ARG_ENABLE([origin-rpath],
[link the binaries without a binary-relative rpath]),
[hts_origin_asked=yes], [enable_origin_rpath=yes; hts_origin_asked=no])
# hts_origin_ref_make survives make's expansion and libtool's eval of the link line.
hts_origin_ref='$ORIGIN'
hts_origin_ref_make='\$$ORIGIN'
AS_CASE([$host_os], [darwin*], [hts_origin_darwin=yes], [hts_origin_darwin=no])
# hts_origin_ref is the raw token the loader expands; hts_origin_ref_make is the
# spelling that survives make's expansion and libtool's eval of the link line.
AS_CASE([$host_os],
[darwin*], [hts_origin_darwin=yes
hts_origin_ref='@loader_path'
hts_origin_ref_make='@loader_path'],
[hts_origin_darwin=no
hts_origin_ref='$ORIGIN'
hts_origin_ref_make='\$$ORIGIN'])
hts_save_prefix=$prefix
hts_save_exec_prefix=$exec_prefix
@@ -236,14 +242,28 @@ AS_CASE([$hts_libdir],
[/usr/lib | /usr/lib/* | /usr/lib32 | /usr/lib32/* | /usr/lib64 | /usr/lib64/* | /usr/libx32 | /usr/libx32/*], [hts_origin_sysdir=yes],
[hts_origin_sysdir=no])
# dyld follows LC_RPATH only for a load path spelled @rpath/..., so Darwin also
# needs libhttrack's own id. That comes from libtool's archive_cmds template and
# not from LDFLAGS, so rewrite it; an unrecognised template leaves the gate off.
hts_origin_id=no
if test "x$hts_origin_darwin" = xyes; then
hts_origin_subst='s|-install_name $rpath/|-install_name @rpath/|'
hts_origin_archive=`printf '%s\n' "$archive_cmds" | $SED "$hts_origin_subst"`
hts_origin_expsym=`printf '%s\n' "$archive_expsym_cmds" | $SED "$hts_origin_subst"`
# Both templates or neither: should they ever diverge, the -export-symbols
# path would keep stamping the absolute id the rpath cannot answer.
if test "x$hts_origin_archive" != "x$archive_cmds" &&
test "x$hts_origin_expsym" != "x$archive_expsym_cmds"; then
hts_origin_id=yes
fi
fi
ORIGIN_RPATH=no
RPATH_ORIGIN_LDFLAGS=""
if test "x$enable_origin_rpath" = xno; then
AC_MSG_NOTICE([binary-relative rpath disabled by request])
elif test "x$hts_origin_darwin" = xyes; then
# dyld reads LC_RPATH only for an @rpath/ load path, and libtool stamps the
# dylib with an absolute -install_name, so the entry would never be consulted.
AC_MSG_NOTICE([binary-relative rpath not used: libtool records an absolute install_name on Darwin])
elif test "x$hts_origin_darwin" = xyes && test "x$hts_origin_id" = xno; then
AC_MSG_NOTICE([binary-relative rpath not used: unrecognised install_name in libtool archive_cmds])
elif test "x$enable_shared" = xno; then
AC_MSG_NOTICE([binary-relative rpath not needed: no shared libraries])
elif test "x$hts_origin_sysdir" = xyes; then
@@ -258,6 +278,7 @@ else
# Cross compiling cannot run the probe, so take the stricter spelling first.
hts_origin_cands="none origin"
test "x$cross_compiling" = xyes && hts_origin_cands="origin none"
test "x$hts_origin_darwin" = xyes && hts_origin_cands=loader_path
rm -rf conftest.origin
AS_MKDIR_P([conftest.origin/bin])
AS_MKDIR_P([conftest.origin/lib])
@@ -270,7 +291,12 @@ _ACEOF
int hts_origin_probe(void);
int main(void) { return hts_origin_probe() == 42 ? 0 : 1; }
_ACEOF
hts_origin_mklib="$CC $CFLAGS -fPIC -shared -o conftest.origin/lib/libhtsorigin.so conftest.origin/lib.c"
if test "x$hts_origin_darwin" = xyes; then
# The same @rpath id, so a dead LC_RPATH cannot pass for a live one.
hts_origin_mklib="$CC $CFLAGS -dynamiclib -install_name @rpath/libhtsorigin.dylib -o conftest.origin/lib/libhtsorigin.dylib conftest.origin/lib.c"
else
hts_origin_mklib="$CC $CFLAGS -fPIC -shared -o conftest.origin/lib/libhtsorigin.so conftest.origin/lib.c"
fi
hts_origin_link="-Lconftest.origin/lib -lhtsorigin"
if $hts_origin_mklib >&AS_MESSAGE_LOG_FD 2>&1; then
for hts_origin_cand in $hts_origin_cands; do
@@ -282,7 +308,7 @@ _ACEOF
>&AS_MESSAGE_LOG_FD 2>&1 || continue
# From an unrelated cwd, so a loader taking the token literally fails.
if test "x$cross_compiling" = xyes ||
(unset LD_LIBRARY_PATH
(unset LD_LIBRARY_PATH DYLD_LIBRARY_PATH
cd conftest.origin/run && ../bin/probe) >&AS_MESSAGE_LOG_FD 2>&1; then
hts_cv_origin_rpath=$hts_origin_cand
break
@@ -300,6 +326,10 @@ _ACEOF
RPATH_ORIGIN_LDFLAGS='-Wl,-rpath,$(libdir)'
test "x$hts_cv_origin_rpath" = xorigin && RPATH_ORIGIN_LDFLAGS="$RPATH_ORIGIN_LDFLAGS -Wl,-z,origin"
RPATH_ORIGIN_LDFLAGS="$RPATH_ORIGIN_LDFLAGS -Wl,-rpath,$hts_origin_ref_make/../lib"
if test "x$hts_origin_darwin" = xyes; then
archive_cmds=$hts_origin_archive
archive_expsym_cmds=$hts_origin_expsym
fi
ORIGIN_RPATH=yes
AC_MSG_RESULT([yes ($hts_origin_ref/../lib)])
])
@@ -307,6 +337,19 @@ fi
AC_SUBST([ORIGIN_RPATH])
AC_SUBST([RPATH_ORIGIN_LDFLAGS])
# The @rpath id is only safe for a consumer carrying its own LC_RPATH, so
# libhttrack.pc hands one out -- under the gates the executables' own rpath
# respects, so that --disable-origin-rpath and an archive-only build stay
# rpath-free too. ${libdir} is pkg-config's own variable, expanded on query.
PKGCONFIG_RPATH=no
PKGCONFIG_RPATH_LDFLAG=
AS_IF([test "x$enable_origin_rpath" != xno && test "x$enable_shared" != xno &&
test "x$hts_origin_sysdir" = xno],
[PKGCONFIG_RPATH=yes
PKGCONFIG_RPATH_LDFLAG=' -Wl,-rpath,${libdir}'])
AC_SUBST([PKGCONFIG_RPATH])
AC_SUBST([PKGCONFIG_RPATH_LDFLAG])
### Check for -fvisibility=hidden support
gl_VISIBILITY
AM_CFLAGS="$AM_CFLAGS $CFLAG_VISIBILITY"
@@ -393,7 +436,8 @@ AC_CHECK_LIB(crypto, EVP_get_digestbyname,
LIBS="$LIBS $OPENSSL_LIBS"
AC_CHECK_LIB(ssl, SSL_CTX_new,
[
OPENSSL_LIBS="$OPENSSL_LIBS -lssl"
# -lssl first: an archive link needs it ahead of the -lcrypto it uses.
OPENSSL_LIBS="-lssl $OPENSSL_LIBS"
AC_DEFINE(HTS_USEOPENSSL, 1, [Check for OpenSSL])
],
[
@@ -461,6 +505,34 @@ AC_DEFINE(DLLIB, 1,[Check for dlopen in c])], AC_MSG_WARN([*** not found * this
AC_SUBST(DL_LIBS)
fi
### What libhttrack.pc tells a consumer about its private dependencies
# A static link needs each library ahead of the ones it uses, which only a
# dependency's own .pc gets right (and which alone carries the -L an
# out-of-the-way prefix needs), so name the module wherever one is installed.
# Dependents first: pkg-config emits Requires.private after Libs.private.
AC_ARG_VAR([PKG_CONFIG], [path to the pkg-config utility])
AC_PATH_TOOL([PKG_CONFIG], [pkg-config], [false])
PKGCONFIG_REQUIRES_PRIVATE=
PKGCONFIG_LIBS_PRIVATE=
hts_pc_dep() { # hts_pc_dep <module or empty> <link line>
test -n "$2" || return 0
if test -n "$1" && $PKG_CONFIG --exists "$1" 2>/dev/null; then
PKGCONFIG_REQUIRES_PRIVATE="${PKGCONFIG_REQUIRES_PRIVATE:+$PKGCONFIG_REQUIRES_PRIVATE }$1"
else
PKGCONFIG_LIBS_PRIVATE="${PKGCONFIG_LIBS_PRIVATE:+$PKGCONFIG_LIBS_PRIVATE }$2"
fi
}
# Whichever half of OpenSSL was actually found; https=auto can settle for libcrypto.
AS_CASE([" $OPENSSL_LIBS "],
[*" -lssl "*], [hts_pc_openssl=openssl],
[*" -lcrypto "*], [hts_pc_openssl=libcrypto],
[hts_pc_openssl=])
hts_pc_dep "$hts_pc_openssl" "$OPENSSL_LIBS"
hts_pc_dep libbrotlidec "$BROTLI_LIBS"
hts_pc_dep libzstd "$ZSTD_LIBS"
AC_SUBST([PKGCONFIG_REQUIRES_PRIVATE])
AC_SUBST([PKGCONFIG_LIBS_PRIVATE])
### Check for setuid
AC_CHECK_LIB(c, setuid, [
AC_DEFINE(SETUID, 1,[Check for setuid])], AC_MSG_RESULT([not found]))
@@ -528,6 +600,7 @@ AM_CONDITIONAL([FUZZERS], [test x"$fuzzers" = x"yes"])
AC_CONFIG_FILES([
Makefile
src/Makefile
src/libhttrack.pc
man/Makefile
m4/Makefile
templates/Makefile

25
debian/changelog vendored
View File

@@ -1,8 +1,27 @@
httrack (3.49.17-1) unstable; urgency=medium
* New upstream release: security fixes in the FTP and HTTP request paths and
in both WebHTTrack interfaces, where a crawled page could reach the control
panel, plus crash-report and stack-diagnostic fixes; full list in
history.txt. Also fixes the 3.49.16-1 build failures on armhf, loong64 and
hppa, both in the crash-report test's unwinding assumptions and in the
per-test time budget.
* libhttrack-dev ships $(libdir)/pkgconfig/libhttrack.pc, and so now depends
on libhttrack3 and on the -dev packages whose modules that file names:
libssl-dev, libbrotli-dev and libzstd-dev.
* Build with --disable-online-unit-tests. Policy 4.9 forbids network access
during a build, and the previous "auto" probe found one on hurd-i386, whose
crawl tests then failed on DNS.
-- Xavier Roche <xavier@debian.org> Wed, 05 Aug 2026 12:28:23 +0200
httrack (3.49.16-1) unstable; urgency=medium
* New upstream release: two ProxyTrack buffer overflows fixed, chunked
trailer handling, relocatable installs, crash-report backtraces on armhf,
a hicolor icon theme and translation fixes; full list in history.txt.
* New upstream release: a stack overflow in the cache header writer, which a
server's response headers could trigger in httrack and libhttrack3 as well
as in proxytrack, and a ProxyTrack DAV buffer overflow; chunked trailer
handling, relocatable installs, crash-report backtraces on armhf, a hicolor
icon theme and translation fixes; full list in history.txt.
-- Xavier Roche <xavier@debian.org> Tue, 04 Aug 2026 18:32:40 +0200

5
debian/control vendored
View File

@@ -72,7 +72,10 @@ Package: libhttrack-dev
Architecture: any
Multi-Arch: same
Section: libdevel
Depends: ${misc:Depends}, ${shlibs:Depends}, zlib1g-dev
# libhttrack3: what the .so symlink and libhttrack.pc's -lhttrack resolve onto. The
# other -dev packages: libhttrack.pc names their modules in Requires.private, and
# pkg-config fails outright when one of those .pc files is missing.
Depends: ${misc:Depends}, ${shlibs:Depends}, libhttrack3 (= ${binary:Version}), zlib1g-dev, libssl-dev, libbrotli-dev, libzstd-dev
Description: Httrack website copier includes and development files
This package adds supplemental files for using the httrack website copier
library

View File

@@ -1,4 +1,5 @@
usr/include/httrack
usr/lib/*/libhttrack.{so}
usr/lib/*/pkgconfig/libhttrack.pc
usr/share/httrack/libtest
usr/lib/*/httrack/libtest

4
debian/rules vendored
View File

@@ -34,7 +34,9 @@ configure-stamp:
dh_autoreconf
# note: dpkg-buildflags to be removed when compat=9
dh_auto_configure -- --enable-online-unit-tests=auto $(shell dpkg-buildflags --export=configure)
# Policy 4.9 forbids network access during a build, and "auto" made it a lottery:
# hurd-i386 FTBFS'd on a crawl whose DNS went away after the probe had said yes.
dh_auto_configure -- --disable-online-unit-tests $(shell dpkg-buildflags --export=configure)
touch configure-stamp

View File

@@ -4,11 +4,32 @@ HTTrack Website Copier release history:
This file lists all changes and fixes that have been made for HTTrack
3.49-17
+ New: a pkg-config file, libhttrack.pc, ships with the development headers (#1018)
+ Fixed: a crawled page could read the WebHTTrack session id and drive the control panel; the id is now unguessable and cross-origin commands are refused (#877)
+ Fixed: an ftp:// link carrying encoded line breaks could append commands to the FTP control channel, and an unquoted LIST argument let a path holding a space list something else (#1010)
+ Fixed: a crawled link could inject header lines into the requests HTTrack sends (#1008)
+ Fixed: a control byte in a link made the crawler fetch a URL the page never wrote, and stripping one left the original tail glued to the result (#982, #974)
+ Fixed: the WebHTTrack progress panel rendered crawled URLs into HTML and JavaScript unescaped (#973)
+ Fixed: the WebHTTrack command box rendered its settings unescaped, so a posted value could break out of the box it was shown in (#998)
+ Fixed: the WebHTTrack settings pages interpolated values into attributes unescaped, lost a value containing a quote, and mangled a Windows path such as C:\ab (#989)
+ Fixed: a worker thread exhausting its stack died with no diagnostic (#969)
+ Fixed: a crash report named no frame of the executable on a non-PIE build, and named the dynamic loader rather than the program when launched through it (#995, #996)
+ Fixed: the crash report could hang, its symbolizer being forked from the signal handler (#968)
+ Fixed: the installed headers declared symbols the library does not export, and scoped struct addrinfo inside a parameter list (#977, #987)
+ Fixed: an install moved away from its configured prefix could not find libhttrack on macOS (#978)
+ Fixed: 3.49.16 did not build on armhf, loong64, hppa or hurd-i386 (#1015)
+ Changed: mirrored pages are served sandboxed, so a saved page can no longer script the WebHTTrack panel, and loses cookies, storage and cross-frame access (#877)
+ Changed: htsserver exits with a diagnostic rather than fall back to a guessable session id when no entropy source answers (#877)
+ Changed: an FTP file whose name contains a tab is no longer fetched (#1010)
+ Changed: on macOS, libhttrack records @rpath/libhttrack.3.dylib, so anything linked against it afterwards supplies its own rpath (#978)
+ Changed: multiple internal hardening, build, test and CI improvements
3.49-16
+ New: macOS ships a signed and notarized HTTrack.app with its own icon, bundling the OpenSSL it needs so a downloaded copy launches (#890, #900, #901, #950)
+ New: the desktop icons install into the hicolor theme, scalable SVG included, so Icon=httrack resolves in a launcher (#932, #933)
+ Fixed: an unauthenticated PROPFIND overflowed the ProxyTrack DAV item buffer (#836)
+ Fixed: the cached-headers block was built with an unbounded sprintf (#841)
+ Fixed: response headers from the server overflowed the stack buffer holding the cache header block, in both httrack and ProxyTrack (#841)
+ Fixed: a chunked response carrying trailers was rejected as "Invalid chunk" (#855)
+ Fixed: an over-long URL aborted the whole mirror inside the cache instead of reading as a miss (#935, #936)
+ Fixed: ProxyTrack could not re-read the .arc it writes, and crashed on a record whose body it could not read (#834, #929, #931)

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
@@ -115,7 +115,7 @@ ${do:end-if}
<br>
<form action="${thisfile}" name="form">
<form action="${attr:thisfile}" name="form">
<table width="100%">
<tr><td>${LANG_T2}</td><td>http://<input name="urladr"
title='${html:LANG_T10}' onMouseOver="info('${js:LANG_T10}'); return true" onMouseOut="info('&nbsp;'); return true"
@@ -130,7 +130,7 @@ ${do:end-if}
<tr><td>${LANG_T6}:</td><td><input name="urlpass"
title='${html:LANG_T13}' onMouseOver="info('${js:LANG_T13}'); return true" onMouseOut="info('&nbsp;'); return true"
></td></tr>
<tr><td>${LANG_T7}:</td><td><input type="button" value="${LANG_T8}" onClick="alert('not yet implemented!')"
<tr><td>${LANG_T7}:</td><td><input type="button" value="${attr:LANG_T8}" onClick="alert('not yet implemented!')"
title='${html:LANG_T14}' onMouseOver="info('${js:LANG_T14}'); return true" onMouseOut="info('&nbsp;'); return true"
></td></tr>
</table>

View File

@@ -51,6 +51,17 @@
<content_rating type="oars-1.1"/>
<!-- Newest first; tests/01_engine-version-macros.test enforces it. -->
<releases>
<release version="3.49.17" date="2026-08-05">
<description>
<ul>
<li>A mirrored page opened in your browser can no longer take control of the WebHTTrack interface</li>
<li>Web addresses shown while a mirror runs no longer break the progress page</li>
<li>Settings keep values containing quotes or ampersands, and Windows paths such as C:\ab</li>
<li>A malformed link on a crawled site can no longer alter the requests HTTrack sends</li>
<li>Crash reports name the function and source line where the problem occurred</li>
</ul>
</description>
</release>
<release version="3.49.16" date="2026-08-04">
<description>
<ul>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -52,7 +52,7 @@ ${LANG_FATALERR}:
</h2>
<br>
<tt>
${error}
${html:error}
</tt>
<br>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<script language="javascript">
<!--

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -71,7 +71,7 @@ ${do:loadhash}
<form method="GET" action="" name="form">
${LANG_S11b}
<select name="name"
onChange="window.open('file://${path}/' + form.name.value + '/index.html')">
onChange="window.open('file://${js:path}/' + form.name.value + '/index.html')">
<option value="">&nbsp;</option>
${liststr:winprofile}
</select>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -79,7 +79,7 @@ ${do:end-if}
<pre>
${do:if-not-empty:commandReturn}
${do:output-mode:html}${do:if-not-empty:commandReturn}
${LANG_F19}
<font color="red">
${commandReturnMsg}
@@ -89,7 +89,8 @@ ${LANG_F20}
httrack ${commandReturnCmdl}
</font>
${LANG_F21}
${do:end-if}
${/* end-if clears the output mode rather than restoring it, so re-arm it */}
${do:end-if}${do:output-mode:html}
${do:if-empty:commandReturn}
${LANG_F22}
@@ -99,7 +100,7 @@ ${do:end-if}
${LANG_G8} :
${/* an http: page cannot navigate to file:, so the mirror is reached through the server */}
<a href="/website/index.html" target="_new">
${path}/${projname}
${html:path}/${html:projname}
</a></li>
<ul>
<li><a href="/website/index.html" target="_new">${LANG_D8}</a></li>
@@ -107,7 +108,7 @@ ${path}/${projname}
</ul>
<form method="POST" action="exit.html" name="form">
<input type="hidden" name="sid" value="${sid}">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="command" value="quit">
<table border="0" width="100%">
@@ -120,7 +121,7 @@ ${path}/${projname}
<table width="100%" border="0"><tr><td align="left">
&nbsp;
</td><td align="right">
<input type="submit" value=" ${LANG_QUIT} "
<input type="submit" value=" ${attr:LANG_QUIT} "
onClick="form.submit()">
</td></tr></table>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -66,7 +66,7 @@ ${LANG_O16}...
<tr><td class="tabCtrl" align="left">
<a style="background:black;color: white"
href="http://www.httrack.com/update.php3?Product=HTTrack&Version=${HTTRACK_VERSIONID}&VersionStr=${HTTRACK_VERSION}&Platform=${HTS_PLATFORM}&LanguageId=${lang}" target="_new"
href="http://www.httrack.com/update.php3?Product=HTTrack&Version=${attr:HTTRACK_VERSIONID}&VersionStr=${attr:HTTRACK_VERSION}&Platform=${attr:HTS_PLATFORM}&LanguageId=${attr:lang}" target="_new"
title='${html:LANG_O17}' onMouseOver="info('${js:LANG_O17}'); return true" onMouseOut="info('&nbsp;'); return true"
>
${LANG_O17}...

View File

@@ -1,9 +1,9 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>HTTrack Website Copier - Offline Browser</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
@@ -98,7 +98,7 @@ ${LANG_THANKYOU}!
<br><br>
<form method="POST" action="step2.html" name="form">
<input type="hidden" name="sid" value="${sid}">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<table border="0" width="100%">
@@ -112,7 +112,7 @@ ${LANG_THANKYOU}!
<tr><td align="right">
<input name="nextBtn" type="submit" value=" ${LANG_NEXT} >> "
<input name="nextBtn" type="submit" value=" ${attr:LANG_NEXT} >> "
title='${html:LANG_TIPNEXT}' onMouseOver="info('${js:LANG_TIPNEXT}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -93,8 +93,8 @@ ${do:end-if}
<table>
<tr><td>
<form method="POST" action="${thisfile}" name="form">
<input type="hidden" name="sid" value="${sid}">
<form method="POST" action="${attr:thisfile}" name="form">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<input type="hidden" name="closeme" value="">
@@ -123,12 +123,12 @@ ${do:end-if}
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK}"
<input type="submit" value="${attr:LANG_OK}"
onClick="form.closeme.value=1; form.submit(); return false;"
title='${html:LANG_TIPOK}' onMouseOver="info('${js:LANG_TIPOK}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input type="button" value="${LANG_CANCEL}"
<input type="button" value="${attr:LANG_CANCEL}"
onClick="window.close();"
title='${html:LANG_TIPCANCEL}' onMouseOver="info('${js:LANG_TIPCANCEL}'); return true" onMouseOut="info('&nbsp;'); return true"
>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -93,8 +93,8 @@ ${do:end-if}
<table>
<tr><td>
<form method="POST" action="${thisfile}" name="form">
<input type="hidden" name="sid" value="${sid}">
<form method="POST" action="${attr:thisfile}" name="form">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<input type="hidden" name="closeme" value="">
@@ -112,10 +112,10 @@ ${LANG_PROXYTYPE}
<br><br>
${LANG_IOPT10}:
<input name="prox" value="${prox}" size="32"
<input name="prox" value="${attr:prox}" size="32"
title='${html:LANG_G14}' onMouseOver="info('${js:LANG_G14}'); return true" onMouseOut="info('&nbsp;'); return true"
>:
<input name="portprox" value="${portprox}" size="4"
<input name="portprox" value="${attr:portprox}" size="4"
title='${html:LANG_G15}' onMouseOver="info('${js:LANG_G15}'); return true" onMouseOut="info('&nbsp;'); return true"
>
<br>
@@ -128,12 +128,12 @@ ${LANG_IOPT10}:
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK}"
<input type="submit" value="${attr:LANG_OK}"
onClick="form.closeme.value=1; form.submit(); return false;"
title='${html:LANG_TIPOK}' onMouseOver="info('${js:LANG_TIPOK}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input type="button" value="${LANG_CANCEL}"
<input type="button" value="${attr:LANG_CANCEL}"
onClick="window.close();"
title='${html:LANG_TIPCANCEL}' onMouseOver="info('${js:LANG_TIPCANCEL}'); return true" onMouseOut="info('&nbsp;'); return true"
>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -93,8 +93,8 @@ ${do:end-if}
<table>
<tr><td>
<form method="POST" action="${thisfile}" name="form">
<input type="hidden" name="sid" value="${sid}">
<form method="POST" action="${attr:thisfile}" name="form">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<input type="hidden" name="closeme" value="">
@@ -111,97 +111,97 @@ ${LANG_W3}
</td></tr>
<td align="left">
<input name="ext1" value="${ext1}"
<input name="ext1" value="${attr:ext1}"
title='${html:LANG_W4}' onMouseOver="info('${js:LANG_W4}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="left">
&#x21d4;
</td><td align="left">
<input name="mime1" value="${mime1}"
<input name="mime1" value="${attr:mime1}"
title='${html:LANG_W5}' onMouseOver="info('${js:LANG_W5}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
<!-- -->
<td align="left">
<input name="ext2" value="${ext2}"
<input name="ext2" value="${attr:ext2}"
title='${html:LANG_W4}' onMouseOver="info('${js:LANG_W4}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="left">
&#x21d4;
</td><td align="left">
<input name="mime2" value="${mime2}"
<input name="mime2" value="${attr:mime2}"
title='${html:LANG_W5}' onMouseOver="info('${js:LANG_W5}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
<!-- -->
<td align="left">
<input name="ext3" value="${ext3}"
<input name="ext3" value="${attr:ext3}"
title='${html:LANG_W4}' onMouseOver="info('${js:LANG_W4}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="left">
&#x21d4;
</td><td align="left">
<input name="mime3" value="${mime3}"
<input name="mime3" value="${attr:mime3}"
title='${html:LANG_W5}' onMouseOver="info('${js:LANG_W5}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
<!-- -->
<td align="left">
<input name="ext4" value="${ext4}"
<input name="ext4" value="${attr:ext4}"
title='${html:LANG_W4}' onMouseOver="info('${js:LANG_W4}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="left">
&#x21d4;
</td><td align="left">
<input name="mime4" value="${mime4}"
<input name="mime4" value="${attr:mime4}"
title='${html:LANG_W5}' onMouseOver="info('${js:LANG_W5}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
<!-- -->
<td align="left">
<input name="ext5" value="${ext5}"
<input name="ext5" value="${attr:ext5}"
title='${html:LANG_W4}' onMouseOver="info('${js:LANG_W4}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="left">
&#x21d4;
</td><td align="left">
<input name="mime5" value="${mime5}"
<input name="mime5" value="${attr:mime5}"
title='${html:LANG_W5}' onMouseOver="info('${js:LANG_W5}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
<!-- -->
<td align="left">
<input name="ext6" value="${ext6}"
<input name="ext6" value="${attr:ext6}"
title='${html:LANG_W4}' onMouseOver="info('${js:LANG_W4}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="left">
&#x21d4;
</td><td align="left">
<input name="mime6" value="${mime6}"
<input name="mime6" value="${attr:mime6}"
title='${html:LANG_W5}' onMouseOver="info('${js:LANG_W5}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
<!-- -->
<td align="left">
<input name="ext7" value="${ext7}"
<input name="ext7" value="${attr:ext7}"
title='${html:LANG_W4}' onMouseOver="info('${js:LANG_W4}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="left">
&#x21d4;
</td><td align="left">
<input name="mime7" value="${mime7}"
<input name="mime7" value="${attr:mime7}"
title='${html:LANG_W5}' onMouseOver="info('${js:LANG_W5}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
<!-- -->
<td align="left">
<input name="ext8" value="${ext8}"
<input name="ext8" value="${attr:ext8}"
title='${html:LANG_W4}' onMouseOver="info('${js:LANG_W4}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="left">
&#x21d4;
</td><td align="left">
<input name="mime8" value="${mime8}"
<input name="mime8" value="${attr:mime8}"
title='${html:LANG_W5}' onMouseOver="info('${js:LANG_W5}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -215,12 +215,12 @@ ${LANG_W3}
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK}"
<input type="submit" value="${attr:LANG_OK}"
onClick="form.closeme.value=1; form.submit(); return false;"
title='${html:LANG_TIPOK}' onMouseOver="info('${js:LANG_TIPOK}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input type="button" value="${LANG_CANCEL}"
<input type="button" value="${attr:LANG_CANCEL}"
onClick="window.close();"
title='${html:LANG_TIPCANCEL}' onMouseOver="info('${js:LANG_TIPCANCEL}'); return true" onMouseOut="info('&nbsp;'); return true"
>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -93,8 +93,8 @@ ${do:end-if}
<table>
<tr><td>
<form method="POST" action="${thisfile}" name="form">
<input type="hidden" name="sid" value="${sid}">
<form method="POST" action="${attr:thisfile}" name="form">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<input type="hidden" name="closeme" value="">
@@ -113,7 +113,7 @@ ${LANG_I33}
>
${listid:build:LISTDEF_3}
</select>
<input type="button" value="${LANG_O2}"
<input type="button" value="${attr:LANG_O2}"
onClick="form.build.selectedIndex=14; window.open('option2b.html', 'option2b', 'toolbar=no, location=no, directories=no, status=yes, menubar=no, scrollbars=yes, resizable=yes, width=640, height=480').wparent=document; form.submit();"
title='${html:LANG_I4}' onMouseOver="info('${js:LANG_I4}'); return true" onMouseOut="info('&nbsp;'); return true"
>
@@ -148,7 +148,7 @@ ${listid:build:LISTDEF_3}
title='${html:LANG_SINGLEFILETIP}' onMouseOver="info('${js:LANG_SINGLEFILETIP}'); return true" onMouseOut="info('&nbsp;'); return true"
> ${LANG_SINGLEFILE}</td></tr>
<tr><td>${LANG_SINGLEFILEMAX}
<input name="singlefilemax" value="${singlefilemax}" size="12"
<input name="singlefilemax" value="${attr:singlefilemax}" size="12"
title='${html:LANG_SINGLEFILEMAXTIP}' onMouseOver="info('${js:LANG_SINGLEFILEMAXTIP}'); return true" onMouseOut="info('&nbsp;'); return true"
></td></tr>
</table>
@@ -156,12 +156,12 @@ ${listid:build:LISTDEF_3}
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK}"
<input type="submit" value="${attr:LANG_OK}"
onClick="form.closeme.value=1; form.submit(); return false;"
title='${html:LANG_TIPOK}' onMouseOver="info('${js:LANG_TIPOK}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input type="button" value="${LANG_CANCEL}"
<input type="button" value="${attr:LANG_CANCEL}"
onClick="window.close();"
title='${html:LANG_TIPCANCEL}' onMouseOver="info('${js:LANG_TIPCANCEL}'); return true" onMouseOut="info('&nbsp;'); return true"
>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -81,12 +81,12 @@ ${do:end-if}
<table>
<tr><td>
<form method="POST" action="${thisfile}" name="form">
<input type="hidden" name="sid" value="${sid}">
<form method="POST" action="${attr:thisfile}" name="form">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<input type="hidden" name="closeme" value="">
<input name="BuildString" size="60" value="${BuildString}">
<input name="BuildString" size="60" value="${attr:BuildString}">
<br>
@@ -103,14 +103,14 @@ ${LANG_Q3}
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK}"
<input type="submit" value="${attr:LANG_OK}"
${do:output-mode:html-urlescaped}
onClick="if (confirm(str_replace(str_replace('${js:LANG_DIAL7}', '%20', ' '), '%0a', ' '))) { form.closeme.value=1; form.submit(); } return false;"
${do:output-mode:}
title='${html:LANG_TIPOK}' onMouseOver="info('${js:LANG_TIPOK}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input type="button" value="${LANG_CANCEL}"
<input type="button" value="${attr:LANG_CANCEL}"
onClick="window.close();"
title='${html:LANG_TIPCANCEL}' onMouseOver="info('${js:LANG_TIPCANCEL}'); return true" onMouseOut="info('&nbsp;'); return true"
>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -93,8 +93,8 @@ ${do:end-if}
<table>
<tr><td>
<form method="POST" action="${thisfile}" name="form">
<input type="hidden" name="sid" value="${sid}">
<form method="POST" action="${attr:thisfile}" name="form">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<input type="hidden" name="closeme" value="">
@@ -159,12 +159,12 @@ ${listid:travel3:LISTDEF_11}
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK}"
<input type="submit" value="${attr:LANG_OK}"
onClick="form.closeme.value=1; form.submit(); return false;"
title='${html:LANG_TIPOK}' onMouseOver="info('${js:LANG_TIPOK}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input type="button" value="${LANG_CANCEL}"
<input type="button" value="${attr:LANG_CANCEL}"
onClick="window.close();"
title='${html:LANG_TIPCANCEL}' onMouseOver="info('${js:LANG_TIPCANCEL}'); return true" onMouseOut="info('&nbsp;'); return true"
>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -94,7 +94,7 @@ ${do:end-if}
<tr><td>
<form method="POST" action="refresh.html" name="form">
<input type="hidden" name="sid" value="${sid}">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<input type="hidden" name="closeme" value="">
@@ -108,7 +108,7 @@ ${do:end-if}
<tr><td>
${LANG_I44}
</td><td>
<input name="connexion" value="${connexion}" size="4"
<input name="connexion" value="${attr:connexion}" size="4"
title='${html:LANG_I12}' onMouseOver="info('${js:LANG_I12}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr><tr><td></td><td>
@@ -120,7 +120,7 @@ ${LANG_I44}
<tr><td>
${LANG_I47d}
</td><td>
<input name="timeout" value="${timeout}" size="4"
<input name="timeout" value="${attr:timeout}" size="4"
title='${html:LANG_I13}' onMouseOver="info('${js:LANG_I13}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr><tr><td></td><td>
@@ -132,7 +132,7 @@ ${LANG_I47d}
<tr><td>
${LANG_I48}
</td><td>
<input name="retry" value="${retry}" size="4"
<input name="retry" value="${attr:retry}" size="4"
title='${html:LANG_I17}' onMouseOver="info('${js:LANG_I17}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -140,7 +140,7 @@ ${LANG_I48}
<tr><td>
${LANG_I46}
</td><td>
<input name="rate" value="${rate}" size="4"
<input name="rate" value="${attr:rate}" size="4"
title='${html:LANG_I15}' onMouseOver="info('${js:LANG_I15}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr><tr><td></td><td>
@@ -154,12 +154,12 @@ ${LANG_I46}
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK}"
<input type="submit" value="${attr:LANG_OK}"
onClick="form.closeme.value=1; form.submit(); return false;"
title='${html:LANG_TIPOK}' onMouseOver="info('${js:LANG_TIPOK}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input type="button" value="${LANG_CANCEL}"
<input type="button" value="${attr:LANG_CANCEL}"
onClick="window.close();"
title='${html:LANG_TIPCANCEL}' onMouseOver="info('${js:LANG_TIPCANCEL}'); return true" onMouseOut="info('&nbsp;'); return true"
>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -93,8 +93,8 @@ ${do:end-if}
<table>
<tr><td>
<form method="POST" action="${thisfile}" name="form">
<input type="hidden" name="sid" value="${sid}">
<form method="POST" action="${attr:thisfile}" name="form">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<input type="hidden" name="closeme" value="">
@@ -103,7 +103,7 @@ ${do:end-if}
<tr><td>
${LANG_G32}
</td><td>
<input name="depth" value="${depth}" size="4"
<input name="depth" value="${attr:depth}" size="4"
title='${html:LANG_I1g}' onMouseOver="info('${js:LANG_I1g}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -111,7 +111,7 @@ ${LANG_G32}
<tr><td>
${LANG_G32b}
</td><td>
<input name="depth2" value="${depth2}" size="4"
<input name="depth2" value="${attr:depth2}" size="4"
title='${html:LANG_I1g2}' onMouseOver="info('${js:LANG_I1g2}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -119,7 +119,7 @@ ${LANG_G32b}
<tr><td>
${LANG_I50}
</td><td>
<input name="maxhtml" value="${maxhtml}" size="4"
<input name="maxhtml" value="${attr:maxhtml}" size="4"
title='${html:LANG_I18}' onMouseOver="info('${js:LANG_I18}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -127,7 +127,7 @@ ${LANG_I50}
<tr><td>
${LANG_I50b}
</td><td>
<input name="othermax" value="${othermax}" size="4"
<input name="othermax" value="${attr:othermax}" size="4"
title='${html:LANG_I19}' onMouseOver="info('${js:LANG_I19}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -135,7 +135,7 @@ ${LANG_I50b}
<tr><td>
${LANG_I51}
</td><td>
<input name="sizemax" value="${sizemax}" size="4"
<input name="sizemax" value="${attr:sizemax}" size="4"
title='${html:LANG_I20}' onMouseOver="info('${js:LANG_I20}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -143,7 +143,7 @@ ${LANG_I51}
<tr><td>
${LANG_I65}
</td><td>
<input name="pausebytes" value="${pausebytes}" size="4"
<input name="pausebytes" value="${attr:pausebytes}" size="4"
title='${html:LANG_I20b}' onMouseOver="info('${js:LANG_I20b}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -151,7 +151,7 @@ ${LANG_I65}
<tr><td>
${LANG_PAUSEFILES}
</td><td>
<input name="pausefiles" value="${pausefiles}" size="8"
<input name="pausefiles" value="${attr:pausefiles}" size="8"
title='${html:LANG_PAUSEFILESTIP}' onMouseOver="info('${js:LANG_PAUSEFILESTIP}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -159,7 +159,7 @@ ${LANG_PAUSEFILES}
<tr><td>
${LANG_I52}
</td><td>
<input name="maxtime" value="${maxtime}" size="4"
<input name="maxtime" value="${attr:maxtime}" size="4"
title='${html:LANG_I21}' onMouseOver="info('${js:LANG_I21}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -167,7 +167,7 @@ ${LANG_I52}
<tr><td>
${LANG_I54}
</td><td>
<input name="maxrate" value="${maxrate}" size="4"
<input name="maxrate" value="${attr:maxrate}" size="4"
title='${html:LANG_I22}' onMouseOver="info('${js:LANG_I22}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -175,7 +175,7 @@ ${LANG_I54}
<tr><td>
${LANG_I64}
</td><td>
<input name="maxconn" value="${maxconn}" size="4"
<input name="maxconn" value="${attr:maxconn}" size="4"
title='${html:LANG_I22b}' onMouseOver="info('${js:LANG_I22b}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -183,7 +183,7 @@ ${LANG_I64}
<tr><td>
${LANG_I64b}
</td><td>
<input name="maxlinks" value="${maxlinks}" size="4"
<input name="maxlinks" value="${attr:maxlinks}" size="4"
title='${html:LANG_I22c}' onMouseOver="info('${js:LANG_I22c}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -191,12 +191,12 @@ ${LANG_I64b}
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK}"
<input type="submit" value="${attr:LANG_OK}"
onClick="form.closeme.value=1; form.submit(); return false;"
title='${html:LANG_TIPOK}' onMouseOver="info('${js:LANG_TIPOK}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input type="button" value="${LANG_CANCEL}"
<input type="button" value="${attr:LANG_CANCEL}"
onClick="window.close();"
title='${html:LANG_TIPCANCEL}' onMouseOver="info('${js:LANG_TIPCANCEL}'); return true" onMouseOut="info('&nbsp;'); return true"
>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -93,8 +93,8 @@ ${do:end-if}
<table>
<tr><td>
<form method="POST" action="${thisfile}" name="form">
<input type="hidden" name="sid" value="${sid}">
<form method="POST" action="${attr:thisfile}" name="form">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<input type="hidden" name="closeme" value="">
@@ -103,7 +103,7 @@ ${do:end-if}
<tr><td>
${LANG_I43}
</td><td>
<input name="user" value="${user}" size="60"
<input name="user" value="${attr:user}" size="60"
title='${html:LANG_I23}' onMouseOver="info('${js:LANG_I23}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -111,7 +111,7 @@ ${LANG_I43}
<tr><td>
${LANG_I43b}
</td><td>
<input name="footer" value="${footer}" size="60"
<input name="footer" value="${attr:footer}" size="60"
title='${html:LANG_I23b}' onMouseOver="info('${js:LANG_I23b}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr>
@@ -121,12 +121,12 @@ ${LANG_I43b}
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK}"
<input type="submit" value="${attr:LANG_OK}"
onClick="form.closeme.value=1; form.submit(); return false;"
title='${html:LANG_TIPOK}' onMouseOver="info('${js:LANG_TIPOK}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input type="button" value="${LANG_CANCEL}"
<input type="button" value="${attr:LANG_CANCEL}"
onClick="window.close();"
title='${html:LANG_TIPCANCEL}' onMouseOver="info('${js:LANG_TIPCANCEL}'); return true" onMouseOut="info('&nbsp;'); return true"
>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -93,8 +93,8 @@ ${do:end-if}
<table>
<tr><td>
<form method="POST" action="${thisfile}" name="form">
<input type="hidden" name="sid" value="${sid}">
<form method="POST" action="${attr:thisfile}" name="form">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<input type="hidden" name="closeme" value="">
@@ -103,7 +103,7 @@ ${LANG_B10}
</tt>
<textarea name="url2" cols="60" rows="8"
title='${html:LANG_C3}' onMouseOver="info('${js:LANG_C3}'); return true" onMouseOut="info('&nbsp;'); return true"
>${url2}</textarea>
>${do:output-mode:html}${url2}${do:output-mode:}</textarea>
<br>
<tt>
@@ -113,12 +113,12 @@ ${LANG_B13}
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK}"
<input type="submit" value="${attr:LANG_OK}"
onClick="form.closeme.value=1; form.submit(); return false;"
title='${html:LANG_TIPOK}' onMouseOver="info('${js:LANG_TIPOK}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input type="button" value="${LANG_CANCEL}"
<input type="button" value="${attr:LANG_CANCEL}"
onClick="window.close();"
title='${html:LANG_TIPCANCEL}' onMouseOver="info('${js:LANG_TIPCANCEL}'); return true" onMouseOut="info('&nbsp;'); return true"
>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -93,8 +93,8 @@ ${do:end-if}
<table>
<tr><td>
<form method="POST" action="${thisfile}" name="form">
<input type="hidden" name="sid" value="${sid}">
<form method="POST" action="${attr:thisfile}" name="form">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<input type="hidden" name="closeme" value="">
@@ -116,7 +116,7 @@ ${do:end-if}
<br><br>
${LANG_COOKIEFILE}
<input name="cookiesfile" value="${cookiesfile}" size="40"
<input name="cookiesfile" value="${attr:cookiesfile}" size="40"
title='${html:LANG_COOKIEFILETIP}' onMouseOver="info('${js:LANG_COOKIEFILETIP}'); return true" onMouseOut="info('&nbsp;'); return true"
>
<br><br>
@@ -150,7 +150,7 @@ ${listid:robots:LISTDEF_8}
<br><br>
${LANG_SITEMAPURL}
<input name="sitemapurl" value="${sitemapurl}" size="40"
<input name="sitemapurl" value="${attr:sitemapurl}" size="40"
title='${html:LANG_SITEMAPURLTIP}' onMouseOver="info('${js:LANG_SITEMAPURLTIP}'); return true" onMouseOut="info('&nbsp;'); return true"
>
<br><br>
@@ -181,7 +181,7 @@ ${LANG_SITEMAPURL}
<br><br>
${LANG_STRIPQUERY}
<input name="stripquery" value="${stripquery}" size="40"
<input name="stripquery" value="${attr:stripquery}" size="40"
title='${html:LANG_STRIPQUERYTIP}' onMouseOver="info('${js:LANG_STRIPQUERYTIP}'); return true" onMouseOut="info('&nbsp;'); return true"
>
<br><br>
@@ -200,12 +200,12 @@ ${LANG_STRIPQUERY}
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK}"
<input type="submit" value="${attr:LANG_OK}"
onClick="form.closeme.value=1; form.submit(); return false;"
title='${html:LANG_TIPOK}' onMouseOver="info('${js:LANG_TIPOK}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input type="button" value="${LANG_CANCEL}"
<input type="button" value="${attr:LANG_CANCEL}"
onClick="window.close();"
title='${html:LANG_TIPCANCEL}' onMouseOver="info('${js:LANG_TIPCANCEL}'); return true" onMouseOut="info('&nbsp;'); return true"
>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -93,8 +93,8 @@ ${do:end-if}
<table>
<tr><td>
<form method="POST" action="${thisfile}" name="form">
<input type="hidden" name="sid" value="${sid}">
<form method="POST" action="${attr:thisfile}" name="form">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<input type="hidden" name="closeme" value="">
@@ -119,13 +119,13 @@ ${do:end-if}
<br><br>
${LANG_WARCFILE}
<input name="warcfile" value="${warcfile}" size="40"
<input name="warcfile" value="${attr:warcfile}" size="40"
title='${html:LANG_WARCFILETIP}' onMouseOver="info('${js:LANG_WARCFILETIP}'); return true" onMouseOut="info('&nbsp;'); return true"
>
<br><br>
${LANG_WARCMAXSIZE}
<input name="warcmaxsize" value="${warcmaxsize}" size="12"
<input name="warcmaxsize" value="${attr:warcmaxsize}" size="12"
title='${html:LANG_WARCMAXSIZETIP}' onMouseOver="info('${js:LANG_WARCMAXSIZETIP}'); return true" onMouseOut="info('&nbsp;'); return true"
>
<br><br>
@@ -173,12 +173,12 @@ ${listid:logtype:LISTDEF_9}
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK}"
<input type="submit" value="${attr:LANG_OK}"
onClick="form.closeme.value=1; form.submit(); return false;"
title='${html:LANG_TIPOK}' onMouseOver="info('${js:LANG_TIPOK}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input type="button" value="${LANG_CANCEL}"
<input type="button" value="${attr:LANG_CANCEL}"
onClick="window.close();"
title='${html:LANG_TIPCANCEL}' onMouseOver="info('${js:LANG_TIPCANCEL}'); return true" onMouseOut="info('&nbsp;'); return true"
>

View File

@@ -1,11 +1,11 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<META http-equiv="refresh" content="30; URL=finished.html">
<title>'${projname}' - HTTrack Website Copier</title>
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -116,17 +116,17 @@ ${do:end-if}
<table border="0" width="100%">
<tr>
<td>${LANG_H8}</td><td>${info.stat_bytes}</td>
<td>${LANG_H9}</td><td>${info.lien_n}/${info.lien_tot} (+${info.stat_back})</td>
<td>${LANG_H8}</td><td>${html:info.stat_bytes}</td>
<td>${LANG_H9}</td><td>${html:info.lien_n}/${html:info.lien_tot} (+${html:info.stat_back})</td>
</tr><tr>
<td>${LANG_H10}</td><td>${info.stat_time_str}</td>
<td>${LANG_H17}</td><td>${info.stat_written}</td>
<td>${LANG_H10}</td><td>${html:info.stat_time_str}</td>
<td>${LANG_H17}</td><td>${html:info.stat_written}</td>
</tr><tr>
<td>${LANG_H14}</td><td>${info.irate} (${info.rate})</td>
<td>${LANG_H18}</td><td>${info.stat_updated}</td>
<td>${LANG_H14}</td><td>${html:info.irate} (${html:info.rate})</td>
<td>${LANG_H18}</td><td>${html:info.stat_updated}</td>
</tr><tr>
<td>${LANG_H11}</td><td>${info.stat_nsocket}</td>
<td>${LANG_H19}</td><td>${info.stat_errors}</td>
<td>${LANG_H11}</td><td>${html:info.stat_nsocket}</td>
<td>${LANG_H19}</td><td>${html:info.stat_errors}</td>
</tr>
</tr>
@@ -138,7 +138,7 @@ ${do:end-if}
<tr>
<td>
${LANG_H20} ${info.currentjob}
${LANG_H20} ${html:info.currentjob}
</td>
</tr>
@@ -147,25 +147,25 @@ ${LANG_H20} ${info.currentjob}
<br>
<form method="POST" action="step4.html" name="form">
<input type="hidden" name="sid" value="${sid}">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="command" value="">
<table border="0" width="100%">
<tr><td>${info.state[0]}</td><td>${info.name[0]}</td><td>${info.file[0]}</td><td>${info.size[0]}/${info.sizetot[0]}</td><td><input type="submit" value="${LANG_H15}" onClick="form.command.value='cancel-file=${info.url_sav[0]}'; form.submit()" title='${html:LANG_H6}' onMouseOver="info('${js:LANG_H6}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${info.state[1]}</td><td>${info.name[1]}</td><td>${info.file[1]}</td><td>${info.size[1]}/${info.sizetot[1]}</td><td><input type="submit" value="${LANG_H15}" onClick="form.command.value='cancel-file=${info.url_sav[1]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${info.state[2]}</td><td>${info.name[2]}</td><td>${info.file[2]}</td><td>${info.size[2]}/${info.sizetot[2]}</td><td><input type="submit" value="${LANG_H15}" onClick="form.command.value='cancel-file=${info.url_sav[2]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${info.state[3]}</td><td>${info.name[3]}</td><td>${info.file[3]}</td><td>${info.size[3]}/${info.sizetot[3]}</td><td><input type="submit" value="${LANG_H15}" onClick="form.command.value='cancel-file=${info.url_sav[3]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${info.state[4]}</td><td>${info.name[4]}</td><td>${info.file[4]}</td><td>${info.size[4]}/${info.sizetot[4]}</td><td><input type="submit" value="${LANG_H15}" onClick="form.command.value='cancel-file=${info.url_sav[4]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${info.state[5]}</td><td>${info.name[5]}</td><td>${info.file[5]}</td><td>${info.size[5]}/${info.sizetot[5]}</td><td><input type="submit" value="${LANG_H15}" onClick="form.command.value='cancel-file=${info.url_sav[5]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${info.state[6]}</td><td>${info.name[6]}</td><td>${info.file[6]}</td><td>${info.size[6]}/${info.sizetot[6]}</td><td><input type="submit" value="${LANG_H15}" onClick="form.command.value='cancel-file=${info.url_sav[6]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${info.state[7]}</td><td>${info.name[7]}</td><td>${info.file[7]}</td><td>${info.size[7]}/${info.sizetot[7]}</td><td><input type="submit" value="${LANG_H15}" onClick="form.command.value='cancel-file=${info.url_sav[7]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${info.state[8]}</td><td>${info.name[8]}</td><td>${info.file[8]}</td><td>${info.size[8]}/${info.sizetot[8]}</td><td><input type="submit" value="${LANG_H15}" onClick="form.command.value='cancel-file=${info.url_sav[8]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${info.state[9]}</td><td>${info.name[9]}</td><td>${info.file[9]}</td><td>${info.size[9]}/${info.sizetot[9]}</td><td><input type="submit" value="${LANG_H15}" onClick="form.command.value='cancel-file=${info.url_sav[9]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${info.state[10]}</td><td>${info.name[10]}</td><td>${info.file[10]}</td><td>${info.size[10]}/${info.sizetot[10]}</td><td><input type="submit" value="${LANG_H15}" onClick="form.command.value='cancel-file=${info.url_sav[10]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${info.state[11]}</td><td>${info.name[11]}</td><td>${info.file[11]}</td><td>${info.size[11]}/${info.sizetot[11]}</td><td><input type="submit" value="${LANG_H15}" onClick="form.command.value='cancel-file=${info.url_sav[11]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${info.state[12]}</td><td>${info.name[12]}</td><td>${info.file[12]}</td><td>${info.size[12]}/${info.sizetot[12]}</td><td><input type="submit" value="${LANG_H15}" onClick="form.command.value='cancel-file=${info.url_sav[12]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${info.state[13]}</td><td>${info.name[13]}</td><td>${info.file[13]}</td><td>${info.size[13]}/${info.sizetot[13]}</td><td><input type="submit" value="${LANG_H15}" onClick="form.command.value='cancel-file=${info.url_sav[13]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${html:info.state[0]}</td><td>${html:info.name[0]}</td><td>${html:info.file[0]}</td><td>${html:info.size[0]}/${html:info.sizetot[0]}</td><td><input type="submit" value="${attr:LANG_H15}" onClick="form.command.value='cancel-file=${js:info.url_sav[0]}'; form.submit()" title='${html:LANG_H6}' onMouseOver="info('${js:LANG_H6}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${html:info.state[1]}</td><td>${html:info.name[1]}</td><td>${html:info.file[1]}</td><td>${html:info.size[1]}/${html:info.sizetot[1]}</td><td><input type="submit" value="${attr:LANG_H15}" onClick="form.command.value='cancel-file=${js:info.url_sav[1]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${html:info.state[2]}</td><td>${html:info.name[2]}</td><td>${html:info.file[2]}</td><td>${html:info.size[2]}/${html:info.sizetot[2]}</td><td><input type="submit" value="${attr:LANG_H15}" onClick="form.command.value='cancel-file=${js:info.url_sav[2]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${html:info.state[3]}</td><td>${html:info.name[3]}</td><td>${html:info.file[3]}</td><td>${html:info.size[3]}/${html:info.sizetot[3]}</td><td><input type="submit" value="${attr:LANG_H15}" onClick="form.command.value='cancel-file=${js:info.url_sav[3]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${html:info.state[4]}</td><td>${html:info.name[4]}</td><td>${html:info.file[4]}</td><td>${html:info.size[4]}/${html:info.sizetot[4]}</td><td><input type="submit" value="${attr:LANG_H15}" onClick="form.command.value='cancel-file=${js:info.url_sav[4]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${html:info.state[5]}</td><td>${html:info.name[5]}</td><td>${html:info.file[5]}</td><td>${html:info.size[5]}/${html:info.sizetot[5]}</td><td><input type="submit" value="${attr:LANG_H15}" onClick="form.command.value='cancel-file=${js:info.url_sav[5]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${html:info.state[6]}</td><td>${html:info.name[6]}</td><td>${html:info.file[6]}</td><td>${html:info.size[6]}/${html:info.sizetot[6]}</td><td><input type="submit" value="${attr:LANG_H15}" onClick="form.command.value='cancel-file=${js:info.url_sav[6]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${html:info.state[7]}</td><td>${html:info.name[7]}</td><td>${html:info.file[7]}</td><td>${html:info.size[7]}/${html:info.sizetot[7]}</td><td><input type="submit" value="${attr:LANG_H15}" onClick="form.command.value='cancel-file=${js:info.url_sav[7]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${html:info.state[8]}</td><td>${html:info.name[8]}</td><td>${html:info.file[8]}</td><td>${html:info.size[8]}/${html:info.sizetot[8]}</td><td><input type="submit" value="${attr:LANG_H15}" onClick="form.command.value='cancel-file=${js:info.url_sav[8]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${html:info.state[9]}</td><td>${html:info.name[9]}</td><td>${html:info.file[9]}</td><td>${html:info.size[9]}/${html:info.sizetot[9]}</td><td><input type="submit" value="${attr:LANG_H15}" onClick="form.command.value='cancel-file=${js:info.url_sav[9]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${html:info.state[10]}</td><td>${html:info.name[10]}</td><td>${html:info.file[10]}</td><td>${html:info.size[10]}/${html:info.sizetot[10]}</td><td><input type="submit" value="${attr:LANG_H15}" onClick="form.command.value='cancel-file=${js:info.url_sav[10]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${html:info.state[11]}</td><td>${html:info.name[11]}</td><td>${html:info.file[11]}</td><td>${html:info.size[11]}/${html:info.sizetot[11]}</td><td><input type="submit" value="${attr:LANG_H15}" onClick="form.command.value='cancel-file=${js:info.url_sav[11]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${html:info.state[12]}</td><td>${html:info.name[12]}</td><td>${html:info.file[12]}</td><td>${html:info.size[12]}/${html:info.sizetot[12]}</td><td><input type="submit" value="${attr:LANG_H15}" onClick="form.command.value='cancel-file=${js:info.url_sav[12]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
<tr><td>${html:info.state[13]}</td><td>${html:info.name[13]}</td><td>${html:info.file[13]}</td><td>${html:info.size[13]}/${html:info.sizetot[13]}</td><td><input type="submit" value="${attr:LANG_H15}" onClick="form.command.value='cancel-file=${js:info.url_sav[13]}'; form.submit()" title='${html:LANG_H7}' onMouseOver="info('${js:LANG_H7}'); return true" onMouseOut="info('&nbsp;'); return true"></td></tr>
</table>
@@ -179,7 +179,7 @@ ${LANG_H20} ${info.currentjob}
<table width="100%" border="0"><tr><td align="left">
&nbsp;
</td><td align="right">
<input type="submit" value=" ${LANG_V4} "
<input type="submit" value=" ${attr:LANG_V4} "
title='${html:LANG_D3}' onMouseOver="disable_timer(); info('${js:LANG_D3}'); return true"
onMouseOut="info('&nbsp;'); enable_timer(); return true"
onClick="if (do_confirm('${js:LANG_G1}')) { form.command.value='cancel'; form.submit(); } return false"

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -28,7 +28,7 @@ function checkname() {
if (form.projname.value != '') {
return true;
}
window.status='${LANG_S1}';
window.status='${js:LANG_S1}';
form.projname.select();
return false;
}
@@ -202,7 +202,7 @@ ${/* End convert winprofile.ini into internal variables */}
<br>
<form method="POST" action="step3.html" name="form">
<input type="hidden" name="sid" value="${sid}">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<input type="hidden" name="todo" value="1">
${do:if-project-file-exists:/hts-cache/winprofile.ini}
@@ -227,7 +227,7 @@ ${do:loadhash}
<br>
${LANG_S11}
<input name="projname" value="${projname}"
<input name="projname" value="${attr:projname}"
title='${html:LANG_S1}' onMouseOver="info('${js:LANG_S1}'); return true" onMouseOut="info('&nbsp;'); return true"
>
@@ -242,7 +242,7 @@ ${do:loadhash}
${liststr:wincateg}
</select>
</td><td>
<input name="projcateg" value="${projcateg}"
<input name="projcateg" value="${attr:projcateg}"
title='${html:LANG_S5}' onMouseOver="info('${js:LANG_S5}'); return true" onMouseOut="info('&nbsp;'); return true"
>
@@ -250,7 +250,7 @@ ${do:loadhash}
<br>
${LANG_S12}
<input name="path" value="${path}"
<input name="path" value="${attr:path}"
title='${html:LANG_S2}' onMouseOver="info('${js:LANG_S2}'); return true" onMouseOut="info('&nbsp;'); return true"
>
<input type="button" value="refresh" onClick="form.redirect.value='step2.html'; form.submit()">
@@ -259,16 +259,16 @@ ${do:loadhash}
<tr><td>
<table width="100%" border="0"><tr><td align="left">
<input type="submit" value=" << ${LANG_PREVIOUS} " onClick="form.redirect.value='index.html'; form.submit()"
<input type="submit" value=" << ${attr:LANG_PREVIOUS} " onClick="form.redirect.value='index.html'; form.submit()"
title='${html:LANG_TIPPREV}' onMouseOver="info('${js:LANG_TIPPREV}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input name="nextBtn" type="submit" value=" ${LANG_NEXT} >> " onClick="return checkname();" default
<input name="nextBtn" type="submit" value=" ${attr:LANG_NEXT} >> " onClick="return checkname();" default
title='${html:LANG_TIPNEXT}' onMouseOver="info('${js:LANG_TIPNEXT}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr></table>
${do:if-not-empty:urls}
${do:output-mode:html}${do:if-not-empty:urls}
<br>
<h2>${LANG_URLS}:</h2><br>
<h3><pre>${urls}</pre></h3>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -111,7 +111,7 @@ ${do:end-if}
<br>
<form method="POST" action="step4.html" name="form">
<input type="hidden" name="sid" value="${sid}">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<table border="0" width="100%">
@@ -129,7 +129,7 @@ ${do:end-if}
<tr><td>
${LANG_G44}
</td><td>
<input type="button" value="${LANG_G43}"
<input type="button" value="${attr:LANG_G43}"
onClick="doOpenWindow()"
title='${html:LANG_G24b}' onMouseOver="info('${js:LANG_G24b}'); return true" onMouseOut="info('&nbsp;'); return true"
>
@@ -149,7 +149,7 @@ ${do:output-mode:}
<tr><td>
${LANG_URLLIST}:
</td><td>
<input name="filelist" value="${filelist}" size="40"
<input name="filelist" value="${attr:filelist}" size="40"
title='${html:LANG_G24c}' onMouseOver="info('${js:LANG_G24c}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr></table>
@@ -160,7 +160,7 @@ ${do:output-mode:}
<tr><td>
${LANG_G41}
</td><td>
<input type="button" value="${LANG_G40}" onClick="window.open('option1.html', 'option1', 'toolbar=no, location=no, directories=no, status=yes, menubar=no, scrollbars=yes, resizable=yes, width=640, height=480'); return false"
<input type="button" value="${attr:LANG_G40}" onClick="window.open('option1.html', 'option1', 'toolbar=no, location=no, directories=no, status=yes, menubar=no, scrollbars=yes, resizable=yes, width=640, height=480'); return false"
title='${html:LANG_G24}' onMouseOver="info('${js:LANG_G24}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr></table>
@@ -169,11 +169,11 @@ ${do:output-mode:}
<tr><td align="right">
<table width="100%" border="0"><tr><td align="left">
<input type="submit" value=" << ${LANG_PREVIOUS} " onClick="form.redirect.value='step2.html'; form.submit()"
<input type="submit" value=" << ${attr:LANG_PREVIOUS} " onClick="form.redirect.value='step2.html'; form.submit()"
title='${html:LANG_TIPPREV}' onMouseOver="info('${js:LANG_TIPPREV}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input name="nextBtn" type="submit" value=" ${LANG_NEXT} >> "
<input name="nextBtn" type="submit" value=" ${attr:LANG_NEXT} >> "
title='${html:LANG_TIPNEXT}' onMouseOver="info('${js:LANG_TIPNEXT}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr></table>

View File

@@ -1,10 +1,10 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="${LANGUAGE_ISO}">
<html xmlns="http://www.w3.org/1999/xhtml" lang="${attr:LANGUAGE_ISO}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=${LANGUAGE_CHARSET}" />
<meta name="description" content="${LANG_METADESC}" />
<meta name="keywords" content="${LANG_METAKEYW}" />
<title>'${projname}' - HTTrack Website Copier</title>
<meta http-equiv="Content-Type" content="text/html; charset=${attr:LANGUAGE_CHARSET}" />
<meta name="description" content="${attr:LANG_METADESC}" />
<meta name="keywords" content="${attr:LANG_METAKEYW}" />
<title>'${html:projname}' - HTTrack Website Copier</title>
<link rel="stylesheet" type="text/css" media="screen" href="style.css">
<script type="text/javascript" src="ping.js" defer></script>
@@ -93,9 +93,9 @@ ${do:end-if}
<br>
<form method="POST" name="form" action="step4.html">
<input type="hidden" name="sid" value="${sid}">
<input type="hidden" name="sid" value="${attr:sid}">
<input type="hidden" name="redirect" value="">
<input type="hidden" name="projpath" value="${path}/${projname}/">
<input type="hidden" name="projpath" value="${attr:path}/${attr:projname}/">
<input type="hidden" name="command_arg_profile" value="path,lang">
@@ -132,7 +132,8 @@ httrack \
\
${do:if-not-empty:BuildString}
--structure "${arg:BuildString}"
${do:end-if}
${/* end-if clears the output mode rather than restoring it, so re-arm it */}
${do:end-if}${do:output-mode:html}
${test:build:-N0:-N0:-N1:-N2:-N3:-N4:-N5:-N100:-N101:-N102:-N103:-N104:-N105:-N99:-N199:}
\
${ztest:dos::--long-names=0:--long-names=2}
@@ -189,7 +190,7 @@ ${/* -m<n> resets the html limit, so the bare form must precede the -m,<n> one *
${test:http10:--http-10}
${test:cache2:--store-all-in-cache}
${test:sitemap:--sitemap}
${test:sitemapurl:--sitemap-url "}${html:sitemapurl}${test:sitemapurl:"}
${test:sitemapurl:--sitemap-url "}${arg:sitemapurl}${test:sitemapurl:"}
${test:warc:--warc}
${test:warcfile:--warc-file "}${arg:warcfile}${test:warcfile:"}
${test:warcmaxsize:--warc-max-size=}${unquoted:warcmaxsize}
@@ -318,11 +319,11 @@ ${do:output-mode:}
</td></tr><tr><td>
<table width="100%" border="0"><tr><td align="left">
<input type="submit" value=" << ${LANG_PREVIOUS} " onClick="form.command.value=''; form.redirect.value='step3.html'; form.submit()"
<input type="submit" value=" << ${attr:LANG_PREVIOUS} " onClick="form.command.value=''; form.redirect.value='step3.html'; form.submit()"
title='${html:LANG_TIPPREV}' onMouseOver="info('${js:LANG_TIPPREV}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td><td align="right">
<input name="nextBtn" type="submit" value=" ${LANG_J9} >> "
<input name="nextBtn" type="submit" value=" ${attr:LANG_J9} >> "
title='${html:LANG_TIPNEXT}' onMouseOver="info('${js:LANG_TIPNEXT}'); return true" onMouseOut="info('&nbsp;'); return true"
>
</td></tr></table>

View File

@@ -15,6 +15,10 @@ DevIncludes_DATA = \
htsarrays.h \
htssafe.h
# What hands a consumer the include path and the rpath the @rpath id needs (#978).
pkgconfigdir = $(libdir)/pkgconfig
pkgconfig_DATA = libhttrack.pc
AM_CPPFLAGS = \
@DEFAULT_CFLAGS@ \
@THREADS_CFLAGS@ \
@@ -42,8 +46,8 @@ httrack_CFLAGS = $(AM_CFLAGS) $(CFLAGS_PIE)
proxytrack_CFLAGS = $(AM_CFLAGS) $(CFLAGS_PIE) -DNO_MALLOCT -DZLIB_CONST -DHTS_INTHASH_USES_MD5
htsserver_CFLAGS = $(AM_CFLAGS) $(CFLAGS_PIE) -DZLIB_CONST -DHTS_INTHASH_USES_MD5
# @RPATH_ORIGIN_LDFLAGS@ makes a copied tree find libhttrack next to it (#906);
# proxytrack links no shared library of ours, so it needs none.
# @RPATH_ORIGIN_LDFLAGS@ makes a copied tree find libhttrack next to it (#906),
# pairing on Darwin with the dylib's @rpath id (#978); proxytrack needs none of it.
httrack_LDFLAGS = $(AM_LDFLAGS) $(LDFLAGS_PIE) @RPATH_ORIGIN_LDFLAGS@
proxytrack_LDFLAGS = $(AM_LDFLAGS) $(LDFLAGS_PIE)
htsserver_LDFLAGS = $(AM_LDFLAGS) $(LDFLAGS_PIE) @RPATH_ORIGIN_LDFLAGS@

View File

@@ -63,42 +63,101 @@ Please visit our Website: http://www.httrack.com
#include <execinfo.h>
#include <link.h>
#include <signal.h>
#include <spawn.h>
#include <time.h>
#include <sys/stat.h>
#include <sys/wait.h>
#define USES_BACKTRACE
#endif
#ifdef _WIN32
#define BT_REPORT_FD 2 /* MSVC ships no <unistd.h> */
#else
#define BT_REPORT_FD STDERR_FILENO
#endif
#ifdef USES_BACKTRACE
#define BT_MAX_FRAMES 64 /* frames we try to name */
#define BT_MAX_MODULES 8 /* distinct modules, one child each */
#define BT_HEX_SIZE 19 /* "0x" + 16 nibbles + NUL */
#define BT_PATH_SIZE 1024 /* module path; longer is skipped */
#define BT_WAIT_TICKS 300 /* 10ms ticks, shared: cap a slow child */
#define BT_NO_SYMBOLIZER 127 /* child exit: execvp() found none */
#define BT_MAX_FRAMES 64 /* frames we try to name */
#define BT_MAX_MODULES 8 /* distinct modules, one child each */
#define BT_HEX_SIZE 19 /* "0x" + 16 nibbles + NUL */
#define BT_PATH_SIZE 1024 /* module path; longer is skipped */
#define BT_WAIT_TICKS 300 /* 10ms ticks, shared: cap a slow child */
static hts_boolean symbolize_crash = HTS_TRUE;
/* dladdr() names the main program after argv[0], which the symbolizer cannot
open when the binary came off PATH; /proc/self/exe always names the file. */
static char main_path[BT_PATH_SIZE];
static const void *main_base;
/* Built at init: assembling file actions allocates; the crash path cannot. */
static posix_spawn_file_actions_t spawn_actions;
static posix_spawn_file_actions_t *spawn_redirect = NULL;
/* dladdr() names the main program after argv[0], which the symbolizer cannot
open when the binary came off PATH; /proc/self/exe always names the file.
hts_self_path() (htscoremain.c) resolves the same path, out of reach here:
it is library-side and hidden by -fvisibility=hidden (#997). */
static char main_path[BT_PATH_SIZE];
/* Main program's mapped range, and the bias to subtract for addr2line (0 on an
ET_EXEC). */
static uintptr_t main_lo, main_hi, main_bias;
static int record_main_range(struct dl_phdr_info *info, size_t size,
void *data) {
size_t i;
static int record_main_base(struct dl_phdr_info *info, size_t size,
void *data) {
(void) size;
(void) data;
main_base = (const void *) info->dlpi_addr;
main_bias = (uintptr_t) info->dlpi_addr;
main_lo = (uintptr_t) -1; /* an empty range matches no frame */
main_hi = 0;
for (i = 0; i < info->dlpi_phnum; i++) {
const ElfW(Phdr) *const phdr = &info->dlpi_phdr[i];
const uintptr_t start = main_bias + (uintptr_t) phdr->p_vaddr;
if (phdr->p_type != PT_LOAD)
continue;
if (start < main_lo)
main_lo = start;
if (start + phdr->p_memsz > main_hi)
main_hi = start + phdr->p_memsz;
}
return 1; /* dl_iterate_phdr starts at the main program */
}
/* Address range, not load base: the two disagree on an ET_EXEC (#995). */
static hts_boolean is_main_frame(const void *addr) {
const uintptr_t value = (uintptr_t) addr;
return value >= main_lo && value < main_hi ? HTS_TRUE : HTS_FALSE;
}
/* Non-zero once a named link-map entry is the file *data stats to: the main
program's entry is the unnamed one, so a hit means a shared object. */
static int names_a_shared_object(struct dl_phdr_info *info, size_t size,
void *data) {
const struct stat *const self = (const struct stat *) data;
struct stat st;
(void) size;
if (info->dlpi_name == NULL || info->dlpi_name[0] == '\0')
return 0;
if (stat(info->dlpi_name, &st) != 0)
return 0; /* vDSO and the like have no file */
return st.st_dev == self->st_dev && st.st_ino == self->st_ino;
}
static void find_main_object(void) {
const ssize_t len = readlink("/proc/self/exe", main_path, sizeof(main_path));
struct stat self;
dl_iterate_phdr(record_main_range, NULL);
/* A full buffer is a clipped path, which readlink() cannot report: its prefix
names another file, and the symbolizer would happily open that one. */
if (len > 0 && (size_t) len < sizeof(main_path)) {
main_path[len] = '\0';
if (dl_iterate_phdr(record_main_base, NULL) == 1)
/* Started through the loader, /proc/self/exe is ld.so, which the link map
carries under its own name (#996). Identity, not dladdr()'s name: that is
argv[0] verbatim, so a symlinked or renamed program would lose its
module. */
if (stat(main_path, &self) == 0 &&
dl_iterate_phdr(names_a_shared_object, &self) == 0)
return;
}
main_path[0] = '\0'; /* unresolved: keep whatever dladdr() reported */
@@ -136,34 +195,27 @@ static hts_boolean copy_bounded(char *dest, size_t size, const char *src) {
return src[i] == '\0' ? HTS_TRUE : HTS_FALSE;
}
/* Run the symbolizer on argv, output on fd, within *budget ticks. HTS_FALSE
only if none could be run at all; otherwise silent, the raw trace stands. */
static hts_boolean spawn_symbolizer(char **argv, int fd, int *budget) {
const pid_t pid = fork();
int status = 0;
/* Run the symbolizer on argv within *budget ticks; HTS_FALSE only if none could
be run at all. Not fork(): it runs the pthread_atfork handlers, and glibc's
malloc registers one taking every arena lock a signal inside malloc holds
(#968). posix_spawn() clones with CLONE_VFORK instead, and counts as
async-signal-safe as of POSIX.1-2024. */
static hts_boolean spawn_symbolizer(char **argv, int *budget) {
static char llvm_prog[] = "llvm-symbolizer";
static char llvm_opts[] = "-p";
pid_t pid;
if (pid == -1)
return HTS_FALSE;
if (pid == 0) {
static char llvm_prog[] = "llvm-symbolizer";
static char llvm_opts[] = "-p";
dup2(fd, 1); /* both symbolizers write on stdout */
execvp(argv[0], argv);
if (posix_spawnp(&pid, argv[0], spawn_redirect, NULL, argv, environ) != 0) {
argv[0] = llvm_prog; /* an LLVM-only install ships no addr2line */
argv[1] = llvm_opts;
execvp(argv[0], argv);
_exit(BT_NO_SYMBOLIZER);
if (posix_spawnp(&pid, argv[0], spawn_redirect, NULL, argv, environ) != 0)
return HTS_FALSE;
}
for (; *budget > 0; (*budget)--) {
const struct timespec tick = {0, 10 * 1000 * 1000};
const pid_t reaped = waitpid(pid, &status, WNOHANG);
const pid_t reaped = waitpid(pid, NULL, WNOHANG);
if (reaped == pid)
return WIFEXITED(status) && WEXITSTATUS(status) == BT_NO_SYMBOLIZER
? HTS_FALSE
: HTS_TRUE;
if (reaped == -1 && errno != EINTR)
if (reaped == pid || (reaped == -1 && errno != EINTR))
return HTS_TRUE;
nanosleep(&tick, NULL);
}
@@ -175,7 +227,7 @@ static hts_boolean spawn_symbolizer(char **argv, int fd, int *budget) {
/* Name the frames backtrace_symbols_fd() leaves as module+offset:
-fvisibility=hidden keeps them out of .dynsym, but DWARF has them. dladdr()
is not formally async-signal-safe; accepted, this path is already fatal. */
static void symbolize_backtrace(void *const *stack, int size, int fd) {
static void symbolize_backtrace(void *const *stack, int size) {
static char prog[] = "addr2line";
static char opts[] = "-Cfipa";
static char dashe[] = "-e";
@@ -183,11 +235,14 @@ static void symbolize_backtrace(void *const *stack, int size, int fd) {
const void *base[BT_MAX_FRAMES];
const char *name[BT_MAX_FRAMES];
hts_boolean grouped[BT_MAX_FRAMES];
hts_boolean is_main[BT_MAX_FRAMES];
char module[BT_PATH_SIZE];
char *argv[4 + BT_MAX_FRAMES + 1];
int budget = BT_WAIT_TICKS;
int i, spawned;
if (spawn_redirect == NULL) /* init could not prepare the child's redirect */
return;
if (size > BT_MAX_FRAMES)
size = BT_MAX_FRAMES;
@@ -200,8 +255,10 @@ static void symbolize_backtrace(void *const *stack, int size, int fd) {
continue;
base[i] = info.dli_fbase;
name[i] = info.dli_fname;
print_hex(hex[i], (uintptr_t) ((const char *) stack[i] -
(const char *) info.dli_fbase));
is_main[i] = is_main_frame(stack[i]);
print_hex(hex[i],
(uintptr_t) stack[i] -
(is_main[i] ? main_bias : (uintptr_t) info.dli_fbase));
grouped[i] = HTS_FALSE;
}
@@ -215,8 +272,7 @@ static void symbolize_backtrace(void *const *stack, int size, int fd) {
;
if (first >= size)
break;
path = main_path[0] != '\0' && base[first] == main_base ? main_path
: name[first];
path = main_path[0] != '\0' && is_main[first] ? main_path : name[first];
argv[argc++] = prog;
argv[argc++] = opts;
argv[argc++] = dashe;
@@ -235,9 +291,9 @@ static void symbolize_backtrace(void *const *stack, int size, int fd) {
const size_t len = strlen(module);
/* addr2line -a prints offsets only: say which module they are in. */
(void) (write(fd, module, len) == (ssize_t) len);
(void) (write(fd, ":\n", 2) == 2);
if (!spawn_symbolizer(argv, fd, &budget))
(void) (write(BT_REPORT_FD, module, len) == (ssize_t) len);
(void) (write(BT_REPORT_FD, ":\n", 2) == 2);
if (!spawn_symbolizer(argv, &budget))
break; /* no symbolizer: stop at one header */
}
}
@@ -250,6 +306,10 @@ void hts_backtrace_init(void) {
symbolize_crash =
getenv("HTTRACK_NO_SYMBOLIZE") == NULL ? HTS_TRUE : HTS_FALSE;
if (symbolize_crash && posix_spawn_file_actions_init(&spawn_actions) == 0 &&
posix_spawn_file_actions_adddup2(&spawn_actions, BT_REPORT_FD,
STDOUT_FILENO) == 0)
spawn_redirect = &spawn_actions; /* both symbolizers write on stdout */
find_main_object();
/* Pay for the unwinder now: glibc's first backtrace() dlopen()s libgcc_s,
which allocates and takes the loader lock the crashing thread may hold. */
@@ -257,7 +317,7 @@ void hts_backtrace_init(void) {
#endif
}
hts_boolean hts_backtrace_altstack(void) {
void *hts_backtrace_altstack(void) {
#ifdef USES_SIGALTSTACK
/* Not a constant since glibc 2.34: SIGSTKSZ is a sysconf() call. */
const size_t size =
@@ -268,23 +328,43 @@ hts_boolean hts_backtrace_altstack(void) {
/* Never take one over, whatever its size: a sanitizer runtime installs its
own and sizes it for its own handlers (ASan: 32kB, ours needs ~10kB). */
if (sigaltstack(NULL, &ss) == 0 && (ss.ss_flags & SS_DISABLE) == 0)
return HTS_TRUE;
return NULL;
/* Mapped, not allocated: a stack for the handler must not live in the heap
whose corruption we may be reporting. */
sp = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1,
0);
if (sp == MAP_FAILED)
return HTS_FALSE;
return NULL;
ss.ss_sp = sp;
ss.ss_size = size;
ss.ss_flags = 0;
if (sigaltstack(&ss, NULL) != 0) {
munmap(sp, size);
return HTS_FALSE;
return NULL;
}
return HTS_TRUE;
return sp;
#else
return HTS_FALSE;
return NULL;
#endif
}
void hts_backtrace_altstack_release(void *stack) {
#ifdef USES_SIGALTSTACK
stack_t ss;
size_t size;
if (stack == NULL)
return;
/* Whatever replaced ours since is not ours to unmap. */
if (sigaltstack(NULL, &ss) != 0 || ss.ss_sp != stack)
return;
size = ss.ss_size;
ss.ss_flags = SS_DISABLE;
/* Disable first: the kernel must never send a handler to an unmapped page. */
if (sigaltstack(&ss, NULL) == 0)
munmap(stack, size);
#else
(void) stack;
#endif
}
@@ -295,31 +375,31 @@ static void print_no_trace(int fd, const char *msg, size_t len) {
}
}
void hts_print_backtrace(int fd) {
void hts_print_backtrace(void) {
#ifdef USES_BACKTRACE
void *stack[256];
const int size = backtrace(stack, sizeof(stack) / sizeof(stack[0]));
/* A fault inside the handler lands back here: symbolizing twice interleaves
two traces on fd and spends a second budget. */
two traces on the report fd and spends a second budget. */
static volatile sig_atomic_t entered = 0;
if (size != 0) {
backtrace_symbols_fd(stack, size, fd);
backtrace_symbols_fd(stack, size, BT_REPORT_FD);
if (symbolize_crash && entered == 0) {
entered = 1;
symbolize_backtrace(stack, size, fd);
symbolize_backtrace(stack, size);
entered = 0;
}
} else {
/* An empty trace means the build carries no unwind tables. */
const char msg[] = "No stack trace available: unwinding failed\n";
print_no_trace(fd, msg, sizeof(msg) - 1);
print_no_trace(BT_REPORT_FD, msg, sizeof(msg) - 1);
}
#else
const char msg[] = "No stack trace available on this OS :(\n";
print_no_trace(fd, msg, sizeof(msg) - 1);
print_no_trace(BT_REPORT_FD, msg, sizeof(msg) - 1);
#endif
}

View File

@@ -35,20 +35,27 @@ Please visit our Website: http://www.httrack.com
#include "htsglobal.h"
/* Sample HTTRACK_NO_SYMBOLIZE and load the unwinder before any crash: neither
is signal-safe. Call once, from the process installing the fatal handlers. */
/* Sample HTTRACK_NO_SYMBOLIZE, prepare the symbolizer spawn and load the
unwinder: none of it is signal-safe. Call once, from the process installing
the fatal handlers. */
void hts_backtrace_init(void);
/* Ensure the calling thread has an alternate signal stack, so a handler
registered with SA_ONSTACK still runs when the fault is stack exhaustion. One
already installed is left alone, whoever owns it. Per-thread and kept until
the thread dies; HTS_FALSE if none could be provided, the handler then
already installed is left alone, whoever owns it. Returns the mapping to hand
back when the thread ends, NULL if nothing was installed, the handler then
running on the faulting stack as it used to. */
hts_boolean hts_backtrace_altstack(void);
void *hts_backtrace_altstack(void);
/* Write the calling thread's stack to fd, callable from a fatal signal handler:
raw frames first, then whatever an external symbolizer can name. Allocates
nothing; prints a one-line notice where the OS has no backtrace(). */
void hts_print_backtrace(int fd);
/* Unmap what hts_backtrace_altstack() returned, from that same thread: one
alternate stack per worker adds up over a crawl. */
void hts_backtrace_altstack_release(void *stack);
/* Write the calling thread's stack to stderr, callable from a fatal signal
handler: raw frames first, then whatever an external symbolizer can name.
Stderr and not a parameter: hts_backtrace_init() pre-builds the child's
redirect onto it. Allocates nothing; prints a one-line notice where the OS
has no backtrace(). */
void hts_print_backtrace(void);
#endif

View File

@@ -84,9 +84,12 @@ extern "C" {
/* OpenSSL structure */
#include <openssl/bio.h>
/* Engine-only: not exported, so the installed header must not offer it. */
#ifdef HTS_INTERNAL_BYTECODE
/** Process-wide OpenSSL client context, created lazily on first TLS use;
shared by all connections. NULL until initialized. */
extern SSL_CTX *openssl_ctx;
#endif
#endif
#endif

View File

@@ -87,6 +87,8 @@ static int datadir_has_templates(const char *dir) {
"templates/index-header.html"));
}
/* htsbacktrace.c copies the Linux branch: it is program-side and cannot reach
this hidden symbol, so a fix here belongs there too (#997). */
const char *hts_self_path(char *dst, size_t dstsize) {
#if defined(_WIN32)
const DWORD n = GetModuleFileNameA(NULL, dst, (DWORD) dstsize);

View File

@@ -36,11 +36,16 @@ Please visit our Website: http://www.httrack.com
#include "htscrashtest.h"
#include "htssafe.h"
#include "htsthread.h"
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#ifndef _WIN32
#include <pthread.h>
#define CRASH_HAS_ATFORK
#endif
#if defined(_MSC_VER)
#define CRASH_NOINLINE __declspec(noinline)
@@ -95,6 +100,41 @@ static CRASH_NOINLINE char blow_the_stack(size_t depth) {
/* Faults with no stack left for the handler, unless it runs on an altstack. */
static CRASH_NOINLINE void crash_stack(void) { (void) blow_the_stack(0); }
static void crash_stack_thread(void *arg) {
(void) arg;
fprintf(stderr, "** Crash test worker thread started\n");
fflush(stderr);
crash_stack();
}
/* Same runaway recursion in an engine worker: the fatal handler needs an
alternate stack in every thread, not just the main one (#969). */
static CRASH_NOINLINE void crash_threadstack(void) {
/* Aborting on a spawn failure keeps the caller's exit status a crash, so the
test reads "no worker started" rather than "the handler never ran". */
if (hts_newthread(crash_stack_thread, NULL) != 0)
abortLog("crash test: cannot spawn a worker thread");
htsthread_wait_n(0); /* the worker takes the process down from there */
}
#ifdef CRASH_HAS_ATFORK
static pthread_mutex_t crash_fork_lock = PTHREAD_MUTEX_INITIALIZER;
static void crash_fork_prepare(void) { pthread_mutex_lock(&crash_fork_lock); }
static void crash_fork_parent(void) { pthread_mutex_unlock(&crash_fork_lock); }
#endif
/* Faults holding a lock a pthread_atfork prepare handler wants (#968). Plain
segv where there is no atfork. */
static CRASH_NOINLINE void crash_atfork(void) {
#ifdef CRASH_HAS_ATFORK
pthread_atfork(crash_fork_prepare, crash_fork_parent, NULL);
pthread_mutex_lock(&crash_fork_lock);
#endif
crash_segv();
}
static const struct {
const char *name;
void (*fn)(void);
@@ -103,21 +143,32 @@ static const struct {
{"abort", crash_abort},
{"trap", crash_trap},
{"stack", crash_stack},
{"threadstack", crash_threadstack},
{"atfork", crash_atfork},
};
#define CRASH_KINDS_COUNT (sizeof(crash_kinds) / sizeof(crash_kinds[0]))
/* Appends what fits and drops the rest, where strcatbuff() would abort. */
static void kinds_append(char *dest, size_t size, const char *src) {
const size_t used = strlen(dest);
strlncatbuff(dest, src, size, size - used - 1);
}
const char *hts_crash_test_kinds(void) {
static char list[64];
/* Clipped rather than aborted: a table outgrowing this must not cost the
process, the string only feeds a usage line. */
static char list[256];
if (list[0] == '\0') {
size_t i;
for (i = 0; i < CRASH_KINDS_COUNT; i++) {
if (i != 0) {
strcatbuff(list, ", ");
kinds_append(list, sizeof(list), ", ");
}
strcatbuff(list, crash_kinds[i].name);
kinds_append(list, sizeof(list), crash_kinds[i].name);
}
}
return list;

View File

@@ -149,11 +149,12 @@ void ftp_split_userpass(const char *src, const char *end, char *user,
}
}
/* Build "<verb> <path>", quoting a path the server could not parse bare. */
static void ftp_command(char *line, size_t line_size, const char *verb,
const char *path) {
if (strchr(path, ' ') != NULL || strchr(path, '\"') != NULL ||
strchr(path, '\'') != NULL)
/* Build "<verb> <path>" (see htsftp.h). */
void ftp_command(char *line, size_t line_size, const char *verb,
const char *path) {
/* A leading '-' would reach a server that shells out to ls as a flag. */
if (path[0] == '-' || strchr(path, ' ') != NULL ||
strchr(path, '\"') != NULL || strchr(path, '\'') != NULL)
snprintf(line, line_size, "%s \"%s\"", verb, path);
else
snprintf(line, line_size, "%s %s", verb, path);
@@ -218,7 +219,7 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
#endif
char BIGSTK adr_ip[1024];
char *adr, *real_adr;
const char *ftp_filename = "";
char BIGSTK ftp_path[CATBUFF_SIZE]; // the decoded URL path, screened once
int timeout = 300; // timeout
int timeout_onfly = 8; // attente réponse supplémentaire
int transfer_list = 0; // directory
@@ -231,7 +232,7 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
SOCaddr server_data;
//
line_retr[0] = adr_ip[0] = '\0';
line_retr[0] = adr_ip[0] = ftp_path[0] = '\0';
timeout = 300;
@@ -258,22 +259,16 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
a = back->url_fil;
if (a != NULL && *a != '\0') {
ftp_filename = a;
if (strnotempty(a)) {
char catbuff[CATBUFF_SIZE];
char *ua = unescape_http(catbuff, sizeof(catbuff), a);
int len_a = (int) strlen(ua);
const size_t len_a =
strlen(unescape_http(ftp_path, sizeof(ftp_path), a));
if (len_a > 0 && ua[len_a - 1] == '/') { /* obviously a directory listing */
if (len_a > 0 &&
ftp_path[len_a - 1] == '/') { /* obviously a directory listing */
transfer_list = 1;
snprintf(line_retr, sizeof(line_retr), "LIST -A %s", ua);
} else if ((strchr(ua, ' '))
|| (strchr(ua, '\"'))
|| (strchr(ua, '\''))
) {
snprintf(line_retr, sizeof(line_retr), "RETR \"%s\"", ua);
} else { /* Regular one */
snprintf(line_retr, sizeof(line_retr), "RETR %s", ua);
ftp_command(line_retr, sizeof(line_retr), "LIST -A", ftp_path);
} else {
ftp_command(line_retr, sizeof(line_retr), "RETR", ftp_path);
}
} else {
transfer_list = 1;
@@ -285,6 +280,15 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
}
}
// fail here, or send_line() drops the command and the reply read times out.
// Every command below is a literal verb plus one of these three.
if (!hts_is_control_free(user) || !hts_is_control_free(pass) ||
!hts_is_control_free(ftp_path)) {
strcpybuff(back->r.msg, "Invalid control character in FTP URL");
back->r.statuscode = STATUSCODE_INVALID;
_HALT_FTP return 0;
}
#if FTP_DEBUG
printf("Connecting to %s...\n", adr);
#endif
@@ -511,10 +515,7 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
// SIZE
if (back->r.statuscode != -1) {
if (!transfer_list) {
char catbuff[CATBUFF_SIZE];
char *ua = unescape_http(catbuff, sizeof(catbuff), ftp_filename);
ftp_command(line, sizeof(line), "SIZE", ua);
ftp_command(line, sizeof(line), "SIZE", ftp_path);
// SIZE?
strcpybuff(back->info, "size");
@@ -536,7 +537,7 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
}
// MDTM?
ftp_command(line, sizeof(line), "MDTM", ua);
ftp_command(line, sizeof(line), "MDTM", ftp_path);
strcpybuff(back->info, "mdtm");
send_line(soc_ctl, line);
get_ftp_line(soc_ctl, line, sizeof(line), timeout);
@@ -923,6 +924,10 @@ FILE *dd = NULL;
int send_line(T_SOC soc, const char *data) {
char BIGSTK line[1024];
// backstop: the driver fails earlier, but no injected byte reaches the wire
if (!hts_is_control_free(data))
return 0;
if (_DEBUG_HEAD) {
if (ioinfo) {
fprintf(ioinfo, "---> %s\x0d\x0a", data);

View File

@@ -75,6 +75,11 @@ int get_ftp_line(T_SOC soc, char *line, size_t line_size, int timeout);
Both sizes must be nonzero. */
void ftp_split_userpass(const char *src, const char *end, char *user,
size_t user_size, char *pass, size_t pass_size);
/* Build "<verb> <path>" into line[line_size], truncating to fit. The path is
quoted whenever a bare one would give the server a second token; it must
already have been screened for control bytes. */
void ftp_command(char *line, size_t line_size, const char *verb,
const char *path);
T_SOC get_datasocket(char *to_send, size_t to_send_size);
int stop_ftp(lien_back * back);
char *linejmp(char *line);

View File

@@ -43,8 +43,8 @@ Please visit our Website: http://www.httrack.com
configure.ac, decoupled from these). VERSION is the display form, VERSIONID
the dotted numeric form, AFF_VERSION the short form shown in footers,
LIB_VERSION the data/cache format generation. */
#define HTTRACK_VERSION "3.49-16"
#define HTTRACK_VERSIONID "3.49.16"
#define HTTRACK_VERSION "3.49-17"
#define HTTRACK_VERSIONID "3.49.17"
#define HTTRACK_AFF_VERSION "3.x"
#define HTTRACK_LIB_VERSION "2.0"

View File

@@ -926,6 +926,10 @@ int http_sendhead(httrackp * opt, t_cookie * cookie, int mode,
int direct_url = 0; // ne pas analyser l'url (exemple: ftp://)
const char *search_tag = NULL;
/* adr and the referer come off the network and can carry raw CR/LF; capped
at their own source buffers, so the worst case emitted does not grow. */
char BIGSTK esc[HTS_URLMAXSIZE * 2];
// Initialize buffer
buffer_head_request[0] = '\0';
@@ -954,9 +958,10 @@ int http_sendhead(httrackp * opt, t_cookie * cookie, int mode,
if (retour->req.proxy.active &&
!hts_proxy_is_socks(retour->req.proxy.name) &&
!hts_proxy_is_connect(retour->req.proxy.name)) {
print_buffer(&bstr,
"%s http://%s%s %s\r\n", method, adr, url,
protocol);
/* only adr is raw here: the other three are %s-scanned */
print_buffer(&bstr, "%s http://%s%s %s\r\n", method,
escape_check_url_addr(adr, esc, sizeof(esc)), url,
protocol);
} else {
print_buffer(&bstr,
"%s %s %s\r\n", method, url, protocol);
@@ -999,14 +1004,16 @@ int http_sendhead(httrackp * opt, t_cookie * cookie, int mode,
printf("Proxy Use: for %s%s proxy %d port %d\n", adr, fil,
retour->req.proxy.name, retour->req.proxy.port);
#endif
print_buffer(&bstr, "http://%s", jump_identification_const(adr));
print_buffer(&bstr, "http://%s",
escape_check_url_addr(jump_identification_const(adr), esc,
sizeof(esc)));
} else { // ftp:// en proxy http
#if HDEBUG
printf("Proxy Use for ftp: for %s%s proxy %d port %d\n", adr, fil,
retour->req.proxy.name, retour->req.proxy.port);
#endif
direct_url = 1; // ne pas analyser user/pass
print_buffer(&bstr, "%s", adr);
print_buffer(&bstr, "%s", escape_check_url_addr(adr, esc, sizeof(esc)));
}
}
// NOM DU FICHIER
@@ -1071,8 +1078,13 @@ int http_sendhead(httrackp * opt, t_cookie * cookie, int mode,
||(strncmp(adr, "https://", 8) == 0) /* or referer AND addresses are https */
)
) { // PAS file://
print_buffer(&bstr, "Referer: http://%s%s"H_CRLF,
jump_identification_const(referer_adr), referer_fil);
/* one escape per piece, so neither can outgrow its own source buffer */
print_buffer(
&bstr, "Referer: http://%s",
escape_check_url_addr(jump_identification_const(referer_adr), esc,
sizeof(esc)));
print_buffer(&bstr, "%s" H_CRLF,
escape_check_url_addr(referer_fil, esc, sizeof(esc)));
}
}
// HTTP field: referer
@@ -1105,7 +1117,8 @@ int http_sendhead(httrackp * opt, t_cookie * cookie, int mode,
// Mandatory per RFC2616
if (!direct_url) { // pas ftp:// par exemple
print_buffer(&bstr, "Host: %s"H_CRLF, real_adr);
print_buffer(&bstr, "Host: %s" H_CRLF,
escape_check_url_addr(real_adr, esc, sizeof(esc)));
}
// HTTP field: from
@@ -3736,6 +3749,20 @@ const char *jump_protocol_const(const char *source) {
DECLARE_NON_CONST_VERSION(jump_protocol)
hts_boolean hts_is_control_free_sized(const char *str, size_t len) {
size_t i;
for (i = 0; i < len; i++) {
if ((unsigned char) str[i] < ' ')
return HTS_FALSE;
}
return HTS_TRUE;
}
hts_boolean hts_is_control_free(const char *str) {
return hts_is_control_free_sized(str, strlen(str));
}
hts_boolean hts_proxy_is_socks(const char *name) {
if (name == NULL)
return HTS_FALSE;
@@ -3960,11 +3987,16 @@ HTSEXT_API size_t escape_uri_utf(const char *const src,
return x_escape_http(src, dest, size, 30);
}
HTSEXT_API size_t escape_check_url(const char *const src,
char *const dest, const size_t size) {
HTSEXT_API size_t escape_check_url(const char *const src, char *const dest,
const size_t size) {
return x_escape_http(src, dest, size, 0);
}
HTSEXT_API size_t escape_control_url(const char *const src, char *const dest,
const size_t size) {
return x_escape_http(src, dest, size, 4);
}
// same as escape_check_url, but returns char*
HTSEXT_API char *escape_check_url_addr(const char *const src,
char *const dest, const size_t size) {
@@ -4050,6 +4082,8 @@ HTSEXT_API void escape_remove_control(char *const s) {
j++;
}
}
// compaction left the original tail sitting past j
s[j] = '\0';
}
#undef ADD_CHAR
@@ -4094,6 +4128,8 @@ HTSEXT_API size_t x_escape_http(const char *const s, char *const dest,
else if (mode == 3) // échapper que ce qui est nécessaire
test = CHAR_SPECIAL(c)
|| CHAR_XXAVOID(c);
else if (mode == 4) // C0 controls only, leaving high bytes (UTF-8)
test = CHAR_LOW(c);
else if (mode == 30) // échapper que ce qui est nécessaire
test = (c != '/' && CHAR_RESERVED(c))
|| CHAR_DELIM(c)

View File

@@ -237,6 +237,14 @@ int check_readinput(htsblk * r);
int check_readinput_t(T_SOC soc, int timeout);
int check_writeinput_t(T_SOC soc, int timeout);
/* TRUE if str[0..len) holds no byte below ' '. A control byte in a value the
client puts on a protocol line (an FTP command, a CONNECT authority, a
SOCKS5 host) smuggles a line or a field of its own. */
hts_boolean hts_is_control_free_sized(const char *str, size_t len);
/* Same over a NUL-terminated string. */
hts_boolean hts_is_control_free(const char *str);
/* TRUE if this -P proxy name (which keeps its scheme) is a SOCKS5 proxy. */
hts_boolean hts_proxy_is_socks(const char *name);

View File

@@ -305,6 +305,8 @@ static HTS_UNUSED void SOCaddr_inetntoa_(char *namebuf, size_t namebuflen,
typedef socklen_t SOClen;
#if HTS_INET6 != 0
/* Engine-only: not exported, and the type is useless without the setter. */
#ifdef HTS_INTERNAL_BYTECODE
/** Resolver backend: getaddrinfo/freeaddrinfo as a swappable pair, so the
self-test can script DNS answers (families, multiplicity, errors)
in-process. The free function must match its getaddrinfo (a fake allocates
@@ -317,6 +319,10 @@ typedef socklen_t SOClen;
#define HTS_RESOLVER_CALL
#endif
/* File scope, or the tag below is a fresh type scoped to its own prototype
wherever <netdb.h> has not already declared it. */
struct addrinfo;
typedef struct hts_resolver_backend {
int(HTS_RESOLVER_CALL *getaddrinfo)(const char *node, const char *service,
const struct addrinfo *hints,
@@ -328,6 +334,7 @@ typedef struct hts_resolver_backend {
Test-only seam, not thread-safe; callers must serialize against resolves. */
void hts_dns_set_resolver_backend(const hts_resolver_backend *backend);
#endif
#endif
#ifdef __cplusplus
}

View File

@@ -1943,10 +1943,8 @@ int htsparse(htsmoduleStruct * str, htsmoduleStructExtended * stre) {
if (ok != -1) { // continuer
// découper le lien
do {
if ((unsigned char) *eadr < 32) { // caractère de contrôle (ou \0)
if (!is_space(*eadr))
ok = 0;
}
if (*eadr == '\0') // end of the parsed buffer
ok = 0;
if (eadr - html > HTS_URLMAXSIZE) // ** trop long, >HTS_URLMAXSIZE caractères (on prévoit HTS_URLMAXSIZE autres pour path)
ok = -1; // ne pas traiter ce lien
@@ -2101,18 +2099,20 @@ int htsparse(htsmoduleStruct * str, htsmoduleStructExtended * stre) {
char *a = lien;
size_t llen;
// strip ending spaces
// strip both ends of every C0 control or space, as a browser
// does; encoding one there would 404 a link that fetched fine
llen = (*a != '\0') ? strlen(a) : 0;
while(llen > 0 && is_realspace(lien[llen - 1])) {
while (llen > 0 && (unsigned char) lien[llen - 1] <= ' ') {
a[--llen] = '\0';
}
// skip leading ones
while(is_realspace(*a))
// '\0' is <= ' ' too, and an all-control link ends up empty
while (*a != '\0' && (unsigned char) *a <= ' ')
a++;
// strip cr, lf, tab inside URL
// strip cr, lf, tab inside URL, as a browser does; every
// other control byte percent-encodes below
llen = 0;
while(*a) {
if (*a != '\n' && *a != '\r' && *a != '\t') {
if (!is_retorsep(*a)) {
lien[llen++] = *a;
}
a++;
@@ -2152,8 +2152,22 @@ int htsparse(htsmoduleStruct * str, htsmoduleStructExtended * stre) {
strcpybuff(lien,
unescape_http_unharm(catbuff, sizeof(catbuff), lien, 1 | 2)); /* note: '%' is still escaped */
// Force to encode non-printable chars (should never happend)
escape_remove_control(lien);
// Percent-encode the control bytes as a browser does (#982);
// a byte grows to three, so a link that outgrows the buffer
// is dropped rather than clipped to a URL nobody wrote.
{
char BIGSTK tempo[sizeof(lien)];
if (escape_control_url(lien, tempo, sizeof(tempo)) <
sizeof(tempo)) {
strcpybuff(lien, tempo);
} else {
error = 1;
hts_log_print(
opt, LOG_DEBUG,
"link rejected (control bytes do not fit) %s", lien);
}
}
// charset conversion for the URI filename (not the query
// string), unless the bytes already are valid UTF-8:
@@ -2204,10 +2218,22 @@ int htsparse(htsmoduleStruct * str, htsmoduleStructExtended * stre) {
"could not URL-decode string '%s'", lien);
}
// we need to encode query string non-ascii chars,
// we need to encode query string non-ascii chars,
// leaving the encoding as-is (unlike the file part)
// and copy back query
append_escape_check_url(query, lien, sizeof(lien));
{
const size_t used = strlen(lien);
// the append grows the query too, and clips silently on
// overflow: drop, or we fetch a query nobody wrote (#982)
if (append_escape_check_url(query, lien, sizeof(lien)) >=
sizeof(lien) - used) {
error = 1;
hts_log_print(opt, LOG_DEBUG,
"link rejected (query does not fit) %s",
lien);
}
}
}
// convertir les éventuels \ en des / pour éviter des problèmes de reconnaissance!

View File

@@ -102,15 +102,9 @@ int http_proxy_tunnel(httrackp *opt, htsblk *retour, const char *adr,
// backstop: never let a stray CR/LF in the host smuggle a second line into
// the CONNECT request (the host is already sanitized upstream)
{
const char *c;
for (c = authority; *c != '\0'; c++) {
if ((unsigned char) *c < ' ') {
strcpybuff(retour->msg, "proxy CONNECT: invalid host");
return 0;
}
}
if (!hts_is_control_free(authority)) {
strcpybuff(retour->msg, "proxy CONNECT: invalid host");
return 0;
}
snprintf(req, sizeof(req), "CONNECT %s HTTP/1.0" H_CRLF "Host: %s" H_CRLF,
@@ -480,17 +474,14 @@ static int socks5_handshake_stream(httrackp *opt, socks5_stream *st,
size_t userlen = 0, passlen = 0;
int want_auth = 0;
int port = ssl ? 443 : 80;
size_t i;
if (hostlen == 0 || hostlen > SOCKS5_MAXFIELD)
return socks5_fail(msg, msgsize, "SOCKS5: invalid origin host");
if (host[0] == '[') // ATYP=domain cannot carry an IPv6 literal
return socks5_fail(msg, msgsize,
"SOCKS5: IPv6 literal origin is not supported");
for (i = 0; i < hostlen; i++) {
if ((unsigned char) host[i] < ' ')
return socks5_fail(msg, msgsize, "SOCKS5: invalid origin host");
}
if (!hts_is_control_free_sized(host, hostlen))
return socks5_fail(msg, msgsize, "SOCKS5: invalid origin host");
// the old range check ran after sscanf("%d") had wrapped a huge value into a
// plausible port (#614). An empty "host:" stays refused here, unlike the
// direct path, as it was before #614.

View File

@@ -2008,6 +2008,21 @@ static int st_socks5(httrackp *opt, int argc, char **argv) {
}
}
/* a control byte in the host would be a field of its own in the ATYP=domain
request; the port that follows it must not hide it (#1010) */
{
static const char *const hostile[] = {"ori\rgin.test", "ori\rgin.test:80"};
size_t k;
for (k = 0; k < sizeof(hostile) / sizeof(hostile[0]); k++) {
len = socks5_reply(script, 0x01, v4, sizeof(v4));
io.reply = script;
io.reply_len = len;
assertf(socks5_handshake_scripted(opt, hostile[k], proxy, &io) == 0);
assertf(io.sent_len == 0);
}
}
/* credentials: split on the first colon of the escaped userinfo, so %3a stays
inside the username and a colon in the password is not a delimiter */
{
@@ -2310,6 +2325,67 @@ static int st_sniff(httrackp *opt, int argc, char **argv) {
return 0;
}
/* escape_remove_control() compacts in place, so it has to terminate at the new
end or the caller reads the compacted head plus the original tail (#974). */
static int st_escape_control(httrackp *opt, int argc, char **argv) {
static const struct {
const char *in;
const char *out;
} cases[] = {
/* VT and FF are the ones that reach here: is_space() passes them */
{"/a\013bc", "/abc"},
{"\014abc", "abc"},
/* nothing moves, but the end does */
{"abc\013", "abc"},
{"abc\001def", "abcdef"},
{"\001\002\003", ""},
/* untouched inputs: the terminator must stay where it was */
{"abc", "abc"},
{"", ""},
/* only bytes below 32 go: DEL and high bytes are not control here */
{"a\177\303\251", "a\177\303\251"},
/* both sides of the >= 32 cut, and a shrink of more than one byte */
{"a b", "a b"},
{"a\037b\036c", "abc"},
};
const size_t ncases = sizeof(cases) / sizeof(cases[0]);
char buf[1024];
size_t k, m;
(void) opt;
if (argc > 0) {
const size_t n = st_decode_body(argv[0], buf, sizeof(buf));
assertf(n < sizeof(buf));
escape_remove_control(buf);
printf("escape-control: len=%d out=hex:", (int) strlen(buf));
for (k = 0; buf[k] != '\0'; k++) {
printf("%02x", (unsigned char) buf[k]);
}
printf("\n");
return 0;
}
for (k = 0; k < ncases; k++) {
const size_t inlen = strlen(cases[k].in);
const size_t outlen = strlen(cases[k].out);
/* poison, so a stray write shows up as a byte a zeroed buffer would hide */
memset(buf, '#', sizeof(buf));
memcpy(buf, cases[k].in, inlen + 1);
escape_remove_control(buf);
assertf(strlen(buf) == outlen);
assertf(memcmp(buf, cases[k].out, outlen + 1) == 0);
/* the terminator belongs inside the original string, never past its NUL */
for (m = inlen + 1; m < sizeof(buf); m++) {
assertf(buf[m] == '#');
}
}
printf("escape-control self-test OK\n");
return 0;
}
/* fsize()/fsize_utf8()/fpsize() must report a size past 4GB: 32-bit wraps both
ways there (MSVC's off_t and struct _stat st_size are long, 32-bit even on
x64), and a size under 4GB would survive an *unsigned* 32-bit truncation. */
@@ -4464,6 +4540,72 @@ static int st_ftpuser(httrackp *opt, int argc, char **argv) {
return 0;
}
/* send_line() must drop a command line carrying a control byte (#1010). */
static int st_ftpctrl(httrackp *opt, int argc, char **argv) {
/* Verb and URL path as run_launch_ftp() hands them over, then the line the
wire must carry; NULL for a command that must never leave. */
static const struct {
const char *verb;
const char *path;
const char *sent;
} cases[] = {
{"RETR", "/f.txt%0d%0aDELE%20secret.txt", NULL},
{"RETR", "/f.txt%0dDELE%20secret.txt", NULL},
{"RETR", "/f.txt%0aDELE%20secret.txt", NULL},
{"LIST -A", "/d%0d%0aDELE%20secret.txt/", NULL},
{"RETR", "/plain.txt", "RETR /plain.txt"},
{"RETR", "/a%20b.txt", "RETR \"/a b.txt\""},
{"RETR", "%2Fa%25b.txt", "RETR /a%b.txt"},
/* High bytes must still go out: a plain-char check reads them negative
and rejects them. */
{"RETR", "/caf%e9.txt", "RETR /caf\xe9.txt"},
/* Bare, these two would hand a server that shells out to ls a flag. */
{"LIST -A", "/x%20-la/", "LIST -A \"/x -la/\""},
{"LIST -A", "-la", "LIST -A \"-la\""},
};
char BIGSTK catbuff[CATBUFF_SIZE];
char cmd[512];
char expect[512];
char wire[512];
T_SOC sv[2];
size_t got = 0, dropped = 1, i;
(void) opt;
(void) argc;
(void) argv;
expect[0] = '\0';
assertf(st_socketpair(sv) == 0);
assertf(send_line(sv[0], "USER bob\001") == 0); // any field, not just a path
for (i = 0; i < sizeof(cases) / sizeof(cases[0]); i++) {
ftp_command(cmd, sizeof(cmd), cases[i].verb,
unescape_http(catbuff, sizeof(catbuff), cases[i].path));
if (cases[i].sent == NULL) {
assertf(strstr(cmd, "DELE") != NULL); // the payload did reach the builder
assertf(send_line(sv[0], cmd) == 0);
dropped++;
} else {
assertf(send_line(sv[0], cmd) != 0);
strcatbuff(expect, cases[i].sent);
strcatbuff(expect, "\r\n");
}
}
deletesoc(sv[0]); // EOF, so the read below sees the whole wire
for (;;) {
const int n = (int) recv(sv[1], wire + got, (int) (sizeof(wire) - got), 0);
if (n <= 0)
break;
got += (size_t) n;
}
deletesoc(sv[1]);
assertf(got == strlen(expect));
assertf(memcmp(wire, expect, got) == 0);
printf("ftp-ctrlchars self-test OK (%d bytes sent, %d rejected)\n", (int) got,
(int) dropped);
return 0;
}
/* Slurp a whole file into a malloc'd buffer; sets *len. NULL on error. */
static unsigned char *warc_slurp(const char *path, size_t *len) {
FILE *f = FOPEN(path, "rb");
@@ -7909,6 +8051,9 @@ static const struct selftest_entry {
"local save-name for a URL", st_savename},
{"sniff", "<content-type> <hex:..|text>", "MIME magic consistency",
st_sniff},
{"escape-control", "[hex:..|string]",
"escape_remove_control() terminates at the compacted end",
st_escape_control},
{"fsize", "<dir>", "file size past the 2GB signed-32-bit wrap", st_fsize},
{"growsize", "", "buffer capacity for a 64-bit file size (no int wrap)",
st_growsize},
@@ -7973,6 +8118,8 @@ static const struct selftest_entry {
{"ftp-line", "", "get_ftp_line bounds a hostile FTP reply line",
st_ftpline},
{"ftp-userpass", "", "ftp_split_userpass bounds URL userinfo", st_ftpuser},
{"ftp-ctrlchars", "", "send_line rejects a control byte in an FTP command",
st_ftpctrl},
{"warc", "<dir>", "WARC/1.1 writer: framing, digests, revisit dedup",
st_warc},
{"warc-trunc", "<dir>", "WARC-Truncated on a cap-truncated body",

View File

@@ -366,6 +366,25 @@ typedef struct {
so anything near this is already invalid and is rejected unread. */
#define SID_VALUE_MAX 64
/** Does this Origin name the panel itself?
Only our own plain-http authority passes: a sandboxed page sends "null", a
foreign one its own host, and an empty Host cannot be matched at all. */
static hts_boolean origin_is_self(const char *origin, const char *host) {
const int p = strfield(origin, "http://");
const char *const authority = origin + p;
return host[0] != '\0' && p != 0 && strfield2(authority, host) != 0;
}
/** Header value with leading blanks dropped, clipped to fit dst. */
static void copy_header_value(char *dst, size_t size, const char *value) {
while (*value == ' ' || *value == '\t') {
value++;
}
dst[0] = '\0';
strlncatbuff(dst, value, size, size - 1);
}
/** Does the urlencoded request body present the expected session id?
True only if at least one "sid" field is present and every occurrence
matches, so it holds whichever one a later last-write-wins parse keeps.
@@ -444,6 +463,27 @@ static hts_boolean cat_html_escaped(String *dst, char c) {
return HTS_TRUE;
}
/* Same, for a double-quoted attribute: the quote included, which
cat_html_escaped() leaves raw for the single-quoted tooltips. */
static hts_boolean cat_attr_escaped_char(String *dst, char c) {
if (c == '\"') {
StringCat(*dst, "&#34;");
return HTS_TRUE;
}
return cat_html_escaped(dst, c);
}
/* Append value escaped for a double-quoted HTML attribute. */
static void cat_attr_escaped(String *dst, const char *value) {
const char *a;
for (a = value; *a != '\0'; a++) {
if (!cat_attr_escaped_char(dst, *a)) {
StringMemcat(*dst, a, 1);
}
}
}
/* Append value escaped for a single-quoted JS literal inside a double-quoted
HTML attribute. Every escape is a \xNN group, so the only bytes it adds are
'\', 'x' and hex digits: nothing the attribute decode can expand back into a
@@ -578,9 +618,13 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
LLint length = 0;
const char *error_redirect = NULL;
hts_boolean denied = HTS_FALSE;
char origin[256];
char host[256];
line[0] = '\0';
buffer[0] = '\0';
origin[0] = '\0';
host[0] = '\0';
StringClear(headers);
StringClear(output);
StringClear(tmpbuff);
@@ -642,6 +686,10 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
tmp[0] = '\0';
strncatbuff(tmp, s, 2);
/*l = LANG_SEARCH(path, tmp); */
} else if ((p = strfield(line, "Origin:")) != 0) {
copy_header_value(origin, sizeof(origin), line + p);
} else if ((p = strfield(line, "Host:")) != 0) {
copy_header_value(host, sizeof(host), line + p);
}
}
if (meth == 2) {
@@ -689,6 +737,16 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
}
/* CSP stops a mirrored page reading /server/, not posting to it blind:
a no-cors POST still runs the command. Origin is browser-set and script
cannot forge it. Absent is allowed, most non-browser clients send none.
*/
if (meth == 2 && origin[0] != '\0' && !origin_is_self(origin, host)) {
buffer[0] = '\0';
meth = 0;
denied = HTS_TRUE;
}
/* Authenticate the body before parsing it: every field it carries is
written straight into the global key store below, "command" included,
and that one reaches the engine. Checking afterwards cannot work — the
@@ -1118,6 +1176,8 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
int format = 0;
int listDefault = 0;
hts_boolean unquoted = HTS_FALSE;
/* value comes from the template, not from the settings */
hts_boolean literal = HTS_FALSE;
name[0] = '\0';
strlncatbuff(name, str, sizeof(name_), n);
@@ -1127,6 +1187,9 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
} else if ((p = strfield(name, "html:"))) {
name += p;
format = 1;
} else if ((p = strfield(name, "attr:"))) {
name += p;
format = 7;
} else if ((p = strfield(name, "js:"))) {
name += p;
format = 6;
@@ -1147,6 +1210,7 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
name += p;
format = 0;
literal = HTS_TRUE;
pos2 = strchr(name, ':');
langstr = "";
if (pos2 != NULL) {
@@ -1276,6 +1340,7 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
int ztest = (name[0] == 'z');
langstr = "";
literal = HTS_TRUE;
name += p;
pos2 = strchr(name, ':');
if (pos2 != NULL) {
@@ -1386,8 +1451,10 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
const char *a = langstr;
while(*a) {
if (a[0] == '\\' && isxdigit(a[1])
&& isxdigit(a[2])) {
/* the ini writer has no inverse for it, so a lone
backslash in a settings value must stay one */
if (literal && a[0] == '\\' && isxdigit(a[1]) &&
isxdigit(a[2])) {
int n;
char c;
@@ -1396,10 +1463,11 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
StringMemcat(output, &c, 1);
}
a += 2;
} else if (unquoted && a[0] == '\"') {
/* the browser posts an entity back as a raw
quote, which would open a quoted run in the
argv splitter; a URI cannot hold one anyway */
} else if ((unquoted || outputmode == 3) &&
a[0] == '\"') {
/* an entity decodes back to a quote, which opens
a quoted run in the argv splitter or ends the
attribute the URL sits in; no URI holds one */
StringCat(output, "%22");
} else if (outputmode &&
cat_html_escaped(&output, a[0])) {
@@ -1431,6 +1499,9 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
case 6:
cat_js_escaped(&output, langstr);
break;
case 7:
cat_attr_escaped(&output, langstr);
break;
default:
if (*langstr) {
int id = 1;
@@ -1472,7 +1543,11 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
StringClear(tmpbuff);
break;
default:
if (!cat_html_escaped(&tmpbuff, *fstr)) {
/* format -2 writes its value into the option's
value="" as well, so the quote must go too */
if (!(format == -2
? cat_attr_escaped_char(&tmpbuff, *fstr)
: cat_html_escaped(&tmpbuff, *fstr))) {
StringMemcat(tmpbuff, fstr, 1);
}
break;
@@ -1524,6 +1599,13 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
type != NULL ? type : "application/octet-stream");
StringCat(headers, "\r\n");
}
if (virtualpath) {
/* No allow-same-origin: an opaque origin keeps script in a
crawled page from reading the session id out of /server/ */
StringCat(headers, "Content-Security-Policy: sandbox "
"allow-scripts allow-forms allow-popups "
"allow-downloads\r\n");
}
while(!feof(fp)) {
int n = (int) fread(line, 1, sizeof(line) - 2, fp);
@@ -1555,7 +1637,8 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
StringCat(headers, "HTTP/1.0 403 Forbidden\r\n"
"Server: httrack small server\r\n"
"Content-type: text/html\r\n");
StringCat(output, "Missing or invalid session id.\r\n");
StringCat(output,
"Missing or invalid session id, or foreign origin.\r\n");
} else {
#ifdef _DEBUG
char error_hdr[] =
@@ -1574,12 +1657,12 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
StringCat(headers, tmp);
}
StringCat(headers, "\r\n");
/* a refusal cleared meth, yet the Content-length above promises a body */
if ((send(soc_c, StringBuff(headers), (int) StringLength(headers), 0) !=
StringLength(headers))
|| ((meth == 1)
&& (send(soc_c, StringBuff(output), (int) StringLength(output), 0)
!= StringLength(output)))
) {
StringLength(headers)) ||
((meth == 1 || denied) &&
(send(soc_c, StringBuff(output), (int) StringLength(output), 0) !=
StringLength(output)))) {
#ifdef _DEBUG
#endif
}

View File

@@ -98,6 +98,19 @@ typedef struct hts_thread_s {
void (*fun) (void *arg);
} hts_thread_s;
/* Set once before any thread is spawned, hence unlocked. */
static void *(*thread_enter)(void) = NULL;
static void (*thread_leave)(void *cookie) = NULL;
HTSEXT_API void hts_set_thread_hooks(void *(*enter)(void),
void (*leave)(void *cookie)) {
/* Never half a pair: 'leave' must not see a cookie no 'enter' produced. */
const int paired = enter != NULL && leave != NULL;
thread_enter = paired ? enter : NULL;
thread_leave = paired ? leave : NULL;
}
#ifdef _WIN32
static unsigned int __stdcall hts_entry_point(void *tharg)
#else
@@ -107,11 +120,15 @@ static void *hts_entry_point(void *tharg)
hts_thread_s *s_args = (hts_thread_s *) tharg;
void *const arg = s_args->arg;
void (*fun) (void *arg) = s_args->fun;
void *cookie;
freet(tharg);
cookie = thread_enter != NULL ? thread_enter() : NULL;
/* run */
fun(arg);
if (thread_leave != NULL)
thread_leave(cookie);
process_chain_add(-1);
#ifdef _WIN32

View File

@@ -64,6 +64,12 @@ struct htsmutex_s {
/* Library internal definictions */
HTSEXT_API int hts_newthread(void (*fun) (void *arg), void *arg);
/* Extends per-thread state to the workers: 'enter' runs at each one's start,
'leave' at its end with the cookie 'enter' returned. Set before spawning; a
NULL in either clears the pair, since neither hook is useful alone. */
HTSEXT_API void hts_set_thread_hooks(void *(*enter)(void),
void (*leave)(void *cookie));
HTSEXT_API void htsthread_wait_n(int n_wait);
/* Locking functions */

View File

@@ -101,6 +101,50 @@ static void htsweb_sig_brpipe(int code) {
/* ignore */
}
#ifdef _WIN32
/* RtlGenRandom, resolved at runtime so no import library is needed. */
typedef BOOLEAN(WINAPI *hts_rtlgenrandom_t)(PVOID buffer, ULONG length);
#endif
/* Fill buffer with system entropy; HTS_FALSE if no source answered. There is
deliberately no weak fallback: the caller mints a secret with it. */
static hts_boolean hts_random_bytes(void *buffer, size_t size) {
#ifdef _WIN32
hts_boolean ok = HTS_FALSE;
HMODULE dll = LoadLibraryA("advapi32.dll");
if (dll != NULL) {
hts_rtlgenrandom_t gen =
(hts_rtlgenrandom_t) GetProcAddress(dll, "SystemFunction036");
/* the ULONG cast must not truncate; callers ask for a few dozen bytes */
if (gen != NULL && size <= 0x10000 && gen(buffer, (ULONG) size)) {
ok = HTS_TRUE;
}
FreeLibrary(dll);
}
return ok;
#else
unsigned char *dst = (unsigned char *) buffer;
size_t done = 0;
FILE *fp = fopen("/dev/urandom", "rb");
if (fp == NULL) {
return HTS_FALSE;
}
while (done < size) {
const size_t n = fread(dst + done, 1, size - done, fp);
if (n == 0) { /* short read is a hard failure, not partial credit */
break;
}
done += n;
}
fclose(fp);
return done == size ? HTS_TRUE : HTS_FALSE;
#endif
}
/* Threads that never return; no wait may count on them draining. */
static int nonjoinable_threads = 0;
@@ -247,14 +291,19 @@ int main(int argc, char *argv[]) {
smallserver_setkey("HTTRACK_INCOMPATIBLE_VERSIONID", hts_version());
}
/* protected session-id */
/* Session id: the only thing authenticating a command, so it is seeded from
the system CSPRNG. A clock-derived one is guessable from the "Mirrored
from" stamp every mirrored page carries (#877). */
{
char buff[1024];
unsigned char seed[32];
char digest[32 + 2];
srand((unsigned int) time(NULL));
snprintf(buff, sizeof(buff), "%d-%d", (int) time(NULL), (int) rand());
domd5mem(buff, strlen(buff), digest, 1);
if (!hts_random_bytes(seed, sizeof(seed))) {
fprintf(stderr,
"** CRITICAL: no system entropy source to build a session id\n");
return -1;
}
domd5mem((const char *) seed, sizeof(seed), digest, 1);
smallserver_setkey("sid", digest);
smallserver_setkey("_sid", digest);
}

View File

@@ -521,6 +521,11 @@ HTSEXT_API size_t escape_uri_utf(const char *const src, char *const dest,
HTSEXT_API size_t escape_check_url(const char *const src, char *const dest,
const size_t size);
/** Percent-escape the C0 control bytes of @p src (0x00-0x1f) and copy every
other byte verbatim, so bytes >= 0x7f keep the UTF-8 they carry. */
HTSEXT_API size_t escape_control_url(const char *const src, char *const dest,
const size_t size);
/** Append-variant of escape_spc_url(): escapes @p src after the existing
NUL-terminated content of @p dest. Returns the bytes appended (excluding the
NUL). */
@@ -572,7 +577,8 @@ HTSEXT_API size_t make_content_id(const char *const adr, const char *const fil,
/** Low-level percent-escaper backing the escape_* family. @p mode selects the
character class to escape: 0 check_url, 1 in_url, 2 spc_url, 3 uri,
30 uri_utf. @p max_size is the dest capacity including the NUL. */
4 control_url, 30 uri_utf. @p max_size is the dest capacity including the
NUL. */
HTSEXT_API size_t x_escape_http(const char *const s, char *const dest,
const size_t max_size, const int mode);

View File

@@ -47,6 +47,7 @@ Please visit our Website: http://www.httrack.com
#include "htslib.h"
#include "htscharset.h" // after htslib.h: winsock2.h must precede windows.h
#include "htsbacktrace.h"
#include "htsthread.h"
/* Static definitions */
static int fexist(const char *s);
@@ -909,7 +910,7 @@ static void sig_fatal(int code) {
size += print_num(&buffer[size], code);
buffer[size++] = '\n';
(void) (write(FD_ERR, buffer, size) == size);
hts_print_backtrace(FD_ERR);
hts_print_backtrace();
(void) (write(FD_ERR, msgreport, sizeof(msgreport) - 1)
== sizeof(msgreport) - 1);
abort();
@@ -950,8 +951,9 @@ static void install_fatal_handler(int code) {
static void signal_handlers(void) {
hts_backtrace_init();
/* Main thread only: engine threads keep the old behaviour. */
(void) hts_backtrace_altstack();
/* The crawl recurses in the engine's workers, so they need one too (#969). */
hts_set_thread_hooks(hts_backtrace_altstack, hts_backtrace_altstack_release);
#ifdef _WIN32
signal(SIGINT, sig_leave); // ^C
signal(SIGTERM, sig_finish); // kill <process>

17
src/libhttrack.pc.in Normal file
View File

@@ -0,0 +1,17 @@
prefix=@prefix@
exec_prefix=@exec_prefix@
libdir=@libdir@
includedir=@includedir@
Name: libhttrack
Description: HTTrack Website Copier engine
URL: @PACKAGE_URL@
Version: @PACKAGE_VERSION@
# Flat, as the installed headers and the libtest examples include each other; same
# flags libtest itself builds with. config.h, not these, carries the ABI gating.
Cflags: -I${includedir}/httrack @V6_FLAG@ @LFS_FLAG@
# Carries its own leading space, and is empty wherever configure gates the rpath off (#978).
Libs: -L${libdir} -lhttrack@PKGCONFIG_RPATH_LDFLAG@
# Ordered for an archive link: a library ahead of the ones it uses.
Requires.private: @PKGCONFIG_REQUIRES_PRIVATE@
Libs.private: @PKGCONFIG_LIBS_PRIVATE@ @LIBS@ @THREADS_LIBS@ @DL_LIBS@ @SOCKET_LIBS@ @ICONV_LIBS@

View File

@@ -17,8 +17,8 @@
#endif
VS_VERSION_INFO VERSIONINFO
FILEVERSION 3, 49, 16, 0
PRODUCTVERSION 3, 49, 16, 0
FILEVERSION 3, 49, 17, 0
PRODUCTVERSION 3, 49, 17, 0
FILEFLAGSMASK VS_FFI_FILEFLAGSMASK
#ifdef _DEBUG
FILEFLAGS VS_FF_DEBUG
@@ -35,12 +35,12 @@ BEGIN
BEGIN
VALUE "CompanyName", "Xavier Roche"
VALUE "FileDescription", VER_FILE_DESCRIPTION
VALUE "FileVersion", "3.49.16"
VALUE "FileVersion", "3.49.17"
VALUE "InternalName", VER_ORIGINAL_FILENAME
VALUE "LegalCopyright", "Copyright (C) 1998-2026 Xavier Roche and other contributors. GNU GPL v3 or later."
VALUE "OriginalFilename", VER_ORIGINAL_FILENAME
VALUE "ProductName", "HTTrack Website Copier"
VALUE "ProductVersion", "3.49-16"
VALUE "ProductVersion", "3.49-17"
END
END
BLOCK "VarFileInfo"

View File

@@ -102,6 +102,43 @@ HTTRACK_TEST_TIMEOUT=0 bash "$driver" "$tmp/93_off.test" >"$out" 2>&1 || rc=$?
test "$rc" -eq 3 || fail "disabled guard changed the exit status ($rc)"
grep -q 'guard off' "$out" || fail "output lost with the guard disabled"
# --- the test reads the same budget the guard enforces ----------------------
# skip_if_out_of_budget paces against it, so a raw or absent value would pace
# against a number the guard is not using.
saw_budget() { # saw_budget <want> <label>
grep -qx "budget=$1" "$out" ||
fail "$2 reached the test as '$(cat "$out")', want budget=$1"
}
# shellcheck disable=SC2016 # the fixture has to read the variable, not us
printf 'echo "budget=${HTTRACK_TEST_TIMEOUT-unset}"\n' >"$tmp/95_budget.test"
(unset HTTRACK_TEST_TIMEOUT && bash "$driver" "$tmp/95_budget.test") >"$out" 2>&1
saw_budget 600 "an unset budget"
HTTRACK_TEST_TIMEOUT=garbage bash "$driver" "$tmp/95_budget.test" >"$out" 2>&1
saw_budget 600 "a non-numeric budget"
HTTRACK_TEST_TIMEOUT=45 bash "$driver" "$tmp/95_budget.test" >"$out" 2>&1
saw_budget 45 "an explicit budget"
# 0 execs the test directly, so the pacer has to see the guard is off too.
HTTRACK_TEST_TIMEOUT=0 bash "$driver" "$tmp/95_budget.test" >"$out" 2>&1
saw_budget 0 "a disabled guard"
# --- a test too slow to finish skips instead of being killed ----------------
# hppa spends ~150s on one configure run, and 124 takes the build down where 77
# does not.
cat >"$tmp/94_pacer.test" <<EOF
. "${testdir}/testlib.sh"
skip_if_out_of_budget 0 9999 # nothing left to run
skip_if_out_of_budget 5 0 # steps that cost nothing
skip_if_out_of_budget 5 40 # the next step alone wants 60s
echo "the pacer let it through"
EOF
rc=0
HTTRACK_TEST_TIMEOUT=20 bash "$driver" "$tmp/94_pacer.test" >"$out" 2>&1 || rc=$?
test "$rc" -eq 77 || fail "a 60s step under a 20s budget reported $rc, want 77"
rc=0
HTTRACK_TEST_TIMEOUT=600 bash "$driver" "$tmp/94_pacer.test" >"$out" 2>&1 || rc=$?
test "$rc" -eq 0 || fail "a 60s step under a 600s budget reported $rc, want 0"
grep -q 'the pacer let it through' "$out" || fail "the pacer skipped a test that fits"
# --- a wedged crawl yields a symbolized engine stack ------------------------
# The whole point of the dump: name the frame the engine is stuck in. Windows has
# neither half (MSYS signals do not reach a native httrack.exe, and that build has

View File

@@ -6,6 +6,10 @@
set -euo pipefail
testdir=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
sh=${BASH_SHELL:-}
test -n "$sh" || {
echo "BASH_SHELL is empty; tests/Makefile.am must export the configured value" >&2
@@ -50,11 +54,12 @@ mkfifo "$tmp/fifo"
chmod 755 "$tmp/fifo"
n=0
cases=16 # reject/accept calls below; pinned again once they have all run
status=0
log=
rundir=
run() { # run <label> <env argument>...
local label=$1
local label=$1 began=$SECONDS
shift
n=$((n + 1))
rundir="$tmp/run$n"
@@ -78,6 +83,7 @@ run() { # run <label> <env argument>...
wait "$pid" || status=$?
log=$(cat "$rundir/log")
echo "run $n ($label): exit $status"
skip_if_out_of_budget "$((cases - n))" "$((SECONDS - began))"
}
reject() { # reject <label> <expected message> <env argument>...
@@ -163,4 +169,8 @@ accept empty '' '' BASH_SHELL=
accept searched "$tmp/fakebin/bash" 'no usable bash found' -u BASH_SHELL "PATH=$tmp/fakebin:$PATH"
accept searched-posix-env '' 'POSIXLY_CORRECT or SHELLOPTS' -u BASH_SHELL POSIXLY_CORRECT=1
test "$n" -eq "$cases" || {
echo "ran $n cases, not the $cases the budget is paced against" >&2
exit 1
}
echo "configure validated $n BASH_SHELL values"

View File

@@ -0,0 +1,41 @@
#!/bin/bash
#
# escape_remove_control() left a dropped byte's tail glued to the result (#974).
# Vectors are hex: raw control bytes do not survive a Windows argv.
set -euo pipefail
fail() {
echo "FAIL: $1" >&2
exit 1
}
# input : expected length : expected output
cases=(
"2f610b6263:4:2f616263" # VT mid-path, the shape the parser can reach
"0c616263:3:616263" # FF first: every byte moves
"6162630b:3:616263" # VT last: nothing moves, but the end does
"61626301646566:6:616263646566"
"010203:0:" # all control
"616263:3:616263" # untouched: the terminator must stay put
":0:"
"617fc3a9:4:617fc3a9" # DEL and high bytes are not stripped
"612062:3:612062" # space is the first byte kept
"611f621e63:3:616263" # top of the control range, and two bytes dropped
)
for c in "${cases[@]}"; do
in=${c%%:*}
rest=${c#*:}
len=${rest%%:*}
out=${rest#*:}
got=$(httrack "-#test=escape-control" "hex:$in") ||
fail "hex:$in exited non-zero"
test "$got" = "escape-control: len=$len out=hex:$out" ||
fail "hex:$in: expected len=$len out=hex:$out, got: $got"
done
# The in-binary table also checks the bytes past the NUL, which printing cannot show.
got=$(httrack "-#test=escape-control") || fail "self-test exited non-zero"
grep -q "escape-control self-test OK" <<<"$got" ||
fail "unexpected self-test output: $got"

View File

@@ -8,7 +8,7 @@ testdir=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
builddir="${abs_top_builddir:?not run under make check}"
: "${abs_top_builddir:?not run under make check}"
bindir="${CONFIGURED_BINDIR:?not run under make check}"
libdir="${CONFIGURED_LIBDIR:?not run under make check}"
@@ -37,11 +37,8 @@ trap 'rm -rf "${work}"' HUP INT QUIT PIPE TERM
# The installed ELF, not the build tree's libtool wrapper: the wrapper re-execs
# its binary by absolute path, which is the argv[0] this test must not have.
stage=${work}/stage
env -u MAKEFLAGS -u MAKELEVEL "${MAKE:-make}" -C "${builddir}/src" install-exec \
DESTDIR="${stage}" >"${work}/install.log" 2>&1 || {
cat "${work}/install.log" >&2
stage_install_exec "${stage}" "${work}/install.log" ||
fail "make install-exec DESTDIR=${stage}"
}
bin=${stage}${bindir}/httrack
test -r "${bin}" || fail "no httrack in ${stage}${bindir}"
@@ -63,7 +60,8 @@ fi
# Control: without a bare argv[0] in the raw trace there is nothing to repair,
# and the assertions below would pass on a report that never needed the fix.
grep -qE '^httrack\(\+0x[0-9a-f]+\)' "${out}" ||
# glibc drops the "+0x" offset when the load bias is zero, as in a non-PIE build.
grep -qE '^httrack\((\+0x[0-9a-f]+)?\)' "${out}" ||
skip "the main program is not reported as a bare argv[0] here"
headers=$(grep -E '^/.*/httrack:$' "${out}" || true)

View File

@@ -0,0 +1,119 @@
#!/bin/bash
# Issue #982: a control byte inside a link used to end the URL where it stood,
# or, for VT and FF, be deleted from it, so the crawler fetched a URL the page
# never wrote. It must percent-encode instead, as a browser does. Each hostile
# href also has its old truncation and deletion targets on the server, so a
# mirror holding either one names which rewrite came back.
set -eu
: "${top_srcdir:=..}"
testdir=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
# The fixtures carry the raw bytes the requests decode back to, which NTFS
# refuses.
! is_windows || ! echo "control bytes are illegal in a Windows filename" >&2 ||
exit 77
tmpdir=$(mktemp -d "${TMPDIR:-/tmp}/httrack_982.XXXXXX") || exit 1
serverpid=
cleanup() {
stop_server "$serverpid"
rm -rf "$tmpdir"
}
trap 'set +e; cleanup' EXIT
trap cleanup HUP INT QUIT PIPE TERM
fail() {
echo "FAIL: $1"
exit 1
}
doc="${tmpdir}/doc"
mkdir -p "$doc"
# What the links must reach, named with the raw byte the server decodes back to.
vt=$(printf 'a\013bc.html')
ff=$(printf 'd\014ef.html')
soh=$(printf 's\001oh.html')
# CR, LF and TAB are stripped, not encoded, so this href must reach strip.html.
strip=$(printf 's\011tr\015ip\012.html')
# A control byte at either end is stripped like a space, so these two must reach
# the plain names: encoding one there would 404 a link that fetches fine.
trail=$(printf 'page.html\001')
lead=$(printf '\037lead.html')
# "abc.html" and "s" are what the old deletion and truncation would have hit.
for f in clean.html abc.html def.html s d strip.html page.html lead.html \
"$vt" "$ff" "$soh"; do
printf 'body of %s\n' "$f" >"${doc}/${f}"
done
# Stripping both ends empties this one, which resolves to "./" and mirrors the
# directory as index-2.html. The leading strip has to stop on the terminator the
# trailing strip just wrote, or it reads on into the last link's leftovers and
# fetches those.
allctrl=$(printf '\013\013\013')
printf '<html><body>\n<a href="clean.html">c</a>\n<a href="%s">v</a>\n<a href="%s">f</a>\n<a href="%s">s</a>\n<a href="%s">t</a>\n<a href="%s">e</a>\n<a href="%s">l</a>\n<a href="%s">a</a>\n<a href="big.html">b</a>\n</body></html>\n' \
"$vt" "$ff" "$soh" "$strip" "$trail" "$lead" "$allctrl" >"${doc}/index.html"
# 700 control bytes encode to 2100, past the 2*HTS_URLMAXSIZE link buffer, so
# the growth no longer fits and the link has to go rather than be clipped.
{
printf '<html><body><a href="big/'
awk 'BEGIN { while (i++ < 700) printf "%c", 11 }'
printf 'end.html">x</a></body></html>\n'
} >"${doc}/big.html"
# clean.html is the positive control: an ordinary link still crawls untouched,
# and the three encoded fetches are what a browser would have requested.
bash "$top_srcdir/tests/local-crawl.sh" --root "$(nativepath "$doc")" \
--errors 0 --files 10 \
--found 'clean.html' \
--found 'strip.html' \
--found 'page.html' \
--found 'lead.html' \
--found 'index-2.html' \
--found 'a%0bbc.html' \
--found 'd%0cef.html' \
--found 's%01oh.html' \
--not-found 'abc.html' \
--not-found 'def.html' \
--not-found 's' \
--not-found 'd' \
--log-found 'link rejected \(control bytes do not fit\)' \
httrack 'BASEURL/index.html' --debug-log
# The query is escaped separately and appended back, so its own growth has to
# fit too. Absolute, because a clipped relative link dies on the URL-length gate
# anyway while a clipped absolute one lands just under it and gets fetched. The
# port is only known once the server is up, hence a second crawl of our own.
python=$(find_python) || ! echo "python3 not found; skipping" >&2 || exit 77
qdoc="${tmpdir}/qdoc"
mkdir -p "$qdoc"
printf 'placeholder\n' >"${qdoc}/index.html"
serverlog="${tmpdir}/server.log"
: >"$serverlog"
LOCAL_SERVER_VERBOSE=1 "$python" "$(nativepath "${testdir}/local-server.py")" \
--root "$(nativepath "$qdoc")" >"$serverlog" 2>&1 &
serverpid=$!
port=$(discover_server_port "$serverlog" "$serverpid") || exit 1
# 300 control bytes of path escape to ~900, under the URL-length gate; 380 more
# of query escape to ~1140, which together overrun the 2 KB link buffer.
{
printf '<html><body><a href="http://127.0.0.1:%s/a' "$port"
awk 'BEGIN { while (i++ < 300) printf "%c", 11 }'
printf 'b.html?q='
awk 'BEGIN { while (i++ < 380) printf "%c", 11 }'
printf 'END">x</a></body></html>\n'
} >"${qdoc}/index.html"
out="${tmpdir}/qout"
httrack "http://127.0.0.1:${port}/index.html" -O "$out" -q -r3 -%v0 ||
fail "the query crawl did not run"
# A clipped link went out as a 2 KB request and 404ed, so assert the outcome
# and not just the refusal: nothing oversized asked for, nothing errored.
errors=$(grep -acE '^[0-9:]*[[:space:]]Error:' "${out}/hts-log.txt" || true)
test "$errors" -eq 0 || fail "the oversized query was fetched ($errors errors)"
longest=$(awk '{ if (length($0) > n) n = length($0) } END { print n + 0 }' \
"$serverlog")
test "$longest" -lt 1024 || fail "a ${longest}-byte request line went out"

View File

@@ -0,0 +1,76 @@
#!/bin/bash
#
# A link carrying &#13;&#10; keeps raw CR/LF past the parser's control-byte
# escape, so a request field built from it can start a header line of the
# site's choosing. Graded on the wire bytes: a log cannot show a split line.
set -euo pipefail
: "${top_srcdir:=..}"
# shellcheck source=tests/testlib.sh
. "$top_srcdir/tests/testlib.sh"
python=$(find_python) || {
echo "python3 missing, skipping"
exit 77
}
server=$(nativepath "$top_srcdir/tests/header-injection-server.py")
checker=$(nativepath "$top_srcdir/tests/header-injection-check.py")
tmpdir=$(mktemp -d)
serverpid=
cleanup() {
stop_server "$serverpid"
rm -rf "$tmpdir"
}
trap 'set +e; cleanup' EXIT
trap cleanup HUP INT QUIT PIPE TERM
start_probe() {
local tag="$1"
"$python" "$server" "$tmpdir/$tag.wire" >"$tmpdir/$tag.out" 2>"$tmpdir/$tag.err" &
serverpid=$!
port=$(discover_server_port "$tmpdir/$tag.out" "$serverpid") || exit 1
}
# a crawl may end non-zero, but never on the deadline or on a signal: an
# assertf abort would otherwise be graded on whatever it managed to send
crawl() {
local rc=0
run_with_timeout 90 httrack "$@" >>"$tmpdir/log" 2>&1 || rc=$?
test "$rc" -ne 124 || {
echo "crawl hit the deadline" >&2
exit 1
}
test "$rc" -le 128 || {
echo "crawl ended abnormally ($rc)" >&2
exit 1
}
}
# 1. Referer: the poisoned page links onward, so its own URL is the referer of
# the next request. No proxy involved; this is a plain crawl.
start_probe direct
crawl "http://127.0.0.1:$port/" -O "$tmpdir/m1" \
-c1 --robots=0 --retries=0 --quiet -%v0
stop_server "$serverpid"
serverpid=
"$python" "$checker" "$tmpdir/direct.wire" direct "$port" || {
cat "$tmpdir/log" >&2
exit 1
}
# 2. Host, the proxy absolute-URI request line and its ftp:// variant. Only
# reachable through an http proxy: without one the poisoned authority fails to
# resolve and no request is ever built.
start_probe proxy
crawl "http://start.example/" '+*' -P "127.0.0.1:$port" \
-O "$tmpdir/m2" -c1 --robots=0 --retries=0 --quiet -%v0
stop_server "$serverpid"
serverpid=
"$python" "$checker" "$tmpdir/proxy.wire" proxy || {
cat "$tmpdir/log" >&2
exit 1
}

View File

@@ -54,6 +54,11 @@ deepest_repeat() {
sort <<<"$addrs" | uniq -c | sort -rn | awk 'NR==1 { print $1 + 0 }'
}
# Raw frames in the report: backtrace_symbols_fd() closes every one with [0xADDR].
frame_count() {
grep -c '\[0x' <<<"$1" || true
}
# Control: an ordinary SIGSEGV was already reported before the fix, so a build
# or harness that can no longer see any crash fails here rather than passing the
# real case vacuously.
@@ -67,10 +72,24 @@ fi
grep -q "^Caught signal 11$" <<<"$segv" || fail "-#c=segv: no 'Caught signal 11' line"
grep -q "^Caught signal 11$" <<<"$stack" || fail "-#c=stack: no 'Caught signal 11' line"
# Measured on x86-64: ~250 repeats for the recursion, 1 for the control.
# Measured on x86-64: recursion 256 frames/~250 repeats, control 11/1.
stack_deep=$(deepest_repeat "$stack")
segv_deep=$(deepest_repeat "$segv")
stack_frames=$(frame_count "$stack")
# armhf and loong64 cannot unwind out of the frame that faulted on the guard page,
# leaving the handler's own three frames and nothing to count. An absolute floor,
# not a compare against the control: the two call chains differ by a frame that
# only -O2 tail-calls away, and a crash_stack that merely stopped recursing still
# unwinds 11. Killing that mutant is what the count below is for.
if [ "$stack_frames" -lt 6 ]; then
echo "-#c=stack: $stack_frames frames, too few for this unwinder to" \
"show the recursion:" >&2
head -5 <<<"$stack" >&2
exit 0
fi
test "$stack_deep" -ge 20 ||
fail "-#c=stack: deepest frame repeats $stack_deep times, expected a recursion"
fail "-#c=stack: deepest frame repeats $stack_deep times over $stack_frames frames," \
"expected a recursion"
test "$segv_deep" -lt 20 ||
fail "-#c=segv: deepest frame repeats $segv_deep times, the threshold no longer discriminates"

View File

@@ -0,0 +1,74 @@
#!/bin/bash
# -#c=atfork reproduces the #968 regime; the timeout turns a wedge into a failure.
set -euo pipefail
ulimit -c 0 # a deliberate crash must not litter the box with cores
fail() {
echo "$*" >&2
exit 1
}
# The spawn needs backtrace(), so Linux is necessary but not sufficient: the
# runtime notice below is what actually rules a musl build out.
if [ "$(uname -s)" != "Linux" ]; then
echo "the symbolizer spawn is Linux-only, skipping" >&2
exit 77
fi
command -v httrack >/dev/null || fail "could not find httrack"
# A wedged httrack still catches SIGTERM and only sets a flag, hence -s KILL.
command -v timeout >/dev/null || fail "no timeout(1): a hang would never end here"
# 134 is SIGABRT, the handler's own last statement; 137 is the wedge.
crash_report() {
local kind="$1" out rc=0
out=$(timeout -s KILL 60 httrack "-#c=$kind" 2>&1) || rc=$?
test "$rc" -ne 137 || fail "-#c=$kind wedged: the handler never came back"
test "$rc" -eq 134 || fail "-#c=$kind exited $rc, expected 134"
grep -q "Please report the problem" <<<"$out" ||
fail "-#c=$kind: report truncated, the handler died halfway"
printf '%s\n' "$out"
}
# Control: an ordinary fault walks the same spawn path with no lock held, so a
# build that cannot report at all fails here rather than passing the real case.
plain=$(crash_report segv)
if grep -q 'No stack trace available on this OS' <<<"$plain"; then
echo "no backtrace() in this build; skipping" >&2
exit 77
fi
# 32-bit ARM may still build with no unwind tables, as test 80 allows for: an
# empty trace has no module block to assert on.
case "$(uname -m)" in
arm | armv*)
if grep -q 'No stack trace available' <<<"$plain"; then
echo "no unwind tables in this ARM build; skipping" >&2
exit 77
fi
;;
esac
locked=$(crash_report atfork)
# The module header is written just before the spawn: no header, no code under test.
if command -v addr2line >/dev/null || command -v llvm-symbolizer >/dev/null; then
grep -qE '^/.+:$' <<<"$plain" || fail "-#c=segv: no symbolizer was spawned"
grep -qE '^/.+:$' <<<"$locked" ||
fail "-#c=atfork: no symbolizer was spawned, the deadlock went untested"
else
echo "no addr2line and no llvm-symbolizer: spawn path not exercised" >&2
fi
# addr2line runs first, and its -a address line is what proves the child ran.
# Unanchored width: -a pads to the target pointer size, 8 nibbles on 32-bit.
if command -v addr2line >/dev/null; then
# A name, not just the shape: an all-"??" regression still emits the offsets.
grep -qE '^0x[0-9a-f]+: [A-Za-z_]' <<<"$locked" ||
fail "-#c=atfork: no symbolized frame, addr2line never ran"
# The child's stdout is redirected onto the report fd, prebuilt at startup:
# symbolized frames belong on stderr, never on the crawler's own stdout.
leaked=$(timeout -s KILL 60 httrack '-#c=segv' 2>/dev/null |
grep -cE '^0x[0-9a-f]+: ' || true)
test "$leaked" -eq 0 || fail "$leaked symbolized frames leaked to stdout"
fi

149
tests/183_altstack-worker.test Executable file
View File

@@ -0,0 +1,149 @@
#!/bin/bash
# A worker that ran out of stack was killed outright with no report: #866 gave
# the alternate signal stack to the main thread, and sigaltstack() is per-thread
# (#969). Second half of the file: the stacks have to be given back too.
set -euo pipefail
ulimit -c 0 # a deliberate crash must not litter the box with cores
fail() {
echo "$*" >&2
exit 1
}
case "$(uname -s)" in
Linux) traced=1 ;;
MINGW* | MSYS* | CYGWIN*)
echo "no sigaltstack on Windows, skipping" >&2
exit 77
;;
# sigaltstack() is POSIX, so the fix applies; only the trace text below is
# gated on Linux + execinfo.h (USES_BACKTRACE in htsbacktrace.c).
*) traced=0 ;;
esac
command -v httrack >/dev/null || fail "could not find httrack"
# Dies from a caught fault, running the report to its end. The handler finishes
# in abort(), so the shell sees SIGABRT (134); 139 is the kernel killing us.
crash_report() {
local kind="$1" out rc=0
out=$(httrack "-#c=$kind" 2>&1) || rc=$?
test "$rc" -eq 134 ||
fail "-#c=$kind exited $rc, expected 134 (139 means the handler never ran)"
grep -q "Please report the problem" <<<"$out" ||
fail "-#c=$kind: report truncated, the handler died halfway"
printf '%s\n' "$out"
}
# Frames the raw trace printed. A runaway recursion fills backtrace()'s buffer;
# an ordinary fault is a dozen deep. Measured: 256 against 11.
frame_count() {
grep -cE '\[0x[0-9a-f]+\]$' <<<"$1" || true
}
# Control: an ordinary fault was already reported before the fix, so a build or
# harness that can no longer see any crash fails here rather than passing the
# real case vacuously.
plain=$(crash_report segv)
worker=$(crash_report threadstack)
grep -q "Crash test worker thread started" <<<"$worker" ||
fail "-#c=threadstack: no worker was spawned, nothing was tested"
if [ "$traced" -eq 0 ]; then
exit 0
fi
grep -q "^Caught signal 11$" <<<"$worker" || fail "-#c=threadstack: no 'Caught signal 11' line"
# The main thread is parked in htsthread_wait_n() once the worker announced
# itself, and only the worker recurses. A trace that deep is therefore the
# worker's, which is what had no alternate stack.
worker_frames=$(frame_count "$worker")
plain_frames=$(frame_count "$plain")
test "$worker_frames" -ge 100 ||
fail "-#c=threadstack: $worker_frames frames, expected a runaway recursion"
test "$plain_frames" -lt 100 ||
fail "-#c=segv: $plain_frames frames, the threshold no longer discriminates"
# Everything above only proves the stack gets installed: the crashing worker
# never returns, so the release hook never runs there. -#test=threadwait spawns
# workers that do return, and the syscall trace says what each did with its
# stack -- 64kB a worker, and an unmap the kernel is not told about first.
[ -r "${ALTSTACKPROBE_LA:-}" ] || fail "${ALTSTACKPROBE_LA:-\$ALTSTACKPROBE_LA} was not built"
if grep -q "^dlname=''" "$ALTSTACKPROBE_LA"; then
echo "static-only build, no release trace" >&2
exit 0
fi
[ -r "${ALTSTACKPROBE_LIB:-}" ] || fail "${ALTSTACKPROBE_LIB:-\$ALTSTACKPROBE_LIB} was not built"
tmp=$(mktemp -d)
trap 'set +e; rm -rf "$tmp"' EXIT
trap 'exit 1' HUP INT TERM
# See 181: the shim loads ahead of libasan and allocates nothing.
export ASAN_OPTIONS="${ASAN_OPTIONS:+$ASAN_OPTIONS:}verify_asan_link_order=0"
trace="$tmp/altstack.trace"
rc=0
out=$(ALTSTACK_TRACE="$trace" LD_PRELOAD="$ALTSTACKPROBE_LIB" \
httrack -#test=threadwait 2>&1) || rc=$?
test "$rc" -eq 0 || fail "-#test=threadwait exited $rc: $out"
grep -q "threadwait self-test: OK" <<<"$out" ||
fail "-#test=threadwait did not report OK, no workers to judge: $out"
[ -s "$trace" ] || fail "the probe traced no sigaltstack() call at all"
# Pairs each thread's events: the unmap of one of our stacks has to come
# straight after its SS_DISABLE, and no worker may end still holding one.
read -r foreign workers released errors main_has <<<"$(awk '
{
key = $1 ":" $2
is_main = ($1 == $2)
if (is_main && $4 != "off" && $4 != "-") { main_has = 1 }
if ($3 == "query") {
# Installed before anyone traced doing it: not ours either.
if (!(key in seen) && $4 == "on") { alien[key] = 1 }
prev[key] = "query " $4
} else if ($3 == "set" && $4 != "off") {
if ($4 == "own") { own[key] = $5; main_of[key] = is_main }
else { alien[key] = 1 }
prev[key] = "set-on " $5
} else if ($3 == "set") {
prev[key] = "set-off " $5
} else if ($3 == "munmap") {
if (prev[key] != "set-off " $5) {
print "thread " key ": unmapped " $5 " with the kernel still pointed at it" > "/dev/stderr"
errors++
}
if (is_main) {
print "thread " key ": the main thread gave up its alternate stack" > "/dev/stderr"
errors++
}
if (own[key] == $5) { freed[key] = 1 }
prev[key] = "munmap " $5
}
seen[key] = 1
}
END {
for (k in alien) { foreign++ }
for (k in own) {
if (!main_of[k]) { workers++; if (freed[k]) released++ }
}
print foreign+0, workers+0, released+0, errors+0, main_has+0
}' "$trace")"
# A sanitizer runtime installs its own in every thread, which we then leave
# alone: nothing of ours to release, so nothing to judge. Cannot swallow a plain
# build, where no alternate stack predates hts_backtrace_altstack().
if [ "$foreign" -gt 0 ] && [ "$workers" -lt 8 ]; then
echo "$foreign thread(s) had an alternate stack already, no release trace" >&2
exit 0
fi
test "$errors" -eq 0 || fail "$errors alternate-stack lifecycle violation(s), listed above"
test "$main_has" -eq 1 || fail "the main thread ran with no alternate stack at all"
test "$workers" -ge 8 ||
fail "only $workers worker(s) got an alternate stack, too few to judge the release"
test "$released" -eq "$workers" ||
fail "$((workers - released)) of $workers workers ended without giving their alternate stack back"

View File

@@ -22,43 +22,180 @@ python=$(find_python) || {
exit 77
}
printf '[each template puts js: in handlers and html: in text] ..\t'
"${python}" - "${distdir}" <<'PY' || fail "the templates mix the two escaping modes (see above)"
printf '[each template escapes for the position it interpolates into] ..\t'
"${python}" - "${distdir}" <<'PY' || fail "the templates mix the escaping modes (see above)"
import glob, os, re, sys
# A handler value is JS, everything else is HTML text: the two never swap.
handler = re.compile(r'\bon[A-Za-z]+="([^"]*)"')
handler = re.compile(r"on[A-Za-z]+$")
script = re.compile(r"<script\b[^>]*>(.*?)</script>", re.S | re.I)
subst = re.compile(r"\$\{([^}]*)\}")
# Directives expand to template-written markup, or (arg:) escape the value
# themselves. unquoted: only rewrites ", so it needs the mode like a bare key.
directive = re.compile(r"/\*|^(do|test|ztest|checked|list|liststr|listid|fexist"
r"|file-exists|arg):")
# Shipped translations and build-time constants; everything else is runtime data.
constant = re.compile(r"^(LANG_|LANGUAGE_|HTTRACK_|HTS_PLATFORM)")
# One attribute; an unquoted value ends on whitespace, so it closes as easily.
attribute = re.compile(r"""\s*(?P<name>[A-Za-z][A-Za-z0-9:_.-]*)
(?:\s*=\s*(?:"(?P<dq>[^"]*)"|'(?P<sq>[^']*)'
|(?P<uq>[^\s"'=<>`]+)))?""", re.X)
files = sorted(glob.glob(os.path.join(sys.argv[1], "html", "server", "*.html")))
def attributes(text):
"""(kind, name, start, end) per attribute value; tags are walked in order, so
a > inside a quoted value cannot end its tag early."""
i, n = 0, len(text)
while i < n:
i = text.find("<", i)
if i < 0:
return
j = i + (2 if text[i:i + 2] == "</" else 1)
if j >= n or not text[j].isalpha():
i += 1
continue
while j < n and (text[j].isalnum() or text[j] in ":_-"):
j += 1
while j < n and text[j] != ">":
m = attribute.match(text, j)
if m is None or m.end() == j:
j += 1
continue
for kind in ("dq", "sq", "uq"):
if m.group(kind) is not None:
yield kind, m.group("name"), m.start(kind), m.end(kind)
j = m.end()
i = j + 1
def scan(name, text):
"""(complaints, per-class site counts) for one template."""
bad, seen = [], {}
attrs = {"dq": [], "sq": [], "uq": [], "handler": []}
for kind, aname, start, end in attributes(text):
attrs["handler" if handler.match(aname) else kind].append((start, end))
scripts = [m.span(1) for m in script.finditer(text)]
def count(cls):
seen[cls] = seen.get(cls, 0) + 1
# Every value on the engine command line is a shell token: a quoted one takes
# arg:, which escapes " and \, a bare one unquoted:. html: escapes neither.
for ta in re.finditer(r'<textarea name="command".*?</textarea>', text, re.S):
count("cmdline")
for c in subst.finditer(ta.group(0)):
if c.group(1).startswith("html:"):
bad.append("%s: command line interpolates ${%s}"
% (name, c.group(1)))
# ${do:output-mode:X} escapes every bare value until the next end-if clears
# it, which is how a <pre> or <textarea> keeps its newlines (src/htsserver.c).
mode, pending = "", []
for m in subst.finditer(text):
key = m.group(1)
if key.startswith("do:output-mode:"):
newmode = key[len("do:output-mode:"):]
elif key.startswith("do:end-if"):
# end-if clears the mode rather than restoring it: the rest of an
# output-mode region that holds one renders raw.
newmode = ""
else:
newmode = mode
if mode == "html-urlescaped" and newmode != mode:
pending = []
mode = newmode
if directive.match(key):
continue
# counted after the skip, so re-adding unquoted: to it trips the floor
if key.startswith("unquoted:"):
count("unquoted")
def within(kind):
return any(a <= m.start() and m.end() <= b for a, b in attrs[kind])
if within("handler"):
count("handler")
# Only js: escapes the quotes and backslashes a JS literal ends on;
# ${LANG_x} and ${html:LANG_x} both reach the JS parser as HTML text.
if not key.startswith("js:"):
bad.append("%s: handler interpolates ${%s}" % (name, key))
elif any(a <= m.start() and m.end() <= b for a, b in scripts):
count("script")
# a script block is JS the moment the tags are stripped, handler or not
if not key.startswith("js:"):
bad.append("%s: script block interpolates ${%s}" % (name, key))
elif within("dq") or within("uq"):
count("attr")
# html: keeps the double quote that ends the attribute, so only attr:
# is safe here; html-urlescaped is, because it %22-encodes it.
if key.startswith("attr:"):
pass
elif mode != "html-urlescaped":
bad.append("%s: attribute interpolates ${%s}" % (name, key))
else:
# held back: the exemption is only real if the region is closed
pending.append("%s: attribute interpolates ${%s} in an"
" output-mode region the file never closes"
% (name, key))
elif within("sq"):
count("sqattr")
# a bare key keeps the apostrophe that ends this one
if not key.startswith(("html:", "attr:", "js:")):
bad.append("%s: attribute interpolates ${%s}" % (name, key))
elif key.startswith(("js:", "attr:")):
bad.append("%s: HTML text interpolates ${%s}" % (name, key))
elif key.startswith("html:"):
count("text")
elif not constant.match(key) and not mode:
bad.append("%s: element body interpolates ${%s} unescaped" % (name, key))
else:
count("body")
return bad + pending, seen
# A scan that silently stops parsing reads as clean, so every form the
# classifier is meant to see has to fire on a synthetic template first.
for what, text in [
("a bare key in a double-quoted attribute", '<input value="${LANG_X}">'),
("spaces around the =", '<input value = "${LANG_X}">'),
("an unquoted attribute", "<input value=${LANG_X}>"),
("a single-quoted attribute", "<input title='${LANG_X}'>"),
("a value holding the tag delimiter", '<input value=" >> ${LANG_X} ">'),
("a value split across lines", '<input value="a\n${LANG_X}">'),
("a dotted key", '<input value="${attr.LANG_X}">'),
("html: in a double-quoted attribute", '<input value="${html:LANG_X}">'),
("attr: outside an attribute", "<p>${attr:LANG_X}</p>"),
("a bare key in a handler", "<a onClick=\"f('${LANG_X}')\">"),
("an unclosed output-mode region",
'${do:output-mode:html-urlescaped}\n<input value="${LANG_X}">')]:
if not scan("control.html", text)[0]:
sys.exit("the scanner does not see %s" % what)
for what, text in [
("attr:", '<input value="${attr:LANG_X}">'),
("html: in a single-quoted attribute", "<input title='${html:LANG_X}'>"),
("a closed output-mode region",
'${do:output-mode:html-urlescaped}\n<input value="${LANG_X}">\n'
"${do:output-mode:}")]:
if scan("control.html", text)[0]:
sys.exit("the scanner rejects %s" % what)
files = sorted(glob.glob(os.path.join(sys.argv[1], "html", "server", "**",
"*.html"), recursive=True))
if len(files) < 20:
sys.exit("only %d server templates found" % len(files))
bad, in_handler, in_text = [], {}, 0
bad, seen = [], {}
for path in files:
name = os.path.basename(path)
text = open(path, encoding="latin-1").read()
spans = [m.span(1) for m in handler.finditer(text)]
for m in subst.finditer(text):
key = m.group(1)
if any(a <= m.start() and m.end() <= b for a, b in spans):
in_handler[name] = in_handler.get(name, 0) + 1
# ${LANG_x}, ${html:LANG_x} and ${html:html:LANG_x} all reach the JS
# parser as HTML text; only the last renders empty instead of broken.
if re.match(r"(html:)*LANG_", key):
bad.append("%s: handler interpolates ${%s}" % (name, key))
else:
if key.startswith("js:"):
bad.append("%s: HTML text interpolates ${%s}" % (name, key))
elif key.startswith("html:"):
in_text += 1
b, s = scan(os.path.basename(path), open(path, encoding="latin-1").read())
bad += b
for cls, n in s.items():
seen[cls] = seen.get(cls, 0) + n
if bad:
sys.exit("\n".join(bad))
# Neither leg may go vacuous; step2.html is named because it is the one the
# ${html:html:} spelling hid in.
if sum(in_handler.values()) < 300 or in_text < 100 or in_handler.get("step2.html", 0) < 8:
sys.exit("scanned too little: %d handler (%d in step2.html), %d text"
% (sum(in_handler.values()), in_handler.get("step2.html", 0), in_text))
# The property above is the assertion; this only catches a leg going empty.
if seen.get("cmdline") != 1 or not all(
seen.get(cls) for cls in ("handler", "script", "attr", "sqattr", "text",
"body", "unquoted")):
sys.exit("scanned too little: %r" % seen)
PY
echo OK

View File

@@ -0,0 +1,131 @@
#!/bin/bash
#
# The progress panel prints URLs a crawled site chose, so a link path holding
# < > or ' must reach the browser escaped for the position it lands in (#973).
set -euo pipefail
testdir=$(cd "$(dirname "$0")" && pwd)
distdir=${top_srcdir:-$(cd "${testdir}/.." && pwd)}
distdir=$(cd "${distdir}" && pwd)
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
fail() {
echo "FAIL: $*" >&2
exit 1
}
command -v htsserver >/dev/null || fail "no htsserver in PATH"
python=$(find_python) || {
echo "python3 not found; skipping" >&2
exit 77
}
log=$(mktemp)
work=$(mktemp -d)
csrv=
# start() runs in a command substitution, so its $! never reaches this shell.
srvpid() { sed -n 's/^PID=//p' "${log}" 2>/dev/null | head -1; }
cleanup() {
local pid
pid=$(srvpid)
test -z "${pid}" || kill -9 "${pid}" 2>/dev/null || true
test -z "${csrv}" || kill -9 "${csrv}" 2>/dev/null || true
wait "${csrv}" 2>/dev/null || true # absorb bash's async "Killed" notice
rm -rf "${log}" "${work}"
}
trap 'set +e; cleanup' EXIT
trap cleanup HUP INT QUIT PIPE TERM
# An isolated HOME keeps a developer's ~/.httrack.ini out of the served settings.
export HOME="${work}"
port=$("${python}" -c 'import socket
s = socket.socket()
s.bind(("127.0.0.1", 0))
print(s.getsockname()[1])
s.close()')
(
trap '' TERM TTOU
exec htsserver "${distdir}/" --port "${port}" >"${log}" 2>&1
) &
for _ in $(seq 1 40); do
url=$(sed -n 's/^URL=//p' "${log}" 2>/dev/null) && test -n "${url}" && break
sleep 0.25
done
test -n "${url:-}" || fail "htsserver did not come up: $(cat "${log}")"
clog="${work}/content.log"
"${python}" "${testdir}/local-server.py" --root "${work}" >"${clog}" 2>&1 &
csrv=$!
cport=$(discover_server_port "${clog}" "${csrv}") ||
fail "content server did not come up"
printf '[a hostile link path is escaped in the progress panel] ..\t'
"${python}" - "${port}" "${cport}" "${work}" <<'PY' || fail "the panel leaks the crawled URL (see above)"
import re, sys, time, urllib.parse, urllib.request
port, cport, work = sys.argv[1:4]
base = "http://127.0.0.1:%s" % port
form = urllib.request.urlopen(base + "/server/index.html", timeout=20).read()
m = re.search(rb'name="sid" value="([0-9a-f]+)"', form)
if not m:
sys.exit("no session id in server/index.html")
sid = m.group(1).decode()
# /xssjob/ links one .bin whose directory and leaf both carry the metacharacters
# and trickles it, so the job sits in the transfer table while the panel renders.
fields = {
"sid": sid, "path": work, "projname": "crawl", "winprofile": "x",
"command_do": "start",
"command": "httrack --quiet --robots=0 -c1 http://127.0.0.1:%s/xssjob/ -O %s/crawl"
% (cport, work),
}
urllib.request.urlopen(urllib.request.Request(
base + "/step4.html", data=urllib.parse.urlencode(fields).encode(),
method="POST"), timeout=60).read()
# The panel is the page any .html request is overridden to while a crawl runs.
PAYLOAD = "img src=x onerror=alert(1)"
# Either form: breaking on the escaped one alone would make the control circular.
populated = re.compile(r"d0(&lt;|<)i")
deadline = time.time() + 90
while True:
body = urllib.request.urlopen(base + "/server/refresh.html",
timeout=20).read().decode("latin-1")
if populated.search(body):
break
if time.time() > deadline:
sys.exit("the hostile job never reached the panel:\n%s" % body[-4000:])
time.sleep(1)
# Positive control: assert the escaped form too, or an empty table passes below.
# Name and file are pinned apart, since either column can regress on its own.
for want in ("d0&lt;i&#39;&gt;", "p0&lt;" + PAYLOAD + "&gt;&#39;quote.bin"):
if want not in body:
sys.exit("panel does not escape the link path as %r:\n%s" % (want, body[-4000:]))
for raw in ("d0<i", "<" + PAYLOAD):
if raw in body:
sys.exit("panel emits %r as raw markup" % raw)
# The save name reaches a single-quoted JS literal: url_savename() maps < and >
# to _ but keeps the apostrophe, which closes the literal unless it is escaped.
handlers = [h for h in re.findall(r'onClick="([^"]*)"', body)
if h.startswith("form.command.value='cancel-file=")]
if not handlers:
sys.exit("no per-row cancel handler in the panel")
hit = 0
for h in handlers:
m = re.fullmatch(r"form\.command\.value='([^']*)'; form\.submit\(\)", h)
if not m:
sys.exit("handler is not a single JS literal: %r" % h)
if "quote.bin" in m.group(1):
hit += 1
if r"\x27quote.bin" not in m.group(1):
sys.exit("save name keeps a bare apostrophe: %r" % m.group(1))
if hit == 0:
sys.exit("no handler carried the hostile save name:\n%s" % body[-4000:])
PY
echo OK

View File

@@ -1,10 +1,14 @@
#!/bin/bash
# An install copied away from its configured prefix still has to find
# libhttrack, which it does through a binary-relative rpath (#906).
# libhttrack, which it does through a binary-relative rpath (#906). On Darwin
# that also takes an @rpath install_name (#978).
set -euo pipefail
builddir="${abs_top_builddir:?not run under make check}"
# shellcheck source=tests/testlib.sh
. "$(dirname "$0")/testlib.sh"
: "${abs_top_builddir:?not run under make check}"
bindir="${CONFIGURED_BINDIR:?not run under make check}"
libdir="${CONFIGURED_LIBDIR:?not run under make check}"
libdir=${libdir%/}
@@ -24,15 +28,9 @@ work=$(mktemp -d "${TMPDIR:-/tmp}/relocate.XXXXXX") || {
}
trap 'set +e; rm -rf "${work}"' EXIT
# DESTDIR keeps the configured prefix, so the absolute rpath libtool baked in
# still points outside the staged tree, and libtool has no reason to relink
# while the rest of the suite runs against the same build directory.
stage=${work}/stage
env -u MAKEFLAGS -u MAKELEVEL "${MAKE:-make}" -C "${builddir}/src" install-exec \
DESTDIR="${stage}" >"${work}/install.log" 2>&1 || {
cat "${work}/install.log" >&2
stage_install_exec "${stage}" "${work}/install.log" ||
fail "make install-exec DESTDIR=${stage}"
}
bin=${stage}${bindir}/httrack
test -r "${bin}" || fail "no httrack in ${stage}${bindir}"

View File

@@ -1,9 +1,14 @@
#!/bin/bash
# The binary-relative rpath must stay off where it is unwanted (#906): a
# distribution libdir the loader already searches, and an explicit opt-out.
# distribution libdir the loader already searches, and an explicit opt-out. The
# rpath libhttrack.pc hands a consumer answers to the same gates (#978).
set -euo pipefail
testdir=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
srcdir="${abs_top_srcdir:?not run under make check}"
test -r "${srcdir}/configure" || {
echo "no configure in ${srcdir}; skipping" >&2
@@ -35,11 +40,12 @@ test -r "${shadow}/configure" || fail "the shadow source tree has no configure"
# not depend on the prefix.
cache=${work}/config.cache
n=0
cases=6 # expect calls below, what the budget is paced against
expect() {
local want=$1 desc=$2
shift 2
local dir got
local dir got line began=$SECONDS
n=$((n + 1))
dir=${work}/build${n}
mkdir -p "${dir}"
@@ -56,16 +62,20 @@ expect() {
grep -i 'binary-relative' "${dir}/configure.log" >&2 || true
fail "${desc}: ORIGIN_RPATH is '${got}', expected '${want}'"
}
# Same answer here: no case below turns on the $ORIGIN probe, which the .pc's
# absolute rpath does not need anyway.
line=$(sed -n 's/^Libs: //p' "${dir}/src/libhttrack.pc")
test -n "${line}" || fail "${desc}: libhttrack.pc carries no Libs line"
case "${line}" in
*-rpath*) got=yes ;;
*) got=no ;;
esac
test "${got}" = "${want}" ||
fail "${desc}: libhttrack.pc rpath is '${got}', expected '${want}': ${line}"
echo "ok: ${desc} -> ${want}"
skip_if_out_of_budget "$((cases - n))" "$((SECONDS - began))"
}
# Darwin has no positive case: configure turns the rpath off there whatever the
# prefix, so assert that one row rather than run "no" rows that cannot fail.
if [ "$(uname -s)" = Darwin ]; then
expect no "Darwin opts out whatever the prefix" --prefix="${work}/inst"
exit 0
fi
# The positive control first, or every "no" below could come from a probe that
# simply never succeeds on this machine.
expect yes "a private prefix gets the rpath" --prefix="${work}/inst"
@@ -74,3 +84,5 @@ expect no "a multiarch libdir does not" --prefix=/usr --libdir=/usr/lib/x86_64-l
expect no "a 32-bit multilib libdir does not" --prefix=/usr --libdir=/usr/lib32
expect no "--disable-origin-rpath is honoured" --prefix="${work}/inst" --disable-origin-rpath
expect no "a static build does not" --prefix="${work}/inst" --disable-shared
test "${n}" -eq "${cases}" || fail "ran ${n} cases, not the ${cases} the budget is paced against"

View File

@@ -0,0 +1,158 @@
#!/bin/bash
#
# An installed header must not offer a name libhttrack.so keeps hidden: the
# consumer compiles, then fails to link (#977). Linked for real, not grepped.
# Blind to a wrong signature, which the linker cannot see either.
set -euo pipefail
fail() {
echo "FAIL: $1" >&2
exit 1
}
skip() {
echo "$1, skipping" >&2
exit 77
}
# Unset means no automake run (the Windows job runs the scripts directly), so
# there is no install rule to drive.
[ -n "${abs_top_builddir:-}" ] || skip "abs_top_builddir unset, no automake environment"
[ -f "$abs_top_builddir/src/Makefile" ] || fail "$abs_top_builddir/src is not configured"
lib=${HTTRACK_SHLIB:-}
[ -f "${lib:-/nonexistent}" ] || skip "no shared libhttrack at '${lib:-unset}'"
# CC carries flags on some legs ("gcc -m32", "ccache gcc"), so split it.
read -r -a cc_argv <<<"${CC:-cc}"
command -v "${cc_argv[0]}" >/dev/null 2>&1 || skip "no C compiler (${cc_argv[0]})"
make=${MAKE:-make}
command -v "$make" >/dev/null 2>&1 || skip "no make ($make)"
nm=${NM:-nm}
command -v "$nm" >/dev/null 2>&1 || skip "no nm ($nm)"
tmp=$(mktemp -d) || exit 1
trap 'set +e; rm -rf "$tmp"' EXIT
# ELF only: BSD nm has no -D.
"$nm" -D --defined-only "$lib" >"$tmp/exported.raw" 2>/dev/null ||
skip "$nm cannot read the dynamic table of $lib"
"$nm" --defined-only "$lib" >"$tmp/all.raw" 2>/dev/null ||
skip "$nm cannot read the symbol table of $lib"
awk 'NF >= 3 { print $3 }' "$tmp/exported.raw" | sort -u >"$tmp/exported"
awk 'NF >= 3 { print $3 }' "$tmp/all.raw" | sort -u >"$tmp/all"
comm -23 "$tmp/all" "$tmp/exported" >"$tmp/hidden"
n_exported=$(wc -l <"$tmp/exported")
n_hidden=$(wc -l <"$tmp/hidden")
[ "$n_exported" -ge 50 ] || fail "$lib exports only $n_exported symbols, that cannot be right"
[ "$n_hidden" -ge 100 ] || skip "only $n_hidden local symbols in $lib, it looks stripped"
cpp_argv=(-I"$tmp/include")
if [ -n "${TEST_CPPFLAGS:-}" ]; then
read -r -a extra_argv <<<"$TEST_CPPFLAGS"
cpp_argv+=("${extra_argv[@]}")
fi
# volatile, or the compiler folds "address of a known symbol is non-null" to 1
# and drops the relocation the link is being asked about.
probe_body() {
printf 'int main(void) { const void *volatile p = (const void *) &%s; return p != 0; }\n' "$1"
}
# A function declaration is a lie the linker ignores, and stays PIE-clean
# whichever kind $1 is. Compiled apart, so a malformed probe is not a link error.
links() {
{
printf 'extern void %s(void);\n' "$1"
probe_body "$1"
} >"$tmp/ctl.c"
"${cc_argv[@]}" -w -c -o "$tmp/ctl.o" "$tmp/ctl.c" 2>"$tmp/ctl.log" || {
head -5 "$tmp/ctl.log" >&2
fail "the control probe for $1 does not compile"
}
"${cc_argv[@]}" -w -o "$tmp/ctl" "$tmp/ctl.o" "$lib" 2>"$tmp/ctl.log"
}
# Without both controls every verdict below is vacuous. Compiler-generated
# statics ("a.2") are not spellable in C, so skip past them.
pos=$(grep -x hts_is_available "$tmp/exported" || head -1 "$tmp/exported")
neg=$(awk '/^[A-Za-z_][A-Za-z0-9_]*$/ { print; exit }' "$tmp/hidden")
links "$pos" || {
head -5 "$tmp/ctl.log" >&2
skip "cannot link the exported $pos against $lib"
}
! links "$neg" || fail "the hidden $neg links anyway, so the probe cannot see the defect"
# Override the install dir rather than DESTDIR: same rule and file list, no
# prefix guessing. Empty MAKEFLAGS, or the sub-make hunts a jobserver it lacks.
env MAKEFLAGS= MFLAGS= "$make" -C "$abs_top_builddir/src" install-DevIncludesDATA \
DevIncludesdir="$tmp/include/httrack" >"$tmp/install.log" 2>&1 ||
{
cat "$tmp/install.log" >&2
fail "install-DevIncludesDATA failed"
}
# A synthetic leak the loop must report: every real candidate is a static defined
# in the header, which links from the probe's own copy and so proves nothing.
canary=zzz-canary.h
printf 'extern void %s(void);\n' "$neg" >"$tmp/include/httrack/$canary"
headers=("$tmp/include/httrack"/*.h)
[ "${#headers[@]}" -ge 10 ] || fail "only ${#headers[@]} headers installed, the list cannot be right"
# Identifiers of the preprocessed unit, narrowed by cpp's line markers to the
# installed headers, so <openssl/ssl.h> and friends stay out of the candidates.
cat >"$tmp/ids.awk" <<'EOF'
/^# [0-9]+ "/ {
s = $0
sub(/^# [0-9]+ "/, "", s)
sub(/".*$/, "", s)
ours = (index(s, PREFIX) == 1)
next
}
ours {
while (match($0, /[A-Za-z_][A-Za-z0-9_]*/)) {
print substr($0, RSTART, RLENGTH)
$0 = substr($0, RSTART + RLENGTH)
}
}
EOF
probed=0
leaks=""
for h in "${headers[@]}"; do
b=$(basename "$h")
# config.h first, as a consumer must: it is what turns HTS_USEOPENSSL on, and
# without it htsbasenet.h preprocesses its OpenSSL half away.
printf '#include <httrack/config.h>\n#include <httrack/%s>\n' "$b" >"$tmp/tu.c"
"${cc_argv[@]}" "${cpp_argv[@]}" -UHTS_INTERNAL_BYTECODE -E -o "$tmp/tu.i" "$tmp/tu.c" \
2>"$tmp/cc.log" || {
head -5 "$tmp/cc.log" >&2
fail "$b does not preprocess the way a consumer includes it"
}
awk -v PREFIX="$tmp/include/httrack/" -f "$tmp/ids.awk" "$tmp/tu.i" | sort -u >"$tmp/ids"
while read -r sym; do
[ -n "$sym" ] || continue
{
printf '#include <httrack/config.h>\n#include <httrack/%s>\n' "$b"
probe_body "$sym"
} >"$tmp/probe.c"
# Undeclared means the header never put it on the consumer's surface.
"${cc_argv[@]}" "${cpp_argv[@]}" -UHTS_INTERNAL_BYTECODE -w -c -o "$tmp/probe.o" \
"$tmp/probe.c" 2>/dev/null || continue
probed=$((probed + 1))
"${cc_argv[@]}" -w -o "$tmp/probe" "$tmp/probe.o" "$lib" 2>/dev/null ||
leaks="$leaks $b:$sym"
done < <(comm -12 "$tmp/hidden" "$tmp/ids")
done
echo "linked $probed reachable symbol(s) from ${#headers[@]} installed headers" \
"($n_hidden hidden, $n_exported exported)"
[ "$probed" -ge 5 ] || fail "only $probed symbols reached the link probe, the candidate list is broken"
[ "${leaks#* "$canary":"$neg"}" != "$leaks" ] ||
fail "the synthetic $canary:$neg leak went unreported, the candidate list is broken"
leaks=${leaks/ "$canary":"$neg"/}
[ -z "$leaks" ] || fail "installed headers declare symbols $lib does not export:$leaks"
exit 0

View File

@@ -0,0 +1,8 @@
#!/bin/bash
#
set -euo pipefail
# send_line() drops an FTP command carrying a control byte (#1010).
out=$(httrack -O /dev/null -#test=ftp-ctrlchars run)
grep -q "ftp-ctrlchars self-test OK" <<<"$out"

View File

@@ -0,0 +1,214 @@
#!/bin/bash
#
# The settings pages put engine values and translations inside double-quoted
# attributes, so a " or a < in one must arrive as an entity (#989).
set -euo pipefail
testdir=$(cd "$(dirname "$0")" && pwd)
distdir=${top_srcdir:-$(cd "${testdir}/.." && pwd)}
distdir=$(cd "${distdir}" && pwd)
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
fail() {
echo "FAIL: $*" >&2
exit 1
}
command -v htsserver >/dev/null || fail "no htsserver in PATH"
python=$(find_python) || {
echo "python3 not found; skipping" >&2
exit 77
}
work=$(mktemp -d "${TMPDIR:-/tmp}/webhttrack_attresc.XXXXXX") || fail "no tmpdir"
srvlog=$(mktemp)
srv=
srvpid=
cleanup() {
# htsserver keeps SIGTERM ignored across its exec, so only -9 reaps it.
test -z "${srvpid}" || kill -9 "${srvpid}" 2>/dev/null || true
test -z "${srv}" || kill -9 "${srv}" 2>/dev/null || true
wait "${srv}" 2>/dev/null || true # absorb bash's async "Killed" notice
rm -rf "${work}" "${srvlog}"
}
trap 'set +e; cleanup' EXIT
trap cleanup HUP INT QUIT PIPE TERM
# A private data root, so LANGUAGE_2 can carry the hostile strings. distcheck's
# srcdir is read-only and cp keeps that mode.
root="${work}/root"
mkdir -p "${root}"
cp -R "${distdir}/html" "${distdir}/lang" "${distdir}/lang.def" "${root}/"
chmod -R u+w "${root}"
# Latin-1, like every shipped translation: an escaper that entity-encodes bytes
# >= 0x80 has to fail here rather than read as clean.
eacute=$(printf '\351')
# CRLF line pairs, English key then translation. "OK" lands in a value="
# attribute, "Click to confirm" in a single-quoted title=.
printf '%s\r\n' \
LANGUAGE_NAME 'Francais-989' \
LANGUAGE_FILE 'Francais' \
LANGUAGE_ISO 'fr' \
LANGUAGE_CHARSET 'iso-8859-1' \
OK "Va \"y\" <ok> d${eacute}j${eacute}" \
'Click to confirm' "Cliquez \"ici\" <maintenant> d${eacute}j${eacute}" \
Cancel 'Annuler' \
>"${root}/lang/Francais.txt"
# A project whose directory name and Category= line both hold a quote.
projects="${work}/projects"
mkdir -p "${projects}/p\"q/hts-cache"
printf 'Category=c%%22d\r\n' >"${projects}/p\"q/hts-cache/winprofile.ini"
# Loaded on the first request; a Windows path, so the render has to leave the
# backslash alone for what the browser posts back to rewrite the same file.
printf 'path=C:\\ab\r\nlang=\r\n' >"${work}/.httrack.ini"
printf '[a hostile settings value is entity-escaped in its attribute] ..\t'
sport=$("${python}" -c 'import socket
s = socket.socket()
s.bind(("127.0.0.1", 0))
print(s.getsockname()[1])
s.close()')
(
trap '' TERM TTOU
# An isolated HOME, so the ~/.httrack.ini above is the only one served.
export HOME="${work}"
exec htsserver "${root}/" --port "${sport}" >"${srvlog}" 2>&1
) &
srv=$!
for _ in $(seq 1 40); do
url=$(sed -n 's/^URL=//p' "${srvlog}") && test -n "${url}" && break
kill -0 "${srv}" 2>/dev/null || break
sleep 0.25
done
test -n "${url:-}" || fail "htsserver did not start: $(cat "${srvlog}")"
srvpid=$(sed -n 's/^PID=//p' "${srvlog}") # absent on Windows
"${python}" - "${url}" "${projects}" <<'PY' || fail "an attribute takes the value raw (see above)"
import html, re, sys, urllib.parse, urllib.request
url = sys.argv[1].rstrip("/")
projects = sys.argv[2]
form = urllib.request.urlopen(url + "/server/index.html", timeout=20).read()
m = re.search(rb'name="sid" value="([0-9a-f]+)"', form)
if not m:
sys.exit("no session id in server/index.html")
sid = m.group(1).decode()
# The quote ends the attribute and the rest is a working injection; each field
# gets its own leader, so one field echoing another's value cannot pass.
BREAK = "\"><img src=q onerror=alert(1)>&'"
def escaped(value):
for c, e in (("&", "&amp;"), ("<", "&lt;"), (">", "&gt;"), ("'", "&#39;"),
("\"", "&#34;")):
value = value.replace(c, e)
return value
def render(page, fields):
# lang=2 selects LANGUAGE_2 and stores the posted fields in one shot.
fields = dict(fields, sid=sid, lang="2")
req = urllib.request.Request(
url + "/server/" + page,
data=urllib.parse.urlencode(fields).encode("latin-1"), method="POST")
resp = urllib.request.urlopen(req, timeout=20)
if resp.status != 200:
sys.exit("%s answered %d" % (page, resp.status))
body = resp.read().decode("latin-1")
if 'lang="fr"' not in body:
sys.exit("%s did not switch to LANGUAGE_2:\n%s" % (page, body[:2000]))
return body
pages = [
("option6.html", {"user": "u" + BREAK, "footer": r"Mozilla/5.0 C:\ab"}, [
# a settings field the local user types
'name="user" value="%s"' % escaped("u" + BREAK),
# negative control: an ordinary value is not rewritten, and the mode
# does not decode \ab into one byte the way a bare ${key} does
r'name="footer" value="Mozilla/5.0 C:\ab"',
# a translation lands in a value= attribute as well, high bytes and all
'value="%s"' % escaped("Va \"y\" <ok> d\xe9j\xe9"),
# ... and in a single-quoted title=, which keeps the quote raw
"title='Cliquez \"ici\" &lt;maintenant&gt; d\xe9j\xe9'",
]),
# a second template, and two fields in it: a per-site sweep miss shows up
# nowhere else
("option11.html", {"ext1": "e" + BREAK, "mime1": "m" + BREAK}, [
'name="ext1" value="%s"' % escaped("e" + BREAK),
'name="mime1" value="%s"' % escaped("m" + BREAK),
]),
("option8.html", {"cookiesfile": "c" + BREAK}, [
'name="cookiesfile" value="%s"' % escaped("c" + BREAK),
]),
]
for page, fields, wants in pages:
body = render(page, fields)
for want in wants:
if want not in body:
sys.exit("%s does not emit %r:\n%s" % (page, want, body[:4000]))
if BREAK in body or "<ok>" in body:
sys.exit("%s emits the value as raw markup" % page)
# every attribute the templates open must still be closed by the next quote
for value in re.findall(r'\b[A-Za-z][A-Za-z0-9:_-]*="([^"]*)"', body):
if "<" in value or ">" in value:
sys.exit("%s leaves markup in the attribute %r" % (page, value))
def attr(body, page, pattern):
m = re.search(pattern, body, re.S)
if not m:
sys.exit("%s emits no %s:\n%s" % (page, pattern, body[:4000]))
return m.group(1)
# The browser posts the attribute back decoded, so an escaper that is stable on
# display can still lose the value on the next submit.
shown = attr(render("option6.html", {"user": "u" + BREAK}), "option6.html",
r'name="user" value="([^"]*)"')
if html.unescape(shown) != "u" + BREAK:
sys.exit("option6.html value decodes to %r" % html.unescape(shown))
again = attr(render("option6.html", {"user": html.unescape(shown)}),
"option6.html", r'name="user" value="([^"]*)"')
if again != shown:
sys.exit("resubmitting the value gives %r, not %r" % (again, shown))
# ~/.httrack.ini holds a Windows path; the ini reader has no backslash escape,
# so both the display and the textarea the browser posts back must keep it.
body = render("step2.html", {})
if attr(body, "step2.html", r'name="path" value="([^"]*)"') != r"C:\ab":
sys.exit("step2.html mangles the loaded path:\n%s" % body[:4000])
body = render("step4.html", {"prox": "px.example", "portprox": "3128",
"proxytype": "1"})
saved = attr(body, "step4.html",
r'<textarea name="userprofile"[^>]*>(.*?)</textarea>')
if "\r\npath=C:\\ab\r\n" not in saved:
sys.exit("step4.html would save %r" % saved)
# ... while a \3A the template itself writes still decodes to the separator
if '--proxy "socks5://px.example:3128"' not in body:
sys.exit("step4.html drops the template's \\3A escape:\n%s" % body[:4000])
# ${liststr:} builds its own <option value=""> in C, so no template edit reaches
# it; the two static categories it prepends are not user data.
body = render("step2.html", {"path": projects})
options = re.findall(r"<option value=\"([^\"]*)\">", body)
for want in ("p&#34;q", "c&#34;d"):
if want not in options:
sys.exit("step2.html lists %r, not %r" % (options, want))
if '<option value="p"q">' in body or '<option value="c"d">' in body:
sys.exit("step2.html closes an option value early:\n%s" % body[:4000])
# The one attribute left on the URL mode, where the quote takes %22 instead:
# an entity would decode back to a quote in the file:// URL it builds.
body = render("file.html", {"path": "/tmp/p" + BREAK})
want = 'href="file:///tmp/p%22&gt;&lt;img%20src=q%20onerror=alert(1)&gt;&amp;&#39;/"'
if want not in body:
sys.exit("file.html does not emit %r:\n%s" % (want, body[:4000]))
PY
echo OK

View File

@@ -0,0 +1,142 @@
#!/bin/bash
# A non-PIE crash report must name the executable and resolve its frames to
# source lines (#995). /proc/self/exe must be rejected when it is the loader,
# and kept when argv[0] merely differs from the file's name (#996).
set -euo pipefail
testdir=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
srcdir=${abs_top_srcdir:?not run under make check}
builddir=${abs_top_builddir:?not run under make check}
out=
skip() {
echo "$*; skipping" >&2
exit 77
}
fail() {
echo "FAIL: $*" >&2
test -z "${out}" || sed 's/^/ | /' <"${out}" >&2
exit 1
}
test "$(uname -s)" = "Linux" || skip "the ELF load bias is a Linux matter"
if ! command -v addr2line >/dev/null && ! command -v llvm-symbolizer >/dev/null; then
skip "no symbolizer installed"
fi
# CC carries flags on some legs ("gcc -m32", "ccache gcc"), so split it.
read -r -a cc_argv <<<"${CC:-cc}"
command -v "${cc_argv[0]}" >/dev/null || skip "no C compiler (${cc_argv[0]})"
work=$(mktemp -d "${TMPDIR:-/tmp}/httrack_nopie.XXXXXX") || exit 1
trap 'set +e; rm -rf "${work}"' EXIT
trap 'rm -rf "${work}"' HUP INT QUIT PIPE TERM
# The crash printer is linked into a probe of our own: every target the build
# tree offers is a PIE, which is the case that already worked.
cflags=(-g -O0 -I"${builddir}" -I"${builddir}/src" -I"${srcdir}/src")
if [ -n "${TEST_CPPFLAGS:-}" ]; then
read -r -a extra_argv <<<"${TEST_CPPFLAGS}"
cflags+=("${extra_argv[@]}")
fi
sources=("${srcdir}/tests/nopieprobe.c" "${srcdir}/src/htsbacktrace.c")
build() { # $1 output binary, rest extra flags
local bin=$1
shift
# shellcheck disable=SC2086 # DL_LIBS is a flag list; splitting is the point
"${cc_argv[@]}" "${cflags[@]}" "$@" -o "${bin}" "${sources[@]}" ${DL_LIBS:-} \
>"${work}/cc.log" 2>&1
}
# Control, run by absolute path so neither the load base nor argv[0] is in play:
# it says whether this box symbolizes anything at all.
build "${work}/probe-pie" || {
head -20 "${work}/cc.log" >&2
fail "the crash printer does not build standalone"
}
out=${work}/pie.log
LC_ALL=C "${work}/probe-pie" >"${out}" 2>&1 || fail "the control probe exited non-zero"
if grep -q 'No stack trace available' "${out}"; then
skip "this build produced no backtrace"
fi
grep -qE 'probe_frame at .*nopieprobe\.c:[0-9]+' "${out}" ||
skip "nothing is symbolized here, even from an absolute path"
# Same probe under the loader: there /proc/self/exe is ld.so, and only dladdr()
# still names the program.
interp=
if command -v readelf >/dev/null; then
interp=$(LC_ALL=C readelf -l "${work}/probe-pie" |
sed -n 's/.*Requesting program interpreter: \([^]]*\)].*/\1/p')
fi
if [ -n "${interp}" ] && [ -x "${interp}" ]; then
out=${work}/loader.log
rc=0
LC_ALL=C run_with_timeout 60 "${interp}" "${work}/probe-pie" >"${out}" 2>&1 || rc=$?
test "${rc}" -ne 124 || fail "the loader-launched probe did not finish in time"
grep -qxF "${work}/probe-pie:" "${out}" ||
fail "launched through ${interp}, the report heads no block with the program"
grep -qE 'probe_frame at .*nopieprobe\.c:[0-9]+' "${out}" ||
fail "launched through ${interp}, the report resolves no source line"
fi
# Same probe under another name: dladdr() echoes argv[0], so anything matching
# the two names drops the module here instead of keeping the real file.
altdir=${work}/alt
mkdir "${altdir}"
ln -s "${work}/probe-pie" "${altdir}/probe-renamed"
out=${work}/renamed.log
rc=0
LC_ALL=C PATH="${altdir}:${PATH}" \
run_with_timeout 60 probe-renamed >"${out}" 2>&1 || rc=$?
test "${rc}" -ne 124 || fail "the symlinked probe did not finish in time"
grep -qxF "${work}/probe-pie:" "${out}" ||
fail "launched as probe-renamed, the report heads no block with the program"
grep -qE 'probe_frame at .*nopieprobe\.c:[0-9]+' "${out}" ||
fail "launched as probe-renamed, the report resolves no source line"
if ! build "${work}/probe-nopie" -no-pie -fno-pie; then
head -5 "${work}/cc.log" >&2
skip "this toolchain takes no -no-pie"
fi
bindir=${work}/bin
mkdir "${bindir}"
mv "${work}/probe-nopie" "${bindir}/probe-nopie"
if command -v readelf >/dev/null; then
LC_ALL=C readelf -h "${bindir}/probe-nopie" >"${work}/elf.txt"
grep -qE '^[[:space:]]*Type:[[:space:]]+EXEC' "${work}/elf.txt" ||
skip "the toolchain ignored -no-pie and linked a PIE anyway"
fi
# No probe-nopie here, so a report that kept argv[0] has nothing to open.
cd "${work}"
out=${work}/nopie.log
rc=0
# LC_ALL=C: addr2line translates its " at " separator, which the match below pins.
LC_ALL=C PATH="${bindir}:${PATH}" \
run_with_timeout 60 probe-nopie >"${out}" 2>&1 || rc=$?
test "${rc}" -ne 124 || fail "the probe did not finish within the deadline"
if ! command -v readelf >/dev/null; then
# glibc drops the "+0x" offset when the load bias is zero.
grep -qE '^probe-nopie\(\) \[0x' "${out}" ||
skip "no zero-bias frame in the report; this is no ET_EXEC"
fi
headers=$(grep -E '^/.*/probe-nopie:$' "${out}" || true)
test -n "${headers}" || fail "the report has no symbolized block for the executable"
# Every frame of a module is claimed in one pass, so a second block for the
# executable means library frames were misattributed to it.
test "$(wc -l <<<"${headers}")" -eq 1 || fail "several executable blocks: ${headers}"
exe=${headers%:}
# Device+inode, so a symlinked TMPDIR does not read as a different binary.
test "${exe}" -ef "${bindir}/probe-nopie" || fail "the report names ${exe}"
block=$(awk -v h="${headers}" '$0 == h { on = 1; next } /^\// { on = 0 } on' "${out}")
# " at " pins the symbol column: both addr2line -Cfipa and llvm-symbolizer -p
# emit it. A wrong offset resolves to "?? ??:0" instead.
grep -qE '(^|[^[:alnum:]_])probe_frame at .*nopieprobe\.c:[0-9]+' <<<"${block}" ||
fail "the executable's block resolves no source line: ${block}"
echo "the non-PIE report named ${exe} and resolved its own frames"

View File

@@ -0,0 +1,110 @@
#!/bin/bash
# dyld follows an rpath only for a load path spelled @rpath/..., so a relocatable
# install on Darwin takes libhttrack's own id too (#978). Read the load commands
# the install produced.
set -euo pipefail
# shellcheck source=tests/testlib.sh
. "$(dirname "$0")/testlib.sh"
fail() {
echo "FAIL: $*" >&2
exit 1
}
skip() {
echo "$*; skipping" >&2
exit 77
}
# Unset means no automake run (the Windows job runs the scripts directly), so
# there is no install rule to drive.
[ -n "${abs_top_builddir:-}" ] || skip "abs_top_builddir unset, no automake environment"
bindir="${CONFIGURED_BINDIR:?not run under make check}"
libdir="${CONFIGURED_LIBDIR:?not run under make check}"
libdir=${libdir%/}
test "$(uname -s)" = Darwin || skip "no Mach-O install commands to read here"
command -v otool >/dev/null 2>&1 || skip "no otool"
work=$(mktemp -d "${TMPDIR:-/tmp}/rpathid.XXXXXX") || {
echo "no tmpdir" >&2
exit 1
}
trap 'set +e; rm -rf "${work}"' EXIT
stage=${work}/stage
stage_install_exec "${stage}" "${work}/install.log" ||
fail "make install-exec DESTDIR=${stage}"
# The install name, empty when absent. The awk drains its input rather than
# exiting on the match, or otool takes SIGPIPE and pipefail returns it.
dylib_id() {
otool -l "$1" |
awk '/LC_ID_DYLIB/ { f = 1 } f && $1 == "name" { if (!p) { print $2; p = 1 } f = 0 }'
}
# The dependencies, one per line: only the indented lines of otool -L are those.
dylib_deps() {
otool -L "$1" | awk '/^[[:space:]]/ { print $1 }'
}
# The LC_RPATH entries, one per line.
dylib_rpaths() {
otool -l "$1" | awk '/LC_RPATH/ { f = 1 } f && $1 == "path" { print $2; f = 0 }'
}
ids=()
for lib in "${stage}${libdir}"/libhttrack*.dylib; do
# The version links point at the file below them, and the glob itself is what
# is left when a --disable-shared build installed no library at all.
test -f "${lib}" || continue
test ! -L "${lib}" || continue
id=$(dylib_id "${lib}")
test -n "${id}" || fail "${lib} carries no LC_ID_DYLIB"
ids+=("${id}")
done
# Must stay ahead of every "${ids[@]}" below: bash 3.2 calls an empty array unbound.
test "${#ids[@]}" -gt 0 || skip "this build installed no libhttrack dylib"
if [ "${ORIGIN_RPATH:-no}" != yes ]; then
# Both directions: with the gate off the id has to stay absolute, or the
# binaries would record an @rpath nothing puts an entry in them for.
for id in "${ids[@]}"; do
case "${id}" in
@rpath/*) fail "configure suppressed the rpath but ${id} is relative anyway" ;;
esac
done
skip "configure did not enable a binary-relative rpath in this build"
fi
for id in "${ids[@]}"; do
test "${id}" = "@rpath/${id##*/}" ||
fail "installed libhttrack keeps the id ${id}, which no rpath answers"
done
# The id only pays off if the executables recorded it and hold an entry answering
# it: an absolute dependency resolves before any rpath, and an @rpath one without
# the entry resolves to nothing at all.
idlist=$(printf '%s\n' "${ids[@]}")
for exe in httrack htsserver; do
bin=${stage}${bindir}/${exe}
test -r "${bin}" || fail "no ${exe} in ${stage}${bindir}"
rpaths=$(dylib_rpaths "${bin}")
grep -qxF '@loader_path/../lib' <<<"${rpaths}" ||
fail "${exe} carries no @loader_path/../lib rpath, so a copied tree resolves nothing"
found=0
while read -r dep; do
case "${dep}" in
*libhttrack*) ;;
*) continue ;;
esac
found=$((found + 1))
grep -qxF "${dep}" <<<"${idlist}" ||
fail "${exe} loads ${dep}, which is not the id the installed library carries"
done < <(dylib_deps "${bin}")
test "${found}" -ge 1 ||
fail "${exe} holds no libhttrack dependency, so this proved nothing"
done
echo "installed libhttrack is ${ids[*]}, loaded as such by httrack and htsserver"

View File

@@ -0,0 +1,210 @@
#!/bin/bash
#
# A mirrored page must not reach the control panel: it lands in an opaque
# origin, and a POST claiming a foreign origin is refused before it runs.
set -euo pipefail
testdir=$(cd "$(dirname "$0")" && pwd)
distdir=${top_srcdir:-$(cd "${testdir}/.." && pwd)}
distdir=$(cd "${distdir}" && pwd)
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
fail() {
echo "FAIL: $*" >&2
exit 1
}
command -v htsserver >/dev/null || fail "no htsserver in PATH"
python=$(find_python) || {
echo "python3 not found; skipping" >&2
exit 77
}
# The panel assets the negative side reads have to exist, or it goes vacuous.
for asset in server/index.html server/ping.js; do
test -f "${distdir}/html/${asset}" || fail "missing GUI asset ${asset}"
done
work=$(mktemp -d "${TMPDIR:-/tmp}/webhttrack_sandbox.XXXXXX") || fail "no tmpdir"
srvlog=$(mktemp)
srv=
srvpid=
cleanup() {
# htsserver keeps SIGTERM ignored across its exec, so only -9 reaps it.
test -z "${srvpid}" || kill -9 "${srvpid}" 2>/dev/null || true
test -z "${srv}" || kill -9 "${srv}" 2>/dev/null || true
wait "${srv}" 2>/dev/null || true # absorb bash's async "Killed" notice
rm -rf "${work}" "${srvlog}"
}
trap 'set +e; cleanup' EXIT
trap cleanup HUP INT QUIT PIPE TERM
# Stand in for a finished mirror: the crawl itself is not under test.
proj="${work}/websites/proj"
mkdir -p "${proj}/site.example"
printf '<html><body>MIRROR-HTML-OK<script>1</script></body></html>\n' >"${proj}/page.html"
# A crawled page lives one level down, under its host; the root files above
# cannot tell a per-file gate from one keyed on the /website/ prefix.
printf '<html><body>MIRROR-SUB-OK</body></html>\n' >"${proj}/site.example/deep.html"
# #875 made a mirrored .svg render inline, so it carries script too.
printf '<svg xmlns="http://www.w3.org/2000/svg">MIRROR-SVG-OK</svg>\n' >"${proj}/probe.svg"
# Untyped: sniffing can still hand it to the HTML parser.
printf 'MIRROR-BLOB-OK\n' >"${proj}/data.bin"
# An isolated HOME keeps a stray ~/.httrack.ini out of the server's settings.
sport=$("${python}" -c 'import socket
s = socket.socket()
s.bind(("127.0.0.1", 0))
print(s.getsockname()[1])
s.close()')
(
trap '' TERM TTOU
export HOME="${work}"
exec htsserver "${distdir}/" --port "${sport}" >"${srvlog}" 2>&1
) &
srv=$!
for _ in $(seq 1 40); do
url=$(sed -n 's/^URL=//p' "${srvlog}") && test -n "${url}" && break
kill -0 "${srv}" 2>/dev/null || break
sleep 0.25
done
test -n "${url:-}" || fail "htsserver did not start: $(cat "${srvlog}")"
srvpid=$(sed -n 's/^PID=//p' "${srvlog}") # absent on Windows
# htsserver resolves the posted paths itself, so hand it native ones.
"${python}" - "${url}" "$(nativepath "${work}/websites")" <<'PY' || fail "mirror isolation checks failed"
import re, socket, sys, urllib.parse, urllib.request
url, base = sys.argv[1].rstrip("/"), sys.argv[2]
expected = "sandbox allow-scripts allow-forms allow-popups allow-downloads"
parts = urllib.parse.urlsplit(url)
rc = 0
def check(ok, what):
global rc
print(("ok: " if ok else "FAIL: ") + what)
if not ok:
rc = 1
def raw(request, whole=False):
"""Raw so the status line is read verbatim and no redirect is followed."""
sock = socket.create_connection((parts.hostname, parts.port), timeout=20)
try:
sock.sendall(request)
got = b""
while len(got) < 65536 and (whole or b"\r\n\r\n" not in got):
chunk = sock.recv(4096)
if not chunk:
break
got += chunk
finally:
sock.close()
return got
def get(path):
return raw(b"GET " + path.encode() + b" HTTP/1.0\r\n\r\n")
def post(path, body, origin=None):
head = "POST %s HTTP/1.0\r\nHost: %s\r\n" % (path, parts.netloc)
head += "Content-type: application/x-www-form-urlencoded\r\n"
head += "Content-length: %d\r\n" % len(body)
if origin is not None:
head += "Origin: %s\r\n" % origin
return raw(head.encode("latin-1") + b"\r\n" + body.encode("latin-1"), True)
def body_is_complete(got):
"""A Content-length the response never delivers leaves the client hanging."""
head, sep, payload = got.partition(b"\r\n\r\n")
m = re.search(rb"\r\nContent-length: (\d+)$", head)
return sep != b"" and m is not None and len(payload) == int(m.group(1))
def fetch(path):
r = urllib.request.urlopen(url + path, timeout=20)
return r.headers, r.read()
def check_sandboxed(path, marker):
hdrs, body = fetch(path)
got = hdrs.get("Content-Security-Policy", "")
check(got == expected, "%s is sandboxed (got %r)" % (path, got))
# A truncated or empty body would satisfy the header check on its own.
check(marker in body, "%s served its content" % path)
# allow-same-origin is the omission that matters: with it /server/ is readable again.
check("allow-same-origin" not in got, "%s keeps an opaque origin" % path)
def check_unsandboxed(path, marker, when):
hdrs, body = fetch(path)
check("Content-Security-Policy" not in hdrs,
"%s is not sandboxed %s" % (path, when))
check(marker in body, "%s served its content %s" % (path, when))
# Before any project is set: index.html renders the sid a sandbox would hide.
panel = urllib.request.urlopen(url + "/server/index.html", timeout=20).read()
sid = re.search(rb'name="sid" value="([0-9a-f]+)"', panel)
if sid is None:
print("FAIL: no sid in the rendered form")
sys.exit(1)
check_unsandboxed("/server/index.html", b'name="sid"', "before the save")
check_unsandboxed("/server/ping.js", b"function", "before the save")
# What step4.html posts. command_do=save records the served root and runs no
# crawl, so the mirror answering afterwards is the side effect to watch.
fields = [("sid", sid.group(1).decode()), ("path", base), ("projname", "proj"),
("command", "httrack"), ("command_do", "save"), ("winprofile", "x")]
body = "&".join("%s=%s" % (k, urllib.parse.quote(v, safe="")) for k, v in fields)
# A sandboxed page sends "null", any other page its own origin. Neither may
# take effect, and a 403 on its own would not prove the command did not run.
for origin in ("null", "http://crawled.example"):
got = post("/server/step4.html", body, origin)
check(got.startswith(b"HTTP/1.0 403 "), "Origin: %s is refused (got %r)"
% (origin, got.split(b"\r\n")[0]))
check(body_is_complete(got), "the refusal delivers the body it announces")
check(get("/website/page.html").startswith(b"HTTP/1.0 404 "),
"the refused save bound no mirror")
# urllib sends no Origin, and neither do most non-browser clients.
check(not post("/server/step4.html", body).startswith(b"HTTP/1.0 403 "),
"a POST without an Origin is accepted")
check(get("/website/page.html").startswith(b"HTTP/1.0 200 "),
"the accepted save bound the mirror")
# The panel's own forms carry a matching Origin, so refusing every one is wrong.
check(not post("/server/step4.html", body, url).startswith(b"HTTP/1.0 403 "),
"a POST from the panel's own origin is accepted")
check_sandboxed("/website/page.html", b"MIRROR-HTML-OK")
check_sandboxed("/website/site.example/deep.html", b"MIRROR-SUB-OK")
check_sandboxed("/website/probe.svg", b"MIRROR-SVG-OK")
check_sandboxed("/website/data.bin", b"MIRROR-BLOB-OK")
# Repeated now that a project is bound: a gate keyed on server state rather
# than on the /website/ prefix only starts sandboxing the panel here.
check_unsandboxed("/server/ping.js", b"function", "after the save")
check_unsandboxed("/server/finished.html", b"/website/index.html", "after the save")
# urllib joins folded headers and hides framing, so read the raw block once.
head = get("/website/page.html").split(b"\r\n\r\n")[0]
check(head.startswith(b"HTTP/1.0 200 OK\r\n"), "mirrored page is a 200")
check(head.count(b"Content-Security-Policy:") == 1, "one policy header")
check(b"\r\nContent-Security-Policy: " + expected.encode() + b"\r\n" in head,
"policy line ends CRLF")
check(b"\n" not in head.replace(b"\r\n", b""), "no bare LF in the header block")
sys.exit(rc)
PY
# A leaked htsserver wedges the parallel harness behind a green log.
cleanup
! kill -0 "${srv}" 2>/dev/null || fail "htsserver ${srv} survived"
echo "PASS"

View File

@@ -0,0 +1,79 @@
#!/bin/bash
#
# A %0d%0a in an FTP URL decoded into a command line of its own (#1010). The
# run has to end on an error: send_line() alone drops the command and leaves
# the engine on a 300s read. A bare listing argument had the same shape, a
# space in it giving the server a second token.
set -euo pipefail
: "${top_srcdir:=..}"
testdir=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
python=$(find_python) || ! echo "python3 not found; skipping" >&2 || exit 77
command -v httrack >/dev/null || {
echo "could not find httrack" >&2
exit 1
}
server=$(nativepath "${testdir}/ftp-server.py")
tmpdir=$(mktemp -d "${TMPDIR:-/tmp}/httrack_ftp.XXXXXX")
serverpid=
cleanup() {
stop_server "$serverpid"
rm -rf "$tmpdir"
}
trap 'set +e; cleanup' EXIT
trap cleanup HUP INT QUIT PIPE TERM
root="${tmpdir}/root"
out="${tmpdir}/crawl"
cmds="${tmpdir}/cmds"
mkdir -p "${root}/d -la" "$out"
printf 'body\n' >"${root}/f.txt"
printf 'secret\n' >"${root}/x.txt"
printf 'listed\n' >"${root}/d -la/inside.txt"
serverlog="${tmpdir}/server.out"
"$python" "$server" --root "$(nativepath "$root")" \
--log "$(nativepath "$cmds")" >"$serverlog" 2>&1 &
serverpid=$!
port=$(discover_server_port "$serverlog" "$serverpid") || exit 1
fail() {
echo "FAIL: $*" >&2
exit 1
}
ok() { echo "OK: $*"; }
# 60s is the wedge detector: a dropped command leaves the engine on a 300s read.
crawl() {
run_with_timeout 60 httrack "$1" -O "$out" --quiet --disable-security-limits \
--robots=0 --timeout=20 --max-time=45 --retries=1 -c1
}
# --- the injected command line ----------------------------------------------
: >"$cmds"
start=$SECONDS
crawl "ftp://127.0.0.1:${port}/f%0d%0aDELE%20x.txt" >"${tmpdir}/log1" 2>&1 ||
fail "the run did not finish in $((SECONDS - start))s"
grep -aq 'Invalid control character in FTP URL' "${out}/hts-log.txt" ||
fail "no refusal in the log: $(grep -a 'Error' "${out}/hts-log.txt" || true)"
ok "the injected URL is refused in $((SECONDS - start))s"
sent=$(cat "$cmds")
test -z "$sent" || fail "the engine still talked to the server: ${sent}"
ok "nothing reached the control channel"
# --- a space in a listing argument ------------------------------------------
: >"$cmds"
crawl "ftp://127.0.0.1:${port}/d%20-la/" >"${tmpdir}/log2" 2>&1 ||
fail "the listing crawl never finished"
listcmd=$(grep -a '^LIST ' "$cmds" || true)
test "$listcmd" = 'LIST -A "/d -la/"' ||
fail "the listing path is not one quoted token: [${listcmd}]"
grep -aq inside.txt "${out}/127.0.0.1_${port}/d -la/index.txt" ||
fail "the quoted listing brought nothing back; mirror holds $(find "$out" \
-type f -not -path '*hts-cache*')"
ok "the listing path goes out as a single token, and still lists"

View File

@@ -0,0 +1,140 @@
#!/bin/bash
# libhttrack.pc hands a consumer the include path and, outside a system libdir,
# the rpath the @rpath install_name needs (#978). Compiled, linked and run, since
# a compile alone passes on the broken case.
set -euo pipefail
# shellcheck source=tests/testlib.sh
. "$(dirname "$0")/testlib.sh"
fail() {
echo "FAIL: $*" >&2
exit 1
}
skip() {
echo "$*; skipping" >&2
exit 77
}
# Unset means no automake run (the Windows job runs the scripts directly).
[ -n "${abs_top_builddir:-}" ] || skip "abs_top_builddir unset, no automake environment"
[ -f "${abs_top_builddir}/src/Makefile" ] || fail "${abs_top_builddir}/src is not configured"
prefix=${CONFIGURED_PREFIX:?not run under make check}
libdir=${CONFIGURED_LIBDIR:?not run under make check}
includedir=${CONFIGURED_INCLUDEDIR:?not run under make check}
pkgconfig=${PKG_CONFIG:-pkg-config}
command -v "${pkgconfig}" >/dev/null 2>&1 || skip "no ${pkgconfig}"
# CC carries flags on some legs ("gcc -m32", "ccache gcc"), so split it.
read -r -a cc_argv <<<"${CC:-cc}"
command -v "${cc_argv[0]}" >/dev/null 2>&1 || skip "no C compiler (${cc_argv[0]})"
command -v "${MAKE:-make}" >/dev/null 2>&1 || skip "no make (${MAKE:-make})"
# The .pc carries no sanitizer flags, so an ASan runtime arrives through libhttrack.
export ASAN_OPTIONS="${ASAN_OPTIONS:+$ASAN_OPTIONS:}verify_asan_link_order=0"
work=$(mktemp -d "${TMPDIR:-/tmp}/pkgconf.XXXXXX") || fail "no tmpdir"
trap 'set +e; rm -rf "${work}"' EXIT
stage=${work}/stage
stage_install_target install "${stage}" "${work}/install.log" ||
fail "make install DESTDIR=${stage}"
staged_libdir=${stage}${libdir%/}
test -r "${staged_libdir}/pkgconfig/libhttrack.pc" ||
fail "no libhttrack.pc in ${staged_libdir}/pkgconfig"
# Only a shared build has anything for an rpath to resolve.
shared=$(find "${staged_libdir}" -maxdepth 1 \( -name 'libhttrack.so*' -o -name 'libhttrack*.dylib' \))
[ -n "${shared}" ] || skip "this build installed no shared libhttrack"
# Without the @rpath id there is nothing an rpath can answer: the dylib keeps an
# absolute install name, and only the real libdir resolves.
[ "$(uname -s)" != Darwin ] || [ "${ORIGIN_RPATH:-no}" = yes ] ||
skip "configure did not enable a binary-relative rpath in this build"
export PKG_CONFIG_PATH=${staged_libdir}/pkgconfig
# --define-variable OVERRIDES the file's own assignment, so every relocated query
# below would pass on a .pc naming the wrong directory. Read the raw values first.
expect_variable() { # <variable> <what configure was given>
local got
got=$("${pkgconfig}" --variable="$1" libhttrack) || fail "pkg-config --variable=$1 failed"
test "${got%/}" = "${2%/}" || fail "libhttrack.pc says $1=${got}, configure was given $2"
}
expect_variable prefix "${prefix}"
expect_variable libdir "${libdir}"
expect_variable includedir "${includedir}"
# The staged copy still names the configured directories, so point every one of
# them at the stage to build against what was just installed.
relocate=(--define-variable=prefix="${stage}${prefix}"
--define-variable=libdir="${staged_libdir}"
--define-variable=includedir="${stage}${includedir}")
pc_query() { "${pkgconfig}" "${relocate[@]}" "$@" libhttrack; }
read -r -a cflags_argv <<<"$(pc_query --cflags)"
read -r -a libs_argv <<<"$(pc_query --libs)"
version=$(pc_query --modversion) || fail "pkg-config --modversion failed"
# Flat, as the installed headers include each other and the libtest examples do:
# an include path one level too high would still resolve <httrack/...>.
cat >"${work}/consumer.c" <<'EOF'
#include <stdio.h>
#include <httrack-library.h>
int main(void) {
const char *const v = hts_version();
printf("%s\n", v != NULL ? v : "");
return v == NULL;
}
EOF
"${cc_argv[@]}" "${cflags_argv[@]}" -o "${work}/consumer" "${work}/consumer.c" \
"${libs_argv[@]}" 2>"${work}/cc.log" || {
head -20 "${work}/cc.log" >&2
fail "a consumer does not build from pkg-config --cflags --libs libhttrack"
}
# With no rpath by design, the stage is not on the loader's path.
runenv=(env -u LD_LIBRARY_PATH -u DYLD_LIBRARY_PATH)
[ "${PKGCONFIG_RPATH:-no}" = yes ] ||
runenv=(env "LD_LIBRARY_PATH=${staged_libdir}" "DYLD_LIBRARY_PATH=${staged_libdir}")
run_consumer() { # <binary> <what it is>
local out
out=$("${runenv[@]}" "$1" 2>&1) || fail "$2 did not run: ${out}"
test "${out}" = "${version}" ||
fail "libhttrack.pc says Version ${version}, $2 resolved ${out}"
}
# From an unrelated cwd, so nothing resolves through a relative path.
cd "${work}"
run_consumer "${work}/consumer" "the consumer"
# An archive link needs each library ahead of the ones it uses, an order the
# shared link above cannot exercise.
read -r -a static_cflags_argv <<<"$(pc_query --static --cflags)"
read -r -a static_libs_argv <<<"$(pc_query --static --libs)"
if [ ! -f "${staged_libdir}/libhttrack.a" ] ||
! "${cc_argv[@]}" -static -o "${work}/staticprobe" -x c - \
<<<'int main(void) { return 0; }' >/dev/null 2>&1; then
echo "no libhttrack.a, or no working -static here; skipping the archive link" >&2
elif ! "${cc_argv[@]}" "${static_cflags_argv[@]}" -static -o "${work}/consumer-static" \
"${work}/consumer.c" "${static_libs_argv[@]}" 2>"${work}/cc-static.log"; then
# A sanitized build's archive needs the runtime -static cannot supply.
if grep -q -e __asan -e __ubsan -e __tsan -e __msan "${work}/cc-static.log"; then
echo "the build is sanitized, its archive does not link -static; skipping" >&2
else
head -20 "${work}/cc-static.log" >&2
fail "a consumer does not link from pkg-config --static --cflags --libs libhttrack"
fi
else
run_consumer "${work}/consumer-static" "the static consumer"
fi
# Take the library away and the consumer has to stop working. If it still runs,
# another libhttrack answered and the run above proved nothing.
mv "${staged_libdir}" "${staged_libdir}.gone"
! "${runenv[@]}" "${work}/consumer" >/dev/null 2>&1 ||
skip "a libhttrack outside the stage answers, so the run proves nothing"
echo "a pkg-config consumer built and ran against the staged libhttrack ${version}"

View File

@@ -0,0 +1,122 @@
#!/bin/bash
#
# The session id authenticates every panel command, so it must not be derivable
# from the clock: two servers started in the same second must still differ.
set -euo pipefail
testdir=$(cd "$(dirname "$0")" && pwd)
distdir=${top_srcdir:-$(cd "${testdir}/.." && pwd)}
distdir=$(cd "${distdir}" && pwd)
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
fail() {
echo "FAIL: $*" >&2
exit 1
}
command -v htsserver >/dev/null || fail "no htsserver in PATH"
python=$(find_python) || {
echo "python3 not found; skipping" >&2
exit 77
}
work=$(mktemp -d "${TMPDIR:-/tmp}/webhttrack_sid.XXXXXX") || fail "no tmpdir"
pids=
cleanup() {
# htsserver keeps SIGTERM ignored across its exec, so only -9 reaps it.
for p in ${pids}; do
kill -9 "${p}" 2>/dev/null || true
done
wait 2>/dev/null || true # absorb bash's async "Killed" notice
rm -rf "${work}"
}
trap 'set +e; cleanup' EXIT
trap cleanup HUP INT QUIT PIPE TERM
# Both bound at once, or the two picks could land on the same free port.
read -r porta portb <<<"$("${python}" -c 'import socket
s = [socket.socket() for _ in range(2)]
for x in s:
x.bind(("127.0.0.1", 0))
print(*[x.getsockname()[1] for x in s])
for x in s:
x.close()')"
# An isolated HOME keeps a stray ~/.httrack.ini out of the server's settings.
launch() { # $1 = tag, $2 = port
mkdir -p "${work}/$1"
(
trap '' TERM TTOU
export HOME="${work}/$1"
exec htsserver "${distdir}/" --port "$2" >"${work}/$1.log" 2>&1
) &
pids="${pids} $!"
}
started() { # $1 = tag; prints the URL once the server is up
for _ in $(seq 1 100); do
url=$(sed -n 's/^URL=//p' "${work}/$1.log" 2>/dev/null) &&
test -n "${url}" && {
echo "${url}"
return 0
}
sleep 0.05
done
return 1
}
# Both servers have to read the same second off the clock, or the id they used
# to derive from it would differ for a reason that is not entropy. Launching
# just after a tick leaves a whole second to get there; the bracket confirms it.
urla=
urlb=
for _ in $(seq 1 10); do
pids=
rm -f "${work}"/*.log
prev=$(date +%s)
while test "$(date +%s)" = "${prev}"; do
sleep 0.02
done
t0=$(date +%s)
launch a "${porta}"
launch b "${portb}"
urla=$(started a) || fail "htsserver a did not start: $(cat "${work}/a.log")"
urlb=$(started b) || fail "htsserver b did not start: $(cat "${work}/b.log")"
t1=$(date +%s)
test "${t0}" != "${t1}" || break
cleanup
urla=
done
test -n "${urla}" || {
echo "the box never started two servers inside one second; skipping" >&2
exit 77
}
"${python}" - "${urla}" "${urlb}" <<'PY' || fail "session id checks failed"
import re, sys, urllib.request
ids = []
for u in sys.argv[1:3]:
body = urllib.request.urlopen(u.rstrip("/") + "/server/index.html",
timeout=20).read()
m = re.search(rb'name="sid" value="([^"]*)"', body)
if m is None:
print("FAIL: no sid in the form rendered by " + u)
sys.exit(1)
ids.append(m.group(1).decode())
rc = 0
for sid in ids:
# The templates and the server's own SID_VALUE_MAX assume an md5 digest.
ok = re.fullmatch(r"[0-9a-f]{32}", sid) is not None
print(("ok: " if ok else "FAIL: ") + "%r is an md5 hex digest" % sid)
rc |= 0 if ok else 1
ok = ids[0] != ids[1]
print(("ok: " if ok else "FAIL: ") + "same-second ids differ (%r)" % ids)
sys.exit(rc | (0 if ok else 1))
PY
cleanup
echo "PASS"

View File

@@ -66,9 +66,11 @@ command -v addr2line >/dev/null || {
}
# Control: resolve the same frames ourselves, so a stripped build cannot turn
# the assertion below into a vacuous pass.
# the assertion below into a vacuous pass. Second form: glibc drops the "+0x"
# when the load bias is zero, and the bracketed address is then the file one.
oracle="${tmpdir}/oracle"
sed -n 's/^\([^(]*\)(+\(0x[0-9a-f]*\)).*$/\1 \2/p' "$out" |
sed -n -e 's/^\([^(]*\)(+\(0x[0-9a-f]*\)).*$/\1 \2/p' \
-e 's/^\([^(]*\)() \[\(0x[0-9a-f]*\)\].*$/\1 \2/p' "$out" |
while read -r mod off; do
# The main program is reported as argv[0], a bare name off PATH (#889).
test -f "$mod" || mod=$(command -v "$mod" 2>/dev/null) || continue

View File

@@ -103,12 +103,11 @@ test -n "${srv}" || fail "htsserver did not report its pid"
sid=$(scrape_sid "${port}")
test "${#sid}" -eq 32 || fail "did not scrape a 32-hex sid from the page (got '${sid}')"
# Fill the wizard fields the command line quotes, then read back the generated
# command line: the user-agent carries a break-out attempt, the footer a
# backslash (which must survive the escape round trip), the project name a plain
# value.
# Fill the wizard fields with what would break each position, then read back the
# generated command line; the project name stays plain as a control.
body=$(formencode "sid=${sid}" 'user=Moz" -V "touch /tmp/pwn' 'footer=a\b"c' \
"path=/tmp/p" "projname=plain proj" 'urls=http://x/a"b' 'url2=+*.png"')
"path=/tmp/p" "projname=plain proj" 'urls=http://x/a"b' 'url2=+*.png"</textarea>' \
'sitemapurl=http://x/s.xml"b')
request "${port}" "${body}" >/dev/null
cmdline=$(request "${port}" "" step4 |
sed -n '/<textarea name="command"/,/<\/textarea>/p')
@@ -146,4 +145,15 @@ grep -qF -- '+*.png%22' <<<"${cmdline}" ||
grep -qF -- 'http://x/a"b' <<<"${cmdline}" &&
fail "the url field still emits a raw quote: ${cmdline}"
# The whole textarea is HTML too: a field holding its closing tag must come back
# as entities, or the value lands in the page as markup.
grep -qF -- '+*.png%22&lt;/textarea&gt;' <<<"${cmdline}" ||
fail "the filter field was not HTML-escaped: ${cmdline}"
grep -qF -- '+*.png%22</textarea>' <<<"${cmdline}" &&
fail "the filter field still closes the textarea: ${cmdline}"
# The sitemap url is quoted like the user-agent, so it takes the same escape.
grep -qF -- '--sitemap-url "http://x/s.xml\"b"' <<<"${cmdline}" ||
fail "the quote in the sitemap url was not escaped: ${cmdline}"
echo "PASS"

View File

@@ -4,9 +4,10 @@ include $(srcdir)/tests-list.mk
# Committed binary fixture read by 01_zlib-cache-golden.test. List it
# explicitly: automake does not expand wildcards in EXTRA_DIST, so a glob would
# silently drop it from the dist tarball and break "make distcheck".
EXTRA_DIST = $(TESTS) renamefail.c threadattrfail.c nobacktrace.c altstackprobe.c crawl-test.sh run-all-tests.sh check-network.sh \
EXTRA_DIST = $(TESTS) renamefail.c threadattrfail.c nobacktrace.c altstackprobe.c nopieprobe.c crawl-test.sh run-all-tests.sh check-network.sh \
proxy-https-server.py socks5-server.py proxy-connect-server.py \
proxytestlib.py tls-stall-server.py warc-validate.py wacz-validate.py \
header-injection-server.py header-injection-check.py \
pty-resize.py test-timeout.sh \
local-crawl.sh local-server.py ftp-server.py testlib.sh \
ci-windows-suite.sh \
@@ -34,12 +35,22 @@ TESTS_ENVIRONMENT += CONFIGURED_DATADIR=$(datadir)
TESTS_ENVIRONMENT += CONFIGURED_LIBDIR=$(libdir)
TESTS_ENVIRONMENT += CONFIGURED_BINDIR=$(bindir)
TESTS_ENVIRONMENT += ORIGIN_RPATH=$(ORIGIN_RPATH)
# 222_pkgconfig-consumer.test asserts the staged .pc against these, and both
# sides of the gate deciding whether it hands a consumer an rpath.
TESTS_ENVIRONMENT += CONFIGURED_PREFIX=$(prefix)
TESTS_ENVIRONMENT += CONFIGURED_INCLUDEDIR=$(includedir)
TESTS_ENVIRONMENT += PKGCONFIG_RPATH=$(PKGCONFIG_RPATH)
TESTS_ENVIRONMENT += MAKE="$(MAKE)"
# 205_install-headers.test compiles the installed headers with the build's compiler.
# CPPFLAGS carries the dependency include paths a consumer needs too: brew's
# openssl@3 is keg-only, so htsbasenet.h finds <openssl/ssl.h> only through it.
TESTS_ENVIRONMENT += CC="$(CC)"
TESTS_ENVIRONMENT += TEST_CPPFLAGS="$(CPPFLAGS)"
# 218_crash-nopie-frames.test links the crash printer, which calls dladdr().
TESTS_ENVIRONMENT += DL_LIBS="$(DL_LIBS)"
# 207_install-headers-symbols.test links a consumer probe against it; absent on a
# static-only build and named .dylib on macOS, where that ELF-only test skips.
TESTS_ENVIRONMENT += HTTRACK_SHLIB=$(abs_top_builddir)/src/$(LT_CV_OBJDIR)/libhttrack.so
TESTS_ENVIRONMENT += RENAMEFAIL_LA=$(abs_builddir)/librenamefail.la
TESTS_ENVIRONMENT += RENAMEFAIL_LIB=$(abs_builddir)/$(LT_CV_OBJDIR)/librenamefail.so
TESTS_ENVIRONMENT += THREADATTRFAIL_LA=$(abs_builddir)/libthreadattrfail.la
@@ -73,9 +84,10 @@ libthreadattrfail_la_LIBADD = $(DL_LIBS)
libnobacktrace_la_SOURCES = nobacktrace.c
libnobacktrace_la_LDFLAGS = -module -avoid-version -rpath $(abs_builddir)
# alt-stack observer for 181_altstack-honoured.test
# alt-stack observer for 181_altstack-honoured.test and 183_altstack-worker.test
libaltstackprobe_la_SOURCES = altstackprobe.c
libaltstackprobe_la_LDFLAGS = -module -avoid-version -rpath $(abs_builddir)
libaltstackprobe_la_LIBADD = $(DL_LIBS)
# realloc stub for 152_engine-string-oom.test; links nothing, so it stays a
# plain binary rather than a libtool wrapper script.

View File

@@ -1,13 +1,27 @@
/* Says what happened to the alternate signal stack a process was handed, for
181_altstack-honoured.test. ALTSTACK_MODE picks the state main() inherits:
"keep" installs one first, "none" leaves none installed. */
"keep" installs one first, "none" leaves none installed.
ALTSTACK_TRACE names a file to log the per-thread lifecycle to, for
183_altstack-worker.test, Linux only as that leg is: "<pid> <tid> query|set
on|off|own <sp>" for every sigaltstack(), "<pid> <tid> munmap - <sp>" for
every unmap of a stack seen installed. Only the ordering across those two
syscalls can show that a worker gives its stack back, disabled first.
"own" is an install of a mapping this shim watched mmap() hand out, which is
what tells httrack's stacks apart from a sanitizer runtime's: those come from
a raw syscall no interposer sees, though installed the same way. */
#define _GNU_SOURCE
#include <dlfcn.h>
#include <fcntl.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/syscall.h>
#include <unistd.h>
#define PROBE_STACK_SIZE (128 * 1024)
@@ -61,3 +75,156 @@ static void __attribute__((destructor)) probe_report(void) {
SAY("ALTSTACK-OWN\n");
}
}
/* Linux-only, like the leg of 183 that reads the trace: gettid() and the
LD_PRELOAD interposition below have no portable spelling, and interposing
mmap() process-wide where nothing consumes the result buys nothing. */
#ifdef __linux__
/* -fvisibility=hidden across the tree would otherwise hide the interposers. */
#define SHIM_EXPORT __attribute__((visibility("default")))
/* One per worker, so several crawls worth of them fit. */
#define TRACE_STACKS 256
static int trace_fd = -1;
static void *trace_stacks[TRACE_STACKS];
static int trace_stacks_used = 0;
static __thread void *trace_last_mmap = NULL;
static int (*real_sigaltstack)(const stack_t *, stack_t *) = NULL;
static int (*real_munmap)(void *, size_t) = NULL;
static void *(*real_mmap)(void *, size_t, int, int, int, off_t) = NULL;
#ifdef __GLIBC__
static void *(*real_mmap64)(void *, size_t, int, int, int, off64_t) = NULL;
#endif
/* Runs ahead of probe_setup(), whose own sigaltstack() call would otherwise
reach an unresolved interposer. */
static void __attribute__((constructor(101))) trace_setup(void) {
const char *const path = getenv("ALTSTACK_TRACE");
*(void **) &real_sigaltstack = dlsym(RTLD_NEXT, "sigaltstack");
*(void **) &real_munmap = dlsym(RTLD_NEXT, "munmap");
*(void **) &real_mmap = dlsym(RTLD_NEXT, "mmap");
#ifdef __GLIBC__
*(void **) &real_mmap64 = dlsym(RTLD_NEXT, "mmap64");
#endif
if (path == NULL || real_sigaltstack == NULL || real_munmap == NULL ||
real_mmap == NULL) {
return;
}
/* Appended to by every process under the preload, hence the pid per line. */
trace_fd = open(path, O_WRONLY | O_CREAT | O_APPEND | O_CLOEXEC, 0600);
}
static void trace_line(const char *event, const char *state, const void *sp) {
char line[128];
int len;
if (trace_fd < 0) {
return;
}
len = snprintf(line, sizeof(line), "%ld %ld %s %s %p\n", (long) getpid(),
(long) syscall(SYS_gettid), event, state, sp);
/* One write per line, so O_APPEND keeps the threads from interleaving. */
if (len > 0 && (size_t) len < sizeof(line)) {
(void) (write(trace_fd, line, (size_t) len) == len);
}
}
static void trace_remember(void *sp) {
const int slot = __atomic_fetch_add(&trace_stacks_used, 1, __ATOMIC_RELAXED);
if (slot < TRACE_STACKS) {
__atomic_store_n(&trace_stacks[slot], sp, __ATOMIC_RELEASE);
}
}
static int trace_is_stack(const void *sp) {
const int used = __atomic_load_n(&trace_stacks_used, __ATOMIC_RELAXED);
int i;
for (i = 0; i < used && i < TRACE_STACKS; i++) {
if (__atomic_load_n(&trace_stacks[i], __ATOMIC_ACQUIRE) == sp) {
return 1;
}
}
return 0;
}
SHIM_EXPORT int sigaltstack(const stack_t *ss, stack_t *old);
SHIM_EXPORT int sigaltstack(const stack_t *ss, stack_t *old) {
const int rc = real_sigaltstack != NULL
? real_sigaltstack(ss, old)
: (int) syscall(SYS_sigaltstack, ss, old);
if (rc != 0 || trace_fd < 0) {
return rc;
}
if (ss != NULL) {
const int off = (ss->ss_flags & SS_DISABLE) != 0;
trace_line("set",
off ? "off"
: ss->ss_sp == trace_last_mmap ? "own"
: "on",
ss->ss_sp);
if (!off) {
trace_remember(ss->ss_sp);
}
} else if (old != NULL) {
trace_line("query", (old->ss_flags & SS_DISABLE) != 0 ? "off" : "on",
old->ss_sp);
}
return rc;
}
static void *trace_mapped(void *sp) {
if (sp != MAP_FAILED) {
trace_last_mmap = sp;
}
return sp;
}
SHIM_EXPORT void *mmap(void *addr, size_t len, int prot, int flags, int fd,
off_t off);
/* hts_backtrace_altstack() installs what it just mapped, with nothing in
between, so the thread's last mapping is the whole of the check. */
SHIM_EXPORT void *mmap(void *addr, size_t len, int prot, int flags, int fd,
off_t off) {
return trace_mapped(
real_mmap != NULL
? real_mmap(addr, len, prot, flags, fd, off)
: (void *) syscall(SYS_mmap, addr, len, prot, flags, fd, off));
}
/* glibc only, and the name that matters: _FILE_OFFSET_BITS=64 redirects the
engine's own mmap() call to mmap64(). musl has no such split. */
#ifdef __GLIBC__
SHIM_EXPORT void *mmap64(void *addr, size_t len, int prot, int flags, int fd,
off64_t off);
SHIM_EXPORT void *mmap64(void *addr, size_t len, int prot, int flags, int fd,
off64_t off) {
return real_mmap64 != NULL
? trace_mapped(real_mmap64(addr, len, prot, flags, fd, off))
: mmap(addr, len, prot, flags, fd, (off_t) off);
}
#endif
SHIM_EXPORT int munmap(void *addr, size_t len);
SHIM_EXPORT int munmap(void *addr, size_t len) {
/* Only the stacks: a process unmaps plenty else, and the trace is read as an
unbroken per-thread sequence. Logged before the call, so a stack the kernel
is still pointed at leaves its record behind. */
if (trace_fd >= 0 && trace_is_stack(addr)) {
trace_line("munmap", "-", addr);
}
return real_munmap != NULL ? real_munmap(addr, len)
: (int) syscall(SYS_munmap, addr, len);
}
#endif /* __linux__ */

View File

@@ -215,9 +215,12 @@ echo "ran=$((pass + fail + skip)) pass=$pass fail=$fail skip=$skip" |
# equivalent of;
# string-oom drives a helper binary that only the automake build produces;
# datadir-ospath copies the unwrapped binary the automake build leaves in .libs,
# and needs the loader variable libtool picked, neither of which this job has.
# and needs the loader variable libtool picked, neither of which this job has;
# link-control-bytes names its fixtures with the raw control bytes the requests
# decode back to, which NTFS refuses.
expected_skips="01_engine-footer-overflow.test
100_local-purge-longpath.test
158_local-link-control-bytes.test
114_local-update-304-leak.test
120_local-proxytrack-webdav-default.test
143_engine-backtrace-empty.test

View File

@@ -194,7 +194,11 @@ tmpdir=
crawlpid=
nopurge=
verbose=
trap cleanup EXIT HUP INT QUIT ILL TRAP ABRT BUS FPE SEGV PIPE ALRM TERM STKFLT XCPU XFSZ
# One at a time: an unknown name (STKFLT is Linux-only) makes the whole trap
# command complain, on every run, on GNU/Hurd and the BSDs.
for sig in EXIT HUP INT QUIT ILL TRAP ABRT BUS FPE SEGV PIPE ALRM TERM STKFLT XCPU XFSZ; do
trap cleanup "$sig" 2>/dev/null || true
done
# working directory
tmpdir="${tmptopdir}/httrack_ut.$$"

View File

@@ -0,0 +1,114 @@
#!/usr/bin/env python3
"""Grade the request bytes header-injection-server.py captured.
Usage: header-injection-check.py <logfile> direct|proxy [port]
"""
import sys
MARKER = b"foo:"
POISON_HOST = b"evil%0d%0afoo:%20injected.example"
FTP_POISON_HOST = b"ftpevil%0d%0afoo:%20x.example"
def requests(path):
blob = open(path, "rb").read()
return [r for r in blob.split(b"=== REQUEST ===\n") if r.strip()]
def lines(req):
"""Split the way a tolerant server would, so a bare LF also ends a line."""
return [line.rstrip(b"\r") for line in req.rstrip(b"\n").split(b"\n")]
def fail(msg, req=None):
print("FAIL: %s" % msg, file=sys.stderr)
if req is not None:
print(repr(req), file=sys.stderr)
sys.exit(1)
def no_injected_header(reqs):
"""Nothing the site controls may start a line of its own."""
for req in reqs:
for line in lines(req)[1:]:
if line.lower().startswith(MARKER):
fail("attacker header line reached the wire", req)
def find(reqs, needle):
return [r for r in reqs if needle in lines(r)[0]]
def check_direct(reqs, port):
host = b"127.0.0.1:%d" % port
deep = find(reqs, b"GET /deep.html ")
if len(deep) != 1:
fail("expected one /deep.html request, got %d" % len(deep))
want = b"Referer: http://" + host + b"/x%0d%0aFoo:%20injected.html"
if want not in lines(deep[0]):
fail("poisoned referer not sent percent-encoded, wanted %r" % want, deep[0])
# positive control: an ordinary referer is still emitted, and unmangled
plain = find(reqs, b"GET /plain.html ")
if len(plain) != 1:
fail("expected one /plain.html request, got %d" % len(plain))
want = b"Referer: http://" + host + b"/"
if want not in lines(plain[0]):
fail("ordinary referer missing or malformed, wanted %r" % want, plain[0])
def check_proxy(reqs):
# the authority is lowercased upstream, hence "foo:" and not "Foo:"
poison = find(reqs, b"/p.html")
poison = [r for r in poison if b"evil" in lines(r)[0]]
if not poison:
fail("the poisoned host was never requested through the proxy")
want_line = b"GET http://" + POISON_HOST + b"/p.html "
want_host = b"Host: " + POISON_HOST
for req in poison:
if not lines(req)[0].startswith(want_line):
fail("proxy request line not escaped, wanted %r" % want_line, req)
if want_host not in lines(req):
fail("Host not escaped, wanted %r" % want_host, req)
# the poisoned host must also reach a Referer, or that escape goes ungraded
deep = find(reqs, b"http://" + POISON_HOST + b"/deep2.html")
if len(deep) != 1:
fail("expected one poisoned-host deep2.html request, got %d" % len(deep))
want = b"Referer: http://" + POISON_HOST + b"/p.html"
if want not in lines(deep[0]):
fail("poisoned referer host not escaped, wanted %r" % want, deep[0])
# ftp through an http proxy is built by its own emission site
ftp = find(reqs, b"ftp://")
if len(ftp) != 1:
fail("expected one ftp-through-proxy request, got %d" % len(ftp))
want = b"GET ftp://" + FTP_POISON_HOST + b"/f.txt "
if not lines(ftp[0])[0].startswith(want):
fail("ftp request line not escaped, wanted %r" % want, ftp[0])
# positive control: an ordinary absolute-URI request is unaffected
plain = find(reqs, b"GET http://plain.example/p.html ")
if len(plain) != 1:
fail("expected one plain.example/p.html request, got %d" % len(plain))
if b"Host: plain.example" not in lines(plain[0]):
fail("ordinary Host missing or malformed", plain[0])
def main():
path, mode = sys.argv[1], sys.argv[2]
reqs = requests(path)
if not reqs:
fail("no request was captured at all")
no_injected_header(reqs)
if mode == "direct":
check_direct(reqs, int(sys.argv[3]))
else:
check_proxy(reqs)
print("OK: %s, %d requests, none carrying an injected header" % (mode, len(reqs)))
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,104 @@
#!/usr/bin/env python3
"""Raw-socket probe for 159_local-header-injection.
Speaks HTTP off the socket so a split header line stays visible; serves an
origin-form site and an http-proxy absolute-URI one on the same port.
Appends "=== REQUEST ===\\n<bytes>\\n" per request to the log named on argv, and
prints "PORT <n>" once listening.
"""
import os
import socket
import sys
import threading
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from proxytestlib import bind_ephemeral # noqa: E402
# &#13;&#10; outlives the parser's control-byte escape, so these land as raw CR/LF
POISON_PATH = "/x&#13;&#10;Foo:%20injected.html"
POISON_HOST = "evil&#13;&#10;Foo:%20injected.example"
FTP_POISON_HOST = "ftpevil&#13;&#10;Foo:%20x.example"
def page(body):
return ("<html><body>%s</body></html>" % body).encode()
def root_page(port):
return page(
'<a href="http://127.0.0.1:%d%s">poison</a>'
'<a href="http://127.0.0.1:%d/plain.html">plain</a>' % (port, POISON_PATH, port)
)
# the poisoned page must link onward, or its URL never becomes a Referer
POISON_PAGE = page('<a href="/deep.html">deep</a>')
# the ftp link takes the ftp-through-proxy emission site, which has no Host:
PROXY_ROOT = page(
'<a href="ftp://' + FTP_POISON_HOST + '/f.txt">ftp</a>'
'<a href="http://' + POISON_HOST + '/p.html">poison</a>'
'<a href="http://plain.example/p.html">plain</a>'
)
def body_for(request, port):
"""Pick a body from the request line alone; a split one just gets a leaf."""
line = request.split(b"\r\n", 1)[0]
if b"start.example" in line:
return PROXY_ROOT
if line.startswith(b"GET / "):
return root_page(port)
if b"/x" in line:
return POISON_PAGE
# the proxied pages link onward too, so a poisoned *host* becomes a referer
if b"/p.html" in line:
return page('<a href="/deep2.html">deep</a>')
return page("leaf")
def handle(conn, port, logf, lock):
conn.settimeout(10)
data = b""
try:
while b"\r\n\r\n" not in data:
chunk = conn.recv(4096)
if not chunk:
break
data += chunk
except (OSError, socket.timeout):
pass
if data:
with lock:
logf.write(b"=== REQUEST ===\n" + data + b"\n")
logf.flush()
body = body_for(data, port)
try:
conn.sendall(
b"HTTP/1.1 200 OK\r\nContent-Type: text/html\r\n"
b"Content-Length: %d\r\nConnection: close\r\n\r\n" % len(body) + body
)
except OSError:
pass
try:
conn.close()
except OSError:
pass
def main():
srv, port = bind_ephemeral()
lock = threading.Lock()
with open(sys.argv[1], "wb") as logf:
print("PORT %d" % port, flush=True)
while True:
conn, _ = srv.accept()
threading.Thread(
target=handle, args=(conn, port, logf, lock), daemon=True
).start()
if __name__ == "__main__":
main()

View File

@@ -2084,6 +2084,16 @@ class Handler(SimpleHTTPRequestHandler):
except OSError:
pass
# #973: quoted, so the link parser keeps the < and > (only an unquoted > ends
# a link) and the whole run reaches the progress panel.
XSS_NAME = "p0<img src=x onerror=alert(1)>'quote.bin"
# The panel splits the URL at the last slash, so only a directory reaches its
# name column. Short, because the engine elides that column past 40 chars.
XSS_DIR = "d0<i'>"
def route_xssjob_index(self):
self.send_html('\t<a href="%s/%s">job</a>\n' % (self.XSS_DIR, self.XSS_NAME))
# #483: trickled .bin pages so the -E stop lands in the type waiter's
# unlock-to-patch window with body bytes pending.
def route_dcancel_index(self):
@@ -2464,6 +2474,8 @@ class Handler(SimpleHTTPRequestHandler):
"/cdispo/evil.php": route_cdispo,
"/delayed/index.html": route_delayed_index,
"/trickle/index.html": route_trickle_index,
"/xssjob/": route_xssjob_index,
"/xssjob/index.html": route_xssjob_index,
"/trickle/p0.bin": route_trickle_page,
"/trickle/p1.bin": route_trickle_page,
"/trickle/p2.bin": route_trickle_page,
@@ -2740,6 +2752,9 @@ class Handler(SimpleHTTPRequestHandler):
handler = type(self).route_sitemap_capset
elif re.fullmatch(r"/sitemapdir/cap\d+\.html", path):
handler = type(self).route_sitemap_cappage
elif path.startswith("/xssjob/"):
# Whatever the engine made of the metacharacters, the job trickles.
handler = type(self).route_trickle_page
if handler is not None:
handler(self)
return True

12
tests/nopieprobe.c Normal file
View File

@@ -0,0 +1,12 @@
/* Crash-report driver for 218_crash-nopie-frames.test: links htsbacktrace.c
into a non-PIE program of its own, which the build tree cannot produce. */
#include "htsbacktrace.h"
static void probe_frame(void) { hts_print_backtrace(); }
int main(void) {
hts_backtrace_init();
probe_frame();
return 0;
}

View File

@@ -24,6 +24,9 @@ budget=${HTTRACK_TEST_TIMEOUT:-600}
case "$budget" in
'' | *[!0-9]*) budget=600 ;;
esac
# Exported so a test can pace itself against the same number (skip_if_out_of_budget)
# instead of being killed halfway.
export HTTRACK_TEST_TIMEOUT="$budget"
test "$budget" -gt 0 || exec "$BASH" "$@"
# The test script is the last argument; automake passes no others today.

View File

@@ -48,6 +48,22 @@ expect_ok() {
esac
}
# Run src/ install target $1 into DESTDIR $2, logging to $3. The configured prefix
# survives DESTDIR, so libtool will not relink the build the rest of the suite runs
# against; MAKEFLAGS and MAKELEVEL cleared, no jobserver to hunt.
stage_install_target() {
local target=$1 dest=$2 log=$3
env -u MAKEFLAGS -u MAKELEVEL "${MAKE:-make}" -C "${abs_top_builddir:?}/src" \
"${target}" DESTDIR="${dest}" >"${log}" 2>&1 && return 0
cat "${log}" >&2
return 1
}
# The binaries and the library alone, for a caller reading what was linked.
stage_install_exec() {
stage_install_target install-exec "$1" "$2"
}
is_windows() {
case "$(uname -s)" in
MINGW* | MSYS* | CYGWIN*) return 0 ;;
@@ -350,6 +366,21 @@ EOF
export PATH
}
# Skip when the next of $1 remaining steps, at 1.5x the $2 seconds the last one
# took, no longer fits the budget meant to catch a wedge (hppa spends ~150s on one
# configure run and would else FTBFS). One step ahead rather than all of them: 196
# shares a config.cache, so its first step costs several times the rest and
# projecting it over them would skip a run that fits.
skip_if_out_of_budget() { # skip_if_out_of_budget <steps left> <seconds the last took>
local budget=${HTTRACK_TEST_TIMEOUT:-600} need=$(($2 + $2 / 2))
case "$budget" in '' | *[!0-9]*) budget=600 ;; esac
test "$1" -gt 0 && test "$budget" -gt 0 || return 0
test "$((SECONDS + need))" -ge "$budget" || return 0
echo "$1 steps left, the last took ${2}s and the budget is ${budget}s; skipping" >&2
exit 77
}
# Collect a killed job, giving up after REAP_GRACE seconds. kill_tree can fail to
# reap a native Windows descendant -- the very case these watchdogs exist for --
# and a bare `wait` then blocks the watchdog itself forever, so the timeout it was

View File

@@ -241,6 +241,21 @@ TESTS += 205_install-headers.test
TESTS += 185_webhttrack-js-escaping.test
TESTS += 200_pixmaps-fallback.test
TESTS += 210_appstream-metainfo.test
TESTS += 183_altstack-worker.test
TESTS += 186_webhttrack-url-escaping.test
TESTS += 207_install-headers-symbols.test
TESTS += 182_crash-fork-safety.test
TESTS += 155_engine-escape-control.test
TESTS += 206_install-headers-c99.test
TESTS += 215_engine-datadir-ospath.test
TESTS += 157_crash-argv0-path.test
TESTS += 159_local-header-injection.test
TESTS += 158_local-link-control-bytes.test
TESTS += 219_install-rpath-darwin.test
TESTS += 220_webhttrack-mirror-isolation.test
TESTS += 223_webhttrack-session-id.test
TESTS += 217_webhttrack-attr-escaping.test
TESTS += 216_engine-ftp-ctrlchars.test
TESTS += 221_local-ftp-ctrlchars.test
TESTS += 218_crash-nopie-frames.test
TESTS += 222_pkgconfig-consumer.test

View File

@@ -71,7 +71,8 @@ mkdir -p "$app/Contents/MacOS" "$app/Contents/Resources"
cp "$plist" "$app/Contents/Info.plist"
cp "$icon" "$app/Contents/Resources/$(basename "$icon")"
cp -R "$prefix"/. "$app/Contents/Resources/"
rm -rf "$app/Contents/Resources/include"
# Development surface only, and the .pc would carry the staging prefix in with it.
rm -rf "$app/Contents/Resources/include" "$app/Contents/Resources/lib/pkgconfig"
find "$app/Contents/Resources" \( -name '*.la' -o -name '*.a' \) -delete
# webhttrack carries the configure-time datadir as a --datadir fallback (#887);