Compare commits

..

9 Commits

Author SHA1 Message Date
Xavier Roche
2b3f1032ef Merge origin/master into warnfix-dead
# Conflicts:
#	tests/Makefile.am
2026-07-26 14:36:48 +02:00
Xavier Roche
0b1924194f tests: skip the WebDAV mime test on Windows
It is the first test to run proxytrack as a live listener, and MSYS cannot
reap a native one: the orphan wedged the whole Windows suite past its
45-minute budget, twice, destroying the log upload with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-26 14:12:50 +02:00
Xavier Roche
feace15693 tests: bound the PROPFIND request and cut the header comment
curl had no --max-time; an unbounded read wedges the runner instead of
failing it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-26 12:52:51 +02:00
Xavier Roche
bd8b2f8d15 tests: prove the WebDAV mime/timestamp fallback survives an empty field
proxytrack's DAV PROPFIND response computes a fallback content-type and
timestamp when a cache entry has no Content-Type/Last-Modified; that
fallback already existed before commit eae1dd0 changed the surrounding
always-true array-address checks, so this test guards the equivalence
rather than a bug. Verified it fails when the fallback default is
disabled, and passes unmodified against the pre-eae1dd0 code too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-26 12:23:58 +02:00
Xavier Roche
102fafffa7 Merge origin/master into warnfix-dead 2026-07-26 11:37:39 +02:00
Xavier Roche
5a52f90862 Fix the discarded const qualifiers rather than casting them away
`binput` and `cache_binput` only ever read through their source pointer,
so they take `const char *` now; that alone clears the cast in
htsrobots.c, and neither is exported nor declared in an installed header,
so no ABI question arises. `treathead` keeps `char *rcvd` because it does
NUL-cut the header in place, and the two selftest calls that fed it a
string literal get a mutable buffer instead, matching their three
siblings and removing a latent write to .rodata. The remaining two are
one-liners: zlib's `next_in` is already `const` under -DZLIB_CONST, and
htsback can call the non-const `jump_protocol` twin on its mutable
`url_adr`. libhttrack.vcxproj gains ZLIB_CONST so the MSVC build agrees
with autotools, as webhttrack and proxytrack already do.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-26 11:37:34 +02:00
Xavier Roche
eae1dd0b77 proxytrack: test the WebDAV header fields, not their addresses
`PT_Element::lastmodified` and `::contenttype` are inline arrays, so both
`if`s were constant-true (-Waddress); use the `[0]` form the same file
already uses when it emits the GET headers. Neither is observable:
get_time_rfc822("") returns 0 and falls through to the index timestamp,
and proxytrack_add_DAV_Item already substitutes application/octet-stream
for an empty mime, which a PROPFIND probe against a cache entry carrying
no Content-Type confirms both before and after.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-26 10:28:30 +02:00
Xavier Roche
cd0d9c4d88 help_wizard: check the allocation, not the arrays it contains
The out-of-memory guard has been constant-false since d593418 folded the
nine separate wizard buffers into one struct: the names it tests are now
inline arrays, so `malloct()`'s result is never checked and an exhausted
heap gets a NULL-page write instead of the intended message. Also switch
the raw free() to freet() and release the struct on the two early returns
that leaked it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-26 10:18:55 +02:00
Xavier Roche
b15362e6d6 Drop NULL tests on inline array members
`lien_back::url_sav`, `htsblk::msg` and POSIX `dirent::d_name` are arrays,
so testing their address folds to a constant and gcc/clang report it
(-Waddress, -Wpointer-bool-conversion). Every site keeps whatever real
condition sat beside the dead one, so behavior is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-26 10:17:39 +02:00
33 changed files with 200 additions and 1392 deletions

View File

@@ -100,8 +100,7 @@ AX_CHECK_COMPILE_FLAG([-fstack-protector-strong], [DEFAULT_CFLAGS="$DEFAULT_CFLA
[AX_CHECK_COMPILE_FLAG([-fstack-protector], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstack-protector"], [], [-Werror])], [-Werror])
AX_CHECK_COMPILE_FLAG([-fstack-clash-protection], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fstack-clash-protection"], [], [-Werror])
AX_CHECK_COMPILE_FLAG([-fcf-protection], [DEFAULT_CFLAGS="$DEFAULT_CFLAGS -fcf-protection"], [], [-Werror])
# No --discard-all: it drops the local symbols naming every static function, so
# a trace misattributes them to the nearest surviving global. Costs 0.6% size.
AX_CHECK_LINK_FLAG([-Wl,--discard-all], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,--discard-all"])
AX_CHECK_LINK_FLAG([-Wl,--no-undefined], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,--no-undefined"])
AX_CHECK_LINK_FLAG([-Wl,-z,relro,-z,now], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,-z,relro,-z,now"])
AX_CHECK_LINK_FLAG([-Wl,-z,noexecstack], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,-z,noexecstack"])
@@ -128,8 +127,8 @@ AX_CHECK_LINK_FLAG([-pie], [LDFLAGS_PIE="-pie"])
AC_SUBST([CFLAGS_PIE])
AC_SUBST([LDFLAGS_PIE])
# Ties a crash trace from a stripped build back to its separate debug symbols.
AX_CHECK_LINK_FLAG([-Wl,--build-id], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -Wl,--build-id"])
## -rdynamic must be a link flag; DEFAULT_CFLAGS (AM_CPPFLAGS) never reaches the linker.
AX_CHECK_LINK_FLAG([-rdynamic], [DEFAULT_LDFLAGS="$DEFAULT_LDFLAGS -rdynamic"])
### Check for -fvisibility=hidden support
gl_VISIBILITY

View File

@@ -103,7 +103,7 @@ ${LANG_Q3}
<tr><td>
<table width="100%">
<tr><td align="left">
<input type="submit" value="${LANG_OK}"
<input type="submit" value="${LANG_OK]"
${do:output-mode:html-urlescaped}
onClick="if (confirm(str_replace(str_replace('${LANG_DIAL7}', '%20', ' '), '%0a', ' '))) { form.closeme.value=1; form.submit(); } return false;"
${do:output-mode:}

View File

@@ -121,9 +121,9 @@ httrack \
--quiet \
--build-top-index \
${test:todo:--mirror:--mirror:--mirror-wizard:--get:--mirrorlinks:--testlinks:--continue:--update}
${unquoted:urls}
${test:filelist:-%L "}${arg:filelist}${test:filelist:"}
--path "${arg:path}/${arg:projname}"
${urls}
${test:filelist:-%L "}${filelist}${test:filelist:"}
--path "${html:path}/${html:projname}"
\
${test:parseall:--near}
${test:link:--test}
@@ -131,7 +131,7 @@ httrack \
${test:htmlfirst::--priority=7}
\
${do:if-not-empty:BuildString}
--structure "${arg:BuildString}"
--structure "${BuildString}"
${do:end-if}
${test:build:-N0:-N0:-N1:-N2:-N3:-N4:-N5:-N100:-N101:-N102:-N103:-N104:-N105:-N99:-N199:}
\
@@ -150,30 +150,29 @@ ${do:end-if}
${test:travel3::--keep-links=0:--keep-links:--keep-links=3:--keep-links=4}
${test:windebug:--debug-headers}
\
${test:connexion:--sockets=}${unquoted:connexion}
${test:connexion:--sockets=}${connexion}
${test:ka:--keep-alive}
${test:timeout:--timeout=}${unquoted:timeout}
${test:timeout:--timeout=}${timeout}
${test:remt:--host-control=1}
${test:retry:--retries=}${unquoted:retry}
${test:rate:--min-rate=}${unquoted:rate}
${test:retry:--retries=}${retry}
${test:rate:--min-rate=}${rate}
${test:rems:--host-control=2}
\
${test:depth:--depth=}${unquoted:depth}
${test:depth2:--ext-depth=}${unquoted:depth2}
${/* -m<n> resets the html limit, so the bare form must precede the -m,<n> one */}
${test:othermax:--max-files=}${unquoted:othermax}
${test:maxhtml:--max-files=,}${unquoted:maxhtml}
${test:sizemax:--max-size=}${unquoted:sizemax}
${test:pausebytes:--max-pause=}${unquoted:pausebytes}
${test:maxtime:--max-time=}${unquoted:maxtime}
${test:maxrate:--max-rate=}${unquoted:maxrate}
${test:maxconn:--connection-per-second=}${unquoted:maxconn}
${test:maxlinks:--advanced-maxlinks=}${unquoted:maxlinks}
${test:depth:--depth=}${depth}
${test:depth2:--ext-depth=}${depth2}
${test:maxhtml:--max-files=,}${maxhtml}
${test:othermax:--max-files=}${othermax}
${test:sizemax:--max-files=}${sizemax}
${test:pausebytes:--max-pause=}${pausebytes}
${test:maxtime:--max-time=}${maxtime}
${test:maxrate:--max-rate=}${maxrate}
${test:maxconn:--connection-per-second=}${maxconn}
${test:maxlinks:--advanced-maxlinks=}${maxlinks}
\
--user-agent "${arg:user}"
--footer "${arg:footer}"
--user-agent "${html:user}"
--footer "${html:footer}"
\
${unquoted:url2}
${url2}
\
${test:cookies:--cookies=0:}
${test:parsejava:--parse-java=0:}
@@ -182,20 +181,20 @@ ${/* -m<n> resets the html limit, so the bare form must precede the -m,<n> one *
${test:keepwww:--keep-www-prefix}
${test:keepslashes:--keep-double-slashes}
${test:keepqueryorder:--keep-query-order}
${test:cookiesfile:--cookies-file "}${arg:cookiesfile}${test:cookiesfile:"}
${test:pausefiles:--pause "}${arg:pausefiles}${test:pausefiles:"}
${test:stripquery:--strip-query "}${arg:stripquery}${test:stripquery:"}
${test:cookiesfile:--cookies-file "}${html:cookiesfile}${test:cookiesfile:"}
${test:pausefiles:--pause "}${pausefiles}${test:pausefiles:"}
${test:stripquery:--strip-query "}${html:stripquery}${test:stripquery:"}
${test:toler:--tolerant}
${test:http10:--http-10}
${test:cache2:--store-all-in-cache}
${test:warc:--warc}
${test:warcfile:--warc-file "}${arg:warcfile}${test:warcfile:"}
${test:warcfile:--warc-file "}${html:warcfile}${test:warcfile:"}
${test:norecatch:--do-not-recatch}
${test:logf:--single-log}
${test:logtype:::--extra-log:--debug-log}
${test:index:--index=0:}
${test:index2:--search-index=0:--search-index}
${test:prox:--proxy "}${do:if-not-empty:prox}${test:proxytype::socks5:connect}${test:proxytype:\3A//}${do:end-if}${do:output-mode:html}${arg:prox}${test:prox:\3A}${arg:portprox}${test:prox:"}
${test:prox:--proxy "}${do:if-not-empty:prox}${test:proxytype::socks5:connect}${test:proxytype:\3A//}${do:end-if}${do:output-mode:html}${prox}${test:prox:\3A}${portprox}${test:prox:"}
${test:ftpprox:--httpproxy-ftp=0:--httpproxy-ftp}
</textarea>
@@ -214,7 +213,7 @@ ParseAll=${ztest:parseall:0:1}
HTMLFirst=${ztest:htmlfirst:0:1}
Cache=${ztest:cache:0:1}
NoRecatch=${ztest:norecatch:0:1}
Dos=${dos}
Dos=${dos
Index=${ztest:index:0:1}
WordIndex=${ztest:index2:0:1}
Log=${ztest:logf:0:1:2}

View File

@@ -32,9 +32,7 @@ AM_LDFLAGS = \
bin_PROGRAMS = proxytrack httrack htsserver
httrack_SOURCES = httrack.c htsbacktrace.c htsbacktrace.h
# $(DL_LIBS): dladdr() in the crash handler, still in libdl on pre-2.34 glibc.
httrack_LDADD = $(THREADS_LIBS) $(DL_LIBS) libhttrack.la
httrack_LDADD = $(THREADS_LIBS) libhttrack.la
htsserver_LDADD = $(THREADS_LIBS) $(SOCKET_LIBS) libhttrack.la
proxytrack_LDADD = $(THREADS_LIBS) $(SOCKET_LIBS)
@@ -49,7 +47,6 @@ htsserver_LDFLAGS = $(AM_LDFLAGS) $(LDFLAGS_PIE)
lib_LTLIBRARIES = libhttrack.la
htsserver_SOURCES = htsserver.c htsserver.h htsweb.c htsweb.h \
htscmdline.c htscmdline.h \
htsurlport.c htsurlport.h
proxytrack_SOURCES = proxy/main.c \
proxy/proxytrack.c proxy/store.c \
@@ -63,7 +60,7 @@ whttrackrun_SCRIPTS = webhttrack
libhttrack_la_SOURCES = htscore.c htsparse.c htsback.c htscache.c \
htscache_selftest.c htsdns_selftest.c htsselftest.c \
htscatchurl.c htsfilters.c htsftp.c htshash.c coucal/coucal.c \
htscmdline.c htshelp.c htslib.c htsurlport.c htscoremain.c \
htshelp.c htslib.c htsurlport.c htscoremain.c \
htsname.c htsrobots.c htstools.c htswizard.c \
htsalias.c htsthread.c htsindex.c htsbauth.c \
htsmd5.c htscodec.c htswarc.c htsproxy.c htszlib.c htswrap.c htsconcat.c \
@@ -72,7 +69,7 @@ libhttrack_la_SOURCES = htscore.c htsparse.c htsback.c htscache.c \
minizip/ioapi.c minizip/mztools.c minizip/unzip.c minizip/zip.c \
hts-indextmpl.h htsalias.h htsback.h htsbase.h htssafe.h \
htsbasenet.h htsbauth.h htscache.h htscache_selftest.h htsdns_selftest.h htsselftest.h htscatchurl.h \
htscmdline.h htsconfig.h htscore.h htsparse.h htscoremain.h htsdefines.h \
htsconfig.h htscore.h htsparse.h htscoremain.h htsdefines.h \
htsfilters.h htsftp.h htsglobal.h htshash.h coucal/coucal.h \
htshelp.h htsindex.h htslib.h htsurlport.h htsmd5.h \
htsmodules.h htsname.h htsnet.h htssniff.h \

View File

@@ -542,14 +542,8 @@ static int create_back_tmpfile(httrackp *opt, lien_back *const back,
// do not use tempnam() but a regular filename
back->tmpfile_buffer[0] = '\0';
if (back->url_sav[0] != '\0') {
/* same capacity as url_sav, so truncation drops the extension and aliases
the temp name onto the live file that back_finalize_backup() UNLINKs */
if (!sprintfbuff(back->tmpfile_buffer, "%s.%s", back->url_sav, ext)) {
hts_log_print(opt, LOG_WARNING, "temporary filename too long for %s",
back->url_sav);
back->tmpfile_buffer[0] = '\0';
return -1;
}
snprintf(back->tmpfile_buffer, sizeof(back->tmpfile_buffer), "%s.%s",
back->url_sav, ext);
back->tmpfile = back->tmpfile_buffer;
if (structcheck(back->tmpfile) != 0) {
hts_log_print(opt, LOG_WARNING, "can not create directory to %s",
@@ -557,15 +551,8 @@ static int create_back_tmpfile(httrackp *opt, lien_back *const back,
return -1;
}
} else {
/* truncation here would collide distinct tmpnameid's onto one name */
if (!sprintfbuff(back->tmpfile_buffer, "%s/tmp%d.%s",
StringBuff(opt->path_html_utf8), opt->state.tmpnameid++,
ext)) {
hts_log_print(opt, LOG_WARNING, "temporary filename too long in %s",
StringBuff(opt->path_html_utf8));
back->tmpfile_buffer[0] = '\0';
return -1;
}
snprintf(back->tmpfile_buffer, sizeof(back->tmpfile_buffer), "%s/tmp%d.%s",
StringBuff(opt->path_html_utf8), opt->state.tmpnameid++, ext);
back->tmpfile = back->tmpfile_buffer;
}
/* OK */

View File

@@ -1,238 +0,0 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* File: crash backtrace printer */
/* Author: Xavier Roche */
/* ------------------------------------------------------------ */
/* Before every header: glibc gates dladdr() on it. */
#if defined(__linux) && !defined(_GNU_SOURCE)
#define _GNU_SOURCE
#endif
#include "htsbacktrace.h"
#include "htsglobal.h"
#include <stdlib.h>
#include <string.h>
#ifdef _WIN32
#include <io.h> /* write */
#endif
#ifdef HAVE_UNISTD_H
#include <unistd.h>
#endif
#if (defined(__linux) && defined(HAVE_EXECINFO_H))
#include <dlfcn.h>
#include <errno.h>
#include <execinfo.h>
#include <signal.h>
#include <time.h>
#include <sys/wait.h>
#define USES_BACKTRACE
#endif
#ifdef USES_BACKTRACE
#define BT_MAX_FRAMES 64 /* frames we try to name */
#define BT_MAX_MODULES 8 /* distinct modules, one child each */
#define BT_HEX_SIZE 19 /* "0x" + 16 nibbles + NUL */
#define BT_PATH_SIZE 1024 /* module path; longer is skipped */
#define BT_WAIT_TICKS 300 /* 10ms ticks, shared: cap a slow child */
#define BT_NO_SYMBOLIZER 127 /* child exit: execvp() found none */
static hts_boolean symbolize_crash = HTS_TRUE;
/* "0x"-prefixed hex: the handler must stay stdio-free. */
static void print_hex(char *buffer, uintptr_t value) {
static const char digits[] = "0123456789abcdef";
size_t i = 2, a, b;
buffer[0] = '0';
buffer[1] = 'x';
do {
buffer[i++] = digits[value & 0xf];
value >>= 4;
} while (value != 0);
buffer[i] = '\0';
for (a = 2, b = i - 1; a < b; a++, b--) {
const char c = buffer[a];
buffer[a] = buffer[b];
buffer[b] = c;
}
}
/* HTS_FALSE if src does not fit: a truncated module path would point the
symbolizer at the wrong file. */
static hts_boolean copy_bounded(char *dest, size_t size, const char *src) {
size_t i;
for (i = 0; i < size - 1 && src[i] != '\0'; i++) {
dest[i] = src[i];
}
dest[i] = '\0';
return src[i] == '\0' ? HTS_TRUE : HTS_FALSE;
}
/* Run the symbolizer on argv, output on fd, within *budget ticks. HTS_FALSE
only if none could be run at all; otherwise silent, the raw trace stands. */
static hts_boolean spawn_symbolizer(char **argv, int fd, int *budget) {
const pid_t pid = fork();
int status = 0;
if (pid == -1)
return HTS_FALSE;
if (pid == 0) {
static char llvm_prog[] = "llvm-symbolizer";
static char llvm_opts[] = "-p";
dup2(fd, 1); /* both symbolizers write on stdout */
execvp(argv[0], argv);
argv[0] = llvm_prog; /* an LLVM-only install ships no addr2line */
argv[1] = llvm_opts;
execvp(argv[0], argv);
_exit(BT_NO_SYMBOLIZER);
}
for (; *budget > 0; (*budget)--) {
const struct timespec tick = {0, 10 * 1000 * 1000};
const pid_t reaped = waitpid(pid, &status, WNOHANG);
if (reaped == pid)
return WIFEXITED(status) && WEXITSTATUS(status) == BT_NO_SYMBOLIZER
? HTS_FALSE
: HTS_TRUE;
if (reaped == -1 && errno != EINTR)
return HTS_TRUE;
nanosleep(&tick, NULL);
}
kill(pid, SIGKILL);
waitpid(pid, NULL, 0);
return HTS_TRUE;
}
/* Name the frames backtrace_symbols_fd() leaves as module+offset:
-fvisibility=hidden keeps them out of .dynsym, but DWARF has them. dladdr()
is not formally async-signal-safe; accepted, this path is already fatal. */
static void symbolize_backtrace(void *const *stack, int size, int fd) {
static char prog[] = "addr2line";
static char opts[] = "-Cfipa";
static char dashe[] = "-e";
char hex[BT_MAX_FRAMES][BT_HEX_SIZE];
const void *base[BT_MAX_FRAMES];
const char *name[BT_MAX_FRAMES];
hts_boolean grouped[BT_MAX_FRAMES];
char module[BT_PATH_SIZE];
char *argv[4 + BT_MAX_FRAMES + 1];
int budget = BT_WAIT_TICKS;
int i, spawned;
if (size > BT_MAX_FRAMES)
size = BT_MAX_FRAMES;
for (i = 0; i < size; i++) {
Dl_info info;
grouped[i] = HTS_TRUE; /* skipped unless dladdr() places the frame */
if (dladdr(stack[i], &info) == 0 || info.dli_fname == NULL ||
info.dli_fname[0] == '\0')
continue;
base[i] = info.dli_fbase;
name[i] = info.dli_fname;
print_hex(hex[i], (uintptr_t) ((const char *) stack[i] -
(const char *) info.dli_fbase));
grouped[i] = HTS_FALSE;
}
/* One child per module: addr2line takes a single -e. Each frame is claimed
once, so argc cannot exceed argv[]. */
for (spawned = 0; spawned < BT_MAX_MODULES; spawned++) {
int first, j, argc = 0;
for (first = 0; first < size && grouped[first]; first++)
;
if (first >= size)
break;
argv[argc++] = prog;
argv[argc++] = opts;
argv[argc++] = dashe;
argv[argc++] = module;
for (j = first; j < size; j++) {
if (grouped[j] || base[j] != base[first])
continue;
grouped[j] = HTS_TRUE;
argv[argc++] = hex[j];
}
argv[argc] = NULL;
/* access(): skip pseudo-modules like linux-vdso, which have no file and
would draw nothing but an addr2line complaint. */
if (copy_bounded(module, sizeof(module), name[first]) &&
access(module, R_OK) == 0) {
const size_t len = strlen(module);
/* addr2line -a prints offsets only: say which module they are in. */
(void) (write(fd, module, len) == (ssize_t) len);
(void) (write(fd, ":\n", 2) == 2);
if (!spawn_symbolizer(argv, fd, &budget))
break; /* no symbolizer: stop at one header */
}
}
}
#endif
void hts_backtrace_init(void) {
#ifdef USES_BACKTRACE
symbolize_crash =
getenv("HTTRACK_NO_SYMBOLIZE") == NULL ? HTS_TRUE : HTS_FALSE;
#endif
}
void hts_print_backtrace(int fd) {
#ifdef USES_BACKTRACE
void *stack[256];
const int size = backtrace(stack, sizeof(stack) / sizeof(stack[0]));
/* A fault inside the handler lands back here: symbolizing twice interleaves
two traces on fd and spends a second budget. */
static volatile sig_atomic_t entered = 0;
if (size != 0) {
backtrace_symbols_fd(stack, size, fd);
if (symbolize_crash && entered == 0) {
entered = 1;
symbolize_backtrace(stack, size, fd);
entered = 0;
}
}
#else
const char msg[] = "No stack trace available on this OS :(\n";
if (write(fd, msg, sizeof(msg) - 1) != sizeof(msg) - 1) {
/* sorry GCC */
}
#endif
}

View File

@@ -1,45 +0,0 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* File: crash backtrace printer */
/* Author: Xavier Roche */
/* ------------------------------------------------------------ */
#ifndef HTSBACKTRACE_DEFH
#define HTSBACKTRACE_DEFH
/* Sample HTTRACK_NO_SYMBOLIZE before any crash: getenv() is not signal-safe.
Call once, from the process that installs the fatal-signal handlers. */
void hts_backtrace_init(void);
/* Write the calling thread's stack to fd, callable from a fatal signal handler:
raw frames first, then whatever an external symbolizer can name. Allocates
nothing; prints a one-line notice where the OS has no backtrace(). */
void hts_print_backtrace(int fd);
#endif

View File

@@ -1,95 +0,0 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* File: command line splitter, shared by the engine and */
/* htsserver */
/* Author: Xavier Roche */
/* ------------------------------------------------------------ */
#include "htscmdline.h"
#include "htssafe.h"
#include <limits.h>
#include <stdint.h>
char **hts_split_cmdline(char *cmd, int *nargs) {
size_t nsep = 0;
size_t capacity;
size_t r;
size_t w;
int argc = 0;
hts_boolean quoted = HTS_FALSE;
char **argv;
*nargs = 0;
/* fold the other separators, so counting them sizes the vector exactly */
for (r = 0; cmd[r] != '\0'; r++) {
if (cmd[r] == '\t' || cmd[r] == '\r' || cmd[r] == '\n') {
cmd[r] = ' ';
}
if (cmd[r] == ' ') {
nsep++;
}
}
/* at most one argument per separator, plus the leading one and the NULL */
if (nsep > (size_t) INT_MAX - 1 || nsep > SIZE_MAX / sizeof(char *) - 2) {
return NULL;
}
capacity = nsep + 2;
argv = (char **) malloct(capacity * sizeof(char *));
if (argv == NULL) {
return NULL;
}
argv[argc++] = cmd;
for (r = 0, w = 0; cmd[r] != '\0';) {
if (quoted && cmd[r] == '\\' &&
(cmd[r + 1] == '\\' || cmd[r + 1] == '\"')) {
r++;
cmd[w++] = cmd[r++];
} else if (cmd[r] == '\"') {
quoted = !quoted;
cmd[w++] = cmd[r++];
} else if (cmd[r] == ' ' && !quoted) {
cmd[w++] = '\0';
assertf((size_t) argc < capacity - 1); /* the last slot holds the NULL */
argv[argc++] = cmd + w;
r++;
} else {
cmd[w++] = cmd[r++];
}
}
cmd[w] = '\0';
argv[argc] = NULL; /* callers may rely on argv[argc] == NULL */
*nargs = argc;
return argv;
}

View File

@@ -1,45 +0,0 @@
/* ------------------------------------------------------------ */
/*
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998 Xavier Roche and other contributors
SPDX-License-Identifier: GPL-3.0-or-later
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Ethical use: we kindly ask that you NOT use this software to harvest email
addresses or to collect any other private information about people. Doing so
would dishonor our work and waste the many hours we have spent on it.
Please visit our Website: http://www.httrack.com
*/
/* ------------------------------------------------------------ */
/* File: command line splitter, shared by the engine and */
/* htsserver */
/* Author: Xavier Roche */
/* ------------------------------------------------------------ */
#ifndef HTSCMDLINE_DEFH
#define HTSCMDLINE_DEFH
#include "htsglobal.h"
/* Split "cmd" in place into a NULL-terminated argv vector of *nargs entries,
argv[0] being the program name and quotes left for the engine to strip.
Returns a malloct'ed vector of pointers into cmd (freet the vector, never its
entries), or NULL when it cannot be sized or allocated. */
char **hts_split_cmdline(char *cmd, int *nargs);
#endif

View File

@@ -447,22 +447,13 @@ void hts_finish_makeindex(httrackp *opt, int *makeindex_done,
const char *fil) {
if (!*makeindex_done) {
if (*makeindex_fp) {
/* sized off link_escaped below: at the old flat 1024 a long first link
produced a redirect to a clipped URL */
char BIGSTK tempo[HTS_URLMAXSIZE * 2 + 64];
char BIGSTK tempo[1024];
if (makeindex_links == 1) {
char BIGSTK link_escaped[HTS_URLMAXSIZE * 2];
escape_uri_utf(makeindex_firstlink, link_escaped, sizeof(link_escaped));
/* no redirect beats one pointing at a clipped URL */
if (!sprintfbuff(
tempo,
"<meta HTTP-EQUIV=\"Refresh\" CONTENT=\"0; URL=%s\">" CRLF,
link_escaped)) {
hts_log_print(opt, LOG_WARNING,
"index redirect omitted: first link too long (%s)",
makeindex_firstlink);
tempo[0] = '\0';
}
snprintf(tempo, sizeof(tempo),
"<meta HTTP-EQUIV=\"Refresh\" CONTENT=\"0; URL=%s\">" CRLF,
link_escaped);
} else
tempo[0] = '\0';
hts_template_format(*makeindex_fp, template_footer,

View File

@@ -251,7 +251,7 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
// folding a nonsense port into 1..65535 fetches one the link never named;
// an empty "host:" just means the default (#614)
if (a[1] != '\0' && !hts_parse_url_port(a + 1, &port)) {
htsblk_failf(&back->r, "Invalid port: %s", a + 1);
snprintf(back->r.msg, sizeof(back->r.msg), "Invalid port: %s", a + 1);
back->r.statuscode = STATUSCODE_INVALID; // permanent, unlike a DNS miss
_HALT_FTP return 0;
}
@@ -262,7 +262,8 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
// récupérer adresse résolue
strcpybuff(back->info, "host name");
if (hts_dns_resolve2(opt, _adr, &server, &error) == NULL) {
htsblk_failf(&back->r, "Unable to get server's address: %s", error);
snprintf(back->r.msg, sizeof(back->r.msg),
"Unable to get server's address: %s", error);
back->r.statuscode = STATUSCODE_NON_FATAL;
_HALT_FTP return 0;
}
@@ -331,15 +332,18 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
}
} else {
htsblk_failf(&back->r, "Bad password: %s", linejmp(line));
snprintf(back->r.msg, sizeof(back->r.msg), "Bad password: %s",
linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
}
} else {
htsblk_failf(&back->r, "Bad user name: %s", linejmp(line));
snprintf(back->r.msg, sizeof(back->r.msg), "Bad user name: %s",
linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
}
} else {
htsblk_failf(&back->r, "Connection refused: %s", linejmp(line));
snprintf(back->r.msg, sizeof(back->r.msg), "Connection refused: %s",
linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
}
@@ -406,7 +410,8 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
}
// -- fin analyse de l'adresse IP et du port --
} else {
htsblk_failf(&back->r, "PASV incorrect: %s", linejmp(line));
snprintf(back->r.msg, sizeof(back->r.msg), "PASV incorrect: %s",
linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
} // sinon on est prêts
} else {
@@ -437,11 +442,13 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
}
}
} else {
htsblk_failf(&back->r, "EPSV incorrect: %s", linejmp(line));
snprintf(back->r.msg, sizeof(back->r.msg), "EPSV incorrect: %s",
linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
}
} else {
htsblk_failf(&back->r, "PASV/EPSV error: %s", linejmp(line));
snprintf(back->r.msg, sizeof(back->r.msg), "PASV/EPSV error: %s",
linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
} // sinon on est prêts
}
@@ -547,8 +554,8 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
deletesoc(soc_dat);
soc_dat = INVALID_SOCKET;
//
htsblk_failf(&back->r, "RETR command error: %s",
linejmp(line));
snprintf(back->r.msg, sizeof(back->r.msg),
"RETR command error: %s", linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
} // sinon on est prêts
} else {
@@ -566,12 +573,13 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
back->r.statuscode = STATUSCODE_INVALID;
} // sinon on est prêts
} else {
htsblk_failf(&back->r, "Unable to resolve IP %s: %s", adr_ip,
error);
snprintf(back->r.msg, sizeof(back->r.msg),
"Unable to resolve IP %s: %s", adr_ip, error);
back->r.statuscode = STATUSCODE_INVALID;
} // sinon on est prêts
} else {
htsblk_failf(&back->r, "PASV incorrect: %s", linejmp(line));
snprintf(back->r.msg, sizeof(back->r.msg), "PASV incorrect: %s",
linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
} // sinon on est prêts
#else
@@ -595,11 +603,13 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
back->r.statuscode = STATUSCODE_INVALID;
}
} else {
htsblk_failf(&back->r, "RETR command error: %s", linejmp(line));
snprintf(back->r.msg, sizeof(back->r.msg),
"RETR command error: %s", linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
}
} else {
htsblk_failf(&back->r, "PORT command error: %s", linejmp(line));
snprintf(back->r.msg, sizeof(back->r.msg), "PORT command error: %s",
linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
}
#ifdef _WIN32
@@ -642,7 +652,8 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
len = 0; // fin
break;
case 0:
htsblk_failf(&back->r, "Time out (%d)", timeout);
snprintf(back->r.msg, sizeof(back->r.msg), "Time out (%d)",
timeout);
back->r.statuscode = STATUSCODE_INVALID;
len = 0; // fin
break;
@@ -705,7 +716,8 @@ int run_launch_ftp(FTPDownloadStruct * pStruct) {
strcpybuff(back->r.msg, "OK");
back->r.statuscode = HTTP_OK;
} else {
htsblk_failf(&back->r, "RETR incorrect: %s", linejmp(line));
snprintf(back->r.msg, sizeof(back->r.msg), "RETR incorrect: %s",
linejmp(line));
back->r.statuscode = STATUSCODE_INVALID;
}
} else {

View File

@@ -72,8 +72,7 @@ Please visit our Website: http://www.httrack.com
HTS_UNUSED: suppress unused-symbol warnings. HTS_STATIC: an unused-safe
static. HTS_PRINTF_FUN(fmt, arg): mark a printf-like function so the
compiler type-checks the format string at argument index fmt against the
varargs starting at arg. HTS_CHECK_RESULT: the return value carries the only
error signal, so dropping it is a bug; a (void) cast does not silence it. */
varargs starting at arg. */
#ifndef HTS_UNUSED
#ifdef __GNUC__
#define HTS_UNUSED __attribute__((unused))
@@ -81,13 +80,10 @@ Please visit our Website: http://www.httrack.com
#define HTS_STATIC static __attribute__((unused))
#define HTS_PRINTF_FUN(fmt, arg) __attribute__((format(printf, fmt, arg)))
#define HTS_CHECK_RESULT __attribute__((warn_unused_result))
#else
#define HTS_UNUSED
#define HTS_STATIC static
#define HTS_PRINTF_FUN(fmt, arg)
#define HTS_CHECK_RESULT
#endif
#endif

View File

@@ -124,9 +124,7 @@ typedef struct help_wizard_buffers {
char stropt[2048]; // options
char stropt2[2048]; // options longues
char strwild[2048]; // wildcards
/* holds all four of the above plus separators: at 4096 a long answer set
clipped the filters off the command line */
char cmd[HTS_URLMAXSIZE * 2 + 3 * 2048 + 4];
char cmd[4096];
char str[256];
char *argv[256];
} help_wizard_buffers;
@@ -310,14 +308,7 @@ void help_wizard(httrackp * opt) {
printf("\n");
if (strlen(stropt) == 1)
stropt[0] = '\0'; // aucune
/* the tail is the filter list, and cmd is split into the argv handed to
hts_main() below: a clipped line would silently widen the crawl */
if (!sprintfbuff(cmd, "%s %s %s %s", urls, stropt, stropt2, strwild)) {
printf("* command line too long (%d bytes max)\n",
(int) sizeof(cmd) - 1);
freet(buffers);
return;
}
snprintf(cmd, sizeof(cmd), "%s %s %s %s", urls, stropt, stropt2, strwild);
printf("---> Wizard command line: httrack %s\n\n", cmd);
printf("Ready to launch the mirror? (Y/n) :");
fflush(stdout);

View File

@@ -2537,15 +2537,6 @@ void fil_simplifie(char *f) {
}
}
void htsblk_failf(htsblk *r, const char *fmt, ...) {
va_list args;
va_start(args, fmt);
// deliberate clip: the reason is quoted from a remote reply
(void) vslprintfbuff(r->msg, sizeof(r->msg), fmt, args);
va_end(args);
}
// fermer liaison fichier ou socket
void deletehttp(htsblk * r) {
#if HTS_DEBUG_CLOSESOCK

View File

@@ -199,10 +199,6 @@ T_SOC newhttp(httrackp * opt, const char *iadr, htsblk * retour, int port,
etc.). */
T_SOC newhttp_addr(httrackp *opt, const char *iadr, htsblk *retour, int port,
int waitconnect, int addr_index, int *addr_count);
/* Clips the formatted failure reason into r->msg, which also round-trips
through the cache as X-StatusMessage. Leaves r->statuscode to the caller. */
void htsblk_failf(htsblk *r, const char *fmt, ...) HTS_PRINTF_FUN(2, 3);
HTS_INLINE void deletehttp(htsblk * r);
HTS_INLINE int deleteaddr(htsblk * r);
HTS_INLINE void deletesoc(T_SOC soc);

View File

@@ -173,8 +173,8 @@ int http_proxy_tunnel(httrackp *opt, htsblk *retour, const char *adr,
if (sscanf(line, "HTTP/%*d.%*d %d", &code) < 1)
code = 0;
if (code < 200 || code >= 300) {
htsblk_failf(retour, "proxy CONNECT refused: %s",
strnotempty(line) ? line : "(no status)");
snprintf(retour->msg, sizeof(retour->msg), "proxy CONNECT refused: %s",
strnotempty(line) ? line : "(no status)");
return 0;
}

View File

@@ -33,7 +33,6 @@ Please visit our Website: http://www.httrack.com
#ifndef HTSSAFE_DEFH
#define HTSSAFE_DEFH
#include <stdarg.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
@@ -460,49 +459,6 @@ static HTS_INLINE HTS_UNUSED const char *htsbuff_str(const htsbuff *b) {
return b->buf;
}
/**
* Callers that deliberately ignore truncation use this instead of
* slprintfbuff(), so it is not HTS_CHECK_RESULT.
*/
static HTS_INLINE HTS_UNUSED HTS_PRINTF_FUN(3, 0) hts_boolean
vslprintfbuff(char *dest, size_t size, const char *fmt, va_list args) {
int ret;
assertf(dest != NULL && size != 0);
ret = vsnprintf(dest, size, fmt, args);
/* pre-C99 runtimes (msvcrt _vsnprintf) return -1 and do not terminate */
dest[size - 1] = '\0';
return ret >= 0 && (size_t) ret < size ? HTS_TRUE : HTS_FALSE;
}
/**
* Formatted print into dest (capacity size, NUL included), truncating to fit
* and always NUL-terminating. Returns HTS_TRUE if the whole output fit; the
* result is the only truncation signal, so it must be acted on. Unlike
* strcpybuff() it never aborts, so it suits text built from remote input.
*/
static HTS_INLINE HTS_UNUSED HTS_CHECK_RESULT HTS_PRINTF_FUN(3, 4) hts_boolean
slprintfbuff(char *dest, size_t size, const char *fmt, ...) {
va_list args;
hts_boolean ret;
va_start(args, fmt);
ret = vslprintfbuff(dest, size, fmt, args);
va_end(args);
return ret;
}
/**
* slprintfbuff() over the in-scope array ARR (capacity = sizeof(ARR)).
* On GCC/Clang a pointer is a compile error; use slprintfbuff() for those.
*/
#if (defined(__GNUC__) && !defined(__cplusplus))
#define sprintfbuff(ARR, ...) \
slprintfbuff((ARR), sizeof(ARR) + htsbuff_must_be_array_(ARR), __VA_ARGS__)
#else
#define sprintfbuff(ARR, ...) slprintfbuff((ARR), sizeof(ARR), __VA_ARGS__)
#endif
/* Thin aliases over the libc allocator/memcpy (historical "t" suffix); no
added bounds checking. freet() also NULLs the freed pointer and tolerates
NULL. memcpybuff() despite the name is a raw memcpy: the caller owns the

View File

@@ -51,7 +51,6 @@ Please visit our Website: http://www.httrack.com
#include "htscache_selftest.h"
#include "htsdns_selftest.h"
#include "htscharset.h"
#include "htscmdline.h"
#include "htsencoding.h"
#include "htsftp.h"
#include "htsmd5.h"
@@ -569,97 +568,6 @@ static int string_safety_selftests(void) {
return 1;
}
/* sprintfbuff: truncate-and-report. Must never abort (its callers format
remote banners) nor write past the array, which the canary catches. */
{
struct {
char dst[8];
char canary[8];
} s;
const char *const big = "0123456789abcdefghijklmnopqrstuvwxyz";
/* repoison before every call, or an implementation that measures first and
writes nothing "passes" the truncating cases on the previous content */
#define POISON_DST() memset(s.dst, '#', sizeof(s.dst))
memset(&s, '#', sizeof(s));
if (!sprintfbuff(s.dst, "%s-%d", "ab", 42) || strcmp(s.dst, "ab-42") != 0)
return 1;
/* exact fit: 7 characters plus the NUL */
POISON_DST();
if (!sprintfbuff(s.dst, "%s", "1234567") || strcmp(s.dst, "1234567") != 0)
return 1;
/* one over, then far over: truncated to the prefix, terminated, reported */
POISON_DST();
if (sprintfbuff(s.dst, "%s", "12345678") || strcmp(s.dst, "1234567") != 0)
return 1;
POISON_DST();
if (sprintfbuff(s.dst, "%s", big) || strcmp(s.dst, "0123456") != 0)
return 1;
/* explicit-capacity form, down to the degenerate size 1 */
{
char *const p = s.dst;
POISON_DST();
if (slprintfbuff(p, 1, "%s", "x") || p[0] != '\0')
return 1;
POISON_DST();
if (!slprintfbuff(p, sizeof(s.dst), "%s", "ok") || strcmp(p, "ok") != 0)
return 1;
}
#undef POISON_DST
if (memcmp(s.canary, "########", sizeof(s.canary)) != 0)
return 1;
}
/* htsblk_failf: clips a reason quoted from a remote reply into msg[] and
touches nothing else in the block */
{
htsblk r;
char expect[sizeof(r.msg)];
char big[4 * sizeof(r.msg)];
/* contenttype abuts msg, so a one-past-the-end store lands in it rather
than in padding; checking it is this block's canary */
#define NEIGHBOURS_INTACT() (r.contenttype[0] == '\0' && r.statuscode == 1234)
memset(&r, 0, sizeof(r));
r.statuscode = 1234;
memset(r.msg, '#', sizeof(r.msg));
htsblk_failf(&r, "PASV incorrect: %s", "220 ok");
if (strcmp(r.msg, "PASV incorrect: 220 ok") != 0 || !NEIGHBOURS_INTACT())
return 1;
/* exact fit: capacity - 1 characters plus the NUL */
memset(expect, 'y', sizeof(expect) - 1);
expect[sizeof(expect) - 1] = '\0';
memcpy(expect, "Bad password: ", sizeof("Bad password: ") - 1);
memset(r.msg, '#', sizeof(r.msg));
htsblk_failf(&r, "%s", expect);
if (strcmp(r.msg, expect) != 0 || !NEIGHBOURS_INTACT())
return 1;
/* far over: the expected bytes differ from the cases above, so writing
nothing cannot pass on the leftovers */
memset(big, 'z', sizeof(big) - 1);
big[sizeof(big) - 1] = '\0';
memset(expect, 'z', sizeof(expect) - 1);
expect[sizeof(expect) - 1] = '\0';
memcpy(expect, "Bad user name: ", sizeof("Bad user name: ") - 1);
memset(r.msg, '#', sizeof(r.msg));
htsblk_failf(&r, "Bad user name: %s", big);
if (strcmp(r.msg, expect) != 0 || !NEIGHBOURS_INTACT())
return 1;
#undef NEIGHBOURS_INTACT
}
/* StringCatN/StringSetLength must eval SIZE once: (n_eval++, V) leaves
n_eval == 2 on a double-eval macro. */
{
@@ -1218,128 +1126,6 @@ static int st_unescape_bounds(httrackp *opt, int argc, char **argv) {
return 0;
}
// hts_split_cmdline(): the vector must grow with the argument count, and a
// quote inside a value must not end the argument and hand -V to the parser.
static int st_cmdlinesplit(httrackp *opt, int argc, char **argv) {
char line[512];
char **args;
int nargs = 0;
(void) opt;
(void) argc;
(void) argv;
// control: every separator splits, and argv[0] is the program name
strcpybuff(line, "httrack http://x/ --quiet\t-c8\n-O out");
args = hts_split_cmdline(line, &nargs);
assertf(args != NULL && nargs == 6);
assertf(args[nargs] == NULL); // callers may walk to the terminator
assertf(strcmp(args[0], "httrack") == 0);
assertf(strcmp(args[1], "http://x/") == 0);
assertf(strcmp(args[2], "--quiet") == 0);
assertf(strcmp(args[3], "-c8") == 0);
assertf(strcmp(args[4], "-O") == 0);
assertf(strcmp(args[5], "out") == 0);
freet(args);
// the template pads with whitespace: empty arguments are kept (the engine
// skips them), so the count is one per separator
strcpybuff(line, "httrack --quiet");
args = hts_split_cmdline(line, &nargs);
assertf(nargs == 3 && args[1][0] == '\0');
assertf(strcmp(args[2], "--quiet") == 0);
freet(args);
// a quoted run keeps both its spaces and its quotes: the engine unquotes
strcpybuff(line, "httrack --user-agent \"Mozilla 5.0\" -c8");
args = hts_split_cmdline(line, &nargs);
assertf(nargs == 4);
assertf(strcmp(args[2], "\"Mozilla 5.0\"") == 0);
assertf(strcmp(args[3], "-c8") == 0);
freet(args);
// an escaped quote is a literal quote, not the end of the argument: the
// engine strips only the outer pair
strcpybuff(line, "httrack --user-agent \"x\\\" -V \\\"touch /tmp/pwn\" -c8");
args = hts_split_cmdline(line, &nargs);
assertf(nargs == 4);
assertf(strcmp(args[2], "\"x\" -V \"touch /tmp/pwn\"") == 0);
assertf(strcmp(args[3], "-c8") == 0);
freet(args);
// \\ is a literal backslash, so a Windows path survives
strcpybuff(line, "httrack --path \"C:\\\\dir\\\\sub\"");
args = hts_split_cmdline(line, &nargs);
assertf(nargs == 3);
assertf(strcmp(args[2], "\"C:\\dir\\sub\"") == 0);
freet(args);
// outside a quoted run a backslash is literal: the url and wildcard-filter
// fields, which the wizard cannot quote, read as before
strcpybuff(line, "httrack -*\\** +*.png");
args = hts_split_cmdline(line, &nargs);
assertf(nargs == 3);
assertf(strcmp(args[1], "-*\\**") == 0);
assertf(strcmp(args[2], "+*.png") == 0);
freet(args);
// a quoted run leaves slots unused, so the terminator has to be written and
// not inherited: size the vector from a full line first, so freeing it hands
// the same chunk back with stale pointers in those slots
strcpybuff(line, "httrack a b c d e");
args = hts_split_cmdline(line, &nargs);
assertf(nargs == 6);
freet(args);
strcpybuff(line, "httrack \"a b c d e\"");
args = hts_split_cmdline(line, &nargs);
assertf(nargs == 2);
assertf(args[nargs] == NULL);
freet(args);
// an unterminated quote protects the rest of the line, as one argument
strcpybuff(line, "httrack --footer \"unbalanced -V x");
args = hts_split_cmdline(line, &nargs);
assertf(nargs == 3);
assertf(strcmp(args[2], "\"unbalanced -V x") == 0);
freet(args);
// past the 1024 entries the vector used to hold: distinct arguments, so a
// write beyond the allocation cannot read back as the expected parse
{
const int n = 2000;
const size_t size = 16 * (size_t) n + 16;
char *big = malloct(size);
size_t pos = 0;
int i;
assertf(big != NULL);
pos = (size_t) snprintf(big, size, "httrack");
assertf(pos < size);
for (i = 0; i < n; i++) {
// snprintf returns what it wanted to write, so accumulating it blind
// would let the next size argument wrap
const int len = snprintf(big + pos, size - pos, " a%d", i);
assertf(len > 0 && (size_t) len < size - pos);
pos += (size_t) len;
}
args = hts_split_cmdline(big, &nargs);
assertf(args != NULL && nargs == n + 1);
assertf(args[nargs] == NULL);
for (i = 0; i < n; i++) {
char expect[16];
snprintf(expect, sizeof(expect), "a%d", i);
assertf(strcmp(args[i + 1], expect) == 0);
}
freet(args);
freet(big);
}
printf("cmdline-split self-test OK\n");
return 0;
}
static int st_hashtable(httrackp *opt, int argc, char **argv) {
char *snum;
unsigned long count = 0;
@@ -2976,32 +2762,6 @@ static int st_makeindex(httrackp *opt, int argc, char **argv) {
assertf(strstr(buf, "Refresh") != NULL);
assertf(strstr(buf, "example.com") != NULL);
/* a first link whose escaped form overruns the old flat 1024-byte tempo: the
redirect must carry the whole URL, not a clipped prefix */
{
char BIGSTK link[HTS_URLMAXSIZE * 2];
char *p = link;
strcpybuff(link, "http://example.com/");
p += strlen(link);
memset(p, 'a', 1200);
p += 1200;
strcpy(p, "/end.html");
done = 0;
fp = fopen(path, "wb");
assertf(fp != NULL);
hts_finish_makeindex(opt, &done, &fp, 1, link, "%s%s", "", "");
assertf(fp == NULL);
fp = fopen(path, "rb");
assertf(fp != NULL);
n = fread(buf, 1, sizeof(buf) - 1, fp);
fclose(fp);
buf[n] = '\0';
/* the closing quote proves the URL was not clipped mid-way */
assertf(strstr(buf, "/end.html\">") != NULL);
}
/* no single link: footer only, no refresh meta */
done = 0;
fp = fopen(path, "wb");
@@ -5041,10 +4801,10 @@ static int st_cookieimport(httrackp *opt, int argc, char **argv) {
char fpath[HTS_URLMAXSIZE * 2];
char file[HTS_URLMAXSIZE * 2];
assertf(sprintfbuff(fpath, "%s/", dir)); /* IE glob wants a trailing sep */
snprintf(fpath, sizeof(fpath), "%s/", dir); /* IE glob wants a trailing sep */
/* cookies.txt: one Netscape record (host, _, path, _, _, name, value). */
assertf(sprintfbuff(file, "%scookies.txt", fpath));
snprintf(file, sizeof(file), "%scookies.txt", fpath);
{
FILE *fp = FOPEN(file, "wb");
@@ -5054,7 +4814,7 @@ static int st_cookieimport(httrackp *opt, int argc, char **argv) {
}
/* A copied IE cookie u@v.txt: name, value, url, then 6 unused fields. */
assertf(sprintfbuff(file, "%su@v.txt", fpath));
snprintf(file, sizeof(file), "%su@v.txt", fpath);
{
FILE *fp = FOPEN(file, "wb");
@@ -5076,7 +4836,7 @@ static int st_cookieimport(httrackp *opt, int argc, char **argv) {
#endif
(void) UNLINK(file); /* u@v.txt (already gone on Windows) */
assertf(sprintfbuff(file, "%scookies.txt", fpath));
snprintf(file, sizeof(file), "%scookies.txt", fpath);
(void) UNLINK(file);
dir[dirlen] = '\0';
while (strlen(dir) > base) {
@@ -5150,9 +4910,6 @@ static const struct selftest_entry {
st_footerfmt},
{"unescape-bounds", "", "unescapers reserve the NUL byte (no 1-byte OOB)",
st_unescape_bounds},
{"cmdline-split", "",
"webhttrack command-line to argv split (bounds, quoting)",
st_cmdlinesplit},
{"hashtable", "<count|file>", "coucal hashtable stress test", st_hashtable},
{"strsafe", "[overflow|overflow-buff|overflow-src [str]]",
"bounded string-op self-test", st_strsafe},

View File

@@ -363,43 +363,6 @@ static hts_boolean body_sid_is_valid(const char *body, const char *expected) {
return seen;
}
/* Append c to dst as an HTML entity, or return HTS_FALSE if it needs none. */
static hts_boolean cat_html_escaped(String *dst, char c) {
switch (c) {
case '<':
StringCat(*dst, "&lt;");
break;
case '>':
StringCat(*dst, "&gt;");
break;
case '&':
StringCat(*dst, "&amp;");
break;
case '\'':
StringCat(*dst, "&#39;");
break;
default:
return HTS_FALSE;
}
return HTS_TRUE;
}
/* Append the value of a double-quoted command-line argument: escaped for HTML,
which the browser undoes when it posts the command line back, and for the
argv splitter, which does not. */
static void cat_cmdline_arg(String *output, const char *value) {
const char *a;
for (a = value; *a != '\0'; a++) {
if (*a == '\\' || *a == '\"') {
StringCat(*output, "\\");
}
if (!cat_html_escaped(output, *a)) {
StringMemcat(*output, a, 1);
}
}
}
int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
int timeout = 30;
int retour = 0;
@@ -1030,7 +993,6 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
int p;
int format = 0;
int listDefault = 0;
hts_boolean unquoted = HTS_FALSE;
name[0] = '\0';
strlncatbuff(name, str, sizeof(name_), n);
@@ -1040,12 +1002,6 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
} else if ((p = strfield(name, "html:"))) {
name += p;
format = 1;
} else if ((p = strfield(name, "unquoted:"))) {
name += p;
unquoted = HTS_TRUE;
} else if ((p = strfield(name, "arg:"))) {
name += p;
format = 5;
} else if ((p = strfield(name, "list:"))) {
name += p;
format = 2;
@@ -1182,8 +1138,8 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
test:<if ==0>:<if ==1>:<if == 2>..
ztest:<if == 0 || !exist>:<if == 1>:<if == 2>..
*/
else if ((p = strfield(name, "test:")) ||
(p = strfield(name, "ztest:"))) {
else if ((p = strfield(name, "test:"))
|| (p = strfield(name, "ztest:"))) {
intptr_t adr = 0;
char *pos2;
int ztest = (name[0] == 'z');
@@ -1286,12 +1242,6 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
}
}
/* consumed here: it shares nothing with the list and
option formats below */
if (format == 5 && langstr != NULL && outputmode != -1) {
cat_cmdline_arg(&output, langstr);
langstr = NULL;
}
if (langstr && outputmode != -1) {
switch (format) {
case 0:
@@ -1309,18 +1259,18 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
StringMemcat(output, &c, 1);
}
a += 2;
} else if (unquoted && a[0] == '\"') {
/* the browser posts an entity back as a raw
quote, which would open a quoted run in the
argv splitter; a URI cannot hold one anyway */
StringCat(output, "%22");
} else if (outputmode &&
cat_html_escaped(&output, a[0])) {
/* appended as an entity */
} else if (outputmode && a[0] == '<') {
StringCat(output, "&lt;");
} else if (outputmode && a[0] == '>') {
StringCat(output, "&gt;");
} else if (outputmode && a[0] == '&') {
StringCat(output, "&amp;");
} else if (outputmode && a[0] == '\'') {
StringCat(output, "&#39;");
} else if (outputmode == 3 && a[0] == ' ') {
StringCat(output, "%20");
} else if (outputmode >= 2 &&
((unsigned char) a[0]) < 32) {
} else if (outputmode >= 2
&& ((unsigned char) a[0]) < 32) {
char tmp[32];
sprintf(tmp, "%%%02x", (unsigned char) a[0]);
@@ -1381,10 +1331,20 @@ int smallserver(T_SOC soc, char *url, char *method, char *data, char *path) {
}
StringClear(tmpbuff);
break;
case '<':
StringCat(tmpbuff, "&lt;");
break;
case '>':
StringCat(tmpbuff, "&gt;");
break;
case '&':
StringCat(tmpbuff, "&amp;");
break;
case '\'':
StringCat(tmpbuff, "&#39;");
break;
default:
if (!cat_html_escaped(&tmpbuff, *fstr)) {
StringMemcat(tmpbuff, fstr, 1);
}
StringMemcat(tmpbuff, fstr, 1);
break;
}
fstr++;

View File

@@ -62,7 +62,6 @@ Please visit our Website: http://www.httrack.com
#include "htsmd5.c"
#include "md5.c"
#include "htscmdline.h"
#include "htsserver.h"
#include "htsurlport.h"
#include "htsweb.h"
@@ -320,8 +319,10 @@ int main(int argc, char *argv[]) {
static int webhttrack_runmain(httrackp * opt, int argc, char **argv);
static void back_launch_cmd(void *pP) {
char *cmd = (char *) pP;
char **argv;
char **argv = (char **) malloct(1024 * sizeof(char *));
int argc = 0;
int i = 0;
int g = 0;
//
httrackp *opt;
@@ -332,19 +333,28 @@ static void back_launch_cmd(void *pP) {
commandReturnCmdl = strdup(cmd);
/* split */
argv = hts_split_cmdline(cmd, &argc);
if (argv == NULL) {
if (commandReturnMsg)
free(commandReturnMsg);
commandReturnMsg = strdup("could not parse the command line");
commandReturn = -1;
commandRunning = 0;
commandEnd = 1;
free(cmd);
return;
argv[0] = strdup("webhttrack");
argv[1] = cmd;
argc++;
i = 0;
while(cmd[i]) {
if (cmd[i] == '\t' || cmd[i] == '\r' || cmd[i] == '\n') {
cmd[i] = ' ';
}
i++;
}
i = 0;
while(cmd[i]) {
if (cmd[i] == '\"')
g = !g;
if (cmd[i] == ' ') {
if (!g) {
cmd[i] = '\0';
argv[argc++] = cmd + i + 1;
}
}
i++;
}
/* drop the program name the posted command line carries */
argv[0] = strdupt("webhttrack");
/* init */
hts_init();
@@ -373,7 +383,6 @@ static void back_launch_cmd(void *pP) {
/* free */
free(cmd);
freet(argv[0]);
freet(argv);
return;
}

View File

@@ -42,7 +42,7 @@ Please visit our Website: http://www.httrack.com
typedef struct t_StatsBuffer {
char name[1024];
char file[1024];
char state[288]; // a short label plus back->info[256]
char state[256];
char url_sav[HTS_URLMAXSIZE * 2]; // pour cancel
char url_adr[HTS_URLMAXSIZE * 2];
char url_fil[HTS_URLMAXSIZE * 2];

View File

@@ -46,7 +46,6 @@ Please visit our Website: http://www.httrack.com
#include "httrack.h"
#include "htslib.h"
#include "htscharset.h" // after htslib.h: winsock2.h must precede windows.h
#include "htsbacktrace.h"
/* Static definitions */
static int fexist(const char *s);
@@ -73,6 +72,10 @@ static int linput(FILE * fp, char *s, int max);
#include <sys/ioctl.h>
#endif
#include <ctype.h>
#if (defined(__linux) && defined(HAVE_EXECINFO_H))
#include <execinfo.h>
#define USES_BACKTRACE
#endif
/* END specific definitions */
static void __cdecl htsshow_init(t_hts_callbackarg * carg);
@@ -877,6 +880,21 @@ static void sig_doback(int blind) { // mettre en backing
#undef FD_ERR
#define FD_ERR 2
static void print_backtrace(void) {
#ifdef USES_BACKTRACE
void *stack[256];
const int size = backtrace(stack, sizeof(stack)/sizeof(stack[0]));
if (size != 0) {
backtrace_symbols_fd(stack, size, FD_ERR);
}
#else
const char msg[] = "No stack trace available on this OS :(\n";
if (write(FD_ERR, msg, sizeof(msg) - 1) != sizeof(msg) - 1) {
/* sorry GCC */
}
#endif
}
static size_t print_num(char *buffer, int num) {
size_t i, j;
if (num < 0) {
@@ -910,7 +928,7 @@ static void sig_fatal(int code) {
size += print_num(&buffer[size], code);
buffer[size++] = '\n';
(void) (write(FD_ERR, buffer, size) == size);
hts_print_backtrace(FD_ERR);
print_backtrace();
(void) (write(FD_ERR, msgreport, sizeof(msgreport) - 1)
== sizeof(msgreport) - 1);
abort();
@@ -933,7 +951,6 @@ static void sig_leave(int code) {
}
static void signal_handlers(void) {
hts_backtrace_init();
#ifdef _WIN32
signal(SIGINT, sig_leave); // ^C
signal(SIGTERM, sig_finish); // kill <process>

View File

@@ -44,7 +44,7 @@ typedef struct t_StatsBuffer t_StatsBuffer;
struct t_StatsBuffer {
char name[1024];
char file[1024];
char state[288]; // a short label plus back->info[256]
char state[256];
char BIGSTK url_sav[HTS_URLMAXSIZE * 2]; // pour cancel
char BIGSTK url_adr[HTS_URLMAXSIZE * 2];
char BIGSTK url_fil[HTS_URLMAXSIZE * 2];

View File

@@ -100,7 +100,6 @@
<ItemGroup>
<ClCompile Include="httrack.c" />
<ClCompile Include="htsbacktrace.c" />
</ItemGroup>
<!-- Pulls in libhttrack.lib and builds it first. -->

View File

@@ -122,7 +122,6 @@
<ClCompile Include="htshelp.c" />
<ClCompile Include="htsindex.c" />
<ClCompile Include="htslib.c" />
<ClCompile Include="htscmdline.c" />
<ClCompile Include="htsurlport.c" />
<ClCompile Include="htsmd5.c" />
<ClCompile Include="htsmodules.c" />

View File

@@ -688,7 +688,7 @@ static PT_Element proxytrack_process_DAV_Request(PT_Indexes indexes,
PT_Element elt = PT_ElementNew();
elt->statuscode = 405;
strcpybuff(elt->msg, "Method Not Allowed");
strcpy(elt->msg, "Method Not Allowed");
return elt;
}
@@ -822,7 +822,7 @@ static PT_Element proxytrack_process_DAV_Request(PT_Indexes indexes,
elt->statuscode = 207; /* Multi-Status */
strcpy(elt->charset, "utf-8");
strcpy(elt->contenttype, "text/xml");
strcpybuff(elt->msg, "Multi-Status");
strcpy(elt->msg, "Multi-Status");
StringFree(response);
fprintf(stderr, "RESPONSE:\n%s\n", elt->adr);
@@ -879,7 +879,7 @@ static PT_Element proxytrack_process_HTTP_List(PT_Indexes indexes,
elt->statuscode = HTTP_OK;
strcpy(elt->charset, "iso-8859-1");
strcpy(elt->contenttype, "text/html");
strcpybuff(elt->msg, "OK");
strcpy(elt->msg, "OK");
StringFree(html);
return elt;
}

View File

@@ -407,7 +407,7 @@ PT_Element PT_Index_HTML_BuildRootInfo(PT_Indexes indexes) {
elt->statuscode = HTTP_OK;
strcpy(elt->charset, "iso-8859-1");
strcpy(elt->contenttype, "text/html");
strcpybuff(elt->msg, "OK");
strcpy(elt->msg, "OK");
StringFree(html);
return elt;
}
@@ -579,10 +579,9 @@ PT_Index PT_LoadCache(const char *filename) {
if (chain != NULL) {
const char *scheme = link_has_authority(chain->name) ? "" : "http://";
/* dropped rather than truncated: empty already reads as "unset" */
if (!sprintfbuff(index->slots.common.startUrl, "%s%s", scheme,
(const char *) chain->name))
index->slots.common.startUrl[0] = '\0';
snprintf(index->slots.common.startUrl,
sizeof(index->slots.common.startUrl), "%s%s", scheme,
(const char *) chain->name);
}
}
}
@@ -828,18 +827,6 @@ PT_Element PT_ElementNew(void) {
return r;
}
/* ProxyTrack's htsblk_failf(): a clipped, diagnostic-only failure reason. */
static void PT_Element_failf(PT_Element r, const char *fmt, ...)
HTS_PRINTF_FUN(2, 3);
static void PT_Element_failf(PT_Element r, const char *fmt, ...) {
va_list args;
va_start(args, fmt);
(void) vslprintfbuff(r->msg, sizeof(r->msg), fmt, args);
va_end(args);
}
PT_Element PT_ReadCache(PT_Index index, const char *url, int flags) {
if (index != NULL && SAFE_INDEX(index)) {
return _IndexFuncts[index->type].PT_ReadCache(index, url, flags);
@@ -985,12 +972,9 @@ int PT_LoadCache__New(PT_Index index_, const char *filename) {
const char *scheme =
link_has_authority(filenameIndex) ? "" : "http://";
/* dropped rather than truncated; try the next entry */
if (sprintfbuff(index->startUrl, "%s%s", scheme,
filenameIndex))
firstSeen = 1;
else
index->startUrl[0] = '\0';
firstSeen = 1;
snprintf(index->startUrl, sizeof(index->startUrl), "%s%s",
scheme, filenameIndex);
}
}
} else {
@@ -1151,9 +1135,8 @@ static PT_Element PT_ReadCache__New_u(PT_Index index_, const char *url,
snprintf(previous_save, sizeof(previous_save), "%s%s",
index->path, previous_save_ + index->fixedPath);
} else {
PT_Element_failf(
r, "Bogus fixePath prefix for %s (prefixLen=%d)",
previous_save_, (int) index->fixedPath);
snprintf(r->msg, sizeof(r->msg), "Bogus fixePath prefix for %s (prefixLen=%d)",
previous_save_, (int) index->fixedPath);
r->statuscode = STATUSCODE_INVALID;
}
} else {
@@ -1172,7 +1155,7 @@ static PT_Element PT_ReadCache__New_u(PT_Index index_, const char *url,
// Peut-on stocker le fichier directement sur disque?
if (ok) {
if (r->msg[0] == '\0') {
strcpybuff(r->msg, "Cache Read Error : Unexpected error");
strcpy(r->msg, "Cache Read Error : Unexpected error");
}
} else { // lire en mémoire
@@ -1190,27 +1173,24 @@ static PT_Element PT_ReadCache__New_u(PT_Index index_, const char *url,
int last_errno = errno;
r->statuscode = STATUSCODE_INVALID;
PT_Element_failf(r,
"Read error in cache disk data: %s",
strerror(last_errno));
sprintf(r->msg, "Read error in cache disk data: %s",
strerror(last_errno));
}
r->adr[r->size] = '\0';
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg,
"Read error (memory exhausted) from cache");
strcpy(r->msg,
"Read error (memory exhausted) from cache");
}
fclose(fp);
} else {
r->statuscode = STATUSCODE_INVALID;
PT_Element_failf(
r, "Read error (can't open '%s') from cache",
file_convert(catbuff, sizeof(catbuff),
previous_save));
snprintf(r->msg, sizeof(r->msg), "Read error (can't open '%s') from cache",
file_convert(catbuff, sizeof(catbuff), previous_save));
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cached file name is invalid");
strcpy(r->msg, "Cached file name is invalid");
}
}
} else {
@@ -1222,12 +1202,12 @@ static PT_Element PT_ReadCache__New_u(PT_Index index_, const char *url,
free(r->adr);
r->adr = NULL;
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cache Read Error : Read Data");
strcpy(r->msg, "Cache Read Error : Read Data");
} else
*(r->adr + r->size) = '\0';
} else { // erreur
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cache Memory Error");
strcpy(r->msg, "Cache Memory Error");
}
}
}
@@ -1235,21 +1215,21 @@ static PT_Element PT_ReadCache__New_u(PT_Index index_, const char *url,
} // si save==null, ne rien charger (juste en tête)
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cache Read Error : Read Header Data");
strcpy(r->msg, "Cache Read Error : Read Header Data");
}
unzCloseCurrentFile(index->zFile);
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cache Read Error : Open File");
strcpy(r->msg, "Cache Read Error : Open File");
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cache Read Error : Bad Offset");
strcpy(r->msg, "Cache Read Error : Bad Offset");
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "File Cache Entry Not Found");
strcpy(r->msg, "File Cache Entry Not Found");
}
if (r->location[0] != '\0') {
r->location = strdup(r->location);
@@ -1590,11 +1570,9 @@ static int PT_LoadCache__Old(PT_Index index_, const char *filename) {
const char *scheme =
link_has_authority(line) ? "" : "http://";
/* dropped rather than truncated; try the next entry */
if (sprintfbuff(index->startUrl, "%s%s", scheme, line))
firstSeen = 1;
else
index->startUrl[0] = '\0';
firstSeen = 1;
snprintf(index->startUrl, sizeof(index->startUrl), "%s%s",
scheme, line);
}
}
@@ -1815,9 +1793,8 @@ static PT_Element PT_ReadCache__Old_u(PT_Index index_, const char *url,
snprintf(previous_save, sizeof(previous_save), "%s%s",
index->path, previous_save_ + index->fixedPath);
} else {
PT_Element_failf(r,
"Bogus fixePath prefix for %s (prefixLen=%d)",
previous_save_, (int) index->fixedPath);
snprintf(r->msg, sizeof(r->msg), "Bogus fixePath prefix for %s (prefixLen=%d)",
previous_save_, (int) index->fixedPath);
r->statuscode = STATUSCODE_INVALID;
}
} else {
@@ -1843,18 +1820,17 @@ static PT_Element PT_ReadCache__Old_u(PT_Index index_, const char *url,
if (r->adr != NULL) {
if (r->size > 0 && fread(r->adr, 1, r->size, fp) != r->size) {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Read error in cache disk data");
strcpy(r->msg, "Read error in cache disk data");
}
r->adr[r->size] = '\0';
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg,
"Read error (memory exhausted) from cache");
strcpy(r->msg, "Read error (memory exhausted) from cache");
}
fclose(fp);
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Previous cache file not found (2)");
strcpy(r->msg, "Previous cache file not found (2)");
}
}
} else {
@@ -1866,30 +1842,30 @@ static PT_Element PT_ReadCache__Old_u(PT_Index index_, const char *url,
free(r->adr);
r->adr = NULL;
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cache Read Error : Read Data");
strcpy(r->msg, "Cache Read Error : Read Data");
} else
r->adr[r->size] = '\0';
} else { // erreur
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cache Memory Error");
strcpy(r->msg, "Cache Memory Error");
}
}
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cache Read Error : Bad Data");
strcpy(r->msg, "Cache Read Error : Bad Data");
}
} else { // erreur
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cache Read Error : Read Header");
strcpy(r->msg, "Cache Read Error : Read Header");
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cache Read Error : Seek Failed");
strcpy(r->msg, "Cache Read Error : Seek Failed");
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "File Cache Entry Not Found");
strcpy(r->msg, "File Cache Entry Not Found");
}
if (r->location[0] != '\0') {
r->location = strdup(r->location);
@@ -2270,7 +2246,7 @@ static PT_Element PT_ReadCache__Arc_u(PT_Index index_, const char *url,
fetchSize = dataLength - metaSize;
} else if (fetchSize > dataLength - metaSize) {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cache Read Error : Truncated Data");
strcpy(r->msg, "Cache Read Error : Truncated Data");
}
r->size = 0;
if (r->statuscode != STATUSCODE_INVALID) {
@@ -2283,13 +2259,12 @@ static PT_Element PT_ReadCache__Arc_u(PT_Index index_, const char *url,
int last_errno = errno;
r->statuscode = STATUSCODE_INVALID;
PT_Element_failf(r, "Read error in cache disk data: %s",
strerror(last_errno));
sprintf(r->msg, "Read error in cache disk data: %s",
strerror(last_errno));
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg,
"Read error (memory exhausted) from cache");
strcpy(r->msg, "Read error (memory exhausted) from cache");
}
}
}
@@ -2297,21 +2272,21 @@ static PT_Element PT_ReadCache__Arc_u(PT_Index index_, const char *url,
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cache Read Error : Read Header Error");
strcpy(r->msg, "Cache Read Error : Read Header Error");
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cache Read Error : Read Header Error");
strcpy(r->msg, "Cache Read Error : Read Header Error");
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "Cache Read Error : Seek Error");
strcpy(r->msg, "Cache Read Error : Seek Error");
}
} else {
r->statuscode = STATUSCODE_INVALID;
strcpybuff(r->msg, "File Cache Entry Not Found");
strcpy(r->msg, "File Cache Entry Not Found");
}
if (r->location[0] != '\0') {
r->location = strdup(r->location);

View File

@@ -97,7 +97,6 @@
<ItemGroup>
<ClCompile Include="htsserver.c" />
<ClCompile Include="htsweb.c" />
<ClCompile Include="htscmdline.c" />
<ClCompile Include="htsurlport.c" />
</ItemGroup>

View File

@@ -1,8 +0,0 @@
#!/bin/bash
#
set -euo pipefail
# webhttrack posts its command line as one string: the argv split must grow past
# 1024 arguments and keep a quote inside a value out of the option parser.
httrack -O /dev/null -#test=cmdline-split run | grep -q "cmdline-split self-test OK"

View File

@@ -1,97 +0,0 @@
#!/bin/bash
# A fatal signal prints the raw backtrace, then names the frames
# backtrace_symbols_fd() leaves as module+offset (-fvisibility=hidden hides them).
set -eu
testdir=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
if is_windows; then
echo "no backtrace() on Windows; skipping" >&2
exit 77
fi
command -v httrack >/dev/null || {
echo "could not find httrack" >&2
exit 1
}
tmpdir=$(mktemp -d "${TMPDIR:-/tmp}/httrack_symbolize.XXXXXX") || exit 1
trap 'rm -rf "$tmpdir"' EXIT HUP INT QUIT PIPE TERM
out="${tmpdir}/trace"
raw="${tmpdir}/trace-optout"
overflow="this string is far too long for the buffer"
# A frame glibc could not name. Stop at the offset: it prints the trailing
# "[0xADDR]" with or without a leading space depending on its version.
rawframe='(+0x[0-9a-f][0-9a-f]*)'
# The strsafe selftest aborts on purpose, which lands in sig_fatal.
rc=0
run_with_timeout 60 httrack -#test=strsafe overflow "$overflow" >"$out" 2>&1 || rc=$?
test "$rc" -ne 124 || {
echo "the crash handler did not finish within the deadline" >&2
exit 1
}
grep -q '^Caught signal ' "$out" || {
echo "no 'Caught signal' line:" >&2
cat "$out" >&2
exit 1
}
if grep -q 'No stack trace available on this OS' "$out"; then
echo "no backtrace() on this platform; skipping" >&2
exit 77
fi
# Unconditional and first: symbolizing must never cost the raw trace.
grep -q "$rawframe" "$out" || {
echo "raw module+offset frames are gone:" >&2
cat "$out" >&2
exit 1
}
command -v addr2line >/dev/null || {
echo "addr2line not installed; skipping" >&2
exit 77
}
# Control: resolve the same frames ourselves, so a stripped build cannot turn
# the assertion below into a vacuous pass.
oracle="${tmpdir}/oracle"
sed -n 's/^\([^(]*\)(+\(0x[0-9a-f]*\)).*$/\1 \2/p' "$out" |
while read -r mod off; do
test -f "$mod" || continue
addr2line -Cfip -e "$mod" "$off" 2>/dev/null || true
done >"$oracle"
grep -qE 'st_strsafe|strcpy_safe_' "$oracle" || {
echo "addr2line names no hidden symbol in this build; skipping" >&2
exit 77
}
# The payload. Dropping the raw frames first is what makes it specific: both
# names are static, so .dynsym could never have carried them. Not anchored on
# the "0xOFF:" line, the inline chain puts the name on either half.
grep -v "$rawframe" "$out" | grep -qE 'st_strsafe|strcpy_safe_' || {
echo "the handler did not name the hidden frames:" >&2
cat "$out" >&2
exit 1
}
# Opt-out: raw trace kept, nothing spawned.
export HTTRACK_NO_SYMBOLIZE=1
rc=0
run_with_timeout 60 httrack -#test=strsafe overflow "$overflow" >"$raw" 2>&1 || rc=$?
unset HTTRACK_NO_SYMBOLIZE
test "$rc" -ne 124 || {
echo "the opt-out run did not finish within the deadline" >&2
exit 1
}
grep -q "$rawframe" "$raw" || {
echo "the opt-out lost the raw trace:" >&2
cat "$raw" >&2
exit 1
}
! grep -q '^0x[0-9a-f]*: ' "$raw" || {
echo "the opt-out still symbolized:" >&2
cat "$raw" >&2
exit 1
}

View File

@@ -1,142 +0,0 @@
#!/bin/bash
#
# The wizard's size fields must render as distinct caps: sizemax is -M, othermax
# then maxhtml share -m.
set -euo pipefail
testdir=$(cd "$(dirname "$0")" && pwd)
distdir=${top_srcdir:-$(cd "${testdir}/.." && pwd)}
distdir=$(cd "${distdir}" && pwd)
# shellcheck source=tests/testlib.sh
. "${testdir}/testlib.sh"
fail() {
echo "FAIL: $*" >&2
exit 1
}
command -v htsserver >/dev/null || fail "no htsserver in PATH"
python=$(find_python) || {
echo "python3 not found; skipping" >&2
exit 77
}
work=$(mktemp -d "${TMPDIR:-/tmp}/webhttrack_maxsize.XXXXXX") || fail "no tmpdir"
srvlog=$(mktemp)
srv=
cleanup() {
# htsserver keeps SIGTERM ignored across its exec, so only -9 reaps it.
test -z "${srv}" || kill -9 "${srv}" 2>/dev/null || true
wait "${srv}" 2>/dev/null || true # absorb bash's async "Killed" notice
srv=
rm -rf "${work}" "${srvlog}"
}
trap cleanup EXIT HUP INT QUIT PIPE TERM
# webhttrack server on a pre-picked port; an isolated HOME keeps a stray
# ~/.httrack.ini out of it.
sport=$("${python}" -c 'import socket
s = socket.socket()
s.bind(("127.0.0.1", 0))
print(s.getsockname()[1])
s.close()')
(
trap '' TERM TTOU
export HOME="${work}"
exec htsserver "${distdir}/" --port "${sport}" >"${srvlog}" 2>&1
) &
srv=$!
for _ in $(seq 1 40); do
url=$(sed -n 's/^URL=//p' "${srvlog}") && test -n "${url}" && break
kill -0 "${srv}" 2>/dev/null || break
sleep 0.25
done
test -n "${url:-}" || fail "htsserver did not start: $(cat "${srvlog}")"
# Audit what step4.html renders back; without command_do nothing is launched.
"${python}" - "${url}" <<'PY' || fail "wizard rendered the wrong options (see above)"
import re, sys, urllib.parse, urllib.request
url = sys.argv[1]
html, other, site = "111111", "222222", "333333"
def post(fields):
# The body is refused without the session id the server puts in each form.
form = urllib.request.urlopen(url + "server/index.html", timeout=20).read()
m = re.search(rb'name="sid" value="([0-9a-f]+)"', form)
if not m:
raise SystemExit("no session id in server/index.html")
fields = [("sid", m.group(1).decode())] + fields
body = "&".join("%s=%s" % (k, urllib.parse.quote(v)) for k, v in fields)
req = urllib.request.Request(url + "server/step4.html",
data=body.encode("latin-1"), method="POST")
return urllib.request.urlopen(req, timeout=20).read().decode("latin-1")
def textarea(page, name):
m = re.search(r'<textarea name="%s".*?>(.*?)</textarea>' % name, page, re.S)
if m is None:
sys.exit("no %s textarea in the rendered step4.html" % name)
return m.group(1)
def want(ok, msg, ctx):
if not ok:
sys.exit("%s\nrendered:%s" % (msg, ctx))
page = post([("maxhtml", html), ("othermax", other), ("sizemax", site),
("dos", "2")])
cmd = textarea(page, "command")
want("--max-size=" + site in cmd, "site cap not emitted as --max-size", cmd)
want("--max-files=" + site not in cmd, "site cap still an --max-files", cmd)
# Positive controls: both per-file caps still ride -m, one option each.
want("--max-files=" + other in cmd, "non-HTML cap not emitted as --max-files",
cmd)
want("--max-files=," + html in cmd, "HTML cap not emitted as --max-files=,",
cmd)
want(cmd.count("--max-files=") == 2, "unexpected --max-files count", cmd)
want(cmd.index("--max-files=" + other) < cmd.index("--max-files=," + html),
"the bare --max-files must precede the --max-files=, form", cmd)
# winprofile.ini feeds the same three caps to the Windows GUI, one key each.
ini = textarea(page, "winprofile").replace("\r\n", "\n") # the ini is CRLF
want("\nDos=2" in ini, "Dos not written from the dos field", ini)
want("\nMaxHtml=" + html + "\n" in ini, "MaxHtml not written from maxhtml", ini)
want("\nMaxOther=" + other + "\n" in ini, "MaxOther not written from othermax",
ini)
want("\nMaxAll=" + site + "\n" in ini, "MaxAll not written from sizemax", ini)
# A bare --max-files= (an unguarded empty field) parses as -m with no digits,
# silently clearing the html cap.
page = post([("maxhtml", ""), ("othermax", ""), ("sizemax", ""), ("dos", "2")])
cmd = textarea(page, "command")
want("--max-rate=" in cmd, "empty size fields rendered no command line", cmd)
want("--max-files" not in cmd and "--max-size" not in cmd,
"empty size fields still rendered a size option", cmd)
# A mistyped ${LANG_OK] key renders the OK button's label empty.
opts = urllib.request.urlopen(url + "server/option2b.html",
timeout=20).read().decode("latin-1")
want("LANG_OK" not in opts, "unexpanded LANG_OK in option2b.html", "")
want('<input type="submit" value="OK"' in opts, "no OK label in option2b.html",
"")
PY
cleanup
# A bare -m<n> resets the html limit, so only the bare-then-comma order keeps
# both caps live. basic.html is 487 bytes, well past the 10-byte html cap.
crawl() {
bash "${distdir}/tests/local-crawl.sh" "$@"
}
crawl --errors 1 --log-found 'File too big' \
httrack 'BASEURL/simple/basic.html' '--max-files=,10'
crawl --errors 1 --log-found 'File too big' \
httrack 'BASEURL/simple/basic.html' '--max-files=500000' '--max-files=,10'
crawl --errors 0 --found 'simple/basic.html' --log-not-found 'File too big' \
httrack 'BASEURL/simple/basic.html' '--max-files=,10' '--max-files=500000'
echo "PASS"

View File

@@ -1,148 +0,0 @@
#!/bin/bash
#
# The wizard renders its httrack command line as one string, which the engine
# splits back into argv: a double quote in a field must reach the split escaped,
# or the rest of the value is parsed as fresh options (-V runs a shell command).
set -euo pipefail
testdir=$(cd "$(dirname "$0")" && pwd)
distdir=${top_srcdir:-$(cd "${testdir}/.." && pwd)}
distdir=$(cd "${distdir}" && pwd)
fail() {
echo "FAIL: $*" >&2
exit 1
}
command -v htsserver >/dev/null || fail "no htsserver in PATH"
command -v python3 >/dev/null || {
echo "python3 not found; skipping" >&2
exit 77
}
srv=
log=$(mktemp)
cleanup() {
test -z "${srv}" || kill -9 "${srv}" 2>/dev/null || true
rm -f "${log}"
}
trap cleanup EXIT HUP INT QUIT PIPE TERM
freeport() {
python3 -c 'import socket
s = socket.socket()
s.bind(("127.0.0.1", 0))
print(s.getsockname()[1])
s.close()'
}
# Echo the announced URL. Runs in a command substitution, so it is a
# subshell and cannot export the pid: the caller reads it back with srvpid.
start() {
local port url
port=$(freeport)
: >"${log}"
(
trap '' TERM TTOU
exec htsserver "${distdir}/" --port "${port}" >"${log}" 2>&1
) &
for _ in $(seq 1 40); do
url=$(sed -n 's/^URL=//p' "${log}" 2>/dev/null) && test -n "${url}" && break
sleep 0.25
done
test -n "${url:-}" || fail "htsserver did not come up: $(cat "${log}")"
echo "${url}"
}
srvpid() { sed -n 's/^PID=//p' "${log}" | head -1; }
portof() { echo "${1##*:}" | tr -d /; }
# Raw request to 127.0.0.1:$1; $2 is the body ("" for a GET of the $3 page,
# default index). Prints the reply.
request() {
python3 -c 'import socket, sys
port, body = int(sys.argv[1]), sys.argv[2]
page = sys.argv[3] if len(sys.argv) > 3 else "index"
if body:
req = ("POST / HTTP/1.0\r\nHost: 127.0.0.1\r\n"
"Content-type: application/x-www-form-urlencoded\r\n"
"Content-length: %d\r\n\r\n%s" % (len(body), body))
else:
req = ("GET /server/%s.html HTTP/1.0\r\nHost: 127.0.0.1\r\n\r\n" % page)
s = socket.create_connection(("127.0.0.1", port), 10)
s.settimeout(30)
s.sendall(req.encode())
out = b""
while True:
b = s.recv(65536)
if not b:
break
out += b
s.close()
sys.stdout.write(out.decode("latin-1"))' "$1" "$2" ${3+"$3"}
}
scrape_sid() {
request "$1" "" | sed -n 's/.*name="sid" value="\([0-9a-f]*\)".*/\1/p' | head -1
}
# urlencode the key=value pairs given as arguments
formencode() {
python3 -c 'import sys, urllib.parse
print(urllib.parse.urlencode([tuple(a.split("=", 1)) for a in sys.argv[1:]]))' "$@"
}
url=$(start)
port=$(portof "${url}")
srv=$(srvpid)
test -n "${srv}" || fail "htsserver did not report its pid"
sid=$(scrape_sid "${port}")
test "${#sid}" -eq 32 || fail "did not scrape a 32-hex sid from the page (got '${sid}')"
# Fill the wizard fields the command line quotes, then read back the generated
# command line: the user-agent carries a break-out attempt, the footer a
# backslash (which must survive the escape round trip), the project name a plain
# value.
body=$(formencode "sid=${sid}" 'user=Moz" -V "touch /tmp/pwn' 'footer=a\b"c' \
"path=/tmp/p" "projname=plain proj" 'urls=http://x/a"b' 'url2=+*.png"')
request "${port}" "${body}" >/dev/null
cmdline=$(request "${port}" "" step4 |
sed -n '/<textarea name="command"/,/<\/textarea>/p')
# Control: without the fields in the page every assertion below is vacuous.
grep -q -- '--user-agent' <<<"${cmdline}" ||
fail "no --user-agent in the generated command line (probe blind)"
# A plain value is passed through untouched: the escaping must not mangle the
# ordinary case.
grep -qF -- '--path "/tmp/p/plain proj"' <<<"${cmdline}" ||
fail "a plain quoted value was not passed through: ${cmdline}"
# The quote is escaped, so the split keeps it inside the value...
grep -qF -- '--user-agent "Moz\" -V \"touch /tmp/pwn"' <<<"${cmdline}" ||
fail "the quote in the user-agent was not escaped: ${cmdline}"
# ...and the raw form, which would end the argument and hand -V to the option
# parser, is gone.
grep -qF -- '--user-agent "Moz" -V "touch' <<<"${cmdline}" &&
fail "the user-agent still closes its argument early: ${cmdline}"
# A backslash is escaped too, or the split would eat it along with the quote
# that follows.
grep -qF -- '--footer "a\\b\"c"' <<<"${cmdline}" ||
fail "the backslash in the footer was not escaped: ${cmdline}"
# The url and filter fields sit outside quotes, where a backslash cannot escape
# anything: one quote there flips the parity of every quote after it, so the
# escaping above would protect nothing. They must not emit a raw quote at all.
grep -qF -- 'http://x/a%22b' <<<"${cmdline}" ||
fail "the quote in the url field was not neutralised: ${cmdline}"
grep -qF -- '+*.png%22' <<<"${cmdline}" ||
fail "the quote in the filter field was not neutralised: ${cmdline}"
grep -qF -- 'http://x/a"b' <<<"${cmdline}" &&
fail "the url field still emits a raw quote: ${cmdline}"
echo "PASS"

View File

@@ -32,7 +32,6 @@ TESTS = \
00_runnable.test \
01_engine-charset.test \
01_engine-cmdline.test \
01_engine-cmdline-split.test \
01_engine-cookies.test \
01_engine-copyopt.test \
01_engine-crange.test \
@@ -176,9 +175,6 @@ TESTS = \
76_cli-resize.test \
77_webhttrack-redirect.test \
78_webhttrack-sid.test \
79_local-proxytrack-webdav-mime.test \
80_engine-crash-symbolize.test \
81_webhttrack-maxsize.test \
83_webhttrack-argescape.test
79_local-proxytrack-webdav-mime.test
CLEANFILES = check-network_sh.cache