Compare commits

..

2 Commits

Author SHA1 Message Date
Xavier Roche
84701fe609 tests: international crawl tracked the double-encoded behavior
The default.html sub-crawl asserted the pre-fix output: two mojibake
café*.html filenames from double-encoded fetches and 4 errors where
the fix leaves 2 (the caf%e9.html Latin-1 escapes the server really
lacks). Update to the corrected names and counts; the other three
sub-crawls are unchanged. Only runs where ut.httrack.com is reachable
(in-tree builds), which is why just the deb CI job caught it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-10 20:18:14 +02:00
Xavier Roche
c4e8c94b92 Fix double-encoded requests for raw UTF-8 links (#180)
Links holding raw non-ASCII bytes were always converted from the page
charset to UTF-8, even when already UTF-8: with the iso-8859-1 fallback
(reached whenever the charset was declared via HTML5 <meta charset=..>,
which the meta scanner predates), each UTF-8 byte was re-encoded and the
mirror requested %c3%a7%c2%bb... instead of %e7%bb..., saving a 404.

Gate the conversion on a strict (RFC 3629) hts_isStringUTF8, and rewrite
hts_getCharsetFromMeta as a size-bounded attribute scanner that handles
both the HTML5 and http-equiv forms in any attribute order, first tag
wins. Drop the dormant, broken hts_getCharsetFromContentType.

Covered by -#test=metacharset/-#test=isutf8 engine self-tests, an
11-variant local-server crawl matrix with an update pass
(41_local-utf8-link.test), and new fuzz corpus seeds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-10 20:07:10 +02:00
13 changed files with 86 additions and 255 deletions

View File

@@ -1,6 +1,6 @@
AC_PREREQ([2.71])
AC_INIT([httrack], [3.49.12], [roche+packaging@httrack.com], [httrack], [http://www.httrack.com/])
AC_INIT([httrack], [3.49.11], [roche+packaging@httrack.com], [httrack], [http://www.httrack.com/])
AC_COPYRIGHT([
HTTrack Website Copier, Offline Browser for Windows and Unix
Copyright (C) 1998-2015 Xavier Roche and other contributors
@@ -29,11 +29,11 @@ AC_CONFIG_SRCDIR(src/httrack.c)
AC_CONFIG_MACRO_DIR([m4])
AC_CONFIG_HEADERS(config.h)
AM_INIT_AUTOMAKE([subdir-objects])
# 3:4:0: 3.49.12 tail-appends one field (why_url) to httrackp and otherwise only
# deletes dead comments; no struct layout or exported signature broke vs 3.49.11,
# so it stays soname .so.3; bump revision.
# 3:3:0: 3.49.11 only adds enum values, macros and inline helpers to the
# installed headers (no struct layout or exported signature changed vs
# 3.49.10), so it stays soname .so.3; bump revision.
# (3:0:0 was the htsblk mime-buffer widening, the ABI break that moved .so.2 -> .so.3.)
VERSION_INFO="3:4:0"
VERSION_INFO="3:3:0"
AM_MAINTAINER_MODE
AC_USE_SYSTEM_EXTENSIONS

10
debian/changelog vendored
View File

@@ -1,13 +1,3 @@
httrack (3.49.12-1) unstable; urgency=medium
* New upstream release: security and crawl-correctness fixes (remote stack
overflow in Content-Type/-Encoding parsing, fuzzer-found parser over-reads,
world-readable cookies.txt, filter-pattern denial of service) plus a new
--why filter diagnostic; full list in history.txt.
* Build with _FORTIFY_SOURCE and -fstack-protector-strong.
-- Xavier Roche <xavier@debian.org> Fri, 10 Jul 2026 21:11:09 +0200
httrack (3.49.11-1) unstable; urgency=medium
* New upstream release: crawl correctness and security fixes (network-facing

View File

@@ -4,23 +4,6 @@ HTTrack Website Copier release history:
This file lists all changes and fixes that have been made for HTTrack
3.49-12
+ New: --why explains which filter rule accepts or rejects a given URL, then exits (#505)
+ Fixed: links carrying raw UTF-8 bytes were fetched double-encoded and 404'd (#516)
+ Fixed: an uncompressed body mislabeled as gzip no longer loses the page (#515)
+ Fixed: remote stack overflow and uninitialized read in Content-Type/-Encoding parsing (#506)
+ Fixed: several over-reads and a leak in the filter, URL and IDNA parsers (#499)
+ Fixed: bound the cache-index (.ndx) parser to its buffer (#507)
+ Fixed: catastrophic backtracking on '*'-heavy filter patterns (#513)
+ Fixed: cookies.txt was created world-readable (#511)
+ Fixed: a single corrupt cache entry no longer aborts the whole mirror (#494)
+ Fixed: cache-reconcile policy was broken for zip caches (#491, #493, #495)
+ Fixed: cancelling a crawl mid type-check no longer orphans .delayed placeholders (#496)
+ Fixed: detect URLs after the first inline script and in mid-tag attributes (#497)
+ Changed: build with _FORTIFY_SOURCE and -fstack-protector-strong (#504)
+ Changed: removed the pre-3.31 (.dat/.ndx) cache import (#512)
+ Changed: multiple internal hardening and build improvements (libFuzzer harnesses, CodeQL, dead-code removal)
3.49-11
+ New: parse robots.txt Allow rules and path wildcards per RFC 9309 (#452)
+ New: advertise deflate in Accept-Encoding and decode deflate responses (#450)

View File

@@ -68,15 +68,6 @@ static int slot_can_be_cached_on_disk(const lien_back * back);
static int slot_can_be_cleaned(const lien_back * back);
static int slot_can_be_finalized(httrackp * opt, const lien_back * back);
/* Which hard quota, if any, is currently aborting the mirror. */
typedef enum {
HTS_MIRROR_LIMIT_NONE = 0,
HTS_MIRROR_LIMIT_SIZE,
HTS_MIRROR_LIMIT_TIME,
} hts_mirror_limit;
static hts_mirror_limit back_mirror_limit(httrackp *opt);
struct_back *back_new(httrackp *opt, int back_max) {
int i;
struct_back *sback = calloct(1, sizeof(struct_back));
@@ -2059,9 +2050,13 @@ int back_add(struct_back * sback, httrackp * opt, cache_back * cache, const char
if (!back_trylive(opt, cache, sback, p)) {
#if HTS_XGETHOST
back[p].status = STATUS_WAIT_DNS; // host name resolution attempt
soc = INVALID_SOCKET; // not opened yet
if (host_wait(opt, &back[p])) { // ready (file, or cached dns)
#if HDEBUG
printf("back_solve..\n");
#endif
back[p].status = STATUS_WAIT_DNS; // tentative de résolution du nom de host
soc = INVALID_SOCKET; // pas encore ouverte
back_solve(opt, &back[p]); // préparer
if (host_wait(opt, &back[p])) { // prêt, par ex fichier ou dispo dans dns
#if HDEBUG
printf("ok, dns cache ready..\n");
#endif
@@ -2204,9 +2199,37 @@ int back_add(struct_back * sback, httrackp * opt, cache_back * cache, const char
}
#if HTS_XGETHOST
// Resolution is synchronous inside the connect path; no pre-resolve step, so
// the host is always immediately ready.
int host_wait(httrackp *opt, lien_back *back) { return 1; }
// attendre que le host (ou celui du proxy) ait été résolu
// si c'est un fichier, la résolution est immédiate
// idem pour ftp://
void back_solve(httrackp * opt, lien_back * back) {
assertf(opt != NULL);
assertf(back != NULL);
if ((!strfield(back->url_adr, "file://"))
&& !strfield(back->url_adr, "ftp://")
) {
const char *a;
if (!(back->r.req.proxy.active))
a = back->url_adr;
else
a = back->r.req.proxy.name;
assertf(a != NULL);
a = jump_protocol_const(a);
if (check_hostname_dns(a)) {
hts_log_print(opt, LOG_DEBUG, "resolved: %s", a);
} else {
hts_log_print(opt, LOG_DEBUG, "failed to resolve: %s", a);
}
}
}
// détermine si le host a pu être résolu
int host_wait(httrackp * opt, lien_back * back) {
// Always synchronous. No more background DNS resolution
// (does not really improve performances)
return 1;
}
#endif
// élimine les fichiers non html en backing (anticipation)
@@ -2393,15 +2416,9 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
back_clean(opt, cache, sback);
#endif
/* Time/size limit exceeded past grace: abort in-flight transfers so no wait
loop starves (#481, #77). FTP slots stay, their thread owns the socket. */
/* Time limit exceeded past grace: abort in-flight transfers so no wait loop
starves (#481). FTP slots stay, their thread owns the socket. */
if (!back_checkmirror(opt)) {
const hts_mirror_limit limit = back_mirror_limit(opt);
const char *const reason =
(limit == HTS_MIRROR_LIMIT_SIZE) ? "size limit" : "time limit";
const char *const slotmsg = (limit == HTS_MIRROR_LIMIT_SIZE)
? "Mirror Size Limit"
: "Mirror Time Out";
int aborted = 0;
unsigned int i;
@@ -2415,15 +2432,15 @@ void back_wait(struct_back * sback, httrackp * opt, cache_back * cache,
if (back[i].r.is_write && IS_DELAYED_EXT(back[i].url_sav))
back_delayed_discard(opt, &back[i]);
back[i].r.statuscode = STATUSCODE_TIMEOUT;
strcpybuff(back[i].r.msg, slotmsg);
strcpybuff(back[i].r.msg, "Mirror Time Out");
back[i].status = STATUS_READY;
back_set_finished(sback, i);
aborted++;
}
}
if (aborted > 0)
hts_log_print(opt, LOG_WARNING, "%s reached, %d transfer(s) aborted",
reason, aborted);
hts_log_print(opt, LOG_WARNING,
"time limit reached, %d transfer(s) aborted", aborted);
return;
}
@@ -4128,43 +4145,38 @@ static int back_maxtime_grace(const int maxtime) {
return maximum(5, minimum(30, maxtime / 10));
}
/* Bytes the smooth stop may overrun before in-flight transfers are aborted.
No floor (unlike the time grace): a size overrun should abort promptly. */
static LLint back_maxsize_grace(const LLint maxsite) { return maxsite / 10; }
/* Which cap has overrun its grace and must hard-stop the mirror (#77, #481). */
static hts_mirror_limit back_mirror_limit(httrackp *opt) {
if (opt->maxsite > 0 && HTS_STAT.stat_bytes >= opt->maxsite &&
HTS_STAT.stat_bytes - opt->maxsite >= back_maxsize_grace(opt->maxsite))
return HTS_MIRROR_LIMIT_SIZE;
int back_checkmirror(httrackp * opt) {
// Check max size
if ((opt->maxsite > 0) && (HTS_STAT.stat_bytes >= opt->maxsite)) {
if (!opt->state.stop) { /* not yet stopped */
hts_log_print(opt, LOG_ERROR,
"More than " LLintP
" bytes have been transferred.. giving up",
(LLint) opt->maxsite);
/* cancel mirror smoothly */
hts_request_stop(opt, 0);
}
return 1; /* don'k break mirror too sharply for size limits, but stop requested */
/*return 0;
*/
}
// Check max time
if (opt->maxtime > 0) {
const TStamp elapsed = time_local() - HTS_STAT.stat_timestart;
if (elapsed >= opt->maxtime &&
elapsed - opt->maxtime >= back_maxtime_grace(opt->maxtime))
return HTS_MIRROR_LIMIT_TIME;
if (elapsed >= opt->maxtime) {
if (!opt->state.stop) { /* not yet stopped */
hts_log_print(opt, LOG_ERROR, "More than %d seconds passed.. giving up",
opt->maxtime);
/* cancel mirror smoothly */
hts_request_stop(opt, 0);
}
/* smooth stop starved past the grace period: stop waiting (#481) */
if (elapsed - opt->maxtime >= back_maxtime_grace(opt->maxtime))
return 0;
}
}
return HTS_MIRROR_LIMIT_NONE;
}
int back_checkmirror(httrackp *opt) {
/* request a smooth stop the first time each cap is reached */
if (opt->maxsite > 0 && HTS_STAT.stat_bytes >= opt->maxsite &&
!opt->state.stop) {
hts_log_print(opt, LOG_ERROR,
"More than " LLintP
" bytes have been transferred.. giving up",
(LLint) opt->maxsite);
hts_request_stop(opt, 0);
}
if (opt->maxtime > 0 && !opt->state.stop &&
(time_local() - HTS_STAT.stat_timestart) >= opt->maxtime) {
hts_log_print(opt, LOG_ERROR, "More than %d seconds passed.. giving up",
opt->maxtime);
hts_request_stop(opt, 0);
}
/* hard stop once a cap overruns its grace (callers must stop waiting) */
return back_mirror_limit(opt) == HTS_MIRROR_LIMIT_NONE;
return 1; /* Ok, go on */
}
// octets transférés + add

View File

@@ -138,11 +138,12 @@ LLint back_transferred(LLint add, struct_back * sback);
// hostback
#if HTS_XGETHOST
void back_solve(httrackp * opt, lien_back * sback);
int host_wait(httrackp * opt, lien_back * sback);
#endif
int back_checksize(httrackp * opt, lien_back * eback, int check_only_totalsize);
/* Enforce -M/-E quotas: smooth-stops when reached; returns 0 once the -M cap
or -E deadline overruns its grace period (callers must stop waiting). */
/* Enforce -M/-E quotas: requests a smooth stop when reached; returns 0 once
the -E deadline overran its grace period (callers must stop waiting). */
int back_checkmirror(httrackp * opt);
#endif

View File

@@ -297,26 +297,6 @@ int dns_selftests(httrackp *opt) {
IPV6_resolver = 0;
}
/* "host:port" resolves as the bare host: the cache/connect path strips
":port" before both the backend and the cache key (#181). */
mock_reset_calls();
{
SOCaddr a;
char ip[64];
const char *err = NULL;
/* cache miss: the backend is hit with the stripped host, not "host:port" */
CHECK(hts_dns_resolve2(opt, "v6only.test:8080", &a, &err) != NULL);
CHECK(mock_find("v6only.test")->calls == 1);
/* the bare host shares that one cache entry (stripped key) */
CHECK(hts_dns_resolve2(opt, "v6only.test", &a, &err) != NULL);
CHECK(mock_find("v6only.test")->calls == 1);
/* a port variant of an already-cached host also hits, right address */
CHECK(hts_dns_resolve2(opt, "v4only.test:1234", &a, &err) != NULL);
SOCaddr_inetntoa(ip, sizeof(ip), a);
CHECK(strcmp(ip, "1.2.3.4") == 0);
}
/* newhttp_addr() must connect to the addr_index-th address, not always the
first: this is what back_connect_next relies on to reach the fallback. */
{

View File

@@ -43,8 +43,8 @@ Please visit our Website: http://www.httrack.com
configure.ac, decoupled from these). VERSION is the display form, VERSIONID
the dotted numeric form, AFF_VERSION the short form shown in footers,
LIB_VERSION the data/cache format generation. */
#define HTTRACK_VERSION "3.49-12"
#define HTTRACK_VERSIONID "3.49.12"
#define HTTRACK_VERSION "3.49-11"
#define HTTRACK_VERSIONID "3.49.11"
#define HTTRACK_AFF_VERSION "3.x"
#define HTTRACK_LIB_VERSION "2.0"

View File

@@ -2147,19 +2147,6 @@ static int ae_write_collision(const char *path, const unsigned char *src,
return ok ? 0 : 1;
}
/* Write src[0..len) to path as-is; 0 on success. */
static int ae_write_raw(const char *path, const unsigned char *src,
size_t len) {
FILE *const f = FOPEN(path, "wb");
int ok;
if (f == NULL)
return 1;
ok = fwrite(src, 1, len, f) == len;
fclose(f);
return ok ? 0 : 1;
}
/* Compare path's bytes to expect[0..len); 0 if equal. Streams (large files). */
static int ae_check_decoded(const char *path, const unsigned char *expect,
size_t len) {
@@ -2227,46 +2214,6 @@ static int st_acceptencoding(httrackp *opt, int argc, char **argv) {
assertf(ae_write_collision(inpath, body, 64) == 0);
assertf(hts_zunpack(inpath, outpath) == 64);
assertf(ae_check_decoded(outpath, body, 64) == 0);
/* Identity fallback (#47): a plain body mislabeled as compressed is kept
verbatim, small and multi-chunk (> one 8 KiB fread). */
assertf(ae_write_raw(inpath, small, slen) == 0);
assertf(hts_zunpack(inpath, outpath) == (int) slen);
assertf(ae_check_decoded(outpath, small, slen) == 0);
{
const size_t ilen = 16 * 1024;
unsigned char *idbody = malloct(ilen);
assertf(idbody != NULL);
for (i = 0; i < ilen; i++)
idbody[i] = small[i % slen];
assertf(ae_write_raw(inpath, idbody, ilen) == 0);
assertf(hts_zunpack(inpath, outpath) == (int) ilen);
assertf(ae_check_decoded(outpath, idbody, ilen) == 0);
freet(idbody);
}
/* Truncated gzip (CRC+ISIZE cut), zlib (ADLER32 cut) and raw deflate
must all still fail, not fall back to a verbatim copy. */
{
static const struct {
int wb;
size_t cut;
} tr[] = {{16 + MAX_WBITS, 8}, {MAX_WBITS, 4}, {-MAX_WBITS, 5}};
for (i = 0; i < sizeof(tr) / sizeof(tr[0]); i++) {
unsigned char z[512];
size_t zlen;
FILE *f;
assertf(ae_write_packed(inpath, tr[i].wb, small, slen) == 0);
f = FOPEN(inpath, "rb");
assertf(f != NULL);
zlen = fread(z, 1, sizeof(z), f);
fclose(f);
assertf(zlen > tr[i].cut && zlen < sizeof(z));
assertf(ae_write_raw(inpath, z, zlen - tr[i].cut) == 0);
assertf(hts_zunpack(inpath, outpath) < 0);
}
}
freet(body);
}
#else

View File

@@ -48,7 +48,6 @@ Please visit our Website: http://www.httrack.com
/*
Unpack file into a new file (gzip, zlib RFC1950 or raw deflate RFC1951).
A body provably in no deflate framing is copied verbatim (identity).
Return value: size of the new file, or -1 if an error occurred
*/
/* Note: utf-8 */
@@ -69,10 +68,6 @@ int hts_zunpack(char *filename, char *newfile) {
((inbuf[0] & 0x0f) == Z_DEFLATED &&
(((unsigned) inbuf[0] << 8 | inbuf[1]) % 31) == 0)));
int attempt;
/* not_deflate: the raw attempt hit a data error, so the body is in no
deflate framing at all. env_error: local I/O or memory failure. */
hts_boolean not_deflate = HTS_FALSE;
hts_boolean env_error = HTS_FALSE;
/* deflate is ambiguous; on failure retry with the other windowBits */
for (attempt = 0; attempt < 2 && ret < 0; attempt++) {
@@ -83,20 +78,15 @@ int hts_zunpack(char *filename, char *newfile) {
if (attempt > 0) {
/* rewind input; reopening fpout "wb" discards the partial output */
if (fseek(in, 0, SEEK_SET) != 0) {
env_error = HTS_TRUE;
if (fseek(in, 0, SEEK_SET) != 0)
break;
}
navail = fread(inbuf, 1, sizeof(inbuf), in);
}
fpout = FOPEN(fconv(catbuff, sizeof(catbuff), newfile), "wb");
if (fpout == NULL) {
env_error = HTS_TRUE;
if (fpout == NULL)
break;
}
memset(&strm, 0, sizeof(strm));
if (inflateInit2(&strm, windowBits) != Z_OK) {
env_error = HTS_TRUE;
fclose(fpout);
break;
}
@@ -118,17 +108,12 @@ int hts_zunpack(char *filename, char *newfile) {
zerr = inflate(&strm, Z_NO_FLUSH);
if (zerr == Z_NEED_DICT || zerr == Z_DATA_ERROR ||
zerr == Z_MEM_ERROR || zerr == Z_STREAM_ERROR) {
if (zerr == Z_MEM_ERROR || zerr == Z_STREAM_ERROR)
env_error = HTS_TRUE;
else if (windowBits < 0)
not_deflate = HTS_TRUE;
ok = HTS_FALSE;
break;
}
produced = sizeof(outbuf) - strm.avail_out;
if (produced > 0 &&
fwrite(outbuf, 1, produced, fpout) != produced) {
env_error = HTS_TRUE;
ok = HTS_FALSE;
break;
}
@@ -144,28 +129,6 @@ int hts_zunpack(char *filename, char *newfile) {
inflateEnd(&strm);
fclose(fpout);
}
/* keep a mislabeled identity body verbatim only when provably not
deflate; truncation or a local failure must keep failing (#47) */
if (ret < 0 && !wrapped && not_deflate && !env_error && !ferror(in)) {
FILE *const fpout =
FOPEN(fconv(catbuff, sizeof(catbuff), newfile), "wb");
if (fpout != NULL && fseek(in, 0, SEEK_SET) == 0) {
int size = 0;
while ((navail = fread(inbuf, 1, sizeof(inbuf), in)) > 0) {
if (fwrite(inbuf, 1, navail, fpout) != navail) {
size = -1;
break;
}
size += (int) navail;
}
if (size >= 0 && !ferror(in))
ret = size;
}
if (fpout != NULL)
fclose(fpout);
}
fclose(in);
}
}

View File

@@ -1,7 +1,7 @@
#!/bin/bash
#
# Diverse seeded /big/ crawl: 12 pattern families, decoy absence, update pass
# must 304-revalidate. 361 = 1 index + 96 pages + 192 imgs + 5 shared + 61
# must 304-revalidate. 360 = 1 index + 96 pages + 192 imgs + 5 shared + 60
# family + 6 singles; the 4 planted errors write -o1 pages, not counted.
set -euo pipefail
@@ -9,7 +9,7 @@ set -euo pipefail
: "${top_srcdir:=..}"
bash "$top_srcdir/tests/local-crawl.sh" --rerun \
--errors 4 --files 361 \
--errors 4 --files 360 \
--found 'big/p/95.html' \
--found 'big/a/d1/d2/d3/d4/d5/d6/d7/d8/deep.png' \
--found 'big/a/f2-2x.png' \
@@ -40,7 +40,6 @@ bash "$top_srcdir/tests/local-crawl.sh" --rerun \
--not-found 'big/x/bj.jar' \
--not-found 'big/x/is1.png' \
--not-found 'big/x/concat.html' \
--file-matches 'big/f1/gzid.html' '<p>gzid</p>' \
--file-matches 'big/p/2.html' 'srcset="\.\./a/f2-1x\.png 1x, \.\./a/f2-2x\.png 2x"' \
--file-matches 'big/a/blk2.css' 'url\(blk2-bg\.png\)' \
--file-matches 'big/p/5.html' "document\\.write\\('<a href=\"\\.\\./f5/dw\\.html\"" \

View File

@@ -1,21 +0,0 @@
#!/bin/bash
#
# -M byte cap under a slow server (#77): p0 overruns -M at once while p1..p3
# trickle for a minute; the cap must abort the in-flight transfers, not wait
# them out. Unfixed, the smooth stop drains the trickle at server pace.
set -euo pipefail
: "${top_srcdir:=..}"
start=$(date +%s)
bash "$top_srcdir/tests/local-crawl.sh" \
--log-found 'More than 400000 bytes have been transferred.. giving up' \
httrack 'BASEURL/bigtrickle/index.html' -M400000 -c4
wall=$(($(date +%s) - start))
# trickle runs 60s; a smooth-only stop waits it out, the hard stop lands near 8s.
# Wall clock is the discriminating check: the log line above fires either way.
if [ "$wall" -ge 30 ]; then
echo "crawl took ${wall}s, -M hard stop did not engage" >&2
exit 1
fi

View File

@@ -110,7 +110,6 @@ TESTS = \
38_local-update-304.test \
39_local-delayed-cancel.test \
40_local-why.test \
41_local-utf8-link.test \
42_local-maxsize-slow.test
41_local-utf8-link.test
CLEANFILES = check-network_sh.cache

View File

@@ -374,7 +374,6 @@ def big_index(port):
'<img src="/big/a/d1/d2/d3/d4/d5/d6/d7/d8/deep.png">'
'<a href="/big/f1/long.html?x=%s">long</a>'
'<a href="/big/f1/gzok.html">gzok</a>'
'<a href="/big/f1/gzid.html">gzid</a>'
'<a href="//127.0.0.1:%d/big/f1/protorel.html">protorel</a>'
'<a href="http://127.0.0.1:%d/big/f1/abshost.html">abshost</a>'
'<a href="/big/e/404.html">e404</a>'
@@ -400,7 +399,6 @@ BIG_SIMPLE_PAGES = {
"/big/f1/dir/": "dir index",
"/big/f1/long.html": "long",
"/big/f1/gzok.html": "gzok",
"/big/f1/gzid.html": "gzid",
"/big/f1/protorel.html": "protorel",
"/big/f1/abshost.html": "abshost",
"/big/f5/dw.html": "dw target",
@@ -1035,14 +1033,6 @@ class Handler(SimpleHTTPRequestHandler):
def route_bigfile(self):
self.send_raw(b"x" * self.BIGFILE_BYTES, "application/octet-stream")
# -M under a slow server (#77): p0 is a fast 640KB file that alone overruns
# -M; p1..p3 trickle for a minute. The cap must abort those in-flight
# transfers, not wait them out.
def route_bigtrickle_index(self):
self.send_html(
"".join('\t<a href="p%d.bin">p%d</a>\n' % (i, i) for i in range(4))
)
ROUTES = {
"/cookies/entrance.php": route_entrance,
"/cookies/second.php": route_second,
@@ -1115,11 +1105,6 @@ class Handler(SimpleHTTPRequestHandler):
"/bigfiles/p5.bin": route_bigfile,
"/bigfiles/p6.bin": route_bigfile,
"/bigfiles/p7.bin": route_bigfile,
"/bigtrickle/index.html": route_bigtrickle_index,
"/bigtrickle/p0.bin": route_bigfile,
"/bigtrickle/p1.bin": route_trickle_page,
"/bigtrickle/p2.bin": route_trickle_page,
"/bigtrickle/p3.bin": route_trickle_page,
"/delayed/noloc.php": route_delayed_noloc,
"/delayed/selfloop.php": route_delayed_selfloop,
"/delayed/redir.php": route_delayed_redir,
@@ -1185,13 +1170,6 @@ class Handler(SimpleHTTPRequestHandler):
"text/html",
extra=[("Content-Encoding", "gzip")],
)
elif path == "/big/f1/gzid.html":
# Plain body mislabeled as gzip: identity fallback keeps it (#47)
self.big_send(
body,
"text/html",
extra=[("Content-Encoding", "gzip")],
)
else:
self.big_send(body, "text/html")
elif path == "/big/f1/list.html":