mirror of
https://github.com/xroche/httrack.git
synced 2026-07-23 17:19:17 +03:00
9ec5a3a067
* Decode the br and zstd content codings, and bound every decode httrack advertised gzip and deflate only, so it took the oldest coding on offer while every browser negotiates brotli (46% of CDN traffic) or zstd. Worse, the response side had no codec identity at all: Content-Encoding was collapsed into a boolean and hts_zunpack guessed the framing from the body bytes, so a br or zstd body sent unsolicited (mis-keyed Vary caches do this) fell through the identity path and was saved as the page, coded bytes and all. Content-Encoding now maps to a codec (htscodec.c), and the decode dispatches on it: brotli and zstd get streaming decoders, a known coding we cannot undo fails the fetch instead of saving garbage, and an unrecognized token still means identity, because servers do put charsets in that header. br and zstd are advertised over TLS only, as browsers do. On any decompression failure the undecoded body is now dropped rather than left in memory for the writer to commit as the page. That closes the coded-bytes-as- page hole for the new unsupported-coding path, and with it a latent leak on the gzip path, where a body that failed to inflate (a truncated stream) was written to disk verbatim. Every decode is now bounded: 4096x the coded body, floor 1 MiB, ceiling INT_MAX. Nothing capped the decoded size before, which was survivable while deflate was the only codec (it cannot pass 1032x) but not with brotli and zstd, which reach a million to one. The zlib accumulator also moves to LLint, so a body over 2 GiB fails instead of overflowing an int. libbrotlidec and libzstd are optional at configure time and required on Windows, where libhttrack.dll links them from vcpkg and CI runs the self-tests to prove the decoders are in. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Fix copyright year on the new codec files (2026, not 1998) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * tests: probe the direct-to-disk decode-failure discard, and assert absence The wire test only exercised the in-memory discard path (bad.html is HTML). Add a non-HTML body under an unsupported coding (bin.dat) so the is_write branch is covered too, and assert both are absent from the mirror rather than grepping a file that the fix now removes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Factor the budget-enforced write shared by the brotli and zstd decoders The two streaming decoders duplicated the decoded-size budget check, the security-critical part of each loop. Pull it into one codec_sink helper so the bomb ceiling lives in a single place. Behavior-preserving; the codec self-test (exact decoded bytes, truncation and bomb rejection) proves equivalence. Also compile the streaming-decode body only when a streaming codec is built, which drops an unused-variable warning in the --without-brotli --without-zstd build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> --------- Signed-off-by: Xavier Roche <roche@httrack.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
102 lines
4.0 KiB
Plaintext
102 lines
4.0 KiB
Plaintext
Source: httrack
|
|
Section: web
|
|
Maintainer: Xavier Roche <roche@httrack.com>
|
|
Standards-Version: 4.7.4
|
|
Build-Depends: debhelper-compat (= 14), autoconf, autoconf-archive, automake, libtool, zlib1g-dev, libssl-dev, libbrotli-dev, libzstd-dev
|
|
Rules-Requires-Root: no
|
|
Homepage: http://www.httrack.com
|
|
Vcs-Git: https://github.com/xroche/httrack.git
|
|
Vcs-Browser: https://github.com/xroche/httrack
|
|
|
|
Package: httrack
|
|
Architecture: any
|
|
Multi-Arch: foreign
|
|
Depends: ${misc:Depends}, ${shlibs:Depends}
|
|
Suggests: webhttrack, httrack-doc
|
|
Description: Copy websites to your computer (Offline browser)
|
|
HTTrack is an offline browser utility, allowing you to download a World
|
|
Wide website from the Internet to a local directory, building recursively
|
|
all directories, getting html, images, and other files from the server to
|
|
your computer.
|
|
.
|
|
HTTrack arranges the original site's relative link-structure. Simply
|
|
open a page of the "mirrored" website in your browser, and you can
|
|
browse the site from link to link, as if you were viewing it online.
|
|
HTTrack can also update an existing mirrored site, and resume
|
|
interrupted downloads. HTTrack is fully configurable, and has an
|
|
integrated help system.
|
|
|
|
Package: webhttrack
|
|
Architecture: any
|
|
Multi-Arch: foreign
|
|
Depends: ${misc:Depends}, ${shlibs:Depends}, webhttrack-common, sensible-utils, chromium | firefox-esr | www-browser
|
|
Replaces: webhttrack-common (<< 3.43.9-2)
|
|
Breaks: webhttrack-common (<< 3.43.9-2)
|
|
Suggests: httrack, httrack-doc
|
|
Enhances: httrack
|
|
Description: Copy websites to your computer, httrack with a Web interface
|
|
WebHTTrack is an offline browser utility, allowing you to download a World
|
|
Wide website from the Internet to a local directory, building recursively
|
|
all directories, getting html, images, and other files from the server to
|
|
your computer, using a step-by-step web interface.
|
|
.
|
|
WebHTTrack arranges the original site's relative link-structure. Simply
|
|
open a page of the "mirrored" website in your browser, and you can
|
|
browse the site from link to link, as if you were viewing it online.
|
|
HTTrack can also update an existing mirrored site, and resume
|
|
interrupted downloads. WebHTTrack is fully configurable, and has an
|
|
integrated help system.
|
|
.
|
|
Snapshots: http://www.httrack.com/page/21/
|
|
|
|
Package: webhttrack-common
|
|
Architecture: all
|
|
Multi-Arch: foreign
|
|
Depends: ${misc:Depends}
|
|
Description: webhttrack common files
|
|
This package is the common files of webhttrack, website copier and
|
|
mirroring utility
|
|
|
|
Package: libhttrack3
|
|
Architecture: any
|
|
Multi-Arch: same
|
|
Section: libs
|
|
Depends: ${misc:Depends}, ${shlibs:Depends}
|
|
Replaces: libhttrack2, httrack (<< 3.49.8-2~)
|
|
Breaks: libhttrack2, httrack (<< 3.49.8-2~)
|
|
Description: Httrack website copier library
|
|
This package is the library part of httrack, website copier and mirroring
|
|
utility
|
|
|
|
Package: libhttrack-dev
|
|
Architecture: any
|
|
Multi-Arch: same
|
|
Section: libdevel
|
|
Depends: ${misc:Depends}, ${shlibs:Depends}, zlib1g-dev
|
|
Description: Httrack website copier includes and development files
|
|
This package adds supplemental files for using the httrack website copier
|
|
library
|
|
|
|
Package: httrack-doc
|
|
Architecture: all
|
|
Multi-Arch: foreign
|
|
Section: doc
|
|
Depends: ${misc:Depends}
|
|
Description: Httrack website copier additional documentation
|
|
This package adds supplemental documentation for httrack and webhttrack
|
|
as a browsable html documentation
|
|
|
|
Package: proxytrack
|
|
Architecture: any
|
|
Multi-Arch: foreign
|
|
Depends: ${misc:Depends}, ${shlibs:Depends}
|
|
Suggests: squid, httrack
|
|
Description: Build HTTP Caches using archived websites copied by HTTrack
|
|
ProxyTrack is a simple proxy server aimed to deliver content archived by
|
|
HTTrack sessions. It can aggregate multiple download caches, for direct
|
|
use (through any browser) or as an upstream cache slave server.
|
|
This proxy can handle HTTP/1.1 proxy connections, and is able to reply to
|
|
ICPv2 requests for an efficient integration within other cache servers,
|
|
such as Squid. It can also handle transparent HTTP requests to allow
|
|
cached live connections inside an offline network.
|