16bbf2ad27
* Percent-escape a query-string character reference the page charset cannot represent On a page with no declared charset (iso-8859-1 fallback), a reference like € was left as literal source text in the query string: its '&' and '#' then act as delimiters, so the origin parsed more parameters than the document expressed. The URL Standard writes such a code point %26%23<decimal>%3B. Adds hts_unescapeEntitiesWithCharsetSpecial() with an opt-in UNESCAPE_ENTITIES_URL_QUERY flag, set only by the query-string call site; the plain decoder keeps its text contract and its in-place guarantee. The query is now decoded out-of-place, since the escape grows the string. Closes #854 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Detect an unrepresentable code point with the encoder, not a round trip The round-trip comparison escaped code points the charset does represent: hts_convertStringToUTF8() short-circuits on an ASCII string and hands the bytes back undecoded, so every reference on ISO-2022-JP, UTF-7 or HZ compared unequal, and glibc's CJK tables are not injective (cp932 U+301C came back as U+FF5E). On POSIX iconv already fails outright on a code point it cannot encode, so the NULL return is the answer. Windows substitutes instead of failing, and reports it through WideCharToMultiByte's lpUsedDefaultChar, which we now pass; hts_convertStringFromUTF8Strict() turns that into a NULL. Test 139 grows a declared shift_jis and a declared iso-2022-jp page, which the two charset cases it had could not have caught. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Assert the ISO-2022-JP query per field, and let the two cp932 tables differ Windows maps 0x8160 to U+FF5E rather than U+301C and reports the substitution, so 〜 is genuinely unencodable there and the escape is right: accept either encoding for the shift_jis page. ISO-2022-JP's trailing shift back to ASCII also differs between iconv and Windows, so that page is asserted field by field, which is what the round-trip check would have broken anyway. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Let Windows substitute the euro sign on a stateful codepage WideCharToMultiByte refuses lpUsedDefaultChar on ISO-2022-JP, so the substitution cannot be seen and € goes out as '?' there, as it did before this branch. It splits no query, and the page is in the test for the references the charset does represent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> --------- Signed-off-by: Xavier Roche <roche@httrack.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
HTTrack Website Copier - Development Repository
About
Copy websites to your computer (Offline browser)
HTTrack is an offline browser utility, allowing you to download a World Wide website from the Internet to a local directory, building recursively all directories, getting html, images, and other files from the server to your computer.
HTTrack arranges the original site's relative link-structure. Simply open a page of the "mirrored" website in your browser, and you can browse the site from link to link, as if you were viewing it online.
HTTrack can also update an existing mirrored site, and resume interrupted downloads. HTTrack is fully configurable, and has an integrated help system.
WinHTTrack is the Windows 2000/XP/Vista/Seven release of HTTrack, and WebHTTrack the Linux/Unix/BSD release.
Website
Main Website: http://www.httrack.com/
Compile trunk release
A git checkout ships only the autotools sources, so ./bootstrap (which runs
autoreconf) regenerates configure first; this needs autoconf, automake and
libtool. Released tarballs already include configure, so building from a
tarball skips ./bootstrap.
git clone https://github.com/xroche/httrack.git --recurse-submodules
cd httrack
./bootstrap
./configure --prefix=$HOME/usr && make -j8 && make install
Or use the one-shot wrapper (bootstrap + configure + make), which forwards its
arguments to configure:
./build.sh --prefix=$HOME/usr