mirror of
https://github.com/xroche/httrack.git
synced 2026-08-13 11:12:19 +03:00
f5fbd1a82edc9362bb052d784c97294f94d21fae
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Xavier Roche
|
2a44dc1728 |
doc: note the --changes example is one captured run (#1027)
generator and date are genuine per-run fields (htschanges.c stamps HTTRACK_VERSION and the current time on every write), so dropping them would misrepresent the payload. Label the sample instead, so a reader does not compare it against their own output as if it should match. Signed-off-by: Xavier Roche <roche@httrack.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Xavier Roche
|
8168aa3a45 |
The masthead wordmark is a 400x34 GIF that blurs on any hi-DPI screen (#916)
* The masthead wordmark is a 400x34 GIF that blurs on any hi-DPI screen Replaces it with an outlined SVG across the 38 documentation and WebHTTrack pages that carry it. The original was set in Futura, so the lockup was refitted in Jost*, the closest free Futura revival, taking weight from the measured stem thickness, size from the cap heights and tracking by least squares against the glyph positions in the old bitmap. tests/82 now asserts that every image a GUI page names is actually served. Signed-off-by: Xavier Roche <xroche@gmail.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * Point the shared chrome generator at the new wordmark The masthead of the 13 generated pages comes from tools/doc-chrome.py, so editing the pages alone left the generator disagreeing with its own output and --check red. Signed-off-by: Xavier Roche <xroche@gmail.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> --------- Signed-off-by: Xavier Roche <xroche@gmail.com> Signed-off-by: Xavier Roche <roche@httrack.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Xavier Roche
|
f1ace40d47 |
Fifteen documentation pages are dead ends in a 2007 table layout (#879)
Nine of the twenty pages linked nowhere at all and seven had a single inbound link, so a reader landing on cache.html from a search engine got no navigation, no breadcrumb and no way back. Each page also carried its own copy of the same inline stylesheet inside a six-table shell with a 400px floor, which is why sixteen of them scrolled sideways on a phone. They now share the guide's chrome: masthead, sidebar, footer, and where the headings allow it a generated "On this page" list. file:// has no server includes, so the navigation is real markup in every page, written by tools/doc-chrome.py and verified by CI; tools/doc-links.py resolves every relative link and fragment. Both refuse to pass vacuously, and both were checked against planted defects. The prose is untouched. For twelve of the fifteen pages the content text is identical word for word; the three exceptions are cmdguide.html losing the contents list the sidebar now carries, contact.html re-encoded to UTF-8, and one font tag whose removal joined two words the browser already ran together. Each page also gains a real title and its own description, replacing the shared blurb whose keyword list still advertised Windows 95 and AIX 4.0. Image zoom and sidebar highlighting move to doc.js so every page has them, leaving guide.js the platform switcher and the option filter. The guide renders identically at 1100px in light and dark, and thirteen behaviours pass unchanged before and after; at 390px one Proxy paragraph that used to overflow its container by 37px now wraps. Nothing scrolls sideways at 390px or 1100px, and no page throws a script error. httrack.man.html keeps its own bare styling for now, since reskinning it belongs in the generator that writes it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> |
||
|
Xavier Roche
|
7e2209d4cc |
No way to see what changed between two mirrors (#721)
* Report what a crawl changed against the previous mirror (--changes) --update already knows which resources were new, which changed and which the server called unchanged, and throws it away: the flags reach file_notify() and go no further than a log line, while deletions exist only as a side effect of purging. --changes (-%d) keeps all of it and writes hts-changes.json plus a one-line summary in the log. "Changed" means the bytes differ, not that the server re-sent the resource. Comparing the mirrored files directly would not work: HTTrack stamps every parsed page with the crawl date via the footer, so those bytes differ on every run. Payloads are compared instead, the previous one coming from the cache for parsed pages and from the local copy sampled just before it is overwritten for everything else. The mirror-relative path, not the URL, is the accumulator's key, so a redirect and its target that share a save name are one entry; and only the first notify for a file samples its pre-run state, so a retried transfer is not counted twice. What counts as already mirrored comes from the previous run's file index rather than from the file's presence on disk: a partial left by this crawl's own failed attempt is on disk but was never part of the previous mirror. The deleted set is now computed whether or not purging is enabled; unlinking still happens only under --purge-old. Closes #714 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <xroche@gmail.com> * changes: never leave a stale report behind A crawl that mirrored nothing created no accumulator, so hts_changes_close_opt returned without writing and the previous run's report stayed on disk as if it described this one. Write it whenever --changes is on. The no-data rollback is the deliberate exception, and is now documented: it restores the previous cache generation, so leaving the matching report alone is the consistent behaviour. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <xroche@gmail.com> * changes: drop em dashes from the format page Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <xroche@gmail.com> * changes: fix the review's three findings The size shortcut compared rendered on-disk sizes even for parsed pages, whose payload digests describe something else entirely, so it decided the outcome before the payload comparison could run: a page whose payload never changed but whose rewritten links moved read as changed. It now only applies when both digests describe the file on disk. file_notify() reaches the accumulator from the FTP download thread as well as the main one, and the lazy allocation, the coucal write and the entries realloc were all unguarded. Every entry point now takes a mutex, and the HTML hook does its cache read before taking it, since that read can itself re-enter file_notify() and move the array. Two fixtures cover what nothing did: a gzipped direct-to-disk body that changes at constant length (without the pre-sample before the decoded temp is renamed, it reads as unchanged), and a page with a fixed payload behind a redirect whose target is renamed, so its file on disk changes length while its bytes do not. Both were checked against builds with the respective fix reverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <xroche@gmail.com> * changes: document the renamed-file case in the format page Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <xroche@gmail.com> * changes: refresh two stale test comments Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <xroche@gmail.com> * Merge origin/master into feat/change-report Both sides appended to tests/Makefile.am's TESTS; kept master's 86_local-proxytrack-cache-longfields.test alongside 88_local-changes.test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <xroche@gmail.com> * changes: translate the new GUI strings into the remaining 28 locales (#740) The feature PR added the two LANG_CHANGES* entries to lang.def with English and Francais only; every other locale fell back to English in the WebHTTrack form. Each file is written in its own declared charset. Signed-off-by: Xavier Roche <roche@httrack.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * changes: fix the review's blocking findings Lock the report path against the FTP thread the crawl never joins, seal the accumulator once the report is written, key entries off the project directory so the report survives --cache=0, stop calling a file gone when the crawl only failed to re-fetch it, and skip the hook's work entirely when --changes is off. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * changes: prove the fixes, and give the report a cache-off mode Adds a changes-race self-test (the FTP shape: notifier threads against the report path), fixtures for a transfer the crawl never completes, for a leftover file at a name the crawl mirrors fresh, and for a cache-off mirror, plus a pass with purging on. Registers the web GUI's --changes box with the clearing companion master's #725 now requires, and documents the degraded mode. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> * changes: keep the failed-transfer case portable A connection killed before the status line surfaces differently on macOS, where it truncates the mirrored file to zero (#748). Assert only what holds on both: it is never reported gone, and its file survives. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com> --------- Signed-off-by: Xavier Roche <xroche@gmail.com> Signed-off-by: Xavier Roche <roche@httrack.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |