* Non-ASCII single -O drops the hts-cache into a mangled twin directory on Windows
PR #636 fixed the log half of #630 and left the cache for a follow-up. With a
single -O café, path_log holds UTF-8 bytes, but cache_init created hts-cache and
opened new.zip through ANSI calls, so on Windows the cache landed in a second
mangled café-twin directory beside the mirror, and a later --update read the
cache from the wrong place.
This routes every path_log filesystem op in cache_init and the reconcile helpers
through the UTF-8 wrappers (structcheck_utf8, MKDIR, FOPEN, UNLINK, RENAME,
fexist_utf8, fsize_utf8), and opens the cache ZIP through minizip's filefunc
entry points (zipOpen2/unzOpen2) backed by hts_fopen_utf8. On POSIX the wrappers
resolve to the same libc calls, so only Windows changes. The conversion is
all-or-nothing: a directory created UTF-8 but a ZIP opened ANSI would split the
cache, so the whole cache-open path flips together.
The corrupt-cache repair path stays ANSI. mztools' unzRepair takes no filefunc,
so a corrupt cache under a non-ASCII path fails to repair (the site is
re-crawled) rather than forking a twin.
Test 69 now also asserts the cache lands under the café directory through a new
--cache-under-logroot audit. Like the log assertion it only bites on the Windows
CI leg; on POSIX it exercises the code path without reproducing the mojibake.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Cache UTF-8 filefunc must keep 64-bit offsets and the cache dir 0700
The #630 cache-UTF-8 hook built a 32-bit zlib_filefunc_def via
fill_fopen_filefunc and called unzOpen2/zipOpen2. Minizip runs a 32-bit
filefunc through fill_zlib_filefunc64_32_def_from_filefunc32, which NULLs the
64-bit seek so every seek truncates the offset to uLong: on Windows LLP64 a
cache >=4GB hard-fails and >=2GB corrupts. The ANSI code this replaced went
through fill_fopen64_filefunc and never truncated. Keep 64-bit: fill the
zlib_filefunc64_def and override only zopen64_file with the UTF-8 opener, then
call unzOpen2_64/zipOpen2_64.
Also restore the POSIX cache-dir mode: the MKDIR macro creates with
HTS_ACCESS_FOLDER (0755), loosening the 0700 the explicit mkdir used. There is
no UTF-8 mkdir-with-mode, so keep the platform split - UTF-8 MKDIR on Windows
(which ignores the mode), mkdir(HTS_PROTECT_FOLDER) on POSIX.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Tighten the #630 comments
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
0d25999069