rms/httrack
1
0
Fork 0
mirror of https://github.com/xroche/httrack.git synced 2026-06-28 21:17:57 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,242 Commits 135 Branches 24 Tags
174bc565fc75423b8e9d2777f3c8347bdb813061
Commit Graph

23 Commits

Author SHA1 Message Date
Xavier Roche
085937b305 Fix get_httptype contenttype overflow; bound the mime/normalize APIs 
get_httptype() took the caller buffer as a bare char* and raw-strcpy'd the MIME
string into it, so crawling a URL ending in .docx/.pptx/.xlsx (whose table MIME
types reach 73 chars) overflowed the 64-byte htsblk.contenttype that the htsback
and htslib callers pass, corrupting the adjacent struct fields. Remotely
triggerable.

* Widen htsblk contenttype/charset/contentencoding to HTS_MIMETYPE_SIZE (128, a
  new named constant holding the longest registered MIME type). This changes the
  installed htsblk layout, so bump the library soname (VERSION_INFO 2:49:0 ->
  3:0:0).
* Add bounded get_httptype_sized(), guess_httptype_sized() and
  adr_normalized_sized() that take the destination size and use
  strlcpybuff/snprintf. The old get_httptype(), guess_httptype() and
  adr_normalized() stay as wrappers, now marked HTS_DEPRECATED (portable:
  GCC/Clang attribute, MSVC __declspec, nothing elsewhere). Internal callers
  pass the real buffer size; the deprecated wrappers bound to the implicit
  contract their old callers relied on (HTS_MIMETYPE_SIZE for the mime buffer,
  HTS_URLMAXSIZE*2 for the URL buffer) rather than staying unbounded, so they
  abort on overflow instead of silently corrupting memory.
* get_httptype_sized(), guess_httptype_sized() and give_mimext() now report
  whether a type/extension was written; callers check the result and bail
  rather than use a possibly-empty buffer (e.g. the is_hypertext_mime helpers).
  A user "--assume cgi=" rule (empty value) matches but writes nothing, so
  get_httptype_sized() returns the buffer's emptiness, matching the old callers'
  strnotempty(s) test rather than reporting a bogus recognized type.
* -#7 basic_selftests: a .pptx MIME (73 chars) is stored whole into a real
  htsblk.contenttype (a [64] field makes the bounded copy abort); give_mimext
  and get_httptype_sized return values; the octet-stream fallback; the empty
  --assume rule; plus fil_normalized "//"-in-query preservation and cut_path
  trailing-slash / single-char branches.

Signed-off-by: Xavier Roche <roche@httrack.com>
 2026-06-16 11:10:49 +02:00
Xavier Roche
7e7b06b55a Reword the ethical-use notice in source headers 
The old license-header note ("We hereby ask people using this source NOT to
use it in purpose of grabbing emails addresses...") read awkwardly: "in
purpose of", "emails addresses", "on persons", and a one-item bullet list
under an "Important notes:" heading.

Replace it across all source headers, configure/configure.ac, and README with
a single clean sentence, and align the wording everywhere:

  Ethical use: we kindly ask that you NOT use this software to harvest email
  addresses or to collect any other private information about people. Doing so
  would dishonor our work and waste the many hours we have spent on it.

Pure text change, no behavior impact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
 2026-06-14 18:38:17 +02:00
Xavier Roche
995cc6c86e Normalize copyright years and add SPDX identifiers 
Collapse the stale "1998-2017" copyright ranges (and a handful of other
ranges) in HTTrack-authored source to a single earliest year taken from
git history: 1998 for files tracing back to the 2012 release-history
import, and the real first-commit year for later additions (2013 for
htsencoding, 2014 for htsarrays/htssafe/htsconcat, 2026 for the cache
self-test). Each header also gains an SPDX-License-Identifier:
GPL-3.0-or-later line.

The runtime "about" banners (httrack, proxytrack) and the man pages keep
a range, but now end in the current year computed at build time: via
__DATE__ for the C banners and makeman.sh for the generated httrack.1,
so they no longer freeze at a stale year.

Third-party notices (Even Rouault, Mathias Svensson, Info-ZIP, Eric
Young) and the BSD-licensed coucal submodule are left untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
 2026-06-14 18:15:40 +02:00
Xavier Roche
fa0e5f72e0 Fixed typos 2023-01-14 17:21:57 +01:00
Xavier Roche
340c0d940f Fixed sprintf 2023-01-14 15:12:07 +01:00
Xavier Roche
50f57ab6ae Updated year to 2017 2017-04-01 21:39:27 +02:00
Xavier Roche
f6d076d743 Updated year 2016 2016-04-26 19:25:50 +02:00
Xavier Roche
e81acec4fc 2014 is so last year! 2015-03-14 15:01:53 +01:00
Xavier Roche
cefcc04266 "const correctness" cleanup 
added the following default flags:
  -Wformat
  -Wformat-security
  -Wmultichar
  -Wwrite-strings
fixed several other warnings
 2014-05-23 21:33:43 +00:00
Xavier Roche
5544f503bf Big cleanup in functions writing to a char buffer without proper size boundary. 2014-05-02 15:13:29 +00:00
Xavier Roche
792311df15 2013 is so last year. 2014-04-24 19:28:32 +00:00
Xavier Roche
f967595e08 Fixed warnings 2013-10-09 13:11:06 +00:00
Xavier Roche
60644a7309 Fixed FSF GPL reference (postal address removed, added website) 
Fixed year notice
 2013-09-13 15:54:23 +00:00
Xavier Roche
5ab61bd6e3 Some cleanup in obviously wrong 15-year-old code. 2013-08-17 07:57:44 +00:00
Xavier Roche
07f4b00367 Indenting cleanup for all *.[ch] files, except htsparse.c (too ugly to be automatically indented for now) and /minizip/, /mmsrip/ (external files) 
setup:
  indent -l80 -lc80 -nhnl -nut -bad -bap -bbo -br -brf -bli2 -brs -bls -br -ss -sai -pmt -nsaw -nsaf -nprs -i2 -ce -npsl -npcs -cs -sob -cdw -nbc -lp

logs:
indent: ./src/htsback.c:157: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsback.c:1417: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsback.c:1826: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsback.c:1833: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsback.c:1981: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsback.c:2685: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsback.c:2747: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsback.c:2861: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsback.c:3128: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsback.c:3512: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htswizard.c:140: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htswizard.c:597: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htswizard.c:598: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htswizard.c:611: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htscoremain.c:97: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htscoremain.c:106: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htscoremain.c:106: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htscoremain.c:256: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htscoremain.c:262: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htscoremain.c:935: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/httrack.c:271: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/httrack.c:272: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/httrack.c:273: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/httrack.c:274: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/httrack.c:275: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/httrack.c:276: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/httrack.c:277: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/httrack.c:278: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/httrack.c:279: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/httrack.c:280: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/httrack.c:281: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/httrack.c:662: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/httrack.c:752: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsweb.c:413: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsweb.c:414: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsweb.c:415: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsweb.c:416: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsweb.c:417: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsweb.c:418: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsweb.c:419: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsweb.c:420: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsweb.c:421: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsweb.c:422: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsweb.c:423: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.c:188: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.c:507: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.c:508: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.c:509: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.c:510: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.c:511: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.c:512: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.c:580: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.c:581: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.h:115: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.h:246: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.h:247: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.h:248: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.h:249: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.h:250: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.h:251: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.h:314: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/proxy/proxytrack.h:315: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htstools.c:151: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htstools.c:158: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htstools.c:161: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htstools.c:166: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htstools.c:174: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htstools.c:177: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htstools.c:188: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htstools.c:243: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htstools.c:245: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htstools.c:248: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htstools.c:257: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htstools.c:647: Warning:old style assignment ambiguity in "=*".  Assuming "= *"
indent: ./src/htsbauth.c:364: Warning:old style assignment ambiguity in "=&".  Assuming "= &"
indent: ./src/htsbauth.c:387: Warning:old style assignment ambiguity in "=&".  Assuming "= &"
indent: ./src/htscache.c:473: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsserver.h:112: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsserver.h:187: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsjava.c:312: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsjava.c:379: Warning:old style assignment ambiguity in "=&".  Assuming "= &"
indent: ./src/htsjava.c:407: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsjava.c:472: Warning:old style assignment ambiguity in "=&".  Assuming "= &"
indent: ./src/htsjava.c:483: Warning:old style assignment ambiguity in "=*".  Assuming "= *"
indent: ./src/htslib.c:593: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:809: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:1743: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:1874: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:1896: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:1984: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:2085: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:2604: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:2605: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:2606: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:2607: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:2608: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:2609: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:2672: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:2673: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:2906: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:2928: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:2996: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:4802: Warning:old style assignment ambiguity in "=&".  Assuming "= &"
indent: ./src/htslib.c:5353: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:5354: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:5355: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:5357: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:5429: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htslib.c:5435: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htscore.c:208: Warning:old style assignment ambiguity in "=&".  Assuming "= &"
indent: ./src/htscore.c:277: Warning:old style assignment ambiguity in "=&".  Assuming "= &"
indent: ./src/htscore.c:279: Warning:old style assignment ambiguity in "=&".  Assuming "= &"
indent: ./src/htscore.c:357: Warning:old style assignment ambiguity in "=*".  Assuming "= *"
indent: ./src/htscore.c:394: Warning:old style assignment ambiguity in "=*".  Assuming "= *"
indent: ./src/htscore.c:1544: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htscore.c:3330: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htscore.c:3361: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htshash.c:140: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htshash.c:217: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsname.c:733: Warning:old style assignment ambiguity in "=*".  Assuming "= *"
indent: ./src/htsname.c:749: Warning:old style assignment ambiguity in "=*".  Assuming "= *"
indent: ./src/htsname.c:933: Warning:old style assignment ambiguity in "=*".  Assuming "= *"
indent: ./src/htsname.c:1520: Warning:old style assignment ambiguity in "=*".  Assuming "= *"
indent: ./src/htszlib.c:76: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htszlib.c:81: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htscatchurl.c:268: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsinthash.c:156: Warning:old style assignment ambiguity in "=&".  Assuming "= &"
indent: ./src/htsinthash.c:159: Warning:old style assignment ambiguity in "=&".  Assuming "= &"
indent: ./src/htsinthash.c:176: Warning:old style assignment ambiguity in "=&".  Assuming "= &"
indent: ./src/htsinthash.c:179: Warning:old style assignment ambiguity in "=&".  Assuming "= &"
indent: ./src/htsinthash.c:226: Warning:old style assignment ambiguity in "=&".  Assuming "= &"
indent: ./src/htsinthash.c:253: Warning:old style assignment ambiguity in "=&".  Assuming "= &"
indent: ./src/htsftp.c:169: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsftp.c:177: Warning:old style assignment ambiguity in "=-".  Assuming "= -"
indent: ./src/htsfilters.c:67: Warning:old style assignment ambiguity in "=*".  Assuming "= *"
indent: ./src/htsfilters.c:80: Warning:old style assignment ambiguity in "=*".  Assuming "= *"
 2013-05-14 20:30:28 +00:00
Xavier Roche
0573b166b2 Merge sources from windows-1252 to utf-8 2013-05-14 19:20:14 +00:00
Xavier Roche
78173ef819 Build warning cleanup. 
* introduced SOClen type (aka. socklen_t)
 2013-04-21 18:46:35 +00:00
Xavier Roche
525118dfe8 Charset fixes 2012-05-07 10:02:58 +00:00
Xavier Roche
09be6e615e GPL v3 2012-03-24 12:21:42 +00:00
Xavier Roche
660b569b09 httrack 3.41.2 2012-03-19 12:59:03 +00:00
Xavier Roche
844ecc3707 httrack 3.33.16 2012-03-19 12:55:42 +00:00
Xavier Roche
25adbdabb4 httrack 3.30.1 2012-03-19 12:51:31 +00:00
Xavier Roche
ad5b7acc19 Imported httrack 3.20.2 2012-03-19 12:36:11 +00:00