Compare commits

...

1 Commits

Author SHA1 Message Date
Xavier Roche
3a20de8ec4 Raise the test server's listen backlog so -c8 bigcrawl stops flaking on macOS
local-server.py used ThreadingHTTPServer's default request_queue_size of 5.
macOS/BSD drop SYNs once the accept queue overflows; Linux is lenient. Under a
parallel make check the single-threaded accept loop is starved for CPU, drains
the 5-deep queue slowly, and an -c8 bigcrawl burst overflows it. The
36_local-bigcrawl file count then flaked (e.g. 361 -> 283) because --retries=0
turns each dropped connection into a lost file. Raise the backlog to 128 for
ample headroom; -c8 is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
2026-07-12 10:36:15 +02:00

View File

@@ -1645,7 +1645,12 @@ def main():
def factory(*a, **kw):
return Handler(*a, directory=root, **kw)
httpd = ThreadingHTTPServer((args.bind, 0), factory)
# macOS/BSD drop SYNs when the listen backlog overflows (Linux is lenient);
# raise it from Python's default 5 so a busy -c8 crawl can't lose fetches.
class BacklogHTTPServer(ThreadingHTTPServer):
request_queue_size = 128
httpd = BacklogHTTPServer((args.bind, 0), factory)
if args.tls:
import ssl