mirror of
https://github.com/xroche/httrack.git
synced 2026-07-23 17:19:17 +03:00
e17f4f12a0
A new --pause MIN[:MAX] (seconds, -%G) waits a random MIN..MAX between files so a crawl looks less like a bot and is gentler on the server; a single value is a fixed delay. Disabled by default. It reuses the existing non-blocking launch gate (back_pluggable_sockets_strict): rather than Sleep() -- which would freeze the single select() pump and stall the other in-flight transfers -- the gate just withholds new launches until the delay elapses, one file per gap. The per-gap target is derived from the last-request timestamp so it stays stable across the many gate evaluations within a gap yet rerolls on each launch; sampling rand() per evaluation would instead bias the realized delay toward MIN. Two int fields appended at the httrackp tail (ABI-stable, no soname bump). Covered by a pure-function self-test (range + spread, with teeth against the min-bias bug) and a local-server crawl that asserts the pause slows a multi-file mirror. Closes #185 Signed-off-by: Xavier Roche <roche@httrack.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
16 lines
436 B
Bash
Executable File
16 lines
436 B
Bash
Executable File
#!/bin/bash
|
|
#
|
|
# --pause (#185): the inter-file pause target must stay in [min,max] and spread
|
|
# across it (a per-call rand() would collapse it toward min). Driven by the
|
|
# in-process 'httrack -#test=pause' test. POSIX-portable ($(BASH) is /bin/sh on macOS).
|
|
|
|
set -eu
|
|
|
|
# 'run' is an ignored placeholder argument.
|
|
out=$(httrack -#test=pause run)
|
|
|
|
test "$out" = "pause: OK" || {
|
|
echo "expected 'pause: OK', got: $out" >&2
|
|
exit 1
|
|
}
|