Compare commits

..

6 Commits

Author SHA1 Message Date
Nikolay
19ae54bcc7 Merge branch 'master' into vmauth-slowdown-unauthenticated
Signed-off-by: Nikolay <nik@victoriametrics.com>
2026-06-30 21:01:09 +02:00
f41gh7
f84c1055da add changelog entry and address review comments 2026-06-30 21:00:08 +02:00
Max Kotliar
7b28b34e3a abort slow down if client cancelled request 2026-06-30 20:30:46 +03:00
Max Kotliar
d07286ea20 rm slowdown from internal service 2026-06-30 20:13:11 +03:00
Max Kotliar
6d0c5f5099 app/vmauth: do not slow down 401 response if no creds has been provided 2026-06-30 20:11:26 +03:00
Max Kotliar
ed1e3965db app/vmauth: slow down unauthorized responses to mitigate brute-force attacks
Without a delay, an attacker can attempt thousands of authentication
requests per second at minimal cost. The only indication of such an
attack is the
vmauth_http_request_errors_total{reason="invalid_auth_token"} metric,
which may go unnoticed if it isn't being monitored.

Introduce a random delay of 2–3 seconds before returning 401
Unauthorized responses. This significantly increases the cost of
sustaining a brute-force attack while having negligible impact on
legitimate clients, which are not expected to generate unauthorized
requests frequently.

This change also increases the cost of accidental authentication
misconfigurations, but that trade-off is acceptable.

The added delay could itself be abused as part of a DoS attack by tying
up request-processing resources. However, such an attack is easier to
detect because it generates sustained unauthorized traffic from
identifiable source IPs, allowing operators to block or rate-limit the
offending clients by specialized tools.

Partly fixes
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11180
2026-06-30 18:56:53 +03:00
2 changed files with 19 additions and 0 deletions

View File

@@ -6,6 +6,7 @@ import (
"flag"
"fmt"
"io"
"math/rand/v2"
"net"
"net/http"
"net/textproto"
@@ -31,6 +32,7 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/lib/procutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/promauth"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/pushmetrics"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/timerpool"
)
var (
@@ -206,6 +208,7 @@ func requestHandler(w http.ResponseWriter, r *http.Request) bool {
}
invalidAuthTokenRequests.Inc()
slowdownUnauthorizedResponse(r)
if *logInvalidAuthTokens {
err := fmt.Errorf("cannot authorize request with auth tokens %q", ats)
err = &httpserver.ErrorWithStatusCode{
@@ -889,3 +892,18 @@ func debugInfo(u *url.URL, r *http.Request) string {
fmt.Fprint(s, ")")
return s.String()
}
// SlowdownUnauthorizedResponse adds a random delay in the [2..3] seconds range
// before returning an unauthorized response.
// This reduces the effectiveness of brute-force.
func slowdownUnauthorizedResponse(r *http.Request) {
d := 2*time.Second + time.Duration(rand.IntN(1000))*time.Millisecond
t := timerpool.Get(d)
select {
case <-t.C:
case <-r.Context().Done():
}
timerpool.Put(t)
}

View File

@@ -34,6 +34,7 @@ See also [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-rel
* BUGFIX: [stream aggregation](https://docs.victoriametrics.com/victoriametrics/stream-aggregation/): fix possible unexpected increases in `rate_avg` and `rate_sum` if an out-of-order sample is ingested after the previous flush. See [#11140](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11140).
* FEATURE: [vmauth](https://docs.victoriametrics.com/victoriametrics/vmauth/): add `default_vm_access_claim` field into `jwt` section of auth config. It could be used at [JWT claim placeholders](https://docs.victoriametrics.com/victoriametrics/vmauth/#jwt-claim-based-request-templating), if `JWT` token doesn't have `vm_access` claim. See [#11054](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11054).
* FEATURE: [vmauth](https://docs.victoriametrics.com/victoriametrics/vmauth/): prevent possible password brute-force attacks with an artificial 2-3 second delay as recommended by [OWASP](https://owasp.org/Top10/2025/A07_2025-Authentication_Failures). See [#11180](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11180).
* FEATURE: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): reduces CPU usage by 10% at [sharding among remote storages](https://docs.victoriametrics.com/victoriametrics/vmagent/#sharding-among-remote-storages). See [#11113](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11113). Thanks to @bennf for contribution.
* FEATURE: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): add `optimize_repeated_binary_op_subexprs=1` query arg to [/api/v1/query_range](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#range-query) for executing binary operator sides sequentially when they share the same optimized aggregate rollup result expression. This allows the second side to reuse rollup result cache populated by the first side. See [#10575](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10575).