Compare commits

...

8 Commits

Author SHA1 Message Date
Max Kotliar
b96f63b588 upd desc 2026-07-01 15:23:22 +03:00
Max Kotliar
6b980bdb6f add changelog 2026-07-01 15:09:06 +03:00
Max Kotliar
222bfb0f0e fix dashboard link 2026-07-01 15:06:15 +03:00
Max Kotliar
13036b9297 upd 2026-07-01 12:53:51 +03:00
Max Kotliar
359c634157 Merge remote-tracking branch 'opensource/master' into vmauth-invalid-auth-token-alert 2026-07-01 12:48:31 +03:00
Max Kotliar
290029897b specialize alert 2026-06-30 20:09:39 +03:00
Max Kotliar
9a150c309c fix dashboard link 2026-06-30 19:42:06 +03:00
Max Kotliar
2a40a40e9e deployment: add TooManyInvalidAuthTokenRequests alert
The alert should help surface client auth misconfigurations or potential
brute force attacks.

Related to
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11180
2026-06-30 19:31:05 +03:00
2 changed files with 17 additions and 0 deletions

View File

@@ -56,3 +56,19 @@ groups:
summary: "Too many errors served for user {{ $labels.username }} (instance {{ $labels.instance }})"
description: "Requests from user {{ $labels.username }} are receiving errors.
Please check the vmauth logs to verify that the configuration is correct and clients are sending valid requests."
- alert: PotentialBruteForceAttack
expr: sum(increase(vmauth_http_request_errors_total{reason="invalid_auth_token"}[5m])) without (instance, reason) > 5000
for: 1m
labels:
severity: warning
annotations:
dashboard: "{{ $externalURL }}/d/nbuo5Mr4k?viewPanel=16&var-job={{ $labels.job }}"
summary: "vmauth {{ $labels.job }} is receiving too many requests with invalid auth tokens"
description: |
vmauth {{ $labels.job }} received {{ $value }} requests with invalid auth tokens in the last 5 minutes.
This may indicate a brute-force attack or a misconfiguration on the client side.
For brute force check the remote_addr in access logs to identify and block the source.
See https://docs.victoriametrics.com/victoriametrics/vmauth/#access-log
For misconfiguration check the clients metrics/logs or enable -logInvalidAuthTokens.

View File

@@ -32,6 +32,7 @@ See also [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-rel
* FEATURE: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): reduces CPU usage by 10% at [sharding among remote storages](https://docs.victoriametrics.com/victoriametrics/vmagent/#sharding-among-remote-storages). See [#11113](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11113). Thanks to @bennf for contribution.
* FEATURE: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): add `optimize_repeated_binary_op_subexprs=1` query arg to [/api/v1/query_range](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#range-query) for executing binary operator sides sequentially when they share the same optimized aggregate rollup result expression. This allows the second side to reuse rollup result cache populated by the first side. See [#10575](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10575).
* FEATURE: [vmauth](https://docs.victoriametrics.com/victoriametrics/vmauth/): prevent possible password brute-force attacks with an artificial 2-3 second delay as recommended by [OWASP](https://owasp.org/Top10/2025/A07_2025-Authentication_Failures). See [#11180](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11180).
* FEATURE: [alerts](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/rules): add `PotentialBruteForceAttack` alerting rule to [vmauth alerts](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/rules/alerts-vmauth.yml). The new rule notifies when vmauth receives too many requests with invalid auth tokens, which may indicate a brute-force attack or client misconfiguration. See [#11180](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11180).
* BUGFIX: all VictoriaMetrics components: cancel in-flight HTTP requests shortly before `-http.maxGracefulShutdownDuration` elapses during graceful shutdown, so they can drain and the shutdown completes cleanly within that window instead of timing out and exiting via `logger.Fatalf` -> `os.Exit`. This prevents skipping the storage flush and losing in-memory data when long-lived requests are in flight (such as VictoriaLogs live tailing). See [#1502](https://github.com/VictoriaMetrics/VictoriaLogs/issues/1502).
* BUGFIX: `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): fixes unexpected rare rerouting. See [#11162](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11162).