The outdated link to the slides for this talk has been dropped in the commit f0a147fdf7 .
The video recording for the talk is still available at YouTube ( https://www.youtube.com/watch?v=ZJQYW-cFOms ),
so put it to the articles page.
Enterprise version of VictoriaTraces isn't available yet, but it is better to mention it
at the https://docs.victoriametrics.com/victoriametrics/enterprise/ page for the sake of consistency.
While at it, consistently use absolute links, even if they point to the same document.
This simplifies moving the text between docs without breaking the links.
This change should clearly distinguish different multitnenacy scenarios
for vmagent. It is expected to be easier to read and follow for users.
---------
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Pablo Fernandez <46322567+TomFern@users.noreply.github.com>
The core `lib/promauth` already supports `usernameFile`
configs, but the CLI flags for vmagent remotewrite and vmalert
datasource/remotewrite/remoteread/notifier only expose
`basicAuth.username`.
This commit adds the corresponding `basicAuth.usernameFile` flags to match
the existing `basicAuth.passwordFile` pattern, closing the gap between
YAML and CLI configuration.
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9436
This commit adds a warning message, if `-memory.allowedBytes` has value less than 1MB.
It should help to debug possible issues, if there is a problem with app start-up due to low memory limit.
For example, fastcache could panic at `-memory.allowedBytes=`
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10935
In most cases, vmalert is configured to write to vm components like
vminsert or vmagent, using VictoriaMetrics remote write protocol can
save network bandwidth.
The VictoriaMetrics remote write protocol is used by default, and the
protocol is downgraded from VictoriaMetrics to Prometheus remote write
if one request fails with protocol error.
Related PR https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10929
Replace the pattern of `git checkout <tag> && make <binary>` with `git
worktree add /tmp/vm-* <tag>` so that flag updates no longer switch the
working tree of the current repository. Each variant (opensource,
enterprise, cluster) gets its own worktree, removing the need to restore
the original branch between steps.
Also normalize dynamic default values in vmctl prometheus flags
(-prom-tmp-dir-path) to `os.TempDir()` to reduce noisy diffs caused by
machine-specific temp paths.
Add '=' separator between label name and value when computing the hash
to prevent false collisions, like {a="bc"} and {ab="c"} hashing to the
same value.
getLabelsHashForShard is added to avoid sharding disruptions in vmagent
(-remoteWrite.shardByURL=true mode). The function preserves previous
behavior, without '=' between name and value.
PR https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10937
### Describe Your Changes
Fix stale `quantiles(...)` stream aggregation output for series without
samples in the current aggregation interval.
Previously, `quantilesAggrConfig` reused the `quantiles` buffer across
aggregation values. If `quantilesAggrValue.flush` was called for a
series without samples after another series had already calculated
quantiles, the stale quantile
values could be emitted for the empty series.
This could produce unrealistic `*_quantiles` output values and make the
same aggregated value appear across unrelated labelsets.
The PR skips `quantiles(...)` output when there is no histogram for the
current interval and adds a regression test for this case.
### Checklist
The following checks are **mandatory**:
- [x] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
- [x] My change adheres to [VictoriaMetrics development
goals](https://docs.victoriametrics.com/victoriametrics/goals/).
---------
Co-authored-by: hagen1778 <roman@victoriametrics.com>
When a request contains both URL path query params and POST form values
for extra_label and extra_filters[], URL query params now take
precedence. This resolves the conflict between the two sources and
simplifies security enforcement for extra_label/extra_filters policies
via vmauth or any other http proxy.
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10908
This commit introduces a new metric to expose fs type for the provided path.
For example:
```
vm_fs_info{path="/vmstorage-data", fs_type="xfs"}
```
Path must be registered with new method `fs.RegisterPathFsMetrics`.
fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10482
Add a new `Kafka (Enterprise)` row to both vmagent dashboards:
- `dashboards/vmagent.json`
- `dashboards/vm/vmagent.json`
The row is placed before `Drilldown` and contains three Kafka-specific
panels:
- `Kafka bytes`
- `Kafka messages in/out`
- `Kafka and consumer errors`
The goal is to provide a compact Kafka-focused view for enterprise
vmagent deployments without duplicating the existing generic remote
write panels such as connection saturation and persistent queue size.
The new row helps distinguish:
- producer vs consumer throughput at the Kafka topic level
- message-rate shifts that may indicate smaller Kafka payloads and
higher per-message overhead
- producer-side Kafka errors vs consumer-side Kafka errors
Descriptions include links to the relevant Kafka documentation sections.
PR https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10728
---------
Co-authored-by: Max Kotliar <mkotlyar@victoriametrics.com>
This commit adds possibility to omit tenantID in the URL path. In this case,
tenantID will be fetched from HTTP headers `AccountID` and `ProjectID`.
If headers are missing too, then default `0:0` tenantID is used.
This functionality can be enabled only if -enableMultitenantHandlers
cmd-line flag was set to vminsert, vmselect or vmagent.
Motivation: this change makes VM configuration for multienancy
consistent with VL configuration - see
https://docs.victoriametrics.com/victorialogs/#multitenancy. And keeps
backward compatibility in the same time.
fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4241
Before, some of the template examples were wrongly renderred by hugo.
For example:
```
http://vm-grafana.com/<dashboard-id>?viewPanel=<panel-id>&from={{($activeAt.Add (parseDurationTime \"-1h\")).UnixMilli}}&to={{($activeAt.Add (parseDurationTime \"1h\")).UnixMilli}}
```
was renderred like:
```
http://vm-grafana.com/ ?viewPanel=&from={{($activeAt.Add (parseDurationTime "-1h")).UnixMilli}}&to={{($activeAt.Add (parseDurationTime "1h")).UnixMilli}}
```
Wrapping examples in ` helps to render them raw.
While there, also fixed some examples.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
On 2025-12-16, Hetzner Cloud deprecated the `datacenter` field in their
Servers API and introduced a top-level `location` field carrying the
same data. The `datacenter` field will be removed after 2026-07-01.
Without this change, `__meta_hetzner_hcloud_datacenter_location`, and
`__meta_hetzner_hcloud_datacenter_location_network_zone` would silently
become empty for the `hcloud` role after that date.
This mirrors the change made in Prometheus v3.11.0
([prometheus/prometheus#17850](https://github.com/prometheus/prometheus/pull/17850)).
## Changes
**`hcloud` role:**
- Add `HCloudLocation` struct and `Location` field on `HCloudServer`,
mapped to the new top-level `location` API field
- Emit two new canonical labels: `__meta_hetzner_hcloud_location` and
`__meta_hetzner_hcloud_location_network_zone`
- Keep the deprecated `__meta_hetzner_hcloud_datacenter_location` and
`__meta_hetzner_hcloud_datacenter_location_network_zone` labels, now
sourced from the new `location` field so they continue to work past
2026-07-01
- `__meta_hetzner_datacenter` (the datacenter name, e.g. `fsn1-dc14`) is
unaffected for this role — the datacenter name is a distinct concept
from location and is kept as-is (this will stop working starting
2026-07-01)
**`robot` role:**
- Add `__meta_hetzner_robot_datacenter` as the canonical replacement for
`__meta_hetzner_datacenter`; the old label is kept for backward
compatibility
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10909
At v1.142.0 was introduced a bug, when changes from OSS version were
back-ported into Enterprise branch. It changed the order of storage
nodes discovery. And resulted into:
* overwrite of discovered storage nodes
* duplicate of per storage node metrics
This bug only affects enterprise vminsert version.
Mention -rule.stripFilePath cmd-limne flag in security recommendations,
so users can be aware of it.
---------
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Haley Wang <haley@victoriametrics.com>
The change adds `AI observability` section to `AI tools` documentation.
It mentions excellent @Amper articles describing these integrations in
all details.
The doc change doesn't repeat the articles, but rather helps users to
discover them.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
