rms/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2026-05-17 00:26:36 +03:00
Code Issues Packages Projects Releases Wiki Activity

docs: structurize multitenancy doc for vmagent (#10943)

Browse Source 
This change should clearly distinguish different multitnenacy scenarios
for vmagent. It is expected to be easier to read and follow for users.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Pablo Fernandez <46322567+TomFern@users.noreply.github.com>
(cherry picked from commit 5f5a2109e8)
This commit is contained in:
Roman Khavronenko
2026-05-13 13:07:33 +02:00
committed by hagen1778
parent 7eba836ec2
commit de3e79aba6
1 changed files with 61 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

80
docs/victoriametrics/vmagent.md
View File

@@ -483,15 +483,29 @@ by specifying `-remoteWrite.forcePromProto` command-line flag for the correspond
By default `vmagent` collects the data without [tenant](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy) identifiers
and routes it to the remote storage specified via `-remoteWrite.url` command-line flag. The `-remoteWrite.url` can point to `/insert/<tenant_id>/prometheus/api/v1/write` path
at `vminsert` according to [these docs](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#url-format). In this case all the metrics are written to the given `<tenant_id>` tenant.
at `vminsert` according to [these docs](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#url-format).
```mermaid
flowchart LR
A["requests_total{instance=foo}"] --> |/api/v1/write| V[vmagent]
B["requests_total{instance=bar}"] <--> |scrape| V
V --> |"/insert/#60;tenant_id#62;/#60;suffix#62;"| C[vminsert]
```
In this case, all the metrics written to `/insert/tenant_id/prometheus/api/v1/write` will belong to specified `<tenant_id>` tenant.
### Multitenancy via labels
vmagent can write data to multiple distinct tenants if `-remoteWrite.url` points to [multitenant url at VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy-via-labels)
and tenant is specified via [multitenancy labels](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy-via-labels):
```mermaid
flowchart LR
A["requests_total{instance=foo, vm_account_id=0}"] --> |/api/v1/write| V[vmagent]
B["requests_total{instance=bar, vm_account_id=1}"] <--> |scrape| V
V --> |"/insert/multitenant/#60;suffix#62;"| C[vminsert]
```
`<tenant_id>` is extracted from the `vm_account_id` and `vm_project_id` labels.
The easiest way to write data to multiple distinct tenants is to specify the needed tenants via `vm_account_id` and `vm_project_id` labels
and then to push metrics with these labels to [multitenant url at VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy-via-labels).
The `vm_account_id` and `vm_project_id` labels can be specified via [relabeling](https://docs.victoriametrics.com/victoriametrics/relabeling/) before sending the metrics to `-remoteWrite.url`.
For example, the following relabeling rule instructs sending metrics to `<account_id>:0` [tenant](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy)
defined in the `prometheus.io/account_id` annotation of Kubernetes pod deployment:
For example, the following relabeling rule instructs sending metrics to `<account_id>:0` tenant defined in the `prometheus.io/account_id` annotation of Kubernetes pod deployment:
```yaml
scrape_configs:
- kubernetes_sd_configs:
@@ -501,12 +515,10 @@ scrape_configs:
target_label: vm_account_id
```
In addition, vmagent could obtain tenant identifier from `__tenant_id__` label at target discovery phase.
vmagent can get tenant identifier from `__tenant_id__` label at target discovery phase.
It implicitly converts `__tenant_id__` label into `vm_account_id` and `vm_project_id` labels and attaches
it to the scraped metrics and metrics metadata.
For example, the following relabeling rule instructs sending metrics to `10:5` [tenant](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy)
defined in the `prometheus.io/tenant_id: 10:5` annotation of Kubernetes pod deployment:
For example, the following relabeling rule instructs sending metrics to `10:5` tenant defined in the `prometheus.io/tenant_id: 10:5` annotation of Kubernetes pod deployment:
```yaml
scrape_configs:
- kubernetes_sd_configs:
@@ -516,15 +528,45 @@ scrape_configs:
target_label: __tenant_id__
```
`vmagent` can accept data via the same multitenant endpoints (`/insert/<accountID>/<suffix>`) as `vminsert` at [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/)
does according to [these docs](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#url-format) if `-enableMultitenantHandlers` command-line flag is set.
In this case, vmagent automatically converts tenant identifiers from the URL or headers to `vm_account_id` and `vm_project_id` labels and sets tenant info in metadata.
These tenant labels are added before applying [relabeling](https://docs.victoriametrics.com/victoriametrics/relabeling/) specified via `-remoteWrite.relabelConfig`
and `-remoteWrite.urlRelabelConfig` command-line flags. Metrics with `vm_account_id` and `vm_project_id` labels can be routed to the corresponding tenants
when specifying `-remoteWrite.url` to [multitenant url at VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy-via-labels).
vmagent can [enforce adding labels](https://docs.victoriametrics.com/victoriametrics/vmagent/#adding-labels-to-metrics) to all scraped
or forwarded metrics.
`vmagent` can accept tenant IDs specified via HTTP headers if both `-enableMultitenantHandlers` and `-enableMultitenancyViaHeaders` command-line flags are set.
See more about [multitenancy via headers](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy-via-headers).
### Multitenancy via path
vmagent can write data to multiple distinct tenants if `-remoteWrite.url` points to [multitenant url at VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy-via-labels),
tenant is specified in the [write path](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#url-format) and `-enableMultitenantHandlers` command-line flag is set:
```mermaid
flowchart LR
A["requests_total{instance=foo}"] --> |/insert/0/#60;suffix#62;| V[vmagent]
B["requests_total{instance=bar}"] --> |/insert/1/#60;suffix#62;| V
V --> |"/insert/multitenant/#60;suffix#62;"| C[vminsert]
```
In this configuration, vmagent accepts writes via the same multitenant endpoints (`/insert/<accountID>/<suffix>`) [as vminsert](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#url-format).
For all received data, vmagent will automatically convert tenant identifiers from the URL to `vm_account_id` and `vm_project_id` labels and sets tenant info in metadata.
These tenant labels are added before applying [relabeling](https://docs.victoriametrics.com/victoriametrics/relabeling/) specified via `-remoteWrite.relabelConfig`
and `-remoteWrite.urlRelabelConfig` command-line flags.
### Multitenancy via headers
vmagent can write data to multiple distinct tenants if `-remoteWrite.url` points to [multitenant url at VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy-via-labels),
tenant is specified [via headers](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy-via-headers) {{% available_from "v1.143.0" %}}, both `-enableMultitenantHandlers` and `-enableMultitenancyViaHeaders` command-line flags are set:
```mermaid
flowchart LR
A["requests_total{instance=foo}"] --> |/insert/#60;suffix#62; <br>--header AccountID: 0| V[vmagent]
B["requests_total{instance=bar}"] --> |/insert/#60;suffix#62; <br>--header AccountID: 1| V
V --> |"/insert/multitenant/#60;suffix#62;"| C[vminsert]
```
In this configuration, vmagent accepts writes via the same simplified multitenant endpoints (`/insert/<suffix>`) [as vminsert](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#url-format).
The tenant information is extracted from the `AccountID` and `ProjectID` HTTP headers, that are expected to be attached to all the incoming requests. If headers are missing, then tenant is set to `0:0` as default.
For all received data, vmagent will automatically convert tenant identifiers from the headers to `vm_account_id` and `vm_project_id` labels and sets tenant info in metadata.
These tenant labels are added before applying [relabeling](https://docs.victoriametrics.com/victoriametrics/relabeling/) specified via `-remoteWrite.relabelConfig`
and `-remoteWrite.urlRelabelConfig` command-line flags.
vmauth can [enforce adding headers](https://docs.victoriametrics.com/victoriametrics/vmauth/#modifying-http-headers) to all
forwarded requests via `headers` param in the config file.
## Adding labels to metrics