rms/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2026-05-17 00:26:36 +03:00
Code Issues Packages Projects Releases Wiki Activity

dashboards: add metadata ingestion row rate queries to vmagent&vmcluster dashboards (#10868)

Browse Source 
Metadata is enabled by default since v1.137.0, and the metadata volume
can be a big contributor to resource usage and network traffic.

vmagent dahsboard:
1. `Troubleshooting` section: rename `Datapoints rate` panel to `Rows
rate` to include metadata rate;
2. `Ingestion` section: add metadata rate to existing `Rows rate` panel.
(The difference between this panel and the one above is that this panel
only contains data from write requests, while the above panel also
includes the scraping part.)


vmcluster dashboard:
1. `vminsert` section: add `Rows rate` panel

Didn’t see a good place for it in the vmsingle dashboard, since it
doesn’t have a dedicated insert section, and I don’t want to add it to
`overview` yet.

https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10868
This commit is contained in:
Hui Wang
2026-04-24 19:07:31 +08:00
committed by GitHub
parent 2e7591d567
commit 66b9890025
5 changed files with 785 additions and 378 deletions
Download Patch File Download Diff File Expand all files Collapse all files

529
dashboards/victoriametrics-cluster.json
View File

@@ -9093,18 +9093,20 @@
"type": "prometheus",
"uid": "$ds"
},
"description": "* `*` - unsupported query path\n* `/write` - insert into VM\n* `/metrics` - query VM system metrics\n* `/query` - query instant values\n* `/query_range` - query over a range of time\n* `/series` - match a certain label set\n* `/label/{}/values` - query a list of label values (variables mostly)",
"description": "Shows the rate of data rows ingested from write requests. There are two kinds of rows:\n1. Raw sample: each sample consists of a value and a timestamp, see https://docs.victoriametrics.com/victoriametrics/keyconcepts/#raw-samples. \n2. Metric metadata: refers to descriptive information about metrics. It can be disabled by setting `-enableMetadata=false`, see https://docs.victoriametrics.com/victoriametrics/vmagent/#metric-metadata.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -9113,6 +9115,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -9120,6 +9123,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -9129,6 +9133,7 @@
"mode": "off"
}
},
"decimals": 2,
"links": [],
"mappings": [],
"min": 0,
@@ -9136,7 +9141,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -9152,9 +9158,9 @@
"h": 8,
"w": 12,
"x": 0,
"y": 8385
"y": 43
},
"id": 97,
"id": 227,
"options": {
"legend": {
"calcs": [
@@ -9169,11 +9175,12 @@
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -9181,15 +9188,30 @@
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vm_http_requests_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])) by (path) > 0",
"expr": "sum(rate(vm_rows_inserted_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])) by (type) > 0 ",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "__auto",
"legendFormat": "sample: {{type}}",
"range": true,
"refId": "A"
},
{
"datasource": {
"type": "prometheus",
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vm_metadata_rows_inserted_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])) by (type) > 0 ",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "metadata: {{type}}",
"range": true,
"refId": "B"
}
],
"title": "Requests rate ($instance)",
"title": "Rows rate ($instance)",
"type": "timeseries"
},
{
@@ -9204,11 +9226,13 @@
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
@@ -9217,6 +9241,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -9224,6 +9249,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -9241,7 +9267,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -9277,7 +9304,7 @@
"h": 8,
"w": 12,
"x": 12,
"y": 8385
"y": 43
},
"id": 99,
"options": {
@@ -9294,11 +9321,12 @@
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -9336,17 +9364,20 @@
"type": "prometheus",
"uid": "$ds"
},
"description": "* `*` - unsupported query path\n* `/write` - insert into VM\n* `/metrics` - query VM system metrics\n* `/query` - query instant values\n* `/query_range` - query over a range of time\n* `/series` - match a certain label set\n* `/label/{}/values` - query a list of label values (variables mostly)",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -9355,6 +9386,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -9362,6 +9394,116 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"links": [],
"mappings": [],
"min": 0,
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
},
{
"color": "red",
"value": 80
}
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 51
},
"id": 97,
"options": {
"legend": {
"calcs": [
"mean",
"lastNotNull",
"max"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true,
"sortBy": "Last *",
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vm_http_requests_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])) by (path) > 0",
"format": "time_series",
"intervalFactor": 1,
"legendFormat": "__auto",
"range": true,
"refId": "A"
}
],
"title": "Requests rate ($instance)",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "$ds"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -9384,7 +9526,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -9399,8 +9542,8 @@
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 8393
"x": 12,
"y": 51
},
"id": 185,
"options": {
@@ -9417,11 +9560,12 @@
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "none"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -9447,7 +9591,6 @@
"exemplar": true,
"expr": "min(\n rate(process_cpu_seconds_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])\n /\n process_cpu_cores_available{job=~\"$job_insert\", instance=~\"$instance\"}\n)",
"format": "time_series",
"hide": false,
"interval": "",
"intervalFactor": 1,
"legendFormat": "min",
@@ -9463,7 +9606,6 @@
"exemplar": true,
"expr": "quantile(0.5,\n rate(process_cpu_seconds_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])\n /\n process_cpu_cores_available{job=~\"$job_insert\", instance=~\"$instance\"}\n)",
"format": "time_series",
"hide": false,
"interval": "",
"intervalFactor": 1,
"legendFormat": "median",
@@ -9486,11 +9628,13 @@
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -9499,6 +9643,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -9506,6 +9651,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -9528,7 +9674,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -9543,8 +9690,8 @@
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 8393
"x": 0,
"y": 59
},
"id": 187,
"options": {
@@ -9561,11 +9708,12 @@
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "none"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -9591,7 +9739,6 @@
"exemplar": true,
"expr": "min(\n max_over_time(process_resident_memory_anon_bytes{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])\n /\n vm_available_memory_bytes{job=~\"$job_insert\", instance=~\"$instance\"}\n)",
"format": "time_series",
"hide": false,
"interval": "",
"intervalFactor": 1,
"legendFormat": "min",
@@ -9607,7 +9754,6 @@
"exemplar": true,
"expr": "quantile(0.5,\n max_over_time(process_resident_memory_anon_bytes{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])\n /\n vm_available_memory_bytes{job=~\"$job_insert\", instance=~\"$instance\"}\n)",
"format": "time_series",
"hide": false,
"interval": "",
"intervalFactor": 1,
"legendFormat": "median",
@@ -9623,18 +9769,20 @@
"type": "prometheus",
"uid": "$ds"
},
"description": "Shows the saturation level of connection between vminsert and vmstorage components. \n\nIf the threshold of 0.9sec is reached, then the connection is saturated by more than 90% and vminsert won't be able to keep up. This usually means that either vminsert or vmstorage nodes are struggling with the load. Verify CPU/mem saturation of both components and network saturation between them.\nIf vminsert resources are saturated - consider adding more resources or scale vminserts horizontally.\n\nIf vminsert resources and network are fine, check vmstorage metrics for anomalies.",
"description": "Shows when vmstorage node is unreachable for vminsert.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -9643,6 +9791,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -9650,6 +9799,116 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"decimals": 0,
"links": [],
"mappings": [],
"min": 0,
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
},
{
"color": "red",
"value": 80
}
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 59
},
"id": 114,
"options": {
"legend": {
"calcs": [
"mean",
"lastNotNull"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "$ds"
},
"editorMode": "code",
"expr": "vm_rpc_vmstorage_is_reachable{job=~\"$job\", instance=~\"$instance\"} != 1",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "{{instance}} => {{addr}}",
"range": true,
"refId": "A"
}
],
"title": "Storage reachability ($instance)",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "$ds"
},
"description": "Shows the saturation level of connection between vminsert and vmstorage components. \n\nIf the threshold of 0.9sec is reached, then the connection is saturated by more than 90% and vminsert won't be able to keep up. This usually means that either vminsert or vmstorage nodes are struggling with the load. Verify CPU/mem saturation of both components and network saturation between them.\nIf vminsert resources are saturated - consider adding more resources or scale vminserts horizontally.\n\nIf vminsert resources and network are fine, check vmstorage metrics for anomalies.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -9667,7 +9926,8 @@
"mode": "absolute",
"steps": [
{
"color": "transparent"
"color": "transparent",
"value": 0
},
{
"color": "red",
@@ -9683,7 +9943,7 @@
"h": 8,
"w": 12,
"x": 0,
"y": 8401
"y": 67
},
"id": 139,
"options": {
@@ -9699,11 +9959,12 @@
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -9729,18 +9990,20 @@
"type": "prometheus",
"uid": "$ds"
},
"description": "Shows when vmstorage node is unreachable for vminsert.",
"description": "Shows network usage between vminserts and vmstorages.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -9749,6 +10012,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -9756,6 +10020,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -9765,15 +10030,14 @@
"mode": "off"
}
},
"decimals": 0,
"links": [],
"mappings": [],
"min": 0,
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -9781,17 +10045,30 @@
}
]
},
"unit": "short"
"unit": "bps"
},
"overrides": []
"overrides": [
{
"matcher": {
"id": "byRegexp",
"options": "/read.*/"
},
"properties": [
{
"id": "custom.transform",
"value": "negative-Y"
}
]
}
]
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 8401
"y": 67
},
"id": 114,
"id": 209,
"options": {
"legend": {
"calcs": [
@@ -9800,14 +10077,17 @@
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
"showLegend": true,
"sortBy": "Last *",
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -9815,16 +10095,28 @@
"uid": "$ds"
},
"editorMode": "code",
"expr": "vm_rpc_vmstorage_is_reachable{job=~\"$job\", instance=~\"$instance\"} != 1",
"expr": "sum(rate(vm_tcpdialer_read_bytes_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])) * 8 > 0",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "{{instance}} => {{addr}}",
"legendFormat": "read from vmstorage",
"range": true,
"refId": "A"
},
{
"datasource": {
"type": "prometheus",
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vm_tcpdialer_written_bytes_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])) * 8 > 0",
"format": "time_series",
"intervalFactor": 1,
"legendFormat": "write to vmstorage",
"range": true,
"refId": "B"
}
],
"title": "Storage reachability ($instance)",
"title": "Network usage: vmstorage ($instance)",
"type": "timeseries"
},
{
@@ -9839,11 +10131,13 @@
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -9852,6 +10146,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -9859,6 +10154,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -9874,7 +10170,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -9903,7 +10200,7 @@
"h": 8,
"w": 12,
"x": 0,
"y": 8409
"y": 75
},
"id": 208,
"options": {
@@ -9919,11 +10216,12 @@
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -9946,7 +10244,6 @@
"editorMode": "code",
"expr": "sum(rate(vm_tcplistener_written_bytes_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])) * 8 > 0",
"format": "time_series",
"hide": false,
"intervalFactor": 1,
"legendFormat": "write to client",
"range": true,
@@ -9961,147 +10258,19 @@
"type": "prometheus",
"uid": "$ds"
},
"description": "Shows network usage between vminserts and vmstorages.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"links": [],
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green"
},
{
"color": "red",
"value": 80
}
]
},
"unit": "bps"
},
"overrides": [
{
"matcher": {
"id": "byRegexp",
"options": "/read.*/"
},
"properties": [
{
"id": "custom.transform",
"value": "negative-Y"
}
]
}
]
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 8409
},
"id": 209,
"options": {
"legend": {
"calcs": [
"mean",
"lastNotNull"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true,
"sortBy": "Last *",
"sortDesc": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.1.0",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vm_tcpdialer_read_bytes_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])) * 8 > 0",
"format": "time_series",
"intervalFactor": 1,
"legendFormat": "read from vmstorage",
"range": true,
"refId": "A"
},
{
"datasource": {
"type": "prometheus",
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vm_tcpdialer_written_bytes_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])) * 8 > 0",
"format": "time_series",
"hide": false,
"intervalFactor": 1,
"legendFormat": "write to vmstorage",
"range": true,
"refId": "B"
}
],
"title": "Network usage: vmstorage ($instance)",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "$ds"
},
"description": "",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -10110,6 +10279,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -10117,6 +10287,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -10134,7 +10305,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -10150,7 +10322,7 @@
"h": 8,
"w": 12,
"x": 12,
"y": 8417
"y": 75
},
"id": 88,
"options": {
@@ -10167,11 +10339,12 @@
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {

529
dashboards/vm/victoriametrics-cluster.json
View File

@@ -9094,18 +9094,20 @@
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"description": "* `*` - unsupported query path\n* `/write` - insert into VM\n* `/metrics` - query VM system metrics\n* `/query` - query instant values\n* `/query_range` - query over a range of time\n* `/series` - match a certain label set\n* `/label/{}/values` - query a list of label values (variables mostly)",
"description": "Shows the rate of data rows ingested from write requests. There are two kinds of rows:\n1. Raw sample: each sample consists of a value and a timestamp, see https://docs.victoriametrics.com/victoriametrics/keyconcepts/#raw-samples. \n2. Metric metadata: refers to descriptive information about metrics. It can be disabled by setting `-enableMetadata=false`, see https://docs.victoriametrics.com/victoriametrics/vmagent/#metric-metadata.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -9114,6 +9116,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -9121,6 +9124,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -9130,6 +9134,7 @@
"mode": "off"
}
},
"decimals": 2,
"links": [],
"mappings": [],
"min": 0,
@@ -9137,7 +9142,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -9153,9 +9159,9 @@
"h": 8,
"w": 12,
"x": 0,
"y": 8385
"y": 43
},
"id": 97,
"id": 227,
"options": {
"legend": {
"calcs": [
@@ -9170,11 +9176,12 @@
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -9182,15 +9189,30 @@
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vm_http_requests_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])) by (path) > 0",
"expr": "sum(rate(vm_rows_inserted_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])) by (type) > 0 ",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "__auto",
"legendFormat": "sample: {{type}}",
"range": true,
"refId": "A"
},
{
"datasource": {
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vm_metadata_rows_inserted_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])) by (type) > 0 ",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "metadata: {{type}}",
"range": true,
"refId": "B"
}
],
"title": "Requests rate ($instance)",
"title": "Rows rate ($instance)",
"type": "timeseries"
},
{
@@ -9205,11 +9227,13 @@
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
@@ -9218,6 +9242,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -9225,6 +9250,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -9242,7 +9268,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -9278,7 +9305,7 @@
"h": 8,
"w": 12,
"x": 12,
"y": 8385
"y": 43
},
"id": 99,
"options": {
@@ -9295,11 +9322,12 @@
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -9337,17 +9365,20 @@
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"description": "* `*` - unsupported query path\n* `/write` - insert into VM\n* `/metrics` - query VM system metrics\n* `/query` - query instant values\n* `/query_range` - query over a range of time\n* `/series` - match a certain label set\n* `/label/{}/values` - query a list of label values (variables mostly)",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -9356,6 +9387,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -9363,6 +9395,116 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"links": [],
"mappings": [],
"min": 0,
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
},
{
"color": "red",
"value": 80
}
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 51
},
"id": 97,
"options": {
"legend": {
"calcs": [
"mean",
"lastNotNull",
"max"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true,
"sortBy": "Last *",
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vm_http_requests_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])) by (path) > 0",
"format": "time_series",
"intervalFactor": 1,
"legendFormat": "__auto",
"range": true,
"refId": "A"
}
],
"title": "Requests rate ($instance)",
"type": "timeseries"
},
{
"datasource": {
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -9385,7 +9527,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -9400,8 +9543,8 @@
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 8393
"x": 12,
"y": 51
},
"id": 185,
"options": {
@@ -9418,11 +9561,12 @@
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "none"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -9448,7 +9592,6 @@
"exemplar": true,
"expr": "min(\n rate(process_cpu_seconds_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])\n /\n process_cpu_cores_available{job=~\"$job_insert\", instance=~\"$instance\"}\n)",
"format": "time_series",
"hide": false,
"interval": "",
"intervalFactor": 1,
"legendFormat": "min",
@@ -9464,7 +9607,6 @@
"exemplar": true,
"expr": "quantile(0.5,\n rate(process_cpu_seconds_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])\n /\n process_cpu_cores_available{job=~\"$job_insert\", instance=~\"$instance\"}\n)",
"format": "time_series",
"hide": false,
"interval": "",
"intervalFactor": 1,
"legendFormat": "median",
@@ -9487,11 +9629,13 @@
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -9500,6 +9644,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -9507,6 +9652,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -9529,7 +9675,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -9544,8 +9691,8 @@
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 8393
"x": 0,
"y": 59
},
"id": 187,
"options": {
@@ -9562,11 +9709,12 @@
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "none"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -9592,7 +9740,6 @@
"exemplar": true,
"expr": "min(\n max_over_time(process_resident_memory_anon_bytes{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])\n /\n vm_available_memory_bytes{job=~\"$job_insert\", instance=~\"$instance\"}\n)",
"format": "time_series",
"hide": false,
"interval": "",
"intervalFactor": 1,
"legendFormat": "min",
@@ -9608,7 +9755,6 @@
"exemplar": true,
"expr": "quantile(0.5,\n max_over_time(process_resident_memory_anon_bytes{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])\n /\n vm_available_memory_bytes{job=~\"$job_insert\", instance=~\"$instance\"}\n)",
"format": "time_series",
"hide": false,
"interval": "",
"intervalFactor": 1,
"legendFormat": "median",
@@ -9624,18 +9770,20 @@
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"description": "Shows the saturation level of connection between vminsert and vmstorage components. \n\nIf the threshold of 0.9sec is reached, then the connection is saturated by more than 90% and vminsert won't be able to keep up. This usually means that either vminsert or vmstorage nodes are struggling with the load. Verify CPU/mem saturation of both components and network saturation between them.\nIf vminsert resources are saturated - consider adding more resources or scale vminserts horizontally.\n\nIf vminsert resources and network are fine, check vmstorage metrics for anomalies.",
"description": "Shows when vmstorage node is unreachable for vminsert.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -9644,6 +9792,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -9651,6 +9800,116 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"decimals": 0,
"links": [],
"mappings": [],
"min": 0,
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
},
{
"color": "red",
"value": 80
}
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 59
},
"id": 114,
"options": {
"legend": {
"calcs": [
"mean",
"lastNotNull"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"editorMode": "code",
"expr": "vm_rpc_vmstorage_is_reachable{job=~\"$job\", instance=~\"$instance\"} != 1",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "{{instance}} => {{addr}}",
"range": true,
"refId": "A"
}
],
"title": "Storage reachability ($instance)",
"type": "timeseries"
},
{
"datasource": {
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"description": "Shows the saturation level of connection between vminsert and vmstorage components. \n\nIf the threshold of 0.9sec is reached, then the connection is saturated by more than 90% and vminsert won't be able to keep up. This usually means that either vminsert or vmstorage nodes are struggling with the load. Verify CPU/mem saturation of both components and network saturation between them.\nIf vminsert resources are saturated - consider adding more resources or scale vminserts horizontally.\n\nIf vminsert resources and network are fine, check vmstorage metrics for anomalies.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -9668,7 +9927,8 @@
"mode": "absolute",
"steps": [
{
"color": "transparent"
"color": "transparent",
"value": 0
},
{
"color": "red",
@@ -9684,7 +9944,7 @@
"h": 8,
"w": 12,
"x": 0,
"y": 8401
"y": 67
},
"id": 139,
"options": {
@@ -9700,11 +9960,12 @@
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -9730,18 +9991,20 @@
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"description": "Shows when vmstorage node is unreachable for vminsert.",
"description": "Shows network usage between vminserts and vmstorages.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -9750,6 +10013,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -9757,6 +10021,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -9766,15 +10031,14 @@
"mode": "off"
}
},
"decimals": 0,
"links": [],
"mappings": [],
"min": 0,
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -9782,17 +10046,30 @@
}
]
},
"unit": "short"
"unit": "bps"
},
"overrides": []
"overrides": [
{
"matcher": {
"id": "byRegexp",
"options": "/read.*/"
},
"properties": [
{
"id": "custom.transform",
"value": "negative-Y"
}
]
}
]
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 8401
"y": 67
},
"id": 114,
"id": 209,
"options": {
"legend": {
"calcs": [
@@ -9801,14 +10078,17 @@
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
"showLegend": true,
"sortBy": "Last *",
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -9816,16 +10096,28 @@
"uid": "$ds"
},
"editorMode": "code",
"expr": "vm_rpc_vmstorage_is_reachable{job=~\"$job\", instance=~\"$instance\"} != 1",
"expr": "sum(rate(vm_tcpdialer_read_bytes_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])) * 8 > 0",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "{{instance}} => {{addr}}",
"legendFormat": "read from vmstorage",
"range": true,
"refId": "A"
},
{
"datasource": {
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vm_tcpdialer_written_bytes_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])) * 8 > 0",
"format": "time_series",
"intervalFactor": 1,
"legendFormat": "write to vmstorage",
"range": true,
"refId": "B"
}
],
"title": "Storage reachability ($instance)",
"title": "Network usage: vmstorage ($instance)",
"type": "timeseries"
},
{
@@ -9840,11 +10132,13 @@
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -9853,6 +10147,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -9860,6 +10155,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -9875,7 +10171,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -9904,7 +10201,7 @@
"h": 8,
"w": 12,
"x": 0,
"y": 8409
"y": 75
},
"id": 208,
"options": {
@@ -9920,11 +10217,12 @@
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -9947,7 +10245,6 @@
"editorMode": "code",
"expr": "sum(rate(vm_tcplistener_written_bytes_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])) * 8 > 0",
"format": "time_series",
"hide": false,
"intervalFactor": 1,
"legendFormat": "write to client",
"range": true,
@@ -9962,147 +10259,19 @@
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"description": "Shows network usage between vminserts and vmstorages.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"links": [],
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green"
},
{
"color": "red",
"value": 80
}
]
},
"unit": "bps"
},
"overrides": [
{
"matcher": {
"id": "byRegexp",
"options": "/read.*/"
},
"properties": [
{
"id": "custom.transform",
"value": "negative-Y"
}
]
}
]
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 8409
},
"id": 209,
"options": {
"legend": {
"calcs": [
"mean",
"lastNotNull"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true,
"sortBy": "Last *",
"sortDesc": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.1.0",
"targets": [
{
"datasource": {
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vm_tcpdialer_read_bytes_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])) * 8 > 0",
"format": "time_series",
"intervalFactor": 1,
"legendFormat": "read from vmstorage",
"range": true,
"refId": "A"
},
{
"datasource": {
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vm_tcpdialer_written_bytes_total{job=~\"$job_insert\", instance=~\"$instance\"}[$__rate_interval])) * 8 > 0",
"format": "time_series",
"hide": false,
"intervalFactor": 1,
"legendFormat": "write to vmstorage",
"range": true,
"refId": "B"
}
],
"title": "Network usage: vmstorage ($instance)",
"type": "timeseries"
},
{
"datasource": {
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"description": "",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -10111,6 +10280,7 @@
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
@@ -10118,6 +10288,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -10135,7 +10306,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -10151,7 +10323,7 @@
"h": 8,
"w": 12,
"x": 12,
"y": 8417
"y": 75
},
"id": 88,
"options": {
@@ -10168,11 +10340,12 @@
"sortDesc": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.1.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {

52
dashboards/vm/vmagent.json
View File

@@ -4020,7 +4020,7 @@
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"description": "Shows the rate of parsed datapoints from write or scrape requests.",
"description": "Shows the rate of data rows parsed from write requests and scraping. \nThere are two kinds of rows:\n1. Raw sample: each sample consists of a value and a timestamp, see https://docs.victoriametrics.com/victoriametrics/keyconcepts/#raw-samples \n2. Metric metadata: refers to descriptive information about metrics. It can be disabled by setting `-enableMetadata=false`, see https://docs.victoriametrics.com/victoriametrics/vmagent/#metric-metadata.",
"fieldConfig": {
"defaults": {
"color": {
@@ -4050,6 +4050,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -4066,7 +4067,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -4102,7 +4104,7 @@
"sort": "desc"
}
},
"pluginVersion": "11.5.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -4113,12 +4115,24 @@
"exemplar": true,
"expr": "sum(rate(vm_protoparser_rows_read_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])) by(job, type) > 0",
"interval": "",
"legendFormat": "{{ type }} ({{job}})",
"legendFormat": "sample: {{ type }} ({{job}})",
"range": true,
"refId": "A"
},
{
"datasource": {
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vm_protoparser_metadata_read_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])) by(job, type) > 0",
"instant": false,
"legendFormat": "metadata: {{ type }} ({{job}})",
"range": true,
"refId": "B"
}
],
"title": "Datapoints rate ($instance)",
"title": "Rows rate ($instance)",
"type": "timeseries"
},
{
@@ -5722,7 +5736,7 @@
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"description": "Shows the rate of rows ingested in vmagent via push protocols.",
"description": "Shows the rate of data rows ingested from write requests. There are two kinds of rows:\n1. Raw sample: each sample consists of a value and a timestamp, see https://docs.victoriametrics.com/victoriametrics/keyconcepts/#raw-samples. \n2. Metric metadata: refers to descriptive information about metrics. It can be disabled by setting `-enableMetadata=false`, see https://docs.victoriametrics.com/victoriametrics/vmagent/#metric-metadata.",
"fieldConfig": {
"defaults": {
"color": {
@@ -5735,6 +5749,7 @@
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -5751,6 +5766,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -5767,7 +5783,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -5783,7 +5800,7 @@
"h": 8,
"w": 12,
"x": 12,
"y": 3931
"y": 41
},
"id": 131,
"options": {
@@ -5798,11 +5815,12 @@
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.2.6",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -5813,12 +5831,24 @@
"exemplar": true,
"expr": "sum(rate(vmagent_rows_inserted_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])) by(job, type) > 0",
"interval": "",
"legendFormat": "{{ type }} ({{job}})",
"legendFormat": "sample: {{ type }} ({{job}})",
"range": true,
"refId": "A"
},
{
"datasource": {
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vmagent_metadata_inserted_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])) by(job, type) > 0",
"instant": false,
"legendFormat": "metadata:{{ type }} ({{job}})",
"range": true,
"refId": "B"
}
],
"title": "Rows rate ($instance)",
"title": "Rows rate ($instance)",
"type": "timeseries"
},
{

52
dashboards/vmagent.json
View File

@@ -4019,7 +4019,7 @@
"type": "prometheus",
"uid": "$ds"
},
"description": "Shows the rate of parsed datapoints from write or scrape requests.",
"description": "Shows the rate of data rows parsed from write requests and scraping. \nThere are two kinds of rows:\n1. Raw sample: each sample consists of a value and a timestamp, see https://docs.victoriametrics.com/victoriametrics/keyconcepts/#raw-samples \n2. Metric metadata: refers to descriptive information about metrics. It can be disabled by setting `-enableMetadata=false`, see https://docs.victoriametrics.com/victoriametrics/vmagent/#metric-metadata.",
"fieldConfig": {
"defaults": {
"color": {
@@ -4049,6 +4049,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -4065,7 +4066,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -4101,7 +4103,7 @@
"sort": "desc"
}
},
"pluginVersion": "11.5.0",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -4112,12 +4114,24 @@
"exemplar": true,
"expr": "sum(rate(vm_protoparser_rows_read_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])) by(job, type) > 0",
"interval": "",
"legendFormat": "{{ type }} ({{job}})",
"legendFormat": "sample: {{ type }} ({{job}})",
"range": true,
"refId": "A"
},
{
"datasource": {
"type": "prometheus",
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vm_protoparser_metadata_read_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])) by(job, type) > 0",
"instant": false,
"legendFormat": "metadata: {{ type }} ({{job}})",
"range": true,
"refId": "B"
}
],
"title": "Datapoints rate ($instance)",
"title": "Rows rate ($instance)",
"type": "timeseries"
},
{
@@ -5721,7 +5735,7 @@
"type": "prometheus",
"uid": "$ds"
},
"description": "Shows the rate of rows ingested in vmagent via push protocols.",
"description": "Shows the rate of data rows ingested from write requests. There are two kinds of rows:\n1. Raw sample: each sample consists of a value and a timestamp, see https://docs.victoriametrics.com/victoriametrics/keyconcepts/#raw-samples. \n2. Metric metadata: refers to descriptive information about metrics. It can be disabled by setting `-enableMetadata=false`, see https://docs.victoriametrics.com/victoriametrics/vmagent/#metric-metadata.",
"fieldConfig": {
"defaults": {
"color": {
@@ -5734,6 +5748,7 @@
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
@@ -5750,6 +5765,7 @@
"type": "linear"
},
"showPoints": "never",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
@@ -5766,7 +5782,8 @@
"mode": "absolute",
"steps": [
{
"color": "green"
"color": "green",
"value": 0
},
{
"color": "red",
@@ -5782,7 +5799,7 @@
"h": 8,
"w": 12,
"x": 12,
"y": 3931
"y": 41
},
"id": 131,
"options": {
@@ -5797,11 +5814,12 @@
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "9.2.6",
"pluginVersion": "12.4.3",
"targets": [
{
"datasource": {
@@ -5812,12 +5830,24 @@
"exemplar": true,
"expr": "sum(rate(vmagent_rows_inserted_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])) by(job, type) > 0",
"interval": "",
"legendFormat": "{{ type }} ({{job}})",
"legendFormat": "sample: {{ type }} ({{job}})",
"range": true,
"refId": "A"
},
{
"datasource": {
"type": "prometheus",
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(vmagent_metadata_inserted_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])) by(job, type) > 0",
"instant": false,
"legendFormat": "metadata:{{ type }} ({{job}})",
"range": true,
"refId": "B"
}
],
"title": "Rows rate ($instance)",
"title": "Rows rate ($instance)",
"type": "timeseries"
},
{

1
docs/victoriametrics/changelog/CHANGELOG.md
View File

@@ -32,6 +32,7 @@ See also [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-rel
* FEATURE: [alerts](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/rules): add new `MetricNameStatsCacheUtilizationIsTooHigh` alerting rule to track overutilization of [Metric names usage stats tracker](https://docs.victoriametrics.com/victoriametrics/#track-ingested-metrics-usage) (used in [Cardinality Explorer](https://docs.victoriametrics.com/victoriametrics/#cardinality-explorer)). See [#10840](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10840).
* FEATURE: [stream aggregation](https://docs.victoriametrics.com/victoriametrics/stream-aggregation/): add `vm_streamaggr_counter_resets_total` metric for `total*`, `increase*` and `rate*` outputs that is useful for aggregation behaviour tracking. These metrics help to identify issues described in [Troubleshooting: counter resets](https://docs.victoriametrics.com/victoriametrics/stream-aggregation/#counter-resets).
* FEATURE: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmstorage` and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): add the support of ingestion and retrieval of samples with timestamps in the future. The new `-futureRetention` flag controls how far in the future the timestamps are allowed to be. See [827](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/827) and [10718](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10718).
* FEATURE: [dashboards/vmagent](https://grafana.com/grafana/dashboards/12683), [dashboards/cluster](https://grafana.com/grafana/dashboards/11176): add metadata row ingestion rate queries to related panels. See [#10868](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10868).
* BUGFIX: `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): fix increased memory usage after upgrade to [v1.140.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.140.0) by properly accounting for internal buffer count when calculating per-storage buffer size. See [#10725](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10725#issuecomment-4282256709).
* BUGFIX: all VictoriaMetrics components: properly parse IPv6 source address when accepting connections with proxy protocol v2 enabled. See [#10839](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10839). Thanks to @andriibeee for the contribution.