mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2026-07-11 11:27:17 +03:00
Compare commits
1 Commits
VMSelectCo
...
RequestErr
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2edf1729d3 |
@@ -18,7 +18,7 @@ groups:
|
|||||||
concurrency: 2
|
concurrency: 2
|
||||||
rules:
|
rules:
|
||||||
- alert: RequestErrorsToAPI
|
- alert: RequestErrorsToAPI
|
||||||
expr: increase(vm_http_request_errors_total[5m]) > 0
|
expr: increase(vm_http_request_errors_total{path=~".+"}[5m]) > 0
|
||||||
for: 15m
|
for: 15m
|
||||||
labels:
|
labels:
|
||||||
severity: warning
|
severity: warning
|
||||||
@@ -37,4 +37,4 @@ groups:
|
|||||||
dashboard: "http://localhost:3000/d/wNf0q_kZk?viewPanel=67&var-instance={{ $labels.instance }}"
|
dashboard: "http://localhost:3000/d/wNf0q_kZk?viewPanel=67&var-instance={{ $labels.instance }}"
|
||||||
summary: "Too many logs printed for job \"{{ $labels.job }}\" ({{ $labels.instance }})"
|
summary: "Too many logs printed for job \"{{ $labels.job }}\" ({{ $labels.instance }})"
|
||||||
description: "Logging rate for job \"{{ $labels.job }}\" ({{ $labels.instance }}) is {{ $value }} for last 15m.\n
|
description: "Logging rate for job \"{{ $labels.job }}\" ({{ $labels.instance }}) is {{ $value }} for last 15m.\n
|
||||||
Worth to check logs for specific error messages."
|
Worth to check logs for specific error messages."
|
||||||
|
|||||||
@@ -75,7 +75,7 @@ groups:
|
|||||||
Consider to limit the ingestion rate, decrease retention or scale the disk space if possible."
|
Consider to limit the ingestion rate, decrease retention or scale the disk space if possible."
|
||||||
|
|
||||||
- alert: RequestErrorsToAPI
|
- alert: RequestErrorsToAPI
|
||||||
expr: increase(vm_http_request_errors_total[5m]) > 0
|
expr: increase(vm_http_request_errors_total{path=~".+"}[5m]) > 0
|
||||||
for: 15m
|
for: 15m
|
||||||
labels:
|
labels:
|
||||||
severity: warning
|
severity: warning
|
||||||
@@ -207,9 +207,9 @@ groups:
|
|||||||
annotations:
|
annotations:
|
||||||
summary: "IndexDB skipped registering items during data ingestion with reason={{ $labels.reason }}."
|
summary: "IndexDB skipped registering items during data ingestion with reason={{ $labels.reason }}."
|
||||||
description: |
|
description: |
|
||||||
VictoriaMetrics could skip registering new timeseries during ingestion if they fail the validation process.
|
VictoriaMetrics could skip registering new timeseries during ingestion if they fail the validation process.
|
||||||
For example, `reason=too_long_item` means that time series cannot exceed 64KB. Please, reduce the number
|
For example, `reason=too_long_item` means that time series cannot exceed 64KB. Please, reduce the number
|
||||||
of labels or label values for such series. Or enforce these limits via `-maxLabelsPerTimeseries` and
|
of labels or label values for such series. Or enforce these limits via `-maxLabelsPerTimeseries` and
|
||||||
`-maxLabelValueLen` command-line flags.
|
`-maxLabelValueLen` command-line flags.
|
||||||
|
|
||||||
- alert: TooManyTSIDMisses
|
- alert: TooManyTSIDMisses
|
||||||
@@ -223,4 +223,4 @@ groups:
|
|||||||
Unexpected TSID misses for \"{{ $labels.job }}\" ({{ $labels.instance }}) for the last 15 minutes.
|
Unexpected TSID misses for \"{{ $labels.job }}\" ({{ $labels.instance }}) for the last 15 minutes.
|
||||||
If this happens after unclean shutdown of VictoriaMetrics process (via \"kill -9\", OOM or power off),
|
If this happens after unclean shutdown of VictoriaMetrics process (via \"kill -9\", OOM or power off),
|
||||||
then this is OK - the alert must go away in a few minutes after the restart.
|
then this is OK - the alert must go away in a few minutes after the restart.
|
||||||
Otherwise this may point to the corruption of index data.
|
Otherwise this may point to the corruption of index data.
|
||||||
|
|||||||
@@ -75,7 +75,7 @@ groups:
|
|||||||
Consider to limit the ingestion rate, decrease retention or scale the disk space if possible."
|
Consider to limit the ingestion rate, decrease retention or scale the disk space if possible."
|
||||||
|
|
||||||
- alert: RequestErrorsToAPI
|
- alert: RequestErrorsToAPI
|
||||||
expr: increase(vm_http_request_errors_total[5m]) > 0
|
expr: increase(vm_http_request_errors_total{path=~".+"}[5m]) > 0
|
||||||
for: 15m
|
for: 15m
|
||||||
labels:
|
labels:
|
||||||
severity: warning
|
severity: warning
|
||||||
@@ -173,9 +173,9 @@ groups:
|
|||||||
annotations:
|
annotations:
|
||||||
summary: "IndexDB skipped registering items during data ingestion with reason={{ $labels.reason }}."
|
summary: "IndexDB skipped registering items during data ingestion with reason={{ $labels.reason }}."
|
||||||
description: |
|
description: |
|
||||||
VictoriaMetrics could skip registering new timeseries during ingestion if they fail the validation process.
|
VictoriaMetrics could skip registering new timeseries during ingestion if they fail the validation process.
|
||||||
For example, `reason=too_long_item` means that time series cannot exceed 64KB. Please, reduce the number
|
For example, `reason=too_long_item` means that time series cannot exceed 64KB. Please, reduce the number
|
||||||
of labels or label values for such series. Or enforce these limits via `-maxLabelsPerTimeseries` and
|
of labels or label values for such series. Or enforce these limits via `-maxLabelsPerTimeseries` and
|
||||||
`-maxLabelValueLen` command-line flags.
|
`-maxLabelValueLen` command-line flags.
|
||||||
|
|
||||||
- alert: TooManyTSIDMisses
|
- alert: TooManyTSIDMisses
|
||||||
@@ -189,4 +189,4 @@ groups:
|
|||||||
Unexpected TSID misses for \"{{ $labels.job }}\" ({{ $labels.instance }}) for the last 15 minutes.
|
Unexpected TSID misses for \"{{ $labels.job }}\" ({{ $labels.instance }}) for the last 15 minutes.
|
||||||
If this happens after unclean shutdown of VictoriaMetrics process (via \"kill -9\", OOM or power off),
|
If this happens after unclean shutdown of VictoriaMetrics process (via \"kill -9\", OOM or power off),
|
||||||
then this is OK - the alert must go away in a few minutes after the restart.
|
then this is OK - the alert must go away in a few minutes after the restart.
|
||||||
Otherwise this may point to the corruption of index data.
|
Otherwise this may point to the corruption of index data.
|
||||||
|
|||||||
Reference in New Issue
Block a user