Compare commits

..

33 Commits

Author SHA1 Message Date
Pablo Fernandez
2481c37f23 Fix a few more typos 2026-08-11 19:32:37 +01:00
Max Kotliar
14549bd30d Apply suggestion from @cubic-dev-ai[bot]
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
Signed-off-by: Max Kotliar <kotlyar.maksim@gmail.com>
2026-08-11 16:07:09 +03:00
Victoria Nduka
e1ad634242 Fix typos and grammar in keyConcepts.md
Signed-off-by: Victoria Nduka <122698422+nwanduka@users.noreply.github.com>
2026-08-11 12:55:34 +01:00
Max Kotliar
578754ef49 docs/changelog: add update note about vmalert logsqql breaking change 2026-08-11 12:15:34 +03:00
Max Kotliar
d88c3f6447 docs/changelog: add update note about vmalert logsql breaking change
See
https://github.com/VictoriaMetrics/VictoriaLogs/blob/master/docs/victorialogs/CHANGELOG.md#v1510,
dce8193c16
2026-08-11 12:10:03 +03:00
Yury Moladau
8a3757d21b app/vmui: fix custom step synchronization between state and URL (#11350)
### Describe Your Changes

Fix custom query step synchronization in vmui.

Previously, the custom step specified via the `g0.step_input` URL
parameter could be overwritten by the automatically calculated step
during page initialization. This could also potentially cause the vmui
Dashboards page to freeze.

Related issue: #11137

Signed-off-by: Yury Molodov <yurymolodov@gmail.com>
Co-authored-by: Max Kotliar <mkotlyar@victoriametrics.com>
2026-08-10 19:29:10 +03:00
Yury Moladau
b6952cf346 app/vmui: add an option to customize the favicon color (#11331)
Make VictoriaMetrics instances easier to distinguish by allowing users to
customize the favicon color. This PR also refreshes the Settings modal
layout and controls.

Mirror of https://github.com/VictoriaMetrics/VictoriaLogs/pull/1646
Related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1634

<img width="497" height="47" alt="image"
src="https://github.com/user-attachments/assets/a250fbb1-b21a-4425-92f7-3aee6dc78a2b"
/>

Changes:
- add per-instance favicon color customization with 12 predefined colors
and a reset option
- persist the selected color in `localStorage` and synchronize it across
browser tabs
- refresh the Settings modal layout and controls

### Browser compatibility

Safari keeps the default favicon because it doesn't support data URL
favicons.

### Screenshots

| Before | After |
|---|---|
| <img width="643" height="726" alt="image"
src="https://github.com/user-attachments/assets/4fda4423-011f-4721-bf21-355135d62a80"
/> | <img width="643" height="726" alt="image"
src="https://github.com/user-attachments/assets/2eee5767-bcad-4e06-85bc-e74704d0f660"
/> |

Signed-off-by: Yury Molodov <yurymolodov@gmail.com>
Co-authored-by: Max Kotliar <mkotlyar@victoriametrics.com>
2026-08-10 19:22:11 +03:00
Andrii Chubatiuk
d142a1682f docs: replace distributed chart links with VMDistributed CR (#11364)
replaced links to deprecated distributed chart with ones to
VMDistributed resource
2026-08-10 14:38:51 +03:00
Hui Wang
bcb653611c docs/changelog: fix misplaced changelog entry location (#11373) 2026-08-10 14:28:01 +03:00
Pablo (Tomas) Fernandez
6cc9a6a2f3 docs: add missing frontmatter in READMEs. Remove duplicate URLs (#11367)
Some README.md do not have a frontmatter. This leads to duplicated URLs
when published.

For example, these two URLs render the same page, which leads to
duplication. The first one should return 404.
- https://docs.victoriametrics.com/opentelemetry/readme/index.html 
- https://docs.victoriametrics.com/opentelemetry/

This PR adds missing frontmatter so all duplicate URLs are removed and
never rendered.

```yaml
---
build:
  list: never
  publishResources: false
  render: never
sitemap:
  disable: true
---
```

(issue reported by @hagen1778, thank you!)
2026-08-10 14:25:35 +03:00
Fred Navruzov
c620cb30b8 docs: update vmanomaly for v1.30.1 (#11363)
## Summary

- Updated examples to prefer online models.
- Marked offline models for future deprecation.
- Updated vmanomaly documentation and deployment references for v1.30.1.
2026-08-06 17:57:56 +03:00
JAYICE
0033834d3c lib/timeutil: properly parse small unix timestamps with fractional part (#11335)
Fix timeutil.TryParseUnixTimestamp returing different results for
equivalent integer and fractional timestamps, such as `12` and `12.0`.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11324

Signed-off-by: “Jayice” <jzhou@victoriametrics.com>
2026-08-06 15:28:42 +02:00
Max Kotliar
ce7b1fba58 .github/build: fix Go build cache collisions between cross-compile targets (#11356)
setup-go's built-in cache uses the runner's OS/arch in the key, not the
target GOOS/GOARCH. When multiple cross-compile jobs run in parallel on
the same runner, they share the same cache key and overwrite each other.

Replace setup-go caching (cache: false) with an explicit actions/cache
step whose key includes matrix.os and matrix.arch, giving each target
its own isolated cache slot. 

The change significantly improves build time. See screenshots.

On master:
<img width="1062" height="638" alt="Screenshot 2026-08-05 at 18 02 43"
src="https://github.com/user-attachments/assets/008bf237-4116-4f65-81fe-30d50bb39295"
/>
<img width="1047" height="625" alt="Screenshot 2026-08-05 at 18 02 59"
src="https://github.com/user-attachments/assets/3abc047f-9430-4ae9-a50a-2bfad13d7e5d"
/>

This commit:
<img width="1073" height="744" alt="Screenshot 2026-08-05 at 17 57 56"
src="https://github.com/user-attachments/assets/3cc0d3eb-b88a-472d-8261-2f25f6788256"
/>
<img width="1058" height="744" alt="Screenshot 2026-08-05 at 17 58 03"
src="https://github.com/user-attachments/assets/9c592596-7577-4503-8482-080bd739a4ac"
/>
2026-08-05 18:19:22 +03:00
Max Kotliar
8855e983b9 docs: update flags with actual v1.149.0 binaries
Signed-off-by: Max Kotliar <mkotlyar@victoriametrics.com>
2026-08-05 15:48:53 +03:00
Max Kotliar
6b3dc18654 docs: bump version to v1.149.0
Signed-off-by: Max Kotliar <mkotlyar@victoriametrics.com>
2026-08-05 15:40:28 +03:00
Max Kotliar
e24adb1501 deplyoment/docker: bump version to v1.149.0
Signed-off-by: Max Kotliar <mkotlyar@victoriametrics.com>
2026-08-05 15:39:20 +03:00
Max Kotliar
0c2dd583c8 docs: forward port LTS v1.136.15 changelog to upstream
Signed-off-by: Max Kotliar <mkotlyar@victoriametrics.com>
2026-08-05 15:37:13 +03:00
Max Kotliar
029540c356 docs: forward port LTS v1.148.1 changelog to upstream
Signed-off-by: Max Kotliar <mkotlyar@victoriametrics.com>
2026-08-05 15:36:44 +03:00
JAYICE
4baba77b15 lib/timeutil: refactor the format of uint64 in time_test (#11353)
Make long numbers used in tests more readable.

Extracted from
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11335 to reduce
diff in the original PR.

Signed-off-by: “Jayice” <jzhou@victoriametrics.com>
2026-08-05 11:27:44 +02:00
Max Kotliar
a8759a539c docs/changelog: cut release v1.149.0
Signed-off-by: Max Kotliar <mkotlyar@victoriametrics.com>
2026-07-31 13:39:51 +03:00
Max Kotliar
f32b743efe docs: update version to v1.149.0
Signed-off-by: Max Kotliar <mkotlyar@victoriametrics.com>
2026-07-31 13:39:32 +03:00
Max Kotliar
8fbf865d9e app/vmselect: run make vmui-update
Signed-off-by: Max Kotliar <mkotlyar@victoriametrics.com>
2026-07-31 13:37:07 +03:00
Max Kotliar
80b6b56028 apptest: fix flaky TestClusterSearchWithDisabledPerDayIndex (#11336)
The test used hardcoded ports for vmstorage instances, which could
already be in use by other processes or tests, causing intermittent
failures like:

    cannot create a server with -vminsertAddr=127.0.0.1:62002:
    unable to listen vminsertAddr 127.0.0.1:62002:
    listen tcp4 127.0.0.1:62002: bind: address already in use

The ports were hardcoded to ensure consistent sharding across cluster
restarts so that the same metrics land on the same storage. However, the
test only verifies query correctness when the per-day index is disabled,
not sharding behavior. Sharding is already covered by
`TestClusterVminsertShardsDataVmselectBuildsFullResultFromShards`.

Switch to a single vmstorage with dynamic ports to eliminate the
flakiness without losing test coverage.
2026-07-31 12:55:21 +03:00
Artem Fetishev
5bdcc5050e lib/storage: reserve 1970-01-01 date for global index search (#11326)
VictoriaMetrics does not support samples with negative timestamps and
limits the min timestamp to `0` (i.e. `1970-01-01T00:00:00Z`).

While samples from the first day (`1970-01-01`) are currently valid (can
be ingested and retrieved), this first day has a special meaning in
`vmstorage` and is used to indicate the search in `global index` instead
of `per-day index`.

This will stop working once the `global index` will be disabled
(https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11196).
I.e. when search is performed on `1970-01-01`, `vmstorage`
will return no results even if the samples exist.

To fix this, we reserve `1970-01-01` for internal use, and allow samples
to have timestamps starting from `1970-01-02`:
- Ingested samples with timestamps from `1970-01-01` will be rejected
and will increment the `vm_rows_ignored_total{reason="small_timestamp"}`
metric.
- Searches whose time range falls completely within the `1970-01-01`
will return empty results.

---------

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2026-07-31 11:19:00 +02:00
Hill Patel
e425aebbc2 lib/timeutil: accept scientific-notation timestamps with sub-second precision (#11278)
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11268

`/api/v1/query` and `/api/v1/query_range` reject `start`/`end` values written in scientific notation whose mantissa carries more fractional digits than the exponent shifts — e.g. `start=1.784144612388E9` returns
`HTTP 422`, while the mathematically equal `start=1784144612.388` returns `200`. Prometheus accepts both (it parses the timestamp with `strconv.ParseFloat`), so this is a Prometheus-compatibility gap.

**Root cause:** in `lib/timeutil/time.go`,
`tryParseScientificNumberForUnixTimestamp` had a guard `if decimalExp <
len(fracStr) { return 0, false }`. For `1.784144612388E9` the mantissa
has 12 fractional digits and the exponent is 9, so `9 < 12` rejected it
— even though the value (`1784144612.388`) is a valid sub-second
timestamp.

**Fix:** when the exponent leaves sub-second fractional digits (`0 <=
decimalExp < len(fracStr)`), keep the mantissa (already
decimal-point-removed) and pad it up to the nearest milli/micro/nano
boundary so `getUnixTimestampNanoseconds` classifies its unit correctly
— exactly how the equivalent plain fractional timestamp is already
parsed by `TryParseUnixTimestamp`.

Kept deliberately in scope, per the discussion on the issue:
- **Negative exponents** on a fractional mantissa (e.g.
`17841446121e-1`) remain unsupported, as @JayiceZ and @valyala decided.
- Sub-second scientific values too small to be a millisecond-scale
timestamp (e.g. `1.23e1`) remain rejected as before, so **no
previously-rejected input changes behaviour** — the change is purely
additive for the reported class of large sub-second timestamps.

## Test plan

- Added success cases to `TestTryParseUnixTimestamp_Success` covering
`1.784144612388E9`, its lowercase and negative forms, and
`1.5000000005e9`, asserting they equal the corresponding
plain-fractional result.
- Verified `TestTryParseUnixTimestamp_Failure` still passes unchanged
(`1.23e1`, `1.234e0`, `1E-1`, negative-exponent forms all still
rejected).
- `go test ./lib/timeutil/`, `go vet ./lib/timeutil/`, and `gofmt` all
clean.

## Disclosure

This change was prepared with AI assistance; I have reviewed the diff,
verified the root cause against the source, and am responsible for the
change and able to explain it.

---------

Co-authored-by: Max Kotliar <mkotlyar@victoriametrics.com>
2026-07-30 21:02:33 +03:00
beyond-infra
f52771ceaf lib/streamaggr: fix sum_samples_total non-monotonic output with enable_windows (#11262)
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11261

`sum_samples_total` produces non-monotonic and undercounted output when`enable_windows: true` is set.


When windows are enabled, blue and green window values are initialized via:
```go
nv.blue[idx]  = ac.getValue(nil)
nv.green[idx] = ac.getValue(nv.blue[idx].state())
```

`sum_samples_total`'s `state()` returned `nil` and `getValue()` ignored its argument, so each window maintained an **independent** cumulative sum. They alternated flushing under the same metric name, producing
repeated, undercounted, and non-monotonic values — e.g. `1, 1, 2, 2` instead of `1, 2, 3, 4`.

## Fix

Introduced `sumSamplesAggrValueShared`, mirroring the pattern already used in `total.go`:
- The shared struct holds the cross-window cumulative `total`
- Both windows receive a pointer to the **same** shared instance via
`state()` / `getValue(s)`
- Each flush adds the window-local `delta` into `shared.total` and outputs that value

`sum_samples` (`resetTotalOnFlush=true`) is unchanged — it has no
cumulative state and uses a plain per-window value as before.

## Test

Added a regression test in `streamaggr_synctest_test.go` that sends 4
batches of `delta=1` with `enable_windows: true` and asserts the output
is monotonically `1, 2, 3, 4`. Before the fix the test fails with `1, 1,
2, 2`.

PR https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11262

---------

Co-authored-by: Hui Wang <haley@victoriametrics.com>
Co-authored-by: Max Kotliar <mkotlyar@victoriametrics.com>
2026-07-30 20:41:12 +03:00
Max Kotliar
eeef07836e docs/changelog: chore changelog 2026-07-30 20:32:29 +03:00
面壁
f1a9c61ba0 lib/promscrape: ignore proxies for Unix socket targets
Unix socket scrape targets inherit `http.DefaultTransport.Proxy` through
`httputil.NewTransport`. When proxy environment variables are
configured, HTTP requests over Unix sockets are altered as proxy
requests, while HTTPS scrapes send `CONNECT` to the Unix socket and fail
the TLS handshake.

Unix sockets are explicit local transport endpoints and cannot be
reached through HTTP proxies. Clear the transport proxy for targets
configured with `__unix_socket__`, matching the existing rejection of
`proxy_url` for these targets.
The bug was introduced with Unix socket scraping support in v1.148.0  at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11193

Related PR https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11318
2026-07-29 18:33:40 +02:00
f41gh7
6cb014fde5 docs/changelog: re-order entries 2026-07-29 15:42:42 +02:00
刘旭
565ecdc4fb app/vmctl: support Prometheus native histograms in remote read mode
This commit add support migrating [Prometheus native
histograms](https://prometheus.io/docs/specs/native_histograms/) in
`vmctl` remote read mode.

- `SAMPLES` mode: process `TimeSeries.Histograms` in addition to
`TimeSeries.Samples`. Previously native histogram samples were silently
ignored.
- `STREAMED_XOR_CHUNKS` mode: dispatch on the chunk encoding and decode
`HISTOGRAM` / `FLOAT_HISTOGRAM` chunks. Previously all chunks were
parsed as XOR, which failed with `EOF` error on native histogram chunks.
Unknown chunk encodings now return an explicit error. `UNKNOWN` (unset)
chunk type is parsed as XOR for compatibility with senders predating
native histograms support.

Native histograms are converted into `_count`, `_sum` and `_bucket`
series with `vmrange` labels in the same way as VictoriaMetrics converts
native histograms received via Prometheus remote write protocol
(`lib/prompb`), so the migration result matches direct remote write
ingestion.

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11292
2026-07-29 15:23:15 +02:00
Hui Wang
06f4fde931 app/vmalert: add flag -replay.continueWithExecutionErr to allow continuing with evaluation errors
Previously, if a rule has a bad syntax, or a query fails due to a bad
expression that causes a duplicate time series on the left side of
error, or the datasource enforces a resource limit that rejects the
request, the replay process exits immediately and the user must fix the
rule before proceeding.

However, since rules are often managed by different teams, bad rules are
not always easy to fix promptly. In such cases, users may want to
continue replaying other rules even when specific rules are problematic.

 This commit adds new `-replay.continueWithExecutionErr` flag to tolerate the 422 response
code from datasource, which indicates that an expression was executed
but failed, see
https://prometheus.io/docs/prometheus/latest/querying/api/#format-overview.
Unlike https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8746,
other errors, such as an unreachable datasource or 5xx responses, are
not covered by this flag, since they indicate a datasource or network
issue that is global in nature rather than specific to a given rule.

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11313
2026-07-29 15:20:09 +02:00
Phuong Le
203eb3a2b4 README: remove retired Go Report Card badge (#11321)
Go Report Card has been sunset: https://goreportcard.com/. Code quality
checks remain covered by the existing CI workflow.
2026-07-29 15:18:34 +02:00
Solomon Wakhungu
7827647b96 app/vmalert: expose evaluation interval as template variable
This commit exposes the alert group's evaluation interval as a new `.Interval`
template variable in vmalert, making it available for use in alert
annotations, labels, and dashboard links without hardcoding.

## Problem

Previously, vmalert templates had access to variables like `.Expr`,
`.ActiveAt`, `.Labels`, `.Value`, `.For`, etc., but not the evaluation
interval. Users generating dashboard links from alert templates need to
know the interval to set the correct time range. Without it, they must
hardcode the interval value, which breaks when the interval changes.

For example, with a 1h evaluation interval, a dashboard link starting
from `.ActiveAt` shows data for the next hour instead of the hour before
the alert became active. With `.Interval`, the link can be generated
relative to the interval.

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11232
2026-07-29 15:13:26 +02:00
100 changed files with 2804 additions and 1271 deletions

View File

@@ -73,11 +73,19 @@ jobs:
id: go
uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
cache-dependency-path: |
go.sum
Makefile
app/**/Makefile
cache: false
go-version-file: 'go.mod'
- name: Cache Go build artifacts
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: go-build-${{ matrix.os }}-${{ matrix.arch }}-${{ hashFiles('go.sum', 'Makefile') }}
restore-keys: |
go-build-${{ matrix.os }}-${{ matrix.arch }}-
- run: go version
- name: Build victoria-metrics for ${{ matrix.os }}-${{ matrix.arch }}

View File

@@ -2,7 +2,6 @@
[![Latest Release](https://img.shields.io/github/v/release/VictoriaMetrics/VictoriaMetrics?sort=semver&label=&filter=!*-victorialogs&logo=github&labelColor=gray&color=gray&link=https%3A%2F%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics%2Freleases%2Flatest)](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
[![Docker Pulls](https://img.shields.io/docker/pulls/victoriametrics/victoria-metrics?label=&logo=docker&logoColor=white&labelColor=2496ED&color=2496ED&link=https%3A%2F%2Fhub.docker.com%2Fr%2Fvictoriametrics%2Fvictoria-metrics)](https://hub.docker.com/u/victoriametrics)
[![Go Report](https://goreportcard.com/badge/github.com/VictoriaMetrics/VictoriaMetrics?link=https%3A%2F%2Fgoreportcard.com%2Freport%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics)](https://goreportcard.com/report/github.com/VictoriaMetrics/VictoriaMetrics)
[![Build Status](https://github.com/VictoriaMetrics/VictoriaMetrics/actions/workflows/build.yml/badge.svg?branch=master&link=https%3A%2F%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics%2Factions)](https://github.com/VictoriaMetrics/VictoriaMetrics/actions/workflows/build.yml)
[![License](https://img.shields.io/github/license/VictoriaMetrics/VictoriaMetrics?labelColor=green&label=&link=https%3A%2F%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics%2Fblob%2Fmaster%2FLICENSE)](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/LICENSE)
[![Join Slack](https://img.shields.io/badge/Join%20Slack-4A154B?logo=slack)](https://slack.victoriametrics.com)

View File

@@ -11,6 +11,7 @@ import (
"strings"
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/netutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/promauth"
@@ -267,7 +268,11 @@ func (c *Client) do(req *http.Request) (*http.Response, error) {
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
_ = resp.Body.Close()
return nil, fmt.Errorf("unexpected response code %d for %s. Response body %s", resp.StatusCode, ru, body)
err = &httpserver.ErrorWithStatusCode{
StatusCode: resp.StatusCode,
Err: fmt.Errorf("unexpected response code %d for %s. Response body %s", resp.StatusCode, ru, body),
}
return nil, err
}
return resp, nil
}

View File

@@ -31,6 +31,8 @@ type Alert struct {
State AlertState
// Expr contains expression that was executed to generate the Alert
Expr string
// Interval contains the evaluation interval of the Alert's group
Interval time.Duration
// ActiveAt defines the moment of time when Alert has become active
ActiveAt time.Time
// Start defines the moment of time when Alert has become firing
@@ -84,6 +86,7 @@ type AlertTplData struct {
Labels map[string]string
Value float64
Expr string
Interval time.Duration
AlertID uint64
GroupID uint64
ActiveAt time.Time
@@ -96,6 +99,7 @@ var tplHeaders = []string{
"{{ $type := .Type }}",
"{{ $labels := .Labels }}",
"{{ $expr := .Expr }}",
"{{ $interval := .Interval }}",
"{{ $externalLabels := .ExternalLabels }}",
"{{ $externalURL := .ExternalURL }}",
"{{ $alertID := .AlertID }}",
@@ -115,6 +119,7 @@ func (a *Alert) ExecTemplate(q templates.QueryFn, labels, annotations map[string
Type: a.Type,
Labels: labels,
Expr: a.Expr,
Interval: a.Interval,
AlertID: a.ID,
GroupID: a.GroupID,
ActiveAt: a.ActiveAt,

View File

@@ -129,6 +129,17 @@ func TestAlertExecTemplate(t *testing.T) {
"exprEscapedHTML": "vm_rows{&quot;label&quot;=&quot;bar&quot;}&lt;0",
})
// interval-template
f(&Alert{
Interval: 10 * time.Second,
}, map[string]string{
"interval": "{{ .Interval }}",
"intervalVariable": "{{ $interval }}",
}, map[string]string{
"interval": "10s",
"intervalVariable": "10s",
})
// query
f(&Alert{
Expr: `vm_rows{"label"="bar"}>0`,

View File

@@ -25,11 +25,12 @@ var (
replayMaxDatapoints = flag.Int("replay.maxDatapointsPerQuery", 1e3,
"Max number of data points expected in one request. It affects the max time range for every '/query_range' request during the replay. The higher the value, the less requests will be made during replay.")
replayRuleRetryAttempts = flag.Int("replay.ruleRetryAttempts", 5,
"Defines how many retries to make before giving up on rule if request for it returns an error.")
"Defines how many retries to make before giving up on rule if request for it returns a retriable error.")
disableProgressBar = flag.Bool("replay.disableProgressBar", false, "Whether to disable rendering progress bars during the replay. "+
"Progress bar rendering might be verbose or break the logs parsing, so it is recommended to be disabled when not used in interactive mode.")
ruleEvaluationConcurrency = flag.Int("replay.ruleEvaluationConcurrency", 1, "The maximum number of concurrent '/query_range' requests when replay recording rule or alerting rule with for=0. "+
"Increasing this value when replaying for a long time, since each request is limited by -replay.maxDatapointsPerQuery.")
continueWithExecutionErr = flag.Bool("replay.continueWithExecutionErr", false, "Whether to continue replaying other rules if a rule execution fails with a 422 response code, which can happen due to an expression syntax error or a resource limit being hit.")
)
func replay(groupsCfg []config.Group, qb datasource.QuerierBuilder, rw remotewrite.RWClient) (totalRows, droppedRows int, err error) {
@@ -73,7 +74,7 @@ func replay(groupsCfg []config.Group, qb datasource.QuerierBuilder, rw remotewri
for _, cfg := range groupsCfg {
ng := rule.NewGroup(cfg, qb, *evaluationInterval, labels)
totalRows += ng.Replay(tFrom, tTo, rw, *replayMaxDatapoints, *replayRuleRetryAttempts, *replayRulesDelay, *disableProgressBar, *ruleEvaluationConcurrency)
totalRows += ng.Replay(tFrom, tTo, rw, *replayMaxDatapoints, *replayRuleRetryAttempts, *replayRulesDelay, *disableProgressBar, *ruleEvaluationConcurrency, *continueWithExecutionErr)
}
logger.Infof("replay evaluation finished, generated %d samples", totalRows)
if err := rw.Close(); err != nil {

View File

@@ -8,8 +8,10 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmalert/config"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmalert/datasource"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/prompb"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/promutil"
"github.com/VictoriaMetrics/metricsql"
)
type fakeReplayQuerier struct {
@@ -32,6 +34,14 @@ func (fc *fakeRWClient) Close() error {
}
func (fr *fakeReplayQuerier) QueryRange(_ context.Context, q string, from, to time.Time) (res datasource.Result, err error) {
_, err = metricsql.Parse(q)
if err != nil {
return res, &httpserver.ErrorWithStatusCode{
StatusCode: 422,
Err: err,
}
}
key := fmt.Sprintf("%s+%s", from.Format("15:04:05"), to.Format("15:04:05"))
dps, ok := fr.registry[q]
if !ok {
@@ -275,4 +285,28 @@ func TestReplay(t *testing.T) {
},
},
}, 10)
// rule with wrong expression won't break the other rule with continueWithExecutionErr
continueWithExecutionErrOld := *continueWithExecutionErr
defer func() {
*continueWithExecutionErr = continueWithExecutionErrOld
}()
*continueWithExecutionErr = true
f("2021-01-01T12:00:00.000Z", "2021-01-01T12:02:30.000Z", 1, 1, time.Millisecond, []config.Group{
{Rules: []config.Rule{{Record: "foo", Expr: "sum(up)"}}},
{Rules: []config.Rule{{Record: "bar", Expr: "up ++"}}},
}, &fakeReplayQuerier{
registry: map[string]map[string][]datasource.Metric{
"sum(up)": {
"12:00:00+12:01:00": {
{
Timestamps: []int64{1, 2},
Values: []float64{1, 2},
},
},
"12:01:00+12:02:00": {},
"12:02:00+12:02:30": {},
},
},
}, 2)
}

View File

@@ -530,6 +530,7 @@ func (ar *AlertingRule) exec(ctx context.Context, ts time.Time, limit int) ([]pr
ar.logDebugf(ts, a, "INACTIVE => PENDING")
}
a.Value = m.Values[0]
a.Interval = ar.EvalInterval
a.Annotations = annotations
a.KeepFiringSince = time.Time{}
continue
@@ -612,6 +613,7 @@ func (ar *AlertingRule) expandAnnotationTemplates(m datasource.Metric, qFn templ
Type: ar.Type.String(),
Labels: ls.origin,
Expr: ar.Expr,
Interval: ar.EvalInterval,
AlertID: hash(ls.processed),
GroupID: ar.GroupID,
ActiveAt: activeAt,
@@ -673,6 +675,7 @@ func (ar *AlertingRule) newAlert(m datasource.Metric, start time.Time, labels, a
Name: ar.Name,
Type: ar.Type.String(),
Expr: ar.Expr,
Interval: ar.EvalInterval,
For: ar.For,
ActiveAt: start,
Value: m.Values[0],

View File

@@ -662,6 +662,7 @@ func TestAlertingRuleExecRange(t *testing.T) {
GroupID: fakeGroup.GetID(),
Name: "for-pending",
Type: config.NewPrometheusType().String(),
Interval: time.Second,
Labels: map[string]string{"alertname": "for-pending"},
Annotations: map[string]string{},
State: notifier.StatePending,
@@ -682,6 +683,7 @@ func TestAlertingRuleExecRange(t *testing.T) {
GroupID: fakeGroup.GetID(),
Name: "for-firing",
Type: config.NewPrometheusType().String(),
Interval: 3 * time.Second,
Labels: map[string]string{"alertname": "for-firing"},
Annotations: map[string]string{},
State: notifier.StateFiring,
@@ -703,6 +705,7 @@ func TestAlertingRuleExecRange(t *testing.T) {
GroupID: fakeGroup.GetID(),
Name: "for-hold-pending",
Type: config.NewPrometheusType().String(),
Interval: time.Second,
Labels: map[string]string{"alertname": "for-hold-pending"},
Annotations: map[string]string{},
State: notifier.StatePending,
@@ -759,6 +762,7 @@ func TestAlertingRuleExecRange(t *testing.T) {
GroupID: fakeGroup.GetID(),
Name: "multi-series",
Type: config.NewPrometheusType().String(),
Interval: 3 * time.Second,
Labels: map[string]string{"alertname": "multi-series"},
Annotations: map[string]string{},
State: notifier.StateFiring,
@@ -771,6 +775,7 @@ func TestAlertingRuleExecRange(t *testing.T) {
GroupID: fakeGroup.GetID(),
Name: "multi-series",
Type: config.NewPrometheusType().String(),
Interval: 3 * time.Second,
Labels: map[string]string{"alertname": "multi-series", "foo": "bar"},
Annotations: map[string]string{},
State: notifier.StatePending,
@@ -1134,7 +1139,8 @@ func TestAlertingRule_Template(t *testing.T) {
fq.Add(metrics...)
fq.SetPartialResponse(isResponsePartial)
if _, err := rule.exec(context.TODO(), time.Now(), 0); err != nil {
ts := time.Unix(3600, 0)
if _, err := rule.exec(context.TODO(), ts, 0); err != nil {
t.Fatalf("unexpected error: %s", err)
}
for hash, expAlert := range alertsExpected {
@@ -1152,12 +1158,14 @@ func TestAlertingRule_Template(t *testing.T) {
}
f(&AlertingRule{
Name: "common",
Name: "common",
EvalInterval: time.Hour,
Labels: map[string]string{
"region": "east",
},
Annotations: map[string]string{
"summary": `{{ $labels.alertname }}: Too high connection number for "{{ $labels.instance }}"`,
"summary": `{{ $labels.alertname }}: Too high connection number for "{{ $labels.instance }}"`,
"dashboard": `&from={{ ($activeAt.Add (parseDurationTime (printf "-%s" .Interval))).UnixMilli }}&to={{ $activeAt.UnixMilli }}`,
},
alerts: make(map[uint64]*notifier.Alert),
}, []datasource.Metric{
@@ -1166,7 +1174,8 @@ func TestAlertingRule_Template(t *testing.T) {
}, false, map[uint64]*notifier.Alert{
hash(map[string]string{alertNameLabel: "common", "region": "east", "instance": "foo"}): {
Annotations: map[string]string{
"summary": `common: Too high connection number for "foo"`,
"summary": `common: Too high connection number for "foo"`,
"dashboard": "&from=0&to=3600000",
},
Labels: map[string]string{
alertNameLabel: "common",
@@ -1176,7 +1185,8 @@ func TestAlertingRule_Template(t *testing.T) {
},
hash(map[string]string{alertNameLabel: "common", "region": "east", "instance": "bar"}): {
Annotations: map[string]string{
"summary": `common: Too high connection number for "bar"`,
"summary": `common: Too high connection number for "bar"`,
"dashboard": "&from=0&to=3600000",
},
Labels: map[string]string{
alertNameLabel: "common",
@@ -1388,7 +1398,7 @@ func TestAlertingRule_ToLabels(t *testing.T) {
"alertname": "ConfigurationReloadFailure",
"alertgroup": "vmalert",
"pod": "vmalert-0",
"invalid_label": `error evaluating template: template: :1:298: executing "" at <.Values.mustRuntimeFail>: can't evaluate field Values in type notifier.tplData`,
"invalid_label": `error evaluating template: template: :1:326: executing "" at <.Values.mustRuntimeFail>: can't evaluate field Values in type notifier.tplData`,
}
expectedProcessedLabels := map[string]string{
@@ -1398,7 +1408,7 @@ func TestAlertingRule_ToLabels(t *testing.T) {
"exported_alertname": "ConfigurationReloadFailure",
"group": "vmalert",
"alertgroup": "vmalert",
"invalid_label": `error evaluating template: template: :1:298: executing "" at <.Values.mustRuntimeFail>: can't evaluate field Values in type notifier.tplData`,
"invalid_label": `error evaluating template: template: :1:326: executing "" at <.Values.mustRuntimeFail>: can't evaluate field Values in type notifier.tplData`,
}
ls, err := ar.toLabels(metric, nil)

View File

@@ -548,7 +548,7 @@ func (g *Group) infof(format string, args ...any) {
}
// Replay performs group replay
func (g *Group) Replay(start, end time.Time, rw remotewrite.RWClient, maxDataPoint, replayRuleRetryAttempts int, replayDelay time.Duration, disableProgressBar bool, ruleEvaluationConcurrency int) int {
func (g *Group) Replay(start, end time.Time, rw remotewrite.RWClient, maxDataPoint, replayRuleRetryAttempts int, replayDelay time.Duration, disableProgressBar bool, ruleEvaluationConcurrency int, continueWithExecutionErr bool) int {
var total int
step := g.Interval * time.Duration(maxDataPoint)
ri := rangeIterator{start: start, end: end, step: step}
@@ -576,7 +576,7 @@ func (g *Group) Replay(start, end time.Time, rw remotewrite.RWClient, maxDataPoi
if !disableProgressBar {
bar = pb.StartNew(iterations)
}
total += replayRuleRange(rule, ri, bar, rw, replayRuleRetryAttempts, ruleEvaluationConcurrency)
total += replayRuleRange(rule, ri, bar, rw, replayRuleRetryAttempts, ruleEvaluationConcurrency, continueWithExecutionErr)
if bar != nil {
bar.Finish()
}
@@ -598,7 +598,7 @@ func (g *Group) Replay(start, end time.Time, rw remotewrite.RWClient, maxDataPoi
rule := g.Rules[i]
sem <- struct{}{}
wg.Go(func() {
res <- replayRuleRange(rule, ri, bar, rw, replayRuleRetryAttempts, ruleEvaluationConcurrency)
res <- replayRuleRange(rule, ri, bar, rw, replayRuleRetryAttempts, ruleEvaluationConcurrency, continueWithExecutionErr)
<-sem
})
}
@@ -618,7 +618,7 @@ func (g *Group) Replay(start, end time.Time, rw remotewrite.RWClient, maxDataPoi
return total
}
func replayRuleRange(r Rule, ri rangeIterator, bar *pb.ProgressBar, rw remotewrite.RWClient, replayRuleRetryAttempts, ruleEvaluationConcurrency int) int {
func replayRuleRange(r Rule, ri rangeIterator, bar *pb.ProgressBar, rw remotewrite.RWClient, replayRuleRetryAttempts, ruleEvaluationConcurrency int, continueWithExecutionErr bool) int {
fmt.Printf("> Rule %q (ID: %d)\n", r, r.ID())
// alerting rule with for>0 can't be replayed concurrently, since the status change might depend on the previous evaluation
// see https://github.com/VictoriaMetrics/VictoriaMetrics/commit/abcb21aa5ee918ba9a4e9cde495dba06e1e9564c
@@ -633,7 +633,7 @@ func replayRuleRange(r Rule, ri rangeIterator, bar *pb.ProgressBar, rw remotewri
start := ri.s
end := ri.e
wg.Go(func() {
n, err := replayRule(r, start, end, rw, replayRuleRetryAttempts)
n, err := replayRule(r, start, end, rw, replayRuleRetryAttempts, continueWithExecutionErr)
if err != nil {
logger.Fatalf("rule %q: %s", r, err)
}

View File

@@ -4,12 +4,14 @@ import (
"context"
"errors"
"fmt"
"net/http"
"sync"
"time"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmalert/remotewrite"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/prompb"
)
@@ -118,7 +120,7 @@ func (s *ruleState) add(e StateEntry) {
s.entries[s.cur] = e
}
func replayRule(r Rule, start, end time.Time, rw remotewrite.RWClient, replayRuleRetryAttempts int) (int, error) {
func replayRule(r Rule, start, end time.Time, rw remotewrite.RWClient, replayRuleRetryAttempts int, continueWithExecutionErr bool) (int, error) {
var err error
var tss []prompb.TimeSeries
for i := range replayRuleRetryAttempts {
@@ -126,6 +128,21 @@ func replayRule(r Rule, start, end time.Time, rw remotewrite.RWClient, replayRul
if err == nil {
break
}
// retry request if possible to tolerate temporary network or datasource unavailability issues
var esc *httpserver.ErrorWithStatusCode
if errors.As(err, &esc) {
statusCode := esc.StatusCode
// if the status code is 422, it means that the query was executed but failed due to an expression syntax error or a the resource limit being hit,
// continue replaying but skip the problematic execution if continueWithExecutionErr is true, otherwise, return the error without retry.
if statusCode == http.StatusUnprocessableEntity {
if continueWithExecutionErr {
logger.Errorf("rule %q: %s", r, err)
return 0, nil
} else {
return 0, err
}
}
}
logger.Errorf("attempt %d to execute rule %q failed: %s", i+1, r, err)
time.Sleep(time.Second)
}

View File

@@ -8,12 +8,14 @@ import (
"io"
"net/http"
"net/url"
"strconv"
"strings"
"time"
"github.com/gogo/protobuf/proto"
"github.com/golang/snappy"
"github.com/prometheus/prometheus/config"
"github.com/prometheus/prometheus/model/histogram"
"github.com/prometheus/prometheus/prompb"
"github.com/prometheus/prometheus/storage/remote"
"github.com/prometheus/prometheus/tsdb/chunkenc"
@@ -234,9 +236,29 @@ func processResponse(body io.ReadCloser, callback StreamCallback) error {
// shouldn't be accounted as an error.
for _, res := range readResp.Results {
for _, ts := range res.Timeseries {
vmTs := convertSamples(ts.Samples, ts.Labels)
if err := callback(vmTs); err != nil {
return err
// A series contains either float samples or native histogram samples.
// Both fields are processed independently, since a series may switch
// from float to native histogram representation at some point in time,
// so the requested time range may contain samples of both types.
if len(ts.Samples) > 0 {
vmTs := convertSamples(ts.Samples, ts.Labels)
if err := callback(vmTs); err != nil {
return err
}
}
if len(ts.Histograms) > 0 {
hSamples := make([]histogramSample, 0, len(ts.Histograms))
for _, h := range ts.Histograms {
hSamples = append(hSamples, histogramSample{
timestamp: h.Timestamp,
fh: h.ToFloatHistogram(),
})
}
for _, vmTs := range convertHistograms(hSamples, ts.Labels) {
if err := callback(vmTs); err != nil {
return err
}
}
}
}
}
@@ -263,17 +285,45 @@ func processStreamResponse(body io.ReadCloser, callback StreamCallback) error {
for _, series := range res.ChunkedSeries {
samples := make([]prompb.Sample, 0)
var hSamples []histogramSample
for _, chunk := range series.Chunks {
s, err := parseSamples(chunk.Data)
if err != nil {
return err
switch chunk.Type {
case prompb.Chunk_XOR, prompb.Chunk_UNKNOWN:
// In proto3 the `type` field may be left unset (UNKNOWN) for XOR chunks.
// Prometheus remote.proto: "REQUIREMENT: when using proto3, this field
// MUST be set when using anything else than XOR". Senders before native
// histograms support (Prometheus < 2.40) do not set this field at all,
// so UNKNOWN chunks must be parsed as XOR ones.
s, err := parseSamples(chunk.Data)
if err != nil {
return err
}
samples = append(samples, s...)
case prompb.Chunk_HISTOGRAM, prompb.Chunk_FLOAT_HISTOGRAM:
hs, err := parseHistograms(chunk.Type, chunk.Data)
if err != nil {
return err
}
hSamples = append(hSamples, hs...)
default:
return fmt.Errorf("unsupported chunk encoding %q", chunk.Type)
}
samples = append(samples, s...)
}
ts := convertSamples(samples, series.Labels)
if err := callback(ts); err != nil {
return err
// A series contains either XOR chunks or native histogram chunks.
// Both are processed independently, since a series may switch
// from float to native histogram representation at some point in time,
// so the requested time range may contain chunks of both types.
if len(samples) > 0 {
ts := convertSamples(samples, series.Labels)
if err := callback(ts); err != nil {
return err
}
}
for _, ts := range convertHistograms(hSamples, series.Labels) {
if err := callback(ts); err != nil {
return err
}
}
}
}
@@ -312,6 +362,151 @@ func parseSamples(chunk []byte) ([]prompb.Sample, error) {
return samples, it.Err()
}
// histogramSample represents a single native histogram sample.
type histogramSample struct {
timestamp int64
fh *histogram.FloatHistogram
}
func parseHistograms(encoding prompb.Chunk_Encoding, chunk []byte) ([]histogramSample, error) {
var enc chunkenc.Encoding
switch encoding {
case prompb.Chunk_HISTOGRAM:
enc = chunkenc.EncHistogram
case prompb.Chunk_FLOAT_HISTOGRAM:
enc = chunkenc.EncFloatHistogram
default:
return nil, fmt.Errorf("unsupported histogram chunk encoding %q", encoding)
}
c, err := chunkenc.FromData(enc, chunk)
if err != nil {
return nil, fmt.Errorf("error read chunk: %w", err)
}
var hSamples []histogramSample
it := c.Iterator(nil)
for {
typ := it.Next()
if typ == chunkenc.ValNone {
break
}
switch typ {
case chunkenc.ValHistogram:
ts, h := it.AtHistogram(nil)
hSamples = append(hSamples, histogramSample{
timestamp: ts,
fh: h.ToFloat(nil),
})
case chunkenc.ValFloatHistogram:
ts, fh := it.AtFloatHistogram(nil)
hSamples = append(hSamples, histogramSample{
timestamp: ts,
fh: fh,
})
default:
// Skip unsupported values
continue
}
}
if err := it.Err(); err != nil {
return nil, fmt.Errorf("error iterate over chunks: %w", err)
}
return hSamples, nil
}
// convertHistograms converts native histogram samples into VictoriaMetrics histogram
// time series in the same way as VictoriaMetrics converts native histograms
// received via Prometheus remote write protocol: every native histogram sample
// is converted into `<name>_count` and `<name>_sum` series plus a set of
// `<name>_bucket` series with `vmrange` labels containing non-cumulative bucket counts.
// The only difference is that for native histograms with custom buckets (NHCB)
// bucket bounds are taken from the custom values, while the remote write protocol
// parser ignores custom values and estimates the bounds with the exponential formula.
// See https://prometheus.io/docs/specs/native_histograms/#data-model
func convertHistograms(hSamples []histogramSample, labels []prompb.Label) []*vm.TimeSeries {
if len(hSamples) == 0 {
return nil
}
labelPairs := make([]vm.LabelPair, 0, len(labels))
nameValue := ""
for _, label := range labels {
if label.Name == "__name__" {
nameValue = label.Value
continue
}
labelPairs = append(labelPairs, vm.LabelPair{Name: label.Name, Value: label.Value})
}
// the metric has no name, skip it in the same way as VictoriaMetrics does
// when it receives a native histogram without the metric name via remote write protocol.
if nameValue == "" {
return nil
}
countSeries := &vm.TimeSeries{
Name: nameValue + "_count",
LabelPairs: labelPairs,
}
sumSeries := &vm.TimeSeries{
Name: nameValue + "_sum",
LabelPairs: labelPairs,
}
bucketSeries := make(map[string]*vm.TimeSeries)
// vmranges preserves the order of bucketSeries creation
// in order to get deterministic results.
var vmranges []string
for _, hs := range hSamples {
fh := hs.fh
countSeries.Timestamps = append(countSeries.Timestamps, hs.timestamp)
countSeries.Values = append(countSeries.Values, fh.Count)
sumSeries.Timestamps = append(sumSeries.Timestamps, hs.timestamp)
sumSeries.Values = append(sumSeries.Values, fh.Sum)
it := fh.AllBucketIterator()
for it.Next() {
b := it.At()
if b.Count <= 0 {
continue
}
vmrange := formatVmrange(b.Lower, b.Upper)
s := bucketSeries[vmrange]
if s == nil {
bucketLabelPairs := make([]vm.LabelPair, len(labelPairs), len(labelPairs)+1)
copy(bucketLabelPairs, labelPairs)
bucketLabelPairs = append(bucketLabelPairs, vm.LabelPair{Name: "vmrange", Value: vmrange})
s = &vm.TimeSeries{
Name: nameValue + "_bucket",
LabelPairs: bucketLabelPairs,
}
bucketSeries[vmrange] = s
vmranges = append(vmranges, vmrange)
}
s.Timestamps = append(s.Timestamps, hs.timestamp)
s.Values = append(s.Values, b.Count)
}
}
tss := make([]*vm.TimeSeries, 0, 2+len(vmranges))
tss = append(tss, countSeries, sumSeries)
for _, vmrange := range vmranges {
tss = append(tss, bucketSeries[vmrange])
}
return tss
}
// formatVmrange formats the given bucket bounds into `vmrange` label value
// in the same way as VictoriaMetrics does for native histograms
// received via Prometheus remote write protocol.
func formatVmrange(lower, upper float64) string {
b := make([]byte, 0, 24)
b = strconv.AppendFloat(b, lower, 'e', 3, 64)
b = append(b, "..."...)
b = strconv.AppendFloat(b, upper, 'e', 3, 64)
return string(b)
}
type keyValue struct {
key string
value string

View File

@@ -0,0 +1,334 @@
package remoteread
import (
"bytes"
"io"
"reflect"
"testing"
"github.com/gogo/protobuf/proto"
"github.com/golang/snappy"
"github.com/prometheus/prometheus/model/histogram"
"github.com/prometheus/prometheus/prompb"
"github.com/prometheus/prometheus/storage/remote"
"github.com/prometheus/prometheus/tsdb/chunkenc"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/vm"
)
func testHistogram(mul int64) *histogram.Histogram {
return &histogram.Histogram{
Schema: 0,
Count: uint64(10 * mul),
Sum: 25.5 * float64(mul),
ZeroThreshold: 0.001,
ZeroCount: uint64(2 * mul),
PositiveSpans: []histogram.Span{{Offset: 0, Length: 2}},
PositiveBuckets: []int64{1 * mul, 2 * mul},
NegativeSpans: []histogram.Span{{Offset: 0, Length: 1}},
NegativeBuckets: []int64{4 * mul},
}
}
func TestConvertHistograms(t *testing.T) {
f := func(hSamples []histogramSample, labels []prompb.Label, expected []*vm.TimeSeries) {
t.Helper()
tss := convertHistograms(hSamples, labels)
if !reflect.DeepEqual(tss, expected) {
t.Fatalf("unexpected result\ngot:\n%v\nwant:\n%v", tss, expected)
}
}
// series without samples
f(nil, []prompb.Label{{Name: "__name__", Value: "foo"}}, nil)
// series without the metric name must be skipped
f([]histogramSample{
{timestamp: 1000, fh: testHistogram(1).ToFloat(nil)},
}, []prompb.Label{{Name: "job", Value: "bar"}}, nil)
// native histogram must be converted to _count, _sum and _bucket series
// in the same way as VictoriaMetrics does for Prometheus remote write protocol
labels := []prompb.Label{
{Name: "__name__", Value: "request_duration_seconds"},
{Name: "job", Value: "bar"},
}
jobLabel := []vm.LabelPair{{Name: "job", Value: "bar"}}
bucketLabels := func(vmrange string) []vm.LabelPair {
return []vm.LabelPair{
{Name: "job", Value: "bar"},
{Name: "vmrange", Value: vmrange},
}
}
f([]histogramSample{
{timestamp: 1000, fh: testHistogram(1).ToFloat(nil)},
{timestamp: 2000, fh: testHistogram(2).ToFloat(nil)},
}, labels, []*vm.TimeSeries{
{
Name: "request_duration_seconds_count",
LabelPairs: jobLabel,
Timestamps: []int64{1000, 2000},
Values: []float64{10, 20},
},
{
Name: "request_duration_seconds_sum",
LabelPairs: jobLabel,
Timestamps: []int64{1000, 2000},
Values: []float64{25.5, 51},
},
{
Name: "request_duration_seconds_bucket",
LabelPairs: bucketLabels("-1.000e+00...-5.000e-01"),
Timestamps: []int64{1000, 2000},
Values: []float64{4, 8},
},
{
Name: "request_duration_seconds_bucket",
LabelPairs: bucketLabels("-1.000e-03...1.000e-03"),
Timestamps: []int64{1000, 2000},
Values: []float64{2, 4},
},
{
Name: "request_duration_seconds_bucket",
LabelPairs: bucketLabels("5.000e-01...1.000e+00"),
Timestamps: []int64{1000, 2000},
Values: []float64{1, 2},
},
{
Name: "request_duration_seconds_bucket",
LabelPairs: bucketLabels("1.000e+00...2.000e+00"),
Timestamps: []int64{1000, 2000},
Values: []float64{3, 6},
},
})
}
func TestParseHistograms(t *testing.T) {
c := chunkenc.NewHistogramChunk()
app, err := c.Appender()
if err != nil {
t.Fatalf("cannot create chunk appender: %s", err)
}
if _, _, _, err := app.AppendHistogram(nil, 0, 1000, testHistogram(1), true); err != nil {
t.Fatalf("cannot append histogram: %s", err)
}
if _, _, _, err := app.AppendHistogram(nil, 0, 2000, testHistogram(2), true); err != nil {
t.Fatalf("cannot append histogram: %s", err)
}
hSamples, err := parseHistograms(prompb.Chunk_HISTOGRAM, c.Bytes())
if err != nil {
t.Fatalf("cannot parse histogram chunk: %s", err)
}
if len(hSamples) != 2 {
t.Fatalf("unexpected number of histogram samples; got %d; want 2", len(hSamples))
}
for i, expected := range []struct {
timestamp int64
count float64
sum float64
}{
{timestamp: 1000, count: 10, sum: 25.5},
{timestamp: 2000, count: 20, sum: 51},
} {
if hSamples[i].timestamp != expected.timestamp {
t.Fatalf("unexpected timestamp; got %d; want %d", hSamples[i].timestamp, expected.timestamp)
}
if hSamples[i].fh.Count != expected.count {
t.Fatalf("unexpected count; got %f; want %f", hSamples[i].fh.Count, expected.count)
}
if hSamples[i].fh.Sum != expected.sum {
t.Fatalf("unexpected sum; got %f; want %f", hSamples[i].fh.Sum, expected.sum)
}
}
// unsupported chunk encoding must return error
if _, err := parseHistograms(prompb.Chunk_XOR, c.Bytes()); err == nil {
t.Fatalf("expecting non-nil error for unsupported chunk encoding")
}
}
func TestProcessResponse(t *testing.T) {
readResp := &prompb.ReadResponse{
Results: []*prompb.QueryResult{
{
Timeseries: []*prompb.TimeSeries{
{
Labels: []prompb.Label{
{Name: "__name__", Value: "cpu_usage"},
{Name: "job", Value: "bar"},
},
Samples: []prompb.Sample{
{Timestamp: 1000, Value: 1.5},
},
},
{
Labels: []prompb.Label{
{Name: "__name__", Value: "request_duration_seconds"},
{Name: "job", Value: "bar"},
},
Histograms: []prompb.Histogram{
prompb.FromIntHistogram(1000, testHistogram(1)),
},
},
},
},
},
}
data, err := proto.Marshal(readResp)
if err != nil {
t.Fatalf("cannot marshal ReadResponse: %s", err)
}
compressed := snappy.Encode(nil, data)
var tss []*vm.TimeSeries
err = processResponse(io.NopCloser(bytes.NewReader(compressed)), func(ts *vm.TimeSeries) error {
tss = append(tss, ts)
return nil
})
if err != nil {
t.Fatalf("cannot process response: %s", err)
}
// 1 float series + _count + _sum + 4 buckets
if len(tss) != 7 {
t.Fatalf("unexpected number of time series; got %d; want 7", len(tss))
}
if tss[0].Name != "cpu_usage" || !reflect.DeepEqual(tss[0].Values, []float64{1.5}) {
t.Fatalf("unexpected float series: %v", tss[0])
}
if tss[1].Name != "request_duration_seconds_count" || !reflect.DeepEqual(tss[1].Values, []float64{10}) {
t.Fatalf("unexpected _count series: %v", tss[1])
}
if tss[2].Name != "request_duration_seconds_sum" || !reflect.DeepEqual(tss[2].Values, []float64{25.5}) {
t.Fatalf("unexpected _sum series: %v", tss[2])
}
for _, ts := range tss[3:] {
if ts.Name != "request_duration_seconds_bucket" {
t.Fatalf("unexpected bucket series name %q", ts.Name)
}
}
}
type nopFlusher struct{}
func (nopFlusher) Flush() {}
func TestProcessStreamResponse(t *testing.T) {
// build a histogram chunk
hc := chunkenc.NewHistogramChunk()
hApp, err := hc.Appender()
if err != nil {
t.Fatalf("cannot create histogram chunk appender: %s", err)
}
if _, _, _, err := hApp.AppendHistogram(nil, 0, 1000, testHistogram(1), true); err != nil {
t.Fatalf("cannot append histogram: %s", err)
}
// build a float chunk
xc := chunkenc.NewXORChunk()
xApp, err := xc.Appender()
if err != nil {
t.Fatalf("cannot create xor chunk appender: %s", err)
}
xApp.Append(0, 1000, 1.5)
res := &prompb.ChunkedReadResponse{
ChunkedSeries: []*prompb.ChunkedSeries{
{
Labels: []prompb.Label{
{Name: "__name__", Value: "request_duration_seconds"},
{Name: "job", Value: "bar"},
},
Chunks: []prompb.Chunk{
{Type: prompb.Chunk_HISTOGRAM, Data: hc.Bytes()},
},
},
{
Labels: []prompb.Label{
{Name: "__name__", Value: "cpu_usage"},
},
Chunks: []prompb.Chunk{
{Type: prompb.Chunk_XOR, Data: xc.Bytes()},
},
},
{
Labels: []prompb.Label{
{Name: "__name__", Value: "memory_usage"},
},
Chunks: []prompb.Chunk{
// the `type` field may be unset for XOR chunks,
// such chunks must be parsed as XOR ones
{Type: prompb.Chunk_UNKNOWN, Data: xc.Bytes()},
},
},
},
}
data, err := proto.Marshal(res)
if err != nil {
t.Fatalf("cannot marshal ChunkedReadResponse: %s", err)
}
var buf bytes.Buffer
cw := remote.NewChunkedWriter(&buf, nopFlusher{})
if _, err := cw.Write(data); err != nil {
t.Fatalf("cannot write chunked response: %s", err)
}
var tss []*vm.TimeSeries
err = processStreamResponse(io.NopCloser(&buf), func(ts *vm.TimeSeries) error {
tss = append(tss, ts)
return nil
})
if err != nil {
t.Fatalf("cannot process stream response: %s", err)
}
// _count + _sum + 4 buckets + 1 float series + 1 float series from UNKNOWN chunk
if len(tss) != 8 {
t.Fatalf("unexpected number of time series; got %d; want 8", len(tss))
}
if tss[0].Name != "request_duration_seconds_count" || !reflect.DeepEqual(tss[0].Values, []float64{10}) {
t.Fatalf("unexpected _count series: %v", tss[0])
}
if tss[1].Name != "request_duration_seconds_sum" || !reflect.DeepEqual(tss[1].Values, []float64{25.5}) {
t.Fatalf("unexpected _sum series: %v", tss[1])
}
for _, ts := range tss[2:6] {
if ts.Name != "request_duration_seconds_bucket" {
t.Fatalf("unexpected bucket series name %q", ts.Name)
}
}
if tss[6].Name != "cpu_usage" || !reflect.DeepEqual(tss[6].Values, []float64{1.5}) {
t.Fatalf("unexpected float series: %v", tss[6])
}
if tss[7].Name != "memory_usage" || !reflect.DeepEqual(tss[7].Values, []float64{1.5}) {
t.Fatalf("unexpected float series from UNKNOWN chunk: %v", tss[7])
}
}
func TestParseFloatHistograms(t *testing.T) {
c := chunkenc.NewFloatHistogramChunk()
app, err := c.Appender()
if err != nil {
t.Fatalf("cannot create chunk appender: %s", err)
}
fh := testHistogram(1).ToFloat(nil)
if _, _, _, err := app.AppendFloatHistogram(nil, 0, 1000, fh, true); err != nil {
t.Fatalf("cannot append float histogram: %s", err)
}
hSamples, err := parseHistograms(prompb.Chunk_FLOAT_HISTOGRAM, c.Bytes())
if err != nil {
t.Fatalf("cannot parse float histogram chunk: %s", err)
}
if len(hSamples) != 1 {
t.Fatalf("unexpected number of histogram samples; got %d; want 1", len(hSamples))
}
if hSamples[0].timestamp != 1000 {
t.Fatalf("unexpected timestamp; got %d; want 1000", hSamples[0].timestamp)
}
if hSamples[0].fh.Count != 10 {
t.Fatalf("unexpected count; got %f; want 10", hSamples[0].fh.Count)
}
}

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -37,7 +37,7 @@
<meta property="og:title" content="UI for VictoriaMetrics">
<meta property="og:url" content="https://victoriametrics.com/">
<meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data">
<script type="module" crossorigin src="./assets/index-D5egN2id.js"></script>
<script type="module" crossorigin src="./assets/index-B1dXK3k7.js"></script>
<link rel="modulepreload" crossorigin href="./assets/rolldown-runtime-CNC7AqOf.js">
<link rel="modulepreload" crossorigin href="./assets/vendor-DwJYpOdw.js">
<link rel="stylesheet" crossorigin href="./assets/vendor-CnsZ1jie.css">

View File

@@ -0,0 +1,5 @@
<svg width="48" height="48" fill="#020202" xmlns="http://www.w3.org/2000/svg">
<path d="M24.5475 0C10.3246.0265251 1.11379 3.06365 4.40623 6.10077c0 0 12.32997 11.23333 16.58217 14.84083.8131.6896 2.1728 1.1936 3.5191 1.2201h.1199c1.3463-.0265 2.706-.5305 3.5191-1.2201 4.2522-3.5942 16.5422-14.84083 16.5422-14.84083C48.0478 3.06365 38.8636.0265251 24.6674 0"/>
<path d="M28.1579 27.0159c-.8131.6896-2.1728 1.1936-3.5191 1.2201h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2201-2.9725-2.5067-13.35639-11.87-17.26201-15.3979v5.4112c0 .5968.22661 1.3793.6265 1.7506C7.00358 21.1936 17.2675 30.5437 20.9731 33.6737c.8132.6896 2.1728 1.1936 3.5191 1.2201h.12c1.3463-.0265 2.7059-.5305 3.519-1.2201 3.679-3.13 13.9429-12.4536 16.6089-14.8939.4132-.3713.6265-1.1538.6265-1.7506V11.618c-3.9323 3.5411-14.3162 12.931-17.2354 15.3979h.0267Z"/>
<path d="M28.1579 39.748c-.8131.6897-2.1728 1.1937-3.5191 1.2202h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2202-2.9725-2.4933-13.35639-11.8567-17.26201-15.3978v5.4111c0 .5969.22661 1.3793.6265 1.7507C7.00358 33.9258 17.2675 43.2759 20.9731 46.4058c.8132.6897 2.1728 1.1937 3.5191 1.2202h.12c1.3463-.0265 2.7059-.5305 3.519-1.2202 3.679-3.1299 13.9429-12.4535 16.6089-14.8938.4132-.3714.6265-1.1538.6265-1.7507v-5.4111c-3.9323 3.5411-14.3162 12.931-17.2354 15.3978h.0267Z"/>
</svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

View File

@@ -2,9 +2,9 @@
<html lang="en">
<head>
<meta charset="utf-8"/>
<link rel="icon" href="/favicon.svg"/>
<link rel="apple-touch-icon" href="/favicon.svg"/>
<link rel="mask-icon" href="/favicon.svg" color="#000000">
<link id="favicon" rel="icon" href="/assets/favicon.svg" />
<link rel="apple-touch-icon" href="/assets/favicon.svg" />
<link id="mask-icon" rel="mask-icon" href="/assets/favicon.svg?no-inline" color="#000000">
<meta name="robots" content="noindex">
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=5"/>

View File

@@ -1 +0,0 @@
<svg width="48" height="48" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M24.5475 0C10.3246.0265251 1.11379 3.06365 4.40623 6.10077c0 0 12.32997 11.23333 16.58217 14.84083.8131.6896 2.1728 1.1936 3.5191 1.2201h.1199c1.3463-.0265 2.706-.5305 3.5191-1.2201 4.2522-3.5942 16.5422-14.84083 16.5422-14.84083C48.0478 3.06365 38.8636.0265251 24.6674 0" fill="#020202"/><path d="M28.1579 27.0159c-.8131.6896-2.1728 1.1936-3.5191 1.2201h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2201-2.9725-2.5067-13.35639-11.87-17.26201-15.3979v5.4112c0 .5968.22661 1.3793.6265 1.7506C7.00358 21.1936 17.2675 30.5437 20.9731 33.6737c.8132.6896 2.1728 1.1936 3.5191 1.2201h.12c1.3463-.0265 2.7059-.5305 3.519-1.2201 3.679-3.13 13.9429-12.4536 16.6089-14.8939.4132-.3713.6265-1.1538.6265-1.7506V11.618c-3.9323 3.5411-14.3162 12.931-17.2354 15.3979h.0267Z" fill="#020202"/><path d="M28.1579 39.748c-.8131.6897-2.1728 1.1937-3.5191 1.2202h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2202-2.9725-2.4933-13.35639-11.8567-17.26201-15.3978v5.4111c0 .5969.22661 1.3793.6265 1.7507C7.00358 33.9258 17.2675 43.2759 20.9731 46.4058c.8132.6897 2.1728 1.1937 3.5191 1.2202h.12c1.3463-.0265 2.7059-.5305 3.519-1.2202 3.679-3.1299 13.9429-12.4535 16.6089-14.8938.4132-.3714.6265-1.1538.6265-1.7507v-5.4111c-3.9323 3.5411-14.3162 12.931-17.2354 15.3978h.0267Z" fill="#020202"/></svg>

Before

Width:  |  Height:  |  Size: 1.3 KiB

View File

@@ -3,7 +3,7 @@
"name": "vmui",
"icons": [
{
"src": "favicon.svg",
"src": "./assets/favicon.svg",
"sizes": "any",
"type": "image/svg+xml"
}

View File

@@ -0,0 +1,61 @@
import { FC, useMemo } from "preact/compat";
import "./style.scss";
import { createFaviconUrl } from "../../../../utils/favicon";
import classNames from "classnames";
import { useBrowserTabSync } from "./hooks/useBrowserTabSync";
import { CloseIcon } from "../../../Main/Icons";
import { faviconColors } from "../../../../constants/faviconColors";
const BrowserTabController: FC = () => {
const { faviconColor, changeFaviconColor } = useBrowserTabSync();
const faviconUrl = useMemo(() => {
return createFaviconUrl(faviconColor);
}, [faviconColor]);
const createHandlerClick = (color?: string) => () => {
changeFaviconColor(color);
};
return (
<div className="vm-browser-tab-controller">
<p className="vm-server-configurator__title">Favicon color</p>
<div className="vm-browser-tab-controller-palette">
<div className="vm-browser-tab-controller-palette-list">
<button
className="vm-browser-tab-controller-palette-list__item vm-browser-tab-controller-palette-list__item_reset"
type="button"
onClick={createHandlerClick()}
aria-label="Reset favicon color"
>
<CloseIcon />
</button>
{faviconColors.map(color => (
<button
className={classNames({
"vm-browser-tab-controller-palette-list__item": true,
"vm-browser-tab-controller-palette-list__item_selected": faviconColor === color
})}
key={color}
type="button"
style={{ color }}
onClick={createHandlerClick(color)}
aria-label={`Set favicon color to ${color}`}
aria-pressed={faviconColor === color}
/>
))}
</div>
<img
className="vm-browser-tab-controller-palette__preview"
src={faviconUrl}
alt="Favicon preview"
/>
</div>
</div>
);
};
export default BrowserTabController;

View File

@@ -0,0 +1,41 @@
import useEventListener from "../../../../../hooks/useEventListener";
import { useEffect, useState } from "preact/compat";
import { getFromStorage, removeFromStorage, saveToStorage } from "../../../../../utils/storage";
import { getFaviconStorageKey, updateFaviconColor } from "../../../../../utils/favicon";
const storageKey = `FAVICON_COLOR:${getFaviconStorageKey()}` as const;
const getColorFromStorage = () => {
return getFromStorage(storageKey) as string | undefined;
};
export const useBrowserTabSync = () => {
const [faviconColor, setFaviconColor] = useState(getColorFromStorage);
const handleUpdateColor = () => {
setFaviconColor(getColorFromStorage());
};
const changeFaviconColor = (color?: string) => {
if (color) {
saveToStorage(storageKey, color);
} else {
removeFromStorage([storageKey]);
}
};
useEffect(() => {
handleUpdateColor();
}, []);
useEffect(() => {
updateFaviconColor(faviconColor);
}, [faviconColor]);
useEventListener("storage", handleUpdateColor);
return {
faviconColor,
changeFaviconColor,
};
};

View File

@@ -0,0 +1,71 @@
@use "src/styles/variables" as *;
$color-item-size: 28px;
$outline-width: 2px;
$outline-offset: 2px;
$outline-space: $outline-width + $outline-offset;
.vm-browser-tab-controller {
.vm-server-configurator__title {
padding-bottom: 2px;
}
&-palette {
display: flex;
align-items: flex-start;
justify-content: space-between;
gap: calc($padding-large * 2);
&-list {
display: flex;
align-items: center;
justify-content: flex-start;
flex-wrap: wrap;
gap: $padding-small;
&__item {
width: $color-item-size;
height: $color-item-size;
aspect-ratio: 1;
border-radius: 50%;
background-color: currentColor;
cursor: pointer;
transition-property: transform;
transition-duration: 0.15s;
transition-timing-function: linear;
&:hover {
transform: scale(1.1);
}
&:focus-visible {
transform: scale(1.1);
}
&_reset {
display: flex;
align-items: center;
justify-content: center;
background: transparent;
border: $border-divider;
color: $color-text-disabled;
padding: calc($padding-small / 2);
}
&_selected {
width: $color-item-size - 2 * $outline-space;
height: $color-item-size - 2 * $outline-space;
margin: $outline-space;
outline: $outline-width solid currentColor;
outline-offset: $outline-offset;
pointer-events: none;
}
}
}
&__preview {
width: $color-item-size;
height: auto;
}
}
}

View File

@@ -8,10 +8,12 @@ import Tooltip from "../../Main/Tooltip/Tooltip";
import LimitsConfigurator from "./LimitsConfigurator/LimitsConfigurator";
import { getAppModeEnable } from "../../../utils/app-mode";
import classNames from "classnames";
import Timezones from "./Timezones/Timezones";
import TimezonesPicker from "./Timezones/TimezonesPicker";
import ThemeControl from "../ThemeControl/ThemeControl";
import useDeviceDetect from "../../../hooks/useDeviceDetect";
import useBoolean from "../../../hooks/useBoolean";
import BrowserTabController from "./BrowserTabController/BrowserTabController";
import LegendCollapseController from "./LegendCollapseController/LegendCollapseController";
const title = "Settings";
@@ -26,7 +28,6 @@ const GlobalSettings: FC = () => {
const serverSettingRef = useRef<ChildComponentHandle>(null);
const limitsSettingRef = useRef<ChildComponentHandle>(null);
const timezoneSettingRef = useRef<ChildComponentHandle>(null);
const {
value: open,
@@ -37,7 +38,6 @@ const GlobalSettings: FC = () => {
const handleApply = () => {
serverSettingRef.current && serverSettingRef.current.handleApply();
limitsSettingRef.current && limitsSettingRef.current.handleApply();
timezoneSettingRef.current && timezoneSettingRef.current.handleApply();
handleClose();
};
@@ -49,6 +49,10 @@ const GlobalSettings: FC = () => {
onClose={handleClose}
/>
},
{
show: true,
component: <TimezonesPicker/>
},
{
show: true,
component: <LimitsConfigurator
@@ -58,12 +62,16 @@ const GlobalSettings: FC = () => {
},
{
show: true,
component: <Timezones ref={timezoneSettingRef}/>
component: <LegendCollapseController/>
},
{
show: !appModeEnable,
component: <ThemeControl/>
}
},
{
show: true,
component: <BrowserTabController/>
},
].filter(control => control.show);
return <>

View File

@@ -0,0 +1,30 @@
import { FC, useEffect, useState } from "preact/compat";
import { getFromStorage, saveToStorage } from "../../../../utils/storage";
import Switch from "../../../Main/Switch/Switch";
import { LEGEND_COLLAPSE_SERIES_LIMIT } from "../../../../constants/graph";
import "./style.scss";
const LegendCollapseController: FC = () => {
const storageCollapse = getFromStorage("LEGEND_AUTO_COLLAPSE");
const [legendCollapse, setLegendCollapse] = useState(storageCollapse ? storageCollapse === "true" : true);
useEffect(() => {
saveToStorage("LEGEND_AUTO_COLLAPSE", `${legendCollapse}`);
}, [legendCollapse]);
return (
<div className="vm-legend-collapse-controller">
<Switch
fullWidth
color="neutral"
value={legendCollapse}
onChange={setLegendCollapse}
label={<span className="vm-server-configurator__title">Auto-collapse legend</span>}
/>
<span className="vm-legend-collapse-controller__description">
Collapses the legend when series count exceeds {LEGEND_COLLAPSE_SERIES_LIMIT} to reduce UI load.
</span>
</div>);
};
export default LegendCollapseController;

View File

@@ -0,0 +1,19 @@
@use "src/styles/variables" as *;
.vm-legend-collapse-controller {
background-color: $color-hover-black;
border-radius: $border-radius-medium;
padding: $padding-large;
border: $border-divider;
.vm-graph-settings-row__label {
margin: 0;
}
&__description {
padding-top: $padding-global;
font-size: $font-size-small;
line-height: 1.3;
text-wrap: pretty;
}
}

View File

@@ -1,23 +1,21 @@
import { forwardRef, useCallback, useEffect, useImperativeHandle, useState } from "preact/compat";
import { forwardRef, useCallback, useImperativeHandle, useState } from "preact/compat";
import { DisplayType, ErrorTypes } from "../../../../types";
import TextField from "../../../Main/TextField/TextField";
import Tooltip from "../../../Main/Tooltip/Tooltip";
import { InfoIcon, RestartIcon } from "../../../Main/Icons";
import Button from "../../../Main/Button/Button";
import { DEFAULT_MAX_SERIES, LEGEND_COLLAPSE_SERIES_LIMIT } from "../../../../constants/graph";
import { DEFAULT_MAX_SERIES } from "../../../../constants/graph";
import "./style.scss";
import classNames from "classnames";
import useDeviceDetect from "../../../../hooks/useDeviceDetect";
import { ChildComponentHandle } from "../GlobalSettings";
import { useCustomPanelDispatch, useCustomPanelState } from "../../../../state/customPanel/CustomPanelStateContext";
import Switch from "../../../Main/Switch/Switch";
import { getFromStorage, saveToStorage } from "../../../../utils/storage";
interface ServerConfiguratorProps {
onClose: () => void
onClose: () => void;
}
const fields: {label: string, type: DisplayType}[] = [
const fields: { label: string, type: DisplayType }[] = [
{ label: "Graph", type: DisplayType.chart },
{ label: "JSON", type: DisplayType.code },
{ label: "Table", type: DisplayType.table }
@@ -29,8 +27,7 @@ const LimitsConfigurator = forwardRef<ChildComponentHandle, ServerConfiguratorPr
const { seriesLimits } = useCustomPanelState();
const customPanelDispatch = useCustomPanelDispatch();
const storageCollapse = getFromStorage("LEGEND_AUTO_COLLAPSE");
const [legendCollapse, setLegendCollapse] = useState(storageCollapse ? storageCollapse === "true" : true);
const [limits, setLimits] = useState(seriesLimits);
const [error, setError] = useState({
@@ -43,7 +40,7 @@ const LimitsConfigurator = forwardRef<ChildComponentHandle, ServerConfiguratorPr
setLimits(DEFAULT_MAX_SERIES);
};
const createChangeHandler = (type: DisplayType) => (val: string) => {
const createChangeHandler = (type: DisplayType) => (val: string) => {
const value = val || "";
setError(prev => ({ ...prev, [type]: +value < 0 ? ErrorTypes.positiveNumber : "" }));
setLimits({
@@ -57,10 +54,6 @@ const LimitsConfigurator = forwardRef<ChildComponentHandle, ServerConfiguratorPr
onClose();
}, [limits]);
useEffect(() => {
saveToStorage("LEGEND_AUTO_COLLAPSE", `${legendCollapse}`);
}, [legendCollapse]);
useImperativeHandle(ref, () => ({ handleApply }), [handleApply]);
return (
@@ -106,19 +99,6 @@ const LimitsConfigurator = forwardRef<ChildComponentHandle, ServerConfiguratorPr
</div>
))}
</div>
<div className="vm-graph-settings-row">
<span className="vm-graph-settings-row__label">Auto-collapse legend</span>
<Switch
value={legendCollapse}
onChange={setLegendCollapse}
label={legendCollapse ? "Enabled" : "Disabled"}
fullWidth={isMobile}
/>
<span className="vm-legend-configs-item__info">
Collapses the legend when series count exceeds {LEGEND_COLLAPSE_SERIES_LIMIT} to reduce UI load.
</span>
</div>
</div>
);
});

View File

@@ -1,183 +0,0 @@
import { FC, forwardRef, useEffect, useImperativeHandle, useMemo, useRef, useState } from "preact/compat";
import { getBrowserTimezone, getTimezoneList, getUTCByTimezone } from "../../../../utils/time";
import { ArrowDropDownIcon } from "../../../Main/Icons";
import classNames from "classnames";
import Popper from "../../../Main/Popper/Popper";
import Accordion from "../../../Main/Accordion/Accordion";
import TextField from "../../../Main/TextField/TextField";
import { Timezone } from "../../../../types";
import "./style.scss";
import useDeviceDetect from "../../../../hooks/useDeviceDetect";
import useBoolean from "../../../../hooks/useBoolean";
import WarningTimezone from "./WarningTimezone";
import { useTimeDispatch, useTimeState } from "../../../../state/time/TimeStateContext";
interface PinnedTimezone extends Timezone {
title: string;
isInvalid?: boolean;
}
const browserTimezone = getBrowserTimezone();
const Timezones: FC = forwardRef((props, ref) => {
const { isMobile } = useDeviceDetect();
const timezones = getTimezoneList();
const { timezone: stateTimezone, defaultTimezone } = useTimeState();
const timeDispatch = useTimeDispatch();
const [timezone, setTimezone] = useState(stateTimezone);
const [search, setSearch] = useState("");
const targetRef = useRef<HTMLDivElement>(null);
const {
value: openList,
toggle: toggleOpenList,
setFalse: handleCloseList,
} = useBoolean(false);
const pinnedTimezones = useMemo(() => [
{
title: `Default time (${defaultTimezone})`,
region: defaultTimezone,
utc: defaultTimezone ? getUTCByTimezone(defaultTimezone) : "UTC"
},
{
title: browserTimezone.title,
region: browserTimezone.region,
utc: getUTCByTimezone(browserTimezone.region),
isInvalid: !browserTimezone.isValid
},
{
title: "UTC (Coordinated Universal Time)",
region: "UTC",
utc: "UTC"
},
].filter(t => t.region) as PinnedTimezone[], [defaultTimezone]);
const searchTimezones = useMemo(() => {
if (!search) return timezones;
try {
return getTimezoneList(search);
} catch (e) {
return {};
}
}, [search, timezones]);
const timezonesGroups = useMemo(() => Object.keys(searchTimezones), [searchTimezones]);
const activeTimezone = useMemo(() => ({
region: timezone,
utc: getUTCByTimezone(timezone)
}), [timezone]);
const handleChangeSearch = (val: string) => {
setSearch(val);
};
const handleSetTimezone = (val: Timezone) => {
setTimezone(val.region);
setSearch("");
handleCloseList();
};
const createHandlerSetTimezone = (val: Timezone) => () => {
handleSetTimezone(val);
};
useEffect(() => {
setTimezone(stateTimezone);
}, [stateTimezone]);
useImperativeHandle(ref, () => ({
handleApply: () => {
timeDispatch({ type: "SET_TIMEZONE", payload: timezone });
}
}), [timezone]);
return (
<div className="vm-timezones">
<div className="vm-server-configurator__title">
Time zone
</div>
<div
className="vm-timezones-item vm-timezones-item_selected"
onClick={toggleOpenList}
ref={targetRef}
>
<div className="vm-timezones-item__title">{activeTimezone.region}</div>
<div className="vm-timezones-item__utc">{activeTimezone.utc}</div>
<div
className={classNames({
"vm-timezones-item__icon": true,
"vm-timezones-item__icon_open": openList
})}
>
<ArrowDropDownIcon/>
</div>
</div>
<Popper
open={openList}
buttonRef={targetRef}
placement="bottom-left"
onClose={handleCloseList}
fullWidth
title={isMobile ? "Time zone" : undefined}
>
<div
className={classNames({
"vm-timezones-list": true,
"vm-timezones-list_mobile": isMobile,
})}
>
<div className="vm-timezones-list-header">
<div className="vm-timezones-list-header__search">
<TextField
autofocus
label="Search"
value={search}
onChange={handleChangeSearch}
/>
</div>
{pinnedTimezones.map((t, i) => t && (
<div
key={`${i}_${t.region}`}
className="vm-timezones-item vm-timezones-list-group-options__item"
onClick={createHandlerSetTimezone(t)}
>
<div className="vm-timezones-item__title">{t.title}{t.isInvalid && <WarningTimezone/>}</div>
<div className="vm-timezones-item__utc">{t.utc}</div>
</div>
))}
</div>
{timezonesGroups.map(t => (
<div
className="vm-timezones-list-group"
key={t}
>
<Accordion
defaultExpanded={true}
title={<div className="vm-timezones-list-group__title">{t}</div>}
>
<div className="vm-timezones-list-group-options">
{searchTimezones[t] && searchTimezones[t].map(item => (
<div
className="vm-timezones-item vm-timezones-list-group-options__item"
onClick={createHandlerSetTimezone(item)}
key={item.search}
>
<div className="vm-timezones-item__title">{item.region}</div>
<div className="vm-timezones-item__utc">{item.utc}</div>
</div>
))}
</div>
</Accordion>
</div>
))}
</div>
</Popper>
</div>
);
});
export default Timezones;

View File

@@ -0,0 +1,129 @@
import { FC, useMemo, useState } from "preact/compat";
import { getBrowserTimezone, getTimezoneList, getUTCByTimezone } from "../../../../utils/time";
import classNames from "classnames";
import Accordion from "../../../Main/Accordion/Accordion";
import TextField from "../../../Main/TextField/TextField";
import { Timezone } from "../../../../types";
import "./style.scss";
import useDeviceDetect from "../../../../hooks/useDeviceDetect";
import WarningTimezone from "./WarningTimezone";
import { useTimeState } from "../../../../state/time/TimeStateContext";
interface PinnedTimezone extends Timezone {
title: string;
isInvalid?: boolean;
}
type Props = {
onChange: (tz: Timezone) => void;
}
const browserTimezone = getBrowserTimezone();
const TimezonesList: FC<Props> = ({ onChange }) => {
const { isMobile } = useDeviceDetect();
const { defaultTimezone } = useTimeState();
const timezones = useMemo(() => getTimezoneList(), []);
const [search, setSearch] = useState("");
const pinnedTimezones = useMemo(() => [
{
title: `Default time (${defaultTimezone})`,
region: defaultTimezone,
utc: defaultTimezone ? getUTCByTimezone(defaultTimezone) : "UTC"
},
{
title: browserTimezone.title,
region: browserTimezone.region,
utc: getUTCByTimezone(browserTimezone.region),
isInvalid: !browserTimezone.isValid
},
{
title: "UTC (Coordinated Universal Time)",
region: "UTC",
utc: "UTC"
},
].filter(t => t.region) as PinnedTimezone[], [defaultTimezone]);
const searchTimezones = useMemo(() => {
if (!search) return timezones;
try {
return getTimezoneList(search);
} catch (e) {
return {};
}
}, [search, timezones]);
const timezonesGroups = useMemo(() => Object.keys(searchTimezones), [searchTimezones]);
const handleChangeSearch = (val: string) => {
setSearch(val);
};
const handleSetTimezone = (tz: Timezone) => {
onChange(tz);
setSearch("");
};
const createHandlerSetTimezone = (val: Timezone) => () => {
handleSetTimezone(val);
};
return (
<div
className={classNames({
"vm-list": true,
"vm-timezones-list": true,
"vm-timezones-list_mobile": isMobile,
})}
>
<div className="vm-timezones-list-header">
<div className="vm-timezones-list-header__search">
<TextField
label="Search"
value={search}
onChange={handleChangeSearch}
/>
</div>
</div>
{pinnedTimezones.map((t, i) => t && (
<div
key={`${i}_${t.region}`}
className="vm-list-item vm-timezones-item vm-timezones-list-group-options__item"
onClick={createHandlerSetTimezone(t)}
>
<div className="vm-timezones-item__title">{t.title}{t.isInvalid && <WarningTimezone/>}</div>
<div className="vm-timezones-item__utc">{t.utc}</div>
</div>
))}
{timezonesGroups.map(t => (
<div
className="vm-timezones-list-group"
key={t}
>
<Accordion
defaultExpanded={true}
title={<div className="vm-timezones-list-group__title">{t}</div>}
>
<div className="vm-timezones-list-group-options">
{searchTimezones[t] && searchTimezones[t].map(item => (
<div
className="vm-list-item vm-timezones-item vm-timezones-list-group-options__item"
onClick={createHandlerSetTimezone(item)}
key={item.search}
>
<div className="vm-timezones-item__title">{item.region}</div>
<div className="vm-timezones-item__utc">{item.utc}</div>
</div>
))}
</div>
</Accordion>
</div>
))}
</div>
);
};
export default TimezonesList;

View File

@@ -0,0 +1,71 @@
import { FC, useMemo, useRef } from "preact/compat";
import { getUTCByTimezone } from "../../../../utils/time";
import { ArrowDropDownIcon } from "../../../Main/Icons";
import classNames from "classnames";
import { Timezone } from "../../../../types";
import "./style.scss";
import useBoolean from "../../../../hooks/useBoolean";
import { useTimeDispatch, useTimeState } from "../../../../state/time/TimeStateContext";
import TimezonesList from "./TimezonesList";
import Popper from "../../../Main/Popper/Popper";
import useDeviceDetect from "../../../../hooks/useDeviceDetect";
const TimezonesPicker: FC = () => {
const { isMobile } = useDeviceDetect();
const { timezone: stateTimezone } = useTimeState();
const timeDispatch = useTimeDispatch();
const triggerRef = useRef<HTMLDivElement>(null);
const {
value: isOpenList,
toggle: toggleOpenList,
setFalse: handleCloseList,
} = useBoolean(false);
const activeTimezone = useMemo(() => ({
region: stateTimezone,
utc: getUTCByTimezone(stateTimezone)
}), [stateTimezone]);
const handleSetTimezone = (tz: Timezone) => {
timeDispatch({ type: "SET_TIMEZONE", payload: tz.region });
handleCloseList();
};
return (
<div className="vm-timezones">
<div className="vm-server-configurator__title">
Time zone
</div>
<div
className="vm-timezones-item vm-timezones-item_selected"
onClick={toggleOpenList}
ref={triggerRef}
>
<div className="vm-timezones-item__title">{activeTimezone.region}</div>
<div className="vm-timezones-item__utc">{activeTimezone.utc}</div>
<div
className={classNames({
"vm-timezones-item__icon": true,
"vm-timezones-item__icon_open": isOpenList
})}
>
<ArrowDropDownIcon/>
</div>
</div>
<Popper
open={isOpenList}
buttonRef={triggerRef}
placement="bottom-left"
onClose={handleCloseList}
fullWidth
title={isMobile ? "Time zone" : undefined}
>
<TimezonesList onChange={handleSetTimezone}/>
</Popper>
</div>
);
};
export default TimezonesPicker;

View File

@@ -16,6 +16,7 @@
}
&__title {
flex-grow: 1;
display: flex;
align-items: center;
gap: $padding-small;
@@ -34,6 +35,7 @@
background-color: $color-hover-black;
padding: calc($padding-small/2);
border-radius: $border-radius-small;
font-size: $font-size-small;
}
&__icon {
@@ -54,9 +56,11 @@
}
&-list {
padding-top: 0;
max-height: 300px;
background-color: $color-background-block;
border-radius: $border-radius-medium;
font-size: $font-size-small;
overflow: auto;
&_mobile {
@@ -72,10 +76,9 @@
top: 0;
background-color: $color-background-block;
z-index: 2;
border-bottom: $border-divider;
&__search {
padding: $padding-small;
padding: $padding-small $padding-small calc($padding-small / 2);
}
}
@@ -91,6 +94,7 @@
font-weight: bold;
color: $color-text-secondary;
padding: $padding-small $padding-global;
font-size: $font-size-small;
}
&-options {
@@ -98,7 +102,7 @@
align-items: flex-start;
&__item {
padding: $padding-small $padding-global;
padding: calc($padding-small / 2) $padding-global;
transition: background-color 200ms ease;
&:hover {

View File

@@ -4,9 +4,9 @@
display: flex;
flex-direction: column;
align-items: center;
gap: $padding-large;
gap: calc($padding-global * 2);
width: 600px;
padding-bottom: $padding-medium;
padding-inline: $padding-large;
&_mobile {
grid-auto-rows: min-content;
@@ -62,6 +62,7 @@
justify-content: flex-end;
gap: $padding-small;
width: 100%;
padding-block: $padding-global;
}
&_mobile &-footer {

View File

@@ -22,12 +22,10 @@ const StepConfigurator: FC = () => {
const { isMobile } = useDeviceDetect();
const { customStep: value, isHistogram } = useGraphState();
const { period: { step, end, start } } = useTimeState();
const { period: { end, start } } = useTimeState();
const graphDispatch = useGraphDispatch();
const { displayType } = useCustomPanelState();
const prevDuration = usePrevious(end - start);
const defaultStep = useMemo(() => {
return getStepFromDuration(end - start, isHistogram, displayType);
}, [end, start, isHistogram, displayType]);
@@ -106,16 +104,14 @@ const StepConfigurator: FC = () => {
}, [defaultStep]);
useEffect(() => {
const dur = end - start;
if (dur === prevDuration || !prevDuration || value !== prevDefaultStep) return;
if (defaultStep) {
handleApply(defaultStep);
}
}, [prevDuration, defaultStep]);
if (!prevDefaultStep) return;
if (value !== prevDefaultStep) return;
if (value === defaultStep) return;
useEffect(() => {
if (step === value || step === defaultStep) handleApply(defaultStep);
}, [isHistogram, displayType]);
graphDispatch({ type: "SET_CUSTOM_STEP", payload: defaultStep });
setCustomStep(defaultStep);
setError("");
}, [defaultStep, prevDefaultStep, value, graphDispatch]);
return (
<div

View File

@@ -5,8 +5,20 @@ import useDeviceDetect from "../../../hooks/useDeviceDetect";
import classNames from "classnames";
import { FC } from "preact/compat";
import { useAppDispatch, useAppState } from "../../../state/common/StateContext";
import { DarkIcon, LightIcon, SystemIcon } from "../../Main/Icons";
const themeIcons = {
[Theme.system]: <SystemIcon/>,
[Theme.light]: <LightIcon/>,
[Theme.dark]: <DarkIcon/>,
};
const options = Object.values(Theme).map(value => ({
title: value,
value,
icon: themeIcons[value],
}));
const options = Object.values(Theme).map(value => ({ title: value, value }));
const ThemeControl: FC = () => {
const { isMobile } = useDeviceDetect();
const dispatch = useAppDispatch();
@@ -25,13 +37,14 @@ const ThemeControl: FC = () => {
})}
>
<div className="vm-server-configurator__title">
Theme preferences
Theme
</div>
<div
className="vm-theme-control__toggle"
key={`${isMobile}`}
>
<Toggle
size="large"
options={options}
value={theme}
onChange={handleClickItem}

View File

@@ -4,7 +4,7 @@
&__toggle {
display: inline-flex;
min-width: 300px;
width: 100%;
text-transform: capitalize;
}

View File

@@ -633,3 +633,60 @@ export const DebugIcon = () => (
/>
</svg>
);
export const SystemIcon = () => (
<svg
viewBox="0 0 24 24"
fill="currentColor"
>
<path
d="M18 4C18.7957 4 19.5595 4.3163 20.1221 4.87891C20.6845 5.44148 21 6.20452 21 7V15.5264L21.0069 15.6426C21.0203 15.7579 21.0542 15.8702 21.1065 15.9746L22.1729 18.0996C22.3271 18.4056 22.4009 18.7466 22.3858 19.0889C22.3705 19.431 22.2675 19.7637 22.0869 20.0547C21.9063 20.3457 21.6534 20.5855 21.3535 20.751C21.0555 20.9154 20.7202 21.0003 20.3799 20.999L3.62013 21C3.28002 21.0012 2.94535 20.9153 2.64748 20.751C2.34748 20.5855 2.09477 20.3458 1.91408 20.0547C1.73343 19.7636 1.63047 19.4311 1.61525 19.0889C1.60006 18.7466 1.67297 18.4056 1.82716 18.0996L2.89455 15.9746L2.94045 15.8682C2.9801 15.7589 3.00007 15.6432 3.00002 15.5264V7C3.00002 6.20442 3.3164 5.4415 3.87892 4.87891C4.44146 4.31636 5.20447 4.00007 6.00002 4H18ZM4.62404 16.9873L3.61427 18.999L3.6133 19H20.3877L20.3867 18.999L19.376 16.9873H4.62404ZM6.00002 6C5.7349 6.00007 5.48045 6.1055 5.29298 6.29297C5.10554 6.48049 5.00002 6.73485 5.00002 7V14.9873H19V7C19 6.73478 18.8946 6.48051 18.707 6.29297C18.5195 6.10552 18.2652 6 18 6H6.00002Z"
/>
</svg>
);
export const LightIcon = () => (
<svg
viewBox="0 0 24 24"
fill="currentColor"
>
<path
d="M12 19C12.5523 19 13 19.4477 13 20V22C13 22.5523 12.5523 23 12 23C11.4477 23 11 22.5523 11 22V20C11 19.4477 11.4477 19 12 19Z"
/>
<path
d="M5.63281 16.9531C6.02334 16.5627 6.65638 16.5626 7.04688 16.9531C7.43717 17.3436 7.43725 17.9767 7.04688 18.3672L5.63672 19.7773C5.24625 20.1676 4.61313 20.1676 4.22266 19.7773C3.8322 19.3869 3.83233 18.7538 4.22266 18.3633L5.63281 16.9531Z"
/>
<path
d="M16.9531 16.9531C17.3436 16.5626 17.9767 16.5626 18.3672 16.9531L19.7773 18.3633C20.1676 18.7538 20.1678 19.3869 19.7773 19.7773C19.3869 20.1677 18.7538 20.1675 18.3633 19.7773L16.9531 18.3672C16.5626 17.9767 16.5627 17.3437 16.9531 16.9531Z"
/>
<path
d="M12 7C14.7614 7 17 9.23858 17 12C17 14.7614 14.7614 17 12 17C9.23858 17 7 14.7614 7 12C7 9.23858 9.23858 7 12 7ZM12 9C10.3431 9 9 10.3431 9 12C9 13.6569 10.3431 15 12 15C13.6569 15 15 13.6569 15 12C15 10.3431 13.6569 9 12 9Z"
/>
<path
d="M4 11C4.55228 11 5 11.4477 5 12C5 12.5523 4.55228 13 4 13H2C1.44772 13 1 12.5523 1 12C1 11.4477 1.44772 11 2 11H4Z"
/>
<path
d="M22 11C22.5523 11 23 11.4477 23 12C23 12.5523 22.5523 13 22 13H20C19.4477 13 19 12.5523 19 12C19 11.4477 19.4477 11 20 11H22Z"
/>
<path
d="M4.22266 4.22266C4.61315 3.83229 5.24623 3.83229 5.63672 4.22266L7.04688 5.63281C7.4372 6.02331 7.43723 6.65639 7.04688 7.04688C6.6564 7.43735 6.02335 7.43724 5.63281 7.04688L4.22266 5.63672C3.83225 5.24618 3.83217 4.61314 4.22266 4.22266Z"
/>
<path
d="M18.3633 4.22266C18.7538 3.83237 19.3869 3.83232 19.7773 4.22266C20.1677 4.61312 20.1676 5.2462 19.7773 5.63672L18.3672 7.04688C17.9767 7.4373 17.3436 7.4373 16.9531 7.04688C16.5627 6.65637 16.5627 6.0233 16.9531 5.63281L18.3633 4.22266Z"
/>
<path
d="M12 1C12.5523 1 13 1.44772 13 2V4C13 4.55228 12.5523 5 12 5C11.4477 5 11 4.55228 11 4V2C11 1.44772 11.4477 1 12 1Z"
/>
</svg>
);
export const DarkIcon = () => (
<svg
viewBox="0 0 24 24"
fill="currentColor"
>
<path
d="M11.5809 2.01318C12.1851 2.02907 12.6373 2.40742 12.8475 2.84717C13.0627 3.29758 13.0619 3.86893 12.7615 4.34814L12.7606 4.34717C12.1616 5.30581 11.9061 6.43987 12.034 7.56299C12.162 8.68628 12.6672 9.73328 13.4666 10.5327C14.2661 11.332 15.3131 11.8364 16.4363 11.9644C17.5596 12.0922 18.6934 11.836 19.6522 11.2368L19.8348 11.1382C20.2701 10.9405 20.7563 10.9617 21.1512 11.1499C21.6217 11.3742 22.0194 11.8752 21.9832 12.5405C21.8789 14.4693 21.2181 16.3268 20.0809 17.8882C18.9435 19.4496 17.378 20.6484 15.574 21.3394C13.7701 22.0302 11.8042 22.184 9.91485 21.7817C8.02549 21.3794 6.29356 20.4376 4.92754 19.0718C3.56149 17.7059 2.62012 15.9739 2.21758 14.0845C1.81507 12.195 1.96826 10.2294 2.65899 8.42529C3.34975 6.62115 4.5487 5.05596 6.11016 3.91846C7.6716 2.781 9.52882 2.11947 11.4578 2.01514L11.5809 2.01318ZM10.6209 4.12061C9.42038 4.33038 8.27873 4.81214 7.28692 5.53467C6.03798 6.44459 5.07973 7.69705 4.52715 9.14014C3.97456 10.5834 3.85165 12.156 4.17364 13.6675C4.49565 15.179 5.24872 16.5649 6.34161 17.6577C7.43448 18.7505 8.82026 19.5038 10.3318 19.8257C11.8434 20.1475 13.416 20.0239 14.8592 19.4712C16.3024 18.9184 17.5548 17.9597 18.4647 16.7104C19.1869 15.7188 19.6671 14.5776 19.8768 13.3774C18.7333 13.8927 17.4674 14.0949 16.2098 13.9517C14.637 13.7725 13.1709 13.0661 12.0516 11.9468C10.9324 10.8275 10.2258 9.36126 10.0467 7.78857C9.90352 6.53075 10.1054 5.26417 10.6209 4.12061Z"
/>
</svg>
);

View File

@@ -1,11 +1,10 @@
import { ReactNode } from "react";
import { FC, ReactNode } from "preact/compat";
import classNames from "classnames";
import "./style.scss";
import { FC } from "preact/compat";
interface SwitchProps {
value: boolean
color?: "primary" | "secondary" | "error"
color?: "primary" | "secondary" | "error" | "neutral"
disabled?: boolean
label?: string | ReactNode
fullWidth?: boolean

View File

@@ -29,6 +29,10 @@ $switch-border-radius: $switch-handle-size + ($switch-padding * 2);
background-color: $color-secondary;
}
&_neutral_active &-track {
background-color: $color-text;
}
&_primary_active &-track {
background-color: $color-primary;
}

View File

@@ -1,22 +1,21 @@
import { FC, useEffect, useRef, useState } from "preact/compat";
import { FC, useEffect, useRef, useState, ReactNode } from "preact/compat";
import classNames from "classnames";
import { ReactNode } from "react";
import "./style.scss";
interface ToggleProps {
options: {value: string, title?: string, icon?: ReactNode}[]
value: string
onChange: (val: string) => void
label?: string
options: { value: string, title?: string, icon?: ReactNode }[];
value: string;
onChange: (val: string) => void;
label?: string;
size?: "medium" | "large";
}
const Toggle: FC<ToggleProps> = ({ options, value, label, onChange }) => {
const Toggle: FC<ToggleProps> = ({ options, value, label, size = "medium", onChange }) => {
const activeRef = useRef<HTMLDivElement>(null);
const [position, setPosition] = useState({
width: "0px",
left: "0px",
borderRadius: "0px"
});
const createHandlerChange = (value: string) => () => {
@@ -28,35 +27,25 @@ const Toggle: FC<ToggleProps> = ({ options, value, label, onChange }) => {
setPosition({
width: "0px",
left: "0px",
borderRadius: "0px"
});
return;
}
const index = options.findIndex(o => o.value === value);
const { width: widthRect } = activeRef.current.getBoundingClientRect();
let width = widthRect;
let left = index * width;
let borderRadius = "0";
if (index === 0) borderRadius = "16px 0 0 16px";
const width = widthRect;
const left = index * width;
if (index === options.length - 1) {
borderRadius = "10px";
left -= 1;
borderRadius = "0 16px 16px 0";
}
if (index !== 0 && (index !== options.length - 1)) {
width += 1;
left -= 1;
}
setPosition({ width: `${width}px`, left: `${left}px`, borderRadius });
setPosition({ width: `${width}px`, left: `${left}px` });
}, [activeRef, value, options]);
return (
<div className="vm-toggles">
<div
className={classNames({
"vm-toggles": true,
[`vm-toggles_${size}`]: size,
})}
>
{label && (
<label className="vm-toggles__label">
{label}
@@ -66,15 +55,14 @@ const Toggle: FC<ToggleProps> = ({ options, value, label, onChange }) => {
className="vm-toggles-group"
style={{ gridTemplateColumns: `repeat(${options.length}, 1fr)` }}
>
{position.borderRadius && <div
<div
className="vm-toggles-group__highlight"
style={position}
/>}
{options.map((option, i) => (
/>
{options.map((option) => (
<div
className={classNames({
"vm-toggles-group-item": true,
"vm-toggles-group-item_first": i === 0,
"vm-toggles-group-item_active": option.value === value,
"vm-toggles-group-item_icon": option.icon && option.title
})}

View File

@@ -20,6 +20,8 @@
align-items: center;
justify-content: center;
overflow: hidden;
border-radius: $border-radius-small;
background: $color-hover-black;
&-item {
position: relative;
@@ -27,55 +29,68 @@
align-items: center;
justify-content: center;
padding: $padding-small;
border-right: $border-divider;
border-top: $border-divider;
border-bottom: $border-divider;
font-size: $font-size-small;
color: $color-text-secondary;
font-weight: bold;
font-weight: 500;
cursor: pointer;
text-align: center;
transition: color 150ms ease-in;
transition: opacity 150ms ease-in, color 150ms ease-in;
z-index: 2;
user-select: none;
&_first {
border-radius: 16px 0 0 16px;
border-left: $border-divider
}
&:last-child {
border-radius: 0 16px 16px 0;
border-left: none;
}
&_icon {
grid-template-columns: 14px auto;
gap: 4px;
gap: calc($padding-small / 2);
}
&:hover {
color: $color-primary;
&:hover:not(&_active) {
opacity: 0.8;
}
&_active {
color: $color-primary;
border-color: transparent;
&:hover {
background-color: transparent;
}
color: $color-text;
font-weight: 600;
}
}
&__highlight {
position: absolute;
top: 0;
display: flex;
align-items: center;
justify-content: center;
padding: 3px;
height: 100%;
background-color: rgba($color-primary, 0.08);
border: 1px solid $color-primary;
transition: left 200ms cubic-bezier(0.280, 0.840, 0.420, 1), border-radius 200ms linear;
z-index: 1;
&:after {
content: '';
height: 100%;
width: 100%;
background-color: $color-background-block;
border-radius: $border-radius-small;
box-shadow: $box-shadow;
}
}
}
&_large &-group {
border-radius: $border-radius-medium;
&-item {
padding: $padding-global $padding-small;
&_icon {
grid-template-columns: 16px auto;
gap: $padding-small;
}
}
&__highlight {
padding: 4px;
border-radius: $border-radius-medium;
}
}
}

View File

@@ -0,0 +1,14 @@
export const faviconColors = [
"#A1A1AA",
"#71717A",
"#020202",
"#E94600",
"#FF7A00",
"#F2B705",
"#84CC16",
"#16B86A",
"#00AFAF",
"#2979FF",
"#8B5CF6",
"#E83E9A",
] as const;

View File

@@ -14,6 +14,9 @@ import useFetchDefaultTimezone from "../../hooks/useFetchDefaultTimezone";
import useFetchAppConfig from "../../hooks/useFetchAppConfig";
import WebStorageCheck from "../../components/WebStorageCheck/WebStorageCheck";
import { migrateStorageToPrefixedKeys } from "../../utils/storage";
import {
useBrowserTabSync
} from "../../components/Configurators/GlobalSettings/BrowserTabController/hooks/useBrowserTabSync";
const MainLayout: FC = () => {
const appModeEnable = getAppModeEnable();
@@ -21,6 +24,7 @@ const MainLayout: FC = () => {
const { pathname } = useLocation();
const [searchParams, setSearchParams] = useSearchParams();
useBrowserTabSync();
useFetchDashboards();
useFetchDefaultTimezone();
useFetchAppConfig();

View File

@@ -0,0 +1,29 @@
import faviconRaw from "../../assets/favicon.svg?raw";
export const createFaviconUrl = (color = "#020202"): string => {
const svgDocument = new DOMParser().parseFromString(faviconRaw, "image/svg+xml");
const svg = svgDocument.documentElement;
if (svg.localName !== "svg") {
throw new Error("Invalid favicon SVG");
}
svg.setAttribute("fill", color);
const serializedSvg = new XMLSerializer().serializeToString(svg);
return `data:image/svg+xml;charset=utf-8,${encodeURIComponent(serializedSvg)}`;
};
export const updateFaviconColor = (color = "#020202"): void => {
const favicon = document.querySelector<HTMLLinkElement>("#favicon");
if (favicon) {
favicon.href = createFaviconUrl(color);
}
const maskIcon = document.querySelector<HTMLLinkElement>("#mask-icon");
if (maskIcon) {
maskIcon.setAttribute("color", color);
}
};
export const getFaviconStorageKey = () => window.location.pathname.replace(/\/+$/, "") || "/";

View File

@@ -17,7 +17,11 @@ export const ALL_STORAGE_KEYS = [
"POINTS_SHOW_ALL",
] as const;
export type StorageKeys = (typeof ALL_STORAGE_KEYS)[number];
export type FaviconStorageKey = `FAVICON_COLOR:${string}`;
export type StorageKeys =
| (typeof ALL_STORAGE_KEYS)[number]
| FaviconStorageKey;
type PrefixedStorageKeys = `${typeof STORAGE_PREFIX}${StorageKeys}`;
@@ -58,7 +62,10 @@ export const getFromStorage = (key: StorageKeys, withPrefix = true): undefined |
export const removeFromStorage = (keys: StorageKeys[], withPrefix = true): void => {
const storageKeys = withPrefix ? keys.map(toPrefixedKey) : keys;
storageKeys.forEach(k => window.localStorage.removeItem(k));
storageKeys.forEach(k => {
window.localStorage.removeItem(k);
window.dispatchEvent(new StorageEvent("storage", { key: k }));
});
};
/**

View File

@@ -205,19 +205,21 @@ export const getUTCByTimezone = (timezone: string) => {
};
export const getTimezoneList = (search = "") => {
const regexp = new RegExp(search, "i");
const normalizedSearch = search.toLowerCase();
return supportedTimezones.reduce((acc: {[key: string]: Timezone[]}, region) => {
return supportedTimezones.reduce((acc: { [key: string]: Timezone[] }, region) => {
const zone = (region.match(/^(.*?)\//) || [])[1] || "unknown";
const utc = getUTCByTimezone(region);
const utcForSearch = utc.replace(/UTC|0/, "");
const utcForSearch = utc.replace(/^UTC/, "");
const regionForSearch = region.replace(/[/_]/g, " ");
const item = {
region,
utc,
search: `${region} ${utc} ${regionForSearch} ${utcForSearch}`
};
const includeZone = !search || (search && regexp.test(item.search));
const includeZone = !normalizedSearch || item.search.toLowerCase().includes(normalizedSearch);
if (includeZone && acc[zone]) {
acc[zone].push(item);

View File

@@ -50,6 +50,13 @@ export default defineConfig(() => {
return "vendor";
}
},
assetFileNames: (assetInfo) => {
if (assetInfo.names.includes("favicon.svg")) {
return "assets/favicon.svg";
}
return "assets/[name]-[hash][extname]";
},
},
},
},

View File

@@ -26,33 +26,20 @@ func TestClusterSearchWithDisabledPerDayIndex(t *testing.T) {
defer tc.Stop()
testSearchWithDisabledPerDayIndex(tc, func(name string, disablePerDayIndex bool) apptest.PrometheusWriteQuerier {
// Using static ports for vmstorage because random ports may cause
// changes in how data is sharded.
vmstorage1 := tc.MustStartVmstorage("vmstorage1-"+name, []string{
"-storageDataPath=" + tc.Dir() + "/vmstorage1",
vmstorage := tc.MustStartVmstorage("vmstorage-"+name, []string{
"-storageDataPath=" + tc.Dir() + "/vmstorage",
"-retentionPeriod=100y",
"-httpListenAddr=127.0.0.1:61001",
"-vminsertAddr=127.0.0.1:61002",
"-vmselectAddr=127.0.0.1:61003",
fmt.Sprintf("-disablePerDayIndex=%t", disablePerDayIndex),
})
vmstorage2 := tc.MustStartVmstorage("vmstorage2-"+name, []string{
"-storageDataPath=" + tc.Dir() + "/vmstorage2",
"-retentionPeriod=100y",
"-httpListenAddr=127.0.0.1:62001",
"-vminsertAddr=127.0.0.1:62002",
"-vmselectAddr=127.0.0.1:62003",
fmt.Sprintf("-disablePerDayIndex=%t", disablePerDayIndex),
})
vminsert := tc.MustStartVminsert("vminsert-"+name, []string{
"-storageNode=" + vmstorage1.VminsertAddr() + "," + vmstorage2.VminsertAddr(),
"-storageNode=" + vmstorage.VminsertAddr(),
})
vmselect := tc.MustStartVmselect("vmselect"+name, []string{
"-storageNode=" + vmstorage1.VmselectAddr() + "," + vmstorage2.VmselectAddr(),
"-storageNode=" + vmstorage.VmselectAddr(),
"-search.maxStalenessInterval=1m",
})
return &apptest.Vmcluster{
Vmstorages: []*apptest.Vmstorage{vmstorage1, vmstorage2},
Vmstorages: []*apptest.Vmstorage{vmstorage},
Vminsert: vminsert,
Vmselect: vmselect,
}

View File

@@ -59,7 +59,7 @@ func (ms *PrometheusMockStorage) Read(_ context.Context, query *prompb.Query, so
}
if !notMatch {
q.Timeseries = append(q.Timeseries, &prompb.TimeSeries{Labels: s.Labels, Samples: s.Samples})
q.Timeseries = append(q.Timeseries, &prompb.TimeSeries{Labels: s.Labels, Samples: s.Samples, Histograms: s.Histograms})
}
}

View File

@@ -12,6 +12,7 @@ import (
"github.com/gogo/protobuf/proto"
"github.com/golang/snappy"
"github.com/prometheus/prometheus/model/histogram"
"github.com/prometheus/prometheus/model/labels"
"github.com/prometheus/prometheus/prompb"
"github.com/prometheus/prometheus/storage/remote"
@@ -86,10 +87,17 @@ func (rrs *RemoteReadServer) getReadHandler(t *testing.T) http.Handler {
samples = append(samples, sample)
}
}
var histograms []prompb.Histogram
for _, h := range s.Histograms {
if h.Timestamp >= startTs && h.Timestamp < endTs {
histograms = append(histograms, h)
}
}
var series prompb.TimeSeries
if len(samples) > 0 {
if len(samples) > 0 || len(histograms) > 0 {
series.Labels = s.Labels
series.Samples = samples
series.Histograms = histograms
}
ts[i] = &series
}
@@ -317,6 +325,37 @@ func generateRemoteReadSamples(idx int, startTime, endTime, numOfSamples int64)
return samples
}
// GenerateRemoteReadHistogramSeries generates a remote read series
// with native histogram samples within the given time range.
func GenerateRemoteReadHistogramSeries(start, end, numOfSamples int64) []*prompb.TimeSeries {
timeSeries := &prompb.TimeSeries{
Labels: []prompb.Label{
{Name: labels.MetricName, Value: "vm_histogram_metric"},
{Name: "job", Value: "0"},
},
}
delta := (end - start) / numOfSamples
mul := int64(0)
for t := start; t != end; t += delta {
mul++
h := &histogram.Histogram{
Schema: 0,
Count: uint64(10 * mul),
Sum: 25.5 * float64(mul),
ZeroThreshold: 0.001,
ZeroCount: uint64(2 * mul),
PositiveSpans: []histogram.Span{{Offset: 0, Length: 2}},
PositiveBuckets: []int64{mul, 2 * mul},
NegativeSpans: []histogram.Span{{Offset: 0, Length: 1}},
NegativeBuckets: []int64{4 * mul},
}
timeSeries.Histograms = append(timeSeries.Histograms, prompb.FromIntHistogram(t*1000, h))
}
return []*prompb.TimeSeries{timeSeries}
}
func labelsToLabelsProto(ls labels.Labels) []prompb.Label {
result := make([]prompb.Label, 0, ls.Len())
ls.Range(func(l labels.Label) {

View File

@@ -75,6 +75,118 @@ func TestClusterVmctlRemoteReadProtocol(t *testing.T) {
testRemoteReadProtocol(tc, clusterDst, newRemoteReadServer, vmctlFlags)
}
func TestSingleVmctlRemoteReadNativeHistograms(t *testing.T) {
fs.MustRemoveDir(t.Name())
tc := apptest.NewTestCase(t)
defer tc.Stop()
vmsingleDst := tc.MustStartDefaultVmsingle()
vmAddr := fmt.Sprintf("http://%s/", vmsingleDst.HTTPAddr())
vmctlFlags := []string{
`remote-read`,
`--remote-read-filter-time-start=2025-06-11T15:31:10Z`,
`--remote-read-filter-time-end=2025-06-11T15:31:20Z`,
`--remote-read-step-interval=minute`,
`--vm-addr=` + vmAddr,
`--disable-progress-bar=true`,
}
testRemoteReadNativeHistograms(tc, vmsingleDst, NewRemoteReadServer, vmctlFlags)
}
func TestSingleVmctlRemoteReadStreamNativeHistograms(t *testing.T) {
fs.MustRemoveDir(t.Name())
tc := apptest.NewTestCase(t)
defer tc.Stop()
vmsingleDst := tc.MustStartDefaultVmsingle()
vmAddr := fmt.Sprintf("http://%s/", vmsingleDst.HTTPAddr())
vmctlFlags := []string{
`remote-read`,
`--remote-read-filter-time-start=2025-06-11T15:31:10Z`,
`--remote-read-filter-time-end=2025-06-11T15:31:20Z`,
`--remote-read-step-interval=minute`,
`--vm-addr=` + vmAddr,
`--remote-read-use-stream=true`,
`--disable-progress-bar=true`,
}
testRemoteReadNativeHistograms(tc, vmsingleDst, NewRemoteReadStreamServer, vmctlFlags)
}
// testRemoteReadNativeHistograms verifies that native histograms are migrated
// as _count, _sum and _bucket series with vmrange labels in the same way
// as VictoriaMetrics converts native histograms received via Prometheus remote write protocol.
func testRemoteReadNativeHistograms(tc *apptest.TestCase, sut apptest.PrometheusWriteQuerier, newRemoteReadServer func(t *testing.T, series []*prompb.TimeSeries) *RemoteReadServer, vmctlFlags []string) {
t := tc.T()
t.Helper()
series := GenerateRemoteReadHistogramSeries(1749655870, 1749655880, 2)
rrs := newRemoteReadServer(t, series)
defer rrs.Close()
vmctlFlags = append(vmctlFlags, `--remote-read-src-addr=`+rrs.HTTPAddr())
tc.MustStartVmctl("vmctl", vmctlFlags)
sut.ForceFlush(t)
tc.Assert(&apptest.AssertOptions{
Retries: 300,
Msg: `unexpected native histogram metrics stored on vmsingle via the prometheus protocol`,
Got: func() any {
got := sut.PrometheusAPIV1Export(t, `{__name__=~".*"}`, apptest.QueryOpts{
Start: "2025-06-11T15:31:10Z",
End: "2025-06-11T15:32:20Z",
})
got.Sort()
return got.Data.Result
},
Want: expectedNativeHistogramQueryResult(),
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(apptest.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
},
})
}
// expectedNativeHistogramQueryResult returns the series expected to be stored in VictoriaMetrics
// after migrating the series generated by GenerateRemoteReadHistogramSeries(1749655870, 1749655880, 2).
func expectedNativeHistogramQueryResult() []*apptest.QueryResult {
metric := func(name, vmrange string) map[string]string {
m := map[string]string{
"__name__": name,
"job": "0",
}
if vmrange != "" {
m["vmrange"] = vmrange
}
return m
}
samples := func(v1, v2 float64) []*apptest.Sample {
return []*apptest.Sample{
{Timestamp: 1749655870000, Value: v1},
{Timestamp: 1749655875000, Value: v2},
}
}
resp := &apptest.PrometheusAPIV1QueryResponse{
Data: &apptest.QueryData{
Result: []*apptest.QueryResult{
{Metric: metric("vm_histogram_metric_count", ""), Samples: samples(10, 20)},
{Metric: metric("vm_histogram_metric_sum", ""), Samples: samples(25.5, 51)},
{Metric: metric("vm_histogram_metric_bucket", "-1.000e+00...-5.000e-01"), Samples: samples(4, 8)},
{Metric: metric("vm_histogram_metric_bucket", "-1.000e-03...1.000e-03"), Samples: samples(2, 4)},
{Metric: metric("vm_histogram_metric_bucket", "5.000e-01...1.000e+00"), Samples: samples(1, 2)},
{Metric: metric("vm_histogram_metric_bucket", "1.000e+00...2.000e+00"), Samples: samples(3, 6)},
},
},
}
// sort in the same way as the exported result
resp.Sort()
return resp.Data.Result
}
func testRemoteReadProtocol(tc *apptest.TestCase, sut apptest.PrometheusWriteQuerier, newRemoteReadServer func(t *testing.T) *RemoteReadServer, vmctlFlags []string) {
t := tc.T()
t.Helper()

View File

@@ -3,7 +3,7 @@ services:
# It scrapes targets defined in --promscrape.config
# And forward them to --remoteWrite.url
vmagent:
image: victoriametrics/vmagent:v1.148.0
image: victoriametrics/vmagent:v1.149.0
depends_on:
- "vmauth"
ports:
@@ -42,14 +42,14 @@ services:
# vmstorage shards. Each shard receives 1/N of all metrics sent to vminserts,
# where N is number of vmstorages (2 in this case).
vmstorage-1:
image: victoriametrics/vmstorage:v1.148.0-cluster
image: victoriametrics/vmstorage:v1.149.0-cluster
volumes:
- strgdata-1:/storage
command:
- "--storageDataPath=/storage"
restart: always
vmstorage-2:
image: victoriametrics/vmstorage:v1.148.0-cluster
image: victoriametrics/vmstorage:v1.149.0-cluster
volumes:
- strgdata-2:/storage
command:
@@ -59,7 +59,7 @@ services:
# vminsert is ingestion frontend. It receives metrics pushed by vmagent,
# pre-process them and distributes across configured vmstorage shards.
vminsert-1:
image: victoriametrics/vminsert:v1.148.0-cluster
image: victoriametrics/vminsert:v1.149.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -68,7 +68,7 @@ services:
- "--storageNode=vmstorage-2:8400"
restart: always
vminsert-2:
image: victoriametrics/vminsert:v1.148.0-cluster
image: victoriametrics/vminsert:v1.149.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -80,7 +80,7 @@ services:
# vmselect is a query fronted. It serves read queries in MetricsQL or PromQL.
# vmselect collects results from configured `--storageNode` shards.
vmselect-1:
image: victoriametrics/vmselect:v1.148.0-cluster
image: victoriametrics/vmselect:v1.149.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -90,7 +90,7 @@ services:
- "--vmalert.proxyURL=http://vmalert:8880"
restart: always
vmselect-2:
image: victoriametrics/vmselect:v1.148.0-cluster
image: victoriametrics/vmselect:v1.149.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -105,7 +105,7 @@ services:
# read requests from Grafana, vmui, vmalert among vmselects.
# It can be used as an authentication proxy.
vmauth:
image: victoriametrics/vmauth:v1.148.0
image: victoriametrics/vmauth:v1.149.0
depends_on:
- "vmselect-1"
- "vmselect-2"
@@ -119,7 +119,7 @@ services:
# vmalert executes alerting and recording rules
vmalert:
image: victoriametrics/vmalert:v1.148.0
image: victoriametrics/vmalert:v1.149.0
depends_on:
- "vmauth"
ports:

View File

@@ -3,7 +3,7 @@ services:
# It scrapes targets defined in --promscrape.config
# And forward them to --remoteWrite.url
vmagent:
image: victoriametrics/vmagent:v1.148.0
image: victoriametrics/vmagent:v1.149.0
depends_on:
- "victoriametrics"
ports:
@@ -18,7 +18,7 @@ services:
# VictoriaMetrics instance, a single process responsible for
# storing metrics and serve read requests.
victoriametrics:
image: victoriametrics/victoria-metrics:v1.148.0
image: victoriametrics/victoria-metrics:v1.149.0
ports:
- 8428:8428
- 8089:8089
@@ -59,7 +59,7 @@ services:
# vmalert executes alerting and recording rules
vmalert:
image: victoriametrics/vmalert:v1.148.0
image: victoriametrics/vmalert:v1.149.0
depends_on:
- "victoriametrics"
- "alertmanager"

View File

@@ -1,6 +1,6 @@
services:
vmagent:
image: victoriametrics/vmagent:v1.148.0
image: victoriametrics/vmagent:v1.149.0
depends_on:
- "victoriametrics"
ports:
@@ -14,7 +14,7 @@ services:
restart: always
victoriametrics:
image: victoriametrics/victoria-metrics:v1.148.0
image: victoriametrics/victoria-metrics:v1.149.0
ports:
- 8428:8428
volumes:
@@ -40,7 +40,7 @@ services:
restart: always
vmalert:
image: victoriametrics/vmalert:v1.148.0
image: victoriametrics/vmalert:v1.149.0
depends_on:
- "victoriametrics"
ports:
@@ -59,7 +59,7 @@ services:
- '--external.alert.source=explore?orgId=1&left=["now-1h","now","VictoriaMetrics",{"expr": },{"mode":"Metrics"},{"ui":[true,true,true,"none"]}]'
restart: always
vmanomaly:
image: victoriametrics/vmanomaly:v1.30.0
image: victoriametrics/vmanomaly:v1.30.1
depends_on:
- "victoriametrics"
ports:

View File

@@ -1,3 +1,11 @@
---
build:
list: never
publishResources: false
render: never
sitemap:
disable: true
---
VictoriaMetrics Observability Stack integrates with AI assistants through [MCP servers](https://docs.victoriametrics.com/ai-tools/#mcp-servers)
and [agent skills](https://docs.victoriametrics.com/ai-tools/#agent-skills).
The integrations allow AI agents and automation tools to query Metrics, Logs, and Traces, analyze telemetry data,

View File

@@ -16,6 +16,27 @@ Please find the changelog for VictoriaMetrics Anomaly Detection below.
{{% collapse name="2026" open=true %}}
## v1.30.1
Released: 2026-08-06
- UI: Updated [vmanomaly UI](https://docs.victoriametrics.com/anomaly-detection/ui/) from [v1.8.0](https://docs.victoriametrics.com/anomaly-detection/ui/#v180) to [v1.8.1](https://docs.victoriametrics.com/anomaly-detection/ui/#v181). The update improves UX validation and fixes regressions introduced by new design.
- IMPROVEMENT: Reduced fit and inference latency for the Z-score, MAD, standard deviation, Seasonal Quantile, and Rolling Quantile online models. Representative service-stage gains range from 1.5-2.6x for fit and 1.7-2.3x for inference, depending on model, storage mode, and data size.
- IMPROVEMENT: Removed forwarded datasource credentials from in-memory state for completed, failed, canceled, and shutting-down [analysis and autotune tasks](https://docs.victoriametrics.com/anomaly-detection/components/server/#time-series-analysis-and-autotune-api).
- BUGFIX: Stabilized [Temporal Envelope](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope) after fitting across a late level shift. Its level, trend, residual, and supported calendar state now initialize coherently from the recent regime, avoiding stale fitted magnitudes and false seasonal oscillations when periodic inference starts.
- BUGFIX: Corrected `/api/v1/timeseries/characteristics` seasonality detection for time series whose timestamps are offset from whole sampling intervals. Trend interpolation now preserves the original observation grid, allowing daily and weekly patterns to be detected on shifted grids.
- BUGFIX: Restored backward-compatible `inference_only` [backtesting](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/#backtesting-scheduler) for configurations that omit `infer_every`. The scheduler derives its inference grid from the query step or reader sampling period and preserves valid single-timestamp range queries.
- BUGFIX: Aligned periodic inference for exact-capable online models with exact backtesting (used in [UI](https://docs.victoriametrics.com/anomaly-detection/ui/) experiments) by applying the configured `infer_every` as the causal update cadence.
- BUGFIX: Corrected [self-monitoring](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#writer-behaviour-metrics) accounting so failed VictoriaMetrics write attempts contribute to `vmanomaly_writer_request_duration_seconds`, including connection retries, and inference counts only unseen *valid* rows in `vmanomaly_model_datapoints_accepted`.
- BUGFIX: Fixed service-level [`settings.anomaly_score_outside_data_range`](https://docs.victoriametrics.com/anomaly-detection/components/settings/#anomaly-score-outside-data-range) propagation so its configured score applies to every model unless the model defines its own override.
## v1.30.0
Released: 2026-07-23

View File

@@ -24,7 +24,7 @@ The decision to set the changepoint at `1.0` is made to ensure consistency acros
> `anomaly_score` is a metric itself, which preserves all labels found in input data and (optionally) appends [custom labels, specified in writer](https://docs.victoriametrics.com/anomaly-detection/components/writer/#metrics-formatting) - follow the link for detailed output example.
## How is anomaly score calculated?
For most of the [univariate models](https://docs.victoriametrics.com/anomaly-detection/components/models/#univariate-models) that can generate `yhat`, `yhat_lower`, and `yhat_upper` time series in [their output](https://docs.victoriametrics.com/anomaly-detection/components/models/#vmanomaly-output) (such as [Prophet](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) or [Z-score](https://docs.victoriametrics.com/anomaly-detection/components/models/#z-score)), the anomaly score is calculated as follows:
For most of the [univariate models](https://docs.victoriametrics.com/anomaly-detection/components/models/#univariate-models) that can generate `yhat`, `yhat_lower`, and `yhat_upper` time series in [their output](https://docs.victoriametrics.com/anomaly-detection/components/models/#vmanomaly-output) (such as [Temporal Envelope](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope) or [Online Z-score](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-z-score)), the anomaly score is calculated as follows:
- If `yhat` (expected series behavior) equals `y` (actual value observed), then the anomaly score is 0.
- If `y` (actual value observed) falls within the `[yhat_lower, yhat_upper]` confidence interval, the anomaly score will gradually approach 1, the closer `y` is to the boundary.
- If `y` (actual value observed) strictly exceeds the `[yhat_lower, yhat_upper]` interval, the anomaly score will be greater than 1, increasing as the margin between the actual value and the expected range grows.
@@ -82,7 +82,7 @@ reader:
`vmanomaly` supports timezone-aware anomaly detection {{% available_from "v1.18.0" anomaly %}} through a `tz` argument, available both at the [reader level](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) and at the [query level](https://docs.victoriametrics.com/anomaly-detection/components/reader/#per-query-parameters).
For models that depend on seasonality, such as [`ProphetModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) and [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile), handling timezone shifts is crucial. Changes like Daylight Saving Time (DST) can disrupt seasonality patterns learned by models, resulting in inaccurate anomaly predictions as the periodic patterns shift with time. Proper timezone configuration ensures that seasonal cycles align with expected intervals, even as DST changes occur.
For models that depend on seasonality, such as [`TemporalEnvelopeModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope) and [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile), handling timezone shifts is crucial. Changes like Daylight Saving Time (DST) can disrupt seasonality patterns learned by models, resulting in inaccurate anomaly predictions as the periodic patterns shift with time. Proper timezone configuration ensures that seasonal cycles align with expected intervals, even as DST changes occur.
To enable timezone handling:
1. **Reader-level**: Set `tz` in the [`reader`](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) section to a specific timezone (e.g., `Europe/Berlin`) to apply this setting to all queries.
@@ -100,9 +100,9 @@ reader:
tz: 'Europe/London' # per-query override
models:
seasonal_model:
class: 'prophet'
class: 'temporal_envelope'
queries: ['your_query']
# other model params ...
seasonalities: ['hod_smooth', 'dow_smooth']
```
## Output produced by vmanomaly
@@ -124,9 +124,8 @@ Selecting the best model for `vmanomaly` depends on the data's nature and the [t
- Use [Online MAD](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-mad) for simple, mostly stationary data with no-to-slow trend, when robustness to outliers is important.
- Use [Online Z-score](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-z-score) for simple, light-tailed data where standard-deviation units are meaningful.
- Use [Temporal Envelope](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope) {{% available_from "v1.30.0" anomaly %}} for complex data with trends, calendar patterns, holidays, or persistent shifts. It is the preferred *online* alternative to Prophet (which will be deprecated in the future releases).
- Use [Temporal Envelope](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope) {{% available_from "v1.30.0" anomaly %}} for complex data with trends, calendar patterns, holidays, or persistent shifts. It is the preferred online migration target for existing Prophet configurations.
- Use multivariate Temporal Envelope when normal relationships between aligned metrics matter. This should replace [Isolation Forest](https://docs.victoriametrics.com/anomaly-detection/components/models/#isolation-forest-multivariate) used in previous versions of `vmanomaly`, which will be deprecated in future releases.
- Use [Prophet](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) when Prophet-specific decomposition outputs, or offline batch behavior are required. Consider using Temporal Envelope instead, as it is more efficient and provides better results in most cases.
There is also an option to auto-tune the most important parameters of a selected model class {{% available_from "v1.12.0" anomaly %}}. {{% available_from "v1.30.0" anomaly %}} The asynchronous autotune API can first profile a bounded sample through `/api/v1/timeseries/characteristics`, then tune a shared concrete configuration through `/api/v1/autotune/tasks`. See the [autotune workflow](https://docs.victoriametrics.com/anomaly-detection/components/models/#shared-asynchronous-autotune-workflow).
@@ -254,7 +253,7 @@ Configuration above will produce N intervals of full length (`fit_window`=14d +
## Forecasting
`vmanomaly` can generate future forecasts using [Temporal Envelope](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope) {{% available_from "v1.30.0" anomaly %}} or [ProphetModel](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) {{% available_from "v1.25.3" anomaly %}}. This is helpful for capacity planning, resource allocation, or trend analysis when the underlying data is complex and exceeds what inline MetricsQL queries, including [predict_linear](https://docs.victoriametrics.com/victoriametrics/metricsql/#predict_linear), can handle.
`vmanomaly` can generate future forecasts with [Temporal Envelope](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope) {{% available_from "v1.30.0" anomaly %}}, the preferred online forecasting model. [ProphetModel](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) {{% available_from "v1.25.3" anomaly %}} also supports forecasting for existing offline configurations. Forecasts help with capacity planning, resource allocation, or trend analysis when the underlying data is complex and exceeds what inline MetricsQL queries, including [predict_linear](https://docs.victoriametrics.com/victoriametrics/metricsql/#predict_linear), can handle.
> However, please note that this mode should be used with care, as the model will produce `yhat_{h}` (and probably `yhat_lower_{h}`, and `yhat_upper_{h}`) time series **for each timeseries returned by input queries and for each forecasting horizon specified in `forecast_at` argument, which can lead to a significant increase in the number of active timeseries in VictoriaMetrics TSDB**.
@@ -432,7 +431,7 @@ services:
# ...
vmanomaly:
container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.30.0
image: victoriametrics/vmanomaly:v1.30.1
# ...
restart: always
volumes:
@@ -554,7 +553,8 @@ reader:
expr: 'sum(ALERTS{alertstate=~'(pending|firing)'}) by (alertstate)'
max_points_per_query: 5000 # query-level override
models:
prophet:
temporal_envelope:
class: temporal_envelope
# other model args
queries: [
'sum_alerts',
@@ -575,7 +575,8 @@ reader:
sum_alerts:
expr: 'sum(ALERTS{alertstate=~'(pending|firing)'}) by (alertstate)'
models:
prophet:
temporal_envelope:
class: temporal_envelope
# other model args
queries: [
'sum_alerts',
@@ -594,7 +595,8 @@ reader:
sum_alerts_firing:
expr: 'sum(ALERTS{alertstate='firing'}) by ()'
models:
prophet:
temporal_envelope:
class: temporal_envelope
# other model args
queries: [
'sum_alerts_pending',
@@ -652,7 +654,7 @@ options:
Heres an example of using the config splitter to divide configurations based on the `extra_filters` argument from the reader section:
```sh
docker pull victoriametrics/vmanomaly:v1.30.0 && docker image tag victoriametrics/vmanomaly:v1.30.0 vmanomaly
docker pull victoriametrics/vmanomaly:v1.30.1 && docker image tag victoriametrics/vmanomaly:v1.30.1 vmanomaly
```
```sh

View File

@@ -45,7 +45,7 @@ There are 2 types of compatibility to consider when migrating in stateful mode:
| Group start | Group end | Compatibility | Notes |
|---------|--------- |------------|-------|
| [v1.29.1](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1291) | [v1.30.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1300) | Fully Compatible | v1.30.0 adds new [Temporal Envelope](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope) model state without changing the compatibility of existing model and data artifacts. |
| [v1.29.1](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1291) | [v1.30.1](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1301) | Fully Compatible | v1.30.0 adds new [Temporal Envelope](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope) model state without changing the compatibility of existing model and data artifacts. v1.30.1 remains compatible with v1.30.0 state and its compatible predecessors. |
| [v1.28.7](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1287) | [v1.29.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1290) | Partially compatible* | Dumped models of class [prophet](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) and [seasonal quantile](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile) have problems with loading to [v1.29.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1290) due to dropped `pytz` library. **Upgrading directly from v1.28.7 to [v1.29.1](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1291) with a fix is suggested** |
| [v1.26.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1262) | [v1.28.7](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1287) | Fully Compatible | [v1.28.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1280) introduced [rolling](https://docs.victoriametrics.com/anomaly-detection/components/models/#rolling-models) model class drop in favor of [online](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-models) models (`rolling_quantile` and `std` models), however, it does not impact compatibility, as artifacts were not produced by default for rolling models. Also, offline `mad` and `zscore` models are redirecting to their respective online counterparts since [v1.28.4](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1284). |
| [v1.25.3](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1253) | [v1.26.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1270) | Partially Compatible* | [v1.25.3](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1253) introduced `forecast_at` argument for base [univariate](https://docs.victoriametrics.com/anomaly-detection/components/models/#univariate-models) and `Prophet` [models](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet), however, itself remains backward-reversible from newer states like [v1.26.2](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1262), [v1.27.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1270). (All models except `isolation_forest_multivariate` class will be dropped) |
@@ -71,7 +71,7 @@ In stateless mode, the migration process is almost straightforward as there are
**Breaking Changes**
- [v1.12.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1120) **ARIMA** model is removed from [built-in models](https://docs.victoriametrics.com/anomaly-detection/components/models/#built-in-models); Action: replace ARIMA by [Prophet](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) or alternative seasonal models in `model(s)` section of your configuration files.
- [v1.12.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1120) **ARIMA** model is removed from [built-in models](https://docs.victoriametrics.com/anomaly-detection/components/models/#built-in-models). Action: for vmanomaly v1.30.0 and newer, replace ARIMA with [Temporal Envelope](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope) {{% available_from "v1.30.0" anomaly %}}; for older releases, use another supported seasonal model in the `models` section of the configuration.
- [v1.9.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v190) The `sampling_period` parameter is now mandatory in `VmReader`. This change aims to clarify and standardize the frequency of input/output in `vmanomaly`, thereby reducing uncertainty and aligning with user expectations; Action: Add the `sampling_period` parameter to your `VmReader` configuration, e.g.:

View File

@@ -126,13 +126,18 @@ groups:
> docker pull quay.io/victoriametrics/vmanomaly:vX.Y.Z
> ```
> [!NOTE] ARM64 startup on affected Apple Silicon virtualization
> On some `linux/arm64` environments running through virtualization on Apple M4/M5 hosts, `vmanomaly` may exit with `SIGILL` (exit code `132`) before startup. This is caused by the virtualized host advertising an SVE2 capability that traps when used by OpenSSL 4.x; it does not affect all ARM64 systems.
>
> On affected hosts, add `-e OPENSSL_armcap=0` to `docker run`, or add `- OPENSSL_armcap=0` under the service's Docker Compose `environment`, matching the list syntax used below. This disables ARM cryptographic acceleration, so apply it only as a temporary workaround on affected hosts.
Below are the steps to get `vmanomaly` up and running inside a Docker container:
1. Pull Docker image:
```sh
docker pull victoriametrics/vmanomaly:v1.30.0
docker pull victoriametrics/vmanomaly:v1.30.1
```
2. Create the license file with your license key.
@@ -152,7 +157,7 @@ docker run -it \
-v ./license:/license \
-v ./config.yaml:/config.yaml \
-p 8490:8490 \
victoriametrics/vmanomaly:v1.30.0 \
victoriametrics/vmanomaly:v1.30.1 \
/config.yaml \
--licenseFile=/license \
--loggerLevel=INFO \
@@ -169,7 +174,7 @@ docker run -it \
-e VMANOMALY_DATA_DUMPS_DIR=/tmp/vmanomaly/data \
-e VMANOMALY_MODEL_DUMPS_DIR=/tmp/vmanomaly/models \
-p 8490:8490 \
victoriametrics/vmanomaly:v1.30.0 \
victoriametrics/vmanomaly:v1.30.1 \
/config.yaml \
--licenseFile=/license \
--loggerLevel=INFO \
@@ -182,7 +187,7 @@ services:
# ...
vmanomaly:
container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.30.0
image: victoriametrics/vmanomaly:v1.30.1
# ...
restart: always
volumes:

View File

@@ -193,7 +193,7 @@ The best applications of this mode are:
### What you can do with Copilot
- **Ask questions** about any model (e.g. [Temporal Envelope](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope), [Prophet](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet), or [Z-score](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-z-score) - parameters, trade-offs, when to use each)
- **Ask questions** about any model (e.g. [Temporal Envelope](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope), [Online Seasonal Quantile](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile), or [Online Z-score](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-z-score) - parameters, trade-offs, when to use each)
- **Improve detection quality** - describe what's wrong ("too many false positives", "missing spikes") and Copilot reads the config, searches the docs, and proposes a validated configuration change to fix the issue.
- **Get config suggestions inline** - suggestions appear as interactive cards with an explanation and a YAML diff; click **Apply** to write the change directly to your current settings, or **Decline** to keep the conversation going.
- {{% available_from "v1.30.0" anomaly %}} **Profile and tune the real query** - with [mcp-vmanomaly](#mcp-tools-server) connected, Copilot can inspect bounded time-series characteristics, recommend an online model, start an asynchronous autotune task, and apply its validated query and model suggestions.
@@ -316,7 +316,7 @@ docker run -it --rm \
-e VMANOMALY_MCP_SERVER_URL=http://mcp-vmanomaly:8081/mcp \
-p 8080:8080 \
-p 8490:8490 \
victoriametrics/vmanomaly:v1.30.0 \
victoriametrics/vmanomaly:v1.30.1 \
vmanomaly_config.yaml
```
@@ -569,7 +569,7 @@ Set up the time range and resolution (step) for data visualization and anomaly d
![vmanomaly-ui-sections-explore](vmanomaly-ui-sections-explore.webp)
Pay attention to trends, seasonality, noise, outliers, and other patterns in the data, which can influence the choice of anomaly detection model and its hyperparameters (e.g. use seasonal models for seasonal data - like `Prophet`, robust models for noisy de-seasonalized data - like `MAD`, etc.).
Pay attention to trends, seasonality, noise, outliers, and other patterns in the data, which can influence the choice of anomaly detection model and its hyperparameters. Use [Temporal Envelope](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope) for complex data with trend or calendar patterns, and [Online MAD](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-mad) for simple, mostly stationary data where robustness to outliers matters.
![vmanomaly-ui-sections-plot-area-query-mode](vmanomaly-ui-sections-plot-area-query-mode.webp)
@@ -645,6 +645,19 @@ If the **results** look good and the **model configuration should be deployed in
{{% collapse name="Release history" %}}
### v1.8.1
Released: 2026-08-06
vmanomaly version: [v1.30.1](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1301)
- IMPROVEMENT: Model settings are validated and normalized when applied. Invalid drafts remain open with actionable feedback, and advanced-setting summaries open the corresponding editor directly.
- BUGFIX: Server-query counts load when the query drawer opens, and numeric model fields preserve valid scalar and range values while reporting parsing errors on blur.
- BUGFIX: The anomaly visualization empty state now follows the active theme instead of using light-theme colors in dark mode.
- BUGFIX: Tenant selection now follows the datasource URL resolved by the server, avoiding an incorrect switch to tenant `0` when it is unavailable.
### v1.8.0
Released: 2026-07-23

View File

@@ -54,11 +54,11 @@ schedulers:
fit_window: "3d" # how much historical data to use for fit stage
start_from: "00:00" # align the annual fit schedule to midnight in the configured timezone
tz: "Europe/Kyiv" # timezone to use for start_from
periodic_offline_1w:
periodic_online_weekly:
class: 'periodic'
infer_every: "15m"
scatter_infer_jobs: true
fit_every: "24h"
fit_every: "365d" # online state continues adapting between infrequent full re-fits
fit_window: "14d"
# if no start_from is specified, jobs will start immediately after service starts
@@ -75,18 +75,16 @@ models:
min_dev_from_expected: 0.0 # turned off. if |y - yhat| < min_dev_from_expected, anomaly score will be 0
detection_direction: 'above_expected' # detect anomalies only when y > yhat, "peaks"
clip_predictions: True # clip predictions to expected data range, i.e. [0, inf] for this query `host_network_receive_errors
prophet_weekly: # we can set up alias for model
class: 'prophet'
envelope_weekly: # we can set up alias for model
class: 'temporal_envelope'
provide_series: ['anomaly_score', 'y', 'yhat', 'yhat_lower', 'yhat_upper']
queries: ['cpu_seconds_total']
schedulers: ['periodic_offline_1w'] # will be attached to 1-week scheduler, re-fit every 24h and infer every 15m
schedulers: ['periodic_online_weekly'] # fit on two weekly cycles, then update online every 15m
min_dev_from_expected: [0.01, 0.01] # minimum deviation from expected value to be even considered as anomaly
anomaly_score_outside_data_range: 1.5 # override default anomaly score outside expected data range
detection_direction: 'above_expected'
clip_predictions: True # clip predictions to expected data range, i.e. [0, inf] for this query `cpu_seconds_total`
args: # model-specific arguments
interval_width: 0.98
yearly_seasonality: False # disable yearly seasonality, since we have only 7 days of data
seasonalities: ['hod_smooth', 'dow_smooth']
# where to read data from
# https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader
@@ -113,7 +111,7 @@ reader:
# https://docs.victoriametrics.com/anomaly-detection/components/writer/
writer:
datasource_url: "http://victoriametrics:8428/"
# tenant_id: "0:0" # for VictoriaMetrics cluster, can support "multitenant"
tenant_id: "0:0" # for VictoriaMetrics cluster, can support "multitenant"
# https://docs.victoriametrics.com/anomaly-detection/components/writer/#metrics-formatting
metric_format:
__name__: $VAR
@@ -204,6 +202,7 @@ models:
writer:
datasource_url: "http://victoriametrics:8428/"
tenant_id: "0:0"
monitoring:
push:

View File

@@ -27,16 +27,13 @@ This section covers the `Models` component of VictoriaMetrics Anomaly Detection
```yaml
models:
model_univariate_1:
class: 'zscore' # or 'model.zscore.ZscoreModel' until v1.13.0
class: 'zscore_online'
z_threshold: 2.5
queries: ['query_alias2'] # referencing queries defined in `reader` section
model_multivariate_1:
class: 'isolation_forest_multivariate' # or model.isolation_forest.IsolationForestMultivariateModel until v1.13.0
contamination: 'auto'
args:
n_estimators: 100
# i.e. to assure reproducibility of produced results each time model is fit on the same input
random_state: 42
class: 'temporal_envelope_multivariate'
seasonalities: ['hod_smooth', 'dow_smooth']
provide_series: ['anomaly_score']
# if there is no explicit `queries` arg, then the model will be run on ALL queries found in reader section
# ...
```
@@ -334,9 +331,10 @@ reader:
+ sum(rate(node_network_transmit_bytes_total[5m])) by (host)
models:
iforest: # alias for the model
class: isolation_forest_multivariate
contamination: 0.01
envelope: # alias for the model
class: temporal_envelope_multivariate
seasonalities: [hod_smooth, dow_smooth]
provide_series: [anomaly_score]
# the multivariate model can be trained on 2+ timeseries returned by 1+ queries
queries: [cpu, ram, network]
# train a distinct multivariate model for each unique value found in the `host` label
@@ -548,7 +546,7 @@ If during an inference, you got a series having **new labelset** (not present in
**Implications:** Univariate models are a go-to default, when your queries returns **changing** amount of **individual** time series of **different** magnitude, [trend](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) or [seasonality](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality), so you won't be mixing incompatible data with different behavior within a single fit model (context isolation).
**Examples:** [Prophet](#prophet), [Holt-Winters](#holt-winters)
**Examples:** [Temporal Envelope](#temporal-envelope), [Online MAD](#online-mad), [Online Z-score](#online-z-score), [Online Seasonal Quantile](#online-seasonal-quantile)
![Univariate model lifecycle](model-lifecycle-univariate.svg)
@@ -565,11 +563,11 @@ If during an inference, you got a **different amount of series** or some series
**Implications:** Multivariate models are a go-to default, when your queries returns **fixed** amount of **individual** time series (say, some aggregations), to be used for adding cross-series (and cross-query) context, useful for catching [collective anomalies](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-2/#collective-anomalies) or [novelties](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-2/#novelties) (expanded to multi-input scenario). For example, you may set it up for anomaly detection of CPU usage in different modes (`idle`, `user`, `system`, etc.) and use its cross-dependencies to detect **unseen (in fit data)** behavior.
**Examples:** [Temporal Envelope](#temporal-envelope), [Isolation Forest](#isolation-forest-multivariate)
**Recommended:** [Temporal Envelope](#temporal-envelope). Existing [Isolation Forest](#isolation-forest-multivariate) configurations can migrate to its multivariate form.
![Multivariate model lifecycle](model-lifecycle-multivariate.svg)
The following configuration applies both models to the same aligned input series. Start with Temporal Envelope when temporal profiles and online adaptation matter; use Isolation Forest as an offline alternative when feature-space outliers are the primary concern.
The following configuration applies a multivariate Temporal Envelope model to the same aligned input series:
```yaml
models:
@@ -582,16 +580,6 @@ models:
seasonalities: [hod_smooth, dow_smooth]
provide_series: [anomaly_score]
service_dependency_isolation_forest:
class: isolation_forest_multivariate
queries: [request_rate, error_rate, latency]
groupby: [cluster]
contamination: 0.01
seasonal_features: [hod, dow]
args:
n_estimators: 100
random_state: 42
provide_series: [anomaly_score]
```
@@ -634,6 +622,9 @@ Each of the ([built-in](#built-in-models) or [custom](#custom-model-guide)) onli
Every other model that isn't [online](#online-models). Offline models are completely re-trained during `fit` call and aren't updated during consecutive `infer` calls.
> [!NOTE]
> Built-in offline model classes are planned for deprecation in a future release in favor of online counterparts. For complex temporal data, prefer [Temporal Envelope](#temporal-envelope), which supports incremental adaptation, forecasting, and both univariate and multivariate operation.
## Built-in Models
@@ -649,14 +640,14 @@ Built-in models support 2 groups of arguments:
**Models**:
- [AutoTuned](#autotuned) - designed to take the cognitive load off the user, allowing any of built-in models below to be re-tuned for best hyperparameters on data seen during each `fit` phase of the algorithm. Tradeoff is between increased computational time and optimized results / simpler maintenance.
- [Temporal Envelope](#temporal-envelope) - the preferred **online model for complex operational data** with trends, changepoints, multiple calendar patterns, holidays, capable of [forecasting](https://docs.victoriametrics.com/anomaly-detection/faq/#forecasting). Its multivariate form also learns cross-series relationships.
- [Prophet](#prophet) - an offline forecasting alternative when Prophet-specific decomposition outputs are required. Favor `Temporal Envelope` for online adaptation and multivariate support.
- [Online Z-score](#online-z-score) - useful for initial testing and for simpler data ([de-trended](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) data without strict [seasonality](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality) and with anomalies of similar magnitude as your "normal" data)
- [MAD](#online-mad) - similarly to [Z-score](#online-z-score), is effective for **identifying outliers in relatively consistent data**. Useful for detecting sudden, stark deviations from the median, being less prone to outlier's magnitude than z-score.
- [Rolling Quantile](#rolling-quantile) - best for **data with evolving patterns**, as it adapts to changes over a rolling window.
- [Online Seasonal Quantile](#online-seasonal-quantile) - best used on **[de-trended](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) data with strong (possibly multiple) [seasonalities](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality)**. Can act as a (slightly less powerful) [online](#online-models) replacement to [`ProphetModel`](#prophet).
- [Seasonal Trend Decomposition](#seasonal-trend-decomposition) - similarly to Holt-Winters, is best for **data with pronounced [seasonal](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality) and [trend](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) components**
- [Isolation forest (Multivariate)](#isolation-forest-multivariate) - an offline alternative for **metrics data interaction** (several queries/metrics -> single anomaly score) and high-dimensional feature-space outliers. Prefer multivariate Temporal Envelope when temporal profiles and *online* adaptation matter.
- [Holt-Winters](#holt-winters) - well-suited for **data with moderate complexity**, exhibiting distinct [trends](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) and/or [single seasonal pattern](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality).
- [Prophet](#prophet) - an offline model retained for existing deployments. Migrate forecasting and seasonal anomaly-detection configurations to [Temporal Envelope](#temporal-envelope), unless Prophet-specific decomposition output must be preserved.
- [Isolation forest (Multivariate)](#isolation-forest-multivariate) - an offline model retained for existing univariate and multivariate deployments. Migrate to the corresponding [Temporal Envelope](#temporal-envelope) form for online adaptation and temporal or cross-series context.
- [Holt-Winters](#holt-winters) - an offline model retained for existing trend and single-seasonality configurations. Migrate these configurations to [Temporal Envelope](#temporal-envelope).
- [Custom model](#custom-model-guide) - benefit from your own models and expertise to better support your **unique use case**.
@@ -785,6 +776,8 @@ The requested anomaly percentage is treated as an alert-volume constraint rather
{{% available_from "v1.30.0" anomaly %}} Temporal Envelope is the preferred online model for complex operational and business metrics. It learns an evolving expected range from robust trend, calendar and holiday patterns, persistent level shifts, uncertainty, and optional future forecasts. The model adapts during inference while limiting the lasting influence of short-lived spikes.
{{% available_from "v1.30.1" anomaly %}} When the fit window ends in a recently established level, the model initializes its adaptive state from that recent regime while preserving supported calendar structure. This improves the first periodic predictions after a level shift and reduces false seasonal oscillation without requiring additional configuration.
> `TemporalEnvelopeModel` is [univariate](#univariate-models) and [online](#online-models). `TemporalEnvelopeMultivariateModel` also learns normal cross-series relationships as a [multivariate](#multivariate-models) model.
Use it for:
@@ -793,7 +786,7 @@ Use it for:
- deployments, traffic migrations, and capacity changes that create persistent shifts, including short-horizon forecasts through `forecast_at`;
- aligned related metrics where each channel keeps its own temporal pattern while their joint behavior contributes to one anomaly score.
For simple profiles without strong trend or seasonality, prefer [Online MAD](#online-mad) or [Online Z-score](#online-z-score). [Prophet](#prophet) and [Isolation Forest](#isolation-forest-multivariate) remain offline alternatives when their distinct capabilities are required or validation favors them.
For simple profiles without strong trend or seasonality, prefer [Online MAD](#online-mad) or [Online Z-score](#online-z-score). Existing [Prophet](#prophet) and [Isolation Forest](#isolation-forest-multivariate) configurations can be migrated to the corresponding univariate or multivariate Temporal Envelope form.
<div class="model-details">
@@ -873,11 +866,224 @@ For independent per-series detection, use `temporal_envelope`. Use `temporal_env
</div>
### Online MAD
> `OnlineMADModel` is a [univariate](#univariate-models), [online](#online-models) model.
The MAD model is a robust method for anomaly detection that is *less sensitive* to outliers in data compared to standard deviation-based models. It considers a point as an anomaly if the absolute deviation from the median is significantly large. This is the online approximate version, based on [t-digests](https://www.sciencedirect.com/science/article/pii/S2665963820300403) for online quantile estimation{{% available_from "v1.15.0" anomaly %}}.
<div class="model-details">
{{% collapse name="Model-specific arguments" %}}
- `class` (string) - model class name `"model.online.OnlineMADModel"` (or `mad_online` with class alias support{{% available_from "v1.13.0" anomaly %}})
- `threshold` (float, optional) - The threshold multiplier for the MAD to determine anomalies. Defaults to `2.5`. Higher values will identify fewer points as anomalies.
- `min_n_samples_seen` (int, optional) - the minimum number of samples to be seen (`n_samples_seen_` property) before computing the anomaly score. Otherwise, the **anomaly score will be 0**, as there is not enough data to trust the model's predictions. Defaults to 16.
- `history_strength` (float, optional) - {{% available_from "v1.30.0" anomaly %}} strength of the initial history learned by `fit`. Values above `1` preserve fitted quantiles initially but reduce the leverage of subsequent updates. Defaults to `1`.
- `compression` (int, optional) - the compression parameter for underlying [t-digest](https://www.sciencedirect.com/science/article/pii/S2665963820300403). Higher values mean higher accuracy but higher memory usage. By default 100.
{{% /collapse %}}
{{% collapse name="Configuration example" %}}
```yaml
models:
your_desired_alias_for_a_model:
class: "mad_online" # or 'model.online.OnlineMADModel'
threshold: 2.5
min_n_samples_seen: 128 # i.e. calculate it as full seasonality / data freq
history_strength: 2 # retain fitted history as a stronger prior
compression: 100 # higher values mean higher accuracy but higher memory usage
provide_series: ['anomaly_score', 'yhat'] # common arg example
# Common arguments for built-in model, if not set, default to
# See https://docs.victoriametrics.com/anomaly-detection/components/models/#common-args
#
# provide_series: ['anomaly_score', 'yhat', 'yhat_lower', 'yhat_upper']
# schedulers: [all scheduler aliases defined in `scheduler` section]
# queries: [all query aliases defined in `reader.queries` section]
# detection_direction: 'both' # meaning both drops and spikes will be captured
# min_dev_from_expected: [0.0, 0.0] # meaning, no minimal threshold is applied to prevent smaller anomalies
# scale: [1.0, 1.0] # if needed, prediction intervals' width can be increased (>1) or narrowed (<1)
# clip_predictions: False # if data_range for respective `queries` is set in reader, `yhat.*` columns will be clipped
# anomaly_score_outside_data_range: 1.01 # auto anomaly score (1.01) if `y` (real value) is outside of data_range, if set
```
{{% /collapse %}}
</div>
Resulting metrics of the model are described [here](#vmanomaly-output).
### Online Seasonal Quantile
> `OnlineQuantileModel` is a [univariate](#univariate-models), [online](#online-models) model.
Online (seasonal) quantile utilizes a set of approximate distributions, based on [t-digests](https://www.sciencedirect.com/science/article/pii/S2665963820300403) for online quantile estimation {{% available_from "v1.15.0" anomaly %}}.
Best used on **[de-trended](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) data with strong (potentially multiple) [seasonalities](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality)**. Can act as a (slightly less flexible) replacement to [`ProphetModel`](#prophet).
It uses the `quantiles` triplet to calculate `yhat_lower`, `yhat`, and `yhat_upper` [output](#vmanomaly-output), respectively, for each of the `min_subseason` sub-intervals contained in `seasonal_interval`. For example, with '4d' + '2h' seasonality patterns (multiple), it will hold and update 24*4 / 2 = 48 consecutive estimates (each 2 hours long).
<div class="model-details">
{{% collapse name="Model-specific arguments" %}}
- `class` (string) - model class name `"model.online.OnlineQuantileModel"` (or `quantile_online` with class alias support{{% available_from "v1.13.0" anomaly %}})
- `quantiles` (list[float], optional) - The quantiles to estimate. `yhat_lower`, `yhat`, `yhat_upper` are the quantile order. By default (0.01, 0.5, 0.99).
- `iqr_threshold` (float, optional) - {{% available_from "v1.25.0" anomaly %}} The [interquartile range (IQR)](https://en.wikipedia.org/wiki/Interquartile_range) multiplier to increase the width of the prediction intervals. Defaults to 0 (no adjustment) for backward compatibility. If set > 0, the model will add IQR * `iqr_threshold` to `yhat_lower` and `yhat_upper` (respecting `min_subseason` seasonal buckets). This is useful for data with high variance or outliers, as it helps to avoid false positives in anomaly detection. Best used with **robust** `quantiles` set to (0.25, 0.5, 0.75) or similar.
- `seasonal_interval` (string, optional) - the interval for the seasonal adjustment. If not set, the model will equal to a simple online quantile model. By default not set.
- `min_subseason` (str, optional) - the minimum interval to estimate quantiles for. By default not set. Note that the minimum interval should be a multiple of the seasonal interval, i.e. if seasonal_interval='2h', then min_subseason='15m' is valid, but '37m' is not.
- `use_transform` (bool, optional) - whether to internally apply a `log1p(abs(x)) * sign(x)` transformation to the data to stabilize internal quantile estimation. Does not affect the scale of produced output (i.e. `yhat`) By default False.
- `global_smoothing` (float, optional) - the smoothing parameter for the global quantiles. i.e. the output is a weighted average of the global and seasonal quantiles (if `seasonal_interval` and `min_subseason` args are set). Should be from `[0, 1]` interval, where 0 means no smoothing and 1 means using only global quantile values.
- `scale` (float, optional) - Is used to adjust the margins between `yhat` and [`yhat_lower`, `yhat_upper`]. New margin = `|yhat_* - yhat_lower| * scale`. Defaults to 1 (no scaling is applied). See `scale`[common arg](https://docs.victoriametrics.com/anomaly-detection/components/models/#scale) section for detailed instructions and 2-sided option.
- `season_starts_from` (str, optional) - the start date for the seasonal adjustment, as a reference point to start counting the intervals. By default '1970-01-01'.
- `min_n_samples_seen` (int, optional) - the minimum number of samples to be seen (`n_samples_seen_` property) before computing the anomaly score. Otherwise, the **anomaly score will be 0**, as there is not enough data to trust the model's predictions. Defaults to 16.
- `history_strength` (float, optional) - {{% available_from "v1.30.0" anomaly %}} strength of the initial history learned by `fit`. Values above `1` preserve fitted quantiles initially but reduce the leverage of subsequent updates. Defaults to `1`.
- `compression` (int, optional) - the compression parameter for the underlying [t-digests](https://www.sciencedirect.com/science/article/pii/S2665963820300403). Higher values mean higher accuracy but higher memory usage. By default 100.
{{% /collapse %}}
{{% collapse name="Configuration example" %}}
Suppose we have a data with strong intra-day (hourly) and intra-week (daily) seasonality, data granularity is '5m' with up to 5% expected outliers present in data. Then you can apply similar config:
```yaml
models:
your_desired_alias_for_a_model:
class: "quantile_online" # or 'model.online.OnlineQuantileModel'
quantiles: [0.25, 0.5, 0.75] # lowered to exclude anomalous edges, can be compensated by `scale` param > 1 and `iqr_threshold` > 0
iqr_threshold: 2.5 # to increase prediction intervals' width to avoid false positives while still keeping the model robust
seasonal_interval: '7d' # longest seasonality (week, day) = week, starting from `season_starts_from`
min_subseason: '1h' # smallest seasonality (week, day, hour) = hour, will have its own quantile estimates
min_n_samples_seen: 288 # 1440 / 5 - at least 1 full day, ideal = 1440 / 5 * 7 - one full week (seasonal_interval)
history_strength: 2 # retain fitted history as a stronger prior
scale: 1.1 # to compensate lowered quantile boundaries with wider intervals
season_starts_from: '2024-01-01' # interval calculation starting point, especially for uncommon seasonalities like '36h' or '12d'
compression: 100 # higher values mean higher accuracy but higher memory usage
provide_series: ['anomaly_score', 'yhat'] # common arg example
# Common arguments for built-in model, if not set, default to
# See https://docs.victoriametrics.com/anomaly-detection/components/models/#common-args
#
# provide_series: ['anomaly_score', 'yhat', 'yhat_lower', 'yhat_upper']
# schedulers: [all scheduler aliases defined in `scheduler` section]
# queries: [all query aliases defined in `reader.queries` section]
# detection_direction: 'both' # meaning both drops and spikes will be captured
# min_dev_from_expected: [0.0, 0.0] # meaning, no minimal threshold is applied to prevent smaller anomalies
# scale: [1.0, 1.0] # if needed, prediction intervals' width can be increased (>1) or narrowed (<1)
# clip_predictions: False # if data_range for respective `queries` is set in reader, `yhat.*` columns will be clipped
# anomaly_score_outside_data_range: 1.01 # auto anomaly score (1.01) if `y` (real value) is outside of data_range, if set
```
{{% /collapse %}}
</div>
Resulting metrics of the model are described [here](#vmanomaly-output).
### Online Z-score
> `OnlineZscoreModel` is a [univariate](#univariate-models), [online](#online-models) model.
Online version of existing [Z-score](#z-score) implementation with the same exact behavior and implications {{% available_from "v1.15.0" anomaly %}}.
<div class="model-details">
{{% collapse name="Model-specific arguments" %}}
- `class` (string) - model class name `"model.online.OnlineZscoreModel"` (or `zscore_online`with class alias support{{% available_from "v1.13.0" anomaly %}})
- `z_threshold` (float, optional) - [standard score](https://en.wikipedia.org/wiki/Standard_score) for calculation boundaries and anomaly score. Defaults to `2.5`.
- `min_n_samples_seen` (int, optional) - the minimum number of samples to be seen (`n_samples_seen_` property) before computing the anomaly score. Otherwise, the **anomaly score will be 0**, as there is not enough data to trust the model's predictions. Defaults to 16.
- `history_strength` (float, optional) - {{% available_from "v1.30.0" anomaly %}} strength of the initial history learned by `fit`. Values above `1` keep fitted mean and variance unchanged initially but reduce the leverage of subsequent updates. Defaults to `1`.
{{% /collapse %}}
{{% collapse name="Configuration example" %}}
```yaml
models:
your_desired_alias_for_a_model:
class: "zscore_online" # or 'model.online.OnlineZscoreModel'
z_threshold: 3.5
min_n_samples_seen: 128 # i.e. calculate it as full seasonality / data freq
history_strength: 2 # retain fitted history as a stronger prior
provide_series: ['anomaly_score', 'yhat'] # common arg example
# Common arguments for built-in model, if not set, default to
# See https://docs.victoriametrics.com/anomaly-detection/components/models/#common-args
#
# provide_series: ['anomaly_score', 'yhat', 'yhat_lower', 'yhat_upper']
# schedulers: [all scheduler aliases defined in `scheduler` section]
# queries: [all query aliases defined in `reader.queries` section]
# detection_direction: 'both' # meaning both drops and spikes will be captured
# min_dev_from_expected: [0.0, 0.0] # meaning, no minimal threshold is applied to prevent smaller anomalies
# scale: [1.0, 1.0] # if needed, prediction intervals' width can be increased (>1) or narrowed (<1)
# clip_predictions: False # if data_range for respective `queries` is set in reader, `yhat.*` columns will be clipped
# anomaly_score_outside_data_range: 1.01 # auto anomaly score (1.01) if `y` (real value) is outside of data_range, if set
```
{{% /collapse %}}
</div>
Resulting metrics of the model are described [here](#vmanomaly-output).
### [Rolling Quantile](https://en.wikipedia.org/wiki/Quantile)
> `RollingQuantileModel` **is** {{% available_from "v1.28.0" anomaly %}} a [univariate](#univariate-models), [online](#online-models) model. It **was** {{% deprecated_from "v1.28.0" anomaly %}} a [univariate](#univariate-models), [rolling](#rolling-models), [offline](#offline-models) model.
This model is best used on **data with short evolving patterns** (i.e. 10-100 datapoints of particular frequency), as it adapts to changes over a rolling window.
<div class="model-details">
{{% collapse name="Model-specific arguments" %}}
- `class` (string) - model class name `"model.rolling_quantile.RollingQuantileModel"` (or `rolling_quantile` with class alias support {{% available_from "v1.13.0" anomaly %}})
- `quantile` (float) - quantile value, from 0.5 to 1.0. This constraint is implied by 2-sided confidence interval.
- `window_steps` (integer) - size of the moving window. (see 'sampling_period')
- `iqr_threshold` (float, optional) - {{% available_from "v1.25.0" anomaly %}} The [interquartile range (IQR)](https://en.wikipedia.org/wiki/Interquartile_range) multiplier to increase the width of the prediction intervals. Defaults to 0 (no adjustment) for backward compatibility. If set > 0, the model will add half IQR * `iqr_threshold` to `yhat_lower` and `yhat_upper`. This is useful for data with high variance or outliers, as it helps to avoid false positives in anomaly detection.
{{% /collapse %}}
{{% collapse name="Configuration example" %}}
```yaml
models:
your_desired_alias_for_a_model:
class: "rolling_quantile"
quantile: 0.9
window_steps: 96
iqr_threshold: 1
# Common arguments for built-in model, if not set, default to
# See https://docs.victoriametrics.com/anomaly-detection/components/models/#common-args
#
# provide_series: ['anomaly_score', 'yhat', 'yhat_lower', 'yhat_upper']
# schedulers: [all scheduler aliases defined in `scheduler` section]
# queries: [all query aliases defined in `reader.queries` section]
# detection_direction: 'both' # meaning both drops and spikes will be captured
# min_dev_from_expected: [0.0, 0.0] # meaning, no minimal threshold is applied to prevent smaller anomalies
# scale: [1.0, 1.0] # if needed, prediction intervals' width can be increased (>1) or narrowed (<1)
# clip_predictions: False # if data_range for respective `queries` is set in reader, `yhat.*` columns will be clipped
# anomaly_score_outside_data_range: 1.01 # auto anomaly score (1.01) if `y` (real value) is outside of data_range, if set
```
{{% /collapse %}}
</div>
Resulting metrics of the model are described [here](#vmanomaly-output).
### [Prophet](https://facebook.github.io/prophet/)
`vmanomaly` uses the Facebook Prophet implementation for time series forecasting, with detailed usage provided in the [Prophet library documentation](https://facebook.github.io/prophet/docs/quick_start#python-api). All original Prophet parameters are supported and can be directly passed to the model via `args` argument.
> `ProphetModel` is a [univariate](#univariate-models), [offline](#offline-models) model.
> [!NOTE]
> Prophet is planned for deprecation in a future release. For new forecasting and anomaly-detection deployments, prefer the online [Temporal Envelope](#temporal-envelope) model unless Prophet-specific decomposition output is required.
> {{% available_from "v1.25.3" anomaly %}} Producing forecasts for future timestamps is now supported. To enable this, set the `forecast_at` argument to a list of relative future offsets (e.g., `['1h', '1d']`). The model will then generate forecasts for these future timestamps, which can be useful for planning and resource allocation. Output series are affected by [provide_series](#provide-series) argument, which need to include at least `yhat` for point-wise forecasts (and `yhat_lower` or/and `yhat_upper` for respective confidence intervals). See the example below for more details.
<div class="model-details">
@@ -993,271 +1199,15 @@ Depending on chosen `seasonality` parameter FB Prophet can return additional met
Resulting metrics of the model are described [here](#vmanomaly-output)
### Online Z-score
> `OnlineZscoreModel` is a [univariate](#univariate-models), [online](#online-models) model.
Online version of existing [Z-score](#z-score) implementation with the same exact behavior and implications {{% available_from "v1.15.0" anomaly %}}.
<div class="model-details">
{{% collapse name="Model-specific arguments" %}}
- `class` (string) - model class name `"model.online.OnlineZscoreModel"` (or `zscore_online`with class alias support{{% available_from "v1.13.0" anomaly %}})
- `z_threshold` (float, optional) - [standard score](https://en.wikipedia.org/wiki/Standard_score) for calculation boundaries and anomaly score. Defaults to `2.5`.
- `min_n_samples_seen` (int, optional) - the minimum number of samples to be seen (`n_samples_seen_` property) before computing the anomaly score. Otherwise, the **anomaly score will be 0**, as there is not enough data to trust the model's predictions. Defaults to 16.
- `history_strength` (float, optional) - {{% available_from "v1.30.0" anomaly %}} strength of the initial history learned by `fit`. Values above `1` keep fitted mean and variance unchanged initially but reduce the leverage of subsequent updates. Defaults to `1`.
{{% /collapse %}}
{{% collapse name="Configuration example" %}}
```yaml
models:
your_desired_alias_for_a_model:
class: "zscore_online" # or 'model.online.OnlineZscoreModel'
z_threshold: 3.5
min_n_samples_seen: 128 # i.e. calculate it as full seasonality / data freq
history_strength: 2 # retain fitted history as a stronger prior
provide_series: ['anomaly_score', 'yhat'] # common arg example
# Common arguments for built-in model, if not set, default to
# See https://docs.victoriametrics.com/anomaly-detection/components/models/#common-args
#
# provide_series: ['anomaly_score', 'yhat', 'yhat_lower', 'yhat_upper']
# schedulers: [all scheduler aliases defined in `scheduler` section]
# queries: [all query aliases defined in `reader.queries` section]
# detection_direction: 'both' # meaning both drops and spikes will be captured
# min_dev_from_expected: [0.0, 0.0] # meaning, no minimal threshold is applied to prevent smaller anomalies
# scale: [1.0, 1.0] # if needed, prediction intervals' width can be increased (>1) or narrowed (<1)
# clip_predictions: False # if data_range for respective `queries` is set in reader, `yhat.*` columns will be clipped
# anomaly_score_outside_data_range: 1.01 # auto anomaly score (1.01) if `y` (real value) is outside of data_range, if set
```
{{% /collapse %}}
</div>
Resulting metrics of the model are described [here](#vmanomaly-output).
### Online MAD
> `OnlineMADModel` is a [univariate](#univariate-models), [online](#online-models) model.
The MAD model is a robust method for anomaly detection that is *less sensitive* to outliers in data compared to standard deviation-based models. It considers a point as an anomaly if the absolute deviation from the median is significantly large. This is the online approximate version, based on [t-digests](https://www.sciencedirect.com/science/article/pii/S2665963820300403) for online quantile estimation{{% available_from "v1.15.0" anomaly %}}.
<div class="model-details">
{{% collapse name="Model-specific arguments" %}}
- `class` (string) - model class name `"model.online.OnlineMADModel"` (or `mad_online` with class alias support{{% available_from "v1.13.0" anomaly %}})
- `threshold` (float, optional) - The threshold multiplier for the MAD to determine anomalies. Defaults to `2.5`. Higher values will identify fewer points as anomalies.
- `min_n_samples_seen` (int, optional) - the minimum number of samples to be seen (`n_samples_seen_` property) before computing the anomaly score. Otherwise, the **anomaly score will be 0**, as there is not enough data to trust the model's predictions. Defaults to 16.
- `history_strength` (float, optional) - {{% available_from "v1.30.0" anomaly %}} strength of the initial history learned by `fit`. Values above `1` preserve fitted quantiles initially but reduce the leverage of subsequent updates. Defaults to `1`.
- `compression` (int, optional) - the compression parameter for underlying [t-digest](https://www.sciencedirect.com/science/article/pii/S2665963820300403). Higher values mean higher accuracy but higher memory usage. By default 100.
{{% /collapse %}}
{{% collapse name="Configuration example" %}}
```yaml
models:
your_desired_alias_for_a_model:
class: "mad_online" # or 'model.online.OnlineMADModel'
threshold: 2.5
min_n_samples_seen: 128 # i.e. calculate it as full seasonality / data freq
history_strength: 2 # retain fitted history as a stronger prior
compression: 100 # higher values mean higher accuracy but higher memory usage
provide_series: ['anomaly_score', 'yhat'] # common arg example
# Common arguments for built-in model, if not set, default to
# See https://docs.victoriametrics.com/anomaly-detection/components/models/#common-args
#
# provide_series: ['anomaly_score', 'yhat', 'yhat_lower', 'yhat_upper']
# schedulers: [all scheduler aliases defined in `scheduler` section]
# queries: [all query aliases defined in `reader.queries` section]
# detection_direction: 'both' # meaning both drops and spikes will be captured
# min_dev_from_expected: [0.0, 0.0] # meaning, no minimal threshold is applied to prevent smaller anomalies
# scale: [1.0, 1.0] # if needed, prediction intervals' width can be increased (>1) or narrowed (<1)
# clip_predictions: False # if data_range for respective `queries` is set in reader, `yhat.*` columns will be clipped
# anomaly_score_outside_data_range: 1.01 # auto anomaly score (1.01) if `y` (real value) is outside of data_range, if set
```
{{% /collapse %}}
</div>
Resulting metrics of the model are described [here](#vmanomaly-output).
### [Rolling Quantile](https://en.wikipedia.org/wiki/Quantile)
> `RollingQuantileModel` **is** {{% available_from "v1.28.0" anomaly %}} a [univariate](#univariate-models), [online](#online-models) model. It **was** {{% deprecated_from "v1.28.0" anomaly %}} a [univariate](#univariate-models), [rolling](#rolling-models), [offline](#offline-models) model.
This model is best used on **data with short evolving patterns** (i.e. 10-100 datapoints of particular frequency), as it adapts to changes over a rolling window.
<div class="model-details">
{{% collapse name="Model-specific arguments" %}}
- `class` (string) - model class name `"model.rolling_quantile.RollingQuantileModel"` (or `rolling_quantile` with class alias support {{% available_from "v1.13.0" anomaly %}})
- `quantile` (float) - quantile value, from 0.5 to 1.0. This constraint is implied by 2-sided confidence interval.
- `window_steps` (integer) - size of the moving window. (see 'sampling_period')
- `iqr_threshold` (float, optional) - {{% available_from "v1.25.0" anomaly %}} The [interquartile range (IQR)](https://en.wikipedia.org/wiki/Interquartile_range) multiplier to increase the width of the prediction intervals. Defaults to 0 (no adjustment) for backward compatibility. If set > 0, the model will add half IQR * `iqr_threshold` to `yhat_lower` and `yhat_upper`. This is useful for data with high variance or outliers, as it helps to avoid false positives in anomaly detection.
{{% /collapse %}}
{{% collapse name="Configuration example" %}}
```yaml
models:
your_desired_alias_for_a_model:
class: "rolling_quantile"
quantile: 0.9
window_steps: 96
iqr_threshold: 1
# Common arguments for built-in model, if not set, default to
# See https://docs.victoriametrics.com/anomaly-detection/components/models/#common-args
#
# provide_series: ['anomaly_score', 'yhat', 'yhat_lower', 'yhat_upper']
# schedulers: [all scheduler aliases defined in `scheduler` section]
# queries: [all query aliases defined in `reader.queries` section]
# detection_direction: 'both' # meaning both drops and spikes will be captured
# min_dev_from_expected: [0.0, 0.0] # meaning, no minimal threshold is applied to prevent smaller anomalies
# scale: [1.0, 1.0] # if needed, prediction intervals' width can be increased (>1) or narrowed (<1)
# clip_predictions: False # if data_range for respective `queries` is set in reader, `yhat.*` columns will be clipped
# anomaly_score_outside_data_range: 1.01 # auto anomaly score (1.01) if `y` (real value) is outside of data_range, if set
```
{{% /collapse %}}
</div>
Resulting metrics of the model are described [here](#vmanomaly-output).
### Online Seasonal Quantile
> `OnlineQuantileModel` is a [univariate](#univariate-models), [online](#online-models) model.
Online (seasonal) quantile utilizes a set of approximate distributions, based on [t-digests](https://www.sciencedirect.com/science/article/pii/S2665963820300403) for online quantile estimation {{% available_from "v1.15.0" anomaly %}}.
Best used on **[de-trended](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) data with strong (potentially multiple) [seasonalities](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality)**. Can act as a (slightly less flexible) replacement to [`ProphetModel`](#prophet).
It uses the `quantiles` triplet to calculate `yhat_lower`, `yhat`, and `yhat_upper` [output](#vmanomaly-output), respectively, for each of the `min_subseasons` sub-intervals contained in `seasonal_interval`. For example, with '4d' + '2h' seasonality patterns (multiple), it will hold and update 24*4 / 2 = 48 consecutive estimates (each 2 hours long).
<div class="model-details">
{{% collapse name="Model-specific arguments" %}}
- `class` (string) - model class name `"model.online.OnlineQuantileModel"` (or `quantile_online` with class alias support{{% available_from "v1.13.0" anomaly %}})
- `quantiles` (list[float], optional) - The quantiles to estimate. `yhat_lower`, `yhat`, `yhat_upper` are the quantile order. By default (0.01, 0.5, 0.99).
- `iqr_threshold` (float, optional) - {{% available_from "v1.25.0" anomaly %}} The [interquartile range (IQR)](https://en.wikipedia.org/wiki/Interquartile_range) multiplier to increase the width of the prediction intervals. Defaults to 0 (no adjustment) for backward compatibility. If set > 0, the model will add IQR * `iqr_threshold` to `yhat_lower` and `yhat_upper` (respecting `min_subseason` seasonal buckets). This is useful for data with high variance or outliers, as it helps to avoid false positives in anomaly detection. Best used with **robust** `quantiles` set to (0.25, 0.5, 0.75) or similar.
- `seasonal_interval` (string, optional) - the interval for the seasonal adjustment. If not set, the model will equal to a simple online quantile model. By default not set.
- `min_subseason` (str, optional) - the minimum interval to estimate quantiles for. By default not set. Note that the minimum interval should be a multiple of the seasonal interval, i.e. if seasonal_interval='2h', then min_subseason='15m' is valid, but '37m' is not.
- `use_transform` (bool, optional) - whether to internally apply a `log1p(abs(x)) * sign(x)` transformation to the data to stabilize internal quantile estimation. Does not affect the scale of produced output (i.e. `yhat`) By default False.
- `global_smoothing` (float, optional) - the smoothing parameter for the global quantiles. i.e. the output is a weighted average of the global and seasonal quantiles (if `seasonal_interval` and `min_subseason` args are set). Should be from `[0, 1]` interval, where 0 means no smoothing and 1 means using only global quantile values.
- `scale` (float, optional) - Is used to adjust the margins between `yhat` and [`yhat_lower`, `yhat_upper`]. New margin = `|yhat_* - yhat_lower| * scale`. Defaults to 1 (no scaling is applied). See `scale`[common arg](https://docs.victoriametrics.com/anomaly-detection/components/models/#scale) section for detailed instructions and 2-sided option.
- `season_starts_from` (str, optional) - the start date for the seasonal adjustment, as a reference point to start counting the intervals. By default '1970-01-01'.
- `min_n_samples_seen` (int, optional) - the minimum number of samples to be seen (`n_samples_seen_` property) before computing the anomaly score. Otherwise, the **anomaly score will be 0**, as there is not enough data to trust the model's predictions. Defaults to 16.
- `history_strength` (float, optional) - {{% available_from "v1.30.0" anomaly %}} strength of the initial history learned by `fit`. Values above `1` preserve fitted quantiles initially but reduce the leverage of subsequent updates. Defaults to `1`.
- `compression` (int, optional) - the compression parameter for the underlying [t-digests](https://www.sciencedirect.com/science/article/pii/S2665963820300403). Higher values mean higher accuracy but higher memory usage. By default 100.
{{% /collapse %}}
{{% collapse name="Configuration example" %}}
Suppose we have a data with strong intra-day (hourly) and intra-week (daily) seasonality, data granularity is '5m' with up to 5% expected outliers present in data. Then you can apply similar config:
```yaml
models:
your_desired_alias_for_a_model:
class: "quantile_online" # or 'model.online.OnlineQuantileModel'
quantiles: [0.25, 0.5, 0.75] # lowered to exclude anomalous edges, can be compensated by `scale` param > 1 and `iqr_threshold` > 0
iqr_threshold: 2.5 # to increase prediction intervals' width to avoid false positives while still keeping the model robust
seasonal_interval: '7d' # longest seasonality (week, day) = week, starting from `season_starts_from`
min_subseason: '1h' # smallest seasonality (week, day, hour) = hour, will have its own quantile estimates
min_n_samples_seen: 288 # 1440 / 5 - at least 1 full day, ideal = 1440 / 5 * 7 - one full week (seasonal_interval)
history_strength: 2 # retain fitted history as a stronger prior
scale: 1.1 # to compensate lowered quantile boundaries with wider intervals
season_starts_from: '2024-01-01' # interval calculation starting point, especially for uncommon seasonalities like '36h' or '12d'
compression: 100 # higher values mean higher accuracy but higher memory usage
provide_series: ['anomaly_score', 'yhat'] # common arg example
# Common arguments for built-in model, if not set, default to
# See https://docs.victoriametrics.com/anomaly-detection/components/models/#common-args
#
# provide_series: ['anomaly_score', 'yhat', 'yhat_lower', 'yhat_upper']
# schedulers: [all scheduler aliases defined in `scheduler` section]
# queries: [all query aliases defined in `reader.queries` section]
# detection_direction: 'both' # meaning both drops and spikes will be captured
# min_dev_from_expected: [0.0, 0.0] # meaning, no minimal threshold is applied to prevent smaller anomalies
# scale: [1.0, 1.0] # if needed, prediction intervals' width can be increased (>1) or narrowed (<1)
# clip_predictions: False # if data_range for respective `queries` is set in reader, `yhat.*` columns will be clipped
# anomaly_score_outside_data_range: 1.01 # auto anomaly score (1.01) if `y` (real value) is outside of data_range, if set
```
{{% /collapse %}}
</div>
Resulting metrics of the model are described [here](#vmanomaly-output).
### [Seasonal Trend Decomposition](https://en.wikipedia.org/wiki/Seasonal_adjustment)
> `StdModel` **is** {{% available_from "v1.28.0" anomaly %}} a [univariate](#univariate-models), [online](#online-models) model. It **was** {{% deprecated_from "v1.28.0" anomaly %}} a [univariate](#univariate-models), [rolling](#rolling-models), [offline](#offline-models) model.
Here we use Seasonal Decompose implementation from `statsmodels` [library](https://www.statsmodels.org/dev/generated/statsmodels.tsa.seasonal.seasonal_decompose). Parameters from this library can be passed to the model. Some parameters are specifically predefined in `vmanomaly` and can't be changed by user (`model`='additive', `two_sided`=False).
<div class="model-details">
{{% collapse name="Model-specific arguments" %}}
- `class` (string) - model class name `"model.std.StdModel"` (or `std` with class alias support{{% available_from "v1.13.0" anomaly %}})
- `period` (integer) - Number of datapoints in one season.
- `z_threshold` (float, optional) - [standard score](https://en.wikipedia.org/wiki/Standard_score) for calculating boundaries to define anomaly score. Defaults to `2.5`.
{{% /collapse %}}
{{% collapse name="Configuration example" %}}
```yaml
models:
your_desired_alias_for_a_model:
class: "std" # or 'model.std.StdModel' starting from v1.13.0
period: 2
# Common arguments for built-in model, if not set, default to
# See https://docs.victoriametrics.com/anomaly-detection/components/models/#common-args
#
# provide_series: ['anomaly_score', 'yhat', 'yhat_lower', 'yhat_upper']
# schedulers: [all scheduler aliases defined in `scheduler` section]
# queries: [all query aliases defined in `reader.queries` section]
# detection_direction: 'both' # meaning both drops and spikes will be captured
# min_dev_from_expected: [0.0, 0.0] # meaning, no minimal threshold is applied to prevent smaller anomalies
# scale: [1.0, 1.0] # if needed, prediction intervals' width can be increased (>1) or narrowed (<1)
# clip_predictions: False # if data_range for respective `queries` is set in reader, `yhat.*` columns will be clipped
# anomaly_score_outside_data_range: 1.01 # auto anomaly score (1.01) if `y` (real value) is outside of data_range, if set
```
{{% /collapse %}}
</div>
Resulting metrics of the model are described [here](#vmanomaly-output).
**Additional output metrics produced by Seasonal Trend Decomposition model**
- `resid` - The residual component of the data series.
- `trend` - The trend component of the data series.
- `seasonal` - The seasonal component of the data series.
### [Isolation forest](https://en.wikipedia.org/wiki/Isolation_forest) (Multivariate)
> `IsolationForestModel` is a [univariate](#univariate-models), [offline](#offline-models) model.
> `IsolationForestMultivariateModel` is a [multivariate](#multivariate-models), [offline](#offline-models) model.
> [!NOTE]
> Both univariate `isolation_forest` and multivariate `isolation_forest_multivariate` are planned for deprecation in a future release. For new deployments, use the corresponding univariate or multivariate online [Temporal Envelope](#temporal-envelope) model.
Detects anomalies using binary trees. The algorithm has a linear time complexity and a low memory requirement, which works well with high-volume data. It can be used on both univariate and multivariate data, but it is more effective in multivariate case.
**Important**: Be aware of [the curse of dimensionality](https://en.wikipedia.org/wiki/Curse_of_dimensionality). Don't use single multivariate model if you expect your queries to return many time series of less datapoints that the number of metrics. In such case it is hard for a model to learn meaningful dependencies from too sparse data hypercube.
@@ -1319,6 +1269,9 @@ Resulting metrics of the model are described [here](#vmanomaly-output).
> `HoltWinters` is a [univariate](#univariate-models), [offline](#offline-models) model.
> [!NOTE]
> Holt-Winters is planned for deprecation in a future release. For new deployments, prefer the online [Temporal Envelope](#temporal-envelope) model.
Here we use Holt-Winters Exponential Smoothing implementation from `statsmodels` [library](https://www.statsmodels.org/dev/generated/statsmodels.tsa.holtwinters.ExponentialSmoothing). All parameters from this library can be passed to the model.
<div class="model-details">
@@ -1378,6 +1331,55 @@ models:
Resulting metrics of the model are described [here](#vmanomaly-output).
### [Seasonal Trend Decomposition](https://en.wikipedia.org/wiki/Seasonal_adjustment)
> `StdModel` **is** {{% available_from "v1.28.0" anomaly %}} a [univariate](#univariate-models), [online](#online-models) model. It **was** {{% deprecated_from "v1.28.0" anomaly %}} a [univariate](#univariate-models), [rolling](#rolling-models), [offline](#offline-models) model.
Here we use Seasonal Decompose implementation from `statsmodels` [library](https://www.statsmodels.org/dev/generated/statsmodels.tsa.seasonal.seasonal_decompose). Parameters from this library can be passed to the model. Some parameters are specifically predefined in `vmanomaly` and can't be changed by user (`model`='additive', `two_sided`=False).
<div class="model-details">
{{% collapse name="Model-specific arguments" %}}
- `class` (string) - model class name `"model.std.StdModel"` (or `std` with class alias support{{% available_from "v1.13.0" anomaly %}})
- `period` (integer) - Number of datapoints in one season.
- `z_threshold` (float, optional) - [standard score](https://en.wikipedia.org/wiki/Standard_score) for calculating boundaries to define anomaly score. Defaults to `2.5`.
{{% /collapse %}}
{{% collapse name="Configuration example" %}}
```yaml
models:
your_desired_alias_for_a_model:
class: "std" # or 'model.std.StdModel' starting from v1.13.0
period: 2
# Common arguments for built-in model, if not set, default to
# See https://docs.victoriametrics.com/anomaly-detection/components/models/#common-args
#
# provide_series: ['anomaly_score', 'yhat', 'yhat_lower', 'yhat_upper']
# schedulers: [all scheduler aliases defined in `scheduler` section]
# queries: [all query aliases defined in `reader.queries` section]
# detection_direction: 'both' # meaning both drops and spikes will be captured
# min_dev_from_expected: [0.0, 0.0] # meaning, no minimal threshold is applied to prevent smaller anomalies
# scale: [1.0, 1.0] # if needed, prediction intervals' width can be increased (>1) or narrowed (<1)
# clip_predictions: False # if data_range for respective `queries` is set in reader, `yhat.*` columns will be clipped
# anomaly_score_outside_data_range: 1.01 # auto anomaly score (1.01) if `y` (real value) is outside of data_range, if set
```
{{% /collapse %}}
</div>
Resulting metrics of the model are described [here](#vmanomaly-output).
**Additional output metrics produced by Seasonal Trend Decomposition model**
- `resid` - The residual component of the data series.
- `trend` - The trend component of the data series.
- `seasonal` - The seasonal component of the data series.
## vmanomaly output
`vmanomaly` generates model-dependent output series. Their metric names can be configured in the writer section.
@@ -1559,7 +1561,7 @@ See the [component configuration reference](https://docs.victoriametrics.com/ano
Pull the `vmanomaly` image:
```sh
docker pull victoriametrics/vmanomaly:v1.30.0
docker pull victoriametrics/vmanomaly:v1.30.1
```
Mount the module at `/vmanomaly/src/model/custom.py`, which matches the configured import path `model.custom.CustomModel`. Validate the complete configuration with `--dryRun` before starting the long-running service.
@@ -1569,7 +1571,7 @@ docker run --rm \
-v "$PWD/license:/license:ro" \
-v "$PWD/custom_model.py:/vmanomaly/src/model/custom.py:ro" \
-v "$PWD/config.yaml:/config.yaml:ro" \
victoriametrics/vmanomaly:v1.30.0 \
victoriametrics/vmanomaly:v1.30.1 \
/config.yaml \
--licenseFile=/license \
--dryRun

View File

@@ -588,7 +588,7 @@ Label names [description](#labelnames)
`Counter`
</td>
<td>The number of valid datapoints accepted by `model_alias`, excluding NaN and Inf values, during `fit`, `infer`, or combined `fit_infer` execution for the `query_key` query.</td>
<td>The number of valid datapoints accepted by `model_alias`, excluding NaN and Inf values, during `fit`, `infer`, or combined `fit_infer` execution for the `query_key` query. During inference, only previously unseen valid rows are counted {{% available_from "v1.30.1" anomaly %}}.</td>
<td>
`stage`, `query_key`, `model_alias`, `scheduler_alias`, `preset`
@@ -687,7 +687,7 @@ Label names [description](#labelnames)
`Histogram` (was `Summary`{{% deprecated_from "v1.17.0" anomaly %}})
</td>
<td>The total time (in seconds) taken by write requests to VictoriaMetrics `url` for the `query_key` query within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.
<td>The total time (in seconds) taken by write requests to VictoriaMetrics `url` for the `query_key` query within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode. Successful and handled failed attempts, including connection retries, are observed {{% available_from "v1.30.1" anomaly %}}.
</td>
<td>

View File

@@ -87,7 +87,7 @@ There is change {{% available_from "v1.13.0" anomaly %}} of [`queries`](https://
- `max_points_per_query`{{% available_from "v1.17.0" anomaly %}} (int): Optional arg, overrides how `search.maxPointsPerTimeseries` flag{{% available_from "v1.14.1" anomaly %}} impacts `vmanomaly` on splitting long `fit_window` [queries](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) into smaller sub-intervals. This helps users avoid hitting the `search.maxQueryDuration` limit for individual queries by distributing initial query across multiple subquery requests with minimal overhead. Set less than `search.maxPointsPerTimeseries` if hitting `maxQueryDuration` limits. If set on a query-level, it overrides the global `max_points_per_query` (reader-level).
- `tz`{{% available_from "v1.18.0" anomaly %}} (string): this optional argument enables timezone specification per query, overriding the readers default `tz`. This setting helps to account for local timezone shifts, such as [DST](https://en.wikipedia.org/wiki/Daylight_saving_time), in models that are sensitive to seasonal variations (e.g., [`ProphetModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) or [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile)).
- `tz`{{% available_from "v1.18.0" anomaly %}} (string): this optional argument enables timezone specification per query, overriding the readers default `tz`. This setting helps to account for local timezone shifts, such as [DST](https://en.wikipedia.org/wiki/Daylight_saving_time), in models that are sensitive to seasonal variations (e.g., [`TemporalEnvelopeModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope) or [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile)).
- `tenant_id` {{% available_from "v1.19.0" anomaly %}} (string): this optional argument enables tenant-level separation for queries (e.g. `query1` to get the data from tenant "0:0", `query2` - from tenant "1:0"). It works as follows:
- if *not set, inherits* reader-level `tenant_id`
@@ -441,7 +441,7 @@ Optional arg{{% available_from "v1.17.0" anomaly %}} overrides how `search.maxPo
`UTC`
</td>
<td>
Optional argument {{% available_from "v1.18.0" anomaly %}} specifies the [IANA](https://nodatime.org/TimeZones) timezone to account for local shifts, like [DST](https://en.wikipedia.org/wiki/Daylight_saving_time), in models sensitive to seasonal patterns (e.g., [`TemporalEnvelopeModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope), [`ProphetModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet), or [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile)). Defaults to `UTC` if not set and can be overridden on a [per-query basis](#per-query-parameters).
Optional argument {{% available_from "v1.18.0" anomaly %}} specifies the [IANA](https://nodatime.org/TimeZones) timezone to account for local shifts, like [DST](https://en.wikipedia.org/wiki/Daylight_saving_time), in models sensitive to seasonal patterns (e.g., [`TemporalEnvelopeModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope) or [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile)). Defaults to `UTC` if not set and can be overridden on a [per-query basis](#per-query-parameters).
</td>
</tr>
<tr>
@@ -825,7 +825,7 @@ Frequency of the points returned. Will be converted to `/select/stats_query_rang
`America/New_York`
</td>
<td>
(Optional) Specifies the [IANA](https://nodatime.org/TimeZones) timezone to account for local shifts, like [DST](https://en.wikipedia.org/wiki/Daylight_saving_time), in models sensitive to seasonal patterns (e.g., [`ProphetModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) or [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile)). Defaults to `UTC` if not set and can be overridden on a [per-query basis](#per-query-parameters).
(Optional) Specifies the [IANA](https://nodatime.org/TimeZones) timezone to account for local shifts, like [DST](https://en.wikipedia.org/wiki/Daylight_saving_time), in models sensitive to seasonal patterns (e.g., [`TemporalEnvelopeModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#temporal-envelope) or [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile)). Defaults to `UTC` if not set and can be overridden on a [per-query basis](#per-query-parameters).
</td>
</tr>
<tr>

View File

@@ -74,6 +74,8 @@ options={`"scheduler.periodic.PeriodicScheduler"`, `"scheduler.oneoff.OneoffSche
> {{% available_from "v1.30.0" anomaly %}} If a periodic scheduler worker exits unexpectedly, the service attempts bounded restarts with exponential backoff instead of shutting down unrelated schedulers. Monitor [`vmanomaly_scheduler_alive`](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#startup-metrics) and `vmanomaly_scheduler_restarts_total` to alert on persistent failures.
> {{% available_from "v1.30.1" anomaly %}} For exact-capable [online models](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-models), `infer_every` is also the causal model-update cadence. If a delayed periodic job fetches several observations at once, they are processed on the same chronological grid used by exact backtesting rather than as one behaviorally different batch.
### Parameters
For periodic scheduler parameters are defined as differences in times, expressed in difference units, e.g. days, hours, minutes, seconds. Time granularity is defined by the last characters of a string. Examples: `"50s"` (seconds), `"4m"` (minutes), `"3h"` (hours), `"2d"` (days), `"1w"` (weeks).
@@ -440,7 +442,7 @@ In **Inference only** mode {{% available_from "v1.22.1" anomaly %}}, the schedul
- `fit_window`: Duration of historical data used for each training run (e.g. `P7D`, `PT1H`).
- `fit_every`: Interval between consecutive training/inference cycles.
- {{% available_from "v1.28.0" anomaly %}} `exact`: If set to `true`, BacktestingScheduler will execute inference for online models in small chronological batches equal to `infer_every` to mimic the production scheduler. (default: `false`)
- {{% available_from "v1.28.0" anomaly %}} `infer_every`: Optional inference cadence for exact mode, defining how often the scheduler should call infer between two fits, otherwise defaults to `fit_every` when unset.
- {{% available_from "v1.28.0" anomaly %}} `infer_every`: Optional inference grid and, in exact mode, model-call cadence between two fits. {{% available_from "v1.30.1" anomaly %}} In `inference_only` mode, an omitted value is derived from the effective query step or reader sampling period and capped by `fit_every`; it falls back to `fit_every` only when neither reader value is available.
- `n_jobs`: Number of parallel jobs for backtesting (default: `1`).
#### Example

View File

@@ -74,5 +74,7 @@ Rest API endpoints (e.g. `/metrics`) can be accessed at `<vmanomaly-host>:8490/v
- `GET /api/v1/autotune/tasks/{task_id}` returns progress and the concrete suggested `modelConfig` when complete.
- `DELETE /api/v1/autotune/tasks/{task_id}` cancels pending work cooperatively.
{{% available_from "v1.30.1" anomaly %}} Seasonality analysis preserves the original timestamp grid when samples are offset from whole step boundaries. This avoids missing daily or weekly patterns solely because timestamps are shifted within the configured sampling interval.
> [!TIP]
> For a complete request and recommended workflow, see [Shared asynchronous autotune workflow](https://docs.victoriametrics.com/anomaly-detection/components/models/#shared-asynchronous-autotune-workflow). OpenAPI schemas for the running version are available at `/docs` endpoint of a running `vmanomaly` instance.

View File

@@ -42,7 +42,7 @@ schedulers:
# other schedulers
models:
zscore_online_override:
zscore_online_inherited:
class: zscore_online
z_threshold: 3.5
clip_predictions: True
@@ -73,6 +73,7 @@ reader:
writer:
class: "vm"
datasource_url: http://localhost:8428
tenant_id: "0"
metric_format:
__name__: "$VAR"
for: "$QUERY_KEY"
@@ -249,12 +250,11 @@ models:
class: zscore_online
z_threshold: 3.5
schedulers: ['periodic_1d']
prophet:
class: prophet
temporal_envelope:
class: temporal_envelope
schedulers: ['periodic_1d']
queries: ['q1', 'q2']
args:
interval_width: 0.98
seasonalities: ['hod_smooth', 'dow_smooth']
reader:
class: vm
datasource_url: 'https://play.victoriametrics.com'
@@ -268,7 +268,7 @@ reader:
# other components like writer, monitoring, etc.
```
if the service is restarted in less than 1 hour after the last training (now < next scheduled fit time), it will restore the state of the `zscore_online` and `prophet` models if their signature (class, hyperparameters, schedulers, etc.) has not changed. It will load the trained model instances or their training data from disk and continue producing [anomaly scores](https://docs.victoriametrics.com/anomaly-detection/faq/#what-is-anomaly-score) without retraining. If there are changes or new queries added to the configuration, the service will add these to scheduled jobs for fit and infer. That's what is changed and what is restored in a config below:
if the service is restarted in less than 1 hour after the last training (now < next scheduled fit time), it will restore the state of the `zscore_online` and `temporal_envelope` models if their signature (class, hyperparameters, schedulers, etc.) has not changed. It will load the trained model instances or their training data from disk and continue producing [anomaly scores](https://docs.victoriametrics.com/anomaly-detection/faq/#what-is-anomaly-score) without retraining. If there are changes or new queries added to the configuration, the service will add these to scheduled jobs for fit and infer. That's what is changed and what is restored in a config below:
```yaml
settings:
@@ -285,12 +285,11 @@ models:
class: zscore_online # unchanged, still the same model class
z_threshold: 3.0 # changed, needs retraining!
schedulers: ['periodic_1d'] # unchanged, still attached to the same scheduler
prophet: # can be partially reused, because its class and schedulers are unchanged but queries have changed
class: prophet # unchanged, still the same model class
temporal_envelope: # can be partially reused, because its class and schedulers are unchanged but queries have changed
class: temporal_envelope # unchanged, still the same model class
schedulers: ['periodic_1d'] # unchanged, still attached to the same scheduler
queries: ['q1', 'q3'] # changed, added new query 'q3', drops 'q2', so (prophet, q2) should be trained from scratch
args:
interval_width: 0.98 # unchanged, still the same argument
queries: ['q1', 'q3'] # changed, added new query 'q3', drops 'q2', so (temporal_envelope, q2) should be trained from scratch
seasonalities: ['hod_smooth', 'dow_smooth'] # unchanged
reader: # can be partially reused, because its class and datasource URL are unchanged, but queries have changed
class: vm # unchanged, still the same reader class
datasource_url: 'https://play.victoriametrics.com' # unchanged, still the same datasource URL
@@ -301,13 +300,13 @@ reader: # can be partially reused, because its class and datasource URL are unc
q2:
expr: 'some_metricsql_query_2' # will be removed, no longer used by any model
q3:
expr: 'some_metricsql_query_3' # new query, added to the reader, and used by the `prophet` model
expr: 'some_metricsql_query_3' # new query, added to the reader, and used by the `temporal_envelope` model
sampling_period: 30s # unchanged, still the same sampling period
# other components like writer, monitoring, etc. remain unchanged
```
This means that the service upon restart:
1. Won't restore the state of `zscore_online` model, because its `z_threshold` argument **has changed**, retraining from scratch is needed on the last `fit_window` = 24 hours of data for `q1`, `q2` and `q3` (as model's `queries` arg is not set so it defaults to all queries found in the reader).
2. Will **partially** restore the state of `prophet` model, because its class and schedulers are unchanged, but **only instances trained on timeseries returned by `q1` query**. New fit/infer jobs will be set for new query `q3`. The old query `q2` artifacts will be dropped upon restart - all respective models and data for (`prophet`, `q2`) combination will be removed from the database file and from the disk.
2. Will **partially** restore the state of `temporal_envelope` model, because its class and schedulers are unchanged, but **only instances trained on timeseries returned by `q1` query**. New fit/infer jobs will be set for new query `q3`. The old query `q2` artifacts will be dropped upon restart - all respective models and data for (`temporal_envelope`, `q2`) combination will be removed from the database file and from the disk.
{{% /collapse %}}
@@ -371,7 +370,7 @@ models:
queries: ['q1']
# other model args
m2: # model instances will be likely dropped during retention checks due to high churn rate
class: prophet
class: temporal_envelope
schedulers: ['s1']
queries: ['q2']
# other model args
@@ -406,7 +405,7 @@ settings:
restore_state: True # enables state restoration
logger_levels:
reader.vm: DEBUG # affects only VmReader logs
model: WARNING # applies to all components with 'model' prefix, such as 'model.zscore_online', 'model.prophet', etc.
model: WARNING # applies to all components with 'model' prefix, such as 'model.zscore_online', 'model.online.temporal_envelope', etc.
# once commented out in hot-reload mode, will use the default logger level set by --loggerLevel command line argument
# monitoring.push: critical
```

View File

@@ -10,9 +10,9 @@ sitemap:
- To use *vmanomaly*, part of the enterprise package, a license key is required. Obtain your key [here](https://victoriametrics.com/products/enterprise/trial/) for this tutorial or for enterprise use.
- In the tutorial, we'll be using the following VictoriaMetrics components:
- [VictoriaMetrics Single-Node](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) (v1.148.0)
- [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/) (v1.148.0)
- [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) (v1.148.0)
- [VictoriaMetrics Single-Node](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) (v1.149.0)
- [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/) (v1.149.0)
- [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) (v1.149.0)
- [Grafana](https://grafana.com/) (v12.2.0)
- [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/)
- [Node exporter](https://github.com/prometheus/node_exporter#node-exporter) (v1.9.1) and [Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/) (v0.28.1)
@@ -323,7 +323,7 @@ Let's wrap it all up together into the `docker-compose.yml` file.
services:
vmagent:
container_name: vmagent
image: victoriametrics/vmagent:v1.148.0
image: victoriametrics/vmagent:v1.149.0
depends_on:
- "victoriametrics"
ports:
@@ -340,7 +340,7 @@ services:
victoriametrics:
container_name: victoriametrics
image: victoriametrics/victoria-metrics:v1.148.0
image: victoriametrics/victoria-metrics:v1.149.0
ports:
- 8428:8428
volumes:
@@ -373,7 +373,7 @@ services:
vmalert:
container_name: vmalert
image: victoriametrics/vmalert:v1.148.0
image: victoriametrics/vmalert:v1.149.0
depends_on:
- "victoriametrics"
ports:
@@ -395,7 +395,7 @@ services:
restart: always
vmanomaly:
container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.30.0
image: victoriametrics/vmanomaly:v1.30.1
depends_on:
- "victoriametrics"
ports:

View File

@@ -1,3 +1,11 @@
---
build:
list: never
publishResources: false
render: never
sitemap:
disable: true
---
Several VictoriaMetrics components can connect to cloud storage to read or write object data.
The following table shows the supported types of storage for each component:

View File

@@ -1,3 +1,11 @@
---
build:
list: never
publishResources: false
render: never
sitemap:
disable: true
---
Using [Grafana](https://grafana.com/) with [vmauth](https://docs.victoriametrics.com/victoriametrics/vmauth/) is an effective way to provide [multi-tenant](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy) access to your metrics, logs, and traces.
vmauth provides a way to authenticate users using [JWT tokens](https://en.wikipedia.org/wiki/JSON_Web_Token) {{% available_from "v1.138.0" %}} issued by an external identity provider.
Those tokens can include information about the user and their tenant, which vmauth can use to restrict access so users only see metrics in their own tenant.
@@ -240,23 +248,23 @@ vmagent will write data into VictoriaMetrics single-node and cluster (with tenan
# compose.yaml
services:
vmsingle:
image: victoriametrics/victoria-metrics:v1.148.0
image: victoriametrics/victoria-metrics:v1.149.0
vmstorage:
image: victoriametrics/vmstorage:v1.148.0-cluster
image: victoriametrics/vmstorage:v1.149.0-cluster
vminsert:
image: victoriametrics/vminsert:v1.148.0-cluster
image: victoriametrics/vminsert:v1.149.0-cluster
command:
- -storageNode=vmstorage:8400
vmselect:
image: victoriametrics/vmselect:v1.148.0-cluster
image: victoriametrics/vmselect:v1.149.0-cluster
command:
- -storageNode=vmstorage:8401
vmagent:
image: victoriametrics/vmagent:v1.148.0
image: victoriametrics/vmagent:v1.149.0
volumes:
- ./scrape.yaml:/etc/vmagent/config.yaml
command:
@@ -308,7 +316,7 @@ Now add the vmauth service to `compose.yaml`:
# compose.yaml
services:
vmauth:
image: docker.io/victoriametrics/vmauth:v1.148.0
image: docker.io/victoriametrics/vmauth:v1.149.0
ports:
- 8427:8427
volumes:

View File

@@ -155,15 +155,15 @@ These services will store and query the metrics scraped by vmagent.
# compose.yaml
services:
vmstorage:
image: victoriametrics/vmstorage:v1.148.0-cluster
image: victoriametrics/vmstorage:v1.149.0-cluster
vminsert:
image: victoriametrics/vminsert:v1.148.0-cluster
image: victoriametrics/vminsert:v1.149.0-cluster
command:
- -storageNode=vmstorage:8400
vmselect:
image: victoriametrics/vmselect:v1.148.0-cluster
image: victoriametrics/vmselect:v1.149.0-cluster
command:
- -storageNode=vmstorage:8401
ports:
@@ -196,7 +196,7 @@ Add the vmauth service to `compose.yaml`:
# compose.yaml
services:
vmauth:
image: victoriametrics/vmauth:v1.148.0-enterprise
image: victoriametrics/vmauth:v1.149.0-enterprise
ports:
- 8427:8427
volumes:
@@ -251,7 +251,7 @@ Add the vmagent service to `compose.yaml` with OAuth2 configuration:
# compose.yaml
services:
vmagent:
image: victoriametrics/vmagent:v1.148.0
image: victoriametrics/vmagent:v1.149.0
volumes:
- ./scrape.yaml:/etc/vmagent/config.yaml
command:

View File

@@ -107,7 +107,7 @@ The final piece is the Docker Compose file. This ties all the services together
# compose.yml
services:
victoriametrics:
image: victoriametrics/victoria-metrics:v1.148.0
image: victoriametrics/victoria-metrics:v1.149.0
command:
- "--storageDataPath=/victoria-metrics-data"
- "--selfScrapeInterval=10s"
@@ -128,7 +128,7 @@ services:
- ./alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro
vmalert:
image: victoriametrics/vmalert:v1.148.0
image: victoriametrics/vmalert:v1.149.0
depends_on:
- victoriametrics
- alertmanager

View File

@@ -1,3 +1,11 @@
---
build:
list: never
publishResources: false
render: never
sitemap:
disable: true
---
VictoriaMetrics software provides native [OpenTelemetry](https://opentelemetry.io/) ingestion across **metrics**, **logs**, and **traces** via dedicated components.
This allows running OpenTelemetry-based observability pipeline with VictoriaMetrics software as your backend.
@@ -88,4 +96,4 @@ Depending on the Grafana datasource plugin there could be multiple correlations
1. Trace to metrics, metric to logs, metric to traces - see [correlations via VictoriaMetrics plugin](https://docs.victoriametrics.com/victoriametrics/integrations/grafana/datasource/#correlations).
1. Metrics to logs or traces correlations are possible via Prometheus datasource as well.
1. Plugins Tempo, Jaeger, and Zipkin can correlate with logs or metrics using [Trace to logs](https://grafana.com/docs/grafana/latest/explore/trace-integration/#trace-to-logs)
and [Trace to metrics](https://grafana.com/docs/grafana/latest/visualizations/explore/trace-integration/#trace-to-metrics) feature.
and [Trace to metrics](https://grafana.com/docs/grafana/latest/visualizations/explore/trace-integration/#trace-to-metrics) feature.

View File

@@ -1,3 +1,11 @@
---
build:
list: never
publishResources: false
render: never
sitemap:
disable: true
---
VictoriaMetrics offers public playgrounds where you can try the full observability stack online.
Some playgrounds are based on the [OpenTelemetry Astronomy Shop demo](https://github.com/open-telemetry/opentelemetry-demo), a sample microservices application that generates realistic metrics, logs, and traces. Other playgrounds use benchmark workloads such as [prometheus-benchmark](https://github.com/VictoriaMetrics/prometheus-benchmark) to demonstrate ingestion and query performance for Prometheus-compatible systems.
@@ -158,4 +166,4 @@ Iximiuz Labs provides various [learning-by-doing resources for VictoriaMetrics](
- [VictoriaMetrics cluster](https://labs.iximiuz.com/playgrounds/victoriametrics-cluster)
- [VictoriaMetrics on Kubernetes](https://labs.iximiuz.com/playgrounds/victoriametrics-kubernetes)
Iximiuz Labs requires a [free account](https://labs.iximiuz.com/signup) to access the materials.
Iximiuz Labs requires a [free account](https://labs.iximiuz.com/signup) to access the materials.

View File

@@ -156,13 +156,6 @@ See [our blog](https://victoriametrics.com/blog) for the latest articles written
* [Why irate from Prometheus doesn't capture spikes](https://valyala.medium.com/why-irate-from-prometheus-doesnt-capture-spikes-45f9896d7832)
* [VictoriaMetrics: PromQL compliance](https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e)
* [How do open source solutions for logs work: Elasticsearch, Loki and VictoriaLogs](https://itnext.io/how-do-open-source-solutions-for-logs-work-elasticsearch-loki-and-victorialogs-9f7097ecbc2f)
* [How vmagent Collects and Ships Metrics Fast with Aggregation, Deduplication, and More](https://victoriametrics.com/blog/vmagent-how-it-works/)
* [When Metrics Meet vminsert: A Data-Delivery Story](https://victoriametrics.com/blog/vminsert-how-it-works/)
* [How vmstorage Handles Data Ingestion From vminsert](https://victoriametrics.com/blog/vmstorage-how-it-handles-data-ingestion/)
* [How vmstorage Processes Data: Retention, Merging, Deduplication...](https://victoriametrics.com/blog/vmstorage-retention-merging-deduplication/)
* [How vmstorage's IndexDB Works](https://victoriametrics.com/blog/vmstorage-how-indexdb-works/)
* [How vmstorage Handles Query Requests From vmselect](https://victoriametrics.com/blog/vmstorage-how-it-handles-query-requests/)
* [Inside vmselect: The Query Processing Engine of VictoriaMetrics](https://victoriametrics.com/blog/vmselect-how-it-works/)
### Tutorials, guides and how-to articles
@@ -180,12 +173,6 @@ See [our guides](https://docs.victoriametrics.com/guides/) for the up-to-date gu
* [Prometheus storage: tech terms for humans](https://valyala.medium.com/prometheus-storage-technical-terms-for-humans-4ab4de6c3d48)
* [Cardinality explorer](https://victoriametrics.com/blog/cardinality-explorer/)
* [Rules backfilling via vmalert](https://victoriametrics.com/blog/rules-replay/)
* [vmagent: Key Features Explained in Under 15 Minutes](https://victoriametrics.com/blog/vmagent-key-features-explained/)
* [Prometheus Metrics Explained: Counters, Gauges, Histograms & Summaries](https://victoriametrics.com/blog/prometheus-monitoring-metrics-counters-gauges-histogram-summaries/)
* [Prometheus Monitoring: Instant Queries and Range Queries Explained](https://victoriametrics.com/blog/prometheus-monitoring-instant-range-query/)
* [Prometheus Monitoring: Functions, Subqueries, Operators, and Modifiers](https://victoriametrics.com/blog/prometheus-monitoring-function-operator-modifier/)
* [Prometheus Alerting 101: Rules, Recording Rules, and Alertmanager](https://victoriametrics.com/blog/alerting-recording-rules-alertmanager/)
* [Alerting Best Practices](https://victoriametrics.com/blog/alerting-best-practices/)
### Other articles

View File

@@ -59,11 +59,6 @@ It increases cluster availability, and simplifies cluster maintenance as well as
![Cluster Scheme](Cluster-VictoriaMetrics-components.webp)
> Further reading, deep dives into how each service works internally:
> - `vmstorage`: [How vmstorage Handles Data Ingestion From vminsert](https://victoriametrics.com/blog/vmstorage-how-it-handles-data-ingestion/), [How vmstorage's IndexDB Works](https://victoriametrics.com/blog/vmstorage-how-indexdb-works/), [How vmstorage Handles Query Requests From vmselect](https://victoriametrics.com/blog/vmstorage-how-it-handles-query-requests/).
> - `vminsert`: [When Metrics Meet vminsert: A Data-Delivery Story](https://victoriametrics.com/blog/vminsert-how-it-works/).
> - `vmselect`: [Inside vmselect: The Query Processing Engine of VictoriaMetrics](https://victoriametrics.com/blog/vmselect-how-it-works/).
## vmui
VictoriaMetrics cluster version provides UI for query troubleshooting and exploration. The UI is available at
@@ -296,7 +291,7 @@ If you need multi-AZ setup, then it is recommended running independent clusters
into all the cluster - see [these docs](https://docs.victoriametrics.com/victoriametrics/vmagent/#multitenancy) for details.
Then an additional `vmselect` nodes can be configured for reading the data from multiple clusters according to [these docs](#multi-level-cluster-setup).
See [victoria-metrics-distributed chart](https://docs.victoriametrics.com/helm/victoria-metrics-distributed/) for an example.
See [VMDistributed](https://docs.victoriametrics.com/operator/resources/vmdistributed/) Kubernetes operator resource for an example.
## Cluster setup
@@ -834,9 +829,9 @@ See also [minimum downtime strategy](#minimum-downtime-strategy).
## Slowness-based re-routing
By default{{% available_from "#" %}}, `vminsert` automatically [re-route writes](https://victoriametrics.com/blog/vminsert-how-it-works/#31-rerouting)
away from the slowest `vmstorage` node to preserve maximum ingestion throughput. This prevents a single slow `vmstorage`
node from throttling the entire cluster.
By default{{% available_from "v1.149.0" %}}, `vminsert` automatically re-routes writes away from the slowest `vmstorage` node
to preserve maximum ingestion throughput. This prevents a single slow `vmstorage` node
from throttling the entire cluster.
Re-routing occurs only when all of the following conditions hold:
- the storage send buffer is full.
@@ -848,7 +843,7 @@ Disable slowness-based re-routing with `-disableRerouting=true` when keeping met
perfectly balanced across nodes or minimizing the number of [active time series](https://docs.victoriametrics.com/victoriametrics/faq/#what-is-an-active-time-series)
matters more than peak write throughput.
Slowness-based re-routing is automatically disabled{{% available_from "#" %}} when `-replicationFactor` is greater than `1`,
Slowness-based re-routing is automatically disabled{{% available_from "v1.149.0" %}} when `-replicationFactor` is greater than `1`,
because rerouting does not guarantee that replicated copies land on distinct storage nodes,
which violates the replication contract.
@@ -883,7 +878,7 @@ See also [resource usage limits docs](#resource-usage-limits).
## Rebalancing
Every `vminsert` node [evenly spreads (shards) incoming data](https://victoriametrics.com/blog/vminsert-how-it-works/#3-sharding-and-buffering) among `vmstorage` nodes specified in the `-storageNode` command-line flag.
Every `vminsert` node evenly spreads (shards) incoming data among `vmstorage` nodes specified in the `-storageNode` command-line flag.
This guarantees even distribution of the ingested data among `vmstorage` nodes. When new `vmstorage` nodes are added to the `-storageNode`
command-line flag at `vminsert`, then only newly ingested data is distributed evenly among old and new `vmstorage` nodes, while
historical data remains on the old `vmstorage` nodes. This speeds up data ingestion and querying for the majority of production workloads,
@@ -1031,7 +1026,7 @@ By default, VictoriaMetrics offloads replication to the underlying storage point
which guarantees data durability. VictoriaMetrics supports application-level replication if replicated durable persistent disks cannot be used for some reason.
The replication can be enabled by passing `-replicationFactor=N` command-line flag to `vminsert`. This instructs `vminsert` to store `N` copies for every ingested sample
on `N` distinct `vmstorage` nodes. This guarantees that all the stored data remains available for querying if up to `N-1` `vmstorage` nodes are unavailable. See [how `vminsert` replicates each sample to `N` `vmstorage` nodes](https://victoriametrics.com/blog/vminsert-how-it-works/#4-replication-and-sending-data-to-vmstorage) for details.
on `N` distinct `vmstorage` nodes. This guarantees that all the stored data remains available for querying if up to `N-1` `vmstorage` nodes are unavailable.
Passing `-replicationFactor=N` command-line flag to `vmselect` instructs it to not mark responses as `partial` if less than `-replicationFactor` vmstorage nodes are unavailable during the query.
See [cluster availability docs](#cluster-availability) for details.
@@ -1066,7 +1061,7 @@ deduplication can't be guaranteed when samples and sample duplicates for the sam
- when `vmstorage` node has no enough capacity for processing incoming data stream. Then `vminsert` re-routes new samples to other `vmstorage` nodes.
It is recommended to set **the same** `-dedup.minScrapeInterval` command-line flag value to both `vmselect` and `vmstorage` nodes
to ensure query results consistency, even if [storage layer didn't complete deduplication](https://victoriametrics.com/blog/vmstorage-retention-merging-deduplication/#deduplication) yet.
to ensure query results consistency, even if storage layer didn't complete deduplication yet.
## Metrics Metadata

View File

@@ -27,5 +27,5 @@ to [the latest available releases](https://docs.victoriametrics.com/victoriametr
## Currently supported LTS release lines
- v1.148.x - the latest one is [v1.148.0 LTS release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.148.0)
- v1.136.x - the latest one is [v1.136.14 LTS release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.136.14)
- v1.148.x - the latest one is [v1.148.1 LTS release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.148.1)
- v1.136.x - the latest one is [v1.136.15 LTS release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.136.15)

View File

@@ -53,8 +53,8 @@ and unpack it. It contains a single `victoria-metrics-prod` binary.
For example, on Linux with `amd64` architecture:
```sh
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.148.0/victoria-metrics-linux-amd64-v1.148.0.tar.gz
tar xzf victoria-metrics-linux-amd64-v1.148.0.tar.gz
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.149.0/victoria-metrics-linux-amd64-v1.149.0.tar.gz
tar xzf victoria-metrics-linux-amd64-v1.149.0.tar.gz
```
The binary is self-contained and requires no installation - it is ready to run as is.
@@ -229,9 +229,9 @@ Download the newest available [VictoriaMetrics release](https://docs.victoriamet
from [DockerHub](https://hub.docker.com/r/victoriametrics/victoria-metrics) or [Quay](https://quay.io/repository/victoriametrics/victoria-metrics?tab=tags):
```sh
docker pull victoriametrics/victoria-metrics:v1.148.0
docker pull victoriametrics/victoria-metrics:v1.149.0
docker run -it --rm -v `pwd`/victoria-metrics-data:/victoria-metrics-data -p 8428:8428 \
victoriametrics/victoria-metrics:v1.148.0 --selfScrapeInterval=5s -storageDataPath=victoria-metrics-data
victoriametrics/victoria-metrics:v1.149.0 --selfScrapeInterval=5s -storageDataPath=victoria-metrics-data
```
_For Enterprise images, see [this link](https://docs.victoriametrics.com/victoriametrics/enterprise/#docker-images)._

View File

@@ -39,6 +39,8 @@ VictoriaMetrics has the following prominent features:
* Easy and fast backups from [instant snapshots](https://medium.com/@valyala/how-victoriametrics-makes-instant-snapshots-for-multi-terabyte-time-series-data-e1f3fb0e0282)
can be done with [vmbackup](https://docs.victoriametrics.com/victoriametrics/vmbackup/) / [vmrestore](https://docs.victoriametrics.com/victoriametrics/vmrestore/) tools.
See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time-series-databases-533c1a927883) for more details.
* It supports storage and retrieval of samples with timestamps that fall within the `[1970-01-02T00:00:00.000Z, 2262-03-31T23:59:59.999Z]` time range with millisecond precision.
See [Retention](#retention) for details.
* It implements a PromQL-like query language - [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/), which provides improved functionality on top of PromQL.
* It provides a global query view. Multiple Prometheus instances or any other data sources may ingest data into VictoriaMetrics. Later this data may be queried via a single query.
* It provides high performance and good vertical and horizontal scalability for both
@@ -1304,7 +1306,7 @@ since it uses lower amounts of RAM, CPU and network bandwidth than Prometheus.
If you use identically configured [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) instances for collecting the same data
and sending it to VictoriaMetrics, then do not forget enabling [deduplication](#deduplication) at VictoriaMetrics side.
See [victoria-metrics-distributed chart](https://docs.victoriametrics.com/helm/victoria-metrics-distributed/) for an example.
See [VMDistributed](https://docs.victoriametrics.com/operator/resources/vmdistributed/) Kubernetes operator resource for an example.
## Deduplication
@@ -1405,9 +1407,6 @@ for fast block lookups, which belong to the given `TSID` and cover the given tim
* various background maintenance tasks such as [de-duplication](#deduplication), [downsampling](#downsampling)
and [freeing up disk space for the deleted time series](#how-to-delete-time-series) are performed during the merge
See how `vmstorage` [selects parts for background merging](https://victoriametrics.com/blog/vmstorage-retention-merging-deduplication/#merge-process),
including merge limits and monitoring metrics.
Newly added `parts` either successfully appear in the storage or fail to appear.
The newly added `part` is atomically registered in the `parts.json` file under the corresponding partition
after it is fully written and [fsynced](https://man7.org/linux/man-pages/man2/fsync.2.html) to the storage.
@@ -1535,8 +1534,6 @@ are **eventually deleted** during [background merge](https://medium.com/@valyala
The time range covered by data part is **not limited by retention period unit**. One data part can cover hours or days of
data. Hence, a data part can be deleted only **when fully outside the configured retention**.
See more about partitions and parts in the [Storage section](#storage).
See how the [retention and free-disk watchers manage storage](https://victoriametrics.com/blog/vmstorage-retention-merging-deduplication/#retention-free-disk-space-guard-and-downsampling)
for implementation details and monitoring metrics.
The maximum disk space usage for a given `-retentionPeriod` is going to be (`-retentionPeriod` + 1) months.
For example, if `-retentionPeriod` is set to 1, data for January is deleted on March 1st.
@@ -1545,6 +1542,9 @@ It is safe to extend `-retentionPeriod` on existing data. If `-retentionPeriod`
value than before, then data outside the configured period will be eventually deleted.
VictoriaMetrics does not support indefinite retention, but you can specify an arbitrarily high duration, e.g. `-retentionPeriod=100y`.
Just keep in mind that VictoriaMetrics does not support samples with negative timestamps. Timestamps at `1970-01-01` are also not
supported because this date has a special meaning internally. It therefore rejects samples with timestamps before
`1970-01-02T00:00:00.000Z`.
By default, VictoriaMetrics doesn't accept samples with timestamps bigger than `now+2d`, e.g. 2 days in the future.
If you need accepting samples with bigger timestamps, then specify the desired "future retention" via `-futureRetention` command-line flag.
@@ -1556,6 +1556,9 @@ For example, the following command starts VictoriaMetrics, which accepts samples
/path/to/victoria-metrics -futureRetention=1y
```
VictoriaMetrics does not support stamples after `2262-03-31T23:59:59.999Z`. If the future retention includes dates after this timestamp,
the samples for those dates will be rejected.
By default, VictoriaMetrics accepts samples with timestamps as old as the configured `-retentionPeriod` allows, e.g. it accepts backfilled
historical data as long as it fits into the retention. If you need rejecting samples with historical timestamps older than the specified
duration, then specify the desired duration via the `-maxBackfillAge` command-line flag. This can be useful for limiting ingestion of

View File

@@ -26,22 +26,57 @@ See also [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-rel
## tip
* FEATURE: [vmui](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui): add an option to customize the favicon color. This makes it easier to distinguish between different installations opened in multiple browser tabs. See [#11329](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11329).
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/), `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/) and [vmctl](https://docs.victoriametrics.com/victoriametrics/vmctl/): properly parse small fractional Unix timestamps in timestamp args such as `start` and `end` in `/api/v1/query_range` and `--vm-native-filter-time-start` and `--vm-native-filter-time-end` in `vmctl`. Previously, fractional Unix timestamps with the integer part below `9223372` were interpreted with the wrong unit, for example `12.0` was parsed as `12000` seconds instead of `12` seconds. See [#11324](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11324).
* BUGFIX: [vmui](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui): respect the custom query step specified via `g0.step_input` when opening a URL. Previously, it could be reset to the automatically calculated step and potentially cause dashboards to freeze. See [#11137](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11137).
## [v1.149.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.149.0)
Released at 2026-08-05
**Update Note 1:** `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): the default value of `-disableRerouting` flag has changed from `true` to `false`, enabling [slowness-based re-routing](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#slowness-based-re-routing) by default. Slowness re-routing is automatically disabled when `-replicationFactor` is greater than 1. If you rely on the old behavior, pass `-disableRerouting` command-line flag to `vminsert`. See [#11287](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11287).
**Update Note 2:** [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): the `/api/v1/admin/tsdb/delete_series`, `/tags/delSeries` endpoints now require `POST` method. Previously, it also accepted `GET` requests. If you use `GET` requests for this endpoint, update your scripts or tooling to use `POST` instead. See [#5552](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5552).
* SECURITY: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): restrict `/api/v1/admin/tsdb/delete_series`, `/tags/delSeries` endpoints to `POST` method only to prevent some [SSRF](https://en.wikipedia.org/wiki/Server-side_request_forgery)-based data deletion attacks. See [#5552](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5552).
* FEATURE: [dashboards](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/dashboards): add `Fsync avg duration` panel to the Troubleshooting section of the single-node, cluster, and vmagent dashboards. This panel surfaces degradation of IO operation for faster incident triage. See [#10432](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10432).
* FEATURE: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): add `name` label identifying the corresponding `-remoteWrite.url` target to the `vm_persistentqueue_*` metrics exposed by persistent queue. See [#7944](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7944). Thanks to @tIGO for contribution.
* FEATURE: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): add `-remoteWrite.obfuscateLabels` flag for hashing values of the specified labels before sending metrics to the corresponding `-remoteWrite.url`. This allows sharing metrics with external systems while keeping sensitive label values hidden. See [#10599](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10599).
* FEATURE: [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): add `-replay.continueWithExecutionErr` flag to allow continuing to replay other rules when a rule execution fails with a 422 response code, which can happen due to an expression syntax error or a resource limit being hit. See [11313](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11313).
* FEATURE: [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): add template variable `$interval` to expose the alerting rule group's evaluation interval. This allows generating dashboard links with a lookback window relative to the rule's interval, for example `&from={{ ($activeAt.Add (parseDurationTime (printf "-%s" .Interval))).UnixMilli }}&to={{ $activeAt.UnixMilli }}`. See [#11232](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11232). Thanks to @1solomonwakhungu for contribution.
* FEATURE: [vmbackupmanager](https://docs.victoriametrics.com/victoriametrics/vmbackupmanager/) and [alerts](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/rules): introduce `vm_backup_last_success_at` metric to track the last successful backup by type. Add [alerting rules](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/rules/alerts-vmbackupmanager.yml) `NoLatestBackupWithinLastDay`, `NoHourlyBackupWithinLastDay`, `NoDailyBackupWithinLast3Days`, `NoWeeklyBackupWithinLast14Days` and `NoMonthlyBackupWithinLast62Days` to remind users about the missing backups. See [#11217](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11217).
* FEATURE: [vmctl](https://docs.victoriametrics.com/victoriametrics/vmctl/): support [Prometheus native histograms](https://prometheus.io/docs/specs/native_histograms/) migration in [remote read mode](https://docs.victoriametrics.com/victoriametrics/vmctl/remoteread/). Native histograms are converted into `_count`, `_sum` and `_bucket` series with `vmrange` labels in the same way as VictoriaMetrics [converts native histograms received via Prometheus remote write protocol](https://docs.victoriametrics.com/victoriametrics/integrations/prometheus/#native-histograms), except that for native histograms with custom buckets the original bucket bounds are preserved instead of being estimated with the exponential formula. Previously native histograms were silently ignored in `SAMPLES` mode, while in stream mode the migration failed with `EOF` error. See [#11292](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11292). Thanks to @liuxu623 for contribution.
* FEATURE: `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): enable [slowness-based re-routing](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#slowness-based-re-routing) by default. Previously, `-disableRerouting` defaulted to `true`, which limited ingestion throughput to the slowest `vmstorage` node. Now `-disableRerouting` defaults to `false`, so `vminsert` automatically routes data away from the slowest `vmstorage` node, improving overall ingestion performance. Slowness re-routing is automatically disabled when `-replicationFactor` is greater than 1. See [#11287](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11287).
* FEATURE: [vmui](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui): persist the selected auto-refresh interval in the URL. See [VictoriaLogs#1310](https://github.com/VictoriaMetrics/VictoriaLogs/issues/1310).
* FEATURE: [dashboards](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/dashboards): add `Fsync avg duration` panel to the Troubleshooting section of the single-node, cluster, and vmagent dashboards. This panel surfaces degradation of IO operation for faster incident triage. See [#10432](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10432).
* FEATURE: [vmbackupmanager](https://docs.victoriametrics.com/victoriametrics/vmbackupmanager/) and [alerts](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/rules): introduce `vm_backup_last_success_at` metric to track the last successful backup by type. Add [alerting rules](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/rules/alerts-vmbackupmanager.yml) `NoLatestBackupWithinLastDay`, `NoHourlyBackupWithinLastDay`, `NoDailyBackupWithinLast3Days`, `NoWeeklyBackupWithinLast14Days` and `NoMonthlyBackupWithinLast62Days` to remind users about the missing backups. See [#11217](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11217).
* FEATURE: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): add `-remoteWrite.obfuscateLabels` flag for hashing values of the specified labels before sending metrics to the corresponding `-remoteWrite.url`. This allows sharing metrics with external systems while keeping sensitive label values hidden. See [#10599](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10599).
* FEATURE: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) and [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): add client side least-loaded load-balancing with `DNS` discovery. See [#2388](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2388) and these [vmagent DNS URLs](https://docs.victoriametrics.com/victoriametrics/vmagent/#dns-urls), [vmalert DNS URLs](https://docs.victoriametrics.com/victoriametrics/vmalert/#dns-urls).
* BUGFIX: [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/): properly drop data points filtered out by an inner [comparison operation](https://prometheus.io/docs/prometheus/latest/querying/operators/#comparison-binary-operators) when its result is used on the right side of another comparison. Previously, queries like `foo != (bar > 100)` could return unexpected results because filtered-out data points are represented internally as `NaN`, and `value != NaN` evaluates to `true`. Comparisons against explicitly present `NaN` values keep the previous behavior. See [#10018](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10018). Thanks to @zasdaym for contribution.
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) and [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/): ignore HTTP proxy environment variables when scraping targets over Unix domain sockets. See [#11318](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11318). Thanks to @lwmacct for contribution.
* BUGFIX: [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): fixed the display of rule state badges on the `Groups` page in the web UI. See [#11160](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11160).
* BUGFIX: [vmbackupmanager](https://docs.victoriametrics.com/victoriametrics/vmbackupmanager/): previously, `vmbackupmanager` was crashing on startup when it failed to restore backup state from remote storage, causing a crash loop. Now it logs the error and continues running, retrying the state restore before each scheduled backup. Added `vm_backup_errors_total{type="restoreState"}` metric to track backup state restore failures. See [#11217](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11217).
* BUGFIX: [stream aggregation](https://docs.victoriametrics.com/victoriametrics/stream-aggregation/): fix incorrect [sum_samples_total](https://docs.victoriametrics.com/victoriametrics/stream-aggregation/configuration/#sum_samples_total) results when `enable_windows: true` is set. See [#11261](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11261). Thanks to @beyond-infra for contribution.
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/), `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/) and [vmctl](https://docs.victoriametrics.com/victoriametrics/vmctl/): accept scientific notation with sub-second precision (e.g. `1.784144612388E9`) for timestamp args such as `start` and `end` in `/api/v1/query_range` and `--vm-native-filter-time-start` and `--vm-native-filter-time-end` in `vmctl`. Previously, values with this pattern were rejected, which is incompatible with Prometheus. See [#11268](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11268). Thanks to @STiFLeR7 for contribution.
## [v1.148.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.148.1)
Released at 2026-07-31
**v1.148.x is a line of [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-releases/). It contains important up-to-date bugfixes for [VictoriaMetrics enterprise](https://docs.victoriametrics.com/victoriametrics/enterprise/).
All these fixes are also included in [the latest community release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest).
The v1.148.x line will be supported for at least 12 months since [v1.148.0](https://docs.victoriametrics.com/victoriametrics/changelog/#v11480) release**
**Update Note 1:** [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): the `/api/v1/admin/tsdb/delete_series`, `/tags/delSeries` endpoints now require `POST` method. Previously, it also accepted `GET` requests. If you use `GET` requests for this endpoint, update your scripts or tooling to use `POST` instead. See [#5552](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5552).
* SECURITY: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): restrict `/api/v1/admin/tsdb/delete_series`, `/tags/delSeries` endpoints to `POST` method only to prevent some [SSRF](https://en.wikipedia.org/wiki/Server-side_request_forgery)-based data deletion attacks. See [#5552](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5552).
* BUGFIX: [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/): properly drop data points filtered out by an inner [comparison operation](https://prometheus.io/docs/prometheus/latest/querying/operators/#comparison-binary-operators) when its result is used on the right side of another comparison. Previously, queries like `foo != (bar > 100)` could return unexpected results because filtered-out data points are represented internally as `NaN`, and `value != NaN` evaluates to `true`. Comparisons against explicitly present `NaN` values keep the previous behavior. See [#10018](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10018). Thanks to @zasdaym for contribution.
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) and [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/): ignore HTTP proxy environment variables when scraping targets over Unix domain sockets. See [#11318](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11318). Thanks to @lwmacct for contribution.
* BUGFIX: [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): fixed the display of rule state badges on the `Groups` page in the web UI. See [#11160](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11160).
* BUGFIX: [vmbackupmanager](https://docs.victoriametrics.com/victoriametrics/vmbackupmanager/): previously, `vmbackupmanager` was crashing on startup when it failed to restore backup state from remote storage, causing a crash loop. Now it logs the error and continues running, retrying the state restore before each scheduled backup. Added `vm_backup_errors_total{type="restoreState"}` metric to track backup state restore failures. See [#11217](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11217).
* BUGFIX: [stream aggregation](https://docs.victoriametrics.com/victoriametrics/stream-aggregation/): fix incorrect [sum_samples_total](https://docs.victoriametrics.com/victoriametrics/stream-aggregation/configuration/#sum_samples_total) results when `enable_windows: true` is set. See [#11261](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11261). Thanks to @beyond-infra for contribution.
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/), `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/) and [vmctl](https://docs.victoriametrics.com/victoriametrics/vmctl/): accept scientific notation with sub-second precision (e.g. `1.784144612388E9`) for timestamp args such as `start` and `end` in `/api/v1/query_range` and `--vm-native-filter-time-start` and `--vm-native-filter-time-end` in `vmctl`. Previously, values with this pattern were rejected, which is incompatible with Prometheus. See [#11268](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11268). Thanks to @STiFLeR7 for contribution.
## [v1.148.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.148.0)
@@ -53,7 +88,6 @@ Released at 2026-07-20
* FEATURE: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) and [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/): support scraping metrics over Unix domain sockets. The socket path can be configured via the `__unix_socket__` target label. See [#11156](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11156). Thanks to @vinyas-bharadwaj for contribution.
* FEATURE: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) and [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/): Improve background discovery performance for [http_sd](https://docs.victoriametrics.com/victoriametrics/sd_configs/#http_sd_configs) discovery. See [#8838](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8838).
* FEATURE: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) and [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/): allow overriding `max_scrape_size` on a per-target basis via the `__max_scrape_size__` label during target relabeling. See [#11188](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11188).
* FEATURE: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) and [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): add client side least-loaded load-balancing with `DNS` discovery. See [#2388](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2388) and these [vmagent DNS URLs](https://docs.victoriametrics.com/victoriametrics/vmagent/#dns-urls), [vmalert DNS URLs](https://docs.victoriametrics.com/victoriametrics/vmalert/#dns-urls).
* FEATURE: [vmstorage](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/) and [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/): add `-maxBackfillAge` command-line flag for limiting ingestion of samples with historical timestamps, for example, when older data has been moved between storage tiers (nvme/hdd, hot/cold). See [#11199](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11199). Thanks to @AshwinRamaniPsg for contribution.
* FEATURE: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): automatically preload relabeling rules configured via `-remoteWrite.relabelConfig` and `-remoteWrite.urlRelabelConfig` in the [metrics relabel debug UI](https://docs.victoriametrics.com/victoriametrics/relabeling/#relabel-debugging). See [#9918](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9918).
@@ -75,6 +109,8 @@ Released at 2026-07-06
**Update Note 1:** [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): contains a bug that causes increased CPU and memory usage when `-remoteWrite.urlRelabelConfig` or `-remoteWrite.streamAggr.config` flags are used. The bug was introduced in [#10854](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10854). Upgrade to v1.148.0 or rollback to v1.146.0. See [#11250](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11250).
**Update Note 2:** [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): vmalert updates VictoriaLogs LogsQL query parser to [v1.51.0](https://docs.victoriametrics.com/victorialogs/changelog/#v1510), which contains a breaking change in LogsQL filter pipes handling. If you used vmalert with `vlogs` query type and query expressions contained deprecated syntax - these rules will fail the validation on vmalert restart. Please review the [VictoriaLogs v1.51.0 changelog](https://docs.victoriametrics.com/victorialogs/changelog/#v1510) and update your alerting rules accordingly before upgrading.
* SECURITY: upgrade base docker image (Alpine) from 3.23.4 to 3.24.1. See [Alpine 3.24.1 release notes](https://www.alpinelinux.org/posts/Alpine-3.24.1-released.html).
* FEATURE: [vmauth](https://docs.victoriametrics.com/victoriametrics/vmauth/): add `default_vm_access_claim` field into `jwt` section of auth config. It could be used at [JWT claim placeholders](https://docs.victoriametrics.com/victoriametrics/vmauth/#jwt-claim-based-request-templating), if `JWT` token doesn't have `vm_access` claim. See [#11054](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11054).
@@ -365,6 +401,20 @@ It enables back `Discovered targets` debug UI by default.
* BUGFIX: `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): properly apply `extra_filters[]` filter when querying `vm_account_id` or `vm_project_id` labels via [multitenant](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy) request for `/api/v1/label/…/values` API. Before, `extra_filters` was ignored. See [#10503](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10503).
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): revert the use of rollup result cache for [instant queries](https://docs.victoriametrics.com/keyConcepts.html#instant-query) that contain [`rate`](https://docs.victoriametrics.com/MetricsQL.html#rate) function with a lookbehind window larger than `-search.minWindowForInstantRollupOptimization`. The cache usage was removed since [v1.132.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.132.0). See [#10098](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10098#issuecomment-3895011084) for more details.
## [v1.136.15](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.136.15)
Released at 2026-07-31
**v1.136.x is a line of [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-releases/). It contains important up-to-date bugfixes for [VictoriaMetrics enterprise](https://docs.victoriametrics.com/victoriametrics/enterprise/).
All these fixes are also included in [the latest community release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest).
The v1.136.x line will be supported for at least 12 months since [v1.136.0](https://docs.victoriametrics.com/victoriametrics/changelog/#v11360) release**
**Update Note 1:** [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): the `/api/v1/admin/tsdb/delete_series`, `/tags/delSeries` endpoints now require `POST` method. Previously, it also accepted `GET` requests. If you use `GET` requests for this endpoint, update your scripts or tooling to use `POST` instead. See [#5552](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5552).
* SECURITY: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): restrict `/api/v1/admin/tsdb/delete_series`, `/tags/delSeries` endpoints to `POST` method only to prevent some [SSRF](https://en.wikipedia.org/wiki/Server-side_request_forgery)-based data deletion attacks. See [#5552](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5552).
* BUGFIX: [vmbackupmanager](https://docs.victoriametrics.com/victoriametrics/vmbackupmanager/): previously, `vmbackupmanager` was crashing on startup when it failed to restore backup state from remote storage, causing a crash loop. Now it logs the error and continues running, retrying the state restore before each scheduled backup. Added `vm_backup_errors_total{type="restoreState"}` metric to track backup state restore failures. See [#11217](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11217).
## [v1.136.14](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.136.14)
Released at 2026-07-17
@@ -394,6 +444,8 @@ Released at 2026-07-03
All these fixes are also included in [the latest community release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest).
The v1.136.x line will be supported for at least 12 months since [v1.136.0](https://docs.victoriametrics.com/victoriametrics/changelog/#v11360) release**
**Update Note 1:** [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): vmalert updates VictoriaLogs LogsQL query parser to [v1.51.0](https://docs.victoriametrics.com/victorialogs/changelog/#v1510), which contains a breaking change in LogsQL filter pipes handling. If you used vmalert with `vlogs` query type and query expressions contained deprecated syntax - these rules will fail the validation on vmalert restart. Please review the [VictoriaLogs v1.51.0 changelog](https://docs.victoriametrics.com/victorialogs/changelog/#v1510) and update your alerting rules accordingly before upgrading.
* SECURITY: upgrade base docker image (Alpine) from 3.23.4 to 3.24.1. See [Alpine 3.24.1 release notes](https://www.alpinelinux.org/posts/Alpine-3.24.1-released.html).
* BUGFIX: `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/) and [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/): properly check values range for the limits configured with flags `-maxLabelsPerTimeseries`, `-maxLabelNameLen` and `-maxLabelValueLen`. It must be in range `1..65535`. See [#11128](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11128).
@@ -804,6 +856,8 @@ Released at 2026-07-03
All these fixes are also included in [the latest community release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest).
The v1.122.x line will be supported for at least 12 months since [v1.122.0](https://docs.victoriametrics.com/victoriametrics/changelog/#v11220) release**
**Update Note 1:** [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): vmalert updates VictoriaLogs LogsQL query parser to [v1.51.0](https://docs.victoriametrics.com/victorialogs/changelog/#v1510), which contains a breaking change in LogsQL filter pipes handling. If you used vmalert with `vlogs` query type and query expressions contained deprecated syntax - these rules will fail the validation on vmalert restart. Please review the [VictoriaLogs v1.51.0 changelog](https://docs.victoriametrics.com/victorialogs/changelog/#v1510) and update your alerting rules accordingly before upgrading.
* SECURITY: upgrade base docker image (Alpine) from 3.23.4 to 3.24.1. See [Alpine 3.24.1 release notes](https://www.alpinelinux.org/posts/Alpine-3.24.1-released.html).
* BUGFIX: `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/) and [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/): properly check values range for the limits configured with flags `-maxLabelsPerTimeseries`, `-maxLabelNameLen` and `-maxLabelValueLen`. It must be in range `1..65535`. See [#11128](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11128).

View File

@@ -121,7 +121,7 @@ It is allowed to run Enterprise components in [cases listed here](https://docs.v
Binary releases of Enterprise components are available at [the releases page for VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest),
[the releases page for VictoriaLogs](https://github.com/VictoriaMetrics/VictoriaLogs/releases/latest)
and [the releases page for VictoriaTraces](https://github.com/VictoriaMetrics/VictoriaTraces/releases/latest).
Enterprise binaries and packages have `enterprise` suffix in their names. For example, `victoria-metrics-linux-amd64-v1.148.0-enterprise.tar.gz`.
Enterprise binaries and packages have `enterprise` suffix in their names. For example, `victoria-metrics-linux-amd64-v1.149.0-enterprise.tar.gz`.
In order to run binary release of Enterprise component, please download the `*-enterprise.tar.gz` archive for your OS and architecture
from the corresponding releases page and unpack it. Then run the unpacked binary.
@@ -139,8 +139,8 @@ For example, the following command runs VictoriaMetrics Enterprise binary with t
obtained at [this page](https://victoriametrics.com/products/enterprise/trial/):
```sh
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.148.0/victoria-metrics-linux-amd64-v1.148.0-enterprise.tar.gz
tar -xzf victoria-metrics-linux-amd64-v1.148.0-enterprise.tar.gz
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.149.0/victoria-metrics-linux-amd64-v1.149.0-enterprise.tar.gz
tar -xzf victoria-metrics-linux-amd64-v1.149.0-enterprise.tar.gz
./victoria-metrics-prod -license=BASE64_ENCODED_LICENSE_KEY
```
@@ -155,7 +155,7 @@ Alternatively, VictoriaMetrics Enterprise license can be stored in the file and
It is allowed to run Enterprise components in [cases listed here](https://docs.victoriametrics.com/victoriametrics/enterprise/#valid-cases-for-victoriametrics-enterprise).
Docker images for Enterprise components are available at [VictoriaMetrics Docker Hub](https://hub.docker.com/u/victoriametrics) and [VictoriaMetrics Quay](https://quay.io/organization/victoriametrics).
Enterprise docker images have `enterprise` suffix in their names. For example, `victoriametrics/victoria-metrics:v1.148.0-enterprise`.
Enterprise docker images have `enterprise` suffix in their names. For example, `victoriametrics/victoria-metrics:v1.149.0-enterprise`.
In order to run Docker image of VictoriaMetrics Enterprise component, it is required to provide the license key via the command-line
flag as described in the [binary-releases](https://docs.victoriametrics.com/victoriametrics/enterprise/#binary-releases) section.
@@ -165,13 +165,13 @@ Enterprise license key can be obtained at [this page](https://victoriametrics.co
For example, the following command runs VictoriaMetrics Enterprise Docker image with the specified license key:
```sh
docker run --name=victoria-metrics victoriametrics/victoria-metrics:v1.148.0-enterprise -license=BASE64_ENCODED_LICENSE_KEY
docker run --name=victoria-metrics victoriametrics/victoria-metrics:v1.149.0-enterprise -license=BASE64_ENCODED_LICENSE_KEY
```
Alternatively, the license code can be stored in the file and then referred via `-licenseFile` command-line flag:
```sh
docker run --name=victoria-metrics -v /vm-license:/vm-license victoriametrics/victoria-metrics:v1.148.0-enterprise -licenseFile=/path/to/vm-license
docker run --name=victoria-metrics -v /vm-license:/vm-license victoriametrics/victoria-metrics:v1.149.0-enterprise -licenseFile=/path/to/vm-license
```
Example docker-compose configuration:
@@ -181,7 +181,7 @@ version: "3.5"
services:
victoriametrics:
container_name: victoriametrics
image: victoriametrics/victoria-metrics:v1.148.0
image: victoriametrics/victoria-metrics:v1.149.0
ports:
- 8428:8428
volumes:
@@ -213,7 +213,7 @@ is used to provide the license key in plain-text:
```yaml
server:
image:
tag: v1.148.0-enterprise
tag: v1.149.0-enterprise
license:
key: {BASE64_ENCODED_LICENSE_KEY}
@@ -224,7 +224,7 @@ In order to provide the license key via existing secret, the following values fi
```yaml
server:
image:
tag: v1.148.0-enterprise
tag: v1.149.0-enterprise
license:
secret:
@@ -274,7 +274,7 @@ spec:
license:
key: {BASE64_ENCODED_LICENSE_KEY}
image:
tag: v1.148.0-enterprise
tag: v1.149.0-enterprise
```
In order to provide the license key via an existing secret, the following custom resource is used:
@@ -291,7 +291,7 @@ spec:
name: vm-license
key: license
image:
tag: v1.148.0-enterprise
tag: v1.149.0-enterprise
```
Example secret with license key:
@@ -342,7 +342,7 @@ Builds are available for amd64 and arm64 architectures.
Example archive:
`victoria-metrics-linux-amd64-v1.148.0-enterprise.tar.gz`
`victoria-metrics-linux-amd64-v1.149.0-enterprise.tar.gz`
Includes:
@@ -351,7 +351,7 @@ Includes:
Example Docker image:
`victoriametrics/victoria-metrics:v1.148.0-enterprise-fips` uses the FIPS-compatible binary and based on `scratch` image.
`victoriametrics/victoria-metrics:v1.149.0-enterprise-fips` uses the FIPS-compatible binary and based on `scratch` image.
## What Happens to Licensed Components When a License Expires

View File

@@ -6,15 +6,16 @@ build:
sitemap:
disable: true
---
## Data model
### What is a metric
Simply put, `metric` is a numeric measure or observation of something.
The most common use-cases for metrics are:
The most common use cases for metrics are:
- check how the system behaves at the particular time period;
- check how the system behaves at a particular time period;
- correlate behavior changes to other measurements;
- observe or forecast trends;
- trigger events (alerts) if the metric exceeds a threshold.
@@ -25,7 +26,7 @@ Let's start with an example. To track how many requests our application serves,
name `requests_total`.
You can be more specific here by saying `requests_success_total` (for only successful requests)
or `request_errors_total` (for requests which failed). Choosing a metric name is very important and supposed to clarify
or `request_errors_total` (for requests which failed). Choosing a metric name is very important and is supposed to clarify
what is actually measured to every person who reads it, just like **variable names** in programming.
#### Labels
@@ -54,14 +55,14 @@ requests_total{path="/", code="200"}
Labels can be automatically attached to the [time series](#time-series)
written via [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/#adding-labels-to-metrics)
or [Prometheus](https://docs.victoriametrics.com/victoriametrics/integrations/prometheus/).
VictoriaMetrics supports enforcing of label filters for [query API](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-querying-api-enhancements)
VictoriaMetrics supports enforcing label filters for the [query API](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-querying-api-enhancements)
to emulate data isolation. However, the real data isolation can be achieved via [multi-tenancy](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy).
#### Time series
A combination of a metric name and its labels defines a `time series`. For example,
`requests_total{path="/", code="200"}` and `requests_total{path="/", code="403"}`
are two different time series because they have different values for `code` label.
are two different time series because they have different values for the `code` label.
The number of unique time series has an impact on database resource usage.
See [what is an active time series](https://docs.victoriametrics.com/victoriametrics/faq/#what-is-an-active-time-series) and
@@ -69,8 +70,8 @@ See [what is an active time series](https://docs.victoriametrics.com/victoriamet
#### Cardinality
The number of unique [time series](#time-series) is named `cardinality`. Too big number of unique time series is named `high cardinality`.
High cardinality may result in increased resource usage at VictoriaMetrics.
The number of unique [time series](#time-series) is named `cardinality`. Having too many unique time series is named `high cardinality`.
High cardinality may result in increased resource usage in VictoriaMetrics.
See [these docs](https://docs.victoriametrics.com/victoriametrics/faq/#what-is-high-cardinality) for more details.
#### Raw samples
@@ -108,13 +109,13 @@ of the [time series](https://docs.victoriametrics.com/victoriametrics/keyconcept
| requests_total{path="/health", code="200"} | 4 | 1676297730 |
....
```
Here we have a time series `requests_total{path="/health", code="200"}` which has a value update each `30s`.
This means, its resolution is also a `30s`.
Here we have a time series `requests_total{path="/health", code="200"}` which has a value updated every `30s`.
This means its resolution is also `30s`.
> In terms of [pull model](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#pull-model), resolution is equal
> to `scrape_interval` and is controlled by the monitoring system (server).
> For [push model](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#push-model), resolution is an interval between
> samples timestamps and is controlled by a client (metrics collector).
> sample timestamps and is controlled by a client (metrics collector).
Try to keep time series resolution consistent, since some [MetricsQL](#metricsql) functions may expect it to be so.
@@ -126,10 +127,10 @@ type exists specifically to help users to understand how the metric was measured
#### Counter
Counter is a metric, which counts some events. Its value increases or stays the same over time.
It cannot decrease in general case. The only exception is e.g. `counter reset`,
A counter is a metric that counts an event. Its value increases or stays the same over time.
It cannot decrease in the general case. The only exception is, e.g., `counter reset`,
when the metric resets to zero. The `counter reset` can occur when the service, which exposes the counter, restarts.
So, the `counter` metric shows the number of observed events since the service start.
So, the `counter` metric shows the number of observed events since the service started.
In programming, `counter` is a variable that you **increment** each time something happens.
@@ -139,7 +140,7 @@ In programming, `counter` is a variable that you **increment** each time somethi
above is that time series `vm_http_requests_total{instance="localhost:8428", job="victoriametrics", path="api/v1/query_range"}`
was rapidly changing from 1:38 pm to 1:39 pm, then there were no changes until 1:41 pm.
Counter is used for measuring the number of events, like the number of requests, errors, logs, messages, etc.
A counter is used for measuring the number of events, like the number of requests, errors, logs, messages, etc.
The most common [MetricsQL](#metricsql) functions used with counters are:
* [rate](https://docs.victoriametrics.com/victoriametrics/metricsql/#rate) - calculates the average per-second speed of metric change.
@@ -148,7 +149,7 @@ The most common [MetricsQL](#metricsql) functions used with counters are:
time period specified in square brackets.
For example, `increase(requests_total[1h])` shows the number of requests served over the last hour.
It is OK to have fractional counters. For example, `request_duration_seconds_sum` counter may sum the durations of all the requests.
It is OK to have fractional counters. For example, the `request_duration_seconds_sum` counter may sum the durations of all the requests.
Every duration may have a fractional value in seconds, e.g. `0.5` of a second. So the cumulative sum of all the request durations
may be fractional too.
@@ -162,12 +163,12 @@ Gauge is used for measuring a value that can go up and down:
![gauge](gauge.webp)
The metric `process_resident_memory_anon_bytes` on the graph shows the memory usage of the application at every given time.
It is changing frequently, going up and down showing how the process allocates and frees the memory.
It is changing frequently, going up and down, showing how the process allocates and frees the memory.
In programming, `gauge` is a variable to which you **set** a specific value as it changes.
Gauge is used in the following scenarios:
* measuring temperature, memory usage, disk usage etc;
* measuring temperature, memory usage, disk usage, etc;
* storing the state of some process. For example, gauge `config_reloaded_successful` can be set to `1` if everything is
good, and to `0` if configuration failed to reload;
* storing the timestamp when the event happened. For example, `config_last_reload_success_timestamp_seconds`
@@ -178,11 +179,11 @@ and [rollup functions](https://docs.victoriametrics.com/victoriametrics/metricsq
#### Histogram
Histogram is a set of [counter](#counter) metrics with different `vmrange` or `le` labels.
A histogram is a set of [counter](#counter) metrics with different `vmrange` or `le` labels.
The `vmrange` or `le` labels define measurement boundaries of a particular bucket.
When the observed measurement hits a particular bucket, then the corresponding counter is incremented.
Histogram buckets usually have `_bucket` suffix in their names.
Histogram buckets usually have a `_bucket` suffix in their names.
For example, VictoriaMetrics tracks the distribution of rows processed per query with the `vm_rows_read_per_query` histogram.
The exposition format for this histogram has the following form:
@@ -200,10 +201,10 @@ The `vm_rows_read_per_query_bucket{vmrange="4.084e+02...4.642e+02"} 2` line mean
that there were 2 queries with the number of rows in the range `(408.4 - 464.2]`
since the last VictoriaMetrics start.
The counters ending with `_bucket` suffix allow estimating arbitrary percentile
The counters ending with the `_bucket` suffix allow estimating arbitrary percentiles
for the observed measurement with the help of [histogram_quantile](https://docs.victoriametrics.com/victoriametrics/metricsql/#histogram_quantile)
function. For example, the following query returns the estimated 99th percentile
on the number of rows read per each query during the last hour (see `1h` in square brackets):
on the number of rows read per query during the last hour (see `1h` in square brackets):
```metricsql
histogram_quantile(0.99, sum(increase(vm_rows_read_per_query_bucket[1h])) by (vmrange))
@@ -215,15 +216,15 @@ This query works in the following way:
number of events over the last hour.
1. The `sum(...) by (vmrange)` calculates per-bucket events by summing per-instance buckets
with the same `vmrange` values.
1. The `histogram_quantile(0.99, ...)` calculates 99th percentile over `vmrange` buckets returned at step 2.
1. The `histogram_quantile(0.99, ...)` calculates the 99th percentile over `vmrange` buckets returned at step 2.
Histogram metric type exposes two additional counters ending with `_sum` and `_count` suffixes:
- the `vm_rows_read_per_query_sum` is a sum of all the observed measurements,
e.g. the sum of rows served by all the queries since the last VictoriaMetrics start.
e.g., the sum of rows served by all the queries since the last VictoriaMetrics start.
- the `vm_rows_read_per_query_count` is the total number of observed events,
e.g. the total number of observed queries since the last VictoriaMetrics start.
e.g., the total number of observed queries since the last VictoriaMetrics start.
These counters allow calculating the average measurement value on a particular lookbehind window.
For example, the following query calculates the average number of rows read per query
@@ -233,7 +234,7 @@ during the last 5 minutes (see `5m` in square brackets):
increase(vm_rows_read_per_query_sum[5m]) / increase(vm_rows_read_per_query_count[5m])
```
The `vm_rows_read_per_query` histogram may be used in Go application in the following way
The `vm_rows_read_per_query` histogram may be used in a Go application in the following way
by using the [github.com/VictoriaMetrics/metrics](https://github.com/VictoriaMetrics/metrics) package:
```go
@@ -246,7 +247,7 @@ for _, query := range queries {
}
```
Now let's see what happens each time when `rowsReadPerQuery.Update` is called:
Now let's see what happens each time `rowsReadPerQuery.Update` is called:
* counter `vm_rows_read_per_query_sum` is incremented by value of `len(query.Rows)` expression;
* counter `vm_rows_read_per_query_count` increments by 1;
@@ -262,7 +263,7 @@ and calculating [quantiles](https://prometheus.io/docs/practices/histograms/#qua
Grafana doesn't understand buckets with `vmrange` labels, so the [prometheus_buckets](https://docs.victoriametrics.com/victoriametrics/metricsql/#prometheus_buckets)
function must be used for converting buckets with `vmrange` labels to buckets with `le` labels before building heatmaps in Grafana.
Histograms are usually used for measuring the distribution of latency, sizes of elements (batch size, for example) etc. There are two
Histograms are usually used for measuring the distribution of latency, sizes of elements (batch size, for example), etc. There are two
implementations of a histogram supported by VictoriaMetrics:
1. [Prometheus histogram](https://prometheus.io/docs/practices/histograms/). The canonical histogram implementation is
@@ -271,7 +272,7 @@ implementations of a histogram supported by VictoriaMetrics:
histogram requires a user to define ranges (`buckets`) statically.
1. [VictoriaMetrics histogram](https://valyala.medium.com/improving-histogram-usability-for-prometheus-and-grafana-bc7e5df0e350)
supported by [VictoriaMetrics/metrics](https://github.com/VictoriaMetrics/metrics) instrumentation library.
Victoriametrics histogram automatically handles bucket boundaries, so users don't need to think about them.
VictoriaMetrics histogram automatically handles bucket boundaries, so users don't need to think about them.
We recommend reading the following articles before you start using histograms:
@@ -303,7 +304,7 @@ The visualization of summaries is pretty straightforward:
Such an approach makes summaries easier to use but also puts significant limitations compared to [histograms](#histogram):
- It is impossible to calculate quantile over multiple summary metrics, e.g. `sum(go_gc_duration_seconds{quantile="0.75"})`,
- It is impossible to calculate a quantile over multiple summary metrics, e.g. `sum(go_gc_duration_seconds{quantile="0.75"})`,
`avg(go_gc_duration_seconds{quantile="0.75"})` or `max(go_gc_duration_seconds{quantile="0.75"})`
won't return the expected 75th percentile over `go_gc_duration_seconds` metrics collected from multiple instances
of the application. See [this article](https://latencytipoftheday.blogspot.de/2014/06/latencytipoftheday-you-cant-average.html) for details.
@@ -313,16 +314,16 @@ Such an approach makes summaries easier to use but also puts significant limitat
- It is impossible to calculate quantiles for measurements collected over an arbitrary time range. Usually, `summary`
quantiles are calculated over a fixed time range such as the last 5 minutes.
Summaries are usually used for tracking the pre-defined percentiles for latency, sizes of elements (batch size, for example) etc.
Summaries are usually used for tracking the pre-defined percentiles for latency, sizes of elements (batch size, for example), etc.
### Instrumenting application with metrics
As was said at the beginning of the [types of metrics](#types-of-metrics) section, metric type defines how it was
measured. VictoriaMetrics TSDB doesn't know about metric types. All it sees are metric names, labels, values, and timestamps.
What are these metrics, what do they measure, and how - all this depends on the application which emits them.
What these metrics are, what they measure, and how - all these depend on the application which emits them.
To instrument your application with metrics compatible with VictoriaMetrics we recommend
using [github.com/VictoriaMetrics/metrics](https://github.com/VictoriaMetrics/metrics) package.
To instrument your application with metrics compatible with VictoriaMetrics, we recommend
using the [github.com/VictoriaMetrics/metrics](https://github.com/VictoriaMetrics/metrics) package.
See more details on how to use it in [this article](https://victoriametrics.medium.com/how-to-monitor-go-applications-with-victoriametrics-c04703110870).
VictoriaMetrics is also compatible with [Prometheus client libraries for metrics instrumentation](https://prometheus.io/docs/instrumenting/clientlibs/).
@@ -331,20 +332,20 @@ VictoriaMetrics is also compatible with [Prometheus client libraries for metrics
We recommend following [Prometheus naming convention for metrics](https://prometheus.io/docs/practices/naming/). There
are no strict restrictions, so any metric name and labels are accepted by VictoriaMetrics.
But the convention helps to keep names meaningful, descriptive, and clear to other people.
Following convention is a good practice.
But this convention helps to keep names meaningful, descriptive, and clear to other people.
Following the convention is a good practice.
#### Labels
Every measurement can contain an arbitrary number of `key="value"` labels. The good practice is to keep this number limited.
Otherwise, it would be difficult to deal with measurements containing a big number of labels.
Otherwise, it would be difficult to deal with measurements containing a large number of labels.
By default, VictoriaMetrics limits the number of labels per measurement to `40` and drops other labels.
This limit can be changed via `-maxLabelsPerTimeseries` command-line flag if necessary (but this isn't recommended).
This limit can be changed via the `-maxLabelsPerTimeseries` command-line flag if necessary (but this isn't recommended).
Every label value can contain an arbitrary string value. The good practice is to use short and meaningful label values to
describe the attribute of the metric, not to tell the story about it. For example, label-value pair
`environment="prod"` is ok, but `log_message="long log message with a lot of details..."` is not ok. By default,
VictoriaMetrics limits label's value size with 4KiB. This limit can be changed via `-maxLabelValueLen` command-line flag.
`environment="prod"` is OK, but `log_message="long log message with a lot of details..."` is not OK. By default,
VictoriaMetrics limits label values to 4KiB. This limit can be changed via the `-maxLabelValueLen` command-line flag.
It is very important to keep under control the number of unique label values, since every unique label value
leads to a new [time series](#time-series). Try to avoid using volatile label values such as session ID or query ID in order to
@@ -356,7 +357,7 @@ avoid excessive resource usage and database slowdown.
supports [multi-tenancy](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy)
for data isolation.
Multi-tenancy can be emulated for [single-server](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/)
Multi-tenancy can be emulated for the [single-server](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/)
version of VictoriaMetrics by adding [labels](#labels) on [write path](#write-data)
and enforcing [labels filtering](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-querying-api-enhancements)
on [read path](#query-data).
@@ -391,10 +392,10 @@ It is allowed to push/write metrics to [single-node VictoriaMetrics](https://doc
to [cluster component vminsert](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#architecture-overview)
and to [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/).
The pros of push model:
The pros of the push model:
* Simpler configuration at VictoriaMetrics side - there is no need to configure VictoriaMetrics with locations of the monitored applications.
There is no need in complex [service discovery schemes](https://docs.victoriametrics.com/victoriametrics/sd_configs/).
There is no need for complex [service discovery schemes](https://docs.victoriametrics.com/victoriametrics/sd_configs/).
* Simpler security setup - there is no need to set up access from VictoriaMetrics to each monitored application.
See [Foiled by the Firewall: A Tale of Transition From Prometheus to VictoriaMetrics](https://www.percona.com/blog/2020/12/01/foiled-by-the-firewall-a-tale-of-transition-from-prometheus-to-victoriametrics/)
@@ -406,18 +407,18 @@ The cons of push protocol:
Every application needs to be individually configured with the address of the monitoring system
for metrics delivery. It also needs to be configured with the interval between metric pushes
and the strategy in case of metric delivery failure.
* Non-trivial setup for metrics' delivery into multiple monitoring systems.
* Non-trivial setup for metrics delivery into multiple monitoring systems.
* It may be hard to tell whether the application went down or just stopped sending metrics for a different reason.
* Applications can overload the monitoring system by pushing metrics at too short intervals.
### Pull model
Pull model is an approach popularized by [Prometheus](https://prometheus.io/), where the monitoring system decides when
The pull model is an approach popularized by [Prometheus](https://prometheus.io/), where the monitoring system decides when
and where to pull metrics from:
![pull model](pull_model.webp)
In pull model, the monitoring system needs to be aware of all the applications it needs to monitor. The metrics are
In the pull model, the monitoring system needs to be aware of all the applications it needs to monitor. The metrics are
scraped (pulled) from the known applications (aka `scrape targets`) via HTTP protocol on a regular basis (aka `scrape_interval`).
VictoriaMetrics supports discovering Prometheus-compatible targets and scraping metrics from them in the same way as Prometheus does -
@@ -431,7 +432,7 @@ The pros of the pull model:
* Easier to debug - VictoriaMetrics knows about all the monitored applications (aka `scrape targets`).
The `up == 0` query instantly shows unavailable scrape targets.
The actual information about scrape targets is available at `http://victoriametrics:8428/targets` and `http://vmagent:8429/targets`.
* Monitoring system controls the frequency of metrics' scrape, so it is easier to control its load.
* The monitoring system controls the frequency of metrics scraping, so it is easier to control its load.
* Applications aren't aware of the monitoring system and don't need to implement the logic for metrics delivery.
The cons of the pull model:
@@ -448,13 +449,13 @@ The most common approach for data collection is using both models:
![data collection](data_collection.webp)
In this approach the additional component is used - [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/). Vmagent is
a lightweight agent whose main purpose is to collect, filter, relabel and deliver metrics to VictoriaMetrics.
In this approach, the additional component is used - [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/). Vmagent is
a lightweight agent whose main purpose is to collect, filter, relabel, and deliver metrics to VictoriaMetrics.
It supports all [push](#push-model) and [pull](#pull-model) protocols mentioned above.
The basic monitoring setup of VictoriaMetrics and vmagent is described
in the [example docker-compose manifest](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker#readme).
In this example vmagent [scrapes a list of targets](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/prometheus-vm-single.yml)
In this example, vmagent [scrapes a list of targets](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/prometheus-vm-single.yml)
and [forwards collected data to VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/9751ea10983d42068487624849cac7ad6fd7e1d8/deployment/docker/compose-vm-single.yml#L16).
VictoriaMetrics is then used as a [datasource for Grafana](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/provisioning/datasources/prometheus/single.yml)
installation for querying collected data.
@@ -480,7 +481,7 @@ The API consists of two main handlers for serving [instant queries](#instant-que
### Instant query
Instant query executes the `query` expression at the given `time`:
An instant query executes the `query` expression at the given `time`:
```
GET | POST /api/v1/query?query=...&time=...&step=...&timeout=...
@@ -497,13 +498,13 @@ Params:
For example, the request `/api/v1/query?query=up&step=1m` looks for the last written raw sample for the metric `up`
in the `(now()-1m, now()]` interval (the first millisecond is not included). If omitted, `step` is set to `5m` (5 minutes)
by default.
* `timeout` - optional query timeout. For example, `timeout=5s`. Query is canceled when the timeout is reached.
By default the timeout is set to the value of `-search.maxQueryDuration` command-line flag passed to single-node VictoriaMetrics
or to `vmselect` component of VictoriaMetrics cluster.
* `timeout` - optional query timeout. For example, `timeout=5s`. The query is canceled when the timeout is reached.
By default, the timeout is set to the value of the `-search.maxQueryDuration` command-line flag passed to the single-node VictoriaMetrics
or to the `vmselect` component of the VictoriaMetrics cluster.
The result of Instant query is a list of [time series](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#time-series)
matching the filter in `query` expression. Each returned series contains exactly one `(timestamp, value)` entry,
where `timestamp` equals to the `time` query arg, while the `value` contains `query` result at the requested `time`.
The result of an Instant query is a list of [time series](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#time-series)
matching the filter in the `query` expression. Each returned series contains exactly one `(timestamp, value)` entry,
where `timestamp` equals the `time` query arg, while the `value` contains the `query` result at the requested `time`.
To understand how instant queries work, let's begin with a data sample:
@@ -530,7 +531,7 @@ ranging from 1m to 3m. If we plot this data sample on the graph, it will have th
{width="500"}
To get the value of the `foo_bar` series at some specific moment of time, for example `2022-05-10T08:03:00Z`, in
VictoriaMetrics we need to issue an **instant query**:
VictoriaMetrics, we need to issue an **instant query**:
```sh
curl "http://<victoria-metrics-addr>/api/v1/query?query=foo_bar&time=2022-05-10T08:03:00.000Z"
@@ -595,13 +596,13 @@ Params:
The `query` is executed at `start`, `start+step`, `start+2*step`, ..., `start+N*step` timestamps,
where `N` is the whole number of steps that fit between `start` and `end`.
`end` is included only when it equals to `start+N*step`.
If the `step` isn't set, then it default to `5m` (5 minutes).
* `timeout` - optional query timeout. For example, `timeout=5s`. Query is canceled when the timeout is reached.
By default the timeout is set to the value of `-search.maxQueryDuration` command-line flag passed to single-node VictoriaMetrics
or to `vmselect` component in VictoriaMetrics cluster.
If the `step` isn't set, then it defaults to `5m` (5 minutes).
* `timeout` - optional query timeout. For example, `timeout=5s`. The query is canceled when the timeout is reached.
By default, the timeout is set to the value of the `-search.maxQueryDuration` command-line flag passed to the single-node VictoriaMetrics
or to the `vmselect` component in a VictoriaMetrics cluster.
The result of Range query is a list of [time series](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#time-series)
matching the filter in `query` expression. Each returned series contains `(timestamp, value)` results for the `query` executed
The result of a Range query is a list of [time series](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#time-series)
matching the filter in the `query` expression. Each returned series contains `(timestamp, value)` results for the `query` executed
at `start`, `start+step`, `start+2*step`, ..., `start+N*step` timestamps. In other words, Range query is an [Instant query](#instant-query)
executed independently at `start`, `start+step`, ..., `start+N*step` timestamps with the only difference that an instant query
does not return `ephemeral` samples (see below). Instead, if the database does not contain any samples for the requested time and step,
@@ -705,7 +706,7 @@ In response, VictoriaMetrics returns `17` sample-timestamp pairs for the series
from `2022-05-10T07:59:00Z` to `2022-05-10T08:17:00Z`. But, if we take a look at the original data sample again, we'll
see that it contains only 13 raw samples. What happens here is that the range query is actually
an [instant query](#instant-query) executed `1 + (start-end)/step` times on the time range from `start` to `end`. If we plot
this request in VictoriaMetrics the graph will be shown as the following:
this request in VictoriaMetrics, the graph will be shown as follows:
![range query](range_query.webp)
{width="500"}
@@ -720,13 +721,13 @@ This behavior of adding ephemeral data points comes from the specifics of the [p
* Scrape may be skipped if the monitoring system is overloaded.
* Scrape may fail due to network issues.
According to these specifics, the range query assumes that if there is a missing raw sample then it is likely a missed
According to these specifics, the range query assumes that if there is a missing raw sample, then it is likely a missed
scrape, so it fills it with the previous raw sample. The same will work for cases when `step` is lower than the actual
interval between samples. In fact, if we set `step=1s` for the same request, we'll get about 1 thousand data points in
response, where most of them are `ephemeral`.
Sometimes, the lookbehind window for locating the datapoint isn't big enough and the graph will contain a gap. For range
queries, lookbehind window isn't equal to the `step` parameter. It is calculated as the median of the intervals between
queries, the lookbehind window isn't equal to the `step` parameter. It is calculated as the median of the intervals between
the last 20 raw samples in the requested time range. In this way, VictoriaMetrics automatically adjusts the lookbehind
window to fill gaps and detect stale series at the same time.
@@ -734,7 +735,7 @@ Range queries are mostly used for plotting time series data over specified time
useful in the following scenarios:
* Track the state of a metric on the given time interval;
* Correlate changes between multiple metrics on the time interval;
* Correlate changes between multiple metrics over the time interval;
* Observe trends and dynamics of the metric change.
If you need to export raw samples from VictoriaMetrics, then take a look at [export APIs](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-export-time-series).
@@ -745,7 +746,7 @@ By default, Victoria Metrics does not immediately return the recently written sa
written prior to the time specified by the `-search.latencyOffset` command-line flag, which has a default offset of 30 seconds.
This is true for both `query` and `query_range` and may give the impression that data is written to the VM with a 30-second delay.
This flag prevents from non-consistent results due to the fact that only part of the values are scraped in the last scrape interval.
This flag prevents inconsistent results due to the fact that only part of the values are scraped in the last scrape interval.
Here is an illustration of a potential problem when `-search.latencyOffset` is set to zero:
@@ -758,12 +759,12 @@ duration throughout the `-search.latencyOffset` duration:
![with latency offset](with_latencyOffset.webp)
{width="1000"}
It can be overridden on per-query basis via `latency_offset` query arg.
It can be overridden on a per-query basis via the `latency_offset` query arg.
VictoriaMetrics buffers recently ingested samples in memory for up to a few seconds and then periodically flushes these samples to disk.
This buffering improves data ingestion performance. The buffered samples are invisible in query results, even if `-search.latencyOffset` command-line flag is set to 0,
This buffering improves data ingestion performance. The buffered samples are invisible in query results, even if the `-search.latencyOffset` command-line flag is set to 0,
or if `latency_offset` query arg is set to 0.
You can send GET request to `/internal/force_flush` http handler at single-node VictoriaMetrics
You can send a GET request to the `/internal/force_flush` HTTP handler at a single-node VictoriaMetrics
or to `vmstorage` at [cluster version of VictoriaMetrics](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/)
in order to forcibly flush the buffered samples to disk, so they become visible for querying. The `/internal/force_flush` handler
is provided for debugging and testing purposes only. Do not call it in production, since this may significantly slow down data ingestion
@@ -771,7 +772,7 @@ performance and increase resource usage.
### MetricsQL
VictoriaMetrics provide a special query language for executing read queries - [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/).
VictoriaMetrics provides a special query language for executing read queries - [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/).
It is a [PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics)-like query language with a powerful set of
functions and features for working specifically with time series data. MetricsQL is backward-compatible with PromQL,
so it shares most of the query concepts. The basic concepts for PromQL and MetricsQL are
@@ -779,7 +780,7 @@ described [here](https://valyala.medium.com/promql-tutorial-for-beginners-9ab455
#### Filtering
In sections [instant query](#instant-query) and [range query](#range-query) we've already used MetricsQL to get data for
In sections [instant query](#instant-query) and [range query](#range-query), we've already used MetricsQL to get data for
metric `foo_bar`. It is as simple as just writing a metric name in the query:
```metricsql
@@ -793,14 +794,14 @@ requests_total{path="/", code="200"}
requests_total{path="/", code="403"}
```
To select only time series with specific label value specify the matching filter in curly braces:
To select only time series with a specific label value, specify the matching filter in curly braces:
```metricsql
requests_total{code="200"}
```
The query above returns all time series with the name `requests_total` and label `code="200"`. We use the operator `=` to
match label value. For negative match use `!=` operator. Filters also support positive regex matching via `=~`
match the label value. For negative matches, use the `!=` operator. Filters also support positive regex matching via `=~`
and negative regex matching via `!~`:
```metricsql
@@ -813,7 +814,7 @@ Filters can also be combined:
requests_total{code=~"200", path="/home"}
```
The query above returns all time series with `requests_total` name, which simultaneously have labels `code="200"` and `path="/home"`.
The query above returns all time series with the `requests_total` name, which simultaneously have labels `code="200"` and `path="/home"`.
#### Filtering by name
@@ -829,7 +830,7 @@ The query above returns series for two metrics: `requests_error_total` and `requ
#### Filtering by multiple "or" filters
[MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/) supports selecting time series, which match at least one of multiple "or" filters.
[MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/) supports selecting time series that match at least one of multiple "or" filters.
Such filters must be delimited by `or` inside curly braces. For example, the following query selects time series with
`{job="app1",env="prod"}` or `{job="app2",env="dev"}` labels:
@@ -838,7 +839,7 @@ Such filters must be delimited by `or` inside curly braces. For example, the fol
```
The number of `or` groups can be arbitrary. The number of `,`-delimited label filters per each `or` group can be arbitrary.
Per-group filters are applied with `and` operation, e.g. they select series simultaneously matching all the filters in the group.
Per-group filters are applied with the `and` operation, e.g., they select series simultaneously matching all the filters in the group.
This functionality allows passing the selected series to [rollup functions](https://docs.victoriametrics.com/victoriametrics/metricsql/#rollup-functions)
such as [rate()](https://docs.victoriametrics.com/victoriametrics/metricsql/#rate)
@@ -849,7 +850,7 @@ rate({job="app1",env="prod" or job="app2",env="dev"}[5m])
```
If you need to select series matching multiple filters for the same label, then it is better from performance PoV
If you need to select series matching multiple filters for the same label, then it is better from a performance PoV
to use regexp filter `{label=~"value1|...|valueN"}` instead of `{label="value1" or ... or label="valueN"}`.
@@ -878,8 +879,8 @@ query may break or may lead to incorrect results. The basics of the matching rul
* MetricsQL engine strips metric names from all the time series on the left and right side of the arithmetic operation
without touching labels.
* For each time series on the left side MetricsQL engine searches for the corresponding time series on the right side
with the same set of labels, applies the operation for each data point and returns the resulting time series with the
* For each time series on the left side, the MetricsQL engine searches for the corresponding time series on the right side
with the same set of labels, applies the operation for each data point, and returns the resulting time series with the
same set of labels. If there are no matches, then the time series is dropped from the result.
* The matching rules may be augmented with `ignoring`, `on`, `group_left` and `group_right` modifiers.
See [these docs](https://prometheus.io/docs/prometheus/latest/querying/operators/#vector-matching) for details.
@@ -896,7 +897,7 @@ MetricsQL supports the following comparison operators:
* less-or-equal - `<=`
These operators may be applied to arbitrary MetricsQL expressions as with arithmetic operators. The result of the
comparison operation is time series with only matching data points. For instance, the following query would return
comparison operation is a time series with only matching data points. For instance, the following query would return
series only for processes where memory usage exceeds `100MB`:
```metricsql
@@ -906,7 +907,7 @@ process_resident_memory_bytes > 100*1024*1024
#### Aggregation and grouping functions
MetricsQL allows aggregating and grouping of time series. Time series are grouped by the given set of labels and then the
given aggregation function is applied individually per each group. For instance, the following query returns
given aggregation function is applied individually to each group. For instance, the following query returns
summary memory usage for each `job`:
```metricsql
@@ -919,14 +920,14 @@ See [docs for aggregate functions in MetricsQL](https://docs.victoriametrics.com
One of the most widely used functions for [counters](#counter)
is [rate](https://docs.victoriametrics.com/victoriametrics/metricsql/#rate). It calculates the average per-second increase rate individually
per each matching time series. For example, the following query shows the average per-second data receive speed
per each monitored `node_exporter` instance, which exposes the `node_network_receive_bytes_total` metric:
for each matching time series. For example, the following query shows the average per-second data receive speed
for each monitored `node_exporter` instance, which exposes the `node_network_receive_bytes_total` metric:
```metricsql
rate(node_network_receive_bytes_total)
```
By default, VictoriaMetrics calculates the `rate` over [raw samples](#raw-samples) on the lookbehind window specified in the `step` param
By default, VictoriaMetrics calculates the `rate` over [raw samples](#raw-samples) on the lookbehind window specified in the `step` parameter
passed either to [instant query](#instant-query) or to [range query](#range-query).
The interval on which `rate` needs to be calculated can be specified explicitly
as [duration](https://prometheus.io/docs/prometheus/latest/querying/basics/#float-literals-and-time-durations) in square brackets:
@@ -935,10 +936,10 @@ as [duration](https://prometheus.io/docs/prometheus/latest/querying/basics/#floa
rate(node_network_receive_bytes_total[5m])
```
In this case VictoriaMetrics uses the specified lookbehind window - `5m` (5 minutes) - for calculating the average per-second increase rate.
In this case, VictoriaMetrics uses the specified lookbehind window - `5m` (5 minutes) - for calculating the average per-second increase rate.
Bigger lookbehind windows usually lead to smoother graphs.
`rate` strips metric name while leaving all the labels for the inner time series. If you need to keep the metric name,
`rate` strips the metric name while leaving all the labels for the inner time series. If you need to keep the metric name,
then add [keep_metric_names](https://docs.victoriametrics.com/victoriametrics/metricsql/#keep_metric_names) modifier
after the `rate(..)`. For example, the following query leaves metric names after calculating the `rate()`:
@@ -952,7 +953,7 @@ rate(node_network_receive_bytes_total) keep_metric_names
VictoriaMetrics has a built-in graphical User Interface for querying and visualizing metrics -
[VMUI](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui).
Open `http://victoriametrics:8428/vmui` page, type the query and see the results:
Open the `http://victoriametrics:8428/vmui` page, type the query, and see the results:
![vmui](vmui.webp)
@@ -963,8 +964,8 @@ in the same way as Grafana queries Prometheus.
## Modify data
VictoriaMetrics stores time series data in [MergeTree](https://en.wikipedia.org/wiki/Log-structured_merge-tree)-like
data structures. While this approach is very efficient for write-heavy databases, it applies some limitations on data
updates. In short, modifying already written [time series](#time-series) requires re-writing the whole data block where
data structures. While this approach is very efficient for write-heavy databases, it imposes some limitations on data
updates. In short, modifying already written [time series](#time-series) requires rewriting the whole data block where
it is stored. Due to this limitation, VictoriaMetrics does not support direct data modification.
### Deletion

View File

@@ -35,8 +35,8 @@ scrape_configs:
After you created the `scrape.yaml` file, download and unpack [single-node VictoriaMetrics](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) to the same directory:
```sh
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.148.0/victoria-metrics-linux-amd64-v1.148.0.tar.gz
tar xzf victoria-metrics-linux-amd64-v1.148.0.tar.gz
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.149.0/victoria-metrics-linux-amd64-v1.149.0.tar.gz
tar xzf victoria-metrics-linux-amd64-v1.149.0.tar.gz
```
Then start VictoriaMetrics and instruct it to scrape targets defined in `scrape.yaml` and save scraped metrics
@@ -150,8 +150,8 @@ Then start [single-node VictoriaMetrics](https://docs.victoriametrics.com/victor
```yaml
# Download and unpack single-node VictoriaMetrics
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.148.0/victoria-metrics-linux-amd64-v1.148.0.tar.gz
tar xzf victoria-metrics-linux-amd64-v1.148.0.tar.gz
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.149.0/victoria-metrics-linux-amd64-v1.149.0.tar.gz
tar xzf victoria-metrics-linux-amd64-v1.149.0.tar.gz
# Run single-node VictoriaMetrics with the given scrape.yaml
./victoria-metrics-prod -promscrape.config=scrape.yaml

View File

@@ -640,7 +640,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/
-vmalert.proxyURL string
Optional URL for proxying requests to vmalert. For example, if -vmalert.proxyURL=http://vmalert:8880 , then alerting API requests such as /api/v1/rules from Grafana will be proxied to http://vmalert:8880/api/v1/rules . See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmalert
-vmselectAddr string
TCP address to accept connections from vmselect services
TCP address to listen for incoming connections from vmselect. When set, the node will be able to accept cluster-native vmselect RPC requests as if it were vmstorage. The tenant ID assigned to this node's data is controlled by -accountID and -projectID flags. See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#multi-tenancy
-vmui.customDashboardsPath string
Optional path to vmui dashboards. See https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmui/packages/vmui/public/dashboards
-vmui.defaultTimezone string

View File

@@ -141,7 +141,6 @@ to other remote storage systems that support Prometheus `remote_write` protocol
If a single remote storage instance is temporarily unavailable, the collected data remains available on the other remote storage instances.
`vmagent` buffers the collected data in files at `-remoteWrite.tmpDataPath` until the remote storage becomes available again.
Then it sends the buffered data to the remote storage in order to prevent data gaps.
See how `vmagent` [selects shards and places replicas](https://victoriametrics.com/blog/vmagent-how-it-works/#step-4-sharding--replication) for implementation details.
[VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/) already supports replication,
so there is no need to specify multiple `-remoteWrite.url` flags when writing data to the same cluster.
@@ -151,8 +150,7 @@ See [these docs](https://docs.victoriametrics.com/victoriametrics/cluster-victor
`vmagent` can add, remove, or update labels on the collected data before sending it to the remote storage.
It can filter scrape targets or remove unwanted samples via Prometheus-like relabeling.
Please see the [Relabeling cookbook](https://docs.victoriametrics.com/victoriametrics/relabeling/) for configuration examples.
For ingestion pipeline internals, see how `vmagent` [applies global relabeling and cardinality limits](https://victoriametrics.com/blog/vmagent-how-it-works/#step-2-global-relabeling-cardinality-reduction).
Please see the [Relabeling cookbook](https://docs.victoriametrics.com/victoriametrics/relabeling/) for details.
### Sharding among remote storages
@@ -270,9 +268,7 @@ for the collected samples. Examples:
```sh
./vmagent -remoteWrite.url=http://remote-storage/api/v1/write -streamAggr.dropInputLabels=replica -streamAggr.dedupInterval=60s
```
See how `vmagent` [orders global deduplication and stream aggregation](https://victoriametrics.com/blog/vmagent-how-it-works/#step-3-global-deduplication--stream-aggregation) in the ingestion pipeline.
### Monitoring Data eXchange
The Monitoring Data eXchange (MDX){{% available_from "v1.147.0" %}} feature allows `vmagent` to forward only VictoriaMetrics metrics to selected `-remoteWrite.url` destinations while dropping metrics from non-VictoriaMetrics services.
@@ -363,8 +359,6 @@ in addition to the pull-based Prometheus-compatible targets' scraping:
* Prometheus exposition format via `http://<vmagent>:8429/api/v1/import/prometheus`. See [these docs](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-import-data-in-prometheus-exposition-format) for details.
* Arbitrary CSV data via `http://<vmagent>:8429/api/v1/import/csv`. See [these docs](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-import-csv-data).
See how `vmagent` [handles concurrency, decompression, and stream parsing](https://victoriametrics.com/blog/vmagent-how-it-works/#step-1-receiving-data-via-api-or-scrape) during ingestion.
## How to collect metrics in Prometheus format
Specify the path to the `prometheus.yml` file via the `-promscrape.config` command-line flag. `vmagent` takes into account the following
@@ -694,7 +688,7 @@ Extra labels can be added to metrics collected by `vmagent` via the following me
## Obfuscating label values
`vmagent` can obfuscate the values of specified labels before sending metrics to `-remoteWrite.url`
via `-remoteWrite.obfuscateLabels`{{% available_from "#" %}}.
via `-remoteWrite.obfuscateLabels`{{% available_from "v1.149.0" %}}.
This is useful when one or more `-remoteWrite.url` endpoints point to external monitoring services
outside the organization, and sensitive label values such as `ip`, `host`, `instance`, or `datacenter`
@@ -1072,7 +1066,6 @@ This behavior can be changed with the `-remoteWrite.inmemoryQueues` {{% availabl
When set to a non-zero value, vmagent starts the given number of additional workers,
which send only recently ingested data from the in-memory queue, while the workers configured via `-remoteWrite.queues` drain the file-based backlog concurrently.
This reduces the delivery lag for fresh samples after remote storage outages or slowdowns. The flag can be set individually per each `-remoteWrite.url`.
See how the [in-memory and file-based queues manage blocks](https://victoriametrics.com/blog/vmagent-how-it-works/#in-memory-queue) for implementation details.
Note that these workers are started in addition to the workers configured via `-remoteWrite.queues`, so the total number of concurrent connections to
the remote storage becomes the sum of both flags. Take this into account if the remote storage limits the number of concurrent requests.

View File

@@ -468,7 +468,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/vmagent/ .
-remoteWrite.maxRowsPerBlock int
The maximum number of samples to send in each block to remote storage. Higher number may improve performance at the cost of the increased memory usage. See also -remoteWrite.maxBlockSize (default 10000)
-remoteWrite.mdx.enable array
Whether to only retain metrics from VictoriaMetrics services before sending them to the corresponding -remoteWrite.url. Please see https://docs.victoriametrics.com/victoriametrics/vmagent/#monitoring-data-exchange
Whether to only retain metrics from VictoriaMetrics services before sending them to the corresponding -remoteWrite.url. Can be combined with -remoteWrite.obfuscateLabels to hide sensitive label values in the forwarded metrics. Please see https://docs.victoriametrics.com/victoriametrics/vmagent/#monitoring-data-exchange
Supports array of values separated by comma or specified via multiple flags.
Empty values are set to false.
-remoteWrite.oauth2.clientID array
@@ -495,6 +495,10 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/vmagent/ .
Optional OAuth2 tokenURL to use for the corresponding -remoteWrite.url
Supports an array of values separated by comma or specified via multiple flags.
Each array item can contain comma inside single-quoted or double-quoted string, {}, [] and () braces.
-remoteWrite.obfuscateLabels ^^
List of label names whose values will be obfuscated before being sent to the corresponding -remoteWrite.url. Multiple label names should be separated by ^^, e.g. "job^^instance,ip". Can be combined with -remoteWrite.mdx.enable to hide sensitive label values in VictoriaMetrics self-monitoring metrics. Please see https://docs.victoriametrics.com/victoriametrics/vmagent/#obfuscating-label-values
Supports an `array` of values separated by comma or specified via multiple flags.
Each array item can contain comma inside single-quoted or double-quoted string, {}, [] and () braces.
-remoteWrite.proxyURL array
Optional proxy URL for writing data to the corresponding -remoteWrite.url. Supported proxies: http, https, socks5. Example: -remoteWrite.proxyURL=socks5://proxy:1234
Supports an array of values separated by comma or specified via multiple flags.

View File

@@ -341,6 +341,7 @@ The following variables are available in templating:
| $externalLabels or .ExternalLabels | List of labels configured via `-external.label` command-line flag. | `Issues with {{ $labels.instance }} (datacenter-{{ $externalLabels.dc }})` |
| $externalURL or .ExternalURL | URL configured via `-external.url` command-line flag. Used for cases when vmalert is hidden behind proxy. | `Visit {{ $externalURL }} for more details` |
| $isPartial or .IsPartial | Indicates whether the latest rule query response from the datasource(that supports returning `isPartial` option, such as vmcluster) could be partial. | `{{ if $isPartial }}WARNING: The latest alert state may be a false alarm due to a partial response from the datasource.{{ end }}` |
| $interval or .Interval | Alerting rule group's evaluation interval. | `http://vm-grafana.com/<dashboard-id>?viewPanel=<panel-id>&from={{ ($activeAt.Add (parseDurationTime (printf "-%s" .Interval))).UnixMilli }}&to={{ $activeAt.UnixMilli }}` |
Additionally, `vmalert` provides some extra templating functions listed in [template functions](#template-functions) and [reusable templates](#reusable-templates).
@@ -1024,9 +1025,6 @@ Try the following tips to avoid common issues:
In that case, the default step will be used (`-datasource.queryStep`) and may cause unexpected results compared to
executing this query in vmui/Grafana, where step is adjusted differently.
See [practical examples for reducing alert noise](https://victoriametrics.com/blog/alerting-best-practices/#reducing-noise),
including aggregating alerts and configuring inhibition.
### Rule state
vmalert keeps the last `-rule.updateEntriesLimit` updates (or `update_entries_limit` [per-rule config](https://docs.victoriametrics.com/victoriametrics/vmalert/#alerting-rules))
@@ -1082,9 +1080,6 @@ Sometimes, it's hard to understand why a specific alert fired or not. Keep in mi
If evaluation returns error (i.e. datasource is unavailable), alert state doesn't change.
If at least one evaluation returns no data, then alert's `for` state resets.
See [how to tune the `for` parameter](https://victoriametrics.com/blog/alerting-best-practices/#the-for-param),
including its tradeoff with the query lookbehind window.
> Note: The alert state is tracked separately for each time series returned during evaluation.
> For example, if the 1st evaluation returns series A and B, and the 2nd evaluation returns only B the alert will remain active **only for B**.

View File

@@ -404,6 +404,8 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/vmalert/ .
Optional TLS server name to use for connections to -remoteWrite.url. By default, the server name from -remoteWrite.url is used
-remoteWrite.url string
Optional URL to persist alerts state and recording rules results in form of timeseries. It must support either VictoriaMetrics remote write protocol or Prometheus remote_write protocol. Supports address in the form of IP address with a port (e.g., http://127.0.0.1:8428) or DNS SRV record. For example, if -remoteWrite.url=http://127.0.0.1:8428 is specified, then the alerts state will be written to http://127.0.0.1:8428/api/v1/write . See also -remoteWrite.disablePathAppend, '-remoteWrite.showURL'.
-replay.continueWithExecutionErr
Whether to continue replaying other rules if a rule execution fails with a 422 response code, which can happen due to an expression syntax error or a resource limit being hit.
-replay.disableProgressBar
Whether to disable rendering progress bars during the replay. Progress bar rendering might be verbose or break the logs parsing, so it is recommended to be disabled when not used in interactive mode.
-replay.maxDatapointsPerQuery int
@@ -411,7 +413,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/vmalert/ .
-replay.ruleEvaluationConcurrency int
The maximum number of concurrent '/query_range' requests when replay recording rule or alerting rule with for=0. Increasing this value when replaying for a long time, since each request is limited by -replay.maxDatapointsPerQuery. (default 1)
-replay.ruleRetryAttempts int
Defines how many retries to make before giving up on rule if request for it returns an error. (default 5)
Defines how many retries to make before giving up on rule if request for it returns a retriable error. (default 5)
-replay.rulesDelay duration
Delay before evaluating the next rule within the group. Is important for chained rules. Keep it equal or bigger than -remoteWrite.flushInterval. When set to >0, replay ignores group's concurrency setting. (default 1s)
-replay.timeFrom string

View File

@@ -480,7 +480,7 @@ Clusters here are referred to as `source` and `destination`.
To verify that `vmbackupmanager` is executing backup tasks normally, the following metrics can help:
* `vm_backup_last_success_at{type="<backup_type>"}` - unix timestamp of the last successful backup{{% available_from "#" %}}. Remains `0` if no backup has completed successfully since startup. Check error logs and verify remote storage accessibility if this persists.
* `vm_backup_last_success_at{type="<backup_type>"}` - unix timestamp of the last successful backup{{% available_from "v1.149.0" %}}. Remains `0` if no backup has completed successfully since startup. Check error logs and verify remote storage accessibility if this persists.
* `vm_backup_last_run_failed{type="<backup_type>"}` - whether the last backup task for the given backup type failed. The value `1` means the last task failed. Check the error logs of `vmbackupmanager` for the root cause
* `vm_backup_errors_total{type="<backup_type>"}` - total number of backup errors for the given backup type.

View File

@@ -34,9 +34,9 @@ vmctl command-line tool is available as:
Download and unpack vmctl:
```sh
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.148.0/vmutils-darwin-arm64-v1.148.0.tar.gz
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.149.0/vmutils-darwin-arm64-v1.149.0.tar.gz
tar xzf vmutils-darwin-arm64-v1.148.0.tar.gz
tar xzf vmutils-darwin-arm64-v1.149.0.tar.gz
```
Once binary is unpacked, see the full list of supported modes by running the following command:

View File

@@ -31,7 +31,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/cluster-victori
-denyQueryTracing
Whether to disable the ability to trace queries. See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#query-tracing
-disableRerouting
Whether to disable re-routing when some of vmstorage nodes accept incoming data at slower speed compared to other storage nodes. Disabled re-routing limits the ingestion rate by the slowest vmstorage node. On the other side, disabled re-routing minimizes the number of active time series in the cluster during rolling restarts and during spikes in series churn rate. See also -disableReroutingOnUnavailable and -dropSamplesOnOverload (default true)
Whether to disable re-routing when some of vmstorage nodes accept incoming data at slower speed compared to other storage nodes. Disabled re-routing limits the ingestion rate by the slowest vmstorage node. On the other side, disabled re-routing minimizes the number of active time series in the cluster during rolling restarts and during spikes in series churn rate. See also -disableReroutingOnUnavailable and -dropSamplesOnOverload
-disableReroutingOnUnavailable
Whether to disable re-routing when some of vmstorage nodes are unavailable. Disabled re-routing stops ingestion when some storage nodes are unavailable. On the other side, disabled re-routing minimizes the number of active time series in the cluster during rolling restarts and during spikes in series churn rate. See also -disableRerouting
-dropSamplesOnOverload

View File

@@ -132,7 +132,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/cluster-victori
-loggerWarnsPerSecondLimit int
Per-second limit on the number of WARN messages. If more than the given number of warns are emitted per second, then the remaining warns are suppressed. Zero values disable the rate limit
-maxConcurrentInserts int
The maximum number of concurrent insert requests. Set higher value when clients send data over slow networks. Default value depends on the number of available CPU cores. It should work fine in most cases since it minimizes resource usage. See also -insert.maxQueueDuration (default 24)
The maximum number of concurrent insert requests. Set higher value when clients send data over slow networks. Default value depends on the number of available CPU cores. It should work fine in most cases since it minimizes resource usage. See also -insert.maxQueueDuration (default 20)
-memory.allowedBytes size
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage. The process may behave unexpectedly if this flag is set too small (e.g., 1 byte).
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)

View File

@@ -90,7 +90,11 @@ func newClient(ctx context.Context, sw *ScrapeWork) (*client, error) {
}
tr := httputil.NewTransport(false, "vm_promscrape")
if proxyURLFunc != nil {
if sw.UnixSocket != "" {
// Unix sockets are direct local endpoints and cannot be reached via HTTP proxies.
// Proxy could be set implicitly via global env variable HTTP_PROXY
tr.Proxy = nil
} else if proxyURLFunc != nil {
tr.Proxy = proxyURLFunc
}
tr.TLSHandshakeTimeout = 10 * time.Second

View File

@@ -1118,6 +1118,14 @@ func searchAndMerge[T any](qt *querytracer.Tracer, s *Storage, tr TimeRange, sea
qt = qt.NewChild("search indexDBs: timeRange=%v", &tr)
defer qt.Done()
var zeroValue T
if tr.MinTimestamp < minUnixMilli {
tr.MinTimestamp = minUnixMilli
}
if tr.MaxTimestamp < tr.MinTimestamp {
return zeroValue, nil
}
var idbts []indexDBWithType
ptws := s.tb.GetPartitions(tr)

View File

@@ -402,6 +402,10 @@ func TestStorageDeletePendingSeries(t *testing.T) {
defer testRemoveAll(t)
const numMonths = 10
start := time.Date(1971, 1, 1, 0, 0, 0, 0, time.UTC)
middle := start.AddDate(0, (numMonths-1)/2, 0)
end := start.AddDate(0, numMonths-1, 0)
s := MustOpenStorage(t.Name(), OpenOptions{})
var metricGroupName = []byte("metric")
@@ -456,7 +460,7 @@ func TestStorageDeletePendingSeries(t *testing.T) {
assertCountMonthsWithLabels := func(count int) {
t.Helper()
ts := time.Unix(0, 0)
ts := start
n := 0
for range numMonths {
lns, err := s.SearchLabelNames(nil, nil, TimeRange{ts.UnixMilli(), ts.UnixMilli()}, 1e5, 1e9, noDeadline)
@@ -481,7 +485,7 @@ func TestStorageDeletePendingSeries(t *testing.T) {
var search Search
defer search.MustClose()
search.Init(nil, s, []*TagFilters{tfs}, TimeRange{0, math.MaxInt64}, 1e5, noDeadline)
search.Init(nil, s, []*TagFilters{tfs}, TimeRange{start.UnixMilli(), math.MaxInt64}, 1e5, noDeadline)
n := 0
for search.NextMetricBlock() {
var b Block
@@ -498,10 +502,6 @@ func TestStorageDeletePendingSeries(t *testing.T) {
// Verify no metrics exist
assertCountRows(0)
start := time.Unix(0, 0)
middle := start.AddDate(0, (numMonths-1)/2, 0)
end := start.AddDate(0, numMonths-1, 0)
// Add some rows and flush, so next DeleteSeries() can delete them
addRows(start, middle, false)
s.DebugFlush()
@@ -3385,53 +3385,190 @@ func TestStorageQueryWithoutIndex(t *testing.T) {
testStorageSearchWithoutIndex(t, &opts)
}
func TestStorageAddRows_SamplesWithZeroDate(t *testing.T) {
func TestStorageAddRowsWithZeroDate(t *testing.T) {
defer testRemoveAll(t)
f := func(t *testing.T, disablePerDayIndex bool) {
t.Helper()
s := MustOpenStorage(t.Name(), OpenOptions{
DisablePerDayIndex: disablePerDayIndex,
})
defer s.MustClose()
mn := MetricName{MetricGroup: []byte("metric")}
mr := MetricRow{MetricNameRaw: mn.marshalRaw(nil)}
for range 10 {
mr.Timestamp = rand.Int63n(msecPerDay)
mr.Value = float64(rand.Intn(1000))
s.AddRows([]MetricRow{mr}, defaultPrecisionBits)
s.DebugFlush()
// Reset TSID cache so that insertion takes the path that involves
// checking whether the index contains metricName->TSID mapping.
s.resetAndSaveTSIDCache()
}
want := 1
firstUnixDay := TimeRange{
MinTimestamp: 0,
MaxTimestamp: msecPerDay - 1,
}
if got := s.newTimeseriesCreated.Load(); got != uint64(want) {
t.Errorf("unexpected new timeseries count: got %d, want %d", got, want)
}
if got := testCountAllMetricNames(s, firstUnixDay); got != want {
t.Errorf("unexpected metric name count: got %d, want %d", got, want)
}
if got := testCountAllMetricIDs(s, firstUnixDay); got != want {
t.Errorf("unexpected metric id count: got %d, want %d", got, want)
}
}
for _, disablePerDayIndex := range []bool{false, true} {
name := fmt.Sprintf("disablePerDayIndex=%t", disablePerDayIndex)
t.Run(name, func(t *testing.T) {
f(t, disablePerDayIndex)
testStorageAddRowsWithZeroDate(t, disablePerDayIndex)
})
}
}
func testStorageAddRowsWithZeroDate(t *testing.T, disablePerDayIndex bool) {
s := MustOpenStorage(t.Name(), OpenOptions{
DisablePerDayIndex: disablePerDayIndex,
})
defer s.MustClose()
const numDays = 4
var metricNamesAll []string
labelNamesAll := []string{"__name__", "label"}
var labelValuesAll []string
mrs := make([]MetricRow, numDays)
for day := range numDays {
metricName := fmt.Sprintf("metric_%02d", day)
labelName := fmt.Sprintf("label_%02d", day)
labelValue := fmt.Sprintf("value_%02d", day)
if day != 0 {
metricNamesAll = append(metricNamesAll, metricName)
labelNamesAll = append(labelNamesAll, labelName)
labelValuesAll = append(labelValuesAll, labelValue)
}
mn := MetricName{
MetricGroup: []byte(metricName),
Tags: []Tag{
{Key: []byte(labelName), Value: []byte("value")},
{Key: []byte("label"), Value: []byte(labelValue)},
},
}
mn.sortTags()
mrs[day].MetricNameRaw = mn.marshalRaw(nil)
mrs[day].Timestamp = int64(day * msecPerDay)
}
s.AddRows(mrs, defaultPrecisionBits)
s.DebugFlush()
if got, want := s.newTimeseriesCreated.Load(), uint64(numDays-1); got != want {
t.Fatalf("unexpected new timeseries count: got %d, want %d", got, want)
}
if got, want := s.tooSmallTimestampRows.Load(), uint64(1); got != want {
t.Fatalf("unexpected rows with too small timestamp: got %d, want %d", got, want)
}
assertMetricNames := func(tr TimeRange, want []string) {
t.Helper()
tfs := NewTagFilters()
if err := tfs.Add(nil, []byte("metric_.*"), false, true); err != nil {
t.Fatalf("unexpected error in TagFilters.Add: %v", err)
}
got, err := s.SearchMetricNames(nil, []*TagFilters{tfs}, tr, 1e9, noDeadline)
if err != nil {
t.Fatalf("SearchMetricNames(%v, %v) failed unexpectedly: %v", tfs, &tr, err)
}
for i, name := range got {
var mn MetricName
if err := mn.UnmarshalString(name); err != nil {
t.Fatalf("Could not unmarshal metric name %q: %v", name, err)
}
got[i] = string(mn.MetricGroup)
}
slices.Sort(got)
slices.Sort(want)
if diff := cmp.Diff(want, got); diff != "" {
t.Fatalf("unexpected metric names (-want, +got):\n%s", diff)
}
}
assertLabelNames := func(tr TimeRange, want []string) {
t.Helper()
tfs := NewTagFilters()
if err := tfs.Add(nil, []byte("metric_.*"), false, true); err != nil {
t.Fatalf("unexpected error in TagFilters.Add: %v", err)
}
got, err := s.SearchLabelNames(nil, []*TagFilters{tfs}, tr, 1e9, 1e9, noDeadline)
if err != nil {
t.Fatalf("SearchLabelNames(%v, %v) failed unexpectedly: %s", tfs, &tr, err)
}
slices.Sort(got)
slices.Sort(want)
if diff := cmp.Diff(want, got); diff != "" {
t.Fatalf("unexpected label names (-want, +got):\n%s", diff)
}
}
assertLabelValues := func(tr TimeRange, want []string) {
t.Helper()
tfs := NewTagFilters()
if err := tfs.Add([]byte("label"), []byte("value_.*"), false, true); err != nil {
t.Fatalf("unexpected error in TagFilters.Add: %v", err)
}
got, err := s.SearchLabelValues(nil, "label", []*TagFilters{tfs}, tr, 1e9, 1e9, noDeadline)
if err != nil {
t.Fatalf("SearchLabelValues(%v, %v) failed unexpectedly: %s", tfs, tr, err)
}
slices.Sort(got)
slices.Sort(want)
if diff := cmp.Diff(want, got); diff != "" {
t.Fatalf("unexpected label values (-want, +got):\n%s", diff)
}
}
assertData := func(tr TimeRange, want []MetricRow) {
t.Helper()
tfs := NewTagFilters()
if err := tfs.Add(nil, []byte("metric_.*"), false, true); err != nil {
t.Fatalf("TagFilters.Add() failed unexpectedly: %v", err)
}
if err := testAssertSearchResult(s, tr, tfs, want); err != nil {
t.Fatalf("Search(%v, %v) failed unexpectedly: %v", tfs, tr, err)
}
}
var tr TimeRange
// Empty time range.
// Expect empty search results
tr = TimeRange{}
assertMetricNames(tr, nil)
assertLabelNames(tr, []string{})
assertLabelValues(tr, []string{})
assertData(tr, nil)
// First day time range.
// Expect empty search results
tr = TimeRange{
MinTimestamp: 0,
MaxTimestamp: msecPerDay - 1,
}
assertMetricNames(tr, nil)
assertLabelNames(tr, []string{})
assertLabelValues(tr, []string{})
assertData(tr, nil)
// Second day time range.
tr = TimeRange{
MinTimestamp: msecPerDay,
MaxTimestamp: 2*msecPerDay - 1,
}
if disablePerDayIndex {
// Expect index search results for all days if per-day index is
// disabled.
assertMetricNames(tr, metricNamesAll)
assertLabelNames(tr, labelNamesAll)
assertLabelValues(tr, labelValuesAll)
} else {
// Expect index search results on second day only if per-day index is
// enabled.
assertMetricNames(tr, []string{"metric_01"})
assertLabelNames(tr, []string{"__name__", "label", "label_01"})
assertLabelValues(tr, []string{"value_01"})
}
assertData(tr, mrs[1:2])
// First two days time range.
// Expect results on second day only.
tr = TimeRange{
MinTimestamp: 0,
MaxTimestamp: 2*msecPerDay - 1,
}
if disablePerDayIndex {
// Expect index search results for all days if per-day index is
// disabled.
assertMetricNames(tr, metricNamesAll)
assertLabelNames(tr, labelNamesAll)
assertLabelValues(tr, labelValuesAll)
} else {
// Expect index search results on second day only if per-day index is
// enabled.
assertMetricNames(tr, []string{"metric_01"})
assertLabelNames(tr, []string{"__name__", "label", "label_01"})
assertLabelValues(tr, []string{"value_01"})
}
assertData(tr, mrs[1:2])
}
// testSearchMetricIDs returns metricIDs for the given tfss and tr.
//
// The returned metricIDs are sorted. The function panics in in case of error.

View File

@@ -429,9 +429,8 @@ func (tb *table) getMinMaxIngestionTimestamps() (int64, int64) {
func (tb *table) getMinMaxTimestampsForAge(minAgeMsecs int64) (int64, int64) {
now := int64(fasttime.UnixTimestamp() * 1000)
minTimestamp := now - minAgeMsecs
if minTimestamp < 0 {
// Negative timestamps aren't supported by the storage.
minTimestamp = 0
if minTimestamp < minUnixMilli {
minTimestamp = minUnixMilli
}
maxTimestamp := int64(maxUnixMilli)
if maxUnixMilli-now > tb.s.futureRetentionMsecs {

View File

@@ -40,12 +40,6 @@ type TimeRange struct {
MaxTimestamp int64
}
// Zero time range and zero date are used to force global index search.
var (
globalIndexTimeRange = TimeRange{}
globalIndexDate = uint64(0)
)
// DateRange returns the date range for the given time range.
func (tr *TimeRange) DateRange() (uint64, uint64) {
minDate := uint64(tr.MinTimestamp) / msecPerDay
@@ -117,10 +111,29 @@ func (tr *TimeRange) contains(timestamp int64) bool {
return tr.MinTimestamp <= timestamp && timestamp <= tr.MaxTimestamp
}
// Zero time range and zero date are used to force global index search.
var (
globalIndexDate = uint64(0)
globalIndexTimeRange = TimeRange{}
)
const (
msecPerDay = 24 * 3600 * 1000
msecPerHour = 3600 * 1000
// minUnixMilli is the min millisecond that is allowed to be used as the
// sample timestamp.
//
// It corresponds to the first millisecond of the second day of the Unix
// Epoch, i.e. 1970-01-02T00:00:00.000Z.
//
// The first day of the Unix Epoch is reserved: zero date and zero time
// range are used for indicating that the the global index search is
// required. See globalIndexDate and globalIndexTimeRange above.
//
// Negative timestamps aren't supported.
minUnixMilli = msecPerDay
// maxUnixMilli is the max millisecond that is allowed to be used as the
// sample timestamp.
//
@@ -130,6 +143,6 @@ const (
// time.UnixMicro(math.MaxInt64/1000) == 2262-04-11 23:47:16.854775 UTC.
//
// Round it to the last millisecond of the last complete partition:
// 2262-03-31 23:59:59.999 UTC.
// 2262-03-31T23:59:59.999Z.
maxUnixMilli = 9222422399999
)

View File

@@ -935,4 +935,25 @@ foo:1m_sum_samples{baz="qwe"} 10
dedup_interval: 30s
outputs: [sum_samples]
`, "11111111")
// Reproduce issue #11261: sum_samples_total must be monotonic with enable_windows: true
// See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11262
f([]string{`
test_delta 1
`, `
test_delta 1
`, `
test_delta 1
`, `
test_delta 1
`}, time.Minute, `test_delta 1
test_delta 2
test_delta 3
test_delta 4
`, `
- interval: 1m
keep_metric_names: true
outputs: [sum_samples_total]
enable_windows: true
`, "1111")
}

View File

@@ -4,30 +4,39 @@ import (
"math"
)
type sumSamplesAggrValueShared struct {
total float64
}
type sumSamplesAggrValue struct {
sum float64
delta float64
shared *sumSamplesAggrValueShared
}
func (av *sumSamplesAggrValue) pushSample(_ aggrConfig, sample *pushSample, _ string, _ int64) {
if math.Abs(av.sum) >= (1 << 53) {
// It is time to reset the entry, since it starts losing float64 precision
av.sum = 0
}
av.sum += sample.value
av.delta += sample.value
}
func (av *sumSamplesAggrValue) flush(c aggrConfig, ctx *flushCtx, key string, _ bool) {
ac := c.(*sumSamplesAggrConfig)
if ac.resetTotalOnFlush {
ctx.appendSeries(key, "sum_samples", av.sum)
av.sum = 0
ctx.appendSeries(key, "sum_samples", av.delta)
av.delta = 0
return
}
ctx.appendSeries(key, "sum_samples_total", av.sum)
total := av.shared.total + av.delta
av.delta = 0
if math.Abs(total) >= (1 << 53) {
// It is time to reset the entry, since it starts losing float64 precision
av.shared.total = 0
} else {
av.shared.total = total
}
ctx.appendSeries(key, "sum_samples_total", total)
}
func (*sumSamplesAggrValue) state() any {
return nil
func (av *sumSamplesAggrValue) state() any {
return av.shared
}
func newSumSamplesAggrConfig(resetTotalOnFlush bool) aggrConfig {
@@ -40,6 +49,14 @@ type sumSamplesAggrConfig struct {
resetTotalOnFlush bool
}
func (*sumSamplesAggrConfig) getValue(_ any) aggrValue {
return &sumSamplesAggrValue{}
func (*sumSamplesAggrConfig) getValue(s any) aggrValue {
var shared *sumSamplesAggrValueShared
if s == nil {
shared = &sumSamplesAggrValueShared{}
} else {
shared = s.(*sumSamplesAggrValueShared)
}
return &sumSamplesAggrValue{
shared: shared,
}
}

View File

@@ -148,11 +148,11 @@ func TryParseUnixTimestamp(s string) (int64, bool) {
if !ok {
return 0, false
}
n, ok := tryParseScientificNumberForUnixTimestamp(s[:expIdx], decimalExp)
n, ok := tryParseScientificUnixTimestamp(s[:expIdx], decimalExp)
if !ok {
return 0, false
}
return getUnixTimestampNanoseconds(n), true
return n, true
}
dotIdx := strings.IndexByte(s, '.')
@@ -168,22 +168,11 @@ func TryParseUnixTimestamp(s string) (int64, bool) {
// The timestamp is fractional.
intStr := s[:dotIdx]
fracStr := s[dotIdx+1:]
n, ok := tryParseFractionalNumberForUnixTimestamp(intStr, fracStr)
n, ok := tryParseFractionalUnixTimestamp(intStr, fracStr)
if !ok {
return 0, false
}
// Adjust the n to multiples of thousands, since this is expected by getUnixTimestampNanoseconds.
decimalExp := len(fracStr)
for decimalExp%3 != 0 {
if n >= 0 && n > math.MaxInt64/10 || n < 0 && n < math.MinInt64/10 {
return 0, false
}
n *= 10
decimalExp++
}
return getUnixTimestampNanoseconds(n), true
return n, true
}
func getExpIndex(s string) int {
@@ -196,30 +185,52 @@ func getExpIndex(s string) int {
return -1
}
func tryParseScientificNumberForUnixTimestamp(s string, decimalExp int64) (int64, bool) {
func tryParseScientificUnixTimestamp(s string, decimalExp int64) (int64, bool) {
if decimalExp < 0 {
// Negative exponents on a fractional mantissa are intentionally not
// supported. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11268
return 0, false
}
dotIdx := strings.IndexByte(s, '.')
if dotIdx < 0 {
n, ok := tryParseInt64(s)
if !ok {
return 0, false
}
return multiplyByDecimalExp(n, decimalExp)
n, ok = multiplyByDecimalExp(n, decimalExp)
if !ok {
return 0, false
}
return getUnixTimestampNanoseconds(n), true
}
intStr := s[:dotIdx]
fracStr := s[dotIdx+1:]
if decimalExp < int64(len(fracStr)) {
if decimalExp >= int64(len(fracStr)) {
// The exponent shifts the decimal point past every fractional digit.
n, ok := tryParseDecimalMantissaAsInt(intStr, fracStr)
if !ok {
return 0, false
}
decimalExp -= int64(len(fracStr))
n, ok = multiplyByDecimalExp(n, decimalExp)
if !ok {
return 0, false
}
return getUnixTimestampNanoseconds(n), true
}
// The exponent leaves fractional digits, e.g. 1.784144612388E9 == 1784144612.388
if decimalExp >= int64(len(decimalMultipliers)) {
return 0, false
}
n, ok := tryParseFractionalNumberForUnixTimestamp(intStr, fracStr)
if !ok {
return 0, false
}
decimalExp -= int64(len(fracStr))
return multiplyByDecimalExp(n, decimalExp)
decimalExpInt := int(decimalExp)
intStr = s[:dotIdx] + fracStr[:decimalExpInt]
fracStr = fracStr[decimalExpInt:]
return tryParseFractionalUnixTimestamp(intStr, fracStr)
}
func tryParseFractionalNumberForUnixTimestamp(intStr, fracStr string) (int64, bool) {
func tryParseDecimalMantissaAsInt(intStr, fracStr string) (int64, bool) {
n, ok := tryParseInt64(intStr)
if !ok {
return 0, false
@@ -251,6 +262,53 @@ func tryParseFractionalNumberForUnixTimestamp(intStr, fracStr string) (int64, bo
return num, true
}
func tryParseFractionalUnixTimestamp(intStr, fracStr string) (int64, bool) {
n, ok := tryParseInt64(intStr)
if !ok {
return 0, false
}
isNegative := n < 0 || n == 0 && strings.HasPrefix(intStr, "-")
multiplier, maxFracDigits := getUnixTimestampMultiplier(n)
// Truncate the fractional digits to valid length according to the unit precision.
if len(fracStr) > maxFracDigits {
// 1.123456789XXX is invalid.
tail := fracStr[maxFracDigits:]
for i := 0; i < len(tail); i++ {
if tail[i] < '0' || tail[i] > '9' {
return 0, false
}
}
fracStr = fracStr[:maxFracDigits]
}
if len(fracStr) == 0 {
return n * multiplier, true
}
frac, ok := tryParseInt64(fracStr)
if !ok {
return 0, false
}
decimalExp := len(fracStr)
if decimalExp >= len(decimalMultipliers) {
return 0, false
}
n *= multiplier
scale := decimalMultipliers[decimalExp]
frac *= multiplier / scale
if isNegative {
if n < math.MinInt64+frac {
return 0, false
}
return n - frac, true
}
if n > math.MaxInt64-frac {
return 0, false
}
return n + frac, true
}
func multiplyByDecimalExp(n int64, decimalExp int64) (int64, bool) {
if decimalExp < 0 {
return 0, false
@@ -283,20 +341,25 @@ const (
)
func getUnixTimestampNanoseconds(n int64) int64 {
multiplier, _ := getUnixTimestampMultiplier(n)
return n * multiplier
}
func getUnixTimestampMultiplier(n int64) (int64, int) {
if n <= maxValidSecond && n >= minValidSecond {
// The timestamp is in seconds.
return n * 1e9
return 1e9, 9
}
if n <= maxValidMilli && n >= minValidMilli {
// The timestamp is in milliseconds.
return n * 1e6
return 1e6, 6
}
if n <= maxValidMicro && n >= minValidMicro {
// The timestamp is in microseconds.
return n * 1e3
return 1e3, 3
}
// The timestamp is in nanoseconds
return n
return 1, 0
}
func tryParseInt64(s string) (int64, bool) {

View File

@@ -24,51 +24,83 @@ func TestTryParseUnixTimestamp_Success(t *testing.T) {
f("0", 0)
// nanoseconds
f("-1234567890123456789", -1234567890123456789)
f("1234567890123456789", 1234567890123456789)
f("1234567890123456.789", 1234567890123456789)
f("-1234567890123456789", -1234567890_123_456_789)
f("1234567890123456789", 1234567890_123_456_789)
f("1234567890123456.789", 1234567890_123_456_789)
f("12345678901234567.891", 12345678_901_234_567)
f("12345678901234567.89", 12345678_901_234_567)
f("12345678901234567.8", 12345678_901_234_567)
// microseconds
f("-1234567890123456", -1234567890123456000)
f("1234567890123456", 1234567890123456000)
f("1234567890123456.789", 1234567890123456789)
f("-1234567890123456", -1234567890_123_456_000)
f("1234567890123456", 1234567890_123_456_000)
f("1234567890123456.789", 1234567890_123_456_789)
f("12345678901234.5000", 12345678_901_234_500)
f("12345678901234.5123", 12345678_901_234_512)
f("12345678901234.567891", 12345678_901_234_567)
// milliseconds
f("-1234567890123", -1234567890123000000)
f("1234567890123", 1234567890123000000)
f("1234567890123.456", 1234567890123456000)
f("-1234567890123", -1234567890_123_000_000)
f("1234567890123", 1234567890_123_000_000)
f("1234567890123.456", 1234567890_123_456_000)
// seconds
f("-1234567890", -1234567890000000000)
f("1234567890", 1234567890000000000)
f("1234567890.123456789", 1234567890123456789)
f("1234567890.12345678", 1234567890123456780)
f("1234567890.1234567", 1234567890123456700)
f("-1234567890.123456", -1234567890123456000)
f("-1234567890.12345", -1234567890123450000)
f("-1234567890.1234", -1234567890123400000)
f("-1234567890.123", -1234567890123000000)
f("-1234567890.12", -1234567890120000000)
f("-1234567890.1", -1234567890100000000)
f("-1234567890", -1234567890_000_000_000)
f("1234567890", 1234567890_000_000_000)
f("1234567890.123456789", 1234567890_123_456_789)
f("1234567890.12345678", 1234567890_123_456_780)
f("1234567890.1234567", 1234567890_123_456_700)
f("-1234567890.123456", -1234567890_123_456_000)
f("-1234567890.12345", -1234567890_123_450_000)
f("-1234567890.1234", -1234567890_123_400_000)
f("-1234567890.123", -1234567890_123_000_000)
f("-1234567890.12", -1234567890_120_000_000)
f("-1234567890.1", -1234567890_100_000_000)
f("12", 12_000_000_000)
f("12.", 12_000_000_000)
f("12.0", 12_000_000_000)
f("12.34", 12_340_000_000)
f("12.999999999000000000", 12_999_999_999)
f("0.1234567890123456789123", 123_456_789)
f("-0.1234567890123456789123", -123_456_789)
f("-12", -12_000_000_000)
f("-12.34", -12_340_000_000)
f("8223372", 8223372_000_000_000)
f("8223372.0", 8223372_000_000_000)
f("1700000000", 1700000000_000_000_000)
f("1700000000.0", 1700000000_000_000_000)
// scientific notation
f("1e9", 1000000000000000000)
f("1.234e9", 1234000000000000000)
f("-1.23456789e9", -1234567890000000000)
f("1.234567890123456789e18", 1234567890123456789)
f("-1.234567890123456789e18", -1234567890123456789)
f("0.23456789e9", 234567890000000000)
f("123.456789123e9", 123456789123000000)
f("-1234.5678912e9", -1234567891200000000)
f("123.678912e7", 1236789120000000000)
f("1.23e7", 12300000000000000)
f("1.23e6", 1230000000000000)
f("1.23e5", 123000000000000)
f("1.23e4", 12300000000000)
f("1.23e3", 1230000000000)
f("1.23e2", 123000000000)
f("1.2e1", 12000000000)
f("1123.456789123456789E15", 1123456789123456789)
f("1e9", 1000000000_000_000_000)
f("1.234e9", 1234000000_000_000_000)
f("-1.23456789e9", -1234567890_000_000_000)
f("1.234567890123456789e18", 1234567890_123_456_789)
f("-1.234567890123456789e18", -1234567890_123_456_789)
f("0.23456789e9", 234567890_000_000_000)
f("123.456789123e9", 123456789_123_000_000)
f("-1234.5678912e9", -1234567891_200_000_000)
f("123.678912e7", 1236789120_000_000_000)
f("1.23e7", 12300000_000_000_000)
f("1.23e6", 1230000_000_000_000)
f("1.23e5", 123000_000_000_000)
f("1.23e4", 12300_000_000_000)
f("1.23e3", 1230_000_000_000)
f("1.23e2", 123_000_000_000)
f("1.2e1", 12_000_000_000)
f("1123.456789123456789E15", 1123456789_123_456_789)
// scientific notation with sub-second precision, i.e. more fractional digits
// than the exponent shifts (https://github.com/VictoriaMetrics/VictoriaMetrics/issues/11268).
// These must match the equivalent plain fractional form.
f("1.784144612388E9", 1784144612_388_000_000) // == 1784144612.388
f("1.784144612388e9", 1784144612_388_000_000)
f("-1.784144612388e9", -1784144612_388_000_000)
f("1.5000000005e9", 1500000000_500_000_000) // == 1500000000.5
f("1.23456789e9", 1234567890_000_000_000) // exponent consumes all frac digits (integer result)
f("1.23e1", 12_300_000_000) // == 12.3
f("1.234e0", 1_234_000_000) // == 1.234
f("1234567890123456789.0e0", 1234567890_123_456_789)
}
func TestTryParseUnixTimestamp_Failure(t *testing.T) {
@@ -86,40 +118,29 @@ func TestTryParseUnixTimestamp_Failure(t *testing.T) {
f("foobar")
f("foo.bar")
f("1.12345671x34")
f("1.123456789x")
f("1.3e12345678x0123")
f("1xs.12345671")
f("1xs.12345671e5")
f("-1xs.12345671e5")
// missing fractional part
f("1233344.")
// too big timestamp
f("12345678901234567.891")
f("12345678901234567890")
f("12345678901234.567891")
f("12345678901234567890e3")
f("12345678901234567890.234e3")
f("-12345678901234567890")
f("12345678901234567890.235424")
f("12345678901234567890.235424e3")
f("-12345678901234567890.235424")
f("12345678901234567.89")
f("12345678901234567.8")
// too big fractional part
f("0.1234567890123456789123")
f("-0.1234567890123456789123")
// too big decimal exponent
f("1e19")
f("1.3e123456789090123")
// too small decimal exponent
f("1.23e1")
f("1.234e0")
// negative decimal exponent
f("1E-1")
f("1.3e-123456789090123")
}
func TestParseTimeAtSuccess(t *testing.T) {
@@ -137,27 +158,27 @@ func TestParseTimeAtSuccess(t *testing.T) {
now := time.Now().UnixNano()
// unix timestamp in seconds
f("1562529662", now, 1562529662*1e9)
f("1562529662.6", now, 1562529662600*1e6)
f("1562529662.67", now, 1562529662670*1e6)
f("1562529662.678", now, 1562529662678*1e6)
f("1562529662.678123", now, 1562529662678123*1e3)
f("1562529662.678123456", now, 1562529662678123456)
f("1562529662", now, 1562529662_000_000_000)
f("1562529662.6", now, 1562529662_600_000_000)
f("1562529662.67", now, 1562529662_670_000_000)
f("1562529662.678", now, 1562529662_678_000_000)
f("1562529662.678123", now, 1562529662_678_123_000)
f("1562529662.678123456", now, 1562529662_678_123_456)
// unix timestamp in milliseconds
f("1562529662678", now, 1562529662678*1e6)
f("1562529662678.9", now, 1562529662678900*1e3)
f("1562529662678.901", now, 1562529662678901*1e3)
f("1562529662678.901324", now, 1562529662678901324)
f("1562529662678", now, 1562529662_678_000_000)
f("1562529662678.9", now, 1562529662_678_900_000)
f("1562529662678.901", now, 1562529662_678_901_000)
f("1562529662678.901324", now, 1562529662_678_901_324)
// unix timestamp in microseconds
f("1562529662678901", now, 1562529662678901*1e3)
f("1562529662678901.3", now, 1562529662678901300)
f("1562529662678901.32", now, 1562529662678901320)
f("1562529662678901.321", now, 1562529662678901321)
f("1562529662678901", now, 1562529662_678_901_000)
f("1562529662678901.3", now, 1562529662_678_901_300)
f("1562529662678901.32", now, 1562529662_678_901_320)
f("1562529662678901.321", now, 1562529662_678_901_321)
// unix timestamp in nanoseconds
f("1562529662678901234", now, 1562529662678901234)
f("1562529662678901234", now, 1562529662_678_901_234)
// duration relative to the current time
f("now", now, now)
@@ -166,7 +187,7 @@ func TestParseTimeAtSuccess(t *testing.T) {
// negative duration relative to the current time
f("-5m", now, now-5*60*1e9)
f("-123", now, now-123*1e9)
f("-123.456", now, now-123456*1e6)
f("-123.456", now, now-123_456*1e6)
f("now-1h5m", now, now-(3600+5*60)*1e9)
// Year
@@ -200,9 +221,9 @@ func TestParseTimeAtSuccess(t *testing.T) {
f("2023-05-20T04:57:43-02:30", now, 1.684567663e+09*1e9)
// milliseconds
f("2023-05-20T04:57:43.123Z", now, 1684558663123000000)
f("2023-05-20T04:57:43.123456789+02:30", now, 1684549663123456789)
f("2023-05-20T04:57:43.123456789-02:30", now, 1684567663123456789)
f("2023-05-20T04:57:43.123Z", now, 1684558663_123_000_000)
f("2023-05-20T04:57:43.123456789+02:30", now, 1684549663_123_456_789)
f("2023-05-20T04:57:43.123456789-02:30", now, 1684567663_123_456_789)
}
func TestParseTimeAtLimits(t *testing.T) {