Compare commits

...

66 Commits

Author SHA1 Message Date
Max Kotliar
66319ef2eb deployment/docker: Build multiarch base image and publish it
It turned out that the release-xxx commands [do not rely on the base
image](31adbf9094/deployment/docker/Makefile (L88-L89)),
so the security fix introduced in
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9805 was not
applied.

Attempted to build a multi-architecture Docker image locally. While
building per-architecture images worked, creating a manifest failed.

Docker does not support creating manifests purely from local images. It
always attempts to contact a remote registry, resulting in an
authentication error:
```
docker manifest create local/base:alpine_3.22.1-alpine_3.22.1-17ca1ba8c4
local/base:alpine_3.22.1-alpine_3.22.1-17ca1ba8c4-amd64
local/base:alpine_3.22.1-alpine_3.22.1-17ca1ba8c4-arm --amend
...
denied: requested access to the resource is denied
unauthorized: authentication required
```

As a result, the only viable approach is to build and push the
multi-arch base image to our registry first (same way as we do for
apps), and then use it as the ROOT_IMAGE\ CERTS_IMAGE in release-xxx
builds.
2025-10-06 16:13:13 +03:00
Artem Fetishev
31adbf9094 docs/CHANGELOG.md: cut v1.127.0
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2025-10-03 18:27:29 +02:00
Artem Fetishev
dd508b9542 make vmui-update
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2025-10-03 16:09:33 +00:00
Roman Khavronenko
f151668188 apptest: tolerate time drift (#9801)
Shift time selectors by 1s to prevent time drifting in tests. Because of
shifting, test was flapping based on when it is run. See
https://github.com/VictoriaMetrics/VictoriaMetrics/actions/runs/18186734439/job/51824319927

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9770

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-10-03 16:54:16 +02:00
Roman Khavronenko
17ca1ba8c4 deployment: upgrade libcrypto3 and libssl3 to 3.5.4-r0 (#9805)
Addresses CVE-2025-9230, CVE-2025-9231, CVE-2025-9232.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-10-03 14:22:03 +02:00
Fred Navruzov
7654536aab docs/vmanomaly: typos and 404 after 1.26.0 release (#9800)
### Describe Your Changes

fixing broken links in UI/FAQ pages

### Checklist

The following checks are **mandatory**:

- [x] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
- [x] My change adheres to [VictoriaMetrics development
goals](https://docs.victoriametrics.com/victoriametrics/goals/).
2025-10-02 21:04:19 +02:00
Fred Navruzov
901de22894 change page title from GUI to UI (#9799)
### Describe Your Changes

Fixing a typo in page name

### Checklist

The following checks are **mandatory**:

- [x] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
- [x] My change adheres to [VictoriaMetrics development
goals](https://docs.victoriametrics.com/victoriametrics/goals/).
2025-10-02 17:40:42 +02:00
Fred Navruzov
96e514439d docs/vmanomaly: release v1.26.0 (#9793)
### Describe Your Changes

Docs update for v1.26.0 release of `vmanomaly`, including vmui-like GUI
docs and `VLogsReader`

### Checklist

The following checks are **mandatory**:

- [x] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
- [x] My change adheres to [VictoriaMetrics development
goals](https://docs.victoriametrics.com/victoriametrics/goals/).
2025-10-02 17:34:03 +02:00
Aliaksandr Valialkin
43e189a56c docs/victoriametrics/enterprise.md: typo fix in the 'VictoriaLogs Enterprise features' chapter: VictoriaMetrics -> VictoriaLogs 2025-10-02 13:00:49 +02:00
Aliaksandr Valialkin
4dbf899d89 docs/victoriametrics/enterprise.md: properly state that Monitoring of Monitoring feature at VictoriaLogs enterprise helps preventing issues in VictoriaLogs setups, not VictoriaMetrics setups 2025-10-02 12:58:26 +02:00
Evgeny
2fcbf75539 app/vmalert: restore usage of query template in labels
- Fixes regression from https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9543
- Issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9783
2025-10-02 09:45:13 +02:00
Aliaksandr Valialkin
676a88793a Revert "integration test: prevent GetMetric from interrupting the test when metric not found"
This reverts commit ccf97a4143.

reason for revert: this change may break tests, which expect that ServesMetrics.GetMetric() fails
when the given metric doesn't exist in the output.

It is better to add 'TryGetMetric() (float64, bool)' function, which would return '(0, false)'
when the given metric doesn't exist, so the caller could decide what to do next.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9773
2025-10-02 02:43:00 +02:00
Aliaksandr Valialkin
8d3e9d1dac app/vmui/packages/vmui: deny indexing vmui page by Google and other web crawlers
The vmui page has zero interesting contents for indexing.
2025-10-01 13:53:40 +02:00
hagen1778
09251f0a1e docs: fix markdown formatting typo
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-10-01 13:34:11 +02:00
Hui Wang
4ea5f8a84d vmselect: prevent duplicate offset modifier when instant query uses r… (#9770)
…ollup functions rate() and avg_over_time() with cache available

fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9762
2025-10-01 13:31:08 +02:00
Roman Khavronenko
cd52978096 vendor: update metrics package to v1.40.2 (#9780)
Restore sorting order of summary and quantile metrics exposed by
VictoriaMetrics components on `/metrics` page.

https://github.com/VictoriaMetrics/metrics/pull/105

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-10-01 13:28:46 +02:00
Roman Khavronenko
f65e24b2ab docs: add Life of a sample section to vmagent docs (#9719)
The routing section aims to describe the processing flow in the exact
order to the user. It substitutes previous incomplete and verbose
routing documentation in Stream Aggregation docs
https://docs.victoriametrics.com/victoriametrics/stream-aggregation/#routing

The processing order is taken from picture in
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9646#issue-3367074827

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Max Kotliar <mkotlyar@victoriametrics.com>
Co-authored-by: func25 <phuongle0205@gmail.com>
Co-authored-by: Phuong Le <39565248+func25@users.noreply.github.com>
2025-10-01 13:20:53 +02:00
Andrii Chubatiuk
0579e68409 dashboards: add adhoc filter to query stats and operator (#9774)
Add ad-hoc filters to query stats and operator dashboards.
These filters are useful for exploring non-uniform metrics sets
without distinct job/instance filters.
2025-10-01 13:19:37 +02:00
Roman Khavronenko
f2aea8532f docs: clarify how vmagent addresses multi-level ingestion shortcomings (#9785)
The previous text didn't contain links to vmagent's capabilities.
Instead, it contained misleading multitenancy-mode link that doesn't
seem to be related to the subject.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-10-01 13:17:05 +02:00
hagen1778
94473ed262 docs: rm unreachable link
https://www.vultr.com/docs/install-and-configure-victoriametrics-on-debian is not reachable anymore.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-10-01 13:13:58 +02:00
Roman Khavronenko
c646a66b60 app/{vmbackup/vmrestore}: push metrics on shutdown
Push metrics on shutdown if `-pushmetrics.url` is configured. Before
metrics reporting might have been skipped because of shutdown.

Obsoletes https://github.com/VictoriaMetrics/metrics/pull/103

--------------

To test:
1. Run local VictoriaMetrics instance
2. Build and run vmbackup or vmrestore:
```
make vmbackup && ./bin/vmbackup -storageDataPath=victoria-metrics-data -snapshot.createURL="http://user:pass@localhost:8428/snapshot/create?authKey=foobar" -dst=fs:////vmbackup/dir -pushmetrics.url=http://localhost:8428/api/v1/import/prometheus,http://127.0.0.1:8428/api/v1/import/prometheus
```
3. Try playing with `-pushmetrics.url` (good/bad/many addresses) and
observe logs

Related PR https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9767
2025-09-30 09:28:25 +02:00
Zhu Jiekun
ccf97a4143 integration test: prevent GetMetric from interrupting the test when metric not found
Previously, `GetMetric` do `t.Fatalf` immediately when the target metric
not exist in `/metrics` page.

However, some metrics may start to appear after the process has been
running for a while. `t.Fatalf` invalidates the retry mechanism of
assertions, if the metric is not found the first time, the test case
will terminate.

This commit request changes `t.Fatalf` to `t.Logf` (instead of `t.Errorf`,
because error output may be considered a test case failure in some
scenarios).

Related PR https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9773
2025-09-30 09:27:03 +02:00
Nikolay
66df8a5003 lib/workingsetache:add runtime finalizer to the cache
Follow-up for cea9505bab

 fastcache.Cache allocates off-heap memory, which must be explicitly
returned back to the pool with Reset method call.

 After changed made at commit above, during cache transit from whole to
split mode, it's possible that current cache is referenced by Cache.Get
or Cache.Call atomic pointers. It leads to potential memory leaks, since
we don't have any memory synchronization for atomic.Pointer.Store calls.

 This commit adds `Finalizer` to the `fastcache.Cache` instances.
It properly releases memory, when cache is no reachable.

Related PR https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9769
2025-09-30 09:25:57 +02:00
Nikolay
cea9505bab lib/workingsetcache: properly transit cache state
Previously, cache state transition from split into whole could left
cache into broken state, if Reset cache method was called in switching
mode.

 Also, cache Reset didn't start background workers and didn't change
cache size.

 This commit properly check mode during cache transition. In addition,
it no longer stops background workers after whole mode transition and
always start workers during start-up.

 Access to the prev, curr and mode Cache fields are properly locked
in order to mitigate possible race conditions.

Related PR https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9769
2025-09-29 14:54:17 +02:00
Hui Wang
30ac8cd3fa vmalert: add -rule.resultLimit command-line flag to allow limiting … (#9737)
…the number of alerts or recording results a single rule can produce

fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5792
2025-09-29 13:07:37 +02:00
hagen1778
a1f0b792af apptest: remove vlogs related code
VictoriaLogs has a new home for integration tests
https://github.com/VictoriaMetrics/VictoriaLogs/tree/master/apptest

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-09-29 12:57:51 +02:00
hagen1778
50f75d751f lib/streamaggr: prevent compilator from overoptimizing testing path
It seems like go compilator skipped computations and allocations for samples
as they weren't used afterwards. Sinking results into global variable removes
this optimizations and benchmark starts showing allocations within `pushSamples` fn.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-09-29 12:52:35 +02:00
hagen1778
27f7bc81e0 docs: fix links leading to legacy anchors
Change link to point to up-to-date documents instead
of pointing to legacy links.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-09-29 12:37:53 +02:00
Artem Fetishev
90d23d7c9f lib/storage: refactor tsid search (#9765)
- Make SearchTSIDs look similar to SearchMetricNames, i.e. search for metricIDs within the method
- Make the corresponding corrupted index test look similar to one for metric names search

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2025-09-26 15:44:02 +02:00
Artem Fetishev
f68c028673 lib/storage: remove unused storage field from Search type
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2025-09-25 16:18:58 +02:00
hagen1778
f24bf391a4 deployment/docker: update Go builder from Go1.25.0 to Go1.25.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.25.1%20label%3ACherryPickApproved

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-09-25 11:41:10 +02:00
hagen1778
bc64ecfa3d deployment: bump Grafana to v12.2.0
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-09-25 11:34:22 +02:00
hagen1778
f0bbf6ec15 deployment: bump node-exporter to v1.9.1
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-09-25 11:33:34 +02:00
hagen1778
cff4bde4d6 deployment: bump alertmanager to v0.28.1
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-09-25 11:32:51 +02:00
hagen1778
1716f11677 deployment: drop vlogs-example-alerts
It was moved to VictoriaLogs repo https://github.com/VictoriaMetrics/VictoriaLogs/blob/master/deployment/docker/vlogs-example-alerts.yml

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-09-25 11:31:44 +02:00
Phuong Le
b4932ed2da docs: update internal vendoring contribution guide (#9739)
### Describe Your Changes

Consistently use the `v0.0.0-YYYYMMDDHHMMSS-commit_hash` reference for
the internal deps such as `github.com/VictoriaMetrics/VictoriaMetrics`
dependency, since it allows referring any commit without waiting for the
release tag.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
- [x] My change adheres to [VictoriaMetrics development
goals](https://docs.victoriametrics.com/victoriametrics/goals/).
2025-09-25 10:40:32 +02:00
hagen1778
77f2ab139f fix a small typo
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-09-25 10:39:49 +02:00
Hui Wang
5537140074 lib/protoparser: remove error log when marshaling an invalid comment or an empty HELP metadata line (#9732)
fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9710

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2025-09-25 10:36:58 +02:00
i2blind
5d766bf7f1 docs/stream-aggregation: streamAggr.dedupInterval comments on old samples (#9731)
### Describe Your Changes

- Add comments to stream-aggregation README.md to clarify the effect
that the +flag will have on old samples
- Fix a spelling error with peridically to periodically in several files
that codespell-check caught.

Related to [#6775]

### Checklist

The following checks are **mandatory**:

- [x] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
- [x] My change adheres to [VictoriaMetrics development
goals](https://docs.victoriametrics.com/victoriametrics/goals/).

---------

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
2025-09-25 10:32:02 +02:00
Andrii Chubatiuk
5907239181 app/vmui: reset select values, when 'ALL' selected (#9702)
### Describe Your Changes

resetting Select component selected items, when all items are selected,
this should speed up filtering on alerting page on VMUI

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
- [ ] My change adheres to [VictoriaMetrics development
goals](https://docs.victoriametrics.com/victoriametrics/goals/).

---------

Co-authored-by: Artur Minchukou <aminchukov@victoriametrics.com>
2025-09-25 10:13:15 +02:00
Andrii Chubatiuk
720c2bfa1d app/vmui: fix disabled state for select, textfield and datetimepicker components (#9698)
### Describe Your Changes

select and textfield components look confusing, while disabled. it's
impossible to guess if it's disabled or not before interaction. updated
colors for components, when they are disabled



### Checklist

The following checks are **mandatory**:

- [ ] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
- [ ] My change adheres to [VictoriaMetrics development
goals](https://docs.victoriametrics.com/victoriametrics/goals/).
2025-09-25 10:12:41 +02:00
Arie Heinrich
e971e6102e docs: markdown, grammar and spelling (#9695)
### Describe Your Changes

This pull request consists of the following:

1. Markdown fixes
    following https://www.markdownguide.org/basic-syntax/
and https://github.com/markdownlint/markdownlint/blob/main/docs/RULES.md

- Add empty lines after headers or lists
- Remove extra lines between paragraphs
- Remove extra spaces at the end of a line
- Add language to code quote
- Consistent list (dont mix astrixes and dashes on same file, choose one
and be consistent in the same file)
- Proper URL links
- Use meaningful context to URLs instead of "here".

2. Concise language

3. Grammar fixes

- removing extra spaces between words
- there are multiple ones but i picked the basic ones that triggered my
eye :)

4. Spelling fixes

### Checklist

The following checks are **mandatory**:

- [x] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
- [x] My change adheres to [VictoriaMetrics development
goals](https://docs.victoriametrics.com/victoriametrics/goals/).

---------

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2025-09-25 10:12:01 +02:00
Andrii Chubatiuk
5cd6d7cfba app/vmui: add minDate and maxDate parameters for DatePicker to allow limiting available dates to select (#9694)
add ability to limit available in datePicker dates using `minDate` and
`maxDate` parameters. all dates before `minDate` and after `maxDate`
cannot be picked. lower and upper bounds can be set independently.

This `minDate` and `maxDate` parameters aren't set by default in vmui.
The datepicker component with these params is re-used elsewhere.
2025-09-25 09:48:47 +02:00
hagen1778
907aa1973a docs: add question about old and out-of-order metrics to FAQ
The change also explciitly mentions `out-of-order` phrase, as it is commonly
used in Prometheus ecosystem.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-09-25 09:31:39 +02:00
Artem Fetishev
d6dacd9771 lib/storage: introduce metricNameSearch type
Searching metricName by metricID happens many times during a single API
call. This requires getting the current set of idbs before those calls
happen. Which is fine but requires propagating idbs across the code
base. This is also fine in case of OSS version as it is used in Search
only.

Propagating idbs across the code base becomes a problem in Enterprise
version as it is used in at least 3 places. As a result it becomes very
difficult to merge things from OSS to Ent.

Localizing the all the dependencies in one searchMetricName type and
reusing this type everywhere should make things simpler.

Related enterprise changes:
https://github.com/VictoriaMetrics/VictoriaMetrics-enterprise/compare/search-metric-name-ent?expand=1

Related PR https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9756
2025-09-24 15:25:48 +02:00
Artem Fetishev
5bb67a7f00 lib/storage: Move searchTSIDs to Storage
A small refactoring that reduces Search dependency on Storage:

- Move searchTSIDs() from Search to Storage because this method does not
depend on anything Search-specific but does depend on Storage.
- Use metricsTracker instead of storage.metricTracker.

Related PR https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9754
2025-09-24 15:17:21 +02:00
Artem Fetishev
8c1c92d4c9 lib/storage: rewrite search benchmarks to allow to make it easy adding new cases (#9691)
Benchmarking storage search api requires taking into account many
parameters, such as:
- data configuration: how many series, deleted series, search time range
- where the index data recides: prev and or indexDB
- which search operation to measure

While adding a new benchmark use case involves a lot boilerplate code.

This pr implements a framework for testing storage search ops that can
be relatively easily extended. This come in expecially handy when adding
new cases for parition index.

The current set of params will result of a lot of benchmarks to be run
which most probably does not make sense because:
- it will take a lot of time and
- the output data is hard to compare manually.

However, these benchmarks are very useful when only small set of params
is of interest. For example, if I want to compare the search of 100k
metric names when the index data resides in prevOnly, currOnly or
prevAndCurr indexDBs. This would translate in the following cmd:

```shell
go test ./lib/storage --loggerLevel=ERROR -run=^$ -bench=^BenchmarkSearch/MetricNames/.*/VariableSeries/100000$
```

Why this change:
- I often need to run benchmarks with configs that I did not have
before, requires either modifying the existing one or writing a new one.
It is easy to get lost and make benchmark non-comparable
- I need some way to make legacy and pt index benchmarks comparable

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2025-09-22 15:05:32 +02:00
Roman Khavronenko
95ca45d05a docs: replace link to WITH templates playgorund (#9729)
The new link is shorter and has nice UI.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-09-22 13:52:42 +02:00
Nils K
828a2aaf17 docs: fix typo of dree -> free in formula (#9743) 2025-09-22 13:52:13 +02:00
hagen1778
007ae5a3f0 docs: fix a few typos in the changelog
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-09-22 13:51:40 +02:00
Zakhar Bessarab
dcd23da4ba docs/vmbackupmanager: add docs to clarify unsafe usage of lifecycle rules (#9728)
- state that it is unsafe to use lifecycle rules and describe the reason
- update formatting according latest changes in docs


---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Max Kotliar <mkotlyar@victoriametrics.com>
2025-09-22 11:56:48 +04:00
Roman Khavronenko
e33dbaf3d2 docs: update vmagent diagram image (#9727)
The original image seems outdated by now.
Replacing it with the updated and more detailed version from
https://victoriametrics.com/blog/vmagent-key-features-explained/

Picture is created by @func25

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: func25 <phuongle0205@gmail.com>
2025-09-17 16:34:52 +03:00
Arie Heinrich
c68973a247 Markdown, grammar and spelling (#9692)
### Describe Your Changes

This pull request consists of the following:

1. Markdown fixes
    following https://www.markdownguide.org/basic-syntax/
and https://github.com/markdownlint/markdownlint/blob/main/docs/RULES.md
- Add empty lines after headers or lists
- Remove extra lines between paragraphs
- Remove extra spaces at the end of a line
- Add language to code quote
- Consistent list (dont mix astrixes and dashes on same file, choose one
and be consistent in the same file)
- Proper URL links
- Use meaningful context to URLs instead of "here".

2. Concise language

3. Grammar fixes

- removing extra spaces between words
- there are multiple ones but i picked the basic ones that triggered my
eye :)

4. Spelling fixes

### Checklist

The following checks are **mandatory**:

- [x] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
- [x] My change adheres to [VictoriaMetrics development
goals](https://docs.victoriametrics.com/victoriametrics/goals/).
2025-09-17 13:52:43 +03:00
Aliaksandr Valialkin
2c72ef0f38 app/vmauth: follow-up for 8ce4636bc0
- Rename copyStream to copyStreamToClient in order to make it more clear
  that the stream must be copied from backend to client.

- Make sure that the client implements net/http.Flusher interface.
  It is a programming error (BUG) if the client passed to copyStreamToClient
  doesn't implement net/http.Flusher interface.

- Do not write zero-length data to the backend.

Updates https://github.com/VictoriaMetrics/VictoriaLogs/issues/667
2025-09-17 10:26:40 +02:00
Roman Khavronenko
bd0551da3b deployment: drop logs-benchmark (#9726)
It has a new home now - see
https://github.com/VictoriaMetrics/VictoriaLogs/tree/master/deployment/logs-benchmark

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-09-16 17:14:33 +03:00
Dima Shur
9f52c40b0b Improvements for backup description and configuration for single node, cluster , quick start (#9459)
### Describe Your Changes

Updating backup-related documentation:
vmbackup, single node, cluster node, quick start to increase clarity and
improve doc structure

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
- [ ] My change adheres to [VictoriaMetrics development
goals](https://docs.victoriametrics.com/victoriametrics/goals/).

---------

Co-authored-by: Max Kotliar <mkotlyar@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2025-09-16 16:41:54 +03:00
dependabot[bot]
ba3b50df1d build(deps): bump vite from 7.0.4 to 7.1.5 in /app/vmui/packages/vmui (#9706)
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite)
from 7.0.4 to 7.1.5.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/vitejs/vite/releases">vite's
releases</a>.</em></p>
<blockquote>
<h2>v7.1.5</h2>
<p>Please refer to <a
href="https://github.com/vitejs/vite/blob/v7.1.5/packages/vite/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v7.1.4</h2>
<p>Please refer to <a
href="https://github.com/vitejs/vite/blob/v7.1.4/packages/vite/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v7.1.3</h2>
<p>Please refer to <a
href="https://github.com/vitejs/vite/blob/v7.1.3/packages/vite/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v7.1.2</h2>
<p>Please refer to <a
href="https://github.com/vitejs/vite/blob/v7.1.2/packages/vite/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v7.1.1</h2>
<p>Please refer to <a
href="https://github.com/vitejs/vite/blob/v7.1.1/packages/vite/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>create-vite@7.1.1</h2>
<p>Please refer to <a
href="https://github.com/vitejs/vite/blob/create-vite@7.1.1/packages/create-vite/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>plugin-legacy@7.1.0</h2>
<p>Please refer to <a
href="https://github.com/vitejs/vite/blob/plugin-legacy@7.1.0/packages/plugin-legacy/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>create-vite@7.1.0</h2>
<p>Please refer to <a
href="https://github.com/vitejs/vite/blob/create-vite@7.1.0/packages/create-vite/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v7.1.0</h2>
<p>Please refer to <a
href="https://github.com/vitejs/vite/blob/v7.1.0/packages/vite/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v7.1.0-beta.1</h2>
<p>Please refer to <a
href="https://github.com/vitejs/vite/blob/v7.1.0-beta.1/packages/vite/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v7.1.0-beta.0</h2>
<p>Please refer to <a
href="https://github.com/vitejs/vite/blob/v7.1.0-beta.0/packages/vite/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v7.0.7</h2>
<p>Please refer to <a
href="https://github.com/vitejs/vite/blob/v7.0.7/packages/vite/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v7.0.6</h2>
<p>Please refer to <a
href="https://github.com/vitejs/vite/blob/v7.0.6/packages/vite/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v7.0.5</h2>
<p>Please refer to <a
href="https://github.com/vitejs/vite/blob/v7.0.5/packages/vite/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md">vite's
changelog</a>.</em></p>
<blockquote>
<h2><!-- raw HTML omitted --><a
href="https://github.com/vitejs/vite/compare/v7.1.4...v7.1.5">7.1.5</a>
(2025-09-08)<!-- raw HTML omitted --></h2>
<h3>Bug Fixes</h3>
<ul>
<li>apply <code>fs.strict</code> check to HTML files (<a
href="https://redirect.github.com/vitejs/vite/issues/20736">#20736</a>)
(<a
href="14015d794f">14015d7</a>)</li>
<li><strong>deps:</strong> update all non-major dependencies (<a
href="https://redirect.github.com/vitejs/vite/issues/20732">#20732</a>)
(<a
href="122bfbabeb">122bfba</a>)</li>
<li>upgrade sirv to 3.0.2 (<a
href="https://redirect.github.com/vitejs/vite/issues/20735">#20735</a>)
(<a
href="09f2b52e8d">09f2b52</a>)</li>
</ul>
<h2><!-- raw HTML omitted --><a
href="https://github.com/vitejs/vite/compare/v7.1.3...v7.1.4">7.1.4</a>
(2025-09-01)<!-- raw HTML omitted --></h2>
<h3>Bug Fixes</h3>
<ul>
<li>add missing awaits (<a
href="https://redirect.github.com/vitejs/vite/issues/20697">#20697</a>)
(<a
href="79d10ed634">79d10ed</a>)</li>
<li><strong>deps:</strong> update all non-major dependencies (<a
href="https://redirect.github.com/vitejs/vite/issues/20676">#20676</a>)
(<a
href="5a274b29df">5a274b2</a>)</li>
<li><strong>deps:</strong> update all non-major dependencies (<a
href="https://redirect.github.com/vitejs/vite/issues/20709">#20709</a>)
(<a
href="0401feba17">0401feb</a>)</li>
<li>pass rollup watch options when building in watch mode (<a
href="https://redirect.github.com/vitejs/vite/issues/20674">#20674</a>)
(<a
href="f367453ca2">f367453</a>)</li>
</ul>
<h3>Miscellaneous Chores</h3>
<ul>
<li>remove unused constants entry from rolldown.config.ts (<a
href="https://redirect.github.com/vitejs/vite/issues/20710">#20710</a>)
(<a
href="537fcf9186">537fcf9</a>)</li>
</ul>
<h3>Code Refactoring</h3>
<ul>
<li>remove unnecessary <code>minify</code> parameter from
<code>finalizeCss</code> (<a
href="https://redirect.github.com/vitejs/vite/issues/20701">#20701</a>)
(<a
href="8099582e53">8099582</a>)</li>
</ul>
<h2><!-- raw HTML omitted --><a
href="https://github.com/vitejs/vite/compare/v7.1.2...v7.1.3">7.1.3</a>
(2025-08-19)<!-- raw HTML omitted --></h2>
<h3>Features</h3>
<ul>
<li><strong>cli:</strong> add Node.js version warning for unsupported
versions (<a
href="https://redirect.github.com/vitejs/vite/issues/20638">#20638</a>)
(<a
href="a1be1bf090">a1be1bf</a>)</li>
<li>generate code frame for parse errors thrown by terser (<a
href="https://redirect.github.com/vitejs/vite/issues/20642">#20642</a>)
(<a
href="a9ba0174a5">a9ba017</a>)</li>
<li>support long lines in <code>generateCodeFrame</code> (<a
href="https://redirect.github.com/vitejs/vite/issues/20640">#20640</a>)
(<a
href="1559577317">1559577</a>)</li>
</ul>
<h3>Bug Fixes</h3>
<ul>
<li><strong>deps:</strong> update all non-major dependencies (<a
href="https://redirect.github.com/vitejs/vite/issues/20634">#20634</a>)
(<a
href="4851cab3ba">4851cab</a>)</li>
<li><strong>optimizer:</strong> incorrect incompatible error (<a
href="https://redirect.github.com/vitejs/vite/issues/20439">#20439</a>)
(<a
href="446fe83033">446fe83</a>)</li>
<li>support multiline new URL(..., import.meta.url) expressions (<a
href="https://redirect.github.com/vitejs/vite/issues/20644">#20644</a>)
(<a
href="9ccf142764">9ccf142</a>)</li>
</ul>
<h3>Performance Improvements</h3>
<ul>
<li><strong>cli:</strong> dynamically import <code>resolveConfig</code>
(<a
href="https://redirect.github.com/vitejs/vite/issues/20646">#20646</a>)
(<a
href="f691f57e46">f691f57</a>)</li>
</ul>
<h3>Miscellaneous Chores</h3>
<ul>
<li><strong>deps:</strong> update rolldown-related dependencies (<a
href="https://redirect.github.com/vitejs/vite/issues/20633">#20633</a>)
(<a
href="98b92e8c4b">98b92e8</a>)</li>
</ul>
<h3>Code Refactoring</h3>
<ul>
<li>replace startsWith with strict equality (<a
href="https://redirect.github.com/vitejs/vite/issues/20603">#20603</a>)
(<a
href="42816dee0e">42816de</a>)</li>
<li>use <code>import</code> in worker threads (<a
href="https://redirect.github.com/vitejs/vite/issues/20641">#20641</a>)
(<a
href="530687a344">530687a</a>)</li>
</ul>
<h3>Tests</h3>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="564754061e"><code>5647540</code></a>
release: v7.1.5</li>
<li><a
href="09f2b52e8d"><code>09f2b52</code></a>
fix: upgrade sirv to 3.0.2 (<a
href="https://github.com/vitejs/vite/tree/HEAD/packages/vite/issues/20735">#20735</a>)</li>
<li><a
href="14015d794f"><code>14015d7</code></a>
fix: apply <code>fs.strict</code> check to HTML files (<a
href="https://github.com/vitejs/vite/tree/HEAD/packages/vite/issues/20736">#20736</a>)</li>
<li><a
href="122bfbabeb"><code>122bfba</code></a>
fix(deps): update all non-major dependencies (<a
href="https://github.com/vitejs/vite/tree/HEAD/packages/vite/issues/20732">#20732</a>)</li>
<li><a
href="bcc31449c0"><code>bcc3144</code></a>
release: v7.1.4</li>
<li><a
href="0401feba17"><code>0401feb</code></a>
fix(deps): update all non-major dependencies (<a
href="https://github.com/vitejs/vite/tree/HEAD/packages/vite/issues/20709">#20709</a>)</li>
<li><a
href="537fcf9186"><code>537fcf9</code></a>
chore: remove unused constants entry from rolldown.config.ts (<a
href="https://github.com/vitejs/vite/tree/HEAD/packages/vite/issues/20710">#20710</a>)</li>
<li><a
href="79d10ed634"><code>79d10ed</code></a>
fix: add missing awaits (<a
href="https://github.com/vitejs/vite/tree/HEAD/packages/vite/issues/20697">#20697</a>)</li>
<li><a
href="8099582e53"><code>8099582</code></a>
refactor: remove unnecessary <code>minify</code> parameter from
<code>finalizeCss</code> (<a
href="https://github.com/vitejs/vite/tree/HEAD/packages/vite/issues/20701">#20701</a>)</li>
<li><a
href="f367453ca2"><code>f367453</code></a>
fix: pass rollup watch options when building in watch mode (<a
href="https://github.com/vitejs/vite/tree/HEAD/packages/vite/issues/20674">#20674</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/vitejs/vite/commits/v7.1.5/packages/vite">compare
view</a></li>
</ul>
</details>
<details>
<summary>Maintainer changes</summary>
<p>This version was pushed to npm by [GitHub Actions](<a
href="https://www.npmjs.com/~GitHub">https://www.npmjs.com/~GitHub</a>
Actions), a new releaser for vite since your current version.</p>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=vite&package-manager=npm_and_yarn&previous-version=7.0.4&new-version=7.1.5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/VictoriaMetrics/VictoriaMetrics/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-16 15:56:42 +03:00
Aliaksandr Valialkin
3cfeae7f1a app/vmauth: do not log requests canceled by the client, since this is an expected condition
See https://github.com/VictoriaMetrics/VictoriaLogs/issues/667#issuecomment-3297270128
2025-09-16 11:59:06 +02:00
Max Kotliar
32da04725b docs: use canonical link 2025-09-16 12:42:59 +03:00
Aliaksandr Valialkin
8ce4636bc0 app/vmauth: flush data chunks from backends to clients as soon as possible without bufferring them at vmauth side
This allows the proper live tailing of responses from backends
such as VictoriaLogs live tailing - https://docs.victoriametrics.com/victorialogs/querying/#live-tailing

See https://github.com/VictoriaMetrics/VictoriaLogs/issues/667

Thanks to @func25 for the initial pull request at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9723
2025-09-16 11:10:38 +02:00
Andrii Chubatiuk
6167ce655e lib/timerpool: removed unneeded code, unified package usage (#9735)
### Describe Your Changes

after golang 1.23 it's enough just to stop timer, no need to drain a
channel

related issue
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9721, but this
is not a fix for it

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
- [ ] My change adheres to [VictoriaMetrics development
goals](https://docs.victoriametrics.com/victoriametrics/goals/).
2025-09-16 09:55:33 +02:00
Max Kotliar
f1e294aa2b docs: use canonical links 2025-09-16 10:19:20 +03:00
Max Kotliar
b72bf6961d docs: bump latest version in docs 2025-09-15 14:21:07 +03:00
Max Kotliar
2b880fe7db deployment/docker: bump version 2025-09-15 14:08:19 +03:00
Max Kotliar
9898743fbd docs: bump last LTS versions 2025-09-15 12:28:56 +03:00
Max Kotliar
ca372168ae docs/CHANGELOG.md: update changelog with LTS release notes 2025-09-15 12:24:34 +03:00
163 changed files with 3242 additions and 3941 deletions

View File

@@ -31,7 +31,7 @@ type Group struct {
// EvalDelay will adjust the `time` parameter of rule evaluation requests to compensate intentional query delay from datasource.
// see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5155
EvalDelay *promutil.Duration `yaml:"eval_delay,omitempty"`
Limit int `yaml:"limit,omitempty"`
Limit *int `yaml:"limit,omitempty"`
Rules []Rule `yaml:"rules"`
Concurrency int `yaml:"concurrency"`
// Labels is a set of label value pairs, that will be added to every rule.
@@ -91,8 +91,8 @@ func (g *Group) Validate(validateTplFn ValidateTplFn, validateExpressions bool)
if g.EvalOffset != nil && g.EvalDelay != nil {
return fmt.Errorf("eval_offset cannot be used with eval_delay")
}
if g.Limit < 0 {
return fmt.Errorf("invalid limit %d, shouldn't be less than 0", g.Limit)
if g.Limit != nil && *g.Limit < 0 {
return fmt.Errorf("invalid limit %d, shouldn't be less than 0", *g.Limit)
}
if g.Concurrency < 0 {
return fmt.Errorf("invalid concurrency %d, shouldn't be less than 0", g.Concurrency)

View File

@@ -181,9 +181,10 @@ func TestGroupValidate_Failure(t *testing.T) {
EvalOffset: promutil.NewDuration(2 * time.Minute),
}, false, "eval_offset should be smaller than interval")
limit := -1
f(&Group{
Name: "wrong limit",
Limit: -1,
Limit: &limit,
}, false, "invalid limit")
f(&Group{

View File

@@ -389,7 +389,7 @@ func (ar *AlertingRule) execRange(ctx context.Context, start, end time.Time) ([]
return []datasource.Metric{{Timestamps: []int64{0}, Values: []float64{math.NaN()}}}, nil
}
for _, s := range res.Data {
ls, err := ar.expandLabelTemplates(s)
ls, err := ar.expandLabelTemplates(s, qFn)
if err != nil {
return nil, err
}
@@ -482,7 +482,7 @@ func (ar *AlertingRule) exec(ctx context.Context, ts time.Time, limit int) ([]pr
expandedLabels := make([]*labelSet, len(res.Data))
expandedAnnotations := make([]map[string]string, len(res.Data))
for i, m := range res.Data {
ls, err := ar.expandLabelTemplates(m)
ls, err := ar.expandLabelTemplates(m, qFn)
if err != nil {
curState.Err = err
return nil, curState.Err
@@ -604,10 +604,7 @@ func (ar *AlertingRule) exec(ctx context.Context, ts time.Time, limit int) ([]pr
return append(tss, ar.toTimeSeries(ts.Unix())...), nil
}
func (ar *AlertingRule) expandLabelTemplates(m datasource.Metric) (*labelSet, error) {
qFn := func(_ string) ([]datasource.Metric, error) {
return nil, fmt.Errorf("`query` template isn't supported in rule label")
}
func (ar *AlertingRule) expandLabelTemplates(m datasource.Metric, qFn templates.QueryFn) (*labelSet, error) {
ls, err := ar.toLabels(m, qFn)
if err != nil {
return nil, fmt.Errorf("failed to expand label templates: %s", err)

View File

@@ -10,6 +10,7 @@ import (
"strings"
"sync"
"testing"
"testing/synctest"
"time"
"github.com/VictoriaMetrics/metrics"
@@ -1429,3 +1430,142 @@ func TestAlertingRuleExec_Partial(t *testing.T) {
t.Fatalf("unexpected error: %s", err)
}
}
func TestAlertingRule_QueryTemplateInLabels(t *testing.T) {
fq := &datasource.FakeQuerier{}
fakeGroup := Group{
Name: "TestQueryTemplateInLabels",
}
ar := &AlertingRule{
Name: "test_alert",
Labels: map[string]string{
"suppress_for_mass_alert": `{{ if (printf "ALERTS{alertname='SomeAlert', alertstate='firing', device='%s'} == 1" $labels.device | query) }}true{{ else }}false{{ end }}`,
},
Annotations: map[string]string{
"summary": "Test alert with query template in labels",
},
alerts: make(map[uint64]*notifier.Alert),
}
ar.GroupID = fakeGroup.GetID()
ar.q = fq
ar.state = &ruleState{
entries: make([]StateEntry, 10),
}
// Add a metric that should trigger the alert
fq.Add(metricWithValueAndLabels(t, 1, "device", "sda1"))
ts := time.Now()
_, err := ar.exec(context.TODO(), ts, 0)
if err != nil {
t.Fatalf("unexpected error with query template in labels: %s", err)
}
// Verify that the alert was created and the query template was executed
if len(ar.alerts) != 1 {
t.Fatalf("expected 1 alert, got %d", len(ar.alerts))
}
alert := ar.GetAlerts()[0]
suppressLabel, exists := alert.Labels["suppress_for_mass_alert"]
if !exists {
t.Fatalf("expected 'suppress_for_mass_alert' label to exist")
}
// The query template should have been executed (even if it returns false due to mock data)
if suppressLabel != "true" && suppressLabel != "false" {
t.Fatalf("expected 'suppress_for_mass_alert' label to be 'true' or 'false', got '%s'", suppressLabel)
}
}
// TestAlertingRule_ActiveAtPreservedInAnnotations ensures that the fix for
// https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9543 is preserved
// while allowing query templates in labels (https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9783)
func TestAlertingRule_ActiveAtPreservedInAnnotations(t *testing.T) {
// wrap into synctest because of time manipulations
synctest.Test(t, func(t *testing.T) {
fq := &datasource.FakeQuerier{}
ar := &AlertingRule{
Name: "TestActiveAtPreservation",
Labels: map[string]string{
"test_query_in_label": `{{ "static_value" }}`,
},
Annotations: map[string]string{
"description": "Alert active since {{ $activeAt }}",
},
alerts: make(map[uint64]*notifier.Alert),
q: fq,
state: &ruleState{
entries: make([]StateEntry, 10),
},
}
// Mock query result - return empty result to make suppress_for_mass_alert = false
// (no need to add anything to fq for empty result)
// Add a metric that should trigger the alert
fq.Add(metricWithValueAndLabels(t, 1, "instance", "server1"))
// First execution - creates new alert
ts1 := time.Now()
_, err := ar.exec(context.TODO(), ts1, 0)
if err != nil {
t.Fatalf("unexpected error on first exec: %s", err)
}
if len(ar.alerts) != 1 {
t.Fatalf("expected 1 alert, got %d", len(ar.alerts))
}
firstAlert := ar.GetAlerts()[0]
// Verify first execution: activeAt should be ts1 and annotation should reflect it
if !firstAlert.ActiveAt.Equal(ts1) {
t.Fatalf("expected activeAt to be %v, got %v", ts1, firstAlert.ActiveAt)
}
// Extract time from annotation (format will be like "Alert active since 2025-09-30 08:55:13.638551611 -0400 EDT m=+0.002928464")
expectedTimeStr := ts1.Format("2006-01-02 15:04:05")
if !strings.Contains(firstAlert.Annotations["description"], expectedTimeStr) {
t.Fatalf("first exec annotation should contain time %s, got: %s", expectedTimeStr, firstAlert.Annotations["description"])
}
// Second execution - should preserve activeAt in annotation
// Ensure different timestamp with different seconds
// sleep is non-blocking thanks to synctest
time.Sleep(2 * time.Second)
ts2 := time.Now()
_, err = ar.exec(context.TODO(), ts2, 0)
if err != nil {
t.Fatalf("unexpected error on second exec: %s", err)
}
// Get the alert again (should be the same alert)
if len(ar.alerts) != 1 {
t.Fatalf("expected 1 alert, got %d", len(ar.alerts))
}
secondAlert := ar.GetAlerts()[0]
// Critical test: activeAt should still be ts1, not ts2
if !secondAlert.ActiveAt.Equal(ts1) {
t.Fatalf("activeAt should be preserved as %v, but got %v", ts1, secondAlert.ActiveAt)
}
// Critical test: annotation should still contain ts1 time, not ts2
if !strings.Contains(secondAlert.Annotations["description"], expectedTimeStr) {
t.Fatalf("second exec annotation should still contain original time %s, got: %s", expectedTimeStr, secondAlert.Annotations["description"])
}
// Additional verification: annotation should NOT contain ts2 time
ts2TimeStr := ts2.Format("2006-01-02 15:04:05")
if strings.Contains(secondAlert.Annotations["description"], ts2TimeStr) {
t.Fatalf("annotation should NOT contain new eval time %s, got: %s", ts2TimeStr, secondAlert.Annotations["description"])
}
// Verify query template in labels still works (this would fail if query templates were broken)
if firstAlert.Labels["test_query_in_label"] != "static_value" {
t.Fatalf("expected test_query_in_label=static_value, got %s", firstAlert.Labels["test_query_in_label"])
}
})
}

View File

@@ -24,6 +24,10 @@ import (
)
var (
ruleResultsLimit = flag.Int("rule.resultsLimit", 0, "Limits the number of alerts or recording results a single rule can produce. "+
"Can be overridden by the limit option under group if specified. "+
"If exceeded, the rule will be marked with an error and all its results will be discarded. "+
"0 means no limit.")
ruleUpdateEntriesLimit = flag.Int("rule.updateEntriesLimit", 20, "Defines the max number of rule's state updates stored in-memory. "+
"Rule's updates are available on rule's Details page and are used for debugging purposes. The number of stored updates can be overridden per rule via update_entries_limit param.")
resendDelay = flag.Duration("rule.resendDelay", 0, "MiniMum amount of time to wait before resending an alert to notifier.")
@@ -111,7 +115,6 @@ func NewGroup(cfg config.Group, qb datasource.QuerierBuilder, defaultInterval ti
Name: cfg.Name,
File: cfg.File,
Interval: cfg.Interval.Duration(),
Limit: cfg.Limit,
Concurrency: cfg.Concurrency,
checksum: cfg.Checksum,
Params: cfg.Params,
@@ -128,6 +131,11 @@ func NewGroup(cfg config.Group, qb datasource.QuerierBuilder, defaultInterval ti
if g.Interval == 0 {
g.Interval = defaultInterval
}
if cfg.Limit != nil {
g.Limit = *cfg.Limit
} else {
g.Limit = *ruleResultsLimit
}
if g.Concurrency < 1 {
g.Concurrency = 1
}

View File

@@ -372,20 +372,54 @@ func tryProcessingRequest(w http.ResponseWriter, r *http.Request, targetURL *url
updateHeadersByConfig(w.Header(), hc.ResponseHeaders)
w.WriteHeader(res.StatusCode)
copyBuf := copyBufPool.Get()
copyBuf.B = bytesutil.ResizeNoCopyNoOverallocate(copyBuf.B, 16*1024)
_, err = io.CopyBuffer(w, res.Body, copyBuf.B)
copyBufPool.Put(copyBuf)
err = copyStreamToClient(w, res.Body)
_ = res.Body.Close()
if err != nil && !netutil.IsTrivialNetworkError(err) {
if err != nil && !netutil.IsTrivialNetworkError(err) && !errors.Is(err, context.Canceled) {
remoteAddr := httpserver.GetQuotedRemoteAddr(r)
requestURI := httpserver.GetRequestURI(r)
logger.Warnf("remoteAddr: %s; requestURI: %s; error when proxying response body from %s: %s", remoteAddr, requestURI, targetURL, err)
return true, false
}
return true, false
}
func copyStreamToClient(client io.Writer, backend io.Reader) error {
copyBuf := copyBufPool.Get()
copyBuf.B = bytesutil.ResizeNoCopyNoOverallocate(copyBuf.B, 16*1024)
defer copyBufPool.Put(copyBuf)
buf := copyBuf.B
flusher, ok := client.(http.Flusher)
if !ok {
logger.Panicf("BUG: client must implement net/http.Flusher interface; got %T", client)
}
for {
n, backendErr := backend.Read(buf)
if n > 0 {
data := buf[:n]
n, clientErr := client.Write(data)
if clientErr != nil {
return fmt.Errorf("cannot write data to client: %w", clientErr)
}
if n != len(data) {
logger.Panicf("BUG: unexpected number of bytes written returned by client.Write; got %d; want %d", n, len(data))
}
// Flush the read data from the backend to the client as fast as possible
// in order to reduce delays for data propagation.
// See https://github.com/VictoriaMetrics/VictoriaLogs/issues/667
flusher.Flush()
}
if backendErr != nil {
if backendErr == io.EOF {
return nil
}
return fmt.Errorf("cannot read data from backend: %w", backendErr)
}
}
}
var copyBufPool bytesutil.ByteBufferPool
func copyHeader(dst, src http.Header) {

View File

@@ -514,6 +514,11 @@ func (w *fakeResponseWriter) getResponse() string {
return w.bb.String()
}
// Flush implements net/http.Flusher
func (w *fakeResponseWriter) Flush() {
// Nothing to do.
}
func (w *fakeResponseWriter) Header() http.Header {
if w.h == nil {
w.h = http.Header{}

View File

@@ -115,7 +115,7 @@ func main() {
if err != nil {
logger.Fatalf("cannot create backup: %s", err)
}
pushmetrics.Stop()
pushmetrics.StopAndPush()
startTime := time.Now()
logger.Infof("gracefully shutting down http server for metrics at %q", listenAddrs)

View File

@@ -68,7 +68,7 @@ func main() {
if err := a.Run(ctx); err != nil {
logger.Fatalf("cannot restore from backup: %s", err)
}
pushmetrics.Stop()
pushmetrics.StopAndPush()
srcFS.MustStop()
dstFS.MustStop()

View File

@@ -197,13 +197,13 @@ func newNextSeriesForSearchQuery(ec *evalConfig, sq *storage.SearchQuery, expr g
}
s.summarize(aggrAvg, ec.startTime, ec.endTime, ec.storageStep, 0)
t := timerpool.Get(30 * time.Second)
defer timerpool.Put(t)
select {
case seriesCh <- s:
case <-t.C:
logger.Errorf("resource leak when processing the %s (full query: %s); please report this error to VictoriaMetrics developers",
expr.AppendString(nil), ec.originalQuery)
}
timerpool.Put(t)
return nil
})
close(seriesCh)

View File

@@ -1150,15 +1150,23 @@ func evalInstantRollup(qt *querytracer.Tracer, ec *EvalConfig, funcName string,
}
qt.Printf("optimized calculation for instant rollup avg_over_time(m[d]) as (sum_over_time(m[d]) / count_over_time(m[d]))")
fe := expr.(*metricsql.FuncExpr)
feSum := *fe
feSum.Name = "sum_over_time"
feCount := *fe
feCount.Name = "count_over_time"
// copy RollupExpr to drop possible offset,
// see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9762
newArg := copyRollupExpr(fe.Args[0].(*metricsql.RollupExpr))
newArg.Offset = nil
be := &metricsql.BinaryOpExpr{
Op: "/",
KeepMetricNames: fe.KeepMetricNames,
Left: &feSum,
Right: &feCount,
Left: &metricsql.FuncExpr{
Name: "sum_over_time",
Args: []metricsql.Expr{newArg},
KeepMetricNames: fe.KeepMetricNames,
},
Right: &metricsql.FuncExpr{
Name: "count_over_time",
Args: []metricsql.Expr{newArg},
KeepMetricNames: fe.KeepMetricNames,
},
}
return evalExpr(qt, ec, be)
case "rate":
@@ -1172,8 +1180,12 @@ func evalInstantRollup(qt *querytracer.Tracer, ec *EvalConfig, funcName string,
fe := afe.Args[0].(*metricsql.FuncExpr)
feIncrease := *fe
feIncrease.Name = "increase"
re := fe.Args[0].(*metricsql.RollupExpr)
d := re.Window.Duration(ec.Step)
// copy RollupExpr to drop possible offset,
// see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9762
newArg := copyRollupExpr(fe.Args[0].(*metricsql.RollupExpr))
newArg.Offset = nil
feIncrease.Args = []metricsql.Expr{newArg}
d := newArg.Window.Duration(ec.Step)
if d == 0 {
d = ec.Step
}
@@ -1193,8 +1205,12 @@ func evalInstantRollup(qt *querytracer.Tracer, ec *EvalConfig, funcName string,
fe := expr.(*metricsql.FuncExpr)
feIncrease := *fe
feIncrease.Name = "increase"
re := fe.Args[0].(*metricsql.RollupExpr)
d := re.Window.Duration(ec.Step)
// copy RollupExpr to drop possible offset,
// see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9762
newArg := copyRollupExpr(fe.Args[0].(*metricsql.RollupExpr))
newArg.Offset = nil
feIncrease.Args = []metricsql.Expr{newArg}
d := newArg.Window.Duration(ec.Step)
if d == 0 {
d = ec.Step
}
@@ -1999,3 +2015,23 @@ func dropStaleNaNs(funcName string, values []float64, timestamps []int64) ([]flo
}
return dstValues, dstTimestamps
}
func copyRollupExpr(re *metricsql.RollupExpr) *metricsql.RollupExpr {
var newRe metricsql.RollupExpr
newRe.Expr = re.Expr
newRe.InheritStep = re.InheritStep
newRe.At = re.At
if re.Window != nil {
newRe.Window = &metricsql.DurationExpr{}
*newRe.Window = *re.Window
}
if re.Offset != nil {
newRe.Offset = &metricsql.DurationExpr{}
*newRe.Offset = *re.Offset
}
if re.Step != nil {
newRe.Step = &metricsql.DurationExpr{}
*newRe.Step = *re.Step
}
return &newRe
}

View File

@@ -123,7 +123,7 @@ The list of MetricsQL features on top of PromQL:
* `if` binary operator. `q1 if q2` removes values from `q1` for missing values from `q2`.
* `ifnot` binary operator. `q1 ifnot q2` removes values from `q1` for existing values from `q2`.
* `WITH` templates. This feature simplifies writing and managing complex queries.
Go to [WITH templates playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/expand-with-exprs) and try it.
Go to [WITH templates playground](https://play.victoriametrics.com/select/0/prometheus/graph/#/expand-with-exprs) and try it.
* String literals may be concatenated. This is useful with `WITH` templates:
`WITH (commonPrefix="long_metric_prefix_") {__name__=commonPrefix+"suffix1"} / {__name__=commonPrefix+"suffix2"}`.
* `keep_metric_names` modifier can be applied to all the [rollup functions](#rollup-functions), [transform functions](#transform-functions)

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -6,6 +6,7 @@
<link rel="apple-touch-icon" href="./favicon.svg"/>
<link rel="mask-icon" href="./favicon.svg" color="#000000">
<meta name="robots" content="noindex">
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=5"/>
<meta name="theme-color" content="#000000"/>
<meta name="description" content="Explore and troubleshoot your VictoriaMetrics data"/>
@@ -36,10 +37,10 @@
<meta property="og:title" content="UI for VictoriaMetrics">
<meta property="og:url" content="https://victoriametrics.com/">
<meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data">
<script type="module" crossorigin src="./assets/index-DQcPcJrn.js"></script>
<script type="module" crossorigin src="./assets/index-DfyHQqg-.js"></script>
<link rel="modulepreload" crossorigin href="./assets/vendor-DY9kCvzk.js">
<link rel="stylesheet" crossorigin href="./assets/vendor-D1GxaB_c.css">
<link rel="stylesheet" crossorigin href="./assets/index-dWApsAEM.css">
<link rel="stylesheet" crossorigin href="./assets/index-Bhu7ieUw.css">
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>

View File

@@ -1,4 +1,4 @@
FROM golang:1.25.0 AS build-web-stage
FROM golang:1.25.1 AS build-web-stage
COPY build /build
WORKDIR /build

View File

@@ -6,6 +6,7 @@
<link rel="apple-touch-icon" href="/favicon.svg"/>
<link rel="mask-icon" href="/favicon.svg" color="#000000">
<meta name="robots" content="noindex">
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=5"/>
<meta name="theme-color" content="#000000"/>
<meta name="description" content="Explore and troubleshoot your VictoriaMetrics data"/>

View File

@@ -17,7 +17,7 @@
"react-input-mask": "^2.0.4",
"react-router-dom": "^7.6.3",
"uplot": "^1.6.32",
"vite": "^7.0.4",
"vite": "^7.1.5",
"web-vitals": "^5.0.3"
},
"devDependencies": {
@@ -7321,13 +7321,13 @@
"license": "MIT"
},
"node_modules/tinyglobby": {
"version": "0.2.14",
"resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.14.tgz",
"integrity": "sha512-tX5e7OM1HnYr2+a2C/4V0htOcSQcoSTH9KgJnVvNm5zm/cyEWKJ7j7YutsH9CxMdtOkkLFy2AHrMci9IM8IPZQ==",
"version": "0.2.15",
"resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.15.tgz",
"integrity": "sha512-j2Zq4NyQYG5XMST4cbs02Ak8iJUdxRM0XI5QyxXuZOzKOINmWurp3smXu3y5wDcJrptwpSjgXHzIQxR0omXljQ==",
"license": "MIT",
"dependencies": {
"fdir": "^6.4.4",
"picomatch": "^4.0.2"
"fdir": "^6.5.0",
"picomatch": "^4.0.3"
},
"engines": {
"node": ">=12.0.0"
@@ -7337,10 +7337,13 @@
}
},
"node_modules/tinyglobby/node_modules/fdir": {
"version": "6.4.6",
"resolved": "https://registry.npmjs.org/fdir/-/fdir-6.4.6.tgz",
"integrity": "sha512-hiFoqpyZcfNm1yc4u8oWCf9A2c4D3QjCrks3zmoVKVxpQRzmPNar1hUJcBG2RQHvEVGDN+Jm81ZheVLAQMK6+w==",
"version": "6.5.0",
"resolved": "https://registry.npmjs.org/fdir/-/fdir-6.5.0.tgz",
"integrity": "sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==",
"license": "MIT",
"engines": {
"node": ">=12.0.0"
},
"peerDependencies": {
"picomatch": "^3 || ^4"
},
@@ -7351,9 +7354,9 @@
}
},
"node_modules/tinyglobby/node_modules/picomatch": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.2.tgz",
"integrity": "sha512-M7BAV6Rlcy5u+m6oPhAPFgJTzAioX/6B0DxyvDlo9l8+T3nLKbrczg2WLUyzd45L8RqfUMyGPzekbMvX2Ldkwg==",
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"license": "MIT",
"engines": {
"node": ">=12"
@@ -7657,17 +7660,17 @@
"license": "MIT"
},
"node_modules/vite": {
"version": "7.0.4",
"resolved": "https://registry.npmjs.org/vite/-/vite-7.0.4.tgz",
"integrity": "sha512-SkaSguuS7nnmV7mfJ8l81JGBFV7Gvzp8IzgE8A8t23+AxuNX61Q5H1Tpz5efduSN7NHC8nQXD3sKQKZAu5mNEA==",
"version": "7.1.5",
"resolved": "https://registry.npmjs.org/vite/-/vite-7.1.5.tgz",
"integrity": "sha512-4cKBO9wR75r0BeIWWWId9XK9Lj6La5X846Zw9dFfzMRw38IlTk2iCcUt6hsyiDRcPidc55ZParFYDXi0nXOeLQ==",
"license": "MIT",
"dependencies": {
"esbuild": "^0.25.0",
"fdir": "^6.4.6",
"picomatch": "^4.0.2",
"fdir": "^6.5.0",
"picomatch": "^4.0.3",
"postcss": "^8.5.6",
"rollup": "^4.40.0",
"tinyglobby": "^0.2.14"
"rollup": "^4.43.0",
"tinyglobby": "^0.2.15"
},
"bin": {
"vite": "bin/vite.js"
@@ -7772,10 +7775,13 @@
}
},
"node_modules/vite/node_modules/fdir": {
"version": "6.4.6",
"resolved": "https://registry.npmjs.org/fdir/-/fdir-6.4.6.tgz",
"integrity": "sha512-hiFoqpyZcfNm1yc4u8oWCf9A2c4D3QjCrks3zmoVKVxpQRzmPNar1hUJcBG2RQHvEVGDN+Jm81ZheVLAQMK6+w==",
"version": "6.5.0",
"resolved": "https://registry.npmjs.org/fdir/-/fdir-6.5.0.tgz",
"integrity": "sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==",
"license": "MIT",
"engines": {
"node": ">=12.0.0"
},
"peerDependencies": {
"picomatch": "^3 || ^4"
},
@@ -7786,9 +7792,9 @@
}
},
"node_modules/vite/node_modules/picomatch": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.2.tgz",
"integrity": "sha512-M7BAV6Rlcy5u+m6oPhAPFgJTzAioX/6B0DxyvDlo9l8+T3nLKbrczg2WLUyzd45L8RqfUMyGPzekbMvX2Ldkwg==",
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"license": "MIT",
"engines": {
"node": ">=12"

View File

@@ -29,7 +29,7 @@
"react-input-mask": "^2.0.4",
"react-router-dom": "^7.6.3",
"uplot": "^1.6.32",
"vite": "^7.0.4",
"vite": "^7.1.5",
"web-vitals": "^5.0.3"
},
"devDependencies": {

View File

@@ -123,7 +123,7 @@ The list of MetricsQL features on top of PromQL:
* `if` binary operator. `q1 if q2` removes values from `q1` for missing values from `q2`.
* `ifnot` binary operator. `q1 ifnot q2` removes values from `q1` for existing values from `q2`.
* `WITH` templates. This feature simplifies writing and managing complex queries.
Go to [WITH templates playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/expand-with-exprs) and try it.
Go to [WITH templates playground](https://play.victoriametrics.com/select/0/prometheus/graph/#/expand-with-exprs) and try it.
* String literals may be concatenated. This is useful with `WITH` templates:
`WITH (commonPrefix="long_metric_prefix_") {__name__=commonPrefix+"suffix1"} / {__name__=commonPrefix+"suffix2"}`.
* `keep_metric_names` modifier can be applied to all the [rollup functions](#rollup-functions), [transform functions](#transform-functions)

View File

@@ -13,6 +13,8 @@ import Button from "../../Button/Button";
interface DatePickerProps {
date: Date | Dayjs
format?: string
minDate?: Date | Dayjs
maxDate?: Date | Dayjs
onChange: (date: string) => void
}
@@ -24,6 +26,8 @@ enum CalendarTypeView {
const Calendar: FC<DatePickerProps> = ({
date,
minDate,
maxDate,
format = DATE_TIME_FORMAT,
onChange,
}) => {
@@ -34,6 +38,8 @@ const Calendar: FC<DatePickerProps> = ({
const today = dayjs.tz();
const viewDateIsToday = today.format(DATE_FORMAT) === viewDate.format(DATE_FORMAT);
const { isMobile } = useDeviceDetect();
const min = minDate ? dayjs(minDate) : undefined;
const max = maxDate ? dayjs(maxDate) : undefined;
const toggleDisplayYears = () => {
setViewType(prev => prev === CalendarTypeView.years ? CalendarTypeView.days : CalendarTypeView.years);
@@ -75,9 +81,13 @@ const Calendar: FC<DatePickerProps> = ({
onChangeViewDate={handleChangeViewDate}
toggleDisplayYears={toggleDisplayYears}
showArrowNav={viewType === CalendarTypeView.days}
hasPrev={viewType === CalendarTypeView.days && (!min || viewDate.startOf("month").isAfter(min))}
hasNext={viewType === CalendarTypeView.days && (!max || viewDate.endOf("month").isBefore(max))}
/>
{viewType === CalendarTypeView.days && (
<CalendarBody
minDate={min}
maxDate={max}
viewDate={viewDate}
selectDate={selectDate}
onChangeSelectDate={handleChangeSelectDate}
@@ -85,12 +95,16 @@ const Calendar: FC<DatePickerProps> = ({
)}
{viewType === CalendarTypeView.years && (
<YearsList
minDate={min}
maxDate={max}
viewDate={viewDate}
onChangeViewDate={handleChangeViewDate}
/>
)}
{viewType === CalendarTypeView.months && (
<MonthsList
minDate={min}
maxDate={max}
selectDate={selectDate}
viewDate={viewDate}
onChangeViewDate={handleChangeViewDate}

View File

@@ -4,6 +4,8 @@ import classNames from "classnames";
import Tooltip from "../../../Tooltip/Tooltip";
interface CalendarBodyProps {
minDate?: Dayjs
maxDate?: Dayjs
viewDate: Dayjs
selectDate: Dayjs
onChangeSelectDate: (date: Dayjs) => void
@@ -11,7 +13,7 @@ interface CalendarBodyProps {
const weekday = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"];
const CalendarBody: FC<CalendarBodyProps> = ({ viewDate: date, selectDate, onChangeSelectDate }) => {
const CalendarBody: FC<CalendarBodyProps> = ({ minDate, maxDate, viewDate: date, selectDate, onChangeSelectDate }) => {
const format = "YYYY-MM-DD";
const today = dayjs.tz();
const viewDate = dayjs(date.format(format));
@@ -44,21 +46,25 @@ const CalendarBody: FC<CalendarBodyProps> = ({ viewDate: date, selectDate, onCha
</Tooltip>
))}
{days.map((d, i) => (
<div
className={classNames({
"vm-calendar-body-cell": true,
"vm-calendar-body-cell_day": true,
"vm-calendar-body-cell_day_empty": !d,
"vm-calendar-body-cell_day_active": (d && d.format(format)) === selectDate.format(format),
"vm-calendar-body-cell_day_today": (d && d.format(format)) === today.format(format)
})}
key={d ? d.format(format) : i}
onClick={createHandlerSelectDate(d)}
>
{d && d.format("D")}
</div>
))}
{days.map((d, i) => {
const isDisabled = d && ((minDate && d.isBefore(minDate)) || (maxDate && d.isAfter(maxDate)));
return (
<div
className={classNames({
"vm-calendar-body-cell": true,
"vm-calendar-body-cell_day": true,
"vm-calendar-body-cell_day_empty": !d,
"vm-calendar-body-cell_day_active": (d && d.format(format)) === selectDate.format(format),
"vm-calendar-body-cell_day_today": (d && d.format(format)) === today.format(format),
"vm-calendar-body-cell_day_disabled": isDisabled,
})}
key={d ? d.format(format) : i}
onClick={createHandlerSelectDate(d)}
>
{d && d.format("D")}
</div>
);
})}
</div>
);
};

View File

@@ -1,15 +1,18 @@
import { FC } from "preact/compat";
import { Dayjs } from "dayjs";
import { ArrowDownIcon, ArrowDropDownIcon } from "../../../Icons";
import classNames from "classnames";
interface CalendarHeaderProps {
viewDate: Dayjs
onChangeViewDate: (date: Dayjs) => void
showArrowNav: boolean
toggleDisplayYears: () => void
hasNext: boolean
hasPrev: boolean
}
const CalendarHeader: FC<CalendarHeaderProps> = ({ viewDate, showArrowNav, onChangeViewDate, toggleDisplayYears }) => {
const CalendarHeader: FC<CalendarHeaderProps> = ({ hasPrev, hasNext, viewDate, showArrowNav, onChangeViewDate, toggleDisplayYears }) => {
const setPrevMonth = () => {
onChangeViewDate(viewDate.subtract(1, "month"));
@@ -35,14 +38,20 @@ const CalendarHeader: FC<CalendarHeaderProps> = ({ viewDate, showArrowNav, onCha
{showArrowNav && (
<div className="vm-calendar-header-right">
<div
className="vm-calendar-header-right__prev"
onClick={setPrevMonth}
className={classNames({
"vm-calendar-header-right__prev": true,
"vm-calendar-header-right_disabled": !hasPrev,
})}
onClick={hasPrev ? setPrevMonth : undefined}
>
<ArrowDownIcon/>
</div>
<div
className="vm-calendar-header-right__next"
onClick={setNextMonth}
className={classNames({
"vm-calendar-header-right__next": true,
"vm-calendar-header-right_disabled": !hasNext,
})}
onClick={hasNext ? setNextMonth : undefined}
>
<ArrowDownIcon/>
</div>

View File

@@ -3,13 +3,14 @@ import dayjs, { Dayjs } from "dayjs";
import classNames from "classnames";
interface CalendarMonthsProps {
minDate?: Dayjs
maxDate?: Dayjs
viewDate: Dayjs,
selectDate: Dayjs
onChangeViewDate: (date: Dayjs) => void
}
const MonthsList: FC<CalendarMonthsProps> = ({ viewDate, selectDate, onChangeViewDate }) => {
const MonthsList: FC<CalendarMonthsProps> = ({ minDate, maxDate, viewDate, selectDate, onChangeViewDate }) => {
const today = dayjs().format("MM");
const currentMonths = useMemo(() => selectDate.format("MM"), [selectDate]);
@@ -29,20 +30,24 @@ const MonthsList: FC<CalendarMonthsProps> = ({ viewDate, selectDate, onChangeVie
return (
<div className="vm-calendar-years">
{months.map(m => (
<div
className={classNames({
"vm-calendar-years__year": true,
"vm-calendar-years__year_selected": m.format("MM") === currentMonths,
"vm-calendar-years__year_today": m.format("MM") === today
})}
id={`vm-calendar-year-${m.format("MM")}`}
key={m.format("MM")}
onClick={createHandlerClick(m)}
>
{m.format("MMMM")}
</div>
))}
{months.map(m => {
const isDisabled = m && ((minDate && m.isBefore(minDate)) || (maxDate && m.isAfter(maxDate)));
return (
<div
className={classNames({
"vm-calendar-years__year": true,
"vm-calendar-years__year_selected": m.format("MM") === currentMonths,
"vm-calendar-years__year_today": m.format("MM") === today,
"vm-calendar-years__year_disabled": isDisabled,
})}
id={`vm-calendar-year-${m.format("MM")}`}
key={m.format("MM")}
onClick={isDisabled ? undefined : createHandlerClick(m)}
>
{m.format("MMMM")}
</div>
);
})}
</div>
);
};

View File

@@ -3,11 +3,13 @@ import dayjs, { Dayjs } from "dayjs";
import classNames from "classnames";
interface CalendarYearsProps {
minDate?: Dayjs
maxDate?: Dayjs
viewDate: Dayjs
onChangeViewDate: (date: Dayjs) => void
}
const YearsList: FC<CalendarYearsProps> = ({ viewDate, onChangeViewDate }) => {
const YearsList: FC<CalendarYearsProps> = ({ minDate, maxDate, viewDate, onChangeViewDate }) => {
const today = dayjs().format("YYYY");
const currentYear = useMemo(() => viewDate.format("YYYY"), [viewDate]);
@@ -30,20 +32,24 @@ const YearsList: FC<CalendarYearsProps> = ({ viewDate, onChangeViewDate }) => {
return (
<div className="vm-calendar-years">
{years.map(y => (
<div
className={classNames({
"vm-calendar-years__year": true,
"vm-calendar-years__year_selected": y.format("YYYY") === currentYear,
"vm-calendar-years__year_today": y.format("YYYY") === today
})}
id={`vm-calendar-year-${y.format("YYYY")}`}
key={y.format("YYYY")}
onClick={createHandlerClick(y)}
>
{y.format("YYYY")}
</div>
))}
{years.map(y => {
const isDisabled = y && (minDate && y.isBefore(minDate)) || (maxDate && y.isAfter(maxDate));
return (
<div
className={classNames({
"vm-calendar-years__year": true,
"vm-calendar-years__year_selected": y.format("YYYY") === currentYear,
"vm-calendar-years__year_today": y.format("YYYY") === today,
"vm-calendar-years__year_disabled": isDisabled,
})}
id={`vm-calendar-year-${y.format("YYYY")}`}
key={y.format("YYYY")}
onClick={isDisabled ? undefined : createHandlerClick(y)}
>
{y.format("YYYY")}
</div>
);
})}
</div>
);
};

View File

@@ -69,6 +69,10 @@
}
}
&_disabled {
color: $color-text-disabled;
}
&__prev {
transform: rotate(90deg);
}
@@ -108,7 +112,12 @@
cursor: pointer;
transition: color 200ms ease, background-color 300ms ease-in-out;
&:hover {
&_disabled {
cursor: unset;
color: $color-text-disabled;
}
&:not(&_disabled):hover {
background-color: $color-hover-black;
}
@@ -148,7 +157,12 @@
cursor: pointer;
transition: color 200ms ease, background-color 300ms ease-in-out;
&:hover {
&_disabled {
cursor: unset;
color: $color-text-disabled;
}
&:not(&_disabled):hover {
background-color: $color-hover-black;
}

View File

@@ -10,8 +10,10 @@ import useEventListener from "../../../hooks/useEventListener";
interface DatePickerProps {
date: string | Date | Dayjs,
targetRef: React.RefObject<HTMLElement>;
format?: string
label?: string
format?: string;
label?: string;
minDate?: Date | Dayjs;
maxDate?: Date | Dayjs;
onChange: (val: string) => void
}
@@ -20,7 +22,9 @@ const DatePicker = forwardRef<HTMLDivElement, DatePickerProps>(({
targetRef,
format = DATE_TIME_FORMAT,
onChange,
label
label,
minDate,
maxDate
}, ref) => {
const dateDayjs = useMemo(() => dayjs(date).isValid() ? dayjs.tz(date) : dayjs().tz(), [date]);
const { isMobile } = useDeviceDetect();
@@ -56,6 +60,8 @@ const DatePicker = forwardRef<HTMLDivElement, DatePickerProps>(({
date={dateDayjs}
format={format}
onChange={handleChangeDate}
minDate={minDate}
maxDate={maxDate}
/>
</div>
</Popper>

View File

@@ -3,39 +3,52 @@ import { ChangeEvent, KeyboardEvent } from "react";
import { CalendarIcon } from "../../Icons";
import DatePicker from "../DatePicker";
import Button from "../../Button/Button";
import { DATE_TIME_FORMAT } from "../../../../constants/date";
import { DATE_ISO_FORMAT, DATE_FORMAT, DATE_TIME_FORMAT } from "../../../../constants/date";
import InputMask from "react-input-mask";
import dayjs from "dayjs";
import dayjs, { Dayjs } from "dayjs";
import classNames from "classnames";
import "./style.scss";
const formatStringDate = (val: string) => {
return dayjs(val).isValid() ? dayjs.tz(val).format(DATE_TIME_FORMAT) : val;
const formatStringDate = (val: string, format: string) => {
return dayjs(val).isValid() ? dayjs.tz(val).format(format) : val;
};
interface DateTimeInputProps {
value?: string;
label: string;
pickerLabel: string;
dateOnly?: boolean;
format?: string;
pickerRef: React.RefObject<HTMLDivElement>;
onChange: (date: string) => void;
onEnter: () => void;
disabled?: boolean;
minDate?: Date | Dayjs;
maxDate?: Date | Dayjs;
}
const masks: Record<string, string> = {
[DATE_ISO_FORMAT]: "9999-99-99T99:99:99",
[DATE_FORMAT]: "9999-99-99",
[DATE_TIME_FORMAT]: "9999-99-99 99:99:99"
};
const DateTimeInput: FC<DateTimeInputProps> = ({
value = "",
dateOnly = false,
format = DATE_TIME_FORMAT,
minDate,
maxDate,
label,
pickerLabel,
pickerRef,
onChange,
onEnter
onEnter,
disabled
}) => {
const wrapperRef = useRef<HTMLDivElement>(null);
const [inputRef, setInputRef] = useState<HTMLInputElement | null>(null);
const mask = masks[format];
const [maskedValue, setMaskedValue] = useState(formatStringDate(value));
const [maskedValue, setMaskedValue] = useState(formatStringDate(value, format));
const [focusToTime, setFocusToTime] = useState(false);
const [awaitChangeForEnter, setAwaitChangeForEnter] = useState(false);
const error = dayjs(maskedValue).isValid() ? "" : "Invalid date format";
@@ -55,16 +68,13 @@ const DateTimeInput: FC<DateTimeInputProps> = ({
}
};
const mask = dateOnly ? "9999-99-99" : "9999-99-99 99:99:99";
const placeholder = dateOnly ? "YYYY-MM-DD" : "YYYY-MM-DD HH:mm:ss";
const handleChangeDate = (val: string) => {
setMaskedValue(val);
setFocusToTime(true);
};
useEffect(() => {
const newValue = formatStringDate(value);
const newValue = formatStringDate(value, format);
if (newValue !== maskedValue) {
setMaskedValue(newValue);
}
@@ -87,7 +97,8 @@ const DateTimeInput: FC<DateTimeInputProps> = ({
<div
className={classNames({
"vm-date-time-input": true,
"vm-date-time-input_error": error
"vm-date-time-input_error": error,
"vm-date-time-input_disabled": disabled,
})}
>
<label>{label}</label>
@@ -95,7 +106,7 @@ const DateTimeInput: FC<DateTimeInputProps> = ({
tabIndex={1}
inputRef={setInputRef}
mask={mask}
placeholder={placeholder}
placeholder={format}
value={maskedValue}
autoCapitalize={"none"}
inputMode={"numeric"}
@@ -103,6 +114,7 @@ const DateTimeInput: FC<DateTimeInputProps> = ({
onChange={handleMaskedChange}
onBlur={handleBlur}
onKeyUp={handleKeyUp}
disabled={disabled}
/>
{error && (
<span className="vm-date-time-input__error-text">{error}</span>
@@ -117,6 +129,7 @@ const DateTimeInput: FC<DateTimeInputProps> = ({
size="small"
startIcon={<CalendarIcon/>}
ariaLabel="calendar"
disabled={disabled}
/>
</div>
<DatePicker
@@ -125,6 +138,9 @@ const DateTimeInput: FC<DateTimeInputProps> = ({
date={maskedValue}
onChange={handleChangeDate}
targetRef={wrapperRef}
minDate={minDate}
maxDate={maxDate}
format={format}
/>
</div>
);

View File

@@ -23,6 +23,14 @@
user-select: none;
}
&_disabled {
cursor: default;
pointer-events: none;
* {
color: $color-text-disabled !important;
}
}
&__icon {
position: absolute;
bottom: 2px;

View File

@@ -46,11 +46,12 @@ const Select: FC<SelectProps> = ({
const autocompleteAnchorEl = useRef<HTMLDivElement>(null);
const [wrapperRef, setWrapperRef] = useState<React.RefObject<HTMLElement> | null>(null);
const [openList, setOpenList] = useState(false);
const resultList = [...list];
const inputRef = useRef<HTMLInputElement>(null);
const isMultiple = Array.isArray(value);
const selectedValues = Array.isArray(value) ? value.slice() : [];
let selectedValues = Array.isArray(value) ? value.slice() : [];
const hideInput = isMobile && isMultiple && !!selectedValues?.length;
const textFieldValue = useMemo(() => {
@@ -77,7 +78,7 @@ const Select: FC<SelectProps> = ({
};
const handleBlur = () => {
list.includes(search) && onChange(search);
resultList.includes(search) && onChange(search);
};
const handleToggleList = (e: MouseEvent<HTMLDivElement>) => {
@@ -123,8 +124,10 @@ const Select: FC<SelectProps> = ({
useEventListener("keyup", handleKeyUp);
useClickOutside(autocompleteAnchorEl, handleCloseList, wrapperRef);
includeAll && !list.includes("All") && list.push("All");
includeAll && !selectedValues?.length && selectedValues.push("All");
if (includeAll && !resultList.includes("All")) resultList.push("All");
if (includeAll && (!selectedValues?.length || selectedValues?.length === resultList?.length)) {
selectedValues = ["All"];
}
return (
<div
@@ -155,6 +158,7 @@ const Select: FC<SelectProps> = ({
onInput={handleChange}
onFocus={handleFocus}
onBlur={handleBlur}
disabled={disabled}
ref={inputRef}
readOnly={isMobile || !searchable}
/>
@@ -182,7 +186,7 @@ const Select: FC<SelectProps> = ({
itemClassName={itemClassName}
label={label}
value={autocompleteValue}
options={list.map(l => ({ value: l }))}
options={resultList.map(l => ({ value: l }))}
anchor={autocompleteAnchorEl}
selected={selectedValues}
minLength={1}

View File

@@ -128,8 +128,14 @@
}
&_disabled {
pointer-events: none;
* {
cursor: not-allowed;
color: var(--color-text-disabled);
cursor: default;
}
input::placeholder {
color: var(--color-text-disabled);
}
.vm-select-input {

View File

@@ -72,7 +72,6 @@ const TextField: FC<TextFieldProps> = ({
"vm-text-field__input_error": error,
"vm-text-field__input_warning": !error && warning,
"vm-text-field__input_icon-start": startIcon,
"vm-text-field__input_disabled": disabled,
"vm-text-field__input_textarea": type === "textarea",
});
@@ -136,7 +135,8 @@ const TextField: FC<TextFieldProps> = ({
className={classNames({
"vm-text-field": true,
"vm-text-field_textarea": type === "textarea",
"vm-text-field_dark": isDarkTheme
"vm-text-field_dark": isDarkTheme,
"vm-text-field_disabled": disabled
})}
data-replicated-value={value}
>

View File

@@ -6,6 +6,15 @@
margin: 6px 0;
width: 100%;
&_disabled {
color: $color-text-disabled;
pointer-events: none;
}
&:is(&_disabled) > &__label {
color: $color-text-disabled;
}
&_textarea:after {
content: attr(data-replicated-value) " ";
white-space: pre-wrap;

View File

@@ -155,15 +155,18 @@ const ExploreRules: FC = () => {
[groups, types, states, searchInput]
);
const selectedTypes = allTypes.size === types.length ? [] : types;
const selectedStates = allStates.size === states.length ? [] : states;
return (
<>
{modalOpen && getModal()}
{(!modalOpen || !!allStates?.size) && (
<div className="vm-explore-alerts">
<RulesHeader
types={types}
types={selectedTypes}
allTypes={Array.from(allTypes)}
states={states}
states={selectedStates}
allStates={Array.from(allStates)}
onChangeTypes={handleChangeTypes}
onChangeStates={handleChangeStates}

View File

@@ -1,11 +1,8 @@
package apptest
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"math"
"net/url"
"slices"
@@ -503,44 +500,3 @@ func sortTSDBStatusResponseEntries(entries []TSDBStatusResponseEntry) {
return left.Count < right.Count
})
}
// LogsQLQueryResponse is an in-memory representation of the
// /select/logsql/query response.
type LogsQLQueryResponse struct {
LogLines []string
}
// NewLogsQLQueryResponse is a test helper function that creates a new
// instance of LogsQLQueryResponse by unmarshalling a json string.
func NewLogsQLQueryResponse(t *testing.T, s string) *LogsQLQueryResponse {
t.Helper()
res := &LogsQLQueryResponse{}
if len(s) == 0 {
return res
}
bs := bytes.NewBufferString(s)
for {
logLine, err := bs.ReadString('\n')
if err != nil {
if errors.Is(err, io.EOF) {
if len(logLine) > 0 {
t.Fatalf("BUG: unexpected non-empty line=%q with io.EOF", logLine)
}
break
}
t.Fatalf("BUG: cannot read logline from buffer: %s", err)
}
var lv map[string]any
if err := json.Unmarshal([]byte(logLine), &lv); err != nil {
t.Fatalf("cannot parse log line=%q: %s", logLine, err)
}
delete(lv, "_stream_id")
normalizedLine, err := json.Marshal(lv)
if err != nil {
t.Fatalf("cannot marshal parsed logline=%q: %s", logLine, err)
}
res.LogLines = append(res.LogLines, string(normalizedLine))
}
return res
}

View File

@@ -52,6 +52,7 @@ func testSpecialQueryRegression(tc *apptest.TestCase, sut apptest.PrometheusWrit
testTooBigLookbehindWindow(tc, sut)
testMatchSeries(tc, sut)
testNegativeIncrease(tc, sut)
testInstantQueryWithOffsetUsingCache(tc, sut)
// graphite
testComparisonNotInfNotNan(tc, sut)
@@ -292,6 +293,45 @@ func testNegativeIncrease(tc *apptest.TestCase, sut apptest.PrometheusWriteQueri
})
}
func testInstantQueryWithOffsetUsingCache(tc *apptest.TestCase, sut apptest.PrometheusWriteQuerier) {
t := tc.T()
// unexpected /api/v1/query response due to wrong applied offset to request range
// see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9762
sut.PrometheusAPIV1ImportPrometheus(t, []string{
`vm_http_requests_total 1 1758196800000`, // 2025-09-18 12:00:00
`vm_http_requests_total 2 1758218400000`, // 2025-09-18 18:00:00
`vm_http_requests_total 3 1758240000000`, // 2025-09-19 00:00:00
`vm_http_requests_total 4 1758261600000`, // 2025-09-19 06:00:00
`vm_http_requests_total 5 1758283200000`, // 2025-09-19 12:00:00
`vm_http_requests_total 6 1758304800000`, // 2025-09-19 18:00:00
`vm_http_requests_total 7 1758326400000`, // 2025-09-20 00:00:00
}, apptest.QueryOpts{})
sut.ForceFlush(t)
tc.Assert(&apptest.AssertOptions{
Msg: "unexpected /api/v1/query response",
DoNotRetry: true,
Got: func() any {
return sut.PrometheusAPIV1Query(t, `avg_over_time(vm_http_requests_total[1d] offset 12h)`, apptest.QueryOpts{
Time: "2025-09-20T12:00:01.000Z",
})
},
Want: &apptest.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &apptest.QueryData{
ResultType: "vector",
Result: []*apptest.QueryResult{
{
Metric: map[string]string{},
Sample: &apptest.Sample{Timestamp: 1758369601000, Value: 5.5},
},
},
},
},
})
}
func testComparisonNotInfNotNan(tc *apptest.TestCase, sut apptest.PrometheusWriteQuerier) {
t := tc.T()

View File

@@ -1786,4 +1786,4 @@
"uid": "gF-lxRdVz",
"version": 1,
"weekStart": ""
}
}

View File

@@ -1168,7 +1168,7 @@
"uid": "$ds"
},
"editorMode": "code",
"expr": "histogram_quantile(0.99,sum(rate(controller_runtime_reconcile_time_seconds_bucket{job=~\"$job\"}[$__rate_interval])) by(le,controller) )",
"expr": "histogram_quantile(0.99, sum(rate(controller_runtime_reconcile_time_seconds_bucket{job=~\"$job\"}[$__rate_interval])) by (le, controller) )",
"legendFormat": "q.99 {{controller}}",
"range": true,
"refId": "A"
@@ -1265,7 +1265,7 @@
"uid": "$ds"
},
"editorMode": "code",
"expr": "sum(rate(rest_client_requests_total{job=~\"$job\"}[$__interval])) by (method,code)",
"expr": "sum(rate(rest_client_requests_total{job=~\"$job\"}[$__interval])) by (method, code)",
"instant": false,
"legendFormat": "{{method}} {{code}}",
"range": true,
@@ -1489,7 +1489,7 @@
"uid": "$ds"
},
"editorMode": "code",
"expr": "max(histogram_quantile(0.99, sum(rate(go_sched_latencies_seconds_bucket{job=~\"$job\"}[$__rate_interval])) by (job, instance, le))) by(job)",
"expr": "max(histogram_quantile(0.99, sum(rate(go_sched_latencies_seconds_bucket{job=~\"$job\"}[$__rate_interval])) by (job, instance, le))) by (job)",
"instant": false,
"legendFormat": "__auto",
"range": true,
@@ -1588,7 +1588,7 @@
"uid": "$ds"
},
"editorMode": "code",
"expr": "histogram_quantile(0.99,sum(rate(rest_client_request_duration_seconds_bucket{job=~\"$job\"})) by(le,method,api) )",
"expr": "histogram_quantile(0.99, sum(rate(rest_client_request_duration_seconds_bucket{job=~\"$job\"}[$__rate_interval])) by (le, method, api))",
"instant": false,
"legendFormat": "{{method}} {{api}}",
"range": true,
@@ -2135,6 +2135,16 @@
"skipUrlSync": false,
"sort": 2,
"type": "query"
},
{
"baseFilters": [],
"datasource": {
"type": "prometheus",
"uid": "$ds"
},
"filters": [],
"name": "adhoc",
"type": "adhoc"
}
]
},

View File

@@ -1950,6 +1950,16 @@
],
"query": "*",
"type": "textbox"
},
{
"baseFilters": [],
"datasource": {
"type": "victoriametrics-logs-datasource",
"uid": "$ds"
},
"filters": [],
"name": "adhoc",
"type": "adhoc"
}
]
},
@@ -1962,4 +1972,4 @@
"title": "Query Stats (cluster)",
"uid": "feg3od1zt1fy8e",
"version": 1
}
}

View File

@@ -1787,4 +1787,4 @@
"uid": "gF-lxRdVz_vm",
"version": 1,
"weekStart": ""
}
}

View File

@@ -1994,7 +1994,7 @@
"baseFilters": [],
"datasource": {
"type": "victoriametrics-metrics-datasource",
"uid": "PE8D8DB4BEE4E4B22"
"uid": "$ds"
},
"filters": [],
"name": "adhoc",

View File

@@ -2136,6 +2136,16 @@
"skipUrlSync": false,
"sort": 2,
"type": "query"
},
{
"baseFilters": [],
"datasource": {
"type": "victoriametrics-metrics-datasource",
"uid": "$ds"
},
"filters": [],
"name": "adhoc",
"type": "adhoc"
}
]
},

View File

@@ -4238,4 +4238,4 @@
"title": "VictoriaMetrics - vmalert (VM)",
"uid": "LzldHAVnz_vm",
"version": 1
}
}

View File

@@ -2652,7 +2652,7 @@
{
"datasource": {
"type": "victoriametrics-datasource",
"uid": "P38648FE0F8C5BEA2"
"uid": "$ds"
},
"filters": [],
"hide": 0,

View File

@@ -3,14 +3,12 @@
DOCKER_REGISTRIES ?= docker.io quay.io
DOCKER_NAMESPACE ?= victoriametrics
ROOT_IMAGE ?= alpine:3.22.1
ROOT_IMAGE_SCRATCH ?= scratch
CERTS_IMAGE := alpine:3.22.1
GO_BUILDER_IMAGE := golang:1.25.0
GO_BUILDER_IMAGE := golang:1.25.1
BUILDER_IMAGE := local/builder:2.0.0-$(shell echo $(GO_BUILDER_IMAGE) | tr :/ __)-1
BASE_IMAGE := local/base:1.1.4-$(shell echo $(ROOT_IMAGE) | tr :/ __)-$(shell echo $(CERTS_IMAGE) | tr :/ __)
BASE_IMAGE := victoriametrics/base:$(shell git log -1 --format="%h" -- deployment/docker/base/Dockerfile)$(shell git diff-index --quiet HEAD -- deployment/docker/base/Dockerfile || echo '-dirty-'$$(git diff-index -u HEAD -- deployment/docker/base/Dockerfile | openssl sha1 | cut -d' ' -f2 | cut -c 1-8))
DOCKER ?= docker
DOCKER_RUN ?= $(DOCKER) run
DOCKER_BUILD ?= $(DOCKER) build
@@ -20,8 +18,16 @@ DOCKER_IMAGE_LS ?= $(DOCKER) image ls --format '{{.Repository}}:{{.Tag}}'
package-base:
($(DOCKER_IMAGE_LS) | grep -q '$(BASE_IMAGE)$$') \
|| $(DOCKER_BUILD) \
--build-arg root_image=$(ROOT_IMAGE) \
--build-arg certs_image=$(CERTS_IMAGE) \
--tag $(BASE_IMAGE) \
deployment/docker/base
publish-base:
($(DOCKER_IMAGE_LS) | grep -q '$(BASE_IMAGE)$$') \
|| $(DOCKER) buildx build \
-o type=image \
--provenance=false \
--platform=linux/amd64,linux/arm,linux/arm64,linux/ppc64le,linux/386 \
--push \
--tag $(BASE_IMAGE) \
deployment/docker/base
@@ -77,7 +83,7 @@ package-via-docker: package-base
--tag $(DOCKER_NAMESPACE)/$(APP_NAME):$(PKG_TAG)$(APP_SUFFIX)$(RACE) \
-f app/$(APP_NAME)/deployment/Dockerfile bin)
publish-via-docker:
publish-via-docker: publish-base
$(MAKE_PARALLEL) app-via-docker-linux-amd64 \
app-via-docker-linux-arm \
app-via-docker-linux-arm64 \
@@ -85,8 +91,8 @@ publish-via-docker:
app-via-docker-linux-386
$(DOCKER) buildx build \
--platform=linux/amd64,linux/arm,linux/arm64,linux/ppc64le,linux/386 \
--build-arg certs_image=$(CERTS_IMAGE) \
--build-arg root_image=$(ROOT_IMAGE) \
--build-arg certs_image=$(BASE_IMAGE) \
--build-arg root_image=$(BASE_IMAGE) \
--build-arg APP_NAME=$(APP_NAME) \
--build-arg BINARY_SUFFIX="" \
--label "org.opencontainers.image.source=https://github.com/VictoriaMetrics/VictoriaMetrics" \
@@ -105,7 +111,7 @@ publish-via-docker:
bin
$(DOCKER) buildx build \
--platform=linux/amd64,linux/arm,linux/arm64,linux/ppc64le,linux/386 \
--build-arg certs_image=$(CERTS_IMAGE) \
--build-arg certs_image=$(BASE_IMAGE) \
--build-arg root_image=$(ROOT_IMAGE_SCRATCH) \
--build-arg APP_NAME=$(APP_NAME) \
--build-arg BINARY_SUFFIX="" \

View File

@@ -1,10 +1,13 @@
# NOTE: Once the changes made in this file commited, run make publish-base
# See https://medium.com/on-docker/use-multi-stage-builds-to-inject-ca-certs-ad1e8f01de1b
ARG certs_image=non-existing
ARG root_image=non-existing
FROM $certs_image AS certs
FROM alpine:3.22.1 AS certs
RUN apk update && apk upgrade && apk --update --no-cache add ca-certificates
FROM $root_image
FROM alpine:3.22.1
# Temporary fix for CVE-2025-9230, CVE-2025-9231, CVE-2025-9232 until Alpine releases a fixed image.
RUN apk add --no-cache 'libcrypto3=3.5.4-r0' 'libssl3=3.5.4-r0'
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt

View File

@@ -3,7 +3,7 @@ services:
# It scrapes targets defined in --promscrape.config
# And forward them to --remoteWrite.url
vmagent:
image: victoriametrics/vmagent:v1.125.1
image: victoriametrics/vmagent:v1.126.0
depends_on:
- "vmauth"
ports:
@@ -19,7 +19,7 @@ services:
restart: always
grafana:
image: grafana/grafana:12.1.1
image: grafana/grafana:12.2.0
depends_on:
- "vmauth"
ports:
@@ -37,14 +37,14 @@ services:
# vmstorage shards. Each shard receives 1/N of all metrics sent to vminserts,
# where N is number of vmstorages (2 in this case).
vmstorage-1:
image: victoriametrics/vmstorage:v1.125.1-cluster
image: victoriametrics/vmstorage:v1.126.0-cluster
volumes:
- strgdata-1:/storage
command:
- "--storageDataPath=/storage"
restart: always
vmstorage-2:
image: victoriametrics/vmstorage:v1.125.1-cluster
image: victoriametrics/vmstorage:v1.126.0-cluster
volumes:
- strgdata-2:/storage
command:
@@ -54,7 +54,7 @@ services:
# vminsert is ingestion frontend. It receives metrics pushed by vmagent,
# pre-process them and distributes across configured vmstorage shards.
vminsert-1:
image: victoriametrics/vminsert:v1.125.1-cluster
image: victoriametrics/vminsert:v1.126.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -63,7 +63,7 @@ services:
- "--storageNode=vmstorage-2:8400"
restart: always
vminsert-2:
image: victoriametrics/vminsert:v1.125.1-cluster
image: victoriametrics/vminsert:v1.126.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -75,7 +75,7 @@ services:
# vmselect is a query fronted. It serves read queries in MetricsQL or PromQL.
# vmselect collects results from configured `--storageNode` shards.
vmselect-1:
image: victoriametrics/vmselect:v1.125.1-cluster
image: victoriametrics/vmselect:v1.126.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -85,7 +85,7 @@ services:
- "--vmalert.proxyURL=http://vmalert:8880"
restart: always
vmselect-2:
image: victoriametrics/vmselect:v1.125.1-cluster
image: victoriametrics/vmselect:v1.126.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -100,7 +100,7 @@ services:
# read requests from Grafana, vmui, vmalert among vmselects.
# It can be used as an authentication proxy.
vmauth:
image: victoriametrics/vmauth:v1.125.1
image: victoriametrics/vmauth:v1.126.0
depends_on:
- "vmselect-1"
- "vmselect-2"
@@ -114,7 +114,7 @@ services:
# vmalert executes alerting and recording rules
vmalert:
image: victoriametrics/vmalert:v1.125.1
image: victoriametrics/vmalert:v1.126.0
depends_on:
- "vmauth"
ports:
@@ -138,7 +138,7 @@ services:
# alertmanager receives alerting notifications from vmalert
# and distributes them according to --config.file.
alertmanager:
image: prom/alertmanager:v0.28.0
image: prom/alertmanager:v0.28.1
volumes:
- ./alertmanager.yml:/config/alertmanager.yml
command:

View File

@@ -3,7 +3,7 @@ services:
# It scrapes targets defined in --promscrape.config
# And forward them to --remoteWrite.url
vmagent:
image: victoriametrics/vmagent:v1.125.1
image: victoriametrics/vmagent:v1.126.0
depends_on:
- "victoriametrics"
ports:
@@ -18,7 +18,7 @@ services:
# VictoriaMetrics instance, a single process responsible for
# storing metrics and serve read requests.
victoriametrics:
image: victoriametrics/victoria-metrics:v1.125.1
image: victoriametrics/victoria-metrics:v1.126.0
ports:
- 8428:8428
- 8089:8089
@@ -38,7 +38,7 @@ services:
restart: always
grafana:
image: grafana/grafana:12.1.1
image: grafana/grafana:12.2.0
depends_on:
- "victoriametrics"
ports:
@@ -54,7 +54,7 @@ services:
# vmalert executes alerting and recording rules
vmalert:
image: victoriametrics/vmalert:v1.125.1
image: victoriametrics/vmalert:v1.126.0
depends_on:
- "victoriametrics"
- "alertmanager"
@@ -79,7 +79,7 @@ services:
# alertmanager receives alerting notifications from vmalert
# and distributes them according to --config.file.
alertmanager:
image: prom/alertmanager:v0.28.0
image: prom/alertmanager:v0.28.1
volumes:
- ./alertmanager.yml:/config/alertmanager.yml
command:

View File

@@ -1,15 +0,0 @@
groups:
- name: log-rules
type: vlogs
interval: 30s
rules:
- alert: AlwaysFiring
expr: '* | stats count()'
annotations:
description: "Generated more than {{$value}} log entries in the last 1 minute"
- alert: TooManyLogs
expr: '* | stats by (path) count() as total | filter total:>50'
annotations:
description: "Path {{$labels.path}} generated more than 50 log entries in the last 1 minute: {{$value}}"
- record: path:logs:count
expr: '* | stats by (path) count()'

View File

@@ -1,6 +1,6 @@
services:
vmagent:
image: victoriametrics/vmagent:v1.125.1
image: victoriametrics/vmagent:v1.126.0
depends_on:
- "victoriametrics"
ports:
@@ -14,7 +14,7 @@ services:
restart: always
victoriametrics:
image: victoriametrics/victoria-metrics:v1.125.1
image: victoriametrics/victoria-metrics:v1.126.0
ports:
- 8428:8428
volumes:
@@ -27,7 +27,7 @@ services:
restart: always
grafana:
image: grafana/grafana:12.1.1
image: grafana/grafana:12.2.0
depends_on:
- "victoriametrics"
ports:
@@ -40,7 +40,7 @@ services:
restart: always
vmalert:
image: victoriametrics/vmalert:v1.125.1
image: victoriametrics/vmalert:v1.126.0
depends_on:
- "victoriametrics"
ports:
@@ -73,7 +73,7 @@ services:
- "/config.yaml"
- "--licenseFile=/license"
alertmanager:
image: prom/alertmanager:v0.28.0
image: prom/alertmanager:v0.28.1
volumes:
- ./alertmanager.yml:/config/alertmanager.yml
command:
@@ -83,7 +83,7 @@ services:
restart: always
node-exporter:
image: quay.io/prometheus/node-exporter:v1.7.0
image: quay.io/prometheus/node-exporter:v1.9.1
ports:
- 9100:9100
pid: host

View File

@@ -1,17 +0,0 @@
prepare-logs:
cd ./source_logs && bash download.sh
docker-up-elk:
docker-compose -f docker-compose.yml -f docker-compose-elk.yml up -d
docker-stop-elk:
docker-compose -f docker-compose.yml -f docker-compose-elk.yml stop
docker-up-loki:
docker-compose -f docker-compose.yml -f docker-compose-loki.yml up -d
docker-stop-loki:
docker-compose -f docker-compose.yml -f docker-compose-loki.yml stop
docker-cleanup:
docker-compose -f docker-compose.yml -f docker-compose-elk.yml -f docker-compose-loki.yml down -v --remove-orphans

View File

@@ -1,69 +0,0 @@
version: "3"
services:
filebeat-elastic:
image: docker.elastic.co/beats/filebeat:8.8.0
restart: on-failure
volumes:
- ./elk/filebeat/filebeat-elastic.yml:/usr/share/filebeat/filebeat.yml:ro
depends_on:
- elastic
filebeat-vlogs:
image: docker.elastic.co/beats/filebeat:8.8.0
restart: on-failure
volumes:
- ./elk/filebeat/filebeat-vlogs.yml:/usr/share/filebeat/filebeat.yml:ro
depends_on:
- vlogs
generator:
image: golang:1.25.0-alpine
restart: always
working_dir: /go/src/app
volumes:
- ./generator:/go/src/app
- ./source_logs:/go/src/source_logs
command:
- go
- run
- main.go
- -logsPath=/go/src/source_logs/logs
- -outputRateLimitItems=10000
- -syslog.addr=filebeat-elastic:12345
- -syslog.addr2=filebeat-vlogs:12345
- -logs.randomSuffix=false
depends_on: [filebeat-elastic, filebeat-vlogs]
elastic:
image: docker.elastic.co/elasticsearch/elasticsearch:8.8.0
volumes:
- ./elk/elastic/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- elastic:/usr/share/elasticsearch/data
environment:
ES_JAVA_OPTS: "-Xmx2048m"
kibana:
image: docker.elastic.co/kibana/kibana:8.8.0
volumes:
- ./elk/kibana/kibana.yml:/usr/share/kibana/config/kibana.yml
ports:
- "5601:5601"
depends_on: [elastic]
beat-exporter-elastic:
image: trustpilot/beat-exporter:0.4.0
command:
- -beat.uri=http://filebeat-elastic:5066
depends_on:
- filebeat-elastic
beat-exporter-vlogs:
image: trustpilot/beat-exporter:0.4.0
command:
- -beat.uri=http://filebeat-vlogs:5066
depends_on:
- filebeat-vlogs
volumes:
elastic:

View File

@@ -1,51 +0,0 @@
version: "3"
services:
generator:
image: golang:1.25.0-alpine
restart: always
working_dir: /go/src/app
volumes:
- ./generator:/go/src/app
- ./source_logs:/go/src/source_logs
command:
- go
- run
- main.go
- -logsPath=/go/src/source_logs/logs
- -outputRateLimitItems=10000
- -outputRateLimitPeriod=1s
- -syslog.addr=rsyslog:514
- -syslog.addr2=rsyslog:514
- -logs.randomSuffix=false
depends_on: [rsyslog]
loki:
image: grafana/loki:2.9.0
user: 0:0
ports:
- "3100:3100"
command: -config.file=/etc/loki/loki-config.yaml
volumes:
- loki:/tmp/loki
- ./loki/:/etc/loki/
promtail:
image: grafana/promtail:2.9.0
command: -config.file=/etc/promtail/promtail-config.yaml
volumes:
- ./loki/:/etc/promtail/
depends_on:
- loki
- vlogs
rsyslog:
build:
dockerfile: Dockerfile
context: rsyslog
volumes:
- ./rsyslog/rsyslog.conf:/etc/rsyslog.conf
depends_on: [promtail]
volumes:
loki:

View File

@@ -1,74 +0,0 @@
version: "3"
services:
# Run `make package-victoria-logs` to build victoria-logs image
vlogs:
image: docker.io/victoriametrics/victoria-logs:v1.24.0-victorialogs
volumes:
- vlogs:/vlogs
ports:
- "9428:9428"
command:
- -storageDataPath=/vlogs
cadvisor:
image: gcr.io/cadvisor/cadvisor:v0.47.0
restart: unless-stopped
privileged: true
volumes:
- /:/rootfs:ro
- /var/run:/var/run:ro
- /sys:/sys:ro
- /var/lib/docker/:/var/lib/docker:ro
- /dev/disk/:/dev/disk:ro
node-exporter:
image: prom/node-exporter:latest
restart: unless-stopped
volumes:
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /:/rootfs:ro
command:
- "--path.procfs=/host/proc"
- "--path.rootfs=/rootfs"
- "--path.sysfs=/host/sys"
- "--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)"
du-exporter:
image: ghcr.io/dundee/disk_usage_exporter/disk_usage_exporter-c4084307c537335c2ddb6f4b9b527422:latest
restart: unless-stopped
user: "root"
volumes:
- /var/lib/docker/volumes:/var/lib/docker/volumes:ro
- ./du/config.yml:/config.yml:ro
command:
- "--config=/config.yml"
vmsingle:
image: victoriametrics/victoria-metrics:v1.109.0
ports:
- "8428:8428"
command:
- -storageDataPath=/vmsingle
- -promscrape.config=/promscrape.yml
- -promscrape.maxScrapeSize=1Gb
volumes:
- vmsingle:/vmsingle
- ./vmsingle/promscrape.yml:/promscrape.yml
grafana:
image: grafana/grafana:12.1.1
depends_on: [vmsingle]
ports:
- 3000:3000
volumes:
- grafanadata:/var/lib/grafana
- ./grafana/provisioning/:/etc/grafana/provisioning/
- ./grafana/dashboards:/var/lib/grafana/dashboards/
restart: always
volumes:
vlogs:
vmsingle:
grafanadata: {}

View File

@@ -1,3 +0,0 @@
analyzed-path: /var/lib/docker/volumes
bind-address: 0.0.0.0:9995
dir-level: 1

View File

@@ -1,4 +0,0 @@
cluster.name: "bench"
network.host: 0.0.0.0
xpack.security.enabled: false
discovery.type: single-node

View File

@@ -1,15 +0,0 @@
filebeat.inputs:
- type: syslog
format: rfc3164
protocol.tcp:
host: "0.0.0.0:12345"
output.elasticsearch:
hosts: [ "http://elastic:9200" ]
worker: 5
bulk_max_size: 1000
http:
enabled: true
host: 0.0.0.0
port: 5066

View File

@@ -1,19 +0,0 @@
filebeat.inputs:
- type: syslog
format: rfc3164
protocol.tcp:
host: "0.0.0.0:12345"
output.elasticsearch:
hosts: [ "http://vlogs:9428/insert/elasticsearch/" ]
worker: 5
bulk_max_size: 1000
parameters:
_msg_field: "message"
_time_field: "@timestamp"
_stream_fields: "host.name,process.program,process.pid"
http:
enabled: true
host: 0.0.0.0
port: 5066

View File

@@ -1,3 +0,0 @@
server.name: kibana
server.host: "0.0.0.0"
elasticsearch.hosts: [ "http://elastic:9200" ]

View File

@@ -1,106 +0,0 @@
package main
import (
"bufio"
"encoding/binary"
"flag"
"fmt"
"log"
"log/syslog"
"math/rand"
"net"
"os"
"strconv"
"strings"
"time"
)
var (
logsPath = flag.String("logsPath", "", "Path to logs directory")
syslogAddr = flag.String("syslog.addr", "logstash:12345", "Addr to send logs to")
syslogAddr2 = flag.String("syslog.addr2", "logstash:12345", "Addr to send logs to")
randomSuffix = flag.Bool("logs.randomSuffix", false, "Whether to add a random suffix to a log line")
outputRateLimitItems = flag.Int("outputRateLimitItems", 100, "Number of items to send per second")
outputRateLimitPeriod = flag.Duration("outputRateLimitPeriod", time.Second, "Period of time to send items")
)
func main() {
flag.Parse()
startedAt := time.Now().Unix()
logFiles, err := os.ReadDir(*logsPath)
if err != nil {
panic(fmt.Errorf("error reading directory %s:%w", *logsPath, err))
}
sourceFiles := make([]string, 0)
for _, logFile := range logFiles {
if strings.HasSuffix(logFile.Name(), ".log") {
sourceFiles = append(sourceFiles, logFile.Name())
}
}
log.Printf("sourceFiles: %v", sourceFiles)
log.Printf("running with rate limit: %d items per %s", *outputRateLimitItems, *outputRateLimitPeriod)
limitTicker := time.NewTicker(*outputRateLimitPeriod)
limitItems := *outputRateLimitItems
limiter := make(chan struct{}, limitItems)
go func() {
for {
<-limitTicker.C
for i := 0; i < limitItems; i++ {
limiter <- struct{}{}
}
}
}()
for _, sourceFile := range sourceFiles {
log.Printf("sourceFile: %s", sourceFile)
f, err := os.Open(*logsPath + "/" + sourceFile)
if err != nil {
panic(err)
}
syslogTag := "logs-benchmark-" + sourceFile + "-" + strconv.FormatInt(startedAt, 10)
// Loki uses RFC5424 syslog format, which has a 48 character limit on the tag.
tagLen := len(syslogTag)
if tagLen > 48 {
truncate := tagLen - 48
syslogTag = syslogTag[truncate:]
}
logger, err := syslog.Dial("tcp", *syslogAddr, syslog.LOG_INFO, syslogTag)
if err != nil {
panic(fmt.Errorf("error dialing syslog: %w", err))
}
logger2, err := syslog.Dial("tcp", *syslogAddr2, syslog.LOG_INFO, syslogTag)
if err != nil {
panic(fmt.Errorf("error dialing syslog: %w", err))
}
scanner := bufio.NewScanner(f)
for scanner.Scan() {
<-limiter
line := scanner.Text()
if *randomSuffix {
line = line + " " + randomString()
}
_ = logger.Info(line)
_ = logger2.Info(line)
}
logger.Close()
logger2.Close()
}
}
func randomString() string {
buf := make([]byte, 4)
ip := rand.Uint32()
binary.LittleEndian.PutUint32(buf, ip)
return net.IP(buf).String()
}

View File

@@ -1,393 +0,0 @@
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": {
"type": "grafana",
"uid": "-- Grafana --"
},
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"target": {
"limit": 100,
"matchAny": false,
"tags": [],
"type": "dashboard"
},
"type": "dashboard"
}
]
},
"description": "",
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 1,
"links": [],
"liveNow": false,
"panels": [
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "percent"
},
"overrides": [
{
"matcher": {
"id": "byRegexp",
"options": ".*vlogs.*"
},
"properties": [
{
"id": "displayName",
"value": "victoria-logs"
},
{
"id": "color",
"value": {
"fixedColor": "purple",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byRegexp",
"options": ".*elastic.*"
},
"properties": [
{
"id": "displayName",
"value": "elasticsearch"
},
{
"id": "color",
"value": {
"fixedColor": "yellow",
"mode": "fixed"
}
}
]
}
]
},
"gridPos": {
"h": 11,
"w": 5,
"x": 0,
"y": 0
},
"id": 2,
"options": {
"displayMode": "gradient",
"minVizHeight": 10,
"minVizWidth": 0,
"orientation": "auto",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showUnfilled": true
},
"pluginVersion": "9.2.7",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"editorMode": "code",
"expr": "sum(rate(container_cpu_usage_seconds_total{name=~\"$containers\"}[5m])) by (name) * 100",
"legendFormat": "__auto",
"range": true,
"refId": "A"
}
],
"title": "CPU Usage",
"type": "bargauge"
},
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "bytes"
},
"overrides": [
{
"matcher": {
"id": "byRegexp",
"options": ".*vlogs.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "purple",
"mode": "fixed"
}
},
{
"id": "displayName",
"value": "victoria-logs"
}
]
},
{
"matcher": {
"id": "byRegexp",
"options": ".*elastic.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "yellow",
"mode": "fixed"
}
},
{
"id": "displayName",
"value": "elasticsearch"
}
]
}
]
},
"gridPos": {
"h": 11,
"w": 5,
"x": 5,
"y": 0
},
"id": 3,
"options": {
"displayMode": "gradient",
"minVizHeight": 10,
"minVizWidth": 0,
"orientation": "auto",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showUnfilled": true
},
"pluginVersion": "9.2.7",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"editorMode": "code",
"expr": "sum(container_memory_rss{name=~\"$containers\"}[5m]) by (name)",
"legendFormat": "__auto",
"range": true,
"refId": "A"
}
],
"title": "Memory Usage",
"type": "bargauge"
},
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "decbytes"
},
"overrides": [
{
"matcher": {
"id": "byRegexp",
"options": ".*vlogs.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "purple",
"mode": "fixed"
}
},
{
"id": "displayName",
"value": "victoria-logs"
}
]
},
{
"matcher": {
"id": "byRegexp",
"options": ".*elastic.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "yellow",
"mode": "fixed"
}
},
{
"id": "displayName",
"value": "elasticsearch"
}
]
}
]
},
"gridPos": {
"h": 11,
"w": 5,
"x": 10,
"y": 0
},
"id": 5,
"options": {
"displayMode": "gradient",
"minVizHeight": 10,
"minVizWidth": 0,
"orientation": "auto",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showUnfilled": true
},
"pluginVersion": "9.2.7",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"editorMode": "code",
"expr": "sum(node_disk_usage_bytes{path=~\"$containers_selector\"}[5m]) by (path)",
"legendFormat": "__auto",
"range": true,
"refId": "A"
}
],
"title": "Disk space used",
"type": "bargauge"
}
],
"refresh": false,
"schemaVersion": 37,
"style": "dark",
"tags": [],
"templating": {
"list": [
{
"hide": 2,
"name": "containers_selector",
"query": ".*(vlogs|elastic).*",
"skipUrlSync": false,
"type": "constant"
},
{
"current": {
"selected": true,
"text": [
"logs-benchmark-elastic-1",
"logs-benchmark-vlogs-1"
],
"value": [
"logs-benchmark-elastic-1",
"logs-benchmark-vlogs-1"
]
},
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"definition": "label_values(container_cpu_usage_seconds_total{name!=\"\",}, name)",
"hide": 0,
"includeAll": true,
"label": "",
"multi": true,
"name": "containers",
"options": [],
"query": {
"query": "label_values(container_cpu_usage_seconds_total{name!=\"\",}, name)",
"refId": "StandardVariableQuery"
},
"refresh": 1,
"regex": ".*vlogs|elastic.*",
"skipUrlSync": false,
"sort": 1,
"type": "query"
}
]
},
"time": {
"from": "now-30m",
"to": "now"
},
"timepicker": {},
"timezone": "",
"title": "Elastic vs VLogs - stats only",
"uid": "hkm6P6_4J",
"version": 2,
"weekStart": ""
}

View File

@@ -1,554 +0,0 @@
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": {
"type": "grafana",
"uid": "-- Grafana --"
},
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"target": {
"limit": 100,
"matchAny": false,
"tags": [],
"type": "dashboard"
},
"type": "dashboard"
}
]
},
"description": "",
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 1,
"links": [],
"liveNow": false,
"panels": [
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 7,
"w": 24,
"x": 0,
"y": 0
},
"id": 4,
"options": {
"legend": {
"calcs": [
"last"
],
"displayMode": "table",
"placement": "right",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"editorMode": "code",
"expr": "sum(rate(filebeat_libbeat_output_events{type=\"acked\"})) by (instance, type)",
"legendFormat": "{{instance}} - {{type}}",
"range": true,
"refId": "A"
}
],
"title": "Filebeat items",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "none"
},
"overrides": [
{
"matcher": {
"id": "byRegexp",
"options": ".*vlogs.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "yellow",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byRegexp",
"options": ".*elastic.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "green",
"mode": "fixed"
}
}
]
}
]
},
"gridPos": {
"h": 11,
"w": 12,
"x": 0,
"y": 7
},
"id": 2,
"options": {
"legend": {
"calcs": [
"mean",
"min",
"max"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"editorMode": "code",
"expr": "sum(rate(container_cpu_usage_seconds_total{name=~\"$containers\"})) by (name)",
"legendFormat": "__auto",
"range": true,
"refId": "A"
}
],
"title": "CPU cores usage",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "bytes"
},
"overrides": [
{
"matcher": {
"id": "byRegexp",
"options": ".*vlogs.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "yellow",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byRegexp",
"options": ".*elastic.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "green",
"mode": "fixed"
}
}
]
}
]
},
"gridPos": {
"h": 11,
"w": 12,
"x": 12,
"y": 7
},
"id": 3,
"options": {
"legend": {
"calcs": [
"min",
"max",
"mean"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"editorMode": "code",
"expr": "sum(container_memory_rss{name=~\"$containers\"}) by (name)",
"legendFormat": "__auto",
"range": true,
"refId": "A"
}
],
"title": "Memory Usage",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "decbytes"
},
"overrides": [
{
"matcher": {
"id": "byRegexp",
"options": ".*vlogs.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "yellow",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byRegexp",
"options": ".*elastic.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "green",
"mode": "fixed"
}
}
]
}
]
},
"gridPos": {
"h": 11,
"w": 12,
"x": 0,
"y": 18
},
"id": 5,
"options": {
"legend": {
"calcs": [
"min",
"max",
"mean"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"editorMode": "code",
"expr": "sum(node_disk_usage_bytes{path=~\"$containers_selector\"}) by (path)",
"legendFormat": "__auto",
"range": true,
"refId": "A"
}
],
"title": "Disk space used",
"type": "timeseries"
}
],
"refresh": "10s",
"schemaVersion": 37,
"style": "dark",
"tags": [],
"templating": {
"list": [
{
"hide": 2,
"name": "containers_selector",
"query": ".*(vlogs|elastic).*",
"skipUrlSync": false,
"type": "constant"
},
{
"current": {
"selected": true,
"text": [
"All"
],
"value": [
"$__all"
]
},
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"definition": "label_values(container_cpu_usage_seconds_total{name!=\"\",}, name)",
"hide": 0,
"includeAll": true,
"label": "",
"multi": true,
"name": "containers",
"options": [],
"query": {
"query": "label_values(container_cpu_usage_seconds_total{name!=\"\",}, name)",
"refId": "StandardVariableQuery"
},
"refresh": 1,
"regex": ".*benchmark-vlogs|benchmark-elastic.*",
"skipUrlSync": false,
"sort": 1,
"type": "query"
}
]
},
"time": {
"from": "now-15m",
"to": "now"
},
"timepicker": {},
"timezone": "",
"title": "Elastic vs VLogs",
"uid": "hkm6P6_4z",
"version": 3,
"weekStart": ""
}

View File

@@ -1,597 +0,0 @@
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": {
"type": "grafana",
"uid": "-- Grafana --"
},
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"target": {
"limit": 100,
"matchAny": false,
"tags": [],
"type": "dashboard"
},
"type": "dashboard"
}
]
},
"description": "",
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 1,
"links": [],
"liveNow": false,
"panels": [
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": [
{
"matcher": {
"id": "byRegexp",
"options": ".*VictoriaLogs.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "yellow",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byRegexp",
"options": ".*Loki.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "green",
"mode": "fixed"
}
}
]
}
]
},
"gridPos": {
"h": 7,
"w": 24,
"x": 0,
"y": 0
},
"id": 4,
"options": {
"legend": {
"calcs": [
"last"
],
"displayMode": "table",
"placement": "right",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"editorMode": "code",
"expr": "sum(rate(vl_rows_ingested_total))",
"legendFormat": "VictoriaLogs",
"range": true,
"refId": "A"
},
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"editorMode": "code",
"expr": "rate(loki_distributor_lines_received_total)",
"hide": false,
"legendFormat": "Loki",
"range": true,
"refId": "B"
}
],
"title": "Ingestion rate",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "none"
},
"overrides": [
{
"matcher": {
"id": "byRegexp",
"options": ".*vlogs.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "yellow",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byRegexp",
"options": ".*loki.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "green",
"mode": "fixed"
}
}
]
}
]
},
"gridPos": {
"h": 11,
"w": 12,
"x": 0,
"y": 7
},
"id": 2,
"options": {
"legend": {
"calcs": [
"mean",
"min",
"max"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"editorMode": "code",
"expr": "sum(rate(container_cpu_usage_seconds_total{name=~\"$containers\"})) by (name)",
"legendFormat": "__auto",
"range": true,
"refId": "A"
}
],
"title": "CPU cores usage",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "bytes"
},
"overrides": [
{
"matcher": {
"id": "byRegexp",
"options": ".*vlogs.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "yellow",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byRegexp",
"options": ".*loki.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "green",
"mode": "fixed"
}
}
]
}
]
},
"gridPos": {
"h": 11,
"w": 12,
"x": 12,
"y": 7
},
"id": 3,
"options": {
"legend": {
"calcs": [
"min",
"max",
"mean"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"editorMode": "code",
"expr": "sum(container_memory_rss{name=~\"$containers\"}) by (name)",
"legendFormat": "__auto",
"range": true,
"refId": "A"
}
],
"title": "Memory Usage",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "decbytes"
},
"overrides": [
{
"matcher": {
"id": "byRegexp",
"options": ".*vlogs.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "yellow",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byRegexp",
"options": ".*loki.*"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "green",
"mode": "fixed"
}
}
]
}
]
},
"gridPos": {
"h": 11,
"w": 12,
"x": 0,
"y": 18
},
"id": 5,
"options": {
"legend": {
"calcs": [
"min",
"max",
"mean"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"editorMode": "code",
"expr": "sum(node_disk_usage_bytes{path=~\"$containers_selector\"}) by (path)",
"legendFormat": "__auto",
"range": true,
"refId": "A"
}
],
"title": "Disk space used",
"type": "timeseries"
}
],
"refresh": "10s",
"schemaVersion": 37,
"style": "dark",
"tags": [],
"templating": {
"list": [
{
"hide": 2,
"name": "containers_selector",
"query": ".*(vlogs|loki).*",
"skipUrlSync": false,
"type": "constant"
},
{
"current": {
"selected": true,
"text": [
"All"
],
"value": [
"$__all"
]
},
"datasource": {
"type": "prometheus",
"uid": "P4169E866C3094E38"
},
"definition": "label_values(container_cpu_usage_seconds_total{name!=\"\",}, name)",
"hide": 0,
"includeAll": true,
"label": "",
"multi": true,
"name": "containers",
"options": [],
"query": {
"query": "label_values(container_cpu_usage_seconds_total{name!=\"\",}, name)",
"refId": "StandardVariableQuery"
},
"refresh": 1,
"regex": ".*benchmark-vlogs|benchmark-loki.*",
"skipUrlSync": false,
"sort": 1,
"type": "query"
}
]
},
"time": {
"from": "now-1h",
"to": "now"
},
"timepicker": {},
"timezone": "",
"title": "Loki vs VLogs",
"uid": "hkm6P6_4y",
"version": 1,
"weekStart": ""
}

View File

@@ -1,9 +0,0 @@
apiVersion: 1
providers:
- name: Prometheus
orgId: 1
folder: ''
type: file
options:
path: /var/lib/grafana/dashboards

View File

@@ -1,8 +0,0 @@
apiVersion: 1
datasources:
- name: VictoriaMetrics
type: prometheus
access: proxy
url: http://vmsingle:8428
isDefault: true

View File

@@ -1,63 +0,0 @@
auth_enabled: false
server:
http_listen_port: 3100
grpc_listen_port: 9096
common:
instance_addr: 0.0.0.0
path_prefix: /tmp/loki
storage:
filesystem:
chunks_directory: /tmp/loki/chunks
rules_directory: /tmp/loki/rules
replication_factor: 1
ring:
kvstore:
store: inmemory
query_range:
results_cache:
cache:
embedded_cache:
enabled: true
max_size_mb: 100
schema_config:
configs:
- from: 2020-10-24
store: boltdb-shipper
object_store: filesystem
schema: v11
index:
prefix: index_
period: 24h
ruler:
alertmanager_url: http://localhost:9093
# By default, Loki will send anonymous, but uniquely-identifiable usage and configuration
# analytics to Grafana Labs. These statistics are sent to https://stats.grafana.org/
#
# Statistics help us better understand how Loki is used, and they show us performance
# levels for most users. This helps us prioritize features and documentation.
# For more information on what's sent, look at
# https://github.com/grafana/loki/blob/main/pkg/usagestats/stats.go
# Refer to the buildReport method to see what goes into a report.
#
# If you would like to disable reporting, uncomment the following lines:
analytics:
reporting_enabled: false
limits_config:
ingestion_rate_mb: 10240
ingestion_burst_size_mb: 10240
max_streams_per_user: 10000000
max_global_streams_per_user: 10000000
retention_period: 30d
per_stream_rate_limit: 10240M
per_stream_rate_limit_burst: 10240M
cardinality_limit: 20000000

View File

@@ -1,26 +0,0 @@
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: http://loki:3100/loki/api/v1/push
- url: http://vlogs:9428/insert/loki/api/v1/push?_stream_fields=hostname,application
# batchwait: 5s
# batchsize: 5242880
scrape_configs:
- job_name: syslog
syslog:
listen_address: 0.0.0.0:5140
idle_timeout: 12h
use_incoming_timestamp: true
labels:
job: syslog
relabel_configs:
- source_labels: [__syslog_message_hostname]
target_label: hostname
- source_labels: [__syslog_message_app_name]
target_label: application

View File

@@ -1,125 +0,0 @@
# Benchmark for VictoriaLogs
Benchmark compares VictoriaLogs with ELK stack and Grafana Loki.
Benchmark is based on:
- Logs from this repository - [https://github.com/logpai/loghub](https://github.com/logpai/loghub)
- [logs generator](./generator)
For ELK suite it uses:
- filebeat - [https://www.elastic.co/beats/filebeat](https://www.elastic.co/beats/filebeat)
- elastic + kibana
For Grafana Loki suite it uses:
- [Promtail](https://grafana.com/docs/loki/latest/send-data/promtail/)
- rsyslog - required to push logs in RFC5424 format to Promtail
- [Loki](https://grafana.com/oss/loki/)
## How it works
[docker-compose.yml](./docker-compose.yml) contains common configurations for all suites:
- VictoriaLogs instance
- vmsingle - port forwarded to `localhost:8428` to see UI
- exporters for system metrics
ELK suite uses [docker-compose-elk.yml](./docker-compose-elk.yml) with the following services:
- [logs generator](./generator) which generates logs and sends them to filebeat instances via syslog
- 2 filebeat instances - one for elastic and one for VictoriaLogs.
- elastic instance
- kibana instance - port forwarded to `localhost:5601` to see UI
Loki suite uses [docker-compose-loki.yml](./docker-compose-loki.yml) with the following services:
- [logs generator](./generator) which generates logs and sends them rsyslog
- rsyslog instance - sends logs to Promtail
- Promtail instance - sends logs to Loki and VictoriaLogs
- Loki instance
[Logs generator](./generator) generates logs based on logs located at `./source_logs/logs` and sends them to filebeat
instances via syslog.
Logs are generated by reading files line by line, adding randomized suffix to each line and sending them to filebeat via
syslog.
By default, generator will exit once all files are read. `docker-compose` will restart it and files will be read again
generating new logs.
Each filebeat than writes logs to elastic and VictoriaLogs via elasticsearch-compatible API.
## How to run
1. Download and unarchive logs by running:
```shell
cd source_logs
bash download.sh
```
Note that with logs listed in `download.sh` it will require 49GB of free space:
- 3GB for archives
- 46GB for unarchived logs
If it is needed to minimize disk footprint, you can download only some of them by commenting out lines in `download.sh`.
Unarchived logs size per file for reference:
```shell
2.3M Linux.log
73M SSH.log
32G Thunderbird.log
5.1M Apache.log
13G hadoop-*.log
```
1. (optional) If needed, adjust amount of logs sent by generator by modifying `-outputRateLimitItems` and
`outputRateLimitPeriod` parameters in [docker-compose.yml](./docker-compose.yml). By default, it is configured to
send 10000 logs per second.
1. (optional) Build victoria-logs image and adjust `image` parameter in [docker-compose.yml](./docker-compose.yml):
```shell
make package-victoria-logs
```
Image name should be replaced at `vlogs` service in [docker-compose.yml](./docker-compose.yml).
It is also possible to configure filebeat to send logs to VictoriaLogs running on local machine.
To do this modify [filebeat config for vlogs](./elk/filebeat/filebeat-vlogs.yml) and replace `vlogs` address
with address of local VictoriaLogs instance:
```yaml
output.elasticsearch:
hosts: [ "http://vlogs:9428/insert/elasticsearch/" ]
```
1. Choose a suite to run.
In order to run ELK suite use the following command:
```sh
make docker-up-elk
```
In order to run Loki suite use the following command:
```sh
make docker-up-loki
```
1. Navigate to `http://localhost:3000/` to see Grafana dashboards with resource usage
comparison.
Navigate to `http://localhost:3000/d/hkm6P6_4z/elastic-vs-vlogs` to see ELK suite results.
Navigate to `http://localhost:3000/d/hkm6P6_4y/loki-vs-vlogs` to see Loki suite results.
Example results vs ELK:
![elk-grafana-dashboard.png](results/elk-grafana-dashboard.png)
Example results vs Loki:
![loki-grafana-dashboard.png](results/loki-grafana-dashboard.png)

Binary file not shown.

Before

Width:  |  Height:  |  Size: 157 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 176 KiB

View File

@@ -1,9 +0,0 @@
FROM alpine:3
RUN apk add --no-cache rsyslog rsyslog-tls tzdata
COPY rsyslog.conf /etc/rsyslog.conf
VOLUME /var/run/rsyslog/dev
EXPOSE 514 10514
CMD ["/usr/sbin/rsyslogd", "-n"]

View File

@@ -1,12 +0,0 @@
module(load="imudp")
input(type="imudp" port="514")
module(load="imtcp")
input(type="imtcp" port="514")
*.* action(type="omfwd"
protocol="tcp" target="promtail" port="5140"
Template="RSYSLOG_SyslogProtocol23Format"
TCP_Framing="octet-counted" KeepAlive="on"
action.resumeRetryCount="-1"
queue.type="linkedlist" queue.size="1000000")

View File

@@ -1,2 +0,0 @@
*.log
*.tar.gz

View File

@@ -1,34 +0,0 @@
#!/bin/bash
set -ex
# Unarchived size: 5.1M Apache.log
if [ ! -f Apache.tar.gz ]; then
curl -o Apache.tar.gz -L -C - https://zenodo.org/records/3227177/files/Apache.tar.gz?download=1
fi
# Unarchived size: 13G hadoop-*.log
if [ ! -f HDFS_2.tar.gz ]; then
curl -o HDFS_2.tar.gz -L -C - https://zenodo.org/records/3227177/files/HDFS_2.tar.gz?download=1
fi
# Unarchived size: 2.3M Linux.log
if [ ! -f Linux.tar.gz ]; then
curl -o Linux.tar.gz -L -C - https://zenodo.org/records/3227177/files/Linux.tar.gz?download=1
fi
# Unarchived size: 32G Thunderbird.log
if [ ! -f Thunderbird.tar.gz ]; then
curl -o Thunderbird.tar.gz -L -C - https://zenodo.org/records/3227177/files/Thunderbird.tar.gz?download=1
fi
# Unarchived size: 73M SSH.log
if [ ! -f SSH.tar.gz ]; then
curl -o SSH.tar.gz -L -C - https://zenodo.org/records/3227177/files/SSH.tar.gz?download=1
fi
mkdir -p logs
for file in *.tar.gz; do
tar -xzf $file -C logs
done

View File

@@ -1,30 +0,0 @@
scrape_configs:
- job_name: "filebeat"
scrape_interval: 30s
static_configs:
- targets:
- beat-exporter-elastic:9479
- beat-exporter-vlogs:9479
- job_name: "victoria-logs"
scrape_interval: 30s
static_configs:
- targets:
- vlogs:9428
- job_name: "cadvisor"
scrape_interval: 30s
metric_relabel_configs:
- action: labeldrop
regex: "container_label_.*"
static_configs:
- targets:
- cadvisor:8080
- job_name: 'node'
static_configs:
- targets: ['node-exporter:9100']
- targets: ['du-exporter:9995']
- job_name: 'loki'
static_configs:
- targets: ['loki:3100']
- job_name: 'promtail'
static_configs:
- targets: ['promtail:9080']

View File

@@ -14,6 +14,21 @@ aliases:
---
Please find the changelog for VictoriaMetrics Anomaly Detection below.
## v1.26.0
Released: 2025-10-02
- FEATURE: Introduced vmui-like [UI](https://docs.victoriametrics.com/anomaly-detection/ui/) for `vmanomaly` service to simplify the configuration and backtesting of anomaly detection models before it goes to production. It provides an intuitive interface to finetune model configurations, visualize its predictions and anomaly scores, and perform backtesting on historical data. The UI is accessible via a web browser and can be run as a [standalone service](https://docs.victoriametrics.com/anomaly-detection/ui/#preset-usage) or [integrated with productionalized deployments](https://docs.victoriametrics.com/anomaly-detection/ui/#mixed-usage). For more details, refer to the [documentation](https://docs.victoriametrics.com/anomaly-detection/ui/).
- FEATURE: Added support for reading data from [VictoriaLogs stats queries](https://docs.victoriametrics.com/victorialogs/querying/#querying-log-range-stats) with `VLogsReader`. This reader allows quering and analyzing log data stored in VictoriaLogs, enabling anomaly detection on metrics generated from logs. It supports similar configuration options as `VmReader`, including `datasource_url`, `tenant_id`, `queries`, etc. For more details, refer to the [documentation](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vlogs-reader). It can be also used in [UI mode](https://docs.victoriametrics.com/anomaly-detection/ui/) for backtesting log-based anomaly detection configurations.
- IMPROVEMENT: Resolved the case in the [`IsolationForestModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#isolation-forest-multivariate) with `provide_series` common model [argument](https://docs.victoriametrics.com/anomaly-detection/components/models/#provide-series) including `yhat.*` series (prediction and confidence boundaries), which are not produced by this model. Now config validation will fail with a clear error message if such series names are requested.
- BUGFIX: Recursive and shallow merging of the config files with mixed class names (`class` argument, with aliases like `zscore` and fully qualified names like `model.zscore.ZscoreModel`) now works as expected and is properly resolved to the same entity. Previously, this could lead to validation errors during service startup.
- BUGFIX: Fixed an issue with `anomaly_score_outside_data_range` [argument](https://docs.victoriametrics.com/anomaly-detection/components/models/#score-outside-data-range) not being properly set for some models, resulting in default value (1.01) being used instead of user-defined override.
- BUGFIX: Fixed an issue with `decay` [parameter](https://docs.victoriametrics.com/anomaly-detection/components/models/#decay) not being properly applied to the global smoothing in the `OnlineQuantileModel` (when `seasonal_interval` is not set), resulted in no decay being applied (equivalent to `decay=1.0`).
## v1.25.3
Released: 2025-08-19
@@ -21,7 +36,7 @@ Released: 2025-08-19
- IMPROVEMENT: Added `logger_levels` argument to `settings` [config section](https://docs.victoriametrics.com/anomaly-detection/components/settings/#logger-levels) to allow setting specific log levels for individual components. Useful for debugging specific components. For example, `logger_levels: { "reader.vm": "DEBUG" }` will set the log level for the `VmReader` component to `DEBUG`, while leaving other components at their default log levels. Also is supported in [hot reload](https://docs.victoriametrics.com/anomaly-detection/components/#hot-reload) mode, allowing for dynamic log level changes without service restarts.
- IMPROVEMENT: Added logging of URLs used for querying VictoriaMetrics TSDB in [`VmReader`](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) to ease the debugging of incomplete data retrieval, incorrect endpoints, or misconfigured tenant IDs. The URLs are logged at the `DEBUG` level, so you can control their verbosity using the `--loggerLevelComponents` argument with `reader.vm=DEBUG` or `reader=DEBUG` to see the URLs in the logs.
- IMPROVEMENT: Added logging of URLs used for querying VictoriaMetrics TSDB in [`VmReader`](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) to ease the debugging of incomplete data retrieval, incorrect endpoints, or misconfigured tenant IDs. The URLs are logged at the `DEBUG` level, so you can override individual verbosity using the [settings.logger_levels](https://docs.victoriametrics.com/anomaly-detection/components/settings/#logger-levels) configuration.
- IMPROVEMENT: Added `offset` [argument](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) to `VmReader` on reader and query levels to allow for flexible time offset adjustments in the reader. Useful for correcting for data collection delays. The `offset` can be specified as a string (e.g., "15s", "-20s") and will be applied to all queries processed by the reader. See [FAQ](https://docs.victoriametrics.com/anomaly-detection/faq/#using-offsets) for more details.
@@ -148,7 +163,7 @@ Released: 2025-04-11
## v1.21.0
Released: 2025-03-19
- FEATURE: Introduced [horizontal scalability](https://docs.victoriametrics.com/anomaly-detection/scaling-vmanomaly/#horizontal-scalability) and [high availability](https://docs.victoriametrics.com/anomaly-detection/scaling-vmanomaly/#high-availability) in `vmanomaly` service. Dedicated page can be found [here](https://docs.victoriametrics.com/anomaly-detection/scaling-vmanomaly/).
- FEATURE: Introduced [horizontal scalability](https://docs.victoriametrics.com/anomaly-detection/scaling-vmanomaly#horizontal-scalability) and [high availability](https://docs.victoriametrics.com/anomaly-detection/scaling-vmanomaly#high-availability) in `vmanomaly` service. Dedicated page can be found [here](https://docs.victoriametrics.com/anomaly-detection/scaling-vmanomaly).
## v1.20.1
Released: 2025-03-16
@@ -162,7 +177,7 @@ Released: 2025-03-03
- FEATURE: The `scale` argument is now a [common argument](https://docs.victoriametrics.com/anomaly-detection/components/models/#scale), previously supported only by [`ProphetModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) and [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile). Additionally, `scale` is now **two-sided**, represented as `[scale_lb, scale_ub]`. The previous format (`scale: x`) remains supported and will be automatically converted to `scale: [x, x]`.
- FEATURE: Introduced a post-processing step to clip `yhat`, `yhat_lower`, and `yhat_upper` to the configured `data_range` [values](https://docs.victoriametrics.com/anomaly-detection/components/reader/#config-parameters) in `VmReader`, if defined. This feature is disabled by default for backward compatibility. It can be enabled for models that generate predictions and estimates, such as [`ProphetModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet), by setting the [common argument](https://docs.victoriametrics.com/anomaly-detection/components/models/#clip-predictions) `clip_predictions` to `True`.
- FEATURE: Introduced a post-processing step to clip `yhat`, `yhat_lower`, and `yhat_upper` to the configured `data_range` [values](https://docs.victoriametrics.com/anomaly-detection/components/reader/?highlight=data_range#config-parameters) in `VmReader`, if defined. This feature is disabled by default for backward compatibility. It can be enabled for models that generate predictions and estimates, such as [`ProphetModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet), by setting the [common argument](https://docs.victoriametrics.com/anomaly-detection/components/models/#clip-predictions) `clip_predictions` to `True`.
- IMPROVEMENT: Introduced the `anomaly_score_outside_data_range` [parameter](https://docs.victoriametrics.com/anomaly-detection/components/models/#score-outside-data-range) to allow overriding the default anomaly score (`1.01`) assigned when input values (`y`) fall outside the defined `data_range` (data domain violation). It improves flexibility for alerting rules and enables clearer visual distinction between different anomaly scenarios. Override can be configured at the **service level** (`settings`) or per **model instance** (`models.model_xxx`), with model-level values taking priority. If not explicitly set, the default anomaly score remains `1.01` for backward compatibility.
@@ -195,12 +210,12 @@ Released: 2025-01-20
> This release contains a bug introduced in [v1.18.7](#v1187) - [`PeriodicScheduler`](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/#periodic-scheduler) where configurations with `fit_every` > `fit_window` could cause inference to be skipped for |fit_every - fit_window| time, until the next `fit_every` call happens. For `fit_every` > `fit_window` configurations we recommend upgrading to [v1.20.1](#v1201), which resolves this issue.
- FEATURE: Added support for per-query `tenant_id` in the [`VmReader`](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader). This allows overriding the reader-level `tenant_id` within a single global `vmanomaly` configuration on a *per-query* basis, enabling isolation of data for different tenants in separate queries when querying the [VictoriaMetrics cluster version](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/). For details, see the [documentation](https://docs.victoriametrics.com/anomaly-detection/components/reader/#per-query-parameters).
- FEATURE: Added support for per-query `tenant_id` in the [`VmReader`](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader). This allows overriding the reader-level `tenant_id` within a single global `vmanomaly` configuration on a *per-query* basis, enabling isolation of data for different tenants in separate queries when querying the [VictoriaMetrics cluster version](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/). For details, see the [documentation](https://docs.victoriametrics.com/anomaly-detection/components/reader/?highlight=tenant_id#per-query-parameters).
- IMPROVEMEMT: Speedup the model infer stage on multicore systems.
- IMPROVEMEMT: Speedup the model fitting stage by 1.253x, depending on configuration complexity.
- IMPROVEMENT: Reduced service RAM usage by 5-10%, depending on configuration complexity.
- BUGFIX: Now [`VmReader`](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) properly handles the cases where the number of queries processed in parallel (up to `reader.queries` cardinality) exceeds the default limit of 10 HTTP(S) connections, preventing potential data loss from discarded queries. The pool limit will automatically adjust to match `reader.queries` cardinality.
- BUGFIX: Corrected the construction of write endpoints for cluster VictoriaMetrics `url`s (`tenant_id` arg is set) in `monitoring.push` [section configurations](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#push-config-parameters).
- BUGFIX: Corrected the construction of write endpoints for cluster VictoriaMetrics `url`s (`tenant_id` arg is set) in `monitoring.push` [section configurations](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/?highlight=tenant_id#push-config-parameters).
## v1.18.8
Released: 2024-12-03
@@ -216,7 +231,7 @@ Released: 2024-12-02
> This release introduced a bug in [`PeriodicScheduler`](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/#periodic-scheduler) where configurations with `fit_every` > `fit_window` could cause inference to be skipped for |fit_every - fit_window| time, until the next `fit_every` call happens. For `fit_every` > `fit_window` configurations we recommend upgrading to [v1.20.1](#v1201), which resolves this issue.
- IMPROVEMENT: Introduced a new `push_frequency` parameter for the [monitoring.push component](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#push-config-parameters), with a default value of 15m. This enhancement ensures better alignment with pull-based monitoring behavior and improves [self-monitoring experience](https://docs.victoriametrics.com/anomaly-detection/self-monitoring/) of `vmanomaly` in setups with infrequent schedules (e.g., rare `fit_every` or `infer_every` intervals) to deal with data staleness.
- IMPROVEMENT: Introduced a new `push_frequency` parameter for the [monitoring.push component](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#push-config-parameters), with a default value of 15m. This enhancement ensures better alignment with pull-based monitoring behavior and improves [self-monitoring experience](https://docs.victoriametrics.com/anomaly-detection/self-monitoring) of `vmanomaly` in setups with infrequent schedules (e.g., rare `fit_every` or `infer_every` intervals) to deal with data staleness.
- BUGFIX: Fixed a bug, introduced in [v1.18.5](#v1185), that prevented the [monitoring.push component](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#push-config-parameters) from properly instantiating and pushing [self-monitoring metrics](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#metrics-generated-by-vmanomaly).
@@ -235,7 +250,7 @@ Released: 2024-11-27
- IMPROVEMENT: Introduced the ability to run `vmanomaly` using a configuration directory. This enhancement allows users to recursively merge multiple full configuration files (previously limited to merging specific sections, such as `reader`) and execute a single instance of the service with the combined configuration.
- IMPROVEMENT: Added a new utility, `config_splitter.py`, to streamline the process of splitting a single configuration file into multiple standalone configurations. The configurations are split by specified entities like `schedulers`, `models`, `queries` or `extra_filters`. The split configurations can be saved to a designated directory. It simplifies scaling `vmanomaly` and enhances user experience by automating the process of separating config files so they can be run on separate instances of vmanomaly. For more details, refer to [this section](https://docs.victoriametrics.com/anomaly-detection/faq/#splitting-the-config).
- IMPROVEMENT: Introduced the ability to configure the [`PeriodicScheduler`](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/#periodic-scheduler) to start at a specific time using the `start_from` and `tz` parameters. The `start_from` parameter accepts either `HH:MM` or [ISO 8601 formats](https://en.wikipedia.org/wiki/ISO_8601), with `tz` defaulting to `UTC`. If `start_from` is in the past, the next valid start time is automatically calculated based on the `fit_every` interval.
- IMPROVEMENT: Introduced the ability to configure the [`PeriodicScheduler`](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/?highlight=start_from#periodic-scheduler) to start at a specific time using the `start_from` and `tz` parameters. The `start_from` parameter accepts either `HH:MM` or [ISO 8601 formats](https://en.wikipedia.org/wiki/ISO_8601), with `tz` defaulting to `UTC`. If `start_from` is in the past, the next valid start time is automatically calculated based on the `fit_every` interval.
## v1.18.4
Released: 2024-11-18
@@ -290,7 +305,7 @@ Released: 2024-10-28
Released: 2024-10-22
- IMPROVEMENT: Added `vmanomaly_version_info` (service) and `vmanomaly_ui_version_info` (vmui) gauges to self-monitoring metrics.
- IMPROVEMENT: Added `instance` and `job` labels to [pushed](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#push-model) metrics so they have the same labels as vmanomaly metrics that are [pulled](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#pull-model)/scraped. Metric labels can be customized via the [`extra_labels` argument](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#push-config-parameters). By default job label will be `vmanomaly` and the instance label will be `f'{hostname}:{vmanomaly_port}`. See [monitoring.push](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#push-config-parameters) for examples and details.
- IMPROVEMENT: Added `instance` and `job` labels to [pushed](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#push-model) metrics so they have the same labels as vmanomaly metrics that are [pulled](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#pull-model)/scraped. Metric labels can be customized via the [`extra_labels` argument](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/?highlight=extra_labels#push-config-parameters). By default job label will be `vmanomaly` and the instance label will be `f'{hostname}:{vmanomaly_port}`. See [monitoring.push](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#push-config-parameters) for examples and details.
- IMPROVEMENT: Added a subsection to [monitoring](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#logs-generated-by-vmanomaly) page with detailed per-component service logs, including reader and writer logs, error handling, metrics updates, and multi-tenancy warnings.
- IMPROVEMENT: Added a new [Command-line arguments](https://docs.victoriametrics.com/anomaly-detection/quickstart/#command-line-arguments) subsection to the [Quickstart guide](https://docs.victoriametrics.com/anomaly-detection/quickstart/), providing details on available options for configuring `vmanomaly`.
@@ -317,7 +332,7 @@ Released: 2024-10-17
- Updated [docs](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#metrics-generated-by-vmanomaly) for better clarity.
- IMPROVEMENT: Accelerated performance of model fitting stages on multicore systems.
- IMPROVEMENT: Optimized query handling in multi-[scheduler](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/) setups by filtering [queries](https://docs.victoriametrics.com/anomaly-detection/components/models/#queries) for each scheduler based on model requirements. This reduces unnecessary data fetching from VictoriaMetrics, ensuring only relevant queries are processed by the [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader), leading to better performance and efficiency of configs with multiple active schedulers.
- IMPROVEMENT: Optimized query handling in multi-[scheduler](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/) setups by filtering [queries](https://docs.victoriametrics.com/anomaly-detection/components/models/#queries) for each scheduler based on model requirements. This reduces unnecessary data fetching from VictoriaMetrics, ensuring only relevant queries are processed by the [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader#vm-reader), leading to better performance and efficiency of configs with multiple active schedulers.
- IMPROVEMENT: Implemented automatic cleanup of files in subdirectories within `/tmp` ([generated by the Stan backend](https://mc-stan.org/cmdstanpy/users-guide/outputs) when utilizing [Prophet](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) models) after each `fit` operation. This prevents the accumulation of unused data over time in `/tmp`, addressing a potential issue where these files would only be deleted upon termination of the current Python session or service, leading to uncontrolled disk growth.
@@ -330,8 +345,8 @@ Released: 2024-10-08
## v1.16.2
Released: 2024-10-06
- FEATURE: Added support for `multitenant` value in `tenant_id` arg to enable querying across multiple tenants in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/) (option available from [v1.104.0](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy-via-labels)):
- Applied when reading input data from `vmselect` via the [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader).
- Applied when writing generated results through `vminsert` via the [VmWriter](https://docs.victoriametrics.com/anomaly-detection/components/writer/#vm-writer).
- Applied when reading input data from `vmselect` via the [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader#vm-reader).
- Applied when writing generated results through `vminsert` via the [VmWriter](https://docs.victoriametrics.com/anomaly-detection/components/writer#vm-writer).
- For more details, refer to the `tenant_id` arg description in the documentation of the mentioned components.
- BUGFIX: Resolved an issue with handling an empty `preset` value (e.g., `preset: ""`) that was preventing the [default helm chart](https://github.com/VictoriaMetrics/helm-charts/blob/7f5a2c00b14c2c088d7d8d8bcee7a440a5ff11c6/charts/victoria-metrics-anomaly/values.yaml#L139) from being deployed.
@@ -345,9 +360,9 @@ Released: 2024-10-01
> A bug was discovered in this release that causes the service to crash. Please use the patch [v1.16.1](#v1161) to resolve this issue.
- FEATURE: Introduced data dumps to a host filesystem for [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader). Resource-intensive setups (multiple queries returning many metrics, bigger `fit_window` arg) will have RAM consumption reduced during fit calls.
- IMPROVEMENT: Added a `groupby` argument for logical grouping in [multivariate models](https://docs.victoriametrics.com/anomaly-detection/components/models/#multivariate-models). When specified, a separate multivariate model is trained for each unique combination of label values in the `groupby` columns. For example, to perform multivariate anomaly detection on metrics at the machine level without cross-entity interference, you can use `groupby: [host]` or `groupby: [instance]`, ensuring one model per entity being trained (e.g., per host). Please find more details [here](https://docs.victoriametrics.com/anomaly-detection/components/models/#group-by).
- IMPROVEMENT: Improved performance of [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) on multicore instances for reading and data processing.
- FEATURE: Introduced data dumps to a host filesystem for [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader#vm-reader). Resource-intensive setups (multiple queries returning many metrics, bigger `fit_window` arg) will have RAM consumption reduced during fit calls.
- IMPROVEMENT: Added a `groupby` argument for logical grouping in [multivariate models](https://docs.victoriametrics.com/anomaly-detection/components/models#multivariate-models). When specified, a separate multivariate model is trained for each unique combination of label values in the `groupby` columns. For example, to perform multivariate anomaly detection on metrics at the machine level without cross-entity interference, you can use `groupby: [host]` or `groupby: [instance]`, ensuring one model per entity being trained (e.g., per host). Please find more details [here](https://docs.victoriametrics.com/anomaly-detection/components/models/#group-by).
- IMPROVEMENT: Improved performance of [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader#vm-reader) on multicore instances for reading and data processing.
- IMPROVEMENT: Introduced new CLI argument aliases to enhance compatibility with [Helm charts](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-anomaly/README.md) (i.e. using secrets) and better align with [VictoriaMetrics flags](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#list-of-command-line-flags):
- `--licenseFile` as an alias for `--license-file`
- `--license.forceOffline` as an alias for `--license-verify-offline`
@@ -397,9 +412,9 @@ Released: 2024-08-10
- FEATURE: Introduced backward-compatible `data_range` [query-specific parameter](https://docs.victoriametrics.com/anomaly-detection/components/reader/#per-query-parameters) to the [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader). It enables the definition of **valid** data ranges for input per individual query in `queries`, resulting in:
- **High anomaly scores** (>1) when the *data falls outside the expected range*, indicating a data constraint violation.
- **Lowest anomaly scores** (=0) when the *model's predictions (`yhat`) fall outside the expected range*, signaling uncertain predictions.
- For more details, please refer to the [documentation](https://docs.victoriametrics.com/anomaly-detection/components/reader/#per-query-parameters).
- For more details, please refer to the [documentation](https://docs.victoriametrics.com/anomaly-detection/components/reader/?highlight=data_range#per-query-parameters).
- IMPROVEMENT: Added `latency_offset` argument to the [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) to override the default `-search.latencyOffset` [flag of VictoriaMetrics](https://docs.victoriametrics.com/victoriametrics/#list-of-command-line-flags) (30s). The default value is set to 1ms, which should help in cases where `sampling_frequency` is low (10-60s) and `sampling_frequency` equals `infer_every` in the [PeriodicScheduler](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/#periodic-scheduler). This prevents users from receiving `service - WARNING - [Scheduler [scheduler_alias]] No data available for inference.` warnings in logs and allows for consecutive `infer` calls without gaps. To restore the backward compatible behavior, set it equal to your `-search.latencyOffset` value in [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) config section.
- IMPROVEMENT: Added `latency_offset` argument to the [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) to override the default `-search.latencyOffset` [flag of VictoriaMetrics](https://docs.victoriametrics.com/?highlight=search.latencyOffset#list-of-command-line-flags) (30s). The default value is set to 1ms, which should help in cases where `sampling_frequency` is low (10-60s) and `sampling_frequency` equals `infer_every` in the [PeriodicScheduler](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/?highlight=infer_every#periodic-scheduler). This prevents users from receiving `service - WARNING - [Scheduler [scheduler_alias]] No data available for inference.` warnings in logs and allows for consecutive `infer` calls without gaps. To restore the backward compatible behavior, set it equal to your `-search.latencyOffset` value in [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) config section.
- BUGFIX: Ensure the `use_transform` argument of the [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile) functions as intended.
- BUGFIX: Add a docstring for `query_from_last_seen_timestamp` arg of [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader).
@@ -423,8 +438,8 @@ Released: 2024-07-26
## v1.14.1
Released: 2024-07-26
- FEATURE: Allow to process larger data chunks in [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) that exceed `-search.maxPointsPerTimeseries` [constraint in VictoriaMetrics](https://docs.victoriametrics.com/victoriametrics/#resource-usage-limits) by splitting the range and sending multiple requests. A warning is printed in logs, suggesting reducing the range or step, or increasing `search.maxPointsPerTimeseries` constraint in VictoriaMetrics, which is still a recommended option.
- FEATURE: Backward-compatible redesign of [`queries`](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) arg of [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader). Old format of `{q_alias1: q_expr1, q_alias2: q_expr2, ...}` will be implicitly converted to a new one with a warning raised in logs. New format allows to specify per-query parameters, like `step` to reduce amount of data read from VictoriaMetrics TSDB and to allow config flexibility. Find out more in [Per-query parameters section of VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#per-query-parameters).
- FEATURE: Allow to process larger data chunks in [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader#vm-reader) that exceed `-search.maxPointsPerTimeseries` [constraint in VictoriaMetrics](https://docs.victoriametrics.com/?highlight=search.maxPointsPerTimeseries#resource-usage-limits) by splitting the range and sending multiple requests. A warning is printed in logs, suggesting reducing the range or step, or increasing `search.maxPointsPerTimeseries` constraint in VictoriaMetrics, which is still a recommended option.
- FEATURE: Backward-compatible redesign of [`queries`](https://docs.victoriametrics.com/anomaly-detection/components/reader?highlight=queries#vm-reader) arg of [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader#vm-reader). Old format of `{q_alias1: q_expr1, q_alias2: q_expr2, ...}` will be implicitly converted to a new one with a warning raised in logs. New format allows to specify per-query parameters, like `step` to reduce amount of data read from VictoriaMetrics TSDB and to allow config flexibility. Find out more in [Per-query parameters section of VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#per-query-parameters).
- IMPROVEMENT: Added multi-platform builds for `linux/amd64` and `linux/arm64` architectures.

View File

@@ -47,12 +47,15 @@ Please see example graph illustrating this logic below:
## What data does vmanomaly operate on?
`vmanomaly` operates on data fetched from VictoriaMetrics, where you can leverage full power of [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/) for data selection, sampling, and processing. Users can also [apply global filters](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-querying-api-enhancements) for more targeted data analysis, enhancing scope limitation and tenant visibility.
Respective config is defined in a [`reader`](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) section.
`vmanomaly` operates on timeseries (metrics) data, and supports both **VictoriaMetrics** and **VictoriaLogs** as data sources. Choose the source depending on the use case.
**VictoriaMetrics (metrics):** use full [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/) for selection, sampling, and processing; [global filters](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-querying-api-enhancements) are also supported. See the [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) for the details.
**VictoriaLogs (logs → metrics):** {{% available_from "v1.26.0" anomaly %}} use [LogsQL](https://docs.victoriametrics.com/victorialogs/logsql/) via the [`VLogsReader`](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vlogs-reader) to create log-derived metrics for anomaly detection (e.g., error rates, request latencies).
> Please note that only LogsQL queries with [stats pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe) functions [subset](http://localhost:1313/anomaly-detection/components/reader/#valid-stats-functions) are supported, as they produce **numeric** time series.
## Handling noisy input data
`vmanomaly` operates on data fetched from VictoriaMetrics using [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/) queries, so the initial data quality can be fine-tuned with aggregation, grouping, and filtering to reduce noise and improve anomaly detection accuracy.
## Using offsets
`vmanomaly` supports {{% available_from "v1.25.3" anomaly %}} the use of offsets in the [`reader`](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) section to adjust the time range of the data being queried. This can be particularly useful for correcting for data collection delays or other timing issues. It can be also defined or overridden on [per-query basis](https://docs.victoriametrics.com/anomaly-detection/components/reader/#per-query-parameters).
@@ -110,6 +113,8 @@ To visualize and interact with both [self-monitoring metrics](https://docs.victo
- For guidance on using the `vmanomaly` Grafana dashboard and drilling down into anomaly score visualizations, refer to the [default preset section](https://docs.victoriametrics.com/anomaly-detection/presets/#default).
- To monitor `vmanomaly` health, operational performance, and potential issues in real time, visit the [self-monitoring section](https://docs.victoriametrics.com/anomaly-detection/self-monitoring/).
- {{% available_from "v1.26.0" anomaly %}} For rapid exploration of how different models, their configurations and included domain knowledge impacts the results of anomaly detection, use the built-in [vmanomaly UI](https://docs.victoriametrics.com/anomaly-detection/ui/).
![vmanomaly-ui-overview](vmanomaly-ui-overview.webp)
## Is vmanomaly stateful?
By default, `vmanomaly` is **stateless**, meaning it does not retain any state between service restarts. However, it can be configured {{% available_from "v1.24.0" anomaly %}} to be **stateful** by enabling the `restore_state` setting in the [settings section](https://docs.victoriametrics.com/anomaly-detection/components/settings/). This allows the service to restore its state from a previous run (training data, trained models), ensuring that models continue to produce [anomaly scores](#what-is-anomaly-score) right after restart and without requiring a full retraining process or re-querying training data from VictoriaMetrics. This is particularly useful for long-running services that need to maintain continuity in anomaly detection without losing previously learned patterns, especially when using [online models](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-models) that continuously adapt to new data and update their internal state. Also, [hot-reloading](https://docs.victoriametrics.com/anomaly-detection/components/#hot-reload) works well with state restoration, allowing for on-the-fly configuration changes without losing the current state of the models and reusing unchanged models/data/scheduler combinations.
@@ -219,6 +224,9 @@ Produced anomaly scores are designed in such a way that values from 0.0 to 1.0 i
## How to backtest particular configuration on historical data?
> {{% available_from "v1.26.0" anomaly %}} You can use the [vmanomaly UI](https://docs.victoriametrics.com/anomaly-detection/ui/) to backtest particular configuration on historical data with easy-to-use [preset](https://docs.victoriametrics.com/anomaly-detection/presets/#ui).
![vmanomaly-ui-overview](vmanomaly-ui-overview.webp)
Anomaly scores for historical ([backtesting](https://wikipedia.org/wiki/Backtesting)) data can be produced using [`backtesting` scheduler](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/#backtesting-scheduler) in `vmanomaly` config. This scheduler allows you to define a historical period for which the models will be trained and then used to produce anomaly scores, imitating the behavior of the [PeriodicScheduler](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/#periodic-scheduler) for that period. Especially useful for testing new models or configurations on historical data before deploying them in production, around labelled incidents.
```yaml
@@ -389,7 +397,7 @@ services:
# ...
vmanomaly:
container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.25.3
image: victoriametrics/vmanomaly:v1.26.0
# ...
ports:
- "8490:8490"
@@ -574,6 +582,8 @@ For **horizontal** scalability, `vmanomaly` can be deployed as multiple independ
### Splitting the config
> Use this approach for versions older than `v1.21.0` (or if you prefer manual control over the splitting process). Prefer the newer approach described on [dedicated page](https://docs.victoriametrics.com/anomaly-detection/scaling-vmanomaly/) for horizontal scalability and high availability.
CLI utility named `config_splitter` is available in `vmanomaly` {{% available_from "v1.18.5" anomaly %}}. The config splitter tool enables splitting a parent vmanomaly YAML configuration file into multiple sub-configurations based on logical entities such as `schedulers`, `queries`, `models`, `extra_filters` and `complete` {{% available_from "v1.19.2" anomaly %}}. The resulting sub-configurations are fully validated, functional, account for many-to-many relationships between models and their associated queries, and the schedulers they are linked to. These sub-configurations can then be saved to a specified directory for further use:
```shellhelp
@@ -602,7 +612,7 @@ options:
Heres an example of using the config splitter to divide configurations based on the `extra_filters` argument from the reader section:
```sh
docker pull victoriametrics/vmanomaly:v1.25.3 && docker image tag victoriametrics/vmanomaly:v1.25.3 vmanomaly
docker pull victoriametrics/vmanomaly:v1.26.0 && docker image tag victoriametrics/vmanomaly:v1.26.0 vmanomaly
```
```sh

View File

@@ -22,22 +22,31 @@ Each preset, including the [default](#default), comes with premade downstream as
**Available presets:**
- [UI](#ui)
- [Default](#default)
- [Node-Exporter](#node-exporter)
To enable preset mode, `preset` arg should be set to particular preset name:
```yaml
preset: "chosen_preset_name" # i.e. "node-exporter"
preset: "chosen_preset_name" # i.e. "node-exporter", "ui", etc.
```
Also, additional minimal set of arguments may be required from user to run the preset. See corresponding preset sections below for the details.
Run a service using config file with one of the [available options](https://docs.victoriametrics.com/anomaly-detection/quickstart/#how-to-install-and-run-vmanomaly).
After you run `vmanomaly` with `preset` arg specified, available assets can be viewed, copied and downloaded at `http://localhost:8490/presets/` endpoint.
## UI
![preset-localhost](presets-localhost.webp)
The UI preset is the easiest to start from and is designed for users who want to quickly experiment with `vmanomaly` in [UI mode](https://docs.victoriametrics.com/anomaly-detection/ui/). It requires minimal configuration and provides a user-friendly interface for exploring anomaly detection capabilities:
```yaml
preset: "ui"
```
Please refer to the [UI documentation](https://docs.victoriametrics.com/anomaly-detection/ui/) for detailed instructions on how to use the interface and [backtest your anomaly detection configurations guide](https://docs.victoriametrics.com/anomaly-detection/ui/#example-usage).
![vmanomaly-ui-overview](vmanomaly-ui-overview.webp)
## Default

View File

@@ -121,13 +121,13 @@ Below are the steps to get `vmanomaly` up and running inside a Docker container:
1. Pull Docker image:
```sh
docker pull victoriametrics/vmanomaly:v1.25.3
docker pull victoriametrics/vmanomaly:v1.26.0
```
2. (Optional step) tag the `vmanomaly` Docker image:
```sh
docker image tag victoriametrics/vmanomaly:v1.25.3 vmanomaly
docker image tag victoriametrics/vmanomaly:v1.26.0 vmanomaly
```
3. Start the `vmanomaly` Docker container with a *license file*, use the command below.
@@ -163,7 +163,7 @@ docker run -it --user 1000:1000 \
services:
# ...
vmanomaly:
image: victoriametrics/vmanomaly:v1.25.3
image: victoriametrics/vmanomaly:v1.26.0
volumes:
$YOUR_LICENSE_FILE_PATH:/license
$YOUR_CONFIG_FILE_PATH:/config.yml
@@ -292,6 +292,8 @@ For optimal service behavior, consider the following tweaks when configuring `vm
- Define queries for input data using [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/) under `reader.queries` section. Note, it's possible to override reader-level arguments at query level for increased flexibility, e.g. specifying per-query [timezone](https://docs.victoriametrics.com/anomaly-detection/faq/#handling-timezones) or [sampling period](https://docs.victoriametrics.com/anomaly-detection/components/reader/#sampling-period).
- For longer `fit_window` intervals in scheduler, consider splitting queries into smaller time ranges to avoid excessive memory usage, timeouts and hitting server-side constraints, so they can be queried separately and reconstructed on `vmanomaly` side. Please refer to this [example](https://docs.victoriametrics.com/anomaly-detection/faq/#handling-large-queries-in-vmanomaly) for more details.
> If applicable - consider trying [`VLogsReader`](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vlogs-reader) {{% available_from "v1.26.0" anomaly %}} to perform anomaly detection on log-derived **metrics**. This is particularly useful for scenarios where log data needs to be analyzed for unusual patterns or behaviors, such as error rates or request latencies.
**Writer**:
- Specify where and how to store anomaly detection metrics in the [writer](https://docs.victoriametrics.com/anomaly-detection/components/writer/) section.
- Include tenant ID if using a [cluster version of VictoriaMetrics](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/) for writing the results.

View File

@@ -6,20 +6,21 @@ build:
sitemap:
disable: true
---
In today's fast-paced and complex landscape of system monitoring, [VictoriaMetrics Anomaly Detection](https://victoriametrics.com/products/enterprise/anomaly-detection/) (`vmanomaly`), part of our [Enterprise offering](https://victoriametrics.com/products/enterprise/), serves as a **powerful observability tool** for SREs and DevOps teams. It **automates the detection of anomalies in time-series data**, reducing manual efforts required to identify abnormal system behavior.
In today's fast-paced and complex landscape of system monitoring, [VictoriaMetrics Anomaly Detection](https://victoriametrics.com/products/enterprise/anomaly-detection/) (`vmanomaly`), a part of our [Enterprise offering](https://victoriametrics.com/products/enterprise/), serves as an **observability layer** for SREs and DevOps teams atop of collected data to **automate the detection of anomalies in time-series data**, reducing manual efforts required to identify abnormal system behavior.
Unlike traditional threshold-based alerting, which relies on **raw metric values** and requires constant tuning and maintenance of thresholds and alerting rules, `vmanomaly` introduces a **unified, interpretable [anomaly score](https://docs.victoriametrics.com/anomaly-detection/faq/#what-is-anomaly-score)** - a **de-trended, de-seasonalized metric** generated through machine learning. This approach eliminates the need for frequent manual adjustments by enabling **stable, long-term static thresholds (as simple as `anomaly_score > 1`)** that remain effective over time through continuous model retraining.
By shifting to anomaly-based detection, teams can **identify and respond to potential issues faster**, enhancing system reliability and operational efficiency while significantly **reducing the engineering effort spent on maintaining alerting rules**.
By shifting to anomaly-based detection, teams can **identify and respond to potential issues faster**, enhancing system reliability and operational efficiency while significantly **reducing the engineering effort spent on handcrafting and maintaining alerting rules**.
## What does it do?
`vmanomaly` is designed to **periodically analyze new data points** across selected metrics, generating a **unified metric** called [anomaly score](https://docs.victoriametrics.com/anomaly-detection/faq/#what-is-anomaly-score).
`vmanomaly` is designed to **periodically analyze new data points** across selected metrics (either requested from [VictoriaMetrics TSDB](https://docs.victoriametrics.com/victoriametrics/) or produced by [VictoriaLogs](https://docs.victoriametrics.com/victorialogs/) metrics [endpoint](https://docs.victoriametrics.com/victorialogs/querying/#querying-log-range-stats)), generating a **unified metric** called [anomaly score](https://docs.victoriametrics.com/anomaly-detection/faq/#what-is-anomaly-score).
Key functions:
- **Automated anomaly detection** continuously scans time-series data to identify deviations from expected behavior.
- **Seamless integration** anomaly scores are stored in VictoriaMetrics TSDB for use in **alerting, visualization, and downstream analytics**.
- **Automated anomaly detection** - continuously scans time-series data to identify deviations from expected behavior.
- **Seamless integration** - anomaly scores are stored in VictoriaMetrics TSDB for use in **alerting, visualization, and downstream analytics**.
The diagram below illustrates how `vmanomaly` fits into an observability setup, such as detecting anomalies in metrics collected by `node_exporter`:

View File

@@ -0,0 +1,326 @@
---
weight: 2
title: UI
menu:
docs:
parent: "anomaly-detection"
identifier: "vmanomaly-ui"
weight: 2
title: UI
tags:
- metrics
- enterprise
- ui
aliases:
- /anomaly-detection/ui.html
- /anomaly-detection/ui
- /anomaly-detection/UI
- /anomaly-detection/UI/
---
## Introduction
{{% available_from "v1.26.0" anomaly %}} `vmanomaly` is shipped with a built-in [vmui-like](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui) [UI](https://en.wikipedia.org/wiki/Graphical_user_interface) that provides an intuitive interface for rapid exploration of how anomaly detection models, their configurations and included domain knowledge impacts the results of anomaly detection, before such configurations are deployed in production.
![vmanomaly-ui-overview](vmanomaly-ui-overview.webp)
## Accessing the UI
The UI is available at `http://<vmanomaly-host>:8490` by default, however, the port can be changed in `server` section of the [configuration file](https://docs.victoriametrics.com/anomaly-detection/components/) using the `port` parameter:
```yaml
server:
port: 8490
# other vmanomaly configuration sections, e.g. schedulers, models, reader, writer, etc.
```
For impactful parameters please refer to [optimize resource usage](#optimize-resource-usage) section of this page.
## Preset
Vmanomaly can be deployed in efficient "UI mode" [preset](https://docs.victoriametrics.com/anomaly-detection/presets/#ui), with as simple configuration as:
```yaml
preset: ui
```
This will start vmanomaly instance with a no-operation schedulers, readers, models and the other components, and **will only enable the UI for interactive exploration of anomaly detection models**, so all the resources are dedicated to the UI and no resources are wasted on other components, such as production jobs of anomaly detection (retrieving metrics, running models, writing results, etc.).
The best applications of this mode are:
- Deploy vmanomaly UI as a service to hide the deployment complexity and allow internal users to explore anomaly detection models and their configurations, without the need to run any production jobs of anomaly detection.
- When there is a quick need to explore and validate anomaly detection models and their configurations before deploying them in production jobs, at minimal resource usage.
> However, the UI can be **combined with existing production jobs of anomaly detection, as it is available in non-blocking mode for all running vmanomaly instances** {{% available_from "v1.26.0" anomaly %}}, regardless of the preset or configuration used, just at a cost of increased resource usage.
## UI Navigation
The vmanomaly UI provides a user-friendly interface for exploring and configuring anomaly detection models. The main components of the UI include:
- [**Query Explorer**](#query-explorer): A vmui-like interface for typing and executing MetricsQL/LogsQL queries to visualize data.
- [**Model Panel**](#model-panel): A form for editing anomaly detection model hyperparameters and applying domain knowledge settings.
- [**Visualization Panel**](#visualization-panel): A space for visualizing model performance and anomalies.
- [**Settings Panel**](#settings-panel): A section for configuring global settings and preferences for the vmanomaly UI, including data source configuration.
[Back to UI navigation](#ui-navigation)
### Query Explorer
The Query Explorer provides a vmui-like interface for typing and executing MetricsQL/LogsQL queries to visualize data.
![vmanomaly-ui-sections-explore](vmanomaly-ui-sections-explore.webp)
Users can:
- Enter, autocomplete, prettify and execute queries to retrieve and plot the data from the configured data source (see [settings panel](#settings-panel) for data source configuration).
- Adjust the (inference) time range and resolution (step) for data visualization and anomaly detection purposes.
- Access query history and saved queries for quick access to frequently used queries.
- Switch tenants (if data source supports multi-tenancy) and access [settings panel](#settings-panel) for global UI configuration.
[Back to UI navigation](#ui-navigation)
### Visualization Panel
The Visualization Panel has 2 modes of displaying data - either raw queried data or data with detected anomalies, depending on the action taken in the Model Panel.
**Visualizations of the queried data** ("Execute Query" button)
![vmanomaly-ui-sections-plot-area-query-mode](vmanomaly-ui-sections-plot-area-query-mode.webp)
> All the metrics are shown in a single plot, similar to vmui, with zooming and panning capabilities.
**Initial data with detected anomalies** ("Detect Anomalies" button)
![vmanomaly-ui-sections-plot-area-detect-mode](vmanomaly-ui-sections-plot-area-detect-mode.webp)
> The plot shows the queried data, **grouped by individual series**, iterated over legend, with the actual values (`y`) compared to the expected values (model predictions, `y_hat`), confidence intervals (`y_hat_lower`, `y_hat_upper`), and detected anomalies. The anomalies are marked with red circles, and hovering over them provides additional information such as the anomaly score and associated labels.
Also, timeseries (such as `y`, `y_hat`, etc.) can be toggled on/off by clicking on the legend items.
[Back to UI navigation](#ui-navigation)
### Model Panel
![vmanomaly-ui-sections-model](vmanomaly-ui-sections-model.webp)
The Model Panel provides:
Parameters, such as "Fit Every" and "Fit Window", to control how often and over what time window the model is retrained on new data to imitate production behavior, as well as overriding default anomaly detection thresholds (1.0).
Controls for running/canceling anomaly detection on the queried data, downloading the results as CSV/JSON, accessing and downloading the model configuration in YAML format.
A form-based menu for finetuning model hyperparameters and applying domain knowledge settings:
- Model type selection (e.g., rolling quantile, Prophet, etc.)
![vmanomaly-ui-model-config-menu](vmanomaly-ui-model-config-menu.webp)
- Wizard with **model-agnostic parameters** (e.g., detection direction, data range, scale, clipping, minimum deviation from expected, etc.) and **model-specific hyperparameters** for chosen model type (e.g., quantile and window steps for [rolling quantile](https://docs.victoriametrics.com/anomaly-detection/components/models/#rolling-quantile) model).
![vmanomaly-ui-model-config-wizard](vmanomaly-ui-model-config-wizard.webp)
[Back to UI navigation](#ui-navigation)
### Settings Panel
The vmui-like "Settings" panel allows users to configure global settings and preferences for the vmanomaly UI, including:
- Server URL (vmanomaly)
- Datasource URL (VictoriaMetrics, VictoriaLogs)
- Timezone
- UI Theme
![vmanomaly-ui-sections-settings](vmanomaly-ui-sections-settings.webp)
[Back to navigation](#ui-navigation)
## Configuration Sharing
Based on the needs, either
- Full UI state can be [shared via URL](#url-sharing)
- Or model part / full service configuration can be [viewed and exported in production-ready YAML format](#yaml-configuration).
### URL Sharing
Similarly to vmui, vmanomaly UI supports **configuration sharing via URL**. This allows users to share their UI state (including queries, time ranges, model type and hyperparameters, and other settings) by copying and sharing the URL from the browser's address bar.
![vmanomaly-ui-state-sharing-url](vmanomaly-ui-state-sharing-url.webp)
<p></p>
<p>Example URL content:</p>
```shellhelp
http://localhost:8490/vmui/#/?anomaly_threshold=1.0&fit_window=1d&fit_every=7d&g0.range_input=7d&g0.end_input=2025-09-30T16%3A56%3A13&g0.relative_time=last_7_days&g0.tab=0&g0.tenantID=0&datasourceUrl=https%3A%2F%2Fplay.victoriametrics.com%2Fselect%2F0%2Fprometheus&g0.expr=sum%28rate%28node_cpu_seconds_total%7Bmode%3D%7E%22%28softirq%7Cuser%7Ciowait%29%22%7D%5B10m%5D%29%29+by+%28container%2C+mode%29&g0.step_input=30m&model_config=%257B%2522modelType%2522%253A%2522rolling_quantile%2522%252C%2522settings%2522%253A%257B%2522detection_direction%2522%253A%2522above_expected%2522%252C%2522data_range%2522%253A%255B0%252C100%255D%252C%2522scale%2522%253A%255B1%252C1%255D%252C%2522clip_predictions%2522%253Atrue%252C%2522min_dev_from_expected%2522%253A%255B0%252C6%255D%252C%2522anomaly_score_outside_data_range%2522%253A1.01%252C%2522quantile%2522%253A0.9%252C%2522window_steps%2522%253A48%257D%252C%2522modelSpec%2522%253A%257B%2522class_name%2522%253A%2522rolling_quantile%2522%252C%2522class%2522%253A%2522model.rolling_quantile.RollingQuantileModel%2522%252C%2522detection_direction%2522%253A%2522above_expected%2522%252C%2522data_range%2522%253A%255B0%252C100%255D%252C%2522scale%2522%253A%255B1%252C1%255D%252C%2522clip_predictions%2522%253Atrue%252C%2522min_dev_from_expected%2522%253A%255B0%252C6%255D%252C%2522anomaly_score_outside_data_range%2522%253A1.01%252C%2522window_steps%2522%253A48%252C%2522quantile%2522%253A0.9%257D%252C%2522isValidated%2522%253Atrue%257D
```
### YAML Configuration
Once the configuration is set up and saved in the UI (selected model type and validated hyperparameters), equivalent configuration in production-ready YAML format can be obtained by:
Accessing the "YAML" Tab in the model configuration section
![vmanomaly-ui-model-config-menu](vmanomaly-ui-model-config-menu.webp)
![vmanomaly-ui-model-config-yaml-tab](vmanomaly-ui-model-config-menu-yaml-tab.webp)
Clicking the "Open Config" button to access (model-only or full) configuration and hitting "Download" button to get the configuration as a YAML file.
![vmanomaly-ui-open-config-btn](vmanomaly-ui-open-config-btn.webp)
![vmanomaly-ui-open-config-menu](vmanomaly-ui-open-config-menu.webp)
## Optimize Resource Usage
Based on expected usage patterns (quick experiments, internal team serving, number of users, etc.) it is recommended to tune resource usage by adjusting the following parameters in the `server` and `settings` sections of the configuration file:
```yaml
server:
# Port for the UI server (default: 8490)
port: 8490
# Limit on concurrent tasks to manage UI load (default: 2)
max_concurrent_tasks: 2
settings:
# Number of workers for single job speed-ups (default: 1)
n_workers: 2
# Adjust logging levels to reduce verbosity
logger_levels:
vmanomaly: WARNING
reader: INFO
model: INFO
# ...
```
`max_concurrent_tasks` limits the number of concurrent tasks (e.g. running anomaly detection on queried data) to avoid overloading the UI when multiple users are using it simultaneously, and `n_workers` [option](https://docs.victoriametrics.com/anomaly-detection/components/settings/#parallelization) controls the number of workers for parallel processing of single tasks (e.g. running anomaly detection on queried data faster).
Please note that numbers <= 0 for `n_workers` mean number of available CPU cores, and these workers would be shared with other components of vmanomaly, such as production jobs of anomaly detection (retrieving metrics, running models, writing results, etc.), if they are enabled.
### Preset Usage
If using the `ui` preset, just specify:
```yaml
preset: ui
server:
# Port for the UI server (default: 8490)
port: 8490
# Limit on concurrent tasks to manage UI load (default: 2)
max_concurrent_tasks: 5
settings:
# Number of workers for single job speed-ups (default: 1)
n_workers: 2
# Adjust logging levels to reduce verbosity
logger_levels:
vmanomaly: WARNING
# ...
```
in instance configuration file, it will be recursively merged with preset pre-filled configuration to produce the final configuration.
### Mixed Usage
If mixing the UI alongside production jobs of anomaly detection is expected, it is recommended to adjust the parameters to optimize resource usage, as well as configuring the `reader` and `writer` components appropriately for the production jobs.
```yaml
settings:
# Number of workers for mixed usage (numbers <= 0 mean number of available CPU cores)
n_workers: -1
# Adjust logging levels to reduce verbosity
logger_levels:
vmanomaly: WARNING
reader: INFO
model: INFO
writer: INFO
# ...
server:
# Port for the UI server (default: 8490)
port: 8490
# Limit on concurrent tasks to manage UI load (default: 2)
max_concurrent_tasks: 5
# other production components, e.g. schedulers, models, reader, writer, etc.
```
## Example Usage
Run vmanomaly in `ui` preset mode (experimenting) with a configuration file `config.yaml` (see [preset usage](#preset-usage) section for details):
```yaml
preset: ui
# other optional server/settings parameters, e.g. port, max_concurrent_tasks, n_workers, logger_levels, etc.
```
using one of the [deployment methods](https://docs.victoriametrics.com/anomaly-detection/quickstart/#how-to-install-and-run-vmanomaly) in a [QuickStart guide](https://docs.victoriametrics.com/anomaly-detection/quickstart/#quickstart), e.g. via Docker.
Retrieve the UI at `http://<vmanomaly-host>:<port>` (e.g. at `http://localhost:8490` if running locally with default port) and start exploring anomaly detection models and their configurations interactively.
### Explore Input Data
Start by entering valid MetricsQL/LogsQL queries in the Query Explorer to visualize the data from the configured data source (see [settings panel](#settings-panel) for data source configuration) - that's the data on which anomaly detection models will be applied.
Set appropriate tenants (if data source supports multi-tenancy) and access [settings panel](#settings-panel) for global UI configuration, if needed.
Set up the time range and resolution (step) for data visualization and anomaly detection purposes - e.g. last 7 days with 30m step, especially if the data has daily/weekly seasonality. Also, set the step according to the desired granularity of anomaly detection results (e.g. 30m step for 30m granularity) which itself is based on alerting needs and latency requirements.
![vmanomaly-ui-sections-explore](vmanomaly-ui-sections-explore.webp)
Pay attention to trends, seasonality, noise, outliers, and other patterns in the data, which can influence the choice of anomaly detection model and its hyperparameters (e.g. use seasonal models for seasonal data - like `Prophet`, robust models for noisy de-seasonalized data - like `MAD`, etc.).
![vmanomaly-ui-sections-plot-area-query-mode](vmanomaly-ui-sections-plot-area-query-mode.webp)
### Select and Configure Model
Choose an appropriate anomaly detection model from the Model Panel based on the characteristics of the data and the specific requirements (domain knowledge) of the use case.
![vmanomaly-ui-sections-model](vmanomaly-ui-sections-model.webp)
Set the "Fit Every" and "Fit Window" parameters to control how often and over what time window the model is retrained on new data to imitate production behavior - e.g. fit every 7 days on a rolling window of last 14 days.
Tune the model hyperparameters and apply domain knowledge settings using the form-based menu in the Model Panel. See (i) tooltips for parameter descriptions and [model documentation](https://docs.victoriametrics.com/anomaly-detection/components/models/) link for recommended values and guidelines.
![vmanomaly-ui-model-config-wizard](vmanomaly-ui-model-config-wizard.webp)
For example, for a `rolling quantile` [model](https://docs.victoriametrics.com/anomaly-detection/components/models/#rolling-quantile), that should be run on a query, returning per-mode CPU utilization (as fractions of 1, data range `[0, 1]`), where you are interested in capturing **spikes of at least 5% deviations** from expected behavior:
Set the **model-agnostic** parameters to encode the domain knowledge:
- [detection direction](https://docs.victoriametrics.com/anomaly-detection/components/models/#detection-direction) to `above expected` for capturing high anomalies (spikes)
- data range to `[0, 1]` for CPU utilization fractions and proper prediction clipping
- [minimum deviation from expected](https://docs.victoriametrics.com/anomaly-detection/components/models/#minimal-deviation-from-expected) to `[0, 0.05]` for capturing spikes of at least 5% deviations in magnitude
- [anomaly score outside data range](https://docs.victoriametrics.com/anomaly-detection/components/models/#score-outside-data-range) to `5.0` for automatically marking values outside data range as anomalies (e.g. to catch improper query returning CPU > 100% or < 0%)
- [clip predictions](https://docs.victoriametrics.com/anomaly-detection/components/models/#clip-predictions) to `true` for avoiding nonsensical predictions outside data range
- [scale](https://docs.victoriametrics.com/anomaly-detection/components/models/#scale) to `[1, 1]` unless you will see improper width of confidence intervals from the experiments (e.g. too wide/narrow) given the model-specific hyperparameters and false positives/negatives observed in the results.
Set the **model-specific** hyperparameters:
- quantile to `0.9` for detecting high anomalies
- window steps to `48` for capturing 1 day of active history with 30m step data (24/0.5 = 48)
Check the parameters for validity and consistency by hitting the "Validate" button, which will also provide warnings if some parameters seem inconsistent (e.g. quantile is set outside of `[0, 1]` range or window steps is set to a negative value), then hit "Save" to apply the changes.
### Detect Anomalies
Hit the "Detect Anomalies" button to run anomaly detection on the queried data with the selected model and its configuration.
> If needed, the running task can be canceled by hitting the "Cancel" button.
![vmanomaly-ui-sections-plot-area-detect-mode](vmanomaly-ui-sections-plot-area-detect-mode.webp)
Iterate over the legend to view **individual output series** (e.g. actual values, expected values, confidence intervals, anomalies, etc.) for different series returned by the query.
Iterate over **returned timeseries by query** to see how the model performed on different series, and whether the detected anomalies make sense in the context of the data and the use case (their % of total points, magnitude, duration, etc.).
Until satisfied with the results, finetune the model hyperparameters and domain knowledge settings in the Model Panel (or change the model type), and rerun anomaly detection on the queried data.
Once satisfied with the results, do the following.
> Many tabs can be opened in parallel in the browser, so multiple experiments can be run in parallel, and the results can be compared by switching between tabs. Local settings (e.g. queries, time ranges, model configurations, etc.) are preserved in each tab, so different experiments do not interfere with each other.
### Work with Results
If the **results** do not look good, the model hyperparameters and domain knowledge settings can be further finetuned in the Model Panel, and anomaly detection can be [rerun on the queried](#detect-anomalies) data until satisfactory results are achieved.
If the **results** look good, but should be shared with others first, timeseries can be downloaded as files by hitting the respective button in the Model Panel. See also [configuration sharing](#configuration-sharing) section for details.
If the **results** look good and the **model configuration should be deployed in production jobs of anomaly detection**, the equivalent configuration in production-ready YAML format can be obtained by accessing the "YAML" Tab in the model configuration section and hitting the "Open Config" button to access (model-only or full) configuration and hitting "Download" button to get the configuration as a YAML file.
![vmanomaly-ui-open-config-menu](vmanomaly-ui-open-config-menu.webp)

View File

@@ -90,6 +90,8 @@ reader:
sampling_period: "30s" # what data resolution to fetch from VictoriaMetrics' /query_range endpoint
latency_offset: '1ms'
query_from_last_seen_timestamp: False
tz: "UTC" # timezone to use for queries without explicit timezone
"offset": "0s" # offset to apply to all queries, e.g. to account for data delays, can be overridden on per-query basis
queries: # aliases to MetricsQL expressions
cpu_seconds_total:
expr: 'avg(rate(node_cpu_seconds_total[5m])) by (mode)'

View File

@@ -1312,7 +1312,7 @@ monitoring:
Let's pull the docker image for `vmanomaly`:
```sh
docker pull victoriametrics/vmanomaly:v1.25.3
docker pull victoriametrics/vmanomaly:v1.26.0
```
Now we can run the docker container putting as volumes both config and model file:
@@ -1326,8 +1326,9 @@ docker run -it \
-v $(PWD)/license:/license \
-v $(PWD)/custom_model.py:/vmanomaly/model/custom.py \
-v $(PWD)/custom.yaml:/config.yaml \
victoriametrics/vmanomaly:v1.25.3 /config.yaml \
victoriametrics/vmanomaly:v1.26.0 /config.yaml \
--licenseFile=/license
--watch
```
Please find more detailed instructions (license, etc.) [here](https://docs.victoriametrics.com/anomaly-detection/quickstart/#docker)

View File

@@ -146,6 +146,7 @@ Name of the class needed to enable reading from VictoriaMetrics or Prometheus. V
<span style="white-space: nowrap;">`queries`</span>
</td>
<td>
See [per-query config example](#per-query-config-example) above
</td>
<td>
@@ -226,7 +227,7 @@ Absolute or relative URL address where to check availability of the datasource.
`USERNAME`
</td>
<td>
BasicAuth username
BasicAuth username. If set, it will be used to authenticate the request.
</td>
</tr>
<tr>
@@ -239,7 +240,7 @@ BasicAuth username
`PASSWORD`
</td>
<td>
BasicAuth password
BasicAuth password. If set, it will be used to authenticate the request.
</td>
</tr>
<tr>
@@ -371,7 +372,7 @@ It allows overriding the default `-search.latencyOffset`{{% available_from "v1.1
`10000`
</td>
<td>
Optional arg{{% available_from "v1.17.0" anomaly %}} overrides how `search.maxPointsPerTimeseries` flag{{% available_from "v1.14.1" anomaly %}} impacts `vmanomaly` on splitting long `fit_window` [queries](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) into smaller sub-intervals. This helps users avoid hitting the `search.maxQueryDuration` limit for individual queries by distributing initial query across multiple subquery requests with minimal overhead. Set less than `search.maxPointsPerTimeseries` if hitting `maxQueryDuration` limits. You can also set it on [per-query](#per-query-parameters) basis to override this global one.
Optional arg{{% available_from "v1.17.0" anomaly %}} overrides how `search.maxPointsPerTimeseries` flag{{% available_from "v1.14.1" anomaly %}} impacts `vmanomaly` on splitting long `fit_window` [queries](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) into smaller sub-intervals. This helps users avoid hitting the `search.maxQueryDuration` limit for individual queries by distributing initial query across multiple subquery requests with minimal overhead. Set less than `search.maxPointsPerTimeseries` if hitting `maxQueryDuration` limits. Can be also set on [per-query](#per-query-parameters) basis to override reader-level settings.
</td>
</tr>
<tr>
@@ -477,3 +478,330 @@ reader:
### Healthcheck metrics
`VmReader` exposes [several healthchecks metrics](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#reader-behaviour-metrics).
## VictoriaLogs reader
{{% available_from "v1.26.0" anomaly %}} `vmanomaly` adds support for reading data from [VictoriaLogs stats queries](https://docs.victoriametrics.com/victorialogs/querying/#querying-log-range-stats) endpoint with `VLogsReader`. This reader allows quering and analyzing log data stored in VictoriaLogs, enabling anomaly detection on metrics generated from logs.
Its queries should be expressed in a subset of [LogsQL](https://docs.victoriametrics.com/victorialogs/logsql), which is similar to MetricsQL/PromQL but adapted for log data.
> Please be aware that `VLogsReader` is designed to work with a `/select/stats_query_range` endpoint of [VictoriaLogs](https://docs.victoriametrics.com/victorialogs/), so the `<query>` expressions must contain `stats` [pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe) (see [query-examples](#query-examples) section below). The calculated stats is converted into metrics with labels from `by(...)` clause of the `| stats by(...)` pipe, where `stats_func*` is any of the supported [stats function subset](#valid-stats-functions) of [available stats functions](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe-functions), while the `result_name*` is the name of the log field to store the result of the corresponding stats function. The `as` keyword is optional.
### Valid stats functions
`VLogsReader` relies on [stats pipe functions](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe-functions) that return **numeric values**, which can be used for anomaly detection on timeseries (metrics). The future addition of similar stats functions in VictoriaLogs will be supported automatically, as long as they return **numeric values**.
The supported stats functions currently include:
- `avg` - returns the average value over the given numeric [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
- `count` - returns the number of log entries.
- `count_empty` - returns the number logs with empty [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
- `count_uniq` - returns the number of unique non-empty values for the given [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
- `count_uniq_hash` - returns the number of unique hashes for non-empty values at the given [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
- `max` - returns the maximum value over the given [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
- `median` - returns the [median](https://en.wikipedia.org/wiki/Median) value over the given [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
- `min` - returns the minimum value over the given [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
- `quantile` - returns the given quantile for the given [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
- `rate` - returns the average per-second rate of matching logs on the selected time range.
- `rate_sum` - returns the average per-second rate of sum for the given [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
- `sum` - returns the sum for the given numeric [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
- `sum_len` - returns the sum of lengths for the given [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
### Query Examples
> You can test your LogsQL queries with stats pipe functions using our [VictoriaLogs playground](https://play-vmlogs.victoriametrics.com/). Use either UI to access graphical results or the `/select/logsql/stats_query_range` endpoint to run your queries and see the raw results, e.g. as this [sample query](https://play-vmlogs.victoriametrics.com/select/logsql/stats_query_range?query=_time%3A5m%20%7C%20stats%20by%20%28_stream%29%20count%28%29%20as%20sample_row&step=1m).
Here are examples of simple valid LogsQL queries with stats pipe functions that can be used with `VLogsReader`.
The following query returns the average value for the duration field over logs matching the [filter](https://docs.victoriametrics.com/victorialogs/logsql/#filters) for `error` [word](https://docs.victoriametrics.com/victorialogs/logsql/#word):
```
error | stats avg(duration) as avg_error_duration
```
It is possible to calculate the average over fields with common prefix via `avg(prefix*)` syntax. For example, the following query calculates the number of logs with `foo` prefix having `error` [word](https://docs.victoriametrics.com/victorialogs/logsql/#word):
```
error | stats count(foo*) as foo_error_count
```
### Config parameters
<table class="params">
<thead>
<tr>
<th>Parameter</th>
<th>Example</th>
<th><span style="white-space: nowrap;">Description</span></th>
</tr>
</thead>
<tbody>
<tr>
<td>
<span style="white-space: nowrap;">`class`</span>
</td>
<td>
`vlogs`
</td>
<td>
The class name of the reader, must be `vlogs` (or `reader.vlogs.VLogsReader`).
</td>
</tr>
<tr>
<td>
`queries`
</td>
<td>
See [per-query config example](#per-query-config-example-1) below
</td>
<td>
Dictionary of queries. Keys are query aliases, values are LogsQL queries to select data in format: `QUERY_ALIAS:<query>`, as accepted by `/select/logsql/stats_query_range?query=%s` VictoriaLogs endpoint. The `<query>` must contain `stats` [pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe-functions). The calculated stats is converted into metrics with labels from `by(...)` clause of the `| stats by(...)` pipe. Only functions returning numeric values are supported, e.g. `count()`, `sum()`, `avg()`, `count_uniq()`, `median()`, `quantile()`, etc.
</td>
</tr>
<tr>
<td>
`datasource_url`
</td>
<td>
`https://play-vmlogs.victoriametrics.com/`
</td>
<td>
URL address of the VictoriaLogs datasource. Must be a valid URL.
</td>
</tr>
<tr>
<td>
`tenant_id`
</td>
<td>
`0:0`
</td>
<td>
Tenants are identified by `accountID` or `accountID:projectID`. See VictoriaLogs [multitenancy docs](https://docs.victoriametrics.com/victorialogs/#multitenancy).
</td>
</tr>
<tr>
<td>
`sampling_period`
</td>
<td>
`1m`
</td>
<td>
Frequency of the points returned. Will be converted to `/select/stats_query_range?step=%s` param (in seconds).
</td>
</tr>
<tr>
<td>
<span style="white-space: nowrap;">`data_range`</span>
</td>
<td>
`[0, 'inf']`
</td>
<td>
(Optional) Allows defining **valid** data ranges for input of all the queries in `queries`. Defaults to `["-inf", "inf"]` if not set and can be overridden on a [per-query basis](#per-query-parameters-1).
</td>
</tr>
<tr>
<td>
`tz`
</td>
<td>
`America/New_York`
</td>
<td>
(Optional) Specifies the [IANA](https://nodatime.org/TimeZones) timezone to account for local shifts, like [DST](https://en.wikipedia.org/wiki/Daylight_saving_time), in models sensitive to seasonal patterns (e.g., [`ProphetModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) or [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile)). Defaults to `UTC` if not set and can be overridden on a [per-query basis](#per-query-parameters).
</td>
</tr>
<tr>
<td>
<span style="white-space: nowrap;">`max_points_per_query`</span>
</td>
<td>
`10000`
</td>
<td>
(Optional) For splitting long `fit_window` [queries](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vlogs-reader) into smaller sub-intervals. This helps users avoid hitting the timeout limits for individual queries by distributing initial query across multiple subquery requests with minimal overhead. Can be also set on [per-query](#per-query-parameters-1) basis to override reader-level settings.
</td>
</tr>
<tr>
<td>
<span style="white-space: nowrap;">`offset`</span>
</td>
<td>
`0s`
</td>
<td>
(Optional) Specifies the duration to shift the query window back (or forward) in time. This is useful for accounting for delays in data availability or for aligning the query window with specific events. Can be set on a [per-query basis](#per-query-parameters-1) to override the reader-level setting.
</td>
</tr>
<tr>
<td>
<span style="white-space: nowrap;">`timeout`</span>
</td>
<td>
`30s`
</td>
<td>
(Optional) Specifies the maximum duration to wait for a query to complete before timing out. Can be set on a [per-query basis](#per-query-parameters-1) to override the reader-level setting.
</td>
</tr>
<tr>
<td>
<span style="white-space: nowrap;">`verify_tls`</span>
</td>
<td>
`false`
</td>
<td>
Verify TLS certificate. If `False`, it will not verify the TLS certificate.
If `True`, it will verify the certificate using the system's CA store.
If a path to a CA bundle file (like `ca.crt`), it will verify the certificate using the provided CA bundle.
</td>
</tr>
<tr>
<td>
<span style="white-space: nowrap;">`tls_cert_file`</span>
</td>
<td>
`path/to/cert.crt`
</td>
<td>
(Optional) Path to a file with the client certificate, i.e. `client.crt`.
</td>
</tr>
<tr>
<td>
<span style="white-space: nowrap;">`tls_key_file`</span>
</td>
<td>
`path/to/key.key`
</td>
<td>
(Optional) Path to a file with the client key, i.e. `client.key`.
</td>
</tr>
<tr>
<td>
<span style="white-space: nowrap;">`bearer_token`</span>
</td>
<td>
`token`
</td>
<td>
(Optional) Bearer token for authentication. If set, it will be used to authenticate the request as `Authorization: bearer {token}`.
</td>
</tr>
<tr>
<td>
<span style="white-space: nowrap;">`bearer_token_file`</span>
</td>
<td>
`path/to/token`
</td>
<td>
(Optional) Path to a file containing the bearer token. If set, it will be used to authenticate the request.
</td>
</tr>
<tr>
<td>
`user`
</td>
<td>
`USERNAME`
</td>
<td>
(Optional) Username for BasicAuth authentication. If set, it will be used to authenticate the request.
</td>
</tr>
<tr>
<td>
`password`
</td>
<td>
`PASSWORD`
</td>
<td>
(Optional) Password for authentication. If set, it will be used to authenticate the request.
</td>
</tr>
</tbody>
</table>
### Per-query parameters
The names, types and the logic of the per-query parameters subset used in `VLogsReader` are exactly the same as those of [`VmReader`](#vm-reader), please see [per-query parameters](#per-query-parameters) section above for the details. The only difference is that `expr` parameter should contain a valid [LogsQL](https://docs.victoriametrics.com/victorialogs/logsql/) expression with `stats` [pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe), as described in [query examples](#query-examples) section above.
### Per-query config example
```yaml
reader:
class: 'vlogs' # or 'reader.vlogs.VLogsReader'
# don't include /select/stats_query_range part in the URL, it is added automatically
datasource_url: 'https://play-vmlogs.victoriametrics.com/' # source victorialogs
# tenant_id: '0:0' # for cluster version only
sampling_period: '1m'
max_points_per_query: 10000
data_range: [0, 'inf'] # reader-level
offset: '0s' # reader-level
timeout: '30s'
queries:
# one query returning 1 result fields (avg_duration), it will have __name__ label (series name) as `duration_30m__avg`
duration_avg_30m:
expr: "* | stats avg(duration) as avg" # initial LogsQL expression
step: '2m' # overrides global `sampling_period` of 1m
data_range: [0, 'inf'] # meaning only positive values > 0 are expected, i.e. a value `y` < 0 will trigger anomaly score > 1
tz: 'America/New_York' # to override reader-wise `tz`
# tenant_id: '1:0' # overriding tenant_id to isolate data
# offset: '-15s' # to override reader-wise `offset` and query data 15 seconds earlier to account for data collection delays
# max_points_per_query: 5000 # overrides reader-level value of 10000 for `avg_duration` query
# one query returning 3 result fields (p50, p90, p99), they will have __name__ label (series name) as
# `duration_quantiles_30m__p50`, `duration_quantiles_30m__p90`, `duration_quantiles_30m__p99`, respectively
duration_quantiles_30m:
expr: |
* | stats
quantile(0.5, request_duration_seconds) p50,
quantile(0.9, request_duration_seconds) p90,
quantile(0.99, request_duration_seconds) p99
step: '2m' # overrides global `sampling_period` of 1m
# other per-query parameters as needed
# other reader-level parameters as needed
# other config sections, like models, schedulers, writer, ...
```
### mTLS protection
Please refer to the [mTLS protection](#mtls-protection) section above for details on how to configure mTLS for `VLogsReader`. It uses the same config parameters as `VmReader` for mTLS setup.
### Healthcheck metrics
Similarly to `VmReader`, `VLogsReader` also exposes [several healthchecks metrics](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#reader-behaviour-metrics).

View File

@@ -378,6 +378,8 @@ schedulers:
## Backtesting scheduler
> {{% available_from "v1.26.0" anomaly %}} `BacktestingScheduler` in [inference-only](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/#inference-only-mode) mode is used in UI for backtesting configurations on historical data to verify that it works as expected before it goes live. See [vmanomaly UI](https://docs.victoriametrics.com/anomaly-detection/ui/) on how to access and use the UI.
> As of latest version, the Backtesting scheduler can't be explicitly used with a combination of [state restoration](https://docs.victoriametrics.com/anomaly-detection/components/settings/#state-restoration). It is designed to run once and exit, so it does not maintain state across runs. A warning will be raised in logs and internal state for such scheduler will not be saved and restored upon restart. If you need to run the scheduler periodically and/or maintain state, consider using the [Periodic scheduler](#periodic-scheduler) instead.
> A new, more intuitive backtesting mode is available {{% available_from "v1.22.1" anomaly %}}. In **Inference only** mode, the window you specify via `[from, to]` (or `[from_iso, to_iso]`) is used *solely for inference*, and the corresponding training (“fit”) windows are determined automatically. To enable this behavior, set:

View File

@@ -307,7 +307,7 @@ This means that the service upon restart:
2. Will **partially** restore the state of `prophet` model, because its class and schedulers are unchanged, but **only instances trained on timeseries returned by `q1` query**. New fit/infer jobs will be set for new query `q3`. The old query `q2` artifacts will be dropped upon restart - all respective models and data for (`prophet`, `q2`) combination will be removed from the database file and from the disk.
### Logger Levels
## Logger Levels
{{% available_from "v1.25.3" anomaly %}} `vmanomaly` service supports per-component logger levels, allowing to control the verbosity of logs for each component independently. This can be useful for debugging or monitoring specific components without overwhelming the logs with information from other components. Prefixes are also supported, allowing to set the logger level for all components with a specific prefix.
@@ -324,8 +324,8 @@ settings:
n_workers: 4
restore_state: True # enables state restoration
logger_levels:
reader.vm: debug # affects only VmReader logs
model: warning # applies to all components with 'model' prefix, such as 'model.zscore_online', 'model.prophet', etc.
reader.vm: DEBUG # affects only VmReader logs
model: WARNING # applies to all components with 'model' prefix, such as 'model.zscore_online', 'model.prophet', etc.
# once commented out in hot-reload mode, will use the default logger level set by --loggerLevel command line argument
# monitoring.push: critical
```

View File

@@ -10,9 +10,9 @@ sitemap:
- To use *vmanomaly*, part of the enterprise package, a license key is required. Obtain your key [here](https://victoriametrics.com/products/enterprise/trial/) for this tutorial or for enterprise use.
- In the tutorial, we'll be using the following VictoriaMetrics components:
- [VictoriaMetrics Single-Node](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) (v1.125.1)
- [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/) (v1.125.1)
- [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) (v1.125.1)
- [VictoriaMetrics Single-Node](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) (v1.126.0)
- [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/) (v1.126.0)
- [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) (v1.126.0)
- [Grafana](https://grafana.com/) (v.10.2.1)
- [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/)
- [Node exporter](https://github.com/prometheus/node_exporter#node-exporter) (v1.7.0) and [Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/) (v0.27.0)
@@ -323,7 +323,7 @@ Let's wrap it all up together into the `docker-compose.yml` file.
services:
vmagent:
container_name: vmagent
image: victoriametrics/vmagent:v1.125.1
image: victoriametrics/vmagent:v1.126.0
depends_on:
- "victoriametrics"
ports:
@@ -340,7 +340,7 @@ services:
victoriametrics:
container_name: victoriametrics
image: victoriametrics/victoria-metrics:v1.125.1
image: victoriametrics/victoria-metrics:v1.126.0
ports:
- 8428:8428
volumes:
@@ -373,7 +373,7 @@ services:
vmalert:
container_name: vmalert
image: victoriametrics/vmalert:v1.125.1
image: victoriametrics/vmalert:v1.126.0
depends_on:
- "victoriametrics"
ports:
@@ -395,7 +395,7 @@ services:
restart: always
vmanomaly:
container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.25.3
image: victoriametrics/vmanomaly:v1.26.0
depends_on:
- "victoriametrics"
ports:

Binary file not shown.

Before

Width:  |  Height:  |  Size: 150 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 50 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 180 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 118 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Some files were not shown because too many files have changed in this diff Show More