Compare commits

...

96 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
4701c108ff docs/CHANGELOG.md: cut v1.71.0 2021-12-20 19:10:24 +02:00
Aliaksandr Valialkin
b9363d9726 lib/promscrape: take into account the original job_name when creating an unique key per each scrape target
This should handle the case when the original job_name has been changed in -promscrape.config ,
while the resulting job label remains the same because it is overriden via relabeling.
2021-12-20 18:38:05 +02:00
Aliaksandr Valialkin
afafeb379a all: typo fix: unexected -> unexpected 2021-12-20 17:39:52 +02:00
Yury Molodov
718c352946 vmui: graph fixes (#1982)
* fix: remove disabling custom step when zooming

* feat: add a dynamic calc of the width of the graph

* fix: add validate y-axis limits

* fix: correct axis limits for value 0

* fix: change logic create time series

* fix: change types for tooltip

* fix: correct points on the line

* fix: change the logic for set graph width

* fix: stop checking the period when auto-refresh is enabled

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-20 17:37:02 +02:00
Roman Khavronenko
871528fedb dashboards/vmagent: fix cached datasource uid (#1984)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-12-20 17:32:41 +02:00
Roman Khavronenko
52a3b2d77e Dashboards vmsingle (#1980)
* dashboards/vmsingle: add "Merges deferred" panel

The new panel supposed to show if there were deferred merges
due to insufficient disk space.
It goes within alerting rule which suppose to send a signal
in such cases.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/vmsingle: add "Cache usage" panel

The new panel supposed to show the % of the used cache
compared to allowed size by type.
It should help to determine underutilized types of caches.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/vmsingle: bump version requirement

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/vmsingle: rm alert for `vm_merge_need_free_disk_space`

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-12-20 17:28:35 +02:00
Aliaksandr Valialkin
5a36e241f4 lib/persistentqueue: check that readerOffset doesnt exceed writerOffset after each readerOffset increase
This should help detecting the source of the panic from https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1981
2021-12-20 17:25:11 +02:00
Aliaksandr Valialkin
ad388ecd78 docs: mention that downsampling can be evaluated for free by running enterprise binaries 2021-12-20 15:57:25 +02:00
Aliaksandr Valialkin
6d77cc9b08 app/vmselect/vmui: make vmui-update 2021-12-20 13:51:02 +02:00
Yurii Kravets
40073bbcb5 Update FAQ.md (#1765)
* Update FAQ.md

Adding explanation "Why do same metrics have differences in VictoriaMetrics and Prometheus dashboards?"

* Update FAQ

* Update FAQ.md

* Apply suggestions from code review

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
2021-12-20 13:43:04 +02:00
Aliaksandr Valialkin
974d9c0eee app/vmselect/promql: follow-up after 177e345d8a
* Document changes_prometheus(), increase_prometheus() and delta_prometheus() functions.
* Simplify their implementation
* Mention these functions in docs/CHANGELOG.md
2021-12-20 13:19:44 +02:00
dependabot[bot]
46eee933b7 build(deps): bump react-scripts in /app/vmui/packages/vmui (#1977)
Bumps [react-scripts](https://github.com/facebook/create-react-app/tree/HEAD/packages/react-scripts) from 4.0.3 to 5.0.0.
- [Release notes](https://github.com/facebook/create-react-app/releases)
- [Changelog](https://github.com/facebook/create-react-app/blob/main/CHANGELOG.md)
- [Commits](https://github.com/facebook/create-react-app/commits/react-scripts@5.0.0/packages/react-scripts)

---
updated-dependencies:
- dependency-name: react-scripts
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 14:16:45 +03:00
dependabot[bot]
4ba1f62507 build(deps): bump @mui/icons-material in /app/vmui/packages/vmui (#1978)
Bumps [@mui/icons-material](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-icons-material) from 5.2.1 to 5.2.4.
- [Release notes](https://github.com/mui-org/material-ui/releases)
- [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mui-org/material-ui/commits/v5.2.4/packages/mui-icons-material)

---
updated-dependencies:
- dependency-name: "@mui/icons-material"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 13:59:09 +03:00
dependabot[bot]
e11c09be82 build(deps): bump @types/node in /app/vmui/packages/vmui (#1979)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 16.11.12 to 17.0.1.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 13:54:26 +03:00
dependabot[bot]
d56bd7df19 build(deps): bump @mui/material in /app/vmui/packages/vmui (#1976)
Bumps [@mui/material](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-material) from 5.2.3 to 5.2.4.
- [Release notes](https://github.com/mui-org/material-ui/releases)
- [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mui-org/material-ui/commits/v5.2.4/packages/mui-material)

---
updated-dependencies:
- dependency-name: "@mui/material"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 13:44:44 +03:00
dependabot[bot]
52335bb48e build(deps-dev): bump @babel/plugin-proposal-nullish-coalescing-operator (#1975)
Bumps [@babel/plugin-proposal-nullish-coalescing-operator](https://github.com/babel/babel/tree/HEAD/packages/babel-plugin-proposal-nullish-coalescing-operator) from 7.16.0 to 7.16.5.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.16.5/packages/babel-plugin-proposal-nullish-coalescing-operator)

---
updated-dependencies:
- dependency-name: "@babel/plugin-proposal-nullish-coalescing-operator"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 13:44:19 +03:00
dependabot[bot]
7171ce767f build(deps): bump @codemirror/basic-setup in /app/vmui/packages/vmui (#1974)
Bumps [@codemirror/basic-setup](https://github.com/codemirror/basic-setup) from 0.19.0 to 0.19.1.
- [Release notes](https://github.com/codemirror/basic-setup/releases)
- [Changelog](https://github.com/codemirror/basic-setup/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/basic-setup/compare/0.19.0...0.19.1)

---
updated-dependencies:
- dependency-name: "@codemirror/basic-setup"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 13:36:12 +03:00
匠心零度
177e345d8a add Prometheus semantics function :changes_prometheus、delta_prometheus、increase_prometheus (#1972)
Co-authored-by: lirenzuo <lirenzuo@shein.com>
2021-12-20 12:32:43 +02:00
Aliaksandr Valialkin
d87414c57c docs/FAQ.md: describe main reasons for high churn rate 2021-12-20 12:30:09 +02:00
Roman Khavronenko
bc79bdf68a Dashboards vmagent updates (#1973)
* dashboards/vmagent: shuffle panels for better visibility

More important error/dropped panels were moved higher on the main row.
Network usage panel moved to Resource usage row.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/vmagent: add Troubleshooting row to show top 5 instances/jobs by churn rate

New panels are supposed to show top 5 jobs or targets which generate the most
of the churn rate. They were placed into a new row "Troubleshooting".

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/vmagent: add panels for showing persistent queue saturation

New panels were added to Torubleshooting row to show the persistent queue
saturation. The corresponding alerts were added and linked to these
panels as well.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/vmagent: add alert "RejectedRemoteWriteDataBlocksAreDropped"

New alert suppose to send a notification when vmagent starts to drop
data blocks rejected by configured remote write destiantion.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-12-20 12:16:53 +02:00
dependabot[bot]
36f4130cf1 build(deps-dev): bump @typescript-eslint/eslint-plugin (#1968)
Bumps [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) from 5.6.0 to 5.7.0.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.7.0/packages/eslint-plugin)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 13:15:15 +03:00
dependabot[bot]
2fe74069be build(deps): bump uplot from 1.6.17 to 1.6.18 in /app/vmui/packages/vmui (#1967)
Bumps [uplot](https://github.com/leeoniya/uPlot) from 1.6.17 to 1.6.18.
- [Release notes](https://github.com/leeoniya/uPlot/releases)
- [Commits](https://github.com/leeoniya/uPlot/compare/1.6.17...1.6.18)

---
updated-dependencies:
- dependency-name: uplot
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 13:15:00 +03:00
Aliaksandr Valialkin
7749b47d6a vendor: make vendor-update 2021-12-20 12:07:22 +02:00
dependabot[bot]
a6b86941a1 build(deps-dev): bump @typescript-eslint/parser (#1965)
Bumps [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) from 5.6.0 to 5.7.0.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.7.0/packages/parser)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/parser"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 12:02:54 +03:00
dependabot[bot]
76bb135181 build(deps): bump @mui/lab in /app/vmui/packages/vmui (#1966)
Bumps [@mui/lab](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-lab) from 5.0.0-alpha.59 to 5.0.0-alpha.60.
- [Release notes](https://github.com/mui-org/material-ui/releases)
- [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mui-org/material-ui/commits/HEAD/packages/mui-lab)

---
updated-dependencies:
- dependency-name: "@mui/lab"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 12:02:33 +03:00
dependabot[bot]
be17387682 build(deps): bump typescript in /app/vmui/packages/vmui (#1969)
Bumps [typescript](https://github.com/Microsoft/TypeScript) from 4.5.3 to 4.5.4.
- [Release notes](https://github.com/Microsoft/TypeScript/releases)
- [Commits](https://github.com/Microsoft/TypeScript/compare/v4.5.3...v4.5.4)

---
updated-dependencies:
- dependency-name: typescript
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 12:02:07 +03:00
John Seekins
b9c41ff051 OpenTSDB Migration Fix (#1946)
* Simplify queries to OpenTSDB (and make them properly appear in OpenTSDB query stats) and also tweak defaults a bit

Signed-off-by: John Seekins <jseekins@datto.com>
2021-12-20 09:35:51 +02:00
Aliaksandr Valialkin
16636a458f docs/CHANGELOG.md: document 6814cc6809
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1943
2021-12-17 20:16:22 +02:00
Aliaksandr Valialkin
8a7f08ded3 lib/storage: properly update per-part min_dedup_interval file contents after merge
Previously 0s was always written even if -dedup.minScrapeInterval was set to non-zero value

This is a follow-up for 4ff647137a
2021-12-17 20:13:24 +02:00
Roman Khavronenko
6814cc6809 vmalert: always convert step value to seconds for better compatibility (#1955)
When using `vmalert` with older Prometheus versions, the passed
`step=2m` may be parsed by Prometheus with an err: "cannot parse \"2m0s\" to a valid duration".
In order to improve compatibility vmalert will always convert step duration to seconds.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1943
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-12-17 15:26:34 +02:00
Aliaksandr Valialkin
e6d4641bf0 app/vmselect/vmui: make vmui-update 2021-12-17 11:01:09 +02:00
Aliaksandr Valialkin
193331d522 app/vmselect: de-duplicate data exported via /api/v1/export/csv by default
Previously the exported data wasn't de-duplicated.
Now it is possible to export the raw data without deduplication
by passing reduce_mem_usage=1 query arg to /api/v1/export/csv

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1837
2021-12-17 10:57:39 +02:00
Roman Khavronenko
f30ed13155 docs: add Benchmarks section (#1950)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-12-16 19:15:19 +03:00
Roman Khavronenko
8b0d340c18 docs: update MetricsQL.Subquery section description (#1951)
* simplify sentences;
* fix typo.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-12-16 19:09:25 +03:00
Yury Molodov
eaf82fe411 fix: return query for app mode (#1954) 2021-12-16 11:44:46 +02:00
Aliaksandr Valialkin
a3adf24527 lib/promscrape: allow up to 5 redirects when scraping a target by default
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1945
2021-12-16 00:14:14 +02:00
Aliaksandr Valialkin
5efe377a26 app/vmselect/promql: add timestamp_with_name(m[d]) function
This function works the same as `timestamp()`, but doesn't remove source time series names.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949#issuecomment-995222388
2021-12-15 23:37:07 +02:00
Aliaksandr Valialkin
27a1ae57e5 docs: mention -storage.minFreeDiskSpaceBytes command-line flag at capacity planning section
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1727 and https://github.com/VictoriaMetrics/VictoriaMetrics/issues/269
2021-12-15 21:41:30 +02:00
Yury Molodov
9baad51004 vmui: introduce application mode (#1949)
* feat: add a label for the Query field

* fix: change zoom position

* fix: add description and error code to alerts

* fix: correct logic query history

* fix: correct update query history

* feat: add custom step

* update package-lock.json

* feat: introduce application mode

* build vmui

* Revert "build vmui"

This reverts commit c0e2415550.

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-15 21:33:25 +02:00
Aliaksandr Valialkin
65bef771f6 docs/Cluster-VictoriaMetrics.md: mention about the added downsampling support 2021-12-15 16:40:15 +02:00
Aliaksandr Valialkin
8e1a87491a docs: document the added dowsnampling support in VictoriaMetrics enterprise
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/36
2021-12-15 16:25:51 +02:00
Aliaksandr Valialkin
4ff647137a lib/storage: deduplicate samples more thoroughly
Previously some duplicate samples may be left on disk for time series with high churn rate.
This may result in higher disk space usage.
2021-12-15 15:59:58 +02:00
Aliaksandr Valialkin
92070cbb67 lib/storage: return dedup interval in milliseconds from GetDedupInterval()
This removes duplicate .Milliseconds() calls after GetDedupInterval() calls.
2021-12-15 13:26:38 +02:00
Aliaksandr Valialkin
acd56603b0 app/victoria-metrics: mention https://docs.victoriametrics.com/#downsampling in the description for -dedup.minScrapeInterval command-line flag 2021-12-15 13:18:04 +02:00
Aliaksandr Valialkin
1d20a19c7d lib/storage: explicitly pass dedupInterval to DeduplicateSamples() and deduplicateSamplesDuringMerge()
This improves the code readability and debuggability, since the output of these functions
stops depending on global state.
2021-12-14 20:49:12 +02:00
Aliaksandr Valialkin
e1a715b0f5 lib/storage: convert alternate regexps into Graphite wildcards inside __graphite__ pseudo-label
For example, `{__graphite__=~"foo.(bar|baz)"}` is automatically converted to `{__graphite__=~"foo.{bar,baz}"}` before execution.
This allows using multi-value Grafana template variables such as `{__graphite__=~"foo.($app)"}`.
2021-12-14 19:51:49 +02:00
Aliaksandr Valialkin
496b6e4d3d deployment/docker/docker-compose.yml: update Grafana version from 8.2.2 to 8.3.2
See https://grafana.com/blog/2021/12/10/grafana-8.3.2-and-7.5.12-released-with-moderate-severity-security-fix/
2021-12-14 15:09:49 +02:00
Aliaksandr Valialkin
d456af7499 docs/CHANGELOG.md: link to the issue about unaligned 64-bit atomic opertion panic on 32-bit architectures 2021-12-14 15:00:10 +02:00
Aliaksandr Valialkin
80996d916b docs/CHANGELOG.md: an attempt to properly show $labels.alertname at https://docs.victoriametrics.com/CHANGELOG.html 2021-12-14 14:56:57 +02:00
Yury Molodov
49e6a921df vmui: custom step (#1942)
* feat: add a label for the Query field

* fix: change zoom position

* fix: add description and error code to alerts

* fix: correct logic query history

* fix: correct update query history

* feat: add custom step

* update package-lock.json

* docs: document that VMUI now supports overriding of `step` query arg, which is passed to `/api/v1/query_range`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-14 14:51:45 +02:00
Yury Molodov
b5b701d590 vmui: minor fixes (#1936)
* feat: add a label for the Query field

* fix: change zoom position

* fix: add description and error code to alerts

* fix: correct logic query history

* fix: correct update query history

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document bugfixes

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-13 13:42:37 +02:00
Aliaksandr Valialkin
ce80a0ce5e app/vmselect/vmui: make vmui-update 2021-12-13 13:32:54 +02:00
Aliaksandr Valialkin
0a157f65bd docs/Single-server-VictoriaMetrics.md: added a link to Graphite paths and wildcards - https://graphite.readthedocs.io/en/latest/render_api.html#paths-and-wildcards 2021-12-13 12:01:17 +02:00
dependabot[bot]
f6f1e1821e build(deps): bump @mui/lab in /app/vmui/packages/vmui (#1935)
Bumps [@mui/lab](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-lab) from 5.0.0-alpha.58 to 5.0.0-alpha.59.
- [Release notes](https://github.com/mui-org/material-ui/releases)
- [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mui-org/material-ui/commits/HEAD/packages/mui-lab)

---
updated-dependencies:
- dependency-name: "@mui/lab"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 12:00:19 +03:00
dependabot[bot]
c522630f72 build(deps): bump @codemirror/commands in /app/vmui/packages/vmui (#1933)
Bumps [@codemirror/commands](https://github.com/codemirror/commands) from 0.19.5 to 0.19.6.
- [Release notes](https://github.com/codemirror/commands/releases)
- [Changelog](https://github.com/codemirror/commands/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/commands/compare/0.19.5...0.19.6)

---
updated-dependencies:
- dependency-name: "@codemirror/commands"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 11:56:47 +03:00
dependabot[bot]
e98a863f91 build(deps-dev): bump @typescript-eslint/parser (#1934)
Bumps [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) from 5.5.0 to 5.6.0.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.6.0/packages/parser)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/parser"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 11:52:43 +03:00
dependabot[bot]
1d2708b147 build(deps): bump @codemirror/view in /app/vmui/packages/vmui (#1931)
Bumps [@codemirror/view](https://github.com/codemirror/view) from 0.19.26 to 0.19.29.
- [Release notes](https://github.com/codemirror/view/releases)
- [Changelog](https://github.com/codemirror/view/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/view/compare/0.19.26...0.19.29)

---
updated-dependencies:
- dependency-name: "@codemirror/view"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 11:52:26 +03:00
dependabot[bot]
51e7ba65ff build(deps): bump @emotion/react in /app/vmui/packages/vmui (#1930)
Bumps [@emotion/react](https://github.com/emotion-js/emotion) from 11.7.0 to 11.7.1.
- [Release notes](https://github.com/emotion-js/emotion/releases)
- [Changelog](https://github.com/emotion-js/emotion/blob/main/CHANGELOG.md)
- [Commits](https://github.com/emotion-js/emotion/compare/@emotion/react@11.7.0...@emotion/react@11.7.1)

---
updated-dependencies:
- dependency-name: "@emotion/react"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 11:52:13 +03:00
dependabot[bot]
f583b7bdcf build(deps): bump @mui/icons-material in /app/vmui/packages/vmui (#1929)
Bumps [@mui/icons-material](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-icons-material) from 5.2.0 to 5.2.1.
- [Release notes](https://github.com/mui-org/material-ui/releases)
- [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mui-org/material-ui/commits/v5.2.1/packages/mui-icons-material)

---
updated-dependencies:
- dependency-name: "@mui/icons-material"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 11:51:56 +03:00
dependabot[bot]
9929f08968 build(deps): bump @types/node in /app/vmui/packages/vmui (#1932)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 16.11.11 to 16.11.12.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 11:50:01 +03:00
dependabot[bot]
60f734ecce build(deps): bump @mui/material in /app/vmui/packages/vmui (#1925)
Bumps [@mui/material](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-material) from 5.2.2 to 5.2.3.
- [Release notes](https://github.com/mui-org/material-ui/releases)
- [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mui-org/material-ui/commits/v5.2.3/packages/mui-material)

---
updated-dependencies:
- dependency-name: "@mui/material"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 11:49:28 +03:00
dependabot[bot]
325758317f build(deps): bump @testing-library/jest-dom in /app/vmui/packages/vmui (#1924)
Bumps [@testing-library/jest-dom](https://github.com/testing-library/jest-dom) from 5.16.0 to 5.16.1.
- [Release notes](https://github.com/testing-library/jest-dom/releases)
- [Changelog](https://github.com/testing-library/jest-dom/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testing-library/jest-dom/compare/v5.16.0...v5.16.1)

---
updated-dependencies:
- dependency-name: "@testing-library/jest-dom"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 11:49:08 +03:00
dependabot[bot]
b5cec7fdb9 build(deps): bump typescript in /app/vmui/packages/vmui (#1926)
Bumps [typescript](https://github.com/Microsoft/TypeScript) from 4.5.2 to 4.5.3.
- [Release notes](https://github.com/Microsoft/TypeScript/releases)
- [Commits](https://github.com/Microsoft/TypeScript/compare/v4.5.2...v4.5.3)

---
updated-dependencies:
- dependency-name: typescript
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 11:46:05 +03:00
dependabot[bot]
e37152d74e build(deps): bump @mui/styles in /app/vmui/packages/vmui (#1927)
Bumps [@mui/styles](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-styles) from 5.2.2 to 5.2.3.
- [Release notes](https://github.com/mui-org/material-ui/releases)
- [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mui-org/material-ui/commits/v5.2.3/packages/mui-styles)

---
updated-dependencies:
- dependency-name: "@mui/styles"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 11:45:54 +03:00
dependabot[bot]
b02d655dcf build(deps-dev): bump @typescript-eslint/eslint-plugin (#1928)
Bumps [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) from 5.5.0 to 5.6.0.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.6.0/packages/eslint-plugin)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 11:45:42 +03:00
Aliaksandr Valialkin
c82cc9cd11 docs/CHANGELOG.md: document 7c3b6365f0 2021-12-12 19:09:20 +02:00
Yury Molodov
7c3b6365f0 vmui: add a label for the Query field (#1923)
* feat: add a label for the Query field

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-12 19:06:39 +02:00
Aliaksandr Valialkin
a8ad870bd0 deployment/docker: update Go builder from v1.17.4 to v1.17.5
See https://github.com/golang/go/issues?q=milestone%3AGo1.17.5+label%3ACherryPickApproved
2021-12-12 18:17:49 +02:00
Aliaksandr Valialkin
7d58f57a52 vendor: make vendor-update 2021-12-12 18:10:09 +02:00
Aliaksandr Valialkin
d1f8915ed1 app/vmselect/promql: preserve the order of time series passed to limit_offset() function
Previously time series passed to `limit_offset()` were shuffled according to hash for their labels.
This was unexpected behaviour for most users.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1920 and https://github.com/VictoriaMetrics/VictoriaMetrics/issues/951
2021-12-12 18:04:58 +02:00
Aliaksandr Valialkin
3d4349343d docs/CHANGELOG.md: document 2851709745
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1921
2021-12-10 12:13:34 +02:00
Roman Khavronenko
2851709745 vmalert: update the order of service labels attaching (#1922)
Service labels like `alertname` or `alertgroup` were attached
after template expanding for `labels` section. Because of this,
labels `alertname` or `alertgroup` weren't available for templating
in `labels` section of alert's definition.
This commit changes the order of labels attaching and adds a test
for verifying these labels availability.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1921
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-12-10 12:10:26 +02:00
Aliaksandr Valialkin
b85e88e2db app/vmui/README.md: remove features chapter, since it lists unimportant and/or misleading features
The main user-visible features for vmui are documented at https://docs.victoriametrics.com/#vmui .
2021-12-09 19:50:25 +02:00
Aliaksandr Valialkin
b42981c465 docs/Single-server-VictoriaMetrics.md: add a link to https://github.com/denisgolius/victoriametrics-ru-links 2021-12-09 19:42:27 +02:00
Aliaksandr Valialkin
a2e0275f14 deployment/docker: update Go builder from v1.17.3 to v1.17.4
See https://github.com/golang/go/issues?q=milestone%3AGo1.17.4+label%3ACherryPickApproved
2021-12-09 18:51:58 +02:00
Aliaksandr Valialkin
52eb9c99e2 docs: document the ability to investigate correlation between two queries at /vmui
This is a follow-up for c1fd93e8a0
2021-12-08 17:24:01 +02:00
Yury Molodov
c1fd93e8a0 vmui: multiple queries (#1916)
* feat: change duration by "enter"

* fix: optimize data processing for chart

* feat: set minimum step to 1ms

* update dependencies

* feat: remove save the last query to local storage

* fix: handle an error in a table with subqueries

* feat: store display type in URL

* Revert "feat: store display type in URL"

This reverts commit ccc242c69a.

* feat: store display type in URL

* refactor: move the time setting to a folder

* refactor: move the query configurator to a folder

* refactor: move the auth settings to a folder

* feat: improve styles

* feat: add multi query

* update package-lock

* feat: add display multiple queries

* feat: add limits for multiple queries

* update dependencies

* feat: add history for multiple queries

* feat: add line type to legend

* feat: change style for switch

* feat: change the logic for axes limits for multiple queries

* update package-lock.json

* update dependencies

* feat: add the filter to legend

* wip

* lib/httpserver: add missing 127.0.0.1 hostname to the logged address for http and pprof server if the address starts with ':'

This allows copy-pasting the url to http server from logs.

* lib/httpserver: add missing 127.0.0.1 hostname to the logged address for http and pprof server if the address starts with ':'

This allows copy-pasting the url to http server from logs.

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-08 16:40:15 +02:00
Aliaksandr Valialkin
0288078cfb docs/Single-server-VictoriaMetrics.md: add LinkedIn public channel 2021-12-08 13:15:24 +02:00
Aliaksandr Valialkin
64da5c2bf6 docs/Release-Guide.md: refresh the list of channels for publishing release notes 2021-12-08 13:15:05 +02:00
Nikolay
a581a93c9b reworks snap build with docker (#1910) 2021-12-08 13:05:35 +02:00
Aliaksandr Valialkin
896fa9bb7c app/vmalert/config: sort extra_filter labels before passing them to query args in order to get consistent order of query args across runs
This fixes TestGroupParams test - see https://github.com/VictoriaMetrics/VictoriaMetrics/runs/4432510244?check_suite_focus=true#step:5:288
2021-12-08 13:02:49 +02:00
Aliaksandr Valialkin
2711d2ea55 docs/Single-server-VictoriaMetrics.md: move features chapter above the case studies chapter 2021-12-08 12:49:09 +02:00
Aliaksandr Valialkin
ff15a752c1 app/vmselect: accept optional extra_filters[] query args for all the supported Prometheus querying APIs
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1863
2021-12-06 17:07:09 +02:00
Aliaksandr Valialkin
45d082bbe2 app/vminsert: add -maxLabelValueLen command-line flag
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1908
2021-12-06 11:40:34 +02:00
Aliaksandr Valialkin
732a0cd3e1 app/vmselect/vmui: make vmui-update 2021-12-06 10:19:09 +02:00
Aliaksandr Valialkin
e2d9bf3b57 vendor: make vendor-update 2021-12-06 10:19:09 +02:00
dependabot[bot]
e06d01f0eb build(deps-dev): bump @typescript-eslint/eslint-plugin (#1902)
Bumps [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.5.0/packages/eslint-plugin)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-06 11:01:58 +03:00
dependabot[bot]
0c614f3e9d build(deps): bump @codemirror/view in /app/vmui/packages/vmui (#1901)
Bumps [@codemirror/view](https://github.com/codemirror/view) from 0.19.21 to 0.19.26.
- [Release notes](https://github.com/codemirror/view/releases)
- [Changelog](https://github.com/codemirror/view/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/view/compare/0.19.21...0.19.26)

---
updated-dependencies:
- dependency-name: "@codemirror/view"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-06 10:51:53 +03:00
dependabot[bot]
2f74d17297 build(deps): bump @types/node in /app/vmui/packages/vmui (#1903)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 16.11.10 to 16.11.11.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-06 10:51:26 +03:00
dependabot[bot]
4931da89f0 build(deps): bump @testing-library/jest-dom in /app/vmui/packages/vmui (#1904)
Bumps [@testing-library/jest-dom](https://github.com/testing-library/jest-dom) from 5.15.1 to 5.16.0.
- [Release notes](https://github.com/testing-library/jest-dom/releases)
- [Changelog](https://github.com/testing-library/jest-dom/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testing-library/jest-dom/compare/v5.15.1...v5.16.0)

---
updated-dependencies:
- dependency-name: "@testing-library/jest-dom"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-06 10:51:11 +03:00
dependabot[bot]
27faaec2b9 build(deps-dev): bump @typescript-eslint/parser (#1905)
Bumps [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.5.0/packages/parser)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/parser"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-06 10:51:00 +03:00
Aliaksandr Valialkin
da402fbdfa lib/workingsetcache: fix unaligned 64-bit atomic operation panic on 32-bit architectures
The panic has been introduced in 7275ebf91a
2021-12-03 01:21:51 +02:00
Aliaksandr Valialkin
4888e2c232 docs/Cluster-VictoriaMetrics.md: sync with cluster branch 2021-12-03 00:13:02 +02:00
Aliaksandr Valialkin
06642d97f5 app: allow specifying http and https urls in the following command-line flags
* -promscrape.config
* -relabelConfig
* -remoteWrite.relabelConfig
* -remoteWrite.urlRelabelConfig
2021-12-03 00:10:02 +02:00
Aliaksandr Valialkin
62b4efb3e7 app/vmauth: follow-up for 13368bed18
* Document the ability to specify http or https urls in `-auth.config` at docs/CHANGELOG.md
* Move the ReadFileOrHTTP to lib/fs, so it can be re-used in other places where a file
  should be read from the given path. For example, in `-promscrape.config` at `vmagent`.
2021-12-02 23:32:05 +02:00
Tiago Magalhães
13368bed18 vmauth: support for reading remote auth config file (#1898)
* add support for reading remote auth_config file via http

* fix lint

* fix defer on close body

Co-authored-by: Tiago Magalhães <tmagalhaes@wavecom.pt>
2021-12-02 23:19:05 +02:00
268 changed files with 20912 additions and 29220 deletions

View File

@@ -24,6 +24,8 @@ all: \
include app/*/Makefile
include deployment/*/Makefile
include snap/local/Makefile
clean:
rm -rf bin/*
@@ -84,9 +86,6 @@ vmutils-windows-amd64: \
vmauth-windows-amd64 \
vmctl-windows-amd64
release-snap:
snapcraft
snapcraft upload "victoriametrics_$(PKG_TAG)_multi.snap" --release beta,edge,candidate
publish-release:
git checkout $(TAG) && $(MAKE) release publish && \
@@ -180,6 +179,7 @@ release-vmutils-windows-generic: \
vmctl-windows-$(GOARCH)-prod.exe \
> vmutils-windows-$(GOARCH)-$(PKG_TAG)_checksums.txt
pprof-cpu:
go tool pprof -trim_path=github.com/VictoriaMetrics/VictoriaMetrics@ $(PPROF_FILE)

158
README.md
View File

@@ -13,46 +13,13 @@
VictoriaMetrics is a fast, cost-effective and scalable monitoring solution and time series database.
VictoriaMetrics is available in [binary releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases),
in [Docker images](https://hub.docker.com/r/victoriametrics/victoria-metrics/), in [Snap packages](https://snapcraft.io/victoriametrics)
and in [source code](https://github.com/VictoriaMetrics/VictoriaMetrics). Just download VictoriaMetrics follow [these instructions](#how-to-start-victoriametrics).
[Docker images](https://hub.docker.com/r/victoriametrics/victoria-metrics/), [Snap packages](https://snapcraft.io/victoriametrics)
and [source code](https://github.com/VictoriaMetrics/VictoriaMetrics). Just download VictoriaMetrics and follow [these instructions](#how-to-start-victoriametrics).
Then read [Prometheus setup](#prometheus-setup) and [Grafana setup](#grafana-setup) docs.
Cluster version of VictoriaMetrics is available [here](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html).
[Contact us](mailto:info@victoriametrics.com) if you need enterprise support for VictoriaMetrics.
See [features available in enterprise package](https://victoriametrics.com/enterprise.html).
Enterprise binaries can be downloaded and evaluated for free from [the releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
## Case studies and talks
Case studies:
* [AbiosGaming](https://docs.victoriametrics.com/CaseStudies.html#abiosgaming)
* [adidas](https://docs.victoriametrics.com/CaseStudies.html#adidas)
* [Adsterra](https://docs.victoriametrics.com/CaseStudies.html#adsterra)
* [ARNES](https://docs.victoriametrics.com/CaseStudies.html#arnes)
* [Brandwatch](https://docs.victoriametrics.com/CaseStudies.html#brandwatch)
* [CERN](https://docs.victoriametrics.com/CaseStudies.html#cern)
* [COLOPL](https://docs.victoriametrics.com/CaseStudies.html#colopl)
* [Dreamteam](https://docs.victoriametrics.com/CaseStudies.html#dreamteam)
* [Fly.io](https://docs.victoriametrics.com/CaseStudies.html#flyio)
* [German Research Center for Artificial Intelligence](https://docs.victoriametrics.com/CaseStudies.html#german-research-center-for-artificial-intelligence)
* [Grammarly](https://docs.victoriametrics.com/CaseStudies.html#grammarly)
* [Groove X](https://docs.victoriametrics.com/CaseStudies.html#groove-x)
* [Idealo.de](https://docs.victoriametrics.com/CaseStudies.html#idealode)
* [MHI Vestas Offshore Wind](https://docs.victoriametrics.com/CaseStudies.html#mhi-vestas-offshore-wind)
* [Razorpay](https://docs.victoriametrics.com/CaseStudies.html#razorpay)
* [Percona](https://docs.victoriametrics.com/CaseStudies.html#percona)
* [Sensedia](https://docs.victoriametrics.com/CaseStudies.html#sensedia)
* [Smarkets](https://docs.victoriametrics.com/CaseStudies.html#smarkets)
* [Synthesio](https://docs.victoriametrics.com/CaseStudies.html#synthesio)
* [Wedos.com](https://docs.victoriametrics.com/CaseStudies.html#wedoscom)
* [Wix.com](https://docs.victoriametrics.com/CaseStudies.html#wixcom)
* [Zerodha](https://docs.victoriametrics.com/CaseStudies.html#zerodha)
* [zhihu](https://docs.victoriametrics.com/CaseStudies.html#zhihu)
See also [articles and slides about VictoriaMetrics from our users](https://docs.victoriametrics.com/Articles.html#third-party-articles-and-slides-about-victoriametrics)
[Contact us](mailto:info@victoriametrics.com) if you need enterprise support for VictoriaMetrics. See [features available in enterprise package](https://victoriametrics.com/enterprise.html). Enterprise binaries can be downloaded and evaluated for free from [the releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
## Prominent features
@@ -95,6 +62,37 @@ VictoriaMetrics has the following prominent features:
See also [various Articles about VictoriaMetrics](https://docs.victoriametrics.com/Articles.html).
## Case studies and talks
Case studies:
* [AbiosGaming](https://docs.victoriametrics.com/CaseStudies.html#abiosgaming)
* [adidas](https://docs.victoriametrics.com/CaseStudies.html#adidas)
* [Adsterra](https://docs.victoriametrics.com/CaseStudies.html#adsterra)
* [ARNES](https://docs.victoriametrics.com/CaseStudies.html#arnes)
* [Brandwatch](https://docs.victoriametrics.com/CaseStudies.html#brandwatch)
* [CERN](https://docs.victoriametrics.com/CaseStudies.html#cern)
* [COLOPL](https://docs.victoriametrics.com/CaseStudies.html#colopl)
* [Dreamteam](https://docs.victoriametrics.com/CaseStudies.html#dreamteam)
* [Fly.io](https://docs.victoriametrics.com/CaseStudies.html#flyio)
* [German Research Center for Artificial Intelligence](https://docs.victoriametrics.com/CaseStudies.html#german-research-center-for-artificial-intelligence)
* [Grammarly](https://docs.victoriametrics.com/CaseStudies.html#grammarly)
* [Groove X](https://docs.victoriametrics.com/CaseStudies.html#groove-x)
* [Idealo.de](https://docs.victoriametrics.com/CaseStudies.html#idealode)
* [MHI Vestas Offshore Wind](https://docs.victoriametrics.com/CaseStudies.html#mhi-vestas-offshore-wind)
* [Razorpay](https://docs.victoriametrics.com/CaseStudies.html#razorpay)
* [Percona](https://docs.victoriametrics.com/CaseStudies.html#percona)
* [Sensedia](https://docs.victoriametrics.com/CaseStudies.html#sensedia)
* [Smarkets](https://docs.victoriametrics.com/CaseStudies.html#smarkets)
* [Synthesio](https://docs.victoriametrics.com/CaseStudies.html#synthesio)
* [Wedos.com](https://docs.victoriametrics.com/CaseStudies.html#wedoscom)
* [Wix.com](https://docs.victoriametrics.com/CaseStudies.html#wixcom)
* [Zerodha](https://docs.victoriametrics.com/CaseStudies.html#zerodha)
* [zhihu](https://docs.victoriametrics.com/CaseStudies.html#zhihu)
See also [articles and slides about VictoriaMetrics from our users](https://docs.victoriametrics.com/Articles.html#third-party-articles-and-slides-about-victoriametrics)
## Operation
## How to start VictoriaMetrics
@@ -418,9 +416,15 @@ The `/api/v1/export` endpoint should return the following response:
Data sent to VictoriaMetrics via `Graphite plaintext protocol` may be read via the following APIs:
* [Graphite API](#graphite-api-usage)
* [Prometheus querying API](#prometheus-querying-api-usage). VictoriaMetrics supports `__graphite__` pseudo-label for selecting time series with Graphite-compatible filters in [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html). For example, `{__graphite__="foo.*.bar"}` is equivalent to `{__name__=~"foo[.][^.]*[.]bar"}`, but it works faster and it is easier to use when migrating from Graphite to VictoriaMetrics. VictoriaMetrics also supports [label_graphite_group](https://docs.victoriametrics.com/MetricsQL.html#label_graphite_group) function for extracting the given groups from Graphite metric name.
* [Prometheus querying API](#prometheus-querying-api-usage). See also [selecting Graphite metrics](#selecting-graphite-metrics).
* [go-graphite/carbonapi](https://github.com/go-graphite/carbonapi/blob/main/cmd/carbonapi/carbonapi.example.victoriametrics.yaml)
## Selecting Graphite metrics
VictoriaMetrics supports `__graphite__` pseudo-label for selecting time series with Graphite-compatible filters in [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html). For example, `{__graphite__="foo.*.bar"}` is equivalent to `{__name__=~"foo[.][^.]*[.]bar"}`, but it works faster and it is easier to use when migrating from Graphite to VictoriaMetrics. See [docs for Graphite paths and wildcards](https://graphite.readthedocs.io/en/latest/render_api.html#paths-and-wildcards). VictoriaMetrics also supports [label_graphite_group](https://docs.victoriametrics.com/MetricsQL.html#label_graphite_group) function for extracting the given groups from Graphite metric name.
The `__graphite__` pseudo-label supports e.g. alternate regexp filters such as `(value1|...|valueN)`. They are transparently converted to `{value1,...,valueN}` syntax [used in Graphite](https://graphite.readthedocs.io/en/latest/render_api.html#paths-and-wildcards). This allows using [multi-value template variables in Grafana](https://grafana.com/docs/grafana/latest/variables/formatting-multi-value-variables/) inside `__graphite__` pseudo-label. For example, Grafana expands `{__graphite__=~"foo.($bar).baz"}` into `{__graphite__=~"foo.(x|y).baz"}` if `$bar` template variable contains `x` and `y` values. In this case the query is automatically converted into `{__graphite__=~"foo.{x,y}.baz"}` before execution.
## How to send data from OpenTSDB-compatible agents
VictoriaMetrics supports [telnet put protocol](http://opentsdb.net/docs/build/html/api_telnet/put.html)
@@ -517,9 +521,10 @@ All the Prometheus querying API handlers can be prepended with `/prometheus` pre
### Prometheus querying API enhancements
VictoriaMetrics accepts optional `extra_label=<label_name>=<label_value>` query arg, which can be used for enforcing additional label filters for queries. For example,
`/api/v1/query_range?extra_label=user_id=123&query=<query>` would automatically add `{user_id="123"}` label filter to the given `<query>`. This functionality can be used
for limiting the scope of time series visible to the given tenant. It is expected that the `extra_label` query arg is automatically set by auth proxy sitting
in front of VictoriaMetrics. See [vmauth](https://docs.victoriametrics.com/vmauth.html) and [vmgateway](https://docs.victoriametrics.com/vmgateway.html) as examples of such proxies.
`/api/v1/query_range?extra_label=user_id=123&extra_label=group_id=456&query=<query>` would automatically add `{user_id="123",group_id="456"}` label filters to the given `<query>`. This functionality can be used for limiting the scope of time series visible to the given tenant. It is expected that the `extra_label` query args are automatically set by auth proxy sitting in front of VictoriaMetrics. See [vmauth](https://docs.victoriametrics.com/vmauth.html) and [vmgateway](https://docs.victoriametrics.com/vmgateway.html) as examples of such proxies.
VictoriaMetrics accepts optional `extra_filters[]=series_selector` query arg, which can be used for enforcing arbitrary label filters for queries. For example,
`/api/v1/query_range?extra_filters[]={env=~"prod|staging",user="xyz"}&query=<query>` would automatically add `{env=~"prod|staging",user="xyz"}` label filters to the given `<query>`. This functionality can be used for limiting the scope of time series visible to the given tenant. It is expected that the `extra_filters[]` query args are automatically set by auth proxy sitting in front of VictoriaMetrics. See [vmauth](https://docs.victoriametrics.com/vmauth.html) and [vmgateway](https://docs.victoriametrics.com/vmgateway.html) as examples of such proxies.
VictoriaMetrics accepts relative times in `time`, `start` and `end` query args additionally to unix timestamps and [RFC3339](https://www.ietf.org/rfc/rfc3339.txt).
For example, the following query would return data for the last 30 minutes: `/api/v1/query_range?start=-30m&query=...`.
@@ -556,12 +561,11 @@ VictoriaMetrics supports the following Graphite APIs, which are needed for [Grap
All the Graphite handlers can be pre-pended with `/graphite` prefix. For example, both `/graphite/metrics/find` and `/metrics/find` should work.
VictoriaMetrics accepts optional `extra_label=<label_name>=<label_value>` query arg for all the Graphite APIs. This arg can be used for limiting the scope of time series
visible to the given tenant. It is expected that the `extra_label` query arg is automatically set by auth proxy sitting in front of VictoriaMetrics.
VictoriaMetrics accepts optional query args: `extra_label=<label_name>=<label_value>` and `extra_filters[]=series_selector` query args for all the Graphite APIs. These args can be used for limiting the scope of time series visible to the given tenant. It is expected that the `extra_label` query arg is automatically set by auth proxy sitting in front of VictoriaMetrics. See [vmauth](https://docs.victoriametrics.com/vmauth.html) and [vmgateway](https://docs.victoriametrics.com/vmgateway.html) as examples of such proxies.
[Contact us](mailto:sales@victoriametrics.com) if you need assistance with such a proxy.
VictoriaMetrics supports `__graphite__` pseudo-label for filtering time series with Graphite-compatible filters in [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html).
For example, `{__graphite__="foo.*.bar"}` is equivalent to `{__name__=~"foo[.][^.]*[.]bar"}`, but it works faster and it is easier to use when migrating from Graphite to VictoriaMetrics. See also [label_graphite_group](https://docs.victoriametrics.com/MetricsQL.html#label_graphite_group) function.
VictoriaMetrics supports `__graphite__` pseudo-label for filtering time series with Graphite-compatible filters in [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html). See [these docs](#selecting-graphite-metrics).
### Graphite Render API usage
@@ -612,6 +616,10 @@ Query history can be navigated by holding `Ctrl` (or `Cmd` on MacOS) and pressin
When querying the [backfilled data](https://docs.victoriametrics.com/#backfilling), it may be useful disabling response cache by clicking `Enable cache` checkbox.
VMUI automatically adjusts the interval between datapoints on the graph depending on the horizontal resolution and on the selected time range. The step value can be customized by clickhing `Override step value` checkbox.
VMUI allows investigating correlations between two queries on the same graph. Just click `+Query` button, enter the second query in the newly appeared input field and press `Ctrl+Enter`. Results for both queries should be displayed simultaneously on the same graph. Every query has its own vertical scale, which is displayed on the left and the right side of the graph. Lines for the second query are dashed.
See the [example VMUI at VictoriaMetrics playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/?g0.expr=100%20*%20sum(rate(process_cpu_seconds_total))%20by%20(job)&g0.range_input=1d).
@@ -829,7 +837,7 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data).
The [deduplication](#deduplication) isn't applied for the data exported in CSV. It is expected that the de-duplication is performed during data import.
The [deduplication](#deduplication) is applied for the data exported in CSV by default. It is possible to export raw data without de-duplication by passing `reduce_mem_usage=1` query arg to `/api/v1/export/csv`.
### How to export data in native format
@@ -1025,6 +1033,7 @@ VictoriaMetrics also may scrape Prometheus targets - see [these docs](#how-to-sc
VictoriaMetrics supports Prometheus-compatible relabeling for all the ingested metrics if `-relabelConfig` command-line flag points
to a file containing a list of [relabel_config](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config) entries.
The `-relabelConfig` also can point to http or https url. For example, `-relabelConfig=https://config-server/relabel_config.yml`.
See [this article with relabeling tips and tricks](https://valyala.medium.com/how-to-use-relabeling-in-prometheus-and-victoriametrics-8b90fc22c4b2).
Example contents for `-relabelConfig` file:
@@ -1074,7 +1083,7 @@ It is recommended leaving the following amounts of spare resources:
* 50% of free RAM for reducing the probability of OOM (out of memory) crashes and slowdowns during temporary spikes in workload.
* 50% of spare CPU for reducing the probability of slowdowns during temporary spikes in workload.
* At least 30% of free storage space at the directory pointed by `-storageDataPath` command-line flag.
* At least 30% of free storage space at the directory pointed by `-storageDataPath` command-line flag. See also `-storage.minFreeDiskSpaceBytes` command-line flag description [here](#list-of-command-line-flags).
## High availability
@@ -1162,20 +1171,15 @@ See [these docs](https://docs.victoriametrics.com/guides/guide-vmcluster-multipl
## Downsampling
There is no downsampling support at the moment, but:
[VictoriaMetrics Enterprise](https://victoriametrics.com/enterprise.html) supports multi-level downsampling with `-downsampling.period` command-line flag. For example:
* VictoriaMetrics is optimized for querying big amounts of raw data. See benchmark results for heavy queries
in [this article](https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae).
* VictoriaMetrics has good compression for on-disk data. See [this article](https://medium.com/@valyala/victoriametrics-achieving-better-compression-for-time-series-data-than-gorilla-317bc1f95932)
for details.
* The downsampling doesn't improve query performance on a long time range if the time range contains big number of time series due to [high churn rate](https://docs.victoriametrics.com/FAQ.html#what-is-high-churn-rate). The query performance depends on the number of unique time series on the selected time range, while downsampling doesn't reduce the number of unique time series in the database - it can reduce only the number of samples per each time series.
* `-downsampling.period=30d:5m` instructs VictoriaMetrics to [deduplicate](#deduplication) samples older than 30 days with 5 minutes interval.
These properties reduce the need of downsampling. We plan to implement downsampling in the future.
See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/36) for details.
* `-downsampling.period=30d:5m,180d:1h` instructs VictoriaMetrics to deduplicate samples older than 30 days with 5 minutes interval and to deduplicate samples older than 180 days with 1 hour interval.
It is possible to (ab)use [-dedup.minScrapeInterval](#deduplication) for basic downsampling.
For instance, if interval between the ingested data points is 15s, then `-dedup.minScrapeInterval=5m` will leave
only a single data point out of 20 initial data points per each 5m interval.
Downsampling is applied independently per each time series. It can reduce disk space usage and improve query performance if it is applied to time series with big number of samples per each series. The downsampling doesn't improve query performance if the database contains big number of time series with small number of samples per each series (aka [high churn rate](https://docs.victoriametrics.com/FAQ.html#what-is-high-churn-rate)), since downsampling doesn't reduce the number of time series. So the majority of time is spent on searching for the matching time series.
The downsampling can be evaluated for free by downloading and using enterprise binaries from [the releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
## Multi-tenancy
@@ -1217,7 +1221,8 @@ Consider setting the following command-line flags:
* `-snapshotAuthKey` for protecting `/snapshot*` endpoints. See [how to work with snapshots](#how-to-work-with-snapshots).
* `-forceMergeAuthKey` for protecting `/internal/force_merge` endpoint. See [force merge docs](#forced-merge).
* `-search.resetCacheAuthKey` for protecting `/internal/resetRollupResultCache` endpoint. See [backfilling](#backfilling) for more details.
* `-configAuthKey` for pretecting `/config` endpoint, since it may contain sensitive information such as passwords.
* `-configAuthKey` for protecting `/config` endpoint, since it may contain sensitive information such as passwords.
- `-pprofAuthKey` for protecting `/debug/pprof/*` endpoints, which can be used for [profiling](#profiling).
Explicitly set internal network interface for TCP and UDP ports for data ingestion with Graphite and OpenTSDB formats.
For example, substitute `-graphiteListenAddr=:2003` with `-graphiteListenAddr=<internal_iface_ip>:2003`.
@@ -1372,9 +1377,7 @@ See also more advanced [cardinality limiter in vmagent](https://docs.victoriamet
This prevents from ingesting metrics with too many labels. It is recommended [monitoring](#monitoring) `vm_metrics_with_dropped_labels_total`
metric in order to determine whether `-maxLabelsPerTimeseries` must be adjusted for your workload.
* If you store Graphite metrics like `foo.bar.baz` in VictoriaMetrics, then use `{__graphite__="foo.*.baz"}` syntax for selecting such metrics.
This expression is equivalent to `{__name__=~"foo[.][^.]*[.]baz"}`, but it works faster and it is easier to use when migrating from Graphite.
See also [label_graphite_group](https://docs.victoriametrics.com/MetricsQL.html#label_graphite_group) function, which allows extracting the given groups from Graphite metric names.
* If you store Graphite metrics like `foo.bar.baz` in VictoriaMetrics, then `{__graphite__="foo.*.baz"}` filter can be used for selecting such metrics. See [these docs](#selecting-graphite-metrics) for details.
* VictoriaMetrics ignores `NaN` values during data ingestion.
@@ -1440,6 +1443,18 @@ We also provide [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanag
Enterprise binaries can be downloaded and evaluated for free from [the releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
## Benchmarks
Note, that vendors (including VictoriaMetrics) are often biased when doing such tests. E.g. they try highlighting
the best parts of their product, while highlighting the worst parts of competing products.
So we encourage users and all independent third parties to conduct their becnhmarks for various products
they are evaluating in production and publish the results.
As a reference, please see [benchmarks](https://docs.victoriametrics.com/Articles.html#benchmarks) conducted by
VictoriaMetrics team. Please also see the [helm chart](https://github.com/VictoriaMetrics/benchmark)
for running ingestion benchmarks based on node_exporter metrics.
## Profiling
VictoriaMetrics provides handlers for collecting the following [Go profiles](https://blog.golang.org/profiling-go-programs):
@@ -1493,9 +1508,11 @@ Contact us with any questions regarding VictoriaMetrics at [info@victoriametrics
Feel free asking any questions regarding VictoriaMetrics:
* [slack](https://slack.victoriametrics.com/)
* [linkedin](https://www.linkedin.com/company/victoriametrics/)
* [reddit](https://www.reddit.com/r/VictoriaMetrics/)
* [telegram-en](https://t.me/VictoriaMetrics_en)
* [telegram-ru](https://t.me/VictoriaMetrics_ru1)
* [articles and talks about VictoriaMetrics in Russian](https://github.com/denisgolius/victoriametrics-ru-links)
* [google groups](https://groups.google.com/forum/#!forum/victorametrics-users)
If you like VictoriaMetrics and want to contribute, then we need the following:
@@ -1565,11 +1582,14 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
The maximum size in bytes of a single DataDog POST request to /api/v1/series
Supports the following optional suffixes for size values: KB, MB, GB, KiB, MiB, GiB (default 67108864)
-dedup.minScrapeInterval duration
Leave only the first sample in every time series per each discrete interval equal to -dedup.minScrapeInterval > 0. See https://docs.victoriametrics.com/#deduplication for details
Leave only the first sample in every time series per each discrete interval equal to -dedup.minScrapeInterval > 0. See https://docs.victoriametrics.com/#deduplication and https://docs.victoriametrics.com/#downsampling
-deleteAuthKey string
authKey for metrics' deletion via /api/v1/admin/tsdb/delete_series and /tags/delSeries
-denyQueriesOutsideRetention
Whether to deny queries outside of the configured -retentionPeriod. When set, then /api/v1/query_range would return '503 Service Unavailable' error for queries with 'from' value outside -retentionPeriod. This may be useful when multiple data sources with distinct retentions are hidden behind query-tee
-downsampling.period array
Comma-separated downsampling periods in the format 'offset:period'. For example, '30d:10m' instructs to leave a single sample per 10 minutes for samples older than 30 days. See https://docs.victoriametrics.com/#downsampling for details
Supports an array of values separated by comma or specified via multiple flags.
-dryRun
Whether to check only -promscrape.config and then exit. Unknown config entries are allowed in -promscrape.config by default. This can be changed with -promscrape.config.strictParse
-enableTCP6
@@ -1578,6 +1598,8 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
Whether to enable reading flags from environment variables additionally to command line. Command line flag values have priority over values from environment vars. Flags are read only from command line if this flag isn't set. See https://docs.victoriametrics.com/#environment-variables for more details
-envflag.prefix string
Prefix for environment variables if -envflag.enable is set
-eula
By specifying this flag, you confirm that you have an enterprise license and accept the EULA https://victoriametrics.com/assets/VM_EULA.pdf
-finalMergeDelay duration
The delay before starting final merge for per-month partition after no new data is ingested into it. Final merge may require additional disk IO and CPU resources. Final merge may increase query speed and reduce disk space usage in some cases. Zero value disables final merge
-forceFlushAuthKey string
@@ -1650,8 +1672,10 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
-maxInsertRequestSize size
The maximum size in bytes of a single Prometheus remote_write API request
Supports the following optional suffixes for size values: KB, MB, GB, KiB, MiB, GiB (default 33554432)
-maxLabelValueLen int
The maximum length of label values in the accepted time series. Longer label values are truncated. In this case the vm_too_long_label_values_total metric at /metrics page is incremented (default 16384)
-maxLabelsPerTimeseries int
The maximum number of labels accepted per time series. Superfluous labels are dropped (default 30)
The maximum number of labels accepted per time series. Superfluous labels are dropped. In this case the vm_metrics_with_dropped_labels_total metric at /metrics page is incremented (default 30)
-memory.allowedBytes size
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from OS page cache resulting in higher disk IO usage
Supports the following optional suffixes for size values: KB, MB, GB, KiB, MiB, GiB (default 0)
@@ -1681,7 +1705,7 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
-promscrape.cluster.replicationFactor int
The number of members in the cluster, which scrape the same targets. If the replication factor is greater than 2, then the deduplication must be enabled at remote storage side. See https://docs.victoriametrics.com/#deduplication (default 1)
-promscrape.config string
Optional path to Prometheus config file with 'scrape_configs' section containing targets to scrape. See https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter for details
Optional path to Prometheus config file with 'scrape_configs' section containing targets to scrape. The path can point to local file and to http url. See https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter for details
-promscrape.config.dryRun
Checks -promscrape.config file for errors and unsupported fields and then exits. Returns non-zero exit code on parsing errors and emits these errors to stderr. See also -promscrape.config.strictParse command-line flag. Pass -loggerLevel=ERROR if you don't need to see info messages in the output.
-promscrape.config.strictParse
@@ -1748,7 +1772,7 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
-promscrape.suppressScrapeErrors
Whether to suppress scrape errors logging. The last error for each target is always available at '/targets' page even if scrape errors logging is suppressed
-relabelConfig string
Optional path to a file with relabeling rules, which are applied to all the ingested metrics. See https://docs.victoriametrics.com/#relabeling for details. The config is reloaded on SIGHUP signal
Optional path to a file with relabeling rules, which are applied to all the ingested metrics. The path can point either to local file or to http url. See https://docs.victoriametrics.com/#relabeling for details. The config is reloaded on SIGHUP signal
-relabelDebug
Whether to log metrics before and after relabeling with -relabelConfig. If the -relabelDebug is enabled, then the metrics aren't sent to storage. This is useful for debugging the relabeling configs
-retentionPeriod value
@@ -1760,6 +1784,10 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
Whether to disable automatic response cache reset if a sample with timestamp outside -search.cacheTimestampOffset is inserted into VictoriaMetrics
-search.disableCache
Whether to disable response caching. This may be useful during data backfilling
-search.graphiteMaxPointsPerSeries int
The maximum number of points per series Graphite render API can return (default 1000000)
-search.graphiteStorageStep duration
The interval between datapoints stored in the database. It is used at Graphite Render API handler for normalizing the interval between datapoints in case it isn't normalized. It can be overriden by sending 'storage_step' query arg to /render API or by sending the desired interval via 'Storage-Step' http header during querying /render API (default 10s)
-search.latencyOffset duration
The time when data points become visible in query results after the collection. Too small value can result in incomplete last points for query results (default 30s)
-search.logSlowQueryDuration duration

View File

@@ -25,7 +25,7 @@ import (
var (
httpListenAddr = flag.String("httpListenAddr", ":8428", "TCP address to listen for http connections")
minScrapeInterval = flag.Duration("dedup.minScrapeInterval", 0, "Leave only the first sample in every time series per each discrete interval "+
"equal to -dedup.minScrapeInterval > 0. See https://docs.victoriametrics.com/#deduplication for details")
"equal to -dedup.minScrapeInterval > 0. See https://docs.victoriametrics.com/#deduplication and https://docs.victoriametrics.com/#downsampling")
dryRun = flag.Bool("dryRun", false, "Whether to check only -promscrape.config and then exit. "+
"Unknown config entries are allowed in -promscrape.config by default. This can be changed with -promscrape.config.strictParse")
)
@@ -51,7 +51,7 @@ func main() {
logger.Infof("starting VictoriaMetrics at %q...", *httpListenAddr)
startTime := time.Now()
storage.SetMinScrapeIntervalForDeduplication(*minScrapeInterval)
storage.SetDedupInterval(*minScrapeInterval)
vmstorage.Init(promql.ResetRollupResultCacheIfNeeded)
vmselect.Init()
vminsert.Init()

View File

@@ -46,7 +46,7 @@ to `vmagent` such as the ability to push metrics instead of pulling them. We did
Please download `vmutils-*` archive from [releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases), unpack it
and configure the following flags to the `vmagent` binary in order to start scraping Prometheus targets:
* `-promscrape.config` with the path to Prometheus config file (usually located at `/etc/prometheus/prometheus.yml`)
* `-promscrape.config` with the path to Prometheus config file (usually located at `/etc/prometheus/prometheus.yml`). The path can point either to local file or to http url.
* `-remoteWrite.url` with the remote storage endpoint such as VictoriaMetrics, the `-remoteWrite.url` argument can be specified multiple times to replicate data concurrently to an arbitrary number of remote storage systems.
Example command line:
@@ -214,15 +214,16 @@ The file pointed by `-promscrape.config` may contain `%{ENV_VAR}` placeholders w
## Loading scrape configs from multiple files
`vmagent` supports loading scrape configs from multiple files specified in the `scrape_config_files` section of `-promscrape.config` file. For example, the following `-promscrape.config` instructs `vmagent` loading scrape configs from all the `*.yml` files under `configs` directory plus a `single_scrape_config.yml` file:
`vmagent` supports loading scrape configs from multiple files specified in the `scrape_config_files` section of `-promscrape.config` file. For example, the following `-promscrape.config` instructs `vmagent` loading scrape configs from all the `*.yml` files under `configs` directory, from `single_scrape_config.yml` local file and from `https://config-server/scrape_config.yml` url:
```yml
scrape_config_files:
- configs/*.yml
- single_scrape_config.yml
- https://config-server/scrape_config.yml
```
Every referred file can contain arbitrary number of any [supported scrape configs](#how-to-collect-metrics-in-prometheus-format). There is no need in specifying top-level `scrape_configs` section in these files. For example:
Every referred file can contain arbitrary number of [supported scrape configs](#how-to-collect-metrics-in-prometheus-format). There is no need in specifying top-level `scrape_configs` section in these files. For example:
```yml
- job_name: foo
@@ -279,7 +280,7 @@ The relabeling can be defined in the following places:
* At the `scrape_config -> relabel_configs` section in `-promscrape.config` file. This relabeling is applied to target labels. This relabeling can be debugged by passing `relabel_debug: true` option to the corresponding `scrape_config` section. In this case `vmagent` logs target labels before and after the relabeling and then drops the logged target.
* At the `scrape_config -> metric_relabel_configs` section in `-promscrape.config` file. This relabeling is applied to all the scraped metrics in the given `scrape_config`. This relabeling can be debugged by passing `metric_relabel_debug: true` option to the corresponding `scrape_config` section. In this case `vmagent` logs metrics before and after the relabeling and then drops the logged metrics.
* At the `-remoteWrite.relabelConfig` file. This relabeling is aplied to all the collected metrics before sending them to remote storage. This relabeling can be debugged by passing `-remoteWrite.relabelDebug` command-line option to `vmagent`. In this case `vmagent` logs metrics before and after the relabeling and then drops all the logged metrics instead of sending them to remote storage.
* At the `-remoteWrite.relabelConfig` file. This relabeling is applied to all the collected metrics before sending them to remote storage. This relabeling can be debugged by passing `-remoteWrite.relabelDebug` command-line option to `vmagent`. In this case `vmagent` logs metrics before and after the relabeling and then drops all the logged metrics instead of sending them to remote storage.
* At the `-remoteWrite.urlRelabelConfig` files. This relabeling is applied to metrics before sending them to the corresponding `-remoteWrite.url`. This relabeling can be debugged by passing `-remoteWrite.urlRelabelDebug` command-line options to `vmagent`. In this case `vmagent` logs metrics before and after the relabeling and then drops all the logged metrics instead of sending them to the corresponding `-remoteWrite.url`.
You can read more about relabeling in the following articles:
@@ -806,7 +807,7 @@ See the docs at https://docs.victoriametrics.com/vmagent.html .
-promscrape.cluster.replicationFactor int
The number of members in the cluster, which scrape the same targets. If the replication factor is greater than 2, then the deduplication must be enabled at remote storage side. See https://docs.victoriametrics.com/#deduplication (default 1)
-promscrape.config string
Optional path to Prometheus config file with 'scrape_configs' section containing targets to scrape. See https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter for details
Optional path to Prometheus config file with 'scrape_configs' section containing targets to scrape. The path can point to local file and to http url. See https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter for details
-promscrape.config.dryRun
Checks -promscrape.config file for errors and unsupported fields and then exits. Returns non-zero exit code on parsing errors and emits these errors to stderr. See also -promscrape.config.strictParse command-line flag. Pass -loggerLevel=ERROR if you don't need to see info messages in the output.
-promscrape.config.strictParse
@@ -931,7 +932,7 @@ See the docs at https://docs.victoriametrics.com/vmagent.html .
Optional rate limit in bytes per second for data sent to -remoteWrite.url. By default the rate limit is disabled. It can be useful for limiting load on remote storage when big amounts of buffered data is sent after temporary unavailability of the remote storage
Supports array of values separated by comma or specified via multiple flags.
-remoteWrite.relabelConfig string
Optional path to file with relabel_config entries. These entries are applied to all the metrics before sending them to -remoteWrite.url. See https://docs.victoriametrics.com/vmagent.html#relabeling for details
Optional path to file with relabel_config entries. The path can point either to local file or to http url. These entries are applied to all the metrics before sending them to -remoteWrite.url. See https://docs.victoriametrics.com/vmagent.html#relabeling for details
-remoteWrite.relabelDebug
Whether to log metrics before and after relabeling with -remoteWrite.relabelConfig. If the -remoteWrite.relabelDebug is enabled, then the metrics aren't sent to remote storage. This is useful for debugging the relabeling configs
-remoteWrite.roundDigits array
@@ -966,7 +967,7 @@ See the docs at https://docs.victoriametrics.com/vmagent.html .
Remote storage URL to write data to. It must support Prometheus remote_write API. It is recommended using VictoriaMetrics as remote storage. Example url: http://<victoriametrics-host>:8428/api/v1/write . Pass multiple -remoteWrite.url flags in order to replicate data to multiple remote storage systems. See also -remoteWrite.multitenantURL
Supports an array of values separated by comma or specified via multiple flags.
-remoteWrite.urlRelabelConfig array
Optional path to relabel config for the corresponding -remoteWrite.url
Optional path to relabel config for the corresponding -remoteWrite.url. The path can point either to local file or to http url
Supports an array of values separated by comma or specified via multiple flags.
-remoteWrite.urlRelabelDebug array
Whether to log metrics before and after relabeling with -remoteWrite.urlRelabelConfig. If the -remoteWrite.urlRelabelDebug is enabled, then the metrics aren't sent to the corresponding -remoteWrite.url. This is useful for debugging the relabeling configs

View File

@@ -254,7 +254,7 @@ func (c *client) sendBlockHTTP(block []byte) bool {
again:
req, err := http.NewRequest("POST", c.remoteWriteURL, bytes.NewBuffer(block))
if err != nil {
logger.Panicf("BUG: unexected error from http.NewRequest(%q): %s", c.sanitizedURL, err)
logger.Panicf("BUG: unexpected error from http.NewRequest(%q): %s", c.sanitizedURL, err)
}
h := req.Header
h.Set("User-Agent", "vmagent")

View File

@@ -15,12 +15,14 @@ import (
var (
unparsedLabelsGlobal = flagutil.NewArray("remoteWrite.label", "Optional label in the form 'name=value' to add to all the metrics before sending them to -remoteWrite.url. "+
"Pass multiple -remoteWrite.label flags in order to add multiple labels to metrics before sending them to remote storage")
relabelConfigPathGlobal = flag.String("remoteWrite.relabelConfig", "", "Optional path to file with relabel_config entries. These entries are applied to all the metrics "+
relabelConfigPathGlobal = flag.String("remoteWrite.relabelConfig", "", "Optional path to file with relabel_config entries. "+
"The path can point either to local file or to http url. These entries are applied to all the metrics "+
"before sending them to -remoteWrite.url. See https://docs.victoriametrics.com/vmagent.html#relabeling for details")
relabelDebugGlobal = flag.Bool("remoteWrite.relabelDebug", false, "Whether to log metrics before and after relabeling with -remoteWrite.relabelConfig. "+
"If the -remoteWrite.relabelDebug is enabled, then the metrics aren't sent to remote storage. This is useful for debugging the relabeling configs")
relabelConfigPaths = flagutil.NewArray("remoteWrite.urlRelabelConfig", "Optional path to relabel config for the corresponding -remoteWrite.url")
relabelDebug = flagutil.NewArrayBool("remoteWrite.urlRelabelDebug", "Whether to log metrics before and after relabeling with -remoteWrite.urlRelabelConfig. "+
relabelConfigPaths = flagutil.NewArray("remoteWrite.urlRelabelConfig", "Optional path to relabel config for the corresponding -remoteWrite.url. "+
"The path can point either to local file or to http url")
relabelDebug = flagutil.NewArrayBool("remoteWrite.urlRelabelDebug", "Whether to log metrics before and after relabeling with -remoteWrite.urlRelabelConfig. "+
"If the -remoteWrite.urlRelabelDebug is enabled, then the metrics aren't sent to the corresponding -remoteWrite.url. "+
"This is useful for debugging the relabeling configs")
)

View File

@@ -153,6 +153,13 @@ func (ar *AlertingRule) ExecRange(ctx context.Context, start, end time.Time) ([]
return nil, fmt.Errorf("`query` template isn't supported in replay mode")
}
for _, s := range series {
// set additional labels to identify group and rule name
if ar.Name != "" {
s.SetLabel(alertNameLabel, ar.Name)
}
if !*disableAlertGroupLabel && ar.GroupName != "" {
s.SetLabel(alertGroupNameLabel, ar.GroupName)
}
// extra labels could contain templates, so we expand them first
labels, err := expandLabels(s, qFn, ar)
if err != nil {
@@ -163,13 +170,6 @@ func (ar *AlertingRule) ExecRange(ctx context.Context, start, end time.Time) ([]
// so the hash key will be consistent on restore
s.SetLabel(k, v)
}
// set additional labels to identify group and rule name
if ar.Name != "" {
s.SetLabel(alertNameLabel, ar.Name)
}
if !*disableAlertGroupLabel && ar.GroupName != "" {
s.SetLabel(alertGroupNameLabel, ar.GroupName)
}
a, err := ar.newAlert(s, time.Time{}, qFn) // initial alert
if err != nil {
return nil, fmt.Errorf("failed to create alert: %s", err)
@@ -225,6 +225,13 @@ func (ar *AlertingRule) Exec(ctx context.Context) ([]prompbmarshal.TimeSeries, e
updated := make(map[uint64]struct{})
// update list of active alerts
for _, m := range qMetrics {
// set additional labels to identify group and rule name
if ar.Name != "" {
m.SetLabel(alertNameLabel, ar.Name)
}
if !*disableAlertGroupLabel && ar.GroupName != "" {
m.SetLabel(alertGroupNameLabel, ar.GroupName)
}
// extra labels could contain templates, so we expand them first
labels, err := expandLabels(m, qFn, ar)
if err != nil {
@@ -235,14 +242,6 @@ func (ar *AlertingRule) Exec(ctx context.Context) ([]prompbmarshal.TimeSeries, e
// so the hash key will be consistent on restore
m.SetLabel(k, v)
}
// set additional labels to identify group and rule name
// set additional labels to identify group and rule name
if ar.Name != "" {
m.SetLabel(alertNameLabel, ar.Name)
}
if !*disableAlertGroupLabel && ar.GroupName != "" {
m.SetLabel(alertGroupNameLabel, ar.GroupName)
}
h := hash(m)
if _, ok := updated[h]; ok {
// duplicate may be caused by extra labels

View File

@@ -715,6 +715,44 @@ func TestAlertingRule_Template(t *testing.T) {
},
},
},
{
&AlertingRule{
Name: "ExtraTemplating",
GroupName: "Testing",
Labels: map[string]string{
"name": "alert_{{ $labels.alertname }}",
"group": "group_{{ $labels.alertgroup }}",
"instance": "{{ $labels.instance }}",
},
Annotations: map[string]string{
"summary": `Alert "{{ $labels.alertname }}({{ $labels.alertgroup }})" for instance {{ $labels.instance }}`,
"description": `Alert "{{ $labels.name }}({{ $labels.group }})" for instance {{ $labels.instance }}`,
},
alerts: make(map[uint64]*notifier.Alert),
},
[]datasource.Metric{
metricWithValueAndLabels(t, 1, "instance", "foo"),
},
map[uint64]*notifier.Alert{
hash(metricWithLabels(t, alertNameLabel, "ExtraTemplating",
"name", "alert_ExtraTemplating",
alertGroupNameLabel, "Testing",
"group", "group_Testing",
"instance", "foo")): {
Labels: map[string]string{
alertNameLabel: "ExtraTemplating",
"name": "alert_ExtraTemplating",
alertGroupNameLabel: "Testing",
"group": "group_Testing",
"instance": "foo",
},
Annotations: map[string]string{
"summary": `Alert "ExtraTemplating(Testing)" for instance foo`,
"description": `Alert "alert_ExtraTemplating(group_Testing)" for instance foo`,
},
},
},
},
}
fakeGroup := Group{Name: "TestRule_Exec"}
for _, tc := range testCases {

View File

@@ -46,8 +46,6 @@ type Group struct {
XXX map[string]interface{} `yaml:",inline"`
}
const extraLabelParam = "extra_label"
// UnmarshalYAML implements the yaml.Unmarshaler interface.
func (g *Group) UnmarshalYAML(unmarshal func(interface{}) error) error {
type group Group
@@ -68,8 +66,14 @@ func (g *Group) UnmarshalYAML(unmarshal func(interface{}) error) error {
if g.Params == nil {
g.Params = url.Values{}
}
// Sort extraFilters for consistent order for query args across runs.
extraFilters := make([]string, 0, len(g.ExtraFilterLabels))
for k, v := range g.ExtraFilterLabels {
g.Params.Add(extraLabelParam, fmt.Sprintf("%s=%s", k, v))
extraFilters = append(extraFilters, fmt.Sprintf("%s=%s", k, v))
}
sort.Strings(extraFilters)
for _, extraFilter := range extraFilters {
g.Params.Add("extra_label", extraFilter)
}
}

View File

@@ -538,7 +538,7 @@ extra_filter_labels:
rules:
- alert: ExampleAlertAlwaysFiring
expr: sum by(job) (up == 1)
`, url.Values{extraLabelParam: {"job=victoriametrics", "env=prod"}})
`, url.Values{"extra_label": {"env=prod", "job=victoriametrics"}})
})
t.Run("extra labels and params", func(t *testing.T) {
@@ -552,6 +552,6 @@ params:
rules:
- alert: ExampleAlertAlwaysFiring
expr: sum by(job) (up == 1)
`, url.Values{"nocache": {"1"}, extraLabelParam: {"env=prod", "job=victoriametrics"}})
`, url.Values{"nocache": {"1"}, "extra_label": {"env=prod", "job=victoriametrics"}})
})
}

View File

@@ -159,13 +159,15 @@ func (s *VMStorage) setPrometheusReqParams(r *http.Request, query string) {
}
}
q.Set("query", query)
if s.evaluationInterval > 0 {
// set step as evaluationInterval by default
q.Set("step", s.evaluationInterval.String())
if s.evaluationInterval > 0 { // set step as evaluationInterval by default
// always convert to seconds to keep compatibility with older
// Prometheus versions. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1943
q.Set("step", fmt.Sprintf("%ds", int(s.evaluationInterval.Seconds())))
}
if s.queryStep > 0 {
// override step with user-specified value
q.Set("step", s.queryStep.String())
if s.queryStep > 0 { // override step with user-specified value
// always convert to seconds to keep compatibility with older
// Prometheus versions. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1943
q.Set("step", fmt.Sprintf("%ds", int(s.queryStep.Seconds())))
}
r.URL.RawQuery = q.Encode()
}

View File

@@ -436,7 +436,20 @@ func TestRequestParams(t *testing.T) {
queryStep: time.Minute,
},
func(t *testing.T, r *http.Request) {
exp := fmt.Sprintf("query=%s&step=%v&time=%d", query, time.Minute, timestamp.Unix())
exp := fmt.Sprintf("query=%s&step=%ds&time=%d", query, int(time.Minute.Seconds()), timestamp.Unix())
checkEqualString(t, exp, r.URL.RawQuery)
},
},
{
"step to seconds",
false,
&VMStorage{
evaluationInterval: 3 * time.Hour,
},
func(t *testing.T, r *http.Request) {
evalInterval := 3 * time.Hour
tt := timestamp.Truncate(evalInterval)
exp := fmt.Sprintf("query=%s&step=%ds&time=%d", query, int(evalInterval.Seconds()), tt.Unix())
checkEqualString(t, exp, r.URL.RawQuery)
},
},

View File

@@ -3,7 +3,7 @@
`vmauth` is a simple auth proxy, router and [load balancer](#load-balancing) for [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics).
It reads auth credentials from `Authorization` http header ([Basic Auth](https://en.wikipedia.org/wiki/Basic_access_authentication) and `Bearer token` is supported),
matches them against configs pointed by [-auth.config](#auth-config) command-line flag and proxies incoming HTTP requests to the configured per-user `url_prefix` on successful match.
The `-auth.config` can point to either local file or to http url.
## Quick start
@@ -26,12 +26,10 @@ Pass `-help` to `vmauth` in order to see all the supported command-line flags wi
Feel free [contacting us](mailto:info@victoriametrics.com) if you need customized auth proxy for VictoriaMetrics with the support of LDAP, SSO, RBAC, SAML,
accounting and rate limiting such as [vmgateway](https://docs.victoriametrics.com/vmgateway.html).
## Load balancing
Each `url_prefix` in the [-auth.config](#auth-config) may contain either a single url or a list of urls. In the latter case `vmauth` balances load among the configured urls in a round-robin manner. This feature is useful for balancing the load among multiple `vmselect` and/or `vminsert` nodes in [VictoriaMetrics cluster](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html).
## Auth config
`-auth.config` is represented in the following simple `yml` format:
@@ -124,7 +122,6 @@ users:
The config may contain `%{ENV_VAR}` placeholders, which are substituted by the corresponding `ENV_VAR` environment variable values.
This may be useful for passing secrets to the config.
## Security
Do not transfer Basic Auth headers in plaintext over untrusted networks. Enable https. This can be done by passing the following `-tls*` command-line flags to `vmauth`:
@@ -142,7 +139,6 @@ Alternatively, [https termination proxy](https://en.wikipedia.org/wiki/TLS_termi
It is recommended protecting `/-/reload` endpoint with `-reloadAuthKey` command-line flag, so external users couldn't trigger config reload.
## Monitoring
`vmauth` exports various metrics in Prometheus exposition format at `http://vmauth-host:8427/metrics` page. It is recommended setting up regular scraping of this page
@@ -161,7 +157,6 @@ users:
It is recommended using [binary releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) - `vmauth` is located in `vmutils-*` archives there.
### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.16.
@@ -187,7 +182,6 @@ by setting it via `<ROOT_IMAGE>` environment variable. For example, the followin
ROOT_IMAGE=scratch make package-vmauth
```
## Profiling
`vmauth` provides handlers for collecting the following [Go profiles](https://blog.golang.org/profiling-go-programs):
@@ -208,7 +202,6 @@ The command for collecting CPU profile waits for 30 seconds before returning.
The collected profiles may be analyzed with [go tool pprof](https://github.com/google/pprof).
## Advanced usage
Pass `-help` command-line arg to `vmauth` in order to see all the configuration options:
@@ -221,7 +214,7 @@ vmauth authenticates and authorizes incoming requests and proxies them to Victor
See the docs at https://docs.victoriametrics.com/vmauth.html .
-auth.config string
Path to auth config. See https://docs.victoriametrics.com/vmauth.html for details on the format of this auth config
Path to auth config. It can point either to local file or to http url. See https://docs.victoriametrics.com/vmauth.html for details on the format of this auth config
-enableTCP6
Whether to enable IPv6 for listening and dialing. By default only IPv4 TCP and UDP is used
-envflag.enable
@@ -249,7 +242,7 @@ See the docs at https://docs.victoriametrics.com/vmauth.html .
-httpListenAddr string
TCP address to listen for http connections (default ":8427")
-logInvalidAuthTokens
Whether to log requests with invalid auth tokens. Such requests are always counted at vmagent_http_request_errors_total{reason="invalid_auth_token"} metric, which is exposed at /metrics page
Whether to log requests with invalid auth tokens. Such requests are always counted at vmauth_http_request_errors_total{reason="invalid_auth_token"} metric, which is exposed at /metrics page
-loggerDisableTimestamps
Whether to disable writing timestamps in logs
-loggerErrorsPerSecondLimit int
@@ -272,9 +265,9 @@ See the docs at https://docs.victoriametrics.com/vmauth.html .
-memory.allowedPercent float
Allowed percent of system memory VictoriaMetrics caches may occupy. See also -memory.allowedBytes. Too low a value may increase cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from OS page cache which will result in higher disk IO usage (default 60)
-metricsAuthKey string
Auth key for /metrics. It overrides httpAuth settings
Auth key for /metrics. It must be passed via authKey query arg. It overrides httpAuth.* settings
-pprofAuthKey string
Auth key for /debug/pprof. It overrides httpAuth settings
Auth key for /debug/pprof. It must be passed via authKey query arg. It overrides httpAuth.* settings
-reloadAuthKey string
Auth key for /-/reload http endpoint. It must be passed as authKey=...
-tls

View File

@@ -4,7 +4,6 @@ import (
"encoding/base64"
"flag"
"fmt"
"io/ioutil"
"net/url"
"os"
"regexp"
@@ -14,6 +13,7 @@ import (
"sync/atomic"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/envtemplate"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fs"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/procutil"
"github.com/VictoriaMetrics/metrics"
@@ -21,8 +21,8 @@ import (
)
var (
authConfigPath = flag.String("auth.config", "", "Path to auth config. See https://docs.victoriametrics.com/vmauth.html "+
"for details on the format of this auth config")
authConfigPath = flag.String("auth.config", "", "Path to auth config. It can point either to local file or to http url. "+
"See https://docs.victoriametrics.com/vmauth.html for details on the format of this auth config")
)
// AuthConfig represents auth config.
@@ -237,9 +237,9 @@ var authConfigWG sync.WaitGroup
var stopCh chan struct{}
func readAuthConfig(path string) (map[string]*UserInfo, error) {
data, err := ioutil.ReadFile(path)
data, err := fs.ReadFileOrHTTP(path)
if err != nil {
return nil, fmt.Errorf("cannot read %q: %w", path, err)
return nil, err
}
m, err := parseAuthConfig(data)
if err != nil {

View File

@@ -73,7 +73,6 @@ func main() {
}()
}
logger.Infof("starting http server for exporting metrics at http://%q/metrics", *httpListenAddr)
go httpserver.Serve(*httpListenAddr, nil)
srcFS, err := newSrcFS()

View File

@@ -187,18 +187,28 @@ func (c Client) GetData(series Meta, rt RetentionMeta, start int64, end int64) (
if err != nil {
return Metric{}, fmt.Errorf("failed to send GET request to %q: %s", q, err)
}
/*
There are three potential failures here, none of which should kill the entire
migration run:
1. bad response code
2. failure to read response body
3. bad format of response body
*/
if resp.StatusCode != 200 {
return Metric{}, fmt.Errorf("Bad return from OpenTSDB: %q: %v", resp.StatusCode, resp)
log.Println(fmt.Sprintf("bad response code from OpenTSDB query %v...skipping", resp.StatusCode))
return Metric{}, nil
}
defer func() { _ = resp.Body.Close() }()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return Metric{}, fmt.Errorf("could not retrieve series data from %q: %s", q, err)
log.Println("couldn't read response body from OpenTSDB query...skipping")
return Metric{}, nil
}
var output []OtsdbMetric
err = json.Unmarshal(body, &output)
if err != nil {
return Metric{}, fmt.Errorf("failed to unmarshal response from %q [%v]: %s", q, body, err)
log.Println(fmt.Sprintf("couldn't marshall response body from OpenTSDB query (%s)...skipping", body))
return Metric{}, nil
}
/*
We expect results to look like:
@@ -227,6 +237,8 @@ func (c Client) GetData(series Meta, rt RetentionMeta, start int64, end int64) (
An empty array doesn't cast to a OtsdbMetric struct well, and there's no reason to try, so we should just skip it
Because we're trying to migrate data without transformations, seeing aggregate tags could mean
we're dropping series on the floor.
In all "bad" cases, we don't end the migration, we just don't process that particular message
*/
if len(output) < 1 {
// no results returned...return an empty object without error
@@ -234,11 +246,11 @@ func (c Client) GetData(series Meta, rt RetentionMeta, start int64, end int64) (
}
if len(output) > 1 {
// multiple series returned for a single query. We can't process this right, so...
return Metric{}, fmt.Errorf("Query returned multiple results: %v", output)
return Metric{}, nil
}
if len(output[0].AggregateTags) > 0 {
// This failure means we've suppressed potential series somehow...
return Metric{}, fmt.Errorf("Query somehow has aggregate tags: %v", output[0].AggregateTags)
return Metric{}, nil
}
data := Metric{}
data.Metric = output[0].Metric
@@ -249,7 +261,7 @@ func (c Client) GetData(series Meta, rt RetentionMeta, start int64, end int64) (
*/
data, err = modifyData(data, c.Normalize)
if err != nil {
return Metric{}, fmt.Errorf("invalid series data from %q: %s", q, err)
return Metric{}, nil
}
/*

View File

@@ -43,7 +43,8 @@ var (
"Usually :4242 must be set. Doesn't work if empty")
opentsdbHTTPListenAddr = flag.String("opentsdbHTTPListenAddr", "", "TCP address to listen for OpentTSDB HTTP put requests. Usually :4242 must be set. Doesn't work if empty")
configAuthKey = flag.String("configAuthKey", "", "Authorization key for accessing /config page. It must be passed via authKey query arg")
maxLabelsPerTimeseries = flag.Int("maxLabelsPerTimeseries", 30, "The maximum number of labels accepted per time series. Superfluous labels are dropped")
maxLabelsPerTimeseries = flag.Int("maxLabelsPerTimeseries", 30, "The maximum number of labels accepted per time series. Superfluous labels are dropped. In this case the vm_metrics_with_dropped_labels_total metric at /metrics page is incremented")
maxLabelValueLen = flag.Int("maxLabelValueLen", 16*1024, "The maximum length of label values in the accepted time series. Longer label values are truncated. In this case the vm_too_long_label_values_total metric at /metrics page is incremented")
)
var (
@@ -57,6 +58,7 @@ var (
func Init() {
relabel.Init()
storage.SetMaxLabelsPerTimeseries(*maxLabelsPerTimeseries)
storage.SetMaxLabelValueLen(*maxLabelValueLen)
common.StartUnmarshalWorkers()
writeconcurrencylimiter.Init()
if len(*graphiteListenAddr) > 0 {

View File

@@ -16,6 +16,7 @@ import (
var (
relabelConfig = flag.String("relabelConfig", "", "Optional path to a file with relabeling rules, which are applied to all the ingested metrics. "+
"The path can point either to local file or to http url. "+
"See https://docs.victoriametrics.com/#relabeling for details. The config is reloaded on SIGHUP signal")
relabelDebug = flag.Bool("relabelDebug", false, "Whether to log metrics before and after relabeling with -relabelConfig. If the -relabelDebug is enabled, "+
"then the metrics aren't sent to storage. This is useful for debugging the relabeling configs")

View File

@@ -36,7 +36,6 @@ func main() {
buildinfo.Init()
logger.Init()
logger.Infof("starting http server for exporting metrics at http://%q/metrics", *httpListenAddr)
go httpserver.Serve(*httpListenAddr, nil)
srcFS, err := newSrcFS()

View File

@@ -32,7 +32,7 @@ func TagsDelSeriesHandler(startTime time.Time, w http.ResponseWriter, r *http.Re
var row graphiteparser.Row
var tagsPool []graphiteparser.Tag
ct := startTime.UnixNano() / 1e6
etfs, err := searchutils.GetEnforcedTagFiltersFromRequest(r)
etfs, err := searchutils.GetExtraTagFilters(r)
if err != nil {
return fmt.Errorf("cannot setup tag filters: %w", err)
}
@@ -53,8 +53,8 @@ func TagsDelSeriesHandler(startTime time.Time, w http.ResponseWriter, r *http.Re
Value: []byte(tag.Value),
})
}
tfs = append(tfs, etfs...)
sq := storage.NewSearchQuery(0, ct, [][]storage.TagFilter{tfs})
tfss := joinTagFilterss(tfs, etfs)
sq := storage.NewSearchQuery(0, ct, tfss)
n, err := netstorage.DeleteSeries(sq, deadline)
if err != nil {
return fmt.Errorf("cannot delete series for %q: %w", sq, err)
@@ -181,7 +181,7 @@ func TagsAutoCompleteValuesHandler(startTime time.Time, w http.ResponseWriter, r
valuePrefix := r.FormValue("valuePrefix")
exprs := r.Form["expr"]
var tagValues []string
etfs, err := searchutils.GetEnforcedTagFiltersFromRequest(r)
etfs, err := searchutils.GetExtraTagFilters(r)
if err != nil {
return fmt.Errorf("cannot setup tag filters: %w", err)
}
@@ -266,7 +266,7 @@ func TagsAutoCompleteTagsHandler(startTime time.Time, w http.ResponseWriter, r *
tagPrefix := r.FormValue("tagPrefix")
exprs := r.Form["expr"]
var labels []string
etfs, err := searchutils.GetEnforcedTagFiltersFromRequest(r)
etfs, err := searchutils.GetExtraTagFilters(r)
if err != nil {
return fmt.Errorf("cannot setup tag filters: %w", err)
}
@@ -345,7 +345,7 @@ func TagsFindSeriesHandler(startTime time.Time, w http.ResponseWriter, r *http.R
if len(exprs) == 0 {
return fmt.Errorf("expecting at least one `expr` query arg")
}
etfs, err := searchutils.GetEnforcedTagFiltersFromRequest(r)
etfs, err := searchutils.GetExtraTagFilters(r)
if err != nil {
return fmt.Errorf("cannot setup tag filters: %w", err)
}
@@ -474,14 +474,14 @@ func getInt(r *http.Request, argName string) (int, error) {
return n, nil
}
func getSearchQueryForExprs(startTime time.Time, etfs []storage.TagFilter, exprs []string) (*storage.SearchQuery, error) {
func getSearchQueryForExprs(startTime time.Time, etfs [][]storage.TagFilter, exprs []string) (*storage.SearchQuery, error) {
tfs, err := exprsToTagFilters(exprs)
if err != nil {
return nil, err
}
ct := startTime.UnixNano() / 1e6
tfs = append(tfs, etfs...)
sq := storage.NewSearchQuery(0, ct, [][]storage.TagFilter{tfs})
tfss := joinTagFilterss(tfs, etfs)
sq := storage.NewSearchQuery(0, ct, tfss)
return sq, nil
}
@@ -524,3 +524,7 @@ func parseFilterExpr(s string) (*storage.TagFilter, error) {
IsRegexp: isRegexp,
}, nil
}
func joinTagFilterss(tfs []storage.TagFilter, extraFilters [][]storage.TagFilter) [][]storage.TagFilter {
return searchutils.JoinTagFilterss([][]storage.TagFilter{tfs}, extraFilters)
}

View File

@@ -468,7 +468,8 @@ func (pts *packedTimeseries) Unpack(dst *Result, tbf *tmpBlocksFile, tr storage.
if firstErr != nil {
return firstErr
}
mergeSortBlocks(dst, sbs)
dedupInterval := storage.GetDedupInterval()
mergeSortBlocks(dst, sbs, dedupInterval)
return nil
}
@@ -489,7 +490,7 @@ var sbPool sync.Pool
var metricRowsSkipped = metrics.NewCounter(`vm_metric_rows_skipped_total{name="vmselect"}`)
func mergeSortBlocks(dst *Result, sbh sortBlocksHeap) {
func mergeSortBlocks(dst *Result, sbh sortBlocksHeap, dedupInterval int64) {
// Skip empty sort blocks, since they cannot be passed to heap.Init.
src := sbh
sbh = sbh[:0]
@@ -532,8 +533,7 @@ func mergeSortBlocks(dst *Result, sbh sortBlocksHeap) {
putSortBlock(top)
}
}
timestamps, values := storage.DeduplicateSamples(dst.Timestamps, dst.Values)
timestamps, values := storage.DeduplicateSamples(dst.Timestamps, dst.Values, dedupInterval)
dedups := len(dst.Timestamps) - len(timestamps)
dedupsDuringSelect.Add(dedups)
dst.Timestamps = timestamps

View File

@@ -129,6 +129,7 @@ func ExportCSVHandler(startTime time.Time, w http.ResponseWriter, r *http.Reques
if err != nil {
return err
}
reduceMemUsage := searchutils.GetBool(r, "reduce_mem_usage")
deadline := searchutils.GetDeadlineForExport(r, startTime)
tagFilterss, err := getTagFilterssFromRequest(r)
if err != nil {
@@ -140,30 +141,58 @@ func ExportCSVHandler(startTime time.Time, w http.ResponseWriter, r *http.Reques
defer bufferedwriter.Put(bw)
resultsCh := make(chan *quicktemplate.ByteBuffer, cgroup.AvailableCPUs())
doneCh := make(chan error)
go func() {
err := netstorage.ExportBlocks(sq, deadline, func(mn *storage.MetricName, b *storage.Block, tr storage.TimeRange) error {
if err := bw.Error(); err != nil {
return err
}
if err := b.UnmarshalData(); err != nil {
return fmt.Errorf("cannot unmarshal block during export: %s", err)
}
xb := exportBlockPool.Get().(*exportBlock)
xb.mn = mn
xb.timestamps, xb.values = b.AppendRowsWithTimeRangeFilter(xb.timestamps[:0], xb.values[:0], tr)
if len(xb.timestamps) > 0 {
bb := quicktemplate.AcquireByteBuffer()
WriteExportCSVLine(bb, xb, fieldNames)
resultsCh <- bb
}
xb.reset()
exportBlockPool.Put(xb)
return nil
})
close(resultsCh)
doneCh <- err
}()
writeCSVLine := func(xb *exportBlock) {
if len(xb.timestamps) == 0 {
return
}
bb := quicktemplate.AcquireByteBuffer()
WriteExportCSVLine(bb, xb, fieldNames)
resultsCh <- bb
}
doneCh := make(chan error, 1)
if !reduceMemUsage {
rss, err := netstorage.ProcessSearchQuery(sq, true, deadline)
if err != nil {
return fmt.Errorf("cannot fetch data for %q: %w", sq, err)
}
go func() {
err := rss.RunParallel(func(rs *netstorage.Result, workerID uint) error {
if err := bw.Error(); err != nil {
return err
}
xb := exportBlockPool.Get().(*exportBlock)
xb.mn = &rs.MetricName
xb.timestamps = rs.Timestamps
xb.values = rs.Values
writeCSVLine(xb)
xb.reset()
exportBlockPool.Put(xb)
return nil
})
close(resultsCh)
doneCh <- err
}()
} else {
go func() {
err := netstorage.ExportBlocks(sq, deadline, func(mn *storage.MetricName, b *storage.Block, tr storage.TimeRange) error {
if err := bw.Error(); err != nil {
return err
}
if err := b.UnmarshalData(); err != nil {
return fmt.Errorf("cannot unmarshal block during export: %s", err)
}
xb := exportBlockPool.Get().(*exportBlock)
xb.mn = mn
xb.timestamps, xb.values = b.AppendRowsWithTimeRangeFilter(xb.timestamps[:0], xb.values[:0], tr)
writeCSVLine(xb)
xb.reset()
exportBlockPool.Put(xb)
return nil
})
close(resultsCh)
doneCh <- err
}()
}
// Consume all the data from resultsCh.
for bb := range resultsCh {
// Do not check for error in bw.Write, since this error is checked inside netstorage.ExportBlocks above.
@@ -283,11 +312,11 @@ func ExportHandler(startTime time.Time, w http.ResponseWriter, r *http.Request)
if start >= end {
end = start + defaultStep
}
etf, err := searchutils.GetEnforcedTagFiltersFromRequest(r)
etfs, err := searchutils.GetExtraTagFilters(r)
if err != nil {
return err
}
if err := exportHandler(w, matches, etf, start, end, format, maxRowsPerLine, reduceMemUsage, deadline); err != nil {
if err := exportHandler(w, matches, etfs, start, end, format, maxRowsPerLine, reduceMemUsage, deadline); err != nil {
return fmt.Errorf("error when exporting data for queries=%q on the time range (start=%d, end=%d): %w", matches, start, end, err)
}
return nil
@@ -295,7 +324,7 @@ func ExportHandler(startTime time.Time, w http.ResponseWriter, r *http.Request)
var exportDuration = metrics.NewSummary(`vm_request_duration_seconds{path="/api/v1/export"}`)
func exportHandler(w http.ResponseWriter, matches []string, etf []storage.TagFilter, start, end int64, format string, maxRowsPerLine int, reduceMemUsage bool, deadline searchutils.Deadline) error {
func exportHandler(w http.ResponseWriter, matches []string, etfs [][]storage.TagFilter, start, end int64, format string, maxRowsPerLine int, reduceMemUsage bool, deadline searchutils.Deadline) error {
writeResponseFunc := WriteExportStdResponse
writeLineFunc := func(xb *exportBlock, resultsCh chan<- *quicktemplate.ByteBuffer) {
bb := quicktemplate.AcquireByteBuffer()
@@ -352,7 +381,7 @@ func exportHandler(w http.ResponseWriter, matches []string, etf []storage.TagFil
if err != nil {
return err
}
tagFilterss = addEnforcedFiltersToTagFilterss(tagFilterss, etf)
tagFilterss = searchutils.JoinTagFilterss(tagFilterss, etfs)
sq := storage.NewSearchQuery(start, end, tagFilterss)
w.Header().Set("Content-Type", contentType)
@@ -360,7 +389,7 @@ func exportHandler(w http.ResponseWriter, matches []string, etf []storage.TagFil
defer bufferedwriter.Put(bw)
resultsCh := make(chan *quicktemplate.ByteBuffer, cgroup.AvailableCPUs())
doneCh := make(chan error)
doneCh := make(chan error, 1)
if !reduceMemUsage {
rss, err := netstorage.ProcessSearchQuery(sq, true, deadline)
if err != nil {
@@ -478,13 +507,13 @@ func LabelValuesHandler(startTime time.Time, labelName string, w http.ResponseWr
if err := r.ParseForm(); err != nil {
return fmt.Errorf("cannot parse form values: %w", err)
}
etf, err := searchutils.GetEnforcedTagFiltersFromRequest(r)
etfs, err := searchutils.GetExtraTagFilters(r)
if err != nil {
return err
}
matches := getMatchesFromRequest(r)
var labelValues []string
if len(matches) == 0 && len(etf) == 0 {
if len(matches) == 0 && len(etfs) == 0 {
if len(r.Form["start"]) == 0 && len(r.Form["end"]) == 0 {
var err error
labelValues, err = netstorage.GetLabelValues(labelName, deadline)
@@ -527,7 +556,7 @@ func LabelValuesHandler(startTime time.Time, labelName string, w http.ResponseWr
if err != nil {
return err
}
labelValues, err = labelValuesWithMatches(labelName, matches, etf, start, end, deadline)
labelValues, err = labelValuesWithMatches(labelName, matches, etfs, start, end, deadline)
if err != nil {
return fmt.Errorf("cannot obtain label values for %q, match[]=%q, start=%d, end=%d: %w", labelName, matches, start, end, err)
}
@@ -543,7 +572,7 @@ func LabelValuesHandler(startTime time.Time, labelName string, w http.ResponseWr
return nil
}
func labelValuesWithMatches(labelName string, matches []string, etf []storage.TagFilter, start, end int64, deadline searchutils.Deadline) ([]string, error) {
func labelValuesWithMatches(labelName string, matches []string, etfs [][]storage.TagFilter, start, end int64, deadline searchutils.Deadline) ([]string, error) {
tagFilterss, err := getTagFilterssFromMatches(matches)
if err != nil {
return nil, err
@@ -564,7 +593,7 @@ func labelValuesWithMatches(labelName string, matches []string, etf []storage.Ta
if start >= end {
end = start + defaultStep
}
tagFilterss = addEnforcedFiltersToTagFilterss(tagFilterss, etf)
tagFilterss = searchutils.JoinTagFilterss(tagFilterss, etfs)
if len(tagFilterss) == 0 {
logger.Panicf("BUG: tagFilterss must be non-empty")
}
@@ -648,7 +677,7 @@ func TSDBStatusHandler(startTime time.Time, w http.ResponseWriter, r *http.Reque
if err := r.ParseForm(); err != nil {
return fmt.Errorf("cannot parse form values: %w", err)
}
etf, err := searchutils.GetEnforcedTagFiltersFromRequest(r)
etfs, err := searchutils.GetExtraTagFilters(r)
if err != nil {
return err
}
@@ -679,13 +708,13 @@ func TSDBStatusHandler(startTime time.Time, w http.ResponseWriter, r *http.Reque
topN = n
}
var status *storage.TSDBStatus
if len(matches) == 0 && len(etf) == 0 {
if len(matches) == 0 && len(etfs) == 0 {
status, err = netstorage.GetTSDBStatusForDate(deadline, date, topN)
if err != nil {
return fmt.Errorf(`cannot obtain tsdb status for date=%d, topN=%d: %w`, date, topN, err)
}
} else {
status, err = tsdbStatusWithMatches(matches, etf, date, topN, deadline)
status, err = tsdbStatusWithMatches(matches, etfs, date, topN, deadline)
if err != nil {
return fmt.Errorf("cannot obtain tsdb status with matches for date=%d, topN=%d: %w", date, topN, err)
}
@@ -700,12 +729,12 @@ func TSDBStatusHandler(startTime time.Time, w http.ResponseWriter, r *http.Reque
return nil
}
func tsdbStatusWithMatches(matches []string, etf []storage.TagFilter, date uint64, topN int, deadline searchutils.Deadline) (*storage.TSDBStatus, error) {
func tsdbStatusWithMatches(matches []string, etfs [][]storage.TagFilter, date uint64, topN int, deadline searchutils.Deadline) (*storage.TSDBStatus, error) {
tagFilterss, err := getTagFilterssFromMatches(matches)
if err != nil {
return nil, err
}
tagFilterss = addEnforcedFiltersToTagFilterss(tagFilterss, etf)
tagFilterss = searchutils.JoinTagFilterss(tagFilterss, etfs)
if len(tagFilterss) == 0 {
logger.Panicf("BUG: tagFilterss must be non-empty")
}
@@ -731,13 +760,13 @@ func LabelsHandler(startTime time.Time, w http.ResponseWriter, r *http.Request)
if err := r.ParseForm(); err != nil {
return fmt.Errorf("cannot parse form values: %w", err)
}
etf, err := searchutils.GetEnforcedTagFiltersFromRequest(r)
etfs, err := searchutils.GetExtraTagFilters(r)
if err != nil {
return err
}
matches := getMatchesFromRequest(r)
var labels []string
if len(matches) == 0 && len(etf) == 0 {
if len(matches) == 0 && len(etfs) == 0 {
if len(r.Form["start"]) == 0 && len(r.Form["end"]) == 0 {
var err error
labels, err = netstorage.GetLabels(deadline)
@@ -778,7 +807,7 @@ func LabelsHandler(startTime time.Time, w http.ResponseWriter, r *http.Request)
if err != nil {
return err
}
labels, err = labelsWithMatches(matches, etf, start, end, deadline)
labels, err = labelsWithMatches(matches, etfs, start, end, deadline)
if err != nil {
return fmt.Errorf("cannot obtain labels for match[]=%q, start=%d, end=%d: %w", matches, start, end, err)
}
@@ -794,7 +823,7 @@ func LabelsHandler(startTime time.Time, w http.ResponseWriter, r *http.Request)
return nil
}
func labelsWithMatches(matches []string, etf []storage.TagFilter, start, end int64, deadline searchutils.Deadline) ([]string, error) {
func labelsWithMatches(matches []string, etfs [][]storage.TagFilter, start, end int64, deadline searchutils.Deadline) ([]string, error) {
tagFilterss, err := getTagFilterssFromMatches(matches)
if err != nil {
return nil, err
@@ -802,7 +831,7 @@ func labelsWithMatches(matches []string, etf []storage.TagFilter, start, end int
if start >= end {
end = start + defaultStep
}
tagFilterss = addEnforcedFiltersToTagFilterss(tagFilterss, etf)
tagFilterss = searchutils.JoinTagFilterss(tagFilterss, etfs)
if len(tagFilterss) == 0 {
logger.Panicf("BUG: tagFilterss must be non-empty")
}
@@ -999,7 +1028,7 @@ func QueryHandler(startTime time.Time, w http.ResponseWriter, r *http.Request) e
if len(query) > maxQueryLen.N {
return fmt.Errorf("too long query; got %d bytes; mustn't exceed `-search.maxQueryLen=%d` bytes", len(query), maxQueryLen.N)
}
etf, err := searchutils.GetEnforcedTagFiltersFromRequest(r)
etfs, err := searchutils.GetExtraTagFilters(r)
if err != nil {
return err
}
@@ -1014,7 +1043,7 @@ func QueryHandler(startTime time.Time, w http.ResponseWriter, r *http.Request) e
if end < start {
end = start
}
if err := exportHandler(w, []string{childQuery}, etf, start, end, "promapi", 0, false, deadline); err != nil {
if err := exportHandler(w, []string{childQuery}, etfs, start, end, "promapi", 0, false, deadline); err != nil {
return fmt.Errorf("error when exporting data for query=%q on the time range (start=%d, end=%d): %w", childQuery, start, end, err)
}
queryDuration.UpdateDuration(startTime)
@@ -1030,7 +1059,7 @@ func QueryHandler(startTime time.Time, w http.ResponseWriter, r *http.Request) e
start -= offset
end := start
start = end - window
if err := queryRangeHandler(startTime, w, childQuery, start, end, step, r, ct, etf); err != nil {
if err := queryRangeHandler(startTime, w, childQuery, start, end, step, r, ct, etfs); err != nil {
return fmt.Errorf("error when executing query=%q on the time range (start=%d, end=%d, step=%d): %w", childQuery, start, end, step, err)
}
queryDuration.UpdateDuration(startTime)
@@ -1048,14 +1077,14 @@ func QueryHandler(startTime time.Time, w http.ResponseWriter, r *http.Request) e
queryOffset = 0
}
ec := promql.EvalConfig{
Start: start,
End: start,
Step: step,
QuotedRemoteAddr: httpserver.GetQuotedRemoteAddr(r),
Deadline: deadline,
LookbackDelta: lookbackDelta,
RoundDigits: getRoundDigits(r),
EnforcedTagFilters: etf,
Start: start,
End: start,
Step: step,
QuotedRemoteAddr: httpserver.GetQuotedRemoteAddr(r),
Deadline: deadline,
LookbackDelta: lookbackDelta,
RoundDigits: getRoundDigits(r),
EnforcedTagFilterss: etfs,
}
result, err := promql.Exec(&ec, query, true)
if err != nil {
@@ -1105,17 +1134,17 @@ func QueryRangeHandler(startTime time.Time, w http.ResponseWriter, r *http.Reque
if err != nil {
return err
}
etf, err := searchutils.GetEnforcedTagFiltersFromRequest(r)
etfs, err := searchutils.GetExtraTagFilters(r)
if err != nil {
return err
}
if err := queryRangeHandler(startTime, w, query, start, end, step, r, ct, etf); err != nil {
if err := queryRangeHandler(startTime, w, query, start, end, step, r, ct, etfs); err != nil {
return fmt.Errorf("error when executing query=%q on the time range (start=%d, end=%d, step=%d): %w", query, start, end, step, err)
}
return nil
}
func queryRangeHandler(startTime time.Time, w http.ResponseWriter, query string, start, end, step int64, r *http.Request, ct int64, etf []storage.TagFilter) error {
func queryRangeHandler(startTime time.Time, w http.ResponseWriter, query string, start, end, step int64, r *http.Request, ct int64, etfs [][]storage.TagFilter) error {
deadline := searchutils.GetDeadlineForQuery(r, startTime)
mayCache := !searchutils.GetBool(r, "nocache")
lookbackDelta, err := getMaxLookback(r)
@@ -1138,15 +1167,15 @@ func queryRangeHandler(startTime time.Time, w http.ResponseWriter, query string,
}
ec := promql.EvalConfig{
Start: start,
End: end,
Step: step,
QuotedRemoteAddr: httpserver.GetQuotedRemoteAddr(r),
Deadline: deadline,
MayCache: mayCache,
LookbackDelta: lookbackDelta,
RoundDigits: getRoundDigits(r),
EnforcedTagFilters: etf,
Start: start,
End: end,
Step: step,
QuotedRemoteAddr: httpserver.GetQuotedRemoteAddr(r),
Deadline: deadline,
MayCache: mayCache,
LookbackDelta: lookbackDelta,
RoundDigits: getRoundDigits(r),
EnforcedTagFilterss: etfs,
}
result, err := promql.Exec(&ec, query, false)
if err != nil {
@@ -1254,24 +1283,12 @@ func getMaxLookback(r *http.Request) (int64, error) {
return searchutils.GetDuration(r, "max_lookback", d)
}
func addEnforcedFiltersToTagFilterss(dstTfss [][]storage.TagFilter, enforcedFilters []storage.TagFilter) [][]storage.TagFilter {
if len(dstTfss) == 0 {
return [][]storage.TagFilter{
enforcedFilters,
}
}
for i := range dstTfss {
dstTfss[i] = append(dstTfss[i], enforcedFilters...)
}
return dstTfss
}
func getTagFilterssFromMatches(matches []string) ([][]storage.TagFilter, error) {
tagFilterss := make([][]storage.TagFilter, 0, len(matches))
for _, match := range matches {
tagFilters, err := promql.ParseMetricSelector(match)
tagFilters, err := searchutils.ParseMetricSelector(match)
if err != nil {
return nil, fmt.Errorf("cannot parse %q: %w", match, err)
return nil, fmt.Errorf("cannot parse matches[]=%s: %w", match, err)
}
tagFilterss = append(tagFilterss, tagFilters)
}
@@ -1287,11 +1304,11 @@ func getTagFilterssFromRequest(r *http.Request) ([][]storage.TagFilter, error) {
if err != nil {
return nil, err
}
etf, err := searchutils.GetEnforcedTagFiltersFromRequest(r)
etfs, err := searchutils.GetExtraTagFilters(r)
if err != nil {
return nil, err
}
tagFilterss = addEnforcedFiltersToTagFilterss(tagFilterss, etf)
tagFilterss = searchutils.JoinTagFilterss(tagFilterss, etfs)
return tagFilterss, nil
}

View File

@@ -6,7 +6,6 @@ import (
"testing"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/netstorage"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
)
func TestRemoveEmptyValuesAndTimeseries(t *testing.T) {
@@ -196,38 +195,3 @@ func TestAdjustLastPoints(t *testing.T) {
},
})
}
// helper for tests
func tfFromKV(k, v string) storage.TagFilter {
return storage.TagFilter{
Key: []byte(k),
Value: []byte(v),
}
}
func Test_addEnforcedFiltersToTagFilterss(t *testing.T) {
f := func(t *testing.T, dstTfss [][]storage.TagFilter, enforcedFilters []storage.TagFilter, want [][]storage.TagFilter) {
t.Helper()
got := addEnforcedFiltersToTagFilterss(dstTfss, enforcedFilters)
if !reflect.DeepEqual(got, want) {
t.Fatalf("unxpected result for addEnforcedFiltersToTagFilterss, \ngot: %v,\n want: %v", want, got)
}
}
f(t, [][]storage.TagFilter{{tfFromKV("label", "value")}},
nil,
[][]storage.TagFilter{{tfFromKV("label", "value")}})
f(t, nil,
[]storage.TagFilter{tfFromKV("ext-label", "ext-value")},
[][]storage.TagFilter{{tfFromKV("ext-label", "ext-value")}})
f(t, [][]storage.TagFilter{
{tfFromKV("l1", "v1")},
{tfFromKV("l2", "v2")},
},
[]storage.TagFilter{tfFromKV("ext-l1", "v2")},
[][]storage.TagFilter{
{tfFromKV("l1", "v1"), tfFromKV("ext-l1", "v2")},
{tfFromKV("l2", "v2"), tfFromKV("ext-l1", "v2")},
})
}

View File

@@ -29,7 +29,6 @@ var aggrFuncs = map[string]aggrFunc{
"geomean": newAggrFunc(aggrFuncGeomean),
"group": newAggrFunc(aggrFuncGroup),
"histogram": newAggrFunc(aggrFuncHistogram),
"limit_offset": aggrFuncLimitOffset,
"limitk": aggrFuncLimitK,
"mad": newAggrFunc(aggrFuncMAD),
"max": newAggrFunc(aggrFuncMax),
@@ -1005,37 +1004,12 @@ func aggrFuncLimitK(afa *aggrFuncArg) ([]*timeseries, error) {
if len(limits) > 0 {
limit = int(limits[0])
}
afe := newLimitOffsetAggrFunc(limit, 0)
return aggrFuncExt(afe, args[1], &afa.ae.Modifier, afa.ae.Limit, true)
}
func aggrFuncLimitOffset(afa *aggrFuncArg) ([]*timeseries, error) {
args := afa.args
if err := expectTransformArgsNum(args, 3); err != nil {
return nil, err
}
limit, err := getIntNumber(args[0], 0)
if err != nil {
return nil, fmt.Errorf("cannot obtain limit arg: %w", err)
}
offset, err := getIntNumber(args[1], 1)
if err != nil {
return nil, fmt.Errorf("cannot obtain offset arg: %w", err)
}
afe := newLimitOffsetAggrFunc(limit, offset)
return aggrFuncExt(afe, args[2], &afa.ae.Modifier, afa.ae.Limit, true)
}
func newLimitOffsetAggrFunc(limit, offset int) func(tss []*timeseries, modifier *metricsql.ModifierExpr) []*timeseries {
if offset < 0 {
offset = 0
}
if limit < 0 {
limit = 0
}
return func(tss []*timeseries, modifier *metricsql.ModifierExpr) []*timeseries {
afe := func(tss []*timeseries, modifier *metricsql.ModifierExpr) []*timeseries {
// Sort series by metricName hash in order to get consistent set of output series
// across multiple calls to limitk() and limit_offset() functions.
// across multiple calls to limitk() function.
// Sort series by hash in order to guarantee uniform selection across series.
type hashSeries struct {
h uint64
@@ -1056,15 +1030,12 @@ func newLimitOffsetAggrFunc(limit, offset int) func(tss []*timeseries, modifier
for i, hs := range hss {
tss[i] = hs.ts
}
if offset > len(tss) {
return nil
}
tss = tss[offset:]
if limit < len(tss) {
tss = tss[:limit]
}
return tss
}
return aggrFuncExt(afe, args[1], &afa.ae.Modifier, afa.ae.Limit, true)
}
func getHash(d *xxhash.Digest, mn *storage.MetricName) uint64 {

View File

@@ -104,8 +104,8 @@ type EvalConfig struct {
// How many decimal digits after the point to leave in response.
RoundDigits int
// EnforcedTagFilters used for apply additional label filters to query.
EnforcedTagFilters []storage.TagFilter
// EnforcedTagFilterss may contain additional label filters to use in the query.
EnforcedTagFilterss [][]storage.TagFilter
timestamps []int64
timestampsOnce sync.Once
@@ -121,7 +121,7 @@ func newEvalConfig(src *EvalConfig) *EvalConfig {
ec.MayCache = src.MayCache
ec.LookbackDelta = src.LookbackDelta
ec.RoundDigits = src.RoundDigits
ec.EnforcedTagFilters = src.EnforcedTagFilters
ec.EnforcedTagFilterss = src.EnforcedTagFilterss
// do not copy src.timestamps - they must be generated again.
return &ec
@@ -672,16 +672,15 @@ func evalRollupFuncWithMetricExpr(ec *EvalConfig, funcName string, rf rollupFunc
}
// Fetch the remaining part of the result.
tfs := toTagFilters(me.LabelFilters)
// append external filters.
tfs = append(tfs, ec.EnforcedTagFilters...)
tfs := searchutils.ToTagFilters(me.LabelFilters)
tfss := searchutils.JoinTagFilterss([][]storage.TagFilter{tfs}, ec.EnforcedTagFilterss)
minTimestamp := start - maxSilenceInterval
if window > ec.Step {
minTimestamp -= window
} else {
minTimestamp -= ec.Step
}
sq := storage.NewSearchQuery(minTimestamp, ec.End, [][]storage.TagFilter{tfs})
sq := storage.NewSearchQuery(minTimestamp, ec.End, tfss)
rss, err := netstorage.ProcessSearchQuery(sq, true, ec.Deadline)
if err != nil {
return nil, err
@@ -877,26 +876,6 @@ func mulNoOverflow(a, b int64) int64 {
return a * b
}
func toTagFilters(lfs []metricsql.LabelFilter) []storage.TagFilter {
tfs := make([]storage.TagFilter, len(lfs))
for i := range lfs {
toTagFilter(&tfs[i], &lfs[i])
}
return tfs
}
func toTagFilter(dst *storage.TagFilter, src *metricsql.LabelFilter) {
if src.Label != "__name__" {
dst.Key = []byte(src.Label)
} else {
// This is required for storage.Search.
dst.Key = nil
}
dst.Value = []byte(src.Value)
dst.IsRegexp = src.IsRegexp
dst.IsNegative = src.IsNegative
}
func dropStaleNaNs(funcName string, values []float64, timestamps []int64) ([]float64, []int64) {
if *noStaleMarkers || funcName == "default_rollup" {
// Do not drop Prometheus staleness marks (aka stale NaNs) for default_rollup() function,

View File

@@ -593,6 +593,29 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run("timestamp(alias(time()>=1600))", func(t *testing.T) {
t.Parallel()
q := `timestamp(alias(time()>=1600,"foo"))`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{nan, nan, nan, 1600, 1800, 2000},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run("timestamp_with_name(alias(time()>=1600))", func(t *testing.T) {
t.Parallel()
q := `timestamp_with_name(alias(time()>=1600,"foo"))`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{nan, nan, nan, 1600, 1800, 2000},
Timestamps: timestampsExpected,
}
r.MetricName.MetricGroup = []byte("foo")
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run("time()/100", func(t *testing.T) {
t.Parallel()
q := `time()/100`
@@ -2055,6 +2078,24 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r1, r2, r3}
f(q, resultExpected)
})
t.Run(`limit_offset`, func(t *testing.T) {
t.Parallel()
q := `limit_offset(1, 1, sort_by_label((
label_set(time()*1, "foo", "y"),
label_set(time()*2, "foo", "a"),
label_set(time()*3, "foo", "x"),
), "foo"))`
r := netstorage.Result{
Values: []float64{3000, 3600, 4200, 4800, 5400, 6000},
Timestamps: timestampsExpected,
}
r.MetricName.Tags = []storage.Tag{{
Key: []byte("foo"),
Value: []byte("x"),
}}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`sum(label_graphite_group)`, func(t *testing.T) {
t.Parallel()
q := `sort(sum by (__name__) (
@@ -5161,21 +5202,6 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r1}
f(q, resultExpected)
})
t.Run(`limit_offset()`, func(t *testing.T) {
t.Parallel()
q := `limit_offset(1, 0, (label_set(10, "foo", "bar"), label_set(time()/150, "xbaz", "sss")))`
r1 := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{10, 10, 10, 10, 10, 10},
Timestamps: timestampsExpected,
}
r1.MetricName.Tags = []storage.Tag{{
Key: []byte("foo"),
Value: []byte("bar"),
}}
resultExpected := []netstorage.Result{r1}
f(q, resultExpected)
})
t.Run(`limitk(10)`, func(t *testing.T) {
t.Parallel()
q := `sort(limitk(10, label_set(10, "foo", "bar") or label_set(time()/150, "baz", "sss")))`
@@ -6244,6 +6270,22 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`increase_prometheus(time())`, func(t *testing.T) {
t.Parallel()
q := `increase_prometheus(time())`
f(q, nil)
})
t.Run(`increase_prometheus(time()[201s])`, func(t *testing.T) {
t.Parallel()
q := `increase_prometheus(time()[201s])`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{200, 200, 200, 200, 200, 200},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`running_max(1)`, func(t *testing.T) {
t.Parallel()
q := `running_max(1)`
@@ -6486,6 +6528,22 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`delta_prometheus(time())`, func(t *testing.T) {
t.Parallel()
q := `delta_prometheus(time())`
f(q, nil)
})
t.Run(`delta_prometheus(time()[201s])`, func(t *testing.T) {
t.Parallel()
q := `delta_prometheus(time()[201s])`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{200, 200, 200, 200, 200, 200},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`median_over_time("foo")`, func(t *testing.T) {
t.Parallel()
q := `median_over_time("foo")`
@@ -7379,6 +7437,7 @@ func TestExecError(t *testing.T) {
f(`sort_by_label()`)
f(`sort_by_label_desc()`)
f(`timestamp()`)
f(`timestamp_with_name()`)
f(`vector()`)
f(`histogram_quantile()`)
f(`histogram_quantiles()`)
@@ -7476,6 +7535,12 @@ func TestExecError(t *testing.T) {
f(`bitmap_xor()`)
f(`quantiles()`)
f(`limit_offset()`)
f(`increase()`)
f(`increase_prometheus()`)
f(`changes()`)
f(`changes_prometheus()`)
f(`delta()`)
f(`delta_prometheus()`)
// Invalid argument type
f(`median_over_time({}, 2)`)

View File

@@ -1,9 +1,6 @@
package promql
import (
"fmt"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
"github.com/VictoriaMetrics/metricsql"
)
@@ -43,21 +40,3 @@ func IsMetricSelectorWithRollup(s string) (childQuery string, window, offset *me
wrappedQuery := me.AppendString(nil)
return string(wrappedQuery), re.Window, re.Offset
}
// ParseMetricSelector parses s containing PromQL metric selector
// and returns the corresponding LabelFilters.
func ParseMetricSelector(s string) ([]storage.TagFilter, error) {
expr, err := parsePromQLWithCache(s)
if err != nil {
return nil, err
}
me, ok := expr.(*metricsql.MetricExpr)
if !ok {
return nil, fmt.Errorf("expecting metricSelector; got %q", expr.AppendString(nil))
}
if len(me.LabelFilters) == 0 {
return nil, fmt.Errorf("labelFilters cannot be empty")
}
tfs := toTagFilters(me.LabelFilters)
return tfs, nil
}

View File

@@ -1,50 +0,0 @@
package promql
import (
"testing"
)
func TestParseMetricSelectorSuccess(t *testing.T) {
f := func(s string) {
t.Helper()
tfs, err := ParseMetricSelector(s)
if err != nil {
t.Fatalf("unexpected error when parsing %q: %s", s, err)
}
if tfs == nil {
t.Fatalf("expecting non-nil tfs when parsing %q", s)
}
}
f("foo")
f(":foo")
f(" :fo:bar.baz")
f(`a{}`)
f(`{foo="bar"}`)
f(`{:f:oo=~"bar.+"}`)
f(`foo {bar != "baz"}`)
f(` foo { bar !~ "^ddd(x+)$", a="ss", __name__="sffd"} `)
f(`(foo)`)
f(`\п\р\и\в\е{\ы="111"}`)
}
func TestParseMetricSelectorError(t *testing.T) {
f := func(s string) {
t.Helper()
tfs, err := ParseMetricSelector(s)
if err == nil {
t.Fatalf("expecting non-nil error when parsing %q", s)
}
if tfs != nil {
t.Fatalf("expecting nil tfs when parsing %q", s)
}
}
f("")
f(`{}`)
f(`foo bar`)
f(`foo+bar`)
f(`sum(bar)`)
f(`x{y}`)
f(`x{y+z}`)
f(`foo[5m]`)
f(`foo offset 5m`)
}

View File

@@ -24,6 +24,7 @@ var rollupFuncs = map[string]newRollupFunc{
"ascent_over_time": newRollupFuncOneArg(rollupAscentOverTime),
"avg_over_time": newRollupFuncOneArg(rollupAvg),
"changes": newRollupFuncOneArg(rollupChanges),
"changes_prometheus": newRollupFuncOneArg(rollupChangesPrometheus),
"count_eq_over_time": newRollupCountEQ,
"count_gt_over_time": newRollupCountGT,
"count_le_over_time": newRollupCountLE,
@@ -32,6 +33,7 @@ var rollupFuncs = map[string]newRollupFunc{
"decreases_over_time": newRollupFuncOneArg(rollupDecreases),
"default_rollup": newRollupFuncOneArg(rollupDefault), // default rollup func
"delta": newRollupFuncOneArg(rollupDelta),
"delta_prometheus": newRollupFuncOneArg(rollupDeltaPrometheus),
"deriv": newRollupFuncOneArg(rollupDerivSlow),
"deriv_fast": newRollupFuncOneArg(rollupDerivFast),
"descent_over_time": newRollupFuncOneArg(rollupDescentOverTime),
@@ -45,8 +47,9 @@ var rollupFuncs = map[string]newRollupFunc{
"holt_winters": newRollupHoltWinters,
"idelta": newRollupFuncOneArg(rollupIdelta),
"ideriv": newRollupFuncOneArg(rollupIderiv),
"increase": newRollupFuncOneArg(rollupDelta), // + rollupFuncsRemoveCounterResets
"increase_pure": newRollupFuncOneArg(rollupIncreasePure), // + rollupFuncsRemoveCounterResets
"increase": newRollupFuncOneArg(rollupDelta), // + rollupFuncsRemoveCounterResets
"increase_prometheus": newRollupFuncOneArg(rollupDeltaPrometheus), // + rollupFuncsRemoveCounterResets
"increase_pure": newRollupFuncOneArg(rollupIncreasePure), // + rollupFuncsRemoveCounterResets
"increases_over_time": newRollupFuncOneArg(rollupIncreases),
"integrate": newRollupFuncOneArg(rollupIntegrate),
"irate": newRollupFuncOneArg(rollupIderiv), // + rollupFuncsRemoveCounterResets
@@ -82,11 +85,12 @@ var rollupFuncs = map[string]newRollupFunc{
// `timestamp` function must return timestamp for the last datapoint on the current window
// in order to properly handle offset and timestamps unaligned to the current step.
// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/415 for details.
"timestamp": newRollupFuncOneArg(rollupTlast),
"tlast_over_time": newRollupFuncOneArg(rollupTlast),
"tmax_over_time": newRollupFuncOneArg(rollupTmax),
"tmin_over_time": newRollupFuncOneArg(rollupTmin),
"zscore_over_time": newRollupFuncOneArg(rollupZScoreOverTime),
"timestamp": newRollupFuncOneArg(rollupTlast),
"timestamp_with_name": newRollupFuncOneArg(rollupTlast), // + rollupFuncsKeepMetricGroup
"tlast_over_time": newRollupFuncOneArg(rollupTlast),
"tmax_over_time": newRollupFuncOneArg(rollupTmax),
"tmin_over_time": newRollupFuncOneArg(rollupTmin),
"zscore_over_time": newRollupFuncOneArg(rollupZScoreOverTime),
}
// rollupAggrFuncs are functions that can be passed to `aggr_over_time()`
@@ -130,6 +134,7 @@ var rollupAggrFuncs = map[string]rollupFunc{
"sum2_over_time": rollupSum2,
"tfirst_over_time": rollupTfirst,
"timestamp": rollupTlast,
"timestamp_with_name": rollupTlast,
"tlast_over_time": rollupTlast,
"tmax_over_time": rollupTmax,
"tmin_over_time": rollupTmin,
@@ -159,12 +164,13 @@ var rollupFuncsCanAdjustWindow = map[string]bool{
}
var rollupFuncsRemoveCounterResets = map[string]bool{
"increase": true,
"increase_pure": true,
"irate": true,
"rate": true,
"rollup_increase": true,
"rollup_rate": true,
"increase": true,
"increase_prometheus": true,
"increase_pure": true,
"irate": true,
"rate": true,
"rollup_increase": true,
"rollup_rate": true,
}
// These functions don't change physical meaning of input time series,
@@ -186,6 +192,7 @@ var rollupFuncsKeepMetricGroup = map[string]bool{
"quantiles_over_time": true,
"rollup": true,
"rollup_candlestick": true,
"timestamp_with_name": true,
}
func getRollupAggrFuncNames(expr metricsql.Expr) ([]string, error) {
@@ -1482,6 +1489,18 @@ func rollupDelta(rfa *rollupFuncArg) float64 {
return values[len(values)-1] - prevValue
}
func rollupDeltaPrometheus(rfa *rollupFuncArg) float64 {
// There is no need in handling NaNs here, since they must be cleaned up
// before calling rollup funcs.
values := rfa.values
// Just return the difference between the last and the first sample like Prometheus does.
// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1962
if len(values) < 2 {
return nan
}
return values[len(values)-1] - values[0]
}
func rollupIdelta(rfa *rollupFuncArg) float64 {
// There is no need in handling NaNs here, since they must be cleaned up
// before calling rollup funcs.
@@ -1641,6 +1660,26 @@ func rollupScrapeInterval(rfa *rollupFuncArg) float64 {
return (float64(timestamps[len(timestamps)-1]-rfa.prevTimestamp) / 1e3) / float64(len(timestamps))
}
func rollupChangesPrometheus(rfa *rollupFuncArg) float64 {
// There is no need in handling NaNs here, since they must be cleaned up
// before calling rollup funcs.
values := rfa.values
// Do not take into account rfa.prevValue like Prometheus does.
// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1962
if len(values) < 1 {
return nan
}
prevValue := values[0]
n := 0
for _, v := range values[1:] {
if v != prevValue {
n++
prevValue = v
}
}
return float64(n)
}
func rollupChanges(rfa *rollupFuncArg) float64 {
// There is no need in handling NaNs here, since they must be cleaned up
// before calling rollup funcs.

View File

@@ -194,7 +194,7 @@ func (rrc *rollupResultCache) Get(ec *EvalConfig, expr metricsql.Expr, window in
bb := bbPool.Get()
defer bbPool.Put(bb)
bb.B = marshalRollupResultCacheKey(bb.B[:0], expr, window, ec.Step, ec.EnforcedTagFilters)
bb.B = marshalRollupResultCacheKey(bb.B[:0], expr, window, ec.Step, ec.EnforcedTagFilterss)
metainfoBuf := rrc.c.Get(nil, bb.B)
if len(metainfoBuf) == 0 {
return nil, ec.Start
@@ -214,7 +214,7 @@ func (rrc *rollupResultCache) Get(ec *EvalConfig, expr metricsql.Expr, window in
if len(compressedResultBuf.B) == 0 {
mi.RemoveKey(key)
metainfoBuf = mi.Marshal(metainfoBuf[:0])
bb.B = marshalRollupResultCacheKey(bb.B[:0], expr, window, ec.Step, ec.EnforcedTagFilters)
bb.B = marshalRollupResultCacheKey(bb.B[:0], expr, window, ec.Step, ec.EnforcedTagFilterss)
rrc.c.Set(bb.B, metainfoBuf)
return nil, ec.Start
}
@@ -317,7 +317,7 @@ func (rrc *rollupResultCache) Put(ec *EvalConfig, expr metricsql.Expr, window in
bb.B = key.Marshal(bb.B[:0])
rrc.c.SetBig(bb.B, compressedResultBuf.B)
bb.B = marshalRollupResultCacheKey(bb.B[:0], expr, window, ec.Step, ec.EnforcedTagFilters)
bb.B = marshalRollupResultCacheKey(bb.B[:0], expr, window, ec.Step, ec.EnforcedTagFilterss)
metainfoBuf := rrc.c.Get(nil, bb.B)
var mi rollupResultCacheMetainfo
if len(metainfoBuf) > 0 {
@@ -347,14 +347,19 @@ var tooBigRollupResults = metrics.NewCounter("vm_too_big_rollup_results_total")
// Increment this value every time the format of the cache changes.
const rollupResultCacheVersion = 8
func marshalRollupResultCacheKey(dst []byte, expr metricsql.Expr, window, step int64, filters []storage.TagFilter) []byte {
func marshalRollupResultCacheKey(dst []byte, expr metricsql.Expr, window, step int64, etfs [][]storage.TagFilter) []byte {
dst = append(dst, rollupResultCacheVersion)
dst = encoding.MarshalUint64(dst, rollupResultCacheKeyPrefix)
dst = encoding.MarshalInt64(dst, window)
dst = encoding.MarshalInt64(dst, step)
dst = expr.AppendString(dst)
for _, f := range filters {
dst = f.Marshal(dst)
for i, etf := range etfs {
for _, f := range etf {
dst = f.Marshal(dst)
}
if i+1 < len(etfs) {
dst = append(dst, '|')
}
}
return dst
}

View File

@@ -490,11 +490,14 @@ func TestRollupNewRollupFuncSuccess(t *testing.T) {
f("default_rollup", 34)
f("changes", 11)
f("changes_prometheus", 10)
f("delta", 34)
f("delta_prometheus", -89)
f("deriv", -266.85860231406093)
f("deriv_fast", -712)
f("idelta", 0)
f("increase", 398)
f("increase_prometheus", 275)
f("irate", 0)
f("rate", 2200)
f("resets", 5)
@@ -524,6 +527,7 @@ func TestRollupNewRollupFuncSuccess(t *testing.T) {
f("descent_over_time", 231)
f("zscore_over_time", -0.4254336383156416)
f("timestamp", 0.13)
f("timestamp_with_name", 0.13)
f("mode_over_time", 34)
f("rate_over_sum", 4520)
}
@@ -850,6 +854,20 @@ func TestRollupFuncsNoWindow(t *testing.T) {
timestampsExpected := []int64{0, 40, 80, 120, 160}
testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
})
t.Run("delta_prometheus", func(t *testing.T) {
rc := rollupConfig{
Func: rollupDeltaPrometheus,
Start: 0,
End: 160,
Step: 40,
Window: 0,
}
rc.Timestamps = getTimestamps(rc.Start, rc.End, rc.Step)
values := rc.Do(nil, testValues, testTimestamps)
valuesExpected := []float64{nan, -102, -42, -10, nan}
timestampsExpected := []int64{0, 40, 80, 120, 160}
testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
})
t.Run("idelta", func(t *testing.T) {
rc := rollupConfig{
Func: rollupIdelta,
@@ -948,6 +966,20 @@ func TestRollupFuncsNoWindow(t *testing.T) {
timestampsExpected := []int64{0, 40, 80, 120, 160}
testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
})
t.Run("changes_prometheus", func(t *testing.T) {
rc := rollupConfig{
Func: rollupChangesPrometheus,
Start: 0,
End: 160,
Step: 40,
Window: 0,
}
rc.Timestamps = getTimestamps(rc.Start, rc.End, rc.Step)
values := rc.Do(nil, testValues, testTimestamps)
valuesExpected := []float64{nan, 3, 3, 2, 0}
timestampsExpected := []int64{0, 40, 80, 120, 160}
testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
})
t.Run("changes_small_window", func(t *testing.T) {
rc := rollupConfig{
Func: rollupChanges,

View File

@@ -11,6 +11,7 @@ import (
"strings"
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/searchutils"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/decimal"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
@@ -69,6 +70,7 @@ var transformFuncs = map[string]transformFunc{
"label_transform": transformLabelTransform,
"label_uppercase": transformLabelUppercase,
"label_value": transformLabelValue,
"limit_offset": transformLimitOffset,
"ln": newTransformFuncOneArg(transformLn),
"log2": newTransformFuncOneArg(transformLog2),
"log10": newTransformFuncOneArg(transformLog10),
@@ -218,7 +220,7 @@ func getAbsentTimeseries(ec *EvalConfig, arg metricsql.Expr) []*timeseries {
if !ok {
return rvs
}
tfs := toTagFilters(me.LabelFilters)
tfs := searchutils.ToTagFilters(me.LabelFilters)
for i := range tfs {
tf := &tfs[i]
if len(tf.Key) == 0 {
@@ -1770,6 +1772,29 @@ func transformLabelGraphiteGroup(tfa *transformFuncArg) ([]*timeseries, error) {
var dotSeparator = []byte(".")
func transformLimitOffset(tfa *transformFuncArg) ([]*timeseries, error) {
args := tfa.args
if err := expectTransformArgsNum(args, 3); err != nil {
return nil, err
}
limit, err := getIntNumber(args[0], 0)
if err != nil {
return nil, fmt.Errorf("cannot obtain limit arg: %w", err)
}
offset, err := getIntNumber(args[1], 1)
if err != nil {
return nil, fmt.Errorf("cannot obtain offset arg: %w", err)
}
rvs := args[2]
if len(rvs) >= offset {
rvs = rvs[offset:]
}
if len(rvs) > limit {
rvs = rvs[:limit]
}
return rvs, nil
}
func transformLn(v float64) float64 {
return math.Log(v)
}

View File

@@ -9,9 +9,8 @@ import (
"strings"
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fasttime"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
"github.com/VictoriaMetrics/metricsql"
)
@@ -198,15 +197,17 @@ func (d *Deadline) String() string {
return fmt.Sprintf("%.3f seconds (elapsed %.3f seconds); the timeout can be adjusted with `%s` command-line flag", d.timeout.Seconds(), elapsed.Seconds(), d.flagHint)
}
// GetEnforcedTagFiltersFromRequest returns additional filters from request.
func GetEnforcedTagFiltersFromRequest(r *http.Request) ([]storage.TagFilter, error) {
// fast path.
extraLabels := r.Form["extra_label"]
if len(extraLabels) == 0 {
return nil, nil
}
tagFilters := make([]storage.TagFilter, 0, len(extraLabels))
for _, match := range extraLabels {
// GetExtraTagFilters returns additional label filters from request.
//
// Label filters can be present in extra_label and extra_filters[] query args.
// They are combined. For example, the following query args:
// extra_label=t1=v1&extra_label=t2=v2&extra_filters[]={env="prod",team="devops"}&extra_filters={env=~"dev|staging",team!="devops"}
// should be translated to the following filters joined with "or":
// {env="prod",team="devops",t1="v1",t2="v2"}
// {env=~"dev|staging",team!="devops",t1="v1",t2="v2"}
func GetExtraTagFilters(r *http.Request) ([][]storage.TagFilter, error) {
var tagFilters []storage.TagFilter
for _, match := range r.Form["extra_label"] {
tmp := strings.SplitN(match, "=", 2)
if len(tmp) != 2 {
return nil, fmt.Errorf("`extra_label` query arg must have the format `name=value`; got %q", match)
@@ -216,5 +217,79 @@ func GetEnforcedTagFiltersFromRequest(r *http.Request) ([]storage.TagFilter, err
Value: []byte(tmp[1]),
})
}
return tagFilters, nil
extraFilters := r.Form["extra_filters"]
extraFilters = append(extraFilters, r.Form["extra_filters[]"]...)
if len(extraFilters) == 0 {
if len(tagFilters) == 0 {
return nil, nil
}
return [][]storage.TagFilter{tagFilters}, nil
}
var etfs [][]storage.TagFilter
for _, extraFilter := range extraFilters {
tfs, err := ParseMetricSelector(extraFilter)
if err != nil {
return nil, fmt.Errorf("cannot parse extra_filters=%s: %w", extraFilter, err)
}
tfs = append(tfs, tagFilters...)
etfs = append(etfs, tfs)
}
return etfs, nil
}
// JoinTagFilterss adds etfs to every src filter and returns the result.
func JoinTagFilterss(src, etfs [][]storage.TagFilter) [][]storage.TagFilter {
if len(src) == 0 {
return etfs
}
if len(etfs) == 0 {
return src
}
var dst [][]storage.TagFilter
for _, tf := range src {
for _, etf := range etfs {
tfs := append([]storage.TagFilter{}, tf...)
tfs = append(tfs, etf...)
dst = append(dst, tfs)
}
}
return dst
}
// ParseMetricSelector parses s containing PromQL metric selector and returns the corresponding LabelFilters.
func ParseMetricSelector(s string) ([]storage.TagFilter, error) {
expr, err := metricsql.Parse(s)
if err != nil {
return nil, err
}
me, ok := expr.(*metricsql.MetricExpr)
if !ok {
return nil, fmt.Errorf("expecting metricSelector; got %q", expr.AppendString(nil))
}
if len(me.LabelFilters) == 0 {
return nil, fmt.Errorf("labelFilters cannot be empty")
}
tfs := ToTagFilters(me.LabelFilters)
return tfs, nil
}
// ToTagFilters converts lfs to a slice of storage.TagFilter
func ToTagFilters(lfs []metricsql.LabelFilter) []storage.TagFilter {
tfs := make([]storage.TagFilter, len(lfs))
for i := range lfs {
toTagFilter(&tfs[i], &lfs[i])
}
return tfs
}
func toTagFilter(dst *storage.TagFilter, src *metricsql.LabelFilter) {
if src.Label != "__name__" {
dst.Key = []byte(src.Label)
} else {
// This is required for storage.Search.
dst.Key = nil
}
dst.Value = []byte(src.Value)
dst.IsRegexp = src.IsRegexp
dst.IsNegative = src.IsNegative
}

View File

@@ -5,6 +5,7 @@ import (
"net/http"
"net/url"
"reflect"
"strconv"
"testing"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
@@ -80,47 +81,238 @@ func TestGetTimeError(t *testing.T) {
f("292277025-08-18T07:12:54.999999998Z")
}
// helper for tests
func tfFromKV(k, v string) storage.TagFilter {
return storage.TagFilter{
Key: []byte(k),
Value: []byte(v),
}
}
func TestGetEnforcedTagFiltersFromRequest(t *testing.T) {
httpReqWithForm := func(tfs []string) *http.Request {
func TestGetExtraTagFilters(t *testing.T) {
httpReqWithForm := func(qs string) *http.Request {
q, err := url.ParseQuery(qs)
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
return &http.Request{
Form: map[string][]string{
"extra_label": tfs,
},
Form: q,
}
}
f := func(t *testing.T, r *http.Request, want []storage.TagFilter, wantErr bool) {
f := func(t *testing.T, r *http.Request, want []string, wantErr bool) {
t.Helper()
got, err := GetEnforcedTagFiltersFromRequest(r)
result, err := GetExtraTagFilters(r)
if (err != nil) != wantErr {
t.Fatalf("unexpected error: %v", err)
}
got := tagFilterssToStrings(result)
if !reflect.DeepEqual(got, want) {
t.Fatalf("unxpected result for getEnforcedTagFiltersFromRequest, \ngot: %v,\n want: %v", want, got)
t.Fatalf("unxpected result for GetExtraTagFilters\ngot: %s\nwant: %s", got, want)
}
}
f(t, httpReqWithForm([]string{"label=value"}),
[]storage.TagFilter{
tfFromKV("label", "value"),
},
false)
f(t, httpReqWithForm([]string{"job=vmagent", "dc=gce"}),
[]storage.TagFilter{tfFromKV("job", "vmagent"), tfFromKV("dc", "gce")},
f(t, httpReqWithForm("extra_label=label=value"),
[]string{`{label="value"}`},
false,
)
f(t, httpReqWithForm([]string{"bad_filter"}),
f(t, httpReqWithForm("extra_label=job=vmagent&extra_label=dc=gce"),
[]string{`{job="vmagent",dc="gce"}`},
false,
)
f(t, httpReqWithForm(`extra_filters={foo="bar"}`),
[]string{`{foo="bar"}`},
false,
)
f(t, httpReqWithForm(`extra_filters={foo="bar"}&extra_filters[]={baz!~"aa",x=~"y"}`),
[]string{
`{foo="bar"}`,
`{baz!~"aa",x=~"y"}`,
},
false,
)
f(t, httpReqWithForm(`extra_label=job=vmagent&extra_label=dc=gce&extra_filters={foo="bar"}`),
[]string{`{foo="bar",job="vmagent",dc="gce"}`},
false,
)
f(t, httpReqWithForm(`extra_label=job=vmagent&extra_label=dc=gce&extra_filters[]={foo="bar"}&extra_filters[]={x=~"y|z",a="b"}`),
[]string{
`{foo="bar",job="vmagent",dc="gce"}`,
`{x=~"y|z",a="b",job="vmagent",dc="gce"}`,
},
false,
)
f(t, httpReqWithForm("extra_label=bad_filter"),
nil,
true,
)
f(t, &http.Request{},
nil, false)
f(t, httpReqWithForm(`extra_filters={bad_filter}`),
nil,
true,
)
f(t, httpReqWithForm(`extra_filters[]={bad_filter}`),
nil,
true,
)
f(t, httpReqWithForm(""),
nil,
false,
)
}
func TestParseMetricSelectorSuccess(t *testing.T) {
f := func(s string) {
t.Helper()
tfs, err := ParseMetricSelector(s)
if err != nil {
t.Fatalf("unexpected error when parsing %q: %s", s, err)
}
if tfs == nil {
t.Fatalf("expecting non-nil tfs when parsing %q", s)
}
}
f("foo")
f(":foo")
f(" :fo:bar.baz")
f(`a{}`)
f(`{foo="bar"}`)
f(`{:f:oo=~"bar.+"}`)
f(`foo {bar != "baz"}`)
f(` foo { bar !~ "^ddd(x+)$", a="ss", __name__="sffd"} `)
f(`(foo)`)
f(`\п\р\и\в\е{\ы="111"}`)
}
func TestParseMetricSelectorError(t *testing.T) {
f := func(s string) {
t.Helper()
tfs, err := ParseMetricSelector(s)
if err == nil {
t.Fatalf("expecting non-nil error when parsing %q", s)
}
if tfs != nil {
t.Fatalf("expecting nil tfs when parsing %q", s)
}
}
f("")
f(`{}`)
f(`foo bar`)
f(`foo+bar`)
f(`sum(bar)`)
f(`x{y}`)
f(`x{y+z}`)
f(`foo[5m]`)
f(`foo offset 5m`)
}
func TestJoinTagFilterss(t *testing.T) {
f := func(t *testing.T, src, etfs [][]storage.TagFilter, want []string) {
t.Helper()
result := JoinTagFilterss(src, etfs)
got := tagFilterssToStrings(result)
if !reflect.DeepEqual(got, want) {
t.Fatalf("unxpected result for JoinTagFilterss\ngot: %s\nwant: %v", got, want)
}
}
// Single tag filter
f(t, [][]storage.TagFilter{
mustParseMetricSelector(`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4"}`),
}, nil, []string{
`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4"}`,
})
// Miltiple tag filters
f(t, [][]storage.TagFilter{
mustParseMetricSelector(`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4"}`),
mustParseMetricSelector(`{k5=~"v5"}`),
}, nil, []string{
`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4"}`,
`{k5=~"v5"}`,
})
// Single extra filter
f(t, nil, [][]storage.TagFilter{
mustParseMetricSelector(`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4"}`),
}, []string{
`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4"}`,
})
// Multiple extra filters
f(t, nil, [][]storage.TagFilter{
mustParseMetricSelector(`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4"}`),
mustParseMetricSelector(`{k5=~"v5"}`),
}, []string{
`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4"}`,
`{k5=~"v5"}`,
})
// Single tag filter and a single extra filter
f(t, [][]storage.TagFilter{
mustParseMetricSelector(`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4"}`),
}, [][]storage.TagFilter{
mustParseMetricSelector(`{k5=~"v5"}`),
}, []string{
`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4",k5=~"v5"}`,
})
// Multiple tag filters and a single extra filter
f(t, [][]storage.TagFilter{
mustParseMetricSelector(`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4"}`),
mustParseMetricSelector(`{k5=~"v5"}`),
}, [][]storage.TagFilter{
mustParseMetricSelector(`{k6=~"v6"}`),
}, []string{
`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4",k6=~"v6"}`,
`{k5=~"v5",k6=~"v6"}`,
})
// Single tag filter and multiple extra filters
f(t, [][]storage.TagFilter{
mustParseMetricSelector(`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4"}`),
}, [][]storage.TagFilter{
mustParseMetricSelector(`{k5=~"v5"}`),
mustParseMetricSelector(`{k6=~"v6"}`),
}, []string{
`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4",k5=~"v5"}`,
`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4",k6=~"v6"}`,
})
// Multiple tag filters and multiple extra filters
f(t, [][]storage.TagFilter{
mustParseMetricSelector(`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4"}`),
mustParseMetricSelector(`{k5=~"v5"}`),
}, [][]storage.TagFilter{
mustParseMetricSelector(`{k6=~"v6"}`),
mustParseMetricSelector(`{k7=~"v7"}`),
}, []string{
`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4",k6=~"v6"}`,
`{k1="v1",k2=~"v2",k3!="v3",k4!~"v4",k7=~"v7"}`,
`{k5=~"v5",k6=~"v6"}`,
`{k5=~"v5",k7=~"v7"}`,
})
}
func mustParseMetricSelector(s string) []storage.TagFilter {
tf, err := ParseMetricSelector(s)
if err != nil {
panic(fmt.Errorf("cannot parse %q: %w", s, err))
}
return tf
}
func tagFilterssToStrings(tfss [][]storage.TagFilter) []string {
var a []string
for _, tfs := range tfss {
a = append(a, tagFiltersToString(tfs))
}
return a
}
func tagFiltersToString(tfs []storage.TagFilter) string {
b := []byte("{")
for i, tf := range tfs {
b = append(b, tf.Key...)
if tf.IsNegative {
if tf.IsRegexp {
b = append(b, "!~"...)
} else {
b = append(b, "!="...)
}
} else {
if tf.IsRegexp {
b = append(b, "=~"...)
} else {
b = append(b, "="...)
}
}
b = strconv.AppendQuote(b, string(tf.Value))
if i+1 < len(tfs) {
b = append(b, ',')
}
}
b = append(b, '}')
return string(b)
}

View File

@@ -1,19 +1,12 @@
{
"files": {
"main.css": "./static/css/main.674f8c98.chunk.css",
"main.js": "./static/js/main.f4cab8bc.chunk.js",
"runtime-main.js": "./static/js/runtime-main.f698388d.js",
"static/css/2.77671664.chunk.css": "./static/css/2.77671664.chunk.css",
"static/js/2.bfcf9c30.chunk.js": "./static/js/2.bfcf9c30.chunk.js",
"static/js/3.e51afffb.chunk.js": "./static/js/3.e51afffb.chunk.js",
"index.html": "./index.html",
"static/js/2.bfcf9c30.chunk.js.LICENSE.txt": "./static/js/2.bfcf9c30.chunk.js.LICENSE.txt"
"main.css": "./static/css/main.a33903a8.css",
"main.js": "./static/js/main.23f635e5.js",
"static/js/27.85f0e2b0.chunk.js": "./static/js/27.85f0e2b0.chunk.js",
"index.html": "./index.html"
},
"entrypoints": [
"static/js/runtime-main.f698388d.js",
"static/css/2.77671664.chunk.css",
"static/js/2.bfcf9c30.chunk.js",
"static/css/main.674f8c98.chunk.css",
"static/js/main.f4cab8bc.chunk.js"
"static/css/main.a33903a8.css",
"static/js/main.23f635e5.js"
]
}

View File

@@ -1 +1 @@
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1"/><meta name="theme-color" content="#000000"/><meta name="description" content="VM-UI is a metric explorer for Victoria Metrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,400,500,700&display=swap"/><link href="./static/css/2.77671664.chunk.css" rel="stylesheet"><link href="./static/css/main.674f8c98.chunk.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div><script>!function(e){function r(r){for(var n,i,a=r[0],c=r[1],f=r[2],s=0,p=[];s<a.length;s++)i=a[s],Object.prototype.hasOwnProperty.call(o,i)&&o[i]&&p.push(o[i][0]),o[i]=0;for(n in c)Object.prototype.hasOwnProperty.call(c,n)&&(e[n]=c[n]);for(l&&l(r);p.length;)p.shift()();return u.push.apply(u,f||[]),t()}function t(){for(var e,r=0;r<u.length;r++){for(var t=u[r],n=!0,a=1;a<t.length;a++){var c=t[a];0!==o[c]&&(n=!1)}n&&(u.splice(r--,1),e=i(i.s=t[0]))}return e}var n={},o={1:0},u=[];function i(r){if(n[r])return n[r].exports;var t=n[r]={i:r,l:!1,exports:{}};return e[r].call(t.exports,t,t.exports,i),t.l=!0,t.exports}i.e=function(e){var r=[],t=o[e];if(0!==t)if(t)r.push(t[2]);else{var n=new Promise((function(r,n){t=o[e]=[r,n]}));r.push(t[2]=n);var u,a=document.createElement("script");a.charset="utf-8",a.timeout=120,i.nc&&a.setAttribute("nonce",i.nc),a.src=function(e){return i.p+"static/js/"+({}[e]||e)+"."+{3:"e51afffb"}[e]+".chunk.js"}(e);var c=new Error;u=function(r){a.onerror=a.onload=null,clearTimeout(f);var t=o[e];if(0!==t){if(t){var n=r&&("load"===r.type?"missing":r.type),u=r&&r.target&&r.target.src;c.message="Loading chunk "+e+" failed.\n("+n+": "+u+")",c.name="ChunkLoadError",c.type=n,c.request=u,t[1](c)}o[e]=void 0}};var f=setTimeout((function(){u({type:"timeout",target:a})}),12e4);a.onerror=a.onload=u,document.head.appendChild(a)}return Promise.all(r)},i.m=e,i.c=n,i.d=function(e,r,t){i.o(e,r)||Object.defineProperty(e,r,{enumerable:!0,get:t})},i.r=function(e){"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},i.t=function(e,r){if(1&r&&(e=i(e)),8&r)return e;if(4&r&&"object"==typeof e&&e&&e.__esModule)return e;var t=Object.create(null);if(i.r(t),Object.defineProperty(t,"default",{enumerable:!0,value:e}),2&r&&"string"!=typeof e)for(var n in e)i.d(t,n,function(r){return e[r]}.bind(null,n));return t},i.n=function(e){var r=e&&e.__esModule?function(){return e.default}:function(){return e};return i.d(r,"a",r),r},i.o=function(e,r){return Object.prototype.hasOwnProperty.call(e,r)},i.p="./",i.oe=function(e){throw console.error(e),e};var a=this.webpackJsonpvmui=this.webpackJsonpvmui||[],c=a.push.bind(a);a.push=r,a=a.slice();for(var f=0;f<a.length;f++)r(a[f]);var l=c;t()}([])</script><script src="./static/js/2.bfcf9c30.chunk.js"></script><script src="./static/js/main.f4cab8bc.chunk.js"></script></body></html>
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1"/><meta name="theme-color" content="#000000"/><meta name="description" content="VM-UI is a metric explorer for Victoria Metrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,400,500,700&display=swap"/><script defer="defer" src="./static/js/main.23f635e5.js"></script><link href="./static/css/main.a33903a8.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>

View File

@@ -1 +0,0 @@
.uplot,.uplot *,.uplot :after,.uplot :before{box-sizing:border-box}.uplot{font-family:system-ui,-apple-system,"Segoe UI",Roboto,"Helvetica Neue",Arial,"Noto Sans",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";line-height:1.5;width:-webkit-min-content;width:min-content}.u-title{text-align:center;font-size:18px;font-weight:700}.u-wrap{position:relative;-webkit-user-select:none;-ms-user-select:none;user-select:none}.u-over,.u-under{position:absolute}.u-under{overflow:hidden}.uplot canvas{display:block;position:relative;width:100%;height:100%}.u-axis{position:absolute}.u-legend{font-size:14px;margin:auto;text-align:center}.u-inline{display:block}.u-inline *{display:inline-block}.u-inline tr{margin-right:16px}.u-legend th{font-weight:600}.u-legend th>*{vertical-align:middle;display:inline-block}.u-legend .u-marker{width:1em;height:1em;margin-right:4px;background-clip:padding-box!important}.u-inline.u-live th:after{content:":";vertical-align:middle}.u-inline:not(.u-live) .u-value{display:none}.u-series>*{padding:4px}.u-series th{cursor:pointer}.u-legend .u-off>*{opacity:.3}.u-select{background:rgba(0,0,0,.07)}.u-cursor-x,.u-cursor-y,.u-select{position:absolute;pointer-events:none}.u-cursor-x,.u-cursor-y{left:0;top:0;will-change:transform;z-index:100}.u-hz .u-cursor-x,.u-vt .u-cursor-y{height:100%;border-right:1px dashed #607d8b}.u-hz .u-cursor-y,.u-vt .u-cursor-x{width:100%;border-bottom:1px dashed #607d8b}.u-cursor-pt{position:absolute;top:0;left:0;border-radius:50%;border:0 solid;pointer-events:none;will-change:transform;z-index:100;background-clip:padding-box!important}.u-axis.u-off,.u-cursor-pt.u-off,.u-cursor-x.u-off,.u-cursor-y.u-off,.u-select.u-off{display:none}

View File

@@ -1 +0,0 @@
body{font-family:-apple-system,BlinkMacSystemFont,"Segoe UI","Roboto","Oxygen","Ubuntu","Cantarell","Fira Sans","Droid Sans","Helvetica Neue",sans-serif;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale}code{font-family:source-code-pro,Menlo,Monaco,Consolas,"Courier New",monospace}.MuiAccordionSummary-content{margin:10px 0!important}.cm-activeLine{background-color:inherit!important}.cm-editor{border-radius:4px;border:1px solid #b9b9b9;font-size:10px}.one-line-scroll .cm-editor{height:24px}.cm-gutters{border-radius:4px 0 0 4px;height:100%}.multi-line-scroll .cm-content,.multi-line-scroll .cm-gutters{min-height:64px!important}.one-line-scroll .cm-content,.one-line-scroll .cm-gutters{min-height:auto}.u-tooltip{position:absolute;display:none;grid-gap:12px;max-width:300px;padding:8px;border-radius:4px;background:rgba(57,57,57,.9);color:#fff;font-size:10px;line-height:1.4em;font-weight:500;word-wrap:break-word;font-family:monospace;pointer-events:none;z-index:100}.u-tooltip-data{display:flex;flex-wrap:wrap;align-items:center;font-size:11px;line-height:150%}.u-tooltip-data__value{padding:4px;font-weight:700}.u-tooltip__info{display:grid;grid-gap:4px}.u-tooltip__marker{width:12px;height:12px;margin-right:4px}.legendWrapper{margin-top:20px}.legendItem{display:inline-grid;grid-template-columns:auto auto;grid-gap:4px;align-items:center;justify-content:start;padding:5px 10px;background-color:#fff;cursor:pointer;transition:.2s ease}.legendItemHide{text-decoration:line-through;opacity:.5}.legendItem:hover{background-color:rgba(0,0,0,.1)}.legendMarker{width:12px;height:12px;border-width:2px;border-style:solid;box-sizing:border-box;transition:.2s ease}.legendLabel{font-size:12px;font-weight:600}

View File

@@ -0,0 +1 @@
body{-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen,Ubuntu,Cantarell,Fira Sans,Droid Sans,Helvetica Neue,sans-serif}code{font-family:source-code-pro,Menlo,Monaco,Consolas,Courier New,monospace}.MuiAccordionSummary-content{margin:0!important}.cm-activeLine{background-color:inherit!important}.cm-editor{border:none;border-radius:4px;font-size:10px}.cm-gutters{border:none!important;border-radius:4px 0 0 4px;height:100%;overflow:hidden}.cm-activeLineGutter,.cm-gutters{background-color:#fff!important}.query-editor .cm-scroller{align-items:center!important}.query-editor .cm-editor.cm-focused{outline:none}.query-editor-container{border:1px solid #b9b9b9;border-radius:4px;padding:12px;position:relative}.query-editor-container_focus{border:1px solid #3f51b5}.query-editor-container_error{border-color:#ff4141}.query-editor-container-one-line .query-editor .cm-editor{height:22px}.query-editor-container-one-line{padding:6px}.query-editor-label{background-color:#fff;color:rgba(0,0,0,.6);font-size:12px;font-weight:400;left:4px;letter-spacing:normal;line-height:1;max-width:calc(133% - 24px);overflow:hidden;padding:0 5px;position:absolute;text-overflow:ellipsis;top:-.71875em;-webkit-transform:scale(.75);transform:scale(.75);white-space:nowrap;z-index:1}.query-editor-container_error .query-editor-label{color:#ff4141}.uplot,.uplot *,.uplot :after,.uplot :before{box-sizing:border-box}.uplot{font-family:system-ui,-apple-system,Segoe UI,Roboto,Helvetica Neue,Arial,Noto Sans,sans-serif,Apple Color Emoji,Segoe UI Emoji,Segoe UI Symbol,Noto Color Emoji;line-height:1.5;width:-webkit-min-content;width:min-content}.u-title{font-size:18px;font-weight:700;text-align:center}.u-wrap{position:relative;-webkit-user-select:none;-ms-user-select:none;user-select:none}.u-over,.u-under{position:absolute}.u-under{overflow:hidden}.uplot canvas{display:block;height:100%;position:relative;width:100%}.u-axis{position:absolute}.u-legend{font-size:14px;margin:auto;text-align:center}.u-inline{display:block}.u-inline *{display:inline-block}.u-inline tr{margin-right:16px}.u-legend th{font-weight:600}.u-legend th>*{display:inline-block;vertical-align:middle}.u-legend .u-marker{background-clip:padding-box!important;height:1em;margin-right:4px;width:1em}.u-inline.u-live th:after{content:":";vertical-align:middle}.u-inline:not(.u-live) .u-value{display:none}.u-series>*{padding:4px}.u-series th{cursor:pointer}.u-legend .u-off>*{opacity:.3}.u-select{background:rgba(0,0,0,.07)}.u-cursor-x,.u-cursor-y,.u-select{pointer-events:none;position:absolute}.u-cursor-x,.u-cursor-y{left:0;top:0;will-change:transform;z-index:100}.u-hz .u-cursor-x,.u-vt .u-cursor-y{border-right:1px dashed #607d8b;height:100%}.u-hz .u-cursor-y,.u-vt .u-cursor-x{border-bottom:1px dashed #607d8b;width:100%}.u-cursor-pt{background-clip:padding-box!important;border:0 solid;border-radius:50%;left:0;pointer-events:none;position:absolute;top:0;will-change:transform;z-index:100}.u-axis.u-off,.u-cursor-pt.u-off,.u-cursor-x.u-off,.u-cursor-y.u-off,.u-select.u-off,.u-tooltip{display:none}.u-tooltip{grid-gap:12px;word-wrap:break-word;background:rgba(57,57,57,.9);border-radius:4px;color:#fff;font-family:monospace;font-size:10px;font-weight:500;line-height:1.4em;max-width:300px;padding:8px;pointer-events:none;position:absolute;z-index:100}.u-tooltip-data{align-items:center;display:flex;flex-wrap:wrap;font-size:11px;line-height:150%}.u-tooltip-data__value{font-weight:700;padding:4px}.u-tooltip__info{grid-gap:4px;display:grid}.u-tooltip__marker{height:12px;margin-right:4px;width:12px}.legendWrapper{grid-gap:20px;cursor:default;display:grid;grid-template-columns:repeat(auto-fit,minmax(400px,1fr));margin-top:20px}.legendGroup{margin-bottom:24px}.legendGroupTitle{align-items:center;display:flex;font-size:11px;padding:10px 0 5px}.legendGroupLine{margin:0 10px}.legendItem{grid-gap:6px;align-items:start;background-color:#fff;cursor:pointer;display:inline-grid;grid-template-columns:auto auto;justify-content:start;padding:5px 10px;transition:.2s ease}.legendItemHide{opacity:.5;text-decoration:line-through}.legendItem:hover{background-color:rgba(0,0,0,.1)}.legendMarker{border-style:solid;border-width:2px;box-sizing:border-box;height:12px;margin:3px 0;transition:.2s ease;width:12px}.legendLabel{font-size:11px;font-weight:400}

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1 @@
"use strict";(self.webpackChunkvmui=self.webpackChunkvmui||[]).push([[27],{4027:function(e,n,t){t.r(n),t.d(n,{getCLS:function(){return y},getFCP:function(){return g},getFID:function(){return C},getLCP:function(){return P},getTTFB:function(){return D}});var i,r,a,o,u=function(e,n){return{name:e,value:void 0===n?-1:n,delta:0,entries:[],id:"v2-".concat(Date.now(),"-").concat(Math.floor(8999999999999*Math.random())+1e12)}},c=function(e,n){try{if(PerformanceObserver.supportedEntryTypes.includes(e)){if("first-input"===e&&!("PerformanceEventTiming"in self))return;var t=new PerformanceObserver((function(e){return e.getEntries().map(n)}));return t.observe({type:e,buffered:!0}),t}}catch(e){}},f=function(e,n){var t=function t(i){"pagehide"!==i.type&&"hidden"!==document.visibilityState||(e(i),n&&(removeEventListener("visibilitychange",t,!0),removeEventListener("pagehide",t,!0)))};addEventListener("visibilitychange",t,!0),addEventListener("pagehide",t,!0)},s=function(e){addEventListener("pageshow",(function(n){n.persisted&&e(n)}),!0)},m=function(e,n,t){var i;return function(r){n.value>=0&&(r||t)&&(n.delta=n.value-(i||0),(n.delta||void 0===i)&&(i=n.value,e(n)))}},v=-1,p=function(){return"hidden"===document.visibilityState?0:1/0},d=function(){f((function(e){var n=e.timeStamp;v=n}),!0)},l=function(){return v<0&&(v=p(),d(),s((function(){setTimeout((function(){v=p(),d()}),0)}))),{get firstHiddenTime(){return v}}},g=function(e,n){var t,i=l(),r=u("FCP"),a=function(e){"first-contentful-paint"===e.name&&(f&&f.disconnect(),e.startTime<i.firstHiddenTime&&(r.value=e.startTime,r.entries.push(e),t(!0)))},o=window.performance&&performance.getEntriesByName&&performance.getEntriesByName("first-contentful-paint")[0],f=o?null:c("paint",a);(o||f)&&(t=m(e,r,n),o&&a(o),s((function(i){r=u("FCP"),t=m(e,r,n),requestAnimationFrame((function(){requestAnimationFrame((function(){r.value=performance.now()-i.timeStamp,t(!0)}))}))})))},h=!1,T=-1,y=function(e,n){h||(g((function(e){T=e.value})),h=!0);var t,i=function(n){T>-1&&e(n)},r=u("CLS",0),a=0,o=[],v=function(e){if(!e.hadRecentInput){var n=o[0],i=o[o.length-1];a&&e.startTime-i.startTime<1e3&&e.startTime-n.startTime<5e3?(a+=e.value,o.push(e)):(a=e.value,o=[e]),a>r.value&&(r.value=a,r.entries=o,t())}},p=c("layout-shift",v);p&&(t=m(i,r,n),f((function(){p.takeRecords().map(v),t(!0)})),s((function(){a=0,T=-1,r=u("CLS",0),t=m(i,r,n)})))},E={passive:!0,capture:!0},w=new Date,L=function(e,n){i||(i=n,r=e,a=new Date,F(removeEventListener),S())},S=function(){if(r>=0&&r<a-w){var e={entryType:"first-input",name:i.type,target:i.target,cancelable:i.cancelable,startTime:i.timeStamp,processingStart:i.timeStamp+r};o.forEach((function(n){n(e)})),o=[]}},b=function(e){if(e.cancelable){var n=(e.timeStamp>1e12?new Date:performance.now())-e.timeStamp;"pointerdown"==e.type?function(e,n){var t=function(){L(e,n),r()},i=function(){r()},r=function(){removeEventListener("pointerup",t,E),removeEventListener("pointercancel",i,E)};addEventListener("pointerup",t,E),addEventListener("pointercancel",i,E)}(n,e):L(n,e)}},F=function(e){["mousedown","keydown","touchstart","pointerdown"].forEach((function(n){return e(n,b,E)}))},C=function(e,n){var t,a=l(),v=u("FID"),p=function(e){e.startTime<a.firstHiddenTime&&(v.value=e.processingStart-e.startTime,v.entries.push(e),t(!0))},d=c("first-input",p);t=m(e,v,n),d&&f((function(){d.takeRecords().map(p),d.disconnect()}),!0),d&&s((function(){var a;v=u("FID"),t=m(e,v,n),o=[],r=-1,i=null,F(addEventListener),a=p,o.push(a),S()}))},k={},P=function(e,n){var t,i=l(),r=u("LCP"),a=function(e){var n=e.startTime;n<i.firstHiddenTime&&(r.value=n,r.entries.push(e)),t()},o=c("largest-contentful-paint",a);if(o){t=m(e,r,n);var v=function(){k[r.id]||(o.takeRecords().map(a),o.disconnect(),k[r.id]=!0,t(!0))};["keydown","click"].forEach((function(e){addEventListener(e,v,{once:!0,capture:!0})})),f(v,!0),s((function(i){r=u("LCP"),t=m(e,r,n),requestAnimationFrame((function(){requestAnimationFrame((function(){r.value=performance.now()-i.timeStamp,k[r.id]=!0,t(!0)}))}))}))}},D=function(e){var n,t=u("TTFB");n=function(){try{var n=performance.getEntriesByType("navigation")[0]||function(){var e=performance.timing,n={entryType:"navigation",startTime:0};for(var t in e)"navigationStart"!==t&&"toJSON"!==t&&(n[t]=Math.max(e[t]-e.navigationStart,0));return n}();if(t.value=t.delta=n.responseStart,t.value<0||t.value>performance.now())return;t.entries=[n],e(t)}catch(e){}},"complete"===document.readyState?setTimeout(n,0):addEventListener("pageshow",n)}}}]);

View File

@@ -1 +0,0 @@
(this.webpackJsonpvmui=this.webpackJsonpvmui||[]).push([[3],{351:function(e,t,n){"use strict";n.r(t),n.d(t,"getCLS",(function(){return y})),n.d(t,"getFCP",(function(){return g})),n.d(t,"getFID",(function(){return C})),n.d(t,"getLCP",(function(){return k})),n.d(t,"getTTFB",(function(){return D}));var i,r,a,o,u=function(e,t){return{name:e,value:void 0===t?-1:t,delta:0,entries:[],id:"v2-".concat(Date.now(),"-").concat(Math.floor(8999999999999*Math.random())+1e12)}},c=function(e,t){try{if(PerformanceObserver.supportedEntryTypes.includes(e)){if("first-input"===e&&!("PerformanceEventTiming"in self))return;var n=new PerformanceObserver((function(e){return e.getEntries().map(t)}));return n.observe({type:e,buffered:!0}),n}}catch(e){}},f=function(e,t){var n=function n(i){"pagehide"!==i.type&&"hidden"!==document.visibilityState||(e(i),t&&(removeEventListener("visibilitychange",n,!0),removeEventListener("pagehide",n,!0)))};addEventListener("visibilitychange",n,!0),addEventListener("pagehide",n,!0)},s=function(e){addEventListener("pageshow",(function(t){t.persisted&&e(t)}),!0)},m=function(e,t,n){var i;return function(r){t.value>=0&&(r||n)&&(t.delta=t.value-(i||0),(t.delta||void 0===i)&&(i=t.value,e(t)))}},v=-1,p=function(){return"hidden"===document.visibilityState?0:1/0},d=function(){f((function(e){var t=e.timeStamp;v=t}),!0)},l=function(){return v<0&&(v=p(),d(),s((function(){setTimeout((function(){v=p(),d()}),0)}))),{get firstHiddenTime(){return v}}},g=function(e,t){var n,i=l(),r=u("FCP"),a=function(e){"first-contentful-paint"===e.name&&(f&&f.disconnect(),e.startTime<i.firstHiddenTime&&(r.value=e.startTime,r.entries.push(e),n(!0)))},o=window.performance&&performance.getEntriesByName&&performance.getEntriesByName("first-contentful-paint")[0],f=o?null:c("paint",a);(o||f)&&(n=m(e,r,t),o&&a(o),s((function(i){r=u("FCP"),n=m(e,r,t),requestAnimationFrame((function(){requestAnimationFrame((function(){r.value=performance.now()-i.timeStamp,n(!0)}))}))})))},h=!1,T=-1,y=function(e,t){h||(g((function(e){T=e.value})),h=!0);var n,i=function(t){T>-1&&e(t)},r=u("CLS",0),a=0,o=[],v=function(e){if(!e.hadRecentInput){var t=o[0],i=o[o.length-1];a&&e.startTime-i.startTime<1e3&&e.startTime-t.startTime<5e3?(a+=e.value,o.push(e)):(a=e.value,o=[e]),a>r.value&&(r.value=a,r.entries=o,n())}},p=c("layout-shift",v);p&&(n=m(i,r,t),f((function(){p.takeRecords().map(v),n(!0)})),s((function(){a=0,T=-1,r=u("CLS",0),n=m(i,r,t)})))},E={passive:!0,capture:!0},w=new Date,L=function(e,t){i||(i=t,r=e,a=new Date,F(removeEventListener),S())},S=function(){if(r>=0&&r<a-w){var e={entryType:"first-input",name:i.type,target:i.target,cancelable:i.cancelable,startTime:i.timeStamp,processingStart:i.timeStamp+r};o.forEach((function(t){t(e)})),o=[]}},b=function(e){if(e.cancelable){var t=(e.timeStamp>1e12?new Date:performance.now())-e.timeStamp;"pointerdown"==e.type?function(e,t){var n=function(){L(e,t),r()},i=function(){r()},r=function(){removeEventListener("pointerup",n,E),removeEventListener("pointercancel",i,E)};addEventListener("pointerup",n,E),addEventListener("pointercancel",i,E)}(t,e):L(t,e)}},F=function(e){["mousedown","keydown","touchstart","pointerdown"].forEach((function(t){return e(t,b,E)}))},C=function(e,t){var n,a=l(),v=u("FID"),p=function(e){e.startTime<a.firstHiddenTime&&(v.value=e.processingStart-e.startTime,v.entries.push(e),n(!0))},d=c("first-input",p);n=m(e,v,t),d&&f((function(){d.takeRecords().map(p),d.disconnect()}),!0),d&&s((function(){var a;v=u("FID"),n=m(e,v,t),o=[],r=-1,i=null,F(addEventListener),a=p,o.push(a),S()}))},P={},k=function(e,t){var n,i=l(),r=u("LCP"),a=function(e){var t=e.startTime;t<i.firstHiddenTime&&(r.value=t,r.entries.push(e)),n()},o=c("largest-contentful-paint",a);if(o){n=m(e,r,t);var v=function(){P[r.id]||(o.takeRecords().map(a),o.disconnect(),P[r.id]=!0,n(!0))};["keydown","click"].forEach((function(e){addEventListener(e,v,{once:!0,capture:!0})})),f(v,!0),s((function(i){r=u("LCP"),n=m(e,r,t),requestAnimationFrame((function(){requestAnimationFrame((function(){r.value=performance.now()-i.timeStamp,P[r.id]=!0,n(!0)}))}))}))}},D=function(e){var t,n=u("TTFB");t=function(){try{var t=performance.getEntriesByType("navigation")[0]||function(){var e=performance.timing,t={entryType:"navigation",startTime:0};for(var n in e)"navigationStart"!==n&&"toJSON"!==n&&(t[n]=Math.max(e[n]-e.navigationStart,0));return t}();if(n.value=n.delta=t.responseStart,n.value<0||n.value>performance.now())return;n.entries=[t],e(n)}catch(e){}},"complete"===document.readyState?setTimeout(t,0):addEventListener("pageshow",t)}}}]);

File diff suppressed because one or more lines are too long

View File

@@ -20,7 +20,7 @@ object-assign
* @license MIT
*/
/** @license MUI v5.2.0
/** @license MUI v5.2.4
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.

File diff suppressed because one or more lines are too long

View File

@@ -1 +0,0 @@
!function(e){function r(r){for(var n,i,a=r[0],c=r[1],f=r[2],s=0,p=[];s<a.length;s++)i=a[s],Object.prototype.hasOwnProperty.call(o,i)&&o[i]&&p.push(o[i][0]),o[i]=0;for(n in c)Object.prototype.hasOwnProperty.call(c,n)&&(e[n]=c[n]);for(l&&l(r);p.length;)p.shift()();return u.push.apply(u,f||[]),t()}function t(){for(var e,r=0;r<u.length;r++){for(var t=u[r],n=!0,a=1;a<t.length;a++){var c=t[a];0!==o[c]&&(n=!1)}n&&(u.splice(r--,1),e=i(i.s=t[0]))}return e}var n={},o={1:0},u=[];function i(r){if(n[r])return n[r].exports;var t=n[r]={i:r,l:!1,exports:{}};return e[r].call(t.exports,t,t.exports,i),t.l=!0,t.exports}i.e=function(e){var r=[],t=o[e];if(0!==t)if(t)r.push(t[2]);else{var n=new Promise((function(r,n){t=o[e]=[r,n]}));r.push(t[2]=n);var u,a=document.createElement("script");a.charset="utf-8",a.timeout=120,i.nc&&a.setAttribute("nonce",i.nc),a.src=function(e){return i.p+"static/js/"+({}[e]||e)+"."+{3:"e51afffb"}[e]+".chunk.js"}(e);var c=new Error;u=function(r){a.onerror=a.onload=null,clearTimeout(f);var t=o[e];if(0!==t){if(t){var n=r&&("load"===r.type?"missing":r.type),u=r&&r.target&&r.target.src;c.message="Loading chunk "+e+" failed.\n("+n+": "+u+")",c.name="ChunkLoadError",c.type=n,c.request=u,t[1](c)}o[e]=void 0}};var f=setTimeout((function(){u({type:"timeout",target:a})}),12e4);a.onerror=a.onload=u,document.head.appendChild(a)}return Promise.all(r)},i.m=e,i.c=n,i.d=function(e,r,t){i.o(e,r)||Object.defineProperty(e,r,{enumerable:!0,get:t})},i.r=function(e){"undefined"!==typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},i.t=function(e,r){if(1&r&&(e=i(e)),8&r)return e;if(4&r&&"object"===typeof e&&e&&e.__esModule)return e;var t=Object.create(null);if(i.r(t),Object.defineProperty(t,"default",{enumerable:!0,value:e}),2&r&&"string"!=typeof e)for(var n in e)i.d(t,n,function(r){return e[r]}.bind(null,n));return t},i.n=function(e){var r=e&&e.__esModule?function(){return e.default}:function(){return e};return i.d(r,"a",r),r},i.o=function(e,r){return Object.prototype.hasOwnProperty.call(e,r)},i.p="./",i.oe=function(e){throw console.error(e),e};var a=this.webpackJsonpvmui=this.webpackJsonpvmui||[],c=a.push.bind(a);a.push=r,a=a.slice();for(var f=0;f<a.length;f++)r(a[f]);var l=c;t()}([]);

View File

@@ -2,19 +2,6 @@
Web UI for VictoriaMetrics
Features:
- configurable Server URL
- configurable time range - every variant have own resolution to show around 30 data points
- query editor has basic highlighting and can be multi-line
- chart is responsive by width
- color assignment for series is automatic
- legend with reduced naming
- tooltips for closest data point
- auto-refresh mode with several time interval presets
- table and raw JSON Query viewer
## Docker image build
Run the following command from the root of VictoriaMetrics repository in order to build `victoriametrics/vmui` Docker image:
@@ -65,4 +52,4 @@ Then run the built binary with the following command:
bin/victoria-metrics -selfScrapeInterval=5s
```
Then navigate to `http://localhost:8428/vmui/`
Then navigate to `http://localhost:8428/vmui/`. See [these docs](https://docs.victoriametrics.com/#vmui) for more details.

File diff suppressed because it is too large Load Diff

View File

@@ -5,26 +5,26 @@
"homepage": "./",
"dependencies": {
"@codemirror/autocomplete": "^0.19.9",
"@codemirror/basic-setup": "^0.19.0",
"@codemirror/commands": "^0.19.5",
"@codemirror/basic-setup": "^0.19.1",
"@codemirror/commands": "^0.19.6",
"@codemirror/highlight": "^0.19.6",
"@codemirror/state": "^0.19.6",
"@codemirror/view": "^0.19.21",
"@codemirror/view": "^0.19.29",
"@date-io/dayjs": "^2.11.0",
"@emotion/react": "^11.7.0",
"@emotion/react": "^11.7.1",
"@emotion/styled": "^11.6.0",
"@mui/icons-material": "^5.2.0",
"@mui/lab": "^5.0.0-alpha.58",
"@mui/material": "^5.2.2",
"@mui/styles": "^5.2.2",
"@testing-library/jest-dom": "^5.15.1",
"@mui/icons-material": "^5.2.4",
"@mui/lab": "^5.0.0-alpha.60",
"@mui/material": "^5.2.4",
"@mui/styles": "^5.2.3",
"@testing-library/jest-dom": "^5.16.1",
"@testing-library/react": "^12.1.2",
"@testing-library/user-event": "^13.5.0",
"@types/jest": "^27.0.3",
"@types/lodash.debounce": "^4.0.6",
"@types/lodash.get": "^4.4.6",
"@types/lodash.throttle": "^4.1.6",
"@types/node": "^16.11.10",
"@types/node": "^17.0.1",
"@types/numeral": "^2.0.2",
"@types/qs": "^6.9.7",
"@types/react": "^17.0.37",
@@ -36,13 +36,14 @@
"lodash.get": "^4.4.2",
"lodash.throttle": "^4.1.1",
"numeral": "^2.0.6",
"qs": "^6.10.1",
"qs": "^6.10.2",
"react": "^17.0.2",
"react-dom": "^17.0.2",
"react-draggable": "^4.4.4",
"react-measure": "^2.5.2",
"react-scripts": "4.0.3",
"typescript": "~4.5.2",
"uplot": "^1.6.17",
"react-scripts": "5.0.0",
"typescript": "~4.5.4",
"uplot": "^1.6.18",
"web-vitals": "^2.1.2"
},
"scripts": {
@@ -72,9 +73,9 @@
]
},
"devDependencies": {
"@babel/plugin-proposal-nullish-coalescing-operator": "^7.16.0",
"@typescript-eslint/eslint-plugin": "^5.4.0",
"@typescript-eslint/parser": "^5.4.0",
"@babel/plugin-proposal-nullish-coalescing-operator": "^7.16.5",
"@typescript-eslint/eslint-plugin": "^5.7.0",
"@typescript-eslint/parser": "^5.7.0",
"customize-cra": "^1.0.0",
"eslint-plugin-react": "^7.27.1",
"react-app-rewired": "^2.1.8"

View File

@@ -4,44 +4,15 @@ import HomeLayout from "./components/Home/HomeLayout";
import {StateProvider} from "./state/common/StateContext";
import {AuthStateProvider} from "./state/auth/AuthStateContext";
import {GraphStateProvider} from "./state/graph/GraphStateContext";
import { ThemeProvider, Theme, StyledEngineProvider, createTheme } from "@mui/material/styles";
import { ThemeProvider, StyledEngineProvider } from "@mui/material/styles";
import THEME from "./theme/theme";
import CssBaseline from "@mui/material/CssBaseline";
import LocalizationProvider from "@mui/lab/LocalizationProvider";
// pick a date util library
import DayjsUtils from "@date-io/dayjs";
declare module "@mui/styles/defaultTheme" {
// eslint-disable-next-line @typescript-eslint/no-empty-interface
interface DefaultTheme extends Theme {}
}
const App: FC = () => {
const THEME = createTheme({
palette: {
primary: {
main: "#3F51B5"
},
secondary: {
main: "#F50057"
}
},
components: {
MuiSwitch: {
defaultProps: {
color: "secondary"
}
}
},
typography: {
"fontSize": 10
}
});
return <>
<CssBaseline /> {/* CSS Baseline: kind of normalize.css made by materialUI team - can be scoped */}
<LocalizationProvider dateAdapter={DayjsUtils}> {/* Allows datepicker to work with DayJS */}

View File

@@ -1,4 +1,5 @@
export interface MetricBase {
group: number;
metric: {
[key: string]: string;
};

View File

@@ -26,8 +26,8 @@ import TabPanel from "./AuthTabPanel";
import PersonIcon from "@mui/icons-material/Person";
import LockIcon from "@mui/icons-material/Lock";
import makeStyles from "@mui/styles/makeStyles";
import {useAuthDispatch, useAuthState} from "../../../state/auth/AuthStateContext";
import {AUTH_METHOD, WithCheckbox} from "../../../state/auth/reducer";
import {useAuthDispatch, useAuthState} from "../../../../state/auth/AuthStateContext";
import {AUTH_METHOD, WithCheckbox} from "../../../../state/auth/reducer";
// TODO: make generic when creating second dialog
export interface DialogProps {

View File

@@ -0,0 +1,43 @@
import React, {FC, useCallback, useMemo} from "react";
import {Box, FormControlLabel, TextField} from "@mui/material";
import {useGraphDispatch, useGraphState} from "../../../../state/graph/GraphStateContext";
import debounce from "lodash.debounce";
import BasicSwitch from "../../../../theme/switch";
const AxesLimitsConfigurator: FC = () => {
const { yaxis } = useGraphState();
const graphDispatch = useGraphDispatch();
const axes = useMemo(() => Object.keys(yaxis.limits.range), [yaxis.limits.range]);
const onChangeYaxisLimits = () => { graphDispatch({type: "TOGGLE_ENABLE_YAXIS_LIMITS"}); };
const onChangeLimit = (e: React.ChangeEvent<HTMLInputElement | HTMLTextAreaElement>, axis: string, index: number) => {
const newLimits = yaxis.limits.range;
newLimits[axis][index] = +e.target.value;
if (newLimits[axis][0] === newLimits[axis][1] || newLimits[axis][0] > newLimits[axis][1]) return;
graphDispatch({type: "SET_YAXIS_LIMITS", payload: newLimits});
};
const debouncedOnChangeLimit = useCallback(debounce(onChangeLimit, 500), [yaxis.limits.range]);
return <Box display="grid" alignItems="center" gap={2}>
<FormControlLabel
control={<BasicSwitch checked={yaxis.limits.enable} onChange={onChangeYaxisLimits}/>}
label="Fix the limits for y-axis"
/>
<Box display="grid" alignItems="center" gap={2}>
{axes.map(axis => <Box display="grid" gridTemplateColumns="120px 120px" gap={1} key={axis}>
<TextField label={`Min ${axis}`} type="number" size="small" variant="outlined"
disabled={!yaxis.limits.enable}
defaultValue={yaxis.limits.range[axis][0]}
onChange={(e) => debouncedOnChangeLimit(e, axis, 0)}/>
<TextField label={`Max ${axis}`} type="number" size="small" variant="outlined"
disabled={!yaxis.limits.enable}
defaultValue={yaxis.limits.range[axis][1]}
onChange={(e) => debouncedOnChangeLimit(e, axis, 1)} />
</Box>)}
</Box>
</Box>;
};
export default AxesLimitsConfigurator;

View File

@@ -0,0 +1,64 @@
import SettingsIcon from "@mui/icons-material/Settings";
import React, {FC, useState, useRef} from "react";
import AxesLimitsConfigurator from "./AxesLimitsConfigurator";
import {Box, Button, IconButton, Paper, Typography} from "@mui/material";
import Draggable from "react-draggable";
import makeStyles from "@mui/styles/makeStyles";
import CloseIcon from "@mui/icons-material/Close";
const useStyles = makeStyles({
popover: {
position: "absolute",
display: "grid",
gridGap: "16px",
padding: "0 0 25px",
zIndex: 2,
},
popoverHeader: {
display: "flex",
alignItems: "center",
justifyContent: "space-between",
background: "#3F51B5",
padding: "6px 6px 6px 12px",
borderRadius: "4px 4px 0 0",
color: "#FFF",
cursor: "move",
},
popoverBody: {
display: "grid",
gridGap: "6px",
padding: "0 14px",
}
});
const GraphSettings: FC = () => {
const [open, setOpen] = useState(false);
const draggableRef = useRef<HTMLDivElement>(null);
const position = { x: 173, y: 0 };
const classes = useStyles();
return <Box display="flex" px={2}>
<Button onClick={() => setOpen((old) => !old)} variant="outlined">
<SettingsIcon sx={{fontSize: 16, marginRight: "4px"}}/>
<span style={{lineHeight: 1, paddingTop: "1px"}}>{open ? "Hide" : "Show"} graph settings</span>
</Button>
{open && (
<Draggable nodeRef={draggableRef} defaultPosition={position} handle="#handle">
<Paper elevation={3} className={classes.popover} ref={draggableRef}>
<div id="handle" className={classes.popoverHeader}>
<Typography variant="body1"><b>Graph Settings</b></Typography>
<IconButton size="small" onClick={() => setOpen(false)}>
<CloseIcon style={{color: "white"}}/>
</IconButton>
</div>
<Box className={classes.popoverBody}>
<AxesLimitsConfigurator/>
</Box>
</Paper>
</Draggable>
)}
</Box>;
};
export default GraphSettings;

View File

@@ -0,0 +1,40 @@
import React, {FC} from "react";
import {Box, FormControlLabel} from "@mui/material";
import {saveToStorage} from "../../../../utils/storage";
import {useAppDispatch, useAppState} from "../../../../state/common/StateContext";
import BasicSwitch from "../../../../theme/switch";
import StepConfigurator from "./StepConfigurator";
const AdditionalSettings: FC = () => {
const {queryControls: {autocomplete, nocache}} = useAppState();
const dispatch = useAppDispatch();
const onChangeAutocomplete = () => {
dispatch({type: "TOGGLE_AUTOCOMPLETE"});
saveToStorage("AUTOCOMPLETE", !autocomplete);
};
const onChangeCache = () => {
dispatch({type: "NO_CACHE"});
saveToStorage("NO_CACHE", !nocache);
};
return <Box display="flex" alignItems="center">
<Box>
<FormControlLabel label="Enable autocomplete"
control={<BasicSwitch checked={autocomplete} onChange={onChangeAutocomplete}/>}
/>
</Box>
<Box ml={2}>
<FormControlLabel label="Enable cache"
control={<BasicSwitch checked={!nocache} onChange={onChangeCache}/>}
/>
</Box>
<Box ml={2}>
<StepConfigurator/>
</Box>
</Box>;
};
export default AdditionalSettings;

View File

@@ -0,0 +1,137 @@
import React, {FC, useEffect, useRef, useState} from "react";
import {
Accordion, AccordionDetails, AccordionSummary, Box, Grid, IconButton, Typography, Tooltip, Button
} from "@mui/material";
import QueryEditor from "./QueryEditor";
import {TimeSelector} from "../Time/TimeSelector";
import {useAppDispatch, useAppState} from "../../../../state/common/StateContext";
import ExpandMoreIcon from "@mui/icons-material/ExpandMore";
import HighlightOffIcon from "@mui/icons-material/HighlightOff";
import AddIcon from "@mui/icons-material/Add";
import PlayCircleOutlineIcon from "@mui/icons-material/PlayCircleOutline";
import Portal from "@mui/material/Portal";
import ServerConfigurator from "./ServerConfigurator";
import AdditionalSettings from "./AdditionalSettings";
import {ErrorTypes} from "../../../../types";
export interface QueryConfiguratorProps {
error?: ErrorTypes | string;
}
const QueryConfigurator: FC<QueryConfiguratorProps> = ({error}) => {
const {serverUrl, query, queryHistory, time: {duration}, queryControls: {autocomplete}} = useAppState();
const dispatch = useAppDispatch();
const [expanded, setExpanded] = useState(true);
const queryContainer = useRef<HTMLDivElement>(null);
const queryRef = useRef(query);
useEffect(() => {
queryRef.current = query;
}, [query]);
const onSetDuration = (dur: string) => dispatch({type: "SET_DURATION", payload: dur});
const updateHistory = () => {
dispatch({
type: "SET_QUERY_HISTORY", payload: query.map((q, i) => {
const h = queryHistory[i] || {values: []};
const queryEqual = q === h.values[h.values.length - 1];
return {
index: h.values.length - Number(queryEqual),
values: !queryEqual && q ? [...h.values, q] : h.values
};
})
});
};
const onRunQuery = () => {
updateHistory();
dispatch({type: "SET_QUERY", payload: query});
dispatch({type: "RUN_QUERY"});
};
const onAddQuery = () => dispatch({type: "SET_QUERY", payload: [...queryRef.current, ""]});
const onRemoveQuery = (index: number) => {
const newQuery = [...queryRef.current];
newQuery.splice(index, 1);
dispatch({type: "SET_QUERY", payload: newQuery});
};
const onSetQuery = (value: string, index: number) => {
const newQuery = [...queryRef.current];
newQuery[index] = value;
dispatch({type: "SET_QUERY", payload: newQuery});
};
const setHistoryIndex = (step: number, indexQuery: number) => {
const {index, values} = queryHistory[indexQuery];
const newIndexHistory = index + step;
if (newIndexHistory < 0 || newIndexHistory >= values.length) return;
onSetQuery(values[newIndexHistory] || "", indexQuery);
dispatch({
type: "SET_QUERY_HISTORY_BY_INDEX",
payload: {value: {values, index: newIndexHistory}, queryNumber: indexQuery}
});
};
return <>
<Accordion expanded={expanded} onChange={() => setExpanded(prev => !prev)}>
<AccordionSummary
expandIcon={<IconButton><ExpandMoreIcon/></IconButton>}
aria-controls="panel1a-content"
id="panel1a-header"
sx={{alignItems: "flex-start", padding: "15px"}}
>
<Box mr={2}>
<Typography variant="h6" component="h2">Query Configuration</Typography>
</Box>
<Box flexGrow={1} onClick={e => e.stopPropagation()} onFocusCapture={e => e.stopPropagation()}>
<Portal disablePortal={!expanded} container={queryContainer.current}>
{query.map((q, i) =>
<Box key={i} display="grid" gridTemplateColumns="1fr auto" gap="4px" width="100%"
mb={i === query.length - 1 ? 0 : 2}>
<QueryEditor server={serverUrl} query={query[i]} index={i} oneLiner={!expanded}
autocomplete={autocomplete} queryHistory={queryHistory[i]} error={error}
setHistoryIndex={setHistoryIndex} runQuery={onRunQuery}
setQuery={onSetQuery}/>
{i === 0 && <Tooltip title="Execute Query">
<IconButton onClick={onRunQuery}>
<PlayCircleOutlineIcon/>
</IconButton>
</Tooltip>}
{i > 0 && <Tooltip title="Remove Query">
<IconButton onClick={() => onRemoveQuery(i)}>
<HighlightOffIcon/>
</IconButton>
</Tooltip>}
</Box>)}
</Portal>
</Box>
</AccordionSummary>
<AccordionDetails>
<Grid container columnSpacing={2}>
<Grid item xs={6} minWidth={400}>
<ServerConfigurator error={error}/>
{/* for portal QueryEditor */}
<div ref={queryContainer}/>
{query.length < 2 && <Box display="inline-block" minHeight="40px" mt={2}>
<Button onClick={onAddQuery} variant="outlined">
<AddIcon sx={{fontSize: 16, marginRight: "4px"}}/>
<span style={{lineHeight: 1, paddingTop: "1px"}}>Query</span>
</Button>
</Box>}
</Grid>
<Grid item xs>
<TimeSelector setDuration={onSetDuration} duration={duration}/>
</Grid>
<Grid item xs={12} pt={1}>
<AdditionalSettings/>
</Grid>
</Grid>
</AccordionDetails>
</Accordion>
</>;
};
export default QueryConfigurator;

View File

@@ -2,28 +2,41 @@ import {EditorState} from "@codemirror/state";
import {EditorView, keymap} from "@codemirror/view";
import {defaultKeymap} from "@codemirror/commands";
import React, {FC, useEffect, useRef, useState} from "react";
import { PromQLExtension } from "codemirror-promql";
import { basicSetup } from "@codemirror/basic-setup";
import {QueryHistory} from "../../../state/common/reducer";
import {PromQLExtension} from "codemirror-promql";
import {basicSetup} from "@codemirror/basic-setup";
import {QueryHistory} from "../../../../state/common/reducer";
import {ErrorTypes} from "../../../../types";
export interface QueryEditorProps {
setHistoryIndex: (step: number) => void;
setQuery: (query: string) => void;
runQuery: () => void;
query: string;
queryHistory: QueryHistory;
server: string;
oneLiner?: boolean;
autocomplete: boolean
setHistoryIndex: (step: number, index: number) => void;
setQuery: (query: string, index: number) => void;
runQuery: () => void;
query: string;
index: number;
queryHistory: QueryHistory;
server: string;
oneLiner?: boolean;
autocomplete: boolean;
error?: ErrorTypes | string;
}
const QueryEditor: FC<QueryEditorProps> = ({
query, queryHistory, setHistoryIndex, setQuery, runQuery, server, oneLiner = false, autocomplete
index,
query,
queryHistory,
setHistoryIndex,
setQuery,
runQuery,
server,
oneLiner = false,
autocomplete,
error
}) => {
const ref = useRef<HTMLDivElement>(null);
const [editorView, setEditorView] = useState<EditorView>();
const [focusEditor, setFocusEditor] = useState(false);
// init editor view on load
useEffect(() => {
@@ -41,11 +54,14 @@ const QueryEditor: FC<QueryEditorProps> = ({
useEffect(() => {
const promQL = new PromQLExtension();
promQL.activateCompletion(autocomplete);
promQL.setComplete({ remote: { url: server } });
promQL.setComplete({remote: {url: server}});
const listenerExtension = EditorView.updateListener.of(editorUpdate => {
if (editorUpdate.focusChanged) {
setFocusEditor(editorView?.hasFocus || false);
}
if (editorUpdate.docChanged) {
setQuery(editorUpdate.state.doc.toJSON().map(el => el.trim()).join(""));
setQuery(editorUpdate.state.doc.toJSON().map(el => el.trim()).join(""), index);
}
});
@@ -66,18 +82,20 @@ const QueryEditor: FC<QueryEditorProps> = ({
if (key === "Enter" && ctrlMetaKey) {
runQuery();
} else if (key === "ArrowUp" && ctrlMetaKey) {
setHistoryIndex(-1);
setHistoryIndex(-1, index);
} else if (key === "ArrowDown" && ctrlMetaKey) {
setHistoryIndex(1);
setHistoryIndex(1, index);
}
};
return (
<>
{/*Class one-line-scroll and other codemirror styles are declared in index.css*/}
<div ref={ref} className={oneLiner ? "one-line-scroll" : "multi-line-scroll"} onKeyUp={onKeyUp}/>
</>
);
return <div className={`query-editor-container
${focusEditor ? "query-editor-container_focus" : ""}
query-editor-container-${oneLiner ? "one-line" : "multi-line"}
${error === ErrorTypes.validQuery ? "query-editor-container_error" : ""}`}>
{/*Class one-line-scroll and other codemirror styles are declared in index.css*/}
<label className="query-editor-label">Query</label>
<div className="query-editor" ref={ref} onKeyUp={onKeyUp}/>
</div>;
};
export default QueryEditor;

View File

@@ -0,0 +1,48 @@
import React, {FC, useEffect, useState} from "react";
import {Box, TextField, Tooltip, IconButton} from "@mui/material";
import SecurityIcon from "@mui/icons-material/Security";
import {useAppDispatch, useAppState} from "../../../../state/common/StateContext";
import {AuthDialog} from "../Auth/AuthDialog";
import {ErrorTypes} from "../../../../types";
import {getAppModeEnable, getAppModeParams} from "../../../../utils/app-mode";
export interface ServerConfiguratorProps {
error?: ErrorTypes | string;
}
const ServerConfigurator: FC<ServerConfiguratorProps> = ({error}) => {
const appModeEnable = getAppModeEnable();
const {serverURL: appServerUrl} = getAppModeParams();
const {serverUrl} = useAppState();
const dispatch = useAppDispatch();
const onSetServer = ({target: {value}}: {target: {value: string}}) => {
dispatch({type: "SET_SERVER", payload: value});
};
const [dialogOpen, setDialogOpen] = useState(false);
useEffect(() => {
if (appModeEnable) dispatch({type: "SET_SERVER", payload: appServerUrl});
}, [appServerUrl]);
return <>
<Box display="grid" gridTemplateColumns="1fr auto" gap="4px" alignItems="center" width="100%" mb={2} minHeight={50}>
<TextField variant="outlined" fullWidth label="Server URL" value={serverUrl || ""} disabled={appModeEnable}
error={error === ErrorTypes.validServer || error === ErrorTypes.emptyServer}
inputProps={{style: {fontFamily: "Monospace"}}}
onChange={onSetServer}/>
<Box>
<Tooltip title="Request Auth Settings">
<IconButton onClick={() => setDialogOpen(true)}>
<SecurityIcon/>
</IconButton>
</Tooltip>
</Box>
</Box>
<AuthDialog open={dialogOpen} onClose={() => setDialogOpen(false)}/>
</>;
};
export default ServerConfigurator;

View File

@@ -0,0 +1,50 @@
import React, {FC, useCallback, useEffect, useState} from "react";
import {Box, FormControlLabel, TextField} from "@mui/material";
import BasicSwitch from "../../../../theme/switch";
import {useGraphDispatch, useGraphState} from "../../../../state/graph/GraphStateContext";
import {useAppState} from "../../../../state/common/StateContext";
import debounce from "lodash.debounce";
const StepConfigurator: FC = () => {
const {customStep} = useGraphState();
const graphDispatch = useGraphDispatch();
const [error, setError] = useState(false);
const {time: {period: {step}}} = useAppState();
const onChangeStep = (e: React.ChangeEvent<HTMLInputElement | HTMLTextAreaElement>) => {
const value = +e.target.value;
if (value > 0) {
graphDispatch({type: "SET_CUSTOM_STEP", payload: value});
setError(false);
} else {
setError(true);
}
};
const debouncedOnChangeStep = useCallback(debounce(onChangeStep, 500), [customStep.value]);
const onChangeEnableStep = () => {
setError(false);
graphDispatch({type: "TOGGLE_CUSTOM_STEP"});
};
useEffect(() => {
if (!customStep.enable) graphDispatch({type: "SET_CUSTOM_STEP", payload: step || 1});
}, [step]);
return <Box display="grid" gridTemplateColumns="auto 120px" alignItems="center">
<FormControlLabel
control={<BasicSwitch checked={customStep.enable} onChange={onChangeEnableStep}/>}
label="Override step value"
/>
{customStep.enable &&
<TextField label="Step value" type="number" size="small" variant="outlined"
defaultValue={customStep.value}
error={error}
helperText={error ? "step is out of allowed range" : " "}
onChange={debouncedOnChangeStep}/>
}
</Box>;
};
export default StepConfigurator;

View File

@@ -1,26 +1,32 @@
import {useEffect, useMemo, useState} from "react";
import {getQueryRangeUrl, getQueryUrl} from "../../../api/query-range";
import {useAppState} from "../../../state/common/StateContext";
import {InstantMetricResult, MetricResult} from "../../../api/types";
import {isValidHttpUrl} from "../../../utils/url";
import {useAuthState} from "../../../state/auth/AuthStateContext";
import {TimeParams} from "../../../types";
import {getQueryRangeUrl, getQueryUrl} from "../../../../api/query-range";
import {useAppState} from "../../../../state/common/StateContext";
import {InstantMetricResult, MetricBase, MetricResult} from "../../../../api/types";
import {isValidHttpUrl} from "../../../../utils/url";
import {useAuthState} from "../../../../state/auth/AuthStateContext";
import {ErrorTypes, TimeParams} from "../../../../types";
import {useGraphState} from "../../../../state/graph/GraphStateContext";
import {getAppModeEnable, getAppModeParams} from "../../../../utils/app-mode";
const appModeEnable = getAppModeEnable();
const {serverURL: appServerUrl} = getAppModeParams();
export const useFetchQuery = (): {
fetchUrl?: string,
fetchUrl?: string[],
isLoading: boolean,
graphData?: MetricResult[],
liveData?: InstantMetricResult[],
error?: string,
error?: ErrorTypes | string,
} => {
const {query, displayType, serverUrl, time: {period}, queryControls: {nocache}} = useAppState();
const {query, displayType, serverUrl, time: {period}, queryControls: {nocache, autoRefresh}} = useAppState();
const {basicData, bearerData, authMethod} = useAuthState();
const {customStep} = useGraphState();
const [isLoading, setIsLoading] = useState(false);
const [graphData, setGraphData] = useState<MetricResult[]>();
const [liveData, setLiveData] = useState<InstantMetricResult[]>();
const [error, setError] = useState<string>();
const [error, setError] = useState<ErrorTypes | string>();
const [prevPeriod, setPrevPeriod] = useState<TimeParams>();
useEffect(() => {
@@ -31,7 +37,7 @@ export const useFetchQuery = (): {
}, [error]);
const needUpdateData = useMemo(() => {
if (!prevPeriod) return true;
if (!prevPeriod || autoRefresh) return true;
const duration = (prevPeriod.end - prevPeriod.start) / 3;
const factorLimit = duration / (period.end - period.start) >= 0.7;
const maxLimit = period.end > (prevPeriod.end + duration);
@@ -40,7 +46,7 @@ export const useFetchQuery = (): {
}, [period]);
const fetchData = async () => {
if (!fetchUrl) return;
if (!fetchUrl?.length) return;
setIsLoading(true);
setPrevPeriod(period);
@@ -53,38 +59,49 @@ export const useFetchQuery = (): {
}
try {
const response = await fetch(fetchUrl, { headers });
if (response.ok) {
const responses = await Promise.all(fetchUrl.map(url => fetch(url, {headers})));
const tempData = [];
let counter = 1;
for await (const response of responses) {
const resp = await response.json();
setError(undefined);
displayType === "chart" ? setGraphData(resp.data.result) : setLiveData(resp.data.result);
} else {
setError((await response.json())?.error);
if (response.ok) {
setError(undefined);
tempData.push(...resp.data.result.map((d: MetricBase) => {
d.group = counter;
return d;
}));
counter++;
} else {
setError(`${resp.errorType}\r\n${resp?.error}`);
}
}
displayType === "chart" ? setGraphData(tempData) : setLiveData(tempData);
} catch (e) {
if (e instanceof Error) setError(e.message);
if (e instanceof Error) setError(`${e.name}: ${e.message}`);
}
setIsLoading(false);
};
const fetchUrl = useMemo(() => {
const server = appModeEnable ? appServerUrl : serverUrl;
if (!period) return;
if (!serverUrl) {
setError("Please enter Server URL");
} else if (!query.trim()) {
setError("Please enter a valid Query and execute it");
} else if (isValidHttpUrl(serverUrl)) {
if (!server) {
setError(ErrorTypes.emptyServer);
} else if (query.every(q => !q.trim())) {
setError(ErrorTypes.validQuery);
} else if (isValidHttpUrl(server)) {
const duration = (period.end - period.start) / 2;
const bufferPeriod = {...period, start: period.start - duration, end: period.end + duration};
return displayType === "chart"
? getQueryRangeUrl(serverUrl, query, bufferPeriod, nocache)
: getQueryUrl(serverUrl, query, period);
if (customStep.enable) bufferPeriod.step = customStep.value;
return query.filter(q => q.trim()).map(q => displayType === "chart"
? getQueryRangeUrl(server, q, bufferPeriod, nocache)
: getQueryUrl(server, q, period));
} else {
setError("Please provide a valid URL");
setError(ErrorTypes.validServer);
}
},
[serverUrl, period, displayType]);
[serverUrl, period, displayType, customStep]);
useEffect(() => {
setPrevPeriod(undefined);
@@ -94,7 +111,7 @@ export const useFetchQuery = (): {
// Doing it on each query change - looks to be a bad idea. Probably can be done on blur
useEffect(() => {
fetchData();
}, [serverUrl, displayType]);
}, [serverUrl, displayType, customStep]);
useEffect(() => {
if (needUpdateData) {

View File

@@ -1,148 +0,0 @@
import React, {FC, useRef, useState} from "react";
import { Accordion, AccordionDetails, AccordionSummary, Box, Grid, IconButton, TextField, Typography, FormControlLabel,
Tooltip, Switch } from "@mui/material";
import QueryEditor from "./QueryEditor";
import {TimeSelector} from "./TimeSelector";
import {useAppDispatch, useAppState} from "../../../state/common/StateContext";
import ExpandMoreIcon from "@mui/icons-material/ExpandMore";
import SecurityIcon from "@mui/icons-material/Security";
import {AuthDialog} from "./AuthDialog";
import PlayCircleOutlineIcon from "@mui/icons-material/PlayCircleOutline";
import Portal from "@mui/material/Portal";
import {saveToStorage} from "../../../utils/storage";
import {useGraphDispatch, useGraphState} from "../../../state/graph/GraphStateContext";
import debounce from "lodash.debounce";
const QueryConfigurator: FC = () => {
const {serverUrl, query, queryHistory, time: {duration}, queryControls: {autocomplete, nocache}} = useAppState();
const dispatch = useAppDispatch();
const onChangeAutocomplete = () => {
dispatch({type: "TOGGLE_AUTOCOMPLETE"});
saveToStorage("AUTOCOMPLETE", !autocomplete);
};
const onChangeCache = () => {
dispatch({type: "NO_CACHE"});
saveToStorage("NO_CACHE", !nocache);
};
const { yaxis } = useGraphState();
const graphDispatch = useGraphDispatch();
const onChangeYaxisLimits = () => { graphDispatch({type: "TOGGLE_ENABLE_YAXIS_LIMITS"}); };
const setMinLimit = ({target: {value}}: {target: {value: string}}) => {
graphDispatch({type: "SET_YAXIS_LIMITS", payload: [+value, yaxis.limits.range[1]]});
};
const setMaxLimit = ({target: {value}}: {target: {value: string}}) => {
graphDispatch({type: "SET_YAXIS_LIMITS", payload: [yaxis.limits.range[0], +value]});
};
const [dialogOpen, setDialogOpen] = useState(false);
const [expanded, setExpanded] = useState(true);
const queryContainer = useRef<HTMLDivElement>(null);
const onSetDuration = (dur: string) => dispatch({type: "SET_DURATION", payload: dur});
const onRunQuery = () => {
const { values } = queryHistory;
dispatch({type: "RUN_QUERY"});
if (query === values[values.length - 1]) return;
dispatch({type: "SET_QUERY_HISTORY_INDEX", payload: values.length});
dispatch({type: "SET_QUERY_HISTORY_VALUES", payload: [...values, query]});
};
const onSetQuery = (newQuery: string) => {
if (query === newQuery) return;
dispatch({type: "SET_QUERY", payload: newQuery});
};
const setHistoryIndex = (step: number) => {
const index = queryHistory.index + step;
if (index < -1 || index > queryHistory.values.length) return;
dispatch({type: "SET_QUERY_HISTORY_INDEX", payload: index});
onSetQuery(queryHistory.values[index] || "");
};
const onSetServer = ({target: {value}}: {target: {value: string}}) => {
dispatch({type: "SET_SERVER", payload: value});
};
return <>
<Accordion expanded={expanded} onChange={() => setExpanded(prev => !prev)}>
<AccordionSummary
expandIcon={<ExpandMoreIcon/>}
aria-controls="panel1a-content"
id="panel1a-header"
>
<Box display="flex" alignItems="center" mr={2}><Typography variant="h6" component="h2">Query Configuration</Typography></Box>
<Box flexGrow={1} onClick={e => e.stopPropagation()} onFocusCapture={e => e.stopPropagation()}>
<Portal disablePortal={!expanded} container={queryContainer.current}>
<Box display="flex" alignItems="center">
<Box width="100%">
<QueryEditor server={serverUrl} query={query} oneLiner={!expanded} autocomplete={autocomplete}
queryHistory={queryHistory} setHistoryIndex={setHistoryIndex} runQuery={onRunQuery} setQuery={onSetQuery}/>
</Box>
<Tooltip title="Execute Query">
<IconButton onClick={onRunQuery} size="large"><PlayCircleOutlineIcon /></IconButton>
</Tooltip>
</Box>
</Portal>
</Box>
</AccordionSummary>
<AccordionDetails>
<Grid container spacing={2}>
<Grid item xs={12} md={6}>
<Box display="grid" gap={2} gridTemplateRows="auto 1fr">
<Box display="flex" alignItems="center">
<TextField variant="outlined" fullWidth label="Server URL" value={serverUrl}
inputProps={{style: {fontFamily: "Monospace"}}}
onChange={onSetServer}/>
<Box>
<Tooltip title="Request Auth Settings">
<IconButton onClick={() => setDialogOpen(true)} size="large"><SecurityIcon/></IconButton>
</Tooltip>
</Box>
</Box>
<Box flexGrow={1} ><div ref={queryContainer} />{/* for portal QueryEditor */}</Box>
</Box>
</Grid>
<Grid item xs={8} md={6} >
<Box style={{
minHeight: "128px",
padding: "10px 0",
borderRadius: "4px",
borderColor: "#b9b9b9",
borderStyle: "solid",
borderWidth: "1px"}}>
<TimeSelector setDuration={onSetDuration} duration={duration}/>
</Box>
</Grid>
<Grid item xs={12}>
<Box px={1} display="flex" alignItems="center" minHeight={52}>
<Box><FormControlLabel label="Enable autocomplete"
control={<Switch size="small" checked={autocomplete} onChange={onChangeAutocomplete}/>}
/></Box>
<Box ml={2}><FormControlLabel label="Enable cache"
control={<Switch size="small" checked={!nocache} onChange={onChangeCache}/>}
/></Box>
<Box ml={2} display="flex" alignItems="center">
<FormControlLabel
control={<Switch size="small" checked={yaxis.limits.enable} onChange={onChangeYaxisLimits}/>}
label="Fix the limits for y-axis"
/>
{yaxis.limits.enable && <Box display="grid" gridTemplateColumns="120px 120px" gap={1}>
<TextField label="Min" type="number" size="small" variant="outlined"
defaultValue={yaxis.limits.range[0]} onChange={debounce(setMinLimit, 750)}/>
<TextField label="Max" type="number" size="small" variant="outlined"
defaultValue={yaxis.limits.range[1]} onChange={debounce(setMaxLimit, 750)}/>
</Box>}
</Box>
</Box>
</Grid>
</Grid>
</AccordionDetails>
</Accordion>
<AuthDialog open={dialogOpen} onClose={() => setDialogOpen(false)}/>
</>;
};
export default QueryConfigurator;

View File

@@ -1,9 +1,10 @@
import React, {FC, useEffect, useState} from "react";
import {Box, FormControlLabel, IconButton, Switch, Tooltip} from "@mui/material";
import {Box, FormControlLabel, IconButton, Tooltip} from "@mui/material";
import EqualizerIcon from "@mui/icons-material/Equalizer";
import {useAppDispatch, useAppState} from "../../../state/common/StateContext";
import CircularProgressWithLabel from "../../common/CircularProgressWithLabel";
import {useAppDispatch, useAppState} from "../../../../state/common/StateContext";
import CircularProgressWithLabel from "../../../common/CircularProgressWithLabel";
import makeStyles from "@mui/styles/makeStyles";
import BasicSwitch from "../../../../theme/switch";
const useStyles = makeStyles({
colorizing: {
@@ -69,7 +70,7 @@ export const ExecutionControls: FC = () => {
return <Box display="flex" alignItems="center">
{<FormControlLabel
control={<Switch size="small" className={classes.colorizing} checked={autoRefresh} onChange={handleChange} />}
control={<BasicSwitch className={classes.colorizing} checked={autoRefresh} onChange={handleChange} />}
label="Auto-refresh"
/>}
@@ -78,7 +79,9 @@ export const ExecutionControls: FC = () => {
onClick={() => {iterateDelays();}} />
<Tooltip title="Change delay refresh">
<Box ml={1}>
<IconButton onClick={() => {iterateDelays();}} size="large"><EqualizerIcon style={{color: "white"}} /></IconButton>
<IconButton onClick={() => {iterateDelays();}}>
<EqualizerIcon style={{color: "white"}} />
</IconButton>
</Box>
</Tooltip>
</>}

View File

@@ -1,6 +1,6 @@
import React, {FC} from "react";
import {Paper, Table, TableBody, TableCell, TableContainer, TableHead, TableRow} from "@mui/material";
import {supportedDurations} from "../../../utils/time";
import {supportedDurations} from "../../../../utils/time";
export const TimeDurationPopover: FC = () => {

View File

@@ -2,17 +2,33 @@ import React, {FC, useEffect, useState} from "react";
import {Box, Popover, TextField, Typography} from "@mui/material";
import DateTimePicker from "@mui/lab/DateTimePicker";
import {TimeDurationPopover} from "./TimeDurationPopover";
import {useAppDispatch, useAppState} from "../../../state/common/StateContext";
import {checkDurationLimit, dateFromSeconds, formatDateForNativeInput} from "../../../utils/time";
import {InlineBtn} from "../../common/InlineBtn";
import {useAppDispatch, useAppState} from "../../../../state/common/StateContext";
import {checkDurationLimit, dateFromSeconds, formatDateForNativeInput} from "../../../../utils/time";
import {InlineBtn} from "../../../common/InlineBtn";
import makeStyles from "@mui/styles/makeStyles";
interface TimeSelectorProps {
setDuration: (str: string) => void;
duration: string;
}
const useStyles = makeStyles({
container: {
display: "grid",
gridTemplateColumns: "auto auto",
height: "100%",
padding: "18px 14px",
borderRadius: "4px",
borderColor: "#b9b9b9",
borderStyle: "solid",
borderWidth: "1px"
}
});
export const TimeSelector: FC<TimeSelectorProps> = ({setDuration}) => {
const classes = useStyles();
const [durationStringFocused, setFocused] = useState(false);
const [anchorEl, setAnchorEl] = React.useState<Element | null>(null);
const [until, setUntil] = useState<string>();
@@ -60,7 +76,7 @@ export const TimeSelector: FC<TimeSelectorProps> = ({setDuration}) => {
const open = Boolean(anchorEl);
return <Box m={1} flexDirection="row" display="flex">
return <Box className={classes.container}>
{/*setup duration*/}
<Box px={1}>
<Box>
@@ -72,7 +88,7 @@ export const TimeSelector: FC<TimeSelectorProps> = ({setDuration}) => {
onFocus={() => {setFocused(true);}}
/>
</Box>
<Box my={2}>
<Box mt={2}>
<Typography variant="body2">
<span aria-owns={open ? "mouse-over-popover" : undefined}
aria-haspopup="true"
@@ -119,7 +135,7 @@ export const TimeSelector: FC<TimeSelectorProps> = ({setDuration}) => {
/>
</Box>
<Box my={2}>
<Box mt={2}>
<Typography variant="body2">
Will be changed to current time for auto-refresh mode.&nbsp;
<InlineBtn handler={() => dispatch({type: "RUN_QUERY_TO_NOW"})} text="Switch to now"/>

View File

@@ -1,23 +1,22 @@
import React, {FC} from "react";
import {Alert, AppBar, Box, CircularProgress, Fade, Link, Toolbar, Typography} from "@mui/material";
import {ExecutionControls} from "./Configurator/ExecutionControls";
import {ExecutionControls} from "./Configurator/Time/ExecutionControls";
import {DisplayTypeSwitch} from "./Configurator/DisplayTypeSwitch";
import GraphView from "./Views/GraphView";
import TableView from "./Views/TableView";
import {useAppState} from "../../state/common/StateContext";
import QueryConfigurator from "./Configurator/QueryConfigurator";
import {useFetchQuery} from "./Configurator/useFetchQuery";
import QueryConfigurator from "./Configurator/Query/QueryConfigurator";
import {useFetchQuery} from "./Configurator/Query/useFetchQuery";
import JsonView from "./Views/JsonView";
import {UrlCopy} from "./UrlCopy";
const HomeLayout: FC = () => {
const {displayType, time: {period}} = useAppState();
const {fetchUrl, isLoading, liveData, graphData, error} = useFetchQuery();
const {isLoading, liveData, graphData, error} = useFetchQuery();
return (
<>
<Box id="homeLayout">
<AppBar position="static">
<Toolbar>
<Box display="flex">
@@ -46,12 +45,11 @@ const HomeLayout: FC = () => {
<ExecutionControls/>
</Box>
<DisplayTypeSwitch/>
<UrlCopy url={fetchUrl}/>
</Toolbar>
</AppBar>
<Box p={2} display="grid" gridTemplateRows="auto 1fr" gap={"20px"} style={{minHeight: "calc(100vh - 64px)"}}>
<Box p={4} display="grid" gridTemplateRows="auto 1fr" gap={"20px"} style={{minHeight: "calc(100vh - 64px)"}}>
<Box>
<QueryConfigurator/>
<QueryConfigurator error={error}/>
</Box>
<Box height={"100%"}>
{isLoading && <Fade in={isLoading} style={{
@@ -68,9 +66,9 @@ const HomeLayout: FC = () => {
<CircularProgress/>
</Box>
</Fade>}
{<Box height={"100%"} p={3} bgcolor={"#fff"}>
{<Box height={"100%"} bgcolor={"#fff"}>
{error &&
<Alert color="error" severity="error" style={{fontSize: "14px"}}>
<Alert color="error" severity="error" style={{fontSize: "14px", whiteSpace: "pre-wrap"}}>
{error}
</Alert>}
{graphData && period && (displayType === "chart") &&
@@ -80,7 +78,7 @@ const HomeLayout: FC = () => {
</Box>}
</Box>
</Box>
</>
</Box>
);
};

View File

@@ -1,57 +1,62 @@
import React, {FC, useEffect, useState} from "react";
import React, {FC, useEffect, useMemo, useState} from "react";
import {MetricResult} from "../../../api/types";
import LineChart from "../../LineChart/LineChart";
import {AlignedData as uPlotData, Series as uPlotSeries} from "uplot";
import {Legend, LegendItem} from "../../Legend/Legend";
import Legend from "../../Legend/Legend";
import {useGraphDispatch, useGraphState} from "../../../state/graph/GraphStateContext";
import {getHideSeries, getLegendItem, getLimitsYAxis, getSeriesItem, getTimeSeries} from "../../../utils/uPlot";
import {getHideSeries, getLegendItem, getSeriesItem} from "../../../utils/uplot/series";
import {getLimitsYAxis, getTimeSeries} from "../../../utils/uplot/axes";
import {LegendItem} from "../../../utils/uplot/types";
import GraphSettings from "../Configurator/Graph/GraphSettings";
import {useAppState} from "../../../state/common/StateContext";
export interface GraphViewProps {
data?: MetricResult[];
}
const GraphView: FC<GraphViewProps> = ({data = []}) => {
const { yaxis } = useGraphState();
const graphDispatch = useGraphDispatch();
const {time: {period}} = useAppState();
const { customStep } = useGraphState();
const currentStep = useMemo(() => customStep.enable ? customStep.value : period.step || 1, [period.step, customStep]);
const [dataChart, setDataChart] = useState<uPlotData>([[]]);
const [series, setSeries] = useState<uPlotSeries[]>([]);
const [legend, setLegend] = useState<LegendItem[]>([]);
const [hideSeries, setHideSeries] = useState<string[]>([]);
const [valuesLimit, setValuesLimit] = useState<[number, number]>([0, 1]);
const setLimitsYaxis = (values: number[]) => {
if (!yaxis.limits.enable || (yaxis.limits.range.every(item => !item))) {
const limits = getLimitsYAxis(values);
setValuesLimit(limits);
graphDispatch({type: "SET_YAXIS_LIMITS", payload: limits});
}
const setLimitsYaxis = (values: {[key: string]: number[]}) => {
const limits = getLimitsYAxis(values);
graphDispatch({type: "SET_YAXIS_LIMITS", payload: limits});
};
const onChangeLegend = (label: string, metaKey: boolean) => {
setHideSeries(getHideSeries({hideSeries, label, metaKey, series}));
const onChangeLegend = (legend: LegendItem, metaKey: boolean) => {
setHideSeries(getHideSeries({hideSeries, legend, metaKey, series}));
};
useEffect(() => {
const tempTimes: number[] = [];
const tempValues: number[] = [];
const tempValues: {[key: string]: number[]} = {};
const tempLegend: LegendItem[] = [];
const tempSeries: uPlotSeries[] = [];
data?.forEach(d => {
data?.forEach((d) => {
const seriesItem = getSeriesItem(d, hideSeries);
tempSeries.push(seriesItem);
tempLegend.push(getLegendItem(seriesItem));
tempLegend.push(getLegendItem(seriesItem, d.group));
d.values.forEach(v => {
tempTimes.push(v[0]);
tempValues.push(+v[1]);
tempValues[d.group] ? tempValues[d.group].push(+v[1]) : tempValues[d.group] = [+v[1]];
});
});
const timeSeries = getTimeSeries(tempTimes);
const timeSeries = getTimeSeries(tempTimes, currentStep, period);
setDataChart([timeSeries, ...data.map(d => {
return new Array(timeSeries.length).fill(1).map((v, i) => d.values[i] ? +d.values[i][1] : null);
return timeSeries.map(t => {
const value = d.values.find(v => v[0] === t);
return value ? +value[1] : null;
});
})] as uPlotData);
setLimitsYaxis(tempValues);
@@ -68,7 +73,7 @@ const GraphView: FC<GraphViewProps> = ({data = []}) => {
data?.forEach(d => {
const seriesItem = getSeriesItem(d, hideSeries);
tempSeries.push(seriesItem);
tempLegend.push(getLegendItem(seriesItem));
tempLegend.push(getLegendItem(seriesItem, d.group));
});
setSeries([{}, ...tempSeries]);
setLegend(tempLegend);
@@ -77,7 +82,8 @@ const GraphView: FC<GraphViewProps> = ({data = []}) => {
return <>
{(data.length > 0)
? <div>
<LineChart data={dataChart} series={series} metrics={data} limits={valuesLimit}/>
<GraphSettings/>
<LineChart data={dataChart} series={series} metrics={data}/>
<Legend labels={legend} onChange={onChangeLegend}/>
</div>
: <div style={{textAlign: "center"}}>No data to show</div>}

View File

@@ -1,31 +1,48 @@
import React, {FC} from "react";
import React, {FC, useMemo} from "react";
import {hexToRGB} from "../../utils/color";
import {useAppState} from "../../state/common/StateContext";
import {LegendItem} from "../../utils/uplot/types";
import "./legend.css";
export interface LegendItem {
label: string;
color: string;
checked: boolean;
}
import {getDashLine} from "../../utils/uplot/helpers";
export interface LegendProps {
labels: LegendItem[];
onChange: (legend: string, metaKey: boolean) => void;
onChange: (item: LegendItem, metaKey: boolean) => void;
}
export const Legend: FC<LegendProps> = ({labels, onChange}) => {
const Legend: FC<LegendProps> = ({labels, onChange}) => {
const {query} = useAppState();
const groups = useMemo(() => {
return Array.from(new Set(labels.map(l => l.group)));
}, [labels]);
return <div className="legendWrapper">
{labels.map((legendItem: LegendItem) =>
<div className={legendItem.checked ? "legendItem" : "legendItem legendItemHide"}
key={legendItem.label}
onClick={(e) => onChange(legendItem.label, e.ctrlKey || e.metaKey)}>
<div className="legendMarker"
style={{
borderColor: legendItem.color,
backgroundColor: `rgba(${hexToRGB(legendItem.color)}, 0.1)`
}}/>
<div className="legendLabel">{legendItem.checked} {legendItem.label}</div>
{groups.map((group) => <div className="legendGroup" key={group}>
<div className="legendGroupTitle">
<svg className="legendGroupLine" width="33" height="3" version="1.1" xmlns="http://www.w3.org/2000/svg">
<line strokeWidth="3" x1="0" y1="0" x2="33" y2="0" stroke="#363636"
strokeDasharray={getDashLine(group).join(",")}
/>
</svg>
<b>&quot;{query[group - 1]}&quot;</b>:
</div>
)}
<div>
{labels.filter(l => l.group === group).map((legendItem: LegendItem) =>
<div className={legendItem.checked ? "legendItem" : "legendItem legendItemHide"}
key={`${legendItem.group}.${legendItem.label}`}
onClick={(e) => onChange(legendItem, e.ctrlKey || e.metaKey)}>
<div className="legendMarker"
style={{
borderColor: legendItem.color,
backgroundColor: `rgba(${hexToRGB(legendItem.color)}, 0.1)`
}}/>
<div className="legendLabel">{legendItem.label}</div>
</div>
)}
</div>
</div>)}
</div>;
};
};
export default Legend;

View File

@@ -1,12 +1,31 @@
.legendWrapper {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(400px, 1fr));
grid-gap: 20px;
margin-top: 20px;
cursor: default;
}
.legendGroup {
margin-bottom: 24px;
}
.legendGroupTitle {
display: flex;
align-items: center;
padding: 10px 0 5px;
font-size: 11px;
}
.legendGroupLine {
margin: 0 10px;
}
.legendItem {
display: inline-grid;
grid-template-columns: auto auto;
grid-gap: 4px;
align-items: center;
grid-gap: 6px;
align-items: start;
justify-content: start;
padding: 5px 10px;
background-color: #FFF;
@@ -30,9 +49,10 @@
border-style: solid;
box-sizing: border-box;
transition: 0.2s ease;
margin: 3px 0;
}
.legendLabel {
font-size: 12px;
font-weight: 600;
font-size: 11px;
font-weight: normal;
}

View File

@@ -1,45 +1,45 @@
import React, {FC, useCallback, useEffect, useRef, useState} from "react";
import {useAppDispatch, useAppState} from "../../state/common/StateContext";
import uPlot, {AlignedData as uPlotData, Options as uPlotOptions, Series as uPlotSeries, Range} from "uplot";
import uPlot, {AlignedData as uPlotData, Options as uPlotOptions, Series as uPlotSeries, Range, Scales, Scale} from "uplot";
import {useGraphState} from "../../state/graph/GraphStateContext";
import {defaultOptions, dragChart, setTooltip} from "../../utils/uPlot";
import {defaultOptions} from "../../utils/uplot/helpers";
import {dragChart} from "../../utils/uplot/events";
import {getAxes} from "../../utils/uplot/axes";
import {setTooltip} from "../../utils/uplot/tooltip";
import {MetricResult} from "../../api/types";
import {limitsDurations} from "../../utils/time";
import throttle from "lodash.throttle";
import "uplot/dist/uPlot.min.css";
import "./tooltip.css";
import useResize from "../../hooks/useResize";
export interface LineChartProps {
metrics: MetricResult[]
data: uPlotData;
series: uPlotSeries[],
limits: [number, number]
metrics: MetricResult[];
data: uPlotData;
series: uPlotSeries[];
}
enum typeChartUpdate {xRange = "xRange", yRange = "yRange", data = "data"}
enum typeChartUpdate { xRange = "xRange", yRange = "yRange", data = "data" }
const LineChart: FC<LineChartProps> = ({data, series, metrics = [], limits}) => {
const LineChart: FC<LineChartProps> = ({data, series, metrics = []}) => {
const dispatch = useAppDispatch();
const {time: {period}} = useAppState();
const { yaxis } = useGraphState();
const containerRef = useRef<HTMLDivElement>(null);
const {yaxis} = useGraphState();
const uPlotRef = useRef<HTMLDivElement>(null);
const [isPanning, setPanning] = useState(false);
const [zoomPos, setZoomPos] = useState(0);
const [xRange, setXRange] = useState({ min: period.start, max: period.end });
const [xRange, setXRange] = useState({min: period.start, max: period.end});
const [uPlotInst, setUPlotInst] = useState<uPlot>();
const layoutSize = useResize(document.getElementById("homeLayout"));
const tooltip = document.createElement("div");
tooltip.className = "u-tooltip";
const tooltipIdx = { seriesIdx: 1, dataIdx: 0 };
const tooltipOffset = { left: 0, top: 0 };
const tooltipIdx: {seriesIdx: number | null, dataIdx: number | undefined} = {seriesIdx: null, dataIdx: undefined};
const tooltipOffset = {left: 0, top: 0};
const setScale = ({min, max}: {min: number, max: number}): void => {
const setScale = ({min, max}: { min: number, max: number }): void => {
dispatch({type: "SET_PERIOD", payload: {from: new Date(min * 1000), to: new Date(max * 1000)}});
};
const throttledSetScale = useCallback(throttle(setScale, 500), []);
const setPlotScale = ({u, min, max}: {u: uPlot, min: number, max: number}) => {
const setPlotScale = ({u, min, max}: { u: uPlot, min: number, max: number }) => {
const delta = (max - min) * 1000;
if ((delta < limitsDurations.min) || (delta > limitsDurations.max)) return;
u.setScale("x", {min, max});
@@ -52,22 +52,18 @@ const LineChart: FC<LineChartProps> = ({data, series, metrics = [], limits}) =>
tooltipOffset.left = parseFloat(u.over.style.left);
tooltipOffset.top = parseFloat(u.over.style.top);
u.root.querySelector(".u-wrap")?.appendChild(tooltip);
// wheel drag pan
u.over.addEventListener("mousedown", e => {
dragChart({u, e, setPanning, setPlotScale, factor});
});
u.over.addEventListener("mousedown", e => dragChart({u, e, setPanning, setPlotScale, factor}));
// wheel scroll zoom
u.over.addEventListener("wheel", e => {
if (!e.ctrlKey && !e.metaKey) return;
e.preventDefault();
const {width} = u.over.getBoundingClientRect();
if (u.cursor.left && u.cursor.left > 0) setZoomPos(u.cursor.left);
const zoomPos = u.cursor.left && u.cursor.left > 0 ? u.cursor.left : 0;
const xVal = u.posToVal(zoomPos, "x");
const oxRange = (u.scales.x.max || 0) - (u.scales.x.min || 0);
const nxRange = e.deltaY < 0 ? oxRange * factor : oxRange / factor;
const min = xVal - (zoomPos/width) * nxRange;
const min = xVal - (zoomPos / width) * nxRange;
const max = min + nxRange;
u.batch(() => setPlotScale({u, min, max}));
});
@@ -76,37 +72,39 @@ const LineChart: FC<LineChartProps> = ({data, series, metrics = [], limits}) =>
const setCursor = (u: uPlot) => {
if (tooltipIdx.dataIdx === u.cursor.idx) return;
tooltipIdx.dataIdx = u.cursor.idx || 0;
if (tooltipIdx.seriesIdx && tooltipIdx.dataIdx) {
if (tooltipIdx.seriesIdx !== null && tooltipIdx.dataIdx !== undefined) {
setTooltip({u, tooltipIdx, metrics, series, tooltip, tooltipOffset});
}
};
const seriesFocus = (u: uPlot, sidx: (number | null)) => {
if (tooltipIdx.seriesIdx === sidx) return;
tooltipIdx.seriesIdx = sidx || 0;
sidx && tooltipIdx.dataIdx
tooltipIdx.seriesIdx = sidx;
sidx && tooltipIdx.dataIdx !== undefined
? setTooltip({u, tooltipIdx, metrics, series, tooltip, tooltipOffset})
: tooltip.style.display = "none";
};
const getRangeY = (u: uPlot, min = 0, max = 1): Range.MinMax => {
if (yaxis.limits.enable) return yaxis.limits.range;
return min && max ? [min - (min * 0.05), max + (max * 0.05)] : limits;
const getRangeX = (): Range.MinMax => [xRange.min, xRange.max];
const getRangeY = (u: uPlot, min = 0, max = 1, axis: string): Range.MinMax => {
if (yaxis.limits.enable) return yaxis.limits.range[axis];
return min && max ? [min - (min * 0.25), max + (max * 0.25)] : [-1, 1];
};
const getRangeX = (): Range.MinMax => {
return [xRange.min, xRange.max];
const getScales = (): Scales => {
const scales: { [key: string]: { range: Scale.Range } } = {x: {range: getRangeX}};
Object.keys(yaxis.limits.range).forEach(axis => {
scales[axis] = {range: (u: uPlot, min = 0, max = 1) => getRangeY(u, min, max, axis)};
});
return scales;
};
const options: uPlotOptions = {
...defaultOptions,
width: containerRef.current ? containerRef.current.offsetWidth : 400,
series,
plugins: [{ hooks: { ready: onReadyChart, setCursor, setSeries: seriesFocus }}],
scales: {
x: { range: getRangeX },
y: { range: getRangeY }
}
axes: getAxes(series),
scales: {...getScales()},
width: layoutSize.width ? layoutSize.width - 64 : 400,
plugins: [{hooks: {ready: onReadyChart, setCursor, setSeries: seriesFocus}}],
};
const updateChart = (type: typeChartUpdate): void => {
@@ -116,7 +114,10 @@ const LineChart: FC<LineChartProps> = ({data, series, metrics = [], limits}) =>
uPlotInst.scales.x.range = getRangeX;
break;
case typeChartUpdate.yRange:
uPlotInst.scales.y.range = getRangeY;
Object.keys(yaxis.limits.range).forEach(axis => {
if (!uPlotInst.scales[axis]) return;
uPlotInst.scales[axis].range = (u: uPlot, min = 0, max = 1) => getRangeY(u, min, max, axis);
});
break;
case typeChartUpdate.data:
uPlotInst.setData(data);
@@ -125,21 +126,21 @@ const LineChart: FC<LineChartProps> = ({data, series, metrics = [], limits}) =>
uPlotInst.redraw();
};
useEffect(() => setXRange({ min: period.start, max: period.end }), [period]);
useEffect(() => setXRange({min: period.start, max: period.end}), [period]);
useEffect(() => {
if (!uPlotRef.current) return;
const u = new uPlot(options, data, uPlotRef.current);
setUPlotInst(u);
setXRange({ min: period.start, max: period.end });
setXRange({min: period.start, max: period.end});
return u.destroy;
}, [uPlotRef.current, series]);
}, [uPlotRef.current, series, layoutSize]);
useEffect(() => updateChart(typeChartUpdate.data), [data]);
useEffect(() => updateChart(typeChartUpdate.xRange), [xRange]);
useEffect(() => updateChart(typeChartUpdate.yRange), [yaxis]);
return <div ref={containerRef} style={{pointerEvents: isPanning ? "none" : "auto", height: "500px"}}>
return <div style={{pointerEvents: isPanning ? "none" : "auto", height: "500px"}}>
<div ref={uPlotRef}/>
</div>;
};

View File

@@ -0,0 +1,23 @@
import { useState, useEffect } from "react";
const useResize = (node: HTMLElement | null): {width: number, height: number} => {
const [windowSize, setWindowSize] = useState({
width: 0,
height: 0,
});
useEffect(() => {
if (!node) return;
const handleResize = () => {
setWindowSize({
width: node.offsetWidth,
height: node.offsetHeight,
});
};
window.addEventListener("resize", handleResize);
handleResize();
return () => window.removeEventListener("resize", handleResize);
}, []);
return windowSize;
};
export default useResize;

View File

@@ -7,43 +7,89 @@ body {
}
code {
font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New',
monospace;
font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New', monospace;
}
/*Material UI global classes*/
.MuiAccordionSummary-content {
margin: 10px 0 !important;
margin: 0 !important;
}
/*Codemirror classes*/
/* TODO: find better way to override codemirror styles */
.cm-activeLine {
background-color: inherit !important;
}
.cm-editor {
border: none;
border-radius: 4px;
border-color: #b9b9b9;
border-style: solid;
border-width: 1px;
font-size: 10px;
}
.one-line-scroll .cm-editor {
height: 24px;
}
.cm-gutters {
border-radius: 4px 0 0 4px;
height: 100%;
overflow: hidden;
background-color: #FFFFFF !important;
border: none !important;
}
.multi-line-scroll .cm-content,
.multi-line-scroll .cm-gutters {
min-height: 64px !important;
.cm-activeLineGutter {
background-color: #FFFFFF !important;
}
.one-line-scroll .cm-content,
.one-line-scroll .cm-gutters {
min-height: auto;
.query-editor .cm-scroller {
align-items: center !important;
}
.query-editor .cm-editor.cm-focused {
outline: none;
}
.query-editor-container {
position: relative;
padding: 12px;
border: 1px solid #b9b9b9;
border-radius: 4px;
}
.query-editor-container_focus {
border: 1px solid #3F51B5;
}
.query-editor-container_error {
border-color: #FF4141;
}
.query-editor-container-one-line .query-editor .cm-editor {
height: 22px;
}
.query-editor-container-one-line {
padding: 6px;
}
.query-editor-label {
font-weight: 400;
font-size: 12px;
line-height: 1;
letter-spacing: normal;
color: rgba(0, 0, 0, 0.6);
padding: 0 5px;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
max-width: calc(133% - 24px);
position: absolute;
left: 4px;
top: -0.71875em;
z-index: 1;
background-color: #FFFFFF;
transform: scale(0.75);
}
.query-editor-container_error .query-editor-label {
color: #FF4141;
}

View File

@@ -4,7 +4,7 @@ import {TimeParams, TimePeriod} from "../../types";
import {dateFromSeconds, formatDateToLocal, getDateNowUTC, getDurationFromPeriod, getTimeperiodForDuration} from "../../utils/time";
import {getFromStorage} from "../../utils/storage";
import {getDefaultServer} from "../../utils/default-server-url";
import {getQueryStringValue} from "../../utils/query-string";
import {getQueryArray, getQueryStringValue} from "../../utils/query-string";
export interface TimeState {
duration: string;
@@ -19,9 +19,9 @@ export interface QueryHistory {
export interface AppState {
serverUrl: string;
displayType: DisplayType;
query: string;
query: string[];
time: TimeState;
queryHistory: QueryHistory,
queryHistory: QueryHistory[],
queryControls: {
autoRefresh: boolean;
autocomplete: boolean,
@@ -32,9 +32,9 @@ export interface AppState {
export type Action =
| { type: "SET_DISPLAY_TYPE", payload: DisplayType }
| { type: "SET_SERVER", payload: string }
| { type: "SET_QUERY", payload: string }
| { type: "SET_QUERY_HISTORY_INDEX", payload: number }
| { type: "SET_QUERY_HISTORY_VALUES", payload: string[] }
| { type: "SET_QUERY", payload: string[] }
| { type: "SET_QUERY_HISTORY_BY_INDEX", payload: {value: QueryHistory, queryNumber: number} }
| { type: "SET_QUERY_HISTORY", payload: QueryHistory[] }
| { type: "SET_DURATION", payload: string }
| { type: "SET_UNTIL", payload: Date }
| { type: "SET_PERIOD", payload: TimePeriod }
@@ -46,13 +46,13 @@ export type Action =
const duration = getQueryStringValue("g0.range_input", "1h") as string;
const endInput = formatDateToLocal(getQueryStringValue("g0.end_input", getDateNowUTC()) as Date);
const query = getQueryStringValue("g0.expr", "") as string;
const query = getQueryArray();
export const initialState: AppState = {
serverUrl: getDefaultServer(),
displayType: getQueryStringValue("tab", "chart") as DisplayType,
query: query, // demo_memory_usage_bytes
queryHistory: { index: 0, values: [query] },
queryHistory: query.map(q => ({index: 0, values: [q]})),
time: {
duration,
period: getTimeperiodForDuration(duration, new Date(endInput))
@@ -81,21 +81,16 @@ export function reducer(state: AppState, action: Action): AppState {
...state,
query: action.payload
};
case "SET_QUERY_HISTORY_INDEX":
case "SET_QUERY_HISTORY":
return {
...state,
queryHistory: {
...state.queryHistory,
index: action.payload
}
queryHistory: action.payload
};
case "SET_QUERY_HISTORY_VALUES":
case "SET_QUERY_HISTORY_BY_INDEX":
state.queryHistory.splice(action.payload.queryNumber, 1, action.payload.value);
return {
...state,
queryHistory: {
...state.queryHistory,
values: action.payload
}
queryHistory: state.queryHistory
};
case "SET_DURATION":
return {

View File

@@ -1,21 +1,34 @@
export interface AxisRange {
[key: string]: [number, number]
}
export interface YaxisState {
limits: {
enable: boolean,
range: [number, number]
}
limits: {
enable: boolean,
range: AxisRange
}
}
export interface CustomStep {
enable: boolean,
value: number
}
export interface GraphState {
yaxis: YaxisState
customStep: CustomStep
yaxis: YaxisState
}
export type GraphAction =
| { type: "TOGGLE_ENABLE_YAXIS_LIMITS" }
| { type: "SET_YAXIS_LIMITS", payload: [number, number] }
| { type: "TOGGLE_ENABLE_YAXIS_LIMITS" }
| { type: "SET_YAXIS_LIMITS", payload: AxisRange }
| { type: "TOGGLE_CUSTOM_STEP" }
| { type: "SET_CUSTOM_STEP", payload: number}
export const initialGraphState: GraphState = {
customStep: {enable: false, value: 1},
yaxis: {
limits: {enable: false, range: [0, 0]}
limits: {enable: false, range: {"1": [0, 0]}}
}
};
@@ -32,6 +45,22 @@ export function reducer(state: GraphState, action: GraphAction): GraphState {
}
}
};
case "TOGGLE_CUSTOM_STEP":
return {
...state,
customStep: {
...state.customStep,
enable: !state.customStep.enable
}
};
case "SET_CUSTOM_STEP":
return {
...state,
customStep: {
...state.customStep,
value: action.payload
}
};
case "SET_YAXIS_LIMITS":
return {
...state,

View File

@@ -0,0 +1,25 @@
import {styled} from "@mui/material/styles";
import Switch from "@mui/material/Switch";
const BasicSwitch = styled(Switch)(() => ({
padding: 10,
"& .MuiSwitch-track": {
borderRadius: 14,
"&:before, &:after": {
content: "\"\"",
position: "absolute",
top: "50%",
transform: "translateY(-50%)",
width: 14,
height: 14,
},
},
"& .MuiSwitch-thumb": {
boxShadow: "none",
width: 12,
height: 12,
margin: 4,
},
}));
export default BasicSwitch;

View File

@@ -0,0 +1,87 @@
import {createTheme} from "@mui/material/styles";
const THEME = createTheme({
palette: {
primary: {
main: "#3F51B5"
},
secondary: {
main: "#F50057"
},
error: {
main: "#FF4141"
}
},
components: {
MuiFormHelperText: {
styleOverrides: {
root: {
position: "absolute",
top: "36px",
left: "2px",
margin: 0,
}
}
},
MuiInputLabel: {
styleOverrides: {
root: {
fontSize: "12px",
letterSpacing: "normal",
lineHeight: "1"
}
}
},
MuiInputBase: {
styleOverrides: {
"root": {
"&.Mui-focused fieldset": {
"borderWidth": "1px !important"
}
}
}
},
MuiSwitch: {
defaultProps: {
color: "secondary"
}
},
MuiAccordion: {
styleOverrides: {
root: {
boxShadow: "rgba(0, 0, 0, 0.16) 0px 1px 4px;"
},
},
},
MuiPaper: {
styleOverrides: {
elevation3: {
boxShadow: "rgba(0, 0, 0, 0.2) 0px 3px 8px;"
},
},
},
MuiIconButton: {
defaultProps: {
size: "large",
},
styleOverrides: {
sizeLarge: {
borderRadius: "20%",
height: "40px",
width: "41px"
},
sizeMedium: {
borderRadius: "20%",
},
sizeSmall: {
borderRadius: "20%",
}
}
}
},
typography: {
"fontSize": 10
}
});
export default THEME;

View File

@@ -24,8 +24,13 @@ export interface DataSeries extends MetricBase{
values: DataValue[]; // sorted by key which is timestamp
}
export interface InstantDataSeries {
metadata: string[]; // just ordered columns
value: string;
}
export enum ErrorTypes {
emptyServer = "Please enter Server URL",
validServer = "Please provide a valid Server URL",
validQuery = "Please enter a valid Query and execute it"
}

View File

@@ -0,0 +1,12 @@
export interface AppParams {
serverURL: string
}
const getAppModeParams = (): AppParams => {
const dataParams = document.getElementById("root")?.dataset.params || "{}";
return JSON.parse(dataParams);
};
const getAppModeEnable = (): boolean => !!Object.keys(getAppModeParams()).length;
export {getAppModeEnable, getAppModeParams};

View File

@@ -2,10 +2,9 @@ import qs from "qs";
import get from "lodash.get";
const stateToUrlParams = {
"query": "g0.expr",
"time.duration": "g0.range_input",
"time.period.date": "g0.end_input",
"time.period.step": "g0.step_input",
"time.duration": "range_input",
"time.period.date": "end_input",
"time.period.step": "step_input",
"displayType": "tab"
};
@@ -39,15 +38,19 @@ export const setQueryStringWithoutPageReload = (qsValue: string): void => {
export const setQueryStringValue = (newValue: Record<string, unknown>): void => {
const queryMap = new Map(Object.entries(stateToUrlParams));
const query = get(newValue, "query", "") as string[];
const newQsValue: string[] = [];
queryMap.forEach((queryKey, stateKey) => {
// const queryKeyEncoded = encodeURIComponent(queryKey);
const value = get(newValue, stateKey, "") as string;
if (value) {
const valueEncoded = encodeURIComponent(value);
newQsValue.push(`${queryKey}=${valueEncoded}`);
}
query.forEach((q, i) => {
queryMap.forEach((queryKey, stateKey) => {
const value = get(newValue, stateKey, "") as string;
if (value) {
const valueEncoded = encodeURIComponent(value);
newQsValue.push(`g${i}.${queryKey}=${valueEncoded}`);
}
});
newQsValue.push(`g${i}.expr=${q}`);
});
setQueryStringWithoutPageReload(newQsValue.join("&"));
};
@@ -59,3 +62,10 @@ export const getQueryStringValue = (
const values = qs.parse(queryString, { ignoreQueryPrefix: true });
return get(values, key, defaultValue || "");
};
export const getQueryArray = (): string[] => {
const queryLength = window.location.search.match(/g\d+.expr/gmi)?.length || 1;
return new Array(queryLength).fill(1).map((q, i) => {
return getQueryStringValue(`g${i}.expr`, "") as string;
});
};

View File

@@ -1,147 +0,0 @@
import uPlot, {Series as uPlotSeries, Series} from "uplot";
import {getColorFromString} from "./color";
import dayjs from "dayjs";
import {MetricResult} from "../api/types";
import {LegendItem} from "../components/Legend/Legend";
import {getNameForMetric} from "./metric";
import {getMaxFromArray, getMinFromArray} from "./math";
import {roundTimeSeconds} from "./time";
import numeral from "numeral";
interface SetupTooltip {
u: uPlot,
metrics: MetricResult[],
series: Series[],
tooltip: HTMLDivElement,
tooltipOffset: {left: number, top: number},
tooltipIdx: {seriesIdx: number, dataIdx: number}
}
interface HideSeriesArgs {
hideSeries: string[],
label: string,
metaKey: boolean,
series: Series[]
}
interface DragArgs {
e: MouseEvent,
u: uPlot,
factor: number,
setPanning: (enable: boolean) => void,
setPlotScale: ({u, min, max}: {u: uPlot, min: number, max: number}) => void
}
const stub = (): null => null;
export const defaultOptions = {
height: 500,
legend: { show: false },
axes: [
{ space: 80 },
{
show: true,
font: "10px Arial",
values: (self: uPlot, ticks: number[]): (string | number)[] => ticks.map(n => n > 1000 ? numeral(n).format("0.0a") : n)
}
],
cursor: {
drag: { x: false, y: false },
focus: { prox: 30 },
bind: { mouseup: stub, mousedown: stub, click: stub, dblclick: stub, mouseenter: stub }
},
};
export const setTooltip = ({ u, tooltipIdx, metrics, series, tooltip, tooltipOffset }: SetupTooltip) : void => {
const {seriesIdx, dataIdx} = tooltipIdx;
const dataSeries = u.data[seriesIdx][dataIdx];
const dataTime = u.data[0][dataIdx];
const metric = metrics[seriesIdx - 1]?.metric || {};
const color = getColorFromString(series[seriesIdx].label || "");
const {width, height} = u.over.getBoundingClientRect();
const top = u.valToPos((dataSeries || 0), "y");
const lft = u.valToPos(dataTime, "x");
const {width: tooltipWidth, height: tooltipHeight} = tooltip.getBoundingClientRect();
const overflowX = lft + tooltipWidth >= width;
const overflowY = top + tooltipHeight >= height;
tooltip.style.display = "grid";
tooltip.style.top = `${tooltipOffset.top + top + 10 - (overflowY ? tooltipHeight + 10 : 0)}px`;
tooltip.style.left = `${tooltipOffset.left + lft + 10 - (overflowX ? tooltipWidth + 20 : 0)}px`;
const date = dayjs(new Date(dataTime * 1000)).format("YYYY-MM-DD HH:mm:ss:SSS (Z)");
const info = Object.keys(metric).filter(k => k !== "__name__").map(k => `<div><b>${k}</b>: ${metric[k]}</div>`).join("");
const marker = `<div class="u-tooltip__marker" style="background: ${color}"></div>`;
tooltip.innerHTML = `<div>${date}</div>
<div class="u-tooltip-data">
${marker}${metric.__name__ || ""}: <b class="u-tooltip-data__value">${dataSeries}</b>
</div>
<div class="u-tooltip__info">${info}</div>`;
};
export const getHideSeries = ({hideSeries, label, metaKey, series}: HideSeriesArgs): string[] => {
const include = hideSeries.includes(label);
const labels = series.map(s => s.label || "").filter(l => l);
if (metaKey && include) {
return [...labels.filter(l => l !== label)];
} else if (metaKey && !include) {
return hideSeries.length === series.length - 2 ? [] : [...labels.filter(l => l !== label)];
}
return include ? hideSeries.filter(l => l !== label) : [...hideSeries, label];
};
export const getTimeSeries = (times: number[]): number[] => {
const allTimes = Array.from(new Set(times)).sort((a,b) => a-b);
const step = getMinFromArray(allTimes.map((t, i) => allTimes[i + 1] - t));
const length = allTimes.length;
const startTime = allTimes[0] || 0;
return new Array(length).fill(startTime).map((d, i) => roundTimeSeconds(d + (step * i)));
};
export const getLimitsYAxis = (values: number[]): [number, number] => {
const min = getMinFromArray(values);
const max = getMaxFromArray(values);
return [min - (min * 0.05), max + (max * 0.05)];
};
export const getSeriesItem = (d: MetricResult, hideSeries: string[]): Series => {
const label = getNameForMetric(d);
return {
label,
width: 1.5,
stroke: getColorFromString(label),
show: !hideSeries.includes(label),
scale: "y"
};
};
export const getLegendItem = (s: uPlotSeries): LegendItem => ({
label: s.label || "",
color: s.stroke as string,
checked: s.show || false
});
export const dragChart = ({e, factor = 0.85, u, setPanning, setPlotScale}: DragArgs): void => {
if (e.button !== 0) return;
e.preventDefault();
setPanning(true);
const leftStart = e.clientX;
const xUnitsPerPx = u.posToVal(1, "x") - u.posToVal(0, "x");
const scXMin = u.scales.x.min || 0;
const scXMax = u.scales.x.max || 0;
const mouseMove = (e: MouseEvent) => {
e.preventDefault();
const dx = xUnitsPerPx * ((e.clientX - leftStart) * factor);
setPlotScale({u, min: scXMin - dx, max: scXMax - dx});
};
const mouseUp = () => {
setPanning(false);
document.removeEventListener("mousemove", mouseMove);
document.removeEventListener("mouseup", mouseUp);
};
document.addEventListener("mousemove", mouseMove);
document.addEventListener("mouseup", mouseUp);
};

View File

@@ -0,0 +1,31 @@
import {Axis, Series} from "uplot";
import {getMaxFromArray, getMinFromArray} from "../math";
import {roundTimeSeconds} from "../time";
import {AxisRange} from "../../state/graph/reducer";
import {formatTicks} from "./helpers";
import {TimeParams} from "../../types";
export const getAxes = (series: Series[]): Axis[] => Array.from(new Set(series.map(s => s.scale))).map(a => {
const axis = {scale: a, show: true, font: "10px Arial", values: formatTicks};
if (!a) return {space: 80};
if (!(Number(a) % 2)) return {...axis, side: 1};
return axis;
});
export const getTimeSeries = (times: number[], defaultStep: number, period: TimeParams): number[] => {
const allTimes = Array.from(new Set(times)).sort((a, b) => a - b);
const length = Math.ceil((period.end - period.start)/defaultStep);
const startTime = allTimes[0] || 0;
return new Array(length*2).fill(startTime).map((d, i) => roundTimeSeconds(d + (defaultStep * i)));
};
export const getLimitsYAxis = (values: { [key: string]: number[] }): AxisRange => {
const result: AxisRange = {};
for (const key in values) {
const numbers = values[key];
const min = getMinFromArray(numbers);
const max = getMaxFromArray(numbers);
result[key] = min && max ? [min - (min * 0.25), max + (max * 0.25)] : [-1, 1];
}
return result;
};

View File

@@ -0,0 +1,25 @@
import {DragArgs} from "./types";
export const dragChart = ({e, factor = 0.85, u, setPanning, setPlotScale}: DragArgs): void => {
if (e.button !== 0) return;
e.preventDefault();
setPanning(true);
const leftStart = e.clientX;
const xUnitsPerPx = u.posToVal(1, "x") - u.posToVal(0, "x");
const scXMin = u.scales.x.min || 0;
const scXMax = u.scales.x.max || 0;
const mouseMove = (e: MouseEvent) => {
e.preventDefault();
const dx = xUnitsPerPx * ((e.clientX - leftStart) * factor);
setPlotScale({u, min: scXMin - dx, max: scXMax - dx});
};
const mouseUp = () => {
setPanning(false);
document.removeEventListener("mousemove", mouseMove);
document.removeEventListener("mouseup", mouseUp);
};
document.addEventListener("mousemove", mouseMove);
document.addEventListener("mouseup", mouseUp);
};

View File

@@ -0,0 +1,38 @@
import uPlot from "uplot";
import numeral from "numeral";
import {getColorFromString} from "../color";
export const defaultOptions = {
height: 500,
legend: {
show: false
},
cursor: {
drag: {
x: false,
y: false
},
focus: {
prox: 30
},
points: {
size: 5.6,
width: 1.4
},
bind: {
mouseup: (): null => null,
mousedown: (): null => null,
click: (): null => null,
dblclick: (): null => null,
mouseenter: (): null => null
}
},
};
export const formatTicks = (u: uPlot, ticks: number[]): (string | number)[] => {
return ticks.map(n => n > 1000 ? numeral(n).format("0.0a") : n);
};
export const getColorLine = (scale: number, label: string): string => getColorFromString(`${scale}${label}`);
export const getDashLine = (group: number): number[] => group <= 1 ? [] : [group*4, group*1.2];

View File

@@ -0,0 +1,45 @@
import {MetricResult} from "../../api/types";
import {Series} from "uplot";
import {getNameForMetric} from "../metric";
import {LegendItem} from "./types";
import {getColorLine, getDashLine} from "./helpers";
import {HideSeriesArgs} from "./types";
export const getSeriesItem = (d: MetricResult, hideSeries: string[]): Series => {
const label = getNameForMetric(d);
return {
label,
dash: getDashLine(d.group),
width: 1.4,
stroke: getColorLine(d.group, label),
show: !includesHideSeries(label, d.group, hideSeries),
scale: String(d.group),
points: {
size: 4.2,
width: 1.4
}
};
};
export const getLegendItem = (s: Series, group: number): LegendItem => ({
group,
label: s.label || "",
color: s.stroke as string,
checked: s.show || false
});
export const getHideSeries = ({hideSeries, legend, metaKey, series}: HideSeriesArgs): string[] => {
const label = `${legend.group}.${legend.label}`;
const include = includesHideSeries(legend.label, legend.group, hideSeries);
const labels = series.map(s => `${s.scale}.${s.label}`);
if (metaKey && include) {
return [...labels.filter(l => l !== label)];
} else if (metaKey && !include) {
return hideSeries.length >= series.length - 1 ? [] : [...labels.filter(l => l !== label)];
}
return include ? hideSeries.filter(l => l !== label) : [...hideSeries, label];
};
export const includesHideSeries = (label: string, group: string | number, hideSeries: string[]): boolean => {
return hideSeries.includes(`${group}.${label}`);
};

View File

@@ -0,0 +1,31 @@
import dayjs from "dayjs";
import {SetupTooltip} from "./types";
import {getColorLine} from "./helpers";
export const setTooltip = ({u, tooltipIdx, metrics, series, tooltip, tooltipOffset}: SetupTooltip): void => {
const {seriesIdx, dataIdx} = tooltipIdx;
if (seriesIdx === null || dataIdx === undefined) return;
const dataSeries = u.data[seriesIdx][dataIdx];
const dataTime = u.data[0][dataIdx];
const metric = metrics[seriesIdx - 1]?.metric || {};
const color = getColorLine(Number(series[seriesIdx].scale || 0), series[seriesIdx].label || "");
const {width, height} = u.over.getBoundingClientRect();
const top = u.valToPos((dataSeries || 0), series[seriesIdx]?.scale || "1");
const lft = u.valToPos(dataTime, "x");
const {width: tooltipWidth, height: tooltipHeight} = tooltip.getBoundingClientRect();
const overflowX = lft + tooltipWidth >= width;
const overflowY = top + tooltipHeight >= height;
tooltip.style.display = "grid";
tooltip.style.top = `${tooltipOffset.top + top + 10 - (overflowY ? tooltipHeight + 10 : 0)}px`;
tooltip.style.left = `${tooltipOffset.left + lft + 10 - (overflowX ? tooltipWidth + 20 : 0)}px`;
const date = dayjs(new Date(dataTime * 1000)).format("YYYY-MM-DD HH:mm:ss:SSS (Z)");
const info = Object.keys(metric).filter(k => k !== "__name__").map(k => `<div><b>${k}</b>: ${metric[k]}</div>`).join("");
const marker = `<div class="u-tooltip__marker" style="background: ${color}"></div>`;
tooltip.innerHTML = `<div>${date}</div>
<div class="u-tooltip-data">
${marker}${metric.__name__ || ""}: <b class="u-tooltip-data__value">${dataSeries}</b>
</div>
<div class="u-tooltip__info">${info}</div>`;
};

View File

@@ -0,0 +1,39 @@
import uPlot, {Series} from "uplot";
import {MetricResult} from "../../api/types";
export interface SetupTooltip {
u: uPlot,
metrics: MetricResult[],
series: Series[],
tooltip: HTMLDivElement,
tooltipOffset: {
left: number,
top: number
},
tooltipIdx: {
seriesIdx: number | null,
dataIdx: number | undefined
}
}
export interface HideSeriesArgs {
hideSeries: string[],
legend: LegendItem,
metaKey: boolean,
series: Series[]
}
export interface DragArgs {
e: MouseEvent,
u: uPlot,
factor: number,
setPanning: (enable: boolean) => void,
setPlotScale: ({u, min, max}: { u: uPlot, min: number, max: number }) => void
}
export interface LegendItem {
group: number;
label: string;
color: string;
checked: boolean;
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -4,7 +4,7 @@ DOCKER_NAMESPACE := victoriametrics
ROOT_IMAGE ?= alpine:3.15.0
CERTS_IMAGE := alpine:3.15.0
GO_BUILDER_IMAGE := golang:1.17.3-alpine
GO_BUILDER_IMAGE := golang:1.17.5-alpine
BUILDER_IMAGE := local/builder:2.0.0-$(shell echo $(GO_BUILDER_IMAGE) | tr :/ __)
BASE_IMAGE := local/base:1.1.3-$(shell echo $(ROOT_IMAGE) | tr :/ __)-$(shell echo $(CERTS_IMAGE) | tr :/ __)

View File

@@ -216,6 +216,16 @@ groups:
description: "Vmagent dropped {{ $value | humanize1024 }} from persistent queue
on instance {{ $labels.instance }} for the last 10m."
- alert: RejectedRemoteWriteDataBlocksAreDropped
expr: sum(increase(vmagent_remotewrite_packets_dropped_total[5m])) by (job, instance) > 0
for: 15m
labels:
severity: warning
annotations:
dashboard: "http://localhost:3000/d/G7Z9GzMGz?viewPanel=79&var-instance={{ $labels.instance }}"
summary: "Job \"{{ $labels.job }}\" on instance {{ $labels.instance }} drops the rejected by
remote-write server data blocks. Check the logs to find the reason for rejects."
- alert: TooManyScrapeErrors
expr: sum(increase(vm_promscrape_scrapes_failed_total[5m])) by (job, instance) > 0
for: 15m
@@ -261,6 +271,30 @@ groups:
This usually means that `-remoteWrite.queues` command-line flag must be increased in order to increase
the number of connections per each remote storage."
- alert: PersistentQueueForWritesIsSaturated
expr: rate(vm_persistentqueue_write_duration_seconds_total[5m]) > 0.9
for: 15m
labels:
severity: warning
annotations:
dashboard: "http://localhost:3000/d/G7Z9GzMGz?viewPanel=98&var-instance={{ $labels.instance }}"
summary: "Persistent queue writes for instance {{ $labels.instance }} are saturated"
description: "Persistent queue writes for vmagent \"{{ $labels.job }}\" (instance {{ $labels.instance }})
are saturated by more than 90% and vmagent won't be able to keep up with flushing data on disk.
In this case, consider to decrease load on the vmagent or improve the disk throughput."
- alert: PersistentQueueForReadsIsSaturated
expr: rate(vm_persistentqueue_read_duration_seconds_total[5m]) > 0.9
for: 15m
labels:
severity: warning
annotations:
dashboard: "http://localhost:3000/d/G7Z9GzMGz?viewPanel=99&var-instance={{ $labels.instance }}"
summary: "Persistent queue reads for instance {{ $labels.instance }} are saturated"
description: "Persistent queue reads for vmagent \"{{ $labels.job }}\" (instance {{ $labels.instance }})
are saturated by more than 90% and vmagent won't be able to keep up with reading data from the disk.
In this case, consider to decrease load on the vmagent or improve the disk throughput."
- alert: SeriesLimitHourReached
expr: (vmagent_hourly_series_limit_current_series / vmagent_hourly_series_limit_max_series) > 0.9
labels:

View File

@@ -39,7 +39,7 @@ services:
restart: always
grafana:
container_name: grafana
image: grafana/grafana:8.2.2
image: grafana/grafana:8.3.2
depends_on:
- "victoriametrics"
ports:

View File

@@ -7,6 +7,31 @@ sort: 15
## tip
## [v1.71.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.71.0)
* FEATURE: [VictoriaMetrics enterprise](https://victoriametrics.com/enterprise.html): add multi-level downsampling support. See [these docs](https://docs.victoriametrics.com/#downsampling) and [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/36).
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add ability to analyze the correlation between two queries on a single graph. Just click `+Query` button, enter the second query in the newly appeared input field and press `Ctrl+Enter`. Results for both queries should be displayed simultaneously on the same graph. Every query has its own vertical scale, which is displayed on the left and the right side of the graph. Lines for the second query are dashed. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1916).
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add ability to override the interval between returned datapoints. By default it is automatically calculated depending on the selected time range and horizontal resolution of the graph. Now it is possible to override it with custom values. This may be useful during data exploration and debugging.
* FEATURE: accept optional `extra_filters[]=series_selector` query args at Prometheus query APIs additionally to `extra_label` query args. This allows enforcing additional filters for all the Prometheus query APIs by using [vmgateway](https://docs.victoriametrics.com/vmgateway.html) or [vmauth](https://docs.victoriametrics.com/vmauth.html). See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1863).
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth.html): allow specifying `http` and `https` urls in `-auth.config` command-line flag. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1898). Thanks for @TFM93 .
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): allow specifying `http` and `https` urls in the following command-line flags: `-promscrape.config`, `-remoteWrite.relabelConfig` and `-remoteWrite.urlRelabelConfig`.
* FEATURE: vminsert: allow specifying `http` and `https` urls in `-relabelConfig` command-line flag.
* FEATURE: vminsert: add `-maxLabelValueLen` command-line flag for the ability to configure the maximum length of label value. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1908).
* FEATURE: preserve the order of time series passed to [limit_offset](https://docs.victoriametrics.com/MetricsQL.html#limit_offset) function. This allows implementing series paging via `limit_offset(limit, offset, sort_by_label(...))`. See [this](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1920) and [this](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/951) issues.
* FEATURE: automaticall convert `(value1|...|valueN)` into `{value1,...,valueN}` inside `__graphite__` pseudo-label. This allows using [Grafana multi-value template variables](https://grafana.com/docs/grafana/latest/variables/formatting-multi-value-variables/) inside `__graphite__` pseudo-label. For example, `{__graphite__=~"foo.($bar)"}` is expanded to `{__graphite__=~"foo.{x,y}"}` if both `x` and `y` are selected for `$bar` template variable. See [these docs](https://docs.victoriametrics.com/#selecting-graphite-metrics) for details.
* FEATURE: add [timestamp_with_name](https://docs.victoriametrics.com/MetricsQL.html#timestamp_with_name) function. It works the same as [timestamp](https://docs.victoriametrics.com/MetricsQL.html#timestamp), but leaves the original time series names, so it can be used in queries, which match multiple time series names: `timestamp_with_name({foo="bar"}[1h])`. See [this comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949#issuecomment-995222388) for more context.
* FEATURE: add [changes_prometheus](https://docs.victoriametrics.com/MetricsQL.html#changes_prometheus), [increase_prometheus](https://docs.victoriametrics.com/MetricsQL.html#increase_prometheus) and [delta_prometheus](https://docs.victoriametrics.com/MetricsQL.html#delta_prometheus) functions, which don't take into account the previous sample before the given lookbehind window specified in square brackets. These functions may be used when the Prometheus behaviour for `changes()`, `increase()` and `delta()` functions is needed to be preserved. VictoriaMetrics uses slightly different behaviour for `changes()`, `increase()` and `delta()` functions by default - see [this article](https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e) for details. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1962).
* BUGFIX: fix `unaligned 64-bit atomic operation` panic on 32-bit architectures, which has been introduced in v1.70.0. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1944).
* BUGFIX: [vmalert](https://docs.victoriametrics.com/vmalert.html): restore the ability to use `$labels.alertname` in labels templating. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1921).
* BUGFIX: [vmui](https://docs.victoriametrics.com/#vmui): add missing `query` caption to the input field for the query. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1900).
* BUGFIX: [vmui](https://docs.victoriametrics.com/#vmui): fix navigation over query history with `Ctrl+up/down` and fix zoom relatively to the cursor position. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1936).
* BUGFIX: deduplicate samples more thoroughly if [deduplication](https://docs.victoriametrics.com/#deduplication) is enabled. Previously some duplicate samples may be left on disk for time series with high churn rate. This may result in bigger storage space requirements.
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent.html): follow up to 5 redirects when `follow_redirects: true` is set for a particular scrape config. Previously only a single redirect was performed in this case. It is expected these redirects are performed to the original hostname. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1945).
* BUGFIX: de-duplicate data exported via [/api/v1/export/csv](https://docs.victoriametrics.com/#how-to-export-csv-data) by default if [deduplication](https://docs.victoriametrics.com/#deduplication) is enabled. The de-duplication can be disabled by passing `reduce_mem_usage=1` query arg to `/api/v1/export/csv`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1837).
* BUGFIX: [vmalert](https://docs.victoriametrics.com/vmalert.html): properly store [historical data](https://docs.victoriametrics.com/vmalert.html#rules-backfilling) to old Prometheus versions. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1943).
## [v1.70.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.70.0)
* FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert.html): add ability to pass arbitrary query args to `-datasource.url` on a per-group basis via `params` option. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1892).
@@ -14,7 +39,7 @@ sort: 15
* FEATURE: vmauth: allow using optional `name` field in configs. This field is then used as `username` label value for `vmauth_user_requests_total` metric. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1805).
* FEATURE: vmagent: export `vm_persistentqueue_read_duration_seconds_total` and `vm_persistentqueue_write_duration_seconds_total` metrics, which can be used for detecting persistent queue saturation with `rate(vm_persistentqueue_write_duration_seconds_total) > 0.9` alerting rule.
* FEATURE: export `vm_filestream_read_duration_seconds_total` and `vm_filestream_write_duration_seconds_total` metrics, which can be used for detecting persistent disk saturation with `rate(vm_filestream_read_duration_seconds_total) > 0.9` alerting rule.
* FEATURE: export `vm_cache_size_max_bytes` metrics, which show capacity for various caches. These metrics can be used for determining caches reaches its capacity with `vm_cache_size_bytes / vm_cache_size_max_bytes > 0.9` query.
* FEATURE: export `vm_cache_size_max_bytes` metrics, which show capacity for various caches. These metrics can be used for determining caches with reach its capacity with `vm_cache_size_bytes / vm_cache_size_max_bytes > 0.9` query.
* FEATURE: [vmbackup](https://docs.victoriametrics.com/vmbackup.html), [vmrestore](https://docs.victoriametrics.com/vmrestore.html): add `-s3ForcePathStyle` command-line flag, which can be used for making backups to [Aliyun OSS](https://www.aliyun.com/product/oss). See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1802).
* FEATURE: [vmctl](https://docs.victoriametrics.com/vmctl.html): improve data migration from OpenTSDB. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1809). Thanks to @johnseekins .
* FEATURE: suppress `connection reset by peer` errors when remote client resets TCP connection to VictoriaMetrics / vmagent while ingesting the data via InfluxDB line protocol, Graphite protocol or OpenTSDB protocol. This error is expected, so there is no need in logging it.
@@ -22,7 +47,7 @@ sort: 15
* FEATURE: vmalert: make `-notifier.url` command-line flag optional. This flag can be omitted if `vmalert` is used solely for recording rules and doesn't evaluate alerting rules. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1870).
* FEATURE: [vmbackup](https://docs.victoriametrics.com/vmbackup.html), [vmrestore](https://docs.victoriametrics.com/vmrestore.html): export internal metrics at `http://vmbackup:8420/metrics` and `http://vmrestore:8421/metrics` for better visibility of the backup/restore process.
* FEATURE: allow trailing whitespace after the timestamp when [parsing Graphite plaintext lines](https://docs.victoriametrics.com/#how-to-send-data-from-graphite-compatible-agents-such-as-statsd). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1865).
* FEATURE: expose `/-/healthy` and `/-/ready` endpoints as Prometheus does. This is needed for improving integration with third-party solutions, which rely on these endpoints. See [tis issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1833).
* FEATURE: expose `/-/healthy` and `/-/ready` endpoints as Prometheus does. This is needed for improving integration with third-party solutions, which rely on these endpoints. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1833).
* BUGFIX: vmagent: prevent from scraping duplicate targets if `-promscrape.dropOriginalLabels` command-line flag is set. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1830). Thanks to @guidao for the fix.
* BUGFIX: vmstorage [enterprise](https://victoriametrics.com/enterprise.html): added missing `vm_tenant_used_tenant_bytes` metric, which shows the approximate per-tenant disk usage. See [these docs](https://docs.victoriametrics.com/PerTenantStatistic.html) and [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1605).

View File

@@ -315,7 +315,8 @@ It is recommended leaving the following amounts of spare resources:
* 50% of free RAM across all the node types for reducing the probability of OOM (out of memory) crashes and slowdowns during temporary spikes in workload.
* 50% of spare CPU across all the node types for reducing the probability of slowdowns during temporary spikes in workload.
* At least 30% of free storage space at the directory pointed by `-storageDataPath` command-line flag at `vmstorage` nodes.
* At least 30% of free storage space at the directory pointed by `-storageDataPath` command-line flag at `vmstorage` nodes. See also `-storage.minFreeDiskSpaceBytes` command-line flag [description for vmstorage](#list-of-command-line-flags-for-vmstorage).
Some capacity planning tips for VictoriaMetrics cluster:
@@ -407,6 +408,11 @@ Restoring from backup:
3. Start `vmstorage` node.
## Downsampling
Downsampling is available in [enterprise version of VictoriaMetrics](https://victoriametrics.com/enterprise.html). It is configured with `-downsampling.period` command-line flag. The same flag value must be passed to both `vmstorage` and `vmselect` nodes. See [these docs](https://docs.victoriametrics.com/#downsampling) for details.
## Profiling
All the cluster components provide the following handlers for [profiling](https://blog.golang.org/profiling-go-programs):
@@ -484,6 +490,8 @@ Below is the output for `/path/to/vminsert -help`:
Whether to enable reading flags from environment variables additionally to command line. Command line flag values have priority over values from environment vars. Flags are read only from command line if this flag isn't set. See https://docs.victoriametrics.com/#environment-variables for more details
-envflag.prefix string
Prefix for environment variables if -envflag.enable is set
-eula
By specifying this flag, you confirm that you have an enterprise license and accept the EULA https://victoriametrics.com/assets/VM_EULA.pdf
-fs.disableMmap
Whether to use pread() instead of mmap() for reading data files. By default mmap() is used for 64-bit arches and pread() is used for 32-bit arches, since they cannot read data files bigger than 2^32 bytes in memory. mmap() is usually faster for reading small data chunks than pread()
-graphiteListenAddr string
@@ -544,8 +552,10 @@ Below is the output for `/path/to/vminsert -help`:
-maxInsertRequestSize size
The maximum size in bytes of a single Prometheus remote_write API request
Supports the following optional suffixes for size values: KB, MB, GB, KiB, MiB, GiB (default 33554432)
-maxLabelValueLen int
The maximum length of label values in the accepted time series. Longer label values are truncated. In this case the vm_too_long_label_values_total metric at /metrics page is incremented (default 16384)
-maxLabelsPerTimeseries int
The maximum number of labels accepted per time series. Superfluous labels are dropped (default 30)
The maximum number of labels accepted per time series. Superfluous labels are dropped. In this case the vm_metrics_with_dropped_labels_total metric at /metrics page is incremented (default 30)
-memory.allowedBytes size
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from OS page cache resulting in higher disk IO usage
Supports the following optional suffixes for size values: KB, MB, GB, KiB, MiB, GiB (default 0)
@@ -563,7 +573,7 @@ Below is the output for `/path/to/vminsert -help`:
-opentsdbhttpTrimTimestamp duration
Trim timestamps for OpenTSDB HTTP data to this duration. Minimum practical duration is 1ms. Higher duration (i.e. 1s) may be used for reducing disk space usage for timestamp data (default 1ms)
-relabelConfig string
Optional path to a file with relabeling rules, which are applied to all the ingested metrics. See https://docs.victoriametrics.com/#relabeling for details. The config is reloaded on SIGHUP signal
Optional path to a file with relabeling rules, which are applied to all the ingested metrics. The path can point either to local file or to http url. See https://docs.victoriametrics.com/#relabeling for details. The config is reloaded on SIGHUP signal
-relabelDebug
Whether to log metrics before and after relabeling with -relabelConfig. If the -relabelDebug is enabled, then the metrics aren't sent to storage. This is useful for debugging the relabeling configs
-replicationFactor int
@@ -594,12 +604,17 @@ Below is the output for `/path/to/vmselect -help`:
Path to directory for cache files. Cache isn't saved if empty
-dedup.minScrapeInterval duration
Leave only the first sample in every time series per each discrete interval equal to -dedup.minScrapeInterval > 0. See https://docs.victoriametrics.com/#deduplication for details
-downsampling.period array
Comma-separated downsampling periods in the format 'offset:period'. For example, '30d:10m' instructs to leave a single sample per 10 minutes for samples older than 30 days. See https://docs.victoriametrics.com/#downsampling for details
Supports an array of values separated by comma or specified via multiple flags.
-enableTCP6
Whether to enable IPv6 for listening and dialing. By default only IPv4 TCP and UDP is used
-envflag.enable
Whether to enable reading flags from environment variables additionally to command line. Command line flag values have priority over values from environment vars. Flags are read only from command line if this flag isn't set. See https://docs.victoriametrics.com/#environment-variables for more details
-envflag.prefix string
Prefix for environment variables if -envflag.enable is set
-eula
By specifying this flag, you confirm that you have an enterprise license and accept the EULA https://victoriametrics.com/assets/VM_EULA.pdf
-fs.disableMmap
Whether to use pread() instead of mmap() for reading data files. By default mmap() is used for 64-bit arches and pread() is used for 32-bit arches, since they cannot read data files bigger than 2^32 bytes in memory. mmap() is usually faster for reading small data chunks than pread()
-graphiteTrimTimestamp duration
@@ -645,6 +660,10 @@ Below is the output for `/path/to/vmselect -help`:
Whether to deny partial responses if a part of -storageNode instances fail to perform queries; this trades availability over consistency; see also -search.maxQueryDuration
-search.disableCache
Whether to disable response caching. This may be useful during data backfilling
-search.graphiteMaxPointsPerSeries int
The maximum number of points per series Graphite render API can return (default 1000000)
-search.graphiteStorageStep duration
The interval between datapoints stored in the database. It is used at Graphite Render API handler for normalizing the interval between datapoints in case it isn't normalized. It can be overriden by sending 'storage_step' query arg to /render API or by sending the desired interval via 'Storage-Step' http header during querying /render API (default 10s)
-search.latencyOffset duration
The time when data points become visible in query results after the collection. Too small value can result in incomplete last points for query results (default 30s)
-search.logSlowQueryDuration duration
@@ -713,12 +732,17 @@ Below is the output for `/path/to/vmstorage -help`:
Leave only the first sample in every time series per each discrete interval equal to -dedup.minScrapeInterval > 0. See https://docs.victoriametrics.com/#deduplication for details
-denyQueriesOutsideRetention
Whether to deny queries outside of the configured -retentionPeriod. When set, then /api/v1/query_range would return '503 Service Unavailable' error for queries with 'from' value outside -retentionPeriod. This may be useful when multiple data sources with distinct retentions are hidden behind query-tee
-downsampling.period array
Comma-separated downsampling periods in the format 'offset:period'. For example, '30d:10m' instructs to leave a single sample per 10 minutes for samples older than 30 days. See https://docs.victoriametrics.com/#downsampling for details
Supports an array of values separated by comma or specified via multiple flags.
-enableTCP6
Whether to enable IPv6 for listening and dialing. By default only IPv4 TCP and UDP is used
-envflag.enable
Whether to enable reading flags from environment variables additionally to command line. Command line flag values have priority over values from environment vars. Flags are read only from command line if this flag isn't set. See https://docs.victoriametrics.com/#environment-variables for more details
-envflag.prefix string
Prefix for environment variables if -envflag.enable is set
-eula
By specifying this flag, you confirm that you have an enterprise license and accept the EULA https://victoriametrics.com/assets/VM_EULA.pdf
-finalMergeDelay duration
The delay before starting final merge for per-month partition after no new data is ingested into it. Final merge may require additional disk IO and CPU resources. Final merge may increase query speed and reduce disk space usage in some cases. Zero value disables final merge
-forceFlushAuthKey string

View File

@@ -282,6 +282,13 @@ If old time series are constantly substituted by new time series at a high rate,
* Increased size of inverted index, which is stored at `<-storageDataPath>/indexdb`, since the inverted index contains entries for every label of every time series with at least a single ingested sample
* Slow down of queries over multiple days.
The main reason for high churn rate is a metric label with frequently changed value. Examples of such labels:
* `queryid`, which changes with each query at `postgres_exporter`.
* `app_name` or `deployment_id`, which changes with each new deployment in Kubernetes.
* A label derived from the current time such as `timestamp`, `minute` or `hour`.
* A `hash` or `uuid` label, which changes frequently.
The solution against high churn rate is to identify and eliminate labels with frequently changed values. The [/api/v1/status/tsdb](https://docs.victoriametrics.com/#tsdb-stats) page can help determining these labels.
@@ -323,3 +330,11 @@ Please see [these docs](https://docs.victoriametrics.com/vmctl.html#migrating-da
## How to migrate data from Graphite to VictoriaMetrics?
Please use [whisper-to-graphite](https://github.com/bzed/whisper-to-graphite) tool for reading the data from Graphite and pushing it to VictoriaMetrics via [Graphite import API](https://docs.victoriametrics.com/#how-to-send-data-from-graphite-compatible-agents-such-as-statsd).
## Why do same metrics have differences in VictoriaMetrics and Prometheus dashboards?
There could be a slight difference in stored values for time series. Due to different compression algorithms, VM may reduce precision for float values with more than 12 significant decimal digits. Please see [this article](https://valyala.medium.com/evaluating-performance-and-correctness-victoriametrics-response-e27315627e87)
The query engine may behave differently for some functions. Please see [this article](https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e)

View File

@@ -30,7 +30,7 @@ MetricsQL implements [PromQL](https://medium.com/@valyala/promql-tutorial-for-be
This functionality can be evaluated at [an editable Grafana dashboard](https://play-grafana.victoriametrics.com/d/4ome8yJmz/node-exporter-on-victoriametrics-demo) or at your own [VictoriaMetrics instance](https://docs.victoriametrics.com/#how-to-start-victoriametrics).
- Graphite-compatible filters can be passed via `{__graphite__="foo.*.bar"}` syntax. This is equivalent to `{__name__=~"foo[.][^.]*[.]bar"}`, but usually works faster and is easier to use when migrating from Graphite. VictoriaMetrics also can be used as Graphite datasource in Grafana. See [these docs](https://docs.victoriametrics.com/#graphite-api-usage) for details. See also [label_graphite_group](#label_graphite_group) function, which can be used for extracting the given groups from Graphite metric name.
- Graphite-compatible filters can be passed via `{__graphite__="foo.*.bar"}` syntax. See [these docs](https://docs.victoriametrics.com/#selecting-graphite-metrics). VictoriaMetrics also can be used as Graphite datasource in Grafana. See [these docs](https://docs.victoriametrics.com/#graphite-api-usage) for details. See also [label_graphite_group](#label_graphite_group) function, which can be used for extracting the given groups from Graphite metric name.
- Lookbehind window in square brackets may be omitted. VictoriaMetrics automatically selects the lookbehind window depending on the current step used for building the graph (e.g. `step` query arg passed to [/api/v1/query_range](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries)). For instance, the following query is valid in VictoriaMetrics: `rate(node_network_receive_bytes_total)`. It is equivalent to `rate(node_network_receive_bytes_total[$__interval])` when used in Grafana.
- [Aggregate functions](#aggregate-functions) accept arbitrary number of args. For example, `avg(q1, q2, q3)` would return the average values for every point across time series returned by `q1`, `q2` and `q3`.
- [offset](https://prometheus.io/docs/prometheus/latest/querying/basics/#offset-modifier), lookbehind window in square brackets and `step` value for [subquery](#subqueries) may refer to the current step aka `$__interval` value from Grafana with `[Ni]` syntax. For instance, `rate(metric[10i] offset 5i)` would return per-second rate over a range covering 10 previous steps with the offset of 5 steps.
@@ -92,7 +92,11 @@ See also [implicit query conversions](#implicit-query-conversions).
#### changes
`changes(series_selector[d])` calculates the number of times the raw samples changed on the given lookbehind window `d` per each time series returned from the given [series_selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors). Metric names are stripped from the resulting rollups. This function is supported by PromQL.
`changes(series_selector[d])` calculates the number of times the raw samples changed on the given lookbehind window `d` per each time series returned from the given [series_selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors). Unlike `changes()` in Prometheus it takes into account the change from the last sample before the given lookbehind window `d`. See [this article](https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e) for details. Metric names are stripped from the resulting rollups. This function is supported by PromQL. See also [changes_prometheus](#changes_prometheus).
#### changes_prometheus
`changes_prometheus(series_selector[d])` calculates the number of times the raw samples changed on the given lookbehind window `d` per each time series returned from the given [series_selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors). It doesn't take into account the change from the last sample before the given lookbehind window `d` in the same way as Prometheus does. See [this article](https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e) for details. Metric names are stripped from the resulting rollups. This function is supported by PromQL. See also [changes](#changes).
#### count_eq_over_time
@@ -124,7 +128,11 @@ See also [implicit query conversions](#implicit-query-conversions).
#### delta
`delta(series_selector[d])` calculates the difference between the first and the last point over the given lookbehind window `d` per each time series returned from the given [series_selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors). Metric names are stripped from the resulting rollups. This function is supported by PromQL. See also [increase](#increase).
`delta(series_selector[d])` calculates the difference between the last sample before the given lookbehind window `d` and the last sample at the given lookbehind window `d` per each time series returned from the given [series_selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors). The behaviour of `delta()` function in MetricsQL is slighly different to the behaviour of `delta()` function in Prometheus. See [this article](https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e) for details. Metric names are stripped from the resulting rollups. This function is supported by PromQL. See also [increase](#increase) and [delta_prometheus](#delta_prometheus).
#### delta_prometheus
`delta_prometheus(series_selector[d])` calculates the difference between the first and the last samples at the given lookbehind window `d` per each time series returned from the given [series_selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors). The behaviour of `delta_prometheus()` is close to the behaviour of `delta()` function in Prometheus. See [this article](https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e) for details. Metric names are stripped from the resulting rollups. See also [delta](#delta).
#### deriv
@@ -180,7 +188,11 @@ See also [implicit query conversions](#implicit-query-conversions).
#### increase
`increase(series_selector[d])` calculates the increase over the given lookbehind window `d` per each time series returned from the given [series_selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors). It is expected that the `series_selector` returns time series of [counter type](https://prometheus.io/docs/concepts/metric_types/#counter). Metric names are stripped from the resulting rollups. This function is supported by PromQL. See also [increase_pure](#increase_pure) and [delta](#delta).
`increase(series_selector[d])` calculates the increase over the given lookbehind window `d` per each time series returned from the given [series_selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors). It is expected that the `series_selector` returns time series of [counter type](https://prometheus.io/docs/concepts/metric_types/#counter). Unlike Prometheus it takes into account the last sample before the given lookbehind window `d` when calculating the result. See [this article](https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e) for details. Metric names are stripped from the resulting rollups. This function is supported by PromQL. See also [increase_pure](#increase_pure), [increase_prometheus](#increase_prometheus) and [delta](#delta).
#### increase_prometheus
`increase_prometheus(series_selector[d])` calculates the increase over the given lookbehind window `d` per each time series returned from the given [series_selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors). It is expected that the `series_selector` returns time series of [counter type](https://prometheus.io/docs/concepts/metric_types/#counter). It doesn't take into account the last sample before the given lookbehind window `d` when calculating the result in the same way as Prometheus does. See [this article](https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e) for details. Metric names are stripped from the resulting rollups. This function is supported by PromQL. See also [increase_pure](#increase_pure) and [increase](#increase).
#### increase_pure
@@ -316,7 +328,11 @@ See also [implicit query conversions](#implicit-query-conversions).
#### timestamp
`timestamp(series_selector[d])` returns the timestamp in seconds for the last raw sample on the given lookbehind window `d` per each time series returned from the given [series_selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors). Metric names are stripped from the resulting rollups. This function is supported by PromQL.
`timestamp(series_selector[d])` returns the timestamp in seconds for the last raw sample on the given lookbehind window `d` per each time series returned from the given [series_selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors). Metric names are stripped from the resulting rollups. This function is supported by PromQL. See also [timestamp_with_name](#timestamp_with_name).
#### timestamp_with_name
`timestamp_with_name(series_selector[d])` returns the timestamp in seconds for the last raw sample on the given lookbehind window `d` per each time series returned from the given [series_selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors). Metric names are preserved in the resulting rollups. See also [timestamp](#timestamp).
#### tfirst_over_time
@@ -487,6 +503,10 @@ See also [implicit query conversions](#implicit-query-conversions).
`keep_next_value(q)` fills gaps with the value of the next non-empty point in every time series returned by `q`. See also [keep_last_value](#keep_last_value) and [interpolate](#interpolate).
#### limit_offset
`limit_offset(limit, offset, q)` skips `offset` time series from series returned by `q` and then returns up to `limit` of the remaining time series per each group. This allows implementing simple paging for `q` time series. See also [limitk](#limitk).
#### ln
`ln(q)` calculates `ln(v)` for every point `v` of every time series returned by `q`. Metric names are stripped from the resulting series. This function is supported by PromQL. See also [exp](#exp) and [log2](#log2).
@@ -823,11 +843,6 @@ See also [implicit query conversions](#implicit-query-conversions).
`histogram(q)` calculates [VictoriaMetrics histogram](https://valyala.medium.com/improving-histogram-usability-for-prometheus-and-grafana-bc7e5df0e350) per each group of points with the same timestamp. Useful for visualizing big number of time series via a heatmap. See [this article](https://medium.com/@valyala/improving-histogram-usability-for-prometheus-and-grafana-bc7e5df0e350) for more details.
#### limit_offset
`limit_offset(limit, offset, q)` skips `offset` time series from series returned by `q` and then returns up to `limit` of the remaining time series. This allows implementing simple paging for `q` time series. See also [limitk](#limitk).
#### limitk
`limitk(k, q) by (group_labels)` returns up to `k` time series per each `group_labels` out of time series returned by `q`. The returned set of time series remain the same across calls. See also [limit_offset](#limit_offset).
@@ -915,11 +930,11 @@ See also [implicit query conversions](#implicit-query-conversions).
## Subqueries
MetricsQL supports and extends PromQL subqueries. See [this article](https://valyala.medium.com/prometheus-subqueries-in-victoriametrics-9b1492b720b3) for details. Any nested [rollup functions](#rollup-functions) form a subquery. Nested rollup functions can be implicit thanks to the [implicit query conversions](#implicit-query-conversions). For example, `delta(sum(m))` is implicitly converted to `delta(sum(default_rollup(m[1i]))[1i:1i])`, so it becomes a subquery, since it contains [default_rollup](#default_rollup) nested into [delta](#delta).
MetricsQL supports and extends PromQL subqueries. See [this article](https://valyala.medium.com/prometheus-subqueries-in-victoriametrics-9b1492b720b3) for details. Any [rollup function](#rollup-functions) for something other than [series selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors) form a subquery. Nested rollup functions can be implicit thanks to the [implicit query conversions](#implicit-query-conversions). For example, `delta(sum(m))` is implicitly converted to `delta(sum(default_rollup(m[1i]))[1i:1i])`, so it becomes a subquery, since it contains [default_rollup](#default_rollup) nested into [delta](#delta).
VictoriaMetrics performs subqueries in the following way:
* It calculates the inner rollup function using the `step` value from the outer rollup function. For example, if `max_over_time(rate(http_requests_total[5m])[1h:30s])` is executed, then the `rate(http_requests_total[5m])` is calculated with the `step` equal to `30s`. The resulting data points are algined by the `step`.
* It calculates the inner rollup function using the `step` value from the outer rollup function. For example, for expression `max_over_time(rate(http_requests_total[5m])[1h:30s])` the inner function `rate(http_requests_total[5m])` is calculated with `step=30s`. The resulting data points are aligned by the `step`.
* It calculates the outer rollup function over the results of the inner rollup function using the `step` value passed by Grafana to [/api/v1/query_range](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries).

View File

@@ -13,46 +13,13 @@
VictoriaMetrics is a fast, cost-effective and scalable monitoring solution and time series database.
VictoriaMetrics is available in [binary releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases),
in [Docker images](https://hub.docker.com/r/victoriametrics/victoria-metrics/), in [Snap packages](https://snapcraft.io/victoriametrics)
and in [source code](https://github.com/VictoriaMetrics/VictoriaMetrics). Just download VictoriaMetrics follow [these instructions](#how-to-start-victoriametrics).
[Docker images](https://hub.docker.com/r/victoriametrics/victoria-metrics/), [Snap packages](https://snapcraft.io/victoriametrics)
and [source code](https://github.com/VictoriaMetrics/VictoriaMetrics). Just download VictoriaMetrics and follow [these instructions](#how-to-start-victoriametrics).
Then read [Prometheus setup](#prometheus-setup) and [Grafana setup](#grafana-setup) docs.
Cluster version of VictoriaMetrics is available [here](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html).
[Contact us](mailto:info@victoriametrics.com) if you need enterprise support for VictoriaMetrics.
See [features available in enterprise package](https://victoriametrics.com/enterprise.html).
Enterprise binaries can be downloaded and evaluated for free from [the releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
## Case studies and talks
Case studies:
* [AbiosGaming](https://docs.victoriametrics.com/CaseStudies.html#abiosgaming)
* [adidas](https://docs.victoriametrics.com/CaseStudies.html#adidas)
* [Adsterra](https://docs.victoriametrics.com/CaseStudies.html#adsterra)
* [ARNES](https://docs.victoriametrics.com/CaseStudies.html#arnes)
* [Brandwatch](https://docs.victoriametrics.com/CaseStudies.html#brandwatch)
* [CERN](https://docs.victoriametrics.com/CaseStudies.html#cern)
* [COLOPL](https://docs.victoriametrics.com/CaseStudies.html#colopl)
* [Dreamteam](https://docs.victoriametrics.com/CaseStudies.html#dreamteam)
* [Fly.io](https://docs.victoriametrics.com/CaseStudies.html#flyio)
* [German Research Center for Artificial Intelligence](https://docs.victoriametrics.com/CaseStudies.html#german-research-center-for-artificial-intelligence)
* [Grammarly](https://docs.victoriametrics.com/CaseStudies.html#grammarly)
* [Groove X](https://docs.victoriametrics.com/CaseStudies.html#groove-x)
* [Idealo.de](https://docs.victoriametrics.com/CaseStudies.html#idealode)
* [MHI Vestas Offshore Wind](https://docs.victoriametrics.com/CaseStudies.html#mhi-vestas-offshore-wind)
* [Razorpay](https://docs.victoriametrics.com/CaseStudies.html#razorpay)
* [Percona](https://docs.victoriametrics.com/CaseStudies.html#percona)
* [Sensedia](https://docs.victoriametrics.com/CaseStudies.html#sensedia)
* [Smarkets](https://docs.victoriametrics.com/CaseStudies.html#smarkets)
* [Synthesio](https://docs.victoriametrics.com/CaseStudies.html#synthesio)
* [Wedos.com](https://docs.victoriametrics.com/CaseStudies.html#wedoscom)
* [Wix.com](https://docs.victoriametrics.com/CaseStudies.html#wixcom)
* [Zerodha](https://docs.victoriametrics.com/CaseStudies.html#zerodha)
* [zhihu](https://docs.victoriametrics.com/CaseStudies.html#zhihu)
See also [articles and slides about VictoriaMetrics from our users](https://docs.victoriametrics.com/Articles.html#third-party-articles-and-slides-about-victoriametrics)
[Contact us](mailto:info@victoriametrics.com) if you need enterprise support for VictoriaMetrics. See [features available in enterprise package](https://victoriametrics.com/enterprise.html). Enterprise binaries can be downloaded and evaluated for free from [the releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
## Prominent features
@@ -95,6 +62,37 @@ VictoriaMetrics has the following prominent features:
See also [various Articles about VictoriaMetrics](https://docs.victoriametrics.com/Articles.html).
## Case studies and talks
Case studies:
* [AbiosGaming](https://docs.victoriametrics.com/CaseStudies.html#abiosgaming)
* [adidas](https://docs.victoriametrics.com/CaseStudies.html#adidas)
* [Adsterra](https://docs.victoriametrics.com/CaseStudies.html#adsterra)
* [ARNES](https://docs.victoriametrics.com/CaseStudies.html#arnes)
* [Brandwatch](https://docs.victoriametrics.com/CaseStudies.html#brandwatch)
* [CERN](https://docs.victoriametrics.com/CaseStudies.html#cern)
* [COLOPL](https://docs.victoriametrics.com/CaseStudies.html#colopl)
* [Dreamteam](https://docs.victoriametrics.com/CaseStudies.html#dreamteam)
* [Fly.io](https://docs.victoriametrics.com/CaseStudies.html#flyio)
* [German Research Center for Artificial Intelligence](https://docs.victoriametrics.com/CaseStudies.html#german-research-center-for-artificial-intelligence)
* [Grammarly](https://docs.victoriametrics.com/CaseStudies.html#grammarly)
* [Groove X](https://docs.victoriametrics.com/CaseStudies.html#groove-x)
* [Idealo.de](https://docs.victoriametrics.com/CaseStudies.html#idealode)
* [MHI Vestas Offshore Wind](https://docs.victoriametrics.com/CaseStudies.html#mhi-vestas-offshore-wind)
* [Razorpay](https://docs.victoriametrics.com/CaseStudies.html#razorpay)
* [Percona](https://docs.victoriametrics.com/CaseStudies.html#percona)
* [Sensedia](https://docs.victoriametrics.com/CaseStudies.html#sensedia)
* [Smarkets](https://docs.victoriametrics.com/CaseStudies.html#smarkets)
* [Synthesio](https://docs.victoriametrics.com/CaseStudies.html#synthesio)
* [Wedos.com](https://docs.victoriametrics.com/CaseStudies.html#wedoscom)
* [Wix.com](https://docs.victoriametrics.com/CaseStudies.html#wixcom)
* [Zerodha](https://docs.victoriametrics.com/CaseStudies.html#zerodha)
* [zhihu](https://docs.victoriametrics.com/CaseStudies.html#zhihu)
See also [articles and slides about VictoriaMetrics from our users](https://docs.victoriametrics.com/Articles.html#third-party-articles-and-slides-about-victoriametrics)
## Operation
## How to start VictoriaMetrics
@@ -418,9 +416,15 @@ The `/api/v1/export` endpoint should return the following response:
Data sent to VictoriaMetrics via `Graphite plaintext protocol` may be read via the following APIs:
* [Graphite API](#graphite-api-usage)
* [Prometheus querying API](#prometheus-querying-api-usage). VictoriaMetrics supports `__graphite__` pseudo-label for selecting time series with Graphite-compatible filters in [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html). For example, `{__graphite__="foo.*.bar"}` is equivalent to `{__name__=~"foo[.][^.]*[.]bar"}`, but it works faster and it is easier to use when migrating from Graphite to VictoriaMetrics. VictoriaMetrics also supports [label_graphite_group](https://docs.victoriametrics.com/MetricsQL.html#label_graphite_group) function for extracting the given groups from Graphite metric name.
* [Prometheus querying API](#prometheus-querying-api-usage). See also [selecting Graphite metrics](#selecting-graphite-metrics).
* [go-graphite/carbonapi](https://github.com/go-graphite/carbonapi/blob/main/cmd/carbonapi/carbonapi.example.victoriametrics.yaml)
## Selecting Graphite metrics
VictoriaMetrics supports `__graphite__` pseudo-label for selecting time series with Graphite-compatible filters in [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html). For example, `{__graphite__="foo.*.bar"}` is equivalent to `{__name__=~"foo[.][^.]*[.]bar"}`, but it works faster and it is easier to use when migrating from Graphite to VictoriaMetrics. See [docs for Graphite paths and wildcards](https://graphite.readthedocs.io/en/latest/render_api.html#paths-and-wildcards). VictoriaMetrics also supports [label_graphite_group](https://docs.victoriametrics.com/MetricsQL.html#label_graphite_group) function for extracting the given groups from Graphite metric name.
The `__graphite__` pseudo-label supports e.g. alternate regexp filters such as `(value1|...|valueN)`. They are transparently converted to `{value1,...,valueN}` syntax [used in Graphite](https://graphite.readthedocs.io/en/latest/render_api.html#paths-and-wildcards). This allows using [multi-value template variables in Grafana](https://grafana.com/docs/grafana/latest/variables/formatting-multi-value-variables/) inside `__graphite__` pseudo-label. For example, Grafana expands `{__graphite__=~"foo.($bar).baz"}` into `{__graphite__=~"foo.(x|y).baz"}` if `$bar` template variable contains `x` and `y` values. In this case the query is automatically converted into `{__graphite__=~"foo.{x,y}.baz"}` before execution.
## How to send data from OpenTSDB-compatible agents
VictoriaMetrics supports [telnet put protocol](http://opentsdb.net/docs/build/html/api_telnet/put.html)
@@ -517,9 +521,10 @@ All the Prometheus querying API handlers can be prepended with `/prometheus` pre
### Prometheus querying API enhancements
VictoriaMetrics accepts optional `extra_label=<label_name>=<label_value>` query arg, which can be used for enforcing additional label filters for queries. For example,
`/api/v1/query_range?extra_label=user_id=123&query=<query>` would automatically add `{user_id="123"}` label filter to the given `<query>`. This functionality can be used
for limiting the scope of time series visible to the given tenant. It is expected that the `extra_label` query arg is automatically set by auth proxy sitting
in front of VictoriaMetrics. See [vmauth](https://docs.victoriametrics.com/vmauth.html) and [vmgateway](https://docs.victoriametrics.com/vmgateway.html) as examples of such proxies.
`/api/v1/query_range?extra_label=user_id=123&extra_label=group_id=456&query=<query>` would automatically add `{user_id="123",group_id="456"}` label filters to the given `<query>`. This functionality can be used for limiting the scope of time series visible to the given tenant. It is expected that the `extra_label` query args are automatically set by auth proxy sitting in front of VictoriaMetrics. See [vmauth](https://docs.victoriametrics.com/vmauth.html) and [vmgateway](https://docs.victoriametrics.com/vmgateway.html) as examples of such proxies.
VictoriaMetrics accepts optional `extra_filters[]=series_selector` query arg, which can be used for enforcing arbitrary label filters for queries. For example,
`/api/v1/query_range?extra_filters[]={env=~"prod|staging",user="xyz"}&query=<query>` would automatically add `{env=~"prod|staging",user="xyz"}` label filters to the given `<query>`. This functionality can be used for limiting the scope of time series visible to the given tenant. It is expected that the `extra_filters[]` query args are automatically set by auth proxy sitting in front of VictoriaMetrics. See [vmauth](https://docs.victoriametrics.com/vmauth.html) and [vmgateway](https://docs.victoriametrics.com/vmgateway.html) as examples of such proxies.
VictoriaMetrics accepts relative times in `time`, `start` and `end` query args additionally to unix timestamps and [RFC3339](https://www.ietf.org/rfc/rfc3339.txt).
For example, the following query would return data for the last 30 minutes: `/api/v1/query_range?start=-30m&query=...`.
@@ -556,12 +561,11 @@ VictoriaMetrics supports the following Graphite APIs, which are needed for [Grap
All the Graphite handlers can be pre-pended with `/graphite` prefix. For example, both `/graphite/metrics/find` and `/metrics/find` should work.
VictoriaMetrics accepts optional `extra_label=<label_name>=<label_value>` query arg for all the Graphite APIs. This arg can be used for limiting the scope of time series
visible to the given tenant. It is expected that the `extra_label` query arg is automatically set by auth proxy sitting in front of VictoriaMetrics.
VictoriaMetrics accepts optional query args: `extra_label=<label_name>=<label_value>` and `extra_filters[]=series_selector` query args for all the Graphite APIs. These args can be used for limiting the scope of time series visible to the given tenant. It is expected that the `extra_label` query arg is automatically set by auth proxy sitting in front of VictoriaMetrics. See [vmauth](https://docs.victoriametrics.com/vmauth.html) and [vmgateway](https://docs.victoriametrics.com/vmgateway.html) as examples of such proxies.
[Contact us](mailto:sales@victoriametrics.com) if you need assistance with such a proxy.
VictoriaMetrics supports `__graphite__` pseudo-label for filtering time series with Graphite-compatible filters in [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html).
For example, `{__graphite__="foo.*.bar"}` is equivalent to `{__name__=~"foo[.][^.]*[.]bar"}`, but it works faster and it is easier to use when migrating from Graphite to VictoriaMetrics. See also [label_graphite_group](https://docs.victoriametrics.com/MetricsQL.html#label_graphite_group) function.
VictoriaMetrics supports `__graphite__` pseudo-label for filtering time series with Graphite-compatible filters in [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html). See [these docs](#selecting-graphite-metrics).
### Graphite Render API usage
@@ -612,6 +616,10 @@ Query history can be navigated by holding `Ctrl` (or `Cmd` on MacOS) and pressin
When querying the [backfilled data](https://docs.victoriametrics.com/#backfilling), it may be useful disabling response cache by clicking `Enable cache` checkbox.
VMUI automatically adjusts the interval between datapoints on the graph depending on the horizontal resolution and on the selected time range. The step value can be customized by clickhing `Override step value` checkbox.
VMUI allows investigating correlations between two queries on the same graph. Just click `+Query` button, enter the second query in the newly appeared input field and press `Ctrl+Enter`. Results for both queries should be displayed simultaneously on the same graph. Every query has its own vertical scale, which is displayed on the left and the right side of the graph. Lines for the second query are dashed.
See the [example VMUI at VictoriaMetrics playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/?g0.expr=100%20*%20sum(rate(process_cpu_seconds_total))%20by%20(job)&g0.range_input=1d).
@@ -829,7 +837,7 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data).
The [deduplication](#deduplication) isn't applied for the data exported in CSV. It is expected that the de-duplication is performed during data import.
The [deduplication](#deduplication) is applied for the data exported in CSV by default. It is possible to export raw data without de-duplication by passing `reduce_mem_usage=1` query arg to `/api/v1/export/csv`.
### How to export data in native format
@@ -1025,6 +1033,7 @@ VictoriaMetrics also may scrape Prometheus targets - see [these docs](#how-to-sc
VictoriaMetrics supports Prometheus-compatible relabeling for all the ingested metrics if `-relabelConfig` command-line flag points
to a file containing a list of [relabel_config](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config) entries.
The `-relabelConfig` also can point to http or https url. For example, `-relabelConfig=https://config-server/relabel_config.yml`.
See [this article with relabeling tips and tricks](https://valyala.medium.com/how-to-use-relabeling-in-prometheus-and-victoriametrics-8b90fc22c4b2).
Example contents for `-relabelConfig` file:
@@ -1074,7 +1083,7 @@ It is recommended leaving the following amounts of spare resources:
* 50% of free RAM for reducing the probability of OOM (out of memory) crashes and slowdowns during temporary spikes in workload.
* 50% of spare CPU for reducing the probability of slowdowns during temporary spikes in workload.
* At least 30% of free storage space at the directory pointed by `-storageDataPath` command-line flag.
* At least 30% of free storage space at the directory pointed by `-storageDataPath` command-line flag. See also `-storage.minFreeDiskSpaceBytes` command-line flag description [here](#list-of-command-line-flags).
## High availability
@@ -1162,20 +1171,15 @@ See [these docs](https://docs.victoriametrics.com/guides/guide-vmcluster-multipl
## Downsampling
There is no downsampling support at the moment, but:
[VictoriaMetrics Enterprise](https://victoriametrics.com/enterprise.html) supports multi-level downsampling with `-downsampling.period` command-line flag. For example:
* VictoriaMetrics is optimized for querying big amounts of raw data. See benchmark results for heavy queries
in [this article](https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae).
* VictoriaMetrics has good compression for on-disk data. See [this article](https://medium.com/@valyala/victoriametrics-achieving-better-compression-for-time-series-data-than-gorilla-317bc1f95932)
for details.
* The downsampling doesn't improve query performance on a long time range if the time range contains big number of time series due to [high churn rate](https://docs.victoriametrics.com/FAQ.html#what-is-high-churn-rate). The query performance depends on the number of unique time series on the selected time range, while downsampling doesn't reduce the number of unique time series in the database - it can reduce only the number of samples per each time series.
* `-downsampling.period=30d:5m` instructs VictoriaMetrics to [deduplicate](#deduplication) samples older than 30 days with 5 minutes interval.
These properties reduce the need of downsampling. We plan to implement downsampling in the future.
See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/36) for details.
* `-downsampling.period=30d:5m,180d:1h` instructs VictoriaMetrics to deduplicate samples older than 30 days with 5 minutes interval and to deduplicate samples older than 180 days with 1 hour interval.
It is possible to (ab)use [-dedup.minScrapeInterval](#deduplication) for basic downsampling.
For instance, if interval between the ingested data points is 15s, then `-dedup.minScrapeInterval=5m` will leave
only a single data point out of 20 initial data points per each 5m interval.
Downsampling is applied independently per each time series. It can reduce disk space usage and improve query performance if it is applied to time series with big number of samples per each series. The downsampling doesn't improve query performance if the database contains big number of time series with small number of samples per each series (aka [high churn rate](https://docs.victoriametrics.com/FAQ.html#what-is-high-churn-rate)), since downsampling doesn't reduce the number of time series. So the majority of time is spent on searching for the matching time series.
The downsampling can be evaluated for free by downloading and using enterprise binaries from [the releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
## Multi-tenancy
@@ -1217,7 +1221,8 @@ Consider setting the following command-line flags:
* `-snapshotAuthKey` for protecting `/snapshot*` endpoints. See [how to work with snapshots](#how-to-work-with-snapshots).
* `-forceMergeAuthKey` for protecting `/internal/force_merge` endpoint. See [force merge docs](#forced-merge).
* `-search.resetCacheAuthKey` for protecting `/internal/resetRollupResultCache` endpoint. See [backfilling](#backfilling) for more details.
* `-configAuthKey` for pretecting `/config` endpoint, since it may contain sensitive information such as passwords.
* `-configAuthKey` for protecting `/config` endpoint, since it may contain sensitive information such as passwords.
- `-pprofAuthKey` for protecting `/debug/pprof/*` endpoints, which can be used for [profiling](#profiling).
Explicitly set internal network interface for TCP and UDP ports for data ingestion with Graphite and OpenTSDB formats.
For example, substitute `-graphiteListenAddr=:2003` with `-graphiteListenAddr=<internal_iface_ip>:2003`.
@@ -1372,9 +1377,7 @@ See also more advanced [cardinality limiter in vmagent](https://docs.victoriamet
This prevents from ingesting metrics with too many labels. It is recommended [monitoring](#monitoring) `vm_metrics_with_dropped_labels_total`
metric in order to determine whether `-maxLabelsPerTimeseries` must be adjusted for your workload.
* If you store Graphite metrics like `foo.bar.baz` in VictoriaMetrics, then use `{__graphite__="foo.*.baz"}` syntax for selecting such metrics.
This expression is equivalent to `{__name__=~"foo[.][^.]*[.]baz"}`, but it works faster and it is easier to use when migrating from Graphite.
See also [label_graphite_group](https://docs.victoriametrics.com/MetricsQL.html#label_graphite_group) function, which allows extracting the given groups from Graphite metric names.
* If you store Graphite metrics like `foo.bar.baz` in VictoriaMetrics, then `{__graphite__="foo.*.baz"}` filter can be used for selecting such metrics. See [these docs](#selecting-graphite-metrics) for details.
* VictoriaMetrics ignores `NaN` values during data ingestion.
@@ -1440,6 +1443,18 @@ We also provide [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanag
Enterprise binaries can be downloaded and evaluated for free from [the releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
## Benchmarks
Note, that vendors (including VictoriaMetrics) are often biased when doing such tests. E.g. they try highlighting
the best parts of their product, while highlighting the worst parts of competing products.
So we encourage users and all independent third parties to conduct their becnhmarks for various products
they are evaluating in production and publish the results.
As a reference, please see [benchmarks](https://docs.victoriametrics.com/Articles.html#benchmarks) conducted by
VictoriaMetrics team. Please also see the [helm chart](https://github.com/VictoriaMetrics/benchmark)
for running ingestion benchmarks based on node_exporter metrics.
## Profiling
VictoriaMetrics provides handlers for collecting the following [Go profiles](https://blog.golang.org/profiling-go-programs):
@@ -1493,9 +1508,11 @@ Contact us with any questions regarding VictoriaMetrics at [info@victoriametrics
Feel free asking any questions regarding VictoriaMetrics:
* [slack](https://slack.victoriametrics.com/)
* [linkedin](https://www.linkedin.com/company/victoriametrics/)
* [reddit](https://www.reddit.com/r/VictoriaMetrics/)
* [telegram-en](https://t.me/VictoriaMetrics_en)
* [telegram-ru](https://t.me/VictoriaMetrics_ru1)
* [articles and talks about VictoriaMetrics in Russian](https://github.com/denisgolius/victoriametrics-ru-links)
* [google groups](https://groups.google.com/forum/#!forum/victorametrics-users)
If you like VictoriaMetrics and want to contribute, then we need the following:
@@ -1565,11 +1582,14 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
The maximum size in bytes of a single DataDog POST request to /api/v1/series
Supports the following optional suffixes for size values: KB, MB, GB, KiB, MiB, GiB (default 67108864)
-dedup.minScrapeInterval duration
Leave only the first sample in every time series per each discrete interval equal to -dedup.minScrapeInterval > 0. See https://docs.victoriametrics.com/#deduplication for details
Leave only the first sample in every time series per each discrete interval equal to -dedup.minScrapeInterval > 0. See https://docs.victoriametrics.com/#deduplication and https://docs.victoriametrics.com/#downsampling
-deleteAuthKey string
authKey for metrics' deletion via /api/v1/admin/tsdb/delete_series and /tags/delSeries
-denyQueriesOutsideRetention
Whether to deny queries outside of the configured -retentionPeriod. When set, then /api/v1/query_range would return '503 Service Unavailable' error for queries with 'from' value outside -retentionPeriod. This may be useful when multiple data sources with distinct retentions are hidden behind query-tee
-downsampling.period array
Comma-separated downsampling periods in the format 'offset:period'. For example, '30d:10m' instructs to leave a single sample per 10 minutes for samples older than 30 days. See https://docs.victoriametrics.com/#downsampling for details
Supports an array of values separated by comma or specified via multiple flags.
-dryRun
Whether to check only -promscrape.config and then exit. Unknown config entries are allowed in -promscrape.config by default. This can be changed with -promscrape.config.strictParse
-enableTCP6
@@ -1578,6 +1598,8 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
Whether to enable reading flags from environment variables additionally to command line. Command line flag values have priority over values from environment vars. Flags are read only from command line if this flag isn't set. See https://docs.victoriametrics.com/#environment-variables for more details
-envflag.prefix string
Prefix for environment variables if -envflag.enable is set
-eula
By specifying this flag, you confirm that you have an enterprise license and accept the EULA https://victoriametrics.com/assets/VM_EULA.pdf
-finalMergeDelay duration
The delay before starting final merge for per-month partition after no new data is ingested into it. Final merge may require additional disk IO and CPU resources. Final merge may increase query speed and reduce disk space usage in some cases. Zero value disables final merge
-forceFlushAuthKey string
@@ -1650,8 +1672,10 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
-maxInsertRequestSize size
The maximum size in bytes of a single Prometheus remote_write API request
Supports the following optional suffixes for size values: KB, MB, GB, KiB, MiB, GiB (default 33554432)
-maxLabelValueLen int
The maximum length of label values in the accepted time series. Longer label values are truncated. In this case the vm_too_long_label_values_total metric at /metrics page is incremented (default 16384)
-maxLabelsPerTimeseries int
The maximum number of labels accepted per time series. Superfluous labels are dropped (default 30)
The maximum number of labels accepted per time series. Superfluous labels are dropped. In this case the vm_metrics_with_dropped_labels_total metric at /metrics page is incremented (default 30)
-memory.allowedBytes size
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from OS page cache resulting in higher disk IO usage
Supports the following optional suffixes for size values: KB, MB, GB, KiB, MiB, GiB (default 0)
@@ -1681,7 +1705,7 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
-promscrape.cluster.replicationFactor int
The number of members in the cluster, which scrape the same targets. If the replication factor is greater than 2, then the deduplication must be enabled at remote storage side. See https://docs.victoriametrics.com/#deduplication (default 1)
-promscrape.config string
Optional path to Prometheus config file with 'scrape_configs' section containing targets to scrape. See https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter for details
Optional path to Prometheus config file with 'scrape_configs' section containing targets to scrape. The path can point to local file and to http url. See https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter for details
-promscrape.config.dryRun
Checks -promscrape.config file for errors and unsupported fields and then exits. Returns non-zero exit code on parsing errors and emits these errors to stderr. See also -promscrape.config.strictParse command-line flag. Pass -loggerLevel=ERROR if you don't need to see info messages in the output.
-promscrape.config.strictParse
@@ -1748,7 +1772,7 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
-promscrape.suppressScrapeErrors
Whether to suppress scrape errors logging. The last error for each target is always available at '/targets' page even if scrape errors logging is suppressed
-relabelConfig string
Optional path to a file with relabeling rules, which are applied to all the ingested metrics. See https://docs.victoriametrics.com/#relabeling for details. The config is reloaded on SIGHUP signal
Optional path to a file with relabeling rules, which are applied to all the ingested metrics. The path can point either to local file or to http url. See https://docs.victoriametrics.com/#relabeling for details. The config is reloaded on SIGHUP signal
-relabelDebug
Whether to log metrics before and after relabeling with -relabelConfig. If the -relabelDebug is enabled, then the metrics aren't sent to storage. This is useful for debugging the relabeling configs
-retentionPeriod value
@@ -1760,6 +1784,10 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
Whether to disable automatic response cache reset if a sample with timestamp outside -search.cacheTimestampOffset is inserted into VictoriaMetrics
-search.disableCache
Whether to disable response caching. This may be useful during data backfilling
-search.graphiteMaxPointsPerSeries int
The maximum number of points per series Graphite render API can return (default 1000000)
-search.graphiteStorageStep duration
The interval between datapoints stored in the database. It is used at Graphite Render API handler for normalizing the interval between datapoints in case it isn't normalized. It can be overriden by sending 'storage_step' query arg to /render API or by sending the desired interval via 'Storage-Step' http header during querying /render API (default 10s)
-search.latencyOffset duration
The time when data points become visible in query results after the collection. Too small value can result in incomplete last points for query results (default 30s)
-search.logSlowQueryDuration duration

View File

@@ -31,10 +31,12 @@ sort: 17
### Public Announcement
1. Publish message in slack (victoriametrics.slack.com, general channel)
2. Post twit with release notes URL
3. Post in subreddit https://www.reddit.com/r/VictoriaMetrics/
4. Post in linkedin
- Publish message in Slack at https://victoriametrics.slack.com
- Post at Twitter at https://twitter.com/MetricsVictoria
- Post in Reddit at https://www.reddit.com/r/VictoriaMetrics/
- Post in Linkedin at https://www.linkedin.com/company/victoriametrics/
- Publish message in Telegram at https://t.me/VictoriaMetrics_en and https://t.me/VictoriaMetrics_ru1
- Publish message in google groups at https://groups.google.com/forum/#!forum/victorametrics-users
## Helm Charts

View File

@@ -17,46 +17,13 @@ sort: 1
VictoriaMetrics is a fast, cost-effective and scalable monitoring solution and time series database.
VictoriaMetrics is available in [binary releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases),
in [Docker images](https://hub.docker.com/r/victoriametrics/victoria-metrics/), in [Snap packages](https://snapcraft.io/victoriametrics)
and in [source code](https://github.com/VictoriaMetrics/VictoriaMetrics). Just download VictoriaMetrics follow [these instructions](#how-to-start-victoriametrics).
[Docker images](https://hub.docker.com/r/victoriametrics/victoria-metrics/), [Snap packages](https://snapcraft.io/victoriametrics)
and [source code](https://github.com/VictoriaMetrics/VictoriaMetrics). Just download VictoriaMetrics and follow [these instructions](#how-to-start-victoriametrics).
Then read [Prometheus setup](#prometheus-setup) and [Grafana setup](#grafana-setup) docs.
Cluster version of VictoriaMetrics is available [here](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html).
[Contact us](mailto:info@victoriametrics.com) if you need enterprise support for VictoriaMetrics.
See [features available in enterprise package](https://victoriametrics.com/enterprise.html).
Enterprise binaries can be downloaded and evaluated for free from [the releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
## Case studies and talks
Case studies:
* [AbiosGaming](https://docs.victoriametrics.com/CaseStudies.html#abiosgaming)
* [adidas](https://docs.victoriametrics.com/CaseStudies.html#adidas)
* [Adsterra](https://docs.victoriametrics.com/CaseStudies.html#adsterra)
* [ARNES](https://docs.victoriametrics.com/CaseStudies.html#arnes)
* [Brandwatch](https://docs.victoriametrics.com/CaseStudies.html#brandwatch)
* [CERN](https://docs.victoriametrics.com/CaseStudies.html#cern)
* [COLOPL](https://docs.victoriametrics.com/CaseStudies.html#colopl)
* [Dreamteam](https://docs.victoriametrics.com/CaseStudies.html#dreamteam)
* [Fly.io](https://docs.victoriametrics.com/CaseStudies.html#flyio)
* [German Research Center for Artificial Intelligence](https://docs.victoriametrics.com/CaseStudies.html#german-research-center-for-artificial-intelligence)
* [Grammarly](https://docs.victoriametrics.com/CaseStudies.html#grammarly)
* [Groove X](https://docs.victoriametrics.com/CaseStudies.html#groove-x)
* [Idealo.de](https://docs.victoriametrics.com/CaseStudies.html#idealode)
* [MHI Vestas Offshore Wind](https://docs.victoriametrics.com/CaseStudies.html#mhi-vestas-offshore-wind)
* [Razorpay](https://docs.victoriametrics.com/CaseStudies.html#razorpay)
* [Percona](https://docs.victoriametrics.com/CaseStudies.html#percona)
* [Sensedia](https://docs.victoriametrics.com/CaseStudies.html#sensedia)
* [Smarkets](https://docs.victoriametrics.com/CaseStudies.html#smarkets)
* [Synthesio](https://docs.victoriametrics.com/CaseStudies.html#synthesio)
* [Wedos.com](https://docs.victoriametrics.com/CaseStudies.html#wedoscom)
* [Wix.com](https://docs.victoriametrics.com/CaseStudies.html#wixcom)
* [Zerodha](https://docs.victoriametrics.com/CaseStudies.html#zerodha)
* [zhihu](https://docs.victoriametrics.com/CaseStudies.html#zhihu)
See also [articles and slides about VictoriaMetrics from our users](https://docs.victoriametrics.com/Articles.html#third-party-articles-and-slides-about-victoriametrics)
[Contact us](mailto:info@victoriametrics.com) if you need enterprise support for VictoriaMetrics. See [features available in enterprise package](https://victoriametrics.com/enterprise.html). Enterprise binaries can be downloaded and evaluated for free from [the releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
## Prominent features
@@ -99,6 +66,37 @@ VictoriaMetrics has the following prominent features:
See also [various Articles about VictoriaMetrics](https://docs.victoriametrics.com/Articles.html).
## Case studies and talks
Case studies:
* [AbiosGaming](https://docs.victoriametrics.com/CaseStudies.html#abiosgaming)
* [adidas](https://docs.victoriametrics.com/CaseStudies.html#adidas)
* [Adsterra](https://docs.victoriametrics.com/CaseStudies.html#adsterra)
* [ARNES](https://docs.victoriametrics.com/CaseStudies.html#arnes)
* [Brandwatch](https://docs.victoriametrics.com/CaseStudies.html#brandwatch)
* [CERN](https://docs.victoriametrics.com/CaseStudies.html#cern)
* [COLOPL](https://docs.victoriametrics.com/CaseStudies.html#colopl)
* [Dreamteam](https://docs.victoriametrics.com/CaseStudies.html#dreamteam)
* [Fly.io](https://docs.victoriametrics.com/CaseStudies.html#flyio)
* [German Research Center for Artificial Intelligence](https://docs.victoriametrics.com/CaseStudies.html#german-research-center-for-artificial-intelligence)
* [Grammarly](https://docs.victoriametrics.com/CaseStudies.html#grammarly)
* [Groove X](https://docs.victoriametrics.com/CaseStudies.html#groove-x)
* [Idealo.de](https://docs.victoriametrics.com/CaseStudies.html#idealode)
* [MHI Vestas Offshore Wind](https://docs.victoriametrics.com/CaseStudies.html#mhi-vestas-offshore-wind)
* [Razorpay](https://docs.victoriametrics.com/CaseStudies.html#razorpay)
* [Percona](https://docs.victoriametrics.com/CaseStudies.html#percona)
* [Sensedia](https://docs.victoriametrics.com/CaseStudies.html#sensedia)
* [Smarkets](https://docs.victoriametrics.com/CaseStudies.html#smarkets)
* [Synthesio](https://docs.victoriametrics.com/CaseStudies.html#synthesio)
* [Wedos.com](https://docs.victoriametrics.com/CaseStudies.html#wedoscom)
* [Wix.com](https://docs.victoriametrics.com/CaseStudies.html#wixcom)
* [Zerodha](https://docs.victoriametrics.com/CaseStudies.html#zerodha)
* [zhihu](https://docs.victoriametrics.com/CaseStudies.html#zhihu)
See also [articles and slides about VictoriaMetrics from our users](https://docs.victoriametrics.com/Articles.html#third-party-articles-and-slides-about-victoriametrics)
## Operation
## How to start VictoriaMetrics
@@ -422,9 +420,15 @@ The `/api/v1/export` endpoint should return the following response:
Data sent to VictoriaMetrics via `Graphite plaintext protocol` may be read via the following APIs:
* [Graphite API](#graphite-api-usage)
* [Prometheus querying API](#prometheus-querying-api-usage). VictoriaMetrics supports `__graphite__` pseudo-label for selecting time series with Graphite-compatible filters in [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html). For example, `{__graphite__="foo.*.bar"}` is equivalent to `{__name__=~"foo[.][^.]*[.]bar"}`, but it works faster and it is easier to use when migrating from Graphite to VictoriaMetrics. VictoriaMetrics also supports [label_graphite_group](https://docs.victoriametrics.com/MetricsQL.html#label_graphite_group) function for extracting the given groups from Graphite metric name.
* [Prometheus querying API](#prometheus-querying-api-usage). See also [selecting Graphite metrics](#selecting-graphite-metrics).
* [go-graphite/carbonapi](https://github.com/go-graphite/carbonapi/blob/main/cmd/carbonapi/carbonapi.example.victoriametrics.yaml)
## Selecting Graphite metrics
VictoriaMetrics supports `__graphite__` pseudo-label for selecting time series with Graphite-compatible filters in [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html). For example, `{__graphite__="foo.*.bar"}` is equivalent to `{__name__=~"foo[.][^.]*[.]bar"}`, but it works faster and it is easier to use when migrating from Graphite to VictoriaMetrics. See [docs for Graphite paths and wildcards](https://graphite.readthedocs.io/en/latest/render_api.html#paths-and-wildcards). VictoriaMetrics also supports [label_graphite_group](https://docs.victoriametrics.com/MetricsQL.html#label_graphite_group) function for extracting the given groups from Graphite metric name.
The `__graphite__` pseudo-label supports e.g. alternate regexp filters such as `(value1|...|valueN)`. They are transparently converted to `{value1,...,valueN}` syntax [used in Graphite](https://graphite.readthedocs.io/en/latest/render_api.html#paths-and-wildcards). This allows using [multi-value template variables in Grafana](https://grafana.com/docs/grafana/latest/variables/formatting-multi-value-variables/) inside `__graphite__` pseudo-label. For example, Grafana expands `{__graphite__=~"foo.($bar).baz"}` into `{__graphite__=~"foo.(x|y).baz"}` if `$bar` template variable contains `x` and `y` values. In this case the query is automatically converted into `{__graphite__=~"foo.{x,y}.baz"}` before execution.
## How to send data from OpenTSDB-compatible agents
VictoriaMetrics supports [telnet put protocol](http://opentsdb.net/docs/build/html/api_telnet/put.html)
@@ -521,9 +525,10 @@ All the Prometheus querying API handlers can be prepended with `/prometheus` pre
### Prometheus querying API enhancements
VictoriaMetrics accepts optional `extra_label=<label_name>=<label_value>` query arg, which can be used for enforcing additional label filters for queries. For example,
`/api/v1/query_range?extra_label=user_id=123&query=<query>` would automatically add `{user_id="123"}` label filter to the given `<query>`. This functionality can be used
for limiting the scope of time series visible to the given tenant. It is expected that the `extra_label` query arg is automatically set by auth proxy sitting
in front of VictoriaMetrics. See [vmauth](https://docs.victoriametrics.com/vmauth.html) and [vmgateway](https://docs.victoriametrics.com/vmgateway.html) as examples of such proxies.
`/api/v1/query_range?extra_label=user_id=123&extra_label=group_id=456&query=<query>` would automatically add `{user_id="123",group_id="456"}` label filters to the given `<query>`. This functionality can be used for limiting the scope of time series visible to the given tenant. It is expected that the `extra_label` query args are automatically set by auth proxy sitting in front of VictoriaMetrics. See [vmauth](https://docs.victoriametrics.com/vmauth.html) and [vmgateway](https://docs.victoriametrics.com/vmgateway.html) as examples of such proxies.
VictoriaMetrics accepts optional `extra_filters[]=series_selector` query arg, which can be used for enforcing arbitrary label filters for queries. For example,
`/api/v1/query_range?extra_filters[]={env=~"prod|staging",user="xyz"}&query=<query>` would automatically add `{env=~"prod|staging",user="xyz"}` label filters to the given `<query>`. This functionality can be used for limiting the scope of time series visible to the given tenant. It is expected that the `extra_filters[]` query args are automatically set by auth proxy sitting in front of VictoriaMetrics. See [vmauth](https://docs.victoriametrics.com/vmauth.html) and [vmgateway](https://docs.victoriametrics.com/vmgateway.html) as examples of such proxies.
VictoriaMetrics accepts relative times in `time`, `start` and `end` query args additionally to unix timestamps and [RFC3339](https://www.ietf.org/rfc/rfc3339.txt).
For example, the following query would return data for the last 30 minutes: `/api/v1/query_range?start=-30m&query=...`.
@@ -560,12 +565,11 @@ VictoriaMetrics supports the following Graphite APIs, which are needed for [Grap
All the Graphite handlers can be pre-pended with `/graphite` prefix. For example, both `/graphite/metrics/find` and `/metrics/find` should work.
VictoriaMetrics accepts optional `extra_label=<label_name>=<label_value>` query arg for all the Graphite APIs. This arg can be used for limiting the scope of time series
visible to the given tenant. It is expected that the `extra_label` query arg is automatically set by auth proxy sitting in front of VictoriaMetrics.
VictoriaMetrics accepts optional query args: `extra_label=<label_name>=<label_value>` and `extra_filters[]=series_selector` query args for all the Graphite APIs. These args can be used for limiting the scope of time series visible to the given tenant. It is expected that the `extra_label` query arg is automatically set by auth proxy sitting in front of VictoriaMetrics. See [vmauth](https://docs.victoriametrics.com/vmauth.html) and [vmgateway](https://docs.victoriametrics.com/vmgateway.html) as examples of such proxies.
[Contact us](mailto:sales@victoriametrics.com) if you need assistance with such a proxy.
VictoriaMetrics supports `__graphite__` pseudo-label for filtering time series with Graphite-compatible filters in [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html).
For example, `{__graphite__="foo.*.bar"}` is equivalent to `{__name__=~"foo[.][^.]*[.]bar"}`, but it works faster and it is easier to use when migrating from Graphite to VictoriaMetrics. See also [label_graphite_group](https://docs.victoriametrics.com/MetricsQL.html#label_graphite_group) function.
VictoriaMetrics supports `__graphite__` pseudo-label for filtering time series with Graphite-compatible filters in [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html). See [these docs](#selecting-graphite-metrics).
### Graphite Render API usage
@@ -616,6 +620,10 @@ Query history can be navigated by holding `Ctrl` (or `Cmd` on MacOS) and pressin
When querying the [backfilled data](https://docs.victoriametrics.com/#backfilling), it may be useful disabling response cache by clicking `Enable cache` checkbox.
VMUI automatically adjusts the interval between datapoints on the graph depending on the horizontal resolution and on the selected time range. The step value can be customized by clickhing `Override step value` checkbox.
VMUI allows investigating correlations between two queries on the same graph. Just click `+Query` button, enter the second query in the newly appeared input field and press `Ctrl+Enter`. Results for both queries should be displayed simultaneously on the same graph. Every query has its own vertical scale, which is displayed on the left and the right side of the graph. Lines for the second query are dashed.
See the [example VMUI at VictoriaMetrics playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/?g0.expr=100%20*%20sum(rate(process_cpu_seconds_total))%20by%20(job)&g0.range_input=1d).
@@ -833,7 +841,7 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data).
The [deduplication](#deduplication) isn't applied for the data exported in CSV. It is expected that the de-duplication is performed during data import.
The [deduplication](#deduplication) is applied for the data exported in CSV by default. It is possible to export raw data without de-duplication by passing `reduce_mem_usage=1` query arg to `/api/v1/export/csv`.
### How to export data in native format
@@ -1029,6 +1037,7 @@ VictoriaMetrics also may scrape Prometheus targets - see [these docs](#how-to-sc
VictoriaMetrics supports Prometheus-compatible relabeling for all the ingested metrics if `-relabelConfig` command-line flag points
to a file containing a list of [relabel_config](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config) entries.
The `-relabelConfig` also can point to http or https url. For example, `-relabelConfig=https://config-server/relabel_config.yml`.
See [this article with relabeling tips and tricks](https://valyala.medium.com/how-to-use-relabeling-in-prometheus-and-victoriametrics-8b90fc22c4b2).
Example contents for `-relabelConfig` file:
@@ -1078,7 +1087,7 @@ It is recommended leaving the following amounts of spare resources:
* 50% of free RAM for reducing the probability of OOM (out of memory) crashes and slowdowns during temporary spikes in workload.
* 50% of spare CPU for reducing the probability of slowdowns during temporary spikes in workload.
* At least 30% of free storage space at the directory pointed by `-storageDataPath` command-line flag.
* At least 30% of free storage space at the directory pointed by `-storageDataPath` command-line flag. See also `-storage.minFreeDiskSpaceBytes` command-line flag description [here](#list-of-command-line-flags).
## High availability
@@ -1166,20 +1175,15 @@ See [these docs](https://docs.victoriametrics.com/guides/guide-vmcluster-multipl
## Downsampling
There is no downsampling support at the moment, but:
[VictoriaMetrics Enterprise](https://victoriametrics.com/enterprise.html) supports multi-level downsampling with `-downsampling.period` command-line flag. For example:
* VictoriaMetrics is optimized for querying big amounts of raw data. See benchmark results for heavy queries
in [this article](https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae).
* VictoriaMetrics has good compression for on-disk data. See [this article](https://medium.com/@valyala/victoriametrics-achieving-better-compression-for-time-series-data-than-gorilla-317bc1f95932)
for details.
* The downsampling doesn't improve query performance on a long time range if the time range contains big number of time series due to [high churn rate](https://docs.victoriametrics.com/FAQ.html#what-is-high-churn-rate). The query performance depends on the number of unique time series on the selected time range, while downsampling doesn't reduce the number of unique time series in the database - it can reduce only the number of samples per each time series.
* `-downsampling.period=30d:5m` instructs VictoriaMetrics to [deduplicate](#deduplication) samples older than 30 days with 5 minutes interval.
These properties reduce the need of downsampling. We plan to implement downsampling in the future.
See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/36) for details.
* `-downsampling.period=30d:5m,180d:1h` instructs VictoriaMetrics to deduplicate samples older than 30 days with 5 minutes interval and to deduplicate samples older than 180 days with 1 hour interval.
It is possible to (ab)use [-dedup.minScrapeInterval](#deduplication) for basic downsampling.
For instance, if interval between the ingested data points is 15s, then `-dedup.minScrapeInterval=5m` will leave
only a single data point out of 20 initial data points per each 5m interval.
Downsampling is applied independently per each time series. It can reduce disk space usage and improve query performance if it is applied to time series with big number of samples per each series. The downsampling doesn't improve query performance if the database contains big number of time series with small number of samples per each series (aka [high churn rate](https://docs.victoriametrics.com/FAQ.html#what-is-high-churn-rate)), since downsampling doesn't reduce the number of time series. So the majority of time is spent on searching for the matching time series.
The downsampling can be evaluated for free by downloading and using enterprise binaries from [the releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
## Multi-tenancy
@@ -1221,7 +1225,8 @@ Consider setting the following command-line flags:
* `-snapshotAuthKey` for protecting `/snapshot*` endpoints. See [how to work with snapshots](#how-to-work-with-snapshots).
* `-forceMergeAuthKey` for protecting `/internal/force_merge` endpoint. See [force merge docs](#forced-merge).
* `-search.resetCacheAuthKey` for protecting `/internal/resetRollupResultCache` endpoint. See [backfilling](#backfilling) for more details.
* `-configAuthKey` for pretecting `/config` endpoint, since it may contain sensitive information such as passwords.
* `-configAuthKey` for protecting `/config` endpoint, since it may contain sensitive information such as passwords.
- `-pprofAuthKey` for protecting `/debug/pprof/*` endpoints, which can be used for [profiling](#profiling).
Explicitly set internal network interface for TCP and UDP ports for data ingestion with Graphite and OpenTSDB formats.
For example, substitute `-graphiteListenAddr=:2003` with `-graphiteListenAddr=<internal_iface_ip>:2003`.
@@ -1376,9 +1381,7 @@ See also more advanced [cardinality limiter in vmagent](https://docs.victoriamet
This prevents from ingesting metrics with too many labels. It is recommended [monitoring](#monitoring) `vm_metrics_with_dropped_labels_total`
metric in order to determine whether `-maxLabelsPerTimeseries` must be adjusted for your workload.
* If you store Graphite metrics like `foo.bar.baz` in VictoriaMetrics, then use `{__graphite__="foo.*.baz"}` syntax for selecting such metrics.
This expression is equivalent to `{__name__=~"foo[.][^.]*[.]baz"}`, but it works faster and it is easier to use when migrating from Graphite.
See also [label_graphite_group](https://docs.victoriametrics.com/MetricsQL.html#label_graphite_group) function, which allows extracting the given groups from Graphite metric names.
* If you store Graphite metrics like `foo.bar.baz` in VictoriaMetrics, then `{__graphite__="foo.*.baz"}` filter can be used for selecting such metrics. See [these docs](#selecting-graphite-metrics) for details.
* VictoriaMetrics ignores `NaN` values during data ingestion.
@@ -1444,6 +1447,18 @@ We also provide [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanag
Enterprise binaries can be downloaded and evaluated for free from [the releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
## Benchmarks
Note, that vendors (including VictoriaMetrics) are often biased when doing such tests. E.g. they try highlighting
the best parts of their product, while highlighting the worst parts of competing products.
So we encourage users and all independent third parties to conduct their becnhmarks for various products
they are evaluating in production and publish the results.
As a reference, please see [benchmarks](https://docs.victoriametrics.com/Articles.html#benchmarks) conducted by
VictoriaMetrics team. Please also see the [helm chart](https://github.com/VictoriaMetrics/benchmark)
for running ingestion benchmarks based on node_exporter metrics.
## Profiling
VictoriaMetrics provides handlers for collecting the following [Go profiles](https://blog.golang.org/profiling-go-programs):
@@ -1497,9 +1512,11 @@ Contact us with any questions regarding VictoriaMetrics at [info@victoriametrics
Feel free asking any questions regarding VictoriaMetrics:
* [slack](https://slack.victoriametrics.com/)
* [linkedin](https://www.linkedin.com/company/victoriametrics/)
* [reddit](https://www.reddit.com/r/VictoriaMetrics/)
* [telegram-en](https://t.me/VictoriaMetrics_en)
* [telegram-ru](https://t.me/VictoriaMetrics_ru1)
* [articles and talks about VictoriaMetrics in Russian](https://github.com/denisgolius/victoriametrics-ru-links)
* [google groups](https://groups.google.com/forum/#!forum/victorametrics-users)
If you like VictoriaMetrics and want to contribute, then we need the following:
@@ -1569,11 +1586,14 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
The maximum size in bytes of a single DataDog POST request to /api/v1/series
Supports the following optional suffixes for size values: KB, MB, GB, KiB, MiB, GiB (default 67108864)
-dedup.minScrapeInterval duration
Leave only the first sample in every time series per each discrete interval equal to -dedup.minScrapeInterval > 0. See https://docs.victoriametrics.com/#deduplication for details
Leave only the first sample in every time series per each discrete interval equal to -dedup.minScrapeInterval > 0. See https://docs.victoriametrics.com/#deduplication and https://docs.victoriametrics.com/#downsampling
-deleteAuthKey string
authKey for metrics' deletion via /api/v1/admin/tsdb/delete_series and /tags/delSeries
-denyQueriesOutsideRetention
Whether to deny queries outside of the configured -retentionPeriod. When set, then /api/v1/query_range would return '503 Service Unavailable' error for queries with 'from' value outside -retentionPeriod. This may be useful when multiple data sources with distinct retentions are hidden behind query-tee
-downsampling.period array
Comma-separated downsampling periods in the format 'offset:period'. For example, '30d:10m' instructs to leave a single sample per 10 minutes for samples older than 30 days. See https://docs.victoriametrics.com/#downsampling for details
Supports an array of values separated by comma or specified via multiple flags.
-dryRun
Whether to check only -promscrape.config and then exit. Unknown config entries are allowed in -promscrape.config by default. This can be changed with -promscrape.config.strictParse
-enableTCP6
@@ -1582,6 +1602,8 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
Whether to enable reading flags from environment variables additionally to command line. Command line flag values have priority over values from environment vars. Flags are read only from command line if this flag isn't set. See https://docs.victoriametrics.com/#environment-variables for more details
-envflag.prefix string
Prefix for environment variables if -envflag.enable is set
-eula
By specifying this flag, you confirm that you have an enterprise license and accept the EULA https://victoriametrics.com/assets/VM_EULA.pdf
-finalMergeDelay duration
The delay before starting final merge for per-month partition after no new data is ingested into it. Final merge may require additional disk IO and CPU resources. Final merge may increase query speed and reduce disk space usage in some cases. Zero value disables final merge
-forceFlushAuthKey string
@@ -1654,8 +1676,10 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
-maxInsertRequestSize size
The maximum size in bytes of a single Prometheus remote_write API request
Supports the following optional suffixes for size values: KB, MB, GB, KiB, MiB, GiB (default 33554432)
-maxLabelValueLen int
The maximum length of label values in the accepted time series. Longer label values are truncated. In this case the vm_too_long_label_values_total metric at /metrics page is incremented (default 16384)
-maxLabelsPerTimeseries int
The maximum number of labels accepted per time series. Superfluous labels are dropped (default 30)
The maximum number of labels accepted per time series. Superfluous labels are dropped. In this case the vm_metrics_with_dropped_labels_total metric at /metrics page is incremented (default 30)
-memory.allowedBytes size
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from OS page cache resulting in higher disk IO usage
Supports the following optional suffixes for size values: KB, MB, GB, KiB, MiB, GiB (default 0)
@@ -1685,7 +1709,7 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
-promscrape.cluster.replicationFactor int
The number of members in the cluster, which scrape the same targets. If the replication factor is greater than 2, then the deduplication must be enabled at remote storage side. See https://docs.victoriametrics.com/#deduplication (default 1)
-promscrape.config string
Optional path to Prometheus config file with 'scrape_configs' section containing targets to scrape. See https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter for details
Optional path to Prometheus config file with 'scrape_configs' section containing targets to scrape. The path can point to local file and to http url. See https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter for details
-promscrape.config.dryRun
Checks -promscrape.config file for errors and unsupported fields and then exits. Returns non-zero exit code on parsing errors and emits these errors to stderr. See also -promscrape.config.strictParse command-line flag. Pass -loggerLevel=ERROR if you don't need to see info messages in the output.
-promscrape.config.strictParse
@@ -1752,7 +1776,7 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
-promscrape.suppressScrapeErrors
Whether to suppress scrape errors logging. The last error for each target is always available at '/targets' page even if scrape errors logging is suppressed
-relabelConfig string
Optional path to a file with relabeling rules, which are applied to all the ingested metrics. See https://docs.victoriametrics.com/#relabeling for details. The config is reloaded on SIGHUP signal
Optional path to a file with relabeling rules, which are applied to all the ingested metrics. The path can point either to local file or to http url. See https://docs.victoriametrics.com/#relabeling for details. The config is reloaded on SIGHUP signal
-relabelDebug
Whether to log metrics before and after relabeling with -relabelConfig. If the -relabelDebug is enabled, then the metrics aren't sent to storage. This is useful for debugging the relabeling configs
-retentionPeriod value
@@ -1764,6 +1788,10 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
Whether to disable automatic response cache reset if a sample with timestamp outside -search.cacheTimestampOffset is inserted into VictoriaMetrics
-search.disableCache
Whether to disable response caching. This may be useful during data backfilling
-search.graphiteMaxPointsPerSeries int
The maximum number of points per series Graphite render API can return (default 1000000)
-search.graphiteStorageStep duration
The interval between datapoints stored in the database. It is used at Graphite Render API handler for normalizing the interval between datapoints in case it isn't normalized. It can be overriden by sending 'storage_step' query arg to /render API or by sending the desired interval via 'Storage-Step' http header during querying /render API (default 10s)
-search.latencyOffset duration
The time when data points become visible in query results after the collection. Too small value can result in incomplete last points for query results (default 30s)
-search.logSlowQueryDuration duration

View File

@@ -50,7 +50,7 @@ to `vmagent` such as the ability to push metrics instead of pulling them. We did
Please download `vmutils-*` archive from [releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases), unpack it
and configure the following flags to the `vmagent` binary in order to start scraping Prometheus targets:
* `-promscrape.config` with the path to Prometheus config file (usually located at `/etc/prometheus/prometheus.yml`)
* `-promscrape.config` with the path to Prometheus config file (usually located at `/etc/prometheus/prometheus.yml`). The path can point either to local file or to http url.
* `-remoteWrite.url` with the remote storage endpoint such as VictoriaMetrics, the `-remoteWrite.url` argument can be specified multiple times to replicate data concurrently to an arbitrary number of remote storage systems.
Example command line:
@@ -218,15 +218,16 @@ The file pointed by `-promscrape.config` may contain `%{ENV_VAR}` placeholders w
## Loading scrape configs from multiple files
`vmagent` supports loading scrape configs from multiple files specified in the `scrape_config_files` section of `-promscrape.config` file. For example, the following `-promscrape.config` instructs `vmagent` loading scrape configs from all the `*.yml` files under `configs` directory plus a `single_scrape_config.yml` file:
`vmagent` supports loading scrape configs from multiple files specified in the `scrape_config_files` section of `-promscrape.config` file. For example, the following `-promscrape.config` instructs `vmagent` loading scrape configs from all the `*.yml` files under `configs` directory, from `single_scrape_config.yml` local file and from `https://config-server/scrape_config.yml` url:
```yml
scrape_config_files:
- configs/*.yml
- single_scrape_config.yml
- https://config-server/scrape_config.yml
```
Every referred file can contain arbitrary number of any [supported scrape configs](#how-to-collect-metrics-in-prometheus-format). There is no need in specifying top-level `scrape_configs` section in these files. For example:
Every referred file can contain arbitrary number of [supported scrape configs](#how-to-collect-metrics-in-prometheus-format). There is no need in specifying top-level `scrape_configs` section in these files. For example:
```yml
- job_name: foo
@@ -283,7 +284,7 @@ The relabeling can be defined in the following places:
* At the `scrape_config -> relabel_configs` section in `-promscrape.config` file. This relabeling is applied to target labels. This relabeling can be debugged by passing `relabel_debug: true` option to the corresponding `scrape_config` section. In this case `vmagent` logs target labels before and after the relabeling and then drops the logged target.
* At the `scrape_config -> metric_relabel_configs` section in `-promscrape.config` file. This relabeling is applied to all the scraped metrics in the given `scrape_config`. This relabeling can be debugged by passing `metric_relabel_debug: true` option to the corresponding `scrape_config` section. In this case `vmagent` logs metrics before and after the relabeling and then drops the logged metrics.
* At the `-remoteWrite.relabelConfig` file. This relabeling is aplied to all the collected metrics before sending them to remote storage. This relabeling can be debugged by passing `-remoteWrite.relabelDebug` command-line option to `vmagent`. In this case `vmagent` logs metrics before and after the relabeling and then drops all the logged metrics instead of sending them to remote storage.
* At the `-remoteWrite.relabelConfig` file. This relabeling is applied to all the collected metrics before sending them to remote storage. This relabeling can be debugged by passing `-remoteWrite.relabelDebug` command-line option to `vmagent`. In this case `vmagent` logs metrics before and after the relabeling and then drops all the logged metrics instead of sending them to remote storage.
* At the `-remoteWrite.urlRelabelConfig` files. This relabeling is applied to metrics before sending them to the corresponding `-remoteWrite.url`. This relabeling can be debugged by passing `-remoteWrite.urlRelabelDebug` command-line options to `vmagent`. In this case `vmagent` logs metrics before and after the relabeling and then drops all the logged metrics instead of sending them to the corresponding `-remoteWrite.url`.
You can read more about relabeling in the following articles:
@@ -810,7 +811,7 @@ See the docs at https://docs.victoriametrics.com/vmagent.html .
-promscrape.cluster.replicationFactor int
The number of members in the cluster, which scrape the same targets. If the replication factor is greater than 2, then the deduplication must be enabled at remote storage side. See https://docs.victoriametrics.com/#deduplication (default 1)
-promscrape.config string
Optional path to Prometheus config file with 'scrape_configs' section containing targets to scrape. See https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter for details
Optional path to Prometheus config file with 'scrape_configs' section containing targets to scrape. The path can point to local file and to http url. See https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter for details
-promscrape.config.dryRun
Checks -promscrape.config file for errors and unsupported fields and then exits. Returns non-zero exit code on parsing errors and emits these errors to stderr. See also -promscrape.config.strictParse command-line flag. Pass -loggerLevel=ERROR if you don't need to see info messages in the output.
-promscrape.config.strictParse
@@ -935,7 +936,7 @@ See the docs at https://docs.victoriametrics.com/vmagent.html .
Optional rate limit in bytes per second for data sent to -remoteWrite.url. By default the rate limit is disabled. It can be useful for limiting load on remote storage when big amounts of buffered data is sent after temporary unavailability of the remote storage
Supports array of values separated by comma or specified via multiple flags.
-remoteWrite.relabelConfig string
Optional path to file with relabel_config entries. These entries are applied to all the metrics before sending them to -remoteWrite.url. See https://docs.victoriametrics.com/vmagent.html#relabeling for details
Optional path to file with relabel_config entries. The path can point either to local file or to http url. These entries are applied to all the metrics before sending them to -remoteWrite.url. See https://docs.victoriametrics.com/vmagent.html#relabeling for details
-remoteWrite.relabelDebug
Whether to log metrics before and after relabeling with -remoteWrite.relabelConfig. If the -remoteWrite.relabelDebug is enabled, then the metrics aren't sent to remote storage. This is useful for debugging the relabeling configs
-remoteWrite.roundDigits array
@@ -970,7 +971,7 @@ See the docs at https://docs.victoriametrics.com/vmagent.html .
Remote storage URL to write data to. It must support Prometheus remote_write API. It is recommended using VictoriaMetrics as remote storage. Example url: http://<victoriametrics-host>:8428/api/v1/write . Pass multiple -remoteWrite.url flags in order to replicate data to multiple remote storage systems. See also -remoteWrite.multitenantURL
Supports an array of values separated by comma or specified via multiple flags.
-remoteWrite.urlRelabelConfig array
Optional path to relabel config for the corresponding -remoteWrite.url
Optional path to relabel config for the corresponding -remoteWrite.url. The path can point either to local file or to http url
Supports an array of values separated by comma or specified via multiple flags.
-remoteWrite.urlRelabelDebug array
Whether to log metrics before and after relabeling with -remoteWrite.urlRelabelConfig. If the -remoteWrite.urlRelabelDebug is enabled, then the metrics aren't sent to the corresponding -remoteWrite.url. This is useful for debugging the relabeling configs

View File

@@ -103,12 +103,24 @@ name: <string>
# By default "prometheus" type is used.
[ type: <string> ]
# Optional list of label filters applied to every rule's
# request withing a group. Is compatible only with VM datasource.
# See more details at https://docs.victoriametrics.com#prometheus-querying-api-enhancements
# Warning: DEPRECATED
# Please use `params` instead:
# params:
# extra_label: ["job=nodeexporter", "env=prod"]
extra_filter_labels:
[ <labelname>: <labelvalue> ... ]
# Optional list of HTTP URL parameters
# applied for all rules requests within a group
# For example:
# params:
# nocache: ["1"] # disable caching for vmselect
# denyPartialResponse: ["true"] # fail if one or more vmstorage nodes returned an error
# extra_label: ["env=dev"] # apply additional label filter "env=dev" for all requests
# see more details at https://docs.victoriametrics.com#prometheus-querying-api-enhancements
params:
[ <string>: [<string>, ...]]
# Optional list of labels added to every rule within a group.
# It has priority over the external labels.
# Labels are commonly used for adding environment
@@ -476,6 +488,8 @@ a review to the dashboard.
## Configuration
### Flags
Pass `-help` to `vmalert` in order to see the full list of supported
command-line flags with their descriptions.
@@ -697,12 +711,32 @@ The shortlist of configuration flags is the following:
Show VictoriaMetrics version
```
### Hot config reload
`vmalert` supports "hot" config reload via the following methods:
* send SIGHUP signal to `vmalert` process;
* send GET request to `/-/reload` endpoint;
* configure `-rule.configCheckInterval` flag for periodic reload
on config change.
### URL params
To set additional URL params for `datasource.url`, `remoteWrite.url` or `remoteRead.url`
just add them in address: `-datasource.url=http://localhost:8428?nocache=1`.
To set additional URL params for specific [group of rules](#Groups) modify
the `params` group:
```yaml
groups:
- name: TestGroup
params:
denyPartialResponse: ["true"]
extra_label: ["env=dev"]
```
Please note, `params` are used only for executing rules expressions (requests to `datasource.url`).
If there would be a conflict between URL params set in `datasource.url` flag and params in group definition
the latter will have higher priority.
## Contributing
`vmalert` is mostly designed and built by VictoriaMetrics community.
@@ -718,7 +752,7 @@ It is recommended using
### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.16.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.17.
2. Run `make vmalert` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmalert` binary and puts it into the `bin` folder.
@@ -735,7 +769,7 @@ ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://b
### Development ARM build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.16.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.17.
2. Run `make vmalert-arm` or `make vmalert-arm64` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmalert-arm` or `vmalert-arm64` binary respectively and puts it into the `bin` folder.

Some files were not shown because too many files have changed in this diff Show More