Compare commits

...

62 Commits

Author SHA1 Message Date
Haley Wang
1cc1d45503 vmalert-dashboard: fix stats panels visualization 2025-01-08 22:24:20 +08:00
cuiweiyuan
d064e14933 chore: fix function name in comment (#7926)
### Describe Your Changes

 fix function name in comment

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: cuiweiyuan <cuiweiyuan@aliyun.com>
2025-01-08 13:58:22 +01:00
Afolabi Badmos
77b0fcfdd9 vmauth: fix bug in discovering ipv6 addresses (#7955)
### Describe Your Changes

Fixes error in `vmauth` when discovering ipv6 addresses.

`vmauth` attempts to [slice till
`:`](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/app/vmauth/auth_config.go#L397)
in the discovered addresses without accounting for ipv6. This causes it
to fail in ipv6 only environments.

```sh
$ nslookup vmselect.ns.svc.cluster.local

...
Name: vmselect.ns.svc.cluster.local
Address: 2600:dead:beef:dead:beef::8
```

```sh
$ kubectl logs -f vmauth

...
error: dial tcp: lookup 2600: no such host
```


### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: f41gh7 <nik@victoriametrics.com>
2025-01-08 16:51:13 +04:00
Yury Molodov
ee7fe11fd2 vmui/logs: add autocomplete support for LogsQL (#6949)
### Describe Your Changes

This pull request adds support for autocomplete in LogsQL queries. The
new feature provides suggestions for field names, field values, and pipe
names as you type.


---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2025-01-08 11:36:37 +01:00
Andrii Chubatiuk
4c26fb6fe5 docs: make badges in docs clickable (#7960)
### Describe Your Changes

added links to badges and made them clickable at
docs.victoriametrics.com

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-08 08:58:43 +01:00
Github Actions
fc135094b3 Automatic update operator docs from VictoriaMetrics/operator@028ab8b (#7981)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2025-01-08 07:45:54 +01:00
Roman Khavronenko
5d42f21abd docs: mention publicly available playgrounds (#7977)
The point of the new section is to highlight publicly available
playgrounds for users. All of them were mentioned in other parts of the
documentation, but we didn't have all of them in one place before.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-07 22:16:50 +01:00
hagen1778
28eeabded1 docs: rm extra new lines as they bring no value
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-07 13:52:35 +01:00
hagen1778
b6910cfff7 docs: make vmui related pages below the vmui parent page
This change only updates the hierarchy of pages within the readme.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-07 13:48:39 +01:00
Zhu Jiekun
8938ef398c docs: clarify extra resource is needed for downsampling/retention filter (#7974)
### Describe Your Changes

clarify extra resource is needed when downsampling with filter(s) or
retention filter(s) is applied

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-07 12:30:19 +01:00
hagen1778
df2b75fa81 docs: fix markdown typos
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-07 09:18:25 +01:00
hagen1778
857734c66c docs: fix markdown typo
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-07 09:06:45 +01:00
Github Actions
bedc0c0f8f Automatic update helm docs from VictoriaMetrics/helm-charts@7004727 (#7958)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2025-01-03 19:06:18 +04:00
YuDong Tang
5a41bdf329 app/select: add command-line flag -search.maxBinaryOpPushdownLabelValues
### Describe Your Changes

Binary operations like `exprFirst op exprSecond` in VictoriaMetrics are
performed in the following way:
1. Execute exprFirst.
2. Extract **common label filters** from the result of step 1.
3. Apply these common label filters to `exprSecond` and execute it, in
order to retrieve less time series from vmstorage nodes.

In step 2, only labels with less than `100` (hard-coded) value could be
used as **common label filter** (e.g. `{common_lb=~"v1|v2|...|v100"}`.

In our scenarios, a label, take `instance` label as an example, could
has thousands of candidate values. Regarding bring more pressure to
vmstorage node, it's still beneficial if labels with more than 100
values could be used as filter in `exprSecond`, with enough vmstorage
resources. After adjusting the value from `100` to `10000`, our query
round-trip time drops significantly from 5s to 2s.

This pull request change the hard-coded value into a configurable flag.
2025-01-03 13:20:50 +01:00
Github Actions
bf5d0dd245 Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@7f94969 (#7911) 2025-01-03 14:58:46 +04:00
Github Actions
1cec37b0f5 Automatic update operator docs from VictoriaMetrics/operator@5992757 (#7954)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2025-01-02 18:46:11 +01:00
f41gh7
c40c25b03c docs/changelog: properly mention vminsert changes
storageNode sorting should be BUGFIX, since previously vminsert performed sort and this behaviour was changed.
Also this change only affects OSS version
2025-01-02 17:53:57 +01:00
kiriklo
82badc3dd5 app/vmselect/promql: improve performance of parseCache on systems with many CPU cores
### Describe Your Changes

Parse cache is a pretty simple implementation of cache. It's just a
standard map with mutex.
Map with mutex overall has poor performance, plus when the cache
overflow occurs, the whole cache locks until 1k elements have been
deleted (now it's 10% of 10000 max elements in the cache). To avoid this
bottleneck and improve performance of cache on systems with many CPU
cores but keep it rather simple, we can implement cache with per bucket
locks like it's done in fastcache. The logic and API remain the same. So
now each bucket will have a map with approximately 78 elements (with 128
buckets), and overflow will occur now for each bucket, and only 7
elements need to be deleted.
Because exec_test.go has about 10k lines of code, it's better to move
the cache into a separate file to add tests and benchmarks for it,
because now it does not have them.

```
goos: windows
goarch: amd64
pkg: github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/promql
cpu: 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz

Current cache implementation performance on 8 cores:
BenchmarkCachePutNoOverFlow-8               1932            618372 ns/op             253 B/op          0 allocs/op
BenchmarkCacheGetNoOverflow-8               6547            211527 ns/op               0 B/op          0 allocs/op
BenchmarkCachePutGetNoOverflow-8            1873            621718 ns/op             261 B/op          0 allocs/op
BenchmarkCachePutOverflow-8                 2262            464328 ns/op              32 B/op          0 allocs/op
BenchmarkCachePutGetOverflow-8              1764            655866 ns/op              38 B/op          0 allocs/op

New cache implementation performance on 8 cores:
BenchmarkCachePutNoOverFlow-8              10408            111412 ns/op               0 B/op          0 allocs/op
BenchmarkCacheGetNoOverflow-8              22407             52809 ns/op               0 B/op          0 allocs/op
BenchmarkCachePutGetNoOverflow-8            6583            168088 ns/op               0 B/op          0 allocs/op
BenchmarkCachePutOverflow-8                 9822            117212 ns/op               2 B/op          0 allocs/op
BenchmarkCachePutGetOverflow-8              6481            175952 ns/op               3 B/op          0 allocs/op

Current cache implementation performance on 16 cores:
BenchmarkCachePutNoOverFlow-16              2331            475307 ns/op             218 B/op          0 allocs/op
BenchmarkCacheGetNoOverflow-16              6069            196905 ns/op               0 B/op          0 allocs/op
BenchmarkCachePutGetNoOverflow-16           1870            644236 ns/op             262 B/op          0 allocs/op
BenchmarkCachePutOverflow-16                2296            509279 ns/op              34 B/op          0 allocs/op
BenchmarkCachePutGetOverflow-16             1726            671510 ns/op              45 B/op          0 allocs/op

New cache implementation performance on 16 cores:
BenchmarkCachePutNoOverFlow-16             13549             82413 ns/op               0 B/op          0 allocs/op
BenchmarkCacheGetNoOverflow-16             30274             38997 ns/op               0 B/op          0 allocs/op
BenchmarkCachePutGetNoOverflow-16           8512            126239 ns/op               0 B/op          0 allocs/op
BenchmarkCachePutOverflow-16               13884             88124 ns/op               1 B/op          0 allocs/op
BenchmarkCachePutGetOverflow-16             7903            131299 ns/op               3 B/op          0 allocs/op
```
From the benchmarks above, we can see that the new implementation is ~5
times faster than the old one.


---------
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2025-01-02 17:43:23 +01:00
Alex Gustafsson
43ded688f7 Add open containers source label to Dockerfiles (#7893)
### Describe Your Changes

In order for third-party tooling to identify the source repository of
VictoriaMetrics, add the org.opencontainers.image label to the
Dockerfiles. This enables a whole suite of tools that scan container
images to further correlate data with the source code.

The lack of these annotations can be identified using docker:

```shell
docker pull victoriametrics/victoria-metrics
docker inspect victoriametrics/victoria-metrics
```

```jsonc
// ...
"Labels": null
// ...
```

If we try an image that has the annotations, we'll see more output.

```shell
docker pull traefik
docker image inspect traefik
```

```jsonc
// ...
"Labels": {
    "org.opencontainers.image.description": "A modern reverse-proxy",
    "org.opencontainers.image.documentation": "https://docs.traefik.io",
    "org.opencontainers.image.source": "https://github.com/traefik/traefik",
    "org.opencontainers.image.title": "Traefik",
    "org.opencontainers.image.url": "https://traefik.io",
    "org.opencontainers.image.vendor": "Traefik Labs",
    "org.opencontainers.image.version": "v3.2.3"
}
// ...
```

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-02 20:43:17 +04:00
Hui Wang
661420fe85 dashboard/vmagent: change metric for Persistent Queue panel
consistently use `vmagent_remotewrite_pending_data_bytes`  on vmagent dashboard to represent persistent queue size.

`vmagent_remotewrite_pending_data_bytes =
vm_persistentqueue_bytes_pending + pendingInmemoryBytes`
According to panel description, `vmagent_remotewrite_pending_data_bytes`
is more accurate.
>Persistent queue size shows size of pending samples in bytes which
hasn't been flushed to remote storage yet.
 
And we already use `vmagent_remotewrite_pending_data_bytes` in other two
panels.

44d2205136/dashboards/vmagent.json (L7132)
2025-01-02 13:04:40 +01:00
Andrii Chubatiuk
7aab967447 Makefile: cspell makefile refactor
- removed absolute paths to run without docker
- set cspell to default entrypoint value
- set cspell config path instead of cspell.json copying and removal
2025-01-02 12:52:24 +01:00
Hui Wang
afb07034ed app/vmalert: fix the auto-generated metrics ALERTS and ALERTS_FOR_STATE
Previously, since labels slice is reused for both `ALERTS` and
`ALERTS_FOR_STATE`, metrics might have incorrect labels and affect the
restore process. Tested the fix under `TestAlertingRule_Exec:
"for-pending=>empty"`.

The bug is introduced in
282f13cf11.
Affected versions: v1.106.1, v1.107...v1.108.x

related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7796
2025-01-02 12:51:05 +01:00
f41gh7
44d2205136 app/vmalert: properly format datasource URL for enterprise group.tenant
At Enterprise version of the vmalert, `group` supports `tenant` field.

`tenant` field value must be added to the `datasource` as a part of the URL path prefix.

But VictoriaLogs can obtain tenant information only from `headers` and defined `tenant` breaks requests to the `VictoriaLogs` datasource.

 This commit properly checks `datasourceType` and skips adding path prefix if `datasourceType` is `vlogs`.

---------
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-12-30 15:42:29 +01:00
f41gh7
b226318f9e app/vmstorage: allow to override the default unique time series limit
previously vmstorage ignored limit values from vmselect component.

This behavior is prohibited starting from v1.105.0, with
85f60237e2.

This breaks the original intent of the -search.maxUniqueTimeseries command-line flag, which has been added at vmselect nodes in the commit b843f0e : to be able to override the default limit at vmstorage on the number of unique time series, at different subsets of vmselect nodes.

The behavior should be the following:

*    If -search.maxUniqueTimeseries command-line flag isn't set at both vmselect and vmstorage nodes, then the limit on  the number of unique time series must be automatically detected at vmstorage nodes according to

* vmstorage: automatically adjust -search.maxUniqueTimeseries max value   . This simplifies configuration of VictoriaMetrics cluster for the typical case.

* If -search.maxUniqueTimeseries command-line flag is explicitly set at vmstorage node, then it must be used as the limit on the number of unique time series, without automatic detection of the limit. Explicitly set limit at vmstorage node cannot be exceeded by the limit from vmselect nodes.
* If the -search.maxUniqueTimeseries command-line flag is explicitly set at vmselect node, then it must override the automatically detected limit at vmstorage node. For example, if vmselect node provides the limit, which exceeds the automatically detected limit at vmstorage node, then the limit from the vmselect node must be applied during query execution at vmstorage node. This will allow properly executing queries from the subset of vmselect nodes for reporting queries described above.

related issue:
 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7852
2024-12-30 15:20:52 +01:00
Github Actions
30999204c9 Automatic update operator docs from VictoriaMetrics/operator@5e0854a (#7938)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-12-30 14:06:27 +01:00
Zhu Jiekun
ffddfa1f94 app/vmctl: properly handle influx series without tags
### Describe Your Changes

Previously, vmctl expect that tag must exist for each measurement, but
it's actually not necessary.


f16a58f14c/app/vmctl/influx/influx.go (L183-L186)

This pull request fix it by removing the check. For influx series
`measurement1_value1{}`, it will be represented as:
```go
Series{
  Measurement: "measurement1",
  Field:       "value1",
  LabelPairs:  []LabelPair{},
  EmptyTags:   []string{},
}
```
and searched by the following query:
```sql
select "value1" from "measurement1"
``` 

 Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7921
2024-12-26 20:39:06 +01:00
f41gh7
fc336bbf20 app/vminsert: properly ingest influx metrics
Commit 71bb9fc0d0 introduced a regression.
If labels are empty and relabeling is not configured, influx ingestion hanlder
performed an earlier exit due to TryPrepareLabels call.
 Due micro-optimisations for this procotol, this check was not valid.
Since it didn't take in account metircName, which added later and skip metrics line.

 This commit removes `TryPrepareLabel` function call from this path and inline it instead.
It properly track empty labels path.

 Adds initial tests implementation for data ingestion protocols.

 Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7933

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-12-26 12:14:42 +01:00
f41gh7
e0b2c1c4f5 docs/changelog: removes duplicate record for maxIngestionRate feature
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-12-24 19:40:44 +01:00
Github Actions
5afbee5f6f Automatic update operator docs from VictoriaMetrics/operator@43f5554 (#7923)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-12-24 19:31:27 +01:00
Github Actions
51459196f9 Automatic update helm docs from VictoriaMetrics/helm-charts@1f33c21 (#7929)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-12-24 19:30:43 +01:00
Dima Shur
7941877233 docs: changed typo in label (enterpriSe instead of enterpriZe) (#7925)
### Describe Your Changes

Fixed typo in contributing.md (enterpriZe -> enterpriSe in the label
name)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-12-24 15:51:35 +04:00
Phuong Le
f303081304 vminsert: sort the storage nodes during initialization (#7899)
Fixes #7898
2024-12-23 19:41:17 +01:00
Ted Possible
a84628f701 app/vminsert: support for rate limiting number of samples/sec with -maxIngestionRate
This commit adds feature to limit sample ingestion rate globally for ingestion protocols. 

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7377
2024-12-23 17:37:30 +01:00
Github Actions
f823a225ac Automatic update operator docs from VictoriaMetrics/operator@471f183 (#7916)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-12-23 16:48:49 +01:00
Andrii Chubatiuk
79f1a37ee6 vlinsert: take into account order of msgfields to have predictable _msg field selection in case of multiple matches (#7784)
### Describe Your Changes

Currently if multiple msgFields are present in a log row it's not
obvious which field is selected as a _msg field. With this PR and order
of msgfield values defined either via headers or query arg params
defines a priority of these values

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-12-23 10:10:02 +01:00
Andrii Chubatiuk
f9cd408ca9 datadog-serverless: fixed metrics and logs ingestion from Datadog serverless extensions for AWS and GCP (#7769)
fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7761

### Describe Your Changes

- datadog /api/v2/logs api supports message field in json format, which
is not documented and is used by serverless extension. This PR allows
message field to be both string and object type. Also added support of
not documented timestamp field
- added `-datadog.streamFields` and `-datadog.ignoreFields` flags to
configure default stream fields for datadog logs, where there's no
alternative option to pass extra headers and query args
- added ingest `max` and `min` values of data, which are ingested using
`datadogsketches` API, which is also actively used by serverless
extensions
- use default `.` separator instead of `_` for sketches metric names
until metrics are not sanitized
2024-12-23 09:57:48 +01:00
Aliaksandr Valialkin
c2811d8d11 docs/VictoriaLogs/LogsQL.md: fix a link to count_uniq_hash stats function docs
It must be consistent with the other stats functions

This is a follow-up for de0ae735aa
2024-12-22 14:39:27 +01:00
Aliaksandr Valialkin
8d981b15c9 deployment: update VictoriaLogs Docker image from v1.3.2-victorialogs to v1.4.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.4.0-victorialogs
2024-12-22 14:36:49 +01:00
Aliaksandr Valialkin
58f09fe3f8 docs/VictoriaLogs/CHANGELOG.md: cut v1.4.0-victorialogs release 2024-12-22 14:31:31 +01:00
Aliaksandr Valialkin
afd926a0b0 lib/logstorage: limit the maximum number of logs and/or log streams, which can be passed to stream_context pipe
This should prevent from excess usage of CPU, RAM and other resources when too many logs
are passed to 'stream_context' pipe.

It is expected that 'stream_context' pipe results are investigated by humans, who cannot inspect
surrounding logs for millions of initial logs. That's why it is OK to limit the number of logs
and/or log streams, which can be passed to 'stream_context' pipe.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7766
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7903
2024-12-22 14:28:50 +01:00
Aliaksandr Valialkin
204c102342 app/vlselect/vmui: run make vmui-logs-update after the commit 1fbc2c0db1
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7288
2024-12-22 13:53:45 +01:00
Aliaksandr Valialkin
c5949af9e8 lib/logstorage: reduce memory allocations when splitting in(...) values into tokens and calculating hashes for these tokens
While at it, reduce memory allocations at Storage.getFieldValuesNoHits and make it more scalable on multi-CPU systems.

This improves performance of in(<query>) filter when the <query> returns big number of values.
2024-12-22 13:13:44 +01:00
Aliaksandr Valialkin
5dc0413bc0 lib/logstorage: allow specifying hits column name in the top pipe via top ... hits as <column_name> syntax 2024-12-22 11:23:19 +01:00
Aliaksandr Valialkin
f919783de9 lib/logstorage: uncommend accidentally commented tests at 60f9f44150 2024-12-22 02:20:57 +01:00
Aliaksandr Valialkin
60f9f44150 lib/logstorage: reduce memory allocations at stats and top pipes
Use chunked allocator in order to reduce memory allocations. It allocates objects from slices of up to 64Kb size.
This improves performance for `stats` and `top` pipes by up to 2x when they are applied to big number of `by (...)` groups.

Also parallelize execution of `count_uniq`, `count_uniq_hash` and `uniq_values` stats functions,
so they are executed faster on hosts with many CPU cores when applied to fields with big number
of unique values.
2024-12-22 02:13:02 +01:00
Github Actions
0fcbe8fdae Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@b18583c (#7910) 2024-12-21 09:42:48 -08:00
Github Actions
458b602938 Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@cbff3fa (#7909) 2024-12-21 09:31:07 -08:00
Aliaksandr Valialkin
471f1d0a09 lib/logstorage: fixed a typo in blockResult.reset()
The commit 4599429f51 improperly set br.cs to nil,
while it should set br.bs to nil instead. This resulted in excess memory allocations
at br.csInit() and br.csInitFast().
2024-12-21 13:39:25 +01:00
hagen1778
7f80c1633f docs: mention filebeat version requirement for vlogs integration
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-12-20 16:21:13 +01:00
Yury Molodov
186b00df6b vmui: add export button for raw query data (#7828)
### Describe Your Changes

1. Added the ability to export data from the `Raw Query` page and import
exported data to the `Query Analyzer` page (related issue #7628).
2. Added a `Title` input field; the `Title` is displayed when importing
data on the `Query Analyzer` page.
3. Implemented `Markdown` support for comments in exported data.  
4. Updated the styling of the `Query Analyzer` page.  
5. Fixed an issue where the `Upload JSON` button on the `Query Analyzer`
page was only clickable on the button text (now clickable on the entire
button area).
6. Added a tooltip with `Deduplication` information on the `Raw Query`
page (related to issue #7763).

<details>
  <summary>Screenshots</summary>
  
#### Data export and `Markdown` preview

<img width="400"
src="https://github.com/user-attachments/assets/bbab31bb-81d3-4335-98c3-d01c8786bde4"/>
<img width="400"
src="https://github.com/user-attachments/assets/3cfd9938-b518-45d6-8ded-e3e7e6ab9299"/>

#### `Query Analyzer` page displaying data from `Raw Query`

<img width="900"
src="https://github.com/user-attachments/assets/008e0e93-92f2-4c25-a20e-3cee90a03397"/>

#### Viewing stats and comments on the `Query Analyzer` page  
    
<img width="600"
src="https://github.com/user-attachments/assets/18bfbba1-a11c-420e-84f2-78229ac7bd25"/>

#### Viewing stats data from the `Query` page

<img width="900"
src="https://github.com/user-attachments/assets/0f7a3009-9fb5-4727-b0c4-257aa196a9c1"/>

#### Tooltip on the `Raw Query` page  

<img width="900"
src="https://github.com/user-attachments/assets/400f86e7-f362-4307-8b1d-24af3c67020e"/>
  
</details>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-12-20 15:51:06 +01:00
Github Actions
4205ae3011 Automatic update helm docs from VictoriaMetrics/helm-charts@feb0675 (#7897)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-12-20 15:27:51 +01:00
Daria Karavaieva
491028774a docs/vmanomaly: popup deprecated_from and available_from for all docs (#7905)
### Describe Your Changes
added deprecated form and available from popups in vmanomaly docs

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-12-20 15:24:16 +01:00
Mathias Palmersheim
565b79c9ca docs: update vmalert+victorialogs doc with multitenant recording (#7779)
### Describe Your Changes
 
- Adds Headers to FAQ questions in vmalert for Victorialogs
- Adds FAQ for multitenant recording rules described in #7656

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Haley Wang <haley@victoriametrics.com>
2024-12-20 15:02:00 +01:00
Aliaksandr Valialkin
5478cc61c2 lib/cgroup: add missing initialization of gogc variable inside SetGOGC
This is a follow-up for 79c08ecac4

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7902
2024-12-20 14:56:59 +01:00
Aliaksandr Valialkin
79c08ecac4 lib/cgroup: use the default GOGC=100 for the most of VictoriaMetrics components
Historically some of VictoriaMetrics components were optimized for the low rate of memory allocations.
These are: vmagent, single-node VictoriaMetrics and vmstorage. These components benefit from the low
GOGC value, since this allow reducing their memory usage in steady state on typical workloads.

Other VictoriaMetrics components aren't optimized for the reduced rate of memory allocations.
This results in the increased CPU usage spent on garbage collection (GC) in these components,
since it must be triggered at higher rate. See https://tip.golang.org/doc/gc-guide#GOGC for details.

These components do not use too much memory, so it is OK increasing the GOGC for these components
from 30 to 100 - this won't affect the most users.

Keep GOGC to 30 only for vmagent, single-node VictoriaMetrics and vmstorage components.
See 077193d87c and 54b9e1d3cb .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7902
2024-12-20 14:48:28 +01:00
hagen1778
f47fd83e54 docs: add example with dots in label name to vlogs rules
This change adds an example of how to use labels with `.` dots
in rule annotations.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-12-20 14:07:48 +01:00
Aliaksandr Valialkin
9c39bac565 lib/logstorage: fix imroper sorting of numeric fields when they are stored as const values at sort pipe
Numeric fields can be stored as const values in the block of logs. In this case the `sort` pipe
was incorrectly comparing such values as strings instead of numbers. This results in incorrect
sort results. For example, 123 was smaller than 2. Fix this by removing the incorrect case
for comparing const fields.

While at it, replace lessString() with strings.LessNatural() in the sortBlockLess.
This improves sorting performance a bit, since the sortBlockLess function already tried
comparing numeric values, and it doesn't need to spend CPU time on such a comparison again inside lessString() call.
The commit 42c9183281 wasn't correct by replacing strings.LessNatural() with lessString()
inside the sortBlockLess() function.
2024-12-20 13:26:20 +01:00
Roman Khavronenko
1042f07498 docs: update OTEL guide (#7887)
* simplify wording
* update styles
* remove extra info about go application details. The details are likely
not needed and we didn't have details for rolling-dice app anyway. So
keep it simple for consstency and brevity.
* update navigation for simplicity sake
* fix typos

follow-up after
40b47601d1

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-12-19 15:13:03 +01:00
Nikolay
79a595c6d0 app/vmauth: properly log host at debugInfo function (#7886)
vmauth started to use request.Host after commit
f4776fec1b for`src_hosts` routing rules.

This commit adds http.Request.Host to the debugInfo output in order to
be consistent with routing logic.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-12-19 15:04:37 +01:00
Andrii Chubatiuk
40b47601d1 docs/guides/otel: added logs integration, updated old otel dependencies
### Describe Your Changes

- added VictoriaLogs to OpenTelemetry guide
- updated deprecated dependencies
- added deltatocumulative processor to example and deltatemporality
selector to one of examples to use for counters by default
- added exponential histograms to example

---
Signed-off-by: Andrii Chubatiuk <andrew.chubatiuk@gmail.com>
2024-12-19 12:32:41 +01:00
Zakhar Bessarab
6bfcbe66f7 docs/release-guide: add a note about versioning in helm charts and ansible
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-12-19 12:28:32 +01:00
f41gh7
94118c63f6 docs: update VM apps version to v1.108.1
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-12-19 12:25:37 +01:00
236 changed files with 6328 additions and 3079 deletions

View File

@@ -1,12 +1,14 @@
# VictoriaMetrics
[![Latest Release](https://img.shields.io/github/release/VictoriaMetrics/VictoriaMetrics.svg?style=flat-square)](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest)
[![Docker Pulls](https://img.shields.io/docker/pulls/victoriametrics/victoria-metrics.svg?maxAge=604800)](https://hub.docker.com/r/victoriametrics/victoria-metrics)
[![Slack](https://img.shields.io/badge/join%20slack-%23victoriametrics-brightgreen.svg)](https://slack.victoriametrics.com/)
[![GitHub license](https://img.shields.io/github/license/VictoriaMetrics/VictoriaMetrics.svg)](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/LICENSE)
[![Go Report](https://goreportcard.com/badge/github.com/VictoriaMetrics/VictoriaMetrics)](https://goreportcard.com/report/github.com/VictoriaMetrics/VictoriaMetrics)
[![Build Status](https://github.com/VictoriaMetrics/VictoriaMetrics/workflows/main/badge.svg)](https://github.com/VictoriaMetrics/VictoriaMetrics/actions)
[![codecov](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/branch/master/graph/badge.svg)](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics)
![Latest Release](https://img.shields.io/github/v/release/VictoriaMetrics/VictoriaMetrics?sort=semver&label=&filter=!*-victorialogs&logo=github&labelColor=gray&color=gray&link=https%3A%2F%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics%2Freleases%2Flatest)
![Docker Pulls](https://img.shields.io/docker/pulls/victoriametrics/victoria-metrics?label=&logo=docker&logoColor=white&labelColor=2496ED&color=2496ED&link=https%3A%2F%2Fhub.docker.com%2Fr%2Fvictoriametrics%2Fvictoria-metrics)
![Go Report](https://goreportcard.com/badge/github.com/VictoriaMetrics/VictoriaMetrics?link=https%3A%2F%2Fgoreportcard.com%2Freport%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics)
![Build Status](https://github.com/VictoriaMetrics/VictoriaMetrics/workflows/main/badge.svg?link=https%3A%2F%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics%2Factions)
![codecov](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/branch/master/graph/badge.svg?link=https%3A%2F%2Fcodecov.io%2Fgh%2FVictoriaMetrics%2FVictoriaMetrics)
![License](https://img.shields.io/github/license/VictoriaMetrics/VictoriaMetrics?labelColor=green&label=&link=https%3A%2F%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics%2Fblob%2Fmaster%2FLICENSE)
![Slack](https://img.shields.io/badge/Join-4A154B?logo=slack&link=https%3A%2F%2Fslack.victoriametrics.com)
![X](https://img.shields.io/twitter/follow/VictoriaMetrics?style=flat&label=Follow&color=black&logo=x&labelColor=black&link=https%3A%2F%2Fx.com%2FVictoriaMetrics)
![Reddit](https://img.shields.io/reddit/subreddit-subscribers/VictoriaMetrics?style=flat&label=Join&labelColor=red&logoColor=white&logo=reddit&link=https%3A%2F%2Fwww.reddit.com%2Fr%2FVictoriaMetrics)
<picture>
<source srcset="docs/logo_white.webp" media="(prefers-color-scheme: dark)">

View File

@@ -14,6 +14,7 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/promql"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmstorage"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/buildinfo"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/cgroup"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/envflag"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/flagutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fs"
@@ -39,9 +40,20 @@ var (
"The saved data survives unclean shutdowns such as OOM crash, hardware reset, SIGKILL, etc. "+
"Bigger intervals may help increase the lifetime of flash storage with limited write cycles (e.g. Raspberry PI). "+
"Smaller intervals increase disk IO load. Minimum supported value is 1s")
maxIngestionRate = flag.Int("maxIngestionRate", 0, "The maximum number of samples vmsingle can receive per second. Data ingestion is paused when the limit is exceeded. "+
"By default there are no limits on samples ingestion rate.")
)
func main() {
// VictoriaMetrics is optimized for reduced memory allocations,
// so it can run with the reduced GOGC in order to reduce the used memory,
// while keeping CPU usage spent in GC at low levels.
//
// Some workloads may need increased GOGC values. Then such values can be set via GOGC environment variable.
// It is recommended increasing GOGC if go_memstats_gc_cpu_fraction metric exposed at /metrics page
// exceeds 0.05 for extended periods of time.
cgroup.SetGOGC(30)
// Write flags and help message to stdout, since it is easier to grep or pipe.
flag.CommandLine.SetOutput(os.Stdout)
flag.Usage = usage
@@ -76,6 +88,7 @@ func main() {
storage.SetDataFlushInterval(*inmemoryDataFlushInterval)
vmstorage.Init(promql.ResetRollupResultCacheIfNeeded)
vmselect.Init()
vminsertcommon.StartIngestionRateLimiter(*maxIngestionRate)
vminsert.Init()
startSelfScraper()
@@ -97,6 +110,7 @@ func main() {
}
logger.Infof("successfully shut down the webservice in %.3f seconds", time.Since(startTime).Seconds())
vminsert.Stop()
vminsertcommon.StopIngestionRateLimiter()
vmstorage.Stop()
vmselect.Stop()

View File

@@ -14,6 +14,7 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/app/vlinsert/insertutils"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vlstorage"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/flagutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logstorage"
@@ -21,6 +22,11 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/lib/writeconcurrencylimiter"
)
var (
datadogStreamFields = flagutil.NewArrayString("datadog.streamFields", "Datadog tags to be used as stream fields.")
datadogIgnoreFields = flagutil.NewArrayString("datadog.ignoreFields", "Datadog tags to ignore.")
)
var parserPool fastjson.ParserPool
// RequestHandler processes Datadog insert requests
@@ -79,6 +85,13 @@ func datadogLogsIngestion(w http.ResponseWriter, r *http.Request) bool {
return true
}
if len(cp.StreamFields) == 0 {
cp.StreamFields = *datadogStreamFields
}
if len(cp.IgnoreFields) == 0 {
cp.IgnoreFields = *datadogIgnoreFields
}
if err := vlstorage.CanWriteData(); err != nil {
httpserver.Errorf(w, r, "%s", err)
return true
@@ -105,6 +118,70 @@ var (
v2LogsRequestDuration = metrics.NewHistogram(`vl_http_request_duration_seconds{path="/insert/datadog/api/v2/logs"}`)
)
// datadog message field has two formats:
// - regular log message with string text
// - nested json format for serverless plugins
// which has folowing format:
// {"message": {"message": "text","lamdba": {"arn": "string","requestID": "string"}, "timestamp": int64} }
//
// See https://github.com/DataDog/datadog-lambda-extension/blob/28b90c7e4e985b72d60b5f5a5147c69c7ac693c4/bottlecap/src/logs/lambda/mod.rs#L24
func appendMsgFields(fields []logstorage.Field, v *fastjson.Value) ([]logstorage.Field, error) {
switch v.Type() {
case fastjson.TypeString:
val := v.GetStringBytes()
fields = append(fields, logstorage.Field{
Name: "_msg",
Value: bytesutil.ToUnsafeString(val),
})
case fastjson.TypeObject:
var firstErr error
v.GetObject().Visit(func(k []byte, v *fastjson.Value) {
if firstErr != nil {
return
}
switch bytesutil.ToUnsafeString(k) {
case "message":
val := v.GetStringBytes()
fields = append(fields, logstorage.Field{
Name: "_msg",
Value: bytesutil.ToUnsafeString(val),
})
case "status":
val := v.GetStringBytes()
fields = append(fields, logstorage.Field{
Name: "status",
Value: bytesutil.ToUnsafeString(val),
})
case "lamdba":
obj, err := v.Object()
if err != nil {
firstErr = err
firstErr = fmt.Errorf("unexpected lambda value type for %q:%q; want object", k, v)
return
}
obj.Visit(func(k []byte, v *fastjson.Value) {
if firstErr != nil {
return
}
val, err := v.StringBytes()
if err != nil {
firstErr = fmt.Errorf("unexpected lambda label value type for %q:%q; want string", k, v)
return
}
fields = append(fields, logstorage.Field{
Name: bytesutil.ToUnsafeString(k),
Value: bytesutil.ToUnsafeString(val),
})
})
}
})
default:
return fields, fmt.Errorf("unsupported message type %q", v.Type().String())
}
return fields, nil
}
// readLogsRequest parses data according to DataDog logs format
// https://docs.datadoghq.com/api/latest/logs/#send-logs
func readLogsRequest(ts int64, data []byte, lmp insertutils.LogMessageProcessor) error {
@@ -129,19 +206,27 @@ func readLogsRequest(ts int64, data []byte, lmp insertutils.LogMessageProcessor)
if err != nil {
return
}
val, e := v.StringBytes()
if e != nil {
err = fmt.Errorf("unexpected label value type for %q:%q; want string", k, v)
return
}
switch string(k) {
switch bytesutil.ToUnsafeString(k) {
case "message":
fields = append(fields, logstorage.Field{
Name: "_msg",
Value: bytesutil.ToUnsafeString(val),
})
fields, err = appendMsgFields(fields, v)
if err != nil {
return
}
case "timestamp":
val, e := v.Int64()
if e != nil {
err = fmt.Errorf("failed to parse timestamp for %q:%q", k, v)
}
if val > 0 {
ts = val * 1e6
}
case "ddtags":
// https://docs.datadoghq.com/getting_started/tagging/
val, e := v.StringBytes()
if e != nil {
err = fmt.Errorf("unexpected label value type for %q:%q; want string", k, v)
return
}
var pair []byte
idx := 0
for idx >= 0 {
@@ -168,12 +253,20 @@ func readLogsRequest(ts int64, data []byte, lmp insertutils.LogMessageProcessor)
}
}
default:
val, e := v.StringBytes()
if e != nil {
err = fmt.Errorf("unexpected label value type for %q:%q; want string", k, v)
return
}
fields = append(fields, logstorage.Field{
Name: bytesutil.ToUnsafeString(k),
Value: bytesutil.ToUnsafeString(val),
})
}
})
if err != nil {
return err
}
lmp.AddRow(ts, fields, nil)
fields = fields[:0]
}

View File

@@ -54,6 +54,12 @@ func TestReadLogsRequestSuccess(t *testing.T) {
"hostname":"127.0.0.1",
"message":"bar",
"service":"test"
}, {
"ddsource":"nginx",
"ddtags":"tag1:value1,tag2:value2",
"hostname":"127.0.0.1",
"message":{"message": "nested"},
"service":"test"
}, {
"ddsource":"nginx",
"ddtags":"tag1:value1,tag2:value2",
@@ -86,8 +92,9 @@ func TestReadLogsRequestSuccess(t *testing.T) {
"service":"test"
}
]`
rowsExpected := 6
rowsExpected := 7
resultExpected := `{"ddsource":"nginx","tag1":"value1","tag2":"value2","hostname":"127.0.0.1","_msg":"bar","service":"test"}
{"ddsource":"nginx","tag1":"value1","tag2":"value2","hostname":"127.0.0.1","_msg":"nested","service":"test"}
{"ddsource":"nginx","tag1":"value1","tag2":"value2","hostname":"127.0.0.1","_msg":"foobar","service":"test"}
{"ddsource":"nginx","tag1":"value1","tag2":"value2","hostname":"127.0.0.1","_msg":"baz","service":"test"}
{"ddsource":"nginx","tag1":"value1","tag2":"value2","hostname":"127.0.0.1","_msg":"xyz","service":"test"}

View File

@@ -1,13 +1,13 @@
{
"files": {
"main.css": "./static/css/main.d05122da.css",
"main.js": "./static/js/main.6082e5a5.js",
"main.css": "./static/css/main.fa83344e.css",
"main.js": "./static/js/main.8ad2bc1f.js",
"static/js/685.f772060c.chunk.js": "./static/js/685.f772060c.chunk.js",
"static/media/MetricsQL.md": "./static/media/MetricsQL.a00044c91d9781cf8557.md",
"index.html": "./index.html"
},
"entrypoints": [
"static/css/main.d05122da.css",
"static/js/main.6082e5a5.js"
"static/css/main.fa83344e.css",
"static/js/main.8ad2bc1f.js"
]
}

View File

@@ -1 +1 @@
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.svg"/><link rel="apple-touch-icon" href="./favicon.svg"/><link rel="mask-icon" href="./favicon.svg" color="#000000"><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="Explore your log data with VictoriaLogs UI"/><link rel="manifest" href="./manifest.json"/><title>UI for VictoriaLogs</title><meta name="twitter:card" content="summary"><meta name="twitter:title" content="UI for VictoriaLogs"><meta name="twitter:site" content="@https://victoriametrics.com/products/victorialogs/"><meta name="twitter:description" content="Explore your log data with VictoriaLogs UI"><meta name="twitter:image" content="./preview.jpg"><meta property="og:type" content="website"><meta property="og:title" content="UI for VictoriaLogs"><meta property="og:url" content="https://victoriametrics.com/products/victorialogs/"><meta property="og:description" content="Explore your log data with VictoriaLogs UI"><script defer="defer" src="./static/js/main.6082e5a5.js"></script><link href="./static/css/main.d05122da.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.svg"/><link rel="apple-touch-icon" href="./favicon.svg"/><link rel="mask-icon" href="./favicon.svg" color="#000000"><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="Explore your log data with VictoriaLogs UI"/><link rel="manifest" href="./manifest.json"/><title>UI for VictoriaLogs</title><meta name="twitter:card" content="summary"><meta name="twitter:title" content="UI for VictoriaLogs"><meta name="twitter:site" content="@https://victoriametrics.com/products/victorialogs/"><meta name="twitter:description" content="Explore your log data with VictoriaLogs UI"><meta name="twitter:image" content="./preview.jpg"><meta property="og:type" content="website"><meta property="og:title" content="UI for VictoriaLogs"><meta property="og:url" content="https://victoriametrics.com/products/victorialogs/"><meta property="og:description" content="Explore your log data with VictoriaLogs UI"><script defer="defer" src="./static/js/main.8ad2bc1f.js"></script><link href="./static/css/main.fa83344e.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -30,6 +30,7 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/lib/auth"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/buildinfo"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/cgroup"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/envflag"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/flagutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver"
@@ -97,6 +98,15 @@ var (
)
func main() {
// vmagent is optimized for reduced memory allocations,
// so it can run with the reduced GOGC in order to reduce the used memory,
// while keeping CPU usage spent in GC at low levels.
//
// Some workloads may need increased GOGC values. Then such values can be set via GOGC environment variable.
// It is recommended increasing GOGC if go_memstats_gc_cpu_fraction metric exposed at /metrics page
// exceeds 0.05 for extended periods of time.
cgroup.SetGOGC(30)
// Write flags and help message to stdout, since it is easier to grep or pipe.
flag.CommandLine.SetOutput(os.Stdout)
flag.Usage = usage

View File

@@ -614,7 +614,7 @@ func (ar *AlertingRule) alertToTimeSeries(a *notifier.Alert, timestamp int64) []
}
func alertToTimeSeries(a *notifier.Alert, timestamp int64) prompbmarshal.TimeSeries {
var labels []prompbmarshal.Label
labels := make([]prompbmarshal.Label, 0, len(a.Labels)+2)
for k, v := range a.Labels {
labels = append(labels, prompbmarshal.Label{
Name: k,
@@ -634,7 +634,7 @@ func alertToTimeSeries(a *notifier.Alert, timestamp int64) prompbmarshal.TimeSer
// alertForToTimeSeries returns a time series that represents
// state of active alerts, where value is time when alert become active
func alertForToTimeSeries(a *notifier.Alert, timestamp int64) prompbmarshal.TimeSeries {
var labels []prompbmarshal.Label
labels := make([]prompbmarshal.Label, 0, len(a.Labels)+1)
for k, v := range a.Labels {
labels = append(labels, prompbmarshal.Label{
Name: k,
@@ -650,21 +650,24 @@ func alertForToTimeSeries(a *notifier.Alert, timestamp int64) prompbmarshal.Time
// for alerts which changed their state from Pending to Inactive or Firing.
func pendingAlertStaleTimeSeries(ls map[string]string, timestamp int64, includeAlertForState bool) []prompbmarshal.TimeSeries {
var result []prompbmarshal.TimeSeries
var baseLabels []prompbmarshal.Label
baseLabels := make([]prompbmarshal.Label, 0, len(ls)+1)
for k, v := range ls {
baseLabels = append(baseLabels, prompbmarshal.Label{
Name: k,
Value: v,
})
}
alertsLabels := make([]prompbmarshal.Label, 0, len(ls)+2)
alertsLabels = append(alertsLabels, baseLabels...)
// __name__ already been dropped, no need to check duplication
alertsLabels := append(baseLabels, prompbmarshal.Label{Name: "__name__", Value: alertMetricName})
alertsLabels = append(alertsLabels, prompbmarshal.Label{Name: "__name__", Value: alertMetricName})
alertsLabels = append(alertsLabels, prompbmarshal.Label{Name: alertStateLabel, Value: notifier.StatePending.String()})
result = append(result, newTimeSeries([]float64{decimal.StaleNaN}, []int64{timestamp}, alertsLabels))
if includeAlertForState {
alertsForStateLabels := append(baseLabels, prompbmarshal.Label{Name: "__name__", Value: alertForStateMetricName})
result = append(result, newTimeSeries([]float64{decimal.StaleNaN}, []int64{timestamp}, alertsForStateLabels))
baseLabels = append(baseLabels, prompbmarshal.Label{Name: "__name__", Value: alertForStateMetricName})
result = append(result, newTimeSeries([]float64{decimal.StaleNaN}, []int64{timestamp}, baseLabels))
}
return result
}
@@ -672,22 +675,25 @@ func pendingAlertStaleTimeSeries(ls map[string]string, timestamp int64, includeA
// firingAlertStaleTimeSeries returns stale `ALERTS` and `ALERTS_FOR_STATE` time series
// for alerts which changed their state from Firing to Inactive.
func firingAlertStaleTimeSeries(ls map[string]string, timestamp int64) []prompbmarshal.TimeSeries {
var baseLabels []prompbmarshal.Label
baseLabels := make([]prompbmarshal.Label, 0, len(ls)+1)
for k, v := range ls {
baseLabels = append(baseLabels, prompbmarshal.Label{
Name: k,
Value: v,
})
}
alertsLabels := make([]prompbmarshal.Label, 0, len(ls)+2)
alertsLabels = append(alertsLabels, baseLabels...)
// __name__ already been dropped, no need to check duplication
alertsLabels := append(baseLabels, prompbmarshal.Label{Name: "__name__", Value: alertMetricName})
alertsLabels = append(alertsLabels, prompbmarshal.Label{Name: "__name__", Value: alertMetricName})
alertsLabels = append(alertsLabels, prompbmarshal.Label{Name: alertStateLabel, Value: notifier.StateFiring.String()})
alertsForStateLabels := append(baseLabels, prompbmarshal.Label{Name: "__name__", Value: alertForStateMetricName})
baseLabels = append(baseLabels, prompbmarshal.Label{Name: "__name__", Value: alertForStateMetricName})
return []prompbmarshal.TimeSeries{
newTimeSeries([]float64{decimal.StaleNaN}, []int64{timestamp}, alertsLabels),
newTimeSeries([]float64{decimal.StaleNaN}, []int64{timestamp}, alertsForStateLabels),
newTimeSeries([]float64{decimal.StaleNaN}, []int64{timestamp}, baseLabels),
}
}

View File

@@ -252,10 +252,14 @@ func TestAlertingRule_Exec(t *testing.T) {
},
map[int][]prompbmarshal.TimeSeries{
0: {
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "empty_labels"}, {Name: "alertstate", Value: "firing"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "empty_labels"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Unix()), Timestamp: ts.UnixNano() / 1e6}}},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "empty_labels"}, {Name: "alertstate", Value: "firing"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "empty_labels"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Unix()), Timestamp: ts.UnixNano() / 1e6}},
},
},
})
@@ -273,22 +277,34 @@ func TestAlertingRule_Exec(t *testing.T) {
4: {{labels: []string{"name", "foo"}, alert: &notifier.Alert{State: notifier.StateInactive}}},
}, map[int][]prompbmarshal.TimeSeries{
0: {
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "single-firing=>inactive=>firing=>inactive=>inactive"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "single-firing=>inactive=>firing=>inactive=>inactive"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Unix()), Timestamp: ts.UnixNano() / 1e6}}},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "single-firing=>inactive=>firing=>inactive=>inactive"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "single-firing=>inactive=>firing=>inactive=>inactive"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Unix()), Timestamp: ts.UnixNano() / 1e6}},
},
},
1: {
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "single-firing=>inactive=>firing=>inactive=>inactive"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "single-firing=>inactive=>firing=>inactive=>inactive"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}}},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "single-firing=>inactive=>firing=>inactive=>inactive"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "single-firing=>inactive=>firing=>inactive=>inactive"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}},
},
},
2: {
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "single-firing=>inactive=>firing=>inactive=>inactive"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "single-firing=>inactive=>firing=>inactive=>inactive"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Add(2 * defaultStep).Unix()), Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}}},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "single-firing=>inactive=>firing=>inactive=>inactive"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "single-firing=>inactive=>firing=>inactive=>inactive"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Add(2 * defaultStep).Unix()), Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}},
},
},
})
@@ -344,34 +360,54 @@ func TestAlertingRule_Exec(t *testing.T) {
},
}, map[int][]prompbmarshal.TimeSeries{
0: {
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Unix()), Timestamp: ts.UnixNano() / 1e6}}},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Unix()), Timestamp: ts.UnixNano() / 1e6}},
},
},
1: {
// stale time series for foo, `firing -> inactive`
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}}},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}},
},
// new time series for foo1
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo1"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "name", Value: "foo1"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Add(defaultStep).Unix()), Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}}},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo1"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "name", Value: "foo1"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Add(defaultStep).Unix()), Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}},
},
},
2: {
// stale time series for foo1
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo1"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "name", Value: "foo1"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}}},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo1"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "name", Value: "foo1"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}},
},
// new time series for foo2
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo2"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "name", Value: "foo2"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Add(2 * defaultStep).Unix()), Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}}},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo2"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "multiple-steps-firing"}, {Name: "name", Value: "foo2"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Add(2 * defaultStep).Unix()), Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}},
},
},
})
@@ -389,50 +425,72 @@ func TestAlertingRule_Exec(t *testing.T) {
1: {{labels: []string{"name", "foo"}, alert: &notifier.Alert{State: notifier.StateFiring}}},
}, map[int][]prompbmarshal.TimeSeries{
0: {
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "for-fired"}, {Name: "alertstate", Value: "pending"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "for-fired"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Unix()), Timestamp: ts.UnixNano() / 1e6}}},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "for-fired"}, {Name: "alertstate", Value: "pending"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "for-fired"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Unix()), Timestamp: ts.UnixNano() / 1e6}},
},
},
1: {
// stale time series for `pending -> firing`
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "for-fired"}, {Name: "alertstate", Value: "pending"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "for-fired"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "for-fired"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Add(defaultStep).Unix()), Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}}},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "for-fired"}, {Name: "alertstate", Value: "pending"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "for-fired"}, {Name: "alertstate", Value: "firing"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "for-fired"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Add(defaultStep).Unix()), Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}},
},
},
})
f(newTestAlertingRule("for-pending=>empty", time.Second), [][]datasource.Metric{
{metricWithLabels(t, "name", "foo")},
{metricWithLabels(t, "name", "foo")},
{metricWithLabels(t, "name", "foo", "a1", "b1", "a2", "b2", "a3", "b3")},
{metricWithLabels(t, "name", "foo", "a1", "b1", "a2", "b2", "a3", "b3")},
// empty step to delete pending alerts
{},
}, map[int][]testAlert{
0: {{labels: []string{"name", "foo"}, alert: &notifier.Alert{State: notifier.StatePending}}},
1: {{labels: []string{"name", "foo"}, alert: &notifier.Alert{State: notifier.StatePending}}},
0: {{labels: []string{"name", "foo", "a1", "b1", "a2", "b2", "a3", "b3"}, alert: &notifier.Alert{State: notifier.StatePending}}},
1: {{labels: []string{"name", "foo", "a1", "b1", "a2", "b2", "a3", "b3"}, alert: &notifier.Alert{State: notifier.StatePending}}},
2: {},
}, map[int][]prompbmarshal.TimeSeries{
0: {
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "for-pending=>empty"}, {Name: "alertstate", Value: "pending"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "for-pending=>empty"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Unix()), Timestamp: ts.UnixNano() / 1e6}}},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "a1", Value: "b1"}, {Name: "a2", Value: "b2"}, {Name: "a3", Value: "b3"}, {Name: "alertname", Value: "for-pending=>empty"}, {Name: "alertstate", Value: "pending"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "a1", Value: "b1"}, {Name: "a2", Value: "b2"}, {Name: "a3", Value: "b3"}, {Name: "alertname", Value: "for-pending=>empty"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Unix()), Timestamp: ts.UnixNano() / 1e6}},
},
},
1: {
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "for-pending=>empty"}, {Name: "alertstate", Value: "pending"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "for-pending=>empty"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Unix()), Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}}},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "a1", Value: "b1"}, {Name: "a2", Value: "b2"}, {Name: "a3", Value: "b3"}, {Name: "alertname", Value: "for-pending=>empty"}, {Name: "alertstate", Value: "pending"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: 1, Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "a1", Value: "b1"}, {Name: "a2", Value: "b2"}, {Name: "a3", Value: "b3"}, {Name: "alertname", Value: "for-pending=>empty"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: float64(ts.Unix()), Timestamp: ts.Add(defaultStep).UnixNano() / 1e6}},
},
},
// stale time series for `pending -> inactive`
2: {
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "alertname", Value: "for-pending=>empty"}, {Name: "alertstate", Value: "pending"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}}},
{Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "alertname", Value: "for-pending=>empty"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}}},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertMetricName}, {Name: "a1", Value: "b1"}, {Name: "a2", Value: "b2"}, {Name: "a3", Value: "b3"}, {Name: "alertname", Value: "for-pending=>empty"}, {Name: "alertstate", Value: "pending"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}},
},
{
Labels: []prompbmarshal.Label{{Name: "__name__", Value: alertForStateMetricName}, {Name: "a1", Value: "b1"}, {Name: "a2", Value: "b2"}, {Name: "a3", Value: "b3"}, {Name: "alertname", Value: "for-pending=>empty"}, {Name: "name", Value: "foo"}},
Samples: []prompbmarshal.Sample{{Value: decimal.StaleNaN, Timestamp: ts.Add(2*defaultStep).UnixNano() / 1e6}},
},
},
})

View File

@@ -7,11 +7,13 @@ import (
"flag"
"fmt"
"math"
"net"
"net/http"
"net/url"
"os"
"regexp"
"sort"
"strconv"
"strings"
"sync"
"sync/atomic"
@@ -348,6 +350,7 @@ func (up *URLPrefix) discoverBackendAddrsIfNeeded() {
hostToAddrs := make(map[string][]string)
for _, bu := range up.busOriginal {
host := bu.Hostname()
port := bu.Port()
if hostToAddrs[host] != nil {
// ips for the given host have been already discovered
continue
@@ -364,7 +367,11 @@ func (up *URLPrefix) discoverBackendAddrsIfNeeded() {
} else {
resolvedAddrs = make([]string, len(addrs))
for i, addr := range addrs {
resolvedAddrs[i] = fmt.Sprintf("%s:%d", addr.Target, addr.Port)
hostPort := port
if hostPort == "" && addr.Port > 0 {
hostPort = strconv.FormatUint(uint64(addr.Port), 10)
}
resolvedAddrs[i] = net.JoinHostPort(addr.Target, hostPort)
}
}
} else {
@@ -375,7 +382,7 @@ func (up *URLPrefix) discoverBackendAddrsIfNeeded() {
} else {
resolvedAddrs = make([]string, len(addrs))
for i, addr := range addrs {
resolvedAddrs[i] = addr.String()
resolvedAddrs[i] = net.JoinHostPort(addr.String(), port)
}
}
}
@@ -389,17 +396,9 @@ func (up *URLPrefix) discoverBackendAddrsIfNeeded() {
var busNew []*backendURL
for _, bu := range up.busOriginal {
host := bu.Hostname()
port := bu.Port()
for _, addr := range hostToAddrs[host] {
buCopy := *bu
buCopy.Host = addr
if port != "" {
if n := strings.IndexByte(buCopy.Host, ':'); n >= 0 {
// Drop the discovered port and substitute it the port specified in bu.
buCopy.Host = buCopy.Host[:n]
}
buCopy.Host += ":" + port
}
busNew = append(busNew, &backendURL{
url: &buCopy,
})

View File

@@ -3,12 +3,14 @@ package main
import (
"bytes"
"fmt"
"net"
"net/url"
"testing"
"gopkg.in/yaml.v2"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/netutil"
)
func TestParseAuthConfigFailure(t *testing.T) {
@@ -799,6 +801,75 @@ func TestBrokenBackend(t *testing.T) {
}
}
func TestDiscoverBackendIPsWithIPV6(t *testing.T) {
f := func(actualUrl, expectedUrl string) {
t.Helper()
up := mustParseURL(actualUrl)
up.discoverBackendIPs = true
up.loadBalancingPolicy = "least_loaded"
up.discoverBackendAddrsIfNeeded()
pbus := up.bus.Load()
bus := *pbus
if len(bus) != 1 {
t.Fatalf("expected url list to be of size 1; got %d instead", len(bus))
}
got := bus[0].url.Host
if got != expectedUrl {
t.Fatalf(`expected url to be %q; got %q instead`, expectedUrl, bus[0].url.Host)
}
}
// Discover backendURL with SRV hostnames
customResolver := &fakeResolver{
Resolver: &net.Resolver{},
// SRV records must return hostname
// not an IP address
lookupSRVResults: map[string][]*net.SRV{
"_vmselect._tcp.selectwithport.": {
{
Target: "vmselect.local",
Port: 8481,
},
},
"_vmselect._tcp.selectwoport.": {
{
Target: "vmselect.local",
},
},
},
lookupIPAddrResults: map[string][]net.IPAddr{
"vminsert.local": {
{
IP: net.ParseIP("10.0.10.13"),
},
},
"ipv6.vminsert.local": {
{
IP: net.ParseIP("2607:f8b0:400a:80b::200e"),
},
},
},
}
origResolver := netutil.Resolver
netutil.Resolver = customResolver
defer func() {
netutil.Resolver = origResolver
}()
f("http://srv+_vmselect._tcp.selectwithport.:8080", "vmselect.local:8080")
f("http://srv+_vmselect._tcp.selectwithport.:", "vmselect.local:8481")
f("http://srv+_vmselect._tcp.selectwoport.:8080", "vmselect.local:8080")
f("http://srv+_vmselect._tcp.selectwoport.", "vmselect.local:")
f("http://vminsert.local:8080", "10.0.10.13:8080")
f("http://vminsert.local", "10.0.10.13:")
f("http://ipv6.vminsert.local:8080", "[2607:f8b0:400a:80b::200e]:8080")
f("http://ipv6.vminsert.local", "[2607:f8b0:400a:80b::200e]:")
}
func getRegexs(paths []string) []*Regex {
var sps []*Regex
for _, path := range paths {

View File

@@ -213,7 +213,7 @@ func processRequest(w http.ResponseWriter, r *http.Request, ui *UserInfo) {
missingRouteRequests.Inc()
var di string
if ui.DumpRequestOnErrors {
di = debugInfo(u, r.Header)
di = debugInfo(u, r)
}
httpserver.Errorf(w, r, "missing route for %q%s", u.String(), di)
return
@@ -668,13 +668,13 @@ func (rtb *readTrackingBody) Close() error {
return nil
}
func debugInfo(u *url.URL, h http.Header) string {
func debugInfo(u *url.URL, r *http.Request) string {
s := &strings.Builder{}
fmt.Fprintf(s, " (host: %q; ", u.Host)
fmt.Fprintf(s, " (host: %q; ", r.Host)
fmt.Fprintf(s, "path: %q; ", u.Path)
fmt.Fprintf(s, "args: %q; ", u.Query().Encode())
fmt.Fprint(s, "headers:")
_ = h.WriteSubset(s, nil)
_ = r.Header.WriteSubset(s, nil)
fmt.Fprint(s, ")")
return s.String()
}

View File

@@ -180,11 +180,7 @@ func (c *Client) Explore() ([]*Series, error) {
log.Printf("skip measurement %q since it has no fields", s.Measurement)
continue
}
tags, ok := measurementTags[s.Measurement]
if !ok {
return nil, fmt.Errorf("failed to find tags of measurement %s", s.Measurement)
}
emptyTags := getEmptyTags(tags, s.LabelPairs)
emptyTags := getEmptyTags(measurementTags[s.Measurement], s.LabelPairs)
for _, field := range fields {
is := &Series{
Measurement: s.Measurement,
@@ -201,11 +197,16 @@ func (c *Client) Explore() ([]*Series, error) {
// getEmptyTags returns tags of a measurement that are missing in a specific series.
// Tags represent all tags of a measurement. LabelPairs represent tags of a specific series.
func getEmptyTags(tags map[string]struct{}, LabelPairs []LabelPair) []string {
if len(tags) == 0 {
// fast path: the measurement does not contain any tag
return nil
}
labelMap := make(map[string]struct{})
for _, pair := range LabelPairs {
labelMap[pair.Name] = struct{}{}
}
result := make([]string, 0, len(labelMap)-len(LabelPairs))
var result []string
for tag := range tags {
if _, ok := labelMap[tag]; !ok {
result = append(result, tag)

View File

@@ -4,16 +4,48 @@ import (
"fmt"
"net/http"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vminsert/relabel"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmstorage"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/prompbmarshal"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/ratelimiter"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/slicesutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/timeserieslimits"
)
// StartIngestionRateLimiter starts ingestion rate limiter.
//
// Ingestion rate limiter must be started before Init() call.
//
// StopIngestionRateLimiter must be called before Stop() call in order to unblock all the callers
// to ingestion rate limiter. Otherwise deadlock may occur at Stop() call.
func StartIngestionRateLimiter(maxIngestionRate int) {
if maxIngestionRate <= 0 {
return
}
ingestionRateLimitReached := metrics.NewCounter(`vm_max_ingestion_rate_limit_reached_total`)
ingestionRateLimiterStopCh = make(chan struct{})
ingestionRateLimiter = ratelimiter.New(int64(maxIngestionRate), ingestionRateLimitReached, ingestionRateLimiterStopCh)
}
// StopIngestionRateLimiter stops ingestion rate limiter.
func StopIngestionRateLimiter() {
if ingestionRateLimiterStopCh == nil {
return
}
close(ingestionRateLimiterStopCh)
ingestionRateLimiterStopCh = nil
}
var (
ingestionRateLimiter *ratelimiter.RateLimiter
ingestionRateLimiterStopCh chan struct{}
)
// InsertCtx contains common bits for data points insertion.
type InsertCtx struct {
Labels sortedLabels
@@ -73,7 +105,7 @@ func (ctx *InsertCtx) TryPrepareLabels(hasRelabeling bool) bool {
if timeserieslimits.Enabled() && timeserieslimits.IsExceeding(ctx.Labels) {
return false
}
ctx.sortLabelsIfNeeded()
ctx.SortLabelsIfNeeded()
return true
}
@@ -172,9 +204,12 @@ func (ctx *InsertCtx) FlushBufs() error {
}
matchIdxsPool.Put(matchIdxs)
}
ingestionRateLimiter.Register(len(ctx.mrs))
// There is no need in limiting the number of concurrent calls to vmstorage.AddRows() here,
// since the number of concurrent FlushBufs() calls should be already limited via writeconcurrencylimiter
// used at every stream.Parse() call under lib/protoparser/*
err := vmstorage.AddRows(ctx.mrs)
ctx.Reset(0)
if err == nil {

View File

@@ -12,8 +12,8 @@ var sortLabels = flag.Bool("sortLabels", false, `Whether to sort labels for inco
`For example, if m{k1="v1",k2="v2"} may be sent as m{k2="v2",k1="v1"}. `+
`Enabled sorting for labels can slow down ingestion performance a bit`)
// sortLabelsIfNeeded sorts labels if -sortLabels command-line flag is set
func (ctx *InsertCtx) sortLabelsIfNeeded() {
// SortLabelsIfNeeded sorts labels if -sortLabels command-line flag is set
func (ctx *InsertCtx) SortLabelsIfNeeded() {
if *sortLabels {
sort.Sort(&ctx.Labels)
}

View File

@@ -118,8 +118,14 @@ func insertRows(db string, rows []parser.Row, extraLabels []prompbmarshal.Label)
}
}
} else {
if !ic.TryPrepareLabels(false) {
continue
// special case for optimisations below
// do not call TryPrepareLabels
// manually apply sort and limits on demand
ic.SortLabelsIfNeeded()
if hasLimitsEnabled {
if timeserieslimits.IsExceeding(ic.Labels) {
continue
}
}
ctx.metricNameBuf = storage.MarshalMetricNameRaw(ctx.metricNameBuf[:0], ic.Labels)
labelsLen := len(ic.Labels)
@@ -132,8 +138,6 @@ func insertRows(db string, rows []parser.Row, extraLabels []prompbmarshal.Label)
ic.Labels = ic.Labels[:labelsLen]
ic.AddLabel("", metricGroup)
if hasLimitsEnabled {
// special case for optimisation above
// check only __name__ label value limits
if timeserieslimits.IsExceeding(ic.Labels[len(ic.Labels)-1:]) {
continue
}

View File

@@ -46,6 +46,8 @@ var (
"so there is no need in spending additional CPU time on its handling. Staleness markers may exist only in data obtained from Prometheus scrape targets")
minWindowForInstantRollupOptimization = flag.Duration("search.minWindowForInstantRollupOptimization", time.Hour*3, "Enable cache-based optimization for repeated queries "+
"to /api/v1/query (aka instant queries), which contain rollup functions with lookbehind window exceeding the given value")
maxBinaryOpPushdownLabelValues = flag.Int("search.maxBinaryOpPushdownLabelValues", 100, "The maximum number of values for a label in the first expression that can be extracted as a common label filter and pushed down to the second expression in a binary operation. "+
"A larger value makes the pushed-down filter more complex but fewer time series will be returned. This flag is useful when selective label contains numerous values, for example `instance`, and storage resources are abundant.")
)
// The minimum number of points per timeseries for enabling time rounding.
@@ -582,7 +584,7 @@ func getCommonLabelFilters(tss []*timeseries) []metricsql.LabelFilter {
}
continue
}
if len(vc.values) > 100 {
if len(vc.values) > *maxBinaryOpPushdownLabelValues {
// Too many unique values found for the given tag.
// Do not make a filter on such values, since it may slow down
// search for matching time series.

View File

@@ -6,8 +6,6 @@ import (
"math"
"sort"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/netstorage"
@@ -16,7 +14,6 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/querytracer"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/metricsql"
)
@@ -270,7 +267,7 @@ func getReverseCmpOp(op string) string {
}
func parsePromQLWithCache(q string) (metricsql.Expr, error) {
pcv := parseCacheV.Get(q)
pcv := parseCacheV.get(q)
if pcv == nil {
e, err := metricsql.Parse(q)
if err == nil {
@@ -284,7 +281,7 @@ func parsePromQLWithCache(q string) (metricsql.Expr, error) {
e: e,
err: err,
}
parseCacheV.Put(q, pcv)
parseCacheV.put(q, pcv)
}
if pcv.err != nil {
return nil, pcv.err
@@ -328,80 +325,3 @@ func escapeDots(s string) string {
}
return string(result)
}
var parseCacheV = func() *parseCache {
pc := &parseCache{
m: make(map[string]*parseCacheValue),
}
metrics.NewGauge(`vm_cache_requests_total{type="promql/parse"}`, func() float64 {
return float64(pc.Requests())
})
metrics.NewGauge(`vm_cache_misses_total{type="promql/parse"}`, func() float64 {
return float64(pc.Misses())
})
metrics.NewGauge(`vm_cache_entries{type="promql/parse"}`, func() float64 {
return float64(pc.Len())
})
return pc
}()
const parseCacheMaxLen = 10e3
type parseCacheValue struct {
e metricsql.Expr
err error
}
type parseCache struct {
requests atomic.Uint64
misses atomic.Uint64
m map[string]*parseCacheValue
mu sync.RWMutex
}
func (pc *parseCache) Requests() uint64 {
return pc.requests.Load()
}
func (pc *parseCache) Misses() uint64 {
return pc.misses.Load()
}
func (pc *parseCache) Len() uint64 {
pc.mu.RLock()
n := len(pc.m)
pc.mu.RUnlock()
return uint64(n)
}
func (pc *parseCache) Get(q string) *parseCacheValue {
pc.requests.Add(1)
pc.mu.RLock()
pcv := pc.m[q]
pc.mu.RUnlock()
if pcv == nil {
pc.misses.Add(1)
}
return pcv
}
func (pc *parseCache) Put(q string, pcv *parseCacheValue) {
pc.mu.Lock()
overflow := len(pc.m) - parseCacheMaxLen
if overflow > 0 {
// Remove 10% of items from the cache.
overflow = int(float64(len(pc.m)) * 0.1)
for k := range pc.m {
delete(pc.m, k)
overflow--
if overflow <= 0 {
break
}
}
}
pc.m[q] = pcv
pc.mu.Unlock()
}

View File

@@ -0,0 +1,142 @@
// Cache for metricsql expressions
// Based on the fastcache idea of locking buckets in order to avoid whole cache locks.
// See: https://github.com/VictoriaMetrics/fastcache
package promql
import (
"sync"
"sync/atomic"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/metricsql"
xxhash "github.com/cespare/xxhash/v2"
)
var parseCacheV = func() *parseCache {
pc := newParseCache()
metrics.NewGauge(`vm_cache_requests_total{type="promql/parse"}`, func() float64 {
return float64(pc.requests())
})
metrics.NewGauge(`vm_cache_misses_total{type="promql/parse"}`, func() float64 {
return float64(pc.misses())
})
metrics.NewGauge(`vm_cache_entries{type="promql/parse"}`, func() float64 {
return float64(pc.len())
})
return pc
}()
const (
parseBucketCount = 128
parseCacheMaxLen int = 10e3
parseBucketMaxLen int = parseCacheMaxLen / parseBucketCount
parseBucketFreePercent float64 = 0.1
)
type parseCacheValue struct {
e metricsql.Expr
err error
}
type parseBucket struct {
m map[string]*parseCacheValue
mu sync.RWMutex
requests atomic.Uint64
misses atomic.Uint64
}
type parseCache struct {
buckets [parseBucketCount]parseBucket
}
func newParseCache() *parseCache {
pc := new(parseCache)
for i := 0; i < parseBucketCount; i++ {
pc.buckets[i] = newParseBucket()
}
return pc
}
func (pc *parseCache) put(q string, pcv *parseCacheValue) {
h := xxhash.Sum64String(q)
idx := h % parseBucketCount
pc.buckets[idx].put(q, pcv)
}
func (pc *parseCache) get(q string) *parseCacheValue {
h := xxhash.Sum64String(q)
idx := h % parseBucketCount
return pc.buckets[idx].get(q)
}
func (pc *parseCache) requests() uint64 {
var n uint64
for i := 0; i < parseBucketCount; i++ {
n += pc.buckets[i].requests.Load()
}
return n
}
func (pc *parseCache) misses() uint64 {
var n uint64
for i := 0; i < parseBucketCount; i++ {
n += pc.buckets[i].misses.Load()
}
return n
}
func (pc *parseCache) len() uint64 {
var n uint64
for i := 0; i < parseBucketCount; i++ {
n += pc.buckets[i].len()
}
return n
}
func newParseBucket() parseBucket {
return parseBucket{
m: make(map[string]*parseCacheValue, parseBucketMaxLen),
}
}
func (pb *parseBucket) len() uint64 {
pb.mu.RLock()
n := len(pb.m)
pb.mu.RUnlock()
return uint64(n)
}
func (pb *parseBucket) get(q string) *parseCacheValue {
pb.requests.Add(1)
pb.mu.RLock()
pcv := pb.m[q]
pb.mu.RUnlock()
if pcv == nil {
pb.misses.Add(1)
}
return pcv
}
func (pb *parseBucket) put(q string, pcv *parseCacheValue) {
pb.mu.Lock()
overflow := len(pb.m) - parseBucketMaxLen
if overflow > 0 {
// Remove parseBucketDeletePercent*100 % of items from the bucket.
overflow = int(float64(len(pb.m)) * parseBucketFreePercent)
for k := range pb.m {
delete(pb.m, k)
overflow--
if overflow <= 0 {
break
}
}
}
pb.m[q] = pcv
pb.mu.Unlock()
}

View File

@@ -0,0 +1,129 @@
package promql
import (
"fmt"
"testing"
"github.com/VictoriaMetrics/metricsql"
)
func testGetParseCacheValue(q string) *parseCacheValue {
e, err := metricsql.Parse(q)
return &parseCacheValue{
e: e,
err: err,
}
}
func testGenerateQueries(items int) []string {
queries := make([]string, items)
for i := 0; i < items; i++ {
queries[i] = fmt.Sprintf(`node_time_seconds{instance="node%d", job="job%d"}`, i, i)
}
return queries
}
func TestParseCache(t *testing.T) {
pc := newParseCache()
if pc.len() != 0 || pc.misses() != 0 || pc.requests() != 0 {
t.Errorf("unexpected pc.Len()=%d, pc.Misses()=%d, pc.Requests()=%d; expected all to be zero.", pc.len(), pc.misses(), pc.requests())
}
q1 := `foo{bar="baz"}`
v1 := testGetParseCacheValue(q1)
q2 := `foo1{bar1="baz1"}`
v2 := testGetParseCacheValue(q2)
pc.put(q1, v1)
if pc.len() != 1 {
t.Errorf("unexpected value obtained; got %d; want %d", pc.len(), 1)
}
if res := pc.get(q2); res != nil {
t.Errorf("unexpected non-empty value obtained from cache: %d ", res)
}
if pc.len() != 1 {
t.Errorf("unexpected value obtained; got %d; want %d", pc.len(), 1)
}
if miss := pc.misses(); miss != 1 {
t.Errorf("unexpected value obtained; got %d; want %d", miss, 1)
}
if req := pc.requests(); req != 1 {
t.Errorf("unexpected value obtained; got %d; want %d", req, 1)
}
pc.put(q2, v2)
if pc.len() != 2 {
t.Errorf("unexpected value obtained; got %d; want %d", pc.len(), 2)
}
if res := pc.get(q1); res != v1 {
t.Errorf("unexpected value obtained; got %v; want %v", res, v1)
}
if res := pc.get(q2); res != v2 {
t.Errorf("unexpected value obtained; got %v; want %v", res, v2)
}
pc.put(q2, v2)
if pc.len() != 2 {
t.Errorf("unexpected value obtained; got %d; want %d", pc.len(), 2)
}
if miss := pc.misses(); miss != 1 {
t.Errorf("unexpected value obtained; got %d; want %d", miss, 1)
}
if req := pc.requests(); req != 3 {
t.Errorf("unexpected value obtained; got %d; want %d", req, 3)
}
if res := pc.get(q2); res != v2 {
t.Errorf("unexpected value obtained; got %v; want %v", res, v2)
}
if pc.len() != 2 {
t.Errorf("unexpected value obtained; got %d; want %d", pc.len(), 2)
}
if miss := pc.misses(); miss != 1 {
t.Errorf("unexpected value obtained; got %d; want %d", miss, 1)
}
if req := pc.requests(); req != 4 {
t.Errorf("unexpected value obtained; got %d; want %d", req, 4)
}
}
func TestParseCacheBucketOverflow(t *testing.T) {
b := newParseBucket()
var expectedLen uint64
// +2 for overflow and clean up
queries := testGenerateQueries(parseBucketMaxLen + 2)
// Same value for all keys
v := testGetParseCacheValue(queries[0])
// Fill bucket
for i := 0; i < parseBucketMaxLen; i++ {
b.put(queries[i], v)
}
expectedLen = uint64(parseBucketMaxLen)
if b.len() != expectedLen {
t.Errorf("unexpected value obtained; got %v; want %v", b.len(), expectedLen)
}
// Overflow bucket
expectedLen = uint64(parseBucketMaxLen + 1)
b.put(queries[parseBucketMaxLen], v)
if b.len() != uint64(expectedLen) {
t.Errorf("unexpected value obtained; got %v; want %v", b.len(), expectedLen)
}
// Clean up;
oldLen := b.len()
overflow := int(float64(oldLen) * parseBucketFreePercent)
expectedLen = oldLen - uint64(overflow) + 1 // +1 for new entry
b.put(queries[parseBucketMaxLen+1], v)
if b.len() != expectedLen {
t.Errorf("unexpected value obtained; got %v; want %v", b.len(), expectedLen)
}
}

View File

@@ -0,0 +1,235 @@
package promql
import (
"testing"
)
func BenchmarkCachePutNoOverFlow(b *testing.B) {
const items int = (parseCacheMaxLen / 2)
pc := newParseCache()
queries := testGenerateQueries(items)
v := testGetParseCacheValue(queries[0])
b.ResetTimer()
b.ReportAllocs()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
for i := 0; i < items; i++ {
pc.put(queries[i], v)
}
}
})
if pc.len() != uint64(items) {
b.Errorf("unexpected value obtained; got %d; want %d", pc.len(), items)
}
}
func BenchmarkCacheGetNoOverflow(b *testing.B) {
const items int = parseCacheMaxLen / 2
pc := newParseCache()
queries := testGenerateQueries(items)
v := testGetParseCacheValue(queries[0])
for i := 0; i < len(queries); i++ {
pc.put(queries[i], v)
}
b.ResetTimer()
b.ReportAllocs()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
for i := 0; i < items; i++ {
if v := pc.get(queries[i]); v == nil {
b.Errorf("unexpected nil value obtained from cache for query: %s ", queries[i])
}
}
}
})
}
func BenchmarkCachePutGetNoOverflow(b *testing.B) {
const items int = parseCacheMaxLen / 2
pc := newParseCache()
queries := testGenerateQueries(items)
v := testGetParseCacheValue(queries[0])
b.ResetTimer()
b.ReportAllocs()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
for i := 0; i < items; i++ {
pc.put(queries[i], v)
if res := pc.get(queries[i]); res == nil {
b.Errorf("unexpected nil value obtained from cache for query: %s ", queries[i])
}
}
}
})
if pc.len() != uint64(items) {
b.Errorf("unexpected value obtained; got %d; want %d", pc.len(), items)
}
}
func BenchmarkCachePutOverflow(b *testing.B) {
const items int = parseCacheMaxLen + (parseCacheMaxLen / 2)
c := newParseCache()
queries := testGenerateQueries(items)
v := testGetParseCacheValue(queries[0])
for i := 0; i < parseCacheMaxLen; i++ {
c.put(queries[i], v)
}
b.ReportAllocs()
b.ResetTimer()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
for i := parseCacheMaxLen; i < items; i++ {
c.put(queries[i], v)
}
}
})
maxElemnts := uint64(parseCacheMaxLen + parseBucketCount)
if c.len() > maxElemnts {
b.Errorf("cache length is more than expected; got %d, expected %d", c.len(), maxElemnts)
}
}
func BenchmarkCachePutGetOverflow(b *testing.B) {
const items int = parseCacheMaxLen + (parseCacheMaxLen / 2)
c := newParseCache()
queries := testGenerateQueries(items)
v := testGetParseCacheValue(queries[0])
for i := 0; i < parseCacheMaxLen; i++ {
c.put(queries[i], v)
}
b.ReportAllocs()
b.ResetTimer()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
for i := parseCacheMaxLen; i < items; i++ {
c.put(queries[i], v)
c.get(queries[i])
}
}
})
maxElemnts := uint64(parseCacheMaxLen + parseBucketCount)
if c.len() > maxElemnts {
b.Errorf("cache length is more than expected; got %d, expected %d", c.len(), maxElemnts)
}
}
var testSimpleQueries = []string{
`m{a="b"}`,
`{a="b"}`,
`m{c="d",a="b"}`,
`{a="b",c="d"}`,
`m1{a="foo"}`,
`m2{a="bar"}`,
`m1{b="foo"}`,
`m2{b="bar"}`,
`m1{a="foo",b="bar"}`,
`m2{b="bar",c="x"}`,
`{b="bar"}`,
}
func BenchmarkParsePromQLWithCacheSimple(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
for j := 0; j < len(testSimpleQueries); j++ {
_, err := parsePromQLWithCache(testSimpleQueries[j])
if err != nil {
b.Errorf("unexpected error: %s", err)
}
}
}
}
func BenchmarkParsePromQLWithCacheSimpleParallel(b *testing.B) {
b.ReportAllocs()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
for i := 0; i < len(testSimpleQueries); i++ {
_, err := parsePromQLWithCache(testSimpleQueries[i])
if err != nil {
b.Errorf("unexpected error: %s", err)
}
}
}
})
}
var testComplexQueries = []string{
`sort_desc(label_set(2, "foo", "bar") * ignoring(a) (label_set(time(), "foo", "bar") or label_set(10, "foo", "qwert")))`,
`sum(a.b{c="d.e",x=~"a.b.+[.a]",y!~"aaa.bb|cc.dd"}) + avg_over_time(1,sum({x=~"aa.bb"}))`,
`sort((label_set(time() offset 100s, "foo", "bar"), label_set(time()+10, "foo", "baz") offset 50s) offset 400s)`,
`sort(label_map((
label_set(time(), "label", "v1"),
label_set(time()+100, "label", "v2"),
label_set(time()+200, "label", "v3"),
label_set(time()+300, "x", "y"),
label_set(time()+400, "label", "v4"),
), "label", "v1", "foo", "v2", "bar", "", "qwe", "v4", ""))`,
`sort(labels_equal((
label_set(10, "instance", "qwe", "host", "rty"),
label_set(20, "instance", "qwe", "host", "qwe"),
label_set(30, "aaa", "bbb", "instance", "foo", "host", "foo"),
), "instance", "host"))`,
`with (
x = (
label_set(time() > 1500, "foo", "123.456", "__name__", "aaa"),
label_set(-time(), "foo", "bar", "__name__", "bbb"),
label_set(-time(), "__name__", "bxs"),
label_set(-time(), "foo", "45", "bar", "xs"),
)
)
sort(x + label_value(x, "foo"))`,
`label_replace(
label_replace(
label_replace(time(), "__name__", "x${1}y", "foo", ".*"),
"xxx", "foo${1}bar(${1})", "__name__", "(.+)"),
"xxx", "AA$1", "xxx", "foox(.+)"
)`,
`sort_desc(union(
label_set(time() > 1400, "__name__", "x", "foo", "bar"),
label_set(time() < 1700, "__name__", "y", "foo", "baz")) default 123)`,
`sort(histogram_quantile(0.6,
label_set(90, "foo", "bar", "le", "10")
or label_set(100, "foo", "bar", "le", "30")
or label_set(300, "foo", "bar", "le", "+Inf")
or label_set(200, "tag", "xx", "le", "10")
or label_set(300, "tag", "xx", "le", "30")
))`,
}
func BenchmarkParsePromQLWithCacheComplex(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
for j := 0; j < len(testComplexQueries); j++ {
_, err := parsePromQLWithCache(testComplexQueries[j])
if err != nil {
b.Errorf("unexpected error: %s", err)
}
}
}
}
func BenchmarkParsePromQLWithCacheComplexParallel(b *testing.B) {
b.ReportAllocs()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
for i := 0; i < len(testComplexQueries); i++ {
_, err := parsePromQLWithCache(testComplexQueries[i])
if err != nil {
b.Errorf("unexpected error: %s", err)
}
}
}
})
}

View File

@@ -48,3 +48,16 @@ export interface LogHits {
[key: string]: string;
};
}
export interface ReportMetaData {
id: number;
title: string;
endpoint: string;
comment: string;
params: Record<string, string>;
}
export interface LogsFiledValues {
value: string;
hits: number;
}

View File

@@ -0,0 +1,154 @@
import React, { FC, useCallback, useEffect, useMemo, useState } from "preact/compat";
import Autocomplete, { AutocompleteOptions } from "../../../Main/Autocomplete/Autocomplete";
import { AUTOCOMPLETE_LIMITS } from "../../../../constants/queryAutocomplete";
import { QueryEditorAutocompleteProps } from "../QueryEditor";
import { getContextData, splitLogicalParts } from "./parser";
import { ContextType, LogicalPart, LogicalPartType } from "./types";
import { useFetchLogsQLOptions } from "./useFetchLogsQLOptions";
import { pipeList } from "./pipes";
const LogsQueryEditorAutocomplete: FC<QueryEditorAutocompleteProps> = ({
value,
anchorEl,
caretPosition,
hasHelperText,
onSelect,
onFoundOptions
}) => {
const [offsetPos, setOffsetPos] = useState({ top: 0, left: 0 });
const fullValue = useMemo(() => {
if (caretPosition[0] !== caretPosition[1]) return { valueBeforeCursor: value, valueAfterCursor: "" };
const valueBeforeCursor = value.substring(0, caretPosition[0]);
const valueAfterCursor = value.substring(caretPosition[1]);
return { valueBeforeCursor, valueAfterCursor };
}, [value, caretPosition]);
const logicalParts = useMemo(() => {
return splitLogicalParts(value);
}, [value]);
const contextData = useMemo(() => {
if (caretPosition[0] !== caretPosition[1]) return;
const part = logicalParts.find(p => caretPosition[0] >= p.position[0] && caretPosition[0] <= p.position[1]);
if (!part) return;
const cursorStartPosition = caretPosition[0] - part.position[0];
return {
...part,
...getContextData(part, cursorStartPosition)
};
}, [logicalParts, caretPosition]);
const { fieldNames, fieldValues, loading } = useFetchLogsQLOptions(contextData);
const options = useMemo(() => {
switch (contextData?.contextType) {
case ContextType.FilterName:
case ContextType.FilterUnknown:
return fieldNames;
case ContextType.FilterValue:
return fieldValues;
case ContextType.PipeName:
return pipeList;
default:
return [];
}
}, [contextData, fieldNames, fieldValues]);
const getUpdatedValue = (insertValue: string, logicalParts: LogicalPart[], id?: number) => {
return logicalParts.reduce((acc, part) => {
const value = part.id === id ? insertValue : part.value;
const separator = part.type === LogicalPartType.Pipe ? " | " : " ";
return `${acc}${separator}${value}`;
}, "").trim();
};
const getModifyInsert = (insert: string, contextType: ContextType, value = "", insertType?: string) => {
let modifiedInsert = insert;
if (insertType === ContextType.FilterName) {
modifiedInsert += ":";
} else if (contextType === ContextType.FilterValue) {
const insertWithQuotes = value.startsWith("_stream:") ? modifiedInsert : `"${modifiedInsert}"`;
modifiedInsert = `${contextData?.filterName || ""}:${insertWithQuotes}`;
}
return modifiedInsert;
};
const handleSelect = useCallback((insert: string, item: AutocompleteOptions) => {
const {
id,
contextType = ContextType.FilterUnknown,
value = "",
position = [0, 0]
} = contextData || {};
const insertValue = getModifyInsert(insert, contextType, value, item.type);
const newValue = getUpdatedValue(insertValue, logicalParts, id);
const updatedPosition = (position[0] || 1) + insertValue.length + (item.type === ContextType.PipeName ? 1 : 0);
onSelect(newValue, updatedPosition);
}, [contextData, logicalParts]);
useEffect(() => {
if (!anchorEl.current) {
setOffsetPos({ top: 0, left: 0 });
return;
}
const element = anchorEl.current.querySelector("textarea") || anchorEl.current;
const style = window.getComputedStyle(element);
const fontSize = `${style.getPropertyValue("font-size")}`;
const fontFamily = `${style.getPropertyValue("font-family")}`;
const lineHeight = parseInt(`${style.getPropertyValue("line-height")}`);
const span = document.createElement("div");
span.style.font = `${fontSize} ${fontFamily}`;
span.style.padding = style.getPropertyValue("padding");
span.style.lineHeight = `${lineHeight}px`;
span.style.width = `${element.offsetWidth}px`;
span.style.maxWidth = `${element.offsetWidth}px`;
span.style.whiteSpace = style.getPropertyValue("white-space");
span.style.overflowWrap = style.getPropertyValue("overflow-wrap");
const marker = document.createElement("span");
span.appendChild(document.createTextNode(fullValue.valueBeforeCursor || ""));
span.appendChild(marker);
span.appendChild(document.createTextNode(fullValue.valueAfterCursor || ""));
document.body.appendChild(span);
const spanRect = span.getBoundingClientRect();
const markerRect = marker.getBoundingClientRect();
const leftOffset = markerRect.left - spanRect.left;
const topOffset = markerRect.bottom - spanRect.bottom - (hasHelperText ? lineHeight : 0);
setOffsetPos({ top: topOffset, left: leftOffset });
span.remove();
marker.remove();
}, [anchorEl, caretPosition, hasHelperText, fullValue]);
return (
<>
<Autocomplete
loading={loading}
disabledFullScreen
value={contextData?.valueContext || ""}
options={options}
anchor={anchorEl}
minLength={0}
offset={offsetPos}
onSelect={handleSelect}
onFoundOptions={onFoundOptions}
maxDisplayResults={{
limit: AUTOCOMPLETE_LIMITS.displayResults,
message: "Please, specify the query more precisely."
}}
/>
</>
);
};
export default LogsQueryEditorAutocomplete;

View File

@@ -0,0 +1,117 @@
import { ContextData, ContextType, LogicalPart, LogicalPartPosition, LogicalPartType } from "./types";
import { pipeList } from "./pipes";
const BUILDER_OPERATORS = ["AND", "OR", "NOT"];
const PIPE_NAMES = pipeList.map(p => p.value);
export const splitLogicalParts = (expr: string) => {
// Replace spaces around the colon (:) with just the colon, removing the spaces
const input = expr; //.replace(/\s*:\s*/g, ":");
const parts: LogicalPart[] = [];
let currentPart = "";
let isPipePart = false;
const quotes = ["'", "\"", "`"];
let insideQuotes = false;
let expectedQuote = "";
const openBrackets = ["(", "[", "{"];
const closeBrackets = [")", "]", "}"];
const brackets = [...openBrackets, ...closeBrackets];
let insideBrackets = 0;
let startIndex = 0;
for (let i = 0; i < input.length; i++) {
const char = input[i];
// Check if the current character is a quote
if (quotes.includes(char)) {
const isClosedQuote: boolean = insideQuotes && (char === expectedQuote);
insideQuotes = !isClosedQuote;
expectedQuote = isClosedQuote ? "" : char;
}
// Check if the current character is a bracket
if (!insideQuotes && brackets.includes(char)) {
const dir = openBrackets.includes(char) ? 1 : -1;
insideBrackets += dir;
}
// Check if the current character is a pipe
if ((!insideQuotes && !insideBrackets && char === "|")) {
isPipePart = true;
const countStartSpaces = currentPart.match(/^ */)?.[0].length || 0;
const countEndSpaces = currentPart.match(/ *$/)?.[0].length || 0;
pushPart(currentPart, true, [startIndex + countStartSpaces, i - countEndSpaces - 1], parts);
currentPart = "";
startIndex = i + 1;
continue;
}
// Check if the current character is a space
if (!isPipePart && !insideQuotes && !insideBrackets && char === " ") {
const nextStr = input.slice(i).replace(/^\s*/, "");
const prevStr = input.slice(0, i).replace(/\s*$/, "");
if (!nextStr.startsWith(":") && !prevStr.endsWith(":")) {
pushPart(currentPart, false, [startIndex, i - 1], parts);
currentPart = "";
startIndex = i + 1;
continue;
}
}
currentPart += char;
}
// push the last part
pushPart(currentPart, isPipePart, [startIndex, input.length], parts);
return parts;
};
const pushPart = (currentPart: string, isPipePart: boolean, position: LogicalPartPosition, parts: LogicalPart[]) => {
const trimmedPart = currentPart.trim();
if (!trimmedPart) return;
const isOperator = BUILDER_OPERATORS.includes(trimmedPart.toUpperCase());
parts.push({
id: parts.length,
value: trimmedPart,
position,
type: isPipePart
? LogicalPartType.Pipe
: isOperator ? LogicalPartType.Operator : LogicalPartType.Filter,
});
};
export const getContextData = (part: LogicalPart, cursorPos: number) => {
const valueBeforeCursor = part.value.substring(0, cursorPos);
const valueAfterCursor = part.value.substring(cursorPos);
const metaData: ContextData = {
valueBeforeCursor,
valueAfterCursor,
valueContext: part.value,
contextType: ContextType.Unknown,
};
if (part.type === LogicalPartType.Filter) {
const noColon = !valueBeforeCursor.includes(":") && !valueAfterCursor.includes(":");
if (noColon) {
metaData.contextType = ContextType.FilterUnknown;
} else if (valueBeforeCursor.includes(":")) {
const [filterName, filterValue] = valueBeforeCursor.split(":");
metaData.contextType = ContextType.FilterValue;
metaData.filterName = filterName;
metaData.valueContext = filterValue;
} else {
metaData.contextType = ContextType.FilterName;
}
} else if (part.type === LogicalPartType.Pipe) {
const valueStartWithPipe = PIPE_NAMES.some(p => part.value.startsWith(p));
metaData.contextType = valueStartWithPipe ? ContextType.PipeValue : ContextType.PipeName;
}
metaData.valueContext = metaData.valueContext.replace(/^["']|["']$/g, "");
return metaData;
};

View File

@@ -0,0 +1,130 @@
import React from "react";
import { ContextType } from "./types";
import { FunctionIcon } from "../../../Main/Icons";
const docsUrl = "https://docs.victoriametrics.com/victorialogs/logsql";
const classLink = "vm-link vm-link_colored";
const prepareDescription = (text: string): string => {
const replaceClass = `$1 target="_blank" class="${classLink}" $2`;
const replaceHref = `$1 $2${docsUrl}#`;
return text
.replace(/(<a) (href=")#/gm, replaceHref)
.replace(/(<a) (href="[^"]+")/gm, replaceClass);
};
export const pipeList = [
{
"value": "copy",
"description": "<a href=\"#copy-pipe\"><code>copy</code></a> copies <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a>."
},
{
"value": "delete",
"description": "<a href=\"#delete-pipe\"><code>delete</code></a> deletes <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a>."
},
{
"value": "drop_empty_fields",
"description": "<a href=\"#drop_empty_fields-pipe\"><code>drop_empty_fields</code></a> drops <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a> with empty values."
},
{
"value": "extract",
"description": "<a href=\"#extract-pipe\"><code>extract</code></a> extracts the specified text into the given log fields."
},
{
"value": "extract_regexp",
"description": "<a href=\"#extract_regexp-pipe\"><code>extract_regexp</code></a> extracts the specified text into the given log fields via <a href=\"https://github.com/google/re2/wiki/Syntax\" rel=\"external\" target=\"_blank\">RE2 regular expressions</a>."
},
{
"value": "field_names",
"description": "<a href=\"#field_names-pipe\"><code>field_names</code></a> returns all the names of <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a>."
},
{
"value": "field_values",
"description": "<a href=\"#field_values-pipe\"><code>field_values</code></a> returns all the values for the given <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log field</a>."
},
{
"value": "fields",
"description": "<a href=\"#fields-pipe\"><code>fields</code></a> selects the given set of <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a>."
},
{
"value": "filter",
"description": "<a href=\"#filter-pipe\"><code>filter</code></a> applies additional <a href=\"#filters\">filters</a> to results."
},
{
"value": "format",
"description": "<a href=\"#format-pipe\"><code>format</code></a> formats output field from input <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a>."
},
{
"value": "limit",
"description": "<a href=\"#limit-pipe\"><code>limit</code></a> limits the number selected logs."
},
{
"value": "math",
"description": "<a href=\"#math-pipe\"><code>math</code></a> performs mathematical calculations over <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a>."
},
{
"value": "offset",
"description": "<a href=\"#offset-pipe\"><code>offset</code></a> skips the given number of selected logs."
},
{
"value": "pack_json",
"description": "<a href=\"#pack_json-pipe\"><code>pack_json</code></a> packs <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a> into JSON object."
},
{
"value": "pack_logfmt",
"description": "<a href=\"#pack_logfmt-pipe\"><code>pack_logfmt</code></a> packs <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a> into <a href=\"https://brandur.org/logfmt\" rel=\"external\" target=\"_blank\">logfmt</a> message."
},
{
"value": "rename",
"description": "<a href=\"#rename-pipe\"><code>rename</code></a> renames <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a>."
},
{
"value": "replace",
"description": "<a href=\"#replace-pipe\"><code>replace</code></a> replaces substrings in the specified <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a>."
},
{
"value": "replace_regexp",
"description": "<a href=\"#replace_regexp-pipe\"><code>replace_regexp</code></a> updates <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a> with regular expressions."
},
{
"value": "sort",
"description": "<a href=\"#sort-pipe\"><code>sort</code></a> sorts logs by the given <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">fields</a>."
},
{
"value": "stats",
"description": "<a href=\"#stats-pipe\"><code>stats</code></a> calculates various stats over the selected logs."
},
{
"value": "stream_context",
"description": "<a href=\"#stream_context-pipe\"><code>stream_context</code></a> allows selecting surrounding logs in front and after the matching logs\nper each <a href=\"/victorialogs/keyconcepts/#stream-fields\">log stream</a>."
},
{
"value": "top",
"description": "<a href=\"#top-pipe\"><code>top</code></a> returns top <code>N</code> field sets with the maximum number of matching logs."
},
{
"value": "uniq",
"description": "<a href=\"#uniq-pipe\"><code>uniq</code></a> returns unique log entires."
},
{
"value": "unpack_json",
"description": "<a href=\"#unpack_json-pipe\"><code>unpack_json</code></a> unpacks JSON messages from <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a>."
},
{
"value": "unpack_logfmt",
"description": "<a href=\"#unpack_logfmt-pipe\"><code>unpack_logfmt</code></a> unpacks <a href=\"https://brandur.org/logfmt\" rel=\"external\" target=\"_blank\">logfmt</a> messages from <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a>."
},
{
"value": "unpack_syslog",
"description": "<a href=\"#unpack_syslog-pipe\"><code>unpack_syslog</code></a> unpacks <a href=\"https://en.wikipedia.org/wiki/Syslog\" rel=\"external\" target=\"_blank\">syslog</a> messages from <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a>."
},
{
"value": "unroll",
"description": "<a href=\"#unroll-pipe\"><code>unroll</code></a> unrolls JSON arrays from <a href=\"https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model\">log fields</a>."
}
].map(item => ({
...item,
type: ContextType.PipeName,
icon: <FunctionIcon/>,
description: prepareDescription(item.description),
}));

View File

@@ -0,0 +1,31 @@
export enum LogicalPartType {
Filter = "Filter",
Pipe = "Pipe",
Operator = "Operator",
}
export type LogicalPartPosition = [start: number, end: number];
export interface LogicalPart {
id: number;
value: string;
type: LogicalPartType;
position: LogicalPartPosition;
}
export interface ContextData {
valueBeforeCursor: string;
valueAfterCursor: string;
contextType: ContextType;
valueContext: string;
filterName?: string;
}
export enum ContextType {
FilterName = "FilterName",
FilterUnknown = "FilterUnknown",
FilterValue = "FilterValue",
PipeName = "Pipes",
PipeValue = "PipeValue",
Unknown = "Unknown",
}

View File

@@ -0,0 +1,137 @@
import React, { useEffect, useState, useRef, Dispatch, SetStateAction } from "preact/compat";
import dayjs from "dayjs";
import { ContextData, ContextType } from "./types";
import { FunctionIcon, LabelIcon, MetricIcon, ValueIcon } from "../../../Main/Icons";
import { AutocompleteOptions } from "../../../Main/Autocomplete/Autocomplete";
import { useAppState } from "../../../../state/common/StateContext";
import { useTimeState } from "../../../../state/time/TimeStateContext";
import { useCallback } from "react";
import { AUTOCOMPLETE_LIMITS } from "../../../../constants/queryAutocomplete";
import { LogsFiledValues } from "../../../../api/types";
import { useLogsDispatch, useLogsState } from "../../../../state/logsPanel/LogsStateContext";
type FetchDataArgs = {
urlSuffix: string;
setter: Dispatch<SetStateAction<AutocompleteOptions[]>>
type: ContextType;
params?: URLSearchParams;
}
const icons = {
[ContextType.FilterName]: <MetricIcon/>,
[ContextType.FilterUnknown]: <MetricIcon/>,
[ContextType.FilterValue]: <ValueIcon/>,
[ContextType.PipeName]: <FunctionIcon/>,
[ContextType.PipeValue]: <LabelIcon/>,
[ContextType.Unknown]: <ValueIcon/>
};
export const useFetchLogsQLOptions = (contextData?: ContextData) => {
const { serverUrl } = useAppState();
const { period: { start, end } } = useTimeState();
const { autocompleteCache } = useLogsState();
const dispatch = useLogsDispatch();
const [loading, setLoading] = useState(false);
const [fieldNames, setFieldNames] = useState<AutocompleteOptions[]>([]);
const [fieldValues, setFieldValues] = useState<AutocompleteOptions[]>([]);
const abortControllerRef = useRef(new AbortController());
const getQueryParams = useCallback((params?: Record<string, string>) => {
const startDay = dayjs(start * 1000).startOf("day").valueOf() / 1000;
const endDay = dayjs(end * 1000).endOf("day").valueOf() / 1000;
return new URLSearchParams({
...(params || {}),
limit: `${AUTOCOMPLETE_LIMITS.queryLimit}`,
start: `${startDay}`,
end: `${endDay}`
});
}, [start, end]);
const processData = (values: LogsFiledValues[], type: ContextType): AutocompleteOptions[] => {
return values.map(v => ({
value: v.value,
type: `${type}`,
icon: icons[type]
}));
};
const fetchData = async ({ urlSuffix, setter, type, params }: FetchDataArgs) => {
// if (!value && type === TypeData.metric) return;
abortControllerRef.current.abort();
abortControllerRef.current = new AbortController();
const { signal } = abortControllerRef.current;
const key = `${urlSuffix}?${params?.toString()}`;
setLoading(true);
try {
const cachedData = autocompleteCache.get(key);
if (cachedData) {
setter(processData(cachedData, type));
setLoading(false);
return;
}
const response = await fetch(`${serverUrl}/select/logsql/${urlSuffix}?${params}`, { signal });
if (response.ok) {
const data = await response.json();
const value = (data?.values || []) as LogsFiledValues[];
setter(value ? processData(value, type) : []);
dispatch({ type: "SET_AUTOCOMPLETE_CACHE", payload: { key, value } });
}
setLoading(false);
} catch (e) {
if (e instanceof Error && e.name !== "AbortError") {
dispatch({ type: "SET_AUTOCOMPLETE_CACHE", payload: { key, value: [] } });
setLoading(false);
console.error(e);
}
}
};
// fetch field names
useEffect(() => {
const validContexts = [ContextType.FilterName, ContextType.FilterUnknown];
const isInvalidContext = !validContexts.includes(contextData?.contextType || ContextType.Unknown);
if (!serverUrl || isInvalidContext) {
return;
}
setFieldNames([]);
fetchData({
urlSuffix: "field_names",
setter: setFieldNames,
type: ContextType.FilterName,
params: getQueryParams({ query: "*" })
});
return () => abortControllerRef.current?.abort();
}, [serverUrl, contextData]);
// fetch field values
useEffect(() => {
const isInvalidContext = contextData?.contextType !== ContextType.FilterValue;
if (!serverUrl || isInvalidContext || !contextData?.filterName) {
return;
}
setFieldValues([]);
fetchData({
urlSuffix: "field_values",
setter: setFieldValues,
type: ContextType.FilterValue,
params: getQueryParams({ query: "*", field: contextData.filterName })
});
return () => abortControllerRef.current?.abort();
}, [serverUrl, contextData]);
return {
fieldNames,
fieldValues,
loading,
};
};

View File

@@ -2,7 +2,6 @@ import React, { FC, useEffect, useRef, useState } from "preact/compat";
import { KeyboardEvent } from "react";
import { ErrorTypes } from "../../../types";
import TextField from "../../Main/TextField/TextField";
import QueryEditorAutocomplete from "./QueryEditorAutocomplete";
import "./style.scss";
import { QueryStats } from "../../../api/types";
import { partialWarning, seriesFetchedWarning } from "./warningText";
@@ -11,6 +10,16 @@ import useDeviceDetect from "../../../hooks/useDeviceDetect";
import { useQueryState } from "../../../state/query/QueryStateContext";
import debounce from "lodash.debounce";
export interface QueryEditorAutocompleteProps {
value: string;
anchorEl: React.RefObject<HTMLInputElement>;
caretPosition: [number, number]; // [start, end]
hasHelperText: boolean;
includeFunctions: boolean;
onSelect: (val: string, caretPosition: number) => void;
onFoundOptions: (val: AutocompleteOptions[]) => void;
}
export interface QueryEditorProps {
onChange: (query: string) => void;
onEnter: () => void;
@@ -19,6 +28,7 @@ export interface QueryEditorProps {
value: string;
oneLiner?: boolean;
autocomplete: boolean;
autocompleteEl?: FC<QueryEditorAutocompleteProps>;
error?: ErrorTypes | string;
stats?: QueryStats;
label: string;
@@ -33,6 +43,7 @@ const QueryEditor: FC<QueryEditorProps> = ({
onArrowUp,
onArrowDown,
autocomplete,
autocompleteEl: AutocompleteEl,
error,
stats,
label,
@@ -47,7 +58,7 @@ const QueryEditor: FC<QueryEditorProps> = ({
const [caretPositionInput, setCaretPositionInput] = useState<[number, number]>([0, 0]);
const autocompleteAnchorEl = useRef<HTMLInputElement>(null);
const [showAutocomplete, setShowAutocomplete] = useState(autocomplete);
const [showAutocomplete, setShowAutocomplete] = useState(!!AutocompleteEl);
const debouncedSetShowAutocomplete = useRef(debounce(setShowAutocomplete, 500)).current;
const warning = [
@@ -113,7 +124,7 @@ const QueryEditor: FC<QueryEditorProps> = ({
};
useEffect(() => {
setOpenAutocomplete(autocomplete);
setOpenAutocomplete(!!AutocompleteEl);
}, [autocompleteQuick]);
useEffect(() => {
@@ -140,8 +151,8 @@ const QueryEditor: FC<QueryEditorProps> = ({
inputmode={"search"}
caretPosition={caretPositionInput}
/>
{showAutocomplete && autocomplete && (
<QueryEditorAutocomplete
{showAutocomplete && autocomplete && AutocompleteEl && (
<AutocompleteEl
value={value}
anchorEl={autocompleteAnchorEl}
caretPosition={caretPositionAutocomplete}

View File

@@ -1,20 +1,11 @@
import React, { FC, useState, useEffect, useMemo, useCallback } from "preact/compat";
import Autocomplete, { AutocompleteOptions } from "../../Main/Autocomplete/Autocomplete";
import Autocomplete from "../../Main/Autocomplete/Autocomplete";
import { useFetchQueryOptions } from "../../../hooks/useFetchQueryOptions";
import { escapeRegexp, hasUnclosedQuotes } from "../../../utils/regexp";
import useGetMetricsQL from "../../../hooks/useGetMetricsQL";
import { QueryContextType } from "../../../types";
import { AUTOCOMPLETE_LIMITS } from "../../../constants/queryAutocomplete";
interface QueryEditorAutocompleteProps {
value: string;
anchorEl: React.RefObject<HTMLElement>;
caretPosition: [number, number]; // [start, end]
hasHelperText: boolean;
includeFunctions: boolean;
onSelect: (val: string, caretPosition: number) => void;
onFoundOptions: (val: AutocompleteOptions[]) => void;
}
import { QueryEditorAutocompleteProps } from "./QueryEditor";
const QueryEditorAutocomplete: FC<QueryEditorAutocompleteProps> = ({
value,

View File

@@ -38,6 +38,10 @@
align-items: flex-start;
gap: $padding-small;
ul {
list-style-position: inside;
}
button {
color: inherit;
min-height: 29px;

View File

@@ -28,7 +28,7 @@ interface AutocompleteProps {
offset?: {top: number, left: number}
maxDisplayResults?: {limit: number, message?: string}
loading?: boolean;
onSelect: (val: string) => void
onSelect: (val: string, item: AutocompleteOptions) => void
onOpenAutocomplete?: (val: boolean) => void
onFoundOptions?: (val: AutocompleteOptions[]) => void
onChangeWrapperRef?: (elementRef: React.RefObject<HTMLElement>) => void
@@ -97,9 +97,9 @@ const Autocomplete: FC<AutocompleteProps> = ({
return noOptionsText && !foundOptions.length;
}, [noOptionsText,foundOptions]);
const createHandlerSelect = (item: string) => () => {
const createHandlerSelect = (item: AutocompleteOptions) => () => {
if (disabled) return;
onSelect(item);
onSelect(item.value, item);
if (!selected) handleCloseAutocomplete();
};
@@ -141,7 +141,7 @@ const Autocomplete: FC<AutocompleteProps> = ({
if (key === "Enter") {
const item = foundOptions[focusOption.index];
item && onSelect(item.value);
item && onSelect(item.value, item);
if (!selected) handleCloseAutocomplete();
}
@@ -206,7 +206,7 @@ const Autocomplete: FC<AutocompleteProps> = ({
})}
id={`$autocomplete$${option.value}`}
key={`${i}${option.value}`}
onClick={createHandlerSelect(option.value)}
onClick={createHandlerSelect(option)}
onMouseEnter={createHandlerMouseEnter(i)}
onMouseLeave={handlerMouseLeave}
>

View File

@@ -570,3 +570,14 @@ export const SpinnerIcon = () => (
</path>
</svg>
);
export const CommentIcon = () => (
<svg
viewBox="0 0 24 24"
fill="currentColor"
>
<path
d="M21.99 4c0-1.1-.89-2-1.99-2H4c-1.1 0-2 .9-2 2v12c0 1.1.9 2 2 2h14l4 4zM18 14H6v-2h12zm0-3H6V9h12zm0-3H6V6h12z"
></path>
</svg>
);

View File

@@ -0,0 +1,62 @@
import React, { FC } from "preact/compat";
import useBoolean from "../../../hooks/useBoolean";
import classNames from "classnames";
import TextField from "../TextField/TextField";
import "./style.scss";
import { marked } from "marked";
interface Props {
value: string;
onChange: (value: string) => void;
}
const tabs = [
{ title: "Write", value: false },
{ title: "Preview", value: true },
];
const MarkdownEditor: FC<Props> = ({ value, onChange }) => {
const {
value: markdownPreview,
setTrue: setMarkdownPreviewTrue,
setFalse: setMarkdownPreviewFalse,
} = useBoolean(false);
return (
<div className="vm-markdown-editor">
<div className="vm-markdown-editor-header">
<div className="vm-markdown-editor-header-tabs">
{tabs.map(({ title, value }) => (
<div
key={title}
className={classNames({
"vm-markdown-editor-header-tabs__tab": true,
"vm-markdown-editor-header-tabs__tab_active": markdownPreview === value,
})}
onClick={value ? setMarkdownPreviewTrue : setMarkdownPreviewFalse}
>
{title}
</div>
))}
</div>
<span className="vm-markdown-editor-header__info">
Markdown is supported
</span>
</div>
{markdownPreview ? (
<div
className="vm-markdown-editor-preview vm-markdown"
dangerouslySetInnerHTML={{ __html: marked(value) as string }}
/>
) : (
<TextField
type="textarea"
value={value}
onChange={onChange}
/>
)}
</div>
);
};
export default MarkdownEditor;

View File

@@ -0,0 +1,75 @@
@use "src/styles/variables" as *;
.vm-markdown-editor {
margin-top: 6px;
padding: 0 6px;
border-radius: $border-radius-small;
border: $border-divider;
overflow: hidden;
&-header {
display: flex;
align-items: center;
background-color: $color-hover-black;
padding-right: $padding-global;
border-bottom: $border-divider;
margin: -1px -7px 6px;
&-tabs {
display: flex;
&__tab {
display: flex;
align-items: center;
justify-content: center;
margin-bottom: -1px;
padding: $padding-small $padding-large;
min-height: 40px;
color: $color-text-secondary;
transition: color 0.3s;
cursor: pointer;
&:hover {
color: $color-text;
}
&_active {
position: relative;
color: $color-text;
background-color: $color-background-body;
border-top-right-radius: $border-radius-small;
border-top-left-radius: $border-radius-small;
z-index: 1;
&:first-child {
border-right: $border-divider;
}
&:last-child {
border-right: $border-divider;
border-left: $border-divider;
}
}
}
}
&__info {
margin-left: auto;
margin-right: 0;
color: $color-text-secondary;
font-size: $font-size-small;
font-weight: 500;
}
}
&-preview {
padding: $padding-small;
margin-bottom: 6px;
}
&-preview,
textarea {
min-height: 200px;
resize: vertical;
}
}

View File

@@ -16,17 +16,20 @@ const UploadJsonButtons: FC<Props> = ({ onOpenModal, onChange }) => (
>
Paste JSON
</Button>
<Button>
Upload Files
<div className="vm-upload-json-buttons__upload">
<Button>
Upload Files
</Button>
<input
id="json"
name="json"
type="file"
accept="application/json"
multiple
title=" "
onChange={onChange}
/>
</Button>
</div>
</div>
);

View File

@@ -6,4 +6,8 @@
gap: $padding-global;
align-items: center;
justify-content: center;
&__upload {
position: relative;
}
}

View File

@@ -1,4 +1,4 @@
import React, { FC, useCallback, useEffect, useMemo, useRef, useState } from "preact/compat";
import React, { FC, useCallback, useEffect, useRef, useState } from "preact/compat";
import { DownloadIcon } from "../../../components/Main/Icons";
import Button from "../../../components/Main/Button/Button";
import Tooltip from "../../../components/Main/Tooltip/Tooltip";
@@ -12,32 +12,65 @@ import TextField from "../../../components/Main/TextField/TextField";
import { useQueryState } from "../../../state/query/QueryStateContext";
import { ErrorTypes } from "../../../types";
import Alert from "../../../components/Main/Alert/Alert";
import qs from "qs";
import Popper from "../../../components/Main/Popper/Popper";
import helperText from "./helperText";
import { Link } from "react-router-dom";
import router from "../../../router";
import { parseLineToJSON } from "../../../utils/json";
import { ExportMetricResult, ReportMetaData } from "../../../api/types";
import { getApiEndpoint } from "../../../utils/url";
import MarkdownEditor from "../../../components/Main/MarkdownEditor/MarkdownEditor";
export enum ReportType {
QUERY_DATA,
RAW_DATA,
}
type Props = {
fetchUrl?: string[];
reportType?: ReportType
}
const getDefaultReportName = () => `vmui_report_${dayjs().utc().format(DATE_FILENAME_FORMAT)}`;
type MetaData = {
id: number;
url: URL;
title: string;
comment: string;
}
const DownloadReport: FC<Props> = ({ fetchUrl }) => {
const getDefaultTitle = (type: ReportType) => {
switch (type) {
case ReportType.RAW_DATA:
return "Raw report";
default:
return "Report";
}
};
const getDefaultFilename = (title: string) => {
const timestamp = dayjs().utc().format(DATE_FILENAME_FORMAT);
return `vmui_${title.toLowerCase().replace(/ /g, "_")}_${timestamp}`;
};
const DownloadReport: FC<Props> = ({ fetchUrl, reportType = ReportType.QUERY_DATA }) => {
const { query } = useQueryState();
const [filename, setFilename] = useState(getDefaultReportName());
const defaultTitle = getDefaultTitle(reportType);
const defaultFilename = getDefaultFilename(defaultTitle);
const [title, setTitle] = useState(defaultTitle);
const [filename, setFilename] = useState(defaultFilename);
const [comment, setComment] = useState("");
const [trace, setTrace] = useState(true);
const [trace, setTrace] = useState(reportType === ReportType.QUERY_DATA);
const [error, setError] = useState<ErrorTypes | string>();
const [isLoading, setIsLoading] = useState(false);
const titleRef = useRef<HTMLDivElement>(null);
const filenameRef = useRef<HTMLDivElement>(null);
const commentRef = useRef<HTMLDivElement>(null);
const traceRef = useRef<HTMLDivElement>(null);
const generateRef = useRef<HTMLDivElement>(null);
const helperRefs = [filenameRef, commentRef, traceRef, generateRef];
const helperRefs = [filenameRef, titleRef, commentRef, traceRef, generateRef];
const [stepHelper, setStepHelper] = useState(0);
const {
@@ -52,13 +85,17 @@ const DownloadReport: FC<Props> = ({ fetchUrl }) => {
setFalse: handleCloseHelper,
} = useBoolean(false);
const fetchUrlReport = useMemo(() => {
const getFetchUrlReport = useCallback(() => {
if (!fetchUrl) return;
return fetchUrl.map((str, i) => {
const url = new URL(str);
trace ? url.searchParams.set("trace", "1") : url.searchParams.delete("trace");
return { id: i, url: url };
});
try {
return fetchUrl.map((str, i) => {
const url = new URL(str);
trace ? url.searchParams.set("trace", "1") : url.searchParams.delete("trace");
return { id: i, url: url };
});
} catch (e) {
setError(String(e));
}
}, [fetchUrl, trace]);
const generateFile = useCallback((data: unknown) => {
@@ -68,7 +105,7 @@ const DownloadReport: FC<Props> = ({ fetchUrl }) => {
const link = document.createElement("a");
link.href = href;
link.download = `${filename || getDefaultReportName()}.json`;
link.download = `${filename || defaultFilename}.json`;
document.body.appendChild(link);
link.click();
@@ -77,9 +114,63 @@ const DownloadReport: FC<Props> = ({ fetchUrl }) => {
handleClose();
}, [filename]);
const getMetaData = ({ id, url, comment, title }: MetaData): ReportMetaData => {
return {
id,
title: title || defaultTitle,
comment,
endpoint: getApiEndpoint(url.pathname) || "",
params: Object.fromEntries(url.searchParams)
};
};
const processJsonLineResponse = async (response: Response, metaData: MetaData) => {
const result: { metric: { [p: string]: string }, values: number[][] }[] = [];
const text = await response.text();
if (response.ok) {
const lines = text.split("\n").filter(line => line);
lines.forEach((line: string) => {
const jsonLine = parseLineToJSON(line) as (ExportMetricResult | null);
if (!jsonLine) return;
result.push({
metric: jsonLine.metric,
values: jsonLine.values.map((value, index) => [(jsonLine.timestamps[index] / 1000), value]),
});
});
} else {
setError(String(text));
}
return { data: { result, resultType: "matrix" }, vmui: getMetaData(metaData) };
};
const processJsonResponse = async (response: Response, metaData: MetaData) => {
const resp = await response.json();
if (response.ok) {
resp.vmui = getMetaData(metaData);
return resp;
} else {
const errorType = resp.errorType ? `${resp.errorType}\r\n` : "";
setError(`${errorType}${resp?.error || resp?.message || "unknown error"}`);
}
};
const processResponse = async (response: Response, metaData: MetaData) => {
switch (reportType) {
case ReportType.RAW_DATA:
return await processJsonLineResponse(response, metaData);
default:
return await processJsonResponse(response, metaData);
}
};
const handleGenerateReport = useCallback(async () => {
const fetchUrlReport = getFetchUrlReport();
if (!fetchUrlReport) {
setError(ErrorTypes.validQuery);
setError(prev => !prev ? ErrorTypes.validQuery : prev);
return;
}
@@ -88,20 +179,12 @@ const DownloadReport: FC<Props> = ({ fetchUrl }) => {
try {
const result = [];
for await (const { url, id } of fetchUrlReport) {
for await (const fetchOps of fetchUrlReport) {
if (!fetchOps) continue;
const { url, id } = fetchOps;
const response = await fetch(url);
const resp = await response.json();
if (response.ok) {
resp.vmui = {
id,
comment,
params: qs.parse(new URL(url).search.replace(/^\?/, ""))
};
result.push(resp);
} else {
const errorType = resp.errorType ? `${resp.errorType}\r\n` : "";
setError(`${errorType}${resp?.error || resp?.message || "unknown error"}`);
}
const data = await processResponse(response, { id, url, comment, title });
result.push(data);
}
result.length && generateFile(result);
} catch (e) {
@@ -111,15 +194,20 @@ const DownloadReport: FC<Props> = ({ fetchUrl }) => {
} finally {
setIsLoading(false);
}
}, [fetchUrlReport, comment, generateFile, query]);
}, [getFetchUrlReport, comment, generateFile, query, title]);
const handleChangeHelp = (step: number) => () => {
setStepHelper(prevStep => prevStep + step);
const findNextRef = (index: number): number => {
const nextIndex = index + step;
if (helperRefs[nextIndex]?.current) return nextIndex;
return findNextRef(nextIndex);
};
setStepHelper(findNextRef);
};
useEffect(() => {
setError("");
setFilename(getDefaultReportName());
setFilename(defaultFilename);
setComment("");
}, [openModal]);
@@ -155,31 +243,41 @@ const DownloadReport: FC<Props> = ({ fetchUrl }) => {
<div className="vm-download-report">
<div className="vm-download-report-settings">
<div ref={filenameRef}>
<div className="vm-download-report-settings__title">Filename</div>
<TextField
label="Filename"
value={filename}
onChange={setFilename}
/>
</div>
<div ref={commentRef}>
<div ref={titleRef}>
<div className="vm-download-report-settings__title">Report title</div>
<TextField
type="textarea"
label="Comment"
value={title}
onChange={setTitle}
/>
</div>
<div ref={commentRef}>
<div className="vm-download-report-settings__title">Comment</div>
<MarkdownEditor
value={comment}
onChange={setComment}
/>
</div>
<div ref={traceRef}>
<Checkbox
checked={trace}
onChange={setTrace}
label={"Include query trace"}
/>
</div>
<Alert variant="info">
If confused with the query results,
try viewing the raw samples for selected series in <RawQueryLink/> tab.
</Alert>
{reportType === ReportType.QUERY_DATA && (
<>
<div ref={traceRef}>
<Checkbox
checked={trace}
onChange={setTrace}
label={"Include query trace"}
/>
</div>
<Alert variant="info">
If confused with the query results,
try viewing the raw samples for selected series in <RawQueryLink/> tab.
</Alert>
</>
)}
</div>
{error && <Alert variant="error">{error}</Alert>}
<div className="vm-download-report__buttons">

View File

@@ -11,6 +11,18 @@ const filename = (
</>
);
const tittle = (
<>
<p>Title - specify the title that will be displayed on the <Link
to={router.queryAnalyzer}
target="_blank"
rel="noreferrer"
className="vm-link vm-link_underlined"
>{routerOptions[router.queryAnalyzer].title}</Link> page.</p>
<p>This helps identify your report in the interface.</p>
</>
);
const comment = (
<>
<p>Comment (optional) - add a comment to your report.</p>
@@ -39,6 +51,7 @@ const generate = (
export default [
filename,
tittle,
comment,
trace,
generate,

View File

@@ -9,10 +9,15 @@
&-settings {
display: grid;
gap: $padding-global;
gap: $padding-large;
textarea {
min-height: 200px;
&__title {
display: flex;
align-items: center;
margin-right: $padding-global;
font-size: $font-size;
font-weight: 600;
white-space: nowrap;
}
}
@@ -34,7 +39,7 @@
line-height: 1.3;
p {
margin-bottom: calc($padding-small/2);
margin-bottom: calc($padding-small / 2);
}
}

View File

@@ -25,6 +25,7 @@ import { QueryStats } from "../../../api/types";
import { usePrettifyQuery } from "./hooks/usePrettifyQuery";
import QueryHistory from "../QueryHistory/QueryHistory";
import AnomalyConfig from "../../../components/ExploreAnomaly/AnomalyConfig";
import QueryEditorAutocomplete from "../../../components/Configurators/QueryEditor/QueryEditorAutocomplete";
export interface QueryConfiguratorProps {
queryErrors: string[];
@@ -216,6 +217,7 @@ const QueryConfigurator: FC<QueryConfiguratorProps> = ({
<QueryEditor
value={stateQuery[i]}
autocomplete={!hideButtons?.autocomplete && (autocomplete || autocompleteQuick)}
autocompleteEl={QueryEditorAutocomplete}
error={queryErrors[i]}
stats={stats[i]}
onArrowUp={createHandlerArrow(-1, i)}

View File

@@ -6,6 +6,9 @@ import useDeviceDetect from "../../../hooks/useDeviceDetect";
import Button from "../../../components/Main/Button/Button";
import QueryEditor from "../../../components/Configurators/QueryEditor/QueryEditor";
import TextField from "../../../components/Main/TextField/TextField";
import LogsQueryEditorAutocomplete from "../../../components/Configurators/QueryEditor/LogsQL/LogsQueryEditorAutocomplete";
import { useQueryDispatch, useQueryState } from "../../../state/query/QueryStateContext";
import Switch from "../../../components/Main/Switch/Switch";
export interface ExploreLogHeaderProps {
query: string;
@@ -27,6 +30,8 @@ const ExploreLogsHeader: FC<ExploreLogHeaderProps> = ({
onRun,
}) => {
const { isMobile } = useDeviceDetect();
const { autocomplete } = useQueryState();
const queryDispatch = useQueryDispatch();
const [errorLimit, setErrorLimit] = useState("");
const [limitInput, setLimitInput] = useState(limit);
@@ -42,6 +47,10 @@ const ExploreLogsHeader: FC<ExploreLogHeaderProps> = ({
}
};
const onChangeAutocomplete = () => {
queryDispatch({ type: "TOGGLE_AUTOCOMPLETE" });
};
useEffect(() => {
setLimitInput(limit);
}, [limit]);
@@ -57,7 +66,8 @@ const ExploreLogsHeader: FC<ExploreLogHeaderProps> = ({
<div className="vm-explore-logs-header-top">
<QueryEditor
value={query}
autocomplete={false}
autocomplete={autocomplete}
autocompleteEl={LogsQueryEditorAutocomplete}
onArrowUp={() => null}
onArrowDown={() => null}
onEnter={onRun}
@@ -75,7 +85,14 @@ const ExploreLogsHeader: FC<ExploreLogHeaderProps> = ({
/>
</div>
<div className="vm-explore-logs-header-bottom">
<div className="vm-explore-logs-header-bottom-contols"></div>
<div className="vm-explore-logs-header-bottom-contols">
<Switch
label={"Autocomplete"}
value={autocomplete}
onChange={onChangeAutocomplete}
fullWidth={isMobile}
/>
</div>
<div className="vm-explore-logs-header-bottom-helpful">
<a
className="vm-link vm-link_with-icon"

View File

@@ -26,6 +26,9 @@
}
&-contols {
display: flex;
align-items: center;
justify-content: flex-start;
flex-grow: 1;
}

View File

@@ -1,13 +1,20 @@
import React, { FC, useMemo } from "preact/compat";
import { DataAnalyzerType } from "../index";
import Button from "../../../components/Main/Button/Button";
import { ClockIcon, InfoIcon, TimelineIcon } from "../../../components/Main/Icons";
import useBoolean from "../../../hooks/useBoolean";
import Modal from "../../../components/Main/Modal/Modal";
import {
ClockIcon,
CommentIcon,
InfoIcon,
TimelineIcon
} from "../../../components/Main/Icons";
import { TimeParams } from "../../../types";
import "./style.scss";
import dayjs from "dayjs";
import { DATE_TIME_FORMAT } from "../../../constants/date";
import useBoolean from "../../../hooks/useBoolean";
import Modal from "../../../components/Main/Modal/Modal";
import { marked } from "marked";
import Button from "../../../components/Main/Button/Button";
import get from "lodash.get";
type Props = {
data: DataAnalyzerType[];
@@ -15,8 +22,23 @@ type Props = {
}
const QueryAnalyzerInfo: FC<Props> = ({ data, period }) => {
const dataWithStats = useMemo(() => data.filter(d => d.stats && d.data.resultType === "matrix"), [data]);
const comment = useMemo(() => data.find(d => d?.vmui?.comment)?.vmui?.comment, [data]);
const dataWithStats = useMemo(() => data.filter(d => d.vmui || d.stats), [data]);
const title = dataWithStats.find(d => d?.vmui?.title)?.vmui?.title || "Report";
const comment = dataWithStats.find(d => d?.vmui?.comment)?.vmui?.comment;
const table = useMemo(() => {
return [
"vmui.endpoint",
...new Set(dataWithStats.flatMap(d => [
...Object.keys(d.vmui?.params || []).map(key => `vmui.params.${key}`),
...Object.keys(d.stats || []).map(key => `stats.${key}`),
"isPartial"
]))
].map(key => ({
column: key.split(".").pop(),
values: dataWithStats.map(data => get(data, key, "-"))
})).filter(({ values }) => values.length && values.every(v => v !== "-"));
}, [dataWithStats]);
const timeRange = useMemo(() => {
if (!period) return "";
@@ -34,59 +56,80 @@ const QueryAnalyzerInfo: FC<Props> = ({ data, period }) => {
return (
<>
<div className="vm-query-analyzer-info-header">
<Button
startIcon={<InfoIcon/>}
variant="outlined"
color="warning"
onClick={handleOpenModal}
>
Show report info
</Button>
{period && (
<>
<div className="vm-query-analyzer-info-header__period">
<TimelineIcon/> step: {period.step}
</div>
<div className="vm-query-analyzer-info-header__period">
<ClockIcon/> {timeRange}
</div>
</>
<h1 className="vm-query-analyzer-info-header__title">{title}</h1>
{timeRange && (
<div className="vm-query-analyzer-info-header__timerange">
<ClockIcon/> {timeRange}
</div>
)}
{period?.step && (
<div className="vm-query-analyzer-info-header__timerange">
<TimelineIcon/> step {period.step}
</div>
)}
{(comment || !!table.length) && (
<div className="vm-query-analyzer-info-header__info">
<Button
startIcon={<InfoIcon/>}
variant="outlined"
color="warning"
onClick={handleOpenModal}
>
Show stats{comment && " & comments"}
</Button>
</div>
)}
</div>
{openModal && (
<Modal
title="Report info"
title={title}
onClose={handleCloseModal}
>
<div className="vm-query-analyzer-info">
{comment && (
<div className="vm-query-analyzer-info-item vm-query-analyzer-info-item_comment">
<div className="vm-query-analyzer-info-item__title">Comment:</div>
<div className="vm-query-analyzer-info-item__text">{comment}</div>
<div className="vm-query-analyzer-info__modal">
{!!table.length && (
<div className="vm-query-analyzer-info-stats">
<div className="vm-query-analyzer-info-comment-header">
<InfoIcon/>
Stats
</div>
<table>
<thead>
<tr>
{table.map(({ column }) => (
<th key={column}>
{column}
</th>
))}
</tr>
</thead>
<tbody>
{table[0]?.values.map((_, rowIndex) => (
<tr key={rowIndex}>
{table.map(({ values }, j) => (
<td key={j}>
{values[rowIndex]}
</td>
))}
</tr>
))}
</tbody>
</table>
</div>
)}
{dataWithStats.map((d, i) => (
<div
className="vm-query-analyzer-info-item"
key={i}
>
<div className="vm-query-analyzer-info-item__title">
{dataWithStats.length > 1 ? `Query ${i + 1}:` : "Stats:"}
</div>
<div className="vm-query-analyzer-info-item__text">
{Object.entries(d.stats || {}).map(([key, value]) => (
<div key={key}>
{key}: {value ?? "-"}
</div>
))}
isPartial: {String(d.isPartial ?? "-")}
{comment && (
<div className="vm-query-analyzer-info-comment">
<div className="vm-query-analyzer-info-comment-header">
<CommentIcon/>
Comments
</div>
<div
className="vm-query-analyzer-info-comment-body vm-markdown"
dangerouslySetInnerHTML={{ __html: (marked(comment) as string) || comment }}
/>
</div>
))}
<div className="vm-query-analyzer-info-type">
{dataWithStats[0]?.vmui?.params ? "The report was created using vmui" : "The report was created manually"}
</div>
)}
</div>
</Modal>
)}

View File

@@ -1,47 +1,115 @@
@use "src/styles/variables" as *;
.vm-query-analyzer-info-header {
display: flex;
gap: $padding-global;
.vm-query-analyzer-info {
&__period {
&-header {
display: flex;
align-items: center;
width: 100%;
height: 100%;
gap: $padding-small;
border: $border-divider;
border-radius: $border-radius-small;
padding: 6px $padding-global;
svg {
width: calc($font-size-small + 1px);
color: $color-primary;
}
}
}
.vm-query-analyzer-info {
display: grid;
gap: $padding-large;
min-width: 300px;
&-type {
text-align: center;
font-style: italic;
color: $color-text-secondary;
}
&-item {
display: grid;
padding-bottom: $padding-large;
border-bottom: $border-divider;
line-height: 130%;
font-size: $font-size-small;
background-color: $color-background-body;
z-index: 1;
&__title {
font-weight: bold;
font-size: $font-size-large;
font-weight: 500;
}
&__text {
white-space: pre-wrap;
&__timerange {
display: flex;
align-items: center;
gap: calc($padding-small / 2);
border: $border-divider;
border-radius: $border-radius-small;
padding: calc($padding-small / 2) $padding-small;
font-size: $font-size-small;
svg {
width: calc($font-size-small + 1px);
color: $color-primary;
}
}
&__info {
margin-left: auto;
margin-right: 0;
}
}
&__modal {
width: min(800px, 90vw);
}
&-comment {
position: relative;
max-width: 800px;
border-radius: $border-radius-medium;
border: $border-divider;
font-size: $font-size-small;
&-header {
display: grid;
grid-template-columns: 16px 1fr;
align-items: center;
gap: $padding-small;
padding: $padding-small;
border-bottom: $border-divider;
background-color: $color-hover-black;
font-weight: 500;
z-index: 1;
svg {
color: $color-primary;
}
}
&-body {
padding: $padding-small;
max-height: 60vh;
overflow: auto;
}
}
&-stats {
border-radius: $border-radius-medium;
border: $border-divider;
font-size: $font-size-small;
margin-bottom: $padding-global;
overflow: hidden;
table {
width: 100%;
}
td, th {
padding: $padding-small;
text-align: left;
}
tr {
border-bottom: $border-divider;
}
thead {
th {
font-weight: 500;
}
}
tbody {
tr {
transition: background-color 0.3s;
&:hover {
background-color: $color-hover-black;
}
&:last-child {
border-bottom: none;
}
}
}
}
}

View File

@@ -9,8 +9,9 @@ import useBoolean from "../../hooks/useBoolean";
import UploadJsonButtons from "../../components/UploadJsonButtons/UploadJsonButtons";
import JsonForm from "./JsonForm/JsonForm";
import "../TracePage/style.scss";
import "./style.scss";
import QueryAnalyzerView from "./QueryAnalyzerView/QueryAnalyzerView";
import { InstantMetricResult, MetricResult, TracingData } from "../../api/types";
import { InstantMetricResult, MetricResult, ReportMetaData, TracingData } from "../../api/types";
import QueryAnalyzerInfo from "./QueryAnalyzerInfo/QueryAnalyzerInfo";
import { TimeParams } from "../../types";
import { dateFromSeconds, formatDateToUTC, humanizeSeconds } from "../../utils/time";
@@ -21,15 +22,8 @@ export type DataAnalyzerType = {
resultType: "vector" | "matrix";
result: MetricResult[] | InstantMetricResult[]
};
stats?: {
seriesFetched?: string;
executionTimeMsec?: number
};
vmui?: {
id: number;
comment: string;
params: Record<string, string>;
};
stats?: Record<string, string>;
vmui?: ReportMetaData;
status: string;
trace?: TracingData;
isPartial?: boolean;
@@ -92,10 +86,12 @@ const QueryAnalyzer: FC = () => {
setData(response);
} else {
setError("Invalid structure - JSON does not match the expected format");
setData([]);
}
} catch (e) {
if (e instanceof Error) {
setError(`${e.name}: ${e.message}`);
setData([]);
}
}
};
@@ -129,33 +125,16 @@ const QueryAnalyzer: FC = () => {
}, [files]);
return (
<div className="vm-trace-page">
<div className="vm-query-analyzer">
{hasData && (
<div className="vm-trace-page-header">
<div className="vm-trace-page-header-errors">
<QueryAnalyzerInfo
data={data}
period={period}
/>
</div>
<div>
<UploadJsonButtons
onOpenModal={handleOpenModal}
onChange={handleChange}
/>
</div>
</div>
)}
{error && (
<div className="vm-trace-page-header-errors-item vm-trace-page-header-errors-item_margin-bottom">
<Alert variant="error">{error}</Alert>
<Button
className="vm-trace-page-header-errors-item__close"
startIcon={<CloseIcon/>}
variant="text"
color="error"
onClick={handleCloseError}
<div className="vm-query-analyzer-header">
<QueryAnalyzerInfo
data={data}
period={period}
/>
<UploadJsonButtons
onOpenModal={handleOpenModal}
onChange={handleChange}
/>
</div>
)}
@@ -185,6 +164,19 @@ const QueryAnalyzer: FC = () => {
</div>
)}
{error && (
<div className="vm-query-analyzer-error">
<Alert variant="error">{error}</Alert>
<Button
className="vm-query-analyzer-error__close"
startIcon={<CloseIcon/>}
variant="text"
color="error"
onClick={handleCloseError}
/>
</div>
)}
{openModal && (
<Modal
title="Paste JSON"

View File

@@ -0,0 +1,29 @@
@use "src/styles/variables" as *;
.vm-query-analyzer {
display: flex;
flex-direction: column;
@media (max-width: 768px) {
padding: $padding-medium 0;
}
&-header {
display: grid;
grid-template-columns: 1fr auto;
gap: $padding-global;
margin-bottom: $padding-global;
}
&-error {
position: relative;
margin: $padding-global 0;
&__close {
position: absolute;
top: $padding-small;
right: $padding-small;
z-index: 2;
}
}
}

View File

@@ -7,6 +7,7 @@ import { useAppState } from "../../../state/common/StateContext";
import { useCustomPanelState } from "../../../state/customPanel/CustomPanelStateContext";
import { isValidHttpUrl } from "../../../utils/url";
import { getExportDataUrl } from "../../../api/query-range";
import { parseLineToJSON } from "../../../utils/json";
interface FetchQueryParams {
hideQuery?: number[];
@@ -24,14 +25,6 @@ interface FetchQueryReturn {
abortFetch: () => void
}
const parseLineToJSON = (line: string): ExportMetricResult | null => {
try {
return JSON.parse(line);
} catch (e) {
return null;
}
};
export const useFetchExport = ({ hideQuery, showAllSeries }: FetchQueryParams): FetchQueryReturn => {
const { query } = useQueryState();
const { period } = useTimeState();
@@ -62,7 +55,7 @@ export const useFetchExport = ({ hideQuery, showAllSeries }: FetchQueryParams):
}
}, [serverUrl, period, hideQuery, reduceMemUsage]);
const fetchData = useCallback(async ( { fetchUrl, stateSeriesLimits, showAllSeries }: {
const fetchData = useCallback(async ({ fetchUrl, stateSeriesLimits, showAllSeries }: {
fetchUrl: string[];
stateSeriesLimits: SeriesLimits;
showAllSeries?: boolean;
@@ -99,12 +92,12 @@ export const useFetchExport = ({ hideQuery, showAllSeries }: FetchQueryParams):
const lines = text.split("\n").filter(line => line);
const lineLimited = lines.slice(0, freeTempSize).sort();
lineLimited.forEach((line: string) => {
const jsonLine = parseLineToJSON(line);
const jsonLine = parseLineToJSON(line) as (ExportMetricResult | null);
if (!jsonLine) return;
tempData.push({
group: counter,
metric: jsonLine.metric,
values: jsonLine.values.map((value, index) => [(jsonLine.timestamps[index]/1000), value]),
values: jsonLine.values.map((value, index) => [(jsonLine.timestamps[index] / 1000), value]),
} as MetricBase);
});
totalLength += lines.length;
@@ -119,7 +112,7 @@ export const useFetchExport = ({ hideQuery, showAllSeries }: FetchQueryParams):
} catch (e) {
setIsLoading(false);
if (e instanceof Error && e.name !== "AbortError") {
setError(error);
setError(String(e));
console.error(e);
}
}

View File

@@ -17,6 +17,7 @@ import { DisplayType } from "../../types";
import Hyperlink from "../../components/Main/Hyperlink/Hyperlink";
import { CloseIcon } from "../../components/Main/Icons";
import Button from "../../components/Main/Button/Button";
import DownloadReport, { ReportType } from "../CustomPanel/DownloadReport/DownloadReport";
const RawSamplesLink = () => (
<Hyperlink
@@ -65,6 +66,7 @@ const RawQueryPage: FC = () => {
queryErrors,
setQueryErrors,
abortFetch,
fetchUrl,
} = useFetchExport({ hideQuery, showAllSeries });
const controlsRef = useRef<HTMLDivElement>(null);
@@ -106,12 +108,22 @@ const RawQueryPage: FC = () => {
{showPageDescription && (
<Alert variant="info">
<div className="vm-explore-metrics-header-description">
<p>
This page provides a dedicated view for querying and displaying <RawSamplesLink/> from VictoriaMetrics.
It expects only <TimeSeriesSelectorLink/> as a query argument.
Users often assume that the <QueryDataLink/> returns data exactly as stored,
but data samples and timestamps may be modified by the API.
</p>
<ul>
<li>
This page provides a dedicated view for querying and displaying <RawSamplesLink/> from VictoriaMetrics.
</li>
<li>
It expects only <TimeSeriesSelectorLink/> as a query argument.
</li>
<li>
Deduplication can only be disabled if it was previously enabled on the server
(<code>-dedup.minScrapeInterval</code>).
</li>
<li>
Users often assume that the <QueryDataLink/> returns data exactly as stored,
but data samples and timestamps may be modified by the API.
</li>
</ul>
<Button
variant="text"
size="small"
@@ -146,6 +158,12 @@ const RawQueryPage: FC = () => {
<div className="vm-custom-panel-body-header__tabs">
<DisplayTypeSwitch tabFilter={(tab) => (tab.value !== DisplayType.table)}/>
</div>
{data && (
<DownloadReport
fetchUrl={fetchUrl}
reportType={ReportType.RAW_DATA}
/>
)}
</div>
<CustomPanelTabs
graphData={data}

View File

@@ -61,7 +61,8 @@
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
justify-content: flex-end;
min-height: calc(($vh * 50) - var(--scrollbar-height));
&__text {
margin-bottom: $padding-global;

View File

@@ -1,15 +1,20 @@
import { getFromStorage, saveToStorage } from "../../utils/storage";
import { LogsFiledValues } from "../../api/types";
import { AUTOCOMPLETE_LIMITS } from "../../constants/queryAutocomplete";
export interface LogsState {
markdownParsing: boolean;
autocompleteCache: Map<string, LogsFiledValues[]>;
}
export type LogsAction =
| { type: "SET_MARKDOWN_PARSING", payload: boolean }
| { type: "SET_AUTOCOMPLETE_CACHE", payload: { key: string, value: LogsFiledValues[] } }
export const initialLogsState: LogsState = {
markdownParsing: getFromStorage("LOGS_MARKDOWN") === "true",
autocompleteCache: new Map<string, LogsFiledValues[]>(),
};
export function reducer(state: LogsState, action: LogsAction): LogsState {
@@ -20,6 +25,18 @@ export function reducer(state: LogsState, action: LogsAction): LogsState {
...state,
markdownParsing: action.payload
};
case "SET_AUTOCOMPLETE_CACHE": {
if (state.autocompleteCache.size >= AUTOCOMPLETE_LIMITS.cacheLimit) {
const firstKey = state.autocompleteCache.keys().next().value;
state.autocompleteCache.delete(firstKey);
}
state.autocompleteCache.set(action.payload.key, action.payload.value);
return {
...state,
autocompleteCache: state.autocompleteCache,
};
}
default:
throw new Error();
}

View File

@@ -0,0 +1,139 @@
@use "src/styles/variables" as *;
.vm-markdown {
display: block;
line-height: 1.5;
pre,
code {
font-size: 1em;
font-family: $font-family-monospace
}
pre {
padding: .5em;
line-height: 1.25;
overflow-x: scroll;
}
a,
a:visited {
color: $color-primary;
text-decoration: underline;
cursor: pointer;
}
body {
line-height: 1.85;
}
p {
font-size: 1em;
margin-bottom: 1.3em;
}
h1, h2, h3, h4, h5, h6 {
margin: .5em 0 0.25em;
font-weight: inherit;
line-height: 1.42;
}
h1 {
font-size: 1.8em;
}
h2 {
font-size: 1.6em;
}
h3 {
font-size: 1.4em;
}
h4 {
font-size: 1.2em;
}
h5 {
font-size: 1em;
}
h6 {
font-size: 0.8em;
}
small {
font-size: 0.8em;
}
img,
canvas,
iframe,
video,
svg,
select,
textarea {
max-width: 100%;
}
pre {
background-color: $color-hover-black;
}
blockquote {
border-left: 3px solid rgba($color-black, 0.2);
padding-left: 1em;
opacity: 0.7;
}
ul, ol {
margin: 0.3em 0;
list-style-position: inside;
li {
margin-bottom: 0.3em;
ul, ol {
margin-left: 1em;
}
}
}
input[type="checkbox"] {
display: none;
}
th,
td {
padding: 0.2em 0.4em;
}
hr {
border-top: $border-divider;
}
strong {
font-weight: bold;
}
em {
font-style: italic;
}
del {
text-decoration: line-through;
}
code:not(pre code) {
display: inline;
vertical-align: middle;
background: $color-hover-black;
border-radius: $border-radius-small;
border: $border-divider;
tab-size: 4;
font-variant-ligatures: none;
padding: 0.12em 0.4em;
font-size: 0.9em;
white-space: nowrap;
}
}

View File

@@ -50,10 +50,6 @@ button {
background: none;
}
strong {
letter-spacing: 1px;
}
input[type='file'] {
opacity: 0;
cursor: pointer;

View File

@@ -10,6 +10,7 @@
@forward "./components/table";
@forward "./components/link";
@forward "./components/dynamic-number";
@forward "./components/markdown";
:root {
/* base palette */

View File

@@ -0,0 +1,7 @@
export const parseLineToJSON = (line: string) => {
try {
return JSON.parse(line);
} catch (e) {
return null;
}
};

View File

@@ -25,3 +25,13 @@ export const isEqualURLSearchParams = (params1: URLSearchParams, params2: URLSea
return true;
};
export const getApiEndpoint = (url: string): string | null => {
try {
const match = url.match(/\/api\/v1\/[^?]+/);
return match ? match[0] : null;
} catch (error) {
console.error("Invalid URL:", error);
return null;
}
};

View File

@@ -128,7 +128,7 @@ func (app *ServesMetrics) GetMetric(t *testing.T, metricName string) float64 {
return res
}
}
t.Fatalf("metic not found: %s", metricName)
t.Fatalf("metric not found: %s", metricName)
return 0
}

View File

@@ -5,6 +5,7 @@ import (
"fmt"
"net/url"
"slices"
"sort"
"strconv"
"strings"
"testing"
@@ -103,6 +104,25 @@ func NewPrometheusAPIV1QueryResponse(t *testing.T, s string) *PrometheusAPIV1Que
return res
}
// Sort performs data.Result sort by metric labels
func (pqr *PrometheusAPIV1QueryResponse) Sort() {
sort.Slice(pqr.Data.Result, func(i, j int) bool {
leftS := make([]string, 0, len(pqr.Data.Result[i].Metric))
rightS := make([]string, 0, len(pqr.Data.Result[j].Metric))
for k, v := range pqr.Data.Result[i].Metric {
leftS = append(leftS, fmt.Sprintf("%s=%s", k, v))
}
for k, v := range pqr.Data.Result[j].Metric {
rightS = append(rightS, fmt.Sprintf("%s=%s", k, v))
}
sort.Strings(leftS)
sort.Strings(rightS)
return strings.Join(leftS, ",") < strings.Join(rightS, ",")
})
}
// QueryData holds the query result along with its type.
type QueryData struct {
ResultType string

View File

@@ -0,0 +1,318 @@
package tests
import (
"os"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
at "github.com/VictoriaMetrics/VictoriaMetrics/apptest"
pb "github.com/VictoriaMetrics/VictoriaMetrics/lib/prompbmarshal"
)
func TestSingleIngestionProtocols(t *testing.T) {
os.RemoveAll(t.Name())
tc := at.NewTestCase(t)
defer tc.Stop()
sut := tc.MustStartDefaultVmsingle()
type opts struct {
query string
wantMetrics []map[string]string
wantSamples []*at.Sample
}
f := func(sut at.PrometheusQuerier, opts *opts) {
t.Helper()
wantResult := []*at.QueryResult{}
for idx, wm := range opts.wantMetrics {
wantResult = append(wantResult, &at.QueryResult{
Metric: wm,
Samples: []*at.Sample{opts.wantSamples[idx]},
})
}
tc.Assert(&at.AssertOptions{
Msg: "unexpected /export query response",
Got: func() any {
got := sut.PrometheusAPIV1Export(t, opts.query, at.QueryOpts{
Start: "2024-02-05T08:50:00.700Z",
End: "2024-02-05T09:00:00.700Z",
})
got.Sort()
return got
},
Want: &at.PrometheusAPIV1QueryResponse{Data: &at.QueryData{Result: wantResult}},
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(at.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
},
})
}
// influx line format
sut.InfluxWrite(t, []string{
`influxline series1=10 1707123456700`, // 2024-02-05T08:57:36.700Z
`influxline,label=foo1,label1=value1,label2=value2 series2=40 1707123456800`, // 2024-02-05T08:57:36.800Z
}, at.QueryOpts{})
sut.ForceFlush(t)
f(sut, &opts{
query: `{__name__=~"influxline.+"}`,
wantMetrics: []map[string]string{
{
"__name__": "influxline_series1",
},
{
"__name__": "influxline_series2",
"label": "foo1",
"label1": "value1",
"label2": "value2",
},
},
wantSamples: []*at.Sample{
{Timestamp: 1707123456700, Value: 10},
{Timestamp: 1707123456800, Value: 40},
},
})
// prometheus text exposition format
sut.PrometheusAPIV1ImportPrometheus(t, []string{
`importprometheus_series 10 1707123456700`, // 2024-02-05T08:57:36.700Z
`importprometheus_series2{label="foo",label1="value1"} 20 1707123456800`, // 2024-02-05T08:57:36.800Z
}, at.QueryOpts{})
sut.ForceFlush(t)
f(sut, &opts{
query: `{__name__=~"importprometheus.+"}`,
wantMetrics: []map[string]string{
{
"__name__": "importprometheus_series",
},
{
"__name__": "importprometheus_series2",
"label": "foo",
"label1": "value1",
},
},
wantSamples: []*at.Sample{
{Timestamp: 1707123456700, Value: 10},
{Timestamp: 1707123456800, Value: 20},
},
})
// prometheus remote write format
pbData := []pb.TimeSeries{
{
Labels: []pb.Label{
{
Name: "__name__",
Value: "prometheusrw_series",
},
},
Samples: []pb.Sample{
{
Value: 10,
Timestamp: 1707123456700, // 2024-02-05T08:57:36.700Z
},
},
},
{
Labels: []pb.Label{
{
Name: "__name__",
Value: "prometheusrw_series2",
},
{
Name: "label",
Value: "foo2",
},
{
Name: "label1",
Value: "value1",
},
},
Samples: []pb.Sample{
{
Value: 20,
Timestamp: 1707123456800, // 2024-02-05T08:57:36.800Z
},
},
},
}
sut.PrometheusAPIV1Write(t, pbData, at.QueryOpts{})
sut.ForceFlush(t)
f(sut, &opts{
query: `{__name__=~"prometheusrw.+"}`,
wantMetrics: []map[string]string{
{
"__name__": "prometheusrw_series",
},
{
"__name__": "prometheusrw_series2",
"label": "foo2",
"label1": "value1",
},
},
wantSamples: []*at.Sample{
{Timestamp: 1707123456700, Value: 10}, // 2024-02-05T08:57:36.700Z
{Timestamp: 1707123456800, Value: 20}, // 2024-02-05T08:57:36.700Z
},
})
}
func TestClusterIngestionProtocols(t *testing.T) {
os.RemoveAll(t.Name())
tc := at.NewTestCase(t)
defer tc.Stop()
vmstorage := tc.MustStartVmstorage("vmstorage", []string{
"-storageDataPath=" + tc.Dir() + "/vmstorage",
"-retentionPeriod=100y",
})
vminsert := tc.MustStartVminsert("vminsert", []string{
"-storageNode=" + vmstorage.VminsertAddr(),
})
vmselect := tc.MustStartVmselect("vmselect", []string{
"-storageNode=" + vmstorage.VmselectAddr(),
})
type opts struct {
query string
wantMetrics []map[string]string
wantSamples []*at.Sample
}
f := func(opts *opts) {
t.Helper()
wantResult := []*at.QueryResult{}
for idx, wm := range opts.wantMetrics {
wantResult = append(wantResult, &at.QueryResult{
Metric: wm,
Samples: []*at.Sample{opts.wantSamples[idx]},
})
}
tc.Assert(&at.AssertOptions{
Msg: "unexpected /export query response",
Got: func() any {
got := vmselect.PrometheusAPIV1Export(t, opts.query, at.QueryOpts{
Start: "2024-02-05T08:50:00.700Z",
End: "2024-02-05T09:00:00.700Z",
})
got.Sort()
return got
},
Want: &at.PrometheusAPIV1QueryResponse{Data: &at.QueryData{Result: wantResult}},
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(at.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
},
})
}
// prometheus text exposition format
vminsert.PrometheusAPIV1ImportPrometheus(t, []string{
`importprometheus_series 10 1707123456700`, // 2024-02-05T08:57:36.700Z
`importprometheus_series2{label="foo",label1="value1"} 20 1707123456800`, // 2024-02-05T08:57:36.800Z
}, at.QueryOpts{})
vmstorage.ForceFlush(t)
f(&opts{
query: `{__name__=~"importprometheus.+"}`,
wantMetrics: []map[string]string{
{
"__name__": "importprometheus_series",
},
{
"__name__": "importprometheus_series2",
"label": "foo",
"label1": "value1",
},
},
wantSamples: []*at.Sample{
{Timestamp: 1707123456700, Value: 10},
{Timestamp: 1707123456800, Value: 20},
},
})
// influx line format
vminsert.InfluxWrite(t, []string{
`influxline series1=10 1707123456700`, // 2024-02-05T08:57:36.700Z
`influxline,label=foo1,label1=value1,label2=value2 series2=40 1707123456800`, // 2024-02-05T08:57:36.800Z
}, at.QueryOpts{})
vmstorage.ForceFlush(t)
f(&opts{
query: `{__name__=~"influxline.+"}`,
wantMetrics: []map[string]string{
{
"__name__": "influxline_series1",
},
{
"__name__": "influxline_series2",
"label": "foo1",
"label1": "value1",
"label2": "value2",
},
},
wantSamples: []*at.Sample{
{Timestamp: 1707123456700, Value: 10},
{Timestamp: 1707123456800, Value: 40},
},
})
// prometheus remote write format
pbData := []pb.TimeSeries{
{
Labels: []pb.Label{
{
Name: "__name__",
Value: "prometheusrw_series",
},
},
Samples: []pb.Sample{
{
Value: 10,
Timestamp: 1707123456700, // 2024-02-05T08:57:36.700Z
},
},
},
{
Labels: []pb.Label{
{
Name: "__name__",
Value: "prometheusrw_series2",
},
{
Name: "label",
Value: "foo2",
},
{
Name: "label1",
Value: "value1",
},
},
Samples: []pb.Sample{
{
Value: 20,
Timestamp: 1707123456800, // 2024-02-05T08:57:36.800Z
},
},
},
}
vminsert.PrometheusAPIV1Write(t, pbData, at.QueryOpts{})
vmstorage.ForceFlush(t)
f(&opts{
query: `{__name__=~"prometheusrw.+"}`,
wantMetrics: []map[string]string{
{
"__name__": "prometheusrw_series",
},
{
"__name__": "prometheusrw_series2",
"label": "foo2",
"label1": "value1",
},
},
wantSamples: []*at.Sample{
{Timestamp: 1707123456700, Value: 10}, // 2024-02-05T08:57:36.700Z
{Timestamp: 1707123456800, Value: 20}, // 2024-02-05T08:57:36.700Z
},
})
}

View File

@@ -3,9 +3,10 @@ package tests
import (
"testing"
at "github.com/VictoriaMetrics/VictoriaMetrics/apptest"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
at "github.com/VictoriaMetrics/VictoriaMetrics/apptest"
)
// Data used in examples in
@@ -49,7 +50,7 @@ func TestClusterKeyConceptsQueryData(t *testing.T) {
testKeyConceptsQueryData(t, sut)
}
// testClusterKeyConceptsQuery verifies cases from https://docs.victoriametrics.com/keyconcepts/#query-data
// testKeyConceptsQueryData verifies cases from https://docs.victoriametrics.com/keyconcepts/#query-data
func testKeyConceptsQueryData(t *testing.T, sut at.PrometheusWriteQuerier) {
// Insert example data from documentation.

View File

@@ -0,0 +1,33 @@
package tests
import (
"testing"
"github.com/VictoriaMetrics/VictoriaMetrics/apptest"
)
// Data used in tests
var testData = []string{
"foo_bar 1.00",
"foo_bar 2.00",
}
func TestSingleMaxIngestionRateIncrementsMetric(t *testing.T) {
tc := apptest.NewTestCase(t)
defer tc.Stop()
sut := tc.MustStartVmsingle("vmsingle", []string{"-maxIngestionRate=1"})
sut.PrometheusAPIV1ImportPrometheus(t, testData, apptest.QueryOpts{})
if got := sut.GetMetric(t, "vm_max_ingestion_rate_limit_reached_total"); got <= 0 {
t.Fatalf("Unexpected vm_max_ingestion_rate_limit_reached_total: got %f, want >0", got)
}
}
func TestSingleMaxIngestionRateDoesNotIncrementMetric(t *testing.T) {
tc := apptest.NewTestCase(t)
defer tc.Stop()
sut := tc.MustStartVmsingle("vmsingle", []string{"-maxIngestionRate=15"})
sut.PrometheusAPIV1ImportPrometheus(t, testData, apptest.QueryOpts{})
if got, want := sut.GetMetric(t, "vm_max_ingestion_rate_limit_reached_total"), 0.0; got != want {
t.Fatalf("Unexpected vm_max_ingestion_rate_limit_reached_total: got %f, want >0", got)
}
}

View File

@@ -78,6 +78,19 @@ func (app *Vminsert) ClusternativeListenAddr() string {
return app.clusternativeListenAddr
}
// InfluxWrite is a test helper function that inserts a
// collection of records in Influx line format by sending a HTTP
// POST request to /influx/write vmsingle endpoint.
//
// See https://docs.victoriametrics.com/url-examples/#influxwrite
func (app *Vminsert) InfluxWrite(t *testing.T, records []string, opts QueryOpts) {
t.Helper()
url := fmt.Sprintf("http://%s/insert/%s/influx/write", app.httpListenAddr, opts.getTenant())
data := []byte(strings.Join(records, "\n"))
app.cli.Post(t, url, "text/plain", data, http.StatusNoContent)
}
// PrometheusAPIV1Write is a test helper function that inserts a
// collection of records in Prometheus remote-write format by sending a HTTP
// POST request to /prometheus/api/v1/write vminsert endpoint.

View File

@@ -1,12 +1,7 @@
FROM node:lts-alpine3.20
ENV PATH="/home/node/node_modules/.bin:$PATH"
WORKDIR /home/node
COPY package.json .
COPY package-lock.json .
RUN npm ci
WORKDIR /victoriametrics
WORKDIR /opt/node
COPY yarn.lock package.json .
RUN yarn install
WORKDIR /vm
ENTRYPOINT ["/opt/node/node_modules/.bin/cspell"]

View File

@@ -2,27 +2,27 @@
# Builds cspell image.
cspell-install:
@ (docker inspect cspell > /dev/null) || (docker build cspell --tag cspell)
@docker build cspell -t cspell
# Checks for spelling errors.
cspell-check: cspell-install
@CMD="cspell --no-progress" $(MAKE) cspell-run-command
cspell-check: CMD="--no-progress -r /vm"
cspell-check: cspell-install cspell-run
# Runs spelling error check.
# A user facing alias to cspell-check command.
spellcheck: cspell-check
# Runs cspell container commands.
cspell-run-command:
@cp cspell/cspell.json cspell.json
cspell-run:
@-docker run \
--entrypoint /bin/sh \
--mount type=bind,src=".",dst=/victoriametrics \
--mount type=bind,src="$(PWD)",dst=/vm \
--rm \
--tty \
cspell -c "$(CMD)"
@rm cspell.json
cspell -c cspell/cspell.json "$(CMD)"
cspell-update-deps: cspell-install
@CMD="cd /victoriametrics/cspell && npm update && rm -rf ./node_modules" $(MAKE) cspell-run-command
@-docker run \
--mount type=bind,src="$(PWD)",dst=/vm \
--entrypoint=/bin/sh \
--workdir=/vm/cspell \
--rm \
cspell -c "yarn install && yarn upgrade && rm -rf ./node_modules"

View File

@@ -7,7 +7,7 @@
{
"addWords": true,
"name": "custom-dict",
"path": "cspell/custom-dict.txt"
"path": "custom-dict.txt"
}
],
"dictionaries": [
@@ -50,18 +50,18 @@
"/vm[a-zA-Z0-9-_]+/i"
],
"import": [
"/home/node/node_modules/@cspell/dict-aws/cspell-ext.json",
"/home/node/node_modules/@cspell/dict-companies/cspell-ext.json",
"/home/node/node_modules/@cspell/dict-data-science/cspell-ext.json",
"/home/node/node_modules/@cspell/dict-en_us/cspell-ext.json",
"/home/node/node_modules/@cspell/dict-fullstack/cspell-ext.json",
"/home/node/node_modules/@cspell/dict-golang/cspell-ext.json",
"/home/node/node_modules/@cspell/dict-k8s/cspell-ext.json",
"/home/node/node_modules/@cspell/dict-people-names/cspell-ext.json",
"/home/node/node_modules/@cspell/dict-ru_ru/cspell-ext.json",
"/home/node/node_modules/@cspell/dict-software-terms/cspell-ext.json",
"/home/node/node_modules/@cspell/dict-uk-ua/cspell-ext.json",
"/home/node/node_modules/@cspell/dict-win32/cspell-ext.json"
"@cspell/dict-aws/cspell-ext.json",
"@cspell/dict-companies/cspell-ext.json",
"@cspell/dict-data-science/cspell-ext.json",
"@cspell/dict-en_us/cspell-ext.json",
"@cspell/dict-fullstack/cspell-ext.json",
"@cspell/dict-golang/cspell-ext.json",
"@cspell/dict-k8s/cspell-ext.json",
"@cspell/dict-people-names/cspell-ext.json",
"@cspell/dict-ru_ru/cspell-ext.json",
"@cspell/dict-software-terms/cspell-ext.json",
"@cspell/dict-uk-ua/cspell-ext.json",
"@cspell/dict-win32/cspell-ext.json"
],
"useGitignore": true
}

View File

@@ -145,3 +145,5 @@ SSZ
DDZ
DOKS
iforest
deltatocumulative
TLSCA

1445
cspell/package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -11,6 +11,6 @@
"@cspell/dict-software-terms": "^4.1.3",
"@cspell/dict-uk-ua": "^4.0.1",
"@cspell/dict-win32": "^2.0.3",
"cspell": "^8.14.2"
"cspell": "^8.17.1"
}
}

793
cspell/yarn.lock Normal file
View File

@@ -0,0 +1,793 @@
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
"@cspell/cspell-bundled-dicts@8.17.1":
version "8.17.1"
resolved "https://registry.yarnpkg.com/@cspell/cspell-bundled-dicts/-/cspell-bundled-dicts-8.17.1.tgz#61adad73f1bb1e12b182ffa04423d6052b18f0fc"
integrity sha512-HmkXS5uX4bk/XxsRS4Q+zRvhgRa81ddGiR2/Xfag9MIi5L5UnEJ4g21EpmIlXkMxYrTu2fp69SZFss5NfcFF9Q==
dependencies:
"@cspell/dict-ada" "^4.0.5"
"@cspell/dict-al" "^1.0.3"
"@cspell/dict-aws" "^4.0.7"
"@cspell/dict-bash" "^4.1.8"
"@cspell/dict-companies" "^3.1.8"
"@cspell/dict-cpp" "^6.0.2"
"@cspell/dict-cryptocurrencies" "^5.0.3"
"@cspell/dict-csharp" "^4.0.5"
"@cspell/dict-css" "^4.0.16"
"@cspell/dict-dart" "^2.2.4"
"@cspell/dict-django" "^4.1.3"
"@cspell/dict-docker" "^1.1.11"
"@cspell/dict-dotnet" "^5.0.8"
"@cspell/dict-elixir" "^4.0.6"
"@cspell/dict-en-common-misspellings" "^2.0.7"
"@cspell/dict-en-gb" "1.1.33"
"@cspell/dict-en_us" "^4.3.28"
"@cspell/dict-filetypes" "^3.0.9"
"@cspell/dict-flutter" "^1.0.3"
"@cspell/dict-fonts" "^4.0.3"
"@cspell/dict-fsharp" "^1.0.4"
"@cspell/dict-fullstack" "^3.2.3"
"@cspell/dict-gaming-terms" "^1.0.9"
"@cspell/dict-git" "^3.0.3"
"@cspell/dict-golang" "^6.0.17"
"@cspell/dict-google" "^1.0.4"
"@cspell/dict-haskell" "^4.0.4"
"@cspell/dict-html" "^4.0.10"
"@cspell/dict-html-symbol-entities" "^4.0.3"
"@cspell/dict-java" "^5.0.10"
"@cspell/dict-julia" "^1.0.4"
"@cspell/dict-k8s" "^1.0.9"
"@cspell/dict-latex" "^4.0.3"
"@cspell/dict-lorem-ipsum" "^4.0.3"
"@cspell/dict-lua" "^4.0.6"
"@cspell/dict-makefile" "^1.0.3"
"@cspell/dict-markdown" "^2.0.7"
"@cspell/dict-monkeyc" "^1.0.9"
"@cspell/dict-node" "^5.0.5"
"@cspell/dict-npm" "^5.1.17"
"@cspell/dict-php" "^4.0.13"
"@cspell/dict-powershell" "^5.0.13"
"@cspell/dict-public-licenses" "^2.0.11"
"@cspell/dict-python" "^4.2.13"
"@cspell/dict-r" "^2.0.4"
"@cspell/dict-ruby" "^5.0.7"
"@cspell/dict-rust" "^4.0.10"
"@cspell/dict-scala" "^5.0.6"
"@cspell/dict-software-terms" "^4.1.19"
"@cspell/dict-sql" "^2.1.8"
"@cspell/dict-svelte" "^1.0.5"
"@cspell/dict-swift" "^2.0.4"
"@cspell/dict-terraform" "^1.0.6"
"@cspell/dict-typescript" "^3.1.11"
"@cspell/dict-vue" "^3.0.3"
"@cspell/cspell-json-reporter@8.17.1":
version "8.17.1"
resolved "https://registry.yarnpkg.com/@cspell/cspell-json-reporter/-/cspell-json-reporter-8.17.1.tgz#c1678665f183589e5fc19a1c0933b8d362165a43"
integrity sha512-EV9Xkh42Xw3aORvDZfxusICX91DDbqQpYdGKBdPGuhgxWOUYYZKpLXsHCmDkhruMPo2m5gDh++/OqjLRPZofKQ==
dependencies:
"@cspell/cspell-types" "8.17.1"
"@cspell/cspell-pipe@8.17.1":
version "8.17.1"
resolved "https://registry.yarnpkg.com/@cspell/cspell-pipe/-/cspell-pipe-8.17.1.tgz#c247d4bd1c8ec43c49c46dc4458f00489e98232b"
integrity sha512-uhC99Ox+OH3COSgShv4fpVHiotR70dNvAOSkzRvKVRzV6IGyFnxHjmyVVPEV0dsqzVLxltwYTqFhwI+UOwm45A==
"@cspell/cspell-resolver@8.17.1":
version "8.17.1"
resolved "https://registry.yarnpkg.com/@cspell/cspell-resolver/-/cspell-resolver-8.17.1.tgz#6377c9c8c05c940fee675c74e31f893b7b2f38ab"
integrity sha512-XEK2ymTdQNgsV3ny60VkKzWskbICl4zNXh/DbxsoRXHqIRg43MXFpTNkEJ7j873EqdX7BU4opQQ+5D4stWWuhQ==
dependencies:
global-directory "^4.0.1"
"@cspell/cspell-service-bus@8.17.1":
version "8.17.1"
resolved "https://registry.yarnpkg.com/@cspell/cspell-service-bus/-/cspell-service-bus-8.17.1.tgz#8d6d82ea3ab0fc9d7efed8523b070e4842780bd1"
integrity sha512-2sFWQtMEWZ4tdz7bw0bAx4NaV1t0ynGfjpuKWdQppsJFKNb+ZPZZ6Ah1dC13AdRRMZaG194kDRFwzNvRaCgWkQ==
"@cspell/cspell-types@8.17.1":
version "8.17.1"
resolved "https://registry.yarnpkg.com/@cspell/cspell-types/-/cspell-types-8.17.1.tgz#5512030b4c2e7881a8822ab3afabbd4f5ddffb6f"
integrity sha512-NJbov7Jp57fh8addoxesjb8atg/APQfssCH5Q9uZuHBN06wEJDgs7fhfE48bU+RBViC9gltblsYZzZZQKzHYKg==
"@cspell/dict-ada@^4.0.5":
version "4.0.5"
resolved "https://registry.yarnpkg.com/@cspell/dict-ada/-/dict-ada-4.0.5.tgz#c14aae2faaecbad2d99f0d701e4700a48c68ef60"
integrity sha512-6/RtZ/a+lhFVmrx/B7bfP7rzC4yjEYe8o74EybXcvu4Oue6J4Ey2WSYj96iuodloj1LWrkNCQyX5h4Pmcj0Iag==
"@cspell/dict-al@^1.0.3":
version "1.0.3"
resolved "https://registry.yarnpkg.com/@cspell/dict-al/-/dict-al-1.0.3.tgz#09e288b5ab56b126dce895d3301faf7c0dd732d6"
integrity sha512-V1HClwlfU/qwSq2Kt+MkqRAsonNu3mxjSCDyGRecdLGIHmh7yeEeaxqRiO/VZ4KP+eVSiSIlbwrb5YNFfxYZbw==
"@cspell/dict-aws@^4.0.4", "@cspell/dict-aws@^4.0.7":
version "4.0.7"
resolved "https://registry.yarnpkg.com/@cspell/dict-aws/-/dict-aws-4.0.7.tgz#f96f3b70cd52a25b895eb08e297de5a5cc3fc5b6"
integrity sha512-PoaPpa2NXtSkhGIMIKhsJUXB6UbtTt6Ao3x9JdU9kn7fRZkwD4RjHDGqulucIOz7KeEX/dNRafap6oK9xHe4RA==
"@cspell/dict-bash@^4.1.8":
version "4.1.8"
resolved "https://registry.yarnpkg.com/@cspell/dict-bash/-/dict-bash-4.1.8.tgz#26dc898e06eddea069cf1ad475ee0e867c89e632"
integrity sha512-I2CM2pTNthQwW069lKcrVxchJGMVQBzru2ygsHCwgidXRnJL/NTjAPOFTxN58Jc1bf7THWghfEDyKX/oyfc0yg==
"@cspell/dict-companies@^3.1.8":
version "3.1.10"
resolved "https://registry.yarnpkg.com/@cspell/dict-companies/-/dict-companies-3.1.10.tgz#1351959ba14a2c1f0678dd99069240748a724aa2"
integrity sha512-KpRLiVDCpTkF+IjWnuYc31B0gyHVh0TSf/MDrWPobl9oYNQRWFUMACAJO9FP+kHI0jzLjTyLC1KpKwqte/88iA==
"@cspell/dict-cpp@^6.0.2":
version "6.0.2"
resolved "https://registry.yarnpkg.com/@cspell/dict-cpp/-/dict-cpp-6.0.2.tgz#e4549ee1bdf4b6402c0b978eb9dd3deac0eb05df"
integrity sha512-yw5eejWvY4bAnc6LUA44m4WsFwlmgPt2uMSnO7QViGMBDuoeopMma4z9XYvs4lSjTi8fIJs/A1YDfM9AVzb8eg==
"@cspell/dict-cryptocurrencies@^5.0.3":
version "5.0.3"
resolved "https://registry.yarnpkg.com/@cspell/dict-cryptocurrencies/-/dict-cryptocurrencies-5.0.3.tgz#502f9fffcb2835a3379668ddebdc487678ce6207"
integrity sha512-bl5q+Mk+T3xOZ12+FG37dB30GDxStza49Rmoax95n37MTLksk9wBo1ICOlPJ6PnDUSyeuv4SIVKgRKMKkJJglA==
"@cspell/dict-csharp@^4.0.5":
version "4.0.5"
resolved "https://registry.yarnpkg.com/@cspell/dict-csharp/-/dict-csharp-4.0.5.tgz#c677c50be09ca5bb3a2cc0be15f3cd05141fd2f7"
integrity sha512-c/sFnNgtRwRJxtC3JHKkyOm+U3/sUrltFeNwml9VsxKBHVmvlg4tk4ar58PdpW9/zTlGUkWi2i85//DN1EsUCA==
"@cspell/dict-css@^4.0.16":
version "4.0.16"
resolved "https://registry.yarnpkg.com/@cspell/dict-css/-/dict-css-4.0.16.tgz#b7b87b5ea0f1157b023205bdb00070a7d231e367"
integrity sha512-70qu7L9z/JR6QLyJPk38fNTKitlIHnfunx0wjpWQUQ8/jGADIhMCrz6hInBjqPNdtGpYm8d1dNFyF8taEkOgrQ==
"@cspell/dict-dart@^2.2.4":
version "2.2.4"
resolved "https://registry.yarnpkg.com/@cspell/dict-dart/-/dict-dart-2.2.4.tgz#8b877161ccdc65cead912b742b71aa55099c1706"
integrity sha512-of/cVuUIZZK/+iqefGln8G3bVpfyN6ZtH+LyLkHMoR5tEj+2vtilGNk9ngwyR8L4lEqbKuzSkOxgfVjsXf5PsQ==
"@cspell/dict-data-science@^2.0.1", "@cspell/dict-data-science@^2.0.5":
version "2.0.5"
resolved "https://registry.yarnpkg.com/@cspell/dict-data-science/-/dict-data-science-2.0.5.tgz#816e9b394c2a423d14cdc9a5de5d6fc6141d3900"
integrity sha512-nNSILXmhSJox9/QoXICPQgm8q5PbiSQP4afpbkBqPi/u/b3K9MbNH5HvOOa6230gxcGdbZ9Argl2hY/U8siBlg==
"@cspell/dict-django@^4.1.3":
version "4.1.3"
resolved "https://registry.yarnpkg.com/@cspell/dict-django/-/dict-django-4.1.3.tgz#a02a4a9ef8c9f47344f2d4a0c3964bcb62069ef5"
integrity sha512-yBspeL3roJlO0a1vKKNaWABURuHdHZ9b1L8d3AukX0AsBy9snSggc8xCavPmSzNfeMDXbH+1lgQiYBd3IW03fg==
"@cspell/dict-docker@^1.1.11":
version "1.1.11"
resolved "https://registry.yarnpkg.com/@cspell/dict-docker/-/dict-docker-1.1.11.tgz#6fce86eb6d86d73f77e18d3e7b9747bad3ca98de"
integrity sha512-s0Yhb16/R+UT1y727ekbR/itWQF3Qz275DR1ahOa66wYtPjHUXmhM3B/LT3aPaX+hD6AWmK23v57SuyfYHUjsw==
"@cspell/dict-dotnet@^5.0.8":
version "5.0.8"
resolved "https://registry.yarnpkg.com/@cspell/dict-dotnet/-/dict-dotnet-5.0.8.tgz#8a110ca302946025e0273a9940079483ec33a88a"
integrity sha512-MD8CmMgMEdJAIPl2Py3iqrx3B708MbCIXAuOeZ0Mzzb8YmLmiisY7QEYSZPg08D7xuwARycP0Ki+bb0GAkFSqg==
"@cspell/dict-elixir@^4.0.6":
version "4.0.6"
resolved "https://registry.yarnpkg.com/@cspell/dict-elixir/-/dict-elixir-4.0.6.tgz#3d8965c558d8afd190356e9a900b02c546741feb"
integrity sha512-TfqSTxMHZ2jhiqnXlVKM0bUADtCvwKQv2XZL/DI0rx3doG8mEMS8SGPOmiyyGkHpR/pGOq18AFH3BEm4lViHIw==
"@cspell/dict-en-common-misspellings@^2.0.7":
version "2.0.7"
resolved "https://registry.yarnpkg.com/@cspell/dict-en-common-misspellings/-/dict-en-common-misspellings-2.0.7.tgz#62861cc9e813c947ebd71c7a50fc720767b4b543"
integrity sha512-qNFo3G4wyabcwnM+hDrMYKN9vNVg/k9QkhqSlSst6pULjdvPyPs1mqz1689xO/v9t8e6sR4IKc3CgUXDMTYOpA==
"@cspell/dict-en-gb@1.1.33":
version "1.1.33"
resolved "https://registry.yarnpkg.com/@cspell/dict-en-gb/-/dict-en-gb-1.1.33.tgz#7f1fd90fc364a5cb77111b5438fc9fcf9cc6da0e"
integrity sha512-tKSSUf9BJEV+GJQAYGw5e+ouhEe2ZXE620S7BLKe3ZmpnjlNG9JqlnaBhkIMxKnNFkLY2BP/EARzw31AZnOv4g==
"@cspell/dict-en_us@^4.3.23", "@cspell/dict-en_us@^4.3.28":
version "4.3.28"
resolved "https://registry.yarnpkg.com/@cspell/dict-en_us/-/dict-en_us-4.3.28.tgz#41169e1ed18465e7ff367a4f4488d4cbc6cf0baa"
integrity sha512-BN1PME7cOl7DXRQJ92pEd1f0Xk5sqjcDfThDGkKcsgwbSOY7KnTc/czBW6Pr3WXIchIm6cT12KEfjNqx7U7Rrw==
"@cspell/dict-filetypes@^3.0.9":
version "3.0.9"
resolved "https://registry.yarnpkg.com/@cspell/dict-filetypes/-/dict-filetypes-3.0.9.tgz#f4d5c35c341e6c3b77c08aec00678412641e1504"
integrity sha512-U7ycC1cE32A5aEgwzp/iE0TVabonUFnVt+Ygbf6NsIWqEuFWZgZChC7gfztA4T1fpuj602nFdp7eOnTWKORsnQ==
"@cspell/dict-flutter@^1.0.3":
version "1.0.3"
resolved "https://registry.yarnpkg.com/@cspell/dict-flutter/-/dict-flutter-1.0.3.tgz#23e552209ab2238733d30ca3f2a141359756af51"
integrity sha512-52C9aUEU22ptpgYh6gQyIdA4MP6NPwzbEqndfgPh3Sra191/kgs7CVqXiO1qbtZa9gnYHUoVApkoxRE7mrXHfg==
"@cspell/dict-fonts@^4.0.3":
version "4.0.3"
resolved "https://registry.yarnpkg.com/@cspell/dict-fonts/-/dict-fonts-4.0.3.tgz#abf578c10a2e7b2bd8f4374002677625288560d9"
integrity sha512-sPd17kV5qgYXLteuHFPn5mbp/oCHKgitNfsZLFC3W2fWEgZlhg4hK+UGig3KzrYhhvQ8wBnmZrAQm0TFKCKzsA==
"@cspell/dict-fsharp@^1.0.4":
version "1.0.4"
resolved "https://registry.yarnpkg.com/@cspell/dict-fsharp/-/dict-fsharp-1.0.4.tgz#19a7263a61ca89cd3ec9c17537e424907b81ef38"
integrity sha512-G5wk0o1qyHUNi9nVgdE1h5wl5ylq7pcBjX8vhjHcO4XBq20D5eMoXjwqMo/+szKAqzJ+WV3BgAL50akLKrT9Rw==
"@cspell/dict-fullstack@^3.2.0", "@cspell/dict-fullstack@^3.2.3":
version "3.2.3"
resolved "https://registry.yarnpkg.com/@cspell/dict-fullstack/-/dict-fullstack-3.2.3.tgz#f6fff74eff00c6759cba510168acada0619004cc"
integrity sha512-62PbndIyQPH11mAv0PyiyT0vbwD0AXEocPpHlCHzfb5v9SspzCCbzQ/LIBiFmyRa+q5LMW35CnSVu6OXdT+LKg==
"@cspell/dict-gaming-terms@^1.0.9":
version "1.0.9"
resolved "https://registry.yarnpkg.com/@cspell/dict-gaming-terms/-/dict-gaming-terms-1.0.9.tgz#6b920386d281b89f70857e6dacea10ab89e88658"
integrity sha512-AVIrZt3YiUnxsUzzGYTZ1XqgtkgwGEO0LWIlEf+SiDUEVLtv4CYmmyXFQ+WXDN0pyJ0wOwDazWrP0Cu7avYQmQ==
"@cspell/dict-git@^3.0.3":
version "3.0.3"
resolved "https://registry.yarnpkg.com/@cspell/dict-git/-/dict-git-3.0.3.tgz#3a3805ab9902bffc9255ec48f648145b957eb30b"
integrity sha512-LSxB+psZ0qoj83GkyjeEH/ZViyVsGEF/A6BAo8Nqc0w0HjD2qX/QR4sfA6JHUgQ3Yi/ccxdK7xNIo67L2ScW5A==
"@cspell/dict-golang@^6.0.12", "@cspell/dict-golang@^6.0.17":
version "6.0.17"
resolved "https://registry.yarnpkg.com/@cspell/dict-golang/-/dict-golang-6.0.17.tgz#8f3c11189b869db7216cb4496514b9882d1e30a5"
integrity sha512-uDDLEJ/cHdLiqPw4+5BnmIo2i/TSR+uDvYd6JlBjTmjBKpOCyvUgYRztH7nv5e7virsN5WDiUWah4/ATQGz4Pw==
"@cspell/dict-google@^1.0.4":
version "1.0.4"
resolved "https://registry.yarnpkg.com/@cspell/dict-google/-/dict-google-1.0.4.tgz#e15a7ea2dee73800231a81840a59d3b50d49346f"
integrity sha512-JThUT9eiguCja1mHHLwYESgxkhk17Gv7P3b1S7ZJzXw86QyVHPrbpVoMpozHk0C9o+Ym764B7gZGKmw9uMGduQ==
"@cspell/dict-haskell@^4.0.4":
version "4.0.4"
resolved "https://registry.yarnpkg.com/@cspell/dict-haskell/-/dict-haskell-4.0.4.tgz#37e9cb9a7f5be337a697bcffd0a0d25e80aab50d"
integrity sha512-EwQsedEEnND/vY6tqRfg9y7tsnZdxNqOxLXSXTsFA6JRhUlr8Qs88iUUAfsUzWc4nNmmzQH2UbtT25ooG9x4nA==
"@cspell/dict-html-symbol-entities@^4.0.3":
version "4.0.3"
resolved "https://registry.yarnpkg.com/@cspell/dict-html-symbol-entities/-/dict-html-symbol-entities-4.0.3.tgz#bf2887020ca4774413d8b1f27c9b6824ba89e9ef"
integrity sha512-aABXX7dMLNFdSE8aY844X4+hvfK7977sOWgZXo4MTGAmOzR8524fjbJPswIBK7GaD3+SgFZ2yP2o0CFvXDGF+A==
"@cspell/dict-html@^4.0.10":
version "4.0.10"
resolved "https://registry.yarnpkg.com/@cspell/dict-html/-/dict-html-4.0.10.tgz#7b536b2adca4b58ed92752c9d3c7ffc724dd5991"
integrity sha512-I9uRAcdtHbh0wEtYZlgF0TTcgH0xaw1B54G2CW+tx4vHUwlde/+JBOfIzird4+WcMv4smZOfw+qHf7puFUbI5g==
"@cspell/dict-java@^5.0.10":
version "5.0.10"
resolved "https://registry.yarnpkg.com/@cspell/dict-java/-/dict-java-5.0.10.tgz#e6383ca645046b9f05a04a2c2e858fcc80c6fc63"
integrity sha512-pVNcOnmoGiNL8GSVq4WbX/Vs2FGS0Nej+1aEeGuUY9CU14X8yAVCG+oih5ZoLt1jaR8YfR8byUF8wdp4qG4XIw==
"@cspell/dict-julia@^1.0.4":
version "1.0.4"
resolved "https://registry.yarnpkg.com/@cspell/dict-julia/-/dict-julia-1.0.4.tgz#e478c20d742cd6857b6de41dc61a92036dafb4bc"
integrity sha512-bFVgNX35MD3kZRbXbJVzdnN7OuEqmQXGpdOi9jzB40TSgBTlJWA4nxeAKV4CPCZxNRUGnLH0p05T/AD7Aom9/w==
"@cspell/dict-k8s@^1.0.6", "@cspell/dict-k8s@^1.0.9":
version "1.0.9"
resolved "https://registry.yarnpkg.com/@cspell/dict-k8s/-/dict-k8s-1.0.9.tgz#e9392a002797c67ffc3e96893156cc15af3774d1"
integrity sha512-Q7GELSQIzo+BERl2ya/nBEnZeQC+zJP19SN1pI6gqDYraM51uYJacbbcWLYYO2Y+5joDjNt/sd/lJtLaQwoSlA==
"@cspell/dict-latex@^4.0.3":
version "4.0.3"
resolved "https://registry.yarnpkg.com/@cspell/dict-latex/-/dict-latex-4.0.3.tgz#a1254c7d9c3a2d70cd6391a9f2f7694431b1b2cb"
integrity sha512-2KXBt9fSpymYHxHfvhUpjUFyzrmN4c4P8mwIzweLyvqntBT3k0YGZJSriOdjfUjwSygrfEwiuPI1EMrvgrOMJw==
"@cspell/dict-lorem-ipsum@^4.0.3":
version "4.0.3"
resolved "https://registry.yarnpkg.com/@cspell/dict-lorem-ipsum/-/dict-lorem-ipsum-4.0.3.tgz#c5fc631d934f1daf8b10c88b795278701a2469ec"
integrity sha512-WFpDi/PDYHXft6p0eCXuYnn7mzMEQLVeqpO+wHSUd+kz5ADusZ4cpslAA4wUZJstF1/1kMCQCZM6HLZic9bT8A==
"@cspell/dict-lua@^4.0.6":
version "4.0.6"
resolved "https://registry.yarnpkg.com/@cspell/dict-lua/-/dict-lua-4.0.6.tgz#7de412bfaead794445e26d566aec222e20ad69ba"
integrity sha512-Jwvh1jmAd9b+SP9e1GkS2ACbqKKRo9E1f9GdjF/ijmooZuHU0hPyqvnhZzUAxO1egbnNjxS/J2T6iUtjAUK2KQ==
"@cspell/dict-makefile@^1.0.3":
version "1.0.3"
resolved "https://registry.yarnpkg.com/@cspell/dict-makefile/-/dict-makefile-1.0.3.tgz#08d3349bf7cbd8f5dacf8641f3d35092ca0b8b38"
integrity sha512-R3U0DSpvTs6qdqfyBATnePj9Q/pypkje0Nj26mQJ8TOBQutCRAJbr2ZFAeDjgRx5EAJU/+8txiyVF97fbVRViw==
"@cspell/dict-markdown@^2.0.7":
version "2.0.7"
resolved "https://registry.yarnpkg.com/@cspell/dict-markdown/-/dict-markdown-2.0.7.tgz#15d6f9eed6bd1b33921b4332426ff387961163f1"
integrity sha512-F9SGsSOokFn976DV4u/1eL4FtKQDSgJHSZ3+haPRU5ki6OEqojxKa8hhj4AUrtNFpmBaJx/WJ4YaEzWqG7hgqg==
"@cspell/dict-monkeyc@^1.0.9":
version "1.0.9"
resolved "https://registry.yarnpkg.com/@cspell/dict-monkeyc/-/dict-monkeyc-1.0.9.tgz#58b5f6f15fc7c11ce0eeffd0742fba4b39fc0b8b"
integrity sha512-Jvf6g5xlB4+za3ThvenYKREXTEgzx5gMUSzrAxIiPleVG4hmRb/GBSoSjtkGaibN3XxGx5x809gSTYCA/IHCpA==
"@cspell/dict-node@^5.0.5":
version "5.0.5"
resolved "https://registry.yarnpkg.com/@cspell/dict-node/-/dict-node-5.0.5.tgz#11653612ebdd833208432e8b3cbe61bd6dd35dc3"
integrity sha512-7NbCS2E8ZZRZwlLrh2sA0vAk9n1kcTUiRp/Nia8YvKaItGXLfxYqD2rMQ3HpB1kEutal6hQLVic3N2Yi1X7AaA==
"@cspell/dict-npm@^5.1.17":
version "5.1.20"
resolved "https://registry.yarnpkg.com/@cspell/dict-npm/-/dict-npm-5.1.20.tgz#5e54b428d7609267263d426d0edc9d51d53f0a6f"
integrity sha512-vE9pFIifCDChsVhhUDuVtnwxygOdtHNluDm+8FkgC84M6LwiUVJr/CuSOI/SCR0oI9iiFp0VvMz194B6XwMv3g==
"@cspell/dict-people-names@^1.1.1":
version "1.1.4"
resolved "https://registry.yarnpkg.com/@cspell/dict-people-names/-/dict-people-names-1.1.4.tgz#e5a9bf51f0d6aae7f51cd1f2d2eb218f8f5bdba8"
integrity sha512-YgnstmeGj/FoIEX996w6SpzTHyNHNxBinwbgozciEo4KJEQkdR7K1GIvJJdRh5Bh81T7xY7PCb6l4nStCFyclA==
"@cspell/dict-php@^4.0.13":
version "4.0.13"
resolved "https://registry.yarnpkg.com/@cspell/dict-php/-/dict-php-4.0.13.tgz#86f1e6fb2174b2b0fa012baf86c448b2730f04f9"
integrity sha512-P6sREMZkhElzz/HhXAjahnICYIqB/HSGp1EhZh+Y6IhvC15AzgtDP8B8VYCIsQof6rPF1SQrFwunxOv8H1e2eg==
"@cspell/dict-powershell@^5.0.13":
version "5.0.13"
resolved "https://registry.yarnpkg.com/@cspell/dict-powershell/-/dict-powershell-5.0.13.tgz#f557aa04ee9bda4fe091308a0bcaea09ed12fa76"
integrity sha512-0qdj0XZIPmb77nRTynKidRJKTU0Fl+10jyLbAhFTuBWKMypVY06EaYFnwhsgsws/7nNX8MTEQuewbl9bWFAbsg==
"@cspell/dict-public-licenses@^2.0.11":
version "2.0.11"
resolved "https://registry.yarnpkg.com/@cspell/dict-public-licenses/-/dict-public-licenses-2.0.11.tgz#37550c4e0cd445991caba528bf4ba58ce7a935c3"
integrity sha512-rR5KjRUSnVKdfs5G+gJ4oIvQvm8+NJ6cHWY2N+GE69/FSGWDOPHxulCzeGnQU/c6WWZMSimG9o49i9r//lUQyA==
"@cspell/dict-python@^4.2.13":
version "4.2.13"
resolved "https://registry.yarnpkg.com/@cspell/dict-python/-/dict-python-4.2.13.tgz#c3dbaa7e2434c835e11540345e2168e5e685190a"
integrity sha512-mZIcmo9qif8LkJ6N/lqTZawcOk2kVTcuWIUOSbMcjyomO0XZ7iWz15TfONyr03Ea/l7o5ULV+MZ4vx76bAUb7w==
dependencies:
"@cspell/dict-data-science" "^2.0.5"
"@cspell/dict-r@^2.0.4":
version "2.0.4"
resolved "https://registry.yarnpkg.com/@cspell/dict-r/-/dict-r-2.0.4.tgz#31b5abd91cc12aebfffdde4be4d2902668789311"
integrity sha512-cBpRsE/U0d9BRhiNRMLMH1PpWgw+N+1A2jumgt1if9nBGmQw4MUpg2u9I0xlFVhstTIdzXiLXMxP45cABuiUeQ==
"@cspell/dict-ru_ru@^2.2.1":
version "2.2.4"
resolved "https://registry.yarnpkg.com/@cspell/dict-ru_ru/-/dict-ru_ru-2.2.4.tgz#a0b4a9272ca360b1721ef999e02559dc636d4043"
integrity sha512-Ub5Y318ZAaFJDAPgeImcLg8ksfthGhxMHsyHGkn9Uf3g9AZUlYsabs1HwgLmh9NtqDNjMlF52S9R11GFDdaWIw==
"@cspell/dict-ruby@^5.0.7":
version "5.0.7"
resolved "https://registry.yarnpkg.com/@cspell/dict-ruby/-/dict-ruby-5.0.7.tgz#3593a955baaffe3c5d28fb178b72fdf93c7eec71"
integrity sha512-4/d0hcoPzi5Alk0FmcyqlzFW9lQnZh9j07MJzPcyVO62nYJJAGKaPZL2o4qHeCS/od/ctJC5AHRdoUm0ktsw6Q==
"@cspell/dict-rust@^4.0.10":
version "4.0.10"
resolved "https://registry.yarnpkg.com/@cspell/dict-rust/-/dict-rust-4.0.10.tgz#8ae6eaf31a0ebce9dc8fd8dd68e5925e1d5290ee"
integrity sha512-6o5C8566VGTTctgcwfF3Iy7314W0oMlFFSQOadQ0OEdJ9Z9ERX/PDimrzP3LGuOrvhtEFoK8pj+BLnunNwRNrw==
"@cspell/dict-scala@^5.0.6":
version "5.0.6"
resolved "https://registry.yarnpkg.com/@cspell/dict-scala/-/dict-scala-5.0.6.tgz#5e925def2fe6dc27ee2ad1c452941c3d6790fb6d"
integrity sha512-tl0YWAfjUVb4LyyE4JIMVE8DlLzb1ecHRmIWc4eT6nkyDqQgHKzdHsnusxFEFMVLIQomgSg0Zz6hJ5S1E4W4ww==
"@cspell/dict-software-terms@^4.1.19", "@cspell/dict-software-terms@^4.1.3":
version "4.2.0"
resolved "https://registry.yarnpkg.com/@cspell/dict-software-terms/-/dict-software-terms-4.2.0.tgz#c16e9d3326c36195dadea812bc035817054bf0c1"
integrity sha512-cTLTNdP9RM6nruZ01FThEFKRi7C4TPN8ndc+FpvCqis9J8iSg4Cr4YQemT/DxXoXz0527NbBPCARunxA0qIgTA==
"@cspell/dict-sql@^2.1.8":
version "2.1.8"
resolved "https://registry.yarnpkg.com/@cspell/dict-sql/-/dict-sql-2.1.8.tgz#45ea53b3e57fd2cc5f839f49b644aa743dac4990"
integrity sha512-dJRE4JV1qmXTbbGm6WIcg1knmR6K5RXnQxF4XHs5HA3LAjc/zf77F95i5LC+guOGppVF6Hdl66S2UyxT+SAF3A==
"@cspell/dict-svelte@^1.0.5":
version "1.0.5"
resolved "https://registry.yarnpkg.com/@cspell/dict-svelte/-/dict-svelte-1.0.5.tgz#09752e01ff6667e737566d9dfc704c8dcc9a6492"
integrity sha512-sseHlcXOqWE4Ner9sg8KsjxwSJ2yssoJNqFHR9liWVbDV+m7kBiUtn2EB690TihzVsEmDr/0Yxrbb5Bniz70mA==
"@cspell/dict-swift@^2.0.4":
version "2.0.4"
resolved "https://registry.yarnpkg.com/@cspell/dict-swift/-/dict-swift-2.0.4.tgz#bc19522418ed68cf914736b612c4e4febbf07e8d"
integrity sha512-CsFF0IFAbRtYNg0yZcdaYbADF5F3DsM8C4wHnZefQy8YcHP/qjAF/GdGfBFBLx+XSthYuBlo2b2XQVdz3cJZBw==
"@cspell/dict-terraform@^1.0.6":
version "1.0.7"
resolved "https://registry.yarnpkg.com/@cspell/dict-terraform/-/dict-terraform-1.0.7.tgz#815a523c86f647cb7695d48b69e4a793c49f875e"
integrity sha512-Ip7tOlAt/qUVdWYyDMA7DlKMpQ6sjtrsXk4vcpqXoYpoJlzMoDce7pw+fPhHshtNOFBAZ4nOrszlLu6APuy+HQ==
"@cspell/dict-typescript@^3.1.11":
version "3.1.11"
resolved "https://registry.yarnpkg.com/@cspell/dict-typescript/-/dict-typescript-3.1.11.tgz#40586f13b0337bd9cba958e0661b35888580b249"
integrity sha512-FwvK5sKbwrVpdw0e9+1lVTl8FPoHYvfHRuQRQz2Ql5XkC0gwPPkpoyD1zYImjIyZRoYXk3yp9j8ss4iz7A7zoQ==
"@cspell/dict-uk-ua@^4.0.1":
version "4.0.4"
resolved "https://registry.yarnpkg.com/@cspell/dict-uk-ua/-/dict-uk-ua-4.0.4.tgz#19ecb8eaa70a363fd53a2adab0ea462f6e57e055"
integrity sha512-4n+ImjsKVWkcZC9O/zQY4oZ9KWm0qkyTZY5/aW1KZsP8sx0cK5IUaA/375R3i4l8SDc0WDZN8K6Wy71cggDUEA==
"@cspell/dict-vue@^3.0.3":
version "3.0.3"
resolved "https://registry.yarnpkg.com/@cspell/dict-vue/-/dict-vue-3.0.3.tgz#295c288f6fd363879898223202ec3be048663b98"
integrity sha512-akmYbrgAGumqk1xXALtDJcEcOMYBYMnkjpmGzH13Ozhq1mkPF4VgllFQlm1xYde+BUKNnzMgPEzxrL2qZllgYA==
"@cspell/dict-win32@^2.0.3":
version "2.0.6"
resolved "https://registry.yarnpkg.com/@cspell/dict-win32/-/dict-win32-2.0.6.tgz#3edaa576906061584b470aca1b41b6d76548eeca"
integrity sha512-znBvliYtgXIpMW1/T/VnrvA2CwEqfJd06I57laVQsa/ST7dUeUU2kTbk4A4ICCr3eemVMPl9NQtR+t3SQu+ogQ==
"@cspell/dynamic-import@8.17.1":
version "8.17.1"
resolved "https://registry.yarnpkg.com/@cspell/dynamic-import/-/dynamic-import-8.17.1.tgz#2b3f3325b6013a067a1a49cda8b69ae73aaed36a"
integrity sha512-XQtr2olYOtqbg49E+8SISd6I5DzfxmsKINDn0ZgaTFeLalnNdF3ewDU4gOEbApIzGffRa1mW9t19MsiVrznSDw==
dependencies:
"@cspell/url" "8.17.1"
import-meta-resolve "^4.1.0"
"@cspell/filetypes@8.17.1":
version "8.17.1"
resolved "https://registry.yarnpkg.com/@cspell/filetypes/-/filetypes-8.17.1.tgz#d193afc5029364334f005ff23f4c4cb80170c374"
integrity sha512-AxYw6j7EPYtDFAFjwybjFpMc9waXQzurfBXmEVfQ5RQRlbylujLZWwR6GnMqofeNg4oGDUpEjcAZFrgdkvMQlA==
"@cspell/strong-weak-map@8.17.1":
version "8.17.1"
resolved "https://registry.yarnpkg.com/@cspell/strong-weak-map/-/strong-weak-map-8.17.1.tgz#2fa88f283ef10222fad25134b5ebb54edaad985f"
integrity sha512-8cY3vLAKdt5gQEMM3Gr57BuQ8sun2NjYNh9qTdrctC1S9gNC7XzFghTYAfHSWR4VrOUcMFLO/izMdsc1KFvFOA==
"@cspell/url@8.17.1":
version "8.17.1"
resolved "https://registry.yarnpkg.com/@cspell/url/-/url-8.17.1.tgz#e7daec1597fa31b4d0a7a685e7a24a11b0c8a193"
integrity sha512-LMvReIndW1ckvemElfDgTt282fb2C3C/ZXfsm0pJsTV5ZmtdelCHwzmgSBmY5fDr7D66XDp8EurotSE0K6BTvw==
array-timsort@^1.0.3:
version "1.0.3"
resolved "https://registry.yarnpkg.com/array-timsort/-/array-timsort-1.0.3.tgz#3c9e4199e54fb2b9c3fe5976396a21614ef0d926"
integrity sha512-/+3GRL7dDAGEfM6TseQk/U+mi18TU2Ms9I3UlLdUMhz2hbvGNTKdj9xniwXfUqgYhHxRx0+8UnKkvlNwVU+cWQ==
braces@^3.0.3:
version "3.0.3"
resolved "https://registry.yarnpkg.com/braces/-/braces-3.0.3.tgz#490332f40919452272d55a8480adc0c441358789"
integrity sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA==
dependencies:
fill-range "^7.1.1"
callsites@^3.0.0, callsites@^3.1.0:
version "3.1.0"
resolved "https://registry.yarnpkg.com/callsites/-/callsites-3.1.0.tgz#b3630abd8943432f54b3f0519238e33cd7df2f73"
integrity sha512-P8BjAsXvZS+VIDUI11hHCQEv74YT67YUi5JJFNWIqL235sBmjX4+qx9Muvls5ivyNENctx46xQLQ3aTuE7ssaQ==
chalk-template@^1.1.0:
version "1.1.0"
resolved "https://registry.yarnpkg.com/chalk-template/-/chalk-template-1.1.0.tgz#ffc55db6dd745e9394b85327c8ac8466edb7a7b1"
integrity sha512-T2VJbcDuZQ0Tb2EWwSotMPJjgpy1/tGee1BTpUNsGZ/qgNjV2t7Mvu+d4600U564nbLesN1x2dPL+xii174Ekg==
dependencies:
chalk "^5.2.0"
chalk@^5.2.0, chalk@^5.3.0:
version "5.4.1"
resolved "https://registry.yarnpkg.com/chalk/-/chalk-5.4.1.tgz#1b48bf0963ec158dce2aacf69c093ae2dd2092d8"
integrity sha512-zgVZuo2WcZgfUEmsn6eO3kINexW8RAE4maiQ8QNs8CtpPCSyMiYsULR3HQYkm3w8FIA3SberyMJMSldGsW+U3w==
clear-module@^4.1.2:
version "4.1.2"
resolved "https://registry.yarnpkg.com/clear-module/-/clear-module-4.1.2.tgz#5a58a5c9f8dccf363545ad7284cad3c887352a80"
integrity sha512-LWAxzHqdHsAZlPlEyJ2Poz6AIs384mPeqLVCru2p0BrP9G/kVGuhNyZYClLO6cXlnuJjzC8xtsJIuMjKqLXoAw==
dependencies:
parent-module "^2.0.0"
resolve-from "^5.0.0"
commander@^12.1.0:
version "12.1.0"
resolved "https://registry.yarnpkg.com/commander/-/commander-12.1.0.tgz#01423b36f501259fdaac4d0e4d60c96c991585d3"
integrity sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA==
comment-json@^4.2.5:
version "4.2.5"
resolved "https://registry.yarnpkg.com/comment-json/-/comment-json-4.2.5.tgz#482e085f759c2704b60bc6f97f55b8c01bc41e70"
integrity sha512-bKw/r35jR3HGt5PEPm1ljsQQGyCrR8sFGNiN5L+ykDHdpO8Smxkrkla9Yi6NkQyUrb8V54PGhfMs6NrIwtxtdw==
dependencies:
array-timsort "^1.0.3"
core-util-is "^1.0.3"
esprima "^4.0.1"
has-own-prop "^2.0.0"
repeat-string "^1.6.1"
core-util-is@^1.0.3:
version "1.0.3"
resolved "https://registry.yarnpkg.com/core-util-is/-/core-util-is-1.0.3.tgz#a6042d3634c2b27e9328f837b965fac83808db85"
integrity sha512-ZQBvi1DcpJ4GDqanjucZ2Hj3wEO5pZDS89BWbkcrvdxksJorwUDDZamX9ldFkp9aw2lmBDLgkObEA4DWNJ9FYQ==
cspell-config-lib@8.17.1:
version "8.17.1"
resolved "https://registry.yarnpkg.com/cspell-config-lib/-/cspell-config-lib-8.17.1.tgz#a87973b91d51bf23a2018042c25aeaaa8a4e69c0"
integrity sha512-x1S7QWprgUcwuwiJB1Ng0ZTBC4G50qP9qQyg/aroMkcdMsHfk26E8jUGRPNt4ftHFzS4YMhwtXuJQ9IgRUuNPA==
dependencies:
"@cspell/cspell-types" "8.17.1"
comment-json "^4.2.5"
yaml "^2.6.1"
cspell-dictionary@8.17.1:
version "8.17.1"
resolved "https://registry.yarnpkg.com/cspell-dictionary/-/cspell-dictionary-8.17.1.tgz#bfc9bfdbd3720d1425260a98091acffab7b03dd5"
integrity sha512-zSl9l3wii+x16yc2NVZl/+CMLeLBAiuEd5YoFkOYPcbTJnfPwdjMNcj71u7wBvNJ+qwbF+kGbutEt15yHW3NBw==
dependencies:
"@cspell/cspell-pipe" "8.17.1"
"@cspell/cspell-types" "8.17.1"
cspell-trie-lib "8.17.1"
fast-equals "^5.0.1"
cspell-gitignore@8.17.1:
version "8.17.1"
resolved "https://registry.yarnpkg.com/cspell-gitignore/-/cspell-gitignore-8.17.1.tgz#38f3213a40ba86480bb5f66a91198db6e0ef37c0"
integrity sha512-bk727Zf4FBCjm9Mwvyreyhgjwe+YhPQEW7PldkHiinKd+Irfez4s8GXLQb1EgV0UpvViqaqBqLmngjZdS30BTA==
dependencies:
"@cspell/url" "8.17.1"
cspell-glob "8.17.1"
cspell-io "8.17.1"
find-up-simple "^1.0.0"
cspell-glob@8.17.1:
version "8.17.1"
resolved "https://registry.yarnpkg.com/cspell-glob/-/cspell-glob-8.17.1.tgz#23d1be46b32fb4933487e4edff347d34db446f5a"
integrity sha512-cUwM5auSt0RvLX7UkP2GEArJRWc85l51B1voArl+3ZIKeMZwcJpJgN3qvImtF8yRTZwYeYCs1sgsihb179q+mg==
dependencies:
"@cspell/url" "8.17.1"
micromatch "^4.0.8"
cspell-grammar@8.17.1:
version "8.17.1"
resolved "https://registry.yarnpkg.com/cspell-grammar/-/cspell-grammar-8.17.1.tgz#8f6619fbfaebff6aeee63b13d17898b4d0c09136"
integrity sha512-H5tLcBuW7aUj9L0rR+FSbnWPEsWb8lWppHVidtqw9Ll1CUHWOZC9HTB2RdrhJZrsz/8DJbM2yNbok0Xt0VAfdw==
dependencies:
"@cspell/cspell-pipe" "8.17.1"
"@cspell/cspell-types" "8.17.1"
cspell-io@8.17.1:
version "8.17.1"
resolved "https://registry.yarnpkg.com/cspell-io/-/cspell-io-8.17.1.tgz#b91f1cac1c64a6fa2b61a388d0dc67437fcf3ada"
integrity sha512-liIOsblt7oVItifzRAbuxiYrwlgw1VOqKppMxVKtYoAn2VUuuEpjCj6jLWpoTqSszR/38o7ChsHY1LHakhJZmw==
dependencies:
"@cspell/cspell-service-bus" "8.17.1"
"@cspell/url" "8.17.1"
cspell-lib@8.17.1:
version "8.17.1"
resolved "https://registry.yarnpkg.com/cspell-lib/-/cspell-lib-8.17.1.tgz#21c76f1ea4e91c90245e55acddbf452d055a6607"
integrity sha512-66n83Q7bK5tnvkDH7869/pBY/65AKmZVfCOAlsbhJn3YMDbNHFCHR0d1oNMlqG+n65Aco89VGwYfXxImZY+/mA==
dependencies:
"@cspell/cspell-bundled-dicts" "8.17.1"
"@cspell/cspell-pipe" "8.17.1"
"@cspell/cspell-resolver" "8.17.1"
"@cspell/cspell-types" "8.17.1"
"@cspell/dynamic-import" "8.17.1"
"@cspell/filetypes" "8.17.1"
"@cspell/strong-weak-map" "8.17.1"
"@cspell/url" "8.17.1"
clear-module "^4.1.2"
comment-json "^4.2.5"
cspell-config-lib "8.17.1"
cspell-dictionary "8.17.1"
cspell-glob "8.17.1"
cspell-grammar "8.17.1"
cspell-io "8.17.1"
cspell-trie-lib "8.17.1"
env-paths "^3.0.0"
fast-equals "^5.0.1"
gensequence "^7.0.0"
import-fresh "^3.3.0"
resolve-from "^5.0.0"
vscode-languageserver-textdocument "^1.0.12"
vscode-uri "^3.0.8"
xdg-basedir "^5.1.0"
cspell-trie-lib@8.17.1:
version "8.17.1"
resolved "https://registry.yarnpkg.com/cspell-trie-lib/-/cspell-trie-lib-8.17.1.tgz#618e5cc671b0a24cf7ec27a9a9b834b197e17392"
integrity sha512-13WNa5s75VwOjlGzWprmfNbBFIfXyA7tYYrbV+LugKkznyNZJeJPojHouEudcLq3SYb2Q6tJ7qyWcuT5bR9qPA==
dependencies:
"@cspell/cspell-pipe" "8.17.1"
"@cspell/cspell-types" "8.17.1"
gensequence "^7.0.0"
cspell@^8.17.1:
version "8.17.1"
resolved "https://registry.yarnpkg.com/cspell/-/cspell-8.17.1.tgz#be3c79a5b0b2e374ac0df8f921eb30ddca170110"
integrity sha512-D0lw8XTXrTycNzOn5DkfPJNUT00X53OgvFDm+0SzhBr1r+na8LEh3CnQ6zKYVU0fL0x8vU82vs4jmGjDho9mPg==
dependencies:
"@cspell/cspell-json-reporter" "8.17.1"
"@cspell/cspell-pipe" "8.17.1"
"@cspell/cspell-types" "8.17.1"
"@cspell/dynamic-import" "8.17.1"
"@cspell/url" "8.17.1"
chalk "^5.3.0"
chalk-template "^1.1.0"
commander "^12.1.0"
cspell-dictionary "8.17.1"
cspell-gitignore "8.17.1"
cspell-glob "8.17.1"
cspell-io "8.17.1"
cspell-lib "8.17.1"
fast-json-stable-stringify "^2.1.0"
file-entry-cache "^9.1.0"
get-stdin "^9.0.0"
semver "^7.6.3"
tinyglobby "^0.2.10"
env-paths@^3.0.0:
version "3.0.0"
resolved "https://registry.yarnpkg.com/env-paths/-/env-paths-3.0.0.tgz#2f1e89c2f6dbd3408e1b1711dd82d62e317f58da"
integrity sha512-dtJUTepzMW3Lm/NPxRf3wP4642UWhjL2sQxc+ym2YMj1m/H2zDNQOlezafzkHwn6sMstjHTwG6iQQsctDW/b1A==
esprima@^4.0.1:
version "4.0.1"
resolved "https://registry.yarnpkg.com/esprima/-/esprima-4.0.1.tgz#13b04cdb3e6c5d19df91ab6987a8695619b0aa71"
integrity sha512-eGuFFw7Upda+g4p+QHvnW0RyTX/SVeJBDM/gCtMARO0cLuT2HcEKnTPvhjV6aGeqrCB/sbNop0Kszm0jsaWU4A==
fast-equals@^5.0.1:
version "5.0.1"
resolved "https://registry.yarnpkg.com/fast-equals/-/fast-equals-5.0.1.tgz#a4eefe3c5d1c0d021aeed0bc10ba5e0c12ee405d"
integrity sha512-WF1Wi8PwwSY7/6Kx0vKXtw8RwuSGoM1bvDaJbu7MxDlR1vovZjIAKrnzyrThgAjm6JDTu0fVgWXDlMGspodfoQ==
fast-json-stable-stringify@^2.1.0:
version "2.1.0"
resolved "https://registry.yarnpkg.com/fast-json-stable-stringify/-/fast-json-stable-stringify-2.1.0.tgz#874bf69c6f404c2b5d99c481341399fd55892633"
integrity sha512-lhd/wF+Lk98HZoTCtlVraHtfh5XYijIjalXck7saUtuanSDyLMxnHhSXEDJqHxD7msR8D0uCmqlkwjCV8xvwHw==
fdir@^6.4.2:
version "6.4.2"
resolved "https://registry.yarnpkg.com/fdir/-/fdir-6.4.2.tgz#ddaa7ce1831b161bc3657bb99cb36e1622702689"
integrity sha512-KnhMXsKSPZlAhp7+IjUkRZKPb4fUyccpDrdFXbi4QL1qkmFh9kVY09Yox+n4MaOb3lHZ1Tv829C3oaaXoMYPDQ==
file-entry-cache@^9.1.0:
version "9.1.0"
resolved "https://registry.yarnpkg.com/file-entry-cache/-/file-entry-cache-9.1.0.tgz#2e66ad98ce93f49aed1b178c57b0b5741591e075"
integrity sha512-/pqPFG+FdxWQj+/WSuzXSDaNzxgTLr/OrR1QuqfEZzDakpdYE70PwUxL7BPUa8hpjbvY1+qvCl8k+8Tq34xJgg==
dependencies:
flat-cache "^5.0.0"
fill-range@^7.1.1:
version "7.1.1"
resolved "https://registry.yarnpkg.com/fill-range/-/fill-range-7.1.1.tgz#44265d3cac07e3ea7dc247516380643754a05292"
integrity sha512-YsGpe3WHLK8ZYi4tWDg2Jy3ebRz2rXowDxnld4bkQB00cc/1Zw9AWnC0i9ztDJitivtQvaI9KaLyKrc+hBW0yg==
dependencies:
to-regex-range "^5.0.1"
find-up-simple@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/find-up-simple/-/find-up-simple-1.0.0.tgz#21d035fde9fdbd56c8f4d2f63f32fd93a1cfc368"
integrity sha512-q7Us7kcjj2VMePAa02hDAF6d+MzsdsAWEwYyOpwUtlerRBkOEPBCRZrAV4XfcSN8fHAgaD0hP7miwoay6DCprw==
flat-cache@^5.0.0:
version "5.0.0"
resolved "https://registry.yarnpkg.com/flat-cache/-/flat-cache-5.0.0.tgz#26c4da7b0f288b408bb2b506b2cb66c240ddf062"
integrity sha512-JrqFmyUl2PnPi1OvLyTVHnQvwQ0S+e6lGSwu8OkAZlSaNIZciTY2H/cOOROxsBA1m/LZNHDsqAgDZt6akWcjsQ==
dependencies:
flatted "^3.3.1"
keyv "^4.5.4"
flatted@^3.3.1:
version "3.3.2"
resolved "https://registry.yarnpkg.com/flatted/-/flatted-3.3.2.tgz#adba1448a9841bec72b42c532ea23dbbedef1a27"
integrity sha512-AiwGJM8YcNOaobumgtng+6NHuOqC3A7MixFeDafM3X9cIUM+xUXoS5Vfgf+OihAYe20fxqNM9yPBXJzRtZ/4eA==
gensequence@^7.0.0:
version "7.0.0"
resolved "https://registry.yarnpkg.com/gensequence/-/gensequence-7.0.0.tgz#bb6aedec8ff665e3a6c42f92823121e3a6ea7718"
integrity sha512-47Frx13aZh01afHJTB3zTtKIlFI6vWY+MYCN9Qpew6i52rfKjnhCF/l1YlC8UmEMvvntZZ6z4PiCcmyuedR2aQ==
get-stdin@^9.0.0:
version "9.0.0"
resolved "https://registry.yarnpkg.com/get-stdin/-/get-stdin-9.0.0.tgz#3983ff82e03d56f1b2ea0d3e60325f39d703a575"
integrity sha512-dVKBjfWisLAicarI2Sf+JuBE/DghV4UzNAVe9yhEJuzeREd3JhOTE9cUaJTeSa77fsbQUK3pcOpJfM59+VKZaA==
global-directory@^4.0.1:
version "4.0.1"
resolved "https://registry.yarnpkg.com/global-directory/-/global-directory-4.0.1.tgz#4d7ac7cfd2cb73f304c53b8810891748df5e361e"
integrity sha512-wHTUcDUoZ1H5/0iVqEudYW4/kAlN5cZ3j/bXn0Dpbizl9iaUVeWSHqiOjsgk6OW2bkLclbBjzewBz6weQ1zA2Q==
dependencies:
ini "4.1.1"
has-own-prop@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/has-own-prop/-/has-own-prop-2.0.0.tgz#f0f95d58f65804f5d218db32563bb85b8e0417af"
integrity sha512-Pq0h+hvsVm6dDEa8x82GnLSYHOzNDt7f0ddFa3FqcQlgzEiptPqL+XrOJNavjOzSYiYWIrgeVYYgGlLmnxwilQ==
import-fresh@^3.3.0:
version "3.3.0"
resolved "https://registry.yarnpkg.com/import-fresh/-/import-fresh-3.3.0.tgz#37162c25fcb9ebaa2e6e53d5b4d88ce17d9e0c2b"
integrity sha512-veYYhQa+D1QBKznvhUHxb8faxlrwUnxseDAbAp457E0wLNio2bOSKnjYDhMj+YiAq61xrMGhQk9iXVk5FzgQMw==
dependencies:
parent-module "^1.0.0"
resolve-from "^4.0.0"
import-meta-resolve@^4.1.0:
version "4.1.0"
resolved "https://registry.yarnpkg.com/import-meta-resolve/-/import-meta-resolve-4.1.0.tgz#f9db8bead9fafa61adb811db77a2bf22c5399706"
integrity sha512-I6fiaX09Xivtk+THaMfAwnA3MVA5Big1WHF1Dfx9hFuvNIWpXnorlkzhcQf6ehrqQiiZECRt1poOAkPmer3ruw==
ini@4.1.1:
version "4.1.1"
resolved "https://registry.yarnpkg.com/ini/-/ini-4.1.1.tgz#d95b3d843b1e906e56d6747d5447904ff50ce7a1"
integrity sha512-QQnnxNyfvmHFIsj7gkPcYymR8Jdw/o7mp5ZFihxn6h8Ci6fh3Dx4E1gPjpQEpIuPo9XVNY/ZUwh4BPMjGyL01g==
is-number@^7.0.0:
version "7.0.0"
resolved "https://registry.yarnpkg.com/is-number/-/is-number-7.0.0.tgz#7535345b896734d5f80c4d06c50955527a14f12b"
integrity sha512-41Cifkg6e8TylSpdtTpeLVMqvSBEVzTttHvERD741+pnZ8ANv0004MRL43QKPDlK9cGvNp6NZWZUBlbGXYxxng==
json-buffer@3.0.1:
version "3.0.1"
resolved "https://registry.yarnpkg.com/json-buffer/-/json-buffer-3.0.1.tgz#9338802a30d3b6605fbe0613e094008ca8c05a13"
integrity sha512-4bV5BfR2mqfQTJm+V5tPPdf+ZpuhiIvTuAB5g8kcrXOZpTT/QwwVRWBywX1ozr6lEuPdbHxwaJlm9G6mI2sfSQ==
keyv@^4.5.4:
version "4.5.4"
resolved "https://registry.yarnpkg.com/keyv/-/keyv-4.5.4.tgz#a879a99e29452f942439f2a405e3af8b31d4de93"
integrity sha512-oxVHkHR/EJf2CNXnWxRLW6mg7JyCCUcG0DtEGmL2ctUo1PNTin1PUil+r/+4r5MpVgC/fn1kjsx7mjSujKqIpw==
dependencies:
json-buffer "3.0.1"
micromatch@^4.0.8:
version "4.0.8"
resolved "https://registry.yarnpkg.com/micromatch/-/micromatch-4.0.8.tgz#d66fa18f3a47076789320b9b1af32bd86d9fa202"
integrity sha512-PXwfBhYu0hBCPw8Dn0E+WDYb7af3dSLVWKi3HGv84IdF4TyFoC0ysxFd0Goxw7nSv4T/PzEJQxsYsEiFCKo2BA==
dependencies:
braces "^3.0.3"
picomatch "^2.3.1"
parent-module@^1.0.0:
version "1.0.1"
resolved "https://registry.yarnpkg.com/parent-module/-/parent-module-1.0.1.tgz#691d2709e78c79fae3a156622452d00762caaaa2"
integrity sha512-GQ2EWRpQV8/o+Aw8YqtfZZPfNRWZYkbidE9k5rpl/hC3vtHHBfGm2Ifi6qWV+coDGkrUKZAxE3Lot5kcsRlh+g==
dependencies:
callsites "^3.0.0"
parent-module@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/parent-module/-/parent-module-2.0.0.tgz#fa71f88ff1a50c27e15d8ff74e0e3a9523bf8708"
integrity sha512-uo0Z9JJeWzv8BG+tRcapBKNJ0dro9cLyczGzulS6EfeyAdeC9sbojtW6XwvYxJkEne9En+J2XEl4zyglVeIwFg==
dependencies:
callsites "^3.1.0"
picomatch@^2.3.1:
version "2.3.1"
resolved "https://registry.yarnpkg.com/picomatch/-/picomatch-2.3.1.tgz#3ba3833733646d9d3e4995946c1365a67fb07a42"
integrity sha512-JU3teHTNjmE2VCGFzuY8EXzCDVwEqB2a8fsIvwaStHhAWJEeVd1o1QD80CU6+ZdEXXSLbSsuLwJjkCBWqRQUVA==
picomatch@^4.0.2:
version "4.0.2"
resolved "https://registry.yarnpkg.com/picomatch/-/picomatch-4.0.2.tgz#77c742931e8f3b8820946c76cd0c1f13730d1dab"
integrity sha512-M7BAV6Rlcy5u+m6oPhAPFgJTzAioX/6B0DxyvDlo9l8+T3nLKbrczg2WLUyzd45L8RqfUMyGPzekbMvX2Ldkwg==
repeat-string@^1.6.1:
version "1.6.1"
resolved "https://registry.yarnpkg.com/repeat-string/-/repeat-string-1.6.1.tgz#8dcae470e1c88abc2d600fff4a776286da75e637"
integrity sha512-PV0dzCYDNfRi1jCDbJzpW7jNNDRuCOG/jI5ctQcGKt/clZD+YcPS3yIlWuTJMmESC8aevCFmWJy5wjAFgNqN6w==
resolve-from@^4.0.0:
version "4.0.0"
resolved "https://registry.yarnpkg.com/resolve-from/-/resolve-from-4.0.0.tgz#4abcd852ad32dd7baabfe9b40e00a36db5f392e6"
integrity sha512-pb/MYmXstAkysRFx8piNI1tGFNQIFA3vkE3Gq4EuA1dF6gHp/+vgZqsCGJapvy8N3Q+4o7FwvquPJcnZ7RYy4g==
resolve-from@^5.0.0:
version "5.0.0"
resolved "https://registry.yarnpkg.com/resolve-from/-/resolve-from-5.0.0.tgz#c35225843df8f776df21c57557bc087e9dfdfc69"
integrity sha512-qYg9KP24dD5qka9J47d0aVky0N+b4fTU89LN9iDnjB5waksiC49rvMB0PrUJQGoTmH50XPiqOvAjDfaijGxYZw==
semver@^7.6.3:
version "7.6.3"
resolved "https://registry.yarnpkg.com/semver/-/semver-7.6.3.tgz#980f7b5550bc175fb4dc09403085627f9eb33143"
integrity sha512-oVekP1cKtI+CTDvHWYFUcMtsK/00wmAEfyqKfNdARm8u1wNVhSgaX7A8d4UuIlUI5e84iEwOhs7ZPYRmzU9U6A==
tinyglobby@^0.2.10:
version "0.2.10"
resolved "https://registry.yarnpkg.com/tinyglobby/-/tinyglobby-0.2.10.tgz#e712cf2dc9b95a1f5c5bbd159720e15833977a0f"
integrity sha512-Zc+8eJlFMvgatPZTl6A9L/yht8QqdmUNtURHaKZLmKBE12hNPSrqNkUp2cs3M/UKmNVVAMFQYSjYIVHDjW5zew==
dependencies:
fdir "^6.4.2"
picomatch "^4.0.2"
to-regex-range@^5.0.1:
version "5.0.1"
resolved "https://registry.yarnpkg.com/to-regex-range/-/to-regex-range-5.0.1.tgz#1648c44aae7c8d988a326018ed72f5b4dd0392e4"
integrity sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ==
dependencies:
is-number "^7.0.0"
vscode-languageserver-textdocument@^1.0.12:
version "1.0.12"
resolved "https://registry.yarnpkg.com/vscode-languageserver-textdocument/-/vscode-languageserver-textdocument-1.0.12.tgz#457ee04271ab38998a093c68c2342f53f6e4a631"
integrity sha512-cxWNPesCnQCcMPeenjKKsOCKQZ/L6Tv19DTRIGuLWe32lyzWhihGVJ/rcckZXJxfdKCFvRLS3fpBIsV/ZGX4zA==
vscode-uri@^3.0.8:
version "3.0.8"
resolved "https://registry.yarnpkg.com/vscode-uri/-/vscode-uri-3.0.8.tgz#1770938d3e72588659a172d0fd4642780083ff9f"
integrity sha512-AyFQ0EVmsOZOlAnxoFOGOq1SQDWAB7C6aqMGS23svWAllfOaxbuFvcT8D1i8z3Gyn8fraVeZNNmN6e9bxxXkKw==
xdg-basedir@^5.1.0:
version "5.1.0"
resolved "https://registry.yarnpkg.com/xdg-basedir/-/xdg-basedir-5.1.0.tgz#1efba19425e73be1bc6f2a6ceb52a3d2c884c0c9"
integrity sha512-GCPAHLvrIH13+c0SuacwvRYj2SxJXQ4kaVTT5xgL3kPrz56XxkF21IGhjSE1+W0aw7gpBWRGXLCPnPby6lSpmQ==
yaml@^2.6.1:
version "2.6.1"
resolved "https://registry.yarnpkg.com/yaml/-/yaml-2.6.1.tgz#42f2b1ba89203f374609572d5349fb8686500773"
integrity sha512-7r0XPzioN/Q9kXBro/XPnA6kznR73DHq+GXh5ON7ZozRO6aMjbmiBuKste2wslTFkC5d1dw0GooOCepZXJ2SAg==

View File

@@ -578,7 +578,7 @@
"type": "victoriametrics-datasource",
"uid": "$ds"
},
"expr": "sum(vm_persistentqueue_bytes_pending{job=~\"$job\", instance=~\"$instance\"})",
"expr": "sum(vmagent_remotewrite_pending_data_bytes{job=~\"$job\", instance=~\"$instance\"})",
"interval": "",
"legendFormat": "",
"refId": "A"

View File

@@ -143,7 +143,7 @@
"type": "victoriametrics-datasource",
"uid": "$ds"
},
"description": "Shows if the last configuration update was successful. \"Not Ok\" means there was an unsuccessful attempt to update the configuration due to some error. Check the log for details.",
"description": "Shows if the last configuration update was successful. \"Not OK\" means there was an unsuccessful attempt to update the configuration due to some error. Check the log for details.",
"fieldConfig": {
"defaults": {
"mappings": [
@@ -152,7 +152,7 @@
"0": {
"color": "green",
"index": 0,
"text": "Ok"
"text": "OK"
}
},
"type": "value"
@@ -163,11 +163,21 @@
"result": {
"color": "red",
"index": 1,
"text": "Not Ok"
"text": "Not OK"
},
"to": 999999
},
"type": "range"
},
{
"options": {
"match": "null",
"result": {
"index": 2,
"text": "OK"
}
},
"type": "special"
}
],
"thresholds": {
@@ -214,7 +224,7 @@
"uid": "$ds"
},
"exemplar": false,
"expr": "count(vmalert_config_last_reload_successful{job=~\"$job\", instance=~\"$instance\"} < 1 ) or 0",
"expr": "count(vmalert_config_last_reload_successful{job=~\"$job\", instance=~\"$instance\"} < 1 )",
"interval": "",
"legendFormat": "",
"refId": "A"
@@ -359,6 +369,7 @@
"defaults": {
"mappings": [],
"min": 0,
"noValue": "0",
"thresholds": {
"mode": "absolute",
"steps": [
@@ -407,7 +418,7 @@
"uid": "$ds"
},
"exemplar": false,
"expr": "(sum(increase(vmalert_alerting_rules_errors_total{job=~\"$job\", instance=~\"$instance\", group=~\"$group\", file=~\"$file\"}[$__rate_interval])) or vector(0)) + \n(sum(increase(vmalert_recording_rules_errors_total{job=~\"$job\", instance=~\"$instance\", group=~\"$group\", file=~\"$file\"}[$__rate_interval])) or vector(0))",
"expr": "sum(increase(vmalert_alerting_rules_errors_total{job=~\"$job\", instance=~\"$instance\", group=~\"$group\", file=~\"$file\"}[$__rate_interval])) + \nsum(increase(vmalert_recording_rules_errors_total{job=~\"$job\", instance=~\"$instance\", group=~\"$group\", file=~\"$file\"}[$__rate_interval]))",
"interval": "",
"legendFormat": "",
"refId": "A"
@@ -426,6 +437,7 @@
"defaults": {
"mappings": [],
"min": 0,
"noValue": "0",
"thresholds": {
"mode": "absolute",
"steps": [
@@ -475,7 +487,7 @@
},
"editorMode": "code",
"exemplar": false,
"expr": "count(vmalert_recording_rules_last_evaluation_samples{job=~\"$job\", instance=~\"$instance\"} < 1) or 0",
"expr": "count(vmalert_recording_rules_last_evaluation_samples{job=~\"$job\", instance=~\"$instance\"} < 1)",
"interval": "",
"legendFormat": "",
"range": true,

View File

@@ -225,7 +225,7 @@
"type": "victoriametrics-datasource",
"uid": "$ds"
},
"description": "Shows if the last configuration update was successful. \"Not Ok\" means there was an unsuccessful attempt to update the configuration due to some error. Check the log for details.",
"description": "Shows if the last configuration update was successful. \"Not OK\" means there was an unsuccessful attempt to update the configuration due to some error. Check the log for details.",
"fieldConfig": {
"defaults": {
"mappings": [
@@ -234,7 +234,7 @@
"0": {
"color": "green",
"index": 0,
"text": "Ok"
"text": "OK"
}
},
"type": "value"
@@ -245,11 +245,21 @@
"result": {
"color": "red",
"index": 1,
"text": "Not Ok"
"text": "Not OK"
},
"to": 999999
},
"type": "range"
},
{
"options": {
"match": "null",
"result": {
"index": 2,
"text": "OK"
}
},
"type": "special"
}
],
"thresholds": {
@@ -296,7 +306,7 @@
"uid": "$ds"
},
"exemplar": false,
"expr": "count(vmauth_config_last_reload_successful{job=~\"$job\", instance=~\"$instance\"} < 1 ) or 0",
"expr": "count(vmauth_config_last_reload_successful{job=~\"$job\", instance=~\"$instance\"} < 1 )",
"interval": "",
"legendFormat": "",
"refId": "A"

View File

@@ -577,7 +577,7 @@
"type": "prometheus",
"uid": "$ds"
},
"expr": "sum(vm_persistentqueue_bytes_pending{job=~\"$job\", instance=~\"$instance\"})",
"expr": "sum(vmagent_remotewrite_pending_data_bytes{job=~\"$job\", instance=~\"$instance\"})",
"interval": "",
"legendFormat": "",
"refId": "A"

View File

@@ -142,7 +142,7 @@
"type": "prometheus",
"uid": "$ds"
},
"description": "Shows if the last configuration update was successful. \"Not Ok\" means there was an unsuccessful attempt to update the configuration due to some error. Check the log for details.",
"description": "Shows if the last configuration update was successful. \"Not OK\" means there was an unsuccessful attempt to update the configuration due to some error. Check the log for details.",
"fieldConfig": {
"defaults": {
"mappings": [
@@ -151,7 +151,7 @@
"0": {
"color": "green",
"index": 0,
"text": "Ok"
"text": "OK"
}
},
"type": "value"
@@ -162,11 +162,21 @@
"result": {
"color": "red",
"index": 1,
"text": "Not Ok"
"text": "Not OK"
},
"to": 999999
},
"type": "range"
},
{
"options": {
"match": "null",
"result": {
"index": 2,
"text": "OK"
}
},
"type": "special"
}
],
"thresholds": {
@@ -213,7 +223,7 @@
"uid": "$ds"
},
"exemplar": false,
"expr": "count(vmalert_config_last_reload_successful{job=~\"$job\", instance=~\"$instance\"} < 1 ) or 0",
"expr": "count(vmalert_config_last_reload_successful{job=~\"$job\", instance=~\"$instance\"} < 1 )",
"interval": "",
"legendFormat": "",
"refId": "A"
@@ -358,6 +368,7 @@
"defaults": {
"mappings": [],
"min": 0,
"noValue": "0",
"thresholds": {
"mode": "absolute",
"steps": [
@@ -406,7 +417,7 @@
"uid": "$ds"
},
"exemplar": false,
"expr": "(sum(increase(vmalert_alerting_rules_errors_total{job=~\"$job\", instance=~\"$instance\", group=~\"$group\", file=~\"$file\"}[$__rate_interval])) or vector(0)) + \n(sum(increase(vmalert_recording_rules_errors_total{job=~\"$job\", instance=~\"$instance\", group=~\"$group\", file=~\"$file\"}[$__rate_interval])) or vector(0))",
"expr": "sum(increase(vmalert_alerting_rules_errors_total{job=~\"$job\", instance=~\"$instance\", group=~\"$group\", file=~\"$file\"}[$__rate_interval])) + \nsum(increase(vmalert_recording_rules_errors_total{job=~\"$job\", instance=~\"$instance\", group=~\"$group\", file=~\"$file\"}[$__rate_interval]))",
"interval": "",
"legendFormat": "",
"refId": "A"
@@ -425,6 +436,7 @@
"defaults": {
"mappings": [],
"min": 0,
"noValue": "0",
"thresholds": {
"mode": "absolute",
"steps": [
@@ -474,7 +486,7 @@
},
"editorMode": "code",
"exemplar": false,
"expr": "count(vmalert_recording_rules_last_evaluation_samples{job=~\"$job\", instance=~\"$instance\"} < 1) or 0",
"expr": "count(vmalert_recording_rules_last_evaluation_samples{job=~\"$job\", instance=~\"$instance\"} < 1)",
"interval": "",
"legendFormat": "",
"range": true,

View File

@@ -224,7 +224,7 @@
"type": "prometheus",
"uid": "$ds"
},
"description": "Shows if the last configuration update was successful. \"Not Ok\" means there was an unsuccessful attempt to update the configuration due to some error. Check the log for details.",
"description": "Shows if the last configuration update was successful. \"Not OK\" means there was an unsuccessful attempt to update the configuration due to some error. Check the log for details.",
"fieldConfig": {
"defaults": {
"mappings": [
@@ -233,7 +233,7 @@
"0": {
"color": "green",
"index": 0,
"text": "Ok"
"text": "OK"
}
},
"type": "value"
@@ -244,11 +244,21 @@
"result": {
"color": "red",
"index": 1,
"text": "Not Ok"
"text": "Not OK"
},
"to": 999999
},
"type": "range"
},
{
"options": {
"match": "null",
"result": {
"index": 2,
"text": "OK"
}
},
"type": "special"
}
],
"thresholds": {
@@ -295,7 +305,7 @@
"uid": "$ds"
},
"exemplar": false,
"expr": "count(vmauth_config_last_reload_successful{job=~\"$job\", instance=~\"$instance\"} < 1 ) or 0",
"expr": "count(vmauth_config_last_reload_successful{job=~\"$job\", instance=~\"$instance\"} < 1 )",
"interval": "",
"legendFormat": "",
"refId": "A"

View File

@@ -66,6 +66,12 @@ package-via-docker: package-base
$(DOCKER_BUILD) \
--build-arg src_binary=$(APP_NAME)$(APP_SUFFIX)-prod \
--build-arg base_image=$(BASE_IMAGE) \
--label "org.opencontainers.image.source=https://github.com/VictoriaMetrics/VictoriaMetrics" \
--label "org.opencontainers.image.documentation=https://docs.victoriametrics.com/" \
--label "org.opencontainers.image.title=$(APP_NAME)" \
--label "org.opencontainers.image.vendor=VictoriaMetrics" \
--label "org.opencontainers.image.version=$(PKG_TAG)" \
--label "org.opencontainers.image.created=$(shell date -u +"%Y-%m-%dT%H:%M:%SZ")" \
--tag $(DOCKER_NAMESPACE)/$(APP_NAME):$(PKG_TAG)$(APP_SUFFIX)$(RACE) \
-f app/$(APP_NAME)/deployment/Dockerfile bin)
@@ -80,6 +86,12 @@ publish-via-docker:
--build-arg certs_image=$(CERTS_IMAGE) \
--build-arg root_image=$(ROOT_IMAGE) \
--build-arg APP_NAME=$(APP_NAME) \
--label "org.opencontainers.image.source=https://github.com/VictoriaMetrics/VictoriaMetrics" \
--label "org.opencontainers.image.documentation=https://docs.victoriametrics.com/" \
--label "org.opencontainers.image.title=$(APP_NAME)" \
--label "org.opencontainers.image.vendor=VictoriaMetrics" \
--label "org.opencontainers.image.version=$(PKG_TAG)" \
--label "org.opencontainers.image.created=$(shell date -u +"%Y-%m-%dT%H:%M:%SZ")" \
--tag $(DOCKER_NAMESPACE)/$(APP_NAME):$(PKG_TAG)$(RACE) \
--tag $(DOCKER_NAMESPACE)/$(APP_NAME):$(LATEST_TAG)$(RACE) \
-o type=image \
@@ -92,6 +104,12 @@ publish-via-docker:
--build-arg certs_image=$(CERTS_IMAGE) \
--build-arg root_image=$(ROOT_IMAGE_SCRATCH) \
--build-arg APP_NAME=$(APP_NAME) \
--label "org.opencontainers.image.source=https://github.com/VictoriaMetrics/VictoriaMetrics" \
--label "org.opencontainers.image.documentation=https://docs.victoriametrics.com/" \
--label "org.opencontainers.image.title=$(APP_NAME)" \
--label "org.opencontainers.image.vendor=VictoriaMetrics" \
--label "org.opencontainers.image.version=$(PKG_TAG)" \
--label "org.opencontainers.image.created=$(shell date -u +"%Y-%m-%dT%H:%M:%SZ")" \
--tag $(DOCKER_NAMESPACE)/$(APP_NAME):$(PKG_TAG)$(RACE)-scratch \
--tag $(DOCKER_NAMESPACE)/$(APP_NAME):$(LATEST_TAG)$(RACE)-scratch \
-o type=image \

View File

@@ -210,3 +210,4 @@ Please see more examples on integration of VictoriaLogs with other log shippers
* [opentelemetry-collector](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker/victorialogs/opentelemetry-collector)
* [telegraf](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker/victorialogs/telegraf)
* [fluentd]((https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker/victorialogs/fluentd)
* [datadog-serverless](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker/victorialogs/datadog-serverless)

View File

@@ -4,7 +4,7 @@ services:
# And forward them to --remoteWrite.url
vmagent:
container_name: vmagent
image: victoriametrics/vmagent:v1.108.0
image: victoriametrics/vmagent:v1.108.1
depends_on:
- "vminsert"
ports:
@@ -39,7 +39,7 @@ services:
# where N is number of vmstorages (2 in this case).
vmstorage-1:
container_name: vmstorage-1
image: victoriametrics/vmstorage:v1.108.0-cluster
image: victoriametrics/vmstorage:v1.108.1-cluster
ports:
- 8482
- 8400
@@ -51,7 +51,7 @@ services:
restart: always
vmstorage-2:
container_name: vmstorage-2
image: victoriametrics/vmstorage:v1.108.0-cluster
image: victoriametrics/vmstorage:v1.108.1-cluster
ports:
- 8482
- 8400
@@ -66,7 +66,7 @@ services:
# pre-process them and distributes across configured vmstorage shards.
vminsert:
container_name: vminsert
image: victoriametrics/vminsert:v1.108.0-cluster
image: victoriametrics/vminsert:v1.108.1-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -81,7 +81,7 @@ services:
# vmselect collects results from configured `--storageNode` shards.
vmselect-1:
container_name: vmselect-1
image: victoriametrics/vmselect:v1.108.0-cluster
image: victoriametrics/vmselect:v1.108.1-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -94,7 +94,7 @@ services:
restart: always
vmselect-2:
container_name: vmselect-2
image: victoriametrics/vmselect:v1.108.0-cluster
image: victoriametrics/vmselect:v1.108.1-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -112,7 +112,7 @@ services:
# It can be used as an authentication proxy.
vmauth:
container_name: vmauth
image: victoriametrics/vmauth:v1.108.0
image: victoriametrics/vmauth:v1.108.1
depends_on:
- "vmselect-1"
- "vmselect-2"
@@ -127,7 +127,7 @@ services:
# vmalert executes alerting and recording rules
vmalert:
container_name: vmalert
image: victoriametrics/vmalert:v1.108.0
image: victoriametrics/vmalert:v1.108.1
depends_on:
- "vmauth"
ports:

View File

@@ -45,7 +45,7 @@ services:
# storing logs and serving read queries.
victorialogs:
container_name: victorialogs
image: victoriametrics/victoria-logs:v1.3.2-victorialogs
image: victoriametrics/victoria-logs:v1.4.0-victorialogs
command:
- "--storageDataPath=/vlogs"
- "--httpListenAddr=:9428"
@@ -60,7 +60,7 @@ services:
# scraping, storing metrics and serve read requests.
victoriametrics:
container_name: victoriametrics
image: victoriametrics/victoria-metrics:v1.108.0
image: victoriametrics/victoria-metrics:v1.108.1
ports:
- 8428:8428
volumes:
@@ -79,7 +79,7 @@ services:
# depending on the requested path.
vmauth:
container_name: vmauth
image: victoriametrics/vmauth:v1.108.0
image: victoriametrics/vmauth:v1.108.1
depends_on:
- "victoriametrics"
- "victorialogs"
@@ -96,7 +96,7 @@ services:
# vmalert executes alerting and recording rules according to given rule type.
vmalert:
container_name: vmalert
image: victoriametrics/vmalert:v1.108.0
image: victoriametrics/vmalert:v1.108.1
depends_on:
- "vmauth"
- "alertmanager"

View File

@@ -4,7 +4,7 @@ services:
# And forward them to --remoteWrite.url
vmagent:
container_name: vmagent
image: victoriametrics/vmagent:v1.108.0
image: victoriametrics/vmagent:v1.108.1
depends_on:
- "victoriametrics"
ports:
@@ -22,7 +22,7 @@ services:
# storing metrics and serve read requests.
victoriametrics:
container_name: victoriametrics
image: victoriametrics/victoria-metrics:v1.108.0
image: victoriametrics/victoria-metrics:v1.108.1
ports:
- 8428:8428
- 8089:8089
@@ -65,7 +65,7 @@ services:
# vmalert executes alerting and recording rules
vmalert:
container_name: vmalert
image: victoriametrics/vmalert:v1.108.0
image: victoriametrics/vmalert:v1.108.1
depends_on:
- "victoriametrics"
- "alertmanager"

View File

@@ -1,11 +1,12 @@
services:
# meta service will be ignored by compose
.victorialogs:
image: docker.io/victoriametrics/victoria-logs:v1.3.2-victorialogs
image: docker.io/victoriametrics/victoria-logs:v1.4.0-victorialogs
command:
- -storageDataPath=/vlogs
- -loggerFormat=json
- -syslog.listenAddr.tcp=0.0.0.0:8094
- -datadog.streamFields=service,hostname,ddsource
- -journald.streamFields=_HOSTNAME,_SYSTEMD_UNIT,_PID
- -journald.ignoreFields=MESSAGE_ID,INVOCATION_ID,USER_INVOCATION_ID,
- -journald.ignoreFields=_BOOT_ID,_MACHINE_ID,_SYSTEMD_INVOCATION_ID,_STREAM_ID,_UID
@@ -17,8 +18,8 @@ services:
timeout: 1s
retries: 10
dd-logs:
image: docker.io/victoriametrics/vmauth:v1.108.0
dd-proxy:
image: docker.io/victoriametrics/vmauth:v1.108.1
restart: on-failure
volumes:
- ./:/etc/vmauth

View File

@@ -11,8 +11,8 @@ services:
- /sys/fs/cgroup/:/host/sys/fs/cgroup:ro
environment:
DD_API_KEY: test
DD_URL: http://victoriametrics:8428/datadog
DD_LOGS_CONFIG_LOGS_DD_URL: http://dd-logs:8427
DD_URL: http://dd-proxy:8427
DD_LOGS_CONFIG_LOGS_DD_URL: http://dd-proxy:8427
DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL: true
DD_LOGS_ENABLED: true
DD_LOGS_CONFIG_USE_HTTP: true

View File

@@ -0,0 +1 @@
**/logs

View File

@@ -0,0 +1,31 @@
# Docker compose Serverless with DataDog extension integration with VictoriaLogs
The folder contains examples of [DataDog serverless](https://docs.datadoghq.com/serverless) integration with VictoriaLogs for:
* [AWS Lambda](./aws)
* [GCP Cloud Run](./gcp)
To spin-up environment `cd` to any of listed above directories run the following command:
```
docker compose up -d
```
To shut down the docker-compose environment run the following command:
```
docker compose down
docker compose rm -f
```
The docker compose file contains the following components:
* dd-proxy - VMAuth proxy, with path-based routing to `victoriametrics` and `victorialogs`
* lambda - Serverless application with Datadog logs collection extension, which is configured to collect and write data to `victorialogs` and `victoriametrics` via `dd-proxy`
* victorialogs - VictoriaLogs log database, which accepts the data from `datadog`
* victoriametrics - VictoriaMetrics metrics database, which collects metrics from `victorialogs` and `datadog`
Querying the data
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.

View File

@@ -0,0 +1,32 @@
FROM golang:1.23-bullseye as aws-lambda-rie
# Install custom aws-lambda-rie till Telemetry API support is not merged
# https://github.com/aws/aws-lambda-runtime-interface-emulator/pull/137
RUN \
git clone https://github.com/VictoriaMetrics/aws-lambda-runtime-interface-emulator -b added-telemetry-api-support /tmp/aws-lambda-rie && \
cd /tmp/aws-lambda-rie && \
CGO_ENABLED=0 go build -buildvcs=false -ldflags "-s -w" -o /aws-lambda-rie ./cmd/aws-lambda-rie
FROM python:3.12-bullseye
RUN \
apt update && \
apt install -y \
curl \
g++ \
make \
cmake \
unzip \
libcurl4-openssl-dev && \
mkdir -p /var/task && \
pip install \
--target /var/task awslambdaric datadog-lambda
WORKDIR /var/task
COPY --from=aws-lambda-rie /aws-lambda-rie /var/task/aws-lambda-rie
COPY main.py /var/task/
COPY --from=public.ecr.aws/datadog/lambda-extension:67 /opt/. /opt/
ENTRYPOINT ["/var/task/aws-lambda-rie"]
CMD ["/usr/local/bin/python", "-m", "awslambdaric", "main.lambda_handler"]

View File

@@ -0,0 +1,25 @@
name: datadog-serverless-aws
include:
- ../../compose-base.yml
services:
lambda:
build: .
restart: on-failure
ports:
- 8080:8080
environment:
DD_LOG_LEVEL: trace
DD_LOGS_ENABLED: true
DD_SOURCE: test
DD_API_KEY: test
DD_DD_URL: http://dd-proxy:8427
DD_EXTENSION_VERSION: compatibility
DD_PROFILING_ENABLED: false
DD_ENHANCED_METRICS: false
DD_LOGS_CONFIG_LOGS_DD_URL: http://dd-proxy:8427
DD_SERVERLESS_FLUSH_STRATEGY: periodically,100
depends_on:
victorialogs:
condition: service_healthy
victoriametrics:
condition: service_healthy

View File

@@ -0,0 +1,15 @@
from datadog_lambda.metric import lambda_metric
def lambda_handler(event, context):
lambda_metric(
metric_name='coffee_house.order_value',
value=12.45,
tags=['product:latte', 'order:online']
)
print('Hello, world!')
return {
'statusCode': 200,
'body': 'Hello from serverless!'
}

View File

@@ -0,0 +1,14 @@
FROM python:3.12-bullseye
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
ENV DD_SERVICE=datadog-demo-run-go
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
RUN pip install Flask gunicorn datadog
WORKDIR /var/task
COPY main.py /var/task/
ENTRYPOINT ["/app/datadog-init"]
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 main:app

View File

@@ -0,0 +1,24 @@
name: datadog-serverless-gcp
include:
- ../../compose-base.yml
services:
cloud-run:
build: .
restart: on-failure
ports:
- 8080:8080
environment:
PORT: 8080
DD_LOG_LEVEL: trace
DD_LOGS_ENABLED: true
DD_SOURCE: test
DD_API_KEY: test
DD_DD_URL: http://dd-proxy:8427
DD_PROFILING_ENABLED: false
DD_ENHANCED_METRICS: false
DD_LOGS_CONFIG_LOGS_DD_URL: http://dd-proxy:8427
depends_on:
victorialogs:
condition: service_healthy
victoriametrics:
condition: service_healthy

View File

@@ -0,0 +1,23 @@
from datadog import initialize, statsd
import os
from flask import Flask
options = {
"statsd_host": "127.0.0.1",
"statsd_port": 8125,
}
initialize(**options)
app = Flask(__name__)
@app.route('/')
def hello_world():
statsd.gauge('active.connections', 1001, tags=["protocol:http"])
target = os.environ.get('TARGET', 'World')
return 'Hello {}!\n'.format(target)
if __name__ == "__main__":
app.run(debug=True,host='0.0.0.0',port=int(os.environ.get('PORT', 8080)))

View File

@@ -4,3 +4,12 @@ unauthorized_user:
- "/api/v2/logs"
- "/api/v1/validate"
url_prefix: "http://victorialogs:9428/insert/datadog/"
- src_paths:
- "/api/v1/series"
- "/api/v2/series"
- "/api/beta/sketches"
- "/api/v1/validate"
- "/api/v1/check_run"
- "/intake"
- "/api/v1/metadata"
url_prefix: "http://victoriametrics:8428/datadog/"

View File

@@ -1,7 +1,7 @@
services:
vmagent:
container_name: vmagent
image: victoriametrics/vmagent:v1.108.0
image: victoriametrics/vmagent:v1.108.1
depends_on:
- "victoriametrics"
ports:
@@ -18,7 +18,7 @@ services:
victoriametrics:
container_name: victoriametrics
image: victoriametrics/victoria-metrics:v1.108.0
image: victoriametrics/victoria-metrics:v1.108.1
ports:
- 8428:8428
volumes:
@@ -50,7 +50,7 @@ services:
vmalert:
container_name: vmalert
image: victoriametrics/vmalert:v1.108.0
image: victoriametrics/vmalert:v1.108.1
depends_on:
- "victoriametrics"
ports:

View File

@@ -3,7 +3,7 @@ version: "3"
services:
# Run `make package-victoria-logs` to build victoria-logs image
vlogs:
image: docker.io/victoriametrics/victoria-logs:v1.3.2-victorialogs
image: docker.io/victoriametrics/victoria-logs:v1.4.0-victorialogs
volumes:
- vlogs:/vlogs
ports:
@@ -46,7 +46,7 @@ services:
- "--config=/config.yml"
vmsingle:
image: victoriametrics/victoria-metrics:v1.108.0
image: victoriametrics/victoria-metrics:v1.108.1
ports:
- "8428:8428"
command:

View File

@@ -41,7 +41,7 @@ We use [labels](https://docs.github.com/en/issues/using-labels-and-milestones-to
to categorize GitHub issues. We have the following labels:
1. A component label: vmalert, vmagent, etc. Add this label to the issue if it is related to a specific component.
1. An issue type: `bug`, `enhancement`, `question`.
1. `enterprize`, assigned to issues related to ENT features
1. `enterprise`, assigned to issues related to ENT features
1. `need more info`, assigned to issues which require elaboration from the issue creator.
For example, if we weren't able to reproduce the reported bug based on the ticket description then we ask additional
questions which could help to reproduce the issue and add `need more info` label. This label helps other maintainers

View File

@@ -720,8 +720,8 @@ Some workloads may need fine-grained resource usage limits. In these cases the f
This means that the maximum memory usage and CPU usage a single query can use at `vmstorage` is proportional to `-search.maxUniqueTimeseries`.
By default, `vmstorage` calculates this limit automatically based on the available memory and the maximum number of concurrent read requests (see `-search.maxConcurrentRequests`).
The calculated limit will be printed during process start-up logs and exposed as `vm_search_max_unique_timeseries` metric.
- `-search.maxUniqueTimeseries` at `vmselect` adjusts the limit with the same name at `vmstorage`. The vmstorage limit can be adjusted
only to **lower value** and can't exceed it. By default, vmselect doesn't apply limit adjustments.
- `-search.maxUniqueTimeseries` at `vmselect` adjusts the limit with the same name at `vmstorage`. The limit cannot exceed the
value set in vmstorage if the `-search.maxUniqueTimeseries` flag is explicitly defined there. By default, vmselect doesn't apply limit adjustments.
- `-search.maxQueryDuration` at `vmselect` limits the duration of a single query. If the query takes longer than the given duration, then it is canceled.
This allows saving CPU and RAM at `vmselect` and `vmstorage` when executing unexpectedly heavy queries.
The limit can be overridden to a smaller value by passing `timeout` GET parameter.
@@ -1576,6 +1576,9 @@ Below is the output for `/path/to/vmselect -help`:
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
-search.logSlowQueryDuration duration
Log queries with execution time exceeding this value. Zero disables slow query logging. See also -search.logQueryMemoryUsage (default 5s)
-search.maxBinaryOpPushdownLabelValues int
The maximum number of values for a label in the first expression that can be extracted as a common label filter and pushed down to the second expression in a binary operation.
A larger value makes the pushed-down filter more complex but fewer time series will be returned. This flag is useful when selective label contains numerous values, for example `instance`, and storage resources are abundant. (default 100)
-search.maxConcurrentRequests int
The maximum number of concurrent search requests. It shouldn't be high, since a single request can saturate all the CPU cores, while many concurrently executed requests may require high amounts of memory. See also -search.maxQueueDuration and -search.maxMemoryPerQuery (default 16)
-search.maxDeleteSeries int
@@ -1633,7 +1636,7 @@ Below is the output for `/path/to/vmselect -help`:
-search.maxTagValueSuffixesPerSearch int
The maximum number of tag value suffixes returned from /metrics/find (default 100000)
-search.maxUniqueTimeseries int
The maximum number of unique time series, which can be selected during /api/v1/query and /api/v1/query_range queries. This option allows limiting memory usage. The limit can't exceed the corresponding -search.maxUniqueTimeseries limit on vmstorage, it can be only set to lower values. (default 0)
The maximum number of unique time series, which can be selected during /api/v1/query and /api/v1/query_range queries. This option allows limiting memory usage. The limit can't exceed the corresponding value set in vmstorage if the `-search.maxUniqueTimeseries` flag is explicitly defined there. (default 0)
-search.maxWorkersPerQuery int
The maximum number of CPU cores a single query can use. The default value should work good for most cases. The flag can be set to lower values for improving performance of big number of concurrently executed queries. The flag can be set to bigger values for improving performance of heavy queries, which scan big number of time series (>10K) and/or big number of samples (>100M). There is no sense in setting this flag to values bigger than the number of CPU cores available on the system (default 16)
-search.minStalenessInterval duration

View File

@@ -1,10 +1,12 @@
[![Latest Release](https://img.shields.io/github/release/VictoriaMetrics/VictoriaMetrics.svg?style=flat-square)](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest)
[![Docker Pulls](https://img.shields.io/docker/pulls/victoriametrics/victoria-metrics.svg?maxAge=604800)](https://hub.docker.com/r/victoriametrics/victoria-metrics)
[![Slack](https://img.shields.io/badge/join%20slack-%23victoriametrics-brightgreen.svg)](https://slack.victoriametrics.com/)
[![GitHub license](https://img.shields.io/github/license/VictoriaMetrics/VictoriaMetrics.svg)](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/LICENSE)
[![Go Report](https://goreportcard.com/badge/github.com/VictoriaMetrics/VictoriaMetrics)](https://goreportcard.com/report/github.com/VictoriaMetrics/VictoriaMetrics)
[![Build Status](https://github.com/VictoriaMetrics/VictoriaMetrics/workflows/main/badge.svg)](https://github.com/VictoriaMetrics/VictoriaMetrics/actions)
[![codecov](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/branch/master/graph/badge.svg)](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics)
![Latest Release](https://img.shields.io/github/v/release/VictoriaMetrics/VictoriaMetrics?sort=semver&label=&filter=!*-victorialogs&logo=github&labelColor=gray&color=gray&link=https%3A%2F%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics%2Freleases%2Flatest)
![Docker Pulls](https://img.shields.io/docker/pulls/victoriametrics/victoria-metrics?label=&logo=docker&logoColor=white&labelColor=2496ED&color=2496ED&link=https%3A%2F%2Fhub.docker.com%2Fr%2Fvictoriametrics%2Fvictoria-metrics)
![Go Report](https://goreportcard.com/badge/github.com/VictoriaMetrics/VictoriaMetrics?link=https%3A%2F%2Fgoreportcard.com%2Freport%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics)
![Build Status](https://github.com/VictoriaMetrics/VictoriaMetrics/workflows/main/badge.svg?link=https%3A%2F%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics%2Factions)
![codecov](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/branch/master/graph/badge.svg?link=https%3A%2F%2Fcodecov.io%2Fgh%2FVictoriaMetrics%2FVictoriaMetrics)
![License](https://img.shields.io/github/license/VictoriaMetrics/VictoriaMetrics?labelColor=green&label=&link=https%3A%2F%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics%2Fblob%2Fmaster%2FLICENSE)
![Slack](https://img.shields.io/badge/Join-4A154B?logo=slack&link=https%3A%2F%2Fslack.victoriametrics.com)
![X](https://img.shields.io/twitter/follow/VictoriaMetrics?style=flat&label=Follow&color=black&logo=x&labelColor=black&link=https%3A%2F%2Fx.com%2FVictoriaMetrics)
![Reddit](https://img.shields.io/reddit/subreddit-subscribers/VictoriaMetrics?style=flat&label=Join&labelColor=red&logoColor=white&logo=reddit&link=https%3A%2F%2Fwww.reddit.com%2Fr%2FVictoriaMetrics)
VictoriaMetrics is a fast, cost-effective and scalable monitoring solution and time series database.
See [case studies for VictoriaMetrics](https://docs.victoriametrics.com/casestudies/).
@@ -226,27 +228,35 @@ and then install it as a service according to the following guide:
See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3781) for more details.
## Playgrounds
VictoriaMetrics has the following publicly available resources:
1. [https://play.victoriametrics.com/](https://play.victoriametrics.com/) - [VMUI](#vmui) of VictoriaMetrics cluster installation.
It is available for testing the query engine, relabeling playground, other tools and pages provided by VMUI.
1. [https://play-grafana.victoriametrics.com/](https://play-grafana.victoriametrics.com/) - Grafana configured with many
typical dashboards using VictoriaMetrics and VictoriaLogs as datasource. It contains VictoriaMetrics cluster dashboard with
3 cluster installations for the recent OS and LTS versions running under the constant becnhmark.
1. [https://play-vmlogs.victoriametrics.com/](https://play-vmlogs.victoriametrics.com/) - [VMUI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui) of VictoriaLogs installation.
It is available for testing the query engine on demo logs set.
Additionally, we provide a [docker-compose environment](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker#docker-compose-environment-for-victoriametrics)
for VictoriaMetrics and VictoriaLogs components. They are already configured, provisioned and interconnected.
It can be used as an example for a [quick start](https://docs.victoriametrics.com/quick-start/).
## Prometheus setup
Add the following lines to Prometheus config file (it is usually located at `/etc/prometheus/prometheus.yml`) in order to send data to VictoriaMetrics:
```yaml
remote_write:
- url: http://<victoriametrics-addr>:8428/api/v1/write
```
Substitute `<victoriametrics-addr>` with hostname or IP address of VictoriaMetrics.
Then apply new config via the following command:
```sh
kill -HUP `pidof prometheus`
```
Prometheus writes incoming data to local storage and replicates it to remote storage in parallel.
This means that data remains available in local storage for `--storage.tsdb.retention.time` duration
even if remote storage is unavailable.
@@ -265,8 +275,6 @@ The label name can be arbitrary - `datacenter` is just an example. The label val
across Prometheus instances, so time series could be filtered and grouped by this label.
For highly loaded Prometheus instances (200k+ samples per second) the following tuning may be applied:
```yaml
remote_write:
- url: http://<victoriametrics-addr>:8428/api/v1/write
@@ -276,7 +284,6 @@ remote_write:
max_shards: 30
```
Using remote write increases memory usage for Prometheus by up to ~25%. If you are experiencing issues with
too high memory consumption of Prometheus, then try to lower `max_samples_per_send` and `capacity` params.
Keep in mind that these two params are tightly connected.
@@ -293,7 +300,6 @@ which can be used as faster and less resource-hungry alternative to Prometheus.
Create [Prometheus datasource](https://grafana.com/docs/grafana/latest/datasources/prometheus/configure-prometheus-data-source/)
in Grafana with the following url:
```url
http://<victoriametrics-addr>:8428
```
@@ -393,7 +399,7 @@ can be loaded in VMUI via `Query Analyzer` tool.
See the [example VMUI at VictoriaMetrics playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/?g0.expr=100%20*%20sum(rate(process_cpu_seconds_total))%20by%20(job)&g0.range_input=1d).
## Top queries
### Top queries
[VMUI](#vmui) provides `top queries` tab, which can help determining the following query types:
@@ -403,7 +409,7 @@ See the [example VMUI at VictoriaMetrics playground](https://play.victoriametric
This information is obtained from the `/api/v1/status/top_queries` HTTP endpoint.
## Active queries
### Active queries
[VMUI](#vmui) provides `active queries` tab, which shows currently execute queries.
It provides the following information per each query:
@@ -414,7 +420,7 @@ It provides the following information per each query:
This information is obtained from the `/api/v1/status/active_queries` HTTP endpoint.
## Metrics explorer
### Metrics explorer
[VMUI](#vmui) provides an ability to explore metrics exported by a particular `job` / `instance` in the following way:
@@ -426,7 +432,7 @@ This information is obtained from the `/api/v1/status/active_queries` HTTP endpo
It is possible to change the selected time range for the graphs in the top right corner.
## Cardinality explorer
### Cardinality explorer
VictoriaMetrics provides an ability to explore time series cardinality at `Explore cardinality` tab in [vmui](#vmui) in the following ways:
@@ -448,7 +454,7 @@ Cardinality explorer is built on top of [/api/v1/status/tsdb](#tsdb-stats).
See [cardinality explorer playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/cardinality).
See the example of using the cardinality explorer [here](https://victoriametrics.com/blog/cardinality-explorer/).
## Cardinality explorer statistic inaccuracy
### Cardinality explorer statistic inaccuracy
In [cluster version of VictoriaMetrics](https://docs.victoriametrics.com/cluster-victoriametrics/) each vmstorage tracks the stored time series individually.
vmselect requests stats via [/api/v1/status/tsdb](#tsdb-stats) API from each vmstorage node and merges the results by summing per-series stats.
@@ -1715,6 +1721,7 @@ See also [resource usage limits docs](#resource-usage-limits).
By default, VictoriaMetrics is tuned for an optimal resource usage under typical workloads. Some workloads may need fine-grained resource usage limits. In these cases the following command-line flags may be useful:
- `-maxIngestionRate` limits samples/second ingested. This may be useful when CPU resources are limited or overloaded.
- `-memory.allowedPercent` and `-memory.allowedBytes` limit the amounts of memory, which may be used for various internal caches at VictoriaMetrics.
Note that VictoriaMetrics may use more memory, since these flags don't limit additional memory, which may be needed on a per-query basis.
- `-search.maxMemoryPerQuery` limits the amounts of memory, which can be used for processing a single query. Queries, which need more memory, are rejected.
@@ -1793,7 +1800,7 @@ By default, VictoriaMetrics is tuned for an optimal resource usage under typical
- `-search.maxExportSeries` limits maximum number of time series, which can be returned from [/api/v1/export* APIs](#how-to-export-data-in-json-line-format).
The duration of the export queries is limited via `-search.maxExportDuration` flag. This option allows limiting memory usage.
- `-search.maxTSDBStatusSeries` limits maximum number of time series, which can be processed during the call to [/api/v1/status/tsdb](#tsdb-stats).
The duration of the status queries is limited via `-search.maxStatusRequestDuration` flag. This option allows limiting memory usage.
The duration of the status queries is limited via `-search.maxStatusRequestDuration` flag. This option allows limiting memory usage.
See also [resource usage limits at VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/#resource-usage-limits),
[cardinality limiter](#cardinality-limiter) and [capacity planning docs](#capacity-planning).
@@ -2064,6 +2071,10 @@ Retention filters configuration can be tested in enterprise version of vmui on t
It is safe updating `-retentionFilter` during VictoriaMetrics restarts - the updated retention filters are applied eventually
to historical data.
It's expected that resource usage will temporarily increase when `-retentionFilter` is applied.
This is because additional operations are required to read the data, filter and apply retention to partitions,
which will cost extra CPU and memory.
See [how to configure multiple retentions in VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/#retention-filters).
See also [downsampling](#downsampling).
@@ -2120,6 +2131,10 @@ or [recording rules in vmalert](https://docs.victoriametrics.com/vmalert/#rules)
Downsampling is performed during [background merges](https://docs.victoriametrics.com/#storage).
It cannot be performed if there is not enough of free disk space or if vmstorage is in [read-only mode](https://docs.victoriametrics.com/cluster-victoriametrics/#readonly-mode).
It's expected that resource usage will temporarily increase when **downsampling with filters** is applied.
This is because additional operations are required to read historical data, downsample, and persist it back,
which will cost extra CPU and memory.
Please, note that intervals of `-downsampling.period` must be multiples of each other.
In case [deduplication](https://docs.victoriametrics.com/#deduplication) is enabled, value of `-dedup.minScrapeInterval` command-line flag must also
be multiple of `-downsampling.period` intervals. This is required to ensure consistency of deduplication and downsampling results.
@@ -2969,6 +2984,9 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
Per-second limit on the number of WARN messages. If more than the given number of warns are emitted per second, then the remaining warns are suppressed. Zero values disable the rate limit
-maxConcurrentInserts int
The maximum number of concurrent insert requests. Set higher value when clients send data over slow networks. Default value depends on the number of available CPU cores. It should work fine in most cases since it minimizes resource usage. See also -insert.maxQueueDuration (default 32)
-maxIngestionRate int
The maximum number of samples vmsingle can receive per second. Data ingestion is paused when the limit is exceeded
By default there are no limits on samples ingestion rate.
-maxInsertRequestSize size
The maximum size in bytes of a single Prometheus remote_write API request
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 33554432)

Some files were not shown because too many files have changed in this diff Show More