Compare commits

...

181 Commits

Author SHA1 Message Date
Fred Navruzov
150ee902fd docs/vmanomaly: patch release 1.16.1 (#7169)
### Describe Your Changes

`vmanomaly` patch release 1.16.1 updates

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-02 18:10:08 +04:00
hagen1778
01bc28eda2 docs: mention version available for multitenant reads
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-02 14:58:06 +02:00
hagen1778
cd8a478a8d docs: actualize multitenancy docs
Follow-up after https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6346

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-02 14:35:29 +02:00
hagen1778
36acde1d11 docs: add missing release notes
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-02 14:20:55 +02:00
Roman Khavronenko
0d4f4b8f7d (app|lib)/vmstorage: do not increment vm_rows_ignored_total on NaNs (#7166)
`vm_rows_ignored_total` metric is a metric for users to signalize about
ingestion issues, such as bad timestamp or parsing error.
In commit
a5424e95b3
this metric started to increment each time vmstorage gets NaN. But NaN
is a valid value for Prometheus data model and for Prometheus metrics
exposition format. Exporters from Prometheus ecosystem could expose NaNs
as values for metrics and these values will be delivered to vmstorage
and increment the metric.
Since there is nothing user can do with this, in opposite to parsing
errors or bad timestamps, there is not much sense in incrementing this
metric. So this commit rolls-back `reason="nan_value"` increments.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-02 12:37:27 +02:00
Artem Navoiev
c04c377d09 docs: cloud API remove link to guide as we don't have it
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-10-02 11:10:38 +02:00
Artem Navoiev
504da7d02b docs: add API doc for cloud (#7164)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-10-02 10:58:35 +02:00
Fred Navruzov
dde2a0cb25 docs/vmanomaly - update versions in examples (#7163)
### Describe Your Changes

update `vnanomaly` versions in examples

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-02 11:53:05 +03:00
Fred Navruzov
0e54cfe350 docs/vmanomaly - release 1.16.0 docs (#7159)
### Describe Your Changes

doc updates for vmanomaly v1.16.0

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-02 00:19:14 +02:00
Aliaksandr Valialkin
456aeda605 app/vlogscli: preserve less output
This simplifies logs' investigation, since it allows copying some text from the previous query output
2024-10-01 21:46:36 +02:00
Aliaksandr Valialkin
3634fefc64 docs/VictoriaLogs/querying/vlogscli.md: typo fixes 2024-10-01 21:10:05 +02:00
f41gh7
776c501cb2 CHANGELOG.md: cut v1.104.0 release 2024-10-01 16:55:04 +02:00
f41gh7
076a1f84e1 vmselect: add support of multi-tenant queries
Added  ability to query data across multiple tenants. See:
VictoriaMetrics/VictoriaMetrics#1434

Currently, the following endpoints work with multi-tenancy:
- /prometheus/api/v1/query
- /prometheus/api/v1/query_range
- /prometheus/api/v1/series
- /prometheus/api/v1/labels
- /prometheus/api/v1/label/<label_name>/values
- /prometheus/api/v1/status/active_queries
- /prometheus/api/v1/status/top_queries
- /prometheus/api/v1/status/tsdb
- /prometheus/api/v1/export
- /prometheus/api/v1/export/csv
- /vmui

A note regarding VMUI: endpoints such as `active_queries` and
`top_queries` have been updated to indicate whether query was a
single-tenant or multi-tenant, but UI needs to be updated to display
this info.
cc: @Loori-R

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2024-10-01 16:49:46 +02:00
Aliaksandr Valialkin
496015aa0e docs/VictoriaLogs/querying/vlogscli.md: provide instructions on how to download run vlogscli 2024-10-01 16:13:12 +02:00
Aliaksandr Valialkin
b05fbee63d deployment: update VictoriaLogs docker image from v0.32.1-victorialogs to v0.33.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.33.0-victorialogs
2024-10-01 13:57:56 +02:00
Aliaksandr Valialkin
234c81754e docs/VictoriaLogs/CHANGELOG.md: cut v0.33.0-victorialogs release 2024-10-01 13:42:18 +02:00
Roman Khavronenko
d57d8b5e60 docs: add note that otel is not using streaming parsing (#7148)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-01 13:39:40 +02:00
Aliaksandr Valialkin
a350be48b6 lib/logstorage: do not count dictionary values which have no matching logs in count_uniq stats function
Create blockResultColumn.forEachDictValue* helper functions for visiting matching
dictionary values. These helper functions should prevent from counting dictionary values
without matching logs in the future.

This is a follow-up for 0c0f013a60
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7152
2024-10-01 13:34:45 +02:00
Aliaksandr Valialkin
630211cfed app/vlogscli: add interactive command-line tool for querying VictoriaLogs 2024-10-01 12:23:07 +02:00
Aliaksandr Valialkin
61ae077e43 docs/VictoriaLogs/FAQ.md: clarify answers a bit 2024-10-01 12:08:27 +02:00
Roman Khavronenko
ead3250b80 docs: old logo seems not referenced anymore (#7151)
6y old :)

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-01 10:26:34 +02:00
Nikolay
fbaa026ae6 dashboards: updates operator dashboard (#7139)
* Replaces deprecated graphs with Timeseries panels
* Adds new latency dashboards for rest client and golang scheduler
* Adds new overview panels
* Adds VM Datasource version of dashboard

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-30 15:35:39 +02:00
Artem Navoiev
0b2d3d7752 docs: cloud add support page. remove title as far it's already there
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-30 14:45:33 +02:00
Zhu Jiekun
7bb8853a5c feature: [vmagent] Add service discovery support for OVH Cloud VPS and dedicated server (#6160)
### Describe Your Changes
related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6071

#### Added
- Added service discovery support for OVH Cloud:
    - VPS.
    - Dedicated server.

#### Docs
- `CHANGELOG.md`, `sd_configs.md`, `vmagent.md` are updated.

#### Note
- Useful links: 
    - OVH Cloud VPS API: https://eu.api.ovh.com/console/#/vps~GET
- OVH Cloud Dedicated server API:
https://eu.api.ovh.com/console/#/dedicated/server~GET
    - OVH Cloud SDK: https://github.com/ovh/go-ovh
- Prometheus SD:
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#ovhcloud_sd_config

Tested on OVH Cloud VPS and dedicated server.
<img width="1722" alt="image"
src="https://github.com/VictoriaMetrics/VictoriaMetrics/assets/30280396/d3f0adc8-b0ef-423e-9379-8a9b9b0792ee">

<img width="1724" alt="image"
src="https://github.com/VictoriaMetrics/VictoriaMetrics/assets/30280396/18b5b730-3512-4fc0-8b2c-f2450ac550fd">

---
Signed-off-by: Jiekun <jiekun@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-09-30 14:42:46 +02:00
Aliaksandr Valialkin
aafa9262c5 deployment/docker: update VictoriaLogs from v0.32.0-victorialogs to v0.32.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.32.1-victorialogs
2024-09-30 14:35:40 +02:00
Aliaksandr Valialkin
82482fca4b docs/VictoriaLogs/CHANGELOG.md: cut v0.32.1-victorialogs release 2024-09-30 14:31:17 +02:00
Artem Navoiev
f5f04c903b docs: cloud add support page (#7143)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-30 14:29:26 +02:00
Hui Wang
664f337c70 stream aggregation: fix possible duplicated aggregation results (#7118)
When ingesting samples with the same labels(duplicated samples or
samples with the same labels after `by` or `without` options). They
could register different entries for the same labelset in
LabelsCompressor.
For example, both index 99 and 100 can be assigned to label `foo=1` in
two concurrent pushes. Then due to differing label indexes in encoded
keys, the samples will appear as distinct in aggrState, resulting in
duplicated results after decompressing the label indexes.

fbde238cdc/lib/streamaggr/streamaggr.go (L933)

In this pull request, since we need to store `idxToLabel` first to
ensure the idx can be searched after `lc.labelToIdxStore`,
the `lc.idxToLabel` still could contain a duplicated entries
[100]="foo=1". But given the low likelihood of this issue and the size
of idxToLabel, it should be fine.
2024-09-30 14:24:59 +02:00
Aliaksandr Valialkin
0c0f013a60 lib/logstorage: skip values with zero hits for 'uniq', 'top' and 'field_values' pipes
See https://github.com/VictoriaMetrics/victorialogs-datasource/issues/72#issuecomment-2352078483
2024-09-30 14:15:07 +02:00
Artem Fetishev
85ea0f80fc Change the default value of the maxDeleteSeries flag to 1 million (#7140)
Change the default value of the maxDeleteSeries flag to 1 million. This
is a follow up for ed5da38ede
---

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2024-09-30 12:40:49 +02:00
f41gh7
758f42fc12 docs: add Update Note for upcoming release changes
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-30 12:37:30 +02:00
Artem Fetishev
ed5da38ede Introduce a flag for limiting the number of time series to delete (#7091)
### Describe Your Changes

Introduce the `-search.maxDeleteSeries` flag that limits the number of
time series that can be deleted with a single
`/api/v1/admin/tsdb/delete_series` call.

Currently, any number can be deleted and if the number is big (millions)
then the operation may result in unaccounted CPU and memory usage spikes
which in some cases may result in OOM kill (see #7027). The flag limits
the number to 30k by default and the users may override it if needed at
the vmstorage start time.


---------

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-09-30 10:02:21 +02:00
Alexander Frolov
80a3c410d4 vmselect: ensure default -search.maxConcurrentRequests is non-decreasing (#6996)
### Describe Your Changes

vmselect determines the default value of `-search.maxConcurrentRequests`
multiplying the number of available CPUs by 2 if and only if the number
is small (to be precise <= 4). That leads
`-search.maxConcurrentRequests` is decreasing at the edge of these two
cases as shown below:
| CPUs | MaxConcurrentRequests | MaxConcurrentRequests (original
proposal) | MaxConcurrentRequests (updated proposal) |
|--------|--------|--------|--------|
| 1 | 2 | 2 | 2 |
| 2 | 4 (prev+2) | 4 (prev+2) | 4 (prev+2) |
| 3 | 6 (prev+2) | 6 (prev+2) | 6 (prev+2) |
| 4 | 8 (prev+2) | 8 (prev+2) | 8 (prev+2) |
| 5 | 5 __(prev-3)__ | 9 __(prev+1)__ | 10 __(prev+2)__ |
| 6 | 6 (prev+1) | 10 (prev+1) | 12 (prev+2) |
| 7 | 7 (prev+1) | 11 (prev+1) | 14 (prev+2) |
| 8 | 8 (prev+1) | 12 (prev+1) | 16 (prev+2) |

I propose to make the default value non-decreasing.
2024-09-30 09:51:54 +02:00
hagen1778
2df2e9b92f docs: use global dedup flag in vmagent's FlexibleDeduplication docs
Current doc is using per-url deduplication, and users might use this example
when they have more than 1 remoteWrite URL. Which would result into extra resource usage.
Changing the example to use global dedup, as it makes more sense.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-30 09:08:56 +02:00
Github Actions
3d01bc3fbe Automatic update helm docs from VictoriaMetrics/helm-charts@de4356f (#7138)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-30 13:03:15 +08:00
Github Actions
5e67e611b8 Automatic update operator docs from VictoriaMetrics/operator@0e44654 (#7136)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-29 22:30:24 +02:00
Aliaksandr Valialkin
2047ad20ef deployment: update VictoriaLogs docker image from v0.31.0-victorialogs to v0.32.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.32.0-victorialogs
2024-09-29 14:54:21 +02:00
Aliaksandr Valialkin
45cfb6b526 docs/VictoriaLogs/CHANGELOG.md: cut v0.32.0-victorialogs 2024-09-29 14:47:31 +02:00
Aliaksandr Valialkin
1da4650143 lib/logstorage: allow using ! in unescaped phrase
Previously the phrase filter with `!` was treated unexpectedly.
For example, `foo!bar` filter was treated at `foo AND NOT bar`,
while most users expect that it matches "foo!bar" phrase.

This commit aligns with users' expectations.
2024-09-29 11:14:15 +02:00
Aliaksandr Valialkin
60183c7c79 lib/logstorage: allow using - instead of ! in front of (...) 2024-09-29 11:12:22 +02:00
Nikolay
3bbb2aed72 fscore: rollback trailing space trim (#7106)
Previous commit 201fd6de1e removed
trailing space trim from data read from file. But common practice is to
remove such trailing space. And it leaded to the authorization errors
for the major group of users.

In first place, this change must help to mitigate an issue with
kubernetes. When authorization information was read from Secret content.
Changes to the operator was made to mitigate such problem at commit
1cf64358c8

We could introduce later optional flag for VictoriaMetrics to disable
trim space behavior.

Related issues:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6986
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7089 
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6947

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Zhu Jiekun <jiekun@victoriametrics.com>
2024-09-29 10:59:25 +02:00
Aliaksandr Valialkin
b52862badf lib/logstorage: return the expected hits results from uniq pipe when the number of unique values reaches the specified limit
Previously `uniq` pipe could return zero `hits` if the number of found unique values equals the specified limit.
This wasn't expected in most cases.
2024-09-29 10:51:09 +02:00
Aliaksandr Valialkin
55eb321f77 lib/logstorage: clear hits slice obtained from encoding.GetUint64s() before updating it with hits for valueTypeDict column
encoding.GetUint64s() returns uninitialized slice, which may contain arbitrary values.
So values in this slice must be reset to zero before using it for counting hits in `uniq` and `top` pipes.
2024-09-29 10:29:13 +02:00
Aliaksandr Valialkin
94afcbd9a9 lib/logstorage: postpone initialization of per-shard stateSizeBudget until the first call to pipeProcessor.writeBlock()
This simplifies pipeProcessor initialization logic a bit.
This also doesn't mangle the original maxStateSize value, which is used in error messages when the state size exceeds maxStateSize.
2024-09-29 10:29:13 +02:00
Andrii Chubatiuk
dfcab4a47f deployment/victorialogs: rename not executable compose files (#7124)
### Describe Your Changes

Renamed base compose files to prevent envs to be created from them

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-29 09:54:45 +02:00
Aliaksandr Valialkin
0b91452ca4 lib/logstorage: add non-empty if (...) condition to automatically generated result names in stats pipe
This allows executing queries with `stats` pipe, which calculate multiple results with the same functions,
but with different `if (...)` conditions. For example:

  _time:5m | count(), count() if (error)

Previously such queries couldn't be executed becasue automatically generated name for the second result
didn't include `if (error)`, so names for both results were identical - `count(*)`.
2024-09-29 09:51:28 +02:00
Aliaksandr Valialkin
8772aea24b lib/logstorage: support order alias for sort pipe
Now the following queries are equivalents:

    _time:5s | sort by (_time)

    _time:5s | order by (_time)

This is needed for convenience, since `order by` is commonly used in other query languages such as SQL.
2024-09-29 09:51:27 +02:00
Artem Navoiev
14a0396f53 docs: changelog fix typo in url
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-28 23:30:19 +02:00
Artem Navoiev
96efe99eef docs: mention new create backup api in docs and changelog (#7104)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-28 14:28:58 -07:00
Aliaksandr Valialkin
806bc2ac58 app/vlinsert: support unix timestamps in seconds and milliseconds in JSON stream data ingestion API 2024-09-28 21:56:50 +02:00
Aliaksandr Valialkin
7d7d7c03bc app/vlinsert: accept unix timestamp in seconds additionally to milliseconds at ElasticSearch bulk API
Timestamps in seconds are sometimes used for data ingestion via ElasticSearch bulk API
2024-09-28 21:19:54 +02:00
Roman Khavronenko
fd890a2771 deployment: separate datasource between single and cluster versions (#7119)
Before, single and cluster deployments were provisioned with both
Grafana datasources: single and cluster. But this resulted into a
problem: single DS didn't work for cluster and vice versa. See
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7113

This PR splits datasource file into 2 files: single and cluster. Now,
these files are separately provisioned to single and cluster deployments
correspondingly.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-27 19:08:05 +02:00
Github Actions
e62b88b2e0 Automatic update operator docs from VictoriaMetrics/operator@d39fc94 (#7129)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-27 19:03:14 +02:00
Andrii Chubatiuk
00912bfa0f docs/victorialogs: change fluentd input type in examples (#7123)
### Describe Your Changes

Use fluentd logging driver in examples to have enriched data in
VictoriaLogs

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-27 15:09:36 +02:00
Roman Khavronenko
59bc63ebc4 app/vmalert: mention labels conflict resolution strategy (#7085)
The change should help users to understand what happens on labels
conflict.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-27 14:41:33 +02:00
Andrii Chubatiuk
05a64a8c14 victorialogs: marked fluentd support in roadmap, added syslog example (#7098)
### Describe Your Changes

Marked fluentd in victorialogs roadmap
Added fluentd syslog example setup

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-27 14:38:39 +02:00
Aliaksandr Valialkin
86c0eb816c deployment: update VictoriaLogs docker image from v0.30.1-victorialogs to v0.31.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.31.0-victorialogs
2024-09-27 13:58:54 +02:00
Aliaksandr Valialkin
58c69386c7 docs/VictoriaLogs/CHANGELOG.md: cut v0.31.0-victorialogs release 2024-09-27 13:54:17 +02:00
Aliaksandr Valialkin
c8e23eefba app/{vmselect,vlselect}: run make vmui-update vmui-logs-update after 25a9802ca4 and 8657d03433
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7088
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5924

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7025
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545#issuecomment-2336805237
2024-09-27 13:50:47 +02:00
Yury Molodov
25a9802ca4 vmui: add link to vmalert (#7088)
### Describe Your Changes

Add link to VMalert when proxy is enabled. The link is displayed when
the `-vmalert.proxyURL` flag is present.

#5924


![image](https://github.com/user-attachments/assets/c45ca884-8912-4bd9-a867-df5919f278a1)

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-09-27 13:22:22 +02:00
Yury Molodov
8657d03433 vmui/logs: improve graph usability (#7025)
### Describe Your Changes

- Show the time range in the tooltip when hovering over staircase
graphs.
- Use bolder lines for staircase graphs.
- Increase the number of steps on the staircase graph to 100.
- Reduce the maximum width of the tooltip to 1/3 of the screen.
- Insert only the label name under the cursor into the query input field
when `Ctrl`-clicking the line legend.

See [this
comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545#issuecomment-2336805237).

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-09-27 13:19:46 +02:00
Aliaksandr Valialkin
09b309a82e lib/logstorage: allow using - instead of ! as a shorthand for NOT operator in LogsQL 2024-09-27 13:14:47 +02:00
Aliaksandr Valialkin
76c1b0b8ea lib/logstorage: support skipping _stream: prefix for stream filters
'_stream:{...}' can be written as '{...}'

This simplifies writing queries with stream filters, and makes them more familier to Loki users.
2024-09-27 13:14:46 +02:00
Hui Wang
fbde238cdc stream aggregation: support configuring multiple labels per `remoteWrite… (#7073)
….url` using `-remoteWrite.streamAggr.dropInputLabels`

Before, labels were set to all the `remoteWrite.url`.

address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6780

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-27 12:21:09 +02:00
Yury Molodov
c896bf340d vmui: add functionality to preserve selected columns (#7037)
### Describe Your Changes

1) Changed table settings from a popup to a modal window to simplify
future functionality additions.
2) Added functionality to save selected columns when data is modified or
the page is reloaded. See #7016.

<details>
  <summary>Example screenshots</summary>
  
<img alt="demo-1" width="600"
src="https://github.com/user-attachments/assets/a5d9a910-363c-4931-8b12-18ea8b3d97d8"/>
  
</details>


### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-27 11:52:01 +02:00
Github Actions
2d26d3e3de Automatic update operator docs from VictoriaMetrics/operator@db0d09f (#7111)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-27 11:42:10 +02:00
Github Actions
ea3b20622a Automatic update helm docs from VictoriaMetrics/helm-charts@ed0f351 (#7115)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-09-27 11:41:24 +02:00
Aliaksandr Valialkin
b670b0e9ff deployment: update VictoriaLogs docker image tag from v0.30.0-victorialogs to v0.30.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.30.1-victorialogs
2024-09-27 11:26:15 +02:00
Aliaksandr Valialkin
8077585303 docs/VictoriaLogs/CHANGELOG.md: cut v0.30.1-victorialogs release 2024-09-27 11:20:58 +02:00
Aliaksandr Valialkin
9367a9a6a2 lib/logstorage: consistently sort stream contexts belonging to different streams by the minimum time seen in the matching logs
This should simplify debugging of stream_context output, since it remains stable over repeated requests.
2024-09-27 11:19:26 +02:00
Aliaksandr Valialkin
b49d1ea809 lib/logstorage: add _msg="---" delimiter between different log streams in stream_context output
This should help investigating contexts, which belong to different log streams.
2024-09-27 11:01:13 +02:00
Aliaksandr Valialkin
13cc4a2618 docs/VictoriaLogs/LogsQL.md: use proper heading for blocks_count pipe docs
All the links in docs assume that the heading for pipe docs should end with `pipe` word.
This fixes broken links.

This is a follow-up for e9950f6307
2024-09-27 09:45:46 +02:00
Aliaksandr Valialkin
6f1fde24dc deployment: update VictoriaLogs docker image from v0.29.0-victorialogs to v0.30.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.30.0-victorialogs
2024-09-27 09:34:09 +02:00
Aliaksandr Valialkin
31117c66d5 docs/VictoriaLogs/CHANGELOG.md: cut v0.30.0-victorialogs release 2024-09-27 09:18:31 +02:00
Aliaksandr Valialkin
f65b976eda vendor: run make vendor-update 2024-09-26 22:33:05 +02:00
Aliaksandr Valialkin
b82bd0c2ec lib/logstorage: improve performance for stream_context pipe over streams with big number of log entries
Do not read timestamps for blocks, which cannot contain surrounding logs.
This should improve peformance for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6730 .

Also optimize min(_time) and max(_time) calculations a bit by avoiding conversion
of timestamp to string when it isn't needed.
This should improve performance for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7070 .
2024-09-26 22:22:23 +02:00
Aliaksandr Valialkin
3646724c6f lib/contextutil: make golanci-lint happy by substituing unused function arg name with _
This is a follow-up for 4b1611267f
2024-09-26 17:06:48 +02:00
Aliaksandr Valialkin
4b1611267f lib/logstorage: properly return surrounding logs outside the selected time range by stream_context pipe
Previously only logs inside the selected time range could be returned by stream_context pipe.
For example, the following query could return up to 10 surrounding logs only for the last 5 minutes,
while most users expect this query should return up to 10 surrounding logs without restrictions on the time range.

    _time:5m panic | stream_context before 10

This enables the ability to implement stream context feature at VictoriaLogs web UI: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7063 .

Reduce memory usage when returning stream context over big log streams with millions of entries.
The new logic scans over all the log messages for the selected log stream, while keeping in memory only
the given number of surrounding logs. Previously all the logs for the given log stream on the selected time range
were loaded in memory before selecting the needed surrounding logs.
This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6730 .

Reduce the scan performance for big log streams by fetching only the requested fields. For example, the following
query should be executed much faster than before if logs contain many fields other than _stream, _msg and _time:

    panic | stream_context after 30 | fields _stream, _msg, _time
2024-09-26 17:03:45 +02:00
Aliaksandr Valialkin
037652d5ae app/vlinsert: support _time field without timezone information during data ingestion
Use local timezone of the host server in this case. The timezone can be overridden
with TZ environment variable if needed.

While at it, allow using whitespace instead of T as a delimiter between data and time
in the ingested _time field. For example, '2024-09-20 10:20:30' is now accepted
during data ingestion. This is valid ISO8601 format, which is used by some log shippers,
so it should be supported. This format is also known as SQL datetime format.

Also assume local time zone when time without timezone information is passed to querying APIs.
Previously such a time was parsed in UTC timezone. Add `Z` to the end of the time string
if the old behaviour is preferred.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6721
2024-09-26 12:49:35 +02:00
Aliaksandr Valialkin
6b775ca68c app/vlinsert/insertutils: add a link to docs why _msg field must be non-empty 2024-09-26 09:53:17 +02:00
Aliaksandr Valialkin
7c86835f3c docs/VictoriaLogs/CHANGELOG.md: typo fix: itentifying -> identifying 2024-09-26 09:41:30 +02:00
Zhu Jiekun
7185fe012b feature: [victorialogs] drop logs without non-empty _msg field (#7056)
### Describe Your Changes

VictoriaLogs allows logs without `_msg` field or `_msg` field is empty.
This lead to incorrect search result. See:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785

This pull request search for non-empty `_msg` field before log entry is
added to `LogRows`.

New counter `vl_rows_dropped_total{reason="msg_not_exist"}` is
introduced.

Example log output:
```
2024-09-23T02:33:19.719Z        warn    app/vlinsert/insertutils/common_params.go:189   dropping log line without _msg field; [{@timestamp 2024-09-18T13:42:16.600000000Z} {Attributes.array.attribute ["many","values"]} {Attributes.boolean.attribute true} {Attributes.double.attribute 637.704} {Attributes.int.attribute 10} {Attributes.map.attribute.some.map.key some value} {Attributes.string.attribute some string} {Body Example ddddddddddlog record} {Resource.service.name my.service} {Scope.my.scope.attribute some scope attribute} {Scope.name my.library} {Scope.version 1.0.0} {SeverityNumber 10} {SeverityText Information} {SpanId eee19b7ec3c1b174} {TraceFlags 0} {TraceId 5b8efff798038103d269b633813fc60c}]
```

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
- [ ] Benchmark for potential performance loss.

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-09-26 09:35:28 +02:00
Aliaksandr Valialkin
2ec0cfec62 docs/VictoriaLogs/CHANGELOG.md: document the fix for Windows build
This is a follow-up for 264c2ec6bd

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6998
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6973
2024-09-26 09:15:27 +02:00
Aliaksandr Valialkin
c6b2cac892 docs/VictoriaLogs/CHANGELOG.md: typo fix after 255d1d4e13: returns -> return 2024-09-26 09:00:55 +02:00
Aliaksandr Valialkin
255d1d4e13 app/vlselect/logsql: clone the query with the current timestamp when performing live tailing requests in the loop
Previously the original timestamp was used in the copied query, so _time:duration filters
were applied to the original time range: (timestamp-duration ... timestamp]. This resulted
in stopped live tailing, since new logs have timestamps bigger than the original time range.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7028
2024-09-26 08:57:23 +02:00
Github Actions
c89a7a0b62 Automatic update operator docs from VictoriaMetrics/operator@5271a59 (#7099)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-25 23:23:08 +02:00
Aliaksandr Valialkin
c66da8b0ba docs/LTS-releases.md: consistently use v prefix in front of VictoriaMetrics releases 2024-09-25 19:29:30 +02:00
Aliaksandr Valialkin
e9950f6307 lib/logstorage: add blocks_count pipe
This pipe is useful for debugging purposes when the number of processed blocks must be calculated for the given query:

    <query> | blocks_count

This helps detecting the root cause of query performance slowdown in cases like https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7070
2024-09-25 19:17:48 +02:00
Aliaksandr Valialkin
65b93b17b1 lib/logstorage: lazily read column headers metadata during queries
This improves performance for analytical queries, which do not need column headers metadata.
For example, the following query doesn't need column headers metadata, since _stream and min(_time)
are stored in block header, which is read separately from colum headers metadata:

  _time:1w | stats by (_stream) min(_time) min_time

This commit significantly improves the performance for this query.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7070
2024-09-25 19:17:48 +02:00
Aliaksandr Valialkin
4599429f51 lib/logstorage: read timestamps column when it is really needed during query execution
Previously timestamps column was read unconditionally on every query.
This could significantly slow down queries, which do not need reading this column
like in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7070 .
2024-09-25 19:17:47 +02:00
Andrii Chubatiuk
f934f71708 docs/victorialogs/data-ingestion: removed FluentBit Elasticsearch from examples (#7093)
removed FluentBit Elasticsearch example from docs as custom headers are
not supported by elasticsearch output till
https://github.com/fluent/fluent-bit/pull/9416 is merged and released

fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6985

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-25 18:43:26 +02:00
Andrii Chubatiuk
e75ae1b274 deployment: restructure victorialogs examples (#6971)
### Describe Your Changes

- Use common compose.yaml file for all victorialogs setups to set
version in a single place and override it on demand for each agent and
protocol
- Replaced multiple victorialogs instances in HA setup with single setup
with `deploy.replica` parameter set
- Added fluentd setup

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-25 18:33:26 +02:00
Github Actions
612be0954c Automatic update operator docs from VictoriaMetrics/operator@1feab7d (#7092)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-25 15:02:24 +02:00
Roman Khavronenko
6b1b47df54 app/vmalert: bump default values for sending data to remoteWrite.url (#7084)
* `remoteWrite.maxQueueSize` from `100_000` to `1_000_000`, this should
improve resiliency of recording rules that produce many series;
* `remoteWrite.maxBatchSize` from `1_000` to `10_000`, this should be
more efficient to send from netwroking perspective;
* `remoteWrite.concurrency` from `1` to `4`, this should imrpove speed
of sending the generated series.

The new settings should improve remote write performance of vmalert with
default settings.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Hui Wang <haley@victoriametrics.com>
2024-09-25 15:01:39 +02:00
Aliaksandr Valialkin
7f1ba18719 lib/logstorage: improve the performance of obtaining _stream column value
Substitute global streamTagsCache with per-blockSearch cache for ((stream.id) -> (_stream value)) entries.
This improves scalability of obtaining _stream values on a machine with many CPU cores, since every CPU
has its own blockSearch instance.

This also should reduce memory usage when querying logs over big number of streams, since per-blockSearch
cache of ((stream.id) -> (_stream value)) entries is limited in size, and its lifetime is bounded by a single query.
2024-09-24 20:57:00 +02:00
Aliaksandr Valialkin
cf2e7d0d92 lib/logstorage/consts.go: document that it isn't recommended setting maxColumnsPerBlock constant to too big values
This should help avoiding cases like this one - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6425#issuecomment-2337446083
2024-09-24 18:51:46 +02:00
Aliaksandr Valialkin
f86e093b20 lib/logstorage: improve performance for streamID.marshalString() by more than 2x
The streamID.marshalString() is executed in hot path if the query selects _stream_id field.

Command to run the benchmark:

go test ./lib/logstorage/ -run=NONE -bench=BenchmarkStreamIDMarshalString -benchtime=5s

Results before the commit:

BenchmarkStreamIDMarshalString-16    	438480714	        14.04 ns/op	  71.23 MB/s	       0 B/op	       0 allocs/op

Results after the commit:

BenchmarkStreamIDMarshalString-16    	982459660	         6.049 ns/op	 165.30 MB/s	       0 B/op	       0 allocs/op
2024-09-24 18:35:04 +02:00
Aliaksandr Valialkin
919d2dc90e lib/logstorage: add benchmark for streamID.marshalString 2024-09-24 18:31:38 +02:00
Roman Khavronenko
9a0f697622 docs: update CONTRIBUTING.md with practical requirements (#7087)
The change supposed to have more practical recommendations and reflect
the real processes for maintaining the project.


Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-24 18:22:18 +02:00
Github Actions
e28265fa39 Automatic update operator docs from VictoriaMetrics/operator@27ad7e1 (#7086)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-24 15:47:30 +02:00
hagen1778
8bb3f2fd43 lib/promscrape: make linter happy
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-24 15:12:55 +02:00
hagen1778
c7569dac50 lib/promscrape: temporary disable TestClientProxyReadOk
This test is very flaky and prevents other tests from running in CI.
Disabling this test should improve tests quality, since it isn't reliable anyway.

There is a ticket to fix this test - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7062

Once fixed, this test should be uncommented.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-24 14:59:25 +02:00
Zhu Jiekun
5319acb8ed vmagent: remote write respect Retry-After in header (#6124)
### Describe Your Changes
related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6097

#### Changed
- Remote write retry policy in `vmagent` is changed into:
  1. Respect `Retry-After` duration if exists.
2. Otherwise, calculate next retry duration by backoff policy (x2) and
max retry duration limit.
 
#### Docs
- `CHANGELOG.md`.

---
### Checklist
The following checks are mandatory:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Zakhar Bessarab <me@zekker-dev.tk>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-09-24 12:44:03 +02:00
Dmytro Kozlov
cbeb7d50e8 lib/promscrape: show only unhealthy targets if show_only_unhealthy filter is enabled (#6960)
### Describe Your Changes

It is better to show only unhealthy targets instead of all of them when
`show_only_unhealthy` filter is enabled.
Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3536

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-24 12:18:24 +02:00
Phuong Le
df665a13c9 docs: update logos files and usage rules (#6980)
### Describe Your Changes

New logos and usage guideline

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-24 11:53:58 +02:00
Zhu Jiekun
fea4433362 docs: [VictoriaLogs] OTel Collector elasticsearchexporter header note (#7074)
### Describe Your Changes

By default, the `elasticsearchexporter` in OTel Collector puts the log
message under a field other than `_msg` (e.g., `Body`). Without
specifying via an HTTP header, those logs may not be queried correctly.
See also:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785.

This pull request updates the example configuration and notes for the
`elasticsearchexporter`.

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-24 11:52:09 +02:00
Dmytro Kozlov
91b28d0527 deployment/docker: update grafana datasources to the latest version (#7083)
### Describe Your Changes

Updated grafana plugins to the latest releases 

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-24 11:51:18 +02:00
Github Actions
524579d9bd Automatic update operator docs from VictoriaMetrics/operator@75bc1b4 (#7080)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-09-24 11:50:50 +02:00
hagen1778
a5c002edef deployment/alerts: fix copy&paste typo in TooHighGoroutineSchedulingLatency
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-24 11:48:19 +02:00
Roman Khavronenko
4d0b41e63b deployment: add panel and alerts for displying go scheduler latency (#7078)
The panel and alerting rule should help to understand whether VM
component doesn't have enough CPU resources or gets throttled. The alert
is applicable for all VM components.
The panel was added to vmalert, vmagent, vmsingle, vm clusert and
victorialogs dashes.

-------------------

This alerting rule should have help us identify resource shortage for
sandbox vmagent - see [this
link](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/?g0.range_input=23d13h25m25s424ms&g0.end_input=2024-09-23T14%3A11%3A00&g0.relative_time=none&g0.tab=0&g0.expr=histogram_quantile%280.99%2C+sum%28rate%28go_sched_latencies_seconds_bucket%7Bjob%3D%22vmagent-monitoring-vmagent%22%7D%5B5m%5D%29%29+by+%28le%2C+job%2C+instance%29%29+%3E+0.1)
for example. We weren't aware of resource shortage, because VM metrics
assumed this vmagent had 1vCPU while in fact its limit was 0.2vCPU.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-23 16:54:42 +02:00
Aliaksandr Valialkin
109772bdc4 lib/cgroup: round GOMAXPROCS to the lower integer value of cpuQuota
Rounding GOMAXPROCS to the upper interger value of cpuQuota increases chances of CPU starvation,
non-optimimal goroutine scheduling and additional CPU overhead related to context switching.

So it is better to round GOMAXPROCS to the lower integer value of cpuQuota.
2024-09-23 16:09:12 +02:00
Aliaksandr Valialkin
3964889705 app/vmselect/promql: consistently replace NaN data points with non-NaN values for range_first and range_last functions
It is expected that range_first and range_last functions return non-nan const value across all the points
if the original series contains at least a single non-NaN value. Previously this rule was violated for NaN data points
in the original series. This could confuse users.

While at it, add tests for series with NaN values across all the range_* and running_* functions, in order to maintain
consistent handling of NaN values across these functions.
2024-09-23 14:59:29 +02:00
hagen1778
3ed172eeeb docs: add note about testing new releases on testing env
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-23 11:58:43 +02:00
Zakhar Bessarab
78ecfede06 license: add ability to reload keys (#775)
* lib/license: add support of license key hot-reload

* docs: add info about license key hot reload

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-09-23 09:04:17 +02:00
Aliaksandr Valialkin
57183c9b61 docs/changelog/CHANGELOG.md: moved the description of the fix for proper usage of -streamAggr.dedupInterval and -remoteWrite.streamAggr.dedupInterval from FEATURE to BUGFIX section
The previous behaviour was incorrect, since it is unexpected that the -streamAggr.dedupInterval
and -remoteWrite.streamAggr.dedupInterval is applied to processed samples only if -streamAggr.config isn't set.

This is a follow-up for d523015f27
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6711
2024-09-23 08:56:33 +02:00
Github Actions
4dc85613a2 Automatic update helm docs from VictoriaMetrics/helm-charts@428cb36 (#7072)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-09-23 13:17:34 +08:00
Aliaksandr Valialkin
0ada781cf2 docs/changelog/CHANGELOG.md: document bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7009
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7064
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7009

This is a follow-up for 55febc0920
2024-09-22 21:57:48 +02:00
Github Actions
8ed9978591 Automatic update operator docs from VictoriaMetrics/operator@d661554 (#7069)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-09-22 18:39:52 +02:00
Artem Fetishev
55febc0920 lib/storage: restore ability to put empty metric ID list into tagFiltersToMetricIDsCache (#7064)
### Describe Your Changes

Currently it the metricID list is empty it won't be mashalled and as the
result won't be put into the tagFiltersToMetricIDsCache which causes the
cache misses for the corresponding tagFilters. In some setups this
causes severe search speed detradation (see #7009).

The empty metric IDs was covered before but then was accidentally
removed in 6c21439.

This PR restores the coverage of this case.

A new unit test can be used as a proof that empty metricID lists are not
added to the cache (just remove the fix in index_db.go and run the test
to see the result)

Also a benchmark has been added to see the implications of the
compression.

```
user@laptop:~/p/github.com/rtm0/VictoriaMetrics/01/src$ go test ./lib/storage/ -run=NONE -bench BenchmarkMarshalUnmarshalMetricIDs --loggerLevel=ERROR
goos: linux
goarch: amd64
pkg: github.com/VictoriaMetrics/VictoriaMetrics/lib/storage
cpu: 13th Gen Intel(R) Core(TM) i7-1355U
BenchmarkMarshalUnmarshalMetricIDs/numMetricIDs-0-12             3237240               363.5 ns/op               0 compression-rate
BenchmarkMarshalUnmarshalMetricIDs/numMetricIDs-1-12             2831049               451.8 ns/op               0.4706 compression-rate
BenchmarkMarshalUnmarshalMetricIDs/numMetricIDs-10-12            1152764              1009 ns/op                 1.667 compression-rate
BenchmarkMarshalUnmarshalMetricIDs/numMetricIDs-100-12            297055              3998 ns/op                 5.755 compression-rate
BenchmarkMarshalUnmarshalMetricIDs/numMetricIDs-1000-12            31172             34566 ns/op                 8.484 compression-rate
BenchmarkMarshalUnmarshalMetricIDs/numMetricIDs-10000-12            4900            289659 ns/op                 9.416 compression-rate
BenchmarkMarshalUnmarshalMetricIDs/numMetricIDs-100000-12            447           2341173 ns/op                 9.456 compression-rate
BenchmarkMarshalUnmarshalMetricIDs/numMetricIDs-1000000-12            42          24926928 ns/op                 9.468 compression-rate
BenchmarkMarshalUnmarshalMetricIDs/numMetricIDs-10000000-12            5         204098872 ns/op                 9.467 compression-rate
PASS
ok      github.com/VictoriaMetrics/VictoriaMetrics/lib/storage  15.018s
```

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Fetishev <wwctrsrx@gmail.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-09-20 17:21:53 +02:00
Artem Navoiev
a2ecba4154 docs: cloud siplify menu items names for notification, user managemenr and notifs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-20 16:16:25 +02:00
Github Actions
8645438a79 Automatic update helm docs from VictoriaMetrics/helm-charts@f0e007f (#7059)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-09-20 16:11:23 +02:00
Artem Navoiev
9e9583357d docs add root menu to _index.md as homepage
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-20 16:06:15 +02:00
Github Actions
75a29655f5 Automatic update operator docs from VictoriaMetrics/operator@fdd3f9f (#7065)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: tenmozes <1381404+tenmozes@users.noreply.github.com>
2024-09-20 16:03:52 +02:00
Andrii Chubatiuk
5708a85499 docs: updated root menu items (#7061)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-20 06:14:29 -07:00
Aliaksandr Valialkin
787b9cd9a0 lib/storage: improve performance for indexSearch.containsTimeRange()
The indexSearch.containsTimeRange() function is called for the current indexDB and the previous indexDB
every time when searching for metricIDs by label filters. This function consumes a lot of additional CPU time
for cases when queries with lightweight label filters are sent to VictoriaMetrics at high rate (e.g. thousands of RPS),
like in the issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7009 .

Optimize indexSearch.containsTimeRange() function in the following ways:

- Unconditionally return true if this function is called for the current indexDB, since there are very high
  chances that the current indexDB contains the data with timestamps in the requested time range.

- Cache the minimum timestamp, which is missing in the indexed data for the previous indexDB.
  This is safe to do, since the previous indexDB is readonly.
  This optimization eliminates potentially slow lookup in the previous indexDB for typical
  use cases when the requested time range is close to the current time.
2024-09-20 13:07:20 +02:00
Aliaksandr Valialkin
6f61e9d49d lib/storage: simplify indexDB.doExtDB() usage by removing the returned value
Previously indexDB.doExtDB() was returning boolean value, which was indicating whether f callback was called.
There is no need in returning this boolean value, since the f callback can determine on itself whether it was called.

This simplifies the code a bit.

While at it, document indexDB.doExtDB().
2024-09-20 11:59:57 +02:00
Roman Khavronenko
218c533874 lib/storage: follow-up after d8f8822fa5 (#7036)
Make function name and comments more clear.

d8f8822fa5

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-09-20 11:50:47 +02:00
Artem Navoiev
7596e239eb docs: set operator menu item weight to 30
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-20 11:22:03 +02:00
Artem Navoiev
36d58588a7 docs: clarify operator name.2
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-20 11:20:16 +02:00
Artem Navoiev
cd384c7547 docs: clarify operator name
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-20 11:11:38 +02:00
Hui Wang
d6d02d7aeb vmalert: fix variable $activeAt value when templating rule annotation in replay mode
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-20 11:07:40 +02:00
hagen1778
6167bccc5a docs: fix more typos in the changelog
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-20 10:54:43 +02:00
hagen1778
59281d5358 docs: rm update node about loggerMaxArgLen as it doesn't have incompatibility effect
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-20 10:42:07 +02:00
hagen1778
6726a5aaed docs: fix typo in link in change line about NaN
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-20 10:39:13 +02:00
Thomas Danielsson
258201af04 docs: fix typo in the changelog
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-20 08:46:29 +02:00
Aliaksandr Valialkin
a3d8077959 lib/logstorage: make sure that getCommonTokens returns common tokens in the original order of tokens inside tokenSets arg
This fixes flaky test TestGetCommonTokensForOrFilters:

    filter_or_test.go:143: unexpected tokens for field "_msg"; got ["foo" "bar"]; want ["bar" "foo"]
2024-09-19 15:59:48 +02:00
hagen1778
c00b64726c app/{vmselect,vlselect}: run make vmui-update vmui-logs-update
Executed after https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6972
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6900

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-19 15:39:40 +02:00
f41gh7
61721303fd docs/changelog: mention vmagent kafka consumer bugfix
Changes were made to the enteprise repository
2024-09-19 15:35:48 +02:00
Yury Molodov
4e976f66f3 vmui: optimize public directory by cleaning up files (#6972)
### Describe Your Changes

### Pull Request Description:

1. **HTML File Structure Optimization**: Adjusted the location of HTML
files for different builds to prevent redundant files in the final
output. See issue #6900
2. **Metadata Fixes**: Corrected metadata in HTML files for each build
configuration.
3. **Favicon Update**: Replaced PNG favicon (`14 KB` and `1.58 KB`) with
SVG (`1.35 KB`).
4. **Social Media Optimization**: Optimized the social preview image,
reducing its size by `60.2 KB`.
5. **Git Ignore Update**: Added `public/index.html` to `.gitignore` as
it is dynamically generated during the build process.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-19 14:37:16 +02:00
Thomas Danielsson
69b976f08e docs: typo filebeat -> fluent-bit (#7000)
### Describe Your Changes

Fix a typo; `filebeat.yml` -> `fluent-bit.conf`

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Thomas Danielsson <thomas@elajt.se>
2024-09-19 14:33:12 +02:00
Yury Molodov
b0bdb92729 vmui: change the query_range request method from GET to POST (#7039)
### Describe Your Changes

change the `/query_range` and `/query` requests method from `GET` to
`POST`. See #6288.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-09-19 14:30:54 +02:00
Roman Khavronenko
e115b85770 lib/logger: increase default value of -loggerMaxArgLen cmd-line fla… (#7008)
…g from 1e3 to 5e3

This should improve visibility on errors produced by very long queries.

The change is classified as BUG in order to port it to LTS releases.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Mathias Palmersheim <mathias@victoriametrics.com>
2024-09-19 14:29:18 +02:00
Yury Molodov
7491f49e9e vmui: update dependencies in package.json to latest versions (#7007)
Update dependencies in `package.json` to latest versions
2024-09-19 11:43:52 +02:00
Yury Molodov
bc9cb69170 vmui/logs: add auto refresh (#7038)
### Describe Your Changes

Add auto refresh 
#7017


![image](https://github.com/user-attachments/assets/20ed1102-d5e4-4d3f-9c24-7d298d93400a)

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-19 11:11:16 +02:00
Thomas Danielsson
75cc7922c3 typo: added missing '{' (#7044)
### Describe Your Changes

Added missing `{` in vmalert rule.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-19 00:11:30 -07:00
Aliaksandr Valialkin
e86891b010 app/vlselect/logsql: call Query.Optimize() on the cloned query in order to replace * filter with filterNoop inside getLastNQueryResults()
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785
2024-09-18 18:24:17 +02:00
Aliaksandr Valialkin
b82e2cabc5 app/vmselect/promql: properly calculate c1 and c2 and c1 or c2 by upgrading github.com/VictoriaMetrics/metricsql to v0.79.0
The fix is in the https://github.com/VictoriaMetrics/metricsql/pull/34
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6637
2024-09-18 17:38:19 +02:00
Github Actions
8bc30b68cd Automatic update operator docs from VictoriaMetrics/operator@fe90f57 (#7026)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-17 19:30:09 +02:00
Dmytro Kozlov
4c228f1e18 docs/victoriametrics-cloud: updated pictures and descriptions with new functionality (#7014)
Improved VictoriaMetrics documentation for cloud

Related issue: https://github.com/VictoriaMetrics/cloud/issues/2143

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-16 23:33:40 -07:00
Nikolay
d8f8822fa5 lib/storage: consistently check for missing metricID index records (#6967)
* Previously, only metricID->metricName missing index records were
tracked with deadline But it was possible a case for missing
metricID->TSID index records. IndexDB metrics fix exposed misleading
metric for such missing records.

* This commit adds check for metricID->TSID missing index records. And
delete missing metricID entry if it hit 60 second deadline.

Related issue
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6931

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-16 10:05:08 +02:00
Nikolay
264c2ec6bd lib/fs: properly call windows APIs (#6998)
Previously we manually imported system windows DDLs
and made direct syscall.

 But golang exposes syscall wrappers with sys/windows package.
It seems, that direct syscall was broken at 1.23 golang release. It was
`GetDiskFreeSpace` syscall in our case.

This commit replaces all manual syscalls with wrappers

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6973

Related golang issue:
https://github.com/golang/go/issues/69029

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-13 12:22:25 +02:00
Dima Lazerka
8207879fa3 docs: fixes misspelled typos
Also tried to make it catch "Authorisation" in the future, fixed a lot
of other misspells along the way, but didn't make it catch
"Authorisation" anyway.

- Fix misspelled "Authorization" header name
- Fix misspelled "organization"
- Fix more misspells
2024-09-13 12:14:24 +02:00
Aliaksandr Valialkin
e92f347336 docs/VictoriaLogs/querying/README.md: typo fixes: use field_values instead field_names where needed 2024-09-11 09:39:22 +02:00
Alexander Marshalov
c0272463d9 [vmcloud]: Cloud tiers assumptions docs (#6948)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-10 12:41:25 +02:00
Artem Navoiev
277fed9990 victorialogs: add HA example for logstash and fluentbit (#6968)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 14:33:05 -07:00
Artem Navoiev
153cceb124 docs: adjust image size
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 21:30:54 +02:00
Artem Navoiev
ae5f98f46b docs: fix links to docker
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 19:56:58 +02:00
Artem Navoiev
71df11dfbb docs: victorialogs just mention telegraf as we support not only http but more
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 17:41:15 +02:00
Artem Navoiev
47cd7bb4c1 docs: update roadmap, mark telegraf and otel as done and link the docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 17:35:14 +02:00
Hui Wang
ae4d376e41 vmalert: do not send message to alertmanager when alert has no label … (#6823)
…pair

`alert_relabel_configs` in [notifier
config](https://docs.victoriametrics.com/vmalert/#notifier-configuration-file)
can drop alert labels when used to filter different tenant alert message
to different notifier.
alertmanager would report error like `msg="Failed to validate alerts"
err="at least one label pair required"` in this case, but the rest of
the alerts inside one request would still be valid in alertmanager, so
it's not severe.
2024-09-09 13:34:48 +02:00
Artem Navoiev
f9a8c09fe8 docs: Add VictoriaLogs HA example for single nodes (#6945)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 03:39:58 -07:00
hagen1778
7983ea4edb docs: mention -dryRun flag next to config example in vmalert
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-09 11:34:52 +02:00
Aliaksandr Valialkin
0ed835026f deployment: update VictoriaLogs from v0.28.0-victorialogs to v0.29.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.29.0-victorialogs
2024-09-08 21:23:16 +02:00
Aliaksandr Valialkin
fb339d7574 docs/VictoriaLogs/CHANGELOG.md: cut v0.29.0 2024-09-08 21:10:17 +02:00
Aliaksandr Valialkin
c6faab77f9 vendor: run make vendor-update 2024-09-08 21:05:06 +02:00
Aliaksandr Valialkin
6e4d29fc03 Makefile: upgrade go mod tidy compatibility mode from Go1.22 to Go1.23
This is a follow-up for 1b9f3b39b4
2024-09-08 21:01:41 +02:00
Aliaksandr Valialkin
563a50b547 Makefile: remove '-d' flag from 'go get', since this flag is deprecated
See https://go.dev/doc/go-get-install-deprecation
2024-09-08 20:11:21 +02:00
Aliaksandr Valialkin
4fbdde5852 deployment/docker: update base Alpine docker image from 3.20.2 to 3.20.3
See https://alpinelinux.org/posts/Alpine-3.17.10-3.18.9-3.19.4-3.20.3-released.html
2024-09-08 19:26:48 +02:00
Aliaksandr Valialkin
657988ac3a app/vlselect: consistently reuse the original query timestamp when executing /select/logsql/query with positive limit=N query arg
Previously the query could return incorrect results, since the query timestamp was updated with every Query.Clone() call
during iterative search for the time range with up to limit=N rows.

While at it, optimize queries, which find low number of matching logs, while spend a lot of CPU time for searching
across big number of logs. The optimization reduces the upper bound of the time range to search if the current time range
contains zero matching rows.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785
2024-09-08 14:32:23 +02:00
Aliaksandr Valialkin
45a3713bdb lib/logstorage: preserve the order of tokens to check against bloom filters in AND filters
Previously tokens from AND filters were extracted in random order. This could slow down
checking them agains bloom filters if the most specific tokens go at the beginning of the AND filters.
Preserve the original order of tokens when matching them against bloom filters,
so the user could control the performance of the query by putting the most specific AND filters
at the beginning of the query.

While at it, add tests for getCommonTokensForAndFilters() and getCommonTokensForOrFilters().

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6554
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6556
2024-09-08 12:27:30 +02:00
Aliaksandr Valialkin
eaee2d7db4 lib/logstorage: improve error logging for incorrect queries passed to /select/logsql/stats_query and /select/logsql/stats_query_range functions 2024-09-08 11:24:44 +02:00
Aliaksandr Valialkin
1cd06ace5a lib/logstorage: properly extract common tokens from unsupported OR filters
Previously the following query could miss rows matching !bar if these rows do not contain foo:

   foo OR !bar

This is because of incorrect detection of common tokens for OR filters - all the unsupported filters
were skipped (including the NOT filter (aka `!`)), while in this case zero common tokens must be returned.

While at it, move repetiteve code in TestFilterAnd and TestFilterOr into f function.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6554
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6556
2024-09-08 11:14:55 +02:00
Aliaksandr Valialkin
0a40064a6f app/vlselect: add /select/logsql/stats_query_range endpoint for building time series panels in VictoriaLogs plugin for Grafana
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6943
Updates https://github.com/VictoriaMetrics/victorialogs-datasource/issues/61
2024-09-07 00:41:47 +02:00
Aliaksandr Valialkin
c9bb4ddeed app/vlselect: add /select/logsql/stats_query endpoint, which is going to be used by vmalert
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6942
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6706
2024-09-06 23:06:43 +02:00
Aliaksandr Valialkin
5261a84119 deployment: update Go builder from Go1.23.0 to Go1.23.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.23.1+label%3ACherryPickApproved
2024-09-06 22:51:15 +02:00
Aliaksandr Valialkin
00e7d5add3 lib/logstorage: substitute | operator with or operator at math pipe
This is needed for avoiding confusion between the `|` operator at `math` pipe and `|` pipe delimiter.
For example, the following query was parsed unexpectedly:

   * | math foo / bar | fields x

as

   * | math foo / (bar | fields) as x

Substituting `|` with `or` inside `math` pipe fixes this ambiguity.
2024-09-06 22:44:14 +02:00
f41gh7
95acca6b52 app/*/multiarch: return back empty value for TARGETARCH
follow-up after 91456ab5bb

docker buildx uses special variables, such as TARGETARCH and it shouldn't be overwritten.

 See this article for details
https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-06 18:12:17 +02:00
f41gh7
feafb30266 docs/changelog: mention storage changes
After a5424e95b3

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-06 18:05:11 +02:00
Artem Fetishev
a5424e95b3 lib/storage: adds metrics that count records that failed to insert
### Describe Your Changes

Add storage metrics that count records that failed to insert:

- `RowsReceivedTotal`: the number of records that have been received by
the storage from the clients
- `RowsAddedTotal`: the number of records that have actually been
persisted. This value must be equal to `RowsReceivedTotal` if all the
records have been valid ones. But it will be smaller otherwise. The
values of the metrics below should provide the insight of why some
records hasn't been added
-   `NaNValueRows`: the number of records whose value was `NaN`
- `StaleNaNValueRows`: the number of records whose value was `Stale NaN`
- `InvalidRawMetricNames`: the number of records whose raw metric name
has failed to unmarshal.

The following metrics existed before this PR and are listed here for
completeness:

- `TooSmallTimestampRows`: the number of records whose timestamp is
negative or is older than retention period
- `TooBigTimestampRows`: the number of records whose timestamp is too
far in the future.
- `HourlySeriesLimitRowsDropped`: the number of records that have not
been added because the hourly series limit has been exceeded.
- `DailySeriesLimitRowsDropped`: the number of records that have not
been added because the daily series limit has been exceeded.

---
Signed-off-by: Artem Fetishev <wwctrsrx@gmail.com>
2024-09-06 17:57:21 +02:00
Zakhar Bessarab
9f7ee4c0bb Vmgateway no prefix string (#784)
* app/vmgateway: allow skipping Bearer prefix, parsing access as string

- allow disabling of "Bearer" prefix check - This is needed in order to support OIDC systems where identity token is provided separately from access token and it does not contain "Bearer" prefix(such as Azure Entra ID, ex AD).a

- support parsing "vm_access" claim as a string - This is helpful for systems where claims can only be mapped to string.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/changelog: mention vmgateway updates

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-09-06 16:20:20 +02:00
Aliaksandr Valialkin
0205170409 lib/logstorage: consistently use nsecsPerDay constant and remove nsecPerDay constant 2024-09-06 16:17:04 +02:00
1127 changed files with 96698 additions and 23561 deletions

View File

@@ -27,6 +27,7 @@ include package/release/Makefile
all: \
victoria-metrics-prod \
victoria-logs-prod \
vlogscli-prod \
vmagent-prod \
vmalert-prod \
vmalert-tool-prod \
@@ -51,6 +52,7 @@ publish: \
package: \
package-victoria-metrics \
package-victoria-logs \
package-vlogscli \
package-vmagent \
package-vmalert \
package-vmalert-tool \
@@ -320,6 +322,63 @@ release-victoria-logs-windows-goarch: victoria-logs-windows-$(GOARCH)-prod
cd bin && rm -rf \
victoria-logs-windows-$(GOARCH)-prod.exe
release-vlogscli:
$(MAKE_PARALLEL) release-vlogscli-linux-386 \
release-vlogscli-linux-amd64 \
release-vlogscli-linux-arm \
release-vlogscli-linux-arm64 \
release-vlogscli-darwin-amd64 \
release-vlogscli-darwin-arm64 \
release-vlogscli-freebsd-amd64 \
release-vlogscli-openbsd-amd64 \
release-vlogscli-windows-amd64
release-vlogscli-linux-386:
GOOS=linux GOARCH=386 $(MAKE) release-vlogscli-goos-goarch
release-vlogscli-linux-amd64:
GOOS=linux GOARCH=amd64 $(MAKE) release-vlogscli-goos-goarch
release-vlogscli-linux-arm:
GOOS=linux GOARCH=arm $(MAKE) release-vlogscli-goos-goarch
release-vlogscli-linux-arm64:
GOOS=linux GOARCH=arm64 $(MAKE) release-vlogscli-goos-goarch
release-vlogscli-darwin-amd64:
GOOS=darwin GOARCH=amd64 $(MAKE) release-vlogscli-goos-goarch
release-vlogscli-darwin-arm64:
GOOS=darwin GOARCH=arm64 $(MAKE) release-vlogscli-goos-goarch
release-vlogscli-freebsd-amd64:
GOOS=freebsd GOARCH=amd64 $(MAKE) release-vlogscli-goos-goarch
release-vlogscli-openbsd-amd64:
GOOS=openbsd GOARCH=amd64 $(MAKE) release-vlogscli-goos-goarch
release-vlogscli-windows-amd64:
GOARCH=amd64 $(MAKE) release-vlogscli-windows-goarch
release-vlogscli-goos-goarch: vlogscli-$(GOOS)-$(GOARCH)-prod
cd bin && \
tar $(TAR_OWNERSHIP) --transform="flags=r;s|-$(GOOS)-$(GOARCH)||" -czf vlogscli-$(GOOS)-$(GOARCH)-$(PKG_TAG).tar.gz \
vlogscli-$(GOOS)-$(GOARCH)-prod \
&& sha256sum vlogscli-$(GOOS)-$(GOARCH)-$(PKG_TAG).tar.gz \
vlogscli-$(GOOS)-$(GOARCH)-prod \
| sed s/-$(GOOS)-$(GOARCH)-prod/-prod/ > vlogscli-$(GOOS)-$(GOARCH)-$(PKG_TAG)_checksums.txt
cd bin && rm -rf vlogscli-$(GOOS)-$(GOARCH)-prod
release-vlogscli-windows-goarch: vlogscli-windows-$(GOARCH)-prod
cd bin && \
zip vlogscli-windows-$(GOARCH)-$(PKG_TAG).zip \
vlogscli-windows-$(GOARCH)-prod.exe \
&& sha256sum vlogscli-windows-$(GOARCH)-$(PKG_TAG).zip \
vlogscli-windows-$(GOARCH)-prod.exe \
> vlogscli-windows-$(GOARCH)-$(PKG_TAG)_checksums.txt
cd bin && rm -rf \
vlogscli-windows-$(GOARCH)-prod.exe
release-vmutils: \
release-vmutils-linux-386 \
release-vmutils-linux-amd64 \
@@ -467,9 +526,9 @@ benchmark-pure:
CGO_ENABLED=0 go test -bench=. ./app/...
vendor-update:
go get -u -d ./lib/...
go get -u -d ./app/...
go mod tidy -compat=1.22
go get -u ./lib/...
go get -u ./app/...
go mod tidy -compat=1.23
go mod vendor
app-local:

View File

@@ -95,30 +95,31 @@ If you like VictoriaMetrics and want to contribute, then please [read these docs
## VictoriaMetrics Logo
[Zip](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/VM_logo.zip) contains three folders with different image orientations (main color and inverted version).
The provided [ZIP file](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/VM_logo.zip) contains three folders with different logo orientations. Each folder includes the following file types:
Files included in each folder:
* JPEG: Preview files
* PNG: Preview files with transparent background
* AI: Adobe Illustrator files
* 2 JPEG Preview files
* 2 PNG Preview files with transparent background
* 2 EPS Adobe Illustrator EPS10 files
### VictoriaMetrics Logo Usage Guidelines
### Logo Usage Guidelines
#### Font
#### Font used
* Lato Black
* Lato Regular
* Font Used: Lato Black
* Download here: [Lato Font](https://fonts.google.com/specimen/Lato)
#### Color Palette
* HEX [#110f0f](https://www.color-hex.com/color/110f0f)
* HEX [#ffffff](https://www.color-hex.com/color/ffffff)
* Black [#000000](https://www.color-hex.com/color/000000)
* Purple [#4d0e82](https://www.color-hex.com/color/4d0e82)
* Orange [#ff2e00](https://www.color-hex.com/color/ff2e00)
* White [#ffffff](https://www.color-hex.com/color/ffffff)
### We kindly ask
### Logo Usage Rules
* Please don't use any other font instead of suggested.
* To keep enough clear space around the logo.
* Do not change spacing, alignment, or relative locations of the design elements.
* Do not change the proportions for any of the design elements or the design itself.
You may resize as needed but must retain all proportions.
* Only use the Lato Black font as specified.
* Maintain sufficient clear space around the logo for visibility.
* Do not modify the spacing, alignment, or positioning of design elements.
* You may resize the logo as needed, but ensure all proportions remain intact.
Thank you for your cooperation!

Binary file not shown.

View File

@@ -8,5 +8,5 @@ FROM $root_image
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
EXPOSE 9428
ENTRYPOINT ["/victoria-logs-prod"]
ARG TARGETARCH=non-existing
ARG TARGETARCH
COPY victoria-logs-linux-${TARGETARCH}-prod ./victoria-logs-prod

View File

@@ -8,5 +8,5 @@ FROM $root_image
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
EXPOSE 8428
ENTRYPOINT ["/victoria-metrics-prod"]
ARG TARGETARCH=non-existing
ARG TARGETARCH
COPY victoria-metrics-linux-${TARGETARCH}-prod ./victoria-metrics-prod

View File

@@ -6,9 +6,7 @@ import (
"flag"
"fmt"
"io"
"math"
"net/http"
"strconv"
"strings"
"time"
@@ -251,19 +249,8 @@ func parseElasticsearchTimestamp(s string) (int64, error) {
return 0, nil
}
if len(s) < len("YYYY-MM-DD") || s[len("YYYY")] != '-' {
// Try parsing timestamp in milliseconds
n, err := strconv.ParseInt(s, 10, 64)
if err != nil {
return 0, fmt.Errorf("cannot parse timestamp in milliseconds from %q: %w", s, err)
}
if n > int64(math.MaxInt64)/1e6 {
return 0, fmt.Errorf("too big timestamp in milliseconds: %d; mustn't exceed %d", n, int64(math.MaxInt64)/1e6)
}
if n < int64(math.MinInt64)/1e6 {
return 0, fmt.Errorf("too small timestamp in milliseconds: %d; must be bigger than %d", n, int64(math.MinInt64)/1e6)
}
n *= 1e6
return n, nil
// Try parsing timestamp in seconds or milliseconds
return insertutils.ParseUnixTimestamp(s)
}
if len(s) == len("YYYY-MM-DD") {
t, err := time.Parse("2006-01-02", s)

View File

@@ -76,17 +76,20 @@ func TestReadBulkRequest_Success(t *testing.T) {
data := `{"create":{"_index":"filebeat-8.8.0"}}
{"@timestamp":"2023-06-06T04:48:11.735Z","log":{"offset":71770,"file":{"path":"/var/log/auth.log"}},"message":"foobar"}
{"create":{"_index":"filebeat-8.8.0"}}
{"@timestamp":"2023-06-06T04:48:12.735Z","message":"baz"}
{"@timestamp":"2023-06-06 04:48:12.735+01:00","message":"baz"}
{"index":{"_index":"filebeat-8.8.0"}}
{"message":"xyz","@timestamp":"2023-06-06T04:48:13.735Z","x":"y"}
{"message":"xyz","@timestamp":"1686026893735","x":"y"}
{"create":{"_index":"filebeat-8.8.0"}}
{"message":"qwe rty","@timestamp":"1686026893"}
`
timeField := "@timestamp"
msgField := "message"
rowsExpected := 3
timestampsExpected := []int64{1686026891735000000, 1686026892735000000, 1686026893735000000}
rowsExpected := 4
timestampsExpected := []int64{1686026891735000000, 1686023292735000000, 1686026893735000000, 1686026893000000000}
resultExpected := `{"@timestamp":"","log.offset":"71770","log.file.path":"/var/log/auth.log","_msg":"foobar"}
{"@timestamp":"","_msg":"baz"}
{"_msg":"xyz","@timestamp":"","x":"y"}`
{"_msg":"xyz","@timestamp":"","x":"y"}
{"_msg":"qwe rty","@timestamp":""}`
f(data, timeField, msgField, rowsExpected, timestampsExpected, resultExpected)
}

View File

@@ -176,6 +176,22 @@ func (lmp *logMessageProcessor) AddRow(timestamp int64, fields []logstorage.Fiel
return
}
// _msg field must be non-empty according to VictoriaLogs data model.
// See https://docs.victoriametrics.com/victorialogs/keyconcepts/#message-field
msgExist := false
for i := range fields {
if fields[i].Name == "_msg" {
msgExist = len(fields[i].Value) > 0
break
}
}
if !msgExist {
rf := logstorage.RowFormatter(fields)
logger.Warnf("dropping log line without _msg field; %s", rf)
rowsDroppedTotalMsgNotValid.Inc()
return
}
lmp.lr.MustAdd(lmp.cp.TenantID, timestamp, fields)
if lmp.cp.Debug {
s := lmp.lr.GetRowString(0)
@@ -225,4 +241,5 @@ func (cp *CommonParams) NewLogMessageProcessor() LogMessageProcessor {
var (
rowsDroppedTotalDebug = metrics.NewCounter(`vl_rows_dropped_total{reason="debug"}`)
rowsDroppedTotalTooManyFields = metrics.NewCounter(`vl_rows_dropped_total{reason="too_many_fields"}`)
rowsDroppedTotalMsgNotValid = metrics.NewCounter(`vl_rows_dropped_total{reason="msg_not_exist"}`)
)

View File

@@ -2,6 +2,8 @@ package insertutils
import (
"fmt"
"math"
"strconv"
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logstorage"
@@ -19,15 +21,49 @@ func ExtractTimestampRFC3339NanoFromFields(timeField string, fields []logstorage
if f.Name != timeField {
continue
}
if f.Value == "" || f.Value == "0" {
return time.Now().UnixNano(), nil
}
nsecs, ok := logstorage.TryParseTimestampRFC3339Nano(f.Value)
if !ok {
return 0, fmt.Errorf("cannot unmarshal rfc3339 timestamp from %s=%q", timeField, f.Value)
nsecs, err := parseTimestamp(f.Value)
if err != nil {
return 0, fmt.Errorf("cannot parse timestamp from field %q: %s", timeField, err)
}
f.Value = ""
if nsecs == 0 {
nsecs = time.Now().UnixNano()
}
return nsecs, nil
}
return time.Now().UnixNano(), nil
}
func parseTimestamp(s string) (int64, error) {
if s == "" || s == "0" {
return time.Now().UnixNano(), nil
}
if len(s) <= len("YYYY") || s[len("YYYY")] != '-' {
return ParseUnixTimestamp(s)
}
nsecs, ok := logstorage.TryParseTimestampRFC3339Nano(s)
if !ok {
return 0, fmt.Errorf("cannot unmarshal rfc3339 timestamp %q", s)
}
return nsecs, nil
}
// ParseUnixTimestamp parses s as unix timestamp in either seconds or milliseconds and returns the parsed timestamp in nanoseconds.
func ParseUnixTimestamp(s string) (int64, error) {
n, err := strconv.ParseInt(s, 10, 64)
if err != nil {
return 0, fmt.Errorf("cannot parse unix timestamp from %q: %w", s, err)
}
if n < (1<<31) && n >= (-1<<31) {
// The timestamp is in seconds. Convert it to milliseconds
n *= 1e3
}
if n > int64(math.MaxInt64)/1e6 {
return 0, fmt.Errorf("too big timestamp in milliseconds: %d; mustn't exceed %d", n, int64(math.MaxInt64)/1e6)
}
if n < int64(math.MinInt64)/1e6 {
return 0, fmt.Errorf("too small timestamp in milliseconds: %d; must be bigger than %d", n, int64(math.MinInt64)/1e6)
}
n *= 1e6
return n, nil
}

View File

@@ -27,25 +27,41 @@ func TestExtractTimestampRFC3339NanoFromFields_Success(t *testing.T) {
}
}
// UTC time
f("time", []logstorage.Field{
{Name: "foo", Value: "bar"},
{Name: "time", Value: "2024-06-18T23:37:20Z"},
}, 1718753840000000000)
// Time with timezone
f("time", []logstorage.Field{
{Name: "foo", Value: "bar"},
{Name: "time", Value: "2024-06-18T23:37:20+08:00"},
}, 1718725040000000000)
// SQL datetime format
f("time", []logstorage.Field{
{Name: "foo", Value: "bar"},
{Name: "time", Value: "2024-06-18T23:37:20.123-05:30"},
{Name: "time", Value: "2024-06-18 23:37:20.123-05:30"},
}, 1718773640123000000)
// Time with nanosecond precision
f("time", []logstorage.Field{
{Name: "time", Value: "2024-06-18T23:37:20.123456789-05:30"},
{Name: "foo", Value: "bar"},
}, 1718773640123456789)
// Unix timestamp in milliseconds
f("time", []logstorage.Field{
{Name: "foo", Value: "bar"},
{Name: "time", Value: "1718773640123"},
}, 1718773640123000000)
// Unix timestamp in seconds
f("time", []logstorage.Field{
{Name: "foo", Value: "bar"},
{Name: "time", Value: "1718773640"},
}, 1718773640000000000)
}
func TestExtractTimestampRFC3339NanoFromFields_Error(t *testing.T) {
@@ -66,9 +82,6 @@ func TestExtractTimestampRFC3339NanoFromFields_Error(t *testing.T) {
f("foobar")
// no Z at the end
f("2024-06-18T23:37:20")
// incomplete time
f("2024-06-18")
f("2024-06-18T23:37")

View File

@@ -23,13 +23,13 @@ func TestProcessStreamInternal_Success(t *testing.T) {
}
data := `{"@timestamp":"2023-06-06T04:48:11.735Z","log":{"offset":71770,"file":{"path":"/var/log/auth.log"}},"message":"foobar"}
{"@timestamp":"2023-06-06T04:48:12.735Z","message":"baz"}
{"message":"xyz","@timestamp":"2023-06-06T04:48:13.735Z","x":"y"}
{"@timestamp":"2023-06-06T04:48:12.735+01:00","message":"baz"}
{"message":"xyz","@timestamp":"2023-06-06 04:48:13.735Z","x":"y"}
`
timeField := "@timestamp"
msgField := "message"
rowsExpected := 3
timestampsExpected := []int64{1686026891735000000, 1686026892735000000, 1686026893735000000}
timestampsExpected := []int64{1686026891735000000, 1686023292735000000, 1686026893735000000}
resultExpected := `{"@timestamp":"","log.offset":"71770","log.file.path":"/var/log/auth.log","_msg":"foobar"}
{"@timestamp":"","_msg":"baz"}
{"_msg":"xyz","@timestamp":"","x":"y"}`

109
app/vlogscli/Makefile Normal file
View File

@@ -0,0 +1,109 @@
# All these commands must run from repository root.
vlogscli:
APP_NAME=vlogscli $(MAKE) app-local
vlogscli-race:
APP_NAME=vlogscli RACE=-race $(MAKE) app-local
vlogscli-prod:
APP_NAME=vlogscli $(MAKE) app-via-docker
vlogscli-pure-prod:
APP_NAME=vlogscli $(MAKE) app-via-docker-pure
vlogscli-linux-amd64-prod:
APP_NAME=vlogscli $(MAKE) app-via-docker-linux-amd64
vlogscli-linux-arm-prod:
APP_NAME=vlogscli $(MAKE) app-via-docker-linux-arm
vlogscli-linux-arm64-prod:
APP_NAME=vlogscli $(MAKE) app-via-docker-linux-arm64
vlogscli-linux-ppc64le-prod:
APP_NAME=vlogscli $(MAKE) app-via-docker-linux-ppc64le
vlogscli-linux-386-prod:
APP_NAME=vlogscli $(MAKE) app-via-docker-linux-386
vlogscli-darwin-amd64-prod:
APP_NAME=vlogscli $(MAKE) app-via-docker-darwin-amd64
vlogscli-darwin-arm64-prod:
APP_NAME=vlogscli $(MAKE) app-via-docker-darwin-arm64
vlogscli-freebsd-amd64-prod:
APP_NAME=vlogscli $(MAKE) app-via-docker-freebsd-amd64
vlogscli-openbsd-amd64-prod:
APP_NAME=vlogscli $(MAKE) app-via-docker-openbsd-amd64
vlogscli-windows-amd64-prod:
APP_NAME=vlogscli $(MAKE) app-via-docker-windows-amd64
package-vlogscli:
APP_NAME=vlogscli $(MAKE) package-via-docker
package-vlogscli-pure:
APP_NAME=vlogscli $(MAKE) package-via-docker-pure
package-vlogscli-amd64:
APP_NAME=vlogscli $(MAKE) package-via-docker-amd64
package-vlogscli-arm:
APP_NAME=vlogscli $(MAKE) package-via-docker-arm
package-vlogscli-arm64:
APP_NAME=vlogscli $(MAKE) package-via-docker-arm64
package-vlogscli-ppc64le:
APP_NAME=vlogscli $(MAKE) package-via-docker-ppc64le
package-vlogscli-386:
APP_NAME=vlogscli $(MAKE) package-via-docker-386
publish-vlogscli:
APP_NAME=vlogscli $(MAKE) publish-via-docker
vlogscli-linux-amd64:
APP_NAME=vlogscli CGO_ENABLED=1 GOOS=linux GOARCH=amd64 $(MAKE) app-local-goos-goarch
vlogscli-linux-arm:
APP_NAME=vlogscli CGO_ENABLED=0 GOOS=linux GOARCH=arm $(MAKE) app-local-goos-goarch
vlogscli-linux-arm64:
APP_NAME=vlogscli CGO_ENABLED=0 GOOS=linux GOARCH=arm64 $(MAKE) app-local-goos-goarch
vlogscli-linux-ppc64le:
APP_NAME=vlogscli CGO_ENABLED=0 GOOS=linux GOARCH=ppc64le $(MAKE) app-local-goos-goarch
vlogscli-linux-s390x:
APP_NAME=vlogscli CGO_ENABLED=0 GOOS=linux GOARCH=s390x $(MAKE) app-local-goos-goarch
vlogscli-linux-loong64:
APP_NAME=vlogscli CGO_ENABLED=0 GOOS=linux GOARCH=loong64 $(MAKE) app-local-goos-goarch
vlogscli-linux-386:
APP_NAME=vlogscli CGO_ENABLED=0 GOOS=linux GOARCH=386 $(MAKE) app-local-goos-goarch
vlogscli-darwin-amd64:
APP_NAME=vlogscli CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 $(MAKE) app-local-goos-goarch
vlogscli-darwin-arm64:
APP_NAME=vlogscli CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 $(MAKE) app-local-goos-goarch
vlogscli-freebsd-amd64:
APP_NAME=vlogscli CGO_ENABLED=0 GOOS=freebsd GOARCH=amd64 $(MAKE) app-local-goos-goarch
vlogscli-openbsd-amd64:
APP_NAME=vlogscli CGO_ENABLED=0 GOOS=openbsd GOARCH=amd64 $(MAKE) app-local-goos-goarch
vlogscli-windows-amd64:
GOARCH=amd64 APP_NAME=vlogscli $(MAKE) app-local-windows-goarch
vlogscli-pure:
APP_NAME=vlogscli $(MAKE) app-local-pure
run-vlogscli:
APP_NAME=vlogscli $(MAKE) run-via-docker

5
app/vlogscli/README.md Normal file
View File

@@ -0,0 +1,5 @@
# vlogscli
Command-line utility for querying [VictoriaLogs](https://docs.victoriametrics.com/victorialogs/).
See [these docs](https://docs.victoriametrics.com/victorialogs/querying/vlogscli/).

View File

@@ -0,0 +1,6 @@
ARG base_image=non-existing
FROM $base_image
ENTRYPOINT ["/vlogscli-prod"]
ARG src_binary=non-existing
COPY $src_binary ./vlogscli-prod

View File

@@ -0,0 +1,73 @@
package main
import (
"encoding/json"
"fmt"
"io"
"sync"
)
type jsonPrettifier struct {
rOriginal io.ReadCloser
d *json.Decoder
pr *io.PipeReader
pw *io.PipeWriter
wg sync.WaitGroup
}
func newJSONPrettifier(r io.ReadCloser) *jsonPrettifier {
d := json.NewDecoder(r)
pr, pw := io.Pipe()
jp := &jsonPrettifier{
rOriginal: r,
d: d,
pr: pr,
pw: pw,
}
jp.wg.Add(1)
go func() {
defer jp.wg.Done()
err := jp.prettifyJSONLines()
jp.closePipesWithError(err)
}()
return jp
}
func (jp *jsonPrettifier) closePipesWithError(err error) {
_ = jp.pr.CloseWithError(err)
_ = jp.pw.CloseWithError(err)
}
func (jp *jsonPrettifier) prettifyJSONLines() error {
for jp.d.More() {
var v any
if err := jp.d.Decode(&v); err != nil {
return err
}
line, err := json.MarshalIndent(v, "", " ")
if err != nil {
panic(fmt.Errorf("BUG: cannot marshal %v to JSON: %w", v, err))
}
if _, err := fmt.Fprintf(jp.pw, "%s\n", line); err != nil {
return err
}
}
return nil
}
func (jp *jsonPrettifier) Close() error {
jp.closePipesWithError(io.ErrUnexpectedEOF)
err := jp.rOriginal.Close()
jp.wg.Wait()
return err
}
func (jp *jsonPrettifier) Read(p []byte) (int, error) {
return jp.pr.Read(p)
}

View File

@@ -0,0 +1,116 @@
package main
import (
"errors"
"fmt"
"io"
"os"
"os/exec"
"os/signal"
"sync"
"syscall"
"github.com/mattn/go-isatty"
)
func isTerminal() bool {
return isatty.IsTerminal(os.Stdout.Fd()) && isatty.IsTerminal(os.Stderr.Fd())
}
func readWithLess(r io.Reader) error {
if !isTerminal() {
// Just write everything to stdout if no terminal is available.
_, err := io.Copy(os.Stdout, r)
if err != nil && !isErrPipe(err) {
return fmt.Errorf("error when forwarding data to stdout: %w", err)
}
if err := os.Stdout.Sync(); err != nil {
return fmt.Errorf("cannot sync data to stdout: %w", err)
}
return nil
}
pr, pw, err := os.Pipe()
if err != nil {
return fmt.Errorf("cannot create pipe: %w", err)
}
defer func() {
_ = pr.Close()
_ = pw.Close()
}()
// Ignore Ctrl+C in the current process, so 'less' could handle it properly
cancel := ignoreSignals(os.Interrupt)
defer cancel()
// Start 'less' process
path, err := exec.LookPath("less")
if err != nil {
return fmt.Errorf("cannot find 'less' command: %w", err)
}
p, err := os.StartProcess(path, []string{"less", "-F", "-X"}, &os.ProcAttr{
Env: append(os.Environ(), "LESSCHARSET=utf-8"),
Files: []*os.File{pr, os.Stdout, os.Stderr},
})
if err != nil {
return fmt.Errorf("cannot start 'less' process: %w", err)
}
// Close pr after 'less' finishes in a parallel goroutine
// in order to unblock forwarding data to stopped 'less' below.
waitch := make(chan *os.ProcessState)
go func() {
// Wait for 'less' process to finish.
ps, err := p.Wait()
if err != nil {
fatalf("unexpected error when waiting for 'less' process: %w", err)
}
_ = pr.Close()
waitch <- ps
}()
// Forward data from r to 'less'
_, err = io.Copy(pw, r)
_ = pw.Sync()
_ = pw.Close()
// Wait until 'less' finished
ps := <-waitch
// Verify 'less' status.
if !ps.Success() {
return fmt.Errorf("'less' finished with unexpected code %d", ps.ExitCode())
}
if err != nil && !isErrPipe(err) {
return fmt.Errorf("error when forwarding data to 'less': %w", err)
}
return nil
}
func isErrPipe(err error) bool {
return errors.Is(err, syscall.EPIPE) || errors.Is(err, io.ErrClosedPipe)
}
func ignoreSignals(sigs ...os.Signal) func() {
ch := make(chan os.Signal, 1)
signal.Notify(ch, sigs...)
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
for {
_, ok := <-ch
if !ok {
return
}
}
}()
return func() {
signal.Stop(ch)
close(ch)
wg.Wait()
}
}

301
app/vlogscli/main.go Normal file
View File

@@ -0,0 +1,301 @@
package main
import (
"context"
"errors"
"flag"
"fmt"
"io"
"io/fs"
"net/http"
"net/url"
"os"
"os/signal"
"strconv"
"strings"
"syscall"
"time"
"github.com/ergochat/readline"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/buildinfo"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/envflag"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/flagutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logstorage"
)
var (
datasourceURL = flag.String("datasource.url", "http://localhost:9428/select/logsql/query", "URL for querying VictoriaLogs; "+
"see https://docs.victoriametrics.com/victorialogs/querying/#querying-logs")
historyFile = flag.String("historyFile", "vlogscli-history", "Path to file with command history")
header = flagutil.NewArrayString("header", "Optional header to pass in request -datasource.url in the form 'HeaderName: value'")
)
const (
firstLinePrompt = ";> "
nextLinePrompt = ""
)
func main() {
// Write flags and help message to stdout, since it is easier to grep or pipe.
flag.CommandLine.SetOutput(os.Stdout)
flag.Usage = usage
envflag.Parse()
buildinfo.Init()
logger.InitNoLogFlags()
hes, err := parseHeaders(*header)
if err != nil {
fatalf("cannot parse -header command-line flag: %s", err)
}
headers = hes
isEmptyLine := true
cfg := &readline.Config{
Prompt: firstLinePrompt,
DisableAutoSaveHistory: true,
Listener: func(line []rune, pos int, _ rune) ([]rune, int, bool) {
isEmptyLine = len(line) == 0
return line, pos, false
},
}
rl, err := readline.NewFromConfig(cfg)
if err != nil {
fatalf("cannot initialize readline: %s", err)
}
fmt.Fprintf(rl, "sending queries to %s\n", *datasourceURL)
runReadlineLoop(rl, &isEmptyLine)
if err := rl.Close(); err != nil {
fatalf("cannot close readline: %s", err)
}
}
func runReadlineLoop(rl *readline.Instance, isEmptyLine *bool) {
historyLines, err := loadFromHistory(*historyFile)
if err != nil {
fatalf("cannot load query history: %s", err)
}
for _, line := range historyLines {
if err := rl.SaveToHistory(line); err != nil {
fatalf("cannot initialize query history: %s", err)
}
}
s := ""
for {
line, err := rl.ReadLine()
if err != nil {
switch err {
case io.EOF:
if s != "" {
// This is non-interactive query execution.
if err := executeQuery(context.Background(), rl, s); err != nil {
fmt.Fprintf(rl, "%s\n", err)
}
}
return
case readline.ErrInterrupt:
if s == "" && *isEmptyLine {
fmt.Fprintf(rl, "interrupted\n")
os.Exit(128 + int(syscall.SIGINT))
}
// Default value for Ctrl+C - clear the prompt
s = ""
rl.SetPrompt(firstLinePrompt)
continue
default:
fatalf("unexpected error in readline: %s", err)
}
}
s += line
if isQuitCommand(s) {
fmt.Fprintf(rl, "bye!\n")
return
}
if s == "" {
// Skip empty lines
continue
}
if line != "" && !strings.HasSuffix(line, ";") {
// Assume the query is incomplete and allow the user finishing the query on the next line
s += "\n"
rl.SetPrompt(nextLinePrompt)
continue
}
// Execute the query
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
err = executeQuery(ctx, rl, s)
cancel()
if err != nil {
if errors.Is(err, context.Canceled) {
fmt.Fprintf(rl, "\n")
} else {
fmt.Fprintf(rl, "%s\n", err)
}
// Save queries in the history even if they weren't finished successfully
}
s = strings.TrimSpace(s)
if len(historyLines) == 0 || historyLines[len(historyLines)-1] != s {
historyLines = append(historyLines, s)
if len(historyLines) > 500 {
historyLines = historyLines[len(historyLines)-500:]
}
if err := saveToHistory(*historyFile, historyLines); err != nil {
fatalf("cannot save query history: %s", err)
}
}
if err := rl.SaveToHistory(s); err != nil {
fatalf("cannot update query history: %s", err)
}
s = ""
rl.SetPrompt(firstLinePrompt)
}
}
func loadFromHistory(filePath string) ([]string, error) {
data, err := os.ReadFile(filePath)
if err != nil {
if errors.Is(err, fs.ErrNotExist) {
return nil, nil
}
return nil, err
}
linesQuoted := strings.Split(string(data), "\n")
lines := make([]string, 0, len(linesQuoted))
i := 0
for _, lineQuoted := range linesQuoted {
i++
if lineQuoted == "" {
continue
}
line, err := strconv.Unquote(lineQuoted)
if err != nil {
return nil, fmt.Errorf("cannot parse line #%d at %s: %w; line: [%s]", i, filePath, err, line)
}
lines = append(lines, line)
}
return lines, nil
}
func saveToHistory(filePath string, lines []string) error {
linesQuoted := make([]string, len(lines))
for i, line := range lines {
lineQuoted := strconv.Quote(line)
linesQuoted[i] = lineQuoted
}
data := strings.Join(linesQuoted, "\n")
return os.WriteFile(filePath, []byte(data), 0600)
}
func isQuitCommand(s string) bool {
switch s {
case "q", "quit", "exit":
return true
default:
return false
}
}
func executeQuery(ctx context.Context, output io.Writer, s string) error {
// Parse the query and convert it to canonical view.
s = strings.TrimSuffix(s, ";")
q, err := logstorage.ParseQuery(s)
if err != nil {
return fmt.Errorf("cannot parse query: %w", err)
}
qStr := q.String()
fmt.Fprintf(output, "executing [%s]...", qStr)
// Prepare HTTP request for VictoriaLogs
args := make(url.Values)
args.Set("query", qStr)
data := strings.NewReader(args.Encode())
req, err := http.NewRequestWithContext(ctx, "POST", *datasourceURL, data)
if err != nil {
panic(fmt.Errorf("BUG: cannot prepare request to server: %w", err))
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
for _, h := range headers {
req.Header.Set(h.Name, h.Value)
}
// Execute HTTP request at VictoriaLogs
startTime := time.Now()
resp, err := httpClient.Do(req)
queryDuration := time.Since(startTime)
fmt.Fprintf(output, "; duration: %.3fs\n", queryDuration.Seconds())
if err != nil {
return fmt.Errorf("cannot execute query: %w", err)
}
defer func() {
_ = resp.Body.Close()
}()
if resp.StatusCode != http.StatusOK {
body, err := io.ReadAll(resp.Body)
if err != nil {
body = []byte(fmt.Sprintf("cannot read response body: %s", err))
}
return fmt.Errorf("unexpected status code: %d; response body:\n%s", resp.StatusCode, body)
}
// Prettify the response and stream it to 'less'.
jp := newJSONPrettifier(resp.Body)
defer func() {
_ = jp.Close()
}()
if err := readWithLess(jp); err != nil {
return fmt.Errorf("error when reading query response: %w", err)
}
return nil
}
var httpClient = &http.Client{}
var headers []headerEntry
type headerEntry struct {
Name string
Value string
}
func parseHeaders(a []string) ([]headerEntry, error) {
hes := make([]headerEntry, len(a))
for i, s := range a {
a := strings.SplitN(s, ":", 2)
if len(a) != 2 {
return nil, fmt.Errorf("cannot parse header=%q; it must contain at least one ':'; for example, 'Cookie: foo'", s)
}
hes[i] = headerEntry{
Name: strings.TrimSpace(a[0]),
Value: strings.TrimSpace(a[1]),
}
}
return hes, nil
}
func fatalf(format string, args ...any) {
fmt.Fprintf(os.Stderr, format+"\n", args...)
os.Exit(1)
}
func usage() {
const s = `
vlogscli is a command-line tool for querying VictoriaLogs.
See the docs at https://docs.victoriametrics.com/victorialogs/querying/vlogscli/
`
flagutil.Usage(s)
}

View File

@@ -0,0 +1,11 @@
# See https://medium.com/on-docker/use-multi-stage-builds-to-inject-ca-certs-ad1e8f01de1b
ARG certs_image=non-existing
ARG root_image=non-existing
FROM $certs_image AS certs
RUN apk update && apk upgrade && apk --update --no-cache add ca-certificates
FROM $root_image
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
ENTRYPOINT ["/vlogscli-prod"]
ARG TARGETARCH
COPY vlogscli-linux-${TARGETARCH}-prod ./vlogscli-prod

View File

@@ -5,6 +5,7 @@ import (
"fmt"
"math"
"net/http"
"slices"
"sort"
"strconv"
"strings"
@@ -44,6 +45,7 @@ func ProcessHitsRequest(ctx context.Context, w http.ResponseWriter, r *http.Requ
}
if step <= 0 {
httpserver.Errorf(w, r, "'step' must be bigger than zero")
return
}
// Obtain offset
@@ -380,6 +382,8 @@ func ProcessStreamsRequest(ctx context.Context, w http.ResponseWriter, r *http.R
}
// ProcessLiveTailRequest processes live tailing request to /select/logsq/tail
//
// See https://docs.victoriametrics.com/victorialogs/querying/#live-tailing
func ProcessLiveTailRequest(ctx context.Context, w http.ResponseWriter, r *http.Request) {
liveTailRequests.Inc()
defer liveTailRequests.Dec()
@@ -413,13 +417,17 @@ func ProcessLiveTailRequest(ctx context.Context, w http.ResponseWriter, r *http.
if !ok {
logger.Panicf("BUG: it is expected that http.ResponseWriter (%T) supports http.Flusher interface", w)
}
qOrig := q
for {
start := end - tailOffsetNsecs
end = time.Now().UnixNano()
qCopy := q.Clone()
qCopy.AddTimeFilter(start, end)
if err := vlstorage.RunQuery(ctxWithCancel, tenantIDs, qCopy, tp.writeBlock); err != nil {
q = qOrig.Clone(end)
q.AddTimeFilter(start, end)
// q.Optimize() call is needed for converting '*' into filterNoop.
// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785#issuecomment-2358547733
q.Optimize()
if err := vlstorage.RunQuery(ctxWithCancel, tenantIDs, q, tp.writeBlock); err != nil {
httpserver.Errorf(w, r, "cannot execute tail query [%s]: %s", q, err)
return
}
@@ -560,9 +568,212 @@ func (tp *tailProcessor) getTailRows() ([][]logstorage.Field, error) {
return tailRows, nil
}
// ProcessStatsQueryRangeRequest handles /select/logsql/stats_query_range request.
//
// See https://docs.victoriametrics.com/victorialogs/querying/#querying-log-range-stats
func ProcessStatsQueryRangeRequest(ctx context.Context, w http.ResponseWriter, r *http.Request) {
q, tenantIDs, err := parseCommonArgs(r)
if err != nil {
httpserver.SendPrometheusError(w, r, err)
return
}
// Obtain step
stepStr := r.FormValue("step")
if stepStr == "" {
stepStr = "1d"
}
step, err := promutils.ParseDuration(stepStr)
if err != nil {
err = fmt.Errorf("cannot parse 'step' arg: %s", err)
httpserver.SendPrometheusError(w, r, err)
return
}
if step <= 0 {
err := fmt.Errorf("'step' must be bigger than zero")
httpserver.SendPrometheusError(w, r, err)
return
}
// Obtain `by(...)` fields from the last `| stats` pipe in q.
// Add `_time:step` to the `by(...)` list.
byFields, err := q.GetStatsByFieldsAddGroupingByTime(int64(step))
if err != nil {
httpserver.SendPrometheusError(w, r, err)
return
}
q.Optimize()
m := make(map[string]*statsSeries)
var mLock sync.Mutex
writeBlock := func(_ uint, timestamps []int64, columns []logstorage.BlockColumn) {
clonedColumnNames := make([]string, len(columns))
for i, c := range columns {
clonedColumnNames[i] = strings.Clone(c.Name)
}
for i := range timestamps {
timestamp := q.GetTimestamp()
labels := make([]logstorage.Field, 0, len(byFields))
for j, c := range columns {
if c.Name == "_time" {
nsec, ok := logstorage.TryParseTimestampRFC3339Nano(c.Values[i])
if ok {
timestamp = nsec
continue
}
}
if slices.Contains(byFields, c.Name) {
labels = append(labels, logstorage.Field{
Name: clonedColumnNames[j],
Value: strings.Clone(c.Values[i]),
})
}
}
var dst []byte
for j, c := range columns {
if !slices.Contains(byFields, c.Name) {
name := clonedColumnNames[j]
dst = dst[:0]
dst = append(dst, name...)
dst = logstorage.MarshalFieldsToJSON(dst, labels)
key := string(dst)
p := statsPoint{
Timestamp: timestamp,
Value: strings.Clone(c.Values[i]),
}
mLock.Lock()
ss := m[key]
if ss == nil {
ss = &statsSeries{
key: key,
Name: name,
Labels: labels,
}
m[key] = ss
}
ss.Points = append(ss.Points, p)
mLock.Unlock()
}
}
}
}
if err := vlstorage.RunQuery(ctx, tenantIDs, q, writeBlock); err != nil {
err = fmt.Errorf("cannot execute query [%s]: %s", q, err)
httpserver.SendPrometheusError(w, r, err)
return
}
// Sort the collected stats by time
rows := make([]*statsSeries, 0, len(m))
for _, ss := range m {
points := ss.Points
sort.Slice(points, func(i, j int) bool {
return points[i].Timestamp < points[j].Timestamp
})
rows = append(rows, ss)
}
sort.Slice(rows, func(i, j int) bool {
return rows[i].key < rows[j].key
})
w.Header().Set("Content-Type", "application/json")
WriteStatsQueryRangeResponse(w, rows)
}
type statsSeries struct {
key string
Name string
Labels []logstorage.Field
Points []statsPoint
}
type statsPoint struct {
Timestamp int64
Value string
}
// ProcessStatsQueryRequest handles /select/logsql/stats_query request.
//
// See https://docs.victoriametrics.com/victorialogs/querying/#querying-log-stats
func ProcessStatsQueryRequest(ctx context.Context, w http.ResponseWriter, r *http.Request) {
q, tenantIDs, err := parseCommonArgs(r)
if err != nil {
httpserver.SendPrometheusError(w, r, err)
return
}
// Obtain `by(...)` fields from the last `| stats` pipe in q.
byFields, err := q.GetStatsByFields()
if err != nil {
httpserver.SendPrometheusError(w, r, err)
return
}
q.Optimize()
var rows []statsRow
var rowsLock sync.Mutex
timestamp := q.GetTimestamp()
writeBlock := func(_ uint, timestamps []int64, columns []logstorage.BlockColumn) {
clonedColumnNames := make([]string, len(columns))
for i, c := range columns {
clonedColumnNames[i] = strings.Clone(c.Name)
}
for i := range timestamps {
labels := make([]logstorage.Field, 0, len(byFields))
for j, c := range columns {
if slices.Contains(byFields, c.Name) {
labels = append(labels, logstorage.Field{
Name: clonedColumnNames[j],
Value: strings.Clone(c.Values[i]),
})
}
}
for j, c := range columns {
if !slices.Contains(byFields, c.Name) {
r := statsRow{
Name: clonedColumnNames[j],
Labels: labels,
Timestamp: timestamp,
Value: strings.Clone(c.Values[i]),
}
rowsLock.Lock()
rows = append(rows, r)
rowsLock.Unlock()
}
}
}
}
if err := vlstorage.RunQuery(ctx, tenantIDs, q, writeBlock); err != nil {
err = fmt.Errorf("cannot execute query [%s]: %s", q, err)
httpserver.SendPrometheusError(w, r, err)
return
}
w.Header().Set("Content-Type", "application/json")
WriteStatsQueryResponse(w, rows)
}
type statsRow struct {
Name string
Labels []logstorage.Field
Timestamp int64
Value string
}
// ProcessQueryRequest handles /select/logsql/query request.
//
// See https://docs.victoriametrics.com/victorialogs/querying/#http-api
// See https://docs.victoriametrics.com/victorialogs/querying/#querying-logs
func ProcessQueryRequest(ctx context.Context, w http.ResponseWriter, r *http.Request) {
q, tenantIDs, err := parseCommonArgs(r)
if err != nil {
@@ -637,6 +848,7 @@ func getLastNQueryResults(ctx context.Context, tenantIDs []logstorage.TenantID,
limitUpper := 2 * limit
q.AddPipeLimit(uint64(limitUpper))
q.Optimize()
rows, err := getQueryResultsWithLimit(ctx, tenantIDs, q, limitUpper)
if err != nil {
return nil, err
@@ -647,32 +859,62 @@ func getLastNQueryResults(ctx context.Context, tenantIDs []logstorage.TenantID,
return rows, nil
}
// Slow path - search for the time range containing up to limitUpper rows.
// Slow path - adjust time range for selecting up to limitUpper rows
start, end := q.GetFilterTimeRange()
d := end/2 - start/2
start += d
qOrig := q
for {
q = qOrig.Clone()
timestamp := qOrig.GetTimestamp()
q = qOrig.Clone(timestamp)
q.AddTimeFilter(start, end)
// q.Optimize() call is needed for converting '*' into filterNoop.
// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785#issuecomment-2358547733
q.Optimize()
rows, err := getQueryResultsWithLimit(ctx, tenantIDs, q, limitUpper)
if err != nil {
return nil, err
}
if len(rows) >= limit && len(rows) < limitUpper || d == 0 {
if d == 0 || start >= end {
// The [start ... end] time range equals one nanosecond.
// Just return up to limit rows.
if len(rows) > limit {
rows = rows[:limit]
}
return rows, nil
}
dLastBit := d & 1
d /= 2
if len(rows) >= limitUpper {
// The number of found rows on the [start ... end] time range exceeds limitUpper,
// so reduce the time range to [start+d ... end].
start += d
continue
}
if len(rows) >= limit {
// The number of found rows is in the range [limit ... limitUpper).
// This means that found rows contains the needed limit rows with the biggest timestamps.
rows = getLastNRows(rows, limit)
return rows, nil
}
lastBit := d & 1
d /= 2
if len(rows) > limit {
start += d
} else {
start -= d + lastBit
// The number of found rows on [start ... end] time range is below the limit.
// This means the time range doesn't cover the needed logs, so it must be extended.
if len(rows) == 0 {
// The [start ... end] time range doesn't contain any rows, so change it to [start-d ... start).
end = start - 1
start -= d + dLastBit
continue
}
// The number of found rows on [start ... end] time range is bigger than 0 but smaller than limit.
// Increase the time range to [start-d ... end].
start -= d + dLastBit
}
}
@@ -693,20 +935,25 @@ func getQueryResultsWithLimit(ctx context.Context, tenantIDs []logstorage.Tenant
var rows []row
var rowsLock sync.Mutex
writeBlock := func(_ uint, timestamps []int64, columns []logstorage.BlockColumn) {
rowsLock.Lock()
defer rowsLock.Unlock()
clonedColumnNames := make([]string, len(columns))
for i, c := range columns {
clonedColumnNames[i] = strings.Clone(c.Name)
}
for i, timestamp := range timestamps {
fields := make([]logstorage.Field, len(columns))
for j := range columns {
f := &fields[j]
f.Name = strings.Clone(columns[j].Name)
f.Name = clonedColumnNames[j]
f.Value = strings.Clone(columns[j].Values[i])
}
rowsLock.Lock()
rows = append(rows, row{
timestamp: timestamp,
fields: fields,
})
rowsLock.Unlock()
}
if len(rows) >= limit {
@@ -728,9 +975,23 @@ func parseCommonArgs(r *http.Request) (*logstorage.Query, []logstorage.TenantID,
}
tenantIDs := []logstorage.TenantID{tenantID}
// Parse optional time arg
timestamp, okTime, err := getTimeNsec(r, "time")
if err != nil {
return nil, nil, err
}
if !okTime {
// If time arg is missing, then evaluate query at the current timestamp
timestamp = time.Now().UnixNano()
}
// decrease timestamp by one nanosecond in order to avoid capturing logs belonging
// to the first nanosecond at the next period of time (month, week, day, hour, etc.)
timestamp--
// Parse query
qStr := r.FormValue("query")
q, err := logstorage.ParseQuery(qStr)
q, err := logstorage.ParseQueryAtTimestamp(qStr, timestamp)
if err != nil {
return nil, nil, fmt.Errorf("cannot parse query [%s]: %s", qStr, err)
}

View File

@@ -0,0 +1,52 @@
{% stripspace %}
// StatsQueryRangeResponse generates response for /select/logsql/stats_query_range
{% func StatsQueryRangeResponse(rows []*statsSeries) %}
{
"status":"success",
"data":{
"resultType":"matrix",
"result":[
{% if len(rows) > 0 %}
{%= formatStatsSeries(rows[0]) %}
{% code rows = rows[1:] %}
{% for i := range rows %}
,{%= formatStatsSeries(rows[i]) %}
{% endfor %}
{% endif %}
]
}
}
{% endfunc %}
{% func formatStatsSeries(ss *statsSeries) %}
{
"metric":{
"__name__":{%q= ss.Name %}
{% if len(ss.Labels) > 0 %}
{% for _, label := range ss.Labels %}
,{%q= label.Name %}:{%q= label.Value %}
{% endfor %}
{% endif %}
},
"values":[
{% code points := ss.Points %}
{% if len(points) > 0 %}
{%= formatStatsPoint(&points[0]) %}
{% code points = points[1:] %}
{% for i := range points %}
,{%= formatStatsPoint(&points[i]) %}
{% endfor %}
{% endif %}
]
}
{% endfunc %}
{% func formatStatsPoint(p *statsPoint) %}
[
{%f= float64(p.Timestamp)/1e9 %},
{%q= p.Value %}
]
{% endfunc %}
{% endstripspace %}

View File

@@ -0,0 +1,188 @@
// Code generated by qtc from "stats_query_range_response.qtpl". DO NOT EDIT.
// See https://github.com/valyala/quicktemplate for details.
// StatsQueryRangeResponse generates response for /select/logsql/stats_query_range
//line app/vlselect/logsql/stats_query_range_response.qtpl:4
package logsql
//line app/vlselect/logsql/stats_query_range_response.qtpl:4
import (
qtio422016 "io"
qt422016 "github.com/valyala/quicktemplate"
)
//line app/vlselect/logsql/stats_query_range_response.qtpl:4
var (
_ = qtio422016.Copy
_ = qt422016.AcquireByteBuffer
)
//line app/vlselect/logsql/stats_query_range_response.qtpl:4
func StreamStatsQueryRangeResponse(qw422016 *qt422016.Writer, rows []*statsSeries) {
//line app/vlselect/logsql/stats_query_range_response.qtpl:4
qw422016.N().S(`{"status":"success","data":{"resultType":"matrix","result":[`)
//line app/vlselect/logsql/stats_query_range_response.qtpl:10
if len(rows) > 0 {
//line app/vlselect/logsql/stats_query_range_response.qtpl:11
streamformatStatsSeries(qw422016, rows[0])
//line app/vlselect/logsql/stats_query_range_response.qtpl:12
rows = rows[1:]
//line app/vlselect/logsql/stats_query_range_response.qtpl:13
for i := range rows {
//line app/vlselect/logsql/stats_query_range_response.qtpl:13
qw422016.N().S(`,`)
//line app/vlselect/logsql/stats_query_range_response.qtpl:14
streamformatStatsSeries(qw422016, rows[i])
//line app/vlselect/logsql/stats_query_range_response.qtpl:15
}
//line app/vlselect/logsql/stats_query_range_response.qtpl:16
}
//line app/vlselect/logsql/stats_query_range_response.qtpl:16
qw422016.N().S(`]}}`)
//line app/vlselect/logsql/stats_query_range_response.qtpl:20
}
//line app/vlselect/logsql/stats_query_range_response.qtpl:20
func WriteStatsQueryRangeResponse(qq422016 qtio422016.Writer, rows []*statsSeries) {
//line app/vlselect/logsql/stats_query_range_response.qtpl:20
qw422016 := qt422016.AcquireWriter(qq422016)
//line app/vlselect/logsql/stats_query_range_response.qtpl:20
StreamStatsQueryRangeResponse(qw422016, rows)
//line app/vlselect/logsql/stats_query_range_response.qtpl:20
qt422016.ReleaseWriter(qw422016)
//line app/vlselect/logsql/stats_query_range_response.qtpl:20
}
//line app/vlselect/logsql/stats_query_range_response.qtpl:20
func StatsQueryRangeResponse(rows []*statsSeries) string {
//line app/vlselect/logsql/stats_query_range_response.qtpl:20
qb422016 := qt422016.AcquireByteBuffer()
//line app/vlselect/logsql/stats_query_range_response.qtpl:20
WriteStatsQueryRangeResponse(qb422016, rows)
//line app/vlselect/logsql/stats_query_range_response.qtpl:20
qs422016 := string(qb422016.B)
//line app/vlselect/logsql/stats_query_range_response.qtpl:20
qt422016.ReleaseByteBuffer(qb422016)
//line app/vlselect/logsql/stats_query_range_response.qtpl:20
return qs422016
//line app/vlselect/logsql/stats_query_range_response.qtpl:20
}
//line app/vlselect/logsql/stats_query_range_response.qtpl:22
func streamformatStatsSeries(qw422016 *qt422016.Writer, ss *statsSeries) {
//line app/vlselect/logsql/stats_query_range_response.qtpl:22
qw422016.N().S(`{"metric":{"__name__":`)
//line app/vlselect/logsql/stats_query_range_response.qtpl:25
qw422016.N().Q(ss.Name)
//line app/vlselect/logsql/stats_query_range_response.qtpl:26
if len(ss.Labels) > 0 {
//line app/vlselect/logsql/stats_query_range_response.qtpl:27
for _, label := range ss.Labels {
//line app/vlselect/logsql/stats_query_range_response.qtpl:27
qw422016.N().S(`,`)
//line app/vlselect/logsql/stats_query_range_response.qtpl:28
qw422016.N().Q(label.Name)
//line app/vlselect/logsql/stats_query_range_response.qtpl:28
qw422016.N().S(`:`)
//line app/vlselect/logsql/stats_query_range_response.qtpl:28
qw422016.N().Q(label.Value)
//line app/vlselect/logsql/stats_query_range_response.qtpl:29
}
//line app/vlselect/logsql/stats_query_range_response.qtpl:30
}
//line app/vlselect/logsql/stats_query_range_response.qtpl:30
qw422016.N().S(`},"values":[`)
//line app/vlselect/logsql/stats_query_range_response.qtpl:33
points := ss.Points
//line app/vlselect/logsql/stats_query_range_response.qtpl:34
if len(points) > 0 {
//line app/vlselect/logsql/stats_query_range_response.qtpl:35
streamformatStatsPoint(qw422016, &points[0])
//line app/vlselect/logsql/stats_query_range_response.qtpl:36
points = points[1:]
//line app/vlselect/logsql/stats_query_range_response.qtpl:37
for i := range points {
//line app/vlselect/logsql/stats_query_range_response.qtpl:37
qw422016.N().S(`,`)
//line app/vlselect/logsql/stats_query_range_response.qtpl:38
streamformatStatsPoint(qw422016, &points[i])
//line app/vlselect/logsql/stats_query_range_response.qtpl:39
}
//line app/vlselect/logsql/stats_query_range_response.qtpl:40
}
//line app/vlselect/logsql/stats_query_range_response.qtpl:40
qw422016.N().S(`]}`)
//line app/vlselect/logsql/stats_query_range_response.qtpl:43
}
//line app/vlselect/logsql/stats_query_range_response.qtpl:43
func writeformatStatsSeries(qq422016 qtio422016.Writer, ss *statsSeries) {
//line app/vlselect/logsql/stats_query_range_response.qtpl:43
qw422016 := qt422016.AcquireWriter(qq422016)
//line app/vlselect/logsql/stats_query_range_response.qtpl:43
streamformatStatsSeries(qw422016, ss)
//line app/vlselect/logsql/stats_query_range_response.qtpl:43
qt422016.ReleaseWriter(qw422016)
//line app/vlselect/logsql/stats_query_range_response.qtpl:43
}
//line app/vlselect/logsql/stats_query_range_response.qtpl:43
func formatStatsSeries(ss *statsSeries) string {
//line app/vlselect/logsql/stats_query_range_response.qtpl:43
qb422016 := qt422016.AcquireByteBuffer()
//line app/vlselect/logsql/stats_query_range_response.qtpl:43
writeformatStatsSeries(qb422016, ss)
//line app/vlselect/logsql/stats_query_range_response.qtpl:43
qs422016 := string(qb422016.B)
//line app/vlselect/logsql/stats_query_range_response.qtpl:43
qt422016.ReleaseByteBuffer(qb422016)
//line app/vlselect/logsql/stats_query_range_response.qtpl:43
return qs422016
//line app/vlselect/logsql/stats_query_range_response.qtpl:43
}
//line app/vlselect/logsql/stats_query_range_response.qtpl:45
func streamformatStatsPoint(qw422016 *qt422016.Writer, p *statsPoint) {
//line app/vlselect/logsql/stats_query_range_response.qtpl:45
qw422016.N().S(`[`)
//line app/vlselect/logsql/stats_query_range_response.qtpl:47
qw422016.N().F(float64(p.Timestamp) / 1e9)
//line app/vlselect/logsql/stats_query_range_response.qtpl:47
qw422016.N().S(`,`)
//line app/vlselect/logsql/stats_query_range_response.qtpl:48
qw422016.N().Q(p.Value)
//line app/vlselect/logsql/stats_query_range_response.qtpl:48
qw422016.N().S(`]`)
//line app/vlselect/logsql/stats_query_range_response.qtpl:50
}
//line app/vlselect/logsql/stats_query_range_response.qtpl:50
func writeformatStatsPoint(qq422016 qtio422016.Writer, p *statsPoint) {
//line app/vlselect/logsql/stats_query_range_response.qtpl:50
qw422016 := qt422016.AcquireWriter(qq422016)
//line app/vlselect/logsql/stats_query_range_response.qtpl:50
streamformatStatsPoint(qw422016, p)
//line app/vlselect/logsql/stats_query_range_response.qtpl:50
qt422016.ReleaseWriter(qw422016)
//line app/vlselect/logsql/stats_query_range_response.qtpl:50
}
//line app/vlselect/logsql/stats_query_range_response.qtpl:50
func formatStatsPoint(p *statsPoint) string {
//line app/vlselect/logsql/stats_query_range_response.qtpl:50
qb422016 := qt422016.AcquireByteBuffer()
//line app/vlselect/logsql/stats_query_range_response.qtpl:50
writeformatStatsPoint(qb422016, p)
//line app/vlselect/logsql/stats_query_range_response.qtpl:50
qs422016 := string(qb422016.B)
//line app/vlselect/logsql/stats_query_range_response.qtpl:50
qt422016.ReleaseByteBuffer(qb422016)
//line app/vlselect/logsql/stats_query_range_response.qtpl:50
return qs422016
//line app/vlselect/logsql/stats_query_range_response.qtpl:50
}

View File

@@ -0,0 +1,36 @@
{% stripspace %}
// StatsQueryResponse generates response for /select/logsql/stats_query
{% func StatsQueryResponse(rows []statsRow) %}
{
"status":"success",
"data":{
"resultType":"vector",
"result":[
{% if len(rows) > 0 %}
{%= formatStatsRow(&rows[0]) %}
{% code rows = rows[1:] %}
{% for i := range rows %}
,{%= formatStatsRow(&rows[i]) %}
{% endfor %}
{% endif %}
]
}
}
{% endfunc %}
{% func formatStatsRow(r *statsRow) %}
{
"metric":{
"__name__":{%q= r.Name %}
{% if len(r.Labels) > 0 %}
{% for _, label := range r.Labels %}
,{%q= label.Name %}:{%q= label.Value %}
{% endfor %}
{% endif %}
},
"value":[{%f= float64(r.Timestamp)/1e9 %},{%q= r.Value %}]
}
{% endfunc %}
{% endstripspace %}

View File

@@ -0,0 +1,133 @@
// Code generated by qtc from "stats_query_response.qtpl". DO NOT EDIT.
// See https://github.com/valyala/quicktemplate for details.
// StatsQueryResponse generates response for /select/logsql/stats_query
//line app/vlselect/logsql/stats_query_response.qtpl:4
package logsql
//line app/vlselect/logsql/stats_query_response.qtpl:4
import (
qtio422016 "io"
qt422016 "github.com/valyala/quicktemplate"
)
//line app/vlselect/logsql/stats_query_response.qtpl:4
var (
_ = qtio422016.Copy
_ = qt422016.AcquireByteBuffer
)
//line app/vlselect/logsql/stats_query_response.qtpl:4
func StreamStatsQueryResponse(qw422016 *qt422016.Writer, rows []statsRow) {
//line app/vlselect/logsql/stats_query_response.qtpl:4
qw422016.N().S(`{"status":"success","data":{"resultType":"vector","result":[`)
//line app/vlselect/logsql/stats_query_response.qtpl:10
if len(rows) > 0 {
//line app/vlselect/logsql/stats_query_response.qtpl:11
streamformatStatsRow(qw422016, &rows[0])
//line app/vlselect/logsql/stats_query_response.qtpl:12
rows = rows[1:]
//line app/vlselect/logsql/stats_query_response.qtpl:13
for i := range rows {
//line app/vlselect/logsql/stats_query_response.qtpl:13
qw422016.N().S(`,`)
//line app/vlselect/logsql/stats_query_response.qtpl:14
streamformatStatsRow(qw422016, &rows[i])
//line app/vlselect/logsql/stats_query_response.qtpl:15
}
//line app/vlselect/logsql/stats_query_response.qtpl:16
}
//line app/vlselect/logsql/stats_query_response.qtpl:16
qw422016.N().S(`]}}`)
//line app/vlselect/logsql/stats_query_response.qtpl:20
}
//line app/vlselect/logsql/stats_query_response.qtpl:20
func WriteStatsQueryResponse(qq422016 qtio422016.Writer, rows []statsRow) {
//line app/vlselect/logsql/stats_query_response.qtpl:20
qw422016 := qt422016.AcquireWriter(qq422016)
//line app/vlselect/logsql/stats_query_response.qtpl:20
StreamStatsQueryResponse(qw422016, rows)
//line app/vlselect/logsql/stats_query_response.qtpl:20
qt422016.ReleaseWriter(qw422016)
//line app/vlselect/logsql/stats_query_response.qtpl:20
}
//line app/vlselect/logsql/stats_query_response.qtpl:20
func StatsQueryResponse(rows []statsRow) string {
//line app/vlselect/logsql/stats_query_response.qtpl:20
qb422016 := qt422016.AcquireByteBuffer()
//line app/vlselect/logsql/stats_query_response.qtpl:20
WriteStatsQueryResponse(qb422016, rows)
//line app/vlselect/logsql/stats_query_response.qtpl:20
qs422016 := string(qb422016.B)
//line app/vlselect/logsql/stats_query_response.qtpl:20
qt422016.ReleaseByteBuffer(qb422016)
//line app/vlselect/logsql/stats_query_response.qtpl:20
return qs422016
//line app/vlselect/logsql/stats_query_response.qtpl:20
}
//line app/vlselect/logsql/stats_query_response.qtpl:22
func streamformatStatsRow(qw422016 *qt422016.Writer, r *statsRow) {
//line app/vlselect/logsql/stats_query_response.qtpl:22
qw422016.N().S(`{"metric":{"__name__":`)
//line app/vlselect/logsql/stats_query_response.qtpl:25
qw422016.N().Q(r.Name)
//line app/vlselect/logsql/stats_query_response.qtpl:26
if len(r.Labels) > 0 {
//line app/vlselect/logsql/stats_query_response.qtpl:27
for _, label := range r.Labels {
//line app/vlselect/logsql/stats_query_response.qtpl:27
qw422016.N().S(`,`)
//line app/vlselect/logsql/stats_query_response.qtpl:28
qw422016.N().Q(label.Name)
//line app/vlselect/logsql/stats_query_response.qtpl:28
qw422016.N().S(`:`)
//line app/vlselect/logsql/stats_query_response.qtpl:28
qw422016.N().Q(label.Value)
//line app/vlselect/logsql/stats_query_response.qtpl:29
}
//line app/vlselect/logsql/stats_query_response.qtpl:30
}
//line app/vlselect/logsql/stats_query_response.qtpl:30
qw422016.N().S(`},"value":[`)
//line app/vlselect/logsql/stats_query_response.qtpl:32
qw422016.N().F(float64(r.Timestamp) / 1e9)
//line app/vlselect/logsql/stats_query_response.qtpl:32
qw422016.N().S(`,`)
//line app/vlselect/logsql/stats_query_response.qtpl:32
qw422016.N().Q(r.Value)
//line app/vlselect/logsql/stats_query_response.qtpl:32
qw422016.N().S(`]}`)
//line app/vlselect/logsql/stats_query_response.qtpl:34
}
//line app/vlselect/logsql/stats_query_response.qtpl:34
func writeformatStatsRow(qq422016 qtio422016.Writer, r *statsRow) {
//line app/vlselect/logsql/stats_query_response.qtpl:34
qw422016 := qt422016.AcquireWriter(qq422016)
//line app/vlselect/logsql/stats_query_response.qtpl:34
streamformatStatsRow(qw422016, r)
//line app/vlselect/logsql/stats_query_response.qtpl:34
qt422016.ReleaseWriter(qw422016)
//line app/vlselect/logsql/stats_query_response.qtpl:34
}
//line app/vlselect/logsql/stats_query_response.qtpl:34
func formatStatsRow(r *statsRow) string {
//line app/vlselect/logsql/stats_query_response.qtpl:34
qb422016 := qt422016.AcquireByteBuffer()
//line app/vlselect/logsql/stats_query_response.qtpl:34
writeformatStatsRow(qb422016, r)
//line app/vlselect/logsql/stats_query_response.qtpl:34
qs422016 := string(qb422016.B)
//line app/vlselect/logsql/stats_query_response.qtpl:34
qt422016.ReleaseByteBuffer(qb422016)
//line app/vlselect/logsql/stats_query_response.qtpl:34
return qs422016
//line app/vlselect/logsql/stats_query_response.qtpl:34
}

View File

@@ -193,6 +193,14 @@ func processSelectRequest(ctx context.Context, w http.ResponseWriter, r *http.Re
logsqlQueryRequests.Inc()
logsql.ProcessQueryRequest(ctx, w, r)
return true
case "/select/logsql/stats_query":
logsqlStatsQueryRequests.Inc()
logsql.ProcessStatsQueryRequest(ctx, w, r)
return true
case "/select/logsql/stats_query_range":
logsqlStatsQueryRangeRequests.Inc()
logsql.ProcessStatsQueryRangeRequest(ctx, w, r)
return true
case "/select/logsql/stream_field_names":
logsqlStreamFieldNamesRequests.Inc()
logsql.ProcessStreamFieldNamesRequest(ctx, w, r)
@@ -232,6 +240,8 @@ var (
logsqlFieldValuesRequests = metrics.NewCounter(`vl_http_requests_total{path="/select/logsql/field_values"}`)
logsqlHitsRequests = metrics.NewCounter(`vl_http_requests_total{path="/select/logsql/hits"}`)
logsqlQueryRequests = metrics.NewCounter(`vl_http_requests_total{path="/select/logsql/query"}`)
logsqlStatsQueryRequests = metrics.NewCounter(`vl_http_requests_total{path="/select/logsql/stats_query"}`)
logsqlStatsQueryRangeRequests = metrics.NewCounter(`vl_http_requests_total{path="/select/logsql/stats_query_range"}`)
logsqlStreamFieldNamesRequests = metrics.NewCounter(`vl_http_requests_total{path="/select/logsql/stream_field_names"}`)
logsqlStreamFieldValuesRequests = metrics.NewCounter(`vl_http_requests_total{path="/select/logsql/stream_field_values"}`)
logsqlStreamIDsRequests = metrics.NewCounter(`vl_http_requests_total{path="/select/logsql/stream_ids"}`)

Binary file not shown.

Before

Width:  |  Height:  |  Size: 14 KiB

View File

@@ -1,13 +1,13 @@
{
"files": {
"main.css": "./static/css/main.c9cc37dd.css",
"main.js": "./static/js/main.7ae2e2c4.js",
"static/js/685.bebe1265.chunk.js": "./static/js/685.bebe1265.chunk.js",
"static/media/MetricsQL.md": "./static/media/MetricsQL.8c2e588d62b87f90dbf0.md",
"main.css": "./static/css/main.cbbca000.css",
"main.js": "./static/js/main.3d2eb957.js",
"static/js/685.f772060c.chunk.js": "./static/js/685.f772060c.chunk.js",
"static/media/MetricsQL.md": "./static/media/MetricsQL.a00044c91d9781cf8557.md",
"index.html": "./index.html"
},
"entrypoints": [
"static/css/main.c9cc37dd.css",
"static/js/main.7ae2e2c4.js"
"static/css/main.cbbca000.css",
"static/js/main.3d2eb957.js"
]
}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.6 KiB

View File

@@ -0,0 +1 @@
<svg width="48" height="48" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M24.5475 0C10.3246.0265251 1.11379 3.06365 4.40623 6.10077c0 0 12.32997 11.23333 16.58217 14.84083.8131.6896 2.1728 1.1936 3.5191 1.2201h.1199c1.3463-.0265 2.706-.5305 3.5191-1.2201 4.2522-3.5942 16.5422-14.84083 16.5422-14.84083C48.0478 3.06365 38.8636.0265251 24.6674 0" fill="#020202"/><path d="M28.1579 27.0159c-.8131.6896-2.1728 1.1936-3.5191 1.2201h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2201-2.9725-2.5067-13.35639-11.87-17.26201-15.3979v5.4112c0 .5968.22661 1.3793.6265 1.7506C7.00358 21.1936 17.2675 30.5437 20.9731 33.6737c.8132.6896 2.1728 1.1936 3.5191 1.2201h.12c1.3463-.0265 2.7059-.5305 3.519-1.2201 3.679-3.13 13.9429-12.4536 16.6089-14.8939.4132-.3713.6265-1.1538.6265-1.7506V11.618c-3.9323 3.5411-14.3162 12.931-17.2354 15.3979h.0267Z" fill="#020202"/><path d="M28.1579 39.748c-.8131.6897-2.1728 1.1937-3.5191 1.2202h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2202-2.9725-2.4933-13.35639-11.8567-17.26201-15.3978v5.4111c0 .5969.22661 1.3793.6265 1.7507C7.00358 33.9258 17.2675 43.2759 20.9731 46.4058c.8132.6897 2.1728 1.1937 3.5191 1.2202h.12c1.3463-.0265 2.7059-.5305 3.519-1.2202 3.679-3.1299 13.9429-12.4535 16.6089-14.8938.4132-.3714.6265-1.1538.6265-1.7507v-5.4111c-3.9323 3.5411-14.3162 12.931-17.2354 15.3978h.0267Z" fill="#020202"/></svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

View File

@@ -1 +1 @@
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="UI for VictoriaMetrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><meta name="twitter:card" content="summary_large_image"><meta name="twitter:image" content="./preview.jpg"><meta name="twitter:title" content="UI for VictoriaMetrics"><meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta name="twitter:site" content="@VictoriaMetrics"><meta property="og:title" content="Metric explorer for VictoriaMetrics"><meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta property="og:image" content="./preview.jpg"><meta property="og:type" content="website"><script defer="defer" src="./static/js/main.7ae2e2c4.js"></script><link href="./static/css/main.c9cc37dd.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.svg"/><link rel="apple-touch-icon" href="./favicon.svg"/><link rel="mask-icon" href="./favicon.svg" color="#000000"><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="Explore your log data with VictoriaLogs UI"/><link rel="manifest" href="./manifest.json"/><title>UI for VictoriaLogs</title><meta name="twitter:card" content="summary"><meta name="twitter:title" content="UI for VictoriaLogs"><meta name="twitter:site" content="@https://victoriametrics.com/products/victorialogs/"><meta name="twitter:description" content="Explore your log data with VictoriaLogs UI"><meta name="twitter:image" content="./preview.jpg"><meta property="og:type" content="website"><meta property="og:title" content="UI for VictoriaLogs"><meta property="og:url" content="https://victoriametrics.com/products/victorialogs/"><meta property="og:description" content="Explore your log data with VictoriaLogs UI"><script defer="defer" src="./static/js/main.3d2eb957.js"></script><link href="./static/css/main.cbbca000.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>

View File

@@ -1,16 +1,11 @@
{
"short_name": "Victoria Metrics UI",
"name": "Victoria Metrics UI is a metric explorer for Victoria Metrics",
"short_name": "vmui",
"name": "vmui",
"icons": [
{
"src": "favicon-32x32.png",
"sizes": "32x32",
"type": "image/png"
},
{
"src": "apple-touch-icon.png",
"type": "image/png",
"sizes": "192x192"
"src": "favicon.svg",
"sizes": "any",
"type": "image/svg+xml"
}
],
"start_url": ".",

Binary file not shown.

Before

Width:  |  Height:  |  Size: 67 KiB

After

Width:  |  Height:  |  Size: 6.5 KiB

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -5,7 +5,7 @@
*/
/**
* @remix-run/router v1.17.0
* @remix-run/router v1.19.2
*
* Copyright (c) Remix Software Inc.
*
@@ -16,7 +16,7 @@
*/
/**
* React Router DOM v6.24.0
* React Router DOM v6.26.2
*
* Copyright (c) Remix Software Inc.
*
@@ -27,7 +27,7 @@
*/
/**
* React Router v6.24.0
* React Router v6.26.2
*
* Copyright (c) Remix Software Inc.
*

File diff suppressed because one or more lines are too long

View File

@@ -103,7 +103,7 @@ The list of MetricsQL features on top of PromQL:
* Trailing commas on all the lists are allowed - label filters, function args and with expressions.
For instance, the following queries are valid: `m{foo="bar",}`, `f(a, b,)`, `WITH (x=y,) x`.
This simplifies maintenance of multi-line queries.
* Metric names and label names may contain any unicode letter. For example `температура{город="Київ"}` is a valid MetricsQL expression.
* Metric names and label names may contain any unicode letter. For example `ტემპერატურა{πόλη="Київ"}` is a valid MetricsQL expression.
* Metric names and labels names may contain escaped chars. For example, `foo\-bar{baz\=aa="b"}` is valid expression.
It returns time series with name `foo-bar` containing label `baz=aa` with value `b`.
Additionally, the following escape sequences are supported:

View File

@@ -1,56 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<link rel="icon" href="%PUBLIC_URL%/favicon.ico"/>
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=5"/>
<meta name="theme-color" content="#000000"/>
<meta
name="description"
content="UI for VictoriaMetrics"
/>
<link rel="apple-touch-icon" href="%PUBLIC_URL%/apple-touch-icon.png"/>
<link rel="icon" type="image/png" sizes="32x32" href="%PUBLIC_URL%/favicon-32x32.png">
<!--
manifest.json provides metadata used when your web app is installed on a
user's mobile device or desktop. See https://developers.google.com/web/fundamentals/web-app-manifest/
-->
<link rel="manifest" href="%PUBLIC_URL%/manifest.json"/>
<!--
Notice the use of %PUBLIC_URL% in the tags above.
It will be replaced with the URL of the `public` folder during the build.
Only files inside the `public` folder can be referenced from the HTML.
Unlike "/favicon.ico" or "favicon.ico", "%PUBLIC_URL%/favicon.ico" will
work correctly both with client-side routing and a non-root public URL.
Learn how to configure a non-root public URL by running `npm run build`.
-->
<title>VM UI</title>
<script src="%PUBLIC_URL%/dashboards/index.js" type="module"></script>
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:image" content="%PUBLIC_URL%/preview.jpg">
<meta name="twitter:title" content="UI for VictoriaMetrics">
<meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data">
<meta name="twitter:site" content="@VictoriaMetrics">
<meta property="og:title" content="Metric explorer for VictoriaMetrics">
<meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data">
<meta property="og:image" content="%PUBLIC_URL%/preview.jpg">
<meta property="og:type" content="website">
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div id="root"></div>
<!--
This HTML file is a template.
If you open it directly in the browser, you will see an empty page.
You can add webfonts, meta tags, or analytics to this file.
The build step will place the bundled scripts into the <body> tag.
To begin the development, run `npm start` or `yarn start`.
To create a production bundle, use `npm run build` or `yarn build`.
-->
</body>
</html>

View File

@@ -1,56 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<link rel="icon" href="%PUBLIC_URL%/favicon.ico"/>
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=5"/>
<meta name="theme-color" content="#000000"/>
<meta
name="description"
content="UI for VictoriaMetrics"
/>
<link rel="apple-touch-icon" href="%PUBLIC_URL%/apple-touch-icon.png"/>
<link rel="icon" type="image/png" sizes="32x32" href="%PUBLIC_URL%/favicon-32x32.png">
<!--
manifest.json provides metadata used when your web app is installed on a
user's mobile device or desktop. See https://developers.google.com/web/fundamentals/web-app-manifest/
-->
<link rel="manifest" href="%PUBLIC_URL%/manifest.json"/>
<!--
Notice the use of %PUBLIC_URL% in the tags above.
It will be replaced with the URL of the `public` folder during the build.
Only files inside the `public` folder can be referenced from the HTML.
Unlike "/favicon.ico" or "favicon.ico", "%PUBLIC_URL%/favicon.ico" will
work correctly both with client-side routing and a non-root public URL.
Learn how to configure a non-root public URL by running `npm run build`.
-->
<title>VM UI</title>
<script src="%PUBLIC_URL%/dashboards/index.js" type="module"></script>
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:image" content="%PUBLIC_URL%/preview.jpg">
<meta name="twitter:title" content="UI for VictoriaMetrics">
<meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data">
<meta name="twitter:site" content="@VictoriaMetrics">
<meta property="og:title" content="Metric explorer for VictoriaMetrics">
<meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data">
<meta property="og:image" content="%PUBLIC_URL%/preview.jpg">
<meta property="og:type" content="website">
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div id="root"></div>
<!--
This HTML file is a template.
If you open it directly in the browser, you will see an empty page.
You can add webfonts, meta tags, or analytics to this file.
The build step will place the bundled scripts into the <body> tag.
To begin the development, run `npm start` or `yarn start`.
To create a production bundle, use `npm run build` or `yarn build`.
-->
</body>
</html>

View File

@@ -8,5 +8,5 @@ FROM $root_image
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
EXPOSE 8429
ENTRYPOINT ["/vmagent-prod"]
ARG TARGETARCH=non-existing
ARG TARGETARCH
COPY vmagent-linux-${TARGETARCH}-prod ./vmagent-prod

View File

@@ -7,12 +7,11 @@ import (
"io"
"net/http"
"net/url"
"strconv"
"strings"
"sync"
"time"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/awsapi"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/flagutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
@@ -23,6 +22,7 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/lib/ratelimiter"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/timerpool"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/timeutil"
"github.com/VictoriaMetrics/metrics"
)
var (
@@ -463,10 +463,10 @@ again:
// Unexpected status code returned
retriesCount++
retryDuration *= 2
if retryDuration > maxRetryDuration {
retryDuration = maxRetryDuration
}
retryAfterHeader := parseRetryAfterHeader(resp.Header.Get("Retry-After"))
retryDuration = getRetryDuration(retryAfterHeader, retryDuration, maxRetryDuration)
// Handle response
body, err := io.ReadAll(resp.Body)
_ = resp.Body.Close()
if err != nil {
@@ -488,3 +488,49 @@ again:
}
var remoteWriteRejectedLogger = logger.WithThrottler("remoteWriteRejected", 5*time.Second)
// getRetryDuration returns retry duration.
// retryAfterDuration has the highest priority.
// If retryAfterDuration is not specified, retryDuration gets doubled.
// retryDuration can't exceed maxRetryDuration.
//
// Also see: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6097
func getRetryDuration(retryAfterDuration, retryDuration, maxRetryDuration time.Duration) time.Duration {
// retryAfterDuration has the highest priority duration
if retryAfterDuration > 0 {
return timeutil.AddJitterToDuration(retryAfterDuration)
}
// default backoff retry policy
retryDuration *= 2
if retryDuration > maxRetryDuration {
retryDuration = maxRetryDuration
}
return retryDuration
}
// parseRetryAfterHeader parses `Retry-After` value retrieved from HTTP response header.
// retryAfterString should be in either HTTP-date or a number of seconds.
// It will return time.Duration(0) if `retryAfterString` does not follow RFC 7231.
func parseRetryAfterHeader(retryAfterString string) (retryAfterDuration time.Duration) {
if retryAfterString == "" {
return retryAfterDuration
}
defer func() {
v := retryAfterDuration.Seconds()
logger.Infof("'Retry-After: %s' parsed into %.2f second(s)", retryAfterString, v)
}()
// Retry-After could be in "Mon, 02 Jan 2006 15:04:05 GMT" format.
if parsedTime, err := time.Parse(http.TimeFormat, retryAfterString); err == nil {
return time.Duration(time.Until(parsedTime).Seconds()) * time.Second
}
// Retry-After could be in seconds.
if seconds, err := strconv.Atoi(retryAfterString); err == nil {
return time.Duration(seconds) * time.Second
}
return 0
}

View File

@@ -0,0 +1,99 @@
package remotewrite
import (
"math"
"net/http"
"testing"
"time"
)
func TestCalculateRetryDuration(t *testing.T) {
// `testFunc` call `calculateRetryDuration` for `n` times
// and evaluate if the result of `calculateRetryDuration` is
// 1. >= expectMinDuration
// 2. <= expectMinDuration + 10% (see timeutil.AddJitterToDuration)
f := func(retryAfterDuration, retryDuration time.Duration, n int, expectMinDuration time.Duration) {
t.Helper()
for i := 0; i < n; i++ {
retryDuration = getRetryDuration(retryAfterDuration, retryDuration, time.Minute)
}
expectMaxDuration := helper(expectMinDuration)
expectMinDuration = expectMinDuration - (1000 * time.Millisecond) // Avoid edge case when calculating time.Until(now)
if !(retryDuration >= expectMinDuration && retryDuration <= expectMaxDuration) {
t.Fatalf(
"incorrect retry duration, want (ms): [%d, %d], got (ms): %d",
expectMinDuration.Milliseconds(), expectMaxDuration.Milliseconds(),
retryDuration.Milliseconds(),
)
}
}
// Call calculateRetryDuration for 1 time.
{
// default backoff policy
f(0, time.Second, 1, 2*time.Second)
// default backoff policy exceed max limit"
f(0, 10*time.Minute, 1, time.Minute)
// retry after > default backoff policy
f(10*time.Second, 1*time.Second, 1, 10*time.Second)
// retry after < default backoff policy
f(1*time.Second, 10*time.Second, 1, 1*time.Second)
// retry after invalid and < default backoff policy
f(0, time.Second, 1, 2*time.Second)
}
// Call calculateRetryDuration for multiple times.
{
// default backoff policy 2 times
f(0, time.Second, 2, 4*time.Second)
// default backoff policy 3 times
f(0, time.Second, 3, 8*time.Second)
// default backoff policy N times exceed max limit
f(0, time.Second, 10, time.Minute)
// retry after 120s 1 times
f(120*time.Second, time.Second, 1, 120*time.Second)
// retry after 120s 2 times
f(120*time.Second, time.Second, 2, 120*time.Second)
}
}
func TestParseRetryAfterHeader(t *testing.T) {
f := func(retryAfterString string, expectResult time.Duration) {
t.Helper()
result := parseRetryAfterHeader(retryAfterString)
// expect `expectResult == result` when retryAfterString is in seconds or invalid
// expect the difference between result and expectResult to be lower than 10%
if !(expectResult == result || math.Abs(float64(expectResult-result))/float64(expectResult) < 0.10) {
t.Fatalf(
"incorrect retry after duration, want (ms): %d, got (ms): %d",
expectResult.Milliseconds(), result.Milliseconds(),
)
}
}
// retry after header in seconds
f("10", 10*time.Second)
// retry after header in date time
f(time.Now().Add(30*time.Second).UTC().Format(http.TimeFormat), 30*time.Second)
// retry after header invalid
f("invalid-retry-after", 0)
// retry after header not in GMT
f(time.Now().Add(10*time.Second).Format("Mon, 02 Jan 2006 15:04:05 FAKETZ"), 0)
}
// helper calculate the max possible time duration calculated by timeutil.AddJitterToDuration.
func helper(d time.Duration) time.Duration {
dv := d / 10
if dv > 10*time.Second {
dv = 10 * time.Second
}
return d + dv
}

View File

@@ -3,6 +3,7 @@ package remotewrite
import (
"flag"
"fmt"
"strings"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fasttime"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/flagutil"
@@ -56,6 +57,7 @@ var (
"See https://docs.victoriametrics.com/stream-aggregation/#ignore-aggregation-intervals-on-start")
streamAggrDropInputLabels = flagutil.NewArrayString("remoteWrite.streamAggr.dropInputLabels", "An optional list of labels to drop from samples "+
"before stream de-duplication and aggregation with -remoteWrite.streamAggr.config and -remoteWrite.streamAggr.dedupInterval at the corresponding -remoteWrite.url. "+
"Multiple labels per remoteWrite.url must be delimited by '^^': -remoteWrite.streamAggr.dropInputLabels='replica^^az,replica'. "+
"See https://docs.victoriametrics.com/stream-aggregation/#dropping-unneeded-labels")
)
@@ -155,7 +157,11 @@ func (rwctx *remoteWriteCtx) initStreamAggrConfig() {
dedupInterval := streamAggrDedupInterval.GetOptionalArg(idx)
if dedupInterval > 0 {
alias := fmt.Sprintf("dedup-%d", idx+1)
rwctx.deduplicator = streamaggr.NewDeduplicator(rwctx.pushInternalTrackDropped, dedupInterval, *streamAggrDropInputLabels, alias)
var dropLabels []string
if streamAggrDropInputLabels.GetOptionalArg(idx) != "" {
dropLabels = strings.Split(streamAggrDropInputLabels.GetOptionalArg(idx), "^^")
}
rwctx.deduplicator = streamaggr.NewDeduplicator(rwctx.pushInternalTrackDropped, dedupInterval, dropLabels, alias)
}
}
@@ -224,9 +230,13 @@ func newStreamAggrConfigPerURL(idx int, pushFunc streamaggr.PushFunc) (*streamag
if *showRemoteWriteURL {
alias = fmt.Sprintf("%d:%s", idx+1, remoteWriteURLs.GetOptionalArg(idx))
}
var dropLabels []string
if streamAggrDropInputLabels.GetOptionalArg(idx) != "" {
dropLabels = strings.Split(streamAggrDropInputLabels.GetOptionalArg(idx), "^^")
}
opts := &streamaggr.Options{
DedupInterval: streamAggrDedupInterval.GetOptionalArg(idx),
DropInputLabels: *streamAggrDropInputLabels,
DropInputLabels: dropLabels,
IgnoreOldSamples: streamAggrIgnoreOldSamples.GetOptionalArg(idx),
IgnoreFirstIntervals: streamAggrIgnoreFirstIntervals.GetOptionalArg(idx),
KeepInput: streamAggrKeepInput.GetOptionalArg(idx),

View File

@@ -8,5 +8,5 @@ FROM $root_image
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
EXPOSE 8429
ENTRYPOINT ["/vmalert-tool-prod"]
ARG TARGETARCH=non-existing
ARG TARGETARCH
COPY vmalert-tool-linux-${TARGETARCH}-prod ./vmalert-tool-prod

View File

@@ -76,7 +76,7 @@ absolute path to all .tpl files in root.
`Link to VMUI: -external.alert.source='vmui/#/?g0.expr={{.Expr|queryEscape}}'. `+
`If empty 'vmalert/alert?group_id={{.GroupID}}&alert_id={{.AlertID}}' is used.`)
externalLabels = flagutil.NewArrayString("external.label", "Optional label in the form 'Name=value' to add to all generated recording rules and alerts. "+
"Pass multiple -label flags in order to add multiple label sets.")
"In case of conflicts, original labels are kept with prefix `exported_`.")
remoteReadIgnoreRestoreErrors = flag.Bool("remoteRead.ignoreRestoreErrors", true, "Whether to ignore errors from remote storage when restoring alerts state on startup. DEPRECATED - this flag has no effect and will be removed in the next releases.")

View File

@@ -8,5 +8,5 @@ FROM $root_image
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
EXPOSE 8880
ENTRYPOINT ["/vmalert-prod"]
ARG TARGETARCH=non-existing
ARG TARGETARCH
COPY vmalert-linux-${TARGETARCH}-prod ./vmalert-prod

View File

@@ -187,7 +187,7 @@ func templateAnnotation(dst io.Writer, text string, data tplData, tmpl *textTpl.
return nil
}
func (a Alert) toPromLabels(relabelCfg *promrelabel.ParsedConfigs) []prompbmarshal.Label {
func (a Alert) applyRelabelingIfNeeded(relabelCfg *promrelabel.ParsedConfigs) []prompbmarshal.Label {
var labels []prompbmarshal.Label
for k, v := range a.Labels {
labels = append(labels, prompbmarshal.Label{

View File

@@ -187,7 +187,7 @@ func TestAlert_toPromLabels(t *testing.T) {
fn := func(labels map[string]string, exp []prompbmarshal.Label, relabel *promrelabel.ParsedConfigs) {
t.Helper()
a := Alert{Labels: labels}
got := a.toPromLabels(relabel)
got := a.applyRelabelingIfNeeded(relabel)
if !reflect.DeepEqual(got, exp) {
t.Fatalf("expected to have: \n%v;\ngot:\n%v",
exp, got)

View File

@@ -8,6 +8,8 @@
{% func amRequest(alerts []Alert, generatorURL func(Alert) string, relabelCfg *promrelabel.ParsedConfigs) %}
[
{% for i, alert := range alerts %}
{% code lbls := alert.applyRelabelingIfNeeded(relabelCfg) %}
{% if len(lbls) == 0 %} {% continue %} {% endif %}
{
"startsAt":{%q= alert.Start.Format(time.RFC3339Nano) %},
"generatorURL": {%q= generatorURL(alert) %},
@@ -15,7 +17,6 @@
"endsAt":{%q= alert.End.Format(time.RFC3339Nano) %},
{% endif %}
"labels": {
{% code lbls := alert.toPromLabels(relabelCfg) %}
{% code ll := len(lbls) %}
{% for idx, l := range lbls %}
{%q= l.Name %}:{%q= l.Value %}{% if idx != ll-1 %}, {% endif %}

View File

@@ -30,111 +30,117 @@ func streamamRequest(qw422016 *qt422016.Writer, alerts []Alert, generatorURL fun
qw422016.N().S(`[`)
//line app/vmalert/notifier/alertmanager_request.qtpl:10
for i, alert := range alerts {
//line app/vmalert/notifier/alertmanager_request.qtpl:10
qw422016.N().S(`{"startsAt":`)
//line app/vmalert/notifier/alertmanager_request.qtpl:12
qw422016.N().Q(alert.Start.Format(time.RFC3339Nano))
//line app/vmalert/notifier/alertmanager_request.qtpl:12
qw422016.N().S(`,"generatorURL":`)
//line app/vmalert/notifier/alertmanager_request.qtpl:13
qw422016.N().Q(generatorURL(alert))
//line app/vmalert/notifier/alertmanager_request.qtpl:13
qw422016.N().S(`,`)
//line app/vmalert/notifier/alertmanager_request.qtpl:14
if !alert.End.IsZero() {
//line app/vmalert/notifier/alertmanager_request.qtpl:14
qw422016.N().S(`"endsAt":`)
//line app/vmalert/notifier/alertmanager_request.qtpl:15
qw422016.N().Q(alert.End.Format(time.RFC3339Nano))
//line app/vmalert/notifier/alertmanager_request.qtpl:15
qw422016.N().S(`,`)
//line app/vmalert/notifier/alertmanager_request.qtpl:16
}
//line app/vmalert/notifier/alertmanager_request.qtpl:16
qw422016.N().S(`"labels": {`)
//line app/vmalert/notifier/alertmanager_request.qtpl:18
lbls := alert.toPromLabels(relabelCfg)
//line app/vmalert/notifier/alertmanager_request.qtpl:11
lbls := alert.applyRelabelingIfNeeded(relabelCfg)
//line app/vmalert/notifier/alertmanager_request.qtpl:19
//line app/vmalert/notifier/alertmanager_request.qtpl:12
if len(lbls) == 0 {
//line app/vmalert/notifier/alertmanager_request.qtpl:12
continue
//line app/vmalert/notifier/alertmanager_request.qtpl:12
}
//line app/vmalert/notifier/alertmanager_request.qtpl:12
qw422016.N().S(`{"startsAt":`)
//line app/vmalert/notifier/alertmanager_request.qtpl:14
qw422016.N().Q(alert.Start.Format(time.RFC3339Nano))
//line app/vmalert/notifier/alertmanager_request.qtpl:14
qw422016.N().S(`,"generatorURL":`)
//line app/vmalert/notifier/alertmanager_request.qtpl:15
qw422016.N().Q(generatorURL(alert))
//line app/vmalert/notifier/alertmanager_request.qtpl:15
qw422016.N().S(`,`)
//line app/vmalert/notifier/alertmanager_request.qtpl:16
if !alert.End.IsZero() {
//line app/vmalert/notifier/alertmanager_request.qtpl:16
qw422016.N().S(`"endsAt":`)
//line app/vmalert/notifier/alertmanager_request.qtpl:17
qw422016.N().Q(alert.End.Format(time.RFC3339Nano))
//line app/vmalert/notifier/alertmanager_request.qtpl:17
qw422016.N().S(`,`)
//line app/vmalert/notifier/alertmanager_request.qtpl:18
}
//line app/vmalert/notifier/alertmanager_request.qtpl:18
qw422016.N().S(`"labels": {`)
//line app/vmalert/notifier/alertmanager_request.qtpl:20
ll := len(lbls)
//line app/vmalert/notifier/alertmanager_request.qtpl:20
//line app/vmalert/notifier/alertmanager_request.qtpl:21
for idx, l := range lbls {
//line app/vmalert/notifier/alertmanager_request.qtpl:21
//line app/vmalert/notifier/alertmanager_request.qtpl:22
qw422016.N().Q(l.Name)
//line app/vmalert/notifier/alertmanager_request.qtpl:21
//line app/vmalert/notifier/alertmanager_request.qtpl:22
qw422016.N().S(`:`)
//line app/vmalert/notifier/alertmanager_request.qtpl:21
//line app/vmalert/notifier/alertmanager_request.qtpl:22
qw422016.N().Q(l.Value)
//line app/vmalert/notifier/alertmanager_request.qtpl:21
//line app/vmalert/notifier/alertmanager_request.qtpl:22
if idx != ll-1 {
//line app/vmalert/notifier/alertmanager_request.qtpl:21
//line app/vmalert/notifier/alertmanager_request.qtpl:22
qw422016.N().S(`,`)
//line app/vmalert/notifier/alertmanager_request.qtpl:21
//line app/vmalert/notifier/alertmanager_request.qtpl:22
}
//line app/vmalert/notifier/alertmanager_request.qtpl:22
//line app/vmalert/notifier/alertmanager_request.qtpl:23
}
//line app/vmalert/notifier/alertmanager_request.qtpl:22
//line app/vmalert/notifier/alertmanager_request.qtpl:23
qw422016.N().S(`},"annotations": {`)
//line app/vmalert/notifier/alertmanager_request.qtpl:25
//line app/vmalert/notifier/alertmanager_request.qtpl:26
c := len(alert.Annotations)
//line app/vmalert/notifier/alertmanager_request.qtpl:26
for k, v := range alert.Annotations {
//line app/vmalert/notifier/alertmanager_request.qtpl:27
for k, v := range alert.Annotations {
//line app/vmalert/notifier/alertmanager_request.qtpl:28
c = c - 1
//line app/vmalert/notifier/alertmanager_request.qtpl:28
//line app/vmalert/notifier/alertmanager_request.qtpl:29
qw422016.N().Q(k)
//line app/vmalert/notifier/alertmanager_request.qtpl:28
//line app/vmalert/notifier/alertmanager_request.qtpl:29
qw422016.N().S(`:`)
//line app/vmalert/notifier/alertmanager_request.qtpl:28
//line app/vmalert/notifier/alertmanager_request.qtpl:29
qw422016.N().Q(v)
//line app/vmalert/notifier/alertmanager_request.qtpl:28
//line app/vmalert/notifier/alertmanager_request.qtpl:29
if c > 0 {
//line app/vmalert/notifier/alertmanager_request.qtpl:28
//line app/vmalert/notifier/alertmanager_request.qtpl:29
qw422016.N().S(`,`)
//line app/vmalert/notifier/alertmanager_request.qtpl:28
//line app/vmalert/notifier/alertmanager_request.qtpl:29
}
//line app/vmalert/notifier/alertmanager_request.qtpl:29
//line app/vmalert/notifier/alertmanager_request.qtpl:30
}
//line app/vmalert/notifier/alertmanager_request.qtpl:29
//line app/vmalert/notifier/alertmanager_request.qtpl:30
qw422016.N().S(`}}`)
//line app/vmalert/notifier/alertmanager_request.qtpl:32
//line app/vmalert/notifier/alertmanager_request.qtpl:33
if i != len(alerts)-1 {
//line app/vmalert/notifier/alertmanager_request.qtpl:32
//line app/vmalert/notifier/alertmanager_request.qtpl:33
qw422016.N().S(`,`)
//line app/vmalert/notifier/alertmanager_request.qtpl:32
//line app/vmalert/notifier/alertmanager_request.qtpl:33
}
//line app/vmalert/notifier/alertmanager_request.qtpl:33
//line app/vmalert/notifier/alertmanager_request.qtpl:34
}
//line app/vmalert/notifier/alertmanager_request.qtpl:33
//line app/vmalert/notifier/alertmanager_request.qtpl:34
qw422016.N().S(`]`)
//line app/vmalert/notifier/alertmanager_request.qtpl:35
//line app/vmalert/notifier/alertmanager_request.qtpl:36
}
//line app/vmalert/notifier/alertmanager_request.qtpl:35
//line app/vmalert/notifier/alertmanager_request.qtpl:36
func writeamRequest(qq422016 qtio422016.Writer, alerts []Alert, generatorURL func(Alert) string, relabelCfg *promrelabel.ParsedConfigs) {
//line app/vmalert/notifier/alertmanager_request.qtpl:35
//line app/vmalert/notifier/alertmanager_request.qtpl:36
qw422016 := qt422016.AcquireWriter(qq422016)
//line app/vmalert/notifier/alertmanager_request.qtpl:35
//line app/vmalert/notifier/alertmanager_request.qtpl:36
streamamRequest(qw422016, alerts, generatorURL, relabelCfg)
//line app/vmalert/notifier/alertmanager_request.qtpl:35
//line app/vmalert/notifier/alertmanager_request.qtpl:36
qt422016.ReleaseWriter(qw422016)
//line app/vmalert/notifier/alertmanager_request.qtpl:35
//line app/vmalert/notifier/alertmanager_request.qtpl:36
}
//line app/vmalert/notifier/alertmanager_request.qtpl:35
//line app/vmalert/notifier/alertmanager_request.qtpl:36
func amRequest(alerts []Alert, generatorURL func(Alert) string, relabelCfg *promrelabel.ParsedConfigs) string {
//line app/vmalert/notifier/alertmanager_request.qtpl:35
//line app/vmalert/notifier/alertmanager_request.qtpl:36
qb422016 := qt422016.AcquireByteBuffer()
//line app/vmalert/notifier/alertmanager_request.qtpl:35
//line app/vmalert/notifier/alertmanager_request.qtpl:36
writeamRequest(qb422016, alerts, generatorURL, relabelCfg)
//line app/vmalert/notifier/alertmanager_request.qtpl:35
//line app/vmalert/notifier/alertmanager_request.qtpl:36
qs422016 := string(qb422016.B)
//line app/vmalert/notifier/alertmanager_request.qtpl:35
//line app/vmalert/notifier/alertmanager_request.qtpl:36
qt422016.ReleaseByteBuffer(qb422016)
//line app/vmalert/notifier/alertmanager_request.qtpl:35
//line app/vmalert/notifier/alertmanager_request.qtpl:36
return qs422016
//line app/vmalert/notifier/alertmanager_request.qtpl:35
//line app/vmalert/notifier/alertmanager_request.qtpl:36
}

View File

@@ -11,6 +11,7 @@ import (
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/promauth"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/promrelabel"
)
func TestAlertManager_Addr(t *testing.T) {
@@ -76,12 +77,33 @@ func TestAlertManager_Send(t *testing.T) {
if a[0].EndAt.IsZero() {
t.Fatalf("expected non-zero end time")
}
if len(a[0].Labels) != 1 {
t.Fatalf("expected 1 labels got %d", len(a[0].Labels))
}
if len(a[0].Annotations) != 2 {
t.Fatalf("expected 2 annotations got %d", len(a[0].Annotations))
}
if r.Header.Get(headerKey) != "bar" {
t.Fatalf("expected header %q to be set to %q; got %q instead", headerKey, headerValue, r.Header.Get(headerKey))
}
case 3:
if r.Header.Get(headerKey) != headerValue {
t.Fatalf("expected header %q to be set to %q; got %q instead", headerKey, headerValue, r.Header.Get(headerKey))
var a []struct {
Labels map[string]string `json:"labels"`
}
if err := json.NewDecoder(r.Body).Decode(&a); err != nil {
t.Fatalf("can not unmarshal data into alert %s", err)
}
if len(a) != 1 {
t.Fatalf("expected 1 alert in array got %d", len(a))
}
if len(a[0].Labels) != 3 {
t.Fatalf("expected 3 labels got %d", len(a[0].Labels))
}
if a[0].Labels["env"] != "prod" {
t.Fatalf("expected env label to be prod during relabeling, got %s", a[0].Labels["env"])
}
if r.Header.Get(headerKey) != "bar" {
t.Fatalf("expected header %q to be set to %q; got %q instead", headerKey, "bar", r.Header.Get(headerKey))
}
}
})
@@ -95,31 +117,58 @@ func TestAlertManager_Send(t *testing.T) {
},
Headers: []string{fmt.Sprintf("%s:%s", headerKey, headerValue)},
}
parsedConfigs, err := promrelabel.ParseRelabelConfigsData([]byte(`
- action: drop
if: '{tenant="0"}'
regex: ".*"
- target_label: "env"
replacement: "prod"
if: '{tenant="1"}'
`))
if err != nil {
t.Fatalf("unexpected error when parse relabeling config: %s", err)
}
am, err := NewAlertManager(srv.URL+alertManagerPath, func(alert Alert) string {
return strconv.FormatUint(alert.GroupID, 10) + "/" + strconv.FormatUint(alert.ID, 10)
}, aCfg, nil, 0)
}, aCfg, parsedConfigs, 0)
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
if err := am.Send(context.Background(), []Alert{{}, {}}, nil); err == nil {
if err := am.Send(context.Background(), []Alert{{Labels: map[string]string{"a": "b"}}}, nil); err == nil {
t.Fatalf("expected connection error got nil")
}
if err := am.Send(context.Background(), []Alert{}, nil); err == nil {
if err := am.Send(context.Background(), []Alert{{Labels: map[string]string{"a": "b"}}}, nil); err == nil {
t.Fatalf("expected wrong http code error got nil")
}
if err := am.Send(context.Background(), []Alert{{
GroupID: 0,
Name: "alert0",
Start: time.Now().UTC(),
End: time.Now().UTC(),
Annotations: map[string]string{"a": "b", "c": "d", "e": "f"},
Labels: map[string]string{"alertname": "alert0"},
Annotations: map[string]string{"a": "b", "c": "d"},
}}, map[string]string{headerKey: "bar"}); err != nil {
t.Fatalf("unexpected error %s", err)
}
if c != 2 {
t.Fatalf("expected 2 calls(count from zero) to server got %d", c)
}
if err := am.Send(context.Background(), nil, map[string]string{headerKey: headerValue}); err != nil {
if err := am.Send(context.Background(), []Alert{
// drop tenant0 alert message during relabeling
{
Name: "alert1",
Labels: map[string]string{"rule": "test", "tenant": "0"},
},
{
Name: "alert2",
Labels: map[string]string{"rule": "test", "tenant": "1"},
},
}, map[string]string{headerKey: "bar"}); err != nil {
t.Fatalf("unexpected error %s", err)
}
if c != 3 {
t.Fatalf("expected 3 calls(count from zero) to server got %d", c)
}
}

View File

@@ -34,9 +34,9 @@ var (
idleConnectionTimeout = flag.Duration("remoteWrite.idleConnTimeout", 50*time.Second, `Defines a duration for idle (keep-alive connections) to exist. Consider settings this value less to the value of "-http.idleConnTimeout". It must prevent possible "write: broken pipe" and "read: connection reset by peer" errors.`)
maxQueueSize = flag.Int("remoteWrite.maxQueueSize", 1e5, "Defines the max number of pending datapoints to remote write endpoint")
maxBatchSize = flag.Int("remoteWrite.maxBatchSize", 1e3, "Defines max number of timeseries to be flushed at once")
concurrency = flag.Int("remoteWrite.concurrency", 1, "Defines number of writers for concurrent writing into remote write endpoint")
maxQueueSize = flag.Int("remoteWrite.maxQueueSize", 1e6, "Defines the max number of pending datapoints to remote write endpoint")
maxBatchSize = flag.Int("remoteWrite.maxBatchSize", 1e4, "Defines max number of timeseries to be flushed at once")
concurrency = flag.Int("remoteWrite.concurrency", 4, "Defines number of writers for concurrent writing into remote write endpoint")
flushInterval = flag.Duration("remoteWrite.flushInterval", 5*time.Second, "Defines interval of flushes to remote write endpoint")
tlsInsecureSkipVerify = flag.Bool("remoteWrite.tlsInsecureSkipVerify", false, "Whether to skip tls verification when connecting to -remoteWrite.url")

View File

@@ -335,6 +335,8 @@ func (ar *AlertingRule) execRange(ctx context.Context, start, end time.Time) ([]
// reset to Pending if there are gaps > EvalInterval between DPs
a.State = notifier.StatePending
a.ActiveAt = at
// re-template the annotations as active timestamp is changed
_, a.Annotations, _ = ar.expandTemplates(s, qFn, at)
a.Start = time.Time{}
} else if at.Sub(a.ActiveAt) >= ar.For && a.State != notifier.StateFiring {
a.State = notifier.StateFiring

View File

@@ -289,7 +289,7 @@ func TestAlertingRule_Exec(t *testing.T) {
4: {{labels: []string{"name", "foo"}, alert: &notifier.Alert{State: notifier.StateFiring}}},
})
f(newTestAlertingRuleWithKeepFiring("for-pending=>firing=>keepfiring=>firing", defaultStep, defaultStep), [][]datasource.Metric{
f(newTestAlertingRuleWithCustomFields("for-pending=>firing=>keepfiring=>firing", defaultStep, 0, defaultStep, nil), [][]datasource.Metric{
{metricWithLabels(t, "name", "foo")},
{metricWithLabels(t, "name", "foo")},
// empty step to keep firing
@@ -302,7 +302,7 @@ func TestAlertingRule_Exec(t *testing.T) {
3: {{labels: []string{"name", "foo"}, alert: &notifier.Alert{State: notifier.StateFiring}}},
})
f(newTestAlertingRuleWithKeepFiring("for-pending=>firing=>keepfiring=>keepfiring=>inactive=>pending=>firing", defaultStep, 2*defaultStep), [][]datasource.Metric{
f(newTestAlertingRuleWithCustomFields("for-pending=>firing=>keepfiring=>keepfiring=>inactive=>pending=>firing", defaultStep, 0, 2*defaultStep, nil), [][]datasource.Metric{
{metricWithLabels(t, "name", "foo")},
{metricWithLabels(t, "name", "foo")},
// empty step to keep firing
@@ -395,7 +395,7 @@ func TestAlertingRuleExecRange(t *testing.T) {
{State: notifier.StateFiring, ActiveAt: time.Unix(3e3, 0)},
}, nil)
f(newTestAlertingRule("for-pending", time.Second), []datasource.Metric{
f(newTestAlertingRuleWithCustomFields("for-pending", time.Second, 0, 0, map[string]string{"activeAt": "{{ $activeAt.UnixMilli }}"}), []datasource.Metric{
{Values: []float64{1, 1, 1}, Timestamps: []int64{1, 3, 5}},
}, []*notifier.Alert{
{State: notifier.StatePending, ActiveAt: time.Unix(1, 0)},
@@ -406,7 +406,7 @@ func TestAlertingRuleExecRange(t *testing.T) {
GroupID: fakeGroup.ID(),
Name: "for-pending",
Labels: map[string]string{"alertname": "for-pending"},
Annotations: map[string]string{},
Annotations: map[string]string{"activeAt": "5000"},
State: notifier.StatePending,
ActiveAt: time.Unix(5, 0),
Value: 1,
@@ -414,7 +414,7 @@ func TestAlertingRuleExecRange(t *testing.T) {
},
})
f(newTestAlertingRule("for-firing", 3*time.Second), []datasource.Metric{
f(newTestAlertingRuleWithCustomFields("for-firing", 3*time.Second, 0, 0, map[string]string{"activeAt": "{{ $activeAt.UnixMilli }}"}), []datasource.Metric{
{Values: []float64{1, 1, 1}, Timestamps: []int64{1, 3, 5}},
}, []*notifier.Alert{
{State: notifier.StatePending, ActiveAt: time.Unix(1, 0)},
@@ -425,7 +425,7 @@ func TestAlertingRuleExecRange(t *testing.T) {
GroupID: fakeGroup.ID(),
Name: "for-firing",
Labels: map[string]string{"alertname": "for-firing"},
Annotations: map[string]string{},
Annotations: map[string]string{"activeAt": "1000"},
State: notifier.StateFiring,
ActiveAt: time.Unix(1, 0),
Start: time.Unix(5, 0),
@@ -434,7 +434,7 @@ func TestAlertingRuleExecRange(t *testing.T) {
},
})
f(newTestAlertingRule("for-hold-pending", time.Second), []datasource.Metric{
f(newTestAlertingRuleWithCustomFields("for-hold-pending", time.Second, 0, 0, map[string]string{"activeAt": "{{ $activeAt.UnixMilli }}"}), []datasource.Metric{
{Values: []float64{1, 1, 1}, Timestamps: []int64{1, 2, 5}},
}, []*notifier.Alert{
{State: notifier.StatePending, ActiveAt: time.Unix(1, 0)},
@@ -445,7 +445,7 @@ func TestAlertingRuleExecRange(t *testing.T) {
GroupID: fakeGroup.ID(),
Name: "for-hold-pending",
Labels: map[string]string{"alertname": "for-hold-pending"},
Annotations: map[string]string{},
Annotations: map[string]string{"activeAt": "5000"},
State: notifier.StatePending,
ActiveAt: time.Unix(5, 0),
Value: 1,
@@ -453,7 +453,7 @@ func TestAlertingRuleExecRange(t *testing.T) {
},
})
f(newTestAlertingRuleWithEvalInterval("firing=>inactive=>inactive=>firing=>firing", 0, time.Second), []datasource.Metric{
f(newTestAlertingRuleWithCustomFields("firing=>inactive=>inactive=>firing=>firing", 0, time.Second, 0, nil), []datasource.Metric{
{Values: []float64{1, 1, 1, 1}, Timestamps: []int64{1, 4, 5, 6}},
}, []*notifier.Alert{
{State: notifier.StateFiring, ActiveAt: time.Unix(1, 0)},
@@ -538,7 +538,6 @@ func TestAlertingRuleExecRange(t *testing.T) {
"source": "vm",
},
},
//
{
State: notifier.StateFiring, ActiveAt: time.Unix(1, 0),
Labels: map[string]string{
@@ -1036,15 +1035,13 @@ func newTestAlertingRule(name string, waitFor time.Duration) *AlertingRule {
return &rule
}
func newTestAlertingRuleWithEvalInterval(name string, waitFor, evalInterval time.Duration) *AlertingRule {
rule := newTestAlertingRule(name, waitFor)
rule.EvalInterval = evalInterval
return rule
}
func newTestAlertingRuleWithKeepFiring(name string, waitFor, keepFiringFor time.Duration) *AlertingRule {
func newTestAlertingRuleWithCustomFields(name string, waitFor, evalInterval, keepFiringFor time.Duration, annotation map[string]string) *AlertingRule {
rule := newTestAlertingRule(name, waitFor)
if evalInterval != 0 {
rule.EvalInterval = evalInterval
}
rule.KeepFiringFor = keepFiringFor
rule.Annotations = annotation
return rule
}

View File

@@ -50,12 +50,12 @@ func TestRequestToCurl(t *testing.T) {
f(req, "curl -k -X POST 'https://user:xxxxx@foo.com?query=up&step=10'")
req = newReq("https://user:pass@foo.com")
req.Header.Set("Authorisation", "Bearer 123456")
f(req, "curl -k -X POST -H 'Authorisation: <secret>' 'https://user:xxxxx@foo.com'")
req.Header.Set("Authorization", "Bearer 123456")
f(req, "curl -k -X POST -H 'Authorization: <secret>' 'https://user:xxxxx@foo.com'")
req = newReq("https://user:pass@foo.com")
req.Header.Set("Authorisation", "Basic 123456")
f(req, "curl -k -X POST -H 'Authorisation: <secret>' 'https://user:xxxxx@foo.com'")
req.Header.Set("Authorization", "Basic 123456")
f(req, "curl -k -X POST -H 'Authorization: <secret>' 'https://user:xxxxx@foo.com'")
req = newReq("https://foo.com")
req.Header.Set("My-Password", "mypassword")
@@ -66,7 +66,7 @@ func TestRequestToCurl(t *testing.T) {
f(req, "curl -k -X POST -H 'Key-For: <secret>' 'https://foo.com'")
req = newReq("https://foo.com")
req.Header.Set("My-Secret-Org", "secret-organisation")
req.Header.Set("My-Secret-Org", "secret-organization")
f(req, "curl -k -X POST -H 'My-Secret-Org: <secret>' 'https://foo.com'")
req = newReq("https://foo.com")

View File

@@ -8,5 +8,5 @@ FROM $root_image
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
EXPOSE 8427
ENTRYPOINT ["/vmauth-prod"]
ARG TARGETARCH=non-existing
ARG TARGETARCH
COPY vmauth-linux-${TARGETARCH}-prod ./vmauth-prod

View File

@@ -7,5 +7,5 @@ RUN apk update && apk upgrade && apk --update --no-cache add ca-certificates
FROM $root_image
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
ENTRYPOINT ["/vmbackup-prod"]
ARG TARGETARCH=non-existing
ARG TARGETARCH
COPY vmbackup-linux-${TARGETARCH}-prod ./vmbackup-prod

View File

@@ -7,5 +7,5 @@ RUN apk update && apk upgrade && apk --update --no-cache add ca-certificates
FROM $root_image
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
ENTRYPOINT ["/vmctl-prod"]
ARG TARGETARCH=non-existing
ARG TARGETARCH
COPY vmctl-linux-${TARGETARCH}-prod ./vmctl-prod

View File

@@ -36,40 +36,37 @@ func TestGetTime_Success(t *testing.T) {
}
// only year
f("2019", time.Date(2019, 1, 1, 0, 0, 0, 0, time.UTC))
f("2019Z", time.Date(2019, 1, 1, 0, 0, 0, 0, time.UTC))
// year and month
f("2019-01", time.Date(2019, 1, 1, 0, 0, 0, 0, time.UTC))
f("2019-01Z", time.Date(2019, 1, 1, 0, 0, 0, 0, time.UTC))
// year and not first month
f("2019-02", time.Date(2019, 2, 1, 0, 0, 0, 0, time.UTC))
f("2019-02Z", time.Date(2019, 2, 1, 0, 0, 0, 0, time.UTC))
// year, month and day
f("2019-02-01", time.Date(2019, 2, 1, 0, 0, 0, 0, time.UTC))
f("2019-02-01Z", time.Date(2019, 2, 1, 0, 0, 0, 0, time.UTC))
// year, month and not first day
f("2019-02-10", time.Date(2019, 2, 10, 0, 0, 0, 0, time.UTC))
f("2019-02-10Z", time.Date(2019, 2, 10, 0, 0, 0, 0, time.UTC))
// year, month, day and time
f("2019-02-02T00", time.Date(2019, 2, 2, 0, 0, 0, 0, time.UTC))
f("2019-02-02T00Z", time.Date(2019, 2, 2, 0, 0, 0, 0, time.UTC))
// year, month, day and one hour time
f("2019-02-02T01", time.Date(2019, 2, 2, 1, 0, 0, 0, time.UTC))
f("2019-02-02T01Z", time.Date(2019, 2, 2, 1, 0, 0, 0, time.UTC))
// time with zero minutes
f("2019-02-02T01:00", time.Date(2019, 2, 2, 1, 0, 0, 0, time.UTC))
f("2019-02-02T01:00Z", time.Date(2019, 2, 2, 1, 0, 0, 0, time.UTC))
// time with one minute
f("2019-02-02T01:01", time.Date(2019, 2, 2, 1, 1, 0, 0, time.UTC))
f("2019-02-02T01:01Z", time.Date(2019, 2, 2, 1, 1, 0, 0, time.UTC))
// time with zero seconds
f("2019-02-02T01:01:00", time.Date(2019, 2, 2, 1, 1, 0, 0, time.UTC))
f("2019-02-02T01:01:00Z", time.Date(2019, 2, 2, 1, 1, 0, 0, time.UTC))
// timezone with one second
f("2019-02-02T01:01:01", time.Date(2019, 2, 2, 1, 1, 1, 0, time.UTC))
// time with two second and timezone
f("2019-07-07T20:01:02Z", time.Date(2019, 7, 7, 20, 1, 02, 0, time.UTC))
f("2019-02-02T01:01:01Z", time.Date(2019, 2, 2, 1, 1, 1, 0, time.UTC))
// time with seconds and timezone
f("2019-07-07T20:47:40+03:00", func() time.Time {

View File

@@ -4,7 +4,6 @@ import (
"context"
"flag"
"fmt"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/barpool"
"log"
"net/http"
"os"
@@ -12,6 +11,7 @@ import (
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/backoff"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/barpool"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/native"
remote_read_integration "github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/testdata/servers_integration_test"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/vm"
@@ -251,7 +251,7 @@ func deleteSeries(name, value string) (int, error) {
if err := tfs.Add([]byte(name), []byte(value), false, true); err != nil {
return 0, fmt.Errorf("unexpected error in TagFilters.Add: %w", err)
}
return vmstorage.DeleteSeries(nil, []*storage.TagFilters{tfs})
return vmstorage.DeleteSeries(nil, []*storage.TagFilters{tfs}, 1e3)
}
func TestBuildMatchWithFilter_Failure(t *testing.T) {

View File

@@ -7,5 +7,5 @@ RUN apk update && apk upgrade && apk --update --no-cache add ca-certificates
FROM $root_image
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
ENTRYPOINT ["/vmrestore-prod"]
ARG TARGETARCH=non-existing
ARG TARGETARCH
COPY vmrestore-linux-${TARGETARCH}-prod ./vmrestore-prod

View File

@@ -2,7 +2,6 @@ package vmselect
import (
"embed"
"errors"
"flag"
"fmt"
"net/http"
@@ -45,10 +44,7 @@ var (
var slowQueries = metrics.NewCounter(`vm_slow_queries_total`)
func getDefaultMaxConcurrentRequests() int {
n := cgroup.AvailableCPUs()
if n <= 4 {
n *= 2
}
n := cgroup.AvailableCPUs() * 2
if n > 16 {
// A single request can saturate all the CPU cores, so there is no sense
// in allowing higher number of concurrent requests - they will just contend
@@ -187,7 +183,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
httpserver.EnableCORS(w, r)
if err := prometheus.LabelValuesHandler(qt, startTime, labelName, w, r); err != nil {
labelValuesErrors.Inc()
sendPrometheusError(w, r, err)
httpserver.SendPrometheusError(w, r, err)
return true
}
return true
@@ -210,7 +206,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
httpserver.EnableCORS(w, r)
if err := prometheus.QueryHandler(qt, startTime, w, r); err != nil {
queryErrors.Inc()
sendPrometheusError(w, r, err)
httpserver.SendPrometheusError(w, r, err)
return true
}
return true
@@ -219,7 +215,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
httpserver.EnableCORS(w, r)
if err := prometheus.QueryRangeHandler(qt, startTime, w, r); err != nil {
queryRangeErrors.Inc()
sendPrometheusError(w, r, err)
httpserver.SendPrometheusError(w, r, err)
return true
}
return true
@@ -228,7 +224,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
httpserver.EnableCORS(w, r)
if err := prometheus.SeriesHandler(qt, startTime, w, r); err != nil {
seriesErrors.Inc()
sendPrometheusError(w, r, err)
httpserver.SendPrometheusError(w, r, err)
return true
}
return true
@@ -237,7 +233,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
httpserver.EnableCORS(w, r)
if err := prometheus.SeriesCountHandler(startTime, w, r); err != nil {
seriesCountErrors.Inc()
sendPrometheusError(w, r, err)
httpserver.SendPrometheusError(w, r, err)
return true
}
return true
@@ -246,7 +242,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
httpserver.EnableCORS(w, r)
if err := prometheus.LabelsHandler(qt, startTime, w, r); err != nil {
labelsErrors.Inc()
sendPrometheusError(w, r, err)
httpserver.SendPrometheusError(w, r, err)
return true
}
return true
@@ -255,7 +251,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
httpserver.EnableCORS(w, r)
if err := prometheus.TSDBStatusHandler(qt, startTime, w, r); err != nil {
statusTSDBErrors.Inc()
sendPrometheusError(w, r, err)
httpserver.SendPrometheusError(w, r, err)
return true
}
return true
@@ -498,7 +494,7 @@ func handleStaticAndSimpleRequests(w http.ResponseWriter, r *http.Request, path
httpserver.EnableCORS(w, r)
if err := prometheus.QueryStatsHandler(w, r); err != nil {
topQueriesErrors.Inc()
sendPrometheusError(w, r, fmt.Errorf("cannot query status endpoint: %w", err))
httpserver.SendPrometheusError(w, r, fmt.Errorf("cannot query status endpoint: %w", err))
return true
}
return true
@@ -575,24 +571,6 @@ func isGraphiteTagsPath(path string) bool {
}
}
func sendPrometheusError(w http.ResponseWriter, r *http.Request, err error) {
logger.WarnfSkipframes(1, "error in %q: %s", httpserver.GetRequestURI(r), err)
w.Header().Set("Content-Type", "application/json")
statusCode := http.StatusUnprocessableEntity
var esc *httpserver.ErrorWithStatusCode
if errors.As(err, &esc) {
statusCode = esc.StatusCode
}
w.WriteHeader(statusCode)
var ure *promql.UserReadableError
if errors.As(err, &ure) {
err = ure
}
prometheus.WriteErrorResponse(w, statusCode, err)
}
var (
requestDuration = metrics.NewHistogram(`vmselect_request_duration_seconds`)

View File

@@ -765,7 +765,7 @@ func putSortBlocksHeap(sbh *sortBlocksHeap) {
var sbhPool sync.Pool
// DeleteSeries deletes time series matching the given tagFilterss.
// DeleteSeries deletes time series matching the given search query.
func DeleteSeries(qt *querytracer.Tracer, sq *storage.SearchQuery, deadline searchutils.Deadline) (int, error) {
qt = qt.NewChild("delete series: %s", sq)
defer qt.Done()
@@ -774,7 +774,7 @@ func DeleteSeries(qt *querytracer.Tracer, sq *storage.SearchQuery, deadline sear
if err != nil {
return 0, err
}
return vmstorage.DeleteSeries(qt, tfss)
return vmstorage.DeleteSeries(qt, tfss, sq.MaxMetrics)
}
// LabelNames returns label names matching the given sq until the given deadline.

View File

@@ -1,61 +0,0 @@
// Code generated by qtc from "error_response.qtpl". DO NOT EDIT.
// See https://github.com/valyala/quicktemplate for details.
// ErrorResponse generates error response for /api/v1/query.See https://prometheus.io/docs/prometheus/latest/querying/api/#format-overview
//line app/vmselect/prometheus/error_response.qtpl:4
package prometheus
//line app/vmselect/prometheus/error_response.qtpl:4
import (
qtio422016 "io"
qt422016 "github.com/valyala/quicktemplate"
)
//line app/vmselect/prometheus/error_response.qtpl:4
var (
_ = qtio422016.Copy
_ = qt422016.AcquireByteBuffer
)
//line app/vmselect/prometheus/error_response.qtpl:4
func StreamErrorResponse(qw422016 *qt422016.Writer, statusCode int, err error) {
//line app/vmselect/prometheus/error_response.qtpl:4
qw422016.N().S(`{"status":"error","errorType":"`)
//line app/vmselect/prometheus/error_response.qtpl:7
qw422016.N().D(statusCode)
//line app/vmselect/prometheus/error_response.qtpl:7
qw422016.N().S(`","error":`)
//line app/vmselect/prometheus/error_response.qtpl:8
qw422016.N().Q(err.Error())
//line app/vmselect/prometheus/error_response.qtpl:8
qw422016.N().S(`}`)
//line app/vmselect/prometheus/error_response.qtpl:10
}
//line app/vmselect/prometheus/error_response.qtpl:10
func WriteErrorResponse(qq422016 qtio422016.Writer, statusCode int, err error) {
//line app/vmselect/prometheus/error_response.qtpl:10
qw422016 := qt422016.AcquireWriter(qq422016)
//line app/vmselect/prometheus/error_response.qtpl:10
StreamErrorResponse(qw422016, statusCode, err)
//line app/vmselect/prometheus/error_response.qtpl:10
qt422016.ReleaseWriter(qw422016)
//line app/vmselect/prometheus/error_response.qtpl:10
}
//line app/vmselect/prometheus/error_response.qtpl:10
func ErrorResponse(statusCode int, err error) string {
//line app/vmselect/prometheus/error_response.qtpl:10
qb422016 := qt422016.AcquireByteBuffer()
//line app/vmselect/prometheus/error_response.qtpl:10
WriteErrorResponse(qb422016, statusCode, err)
//line app/vmselect/prometheus/error_response.qtpl:10
qs422016 := string(qb422016.B)
//line app/vmselect/prometheus/error_response.qtpl:10
qt422016.ReleaseByteBuffer(qb422016)
//line app/vmselect/prometheus/error_response.qtpl:10
return qs422016
//line app/vmselect/prometheus/error_response.qtpl:10
}

View File

@@ -3,7 +3,6 @@ package prometheus
import (
"flag"
"fmt"
"github.com/VictoriaMetrics/metricsql"
"math"
"net/http"
"runtime"
@@ -27,6 +26,7 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/lib/querytracer"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/metricsql"
"github.com/valyala/fastjson/fastfloat"
)
@@ -52,6 +52,7 @@ var (
maxExportSeries = flag.Int("search.maxExportSeries", 10e6, "The maximum number of time series, which can be returned from /api/v1/export* APIs. This option allows limiting memory usage")
maxTSDBStatusSeries = flag.Int("search.maxTSDBStatusSeries", 10e6, "The maximum number of time series, which can be processed during the call to /api/v1/status/tsdb. This option allows limiting memory usage")
maxSeriesLimit = flag.Int("search.maxSeries", 30e3, "The maximum number of time series, which can be returned from /api/v1/series. This option allows limiting memory usage")
maxDeleteSeries = flag.Int("search.maxDeleteSeries", 1e6, "The maximum number of time series, which can be deleted using /api/v1/admin/tsdb/delete_series. This option allows limiting memory usage")
maxLabelsAPISeries = flag.Int("search.maxLabelsAPISeries", 1e6, "The maximum number of time series, which could be scanned when searching for the matching time series "+
"at /api/v1/labels and /api/v1/label/.../values. This option allows limiting memory usage and CPU usage. See also -search.maxLabelsAPIDuration, "+
"-search.maxTagKeys, -search.maxTagValues and -search.ignoreExtraFiltersAtLabelsAPI")
@@ -479,7 +480,7 @@ func DeleteHandler(startTime time.Time, r *http.Request) error {
if !cp.IsDefaultTimeRange() {
return fmt.Errorf("start=%d and end=%d args aren't supported. Remove these args from the query in order to delete all the matching metrics", cp.start, cp.end)
}
sq := storage.NewSearchQuery(cp.start, cp.end, cp.filterss, 0)
sq := storage.NewSearchQuery(cp.start, cp.end, cp.filterss, *maxDeleteSeries)
deletedCount, err := netstorage.DeleteSeries(nil, sq, cp.deadline)
if err != nil {
return fmt.Errorf("cannot delete time series: %w", err)

View File

@@ -19,6 +19,7 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/lib/decimal"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fasttime"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/flagutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/memory"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/querytracer"
@@ -354,7 +355,7 @@ func evalExprInternal(qt *querytracer.Tracer, ec *EvalConfig, e metricsql.Expr)
func evalTransformFunc(qt *querytracer.Tracer, ec *EvalConfig, fe *metricsql.FuncExpr) ([]*timeseries, error) {
tf := getTransformFunc(fe.Name)
if tf == nil {
return nil, &UserReadableError{
return nil, &httpserver.UserReadableError{
Err: fmt.Errorf(`unknown func %q`, fe.Name),
}
}
@@ -376,7 +377,7 @@ func evalTransformFunc(qt *querytracer.Tracer, ec *EvalConfig, fe *metricsql.Fun
}
rv, err := tf(tfa)
if err != nil {
return nil, &UserReadableError{
return nil, &httpserver.UserReadableError{
Err: fmt.Errorf(`cannot evaluate %q: %w`, fe.AppendString(nil), err),
}
}
@@ -407,7 +408,7 @@ func evalAggrFunc(qt *querytracer.Tracer, ec *EvalConfig, ae *metricsql.AggrFunc
}
af := getAggrFunc(ae.Name)
if af == nil {
return nil, &UserReadableError{
return nil, &httpserver.UserReadableError{
Err: fmt.Errorf(`unknown func %q`, ae.Name),
}
}
@@ -802,12 +803,12 @@ func evalRollupFunc(qt *querytracer.Tracer, ec *EvalConfig, funcName string, rf
}
tssAt, err := evalExpr(qt, ec, re.At)
if err != nil {
return nil, &UserReadableError{
return nil, &httpserver.UserReadableError{
Err: fmt.Errorf("cannot evaluate `@` modifier: %w", err),
}
}
if len(tssAt) != 1 {
return nil, &UserReadableError{
return nil, &httpserver.UserReadableError{
Err: fmt.Errorf("`@` modifier must return a single series; it returns %d series instead", len(tssAt)),
}
}
@@ -869,7 +870,7 @@ func evalRollupFuncWithoutAt(qt *querytracer.Tracer, ec *EvalConfig, funcName st
rvs, err = evalRollupFuncWithSubquery(qt, ecNew, funcName, rf, expr, re)
}
if err != nil {
return nil, &UserReadableError{
return nil, &httpserver.UserReadableError{
Err: err,
}
}
@@ -1601,7 +1602,7 @@ func evalRollupFuncWithMetricExpr(qt *querytracer.Tracer, ec *EvalConfig, funcNa
if ec.Start == ec.End {
rvs, err := evalInstantRollup(qt, ec, funcName, rf, expr, me, iafc, window)
if err != nil {
err = &UserReadableError{
err = &httpserver.UserReadableError{
Err: err,
}
return nil, err
@@ -1612,7 +1613,7 @@ func evalRollupFuncWithMetricExpr(qt *querytracer.Tracer, ec *EvalConfig, funcNa
evalWithConfig := func(ec *EvalConfig) ([]*timeseries, error) {
tss, err := evalRollupFuncNoCache(qt, ec, funcName, rf, expr, me, iafc, window, pointsPerSeries)
if err != nil {
err = &UserReadableError{
err = &httpserver.UserReadableError{
Err: err,
}
return nil, err

View File

@@ -35,24 +35,6 @@ var (
"Such conversion can be disabled using -search.disableImplicitConversion.")
)
// UserReadableError is a type of error which supposed to be returned to the user without additional context.
type UserReadableError struct {
// Err is the error which needs to be returned to the user.
Err error
}
// Unwrap returns ure.Err.
//
// This is used by standard errors package. See https://golang.org/pkg/errors
func (ure *UserReadableError) Unwrap() error {
return ure.Err
}
// Error satisfies Error interface
func (ure *UserReadableError) Error() string {
return ure.Err.Error()
}
// Exec executes q for the given ec.
func Exec(qt *querytracer.Tracer, ec *EvalConfig, q string, isFirstPointOnly bool) ([]netstorage.Result, error) {
if querystats.Enabled() {

View File

@@ -3027,6 +3027,12 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`1 and (0 > 1)`, func(t *testing.T) {
// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6637
t.Parallel()
q := `1 and (0 > 1)`
f(q, nil)
})
t.Run(`time() and 2`, func(t *testing.T) {
t.Parallel()
q := `time() and 2`
@@ -7245,6 +7251,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_trim_outliers(time() > 1200)`, func(t *testing.T) {
t.Parallel()
q := `range_trim_outliers(0.5, time() > 1200)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{nan, nan, nan, 1600, 1800, nan},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_trim_spikes()`, func(t *testing.T) {
t.Parallel()
q := `range_trim_spikes(0.2, time())`
@@ -7256,6 +7273,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_trim_spikes(time() > 1200 <= 1800)`, func(t *testing.T) {
t.Parallel()
q := `range_trim_spikes(0.2, time() > 1200 <= 1800)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{nan, nan, nan, 1600, nan, nan},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_trim_zscore()`, func(t *testing.T) {
t.Parallel()
q := `range_trim_zscore(0.9, time())`
@@ -7267,6 +7295,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_trim_zscore(time() > 1200 <= 1800)`, func(t *testing.T) {
t.Parallel()
q := `range_trim_zscore(0.9, time() > 1200 <= 1800)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{nan, nan, nan, 1600, nan, nan},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_zscore()`, func(t *testing.T) {
t.Parallel()
q := `round(range_zscore(time()), 0.1)`
@@ -7278,6 +7317,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_zscore(time() > 1200 < 1800)`, func(t *testing.T) {
t.Parallel()
q := `round(range_zscore(time() > 1200 < 1800), 0.1)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{nan, nan, -1, 1, nan, nan},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_quantile(0.5)`, func(t *testing.T) {
t.Parallel()
q := `range_quantile(0.5, time())`
@@ -7289,6 +7339,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_quantile(0.5, time() > 1200 < 2000)`, func(t *testing.T) {
t.Parallel()
q := `range_quantile(0.5, time() > 1200 < 2000)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{1600, 1600, 1600, 1600, 1600, 1600},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_stddev()`, func(t *testing.T) {
t.Parallel()
q := `round(range_stddev(time()),0.01)`
@@ -7300,6 +7361,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_stddev(time() > 1200 < 1800)`, func(t *testing.T) {
t.Parallel()
q := `round(range_stddev(time() > 1200 < 1800),0.01)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{100, 100, 100, 100, 100, 100},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_stdvar()`, func(t *testing.T) {
t.Parallel()
q := `round(range_stdvar(time()),0.01)`
@@ -7311,6 +7383,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_stdvar(time() > 1200 < 1800)`, func(t *testing.T) {
t.Parallel()
q := `round(range_stdvar(time() > 1200 < 1800),0.01)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{10000, 10000, 10000, 10000, 10000, 10000},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_median()`, func(t *testing.T) {
t.Parallel()
q := `range_median(time())`
@@ -7671,6 +7754,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`running_min(abs(1500-time()) < 400 > 100)`, func(t *testing.T) {
t.Parallel()
q := `running_min(abs(1500-time()) < 400 > 100)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{nan, 300, 300, 300, 300, 300},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`running_max(abs(1300-time()))`, func(t *testing.T) {
t.Parallel()
q := `running_max(abs(1300-time()))`
@@ -7682,6 +7776,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`running_max(abs(1300-time()) > 300 < 700)`, func(t *testing.T) {
t.Parallel()
q := `running_max(abs(1300-time()) > 300 < 700)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{nan, nan, nan, nan, 500, 500},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`running_sum(1)`, func(t *testing.T) {
t.Parallel()
q := `running_sum(1)`
@@ -7704,6 +7809,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`running_sum(time() > 1.2 < 1.8)`, func(t *testing.T) {
t.Parallel()
q := `running_sum(time()/1e3 > 1.2 < 1.8)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{nan, nan, 1.4, 3, 3, 3},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`running_avg(time())`, func(t *testing.T) {
t.Parallel()
q := `running_avg(time())`
@@ -7715,6 +7831,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`running_avg(time() > 1200 < 1800)`, func(t *testing.T) {
t.Parallel()
q := `running_avg(time() > 1200 < 1800)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{nan, nan, 1400, 1500, 1500, 1500},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`smooth_exponential(time(), 1)`, func(t *testing.T) {
t.Parallel()
q := `smooth_exponential(time(), 1)`
@@ -7795,6 +7922,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_min(time() > 1200 < 1800)`, func(t *testing.T) {
t.Parallel()
q := `range_min(time() > 1200 < 1800)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{1400, 1400, 1400, 1400, 1400, 1400},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_normalize(time(),alias(-time(),"negative"))`, func(t *testing.T) {
t.Parallel()
q := `range_normalize(time(),alias(-time(), "negative"))`
@@ -7812,6 +7950,23 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r1, r2}
f(q, resultExpected)
})
t.Run(`range_normalize(time() > 1200 < 1800,alias(-(time() > 1400 < 2000),"negative"))`, func(t *testing.T) {
t.Parallel()
q := `range_normalize(time() > 1200 < 1800,alias(-(time() > 1200 < 2000), "negative"))`
r1 := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{nan, nan, 0, 1, nan, nan},
Timestamps: timestampsExpected,
}
r2 := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{nan, nan, 1, 0.5, 0, nan},
Timestamps: timestampsExpected,
}
r2.MetricName.MetricGroup = []byte("negative")
resultExpected := []netstorage.Result{r1, r2}
f(q, resultExpected)
})
t.Run(`range_first(time())`, func(t *testing.T) {
t.Parallel()
q := `range_first(time())`
@@ -7823,6 +7978,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_first(time() > 1200 < 1800)`, func(t *testing.T) {
t.Parallel()
q := `range_first(time() > 1200 < 1800)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{1400, 1400, 1400, 1400, 1400, 1400},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_mad(time())`, func(t *testing.T) {
t.Parallel()
q := `range_mad(time())`
@@ -7834,6 +8000,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_mad(time() > 1200 < 1800)`, func(t *testing.T) {
t.Parallel()
q := `range_mad(time() > 1200 < 1800)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{100, 100, 100, 100, 100, 100},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_max(time())`, func(t *testing.T) {
t.Parallel()
q := `range_max(time())`
@@ -7845,6 +8022,39 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_max(time() > 1200 < 1800)`, func(t *testing.T) {
t.Parallel()
q := `range_max(time() > 1200 < 1800)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{1600, 1600, 1600, 1600, 1600, 1600},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_sum(time())`, func(t *testing.T) {
t.Parallel()
q := `range_sum(time())`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{9000, 9000, 9000, 9000, 9000, 9000},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_sum(time() > 1200 < 1800)`, func(t *testing.T) {
t.Parallel()
q := `range_sum(time() > 1200 < 1800)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{3000, 3000, 3000, 3000, 3000, 3000},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_last(time())`, func(t *testing.T) {
t.Parallel()
q := `range_last(time())`
@@ -7856,6 +8066,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_last(time() > 1200 < 1800)`, func(t *testing.T) {
t.Parallel()
q := `range_last(time() > 1200 < 1800)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{1600, 1600, 1600, 1600, 1600, 1600},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_linear_regression(time())`, func(t *testing.T) {
t.Parallel()
q := `range_linear_regression(time())`
@@ -7878,6 +8099,17 @@ func TestExecSuccess(t *testing.T) {
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_linear_regression(time() > 1200 < 1800)`, func(t *testing.T) {
t.Parallel()
q := `range_linear_regression(time() > 1200 < 1800)`
r := netstorage.Result{
MetricName: metricNameExpected,
Values: []float64{1000, 1200, 1400, 1600, 1800, 2000},
Timestamps: timestampsExpected,
}
resultExpected := []netstorage.Result{r}
f(q, resultExpected)
})
t.Run(`range_linear_regression(100/time())`, func(t *testing.T) {
t.Parallel()
q := `sort_desc(round((

View File

@@ -1544,10 +1544,8 @@ func transformRangeFirst(tfa *transformFuncArg) ([]*timeseries, error) {
continue
}
vFirst := values[0]
for i, v := range values {
if math.IsNaN(v) {
continue
}
values = ts.Values
for i := range values {
values[i] = vFirst
}
}
@@ -1571,10 +1569,8 @@ func setLastValues(tss []*timeseries) {
continue
}
vLast := values[len(values)-1]
for i, v := range values {
if math.IsNaN(v) {
continue
}
values = ts.Values
for i := range values {
values[i] = vLast
}
}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 14 KiB

View File

@@ -1,13 +1,13 @@
{
"files": {
"main.css": "./static/css/main.81c6ec3a.css",
"main.js": "./static/js/main.9ed27532.js",
"static/js/685.bebe1265.chunk.js": "./static/js/685.bebe1265.chunk.js",
"static/media/MetricsQL.md": "./static/media/MetricsQL.8c2e588d62b87f90dbf0.md",
"main.css": "./static/css/main.d871147a.css",
"main.js": "./static/js/main.621c4b4d.js",
"static/js/685.f772060c.chunk.js": "./static/js/685.f772060c.chunk.js",
"static/media/MetricsQL.md": "./static/media/MetricsQL.a00044c91d9781cf8557.md",
"index.html": "./index.html"
},
"entrypoints": [
"static/css/main.81c6ec3a.css",
"static/js/main.9ed27532.js"
"static/css/main.d871147a.css",
"static/js/main.621c4b4d.js"
]
}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.6 KiB

View File

@@ -0,0 +1 @@
<svg width="48" height="48" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M24.5475 0C10.3246.0265251 1.11379 3.06365 4.40623 6.10077c0 0 12.32997 11.23333 16.58217 14.84083.8131.6896 2.1728 1.1936 3.5191 1.2201h.1199c1.3463-.0265 2.706-.5305 3.5191-1.2201 4.2522-3.5942 16.5422-14.84083 16.5422-14.84083C48.0478 3.06365 38.8636.0265251 24.6674 0" fill="#020202"/><path d="M28.1579 27.0159c-.8131.6896-2.1728 1.1936-3.5191 1.2201h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2201-2.9725-2.5067-13.35639-11.87-17.26201-15.3979v5.4112c0 .5968.22661 1.3793.6265 1.7506C7.00358 21.1936 17.2675 30.5437 20.9731 33.6737c.8132.6896 2.1728 1.1936 3.5191 1.2201h.12c1.3463-.0265 2.7059-.5305 3.519-1.2201 3.679-3.13 13.9429-12.4536 16.6089-14.8939.4132-.3713.6265-1.1538.6265-1.7506V11.618c-3.9323 3.5411-14.3162 12.931-17.2354 15.3979h.0267Z" fill="#020202"/><path d="M28.1579 39.748c-.8131.6897-2.1728 1.1937-3.5191 1.2202h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2202-2.9725-2.4933-13.35639-11.8567-17.26201-15.3978v5.4111c0 .5969.22661 1.3793.6265 1.7507C7.00358 33.9258 17.2675 43.2759 20.9731 46.4058c.8132.6897 2.1728 1.1937 3.5191 1.2202h.12c1.3463-.0265 2.7059-.5305 3.519-1.2202 3.679-3.1299 13.9429-12.4535 16.6089-14.8938.4132-.3714.6265-1.1538.6265-1.7507v-5.4111c-3.9323 3.5411-14.3162 12.931-17.2354 15.3978h.0267Z" fill="#020202"/></svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

View File

@@ -1 +1 @@
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="UI for VictoriaMetrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><script src="./dashboards/index.js" type="module"></script><meta name="twitter:card" content="summary_large_image"><meta name="twitter:image" content="./preview.jpg"><meta name="twitter:title" content="UI for VictoriaMetrics"><meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta name="twitter:site" content="@VictoriaMetrics"><meta property="og:title" content="Metric explorer for VictoriaMetrics"><meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta property="og:image" content="./preview.jpg"><meta property="og:type" content="website"><script defer="defer" src="./static/js/main.9ed27532.js"></script><link href="./static/css/main.81c6ec3a.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.svg"/><link rel="apple-touch-icon" href="./favicon.svg"/><link rel="mask-icon" href="./favicon.svg" color="#000000"><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="Explore and troubleshoot your VictoriaMetrics data"/><link rel="manifest" href="./manifest.json"/><title>vmui</title><script src="./dashboards/index.js" type="module"></script><meta name="twitter:card" content="summary"><meta name="twitter:title" content="UI for VictoriaMetrics"><meta name="twitter:site" content="@https://victoriametrics.com/"><meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta name="twitter:image" content="./preview.jpg"><meta property="og:type" content="website"><meta property="og:title" content="UI for VictoriaMetrics"><meta property="og:url" content="https://victoriametrics.com/"><meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data"><script defer="defer" src="./static/js/main.621c4b4d.js"></script><link href="./static/css/main.d871147a.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>

View File

@@ -1,16 +1,11 @@
{
"short_name": "Victoria Metrics UI",
"name": "Victoria Metrics UI is a metric explorer for Victoria Metrics",
"short_name": "vmui",
"name": "vmui",
"icons": [
{
"src": "favicon-32x32.png",
"sizes": "32x32",
"type": "image/png"
},
{
"src": "apple-touch-icon.png",
"type": "image/png",
"sizes": "192x192"
"src": "favicon.svg",
"sizes": "any",
"type": "image/svg+xml"
}
],
"start_url": ".",

Binary file not shown.

Before

Width:  |  Height:  |  Size: 67 KiB

After

Width:  |  Height:  |  Size: 6.5 KiB

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -5,7 +5,7 @@
*/
/**
* @remix-run/router v1.17.0
* @remix-run/router v1.19.2
*
* Copyright (c) Remix Software Inc.
*
@@ -16,7 +16,7 @@
*/
/**
* React Router DOM v6.24.0
* React Router DOM v6.26.2
*
* Copyright (c) Remix Software Inc.
*
@@ -27,7 +27,7 @@
*/
/**
* React Router v6.24.0
* React Router v6.26.2
*
* Copyright (c) Remix Software Inc.
*

File diff suppressed because one or more lines are too long

View File

@@ -103,7 +103,7 @@ The list of MetricsQL features on top of PromQL:
* Trailing commas on all the lists are allowed - label filters, function args and with expressions.
For instance, the following queries are valid: `m{foo="bar",}`, `f(a, b,)`, `WITH (x=y,) x`.
This simplifies maintenance of multi-line queries.
* Metric names and label names may contain any unicode letter. For example `температура{город="Київ"}` is a valid MetricsQL expression.
* Metric names and label names may contain any unicode letter. For example `ტემპერატურა{πόλη="Київ"}` is a valid MetricsQL expression.
* Metric names and labels names may contain escaped chars. For example, `foo\-bar{baz\=aa="b"}` is valid expression.
It returns time series with name `foo-bar` containing label `baz=aa` with value `b`.
Additionally, the following escape sequences are supported:

View File

@@ -1,55 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<link rel="icon" href="%PUBLIC_URL%/favicon.ico"/>
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=5"/>
<meta name="theme-color" content="#000000"/>
<meta
name="description"
content="UI for VictoriaMetrics"
/>
<link rel="apple-touch-icon" href="%PUBLIC_URL%/apple-touch-icon.png"/>
<link rel="icon" type="image/png" sizes="32x32" href="%PUBLIC_URL%/favicon-32x32.png">
<!--
manifest.json provides metadata used when your web app is installed on a
user's mobile device or desktop. See https://developers.google.com/web/fundamentals/web-app-manifest/
-->
<link rel="manifest" href="%PUBLIC_URL%/manifest.json"/>
<!--
Notice the use of %PUBLIC_URL% in the tags above.
It will be replaced with the URL of the `public` folder during the build.
Only files inside the `public` folder can be referenced from the HTML.
Unlike "/favicon.ico" or "favicon.ico", "%PUBLIC_URL%/favicon.ico" will
work correctly both with client-side routing and a non-root public URL.
Learn how to configure a non-root public URL by running `npm run build`.
-->
<title>VM UI</title>
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:image" content="%PUBLIC_URL%/preview.jpg">
<meta name="twitter:title" content="UI for VictoriaMetrics">
<meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data">
<meta name="twitter:site" content="@VictoriaMetrics">
<meta property="og:title" content="Metric explorer for VictoriaMetrics">
<meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data">
<meta property="og:image" content="%PUBLIC_URL%/preview.jpg">
<meta property="og:type" content="website">
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div id="root"></div>
<!--
This HTML file is a template.
If you open it directly in the browser, you will see an empty page.
You can add webfonts, meta tags, or analytics to this file.
The build step will place the bundled scripts into the <body> tag.
To begin the development, run `npm start` or `yarn start`.
To create a production bundle, use `npm run build` or `yarn build`.
-->
</body>
</html>

View File

@@ -171,9 +171,9 @@ func RegisterMetricNames(qt *querytracer.Tracer, mrs []storage.MetricRow) {
// DeleteSeries deletes series matching tfss.
//
// Returns the number of deleted series.
func DeleteSeries(qt *querytracer.Tracer, tfss []*storage.TagFilters) (int, error) {
func DeleteSeries(qt *querytracer.Tracer, tfss []*storage.TagFilters, maxMetrics int) (int, error) {
WG.Add(1)
n, err := Storage.DeleteSeries(qt, tfss)
n, err := Storage.DeleteSeries(qt, tfss, maxMetrics)
WG.Done()
return n, err
}
@@ -521,12 +521,20 @@ func writeStorageMetrics(w io.Writer, strg *storage.Storage) {
metrics.WriteGaugeUint64(w, `vm_data_size_bytes{type="indexdb/inmemory"}`, idbm.InmemorySizeBytes)
metrics.WriteGaugeUint64(w, `vm_data_size_bytes{type="indexdb/file"}`, idbm.FileSizeBytes)
metrics.WriteCounterUint64(w, `vm_rows_received_by_storage_total`, m.RowsReceivedTotal)
metrics.WriteCounterUint64(w, `vm_rows_added_to_storage_total`, m.RowsAddedTotal)
metrics.WriteCounterUint64(w, `vm_deduplicated_samples_total{type="merge"}`, m.DedupsDuringMerge)
metrics.WriteGaugeUint64(w, `vm_snapshots`, m.SnapshotsCount)
metrics.WriteCounterUint64(w, `vm_rows_ignored_total{reason="big_timestamp"}`, m.TooBigTimestampRows)
metrics.WriteCounterUint64(w, `vm_rows_ignored_total{reason="small_timestamp"}`, m.TooSmallTimestampRows)
metrics.WriteCounterUint64(w, `vm_rows_ignored_total{reason="invalid_raw_metric_name"}`, m.InvalidRawMetricNames)
if *maxHourlySeries > 0 {
metrics.WriteCounterUint64(w, `vm_rows_ignored_total{reason="hourly_limit_exceeded"}`, m.HourlySeriesLimitRowsDropped)
}
if *maxDailySeries > 0 {
metrics.WriteCounterUint64(w, `vm_rows_ignored_total{reason="daily_limit_exceeded"}`, m.DailySeriesLimitRowsDropped)
}
metrics.WriteCounterUint64(w, `vm_timeseries_repopulated_total`, m.TimeseriesRepopulated)
metrics.WriteCounterUint64(w, `vm_timeseries_precreated_total`, m.TimeseriesPreCreated)

View File

@@ -1,4 +1,4 @@
FROM golang:1.23.0 AS build-web-stage
FROM golang:1.23.1 AS build-web-stage
COPY build /build
WORKDIR /build
@@ -6,7 +6,7 @@ COPY web/ /build/
RUN GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o web-amd64 github.com/VictoriMetrics/vmui/ && \
GOOS=windows GOARCH=amd64 CGO_ENABLED=0 go build -o web-windows github.com/VictoriMetrics/vmui/
FROM alpine:3.20.2
FROM alpine:3.20.3
USER root
COPY --from=build-web-stage /build/web-amd64 /app/web

View File

@@ -10,6 +10,7 @@
# production
/build
/public/index.html
# misc
.DS_Store

View File

@@ -8,11 +8,11 @@ const path = require('path');
const pathIndexHTML = (() => {
switch (process.env.REACT_APP_TYPE) {
case 'logs':
return 'public/victorialogs.html';
return 'src/html/victorialogs.html';
case 'anomaly':
return 'public/vmanomaly.html';
return 'src/html/vmanomaly.html';
default:
return 'public/victoriametrics.html';
return 'src/html/victoriametrics.html';
}
})();
const fileContent = fs.readFileSync(path.resolve(__dirname, pathIndexHTML), 'utf8');

View File

@@ -8,45 +8,45 @@
"name": "vmui",
"version": "0.1.0",
"dependencies": {
"@types/lodash.debounce": "^4.0.6",
"@types/lodash.get": "^4.4.6",
"@types/lodash.throttle": "^4.1.6",
"@types/node": "^20.4.0",
"@types/qs": "^6.9.7",
"@types/react-input-mask": "^3.0.2",
"@types/lodash.debounce": "^4.0.9",
"@types/lodash.get": "^4.4.9",
"@types/lodash.throttle": "^4.1.9",
"@types/node": "^22.5.4",
"@types/qs": "^6.9.15",
"@types/react-input-mask": "^3.0.5",
"@types/react-router-dom": "^5.3.3",
"@types/webpack-env": "^1.16.3",
"classnames": "^2.3.2",
"dayjs": "^1.11.0",
"@types/webpack-env": "^1.18.5",
"classnames": "^2.5.1",
"dayjs": "^1.11.13",
"lodash.debounce": "^4.0.8",
"lodash.get": "^4.4.2",
"lodash.throttle": "^4.1.1",
"marked": "^12.0.2",
"marked-emoji": "^1.4.0",
"preact": "^10.7.1",
"qs": "^6.10.3",
"marked": "^14.1.2",
"marked-emoji": "^1.4.2",
"preact": "^10.23.2",
"qs": "^6.13.0",
"react-input-mask": "^2.0.4",
"react-router-dom": "^6.10.0",
"sass": "^1.56.0",
"react-router-dom": "^6.26.2",
"sass": "^1.78.0",
"source-map-explorer": "^2.5.3",
"typescript": "~4.6.2",
"uplot": "^1.6.30",
"web-vitals": "^3.3.2"
"web-vitals": "^4.2.3"
},
"devDependencies": {
"@babel/plugin-proposal-nullish-coalescing-operator": "^7.16.7",
"@babel/plugin-proposal-private-property-in-object": "^7.21.11",
"@types/react-dom": "^18.2.6",
"@types/react-dom": "^18.3.0",
"@typescript-eslint/eslint-plugin": "^5.15.0",
"@typescript-eslint/parser": "^5.15.0",
"cross-env": "^7.0.3",
"customize-cra": "^1.0.0",
"eslint": "^8.44.0",
"eslint-config-react-app": "^7.0.1",
"eslint-plugin-react": "^7.29.4",
"http-proxy-middleware": "^3.0.0",
"eslint-plugin-react": "^7.36.1",
"http-proxy-middleware": "^3.0.2",
"react-app-rewired": "^2.2.1",
"webpack": "^5.88.1"
"webpack": "^5.94.0"
}
},
"node_modules/@alloc/quick-lru": {
@@ -3651,9 +3651,9 @@
}
},
"node_modules/@remix-run/router": {
"version": "1.17.0",
"resolved": "https://registry.npmjs.org/@remix-run/router/-/router-1.17.0.tgz",
"integrity": "sha512-2D6XaHEVvkCn682XBnipbJjgZUU7xjLtA4dGJRBVUKpEaDYOZMENZoZjAOSb7qirxt5RupjzZxz4fK2FO+EFPw==",
"version": "1.19.2",
"resolved": "https://registry.npmjs.org/@remix-run/router/-/router-1.19.2.tgz",
"integrity": "sha512-baiMx18+IMuD1yyvOGaHM9QrVUPGGG0jC+z+IPHnRJWUAUvaKuWKyE8gjDj2rzv3sz9zOGoRSPgeBVHRhZnBlA==",
"engines": {
"node": ">=14.0.0"
}
@@ -4135,21 +4135,12 @@
"resolved": "https://registry.npmjs.org/@types/eslint/-/eslint-8.56.10.tgz",
"integrity": "sha512-Shavhk87gCtY2fhXDctcfS3e6FdxWkCx1iUZ9eEUbh7rTqlZT0/IzOkCOVt0fCjcFuZ9FPYfuezTBImfHCDBGQ==",
"dev": true,
"peer": true,
"dependencies": {
"@types/estree": "*",
"@types/json-schema": "*"
}
},
"node_modules/@types/eslint-scope": {
"version": "3.7.7",
"resolved": "https://registry.npmjs.org/@types/eslint-scope/-/eslint-scope-3.7.7.tgz",
"integrity": "sha512-MzMFlSLBqNF2gcHWO0G1vP/YQyfvrxZ0bF+u7mzUdZ1/xK4A4sru+nraZz5i3iEIk1l1uyicaDVTB4QbbEkAYg==",
"dev": true,
"dependencies": {
"@types/eslint": "*",
"@types/estree": "*"
}
},
"node_modules/@types/estree": {
"version": "1.0.5",
"resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.5.tgz",
@@ -4212,9 +4203,9 @@
"peer": true
},
"node_modules/@types/http-proxy": {
"version": "1.17.14",
"resolved": "https://registry.npmjs.org/@types/http-proxy/-/http-proxy-1.17.14.tgz",
"integrity": "sha512-SSrD0c1OQzlFX7pGu1eXxSEjemej64aaNPRhhVYUGqXh0BtldAAx37MG8btcumvpgKyZp1F5Gn3JkktdxiFv6w==",
"version": "1.17.15",
"resolved": "https://registry.npmjs.org/@types/http-proxy/-/http-proxy-1.17.15.tgz",
"integrity": "sha512-25g5atgiVNTIv0LBDTg1H74Hvayx0ajtJPLLcYE3whFv75J0pWNtOBzaXJQgDTmrX1bx5U9YC2w/n65BN1HwRQ==",
"dev": true,
"dependencies": {
"@types/node": "*"
@@ -4296,11 +4287,11 @@
"peer": true
},
"node_modules/@types/node": {
"version": "20.14.8",
"resolved": "https://registry.npmjs.org/@types/node/-/node-20.14.8.tgz",
"integrity": "sha512-DO+2/jZinXfROG7j7WKFn/3C6nFwxy2lLpgLjEXJz+0XKphZlTLJ14mo8Vfg8X5BWN6XjyESXq+LcYdT7tR3bA==",
"version": "22.5.4",
"resolved": "https://registry.npmjs.org/@types/node/-/node-22.5.4.tgz",
"integrity": "sha512-FDuKUJQm/ju9fT/SeX/6+gBzoPzlVCzfzmGkwKvRHQVxi4BntVbyIwf6a4Xn62mrvndLiml6z/UBXIdEVjQLXg==",
"dependencies": {
"undici-types": "~5.26.4"
"undici-types": "~6.19.2"
}
},
"node_modules/@types/node-forge": {
@@ -5357,18 +5348,6 @@
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/array.prototype.toreversed": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/array.prototype.toreversed/-/array.prototype.toreversed-1.1.2.tgz",
"integrity": "sha512-wwDCoT4Ck4Cz7sLtgUmzR5UV3YF5mFHUlbChCzZBQZ+0m2cl/DH3tKgvphv1nKgFsJ48oCSg6p91q2Vm0I/ZMA==",
"dev": true,
"dependencies": {
"call-bind": "^1.0.2",
"define-properties": "^1.2.0",
"es-abstract": "^1.22.1",
"es-shim-unscopables": "^1.0.0"
}
},
"node_modules/array.prototype.tosorted": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/array.prototype.tosorted/-/array.prototype.tosorted-1.1.4.tgz",
@@ -7143,17 +7122,17 @@
}
},
"node_modules/dayjs": {
"version": "1.11.11",
"resolved": "https://registry.npmjs.org/dayjs/-/dayjs-1.11.11.tgz",
"integrity": "sha512-okzr3f11N6WuqYtZSvm+F776mB41wRZMhKP+hc34YdW+KmtYYK9iqvHSwo2k9FEH3fhGXvOPV6yz2IcSrfRUDg=="
"version": "1.11.13",
"resolved": "https://registry.npmjs.org/dayjs/-/dayjs-1.11.13.tgz",
"integrity": "sha512-oaMBel6gjolK862uaPQOVTA7q3TZhuSvuMQAAglQDOWYO9A91IrAOUJEyKVlqJlHE0vq5p5UXxzdPfMH/x6xNg=="
},
"node_modules/debug": {
"version": "4.3.5",
"resolved": "https://registry.npmjs.org/debug/-/debug-4.3.5.tgz",
"integrity": "sha512-pt0bNEmneDIvdL1Xsd9oDQ/wrQRkXDT4AUWlNZNPKvW5x/jyO9VFXkJUP07vQ2upmw5PlaITaPKc31jK13V+jg==",
"version": "4.3.7",
"resolved": "https://registry.npmjs.org/debug/-/debug-4.3.7.tgz",
"integrity": "sha512-Er2nc/H7RrMXZBFCEim6TCmMk02Z8vLC2Rbi1KEBggpo0fS6l0S1nnapwmIi3yW/+GOJap1Krg4w0Hg80oCqgQ==",
"dev": true,
"dependencies": {
"ms": "2.1.2"
"ms": "^2.1.3"
},
"engines": {
"node": ">=6.0"
@@ -7626,9 +7605,9 @@
}
},
"node_modules/enhanced-resolve": {
"version": "5.17.0",
"resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-5.17.0.tgz",
"integrity": "sha512-dwDPwZL0dmye8Txp2gzFmA6sxALaSvdRDjPH0viLcKrtlOL3tw62nWWweVD1SdILDTJrbrL6tdWVN58Wo6U3eA==",
"version": "5.17.1",
"resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-5.17.1.tgz",
"integrity": "sha512-LMHl3dXhTcfv8gM4kEzIUeTQ+7fpdA0l2tUf34BddXPkz2A5xJ5L/Pchd5BL6rdccM9QGvu0sWZzK1Z1t4wwyg==",
"dev": true,
"dependencies": {
"graceful-fs": "^4.2.4",
@@ -8165,35 +8144,35 @@
}
},
"node_modules/eslint-plugin-react": {
"version": "7.34.3",
"resolved": "https://registry.npmjs.org/eslint-plugin-react/-/eslint-plugin-react-7.34.3.tgz",
"integrity": "sha512-aoW4MV891jkUulwDApQbPYTVZmeuSyFrudpbTAQuj5Fv8VL+o6df2xIGpw8B0hPjAaih1/Fb0om9grCdyFYemA==",
"version": "7.36.1",
"resolved": "https://registry.npmjs.org/eslint-plugin-react/-/eslint-plugin-react-7.36.1.tgz",
"integrity": "sha512-/qwbqNXZoq+VP30s1d4Nc1C5GTxjJQjk4Jzs4Wq2qzxFM7dSmuG2UkIjg2USMLh3A/aVcUNrK7v0J5U1XEGGwA==",
"dev": true,
"dependencies": {
"array-includes": "^3.1.8",
"array.prototype.findlast": "^1.2.5",
"array.prototype.flatmap": "^1.3.2",
"array.prototype.toreversed": "^1.1.2",
"array.prototype.tosorted": "^1.1.4",
"doctrine": "^2.1.0",
"es-iterator-helpers": "^1.0.19",
"estraverse": "^5.3.0",
"hasown": "^2.0.2",
"jsx-ast-utils": "^2.4.1 || ^3.0.0",
"minimatch": "^3.1.2",
"object.entries": "^1.1.8",
"object.fromentries": "^2.0.8",
"object.hasown": "^1.1.4",
"object.values": "^1.2.0",
"prop-types": "^15.8.1",
"resolve": "^2.0.0-next.5",
"semver": "^6.3.1",
"string.prototype.matchall": "^4.0.11"
"string.prototype.matchall": "^4.0.11",
"string.prototype.repeat": "^1.0.0"
},
"engines": {
"node": ">=4"
},
"peerDependencies": {
"eslint": "^3 || ^4 || ^5 || ^6 || ^7 || ^8"
"eslint": "^3 || ^4 || ^5 || ^6 || ^7 || ^8 || ^9.7"
}
},
"node_modules/eslint-plugin-react-hooks": {
@@ -9931,17 +9910,17 @@
}
},
"node_modules/http-proxy-middleware": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-3.0.0.tgz",
"integrity": "sha512-36AV1fIaI2cWRzHo+rbcxhe3M3jUDCNzc4D5zRl57sEWRAxdXYtw7FSQKYY6PDKssiAKjLYypbssHk+xs/kMXw==",
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-3.0.2.tgz",
"integrity": "sha512-fBLFpmvDzlxdckwZRjM0wWtwDZ4KBtQ8NFqhrFKoEtK4myzuiumBuNTxD+F4cVbXfOZljIbrynmvByofDzT7Ag==",
"dev": true,
"dependencies": {
"@types/http-proxy": "^1.17.10",
"debug": "^4.3.4",
"@types/http-proxy": "^1.17.15",
"debug": "^4.3.6",
"http-proxy": "^1.18.1",
"is-glob": "^4.0.1",
"is-plain-obj": "^3.0.0",
"micromatch": "^4.0.5"
"is-glob": "^4.0.3",
"is-plain-object": "^5.0.0",
"micromatch": "^4.0.8"
},
"engines": {
"node": "^14.15.0 || ^16.10.0 || >=18.0.0"
@@ -10446,6 +10425,7 @@
"resolved": "https://registry.npmjs.org/is-plain-obj/-/is-plain-obj-3.0.0.tgz",
"integrity": "sha512-gwsOE28k+23GP1B6vFl1oVh/WOzmawBrKwo5Ev6wMKzPkaXaCDIQKzLnvsA42DRlbVTWorkgTKIviAKCWkfUwA==",
"dev": true,
"peer": true,
"engines": {
"node": ">=10"
},
@@ -10453,6 +10433,15 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/is-plain-object": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/is-plain-object/-/is-plain-object-5.0.0.tgz",
"integrity": "sha512-VRSzKkbMm5jMDoKLbltAkFQ5Qr7VDiTFGXxYFXXowVj387GeGNOCsOH6Msy00SGZ3Fp84b1Naa1psqgcCIEP5Q==",
"dev": true,
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/is-potential-custom-element-name": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/is-potential-custom-element-name/-/is-potential-custom-element-name-1.0.1.tgz",
@@ -13541,9 +13530,9 @@
}
},
"node_modules/marked": {
"version": "12.0.2",
"resolved": "https://registry.npmjs.org/marked/-/marked-12.0.2.tgz",
"integrity": "sha512-qXUm7e/YKFoqFPYPa3Ukg9xlI5cyAtGmyEIzMfW//m6kXwCy2Ps9DYf5ioijFKQ8qyuscrHoY04iJGctu2Kg0Q==",
"version": "14.1.2",
"resolved": "https://registry.npmjs.org/marked/-/marked-14.1.2.tgz",
"integrity": "sha512-f3r0yqpz31VXiDB/wj9GaOB0a2PRLQl6vJmXiFrniNwjkKdvakqJRULhjFKJpxOchlCRiG5fcacoUZY5Xa6PEQ==",
"bin": {
"marked": "bin/marked.js"
},
@@ -13552,11 +13541,11 @@
}
},
"node_modules/marked-emoji": {
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/marked-emoji/-/marked-emoji-1.4.1.tgz",
"integrity": "sha512-3xHWQn8XD1LyhMpHxWpHTDWBZ9bpXLlW8JIqvyXTO6he7okKIB/W9fD/3fTg0DQuZlSQvPZ6Ub5hN6Rnmn7j9g==",
"version": "1.4.2",
"resolved": "https://registry.npmjs.org/marked-emoji/-/marked-emoji-1.4.2.tgz",
"integrity": "sha512-2sP+bp2z76dwbILzQ7ijy2PyjjAJR3iAZCzaNGThD2UijFUBeidkn6MoCdX/j47tPIcWt9nwnjqRQPd01ZrfdA==",
"peerDependencies": {
"marked": ">=4 <14"
"marked": ">=4 <15"
}
},
"node_modules/mdn-data": {
@@ -13622,9 +13611,9 @@
}
},
"node_modules/micromatch": {
"version": "4.0.7",
"resolved": "https://registry.npmjs.org/micromatch/-/micromatch-4.0.7.tgz",
"integrity": "sha512-LPP/3KorzCwBxfeUuZmaR6bG2kdeHSbe0P2tY3FLRU4vYrjYz5hI4QZwV0njUx3jeuKe67YukQ1LSPZBKDqO/Q==",
"version": "4.0.8",
"resolved": "https://registry.npmjs.org/micromatch/-/micromatch-4.0.8.tgz",
"integrity": "sha512-PXwfBhYu0hBCPw8Dn0E+WDYb7af3dSLVWKi3HGv84IdF4TyFoC0ysxFd0Goxw7nSv4T/PzEJQxsYsEiFCKo2BA==",
"dev": true,
"dependencies": {
"braces": "^3.0.3",
@@ -13747,9 +13736,9 @@
}
},
"node_modules/ms": {
"version": "2.1.2",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz",
"integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==",
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
"dev": true
},
"node_modules/multicast-dns": {
@@ -14064,23 +14053,6 @@
"node": ">= 0.4"
}
},
"node_modules/object.hasown": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/object.hasown/-/object.hasown-1.1.4.tgz",
"integrity": "sha512-FZ9LZt9/RHzGySlBARE3VF+gE26TxR38SdmqOqliuTnl9wrKulaQs+4dee1V+Io8VfxqzAfHu6YuRgUy8OHoTg==",
"dev": true,
"dependencies": {
"define-properties": "^1.2.1",
"es-abstract": "^1.23.2",
"es-object-atoms": "^1.0.0"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/object.values": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/object.values/-/object.values-1.2.0.tgz",
@@ -15962,9 +15934,9 @@
"peer": true
},
"node_modules/preact": {
"version": "10.22.0",
"resolved": "https://registry.npmjs.org/preact/-/preact-10.22.0.tgz",
"integrity": "sha512-RRurnSjJPj4rp5K6XoP45Ui33ncb7e4H7WiOHVpjbkvqvA3U+N8Z6Qbo0AE6leGYBV66n8EhEaFixvIu3SkxFw==",
"version": "10.23.2",
"resolved": "https://registry.npmjs.org/preact/-/preact-10.23.2.tgz",
"integrity": "sha512-kKYfePf9rzKnxOAKDpsWhg/ysrHPqT+yQ7UW4JjdnqjFIeNUnNcEJvhuA8fDenxAGWzUqtd51DfVg7xp/8T9NA==",
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/preact"
@@ -16133,9 +16105,9 @@
}
},
"node_modules/qs": {
"version": "6.12.1",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.12.1.tgz",
"integrity": "sha512-zWmv4RSuB9r2mYQw3zxQuHWeU+42aKi1wWig/j4ele4ygELZ7PEO6MM7rim9oAQH2A5MWfsAVf/jPvTPgCbvUQ==",
"version": "6.13.0",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.13.0.tgz",
"integrity": "sha512-+38qI9SOr8tfZ4QmJNplMUxqjbe7LKvvZgWdExBOmd+egZTtjLB67Gu0HRX3u/XOq7UU2Nx6nsjvS16Z9uwfpg==",
"dependencies": {
"side-channel": "^1.0.6"
},
@@ -16487,11 +16459,11 @@
}
},
"node_modules/react-router": {
"version": "6.24.0",
"resolved": "https://registry.npmjs.org/react-router/-/react-router-6.24.0.tgz",
"integrity": "sha512-sQrgJ5bXk7vbcC4BxQxeNa5UmboFm35we1AFK0VvQaz9g0LzxEIuLOhHIoZ8rnu9BO21ishGeL9no1WB76W/eg==",
"version": "6.26.2",
"resolved": "https://registry.npmjs.org/react-router/-/react-router-6.26.2.tgz",
"integrity": "sha512-tvN1iuT03kHgOFnLPfLJ8V95eijteveqdOSk+srqfePtQvqCExB8eHOYnlilbOcyJyKnYkr1vJvf7YqotAJu1A==",
"dependencies": {
"@remix-run/router": "1.17.0"
"@remix-run/router": "1.19.2"
},
"engines": {
"node": ">=14.0.0"
@@ -16501,12 +16473,12 @@
}
},
"node_modules/react-router-dom": {
"version": "6.24.0",
"resolved": "https://registry.npmjs.org/react-router-dom/-/react-router-dom-6.24.0.tgz",
"integrity": "sha512-960sKuau6/yEwS8e+NVEidYQb1hNjAYM327gjEyXlc6r3Skf2vtwuJ2l7lssdegD2YjoKG5l8MsVyeTDlVeY8g==",
"version": "6.26.2",
"resolved": "https://registry.npmjs.org/react-router-dom/-/react-router-dom-6.26.2.tgz",
"integrity": "sha512-z7YkaEW0Dy35T3/QKPYB1LjMK2R1fxnHO8kWpUMTBdfVzZrWOiY9a7CtN8HqdWtDUWd5FY6Dl8HFsqVwH4uOtQ==",
"dependencies": {
"@remix-run/router": "1.17.0",
"react-router": "6.24.0"
"@remix-run/router": "1.19.2",
"react-router": "6.26.2"
},
"engines": {
"node": ">=14.0.0"
@@ -17157,9 +17129,9 @@
"peer": true
},
"node_modules/sass": {
"version": "1.77.6",
"resolved": "https://registry.npmjs.org/sass/-/sass-1.77.6.tgz",
"integrity": "sha512-ByXE1oLD79GVq9Ht1PeHWCPMPB8XHpBuz1r85oByKHjZY6qV6rWnQovQzXJXuQ/XyE1Oj3iPk3lo28uzaRA2/Q==",
"version": "1.78.0",
"resolved": "https://registry.npmjs.org/sass/-/sass-1.78.0.tgz",
"integrity": "sha512-AaIqGSrjo5lA2Yg7RvFZrlXDBCp3nV4XP73GrLGvdRWWwk+8H3l0SDvq/5bA4eF+0RFPLuWUk3E+P1U/YqnpsQ==",
"dependencies": {
"chokidar": ">=3.0.0 <4.0.0",
"immutable": "^4.0.0",
@@ -17369,13 +17341,6 @@
"dev": true,
"peer": true
},
"node_modules/send/node_modules/ms": {
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
"dev": true,
"peer": true
},
"node_modules/serialize-javascript": {
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/serialize-javascript/-/serialize-javascript-6.0.2.tgz",
@@ -18105,6 +18070,16 @@
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/string.prototype.repeat": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/string.prototype.repeat/-/string.prototype.repeat-1.0.0.tgz",
"integrity": "sha512-0u/TldDbKD8bFCQ/4f5+mNRrXwZ8hg2w7ZR8wa16e8z9XpePWl3eGEcUD0OXpEH/VJH/2G3gjUtR3ZOiBe2S/w==",
"dev": true,
"dependencies": {
"define-properties": "^1.1.3",
"es-abstract": "^1.17.5"
}
},
"node_modules/string.prototype.trim": {
"version": "1.2.9",
"resolved": "https://registry.npmjs.org/string.prototype.trim/-/string.prototype.trim-1.2.9.tgz",
@@ -19118,9 +19093,9 @@
"peer": true
},
"node_modules/undici-types": {
"version": "5.26.5",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-5.26.5.tgz",
"integrity": "sha512-JlCMO+ehdEIKqlFxk6IfVoAUVmgz7cU7zD/h9XZ0qzeosSHmUJVOzSQvvYSYWXkFXC+IfLKSIffhv0sVZup6pA=="
"version": "6.19.8",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.19.8.tgz",
"integrity": "sha512-ve2KP6f/JnbPBFyobGHuerC9g1FYGn/F8n1LWTwNxCEzd6IfqTwUQcNXgEtmmQ6DlRrC1hrSrBnCZPokRrDHjw=="
},
"node_modules/unicode-canonical-property-names-ecmascript": {
"version": "2.0.0",
@@ -19416,9 +19391,9 @@
}
},
"node_modules/web-vitals": {
"version": "3.5.2",
"resolved": "https://registry.npmjs.org/web-vitals/-/web-vitals-3.5.2.tgz",
"integrity": "sha512-c0rhqNcHXRkY/ogGDJQxZ9Im9D19hDihbzSQJrsioex+KnFgmMzBiy57Z1EjkhX/+OjyBpclDCzz2ITtjokFmg=="
"version": "4.2.3",
"resolved": "https://registry.npmjs.org/web-vitals/-/web-vitals-4.2.3.tgz",
"integrity": "sha512-/CFAm1mNxSmOj6i0Co+iGFJ58OS4NRGVP+AWS/l509uIK5a1bSoIVaHz/ZumpHTfHSZBpgrJ+wjfpAOrTHok5Q=="
},
"node_modules/webidl-conversions": {
"version": "6.1.0",
@@ -19431,12 +19406,11 @@
}
},
"node_modules/webpack": {
"version": "5.92.1",
"resolved": "https://registry.npmjs.org/webpack/-/webpack-5.92.1.tgz",
"integrity": "sha512-JECQ7IwJb+7fgUFBlrJzbyu3GEuNBcdqr1LD7IbSzwkSmIevTm8PF+wej3Oxuz/JFBUZ6O1o43zsPkwm1C4TmA==",
"version": "5.94.0",
"resolved": "https://registry.npmjs.org/webpack/-/webpack-5.94.0.tgz",
"integrity": "sha512-KcsGn50VT+06JH/iunZJedYGUJS5FGjow8wb9c0v5n1Om8O1g4L6LjtfxwlXIATopoQu+vOXXa7gYisWxCoPyg==",
"dev": true,
"dependencies": {
"@types/eslint-scope": "^3.7.3",
"@types/estree": "^1.0.5",
"@webassemblyjs/ast": "^1.12.1",
"@webassemblyjs/wasm-edit": "^1.12.1",
@@ -19445,7 +19419,7 @@
"acorn-import-attributes": "^1.9.5",
"browserslist": "^4.21.10",
"chrome-trace-event": "^1.0.2",
"enhanced-resolve": "^5.17.0",
"enhanced-resolve": "^5.17.1",
"es-module-lexer": "^1.2.1",
"eslint-scope": "5.1.1",
"events": "^3.2.0",

View File

@@ -4,30 +4,30 @@
"private": true,
"homepage": "./",
"dependencies": {
"@types/lodash.debounce": "^4.0.6",
"@types/lodash.get": "^4.4.6",
"@types/lodash.throttle": "^4.1.6",
"@types/node": "^20.4.0",
"@types/qs": "^6.9.7",
"@types/react-input-mask": "^3.0.2",
"@types/lodash.debounce": "^4.0.9",
"@types/lodash.get": "^4.4.9",
"@types/lodash.throttle": "^4.1.9",
"@types/node": "^22.5.4",
"@types/qs": "^6.9.15",
"@types/react-input-mask": "^3.0.5",
"@types/react-router-dom": "^5.3.3",
"@types/webpack-env": "^1.16.3",
"classnames": "^2.3.2",
"dayjs": "^1.11.0",
"@types/webpack-env": "^1.18.5",
"classnames": "^2.5.1",
"dayjs": "^1.11.13",
"lodash.debounce": "^4.0.8",
"lodash.get": "^4.4.2",
"lodash.throttle": "^4.1.1",
"marked": "^12.0.2",
"marked-emoji": "^1.4.0",
"preact": "^10.7.1",
"qs": "^6.10.3",
"marked": "^14.1.2",
"marked-emoji": "^1.4.2",
"preact": "^10.23.2",
"qs": "^6.13.0",
"react-input-mask": "^2.0.4",
"react-router-dom": "^6.10.0",
"sass": "^1.56.0",
"react-router-dom": "^6.26.2",
"sass": "^1.78.0",
"source-map-explorer": "^2.5.3",
"typescript": "~4.6.2",
"uplot": "^1.6.30",
"web-vitals": "^3.3.2"
"web-vitals": "^4.2.3"
},
"scripts": {
"prestart": "npm run copy-metricsql-docs",
@@ -63,17 +63,17 @@
"devDependencies": {
"@babel/plugin-proposal-nullish-coalescing-operator": "^7.16.7",
"@babel/plugin-proposal-private-property-in-object": "^7.21.11",
"@types/react-dom": "^18.2.6",
"@types/react-dom": "^18.3.0",
"@typescript-eslint/eslint-plugin": "^5.15.0",
"@typescript-eslint/parser": "^5.15.0",
"cross-env": "^7.0.3",
"customize-cra": "^1.0.0",
"eslint": "^8.44.0",
"eslint-config-react-app": "^7.0.1",
"eslint-plugin-react": "^7.29.4",
"http-proxy-middleware": "^3.0.0",
"eslint-plugin-react": "^7.36.1",
"http-proxy-middleware": "^3.0.2",
"react-app-rewired": "^2.2.1",
"webpack": "^5.88.1"
"webpack": "^5.94.0"
},
"overrides": {
"react-app-rewired": {

Binary file not shown.

Before

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.6 KiB

View File

@@ -0,0 +1 @@
<svg width="48" height="48" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M24.5475 0C10.3246.0265251 1.11379 3.06365 4.40623 6.10077c0 0 12.32997 11.23333 16.58217 14.84083.8131.6896 2.1728 1.1936 3.5191 1.2201h.1199c1.3463-.0265 2.706-.5305 3.5191-1.2201 4.2522-3.5942 16.5422-14.84083 16.5422-14.84083C48.0478 3.06365 38.8636.0265251 24.6674 0" fill="#020202"/><path d="M28.1579 27.0159c-.8131.6896-2.1728 1.1936-3.5191 1.2201h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2201-2.9725-2.5067-13.35639-11.87-17.26201-15.3979v5.4112c0 .5968.22661 1.3793.6265 1.7506C7.00358 21.1936 17.2675 30.5437 20.9731 33.6737c.8132.6896 2.1728 1.1936 3.5191 1.2201h.12c1.3463-.0265 2.7059-.5305 3.519-1.2201 3.679-3.13 13.9429-12.4536 16.6089-14.8939.4132-.3713.6265-1.1538.6265-1.7506V11.618c-3.9323 3.5411-14.3162 12.931-17.2354 15.3979h.0267Z" fill="#020202"/><path d="M28.1579 39.748c-.8131.6897-2.1728 1.1937-3.5191 1.2202h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2202-2.9725-2.4933-13.35639-11.8567-17.26201-15.3978v5.4111c0 .5969.22661 1.3793.6265 1.7507C7.00358 33.9258 17.2675 43.2759 20.9731 46.4058c.8132.6897 2.1728 1.1937 3.5191 1.2202h.12c1.3463-.0265 2.7059-.5305 3.519-1.2202 3.679-3.1299 13.9429-12.4535 16.6089-14.8938.4132-.3714.6265-1.1538.6265-1.7507v-5.4111c-3.9323 3.5411-14.3162 12.931-17.2354 15.3978h.0267Z" fill="#020202"/></svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

View File

@@ -1,16 +1,11 @@
{
"short_name": "Victoria Metrics UI",
"name": "Victoria Metrics UI is a metric explorer for Victoria Metrics",
"short_name": "vmui",
"name": "vmui",
"icons": [
{
"src": "favicon-32x32.png",
"sizes": "32x32",
"type": "image/png"
},
{
"src": "apple-touch-icon.png",
"type": "image/png",
"sizes": "192x192"
"src": "favicon.svg",
"sizes": "any",
"type": "image/svg+xml"
}
],
"start_url": ".",

Binary file not shown.

Before

Width:  |  Height:  |  Size: 67 KiB

After

Width:  |  Height:  |  Size: 6.5 KiB

View File

@@ -1,55 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<link rel="icon" href="%PUBLIC_URL%/favicon.ico"/>
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=5"/>
<meta name="theme-color" content="#000000"/>
<meta
name="description"
content="UI for VictoriaMetrics"
/>
<link rel="apple-touch-icon" href="%PUBLIC_URL%/apple-touch-icon.png"/>
<link rel="icon" type="image/png" sizes="32x32" href="%PUBLIC_URL%/favicon-32x32.png">
<!--
manifest.json provides metadata used when your web app is installed on a
user's mobile device or desktop. See https://developers.google.com/web/fundamentals/web-app-manifest/
-->
<link rel="manifest" href="%PUBLIC_URL%/manifest.json"/>
<!--
Notice the use of %PUBLIC_URL% in the tags above.
It will be replaced with the URL of the `public` folder during the build.
Only files inside the `public` folder can be referenced from the HTML.
Unlike "/favicon.ico" or "favicon.ico", "%PUBLIC_URL%/favicon.ico" will
work correctly both with client-side routing and a non-root public URL.
Learn how to configure a non-root public URL by running `npm run build`.
-->
<title>VM UI</title>
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:image" content="%PUBLIC_URL%/preview.jpg">
<meta name="twitter:title" content="UI for VictoriaMetrics">
<meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data">
<meta name="twitter:site" content="@VictoriaMetrics">
<meta property="og:title" content="Metric explorer for VictoriaMetrics">
<meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data">
<meta property="og:image" content="%PUBLIC_URL%/preview.jpg">
<meta property="og:type" content="website">
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div id="root"></div>
<!--
This HTML file is a template.
If you open it directly in the browser, you will see an empty page.
You can add webfonts, meta tags, or analytics to this file.
The build step will place the bundled scripts into the <body> tag.
To begin the development, run `npm start` or `yarn start`.
To create a production bundle, use `npm run build` or `yarn build`.
-->
</body>
</html>

View File

@@ -1,56 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<link rel="icon" href="%PUBLIC_URL%/favicon.ico"/>
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=5"/>
<meta name="theme-color" content="#000000"/>
<meta
name="description"
content="UI for VictoriaMetrics"
/>
<link rel="apple-touch-icon" href="%PUBLIC_URL%/apple-touch-icon.png"/>
<link rel="icon" type="image/png" sizes="32x32" href="%PUBLIC_URL%/favicon-32x32.png">
<!--
manifest.json provides metadata used when your web app is installed on a
user's mobile device or desktop. See https://developers.google.com/web/fundamentals/web-app-manifest/
-->
<link rel="manifest" href="%PUBLIC_URL%/manifest.json"/>
<!--
Notice the use of %PUBLIC_URL% in the tags above.
It will be replaced with the URL of the `public` folder during the build.
Only files inside the `public` folder can be referenced from the HTML.
Unlike "/favicon.ico" or "favicon.ico", "%PUBLIC_URL%/favicon.ico" will
work correctly both with client-side routing and a non-root public URL.
Learn how to configure a non-root public URL by running `npm run build`.
-->
<title>VM UI</title>
<script src="%PUBLIC_URL%/dashboards/index.js" type="module"></script>
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:image" content="%PUBLIC_URL%/preview.jpg">
<meta name="twitter:title" content="UI for VictoriaMetrics">
<meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data">
<meta name="twitter:site" content="@VictoriaMetrics">
<meta property="og:title" content="Metric explorer for VictoriaMetrics">
<meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data">
<meta property="og:image" content="%PUBLIC_URL%/preview.jpg">
<meta property="og:type" content="website">
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div id="root"></div>
<!--
This HTML file is a template.
If you open it directly in the browser, you will see an empty page.
You can add webfonts, meta tags, or analytics to this file.
The build step will place the bundled scripts into the <body> tag.
To begin the development, run `npm start` or `yarn start`.
To create a production bundle, use `npm run build` or `yarn build`.
-->
</body>
</html>

View File

@@ -1,56 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<link rel="icon" href="%PUBLIC_URL%/favicon.ico"/>
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=5"/>
<meta name="theme-color" content="#000000"/>
<meta
name="description"
content="UI for VictoriaMetrics"
/>
<link rel="apple-touch-icon" href="%PUBLIC_URL%/apple-touch-icon.png"/>
<link rel="icon" type="image/png" sizes="32x32" href="%PUBLIC_URL%/favicon-32x32.png">
<!--
manifest.json provides metadata used when your web app is installed on a
user's mobile device or desktop. See https://developers.google.com/web/fundamentals/web-app-manifest/
-->
<link rel="manifest" href="%PUBLIC_URL%/manifest.json"/>
<!--
Notice the use of %PUBLIC_URL% in the tags above.
It will be replaced with the URL of the `public` folder during the build.
Only files inside the `public` folder can be referenced from the HTML.
Unlike "/favicon.ico" or "favicon.ico", "%PUBLIC_URL%/favicon.ico" will
work correctly both with client-side routing and a non-root public URL.
Learn how to configure a non-root public URL by running `npm run build`.
-->
<title>VM UI</title>
<script src="%PUBLIC_URL%/dashboards/index.js" type="module"></script>
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:image" content="%PUBLIC_URL%/preview.jpg">
<meta name="twitter:title" content="UI for VictoriaMetrics">
<meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data">
<meta name="twitter:site" content="@VictoriaMetrics">
<meta property="og:title" content="Metric explorer for VictoriaMetrics">
<meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data">
<meta property="og:image" content="%PUBLIC_URL%/preview.jpg">
<meta property="og:type" content="website">
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div id="root"></div>
<!--
This HTML file is a template.
If you open it directly in the browser, you will see an empty page.
You can add webfonts, meta tags, or analytics to this file.
The build step will place the bundled scripts into the <body> tag.
To begin the development, run `npm start` or `yarn start`.
To create a production bundle, use `npm run build` or `yarn build`.
-->
</body>
</html>

Some files were not shown because too many files have changed in this diff Show More