Compare commits

..

46 Commits

Author SHA1 Message Date
Max Kotliar
83aef6f86d fix link 2025-06-27 15:09:27 +03:00
Max Kotliar
e08534a1c7 docs: [traces] add active development notice 2025-06-27 15:01:48 +03:00
Max Kotliar
4aef00c2a9 docs: [traces] add active development notice 2025-06-27 14:58:25 +03:00
Max Kotliar
88c617c51b docs: [traces] small fixes 2025-06-27 14:46:50 +03:00
Max Kotliar
5a053d5714 docs: [traces] upd quick start 2025-06-27 14:19:44 +03:00
Jiekun
a79a7cea1b docs: [traces] remove typo 2025-06-26 14:07:33 +08:00
Jiekun
58469efd0d docs: [traces] remove vmui 2025-06-26 14:01:17 +08:00
Jiekun
0767fe1367 docs: [traces] add jaeger nginx visualization 2025-06-26 14:00:45 +08:00
Jiekun
d395ddec2d docs: [traces] add visualization 2025-06-26 12:04:37 +08:00
Jiekun
235a789a96 docs: [traces] remove typo 2025-06-26 11:09:44 +08:00
Jiekun
06bf32d1f6 docs: [traces] update links 2025-06-26 11:06:15 +08:00
Jiekun
ff32d15560 docs: [traces] update links 2025-06-26 10:54:31 +08:00
Jiekun
84044db837 docs: [traces] update after review 2025-06-26 10:52:24 +08:00
Jiekun
f1e6aaa949 docs: [traces] update after review 2025-06-26 10:41:47 +08:00
Max Kotliar
27f2cf84e5 docs/victoriatraces: enhance first sentence 2025-06-25 14:22:08 +03:00
Max Kotliar
8fff6e4173 docs/victoriatraces: fix title and postion in left side menu. 2025-06-25 14:01:46 +03:00
Jiekun
9de6941147 docs: [traces] compress img and move to querying part 2025-06-24 18:03:41 +08:00
Jiekun
8a2e208d1b docs: [traces] update key concept and make json collapse 2025-06-24 17:51:28 +08:00
Jiekun
d01cffd3e7 docs: [traces] update how does it work part for traces doc 2025-06-24 17:37:34 +08:00
Jiekun
96a6443be1 docs: [traces] update readme for traces after review 2025-06-24 12:49:28 +08:00
Zhu Jiekun
ebbb921d90 Apply suggestions from code review
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2025-06-24 11:45:16 +08:00
Jiekun
f8ce411a70 docs: [traces] init docs for vt 2025-06-23 15:11:47 +08:00
Aliaksandr Valialkin
1e3791cbf6 lib/storage: move testing/synctest usage into separate files ending with _synctest_test.go
The files with synctest usage are built only if the goexperiment.synctest build tag is set.
This allows running `go test ./lib/...` and `go vet ./lib/...` without the need to pass GOEXPERIMENT=synctest to them.

This is a follow-up for d33d7e20be
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8848
2025-06-21 12:27:22 +02:00
Aliaksandr Valialkin
b1f3d7f3eb app/vlinsert/journald: add a benchark for the isValidFieldName() function 2025-06-21 12:15:45 +02:00
Aliaksandr Valialkin
c2c6f97bd3 lib/fasttime: remove performance overhead when UnixTimestamp() is called from benchmark loops
Move the synctest-related implementation into a separate file protected with go:build goexperiment.synctest.

This is a follow-up for the commit 06c26315df
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8740
2025-06-21 12:09:55 +02:00
Aliaksandr Valialkin
274e38ebee docs/victorialogs/data-ingestion/Journald.md: mention that _TRANSPORT and _SYSTEMD_USER_UNIT fields are also good candidates for log streams
Thanks to @septatrix for the suggestion at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9143#issuecomment-2983752442
2025-06-20 23:26:16 +02:00
Aliaksandr Valialkin
e05738749c docs/victorialogs/CHANGELOG.md: typo fixes in links to stats functions 2025-06-20 22:25:57 +02:00
Aliaksandr Valialkin
3428d0d27d deployment: update VictoriaLogs Docker image tags from v1.23.3-victorialogs to v1.24.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.24.0-victorialogs
2025-06-20 21:29:54 +02:00
Aliaksandr Valialkin
f8170b2471 docs/victorialogs/CHANGELOG.md: cut v1.24.0-victorialogs 2025-06-20 21:20:35 +02:00
Aliaksandr Valialkin
57925c34e6 app/vlselect/vmui: run make vmui-logs-update after 9e60fc8fc8 2025-06-20 21:19:07 +02:00
Artur Minchukou
9e60fc8fc8 app/vmui/logs: move compact mode of VictoriaLogs table to separate tab (#8745)
### Describe Your Changes

Related issue: #7047 and #8438

- removed compact mode of VictoriaLogs table
- added sorting by field to JSON tab 

![image](https://github.com/user-attachments/assets/c02bbbed-cf2c-41e3-86fe-97bc205654a5)
- added sorting of field names to JSON tab


### Checklist

The following checks are **mandatory**:

- [ ] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-06-20 17:26:28 +02:00
f41gh7
07c8a4a9a7 make vmui-update 2025-06-20 17:04:17 +02:00
f41gh7
e098ef901f CHANGELOG.md: cut v1.120.0 release 2025-06-20 16:41:00 +02:00
f41gh7
7693c4bcce docker/Makefile: add EXTRA_TAG_SUFFIX variable to buildx publish
New variable should help to change docker image tag name during release prepare.

Instead of publishing an exact version, like v1.1.0, it should use suffixed version v1.1.0-rc1.
Which should be re-tagged later, when release will be promoted to stable.

Related to https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9136

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2025-06-20 16:39:27 +02:00
Fred Navruzov
8bb56f8ef5 docs/vmanomaly: release v1.24.1 (#9241)
### Describe Your Changes

Patch release doc updates for vmanomaly (v1.24.1)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
2025-06-20 16:35:02 +02:00
Alexander Frolov
b91d249a29 lib/httpserver: option for disabling HTTP keep-alives (#9125)
### Describe Your Changes

Some network configurations may not work optimally with long-lived
connections. \
Address the issue described by #2395 for other components.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).

---------

Signed-off-by: Aleksandr Frolov <fxrlv@nebius.com>
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
Co-authored-by: Max Kotliar <kotlyar.maksim@gmail.com>
Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2025-06-20 12:43:21 +02:00
Hui Wang
7b274e0d6d vmalert: correct the rule evaluation timestamp if the system clock is… (#9228)
… changed during runtime

Strip the monotonic clock with `t.Round(d)` or `t.Truncate(d)` before
apply `Sub()`, to force the wall clock usage which corrects the rule
evaluation timestamp if the system clock is changed during runtime.

fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8790

---------

Co-authored-by: Max Kotliar <kotlyar.maksim@gmail.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2025-06-20 12:17:31 +02:00
Benjamin Nichols-Farquhar
b3c92540e5 vmalert: respect group.concurrency in replay mode (#9214)
### Describe Your Changes

Revival and modification of original PR
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8762 after
discussion on
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7387.

`group.concurrency` is now respected if and only if
`-replay.rulesDelay=0` rather than always. This allows rules to be run
concurrently without ambiguity about rule chaining. If
`-replay.rulesDelay` is set greater than zero concurrency is still
ignored. This will be the default behavior since it defaults to 1s.

Implementation considerations:

I chose to add split some simple logic into a helper function in
preparation for adding `replay.singleRuleEvaluationConcurrency` in a
follow up PR as thats we're we can share that logic.

cc @Haleygo 
### Checklist

The following checks are **mandatory**:

- [x] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).

---------

Co-authored-by: Hui Wang <haley@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2025-06-20 12:14:18 +02:00
Artem Fetishev
0dd63a60d0 lib/storage: Change BenchmarkHeadPostingForMatchers to use global index time range explicitly (#9233)
During the data retrieval, VictoriaMetrics switches to global index
search if the time range is > 40 days. Prior
ba0d7dc2fc, this switch was handled in
indexDB code. That commit moved that logic to the storage level. As the
result, indexDB will search per-day index for whatever time range that
is passed to it.

This broke the BenchmarkHeadPostingForMatchers. It now shows very slow
indexDB performance compared to v1.111.0 (the last release before that
commit). This is because now indexDB tries to search 55 years of data by
spawning a separate goroutine for each day. Even though the data was
inserted only for the current day, running that many goroutines
significantly slows down the search.

The fix is to use global index time range explicitly.

Fixes #9203 

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2025-06-20 11:55:04 +02:00
Benjamin Nichols-Farquhar
32fc801519 vmstorage: add flag for storage metricName cache tuning (#9156)
Similar to other storage caches `storage/metricName` can be very
important to performance, however it is not tunable independently like
other caches.

In high cardinality setups where a large amount of that cardinality is
actively queried we can see a high `metricName` miss rate.

The only way to correct this is to increase available memory either by
provisioning more or by increasing `memory.allowedPercent`, which is
often expensive or undesirable for stability reasons.

Its possible to work around this by increasing `memory.allowedPercent`
and then adjusting `storage.cacheSizeIndexDBDataBlocks` and
`storage.cacheSizeStorageTSID` down as they are the largest caches, but
this is a less ideal solution than being able to directly control this
cache size.

Related to
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8843
2025-06-20 11:33:39 +02:00
Artem Fetishev
771f742842 lib/storage: extract adding metricIDs to the pendingHourEntries into a separate func (#9138)
This change has been requested to be done in a separate PR during the
partition index review. See:
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8134#discussion_r2131692079

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2025-06-20 09:58:11 +02:00
Max Kotliar
a845fe815a lib/logstorage: clarify comment on writeBlockResultFunc usage constraints (#9235)
### Describe Your Changes

The `DataBlock` contains structs with string fields, and while the
original comment mentioned not holding references to `br`, it wasn't
immediately clear that this also applies to fields like strings within
the data.

This change clarifies that the `writeBlockResultFunc` must not retain
references to any part of `br`, including its fields. This makes it
explicit that even seemingly safe types like strings must be copied if
needed.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
2025-06-19 16:30:56 +02:00
Phuong Le
2eadd4c9f9 vlogs: fix inconsistent type for HTTP request duration metric (#9225)
The 'select' HTTP request duration metric is currently implemented as a
summary, while the 'insert' HTTP request duration uses a histogram.

All time series under the same metric name must use the same metric
type. Using the summary type for this metric is preferable, as it
significantly reduces the number of unique time series generated. While
summaries have the limitation of not supporting accurate aggregation
across multiple instances or paths, this trade-off is more manageable
than dealing with a high-cardinality explosion caused by histograms.
2025-06-19 14:49:29 +02:00
Aliaksandr Valialkin
eb7b088c91 lib/logstorage: provide standard string representation for all the priority and severity levels in Syslog and Journald protocols inside the "level" field
It is better from usability PoV to provide string representation for all the priority and severity levels
instead of merging some of them into a common groups.

This is requested at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9209
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8535
2025-06-19 14:39:20 +02:00
Aliaksandr Valialkin
19a50ae7cd app/vlinsert/journald: follow-up for 01d413873e
- Add more tests, which cover various edge cases for binary encoding of log field value in journald format
- Moved common code for reading the next line to fieldsBuf.value. This simplifies the code a bit.
- Added more comments, which try explaining the braindead logic for parsing binary-encoded log field values in journald format

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9070
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9153
2025-06-19 13:50:46 +02:00
Andrii Chubatiuk
01d413873e app/vlinsert/journald: fixed binary value parsing (#9234)
### Describe Your Changes

- removed unneeded loop for binary field value size extraction, since it
should not be delimited by multiple `\n` symbols
- changed loop condition

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
2025-06-19 13:47:56 +02:00
123 changed files with 3497 additions and 2052 deletions

View File

@@ -101,7 +101,7 @@ func datadogLogsIngestion(w http.ResponseWriter, r *http.Request) bool {
var (
v2LogsRequestsTotal = metrics.NewCounter(`vl_http_requests_total{path="/insert/datadog/api/v2/logs"}`)
v2LogsRequestDuration = metrics.NewHistogram(`vl_http_request_duration_seconds{path="/insert/datadog/api/v2/logs"}`)
v2LogsRequestDuration = metrics.NewSummary(`vl_http_request_duration_seconds{path="/insert/datadog/api/v2/logs"}`)
)
// datadog message field has two formats:

View File

@@ -129,7 +129,7 @@ func RequestHandler(path string, w http.ResponseWriter, r *http.Request) bool {
var (
bulkRequestsTotal = metrics.NewCounter(`vl_http_requests_total{path="/insert/elasticsearch/_bulk"}`)
bulkRequestDuration = metrics.NewHistogram(`vl_http_request_duration_seconds{path="/insert/elasticsearch/_bulk"}`)
bulkRequestDuration = metrics.NewSummary(`vl_http_request_duration_seconds{path="/insert/elasticsearch/_bulk"}`)
)
func readBulkRequest(streamName string, r io.Reader, encoding string, timeFields, msgFields []string, lmp insertutil.LogMessageProcessor) (int, error) {

View File

@@ -84,5 +84,5 @@ var (
requestsTotal = metrics.NewCounter(`vl_http_requests_total{path="/internal/insert"}`)
errorsTotal = metrics.NewCounter(`vl_http_errors_total{path="/internal/insert"}`)
requestDuration = metrics.NewHistogram(`vl_http_request_duration_seconds{path="/internal/insert"}`)
requestDuration = metrics.NewSummary(`vl_http_request_duration_seconds{path="/internal/insert"}`)
)

View File

@@ -28,24 +28,6 @@ import (
// See https://github.com/systemd/systemd/blob/main/src/libsystemd/sd-journal/journal-file.c#L1703
const maxFieldNameLen = 64
func isValidJournaldFieldName(s string) bool {
if len(s) == 0 {
return false
}
c := s[0]
if !(c >= 'A' && c <= 'Z' || c == '_') {
return false
}
for i := 1; i < len(s); i++ {
c := s[i]
if !(c >= 'A' && c <= 'Z' || c >= '0' && c <= '9' || c == '_') {
return false
}
}
return true
}
var (
journaldStreamFields = flagutil.NewArrayString("journald.streamFields", "Comma-separated list of fields to use as log stream fields for logs ingested over journald protocol. "+
"See https://docs.victoriametrics.com/victorialogs/data-ingestion/journald/#stream-fields")
@@ -163,7 +145,7 @@ func handleJournald(r *http.Request, w http.ResponseWriter) {
var (
requestsTotal = metrics.NewCounter(`vl_http_requests_total{path="/insert/journald/upload"}`)
errorsTotal = metrics.NewCounter(`vl_http_errors_total{path="/insert/journald/upload"}`)
requestDuration = metrics.NewHistogram(`vl_http_request_duration_seconds{path="/insert/journald/upload"}`)
requestDuration = metrics.NewSummary(`vl_http_request_duration_seconds{path="/insert/journald/upload"}`)
)
func processStreamInternal(streamName string, r io.Reader, lmp insertutil.LogMessageProcessor, cp *insertutil.CommonParams) error {
@@ -214,6 +196,18 @@ func (fb *fieldsBuf) addField(name, value string) {
})
}
func (fb *fieldsBuf) appendNextLineToValue(lr *insertutil.LineReader) error {
if !lr.NextLine() {
if err := lr.Err(); err != nil {
return err
}
return fmt.Errorf("unexpected end of stream")
}
fb.value = append(fb.value, lr.Line...)
fb.value = append(fb.value, '\n')
return nil
}
func getFieldsBuf() *fieldsBuf {
fb := fieldsBufPool.Get()
if fb == nil {
@@ -259,42 +253,35 @@ func readJournaldLogEntry(streamName string, lr *insertutil.LineReader, lmp inse
return nil
}
// line could be either "key=value\n" or "key\n<little_endian_size_64>value\n"
// line could be either "key=value" or "key"
// according to https://systemd.io/JOURNAL_EXPORT_FORMATS/#journal-export-format
if n := bytes.IndexByte(line, '='); n >= 0 {
// "key=value\n"
// line = "key=value"
fb.name = append(fb.name[:0], line[:n]...)
name = bytesutil.ToUnsafeString(fb.name)
fb.value = append(fb.value[:0], line[n+1:]...)
value = bytesutil.ToUnsafeString(fb.value)
} else {
// "key\n<little_endian_size_64>value\n"
// line = "key"
// Parse the binary-encoded value from the next line according to "key\n<little_endian_size_64>value\n" format
fb.name = append(fb.name[:0], line...)
name = bytesutil.ToUnsafeString(fb.name)
fb.value = fb.value[:0]
for len(fb.value) < 8 {
if !lr.NextLine() {
if err := lr.Err(); err != nil {
return fmt.Errorf("cannot read value size: %w", err)
}
return fmt.Errorf("unexpected end of stream while reading value size")
if err := fb.appendNextLineToValue(lr); err != nil {
return fmt.Errorf("cannot read value size: %w", err)
}
fb.value = append(fb.value, lr.Line...)
fb.value = append(fb.value, '\n')
}
size := binary.LittleEndian.Uint64(fb.value[:8])
for size > uint64(len(fb.value[8:])) {
if !lr.NextLine() {
if err := lr.Err(); err != nil {
return fmt.Errorf("cannot read %q value with size %d bytes; read only %d bytes: %w", fb.name, size, len(fb.value[8:]), err)
}
return fmt.Errorf("unexpected end of stream while reading %q value with size %d bytes; read only %d bytes", fb.name, size, len(fb.value[8:]))
// Read the value until its lenth exceeds the given size - the last char in the read value will always be '\n'
// because it is appended by appendNextLineToValue().
for uint64(len(fb.value[8:])) <= size {
if err := fb.appendNextLineToValue(lr); err != nil {
return fmt.Errorf("cannot read %q value with size %d bytes; read only %d bytes: %w", fb.name, size, len(fb.value[8:]), err)
}
fb.value = append(fb.value, lr.Line...)
fb.value = append(fb.value, '\n')
}
value = bytesutil.ToUnsafeString(fb.value[8 : len(fb.value)-1])
if uint64(len(value)) != size {
@@ -314,7 +301,7 @@ func readJournaldLogEntry(streamName string, lr *insertutil.LineReader, lmp inse
logger.Errorf("%s: field name size should not exceed %d bytes; got %d bytes: %q; skipping this field", streamName, maxFieldNameLen, len(name), name)
continue
}
if !isValidJournaldFieldName(name) {
if !isValidFieldName(name) {
logger.Errorf("%s: invalid field name %q; it must consist of `A-Z0-9_` chars and must start from non-digit char; skipping this field", streamName, name)
continue
}
@@ -350,13 +337,19 @@ func journaldPriorityToLevel(priority string) string {
// See https://wiki.archlinux.org/title/Systemd/Journal#Priority_level
// and https://grafana.com/docs/grafana/latest/explore/logs-integration/#log-level
switch priority {
case "0", "1", "2":
case "0":
return "emerg"
case "1":
return "alert"
case "2":
return "critical"
case "3":
return "error"
case "4":
return "warning"
case "5", "6":
case "5":
return "notice"
case "6":
return "info"
case "7":
return "debug"
@@ -364,3 +357,21 @@ func journaldPriorityToLevel(priority string) string {
return priority
}
}
func isValidFieldName(s string) bool {
if len(s) == 0 {
return false
}
c := s[0]
if !(c >= 'A' && c <= 'Z' || c == '_') {
return false
}
for i := 1; i < len(s); i++ {
c := s[i]
if !(c >= 'A' && c <= 'Z' || c >= '0' && c <= '9' || c == '_') {
return false
}
}
return true
}

View File

@@ -8,11 +8,11 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/app/vlinsert/insertutil"
)
func TestIsValidJournaldFieldName(t *testing.T) {
func TestIsValidFieldName(t *testing.T) {
f := func(name string, resultExpected bool) {
t.Helper()
result := isValidJournaldFieldName(name)
result := isValidFieldName(name)
if result != resultExpected {
t.Fatalf("unexpected result for isValidJournaldFieldName(%q); got %v; want %v", name, result, resultExpected)
}
@@ -102,6 +102,24 @@ func TestPushJournald_Success(t *testing.T) {
"{\"E\":\"JobStateChanged\",\"_BOOT_ID\":\"f778b6e2f7584a77b991a2366612a7b5\",\"_UID\":\"0\",\"_GID\":\"0\",\"_MACHINE_ID\":\"a4a970370c30a925df02a13c67167847\",\"_HOSTNAME\":\"ecd5e4555787\",\"_RUNTIME_SCOPE\":\"system\",\"_TRANSPORT\":\"journal\",\"_CAP_EFFECTIVE\":\"1ffffffffff\",\"_SYSTEMD_CGROUP\":\"/init.scope\",\"_SYSTEMD_UNIT\":\"init.scope\",\"_SYSTEMD_SLICE\":\"-.slice\",\"CODE_FILE\":\"\\u003cstdin>\",\"CODE_LINE\":\"1\",\"CODE_FUNC\":\"\\u003cmodule>\",\"SYSLOG_IDENTIFIER\":\"python3\",\"_COMM\":\"python3\",\"_EXE\":\"/usr/bin/python3.12\",\"_CMDLINE\":\"python3\",\"_msg\":\"foo\\nbar\\n\\n\\nasda\\nasda\",\"_PID\":\"2763\",\"_SOURCE_REALTIME_TIMESTAMP\":\"1729698775704375\"}",
)
// Parse binary data with trailing newline
f("__REALTIME_TIMESTAMP=1729698775704404\n_CMDLINE=python3\nMESSAGE\n\x14\x00\x00\x00\x00\x00\x00\x00foo\nbar\n\n\nasda\nasda\n\n_PID=2763\n\n",
[]int64{1729698775704404000},
`{"_CMDLINE":"python3","_msg":"foo\nbar\n\n\nasda\nasda\n","_PID":"2763"}`,
)
f("__REALTIME_TIMESTAMP=1729698775704404\n_CMDLINE=python3\nMESSAGE\n\x00\x00\x00\x00\x00\x00\x00\x00\n_PID=2763\n\n",
[]int64{1729698775704404000},
`{"_CMDLINE":"python3","_PID":"2763"}`,
)
f("__REALTIME_TIMESTAMP=1729698775704404\n_CMDLINE=python3\nMESSAGE\n\x0A\x00\x00\x00\x00\x00\x00\x00123456789\n\n_PID=2763\n\n",
[]int64{1729698775704404000},
`{"_CMDLINE":"python3","_msg":"123456789\n","_PID":"2763"}`,
)
f("__REALTIME_TIMESTAMP=1729698775704404\n_CMDLINE=python3\nMESSAGE\n\x0A\x00\x00\x00\x00\x00\x00\x001234567890\n_PID=2763\n\n",
[]int64{1729698775704404000},
`{"_CMDLINE":"python3","_msg":"1234567890","_PID":"2763"}`,
)
// Empty field name must be ignored
f("__REALTIME_TIMESTAMP=91723819283\na=b\n=Test message", nil, "")
f("__REALTIME_TIMESTAMP=91723819284\nMESSAGE=Test message2\n\n__REALTIME_TIMESTAMP=91723819283\n=Test message\n", []int64{91723819284000}, `{"_msg":"Test message2"}`)
@@ -138,7 +156,10 @@ func TestPushJournald_Failure(t *testing.T) {
}
// too short binary encoded message
f("__CURSOR=s=e0afe8412a6a49d2bfcf66aa7927b588;i=1f06;b=f778b6e2f7584a77b991a2366612a7b5;m=300bdfd420;t=62526e1182354;x=930dc44b370963b7\n__REALTIME_TIMESTAMP=1729698775704404\nMESSAGE\n\x13\x00\x00\x00\x00\x00\x00\x00foo\nbar\n\n\nasdaasda")
f("__CURSOR=s=e0afe8412a6a49d2bfcf66aa7927b588;i=1f06;b=f778b6e2f7584a77b991a2366612a7b5;m=300bdfd420;t=62526e1182354;x=930dc44b370963b7\n__REALTIME_TIMESTAMP=1729698775704404\nMESSAGE\n\x13\x00\x00\x00\x00\x00\x00\x00foo\nbar\n\n\nasdaasd")
f("__REALTIME_TIMESTAMP=1729698775704404\n_CMDLINE=python3\nMESSAGE\n\x00\x00\x00\x00\x00\x00\x00\x00_PID=2763\n\n")
f("__REALTIME_TIMESTAMP=1729698775704404\n_CMDLINE=python3\nMESSAGE\n\x0A\x00\x00\x00\x00\x00\x00\x001234567890_PID=2763\n\n")
f("__REALTIME_TIMESTAMP=1729698775704404\n_CMDLINE=python3\nMESSAGE\n\x0A\x00\x00\x00\x00\x00\x00\x00123456789\n_PID=2763\n\n")
// too long binary encoded message
f("__CURSOR=s=e0afe8412a6a49d2bfcf66aa7927b588;i=1f06;b=f778b6e2f7584a77b991a2366612a7b5;m=300bdfd420;t=62526e1182354;x=930dc44b370963b7\n__REALTIME_TIMESTAMP=1729698775704404\nMESSAGE\n\x13\x00\x00\x00\x00\x00\x00\x00foo\nbar\n\n\nasdaasdakljlsfd")

View File

@@ -4,12 +4,55 @@ import (
"bytes"
"encoding/binary"
"fmt"
"strings"
"testing"
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vlinsert/insertutil"
)
func BenchmarkIsValidFieldName(b *testing.B) {
b.ReportAllocs()
b.SetBytes(int64(len(benchmarkFields)))
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
for _, field := range benchmarkFields {
if !isValidFieldName(field) {
panic(fmt.Errorf("cannot validate field %q", field))
}
}
}
})
}
var benchmarkFields = strings.Split(
"E,_BOOT_ID,_UID,_GID,_MACHINE_ID,_HOSTNAME,_RUNTIME_SCOPE,_TRANSPORT,_CAP_EFFECTIVE,_SYSTEMD_CGROUP,_SYSTEMD_UNIT,"+
"_SYSTEMD_SLICE,CODE_FILE,CODE_LINE,CODE_FUNC,SYSLOG_IDENTIFIER,_COMM,_EXE,_CMDLINE,MESSAGE,_PID,_SOURCE_REALTIME_TIMESTAMP,_REALTIME_TIMESTAMP",
",")
func BenchmarkPushJournaldPerformance(b *testing.B) {
cp := &insertutil.CommonParams{
TimeFields: []string{"__REALTIME_TIMESTAMP"},
MsgFields: []string{"MESSAGE"},
}
const dataChunkSize = 1024 * 1024
data := generateJournaldData(dataChunkSize)
b.ReportAllocs()
b.SetBytes(int64(len(data)))
b.RunParallel(func(pb *testing.PB) {
r := &bytes.Reader{}
blp := &insertutil.BenchmarkLogMessageProcessor{}
for pb.Next() {
r.Reset(data)
if err := processStreamInternal("performance_test", r, blp, cp); err != nil {
panic(fmt.Errorf("unexpected error: %w", err))
}
}
})
}
func generateJournaldData(size int) []byte {
var buf []byte
timestamp := time.Now().UnixMicro()
@@ -37,26 +80,3 @@ func generateJournaldData(size int) []byte {
}
return buf
}
func BenchmarkPushJournaldPerformance(b *testing.B) {
cp := &insertutil.CommonParams{
TimeFields: []string{"__REALTIME_TIMESTAMP"},
MsgFields: []string{"MESSAGE"},
}
const dataChunkSize = 1024 * 1024
data := generateJournaldData(dataChunkSize)
b.ReportAllocs()
b.SetBytes(int64(len(data)))
b.RunParallel(func(pb *testing.PB) {
r := &bytes.Reader{}
blp := &insertutil.BenchmarkLogMessageProcessor{}
for pb.Next() {
r.Reset(data)
if err := processStreamInternal("performance_test", r, blp, cp); err != nil {
panic(fmt.Errorf("unexpected error: %w", err))
}
}
})
}

View File

@@ -119,5 +119,5 @@ var (
requestsTotal = metrics.NewCounter(`vl_http_requests_total{path="/insert/jsonline"}`)
errorsTotal = metrics.NewCounter(`vl_http_errors_total{path="/insert/jsonline"}`)
requestDuration = metrics.NewHistogram(`vl_http_request_duration_seconds{path="/insert/jsonline"}`)
requestDuration = metrics.NewSummary(`vl_http_request_duration_seconds{path="/insert/jsonline"}`)
)

View File

@@ -58,7 +58,7 @@ func handleJSON(r *http.Request, w http.ResponseWriter) {
var (
requestsJSONTotal = metrics.NewCounter(`vl_http_requests_total{path="/insert/loki/api/v1/push",format="json"}`)
requestJSONDuration = metrics.NewHistogram(`vl_http_request_duration_seconds{path="/insert/loki/api/v1/push",format="json"}`)
requestJSONDuration = metrics.NewSummary(`vl_http_request_duration_seconds{path="/insert/loki/api/v1/push",format="json"}`)
)
func parseJSONRequest(data []byte, lmp insertutil.LogMessageProcessor, msgFields []string, useDefaultStreamFields, parseMessage bool) error {

View File

@@ -62,7 +62,7 @@ func handleProtobuf(r *http.Request, w http.ResponseWriter) {
var (
requestsProtobufTotal = metrics.NewCounter(`vl_http_requests_total{path="/insert/loki/api/v1/push",format="protobuf"}`)
requestProtobufDuration = metrics.NewHistogram(`vl_http_request_duration_seconds{path="/insert/loki/api/v1/push",format="protobuf"}`)
requestProtobufDuration = metrics.NewSummary(`vl_http_request_duration_seconds{path="/insert/loki/api/v1/push",format="protobuf"}`)
)
func parseProtobufRequest(data []byte, lmp insertutil.LogMessageProcessor, msgFields []string, useDefaultStreamFields, parseMessage bool) error {

View File

@@ -70,7 +70,7 @@ var (
requestsProtobufTotal = metrics.NewCounter(`vl_http_requests_total{path="/insert/opentelemetry/v1/logs",format="protobuf"}`)
errorsTotal = metrics.NewCounter(`vl_http_errors_total{path="/insert/opentelemetry/v1/logs",format="protobuf"}`)
requestProtobufDuration = metrics.NewHistogram(`vl_http_request_duration_seconds{path="/insert/opentelemetry/v1/logs",format="protobuf"}`)
requestProtobufDuration = metrics.NewSummary(`vl_http_request_duration_seconds{path="/insert/opentelemetry/v1/logs",format="protobuf"}`)
)
func pushProtobufRequest(data []byte, lmp insertutil.LogMessageProcessor, msgFields []string, useDefaultStreamFields bool) error {

View File

@@ -101,7 +101,7 @@ func TestProcessStreamInternal_Success(t *testing.T) {
currentYear := 2023
timestampsExpected := []int64{1685794113000000000, 1685880513000000000, 1685814132345000000}
resultExpected := `{"format":"rfc3164","hostname":"abcd","app_name":"systemd","_msg":"Starting Update the local ESM caches..."}
{"priority":"165","facility_keyword":"local4","level":"info","facility":"20","severity":"5","format":"rfc3164","hostname":"abcd","app_name":"systemd","proc_id":"345","_msg":"abc defg"}
{"priority":"165","facility_keyword":"local4","level":"notice","facility":"20","severity":"5","format":"rfc3164","hostname":"abcd","app_name":"systemd","proc_id":"345","_msg":"abc defg"}
{"priority":"123","facility_keyword":"solaris-cron","level":"error","facility":"15","severity":"3","format":"rfc5424","hostname":"mymachine.example.com","app_name":"appname","proc_id":"12345","msg_id":"ID47","exampleSDID@32473.iut":"3","exampleSDID@32473.eventSource":"Application 123 = ] 56","exampleSDID@32473.eventID":"11211","_msg":"This is a test message with structured data."}`
f(data, currentYear, timestampsExpected, resultExpected)
}

View File

@@ -742,7 +742,23 @@ Metric names are stripped from the resulting rollups. Add [keep_metric_names](#k
This function is supported by PromQL.
See also [irate](#irate) and [rollup_rate](#rollup_rate).
See also [irate](#irate), [rollup_rate](#rollup_rate) and [rate_prometheus](#rate_prometheus).
#### rate_prometheus
`rate_prometheus(series_selector[d])` {{% available_from "#" %}} is a [rollup function](#rollup-functions), which calculates the average per-second
increase rate over the given lookbehind window `d` per each time series returned from the given [series_selector](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#filtering).
The resulting calculation is equivalent to `increase_prometheus(series_selector[d]) / d`.
It doesn't take into account the last sample before the given lookbehind window `d` when calculating the result in the same way as Prometheus does.
See [this article](https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e) for details.
Metric names are stripped from the resulting rollups. Add [keep_metric_names](#keep_metric_names) modifier in order to keep metric names.
This function is usually applied to [counters](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#counter).
See also [increase_prometheus](#increase_prometheus) and [rate](#rate).
#### rate_over_sum

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -35,10 +35,10 @@
<meta property="og:title" content="UI for VictoriaLogs">
<meta property="og:url" content="https://victoriametrics.com/products/victorialogs/">
<meta property="og:description" content="Explore your log data with VictoriaLogs UI">
<script type="module" crossorigin src="./assets/index-DhqzKCNf.js"></script>
<link rel="modulepreload" crossorigin href="./assets/vendor-D8IJGiEn.js">
<script type="module" crossorigin src="./assets/index-721xTF8u.js"></script>
<link rel="modulepreload" crossorigin href="./assets/vendor-V4vnRsM-.js">
<link rel="stylesheet" crossorigin href="./assets/vendor-D1GxaB_c.css">
<link rel="stylesheet" crossorigin href="./assets/index-D5re9hC6.css">
<link rel="stylesheet" crossorigin href="./assets/index-C36SC0pJ.css">
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>

View File

@@ -20,9 +20,8 @@ var (
"The time filter in RFC3339 format to finish the replay by. E.g. '2020-01-01T20:07:00Z'. "+
"By default, is set to the current time.")
replayRulesDelay = flag.Duration("replay.rulesDelay", time.Second,
"Delay between rules evaluation within the group. Could be important if there are chained rules inside the group "+
"and processing need to wait for previous rule results to be persisted by remote storage before evaluating the next rule."+
"Keep it equal or bigger than -remoteWrite.flushInterval.")
"Delay before evaluating the next rule within the group. Is important for chained rules. "+
"Keep it equal or bigger than -remoteWrite.flushInterval. When set to >0, replay ignores group's concurrency setting.")
replayMaxDatapoints = flag.Int("replay.maxDatapointsPerQuery", 1e3,
"Max number of data points expected in one request. It affects the max time range for every '/query_range' request during the replay. The higher the value, the less requests will be made during replay.")
replayRuleRetryAttempts = flag.Int("replay.ruleRetryAttempts", 5,

View File

@@ -39,7 +39,7 @@ func (fr *fakeReplayQuerier) QueryRange(_ context.Context, q string, from, to ti
}
func TestReplay(t *testing.T) {
f := func(from, to string, maxDP int, cfg []config.Group, qb *fakeReplayQuerier) {
f := func(from, to string, maxDP int, ruleDelay time.Duration, cfg []config.Group, qb *fakeReplayQuerier) {
t.Helper()
fromOrig, toOrig, maxDatapointsOrig := *replayFrom, *replayTo, *replayMaxDatapoints
@@ -51,7 +51,7 @@ func TestReplay(t *testing.T) {
}()
*replayRuleRetryAttempts = 1
*replayRulesDelay = time.Millisecond
*replayRulesDelay = ruleDelay
rwb := &remotewrite.DebugClient{}
*replayFrom = from
*replayTo = to
@@ -65,7 +65,7 @@ func TestReplay(t *testing.T) {
}
// one rule + one response
f("2021-01-01T12:00:00.000Z", "2021-01-01T12:02:00.000Z", 10, []config.Group{
f("2021-01-01T12:00:00.000Z", "2021-01-01T12:02:00.000Z", 10, time.Millisecond, []config.Group{
{Rules: []config.Rule{{Record: "foo", Expr: "sum(up)"}}},
}, &fakeReplayQuerier{
registry: map[string]map[string]struct{}{
@@ -74,7 +74,7 @@ func TestReplay(t *testing.T) {
})
// one rule + multiple responses
f("2021-01-01T12:00:00.000Z", "2021-01-01T12:02:30.000Z", 1, []config.Group{
f("2021-01-01T12:00:00.000Z", "2021-01-01T12:02:30.000Z", 1, time.Millisecond, []config.Group{
{Rules: []config.Rule{{Record: "foo", Expr: "sum(up)"}}},
}, &fakeReplayQuerier{
registry: map[string]map[string]struct{}{
@@ -87,7 +87,7 @@ func TestReplay(t *testing.T) {
})
// datapoints per step
f("2021-01-01T12:00:00.000Z", "2021-01-01T15:02:30.000Z", 60, []config.Group{
f("2021-01-01T12:00:00.000Z", "2021-01-01T15:02:30.000Z", 60, time.Millisecond, []config.Group{
{Interval: promutil.NewDuration(time.Minute), Rules: []config.Rule{{Record: "foo", Expr: "sum(up)"}}},
}, &fakeReplayQuerier{
registry: map[string]map[string]struct{}{
@@ -101,7 +101,7 @@ func TestReplay(t *testing.T) {
})
// multiple recording rules + multiple responses
f("2021-01-01T12:00:00.000Z", "2021-01-01T12:02:30.000Z", 1, []config.Group{
f("2021-01-01T12:00:00.000Z", "2021-01-01T12:02:30.000Z", 1, time.Millisecond, []config.Group{
{Rules: []config.Rule{{Record: "foo", Expr: "sum(up)"}}},
{Rules: []config.Rule{{Record: "bar", Expr: "max(up)"}}},
}, &fakeReplayQuerier{
@@ -120,7 +120,7 @@ func TestReplay(t *testing.T) {
})
// multiple alerting rules + multiple responses
f("2021-01-01T12:00:00.000Z", "2021-01-01T12:02:30.000Z", 1, []config.Group{
f("2021-01-01T12:00:00.000Z", "2021-01-01T12:02:30.000Z", 1, time.Millisecond, []config.Group{
{Rules: []config.Rule{{Alert: "foo", Expr: "sum(up) > 1"}}},
{Rules: []config.Rule{{Alert: "bar", Expr: "max(up) < 1"}}},
}, &fakeReplayQuerier{
@@ -137,4 +137,21 @@ func TestReplay(t *testing.T) {
},
},
})
// multiple alerting rules in one group+ multiple responses + concurrency
f("2021-01-01T12:00:00.000Z", "2021-01-01T12:02:30.000Z", 1, 0, []config.Group{
{Rules: []config.Rule{{Alert: "foo", Expr: "sum(up) > 1"}, {Alert: "bar", Expr: "max(up) < 1"}}, Concurrency: 2}}, &fakeReplayQuerier{
registry: map[string]map[string]struct{}{
"sum(up) > 1": {
"12:00:00+12:01:00": {},
"12:01:00+12:02:00": {},
"12:02:00+12:02:30": {},
},
"max(up) < 1": {
"12:00:00+12:01:00": {},
"12:01:00+12:02:00": {},
"12:02:00+12:02:30": {},
},
},
})
}

View File

@@ -445,11 +445,17 @@ func (g *Group) Start(ctx context.Context, nts func() []notifier.Notifier, rw re
g.infof("re-started")
case <-t.C:
missed := (time.Since(evalTS) / g.Interval) - 1
// calculate the real wall clock offset by stripping the monotonic clock first,
// then evalTS can be corrected when wall clock is adjusted.
// see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8790#issuecomment-2986541829
offset := time.Now().Round(0).Sub(evalTS.Round(0))
missed := (offset / g.Interval) - 1
if missed < 0 {
// missed can become < 0 due to irregular delays during evaluation
// which can result in time.Since(evalTS) < g.Interval
// which can result in time.Since(evalTS) < g.Interval;
// or the system wall clock was changed backward
missed = 0
evalTS = time.Now()
}
if missed > 0 {
g.metrics.iterationMissed.Inc()
@@ -514,36 +520,84 @@ func (g *Group) Replay(start, end time.Time, rw remotewrite.RWClient, maxDataPoi
iterations := int(end.Sub(start)/step) + 1
fmt.Printf("\nGroup %q"+
"\ninterval: \t%v"+
"\nrequests to make: \t%d"+
"\nconcurrency: \t %d"+
"\nrequests to make per rule: \t%d"+
"\nmax range per request: \t%v\n",
g.Name, g.Interval, iterations, step)
g.Name, g.Interval, g.Concurrency, iterations, step)
if g.Limit > 0 {
fmt.Printf("\nPlease note, `limit: %d` param has no effect during replay.\n",
fmt.Printf("\nWarning: `limit: %d` param has no effect during replay.\n",
g.Limit)
}
for _, rule := range g.Rules {
fmt.Printf("> Rule %q (ID: %d)\n", rule, rule.ID())
var bar *pb.ProgressBar
if !disableProgressBar {
bar = pb.StartNew(iterations)
}
ri.reset()
for ri.next() {
n, err := replayRule(rule, ri.s, ri.e, rw, replayRuleRetryAttempts)
if err != nil {
logger.Fatalf("rule %q: %s", rule, err)
concurrency := g.Concurrency
if g.Concurrency > 1 && replayDelay > 0 {
fmt.Printf("\nWarning: group concurrency %d will be ignored since `-replay.rulesDelay` is %.3f seconds."+
" Set -replay.rulesDelay=0 to enable concurrency for replay.\n", g.Concurrency, replayDelay.Seconds())
concurrency = 1
}
if concurrency == 1 {
for _, rule := range g.Rules {
var bar *pb.ProgressBar
if !disableProgressBar {
bar = pb.StartNew(iterations)
}
total += n
// pass ri as a copy, so it can be modified within the replayRuleRange
total += replayRuleRange(rule, ri, bar, rw, replayRuleRetryAttempts)
if bar != nil {
bar.Increment()
bar.Finish()
}
// sleep to let remote storage to flush data on-disk
// so chained rules could be calculated correctly
time.Sleep(replayDelay)
}
return total
}
sem := make(chan struct{}, g.Concurrency)
res := make(chan int, len(g.Rules)*iterations)
wg := sync.WaitGroup{}
var bar *pb.ProgressBar
if !disableProgressBar {
bar = pb.StartNew(iterations * len(g.Rules))
}
for _, r := range g.Rules {
sem <- struct{}{}
wg.Add(1)
go func(r Rule, ri rangeIterator) {
// pass ri as a copy, so it can be modified within the replayRuleRange
res <- replayRuleRange(r, ri, bar, rw, replayRuleRetryAttempts)
<-sem
wg.Done()
}(r, ri)
}
wg.Wait()
close(res)
close(sem)
if bar != nil {
bar.Finish()
}
total = 0
for n := range res {
total += n
}
return total
}
func replayRuleRange(r Rule, ri rangeIterator, bar *pb.ProgressBar, rw remotewrite.RWClient, replayRuleRetryAttempts int) int {
fmt.Printf("> Rule %q (ID: %d)\n", r, r.ID())
total := 0
for ri.next() {
n, err := replayRule(r, ri.s, ri.e, rw, replayRuleRetryAttempts)
if err != nil {
logger.Fatalf("rule %q: %s", r, err)
}
if bar != nil {
bar.Finish()
bar.Increment()
}
// sleep to let remote storage to flush data on-disk
// so chained rules could be calculated correctly
time.Sleep(replayDelay)
total += n
}
return total
}
@@ -570,11 +624,10 @@ type rangeIterator struct {
s, e time.Time
}
func (ri *rangeIterator) reset() {
ri.iter = 0
ri.s, ri.e = time.Time{}, time.Time{}
}
// next iterates with given step between start and end
// by modifying iter, s and e.
// Returns true until it reaches end.
// next modifies ri and isn't thread-safe.
func (ri *rangeIterator) next() bool {
ri.s = ri.start.Add(ri.step * time.Duration(ri.iter))
if !ri.end.After(ri.s) {

View File

@@ -742,7 +742,23 @@ Metric names are stripped from the resulting rollups. Add [keep_metric_names](#k
This function is supported by PromQL.
See also [irate](#irate) and [rollup_rate](#rollup_rate).
See also [irate](#irate), [rollup_rate](#rollup_rate) and [rate_prometheus](#rate_prometheus).
#### rate_prometheus
`rate_prometheus(series_selector[d])` {{% available_from "#" %}} is a [rollup function](#rollup-functions), which calculates the average per-second
increase rate over the given lookbehind window `d` per each time series returned from the given [series_selector](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#filtering).
The resulting calculation is equivalent to `increase_prometheus(series_selector[d]) / d`.
It doesn't take into account the last sample before the given lookbehind window `d` when calculating the result in the same way as Prometheus does.
See [this article](https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e) for details.
Metric names are stripped from the resulting rollups. Add [keep_metric_names](#keep_metric_names) modifier in order to keep metric names.
This function is usually applied to [counters](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#counter).
See also [increase_prometheus](#increase_prometheus) and [rate](#rate).
#### rate_over_sum

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -36,10 +36,10 @@
<meta property="og:title" content="UI for VictoriaMetrics">
<meta property="og:url" content="https://victoriametrics.com/">
<meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data">
<script type="module" crossorigin src="./assets/index-D-ssBbZq.js"></script>
<script type="module" crossorigin src="./assets/index-BiQY-19a.js"></script>
<link rel="modulepreload" crossorigin href="./assets/vendor-D8IJGiEn.js">
<link rel="stylesheet" crossorigin href="./assets/vendor-D1GxaB_c.css">
<link rel="stylesheet" crossorigin href="./assets/index-D5re9hC6.css">
<link rel="stylesheet" crossorigin href="./assets/index-ojCMu5lE.css">
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>

View File

@@ -63,6 +63,8 @@ var (
cacheSizeStorageTSID = flagutil.NewBytes("storage.cacheSizeStorageTSID", 0, "Overrides max size for storage/tsid cache. "+
"See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#cache-tuning")
cacheSizeStorageMetricName = flagutil.NewBytes("storage.cacheSizeStorageMetricName", 0, "Overrides max size for storage/metricName cache. "+
"See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#cache-tuning")
cacheSizeIndexDBIndexBlocks = flagutil.NewBytes("storage.cacheSizeIndexDBIndexBlocks", 0, "Overrides max size for indexdb/indexBlocks cache. "+
"See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#cache-tuning")
cacheSizeIndexDBDataBlocks = flagutil.NewBytes("storage.cacheSizeIndexDBDataBlocks", 0, "Overrides max size for indexdb/dataBlocks cache. "+
@@ -111,6 +113,7 @@ func Init(resetCacheIfNeeded func(mrs []storage.MetricRow)) {
storage.SetTSIDCacheSize(cacheSizeStorageTSID.IntN())
storage.SetTagFiltersCacheSize(cacheSizeIndexDBTagFilters.IntN())
storage.SetMetricNamesStatsCacheSize(cacheSizeMetricNamesStats.IntN())
storage.SetMetricNameCacheSize(cacheSizeStorageMetricName.IntN())
mergeset.SetIndexBlocksCacheSize(cacheSizeIndexDBIndexBlocks.IntN())
mergeset.SetDataBlocksCacheSize(cacheSizeIndexDBDataBlocks.IntN())
mergeset.SetDataBlocksSparseCacheSize(cacheSizeIndexDBDataBlocksSparse.IntN())

View File

@@ -10,6 +10,8 @@
"dependencies": {
"@types/lodash.debounce": "^4.0.9",
"@types/lodash.get": "^4.4.9",
"@types/lodash.orderBy": "^4.6.9",
"@types/lodash.throttle": "^4.1.9",
"@types/qs": "^6.9.18",
"@types/react": "^19.1.2",
"@types/react-input-mask": "^3.0.6",
@@ -18,6 +20,8 @@
"dayjs": "^1.11.13",
"lodash.debounce": "^4.0.8",
"lodash.get": "^4.4.2",
"lodash.orderBy": "^4.6.0",
"lodash.throttle": "^4.1.1",
"marked": "^15.0.8",
"marked-emoji": "^2.0.0",
"preact": "^10.26.5",
@@ -2191,6 +2195,24 @@
"@types/lodash": "*"
}
},
"node_modules/@types/lodash.orderBy": {
"version": "4.6.9",
"resolved": "https://registry.npmjs.org/@types/lodash.orderby/-/lodash.orderby-4.6.9.tgz",
"integrity": "sha512-T9o2wkIJOmxXwVTPTmwJ59W6eTi2FseiLR369fxszG649Po/xe9vqFNhf/MtnvT5jrbDiyWKxPFPZbpSVK0SVQ==",
"license": "MIT",
"dependencies": {
"@types/lodash": "*"
}
},
"node_modules/@types/lodash.throttle": {
"version": "4.1.9",
"resolved": "https://registry.npmjs.org/@types/lodash.throttle/-/lodash.throttle-4.1.9.tgz",
"integrity": "sha512-PCPVfpfueguWZQB7pJQK890F2scYKoDUL3iM522AptHWn7d5NQmeS/LTEHIcLr5PaTzl3dK2Z0xSUHHTHwaL5g==",
"license": "MIT",
"dependencies": {
"@types/lodash": "*"
}
},
"node_modules/@types/node": {
"version": "22.14.1",
"resolved": "https://registry.npmjs.org/@types/node/-/node-22.14.1.tgz",
@@ -5742,6 +5764,18 @@
"dev": true,
"license": "MIT"
},
"node_modules/lodash.orderBy": {
"version": "4.6.0",
"resolved": "https://registry.npmjs.org/lodash.orderby/-/lodash.orderby-4.6.0.tgz",
"integrity": "sha512-T0rZxKmghOOf5YPnn8EY5iLYeWCpZq8G41FfqoVHH5QDTAFaghJRmAdLiadEDq+ztgM2q5PjA+Z1fOwGrLgmtg==",
"license": "MIT"
},
"node_modules/lodash.throttle": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/lodash.throttle/-/lodash.throttle-4.1.1.tgz",
"integrity": "sha512-wIkUCfVKpVsWo3JSZlc+8MB5it+2AN5W8J7YVMST30UrvcQNZ1Okbj+rbVniijTWE6FGYy4XJq/rHkas8qJMLQ==",
"license": "MIT"
},
"node_modules/loose-envify": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/loose-envify/-/loose-envify-1.4.0.tgz",

View File

@@ -7,6 +7,8 @@
"dependencies": {
"@types/lodash.debounce": "^4.0.9",
"@types/lodash.get": "^4.4.9",
"@types/lodash.orderBy": "^4.6.9",
"@types/lodash.throttle": "^4.1.9",
"@types/qs": "^6.9.18",
"@types/react": "^19.1.2",
"@types/react-input-mask": "^3.0.6",
@@ -15,6 +17,8 @@
"dayjs": "^1.11.13",
"lodash.debounce": "^4.0.8",
"lodash.get": "^4.4.2",
"lodash.orderBy": "^4.6.0",
"lodash.throttle": "^4.1.1",
"marked": "^15.0.8",
"marked-emoji": "^2.0.0",
"preact": "^10.26.5",

View File

@@ -742,7 +742,23 @@ Metric names are stripped from the resulting rollups. Add [keep_metric_names](#k
This function is supported by PromQL.
See also [irate](#irate) and [rollup_rate](#rollup_rate).
See also [irate](#irate), [rollup_rate](#rollup_rate) and [rate_prometheus](#rate_prometheus).
#### rate_prometheus
`rate_prometheus(series_selector[d])` {{% available_from "#" %}} is a [rollup function](#rollup-functions), which calculates the average per-second
increase rate over the given lookbehind window `d` per each time series returned from the given [series_selector](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#filtering).
The resulting calculation is equivalent to `increase_prometheus(series_selector[d]) / d`.
It doesn't take into account the last sample before the given lookbehind window `d` when calculating the result in the same way as Prometheus does.
See [this article](https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e) for details.
Metric names are stripped from the resulting rollups. Add [keep_metric_names](#keep_metric_names) modifier in order to keep metric names.
This function is usually applied to [counters](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#counter).
See also [increase_prometheus](#increase_prometheus) and [rate](#rate).
#### rate_over_sum

View File

@@ -1,4 +1,4 @@
import React, { FC } from "preact/compat";
import { FC } from "preact/compat";
import classNames from "classnames";
import { MouseEvent as ReactMouseEvent, ReactNode } from "react";
import "./style.scss";
@@ -14,6 +14,7 @@ interface ButtonProps {
disabled?: boolean
children?: ReactNode
className?: string
"data-id"?: string
onClick?: (e: ReactMouseEvent<HTMLButtonElement>) => void
onMouseDown?: (e: ReactMouseEvent<HTMLButtonElement>) => void
}
@@ -31,6 +32,7 @@ const Button: FC<ButtonProps> = ({
disabled,
onClick,
onMouseDown,
"data-id": dataId
}) => {
const classesButton = classNames({
@@ -50,6 +52,7 @@ const Button: FC<ButtonProps> = ({
aria-label={ariaLabel}
onClick={onClick}
onMouseDown={onMouseDown}
data-id={dataId}
>
{startIcon}{children}{endIcon}
</button>

View File

@@ -656,3 +656,33 @@ export const ScrollToTopIcon = () => (
/>
</svg>
);
export const SortIcon = () => (
<svg
viewBox="0 0 24 24"
fill="currentColor"
>
<path d="M4 3 L4 15 L1.5 15 L5.5 21 L9.5 15 L7 15 L7 3 Z"/>
<path d="M13 21 L13 9 L10.5 9 L14.5 3 L18.5 9 L16 9 L16 21 Z"/>
</svg>
);
export const SortArrowDownIcon = () => (
<svg
viewBox="0 0 24 24"
fill="currentColor"
>
<path d="M10.5 3 L10.5 15 L8 15 L12 21 L16 15 L13.5 15 L13.5 3 Z"/>
</svg>
);
export const SortArrowUpIcon = () => (
<svg
viewBox="0 0 24 24"
fill="currentColor"
>
<path d="M10.5 21 L10.5 9 L8 9 L12 3 L16 9 L13.5 9 L13.5 21 Z"/>
</svg>
);

View File

@@ -18,8 +18,8 @@ const title = "Table settings";
interface TableSettingsProps {
columns: string[];
selectedColumns?: string[];
tableCompact: boolean;
toggleTableCompact: () => void;
tableCompact?: boolean;
toggleTableCompact?: () => void;
onChangeColumns: (arr: string[]) => void
}
@@ -195,18 +195,20 @@ const TableSettings: FC<TableSettingsProps> = ({
</div>
</div>
</div>
<div className="vm-table-settings-modal-section">
<div className="vm-table-settings-modal-section__title">
{toggleTableCompact && tableCompact !== undefined && (
<div className="vm-table-settings-modal-section">
<div className="vm-table-settings-modal-section__title">
Table view
</div>
<div className="vm-table-settings-modal-columns-list__item">
<Switch
label={"Compact view"}
value={tableCompact}
onChange={toggleTableCompact}
/>
</div>
</div>
<div className="vm-table-settings-modal-columns-list__item">
<Switch
label={"Compact view"}
value={tableCompact}
onChange={toggleTableCompact}
/>
</div>
</div>
)}
</Modal>)}
</div>
);

View File

@@ -5,7 +5,7 @@
&-header {
background-color: $color-background-block;
z-index: 1;
z-index: 3;
margin: -$padding-medium 0-$padding-medium 0;
position: sticky;
top: 0;

View File

@@ -1,21 +1,67 @@
import React, { FC } from "preact/compat";
import React, { FC, useMemo, useCallback, createPortal } from "preact/compat";
import DownloadLogsButton from "../../../DownloadLogsButton/DownloadLogsButton";
import { createPortal } from "preact/compat";
import JsonViewComponent from "../../../../../components/Views/JsonView/JsonView";
import { ViewProps } from "../../types";
import EmptyLogs from "../components/EmptyLogs/EmptyLogs";
import { useCallback } from "react";
import JsonViewSettings from "./JsonViewSettings/JsonViewSettings";
import { useSearchParams } from "react-router-dom";
import orderBy from "lodash.orderBy";
import "./style.scss";
import { Logs } from "../../../../../api/types";
import { SortDirection } from "./types";
const MemoizedJsonView = React.memo(JsonViewComponent);
const jsonQuerySortParam = "json_sort";
const fieldSortQueryParamName = "json_field_sort";
const JsonView: FC<ViewProps> = ({ data, settingsRef }) => {
const getLogs = useCallback(() => data, [data]);
const [searchParams] = useSearchParams();
const sortParam = searchParams.get(jsonQuerySortParam);
const fieldSortParam = searchParams.get(fieldSortQueryParamName) as SortDirection;
const [sortField, sortDirection] = useMemo(() => {
const [sortField, sortDirection] = sortParam?.split(":").map(decodeURIComponent) || [];
return [sortField, sortDirection as "asc" | "desc" | undefined];
}, [sortParam]);
const fields = useMemo(() => {
const keys = new Set(data.flatMap(Object.keys));
return Array.from(keys);
}, [data]);
const orderedFieldsData = useMemo(() => {
if (!fieldSortParam) return data;
const orderedFields = fields.toSorted((a, b) => fieldSortParam === "asc" ? a.localeCompare(b): b.localeCompare(a));
return data.map((item) => {
return orderedFields.reduce((acc, field) => {
if (item[field]) acc[field] = item[field];
return acc;
}, {} as Logs);
});
}, [fields, fieldSortParam, data]);
const sortedData = useMemo(() => {
if (!sortField || !sortDirection) return orderedFieldsData;
return orderBy(orderedFieldsData, [sortField], [sortDirection]);
}, [orderedFieldsData, sortField, sortDirection]);
const renderSettings = () => {
if (!settingsRef.current) return null;
return createPortal(
data.length > 0 && <DownloadLogsButton getLogs={getLogs} />,
data.length > 0 && (
<div className="vm-json-view__settings-container">
<DownloadLogsButton getLogs={getLogs} />
<JsonViewSettings
fields={fields}
sortQueryParamName={jsonQuerySortParam}
fieldSortQueryParamName={fieldSortQueryParamName}
/>
</div>
),
settingsRef.current
);
};
@@ -25,9 +71,11 @@ const JsonView: FC<ViewProps> = ({ data, settingsRef }) => {
return (
<>
{renderSettings()}
<MemoizedJsonView data={data} />
<MemoizedJsonView
data={sortedData}
/>
</>
);
};
export default JsonView;
export default JsonView;

View File

@@ -0,0 +1,185 @@
import { FC, useMemo, useRef } from "preact/compat";
import Button from "../../../../../../components/Main/Button/Button";
import { SettingsIcon, SortArrowDownIcon, SortArrowUpIcon, SortIcon } from "../../../../../../components/Main/Icons";
import Tooltip from "../../../../../../components/Main/Tooltip/Tooltip";
import Select from "../../../../../../components/Main/Select/Select";
import useBoolean from "../../../../../../hooks/useBoolean";
import { useState, useEffect, useCallback } from "react";
import Modal from "../../../../../../components/Main/Modal/Modal";
import { useSearchParams } from "react-router-dom";
import "./style.scss";
import { SortDirection } from "../types";
const title = "JSON settings";
const directionList = ["asc", "desc"];
interface JsonSettingsProps {
fields: string[];
sortQueryParamName: string;
fieldSortQueryParamName: string;
}
const JsonViewSettings: FC<JsonSettingsProps> = ({
fields,
sortQueryParamName,
fieldSortQueryParamName
}) => {
const [searchParams, setSearchParams] = useSearchParams();
const buttonRef = useRef<HTMLDivElement>(null);
const [fieldSortDirection, setFieldSortDirection] = useState<SortDirection>(null);
const {
value: openSettings,
toggle: toggleOpenSettings,
setFalse: handleClose,
} = useBoolean(false);
const [sortField, setSortField] = useState<string | null>(null);
const [sortDirection, setSortDirection] = useState<SortDirection>(null);
useEffect(() => {
const sortParam = searchParams.get(sortQueryParamName);
const isSortDirection = (value: string) : value is Exclude<SortDirection, null> => directionList.includes(value);
if (sortParam) {
const [field, direction] = sortParam.split(":").map(decodeURIComponent);
if (field && (isSortDirection(direction))) {
setSortField(field);
setSortDirection(direction);
}
}
const fieldSortParam = searchParams.get(fieldSortQueryParamName);
if (fieldSortParam === "asc" || fieldSortParam === "desc") {
setFieldSortDirection(fieldSortParam);
}
}, [searchParams, sortQueryParamName, fieldSortQueryParamName, setSortField, setSortDirection, setFieldSortDirection]);
const updateSortParams = useCallback((field: string | null, direction: SortDirection) => {
const updatedParams = new URLSearchParams(searchParams.toString());
if (!field || !direction) {
updatedParams.delete(sortQueryParamName);
} else {
updatedParams.set(sortQueryParamName, `${field}:${direction || ""}`);
}
setSearchParams(updatedParams);
}, [searchParams, sortQueryParamName]);
const handleSort = (field: string) => {
const newDirection: SortDirection = sortDirection || "asc";
setSortField(field);
setSortDirection(newDirection);
updateSortParams(field, newDirection);
};
const resetSort = () => {
setSortField(null);
setSortDirection(null);
updateSortParams(null, null);
};
const changeFieldSortDirection = useCallback(() => {
let newFieldSortDirection: SortDirection = null;
if (fieldSortDirection === null) {
newFieldSortDirection = "asc";
}else if (fieldSortDirection === "asc") {
newFieldSortDirection = "desc";
}
setFieldSortDirection(newFieldSortDirection);
const updatedParams = new URLSearchParams(searchParams.toString());
if (!newFieldSortDirection) {
updatedParams.delete(fieldSortQueryParamName);
} else {
updatedParams.set(fieldSortQueryParamName, encodeURIComponent(newFieldSortDirection));
}
setSearchParams(updatedParams);
},[fieldSortDirection, searchParams, fieldSortQueryParamName]);
const handleChangeSortDirection = (direction: string) => {
const field = sortField || fields[0];
setSortField(field);
setSortDirection(direction as SortDirection);
updateSortParams(field, direction as SortDirection);
};
const fieldSortMeta = useMemo(() => ({
default: {
title: "Set field sort order. Click to sort in ascending order",
icon: <SortIcon />
},
asc: {
title: "Fields sorted ascending. Click to sort in descending order",
icon: <SortArrowDownIcon />
},
desc: {
title: "Fields sorted descending. Click to reset sort",
icon: <SortArrowUpIcon />
},
}), []);
const fieldSortButton = useMemo(() => {
const { title, icon } = fieldSortMeta[fieldSortDirection ?? "default"];
return <Tooltip title={title}>
<Button
variant="text"
startIcon={icon}
onClick={changeFieldSortDirection}
ariaLabel={title}
/>
</Tooltip>;
}, [fieldSortDirection, toggleOpenSettings, changeFieldSortDirection, fieldSortMeta]);
return (
<div className="vm-json-settings">
{fieldSortButton}
<Tooltip title={title}>
<div ref={buttonRef}>
<Button
variant="text"
startIcon={<SettingsIcon/>}
onClick={toggleOpenSettings}
ariaLabel={title}
/>
</div>
</Tooltip>
{openSettings && (
<Modal
title={title}
className="vm-json-settings-modal"
onClose={handleClose}
>
<div className="vm-json-settings-modal-section">
<div className="vm-json-settings-modal-section__sort-settings-container">
<Select
value={sortField || ""}
onChange={handleSort}
list={fields}
label="Select field"
/>
<Select
value={sortDirection || ""}
onChange={handleChangeSortDirection}
list={directionList}
label="Sort direction"
/>
{(sortField || sortDirection) && (
<Button
variant="outlined"
color="error"
onClick={resetSort}
>
Reset sort
</Button>
)}
</div>
</div>
</Modal>)}
</div>
);
};
export default JsonViewSettings;

View File

@@ -0,0 +1,34 @@
@use "src/styles/variables" as *;
.vm-json-settings {
display: flex;
flex-direction: row;
&-modal {
.vm-modal-content-body {
min-width: clamp(300px, 600px, 90vw);
padding: 0;
}
&-section {
padding-block: $padding-global;
border-top: $border-divider;
&:first-child {
padding-top: 0;
border-top: none;
}
&__sort-settings-container {
display: grid;
padding: $padding-medium;
grid-template-columns: 1fr 1fr 80px;
gap: $padding-medium;
@media (max-width: 500px) {
grid-template-columns: 1fr;
}
}
}
}
}

View File

@@ -0,0 +1,9 @@
@use "src/styles/variables" as *;
.vm-json-view {
&__settings-container {
display: flex;
flex-direction: row;
align-items: center;
}
}

View File

@@ -0,0 +1 @@
export type SortDirection = "asc" | "desc" | null;

View File

@@ -7,7 +7,7 @@ import { useLiveTailingLogs } from "./useLiveTailingLogs";
import { LOGS_DISPLAY_FIELDS, LOGS_URL_PARAMS } from "../../../../../constants/logs";
import { useMemo } from "react";
import { useSearchParams } from "react-router-dom";
import throttle from "lodash/throttle";
import throttle from "lodash.throttle";
import GroupLogsItem from "../../../GroupLogs/GroupLogsItem";
import LiveTailingSettings from "./LiveTailingSettings";
import Alert from "../../../../../components/Main/Alert/Alert";

View File

@@ -3,7 +3,6 @@ import DownloadLogsButton from "../../../DownloadLogsButton/DownloadLogsButton";
import { createPortal } from "preact/compat";
import "./style.scss";
import { ViewProps } from "../../types";
import useBoolean from "../../../../../hooks/useBoolean";
import useStateSearchParams from "../../../../../hooks/useStateSearchParams";
import TableLogs from "../../TableLogs";
import SelectLimit from "../../../../../components/Main/Pagination/SelectLimit/SelectLimit";
@@ -18,7 +17,6 @@ const TableView: FC<ViewProps> = ({ data, settingsRef }) => {
const { setSearchParamsFromKeys } = useSearchParamsFromObject();
const [displayColumns, setDisplayColumns] = useState<string[]>([]);
const [rowsPerPage, setRowsPerPage] = useStateSearchParams(100, "rows_per_page");
const { value: tableCompact, toggle: toggleTableCompact } = useBoolean(false);
const columns = useMemo(() => {
const keys = new Set<string>();
@@ -52,8 +50,6 @@ const TableView: FC<ViewProps> = ({ data, settingsRef }) => {
columns={columns}
selectedColumns={displayColumns}
onChangeColumns={setDisplayColumns}
tableCompact={tableCompact}
toggleTableCompact={toggleTableCompact}
/>
</div>
</div>,
@@ -69,7 +65,7 @@ const TableView: FC<ViewProps> = ({ data, settingsRef }) => {
<MemoizedTableView
logs={data}
displayColumns={displayColumns}
tableCompact={tableCompact}
tableCompact={false}
columns={columns}
rowsPerPage={Number(rowsPerPage)}
/>

View File

@@ -18,22 +18,22 @@ import (
pb "github.com/VictoriaMetrics/VictoriaMetrics/lib/prompbmarshal"
)
// APIQuerier contains methods available to Prometheus-like HTTP API for Querying
type APIQuerier interface {
APIV1Export(t *testing.T, query string, opts QueryOpts) *APIV1QueryResponse
APIV1Query(t *testing.T, query string, opts QueryOpts) *APIV1QueryResponse
APIV1QueryRange(t *testing.T, query string, opts QueryOpts) *APIV1QueryResponse
APIV1Series(t *testing.T, matchQuery string, opts QueryOpts) *APIV1SeriesResponse
APIV1ExportNative(t *testing.T, query string, opts QueryOpts) []byte
// PrometheusQuerier contains methods available to Prometheus-like HTTP API for Querying
type PrometheusQuerier interface {
PrometheusAPIV1Export(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse
PrometheusAPIV1Query(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse
PrometheusAPIV1QueryRange(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse
PrometheusAPIV1Series(t *testing.T, matchQuery string, opts QueryOpts) *PrometheusAPIV1SeriesResponse
PrometheusAPIV1ExportNative(t *testing.T, query string, opts QueryOpts) []byte
}
// APIWriter contains methods for writing new data
type APIWriter interface {
// Prometheus-like APIs
APIV1Write(t *testing.T, records []pb.TimeSeries, opts QueryOpts)
APIV1ImportPrometheus(t *testing.T, records []string, opts QueryOpts)
APIV1ImportCSV(t *testing.T, records []string, opts QueryOpts)
APIV1ImportNative(t *testing.T, data []byte, opts QueryOpts)
// Writer contains methods for writing new data
type Writer interface {
// Prometheus APIs
PrometheusAPIV1Write(t *testing.T, records []pb.TimeSeries, opts QueryOpts)
PrometheusAPIV1ImportPrometheus(t *testing.T, records []string, opts QueryOpts)
PrometheusAPIV1ImportCSV(t *testing.T, records []string, opts QueryOpts)
PrometheusAPIV1ImportNative(t *testing.T, data []byte, opts QueryOpts)
// Graphit APIs
GraphiteWrite(t *testing.T, records []string, opts QueryOpts)
@@ -54,11 +54,11 @@ type StorageMerger interface {
ForceMerge(t *testing.T)
}
// WriteQuerier encompasses the methods for writing, flushing and
// PrometheusWriteQuerier encompasses the methods for writing, flushing and
// querying the data.
type WriteQuerier interface {
APIWriter
APIQuerier
type PrometheusWriteQuerier interface {
Writer
PrometheusQuerier
StorageFlusher
StorageMerger
}
@@ -138,9 +138,9 @@ func (qos *QueryOptsLogs) asURLValues() url.Values {
return uv
}
// APIV1QueryResponse is an inmemory representation of the
// PrometheusAPIV1QueryResponse is an inmemory representation of the
// /prometheus/api/v1/query or /prometheus/api/v1/query_range response.
type APIV1QueryResponse struct {
type PrometheusAPIV1QueryResponse struct {
Status string
Data *QueryData
ErrorType string
@@ -148,12 +148,12 @@ type APIV1QueryResponse struct {
IsPartial bool
}
// NewAPIV1QueryResponse is a test helper function that creates a new
// instance of APIV1QueryResponse by unmarshalling a json string.
func NewAPIV1QueryResponse(t *testing.T, s string) *APIV1QueryResponse {
// NewPrometheusAPIV1QueryResponse is a test helper function that creates a new
// instance of PrometheusAPIV1QueryResponse by unmarshalling a json string.
func NewPrometheusAPIV1QueryResponse(t *testing.T, s string) *PrometheusAPIV1QueryResponse {
t.Helper()
res := &APIV1QueryResponse{}
res := &PrometheusAPIV1QueryResponse{}
if err := json.Unmarshal([]byte(s), res); err != nil {
t.Fatalf("could not unmarshal query response data=\n%s\n: %v", string(s), err)
}
@@ -161,7 +161,7 @@ func NewAPIV1QueryResponse(t *testing.T, s string) *APIV1QueryResponse {
}
// Sort performs data.Result sort by metric labels
func (pqr *APIV1QueryResponse) Sort() {
func (pqr *PrometheusAPIV1QueryResponse) Sort() {
if pqr.Data == nil {
return
}
@@ -257,9 +257,9 @@ func (s *Sample) UnmarshalJSON(b []byte) error {
return nil
}
// APIV1SeriesResponse is an inmemory representation of the
// PrometheusAPIV1SeriesResponse is an inmemory representation of the
// /prometheus/api/v1/series response.
type APIV1SeriesResponse struct {
type PrometheusAPIV1SeriesResponse struct {
Status string
IsPartial bool
Data []map[string]string
@@ -268,12 +268,12 @@ type APIV1SeriesResponse struct {
Error string
}
// NewAPIV1SeriesResponse is a test helper function that creates a new
// instance of APIV1SeriesResponse by unmarshalling a json string.
func NewAPIV1SeriesResponse(t *testing.T, s string) *APIV1SeriesResponse {
// NewPrometheusAPIV1SeriesResponse is a test helper function that creates a new
// instance of PrometheusAPIV1SeriesResponse by unmarshalling a json string.
func NewPrometheusAPIV1SeriesResponse(t *testing.T, s string) *PrometheusAPIV1SeriesResponse {
t.Helper()
res := &APIV1SeriesResponse{}
res := &PrometheusAPIV1SeriesResponse{}
if err := json.Unmarshal([]byte(s), res); err != nil {
t.Fatalf("could not unmarshal series response data:\n%s\n err: %v", string(s), err)
}
@@ -281,7 +281,7 @@ func NewAPIV1SeriesResponse(t *testing.T, s string) *APIV1SeriesResponse {
}
// Sort sorts the response data.
func (r *APIV1SeriesResponse) Sort() *APIV1SeriesResponse {
func (r *PrometheusAPIV1SeriesResponse) Sort() *PrometheusAPIV1SeriesResponse {
str := func(m map[string]string) string {
s := []string{}
for k, v := range m {

View File

@@ -146,7 +146,7 @@ func (tc *TestCase) MustStartVmagent(instance string, flags []string, promScrape
// Vmcluster represents a typical cluster setup: several vmstorage replicas, one
// vminsert, and one vmselect.
//
// Both Vmsingle and Vmcluster implement the WriteQuerier used in
// Both Vmsingle and Vmcluster implement the PrometheusWriteQuerier used in
// business logic tests to abstract out the infrasture.
//
// This type is not suitable for infrastructure tests where custom cluster
@@ -309,8 +309,8 @@ func (tc *TestCase) StopApp(instance string) {
}
}
// StopWriteQuerier stop all apps that are a part of the pwq.
func (tc *TestCase) StopWriteQuerier(pwq WriteQuerier) {
// StopPrometheusWriteQuerier stop all apps that are a part of the pwq.
func (tc *TestCase) StopPrometheusWriteQuerier(pwq PrometheusWriteQuerier) {
tc.t.Helper()
switch t := pwq.(type) {
case *Vmsingle:

View File

@@ -11,10 +11,10 @@ import (
)
type testBackupRestoreOpts struct {
startSUT func() at.WriteQuerier
startSUT func() at.PrometheusWriteQuerier
stopSUT func()
storageDataPaths []string
snapshotCreateURLs func(at.WriteQuerier) []string
snapshotCreateURLs func(at.PrometheusWriteQuerier) []string
}
func TestSingleBackupRestore(t *testing.T) {
@@ -24,7 +24,7 @@ func TestSingleBackupRestore(t *testing.T) {
storageDataPath := filepath.Join(tc.Dir(), "vmsingle")
opts := testBackupRestoreOpts{
startSUT: func() at.WriteQuerier {
startSUT: func() at.PrometheusWriteQuerier {
return tc.MustStartVmsingle("vmsingle", []string{
"-storageDataPath=" + storageDataPath,
"-retentionPeriod=100y",
@@ -37,7 +37,7 @@ func TestSingleBackupRestore(t *testing.T) {
storageDataPaths: []string{
storageDataPath,
},
snapshotCreateURLs: func(sut at.WriteQuerier) []string {
snapshotCreateURLs: func(sut at.PrometheusWriteQuerier) []string {
return []string{
sut.(*at.Vmsingle).SnapshotCreateURL(),
}
@@ -55,7 +55,7 @@ func TestClusterBackupRestore(t *testing.T) {
storage2DataPath := filepath.Join(tc.Dir(), "vmstorage2")
opts := testBackupRestoreOpts{
startSUT: func() at.WriteQuerier {
startSUT: func() at.PrometheusWriteQuerier {
return tc.MustStartCluster(&at.ClusterOptions{
Vmstorage1Instance: "vmstorage1",
Vmstorage1Flags: []string{
@@ -85,7 +85,7 @@ func TestClusterBackupRestore(t *testing.T) {
storage1DataPath,
storage2DataPath,
},
snapshotCreateURLs: func(sut at.WriteQuerier) []string {
snapshotCreateURLs: func(sut at.PrometheusWriteQuerier) []string {
c := sut.(*at.Vmcluster)
return []string{
c.Vmstorages[0].SnapshotCreateURL(),
@@ -127,18 +127,18 @@ func testBackupRestore(tc *at.TestCase, opts testBackupRestoreOpts) {
// assertSeries retrieves set of all metric names from the storage and
// compares it with the expected set.
assertSeries := func(app at.APIQuerier, query string, start, end int64, want []map[string]string) {
assertSeries := func(app at.PrometheusQuerier, query string, start, end int64, want []map[string]string) {
t.Helper()
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/series response",
Got: func() any {
return app.APIV1Series(t, query, at.QueryOpts{
return app.PrometheusAPIV1Series(t, query, at.QueryOpts{
Start: fmt.Sprintf("%d", start),
End: fmt.Sprintf("%d", end),
}).Sort()
},
Want: &at.APIV1SeriesResponse{
Want: &at.PrometheusAPIV1SeriesResponse{
Status: "success",
Data: want,
},
@@ -148,18 +148,18 @@ func testBackupRestore(tc *at.TestCase, opts testBackupRestoreOpts) {
// assertSeries retrieves all data from the storage and compares it with the
// expected result.
assertQueryResults := func(app at.APIQuerier, query string, start, end int64, want []*at.QueryResult) {
assertQueryResults := func(app at.PrometheusQuerier, query string, start, end int64, want []*at.QueryResult) {
t.Helper()
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query_range response",
Got: func() any {
return app.APIV1QueryRange(t, query, at.QueryOpts{
return app.PrometheusAPIV1QueryRange(t, query, at.QueryOpts{
Start: fmt.Sprintf("%d", start),
End: fmt.Sprintf("%d", end),
Step: "60s",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -171,7 +171,7 @@ func testBackupRestore(tc *at.TestCase, opts testBackupRestoreOpts) {
})
}
createBackup := func(sut at.WriteQuerier, name string) {
createBackup := func(sut at.PrometheusWriteQuerier, name string) {
for i, storageDataPath := range opts.storageDataPaths {
replica := fmt.Sprintf("replica-%d", i)
instance := fmt.Sprintf("vmbackup-%s-%s", name, replica)
@@ -216,13 +216,13 @@ func testBackupRestore(tc *at.TestCase, opts testBackupRestoreOpts) {
sut := opts.startSUT()
sut.APIV1ImportPrometheus(t, batch1Data, at.QueryOpts{})
sut.PrometheusAPIV1ImportPrometheus(t, batch1Data, at.QueryOpts{})
sut.ForceFlush(t)
assertSeries(sut, `{__name__=~"batch1.*"}`, start, end, wantBatch1Series)
assertQueryResults(sut, `{__name__=~"batch1.*"}`, start, end, wantBatch1QueryResults)
createBackup(sut, "batch1")
sut.APIV1ImportPrometheus(t, batch2Data, at.QueryOpts{})
sut.PrometheusAPIV1ImportPrometheus(t, batch2Data, at.QueryOpts{})
sut.ForceFlush(t)
assertSeries(sut, `{__name__=~"batch(1|2).*"}`, start, end, wantBatch12Series)
assertQueryResults(sut, `{__name__=~"batch(1|2).*"}`, start, end, wantBatch12QueryResults)

View File

@@ -5,13 +5,12 @@ import (
"testing"
"time"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/VictoriaMetrics/VictoriaMetrics/apptest"
at "github.com/VictoriaMetrics/VictoriaMetrics/apptest"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/decimal"
pb "github.com/VictoriaMetrics/VictoriaMetrics/lib/prompbmarshal"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
)
func TestSingleDeduplication_dedulicationIsOff(t *testing.T) {
@@ -81,7 +80,7 @@ func TestClusterDeduplication_deduplicationIsOn(t *testing.T) {
}
// See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#deduplication
func testDeduplication(tc *at.TestCase, sut at.WriteQuerier, deduplicationIsOn bool) {
func testDeduplication(tc *at.TestCase, sut at.PrometheusWriteQuerier, deduplicationIsOn bool) {
t := tc.T()
firstDayOfThisMonth := func() time.Time {
@@ -135,11 +134,11 @@ func testDeduplication(tc *at.TestCase, sut at.WriteQuerier, deduplicationIsOn b
},
}
sut.APIV1Write(t, data, apptest.QueryOpts{})
sut.PrometheusAPIV1Write(t, data, apptest.QueryOpts{})
sut.ForceFlush(t)
sut.ForceMerge(t)
wantDuplicates := &at.APIV1QueryResponse{
wantDuplicates := &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -167,7 +166,7 @@ func testDeduplication(tc *at.TestCase, sut at.WriteQuerier, deduplicationIsOn b
},
},
}
wantDeduped := &at.APIV1QueryResponse{
wantDeduped := &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -208,7 +207,7 @@ func testDeduplication(tc *at.TestCase, sut at.WriteQuerier, deduplicationIsOn b
tc.Assert(&at.AssertOptions{
Msg: "unexpected response",
Got: func() any {
got := sut.APIV1Export(t, `{__name__=~"metric.*"}`, apptest.QueryOpts{
got := sut.PrometheusAPIV1Export(t, `{__name__=~"metric.*"}`, apptest.QueryOpts{
ReduceMemUsage: "1",
Start: fmt.Sprintf("%d", start.UnixMilli()),
End: fmt.Sprintf("%d", end.UnixMilli()),

View File

@@ -33,9 +33,9 @@ func TestClusterExportImportNative(t *testing.T) {
// testExportImportNative test export and import in VictoriaMetrics native format.
// see: https://docs.victoriametrics.com/#how-to-import-data-in-native-format
func testExportImportNative(t *testing.T, sut at.WriteQuerier) {
func testExportImportNative(t *testing.T, sut at.PrometheusWriteQuerier) {
// create test data
sut.APIV1ImportPrometheus(t, []string{
sut.PrometheusAPIV1ImportPrometheus(t, []string{
`native_export_import 10 1707123456700`, // 2024-02-05T08:57:36.700Z
}, at.QueryOpts{
ExtraLabels: []string{"el1=elv1", "el2=elv2"},
@@ -43,27 +43,27 @@ func testExportImportNative(t *testing.T, sut at.WriteQuerier) {
sut.ForceFlush(t)
// export test data via native export API
exportResult := sut.APIV1ExportNative(t, "native_export_import", at.QueryOpts{
exportResult := sut.PrometheusAPIV1ExportNative(t, "native_export_import", at.QueryOpts{
Start: "2024-02-05T08:50:00.700Z",
End: "2024-02-05T09:00:00.700Z",
})
// re-import test data via native import API
sut.APIV1ImportNative(t, exportResult, at.QueryOpts{})
sut.PrometheusAPIV1ImportNative(t, exportResult, at.QueryOpts{})
sut.ForceFlush(t)
// check query result
got := sut.APIV1QueryRange(t, "native_export_import", at.QueryOpts{
got := sut.PrometheusAPIV1QueryRange(t, "native_export_import", at.QueryOpts{
Start: "2024-02-05T08:57:36.700Z",
End: "2024-02-05T08:57:36.700Z",
Step: "60s",
})
cmpOptions := []cmp.Option{
cmpopts.IgnoreFields(at.APIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.IgnoreFields(at.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.EquateNaNs(),
}
want := at.NewAPIV1QueryResponse(t, `{"data": {"result": [{"metric": {"__name__": "native_export_import", "el1": "elv1", "el2":"elv2"}, "values": []}]}}`)
want := at.NewPrometheusAPIV1QueryResponse(t, `{"data": {"result": [{"metric": {"__name__": "native_export_import", "el1": "elv1", "el2":"elv2"}, "values": []}]}}`)
want.Data.Result[0].Samples = []*at.Sample{
at.NewSample(t, "2024-02-05T08:57:36.700Z", 10),
}

View File

@@ -22,7 +22,7 @@ func TestSingleIngestionProtocols(t *testing.T) {
wantMetrics []map[string]string
wantSamples []*at.Sample
}
f := func(sut at.APIQuerier, opts *opts) {
f := func(sut at.PrometheusQuerier, opts *opts) {
t.Helper()
wantResult := []*at.QueryResult{}
for idx, wm := range opts.wantMetrics {
@@ -35,16 +35,16 @@ func TestSingleIngestionProtocols(t *testing.T) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /export query response",
Got: func() any {
got := sut.APIV1Export(t, opts.query, at.QueryOpts{
got := sut.PrometheusAPIV1Export(t, opts.query, at.QueryOpts{
Start: "2024-02-05T08:50:00.700Z",
End: "2024-02-05T09:00:00.700Z",
})
got.Sort()
return got
},
Want: &at.APIV1QueryResponse{Data: &at.QueryData{Result: wantResult}},
Want: &at.PrometheusAPIV1QueryResponse{Data: &at.QueryData{Result: wantResult}},
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(at.APIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.IgnoreFields(at.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
},
})
}
@@ -109,7 +109,7 @@ func TestSingleIngestionProtocols(t *testing.T) {
})
// CSV import
sut.APIV1ImportCSV(t, []string{
sut.PrometheusAPIV1ImportCSV(t, []string{
`GOOG,1.23,4.56,NYSE,1707123457`,
`MSFT,23,56,NASDAQ,1707123457`,
}, at.QueryOpts{
@@ -158,7 +158,7 @@ func TestSingleIngestionProtocols(t *testing.T) {
})
// prometheus text exposition format
sut.APIV1ImportPrometheus(t, []string{
sut.PrometheusAPIV1ImportPrometheus(t, []string{
`importprometheus_series 10 1707123456700`, // 2024-02-05T08:57:36.700Z
`importprometheus_series2{label="foo",label1="value1"} 20 1707123456800`, // 2024-02-05T08:57:36.800Z
}, at.QueryOpts{
@@ -227,7 +227,7 @@ func TestSingleIngestionProtocols(t *testing.T) {
},
},
}
sut.APIV1Write(t, pbData, at.QueryOpts{})
sut.PrometheusAPIV1Write(t, pbData, at.QueryOpts{})
sut.ForceFlush(t)
f(sut, &opts{
query: `{__name__=~"prometheusrw.+"}`,
@@ -282,22 +282,22 @@ func TestClusterIngestionProtocols(t *testing.T) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /export query response",
Got: func() any {
got := vmselect.APIV1Export(t, opts.query, at.QueryOpts{
got := vmselect.PrometheusAPIV1Export(t, opts.query, at.QueryOpts{
Start: "2024-02-05T08:50:00.700Z",
End: "2024-02-05T09:00:00.700Z",
})
got.Sort()
return got
},
Want: &at.APIV1QueryResponse{Data: &at.QueryData{Result: wantResult}},
Want: &at.PrometheusAPIV1QueryResponse{Data: &at.QueryData{Result: wantResult}},
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(at.APIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.IgnoreFields(at.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
},
})
}
// prometheus text exposition format
vminsert.APIV1ImportPrometheus(t, []string{
vminsert.PrometheusAPIV1ImportPrometheus(t, []string{
`importprometheus_series 10 1707123456700`, // 2024-02-05T08:57:36.700Z
`importprometheus_series2{label="foo",label1="value1"} 20 1707123456800`, // 2024-02-05T08:57:36.800Z
}, at.QueryOpts{
@@ -358,7 +358,7 @@ func TestClusterIngestionProtocols(t *testing.T) {
})
// CSV import
vminsert.APIV1ImportCSV(t, []string{
vminsert.PrometheusAPIV1ImportCSV(t, []string{
`GOOG,1.23,4.56,NYSE,1707123457`, // 2024-02-05T08:57:37.000Z
`MSFT,23,56,NASDAQ,1707123457`, // 2024-02-05T08:57:37.000Z
}, at.QueryOpts{
@@ -474,7 +474,7 @@ func TestClusterIngestionProtocols(t *testing.T) {
},
},
}
vminsert.APIV1Write(t, pbData, at.QueryOpts{})
vminsert.PrometheusAPIV1Write(t, pbData, at.QueryOpts{})
vmstorage.ForceFlush(t)
f(&opts{
query: `{__name__=~"prometheusrw.+"}`,

View File

@@ -55,10 +55,10 @@ func TestClusterKeyConceptsQueryData(t *testing.T) {
}
// testKeyConceptsQueryData verifies cases from https://docs.victoriametrics.com/victoriametrics/keyconcepts/#query-data
func testKeyConceptsQueryData(t *testing.T, sut at.WriteQuerier) {
func testKeyConceptsQueryData(t *testing.T, sut at.PrometheusWriteQuerier) {
// Insert example data from documentation.
sut.APIV1ImportPrometheus(t, docData, at.QueryOpts{})
sut.PrometheusAPIV1ImportPrometheus(t, docData, at.QueryOpts{})
sut.ForceFlush(t)
testInstantQuery(t, sut)
@@ -69,14 +69,14 @@ func testKeyConceptsQueryData(t *testing.T, sut at.WriteQuerier) {
// testInstantQuery verifies the statements made in the `Instant query` section
// of the VictoriaMetrics documentation. See:
// https://docs.victoriametrics.com/victoriametrics/keyconcepts/#instant-query
func testInstantQuery(t *testing.T, q at.APIQuerier) {
func testInstantQuery(t *testing.T, q at.PrometheusQuerier) {
// Get the value of the foo_bar time series at 2022-05-10T08:03:00Z with the
// step of 5m and timeout 5s. There is no sample at exactly this timestamp.
// Therefore, VictoriaMetrics will search for the nearest sample within the
// [time-5m..time] interval.
got := q.APIV1Query(t, "foo_bar", at.QueryOpts{Time: "2022-05-10T08:03:00.000Z", Step: "5m"})
want := at.NewAPIV1QueryResponse(t, `{"data":{"result":[{"metric":{"__name__":"foo_bar"},"value":[1652169780,"3"]}]}}`)
opt := cmpopts.IgnoreFields(at.APIV1QueryResponse{}, "Status", "Data.ResultType")
got := q.PrometheusAPIV1Query(t, "foo_bar", at.QueryOpts{Time: "2022-05-10T08:03:00.000Z", Step: "5m"})
want := at.NewPrometheusAPIV1QueryResponse(t, `{"data":{"result":[{"metric":{"__name__":"foo_bar"},"value":[1652169780,"3"]}]}}`)
opt := cmpopts.IgnoreFields(at.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType")
if diff := cmp.Diff(want, got, opt); diff != "" {
t.Errorf("unexpected response (-want, +got):\n%s", diff)
}
@@ -86,7 +86,7 @@ func testInstantQuery(t *testing.T, q at.APIQuerier) {
// Therefore, VictoriaMetrics will search for the nearest sample within the
// [time-1m..time] interval. Since the nearest sample is 2m away and the
// step is 1m, then the VictoriaMetrics must return empty response.
got = q.APIV1Query(t, "foo_bar", at.QueryOpts{Time: "2022-05-10T08:18:00.000Z", Step: "1m"})
got = q.PrometheusAPIV1Query(t, "foo_bar", at.QueryOpts{Time: "2022-05-10T08:18:00.000Z", Step: "1m"})
if len(got.Data.Result) > 0 {
t.Errorf("unexpected response: got non-empty result, want empty result:\n%v", got)
}
@@ -95,14 +95,14 @@ func testInstantQuery(t *testing.T, q at.APIQuerier) {
// testRangeQuery verifies the statements made in the `Range query` section of
// the VictoriaMetrics documentation. See:
// https://docs.victoriametrics.com/victoriametrics/keyconcepts/#range-query
func testRangeQuery(t *testing.T, q at.APIQuerier) {
func testRangeQuery(t *testing.T, q at.PrometheusQuerier) {
f := func(start, end, step string, wantSamples []*at.Sample) {
t.Helper()
got := q.APIV1QueryRange(t, "foo_bar", at.QueryOpts{Start: start, End: end, Step: step})
want := at.NewAPIV1QueryResponse(t, `{"data": {"result": [{"metric": {"__name__": "foo_bar"}, "values": []}]}}`)
got := q.PrometheusAPIV1QueryRange(t, "foo_bar", at.QueryOpts{Start: start, End: end, Step: step})
want := at.NewPrometheusAPIV1QueryResponse(t, `{"data": {"result": [{"metric": {"__name__": "foo_bar"}, "values": []}]}}`)
want.Data.Result[0].Samples = wantSamples
opt := cmpopts.IgnoreFields(at.APIV1QueryResponse{}, "Status", "Data.ResultType")
opt := cmpopts.IgnoreFields(at.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType")
if diff := cmp.Diff(want, got, opt); diff != "" {
t.Errorf("unexpected response (-want, +got):\n%s", diff)
}
@@ -168,11 +168,11 @@ func testRangeQuery(t *testing.T, q at.APIQuerier) {
// will not produce ephemeral points.
//
// See: https://docs.victoriametrics.com/victoriametrics/keyconcepts/#range-query
func testRangeQueryIsEquivalentToManyInstantQueries(t *testing.T, q at.APIQuerier) {
func testRangeQueryIsEquivalentToManyInstantQueries(t *testing.T, q at.PrometheusQuerier) {
f := func(timestamp string, want *at.Sample) {
t.Helper()
gotInstant := q.APIV1Query(t, "foo_bar", at.QueryOpts{Time: timestamp, Step: "1m"})
gotInstant := q.PrometheusAPIV1Query(t, "foo_bar", at.QueryOpts{Time: timestamp, Step: "1m"})
if want == nil {
if got, want := len(gotInstant.Data.Result), 0; got != want {
t.Errorf("unexpected instant result size: got %d, want %d", got, want)
@@ -185,7 +185,7 @@ func testRangeQueryIsEquivalentToManyInstantQueries(t *testing.T, q at.APIQuerie
}
}
rangeRes := q.APIV1QueryRange(t, "foo_bar", at.QueryOpts{
rangeRes := q.PrometheusAPIV1QueryRange(t, "foo_bar", at.QueryOpts{
Start: "2022-05-10T07:59:00.000Z",
End: "2022-05-10T08:17:00.000Z",
Step: "1m",
@@ -231,7 +231,7 @@ func TestClusterMillisecondPrecisionInInstantQueries(t *testing.T) {
testMillisecondPrecisionInInstantQueries(tc, sut)
}
func testMillisecondPrecisionInInstantQueries(tc *at.TestCase, sut at.WriteQuerier) {
func testMillisecondPrecisionInInstantQueries(tc *at.TestCase, sut at.PrometheusWriteQuerier) {
t := tc.T()
type opts struct {
@@ -242,7 +242,7 @@ func testMillisecondPrecisionInInstantQueries(tc *at.TestCase, sut at.WriteQueri
wantSample *at.Sample
wantSamples []*at.Sample
}
f := func(sut at.APIQuerier, opts *opts) {
f := func(sut at.PrometheusQuerier, opts *opts) {
t.Helper()
wantResult := []*at.QueryResult{}
if opts.wantMetric != nil && (opts.wantSample != nil || len(opts.wantSamples) > 0) {
@@ -255,19 +255,19 @@ func testMillisecondPrecisionInInstantQueries(tc *at.TestCase, sut at.WriteQueri
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query response",
Got: func() any {
return sut.APIV1Query(t, opts.query, at.QueryOpts{
return sut.PrometheusAPIV1Query(t, opts.query, at.QueryOpts{
Time: opts.qtime,
Step: opts.step,
})
},
Want: &at.APIV1QueryResponse{Data: &at.QueryData{Result: wantResult}},
Want: &at.PrometheusAPIV1QueryResponse{Data: &at.QueryData{Result: wantResult}},
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(at.APIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.IgnoreFields(at.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
},
})
}
sut.APIV1ImportPrometheus(t, []string{
sut.PrometheusAPIV1ImportPrometheus(t, []string{
`series1{label="foo"} 10 1707123456700`, // 2024-02-05T08:57:36.700Z
`series1{label="foo"} 20 1707123456800`, // 2024-02-05T08:57:36.800Z
}, at.QueryOpts{})
@@ -326,7 +326,7 @@ func testMillisecondPrecisionInInstantQueries(tc *at.TestCase, sut at.WriteQueri
// Insert samples with different dates. The difference in ms between the two
// timestamps is 4236579304.
sut.APIV1ImportPrometheus(t, []string{
sut.PrometheusAPIV1ImportPrometheus(t, []string{
`series2{label="foo"} 10 1638564958042`, // 2021-12-03T20:55:58.042Z
`series2{label="foo"} 20 1642801537346`, // 2022-01-21T21:45:37.346Z
}, at.QueryOpts{})

View File

@@ -16,7 +16,7 @@ func TestSingleMaxIngestionRateIncrementsMetric(t *testing.T) {
tc := apptest.NewTestCase(t)
defer tc.Stop()
sut := tc.MustStartVmsingle("vmsingle", []string{"-maxIngestionRate=1"})
sut.APIV1ImportPrometheus(t, testData, apptest.QueryOpts{})
sut.PrometheusAPIV1ImportPrometheus(t, testData, apptest.QueryOpts{})
if got := sut.GetMetric(t, "vm_max_ingestion_rate_limit_reached_total"); got <= 0 {
t.Fatalf("Unexpected vm_max_ingestion_rate_limit_reached_total: got %f, want >0", got)
}
@@ -26,7 +26,7 @@ func TestSingleMaxIngestionRateDoesNotIncrementMetric(t *testing.T) {
tc := apptest.NewTestCase(t)
defer tc.Stop()
sut := tc.MustStartVmsingle("vmsingle", []string{"-maxIngestionRate=15"})
sut.APIV1ImportPrometheus(t, testData, apptest.QueryOpts{})
sut.PrometheusAPIV1ImportPrometheus(t, testData, apptest.QueryOpts{})
if got, want := sut.GetMetric(t, "vm_max_ingestion_rate_limit_reached_total"), 0.0; got != want {
t.Fatalf("Unexpected vm_max_ingestion_rate_limit_reached_total: got %f, want >0", got)
}

View File

@@ -37,7 +37,7 @@ func TestSingleMetricNamesStats(t *testing.T) {
}
tsdbMetricNameEntryCmpOpts := cmpopts.IgnoreFields(apptest.TSDBStatusResponseMetricNameEntry{}, "LastRequestTimestamp")
sut.APIV1ImportPrometheus(t, dataSet, at.QueryOpts{})
sut.PrometheusAPIV1ImportPrometheus(t, dataSet, at.QueryOpts{})
sut.ForceFlush(t)
// verify ingest request correctly registered
@@ -55,7 +55,7 @@ func TestSingleMetricNamesStats(t *testing.T) {
}
// verify query request correctly registered
sut.APIV1Query(t, `{__name__!=""}`, at.QueryOpts{Time: ingestDateTime})
sut.PrometheusAPIV1Query(t, `{__name__!=""}`, at.QueryOpts{Time: ingestDateTime})
expected = apptest.MetricNamesStatsResponse{
Records: []at.MetricNamesStatsRecord{
{MetricName: largeMetricName, QueryRequestsCount: 1},
@@ -97,7 +97,7 @@ func TestSingleMetricNamesStats(t *testing.T) {
}
// perform query request for single metric and check counter increase
sut.APIV1Query(t, `metric_name_2`, at.QueryOpts{Time: ingestDateTime})
sut.PrometheusAPIV1Query(t, `metric_name_2`, at.QueryOpts{Time: ingestDateTime})
expected = apptest.MetricNamesStatsResponse{
Records: []at.MetricNamesStatsRecord{
{MetricName: largeMetricName, QueryRequestsCount: 1},
@@ -187,7 +187,7 @@ func TestClusterMetricNamesStats(t *testing.T) {
// ingest per tenant data and verify it with search
tenantIDs := []string{"1:1", "1:15", "15:15"}
for _, tenantID := range tenantIDs {
vminsert.APIV1ImportPrometheus(t, dataSet, apptest.QueryOpts{Tenant: tenantID})
vminsert.PrometheusAPIV1ImportPrometheus(t, dataSet, apptest.QueryOpts{Tenant: tenantID})
vmstorage1.ForceFlush(t)
vmstorage2.ForceFlush(t)
@@ -206,7 +206,7 @@ func TestClusterMetricNamesStats(t *testing.T) {
}
// verify query request registered correctly
vmselect.APIV1Query(t, `{__name__!=""}`, apptest.QueryOpts{
vmselect.PrometheusAPIV1Query(t, `{__name__!=""}`, apptest.QueryOpts{
Tenant: tenantID, Time: ingestDateTime,
})

View File

@@ -46,7 +46,7 @@ func TestClusterInstantQuery(t *testing.T) {
testQueryRangeWithAtModifier(t, sut)
}
func testInstantQueryWithUTFNames(t *testing.T, sut apptest.WriteQuerier) {
func testInstantQueryWithUTFNames(t *testing.T, sut apptest.PrometheusWriteQuerier) {
data := []pb.TimeSeries{
{
Labels: []pb.Label{
@@ -59,18 +59,18 @@ func testInstantQueryWithUTFNames(t *testing.T, sut apptest.WriteQuerier) {
},
}
sut.APIV1Write(t, data, apptest.QueryOpts{})
sut.PrometheusAPIV1Write(t, data, apptest.QueryOpts{})
sut.ForceFlush(t)
var got, want *apptest.APIV1QueryResponse
var got, want *apptest.PrometheusAPIV1QueryResponse
cmpOptions := []cmp.Option{
cmpopts.IgnoreFields(apptest.APIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.IgnoreFields(apptest.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.EquateNaNs(),
}
want = apptest.NewAPIV1QueryResponse(t, `{"data": {"result": [{"metric": {"__name__": "3fooµ¥", "3👋tfにちは": "漢©®€£"}}]}}`)
want = apptest.NewPrometheusAPIV1QueryResponse(t, `{"data": {"result": [{"metric": {"__name__": "3fooµ¥", "3👋tfにちは": "漢©®€£"}}]}}`)
fn := func(query string) {
got = sut.APIV1Query(t, query, apptest.QueryOpts{
got = sut.PrometheusAPIV1Query(t, query, apptest.QueryOpts{
Step: "5m",
Time: "2024-01-01T00:01:00.000Z",
})
@@ -112,24 +112,24 @@ var staleNaNsData = func() []pb.TimeSeries {
}
}()
func testInstantQueryDoesNotReturnStaleNaNs(t *testing.T, sut apptest.WriteQuerier) {
func testInstantQueryDoesNotReturnStaleNaNs(t *testing.T, sut apptest.PrometheusWriteQuerier) {
sut.APIV1Write(t, staleNaNsData, apptest.QueryOpts{})
sut.PrometheusAPIV1Write(t, staleNaNsData, apptest.QueryOpts{})
sut.ForceFlush(t)
var got, want *apptest.APIV1QueryResponse
var got, want *apptest.PrometheusAPIV1QueryResponse
cmpOptions := []cmp.Option{
cmpopts.IgnoreFields(apptest.APIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.IgnoreFields(apptest.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.EquateNaNs(),
}
// Verify that instant query returns the first point.
got = sut.APIV1Query(t, "metric", apptest.QueryOpts{
got = sut.PrometheusAPIV1Query(t, "metric", apptest.QueryOpts{
Step: "5m",
Time: "2024-01-01T00:01:00.000Z",
})
want = apptest.NewAPIV1QueryResponse(t, `{"data": {"result": [{"metric": {"__name__": "metric"}}]}}`)
want = apptest.NewPrometheusAPIV1QueryResponse(t, `{"data": {"result": [{"metric": {"__name__": "metric"}}]}}`)
want.Data.Result[0].Sample = apptest.NewSample(t, "2024-01-01T00:01:00Z", 1)
if diff := cmp.Diff(want, got, cmpOptions...); diff != "" {
t.Errorf("unexpected response (-want, +got):\n%s", diff)
@@ -137,11 +137,11 @@ func testInstantQueryDoesNotReturnStaleNaNs(t *testing.T, sut apptest.WriteQueri
// Verify that instant query does not return stale NaN.
got = sut.APIV1Query(t, "metric", apptest.QueryOpts{
got = sut.PrometheusAPIV1Query(t, "metric", apptest.QueryOpts{
Step: "5m",
Time: "2024-01-01T00:02:00.000Z",
})
want = apptest.NewAPIV1QueryResponse(t, `{"data": {"result": []}}`)
want = apptest.NewPrometheusAPIV1QueryResponse(t, `{"data": {"result": []}}`)
// Empty response, stale NaN is not included into response
if diff := cmp.Diff(want, got, cmpOptions...); diff != "" {
t.Errorf("unexpected response (-want, +got):\n%s", diff)
@@ -151,11 +151,11 @@ func testInstantQueryDoesNotReturnStaleNaNs(t *testing.T, sut apptest.WriteQueri
// while it must not.
// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5806
got = sut.APIV1Query(t, "metric[2m]", apptest.QueryOpts{
got = sut.PrometheusAPIV1Query(t, "metric[2m]", apptest.QueryOpts{
Step: "5m",
Time: "2024-01-01T00:02:00.000Z",
})
want = apptest.NewAPIV1QueryResponse(t, `{"data": {"result": [{"metric": {"__name__": "metric"}, "values": []}]}}`)
want = apptest.NewPrometheusAPIV1QueryResponse(t, `{"data": {"result": [{"metric": {"__name__": "metric"}, "values": []}]}}`)
s := make([]*apptest.Sample, 2)
s[0] = apptest.NewSample(t, "2024-01-01T00:01:00Z", 1)
s[1] = apptest.NewSample(t, "2024-01-01T00:02:00Z", decimal.StaleNaN)
@@ -166,11 +166,11 @@ func testInstantQueryDoesNotReturnStaleNaNs(t *testing.T, sut apptest.WriteQueri
// Verify that exported data contains stale NaN.
got = sut.APIV1Export(t, `{__name__="metric"}`, apptest.QueryOpts{
got = sut.PrometheusAPIV1Export(t, `{__name__="metric"}`, apptest.QueryOpts{
Start: "2024-01-01T00:01:00.000Z",
End: "2024-01-01T00:02:00.000Z",
})
want = apptest.NewAPIV1QueryResponse(t, `{"data": {"result": [{"metric": {"__name__": "metric"}, "values": []}]}}`)
want = apptest.NewPrometheusAPIV1QueryResponse(t, `{"data": {"result": [{"metric": {"__name__": "metric"}, "values": []}]}}`)
s = make([]*apptest.Sample, 2)
s[0] = apptest.NewSample(t, "2024-01-01T00:01:00Z", 1)
s[1] = apptest.NewSample(t, "2024-01-01T00:02:00Z", decimal.StaleNaN)
@@ -184,7 +184,7 @@ func testInstantQueryDoesNotReturnStaleNaNs(t *testing.T, sut apptest.WriteQueri
// See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8444
// However, conversion of math.NaN to int64 could behave differently depending on platform and Go version.
// Hence, this test could succeed for some platforms even if fix is rolled back.
func testQueryRangeWithAtModifier(t *testing.T, sut apptest.WriteQuerier) {
func testQueryRangeWithAtModifier(t *testing.T, sut apptest.PrometheusWriteQuerier) {
data := []pb.TimeSeries{
{
Labels: []pb.Label{
@@ -204,10 +204,10 @@ func testQueryRangeWithAtModifier(t *testing.T, sut apptest.WriteQuerier) {
},
}
sut.APIV1Write(t, data, apptest.QueryOpts{})
sut.PrometheusAPIV1Write(t, data, apptest.QueryOpts{})
sut.ForceFlush(t)
resp := sut.APIV1QueryRange(t, `vector(1) @ up`, apptest.QueryOpts{
resp := sut.PrometheusAPIV1QueryRange(t, `vector(1) @ up`, apptest.QueryOpts{
Start: "2025-01-01T00:00:00Z",
End: "2025-01-01T00:02:00Z",
Step: "10s",
@@ -217,7 +217,7 @@ func testQueryRangeWithAtModifier(t *testing.T, sut apptest.WriteQuerier) {
t.Fatalf("unexpected status: %q", resp.Status)
}
resp = sut.APIV1QueryRange(t, `vector(1) @ metricNaN`, apptest.QueryOpts{
resp = sut.PrometheusAPIV1QueryRange(t, `vector(1) @ metricNaN`, apptest.QueryOpts{
Start: "2025-01-01T00:00:00Z",
End: "2025-01-01T00:02:00Z",
Step: "10s",

View File

@@ -37,7 +37,7 @@ func TestClusterMultilevelSelect(t *testing.T) {
const numMetrics = 1000
records := make([]string, numMetrics)
want := &apptest.APIV1SeriesResponse{
want := &apptest.PrometheusAPIV1SeriesResponse{
Status: "success",
IsPartial: false,
Data: make([]map[string]string, numMetrics),
@@ -49,7 +49,7 @@ func TestClusterMultilevelSelect(t *testing.T) {
}
want.Sort()
qopts := apptest.QueryOpts{Tenant: "0"}
vminsert.APIV1ImportPrometheus(t, records, qopts)
vminsert.PrometheusAPIV1ImportPrometheus(t, records, qopts)
vmstorage.ForceFlush(t)
// Retrieve all time series and verify that both vmselect (L1) and
@@ -60,7 +60,7 @@ func TestClusterMultilevelSelect(t *testing.T) {
tc.Assert(&apptest.AssertOptions{
Msg: "unexpected /api/v1/series response",
Got: func() any {
res := app.APIV1Series(t, `{__name__=~".*"}`, qopts)
res := app.PrometheusAPIV1Series(t, `{__name__=~".*"}`, qopts)
res.Sort()
return res
},

View File

@@ -13,8 +13,8 @@ import (
func TestClusterMultiTenantSelect(t *testing.T) {
os.RemoveAll(t.Name())
cmpOpt := cmpopts.IgnoreFields(apptest.APIV1QueryResponse{}, "Status", "Data.ResultType")
cmpSROpt := cmpopts.IgnoreFields(apptest.APIV1SeriesResponse{}, "Status", "IsPartial")
cmpOpt := cmpopts.IgnoreFields(apptest.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType")
cmpSROpt := cmpopts.IgnoreFields(apptest.PrometheusAPIV1SeriesResponse{}, "Status", "IsPartial")
tc := apptest.NewTestCase(t)
defer tc.Stop()
@@ -37,12 +37,12 @@ func TestClusterMultiTenantSelect(t *testing.T) {
}
// test for empty tenants request
got := vmselect.APIV1Query(t, "foo_bar", apptest.QueryOpts{
got := vmselect.PrometheusAPIV1Query(t, "foo_bar", apptest.QueryOpts{
Tenant: "multitenant",
Step: "5m",
Time: "2022-05-10T08:03:00.000Z",
})
want := apptest.NewAPIV1QueryResponse(t, `{"data":{"result":[]}}`)
want := apptest.NewPrometheusAPIV1QueryResponse(t, `{"data":{"result":[]}}`)
if diff := cmp.Diff(want, got, cmpOpt); diff != "" {
t.Errorf("unexpected response (-want, +got):\n%s", diff)
}
@@ -51,12 +51,12 @@ func TestClusterMultiTenantSelect(t *testing.T) {
tenantIDs := []string{"1:1", "1:15"}
instantCT := "2022-05-10T08:05:00.000Z"
for _, tenantID := range tenantIDs {
vminsert.APIV1ImportPrometheus(t, commonSamples, apptest.QueryOpts{Tenant: tenantID})
vminsert.PrometheusAPIV1ImportPrometheus(t, commonSamples, apptest.QueryOpts{Tenant: tenantID})
vmstorage.ForceFlush(t)
got := vmselect.APIV1Query(t, "foo_bar", apptest.QueryOpts{
got := vmselect.PrometheusAPIV1Query(t, "foo_bar", apptest.QueryOpts{
Tenant: tenantID, Time: instantCT,
})
want := apptest.NewAPIV1QueryResponse(t, `{"data":{"result":[{"metric":{"__name__":"foo_bar"},"value":[1652169900,"3"]}]}}`)
want := apptest.NewPrometheusAPIV1QueryResponse(t, `{"data":{"result":[{"metric":{"__name__":"foo_bar"},"value":[1652169900,"3"]}]}}`)
if diff := cmp.Diff(want, got, cmpOpt); diff != "" {
t.Errorf("unexpected response (-want, +got):\n%s", diff)
}
@@ -64,7 +64,7 @@ func TestClusterMultiTenantSelect(t *testing.T) {
// verify all tenants searchable with multitenant APIs
// /api/v1/query
want = apptest.NewAPIV1QueryResponse(t,
want = apptest.NewPrometheusAPIV1QueryResponse(t,
`{"data":
{"result":[
{"metric":{"__name__":"foo_bar","vm_account_id":"1","vm_project_id": "1"},"value":[1652169900,"3"]},
@@ -73,7 +73,7 @@ func TestClusterMultiTenantSelect(t *testing.T) {
}
}`,
)
got = vmselect.APIV1Query(t, "foo_bar", apptest.QueryOpts{
got = vmselect.PrometheusAPIV1Query(t, "foo_bar", apptest.QueryOpts{
Tenant: "multitenant",
Time: instantCT,
})
@@ -83,14 +83,14 @@ func TestClusterMultiTenantSelect(t *testing.T) {
// /api/v1/query_range aggregated by tenant labels
query := "sum(foo_bar) by(vm_account_id,vm_project_id)"
got = vmselect.APIV1QueryRange(t, query, apptest.QueryOpts{
got = vmselect.PrometheusAPIV1QueryRange(t, query, apptest.QueryOpts{
Tenant: "multitenant",
Start: "2022-05-10T07:59:00.000Z",
End: "2022-05-10T08:05:00.000Z",
Step: "1m",
})
want = apptest.NewAPIV1QueryResponse(t,
want = apptest.NewPrometheusAPIV1QueryResponse(t,
`{"data":
{"result": [
{"metric": {"vm_account_id": "1","vm_project_id":"1"}, "values": [[1652169600,"1"],[1652169660,"2"],[1652169720,"3"],[1652169780,"3"]]},
@@ -104,7 +104,7 @@ func TestClusterMultiTenantSelect(t *testing.T) {
// verify /api/v1/series response
wantSR := apptest.NewAPIV1SeriesResponse(t,
wantSR := apptest.NewPrometheusAPIV1SeriesResponse(t,
`{"data": [
{"__name__":"foo_bar", "vm_account_id":"1", "vm_project_id":"1"},
{"__name__":"foo_bar", "vm_account_id":"1", "vm_project_id":"15"}
@@ -112,7 +112,7 @@ func TestClusterMultiTenantSelect(t *testing.T) {
}`)
wantSR.Sort()
gotSR := vmselect.APIV1Series(t, "foo_bar", apptest.QueryOpts{
gotSR := vmselect.PrometheusAPIV1Series(t, "foo_bar", apptest.QueryOpts{
Tenant: "multitenant",
Start: "2022-05-10T08:03:00.000Z",
})
@@ -129,11 +129,11 @@ func TestClusterMultiTenantSelect(t *testing.T) {
`foo_bar{vm_account_id="5",vm_project_id="15"} 3.00 1652169720000`, // 2022-05-10T08:02:00Z
}
vminsert.APIV1ImportPrometheus(t, tenantLabelsSamples, apptest.QueryOpts{Tenant: "multitenant"})
vminsert.PrometheusAPIV1ImportPrometheus(t, tenantLabelsSamples, apptest.QueryOpts{Tenant: "multitenant"})
vmstorage.ForceFlush(t)
// /api/v1/query with query filters
want = apptest.NewAPIV1QueryResponse(t,
want = apptest.NewPrometheusAPIV1QueryResponse(t,
`{"data":
{"result":[
{"metric":{"__name__":"foo_bar","vm_account_id":"5","vm_project_id": "0"},"value":[1652169900,"1"]},
@@ -142,7 +142,7 @@ func TestClusterMultiTenantSelect(t *testing.T) {
}
}`,
)
got = vmselect.APIV1Query(t, `foo_bar{vm_account_id="5"}`, apptest.QueryOpts{
got = vmselect.PrometheusAPIV1Query(t, `foo_bar{vm_account_id="5"}`, apptest.QueryOpts{
Time: instantCT,
Tenant: "multitenant",
})
@@ -152,14 +152,14 @@ func TestClusterMultiTenantSelect(t *testing.T) {
// /api/v1/series with extra_filters
wantSR = apptest.NewAPIV1SeriesResponse(t,
wantSR = apptest.NewPrometheusAPIV1SeriesResponse(t,
`{"data": [
{"__name__":"foo_bar", "vm_account_id":"5", "vm_project_id":"15"},
{"__name__":"foo_bar", "vm_account_id":"1", "vm_project_id":"15"}
]
}`)
wantSR.Sort()
gotSR = vmselect.APIV1Series(t, "foo_bar", apptest.QueryOpts{
gotSR = vmselect.PrometheusAPIV1Series(t, "foo_bar", apptest.QueryOpts{
Start: "2022-05-10T08:00:00.000Z",
End: "2022-05-10T08:30:00.000Z",
ExtraFilters: []string{`{vm_project_id="15"}`},
@@ -175,7 +175,7 @@ func TestClusterMultiTenantSelect(t *testing.T) {
vmselect.DeleteSeries(t, "foo_bar", apptest.QueryOpts{
Tenant: "5:15",
})
wantSR = apptest.NewAPIV1SeriesResponse(t,
wantSR = apptest.NewPrometheusAPIV1SeriesResponse(t,
`{"data": [
{"__name__":"foo_bar", "vm_account_id":"0", "vm_project_id":"10"},
{"__name__":"foo_bar", "vm_account_id":"1", "vm_project_id":"1"},
@@ -185,7 +185,7 @@ func TestClusterMultiTenantSelect(t *testing.T) {
}`)
wantSR.Sort()
gotSR = vmselect.APIV1Series(t, "foo_bar", apptest.QueryOpts{
gotSR = vmselect.PrometheusAPIV1Series(t, "foo_bar", apptest.QueryOpts{
Tenant: "multitenant",
Start: "2022-05-10T08:03:00.000Z",
})
@@ -199,7 +199,7 @@ func TestClusterMultiTenantSelect(t *testing.T) {
Tenant: "multitenant",
})
wantSR = apptest.NewAPIV1SeriesResponse(t,
wantSR = apptest.NewPrometheusAPIV1SeriesResponse(t,
`{"data": [
{"__name__":"foo_bar", "vm_account_id":"0", "vm_project_id":"10"},
{"__name__":"foo_bar", "vm_account_id":"5", "vm_project_id":"0"}
@@ -207,7 +207,7 @@ func TestClusterMultiTenantSelect(t *testing.T) {
}`)
wantSR.Sort()
gotSR = vmselect.APIV1Series(t, `foo_bar`, apptest.QueryOpts{
gotSR = vmselect.PrometheusAPIV1Series(t, `foo_bar`, apptest.QueryOpts{
Tenant: "multitenant",
Start: "2022-05-10T08:03:00.000Z",
})

View File

@@ -11,7 +11,7 @@ func TestSingleSearchWithDisabledPerDayIndex(t *testing.T) {
tc := at.NewTestCase(t)
defer tc.Stop()
testSearchWithDisabledPerDayIndex(tc, func(name string, disablePerDayIndex bool) at.WriteQuerier {
testSearchWithDisabledPerDayIndex(tc, func(name string, disablePerDayIndex bool) at.PrometheusWriteQuerier {
return tc.MustStartVmsingle("vmsingle-"+name, []string{
"-storageDataPath=" + tc.Dir() + "/vmsingle",
"-retentionPeriod=100y",
@@ -25,7 +25,7 @@ func TestClusterSearchWithDisabledPerDayIndex(t *testing.T) {
tc := at.NewTestCase(t)
defer tc.Stop()
testSearchWithDisabledPerDayIndex(tc, func(name string, disablePerDayIndex bool) at.WriteQuerier {
testSearchWithDisabledPerDayIndex(tc, func(name string, disablePerDayIndex bool) at.PrometheusWriteQuerier {
// Using static ports for vmstorage because random ports may cause
// changes in how data is sharded.
vmstorage1 := tc.MustStartVmstorage("vmstorage1-"+name, []string{
@@ -59,7 +59,7 @@ func TestClusterSearchWithDisabledPerDayIndex(t *testing.T) {
})
}
type startSUTFunc func(name string, disablePerDayIndex bool) at.WriteQuerier
type startSUTFunc func(name string, disablePerDayIndex bool) at.PrometheusWriteQuerier
// testDisablePerDayIndex_Search shows what search results to expect when data
// is first inserted with per-day index enabled and then with per-day index
@@ -78,17 +78,17 @@ func testSearchWithDisabledPerDayIndex(tc *at.TestCase, start startSUTFunc) {
wantSeries []map[string]string
wantQueryResults []*at.QueryResult
}
assertSearchResults := func(sut at.APIQuerier, opts *opts) {
assertSearchResults := func(sut at.PrometheusQuerier, opts *opts) {
t.Helper()
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/series response",
Got: func() any {
return sut.APIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{
return sut.PrometheusAPIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{
Start: opts.start,
End: opts.end,
}).Sort()
},
Want: &at.APIV1SeriesResponse{
Want: &at.PrometheusAPIV1SeriesResponse{
Status: "success",
Data: opts.wantSeries,
},
@@ -96,13 +96,13 @@ func testSearchWithDisabledPerDayIndex(tc *at.TestCase, start startSUTFunc) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query_range response",
Got: func() any {
return sut.APIV1QueryRange(t, `{__name__=~".*"}`, at.QueryOpts{
return sut.PrometheusAPIV1QueryRange(t, `{__name__=~".*"}`, at.QueryOpts{
Start: opts.start,
End: opts.end,
Step: "1d",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -116,7 +116,7 @@ func testSearchWithDisabledPerDayIndex(tc *at.TestCase, start startSUTFunc) {
// is searchable.
sut := start("with-per-day-index", false)
sample1 := []string{"metric1 111 1704067200000"} // 2024-01-01T00:00:00Z
sut.APIV1ImportPrometheus(t, sample1, at.QueryOpts{})
sut.PrometheusAPIV1ImportPrometheus(t, sample1, at.QueryOpts{})
sut.ForceFlush(t)
assertSearchResults(sut, &opts{
start: "2024-01-01T00:00:00Z",
@@ -132,10 +132,10 @@ func testSearchWithDisabledPerDayIndex(tc *at.TestCase, start startSUTFunc) {
// Restart vmsingle with disabled per-day index, insert sample2, and confirm
// that both sample1 and sample2 is searchable.
tc.StopWriteQuerier(sut)
tc.StopPrometheusWriteQuerier(sut)
sut = start("without-per-day-index", true)
sample2 := []string{"metric2 222 1704067200000"} // 2024-01-01T00:00:00Z
sut.APIV1ImportPrometheus(t, sample2, at.QueryOpts{})
sut.PrometheusAPIV1ImportPrometheus(t, sample2, at.QueryOpts{})
sut.ForceFlush(t)
assertSearchResults(sut, &opts{
start: "2024-01-01T00:00:00Z",
@@ -165,9 +165,9 @@ func testSearchWithDisabledPerDayIndex(tc *at.TestCase, start startSUTFunc) {
// - sample2 is not searchable when the time range is <= 40 days
// - sample2 becomes searchable when the time range is > 40 days
sample3 := []string{"metric1 333 1705708800000"} // 2024-01-20T00:00:00Z
sut.APIV1ImportPrometheus(t, sample3, at.QueryOpts{})
sut.PrometheusAPIV1ImportPrometheus(t, sample3, at.QueryOpts{})
sut.ForceFlush(t)
tc.StopWriteQuerier(sut)
tc.StopPrometheusWriteQuerier(sut)
sut = start("with-per-day-index2", false)
// Time range is 1 day (Jan 1st) <= 40 days
@@ -295,14 +295,14 @@ func testClusterActiveTimeseriesMetric(t *testing.T, disablePerDayIndex bool) {
})
}
func testActiveTimeseriesMetric(tc *at.TestCase, sut at.WriteQuerier, getActiveTimeseries func() int) {
func testActiveTimeseriesMetric(tc *at.TestCase, sut at.PrometheusWriteQuerier, getActiveTimeseries func() int) {
t := tc.T()
const numSamples = 1000
samples := make([]string, numSamples)
for i := range numSamples {
samples[i] = fmt.Sprintf("metric_%03d %d", i, i)
}
sut.APIV1ImportPrometheus(t, samples, at.QueryOpts{})
sut.PrometheusAPIV1ImportPrometheus(t, samples, at.QueryOpts{})
sut.ForceFlush(t)
tc.Assert(&at.AssertOptions{
Msg: `unexpected vm_cache_entries{type="storage/hour_metric_ids"} metric value`,

View File

@@ -13,7 +13,7 @@ import (
func TestClusterMaxUniqueTimeseries(t *testing.T) {
os.RemoveAll(t.Name())
cmpOpt := cmpopts.IgnoreFields(apptest.APIV1QueryResponse{}, "Status", "Data.ResultType")
cmpOpt := cmpopts.IgnoreFields(apptest.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType")
tc := apptest.NewTestCase(t)
defer tc.Stop()
@@ -54,14 +54,14 @@ func TestClusterMaxUniqueTimeseries(t *testing.T) {
// write data to two tenants
tenantIDs := []string{"0:0", "1:15"}
for _, tenantID := range tenantIDs {
vminsert.APIV1ImportPrometheus(t, commonSamples, apptest.QueryOpts{Tenant: tenantID})
vminsert.PrometheusAPIV1ImportPrometheus(t, commonSamples, apptest.QueryOpts{Tenant: tenantID})
vmstorage.ForceFlush(t)
}
instantCT := "2022-05-10T08:05:00.000Z"
// success - `/api/v1/query`
want := apptest.NewAPIV1QueryResponse(t,
want := apptest.NewPrometheusAPIV1QueryResponse(t,
`{"data":
{"result":[
{"metric":{"__name__":"foo_bar1","instance":"a"},"value":[1652169900,"1"]}
@@ -69,7 +69,7 @@ func TestClusterMaxUniqueTimeseries(t *testing.T) {
}
}`,
)
queryRes := vmselectSmallLimit.APIV1Query(t, "foo_bar1", apptest.QueryOpts{
queryRes := vmselectSmallLimit.PrometheusAPIV1Query(t, "foo_bar1", apptest.QueryOpts{
Time: instantCT,
})
if diff := cmp.Diff(want, queryRes, cmpOpt); diff != "" {
@@ -78,7 +78,7 @@ func TestClusterMaxUniqueTimeseries(t *testing.T) {
// success - multitenant `/api/v1/query`
// query is split into two queries for each tenant, so the final result can exceed the limit.
want = apptest.NewAPIV1QueryResponse(t,
want = apptest.NewPrometheusAPIV1QueryResponse(t,
`{"data":
{"result":[
{"metric":{"__name__":"foo_bar1","instance":"a","vm_account_id":"0","vm_project_id":"0"},"value":[1652169900,"1"]},
@@ -87,7 +87,7 @@ func TestClusterMaxUniqueTimeseries(t *testing.T) {
}
}`,
)
queryRes = vmselectSmallLimit.APIV1Query(t, "foo_bar1", apptest.QueryOpts{
queryRes = vmselectSmallLimit.PrometheusAPIV1Query(t, "foo_bar1", apptest.QueryOpts{
Time: instantCT,
Tenant: "multitenant",
})
@@ -96,7 +96,7 @@ func TestClusterMaxUniqueTimeseries(t *testing.T) {
}
// fail - `/api/v1/query`, exceed vmselect `maxUniqueTimeseries`
queryRes = vmselectSmallLimit.APIV1Query(t, "foo_bar2", apptest.QueryOpts{
queryRes = vmselectSmallLimit.PrometheusAPIV1Query(t, "foo_bar2", apptest.QueryOpts{
Time: instantCT,
})
if queryRes.ErrorType != "422" {
@@ -104,7 +104,7 @@ func TestClusterMaxUniqueTimeseries(t *testing.T) {
}
// fail - `/api/v1/query`, exceed vmstorage `maxUniqueTimeseries`
queryRes = vmselectNoLimit.APIV1Query(t, "foo_bar3", apptest.QueryOpts{
queryRes = vmselectNoLimit.PrometheusAPIV1Query(t, "foo_bar3", apptest.QueryOpts{
Time: instantCT,
})
if queryRes.ErrorType != "422" {
@@ -112,7 +112,7 @@ func TestClusterMaxUniqueTimeseries(t *testing.T) {
}
// fail - `/api/v1/query`, vmselect `maxUniqueTimeseries` cannot exceed vmstorage `maxUniqueTimeseries`
queryRes = vmselectBigLimit.APIV1Query(t, "foo_bar3", apptest.QueryOpts{
queryRes = vmselectBigLimit.PrometheusAPIV1Query(t, "foo_bar3", apptest.QueryOpts{
Time: instantCT,
})
if queryRes.ErrorType != "422" {
@@ -123,7 +123,7 @@ func TestClusterMaxUniqueTimeseries(t *testing.T) {
func TestClusterMaxSeries(t *testing.T) {
os.RemoveAll(t.Name())
cmpSROpt := cmpopts.IgnoreFields(apptest.APIV1SeriesResponse{}, "Status", "IsPartial")
cmpSROpt := cmpopts.IgnoreFields(apptest.PrometheusAPIV1SeriesResponse{}, "Status", "IsPartial")
tc := apptest.NewTestCase(t)
defer tc.Stop()
@@ -153,18 +153,18 @@ func TestClusterMaxSeries(t *testing.T) {
}
// write data
vminsert.APIV1ImportPrometheus(t, commonSamples, apptest.QueryOpts{})
vminsert.PrometheusAPIV1ImportPrometheus(t, commonSamples, apptest.QueryOpts{})
vmstorage.ForceFlush(t)
// success - `/api/v1/series`, vmselect `maxLabelsAPISeries` can exceed vmstorage `maxLabelsAPISeries``
wantSR := apptest.NewAPIV1SeriesResponse(t,
wantSR := apptest.NewPrometheusAPIV1SeriesResponse(t,
`{"data": [
{"__name__":"foo_bar3","instance":"a"},
{"__name__":"foo_bar3","instance":"b"},
{"__name__":"foo_bar3","instance":"c"}
]
}`)
seriesRes := vmselectBigLimit.APIV1Series(t, "foo_bar3", apptest.QueryOpts{
seriesRes := vmselectBigLimit.PrometheusAPIV1Series(t, "foo_bar3", apptest.QueryOpts{
Start: "2022-05-10T08:03:00.000Z",
})
if diff := cmp.Diff(wantSR.Sort(), seriesRes.Sort(), cmpSROpt); diff != "" {
@@ -172,7 +172,7 @@ func TestClusterMaxSeries(t *testing.T) {
}
// fail - `/api/v1/series`, exceed vmselect `maxSeries`
seriesRes1 := vmselectSmallLimit.APIV1Series(t, "foo_bar3", apptest.QueryOpts{
seriesRes1 := vmselectSmallLimit.PrometheusAPIV1Series(t, "foo_bar3", apptest.QueryOpts{
Start: "2022-05-10T08:03:00.000Z",
})
if seriesRes1.ErrorType != "422" {

View File

@@ -57,7 +57,7 @@ func TestSingleIngestionWithRelabeling(t *testing.T) {
wantMetrics []map[string]string
wantSamples []*at.Sample
}
f := func(sut at.APIQuerier, opts *opts) {
f := func(sut at.PrometheusQuerier, opts *opts) {
t.Helper()
wantResult := []*at.QueryResult{}
for idx, wm := range opts.wantMetrics {
@@ -70,19 +70,19 @@ func TestSingleIngestionWithRelabeling(t *testing.T) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query response",
Got: func() any {
return sut.APIV1Query(t, opts.query, at.QueryOpts{
return sut.PrometheusAPIV1Query(t, opts.query, at.QueryOpts{
Time: opts.qtime,
Step: opts.step,
})
},
Want: &at.APIV1QueryResponse{Data: &at.QueryData{Result: wantResult}},
Want: &at.PrometheusAPIV1QueryResponse{Data: &at.QueryData{Result: wantResult}},
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(at.APIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.IgnoreFields(at.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
},
})
}
sut.APIV1ImportPrometheus(t, []string{
sut.PrometheusAPIV1ImportPrometheus(t, []string{
`importprometheus_series{label="foo"} 10 1707123456700`, // 2024-02-05T08:57:36.700Z
`must_drop_series{label="foo"} 20 1707123456800`, // 2024-02-05T08:57:36.800Z
}, at.QueryOpts{})
@@ -180,7 +180,7 @@ func TestSingleIngestionWithRelabeling(t *testing.T) {
},
},
}
sut.APIV1Write(t, pbData, at.QueryOpts{})
sut.PrometheusAPIV1Write(t, pbData, at.QueryOpts{})
sut.ForceFlush(t)
f(sut, &opts{
query: `{label="foo2"}[120ms]`,

View File

@@ -7,11 +7,10 @@ import (
"testing"
"time"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/VictoriaMetrics/VictoriaMetrics/apptest"
at "github.com/VictoriaMetrics/VictoriaMetrics/apptest"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
)
type clusterWithReplication struct {
@@ -95,7 +94,7 @@ func TestClusterReplication_DataIsWrittenSeveralTimes(t *testing.T) {
for i := range numRecs {
recs[i] = fmt.Sprintf("metric_%d %d", i, rand.IntN(1000))
}
c.vminsert.APIV1ImportPrometheus(t, recs, at.QueryOpts{})
c.vminsert.PrometheusAPIV1ImportPrometheus(t, recs, at.QueryOpts{})
tc.ForceFlush(c.vmstorages...)
// Verify that each storage node has metrics and that total metric count across
@@ -152,7 +151,7 @@ func TestClusterReplication_Deduplication(t *testing.T) {
ts = ts.Add(1 * time.Minute)
}
}
c.vminsert.APIV1ImportPrometheus(t, recs, at.QueryOpts{})
c.vminsert.PrometheusAPIV1ImportPrometheus(t, recs, at.QueryOpts{})
tc.ForceFlush(c.vmstorages...)
// Check /api/v1/series response.
@@ -165,12 +164,12 @@ func TestClusterReplication_Deduplication(t *testing.T) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/series response",
Got: func() any {
return app.APIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{
return app.PrometheusAPIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{
Start: "2024-01-01T00:00:00Z",
End: "2024-01-31T00:00:00Z",
}).Sort()
},
Want: &at.APIV1SeriesResponse{
Want: &at.PrometheusAPIV1SeriesResponse{
Status: "success",
IsPartial: false,
Data: []map[string]string{
@@ -195,12 +194,12 @@ func TestClusterReplication_Deduplication(t *testing.T) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query response",
Got: func() any {
return app.APIV1Query(t, "metric_1", at.QueryOpts{
return app.PrometheusAPIV1Query(t, "metric_1", at.QueryOpts{
Time: "2024-01-01T00:05:00Z",
Step: "5m",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "vector",
@@ -237,12 +236,12 @@ func TestClusterReplication_Deduplication(t *testing.T) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query response",
Got: func() any {
return app.APIV1Query(t, "metric_1[5m]", at.QueryOpts{
return app.PrometheusAPIV1Query(t, "metric_1[5m]", at.QueryOpts{
Time: "2024-01-01T00:05:00Z",
Step: "5m",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -274,13 +273,13 @@ func TestClusterReplication_Deduplication(t *testing.T) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query_range response",
Got: func() any {
return app.APIV1QueryRange(t, "metric_1", at.QueryOpts{
return app.PrometheusAPIV1QueryRange(t, "metric_1", at.QueryOpts{
Start: "2024-01-01T00:00:00Z",
End: "2024-01-01T00:10:00Z",
Step: "5m",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -310,12 +309,12 @@ func TestClusterReplication_Deduplication(t *testing.T) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/export response",
Got: func() any {
return app.APIV1Export(t, `{__name__="metric_1"}`, at.QueryOpts{
return app.PrometheusAPIV1Export(t, `{__name__="metric_1"}`, at.QueryOpts{
Start: "2024-01-01T00:00:00Z",
End: "2024-01-01T00:03:00Z",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -361,7 +360,7 @@ func TestClusterReplication_PartialResponse(t *testing.T) {
for i := range numRecs {
recs[i] = fmt.Sprintf("metric_%d %d", i, rand.IntN(1000))
}
c.vminsert.APIV1ImportPrometheus(t, recs, at.QueryOpts{})
c.vminsert.PrometheusAPIV1ImportPrometheus(t, recs, at.QueryOpts{})
tc.ForceFlush(c.vmstorages...)
// Verify partial vs full response.
@@ -371,14 +370,14 @@ func TestClusterReplication_PartialResponse(t *testing.T) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/series response",
Got: func() any {
return app.APIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{}).Sort()
return app.PrometheusAPIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{}).Sort()
},
Want: &at.APIV1SeriesResponse{
Want: &at.PrometheusAPIV1SeriesResponse{
Status: "success",
IsPartial: wantPartial,
},
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(apptest.APIV1SeriesResponse{}, "Data"),
cmpopts.IgnoreFields(apptest.PrometheusAPIV1SeriesResponse{}, "Data"),
},
})
}
@@ -438,7 +437,7 @@ func TestClusterReplication_SkipSlowReplicas(t *testing.T) {
const numRecs = 1000
recs := make([]string, numRecs)
wantSeries := &at.APIV1SeriesResponse{
wantSeries := &at.PrometheusAPIV1SeriesResponse{
Status: "success",
Data: make([]map[string]string, numRecs),
}
@@ -448,7 +447,7 @@ func TestClusterReplication_SkipSlowReplicas(t *testing.T) {
wantSeries.Data[i] = map[string]string{"__name__": name}
}
wantSeries.Sort()
c.vminsert.APIV1ImportPrometheus(t, recs, at.QueryOpts{})
c.vminsert.PrometheusAPIV1ImportPrometheus(t, recs, at.QueryOpts{})
tc.ForceFlush(c.vmstorages...)
// Verify skipping slow replicas by counting the number of skipSlowReplicas
@@ -459,12 +458,12 @@ func TestClusterReplication_SkipSlowReplicas(t *testing.T) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/series response",
Got: func() any {
return app.APIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{}).Sort()
return app.PrometheusAPIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{}).Sort()
},
Want: wantSeries,
})
res := app.APIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{Trace: "1"})
res := app.PrometheusAPIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{Trace: "1"})
got := res.Trace.Contains("cancel request because -search.skipSlowReplicas is set and every group returned the needed number of responses according to replicationFactor")
if got != want {
t.Errorf("unexpected number of skipSlowReplicas messages in request trace: got %d, want %d (full trace:\n%v)", got, want, res.Trace)
@@ -655,7 +654,7 @@ func TestClusterGroupReplication(t *testing.T) {
numRecs = numMetrics * numSamples
)
var recs []string
wantSeries := &at.APIV1SeriesResponse{
wantSeries := &at.PrometheusAPIV1SeriesResponse{
Status: "success",
Data: make([]map[string]string, numMetrics),
}
@@ -669,7 +668,7 @@ func TestClusterGroupReplication(t *testing.T) {
}
}
wantSeries.Sort()
c.vminsert.APIV1ImportPrometheus(t, recs, at.QueryOpts{})
c.vminsert.PrometheusAPIV1ImportPrometheus(t, recs, at.QueryOpts{})
c.forceFlush(tc)
opts := &testGroupReplicationOpts{
@@ -695,7 +694,7 @@ type testGroupReplicationOpts struct {
numGroups int
numNodes int
numRecs int
wantSeries *at.APIV1SeriesResponse
wantSeries *at.PrometheusAPIV1SeriesResponse
}
// testGroupDataIsWrittenSeveralTimes checks that multiple
@@ -748,7 +747,7 @@ func testGroupDeduplication(tc *at.TestCase, opts *testGroupReplicationOpts) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/series response",
Got: func() any {
return app.APIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{
return app.PrometheusAPIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{
Start: "2024-01-01T00:00:00Z",
End: "2024-01-31T00:00:00Z",
}).Sort()
@@ -769,12 +768,12 @@ func testGroupDeduplication(tc *at.TestCase, opts *testGroupReplicationOpts) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query response",
Got: func() any {
return app.APIV1Query(t, "metric_1", at.QueryOpts{
return app.PrometheusAPIV1Query(t, "metric_1", at.QueryOpts{
Time: "2024-01-01T00:05:00Z",
Step: "5m",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "vector",
@@ -811,12 +810,12 @@ func testGroupDeduplication(tc *at.TestCase, opts *testGroupReplicationOpts) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query response",
Got: func() any {
return app.APIV1Query(t, "metric_1[5m]", at.QueryOpts{
return app.PrometheusAPIV1Query(t, "metric_1[5m]", at.QueryOpts{
Time: "2024-01-01T00:05:00Z",
Step: "5m",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -848,13 +847,13 @@ func testGroupDeduplication(tc *at.TestCase, opts *testGroupReplicationOpts) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query_range response",
Got: func() any {
return app.APIV1QueryRange(t, "metric_1", at.QueryOpts{
return app.PrometheusAPIV1QueryRange(t, "metric_1", at.QueryOpts{
Start: "2024-01-01T00:00:00Z",
End: "2024-01-01T00:10:00Z",
Step: "5m",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -884,12 +883,12 @@ func testGroupDeduplication(tc *at.TestCase, opts *testGroupReplicationOpts) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/export response",
Got: func() any {
return app.APIV1Export(t, `{__name__="metric_1"}`, at.QueryOpts{
return app.PrometheusAPIV1Export(t, `{__name__="metric_1"}`, at.QueryOpts{
Start: "2024-01-01T00:00:00Z",
End: "2024-01-01T00:03:00Z",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -930,7 +929,7 @@ func testGroupSkipSlowReplicas(tc *at.TestCase, opts *testGroupReplicationOpts)
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/series response",
Got: func() any {
return app.APIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{
return app.PrometheusAPIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{
Start: "2024-01-01T00:00:00Z",
End: "2024-01-31T00:00:00Z",
}).Sort()
@@ -938,7 +937,7 @@ func testGroupSkipSlowReplicas(tc *at.TestCase, opts *testGroupReplicationOpts)
Want: opts.wantSeries,
})
res := app.APIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{Trace: "1"})
res := app.PrometheusAPIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{Trace: "1"})
got := res.Trace.Contains("cancel request because -search.skipSlowReplicas is set and every group returned the needed number of responses according to replicationFactor")
if got < wantMin || got > wantMax {
t.Errorf("unexpected number of skipSlowReplicas messages in request trace: got %d, %d <= want <= %d (full trace:\n%v)", got, wantMin, wantMax, res.Trace)
@@ -974,7 +973,7 @@ func testGroupSkipSlowReplicas(tc *at.TestCase, opts *testGroupReplicationOpts)
// The data is replicated across N groups of M nodes. Replication factor is
// globalRF. There is no replication across the nodes within each group or
// it is unknown it there is one.
//it is unknown it there is one.
//
// Max number of nodes to skip is M*(globalRF-1). This corresponds to the
// case when N-globalRF+1 groups have received the response from all of
@@ -1021,17 +1020,17 @@ func testGroupPartialResponse(tc *at.TestCase, opts *testGroupReplicationOpts) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/series response",
Got: func() any {
return app.APIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{
return app.PrometheusAPIV1Series(t, `{__name__=~".*"}`, at.QueryOpts{
Start: "2024-01-01T00:00:00Z",
End: "2024-01-31T00:00:00Z",
}).Sort()
},
Want: &at.APIV1SeriesResponse{
Want: &at.PrometheusAPIV1SeriesResponse{
Status: "success",
IsPartial: wantPartial,
},
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(apptest.APIV1SeriesResponse{}, "Data"),
cmpopts.IgnoreFields(apptest.PrometheusAPIV1SeriesResponse{}, "Data"),
},
})
}
@@ -1129,10 +1128,10 @@ func TestClusterReplication_PartialResponseMultitenant(t *testing.T) {
recs[i] = fmt.Sprintf("metric_%d %d", i, rand.IntN(1000))
}
c.vminsert.APIV1ImportPrometheus(t, recs, at.QueryOpts{
c.vminsert.PrometheusAPIV1ImportPrometheus(t, recs, at.QueryOpts{
Tenant: "0",
})
c.vminsert.APIV1ImportPrometheus(t, recs, at.QueryOpts{
c.vminsert.PrometheusAPIV1ImportPrometheus(t, recs, at.QueryOpts{
Tenant: "1",
})
tc.ForceFlush(c.vmstorages...)
@@ -1145,14 +1144,14 @@ func TestClusterReplication_PartialResponseMultitenant(t *testing.T) {
Msg: "unexpected /api/v1/query response",
Got: func() any {
qo := at.QueryOpts{Tenant: "multitenant", Trace: "1"}
return app.APIV1Query(t, `{__name__=~"metric_.*"}`, qo)
return app.PrometheusAPIV1Query(t, `{__name__=~"metric_.*"}`, qo)
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
IsPartial: wantPartial,
},
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(apptest.APIV1QueryResponse{}, "Data"),
cmpopts.IgnoreFields(apptest.PrometheusAPIV1QueryResponse{}, "Data"),
},
})
}

View File

@@ -13,7 +13,7 @@ import (
func TestClusterRollupResultCache(t *testing.T) {
os.RemoveAll(t.Name())
cmpOpt := cmpopts.IgnoreFields(apptest.APIV1QueryResponse{}, "Status", "Data.ResultType")
cmpOpt := cmpopts.IgnoreFields(apptest.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType")
tc := apptest.NewTestCase(t)
defer tc.Stop()
@@ -34,10 +34,10 @@ func TestClusterRollupResultCache(t *testing.T) {
`foo_bar{vm_account_id="5",vm_project_id="15"} 3.00 1652169720000`, // 2022-05-10T08:02:00Z
}
vminsert.APIV1ImportPrometheus(t, tenantLabelsSamples, apptest.QueryOpts{Tenant: "multitenant"})
vminsert.PrometheusAPIV1ImportPrometheus(t, tenantLabelsSamples, apptest.QueryOpts{Tenant: "multitenant"})
vmstorage.ForceFlush(t)
want := apptest.NewAPIV1QueryResponse(t,
want := apptest.NewPrometheusAPIV1QueryResponse(t,
`{"data":
{"result":[
{"metric":{"__name__":"foo_bar","vm_account_id":"5","vm_project_id": "0"},"values":[[1652169720,"1"],[1652169780,"1"]]},
@@ -47,7 +47,7 @@ func TestClusterRollupResultCache(t *testing.T) {
}`,
)
got := vmselect.APIV1QueryRange(t, `foo_bar{}`, apptest.QueryOpts{
got := vmselect.PrometheusAPIV1QueryRange(t, `foo_bar{}`, apptest.QueryOpts{
Tenant: "multitenant",
Start: "2022-05-10T07:59:00.000Z",
End: "2022-05-10T08:05:00.000Z",
@@ -58,13 +58,13 @@ func TestClusterRollupResultCache(t *testing.T) {
t.Errorf("unexpected response (-want, +got):\n%s", diff)
}
want = apptest.NewAPIV1QueryResponse(t,
want = apptest.NewPrometheusAPIV1QueryResponse(t,
`{"data":
{"result":[]}
}`,
)
got = vmselect.APIV1QueryRange(t, `foo_bar{}`, apptest.QueryOpts{
got = vmselect.PrometheusAPIV1QueryRange(t, `foo_bar{}`, apptest.QueryOpts{
Tenant: "multitenant",
Start: "2022-05-10T07:59:00.000Z",
End: "2022-05-10T08:05:00.000Z",

View File

@@ -38,7 +38,7 @@ func TestClusterVminsertShardsDataVmselectBuildsFullResultFromShards(t *testing.
const numMetrics = 1000
records := make([]string, numMetrics)
want := &apptest.APIV1SeriesResponse{
want := &apptest.PrometheusAPIV1SeriesResponse{
Status: "success",
IsPartial: false,
Data: make([]map[string]string, numMetrics),
@@ -49,7 +49,7 @@ func TestClusterVminsertShardsDataVmselectBuildsFullResultFromShards(t *testing.
want.Data[i] = map[string]string{"__name__": name}
}
want.Sort()
vminsert.APIV1ImportPrometheus(t, records, apptest.QueryOpts{})
vminsert.PrometheusAPIV1ImportPrometheus(t, records, apptest.QueryOpts{})
vmstorage1.ForceFlush(t)
vmstorage2.ForceFlush(t)
@@ -74,7 +74,7 @@ func TestClusterVminsertShardsDataVmselectBuildsFullResultFromShards(t *testing.
tc.Assert(&apptest.AssertOptions{
Msg: "unexpected /api/v1/series response",
Got: func() any {
res := vmselect.APIV1Series(t, `{__name__=~".*"}`, apptest.QueryOpts{})
res := vmselect.PrometheusAPIV1Series(t, `{__name__=~".*"}`, apptest.QueryOpts{})
res.Sort()
return res
},

View File

@@ -28,7 +28,7 @@ func TestSingleSnapshots_CreateListDelete(t *testing.T) {
for i := range numSamples {
samples[i] = fmt.Sprintf("metric_%03d %d", i, i)
}
sut.APIV1ImportPrometheus(t, samples, at.QueryOpts{})
sut.PrometheusAPIV1ImportPrometheus(t, samples, at.QueryOpts{})
sut.ForceFlush(t)
// Create several snapshots using VictoriaMetrics and Prometheus endpoints.
@@ -113,7 +113,7 @@ func TestClusterSnapshots_CreateListDelete(t *testing.T) {
for i := range numSamples {
samples[i] = fmt.Sprintf("metric_%03d %d", i, i)
}
sut.APIV1ImportPrometheus(t, samples, at.QueryOpts{})
sut.PrometheusAPIV1ImportPrometheus(t, samples, at.QueryOpts{})
sut.ForceFlush(t)
// Create several snapshots for both vmstorage replicas using

View File

@@ -4,10 +4,9 @@ import (
"os"
"testing"
at "github.com/VictoriaMetrics/VictoriaMetrics/apptest"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
at "github.com/VictoriaMetrics/VictoriaMetrics/apptest"
)
// TestSingleSpecialQueryRegression is used to test queries that have experienced issues for specific data sets.
@@ -36,7 +35,7 @@ func TestClusterSpecialQueryRegression(t *testing.T) {
testSpecialQueryRegression(tc, sut)
}
func testSpecialQueryRegression(tc *at.TestCase, sut at.WriteQuerier) {
func testSpecialQueryRegression(tc *at.TestCase, sut at.PrometheusWriteQuerier) {
// prometheus
testCaseSensitiveRegex(tc, sut)
testDuplicateLabel(tc, sut)
@@ -51,12 +50,12 @@ func testSpecialQueryRegression(tc *at.TestCase, sut at.WriteQuerier) {
testSubqueryAggregation(tc, sut)
}
func testCaseSensitiveRegex(tc *at.TestCase, sut at.WriteQuerier) {
func testCaseSensitiveRegex(tc *at.TestCase, sut at.PrometheusWriteQuerier) {
t := tc.T()
// case-sensitive-regex
// https://github.com/VictoriaMetrics/VictoriaMetrics/issues/161
sut.APIV1ImportPrometheus(t, []string{
sut.PrometheusAPIV1ImportPrometheus(t, []string{
`prometheus.sensitiveRegex{label="sensitiveRegex"} 10 1707123456700`, // 2024-02-05T08:57:36.700Z
`prometheus.sensitiveRegex{label="SensitiveRegex"} 10 1707123456700`, // 2024-02-05T08:57:36.700Z
}, at.QueryOpts{})
@@ -65,12 +64,12 @@ func testCaseSensitiveRegex(tc *at.TestCase, sut at.WriteQuerier) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/export response",
Got: func() any {
return sut.APIV1Export(t, `{label=~'(?i)sensitiveregex'}`, at.QueryOpts{
return sut.PrometheusAPIV1Export(t, `{label=~'(?i)sensitiveregex'}`, at.QueryOpts{
Start: "2024-02-05T08:50:00.700Z",
End: "2024-02-05T09:00:00.700Z",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -87,17 +86,17 @@ func testCaseSensitiveRegex(tc *at.TestCase, sut at.WriteQuerier) {
},
},
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(at.APIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.IgnoreFields(at.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
},
})
}
func testDuplicateLabel(tc *at.TestCase, sut at.WriteQuerier) {
func testDuplicateLabel(tc *at.TestCase, sut at.PrometheusWriteQuerier) {
t := tc.T()
// duplicate_label
// https://github.com/VictoriaMetrics/VictoriaMetrics/issues/172
sut.APIV1ImportPrometheus(t, []string{
sut.PrometheusAPIV1ImportPrometheus(t, []string{
`prometheus.duplicate_label{label="duplicate", label="duplicate"} 10 1707123456700`, // 2024-02-05T08:57:36.700Z
}, at.QueryOpts{})
sut.ForceFlush(t)
@@ -105,12 +104,12 @@ func testDuplicateLabel(tc *at.TestCase, sut at.WriteQuerier) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/export response",
Got: func() any {
return sut.APIV1Export(t, `{__name__='prometheus.duplicate_label'}`, at.QueryOpts{
return sut.PrometheusAPIV1Export(t, `{__name__='prometheus.duplicate_label'}`, at.QueryOpts{
Start: "2024-02-05T08:50:00.700Z",
End: "2024-02-05T09:00:00.700Z",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -123,17 +122,17 @@ func testDuplicateLabel(tc *at.TestCase, sut at.WriteQuerier) {
},
},
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(at.APIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.IgnoreFields(at.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
},
})
}
func testTooBigLookbehindWindow(tc *at.TestCase, sut at.WriteQuerier) {
func testTooBigLookbehindWindow(tc *at.TestCase, sut at.PrometheusWriteQuerier) {
t := tc.T()
// too big look-behind window
// https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5553
sut.APIV1ImportPrometheus(t, []string{
sut.PrometheusAPIV1ImportPrometheus(t, []string{
`prometheus.too_big_lookbehind{label="foo"} 10 1707123456700`, // 2024-02-05T08:57:36.700Z
}, at.QueryOpts{})
sut.ForceFlush(t)
@@ -141,12 +140,12 @@ func testTooBigLookbehindWindow(tc *at.TestCase, sut at.WriteQuerier) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query response",
Got: func() any {
return sut.APIV1Query(t, `prometheus.too_big_lookbehind{label="foo"}[100y]`, at.QueryOpts{
return sut.PrometheusAPIV1Query(t, `prometheus.too_big_lookbehind{label="foo"}[100y]`, at.QueryOpts{
Step: "5m",
Time: "2024-02-05T08:57:36.700Z",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -164,7 +163,7 @@ func testTooBigLookbehindWindow(tc *at.TestCase, sut at.WriteQuerier) {
// too big look-behind window - query range
// https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5553
sut.APIV1ImportPrometheus(t, []string{
sut.PrometheusAPIV1ImportPrometheus(t, []string{
`prometheus.too_big_lookbehind_range{label="foo"} 13 1707123496700`, // 2024-02-05T08:58:16.700Z
`prometheus.too_big_lookbehind_range{label="foo"} 12 1707123466700`, // 2024-02-05T08:57:46.700Z
`prometheus.too_big_lookbehind_range{label="foo"} 11 1707123436700`, // 2024-02-05T08:57:16.700Z
@@ -175,13 +174,13 @@ func testTooBigLookbehindWindow(tc *at.TestCase, sut at.WriteQuerier) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query_range response",
Got: func() any {
return sut.APIV1QueryRange(t, `prometheus.too_big_lookbehind_range{label="foo"}`, at.QueryOpts{
return sut.PrometheusAPIV1QueryRange(t, `prometheus.too_big_lookbehind_range{label="foo"}`, at.QueryOpts{
Start: "2024-02-05T08:56:46.700Z",
End: "2024-02-05T08:58:16.700Z",
Step: "30s",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -201,12 +200,12 @@ func testTooBigLookbehindWindow(tc *at.TestCase, sut at.WriteQuerier) {
})
}
func testMatchSeries(tc *at.TestCase, sut at.WriteQuerier) {
func testMatchSeries(tc *at.TestCase, sut at.PrometheusWriteQuerier) {
t := tc.T()
// match_series
// https://github.com/VictoriaMetrics/VictoriaMetrics/issues/155
sut.APIV1ImportPrometheus(t, []string{
sut.PrometheusAPIV1ImportPrometheus(t, []string{
`GenBearTemp{db="TenMinute",Park="1",TurbineType="V112"} 10 1707123456700`, // 2024-02-05T08:57:36.700Z
`GenBearTemp{db="TenMinute",Park="2",TurbineType="V112"} 10 1707123456700`, // 2024-02-05T08:57:36.700Z
`GenBearTemp{db="TenMinute",Park="3",TurbineType="V112"} 10 1707123456700`, // 2024-02-05T08:57:36.700Z
@@ -217,12 +216,12 @@ func testMatchSeries(tc *at.TestCase, sut at.WriteQuerier) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/series response",
Got: func() any {
return sut.APIV1Series(t, `{__name__="GenBearTemp"}`, at.QueryOpts{
return sut.PrometheusAPIV1Series(t, `{__name__="GenBearTemp"}`, at.QueryOpts{
Start: "2024-02-04T08:57:36.700Z",
End: "2024-02-05T08:57:36.700Z",
}).Sort()
},
Want: &at.APIV1SeriesResponse{
Want: &at.PrometheusAPIV1SeriesResponse{
Status: "success",
IsPartial: false,
Data: []map[string]string{
@@ -235,7 +234,7 @@ func testMatchSeries(tc *at.TestCase, sut at.WriteQuerier) {
})
}
func testComparisonNotInfNotNan(tc *at.TestCase, sut at.WriteQuerier) {
func testComparisonNotInfNotNan(tc *at.TestCase, sut at.PrometheusWriteQuerier) {
t := tc.T()
// comparison-not-inf-not-nan
@@ -259,13 +258,13 @@ func testComparisonNotInfNotNan(tc *at.TestCase, sut at.WriteQuerier) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query_range response",
Got: func() any {
return sut.APIV1QueryRange(t, `1/(not_nan_not_inf-1)!=inf!=nan`, at.QueryOpts{
return sut.PrometheusAPIV1QueryRange(t, `1/(not_nan_not_inf-1)!=inf!=nan`, at.QueryOpts{
Start: "2024-02-05T06:50:36.000Z",
End: "2024-02-05T09:58:37.000Z",
Step: "60",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -282,7 +281,7 @@ func testComparisonNotInfNotNan(tc *at.TestCase, sut at.WriteQuerier) {
})
}
func testEmptyLabelMatch(tc *at.TestCase, sut at.WriteQuerier) {
func testEmptyLabelMatch(tc *at.TestCase, sut at.PrometheusWriteQuerier) {
t := tc.T()
// empty-label-match
@@ -305,13 +304,13 @@ func testEmptyLabelMatch(tc *at.TestCase, sut at.WriteQuerier) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query_range response",
Got: func() any {
return sut.APIV1QueryRange(t, `empty_label_match{foo=~'bar|'}`, at.QueryOpts{
return sut.PrometheusAPIV1QueryRange(t, `empty_label_match{foo=~'bar|'}`, at.QueryOpts{
Start: "2024-02-05T08:55:36.000Z",
End: "2024-02-05T08:57:36.000Z",
Step: "60s",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -334,7 +333,7 @@ func testEmptyLabelMatch(tc *at.TestCase, sut at.WriteQuerier) {
})
}
func testMaxLookbehind(tc *at.TestCase, sut at.WriteQuerier) {
func testMaxLookbehind(tc *at.TestCase, sut at.PrometheusWriteQuerier) {
t := tc.T()
// max_lookback_set
@@ -358,14 +357,14 @@ func testMaxLookbehind(tc *at.TestCase, sut at.WriteQuerier) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query_range response",
Got: func() any {
return sut.APIV1QueryRange(t, `max_lookback_set{foo=~'bar|'}`, at.QueryOpts{
return sut.PrometheusAPIV1QueryRange(t, `max_lookback_set{foo=~'bar|'}`, at.QueryOpts{
Start: "2024-02-05T08:55:06.000Z",
End: "2024-02-05T08:57:37.000Z",
Step: "10s",
MaxLookback: "1s",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -405,13 +404,13 @@ func testMaxLookbehind(tc *at.TestCase, sut at.WriteQuerier) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query_range response",
Got: func() any {
return sut.APIV1QueryRange(t, `max_lookback_unset{foo=~'bar|'}`, at.QueryOpts{
return sut.PrometheusAPIV1QueryRange(t, `max_lookback_unset{foo=~'bar|'}`, at.QueryOpts{
Start: "2024-02-05T08:55:06.000Z",
End: "2024-02-05T08:57:37.000Z",
Step: "10s",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -442,7 +441,7 @@ func testMaxLookbehind(tc *at.TestCase, sut at.WriteQuerier) {
})
}
func testNonNanAsMissingData(tc *at.TestCase, sut at.WriteQuerier) {
func testNonNanAsMissingData(tc *at.TestCase, sut at.PrometheusWriteQuerier) {
t := tc.T()
// not-nan-as-missing-data
@@ -466,13 +465,13 @@ func testNonNanAsMissingData(tc *at.TestCase, sut at.WriteQuerier) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query_range response",
Got: func() any {
return sut.APIV1QueryRange(t, `not_nan_as_missing_data>1`, at.QueryOpts{
return sut.PrometheusAPIV1QueryRange(t, `not_nan_as_missing_data>1`, at.QueryOpts{
Start: "2024-02-05T08:57:34.000Z",
End: "2024-02-05T08:57:36.000Z",
Step: "1s",
})
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "matrix",
@@ -497,7 +496,7 @@ func testNonNanAsMissingData(tc *at.TestCase, sut at.WriteQuerier) {
})
}
func testSubqueryAggregation(tc *at.TestCase, sut at.WriteQuerier) {
func testSubqueryAggregation(tc *at.TestCase, sut at.PrometheusWriteQuerier) {
t := tc.T()
// subquery-aggregation
@@ -521,14 +520,14 @@ func testSubqueryAggregation(tc *at.TestCase, sut at.WriteQuerier) {
tc.Assert(&at.AssertOptions{
Msg: "unexpected /api/v1/query response",
Got: func() any {
got := sut.APIV1Query(t, `min by (item) (min_over_time(forms_daily_count[10m:1m]))`, at.QueryOpts{
got := sut.PrometheusAPIV1Query(t, `min by (item) (min_over_time(forms_daily_count[10m:1m]))`, at.QueryOpts{
Time: "2024-02-05T08:56:35.000Z",
LatencyOffset: "1ms",
})
got.Sort()
return got
},
Want: &at.APIV1QueryResponse{
Want: &at.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &at.QueryData{
ResultType: "vector",
@@ -547,7 +546,7 @@ func testSubqueryAggregation(tc *at.TestCase, sut at.WriteQuerier) {
})
}
func getRowsInsertedTotal(t *testing.T, sut at.WriteQuerier) int {
func getRowsInsertedTotal(t *testing.T, sut at.PrometheusWriteQuerier) int {
t.Helper()
selector := `vm_rows_inserted_total{type="graphite"}`

View File

@@ -45,12 +45,12 @@ func TestSingleVMAgentReloadConfigs(t *testing.T) {
tc.Assert(&at.AssertOptions{
Msg: `unexpected metrics stored on vmagent remote write`,
Got: func() any {
return vmsingle.APIV1Series(t, `{__name__="foo_bar"}`, at.QueryOpts{
return vmsingle.PrometheusAPIV1Series(t, `{__name__="foo_bar"}`, at.QueryOpts{
Start: "2022-05-10T00:00:00Z",
End: "2022-05-10T23:59:59Z",
}).Sort()
},
Want: &at.APIV1SeriesResponse{
Want: &at.PrometheusAPIV1SeriesResponse{
Status: "success",
Data: []map[string]string{{"__name__": "foo_bar", "label1": "value1"}},
},
@@ -76,12 +76,12 @@ func TestSingleVMAgentReloadConfigs(t *testing.T) {
tc.Assert(&at.AssertOptions{
Msg: `unexpected metrics stored on vmagent remote write`,
Got: func() any {
return vmsingle.APIV1Series(t, `{__name__="bar_foo"}`, at.QueryOpts{
return vmsingle.PrometheusAPIV1Series(t, `{__name__="bar_foo"}`, at.QueryOpts{
Start: "2022-05-10T00:00:00Z",
End: "2022-05-10T23:59:59Z",
}).Sort()
},
Want: &at.APIV1SeriesResponse{
Want: &at.PrometheusAPIV1SeriesResponse{
Status: "success",
Data: []map[string]string{{"__name__": "bar_foo", "label1": "value2"}},
},
@@ -122,12 +122,12 @@ func testSingleVMAgentRemoteWrite(t *testing.T, forcePromProto bool) {
tc.Assert(&at.AssertOptions{
Msg: `unexpected metrics stored on vmagent remote write`,
Got: func() any {
return vmsingle.APIV1Series(t, `{__name__="foo_bar"}`, at.QueryOpts{
return vmsingle.PrometheusAPIV1Series(t, `{__name__="foo_bar"}`, at.QueryOpts{
Start: "2022-05-10T00:00:00Z",
End: "2022-05-10T23:59:59Z",
}).Sort()
},
Want: &at.APIV1SeriesResponse{
Want: &at.PrometheusAPIV1SeriesResponse{
Status: "success",
Data: []map[string]string{{"__name__": "foo_bar"}},
},

View File

@@ -77,19 +77,19 @@ func TestClusterTenantsToTenantsVmctlNativeProtocol(t *testing.T) {
testVmctlNativeProtocol(tc, clusterSrc, clusterDst, flags)
}
func testVmctlNativeProtocol(tc *apptest.TestCase, srcSut apptest.WriteQuerier, dstSut apptest.WriteQuerier, vmctlFlags []string) {
func testVmctlNativeProtocol(tc *apptest.TestCase, srcSut apptest.PrometheusWriteQuerier, dstSut apptest.PrometheusWriteQuerier, vmctlFlags []string) {
t := tc.T()
t.Helper()
cmpOpt := cmpopts.IgnoreFields(apptest.APIV1QueryResponse{}, "Status", "Data.ResultType")
cmpOpt := cmpopts.IgnoreFields(apptest.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType")
// test for empty data request in the source
got := srcSut.APIV1Query(t, `{__name__=~".*"}`, apptest.QueryOpts{
got := srcSut.PrometheusAPIV1Query(t, `{__name__=~".*"}`, apptest.QueryOpts{
Step: "5m",
Time: "2025-05-30T12:45:00Z",
})
want := apptest.NewAPIV1QueryResponse(t, `{"data":{"result":[]}}`)
want := apptest.NewPrometheusAPIV1QueryResponse(t, `{"data":{"result":[]}}`)
if diff := cmp.Diff(want, got, cmpOpt); diff != "" {
t.Errorf("unexpected response (-want, +got):\n%s", diff)
}
@@ -116,14 +116,14 @@ func testVmctlNativeProtocol(tc *apptest.TestCase, srcSut apptest.WriteQuerier,
dataSet[i] = fmt.Sprintf("%s %d %s", metricsName, i, ingestTimestamp)
}
wantResponse := apptest.APIV1QueryResponse{
wantResponse := apptest.PrometheusAPIV1QueryResponse{
Status: "success",
Data: &expectedQueryData,
}
wantResponse.Sort()
srcSut.APIV1ImportPrometheus(t, dataSet, apptest.QueryOpts{})
srcSut.PrometheusAPIV1ImportPrometheus(t, dataSet, apptest.QueryOpts{})
srcSut.ForceFlush(t)
tc.MustStartVmctl("vmctl", vmctlFlags)
@@ -134,7 +134,7 @@ func testVmctlNativeProtocol(tc *apptest.TestCase, srcSut apptest.WriteQuerier,
Retries: 300,
Msg: `unexpected metrics stored on vmsingle via the native protocol`,
Got: func() any {
exported := dstSut.APIV1Export(t, `{__name__=~".*"}`, apptest.QueryOpts{
exported := dstSut.PrometheusAPIV1Export(t, `{__name__=~".*"}`, apptest.QueryOpts{
Start: "2025-05-30T16:39:36Z",
End: "2025-05-30T16:39:37Z",
})
@@ -143,7 +143,7 @@ func testVmctlNativeProtocol(tc *apptest.TestCase, srcSut apptest.WriteQuerier,
},
Want: wantResponse.Data.Result,
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(apptest.APIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.IgnoreFields(apptest.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
},
})
}

View File

@@ -55,19 +55,19 @@ func TestClusterVmctlPrometheusProtocol(t *testing.T) {
testPrometheusProtocol(tc, cluster, vmctlFlags)
}
func testPrometheusProtocol(tc *apptest.TestCase, sut apptest.WriteQuerier, vmctlFlags []string) {
func testPrometheusProtocol(tc *apptest.TestCase, sut apptest.PrometheusWriteQuerier, vmctlFlags []string) {
t := tc.T()
t.Helper()
cmpOpt := cmpopts.IgnoreFields(apptest.APIV1QueryResponse{}, "Status", "Data.ResultType")
cmpOpt := cmpopts.IgnoreFields(apptest.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType")
// test for empty data request
got := sut.APIV1Query(t, `{__name__=~".*"}`, apptest.QueryOpts{
got := sut.PrometheusAPIV1Query(t, `{__name__=~".*"}`, apptest.QueryOpts{
Step: "5m",
Time: "2025-06-02T17:14:00Z",
})
want := apptest.NewAPIV1QueryResponse(t, `{"data":{"result":[]}}`)
want := apptest.NewPrometheusAPIV1QueryResponse(t, `{"data":{"result":[]}}`)
if diff := cmp.Diff(want, got, cmpOpt); diff != "" {
t.Errorf("unexpected response (-want, +got):\n%s", diff)
}
@@ -88,7 +88,7 @@ func testPrometheusProtocol(tc *apptest.TestCase, sut apptest.WriteQuerier, vmct
t.Fatalf("cannot read expected series response file: %s", err)
}
var wantResponse apptest.APIV1QueryResponse
var wantResponse apptest.PrometheusAPIV1QueryResponse
if err := json.Unmarshal(bytes, &wantResponse); err != nil {
t.Fatalf("cannot unmarshal expected series response file: %s", err)
}
@@ -99,7 +99,7 @@ func testPrometheusProtocol(tc *apptest.TestCase, sut apptest.WriteQuerier, vmct
Retries: 300,
Msg: `unexpected metrics stored on vmsingle via the prometheus protocol`,
Got: func() any {
expected := sut.APIV1Export(t, `{__name__="vm_log_messages_total", location=~"VictoriaMetrics/lib/ingestserver/opentsdb/server.go:(48|59)"}`, apptest.QueryOpts{
expected := sut.PrometheusAPIV1Export(t, `{__name__="vm_log_messages_total", location=~"VictoriaMetrics/lib/ingestserver/opentsdb/server.go:(48|59)"}`, apptest.QueryOpts{
Start: "2025-06-02T00:00:00Z",
End: "2025-06-02T23:59:59Z",
})
@@ -108,7 +108,7 @@ func testPrometheusProtocol(tc *apptest.TestCase, sut apptest.WriteQuerier, vmct
},
Want: wantResponse.Data.Result,
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(apptest.APIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.IgnoreFields(apptest.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
},
})
}

View File

@@ -75,7 +75,7 @@ func TestClusterVmctlRemoteReadProtocol(t *testing.T) {
testRemoteReadProtocol(tc, clusterDst, newRemoteReadServer, vmctlFlags)
}
func testRemoteReadProtocol(tc *at.TestCase, sut at.WriteQuerier, newRemoteReadServer func(t *testing.T) *RemoteReadServer, vmctlFlags []string) {
func testRemoteReadProtocol(tc *at.TestCase, sut at.PrometheusWriteQuerier, newRemoteReadServer func(t *testing.T) *RemoteReadServer, vmctlFlags []string) {
t := tc.T()
t.Helper()
@@ -84,14 +84,14 @@ func testRemoteReadProtocol(tc *at.TestCase, sut at.WriteQuerier, newRemoteReadS
expectedResult := transformSeriesToQueryResult(rrs.storage.store)
cmpOpt := cmpopts.IgnoreFields(at.APIV1QueryResponse{}, "Status", "Data.ResultType")
cmpOpt := cmpopts.IgnoreFields(at.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType")
// test for empty data request
got := sut.APIV1Query(t, `{__name__=~".*"}`, at.QueryOpts{
got := sut.PrometheusAPIV1Query(t, `{__name__=~".*"}`, at.QueryOpts{
Step: "5m",
Time: "2025-06-02T17:14:00Z",
})
want := at.NewAPIV1QueryResponse(t, `{"data":{"result":[]}}`)
want := at.NewPrometheusAPIV1QueryResponse(t, `{"data":{"result":[]}}`)
if diff := cmp.Diff(want, got, cmpOpt); diff != "" {
t.Errorf("unexpected response (-want, +got):\n%s", diff)
}
@@ -106,7 +106,7 @@ func testRemoteReadProtocol(tc *at.TestCase, sut at.WriteQuerier, newRemoteReadS
Retries: 300,
Msg: `unexpected metrics stored on vmsingle via the prometheus protocol`,
Got: func() any {
expected := sut.APIV1Export(t, `{__name__=~".*"}`, at.QueryOpts{
expected := sut.PrometheusAPIV1Export(t, `{__name__=~".*"}`, at.QueryOpts{
Start: "2025-06-11T15:31:10Z",
End: "2025-06-11T15:32:20Z",
})
@@ -115,7 +115,7 @@ func testRemoteReadProtocol(tc *at.TestCase, sut at.WriteQuerier, newRemoteReadS
},
Want: expectedResult,
CmpOpts: []cmp.Option{
cmpopts.IgnoreFields(at.APIV1QueryResponse{}, "Status", "Data.ResultType"),
cmpopts.IgnoreFields(at.PrometheusAPIV1QueryResponse{}, "Status", "Data.ResultType"),
},
})
}

View File

@@ -127,12 +127,12 @@ func (app *Vminsert) GraphiteWrite(t *testing.T, records []string, _ QueryOpts)
app.cli.Write(t, app.graphiteListenAddr, records)
}
// APIV1ImportCSV is a test helper function that inserts a collection
// PrometheusAPIV1ImportCSV is a test helper function that inserts a collection
// of records in CSV format for the given tenant by sending an HTTP POST
// request to prometheus/api/v1/import/csv vminsert endpoint.
//
// See https://docs.victoriametrics.com/cluster-victoriametrics/#url-format
func (app *Vminsert) APIV1ImportCSV(t *testing.T, records []string, opts QueryOpts) {
func (app *Vminsert) PrometheusAPIV1ImportCSV(t *testing.T, records []string, opts QueryOpts) {
t.Helper()
url := fmt.Sprintf("http://%s/insert/%s/prometheus/api/v1/import/csv", app.httpListenAddr, opts.getTenant())
@@ -150,12 +150,12 @@ func (app *Vminsert) APIV1ImportCSV(t *testing.T, records []string, opts QueryOp
})
}
// APIV1ImportNative is a test helper function that inserts a collection
// PrometheusAPIV1ImportNative is a test helper function that inserts a collection
// of records in Native format for the given tenant by sending an HTTP POST
// request to prometheus/api/v1/import/native vminsert endpoint.
//
// See https://docs.victoriametrics.com/cluster-victoriametrics/#url-format
func (app *Vminsert) APIV1ImportNative(t *testing.T, data []byte, opts QueryOpts) {
func (app *Vminsert) PrometheusAPIV1ImportNative(t *testing.T, data []byte, opts QueryOpts) {
t.Helper()
url := fmt.Sprintf("http://%s/insert/%s/prometheus/api/v1/import/native", app.httpListenAddr, opts.getTenant())
@@ -195,10 +195,10 @@ func (app *Vminsert) OpenTSDBAPIPut(t *testing.T, records []string, opts QueryOp
})
}
// APIV1Write is a test helper function that inserts a
// PrometheusAPIV1Write is a test helper function that inserts a
// collection of records in Prometheus remote-write format by sending a HTTP
// POST request to /prometheus/api/v1/write vminsert endpoint.
func (app *Vminsert) APIV1Write(t *testing.T, records []pb.TimeSeries, opts QueryOpts) {
func (app *Vminsert) PrometheusAPIV1Write(t *testing.T, records []pb.TimeSeries, opts QueryOpts) {
t.Helper()
url := fmt.Sprintf("http://%s/insert/%s/prometheus/api/v1/write", app.httpListenAddr, opts.getTenant())
@@ -212,13 +212,13 @@ func (app *Vminsert) APIV1Write(t *testing.T, records []pb.TimeSeries, opts Quer
})
}
// APIV1ImportPrometheus is a test helper function that inserts a
// PrometheusAPIV1ImportPrometheus is a test helper function that inserts a
// collection of records in Prometheus text exposition format for the given
// tenant by sending a HTTP POST request to
// /prometheus/api/v1/import/prometheus vminsert endpoint.
//
// See https://docs.victoriametrics.com/victoriametrics/url-examples/#apiv1importprometheus
func (app *Vminsert) APIV1ImportPrometheus(t *testing.T, records []string, opts QueryOpts) {
func (app *Vminsert) PrometheusAPIV1ImportPrometheus(t *testing.T, records []string, opts QueryOpts) {
t.Helper()
url := fmt.Sprintf("http://%s/insert/%s/prometheus/api/v1/import/prometheus", app.httpListenAddr, opts.getTenant())

View File

@@ -61,12 +61,12 @@ func (app *Vmselect) HTTPAddr() string {
return app.httpListenAddr
}
// APIV1Export is a test helper function that performs the export of
// PrometheusAPIV1Export is a test helper function that performs the export of
// raw samples in JSON line format by sending a HTTP POST request to
// /prometheus/api/v1/export vmselect endpoint.
//
// See https://docs.victoriametrics.com/victoriametrics/url-examples/#apiv1export
func (app *Vmselect) APIV1Export(t *testing.T, query string, opts QueryOpts) *APIV1QueryResponse {
func (app *Vmselect) PrometheusAPIV1Export(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse {
t.Helper()
exportURL := fmt.Sprintf("http://%s/select/%s/prometheus/api/v1/export", app.httpListenAddr, opts.getTenant())
@@ -74,15 +74,15 @@ func (app *Vmselect) APIV1Export(t *testing.T, query string, opts QueryOpts) *AP
values.Add("match[]", query)
values.Add("format", "promapi")
res, _ := app.cli.PostForm(t, exportURL, values)
return NewAPIV1QueryResponse(t, res)
return NewPrometheusAPIV1QueryResponse(t, res)
}
// APIV1ExportNative is a test helper function that performs the export of
// PrometheusAPIV1ExportNative is a test helper function that performs the export of
// raw samples in native binary format by sending an HTTP POST request to
// /prometheus/api/v1/export/native vmselect endpoint.
//
// See https://docs.victoriametrics.com/victoriametrics/url-examples/#apiv1exportnative
func (app *Vmselect) APIV1ExportNative(t *testing.T, query string, opts QueryOpts) []byte {
func (app *Vmselect) PrometheusAPIV1ExportNative(t *testing.T, query string, opts QueryOpts) []byte {
t.Helper()
exportURL := fmt.Sprintf("http://%s/select/%s/prometheus/api/v1/export/native", app.httpListenAddr, opts.getTenant())
@@ -93,12 +93,12 @@ func (app *Vmselect) APIV1ExportNative(t *testing.T, query string, opts QueryOpt
return []byte(res)
}
// APIV1Query is a test helper function that performs PromQL/MetricsQL
// PrometheusAPIV1Query is a test helper function that performs PromQL/MetricsQL
// instant query by sending a HTTP POST request to /prometheus/api/v1/query
// vmselect endpoint.
//
// See https://docs.victoriametrics.com/victoriametrics/url-examples/#apiv1query
func (app *Vmselect) APIV1Query(t *testing.T, query string, opts QueryOpts) *APIV1QueryResponse {
func (app *Vmselect) PrometheusAPIV1Query(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse {
t.Helper()
queryURL := fmt.Sprintf("http://%s/select/%s/prometheus/api/v1/query", app.httpListenAddr, opts.getTenant())
@@ -106,15 +106,15 @@ func (app *Vmselect) APIV1Query(t *testing.T, query string, opts QueryOpts) *API
values.Add("query", query)
res, _ := app.cli.PostForm(t, queryURL, values)
return NewAPIV1QueryResponse(t, res)
return NewPrometheusAPIV1QueryResponse(t, res)
}
// APIV1QueryRange is a test helper function that performs
// PrometheusAPIV1QueryRange is a test helper function that performs
// PromQL/MetricsQL range query by sending a HTTP POST request to
// /prometheus/api/v1/query_range vmselect endpoint.
//
// See https://docs.victoriametrics.com/victoriametrics/url-examples/#apiv1query_range
func (app *Vmselect) APIV1QueryRange(t *testing.T, query string, opts QueryOpts) *APIV1QueryResponse {
func (app *Vmselect) PrometheusAPIV1QueryRange(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse {
t.Helper()
queryURL := fmt.Sprintf("http://%s/select/%s/prometheus/api/v1/query_range", app.httpListenAddr, opts.getTenant())
@@ -122,14 +122,14 @@ func (app *Vmselect) APIV1QueryRange(t *testing.T, query string, opts QueryOpts)
values.Add("query", query)
res, _ := app.cli.PostForm(t, queryURL, values)
return NewAPIV1QueryResponse(t, res)
return NewPrometheusAPIV1QueryResponse(t, res)
}
// APIV1Series sends a query to a /prometheus/api/v1/series endpoint
// PrometheusAPIV1Series sends a query to a /prometheus/api/v1/series endpoint
// and returns the list of time series that match the query.
//
// See https://docs.victoriametrics.com/victoriametrics/url-examples/#apiv1series
func (app *Vmselect) APIV1Series(t *testing.T, matchQuery string, opts QueryOpts) *APIV1SeriesResponse {
func (app *Vmselect) PrometheusAPIV1Series(t *testing.T, matchQuery string, opts QueryOpts) *PrometheusAPIV1SeriesResponse {
t.Helper()
seriesURL := fmt.Sprintf("http://%s/select/%s/prometheus/api/v1/series", app.httpListenAddr, opts.getTenant())
@@ -137,7 +137,7 @@ func (app *Vmselect) APIV1Series(t *testing.T, matchQuery string, opts QueryOpts
values.Add("match[]", matchQuery)
res, _ := app.cli.PostForm(t, seriesURL, values)
return NewAPIV1SeriesResponse(t, res)
return NewPrometheusAPIV1SeriesResponse(t, res)
}
// DeleteSeries sends a query to a /prometheus/api/v1/admin/tsdb/delete_series

View File

@@ -143,12 +143,12 @@ func (app *Vmsingle) GraphiteWrite(t *testing.T, records []string, _ QueryOpts)
app.cli.Write(t, app.graphiteWriteAddr, records)
}
// APIV1ImportCSV is a test helper function that inserts a collection
// PrometheusAPIV1ImportCSV is a test helper function that inserts a collection
// of records in CSV format for the given tenant by sending an HTTP POST
// request to /api/v1/import/csv vmsingle endpoint.
//
// See https://docs.victoriametrics.com/single-server-victoriametrics/#how-to-import-csv-data
func (app *Vmsingle) APIV1ImportCSV(t *testing.T, records []string, opts QueryOpts) {
func (app *Vmsingle) PrometheusAPIV1ImportCSV(t *testing.T, records []string, opts QueryOpts) {
t.Helper()
url := fmt.Sprintf("http://%s/api/v1/import/csv", app.httpListenAddr)
@@ -164,12 +164,12 @@ func (app *Vmsingle) APIV1ImportCSV(t *testing.T, records []string, opts QueryOp
}
}
// APIV1ImportNative is a test helper function that inserts a collection
// PrometheusAPIV1ImportNative is a test helper function that inserts a collection
// of records in native format for the given tenant by sending an HTTP POST
// request to /api/v1/import/native vmsingle endpoint.
//
// See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-import-data-in-native-format
func (app *Vmsingle) APIV1ImportNative(t *testing.T, data []byte, opts QueryOpts) {
func (app *Vmsingle) PrometheusAPIV1ImportNative(t *testing.T, data []byte, opts QueryOpts) {
t.Helper()
url := fmt.Sprintf("http://%s/api/v1/import/native", app.httpListenAddr)
@@ -206,10 +206,10 @@ func (app *Vmsingle) OpenTSDBAPIPut(t *testing.T, records []string, opts QueryOp
}
}
// APIV1Write is a test helper function that inserts a
// PrometheusAPIV1Write is a test helper function that inserts a
// collection of records in Prometheus remote-write format by sending a HTTP
// POST request to /prometheus/api/v1/write vmsingle endpoint.
func (app *Vmsingle) APIV1Write(t *testing.T, records []pb.TimeSeries, _ QueryOpts) {
func (app *Vmsingle) PrometheusAPIV1Write(t *testing.T, records []pb.TimeSeries, _ QueryOpts) {
t.Helper()
wr := pb.WriteRequest{Timeseries: records}
@@ -220,12 +220,12 @@ func (app *Vmsingle) APIV1Write(t *testing.T, records []pb.TimeSeries, _ QueryOp
}
}
// APIV1ImportPrometheus is a test helper function that inserts a
// PrometheusAPIV1ImportPrometheus is a test helper function that inserts a
// collection of records in Prometheus text exposition format by sending a HTTP
// POST request to /prometheus/api/v1/import/prometheus vmsingle endpoint.
//
// See https://docs.victoriametrics.com/victoriametrics/url-examples/#apiv1importprometheus
func (app *Vmsingle) APIV1ImportPrometheus(t *testing.T, records []string, opts QueryOpts) {
func (app *Vmsingle) PrometheusAPIV1ImportPrometheus(t *testing.T, records []string, opts QueryOpts) {
t.Helper()
// add extra label
@@ -243,27 +243,27 @@ func (app *Vmsingle) APIV1ImportPrometheus(t *testing.T, records []string, opts
}
}
// APIV1Export is a test helper function that performs the export of
// PrometheusAPIV1Export is a test helper function that performs the export of
// raw samples in JSON line format by sending a HTTP POST request to
// /prometheus/api/v1/export vmsingle endpoint.
//
// See https://docs.victoriametrics.com/victoriametrics/url-examples/#apiv1export
func (app *Vmsingle) APIV1Export(t *testing.T, query string, opts QueryOpts) *APIV1QueryResponse {
func (app *Vmsingle) PrometheusAPIV1Export(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse {
t.Helper()
values := opts.asURLValues()
values.Add("match[]", query)
values.Add("format", "promapi")
res, _ := app.cli.PostForm(t, app.prometheusAPIV1ExportURL, values)
return NewAPIV1QueryResponse(t, res)
return NewPrometheusAPIV1QueryResponse(t, res)
}
// APIV1ExportNative is a test helper function that performs the export of
// PrometheusAPIV1ExportNative is a test helper function that performs the export of
// raw samples in native binary format by sending an HTTP POST request to
// /prometheus/api/v1/export/native vmselect endpoint.
//
// See https://docs.victoriametrics.com/victoriametrics/url-examples/#apiv1exportnative
func (app *Vmsingle) APIV1ExportNative(t *testing.T, query string, opts QueryOpts) []byte {
func (app *Vmsingle) PrometheusAPIV1ExportNative(t *testing.T, query string, opts QueryOpts) []byte {
t.Helper()
t.Helper()
@@ -275,47 +275,47 @@ func (app *Vmsingle) APIV1ExportNative(t *testing.T, query string, opts QueryOpt
return []byte(res)
}
// APIV1Query is a test helper function that performs PromQL/MetricsQL
// PrometheusAPIV1Query is a test helper function that performs PromQL/MetricsQL
// instant query by sending a HTTP POST request to /prometheus/api/v1/query
// vmsingle endpoint.
//
// See https://docs.victoriametrics.com/victoriametrics/url-examples/#apiv1query
func (app *Vmsingle) APIV1Query(t *testing.T, query string, opts QueryOpts) *APIV1QueryResponse {
func (app *Vmsingle) PrometheusAPIV1Query(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse {
t.Helper()
values := opts.asURLValues()
values.Add("query", query)
res, _ := app.cli.PostForm(t, app.prometheusAPIV1QueryURL, values)
return NewAPIV1QueryResponse(t, res)
return NewPrometheusAPIV1QueryResponse(t, res)
}
// APIV1QueryRange is a test helper function that performs
// PrometheusAPIV1QueryRange is a test helper function that performs
// PromQL/MetricsQL range query by sending a HTTP POST request to
// /prometheus/api/v1/query_range vmsingle endpoint.
//
// See https://docs.victoriametrics.com/victoriametrics/url-examples/#apiv1query_range
func (app *Vmsingle) APIV1QueryRange(t *testing.T, query string, opts QueryOpts) *APIV1QueryResponse {
func (app *Vmsingle) PrometheusAPIV1QueryRange(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse {
t.Helper()
values := opts.asURLValues()
values.Add("query", query)
res, _ := app.cli.PostForm(t, app.prometheusAPIV1QueryRangeURL, values)
return NewAPIV1QueryResponse(t, res)
return NewPrometheusAPIV1QueryResponse(t, res)
}
// APIV1Series sends a query to a /prometheus/api/v1/series endpoint
// PrometheusAPIV1Series sends a query to a /prometheus/api/v1/series endpoint
// and returns the list of time series that match the query.
//
// See https://docs.victoriametrics.com/victoriametrics/url-examples/#apiv1series
func (app *Vmsingle) APIV1Series(t *testing.T, matchQuery string, opts QueryOpts) *APIV1SeriesResponse {
func (app *Vmsingle) PrometheusAPIV1Series(t *testing.T, matchQuery string, opts QueryOpts) *PrometheusAPIV1SeriesResponse {
t.Helper()
values := opts.asURLValues()
values.Add("match[]", matchQuery)
res, _ := app.cli.PostForm(t, app.prometheusAPIV1SeriesURL, values)
return NewAPIV1SeriesResponse(t, res)
return NewPrometheusAPIV1SeriesResponse(t, res)
}
// APIV1StatusMetricNamesStats sends a query to a /api/v1/status/metric_names_stats endpoint

View File

@@ -95,7 +95,7 @@ publish-via-docker:
--label "org.opencontainers.image.version=$(PKG_TAG)" \
--label "org.opencontainers.image.created=$(shell date -u +"%Y-%m-%dT%H:%M:%SZ")" \
$(foreach registry,$(DOCKER_REGISTRIES),\
--tag $(registry)/$(DOCKER_NAMESPACE)/$(APP_NAME):$(PKG_TAG)$(RACE) \
--tag $(registry)/$(DOCKER_NAMESPACE)/$(APP_NAME):$(PKG_TAG)$(RACE)$(EXTRA_TAG_SUFFIX) \
) \
-o type=image \
--provenance=false \
@@ -115,7 +115,7 @@ publish-via-docker:
--label "org.opencontainers.image.version=$(PKG_TAG)" \
--label "org.opencontainers.image.created=$(shell date -u +"%Y-%m-%dT%H:%M:%SZ")" \
$(foreach registry,$(DOCKER_REGISTRIES),\
--tag $(registry)/$(DOCKER_NAMESPACE)/$(APP_NAME):$(PKG_TAG)$(RACE)-scratch \
--tag $(registry)/$(DOCKER_NAMESPACE)/$(APP_NAME):$(PKG_TAG)$(RACE)$(EXTRA_TAG_SUFFIX)-scratch \
) \
-o type=image \
--provenance=false \

View File

@@ -36,30 +36,30 @@ services:
user: root
vlinsert:
image: victoriametrics/victoria-logs:v1.23.3-victorialogs
image: victoriametrics/victoria-logs:v1.24.0-victorialogs
command:
- "--storageNode=vlstorage-1:9428"
- "--storageNode=vlstorage-2:9428"
vlselect-1:
image: victoriametrics/victoria-logs:v1.23.3-victorialogs
image: victoriametrics/victoria-logs:v1.24.0-victorialogs
command:
- "--storageNode=vlstorage-1:9428"
- "--storageNode=vlstorage-2:9428"
vlselect-2:
image: victoriametrics/victoria-logs:v1.23.3-victorialogs
image: victoriametrics/victoria-logs:v1.24.0-victorialogs
command:
- "--storageNode=vlstorage-1:9428"
- "--storageNode=vlstorage-2:9428"
vlstorage-1:
image: victoriametrics/victoria-logs:v1.23.3-victorialogs
image: victoriametrics/victoria-logs:v1.24.0-victorialogs
command:
- "--storageDataPath=/vlogs"
volumes:
- vldata-1:/vlogs
vlstorage-2:
image: victoriametrics/victoria-logs:v1.23.3-victorialogs
image: victoriametrics/victoria-logs:v1.24.0-victorialogs
command:
- "--storageDataPath=/vlogs"
volumes:

View File

@@ -38,7 +38,7 @@ services:
# VictoriaLogs instance, a single process responsible for
# storing logs and serving read queries.
victorialogs:
image: victoriametrics/victoria-logs:v1.23.3-victorialogs
image: victoriametrics/victoria-logs:v1.24.0-victorialogs
ports:
- "9428:9428"
command:

View File

@@ -1,7 +1,7 @@
services:
# meta service will be ignored by compose
.victorialogs:
image: docker.io/victoriametrics/victoria-logs:v1.23.3-victorialogs
image: docker.io/victoriametrics/victoria-logs:v1.24.0-victorialogs
command:
- -storageDataPath=/vlogs
- -loggerFormat=json

View File

@@ -59,7 +59,7 @@ services:
- '--external.alert.source=explore?orgId=1&left=["now-1h","now","VictoriaMetrics",{"expr": },{"mode":"Metrics"},{"ui":[true,true,true,"none"]}]'
restart: always
vmanomaly:
image: victoriametrics/vmanomaly:v1.24.0
image: victoriametrics/vmanomaly:v1.24.1
depends_on:
- "victoriametrics"
ports:

View File

@@ -3,7 +3,7 @@ version: "3"
services:
# Run `make package-victoria-logs` to build victoria-logs image
vlogs:
image: docker.io/victoriametrics/victoria-logs:v1.23.3-victorialogs
image: docker.io/victoriametrics/victoria-logs:v1.24.0-victorialogs
volumes:
- vlogs:/vlogs
ports:

View File

@@ -14,6 +14,16 @@ aliases:
---
Please find the changelog for VictoriaMetrics Anomaly Detection below.
## v1.24.1
Released: 2025-06-20
- BUGFIX: Resolved the issue first seen in [v1.23.0](#v1230) where some fit and infer jobs were silently skipped at task submission time (due to a bug in the new background scheduler behind [`PeriodicScheduler`](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/#periodic-scheduler)) followed by similar warnings in the logs later on, such as:
```shellhelp
2025-06-19 14:32:50,568 - apscheduler.executors.default - WARNING - Run time of job "{job_name}" (trigger: interval[1 day, 0:00:00], next run at: 2025-06-20 14:32:50 UTC)" was missed by 0:00:01.024753
```
- BUGFIX: Resolved the issue where `vmanomaly` service on [`PeriodicScheduler`](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/#periodic-scheduler) where `start_from` argument was set and [state restoration](https://docs.victoriametrics.com/anomaly-detection/components/settings/#state-restoration) was enabled, didn't resume infer jobs after respective fitted models were restored from the previous run. This could lead to a situation where the service, *if restore happened in-between fit calls*, would not produce any anomaly scores and stay idle until the next `fit_every` happens, which is *expected in stateless mode*, but not in *stateful* mode with `restore_state` enabled.
## v1.24.0
Released: 2025-06-18
@@ -45,6 +55,13 @@ Released: 2025-06-08
## v1.23.0
Released: 2025-06-05
> There is a known bug that can cause some fit and infer jobs to be silently skipped at task submission time (due to a bug in the new background scheduler behind [`PeriodicScheduler`](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/#periodic-scheduler)) followed by similar warnings in the logs later on, such as:
> ```shellhelp
> 2025-06-19 14:32:50,568 - apscheduler.executors.default - WARNING - Run time of job "{job_name}" (trigger: interval[1 day, 0:00:00], next run at: 2025-06-20 14:32:50 UTC)" was missed by 0:00:01.024753
> ```
> Releases affected: [v1.23.0](#v1230) - [v1.23.3](#v1233).
> **The issue has been resolved in patch [v1.24.1](#v1241), upgrade is recommended.**
- FEATURE: Added `decay` [argument](https://docs.victoriametrics.com/anomaly-detection/components/models/#decay) to [online models](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-models). This parameters allows for newer data to be weighted more heavily in online models. By default this is set to 1 which means all data points are weighted the same to maintain backward compatibility with existing configs. The closer this value is to 0 the more important new data is.
- IMPROVEMENT: **Restored back parallelization** in the read/fit/infer pipeline, previously disabled in [v1.22.0](#v1220-experimental) due to deadlock issues. The new implementation prevents deadlocks, allowing to control the parallelization level via `n_workers` in [settings section](https://docs.victoriametrics.com/anomaly-detection/components/settings/). It's suggested to upgrade from [v1.22.0](#v1220) - [v1.22.1](#v1221) to this version to regain the performance benefits of parallel processing.

View File

@@ -230,7 +230,7 @@ services:
# ...
vmanomaly:
container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.24.0
image: victoriametrics/vmanomaly:v1.24.1
# ...
ports:
- "8490:8490"
@@ -443,7 +443,7 @@ options:
Heres an example of using the config splitter to divide configurations based on the `extra_filters` argument from the reader section:
```sh
docker pull victoriametrics/vmanomaly:v1.24.0 && docker image tag victoriametrics/vmanomaly:v1.24.0 vmanomaly
docker pull victoriametrics/vmanomaly:v1.24.1 && docker image tag victoriametrics/vmanomaly:v1.24.1 vmanomaly
```
```sh

View File

@@ -120,13 +120,13 @@ Below are the steps to get `vmanomaly` up and running inside a Docker container:
1. Pull Docker image:
```sh
docker pull victoriametrics/vmanomaly:v1.24.0
docker pull victoriametrics/vmanomaly:v1.24.1
```
2. (Optional step) tag the `vmanomaly` Docker image:
```sh
docker image tag victoriametrics/vmanomaly:v1.24.0 vmanomaly
docker image tag victoriametrics/vmanomaly:v1.24.1 vmanomaly
```
3. Start the `vmanomaly` Docker container with a *license file*, use the command below.
@@ -160,7 +160,7 @@ docker run -it --user 1000:1000 \
services:
# ...
vmanomaly:
image: victoriametrics/vmanomaly:v1.24.0
image: victoriametrics/vmanomaly:v1.24.1
volumes:
$YOUR_LICENSE_FILE_PATH:/license
$YOUR_CONFIG_FILE_PATH:/config.yml

View File

@@ -1278,7 +1278,7 @@ monitoring:
Let's pull the docker image for `vmanomaly`:
```sh
docker pull victoriametrics/vmanomaly:v1.24.0
docker pull victoriametrics/vmanomaly:v1.24.1
```
Now we can run the docker container putting as volumes both config and model file:
@@ -1292,7 +1292,7 @@ docker run -it \
-v $(PWD)/license:/license \
-v $(PWD)/custom_model.py:/vmanomaly/model/custom.py \
-v $(PWD)/custom.yaml:/config.yaml \
victoriametrics/vmanomaly:v1.24.0 /config.yaml \
victoriametrics/vmanomaly:v1.24.1 /config.yaml \
--licenseFile=/license
```

View File

@@ -387,7 +387,7 @@ services:
restart: always
vmanomaly:
container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.24.0
image: victoriametrics/vmanomaly:v1.24.1
depends_on:
- "victoriametrics"
ports:

View File

@@ -18,6 +18,11 @@ according to [these docs](https://docs.victoriametrics.com/victorialogs/quicksta
## tip
## [v1.24.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.24.0-victorialogs)
Released at 2025-06-20
* FEATURE: add `-http.disableKeepAlive` to disable HTTP keep-alives for incoming connections. The flag could improve load balancing among replicas behind HTTP load balancers. See [#9125](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9125) and [#2395](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2395) for details.
* FEATURE: [`delete` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#delete-pipe): allow deleting all the fields with common prefix via `... | delete prefix*` syntax.
* FEATURE: [`fields` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#fields-pipe): allow keeping all the fields with common prefix via `... | fields prefix*` syntax.
* FEATURE: [`copy` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#copy-pipe): allow copying all the fields with common prefix to fields with another common prefix via `... | copy old_prefix* as new_prefix*` syntax.
@@ -25,10 +30,10 @@ according to [these docs](https://docs.victoriametrics.com/victorialogs/quicksta
* FEATURE: [`unpack_json` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_json-pipe): allow unpacking JSON fields with common prefix via `... fields (prefix*)` syntax.
* FEATURE: [`unpack_logfmt` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_logfmt-pipe): allow unpacking JSON fields with common prefix via `... fields (prefix*)` syntax.
* FEATURE: [`avg` stats function](https://docs.victoriametrics.com/victorialogs/logsql/#avg-stats): allow calculating the average value over all the fields with common prefix via `avg(prefix*)` syntax.
* FEATURE: [`max` stats function](https://docs.victoriametrics.com/victorialogs/logsql/#avg-stats): allow calculating the maximum value over all the fields with common prefix via `max(prefix*)` syntax.
* FEATURE: [`min` stats function](https://docs.victoriametrics.com/victorialogs/logsql/#avg-stats): allow calculating the minimum value over all the fields with common prefix via `min(prefix*)` syntax.
* FEATURE: [`median` stats function](https://docs.victoriametrics.com/victorialogs/logsql/#avg-stats): allow calculating the median value over all the fields with common prefix via `median(prefix*)` syntax.
* FEATURE: [`quantile` stats function](https://docs.victoriametrics.com/victorialogs/logsql/#avg-stats): allow calculating the maximum value over all the fields with common prefix via `quantile(prefix*)` syntax.
* FEATURE: [`max` stats function](https://docs.victoriametrics.com/victorialogs/logsql/#max-stats): allow calculating the maximum value over all the fields with common prefix via `max(prefix*)` syntax.
* FEATURE: [`min` stats function](https://docs.victoriametrics.com/victorialogs/logsql/#min-stats): allow calculating the minimum value over all the fields with common prefix via `min(prefix*)` syntax.
* FEATURE: [`median` stats function](https://docs.victoriametrics.com/victorialogs/logsql/#median-stats): allow calculating the median value over all the fields with common prefix via `median(prefix*)` syntax.
* FEATURE: [`quantile` stats function](https://docs.victoriametrics.com/victorialogs/logsql/#quantile-stats): allow calculating the maximum value over all the fields with common prefix via `quantile(prefix*)` syntax.
* FEATURE: [`sum` stats function](https://docs.victoriametrics.com/victorialogs/logsql/#sum-stats): allow calculating the sum for all the fields with common prefix via `sum(prefix*)` syntax.
* FEATURE: [`sum_len` stats function](https://docs.victoriametrics.com/victorialogs/logsql/#sum_len-stats): allow calculating the sum of byte lengths for all the fields with common prefix via `sum_len(prefix*)` syntax.
* FEATURE: [`count` stats function](https://docs.victoriametrics.com/victorialogs/logsql/#count-stats): allow calculating the number of logs with at least a single non-empty field across fields with common prefix via `count(prefix*)` syntax.
@@ -53,6 +58,7 @@ according to [these docs](https://docs.victoriametrics.com/victorialogs/quicksta
* BUGFIX: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): live tailing tab automatically reconnects when the connection is lost. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9129).
* BUGFIX: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): fix issue with hits chart ignoring selected AccountID and ProjectID. See [#9157](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9157).
* BUGFIX: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): fix missing field values in auto-complete. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8749)
* BUGFIX: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): remove the compact mode of the table tab and add field sorting capabilities to the JSON tab. See [#7047](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7047).
* BUGFIX: [Journald data ingestion](https://docs.victoriametrics.com/victorialogs/data-ingestion/journald/): properly read log timestamp from `__REALTIME_TIMESTAMP` field according to [the docs](https://docs.victoriametrics.com/victorialogs/data-ingestion/journald/#time-field). See [#9144](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9144). The bug has been introduced in [v1.22.0-victorialogs](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.22.0-victorialogs).
* BUGFIX: [data ingestion](https://docs.victoriametrics.com/victorialogs/data-ingestion/): support `-` as a timestamp value, as described in [RFC5424](https://datatracker.ietf.org/doc/html/rfc5424#section-6.2.3).
* BUGFIX: [LogsQL](https://docs.victoriametrics.com/victorialogs/logsql/): properly handle quotes inside quoted strings such as `"\""`. Previously this could lead to panics. See [#9219](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9219).

View File

@@ -36,8 +36,8 @@ Just download archive for the needed Operating system and architecture, unpack i
For example, the following commands download VictoriaLogs archive for Linux/amd64, unpack and run it:
```sh
curl -L -O https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.23.3-victorialogs/victoria-logs-linux-amd64-v1.23.3-victorialogs.tar.gz
tar xzf victoria-logs-linux-amd64-v1.23.3-victorialogs.tar.gz
curl -L -O https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.24.0-victorialogs/victoria-logs-linux-amd64-v1.24.0-victorialogs.tar.gz
tar xzf victoria-logs-linux-amd64-v1.24.0-victorialogs.tar.gz
./victoria-logs-prod -storageDataPath=victoria-logs-data
```
@@ -61,7 +61,7 @@ Here is the command to run VictoriaLogs in a Docker container:
```sh
docker run --rm -it -p 9428:9428 -v ./victoria-logs-data:/victoria-logs-data \
docker.io/victoriametrics/victoria-logs:v1.23.3-victorialogs -storageDataPath=victoria-logs-data
docker.io/victoriametrics/victoria-logs:v1.24.0-victorialogs -storageDataPath=victoria-logs-data
```
See also:

View File

@@ -400,6 +400,8 @@ Pass `-help` to VictoriaLogs in order to see the list of supported command-line
Incoming connections to -httpListenAddr are closed after the configured timeout. This may help evenly spreading load among a cluster of services behind TCP-level load balancer. Zero value disables closing of incoming connections (default 2m0s)
-http.disableCORS
Disable CORS for all origins (*)
-http.disableKeepAlive
Whether to disable HTTP keep-alive for incoming connections at -httpListenAddr
-http.disableResponseCompression
Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
-http.header.csp string

View File

@@ -155,8 +155,8 @@ The following guide covers the following topics for Linux host:
Download and unpack the latest VictoriaLogs release:
```sh
curl -L -O https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.23.3-victorialogs/victoria-logs-linux-amd64-v1.23.3-victorialogs.tar.gz
tar xzf victoria-logs-linux-amd64-v1.23.3-victorialogs.tar.gz
curl -L -O https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.24.0-victorialogs/victoria-logs-linux-amd64-v1.24.0-victorialogs.tar.gz
tar xzf victoria-logs-linux-amd64-v1.24.0-victorialogs.tar.gz
```
Start the first [`vlstorage` node](#architecture), which accepts incoming requests at the port `9491` and stores the ingested logs at `victoria-logs-data-1` directory:

View File

@@ -36,7 +36,14 @@ VictoriaLogs uses `(_MACHINE_ID, _HOSTNAME, _SYSTEMD_UNIT)` as [stream fields](h
for logs ingested via jorunald protocol. The list of log stream fields can be changed via `-journald.streamFields` command-line flag if needed,
by providing comma-separated list of journald fields form [this list](https://www.freedesktop.org/software/systemd/man/latest/systemd.journal-fields.html).
See [the list of supported Journald fields](https://www.freedesktop.org/software/systemd/man/latest/systemd.journal-fields.html).
Please make sure that the log stream fields passed to `-journlad.streamFields` do not contain fields with high number or unbound number of unique values,
since this may lead to [high cardinality issues](https://docs.victoriametrics.com/victorialogs/keyconcepts/#high-cardinality).
The following Journald fields are also good candidates for stream fields:
- `_TRANSPORT`
- `_SYSTEMD_USER_UNIT`
## Dropping fields

View File

@@ -25,8 +25,8 @@ or from docker images at [Docker Hub](https://hub.docker.com/r/victoriametrics/v
### Running `vlogscli` from release binary
```sh
curl -L -O https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.23.3-victorialogs/vlogscli-linux-amd64-v1.23.3-victorialogs.tar.gz
tar xzf vlogscli-linux-amd64-v1.23.3-victorialogs.tar.gz
curl -L -O https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.24.0-victorialogs/vlogscli-linux-amd64-v1.24.0-victorialogs.tar.gz
tar xzf vlogscli-linux-amd64-v1.24.0-victorialogs.tar.gz
./vlogscli-prod
```
@@ -48,7 +48,6 @@ which queries `(AccountID=123, ProjectID=456)` [tenant](https://docs.victoriamet
./vlogscli -header='AccountID: 123' -header='ProjectID: 456'
```
## Multitenancy
`AccountID` and `ProjectID` [values](https://docs.victoriametrics.com/victorialogs/#multitenancy)

View File

@@ -1164,6 +1164,8 @@ Below is the output for `/path/to/vminsert -help`:
Incoming connections to -httpListenAddr are closed after the configured timeout. This may help evenly spreading load among a cluster of services behind TCP-level load balancer. Zero value disables closing of incoming connections (default 2m0s)
-http.disableCORS
Disable CORS for all origins (*)
-http.disableKeepAlive
Whether to disable HTTP keep-alive for incoming connections at -httpListenAddr
-http.disableResponseCompression
Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
-http.header.csp string
@@ -1470,6 +1472,8 @@ Below is the output for `/path/to/vmselect -help`:
Incoming connections to -httpListenAddr are closed after the configured timeout. This may help evenly spreading load among a cluster of services behind TCP-level load balancer. Zero value disables closing of incoming connections (default 2m0s)
-http.disableCORS
Disable CORS for all origins (*)
-http.disableKeepAlive
Whether to disable HTTP keep-alive for incoming connections at -httpListenAddr
-http.disableResponseCompression
Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
-http.header.csp string
@@ -1811,6 +1815,8 @@ Below is the output for `/path/to/vmstorage -help`:
Incoming connections to -httpListenAddr are closed after the configured timeout. This may help evenly spreading load among a cluster of services behind TCP-level load balancer. Zero value disables closing of incoming connections (default 2m0s)
-http.disableCORS
Disable CORS for all origins (*)
-http.disableKeepAlive
Whether to disable HTTP keep-alive for incoming connections at -httpListenAddr
-http.disableResponseCompression
Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
-http.header.csp string

View File

@@ -2459,6 +2459,8 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
Incoming connections to -httpListenAddr are closed after the configured timeout. This may help evenly spreading load among a cluster of services behind TCP-level load balancer. Zero value disables closing of incoming connections (default 2m0s)
-http.disableCORS
Disable CORS for all origins (*)
-http.disableKeepAlive
Whether to disable HTTP keep-alive for incoming connections at -httpListenAddr
-http.disableResponseCompression
Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
-http.header.csp string

View File

@@ -18,13 +18,21 @@ See also [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-rel
## tip
## [v1.120.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.120.0)
Released at 2025-06-20
* SECURITY: upgrade Go builder from Go1.24.3 to Go1.24.4. See [the list of issues addressed in Go1.24.4](https://github.com/golang/go/issues?q=milestone%3AGo1.24.4+label%3ACherryPickApproved).
* SECURITY: upgrade base docker image (Alpine) from 3.21.3 to 3.22.0. See [Alpine 3.22.0 release notes](https://alpinelinux.org/posts/Alpine-3.22.0-released.html).
* FEATURE: all the VictoriaMetrics components: add `-http.disableKeepAlive` to disable HTTP keep-alives for incoming connections. The flag could improve load balancing among replicas behind HTTP load balancers. See [#9125](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9125) and [#2395](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2395) for details.
* FEATURE: [dashboards/cluster](https://grafana.com/grafana/dashboards/11176): add panel `Partitions scheduled for re-processing` to `Troubleshooting` row. It shows the amount of data scheduled for [downsampling](https://docs.victoriametrics.com/#downsampling) or [retention filters](https://docs.victoriametrics.com/#retention-filters). The new panel should help to correlate resource usage with background re-processing of partitions.
* FEATURE: [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/): support [rate_prometheus](https://docs.victoriametrics.com/victoriametrics/metricsql/#rate_prometheus) function, an equivalent to `increase_prometheus(series_selector[d]) / d`. See [#8901](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8901) and [#8891](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8891) for details.
* FEATURE: [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/): respect staleness markers when calculating `rate` and `increase` functions. The new behavior will interrupt rate/increase calculation if last sample on the selected time window is a [staleness marker](https://docs.victoriametrics.com/victoriametrics/vmagent/#prometheus-staleness-markers), making the series to disappear immediately instead of slowly fading away. See more details in [#8891-comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8891#issuecomment-2875542721).
* FEATURE: [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): do not break vmalert process under replay mode when rule uses `query` template, but only logging a warning.
* FEATURE: [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): correct the rule evaluation timestamp if the system clock is changed during runtime. See [#8790](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8790).
* FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert/): respect [group](https://docs.victoriametrics.com/victoriametrics/vmalert/#groups) `concurrency` setting in [replay mode](https://docs.victoriametrics.com/victoriametrics/vmalert/#rules-backfilling) when `-replay.rulesDelay=0`. See this [#7387](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7387) for details. Thanks to @BenNF for the [PR #9214](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9214).
* FEATURE: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): allow overriding default limits for in-memory cache `storage/metricName` via flag `-storage.cacheSizeStorageMetricName`.
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): fix incorrect sorting of tag filters, which led to suboptimal tag filter evaluation order and potentially degraded query performance in rare cases. See [#9127](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9127) for details.
* BUGFIX: [vmbackup](https://docs.victoriametrics.com/vmbackup/), [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanager/): fix server-side copying of objects for Azure Blob Storage when using managed identity for authentication. Previously, it wasn't possible to use [smart backups](https://docs.victoriametrics.com/victoriametrics/vmbackup/#smart-backups) strategy for `vmbackup` as server-side copy would fail. See [#9131](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9131).

View File

@@ -1552,6 +1552,8 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/vmagent/ .
Incoming connections to -httpListenAddr are closed after the configured timeout. This may help evenly spreading load among a cluster of services behind TCP-level load balancer. Zero value disables closing of incoming connections (default 2m0s)
-http.disableCORS
Disable CORS for all origins (*)
-http.disableKeepAlive
Whether to disable HTTP keep-alive for incoming connections at -httpListenAddr
-http.disableResponseCompression
Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
-http.header.csp string

View File

@@ -817,9 +817,11 @@ max range per request: 8h20m0s
2021-06-07T09:59:12.098Z info app/vmalert/replay.go:68 replay finished! Imported 511734 samples
```
In `replay` mode all groups are executed sequentially one-by-one. Rules within the group are
executed sequentially as well (`concurrency` setting is ignored). vmalert sends rule's expression
to [/query_range](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#range-query) endpoint
> In replay mode, groups are executed one after another in sequence. Within each group, rules are also executed sequentially,
regardless of the `concurrency` setting. This ensures that any potential chaining between rules is preserved (see `-replay.rulesDelay`).
If you want rules to run concurrently based on the `concurrency` setting, set `-replay.rulesDelay=0`.
vmalert sends rule's expression to [/query_range](https://docs.victoriametrics.com/keyconcepts/#range-query) endpoint
of the configured `-datasource.url`. Returned data is then processed according to the rule type and
backfilled to `-remoteWrite.url` via [remote Write protocol](https://prometheus.io/docs/prometheus/latest/storage/#remote-storage-integrations).
vmalert respects `evaluationInterval` value set by flag or per-group during the replay.
@@ -853,7 +855,8 @@ There are following non-required `replay` flags:
* `-replay.rulesDelay` - delay between sequential rules execution. Important in cases if there are chaining
(rules which depend on each other) rules. It is expected, that remote storage will be able to persist
previously accepted data during the delay, so data will be available for the subsequent queries.
Keep it equal or bigger than `-remoteWrite.flushInterval`.
Keep it equal or bigger than `-remoteWrite.flushInterval`. When set to `0`, allows executing rules within
the group concurrently.
* `-replay.disableProgressBar` - whether to disable progress bar which shows progress work.
Progress bar may generate a lot of log records, which is not formatted as standard VictoriaMetrics logger.
It could break logs parsing by external system and generate additional load on it.
@@ -1190,6 +1193,8 @@ The shortlist of configuration flags is the following:
Incoming connections to -httpListenAddr are closed after the configured timeout. This may help evenly spreading load among a cluster of services behind TCP-level load balancer. Zero value disables closing of incoming connections (default 2m0s)
-http.disableCORS
Disable CORS for all origins (*)
-http.disableKeepAlive
Whether to disable HTTP keep-alive for incoming connections at -httpListenAddr
-http.disableResponseCompression
Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
-http.header.csp string
@@ -1492,7 +1497,7 @@ The shortlist of configuration flags is the following:
-replay.ruleRetryAttempts int
Defines how many retries to make before giving up on rule if request for it returns an error. (default 5)
-replay.rulesDelay duration
Delay between rules evaluation within the group. Could be important if there are chained rules inside the group and processing need to wait for previous rule results to be persisted by remote storage before evaluating the next rule.Keep it equal or bigger than -remoteWrite.flushInterval. (default 1s)
Delay before evaluating the next rule within the group. Is important for chained rules. Keep it equal or bigger than -remoteWrite.flushInterval. When set to >0, replay ignores group's concurrency setting. (default 1s)
-replay.timeFrom string
The time filter in RFC3339 format to start the replay from. E.g. '2020-01-01T20:07:00Z'
-replay.timeTo string

View File

@@ -1271,6 +1271,8 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/vmauth/ .
Incoming connections to -httpListenAddr are closed after the configured timeout. This may help evenly spreading load among a cluster of services behind TCP-level load balancer. Zero value disables closing of incoming connections (default 2m0s)
-http.disableCORS
Disable CORS for all origins (*)
-http.disableKeepAlive
Whether to disable HTTP keep-alive for incoming connections at -httpListenAddr
-http.disableResponseCompression
Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
-http.header.csp string

View File

@@ -366,6 +366,8 @@ Run `vmbackup -help` in order to see all the available options:
Incoming connections to -httpListenAddr are closed after the configured timeout. This may help evenly spreading load among a cluster of services behind TCP-level load balancer. Zero value disables closing of incoming connections (default 2m0s)
-http.disableCORS
Disable CORS for all origins (*)
-http.disableKeepAlive
Whether to disable HTTP keep-alive for incoming connections at -httpListenAddr
-http.disableResponseCompression
Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
-http.header.csp string

View File

@@ -493,6 +493,8 @@ command-line flags:
Incoming connections to -httpListenAddr are closed after the configured timeout. This may help evenly spreading load among a cluster of services behind TCP-level load balancer. Zero value disables closing of incoming connections (default 2m0s)
-http.disableCORS
Disable CORS for all origins (*)
-http.disableKeepAlive
Whether to disable HTTP keep-alive for incoming connections at -httpListenAddr
-http.disableResponseCompression
Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
-http.header.csp string

View File

@@ -377,6 +377,8 @@ Below is the list of configuration flags (it can be viewed by running `./vmgatew
Incoming connections to -httpListenAddr are closed after the configured timeout. This may help evenly spreading load among a cluster of services behind TCP-level load balancer. Zero value disables closing of incoming connections (default 2m0s)
-http.disableCORS
Disable CORS for all origins (*)
-http.disableKeepAlive
Whether to disable HTTP keep-alive for incoming connections at -httpListenAddr
-http.disableResponseCompression
Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
-http.header.csp string

View File

@@ -90,6 +90,8 @@ Run `vmrestore -help` in order to see all the available options:
Incoming connections to -httpListenAddr are closed after the configured timeout. This may help evenly spreading load among a cluster of services behind TCP-level load balancer. Zero value disables closing of incoming connections (default 2m0s)
-http.disableCORS
Disable CORS for all origins (*)
-http.disableKeepAlive
Whether to disable HTTP keep-alive for incoming connections at -httpListenAddr
-http.disableResponseCompression
Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
-http.header.csp string

View File

@@ -0,0 +1,153 @@
> VictoriaTraces is currently under active development and not ready for production use. It is built on top of VictoriaLogs and therefore shares some flags and APIs. These will be fully separated once VictoriaTraces reaches a stable release. Until then, features may change or break without notice.
VictoriaTraces is an open-source, user-friendly database designed for storing and querying distributed [tracing data](https://en.wikipedia.org/wiki/Tracing_(software)),
built by the [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics) team.
VictoriaTraces provides the following features:
- It is resource-efficient and fast. It uses up to [**3.7x less RAM and up to 2.6x less CPU**](https://victoriametrics.com/blog/dev-note-distributed-tracing-with-victorialogs/) than other solutions such as Grafana Tempo.
- VictoriaTraces' capacity and performance scales linearly with the available resources (CPU, RAM, disk IO, disk space).
- It accepts trace spans in the popular [OpenTelemetry protocol](https://opentelemetry.io/docs/specs/otel/protocol/)(OTLP).
- It provides [Jaeger Query Service JSON APIs](https://www.jaegertracing.io/docs/2.6/apis/#internal-http-json)
to integrate with [Grafana](https://grafana.com/docs/grafana/latest/datasources/jaeger/) or [Jaeger Frontend](https://www.jaegertracing.io/docs/2.6/frontend-ui/).
## Quick Start
The easiest way to get started with VictoriaTraces is by using the pre-built Docker Compose file.
It launches VictoriaTraces, Grafana, and HotROD (a sample application that generates tracing data).
Everything is preconfigured and connected out of the box, so you can start exploring distributed tracing within minutes.
Clone the repository:
```bash
git clone -b victoriatraces --single-branch https://github.com/VictoriaMetrics/VictoriaMetrics.git;
cd VictoriaMetrics;
```
Run VictoriaTraces with Docker Compose:
```bash
make docker-vt-single-up;
```
Now you can open HotROD at [http://localhost:8080](http://localhost:8080) and click around to generate some traces.
Then, you can open Grafana at [http://localhost:3000/explore](http://localhost:3000/explore) and explore the traces using the Jaeger data source.
To stop the services, run:
```bash
make docker-vt-single-down;
```
You can read more about docker compose and what's available there in the [Docker compose environment for VictoriaTraces](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/deployment/victoriatraces/deployment/docker/README.md#victoriatraces-server).
### How to build from sources
Building from sources is reasonable when developing additional features specific to your needs or when testing bugfixes.
{{% collapse name="How to build from sources" %}}
Clone VictoriaMetrics repository:
```bash
git clone -b victoriatraces --single-branch https://github.com/VictoriaMetrics/VictoriaMetrics.git;
cd VictoriaMetrics;
```
#### Build binary with go build
1. [Install Go](https://golang.org/doc/install).
1. Run `make victoria-traces` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaTraces).
It builds `victoria-traces` binary and puts it into the `bin` folder.
#### Build binary with Docker
1. [Install docker](https://docs.docker.com/install/).
1. Run `make victoria-traces-prod` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaTraces).
It builds `victoria-traces-prod` binary and puts it into the `bin` folder.
#### Building docker images
Run `make package-victoria-traces`. It builds `victoriametrics/victoria-traces:<PKG_TAG>` docker image locally.
`<PKG_TAG>` is auto-generated image tag, which depends on source code in the repository.
The `<PKG_TAG>` may be manually set via `PKG_TAG=foobar make package-victoria-traces`.
The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
by setting it via `<ROOT_IMAGE>` environment variable.
For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
```sh
ROOT_IMAGE=scratch make package-victoria-traces
```
{{% /collapse %}}
### Configure and run VictoriaTraces
VictoriaTraces can be run with:
```shell
/path/to/victoria-traces -storageDataPath=victoria-traces-data -retentionPeriod=7d
```
or with Docker:
```shell
docker run --rm -it -p 9428:9428 -v ./victoria-traces-data:/victoria-traces-data \
docker.io/victoriametrics/victoria-traces:latest -storageDataPath=victoria-traces-data
```
VictoriaTraces is configured via command-line flags.
All the command-line flags have sane defaults, so there is no need in tuning them in general case.
VictoriaTraces runs smoothly in most environments without additional configuration.
Pass `-help` to VictoriaTraces in order to see the list of supported command-line flags with their description and default values:
```bash
/path/to/victoria-traces -help
```
The following command-line flags are used the most:
* `-storageDataPath` - VictoriaTraces stores all the data in this directory. The default path is `victoria-traces-data` in the current working directory.
* `-retentionPeriod` - retention for stored data. Older data is automatically deleted. Default retention is 7 days.
Once it's running, it will listen to port `9428` (`-httpListenAddr`) and provide the following APIs:
1. for ingestion:
```
http://<victoria-traces>:9428/insert/opentelemetry/v1/traces
```
2. for querying:
```
http://<victoria-traces>:9428/select/jaeger/<endpoints>
```
See [data ingestion](https://docs.victoriametrics.com/victoriatraces/data-ingestion/) and [querying](https://docs.victoriametrics.com/VictoriaTraces/querying/) for more details.
## How does it work
VictoriaTraces was initially built on top of [VictoriaLogs](https://docs.victoriametrics.com/victorialogs/), a log database.
It receives trace spans in OTLP format, transforms them into structured logs, and provides [Jaeger Query Service JSON APIs](https://www.jaegertracing.io/docs/2.6/apis/#internal-http-json) for querying.
For detailed data model and example, see: [Key Concepts](https://docs.victoriametrics.com/victoriatraces/keyConcepts).
![How does VictoriaTraces work](how-does-it-work.webp)
Building VictoriaTraces in this way enables it to scale easily and linearly with the available resources, like VictoriaLogs.
## List of command-line flags
```shell
-search.traceMaxDurationWindow
The lookbehind/lookahead window of searching for the rest trace spans after finding one span.
It allows extending the search start time and end time by `-search.traceMaxDurationWindow` to make sure all spans are included.
It affects both Jaeger's `/api/traces` and `/api/traces/<trace_id>` APIs. (default: 10m)
-search.traceMaxServiceNameList
The maximum number of service name can return in a get service name request.
This limit affects Jaeger's `/api/services` API. (default: 1000)
-search.traceMaxSpanNameList
The maximum number of span name can return in a get span name request.
This limit affects Jaeger's `/api/services/*/operations` API. (default: 1000)
-search.traceSearchStep
Splits the [0, now] time range into many small time ranges by -search.traceSearchStep
when searching for spans by `trace_id`. Once it finds spans in a time range, it performs an additional search according to `-search.traceMaxDurationWindow` and then stops.
It affects Jaeger's `/api/traces/<trace_id>` API. (default: 1d)
-search.traceServiceAndSpanNameLookbehind
The time range of searching for service names and span names.
It affects Jaeger's `/api/services` and `/api/services/*/operations` APIs. (default: 7d)
```
See also: [VictoriaLogs - List of Command-line flags](https://docs.victoriametrics.com/victorialogs/#list-of-command-line-flags)

View File

@@ -0,0 +1,11 @@
---
title: VictoriaTraces
menu:
docs:
weight: 17
identifier: victoriatraces
pageRef: "/"
tags:
- traces
---
{{% content "README.md" %}}

View File

@@ -0,0 +1,90 @@
> VictoriaTraces is currently under active development and not ready for production use. It is built on top of VictoriaLogs and therefore shares some flags and APIs. These will be fully separated once VictoriaTraces reaches a stable release. Until then, features may change or break without notice.
[VictoriaTraces](https://docs.victoriametrics.com/victoriatraces/) can accept trace spans via [the OpenTelemetry protocol (OTLP)](https://opentelemetry.io/docs/specs/otlp/).
## HTTP APIs
### Opentelemetry API
VictoriaTraces provides the following API for OpenTelemetry data ingestion:
- `/insert/opentelemetry/v1/traces`
See more details [in this docs](https://docs.victoriametrics.com/victoriatraces/data-ingestion/opentelemetry/).
### HTTP parameters
VictoriaTraces accepts optional HTTP parameters at data ingestion HTTP API via [HTTP query string parameters](https://en.wikipedia.org/wiki/Query_string), or via [HTTP headers](https://en.wikipedia.org/wiki/List_of_HTTP_header_fields).
HTTP query string parameters have priority over HTTP Headers.
#### HTTP Query string parameters
All the [HTTP-based data ingestion protocols](#http-apis) support the following [HTTP query string](https://en.wikipedia.org/wiki/Query_string) args:
- `extra_fields` - an optional comma-separated list of [trace fields](https://docs.victoriametrics.com/victoriatraces/keyconcepts/#data-model),
which must be added to all the ingested traces. The format of every `extra_fields` entry is `field_name=field_value`.
If the trace entry contains fields from the `extra_fields`, then they are overwritten by the values specified in `extra_fields`.
- `debug` - if this arg is set to `1`, then the ingested traces aren't stored in VictoriaTraces. Instead,
the ingested data is traceged by VictoriaTraces, so it can be investigated later.
See also [HTTP headers](#http-headers).
#### HTTP headers
All the [HTTP-based data ingestion protocols](#http-apis) support the following [HTTP Headers](https://en.wikipedia.org/wiki/List_of_HTTP_header_fields)
additionally to [HTTP query args](#http-query-string-parameters):
- `AccountID` - accountID of the tenant to ingest data to. See [multitenancy docs](https://docs.victoriametrics.com/victoriatraces/#multitenancy) for details.
- `ProjectID`- projectID of the tenant to ingest data to. See [multitenancy docs](https://docs.victoriametrics.com/victoriatraces/#multitenancy) for details.
- `VL-Extra-Fields` - an optional comma-separated list of [trace fields](https://docs.victoriametrics.com/victoriatraces/keyconcepts/#data-model),
which must be added to all the ingested traces. The format of every `extra_fields` entry is `field_name=field_value`.
If the trace entry contains fields from the `extra_fields`, then they are overwritten by the values specified in `extra_fields`.
- `VL-Debug` - if this parameter is set to `1`, then the ingested traces aren't stored in VictoriaTraces. Instead,
the ingested data is traceged by VictoriaTraces, so it can be investigated later.
See also [HTTP Query string parameters](#http-query-string-parameters).
## Troubleshooting
The following command can be used for verifying whether the data is successfully ingested into VictoriaTraces:
```sh
curl http://<victoria-traces>:9428/select/logsql/query -d 'query=*' | head
```
This command selects all the data ingested into VictoriaTraces via [HTTP query API](https://docs.victoriametrics.com/victoriatraces/querying/#http-api)
using [any value filter](https://docs.victoriametrics.com/victorialogs/logsql/#any-value-filter),
while `head` cancels query execution after reading the first 10 trace spans. See [these docs](https://docs.victoriametrics.com/victoriatraces/querying/#command-line)
for more details on how `head` integrates with VictoriaTraces.
The response by default contains all the [trace span fields](https://docs.victoriametrics.com/victoriatraces/keyconcepts/#data-model).
See [how to query specific fields](https://docs.victoriametrics.com/victoriatraces/logsql/#querying-specific-fields).
VictoriaTraces provides the following command-line flags, which can help debugging data ingestion issues:
- `-logNewStreams` - if this flag is passed to VictoriaTraces, then it traces all the newly
registered [streams](https://docs.victoriametrics.com/victoriatraces/keyconcepts/#stream-fields).
This may help debugging [high cardinality issues](https://docs.victoriametrics.com/victoriatraces/keyconcepts/#high-cardinality).
- `-logIngestedRows` - if this flag is passed to VictoriaTraces, then it traces all the ingested
[trace span entries](https://docs.victoriametrics.com/victoriatraces/keyconcepts/#data-model).
See also `debug` [parameter](#http-parameters).
VictoriaTraces exposes various [metrics](https://docs.victoriametrics.com/victoriatraces/#monitoring), which may help debugging data ingestion issues:
- `vl_rows_ingested_total` - the number of ingested [trace span entries](https://docs.victoriametrics.com/victoriatraces/keyconcepts/#data-model)
since the last VictoriaTraces restart. If this number increases over time, then trace spans are successfully ingested into VictoriaTraces.
The ingested trace spans can be inspected in the following ways:
- By passing `debug=1` parameter to every request to [data ingestion APIs](#http-apis). The ingested spans aren't stored in VictoriaTraces
in this case. Instead, they are logged, so they can be investigated later.
The `vl_rows_dropped_total` [metric](https://docs.victoriametrics.com/victoriatraces/#monitoring) is incremented for each logged row.
- By passing `-logIngestedRows` command-line flag to VictoriaTraces. In this case it traces all the ingested data, so it can be investigated later.
- `vl_streams_created_total` - the number of created [trace streams](https://docs.victoriametrics.com/victoriatraces/keyconcepts/#stream-fields)
since the last VictoriaTraces restart. If this metric grows rapidly during extended periods of time, then this may lead
to [high cardinality issues](https://docs.victoriametrics.com/victoriatraces/keyconcepts/#high-cardinality).
The newly created trace streams can be inspected in traces by passing `-logNewStreams` command-line flag to VictoriaTraces.

View File

@@ -0,0 +1,15 @@
---
title: Data ingestion
weight: 3
menu:
docs:
identifier: victoriatraces-data-ingestion
parent: "victoriatraces"
weight: 3
tags:
- traces
aliases:
- /victoriatraces/data-ingestion/
- /victoriatraces/data-ingestion/index.html
---
{{% content "README.md" %}}

View File

@@ -0,0 +1,82 @@
---
weight: 4
title: OpenTelemetry setup
disableToc: true
menu:
docs:
parent: "victoriatraces-data-ingestion"
weight: 4
tags:
- traces
aliases:
- /victoriatraces/data-ingestion/OpenTelemetry.html
---
> VictoriaTraces is currently under active development and not ready for production use. It is built on top of VictoriaLogs and therefore shares some flags and APIs. These will be fully separated once VictoriaTraces reaches a stable release. Until then, features may change or break without notice.
VictoriaTraces supports both client open-telemetry [SDK](https://opentelemetry.io/docs/languages/) and [collector](https://opentelemetry.io/docs/collector/).
## Client SDK
The OpenTelemetry provides detailed document and examples for various programming languages:
- [C++](https://opentelemetry.io/docs/languages/cpp/)
- [C#/.NET](https://opentelemetry.io/docs/languages/dotnet/)
- [Erlang/Elixir](https://opentelemetry.io/docs/languages/erlang/)
- [Go](https://opentelemetry.io/docs/languages/go/)
- [Java](https://opentelemetry.io/docs/languages/java/)
- [JavaScript](https://opentelemetry.io/docs/languages/js/)
- [PHP](https://opentelemetry.io/docs/languages/php/)
- [Python](https://opentelemetry.io/docs/languages/python/)
- [Ruby](https://opentelemetry.io/docs/languages/ruby/)
- [Rust](https://opentelemetry.io/docs/languages/rust/)
- [Swift](https://opentelemetry.io/docs/languages/swift/)
To send data to VictoriaTraces, specify the `EndpointURL` for http-exporter builder to `http://<victoria-traces>:9428/insert/opentelemetry/v1/traces`.
Consider the following example for Go SDK:
```go
traceExporter, err := otlptracehttp.New(ctx,
otlptracehttp.WithEndpointURL("http://<victoria-traces>:9428/insert/opentelemetry/v1/traces"),
)
```
VictoriaTraces automatically use `service.name` in **resource attributes** and `name` in **span** as [stream fields](https://docs.victoriametrics.com/victoriatraces/keyconcepts/#stream-fields).
While the remaining data (including [resource](https://opentelemetry.io/docs/specs/otel/overview/#resources), [instrumentation scope](https://opentelemetry.io/docs/specs/otel/common/instrumentation-scope/), and fields in [span](https://opentelemetry.io/docs/specs/otel/trace/api/#span), like `trace_id`, `span_id`, span `attributes` and more) are stored as [regular fields](https://docs.victoriametrics.com/victoriatraces/keyconcepts/#data-model):
VictoriaTraces supports other HTTP headers - see the list [here](https://docs.victoriametrics.com/victoriatraces/data-ingestion/#http-headers).
The ingested trace spans can be queried according to [these docs](https://docs.victoriametrics.com/victoriatraces/querying/).
## Collector configuration
VictoriaTraces supports receiving traces from the following OpenTelemetry collector:
* [OpenTelemetry](#opentelemetry)
### OpenTelemetry
To send the collected traces to VictoriaTraces, specify traces endpoint for [OTLP/HTTP exporter](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/otlphttpexporter/README.md) in configuration file:
```yaml
exporters:
otlphttp:
traces_endpoint: http://<victoria-traces>:9428/insert/opentelemetry/v1/traces
```
VictoriaTraces supports various HTTP headers, which can be used during data ingestion - see the list [here](https://docs.victoriametrics.com/victoriatraces/data-ingestion/#http-headers).
These headers can be passed to OpenTelemetry exporter config via `headers` options. For example, the following configs add (or overwrites) `foo: bar` field to each trace span during data ingestion:
```yaml
exporters:
otlphttp:
traces_endpoint: http://<victoria-traces>:9428/insert/opentelemetry/v1/traces
headers:
VL-Extra-Fields: foo=bar
```
See also:
* [Data ingestion troubleshooting](https://docs.victoriametrics.com/victoriatraces/data-ingestion/#troubleshooting).
* [How to query VictoriaTraces](https://docs.victoriametrics.com/victoriatraces/querying/).
* [Docker-compose demo for OpenTelemetry collector integration with VictoriaTraces](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker/victoriatraces/opentelemetry-collector).

Some files were not shown because too many files have changed in this diff Show More