Compare commits

..

12 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
b173f24041 docs/VictoriaLogs/CHANGELOG.md: cut v0.28.0-victorialogs release 2024-07-10 03:08:16 +02:00
Aliaksandr Valialkin
ac06569c49 app/vlinsert/loki: use easyproto instead for parsing Loki protobuf messages 2024-07-10 03:05:17 +02:00
Aliaksandr Valialkin
00c666a6c3 app/vlselect/vmui: run make vmui-logs-update after 662e026279 2024-07-10 00:50:10 +02:00
Aliaksandr Valialkin
aa9bb99527 lib/logstorage: drop all the pipes from the query when calculating the number of matching logs at /select/logsql/hits API 2024-07-10 00:39:28 +02:00
Aliaksandr Valialkin
3c02937a34 all: consistently use 'any' instead of 'interface{}'
'any' type is supported starting from Go1.18. Let's consistently use it
instead of 'interface{}' type across the code base, since `any` is easier to read than 'interface{}'.
2024-07-10 00:20:37 +02:00
Aliaksandr Valialkin
08c32232a6 app/vlinsert/loki: remove unused functions from the generated protobuf code 2024-07-10 00:18:48 +02:00
Aliaksandr Valialkin
a9525da8a4 lib: consistently use f-tests instead of table-driven tests
This makes easier to read and debug these tests. This also reduces test lines count by 15% from 3K to 2.5K
See https://itnext.io/f-tests-as-a-replacement-for-table-driven-tests-in-go-8814a8b19e9e

While at it, consistently use t.Fatal* instead of t.Error*, since t.Error* usually leads
to more complicated and fragile tests, while it doesn't bring any practical benefits over t.Fatal*.
2024-07-09 22:40:50 +02:00
Yury Molodov
662e026279 vmui/logs: add spinner to bar chart (#6577)
Add a spinner to the bar chart 

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6558

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-09 14:58:48 +02:00
Hui Wang
8e9f98e725 security: upgrade base docker image (Alpine) from 3.20.0 to 3.20.1
See https://www.alpinelinux.org/posts/Alpine-3.20.1-released.html

>including security fixes for:
OPENSSL
[CVE-2024-4741](https://security.alpinelinux.org/vuln/CVE-2024-4741)
BUSYBOX
[CVE-2023-42364](https://security.alpinelinux.org/vuln/CVE-2023-42364)
[CVE-2023-42365](https://security.alpinelinux.org/vuln/CVE-2023-42365)
2024-07-09 11:38:05 +02:00
Phuong Le
c341369fc1 docs: use white version of logo for dark theme (#6610)
The logo on the README is hard to recognize in dark mode on GitHub, so I
made it responsive based on user preferences.
2024-07-09 11:04:00 +02:00
hagen1778
6e17255ec0 docs: follow-up 0e1dbdee28
* restore old anchor names to keep links compatibility.
See https://docs.victoriametrics.com/#documentation requirements
* consistently use the same format for commands `sh` as it makes it better
renderred and automatically adds `copy` button to fileds with commands
* simplify the text by removing extra points in the list
* add recommendations for installing the cluster setup
* explicitly mention the ports services are listening on
* add description for `storageNode` cmd-line flag to inform the reader what
values need to be put into it
* fix the incorrect vmui link in cluster installation recommendation
* rename component anchors to be more unique, because URL doesn't respect
hierarchy for the anchored links and may result into conflicts in future

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-09 10:43:26 +02:00
Artem Navoiev
878c727a3a add alt to cluster logo, this allows to hide it on our docs site
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-09 10:26:55 +02:00
168 changed files with 2663 additions and 4420 deletions

View File

@@ -6,14 +6,12 @@ on:
- cluster
- master
paths:
- '.github/workflows/main.yml'
- '**.go'
pull_request:
branches:
- cluster
- master
paths:
- '.github/workflows/main.yml'
- '**.go'
permissions:
@@ -48,25 +46,10 @@ jobs:
key: go-artifacts-${{ runner.os }}-check-all-${{ steps.go.outputs.go-version }}-${{ hashFiles('go.sum', 'Makefile', 'app/**/Makefile') }}
restore-keys: go-artifacts-${{ runner.os }}-check-all-
- name: Run fmt
run: make fmt
- name: Run vet
run: make vet
# Use action instead of `make golangci-lint` to speed up the process
# as it caches data between builds.
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v6
with:
# The version must match "install-golangci-lint" Makefile target version.
version: v1.59.1
- name: Run govulncheck
run: make govulncheck
- name: Check diff
run: git diff --exit-code
- name: Run check-all
run: |
make check-all
git diff --exit-code
test:
name: test

View File

@@ -493,7 +493,6 @@ golangci-lint: install-golangci-lint
golangci-lint run
install-golangci-lint:
# The version must match GitHub main.yml lint job "Run golangci-lint" step version.
which golangci-lint || curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(shell go env GOPATH)/bin v1.59.1
remove-golangci-lint:

View File

@@ -9,7 +9,11 @@
[![Build Status](https://github.com/VictoriaMetrics/VictoriaMetrics/workflows/main/badge.svg)](https://github.com/VictoriaMetrics/VictoriaMetrics/actions)
[![codecov](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/branch/master/graph/badge.svg)](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics)
<img src="docs/logo.webp" width="300" alt="VictoriaMetrics logo">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/logo_white.webp">
<source media="(prefers-color-scheme: light)" srcset="docs/logo.webp">
<img width="300" alt="VictoriaMetrics logo" src="docs/logo.webp">
</picture>
VictoriaMetrics is a fast, cost-effective and scalable monitoring solution and time series database.
See [case studies for VictoriaMetrics](https://docs.victoriametrics.com/casestudies/).

View File

@@ -117,7 +117,7 @@ func (q *Query) metrics() ([]Metric, error) {
type QueryInstant struct {
Result []struct {
Labels map[string]string `json:"metric"`
TV [2]interface{} `json:"value"`
TV [2]any `json:"value"`
} `json:"result"`
}
@@ -140,7 +140,7 @@ func (q QueryInstant) metrics() ([]Metric, error) {
type QueryRange struct {
Result []struct {
Metric map[string]string `json:"metric"`
Values [][]interface{} `json:"values"`
Values [][]any `json:"values"`
} `json:"result"`
}
@@ -432,7 +432,7 @@ func httpReadMetrics(t *testing.T, address, query string) []Metric {
return rows
}
func httpReadStruct(t *testing.T, address, query string, dst interface{}) {
func httpReadStruct(t *testing.T, address, query string, dst any) {
t.Helper()
s := newSuite(t)
resp, err := http.Get(address + query)

View File

@@ -79,12 +79,14 @@ func parseProtobufRequest(data []byte, lmp insertutils.LogMessageProcessor) (int
req := getPushRequest()
defer putPushRequest(req)
err = req.Unmarshal(bb.B)
err = req.UnmarshalProtobuf(bb.B)
if err != nil {
return 0, fmt.Errorf("cannot parse request body: %w", err)
}
var commonFields []logstorage.Field
fields := getFields()
defer putFields(fields)
rowsIngested := 0
streams := req.Streams
currentTimestamp := time.Now().UnixNano()
@@ -92,30 +94,60 @@ func parseProtobufRequest(data []byte, lmp insertutils.LogMessageProcessor) (int
stream := &streams[i]
// st.Labels contains labels for the stream.
// Labels are same for all entries in the stream.
commonFields, err = parsePromLabels(commonFields[:0], stream.Labels)
fields.fields, err = parsePromLabels(fields.fields[:0], stream.Labels)
if err != nil {
return rowsIngested, fmt.Errorf("cannot parse stream labels %q: %w", stream.Labels, err)
}
fields := commonFields
commonFieldsLen := len(fields.fields)
entries := stream.Entries
for j := range entries {
entry := &entries[j]
fields = append(fields[:len(commonFields)], logstorage.Field{
e := &entries[j]
fields.fields = fields.fields[:commonFieldsLen]
for _, lp := range e.StructuredMetadata {
fields.fields = append(fields.fields, logstorage.Field{
Name: lp.Name,
Value: lp.Value,
})
}
fields.fields = append(fields.fields, logstorage.Field{
Name: "_msg",
Value: entry.Line,
Value: e.Line,
})
ts := entry.Timestamp.UnixNano()
ts := e.Timestamp.UnixNano()
if ts == 0 {
ts = currentTimestamp
}
lmp.AddRow(ts, fields)
lmp.AddRow(ts, fields.fields)
}
rowsIngested += len(stream.Entries)
}
return rowsIngested, nil
}
func getFields() *fields {
v := fieldsPool.Get()
if v == nil {
return &fields{}
}
return v.(*fields)
}
func putFields(f *fields) {
f.fields = f.fields[:0]
fieldsPool.Put(f)
}
var fieldsPool sync.Pool
type fields struct {
fields []logstorage.Field
}
// parsePromLabels parses log fields in Prometheus text exposition format from s, appends them to dst and returns the result.
//
// See test data of promtail for examples: https://github.com/grafana/loki/blob/a24ef7b206e0ca63ee74ca6ecb0a09b745cd2258/pkg/push/types_test.go
@@ -181,6 +213,6 @@ func getPushRequest() *PushRequest {
}
func putPushRequest(req *PushRequest) {
req.Reset()
req.reset()
pushReqsPool.Put(req)
}

View File

@@ -36,7 +36,7 @@ func (tlp *testLogMessageProcessor) AddRow(timestamp int64, fields []logstorage.
Entries: []Entry{
{
Timestamp: time.Unix(0, timestamp),
Line: msg,
Line: strings.Clone(msg),
},
},
})
@@ -58,10 +58,7 @@ func TestParseProtobufRequest_Success(t *testing.T) {
t.Fatalf("unexpected number of streams; got %d; want %d", len(tlp.pr.Streams), n)
}
data, err := tlp.pr.Marshal()
if err != nil {
t.Fatalf("unexpected error when marshaling PushRequest: %s", err)
}
data := tlp.pr.MarshalProtobuf(nil)
encodedData := snappy.Encode(nil, data)
tlp2 := &insertutils.TestLogMessageProcessor{}

View File

@@ -9,6 +9,7 @@ import (
"github.com/golang/snappy"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vlinsert/insertutils"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
)
func BenchmarkParseProtobufRequest(b *testing.B) {
@@ -38,29 +39,47 @@ func benchmarkParseProtobufRequest(b *testing.B, streams, rows, labels int) {
})
}
func getProtobufBody(streams, rows, labels int) []byte {
var pr PushRequest
for i := 0; i < streams; i++ {
var st Stream
st.Labels = `{`
for j := 0; j < labels; j++ {
st.Labels += `label_` + strconv.Itoa(j) + `="value_` + strconv.Itoa(j) + `"`
if j < labels-1 {
st.Labels += `,`
func getProtobufBody(streamsCount, rowsCount, labelsCount int) []byte {
var b []byte
var entries []Entry
streams := make([]Stream, streamsCount)
for i := range streams {
b = b[:0]
b = append(b, '{')
for j := 0; j < labelsCount; j++ {
b = append(b, "label_"...)
b = strconv.AppendInt(b, int64(j), 10)
b = append(b, `="value_`...)
b = strconv.AppendInt(b, int64(j), 10)
b = append(b, '"')
if j < labelsCount-1 {
b = append(b, ',')
}
}
st.Labels += `}`
b = append(b, '}')
labels := string(b)
for j := 0; j < rows; j++ {
st.Entries = append(st.Entries, Entry{Timestamp: time.Now(), Line: "value_" + strconv.Itoa(j)})
var rowsBuf []byte
entriesLen := len(entries)
for j := 0; j < rowsCount; j++ {
rowsBufLen := len(rowsBuf)
rowsBuf = append(rowsBuf, "value_"...)
rowsBuf = strconv.AppendInt(rowsBuf, int64(j), 10)
entries = append(entries, Entry{
Timestamp: time.Now(),
Line: bytesutil.ToUnsafeString(rowsBuf[rowsBufLen:]),
})
}
pr.Streams = append(pr.Streams, st)
st := &streams[i]
st.Labels = labels
st.Entries = entries[entriesLen:]
}
pr := PushRequest{
Streams: streams,
}
body, _ := pr.Marshal()
body := pr.MarshalProtobuf(nil)
encodedBody := snappy.Encode(nil, body)
return encodedBody

302
app/vlinsert/loki/pb.go Normal file
View File

@@ -0,0 +1,302 @@
// Code generated by protoc-gen-gogo. DO NOT EDIT.
// source: push_request.proto
// source: https://raw.githubusercontent.com/grafana/loki/main/pkg/push/push_request.proto
// Licensed under the Apache License, Version 2.0 (the "License");
// https://github.com/grafana/loki/blob/main/pkg/push/LICENSE
package loki
import (
"fmt"
"time"
"github.com/VictoriaMetrics/easyproto"
)
var mp easyproto.MarshalerPool
// PushRequest represents Loki PushRequest
//
// See https://github.com/grafana/loki/blob/4220737a52da7ab6c9346b12d5a5d7bedbcd641d/pkg/push/push.proto#L14C1-L14C20
type PushRequest struct {
Streams []Stream
entriesBuf []Entry
labelPairBuf []LabelPair
}
func (pr *PushRequest) reset() {
pr.Streams = pr.Streams[:0]
pr.entriesBuf = pr.entriesBuf[:0]
pr.labelPairBuf = pr.labelPairBuf[:0]
}
// UnmarshalProtobuf unmarshals pr from protobuf message at src.
//
// pr remains valid until src is modified.
func (pr *PushRequest) UnmarshalProtobuf(src []byte) error {
pr.reset()
var err error
pr.entriesBuf, pr.labelPairBuf, err = pr.unmarshalProtobuf(pr.entriesBuf, pr.labelPairBuf, src)
return err
}
// MarshalProtobuf marshals r to protobuf message, appends it to dst and returns the result.
func (pr *PushRequest) MarshalProtobuf(dst []byte) []byte {
m := mp.Get()
pr.marshalProtobuf(m.MessageMarshaler())
dst = m.Marshal(dst)
mp.Put(m)
return dst
}
func (pr *PushRequest) marshalProtobuf(mm *easyproto.MessageMarshaler) {
for _, s := range pr.Streams {
s.marshalProtobuf(mm.AppendMessage(1))
}
}
func (pr *PushRequest) unmarshalProtobuf(entriesBuf []Entry, labelPairBuf []LabelPair, src []byte) ([]Entry, []LabelPair, error) {
// message PushRequest {
// repeated Stream streams = 1;
// }
var err error
var fc easyproto.FieldContext
for len(src) > 0 {
src, err = fc.NextField(src)
if err != nil {
return entriesBuf, labelPairBuf, fmt.Errorf("cannot read next field in PushRequest: %w", err)
}
switch fc.FieldNum {
case 1:
data, ok := fc.MessageData()
if !ok {
return entriesBuf, labelPairBuf, fmt.Errorf("cannot read Stream data")
}
pr.Streams = append(pr.Streams, Stream{})
s := &pr.Streams[len(pr.Streams)-1]
entriesBuf, labelPairBuf, err = s.unmarshalProtobuf(entriesBuf, labelPairBuf, data)
if err != nil {
return entriesBuf, labelPairBuf, fmt.Errorf("cannot unmarshal Stream: %w", err)
}
}
}
return entriesBuf, labelPairBuf, nil
}
// Stream represents Loki stream.
//
// See https://github.com/grafana/loki/blob/4220737a52da7ab6c9346b12d5a5d7bedbcd641d/pkg/push/push.proto#L23
type Stream struct {
Labels string
Entries []Entry
}
func (s *Stream) marshalProtobuf(mm *easyproto.MessageMarshaler) {
mm.AppendString(1, s.Labels)
for _, e := range s.Entries {
e.marshalProtobuf(mm.AppendMessage(2))
}
}
func (s *Stream) unmarshalProtobuf(entriesBuf []Entry, labelPairBuf []LabelPair, src []byte) ([]Entry, []LabelPair, error) {
// message Stream {
// string labels = 1;
// repeated Entry entries = 2;
// }
var err error
var fc easyproto.FieldContext
entriesBufLen := len(entriesBuf)
for len(src) > 0 {
src, err = fc.NextField(src)
if err != nil {
return entriesBuf, labelPairBuf, fmt.Errorf("cannot read next field in Stream: %w", err)
}
switch fc.FieldNum {
case 1:
labels, ok := fc.String()
if !ok {
return entriesBuf, labelPairBuf, fmt.Errorf("cannot read labels")
}
s.Labels = labels
case 2:
data, ok := fc.MessageData()
if !ok {
return entriesBuf, labelPairBuf, fmt.Errorf("cannot read Entry data")
}
entriesBuf = append(entriesBuf, Entry{})
e := &entriesBuf[len(entriesBuf)-1]
labelPairBuf, err = e.unmarshalProtobuf(labelPairBuf, data)
if err != nil {
return entriesBuf, labelPairBuf, fmt.Errorf("cannot unmarshal Entry: %w", err)
}
}
}
s.Entries = entriesBuf[entriesBufLen:]
return entriesBuf, labelPairBuf, nil
}
// Entry represents Loki entry.
//
// See https://github.com/grafana/loki/blob/4220737a52da7ab6c9346b12d5a5d7bedbcd641d/pkg/push/push.proto#L38
type Entry struct {
Timestamp time.Time
Line string
StructuredMetadata []LabelPair
}
func (e *Entry) marshalProtobuf(mm *easyproto.MessageMarshaler) {
marshalTime(mm, 1, e.Timestamp)
mm.AppendString(2, e.Line)
for _, lp := range e.StructuredMetadata {
lp.marshalProtobuf(mm.AppendMessage(3))
}
}
func (e *Entry) unmarshalProtobuf(labelPairBuf []LabelPair, src []byte) ([]LabelPair, error) {
// message Entry {
// Timestamp timestamp = 1;
// string line = 2;
// repeated LabelPair structuredMetadata = 3;
// }
var err error
var fc easyproto.FieldContext
labelPairBufLen := len(labelPairBuf)
for len(src) > 0 {
src, err = fc.NextField(src)
if err != nil {
return labelPairBuf, fmt.Errorf("cannot read next field in Entry: %w", err)
}
switch fc.FieldNum {
case 1:
data, ok := fc.MessageData()
if !ok {
return labelPairBuf, fmt.Errorf("cannot read Timestamp data")
}
timestamp, err := unmarshalTime(data)
if err != nil {
return labelPairBuf, fmt.Errorf("cannot unmarshal Timestamp: %w", err)
}
e.Timestamp = timestamp
case 2:
line, ok := fc.String()
if !ok {
return labelPairBuf, fmt.Errorf("cannot read Line")
}
e.Line = line
case 3:
data, ok := fc.MessageData()
if !ok {
return labelPairBuf, fmt.Errorf("cannot read StructuredMetadata")
}
labelPairBuf = append(labelPairBuf, LabelPair{})
lp := &labelPairBuf[len(labelPairBuf)-1]
if err := lp.unmarshalProtobuf(data); err != nil {
return labelPairBuf, fmt.Errorf("cannot unmarshal StructuredMetadata: %w", err)
}
}
}
e.StructuredMetadata = labelPairBuf[labelPairBufLen:]
return labelPairBuf, nil
}
// LabelPair represents Loki label pair.
//
// See https://github.com/grafana/loki/blob/4220737a52da7ab6c9346b12d5a5d7bedbcd641d/pkg/push/push.proto#L33
type LabelPair struct {
Name string
Value string
}
func (lp *LabelPair) marshalProtobuf(mm *easyproto.MessageMarshaler) {
mm.AppendString(1, lp.Name)
mm.AppendString(2, lp.Value)
}
func (lp *LabelPair) unmarshalProtobuf(src []byte) (err error) {
// message LabelPair {
// string name = 1;
// string value = 2;
// }
var fc easyproto.FieldContext
for len(src) > 0 {
src, err = fc.NextField(src)
if err != nil {
return fmt.Errorf("cannot read next field in LabelPair: %w", err)
}
switch fc.FieldNum {
case 1:
name, ok := fc.String()
if !ok {
return fmt.Errorf("cannot read name")
}
lp.Name = name
case 2:
value, ok := fc.String()
if !ok {
return fmt.Errorf("cannot unmarshal value")
}
lp.Value = value
}
}
return nil
}
func marshalTime(mm *easyproto.MessageMarshaler, fieldNum uint32, timestamp time.Time) {
nsecs := timestamp.UnixNano()
ts := Timestamp{
Seconds: nsecs / 1e9,
Nanos: int32(nsecs % 1e9),
}
ts.marshalProtobuf(mm.AppendMessage(fieldNum))
}
func unmarshalTime(src []byte) (time.Time, error) {
var ts Timestamp
if err := ts.unmarshalProtobuf(src); err != nil {
return time.Time{}, err
}
timestamp := time.Unix(ts.Seconds, int64(ts.Nanos)).UTC()
return timestamp, nil
}
// Timestamp is protobuf well-known timestamp type.
type Timestamp struct {
Seconds int64
Nanos int32
}
func (ts *Timestamp) marshalProtobuf(mm *easyproto.MessageMarshaler) {
mm.AppendInt64(1, ts.Seconds)
mm.AppendInt32(2, ts.Nanos)
}
func (ts *Timestamp) unmarshalProtobuf(src []byte) (err error) {
// message Timestamp {
// int64 seconds = 1;
// int32 nanos = 2;
// }
var fc easyproto.FieldContext
for len(src) > 0 {
src, err = fc.NextField(src)
if err != nil {
return fmt.Errorf("cannot read next field in Timestamp: %w", err)
}
switch fc.FieldNum {
case 1:
seconds, ok := fc.Int64()
if !ok {
return fmt.Errorf("cannot read Seconds")
}
ts.Seconds = seconds
case 2:
nanos, ok := fc.Int32()
if !ok {
return fmt.Errorf("cannot read Nanos")
}
ts.Nanos = nanos
}
}
return nil
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,38 +0,0 @@
syntax = "proto3";
// source: https://raw.githubusercontent.com/grafana/loki/main/pkg/push/push.proto
// Licensed under the Apache License, Version 2.0 (the "License");
// https://github.com/grafana/loki/blob/main/pkg/push/LICENSE
package logproto;
import "gogoproto/gogo.proto";
import "google/protobuf/timestamp.proto";
option go_package = "github.com/VictoriaMetrics/VictoriaMetrics/app/vlinsert/loki";
message PushRequest {
repeated StreamAdapter streams = 1 [
(gogoproto.jsontag) = "streams",
(gogoproto.customtype) = "Stream"
];
}
message StreamAdapter {
string labels = 1 [(gogoproto.jsontag) = "labels"];
repeated EntryAdapter entries = 2 [
(gogoproto.nullable) = false,
(gogoproto.jsontag) = "entries"
];
// hash contains the original hash of the stream.
uint64 hash = 3 [(gogoproto.jsontag) = "-"];
}
message EntryAdapter {
google.protobuf.Timestamp timestamp = 1 [
(gogoproto.stdtime) = true,
(gogoproto.nullable) = false,
(gogoproto.jsontag) = "ts"
];
string line = 2 [(gogoproto.jsontag) = "line"];
}

View File

@@ -1,110 +0,0 @@
package loki
// source: https://raw.githubusercontent.com/grafana/loki/main/pkg/push/timestamp.go
// Licensed under the Apache License, Version 2.0 (the "License");
// https://github.com/grafana/loki/blob/main/pkg/push/LICENSE
import (
"errors"
"strconv"
"time"
"github.com/gogo/protobuf/types"
)
const (
// Seconds field of the earliest valid Timestamp.
// This is time.Date(1, 1, 1, 0, 0, 0, 0, time.UTC).Unix().
minValidSeconds = -62135596800
// Seconds field just after the latest valid Timestamp.
// This is time.Date(10000, 1, 1, 0, 0, 0, 0, time.UTC).Unix().
maxValidSeconds = 253402300800
)
// validateTimestamp determines whether a Timestamp is valid.
// A valid timestamp represents a time in the range
// [0001-01-01, 10000-01-01) and has a Nanos field
// in the range [0, 1e9).
//
// If the Timestamp is valid, validateTimestamp returns nil.
// Otherwise, it returns an error that describes
// the problem.
//
// Every valid Timestamp can be represented by a time.Time, but the converse is not true.
func validateTimestamp(ts *types.Timestamp) error {
if ts == nil {
return errors.New("timestamp: nil Timestamp")
}
if ts.Seconds < minValidSeconds {
return errors.New("timestamp: " + formatTimestamp(ts) + " before 0001-01-01")
}
if ts.Seconds >= maxValidSeconds {
return errors.New("timestamp: " + formatTimestamp(ts) + " after 10000-01-01")
}
if ts.Nanos < 0 || ts.Nanos >= 1e9 {
return errors.New("timestamp: " + formatTimestamp(ts) + ": nanos not in range [0, 1e9)")
}
return nil
}
// formatTimestamp is equivalent to fmt.Sprintf("%#v", ts)
// but avoids the escape incurred by using fmt.Sprintf, eliminating
// unnecessary heap allocations.
func formatTimestamp(ts *types.Timestamp) string {
if ts == nil {
return "nil"
}
seconds := strconv.FormatInt(ts.Seconds, 10)
nanos := strconv.FormatInt(int64(ts.Nanos), 10)
return "&types.Timestamp{Seconds: " + seconds + ",\nNanos: " + nanos + ",\n}"
}
func sizeOfStdTime(t time.Time) int {
ts, err := timestampProto(t)
if err != nil {
return 0
}
return ts.Size()
}
func stdTimeMarshalTo(t time.Time, data []byte) (int, error) {
ts, err := timestampProto(t)
if err != nil {
return 0, err
}
return ts.MarshalTo(data)
}
func stdTimeUnmarshal(t *time.Time, data []byte) error {
ts := &types.Timestamp{}
if err := ts.Unmarshal(data); err != nil {
return err
}
tt, err := timestampFromProto(ts)
if err != nil {
return err
}
*t = tt
return nil
}
func timestampFromProto(ts *types.Timestamp) (time.Time, error) {
// Don't return the zero value on error, because corresponds to a valid
// timestamp. Instead return whatever time.Unix gives us.
var t time.Time
if ts == nil {
t = time.Unix(0, 0).UTC() // treat nil like the empty Timestamp
} else {
t = time.Unix(ts.Seconds, int64(ts.Nanos)).UTC()
}
return t, validateTimestamp(ts)
}
func timestampProto(t time.Time) (types.Timestamp, error) {
ts := types.Timestamp{
Seconds: t.Unix(),
Nanos: int32(t.Nanosecond()),
}
return ts, validateTimestamp(&ts)
}

View File

@@ -1,481 +0,0 @@
package loki
// source: https://raw.githubusercontent.com/grafana/loki/main/pkg/push/types.go
// Licensed under the Apache License, Version 2.0 (the "License");
// https://github.com/grafana/loki/blob/main/pkg/push/LICENSE
import (
"fmt"
"io"
"time"
)
// Stream contains a unique labels set as a string and a set of entries for it.
// We are not using the proto generated version but this custom one so that we
// can improve serialization see benchmark.
type Stream struct {
Labels string `protobuf:"bytes,1,opt,name=labels,proto3" json:"labels"`
Entries []Entry `protobuf:"bytes,2,rep,name=entries,proto3,customtype=EntryAdapter" json:"entries"`
Hash uint64 `protobuf:"varint,3,opt,name=hash,proto3" json:"-"`
}
// Entry is a log entry with a timestamp.
type Entry struct {
Timestamp time.Time `protobuf:"bytes,1,opt,name=timestamp,proto3,stdtime" json:"ts"`
Line string `protobuf:"bytes,2,opt,name=line,proto3" json:"line"`
}
// Marshal implements the proto.Marshaler interface.
func (m *Stream) Marshal() (dAtA []byte, err error) {
size := m.Size()
dAtA = make([]byte, size)
n, err := m.MarshalToSizedBuffer(dAtA[:size])
if err != nil {
return nil, err
}
return dAtA[:n], nil
}
// MarshalTo marshals m to dst.
func (m *Stream) MarshalTo(dAtA []byte) (int, error) {
size := m.Size()
return m.MarshalToSizedBuffer(dAtA[:size])
}
// MarshalToSizedBuffer marshals m to the sized buffer.
func (m *Stream) MarshalToSizedBuffer(dAtA []byte) (int, error) {
i := len(dAtA)
_ = i
var l int
_ = l
if m.Hash != 0 {
i = encodeVarintPush(dAtA, i, m.Hash)
i--
dAtA[i] = 0x18
}
if len(m.Entries) > 0 {
for iNdEx := len(m.Entries) - 1; iNdEx >= 0; iNdEx-- {
{
size, err := m.Entries[iNdEx].MarshalToSizedBuffer(dAtA[:i])
if err != nil {
return 0, err
}
i -= size
i = encodeVarintPush(dAtA, i, uint64(size))
}
i--
dAtA[i] = 0x12
}
}
if len(m.Labels) > 0 {
i -= len(m.Labels)
copy(dAtA[i:], m.Labels)
i = encodeVarintPush(dAtA, i, uint64(len(m.Labels)))
i--
dAtA[i] = 0xa
}
return len(dAtA) - i, nil
}
// Marshal implements the proto.Marshaler interface.
func (m *Entry) Marshal() (dAtA []byte, err error) {
size := m.Size()
dAtA = make([]byte, size)
n, err := m.MarshalToSizedBuffer(dAtA[:size])
if err != nil {
return nil, err
}
return dAtA[:n], nil
}
// MarshalTo marshals m to dst.
func (m *Entry) MarshalTo(dAtA []byte) (int, error) {
size := m.Size()
return m.MarshalToSizedBuffer(dAtA[:size])
}
// MarshalToSizedBuffer marshals m to the sized buffer.
func (m *Entry) MarshalToSizedBuffer(dAtA []byte) (int, error) {
i := len(dAtA)
_ = i
var l int
_ = l
if len(m.Line) > 0 {
i -= len(m.Line)
copy(dAtA[i:], m.Line)
i = encodeVarintPush(dAtA, i, uint64(len(m.Line)))
i--
dAtA[i] = 0x12
}
n7, err7 := stdTimeMarshalTo(m.Timestamp, dAtA[i-sizeOfStdTime(m.Timestamp):])
if err7 != nil {
return 0, err7
}
i -= n7
i = encodeVarintPush(dAtA, i, uint64(n7))
i--
dAtA[i] = 0xa
return len(dAtA) - i, nil
}
// Unmarshal unmarshals the given data into m.
func (m *Stream) Unmarshal(dAtA []byte) error {
l := len(dAtA)
iNdEx := 0
for iNdEx < l {
preIndex := iNdEx
var wire uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowPush
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
wire |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
fieldNum := int32(wire >> 3)
wireType := int(wire & 0x7)
if wireType == 4 {
return fmt.Errorf("proto: StreamAdapter: wiretype end group for non-group")
}
if fieldNum <= 0 {
return fmt.Errorf("proto: StreamAdapter: illegal tag %d (wire type %d)", fieldNum, wire)
}
switch fieldNum {
case 1:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field Labels", wireType)
}
var stringLen uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowPush
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
stringLen |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
intStringLen := int(stringLen)
if intStringLen < 0 {
return ErrInvalidLengthPush
}
postIndex := iNdEx + intStringLen
if postIndex < 0 {
return ErrInvalidLengthPush
}
if postIndex > l {
return io.ErrUnexpectedEOF
}
m.Labels = string(dAtA[iNdEx:postIndex])
iNdEx = postIndex
case 2:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field Entries", wireType)
}
var msglen int
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowPush
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
msglen |= int(b&0x7F) << shift
if b < 0x80 {
break
}
}
if msglen < 0 {
return ErrInvalidLengthPush
}
postIndex := iNdEx + msglen
if postIndex < 0 {
return ErrInvalidLengthPush
}
if postIndex > l {
return io.ErrUnexpectedEOF
}
m.Entries = append(m.Entries, Entry{})
if err := m.Entries[len(m.Entries)-1].Unmarshal(dAtA[iNdEx:postIndex]); err != nil {
return err
}
iNdEx = postIndex
case 3:
if wireType != 0 {
return fmt.Errorf("proto: wrong wireType = %d for field Hash", wireType)
}
m.Hash = 0
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowPush
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
m.Hash |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
default:
iNdEx = preIndex
skippy, err := skipPush(dAtA[iNdEx:])
if err != nil {
return err
}
if skippy < 0 {
return ErrInvalidLengthPush
}
if (iNdEx + skippy) < 0 {
return ErrInvalidLengthPush
}
if (iNdEx + skippy) > l {
return io.ErrUnexpectedEOF
}
iNdEx += skippy
}
}
if iNdEx > l {
return io.ErrUnexpectedEOF
}
return nil
}
// Unmarshal unmarshals the given data into m.
func (m *Entry) Unmarshal(dAtA []byte) error {
l := len(dAtA)
iNdEx := 0
for iNdEx < l {
preIndex := iNdEx
var wire uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowPush
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
wire |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
fieldNum := int32(wire >> 3)
wireType := int(wire & 0x7)
if wireType == 4 {
return fmt.Errorf("proto: EntryAdapter: wiretype end group for non-group")
}
if fieldNum <= 0 {
return fmt.Errorf("proto: EntryAdapter: illegal tag %d (wire type %d)", fieldNum, wire)
}
switch fieldNum {
case 1:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field Timestamp", wireType)
}
var msglen int
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowPush
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
msglen |= int(b&0x7F) << shift
if b < 0x80 {
break
}
}
if msglen < 0 {
return ErrInvalidLengthPush
}
postIndex := iNdEx + msglen
if postIndex < 0 {
return ErrInvalidLengthPush
}
if postIndex > l {
return io.ErrUnexpectedEOF
}
if err := stdTimeUnmarshal(&m.Timestamp, dAtA[iNdEx:postIndex]); err != nil {
return err
}
iNdEx = postIndex
case 2:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field Line", wireType)
}
var stringLen uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowPush
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
stringLen |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
intStringLen := int(stringLen)
if intStringLen < 0 {
return ErrInvalidLengthPush
}
postIndex := iNdEx + intStringLen
if postIndex < 0 {
return ErrInvalidLengthPush
}
if postIndex > l {
return io.ErrUnexpectedEOF
}
m.Line = string(dAtA[iNdEx:postIndex])
iNdEx = postIndex
default:
iNdEx = preIndex
skippy, err := skipPush(dAtA[iNdEx:])
if err != nil {
return err
}
if skippy < 0 {
return ErrInvalidLengthPush
}
if (iNdEx + skippy) < 0 {
return ErrInvalidLengthPush
}
if (iNdEx + skippy) > l {
return io.ErrUnexpectedEOF
}
iNdEx += skippy
}
}
if iNdEx > l {
return io.ErrUnexpectedEOF
}
return nil
}
// Size returns the size of the serialized Stream.
func (m *Stream) Size() (n int) {
if m == nil {
return 0
}
var l int
_ = l
l = len(m.Labels)
if l > 0 {
n += 1 + l + sovPush(uint64(l))
}
if len(m.Entries) > 0 {
for _, e := range m.Entries {
l = e.Size()
n += 1 + l + sovPush(uint64(l))
}
}
if m.Hash != 0 {
n += 1 + sovPush(m.Hash)
}
return n
}
// Size returns the size of the serialized Entry
func (m *Entry) Size() (n int) {
if m == nil {
return 0
}
var l int
_ = l
l = sizeOfStdTime(m.Timestamp)
n += 1 + l + sovPush(uint64(l))
l = len(m.Line)
if l > 0 {
n += 1 + l + sovPush(uint64(l))
}
return n
}
// Equal returns true if the two Streams are equal.
func (m *Stream) Equal(that interface{}) bool {
if that == nil {
return m == nil
}
that1, ok := that.(*Stream)
if !ok {
that2, ok := that.(Stream)
if ok {
that1 = &that2
} else {
return false
}
}
if that1 == nil {
return m == nil
} else if m == nil {
return false
}
if m.Labels != that1.Labels {
return false
}
if len(m.Entries) != len(that1.Entries) {
return false
}
for i := range m.Entries {
if !m.Entries[i].Equal(that1.Entries[i]) {
return false
}
}
return m.Hash == that1.Hash
}
// Equal returns true if the two Entries are equal.
func (m *Entry) Equal(that interface{}) bool {
if that == nil {
return m == nil
}
that1, ok := that.(*Entry)
if !ok {
that2, ok := that.(Entry)
if ok {
that1 = &that2
} else {
return false
}
}
if that1 == nil {
return m == nil
} else if m == nil {
return false
}
if !m.Timestamp.Equal(that1.Timestamp) {
return false
}
if m.Line != that1.Line {
return false
}
return true
}

View File

@@ -70,9 +70,10 @@ func ProcessHitsRequest(ctx context.Context, w http.ResponseWriter, r *http.Requ
fieldsLimit = 0
}
// Prepare the query
q.AddCountByTimePipe(int64(step), int64(offset), fields)
// Prepare the query for hits count.
q.Optimize()
q.DropAllPipes()
q.AddCountByTimePipe(int64(step), int64(offset), fields)
var mLock sync.Mutex
m := make(map[string]*hitsSeries)

View File

@@ -1,13 +1,13 @@
{
"files": {
"main.css": "./static/css/main.1041c3d4.css",
"main.js": "./static/js/main.8988988c.js",
"main.js": "./static/js/main.62a82f4a.js",
"static/js/685.bebe1265.chunk.js": "./static/js/685.bebe1265.chunk.js",
"static/media/MetricsQL.md": "./static/media/MetricsQL.aaabf95f2c9bf356bde4.md",
"index.html": "./index.html"
},
"entrypoints": [
"static/css/main.1041c3d4.css",
"static/js/main.8988988c.js"
"static/js/main.62a82f4a.js"
]
}

View File

@@ -1 +1 @@
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="UI for VictoriaMetrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><script src="./dashboards/index.js" type="module"></script><meta name="twitter:card" content="summary_large_image"><meta name="twitter:image" content="./preview.jpg"><meta name="twitter:title" content="UI for VictoriaMetrics"><meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta name="twitter:site" content="@VictoriaMetrics"><meta property="og:title" content="Metric explorer for VictoriaMetrics"><meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta property="og:image" content="./preview.jpg"><meta property="og:type" content="website"><script defer="defer" src="./static/js/main.8988988c.js"></script><link href="./static/css/main.1041c3d4.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="UI for VictoriaMetrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><script src="./dashboards/index.js" type="module"></script><meta name="twitter:card" content="summary_large_image"><meta name="twitter:image" content="./preview.jpg"><meta name="twitter:title" content="UI for VictoriaMetrics"><meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta name="twitter:site" content="@VictoriaMetrics"><meta property="og:title" content="Metric explorer for VictoriaMetrics"><meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta property="og:image" content="./preview.jpg"><meta property="og:type" content="website"><script defer="defer" src="./static/js/main.62a82f4a.js"></script><link href="./static/css/main.1041c3d4.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -181,7 +181,7 @@ func (rctx *relabelCtx) reset() {
}
var relabelCtxPool = &sync.Pool{
New: func() interface{} {
New: func() any {
return &relabelCtx{}
},
}

View File

@@ -1000,7 +1000,7 @@ func (rwctx *remoteWriteCtx) tryPushInternal(tss []prompbmarshal.TimeSeries) boo
}
var tssPool = &sync.Pool{
New: func() interface{} {
New: func() any {
a := []prompbmarshal.TimeSeries{}
return &a
},

View File

@@ -45,11 +45,11 @@ type Group struct {
// EvalAlignment will make the timestamp of group query requests be aligned with interval
EvalAlignment *bool `yaml:"eval_alignment,omitempty"`
// Catches all undefined fields and must be empty after parsing.
XXX map[string]interface{} `yaml:",inline"`
XXX map[string]any `yaml:",inline"`
}
// UnmarshalYAML implements the yaml.Unmarshaler interface.
func (g *Group) UnmarshalYAML(unmarshal func(interface{}) error) error {
func (g *Group) UnmarshalYAML(unmarshal func(any) error) error {
type group Group
if err := unmarshal((*group)(g)); err != nil {
return err
@@ -142,11 +142,11 @@ type Rule struct {
UpdateEntriesLimit *int `yaml:"update_entries_limit,omitempty"`
// Catches all undefined fields and must be empty after parsing.
XXX map[string]interface{} `yaml:",inline"`
XXX map[string]any `yaml:",inline"`
}
// UnmarshalYAML implements the yaml.Unmarshaler interface.
func (r *Rule) UnmarshalYAML(unmarshal func(interface{}) error) error {
func (r *Rule) UnmarshalYAML(unmarshal func(any) error) error {
type rule Rule
if err := unmarshal((*rule)(r)); err != nil {
return err
@@ -301,7 +301,7 @@ func parseConfig(data []byte) ([]Group, error) {
g := struct {
Groups []Group `yaml:"groups"`
// Catches all undefined fields and must be empty after parsing.
XXX map[string]interface{} `yaml:",inline"`
XXX map[string]any `yaml:",inline"`
}{}
err = yaml.Unmarshal(data, &g)
if err != nil {
@@ -310,7 +310,7 @@ func parseConfig(data []byte) ([]Group, error) {
return g.Groups, checkOverflow(g.XXX, "config")
}
func checkOverflow(m map[string]interface{}, ctx string) error {
func checkOverflow(m map[string]any, ctx string) error {
if len(m) > 0 {
var keys []string
for k := range m {

View File

@@ -29,7 +29,7 @@ func (l *Logger) isDisabled() bool {
}
// Errorf logs error message.
func (l *Logger) Errorf(format string, args ...interface{}) {
func (l *Logger) Errorf(format string, args ...any) {
if l.isDisabled() {
return
}
@@ -37,7 +37,7 @@ func (l *Logger) Errorf(format string, args ...interface{}) {
}
// Warnf logs warning message.
func (l *Logger) Warnf(format string, args ...interface{}) {
func (l *Logger) Warnf(format string, args ...any) {
if l.isDisabled() {
return
}
@@ -45,7 +45,7 @@ func (l *Logger) Warnf(format string, args ...interface{}) {
}
// Infof logs info message.
func (l *Logger) Infof(format string, args ...interface{}) {
func (l *Logger) Infof(format string, args ...any) {
if l.isDisabled() {
return
}
@@ -54,6 +54,6 @@ func (l *Logger) Infof(format string, args ...interface{}) {
// Panicf logs panic message and panics.
// Panicf can't be suppressed
func (l *Logger) Panicf(format string, args ...interface{}) {
func (l *Logger) Panicf(format string, args ...any) {
logger.Panicf(format, args...)
}

View File

@@ -69,7 +69,7 @@ func (t *Type) ValidateExpr(expr string) error {
}
// UnmarshalYAML implements the yaml.Unmarshaler interface.
func (t *Type) UnmarshalYAML(unmarshal func(interface{}) error) error {
func (t *Type) UnmarshalYAML(unmarshal func(any) error) error {
var s string
if err := unmarshal(&s); err != nil {
return err
@@ -87,7 +87,7 @@ func (t *Type) UnmarshalYAML(unmarshal func(interface{}) error) error {
}
// MarshalYAML implements the yaml.Unmarshaler interface.
func (t Type) MarshalYAML() (interface{}, error) {
func (t Type) MarshalYAML() (any, error) {
return t.Name, nil
}
@@ -98,7 +98,7 @@ type Header struct {
}
// UnmarshalYAML implements the yaml.Unmarshaler interface.
func (h *Header) UnmarshalYAML(unmarshal func(interface{}) error) error {
func (h *Header) UnmarshalYAML(unmarshal func(any) error) error {
var s string
if err := unmarshal(&s); err != nil {
return err

View File

@@ -119,7 +119,7 @@ func (pi *promInstant) Unmarshal(b []byte) error {
type promRange struct {
Result []struct {
Labels map[string]string `json:"metric"`
TVs [][2]interface{} `json:"values"`
TVs [][2]any `json:"values"`
} `json:"result"`
}
@@ -147,7 +147,7 @@ func (r promRange) metrics() ([]Metric, error) {
return result, nil
}
type promScalar [2]interface{}
type promScalar [2]any
func (r promScalar) metrics() ([]Metric, error) {
var m Metric

View File

@@ -51,7 +51,7 @@ type Config struct {
Checksum string
// Catches all undefined fields and must be empty after parsing.
XXX map[string]interface{} `yaml:",inline"`
XXX map[string]any `yaml:",inline"`
// This is set to the directory from where the config has been loaded.
baseDir string
@@ -73,7 +73,7 @@ type StaticConfig struct {
}
// UnmarshalYAML implements the yaml.Unmarshaler interface.
func (cfg *Config) UnmarshalYAML(unmarshal func(interface{}) error) error {
func (cfg *Config) UnmarshalYAML(unmarshal func(any) error) error {
type config Config
if err := unmarshal((*config)(cfg)); err != nil {
return err

View File

@@ -183,7 +183,7 @@ func (ar *AlertingRule) GetAlert(id uint64) *notifier.Alert {
return ar.alerts[id]
}
func (ar *AlertingRule) logDebugf(at time.Time, a *notifier.Alert, format string, args ...interface{}) {
func (ar *AlertingRule) logDebugf(at time.Time, a *notifier.Alert, format string, args ...any) {
if !ar.Debug {
return
}

View File

@@ -475,7 +475,7 @@ func delayBeforeStart(ts time.Time, key uint64, interval time.Duration, offset *
return randSleep
}
func (g *Group) infof(format string, args ...interface{}) {
func (g *Group) infof(format string, args ...any) {
msg := fmt.Sprintf(format, args...)
logger.Infof("group %q %s; interval=%v; eval_offset=%v; concurrency=%d",
g.Name, msg, g.Interval, g.EvalOffset, g.Concurrency)

View File

@@ -316,7 +316,7 @@ func templateFuncs() textTpl.FuncMap {
// humanize converts given number to a human readable format
// by adding metric prefixes https://en.wikipedia.org/wiki/Metric_prefix
"humanize": func(i interface{}) (string, error) {
"humanize": func(i any) (string, error) {
v, err := toFloat64(i)
if err != nil {
return "", err
@@ -347,7 +347,7 @@ func templateFuncs() textTpl.FuncMap {
},
// humanize1024 converts given number to a human readable format with 1024 as base
"humanize1024": func(i interface{}) (string, error) {
"humanize1024": func(i any) (string, error) {
v, err := toFloat64(i)
if err != nil {
return "", err
@@ -359,7 +359,7 @@ func templateFuncs() textTpl.FuncMap {
},
// humanizeDuration converts given seconds to a human-readable duration
"humanizeDuration": func(i interface{}) (string, error) {
"humanizeDuration": func(i any) (string, error) {
v, err := toFloat64(i)
if err != nil {
return "", err
@@ -405,7 +405,7 @@ func templateFuncs() textTpl.FuncMap {
},
// humanizePercentage converts given ratio value to a fraction of 100
"humanizePercentage": func(i interface{}) (string, error) {
"humanizePercentage": func(i any) (string, error) {
v, err := toFloat64(i)
if err != nil {
return "", err
@@ -414,7 +414,7 @@ func templateFuncs() textTpl.FuncMap {
},
// humanizeTimestamp converts given timestamp to a human readable time equivalent
"humanizeTimestamp": func(i interface{}) (string, error) {
"humanizeTimestamp": func(i any) (string, error) {
v, err := toFloat64(i)
if err != nil {
return "", err
@@ -427,7 +427,7 @@ func templateFuncs() textTpl.FuncMap {
},
// toTime converts given timestamp to a time.Time.
"toTime": func(i interface{}) (time.Time, error) {
"toTime": func(i any) (time.Time, error) {
v, err := toFloat64(i)
if err != nil {
return time.Time{}, err
@@ -524,8 +524,8 @@ func templateFuncs() textTpl.FuncMap {
// Converts a list of objects to a map with keys arg0, arg1 etc.
// This is intended to allow multiple arguments to be passed to templates.
"args": func(args ...interface{}) map[string]interface{} {
result := make(map[string]interface{})
"args": func(args ...any) map[string]any {
result := make(map[string]any)
for i, a := range args {
result[fmt.Sprintf("arg%d", i)] = a
}
@@ -565,7 +565,7 @@ func (t Time) Time() time.Time {
return time.Unix(int64(t)/second, (int64(t)%second)*nanosPerTick)
}
func toFloat64(v interface{}) (float64, error) {
func toFloat64(v any) (float64, error) {
switch i := v.(type) {
case float64:
return i, nil

View File

@@ -52,10 +52,10 @@ func TestTemplateFuncs(t *testing.T) {
t.Fatalf("unexpected mismatch")
}
formatting := func(funcName string, p interface{}, resultExpected string) {
formatting := func(funcName string, p any, resultExpected string) {
t.Helper()
v := funcs[funcName]
fLocal := v.(func(s interface{}) (string, error))
fLocal := v.(func(s any) (string, error))
result, err := fLocal(p)
if err != nil {
t.Fatalf("unexpected error for %s(%f): %s", funcName, p, err)
@@ -92,7 +92,7 @@ func TestTemplateFuncs(t *testing.T) {
formatting("humanizeTimestamp", 1679055557, "2023-03-17 12:19:17 +0000 UTC")
}
func mkTemplate(current, replacement interface{}) textTemplate {
func mkTemplate(current, replacement any) textTemplate {
tmpl := textTemplate{}
if current != nil {
switch val := current.(type) {

View File

@@ -36,7 +36,7 @@ func TestHandler(t *testing.T) {
}}
rh := &requestHandler{m: m}
getResp := func(t *testing.T, url string, to interface{}, code int) {
getResp := func(t *testing.T, url string, to any, code int) {
t.Helper()
resp, err := http.Get(url)
if err != nil {
@@ -241,7 +241,7 @@ func TestEmptyResponse(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { rhWithNoGroups.handler(w, r) }))
defer ts.Close()
getResp := func(t *testing.T, url string, to interface{}, code int) {
getResp := func(t *testing.T, url string, to any, code int) {
t.Helper()
resp, err := http.Get(url)
if err != nil {

View File

@@ -183,7 +183,7 @@ func (ar apiRule) WebLink() string {
paramGroupID, ar.GroupID, paramRuleID, ar.ID)
}
func ruleToAPI(r interface{}) apiRule {
func ruleToAPI(r any) apiRule {
if ar, ok := r.(*rule.AlertingRule); ok {
return alertingToAPI(ar)
}

View File

@@ -129,7 +129,7 @@ type Header struct {
}
// UnmarshalYAML unmarshals h from f.
func (h *Header) UnmarshalYAML(f func(interface{}) error) error {
func (h *Header) UnmarshalYAML(f func(any) error) error {
var s string
if err := f(&s); err != nil {
return err
@@ -146,7 +146,7 @@ func (h *Header) UnmarshalYAML(f func(interface{}) error) error {
}
// MarshalYAML marshals h to yaml.
func (h *Header) MarshalYAML() (interface{}, error) {
func (h *Header) MarshalYAML() (any, error) {
return h.sOriginal, nil
}
@@ -201,7 +201,7 @@ type QueryArg struct {
}
// UnmarshalYAML unmarshals qa from yaml.
func (qa *QueryArg) UnmarshalYAML(f func(interface{}) error) error {
func (qa *QueryArg) UnmarshalYAML(f func(any) error) error {
var s string
if err := f(&s); err != nil {
return err
@@ -230,7 +230,7 @@ func (qa *QueryArg) UnmarshalYAML(f func(interface{}) error) error {
}
// MarshalYAML marshals qa to yaml.
func (qa *QueryArg) MarshalYAML() (interface{}, error) {
func (qa *QueryArg) MarshalYAML() (any, error) {
return qa.sOriginal, nil
}
@@ -263,7 +263,7 @@ type URLPrefix struct {
nextDiscoveryDeadline atomic.Uint64
// vOriginal contains the original yaml value for URLPrefix.
vOriginal interface{}
vOriginal any
}
func (up *URLPrefix) setLoadBalancingPolicy(loadBalancingPolicy string) error {
@@ -497,8 +497,8 @@ func getLeastLoadedBackendURL(bus []*backendURL, atomicCounter *atomic.Uint32) *
}
// UnmarshalYAML unmarshals up from yaml.
func (up *URLPrefix) UnmarshalYAML(f func(interface{}) error) error {
var v interface{}
func (up *URLPrefix) UnmarshalYAML(f func(any) error) error {
var v any
if err := f(&v); err != nil {
return err
}
@@ -508,7 +508,7 @@ func (up *URLPrefix) UnmarshalYAML(f func(interface{}) error) error {
switch x := v.(type) {
case string:
urls = []string{x}
case []interface{}:
case []any:
if len(x) == 0 {
return fmt.Errorf("`url_prefix` must contain at least a single url")
}
@@ -538,7 +538,7 @@ func (up *URLPrefix) UnmarshalYAML(f func(interface{}) error) error {
}
// MarshalYAML marshals up to yaml.
func (up *URLPrefix) MarshalYAML() (interface{}, error) {
func (up *URLPrefix) MarshalYAML() (any, error) {
return up.vOriginal, nil
}
@@ -562,7 +562,7 @@ func (r *Regex) match(s string) bool {
}
// UnmarshalYAML implements yaml.Unmarshaler
func (r *Regex) UnmarshalYAML(f func(interface{}) error) error {
func (r *Regex) UnmarshalYAML(f func(any) error) error {
var s string
if err := f(&s); err != nil {
return err
@@ -579,7 +579,7 @@ func (r *Regex) UnmarshalYAML(f func(interface{}) error) error {
}
// MarshalYAML implements yaml.Marshaler.
func (r *Regex) MarshalYAML() (interface{}, error) {
func (r *Regex) MarshalYAML() (any, error) {
return r.sOriginal, nil
}

View File

@@ -12,7 +12,7 @@ import (
type queryValues struct {
name string
values map[string][]interface{}
values map[string][]any
}
func parseResult(r influx.Result) ([]queryValues, error) {
@@ -21,7 +21,7 @@ func parseResult(r influx.Result) ([]queryValues, error) {
}
qValues := make([]queryValues, len(r.Series))
for i, row := range r.Series {
values := make(map[string][]interface{}, len(row.Values))
values := make(map[string][]any, len(row.Values))
for _, value := range row.Values {
for idx, v := range value {
key := row.Columns[idx]
@@ -36,7 +36,7 @@ func parseResult(r influx.Result) ([]queryValues, error) {
return qValues, nil
}
func toFloat64(v interface{}) (float64, error) {
func toFloat64(v any) (float64, error) {
switch i := v.(type) {
case json.Number:
return i.Float64()

View File

@@ -61,7 +61,7 @@ func TestSeries_Unmarshal(t *testing.T) {
}
func TestToFloat64(t *testing.T) {
f := func(in interface{}, want float64) {
f := func(in any, want float64) {
t.Helper()
got, err := toFloat64(in)
if err != nil {

View File

@@ -75,12 +75,6 @@ type TimeRange struct {
type MetaResults struct {
Type string `json:"type"`
Results []Meta `json:"results"`
//metric string
//tags interface{}
//limit int
//time int
//startIndex int
//totalResults int
}
// Meta A meta object about a metric
@@ -88,7 +82,6 @@ type MetaResults struct {
type Meta struct {
Metric string `json:"metric"`
Tags map[string]string `json:"tags"`
//tsuid string
}
// OtsdbMetric is a single series in OpenTSDB's returned format

View File

@@ -45,7 +45,7 @@ type cWriter struct {
err error
}
func (cw *cWriter) printf(format string, args ...interface{}) {
func (cw *cWriter) printf(format string, args ...any) {
if cw.err != nil {
return
}

View File

@@ -176,10 +176,10 @@ func (p *vmNativeProcessor) runBackfilling(ctx context.Context, tenantID string,
}
initMessage := "Initing import process from %q to %q with filter %s"
initParams := []interface{}{srcURL, dstURL, p.filter.String()}
initParams := []any{srcURL, dstURL, p.filter.String()}
if p.interCluster {
initMessage = "Initing import process from %q to %q with filter %s for tenant %s"
initParams = []interface{}{srcURL, dstURL, p.filter.String(), tenantID}
initParams = []any{srcURL, dstURL, p.filter.String(), tenantID}
}
fmt.Println("") // extra line for better output formatting

View File

@@ -253,7 +253,7 @@ func putHistogram(h *histogram.Fast) {
}
var histogramPool = &sync.Pool{
New: func() interface{} {
New: func() any {
return histogram.NewFast()
},
}

View File

@@ -16,7 +16,7 @@ import (
func FunctionsHandler(w http.ResponseWriter, r *http.Request) error {
grouped := httputils.GetBool(r, "grouped")
group := r.FormValue("group")
result := make(map[string]interface{})
result := make(map[string]any)
for funcName, fi := range funcs {
if group != "" && fi.Group != group {
continue
@@ -47,7 +47,7 @@ func FunctionDetailsHandler(funcName string, w http.ResponseWriter, r *http.Requ
return writeJSON(result, w, r)
}
func writeJSON(result interface{}, w http.ResponseWriter, r *http.Request) error {
func writeJSON(result any, w http.ResponseWriter, r *http.Request) error {
data, err := json.Marshal(result)
if err != nil {
return fmt.Errorf("cannot marshal response to JSON: %w", err)

View File

@@ -1968,10 +1968,10 @@ func (h *minSeriesHeap) Swap(i, j int) {
a := *h
a[i], a[j] = a[j], a[i]
}
func (h *minSeriesHeap) Push(x interface{}) {
func (h *minSeriesHeap) Push(x any) {
*h = append(*h, x.(*seriesWithWeight))
}
func (h *minSeriesHeap) Pop() interface{} {
func (h *minSeriesHeap) Pop() any {
a := *h
x := a[len(a)-1]
*h = a[:len(a)-1]
@@ -2499,10 +2499,10 @@ func (h *maxSeriesHeap) Swap(i, j int) {
a := *h
a[i], a[j] = a[j], a[i]
}
func (h *maxSeriesHeap) Push(x interface{}) {
func (h *maxSeriesHeap) Push(x any) {
*h = append(*h, x.(*seriesWithWeight))
}
func (h *maxSeriesHeap) Pop() interface{} {
func (h *maxSeriesHeap) Pop() any {
a := *h
x := a[len(a)-1]
*h = a[:len(a)-1]

View File

@@ -734,11 +734,11 @@ func (sbh *sortBlocksHeap) Swap(i, j int) {
sbs[i], sbs[j] = sbs[j], sbs[i]
}
func (sbh *sortBlocksHeap) Push(x interface{}) {
func (sbh *sortBlocksHeap) Push(x any) {
sbh.sbs = append(sbh.sbs, x.(*sortBlock))
}
func (sbh *sortBlocksHeap) Pop() interface{} {
func (sbh *sortBlocksHeap) Pop() any {
sbs := sbh.sbs
v := sbs[len(sbs)-1]
sbs[len(sbs)-1] = nil
@@ -1084,7 +1084,7 @@ func (xw *exportWork) reset() {
}
var exportWorkPool = &sync.Pool{
New: func() interface{} {
New: func() any {
return &exportWork{}
},
}

View File

@@ -461,7 +461,7 @@ func (xb *exportBlock) reset() {
}
var exportBlockPool = &sync.Pool{
New: func() interface{} {
New: func() any {
return &exportBlock{}
},
}
@@ -1238,7 +1238,7 @@ func (sw *scalableWriter) maybeFlushBuffer(bb *bytesutil.ByteBuffer) error {
}
func (sw *scalableWriter) flush() error {
sw.m.Range(func(_, v interface{}) bool {
sw.m.Range(func(_, v any) bool {
bb := v.(*bytesutil.ByteBuffer)
_, err := sw.bw.Write(bb.B)
return err == nil

View File

@@ -742,13 +742,13 @@ func evalExprsInParallel(qt *querytracer.Tracer, ec *EvalConfig, es []metricsql.
return rvs, nil
}
func evalRollupFuncArgs(qt *querytracer.Tracer, ec *EvalConfig, fe *metricsql.FuncExpr) ([]interface{}, *metricsql.RollupExpr, error) {
func evalRollupFuncArgs(qt *querytracer.Tracer, ec *EvalConfig, fe *metricsql.FuncExpr) ([]any, *metricsql.RollupExpr, error) {
var re *metricsql.RollupExpr
rollupArgIdx := metricsql.GetRollupArgIdx(fe)
if len(fe.Args) <= rollupArgIdx {
return nil, nil, fmt.Errorf("expecting at least %d args to %q; got %d args; expr: %q", rollupArgIdx+1, fe.Name, len(fe.Args), fe.AppendString(nil))
}
args := make([]interface{}, len(fe.Args))
args := make([]any, len(fe.Args))
for i, arg := range fe.Args {
if i == rollupArgIdx {
re = getRollupExprArg(arg)

View File

@@ -956,10 +956,10 @@ func derivValues(values []float64, timestamps []int64) {
values[len(values)-1] = prevDeriv
}
type newRollupFunc func(args []interface{}) (rollupFunc, error)
type newRollupFunc func(args []any) (rollupFunc, error)
func newRollupFuncOneArg(rf rollupFunc) newRollupFunc {
return func(args []interface{}) (rollupFunc, error) {
return func(args []any) (rollupFunc, error) {
if err := expectRollupArgsNum(args, 1); err != nil {
return nil, err
}
@@ -968,7 +968,7 @@ func newRollupFuncOneArg(rf rollupFunc) newRollupFunc {
}
func newRollupFuncTwoArgs(rf rollupFunc) newRollupFunc {
return func(args []interface{}) (rollupFunc, error) {
return func(args []any) (rollupFunc, error) {
if err := expectRollupArgsNum(args, 2); err != nil {
return nil, err
}
@@ -977,7 +977,7 @@ func newRollupFuncTwoArgs(rf rollupFunc) newRollupFunc {
}
func newRollupFuncOneOrTwoArgs(rf rollupFunc) newRollupFunc {
return func(args []interface{}) (rollupFunc, error) {
return func(args []any) (rollupFunc, error) {
if len(args) < 1 || len(args) > 2 {
return nil, fmt.Errorf("unexpected number of args; got %d; want 1...2", len(args))
}
@@ -985,7 +985,7 @@ func newRollupFuncOneOrTwoArgs(rf rollupFunc) newRollupFunc {
}
}
func newRollupHoltWinters(args []interface{}) (rollupFunc, error) {
func newRollupHoltWinters(args []any) (rollupFunc, error) {
if err := expectRollupArgsNum(args, 3); err != nil {
return nil, err
}
@@ -1035,7 +1035,7 @@ func newRollupHoltWinters(args []interface{}) (rollupFunc, error) {
return rf, nil
}
func newRollupPredictLinear(args []interface{}) (rollupFunc, error) {
func newRollupPredictLinear(args []any) (rollupFunc, error) {
if err := expectRollupArgsNum(args, 2); err != nil {
return nil, err
}
@@ -1106,7 +1106,7 @@ func areConstValues(values []float64) bool {
return true
}
func newRollupDurationOverTime(args []interface{}) (rollupFunc, error) {
func newRollupDurationOverTime(args []any) (rollupFunc, error) {
if err := expectRollupArgsNum(args, 2); err != nil {
return nil, err
}
@@ -1136,7 +1136,7 @@ func newRollupDurationOverTime(args []interface{}) (rollupFunc, error) {
return rf, nil
}
func newRollupShareLE(args []interface{}) (rollupFunc, error) {
func newRollupShareLE(args []any) (rollupFunc, error) {
return newRollupAvgFilter(args, countFilterLE)
}
@@ -1150,7 +1150,7 @@ func countFilterLE(values []float64, le float64) float64 {
return float64(n)
}
func newRollupShareGT(args []interface{}) (rollupFunc, error) {
func newRollupShareGT(args []any) (rollupFunc, error) {
return newRollupAvgFilter(args, countFilterGT)
}
@@ -1164,7 +1164,7 @@ func countFilterGT(values []float64, gt float64) float64 {
return float64(n)
}
func newRollupShareEQ(args []interface{}) (rollupFunc, error) {
func newRollupShareEQ(args []any) (rollupFunc, error) {
return newRollupAvgFilter(args, countFilterEQ)
}
@@ -1218,7 +1218,7 @@ func countFilterNE(values []float64, ne float64) float64 {
return float64(n)
}
func newRollupAvgFilter(args []interface{}, f func(values []float64, limit float64) float64) (rollupFunc, error) {
func newRollupAvgFilter(args []any, f func(values []float64, limit float64) float64) (rollupFunc, error) {
rf, err := newRollupFilter(args, f)
if err != nil {
return nil, err
@@ -1229,35 +1229,35 @@ func newRollupAvgFilter(args []interface{}, f func(values []float64, limit float
}, nil
}
func newRollupCountEQ(args []interface{}) (rollupFunc, error) {
func newRollupCountEQ(args []any) (rollupFunc, error) {
return newRollupFilter(args, countFilterEQ)
}
func newRollupCountLE(args []interface{}) (rollupFunc, error) {
func newRollupCountLE(args []any) (rollupFunc, error) {
return newRollupFilter(args, countFilterLE)
}
func newRollupCountGT(args []interface{}) (rollupFunc, error) {
func newRollupCountGT(args []any) (rollupFunc, error) {
return newRollupFilter(args, countFilterGT)
}
func newRollupCountNE(args []interface{}) (rollupFunc, error) {
func newRollupCountNE(args []any) (rollupFunc, error) {
return newRollupFilter(args, countFilterNE)
}
func newRollupSumEQ(args []interface{}) (rollupFunc, error) {
func newRollupSumEQ(args []any) (rollupFunc, error) {
return newRollupFilter(args, sumFilterEQ)
}
func newRollupSumLE(args []interface{}) (rollupFunc, error) {
func newRollupSumLE(args []any) (rollupFunc, error) {
return newRollupFilter(args, sumFilterLE)
}
func newRollupSumGT(args []interface{}) (rollupFunc, error) {
func newRollupSumGT(args []any) (rollupFunc, error) {
return newRollupFilter(args, sumFilterGT)
}
func newRollupFilter(args []interface{}, f func(values []float64, limit float64) float64) (rollupFunc, error) {
func newRollupFilter(args []any, f func(values []float64, limit float64) float64) (rollupFunc, error) {
if err := expectRollupArgsNum(args, 2); err != nil {
return nil, err
}
@@ -1278,7 +1278,7 @@ func newRollupFilter(args []interface{}, f func(values []float64, limit float64)
return rf, nil
}
func newRollupHoeffdingBoundLower(args []interface{}) (rollupFunc, error) {
func newRollupHoeffdingBoundLower(args []any) (rollupFunc, error) {
if err := expectRollupArgsNum(args, 2); err != nil {
return nil, err
}
@@ -1293,7 +1293,7 @@ func newRollupHoeffdingBoundLower(args []interface{}) (rollupFunc, error) {
return rf, nil
}
func newRollupHoeffdingBoundUpper(args []interface{}) (rollupFunc, error) {
func newRollupHoeffdingBoundUpper(args []any) (rollupFunc, error) {
if err := expectRollupArgsNum(args, 2); err != nil {
return nil, err
}
@@ -1338,7 +1338,7 @@ func rollupHoeffdingBoundInternal(rfa *rollupFuncArg, phis []float64) (float64,
return bound, vAvg
}
func newRollupQuantiles(args []interface{}) (rollupFunc, error) {
func newRollupQuantiles(args []any) (rollupFunc, error) {
if len(args) < 3 {
return nil, fmt.Errorf("unexpected number of args: %d; want at least 3 args", len(args))
}
@@ -1405,7 +1405,7 @@ func rollupOutlierIQR(rfa *rollupFuncArg) float64 {
return nan
}
func newRollupQuantile(args []interface{}) (rollupFunc, error) {
func newRollupQuantile(args []any) (rollupFunc, error) {
if err := expectRollupArgsNum(args, 2); err != nil {
return nil, err
}
@@ -1445,7 +1445,7 @@ func mad(values []float64) float64 {
return v
}
func newRollupCountValues(args []interface{}) (rollupFunc, error) {
func newRollupCountValues(args []any) (rollupFunc, error) {
if err := expectRollupArgsNum(args, 2); err != nil {
return nil, err
}
@@ -2389,7 +2389,7 @@ func rollupFake(_ *rollupFuncArg) float64 {
return 0
}
func getScalar(arg interface{}, argNum int) ([]float64, error) {
func getScalar(arg any, argNum int) ([]float64, error) {
ts, ok := arg.([]*timeseries)
if !ok {
return nil, fmt.Errorf(`unexpected type for arg #%d; got %T; want %T`, argNum+1, arg, ts)
@@ -2400,7 +2400,7 @@ func getScalar(arg interface{}, argNum int) ([]float64, error) {
return ts[0].Values, nil
}
func getIntNumber(arg interface{}, argNum int) (int, error) {
func getIntNumber(arg any, argNum int) (int, error) {
v, err := getScalar(arg, argNum)
if err != nil {
return 0, err
@@ -2425,7 +2425,7 @@ func getString(tss []*timeseries, argNum int) (string, error) {
return string(ts.MetricName.MetricGroup), nil
}
func expectRollupArgsNum(args []interface{}, expectedNum int) error {
func expectRollupArgsNum(args []any, expectedNum int) error {
if len(args) == expectedNum {
return nil
}

View File

@@ -200,7 +200,7 @@ func TestDerivValues(t *testing.T) {
testRowsEqual(t, values, timestamps, valuesExpected, timestamps)
}
func testRollupFunc(t *testing.T, funcName string, args []interface{}, vExpected float64) {
func testRollupFunc(t *testing.T, funcName string, args []any, vExpected float64) {
t.Helper()
nrf := getRollupFunc(funcName)
if nrf == nil {
@@ -245,7 +245,7 @@ func TestRollupDurationOverTime(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{&metricsql.RollupExpr{Expr: &me}, maxIntervals}
args := []any{&metricsql.RollupExpr{Expr: &me}, maxIntervals}
testRollupFunc(t, "duration_over_time", args, dExpected)
}
f(-123, 0)
@@ -266,7 +266,7 @@ func TestRollupShareLEOverTime(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{&metricsql.RollupExpr{Expr: &me}, les}
args := []any{&metricsql.RollupExpr{Expr: &me}, les}
testRollupFunc(t, "share_le_over_time", args, vExpected)
}
@@ -289,7 +289,7 @@ func TestRollupShareGTOverTime(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{&metricsql.RollupExpr{Expr: &me}, gts}
args := []any{&metricsql.RollupExpr{Expr: &me}, gts}
testRollupFunc(t, "share_gt_over_time", args, vExpected)
}
@@ -312,7 +312,7 @@ func TestRollupShareEQOverTime(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{&metricsql.RollupExpr{Expr: &me}, eqs}
args := []any{&metricsql.RollupExpr{Expr: &me}, eqs}
testRollupFunc(t, "share_eq_over_time", args, vExpected)
}
@@ -331,7 +331,7 @@ func TestRollupCountLEOverTime(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{&metricsql.RollupExpr{Expr: &me}, les}
args := []any{&metricsql.RollupExpr{Expr: &me}, les}
testRollupFunc(t, "count_le_over_time", args, vExpected)
}
@@ -354,7 +354,7 @@ func TestRollupCountGTOverTime(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{&metricsql.RollupExpr{Expr: &me}, gts}
args := []any{&metricsql.RollupExpr{Expr: &me}, gts}
testRollupFunc(t, "count_gt_over_time", args, vExpected)
}
@@ -377,7 +377,7 @@ func TestRollupCountEQOverTime(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{&metricsql.RollupExpr{Expr: &me}, eqs}
args := []any{&metricsql.RollupExpr{Expr: &me}, eqs}
testRollupFunc(t, "count_eq_over_time", args, vExpected)
}
@@ -396,7 +396,7 @@ func TestRollupCountNEOverTime(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{&metricsql.RollupExpr{Expr: &me}, nes}
args := []any{&metricsql.RollupExpr{Expr: &me}, nes}
testRollupFunc(t, "count_ne_over_time", args, vExpected)
}
@@ -415,7 +415,7 @@ func TestRollupSumLEOverTime(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{&metricsql.RollupExpr{Expr: &me}, les}
args := []any{&metricsql.RollupExpr{Expr: &me}, les}
testRollupFunc(t, "sum_le_over_time", args, vExpected)
}
@@ -438,7 +438,7 @@ func TestRollupSumGTOverTime(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{&metricsql.RollupExpr{Expr: &me}, les}
args := []any{&metricsql.RollupExpr{Expr: &me}, les}
testRollupFunc(t, "sum_gt_over_time", args, vExpected)
}
@@ -461,7 +461,7 @@ func TestRollupSumEQOverTime(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{&metricsql.RollupExpr{Expr: &me}, les}
args := []any{&metricsql.RollupExpr{Expr: &me}, les}
testRollupFunc(t, "sum_eq_over_time", args, vExpected)
}
@@ -484,7 +484,7 @@ func TestRollupQuantileOverTime(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{phis, &metricsql.RollupExpr{Expr: &me}}
args := []any{phis, &metricsql.RollupExpr{Expr: &me}}
testRollupFunc(t, "quantile_over_time", args, vExpected)
}
@@ -506,7 +506,7 @@ func TestRollupPredictLinear(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{&metricsql.RollupExpr{Expr: &me}, secs}
args := []any{&metricsql.RollupExpr{Expr: &me}, secs}
testRollupFunc(t, "predict_linear", args, vExpected)
}
@@ -545,7 +545,7 @@ func TestRollupHoltWinters(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{&metricsql.RollupExpr{Expr: &me}, sfs, tfs}
args := []any{&metricsql.RollupExpr{Expr: &me}, sfs, tfs}
testRollupFunc(t, "holt_winters", args, vExpected)
}
@@ -573,7 +573,7 @@ func TestRollupHoeffdingBoundLower(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{phis, &metricsql.RollupExpr{Expr: &me}}
args := []any{phis, &metricsql.RollupExpr{Expr: &me}}
testRollupFunc(t, "hoeffding_bound_lower", args, vExpected)
}
@@ -594,7 +594,7 @@ func TestRollupHoeffdingBoundUpper(t *testing.T) {
Timestamps: []int64{123},
}}
var me metricsql.MetricExpr
args := []interface{}{phis, &metricsql.RollupExpr{Expr: &me}}
args := []any{phis, &metricsql.RollupExpr{Expr: &me}}
testRollupFunc(t, "hoeffding_bound_upper", args, vExpected)
}
@@ -611,7 +611,7 @@ func TestRollupNewRollupFuncSuccess(t *testing.T) {
f := func(funcName string, vExpected float64) {
t.Helper()
var me metricsql.MetricExpr
args := []interface{}{&metricsql.RollupExpr{Expr: &me}}
args := []any{&metricsql.RollupExpr{Expr: &me}}
testRollupFunc(t, funcName, args, vExpected)
}
@@ -668,7 +668,7 @@ func TestRollupNewRollupFuncError(t *testing.T) {
t.Fatalf("expecting nil func; got %p", nrf)
}
f := func(funcName string, args []interface{}) {
f := func(funcName string, args []any) {
t.Helper()
nrf := getRollupFunc(funcName)
@@ -694,13 +694,13 @@ func TestRollupNewRollupFuncError(t *testing.T) {
Timestamps: []int64{123},
}}
me := &metricsql.MetricExpr{}
f("holt_winters", []interface{}{123, 123, 321})
f("holt_winters", []interface{}{me, 123, 321})
f("holt_winters", []interface{}{me, scalarTs, 321})
f("predict_linear", []interface{}{123, 123})
f("predict_linear", []interface{}{me, 123})
f("quantile_over_time", []interface{}{123, 123})
f("quantiles_over_time", []interface{}{123, 123})
f("holt_winters", []any{123, 123, 321})
f("holt_winters", []any{me, 123, 321})
f("holt_winters", []any{me, scalarTs, 321})
f("predict_linear", []any{123, 123})
f("predict_linear", []any{me, 123})
f("quantile_over_time", []any{123, 123})
f("quantiles_over_time", []any{123, 123})
}
func TestRollupNoWindowNoPoints(t *testing.T) {

View File

@@ -6,7 +6,7 @@ COPY web/ /build/
RUN GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o web-amd64 github.com/VictoriMetrics/vmui/ && \
GOOS=windows GOARCH=amd64 CGO_ENABLED=0 go build -o web-windows github.com/VictoriMetrics/vmui/
FROM alpine:3.20.0
FROM alpine:3.20.1
USER root
COPY --from=build-web-stage /build/web-amd64 /app/web

View File

@@ -48,9 +48,8 @@ const ExploreLogs: FC = () => {
const newPeriod = getPeriod();
setPeriod(newPeriod);
fetchLogs(newPeriod).then(() => {
fetchLogHits(newPeriod);
}).catch(e => e);
fetchLogs(newPeriod);
fetchLogHits(newPeriod);
setSearchParamsFromKeys( {
query,
@@ -95,9 +94,10 @@ const ExploreLogs: FC = () => {
{error && <Alert variant="error">{error}</Alert>}
{!error && (
<ExploreLogsBarChart
{...dataLogHits}
query={query}
period={period}
{...dataLogHits}
isLoading={isLoading ? false : dataLogHits.isLoading}
/>
)}
<ExploreLogsBody

View File

@@ -9,6 +9,7 @@ import { AlignedData } from "uplot";
import BarHitsChart from "../../../components/Chart/BarHitsChart/BarHitsChart";
import Alert from "../../../components/Main/Alert/Alert";
import { TimeParams } from "../../../types";
import Spinner from "../../../components/Main/Spinner/Spinner";
interface Props {
query: string;
@@ -18,7 +19,7 @@ interface Props {
isLoading: boolean;
}
const ExploreLogsBarChart: FC<Props> = ({ logHits, period, error }) => {
const ExploreLogsBarChart: FC<Props> = ({ logHits, period, error, isLoading }) => {
const { isMobile } = useDeviceDetect();
const timeDispatch = useTimeDispatch();
@@ -56,6 +57,7 @@ const ExploreLogsBarChart: FC<Props> = ({ logHits, period, error }) => {
"vm-block_mobile": isMobile,
})}
>
{isLoading && <Spinner containerStyles={{ position: "absolute" }}/>}
{!error && noDataMessage && (
<div className="vm-explore-logs-chart__empty">
<Alert variant="info">{noDataMessage}</Alert>

View File

@@ -7,7 +7,7 @@ import { LOGS_BARS_VIEW } from "../../../constants/logs";
export const useFetchLogHits = (server: string, query: string) => {
const [logHits, setLogHits] = useState<LogHits[]>([]);
const [isLoading, setIsLoading] = useState(false);
const [isLoading, setIsLoading] = useState<{[key: number]: boolean;}>([]);
const [error, setError] = useState<ErrorTypes | string>();
const abortControllerRef = useRef(new AbortController());
@@ -36,7 +36,8 @@ export const useFetchLogHits = (server: string, query: string) => {
abortControllerRef.current = new AbortController();
const { signal } = abortControllerRef.current;
setIsLoading(true);
const id = Date.now();
setIsLoading(prev => ({ ...prev, [id]: true }));
setError(undefined);
try {
@@ -47,8 +48,8 @@ export const useFetchLogHits = (server: string, query: string) => {
const text = await response.text();
setError(text);
setLogHits([]);
setIsLoading(false);
return Promise.reject(new Error(text));
setIsLoading(prev => ({ ...prev, [id]: false }));
return;
}
const data = await response.json();
@@ -56,25 +57,22 @@ export const useFetchLogHits = (server: string, query: string) => {
if (!hits) {
const error = "Error: No 'hits' field in response";
setError(error);
return Promise.reject(new Error(error));
}
setLogHits(hits);
setLogHits(!hits ? [] : hits);
} catch (e) {
if (e instanceof Error && e.name !== "AbortError") {
setError(String(e));
console.error(e);
setLogHits([]);
}
return Promise.reject(e);
} finally {
setIsLoading(false);
}
setIsLoading(prev => ({ ...prev, [id]: false }));
}, [url, query]);
return {
logHits,
isLoading,
isLoading: Object.values(isLoading).some(s => s),
error,
fetchLogHits,
};

View File

@@ -6,7 +6,7 @@ import dayjs from "dayjs";
export const useFetchLogs = (server: string, query: string, limit: number) => {
const [logs, setLogs] = useState<Logs[]>([]);
const [isLoading, setIsLoading] = useState(false);
const [isLoading, setIsLoading] = useState<{[key: number]: boolean;}>([]);
const [error, setError] = useState<ErrorTypes | string>();
const abortControllerRef = useRef(new AbortController());
@@ -39,7 +39,8 @@ export const useFetchLogs = (server: string, query: string, limit: number) => {
abortControllerRef.current = new AbortController();
const { signal } = abortControllerRef.current;
setIsLoading(true);
const id = Date.now();
setIsLoading(prev => ({ ...prev, [id]: true }));
setError(undefined);
try {
@@ -50,8 +51,8 @@ export const useFetchLogs = (server: string, query: string, limit: number) => {
if (!response.ok || !response.body) {
setError(text);
setLogs([]);
setIsLoading(false);
return Promise.reject(new Error(text));
setIsLoading(prev => ({ ...prev, [id]: false }));
return;
}
const lines = text.split("\n").filter(line => line).slice(0, limit);
@@ -63,16 +64,13 @@ export const useFetchLogs = (server: string, query: string, limit: number) => {
console.error(e);
setLogs([]);
}
return Promise.reject(e);
} finally {
setIsLoading(false);
}
setIsLoading(false);
setIsLoading(prev => ({ ...prev, [id]: false }));
}, [url, query, limit]);
return {
logs,
isLoading,
isLoading: Object.values(isLoading).some(s => s),
error,
fetchLogs,
};

View File

@@ -3,10 +3,10 @@
DOCKER_REGISTRY ?= docker.io
DOCKER_NAMESPACE ?= victoriametrics
ROOT_IMAGE ?= alpine:3.20.0
ROOT_IMAGE ?= alpine:3.20.1
ROOT_IMAGE_SCRATCH ?= scratch
SKIP_SCRATCH_BUILD ?= false
CERTS_IMAGE := alpine:3.20.0
CERTS_IMAGE := alpine:3.20.1
GO_BUILDER_IMAGE := golang:1.22.5-alpine
BUILDER_IMAGE := local/builder:2.0.0-$(shell echo $(GO_BUILDER_IMAGE) | tr :/ __)-1

View File

@@ -40,7 +40,7 @@ services:
# storing logs and serving read queries.
victorialogs:
container_name: victorialogs
image: docker.io/victoriametrics/victoria-logs:v0.27.1-victorialogs
image: docker.io/victoriametrics/victoria-logs:v0.28.0-victorialogs
command:
- "--storageDataPath=/vlogs"
- "--httpListenAddr=:9428"

View File

@@ -22,7 +22,7 @@ services:
- -beat.uri=http://filebeat-victorialogs:5066
victorialogs:
image: docker.io/victoriametrics/victoria-logs:v0.27.1-victorialogs
image: docker.io/victoriametrics/victoria-logs:v0.28.0-victorialogs
volumes:
- victorialogs-filebeat-docker-vl:/vlogs
ports:

View File

@@ -13,7 +13,7 @@ services:
- "5140:5140"
victorialogs:
image: docker.io/victoriametrics/victoria-logs:v0.27.1-victorialogs
image: docker.io/victoriametrics/victoria-logs:v0.28.0-victorialogs
volumes:
- victorialogs-filebeat-syslog-vl:/vlogs
ports:

View File

@@ -11,7 +11,7 @@ services:
- "5140:5140"
victorialogs:
image: docker.io/victoriametrics/victoria-logs:v0.27.1-victorialogs
image: docker.io/victoriametrics/victoria-logs:v0.28.0-victorialogs
volumes:
- victorialogs-fluentbit-vl:/vlogs
ports:

View File

@@ -14,7 +14,7 @@ services:
- "5140:5140"
victorialogs:
image: docker.io/victoriametrics/victoria-logs:v0.27.1-victorialogs
image: docker.io/victoriametrics/victoria-logs:v0.28.0-victorialogs
volumes:
- victorialogs-logstash-vl:/vlogs
ports:

View File

@@ -12,7 +12,7 @@ services:
- "5140:5140"
vlogs:
image: docker.io/victoriametrics/victoria-logs:v0.27.1-victorialogs
image: docker.io/victoriametrics/victoria-logs:v0.28.0-victorialogs
volumes:
- victorialogs-promtail-docker:/vlogs
ports:

View File

@@ -22,7 +22,7 @@ services:
condition: service_healthy
victorialogs:
image: docker.io/victoriametrics/victoria-logs:v0.27.1-victorialogs
image: docker.io/victoriametrics/victoria-logs:v0.28.0-victorialogs
volumes:
- victorialogs-vector-docker-vl:/vlogs
ports:

View File

@@ -3,7 +3,7 @@ version: '3'
services:
# Run `make package-victoria-logs` to build victoria-logs image
vlogs:
image: docker.io/victoriametrics/victoria-logs:v0.27.1-victorialogs
image: docker.io/victoriametrics/victoria-logs:v0.28.0-victorialogs
volumes:
- vlogs:/vlogs
ports:

View File

@@ -32,6 +32,7 @@ See also [LTS releases](https://docs.victoriametrics.com/lts-releases/).
**Update note 1: support for snap packages was removed due to lack of interest from community. See this [pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6543) for details. Please read about supported package types [here](https://docs.victoriametrics.com/#install).**
* SECURITY: upgrade Go builder from Go1.22.4 to Go1.22.5. See the list of issues addressed in [Go1.22.5](https://github.com/golang/go/issues?q=milestone%3AGo1.22.5+label%3ACherryPickApproved).
* SECURITY: upgrade base docker image (Alpine) from 3.20.0 to 3.20.1. See [alpine 3.20.1 release notes](https://www.alpinelinux.org/posts/Alpine-3.20.1-released.html).
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth/): allow overriding `Host` header with a target host before sending to a downstream. See this [issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6453)
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth/): reduces CPU usage by reusing request body buffer. Allows to disable requests caching with `-maxRequestBodySizeToRetry=0`. See this [PR](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6533) for details.

View File

@@ -11,7 +11,11 @@ aliases:
---
# Cluster version
<img src="logo.webp" width="300">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="logo_white.webp">
<source media="(prefers-color-scheme: light)" srcset="logo.webp">
<img width="300" alt="VictoriaMetrics logo" src="logo.webp">
</picture>
VictoriaMetrics is a fast, cost-effective and scalable time series database. It can be used as a long-term remote storage for Prometheus.

View File

@@ -41,7 +41,7 @@ and [Grafana setup](https://docs.victoriametrics.com/single-server-victoriametri
VictoriaMetrics is developed at a fast pace, so it is recommended periodically checking the [CHANGELOG](https://docs.victoriametrics.com/changelog/) and performing [regular upgrades](https://docs.victoriametrics.com/#how-to-upgrade-victoriametrics).
### Starting VictoriaMetrics Single via Docker
### Starting VM-Single via Docker
The following commands download the latest available
[Docker image of VictoriaMetrics](https://hub.docker.com/r/victoriametrics/victoria-metrics)
@@ -61,7 +61,7 @@ and read [these docs](https://docs.victoriametrics.com/#operation).
There is also [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/)
- horizontally scalable installation, which scales to multiple nodes.
### Starting VictoriaMetrics Cluster via Docker
### Starting VM-Cluster via Docker
The following commands clone the latest available
[VictoriaMetrics repository](https://github.com/VictoriaMetrics/VictoriaMetrics)
@@ -81,38 +81,32 @@ See more details [here](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/
* [Cluster setup](https://docs.victoriametrics.com/cluster-victoriametrics/#cluster-setup)
### Starting VictoriaMetrics Single from a Binary
### Starting VM-Single from a Binary
1. Download the correct archive from [github](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
1. Download the correct binary for your OS and architecture from [GitHub](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
For Enterprise binaries see [this link](https://docs.victoriametrics.com/enterprise/#binary-releases).
For open source it will look like
2. Extract the archive to /usr/local/bin by running:
```sh
sudo tar -xvf <victoriametrics-archive> -C /usr/local/bin
```
Replace `<victoriametrics-archive>` with the path to the archive you downloaded in step 1.
```victoria-metrics-<os>-<architecture>-v<version>.tar.gz```
3. Create a VictoriaMetrics user on the system:
For Enterprise it will look like
```sh
sudo useradd -s /usr/sbin/nologin victoriametrics
```
`victoria-metrics-<os>-<architecture>-v<version>-enterprise.tar.gz`
4. Create a folder for storing VictoriaMetrics data:
In order for VictoriaMetrics Enterprise to start the, the -license flag must be set equal to a valid VictoriaMetrics key or the -licenseFile flag needs to point to a file containing your VictoriaMetrics license. Please, see more information about license configuration [here](https://docs.victoriametrics.com/enterprise/#binary-releases).
```sh
mkdir -p /var/lib/victoria-metrics && chown -R victoriametrics:victoriametrics /var/lib/victoria-metrics
```
2. Extract the archive to /usr/local/bin by running
5. Create a Linux Service by running the following:
`sudo tar -xvf <victoriametrics-archive> -C /usr/local/bin`
Replace victoriametrics-archive with the path to the archive you downloaded in step 1
3. Create a VictoriaMetrics user on the system
`sudo useradd -s /usr/sbin/nologin victoriametrics`
4. Create a folder for storing VictoriaMetrics data
`mkdir -p /var/lib/victoria-metrics && chown -R victoriametrics:victoriametrics /var/lib/victoria-metrics`
5. Create a Linux Service by running the following
```bash
```sh
cat <<END >/etc/systemd/system/victoriametrics.service
[Unit]
Description=VictoriaMetrics service
@@ -137,59 +131,64 @@ WantedBy=multi-user.target
END
```
If you want to deploy VictoriaMetrics single as a Windows Service review the [running as a windows service docs](https://docs.victoriametrics.com/single-server-victoriametrics/#running-as-windows-service)
Extra [command-line flags](https://docs.victoriametrics.com/#list-of-command-line-flags) can be added to `ExecStart` line.
6. Adjust the command line flags in the `ExecStart` line to fit your needs.
If you want to deploy VictoriaMetrics single as a Windows Service review the [running as a Windows service docs](https://docs.victoriametrics.com/single-server-victoriametrics/#running-as-windows-service).
The list of command line flags for VictoriaMetrics can be found [here](https://docs.victoriametrics.com/single-server-victoriametrics/#list-of-command-line-flags)
> Please note, `victoriametrics` service is listening on `:8428` for HTTP connections (see `-httpListenAddr` flag).
7. Start and enable the service by running
6. Start and enable the service by running the following command:
`sudo systemctl daemon-reload && sudo systemctl enable --now victoriametrics.service`
```sh
sudo systemctl daemon-reload && sudo systemctl enable --now victoriametrics.service
```
8. Check the that service started successfully
7. Check that service started successfully:
`sudo systemctl status victoriametrics.service`
```sh
sudo systemctl status victoriametrics.service
```
9. After VictoriaMetrics is Running verify VMUI is working by going to `http://<ip_or_hostname>:8428/vmui`
8. After VictoriaMetrics is in `Running` state, verify [vmui](https://docs.victoriametrics.com/#vmui) is working
by going to `http://<ip_or_hostname>:8428/vmui`.
### Starting VictoriaMetrics Cluster from Binaries
### Starting VM-Cluster from Binaries
On all nodes you will need to do the following
VictoriaMetrics cluster consists of [3 components](https://docs.victoriametrics.com/cluster-victoriametrics/#architecture-overview).
It is recommended to run these components in the same private network (for [security reasons](https://docs.victoriametrics.com/#security)),
but on the separate physical nodes for the best performance.
1. Download the archive that matches your operating system and processor architecture from [github releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
On all nodes you will need to do the following:
For open source it will look like
1. Download the correct binary for your OS and architecture with `-cluster` suffix from [GitHub](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
For Enterprise binaries see [this link](https://docs.victoriametrics.com/enterprise/#binary-releases).
`victoria-metrics-<os>-<architecture>-v<version>-cluster.tar.gz`
2. Extract the archive to /usr/local/bin by running:
For Enterprise versions of VictoriaMetrics the ar will look like
```sh
sudo tar -xvf <victoriametrics-archive> -C /usr/local/bin
```
`victoria-metrics-<os>-<architecture>-v<version>-enterprise-cluster.tar.gz`
Replace `<victoriametrics-archive>` with the path to the archive you downloaded in step 1
In order for VictoriaMetrics Enterprise to start the, the -license flag must be set equal to a valid VictoriaMetrics key or the -licenseFile flag needs to point to a file containing your VictoriaMetrics license. Please, see more information about license configuration [here](https://docs.victoriametrics.com/enterprise/#binary-releases).
3. Create a user account for VictoriaMetrics:
```sh
sudo useradd -s /usr/sbin/nologin victoriametrics
```
2. Extract the archive to /usr/local/bin by running
See recommendations for installing each type of [cluster component](https://docs.victoriametrics.com/cluster-victoriametrics/#architecture-overview) below.
`sudo tar -xvf <victoriametrics-archive> -C /usr/local/bin`
##### Installing vmstorage
Replace victoriametrics-archive with the path to the archive you downloaded in step 1
3. Create a user account for VictoriaMetrics
`sudo useradd -s /usr/sbin/nologin victoriametrics`
##### vmstorage
1. Create a folder for storing VictoriaMetrics data
1. Create a folder for storing `vmstorage` data:
`mkdir -p /var/lib/vmstorage && chown -R victoriametrics:victoriametrics /var/lib/vmstorage`
2. Create a Linux Service for `vmstorage` service by running
2. Create a Linux Service for `vmstorage` service by running the following command:
```bash
```sh
cat <<END >/etc/systemd/system/vmstorage.service
[Unit]
Description=VictoriaMetrics vmstorage service
@@ -211,30 +210,32 @@ WantedBy=multi-user.target
END
```
3. Adjust the command line flags in the ExecStart line to fit your needs.
Extra [command-line flags](https://docs.victoriametrics.com/cluster-victoriametrics/#list-of-command-line-flags-for-vmstorage)
for vmstorage can be added to `ExecStart` line.
The list of command line flags for `vmstorage` can be found [here](https://docs.victoriametrics.com/cluster-victoriametrics/#list-of-command-line-flags-for-vmstorage)
> Please note, `vmstorage` service is listening on `:8400` for vminsert connections (see `-vminsertAddr` flag),
> on `:8401` for vmselect connections (see `--vmselectAddr` flag) and on `:8482` for HTTP connections (see `-httpListenAddr` flag).
4. Start and Enable `vmstorage`
3. Start and Enable `vmstorage`:
`sudo systemctl daemon-reload && systemctl enable --now vmstorage`
```sh
sudo systemctl daemon-reload && systemctl enable --now vmstorage
```
5. Check that the service is running
4. Check that service started successfully:
`sudo systemctl status vmstorage`
```sh
sudo systemctl status vmstorage
```
6. After `vmstorage` is running confirm the service healthy by going to
5. After `vmstorage` is in `Running` state, confirm the service is healthy by visiting `http://<ip_or_hostname>:8482/-/healthy` link.
It should say "VictoriaMetrics is Healthy".
##### Installing vminsert
`http://<ip_or_hostname>:8482/-/healthy`
1. Create a Linux Service for `vminsert` by running the following command:
It should say "VictoriaMetrics is Healthy"
##### vminsert
1. Create a Linux Service for `vminsert` by running
```bash
```sh
cat << END >/etc/systemd/system/vminsert.service
[Unit]
Description=VictoriaMetrics vminsert service
@@ -245,7 +246,7 @@ Type=simple
User=victoriametrics
Group=victoriametrics
Restart=always
ExecStart=/usr/local/bin/vminsert-prod -replicationFactor=1 -storageNode=localhost
ExecStart=/usr/local/bin/vminsert-prod -storageNode=<list of vmstorages>
PrivateTmp=yes
NoNewPrivileges=yes
@@ -256,27 +257,34 @@ WantedBy=multi-user.target
END
```
2. Adjust the command line flags in the ExecStart line to fit your needs.
Replace `<list of vmstorages>` with addresses of previously configured `vmstorage` services.
To specify multiple addresses you can repeat the flag multiple times, or separate addresses with commas
in one flag. See more details in `-storageNode` flag description [here](https://docs.victoriametrics.com/cluster-victoriametrics/#list-of-command-line-flags-for-vminsert).
The list of command line flags for `vminsert` can be found [here](https://docs.victoriametrics.com/cluster-victoriametrics/#list-of-command-line-flags-for-vminsert).
> Please note, `vminsert` service is listening on `:8480` for HTTP connections (see `-httpListenAddr` flag).
3. Start and enable `vminsert`
2. Start and Enable `vminsert`:
`sudo systemctl daemon-reload && sudo systemctl enable --now vminsert.service`
```sh
sudo systemctl daemon-reload && sudo systemctl enable --now vminsert.service
```
4. Make sure `vminsert` is running
3. Check that service started successfully:
`sudo systemctl status vminsert.service`
5. After `vminsert` is started you can confirm that is healthy by going to
`http://<ip_or_hostname>:8480/-/healthy`
```sh
sudo systemctl status vminsert.service
```
4. After `vminsert` is in `Running` state, confirm the service is healthy by visiting `http://<ip_or_hostname>:8480/-/healthy` link.
It should say "VictoriaMetrics is Healthy"
##### vmselect
##### Installing vmselect
1. Create a folder to store query cache data `sudo mkdir -p /var/lib/vmselect-cache && sudo chown -R victoriametrics:victoriametrics /var/lib/vmselect-cache`
1. Create a folder to store temporary cache:
```sh
sudo mkdir -p /var/lib/vmselect-cache && sudo chown -R victoriametrics:victoriametrics /var/lib/vmselect-cache
```
2. Add a Linux Service for `vmselect` by running
@@ -291,7 +299,7 @@ Type=simple
User=victoriametrics
Group=victoriametrics
Restart=always
ExecStart=/usr/local/bin/vmselect-prod -storageNode localhost -cacheDataPath=/var/lib/vmselect-cache
ExecStart=/usr/local/bin/vmselect-prod -storageNode=<list of vmstorages> -cacheDataPath=/var/lib/vmselect-cache
PrivateTmp=yes
NoNewPrivileges=yes
@@ -303,21 +311,26 @@ WantedBy=multi-user.target
END
```
3. Adjust the command line flags in the ExecStart line to fit your needs.
Replace `<list of vmstorages>` with addresses of previously configured `vmstorage` services.
To specify multiple addresses you can repeat the flag multiple times, or separate addresses with commas
in one flag. See more details in `-storageNode` flag description [here](https://docs.victoriametrics.com/cluster-victoriametrics/#list-of-command-line-flags-for-vminsert).
The list of command line flags for `vmselect` can be found [here](https://docs.victoriametrics.com/cluster-victoriametrics/#list-of-command-line-flags-for-vmselect)
> Please note, `vmselect` service is listening on `:8481` for HTTP connections (see `-httpListenAddr` flag).
4. Start and enable `vmselect`
3. Start and Enable `vmselect`:
`sudo systemctl daemon-reload && sudo systemctl enable --now vmselect.service`
```sh
sudo systemctl daemon-reload && sudo systemctl enable --now vmselect.service
```
5. Make sure the `vmselect` service is running
4. Make sure the `vmselect` service is running:
`sudo systemctl status vmselect.service`
```sh
sudo systemctl status vmselect.service
```
6. After `vmselect` is running you can verify it is working by going to VMUI located at
`http://<ip_or_hostname>:8481/select/vmui/vmui/`
5. After `vmselect` is in `Running` state, confirm the service is healthy by visiting `http://<ip_or_hostname>:8481/select/0/vmui` link.
It should open [vmui](https://docs.victoriametrics.com/#vmui) page.
## Write data

View File

@@ -12,7 +12,11 @@ title: VictoriaMetrics
[![Build Status](https://github.com/VictoriaMetrics/VictoriaMetrics/workflows/main/badge.svg)](https://github.com/VictoriaMetrics/VictoriaMetrics/actions)
[![codecov](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/branch/master/graph/badge.svg)](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics)
<img src="logo.webp" width="300" alt="VictoriaMetrics logo">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="logo_white.webp">
<source media="(prefers-color-scheme: light)" srcset="logo.webp">
<img width="300" alt="VictoriaMetrics logo" src="logo.webp">
</picture>
VictoriaMetrics is a fast, cost-effective and scalable monitoring solution and time series database.
See [case studies for VictoriaMetrics](https://docs.victoriametrics.com/casestudies/).

View File

@@ -20,7 +20,11 @@ aliases:
[![Build Status](https://github.com/VictoriaMetrics/VictoriaMetrics/workflows/main/badge.svg)](https://github.com/VictoriaMetrics/VictoriaMetrics/actions)
[![codecov](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/branch/master/graph/badge.svg)](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics)
<img src="logo.webp" width="300" alt="VictoriaMetrics logo">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="logo_white.webp">
<source media="(prefers-color-scheme: light)" srcset="logo.webp">
<img width="300" alt="VictoriaMetrics logo" src="logo.webp">
</picture>
VictoriaMetrics is a fast, cost-effective and scalable monitoring solution and time series database.
See [case studies for VictoriaMetrics](https://docs.victoriametrics.com/casestudies/).

View File

@@ -19,6 +19,14 @@ according to [these docs](https://docs.victoriametrics.com/victorialogs/quicksta
## tip
## [v0.28.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.28.0-victorialogs)
Released at 2024-07-10
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): show a spinner on top of bar chart until user's request is finished. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6558).
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): use compact representation of JSON lines at `JSON` tab if only a single [log field](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) is queried. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6559).
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): properly show the number of matching logs on the selected time range at bar chart for queries with arbitrary [pipes](https://docs.victoriametrics.com/victorialogs/logsql/#pipes), including [`stats` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe) and [`top` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#top-pipe).
## [v0.27.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.27.1-victorialogs)
Released at 2024-07-05

View File

@@ -36,8 +36,8 @@ Just download archive for the needed Operating system and architecture, unpack i
For example, the following commands download VictoriaLogs archive for Linux/amd64, unpack and run it:
```sh
curl -L -O https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v0.27.1-victorialogs/victoria-logs-linux-amd64-v0.27.1-victorialogs.tar.gz
tar xzf victoria-logs-linux-amd64-v0.27.1-victorialogs.tar.gz
curl -L -O https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v0.28.0-victorialogs/victoria-logs-linux-amd64-v0.28.0-victorialogs.tar.gz
tar xzf victoria-logs-linux-amd64-v0.28.0-victorialogs.tar.gz
./victoria-logs-prod
```
@@ -61,7 +61,7 @@ Here is the command to run VictoriaLogs in a Docker container:
```sh
docker run --rm -it -p 9428:9428 -v ./victoria-logs-data:/victoria-logs-data \
docker.io/victoriametrics/victoria-logs:v0.27.1-victorialogs
docker.io/victoriametrics/victoria-logs:v0.28.0-victorialogs
```
See also:

BIN
docs/logo_white.webp Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.1 KiB

View File

@@ -65,7 +65,7 @@ func (c *Cache) MustStop() {
}
// RemoveBlocksForPart removes all the blocks for the given part from the cache.
func (c *Cache) RemoveBlocksForPart(p interface{}) {
func (c *Cache) RemoveBlocksForPart(p any) {
for _, shard := range c.shards {
shard.RemoveBlocksForPart(p)
}
@@ -185,7 +185,7 @@ type cache struct {
mu sync.Mutex
// m contains cached blocks keyed by Key.Part and then by Key.Offset
m map[interface{}]map[uint64]*cacheEntry
m map[any]map[uint64]*cacheEntry
// perKeyMisses contains per-block cache misses.
//
@@ -199,7 +199,7 @@ type cache struct {
// Key represents a key, which uniquely identifies the Block.
type Key struct {
// Part must contain a pointer to part structure where the block belongs to.
Part interface{}
Part any
// Offset is the offset of the block in the part.
Offset uint64
@@ -233,12 +233,12 @@ type cacheEntry struct {
func newCache(getMaxSizeBytes func() int) *cache {
var c cache
c.getMaxSizeBytes = getMaxSizeBytes
c.m = make(map[interface{}]map[uint64]*cacheEntry)
c.m = make(map[any]map[uint64]*cacheEntry)
c.perKeyMisses = make(map[Key]int)
return &c
}
func (c *cache) RemoveBlocksForPart(p interface{}) {
func (c *cache) RemoveBlocksForPart(p any) {
c.mu.Lock()
defer c.mu.Unlock()
@@ -398,13 +398,13 @@ func (lah *lastAccessHeap) Less(i, j int) bool {
h := *lah
return h[i].lastAccessTime < h[j].lastAccessTime
}
func (lah *lastAccessHeap) Push(x interface{}) {
func (lah *lastAccessHeap) Push(x any) {
e := x.(*cacheEntry)
h := *lah
e.heapIdx = len(h)
*lah = append(h, e)
}
func (lah *lastAccessHeap) Pop() interface{} {
func (lah *lastAccessHeap) Pop() any {
h := *lah
e := h[len(h)-1]

View File

@@ -27,7 +27,7 @@ func TestCache(t *testing.T) {
t.Fatalf("unexpected SizeMaxBytes(); got %d; want %d", n, sizeMaxBytes)
}
offset := uint64(1234)
part := (interface{})("foobar")
part := (any)("foobar")
k := Key{
Offset: offset,
Part: part,
@@ -145,7 +145,7 @@ func TestCacheConcurrentAccess(_ *testing.T) {
func testCacheSetGet(c *Cache, worker int) {
for i := 0; i < 1000; i++ {
part := (interface{})(i)
part := (any)(i)
b := testBlock{}
k := Key{
Offset: uint64(worker*1000 + i),

View File

@@ -71,7 +71,7 @@ func (fsm *FastStringMatcher) Match(s string) bool {
// Perform a global cleanup for fsm.m by removing items, which weren't accessed during the last 5 minutes.
m := &fsm.m
deadline := ct - uint64(cacheExpireDuration.Seconds())
m.Range(func(k, v interface{}) bool {
m.Range(func(k, v any) bool {
e := v.(*fsmEntry)
if e.lastAccessTime.Load() < deadline {
m.Delete(k)

View File

@@ -82,7 +82,7 @@ func (fst *FastStringTransformer) Transform(s string) string {
// Perform a global cleanup for fst.m by removing items, which weren't accessed during the last 5 minutes.
m := &fst.m
deadline := ct - uint64(cacheExpireDuration.Seconds())
m.Range(func(k, v interface{}) bool {
m.Range(func(k, v any) bool {
e := v.(*fstEntry)
if e.lastAccessTime.Load() < deadline {
m.Delete(k)

View File

@@ -186,7 +186,7 @@ func whetherToCloseConn(r *http.Request) bool {
return ok && fasttime.UnixTimestamp() > *deadline
}
var connDeadlineTimeKey = interface{}("connDeadlineSecs")
var connDeadlineTimeKey = any("connDeadlineSecs")
// Stop stops the http server on the given addrs, which has been started via Serve func.
func Stop(addrs []string) error {
@@ -617,7 +617,7 @@ func (rwa *responseWriterWithAbort) abort() {
}
// Errorf writes formatted error message to w and to logger.
func Errorf(w http.ResponseWriter, r *http.Request, format string, args ...interface{}) {
func Errorf(w http.ResponseWriter, r *http.Request, format string, args ...any) {
errStr := fmt.Sprintf(format, args...)
remoteAddr := GetQuotedRemoteAddr(r)
requestURI := GetRequestURI(r)

View File

@@ -135,9 +135,13 @@ func TestAuthKeyMetrics(t *testing.T) {
}
func TestHandlerWrapper(t *testing.T) {
*headerHSTS = "foo"
*headerFrameOptions = "bar"
*headerCSP = "baz"
const hstsHeader = "foo"
const frameOptionsHeader = "bar"
const cspHeader = "baz"
*headerHSTS = hstsHeader
*headerFrameOptions = frameOptionsHeader
*headerCSP = cspHeader
defer func() {
*headerHSTS = ""
*headerFrameOptions = ""
@@ -152,13 +156,14 @@ func TestHandlerWrapper(t *testing.T) {
return true
})
if w.Header().Get("Strict-Transport-Security") != "foo" {
t.Errorf("HSTS header not set")
h := w.Header()
if got := h.Get("Strict-Transport-Security"); got != hstsHeader {
t.Fatalf("unexpected HSTS header; got %q; want %q", got, hstsHeader)
}
if w.Header().Get("X-Frame-Options") != "bar" {
t.Errorf("X-Frame-Options header not set")
if got := h.Get("X-Frame-Options"); got != frameOptionsHeader {
t.Fatalf("unexpected X-Frame-Options header; got %q; want %q", got, frameOptionsHeader)
}
if w.Header().Get("Content-Security-Policy") != "baz" {
t.Errorf("CSP header not set")
if got := h.Get("Content-Security-Policy"); got != cspHeader {
t.Fatalf("unexpected CSP header; got %q; want %q", got, cspHeader)
}
}

View File

@@ -9,28 +9,27 @@ func TestTLSConfig(t *testing.T) {
insecureSkipVerify = true
tlsCfg, err := TLSConfig(certFile, keyFile, CAFile, serverName, insecureSkipVerify)
if err != nil {
t.Errorf("unexpected error %s", err)
t.Fatalf("unexpected error %s", err)
}
if tlsCfg == nil {
t.Errorf("expected tlsConfig to be set, got nil")
return
t.Fatalf("expected tlsConfig to be set, got nil")
}
if tlsCfg.ServerName != serverName {
t.Errorf("unexpected ServerName, want %s, got %s", serverName, tlsCfg.ServerName)
t.Fatalf("unexpected ServerName, want %s, got %s", serverName, tlsCfg.ServerName)
}
if tlsCfg.InsecureSkipVerify != insecureSkipVerify {
t.Errorf("unexpected InsecureSkipVerify, want %v, got %v", insecureSkipVerify, tlsCfg.InsecureSkipVerify)
t.Fatalf("unexpected InsecureSkipVerify, want %v, got %v", insecureSkipVerify, tlsCfg.InsecureSkipVerify)
}
certFile = "/path/to/nonexisting/cert/file"
_, err = TLSConfig(certFile, keyFile, CAFile, serverName, insecureSkipVerify)
if err == nil {
t.Errorf("expected keypair error, got nil")
t.Fatalf("expected keypair error, got nil")
}
certFile = ""
CAFile = "/path/to/nonexisting/cert/file"
_, err = TLSConfig(certFile, keyFile, CAFile, serverName, insecureSkipVerify)
if err == nil {
t.Errorf("expected read error, got nil")
t.Fatalf("expected read error, got nil")
}
}
@@ -40,14 +39,14 @@ func TestTransport(t *testing.T) {
URL := "http://victoriametrics.com"
_, err := Transport(URL, certFile, keyFile, CAFile, serverName, insecureSkipVerify)
if err != nil {
t.Errorf("unexpected error %s", err)
t.Fatalf("unexpected error %s", err)
}
URL = "https://victoriametrics.com"
tr, err := Transport(URL, certFile, keyFile, CAFile, serverName, insecureSkipVerify)
if err != nil {
t.Errorf("unexpected error %s", err)
t.Fatalf("unexpected error %s", err)
}
if tr.TLSClientConfig == nil {
t.Errorf("expected TLSClientConfig to be set, got nil")
t.Fatalf("expected TLSClientConfig to be set, got nil")
}
}

View File

@@ -95,45 +95,45 @@ func StdErrorLogger() *log.Logger {
}
// Infof logs info message.
func Infof(format string, args ...interface{}) {
func Infof(format string, args ...any) {
logLevel("INFO", format, args)
}
// Warnf logs warn message.
func Warnf(format string, args ...interface{}) {
func Warnf(format string, args ...any) {
logLevel("WARN", format, args)
}
// Errorf logs error message.
func Errorf(format string, args ...interface{}) {
func Errorf(format string, args ...any) {
logLevel("ERROR", format, args)
}
// WarnfSkipframes logs warn message and skips the given number of frames for the caller.
func WarnfSkipframes(skipframes int, format string, args ...interface{}) {
func WarnfSkipframes(skipframes int, format string, args ...any) {
logLevelSkipframes(skipframes, "WARN", format, args)
}
// ErrorfSkipframes logs error message and skips the given number of frames for the caller.
func ErrorfSkipframes(skipframes int, format string, args ...interface{}) {
func ErrorfSkipframes(skipframes int, format string, args ...any) {
logLevelSkipframes(skipframes, "ERROR", format, args)
}
// Fatalf logs fatal message and terminates the app.
func Fatalf(format string, args ...interface{}) {
func Fatalf(format string, args ...any) {
logLevel("FATAL", format, args)
}
// Panicf logs panic message and panics.
func Panicf(format string, args ...interface{}) {
func Panicf(format string, args ...any) {
logLevel("PANIC", format, args)
}
func logLevel(level, format string, args []interface{}) {
func logLevel(level, format string, args []any) {
logLevelSkipframes(1, level, format, args)
}
func logLevelSkipframes(skipframes int, level, format string, args []interface{}) {
func logLevelSkipframes(skipframes int, level, format string, args []any) {
if shouldSkipLog(level) {
return
}
@@ -141,7 +141,7 @@ func logLevelSkipframes(skipframes int, level, format string, args []interface{}
logMessage(level, msg, 3+skipframes)
}
func formatLogMessage(maxArgLen int, format string, args []interface{}) string {
func formatLogMessage(maxArgLen int, format string, args []any) string {
x := format
// Limit the length of every string-like arg in order to prevent from too long log messages
for i := range args {
@@ -217,7 +217,7 @@ type logWriter struct {
}
func (lw *logWriter) Write(p []byte) (int, error) {
logLevelSkipframes(2, "ERROR", "%s", []interface{}{p})
logLevelSkipframes(2, "ERROR", "%s", []any{p})
return len(p), nil
}

View File

@@ -6,7 +6,7 @@ import (
)
func TestFormatLogMessage(t *testing.T) {
f := func(format string, args []interface{}, maxArgLen int, expectedResult string) {
f := func(format string, args []any, maxArgLen int, expectedResult string) {
t.Helper()
result := formatLogMessage(maxArgLen, format, args)
if result != expectedResult {
@@ -18,8 +18,8 @@ func TestFormatLogMessage(t *testing.T) {
f("foobar", nil, 1, "foobar")
// Format args not exceeding the maxArgLen
f("foo: %d, %s, %s, %s", []interface{}{123, "bar", []byte("baz"), fmt.Errorf("abc")}, 3, "foo: 123, bar, baz, abc")
f("foo: %d, %s, %s, %s", []any{123, "bar", []byte("baz"), fmt.Errorf("abc")}, 3, "foo: 123, bar, baz, abc")
// Format args exceeding the maxArgLen
f("foo: %s, %q, %s", []interface{}{"abcde", fmt.Errorf("foo bar baz"), "xx"}, 4, `foo: a..e, "f..z", xx`)
f("foo: %s, %q, %s", []any{"abcde", fmt.Errorf("foo bar baz"), "xx"}, 4, `foo: a..e, "f..z", xx`)
}

View File

@@ -49,7 +49,7 @@ func newLogThrottler(throttle time.Duration) *LogThrottler {
}
// Errorf logs error message.
func (lt *LogThrottler) Errorf(format string, args ...interface{}) {
func (lt *LogThrottler) Errorf(format string, args ...any) {
select {
case lt.ch <- struct{}{}:
ErrorfSkipframes(1, format, args...)
@@ -58,7 +58,7 @@ func (lt *LogThrottler) Errorf(format string, args ...interface{}) {
}
// Warnf logs warn message.
func (lt *LogThrottler) Warnf(format string, args ...interface{}) {
func (lt *LogThrottler) Warnf(format string, args ...any) {
select {
case lt.ch <- struct{}{}:
WarnfSkipframes(1, format, args...)

View File

@@ -302,12 +302,12 @@ func (h *blockStreamReadersHeap) Swap(i, j int) {
x[i], x[j] = x[j], x[i]
}
func (h *blockStreamReadersHeap) Push(v interface{}) {
func (h *blockStreamReadersHeap) Push(v any) {
bsr := v.(*blockStreamReader)
*h = append(*h, bsr)
}
func (h *blockStreamReadersHeap) Pop() interface{} {
func (h *blockStreamReadersHeap) Pop() any {
x := *h
bsr := x[len(x)-1]
x[len(x)-1] = nil

View File

@@ -281,6 +281,11 @@ func getStreamIDsFromFilterOr(f filter) ([]streamID, bool) {
}
}
// DropAllPipes drops all the pipes from q.
func (q *Query) DropAllPipes() {
q.pipes = nil
}
// AddCountByTimePipe adds '| stats by (_time:step offset off, field1, ..., fieldN) count() hits' to the end of q.
func (q *Query) AddCountByTimePipe(step, off int64, fields []string) {
{

View File

@@ -2078,3 +2078,25 @@ func TestQueryCanLiveTail(t *testing.T) {
f("* | unpack_syslog", true)
f("* | unroll by (a)", true)
}
func TestQueryDropAllPipes(t *testing.T) {
f := func(qStr, resultExpected string) {
t.Helper()
q, err := ParseQuery(qStr)
if err != nil {
t.Fatalf("cannot parse [%s]: %s", qStr, err)
}
q.Optimize()
q.DropAllPipes()
result := q.String()
if result != resultExpected {
t.Fatalf("unexpected result\ngot\n%s\nwant\n%s", result, resultExpected)
}
}
f(`*`, `*`)
f(`foo | stats count()`, `foo`)
f(`foo or bar and baz | top 5 by (x)`, `foo or bar baz`)
f(`foo | filter bar:baz | stats by (x) min(y)`, `foo bar:baz`)
}

View File

@@ -43,11 +43,7 @@ func (st *StreamTags) Reset() {
st.buf = st.buf[:0]
tags := st.tags
for i := range tags {
t := &tags[i]
t.Name = nil
t.Value = nil
}
clear(tags)
st.tags = tags[:0]
}

View File

@@ -129,8 +129,7 @@ func TestParseTenantID(t *testing.T) {
got, err := ParseTenantID(tenant)
if err != nil {
t.Errorf("unexpected error: %s", err)
return
t.Fatalf("unexpected error: %s", err)
}
if got.String() != expected.String() {

View File

@@ -308,13 +308,13 @@ func (lah *lastAccessHeap) Less(i, j int) bool {
h := *lah
return h[i].lastAccessTime < h[j].lastAccessTime
}
func (lah *lastAccessHeap) Push(x interface{}) {
func (lah *lastAccessHeap) Push(x any) {
e := x.(*cacheEntry)
h := *lah
e.heapIdx = len(h)
*lah = append(h, e)
}
func (lah *lastAccessHeap) Pop() interface{} {
func (lah *lastAccessHeap) Pop() any {
h := *lah
e := h[len(h)-1]

View File

@@ -44,7 +44,7 @@ func mergeBlockStreams(ph *partHeader, bsw *blockStreamWriter, bsrs []*blockStre
}
var bsmPool = &sync.Pool{
New: func() interface{} {
New: func() any {
return &blockStreamMerger{}
},
}
@@ -238,14 +238,14 @@ func (bh *bsrHeap) Less(i, j int) bool {
return x[i].CurrItem() < x[j].CurrItem()
}
func (bh *bsrHeap) Pop() interface{} {
func (bh *bsrHeap) Pop() any {
a := *bh
v := a[len(a)-1]
*bh = a[:len(a)-1]
return v
}
func (bh *bsrHeap) Push(x interface{}) {
func (bh *bsrHeap) Push(x any) {
v := x.(*blockStreamReader)
*bh = append(*bh, v)
}

View File

@@ -205,11 +205,11 @@ func (psh *partSearchHeap) Swap(i, j int) {
x[i], x[j] = x[j], x[i]
}
func (psh *partSearchHeap) Push(x interface{}) {
func (psh *partSearchHeap) Push(x any) {
*psh = append(*psh, x.(*partSearch))
}
func (psh *partSearchHeap) Pop() interface{} {
func (psh *partSearchHeap) Pop() any {
a := *psh
v := a[len(a)-1]
*psh = a[:len(a)-1]

View File

@@ -43,12 +43,12 @@ func NewSecret(s string) *Secret {
// MarshalYAML implements yaml.Marshaler interface.
//
// It substitutes the secret with "<secret>" string.
func (s *Secret) MarshalYAML() (interface{}, error) {
func (s *Secret) MarshalYAML() (any, error) {
return "<secret>", nil
}
// UnmarshalYAML implements yaml.Unmarshaler interface.
func (s *Secret) UnmarshalYAML(f func(interface{}) error) error {
func (s *Secret) UnmarshalYAML(f func(any) error) error {
var secret string
if err := f(&secret); err != nil {
return fmt.Errorf("cannot parse secret: %w", err)

View File

@@ -61,8 +61,8 @@ type MultiLineRegex struct {
}
// UnmarshalYAML unmarshals mlr from YAML passed to f.
func (mlr *MultiLineRegex) UnmarshalYAML(f func(interface{}) error) error {
var v interface{}
func (mlr *MultiLineRegex) UnmarshalYAML(f func(any) error) error {
var v any
if err := f(&v); err != nil {
return fmt.Errorf("cannot parse multiline regex: %w", err)
}
@@ -74,12 +74,12 @@ func (mlr *MultiLineRegex) UnmarshalYAML(f func(interface{}) error) error {
return nil
}
func stringValue(v interface{}) (string, error) {
func stringValue(v any) (string, error) {
if v == nil {
return "null", nil
}
switch x := v.(type) {
case []interface{}:
case []any:
a := make([]string, len(x))
for i, xx := range x {
s, err := stringValue(xx)
@@ -106,7 +106,7 @@ func stringValue(v interface{}) (string, error) {
}
// MarshalYAML marshals mlr to YAML.
func (mlr *MultiLineRegex) MarshalYAML() (interface{}, error) {
func (mlr *MultiLineRegex) MarshalYAML() (any, error) {
if strings.ContainsAny(mlr.S, "([") {
// The mlr.S contains groups. Fall back to returning the regexp as is without splitting it into parts.
// This fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2928 .

View File

@@ -8,7 +8,7 @@ import (
)
var graphiteMatchesPool = &sync.Pool{
New: func() interface{} {
New: func() any {
return &graphiteMatches{}
},
}

View File

@@ -53,7 +53,7 @@ func (ie *IfExpression) Parse(s string) error {
// UnmarshalJSON unmarshals ie from JSON data.
func (ie *IfExpression) UnmarshalJSON(data []byte) error {
var v interface{}
var v any
if err := json.Unmarshal(data, &v); err != nil {
return err
}
@@ -72,15 +72,15 @@ func (ie *IfExpression) MarshalJSON() ([]byte, error) {
}
// UnmarshalYAML unmarshals ie from YAML passed to f.
func (ie *IfExpression) UnmarshalYAML(f func(interface{}) error) error {
var v interface{}
func (ie *IfExpression) UnmarshalYAML(f func(any) error) error {
var v any
if err := f(&v); err != nil {
return fmt.Errorf("cannot unmarshal `match` option: %w", err)
}
return ie.unmarshalFromInterface(v)
}
func (ie *IfExpression) unmarshalFromInterface(v interface{}) error {
func (ie *IfExpression) unmarshalFromInterface(v any) error {
ies := ie.ies[:0]
switch t := v.(type) {
case string:
@@ -89,7 +89,7 @@ func (ie *IfExpression) unmarshalFromInterface(v interface{}) error {
return fmt.Errorf("unexpected `match` option: %w", err)
}
ies = append(ies, ieLocal)
case []interface{}:
case []any:
for _, x := range t {
s, ok := x.(string)
if !ok {
@@ -109,7 +109,7 @@ func (ie *IfExpression) unmarshalFromInterface(v interface{}) error {
}
// MarshalYAML marshals ie to YAML
func (ie *IfExpression) MarshalYAML() (interface{}, error) {
func (ie *IfExpression) MarshalYAML() (any, error) {
if ie == nil || len(ie.ies) == 0 {
return nil, nil
}
@@ -198,7 +198,7 @@ func (ie *ifExpression) MarshalJSON() ([]byte, error) {
}
// UnmarshalYAML unmarshals ie from YAML passed to f.
func (ie *ifExpression) UnmarshalYAML(f func(interface{}) error) error {
func (ie *ifExpression) UnmarshalYAML(f func(any) error) error {
var s string
if err := f(&s); err != nil {
return fmt.Errorf("cannot unmarshal `if` option: %w", err)
@@ -210,7 +210,7 @@ func (ie *ifExpression) UnmarshalYAML(f func(interface{}) error) error {
}
// MarshalYAML marshals ie to YAML.
func (ie *ifExpression) MarshalYAML() (interface{}, error) {
func (ie *ifExpression) MarshalYAML() (any, error) {
return ie.s, nil
}

View File

@@ -331,7 +331,7 @@ type ScrapeConfig struct {
}
func (sc *ScrapeConfig) mustStart(baseDir string) {
swosFunc := func(metaLabels *promutils.Labels) interface{} {
swosFunc := func(metaLabels *promutils.Labels) any {
target := metaLabels.Get("__address__")
sw, err := sc.swc.getScrapeWork(target, nil, metaLabels)
if err != nil {

View File

@@ -72,7 +72,7 @@ type apiConfig struct {
type refreshTokenFunc func() (string, time.Duration, error)
func getAPIConfig(sdc *SDConfig, baseDir string) (*apiConfig, error) {
v, err := configMap.Get(sdc, func() (interface{}, error) { return newAPIConfig(sdc, baseDir) })
v, err := configMap.Get(sdc, func() (any, error) { return newAPIConfig(sdc, baseDir) })
if err != nil {
return nil, err
}

View File

@@ -30,7 +30,7 @@ func (ac *apiConfig) mustStop() {
var configMap = discoveryutils.NewConfigMap()
func getAPIConfig(sdc *SDConfig, baseDir string) (*apiConfig, error) {
v, err := configMap.Get(sdc, func() (interface{}, error) { return newAPIConfig(sdc, baseDir) })
v, err := configMap.Get(sdc, func() (any, error) { return newAPIConfig(sdc, baseDir) })
if err != nil {
return nil, err
}

View File

@@ -24,7 +24,7 @@ func (ac *apiConfig) mustStop() {
var configMap = discoveryutils.NewConfigMap()
func getAPIConfig(sdc *SDConfig, baseDir string) (*apiConfig, error) {
v, err := configMap.Get(sdc, func() (interface{}, error) { return newAPIConfig(sdc, baseDir) })
v, err := configMap.Get(sdc, func() (any, error) { return newAPIConfig(sdc, baseDir) })
if err != nil {
return nil, err
}

View File

@@ -51,7 +51,7 @@ func newAPIConfig(sdc *SDConfig, baseDir string) (*apiConfig, error) {
}
func getAPIConfig(sdc *SDConfig, baseDir string) (*apiConfig, error) {
v, err := configMap.Get(sdc, func() (interface{}, error) { return newAPIConfig(sdc, baseDir) })
v, err := configMap.Get(sdc, func() (any, error) { return newAPIConfig(sdc, baseDir) })
if err != nil {
return nil, err
}

View File

@@ -5,20 +5,21 @@ import (
"testing"
)
func Test_parseAPIResponse(t *testing.T) {
type args struct {
data []byte
}
tests := []struct {
name string
args args
want *listDropletResponse
wantErr bool
}{
func TestParseAPIResponse(t *testing.T) {
f := func(data string, responseExpected *listDropletResponse) {
t.Helper()
{
name: "simple parse",
args: args{data: []byte(`{
response, err := parseAPIResponse([]byte(data))
if err != nil {
t.Fatalf("unexpected parseAPIResponse() error: %s", err)
}
if !reflect.DeepEqual(response, responseExpected) {
t.Fatalf("unexpected response\ngot\n%v\nwant\n%v", response, responseExpected)
}
}
data := `
{
"droplets": [
{
"id": 3164444,
@@ -88,88 +89,70 @@ func Test_parseAPIResponse(t *testing.T) {
"next": "https://api.digitalocean.com/v2/droplets?page=2&per_page=1"
}
}
}`)},
want: &listDropletResponse{
Droplets: []droplet{
{
Image: struct {
Name string `json:"name"`
Slug string `json:"slug"`
}(struct {
Name string
Slug string
}{Name: "14.04 x64", Slug: "ubuntu-16-04-x64"}),
Region: struct {
Slug string `json:"slug"`
}(struct{ Slug string }{Slug: "nyc3"}),
Networks: networks{
V6: []network{
{
IPAddress: "2604:A880:0800:0010:0000:0000:02DD:4001",
Type: "public",
},
},
V4: []network{
{
IPAddress: "104.236.32.182",
Type: "public",
},
},
}`
responseExpected := &listDropletResponse{
Droplets: []droplet{
{
Image: dropletImage{
Name: "14.04 x64",
Slug: "ubuntu-16-04-x64",
},
Region: dropletRegion{
Slug: "nyc3",
},
Networks: networks{
V6: []network{
{
IPAddress: "2604:A880:0800:0010:0000:0000:02DD:4001",
Type: "public",
},
},
V4: []network{
{
IPAddress: "104.236.32.182",
Type: "public",
},
SizeSlug: "s-1vcpu-1gb",
Features: []string{"backups", "ipv6", "virtio"},
Tags: []string{"tag1", "tag2"},
Status: "active",
Name: "example.com",
ID: 3164444,
VpcUUID: "f9b0769c-e118-42fb-a0c4-fed15ef69662",
},
},
Links: links{
Pages: struct {
Last string `json:"last,omitempty"`
Next string `json:"next,omitempty"`
}(struct {
Last string
Next string
}{Last: "https://api.digitalocean.com/v2/droplets?page=3&per_page=1", Next: "https://api.digitalocean.com/v2/droplets?page=2&per_page=1"}),
},
SizeSlug: "s-1vcpu-1gb",
Features: []string{"backups", "ipv6", "virtio"},
Tags: []string{"tag1", "tag2"},
Status: "active",
Name: "example.com",
ID: 3164444,
VpcUUID: "f9b0769c-e118-42fb-a0c4-fed15ef69662",
},
},
Links: links{
Pages: linksPages{
Last: "https://api.digitalocean.com/v2/droplets?page=3&per_page=1",
Next: "https://api.digitalocean.com/v2/droplets?page=2&per_page=1",
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := parseAPIResponse(tt.args.data)
if (err != nil) != tt.wantErr {
t.Errorf("parseAPIResponse() error = %v, wantErr %v", err, tt.wantErr)
return
}
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("parseAPIResponse() got = \n%v\n, \nwant \n%v\n", got, tt.want)
}
})
}
f(data, responseExpected)
}
func Test_getDroplets(t *testing.T) {
type args struct {
getAPIResponse func(string) ([]byte, error)
func TestGetDroplets(t *testing.T) {
f := func(getAPIResponse func(string) ([]byte, error), expectedDropletCount int) {
t.Helper()
resp, err := getDroplets(getAPIResponse)
if err != nil {
t.Fatalf("getDroplets() error: %s", err)
}
if len(resp) != expectedDropletCount {
t.Fatalf("unexpected droplets count; got %d; want %d\ndroplets:\n%v", len(resp), expectedDropletCount, resp)
}
}
tests := []struct {
name string
args args
wantDropletCount int
wantErr bool
}{
{
name: "get 4 droples",
args: args{
func(s string) ([]byte, error) {
var resp []byte
switch s {
case dropletsAPIPath:
// return next
resp = []byte(`{ "droplets": [
getAPIResponse := func(s string) ([]byte, error) {
var resp []byte
switch s {
case dropletsAPIPath:
// return next
resp = []byte(`{ "droplets": [
{
"id": 3164444,
"name": "example.com",
@@ -267,9 +250,9 @@ func Test_getDroplets(t *testing.T) {
}
}
}`)
default:
// return with empty next
resp = []byte(`{ "droplets": [
default:
// return with empty next
resp = []byte(`{ "droplets": [
{
"id": 3164444,
"name": "example.com",
@@ -326,24 +309,8 @@ func Test_getDroplets(t *testing.T) {
}
]
}`)
}
return resp, nil
},
},
wantDropletCount: 5,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := getDroplets(tt.args.getAPIResponse)
if (err != nil) != tt.wantErr {
t.Errorf("getDroplets() error = %v, wantErr %v", err, tt.wantErr)
return
}
if len(got) != tt.wantDropletCount {
t.Fatalf("unexpected droplets count: %d, want: %d, \n droplets: %v\n", len(got), tt.wantDropletCount, got)
}
})
}
return resp, nil
}
f(getAPIResponse, 5)
}

View File

@@ -49,18 +49,22 @@ type droplet struct {
Name string `json:"name"`
Status string `json:"status"`
Features []string `json:"features"`
Image struct {
Name string `json:"name"`
Slug string `json:"slug"`
} `json:"image"`
SizeSlug string `json:"size_slug"`
Networks networks `json:"networks"`
Region struct {
Slug string `json:"slug"`
} `json:"region"`
Tags []string `json:"tags"`
VpcUUID string `json:"vpc_uuid"`
Features []string `json:"features"`
Image dropletImage `json:"image"`
SizeSlug string `json:"size_slug"`
Networks networks `json:"networks"`
Region dropletRegion `json:"region"`
Tags []string `json:"tags"`
VpcUUID string `json:"vpc_uuid"`
}
type dropletImage struct {
Name string `json:"name"`
Slug string `json:"slug"`
}
type dropletRegion struct {
Slug string `json:"slug"`
}
func (d *droplet) getIPByNet(netVersion, netType string) string {
@@ -98,10 +102,12 @@ type listDropletResponse struct {
}
type links struct {
Pages struct {
Last string `json:"last,omitempty"`
Next string `json:"next,omitempty"`
} `json:"pages,omitempty"`
Pages linksPages `json:"pages,omitempty"`
}
type linksPages struct {
Last string `json:"last,omitempty"`
Next string `json:"next,omitempty"`
}
func (r *listDropletResponse) nextURLPath() (string, error) {

View File

@@ -7,84 +7,68 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/lib/promutils"
)
func Test_addDropletLabels(t *testing.T) {
type args struct {
droplets []droplet
defaultPort int
func TestAddDropletLabels(t *testing.T) {
f := func(droplets []droplet, labelssExpected []*promutils.Labels) {
t.Helper()
labelss := addDropletLabels(droplets, 9100)
discoveryutils.TestEqualLabelss(t, labelss, labelssExpected)
}
tests := []struct {
name string
args args
want []*promutils.Labels
}{
// base labels add test
droplets := []droplet{
{
name: "base labels add test",
args: args{
droplets: []droplet{
ID: 15,
Tags: []string{"private", "test"},
Status: "active",
Name: "ubuntu-1",
Region: dropletRegion{
Slug: "do",
},
Features: []string{"feature-1", "feature-2"},
SizeSlug: "base-1",
VpcUUID: "vpc-1",
Image: dropletImage{
Name: "ubuntu",
Slug: "18",
},
Networks: networks{
V4: []network{
{
ID: 15,
Tags: []string{"private", "test"},
Status: "active",
Name: "ubuntu-1",
Region: struct {
Slug string `json:"slug"`
}(struct{ Slug string }{Slug: "do"}),
Features: []string{"feature-1", "feature-2"},
SizeSlug: "base-1",
VpcUUID: "vpc-1",
Image: struct {
Name string `json:"name"`
Slug string `json:"slug"`
}(struct {
Name string
Slug string
}{Name: "ubuntu", Slug: "18"}),
Networks: networks{
V4: []network{
{
Type: "public",
IPAddress: "100.100.100.100",
},
{
Type: "private",
IPAddress: "10.10.10.10",
},
},
V6: []network{
{
Type: "public",
IPAddress: "::1",
},
},
},
Type: "public",
IPAddress: "100.100.100.100",
},
{
Type: "private",
IPAddress: "10.10.10.10",
},
},
V6: []network{
{
Type: "public",
IPAddress: "::1",
},
},
defaultPort: 9100,
},
want: []*promutils.Labels{
promutils.NewLabelsFromMap(map[string]string{
"__address__": "100.100.100.100:9100",
"__meta_digitalocean_droplet_id": "15",
"__meta_digitalocean_droplet_name": "ubuntu-1",
"__meta_digitalocean_features": ",feature-1,feature-2,",
"__meta_digitalocean_image": "18",
"__meta_digitalocean_image_name": "ubuntu",
"__meta_digitalocean_private_ipv4": "10.10.10.10",
"__meta_digitalocean_public_ipv4": "100.100.100.100",
"__meta_digitalocean_public_ipv6": "::1",
"__meta_digitalocean_region": "do",
"__meta_digitalocean_size": "base-1",
"__meta_digitalocean_status": "active",
"__meta_digitalocean_tags": ",private,test,",
"__meta_digitalocean_vpc": "vpc-1",
}),
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := addDropletLabels(tt.args.droplets, tt.args.defaultPort)
discoveryutils.TestEqualLabelss(t, got, tt.want)
})
labelssExpected := []*promutils.Labels{
promutils.NewLabelsFromMap(map[string]string{
"__address__": "100.100.100.100:9100",
"__meta_digitalocean_droplet_id": "15",
"__meta_digitalocean_droplet_name": "ubuntu-1",
"__meta_digitalocean_features": ",feature-1,feature-2,",
"__meta_digitalocean_image": "18",
"__meta_digitalocean_image_name": "ubuntu",
"__meta_digitalocean_private_ipv4": "10.10.10.10",
"__meta_digitalocean_public_ipv4": "100.100.100.100",
"__meta_digitalocean_public_ipv6": "::1",
"__meta_digitalocean_region": "do",
"__meta_digitalocean_size": "base-1",
"__meta_digitalocean_status": "active",
"__meta_digitalocean_tags": ",private,test,",
"__meta_digitalocean_vpc": "vpc-1",
}),
}
f(droplets, labelssExpected)
}

View File

@@ -22,7 +22,7 @@ type apiConfig struct {
}
func getAPIConfig(sdc *SDConfig, baseDir string) (*apiConfig, error) {
v, err := configMap.Get(sdc, func() (interface{}, error) { return newAPIConfig(sdc, baseDir) })
v, err := configMap.Get(sdc, func() (any, error) { return newAPIConfig(sdc, baseDir) })
if err != nil {
return nil, err
}

View File

@@ -11,24 +11,32 @@ import (
// See https://github.com/moby/moby/blob/314759dc2f4745925d8dec6d15acc7761c6e5c92/docs/api/v1.41.yaml#L4024
type container struct {
ID string
Names []string
Labels map[string]string
Ports []struct {
IP string
PrivatePort int
PublicPort int
Type string
}
HostConfig struct {
NetworkMode string
}
NetworkSettings struct {
Networks map[string]struct {
IPAddress string
NetworkID string
}
}
ID string
Names []string
Labels map[string]string
Ports []containerPort
HostConfig containerHostConfig
NetworkSettings containerNetworkSettings
}
type containerPort struct {
IP string
PrivatePort int
PublicPort int
Type string
}
type containerHostConfig struct {
NetworkMode string
}
type containerNetworkSettings struct {
Networks map[string]containerNetwork
}
type containerNetwork struct {
IPAddress string
NetworkID string
}
func getContainersLabels(cfg *apiConfig) ([]*promutils.Labels, error) {

View File

@@ -8,20 +8,20 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/lib/promutils"
)
func Test_parseContainers(t *testing.T) {
type args struct {
data []byte
func TePParseContainers(t *testing.T) {
f := func(data string, resultExpected []container) {
t.Helper()
result, err := parseContainers([]byte(data))
if err != nil {
t.Fatalf("parseContainers() error: %s", err)
}
if !reflect.DeepEqual(result, resultExpected) {
t.Fatalf("unexpected result\ngot\n%v\nwant\n%v", result, resultExpected)
}
}
tests := []struct {
name string
args args
want []container
wantErr bool
}{
{
name: "parse two containers",
args: args{
data: []byte(`[
data := `[
{
"Id": "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
"Names": [
@@ -116,115 +116,83 @@ func Test_parseContainers(t *testing.T) {
}
}
}
]`),
]`
resultExpected := []container{
{
ID: "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
Names: []string{"/crow-server"},
Labels: map[string]string{
"com.docker.compose.config-hash": "c9f0bd5bb31921f94cff367d819a30a0cc08d4399080897a6c5cd74b983156ec",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "crowserver",
"com.docker.compose.service": "crow-server",
"com.docker.compose.version": "1.11.2",
},
want: []container{
Ports: []containerPort{
{
ID: "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
Names: []string{"/crow-server"},
Labels: map[string]string{
"com.docker.compose.config-hash": "c9f0bd5bb31921f94cff367d819a30a0cc08d4399080897a6c5cd74b983156ec",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "crowserver",
"com.docker.compose.service": "crow-server",
"com.docker.compose.version": "1.11.2",
},
Ports: []struct {
IP string
PrivatePort int
PublicPort int
Type string
}{{
IP: "0.0.0.0",
PrivatePort: 8080,
PublicPort: 18081,
Type: "tcp",
}},
HostConfig: struct {
NetworkMode string
}{
NetworkMode: "bridge",
},
NetworkSettings: struct {
Networks map[string]struct {
IPAddress string
NetworkID string
}
}{
Networks: map[string]struct {
IPAddress string
NetworkID string
}{
"bridge": {
IPAddress: "172.17.0.2",
NetworkID: "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
},
},
IP: "0.0.0.0",
PrivatePort: 8080,
PublicPort: 18081,
Type: "tcp",
},
},
HostConfig: containerHostConfig{
NetworkMode: "bridge",
},
NetworkSettings: containerNetworkSettings{
Networks: map[string]containerNetwork{
"bridge": {
IPAddress: "172.17.0.2",
NetworkID: "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
},
},
},
},
{
ID: "0e0f72a6eb7d9fb443f0426a66f7b8dd7d3283ab7e3a308b2bed584ac03a33dc",
Names: []string{"/crow-web"},
Labels: map[string]string{
"com.docker.compose.config-hash": "d99ebd0fde8512366c2d78c367e95ddc74528bb60b7cf0c991c9f4835981e00e",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "crowweb",
"com.docker.compose.service": "crow-web",
"com.docker.compose.version": "1.11.2",
},
Ports: []containerPort{
{
ID: "0e0f72a6eb7d9fb443f0426a66f7b8dd7d3283ab7e3a308b2bed584ac03a33dc",
Names: []string{"/crow-web"},
Labels: map[string]string{
"com.docker.compose.config-hash": "d99ebd0fde8512366c2d78c367e95ddc74528bb60b7cf0c991c9f4835981e00e",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "crowweb",
"com.docker.compose.service": "crow-web",
"com.docker.compose.version": "1.11.2",
},
Ports: []struct {
IP string
PrivatePort int
PublicPort int
Type string
}{{
IP: "0.0.0.0",
PrivatePort: 8080,
PublicPort: 18082,
Type: "tcp",
}},
HostConfig: struct {
NetworkMode string
}{
NetworkMode: "bridge",
},
NetworkSettings: struct {
Networks map[string]struct {
IPAddress string
NetworkID string
}
}{
Networks: map[string]struct {
IPAddress string
NetworkID string
}{
"bridge": {
IPAddress: "172.17.0.3",
NetworkID: "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
},
},
IP: "0.0.0.0",
PrivatePort: 8080,
PublicPort: 18082,
Type: "tcp",
},
},
HostConfig: containerHostConfig{
NetworkMode: "bridge",
},
NetworkSettings: containerNetworkSettings{
Networks: map[string]containerNetwork{
"bridge": {
IPAddress: "172.17.0.3",
NetworkID: "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
},
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := parseContainers(tt.args.data)
if (err != nil) != tt.wantErr {
t.Errorf("parseContainers() error = %v, wantErr %v", err, tt.wantErr)
return
}
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("parseNetworks() \ngot %v, \nwant %v", got, tt.want)
}
})
}
f(data, resultExpected)
}
func Test_addContainerLabels(t *testing.T) {
func TestAddContainerLabels(t *testing.T) {
f := func(c container, networkLabels map[string]*promutils.Labels, labelssExpected []*promutils.Labels) {
t.Helper()
labelss := addContainersLabels([]container{c}, networkLabels, 8012, "foobar")
discoveryutils.TestEqualLabelss(t, labelss, labelssExpected)
}
data := []byte(`[
{
"Name": "host",
@@ -314,204 +282,152 @@ func Test_addContainerLabels(t *testing.T) {
}
networkLabels := getNetworkLabelsByNetworkID(networks)
tests := []struct {
name string
c container
want []*promutils.Labels
wantErr bool
}{
{
name: "NetworkMode!=host",
c: container{
ID: "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
Names: []string{"/crow-server"},
Labels: map[string]string{
"com.docker.compose.config-hash": "c9f0bd5bb31921f94cff367d819a30a0cc08d4399080897a6c5cd74b983156ec",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "crowserver",
"com.docker.compose.service": "crow-server",
"com.docker.compose.version": "1.11.2",
},
HostConfig: struct {
NetworkMode string
}{
NetworkMode: "bridge",
},
NetworkSettings: struct {
Networks map[string]struct {
IPAddress string
NetworkID string
}
}{
Networks: map[string]struct {
IPAddress string
NetworkID string
}{
"host": {
IPAddress: "172.17.0.2",
NetworkID: "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
},
},
},
},
want: []*promutils.Labels{
promutils.NewLabelsFromMap(map[string]string{
"__address__": "172.17.0.2:8012",
"__meta_docker_container_id": "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
"__meta_docker_container_label_com_docker_compose_config_hash": "c9f0bd5bb31921f94cff367d819a30a0cc08d4399080897a6c5cd74b983156ec",
"__meta_docker_container_label_com_docker_compose_container_number": "1",
"__meta_docker_container_label_com_docker_compose_oneoff": "False",
"__meta_docker_container_label_com_docker_compose_project": "crowserver",
"__meta_docker_container_label_com_docker_compose_service": "crow-server",
"__meta_docker_container_label_com_docker_compose_version": "1.11.2",
"__meta_docker_container_name": "/crow-server",
"__meta_docker_container_network_mode": "bridge",
"__meta_docker_network_id": "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
"__meta_docker_network_ingress": "false",
"__meta_docker_network_internal": "false",
"__meta_docker_network_ip": "172.17.0.2",
"__meta_docker_network_name": "bridge",
"__meta_docker_network_scope": "local",
}),
},
// NetworkMode != host
c := container{
ID: "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
Names: []string{"/crow-server"},
Labels: map[string]string{
"com.docker.compose.config-hash": "c9f0bd5bb31921f94cff367d819a30a0cc08d4399080897a6c5cd74b983156ec",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "crowserver",
"com.docker.compose.service": "crow-server",
"com.docker.compose.version": "1.11.2",
},
{
name: "NetworkMode=host",
c: container{
ID: "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
Names: []string{"/crow-server"},
Labels: map[string]string{
"com.docker.compose.config-hash": "c9f0bd5bb31921f94cff367d819a30a0cc08d4399080897a6c5cd74b983156ec",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "crowserver",
"com.docker.compose.service": "crow-server",
"com.docker.compose.version": "1.11.2",
},
HostConfig: struct {
NetworkMode string
}{
NetworkMode: "host",
},
NetworkSettings: struct {
Networks map[string]struct {
IPAddress string
NetworkID string
}
}{
Networks: map[string]struct {
IPAddress string
NetworkID string
}{
"host": {
IPAddress: "172.17.0.2",
NetworkID: "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
},
},
},
},
want: []*promutils.Labels{
promutils.NewLabelsFromMap(map[string]string{
"__address__": "foobar",
"__meta_docker_container_id": "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
"__meta_docker_container_label_com_docker_compose_config_hash": "c9f0bd5bb31921f94cff367d819a30a0cc08d4399080897a6c5cd74b983156ec",
"__meta_docker_container_label_com_docker_compose_container_number": "1",
"__meta_docker_container_label_com_docker_compose_oneoff": "False",
"__meta_docker_container_label_com_docker_compose_project": "crowserver",
"__meta_docker_container_label_com_docker_compose_service": "crow-server",
"__meta_docker_container_label_com_docker_compose_version": "1.11.2",
"__meta_docker_container_name": "/crow-server",
"__meta_docker_container_network_mode": "host",
"__meta_docker_network_id": "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
"__meta_docker_network_ingress": "false",
"__meta_docker_network_internal": "false",
"__meta_docker_network_ip": "172.17.0.2",
"__meta_docker_network_name": "bridge",
"__meta_docker_network_scope": "local",
}),
},
HostConfig: containerHostConfig{
NetworkMode: "bridge",
},
{
name: "get labels from a container",
c: container{
ID: "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
Names: []string{"/crow-server"},
Labels: map[string]string{
"com.docker.compose.config-hash": "c9f0bd5bb31921f94cff367d819a30a0cc08d4399080897a6c5cd74b983156ec",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "crowserver",
"com.docker.compose.service": "crow-server",
"com.docker.compose.version": "1.11.2",
NetworkSettings: containerNetworkSettings{
Networks: map[string]containerNetwork{
"host": {
IPAddress: "172.17.0.2",
NetworkID: "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
},
Ports: []struct {
IP string
PrivatePort int
PublicPort int
Type string
}{{
IP: "0.0.0.0",
PrivatePort: 8080,
PublicPort: 18081,
Type: "tcp",
}},
HostConfig: struct {
NetworkMode string
}{
NetworkMode: "bridge",
},
NetworkSettings: struct {
Networks map[string]struct {
IPAddress string
NetworkID string
}
}{
Networks: map[string]struct {
IPAddress string
NetworkID string
}{
"bridge": {
IPAddress: "172.17.0.2",
NetworkID: "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
},
},
},
},
want: []*promutils.Labels{
promutils.NewLabelsFromMap(map[string]string{
"__address__": "172.17.0.2:8080",
"__meta_docker_container_id": "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
"__meta_docker_container_label_com_docker_compose_config_hash": "c9f0bd5bb31921f94cff367d819a30a0cc08d4399080897a6c5cd74b983156ec",
"__meta_docker_container_label_com_docker_compose_container_number": "1",
"__meta_docker_container_label_com_docker_compose_oneoff": "False",
"__meta_docker_container_label_com_docker_compose_project": "crowserver",
"__meta_docker_container_label_com_docker_compose_service": "crow-server",
"__meta_docker_container_label_com_docker_compose_version": "1.11.2",
"__meta_docker_container_name": "/crow-server",
"__meta_docker_container_network_mode": "bridge",
"__meta_docker_network_id": "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
"__meta_docker_network_ingress": "false",
"__meta_docker_network_internal": "false",
"__meta_docker_network_ip": "172.17.0.2",
"__meta_docker_network_name": "bridge",
"__meta_docker_network_scope": "local",
"__meta_docker_port_private": "8080",
"__meta_docker_port_public": "18081",
"__meta_docker_port_public_ip": "0.0.0.0",
}),
},
},
}
labelssExpected := []*promutils.Labels{
promutils.NewLabelsFromMap(map[string]string{
"__address__": "172.17.0.2:8012",
"__meta_docker_container_id": "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
"__meta_docker_container_label_com_docker_compose_config_hash": "c9f0bd5bb31921f94cff367d819a30a0cc08d4399080897a6c5cd74b983156ec",
"__meta_docker_container_label_com_docker_compose_container_number": "1",
"__meta_docker_container_label_com_docker_compose_oneoff": "False",
"__meta_docker_container_label_com_docker_compose_project": "crowserver",
"__meta_docker_container_label_com_docker_compose_service": "crow-server",
"__meta_docker_container_label_com_docker_compose_version": "1.11.2",
"__meta_docker_container_name": "/crow-server",
"__meta_docker_container_network_mode": "bridge",
"__meta_docker_network_id": "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
"__meta_docker_network_ingress": "false",
"__meta_docker_network_internal": "false",
"__meta_docker_network_ip": "172.17.0.2",
"__meta_docker_network_name": "bridge",
"__meta_docker_network_scope": "local",
}),
}
f(c, networkLabels, labelssExpected)
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
labelss := addContainersLabels([]container{tt.c}, networkLabels, 8012, "foobar")
if (err != nil) != tt.wantErr {
t.Errorf("addContainersLabels() error = %v, wantErr %v", err, tt.wantErr)
return
}
discoveryutils.TestEqualLabelss(t, labelss, tt.want)
})
// NetworkMode=host
c = container{
ID: "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
Names: []string{"/crow-server"},
Labels: map[string]string{
"com.docker.compose.config-hash": "c9f0bd5bb31921f94cff367d819a30a0cc08d4399080897a6c5cd74b983156ec",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "crowserver",
"com.docker.compose.service": "crow-server",
"com.docker.compose.version": "1.11.2",
},
HostConfig: containerHostConfig{
NetworkMode: "host",
},
NetworkSettings: containerNetworkSettings{
Networks: map[string]containerNetwork{
"host": {
IPAddress: "172.17.0.2",
NetworkID: "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
},
},
},
}
labelssExpected = []*promutils.Labels{
promutils.NewLabelsFromMap(map[string]string{
"__address__": "foobar",
"__meta_docker_container_id": "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
"__meta_docker_container_label_com_docker_compose_config_hash": "c9f0bd5bb31921f94cff367d819a30a0cc08d4399080897a6c5cd74b983156ec",
"__meta_docker_container_label_com_docker_compose_container_number": "1",
"__meta_docker_container_label_com_docker_compose_oneoff": "False",
"__meta_docker_container_label_com_docker_compose_project": "crowserver",
"__meta_docker_container_label_com_docker_compose_service": "crow-server",
"__meta_docker_container_label_com_docker_compose_version": "1.11.2",
"__meta_docker_container_name": "/crow-server",
"__meta_docker_container_network_mode": "host",
"__meta_docker_network_id": "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
"__meta_docker_network_ingress": "false",
"__meta_docker_network_internal": "false",
"__meta_docker_network_ip": "172.17.0.2",
"__meta_docker_network_name": "bridge",
"__meta_docker_network_scope": "local",
}),
}
f(c, networkLabels, labelssExpected)
// get labels from a container
c = container{
ID: "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
Names: []string{"/crow-server"},
Labels: map[string]string{
"com.docker.compose.config-hash": "c9f0bd5bb31921f94cff367d819a30a0cc08d4399080897a6c5cd74b983156ec",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "crowserver",
"com.docker.compose.service": "crow-server",
"com.docker.compose.version": "1.11.2",
},
Ports: []containerPort{
{
IP: "0.0.0.0",
PrivatePort: 8080,
PublicPort: 18081,
Type: "tcp",
},
},
HostConfig: containerHostConfig{
NetworkMode: "bridge",
},
NetworkSettings: containerNetworkSettings{
Networks: map[string]containerNetwork{
"bridge": {
IPAddress: "172.17.0.2",
NetworkID: "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
},
},
},
}
labelssExpected = []*promutils.Labels{
promutils.NewLabelsFromMap(map[string]string{
"__address__": "172.17.0.2:8080",
"__meta_docker_container_id": "90bc3b31aa13da5c0b11af2e228d54b38428a84e25d4e249ae9e9c95e51a0700",
"__meta_docker_container_label_com_docker_compose_config_hash": "c9f0bd5bb31921f94cff367d819a30a0cc08d4399080897a6c5cd74b983156ec",
"__meta_docker_container_label_com_docker_compose_container_number": "1",
"__meta_docker_container_label_com_docker_compose_oneoff": "False",
"__meta_docker_container_label_com_docker_compose_project": "crowserver",
"__meta_docker_container_label_com_docker_compose_service": "crow-server",
"__meta_docker_container_label_com_docker_compose_version": "1.11.2",
"__meta_docker_container_name": "/crow-server",
"__meta_docker_container_network_mode": "bridge",
"__meta_docker_network_id": "1dd8d1a8bef59943345c7231d7ce8268333ff5a8c5b3c94881e6b4742b447634",
"__meta_docker_network_ingress": "false",
"__meta_docker_network_internal": "false",
"__meta_docker_network_ip": "172.17.0.2",
"__meta_docker_network_name": "bridge",
"__meta_docker_network_scope": "local",
"__meta_docker_port_private": "8080",
"__meta_docker_port_public": "18081",
"__meta_docker_port_public_ip": "0.0.0.0",
}),
}
f(c, networkLabels, labelssExpected)
}

Some files were not shown because too many files have changed in this diff Show More