Compare commits

...

29 Commits

Author SHA1 Message Date
Artem Fetishev
3f1b7a3a7f lib/storage: make lrucache ttl configurable
Previously, the `lrucache` ttl was hardcoded to 3 minutes and the code comment
said it was enough for most queries that are sent to vmstorage repeatedly. But
it appears that it is not always the case.

`indexDB` has this `tagFilters loop cache` that is used in index search
optimizations (see `getMetricIDsForDateAndFilters()`). Until recently, this
cache was implemented with `workingsetcache`. In
[10154](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10154), this
implementation was changed to `lrucache`. After this change, some users reported
huge CPU utilization increase in vmstorage (see
[10297](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10297)).

`workingsetcache` evicts an entry after two rotations that happen every 30
minutes. I.e. the entry ttl is 1h since the last time it was retrieved from the
cache. Hence the assumption that `lrucache` causes the CPU to raise because of
its too aggressive eviction policy.

Reverting back to `workingsetcache` helped but `lrucache` is still preferred
because it occupies less memory. Since `lrucache` is preferred, its ttl needs to
be increased. Instead changing the hardcoded value, the ttl was made
configurable.

The `tagFilters loops cache` was configured to have ttl of 1h to match the
default behavior of `workingsetcache`.

Although no one complained about this yet, the ttl was also increased for
`tfssCache` because previously it was also implemented with `workingsetcache`.

Finally, the `regexpCache` and `prefixesCache` were also configured to have 1h
ttl because these caches are also during the data retrieval.

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2026-02-09 12:01:24 +01:00
Artem Fetishev
5915c57b46 docs/changelog: add known issue to v1.132.0 release notes
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2026-02-09 11:03:19 +01:00
JAYICE
f36e1857c0 app/vmagent: improve kafka consumer performance
Previously, the Kafka consumer in vmagent committed offsets per message
(manual commit). At high message rates, this could overload the commit
path (coordinator, __consumer_offsets topic, and network).

This commit introduces time-based manual commits with a controlled window:
* enable.auto.commit remains false by default.
* After a successful TryPush (data accepted into the buffer before the
  vmagent queue/backend), vmagent adds the message to pending offsets.
* Offsets are committed periodically (every second), as well as during
  shutdown and partition rebalance.

This keeps the commit point tied to TryPush (stronger guarantees than
auto-commit) while significantly reducing commit QPS.

Auto-commit is also time-based, but it advances offsets based on poll()
delivery rather than application-level processing. This means offsets
may be committed before data is actually accepted by the vmagent
pipeline, slightly increasing the risk of data loss on crash or restart.

This change does not make the Kafka consumer fully transactional
end-to-end. Buffers in vmagent/vminsert/vmstorage still imply possible
data loss on hard stops. However, it provides stronger guarantees than
auto-commit, since commits are based on TryPush rather than poll().

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10395
2026-02-06 13:17:14 +01:00
Aliaksandr Valialkin
04f4a28cf4 docs/victoriametrics/integrations/zabbixconnector.md: add an alias - https://docs.victoriametrics.com/victoriametrics/integrations/zabbix/ - seen in the Internet
Visits to this page are seen in Google Analytics reports.
2026-02-05 23:52:08 +01:00
Aliaksandr Valialkin
7f3d370244 deployment/docker: update base Alpine Docker image from 3.23.2 to 3.23.3
See https://www.alpinelinux.org/posts/Alpine-3.20.9-3.21.6-3.22.3-3.23.3-released.html
2026-02-05 19:48:54 +01:00
Aliaksandr Valialkin
c89b7f7ad5 deployment/docker: update Go builder from Go1.25.6 to Go1.25.7
See https://github.com/golang/go/issues?q=milestone%3AGo1.25.7%20label%3ACherryPickApproved
2026-02-05 19:46:56 +01:00
Aliaksandr Valialkin
d9dabea303 docs/victoriametrics: add links on how to tune VictoriaMetrics for IoT and industrial monitoring cases with low churn rate for time series
The link is https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#index-tuning-for-low-churn-rate
Put this link to the docs which mention IoT and industrial monitoring, so users could figure out
how to optimize VictoriaMetrics for these cases.
2026-02-05 17:23:10 +01:00
Aliaksandr Valialkin
09d2ce36e8 lib/protoparser/protoparserutil: do not store byte slices with more than 75% of unused space in the pool
Keeping such byte slices in the pool may increase memory usage when processing a small share of requests
with much bigger sizes than the average processed request.

This should help reducing memory usage at https://github.com/VictoriaMetrics/VictoriaLogs/issues/1042
2026-02-04 15:31:21 +01:00
Max Kotliar
08755c838b docs: update changelog with LTS release notes 2026-02-02 18:45:07 +02:00
Max Kotliar
d2e438ef41 docs: bump version to v1.135.0 2026-02-02 18:38:03 +02:00
Max Kotliar
e508fa5fe2 deplyoment/docker: bump version to v1.135.0 2026-02-02 18:27:28 +02:00
f41gh7
9a7deca207 follow-up for 60cadfbad1
Respect the default value of http.DefaultTransport.Proxy. Previously,
it could be unintentionally overridden with a nil value.

This commit aligns Proxy configuration across all created transports.
2026-02-02 16:39:52 +01:00
Zane DeGraffenried
60cadfbad1 lib/promauth: fix oauth http client overwriting default proxy with nil
Previously, default `Proxy` was unconditionally replaced with config value, which could be nil. 
It made impossible to use  default http client proxy env variables.

This commit adds check in oauth http client builder that only overwrites the
transport proxy if a custom proxy url function is defined.

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10385
2026-02-02 15:39:46 +01:00
Vadim Alekseev
b36c8b1110 app/vminsert/common: reduce allocations when writing metadata
Bug was introduced at 5a587f2006, while porting change from cluster branch.

This commit properlyslice `mms
[]metricsmetadata.Row` slice . Previously, every WriteMetadata call triggered a
slice allocation.
This shouldn't significantly impact overall performance, so I haven't
included benchmarks.

Related PR https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10392
2026-02-02 15:36:33 +01:00
Nikolay
90f0405b11 lib/promscrape: properly expose kubernetes_sd dialer metrics (#10381)
Commit 35b31f904d introduced a bug, where
dialer metrics for Kubernetes discovery were overwritten.

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10382
2026-02-02 14:47:50 +01:00
Nikolay
eac0a7ed86 docs: mention downsampling export API behavior
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10326
2026-02-02 14:44:40 +01:00
Artem Fetishev
a8a99105b1 lib/storage: reduce number of shards in metricIDCache (#10388)
This should reduce cpu utilization while still removing the storage
connection saturation.

Follow-up for 6bc809813b (#10346)

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2026-02-01 18:13:44 +01:00
Max Kotliar
c7f52992e7 docs/changelog: cut v1.135.0 2026-01-30 14:12:05 +02:00
Max Kotliar
5fe14e5479 docs: run make docs-update-flags 2026-01-30 14:09:42 +02:00
Max Kotliar
c7ef079eba docs: run make docs-update-flags 2026-01-30 14:04:39 +02:00
Max Kotliar
424d007a39 app/vmselect: run make vmui-update 2026-01-30 13:58:57 +02:00
Zakhar Bessarab
ad4562cd56 lib/pushmetrics: allow enabling push metrics via config
This is needed in order to allow using lib/pushmetrics for vmctl as it does not use go native flags.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

app/vmctl: add metrics for the migrations

- add flags to allow setting up metrics push
- add metrics to track progress of the migration for all modes
- add metrics for generic backoff and limiter packages

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2026-01-30 13:02:13 +02:00
f41gh7
f9895d7e5e follow-up for a2271284
Remove duplicate line at app/vmui/Makefile
2026-01-30 11:28:50 +01:00
Andrei Baidarov
6bc809813b lib/storage: shard metricIdCache
The current implementation has a bottleneck – a single mutex to access
`prev`/`next` metric sets. Each rotation results in storage utilization
spikes since lock-free `curr` is almost empty, and cache needs to
promote metrics from `prev` to `next`.

This is an attempt to reduce contention by spliting cache into separate
shards.

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10367
2026-01-30 11:19:40 +01:00
Hui Wang
9b40fd00e0 app/vmalert: do not skip sending alert notifications to -notifier.url if remote write requests fail
Note: remote write request won't fail immediately if `-remoteWrite.url`
is unreachable, as vmalert maintains a remote write queue (with capacity
controlled by `-remoteWrite.maxQueueSize(default 1e5)`) and uses a
separate process to batch and push queued data.

vmalert uses error group to print error messages associated with a
single group together, which should assist the group owner in reviewing
relevant error messages.
With this pull request, the error message would be like:
```
2026-01-30T08:26:46.641Z	error	app/vmalert/rule/group.go:395	group "group2": errors(3): 
rule "rule1": remote write failure: failed to push timeseries - queue is full (1 entries). Queue size is controlled by -remoteWrite.maxQueueSize flag
rule "rule1": notifier failure: failed to send alerts to addr "http://non-existing-alertmanager-1/api/v2/alerts": invalid SC 502 from "http://non-existing-alertmanager-1/api/v2/alerts"; response body: 
rule "rule1": notifier failure: failed to send alerts to addr "http://non-existing-alertmanager-2/api/v2/alerts": invalid SC 502 from "http://non-existing-alertmanager-2/api/v2/alerts"; response body: 
2026-01-30T08:26:46.641Z	error	app/vmalert/rule/group.go:395	group "group2": errors(3): 
rule "rule2": remote write failure: failed to push timeseries - queue is full (1 entries). Queue size is controlled by -remoteWrite.maxQueueSize flag
rule "rule2": notifier failure: failed to send alerts to addr "http://non-existing-alertmanager-2/api/v2/alerts": invalid SC 502 from "http://non-existing-alertmanager-2/api/v2/alerts"; response body: 
rule "rule2": notifier failure: failed to send alerts to addr "http://non-existing-alertmanager-1/api/v2/alerts": invalid SC 502 from "http://non-existing-alertmanager-1/api/v2/alerts"; response body: 
2026-01-30T08:26:52.229Z	error	app/vmalert/rule/group.go:395	group "group1": errors(3): 
rule "rule1": remote write failure: failed to push timeseries - queue is full (1 entries). Queue size is controlled by -remoteWrite.maxQueueSize flag
rule "rule1": notifier failure: failed to send alerts to addr "http://non-existing-alertmanager-1/api/v2/alerts": invalid SC 502 from "http://non-existing-alertmanager-1/api/v2/alerts"; response body: 
rule "rule1": notifier failure: failed to send alerts to addr "http://non-existing-alertmanager-2/api/v2/alerts": invalid SC 502 from "http://non-existing-alertmanager-2/api/v2/alerts"; response body: 
2026-01-30T08:26:52.229Z	error	app/vmalert/rule/group.go:395	group "group1": errors(3): 
rule "rule2": remote write failure: failed to push timeseries - queue is full (1 entries). Queue size is controlled by -remoteWrite.maxQueueSize flag
rule "rule2": notifier failure: failed to send alerts to addr "http://non-existing-alertmanager-2/api/v2/alerts": invalid SC 502 from "http://non-existing-alertmanager-2/api/v2/alerts"; response body: 
rule "rule2": notifier failure: failed to send alerts to addr "http://non-existing-alertmanager-1/api/v2/alerts": invalid SC 502 from "http://non-existing-alertmanager-1/api/v2/alerts"; response body: 
```

Related PR https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10376
2026-01-30 11:14:27 +01:00
JAYICE
9d59a31290 expose topN average memory bytes consumption queries in /api/v1/status/top_queries (#10350)
### Describe Your Changes

part of https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9330

### Checklist

The following checks are **mandatory**:

- [x] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
- [x] My change adheres to [VictoriaMetrics development
goals](https://docs.victoriametrics.com/victoriametrics/goals/).

---------

Signed-off-by: JAYICE <1185430411@qq.com>
2026-01-30 10:47:31 +02:00
Zakhar Bessarab
8391be18be app/vmbackupmanager: allow disabling scheduled backups
This commit adds a new flag `disableScheduledBackups` for `vmbackupmanager. Which disables any scheduled backups. It could be useful to keep vmbackupmanager running and serving API calls only.

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10364
2026-01-29 13:46:00 +01:00
Vadim Rutkovsky
8feb8c17aa docs: update examples and documentation after nodes/proxy permission removed
Updated helm-charts and operators no longer come with nodes/proxy
permissions for vmagent/vmsingle roles. In the examples using kubelet's
proxy endpoint we should explicitly create ClusterRoles /
ClusterRoleBinding to grant access.

See https://github.com/VictoriaMetrics/operator/pull/1754 and
https://github.com/VictoriaMetrics/helm-charts/pull/2676

Ref: https://github.com/VictoriaMetrics/operator/issues/1753
2026-01-29 13:20:12 +01:00
Hui Wang
634b4d035d app/vmalert: ensure alert restore retrieve the correct previous alert state if the group takes long time to evaluate
The new `ALERTS_FOR_STATE` may be retrieved during restore when:
1. a group contains multiple heavy rules, alerting rule A may have
already been executed and its state metrics successfully uploaded to the
datasource by the time all rules within the group have finished
executing;
2. the datasource makes data queryable very quickly, for instance, when
users configure a small value for `-search.latencyOffset`.

fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10335
2026-01-29 13:18:32 +01:00
75 changed files with 1113 additions and 560 deletions

View File

@@ -11,7 +11,6 @@ import (
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmalert/datasource"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmalert/vmalertutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/flagutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/httputil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
@@ -291,7 +290,7 @@ func GetTargets() map[TargetType][]Target {
}
// Send sends alerts to all active notifiers
func Send(ctx context.Context, alerts []Alert, notifierHeaders map[string]string) *vmalertutil.ErrGroup {
func Send(ctx context.Context, alerts []Alert, notifierHeaders map[string]string) chan error {
alertsToSend := make([]Alert, 0, len(alerts))
lblss := make([][]prompb.Label, 0, len(alerts))
// apply global relabel config first without modifying original alerts in alerts
@@ -304,17 +303,18 @@ func Send(ctx context.Context, alerts []Alert, notifierHeaders map[string]string
lblss = append(lblss, lbls)
}
errGr := new(vmalertutil.ErrGroup)
wg := sync.WaitGroup{}
activeNotifiers := getActiveNotifiers()
errCh := make(chan error, len(activeNotifiers))
defer close(errCh)
for i := range activeNotifiers {
nt := activeNotifiers[i]
wg.Go(func() {
if err := nt.Send(ctx, alertsToSend, lblss, notifierHeaders); err != nil {
errGr.Add(fmt.Errorf("failed to send alerts to addr %q: %w", nt.Addr(), err))
errCh <- fmt.Errorf("failed to send alerts to addr %q: %w", nt.Addr(), err)
}
})
}
wg.Wait()
return errGr
return errCh
}

View File

@@ -205,7 +205,9 @@ alert_relabel_configs:
},
}
errG := Send(context.Background(), firingAlerts, nil)
if errG.Err() != nil {
t.Fatalf("unexpected error when sending alerts: %s", err)
for err := range errG {
if err != nil {
t.Errorf("unexpected error when sending alerts: %s", err)
}
}
}

View File

@@ -818,7 +818,9 @@ func (ar *AlertingRule) restore(ctx context.Context, q datasource.Querier, ts ti
expr := fmt.Sprintf("default_rollup(%s{%s%s}[%ds])",
alertForStateMetricName, nameStr, labelsFilter, int(lookback.Seconds()))
res, _, err := q.Query(ctx, expr, ts)
// query ALERTS_FOR_STATE at `ts-1s` instead `ts` to avoid retrieving data written in the current run,
// see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10335
res, _, err := q.Query(ctx, expr, ts.Add(-1*time.Second))
if err != nil {
return fmt.Errorf("failed to execute restore query %q: %w ", expr, err)
}

View File

@@ -18,6 +18,7 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmalert/datasource"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmalert/notifier"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmalert/remotewrite"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmalert/vmalertutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/prompb"
)
@@ -374,7 +375,7 @@ func (g *Group) Start(ctx context.Context, rw remotewrite.RWClient, rr datasourc
g.infof("started")
eval := func(ctx context.Context, ts time.Time) {
eval := func(ctx context.Context, ts time.Time) time.Time {
g.metrics.iterationTotal.Inc()
start := time.Now()
@@ -382,7 +383,7 @@ func (g *Group) Start(ctx context.Context, rw remotewrite.RWClient, rr datasourc
if len(g.Rules) < 1 {
g.metrics.iterationDuration.UpdateDuration(start)
g.LastEvaluation = start
return
return ts
}
resolveDuration := getResolveDuration(g.Interval, *resendDelay, *maxResolveDuration)
@@ -396,6 +397,7 @@ func (g *Group) Start(ctx context.Context, rw remotewrite.RWClient, rr datasourc
}
g.metrics.iterationDuration.UpdateDuration(start)
g.LastEvaluation = start
return ts
}
evalCtx, cancel := context.WithCancel(ctx)
@@ -404,7 +406,7 @@ func (g *Group) Start(ctx context.Context, rw remotewrite.RWClient, rr datasourc
g.mu.Unlock()
defer g.evalCancel()
eval(evalCtx, evalTS)
realEvalTS := eval(evalCtx, evalTS)
t := time.NewTicker(g.Interval)
defer t.Stop()
@@ -412,7 +414,7 @@ func (g *Group) Start(ctx context.Context, rw remotewrite.RWClient, rr datasourc
// restore the rules state after the first evaluation
// so only active alerts can be restored.
if rr != nil {
err := g.restore(ctx, rr, evalTS, *remoteReadLookBack)
err := g.restore(ctx, rr, realEvalTS, *remoteReadLookBack)
if err != nil {
logger.Errorf("error while restoring ruleState for group %q: %s", g.Name, err)
}
@@ -755,6 +757,7 @@ func (e *executor) exec(ctx context.Context, r Rule, ts time.Time, resolveDurati
return fmt.Errorf("rule %q: failed to execute: %w", r, err)
}
var errG vmalertutil.ErrGroup
if e.Rw != nil {
pushToRW := func(tss []prompb.TimeSeries) error {
var lastErr error
@@ -766,20 +769,26 @@ func (e *executor) exec(ctx context.Context, r Rule, ts time.Time, resolveDurati
return lastErr
}
if err := pushToRW(tss); err != nil {
return err
errG.Add(err)
}
}
ar, ok := r.(*AlertingRule)
if !ok {
return nil
return errG.Err()
}
alerts := ar.alertsToSend(resolveDuration, *resendDelay)
if len(alerts) < 1 {
return nil
return errG.Err()
}
errGr := notifier.Send(ctx, alerts, e.notifierHeaders)
return errGr.Err()
notifierErr := notifier.Send(ctx, alerts, e.notifierHeaders)
for err := range notifierErr {
if err != nil {
errG.Add(fmt.Errorf("rule %q: notifier failure: %w", r, err))
}
}
return errG.Err()
}

View File

@@ -45,7 +45,7 @@ func (eg *ErrGroup) Error() string {
return ""
}
var b strings.Builder
fmt.Fprintf(&b, "errors(%d): ", len(eg.errs))
fmt.Fprintf(&b, "errors(%d): \n", len(eg.errs))
for i, err := range eg.errs {
b.WriteString(err.Error())
if i != len(eg.errs)-1 {

View File

@@ -30,8 +30,8 @@ func TestErrGroup(t *testing.T) {
}
f(nil, "")
f([]error{errors.New("timeout")}, "errors(1): timeout")
f([]error{errors.New("timeout"), errors.New("deadline")}, "errors(2): timeout\ndeadline")
f([]error{errors.New("timeout")}, "errors(1): \ntimeout")
f([]error{errors.New("timeout"), errors.New("deadline")}, "errors(2): \ntimeout\ndeadline")
}
// TestErrGroupConcurrent supposed to test concurrent

View File

@@ -7,6 +7,8 @@ import (
"math"
"time"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
)
@@ -55,6 +57,7 @@ func (b *Backoff) Retry(ctx context.Context, cb retryableFunc) (uint64, error) {
return attempt, err // fail fast if not recoverable
}
attempt++
retriesTotal.Inc()
backoff := float64(b.minDuration) * math.Pow(b.factor, float64(i))
dur := time.Duration(backoff)
logger.Errorf("got error: %s on attempt: %d; will retry in %v", err, attempt, dur)
@@ -74,3 +77,7 @@ func (b *Backoff) Retry(ctx context.Context, cb retryableFunc) (uint64, error) {
}
return attempt, fmt.Errorf("execution failed after %d retry attempts", b.retries)
}
var (
retriesTotal = metrics.NewCounter(`vmctl_backoff_retries_total`)
)

View File

@@ -14,6 +14,12 @@ const (
globalSilent = "s"
globalVerbose = "verbose"
globalDisableProgressBar = "disable-progress-bar"
globalPushMetricsURL = "pushmetrics.url"
globalPushMetricsInterval = "pushmetrics.interval"
globalPushExtraLabels = "pushmetrics.extraLabel"
globalPushHeaders = "pushmetrics.header"
globalPushDisableCompression = "pushmetrics.disableCompression"
)
var (
@@ -33,6 +39,29 @@ var (
Value: false,
Usage: "Whether to disable progress bar during the import.",
},
&cli.StringSliceFlag{
Name: globalPushMetricsURL,
Usage: "Optional URL to push metrics. See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#push-metrics",
},
&cli.DurationFlag{
Name: globalPushMetricsInterval,
Value: 10 * time.Second,
Usage: "Interval for pushing metrics to every -pushmetrics.url",
},
&cli.StringSliceFlag{
Name: globalPushExtraLabels,
Usage: "Extra labels to add to pushed metrics. In case of collision, label value defined by flag will have priority. " +
"Flag can be set multiple times, to add few additional labels. " +
"For example, -pushmetrics.extraLabel='instance=\"foo\"' adds instance=\"foo\" label to all the metrics pushed to every -pushmetrics.url",
},
&cli.StringSliceFlag{
Name: globalPushHeaders,
Usage: "Optional HTTP headers to add to pushed metrics. Flag can be set multiple times, to add few additional headers.",
},
&cli.BoolFlag{
Name: globalPushDisableCompression,
Usage: "Whether to disable compression when pushing metrics.",
},
}
)

View File

@@ -7,6 +7,8 @@ import (
"log"
"sync"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/barpool"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/influx"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/vm"
@@ -52,6 +54,7 @@ func (ip *influxProcessor) run(ctx context.Context) error {
return nil
}
influxSeriesTotal.Add(len(series))
bar := barpool.AddWithTemplate(fmt.Sprintf(barTpl, "Processing series"), len(series))
if err := barpool.Start(); err != nil {
return err
@@ -67,9 +70,11 @@ func (ip *influxProcessor) run(ctx context.Context) error {
wg.Go(func() {
for s := range seriesCh {
if err := ip.do(s); err != nil {
influxErrorsTotal.Inc()
errCh <- fmt.Errorf("request failed for %q.%q: %s", s.Measurement, s.Field, err)
return
}
influxSeriesProcessed.Inc()
bar.Increment()
}
})
@@ -81,6 +86,7 @@ func (ip *influxProcessor) run(ctx context.Context) error {
case infErr := <-errCh:
return fmt.Errorf("influx error: %s", infErr)
case vmErr := <-ip.im.Errors():
influxErrorsTotal.Inc()
return fmt.Errorf("import process failed: %s", wrapErr(vmErr, ip.isVerbose))
case seriesCh <- s:
}
@@ -93,6 +99,7 @@ func (ip *influxProcessor) run(ctx context.Context) error {
// drain import errors channel
for vmErr := range ip.im.Errors() {
if vmErr.Err != nil {
influxErrorsTotal.Inc()
return fmt.Errorf("import process failed: %s", wrapErr(vmErr, ip.isVerbose))
}
}
@@ -167,3 +174,9 @@ func (ip *influxProcessor) do(s *influx.Series) error {
}
}
}
var (
influxSeriesTotal = metrics.NewCounter(`vmctl_influx_migration_series_total`)
influxSeriesProcessed = metrics.NewCounter(`vmctl_influx_migration_series_processed`)
influxErrorsTotal = metrics.NewCounter(`vmctl_influx_migration_errors_total`)
)

View File

@@ -4,6 +4,8 @@ import (
"sync"
"time"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/timerpool"
)
@@ -45,9 +47,16 @@ func (l *Limiter) Register(dataLen int) {
t := timerpool.Get(d)
<-t.C
timerpool.Put(t)
limiterThrottleEventsTotal.Inc()
}
l.budget += limit
l.deadline = time.Now().Add(time.Second)
}
l.budget -= int64(dataLen)
limiterBytesProcessed.Add(dataLen)
}
var (
limiterBytesProcessed = metrics.NewCounter(`vmctl_limiter_bytes_processed_total`)
limiterThrottleEventsTotal = metrics.NewCounter(`vmctl_limiter_throttle_events_total`)
)

View File

@@ -2,6 +2,7 @@ package main
import (
"context"
"flag"
"fmt"
"log"
"net/http"
@@ -19,7 +20,9 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/barpool"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/native"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/remoteread"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/netutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/pushmetrics"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/influx"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/opentsdb"
@@ -41,11 +44,20 @@ func main() {
ctx, cancelCtx := context.WithCancel(context.Background())
start := time.Now()
beforeFn := func(c *cli.Context) error {
flag.Parse()
logger.Init()
isSilent = c.Bool(globalSilent)
if c.Bool(globalDisableProgressBar) {
barpool.Disable(true)
}
netutil.EnableIPv6()
pushmetrics.InitWith(&pushmetrics.Config{
URLs: c.StringSlice(globalPushMetricsURL),
Interval: c.Duration(globalPushMetricsInterval),
ExtraLabels: c.StringSlice(globalPushExtraLabels),
DisableCompression: c.Bool(globalPushDisableCompression),
Headers: c.StringSlice(globalPushHeaders),
})
return nil
}
app := &cli.App{
@@ -451,6 +463,7 @@ func main() {
log.Fatalln(err)
}
log.Printf("Total time: %v", time.Since(start))
pushmetrics.StopAndPush()
}
func initConfigVM(c *cli.Context) (vm.Config, error) {

View File

@@ -8,6 +8,8 @@ import (
"net/http"
"time"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/auth"
)
@@ -36,12 +38,15 @@ type Response struct {
// Explore finds metric names by provided filter from api/v1/label/__name__/values
func (c *Client) Explore(ctx context.Context, f Filter, tenantID string, start, end time.Time) ([]string, error) {
startTime := time.Now()
exploreRequestsTotal.Inc()
url := fmt.Sprintf("%s/%s", c.Addr, nativeMetricNamesAddr)
if tenantID != "" {
url = fmt.Sprintf("%s/select/%s/prometheus/%s", c.Addr, tenantID, nativeMetricNamesAddr)
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
exploreRequestsErrorsTotal.Inc()
return nil, fmt.Errorf("cannot create request to %q: %s", url, err)
}
@@ -53,37 +58,53 @@ func (c *Client) Explore(ctx context.Context, f Filter, tenantID string, start,
resp, err := c.do(req, http.StatusOK)
if err != nil {
exploreRequestsErrorsTotal.Inc()
exploreDuration.UpdateDuration(startTime)
return nil, fmt.Errorf("series request failed: %s", err)
}
var response Response
if err := json.NewDecoder(resp.Body).Decode(&response); err != nil {
exploreRequestsErrorsTotal.Inc()
exploreDuration.UpdateDuration(startTime)
return nil, fmt.Errorf("cannot decode series response: %s", err)
}
exploreDuration.UpdateDuration(startTime)
return response.MetricNames, resp.Body.Close()
}
// ImportPipe uses pipe reader in request to process data
func (c *Client) ImportPipe(ctx context.Context, dstURL string, pr *io.PipeReader) error {
startTime := time.Now()
importRequestsTotal.Inc()
req, err := http.NewRequestWithContext(ctx, http.MethodPost, dstURL, pr)
if err != nil {
importRequestsErrorsTotal.Inc()
return fmt.Errorf("cannot create import request to %q: %s", c.Addr, err)
}
importResp, err := c.do(req, http.StatusNoContent)
if err != nil {
importRequestsErrorsTotal.Inc()
importDuration.UpdateDuration(startTime)
return fmt.Errorf("import request failed: %s", err)
}
if err := importResp.Body.Close(); err != nil {
importRequestsErrorsTotal.Inc()
importDuration.UpdateDuration(startTime)
return fmt.Errorf("cannot close import response body: %s", err)
}
importDuration.UpdateDuration(startTime)
return nil
}
// ExportPipe makes request by provided filter and return io.ReadCloser which can be used to get data
func (c *Client) ExportPipe(ctx context.Context, url string, f Filter) (io.ReadCloser, error) {
startTime := time.Now()
exportRequestsTotal.Inc()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
exportRequestsErrorsTotal.Inc()
return nil, fmt.Errorf("cannot create request to %q: %s", c.Addr, err)
}
@@ -102,8 +123,11 @@ func (c *Client) ExportPipe(ctx context.Context, url string, f Filter) (io.ReadC
resp, err := c.do(req, http.StatusOK)
if err != nil {
exportRequestsErrorsTotal.Inc()
exportDuration.UpdateDuration(startTime)
return nil, fmt.Errorf("export request failed: %w", err)
}
exportDuration.UpdateDuration(startTime)
return resp.Body, nil
}
@@ -162,3 +186,16 @@ func (c *Client) do(req *http.Request, expSC int) (*http.Response, error) {
}
return resp, err
}
var (
importRequestsTotal = metrics.NewCounter(`vmctl_vm_native_requests_total{type="import"}`)
exportRequestsTotal = metrics.NewCounter(`vmctl_vm_native_requests_total{type="export"}`)
exploreRequestsTotal = metrics.NewCounter(`vmctl_vm_native_requests_total{type="explore"}`)
importRequestsErrorsTotal = metrics.NewCounter(`vmctl_vm_native_request_errors_total{type="import"}`)
exportRequestsErrorsTotal = metrics.NewCounter(`vmctl_vm_native_request_errors_total{type="export"}`)
exploreRequestsErrorsTotal = metrics.NewCounter(`vmctl_vm_native_request_errors_total{type="explore"}`)
importDuration = metrics.NewHistogram(`vmctl_vm_native_import_duration_seconds`)
exportDuration = metrics.NewHistogram(`vmctl_vm_native_export_duration_seconds`)
exploreDuration = metrics.NewHistogram(`vmctl_vm_native_explore_duration_seconds`)
)

View File

@@ -7,6 +7,8 @@ import (
"sync"
"time"
vmetrics "github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/opentsdb"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/vm"
"github.com/cheggaaa/pb/v3"
@@ -57,6 +59,7 @@ func (op *otsdbProcessor) run(ctx context.Context) error {
if !prompt(ctx, question) {
return nil
}
op.im.ResetStats()
var startTime int64
if op.oc.HardTS != 0 {
@@ -84,6 +87,7 @@ func (op *otsdbProcessor) run(ctx context.Context) error {
seriesCh := make(chan queryObj, op.otsdbcc)
errCh := make(chan error)
// we're going to make serieslist * queryRanges queries, so we should represent that in the progress bar
otsdbSeriesTotal.Add(len(serieslist) * queryRanges)
bar := pb.StartNew(len(serieslist) * queryRanges)
defer func(bar *pb.ProgressBar) {
bar.Finish()
@@ -93,9 +97,11 @@ func (op *otsdbProcessor) run(ctx context.Context) error {
wg.Go(func() {
for s := range seriesCh {
if err := op.do(s); err != nil {
otsdbErrorsTotal.Inc()
errCh <- fmt.Errorf("couldn't retrieve series for %s : %s", metric, err)
return
}
otsdbSeriesProcessed.Inc()
bar.Increment()
}
})
@@ -115,6 +121,7 @@ func (op *otsdbProcessor) run(ctx context.Context) error {
case otsdbErr := <-errCh:
return fmt.Errorf("opentsdb error: %s", otsdbErr)
case vmErr := <-op.im.Errors():
otsdbErrorsTotal.Inc()
return fmt.Errorf("import process failed: %s", wrapErr(vmErr, op.isVerbose))
case seriesCh <- queryObj{
Tr: tr, StartTime: startTime,
@@ -139,6 +146,7 @@ func (op *otsdbProcessor) run(ctx context.Context) error {
op.im.Close()
for vmErr := range op.im.Errors() {
if vmErr.Err != nil {
otsdbErrorsTotal.Inc()
return fmt.Errorf("import process failed: %s", wrapErr(vmErr, op.isVerbose))
}
}
@@ -169,3 +177,9 @@ func (op *otsdbProcessor) do(s queryObj) error {
}
return op.im.Input(&ts)
}
var (
otsdbSeriesTotal = vmetrics.NewCounter(`vmctl_opentsdb_migration_series_total`)
otsdbSeriesProcessed = vmetrics.NewCounter(`vmctl_opentsdb_migration_series_processed`)
otsdbErrorsTotal = vmetrics.NewCounter(`vmctl_opentsdb_migration_errors_total`)
)

View File

@@ -11,6 +11,8 @@ import (
"github.com/prometheus/prometheus/tsdb"
"github.com/prometheus/prometheus/tsdb/chunkenc"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/barpool"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/prometheus"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/vm"
@@ -113,6 +115,7 @@ func (pp *prometheusProcessor) do(b tsdb.BlockReader) error {
}
func (pp *prometheusProcessor) processBlocks(blocks []tsdb.BlockReader) error {
promBlocksTotal.Add(len(blocks))
bar := barpool.AddWithTemplate(fmt.Sprintf(barTpl, "Processing blocks"), len(blocks))
if err := barpool.Start(); err != nil {
return err
@@ -128,9 +131,11 @@ func (pp *prometheusProcessor) processBlocks(blocks []tsdb.BlockReader) error {
wg.Go(func() {
for br := range blockReadersCh {
if err := pp.do(br); err != nil {
promErrorsTotal.Inc()
errCh <- fmt.Errorf("read failed for block %q: %s", br.Meta().ULID, err)
return
}
promBlocksProcessed.Inc()
bar.Increment()
}
})
@@ -143,6 +148,7 @@ func (pp *prometheusProcessor) processBlocks(blocks []tsdb.BlockReader) error {
return fmt.Errorf("prometheus error: %s", promErr)
case vmErr := <-pp.im.Errors():
close(blockReadersCh)
promErrorsTotal.Inc()
return fmt.Errorf("import process failed: %s", wrapErr(vmErr, pp.isVerbose))
case blockReadersCh <- br:
}
@@ -156,6 +162,7 @@ func (pp *prometheusProcessor) processBlocks(blocks []tsdb.BlockReader) error {
// drain import errors channel
for vmErr := range pp.im.Errors() {
if vmErr.Err != nil {
promErrorsTotal.Inc()
return fmt.Errorf("import process failed: %s", wrapErr(vmErr, pp.isVerbose))
}
}
@@ -165,3 +172,9 @@ func (pp *prometheusProcessor) processBlocks(blocks []tsdb.BlockReader) error {
return nil
}
var (
promBlocksTotal = metrics.NewCounter(`vmctl_prometheus_migration_blocks_total`)
promBlocksProcessed = metrics.NewCounter(`vmctl_prometheus_migration_blocks_processed`)
promErrorsTotal = metrics.NewCounter(`vmctl_prometheus_migration_errors_total`)
)

View File

@@ -7,6 +7,8 @@ import (
"sync"
"time"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/barpool"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/remoteread"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/stepper"
@@ -51,6 +53,7 @@ func (rrp *remoteReadProcessor) run(ctx context.Context) error {
return nil
}
remoteReadRangesTotal.Add(len(ranges))
bar := barpool.AddWithTemplate(fmt.Sprintf(barTpl, "Processing ranges"), len(ranges))
if err := barpool.Start(); err != nil {
return err
@@ -70,9 +73,11 @@ func (rrp *remoteReadProcessor) run(ctx context.Context) error {
wg.Go(func() {
for r := range rangeC {
if err := rrp.do(ctx, r); err != nil {
remoteReadErrorsTotal.Inc()
errCh <- fmt.Errorf("request failed for: %s", err)
return
}
remoteReadRangesProcessed.Inc()
bar.Increment()
}
})
@@ -83,6 +88,7 @@ func (rrp *remoteReadProcessor) run(ctx context.Context) error {
case infErr := <-errCh:
return fmt.Errorf("remote read error: %s", infErr)
case vmErr := <-rrp.dst.Errors():
remoteReadErrorsTotal.Inc()
return fmt.Errorf("import process failed: %s", wrapErr(vmErr, rrp.isVerbose))
case rangeC <- &remoteread.Filter{
StartTimestampMs: r[0].UnixMilli(),
@@ -98,6 +104,7 @@ func (rrp *remoteReadProcessor) run(ctx context.Context) error {
// drain import errors channel
for vmErr := range rrp.dst.Errors() {
if vmErr.Err != nil {
remoteReadErrorsTotal.Inc()
return fmt.Errorf("import process failed: %s", wrapErr(vmErr, rrp.isVerbose))
}
}
@@ -118,3 +125,9 @@ func (rrp *remoteReadProcessor) do(ctx context.Context, filter *remoteread.Filte
return nil
})
}
var (
remoteReadRangesTotal = metrics.NewCounter(`vmctl_remote_read_migration_ranges_total`)
remoteReadRangesProcessed = metrics.NewCounter(`vmctl_remote_read_migration_ranges_processed`)
remoteReadErrorsTotal = metrics.NewCounter(`vmctl_remote_read_migration_errors_total`)
)

View File

@@ -12,6 +12,8 @@ import (
"sync"
"time"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/backoff"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/barpool"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/limiter"
@@ -80,6 +82,12 @@ type Importer struct {
s *stats
backoff *backoff.Backoff
importRequestsTotal *metrics.Counter
importRequestsErrorsTotal *metrics.Counter
importSamplesTotal *metrics.Counter
importBytesTotal *metrics.Counter
importDuration *metrics.Histogram
}
// ResetStats resets im stats.
@@ -147,6 +155,12 @@ func NewImporter(ctx context.Context, cfg Config) (*Importer, error) {
input: make(chan *TimeSeries, cfg.Concurrency*4),
errors: make(chan *ImportError, cfg.Concurrency),
backoff: cfg.Backoff,
importRequestsTotal: metrics.GetOrCreateCounter(`vmctl_importer_requests_total`),
importRequestsErrorsTotal: metrics.GetOrCreateCounter(`vmctl_importer_request_errors_total`),
importSamplesTotal: metrics.GetOrCreateCounter(`vmctl_importer_samples_total`),
importBytesTotal: metrics.GetOrCreateCounter(`vmctl_importer_bytes_total`),
importDuration: metrics.GetOrCreateHistogram(`vmctl_importer_request_duration_seconds`),
}
if err := im.Ping(); err != nil {
return nil, fmt.Errorf("ping to %q failed: %s", addr, err)
@@ -311,9 +325,13 @@ func (im *Importer) Import(tsBatch []*TimeSeries) error {
return nil
}
startTime := time.Now()
im.importRequestsTotal.Inc()
pr, pw := io.Pipe()
req, err := http.NewRequest(http.MethodPost, im.importPath, pr)
if err != nil {
im.importRequestsErrorsTotal.Inc()
return fmt.Errorf("cannot create request to %q: %s", im.addr, err)
}
if im.user != "" {
@@ -333,6 +351,7 @@ func (im *Importer) Import(tsBatch []*TimeSeries) error {
if im.compress {
zw, err := gzip.NewWriterLevel(w, 1)
if err != nil {
im.importRequestsErrorsTotal.Inc()
return fmt.Errorf("unexpected error when creating gzip writer: %s", err)
}
w = zw
@@ -344,29 +363,39 @@ func (im *Importer) Import(tsBatch []*TimeSeries) error {
for _, ts := range tsBatch {
n, err := ts.write(bw)
if err != nil {
im.importRequestsErrorsTotal.Inc()
return fmt.Errorf("write err: %w", err)
}
totalBytes += n
totalSamples += len(ts.Values)
}
if err := bw.Flush(); err != nil {
im.importRequestsErrorsTotal.Inc()
return err
}
if closer, ok := w.(io.Closer); ok {
err := closer.Close()
if err != nil {
im.importRequestsErrorsTotal.Inc()
return err
}
}
if err := pw.Close(); err != nil {
im.importRequestsErrorsTotal.Inc()
return err
}
requestErr := <-errCh
if requestErr != nil {
im.importRequestsErrorsTotal.Inc()
im.importDuration.UpdateDuration(startTime)
return fmt.Errorf("import request error for %q: %w", im.addr, requestErr)
}
im.importSamplesTotal.Add(totalSamples)
im.importBytesTotal.Add(totalBytes)
im.importDuration.UpdateDuration(startTime)
im.s.Lock()
im.s.bytes += uint64(totalBytes)
im.s.samples += uint64(totalSamples)

View File

@@ -9,6 +9,8 @@ import (
"sync"
"time"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/backoff"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/barpool"
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl/limiter"
@@ -82,13 +84,19 @@ func (p *vmNativeProcessor) run(ctx context.Context) error {
if !prompt(ctx, question) {
return nil
}
migrationTenantsTotal.Set(uint64(len(tenants)))
}
for _, tenantID := range tenants {
err := p.runBackfilling(ctx, tenantID, ranges)
if err != nil {
migrationErrorsTotal.Inc()
return fmt.Errorf("migration failed: %s", err)
}
if p.interCluster {
migrationTenantsProcessed.Inc()
}
}
log.Println("Import finished!")
@@ -156,6 +164,7 @@ func (p *vmNativeProcessor) runSingle(ctx context.Context, f native.Filter, srcU
p.s.bytes += uint64(written)
p.s.requests++
p.s.Unlock()
migrationBytesTransferredTotal.AddInt64(written)
if err := pw.Close(); err != nil {
return err
@@ -199,7 +208,7 @@ func (p *vmNativeProcessor) runBackfilling(ctx context.Context, tenantID string,
var foundSeriesMsg string
var requestsToMake int
var metrics = map[string][][]time.Time{
var metricsMap = map[string][][]time.Time{
"": ranges,
}
@@ -211,11 +220,11 @@ func (p *vmNativeProcessor) runBackfilling(ctx context.Context, tenantID string,
if !p.disablePerMetricRequests {
format = fmt.Sprintf(nativeWithBackoffTpl, barPrefix)
metrics, err = p.explore(ctx, p.src, tenantID, ranges)
metricsMap, err = p.explore(ctx, p.src, tenantID, ranges)
if err != nil {
return fmt.Errorf("failed to explore metric names: %s", err)
}
if len(metrics) == 0 {
if len(metricsMap) == 0 {
errMsg := "no metrics found"
if tenantID != "" {
errMsg = fmt.Sprintf("%s for tenant id: %s", errMsg, tenantID)
@@ -223,10 +232,14 @@ func (p *vmNativeProcessor) runBackfilling(ctx context.Context, tenantID string,
log.Println(errMsg)
return nil
}
for _, m := range metrics {
for _, m := range metricsMap {
requestsToMake += len(m)
}
foundSeriesMsg = fmt.Sprintf("Found %d unique metric names to import. Total import/export requests to make %d", len(metrics), requestsToMake)
foundSeriesMsg = fmt.Sprintf("Found %d unique metric names to import. Total import/export requests to make %d", len(metricsMap), requestsToMake)
migrationMetricsTotal.Add(len(metricsMap))
} else {
requestsToMake = len(ranges)
}
if !p.interCluster {
@@ -240,6 +253,7 @@ func (p *vmNativeProcessor) runBackfilling(ctx context.Context, tenantID string,
log.Print(foundSeriesMsg)
}
migrationRequestsPlanned.Add(requestsToMake)
bar := barpool.NewSingleProgress(format, requestsToMake)
bar.Start()
defer bar.Finish()
@@ -263,12 +277,13 @@ func (p *vmNativeProcessor) runBackfilling(ctx context.Context, tenantID string,
return
}
}
migrationRequestsCompleted.Inc()
}
})
}
// any error breaks the import
for mName, mRanges := range metrics {
for mName, mRanges := range metricsMap {
match, err := buildMatchWithFilter(p.filter.Match, mName)
if err != nil {
logger.Errorf("failed to build filter %q for metric name %q: %s", p.filter.Match, mName, err)
@@ -288,6 +303,9 @@ func (p *vmNativeProcessor) runBackfilling(ctx context.Context, tenantID string,
}:
}
}
if !p.disablePerMetricRequests {
migrationMetricsProcessed.Inc()
}
}
close(filterCh)
@@ -396,3 +414,18 @@ func buildMatchWithFilter(filter string, metricName string) (string, error) {
match := "{" + strings.Join(filters, " or ") + "}"
return match, nil
}
var (
migrationMetricsTotal = metrics.NewCounter(`vmctl_vm_native_migration_metrics_total`)
migrationMetricsProcessed = metrics.NewCounter(`vmctl_vm_native_migration_metrics_processed`)
migrationRequestsPlanned = metrics.NewCounter(`vmctl_vm_native_migration_requests_planned`)
migrationRequestsCompleted = metrics.NewCounter(`vmctl_vm_native_migration_requests_completed`)
migrationErrorsTotal = metrics.NewCounter(`vmctl_vm_native_migration_errors_total`)
migrationTenantsTotal = metrics.NewCounter(`vmctl_vm_native_migration_tenants_total`)
migrationTenantsProcessed = metrics.NewCounter(`vmctl_vm_native_migration_tenants_processed`)
migrationBytesTransferredTotal = metrics.NewCounter(`vmctl_vm_native_migration_bytes_transferred_total`)
)

View File

@@ -182,6 +182,7 @@ func (ctx *InsertCtx) WriteMetadata(mmpbs []prompb.MetricMetadata) error {
mm.Type = mmpb.Type
mm.Unit = bytesutil.ToUnsafeBytes(mmpb.Unit)
}
ctx.mms = mms
err := vmstorage.AddMetadataRows(mms)
if err != nil {
@@ -206,6 +207,7 @@ func (ctx *InsertCtx) WritePromMetadata(mmps []prometheus.Metadata) error {
mm.Help = bytesutil.ToUnsafeBytes(mmpb.Help)
mm.Type = mmpb.Type
}
ctx.mms = mms
err := vmstorage.AddMetadataRows(mms)
if err != nil {

View File

@@ -1713,6 +1713,7 @@ func evalRollupFuncNoCache(qt *querytracer.Tracer, ec *EvalConfig, funcName stri
return nil, err
}
defer rml.Put(uint64(rollupMemorySize))
qs.addMemoryUsage(rollupMemorySize)
qt.Printf("the rollup evaluation needs an estimated %d bytes of RAM for %d series and %d points per series (summary %d points)",
rollupMemorySize, timeseriesLen, pointsPerSeries, rollupPoints)

View File

@@ -37,7 +37,7 @@ func Exec(qt *querytracer.Tracer, ec *EvalConfig, q string, isFirstPointOnly boo
if querystats.Enabled() {
startTime := time.Now()
defer func() {
querystats.RegisterQuery(q, ec.End-ec.Start, startTime)
querystats.RegisterQuery(q, ec.End-ec.Start, startTime, ec.QueryStats.memoryUsage())
ec.QueryStats.addExecutionTimeMsec(startTime)
}()
}

View File

@@ -13,6 +13,8 @@ type QueryStats struct {
ExecutionDuration atomic.Pointer[time.Duration]
// SeriesFetched contains the number of series fetched from storage or cache.
SeriesFetched atomic.Int64
// MemoryUsage contains the estimated memory consumption of the query
MemoryUsage atomic.Int64
at *auth.Token
@@ -53,3 +55,17 @@ func (qs *QueryStats) addExecutionTimeMsec(startTime time.Time) {
d := time.Since(startTime)
qs.ExecutionDuration.Store(&d)
}
func (qs *QueryStats) addMemoryUsage(memoryUsage int64) {
if qs == nil {
return
}
qs.MemoryUsage.Store(memoryUsage)
}
func (qs *QueryStats) memoryUsage() int64 {
if qs == nil {
return 0
}
return qs.MemoryUsage.Load()
}

View File

@@ -8,6 +8,7 @@ import (
"sync"
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/flagutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/stringsutil"
)
@@ -15,7 +16,8 @@ import (
var (
lastQueriesCount = flag.Int("search.queryStats.lastQueriesCount", 20000, "Query stats for /api/v1/status/top_queries is tracked on this number of last queries. "+
"Zero value disables query stats tracking")
minQueryDuration = flag.Duration("search.queryStats.minQueryDuration", time.Millisecond, "The minimum duration for queries to track in query stats at /api/v1/status/top_queries. Queries with lower duration are ignored in query stats")
minQueryDuration = flag.Duration("search.queryStats.minQueryDuration", time.Millisecond, "The minimum duration for queries to track in query stats at /api/v1/status/top_queries. Queries with lower duration are ignored in query stats")
minQueryMemoryUsage = flagutil.NewBytes("search.queryStats.minQueryMemoryUsage", 1024, "The minimum memory bytes consumption for queries to track in query stats at /api/v1/status/top_queries. Queries with lower memory bytes consumption are ignored in query stats")
)
var (
@@ -31,9 +33,9 @@ func Enabled() bool {
// RegisterQuery registers the query on the given timeRangeMsecs, which has been started at startTime.
//
// RegisterQuery must be called when the query is finished.
func RegisterQuery(query string, timeRangeMsecs int64, startTime time.Time) {
func RegisterQuery(query string, timeRangeMsecs int64, startTime time.Time, memoryUsage int64) {
initOnce.Do(initQueryStats)
qsTracker.registerQuery(query, timeRangeMsecs, startTime)
qsTracker.registerQuery(query, timeRangeMsecs, startTime, memoryUsage)
}
// WriteJSONQueryStats writes query stats to given writer in json format.
@@ -54,6 +56,7 @@ type queryStatRecord struct {
timeRangeSecs int64
registerTime time.Time
duration time.Duration
memoryUsage int64
}
type queryStatKey struct {
@@ -66,8 +69,8 @@ func initQueryStats() {
if recordsCount <= 0 {
recordsCount = 1
} else {
logger.Infof("enabled query stats tracking at `/api/v1/status/top_queries` with -search.queryStats.lastQueriesCount=%d, -search.queryStats.minQueryDuration=%s",
*lastQueriesCount, *minQueryDuration)
logger.Infof("enabled query stats tracking at `/api/v1/status/top_queries` with -search.queryStats.lastQueriesCount=%d, -search.queryStats.minQueryDuration=%s, -search.queryStats.minQueryMemoryUsage=%s",
*lastQueriesCount, *minQueryDuration, minQueryMemoryUsage)
}
qsTracker = &queryStatsTracker{
a: make([]queryStatRecord, recordsCount),
@@ -78,6 +81,7 @@ func (qst *queryStatsTracker) writeJSONQueryStats(w io.Writer, topN int, maxLife
fmt.Fprintf(w, `{"topN":"%d","maxLifetime":"%s",`, topN, maxLifetime)
fmt.Fprintf(w, `"search.queryStats.lastQueriesCount":%d,`, *lastQueriesCount)
fmt.Fprintf(w, `"search.queryStats.minQueryDuration":"%s",`, *minQueryDuration)
fmt.Fprintf(w, `"search.queryStats.minQueryMemoryUsage":"%s",`, minQueryMemoryUsage)
fmt.Fprintf(w, `"topByCount":[`)
topByCount := qst.getTopByCount(topN, maxLifetime)
for i, r := range topByCount {
@@ -102,15 +106,28 @@ func (qst *queryStatsTracker) writeJSONQueryStats(w io.Writer, topN int, maxLife
fmt.Fprintf(w, `,`)
}
}
fmt.Fprintf(w, `],"topByAvgMemoryUsage":[`)
topByAvgMemoryConsumption := qst.getTopByAvgMemoryUsage(topN, maxLifetime)
for i, r := range topByAvgMemoryConsumption {
fmt.Fprintf(w, `{"query":%s,"timeRangeSeconds":%d,"avgMemoryBytes":%d,"count":%d}`, stringsutil.JSONString(r.query), r.timeRangeSecs, r.memoryUsage, r.count)
if i+1 < len(topByAvgMemoryConsumption) {
fmt.Fprintf(w, `,`)
}
}
fmt.Fprintf(w, `]}`)
}
func (qst *queryStatsTracker) registerQuery(query string, timeRangeMsecs int64, startTime time.Time) {
func (qst *queryStatsTracker) registerQuery(query string, timeRangeMsecs int64, startTime time.Time, memoryUsage int64) {
registerTime := time.Now()
duration := registerTime.Sub(startTime)
if duration < *minQueryDuration {
return
}
if memoryUsage < int64(minQueryMemoryUsage.IntN()) {
return
}
qst.mu.Lock()
defer qst.mu.Unlock()
@@ -126,6 +143,7 @@ func (qst *queryStatsTracker) registerQuery(query string, timeRangeMsecs int64,
r.timeRangeSecs = timeRangeMsecs / 1000
r.registerTime = registerTime
r.duration = duration
r.memoryUsage = memoryUsage
}
func (r *queryStatRecord) matches(currentTime time.Time, maxLifetime time.Duration) bool {
@@ -257,3 +275,47 @@ func (qst *queryStatsTracker) getTopBySumDuration(topN int, maxLifetime time.Dur
}
return a
}
type queryStatByMemory struct {
query string
timeRangeSecs int64
memoryUsage int64
count int
}
func (qst *queryStatsTracker) getTopByAvgMemoryUsage(topN int, maxLifetime time.Duration) []queryStatByMemory {
currentTime := time.Now()
qst.mu.Lock()
type countSum struct {
count int
sum int64
}
m := make(map[queryStatKey]countSum)
for _, r := range qst.a {
if r.matches(currentTime, maxLifetime) {
k := r.key()
ks := m[k]
ks.count++
ks.sum += r.memoryUsage
m[k] = ks
}
}
qst.mu.Unlock()
var a []queryStatByMemory
for k, ks := range m {
a = append(a, queryStatByMemory{
query: k.query,
timeRangeSecs: k.timeRangeSecs,
memoryUsage: ks.sum / int64(ks.count),
count: ks.count,
})
}
sort.Slice(a, func(i, j int) bool {
return a[i].memoryUsage > a[j].memoryUsage
})
if len(a) > topN {
a = a[:topN]
}
return a
}

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -37,10 +37,10 @@
<meta property="og:title" content="UI for VictoriaMetrics">
<meta property="og:url" content="https://victoriametrics.com/">
<meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data">
<script type="module" crossorigin src="./assets/index-B6lol36n.js"></script>
<script type="module" crossorigin src="./assets/index-C4RD5Sxk.js"></script>
<link rel="modulepreload" crossorigin href="./assets/vendor-EZef-S_8.js">
<link rel="stylesheet" crossorigin href="./assets/vendor-D1GxaB_c.css">
<link rel="stylesheet" crossorigin href="./assets/index-VQRcNK83.css">
<link rel="stylesheet" crossorigin href="./assets/index-D7CzMv1O.css">
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>

View File

@@ -1,4 +1,4 @@
FROM golang:1.25.6 AS build-web-stage
FROM golang:1.25.7 AS build-web-stage
COPY build /build
WORKDIR /build

View File

@@ -2,9 +2,6 @@
copy-metricsql-docs:
cp docs/victoriametrics/MetricsQL.md app/vmui/packages/vmui/src/assets/MetricsQL.md
vmui-package-base-image:
docker build -t vmui-builder-image -f app/vmui/Dockerfile-build ./app/vmui
vmui-run-npm-command: vmui-package-base-image
docker run --rm \
--user $(shell id -u):$(shell id -g) \

View File

@@ -3,11 +3,11 @@
DOCKER_REGISTRIES ?= docker.io quay.io
DOCKER_NAMESPACE ?= victoriametrics
ROOT_IMAGE ?= alpine:3.23.2
ROOT_IMAGE ?= alpine:3.23.3
ROOT_IMAGE_SCRATCH ?= scratch
CERTS_IMAGE := alpine:3.23.2
CERTS_IMAGE := alpine:3.23.3
GO_BUILDER_IMAGE := golang:1.25.6
GO_BUILDER_IMAGE := golang:1.25.7
BUILDER_IMAGE := local/builder:2.0.0-$(shell echo $(GO_BUILDER_IMAGE) | tr :/ __)-1
BASE_IMAGE := local/base:1.1.4-$(shell echo $(ROOT_IMAGE) | tr :/ __)-$(shell echo $(CERTS_IMAGE) | tr :/ __)

View File

@@ -3,7 +3,7 @@ services:
# It scrapes targets defined in --promscrape.config
# And forward them to --remoteWrite.url
vmagent:
image: victoriametrics/vmagent:v1.134.0
image: victoriametrics/vmagent:v1.135.0
depends_on:
- "vmauth"
ports:
@@ -37,14 +37,14 @@ services:
# vmstorage shards. Each shard receives 1/N of all metrics sent to vminserts,
# where N is number of vmstorages (2 in this case).
vmstorage-1:
image: victoriametrics/vmstorage:v1.134.0-cluster
image: victoriametrics/vmstorage:v1.135.0-cluster
volumes:
- strgdata-1:/storage
command:
- "--storageDataPath=/storage"
restart: always
vmstorage-2:
image: victoriametrics/vmstorage:v1.134.0-cluster
image: victoriametrics/vmstorage:v1.135.0-cluster
volumes:
- strgdata-2:/storage
command:
@@ -54,7 +54,7 @@ services:
# vminsert is ingestion frontend. It receives metrics pushed by vmagent,
# pre-process them and distributes across configured vmstorage shards.
vminsert-1:
image: victoriametrics/vminsert:v1.134.0-cluster
image: victoriametrics/vminsert:v1.135.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -63,7 +63,7 @@ services:
- "--storageNode=vmstorage-2:8400"
restart: always
vminsert-2:
image: victoriametrics/vminsert:v1.134.0-cluster
image: victoriametrics/vminsert:v1.135.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -75,7 +75,7 @@ services:
# vmselect is a query fronted. It serves read queries in MetricsQL or PromQL.
# vmselect collects results from configured `--storageNode` shards.
vmselect-1:
image: victoriametrics/vmselect:v1.134.0-cluster
image: victoriametrics/vmselect:v1.135.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -85,7 +85,7 @@ services:
- "--vmalert.proxyURL=http://vmalert:8880"
restart: always
vmselect-2:
image: victoriametrics/vmselect:v1.134.0-cluster
image: victoriametrics/vmselect:v1.135.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -100,7 +100,7 @@ services:
# read requests from Grafana, vmui, vmalert among vmselects.
# It can be used as an authentication proxy.
vmauth:
image: victoriametrics/vmauth:v1.134.0
image: victoriametrics/vmauth:v1.135.0
depends_on:
- "vmselect-1"
- "vmselect-2"
@@ -114,7 +114,7 @@ services:
# vmalert executes alerting and recording rules
vmalert:
image: victoriametrics/vmalert:v1.134.0
image: victoriametrics/vmalert:v1.135.0
depends_on:
- "vmauth"
ports:

View File

@@ -3,7 +3,7 @@ services:
# It scrapes targets defined in --promscrape.config
# And forward them to --remoteWrite.url
vmagent:
image: victoriametrics/vmagent:v1.134.0
image: victoriametrics/vmagent:v1.135.0
depends_on:
- "victoriametrics"
ports:
@@ -18,7 +18,7 @@ services:
# VictoriaMetrics instance, a single process responsible for
# storing metrics and serve read requests.
victoriametrics:
image: victoriametrics/victoria-metrics:v1.134.0
image: victoriametrics/victoria-metrics:v1.135.0
ports:
- 8428:8428
- 8089:8089
@@ -54,7 +54,7 @@ services:
# vmalert executes alerting and recording rules
vmalert:
image: victoriametrics/vmalert:v1.134.0
image: victoriametrics/vmalert:v1.135.0
depends_on:
- "victoriametrics"
- "alertmanager"

View File

@@ -1,6 +1,6 @@
services:
vmagent:
image: victoriametrics/vmagent:v1.134.0
image: victoriametrics/vmagent:v1.135.0
depends_on:
- "victoriametrics"
ports:
@@ -14,7 +14,7 @@ services:
restart: always
victoriametrics:
image: victoriametrics/victoria-metrics:v1.134.0
image: victoriametrics/victoria-metrics:v1.135.0
ports:
- 8428:8428
volumes:
@@ -40,7 +40,7 @@ services:
restart: always
vmalert:
image: victoriametrics/vmalert:v1.134.0
image: victoriametrics/vmalert:v1.135.0
depends_on:
- "victoriametrics"
ports:

View File

@@ -10,9 +10,9 @@ sitemap:
- To use *vmanomaly*, part of the enterprise package, a license key is required. Obtain your key [here](https://victoriametrics.com/products/enterprise/trial/) for this tutorial or for enterprise use.
- In the tutorial, we'll be using the following VictoriaMetrics components:
- [VictoriaMetrics Single-Node](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) (v1.134.0)
- [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/) (v1.134.0)
- [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) (v1.134.0)
- [VictoriaMetrics Single-Node](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) (v1.135.0)
- [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/) (v1.135.0)
- [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) (v1.135.0)
- [Grafana](https://grafana.com/) (v.10.2.1)
- [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/)
- [Node exporter](https://github.com/prometheus/node_exporter#node-exporter) (v1.7.0) and [Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/) (v0.27.0)
@@ -323,7 +323,7 @@ Let's wrap it all up together into the `docker-compose.yml` file.
services:
vmagent:
container_name: vmagent
image: victoriametrics/vmagent:v1.134.0
image: victoriametrics/vmagent:v1.135.0
depends_on:
- "victoriametrics"
ports:
@@ -340,7 +340,7 @@ services:
victoriametrics:
container_name: victoriametrics
image: victoriametrics/victoria-metrics:v1.134.0
image: victoriametrics/victoria-metrics:v1.135.0
ports:
- 8428:8428
volumes:
@@ -373,7 +373,7 @@ services:
vmalert:
container_name: vmalert
image: victoriametrics/vmalert:v1.134.0
image: victoriametrics/vmalert:v1.135.0
depends_on:
- "victoriametrics"
ports:

View File

@@ -37,12 +37,6 @@ config:
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels: [__meta_kubernetes_node_name]
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/$1/proxy/metrics
- job_name: "kubernetes-nodes-cadvisor"
scheme: https
tls_config:
@@ -51,15 +45,12 @@ config:
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
kubernetes_sd_configs:
- role: node
metrics_path: /metrics/cadvisor
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels: [__meta_kubernetes_node_name]
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/$1/proxy/metrics/cadvisor
- source_labels: [__metrics_path__]
target_label: metrics_path
metric_relabel_configs:
- action: replace
source_labels: [pod]

View File

@@ -37,12 +37,6 @@ server:
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels: [ __meta_kubernetes_node_name ]
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/$1/proxy/metrics
- job_name: "kubernetes-nodes-cadvisor"
scheme: https
tls_config:
@@ -51,15 +45,12 @@ server:
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
kubernetes_sd_configs:
- role: node
metrics_path: /metrics/cadvisor
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels: [ __meta_kubernetes_node_name ]
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/$1/proxy/metrics/cadvisor
- source_labels: [__metrics_path__]
target_label: metrics_path
metric_relabel_configs:
- action: replace
source_labels: [pod]
@@ -78,4 +69,4 @@ server:
source_labels: [id]
regex: '^/system\.slice/(.+)\.service$'
target_label: systemd_service_name
replacement: '${1}'
replacement: '${1}'

View File

@@ -249,27 +249,27 @@ services:
- grafana_data:/var/lib/grafana/
vmsingle:
image: victoriametrics/victoria-metrics:v1.134.0
image: victoriametrics/victoria-metrics:v1.135.0
command:
- -httpListenAddr=0.0.0.0:8429
vmstorage:
image: victoriametrics/vmstorage:v1.134.0-cluster
image: victoriametrics/vmstorage:v1.135.0-cluster
vminsert:
image: victoriametrics/vminsert:v1.134.0-cluster
image: victoriametrics/vminsert:v1.135.0-cluster
command:
- -storageNode=vmstorage:8400
- -httpListenAddr=0.0.0.0:8480
vmselect:
image: victoriametrics/vmselect:v1.134.0-cluster
image: victoriametrics/vmselect:v1.135.0-cluster
command:
- -storageNode=vmstorage:8401
- -httpListenAddr=0.0.0.0:8481
vmagent:
image: victoriametrics/vmagent:v1.134.0
image: victoriametrics/vmagent:v1.135.0
volumes:
- ./scrape.yaml:/etc/vmagent/config.yaml
command:
@@ -278,7 +278,7 @@ services:
- -remoteWrite.url=http://vmsingle:8429/api/v1/write
vmgateway-cluster:
image: victoriametrics/vmgateway:v1.134.0-enterprise
image: victoriametrics/vmgateway:v1.135.0-enterprise
ports:
- 8431:8431
volumes:
@@ -294,7 +294,7 @@ services:
- -auth.oidcDiscoveryEndpoints=http://keycloak:8080/realms/master/.well-known/openid-configuration
vmgateway-single:
image: victoriametrics/vmgateway:v1.134.0-enterprise
image: victoriametrics/vmgateway:v1.135.0-enterprise
ports:
- 8432:8431
volumes:
@@ -405,7 +405,7 @@ Once iDP configuration is done, vmagent configuration needs to be updated to use
```yaml
vmagent:
image: victoriametrics/vmagent:v1.134.0
image: victoriametrics/vmagent:v1.135.0
volumes:
- ./scrape.yaml:/etc/vmagent/config.yaml
- ./vmagent-client-secret:/etc/vmagent/oauth2-client-secret

View File

@@ -187,12 +187,6 @@ scrape_configs:
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels: [__meta_kubernetes_node_name]
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/$1/proxy/metrics
- job_name: "kubernetes-nodes-cadvisor"
scheme: https
tls_config:
@@ -201,15 +195,12 @@ scrape_configs:
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
kubernetes_sd_configs:
- role: node
metrics_path: /metrics/cadvisor
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels: [__meta_kubernetes_node_name]
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/$1/proxy/metrics/cadvisor
- source_labels: [__metrics_path__]
target_label: metrics_path
metric_relabel_configs:
- action: replace
source_labels: [pod]

View File

@@ -213,12 +213,6 @@ config:
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels: [__meta_kubernetes_node_name]
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/$1/proxy/metrics
- job_name: "kubernetes-nodes-cadvisor"
scheme: https
tls_config:
@@ -227,15 +221,12 @@ config:
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
kubernetes_sd_configs:
- role: node
metrics_path: /metrics/cadvisor
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels: [__meta_kubernetes_node_name]
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/$1/proxy/metrics/cadvisor
- source_labels: [__metrics_path__]
target_label: metrics_path
metric_relabel_configs:
- action: replace
source_labels: [pod]

View File

@@ -114,12 +114,6 @@ server:
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels: [ __meta_kubernetes_node_name ]
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/$1/proxy/metrics
- job_name: "kubernetes-nodes-cadvisor"
scheme: https
tls_config:
@@ -128,15 +122,12 @@ server:
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
kubernetes_sd_configs:
- role: node
metrics_path: /metrics/cadvisor
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels: [ __meta_kubernetes_node_name ]
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/$1/proxy/metrics/cadvisor
- source_labels: [__metrics_path__]
target_label: metrics_path
metric_relabel_configs:
- action: replace
source_labels: [pod]

View File

@@ -286,7 +286,8 @@ Source code for Victoriametrics can be found in the following locations:
VictoriaMetrics is able to handle data from hundreds of millions of IoT sensors and industrial sensors.
It supports [high cardinality data](https://medium.com/@valyala/high-cardinality-tsdb-benchmarks-victoriametrics-vs-timescaledb-vs-influxdb-13e6ee64dd6b),
perfectly [scales up on a single node](https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae)
and scales horizontally to multiple nodes.
and scales horizontally to multiple nodes in [cluster setup](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/).
It also supports an option for reducing the index size for IoT data - see [these docs](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#index-tuning-for-low-churn-rate).
## What is the difference between single-node and cluster versions of VictoriaMetrics?

View File

@@ -58,9 +58,9 @@ Download the newest available [VictoriaMetrics release](https://docs.victoriamet
from [DockerHub](https://hub.docker.com/r/victoriametrics/victoria-metrics) or [Quay](https://quay.io/repository/victoriametrics/victoria-metrics?tab=tags):
```sh
docker pull victoriametrics/victoria-metrics:v1.134.0
docker pull victoriametrics/victoria-metrics:v1.135.0
docker run -it --rm -v `pwd`/victoria-metrics-data:/victoria-metrics-data -p 8428:8428 \
victoriametrics/victoria-metrics:v1.134.0 --selfScrapeInterval=5s -storageDataPath=victoria-metrics-data
victoriametrics/victoria-metrics:v1.135.0 --selfScrapeInterval=5s -storageDataPath=victoria-metrics-data
```
_For Enterprise images see [this link](https://docs.victoriametrics.com/victoriametrics/enterprise/#docker-images)._

View File

@@ -93,8 +93,10 @@ VictoriaMetrics has the following prominent features:
* It supports metrics [relabeling](#relabeling).
* It can deal with [high cardinality issues](https://docs.victoriametrics.com/victoriametrics/faq/#what-is-high-cardinality) and
[high churn rate](https://docs.victoriametrics.com/victoriametrics/faq/#what-is-high-churn-rate) issues via [series limiter](#cardinality-limiter).
* It ideally works with big amounts of time series data from APM, Kubernetes, IoT sensors, connected cars, industrial telemetry, financial data
and various [Enterprise workloads](https://docs.victoriametrics.com/victoriametrics/enterprise/).
* It ideally works for big amounts of time series with both [high churn rate](https://docs.victoriametrics.com/victoriametrics/faq/#what-is-high-churn-rate) (APM, Kubernetes)
and low churn rate (IoT sensors, connected cars, industrial telemetry, financial data - see
[these docs](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#index-tuning-for-low-churn-rate)),
plus various [Enterprise workloads](https://docs.victoriametrics.com/victoriametrics/enterprise/).
* It has an open source [cluster version](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster).
* It can store data on [NFS-based storages](https://en.wikipedia.org/wiki/Network_File_System) such as [Amazon EFS](https://aws.amazon.com/efs/)
and [Google Filestore](https://cloud.google.com/filestore).
@@ -1671,6 +1673,9 @@ See [Why IndexDB size is so large?](https://docs.victoriametrics.com/victoriamet
Downsampling is performed during [background merges](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#storage).
It cannot be performed if there is not enough of free disk space or if vmstorage is in [read-only mode](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#readonly-mode).
Downsampling period changes `/api/v1/export` API output. During query requests, if export `start` period is not specified and `reduce_mem_usage` param is omitted, the biggest `downsampling.period` is applied.
As an example, export request `/api/v1/export?match[]=series` with `-downsampling.period=30d:1h,180d:24h` will return samples downsampled with `24h` interval.
It's expected that resource usage will temporarily increase when **downsampling with filters** is applied.
This is because additional operations are required to read historical data, downsample, and persist it back,
which will cost extra CPU and memory.

View File

@@ -25,14 +25,28 @@ The sandbox cluster installation runs under the constant load generated by
See also [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-releases/).
## tip
* SECURITY: upgrade Go builder from Go1.25.6 to Go1.25.7. See [the list of issues addressed in Go1.25.7](https://github.com/golang/go/issues?q=milestone%3AGo1.25.7%20label%3ACherryPickApproved).
* SECURITY: upgrade base docker image (Alpine) from 3.23.2 to 3.23.3. See [Alpine 3.23.3 release notes](https://www.alpinelinux.org/posts/Alpine-3.20.9-3.21.6-3.22.3-3.23.3-released.html).
* BUGFIX: all VictoriaMetrics components: respect default http client proxy env variables (HTTP_PROXY,HTTPS_PROXY,NO_PROXY). See [#10385](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10385). Thanks to @zane-deg for the contribution.
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent/) and [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/): properly expose `kubernetes_sd` discovery network dialer metrics `vm_promscrape_discovery_kubernetes_conn_*`. See [#10382](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10382).
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/): slightly reduce memory usage for [metrics-metadata](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#metrics-metadata) ingestion. See [#10392](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10392).
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent/) and [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/): introduce timebased manual offset commit for kafka consumer to fix performance degradation with enabled manual commit. After this change, it will commit partition offsets in batch per second to avoid high commit QPS on the Kafka broker. It's no longer recommended to set `enable.auto.commit=true` in `-kafka.consumer.topic.options`, as `vmagent` will automatically manage it. See [#10395](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10395).
## [v1.135.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.135.0)
Released at 2026-01-30
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent/) and [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/): improved scrape size display. Sizes below 1024 bytes are now shown in `B`, and larger sizes are shown as whole `KiB` (rounded up). This prevents confusion where values like 123.456 KiB were interpreted as 123456 KiB, while the actual size was only 123 KiB. See [#10307](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10307).
* FEATURE: [vmauth](https://docs.victoriametrics.com/victoriametrics/vmauth/): allow buffering request bodies before proxying them to backends. This reduces load on backends when processing requests from slow clients such as IoT devices connected to `vmauth` via slow networks. See [#10309](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10309) and [request body buffering docs](https://docs.victoriametrics.com/victoriametrics/vmauth/#request-body-buffering).
* FEATURE: [vmbackupmanager](https://docs.victoriametrics.com/victoriametrics/vmbackupmanager/): allow completely disabling scheduled backups by using `-disableScheduledBackups` command-line flag. This is useful to run `vmbackupmanager` only for on-demand backups and restores triggered via API. See [#10364](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10364).
* FEATURE: [monitoring](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#monitoring): take into account all the generated log messages at `vm_log_messages_total` metric, including suppressed logs if the `-loggerLevel` command-line flag is set to values other than `INFO`. Add `is_printed` label to the `vm_log_messages_total` metric in order to understand whether the log at the given code `location` was suppressed or not. This simplifies troubleshooting of VictoriaMetrics components when logs aren't available. See [#10304](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10304).
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent/): support configuring different `-remoteWrite.queues` per remoteWrite url. This allows setting `-remoteWrite.queues=1` for backends that do not support out-of-order ingestion (e.g. Mimir), while keeping higher queue counts for other backends such as VictoriaMetrics. Previously, this required running multiple vmagent instances with different queue settings. See [#10270](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10270).
* FEATURE: [alerts](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/rules): add new alerting rules `PersistentQueueRunsOutOfSpaceIn12Hours` and `PersistentQueueRunsOutOfSpaceIn4Hours` for `vmagent` persistent queue capacity. These alerts help users to take proactive actions before `vmagent` starts dropping metrics due to insufficient persistent queue space. See [#10193](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10193)
* FEATURE: All VictoriaMetrics components: add build version information to the home page for consistency with other projects. See [#10249](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10249).
* FEATURE: all VictoriaMetrics components: add flag `fs.disableMincore`, which allows to disable `mincore` syscall. See [#10327](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10327).
* FEATURE: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): expose topN average memory bytes consumption queries in `/api/v1/status/top_queries`. It can help users to find queries that consume a lot of memory and potentially cause OOM. See [#9330](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9330).
* FEATURE: [vmctl](https://docs.victoriametrics.com/victoriametrics/vmctl/): add metrics for tracking the migration progress. See [vmctl - monitoring the migration process](https://docs.victoriametrics.com/victoriametrics/vmctl/#monitoring-the-migration-process) and [#10276](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10276).
* BUGFIX: [vmauth](https://docs.victoriametrics.com/victoriametrics/vmauth/): stop backend health checks for URL prefixes defined in `url_map` during configuration reloads. Previously, stale backends kept being health-checked and produced repeated warning logs after reloads. See [#10334](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10334).
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): properly return [/api/v1/status/tsdb](https://docs.victoriametrics.com/victoriametrics/#tsdb-stats) response for time range outside [partition index](https://docs.victoriametrics.com/victoriametrics/#indexdb). See [#10315](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10315).
@@ -41,7 +55,10 @@ See also [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-rel
* BUGFIX: [vmui](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui): fix "Percentage from total" calculation on the Cardinality Explorer page when multiple metrics match the filter. See [#10323](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10323). Thanks to @PleasingFungus for the contribution.
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): apply `-promscrape.maxScrapeSize` check to decompressed data instead of compressed data. See [#9481](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9481).
* BUGFIX: [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): disallow setting the `-notifier.url` command-line flag to a null value. See [#10355](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10355).
* BUGFIX: [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): fix [alert restore](https://docs.victoriametrics.com/victoriametrics/vmalert/#alerts-state-on-restarts) when a group contains many rules and is slow to complete evaluation. Previously, the restore process might not retrieve the correct previous alert state. See [#10335](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10335).
* BUGFIX: [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): do not skip sending alert notifications to `-notifier.url` if remote write requests to `-remoteWrite.url` fail. See [#10376](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10376).
* BUGFIX: [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/): fix `changes()` function when gaps between samples exceed the lookbehind window. Previously, it could yield a non-zero value even when the sample value remained unchanged. See [#10280](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10280).
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): previously ingestion could hit lock contention that triggered frequent context switches and storage connection saturation spikes; now the contention is removed to keep ingestion steady. See [#10367](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10367).
## [v1.134.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.134.0)
@@ -74,6 +91,10 @@ Released at 2026-01-16
Released at 2026-01-02
**Update Note 1:** [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): Upgrading to per-partition index requires registering all active time series. Expect slow down of data ingestion and queries during upgrade roll-out. This is a one-time operation. Additionally, for users with retention periods shorter than 1 month the disk usage may increase.
**Update Note 2:** [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): Certain data and query patterns may cause high CPU utilization due to [10154](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10154). See [10297](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10297).
**Update Note 3:** [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): lock contention in the ingestion path may cause frequent context switches and storage connection saturation spikes. See [10367](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10367). Addressed in `v1.135.0`.
**Update Note 4:** [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): TSDB status may be empty if the partition index does not have records for the requested date. See [10315](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10315). Addressed in `v1.135.0`.
**Update Note 5:** [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): `indexdb/tagFiltersToMetricIDs`, `indexdb/metricID` and `indexdb/date_metricID` cache metrics are not reported properly. See [10275](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10275). Addressed in `v1.135.0`.
* SECURITY: upgrade base docker image (Alpine) from 3.22.2 to 3.23.2. See [Alpine 3.23.2 release notes](https://www.alpinelinux.org/posts/Alpine-3.23.2-released.html).
@@ -139,6 +160,22 @@ See changes [here](https://docs.victoriametrics.com/victoriametrics/changelog/ch
See changes [here](https://docs.victoriametrics.com/victoriametrics/changelog/changelog_2025/#v11230)
## [v1.122.14](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.122.14)
Released at 2026-01-30
**v1.122.x is a line of [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-releases/). It contains important up-to-date bugfixes for [VictoriaMetrics enterprise](https://docs.victoriametrics.com/victoriametrics/enterprise/).
All these fixes are also included in [the latest community release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest).
The v1.122.x line will be supported for at least 12 months since [v1.122.0](https://docs.victoriametrics.com/victoriametrics/changelog/#v11220) release**
* BUGFIX: [vmauth](https://docs.victoriametrics.com/victoriametrics/vmauth/): stop backend health checks for URL prefixes defined in `url_map` during configuration reloads. Previously, stale backends kept being health-checked and produced repeated warning logs after reloads. See [#10334](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10334).
* BUGFIX: `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): correctly return tenants results for `/admin/tenants` when `start` or `end` are specified. See [#10312](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10312)
* BUGFIX: [vmui](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui): fix "Percentage from total" calculation on the Cardinality Explorer page when multiple metrics match the filter. See [#10323](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10323). Thanks to @PleasingFungus for the contribution.
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): apply `-promscrape.maxScrapeSize` check to decompressed data instead of compressed data. See [#9481](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9481).
* BUGFIX: [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): fix [alert restore](https://docs.victoriametrics.com/victoriametrics/vmalert/#alerts-state-on-restarts) when a group contains many rules and is slow to complete evaluation. Previously, the restore process might not retrieve the correct previous alert state. See [#10335](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10335).
* BUGFIX: [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/): fix `changes()` function when gaps between samples exceed the lookbehind window. Previously, it could yield a non-zero value even when the sample value remained unchanged. See [#10280](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10280).
* BUGFIX: [vmauth](https://docs.victoriametrics.com/victoriametrics/vmauth/): fix an issue where canceling a client request (closing a browser tab or timeout) incorrectly marked all backends as unavailable for `-failTimeout` duration (3s by default), even though backends were healthy. See [#10318](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10318).
## [v1.122.13](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.122.13)
Released at 2026-01-16
@@ -254,6 +291,21 @@ See changes [here](https://docs.victoriametrics.com/victoriametrics/changelog/ch
See changes [here](https://docs.victoriametrics.com/victoriametrics/changelog/changelog_2025/#v11110)
## [v1.110.29](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.110.29)
Released at 2026-01-30
**v1.110.x is a line of [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-releases/). It contains important up-to-date bugfixes for [VictoriaMetrics enterprise](https://docs.victoriametrics.com/victoriametrics/enterprise/).
All these fixes are also included in [the latest community release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest).
The v1.110.x line will be supported for at least 12 months since [v1.110.0](https://docs.victoriametrics.com/victoriametrics/changelog/#v11100) release**
* BUGFIX: [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/): fix `changes()` function when gaps between samples exceed the lookbehind window. Previously, it could yield a non-zero value even when the sample value remained unchanged. See [#10280](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10280).
* BUGFIX: [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): fix [alert restore](https://docs.victoriametrics.com/victoriametrics/vmalert/#alerts-state-on-restarts) when a group contains many rules and is slow to complete evaluation. Previously, the restore process might not retrieve the correct previous alert state. See [#10335](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10335).
* BUGFIX: [vmauth](https://docs.victoriametrics.com/victoriametrics/vmauth/): fix an issue where canceling a client request (closing a browser tab or timeout) incorrectly marked all backends as unavailable for `-failTimeout` duration (3s by default), even though backends were healthy. See [#10318](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10318).
* BUGFIX: [vmauth](https://docs.victoriametrics.com/victoriametrics/vmauth/): stop backend health checks for URL prefixes defined in `url_map` during configuration reloads. Previously, stale backends kept being health-checked and produced repeated warning logs after reloads. See [#10334](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10334).
* BUGFIX: `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): correctly return tenants results for `/admin/tenants` when `start` or `end` are specified. See [#10312](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10312)
* BUGFIX: [vmui](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui): fix "Percentage from total" calculation on the Cardinality Explorer page when multiple metrics match the filter. See [#10323](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10323). Thanks to @PleasingFungus for the contribution.
## [v1.110.28](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.110.28)
Released at 2026-01-16

View File

@@ -78,6 +78,8 @@ Read [VictoriaLogs#869](https://github.com/VictoriaMetrics/VictoriaLogs/issues/8
**Known issue: `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): may shutdown ungracefully (data, indexes, and caches are not flushed to disk) in Kubernetes deployments if the number of `vminserts` is > 1 and `terminationGracePeriodSeconds` < 60s (30s by default). The issue was introduced in [9487](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9487) and [10136](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10136) and was fixed in [10224](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10224). The fix will be included into `v1.133.0`. Even with this fix, ungraceful shutdowns are possible if flushing in-memory contents takes more than 5s. In this case decrease `-storage.vminsertConnsShutdownDuration` and/or increase [terminationGracePeriodSeconds](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination-flow).**
**Known issue:** [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): Certain data and query patterns may cause high CPU utilization due to [10154](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10154). See [10297](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10297).
* SECURITY: upgrade Go builder from Go1.25.4 to Go1.25.5. See [the list of issues addressed in Go1.25.5](https://github.com/golang/go/issues?q=milestone%3AGo1.25.5%20label%3ACherryPickApproved).
* FEATURE: [dashboards/operator](https://grafana.com/grafana/dashboards/17869-victoriametrics-operator/): add panels for flags and configuration parameters values. See [operator#1341](https://github.com/VictoriaMetrics/operator/issues/1341).
@@ -2257,4 +2259,4 @@ See changes [here](https://docs.victoriametrics.com/victoriametrics/changelog/ch
## Previous releases
See changes for older releases [here](https://docs.victoriametrics.com/victoriametrics/changelog/changelog_2024/).
See changes for older releases [here](https://docs.victoriametrics.com/victoriametrics/changelog/changelog_2024/).

View File

@@ -117,7 +117,7 @@ It is allowed to run VictoriaMetrics and VictoriaLogs Enterprise components in [
Binary releases of Enterprise components are available at [the releases page for VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest)
and [the releases page for VictoriaLogs](https://github.com/VictoriaMetrics/VictoriaLogs/releases/latest).
Enterprise binaries and packages have `enterprise` suffix in their names. For example, `victoria-metrics-linux-amd64-v1.134.0-enterprise.tar.gz`.
Enterprise binaries and packages have `enterprise` suffix in their names. For example, `victoria-metrics-linux-amd64-v1.135.0-enterprise.tar.gz`.
In order to run binary release of Enterprise component, please download the `*-enterprise.tar.gz` archive for your OS and architecture
from the corresponding releases page and unpack it. Then run the unpacked binary.
@@ -135,8 +135,8 @@ For example, the following command runs VictoriaMetrics Enterprise binary with t
obtained at [this page](https://victoriametrics.com/products/enterprise/trial/):
```sh
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.134.0/victoria-metrics-linux-amd64-v1.134.0-enterprise.tar.gz
tar -xzf victoria-metrics-linux-amd64-v1.134.0-enterprise.tar.gz
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.135.0/victoria-metrics-linux-amd64-v1.135.0-enterprise.tar.gz
tar -xzf victoria-metrics-linux-amd64-v1.135.0-enterprise.tar.gz
./victoria-metrics-prod -license=BASE64_ENCODED_LICENSE_KEY
```
@@ -151,7 +151,7 @@ Alternatively, VictoriaMetrics Enterprise license can be stored in the file and
It is allowed to run VictoriaMetrics and VictoriaLogs Enterprise components in [cases listed here](#valid-cases-for-victoriametrics-enterprise).
Docker images for Enterprise components are available at [VictoriaMetrics Docker Hub](https://hub.docker.com/u/victoriametrics) and [VictoriaMetrics Quay](https://quay.io/organization/victoriametrics).
Enterprise docker images have `enterprise` suffix in their names. For example, `victoriametrics/victoria-metrics:v1.134.0-enterprise`.
Enterprise docker images have `enterprise` suffix in their names. For example, `victoriametrics/victoria-metrics:v1.135.0-enterprise`.
In order to run Docker image of VictoriaMetrics Enterprise component, it is required to provide the license key via the command-line
flag as described in the [binary-releases](#binary-releases) section.
@@ -161,13 +161,13 @@ Enterprise license key can be obtained at [this page](https://victoriametrics.co
For example, the following command runs VictoriaMetrics Enterprise Docker image with the specified license key:
```sh
docker run --name=victoria-metrics victoriametrics/victoria-metrics:v1.134.0-enterprise -license=BASE64_ENCODED_LICENSE_KEY
docker run --name=victoria-metrics victoriametrics/victoria-metrics:v1.135.0-enterprise -license=BASE64_ENCODED_LICENSE_KEY
```
Alternatively, the license code can be stored in the file and then referred via `-licenseFile` command-line flag:
```sh
docker run --name=victoria-metrics -v /vm-license:/vm-license victoriametrics/victoria-metrics:v1.134.0-enterprise -licenseFile=/path/to/vm-license
docker run --name=victoria-metrics -v /vm-license:/vm-license victoriametrics/victoria-metrics:v1.135.0-enterprise -licenseFile=/path/to/vm-license
```
Example docker-compose configuration:
@@ -177,7 +177,7 @@ version: "3.5"
services:
victoriametrics:
container_name: victoriametrics
image: victoriametrics/victoria-metrics:v1.134.0
image: victoriametrics/victoria-metrics:v1.135.0
ports:
- 8428:8428
volumes:
@@ -209,7 +209,7 @@ is used to provide the license key in plain-text:
```yaml
server:
image:
tag: v1.134.0-enterprise
tag: v1.135.0-enterprise
license:
key: {BASE64_ENCODED_LICENSE_KEY}
@@ -220,7 +220,7 @@ In order to provide the license key via existing secret, the following values fi
```yaml
server:
image:
tag: v1.134.0-enterprise
tag: v1.135.0-enterprise
license:
secret:
@@ -270,7 +270,7 @@ spec:
license:
key: {BASE64_ENCODED_LICENSE_KEY}
image:
tag: v1.134.0-enterprise
tag: v1.135.0-enterprise
```
In order to provide the license key via an existing secret, the following custom resource is used:
@@ -287,7 +287,7 @@ spec:
name: vm-license
key: license
image:
tag: v1.134.0-enterprise
tag: v1.135.0-enterprise
```
Example secret with license key:
@@ -338,7 +338,7 @@ Builds are available for amd64 and arm64 architectures.
Example archive:
`victoria-metrics-linux-amd64-v1.134.0-enterprise.tar.gz`
`victoria-metrics-linux-amd64-v1.135.0-enterprise.tar.gz`
Includes:
@@ -347,7 +347,7 @@ Includes:
Example Docker image:
`victoriametrics/victoria-metrics:v1.134.0-enterprise-fips` uses the FIPS-compatible binary and based on `scratch` image.
`victoriametrics/victoria-metrics:v1.135.0-enterprise-fips` uses the FIPS-compatible binary and based on `scratch` image.
## Monitoring license expiration

View File

@@ -5,6 +5,8 @@ menu:
docs:
parent: "integrations-vm"
weight: 10
aliases:
- /victoriametrics/integrations/zabbix/
---
VictoriaMetrics components like **vmagent**, **vminsert** or **single-node** can receive data from
@@ -61,4 +63,4 @@ curl http://localhost:8428/api/v1/export -d 'match={host="Zabbix server"}'
{"metric":{"__name__":"item_1","host":"ZabbixServer","hostname":"ZabbixServer","group_servers":"exists","tag_foo":"exists"},"values":[0],"timestamps":[1673454303800]}
{"metric":{"__name__":"item_2","host":"ZabbixServer","hostname":"ZabbixServer","group_servers":"exists","tag_foo":"test,exists"},"values":[1],"timestamps":[1673454303832]}
{"metric":{"__name__":"item_3","host":"ZabbixServer","hostname":"ZabbixServer","group_servers":"exists","tag_bar":"test"},"values":[123],"timestamps":[1673454303867]}
```
```

View File

@@ -35,8 +35,8 @@ scrape_configs:
After you created the `scrape.yaml` file, download and unpack [single-node VictoriaMetrics](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) to the same directory:
```sh
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.134.0/victoria-metrics-linux-amd64-v1.134.0.tar.gz
tar xzf victoria-metrics-linux-amd64-v1.134.0.tar.gz
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.135.0/victoria-metrics-linux-amd64-v1.135.0.tar.gz
tar xzf victoria-metrics-linux-amd64-v1.135.0.tar.gz
```
Then start VictoriaMetrics and instruct it to scrape targets defined in `scrape.yaml` and save scraped metrics
@@ -150,8 +150,8 @@ Then start [single-node VictoriaMetrics](https://docs.victoriametrics.com/victor
```yaml
# Download and unpack single-node VictoriaMetrics
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.134.0/victoria-metrics-linux-amd64-v1.134.0.tar.gz
tar xzf victoria-metrics-linux-amd64-v1.134.0.tar.gz
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.135.0/victoria-metrics-linux-amd64-v1.135.0.tar.gz
tar xzf victoria-metrics-linux-amd64-v1.135.0.tar.gz
# Run single-node VictoriaMetrics with the given scrape.yaml
./victoria-metrics-prod -promscrape.config=scrape.yaml
@@ -322,16 +322,11 @@ scrape_configs:
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
metrics_path: /metrics/cadvisor
relabel_configs:
# Cadvisor metrics are better to scrape from Kubernetes API server proxy.
# There is no need to add container, pod and node labels to the scraped metrics,
# since cadvisor adds these labels on itself.
#
- source_labels: [__meta_kubernetes_node_name]
target_label: __address__
regex: '(.+)'
replacement: https://kubernetes.default.svc/api/v1/nodes/$1/proxy/metrics/cadvisor
- source_labels: [__meta_kubernetes_node_name]
target_label: instance
```

View File

@@ -320,7 +320,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/
The maximum size of http response headers from Prometheus scrape targets
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 4096)
-promscrape.maxScrapeSize size
The maximum size of scrape response in bytes to process from Prometheus targets. Bigger responses are rejected. See also max_scrape_size option at https://docs.victoriametrics.com/victoriametrics/sd_configs/#scrape_configs
The maximum size of uncompressed scrape response in bytes to process from Prometheus targets. Bigger uncompressed responses are rejected. See also max_scrape_size option at https://docs.victoriametrics.com/victoriametrics/sd_configs/#scrape_configs
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 16777216)
-promscrape.minResponseSizeForStreamParse size
The minimum target response size for automatic switching to stream parsing mode, which can reduce memory usage. See https://docs.victoriametrics.com/victoriametrics/vmagent/#stream-parsing-mode
@@ -486,6 +486,9 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/
Query stats for /api/v1/status/top_queries is tracked on this number of last queries. Zero value disables query stats tracking (default 20000)
-search.queryStats.minQueryDuration duration
The minimum duration for queries to track in query stats at /api/v1/status/top_queries. Queries with lower duration are ignored in query stats (default 1ms)
-search.queryStats.minQueryMemoryUsage size
The minimum memory bytes consumption for queries to track in query stats at /api/v1/status/top_queries. Queries with lower memory bytes consumption are ignored in query stats
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 1024)
-search.resetCacheAuthKey value
Optional authKey for resetting rollup cache via /internal/resetRollupResultCache call. It could be passed via authKey query arg. It overrides -httpAuth.*
Flag value can be read from the given file when using -search.resetCacheAuthKey=file:///abs/path/to/file or -search.resetCacheAuthKey=file://./relative/path/to/file.

View File

@@ -114,6 +114,8 @@ The maximum on-disk size for the buffered metrics can be limited with `-remoteWr
`vmagent` can save on network bandwidth usage costs by using [VictoriaMetrics remote write protocol](#victoriametrics-remote-write-protocol).
See [how to optimize index size at VictoriaMetrics for IoT and industrial monitoring](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#index-tuning-for-low-churn-rate).
### Drop-in replacement for Prometheus
If you use Prometheus only for scraping metrics from various targets and forwarding these metrics to remote storage

View File

@@ -286,7 +286,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/vmagent/ .
The maximum size of http response headers from Prometheus scrape targets
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 4096)
-promscrape.maxScrapeSize size
The maximum size of scrape response in bytes to process from Prometheus targets. Bigger responses are rejected. See also max_scrape_size option at https://docs.victoriametrics.com/victoriametrics/sd_configs/#scrape_configs
The maximum size of uncompressed scrape response in bytes to process from Prometheus targets. Bigger uncompressed responses are rejected. See also max_scrape_size option at https://docs.victoriametrics.com/victoriametrics/sd_configs/#scrape_configs
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 16777216)
-promscrape.minResponseSizeForStreamParse size
The minimum target response size for automatic switching to stream parsing mode, which can reduce memory usage. See https://docs.victoriametrics.com/victoriametrics/vmagent/#stream-parsing-mode
@@ -459,8 +459,10 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/vmagent/ .
Optional proxy URL for writing data to the corresponding -remoteWrite.url. Supported proxies: http, https, socks5. Example: -remoteWrite.proxyURL=socks5://proxy:1234
Supports an array of values separated by comma or specified via multiple flags.
Each array item can contain comma inside single-quoted or double-quoted string, {}, [] and () braces.
-remoteWrite.queues int
-remoteWrite.queues array
The number of concurrent queues to each -remoteWrite.url. Set more queues if default number of queues isn't enough for sending high volume of collected data to remote storage. Default value depends on the number of available CPU cores. It should work fine in most cases since it minimizes resource usage (default 2*cgroup.AvailableCPUs())
Supports array of values separated by comma or specified via multiple flags.
Empty values are set to default value.
-remoteWrite.rateLimit array
Optional rate limit in bytes per second for data sent to the corresponding -remoteWrite.url. By default, the rate limit is disabled. It can be useful for limiting load on remote storage when big amounts of buffered data is sent after temporary unavailability of the remote storage. See also -maxIngestionRate (default 0)
Supports array of values separated by comma or specified via multiple flags.

View File

@@ -738,7 +738,7 @@ See also [request body buffering](https://docs.victoriametrics.com/victoriametri
## Request body buffering
`vmauth` can buffer request bodies {{% available_from "#" %}} before proxying the requests to backends. This prevent slow-writing clients from occupying connections to backends.
`vmauth` can buffer request bodies {{% available_from "v1.135.0" %}} before proxying the requests to backends. This prevent slow-writing clients from occupying connections to backends.
This is especially important when clients send requests over unreliable or low-bandwidth networks (for example, [IoT](https://en.wikipedia.org/wiki/Internet_of_things) devices over EDGE networks),
where slow uploads can exhaust concurrency limits, increase latency, reduce ingestion rate, and trigger `429 Too Many Requests` responses even when backend resources are not saturated.

View File

@@ -43,6 +43,8 @@ Backup schedule is controlled by the following flags:
* `-disableMonthly` - disable monthly run. Default false
By default, all flags are turned on and Backup Manager backups data every hour for every interval (hourly, daily, weekly and monthly).
Note that if all intervals are disabled, the `vmbackupmanager` will create latest backups every 24 hours.
Using `-disableScheduledBackups`{{% available_from "v1.135.0" %}} allows completely disabling scheduled backups. In this mode API can be used to trigger backups and restores on demand.
The backup manager creates the following directory hierarchy at `-dst`:
@@ -537,6 +539,8 @@ command-line flags:
Disable hourly run. Default false
-disableMonthly
Disable monthly run. Default false
-disableScheduledBackups
Disable all scheduled backups. This is useful in order to run vmbackupmanager in API only mode to allow requesting restore and performing manual backups. Default false
-disableWeekly
Disable weekly run. Default false
-dst string

View File

@@ -34,9 +34,9 @@ vmctl command-line tool is available as:
Download and unpack vmctl:
```sh
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.134.0/vmutils-darwin-arm64-v1.134.0.tar.gz
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.135.0/vmutils-darwin-arm64-v1.135.0.tar.gz
tar xzf vmutils-darwin-arm64-v1.134.0.tar.gz
tar xzf vmutils-darwin-arm64-v1.135.0.tar.gz
```
Once binary is unpacked, see the full list of supported modes by running the following command:
@@ -169,6 +169,71 @@ see `--vm-concurrency` flag.
Please note, you can also use [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/)
as a proxy between `vmctl` and destination with `-remoteWrite.rateLimit` flag enabled.
### Monitoring the migration process
`vmctl` can push internal metrics {{% available_from "v1.135.0" %}} to a remote storage for monitoring migration progress and performance.
This is especially useful for long-running migrations where you want to track progress, detect issues,
or build dashboards to visualize the migration status.
Example usage with VictoriaMetrics as the metrics destination:
```sh
./vmctl influx \
--influx-addr=http://localhost:8086 \
--influx-database=mydb \
--vm-addr=http://localhost:8428 \
--pushmetrics.url=http://localhost:8428/api/v1/import/prometheus \
--pushmetrics.extraLabel='job="vmctl"' \
--pushmetrics.extraLabel='instance="migration-1"'
```
#### Available metrics
The following metrics are exposed by `vmctl`:
General metrics (available for all migration modes):
| Metric | Description |
|--------|-------------|
| `vmctl_backoff_retries_total` | Total number of retry attempts across all operations |
| `vmctl_limiter_bytes_processed_total` | Total bytes processed through rate limiter (when `--vm-rate-limit` is set) |
| `vmctl_limiter_throttle_events_total` | Number of times rate limiting caused a pause |
Mode-specific metrics:
Each migration mode exposes its own set of metrics with the mode name embedded in the metric name:
| Mode | Metrics |
|------|---------|
| `influx` | `vmctl_influx_migration_series_total`, `vmctl_influx_migration_series_processed`, `vmctl_influx_migration_errors_total` |
| `prometheus` | `vmctl_prometheus_migration_blocks_total`, `vmctl_prometheus_migration_blocks_processed`, `vmctl_prometheus_migration_errors_total` |
| `opentsdb` | `vmctl_opentsdb_migration_series_total`, `vmctl_opentsdb_migration_series_processed`, `vmctl_opentsdb_migration_errors_total` |
| `remote-read` | `vmctl_remote_read_migration_ranges_total`, `vmctl_remote_read_migration_ranges_processed`, `vmctl_remote_read_migration_errors_total` |
| `vm-native` | `vmctl_vm_native_migration_metrics_total`, `vmctl_vm_native_migration_metrics_processed`, `vmctl_vm_native_migration_requests_planned`, `vmctl_vm_native_migration_requests_completed`, `vmctl_vm_native_migration_tenants_total`, `vmctl_vm_native_migration_tenants_processed`, `vmctl_vm_native_migration_bytes_transferred_total`, `vmctl_vm_native_migration_errors_total` |
#### Example PromQL queries
Monitor migration progress:
```promql
# Migration completion percentage for influx mode
vmctl_influx_migration_series_processed / vmctl_influx_migration_series_total * 100
# Migration completion percentage for vm-native mode
vmctl_vm_native_migration_metrics_processed / vmctl_vm_native_migration_metrics_total * 100
# Retry rate
rate(vmctl_backoff_retries_total[5m])
# Rate limiter throttling events per second
rate(vmctl_limiter_throttle_events_total[5m])
# Data transfer speed in bytes per second (when rate limiting is enabled)
rate(vmctl_limiter_bytes_processed_total[5m])
# Data transfer speed in MB per second for vm-native mode
rate(vmctl_vm_native_migration_bytes_transferred_total[5m]) / 1Mb
```
## Verifying exported blocks from VictoriaMetrics
In this mode, `vmctl` allows verifying correctness and integrity of data exported via

View File

@@ -15,17 +15,22 @@ USAGE:
vmctl influx [command options]
OPTIONS:
-s Whether to run in silent mode. If set to true no confirmation prompts will appear. (default: false)
--verbose Whether to enable verbosity in logs output. (default: false)
--disable-progress-bar Whether to disable progress bar during the import. (default: false)
--influx-addr value InfluxDB server addr (default: "http://localhost:8086")
--influx-user value InfluxDB user [$INFLUX_USERNAME]
--influx-password value InfluxDB user password [$INFLUX_PASSWORD]
--influx-database value InfluxDB database
--influx-retention-policy value InfluxDB retention policy (default: "autogen")
--influx-chunk-size value The chunkSize defines max amount of series to be returned in one chunk (default: 10000)
--influx-concurrency value Number of concurrently running fetch queries to InfluxDB (default: 1)
--influx-filter-series value InfluxDB filter expression to select series. E.g. "from cpu where arch='x86' AND hostname='host_2753'".
-s Whether to run in silent mode. If set to true no confirmation prompts will appear. (default: false)
--verbose Whether to enable verbosity in logs output. (default: false)
--disable-progress-bar Whether to disable progress bar during the import. (default: false)
--pushmetrics.url value [ --pushmetrics.url value ] Optional URL to push metrics. See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#push-metrics
--pushmetrics.interval value Interval for pushing metrics to every -pushmetrics.url (default: 10s)
--pushmetrics.extraLabel value [ --pushmetrics.extraLabel value ] Extra labels to add to pushed metrics. In case of collision, label value defined by flag will have priority. Flag can be set multiple times, to add few additional labels. For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
--pushmetrics.header value [ --pushmetrics.header value ] Optional HTTP headers to add to pushed metrics. Flag can be set multiple times, to add few additional headers.
--pushmetrics.disableCompression Whether to disable compression when pushing metrics. (default: false)
--influx-addr value InfluxDB server addr (default: "http://localhost:8086")
--influx-user value InfluxDB user [$INFLUX_USERNAME]
--influx-password value InfluxDB user password [$INFLUX_PASSWORD]
--influx-database value InfluxDB database
--influx-retention-policy value InfluxDB retention policy (default: "autogen")
--influx-chunk-size value The chunkSize defines max amount of series to be returned in one chunk (default: 10000)
--influx-concurrency value Number of concurrently running fetch queries to InfluxDB (default: 1)
--influx-filter-series value InfluxDB filter expression to select series. E.g. "from cpu where arch='x86' AND hostname='host_2753'".
See for details https://docs.influxdata.com/influxdb/v1.7/query_language/schema_exploration#show-series
--influx-filter-time-start value The time filter to select timeseries with timestamp equal or higher than provided value. E.g. '2020-01-01T20:07:00Z'
--influx-filter-time-end value The time filter to select timeseries with timestamp equal or lower than provided value. E.g. '2020-01-01T20:07:00Z'
@@ -52,14 +57,14 @@ OPTIONS:
--vm-batch-size value How many samples importer collects before sending the import request to VM (default: 200000)
--vm-significant-figures value The number of significant figures to leave in metric values before importing. See https://en.wikipedia.org/wiki/Significant_figures. Zero value saves all the significant figures. This option may be used for increasing on-disk compression level for the stored metrics. See also --vm-round-digits option (default: 0)
--vm-round-digits value Round metric values to the given number of decimal digits after the point. This option may be used for increasing on-disk compression level for the stored metrics (default: 100)
--vm-extra-label value [ --vm-extra-label value ] Extra labels, that will be added to imported timeseries. In case of collision, label value defined by flagwill have priority. Flag can be set multiple times, to add few additional labels.
--vm-extra-label value [ --vm-extra-label value ] Extra labels, that will be added to imported timeseries. In case of collision, label value defined by flag will have priority. Flag can be set multiple times, to add few additional labels.
--vm-rate-limit value Optional data transfer rate limit in bytes per second.
By default, the rate limit is disabled. It can be useful for limiting load on configured via '--vmAddr' destination. (default: 0)
--vm-cert-file value Optional path to client-side TLS certificate file to use when connecting to '--vmAddr'
--vm-key-file value Optional path to client-side TLS key to use when connecting to '--vmAddr'
--vm-CA-file value Optional path to TLS CA file to use for verifying connections to '--vmAddr'. By default, system CA is used
--vm-server-name value Optional TLS server name to use for connections to '--vmAddr'. By default, the server name from '--vmAddr' is used
--vm-insecure-skip-verify Whether to skip tls verification when connecting to '--vmAddr' (default: false)
By default, the rate limit is disabled. It can be useful for limiting load on configured via '--vm-addr' destination. (default: 0)
--vm-cert-file value Optional path to client-side TLS certificate file to use when connecting to '--vm-addr'
--vm-key-file value Optional path to client-side TLS key to use when connecting to '--vm-addr'
--vm-CA-file value Optional path to TLS CA file to use for verifying connections to '--vm-addr'. By default, system CA is used
--vm-server-name value Optional TLS server name to use for connections to '--vm-addr'. By default, the server name from '--vm-addr' is used
--vm-insecure-skip-verify Whether to skip tls verification when connecting to '--vm-addr' (default: false)
--vm-backoff-retries value How many import retries to perform before giving up. (default: 10)
--vm-backoff-factor value Factor to multiply the base duration after each failed import retry. Must be greater than 1.0 (default: 1.8)
--vm-backoff-min-duration value Minimum duration to wait before the first import retry. Each subsequent import retry will be multiplied by the '--vm-backoff-factor'. (default: 2s)

View File

@@ -15,24 +15,29 @@ USAGE:
vmctl opentsdb [command options]
OPTIONS:
-s Whether to run in silent mode. If set to true no confirmation prompts will appear. (default: false)
--verbose Whether to enable verbosity in logs output. (default: false)
--disable-progress-bar Whether to disable progress bar during the import. (default: false)
--otsdb-addr value OpenTSDB server addr (default: "http://localhost:4242")
--otsdb-concurrency value Number of concurrently running fetch queries to OpenTSDB per metric (default: 1)
--otsdb-retentions value [ --otsdb-retentions value ] Retentions patterns to collect on. Each pattern should describe the aggregation performed for the query, the row size (in HBase) that will define how long each individual query is, and the time range to query for. e.g. sum-1m-avg:1h:3d. The first time range defined should be a multiple of the row size in HBase. e.g. if the row size is 2 hours, 4h is good, 5h less so. We want each query to land on unique rows.
--otsdb-filters value [ --otsdb-filters value ] Filters to process for discovering metrics in OpenTSDB (default: "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z")
--otsdb-offset-days value Days to offset our 'starting' point for collecting data from OpenTSDB (default: 0)
--otsdb-hard-ts-start value A specific timestamp to start from, will override using an offset (default: 0)
--otsdb-query-limit value Result limit on meta queries to OpenTSDB (affects both metric name and tag value queries, recommended to use a value exceeding your largest series) (default: 100000000)
--otsdb-msecstime Whether OpenTSDB is writing values in milliseconds or seconds (default: false)
--otsdb-normalize Whether to normalize all data received to lower case before forwarding to VictoriaMetrics (default: false)
--otsdb-cert-file value Optional path to client-side TLS certificate file to use when connecting to -otsdb-addr
--otsdb-key-file value Optional path to client-side TLS key to use when connecting to -otsdb-addr
--otsdb-CA-file value Optional path to TLS CA file to use for verifying connections to -otsdb-addr. By default, system CA is used
--otsdb-server-name value Optional TLS server name to use for connections to -otsdb-addr. By default, the server name from -otsdb-addr is used
--otsdb-insecure-skip-verify Whether to skip tls verification when connecting to -otsdb-addr (default: false)
--vm-addr value VictoriaMetrics address to perform import requests.
-s Whether to run in silent mode. If set to true no confirmation prompts will appear. (default: false)
--verbose Whether to enable verbosity in logs output. (default: false)
--disable-progress-bar Whether to disable progress bar during the import. (default: false)
--pushmetrics.url value [ --pushmetrics.url value ] Optional URL to push metrics. See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#push-metrics
--pushmetrics.interval value Interval for pushing metrics to every -pushmetrics.url (default: 10s)
--pushmetrics.extraLabel value [ --pushmetrics.extraLabel value ] Extra labels to add to pushed metrics. In case of collision, label value defined by flag will have priority. Flag can be set multiple times, to add few additional labels. For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
--pushmetrics.header value [ --pushmetrics.header value ] Optional HTTP headers to add to pushed metrics. Flag can be set multiple times, to add few additional headers.
--pushmetrics.disableCompression Whether to disable compression when pushing metrics. (default: false)
--otsdb-addr value OpenTSDB server addr (default: "http://localhost:4242")
--otsdb-concurrency value Number of concurrently running fetch queries to OpenTSDB per metric (default: 1)
--otsdb-retentions value [ --otsdb-retentions value ] Retentions patterns to collect on. Each pattern should describe the aggregation performed for the query, the row size (in HBase) that will define how long each individual query is, and the time range to query for. e.g. sum-1m-avg:1h:3d. The first time range defined should be a multiple of the row size in HBase. e.g. if the row size is 2 hours, 4h is good, 5h less so. We want each query to land on unique rows.
--otsdb-filters value [ --otsdb-filters value ] Filters to process for discovering metrics in OpenTSDB (default: "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z")
--otsdb-offset-days value Days to offset our 'starting' point for collecting data from OpenTSDB (default: 0)
--otsdb-hard-ts-start value A specific timestamp to start from, will override using an offset (default: 0)
--otsdb-query-limit value Result limit on meta queries to OpenTSDB (affects both metric name and tag value queries, recommended to use a value exceeding your largest series) (default: 100000000)
--otsdb-msecstime Whether OpenTSDB is writing values in milliseconds or seconds (default: false)
--otsdb-normalize Whether to normalize all data received to lower case before forwarding to VictoriaMetrics (default: false)
--otsdb-cert-file value Optional path to client-side TLS certificate file to use when connecting to -otsdb-addr
--otsdb-key-file value Optional path to client-side TLS key to use when connecting to -otsdb-addr
--otsdb-CA-file value Optional path to TLS CA file to use for verifying connections to -otsdb-addr. By default, system CA is used
--otsdb-server-name value Optional TLS server name to use for connections to -otsdb-addr. By default, the server name from -otsdb-addr is used
--otsdb-insecure-skip-verify Whether to skip tls verification when connecting to -otsdb-addr (default: false)
--vm-addr value VictoriaMetrics address to perform import requests.
Should be the same as --httpListenAddr value for single-node version or vminsert component.
When importing into the clustered version do not forget to set additionally --vm-account-id flag.
Please note, that vmctl performs initial readiness check for the given address by checking /health endpoint. (default: "http://localhost:8428")
@@ -47,14 +52,14 @@ OPTIONS:
--vm-batch-size value How many samples importer collects before sending the import request to VM (default: 200000)
--vm-significant-figures value The number of significant figures to leave in metric values before importing. See https://en.wikipedia.org/wiki/Significant_figures. Zero value saves all the significant figures. This option may be used for increasing on-disk compression level for the stored metrics. See also --vm-round-digits option (default: 0)
--vm-round-digits value Round metric values to the given number of decimal digits after the point. This option may be used for increasing on-disk compression level for the stored metrics (default: 100)
--vm-extra-label value [ --vm-extra-label value ] Extra labels, that will be added to imported timeseries. In case of collision, label value defined by flagwill have priority. Flag can be set multiple times, to add few additional labels.
--vm-extra-label value [ --vm-extra-label value ] Extra labels, that will be added to imported timeseries. In case of collision, label value defined by flag will have priority. Flag can be set multiple times, to add few additional labels.
--vm-rate-limit value Optional data transfer rate limit in bytes per second.
By default, the rate limit is disabled. It can be useful for limiting load on configured via '--vmAddr' destination. (default: 0)
--vm-cert-file value Optional path to client-side TLS certificate file to use when connecting to '--vmAddr'
--vm-key-file value Optional path to client-side TLS key to use when connecting to '--vmAddr'
--vm-CA-file value Optional path to TLS CA file to use for verifying connections to '--vmAddr'. By default, system CA is used
--vm-server-name value Optional TLS server name to use for connections to '--vmAddr'. By default, the server name from '--vmAddr' is used
--vm-insecure-skip-verify Whether to skip tls verification when connecting to '--vmAddr' (default: false)
By default, the rate limit is disabled. It can be useful for limiting load on configured via '--vm-addr' destination. (default: 0)
--vm-cert-file value Optional path to client-side TLS certificate file to use when connecting to '--vm-addr'
--vm-key-file value Optional path to client-side TLS key to use when connecting to '--vm-addr'
--vm-CA-file value Optional path to TLS CA file to use for verifying connections to '--vm-addr'. By default, system CA is used
--vm-server-name value Optional TLS server name to use for connections to '--vm-addr'. By default, the server name from '--vm-addr' is used
--vm-insecure-skip-verify Whether to skip tls verification when connecting to '--vm-addr' (default: false)
--vm-backoff-retries value How many import retries to perform before giving up. (default: 10)
--vm-backoff-factor value Factor to multiply the base duration after each failed import retry. Must be greater than 1.0 (default: 1.8)
--vm-backoff-min-duration value Minimum duration to wait before the first import retry. Each subsequent import retry will be multiplied by the '--vm-backoff-factor'. (default: 2s)

View File

@@ -15,17 +15,22 @@ USAGE:
vmctl prometheus [command options]
OPTIONS:
-s Whether to run in silent mode. If set to true no confirmation prompts will appear. (default: false)
--verbose Whether to enable verbosity in logs output. (default: false)
--disable-progress-bar Whether to disable progress bar during the import. (default: false)
--prom-snapshot value Path to Prometheus snapshot. Pls see for details https://www.robustperception.io/taking-snapshots-of-prometheus-data
--prom-concurrency value Number of concurrently running snapshot readers (default: 1)
--prom-filter-time-start value The time filter in RFC3339 format to select timeseries with timestamp equal or higher than provided value. E.g. '2020-01-01T20:07:00Z'
--prom-filter-time-end value The time filter in RFC3339 format to select timeseries with timestamp equal or lower than provided value. E.g. '2020-01-01T20:07:00Z'
--prom-filter-label value Prometheus label name to filter timeseries by. E.g. '__name__' will filter timeseries by name.
--prom-filter-label-value value Prometheus regular expression to filter label from "prom-filter-label" flag. (default: ".*")
--prom-tmp-dir-path value Path to directory to be used for temporary files. (default: "/var/folders/ds/3kj5p3v17ll0hsyvq380ryvm0000gn/T/")
--vm-addr value VictoriaMetrics address to perform import requests.
-s Whether to run in silent mode. If set to true no confirmation prompts will appear. (default: false)
--verbose Whether to enable verbosity in logs output. (default: false)
--disable-progress-bar Whether to disable progress bar during the import. (default: false)
--pushmetrics.url value [ --pushmetrics.url value ] Optional URL to push metrics. See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#push-metrics
--pushmetrics.interval value Interval for pushing metrics to every -pushmetrics.url (default: 10s)
--pushmetrics.extraLabel value [ --pushmetrics.extraLabel value ] Extra labels to add to pushed metrics. In case of collision, label value defined by flag will have priority. Flag can be set multiple times, to add few additional labels. For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
--pushmetrics.header value [ --pushmetrics.header value ] Optional HTTP headers to add to pushed metrics. Flag can be set multiple times, to add few additional headers.
--pushmetrics.disableCompression Whether to disable compression when pushing metrics. (default: false)
--prom-snapshot value Path to Prometheus snapshot. Pls see for details https://www.robustperception.io/taking-snapshots-of-prometheus-data
--prom-concurrency value Number of concurrently running snapshot readers (default: 1)
--prom-filter-time-start value The time filter in RFC3339 format to select timeseries with timestamp equal or higher than provided value. E.g. '2020-01-01T20:07:00Z'
--prom-filter-time-end value The time filter in RFC3339 format to select timeseries with timestamp equal or lower than provided value. E.g. '2020-01-01T20:07:00Z'
--prom-filter-label value Prometheus label name to filter timeseries by. E.g. '__name__' will filter timeseries by name.
--prom-filter-label-value value Prometheus regular expression to filter label from "prom-filter-label" flag. (default: ".*")
--prom-tmp-dir-path value Path to directory to be used for temporary files. (default: "/var/folders/ds/3kj5p3v17ll0hsyvq380ryvm0000gn/T/")
--vm-addr value VictoriaMetrics address to perform import requests.
Should be the same as --httpListenAddr value for single-node version or vminsert component.
When importing into the clustered version do not forget to set additionally --vm-account-id flag.
Please note, that vmctl performs initial readiness check for the given address by checking /health endpoint. (default: "http://localhost:8428")
@@ -40,14 +45,14 @@ OPTIONS:
--vm-batch-size value How many samples importer collects before sending the import request to VM (default: 200000)
--vm-significant-figures value The number of significant figures to leave in metric values before importing. See https://en.wikipedia.org/wiki/Significant_figures. Zero value saves all the significant figures. This option may be used for increasing on-disk compression level for the stored metrics. See also --vm-round-digits option (default: 0)
--vm-round-digits value Round metric values to the given number of decimal digits after the point. This option may be used for increasing on-disk compression level for the stored metrics (default: 100)
--vm-extra-label value [ --vm-extra-label value ] Extra labels, that will be added to imported timeseries. In case of collision, label value defined by flagwill have priority. Flag can be set multiple times, to add few additional labels.
--vm-extra-label value [ --vm-extra-label value ] Extra labels, that will be added to imported timeseries. In case of collision, label value defined by flag will have priority. Flag can be set multiple times, to add few additional labels.
--vm-rate-limit value Optional data transfer rate limit in bytes per second.
By default, the rate limit is disabled. It can be useful for limiting load on configured via '--vmAddr' destination. (default: 0)
--vm-cert-file value Optional path to client-side TLS certificate file to use when connecting to '--vmAddr'
--vm-key-file value Optional path to client-side TLS key to use when connecting to '--vmAddr'
--vm-CA-file value Optional path to TLS CA file to use for verifying connections to '--vmAddr'. By default, system CA is used
--vm-server-name value Optional TLS server name to use for connections to '--vmAddr'. By default, the server name from '--vmAddr' is used
--vm-insecure-skip-verify Whether to skip tls verification when connecting to '--vmAddr' (default: false)
By default, the rate limit is disabled. It can be useful for limiting load on configured via '--vm-addr' destination. (default: 0)
--vm-cert-file value Optional path to client-side TLS certificate file to use when connecting to '--vm-addr'
--vm-key-file value Optional path to client-side TLS key to use when connecting to '--vm-addr'
--vm-CA-file value Optional path to TLS CA file to use for verifying connections to '--vm-addr'. By default, system CA is used
--vm-server-name value Optional TLS server name to use for connections to '--vm-addr'. By default, the server name from '--vm-addr' is used
--vm-insecure-skip-verify Whether to skip tls verification when connecting to '--vm-addr' (default: false)
--vm-backoff-retries value How many import retries to perform before giving up. (default: 10)
--vm-backoff-factor value Factor to multiply the base duration after each failed import retry. Must be greater than 1.0 (default: 1.8)
--vm-backoff-min-duration value Minimum duration to wait before the first import retry. Each subsequent import retry will be multiplied by the '--vm-backoff-factor'. (default: 2s)

View File

@@ -18,6 +18,11 @@ OPTIONS:
-s Whether to run in silent mode. If set to true no confirmation prompts will appear. (default: false)
--verbose Whether to enable verbosity in logs output. (default: false)
--disable-progress-bar Whether to disable progress bar during the import. (default: false)
--pushmetrics.url value [ --pushmetrics.url value ] Optional URL to push metrics. See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#push-metrics
--pushmetrics.interval value Interval for pushing metrics to every -pushmetrics.url (default: 10s)
--pushmetrics.extraLabel value [ --pushmetrics.extraLabel value ] Extra labels to add to pushed metrics. In case of collision, label value defined by flag will have priority. Flag can be set multiple times, to add few additional labels. For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
--pushmetrics.header value [ --pushmetrics.header value ] Optional HTTP headers to add to pushed metrics. Flag can be set multiple times, to add few additional headers.
--pushmetrics.disableCompression Whether to disable compression when pushing metrics. (default: false)
--remote-read-concurrency value Number of concurrently running remote read readers (default: 1)
--remote-read-filter-time-start value The time filter in RFC3339 format to select timeseries with timestamp equal or higher than provided value. E.g. '2020-01-01T20:07:00Z'
--remote-read-filter-time-end value The time filter in RFC3339 format to select timeseries with timestamp equal or lower than provided value. E.g. '2020-01-01T20:07:00Z'
@@ -55,14 +60,14 @@ OPTIONS:
--vm-batch-size value How many samples importer collects before sending the import request to VM (default: 200000)
--vm-significant-figures value The number of significant figures to leave in metric values before importing. See https://en.wikipedia.org/wiki/Significant_figures. Zero value saves all the significant figures. This option may be used for increasing on-disk compression level for the stored metrics. See also --vm-round-digits option (default: 0)
--vm-round-digits value Round metric values to the given number of decimal digits after the point. This option may be used for increasing on-disk compression level for the stored metrics (default: 100)
--vm-extra-label value [ --vm-extra-label value ] Extra labels, that will be added to imported timeseries. In case of collision, label value defined by flagwill have priority. Flag can be set multiple times, to add few additional labels.
--vm-extra-label value [ --vm-extra-label value ] Extra labels, that will be added to imported timeseries. In case of collision, label value defined by flag will have priority. Flag can be set multiple times, to add few additional labels.
--vm-rate-limit value Optional data transfer rate limit in bytes per second.
By default, the rate limit is disabled. It can be useful for limiting load on configured via '--vmAddr' destination. (default: 0)
--vm-cert-file value Optional path to client-side TLS certificate file to use when connecting to '--vmAddr'
--vm-key-file value Optional path to client-side TLS key to use when connecting to '--vmAddr'
--vm-CA-file value Optional path to TLS CA file to use for verifying connections to '--vmAddr'. By default, system CA is used
--vm-server-name value Optional TLS server name to use for connections to '--vmAddr'. By default, the server name from '--vmAddr' is used
--vm-insecure-skip-verify Whether to skip tls verification when connecting to '--vmAddr' (default: false)
By default, the rate limit is disabled. It can be useful for limiting load on configured via '--vm-addr' destination. (default: 0)
--vm-cert-file value Optional path to client-side TLS certificate file to use when connecting to '--vm-addr'
--vm-key-file value Optional path to client-side TLS key to use when connecting to '--vm-addr'
--vm-CA-file value Optional path to TLS CA file to use for verifying connections to '--vm-addr'. By default, system CA is used
--vm-server-name value Optional TLS server name to use for connections to '--vm-addr'. By default, the server name from '--vm-addr' is used
--vm-insecure-skip-verify Whether to skip tls verification when connecting to '--vm-addr' (default: false)
--vm-backoff-retries value How many import retries to perform before giving up. (default: 10)
--vm-backoff-factor value Factor to multiply the base duration after each failed import retry. Must be greater than 1.0 (default: 1.8)
--vm-backoff-min-duration value Minimum duration to wait before the first import retry. Each subsequent import retry will be multiplied by the '--vm-backoff-factor'. (default: 2s)

View File

@@ -15,10 +15,15 @@ USAGE:
vmctl vm-native [command options]
OPTIONS:
-s Whether to run in silent mode. If set to true no confirmation prompts will appear. (default: false)
--verbose Whether to enable verbosity in logs output. (default: false)
--disable-progress-bar Whether to disable progress bar during the import. (default: false)
--vm-native-filter-match value Time series selector to match series for export. For example, select {instance!="localhost"} will match all series with "instance" label different to "localhost".
-s Whether to run in silent mode. If set to true no confirmation prompts will appear. (default: false)
--verbose Whether to enable verbosity in logs output. (default: false)
--disable-progress-bar Whether to disable progress bar during the import. (default: false)
--pushmetrics.url value [ --pushmetrics.url value ] Optional URL to push metrics. See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#push-metrics
--pushmetrics.interval value Interval for pushing metrics to every -pushmetrics.url (default: 10s)
--pushmetrics.extraLabel value [ --pushmetrics.extraLabel value ] Extra labels to add to pushed metrics. In case of collision, label value defined by flag will have priority. Flag can be set multiple times, to add few additional labels. For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
--pushmetrics.header value [ --pushmetrics.header value ] Optional HTTP headers to add to pushed metrics. Flag can be set multiple times, to add few additional headers.
--pushmetrics.disableCompression Whether to disable compression when pushing metrics. (default: false)
--vm-native-filter-match value Time series selector to match series for export. For example, select {instance!="localhost"} will match all series with "instance" label different to "localhost".
See more details here https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-export-data-in-native-format (default: "{__name__!=\"\"}")
--vm-native-filter-time-start value The time filter may contain different timestamp formats. See more details here https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#timestamp-formats
--vm-native-filter-time-end value The time filter may contain different timestamp formats. See more details here https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#timestamp-formats
@@ -51,7 +56,7 @@ OPTIONS:
--vm-native-dst-ca-file value Optional path to TLS CA file to use for verifying connections to --vm-native-dst-addr. By default, system CA is used
--vm-native-dst-server-name value Optional TLS server name to use for connections to --vm-native-dst-addr. By default, the server name from --vm-native-dst-addr is used
--vm-native-dst-insecure-skip-verify Whether to skip TLS certificate verification when connecting to --vm-native-dst-addr (default: false)
--vm-extra-label value [ --vm-extra-label value ] Extra labels, that will be added to imported timeseries. In case of collision, label value defined by flagwill have priority. Flag can be set multiple times, to add few additional labels.
--vm-extra-label value [ --vm-extra-label value ] Extra labels, that will be added to imported timeseries. In case of collision, label value defined by flag will have priority. Flag can be set multiple times, to add few additional labels.
--vm-rate-limit value Optional data transfer rate limit in bytes per second.
By default, the rate limit is disabled. It can be useful for limiting load on source or destination databases.
Rate limit is applied per worker, see --vm-concurrency. (default: 0)
@@ -60,7 +65,7 @@ OPTIONS:
TenantID will be appended automatically after discovering tenants from src. (default: false)
--vm-concurrency value Number of workers concurrently performing import requests to VM (default: 2)
--vm-native-disable-per-metric-migration Defines whether to disable per-metric migration and migrate all data via one connection. In this mode, vmctl makes less export/import requests, but can't provide a progress bar or retry failed requests. (default: false)
--vm-native-disable-binary-protocol Whether to use https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-export-data-in-json-line-format instead of https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-export-data-in-native-format API.Binary export/import API protocol implies less network and resource usage, as it transfers compressed binary data blocks.Non-binary export/import API is less efficient, but supports deduplication if it is configured on vm-native-src-addr side. (default: false)
--vm-native-disable-binary-protocol Whether to use https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-export-data-in-json-line-format instead of https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-export-data-in-native-format API. Binary export/import API protocol implies less network and resource usage, as it transfers compressed binary data blocks. Non-binary export/import API is less efficient, but supports deduplication if it is configured on vm-native-src-addr side. (default: false)
--vm-native-backoff-retries value How many export/import retries to perform before giving up. (default: 10)
--vm-native-backoff-factor value Factor to multiply the base duration after each failed export/import retry. Must be greater than 1.0 (default: 1.8)
--vm-native-backoff-min-duration value Minimum duration to wait before the first export/import retry. Each subsequent export/import retry will be multiplied by the '--vm-native-backoff-factor'. (default: 2s)

View File

@@ -50,6 +50,8 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/cluster-victori
Auth key for /flags endpoint. It must be passed via authKey query arg. It overrides -httpAuth.*
Flag value can be read from the given file when using -flagsAuthKey=file:///abs/path/to/file or -flagsAuthKey=file://./relative/path/to/file.
Flag value can be read from the given http/https url when using -flagsAuthKey=http://host/path or -flagsAuthKey=https://host/path
-fs.disableMincore
Whether to disable the mincore() syscall for checking mmap()ed files. By default, mincore() is used to detect whether mmap()ed file pages are resident in memory. Disabling mincore() may be needed on older ZFS filesystems (below 2.1.5), since it may trigger ZFS bug. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10327 for details.
-fs.disableMmap
Whether to use pread() instead of mmap() for reading data files. By default, mmap() is used for 64-bit arches and pread() is used for 32-bit arches, since they cannot read data files bigger than 2^32 bytes in memory. mmap() is usually faster for reading small data chunks than pread()
-fs.maxConcurrency int

View File

@@ -269,6 +269,9 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/cluster-victori
Query stats for /api/v1/status/top_queries is tracked on this number of last queries. Zero value disables query stats tracking (default 20000)
-search.queryStats.minQueryDuration duration
The minimum duration for queries to track in query stats at /api/v1/status/top_queries. Queries with lower duration are ignored in query stats (default 1ms)
-search.queryStats.minQueryMemoryUsage size
The minimum memory bytes consumption for queries to track in query stats at /api/v1/status/top_queries. Queries with lower memory bytes consumption are ignored in query stats
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 1024)
-search.resetCacheAuthKey value
Optional authKey for resetting rollup cache via /internal/resetRollupResultCache call. It could be passed via authKey query arg. It overrides -httpAuth.*
Flag value can be read from the given file when using -search.resetCacheAuthKey=file:///abs/path/to/file or -search.resetCacheAuthKey=file://./relative/path/to/file.

2
go.mod
View File

@@ -1,6 +1,6 @@
module github.com/VictoriaMetrics/VictoriaMetrics
go 1.25.6
go 1.25.7
require (
cloud.google.com/go/storage v1.59.1

View File

@@ -28,8 +28,12 @@ type Cache struct {
// NewCache creates new cache.
//
// Cache size in bytes is limited by the value returned by getMaxSizeBytes() callback.
//
// Entries will be removed from the cache if not accessed longer than ttl
// duration but not sooner than ~53 seconds (see cleaner() below).
//
// Call MustStop() in order to free up resources occupied by Cache.
func NewCache(getMaxSizeBytes func() uint64) *Cache {
func NewCache(getMaxSizeBytes func() uint64, ttl time.Duration) *Cache {
cpusCount := cgroup.AvailableCPUs()
shardsCount := cgroup.AvailableCPUs()
// Increase the number of shards with the increased number of available CPU cores.
@@ -45,7 +49,7 @@ func NewCache(getMaxSizeBytes func() uint64) *Cache {
return n / uint64(shardsCount)
}
for i := range shards {
shards[i] = newCache(getMaxShardBytes)
shards[i] = newCache(getMaxShardBytes, ttl)
}
c := &Cache{
shards: shards,
@@ -175,6 +179,11 @@ type cache struct {
// getMaxSizeBytes() is a callback, which returns the maximum allowed cache size in bytes.
getMaxSizeBytes func() uint64
// ttl is the duration in seconds during which an entry is allowed to stay
// in the cache without being retrieved. Once this time passes the entry
// will be removed from the cache.
ttl uint64
// mu protects all the fields below.
mu sync.Mutex
@@ -210,9 +219,10 @@ type cacheEntry struct {
e Entry
}
func newCache(getMaxSizeBytes func() uint64) *cache {
func newCache(getMaxSizeBytes func() uint64, ttl time.Duration) *cache {
var c cache
c.getMaxSizeBytes = getMaxSizeBytes
c.ttl = uint64(ttl.Seconds())
c.m = make(map[string]*cacheEntry)
return &c
}
@@ -231,9 +241,7 @@ func (c *cache) updateSizeBytes(n uint64) {
}
func (c *cache) cleanByTimeout() {
// Delete items accessed more than three minutes ago.
// This time should be enough for repeated queries.
lastAccessTime := fasttime.UnixTimestamp() - 3*60
lastAccessTime := fasttime.UnixTimestamp() - c.ttl
c.mu.Lock()
defer c.mu.Unlock()

View File

@@ -4,6 +4,7 @@ import (
"fmt"
"sync"
"testing"
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/cgroup"
)
@@ -18,7 +19,7 @@ func TestCache(t *testing.T) {
getMaxSize := func() uint64 {
return sizeMaxBytes
}
c := NewCache(getMaxSize)
c := NewCache(getMaxSize, 3*time.Minute)
defer c.MustStop()
if n := c.SizeBytes(); n != 0 {
t.Fatalf("unexpected SizeBytes(); got %d; want %d", n, 0)
@@ -123,7 +124,7 @@ func TestCacheConcurrentAccess(_ *testing.T) {
getMaxSize := func() uint64 {
return sizeMaxBytes
}
c := NewCache(getMaxSize)
c := NewCache(getMaxSize, 3*time.Minute)
defer c.MustStop()
workers := 5

View File

@@ -234,7 +234,9 @@ func urlValuesFromMap(m map[string]string) url.Values {
func (oi *oauth2ConfigInternal) initTokenSource() error {
tr := httputil.NewTransport(false, "vm_oauth_client")
tr.Proxy = oi.proxyURLFunc
if oi.proxyURLFunc != nil {
tr.Proxy = oi.proxyURLFunc
}
c := &http.Client{
Transport: oi.ac.NewRoundTripper(tr),
}

View File

@@ -81,7 +81,9 @@ func newClient(ctx context.Context, sw *ScrapeWork) (*client, error) {
}
tr := httputil.NewTransport(false, "vm_promscrape")
tr.Proxy = proxyURLFunc
if proxyURLFunc != nil {
tr.Proxy = proxyURLFunc
}
tr.TLSHandshakeTimeout = 10 * time.Second
tr.IdleConnTimeout = 2 * sw.ScrapeInterval
tr.DisableKeepAlives = *disableKeepAlive || sw.DisableKeepAlive

View File

@@ -22,7 +22,6 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/lib/cgroup"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/httputil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/netutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/promauth"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/promutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/timerpool"
@@ -269,11 +268,9 @@ func getHTTPClient(ac *promauth.Config, proxyURL *url.URL) *http.Client {
if !*useHTTP2Client {
// Proxy is not supported for http2 client.
// See https://github.com/golang/go/issues/26479
var proxy func(*http.Request) (*url.URL, error)
if proxyURL != nil {
proxy = http.ProxyURL(proxyURL)
tr.Proxy = http.ProxyURL(proxyURL)
}
tr.Proxy = proxy
}
c := &http.Client{
Transport: ac.NewRoundTripper(tr),
@@ -286,7 +283,6 @@ func getHTTPClient(ac *promauth.Config, proxyURL *url.URL) *http.Client {
func newHTTPTransport(enableHTTP2 bool) *http.Transport {
tr := httputil.NewTransport(enableHTTP2, "vm_promscrape_discovery_kubernetes")
tr.DialContext = netutil.Dialer.DialContext
tr.TLSHandshakeTimeout = 10 * time.Second
tr.IdleConnTimeout = *apiServerTimeout
tr.MaxIdleConnsPerHost = 100

View File

@@ -112,7 +112,9 @@ func NewClient(apiServer string, ac *promauth.Config, proxyURL *proxy.URL, proxy
}
tr := httputil.NewTransport(false, "vm_promscrape_discovery")
tr.Proxy = proxyURLFunc
if proxyURLFunc != nil {
tr.Proxy = proxyURLFunc
}
tr.TLSHandshakeTimeout = 10 * time.Second
tr.MaxIdleConnsPerHost = *maxConcurrency
tr.ResponseHeaderTimeout = DefaultClientReadTimeout
@@ -124,7 +126,9 @@ func NewClient(apiServer string, ac *promauth.Config, proxyURL *proxy.URL, proxy
}
trBlocking := httputil.NewTransport(false, "vm_promscrape_discovery")
trBlocking.Proxy = proxyURLFunc
if proxyURLFunc != nil {
trBlocking.Proxy = proxyURLFunc
}
trBlocking.TLSHandshakeTimeout = 10 * time.Second
trBlocking.MaxIdleConnsPerHost = 1000
trBlocking.ResponseHeaderTimeout = BlockingClientReadTimeout

View File

@@ -74,7 +74,14 @@ func ReadUncompressedData(r io.Reader, contentType string, maxDataSize *flagutil
func readUncompressedData(r io.Reader, maxDataSize *flagutil.Bytes, decompress func(dst, src []byte) ([]byte, error), callback func(data []byte) error) error {
return readFull(r, maxDataSize, func(data []byte) error {
dbb := decompressedBufPool.Get()
defer decompressedBufPool.Put(dbb)
defer func() {
if cap(dbb.B) > 64*1024 && cap(dbb.B) > 4*len(dbb.B) {
// Do not store too big dbb to the pool if only a small part of the buffer is used last time.
// This should reduce memory waste.
return
}
decompressedBufPool.Put(dbb)
}()
var err error
dbb.B, err = decompress(dbb.B, data)
@@ -95,7 +102,7 @@ func readFull(r io.Reader, maxDataSize *flagutil.Bytes, callback func(data []byt
bb := fullReaderBufPool.Get()
defer func() {
if len(bb.B) > 1024*1024 && cap(bb.B) > 4*len(bb.B) {
if cap(bb.B) > 64*1024 && cap(bb.B) > 4*len(bb.B) {
// Do not store too big bb to the pool if only a small part of the buffer is used last time.
// This should reduce memory waste.
return

View File

@@ -50,6 +50,29 @@ func Init() {
}
}
type Config struct {
URLs []string
Interval time.Duration
ExtraLabels []string
Headers []string
DisableCompression bool
}
// InitWith must be called after logger.Init
// Supersedes command-line flags related to pushmetrics and initializes pushmetrics with the given config.
// This is needed for vmctl integration, see: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10375
func InitWith(c *Config) {
*pushURL = c.URLs
*pushExtraLabel = c.ExtraLabels
*pushHeader = c.Headers
*disableCompression = c.DisableCompression
if c.Interval > 0 {
*pushInterval = c.Interval
}
Init()
}
// Stop stops the periodic push of metrics.
// It is important to stop the push of metrics before disposing resources
// these metrics attached to. See related https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548

View File

@@ -173,7 +173,7 @@ func mustOpenIndexDB(id uint64, tr TimeRange, name, path string, s *Storage, isR
logger.Panicf("BUG: Storage must not be nil")
}
tfssCache := lrucache.NewCache(getTagFiltersCacheSize)
tfssCache := lrucache.NewCache(getTagFiltersCacheSize, 1*time.Hour)
tb := mergeset.MustOpenTable(path, dataFlushInterval, tfssCache.Reset, mergeTagToMetricIDsRows, isReadOnly)
db := &indexDB{
legacyMinMissingTimestampByKey: make(map[string]int64),
@@ -183,7 +183,7 @@ func mustOpenIndexDB(id uint64, tr TimeRange, name, path string, s *Storage, isR
tb: tb,
s: s,
tagFiltersToMetricIDsCache: tfssCache,
loopsPerDateTagFilterCache: lrucache.NewCache(getTagFiltersLoopsCacheSize),
loopsPerDateTagFilterCache: lrucache.NewCache(getTagFiltersLoopsCacheSize, 1*time.Hour),
metricIDCache: newMetricIDCache(),
dateMetricIDCache: newDateMetricIDCache(),
}

View File

@@ -4,7 +4,10 @@ import (
"sync"
"sync/atomic"
"time"
"unsafe"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/atomicutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/cgroup"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/timeutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/uint64set"
)
@@ -13,9 +16,119 @@ import (
// during data ingestion to decide whether a new entry needs to be added to the
// global index.
//
// The cache avoids synchronization on the read path if possible to reduce
// contention. Based on dateMetricIDCache ideas.
// The cache consists of multiple shards and avoids synchronization on the read
// path if possible to reduce contention.
type metricIDCache struct {
shards []metricIDCacheShard
// The shards are rotated in groups, one group at a time.
// rotationGroupSize tells the number of shards in one group,
// rotationGroupCount tells how many groups to rotate, and
// rotationGroupPeriod tells how often a group is rotated.
rotationGroupSize int
rotationGroupCount int
rotationGroupPeriod time.Duration
stopCh chan struct{}
rotationStoppedCh chan struct{}
}
func newMetricIDCache() *metricIDCache {
// Shards based on the number of CPUs are taken from
// lib/blockcache/blockcache.go.
rotationGroupSize := 1
rotationGroupCount := cgroup.AvailableCPUs()
if rotationGroupCount > 16 {
rotationGroupCount = 16
}
numShards := rotationGroupSize * rotationGroupCount
c := metricIDCache{
shards: make([]metricIDCacheShard, numShards),
rotationGroupSize: rotationGroupSize,
rotationGroupCount: rotationGroupCount,
rotationGroupPeriod: timeutil.AddJitterToDuration(1 * time.Minute),
stopCh: make(chan struct{}),
rotationStoppedCh: make(chan struct{}),
}
for i := range numShards {
c.shards[i].prev = &uint64set.Set{}
c.shards[i].next = &uint64set.Set{}
c.shards[i].curr.Store(&uint64set.Set{})
}
go c.startRotation()
return &c
}
func (c *metricIDCache) MustStop() {
close(c.stopCh)
<-c.rotationStoppedCh
}
func (c *metricIDCache) numShards() uint64 {
return uint64(len(c.shards))
}
func (c *metricIDCache) fullRotationPeriod() time.Duration {
return time.Duration(c.rotationGroupCount) * c.rotationGroupPeriod
}
func (c *metricIDCache) Stats() metricIDCacheStats {
var stats metricIDCacheStats
for i := range len(c.shards) {
s := c.shards[i].Stats()
stats.Size += s.Size
stats.SizeBytes += s.SizeBytes
stats.SyncsCount += s.SyncsCount
stats.RotationsCount += s.RotationsCount
}
return stats
}
func (c *metricIDCache) Has(metricID uint64) bool {
shardIdx := fastHashUint64(metricID) % uint64(len(c.shards))
return c.shards[shardIdx].Has(metricID)
}
func (c *metricIDCache) Set(metricID uint64) {
shardIdx := fastHashUint64(metricID) % uint64(len(c.shards))
c.shards[shardIdx].Set(metricID)
}
func (c *metricIDCache) rotate(rotationGroup int) {
for i := range len(c.shards) {
if i/c.rotationGroupSize == rotationGroup {
c.shards[i].rotate()
}
}
}
func (c *metricIDCache) startRotation() {
ticker := time.NewTicker(c.rotationGroupPeriod)
defer ticker.Stop()
rotationGroup := 0
for {
select {
case <-c.stopCh:
close(c.rotationStoppedCh)
return
case <-ticker.C:
// each tick rotate only subset of size metricIDCacheRotationGroupSize
// to avoid slow access for all shards at once
rotationGroup = (rotationGroup + 1) % c.rotationGroupCount
c.rotate(rotationGroup)
}
}
}
type metricIDCacheStats struct {
Size uint64
SizeBytes uint64
SyncsCount uint64
RotationsCount uint64
}
type metricIDCacheShardNopad struct {
// Contains immutable set of metricIDs.
curr atomic.Pointer[uint64set.Set]
@@ -39,36 +152,16 @@ type metricIDCache struct {
rotationsCount uint64
mu sync.Mutex
stopCh chan struct{}
rotationStoppedCh chan struct{}
}
func newMetricIDCache() *metricIDCache {
c := metricIDCache{
prev: &uint64set.Set{},
next: &uint64set.Set{},
stopCh: make(chan struct{}),
rotationStoppedCh: make(chan struct{}),
}
c.curr.Store(&uint64set.Set{})
go c.startRotation()
return &c
type metricIDCacheShard struct {
metricIDCacheShardNopad
// The padding prevents false sharing
_ [atomicutil.CacheLineSize - unsafe.Sizeof(metricIDCacheShardNopad{})%atomicutil.CacheLineSize]byte
}
func (c *metricIDCache) MustStop() {
close(c.stopCh)
<-c.rotationStoppedCh
}
type metricIDCacheStats struct {
Size uint64
SizeBytes uint64
SyncsCount uint64
RotationsCount uint64
}
func (c *metricIDCache) Stats() metricIDCacheStats {
func (c *metricIDCacheShard) Stats() metricIDCacheStats {
c.mu.Lock()
defer c.mu.Unlock()
@@ -91,7 +184,7 @@ func (c *metricIDCache) Stats() metricIDCacheStats {
return s
}
func (c *metricIDCache) Has(metricID uint64) bool {
func (c *metricIDCacheShard) Has(metricID uint64) bool {
if c.curr.Load().Has(metricID) {
// Fast path. The majority of calls must go here.
return true
@@ -101,7 +194,7 @@ func (c *metricIDCache) Has(metricID uint64) bool {
return c.hasSlow(metricID)
}
func (c *metricIDCache) hasSlow(metricID uint64) bool {
func (c *metricIDCacheShard) hasSlow(metricID uint64) bool {
c.mu.Lock()
defer c.mu.Unlock()
@@ -132,7 +225,7 @@ func (c *metricIDCache) hasSlow(metricID uint64) bool {
return ok
}
func (c *metricIDCache) Set(metricID uint64) {
func (c *metricIDCacheShard) Set(metricID uint64) {
c.mu.Lock()
c.next.Add(metricID)
c.mu.Unlock()
@@ -140,7 +233,7 @@ func (c *metricIDCache) Set(metricID uint64) {
// syncLocked merges data from curr into next and atomically replaces curr with
// next.
func (c *metricIDCache) syncLocked() {
func (c *metricIDCacheShard) syncLocked() {
curr := c.curr.Load()
c.next.Union(curr)
c.curr.Store(c.next)
@@ -148,23 +241,8 @@ func (c *metricIDCache) syncLocked() {
c.syncsCount++
}
func (c *metricIDCache) startRotation() {
d := timeutil.AddJitterToDuration(10 * time.Minute)
ticker := time.NewTicker(d)
defer ticker.Stop()
for {
select {
case <-c.stopCh:
close(c.rotationStoppedCh)
return
case <-ticker.C:
c.rotate()
}
}
}
// rotate atomically rotates next, curr, and prev cache parts.
func (c *metricIDCache) rotate() {
func (c *metricIDCacheShard) rotate() {
c.mu.Lock()
defer c.mu.Unlock()
curr := c.curr.Load()

View File

@@ -5,8 +5,8 @@ import (
"testing/synctest"
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/uint64set"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
)
func TestMetricIDCache_ClearedWhenUnused(t *testing.T) {
@@ -16,9 +16,7 @@ func TestMetricIDCache_ClearedWhenUnused(t *testing.T) {
c := newMetricIDCache()
defer c.MustStop()
c.Set(123)
time.Sleep(15 * time.Minute)
time.Sleep(15 * time.Minute)
time.Sleep(15 * time.Minute)
time.Sleep(3 * c.fullRotationPeriod())
if c.Has(123) {
t.Fatalf("entry is still in cache")
}
@@ -30,12 +28,11 @@ func TestMetricIDCache_ClearedWhenUnused(t *testing.T) {
c := newMetricIDCache()
defer c.MustStop()
c.Set(123)
time.Sleep(5 * time.Minute)
time.Sleep(c.rotationGroupPeriod - time.Second)
if !c.Has(123) {
t.Fatalf("entry not in cache")
}
time.Sleep(15 * time.Minute)
time.Sleep(15 * time.Minute)
time.Sleep(2 * c.fullRotationPeriod())
if c.Has(123) {
t.Fatalf("entry is still in cache")
}
@@ -48,7 +45,7 @@ func TestMetricIDCache_ClearedWhenUnused(t *testing.T) {
defer c.MustStop()
c.Set(123)
for range 10_000 {
time.Sleep(5 * time.Minute)
time.Sleep(c.rotationGroupPeriod - time.Second)
if !c.Has(123) {
t.Fatalf("entry not in cache")
}
@@ -58,7 +55,8 @@ func TestMetricIDCache_ClearedWhenUnused(t *testing.T) {
func TestMetricIDCache_Stats(t *testing.T) {
assertStats := func(t *testing.T, c *metricIDCache, want metricIDCacheStats) {
if diff := cmp.Diff(want, c.Stats()); diff != "" {
t.Helper()
if diff := cmp.Diff(want, c.Stats(), cmpopts.IgnoreFields(metricIDCacheStats{}, "SizeBytes")); diff != "" {
t.Fatalf("unexpected stats (-want, +got):\n%s", diff)
}
}
@@ -72,14 +70,11 @@ func TestMetricIDCache_Stats(t *testing.T) {
// Add metricIDs and check stats.
// At this point, all metricIDs are in next.
metricIDs := uint64set.Set{}
for metricID := range uint64(100_000) {
c.Set(metricID)
metricIDs.Add(metricID)
}
assertStats(t, c, metricIDCacheStats{
Size: 100_000,
SizeBytes: metricIDs.SizeBytes(),
Size: 100_000,
})
// Get all metricIDs and check stats.
@@ -91,26 +86,24 @@ func TestMetricIDCache_Stats(t *testing.T) {
}
assertStats(t, c, metricIDCacheStats{
Size: 100_000,
SizeBytes: metricIDs.SizeBytes(),
SyncsCount: 1,
SyncsCount: c.numShards(),
})
// Wait until next rotation.
// Wait until all groups are rotated.
// curr metricIDs will be moved to prev.
time.Sleep(15 * time.Minute)
time.Sleep(c.fullRotationPeriod() + time.Second)
assertStats(t, c, metricIDCacheStats{
Size: 100_000,
SizeBytes: metricIDs.SizeBytes(),
SyncsCount: 1,
RotationsCount: 1,
SyncsCount: c.numShards(),
RotationsCount: c.numShards(),
})
// Wait until another rotation.
// Wait until all groups are rotated.
// The cache now should be empty.
time.Sleep(15 * time.Minute)
time.Sleep(c.fullRotationPeriod())
assertStats(t, c, metricIDCacheStats{
SyncsCount: 1,
RotationsCount: 2,
SyncsCount: c.numShards(),
RotationsCount: 2 * c.numShards(),
})
})
}

View File

@@ -7,11 +7,29 @@ import (
"time"
)
type benchCacheState int
const (
benchCacheStateCold benchCacheState = iota
benchCacheStateWarm
benchCacheStateRotated
)
var benchCacheStates = [...]benchCacheState{benchCacheStateCold, benchCacheStateWarm, benchCacheStateRotated}
func (s benchCacheState) String() string {
return [...]string{" cold", " warm", "rotated"}[s]
}
func BenchmarkMetricIDCache_Has(b *testing.B) {
f := func(b *testing.B, numMetricIDs, distance int64, hitsOnly, warmUp bool) {
f := func(b *testing.B, numMetricIDs, distance int64, hitsOnly bool, state benchCacheState) {
b.Helper()
c := newMetricIDCache()
defer c.MustStop()
warmUp := state == benchCacheStateWarm || state == benchCacheStateRotated
rotate := state == benchCacheStateRotated
metricIDMin := time.Now().UnixNano()
metricIDMax := metricIDMin + numMetricIDs*distance
for metricID := metricIDMin; metricID <= metricIDMax; metricID += distance {
@@ -20,7 +38,11 @@ func BenchmarkMetricIDCache_Has(b *testing.B) {
b.Fatalf("metricID not in cache: %d", metricID)
}
}
if rotate {
c.rotate(rand.Intn(c.rotationGroupCount))
}
b.ResetTimer()
b.RunParallel(func(pb *testing.PB) {
if hitsOnly {
metricID := metricIDMin + rand.Int63n(numMetricIDs)*distance
@@ -45,27 +67,24 @@ func BenchmarkMetricIDCache_Has(b *testing.B) {
}
})
b.ReportAllocs()
b.ReportMetric(float64(c.Stats().SizeBytes), "sizeBytes")
}
subB := func(numMetricIDs, distance int64, hitsOnly, warmUp bool) {
hitsOrMisses := "hitsss"
subB := func(numMetricIDs, distance int64, hitsOnly bool, state benchCacheState) {
hitsOrMisses := " hits-only"
if !hitsOnly {
hitsOrMisses = "misses"
hitsOrMisses = "misses-only"
}
coldOrWarm := "cold"
if warmUp {
coldOrWarm = "warm"
}
name := fmt.Sprintf("%s/%s/n%d/d%d", hitsOrMisses, coldOrWarm, numMetricIDs, distance)
name := fmt.Sprintf("%s/%s/n%d/d%d", hitsOrMisses, state, numMetricIDs, distance)
b.Run(name, func(b *testing.B) {
f(b, numMetricIDs, distance, hitsOnly, warmUp)
f(b, numMetricIDs, distance, hitsOnly, state)
})
}
for _, hitsOnly := range []bool{true, false} {
for _, warmUp := range []bool{false, true} {
for _, state := range benchCacheStates {
for _, numMetricIDs := range []int64{100_000, 1_000_000, 10_000_000} {
for _, distance := range []int64{1, 10, 100} {
subB(numMetricIDs, distance, hitsOnly, warmUp)
subB(numMetricIDs, distance, hitsOnly, state)
}
}
}

View File

@@ -8,6 +8,7 @@ import (
"sort"
"strings"
"sync"
"time"
"unsafe"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/atomicutil"
@@ -866,7 +867,7 @@ var (
)
var (
regexpCache = lrucache.NewCache(getMaxRegexpCacheSize)
regexpCache = lrucache.NewCache(getMaxRegexpCacheSize, 1*time.Hour)
)
type regexpCacheValue struct {
@@ -921,7 +922,7 @@ var (
)
var (
prefixesCache = lrucache.NewCache(getMaxPrefixesCacheSize)
prefixesCache = lrucache.NewCache(getMaxPrefixesCacheSize, 1*time.Hour)
)
// RegexpPrefixesCacheSize returns the number of cached regexp prefixes for tag filters.