deployment/docker/rules: fixed expression for vmalert rules

Previously, vmselect may crash if there is not enough free disk space to store tmp files from vmstorage. vmselect stores data blocks received from vmstorage if response size exceeds in-memory limit.
 It reduced vmselect availability, because it makes impossible to serve small queries. And huge query may crash vmselect for all users.

 So this commit adds a error check instead. If there is not enough disk space, vmselect will stop query execution and return error back to user.

fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4688

.github/workflows/build.yml

@@ -73,11 +73,19 @@ jobs:
         id: go
         uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c  # v6.4.0
         with:
-          cache-dependency-path: |
-            go.sum
-            Makefile
-            app/**/Makefile
+          cache: false
           go-version-file: 'go.mod'
+
+      - name: Cache Go build artifacts
+        uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae  # v5.0.5
+        with:
+          path: |
+            ~/.cache/go-build
+            ~/go/pkg/mod
+          key: go-build-${{ matrix.os }}-${{ matrix.arch }}-${{ hashFiles('go.sum', 'Makefile') }}
+          restore-keys: |
+            go-build-${{ matrix.os }}-${{ matrix.arch }}-
+
       - run: go version
 
       - name: Build victoria-metrics for ${{ matrix.os }}-${{ matrix.arch }}

README.md

@@ -1,6 +1,6 @@
 # VictoriaMetrics
 
-[![Latest Release](https://img.shields.io/github/v/release/VictoriaMetrics/VictoriaMetrics?sort=semver&label=&filter=!*-victorialogs&logo=github&labelColor=gray&color=gray&link=https%3A%2F%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics%2Freleases%2Flatest)](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
+[![Latest Release](https://img.shields.io/github/v/release/VictoriaMetrics/VictoriaMetrics?logo=github&labelColor=gray&color=gray&label=Release)](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
 [![Docker Pulls](https://img.shields.io/docker/pulls/victoriametrics/victoria-metrics?label=&logo=docker&logoColor=white&labelColor=2496ED&color=2496ED&link=https%3A%2F%2Fhub.docker.com%2Fr%2Fvictoriametrics%2Fvictoria-metrics)](https://hub.docker.com/u/victoriametrics)
 [![Build Status](https://github.com/VictoriaMetrics/VictoriaMetrics/actions/workflows/build.yml/badge.svg?branch=master&link=https%3A%2F%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics%2Factions)](https://github.com/VictoriaMetrics/VictoriaMetrics/actions/workflows/build.yml)
 [![License](https://img.shields.io/github/license/VictoriaMetrics/VictoriaMetrics?labelColor=green&label=&link=https%3A%2F%2Fgithub.com%2FVictoriaMetrics%2FVictoriaMetrics%2Fblob%2Fmaster%2FLICENSE)](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/LICENSE)
@@ -41,8 +41,8 @@ VictoriaMetrics is optimized for timeseries data, even when old time series are
 * **Ideal for big data**: Works well with large amounts of time series data from APM, Kubernetes, IoT sensors, connected cars, industrial telemetry, financial data and various [Enterprise workloads](https://docs.victoriametrics.com/victoriametrics/enterprise/).
 * **Query language**: Supports both PromQL and the more performant MetricsQL.
 * **Easy to setup**: No dependencies, single [small binary](https://medium.com/@valyala/stripping-dependency-bloat-in-victoriametrics-docker-image-983fb5912b0d), configuration through command-line flags, but the default is also fine-tuned; backup and restore with [instant snapshots](https://medium.com/@valyala/how-victoriametrics-makes-instant-snapshots-for-multi-terabyte-time-series-data-e1f3fb0e0282).
-* **Global query view**: Multiple Prometheus instances or any other data sources may ingest data into VictoriaMetrics and queried via a single query.
-* **Various Protocols**: Support metric scraping, ingestion and backfilling in various protocol.
+* **Global query view**: Multiple Prometheus instances or any other data sources may ingest data into VictoriaMetrics and be queried via a single query.
+* **Various Protocols**: Support metric scraping, ingestion and backfilling in various protocols.
     * [Prometheus exporters](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-scrape-prometheus-exporters-such-as-node-exporter), [Prometheus remote write API](https://docs.victoriametrics.com/victoriametrics/integrations/prometheus/), [Prometheus exposition format](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-import-data-in-prometheus-exposition-format).
     * [InfluxDB line protocol](https://docs.victoriametrics.com/victoriametrics/integrations/influxdb/) over HTTP, TCP and UDP.
     * [Graphite plaintext protocol](https://docs.victoriametrics.com/victoriametrics/integrations/graphite/#ingesting) with [tags](https://graphite.readthedocs.io/en/latest/tags.html#carbon).
@@ -78,7 +78,7 @@ We strictly apply security measures in everything we do. VictoriaMetrics has ach
 Some good benchmarks VictoriaMetrics achieved:
 
 * **Minimal memory footprint**: handling millions of unique timeseries with [10x less RAM](https://medium.com/@valyala/insert-benchmarks-with-inch-influxdb-vs-victoriametrics-e31a41ae2893) than InfluxDB, up to [7x less RAM](https://valyala.medium.com/prometheus-vs-victoriametrics-benchmark-on-node-exporter-metrics-4ca29c75590f) than Prometheus, Thanos or Cortex.
-* **Highly scalable and performance** for [data ingestion](https://medium.com/@valyala/high-cardinality-tsdb-benchmarks-victoriametrics-vs-timescaledb-vs-influxdb-13e6ee64dd6b) and [querying](https://medium.com/@valyala/when-size-matters-benchmarking-victoriametrics-vs-timescale-and-influxdb-6035811952d4), [20x outperforms](https://medium.com/@valyala/insert-benchmarks-with-inch-influxdb-vs-victoriametrics-e31a41ae2893) InfluxDB and TimescaleDB.
+* **Highly scalable and performant** for [data ingestion](https://medium.com/@valyala/high-cardinality-tsdb-benchmarks-victoriametrics-vs-timescaledb-vs-influxdb-13e6ee64dd6b) and [querying](https://medium.com/@valyala/when-size-matters-benchmarking-victoriametrics-vs-timescale-and-influxdb-6035811952d4), [20x outperforms](https://medium.com/@valyala/insert-benchmarks-with-inch-influxdb-vs-victoriametrics-e31a41ae2893) InfluxDB and TimescaleDB.
 * **High data compression**: [70x more data points](https://medium.com/@valyala/when-size-matters-benchmarking-victoriametrics-vs-timescale-and-influxdb-6035811952d4) may be stored into limited storage than TimescaleDB, [7x less storage](https://valyala.medium.com/prometheus-vs-victoriametrics-benchmark-on-node-exporter-metrics-4ca29c75590f) space is required than Prometheus, Thanos or Cortex.
 * **Reducing storage costs**: [10x more effective](https://docs.victoriametrics.com/victoriametrics/casestudies/#grammarly) than Graphite according to the Grammarly case study.
 * **A single-node VictoriaMetrics** can replace medium-sized clusters built with competing solutions such as Thanos, M3DB, Cortex, InfluxDB or TimescaleDB. See [VictoriaMetrics vs Thanos](https://medium.com/@valyala/comparing-thanos-to-victoriametrics-cluster-b193bea1683), [Measuring vertical scalability](https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae), [Remote write storage wars - PromCon 2019](https://promcon.io/2019-munich/talks/remote-write-storage-wars/).
@@ -86,7 +86,7 @@ Some good benchmarks VictoriaMetrics achieved:
 
 ## Community and contributions
 
-Feel free asking any questions regarding VictoriaMetrics:
+Feel free to ask any questions regarding VictoriaMetrics:
 
 * [Slack Inviter](https://slack.victoriametrics.com/) and [Slack channel](https://victoriametrics.slack.com/)
 * [X (Twitter)](https://x.com/VictoriaMetrics/)

app/victoria-metrics/main.go

@@ -13,6 +13,7 @@ import (
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect"
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/promql"
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vmstorage"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/appmetrics"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/buildinfo"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/cgroup"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/envflag"
@@ -34,9 +35,10 @@ var (
 		"This can be changed with -promscrape.config.strictParse=false command-line flag")
 	maxIngestionRate = flag.Int("maxIngestionRate", 0, "The maximum number of samples vmsingle can receive per second. Data ingestion is paused when the limit is exceeded. "+
 		"By default there are no limits on samples ingestion rate.")
-	vmselectMaxConcurrentRequests = flag.Int("search.maxConcurrentRequests", getDefaultMaxConcurrentRequests(), "The maximum number of concurrent search requests. "+
-		"It shouldn't be high, since a single request can saturate all the CPU cores, while many concurrently executed requests may require high amounts of memory. "+
-		"See also -search.maxQueueDuration and -search.maxMemoryPerQuery")
+	vmselectMaxConcurrentRequests = flagutil.NewIntWithDynamicDefault("search.maxConcurrentRequests", getDefaultMaxConcurrentRequests(), "vmselect.getDefaultMaxConcurrentRequests()",
+		"The maximum number of concurrent search requests. "+
+			"It shouldn't be high, since a single request can saturate all the CPU cores, while many concurrently executed requests may require high amounts of memory. "+
+			"See also -search.maxQueueDuration and -search.maxMemoryPerQuery")
 	vmselectMaxQueueDuration = flag.Duration("search.maxQueueDuration", 10*time.Second, "The maximum time the request waits for execution when -search.maxConcurrentRequests "+
 		"limit is reached; see also -search.maxQueryDuration")
 )
@@ -63,6 +65,7 @@ func main() {
 	flag.CommandLine.SetOutput(os.Stdout)
 	flag.Usage = usage
 	envflag.Parse()
+	initSecretFlags()
 	buildinfo.Init()
 	logger.Init()
 
@@ -89,7 +92,9 @@ func main() {
 	}
 	logger.Infof("starting VictoriaMetrics at %q...", listenAddrs)
 	startTime := time.Now()
+
 	vmstorage.Init(*vmselectMaxConcurrentRequests, *vmselectMaxQueueDuration, promql.ResetRollupResultCacheIfNeeded)
+	appmetrics.MustCreateUncleanShutdownMarker(vmstorage.DataPath())
 	vmselect.Init(*vmselectMaxConcurrentRequests, *vmselectMaxQueueDuration)
 	vminsertcommon.StartIngestionRateLimiter(*maxIngestionRate)
 	vminsert.Init()
@@ -119,6 +124,7 @@ func main() {
 
 	vmstorage.Stop()
 	vmselect.Stop()
+	appmetrics.MustRemoveUncleanShutdownMarker(vmstorage.DataPath())
 
 	logger.Infof("the VictoriaMetrics has been stopped in %.3f seconds", time.Since(startTime).Seconds())
 }
@@ -170,3 +176,9 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/
 `
 	flagutil.Usage(s)
 }
+
+// initSecretFlags manages the secret flags for this app and must be called after flag parsing and before logger init.
+func initSecretFlags() {
+	pushmetrics.InitSecretFlags()
+	vmselect.InitSecretFlags()
+}

app/vmagent/main.go

@@ -84,7 +84,7 @@ var (
 	maxLabelNameLen        = flag.Int("maxLabelNameLen", 0, "The maximum length of label names in the accepted time series. Series with longer label name are ignored. In this case the vm_rows_ignored_total{reason=\"too_long_label_name\"} metric at /metrics page is incremented")
 	maxLabelValueLen       = flag.Int("maxLabelValueLen", 0, "The maximum length of label values in the accepted time series. Series with longer label value are ignored. In this case the vm_rows_ignored_total{reason=\"too_long_label_value\"} metric at /metrics page is incremented")
 
-	enableMultitenancyViaHeaders = flag.Bool("enableMultitenancyViaHeaders", false, "Enables multitenancy via HTTP headers. "+
+	enableMultitenancyViaHeaders = flag.Bool("enableMultitenancyViaHeaders", true, "Enables multitenancy via HTTP headers. "+
 		"See https://docs.victoriametrics.com/victoriametrics/vmagent/#multitenancy")
 )
 
@@ -115,7 +115,7 @@ func main() {
 	flag.CommandLine.SetOutput(os.Stdout)
 	flag.Usage = usage
 	envflag.Parse()
-	remotewrite.InitSecretFlags()
+	initSecretFlags()
 	buildinfo.Init()
 	logger.Init()
 	opentelemetry.Init()
@@ -843,3 +843,9 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/vmagent/ .
 `
 	flagutil.Usage(s)
 }
+
+// initSecretFlags manages the secret flags for this app and must be called after flag parsing and before logger init.
+func initSecretFlags() {
+	remotewrite.InitSecretFlags()
+	pushmetrics.InitSecretFlags()
+}

app/vmagent/prometheusimport/request_handler_test.go

@@ -52,6 +52,7 @@ func setUp() {
 
 func tearDown() {
 	protoparserutil.StopUnmarshalWorkers()
+	remotewrite.Stop()
 	srv.Close()
 	logger.ResetOutputForTest()
 	tmpDataDir := flag.Lookup("remoteWrite.tmpDataPath").Value.String()

app/vmagent/remotewrite/remotewrite.go

@@ -15,6 +15,7 @@ import (
 	"github.com/VictoriaMetrics/metrics"
 	"github.com/cespare/xxhash/v2"
 
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/appmetrics"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/auth"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/bloomfilter"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/cgroup"
@@ -62,9 +63,10 @@ var (
 		"See also -remoteWrite.maxDiskUsagePerURL and -remoteWrite.disableOnDiskQueue")
 	keepDanglingQueues = flag.Bool("remoteWrite.keepDanglingQueues", false, "Keep persistent queues contents at -remoteWrite.tmpDataPath in case there are no matching -remoteWrite.url. "+
 		"Useful when -remoteWrite.url is changed temporarily and persistent queue files will be needed later on.")
-	queues = flagutil.NewArrayInt("remoteWrite.queues", cgroup.AvailableCPUs()*2, "The number of concurrent queues to each -remoteWrite.url. Set more queues if default number of queues "+
-		"isn't enough for sending high volume of collected data to remote storage. "+
-		"Default value depends on the number of available CPU cores. It should work fine in most cases since it minimizes resource usage")
+	queues = flagutil.NewArrayIntWithDynamicDefault("remoteWrite.queues", cgroup.AvailableCPUs()*2, "2*cgroup.AvailableCPUs()",
+		"The number of concurrent queues to each -remoteWrite.url. Set more queues if default number of queues "+
+			"isn't enough for sending high volume of collected data to remote storage. "+
+			"Default value depends on the number of available CPU cores. It should work fine in most cases since it minimizes resource usage")
 	inmemoryQueues = flagutil.NewArrayInt("remoteWrite.inmemoryQueues", 0, "The number of additional workers per each -remoteWrite.url, which send only recently ingested data from the in-memory queue, "+
 		"while the file-based queue at -remoteWrite.tmpDataPath is drained by workers configured via -remoteWrite.queues. "+
 		"This reduces delivery lag for fresh samples when the file-based queue contains a backlog accumulated during remote storage outages.")
@@ -156,7 +158,8 @@ var maxQueues = cgroup.AvailableCPUs() * 16
 
 const persistentQueueDirname = "persistent-queue"
 
-// InitSecretFlags must be called after flag.Parse and before any logging.
+// InitSecretFlags manages the secret flags for this pkg and must be called by app-level initSecretFlags.
+// It should run before logger initialization and package Init() (if exists).
 func InitSecretFlags() {
 	if !*showRemoteWriteURL {
 		// remoteWrite.url can contain authentication codes, so hide it at `/metrics` output.
@@ -232,6 +235,7 @@ func Init() {
 	initStreamAggrConfigGlobal()
 
 	initRemoteWriteCtxs(*remoteWriteURLs)
+	appmetrics.MustCreateUncleanShutdownMarker(*tmpDataPath)
 
 	disableOnDiskQueues := []bool(*disableOnDiskQueue)
 	disableOnDiskQueueAny = slices.Contains(disableOnDiskQueues, true)
@@ -245,6 +249,8 @@ func Init() {
 	dropDanglingQueues()
 
 	// Start config reloader.
+	configReloaderStopCh = make(chan struct{})
+	configReloaderWG = sync.WaitGroup{}
 	configReloaderWG.Go(func() {
 		for {
 			select {
@@ -331,7 +337,7 @@ func initRemoteWriteCtxs(urls []string) {
 }
 
 var (
-	configReloaderStopCh = make(chan struct{})
+	configReloaderStopCh chan struct{}
 	configReloaderWG     sync.WaitGroup
 )
 
@@ -388,6 +394,8 @@ func Stop() {
 	if sl := dailySeriesLimiter; sl != nil {
 		sl.MustStop()
 	}
+
+	appmetrics.MustRemoveUncleanShutdownMarker(*tmpDataPath)
 }
 
 // PushDropSamplesOnFailure pushes wr to the configured remote storage systems set via -remoteWrite.url

app/vmalert/datasource/init.go

@@ -60,7 +60,8 @@ var (
 		`Only valid for VictoriaMetrics as the datasource.`)
 )
 
-// InitSecretFlags must be called after flag.Parse and before any logging
+// InitSecretFlags manages the secret flags for this pkg and must be called by app-level initSecretFlags.
+// It should run before logger initialization and package Init() (if exists).
 func InitSecretFlags() {
 	if !*showDatasourceURL {
 		flagutil.RegisterSecretFlag("datasource.url")

app/vmalert/main.go

@@ -88,10 +88,7 @@ func main() {
 	flag.CommandLine.SetOutput(os.Stdout)
 	flag.Usage = usage
 	envflag.Parse()
-	remoteread.InitSecretFlags()
-	remotewrite.InitSecretFlags()
-	datasource.InitSecretFlags()
-	notifier.InitSecretFlags()
+	initSecretFlags()
 	buildinfo.Init()
 	logger.Init()
 
@@ -438,3 +435,12 @@ func getLastConfigError() error {
 	defer lastConfigErrMu.RUnlock()
 	return lastConfigErr
 }
+
+// initSecretFlags manages the secret flags for this app and must be called after flag parsing and before logger init.
+func initSecretFlags() {
+	remoteread.InitSecretFlags()
+	remotewrite.InitSecretFlags()
+	datasource.InitSecretFlags()
+	notifier.InitSecretFlags()
+	pushmetrics.InitSecretFlags()
+}

app/vmalert/notifier/init.go

@@ -189,7 +189,8 @@ func Init(extLabels map[string]string, extURL string) error {
 	return nil
 }
 
-// InitSecretFlags must be called after flag.Parse and before any logging
+// InitSecretFlags manages the secret flags for this pkg and must be called by app-level initSecretFlags.
+// It should run before logger initialization and package Init() (if exists).
 func InitSecretFlags() {
 	if !*showNotifierURL {
 		flagutil.RegisterSecretFlag("notifier.url")

app/vmalert/remoteread/init.go

@@ -56,7 +56,8 @@ var (
 	oauth2Scopes   = flag.String("remoteRead.oauth2.scopes", "", "Optional OAuth2 scopes to use for -remoteRead.url. Scopes must be delimited by ';'.")
 )
 
-// InitSecretFlags must be called after flag.Parse and before any logging
+// InitSecretFlags manages the secret flags for this pkg and must be called by app-level initSecretFlags.
+// It should run before logger initialization and package Init() (if exists).
 func InitSecretFlags() {
 	if !*showRemoteReadURL {
 		flagutil.RegisterSecretFlag("remoteRead.url")

app/vmalert/remotewrite/client.go

@@ -284,7 +284,15 @@ func (c *Client) flush(ctx context.Context, wr *prompb.WriteRequest) {
 	bb := writeRequestBufPool.Get()
 	bb.B = wr.MarshalProtobuf(bb.B[:0])
 	zb := compressBufPool.Get()
-	defer compressBufPool.Put(zb)
+	// A failed send may leave the http transport still reading zb.B in a separate goroutine
+	// even after send returns, so zb is returned to the pool only if no send attempt has failed.
+	// See https://pkg.go.dev/net/http#RoundTripper
+	sendFailed := false
+	defer func() {
+		if !sendFailed {
+			compressBufPool.Put(zb)
+		}
+	}()
 	if c.isVMRemoteWrite.Load() {
 		zb.B = zstd.CompressLevel(zb.B[:0], bb.B, 0)
 	} else {
@@ -303,10 +311,13 @@ func (c *Client) flush(ctx context.Context, wr *prompb.WriteRequest) {
 L:
 	for {
 		err := c.send(ctx, zb.B)
-		if err != nil && (errors.Is(err, io.EOF) || netutil.IsTrivialNetworkError(err)) {
-			// Something in the middle between client and destination might be closing
-			// the connection. So we do a one more attempt in hope request will succeed.
-			err = c.send(ctx, zb.B)
+		if err != nil {
+			sendFailed = true
+			if errors.Is(err, io.EOF) || netutil.IsTrivialNetworkError(err) {
+				// Something in the middle between client and destination might be closing
+				// the connection. So we do a one more attempt in hope request will succeed.
+				err = c.send(ctx, zb.B)
+			}
 		}
 		if err == nil {
 			sentRows.Add(len(wr.Timeseries))

app/vmalert/remotewrite/debug_client.go

@@ -36,6 +36,13 @@ func NewDebugClient() (*DebugClient, error) {
 	if err != nil {
 		return nil, fmt.Errorf("failed to create transport for -remoteWrite.url=%q: %w", *addr, err)
 	}
+	tr.IdleConnTimeout = *idleConnectionTimeout
+	// DebugClient sends every series in a separate request, so it needs more idle
+	// connections than the two http.DefaultTransport keeps per host.
+	tr.MaxIdleConnsPerHost = *maxIdleConnections
+	if tr.MaxIdleConns != 0 && tr.MaxIdleConns < tr.MaxIdleConnsPerHost {
+		tr.MaxIdleConns = tr.MaxIdleConnsPerHost
+	}
 	c := &DebugClient{
 		c: &http.Client{
 			Timeout:   *sendTimeout,

app/vmalert/remotewrite/debug_client_conns_test.go

@@ -0,0 +1,45 @@
+package remotewrite
+
+import (
+	"net/http"
+	"testing"
+)
+
+// TestDebugClient_IdleConns makes sure DebugClient keeps enough idle connections
+// to -remoteWrite.url. Every series is pushed in a separate request, so with the
+// two idle connections per host of http.DefaultTransport most of the concurrent
+// requests would open a new connection and leave a socket in TIME_WAIT state.
+func TestDebugClient_IdleConns(t *testing.T) {
+	f := func(maxIdle int) {
+		t.Helper()
+
+		oldAddr, oldMaxIdle := *addr, *maxIdleConnections
+		*addr, *maxIdleConnections = "http://localhost:8428", maxIdle
+		defer func() {
+			*addr, *maxIdleConnections = oldAddr, oldMaxIdle
+		}()
+
+		client, err := NewDebugClient()
+		if err != nil {
+			t.Fatalf("failed to create debug client: %s", err)
+		}
+		tr, ok := client.c.Transport.(*http.Transport)
+		if !ok {
+			t.Fatalf("unexpected transport type %T", client.c.Transport)
+		}
+		if tr.MaxIdleConnsPerHost != maxIdle {
+			t.Fatalf("unexpected MaxIdleConnsPerHost; got %d; want %d", tr.MaxIdleConnsPerHost, maxIdle)
+		}
+		if tr.MaxIdleConns != 0 && tr.MaxIdleConns < maxIdle {
+			t.Fatalf("MaxIdleConns=%d is lower than MaxIdleConnsPerHost=%d", tr.MaxIdleConns, maxIdle)
+		}
+		if tr.IdleConnTimeout != *idleConnectionTimeout {
+			t.Fatalf("unexpected IdleConnTimeout; got %s; want %s", tr.IdleConnTimeout, *idleConnectionTimeout)
+		}
+	}
+
+	f(100)
+
+	// the number of idle connections must be raised together with the total limit
+	f(1000)
+}

app/vmalert/remotewrite/init.go

@@ -34,10 +34,12 @@ var (
 	bearerTokenFile = flag.String("remoteWrite.bearerTokenFile", "", "Optional path to bearer token file to use for -remoteWrite.url.")
 
 	idleConnectionTimeout = flag.Duration("remoteWrite.idleConnTimeout", 50*time.Second, `Defines a duration for idle (keep-alive connections) to exist. Consider settings this value less to the value of "-http.idleConnTimeout". It must prevent possible "write: broken pipe" and "read: connection reset by peer" errors.`)
+	maxIdleConnections    = flag.Int("remoteWrite.maxIdleConnections", 100, `Defines the number of idle (keep-alive connections) to -remoteWrite.url for the vmalert-tool debug writer, which sends every series in a separate request. Too low a value may result in a high number of sockets in TIME_WAIT state.`)
 
-	maxQueueSize  = flag.Int("remoteWrite.maxQueueSize", defaultMaxQueueSize, "Defines the max number of pending datapoints to remote write endpoint")
-	maxBatchSize  = flag.Int("remoteWrite.maxBatchSize", defaultMaxBatchSize, "Defines max number of timeseries to be flushed at once")
-	concurrency   = flag.Int("remoteWrite.concurrency", defaultConcurrency, "Defines number of writers for concurrent writing into remote write endpoint. Default value depends on the number of available CPU cores.")
+	maxQueueSize = flag.Int("remoteWrite.maxQueueSize", defaultMaxQueueSize, "Defines the max number of pending datapoints to remote write endpoint")
+	maxBatchSize = flag.Int("remoteWrite.maxBatchSize", defaultMaxBatchSize, "Defines max number of timeseries to be flushed at once")
+	concurrency  = flagutil.NewIntWithDynamicDefault("remoteWrite.concurrency", defaultConcurrency, "2*cgroup.AvailableCPUs()",
+		"Defines number of writers for concurrent writing into remote write endpoint. Default value depends on the number of available CPU cores.")
 	flushInterval = flag.Duration("remoteWrite.flushInterval", defaultFlushInterval, "Defines interval of flushes to remote write endpoint")
 
 	tlsInsecureSkipVerify = flag.Bool("remoteWrite.tlsInsecureSkipVerify", false, "Whether to skip tls verification when connecting to -remoteWrite.url")
@@ -57,7 +59,8 @@ var (
 	oauth2Scopes   = flag.String("remoteWrite.oauth2.scopes", "", "Optional OAuth2 scopes to use for -notifier.url. Scopes must be delimited by ';'.")
 )
 
-// InitSecretFlags must be called after flag.Parse and before any logging
+// InitSecretFlags manages the secret flags for this pkg and must be called by app-level initSecretFlags.
+// It should run before logger initialization and package Init() (if exists).
 func InitSecretFlags() {
 	if !*showRemoteWriteURL {
 		flagutil.RegisterSecretFlag("remoteWrite.url")

app/vmalert/replay.go

@@ -30,7 +30,8 @@ var (
 		"Progress bar rendering might be verbose or break the logs parsing, so it is recommended to be disabled when not used in interactive mode.")
 	ruleEvaluationConcurrency = flag.Int("replay.ruleEvaluationConcurrency", 1, "The maximum number of concurrent '/query_range' requests when replay recording rule or alerting rule with for=0. "+
 		"Increasing this value when replaying for a long time, since each request is limited by -replay.maxDatapointsPerQuery.")
-	continueWithExecutionErr = flag.Bool("replay.continueWithExecutionErr", false, "Whether to continue replaying other rules if a rule execution fails with a 422 response code, which can happen due to an expression syntax error or a resource limit being hit.")
+	continueWithExecutionErr = flag.Bool("replay.continueWithExecutionErr", false, "Whether to continue replaying other rules if a rule execution fails with a 400 or 422 response code, "+
+		"which can happen due to an expression syntax error or a resource limit being hit.")
 )
 
 func replay(groupsCfg []config.Group, qb datasource.QuerierBuilder, rw remotewrite.RWClient) (totalRows, droppedRows int, err error) {

app/vmalert/rule/alerting.go

@@ -462,7 +462,11 @@ func (ar *AlertingRule) exec(ctx context.Context, ts time.Time, limit int) ([]pr
 	}
 
 	isPartial := isPartialResponse(res)
-	ar.logDebugf(ts, nil, "query returned %d series (elapsed: %s, isPartial: %t)", curState.Samples, curState.Duration, isPartial)
+	seriesFetched := 0
+	if res.SeriesFetched != nil {
+		seriesFetched = *res.SeriesFetched
+	}
+	ar.logDebugf(ts, nil, "query returned %d series (series_fetched: %d, elapsed: %s, isPartial: %t)", curState.Samples, seriesFetched, curState.Duration, isPartial)
 	qFn := func(query string) ([]datasource.Metric, error) {
 		res, _, err := ar.q.Query(ctx, query, ts)
 		return res.Data, err

app/vmalert/rule/group.go

@@ -290,6 +290,8 @@ func (g *Group) updateWith(newGroup *Group) error {
 	g.Headers = newGroup.Headers
 	g.NotifierHeaders = newGroup.NotifierHeaders
 	g.Labels = newGroup.Labels
+	g.EvalDelay = newGroup.EvalDelay
+	g.evalAlignment = newGroup.evalAlignment
 	g.Limit = newGroup.Limit
 	g.checksum = newGroup.checksum
 	g.Rules = newRules
@@ -337,7 +339,7 @@ func (g *Group) Init() {
 		i := g.Interval.Seconds()
 		return i
 	})
-	g.metrics.iterationLimit = g.metrics.set.NewGauge(fmt.Sprintf(`vmalert_rule_group_results_limit{%s}`, labels), func() float64 {
+	g.metrics.iterationLimit = g.metrics.set.NewGauge(fmt.Sprintf(`vmalert_group_rule_results_limit{%s}`, labels), func() float64 {
 		g.mu.RLock()
 		limit := g.Limit
 		g.mu.RUnlock()
@@ -373,7 +375,7 @@ func (g *Group) Start(ctx context.Context, rw remotewrite.RWClient, rr datasourc
 				g.mu.Lock()
 				err := g.updateWith(ng)
 				if err != nil {
-					logger.Errorf("group %q: failed to update: %s", g.Name, err)
+					logger.Errorf("group %q (file=%q): failed to update: %s", g.Name, g.File, err)
 					g.mu.Unlock()
 					continue
 				}
@@ -412,7 +414,7 @@ func (g *Group) Start(ctx context.Context, rw remotewrite.RWClient, rr datasourc
 		errs := e.execConcurrently(ctx, g.Rules, ts, g.Concurrency, resolveDuration, g.Limit)
 		for err := range errs {
 			if err != nil {
-				logger.Errorf("group %q: %s", g.Name, err)
+				logger.Errorf("group %q (file=%q): %s", g.Name, g.File, err)
 			}
 		}
 		g.metrics.iterationDuration.UpdateDuration(start)
@@ -441,17 +443,17 @@ func (g *Group) Start(ctx context.Context, rw remotewrite.RWClient, rr datasourc
 	if rr != nil {
 		err := g.restore(ctx, rr, realEvalTS, *remoteReadLookBack)
 		if err != nil {
-			logger.Errorf("error while restoring ruleState for group %q: %s", g.Name, err)
+			logger.Errorf("error while restoring ruleState for group %q (file=%q): %s", g.Name, g.File, err)
 		}
 	}
 
 	for {
 		select {
 		case <-ctx.Done():
-			logger.Infof("group %q: context cancelled", g.Name)
+			logger.Infof("group %q (file=%q): context cancelled", g.Name, g.File)
 			return
 		case <-g.doneCh:
-			logger.Infof("group %q: received stop signal", g.Name)
+			logger.Infof("group %q (file=%q): received stop signal", g.Name, g.File)
 			return
 		case ng := <-g.updateCh:
 			g.mu.Lock()
@@ -465,7 +467,7 @@ func (g *Group) Start(ctx context.Context, rw remotewrite.RWClient, rr datasourc
 
 			err := g.updateWith(ng)
 			if err != nil {
-				logger.Errorf("group %q: failed to update: %s", g.Name, err)
+				logger.Errorf("group %q (file=%q): failed to update: %s", g.Name, g.File, err)
 				g.mu.Unlock()
 				continue
 			}
@@ -543,8 +545,8 @@ func (g *Group) delayBeforeStart(ts time.Time, maxDelay time.Duration) time.Dura
 
 func (g *Group) infof(format string, args ...any) {
 	msg := fmt.Sprintf(format, args...)
-	logger.Infof("group %q %s; interval=%v; eval_offset=%v; concurrency=%d",
-		g.Name, msg, g.Interval, g.EvalOffset, g.Concurrency)
+	logger.Infof("group %q (file=%q; interval=%v; eval_offset=%v; concurrency=%d) %s",
+		g.Name, g.File, g.Interval, g.EvalOffset, g.Concurrency, msg)
 }
 
 // Replay performs group replay

app/vmalert/rule/group_test.go

@@ -78,6 +78,12 @@ func TestUpdateWith(t *testing.T) {
 		if g.Debug != expect.Debug {
 			t.Fatalf("expected to have debug %v; got %v", expect.Debug, g.Debug)
 		}
+		if !durationPtrEqual(g.EvalDelay, expect.EvalDelay) {
+			t.Fatalf("expected to have eval_delay %v; got %v", expect.EvalDelay, g.EvalDelay)
+		}
+		if !boolPtrEqual(g.evalAlignment, expect.evalAlignment) {
+			t.Fatalf("expected to have eval_alignment %v; got %v", expect.evalAlignment, g.evalAlignment)
+		}
 	}
 
 	// new rule
@@ -237,6 +243,37 @@ func TestUpdateWith(t *testing.T) {
 			{Alert: "foo1", Debug: &debug},
 		},
 	})
+
+	// update group evaluation settings
+	evalDelay := promutil.NewDuration(time.Minute)
+	evalAlignment := false
+	f(config.Group{
+		Rules: []config.Rule{{
+			Record: "foo",
+			Expr:   "max(up)",
+		}},
+	}, config.Group{
+		EvalDelay:     evalDelay,
+		EvalAlignment: &evalAlignment,
+		Rules: []config.Rule{{
+			Record: "foo",
+			Expr:   "min(up)",
+		}},
+	})
+}
+
+func durationPtrEqual(a, b *time.Duration) bool {
+	if a == nil || b == nil {
+		return a == b
+	}
+	return *a == *b
+}
+
+func boolPtrEqual(a, b *bool) bool {
+	if a == nil || b == nil {
+		return a == b
+	}
+	return *a == *b
 }
 
 func TestUpdateDuringRandSleep(t *testing.T) {

app/vmalert/rule/recording.go

@@ -208,7 +208,11 @@ func (rr *RecordingRule) exec(ctx context.Context, ts time.Time, limit int) ([]p
 		return nil, curState.Err
 	}
 
-	rr.logDebugf(ts, "query returned %d samples (elapsed: %s, isPartial: %t)", curState.Samples, curState.Duration, isPartialResponse(res))
+	seriesFetched := 0
+	if res.SeriesFetched != nil {
+		seriesFetched = *res.SeriesFetched
+	}
+	rr.logDebugf(ts, "query returned %d samples (series_fetched: %d, elapsed: %s, isPartial: %t)", curState.Samples, seriesFetched, curState.Duration, isPartialResponse(res))
 
 	qMetrics := res.Data
 	numSeries := len(qMetrics)

app/vmalert/rule/rule.go

@@ -132,9 +132,10 @@ func replayRule(r Rule, start, end time.Time, rw remotewrite.RWClient, replayRul
 		var esc *httpserver.ErrorWithStatusCode
 		if errors.As(err, &esc) {
 			statusCode := esc.StatusCode
-			// if the status code is 422, it means that the query was executed but failed due to an expression syntax error or a the resource limit being hit,
-			// continue replaying but skip the problematic execution if continueWithExecutionErr is true, otherwise, return the error without retry.
-			if statusCode == http.StatusUnprocessableEntity {
+			// if the status code is 400 or 422, the query failed due to reasons such as an expression syntax error or a resource limit being hit,
+			// rather than datasource unavailability.
+			// Continue replaying but skip the problematic execution if continueWithExecutionErr is true, otherwise, return the error without retry.
+			if statusCode == http.StatusUnprocessableEntity || statusCode == http.StatusBadRequest {
 				if continueWithExecutionErr {
 					logger.Errorf("rule %q: %s", r, err)
 					return 0, nil

app/vmauth/main.go

@@ -96,6 +96,7 @@ func main() {
 	flag.CommandLine.SetOutput(os.Stdout)
 	flag.Usage = usage
 	envflag.Parse()
+	initSecretFlags()
 	buildinfo.Init()
 	logger.Init()
 
@@ -911,3 +912,8 @@ func slowdownUnauthorizedResponse(r *http.Request) {
 	}
 	timerpool.Put(t)
 }
+
+// initSecretFlags manages the secret flags for this app and must be called after flag parsing and before logger init.
+func initSecretFlags() {
+	pushmetrics.InitSecretFlags()
+}

app/vmbackup/main.go

@@ -47,9 +47,8 @@ func main() {
 	// Write flags and help message to stdout, since it is easier to grep or pipe.
 	flag.CommandLine.SetOutput(os.Stdout)
 	flag.Usage = usage
-	flagutil.RegisterSecretFlag("snapshot.createURL")
-	flagutil.RegisterSecretFlag("snapshot.deleteURL")
 	envflag.Parse()
+	initSecretFlags()
 	buildinfo.Init()
 	logger.Init()
 
@@ -273,3 +272,10 @@ func newRemoteOriginFS(ctx context.Context) (common.RemoteFS, error) {
 	}
 	return fs, nil
 }
+
+// initSecretFlags manages the secret flags for this app and must be called after flag parsing and before logger init.
+func initSecretFlags() {
+	flagutil.RegisterSecretFlag("snapshot.createURL")
+	flagutil.RegisterSecretFlag("snapshot.deleteURL")
+	pushmetrics.InitSecretFlags()
+}

app/vmctl/main.go

@@ -47,6 +47,7 @@ func main() {
 	start := time.Now()
 	beforeFn := func(c *cli.Context) error {
 		flag.Parse()
+		initSecretFlags()
 		logger.Init()
 		isSilent = c.Bool(globalSilent)
 		if c.Bool(globalDisableProgressBar) {
@@ -619,3 +620,8 @@ func initConfigVM(c *cli.Context) (vm.Config, error) {
 		Backoff:            bf,
 	}, nil
 }
+
+// initSecretFlags manages the secret flags for this app and must be called after flag parsing and before logger init.
+func initSecretFlags() {
+	pushmetrics.InitSecretFlags()
+}

app/vmrestore/main.go

@@ -38,6 +38,7 @@ func main() {
 	flag.CommandLine.SetOutput(os.Stdout)
 	flag.Usage = usage
 	envflag.Parse()
+	initSecretFlags()
 	buildinfo.Init()
 	logger.Init()
 
@@ -112,3 +113,8 @@ func newSrcFS(ctx context.Context) (common.RemoteFS, error) {
 	}
 	return fs, nil
 }
+
+// initSecretFlags manages the secret flags for this app and must be called after flag parsing and before logger init.
+func initSecretFlags() {
+	pushmetrics.InitSecretFlags()
+}

app/vmselect/main.go

@@ -59,6 +59,12 @@ func Init(vmselectMaxConcurrentRequests int, vmselectMaxQueueDuration time.Durat
 	initVMUIConfig()
 
 	vmalertproxy.Init(*vmalertProxyURL)
+
+}
+
+// InitSecretFlags manages the secret flags for this pkg and must be called by app-level initSecretFlags.
+// It should run before logger initialization and package Init() (if exists).
+func InitSecretFlags() {
 	flagutil.RegisterSecretFlag("vmalert.proxyURL")
 }
 

app/vmselect/netstorage/netstorage.go

@@ -20,6 +20,7 @@ import (
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/cgroup"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/fasttime"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/flagutil"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/querytracer"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage/metricnamestats"
@@ -30,11 +31,12 @@ var (
 	maxSamplesPerSeries = flag.Int("search.maxSamplesPerSeries", 30e6, "The maximum number of raw samples a single query can scan per each time series. This option allows limiting memory usage")
 	maxSamplesPerQuery  = flag.Int("search.maxSamplesPerQuery", 1e9, "The maximum number of raw samples a single query can process across all time series. "+
 		"This protects from heavy queries, which select unexpectedly high number of raw samples. See also -search.maxSamplesPerSeries")
-	maxWorkersPerQuery = flag.Int("search.maxWorkersPerQuery", defaultMaxWorkersPerQuery, "The maximum number of CPU cores a single query can use. "+
-		"The default value should work good for most cases. "+
-		"The flag can be set to lower values for improving performance of big number of concurrently executed queries. "+
-		"The flag can be set to bigger values for improving performance of heavy queries, which scan big number of time series (>10K) and/or big number of samples (>100M). "+
-		"There is no sense in setting this flag to values bigger than the number of CPU cores available on the system")
+	maxWorkersPerQuery = flagutil.NewIntWithDynamicDefault("search.maxWorkersPerQuery", defaultMaxWorkersPerQuery, "netstorage.defaultMaxWorkersPerQuery()",
+		"The maximum number of CPU cores a single query can use. "+
+			"The default value should work good for most cases. "+
+			"The flag can be set to lower values for improving performance of big number of concurrently executed queries. "+
+			"The flag can be set to bigger values for improving performance of heavy queries, which scan big number of time series (>10K) and/or big number of samples (>100M). "+
+			"There is no sense in setting this flag to values bigger than the number of CPU cores available on the system")
 )
 
 // Result is a single timeseries result.

app/vmselect/netstorage/tmp_blocks_file.go

@@ -62,6 +62,9 @@ type tmpBlocksFile struct {
 	r *fs.ReaderAt
 
 	offset uint64
+
+	// err stores the first error occurred while writing the temporary blocks file.
+	err error
 }
 
 func getTmpBlocksFile() *tmpBlocksFile {
@@ -82,6 +85,7 @@ func putTmpBlocksFile(tbf *tmpBlocksFile) {
 	tbf.f = nil
 	tbf.r = nil
 	tbf.offset = 0
+	tbf.err = nil
 	tmpBlocksFilePool.Put(tbf)
 }
 
@@ -109,6 +113,10 @@ var (
 // and this must be handled.
 func (tbf *tmpBlocksFile) WriteBlockRefData(b []byte) (tmpBlockAddr, error) {
 	var addr tmpBlockAddr
+	if tbf.err != nil {
+		// Do not write anything to the tbf after the first failed write
+		return addr, tbf.err
+	}
 	addr.offset = tbf.offset
 	addr.size = len(b)
 	tbf.offset += uint64(addr.size)
@@ -122,7 +130,8 @@ func (tbf *tmpBlocksFile) WriteBlockRefData(b []byte) (tmpBlockAddr, error) {
 	if tbf.f == nil {
 		f, err := os.CreateTemp(tmpBlocksDir, "")
 		if err != nil {
-			return addr, err
+			tbf.err = fmt.Errorf("cannot create temporary blocks file at %q: %w", tmpBlocksDir, err)
+			return addr, tbf.err
 		}
 		tbf.f = f
 		tmpBlocksFilesCreated.Inc()
@@ -130,7 +139,9 @@ func (tbf *tmpBlocksFile) WriteBlockRefData(b []byte) (tmpBlockAddr, error) {
 	_, err := tbf.f.Write(tbf.buf)
 	tbf.buf = append(tbf.buf[:0], b...)
 	if err != nil {
-		return addr, fmt.Errorf("cannot write block to %q: %w", tbf.f.Name(), err)
+		// The blocks buffered at tbf.buf could be partially lost, mark the tbf as unusable.
+		tbf.err = fmt.Errorf("cannot write block to %q: %w", tbf.f.Name(), err)
+		return addr, tbf.err
 	}
 	return addr, nil
 }
@@ -141,12 +152,16 @@ func (tbf *tmpBlocksFile) Len() uint64 {
 }
 
 func (tbf *tmpBlocksFile) Finalize() error {
+	if tbf.err != nil {
+		return tbf.err
+	}
 	if tbf.f == nil {
 		return nil
 	}
 	fname := tbf.f.Name()
 	if _, err := tbf.f.Write(tbf.buf); err != nil {
-		return fmt.Errorf("cannot write the remaining %d bytes to %q: %w", len(tbf.buf), fname, err)
+		tbf.err = fmt.Errorf("cannot write the remaining %d bytes to %q: %w", len(tbf.buf), fname, err)
+		return tbf.err
 	}
 	tbf.buf = tbf.buf[:0]
 	r := fs.NewReaderAt(tbf.f)
@@ -166,6 +181,10 @@ func (tbf *tmpBlocksFile) Finalize() error {
 }
 
 func (tbf *tmpBlocksFile) MustReadBlockRefAt(partRef storage.PartRef, addr tmpBlockAddr) storage.BlockRef {
+	if tbf.err != nil {
+		// This should never happen, since Finalize() already returns the error for such a tbf.
+		logger.Panicf("BUG: cannot read block at %s from the temporary blocks file with the failed write: %s", addr, tbf.err)
+	}
 	var buf []byte
 	if tbf.r == nil {
 		buf = tbf.buf[addr.offset : addr.offset+uint64(addr.size)]

app/vmselect/prometheus/prometheus.go

@@ -516,7 +516,7 @@ func DeleteHandler(startTime time.Time, r *http.Request) error {
 	cp.deadline = searchutil.GetDeadlineForDelete(r, startTime)
 
 	if !cp.IsDefaultTimeRange() {
-		return fmt.Errorf("start=%d and end=%d args aren't supported. Remove these args from the query in order to delete all the matching metrics", cp.start, cp.end)
+		return fmt.Errorf("delete API does not support specific time ranges using start and end args, the series can only be deleted completely")
 	}
 	sq := storage.NewSearchQuery(cp.start, cp.end, cp.filterss, *maxDeleteSeries)
 	deletedCount, err := netstorage.DeleteSeries(nil, sq, cp.deadline)
@@ -540,11 +540,11 @@ func LabelValuesHandler(qt *querytracer.Tracer, startTime time.Time, labelName s
 
 	cp, err := getCommonParamsForLabelsAPI(r, startTime, false)
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	limit, err := httputil.GetInt(r, "limit")
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	sq := storage.NewSearchQuery(cp.start, cp.end, cp.filterss, *maxLabelsAPISeries)
 
@@ -584,7 +584,7 @@ func TSDBStatusHandler(qt *querytracer.Tracer, startTime time.Time, w http.Respo
 
 	cp, err := getCommonParams(r, startTime, false)
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	cp.deadline = searchutil.GetDeadlineForStatusRequest(r, startTime)
 
@@ -596,7 +596,7 @@ func TSDBStatusHandler(qt *querytracer.Tracer, startTime time.Time, w http.Respo
 		} else {
 			t, err := time.Parse("2006-01-02", dateStr)
 			if err != nil {
-				return fmt.Errorf("cannot parse `date` arg %q: %w", dateStr, err)
+				return httpserver.InvalidParamError(fmt.Errorf("cannot parse `date` arg %q: %w", dateStr, err))
 			}
 			date = uint64(t.Unix()) / secsPerDay
 		}
@@ -607,7 +607,7 @@ func TSDBStatusHandler(qt *querytracer.Tracer, startTime time.Time, w http.Respo
 	if len(topNStr) > 0 {
 		n, err := strconv.Atoi(topNStr)
 		if err != nil {
-			return fmt.Errorf("cannot parse `topN` arg %q: %w", topNStr, err)
+			return httpserver.InvalidParamError(fmt.Errorf("cannot parse `topN` arg %q: %w", topNStr, err))
 		}
 		if n <= 0 {
 			n = 1
@@ -645,11 +645,11 @@ func LabelsHandler(qt *querytracer.Tracer, startTime time.Time, w http.ResponseW
 
 	cp, err := getCommonParamsForLabelsAPI(r, startTime, false)
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	limit, err := httputil.GetInt(r, "limit")
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	sq := storage.NewSearchQuery(cp.start, cp.end, cp.filterss, *maxLabelsAPISeries)
 	labels, err := netstorage.LabelNames(qt, sq, limit, cp.deadline)
@@ -671,10 +671,9 @@ func LabelsHandler(qt *querytracer.Tracer, startTime time.Time, w http.ResponseW
 //
 // See https://prometheus.io/docs/prometheus/latest/querying/api/#querying-metric-metadata
 func MetadataHandler(qt *querytracer.Tracer, startTime time.Time, w http.ResponseWriter, r *http.Request) error {
-
 	limit, err := httputil.GetInt(r, "limit")
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	if limit < 0 {
 		limit = 0
@@ -734,11 +733,11 @@ func SeriesHandler(qt *querytracer.Tracer, startTime time.Time, w http.ResponseW
 	// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/91
 	cp, err := getCommonParamsForLabelsAPI(r, startTime, true)
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	limit, err := httputil.GetInt(r, "limit")
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 
 	sq := storage.NewSearchQuery(cp.start, cp.end, cp.filterss, *maxSeriesLimit)
@@ -772,19 +771,19 @@ func QueryHandler(qt *querytracer.Tracer, startTime time.Time, w http.ResponseWr
 	mayCache := !httputil.GetBool(r, "nocache")
 	query := r.FormValue("query")
 	if len(query) == 0 {
-		return fmt.Errorf("missing `query` arg")
+		return httpserver.InvalidParamError(fmt.Errorf("missing `query` arg"))
 	}
 	start, err := httputil.GetTime(r, "time", ct)
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	lookbackDelta, err := getMaxLookback(r)
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	step, err := httputil.GetDuration(r, "step", lookbackDelta)
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	if step <= 0 {
 		step = defaultStep
@@ -792,16 +791,16 @@ func QueryHandler(qt *querytracer.Tracer, startTime time.Time, w http.ResponseWr
 
 	maxLen := searchutil.GetMaxQueryLen()
 	if len(query) > maxLen {
-		return fmt.Errorf("too long query; got %d bytes; mustn't exceed `-search.maxQueryLen=%d` bytes", len(query), maxLen)
+		return httpserver.InvalidParamError(fmt.Errorf("too long query; got %d bytes; mustn't exceed `-search.maxQueryLen=%d` bytes", len(query), maxLen))
 	}
 	etfs, err := searchutil.GetExtraTagFilters(r)
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	if childQuery, windowExpr, offsetExpr := promql.IsMetricSelectorWithRollup(query); childQuery != "" {
 		window, err := windowExpr.NonNegativeDuration(step)
 		if err != nil {
-			return fmt.Errorf("cannot parse lookbehind window in square brackets at %s: %w", query, err)
+			return httpserver.InvalidParamError(fmt.Errorf("cannot parse lookbehind window in square brackets at %s: %w", query, err))
 		}
 		offset := offsetExpr.Duration(step)
 		start -= offset
@@ -815,7 +814,7 @@ func QueryHandler(qt *querytracer.Tracer, startTime time.Time, w http.ResponseWr
 
 		tagFilterss, err := getTagFilterssFromMatches([]string{childQuery})
 		if err != nil {
-			return err
+			return httpserver.InvalidParamError(err)
 		}
 		filterss := searchutil.JoinTagFilterss(tagFilterss, etfs)
 
@@ -831,22 +830,25 @@ func QueryHandler(qt *querytracer.Tracer, startTime time.Time, w http.ResponseWr
 		return nil
 	}
 	if childQuery, windowExpr, stepExpr, offsetExpr := promql.IsRollup(query); childQuery != "" {
+		if len(childQuery) > maxLen {
+			return httpserver.InvalidParamError(fmt.Errorf("too long query; got %d bytes; mustn't exceed `-search.maxQueryLen=%d` bytes", len(childQuery), maxLen))
+		}
 		newStep, err := stepExpr.NonNegativeDuration(step)
 		if err != nil {
-			return fmt.Errorf("cannot parse step in square brackets at %s: %w", query, err)
+			return httpserver.InvalidParamError(fmt.Errorf("cannot parse step in square brackets at %s: %w", query, err))
 		}
 		if newStep > 0 {
 			step = newStep
 		}
 		window, err := windowExpr.NonNegativeDuration(step)
 		if err != nil {
-			return fmt.Errorf("cannot parse lookbehind window in square brackets at %s: %w", query, err)
+			return httpserver.InvalidParamError(fmt.Errorf("cannot parse lookbehind window in square brackets at %s: %w", query, err))
 		}
 		offset := offsetExpr.Duration(step)
 		start -= offset
 		end := start
 		start = end - window
-		if err := queryRangeHandler(qt, startTime, w, childQuery, start, end, step, r, ct, etfs); err != nil {
+		if err := queryRangeHandler(qt, startTime, w, childQuery, start, end, step, lookbackDelta, r, ct, etfs); err != nil {
 			return fmt.Errorf("error when executing query=%q on the time range (start=%d, end=%d, step=%d): %w", childQuery, start, end, step, err)
 		}
 		return nil
@@ -854,7 +856,7 @@ func QueryHandler(qt *querytracer.Tracer, startTime time.Time, w http.ResponseWr
 
 	queryOffset, err := getLatencyOffsetMilliseconds(r)
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	if !httputil.GetBool(r, "nocache") && ct-start < queryOffset && start-ct < queryOffset {
 		// Adjust start time only if `nocache` arg isn't set.
@@ -928,45 +930,43 @@ func QueryRangeHandler(qt *querytracer.Tracer, startTime time.Time, w http.Respo
 	ct := startTime.UnixNano() / 1e6
 	query := r.FormValue("query")
 	if len(query) == 0 {
-		return fmt.Errorf("missing `query` arg")
+		return httpserver.InvalidParamError(fmt.Errorf("missing `query` arg"))
+	}
+	maxLen := searchutil.GetMaxQueryLen()
+	if len(query) > maxLen {
+		return httpserver.InvalidParamError(fmt.Errorf("too long query; got %d bytes; mustn't exceed `-search.maxQueryLen=%d` bytes", len(query), maxLen))
 	}
 	start, err := httputil.GetTime(r, "start", ct-defaultStep)
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	end, err := httputil.GetTime(r, "end", ct)
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	step, err := httputil.GetDuration(r, "step", defaultStep)
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
 	etfs, err := searchutil.GetExtraTagFilters(r)
 	if err != nil {
-		return err
+		return httpserver.InvalidParamError(err)
 	}
-	if err := queryRangeHandler(qt, startTime, w, query, start, end, step, r, ct, etfs); err != nil {
+	lookbackDelta, err := getMaxLookback(r)
+	if err != nil {
+		return httpserver.InvalidParamError(err)
+	}
+	if err := queryRangeHandler(qt, startTime, w, query, start, end, step, lookbackDelta, r, ct, etfs); err != nil {
 		return fmt.Errorf("error when executing query=%q on the time range (start=%d, end=%d, step=%d): %w", query, start, end, step, err)
 	}
 	return nil
 }
 
 func queryRangeHandler(qt *querytracer.Tracer, startTime time.Time, w http.ResponseWriter, query string,
-	start, end, step int64, r *http.Request, ct int64, etfs [][]storage.TagFilter) error {
+	start, end, step, lookbackDelta int64, r *http.Request, ct int64, etfs [][]storage.TagFilter) error {
 	deadline := searchutil.GetDeadlineForQuery(r, startTime)
 	mayCache := !httputil.GetBool(r, "nocache")
 	optimizeRepeatedBinaryOpSubexprs := httputil.GetBool(r, "optimize_repeated_binary_op_subexprs")
-	lookbackDelta, err := getMaxLookback(r)
-	if err != nil {
-		return err
-	}
-
-	// Validate input args.
-	maxLen := searchutil.GetMaxQueryLen()
-	if len(query) > maxLen {
-		return fmt.Errorf("too long query; got %d bytes; mustn't exceed `-search.maxQueryLen=%d` bytes", len(query), maxLen)
-	}
 	if start > end {
 		end = start + defaultStep
 	}
@@ -1005,7 +1005,7 @@ func queryRangeHandler(qt *querytracer.Tracer, startTime time.Time, w http.Respo
 	if step < maxStepForPointsAdjustment.Milliseconds() {
 		queryOffset, err := getLatencyOffsetMilliseconds(r)
 		if err != nil {
-			return err
+			return httpserver.InvalidParamError(err)
 		}
 		if ct-queryOffset < end {
 			result = adjustLastPoints(result, ct-queryOffset, ct+step)
@@ -1156,13 +1156,13 @@ func QueryStatsHandler(w http.ResponseWriter, r *http.Request) error {
 	if len(topNStr) > 0 {
 		n, err := strconv.Atoi(topNStr)
 		if err != nil {
-			return fmt.Errorf("cannot parse `topN` arg %q: %w", topNStr, err)
+			return httpserver.InvalidParamError(fmt.Errorf("cannot parse `topN` arg %q: %w", topNStr, err))
 		}
 		topN = n
 	}
 	maxLifetimeMsecs, err := httputil.GetDuration(r, "maxLifetime", 10*60*1000)
 	if err != nil {
-		return fmt.Errorf("cannot parse `maxLifetime` arg: %w", err)
+		return httpserver.InvalidParamError(fmt.Errorf("cannot parse `maxLifetime` arg: %w", err))
 	}
 	maxLifetime := time.Duration(maxLifetimeMsecs) * time.Millisecond
 	w.Header().Set("Content-Type", "application/json")

app/vmselect/promql/exec.go

@@ -11,6 +11,7 @@ import (
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/netstorage"
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/querystats"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/decimal"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/querytracer"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
@@ -46,15 +47,15 @@ func Exec(qt *querytracer.Tracer, ec *EvalConfig, q string, isFirstPointOnly boo
 
 	e, err := parsePromQLWithCache(q)
 	if err != nil {
-		return nil, err
+		return nil, httpserver.InvalidParamError(err)
 	}
 
 	if *disableImplicitConversion || *logImplicitConversion {
 		isInvalid := metricsql.IsLikelyInvalid(e)
 		if isInvalid && *disableImplicitConversion {
 			// we don't add query=%q to err message as it will be added by the caller
-			return nil, fmt.Errorf("query requires implicit conversion and is rejected according to -search.disableImplicitConversion command-line flag. " +
-				"See https://docs.victoriametrics.com/victoriametrics/metricsql/#implicit-query-conversions for details")
+			return nil, httpserver.InvalidParamError(fmt.Errorf("query requires implicit conversion and is rejected according to -search.disableImplicitConversion command-line flag. " +
+				"See https://docs.victoriametrics.com/victoriametrics/metricsql/#implicit-query-conversions for details"))
 		}
 		if isInvalid && *logImplicitConversion {
 			logger.Warnf("query=%q requires implicit conversion, see https://docs.victoriametrics.com/victoriametrics/metricsql/#implicit-query-conversions for details", e.AppendString(nil))

app/vmselect/promql/transform.go

@@ -2,6 +2,7 @@ package promql
 
 import (
 	"bytes"
+	"errors"
 	"fmt"
 	"math"
 	"math/rand"
@@ -2566,6 +2567,11 @@ func isDecimalChar(ch byte) bool {
 func mustParseNum(s string) float64 {
 	f, err := strconv.ParseFloat(s, 64)
 	if err != nil {
+		if errors.Is(err, strconv.ErrRange) {
+			// The number is too large to fit into float64; ParseFloat returns ±Inf in this case.
+			// Use ±Inf for sorting purposes — it is semantically correct.
+			return f
+		}
 		logger.Panicf("BUG: unexpected error when parsing the number %q: %s", s, err)
 	}
 	return f

app/vmselect/promql/transform_test.go

@@ -385,4 +385,12 @@ func TestNumericLess(t *testing.T) {
 	f("12.9", "12.56", false)
 	f("12.56", "12.9", true)
 	f("12.9", "12.9", false)
+
+	// 309-digit numbers - must not panic (regression test for GHSA-9g98-8jgr-x2vv)
+	big := strings.Repeat("9", 309)
+	f(big, "1", false)
+	f("1", big, true)
+	f(big, big, false)
+	f("-"+big, big, true)
+	f(big, "-"+big, false)
 }

app/vmselect/vmui/assets/favicon.svg

@@ -0,0 +1,5 @@
+<svg width="48" height="48" fill="#020202" xmlns="http://www.w3.org/2000/svg">
+    <path d="M24.5475 0C10.3246.0265251 1.11379 3.06365 4.40623 6.10077c0 0 12.32997 11.23333 16.58217 14.84083.8131.6896 2.1728 1.1936 3.5191 1.2201h.1199c1.3463-.0265 2.706-.5305 3.5191-1.2201 4.2522-3.5942 16.5422-14.84083 16.5422-14.84083C48.0478 3.06365 38.8636.0265251 24.6674 0"/>
+    <path d="M28.1579 27.0159c-.8131.6896-2.1728 1.1936-3.5191 1.2201h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2201-2.9725-2.5067-13.35639-11.87-17.26201-15.3979v5.4112c0 .5968.22661 1.3793.6265 1.7506C7.00358 21.1936 17.2675 30.5437 20.9731 33.6737c.8132.6896 2.1728 1.1936 3.5191 1.2201h.12c1.3463-.0265 2.7059-.5305 3.519-1.2201 3.679-3.13 13.9429-12.4536 16.6089-14.8939.4132-.3713.6265-1.1538.6265-1.7506V11.618c-3.9323 3.5411-14.3162 12.931-17.2354 15.3979h.0267Z"/>
+    <path d="M28.1579 39.748c-.8131.6897-2.1728 1.1937-3.5191 1.2202h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2202-2.9725-2.4933-13.35639-11.8567-17.26201-15.3978v5.4111c0 .5969.22661 1.3793.6265 1.7507C7.00358 33.9258 17.2675 43.2759 20.9731 46.4058c.8132.6897 2.1728 1.1937 3.5191 1.2202h.12c1.3463-.0265 2.7059-.5305 3.519-1.2202 3.679-3.1299 13.9429-12.4535 16.6089-14.8938.4132-.3714.6265-1.1538.6265-1.7507v-5.4111c-3.9323 3.5411-14.3162 12.931-17.2354 15.3978h.0267Z"/>
+</svg>

app/vmselect/vmui/favicon.svg

@@ -1 +0,0 @@
-<svg width="48" height="48" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M24.5475 0C10.3246.0265251 1.11379 3.06365 4.40623 6.10077c0 0 12.32997 11.23333 16.58217 14.84083.8131.6896 2.1728 1.1936 3.5191 1.2201h.1199c1.3463-.0265 2.706-.5305 3.5191-1.2201 4.2522-3.5942 16.5422-14.84083 16.5422-14.84083C48.0478 3.06365 38.8636.0265251 24.6674 0" fill="#020202"/><path d="M28.1579 27.0159c-.8131.6896-2.1728 1.1936-3.5191 1.2201h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2201-2.9725-2.5067-13.35639-11.87-17.26201-15.3979v5.4112c0 .5968.22661 1.3793.6265 1.7506C7.00358 21.1936 17.2675 30.5437 20.9731 33.6737c.8132.6896 2.1728 1.1936 3.5191 1.2201h.12c1.3463-.0265 2.7059-.5305 3.519-1.2201 3.679-3.13 13.9429-12.4536 16.6089-14.8939.4132-.3713.6265-1.1538.6265-1.7506V11.618c-3.9323 3.5411-14.3162 12.931-17.2354 15.3979h.0267Z" fill="#020202"/><path d="M28.1579 39.748c-.8131.6897-2.1728 1.1937-3.5191 1.2202h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2202-2.9725-2.4933-13.35639-11.8567-17.26201-15.3978v5.4111c0 .5969.22661 1.3793.6265 1.7507C7.00358 33.9258 17.2675 43.2759 20.9731 46.4058c.8132.6897 2.1728 1.1937 3.5191 1.2202h.12c1.3463-.0265 2.7059-.5305 3.519-1.2202 3.679-3.1299 13.9429-12.4535 16.6089-14.8938.4132-.3714.6265-1.1538.6265-1.7507v-5.4111c-3.9323 3.5411-14.3162 12.931-17.2354 15.3978h.0267Z" fill="#020202"/></svg>

app/vmselect/vmui/index.html

@@ -2,9 +2,9 @@
 <html lang="en">
 <head>
   <meta charset="utf-8"/>
-  <link rel="icon" href="./favicon.svg"/>
-  <link rel="apple-touch-icon" href="./favicon.svg"/>
-  <link rel="mask-icon" href="./favicon.svg" color="#000000">
+  <link id="favicon" rel="icon" href="./assets/favicon.svg" />
+  <link rel="apple-touch-icon" href="./assets/favicon.svg" />
+  <link id="mask-icon" rel="mask-icon" href="./assets/favicon.svg" color="#000000">
 
   <meta name="robots" content="noindex">
   <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=5"/>
@@ -37,11 +37,11 @@
   <meta property="og:title" content="UI for VictoriaMetrics">
   <meta property="og:url" content="https://victoriametrics.com/">
   <meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data">
-  <script type="module" crossorigin src="./assets/index-B1dXK3k7.js"></script>
+  <script type="module" crossorigin src="./assets/index-BiDX4bB6.js"></script>
   <link rel="modulepreload" crossorigin href="./assets/rolldown-runtime-CNC7AqOf.js">
   <link rel="modulepreload" crossorigin href="./assets/vendor-DwJYpOdw.js">
   <link rel="stylesheet" crossorigin href="./assets/vendor-CnsZ1jie.css">
-  <link rel="stylesheet" crossorigin href="./assets/index-BJqoElx2.css">
+  <link rel="stylesheet" crossorigin href="./assets/index-CymA7XYg.css">
 </head>
 <body>
 <noscript>You need to enable JavaScript to run this app.</noscript>

app/vmselect/vmui/manifest.json

@@ -3,7 +3,7 @@
   "name": "vmui",
   "icons": [
     {
-      "src": "favicon.svg",
+      "src": "./assets/favicon.svg",
       "sizes": "any",
       "type": "image/svg+xml"
     }

app/vmui/Dockerfile-web

@@ -1,4 +1,4 @@
-FROM golang:1.26.5 AS build-web-stage
+FROM golang:1.26.6 AS build-web-stage
 COPY build /build
 
 WORKDIR /build

app/vmui/packages/vmui/assets/favicon.svg

@@ -0,0 +1,5 @@
+<svg width="48" height="48" fill="#020202" xmlns="http://www.w3.org/2000/svg">
+    <path d="M24.5475 0C10.3246.0265251 1.11379 3.06365 4.40623 6.10077c0 0 12.32997 11.23333 16.58217 14.84083.8131.6896 2.1728 1.1936 3.5191 1.2201h.1199c1.3463-.0265 2.706-.5305 3.5191-1.2201 4.2522-3.5942 16.5422-14.84083 16.5422-14.84083C48.0478 3.06365 38.8636.0265251 24.6674 0"/>
+    <path d="M28.1579 27.0159c-.8131.6896-2.1728 1.1936-3.5191 1.2201h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2201-2.9725-2.5067-13.35639-11.87-17.26201-15.3979v5.4112c0 .5968.22661 1.3793.6265 1.7506C7.00358 21.1936 17.2675 30.5437 20.9731 33.6737c.8132.6896 2.1728 1.1936 3.5191 1.2201h.12c1.3463-.0265 2.7059-.5305 3.519-1.2201 3.679-3.13 13.9429-12.4536 16.6089-14.8939.4132-.3713.6265-1.1538.6265-1.7506V11.618c-3.9323 3.5411-14.3162 12.931-17.2354 15.3979h.0267Z"/>
+    <path d="M28.1579 39.748c-.8131.6897-2.1728 1.1937-3.5191 1.2202h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2202-2.9725-2.4933-13.35639-11.8567-17.26201-15.3978v5.4111c0 .5969.22661 1.3793.6265 1.7507C7.00358 33.9258 17.2675 43.2759 20.9731 46.4058c.8132.6897 2.1728 1.1937 3.5191 1.2202h.12c1.3463-.0265 2.7059-.5305 3.519-1.2202 3.679-3.1299 13.9429-12.4535 16.6089-14.8938.4132-.3714.6265-1.1538.6265-1.7507v-5.4111c-3.9323 3.5411-14.3162 12.931-17.2354 15.3978h.0267Z"/>
+</svg>

app/vmui/packages/vmui/index.html

@@ -2,9 +2,9 @@
 <html lang="en">
 <head>
   <meta charset="utf-8"/>
-  <link rel="icon" href="/favicon.svg"/>
-  <link rel="apple-touch-icon" href="/favicon.svg"/>
-  <link rel="mask-icon" href="/favicon.svg" color="#000000">
+  <link id="favicon" rel="icon" href="/assets/favicon.svg" />
+  <link rel="apple-touch-icon" href="/assets/favicon.svg" />
+  <link id="mask-icon" rel="mask-icon" href="/assets/favicon.svg?no-inline" color="#000000">
 
   <meta name="robots" content="noindex">
   <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=5"/>

app/vmui/packages/vmui/public/favicon.svg

@@ -1 +0,0 @@
-<svg width="48" height="48" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M24.5475 0C10.3246.0265251 1.11379 3.06365 4.40623 6.10077c0 0 12.32997 11.23333 16.58217 14.84083.8131.6896 2.1728 1.1936 3.5191 1.2201h.1199c1.3463-.0265 2.706-.5305 3.5191-1.2201 4.2522-3.5942 16.5422-14.84083 16.5422-14.84083C48.0478 3.06365 38.8636.0265251 24.6674 0" fill="#020202"/><path d="M28.1579 27.0159c-.8131.6896-2.1728 1.1936-3.5191 1.2201h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2201-2.9725-2.5067-13.35639-11.87-17.26201-15.3979v5.4112c0 .5968.22661 1.3793.6265 1.7506C7.00358 21.1936 17.2675 30.5437 20.9731 33.6737c.8132.6896 2.1728 1.1936 3.5191 1.2201h.12c1.3463-.0265 2.7059-.5305 3.519-1.2201 3.679-3.13 13.9429-12.4536 16.6089-14.8939.4132-.3713.6265-1.1538.6265-1.7506V11.618c-3.9323 3.5411-14.3162 12.931-17.2354 15.3979h.0267Z" fill="#020202"/><path d="M28.1579 39.748c-.8131.6897-2.1728 1.1937-3.5191 1.2202h-.12c-1.3463-.0265-2.7059-.5305-3.519-1.2202-2.9725-2.4933-13.35639-11.8567-17.26201-15.3978v5.4111c0 .5969.22661 1.3793.6265 1.7507C7.00358 33.9258 17.2675 43.2759 20.9731 46.4058c.8132.6897 2.1728 1.1937 3.5191 1.2202h.12c1.3463-.0265 2.7059-.5305 3.519-1.2202 3.679-3.1299 13.9429-12.4535 16.6089-14.8938.4132-.3714.6265-1.1538.6265-1.7507v-5.4111c-3.9323 3.5411-14.3162 12.931-17.2354 15.3978h.0267Z" fill="#020202"/></svg>

app/vmui/packages/vmui/public/manifest.json

@@ -3,7 +3,7 @@
   "name": "vmui",
   "icons": [
     {
-      "src": "favicon.svg",
+      "src": "./assets/favicon.svg",
       "sizes": "any",
       "type": "image/svg+xml"
     }

app/vmui/packages/vmui/src/components/Configurators/GlobalSettings/BrowserTabController/BrowserTabController.tsx

@@ -0,0 +1,61 @@
+import { FC, useMemo } from "preact/compat";
+import "./style.scss";
+import { createFaviconUrl } from "../../../../utils/favicon";
+import classNames from "classnames";
+import { useBrowserTabSync } from "./hooks/useBrowserTabSync";
+import { CloseIcon } from "../../../Main/Icons";
+import { faviconColors } from "../../../../constants/faviconColors";
+
+const BrowserTabController: FC = () => {
+  const { faviconColor, changeFaviconColor } = useBrowserTabSync();
+
+  const faviconUrl = useMemo(() => {
+    return createFaviconUrl(faviconColor);
+  }, [faviconColor]);
+
+  const createHandlerClick = (color?: string) => () => {
+    changeFaviconColor(color);
+  };
+
+  return (
+    <div className="vm-browser-tab-controller">
+      <p className="vm-server-configurator__title">Favicon color</p>
+
+      <div className="vm-browser-tab-controller-palette">
+        <div className="vm-browser-tab-controller-palette-list">
+          <button
+            className="vm-browser-tab-controller-palette-list__item vm-browser-tab-controller-palette-list__item_reset"
+            type="button"
+            onClick={createHandlerClick()}
+            aria-label="Reset favicon color"
+          >
+            <CloseIcon />
+          </button>
+
+          {faviconColors.map(color => (
+            <button
+              className={classNames({
+                "vm-browser-tab-controller-palette-list__item": true,
+                "vm-browser-tab-controller-palette-list__item_selected": faviconColor === color
+              })}
+              key={color}
+              type="button"
+              style={{ color }}
+              onClick={createHandlerClick(color)}
+              aria-label={`Set favicon color to ${color}`}
+              aria-pressed={faviconColor === color}
+            />
+          ))}
+        </div>
+
+        <img
+          className="vm-browser-tab-controller-palette__preview"
+          src={faviconUrl}
+          alt="Favicon preview"
+        />
+      </div>
+    </div>
+  );
+};
+
+export default BrowserTabController;

app/vmui/packages/vmui/src/components/Configurators/GlobalSettings/BrowserTabController/hooks/useBrowserTabSync.ts

@@ -0,0 +1,41 @@
+import useEventListener from "../../../../../hooks/useEventListener";
+import { useEffect, useState } from "preact/compat";
+import { getFromStorage, removeFromStorage, saveToStorage } from "../../../../../utils/storage";
+import { getFaviconStorageKey, updateFaviconColor } from "../../../../../utils/favicon";
+
+const storageKey = `FAVICON_COLOR:${getFaviconStorageKey()}` as const;
+
+const getColorFromStorage = () => {
+  return getFromStorage(storageKey) as string | undefined;
+};
+
+export const useBrowserTabSync = () => {
+  const [faviconColor, setFaviconColor] = useState(getColorFromStorage);
+
+  const handleUpdateColor = () => {
+    setFaviconColor(getColorFromStorage());
+  };
+
+  const changeFaviconColor = (color?: string) => {
+    if (color) {
+      saveToStorage(storageKey, color);
+    } else {
+      removeFromStorage([storageKey]);
+    }
+  };
+
+  useEffect(() => {
+    handleUpdateColor();
+  }, []);
+
+  useEffect(() => {
+    updateFaviconColor(faviconColor);
+  }, [faviconColor]);
+
+  useEventListener("storage", handleUpdateColor);
+
+  return {
+    faviconColor,
+    changeFaviconColor,
+  };
+};

app/vmui/packages/vmui/src/components/Configurators/GlobalSettings/BrowserTabController/style.scss

@@ -0,0 +1,71 @@
+@use "src/styles/variables" as *;
+
+$color-item-size: 28px;
+$outline-width: 2px;
+$outline-offset: 2px;
+$outline-space: $outline-width + $outline-offset;
+
+.vm-browser-tab-controller {
+  .vm-server-configurator__title {
+    padding-bottom: 2px;
+  }
+
+  &-palette {
+    display: flex;
+    align-items: flex-start;
+    justify-content: space-between;
+    gap: calc($padding-large * 2);
+
+    &-list {
+      display: flex;
+      align-items: center;
+      justify-content: flex-start;
+      flex-wrap: wrap;
+      gap: $padding-small;
+
+      &__item {
+        width: $color-item-size;
+        height: $color-item-size;
+        aspect-ratio: 1;
+        border-radius: 50%;
+        background-color: currentColor;
+        cursor: pointer;
+        transition-property: transform;
+        transition-duration: 0.15s;
+        transition-timing-function: linear;
+
+        &:hover {
+          transform: scale(1.1);
+        }
+
+        &:focus-visible {
+          transform: scale(1.1);
+        }
+
+        &_reset {
+          display: flex;
+          align-items: center;
+          justify-content: center;
+          background: transparent;
+          border: $border-divider;
+          color: $color-text-disabled;
+          padding: calc($padding-small / 2);
+        }
+
+        &_selected {
+          width: $color-item-size - 2 * $outline-space;
+          height: $color-item-size - 2 * $outline-space;
+          margin: $outline-space;
+          outline: $outline-width solid currentColor;
+          outline-offset: $outline-offset;
+          pointer-events: none;
+        }
+      }
+    }
+
+    &__preview {
+      width: $color-item-size;
+      height: auto;
+    }
+  }
+}

app/vmui/packages/vmui/src/components/Configurators/GlobalSettings/GlobalSettings.tsx

@@ -1,4 +1,4 @@
-import { FC, useRef } from "preact/compat";
+import { forwardRef, useImperativeHandle, useRef } from "preact/compat";
 import ServerConfigurator from "./ServerConfigurator/ServerConfigurator";
 import { ArrowDownIcon, SettingsIcon } from "../../Main/Icons";
 import Button from "../../Main/Button/Button";
@@ -8,10 +8,12 @@ import Tooltip from "../../Main/Tooltip/Tooltip";
 import LimitsConfigurator from "./LimitsConfigurator/LimitsConfigurator";
 import { getAppModeEnable } from "../../../utils/app-mode";
 import classNames from "classnames";
-import Timezones from "./Timezones/Timezones";
+import TimezonesPicker from "./Timezones/TimezonesPicker";
 import ThemeControl from "../ThemeControl/ThemeControl";
 import useDeviceDetect from "../../../hooks/useDeviceDetect";
 import useBoolean from "../../../hooks/useBoolean";
+import BrowserTabController from "./BrowserTabController/BrowserTabController";
+import LegendCollapseController from "./LegendCollapseController/LegendCollapseController";
 
 const title = "Settings";
 
@@ -19,14 +21,17 @@ export interface ChildComponentHandle {
   handleApply: () => void;
 }
 
-const GlobalSettings: FC = () => {
+export interface GlobalSettingsHandle {
+  open: () => void;
+}
+
+const GlobalSettings = forwardRef<GlobalSettingsHandle>((_, ref) => {
   const { isMobile } = useDeviceDetect();
 
   const appModeEnable = getAppModeEnable();
 
   const serverSettingRef = useRef<ChildComponentHandle>(null);
   const limitsSettingRef = useRef<ChildComponentHandle>(null);
-  const timezoneSettingRef = useRef<ChildComponentHandle>(null);
 
   const {
     value: open,
@@ -37,7 +42,6 @@ const GlobalSettings: FC = () => {
   const handleApply = () => {
     serverSettingRef.current && serverSettingRef.current.handleApply();
     limitsSettingRef.current && limitsSettingRef.current.handleApply();
-    timezoneSettingRef.current && timezoneSettingRef.current.handleApply();
     handleClose();
   };
 
@@ -49,6 +53,10 @@ const GlobalSettings: FC = () => {
         onClose={handleClose}
       />
     },
+    {
+      show: true,
+      component: <TimezonesPicker/>
+    },
     {
       show: true,
       component: <LimitsConfigurator
@@ -58,14 +66,22 @@ const GlobalSettings: FC = () => {
     },
     {
       show: true,
-      component: <Timezones ref={timezoneSettingRef}/>
+      component: <LegendCollapseController/>
     },
     {
       show: !appModeEnable,
       component: <ThemeControl/>
-    }
+    },
+    {
+      show: true,
+      component: <BrowserTabController/>
+    },
   ].filter(control => control.show);
 
+  useImperativeHandle(ref, () => ({
+    open: handleOpen,
+  }));
+
   return <>
     {isMobile ? (
       <div
@@ -131,6 +147,6 @@ const GlobalSettings: FC = () => {
       </Modal>
     )}
   </>;
-};
+});
 
 export default GlobalSettings;

app/vmui/packages/vmui/src/components/Configurators/GlobalSettings/LegendCollapseController/LegendCollapseController.tsx

@@ -0,0 +1,30 @@
+import { FC, useEffect, useState } from "preact/compat";
+import { getFromStorage, saveToStorage } from "../../../../utils/storage";
+import Switch from "../../../Main/Switch/Switch";
+import { LEGEND_COLLAPSE_SERIES_LIMIT } from "../../../../constants/graph";
+import "./style.scss";
+
+const LegendCollapseController: FC = () => {
+  const storageCollapse = getFromStorage("LEGEND_AUTO_COLLAPSE");
+  const [legendCollapse, setLegendCollapse] = useState(storageCollapse ? storageCollapse === "true" : true);
+
+  useEffect(() => {
+    saveToStorage("LEGEND_AUTO_COLLAPSE", `${legendCollapse}`);
+  }, [legendCollapse]);
+
+  return (
+    <div className="vm-legend-collapse-controller">
+      <Switch
+        fullWidth
+        color="neutral"
+        value={legendCollapse}
+        onChange={setLegendCollapse}
+        label={<span className="vm-server-configurator__title">Auto-collapse legend</span>}
+      />
+      <span className="vm-legend-collapse-controller__description">
+        Collapses the legend when series count exceeds {LEGEND_COLLAPSE_SERIES_LIMIT} to reduce UI load.
+      </span>
+    </div>);
+};
+
+export default LegendCollapseController;

app/vmui/packages/vmui/src/components/Configurators/GlobalSettings/LegendCollapseController/style.scss

@@ -0,0 +1,19 @@
+@use "src/styles/variables" as *;
+
+.vm-legend-collapse-controller {
+  background-color: $color-hover-black;
+  border-radius: $border-radius-medium;
+  padding: $padding-large;
+  border: $border-divider;
+
+  .vm-graph-settings-row__label {
+    margin: 0;
+  }
+
+  &__description {
+    padding-top: $padding-global;
+    font-size: $font-size-small;
+    line-height: 1.3;
+    text-wrap: pretty;
+  }
+}

app/vmui/packages/vmui/src/components/Configurators/GlobalSettings/LimitsConfigurator/LimitsConfigurator.tsx

@@ -1,23 +1,21 @@
-import { forwardRef, useCallback, useEffect, useImperativeHandle, useState } from "preact/compat";
+import { forwardRef, useCallback, useImperativeHandle, useState } from "preact/compat";
 import { DisplayType, ErrorTypes } from "../../../../types";
 import TextField from "../../../Main/TextField/TextField";
 import Tooltip from "../../../Main/Tooltip/Tooltip";
 import { InfoIcon, RestartIcon } from "../../../Main/Icons";
 import Button from "../../../Main/Button/Button";
-import { DEFAULT_MAX_SERIES, LEGEND_COLLAPSE_SERIES_LIMIT } from "../../../../constants/graph";
+import { DEFAULT_MAX_SERIES } from "../../../../constants/graph";
 import "./style.scss";
 import classNames from "classnames";
 import useDeviceDetect from "../../../../hooks/useDeviceDetect";
 import { ChildComponentHandle } from "../GlobalSettings";
 import { useCustomPanelDispatch, useCustomPanelState } from "../../../../state/customPanel/CustomPanelStateContext";
-import Switch from "../../../Main/Switch/Switch";
-import { getFromStorage, saveToStorage } from "../../../../utils/storage";
 
 interface ServerConfiguratorProps {
-  onClose: () => void
+  onClose: () => void;
 }
 
-const fields: {label: string, type: DisplayType}[] = [
+const fields: { label: string, type: DisplayType }[] = [
   { label: "Graph", type: DisplayType.chart },
   { label: "JSON", type: DisplayType.code },
   { label: "Table", type: DisplayType.table }
@@ -29,8 +27,7 @@ const LimitsConfigurator = forwardRef<ChildComponentHandle, ServerConfiguratorPr
   const { seriesLimits } = useCustomPanelState();
   const customPanelDispatch = useCustomPanelDispatch();
 
-  const storageCollapse = getFromStorage("LEGEND_AUTO_COLLAPSE");
-  const [legendCollapse, setLegendCollapse] = useState(storageCollapse ? storageCollapse === "true" : true);
+
 
   const [limits, setLimits] = useState(seriesLimits);
   const [error, setError] = useState({
@@ -43,7 +40,7 @@ const LimitsConfigurator = forwardRef<ChildComponentHandle, ServerConfiguratorPr
     setLimits(DEFAULT_MAX_SERIES);
   };
 
-  const createChangeHandler = (type: DisplayType) =>  (val: string) => {
+  const createChangeHandler = (type: DisplayType) => (val: string) => {
     const value = val || "";
     setError(prev => ({ ...prev, [type]: +value < 0 ? ErrorTypes.positiveNumber : "" }));
     setLimits({
@@ -57,10 +54,6 @@ const LimitsConfigurator = forwardRef<ChildComponentHandle, ServerConfiguratorPr
     onClose();
   }, [limits]);
 
-  useEffect(() => {
-    saveToStorage("LEGEND_AUTO_COLLAPSE", `${legendCollapse}`);
-  }, [legendCollapse]);
-
   useImperativeHandle(ref, () => ({ handleApply }), [handleApply]);
 
   return (
@@ -106,19 +99,6 @@ const LimitsConfigurator = forwardRef<ChildComponentHandle, ServerConfiguratorPr
           </div>
         ))}
       </div>
-
-      <div className="vm-graph-settings-row">
-        <span className="vm-graph-settings-row__label">Auto-collapse legend</span>
-        <Switch
-          value={legendCollapse}
-          onChange={setLegendCollapse}
-          label={legendCollapse ? "Enabled" : "Disabled"}
-          fullWidth={isMobile}
-        />
-        <span className="vm-legend-configs-item__info">
-          Collapses the legend when series count exceeds {LEGEND_COLLAPSE_SERIES_LIMIT} to reduce UI load.
-        </span>
-      </div>
     </div>
   );
 });

app/vmui/packages/vmui/src/components/Configurators/GlobalSettings/TimeZonePreview/TimeZonePreview.tsx

@@ -0,0 +1,52 @@
+import { FC } from "preact/compat";
+import Button from "../../../Main/Button/Button";
+import { useTimeState } from "../../../../state/time/TimeStateContext";
+import useDeviceDetect from "../../../../hooks/useDeviceDetect";
+import { getUTCByTimezone } from "../../../../utils/time";
+import { useMemo } from "react";
+import { ArrowDownIcon, PlanetIcon } from "../../../Main/Icons";
+
+type Props = {
+  onOpenSettings?: () => void;
+}
+
+const TimeZonePreview: FC<Props> = ({ onOpenSettings }) => {
+  const { isMobile } = useDeviceDetect();
+
+  const { timezone } = useTimeState();
+  const utcOffset = useMemo(() => getUTCByTimezone(timezone), [timezone]);
+
+  const handleOpenSettings = () => {
+    onOpenSettings && onOpenSettings();
+  };
+
+
+  if (isMobile) {
+    return (
+      <button
+        className="vm-mobile-option"
+        onClick={handleOpenSettings}
+      >
+        <span className="vm-mobile-option__icon"><PlanetIcon/></span>
+        <div className="vm-mobile-option-text">
+          <span className="vm-mobile-option-text__label">Time zone</span>
+          <span className="vm-mobile-option-text__value">{utcOffset}</span>
+        </div>
+        <span className="vm-mobile-option__arrow"><ArrowDownIcon/></span>
+      </button>
+    );
+  }
+
+  return (
+    <Button
+      className="vm-header-button"
+      onClick={handleOpenSettings}
+      startIcon={<PlanetIcon/>}
+    >
+      {utcOffset}
+    </Button>
+  );
+};
+
+
+export default TimeZonePreview;

app/vmui/packages/vmui/src/components/Configurators/GlobalSettings/Timezones/Timezones.tsx

@@ -1,183 +0,0 @@
-import { FC, forwardRef, useEffect, useImperativeHandle, useMemo, useRef, useState } from "preact/compat";
-import { getBrowserTimezone, getTimezoneList, getUTCByTimezone } from "../../../../utils/time";
-import { ArrowDropDownIcon } from "../../../Main/Icons";
-import classNames from "classnames";
-import Popper from "../../../Main/Popper/Popper";
-import Accordion from "../../../Main/Accordion/Accordion";
-import TextField from "../../../Main/TextField/TextField";
-import { Timezone } from "../../../../types";
-import "./style.scss";
-import useDeviceDetect from "../../../../hooks/useDeviceDetect";
-import useBoolean from "../../../../hooks/useBoolean";
-import WarningTimezone from "./WarningTimezone";
-import { useTimeDispatch, useTimeState } from "../../../../state/time/TimeStateContext";
-
-interface PinnedTimezone extends Timezone {
-  title: string;
-  isInvalid?: boolean;
-}
-
-const browserTimezone = getBrowserTimezone();
-
-const Timezones: FC = forwardRef((props, ref) => {
-  const { isMobile } = useDeviceDetect();
-  const timezones = getTimezoneList();
-
-  const { timezone: stateTimezone, defaultTimezone } = useTimeState();
-  const timeDispatch = useTimeDispatch();
-
-  const [timezone, setTimezone] = useState(stateTimezone);
-  const [search, setSearch] = useState("");
-  const targetRef = useRef<HTMLDivElement>(null);
-
-  const {
-    value: openList,
-    toggle: toggleOpenList,
-    setFalse: handleCloseList,
-  } = useBoolean(false);
-
-  const pinnedTimezones = useMemo(() => [
-    {
-      title: `Default time (${defaultTimezone})`,
-      region: defaultTimezone,
-      utc: defaultTimezone ? getUTCByTimezone(defaultTimezone) : "UTC"
-    },
-    {
-      title: browserTimezone.title,
-      region: browserTimezone.region,
-      utc: getUTCByTimezone(browserTimezone.region),
-      isInvalid: !browserTimezone.isValid
-    },
-    {
-      title: "UTC (Coordinated Universal Time)",
-      region: "UTC",
-      utc: "UTC"
-    },
-  ].filter(t => t.region) as PinnedTimezone[], [defaultTimezone]);
-
-  const searchTimezones = useMemo(() => {
-    if (!search) return timezones;
-    try {
-      return getTimezoneList(search);
-    } catch (e) {
-      return {};
-    }
-  }, [search, timezones]);
-
-  const timezonesGroups = useMemo(() => Object.keys(searchTimezones), [searchTimezones]);
-
-  const activeTimezone = useMemo(() => ({
-    region: timezone,
-    utc: getUTCByTimezone(timezone)
-  }), [timezone]);
-
-  const handleChangeSearch = (val: string) => {
-    setSearch(val);
-  };
-
-  const handleSetTimezone = (val: Timezone) => {
-    setTimezone(val.region);
-    setSearch("");
-    handleCloseList();
-  };
-
-  const createHandlerSetTimezone = (val: Timezone) => () => {
-    handleSetTimezone(val);
-  };
-
-  useEffect(() => {
-    setTimezone(stateTimezone);
-  }, [stateTimezone]);
-
-  useImperativeHandle(ref, () => ({
-    handleApply: () => {
-      timeDispatch({ type: "SET_TIMEZONE", payload: timezone });
-    }
-  }), [timezone]);
-
-  return (
-    <div className="vm-timezones">
-      <div className="vm-server-configurator__title">
-        Time zone
-      </div>
-      <div
-        className="vm-timezones-item vm-timezones-item_selected"
-        onClick={toggleOpenList}
-        ref={targetRef}
-      >
-        <div className="vm-timezones-item__title">{activeTimezone.region}</div>
-        <div className="vm-timezones-item__utc">{activeTimezone.utc}</div>
-        <div
-          className={classNames({
-            "vm-timezones-item__icon": true,
-            "vm-timezones-item__icon_open": openList
-          })}
-        >
-          <ArrowDropDownIcon/>
-        </div>
-      </div>
-      <Popper
-        open={openList}
-        buttonRef={targetRef}
-        placement="bottom-left"
-        onClose={handleCloseList}
-        fullWidth
-        title={isMobile ? "Time zone" : undefined}
-      >
-        <div
-          className={classNames({
-            "vm-timezones-list": true,
-            "vm-timezones-list_mobile": isMobile,
-          })}
-        >
-          <div className="vm-timezones-list-header">
-            <div className="vm-timezones-list-header__search">
-              <TextField
-                autofocus
-                label="Search"
-                value={search}
-                onChange={handleChangeSearch}
-              />
-            </div>
-            {pinnedTimezones.map((t, i) => t && (
-              <div
-                key={`${i}_${t.region}`}
-                className="vm-timezones-item vm-timezones-list-group-options__item"
-                onClick={createHandlerSetTimezone(t)}
-              >
-                <div className="vm-timezones-item__title">{t.title}{t.isInvalid && <WarningTimezone/>}</div>
-                <div className="vm-timezones-item__utc">{t.utc}</div>
-              </div>
-            ))}
-          </div>
-          {timezonesGroups.map(t => (
-            <div
-              className="vm-timezones-list-group"
-              key={t}
-            >
-              <Accordion
-                defaultExpanded={true}
-                title={<div className="vm-timezones-list-group__title">{t}</div>}
-              >
-                <div className="vm-timezones-list-group-options">
-                  {searchTimezones[t] && searchTimezones[t].map(item => (
-                    <div
-                      className="vm-timezones-item vm-timezones-list-group-options__item"
-                      onClick={createHandlerSetTimezone(item)}
-                      key={item.search}
-                    >
-                      <div className="vm-timezones-item__title">{item.region}</div>
-                      <div className="vm-timezones-item__utc">{item.utc}</div>
-                    </div>
-                  ))}
-                </div>
-              </Accordion>
-            </div>
-          ))}
-        </div>
-      </Popper>
-    </div>
-  );
-});
-
-export default Timezones;

app/vmui/packages/vmui/src/components/Configurators/GlobalSettings/Timezones/TimezonesList.tsx

@@ -0,0 +1,129 @@
+import { FC, useMemo, useState } from "preact/compat";
+import { getBrowserTimezone, getTimezoneList, getUTCByTimezone } from "../../../../utils/time";
+import classNames from "classnames";
+import Accordion from "../../../Main/Accordion/Accordion";
+import TextField from "../../../Main/TextField/TextField";
+import { Timezone } from "../../../../types";
+import "./style.scss";
+import useDeviceDetect from "../../../../hooks/useDeviceDetect";
+import WarningTimezone from "./WarningTimezone";
+import { useTimeState } from "../../../../state/time/TimeStateContext";
+
+interface PinnedTimezone extends Timezone {
+  title: string;
+  isInvalid?: boolean;
+}
+
+type Props = {
+  onChange: (tz: Timezone) => void;
+}
+
+const browserTimezone = getBrowserTimezone();
+
+const TimezonesList: FC<Props> = ({ onChange }) => {
+  const { isMobile } = useDeviceDetect();
+  const { defaultTimezone } = useTimeState();
+
+  const timezones = useMemo(() => getTimezoneList(), []);
+
+  const [search, setSearch] = useState("");
+
+  const pinnedTimezones = useMemo(() => [
+    {
+      title: `Default time (${defaultTimezone})`,
+      region: defaultTimezone,
+      utc: defaultTimezone ? getUTCByTimezone(defaultTimezone) : "UTC"
+    },
+    {
+      title: browserTimezone.title,
+      region: browserTimezone.region,
+      utc: getUTCByTimezone(browserTimezone.region),
+      isInvalid: !browserTimezone.isValid
+    },
+    {
+      title: "UTC (Coordinated Universal Time)",
+      region: "UTC",
+      utc: "UTC"
+    },
+  ].filter(t => t.region) as PinnedTimezone[], [defaultTimezone]);
+
+  const searchTimezones = useMemo(() => {
+    if (!search) return timezones;
+    try {
+      return getTimezoneList(search);
+    } catch (e) {
+      return {};
+    }
+  }, [search, timezones]);
+
+  const timezonesGroups = useMemo(() => Object.keys(searchTimezones), [searchTimezones]);
+
+  const handleChangeSearch = (val: string) => {
+    setSearch(val);
+  };
+
+  const handleSetTimezone = (tz: Timezone) => {
+    onChange(tz);
+    setSearch("");
+  };
+
+  const createHandlerSetTimezone = (val: Timezone) => () => {
+    handleSetTimezone(val);
+  };
+
+  return (
+    <div
+      className={classNames({
+        "vm-list": true,
+        "vm-timezones-list": true,
+        "vm-timezones-list_mobile": isMobile,
+      })}
+    >
+      <div className="vm-timezones-list-header">
+        <div className="vm-timezones-list-header__search">
+          <TextField
+            label="Search"
+            value={search}
+            onChange={handleChangeSearch}
+          />
+        </div>
+      </div>
+      {pinnedTimezones.map((t, i) => t && (
+        <div
+          key={`${i}_${t.region}`}
+          className="vm-list-item vm-timezones-item vm-timezones-list-group-options__item"
+          onClick={createHandlerSetTimezone(t)}
+        >
+          <div className="vm-timezones-item__title">{t.title}{t.isInvalid && <WarningTimezone/>}</div>
+          <div className="vm-timezones-item__utc">{t.utc}</div>
+        </div>
+      ))}
+      {timezonesGroups.map(t => (
+        <div
+          className="vm-timezones-list-group"
+          key={t}
+        >
+          <Accordion
+            defaultExpanded={true}
+            title={<div className="vm-timezones-list-group__title">{t}</div>}
+          >
+            <div className="vm-timezones-list-group-options">
+              {searchTimezones[t] && searchTimezones[t].map(item => (
+                <div
+                  className="vm-list-item vm-timezones-item vm-timezones-list-group-options__item"
+                  onClick={createHandlerSetTimezone(item)}
+                  key={item.search}
+                >
+                  <div className="vm-timezones-item__title">{item.region}</div>
+                  <div className="vm-timezones-item__utc">{item.utc}</div>
+                </div>
+              ))}
+            </div>
+          </Accordion>
+        </div>
+      ))}
+    </div>
+  );
+};
+
+export default TimezonesList;

app/vmui/packages/vmui/src/components/Configurators/GlobalSettings/Timezones/TimezonesPicker.tsx

@@ -0,0 +1,71 @@
+import { FC, useMemo, useRef } from "preact/compat";
+import { getUTCByTimezone } from "../../../../utils/time";
+import { ArrowDropDownIcon } from "../../../Main/Icons";
+import classNames from "classnames";
+import { Timezone } from "../../../../types";
+import "./style.scss";
+import useBoolean from "../../../../hooks/useBoolean";
+import { useTimeDispatch, useTimeState } from "../../../../state/time/TimeStateContext";
+import TimezonesList from "./TimezonesList";
+import Popper from "../../../Main/Popper/Popper";
+import useDeviceDetect from "../../../../hooks/useDeviceDetect";
+
+const TimezonesPicker: FC = () => {
+  const { isMobile } = useDeviceDetect();
+  const { timezone: stateTimezone } = useTimeState();
+  const timeDispatch = useTimeDispatch();
+
+  const triggerRef = useRef<HTMLDivElement>(null);
+
+  const {
+    value: isOpenList,
+    toggle: toggleOpenList,
+    setFalse: handleCloseList,
+  } = useBoolean(false);
+
+  const activeTimezone = useMemo(() => ({
+    region: stateTimezone,
+    utc: getUTCByTimezone(stateTimezone)
+  }), [stateTimezone]);
+
+  const handleSetTimezone = (tz: Timezone) => {
+    timeDispatch({ type: "SET_TIMEZONE", payload: tz.region });
+    handleCloseList();
+  };
+
+  return (
+    <div className="vm-timezones">
+      <div className="vm-server-configurator__title">
+        Time zone
+      </div>
+      <div
+        className="vm-timezones-item vm-timezones-item_selected"
+        onClick={toggleOpenList}
+        ref={triggerRef}
+      >
+        <div className="vm-timezones-item__title">{activeTimezone.region}</div>
+        <div className="vm-timezones-item__utc">{activeTimezone.utc}</div>
+        <div
+          className={classNames({
+            "vm-timezones-item__icon": true,
+            "vm-timezones-item__icon_open": isOpenList
+          })}
+        >
+          <ArrowDropDownIcon/>
+        </div>
+      </div>
+      <Popper
+        open={isOpenList}
+        buttonRef={triggerRef}
+        placement="bottom-left"
+        onClose={handleCloseList}
+        fullWidth
+        title={isMobile ? "Time zone" : undefined}
+      >
+        <TimezonesList onChange={handleSetTimezone}/>
+      </Popper>
+    </div>
+  );
+};
+
+export default TimezonesPicker;

app/vmui/packages/vmui/src/components/Configurators/GlobalSettings/Timezones/style.scss

@@ -16,6 +16,7 @@
     }
 
     &__title {
+      flex-grow: 1;
       display: flex;
       align-items: center;
       gap: $padding-small;
@@ -34,6 +35,7 @@
       background-color: $color-hover-black;
       padding: calc($padding-small/2);
       border-radius: $border-radius-small;
+      font-size: $font-size-small;
     }
 
     &__icon {
@@ -54,9 +56,11 @@
   }
 
   &-list {
+    padding-top: 0;
     max-height: 300px;
     background-color: $color-background-block;
     border-radius: $border-radius-medium;
+    font-size: $font-size-small;
     overflow: auto;
 
     &_mobile {
@@ -72,10 +76,9 @@
       top: 0;
       background-color: $color-background-block;
       z-index: 2;
-      border-bottom: $border-divider;
 
       &__search {
-        padding: $padding-small;
+        padding: $padding-small $padding-small calc($padding-small / 2);
       }
     }
 
@@ -91,6 +94,7 @@
         font-weight: bold;
         color: $color-text-secondary;
         padding: $padding-small $padding-global;
+        font-size: $font-size-small;
       }
 
       &-options {
@@ -98,7 +102,7 @@
         align-items: flex-start;
 
         &__item {
-          padding: $padding-small $padding-global;
+          padding: calc($padding-small / 2) $padding-global;
           transition: background-color 200ms ease;
 
           &:hover {

app/vmui/packages/vmui/src/components/Configurators/GlobalSettings/style.scss

@@ -4,9 +4,9 @@
   display: flex;
   flex-direction: column;
   align-items: center;
-  gap: $padding-large;
+  gap: calc($padding-global * 2);
   width: 600px;
-  padding-bottom: $padding-medium;
+  padding-inline: $padding-large;
 
   &_mobile {
     grid-auto-rows: min-content;
@@ -62,6 +62,7 @@
     justify-content: flex-end;
     gap: $padding-small;
     width: 100%;
+    padding-block: $padding-global;
   }
 
   &_mobile &-footer {

app/vmui/packages/vmui/src/components/Configurators/StepConfigurator/StepConfigurator.tsx

@@ -22,12 +22,10 @@ const StepConfigurator: FC = () => {
   const { isMobile } = useDeviceDetect();
 
   const { customStep: value, isHistogram } = useGraphState();
-  const { period: { step, end, start } } = useTimeState();
+  const { period: { end, start } } = useTimeState();
   const graphDispatch = useGraphDispatch();
   const { displayType } = useCustomPanelState();
 
-  const prevDuration = usePrevious(end - start);
-
   const defaultStep = useMemo(() => {
     return getStepFromDuration(end - start, isHistogram, displayType);
   }, [end, start, isHistogram, displayType]);
@@ -106,16 +104,16 @@ const StepConfigurator: FC = () => {
   }, [defaultStep]);
 
   useEffect(() => {
-    const dur = end - start;
-    if (dur === prevDuration || !prevDuration || value !== prevDefaultStep) return;
-    if (defaultStep) {
-      handleApply(defaultStep);
-    }
-  }, [prevDuration, defaultStep]);
+    if (!prevDefaultStep) return;
+    if (value !== prevDefaultStep) return;
+    if (value === defaultStep) return;
 
-  useEffect(() => {
-    if (step === value || step === defaultStep) handleApply(defaultStep);
-  }, [isHistogram, displayType]);
+    graphDispatch({ type: "SET_CUSTOM_STEP", payload: defaultStep });
+    setCustomStep(defaultStep);
+    setError("");
+  }, [defaultStep, prevDefaultStep, value, graphDispatch]);
+
+  const textValue = isAutoStep ? `auto (${customStep})` : customStep;
 
   return (
     <div
@@ -130,7 +128,7 @@ const StepConfigurator: FC = () => {
           <span className="vm-mobile-option__icon"><TimelineIcon/></span>
           <div className="vm-mobile-option-text">
             <span className="vm-mobile-option-text__label">Step</span>
-            <span className="vm-mobile-option-text__value">{customStep}</span>
+            <span className="vm-mobile-option-text__value">{textValue}</span>
           </div>
           <span className="vm-mobile-option__arrow"><ArrowDownIcon/></span>
         </div>
@@ -142,7 +140,7 @@ const StepConfigurator: FC = () => {
           startIcon={<TimelineIcon/>}
           onClick={toggleOpenOptions}
         >
-          Step: {isAutoStep ? `auto (${customStep})` : customStep}
+          Step: {textValue}
         </Button>
       )}
       <Popper

app/vmui/packages/vmui/src/components/Configurators/ThemeControl/ThemeControl.tsx

@@ -5,8 +5,20 @@ import useDeviceDetect from "../../../hooks/useDeviceDetect";
 import classNames from "classnames";
 import { FC } from "preact/compat";
 import { useAppDispatch, useAppState } from "../../../state/common/StateContext";
+import { DarkIcon, LightIcon, SystemIcon } from "../../Main/Icons";
+
+const themeIcons = {
+  [Theme.system]: <SystemIcon/>,
+  [Theme.light]: <LightIcon/>,
+  [Theme.dark]: <DarkIcon/>,
+};
+
+const options = Object.values(Theme).map(value => ({
+  title: value,
+  value,
+  icon: themeIcons[value],
+}));
 
-const options = Object.values(Theme).map(value => ({ title: value, value }));
 const ThemeControl: FC = () => {
   const { isMobile } = useDeviceDetect();
   const dispatch = useAppDispatch();
@@ -25,13 +37,14 @@ const ThemeControl: FC = () => {
       })}
     >
       <div className="vm-server-configurator__title">
-        Theme preferences
+        Theme
       </div>
       <div
         className="vm-theme-control__toggle"
         key={`${isMobile}`}
       >
         <Toggle
+          size="large"
           options={options}
           value={theme}
           onChange={handleClickItem}

app/vmui/packages/vmui/src/components/Configurators/ThemeControl/style.scss

@@ -4,7 +4,7 @@
 
   &__toggle {
     display: inline-flex;
-    min-width: 300px;
+    width: 100%;
     text-transform: capitalize;
   }
 

app/vmui/packages/vmui/src/components/Configurators/TimeRangeSettings/TimeSelector/TimeSelector.tsx

@@ -19,7 +19,11 @@ import useBoolean from "../../../../hooks/useBoolean";
 import useWindowSize from "../../../../hooks/useWindowSize";
 import usePrevious from "../../../../hooks/usePrevious";
 
-export const TimeSelector: FC = () => {
+type Props = {
+  onOpenSettings?: () => void;
+}
+
+export const TimeSelector: FC<Props> = ({ onOpenSettings }) => {
   const { isMobile } = useDeviceDetect();
   const { isDarkTheme } = useAppState();
   const wrapperRef = useRef<HTMLDivElement>(null);
@@ -53,7 +57,7 @@ export const TimeSelector: FC = () => {
     setFrom(formatDateForNativeInput(dateFromSeconds(start)));
   }, [timezone, start]);
 
-  const setDuration = ({ duration, until, id }: {duration: string, until: Date, id: string}) => {
+  const setDuration = ({ duration, until, id }: { duration: string, until: Date, id: string }) => {
     dispatch({ type: "SET_RELATIVE_TIME", payload: { duration, until, id } });
     handleCloseOptions();
   };
@@ -75,16 +79,23 @@ export const TimeSelector: FC = () => {
 
   const setTimeAndClosePicker = () => {
     if (from && until) {
-      dispatch({ type: "SET_PERIOD", payload: {
-        from: dayjs.tz(from).toDate(),
-        to: dayjs.tz(until).toDate()
-      } });
+      dispatch({
+        type: "SET_PERIOD", payload: {
+          from: dayjs.tz(from).toDate(),
+          to: dayjs.tz(until).toDate()
+        }
+      });
     }
     handleCloseOptions();
   };
 
   const onSwitchToNow = () => dispatch({ type: "RUN_QUERY_TO_NOW" });
 
+  const handleOpenSettings = () => {
+    onOpenSettings && onOpenSettings();
+    handleCloseOptions();
+  };
+
   const onCancelClick = () => {
     setUntil(formatDateForNativeInput(dateFromSeconds(end)));
     setFrom(formatDateForNativeInput(dateFromSeconds(start)));
@@ -140,6 +151,7 @@ export const TimeSelector: FC = () => {
         </Tooltip>
       )}
     </div>
+
     <Popper
       open={openOptions}
       buttonRef={buttonRef}
@@ -179,13 +191,17 @@ export const TimeSelector: FC = () => {
               onEnter={setTimeAndClosePicker}
             />
           </div>
-          <div className="vm-time-selector-left-timezone">
-            <div className="vm-time-selector-left-timezone__title">{activeTimezone.region}</div>
-            <div className="vm-time-selector-left-timezone__utc">{activeTimezone.utc}</div>
-          </div>
+          <button
+            type="button"
+            className="vm-time-selector-left-timezone"
+            onClick={handleOpenSettings}
+          >
+            <span className="vm-time-selector-left-timezone__title">{activeTimezone.region}</span>
+            <span className="vm-time-selector-left-timezone__utc">{activeTimezone.utc}</span>
+          </button>
           <Button
             variant="text"
-            startIcon={<AlarmIcon />}
+            startIcon={<AlarmIcon/>}
             onClick={onSwitchToNow}
           >
             switch to now

app/vmui/packages/vmui/src/components/Configurators/TimeRangeSettings/TimeSelector/style.scss

@@ -40,8 +40,13 @@
       gap: $padding-small;
       font-size: $font-size-small;
       margin-bottom: $padding-small;
+      color: $color-text;
+      cursor: pointer;
 
-      &__title {}
+      &:hover {
+        color: $color-primary;
+        text-decoration: underline;
+      }
 
       &__utc {
         display: inline-flex;

app/vmui/packages/vmui/src/components/Main/Icons/index.tsx

@@ -633,3 +633,71 @@ export const DebugIcon = () => (
     />
   </svg>
 );
+
+export const PlanetIcon = () => (
+  <svg
+    viewBox="0 0 24 24"
+    fill="currentColor"
+  >
+    <path
+      d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10 10-4.48 10-10S17.52 2 12 2M4 12c0-.61.08-1.21.21-1.78L8.99 15v1c0 1.1.9 2 2 2v1.93C7.06 19.43 4 16.07 4 12m13.89 5.4c-.26-.81-1-1.4-1.9-1.4h-1v-3c0-.55-.45-1-1-1h-6v-2h2c.55 0 1-.45 1-1V7h2c1.1 0 2-.9 2-2v-.41C17.92 5.77 20 8.65 20 12c0 2.08-.81 3.98-2.11 5.4"
+    ></path>
+  </svg>
+);
+
+export const SystemIcon = () => (
+  <svg
+    viewBox="0 0 24 24"
+    fill="currentColor"
+  >
+    <path
+      d="M18 4C18.7957 4 19.5595 4.3163 20.1221 4.87891C20.6845 5.44148 21 6.20452 21 7V15.5264L21.0069 15.6426C21.0203 15.7579 21.0542 15.8702 21.1065 15.9746L22.1729 18.0996C22.3271 18.4056 22.4009 18.7466 22.3858 19.0889C22.3705 19.431 22.2675 19.7637 22.0869 20.0547C21.9063 20.3457 21.6534 20.5855 21.3535 20.751C21.0555 20.9154 20.7202 21.0003 20.3799 20.999L3.62013 21C3.28002 21.0012 2.94535 20.9153 2.64748 20.751C2.34748 20.5855 2.09477 20.3458 1.91408 20.0547C1.73343 19.7636 1.63047 19.4311 1.61525 19.0889C1.60006 18.7466 1.67297 18.4056 1.82716 18.0996L2.89455 15.9746L2.94045 15.8682C2.9801 15.7589 3.00007 15.6432 3.00002 15.5264V7C3.00002 6.20442 3.3164 5.4415 3.87892 4.87891C4.44146 4.31636 5.20447 4.00007 6.00002 4H18ZM4.62404 16.9873L3.61427 18.999L3.6133 19H20.3877L20.3867 18.999L19.376 16.9873H4.62404ZM6.00002 6C5.7349 6.00007 5.48045 6.1055 5.29298 6.29297C5.10554 6.48049 5.00002 6.73485 5.00002 7V14.9873H19V7C19 6.73478 18.8946 6.48051 18.707 6.29297C18.5195 6.10552 18.2652 6 18 6H6.00002Z"
+    />
+  </svg>
+);
+
+export const LightIcon = () => (
+  <svg
+    viewBox="0 0 24 24"
+    fill="currentColor"
+  >
+    <path
+      d="M12 19C12.5523 19 13 19.4477 13 20V22C13 22.5523 12.5523 23 12 23C11.4477 23 11 22.5523 11 22V20C11 19.4477 11.4477 19 12 19Z"
+    />
+    <path
+      d="M5.63281 16.9531C6.02334 16.5627 6.65638 16.5626 7.04688 16.9531C7.43717 17.3436 7.43725 17.9767 7.04688 18.3672L5.63672 19.7773C5.24625 20.1676 4.61313 20.1676 4.22266 19.7773C3.8322 19.3869 3.83233 18.7538 4.22266 18.3633L5.63281 16.9531Z"
+    />
+    <path
+      d="M16.9531 16.9531C17.3436 16.5626 17.9767 16.5626 18.3672 16.9531L19.7773 18.3633C20.1676 18.7538 20.1678 19.3869 19.7773 19.7773C19.3869 20.1677 18.7538 20.1675 18.3633 19.7773L16.9531 18.3672C16.5626 17.9767 16.5627 17.3437 16.9531 16.9531Z"
+    />
+    <path
+      d="M12 7C14.7614 7 17 9.23858 17 12C17 14.7614 14.7614 17 12 17C9.23858 17 7 14.7614 7 12C7 9.23858 9.23858 7 12 7ZM12 9C10.3431 9 9 10.3431 9 12C9 13.6569 10.3431 15 12 15C13.6569 15 15 13.6569 15 12C15 10.3431 13.6569 9 12 9Z"
+    />
+    <path
+      d="M4 11C4.55228 11 5 11.4477 5 12C5 12.5523 4.55228 13 4 13H2C1.44772 13 1 12.5523 1 12C1 11.4477 1.44772 11 2 11H4Z"
+    />
+    <path
+      d="M22 11C22.5523 11 23 11.4477 23 12C23 12.5523 22.5523 13 22 13H20C19.4477 13 19 12.5523 19 12C19 11.4477 19.4477 11 20 11H22Z"
+    />
+    <path
+      d="M4.22266 4.22266C4.61315 3.83229 5.24623 3.83229 5.63672 4.22266L7.04688 5.63281C7.4372 6.02331 7.43723 6.65639 7.04688 7.04688C6.6564 7.43735 6.02335 7.43724 5.63281 7.04688L4.22266 5.63672C3.83225 5.24618 3.83217 4.61314 4.22266 4.22266Z"
+    />
+    <path
+      d="M18.3633 4.22266C18.7538 3.83237 19.3869 3.83232 19.7773 4.22266C20.1677 4.61312 20.1676 5.2462 19.7773 5.63672L18.3672 7.04688C17.9767 7.4373 17.3436 7.4373 16.9531 7.04688C16.5627 6.65637 16.5627 6.0233 16.9531 5.63281L18.3633 4.22266Z"
+    />
+    <path
+      d="M12 1C12.5523 1 13 1.44772 13 2V4C13 4.55228 12.5523 5 12 5C11.4477 5 11 4.55228 11 4V2C11 1.44772 11.4477 1 12 1Z"
+    />
+  </svg>
+);
+
+export const DarkIcon = () => (
+  <svg
+    viewBox="0 0 24 24"
+    fill="currentColor"
+  >
+    <path
+      d="M11.5809 2.01318C12.1851 2.02907 12.6373 2.40742 12.8475 2.84717C13.0627 3.29758 13.0619 3.86893 12.7615 4.34814L12.7606 4.34717C12.1616 5.30581 11.9061 6.43987 12.034 7.56299C12.162 8.68628 12.6672 9.73328 13.4666 10.5327C14.2661 11.332 15.3131 11.8364 16.4363 11.9644C17.5596 12.0922 18.6934 11.836 19.6522 11.2368L19.8348 11.1382C20.2701 10.9405 20.7563 10.9617 21.1512 11.1499C21.6217 11.3742 22.0194 11.8752 21.9832 12.5405C21.8789 14.4693 21.2181 16.3268 20.0809 17.8882C18.9435 19.4496 17.378 20.6484 15.574 21.3394C13.7701 22.0302 11.8042 22.184 9.91485 21.7817C8.02549 21.3794 6.29356 20.4376 4.92754 19.0718C3.56149 17.7059 2.62012 15.9739 2.21758 14.0845C1.81507 12.195 1.96826 10.2294 2.65899 8.42529C3.34975 6.62115 4.5487 5.05596 6.11016 3.91846C7.6716 2.781 9.52882 2.11947 11.4578 2.01514L11.5809 2.01318ZM10.6209 4.12061C9.42038 4.33038 8.27873 4.81214 7.28692 5.53467C6.03798 6.44459 5.07973 7.69705 4.52715 9.14014C3.97456 10.5834 3.85165 12.156 4.17364 13.6675C4.49565 15.179 5.24872 16.5649 6.34161 17.6577C7.43448 18.7505 8.82026 19.5038 10.3318 19.8257C11.8434 20.1475 13.416 20.0239 14.8592 19.4712C16.3024 18.9184 17.5548 17.9597 18.4647 16.7104C19.1869 15.7188 19.6671 14.5776 19.8768 13.3774C18.7333 13.8927 17.4674 14.0949 16.2098 13.9517C14.637 13.7725 13.1709 13.0661 12.0516 11.9468C10.9324 10.8275 10.2258 9.36126 10.0467 7.78857C9.90352 6.53075 10.1054 5.26417 10.6209 4.12061Z"
+    />
+  </svg>
+);

app/vmui/packages/vmui/src/components/Main/Switch/Switch.tsx

@@ -1,11 +1,10 @@
-import { ReactNode } from "react";
+import { FC, ReactNode } from "preact/compat";
 import classNames from "classnames";
 import "./style.scss";
-import { FC } from "preact/compat";
 
 interface SwitchProps {
   value: boolean
-  color?: "primary" | "secondary" | "error"
+  color?: "primary" | "secondary" | "error" | "neutral"
   disabled?: boolean
   label?: string | ReactNode
   fullWidth?: boolean

app/vmui/packages/vmui/src/components/Main/Switch/style.scss

@@ -29,6 +29,10 @@ $switch-border-radius: $switch-handle-size + ($switch-padding * 2);
     background-color: $color-secondary;
   }
 
+  &_neutral_active &-track {
+    background-color: $color-text;
+  }
+
   &_primary_active &-track {
     background-color: $color-primary;
   }

app/vmui/packages/vmui/src/components/Main/Toggle/Toggle.tsx

@@ -1,22 +1,21 @@
-import { FC, useEffect, useRef, useState } from "preact/compat";
+import { FC, useEffect, useRef, useState, ReactNode } from "preact/compat";
 import classNames from "classnames";
-import { ReactNode } from "react";
 import "./style.scss";
 
 interface ToggleProps {
-  options: {value: string, title?: string, icon?: ReactNode}[]
-  value: string
-  onChange: (val: string) => void
-  label?: string
+  options: { value: string, title?: string, icon?: ReactNode }[];
+  value: string;
+  onChange: (val: string) => void;
+  label?: string;
+  size?: "medium" | "large";
 }
 
-const Toggle: FC<ToggleProps> = ({ options, value, label, onChange }) => {
+const Toggle: FC<ToggleProps> = ({ options, value, label, size = "medium", onChange }) => {
 
   const activeRef = useRef<HTMLDivElement>(null);
   const [position, setPosition] = useState({
     width: "0px",
     left: "0px",
-    borderRadius: "0px"
   });
 
   const createHandlerChange = (value: string) => () => {
@@ -28,35 +27,25 @@ const Toggle: FC<ToggleProps> = ({ options, value, label, onChange }) => {
       setPosition({
         width: "0px",
         left: "0px",
-        borderRadius: "0px"
       });
       return;
     }
     const index = options.findIndex(o => o.value === value);
     const { width: widthRect } = activeRef.current.getBoundingClientRect();
 
-    let width = widthRect;
-    let left = index * width;
-    let borderRadius = "0";
-    if (index === 0) borderRadius = "16px 0 0 16px";
+    const width = widthRect;
+    const left = index * width;
 
-    if (index === options.length - 1) {
-      borderRadius = "10px";
-      left -= 1;
-      borderRadius = "0 16px 16px 0";
-    }
-
-    if (index !== 0 && (index !== options.length - 1)) {
-      width += 1;
-      left -= 1;
-    }
-
-
-    setPosition({ width: `${width}px`, left: `${left}px`, borderRadius });
+    setPosition({ width: `${width}px`, left: `${left}px` });
   }, [activeRef, value, options]);
 
   return (
-    <div className="vm-toggles">
+    <div
+      className={classNames({
+        "vm-toggles": true,
+        [`vm-toggles_${size}`]: size,
+      })}
+    >
       {label && (
         <label className="vm-toggles__label">
           {label}
@@ -66,15 +55,14 @@ const Toggle: FC<ToggleProps> = ({ options, value, label, onChange }) => {
         className="vm-toggles-group"
         style={{ gridTemplateColumns: `repeat(${options.length}, 1fr)` }}
       >
-        {position.borderRadius && <div
+        <div
           className="vm-toggles-group__highlight"
           style={position}
-        />}
-        {options.map((option, i) => (
+        />
+        {options.map((option) => (
           <div
             className={classNames({
               "vm-toggles-group-item": true,
-              "vm-toggles-group-item_first": i === 0,
               "vm-toggles-group-item_active": option.value === value,
               "vm-toggles-group-item_icon": option.icon && option.title
             })}

app/vmui/packages/vmui/src/components/Main/Toggle/style.scss

@@ -20,6 +20,8 @@
     align-items: center;
     justify-content: center;
     overflow: hidden;
+    border-radius: $border-radius-small;
+    background: $color-hover-black;
 
     &-item {
       position: relative;
@@ -27,55 +29,68 @@
       align-items: center;
       justify-content: center;
       padding: $padding-small;
-      border-right: $border-divider;
-      border-top: $border-divider;
-      border-bottom: $border-divider;
       font-size: $font-size-small;
       color: $color-text-secondary;
-      font-weight: bold;
+      font-weight: 500;
       cursor: pointer;
       text-align: center;
-      transition: color 150ms ease-in;
+      transition: opacity 150ms ease-in, color 150ms ease-in;
       z-index: 2;
       user-select: none;
 
-      &_first {
-        border-radius: 16px 0 0 16px;
-        border-left: $border-divider
-      }
-
-      &:last-child {
-        border-radius: 0 16px 16px 0;
-        border-left: none;
-      }
-
       &_icon {
         grid-template-columns: 14px auto;
-        gap: 4px;
+        gap: calc($padding-small / 2);
       }
 
-      &:hover {
-        color: $color-primary;
+      &:hover:not(&_active) {
+        opacity: 0.8;
       }
 
       &_active {
-        color: $color-primary;
-        border-color: transparent;
-
-        &:hover {
-          background-color: transparent;
-        }
+        color: $color-text;
+        font-weight: 600;
       }
     }
 
     &__highlight {
       position: absolute;
       top: 0;
+      display: flex;
+      align-items: center;
+      justify-content: center;
+      padding: 3px;
       height: 100%;
       background-color: rgba($color-primary, 0.08);
-      border: 1px solid $color-primary;
       transition: left 200ms cubic-bezier(0.280, 0.840, 0.420, 1), border-radius 200ms linear;
       z-index: 1;
+
+      &:after {
+        content: '';
+        height: 100%;
+        width: 100%;
+        background-color: $color-background-block;
+        border-radius: $border-radius-small;
+        box-shadow: $box-shadow;
+      }
+    }
+  }
+
+  &_large &-group {
+    border-radius: $border-radius-medium;
+
+    &-item {
+      padding: $padding-global $padding-small;
+
+      &_icon {
+        grid-template-columns: 16px auto;
+        gap: $padding-small;
+      }
+    }
+
+    &__highlight {
+      padding: 4px;
+      border-radius: $border-radius-medium;
     }
   }
 }

app/vmui/packages/vmui/src/constants/faviconColors.ts

@@ -0,0 +1,14 @@
+export const faviconColors = [
+  "#A1A1AA",
+  "#71717A",
+  "#020202",
+  "#E94600",
+  "#FF7A00",
+  "#F2B705",
+  "#84CC16",
+  "#16B86A",
+  "#00AFAF",
+  "#2979FF",
+  "#8B5CF6",
+  "#E83E9A",
+] as const;

app/vmui/packages/vmui/src/layouts/Header/HeaderControls/style.scss

@@ -11,6 +11,7 @@
   &_mobile {
     display: grid;
     grid-template-columns: 1fr;
+    gap: 0;
     padding: 0;
     flex-grow: initial;
 

app/vmui/packages/vmui/src/layouts/MainLayout/ControlsMainLayout.tsx

@@ -6,9 +6,11 @@ import StepConfigurator from "../../components/Configurators/StepConfigurator/St
 import { TimeSelector } from "../../components/Configurators/TimeRangeSettings/TimeSelector/TimeSelector";
 import CardinalityDatePicker from "../../components/Configurators/CardinalityDatePicker/CardinalityDatePicker";
 import { ExecutionControls } from "../../components/Configurators/TimeRangeSettings/ExecutionControls/ExecutionControls";
-import GlobalSettings from "../../components/Configurators/GlobalSettings/GlobalSettings";
+import GlobalSettings, { GlobalSettingsHandle } from "../../components/Configurators/GlobalSettings/GlobalSettings";
 import ShortcutKeys from "../../components/Main/ShortcutKeys/ShortcutKeys";
 import { ControlsProps } from "../Header/HeaderControls/HeaderControls";
+import { useRef } from "react";
+import TimeZonePreview from "../../components/Configurators/GlobalSettings/TimeZonePreview/TimeZonePreview";
 
 const ControlsMainLayout: FC<ControlsProps> = ({
   displaySidebar,
@@ -17,6 +19,7 @@ const ControlsMainLayout: FC<ControlsProps> = ({
   accountIds,
   closeModal,
 }) => {
+  const settingsRef = useRef<GlobalSettingsHandle>(null);
 
   return (
     <div
@@ -27,14 +30,15 @@ const ControlsMainLayout: FC<ControlsProps> = ({
     >
       {headerSetup?.tenant && <TenantsConfiguration accountIds={accountIds || []}/>}
       {headerSetup?.stepControl && <StepConfigurator/>}
-      {headerSetup?.timeSelector && <TimeSelector/>}
+      {headerSetup?.timeSelector && <TimeSelector onOpenSettings={() => settingsRef.current?.open()}/>}
       {headerSetup?.cardinalityDatePicker && <CardinalityDatePicker/>}
+      <TimeZonePreview onOpenSettings={() => settingsRef.current?.open()}/>
       {headerSetup?.executionControls && <ExecutionControls
         tooltip={headerSetup?.executionControls?.tooltip}
         useAutorefresh={headerSetup?.executionControls?.useAutorefresh}
         closeModal={closeModal}
       />}
-      <GlobalSettings/>
+      <GlobalSettings ref={settingsRef}/>
       {!displaySidebar && <ShortcutKeys/>}
     </div>
   );

app/vmui/packages/vmui/src/layouts/MainLayout/MainLayout.tsx

@@ -14,6 +14,9 @@ import useFetchDefaultTimezone from "../../hooks/useFetchDefaultTimezone";
 import useFetchAppConfig from "../../hooks/useFetchAppConfig";
 import WebStorageCheck from "../../components/WebStorageCheck/WebStorageCheck";
 import { migrateStorageToPrefixedKeys } from "../../utils/storage";
+import {
+  useBrowserTabSync
+} from "../../components/Configurators/GlobalSettings/BrowserTabController/hooks/useBrowserTabSync";
 
 const MainLayout: FC = () => {
   const appModeEnable = getAppModeEnable();
@@ -21,6 +24,7 @@ const MainLayout: FC = () => {
   const { pathname } = useLocation();
   const [searchParams, setSearchParams] = useSearchParams();
 
+  useBrowserTabSync();
   useFetchDashboards();
   useFetchDefaultTimezone();
   useFetchAppConfig();

app/vmui/packages/vmui/src/styles/components/mobile-option.scss

@@ -1,11 +1,12 @@
 @use "src/styles/variables" as *;
 
 .vm-mobile-option {
-  display: flex;
+  display: grid;
+  grid-template-columns: auto 1fr auto;
   align-items: center;
   justify-content: flex-start;
-  gap: $padding-small;
-  padding: calc($padding-medium/2) 0;
+  gap: $padding-global;
+  padding: $padding-global $padding-small;
   width: 100%;
   user-select: none;
 
@@ -17,14 +18,33 @@
   }
 
   &__icon {
-    width: 22px;
-    height: 22px;
+    position: relative;
+    display: flex;
+    width: 40px;
+    height: 40px;
     color: $color-primary;
+
+    &:after {
+      content: "";
+      position: absolute;
+      top: 0;
+      left: 0;
+      width: 100%;
+      height: 100%;
+      opacity: 0.1;
+      background-color: currentColor;
+      border-radius: $border-radius-medium;
+    }
+
+    svg {
+      width: 21px;
+      height: auto;
+    }
   }
 
   &__arrow {
-    width: 14px;
-    height: 14px;
+    width: 20px;
+    height: 20px;
     transform: rotate(-90deg);
     color: $color-primary;
   }
@@ -32,11 +52,13 @@
   &-text {
     display: grid;
     align-items: center;
-    gap: 2px;
+    height: 100%;
+    gap: calc($padding-small / 2);
     flex-grow: 1;
+    text-align: left;
 
     &__label {
-      font-weight: bold;
+      font-weight: 600;
     }
 
     &__value {

app/vmui/packages/vmui/src/utils/favicon.tsx

@@ -0,0 +1,29 @@
+import faviconRaw from "../../assets/favicon.svg?raw";
+
+export const createFaviconUrl = (color = "#020202"): string => {
+  const svgDocument = new DOMParser().parseFromString(faviconRaw, "image/svg+xml");
+  const svg = svgDocument.documentElement;
+
+  if (svg.localName !== "svg") {
+    throw new Error("Invalid favicon SVG");
+  }
+
+  svg.setAttribute("fill", color);
+
+  const serializedSvg = new XMLSerializer().serializeToString(svg);
+  return `data:image/svg+xml;charset=utf-8,${encodeURIComponent(serializedSvg)}`;
+};
+
+export const updateFaviconColor = (color = "#020202"): void => {
+  const favicon = document.querySelector<HTMLLinkElement>("#favicon");
+  if (favicon) {
+    favicon.href = createFaviconUrl(color);
+  }
+
+  const maskIcon = document.querySelector<HTMLLinkElement>("#mask-icon");
+  if (maskIcon) {
+    maskIcon.setAttribute("color", color);
+  }
+};
+
+export const getFaviconStorageKey = () => window.location.pathname.replace(/\/+$/, "") || "/";

app/vmui/packages/vmui/src/utils/storage.ts

@@ -17,7 +17,11 @@ export const ALL_STORAGE_KEYS = [
   "POINTS_SHOW_ALL",
 ] as const;
 
-export type StorageKeys = (typeof ALL_STORAGE_KEYS)[number];
+export type FaviconStorageKey = `FAVICON_COLOR:${string}`;
+
+export type StorageKeys =
+  | (typeof ALL_STORAGE_KEYS)[number]
+  | FaviconStorageKey;
 
 type PrefixedStorageKeys = `${typeof STORAGE_PREFIX}${StorageKeys}`;
 
@@ -58,7 +62,10 @@ export const getFromStorage = (key: StorageKeys, withPrefix = true): undefined |
 
 export const removeFromStorage = (keys: StorageKeys[], withPrefix = true): void => {
   const storageKeys = withPrefix ? keys.map(toPrefixedKey) : keys;
-  storageKeys.forEach(k => window.localStorage.removeItem(k));
+  storageKeys.forEach(k => {
+    window.localStorage.removeItem(k);
+    window.dispatchEvent(new StorageEvent("storage", { key: k }));
+  });
 };
 
 /**

app/vmui/packages/vmui/src/utils/time.ts

@@ -205,19 +205,21 @@ export const getUTCByTimezone = (timezone: string) => {
 };
 
 export const getTimezoneList = (search = "") => {
-  const regexp = new RegExp(search, "i");
+  const normalizedSearch = search.toLowerCase();
 
-  return supportedTimezones.reduce((acc: {[key: string]: Timezone[]}, region) => {
+  return supportedTimezones.reduce((acc: { [key: string]: Timezone[] }, region) => {
     const zone = (region.match(/^(.*?)\//) || [])[1] || "unknown";
     const utc = getUTCByTimezone(region);
-    const utcForSearch = utc.replace(/UTC|0/, "");
+    const utcForSearch = utc.replace(/^UTC/, "");
     const regionForSearch = region.replace(/[/_]/g, " ");
+
     const item = {
       region,
       utc,
       search: `${region} ${utc} ${regionForSearch} ${utcForSearch}`
     };
-    const includeZone = !search || (search && regexp.test(item.search));
+
+    const includeZone = !normalizedSearch || item.search.toLowerCase().includes(normalizedSearch);
 
     if (includeZone && acc[zone]) {
       acc[zone].push(item);

app/vmui/packages/vmui/vite.config.ts

@@ -50,6 +50,13 @@ export default defineConfig(() => {
               return "vendor";
             }
           },
+          assetFileNames: (assetInfo) => {
+            if (assetInfo.names.includes("favicon.svg")) {
+              return "assets/favicon.svg";
+            }
+
+            return "assets/[name]-[hash][extname]";
+          },
         },
       },
     },

apptest/client.go

@@ -135,7 +135,7 @@ func tenantViaURL(addr, prefix, tenant, suffix string) string {
 }
 
 // tenantViaHeaders returns path in cluster's URL format where tenant is omitted in URL
-// Only supported if -enableMultitenancyViaHeaders is specified
+// Only supported if -enableMultitenancyViaHeaders is enabled
 func tenantViaHeaders(addr, prefix, suffix string) string {
 	return fmt.Sprintf("http://%s/%s/%s", addr, prefix, suffix)
 }

apptest/tests/multitenancy_via_headers_test.go

@@ -25,12 +25,10 @@ func TestClusterMultiTenantSelectViaHeaders(t *testing.T) {
 	})
 	vminsert := tc.MustStartVminsert("vminsert", []string{
 		"-storageNode=" + vmstorage.VminsertAddr(),
-		"-enableMultitenancyViaHeaders",
 	})
 	vmselect := tc.MustStartVmselect("vmselect", []string{
 		"-storageNode=" + vmstorage.VmselectAddr(),
 		"-search.tenantCacheExpireDuration=0",
-		"-enableMultitenancyViaHeaders",
 	})
 
 	multitenant := make(http.Header)

apptest/tests/vmagent_remotewrite_test.go

@@ -594,7 +594,6 @@ func TestSingleVMAgentMultitenancy(t *testing.T) {
 		fmt.Sprintf(`-remoteWrite.url=%s/api/v1/write`, remoteWriteSrv.URL),
 		"-remoteWrite.tmpDataPath=" + tc.Dir() + "/vmagent-multitenancy",
 		"-enableMultitenantHandlers",
-		"-enableMultitenancyViaHeaders",
 	})
 
 	vmagent.APIV1ImportPrometheus(t, []string{

dashboards/backupmanager.json

@@ -59,6 +59,19 @@
         },
         "type": "dashboard"
       },
+      {
+        "datasource": {
+          "type": "prometheus",
+          "uid": "$ds"
+        },
+        "enable": true,
+        "expr": "sum by(version) (\n    label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n    OR\n    vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n) \nunless \n(\n    sum by(version) (\n        label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n        OR\n        vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n    ) offset $__interval\n)",
+        "hide": true,
+        "iconColor": "dark-blue",
+        "name": "version",
+        "textFormat": "{{version}}",
+        "titleFormat": "Version change"
+      },
       {
         "datasource": {
           "type": "prometheus",

dashboards/victoriametrics-cluster.json

@@ -37,7 +37,7 @@
           "uid": "$ds"
         },
         "enable": true,
-        "expr": "sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"}) by(version) unless (sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"} offset $__interval) by(version))",
+        "expr": "sum by(version) (\n    label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n    OR\n    vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n) \nunless \n(\n    sum by(version) (\n        label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n        OR\n        vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n    ) offset $__interval\n)",
         "hide": true,
         "iconColor": "dark-blue",
         "name": "version change",

dashboards/victoriametrics.json

@@ -37,7 +37,7 @@
           "uid": "$ds"
         },
         "enable": true,
-        "expr": "sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"}) by(version) unless (sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"} offset $__interval) by(version))",
+        "expr": "sum by(version) (\n    label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n    OR\n    vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n) \nunless \n(\n    sum by(version) (\n        label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n        OR\n        vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n    ) offset $__interval\n)",
         "hide": true,
         "iconColor": "dark-blue",
         "name": "version",

dashboards/vm/backupmanager.json

@@ -60,6 +60,19 @@
         },
         "type": "dashboard"
       },
+      {
+        "datasource": {
+          "type": "victoriametrics-metrics-datasource",
+          "uid": "$ds"
+        },
+        "enable": true,
+        "expr": "sum by(version) (\n    label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n    OR\n    vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n) \nunless \n(\n    sum by(version) (\n        label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n        OR\n        vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n    ) offset $__interval\n)",
+        "hide": true,
+        "iconColor": "dark-blue",
+        "name": "version",
+        "textFormat": "{{version}}",
+        "titleFormat": "Version change"
+      },
       {
         "datasource": {
           "type": "victoriametrics-metrics-datasource",

dashboards/vm/victoriametrics-cluster.json

@@ -38,7 +38,7 @@
           "uid": "$ds"
         },
         "enable": true,
-        "expr": "sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"}) by(version) unless (sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"} offset $__interval) by(version))",
+        "expr": "sum by(version) (\n    label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n    OR\n    vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n) \nunless \n(\n    sum by(version) (\n        label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n        OR\n        vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n    ) offset $__interval\n)",
         "hide": true,
         "iconColor": "dark-blue",
         "name": "version change",

dashboards/vm/victoriametrics.json

@@ -38,7 +38,7 @@
           "uid": "$ds"
         },
         "enable": true,
-        "expr": "sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"}) by(version) unless (sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"} offset $__interval) by(version))",
+        "expr": "sum by(version) (\n    label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n    OR\n    vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n) \nunless \n(\n    sum by(version) (\n        label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n        OR\n        vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n    ) offset $__interval\n)",
         "hide": true,
         "iconColor": "dark-blue",
         "name": "version",

dashboards/vm/vmagent.json

@@ -26,11 +26,11 @@
           "uid": "$ds"
         },
         "enable": true,
-        "expr": "sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"}) by(short_version) unless (sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"} offset $__interval) by(short_version))",
+        "expr": "sum by(version) (\n    label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n    OR\n    vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n) \nunless \n(\n    sum by(version) (\n        label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n        OR\n        vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n    ) offset $__interval\n)",
         "hide": true,
         "iconColor": "dark-blue",
         "name": "version",
-        "textFormat": "{{short_version}}",
+        "textFormat": "{{version}}",
         "titleFormat": "Version change"
       },
       {

dashboards/vm/vmalert.json

@@ -26,11 +26,11 @@
           "uid": "$ds"
         },
         "enable": true,
-        "expr": "sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"}) by(short_version) unless (sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"} offset $__interval) by(short_version))",
+        "expr": "sum by(version) (\n    label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n    OR\n    vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n) \nunless \n(\n    sum by(version) (\n        label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n        OR\n        vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n    ) offset $__interval\n)",
         "hide": true,
         "iconColor": "dark-blue",
         "name": "version",
-        "textFormat": "{{short_version}}",
+        "textFormat": "{{version}}",
         "titleFormat": "Version change"
       },
       {

dashboards/vm/vmauth.json

@@ -54,6 +54,19 @@
         },
         "type": "dashboard"
       },
+      {
+        "datasource": {
+          "type": "victoriametrics-metrics-datasource",
+          "uid": "$ds"
+        },
+        "enable": true,
+        "expr": "sum by(version) (\n    label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n    OR\n    vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n) \nunless \n(\n    sum by(version) (\n        label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n        OR\n        vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n    ) offset $__interval\n)",
+        "hide": true,
+        "iconColor": "dark-blue",
+        "name": "version",
+        "textFormat": "{{version}}",
+        "titleFormat": "Version change"
+      },
       {
         "datasource": {
           "type": "victoriametrics-metrics-datasource",

dashboards/vmagent.json

@@ -25,11 +25,11 @@
           "uid": "$ds"
         },
         "enable": true,
-        "expr": "sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"}) by(short_version) unless (sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"} offset $__interval) by(short_version))",
+        "expr": "sum by(version) (\n    label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n    OR\n    vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n) \nunless \n(\n    sum by(version) (\n        label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n        OR\n        vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n    ) offset $__interval\n)",
         "hide": true,
         "iconColor": "dark-blue",
         "name": "version",
-        "textFormat": "{{short_version}}",
+        "textFormat": "{{version}}",
         "titleFormat": "Version change"
       },
       {

dashboards/vmalert.json

@@ -25,11 +25,11 @@
           "uid": "$ds"
         },
         "enable": true,
-        "expr": "sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"}) by(short_version) unless (sum(vm_app_version{job=~\"$job\", instance=~\"$instance\"} offset $__interval) by(short_version))",
+        "expr": "sum by(version) (\n    label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n    OR\n    vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n) \nunless \n(\n    sum by(version) (\n        label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n        OR\n        vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n    ) offset $__interval\n)",
         "hide": true,
         "iconColor": "dark-blue",
         "name": "version",
-        "textFormat": "{{short_version}}",
+        "textFormat": "{{version}}",
         "titleFormat": "Version change"
       },
       {

dashboards/vmauth.json

@@ -53,6 +53,19 @@
         },
         "type": "dashboard"
       },
+      {
+        "datasource": {
+          "type": "prometheus",
+          "uid": "$ds"
+        },
+        "enable": true,
+        "expr": "sum by(version) (\n    label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n    OR\n    vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n) \nunless \n(\n    sum by(version) (\n        label_replace(vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version!=\"\"}, \"version\", \"$1\", \"short_version\", \"(.*)\")\n        OR\n        vm_app_version{job=~\"$job\", instance=~\"$instance\", short_version=\"\"}\n    ) offset $__interval\n)",
+        "hide": true,
+        "iconColor": "dark-blue",
+        "name": "version",
+        "textFormat": "{{version}}",
+        "titleFormat": "Version change"
+      },
       {
         "datasource": {
           "type": "prometheus",

deployment/docker/Makefile

@@ -7,7 +7,7 @@ ROOT_IMAGE ?= alpine:3.24.1
 ROOT_IMAGE_SCRATCH ?= scratch
 CERTS_IMAGE := alpine:3.24.1
 
-GO_BUILDER_IMAGE := golang:1.26.5
+GO_BUILDER_IMAGE := golang:1.26.6
 
 BUILDER_IMAGE := local/builder:2.0.0-$(shell echo $(GO_BUILDER_IMAGE) | tr :/ __)-1
 BASE_IMAGE := local/base:1.1.4-$(shell echo $(ROOT_IMAGE) | tr :/ __)-$(shell echo $(CERTS_IMAGE) | tr :/ __)

deployment/docker/compose-vm-cluster.yml

@@ -3,7 +3,7 @@ services:
   #  It scrapes targets defined in --promscrape.config
   #  And forward them to --remoteWrite.url
   vmagent:
-    image: victoriametrics/vmagent:v1.148.0
+    image: victoriametrics/vmagent:v1.150.0
     depends_on:
       - "vmauth"
     ports:
@@ -42,14 +42,14 @@ services:
   # vmstorage shards. Each shard receives 1/N of all metrics sent to vminserts,
   # where N is number of vmstorages (2 in this case).
   vmstorage-1:
-    image: victoriametrics/vmstorage:v1.148.0-cluster
+    image: victoriametrics/vmstorage:v1.150.0-cluster
     volumes:
       - strgdata-1:/storage
     command:
       - "--storageDataPath=/storage"
     restart: always
   vmstorage-2:
-    image: victoriametrics/vmstorage:v1.148.0-cluster
+    image: victoriametrics/vmstorage:v1.150.0-cluster
     volumes:
       - strgdata-2:/storage
     command:
@@ -59,7 +59,7 @@ services:
   # vminsert is ingestion frontend. It receives metrics pushed by vmagent,
   # pre-process them and distributes across configured vmstorage shards.
   vminsert-1:
-    image: victoriametrics/vminsert:v1.148.0-cluster
+    image: victoriametrics/vminsert:v1.150.0-cluster
     depends_on:
       - "vmstorage-1"
       - "vmstorage-2"
@@ -68,7 +68,7 @@ services:
       - "--storageNode=vmstorage-2:8400"
     restart: always
   vminsert-2:
-    image: victoriametrics/vminsert:v1.148.0-cluster
+    image: victoriametrics/vminsert:v1.150.0-cluster
     depends_on:
       - "vmstorage-1"
       - "vmstorage-2"
@@ -80,7 +80,7 @@ services:
   # vmselect is a query fronted. It serves read queries in MetricsQL or PromQL.
   # vmselect collects results from configured `--storageNode` shards.
   vmselect-1:
-    image: victoriametrics/vmselect:v1.148.0-cluster
+    image: victoriametrics/vmselect:v1.150.0-cluster
     depends_on:
       - "vmstorage-1"
       - "vmstorage-2"
@@ -90,7 +90,7 @@ services:
       - "--vmalert.proxyURL=http://vmalert:8880"
     restart: always
   vmselect-2:
-    image: victoriametrics/vmselect:v1.148.0-cluster
+    image: victoriametrics/vmselect:v1.150.0-cluster
     depends_on:
       - "vmstorage-1"
       - "vmstorage-2"
@@ -105,7 +105,7 @@ services:
   # read requests from Grafana, vmui, vmalert among vmselects.
   # It can be used as an authentication proxy.
   vmauth:
-    image: victoriametrics/vmauth:v1.148.0
+    image: victoriametrics/vmauth:v1.150.0
     depends_on:
       - "vmselect-1"
       - "vmselect-2"
@@ -119,7 +119,7 @@ services:
 
   # vmalert executes alerting and recording rules
   vmalert:
-    image: victoriametrics/vmalert:v1.148.0
+    image: victoriametrics/vmalert:v1.150.0
     depends_on:
       - "vmauth"
     ports:

deployment/docker/compose-vm-single.yml

@@ -3,7 +3,7 @@ services:
   #  It scrapes targets defined in --promscrape.config
   #  And forward them to --remoteWrite.url
   vmagent:
-    image: victoriametrics/vmagent:v1.148.0
+    image: victoriametrics/vmagent:v1.150.0
     depends_on:
       - "victoriametrics"
     ports:
@@ -18,7 +18,7 @@ services:
   # VictoriaMetrics instance, a single process responsible for
   # storing metrics and serve read requests.
   victoriametrics:
-    image: victoriametrics/victoria-metrics:v1.148.0
+    image: victoriametrics/victoria-metrics:v1.150.0
     ports:
       - 8428:8428
       - 8089:8089
@@ -59,7 +59,7 @@ services:
 
   # vmalert executes alerting and recording rules
   vmalert:
-    image: victoriametrics/vmalert:v1.148.0
+    image: victoriametrics/vmalert:v1.150.0
     depends_on:
       - "victoriametrics"
       - "alertmanager"

deployment/docker/rules/alerts-health.yml

@@ -16,6 +16,19 @@ groups:
             Job {{ $labels.job }} (instance {{ $labels.instance }}) has restarted more than twice in the last 15 minutes.
             It might be crashlooping.
 
+      - alert: UncleanShutdown
+        expr: vm_app_prev_shutdown_unclean == 1 and time() - vm_app_start_timestamp < 600
+        labels:
+          severity: warning
+        annotations:
+          summary: "{{ $labels.job }} on instance {{ $labels.instance }} started after an unclean shutdown"
+          description: |
+            The previous process run didn't shut down cleanly. Check the logs for OOM, SIGKILL,
+            a host failure, or another unexpected termination. In Kubernetes, a pod may be forcefully
+            killed with SIGKILL if the shutdown takes longer than terminationGracePeriodSeconds.
+            This alert stops firing 10 minutes after startup.
+            See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8443 for more details.
+
       - alert: ServiceDown
         expr: up{job=~".*(victoriametrics|vmselect|vminsert|vmstorage|vmagent|vmalert|vmsingle|vmalertmanager|vmauth).*"} == 0
         for: 2m

deployment/docker/rules/alerts-vmalert.yml

@@ -126,10 +126,10 @@ groups:
           (
             vmalert_alerting_rules_last_evaluation_samples
               > on(group,file) group_left()
-            (vmalert_group_rule_results_limit * 0.9)
+            max by (group,file) (vmalert_group_rule_results_limit * 0.9)
           )
             and on(group,file)
-          (vmalert_group_rule_results_limit > 0)
+          (max by (group,file) (vmalert_group_rule_results_limit) > 0)
         for: 5m
         labels:
           severity: warning
@@ -144,10 +144,10 @@ groups:
           (
             vmalert_recording_rules_last_evaluation_samples
               > on(group,file) group_left()
-            (vmalert_group_rule_results_limit * 0.9)
+            max by (group,file) (vmalert_group_rule_results_limit * 0.9)
           )
             and on(group,file)
-          (vmalert_group_rule_results_limit > 0)
+          (max by (group,file) (vmalert_group_rule_results_limit) > 0)
         for: 5m
         labels:
           severity: warning

deployment/docker/vmanomaly/vmanomaly-integration/compose.yml

@@ -1,6 +1,6 @@
 services:
   vmagent:
-    image: victoriametrics/vmagent:v1.148.0
+    image: victoriametrics/vmagent:v1.150.0
     depends_on:
       - "victoriametrics"
     ports:
@@ -14,7 +14,7 @@ services:
     restart: always
 
   victoriametrics:
-    image: victoriametrics/victoria-metrics:v1.148.0
+    image: victoriametrics/victoria-metrics:v1.150.0
     ports:
       - 8428:8428
     volumes:
@@ -40,7 +40,7 @@ services:
     restart: always
 
   vmalert:
-    image: victoriametrics/vmalert:v1.148.0
+    image: victoriametrics/vmalert:v1.150.0
     depends_on:
       - "victoriametrics"
     ports:
@@ -59,7 +59,7 @@ services:
       - '--external.alert.source=explore?orgId=1&left=["now-1h","now","VictoriaMetrics",{"expr": },{"mode":"Metrics"},{"ui":[true,true,true,"none"]}]'
     restart: always
   vmanomaly:
-    image: victoriametrics/vmanomaly:v1.30.0
+    image: victoriametrics/vmanomaly:v1.30.2
     depends_on:
       - "victoriametrics"
     ports:

deployment/docker/vmanomaly/vmanomaly-integration/vmanomaly_config.yml

@@ -1,7 +1,7 @@
 schedulers:
   periodic:
     infer_every: "1m"
-    fit_every: "100w" # the online model keeps learning during inference
+    fit_every: "1000d" # bootstrap-only schedule; use a finite cadence if accumulated state must be reset
     fit_window: "2w"
 
 models:

docs/Makefile

@@ -90,12 +90,9 @@ endif
 	sed -i 's/\t/ /g' docs/victoriametrics/victoria_metrics_common_flags.md
 	sed -i 's/\t/ /g' docs/victoriametrics/victoria_metrics_enterprise_flags.md
 
-	# adjust flags with dynamic default values
-	# remove after https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9680 implemented
-	sed -i '/The maximum number of concurrent insert requests/ s/(default [0-9]\+)/(default 2*cgroup.AvailableCPUs())/' docs/victoriametrics/victoria_metrics_common_flags.md
-	sed -i '/The maximum number of concurrent search requests\./ s/(default [0-9]\+)/(default vmselect.getDefaultMaxConcurrentRequests())/' docs/victoriametrics/victoria_metrics_common_flags.md
-	sed -i '/The maximum number of CPU cores a single query can use\./ s/(default [0-9]\+)/(default netstorage.defaultMaxWorkersPerQuery())/' docs/victoriametrics/victoria_metrics_common_flags.md
-	sed -i '/The maximum number of concurrent goroutines to work with files;/ s/(default [0-9]\+)/(default fsutil.getDefaultConcurrency())/' docs/victoriametrics/victoria_metrics_common_flags.md
+	# hide the machine-specific value of dynamic defaults, keeping the formula.
+	# the flagutil.New*WithDynamicDefault constructors print them as "(default <value> = <formula>)".
+	sed -i 's/(default [0-9]\+ = \(.*\))$$/(default \1)/' docs/victoriametrics/victoria_metrics_common_flags.md
 
 docs-update-vmauth-flags:
 ifndef TAG
@@ -119,9 +116,9 @@ endif
 	sed -i 's/\t/ /g' docs/victoriametrics/vmauth_common_flags.md
 	sed -i 's/\t/ /g' docs/victoriametrics/vmauth_enterprise_flags.md
 
-	# adjust flags with dynamic default values
-	# remove after https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9680 implemented
-	sed -i '/The maximum number of concurrent goroutines to work with files;/ s/(default [0-9]\+)/(default fsutil.getDefaultConcurrency())/' docs/victoriametrics/vmauth_common_flags.md
+	# hide the machine-specific value of dynamic defaults, keeping the formula.
+	# the flagutil.New*WithDynamicDefault constructors print them as "(default <value> = <formula>)".
+	sed -i 's/(default [0-9]\+ = \(.*\))$$/(default \1)/' docs/victoriametrics/vmauth_common_flags.md
 
 docs-update-vmagent-flags:
 ifndef TAG
@@ -145,11 +142,9 @@ endif
 	sed -i 's/\t/ /g' docs/victoriametrics/vmagent_common_flags.md
 	sed -i 's/\t/ /g' docs/victoriametrics/vmagent_enterprise_flags.md
 
-	# adjust flags with dynamic default values
-	# remove after https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9680 implemented
-	sed -i '/The maximum number of concurrent insert requests/ s/(default [0-9]\+)/(default 2*cgroup.AvailableCPUs())/' docs/victoriametrics/vmagent_common_flags.md
-	sed -i '/The number of concurrent queues to each -remoteWrite.url./ s/(default [0-9]\+)/(default 2*cgroup.AvailableCPUs())/' docs/victoriametrics/vmagent_common_flags.md
-	sed -i '/The maximum number of concurrent goroutines to work with files;/ s/(default [0-9]\+)/(default fsutil.getDefaultConcurrency())/' docs/victoriametrics/vmagent_common_flags.md
+	# hide the machine-specific value of dynamic defaults, keeping the formula.
+	# the flagutil.New*WithDynamicDefault constructors print them as "(default <value> = <formula>)".
+	sed -i 's/(default [0-9]\+ = \(.*\))$$/(default \1)/' docs/victoriametrics/vmagent_common_flags.md
 
 docs-update-vmalert-flags:
 ifndef TAG
@@ -173,10 +168,9 @@ endif
 	sed -i 's/\t/ /g' docs/victoriametrics/vmalert_common_flags.md
 	sed -i 's/\t/ /g' docs/victoriametrics/vmalert_enterprise_flags.md
 
-	# adjust flags with dynamic default values
-	# remove after https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9680 implemented
-	sed -i '/Defines number of writers for concurrent writing into remote write endpoint./ s/(default [0-9]\+)/(default 2*cgroup.AvailableCPUs())/' docs/victoriametrics/vmalert_common_flags.md
-	sed -i '/The maximum number of concurrent goroutines to work with files;/ s/(default [0-9]\+)/(default fsutil.getDefaultConcurrency())/' docs/victoriametrics/vmalert_common_flags.md
+	# hide the machine-specific value of dynamic defaults, keeping the formula.
+	# the flagutil.New*WithDynamicDefault constructors print them as "(default <value> = <formula>)".
+	sed -i 's/(default [0-9]\+ = \(.*\))$$/(default \1)/' docs/victoriametrics/vmalert_common_flags.md
 
 docs-update-vmselect-flags:
 ifndef TAG

docs/ai-tools/README.md

@@ -1,3 +1,11 @@
+---
+build:
+  list: never
+  publishResources: false
+  render: never
+sitemap:
+  disable: true
+---
 VictoriaMetrics Observability Stack integrates with AI assistants through [MCP servers](https://docs.victoriametrics.com/ai-tools/#mcp-servers)
 and [agent skills](https://docs.victoriametrics.com/ai-tools/#agent-skills).
 The integrations allow AI agents and automation tools to query Metrics, Logs, and Traces, analyze telemetry data,