app/vmselect/promql: do not take into account the previous point before time window in square brackets for min_over_time, max_over_time, rollup_first and rollup_last functions

This makes the behaviour for these functions similar to Prometheus when processing broken time series with irregular data points like `gitlab_runner_jobs`. See https://gitlab.com/gitlab-org/gitlab-exporter/issues/50 for details.
vendor: update github.com/valyala/fastjson from v1.4.2 to v1.4.5
2026-06-09 03:43:58 +03:00 · 2020-01-11 00:26:26 +02:00 · 2020-01-10 23:15:15 +02:00 · 2020-01-10 21:57:18 +02:00 · 2020-01-10 21:18:06 +02:00 · 2020-01-10 19:39:28 +02:00
202 changed files with 8846 additions and 5046 deletions
--- a/2
+++ b/2
@@ -175,7 +175,7 @@

   END OF TERMS AND CONDITIONS

-   Copyright 2019 VictoriaMetrics, Inc.
+   Copyright 2019-2020 VictoriaMetrics, Inc.

   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
--- a/README.md
+++ b/README.md
@@ -6,7 +6,7 @@
 [![Build Status](https://github.com/VictoriaMetrics/VictoriaMetrics/workflows/main/badge.svg)](https://github.com/VictoriaMetrics/VictoriaMetrics/actions)
 [![codecov](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/branch/master/graph/badge.svg)](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics)

-<img alt="Victoria Metrics" src="logo.png" height="200px">
+<img alt="Victoria Metrics" src="logo.png">

 ## VictoriaMetrics

@@ -28,7 +28,7 @@ Cluster version is available [here](https://github.com/VictoriaMetrics/VictoriaM
 ## Prominent features

 * Supports [Prometheus querying API](https://prometheus.io/docs/prometheus/latest/querying/api/), so it can be used as Prometheus drop-in replacement in Grafana.
-  Additionally, VictoriaMetrics extends PromQL with opt-in [useful features](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/ExtendedPromQL).
+  VictoriaMetrics implements [MetricsQL](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/ExtendedPromQL) query language, which is inspired by PromQL.
 * Supports global query view. Multiple Prometheus instances may write data into VictoriaMetrics. Later this data may be used in a single query.
 * High performance and good scalability for both [inserts](https://medium.com/@valyala/high-cardinality-tsdb-benchmarks-victoriametrics-vs-timescaledb-vs-influxdb-13e6ee64dd6b)
  and [selects](https://medium.com/@valyala/when-size-matters-benchmarking-victoriametrics-vs-timescale-and-influxdb-6035811952d4).
@@ -38,9 +38,11 @@ Cluster version is available [here](https://github.com/VictoriaMetrics/VictoriaM
 * High data compression, so [up to 70x more data points](https://medium.com/@valyala/when-size-matters-benchmarking-victoriametrics-vs-timescale-and-influxdb-6035811952d4)
  may be crammed into limited storage comparing to TimescaleDB.
 * Optimized for storage with high-latency IO and low IOPS (HDD and network storage in AWS, Google Cloud, Microsoft Azure, etc). See [graphs from these benchmarks](https://medium.com/@valyala/high-cardinality-tsdb-benchmarks-victoriametrics-vs-timescaledb-vs-influxdb-13e6ee64dd6b).
-* A single-node VictoriaMetrics may substitute moderately sized clusters built with competing solutions such as Thanos, Uber M3, Cortex, InfluxDB or TimescaleDB.
-  See [vertical scalability benchmarks](https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae)
-  and [comparing Thanos to VictoriaMetrics cluster](https://medium.com/@valyala/comparing-thanos-to-victoriametrics-cluster-b193bea1683).
+* A single-node VictoriaMetrics may substitute moderately sized clusters built with competing solutions such as Thanos, M3DB, Cortex, InfluxDB or TimescaleDB.
+  See [vertical scalability benchmarks](https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae),
+  [comparing Thanos to VictoriaMetrics cluster](https://medium.com/@valyala/comparing-thanos-to-victoriametrics-cluster-b193bea1683)
+  and [Remote Write Storage Wars](https://promcon.io/2019-munich/talks/remote-write-storage-wars/) talk
+  from [PromCon 2019](https://promcon.io/2019-munich/talks/remote-write-storage-wars/).
 * Easy operation:
  * VictoriaMetrics consists of a single [small executable](https://medium.com/@valyala/stripping-dependency-bloat-in-victoriametrics-docker-image-983fb5912b0d) without external dependencies.
  * All the configuration is done via explicit command-line flags with reasonable defaults.
@@ -136,14 +138,13 @@ It is recommended setting up [monitoring](#monitoring) for VictoriaMetrics.

 ### Prometheus setup

-Add the following lines to Prometheus config file (it is usually located at `/etc/prometheus/prometheus.yml`):
+Prometheus must be configured with [remote_write](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write) 
+in order to send data to VictoriaMetrics. Add the following lines 
+to Prometheus config file (it is usually located at `/etc/prometheus/prometheus.yml`):

 ```yml
 remote_write:
  - url: http://<victoriametrics-addr>:8428/api/v1/write
-    queue_config:
-      max_samples_per_send: 10000
-      max_shards: 30
 ```

 Substitute `<victoriametrics-addr>` with the hostname or IP address of VictoriaMetrics.
@@ -170,6 +171,22 @@ This instructs Prometheus to add `datacenter=dc-123` label to each time series s
 The label name may be arbitrary - `datacenter` is just an example. The label value must be unique
 across Prometheus instances, so those time series may be filtered and grouped by this label.

+For highly loaded Prometheus instances (400k+ samples per second)
+the following tuning may be applied:
+```
+remote_write:
+  - url: http://<victoriametrics-addr>:8428/api/v1/write
+    queue_config:
+      max_samples_per_send: 10000
+      capacity: 20000
+      max_shards: 30
+```
+
+Using remote write increases memory usage for Prometheus up to ~25%
+and depends on the shape of data. If you are experiencing issues with
+too high memory consumption try to lower `max_samples_per_send` 
+and `capacity` params (keep in mind that these two params are tightly connected).
+Read more about tuning remote write for Prometheus [here](https://prometheus.io/docs/practices/remote_write).

 It is recommended upgrading Prometheus to [v2.12.0](https://github.com/prometheus/prometheus/releases) or newer,
 since the previous versions may have issues with `remote_write`.
@@ -544,6 +561,15 @@ Each JSON line would contain data for a single time series. An example output:
 Optional `start` and `end` args may be added to the request in order to limit the time frame for the exported data. These args may contain either
 unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) values.

+Pass `Accept-Encoding: gzip` HTTP header in the request to `/api/v1/export` in order to reduce network bandwidth during exporing big amounts
+of time series data. This enables gzip compression for the exported data. Example for exporting gzipped data:
+
+```
+curl -H 'Accept-Encoding: gzip' http://localhost:8428/api/v1/export -d 'match[]={__name__!=""}' > data.jsonl.gz
+```
+
+The maximum duration for each request to `/api/v1/export` is limited by `-search.maxExportDuration` command-line flag.
+
 Exported data can be imported via POST'ing it to [/api/v1/import](#how-to-import-time-series-data).


@@ -562,10 +588,20 @@ The most efficient protocol for importing data into VictoriaMetrics is `/api/v1/

 ```
 # Export the data from <source-victoriametrics>:
-curl -s 'http://source-victoriametrics:8428/api/v1/export' -d 'match={__name__!=""}' > exported_data.jsonl
+curl http://source-victoriametrics:8428/api/v1/export -d 'match={__name__!=""}' > exported_data.jsonl

 # Import the data to <destination-victoriametrics>:
-curl -X POST 'http://destination-victoriametrics:8428/api/v1/import' -T exported_data.jsonl
+curl -X POST http://destination-victoriametrics:8428/api/v1/import -T exported_data.jsonl
+```
+
+Pass `Content-Encoding: gzip` HTTP request header to `/api/v1/import` for importing gzipped data:
+
+```
+# Export gzipped data from <source-victoriametrics>:
+curl -H 'Accept-Encoding: gzip' http://source-victoriametrics:8428/api/v1/export -d 'match={__name__!=""}' > exported_data.jsonl.gz
+
+# Import gzipped data to <destination-victoriametrics>:
+curl -X POST -H 'Content-Encoding: gzip' http://destination-victoriametrics:8428/api/v1/import -T exported_data.jsonl.gz
 ```

 Each request to `/api/v1/import` can load up to a single vCPU core on VictoriaMetrics. Import speed can be improved by splitting the original file into smaller parts
@@ -591,7 +627,7 @@ A rough estimation of the required resources for ingestion path:
 * RAM size: less than 1KB per active time series. So, ~1GB of RAM is required for 1M active time series.
  Time series is considered active if new data points have been added to it recently or if it has been recently queried.
  The number of active time series may be obtained from `vm_cache_entries{type="storage/hour_metric_ids"}` metric
-  exproted on the `/metrics` page.
+  exported on the `/metrics` page.
  VictoriaMetrics stores various caches in RAM. Memory size for these caches may be limited by `-memory.allowedPercent` flag.

 * CPU cores: a CPU core per 300K inserted data points per second. So, ~4 CPU cores are required for processing
@@ -672,7 +708,7 @@ There is no downsampling support at the moment, but:
 - VictoriaMetrics has good compression for on-disk data. See [this article](https://medium.com/@valyala/victoriametrics-achieving-better-compression-for-time-series-data-than-gorilla-317bc1f95932)
  for details.

-These properties reduce the need in downsampling. We plan to implement downsampling in the future.
+These properties reduce the need of downsampling. We plan to implement downsampling in the future.
 See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/36) for details.


@@ -694,8 +730,10 @@ horizontally scalable long-term remote storage for really large Prometheus deplo

 ### Alerting

-VictoriaMetrics doesn't support rule evaluation and alerting yet, so these actions must be performed either
-on [Prometheus side](https://prometheus.io/docs/alerting/overview/) or on [Grafana side](https://grafana.com/docs/alerting/rules/).
+VictoriaMetrics doesn't support rule evaluation and alerting yet, so these actions can be performed at the following places:
+* At Prometheus - see [the corresponding docs](https://prometheus.io/docs/alerting/overview/).
+* At Promxy - see [the corresponding docs](https://github.com/jacksontj/promxy/blob/master/README.md#how-do-i-use-alertingrecording-rules-in-promxy).
+* At Grafana - see [the corresponding docs](https://grafana.com/docs/alerting/rules/).


 ### Security
@@ -715,14 +753,14 @@ For example, substitute `-graphiteListenAddr=:2003` with `-graphiteListenAddr=<i

 ### Tuning

-* There is no need in VictoriaMetrics tuning since it uses reasonable defaults for command-line flags,
+* There is no need for VictoriaMetrics tuning since it uses reasonable defaults for command-line flags,
  which are automatically adjusted for the available CPU and RAM resources.
-* There is no need in Operating System tuning since VictoriaMetrics is optimized for default OS settings.
+* There is no need for Operating System tuning since VictoriaMetrics is optimized for default OS settings.
  The only option is increasing the limit on [the number of open files in the OS](https://medium.com/@muhammadtriwibowo/set-permanently-ulimit-n-open-files-in-ubuntu-4d61064429a),
  so Prometheus instances could establish more connections to VictoriaMetrics.
 * The recommended filesystem is `ext4`, the recommended persistent storage is [persistent HDD-based disk on GCP](https://cloud.google.com/compute/docs/disks/#pdspecs),
  since it is protected from hardware failures via internal replication and it can be [resized on the fly](https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd).
-  If you plan storing more than 1TB of data on `ext4` partition or plan extending it to more than 16TB,
+  If you plan to store more than 1TB of data on `ext4` partition or plan extending it to more than 16TB,
  then the following options are recommended to pass to `mkfs.ext4`:

 ```
@@ -753,7 +791,7 @@ The most interesting metrics are:
 ### Troubleshooting

 * It is recommended to use default command-line flag values (i.e. don't set them explicitly) until the need
-  in tweaking these flag values arises.
+  of tweaking these flag values arises.

 * If VictoriaMetrics works slowly and eats more than a CPU core per 100K ingested data points per second,
  then it is likely you have too many active time series for the current amount of RAM.
@@ -767,7 +805,7 @@ The most interesting metrics are:
  has at least 20% of free space comparing to disk size.

 * If VictoriaMetrics doesn't work because of certain parts are corrupted due to disk errors,
-  then just remove directoreis with broken parts. This will recover VictoriaMetrics at the cost
+  then just remove directories with broken parts. This will recover VictoriaMetrics at the cost
  of data loss stored in the broken parts. In the future, `vmrecover` tool will be created
  for automatic recovering from such errors.

@@ -818,7 +856,7 @@ The collected profiles may be analyzed with [go tool pprof](https://github.com/g
 - [ ] CLI tool for data migration, re-balancing and adding/removing nodes [#103](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/103)


-The discussion happens [here](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/129). Feel free to comment any item or add own one.
+The discussion happens [here](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/129). Feel free to comment on any item or add you own one.


 ## Contacts
@@ -861,7 +899,7 @@ Report bugs and propose new features [here](https://github.com/VictoriaMetrics/V

 ## Victoria Metrics Logo

-[Zip](VM_logo.zip) contains three folders with different image orientation (main color and inverted version).
+[Zip](VM_logo.zip) contains three folders with different image orientations (main color and inverted version).

 Files included in each folder:

--- a/app/victoria-metrics/Makefile
+++ b/app/victoria-metrics/Makefile
@@ -9,6 +9,9 @@ victoria-metrics-prod:
 victoria-metrics-pure-prod:
 	APP_NAME=victoria-metrics $(MAKE) app-via-docker-pure

+victoria-metrics-amd64-prod:
+	APP_NAME=victoria-metrics $(MAKE) app-via-docker-amd64
+
 victoria-metrics-arm-prod:
 	APP_NAME=victoria-metrics $(MAKE) app-via-docker-arm

@@ -27,6 +30,9 @@ package-victoria-metrics:
 package-victoria-metrics-pure:
 	APP_NAME=victoria-metrics $(MAKE) package-via-docker-pure

+package-victoria-metrics-amd64:
+	APP_NAME=victoria-metrics $(MAKE) package-via-docker-amd64
+
 package-victoria-metrics-arm:
 	APP_NAME=victoria-metrics $(MAKE) package-via-docker-arm

@@ -49,6 +55,9 @@ run-victoria-metrics:
 	ARGS='-graphiteListenAddr=:2003 -opentsdbListenAddr=:4242 -retentionPeriod=12 -search.maxUniqueTimeseries=1000000 -search.maxQueryDuration=10m' \
 	$(MAKE) run-via-docker

+victoria-metrics-amd64:
+	CGO_ENABLED=1 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -mod=vendor -ldflags "$(GO_BUILDINFO)" -o bin/victoria-metrics-amd64 ./app/victoria-metrics
+
 victoria-metrics-arm:
 	CGO_ENABLED=0 GOOS=linux GOARCH=arm GO111MODULE=on go build -mod=vendor -ldflags "$(GO_BUILDINFO)" -o bin/victoria-metrics-arm ./app/victoria-metrics

--- a/app/victoria-metrics/main.go
+++ b/app/victoria-metrics/main.go
@@ -21,7 +21,7 @@ func main() {
 	flag.Parse()
 	buildinfo.Init()
 	logger.Init()
-	logger.Infof("starting VictoraMetrics at %q...", *httpListenAddr)
+	logger.Infof("starting VictoriaMetrics at %q...", *httpListenAddr)
 	startTime := time.Now()
 	vmstorage.Init()
 	vmselect.Init()
--- a/app/vmbackup/Makefile
+++ b/app/vmbackup/Makefile
@@ -9,7 +9,10 @@ vmbackup-prod:
 vmbackup-pure-prod:
 	APP_NAME=vmbackup $(MAKE) app-via-docker-pure

-vmbackup--arm-prod:
+vmbackup-amd64-prod:
+	APP_NAME=vmbackup $(MAKE) app-via-docker-amd64
+
+vmbackup-arm-prod:
 	APP_NAME=vmbackup $(MAKE) app-via-docker-arm

 vmbackup-arm64-prod:
@@ -27,6 +30,9 @@ package-vmbackup:
 package-vmbackup-pure:
 	APP_NAME=vmbackup $(MAKE) package-via-docker-pure

+package-vmbackup-amd64:
+	APP_NAME=vmbackup $(MAKE) package-via-docker-amd64
+
 package-vmbackup-arm:
 	APP_NAME=vmbackup $(MAKE) package-via-docker-arm

@@ -45,6 +51,9 @@ publish-vmbackup:
 vmbackup-pure:
 	APP_NAME=vmbackup $(MAKE) app-local-pure

+vmbackup-amd64:
+	CGO_ENABLED=1 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -mod=vendor -ldflags "$(GO_BUILDINFO)" -o bin/vmbackup-amd64 ./app/vmbackup
+
 vmbackup-arm:
 	CGO_ENABLED=0 GOOS=linux GOARCH=arm GO111MODULE=on go build -mod=vendor -ldflags "$(GO_BUILDINFO)" -o bin/vmbackup-arm ./app/vmbackup

--- a/app/vminsert/main.go
+++ b/app/vminsert/main.go
@@ -83,6 +83,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 			httpserver.Errorf(w, "error in %q: %s", r.URL.Path, err)
 			return true
 		}
+		w.WriteHeader(http.StatusNoContent)
 		return true
 	case "/write", "/api/v2/write":
 		influxWriteRequests.Inc()
--- a/app/vminsert/vmimport/parser_test.go
+++ b/app/vminsert/vmimport/parser_test.go
@@ -1,6 +1,7 @@
 package vmimport

 import (
+	"math"
 	"reflect"
 	"testing"
 )
@@ -101,6 +102,18 @@ func TestRowsUnmarshalSuccess(t *testing.T) {
 		}},
 	})

+	// Inf and nan values
+	f(`{"metric":{"foo":"bar"},"values":[Inf, -Inf],"timestamps":[456, 789]}`, &Rows{
+		Rows: []Row{{
+			Tags: []Tag{{
+				Key:   []byte("foo"),
+				Value: []byte("bar"),
+			}},
+			Values:     []float64{math.Inf(1), math.Inf(-1)},
+			Timestamps: []int64{456, 789},
+		}},
+	})
+
 	// Line with multiple tags
 	f(`{"metric":{"foo":"bar","baz":"xx"},"values":[1.23, -3.21],"timestamps" : [456,789]}`, &Rows{
 		Rows: []Row{{
--- a/app/vmrestore/Makefile
+++ b/app/vmrestore/Makefile
@@ -9,7 +9,10 @@ vmrestore-prod:
 vmrestore-pure-prod:
 	APP_NAME=vmrestore $(MAKE) app-via-docker-pure

-vmrestore--arm-prod:
+vmrestore-amd64-prod:
+	APP_NAME=vmrestore $(MAKE) app-via-docker-amd64
+
+vmrestore-arm-prod:
 	APP_NAME=vmrestore $(MAKE) app-via-docker-arm

 vmrestore-arm64-prod:
@@ -27,6 +30,9 @@ package-vmrestore:
 package-vmrestore-pure:
 	APP_NAME=vmrestore $(MAKE) package-via-docker-pure

+package-vmrestore-amd64:
+	APP_NAME=vmrestore $(MAKE) package-via-docker-amd64
+
 package-vmrestore-arm:
 	APP_NAME=vmrestore $(MAKE) package-via-docker-arm

@@ -45,6 +51,9 @@ publish-vmrestore:
 vmrestore-pure:
 	APP_NAME=vmrestore $(MAKE) app-local-pure

+vmrestore-amd64:
+	CGO_ENABLED=1 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -mod=vendor -ldflags "$(GO_BUILDINFO)" -o bin/vmrestore-amd64 ./app/vmrestore
+
 vmrestore-arm:
 	CGO_ENABLED=0 GOOS=linux GOARCH=arm GO111MODULE=on go build -mod=vendor -ldflags "$(GO_BUILDINFO)" -o bin/vmrestore-arm ./app/vmrestore

--- a/app/vmrestore/main.go
+++ b/app/vmrestore/main.go
@@ -16,8 +16,9 @@ var (
 		"Example: gcs://bucket/path/to/backup/dir, s3://bucket/path/to/backup/dir or fs:///path/to/local/backup/dir")
 	storageDataPath = flag.String("storageDataPath", "victoria-metrics-data", "Destination path where backup must be restored. "+
 		"VictoriaMetrics must be stopped when restoring from backup. -storageDataPath dir can be non-empty. In this case only missing data is downloaded from backup")
-	concurrency       = flag.Int("concurrency", 10, "The number of concurrent workers. Higher concurrency may reduce restore duration")
-	maxBytesPerSecond = flag.Int("maxBytesPerSecond", 0, "The maximum download speed. There is no limit if it is set to 0")
+	concurrency             = flag.Int("concurrency", 10, "The number of concurrent workers. Higher concurrency may reduce restore duration")
+	maxBytesPerSecond       = flag.Int("maxBytesPerSecond", 0, "The maximum download speed. There is no limit if it is set to 0")
+	skipBackupCompleteCheck = flag.Bool("skipBackupCompleteCheck", false, "Whether to skip checking for `backup complete` file in `-src`. This may be useful for restoring from old backups, which were created without `backup complete` file")
 )

 func main() {
@@ -34,9 +35,10 @@ func main() {
 		logger.Fatalf("%s", err)
 	}
 	a := &actions.Restore{
-		Concurrency: *concurrency,
-		Src:         srcFS,
-		Dst:         dstFS,
+		Concurrency:             *concurrency,
+		Src:                     srcFS,
+		Dst:                     dstFS,
+		SkipBackupCompleteCheck: *skipBackupCompleteCheck,
 	}
 	if err := a.Run(); err != nil {
 		logger.Fatalf("cannot restore from backup: %s", err)
--- a/app/vmselect/netstorage/fadvise_darwin.go
+++ b/app/vmselect/netstorage/fadvise_darwin.go
@@ -1,9 +0,0 @@
-package netstorage
-
-import (
-	"os"
-)
-
-func mustFadviseSequentialRead(f *os.File) {
-	// Do nothing :)
-}
--- a/app/vmselect/netstorage/fadvise_freebsd.go
+++ b/app/vmselect/netstorage/fadvise_freebsd.go
@@ -1,15 +0,0 @@
-package netstorage
-
-import (
-	"os"
-
-	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
-	"golang.org/x/sys/unix"
-)
-
-func mustFadviseSequentialRead(f *os.File) {
-	fd := int(f.Fd())
-	if err := unix.Fadvise(int(fd), 0, 0, unix.FADV_SEQUENTIAL|unix.FADV_WILLNEED); err != nil {
-		logger.Panicf("FATAL: error returned from unix.Fadvise(SEQUENTIAL|WILLNEED): %s", err)
-	}
-}
--- a/app/vmselect/netstorage/fadvise_linux.go
+++ b/app/vmselect/netstorage/fadvise_linux.go
@@ -1,15 +0,0 @@
-package netstorage
-
-import (
-	"os"
-
-	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
-	"golang.org/x/sys/unix"
-)
-
-func mustFadviseSequentialRead(f *os.File) {
-	fd := int(f.Fd())
-	if err := unix.Fadvise(int(fd), 0, 0, unix.FADV_SEQUENTIAL|unix.FADV_WILLNEED); err != nil {
-		logger.Panicf("FATAL: error returned from unix.Fadvise(SEQUENTIAL|WILLNEED): %s", err)
-	}
-}
--- a/app/vmselect/netstorage/tmp_blocks_file.go
+++ b/app/vmselect/netstorage/tmp_blocks_file.go
@@ -128,7 +128,7 @@ func (tbf *tmpBlocksFile) Finalize() error {
 	// Hint the OS that the file is read almost sequentiallly.
 	// This should reduce the number of disk seeks, which is important
 	// for HDDs.
-	mustFadviseSequentialRead(tbf.f)
+	fs.MustFadviseSequentialRead(tbf.f, true)
 	return nil
 }

--- a/app/vmselect/prometheus/prometheus.go
+++ b/app/vmselect/prometheus/prometheus.go
@@ -15,6 +15,7 @@ import (
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/netstorage"
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/promql"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
 	"github.com/VictoriaMetrics/metrics"
 	"github.com/valyala/quicktemplate"
@@ -23,9 +24,10 @@ import (
 var (
 	latencyOffset = flag.Duration("search.latencyOffset", time.Second*30, "The time when data points become visible in query results after the colection. "+
 		"Too small value can result in incomplete last points for query results")
-	maxQueryDuration = flag.Duration("search.maxQueryDuration", time.Second*30, "The maximum time for search query execution")
-	maxQueryLen      = flag.Int("search.maxQueryLen", 16*1024, "The maximum search query length in bytes")
-	maxLookback      = flag.Duration("search.maxLookback", 0, "Synonim to `-search.lookback-delta` from Prometheus. "+
+	maxExportDuration = flag.Duration("search.maxExportDuration", 10*time.Minute, "The maximum duration for `/api/v1/export` call")
+	maxQueryDuration  = flag.Duration("search.maxQueryDuration", time.Second*30, "The maximum duration for search query execution")
+	maxQueryLen       = flag.Int("search.maxQueryLen", 16*1024, "The maximum search query length in bytes")
+	maxLookback       = flag.Duration("search.maxLookback", 0, "Synonim to `-search.lookback-delta` from Prometheus. "+
 		"The value is dynamically detected from interval between time series datapoints if not set. It can be overridden on per-query basis via `max_lookback` arg")
 )

@@ -58,7 +60,7 @@ func FederateHandler(w http.ResponseWriter, r *http.Request) error {
 	if err != nil {
 		return err
 	}
-	deadline := getDeadline(r)
+	deadline := getDeadlineForQuery(r)
 	if start >= end {
 		start = end - defaultStep
 	}
@@ -129,7 +131,7 @@ func ExportHandler(w http.ResponseWriter, r *http.Request) error {
 		return err
 	}
 	format := r.FormValue("format")
-	deadline := getDeadline(r)
+	deadline := getDeadlineForExport(r)
 	if start >= end {
 		end = start + defaultStep
 	}
@@ -235,7 +237,7 @@ var deleteDuration = metrics.NewSummary(`vm_request_duration_seconds{path="/api/
 // See https://prometheus.io/docs/prometheus/latest/querying/api/#querying-label-values
 func LabelValuesHandler(labelName string, w http.ResponseWriter, r *http.Request) error {
 	startTime := time.Now()
-	deadline := getDeadline(r)
+	deadline := getDeadlineForQuery(r)

 	if err := r.ParseForm(); err != nil {
 		return fmt.Errorf("cannot parse form values: %s", err)
@@ -333,7 +335,7 @@ var labelValuesDuration = metrics.NewSummary(`vm_request_duration_seconds{path="
 // LabelsCountHandler processes /api/v1/labels/count request.
 func LabelsCountHandler(w http.ResponseWriter, r *http.Request) error {
 	startTime := time.Now()
-	deadline := getDeadline(r)
+	deadline := getDeadlineForQuery(r)
 	labelEntries, err := netstorage.GetLabelEntries(deadline)
 	if err != nil {
 		return fmt.Errorf(`cannot obtain label entries: %s`, err)
@@ -352,7 +354,7 @@ var labelsCountDuration = metrics.NewSummary(`vm_request_duration_seconds{path="
 // See https://prometheus.io/docs/prometheus/latest/querying/api/#getting-label-names
 func LabelsHandler(w http.ResponseWriter, r *http.Request) error {
 	startTime := time.Now()
-	deadline := getDeadline(r)
+	deadline := getDeadlineForQuery(r)

 	if err := r.ParseForm(); err != nil {
 		return fmt.Errorf("cannot parse form values: %s", err)
@@ -442,7 +444,7 @@ var labelsDuration = metrics.NewSummary(`vm_request_duration_seconds{path="/api/
 // SeriesCountHandler processes /api/v1/series/count request.
 func SeriesCountHandler(w http.ResponseWriter, r *http.Request) error {
 	startTime := time.Now()
-	deadline := getDeadline(r)
+	deadline := getDeadlineForQuery(r)
 	n, err := netstorage.GetSeriesCount(deadline)
 	if err != nil {
 		return fmt.Errorf("cannot obtain series count: %s", err)
@@ -482,7 +484,7 @@ func SeriesHandler(w http.ResponseWriter, r *http.Request) error {
 	if err != nil {
 		return err
 	}
-	deadline := getDeadline(r)
+	deadline := getDeadlineForQuery(r)

 	tagFilterss, err := getTagFilterssFromMatches(matches)
 	if err != nil {
@@ -551,7 +553,7 @@ func QueryHandler(w http.ResponseWriter, r *http.Request) error {
 	if err != nil {
 		return err
 	}
-	deadline := getDeadline(r)
+	deadline := getDeadlineForQuery(r)
 	lookbackDelta, err := getMaxLookback(r)
 	if err != nil {
 		return err
@@ -633,14 +635,14 @@ func parseDuration(s string, step int64) (int64, error) {
 	if len(s) == 0 {
 		return 0, nil
 	}
-	return promql.DurationValue(s, step)
+	return metricsql.DurationValue(s, step)
 }

 func parsePositiveDuration(s string, step int64) (int64, error) {
 	if len(s) == 0 {
 		return 0, nil
 	}
-	return promql.PositiveDurationValue(s, step)
+	return metricsql.PositiveDurationValue(s, step)
 }

 // QueryRangeHandler processes /api/v1/query_range request.
@@ -674,7 +676,7 @@ func QueryRangeHandler(w http.ResponseWriter, r *http.Request) error {
 }

 func queryRangeHandler(w http.ResponseWriter, query string, start, end, step int64, r *http.Request, ct int64) error {
-	deadline := getDeadline(r)
+	deadline := getDeadlineForQuery(r)
 	mayCache := !getBool(r, "nocache")
 	lookbackDelta, err := getMaxLookback(r)
 	if err != nil {
@@ -866,12 +868,21 @@ func getMaxLookback(r *http.Request) (int64, error) {
 	return getDuration(r, "max_lookback", d)
 }

-func getDeadline(r *http.Request) netstorage.Deadline {
+func getDeadlineForQuery(r *http.Request) netstorage.Deadline {
+	dMax := int64(maxQueryDuration.Seconds() * 1e3)
+	return getDeadlineWithMaxDuration(r, dMax)
+}
+
+func getDeadlineForExport(r *http.Request) netstorage.Deadline {
+	dMax := int64(maxExportDuration.Seconds() * 1e3)
+	return getDeadlineWithMaxDuration(r, dMax)
+}
+
+func getDeadlineWithMaxDuration(r *http.Request, dMax int64) netstorage.Deadline {
 	d, err := getDuration(r, "timeout", 0)
 	if err != nil {
 		d = 0
 	}
-	dMax := int64(maxQueryDuration.Seconds() * 1e3)
 	if d <= 0 || d > dMax {
 		d = dMax
 	}
--- a/app/vmselect/promql/aggr.go
+++ b/app/vmselect/promql/aggr.go
@@ -8,6 +8,7 @@ import (
 	"strings"

 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
 	"github.com/VictoriaMetrics/metrics"
 	"github.com/valyala/histogram"
@@ -48,7 +49,7 @@ type aggrFunc func(afa *aggrFuncArg) ([]*timeseries, error)

 type aggrFuncArg struct {
 	args [][]*timeseries
-	ae   *aggrFuncExpr
+	ae   *metricsql.AggrFuncExpr
 	ec   *EvalConfig
 }

@@ -57,20 +58,6 @@ func getAggrFunc(s string) aggrFunc {
 	return aggrFuncs[s]
 }

-func isAggrFunc(s string) bool {
-	return getAggrFunc(s) != nil
-}
-
-func isAggrFuncModifier(s string) bool {
-	s = strings.ToLower(s)
-	switch s {
-	case "by", "without":
-		return true
-	default:
-		return false
-	}
-}
-
 func newAggrFunc(afe func(tss []*timeseries) []*timeseries) aggrFunc {
 	return func(afa *aggrFuncArg) ([]*timeseries, error) {
 		args := afa.args
@@ -81,7 +68,7 @@ func newAggrFunc(afe func(tss []*timeseries) []*timeseries) aggrFunc {
 	}
 }

-func removeGroupTags(metricName *storage.MetricName, modifier *modifierExpr) {
+func removeGroupTags(metricName *storage.MetricName, modifier *metricsql.ModifierExpr) {
 	groupOp := strings.ToLower(modifier.Op)
 	switch groupOp {
 	case "", "by":
@@ -93,7 +80,7 @@ func removeGroupTags(metricName *storage.MetricName, modifier *modifierExpr) {
 	}
 }

-func aggrFuncExt(afe func(tss []*timeseries) []*timeseries, argOrig []*timeseries, modifier *modifierExpr, keepOriginal bool) ([]*timeseries, error) {
+func aggrFuncExt(afe func(tss []*timeseries) []*timeseries, argOrig []*timeseries, modifier *metricsql.ModifierExpr, keepOriginal bool) ([]*timeseries, error) {
 	arg := copyTimeseriesMetricNames(argOrig)

 	// Perform grouping.
--- a/app/vmselect/promql/aggr_incremental.go
+++ b/app/vmselect/promql/aggr_incremental.go
@@ -4,10 +4,12 @@ import (
 	"math"
 	"strings"
 	"sync"
+
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql"
 )

 // callbacks for optimized incremental calculations for aggregate functions
-// over rollups over metricExpr.
+// over rollups over metricsql.MetricExpr.
 //
 // These calculations save RAM for aggregates over big number of time series.
 var incrementalAggrFuncCallbacksMap = map[string]*incrementalAggrFuncCallbacks{
@@ -49,7 +51,7 @@ var incrementalAggrFuncCallbacksMap = map[string]*incrementalAggrFuncCallbacks{
 }

 type incrementalAggrFuncContext struct {
-	ae *aggrFuncExpr
+	ae *metricsql.AggrFuncExpr

 	mLock sync.Mutex
 	m     map[uint]map[string]*incrementalAggrContext
@@ -57,7 +59,7 @@ type incrementalAggrFuncContext struct {
 	callbacks *incrementalAggrFuncCallbacks
 }

-func newIncrementalAggrFuncContext(ae *aggrFuncExpr, callbacks *incrementalAggrFuncCallbacks) *incrementalAggrFuncContext {
+func newIncrementalAggrFuncContext(ae *metricsql.AggrFuncExpr, callbacks *incrementalAggrFuncCallbacks) *incrementalAggrFuncContext {
 	return &incrementalAggrFuncContext{
 		ae:        ae,
 		m:         make(map[uint]map[string]*incrementalAggrContext),
--- a/app/vmselect/promql/aggr_incremental_test.go
+++ b/app/vmselect/promql/aggr_incremental_test.go
@@ -7,6 +7,8 @@ import (
 	"runtime"
 	"sync"
 	"testing"
+
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql"
 )

 func TestIncrementalAggr(t *testing.T) {
@@ -42,7 +44,7 @@ func TestIncrementalAggr(t *testing.T) {
 	f := func(name string, valuesExpected []float64) {
 		t.Helper()
 		callbacks := getIncrementalAggrFuncCallbacks(name)
-		ae := &aggrFuncExpr{
+		ae := &metricsql.AggrFuncExpr{
 			Name: name,
 		}
 		tssExpected := []*timeseries{{
--- a/app/vmselect/promql/binary_op.go
+++ b/app/vmselect/promql/binary_op.go
@@ -6,24 +6,26 @@ import (
 	"strings"

 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql/binaryop"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
 )

 var binaryOpFuncs = map[string]binaryOpFunc{
-	"+": newBinaryOpArithFunc(binaryOpPlus),
-	"-": newBinaryOpArithFunc(binaryOpMinus),
-	"*": newBinaryOpArithFunc(binaryOpMul),
-	"/": newBinaryOpArithFunc(binaryOpDiv),
-	"%": newBinaryOpArithFunc(binaryOpMod),
-	"^": newBinaryOpArithFunc(binaryOpPow),
+	"+": newBinaryOpArithFunc(binaryop.Plus),
+	"-": newBinaryOpArithFunc(binaryop.Minus),
+	"*": newBinaryOpArithFunc(binaryop.Mul),
+	"/": newBinaryOpArithFunc(binaryop.Div),
+	"%": newBinaryOpArithFunc(binaryop.Mod),
+	"^": newBinaryOpArithFunc(binaryop.Pow),

 	// cmp ops
-	"==": newBinaryOpCmpFunc(binaryOpEq),
-	"!=": newBinaryOpCmpFunc(binaryOpNeq),
-	">":  newBinaryOpCmpFunc(binaryOpGt),
-	"<":  newBinaryOpCmpFunc(binaryOpLt),
-	">=": newBinaryOpCmpFunc(binaryOpGte),
-	"<=": newBinaryOpCmpFunc(binaryOpLte),
+	"==": newBinaryOpCmpFunc(binaryop.Eq),
+	"!=": newBinaryOpCmpFunc(binaryop.Neq),
+	">":  newBinaryOpCmpFunc(binaryop.Gt),
+	"<":  newBinaryOpCmpFunc(binaryop.Lt),
+	">=": newBinaryOpCmpFunc(binaryop.Gte),
+	"<=": newBinaryOpCmpFunc(binaryop.Lte),

 	// logical set ops
 	"and":    binaryOpAnd,
@@ -31,38 +33,9 @@ var binaryOpFuncs = map[string]binaryOpFunc{
 	"unless": binaryOpUnless,

 	// New op
-	"if":      newBinaryOpArithFunc(binaryOpIf),
-	"ifnot":   newBinaryOpArithFunc(binaryOpIfnot),
-	"default": newBinaryOpArithFunc(binaryOpDefault),
-}
-
-var binaryOpPriorities = map[string]int{
-	"default": -1,
-
-	"if":    0,
-	"ifnot": 0,
-
-	// See https://prometheus.io/docs/prometheus/latest/querying/operators/#binary-operator-precedence
-	"or": 1,
-
-	"and":    2,
-	"unless": 2,
-
-	"==": 3,
-	"!=": 3,
-	"<":  3,
-	">":  3,
-	"<=": 3,
-	">=": 3,
-
-	"+": 4,
-	"-": 4,
-
-	"*": 5,
-	"/": 5,
-	"%": 5,
-
-	"^": 6,
+	"if":      newBinaryOpArithFunc(binaryop.If),
+	"ifnot":   newBinaryOpArithFunc(binaryop.Ifnot),
+	"default": newBinaryOpArithFunc(binaryop.Default),
 }

 func getBinaryOpFunc(op string) binaryOpFunc {
@@ -70,144 +43,8 @@ func getBinaryOpFunc(op string) binaryOpFunc {
 	return binaryOpFuncs[op]
 }

-func isBinaryOp(op string) bool {
-	return getBinaryOpFunc(op) != nil
-}
-
-func binaryOpPriority(op string) int {
-	op = strings.ToLower(op)
-	return binaryOpPriorities[op]
-}
-
-func scanBinaryOpPrefix(s string) int {
-	n := 0
-	for op := range binaryOpFuncs {
-		if len(s) < len(op) {
-			continue
-		}
-		ss := strings.ToLower(s[:len(op)])
-		if ss == op && len(op) > n {
-			n = len(op)
-		}
-	}
-	return n
-}
-
-func isRightAssociativeBinaryOp(op string) bool {
-	// See https://prometheus.io/docs/prometheus/latest/querying/operators/#binary-operator-precedence
-	return op == "^"
-}
-
-func isBinaryOpGroupModifier(s string) bool {
-	s = strings.ToLower(s)
-	switch s {
-	// See https://prometheus.io/docs/prometheus/latest/querying/operators/#vector-matching
-	case "on", "ignoring":
-		return true
-	default:
-		return false
-	}
-}
-
-func isBinaryOpJoinModifier(s string) bool {
-	s = strings.ToLower(s)
-	switch s {
-	case "group_left", "group_right":
-		return true
-	default:
-		return false
-	}
-}
-
-func isBinaryOpBoolModifier(s string) bool {
-	s = strings.ToLower(s)
-	return s == "bool"
-}
-
-func isBinaryOpCmp(op string) bool {
-	switch op {
-	case "==", "!=", ">", "<", ">=", "<=":
-		return true
-	default:
-		return false
-	}
-}
-
-func isBinaryOpLogicalSet(op string) bool {
-	op = strings.ToLower(op)
-	switch op {
-	case "and", "or", "unless":
-		return true
-	default:
-		return false
-	}
-}
-
-func binaryOpConstants(op string, left, right float64, isBool bool) float64 {
-	if isBinaryOpCmp(op) {
-		evalCmp := func(cf func(left, right float64) bool) float64 {
-			if isBool {
-				if cf(left, right) {
-					return 1
-				}
-				return 0
-			}
-			if cf(left, right) {
-				return left
-			}
-			return nan
-		}
-		switch op {
-		case "==":
-			left = evalCmp(binaryOpEq)
-		case "!=":
-			left = evalCmp(binaryOpNeq)
-		case ">":
-			left = evalCmp(binaryOpGt)
-		case "<":
-			left = evalCmp(binaryOpLt)
-		case ">=":
-			left = evalCmp(binaryOpGte)
-		case "<=":
-			left = evalCmp(binaryOpLte)
-		default:
-			logger.Panicf("BUG: unexpected comparison binaryOp: %q", op)
-		}
-	} else {
-		switch op {
-		case "+":
-			left = binaryOpPlus(left, right)
-		case "-":
-			left = binaryOpMinus(left, right)
-		case "*":
-			left = binaryOpMul(left, right)
-		case "/":
-			left = binaryOpDiv(left, right)
-		case "%":
-			left = binaryOpMod(left, right)
-		case "^":
-			left = binaryOpPow(left, right)
-		case "and":
-			// Nothing to do
-		case "or":
-			// Nothing to do
-		case "unless":
-			left = nan
-		case "default":
-			left = binaryOpDefault(left, right)
-		case "if":
-			left = binaryOpIf(left, right)
-		case "ifnot":
-			left = binaryOpIfnot(left, right)
-		default:
-			logger.Panicf("BUG: unexpected non-comparison binaryOp: %q", op)
-		}
-	}
-	return left
-}
-
 type binaryOpFuncArg struct {
-	be    *binaryOpExpr
+	be    *metricsql.BinaryOpExpr
 	left  []*timeseries
 	right []*timeseries
 }
@@ -267,7 +104,7 @@ func newBinaryOpFunc(bf func(left, right float64, isBool bool) float64) binaryOp
 	}
 }

-func adjustBinaryOpTags(be *binaryOpExpr, left, right []*timeseries) ([]*timeseries, []*timeseries, []*timeseries, error) {
+func adjustBinaryOpTags(be *metricsql.BinaryOpExpr, left, right []*timeseries) ([]*timeseries, []*timeseries, []*timeseries, error) {
 	if len(be.GroupModifier.Op) == 0 && len(be.JoinModifier.Op) == 0 {
 		if isScalar(left) {
 			// Fast path: `scalar op vector`
@@ -348,7 +185,7 @@ func adjustBinaryOpTags(be *binaryOpExpr, left, right []*timeseries) ([]*timeser
 	return rvsLeft, rvsRight, dst, nil
 }

-func ensureSingleTimeseries(side string, be *binaryOpExpr, tss []*timeseries) error {
+func ensureSingleTimeseries(side string, be *metricsql.BinaryOpExpr, tss []*timeseries) error {
 	if len(tss) == 0 {
 		logger.Panicf("BUG: tss must contain at least one value")
 	}
@@ -362,7 +199,7 @@ func ensureSingleTimeseries(side string, be *binaryOpExpr, tss []*timeseries) er
 	return nil
 }

-func groupJoin(singleTimeseriesSide string, be *binaryOpExpr, rvsLeft, rvsRight, tssLeft, tssRight []*timeseries) ([]*timeseries, []*timeseries, error) {
+func groupJoin(singleTimeseriesSide string, be *metricsql.BinaryOpExpr, rvsLeft, rvsRight, tssLeft, tssRight []*timeseries) ([]*timeseries, []*timeseries, error) {
 	joinTags := be.JoinModifier.Args
 	var m map[string]*timeseries
 	for _, tsLeft := range tssLeft {
@@ -432,8 +269,8 @@ func mergeNonOverlappingTimeseries(dst, src *timeseries) bool {
 	return true
 }

-func resetMetricGroupIfRequired(be *binaryOpExpr, ts *timeseries) {
-	if isBinaryOpCmp(be.Op) && !be.Bool {
+func resetMetricGroupIfRequired(be *metricsql.BinaryOpExpr, ts *timeseries) {
+	if metricsql.IsBinaryOpCmp(be.Op) && !be.Bool {
 		// Do not reset MetricGroup for non-boolean `compare` binary ops like Prometheus does.
 		return
 	}
@@ -445,90 +282,6 @@ func resetMetricGroupIfRequired(be *binaryOpExpr, ts *timeseries) {
 	ts.MetricName.ResetMetricGroup()
 }

-func binaryOpPlus(left, right float64) float64 {
-	return left + right
-}
-
-func binaryOpMinus(left, right float64) float64 {
-	return left - right
-}
-
-func binaryOpMul(left, right float64) float64 {
-	return left * right
-}
-
-func binaryOpDiv(left, right float64) float64 {
-	return left / right
-}
-
-func binaryOpMod(left, right float64) float64 {
-	return math.Mod(left, right)
-}
-
-func binaryOpPow(left, right float64) float64 {
-	return math.Pow(left, right)
-}
-
-func binaryOpDefault(left, right float64) float64 {
-	if math.IsNaN(left) {
-		return right
-	}
-	return left
-}
-
-func binaryOpIf(left, right float64) float64 {
-	if math.IsNaN(right) {
-		return nan
-	}
-	return left
-}
-
-func binaryOpIfnot(left, right float64) float64 {
-	if math.IsNaN(right) {
-		return left
-	}
-	return nan
-}
-
-func binaryOpEq(left, right float64) bool {
-	// Special handling for nan == nan.
-	// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150 .
-	if math.IsNaN(left) {
-		return math.IsNaN(right)
-	}
-
-	return left == right
-}
-
-func binaryOpNeq(left, right float64) bool {
-	// Special handling for comparison with nan.
-	// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150 .
-	if math.IsNaN(left) {
-		return !math.IsNaN(right)
-	}
-	if math.IsNaN(right) {
-		return true
-	}
-
-	return left != right
-}
-
-func binaryOpGt(left, right float64) bool {
-	return left > right
-}
-
-func binaryOpLt(left, right float64) bool {
-	return left < right
-}
-
-func binaryOpGte(left, right float64) bool {
-	return left >= right
-}
-
-func binaryOpLte(left, right float64) bool {
-	return left <= right
-}
-
 func binaryOpAnd(bfa *binaryOpFuncArg) ([]*timeseries, error) {
 	mLeft, mRight := createTimeseriesMapByTagSet(bfa.be, bfa.left, bfa.right)
 	var rvs []*timeseries
@@ -565,7 +318,7 @@ func binaryOpUnless(bfa *binaryOpFuncArg) ([]*timeseries, error) {
 	return rvs, nil
 }

-func createTimeseriesMapByTagSet(be *binaryOpExpr, left, right []*timeseries) (map[string][]*timeseries, map[string][]*timeseries) {
+func createTimeseriesMapByTagSet(be *metricsql.BinaryOpExpr, left, right []*timeseries) (map[string][]*timeseries, map[string][]*timeseries) {
 	groupTags := be.GroupModifier.Args
 	groupOp := strings.ToLower(be.GroupModifier.Op)
 	if len(groupOp) == 0 {
--- a/app/vmselect/promql/eval.go
+++ b/app/vmselect/promql/eval.go
@@ -11,6 +11,7 @@ import (
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/memory"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
 	"github.com/VictoriaMetrics/metrics"
 )
@@ -57,6 +58,14 @@ func AdjustStartEnd(start, end, step int64) (int64, int64) {
 	if adjust > 0 {
 		end += step - adjust
 	}
+
+	// Make sure that the new number of points is the same as the initial number of points.
+	newPoints := (end-start)/step + 1
+	for newPoints > points {
+		end -= step
+		newPoints--
+	}
+
 	return start, end
 }

@@ -144,25 +153,25 @@ func getTimestamps(start, end, step int64) []int64 {
 	return timestamps
 }

-func evalExpr(ec *EvalConfig, e expr) ([]*timeseries, error) {
-	if me, ok := e.(*metricExpr); ok {
-		re := &rollupExpr{
+func evalExpr(ec *EvalConfig, e metricsql.Expr) ([]*timeseries, error) {
+	if me, ok := e.(*metricsql.MetricExpr); ok {
+		re := &metricsql.RollupExpr{
 			Expr: me,
 		}
-		rv, err := evalRollupFunc(ec, "default_rollup", rollupDefault, re, nil)
+		rv, err := evalRollupFunc(ec, "default_rollup", rollupDefault, e, re, nil)
 		if err != nil {
 			return nil, fmt.Errorf(`cannot evaluate %q: %s`, me.AppendString(nil), err)
 		}
 		return rv, nil
 	}
-	if re, ok := e.(*rollupExpr); ok {
-		rv, err := evalRollupFunc(ec, "default_rollup", rollupDefault, re, nil)
+	if re, ok := e.(*metricsql.RollupExpr); ok {
+		rv, err := evalRollupFunc(ec, "default_rollup", rollupDefault, e, re, nil)
 		if err != nil {
 			return nil, fmt.Errorf(`cannot evaluate %q: %s`, re.AppendString(nil), err)
 		}
 		return rv, nil
 	}
-	if fe, ok := e.(*funcExpr); ok {
+	if fe, ok := e.(*metricsql.FuncExpr); ok {
 		nrf := getRollupFunc(fe.Name)
 		if nrf == nil {
 			args, err := evalExprs(ec, fe.Args)
@@ -192,17 +201,17 @@ func evalExpr(ec *EvalConfig, e expr) ([]*timeseries, error) {
 		if err != nil {
 			return nil, err
 		}
-		rv, err := evalRollupFunc(ec, fe.Name, rf, re, nil)
+		rv, err := evalRollupFunc(ec, fe.Name, rf, e, re, nil)
 		if err != nil {
 			return nil, fmt.Errorf(`cannot evaluate %q: %s`, fe.AppendString(nil), err)
 		}
 		return rv, nil
 	}
-	if ae, ok := e.(*aggrFuncExpr); ok {
+	if ae, ok := e.(*metricsql.AggrFuncExpr); ok {
 		if callbacks := getIncrementalAggrFuncCallbacks(ae.Name); callbacks != nil {
 			fe, nrf := tryGetArgRollupFuncWithMetricExpr(ae)
 			if fe != nil {
-				// There is an optimized path for calculating aggrFuncExpr over rollupFunc over metricExpr.
+				// There is an optimized path for calculating metricsql.AggrFuncExpr over rollupFunc over metricsql.MetricExpr.
 				// The optimized path saves RAM for aggregates over big number of time series.
 				args, re, err := evalRollupFuncArgs(ec, fe)
 				if err != nil {
@@ -213,7 +222,7 @@ func evalExpr(ec *EvalConfig, e expr) ([]*timeseries, error) {
 					return nil, err
 				}
 				iafc := newIncrementalAggrFuncContext(ae, callbacks)
-				return evalRollupFunc(ec, fe.Name, rf, re, iafc)
+				return evalRollupFunc(ec, fe.Name, rf, e, re, iafc)
 			}
 		}
 		args, err := evalExprs(ec, ae.Args)
@@ -235,7 +244,7 @@ func evalExpr(ec *EvalConfig, e expr) ([]*timeseries, error) {
 		}
 		return rv, nil
 	}
-	if be, ok := e.(*binaryOpExpr); ok {
+	if be, ok := e.(*metricsql.BinaryOpExpr); ok {
 		left, err := evalExpr(ec, be.Left)
 		if err != nil {
 			return nil, err
@@ -259,18 +268,18 @@ func evalExpr(ec *EvalConfig, e expr) ([]*timeseries, error) {
 		}
 		return rv, nil
 	}
-	if ne, ok := e.(*numberExpr); ok {
+	if ne, ok := e.(*metricsql.NumberExpr); ok {
 		rv := evalNumber(ec, ne.N)
 		return rv, nil
 	}
-	if se, ok := e.(*stringExpr); ok {
+	if se, ok := e.(*metricsql.StringExpr); ok {
 		rv := evalString(ec, se.S)
 		return rv, nil
 	}
 	return nil, fmt.Errorf("unexpected expression %q", e.AppendString(nil))
 }

-func tryGetArgRollupFuncWithMetricExpr(ae *aggrFuncExpr) (*funcExpr, newRollupFunc) {
+func tryGetArgRollupFuncWithMetricExpr(ae *metricsql.AggrFuncExpr) (*metricsql.FuncExpr, newRollupFunc) {
 	if len(ae.Args) != 1 {
 		return nil, nil
 	}
@@ -281,31 +290,31 @@ func tryGetArgRollupFuncWithMetricExpr(ae *aggrFuncExpr) (*funcExpr, newRollupFu
 	// - rollupFunc(metricExpr)
 	// - rollupFunc(metricExpr[d])

-	if me, ok := e.(*metricExpr); ok {
+	if me, ok := e.(*metricsql.MetricExpr); ok {
 		// e = metricExpr
 		if me.IsEmpty() {
 			return nil, nil
 		}
-		fe := &funcExpr{
+		fe := &metricsql.FuncExpr{
 			Name: "default_rollup",
-			Args: []expr{me},
+			Args: []metricsql.Expr{me},
 		}
 		nrf := getRollupFunc(fe.Name)
 		return fe, nrf
 	}
-	if re, ok := e.(*rollupExpr); ok {
-		if me, ok := re.Expr.(*metricExpr); !ok || me.IsEmpty() || re.ForSubquery() {
+	if re, ok := e.(*metricsql.RollupExpr); ok {
+		if me, ok := re.Expr.(*metricsql.MetricExpr); !ok || me.IsEmpty() || re.ForSubquery() {
 			return nil, nil
 		}
 		// e = metricExpr[d]
-		fe := &funcExpr{
+		fe := &metricsql.FuncExpr{
 			Name: "default_rollup",
-			Args: []expr{re},
+			Args: []metricsql.Expr{re},
 		}
 		nrf := getRollupFunc(fe.Name)
 		return fe, nrf
 	}
-	fe, ok := e.(*funcExpr)
+	fe, ok := e.(*metricsql.FuncExpr)
 	if !ok {
 		return nil, nil
 	}
@@ -315,18 +324,18 @@ func tryGetArgRollupFuncWithMetricExpr(ae *aggrFuncExpr) (*funcExpr, newRollupFu
 	}
 	rollupArgIdx := getRollupArgIdx(fe.Name)
 	arg := fe.Args[rollupArgIdx]
-	if me, ok := arg.(*metricExpr); ok {
+	if me, ok := arg.(*metricsql.MetricExpr); ok {
 		if me.IsEmpty() {
 			return nil, nil
 		}
 		// e = rollupFunc(metricExpr)
-		return &funcExpr{
+		return &metricsql.FuncExpr{
 			Name: fe.Name,
-			Args: []expr{me},
+			Args: []metricsql.Expr{me},
 		}, nrf
 	}
-	if re, ok := arg.(*rollupExpr); ok {
-		if me, ok := re.Expr.(*metricExpr); !ok || me.IsEmpty() || re.ForSubquery() {
+	if re, ok := arg.(*metricsql.RollupExpr); ok {
+		if me, ok := re.Expr.(*metricsql.MetricExpr); !ok || me.IsEmpty() || re.ForSubquery() {
 			return nil, nil
 		}
 		// e = rollupFunc(metricExpr[d])
@@ -335,7 +344,7 @@ func tryGetArgRollupFuncWithMetricExpr(ae *aggrFuncExpr) (*funcExpr, newRollupFu
 	return nil, nil
 }

-func evalExprs(ec *EvalConfig, es []expr) ([][]*timeseries, error) {
+func evalExprs(ec *EvalConfig, es []metricsql.Expr) ([][]*timeseries, error) {
 	var rvs [][]*timeseries
 	for _, e := range es {
 		rv, err := evalExpr(ec, e)
@@ -347,9 +356,12 @@ func evalExprs(ec *EvalConfig, es []expr) ([][]*timeseries, error) {
 	return rvs, nil
 }

-func evalRollupFuncArgs(ec *EvalConfig, fe *funcExpr) ([]interface{}, *rollupExpr, error) {
-	var re *rollupExpr
+func evalRollupFuncArgs(ec *EvalConfig, fe *metricsql.FuncExpr) ([]interface{}, *metricsql.RollupExpr, error) {
+	var re *metricsql.RollupExpr
 	rollupArgIdx := getRollupArgIdx(fe.Name)
+	if len(fe.Args) <= rollupArgIdx {
+		return nil, nil, fmt.Errorf("expecting at least %d args to %q; got %d args; expr: %q", rollupArgIdx, fe.Name, len(fe.Args), fe.AppendString(nil))
+	}
 	args := make([]interface{}, len(fe.Args))
 	for i, arg := range fe.Args {
 		if i == rollupArgIdx {
@@ -366,11 +378,11 @@ func evalRollupFuncArgs(ec *EvalConfig, fe *funcExpr) ([]interface{}, *rollupExp
 	return args, re, nil
 }

-func getRollupExprArg(arg expr) *rollupExpr {
-	re, ok := arg.(*rollupExpr)
+func getRollupExprArg(arg metricsql.Expr) *metricsql.RollupExpr {
+	re, ok := arg.(*metricsql.RollupExpr)
 	if !ok {
-		// Wrap non-rollup arg into rollupExpr.
-		return &rollupExpr{
+		// Wrap non-rollup arg into metricsql.RollupExpr.
+		return &metricsql.RollupExpr{
 			Expr: arg,
 		}
 	}
@@ -378,45 +390,50 @@ func getRollupExprArg(arg expr) *rollupExpr {
 		// Return standard rollup if it doesn't contain subquery.
 		return re
 	}
-	me, ok := re.Expr.(*metricExpr)
+	me, ok := re.Expr.(*metricsql.MetricExpr)
 	if !ok {
 		// arg contains subquery.
 		return re
 	}
 	// Convert me[w:step] -> default_rollup(me)[w:step]
 	reNew := *re
-	reNew.Expr = &funcExpr{
+	reNew.Expr = &metricsql.FuncExpr{
 		Name: "default_rollup",
-		Args: []expr{
-			&rollupExpr{Expr: me},
+		Args: []metricsql.Expr{
+			&metricsql.RollupExpr{Expr: me},
 		},
 	}
 	return &reNew
 }

-func evalRollupFunc(ec *EvalConfig, name string, rf rollupFunc, re *rollupExpr, iafc *incrementalAggrFuncContext) ([]*timeseries, error) {
+func evalRollupFunc(ec *EvalConfig, name string, rf rollupFunc, expr metricsql.Expr, re *metricsql.RollupExpr, iafc *incrementalAggrFuncContext) ([]*timeseries, error) {
 	ecNew := ec
 	var offset int64
 	if len(re.Offset) > 0 {
 		var err error
-		offset, err = DurationValue(re.Offset, ec.Step)
+		offset, err = metricsql.DurationValue(re.Offset, ec.Step)
 		if err != nil {
 			return nil, err
 		}
 		ecNew = newEvalConfig(ec)
 		ecNew.Start -= offset
 		ecNew.End -= offset
-		ecNew.Start, ecNew.End = AdjustStartEnd(ecNew.Start, ecNew.End, ecNew.Step)
+		if ecNew.MayCache {
+			start, end := AdjustStartEnd(ecNew.Start, ecNew.End, ecNew.Step)
+			offset += ecNew.Start - start
+			ecNew.Start = start
+			ecNew.End = end
+		}
 	}
 	var rvs []*timeseries
 	var err error
-	if me, ok := re.Expr.(*metricExpr); ok {
-		rvs, err = evalRollupFuncWithMetricExpr(ecNew, name, rf, me, iafc, re.Window)
+	if me, ok := re.Expr.(*metricsql.MetricExpr); ok {
+		rvs, err = evalRollupFuncWithMetricExpr(ecNew, name, rf, expr, me, iafc, re.Window)
 	} else {
 		if iafc != nil {
 			logger.Panicf("BUG: iafc must be nil for rollup %q over subquery %q", name, re.AppendString(nil))
 		}
-		rvs, err = evalRollupFuncWithSubquery(ecNew, name, rf, re)
+		rvs, err = evalRollupFuncWithSubquery(ecNew, name, rf, expr, re)
 	}
 	if err != nil {
 		return nil, err
@@ -435,12 +452,12 @@ func evalRollupFunc(ec *EvalConfig, name string, rf rollupFunc, re *rollupExpr,
 	return rvs, nil
 }

-func evalRollupFuncWithSubquery(ec *EvalConfig, name string, rf rollupFunc, re *rollupExpr) ([]*timeseries, error) {
-	// Do not use rollupResultCacheV here, since it works only with metricExpr.
+func evalRollupFuncWithSubquery(ec *EvalConfig, name string, rf rollupFunc, expr metricsql.Expr, re *metricsql.RollupExpr) ([]*timeseries, error) {
+	// TODO: determine whether to use rollupResultCacheV here.
 	var step int64
 	if len(re.Step) > 0 {
 		var err error
-		step, err = PositiveDurationValue(re.Step, ec.Step)
+		step, err = metricsql.PositiveDurationValue(re.Step, ec.Step)
 		if err != nil {
 			return nil, err
 		}
@@ -450,7 +467,7 @@ func evalRollupFuncWithSubquery(ec *EvalConfig, name string, rf rollupFunc, re *
 	var window int64
 	if len(re.Window) > 0 {
 		var err error
-		window, err = PositiveDurationValue(re.Window, ec.Step)
+		window, err = metricsql.PositiveDurationValue(re.Window, ec.Step)
 		if err != nil {
 			return nil, err
 		}
@@ -467,9 +484,19 @@ func evalRollupFuncWithSubquery(ec *EvalConfig, name string, rf rollupFunc, re *
 	if err != nil {
 		return nil, err
 	}
+	if len(tssSQ) == 0 {
+		if name == "absent_over_time" {
+			tss := evalNumber(ec, 1)
+			return tss, nil
+		}
+		return nil, nil
+	}

 	sharedTimestamps := getTimestamps(ec.Start, ec.End, ec.Step)
-	preFunc, rcs := getRollupConfigs(name, rf, ec.Start, ec.End, ec.Step, window, ec.LookbackDelta, sharedTimestamps)
+	preFunc, rcs, err := getRollupConfigs(name, rf, expr, ec.Start, ec.End, ec.Step, window, ec.LookbackDelta, sharedTimestamps)
+	if err != nil {
+		return nil, err
+	}
 	tss := make([]*timeseries, 0, len(tssSQ)*len(rcs))
 	var tssLock sync.Mutex
 	removeMetricGroup := !rollupFuncsKeepMetricGroup[name]
@@ -477,6 +504,13 @@ func evalRollupFuncWithSubquery(ec *EvalConfig, name string, rf rollupFunc, re *
 		values, timestamps = removeNanValues(values[:0], timestamps[:0], tsSQ.Values, tsSQ.Timestamps)
 		preFunc(values, timestamps)
 		for _, rc := range rcs {
+			if tsm := newTimeseriesMap(name, sharedTimestamps, &tsSQ.MetricName); tsm != nil {
+				rc.DoTimeseriesMap(tsm, values, timestamps)
+				tssLock.Lock()
+				tss = tsm.AppendTimeseriesTo(tss)
+				tssLock.Unlock()
+				continue
+			}
 			var ts timeseries
 			doRollupForTimeseries(rc, &ts, &tsSQ.MetricName, values, timestamps, sharedTimestamps, removeMetricGroup)
 			tssLock.Lock()
@@ -544,21 +578,22 @@ var (
 	rollupResultCacheMiss        = metrics.NewCounter(`vm_rollup_result_cache_miss_total`)
 )

-func evalRollupFuncWithMetricExpr(ec *EvalConfig, name string, rf rollupFunc, me *metricExpr, iafc *incrementalAggrFuncContext, windowStr string) ([]*timeseries, error) {
+func evalRollupFuncWithMetricExpr(ec *EvalConfig, name string, rf rollupFunc,
+	expr metricsql.Expr, me *metricsql.MetricExpr, iafc *incrementalAggrFuncContext, windowStr string) ([]*timeseries, error) {
 	if me.IsEmpty() {
 		return evalNumber(ec, nan), nil
 	}
 	var window int64
 	if len(windowStr) > 0 {
 		var err error
-		window, err = PositiveDurationValue(windowStr, ec.Step)
+		window, err = metricsql.PositiveDurationValue(windowStr, ec.Step)
 		if err != nil {
 			return nil, err
 		}
 	}

 	// Search for partial results in cache.
-	tssCached, start := rollupResultCacheV.Get(name, ec, me, iafc, window)
+	tssCached, start := rollupResultCacheV.Get(ec, expr, window)
 	if start > ec.End {
 		// The result is fully cached.
 		rollupResultCacheFullHits.Inc()
@@ -571,10 +606,11 @@ func evalRollupFuncWithMetricExpr(ec *EvalConfig, name string, rf rollupFunc, me
 	}

 	// Fetch the remaining part of the result.
+	tfs := toTagFilters(me.LabelFilters)
 	sq := &storage.SearchQuery{
 		MinTimestamp: start - window - maxSilenceInterval,
 		MaxTimestamp: ec.End + ec.Step,
-		TagFilterss:  [][]storage.TagFilter{me.TagFilters},
+		TagFilterss:  [][]storage.TagFilter{tfs},
 	}
 	rss, err := netstorage.ProcessSearchQuery(sq, true, ec.Deadline)
 	if err != nil {
@@ -583,14 +619,22 @@ func evalRollupFuncWithMetricExpr(ec *EvalConfig, name string, rf rollupFunc, me
 	rssLen := rss.Len()
 	if rssLen == 0 {
 		rss.Cancel()
+		var tss []*timeseries
+		if name == "absent_over_time" {
+			tss = getAbsentTimeseries(ec, me)
+		}
 		// Add missing points until ec.End.
 		// Do not cache the result, since missing points
 		// may be backfilled in the future.
-		tss := mergeTimeseries(tssCached, nil, start, ec)
+		tss = mergeTimeseries(tssCached, tss, start, ec)
 		return tss, nil
 	}
 	sharedTimestamps := getTimestamps(start, ec.End, ec.Step)
-	preFunc, rcs := getRollupConfigs(name, rf, start, ec.End, ec.Step, window, ec.LookbackDelta, sharedTimestamps)
+	preFunc, rcs, err := getRollupConfigs(name, rf, expr, start, ec.End, ec.Step, window, ec.LookbackDelta, sharedTimestamps)
+	if err != nil {
+		rss.Cancel()
+		return nil, err
+	}

 	// Verify timeseries fit available memory after the rollup.
 	// Take into account points from tssCached.
@@ -602,8 +646,8 @@ func evalRollupFuncWithMetricExpr(ec *EvalConfig, name string, rf rollupFunc, me
 		if iafc.ae.Modifier.Op != "" {
 			// Increase the number of timeseries for non-empty group list: `aggr() by (something)`,
 			// since each group can have own set of time series in memory.
-			// Estimate the number of such groups is lower than 100 :)
-			timeseriesLen *= 100
+			// Estimate the number of such groups is lower than 1000 :)
+			timeseriesLen *= 1000
 		}
 	}
 	rollupPoints := mulNoOverflow(pointsPerTimeseries, int64(timeseriesLen*len(rcs)))
@@ -622,16 +666,15 @@ func evalRollupFuncWithMetricExpr(ec *EvalConfig, name string, rf rollupFunc, me
 	removeMetricGroup := !rollupFuncsKeepMetricGroup[name]
 	var tss []*timeseries
 	if iafc != nil {
-		tss, err = evalRollupWithIncrementalAggregate(iafc, rss, rcs, preFunc, sharedTimestamps, removeMetricGroup)
+		tss, err = evalRollupWithIncrementalAggregate(name, iafc, rss, rcs, preFunc, sharedTimestamps, removeMetricGroup)
 	} else {
-		tss, err = evalRollupNoIncrementalAggregate(rss, rcs, preFunc, sharedTimestamps, removeMetricGroup)
+		tss, err = evalRollupNoIncrementalAggregate(name, rss, rcs, preFunc, sharedTimestamps, removeMetricGroup)
 	}
 	if err != nil {
 		return nil, err
 	}
 	tss = mergeTimeseries(tssCached, tss, start, ec)
-	rollupResultCacheV.Put(name, ec, me, iafc, window, tss)
-
+	rollupResultCacheV.Put(ec, expr, window, tss)
 	return tss, nil
 }

@@ -647,13 +690,20 @@ func getRollupMemoryLimiter() *memoryLimiter {
 	return &rollupMemoryLimiter
 }

-func evalRollupWithIncrementalAggregate(iafc *incrementalAggrFuncContext, rss *netstorage.Results, rcs []*rollupConfig,
+func evalRollupWithIncrementalAggregate(name string, iafc *incrementalAggrFuncContext, rss *netstorage.Results, rcs []*rollupConfig,
 	preFunc func(values []float64, timestamps []int64), sharedTimestamps []int64, removeMetricGroup bool) ([]*timeseries, error) {
 	err := rss.RunParallel(func(rs *netstorage.Result, workerID uint) {
 		preFunc(rs.Values, rs.Timestamps)
 		ts := getTimeseries()
 		defer putTimeseries(ts)
 		for _, rc := range rcs {
+			if tsm := newTimeseriesMap(name, sharedTimestamps, &rs.MetricName); tsm != nil {
+				rc.DoTimeseriesMap(tsm, rs.Values, rs.Timestamps)
+				for _, ts := range tsm.m {
+					iafc.updateTimeseries(ts, workerID)
+				}
+				continue
+			}
 			ts.Reset()
 			doRollupForTimeseries(rc, ts, &rs.MetricName, rs.Values, rs.Timestamps, sharedTimestamps, removeMetricGroup)
 			iafc.updateTimeseries(ts, workerID)
@@ -670,13 +720,20 @@ func evalRollupWithIncrementalAggregate(iafc *incrementalAggrFuncContext, rss *n
 	return tss, nil
 }

-func evalRollupNoIncrementalAggregate(rss *netstorage.Results, rcs []*rollupConfig,
+func evalRollupNoIncrementalAggregate(name string, rss *netstorage.Results, rcs []*rollupConfig,
 	preFunc func(values []float64, timestamps []int64), sharedTimestamps []int64, removeMetricGroup bool) ([]*timeseries, error) {
 	tss := make([]*timeseries, 0, rss.Len()*len(rcs))
 	var tssLock sync.Mutex
 	err := rss.RunParallel(func(rs *netstorage.Result, workerID uint) {
 		preFunc(rs.Values, rs.Timestamps)
 		for _, rc := range rcs {
+			if tsm := newTimeseriesMap(name, sharedTimestamps, &rs.MetricName); tsm != nil {
+				rc.DoTimeseriesMap(tsm, rs.Values, rs.Timestamps)
+				tssLock.Lock()
+				tss = tsm.AppendTimeseriesTo(tss)
+				tssLock.Unlock()
+				continue
+			}
 			var ts timeseries
 			doRollupForTimeseries(rc, &ts, &rs.MetricName, rs.Values, rs.Timestamps, sharedTimestamps, removeMetricGroup)
 			tssLock.Lock()
@@ -704,62 +761,6 @@ func doRollupForTimeseries(rc *rollupConfig, tsDst *timeseries, mnSrc *storage.M
 	tsDst.denyReuse = true
 }

-func getRollupConfigs(name string, rf rollupFunc, start, end, step, window int64, lookbackDelta int64, sharedTimestamps []int64) (
-	func(values []float64, timestamps []int64), []*rollupConfig) {
-	preFunc := func(values []float64, timestamps []int64) {}
-	if rollupFuncsRemoveCounterResets[name] {
-		preFunc = func(values []float64, timestamps []int64) {
-			removeCounterResets(values)
-		}
-	}
-	newRollupConfig := func(rf rollupFunc, tagValue string) *rollupConfig {
-		return &rollupConfig{
-			TagValue:        tagValue,
-			Func:            rf,
-			Start:           start,
-			End:             end,
-			Step:            step,
-			Window:          window,
-			MayAdjustWindow: rollupFuncsMayAdjustWindow[name],
-			LookbackDelta:   lookbackDelta,
-			Timestamps:      sharedTimestamps,
-		}
-	}
-	appendRollupConfigs := func(dst []*rollupConfig) []*rollupConfig {
-		dst = append(dst, newRollupConfig(rollupMin, "min"))
-		dst = append(dst, newRollupConfig(rollupMax, "max"))
-		dst = append(dst, newRollupConfig(rollupAvg, "avg"))
-		return dst
-	}
-	var rcs []*rollupConfig
-	switch name {
-	case "rollup":
-		rcs = appendRollupConfigs(rcs)
-	case "rollup_rate", "rollup_deriv":
-		preFuncPrev := preFunc
-		preFunc = func(values []float64, timestamps []int64) {
-			preFuncPrev(values, timestamps)
-			derivValues(values, timestamps)
-		}
-		rcs = appendRollupConfigs(rcs)
-	case "rollup_increase", "rollup_delta":
-		preFuncPrev := preFunc
-		preFunc = func(values []float64, timestamps []int64) {
-			preFuncPrev(values, timestamps)
-			deltaValues(values)
-		}
-		rcs = appendRollupConfigs(rcs)
-	case "rollup_candlestick":
-		rcs = append(rcs, newRollupConfig(rollupFirst, "open"))
-		rcs = append(rcs, newRollupConfig(rollupLast, "close"))
-		rcs = append(rcs, newRollupConfig(rollupMin, "low"))
-		rcs = append(rcs, newRollupConfig(rollupMax, "high"))
-	default:
-		rcs = append(rcs, newRollupConfig(rf, ""))
-	}
-	return preFunc, rcs
-}
-
 var bbPool bytesutil.ByteBufferPool

 func evalNumber(ec *EvalConfig, n float64) []*timeseries {
@@ -798,3 +799,23 @@ func mulNoOverflow(a, b int64) int64 {
 	}
 	return a * b
 }
+
+func toTagFilters(lfs []metricsql.LabelFilter) []storage.TagFilter {
+	tfs := make([]storage.TagFilter, len(lfs))
+	for i := range lfs {
+		toTagFilter(&tfs[i], &lfs[i])
+	}
+	return tfs
+}
+
+func toTagFilter(dst *storage.TagFilter, src *metricsql.LabelFilter) {
+	if src.Label != "__name__" {
+		dst.Key = []byte(src.Label)
+	} else {
+		// This is required for storage.Search.
+		dst.Key = nil
+	}
+	dst.Value = []byte(src.Value)
+	dst.IsRegexp = src.IsRegexp
+	dst.IsNegative = src.IsNegative
+}
--- a/app/vmselect/promql/exec.go
+++ b/app/vmselect/promql/exec.go
@@ -11,6 +11,7 @@ import (

 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/netstorage"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql"
 	"github.com/VictoriaMetrics/metrics"
 )

@@ -18,17 +19,6 @@ var logSlowQueryDuration = flag.Duration("search.logSlowQueryDuration", 5*time.S

 var slowQueries = metrics.NewCounter(`vm_slow_queries_total`)

-// ExpandWithExprs expands WITH expressions inside q and returns the resulting
-// PromQL without WITH expressions.
-func ExpandWithExprs(q string) (string, error) {
-	e, err := parsePromQLWithCache(q)
-	if err != nil {
-		return "", err
-	}
-	buf := e.AppendString(nil)
-	return string(buf), nil
-}
-
 // Exec executes q for the given ec.
 func Exec(ec *EvalConfig, q string, isFirstPointOnly bool) ([]netstorage.Result, error) {
 	if *logSlowQueryDuration > 0 {
@@ -85,12 +75,12 @@ func Exec(ec *EvalConfig, q string, isFirstPointOnly bool) ([]netstorage.Result,
 	return result, err
 }

-func maySortResults(e expr, tss []*timeseries) bool {
+func maySortResults(e metricsql.Expr, tss []*timeseries) bool {
 	if len(tss) > 100 {
 		// There is no sense in sorting a lot of results
 		return false
 	}
-	fe, ok := e.(*funcExpr)
+	fe, ok := e.(*metricsql.FuncExpr)
 	if !ok {
 		return true
 	}
@@ -154,10 +144,10 @@ func removeNaNs(tss []*timeseries) []*timeseries {
 	return rvs
 }

-func parsePromQLWithCache(q string) (expr, error) {
+func parsePromQLWithCache(q string) (metricsql.Expr, error) {
 	pcv := parseCacheV.Get(q)
 	if pcv == nil {
-		e, err := parsePromQL(q)
+		e, err := metricsql.Parse(q)
 		pcv = &parseCacheValue{
 			e:   e,
 			err: err,
@@ -189,7 +179,7 @@ var parseCacheV = func() *parseCache {
 const parseCacheMaxLen = 10e3

 type parseCacheValue struct {
-	e   expr
+	e   metricsql.Expr
 	err error
 }

--- a/app/vmselect/promql/exec_test.go
+++ b/app/vmselect/promql/exec_test.go
@@ -8,44 +8,6 @@ import (
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
 )

-func TestExpandWithExprsSuccess(t *testing.T) {
-	f := func(q, qExpected string) {
-		t.Helper()
-		for i := 0; i < 3; i++ {
-			qExpanded, err := ExpandWithExprs(q)
-			if err != nil {
-				t.Fatalf("unexpected error when expanding %q: %s", q, err)
-			}
-			if qExpanded != qExpected {
-				t.Fatalf("unexpected expanded expression for %q;\ngot\n%q\nwant\n%q", q, qExpanded, qExpected)
-			}
-		}
-	}
-
-	f(`1`, `1`)
-	f(`foobar`, `foobar`)
-	f(`with (x = 1) x+x`, `2`)
-	f(`with (f(x) = x*x) 3+f(2)+2`, `9`)
-}
-
-func TestExpandWithExprsError(t *testing.T) {
-	f := func(q string) {
-		t.Helper()
-		for i := 0; i < 3; i++ {
-			qExpanded, err := ExpandWithExprs(q)
-			if err == nil {
-				t.Fatalf("expecting non-nil error when expanding %q", q)
-			}
-			if qExpanded != "" {
-				t.Fatalf("unexpected non-empty qExpanded=%q", qExpanded)
-			}
-		}
-	}
-
-	f(``)
-	f(`  with (`)
-}
-
 func TestExecSuccess(t *testing.T) {
 	start := int64(1000e3)
 	end := int64(2000e3)
@@ -570,6 +532,12 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{}
 		f(q, resultExpected)
 	})
+	t.Run("absent_over_time(time())", func(t *testing.T) {
+		t.Parallel()
+		q := `absent_over_time(time())`
+		resultExpected := []netstorage.Result{}
+		f(q, resultExpected)
+	})
 	t.Run("absent(123)", func(t *testing.T) {
 		t.Parallel()
 		q := `absent(123)`
@@ -593,6 +561,17 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r}
 		f(q, resultExpected)
 	})
+	t.Run("absent_over_time(nan[200s:10s])", func(t *testing.T) {
+		t.Parallel()
+		q := `absent_over_time(nan[200s:10s])`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{1, 1, 1, 1, 1, 1},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
 	t.Run(`absent(scalar(multi-timeseries))`, func(t *testing.T) {
 		t.Parallel()
 		q := `
@@ -609,6 +588,34 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r}
 		f(q, resultExpected)
 	})
+	t.Run(`absent_over_time(scalar(multi-timeseries))`, func(t *testing.T) {
+		t.Parallel()
+		q := `
+		absent_over_time(label_set(scalar(1 or label_set(2, "xx", "foo")), "yy", "foo"))`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{1, 1, 1, 1, 1, 1},
+			Timestamps: timestampsExpected,
+		}
+		r.MetricName.Tags = []storage.Tag{{
+			Key:   []byte("yy"),
+			Value: []byte("foo"),
+		}}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
+	t.Run(`absent(time() > 1500)`, func(t *testing.T) {
+		t.Parallel()
+		q := `
+		absent(time() > 1500)`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{1, 1, 1, nan, nan, nan},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
 	t.Run("clamp_max(time(), 1400)", func(t *testing.T) {
 		t.Parallel()
 		q := `clamp_max(time(), 1400)`
@@ -2301,18 +2308,36 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{}
 		f(q, resultExpected)
 	})
+	t.Run(`histogram_share(scalar)`, func(t *testing.T) {
+		t.Parallel()
+		q := `histogram_share(123, time())`
+		resultExpected := []netstorage.Result{}
+		f(q, resultExpected)
+	})
 	t.Run(`histogram_quantile(single-value-no-le)`, func(t *testing.T) {
 		t.Parallel()
 		q := `histogram_quantile(0.6, label_set(100, "foo", "bar"))`
 		resultExpected := []netstorage.Result{}
 		f(q, resultExpected)
 	})
+	t.Run(`histogram_share(single-value-no-le)`, func(t *testing.T) {
+		t.Parallel()
+		q := `histogram_share(123, label_set(100, "foo", "bar"))`
+		resultExpected := []netstorage.Result{}
+		f(q, resultExpected)
+	})
 	t.Run(`histogram_quantile(single-value-invalid-le)`, func(t *testing.T) {
 		t.Parallel()
 		q := `histogram_quantile(0.6, label_set(100, "le", "foobar"))`
 		resultExpected := []netstorage.Result{}
 		f(q, resultExpected)
 	})
+	t.Run(`histogram_share(single-value-invalid-le)`, func(t *testing.T) {
+		t.Parallel()
+		q := `histogram_share(50, label_set(100, "le", "foobar"))`
+		resultExpected := []netstorage.Result{}
+		f(q, resultExpected)
+	})
 	t.Run(`histogram_quantile(single-value-valid-le)`, func(t *testing.T) {
 		t.Parallel()
 		q := `histogram_quantile(0.6, label_set(100, "le", "200"))`
@@ -2324,6 +2349,39 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r}
 		f(q, resultExpected)
 	})
+	t.Run(`histogram_share(single-value-valid-le)`, func(t *testing.T) {
+		t.Parallel()
+		q := `histogram_share(80, label_set(100, "le", "200"))`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0.4, 0.4, 0.4, 0.4, 0.4, 0.4},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
+	t.Run(`histogram_share(single-value-valid-le)`, func(t *testing.T) {
+		t.Parallel()
+		q := `histogram_share(200, label_set(100, "le", "200"))`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{1, 1, 1, 1, 1, 1},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
+	t.Run(`histogram_share(single-value-valid-le)`, func(t *testing.T) {
+		t.Parallel()
+		q := `histogram_share(300, label_set(100, "le", "200"))`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{1, 1, 1, 1, 1, 1},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
 	t.Run(`histogram_quantile(single-value-valid-le, boundsLabel)`, func(t *testing.T) {
 		t.Parallel()
 		q := `sort(histogram_quantile(0.6, label_set(100, "le", "200"), "foobar"))`
@@ -2353,6 +2411,35 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r1, r2, r3}
 		f(q, resultExpected)
 	})
+	t.Run(`histogram_quantile(single-value-valid-le, boundsLabel)`, func(t *testing.T) {
+		t.Parallel()
+		q := `sort(histogram_share(120, label_set(100, "le", "200"), "foobar"))`
+		r1 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0, 0, 0, 0, 0, 0},
+			Timestamps: timestampsExpected,
+		}
+		r1.MetricName.Tags = []storage.Tag{{
+			Key:   []byte("foobar"),
+			Value: []byte("lower"),
+		}}
+		r2 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0.6, 0.6, 0.6, 0.6, 0.6, 0.6},
+			Timestamps: timestampsExpected,
+		}
+		r3 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{1, 1, 1, 1, 1, 1},
+			Timestamps: timestampsExpected,
+		}
+		r3.MetricName.Tags = []storage.Tag{{
+			Key:   []byte("foobar"),
+			Value: []byte("upper"),
+		}}
+		resultExpected := []netstorage.Result{r1, r2, r3}
+		f(q, resultExpected)
+	})
 	t.Run(`histogram_quantile(single-value-valid-le-max-phi)`, func(t *testing.T) {
 		t.Parallel()
 		q := `histogram_quantile(1, (
@@ -2367,6 +2454,20 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r}
 		f(q, resultExpected)
 	})
+	t.Run(`histogram_share(single-value-valid-le-max-le)`, func(t *testing.T) {
+		t.Parallel()
+		q := `histogram_share(200, (
+			label_set(100, "le", "200"),
+			label_set(0, "le", "55"),
+		))`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{1, 1, 1, 1, 1, 1},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
 	t.Run(`histogram_quantile(single-value-valid-le-min-phi)`, func(t *testing.T) {
 		t.Parallel()
 		q := `histogram_quantile(0, (
@@ -2381,6 +2482,48 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r}
 		f(q, resultExpected)
 	})
+	t.Run(`histogram_share(single-value-valid-le-min-le)`, func(t *testing.T) {
+		t.Parallel()
+		q := `histogram_share(0, (
+			label_set(100, "le", "200"),
+			label_set(0, "le", "55"),
+		))`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0, 0, 0, 0, 0, 0},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
+	t.Run(`histogram_share(single-value-valid-le-low-le)`, func(t *testing.T) {
+		t.Parallel()
+		q := `histogram_share(55, (
+			label_set(100, "le", "200"),
+			label_set(0, "le", "55"),
+		))`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0, 0, 0, 0, 0, 0},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
+	t.Run(`histogram_share(single-value-valid-le-mid-le)`, func(t *testing.T) {
+		t.Parallel()
+		q := `histogram_share(105, (
+			label_set(100, "le", "200"),
+			label_set(0, "le", "55"),
+		))`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0.3448275862068966, 0.3448275862068966, 0.3448275862068966, 0.3448275862068966, 0.3448275862068966, 0.3448275862068966},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
 	t.Run(`histogram_quantile(single-value-valid-le-min-phi-no-zero-bucket)`, func(t *testing.T) {
 		t.Parallel()
 		q := `histogram_quantile(0, label_set(100, "le", "200"))`
@@ -2403,6 +2546,17 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r}
 		f(q, resultExpected)
 	})
+	t.Run(`histogram_share(scalar-phi)`, func(t *testing.T) {
+		t.Parallel()
+		q := `histogram_share(time() / 8, label_set(100, "le", "200"))`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0.625, 0.75, 0.875, 1, 1, 1},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
 	t.Run(`histogram_quantile(valid)`, func(t *testing.T) {
 		t.Parallel()
 		q := `sort(histogram_quantile(0.6,
@@ -2433,6 +2587,36 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r1, r2}
 		f(q, resultExpected)
 	})
+	t.Run(`histogram_share(valid)`, func(t *testing.T) {
+		t.Parallel()
+		q := `sort(histogram_share(25,
+			label_set(90, "foo", "bar", "le", "10")
+			or label_set(100, "foo", "bar", "le", "30")
+			or label_set(300, "foo", "bar", "le", "+Inf")
+			or label_set(200, "tag", "xx", "le", "10")
+			or label_set(300, "tag", "xx", "le", "30")
+		))`
+		r1 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0.325, 0.325, 0.325, 0.325, 0.325, 0.325},
+			Timestamps: timestampsExpected,
+		}
+		r1.MetricName.Tags = []storage.Tag{{
+			Key:   []byte("foo"),
+			Value: []byte("bar"),
+		}}
+		r2 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0.9166666666666666, 0.9166666666666666, 0.9166666666666666, 0.9166666666666666, 0.9166666666666666, 0.9166666666666666},
+			Timestamps: timestampsExpected,
+		}
+		r2.MetricName.Tags = []storage.Tag{{
+			Key:   []byte("tag"),
+			Value: []byte("xx"),
+		}}
+		resultExpected := []netstorage.Result{r1, r2}
+		f(q, resultExpected)
+	})
 	t.Run(`histogram_quantile(negative-bucket-count)`, func(t *testing.T) {
 		t.Parallel()
 		q := `histogram_quantile(0.6,
@@ -2461,7 +2645,7 @@ func TestExecSuccess(t *testing.T) {
 		)`
 		r := netstorage.Result{
 			MetricName: metricNameExpected,
-			Values:     []float64{10, 10, 10, 10, 10, 10},
+			Values:     []float64{30, 30, 30, 30, 30, 30},
 			Timestamps: timestampsExpected,
 		}
 		r.MetricName.Tags = []storage.Tag{{
@@ -2490,6 +2674,25 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r}
 		f(q, resultExpected)
 	})
+	t.Run(`histogram_share(normal-bucket-count)`, func(t *testing.T) {
+		t.Parallel()
+		q := `histogram_share(35,
+			label_set(0, "foo", "bar", "le", "10")
+			or label_set(100, "foo", "bar", "le", "30")
+			or label_set(300, "foo", "bar", "le", "+Inf")
+		)`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333},
+			Timestamps: timestampsExpected,
+		}
+		r.MetricName.Tags = []storage.Tag{{
+			Key:   []byte("foo"),
+			Value: []byte("bar"),
+		}}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
 	t.Run(`histogram_quantile(normal-bucket-count, boundsLabel)`, func(t *testing.T) {
 		t.Parallel()
 		q := `sort(histogram_quantile(0.2,
@@ -2540,6 +2743,56 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r1, r2, r3}
 		f(q, resultExpected)
 	})
+	t.Run(`histogram_share(normal-bucket-count, boundsLabel)`, func(t *testing.T) {
+		t.Parallel()
+		q := `sort(histogram_share(22,
+			label_set(0, "foo", "bar", "le", "10")
+			or label_set(100, "foo", "bar", "le", "30")
+			or label_set(300, "foo", "bar", "le", "+Inf"),
+			"xxx"
+		))`
+		r1 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0, 0, 0, 0, 0, 0},
+			Timestamps: timestampsExpected,
+		}
+		r1.MetricName.Tags = []storage.Tag{
+			{
+				Key:   []byte("foo"),
+				Value: []byte("bar"),
+			},
+			{
+				Key:   []byte("xxx"),
+				Value: []byte("lower"),
+			},
+		}
+		r2 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0.2, 0.2, 0.2, 0.2, 0.2, 0.2},
+			Timestamps: timestampsExpected,
+		}
+		r2.MetricName.Tags = []storage.Tag{{
+			Key:   []byte("foo"),
+			Value: []byte("bar"),
+		}}
+		r3 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333},
+			Timestamps: timestampsExpected,
+		}
+		r3.MetricName.Tags = []storage.Tag{
+			{
+				Key:   []byte("foo"),
+				Value: []byte("bar"),
+			},
+			{
+				Key:   []byte("xxx"),
+				Value: []byte("upper"),
+			},
+		}
+		resultExpected := []netstorage.Result{r1, r2, r3}
+		f(q, resultExpected)
+	})
 	t.Run(`histogram_quantile(zero-bucket-count)`, func(t *testing.T) {
 		t.Parallel()
 		q := `histogram_quantile(0.6,
@@ -3121,6 +3374,150 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r}
 		f(q, resultExpected)
 	})
+	t.Run(`histogram_over_time`, func(t *testing.T) {
+		t.Parallel()
+		q := `sort(histogram_over_time(alias(label_set(rand(0)*1.3+1.1, "foo", "bar"), "xxx")[200s:5s]))`
+		r1 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{13, 14, 12, 8, 12, 13},
+			Timestamps: timestampsExpected,
+		}
+		r1.MetricName.Tags = []storage.Tag{
+			{
+				Key:   []byte("foo"),
+				Value: []byte("bar"),
+			},
+			{
+				Key:   []byte("vmrange"),
+				Value: []byte("1.0e0...1.5e0"),
+			},
+		}
+		r2 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{14, 15, 12, 13, 15, 11},
+			Timestamps: timestampsExpected,
+		}
+		r2.MetricName.Tags = []storage.Tag{
+			{
+				Key:   []byte("foo"),
+				Value: []byte("bar"),
+			},
+			{
+				Key:   []byte("vmrange"),
+				Value: []byte("2.0e0...2.5e0"),
+			},
+		}
+		r3 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{13, 11, 16, 19, 13, 16},
+			Timestamps: timestampsExpected,
+		}
+		r3.MetricName.Tags = []storage.Tag{
+			{
+				Key:   []byte("foo"),
+				Value: []byte("bar"),
+			},
+			{
+				Key:   []byte("vmrange"),
+				Value: []byte("1.5e0...2.0e0"),
+			},
+		}
+		resultExpected := []netstorage.Result{r1, r2, r3}
+		f(q, resultExpected)
+	})
+	t.Run(`sum(histogram_over_time) by (vmrange)`, func(t *testing.T) {
+		t.Parallel()
+		q := `sort(sum(histogram_over_time(alias(label_set(rand(0)*1.3+1.1, "foo", "bar"), "xxx")[200s:5s])) by (vmrange))`
+		r1 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{13, 14, 12, 8, 12, 13},
+			Timestamps: timestampsExpected,
+		}
+		r1.MetricName.Tags = []storage.Tag{
+			{
+				Key:   []byte("vmrange"),
+				Value: []byte("1.0e0...1.5e0"),
+			},
+		}
+		r2 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{14, 15, 12, 13, 15, 11},
+			Timestamps: timestampsExpected,
+		}
+		r2.MetricName.Tags = []storage.Tag{
+			{
+				Key:   []byte("vmrange"),
+				Value: []byte("2.0e0...2.5e0"),
+			},
+		}
+		r3 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{13, 11, 16, 19, 13, 16},
+			Timestamps: timestampsExpected,
+		}
+		r3.MetricName.Tags = []storage.Tag{
+			{
+				Key:   []byte("vmrange"),
+				Value: []byte("1.5e0...2.0e0"),
+			},
+		}
+		resultExpected := []netstorage.Result{r1, r2, r3}
+		f(q, resultExpected)
+	})
+	t.Run(`sum(histogram_over_time)`, func(t *testing.T) {
+		t.Parallel()
+		q := `sum(histogram_over_time(alias(label_set(rand(0)*1.3+1.1, "foo", "bar"), "xxx")[200s:5s]))`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{40, 40, 40, 40, 40, 40},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
+	t.Run(`topk_max(histogram_over_time)`, func(t *testing.T) {
+		t.Parallel()
+		q := `topk_max(1, histogram_over_time(alias(label_set(rand(0)*1.3+1.1, "foo", "bar"), "xxx")[200s:5s]))`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{13, 11, 16, 19, 13, 16},
+			Timestamps: timestampsExpected,
+		}
+		r.MetricName.Tags = []storage.Tag{
+			{
+				Key:   []byte("foo"),
+				Value: []byte("bar"),
+			},
+			{
+				Key:   []byte("vmrange"),
+				Value: []byte("1.5e0...2.0e0"),
+			},
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
+	t.Run(`share_gt_over_time`, func(t *testing.T) {
+		t.Parallel()
+		q := `share_gt_over_time(rand(0)[200s:10s], 0.7)`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0.35, 0.3, 0.5, 0.3, 0.3, 0.25},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
+	t.Run(`share_le_over_time`, func(t *testing.T) {
+		t.Parallel()
+		q := `share_le_over_time(rand(0)[200s:10s], 0.7)`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0.65, 0.7, 0.5, 0.7, 0.7, 0.75},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
 	t.Run(`increases_over_time`, func(t *testing.T) {
 		t.Parallel()
 		q := `increases_over_time(rand(0)[200s:10s])`
@@ -4083,6 +4480,89 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r}
 		f(q, resultExpected)
 	})
+	t.Run(`aggr_over_time(single-func)`, func(t *testing.T) {
+		t.Parallel()
+		q := `aggr_over_time("increase", rand(0)[:10s])`
+		r1 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{5.465672601448873, 6.642207999066246, 6.8400051805114295, 7.182425481980655, 5.1677922402706, 6.594060518641982},
+			Timestamps: timestampsExpected,
+		}
+		r1.MetricName.Tags = []storage.Tag{{
+			Key:   []byte("rollup"),
+			Value: []byte("increase"),
+		}}
+		resultExpected := []netstorage.Result{r1}
+		f(q, resultExpected)
+	})
+	t.Run(`aggr_over_time(multi-func)`, func(t *testing.T) {
+		t.Parallel()
+		q := `sort(aggr_over_time(("min_over_time", "count_over_time", "max_over_time"), round(rand(0),0.1)[:10s]))`
+		r1 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0, 0, 0, 0, 0, 0},
+			Timestamps: timestampsExpected,
+		}
+		r1.MetricName.Tags = []storage.Tag{{
+			Key:   []byte("rollup"),
+			Value: []byte("min_over_time"),
+		}}
+		r2 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0.8, 0.9, 1, 0.9, 1, 0.9},
+			Timestamps: timestampsExpected,
+		}
+		r2.MetricName.Tags = []storage.Tag{{
+			Key:   []byte("rollup"),
+			Value: []byte("max_over_time"),
+		}}
+		r3 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{20, 20, 20, 20, 20, 20},
+			Timestamps: timestampsExpected,
+		}
+		r3.MetricName.Tags = []storage.Tag{{
+			Key:   []byte("rollup"),
+			Value: []byte("count_over_time"),
+		}}
+		resultExpected := []netstorage.Result{r1, r2, r3}
+		f(q, resultExpected)
+	})
+	t.Run(`avg(aggr_over_time(multi-func))`, func(t *testing.T) {
+		t.Parallel()
+		q := `avg(aggr_over_time(("min_over_time", "max_over_time"), time()[:10s]))`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{905, 1105, 1305, 1505, 1705, 1905},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
+	t.Run(`avg(aggr_over_time(multi-func)) by (rollup)`, func(t *testing.T) {
+		t.Parallel()
+		q := `sort(avg(aggr_over_time(("min_over_time", "max_over_time"), time()[:10s])) by (rollup))`
+		r1 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{810, 1010, 1210, 1410, 1610, 1810},
+			Timestamps: timestampsExpected,
+		}
+		r1.MetricName.Tags = []storage.Tag{{
+			Key:   []byte("rollup"),
+			Value: []byte("min_over_time"),
+		}}
+		r2 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{1000, 1200, 1400, 1600, 1800, 2000},
+			Timestamps: timestampsExpected,
+		}
+		r2.MetricName.Tags = []storage.Tag{{
+			Key:   []byte("rollup"),
+			Value: []byte("max_over_time"),
+		}}
+		resultExpected := []netstorage.Result{r1, r2}
+		f(q, resultExpected)
+	})
 	t.Run(`rollup_candlestick()`, func(t *testing.T) {
 		t.Parallel()
 		q := `sort(rollup_candlestick(round(rand(0),0.01)[:10s]))`
@@ -4097,25 +4577,25 @@ func TestExecSuccess(t *testing.T) {
 		}}
 		r2 := netstorage.Result{
 			MetricName: metricNameExpected,
-			Values:     []float64{0.32, 0.82, 0.13, 0.28, 0.86, 0.57},
+			Values:     []float64{0.85, 0.15, 0.43, 0.76, 0.47, 0.21},
 			Timestamps: timestampsExpected,
 		}
 		r2.MetricName.Tags = []storage.Tag{{
 			Key:   []byte("rollup"),
-			Value: []byte("close"),
+			Value: []byte("open"),
 		}}
 		r3 := netstorage.Result{
 			MetricName: metricNameExpected,
-			Values:     []float64{0.9, 0.32, 0.82, 0.13, 0.28, 0.86},
+			Values:     []float64{0.32, 0.82, 0.13, 0.28, 0.86, 0.57},
 			Timestamps: timestampsExpected,
 		}
 		r3.MetricName.Tags = []storage.Tag{{
 			Key:   []byte("rollup"),
-			Value: []byte("open"),
+			Value: []byte("close"),
 		}}
 		r4 := netstorage.Result{
 			MetricName: metricNameExpected,
-			Values:     []float64{0.9, 0.94, 0.97, 0.93, 0.98, 0.92},
+			Values:     []float64{0.85, 0.94, 0.97, 0.93, 0.98, 0.92},
 			Timestamps: timestampsExpected,
 		}
 		r4.MetricName.Tags = []storage.Tag{{
@@ -4163,7 +4643,7 @@ func TestExecSuccess(t *testing.T) {
 		q := `sort(rollup(time()[:50s]))`
 		r1 := netstorage.Result{
 			MetricName: metricNameExpected,
-			Values:     []float64{800, 1000, 1200, 1400, 1600, 1800},
+			Values:     []float64{850, 1050, 1250, 1450, 1650, 1850},
 			Timestamps: timestampsExpected,
 		}
 		r1.MetricName.Tags = []storage.Tag{{
@@ -4784,6 +5264,9 @@ func TestExecError(t *testing.T) {
 	f(`alias(1, "foo", "bar")`)
 	f(`lifetime()`)
 	f(`lag()`)
+	f(`aggr_over_time()`)
+	f(`aggr_over_time(foo)`)
+	f(`aggr_over_time("foo", bar, 1)`)

 	// Invalid argument type
 	f(`median_over_time({}, 2)`)
@@ -4819,6 +5302,8 @@ func TestExecError(t *testing.T) {
 	f(`label_transform(1, "foo", "bar", 4)`)
 	f(`label_transform(1, "foo", "invalid(regexp", "baz`)
 	f(`alias(1, 2)`)
+	f(`aggr_over_time(1, 2)`)
+	f(`aggr_over_time(("foo", "bar"), 3)`)

 	// Duplicate timeseries
 	f(`(label_set(1, "foo", "bar") or label_set(2, "foo", "baz"))
--- a/app/vmselect/promql/parser.go
+++ b/app/vmselect/promql/parser.go
--- a/app/vmselect/promql/parser_test.go
+++ b/app/vmselect/promql/parser_test.go
@@ -47,677 +47,3 @@ func TestParseMetricSelectorError(t *testing.T) {
 	f(`foo[5m]`)
 	f(`foo offset 5m`)
 }
-
-func TestParsePromQLSuccess(t *testing.T) {
-	another := func(s string, sExpected string) {
-		t.Helper()
-
-		e, err := parsePromQL(s)
-		if err != nil {
-			t.Fatalf("unexpected error when parsing %q: %s", s, err)
-		}
-		res := e.AppendString(nil)
-		if string(res) != sExpected {
-			t.Fatalf("unexpected string constructed;\ngot\n%q\nwant\n%q", res, sExpected)
-		}
-	}
-	same := func(s string) {
-		t.Helper()
-		another(s, s)
-	}
-
-	// metricExpr
-	same(`{}`)
-	same(`{}[5m]`)
-	same(`{}[5m:]`)
-	same(`{}[:]`)
-	another(`{}[: ]`, `{}[:]`)
-	same(`{}[:3s]`)
-	another(`{}[: 3s ]`, `{}[:3s]`)
-	same(`{}[5m:3s]`)
-	another(`{}[ 5m : 3s ]`, `{}[5m:3s]`)
-	same(`{} offset 5m`)
-	same(`{} offset -5m`)
-	same(`{}[5m] offset 10y`)
-	same(`{}[5.3m:3.4s] offset 10y`)
-	same(`{}[:3.4s] offset 10y`)
-	same(`{}[:3.4s] offset -10y`)
-	same(`{Foo="bAR"}`)
-	same(`{foo="bar"}`)
-	same(`{foo="bar"}[5m]`)
-	same(`{foo="bar"}[5m:]`)
-	same(`{foo="bar"}[5m:3s]`)
-	same(`{foo="bar"} offset 10y`)
-	same(`{foo="bar"} offset -10y`)
-	same(`{foo="bar"}[5m] offset 10y`)
-	same(`{foo="bar"}[5m:3s] offset 10y`)
-	another(`{foo="bar"}[5m] oFFSEt 10y`, `{foo="bar"}[5m] offset 10y`)
-	same("METRIC")
-	same("metric")
-	same("m_e:tri44:_c123")
-	another("-metric", "0 - metric")
-	same(`metric offset 10h`)
-	same("metric[5m]")
-	same("metric[5m:3s]")
-	same("metric[5m] offset 10h")
-	same("metric[5m:3s] offset 10h")
-	same("metric[5i:3i] offset 10i")
-	same(`metric{foo="bar"}`)
-	same(`metric{foo="bar"} offset 10h`)
-	same(`metric{foo!="bar"}[2d]`)
-	same(`metric{foo="bar"}[2d] offset 10h`)
-	same(`metric{foo="bar", b="sdfsdf"}[2d:3h] offset 10h`)
-	another(`  metric  {  foo  = "bar"  }  [  2d ]   offset   10h  `, `metric{foo="bar"}[2d] offset 10h`)
-	// metric name matching keywords
-	same("rate")
-	same("RATE")
-	same("by")
-	same("BY")
-	same("bool")
-	same("BOOL")
-	same("unless")
-	same("UNLESS")
-	same("Ignoring")
-	same("with")
-	same("WITH")
-	same("With")
-	same("alias")
-	same(`alias{foo="bar"}`)
-	same(`aLIas{alias="aa"}`)
-	another(`al\ias`, `alias`)
-	// identifiers with with escape chars
-	same(`foo\ bar`)
-	same(`foo\-bar\{{baz\+bar="aa"}`)
-	another(`\x2E\x2ef\oo{b\xEF\ar="aa"}`, `\x2e.foo{b\xefar="aa"}`)
-	// Duplicate filters
-	same(`foo{__name__="bar"}`)
-	same(`foo{a="b", a="c", __name__="aaa", b="d"}`)
-	// Metric filters ending with comma
-	another(`m{foo="bar",}`, `m{foo="bar"}`)
-	// String concat in tag value
-	another(`m{foo="bar" + "baz"}`, `m{foo="barbaz"}`)
-
-	// Valid regexp
-	same(`foo{bar=~"x"}`)
-	same(`foo{bar=~"^x"}`)
-	same(`foo{bar=~"^x$"}`)
-	same(`foo{bar=~"^(a[bc]|d)$"}`)
-	same(`foo{bar!~"x"}`)
-	same(`foo{bar!~"^x"}`)
-	same(`foo{bar!~"^x$"}`)
-	same(`foo{bar!~"^(a[bc]|d)$"}`)
-
-	// stringExpr
-	same(`""`)
-	same(`"\n\t\r 12:{}[]()44"`)
-	another(`''`, `""`)
-	another("``", `""`)
-	another("   `foo\"b'ar`  ", "\"foo\\\"b'ar\"")
-	another(`  'foo\'bar"BAZ'  `, `"foo'bar\"BAZ"`)
-	// string concat
-	another(`"foo"+'bar'`, `"foobar"`)
-
-	// numberExpr
-	same(`1`)
-	same(`1.23`)
-	same(`0.23`)
-	same(`1.2e+45`)
-	same(`1.2e-45`)
-	same(`-1`)
-	same(`-1.23`)
-	same(`-0.23`)
-	same(`-1.2e+45`)
-	same(`-1.2e-45`)
-	same(`-1.2e-45`)
-	another(`12.5E34`, `1.25e+35`)
-	another(`-.2`, `-0.2`)
-	another(`-.2E-2`, `-0.002`)
-	same(`NaN`)
-	another(`nan`, `NaN`)
-	another(`NAN`, `NaN`)
-	another(`nAN`, `NaN`)
-	another(`Inf`, `+Inf`)
-	another(`INF`, `+Inf`)
-	another(`inf`, `+Inf`)
-	another(`+Inf`, `+Inf`)
-	another(`-Inf`, `-Inf`)
-	another(`-inF`, `-Inf`)
-
-	// binaryOpExpr
-	another(`nan == nan`, `NaN`)
-	another(`nan ==bool nan`, `1`)
-	another(`nan !=bool nan`, `0`)
-	another(`nan !=bool 2`, `1`)
-	another(`2 !=bool nan`, `1`)
-	another(`nan >bool nan`, `0`)
-	another(`nan <bool nan`, `0`)
-	another(`1 ==bool nan`, `0`)
-	another(`NaN !=bool 1`, `1`)
-	another(`inf >=bool 2`, `1`)
-	another(`-1 >bool -inf`, `1`)
-	another(`-1 <bool -inf`, `0`)
-	another(`nan + 2 *3 * inf`, `NaN`)
-	another(`INF - Inf`, `NaN`)
-	another(`Inf + inf`, `+Inf`)
-	another(`1/0`, `+Inf`)
-	another(`0/0`, `NaN`)
-	another(`-m`, `0 - m`)
-	same(`m + ignoring () n[5m]`)
-	another(`M + IGNORING () N[5m]`, `M + ignoring () N[5m]`)
-	same(`m + on (foo) n[5m]`)
-	another(`m + ON (Foo) n[5m]`, `m + on (Foo) n[5m]`)
-	same(`m + ignoring (a, b) n[5m]`)
-	another(`1 or 2`, `1`)
-	another(`1 and 2`, `1`)
-	another(`1 unless 2`, `NaN`)
-	another(`1 default 2`, `1`)
-	another(`1 default NaN`, `1`)
-	another(`NaN default 2`, `2`)
-	another(`1 > 2`, `NaN`)
-	another(`1 > bool 2`, `0`)
-	another(`3 >= 2`, `3`)
-	another(`3 <= bool 2`, `0`)
-	another(`1 + -2 - 3`, `-4`)
-	another(`1 / 0 + 2`, `+Inf`)
-	another(`2 + -1 / 0`, `-Inf`)
-	another(`-1 ^ 0.5`, `NaN`)
-	another(`512.5 - (1 + 3) * (2 ^ 2) ^ 3`, `256.5`)
-	another(`1 == bool 1 != bool 24 < bool 4 > bool -1`, `1`)
-	another(`1 == bOOl 1 != BOOL 24 < Bool 4 > booL -1`, `1`)
-	another(`m1+on(foo)group_left m2`, `m1 + on (foo) group_left () m2`)
-	another(`M1+ON(FOO)GROUP_left M2`, `M1 + on (FOO) group_left () M2`)
-	same(`m1 + on (foo) group_right () m2`)
-	same(`m1 + on (foo, bar) group_right (x, y) m2`)
-	another(`m1 + on (foo, bar,) group_right (x, y,) m2`, `m1 + on (foo, bar) group_right (x, y) m2`)
-	same(`m1 == bool on (foo, bar) group_right (x, y) m2`)
-	another(`5 - 1 + 3 * 2 ^ 2 ^ 3 - 2  OR Metric {Bar= "Baz", aaa!="bb",cc=~"dd" ,zz !~"ff" } `,
-		`770 or Metric{Bar="Baz", aaa!="bb", cc=~"dd", zz!~"ff"}`)
-	same(`"foo" + bar()`)
-	same(`"foo" + bar{x="y"}`)
-	same(`("foo"[3s] + bar{x="y"})[5m:3s] offset 10s`)
-	same(`("foo"[3s] + bar{x="y"})[5i:3i] offset 10i`)
-	same(`bar + "foo" offset 3s`)
-	same(`bar + "foo" offset 3i`)
-	another(`1+2 if 2>3`, `NaN`)
-	another(`1+4 if 2<3`, `5`)
-	another(`2+6 default 3 if 2>3`, `8`)
-	another(`2+6 if 2>3 default NaN`, `NaN`)
-	another(`42 if 3>2 if 2+2<5`, `42`)
-	another(`42 if 3>2 if 2+2>=5`, `NaN`)
-	another(`1+2 ifnot 2>3`, `3`)
-	another(`1+4 ifnot 2<3`, `NaN`)
-	another(`2+6 default 3 ifnot 2>3`, `8`)
-	another(`2+6 ifnot 2>3 default NaN`, `8`)
-	another(`42 if 3>2 ifnot 2+2<5`, `NaN`)
-	another(`42 if 3>2 ifnot 2+2>=5`, `42`)
-
-	// parensExpr
-	another(`(-foo + ((bar) / (baz))) + ((23))`, `((0 - foo) + (bar / baz)) + 23`)
-	another(`(FOO + ((Bar) / (baZ))) + ((23))`, `(FOO + (Bar / baZ)) + 23`)
-	same(`(foo, bar)`)
-	another(`((foo, bar),(baz))`, `((foo, bar), baz)`)
-	same(`(foo, (bar, baz), ((x, y), (z, y), xx))`)
-	another(`1+(foo, bar,)`, `1 + (foo, bar)`)
-	another(`((foo(bar,baz)), (1+(2)+(3,4)+()))`, `(foo(bar, baz), (3 + (3, 4)) + ())`)
-	same(`()`)
-
-	// funcExpr
-	same(`f()`)
-	another(`f(x,)`, `f(x)`)
-	another(`-f()-Ff()`, `(0 - f()) - Ff()`)
-	same(`F()`)
-	another(`+F()`, `F()`)
-	another(`++F()`, `F()`)
-	another(`--F()`, `0 - (0 - F())`)
-	same(`f(http_server_request)`)
-	same(`f(http_server_request)[4s:5m] offset 10m`)
-	same(`f(http_server_request)[4i:5i] offset 10i`)
-	same(`F(HttpServerRequest)`)
-	same(`f(job, foo)`)
-	same(`F(Job, Foo)`)
-	another(` FOO (bar) + f  (  m  (  ),ff(1 + (  2.5)) ,M[5m ]  , "ff"  )`, `FOO(bar) + f(m(), ff(3.5), M[5m], "ff")`)
-	// funcName matching keywords
-	same(`by(2)`)
-	same(`BY(2)`)
-	same(`or(2)`)
-	same(`OR(2)`)
-	same(`bool(2)`)
-	same(`BOOL(2)`)
-	same(`rate(rate(m))`)
-	same(`rate(rate(m[5m]))`)
-	same(`rate(rate(m[5m])[1h:])`)
-	same(`rate(rate(m[5m])[1h:3s])`)
-	// funcName with escape chars
-	same(`foo\(ba\-r()`)
-
-	// aggrFuncExpr
-	same(`sum(http_server_request) by ()`)
-	same(`sum(http_server_request) by (job)`)
-	same(`sum(http_server_request) without (job, foo)`)
-	another(`sum(x,y,) without (a,b,)`, `sum(x, y) without (a, b)`)
-	another(`sum by () (xx)`, `sum(xx) by ()`)
-	another(`sum by (s) (xx)[5s]`, `(sum(xx) by (s))[5s]`)
-	another(`SUM BY (ZZ, aa) (XX)`, `sum(XX) by (ZZ, aa)`)
-	another(`sum without (a, b) (xx,2+2)`, `sum(xx, 4) without (a, b)`)
-	another(`Sum WIthout (a, B) (XX,2+2)`, `sum(XX, 4) without (a, B)`)
-	same(`sum(a) or sum(b)`)
-	same(`sum(a) by () or sum(b) without (x, y)`)
-	same(`sum(a) + sum(b)`)
-	same(`sum(x) * (1 + sum(a))`)
-
-	// All the above
-	another(`Sum(Ff(M) * M{X=""}[5m] Offset 7m - 123, 35) BY (X, y) * F2("Test")`,
-		`sum((Ff(M) * M{X=""}[5m] offset 7m) - 123, 35) by (X, y) * F2("Test")`)
-	another(`# comment
-		Sum(Ff(M) * M{X=""}[5m] Offset 7m - 123, 35) BY (X, y) # yet another comment
-		* F2("Test")`,
-		`sum((Ff(M) * M{X=""}[5m] offset 7m) - 123, 35) by (X, y) * F2("Test")`)
-
-	// withExpr
-	another(`with () x`, `x`)
-	another(`with (x=1,) x`, `1`)
-	another(`with (x = m offset 5h) x + x`, `m offset 5h + m offset 5h`)
-	another(`with (x = m offset 5i) x + x`, `m offset 5i + m offset 5i`)
-	another(`with (foo = bar{x="x"}) 1`, `1`)
-	another(`with (foo = bar{x="x"}) "x"`, `"x"`)
-	another(`with (f="x") f`, `"x"`)
-	another(`with (foo = bar{x="x"}) x{x="y"}`, `x{x="y"}`)
-	another(`with (foo = bar{x="x"}) 1+1`, `2`)
-	another(`with (foo = bar{x="x"}) f()`, `f()`)
-	another(`with (foo = bar{x="x"}) sum(x)`, `sum(x)`)
-	another(`with (foo = bar{x="x"}) baz{foo="bar"}`, `baz{foo="bar"}`)
-	another(`with (foo = bar) baz`, `baz`)
-	another(`with (foo = bar) foo + foo{a="b"}`, `bar + bar{a="b"}`)
-	another(`with (foo = bar, bar=baz + f()) test`, `test`)
-	another(`with (ct={job="test"}) a{ct} + ct() + f({ct="x"})`, `(a{job="test"} + {job="test"}) + f({ct="x"})`)
-	another(`with (ct={job="test", i="bar"}) ct + {ct, x="d"} + foo{ct, ct} + ctx(1)`,
-		`(({job="test", i="bar"} + {job="test", i="bar", x="d"}) + foo{job="test", i="bar"}) + ctx(1)`)
-	another(`with (foo = bar) {__name__=~"foo"}`, `{__name__=~"foo"}`)
-	another(`with (foo = bar) foo{__name__="foo"}`, `bar`)
-	another(`with (foo = bar) {__name__="foo", x="y"}`, `bar{x="y"}`)
-	another(`with (foo(bar) = {__name__!="bar"}) foo(x)`, `{__name__!="bar"}`)
-	another(`with (foo(bar) = bar{__name__="bar"}) foo(x)`, `x`)
-	another(`with (foo\-bar(baz) = baz + baz) foo\-bar((x,y))`, `(x, y) + (x, y)`)
-	another(`with (foo\-bar(baz) = baz + baz) foo\-bar(x*y)`, `(x * y) + (x * y)`)
-	another(`with (foo\-bar(baz) = baz + baz) foo\-bar(x\*y)`, `x\*y + x\*y`)
-	another(`with (foo\-bar(b\ az) = b\ az + b\ az) foo\-bar(x\*y)`, `x\*y + x\*y`)
-	// override ttf to something new.
-	another(`with (ttf = a) ttf + b`, `a + b`)
-	// override ttf to ru
-	another(`with (ttf = ru(m, n)) ttf`, `(clamp_min(n - clamp_min(m, 0), 0) / clamp_min(n, 0)) * 100`)
-
-	// Verify withExpr recursion and forward reference
-	another(`with (x = x+y, y = x+x) y ^ 2`, `((x + y) + (x + y)) ^ 2`)
-	another(`with (f1(x)=f2(x), f2(x)=f1(x)^2) f1(foobar)`, `f2(foobar)`)
-	another(`with (f1(x)=f2(x), f2(x)=f1(x)^2) f2(foobar)`, `f2(foobar) ^ 2`)
-
-	// Verify withExpr funcs
-	another(`with (x() = y+1) x`, `y + 1`)
-	another(`with (x(foo) = foo+1) x(a)`, `a + 1`)
-	another(`with (x(a, b) = a + b) x(foo, bar)`, `foo + bar`)
-	another(`with (x(a, b) = a + b) x(foo, x(1, 2))`, `foo + 3`)
-	another(`with (x(a) = sum(a) by (b)) x(xx) / x(y)`, `sum(xx) by (b) / sum(y) by (b)`)
-	another(`with (f(a,f,x)=ff(x,f,a)) f(f(x,y,z),1,2)`, `ff(2, 1, ff(z, y, x))`)
-	another(`with (f(x)=1+f(x)) f(foo{bar="baz"})`, `1 + f(foo{bar="baz"})`)
-	another(`with (a=foo, y=bar, f(a)= a+a+y) f(x)`, `(x + x) + bar`)
-	another(`with (f(a, b) = m{a, b}) f({a="x", b="y"}, {c="d"})`, `m{a="x", b="y", c="d"}`)
-	another(`with (xx={a="x"}, f(a, b) = m{a, b}) f({xx, b="y"}, {c="d"})`, `m{a="x", b="y", c="d"}`)
-	another(`with (x() = {b="c"}) foo{x}`, `foo{b="c"}`)
-	another(`with (f(x)=x{foo="bar"} offset 5m) f(m offset 10m)`, `(m{foo="bar"} offset 10m) offset 5m`)
-	another(`with (f(x)=x{foo="bar",bas="a"}[5m]) f(m[10m] offset 3s)`, `(m{foo="bar", bas="a"}[10m] offset 3s)[5m]`)
-	another(`with (f(x)=x{foo="bar"}[5m] offset 10m) f(m{x="y"})`, `m{x="y", foo="bar"}[5m] offset 10m`)
-	another(`with (f(x)=x{foo="bar"}[5m] offset 10m) f({x="y", foo="bar", foo="bar"})`, `{x="y", foo="bar"}[5m] offset 10m`)
-	another(`with (f(m, x)=m{x}[5m] offset 10m) f(foo, {})`, `foo[5m] offset 10m`)
-	another(`with (f(m, x)=m{x, bar="baz"}[5m] offset 10m) f(foo, {})`, `foo{bar="baz"}[5m] offset 10m`)
-	another(`with (f(x)=x[5m] offset 3s) f(foo[3m]+bar)`, `(foo[3m] + bar)[5m] offset 3s`)
-	another(`with (f(x)=x[5m:3s] oFFsEt 1.5m) f(sum(s) by (a,b))`, `(sum(s) by (a, b))[5m:3s] offset 1.5m`)
-	another(`with (x="a", y=x) y+"bc"`, `"abc"`)
-	another(`with (x="a", y="b"+x) "we"+y+"z"+f()`, `"webaz" + f()`)
-	another(`with (f(x) = m{foo=x+"y", bar="y"+x, baz=x} + x) f("qwe")`, `m{foo="qwey", bar="yqwe", baz="qwe"} + "qwe"`)
-	another(`with (f(a)=a) f`, `f`)
-	another(`with (f\q(a)=a) f\q`, `fq`)
-
-	// Verify withExpr for aggr func modifiers
-	another(`with (f(x) = x, y = sum(m) by (f)) y`, `sum(m) by (f)`)
-	another(`with (f(x) = sum(m) by (x)) f(foo)`, `sum(m) by (foo)`)
-	another(`with (f(x) = sum(m) by (x)) f((foo, bar, foo))`, `sum(m) by (foo, bar)`)
-	another(`with (f(x) = sum(m) without (x,y)) f((a, b))`, `sum(m) without (a, b, y)`)
-	another(`with (f(x) = sum(m) without (y,x)) f((a, y))`, `sum(m) without (y, a)`)
-	another(`with (f(x,y) = a + on (x,y) group_left (y,bar) b) f(foo,())`, `a + on (foo) group_left (bar) b`)
-	another(`with (f(x,y) = a + on (x,y) group_left (y,bar) b) f((foo),())`, `a + on (foo) group_left (bar) b`)
-	another(`with (f(x,y) = a + on (x,y) group_left (y,bar) b) f((foo,xx),())`, `a + on (foo, xx) group_left (bar) b`)
-
-	// Verify nested with exprs
-	another(`with (f(x) = (with(x=y) x) + x) f(z)`, `y + z`)
-	another(`with (x=foo) f(a, with (y=x) y)`, `f(a, foo)`)
-	another(`with (x=foo) a * x + (with (y=x) y) / y`, `(a * foo) + (foo / y)`)
-	another(`with (x = with (y = foo) y + x) x/x`, `(foo + x) / (foo + x)`)
-	another(`with (
-		x = {foo="bar"},
-		q = m{x, y="1"},
-		f(x) =
-			with (
-				z(y) = x + y * q
-			)
-			z(foo) / f(x)
-	)
-	f(a)`, `(a + (foo * m{foo="bar", y="1"})) / f(a)`)
-
-	// complex withExpr
-	another(`WITH (
-		treshold = (0.9),
-		commonFilters = {job="cacher", instance=~"1.2.3.4"},
-		hits = rate(cache{type="hit", commonFilters}[5m]),
-		miss = rate(cache{type="miss", commonFilters}[5m]),
-		sumByInstance(arg) = sum(arg) by (instance),
-		hitRatio = sumByInstance(hits) / sumByInstance(hits + miss)
-	)
-	hitRatio < treshold`,
-		`(sum(rate(cache{type="hit", job="cacher", instance=~"1.2.3.4"}[5m])) by (instance) / sum(rate(cache{type="hit", job="cacher", instance=~"1.2.3.4"}[5m]) + rate(cache{type="miss", job="cacher", instance=~"1.2.3.4"}[5m])) by (instance)) < 0.9`)
-	another(`WITH (
-		x2(x) = x^2,
-		f(x, y) = x2(x) + x*y + x2(y)
-	)
-	f(a, 3)
-	`, `((a ^ 2) + (a * 3)) + 9`)
-	another(`WITH (
-		x2(x) = x^2,
-		f(x, y) = x2(x) + x*y + x2(y)
-	)
-	f(2, 3)
-	`, `19`)
-	another(`WITH (
-		commonFilters = {instance="foo"},
-		timeToFuckup(currv, maxv) = (maxv - currv) / rate(currv)
-	)
-	timeToFuckup(diskUsage{commonFilters}, maxDiskSize{commonFilters})`,
-		`(maxDiskSize{instance="foo"} - diskUsage{instance="foo"}) / rate(diskUsage{instance="foo"})`)
-	another(`WITH (
-	       commonFilters = {job="foo", instance="bar"},
-	       sumRate(m, cf) = sum(rate(m{cf})) by (job, instance),
-	       hitRate(hits, misses) = sumRate(hits, commonFilters) / (sumRate(hits, commonFilters) + sumRate(misses, commonFilters))
-	   )
-	   hitRate(cacheHits, cacheMisses)`,
-		`sum(rate(cacheHits{job="foo", instance="bar"})) by (job, instance) / (sum(rate(cacheHits{job="foo", instance="bar"})) by (job, instance) + sum(rate(cacheMisses{job="foo", instance="bar"})) by (job, instance))`)
-	another(`with(y=123,z=5) union(with(y=3,f(x)=x*y) f(2) + f(3), with(x=5,y=2) x*y*z)`, `union(15, 50)`)
-}
-
-func TestParsePromQLError(t *testing.T) {
-	f := func(s string) {
-		t.Helper()
-
-		e, err := parsePromQL(s)
-		if err == nil {
-			t.Fatalf("expecting non-nil error when parsing %q", s)
-		}
-		if e != nil {
-			t.Fatalf("expecting nil expr when parsing %q", s)
-		}
-	}
-
-	// an empty string
-	f("")
-	f("  \t\b\r\n  ")
-
-	// invalid metricExpr
-	f(`{__name__="ff"} offset 55`)
-	f(`foo[55]`)
-	f(`m[-5m]`)
-	f(`{`)
-	f(`foo{`)
-	f(`foo{bar`)
-	f(`foo{bar=`)
-	f(`foo{bar="baz"`)
-	f(`foo{bar="baz",  `)
-	f(`foo{123="23"}`)
-	f(`foo{foo}`)
-	f(`foo{,}`)
-	f(`foo{,foo="bar"}`)
-	f(`foo{foo=}`)
-	f(`foo{foo="ba}`)
-	f(`foo{"foo"="bar"}`)
-	f(`foo{$`)
-	f(`foo{a $`)
-	f(`foo{a="b",$`)
-	f(`foo{a="b"}$`)
-	f(`[`)
-	f(`[]`)
-	f(`f[5m]$`)
-	f(`[5m]`)
-	f(`[5m] offset 4h`)
-	f(`m[5m] offset $`)
-	f(`m[5m] offset 5h $`)
-	f(`m[]`)
-	f(`m[-5m]`)
-	f(`m[5m:`)
-	f(`m[5m:-`)
-	f(`m[5m:-1`)
-	f(`m[5m:-1]`)
-	f(`m[5m:-1s]`)
-	f(`m[-5m:1s]`)
-	f(`m[-5m:-1s]`)
-	f(`m[:`)
-	f(`m[:-`)
-	f(`m[:1]`)
-	f(`m[:-1m]`)
-	f(`m[5]`)
-	f(`m[[5m]]`)
-	f(`m[foo]`)
-	f(`m["ff"]`)
-	f(`m[10m`)
-	f(`m[123`)
-	f(`m["ff`)
-	f(`m[(f`)
-	f(`fd}`)
-	f(`]`)
-	f(`m $`)
-	f(`m{,}`)
-	f(`m{x=y}`)
-	f(`m{x=y/5}`)
-	f(`m{x=y+5}`)
-
-	// Invalid regexp
-	f(`foo{bar=~"x["}`)
-	f(`foo{bar=~"x("}`)
-	f(`foo{bar=~"x)"}`)
-	f(`foo{bar!~"x["}`)
-	f(`foo{bar!~"x("}`)
-	f(`foo{bar!~"x)"}`)
-
-	// invalid stringExpr
-	f(`'`)
-	f(`"`)
-	f("`")
-	f(`"foo`)
-	f(`'foo`)
-	f("`foo")
-	f(`"foo\"bar`)
-	f(`'foo\'bar`)
-	f("`foo\\`bar")
-	f(`"" $`)
-	f(`"foo" +`)
-	f(`n{"foo" + m`)
-
-	// invalid numberExpr
-	f(`12.`)
-	f(`1.2e`)
-	f(`23e-`)
-	f(`23E+`)
-	f(`.`)
-	f(`-12.`)
-	f(`-1.2e`)
-	f(`-23e-`)
-	f(`-23E+`)
-	f(`-.`)
-	f(`-1$$`)
-	f(`-$$`)
-	f(`+$$`)
-	f(`23 $$`)
-
-	// invalid binaryOpExpr
-	f(`+`)
-	f(`1 +`)
-	f(`1 + 2.`)
-	f(`3 unless`)
-	f(`23 + on (foo)`)
-	f(`m + on (,) m`)
-	f(`3 * ignoring`)
-	f(`m * on (`)
-	f(`m * on (foo`)
-	f(`m * on (foo,`)
-	f(`m * on (foo,)`)
-	f(`m * on (,foo)`)
-	f(`m * on (,)`)
-	f(`m == bool (bar) baz`)
-	f(`m == bool () baz`)
-	f(`m * by (baz) n`)
-	f(`m + bool group_left m2`)
-	f(`m + on () group_left (`)
-	f(`m + on () group_left (,`)
-	f(`m + on () group_left (,foo`)
-	f(`m + on () group_left (foo,)`)
-	f(`m + on () group_left (,foo)`)
-	f(`m + on () group_left (foo)`)
-	f(`m + on () group_right (foo) (m`)
-	f(`m or ignoring () group_left () n`)
-	f(`1 + bool 2`)
-	f(`m % bool n`)
-	f(`m * bool baz`)
-	f(`M * BOoL BaZ`)
-	f(`foo unless ignoring (bar) group_left xxx`)
-	f(`foo or bool bar`)
-	f(`foo == bool $$`)
-	f(`"foo" + bar`)
-
-	// invalid parensExpr
-	f(`(`)
-	f(`($`)
-	f(`(+`)
-	f(`(1`)
-	f(`(m+`)
-	f(`1)`)
-	f(`(,)`)
-	f(`(1)$`)
-
-	// invalid funcExpr
-	f(`f $`)
-	f(`f($)`)
-	f(`f[`)
-	f(`f()$`)
-	f(`f(`)
-	f(`f(foo`)
-	f(`f(f,`)
-	f(`f(,`)
-	f(`f(,)`)
-	f(`f(,foo)`)
-	f(`f(,foo`)
-	f(`f(foo,$`)
-	f(`f() by (a)`)
-	f(`f without (x) (y)`)
-	f(`f() foo (a)`)
-	f(`f bar (x) (b)`)
-	f(`f bar (x)`)
-
-	// invalid aggrFuncExpr
-	f(`sum(`)
-	f(`sum $`)
-	f(`sum [`)
-	f(`sum($)`)
-	f(`sum()$`)
-	f(`sum(foo) ba`)
-	f(`sum(foo) ba()`)
-	f(`sum(foo) by`)
-	f(`sum(foo) without x`)
-	f(`sum(foo) aaa`)
-	f(`sum(foo) aaa x`)
-	f(`sum() by $`)
-	f(`sum() by (`)
-	f(`sum() by ($`)
-	f(`sum() by (a`)
-	f(`sum() by (a $`)
-	f(`sum() by (a ]`)
-	f(`sum() by (a)$`)
-	f(`sum() by (,`)
-	f(`sum() by (a,$`)
-	f(`sum() by (,)`)
-	f(`sum() by (,a`)
-	f(`sum() by (,a)`)
-	f(`sum() on (b)`)
-	f(`sum() bool`)
-	f(`sum() group_left`)
-	f(`sum() group_right(x)`)
-	f(`sum ba`)
-	f(`sum ba ()`)
-	f(`sum by (`)
-	f(`sum by (a`)
-	f(`sum by (,`)
-	f(`sum by (,)`)
-	f(`sum by (,a`)
-	f(`sum by (,a)`)
-	f(`sum by (a)`)
-	f(`sum by (a) (`)
-	f(`sum by (a) [`)
-	f(`sum by (a) {`)
-	f(`sum by (a) (b`)
-	f(`sum by (a) (b,`)
-	f(`sum by (a) (,)`)
-	f(`avg by (a) (,b)`)
-	f(`sum by (x) (y) by (z)`)
-	f(`sum(m) by (1)`)
-
-	// invalid withExpr
-	f(`with $`)
-	f(`with a`)
-	f(`with a=b c`)
-	f(`with (`)
-	f(`with (x=b)$`)
-	f(`with ($`)
-	f(`with (foo`)
-	f(`with (foo $`)
-	f(`with (x y`)
-	f(`with (x =`)
-	f(`with (x = $`)
-	f(`with (x= y`)
-	f(`with (x= y $`)
-	f(`with (x= y)`)
-	f(`with (x=(`)
-	f(`with (x=[)`)
-	f(`with (x=() x)`)
-	f(`with ($$)`)
-	f(`with (x $$`)
-	f(`with (x = $$)`)
-	f(`with (x = foo) bar{x}`)
-	f(`with (x = {foo="bar"}[5m]) bar{x}`)
-	f(`with (x = {foo="bar"} offset 5m) bar{x}`)
-	f(`with (x = a, x = b) c`)
-	f(`with (x(a, a) = b) c`)
-	f(`with (x=m{f="x"}) foo{x}`)
-	f(`with (sum = x) y`)
-	f(`with (rate(a) = b) c`)
-	f(`with (clamp_min=x) y`)
-	f(`with (f()`)
-	f(`with (a=b c=d) e`)
-	f(`with (f(x)=x^2) m{x}`)
-	f(`with (f(x)=ff()) m{x}`)
-	f(`with (f(x`)
-	f(`with (x=m) a{x} + b`)
-	f(`with (x=m) b + a{x}`)
-	f(`with (x=m) f(b, a{x})`)
-	f(`with (x=m) sum(a{x})`)
-	f(`with (x=m) (a{x})`)
-	f(`with (f(a)=a) f(1, 2)`)
-	f(`with (f(x)=x{foo="bar"}) f(1)`)
-	f(`with (f(x)=x{foo="bar"}) f(m + n)`)
-	f(`with (f = with`)
-	f(`with (,)`)
-	f(`with (1) 2`)
-	f(`with (f(1)=2) 3`)
-	f(`with (f(,)=x) x`)
-	f(`with (x(a) = {b="c"}) foo{x}`)
-	f(`with (f(x) = m{foo=xx}) f("qwe")`)
-	f(`a + with(f(x)=x) f(1,2)`)
-	f(`with (f(x) = sum(m) by (x)) f({foo="bar"})`)
-	f(`with (f(x) = sum(m) by (x)) f((xx(), {foo="bar"}))`)
-	f(`with (f(x) = m + on (x) n) f(xx())`)
-	f(`with (f(x) = m + on (a) group_right (x) n) f(xx())`)
-}
--- a/app/vmselect/promql/rollup.go
+++ b/app/vmselect/promql/rollup.go
@@ -8,6 +8,9 @@ import (

 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/decimal"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
+	"github.com/VictoriaMetrics/metrics"
 	"github.com/valyala/histogram"
 )

@@ -35,6 +38,7 @@ var rollupFuncs = map[string]newRollupFunc{
 	"quantile_over_time": newRollupQuantile,
 	"stddev_over_time":   newRollupFuncOneArg(rollupStddev),
 	"stdvar_over_time":   newRollupFuncOneArg(rollupStdvar),
+	"absent_over_time":   newRollupFuncOneArg(rollupAbsent),

 	// Additional rollup funcs.
 	"sum2_over_time":      newRollupFuncOneArg(rollupSum2),
@@ -49,12 +53,56 @@ var rollupFuncs = map[string]newRollupFunc{
 	"lifetime":            newRollupFuncOneArg(rollupLifetime),
 	"lag":                 newRollupFuncOneArg(rollupLag),
 	"scrape_interval":     newRollupFuncOneArg(rollupScrapeInterval),
+	"tmin_over_time":      newRollupFuncOneArg(rollupTmin),
+	"tmax_over_time":      newRollupFuncOneArg(rollupTmax),
+	"share_le_over_time":  newRollupShareLE,
+	"share_gt_over_time":  newRollupShareGT,
+	"histogram_over_time": newRollupFuncOneArg(rollupHistogram),
 	"rollup":              newRollupFuncOneArg(rollupFake),
 	"rollup_rate":         newRollupFuncOneArg(rollupFake), // + rollupFuncsRemoveCounterResets
 	"rollup_deriv":        newRollupFuncOneArg(rollupFake),
 	"rollup_delta":        newRollupFuncOneArg(rollupFake),
 	"rollup_increase":     newRollupFuncOneArg(rollupFake), // + rollupFuncsRemoveCounterResets
 	"rollup_candlestick":  newRollupFuncOneArg(rollupFake),
+	"aggr_over_time":      newRollupFuncTwoArgs(rollupFake),
+}
+
+// rollupAggrFuncs are functions that can be passed to `aggr_over_time()`
+var rollupAggrFuncs = map[string]rollupFunc{
+	// Standard rollup funcs from PromQL.
+	"changes":          rollupChanges,
+	"delta":            rollupDelta,
+	"deriv":            rollupDerivSlow,
+	"deriv_fast":       rollupDerivFast,
+	"idelta":           rollupIdelta,
+	"increase":         rollupIncrease,  // + rollupFuncsRemoveCounterResets
+	"irate":            rollupIderiv,    // + rollupFuncsRemoveCounterResets
+	"rate":             rollupDerivFast, // + rollupFuncsRemoveCounterResets
+	"resets":           rollupResets,
+	"avg_over_time":    rollupAvg,
+	"min_over_time":    rollupMin,
+	"max_over_time":    rollupMax,
+	"sum_over_time":    rollupSum,
+	"count_over_time":  rollupCount,
+	"stddev_over_time": rollupStddev,
+	"stdvar_over_time": rollupStdvar,
+	"absent_over_time": rollupAbsent,
+
+	// Additional rollup funcs.
+	"sum2_over_time":      rollupSum2,
+	"geomean_over_time":   rollupGeomean,
+	"first_over_time":     rollupFirst,
+	"last_over_time":      rollupLast,
+	"distinct_over_time":  rollupDistinct,
+	"increases_over_time": rollupIncreases,
+	"decreases_over_time": rollupDecreases,
+	"integrate":           rollupIntegrate,
+	"ideriv":              rollupIderiv,
+	"lifetime":            rollupLifetime,
+	"lag":                 rollupLag,
+	"scrape_interval":     rollupScrapeInterval,
+	"tmin_over_time":      rollupTmin,
+	"tmax_over_time":      rollupTmax,
 }

 var rollupFuncsMayAdjustWindow = map[string]bool{
@@ -66,6 +114,7 @@ var rollupFuncsMayAdjustWindow = map[string]bool{
 	"irate":           true,
 	"rate":            true,
 	"lifetime":        true,
+	"lag":             true,
 	"scrape_interval": true,
 }

@@ -87,15 +136,137 @@ var rollupFuncsKeepMetricGroup = map[string]bool{
 	"geomean_over_time":  true,
 }

+func getRollupAggrFuncNames(expr metricsql.Expr) ([]string, error) {
+	afe, ok := expr.(*metricsql.AggrFuncExpr)
+	if ok {
+		// This is for incremental aggregate function case:
+		//
+		//     sum(aggr_over_time(...))
+		//
+		// See aggr_incremental.go for details.
+		expr = afe.Args[0]
+	}
+	fe, ok := expr.(*metricsql.FuncExpr)
+	if !ok {
+		logger.Panicf("BUG: unexpected expression; want metricsql.FuncExpr; got %T; value: %s", expr, expr.AppendString(nil))
+	}
+	if fe.Name != "aggr_over_time" {
+		logger.Panicf("BUG: unexpected function name: %q; want `aggr_over_time`", fe.Name)
+	}
+	if len(fe.Args) != 2 {
+		return nil, fmt.Errorf("unexpected number of args to aggr_over_time(); got %d; want %d", len(fe.Args), 2)
+	}
+	arg := fe.Args[0]
+	var aggrFuncNames []string
+	if se, ok := arg.(*metricsql.StringExpr); ok {
+		aggrFuncNames = append(aggrFuncNames, se.S)
+	} else {
+		fe, ok := arg.(*metricsql.FuncExpr)
+		if !ok || fe.Name != "" {
+			return nil, fmt.Errorf("%s cannot be passed to aggr_over_time(); expecting quoted aggregate function name or a list of quoted aggregate function names",
+				arg.AppendString(nil))
+		}
+		for _, e := range fe.Args {
+			se, ok := e.(*metricsql.StringExpr)
+			if !ok {
+				return nil, fmt.Errorf("%s cannot be passed here; expecting quoted aggregate function name", e.AppendString(nil))
+			}
+			aggrFuncNames = append(aggrFuncNames, se.S)
+		}
+	}
+	if len(aggrFuncNames) == 0 {
+		return nil, fmt.Errorf("aggr_over_time() must contain at least a single aggregate function name")
+	}
+	for _, s := range aggrFuncNames {
+		if rollupAggrFuncs[s] == nil {
+			return nil, fmt.Errorf("%q cannot be used in `aggr_over_time` function; expecting quoted aggregate function name", s)
+		}
+	}
+	return aggrFuncNames, nil
+}
+
 func getRollupArgIdx(funcName string) int {
 	funcName = strings.ToLower(funcName)
 	if rollupFuncs[funcName] == nil {
 		logger.Panicf("BUG: getRollupArgIdx is called for non-rollup func %q", funcName)
 	}
-	if funcName == "quantile_over_time" {
+	switch funcName {
+	case "quantile_over_time", "aggr_over_time":
 		return 1
+	default:
+		return 0
 	}
-	return 0
+}
+
+func getRollupConfigs(name string, rf rollupFunc, expr metricsql.Expr, start, end, step, window int64, lookbackDelta int64, sharedTimestamps []int64) (
+	func(values []float64, timestamps []int64), []*rollupConfig, error) {
+	preFunc := func(values []float64, timestamps []int64) {}
+	if rollupFuncsRemoveCounterResets[name] {
+		preFunc = func(values []float64, timestamps []int64) {
+			removeCounterResets(values)
+		}
+	}
+	newRollupConfig := func(rf rollupFunc, tagValue string) *rollupConfig {
+		return &rollupConfig{
+			TagValue:        tagValue,
+			Func:            rf,
+			Start:           start,
+			End:             end,
+			Step:            step,
+			Window:          window,
+			MayAdjustWindow: rollupFuncsMayAdjustWindow[name],
+			LookbackDelta:   lookbackDelta,
+			Timestamps:      sharedTimestamps,
+		}
+	}
+	appendRollupConfigs := func(dst []*rollupConfig) []*rollupConfig {
+		dst = append(dst, newRollupConfig(rollupMin, "min"))
+		dst = append(dst, newRollupConfig(rollupMax, "max"))
+		dst = append(dst, newRollupConfig(rollupAvg, "avg"))
+		return dst
+	}
+	var rcs []*rollupConfig
+	switch name {
+	case "rollup":
+		rcs = appendRollupConfigs(rcs)
+	case "rollup_rate", "rollup_deriv":
+		preFuncPrev := preFunc
+		preFunc = func(values []float64, timestamps []int64) {
+			preFuncPrev(values, timestamps)
+			derivValues(values, timestamps)
+		}
+		rcs = appendRollupConfigs(rcs)
+	case "rollup_increase", "rollup_delta":
+		preFuncPrev := preFunc
+		preFunc = func(values []float64, timestamps []int64) {
+			preFuncPrev(values, timestamps)
+			deltaValues(values)
+		}
+		rcs = appendRollupConfigs(rcs)
+	case "rollup_candlestick":
+		rcs = append(rcs, newRollupConfig(rollupFirst, "open"))
+		rcs = append(rcs, newRollupConfig(rollupLast, "close"))
+		rcs = append(rcs, newRollupConfig(rollupMin, "low"))
+		rcs = append(rcs, newRollupConfig(rollupMax, "high"))
+	case "aggr_over_time":
+		aggrFuncNames, err := getRollupAggrFuncNames(expr)
+		if err != nil {
+			return nil, nil, fmt.Errorf("invalid args to %s: %s", expr.AppendString(nil), err)
+		}
+		for _, aggrFuncName := range aggrFuncNames {
+			if rollupFuncsRemoveCounterResets[aggrFuncName] {
+				// There is no need to save the previous preFunc, since it is either empty or the same.
+				preFunc = func(values []float64, timestamps []int64) {
+					removeCounterResets(values)
+				}
+			}
+			rf := rollupAggrFuncs[aggrFuncName]
+			rcs = append(rcs, newRollupConfig(rf, aggrFuncName))
+		}
+	default:
+		rcs = append(rcs, newRollupConfig(rf, ""))
+	}
+	return preFunc, rcs, nil
 }

 func getRollupFunc(funcName string) newRollupFunc {
@@ -103,10 +274,6 @@ func getRollupFunc(funcName string) newRollupFunc {
 	return rollupFuncs[funcName]
 }

-func isRollupFunc(funcName string) bool {
-	return getRollupFunc(funcName) != nil
-}
-
 type rollupFuncArg struct {
 	prevValue     float64
 	prevTimestamp int64
@@ -116,7 +283,12 @@ type rollupFuncArg struct {
 	currTimestamp int64
 	idx           int
 	step          int64
+
+	// Real previous value even if it is located too far from the current window.
+	// It matches prevValue if prevValue is not nan.
 	realPrevValue float64
+
+	tsm *timeseriesMap
 }

 func (rfa *rollupFuncArg) reset() {
@@ -128,6 +300,7 @@ func (rfa *rollupFuncArg) reset() {
 	rfa.idx = 0
 	rfa.step = 0
 	rfa.realPrevValue = nan
+	rfa.tsm = nil
 }

 // rollupFunc must return rollup value for the given rfa.
@@ -166,6 +339,54 @@ var (
 // The maximum interval without previous rows.
 const maxSilenceInterval = 5 * 60 * 1000

+type timeseriesMap struct {
+	origin    *timeseries
+	labelName string
+	h         metrics.Histogram
+	m         map[string]*timeseries
+}
+
+func newTimeseriesMap(funcName string, sharedTimestamps []int64, mnSrc *storage.MetricName) *timeseriesMap {
+	if funcName != "histogram_over_time" {
+		return nil
+	}
+
+	values := make([]float64, len(sharedTimestamps))
+	for i := range values {
+		values[i] = nan
+	}
+	var origin timeseries
+	origin.MetricName.CopyFrom(mnSrc)
+	origin.MetricName.ResetMetricGroup()
+	origin.Timestamps = sharedTimestamps
+	origin.Values = values
+	return &timeseriesMap{
+		origin:    &origin,
+		labelName: "vmrange",
+		m:         make(map[string]*timeseries),
+	}
+}
+
+func (tsm *timeseriesMap) AppendTimeseriesTo(dst []*timeseries) []*timeseries {
+	for _, ts := range tsm.m {
+		dst = append(dst, ts)
+	}
+	return dst
+}
+
+func (tsm *timeseriesMap) GetOrCreateTimeseries(labelValue string) *timeseries {
+	ts := tsm.m[labelValue]
+	if ts != nil {
+		return ts
+	}
+	ts = &timeseries{}
+	ts.CopyFromShallowTimestamps(tsm.origin)
+	ts.MetricName.RemoveTag(tsm.labelName)
+	ts.MetricName.AddTag(tsm.labelName, labelValue)
+	tsm.m[labelValue] = ts
+	return ts
+}
+
 // Do calculates rollups for the given timestamps and values, appends
 // them to dstValues and returns results.
 //
@@ -173,8 +394,19 @@ const maxSilenceInterval = 5 * 60 * 1000
 //
 // timestamps must cover time range [rc.Start - rc.Window - maxSilenceInterval ... rc.End + rc.Step].
 //
-// Cannot be called from concurrent goroutines.
+// Do cannot be called from concurrent goroutines.
 func (rc *rollupConfig) Do(dstValues []float64, values []float64, timestamps []int64) []float64 {
+	return rc.doInternal(dstValues, nil, values, timestamps)
+}
+
+// DoTimeseriesMap calculates rollups for the given timestamps and values and puts them to tsm.
+func (rc *rollupConfig) DoTimeseriesMap(tsm *timeseriesMap, values []float64, timestamps []int64) {
+	ts := getTimeseries()
+	ts.Values = rc.doInternal(ts.Values[:0], tsm, values, timestamps)
+	putTimeseries(ts)
+}
+
+func (rc *rollupConfig) doInternal(dstValues []float64, tsm *timeseriesMap, values []float64, timestamps []int64) []float64 {
 	// Sanity checks.
 	if rc.Step <= 0 {
 		logger.Panicf("BUG: Step must be bigger than 0; got %d", rc.Step)
@@ -192,7 +424,8 @@ func (rc *rollupConfig) Do(dstValues []float64, values []float64, timestamps []i
 	// Extend dstValues in order to remove mallocs below.
 	dstValues = decimal.ExtendFloat64sCapacity(dstValues, len(rc.Timestamps))

-	maxPrevInterval := getMaxPrevInterval(timestamps)
+	scrapeInterval := getScrapeInterval(timestamps)
+	maxPrevInterval := getMaxPrevInterval(scrapeInterval)
 	if rc.LookbackDelta > 0 && maxPrevInterval > rc.LookbackDelta {
 		maxPrevInterval = rc.LookbackDelta
 	}
@@ -207,6 +440,7 @@ func (rc *rollupConfig) Do(dstValues []float64, values []float64, timestamps []i
 	rfa.idx = 0
 	rfa.step = rc.Step
 	rfa.realPrevValue = nan
+	rfa.tsm = tsm

 	i := 0
 	j := 0
@@ -296,7 +530,7 @@ func binarySearchInt64(a []int64, v int64) uint {
 	return i
 }

-func getMaxPrevInterval(timestamps []int64) int64 {
+func getScrapeInterval(timestamps []int64) int64 {
 	if len(timestamps) < 2 {
 		return int64(maxSilenceInterval)
 	}
@@ -312,30 +546,34 @@ func getMaxPrevInterval(timestamps []int64) int64 {
 		h.Update(float64(ts - tsPrev))
 		tsPrev = ts
 	}
-	d := int64(h.Quantile(0.6))
+	scrapeInterval := int64(h.Quantile(0.6))
 	histogram.PutFast(h)
-	if d <= 0 {
+	if scrapeInterval <= 0 {
 		return int64(maxSilenceInterval)
 	}
-	// Increase d more for smaller scrape intervals in order to hide possible gaps
+	return scrapeInterval
+}
+
+func getMaxPrevInterval(scrapeInterval int64) int64 {
+	// Increase scrapeInterval more for smaller scrape intervals in order to hide possible gaps
 	// when high jitter is present.
 	// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/139 .
-	if d <= 2*1000 {
-		return d + 4*d
+	if scrapeInterval <= 2*1000 {
+		return scrapeInterval + 4*scrapeInterval
 	}
-	if d <= 4*1000 {
-		return d + 2*d
+	if scrapeInterval <= 4*1000 {
+		return scrapeInterval + 2*scrapeInterval
 	}
-	if d <= 8*1000 {
-		return d + d
+	if scrapeInterval <= 8*1000 {
+		return scrapeInterval + scrapeInterval
 	}
-	if d <= 16*1000 {
-		return d + d/2
+	if scrapeInterval <= 16*1000 {
+		return scrapeInterval + scrapeInterval/2
 	}
-	if d <= 32*1000 {
-		return d + d/4
+	if scrapeInterval <= 32*1000 {
+		return scrapeInterval + scrapeInterval/4
 	}
-	return d + d/8
+	return scrapeInterval + scrapeInterval/8
 }

 func removeCounterResets(values []float64) {
@@ -414,6 +652,15 @@ func newRollupFuncOneArg(rf rollupFunc) newRollupFunc {
 	}
 }

+func newRollupFuncTwoArgs(rf rollupFunc) newRollupFunc {
+	return func(args []interface{}) (rollupFunc, error) {
+		if err := expectRollupArgsNum(args, 2); err != nil {
+			return nil, err
+		}
+		return rf, nil
+	}
+}
+
 func newRollupHoltWinters(args []interface{}) (rollupFunc, error) {
 	if err := expectRollupArgsNum(args, 3); err != nil {
 		return nil, err
@@ -522,6 +769,56 @@ func linearRegression(rfa *rollupFuncArg) (float64, float64) {
 	return v, k
 }

+func newRollupShareLE(args []interface{}) (rollupFunc, error) {
+	return newRollupShareFilter(args, countFilterLE)
+}
+
+func countFilterLE(values []float64, le float64) int {
+	n := 0
+	for _, v := range values {
+		if v <= le {
+			n++
+		}
+	}
+	return n
+}
+
+func newRollupShareGT(args []interface{}) (rollupFunc, error) {
+	return newRollupShareFilter(args, countFilterGT)
+}
+
+func countFilterGT(values []float64, gt float64) int {
+	n := 0
+	for _, v := range values {
+		if v > gt {
+			n++
+		}
+	}
+	return n
+}
+
+func newRollupShareFilter(args []interface{}, countFilter func(values []float64, limit float64) int) (rollupFunc, error) {
+	if err := expectRollupArgsNum(args, 2); err != nil {
+		return nil, err
+	}
+	limits, err := getScalar(args[1], 1)
+	if err != nil {
+		return nil, err
+	}
+	rf := func(rfa *rollupFuncArg) float64 {
+		// There is no need in handling NaNs here, since they must be cleaned up
+		// before calling rollup funcs.
+		values := rfa.values
+		if len(values) == 0 {
+			return nan
+		}
+		limit := limits[rfa.idx]
+		n := countFilter(values, limit)
+		return float64(n) / float64(len(values))
+	}
+	return rf, nil
+}
+
 func newRollupQuantile(args []interface{}) (rollupFunc, error) {
 	if err := expectRollupArgsNum(args, 2); err != nil {
 		return nil, err
@@ -553,6 +850,21 @@ func newRollupQuantile(args []interface{}) (rollupFunc, error) {
 	return rf, nil
 }

+func rollupHistogram(rfa *rollupFuncArg) float64 {
+	values := rfa.values
+	tsm := rfa.tsm
+	tsm.h.Reset()
+	for _, v := range values {
+		tsm.h.Update(v)
+	}
+	idx := rfa.idx
+	tsm.h.VisitNonZeroBuckets(func(vmrange string, count uint64) {
+		ts := tsm.GetOrCreateTimeseries(vmrange)
+		ts.Values[idx] = float64(count)
+	})
+	return nan
+}
+
 func rollupAvg(rfa *rollupFuncArg) float64 {
 	// Do not use `Rapid calculation methods` at https://en.wikipedia.org/wiki/Standard_deviation,
 	// since it is slower and has no significant benefits in precision.
@@ -573,14 +885,14 @@ func rollupAvg(rfa *rollupFuncArg) float64 {
 func rollupMin(rfa *rollupFuncArg) float64 {
 	// There is no need in handling NaNs here, since they must be cleaned up
 	// before calling rollup funcs.
-	minValue := rfa.prevValue
 	values := rfa.values
-	if math.IsNaN(minValue) {
-		if len(values) == 0 {
-			return nan
-		}
-		minValue = values[0]
+	if len(values) == 0 {
+		// Do not take into account rfa.prevValue, since it may lead
+		// to inconsistent results comparing to Prometheus on broken time series
+		// with irregular data points.
+		return nan
 	}
+	minValue := values[0]
 	for _, v := range values {
 		if v < minValue {
 			minValue = v
@@ -592,14 +904,14 @@ func rollupMin(rfa *rollupFuncArg) float64 {
 func rollupMax(rfa *rollupFuncArg) float64 {
 	// There is no need in handling NaNs here, since they must be cleaned up
 	// before calling rollup funcs.
-	maxValue := rfa.prevValue
 	values := rfa.values
-	if math.IsNaN(maxValue) {
-		if len(values) == 0 {
-			return nan
-		}
-		maxValue = values[0]
+	if len(values) == 0 {
+		// Do not take into account rfa.prevValue, since it may lead
+		// to inconsistent results comparing to Prometheus on broken time series
+		// with irregular data points.
+		return nan
 	}
+	maxValue := values[0]
 	for _, v := range values {
 		if v > maxValue {
 			maxValue = v
@@ -608,6 +920,44 @@ func rollupMax(rfa *rollupFuncArg) float64 {
 	return maxValue
 }

+func rollupTmin(rfa *rollupFuncArg) float64 {
+	// There is no need in handling NaNs here, since they must be cleaned up
+	// before calling rollup funcs.
+	values := rfa.values
+	timestamps := rfa.timestamps
+	if len(values) == 0 {
+		return nan
+	}
+	minValue := values[0]
+	minTimestamp := timestamps[0]
+	for i, v := range values {
+		if v < minValue {
+			minValue = v
+			minTimestamp = timestamps[i]
+		}
+	}
+	return float64(minTimestamp) * 1e-3
+}
+
+func rollupTmax(rfa *rollupFuncArg) float64 {
+	// There is no need in handling NaNs here, since they must be cleaned up
+	// before calling rollup funcs.
+	values := rfa.values
+	timestamps := rfa.timestamps
+	if len(values) == 0 {
+		return nan
+	}
+	maxValue := values[0]
+	maxTimestamp := timestamps[0]
+	for i, v := range values {
+		if v > maxValue {
+			maxValue = v
+			maxTimestamp = timestamps[i]
+		}
+	}
+	return float64(maxTimestamp) * 1e-3
+}
+
 func rollupSum(rfa *rollupFuncArg) float64 {
 	// There is no need in handling NaNs here, since they must be cleaned up
 	// before calling rollup funcs.
@@ -653,6 +1003,13 @@ func rollupGeomean(rfa *rollupFuncArg) float64 {
 	return math.Pow(p, 1/float64(len(values)))
 }

+func rollupAbsent(rfa *rollupFuncArg) float64 {
+	if len(rfa.values) == 0 {
+		return 1
+	}
+	return nan
+}
+
 func rollupCount(rfa *rollupFuncArg) float64 {
 	// There is no need in handling NaNs here, since they must be cleaned up
 	// before calling rollup funcs.
@@ -716,16 +1073,11 @@ func rollupDeltaInternal(rfa *rollupFuncArg, canUseRealPrevValue bool) float64 {
 		if len(values) == 0 {
 			return nan
 		}
-		if len(values) == 1 {
-			if canUseRealPrevValue && !math.IsNaN(rfa.realPrevValue) {
-				// Fix against removeCounterResets.
-				return values[0] - rfa.realPrevValue
-			}
-			// Assume that the previous non-existing value was 0.
-			return values[0]
+		// Assume that the previous non-existing value was 0.
+		prevValue = 0
+		if canUseRealPrevValue && !math.IsNaN(rfa.prevValue) {
+			prevValue = rfa.prevValue
 		}
-		prevValue = values[0]
-		values = values[1:]
 	}
 	if len(values) == 0 {
 		// Assume that the value didn't change on the given interval.
@@ -773,16 +1125,25 @@ func rollupDerivFast(rfa *rollupFuncArg) float64 {
 	prevValue := rfa.prevValue
 	prevTimestamp := rfa.prevTimestamp
 	if math.IsNaN(prevValue) {
-		if len(values) < 2 {
-			// It is impossible to calculate derivative on 0 or 1 values.
+		if len(values) == 0 {
+			return nan
+		}
+		if len(values) == 1 {
+			// It is impossible to determine the duration during which the value changed
+			// from 0 to the current value.
+			// The following attempts didn't work well:
+			// - using scrape interval as the duration. It fails on Prometheus restarts when it
+			//   skips scraping for the counter. This results in too high rate() value for the first point
+			//   after Prometheus restarts.
+			// - using window or step as the duration. It results in too small rate() values for the first
+			//   points of time series.
+			//
+			// So just return nan
 			return nan
 		}
 		prevValue = values[0]
 		prevTimestamp = timestamps[0]
-		values = values[1:]
-		timestamps = timestamps[1:]
-	}
-	if len(values) == 0 {
+	} else if len(values) == 0 {
 		// Assume that the value didn't change on the given interval.
 		return 0
 	}
@@ -799,8 +1160,20 @@ func rollupIderiv(rfa *rollupFuncArg) float64 {
 	values := rfa.values
 	timestamps := rfa.timestamps
 	if len(values) < 2 {
-		if len(values) == 0 || math.IsNaN(rfa.prevValue) {
-			// It is impossible to calculate derivative on 0 or 1 values.
+		if len(values) == 0 {
+			return nan
+		}
+		if math.IsNaN(rfa.prevValue) {
+			// It is impossible to determine the duration during which the value changed
+			// from 0 to the current value.
+			// The following attempts didn't work well:
+			// - using scrape interval as the duration. It fails on Prometheus restarts when it
+			//   skips scraping for the counter. This results in too high rate() value for the first point
+			//   after Prometheus restarts.
+			// - using window or step as the duration. It results in too small rate() values for the first
+			//   points of time series.
+			//
+			// So just return nan
 			return nan
 		}
 		return (values[0] - rfa.prevValue) / (float64(timestamps[0]-rfa.prevTimestamp) * 1e-3)
@@ -955,16 +1328,13 @@ func rollupResets(rfa *rollupFuncArg) float64 {
 }

 func rollupFirst(rfa *rollupFuncArg) float64 {
-	// See https://prometheus.io/docs/prometheus/latest/querying/basics/#staleness
-	v := rfa.prevValue
-	if !math.IsNaN(v) {
-		return v
-	}
-
 	// There is no need in handling NaNs here, since they must be cleaned up
 	// before calling rollup funcs.
 	values := rfa.values
 	if len(values) == 0 {
+		// Do not take into account rfa.prevValue, since it may lead
+		// to inconsistent results comparing to Prometheus on broken time series
+		// with irregular data points.
 		return nan
 	}
 	return values[0]
@@ -977,7 +1347,10 @@ func rollupLast(rfa *rollupFuncArg) float64 {
 	// before calling rollup funcs.
 	values := rfa.values
 	if len(values) == 0 {
-		return rfa.prevValue
+		// Do not take into account rfa.prevValue, since it may lead
+		// to inconsistent results comparing to Prometheus on broken time series
+		// with irregular data points.
+		return nan
 	}
 	return values[len(values)-1]
 }
--- a/app/vmselect/promql/rollup_result_cache.go
+++ b/app/vmselect/promql/rollup_result_cache.go
@@ -12,6 +12,7 @@ import (
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/encoding"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/memory"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/workingsetcache"
 	"github.com/VictoriaMetrics/fastcache"
 	"github.com/VictoriaMetrics/metrics"
@@ -128,7 +129,7 @@ func ResetRollupResultCache() {
 	rollupResultCacheV.c.Reset()
 }

-func (rrc *rollupResultCache) Get(funcName string, ec *EvalConfig, me *metricExpr, iafc *incrementalAggrFuncContext, window int64) (tss []*timeseries, newStart int64) {
+func (rrc *rollupResultCache) Get(ec *EvalConfig, expr metricsql.Expr, window int64) (tss []*timeseries, newStart int64) {
 	if *disableCache || !ec.mayCache() {
 		return nil, ec.Start
 	}
@@ -137,7 +138,7 @@ func (rrc *rollupResultCache) Get(funcName string, ec *EvalConfig, me *metricExp
 	bb := bbPool.Get()
 	defer bbPool.Put(bb)

-	bb.B = marshalRollupResultCacheKey(bb.B[:0], funcName, me, iafc, window, ec.Step)
+	bb.B = marshalRollupResultCacheKey(bb.B[:0], expr, window, ec.Step)
 	metainfoBuf := rrc.c.Get(nil, bb.B)
 	if len(metainfoBuf) == 0 {
 		return nil, ec.Start
@@ -157,7 +158,7 @@ func (rrc *rollupResultCache) Get(funcName string, ec *EvalConfig, me *metricExp
 	if len(compressedResultBuf.B) == 0 {
 		mi.RemoveKey(key)
 		metainfoBuf = mi.Marshal(metainfoBuf[:0])
-		bb.B = marshalRollupResultCacheKey(bb.B[:0], funcName, me, iafc, window, ec.Step)
+		bb.B = marshalRollupResultCacheKey(bb.B[:0], expr, window, ec.Step)
 		rrc.c.Set(bb.B, metainfoBuf)
 		return nil, ec.Start
 	}
@@ -209,7 +210,7 @@ func (rrc *rollupResultCache) Get(funcName string, ec *EvalConfig, me *metricExp

 var resultBufPool bytesutil.ByteBufferPool

-func (rrc *rollupResultCache) Put(funcName string, ec *EvalConfig, me *metricExpr, iafc *incrementalAggrFuncContext, window int64, tss []*timeseries) {
+func (rrc *rollupResultCache) Put(ec *EvalConfig, expr metricsql.Expr, window int64, tss []*timeseries) {
 	if *disableCache || len(tss) == 0 || !ec.mayCache() {
 		return
 	}
@@ -260,7 +261,7 @@ func (rrc *rollupResultCache) Put(funcName string, ec *EvalConfig, me *metricExp
 	bb.B = key.Marshal(bb.B[:0])
 	rrc.c.SetBig(bb.B, compressedResultBuf.B)

-	bb.B = marshalRollupResultCacheKey(bb.B[:0], funcName, me, iafc, window, ec.Step)
+	bb.B = marshalRollupResultCacheKey(bb.B[:0], expr, window, ec.Step)
 	metainfoBuf := rrc.c.Get(nil, bb.B)
 	var mi rollupResultCacheMetainfo
 	if len(metainfoBuf) > 0 {
@@ -288,23 +289,13 @@ var (
 var tooBigRollupResults = metrics.NewCounter("vm_too_big_rollup_results_total")

 // Increment this value every time the format of the cache changes.
-const rollupResultCacheVersion = 6
+const rollupResultCacheVersion = 7

-func marshalRollupResultCacheKey(dst []byte, funcName string, me *metricExpr, iafc *incrementalAggrFuncContext, window, step int64) []byte {
+func marshalRollupResultCacheKey(dst []byte, expr metricsql.Expr, window, step int64) []byte {
 	dst = append(dst, rollupResultCacheVersion)
-	if iafc == nil {
-		dst = append(dst, 0)
-	} else {
-		dst = append(dst, 1)
-		dst = iafc.ae.AppendString(dst)
-	}
-	dst = encoding.MarshalUint64(dst, uint64(len(funcName)))
-	dst = append(dst, funcName...)
 	dst = encoding.MarshalInt64(dst, window)
 	dst = encoding.MarshalInt64(dst, step)
-	for i := range me.TagFilters {
-		dst = me.TagFilters[i].Marshal(dst)
-	}
+	dst = expr.AppendString(dst)
 	return dst
 }

--- a/app/vmselect/promql/rollup_result_cache_test.go
+++ b/app/vmselect/promql/rollup_result_cache_test.go
@@ -3,12 +3,12 @@ package promql
 import (
 	"testing"

+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
 )

 func TestRollupResultCache(t *testing.T) {
 	ResetRollupResultCache()
-	funcName := "foo"
 	window := int64(456)
 	ec := &EvalConfig{
 		Start: 1000,
@@ -17,21 +17,24 @@ func TestRollupResultCache(t *testing.T) {

 		MayCache: true,
 	}
-	me := &metricExpr{
-		TagFilters: []storage.TagFilter{{
-			Key:   []byte("aaa"),
-			Value: []byte("xxx"),
+	me := &metricsql.MetricExpr{
+		LabelFilters: []metricsql.LabelFilter{{
+			Label: "aaa",
+			Value: "xxx",
 		}},
 	}
-	iafc := &incrementalAggrFuncContext{
-		ae: &aggrFuncExpr{
-			Name: "foobar",
-		},
+	fe := &metricsql.FuncExpr{
+		Name: "foo",
+		Args: []metricsql.Expr{me},
+	}
+	ae := &metricsql.AggrFuncExpr{
+		Name: "foobar",
+		Args: []metricsql.Expr{fe},
 	}

 	// Try obtaining an empty value.
 	t.Run("empty", func(t *testing.T) {
-		tss, newStart := rollupResultCacheV.Get(funcName, ec, me, nil, window)
+		tss, newStart := rollupResultCacheV.Get(ec, fe, window)
 		if newStart != ec.Start {
 			t.Fatalf("unexpected newStart; got %d; want %d", newStart, ec.Start)
 		}
@@ -41,7 +44,7 @@ func TestRollupResultCache(t *testing.T) {
 	})

 	// Store timeseries overlapping with start
-	t.Run("start-overlap-no-iafc", func(t *testing.T) {
+	t.Run("start-overlap-no-ae", func(t *testing.T) {
 		ResetRollupResultCache()
 		tss := []*timeseries{
 			{
@@ -49,8 +52,8 @@ func TestRollupResultCache(t *testing.T) {
 				Values:     []float64{0, 1, 2},
 			},
 		}
-		rollupResultCacheV.Put(funcName, ec, me, nil, window, tss)
-		tss, newStart := rollupResultCacheV.Get(funcName, ec, me, nil, window)
+		rollupResultCacheV.Put(ec, fe, window, tss)
+		tss, newStart := rollupResultCacheV.Get(ec, fe, window)
 		if newStart != 1400 {
 			t.Fatalf("unexpected newStart; got %d; want %d", newStart, 1400)
 		}
@@ -62,7 +65,7 @@ func TestRollupResultCache(t *testing.T) {
 		}
 		testTimeseriesEqual(t, tss, tssExpected)
 	})
-	t.Run("start-overlap-with-iafc", func(t *testing.T) {
+	t.Run("start-overlap-with-ae", func(t *testing.T) {
 		ResetRollupResultCache()
 		tss := []*timeseries{
 			{
@@ -70,8 +73,8 @@ func TestRollupResultCache(t *testing.T) {
 				Values:     []float64{0, 1, 2},
 			},
 		}
-		rollupResultCacheV.Put(funcName, ec, me, iafc, window, tss)
-		tss, newStart := rollupResultCacheV.Get(funcName, ec, me, iafc, window)
+		rollupResultCacheV.Put(ec, ae, window, tss)
+		tss, newStart := rollupResultCacheV.Get(ec, ae, window)
 		if newStart != 1400 {
 			t.Fatalf("unexpected newStart; got %d; want %d", newStart, 1400)
 		}
@@ -93,8 +96,8 @@ func TestRollupResultCache(t *testing.T) {
 				Values:     []float64{333, 0, 1, 2},
 			},
 		}
-		rollupResultCacheV.Put(funcName, ec, me, nil, window, tss)
-		tss, newStart := rollupResultCacheV.Get(funcName, ec, me, nil, window)
+		rollupResultCacheV.Put(ec, fe, window, tss)
+		tss, newStart := rollupResultCacheV.Get(ec, fe, window)
 		if newStart != 1000 {
 			t.Fatalf("unexpected newStart; got %d; want %d", newStart, 1000)
 		}
@@ -112,8 +115,8 @@ func TestRollupResultCache(t *testing.T) {
 				Values:     []float64{0, 1, 2},
 			},
 		}
-		rollupResultCacheV.Put(funcName, ec, me, nil, window, tss)
-		tss, newStart := rollupResultCacheV.Get(funcName, ec, me, nil, window)
+		rollupResultCacheV.Put(ec, fe, window, tss)
+		tss, newStart := rollupResultCacheV.Get(ec, fe, window)
 		if newStart != 1000 {
 			t.Fatalf("unexpected newStart; got %d; want %d", newStart, 1000)
 		}
@@ -131,8 +134,8 @@ func TestRollupResultCache(t *testing.T) {
 				Values:     []float64{0, 1, 2},
 			},
 		}
-		rollupResultCacheV.Put(funcName, ec, me, nil, window, tss)
-		tss, newStart := rollupResultCacheV.Get(funcName, ec, me, nil, window)
+		rollupResultCacheV.Put(ec, fe, window, tss)
+		tss, newStart := rollupResultCacheV.Get(ec, fe, window)
 		if newStart != 1000 {
 			t.Fatalf("unexpected newStart; got %d; want %d", newStart, 1000)
 		}
@@ -150,8 +153,8 @@ func TestRollupResultCache(t *testing.T) {
 				Values:     []float64{0, 1, 2},
 			},
 		}
-		rollupResultCacheV.Put(funcName, ec, me, nil, window, tss)
-		tss, newStart := rollupResultCacheV.Get(funcName, ec, me, nil, window)
+		rollupResultCacheV.Put(ec, fe, window, tss)
+		tss, newStart := rollupResultCacheV.Get(ec, fe, window)
 		if newStart != 1000 {
 			t.Fatalf("unexpected newStart; got %d; want %d", newStart, 1000)
 		}
@@ -169,8 +172,8 @@ func TestRollupResultCache(t *testing.T) {
 				Values:     []float64{0, 1, 2, 3, 4, 5, 6, 7},
 			},
 		}
-		rollupResultCacheV.Put(funcName, ec, me, nil, window, tss)
-		tss, newStart := rollupResultCacheV.Get(funcName, ec, me, nil, window)
+		rollupResultCacheV.Put(ec, fe, window, tss)
+		tss, newStart := rollupResultCacheV.Get(ec, fe, window)
 		if newStart != 2200 {
 			t.Fatalf("unexpected newStart; got %d; want %d", newStart, 2200)
 		}
@@ -192,8 +195,8 @@ func TestRollupResultCache(t *testing.T) {
 				Values:     []float64{1, 2, 3, 4, 5, 6},
 			},
 		}
-		rollupResultCacheV.Put(funcName, ec, me, nil, window, tss)
-		tss, newStart := rollupResultCacheV.Get(funcName, ec, me, nil, window)
+		rollupResultCacheV.Put(ec, fe, window, tss)
+		tss, newStart := rollupResultCacheV.Get(ec, fe, window)
 		if newStart != 2200 {
 			t.Fatalf("unexpected newStart; got %d; want %d", newStart, 2200)
 		}
@@ -217,8 +220,8 @@ func TestRollupResultCache(t *testing.T) {
 			}
 			tss = append(tss, ts)
 		}
-		rollupResultCacheV.Put(funcName, ec, me, nil, window, tss)
-		tssResult, newStart := rollupResultCacheV.Get(funcName, ec, me, nil, window)
+		rollupResultCacheV.Put(ec, fe, window, tss)
+		tssResult, newStart := rollupResultCacheV.Get(ec, fe, window)
 		if newStart != 2200 {
 			t.Fatalf("unexpected newStart; got %d; want %d", newStart, 2200)
 		}
@@ -246,10 +249,10 @@ func TestRollupResultCache(t *testing.T) {
 				Values:     []float64{0, 1, 2},
 			},
 		}
-		rollupResultCacheV.Put(funcName, ec, me, nil, window, tss1)
-		rollupResultCacheV.Put(funcName, ec, me, nil, window, tss2)
-		rollupResultCacheV.Put(funcName, ec, me, nil, window, tss3)
-		tss, newStart := rollupResultCacheV.Get(funcName, ec, me, nil, window)
+		rollupResultCacheV.Put(ec, fe, window, tss1)
+		rollupResultCacheV.Put(ec, fe, window, tss2)
+		rollupResultCacheV.Put(ec, fe, window, tss3)
+		tss, newStart := rollupResultCacheV.Get(ec, fe, window)
 		if newStart != 1400 {
 			t.Fatalf("unexpected newStart; got %d; want %d", newStart, 1400)
 		}
--- a/app/vmselect/promql/rollup_test.go
+++ b/app/vmselect/promql/rollup_test.go
@@ -3,6 +3,8 @@ package promql
 import (
 	"math"
 	"testing"
+
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql"
 )

 var (
@@ -57,7 +59,7 @@ func TestRollupIderivDuplicateTimestamps(t *testing.T) {
 	}
 	n = rollupIderiv(rfa)
 	if n != 500 {
-		t.Fatalf("unexpected value; got %v; want %v", n, 0.5)
+		t.Fatalf("unexpected value; got %v; want %v", n, 500)
 	}

 	rfa = &rollupFuncArg{
@@ -157,7 +159,7 @@ func TestDerivValues(t *testing.T) {
 	testRowsEqual(t, values, timestamps, valuesExpected, timestamps)
 }

-func testRollupFunc(t *testing.T, funcName string, args []interface{}, meExpected *metricExpr, vExpected float64) {
+func testRollupFunc(t *testing.T, funcName string, args []interface{}, meExpected *metricsql.MetricExpr, vExpected float64) {
 	t.Helper()
 	nrf := getRollupFunc(funcName)
 	if nrf == nil {
@@ -190,6 +192,52 @@ func testRollupFunc(t *testing.T, funcName string, args []interface{}, meExpecte
 	}
 }

+func TestRollupShareLEOverTime(t *testing.T) {
+	f := func(le, vExpected float64) {
+		t.Helper()
+		les := []*timeseries{{
+			Values:     []float64{le},
+			Timestamps: []int64{123},
+		}}
+		var me metricsql.MetricExpr
+		args := []interface{}{&metricsql.RollupExpr{Expr: &me}, les}
+		testRollupFunc(t, "share_le_over_time", args, &me, vExpected)
+	}
+
+	f(-123, 0)
+	f(0, 0)
+	f(10, 0)
+	f(12, 0.08333333333333333)
+	f(30, 0.16666666666666666)
+	f(50, 0.75)
+	f(100, 0.9166666666666666)
+	f(123, 1)
+	f(1000, 1)
+}
+
+func TestRollupShareGTOverTime(t *testing.T) {
+	f := func(gt, vExpected float64) {
+		t.Helper()
+		gts := []*timeseries{{
+			Values:     []float64{gt},
+			Timestamps: []int64{123},
+		}}
+		var me metricsql.MetricExpr
+		args := []interface{}{&metricsql.RollupExpr{Expr: &me}, gts}
+		testRollupFunc(t, "share_gt_over_time", args, &me, vExpected)
+	}
+
+	f(-123, 1)
+	f(0, 1)
+	f(10, 1)
+	f(12, 0.9166666666666666)
+	f(30, 0.8333333333333334)
+	f(50, 0.25)
+	f(100, 0.08333333333333333)
+	f(123, 0)
+	f(1000, 0)
+}
+
 func TestRollupQuantileOverTime(t *testing.T) {
 	f := func(phi, vExpected float64) {
 		t.Helper()
@@ -197,8 +245,8 @@ func TestRollupQuantileOverTime(t *testing.T) {
 			Values:     []float64{phi},
 			Timestamps: []int64{123},
 		}}
-		var me metricExpr
-		args := []interface{}{phis, &rollupExpr{Expr: &me}}
+		var me metricsql.MetricExpr
+		args := []interface{}{phis, &metricsql.RollupExpr{Expr: &me}}
 		testRollupFunc(t, "quantile_over_time", args, &me, vExpected)
 	}

@@ -219,8 +267,8 @@ func TestRollupPredictLinear(t *testing.T) {
 			Values:     []float64{sec},
 			Timestamps: []int64{123},
 		}}
-		var me metricExpr
-		args := []interface{}{&rollupExpr{Expr: &me}, secs}
+		var me metricsql.MetricExpr
+		args := []interface{}{&metricsql.RollupExpr{Expr: &me}, secs}
 		testRollupFunc(t, "predict_linear", args, &me, vExpected)
 	}

@@ -241,8 +289,8 @@ func TestRollupHoltWinters(t *testing.T) {
 			Values:     []float64{tf},
 			Timestamps: []int64{123},
 		}}
-		var me metricExpr
-		args := []interface{}{&rollupExpr{Expr: &me}, sfs, tfs}
+		var me metricsql.MetricExpr
+		args := []interface{}{&metricsql.RollupExpr{Expr: &me}, sfs, tfs}
 		testRollupFunc(t, "holt_winters", args, &me, vExpected)
 	}

@@ -265,24 +313,26 @@ func TestRollupHoltWinters(t *testing.T) {
 func TestRollupNewRollupFuncSuccess(t *testing.T) {
 	f := func(funcName string, vExpected float64) {
 		t.Helper()
-		var me metricExpr
-		args := []interface{}{&rollupExpr{Expr: &me}}
+		var me metricsql.MetricExpr
+		args := []interface{}{&metricsql.RollupExpr{Expr: &me}}
 		testRollupFunc(t, funcName, args, &me, vExpected)
 	}

 	f("default_rollup", 34)
 	f("changes", 11)
-	f("delta", -89)
+	f("delta", 34)
 	f("deriv", -266.85860231406065)
 	f("deriv_fast", -712)
 	f("idelta", 0)
-	f("increase", 275)
+	f("increase", 398)
 	f("irate", 0)
 	f("rate", 2200)
 	f("resets", 5)
 	f("avg_over_time", 47.083333333333336)
 	f("min_over_time", 12)
 	f("max_over_time", 123)
+	f("tmin_over_time", 0.08)
+	f("tmax_over_time", 0.005)
 	f("sum_over_time", 565)
 	f("sum2_over_time", 37951)
 	f("geomean_over_time", 39.33466603189148)
@@ -327,7 +377,7 @@ func TestRollupNewRollupFuncError(t *testing.T) {
 		Values:     []float64{321},
 		Timestamps: []int64{123},
 	}}
-	me := &metricExpr{}
+	me := &metricsql.MetricExpr{}
 	f("holt_winters", []interface{}{123, 123, 321})
 	f("holt_winters", []interface{}{me, 123, 321})
 	f("holt_winters", []interface{}{me, scalarTs, 321})
@@ -409,7 +459,7 @@ func TestRollupNoWindowPartialPoints(t *testing.T) {
 		}
 		rc.Timestamps = getTimestamps(rc.Start, rc.End, rc.Step)
 		values := rc.Do(nil, testValues, testTimestamps)
-		valuesExpected := []float64{nan, 123, 123, 123, 34, 34}
+		valuesExpected := []float64{nan, 123, nan, 34, nan, 44}
 		timestampsExpected := []int64{0, 5, 10, 15, 20, 25}
 		testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
 	})
@@ -423,7 +473,7 @@ func TestRollupNoWindowPartialPoints(t *testing.T) {
 		}
 		rc.Timestamps = getTimestamps(rc.Start, rc.End, rc.Step)
 		values := rc.Do(nil, testValues, testTimestamps)
-		valuesExpected := []float64{12, 44, 34, nan}
+		valuesExpected := []float64{44, 32, 34, nan}
 		timestampsExpected := []int64{100, 120, 140, 160}
 		testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
 	})
@@ -437,7 +487,7 @@ func TestRollupNoWindowPartialPoints(t *testing.T) {
 		}
 		rc.Timestamps = getTimestamps(rc.Start, rc.End, rc.Step)
 		values := rc.Do(nil, testValues, testTimestamps)
-		valuesExpected := []float64{nan, nan, 123, 54, 44}
+		valuesExpected := []float64{nan, nan, 123, 34, 32}
 		timestampsExpected := []int64{-50, 0, 50, 100, 150}
 		testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
 	})
@@ -499,7 +549,7 @@ func TestRollupFuncsLookbackDelta(t *testing.T) {
 		}
 		rc.Timestamps = getTimestamps(rc.Start, rc.End, rc.Step)
 		values := rc.Do(nil, testValues, testTimestamps)
-		valuesExpected := []float64{99, 12, 44, nan, 32, 34, nan}
+		valuesExpected := []float64{99, nan, 44, nan, 32, 34, nan}
 		timestampsExpected := []int64{80, 90, 100, 110, 120, 130, 140}
 		testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
 	})
@@ -513,7 +563,7 @@ func TestRollupFuncsLookbackDelta(t *testing.T) {
 		}
 		rc.Timestamps = getTimestamps(rc.Start, rc.End, rc.Step)
 		values := rc.Do(nil, testValues, testTimestamps)
-		valuesExpected := []float64{99, 12, 44, 44, 32, 34, nan}
+		valuesExpected := []float64{99, nan, 44, nan, 32, 34, nan}
 		timestampsExpected := []int64{80, 90, 100, 110, 120, 130, 140}
 		testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
 	})
@@ -527,7 +577,7 @@ func TestRollupFuncsLookbackDelta(t *testing.T) {
 		}
 		rc.Timestamps = getTimestamps(rc.Start, rc.End, rc.Step)
 		values := rc.Do(nil, testValues, testTimestamps)
-		valuesExpected := []float64{34, 12, 12, 44, 44, 34, nan}
+		valuesExpected := []float64{99, nan, 44, nan, 32, 34, nan}
 		timestampsExpected := []int64{80, 90, 100, 110, 120, 130, 140}
 		testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
 	})
@@ -544,7 +594,7 @@ func TestRollupFuncsNoWindow(t *testing.T) {
 		}
 		rc.Timestamps = getTimestamps(rc.Start, rc.End, rc.Step)
 		values := rc.Do(nil, testValues, testTimestamps)
-		valuesExpected := []float64{nan, 123, 21, 12, 34}
+		valuesExpected := []float64{nan, 123, 54, 44, 34}
 		timestampsExpected := []int64{0, 40, 80, 120, 160}
 		testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
 	})
@@ -572,7 +622,7 @@ func TestRollupFuncsNoWindow(t *testing.T) {
 		}
 		rc.Timestamps = getTimestamps(rc.Start, rc.End, rc.Step)
 		values := rc.Do(nil, testValues, testTimestamps)
-		valuesExpected := []float64{nan, 21, 12, 12, 34}
+		valuesExpected := []float64{nan, 21, 12, 32, 34}
 		timestampsExpected := []int64{0, 40, 80, 120, 160}
 		testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
 	})
@@ -614,7 +664,7 @@ func TestRollupFuncsNoWindow(t *testing.T) {
 		}
 		rc.Timestamps = getTimestamps(rc.Start, rc.End, rc.Step)
 		values := rc.Do(nil, testValues, testTimestamps)
-		valuesExpected := []float64{nan, -102, -9, 22, 0}
+		valuesExpected := []float64{nan, nan, -9, 22, 0}
 		timestampsExpected := []int64{0, 40, 80, 120, 160}
 		testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
 	})
@@ -772,6 +822,20 @@ func TestRollupFuncsNoWindow(t *testing.T) {
 		timestampsExpected := []int64{0, 40, 80, 120, 160}
 		testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
 	})
+	t.Run("deriv_fast", func(t *testing.T) {
+		rc := rollupConfig{
+			Func:   rollupDerivFast,
+			Start:  0,
+			End:    20,
+			Step:   4,
+			Window: 0,
+		}
+		rc.Timestamps = getTimestamps(rc.Start, rc.End, rc.Step)
+		values := rc.Do(nil, testValues, testTimestamps)
+		valuesExpected := []float64{nan, nan, nan, 0, -8900, 0}
+		timestampsExpected := []int64{0, 4, 8, 12, 16, 20}
+		testRowsEqual(t, values, rc.Timestamps, valuesExpected, timestampsExpected)
+	})
 	t.Run("ideriv", func(t *testing.T) {
 		rc := rollupConfig{
 			Func:   rollupIderiv,
--- a/app/vmselect/promql/transform.go
+++ b/app/vmselect/promql/transform.go
@@ -12,6 +12,7 @@ import (

 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/decimal"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
 	"github.com/valyala/histogram"
 )
@@ -92,6 +93,7 @@ var transformFuncs = map[string]transformFunc{
 	"asin":               newTransformFuncOneArg(transformAsin),
 	"acos":               newTransformFuncOneArg(transformAcos),
 	"prometheus_buckets": transformPrometheusBuckets,
+	"histogram_share":    transformHistogramShare,
 }

 func getTransformFunc(s string) transformFunc {
@@ -99,13 +101,9 @@ func getTransformFunc(s string) transformFunc {
 	return transformFuncs[s]
 }

-func isTransformFunc(s string) bool {
-	return getTransformFunc(s) != nil
-}
-
 type transformFuncArg struct {
 	ec   *EvalConfig
-	fe   *funcExpr
+	fe   *metricsql.FuncExpr
 	args [][]*timeseries
 }

@@ -126,7 +124,7 @@ func newTransformFuncOneArg(tf func(v float64) float64) transformFunc {
 	}
 }

-func doTransformValues(arg []*timeseries, tf func(values []float64), fe *funcExpr) ([]*timeseries, error) {
+func doTransformValues(arg []*timeseries, tf func(values []float64), fe *metricsql.FuncExpr) ([]*timeseries, error) {
 	name := strings.ToLower(fe.Name)
 	keepMetricGroup := transformFuncsKeepMetricGroup[name]
 	for _, ts := range arg {
@@ -149,28 +147,10 @@ func transformAbsent(tfa *transformFuncArg) ([]*timeseries, error) {
 		return nil, err
 	}
 	arg := args[0]
-
 	if len(arg) == 0 {
-		// Copy tags from arg
-		rvs := evalNumber(tfa.ec, 1)
-		rv := rvs[0]
-		me, ok := tfa.fe.Args[0].(*metricExpr)
-		if !ok {
-			return rvs, nil
-		}
-		for i := range me.TagFilters {
-			tf := &me.TagFilters[i]
-			if len(tf.Key) == 0 {
-				continue
-			}
-			if tf.IsRegexp || tf.IsNegative {
-				continue
-			}
-			rv.MetricName.AddTagBytes(tf.Key, tf.Value)
-		}
+		rvs := getAbsentTimeseries(tfa.ec, tfa.fe.Args[0])
 		return rvs, nil
 	}
-
 	for _, ts := range arg {
 		ts.MetricName.ResetMetricGroup()
 		for i, v := range ts.Values {
@@ -185,6 +165,28 @@ func transformAbsent(tfa *transformFuncArg) ([]*timeseries, error) {
 	return arg, nil
 }

+func getAbsentTimeseries(ec *EvalConfig, arg metricsql.Expr) []*timeseries {
+	// Copy tags from arg
+	rvs := evalNumber(ec, 1)
+	rv := rvs[0]
+	me, ok := arg.(*metricsql.MetricExpr)
+	if !ok {
+		return rvs
+	}
+	tfs := toTagFilters(me.LabelFilters)
+	for i := range tfs {
+		tf := &tfs[i]
+		if len(tf.Key) == 0 {
+			continue
+		}
+		if tf.IsRegexp || tf.IsNegative {
+			continue
+		}
+		rv.MetricName.AddTagBytes(tf.Key, tf.Value)
+	}
+	return rvs
+}
+
 func transformCeil(v float64) float64 {
 	return math.Ceil(v)
 }
@@ -398,6 +400,105 @@ func vmrangeBucketsToLE(tss []*timeseries) []*timeseries {
 	return rvs
 }

+func transformHistogramShare(tfa *transformFuncArg) ([]*timeseries, error) {
+	args := tfa.args
+	if len(args) < 2 || len(args) > 3 {
+		return nil, fmt.Errorf("unexpected number of args; got %d; want 2...3", len(args))
+	}
+	les, err := getScalar(args[0], 0)
+	if err != nil {
+		return nil, fmt.Errorf("cannot parse le: %s", err)
+	}
+
+	// Convert buckets with `vmrange` labels to buckets with `le` labels.
+	tss := vmrangeBucketsToLE(args[1])
+
+	// Parse boundsLabel. See https://github.com/prometheus/prometheus/issues/5706 for details.
+	var boundsLabel string
+	if len(args) > 2 {
+		s, err := getString(args[2], 2)
+		if err != nil {
+			return nil, fmt.Errorf("cannot parse boundsLabel (arg #3): %s", err)
+		}
+		boundsLabel = s
+	}
+
+	// Group metrics by all tags excluding "le"
+	m := groupLeTimeseries(tss)
+
+	// Calculate share for les
+
+	share := func(i int, les []float64, xss []leTimeseries) (q, lower, upper float64) {
+		leReq := les[i]
+		if math.IsNaN(leReq) || len(xss) == 0 {
+			return nan, nan, nan
+		}
+		fixBrokenBuckets(i, xss)
+		if leReq < 0 {
+			return 0, 0, 0
+		}
+		if math.IsInf(leReq, 1) {
+			return 1, 1, 1
+		}
+		var vPrev, lePrev float64
+		for _, xs := range xss {
+			v := xs.ts.Values[i]
+			le := xs.le
+			if leReq >= le {
+				vPrev = v
+				lePrev = le
+				continue
+			}
+			// precondition: lePrev <= leReq < le
+			vLast := xss[len(xss)-1].ts.Values[i]
+			lower = vPrev / vLast
+			if math.IsInf(le, 1) {
+				return lower, lower, 1
+			}
+			if lePrev == leReq {
+				return lower, lower, lower
+			}
+			upper = v / vLast
+			q = lower + (v-vPrev)/vLast*(leReq-lePrev)/(le-lePrev)
+			return q, lower, upper
+		}
+		// precondition: leReq > leLast
+		return 1, 1, 1
+	}
+	rvs := make([]*timeseries, 0, len(m))
+	for _, xss := range m {
+		sort.Slice(xss, func(i, j int) bool {
+			return xss[i].le < xss[j].le
+		})
+		dst := xss[0].ts
+		var tsLower, tsUpper *timeseries
+		if len(boundsLabel) > 0 {
+			tsLower = &timeseries{}
+			tsLower.CopyFromShallowTimestamps(dst)
+			tsLower.MetricName.RemoveTag(boundsLabel)
+			tsLower.MetricName.AddTag(boundsLabel, "lower")
+			tsUpper = &timeseries{}
+			tsUpper.CopyFromShallowTimestamps(dst)
+			tsUpper.MetricName.RemoveTag(boundsLabel)
+			tsUpper.MetricName.AddTag(boundsLabel, "upper")
+		}
+		for i := range dst.Values {
+			q, lower, upper := share(i, les, xss)
+			dst.Values[i] = q
+			if len(boundsLabel) > 0 {
+				tsLower.Values[i] = lower
+				tsUpper.Values[i] = upper
+			}
+		}
+		rvs = append(rvs, dst)
+		if len(boundsLabel) > 0 {
+			rvs = append(rvs, tsLower)
+			rvs = append(rvs, tsUpper)
+		}
+	}
+	return rvs, nil
+}
+
 func transformHistogramQuantile(tfa *transformFuncArg) ([]*timeseries, error) {
 	args := tfa.args
 	if len(args) < 2 || len(args) > 3 {
@@ -422,73 +523,35 @@ func transformHistogramQuantile(tfa *transformFuncArg) ([]*timeseries, error) {
 	}

 	// Group metrics by all tags excluding "le"
-	type x struct {
-		le float64
-		ts *timeseries
-	}
-	m := make(map[string][]x)
-	bb := bbPool.Get()
-	for _, ts := range tss {
-		tagValue := ts.MetricName.GetTagValue("le")
-		if len(tagValue) == 0 {
-			continue
-		}
-		le, err := strconv.ParseFloat(bytesutil.ToUnsafeString(tagValue), 64)
-		if err != nil {
-			continue
-		}
-		ts.MetricName.ResetMetricGroup()
-		ts.MetricName.RemoveTag("le")
-		bb.B = marshalMetricTagsSorted(bb.B[:0], &ts.MetricName)
-		m[string(bb.B)] = append(m[string(bb.B)], x{
-			le: le,
-			ts: ts,
-		})
-	}
-	bbPool.Put(bb)
+	m := groupLeTimeseries(tss)

 	// Calculate quantile for each group in m

-	lastNonInf := func(i int, xss []x) float64 {
+	lastNonInf := func(i int, xss []leTimeseries) float64 {
 		for len(xss) > 0 {
 			xsLast := xss[len(xss)-1]
 			v := xsLast.ts.Values[i]
 			if v == 0 {
 				return nan
 			}
-			if !math.IsNaN(v) && !math.IsInf(xsLast.le, 0) {
+			if !math.IsInf(xsLast.le, 0) {
 				return xsLast.le
 			}
 			xss = xss[:len(xss)-1]
 		}
 		return nan
 	}
-	quantile := func(i int, phis []float64, xss []x) (q, lower, upper float64) {
+	quantile := func(i int, phis []float64, xss []leTimeseries) (q, lower, upper float64) {
 		phi := phis[i]
 		if math.IsNaN(phi) {
 			return nan, nan, nan
 		}
-		// Fix broken buckets.
-		// They are already sorted by le, so their values must be in ascending order,
-		// since the next bucket value includes all the previous buckets.
-		vPrev := float64(0)
-		for _, xs := range xss {
-			v := xs.ts.Values[i]
-			if v < vPrev {
-				xs.ts.Values[i] = vPrev
-			} else if !math.IsNaN(v) {
-				vPrev = v
-			}
-		}
-		vLast := nan
-		for len(xss) > 0 {
+		fixBrokenBuckets(i, xss)
+		vLast := float64(0)
+		if len(xss) > 0 {
 			vLast = xss[len(xss)-1].ts.Values[i]
-			if !math.IsNaN(vLast) {
-				break
-			}
-			xss = xss[:len(xss)-1]
 		}
-		if vLast == 0 || math.IsNaN(vLast) {
+		if vLast == 0 {
 			return nan, nan, nan
 		}
 		if phi < 0 {
@@ -498,15 +561,10 @@ func transformHistogramQuantile(tfa *transformFuncArg) ([]*timeseries, error) {
 			return inf, vLast, inf
 		}
 		vReq := vLast * phi
-		vPrev = 0
+		vPrev := float64(0)
 		lePrev := float64(0)
 		for _, xs := range xss {
 			v := xs.ts.Values[i]
-			if math.IsNaN(v) {
-				// Skip NaNs - they may appear if the selected time range
-				// contains multiple different bucket sets.
-				continue
-			}
 			le := xs.le
 			if v <= 0 {
 				// Skip zero buckets.
@@ -565,6 +623,50 @@ func transformHistogramQuantile(tfa *transformFuncArg) ([]*timeseries, error) {
 	return rvs, nil
 }

+type leTimeseries struct {
+	le float64
+	ts *timeseries
+}
+
+func groupLeTimeseries(tss []*timeseries) map[string][]leTimeseries {
+	m := make(map[string][]leTimeseries)
+	bb := bbPool.Get()
+	for _, ts := range tss {
+		tagValue := ts.MetricName.GetTagValue("le")
+		if len(tagValue) == 0 {
+			continue
+		}
+		le, err := strconv.ParseFloat(bytesutil.ToUnsafeString(tagValue), 64)
+		if err != nil {
+			continue
+		}
+		ts.MetricName.ResetMetricGroup()
+		ts.MetricName.RemoveTag("le")
+		bb.B = marshalMetricTagsSorted(bb.B[:0], &ts.MetricName)
+		m[string(bb.B)] = append(m[string(bb.B)], leTimeseries{
+			le: le,
+			ts: ts,
+		})
+	}
+	bbPool.Put(bb)
+	return m
+}
+
+func fixBrokenBuckets(i int, xss []leTimeseries) {
+	// Fix broken buckets.
+	// They are already sorted by le, so their values must be in ascending order,
+	// since the next bucket includes all the previous buckets.
+	vPrev := float64(0)
+	for _, xs := range xss {
+		v := xs.ts.Values[i]
+		if v < vPrev || math.IsNaN(v) {
+			xs.ts.Values[i] = vPrev
+		} else {
+			vPrev = v
+		}
+	}
+}
+
 func transformHour(t time.Time) int {
 	return t.Hour()
 }
@@ -1023,7 +1125,7 @@ func transformLabelTransform(tfa *transformFuncArg) ([]*timeseries, error) {
 		return nil, err
 	}

-	r, err := compileRegexp(regex)
+	r, err := metricsql.CompileRegexp(regex)
 	if err != nil {
 		return nil, fmt.Errorf(`cannot compile regex %q: %s`, regex, err)
 	}
@@ -1052,7 +1154,7 @@ func transformLabelReplace(tfa *transformFuncArg) ([]*timeseries, error) {
 		return nil, err
 	}

-	r, err := compileRegexpAnchored(regex)
+	r, err := metricsql.CompileRegexpAnchored(regex)
 	if err != nil {
 		return nil, fmt.Errorf(`cannot compile regex %q: %s`, regex, err)
 	}
@@ -1163,7 +1265,7 @@ func transformScalar(tfa *transformFuncArg) ([]*timeseries, error) {

 	// Verify whether the arg is a string.
 	// Then try converting the string to number.
-	if se, ok := tfa.fe.Args[0].(*stringExpr); ok {
+	if se, ok := tfa.fe.Args[0].(*metricsql.StringExpr); ok {
 		n, err := strconv.ParseFloat(se.S, 64)
 		if err != nil {
 			n = nan
--- a/deployment/docker/Makefile
+++ b/deployment/docker/Makefile
@@ -84,49 +84,61 @@ run-via-docker: package-via-docker

 app-via-docker-goarch:
 	APP_SUFFIX='-$(GOARCH)' \
-	DOCKER_OPTS='--env CGO_ENABLED=0 --env GOOS=linux --env GOARCH=$(GOARCH)' \
+	DOCKER_OPTS='--env CGO_ENABLED=$(CGO_ENABLED) --env GOOS=linux --env GOARCH=$(GOARCH)' \
 	$(MAKE) app-via-docker

+app-via-docker-goarch-cgo:
+	CGO_ENABLED=1 $(MAKE) app-via-docker-goarch
+
+app-via-docker-goarch-nocgo:
+	CGO_ENABLED=0 $(MAKE) app-via-docker-goarch
+
 app-via-docker-pure:
 	APP_SUFFIX='-pure' DOCKER_OPTS='--env CGO_ENABLED=0' $(MAKE) app-via-docker

 app-via-docker-amd64:
-	GOARCH=amd64 $(MAKE) app-via-docker-goarch
+	GOARCH=amd64 $(MAKE) app-via-docker-goarch-cgo

 app-via-docker-arm:
-	GOARCH=arm $(MAKE) app-via-docker-goarch
+	GOARCH=arm $(MAKE) app-via-docker-goarch-nocgo

 app-via-docker-arm64:
-	GOARCH=arm64 $(MAKE) app-via-docker-goarch
+	GOARCH=arm64 $(MAKE) app-via-docker-goarch-nocgo

 app-via-docker-ppc64le:
-	GOARCH=ppc64le $(MAKE) app-via-docker-goarch
+	GOARCH=ppc64le $(MAKE) app-via-docker-goarch-nocgo

 app-via-docker-386:
-	GOARCH=386 $(MAKE) app-via-docker-goarch
+	GOARCH=386 $(MAKE) app-via-docker-goarch-nocgo

 package-via-docker-goarch:
 	APP_SUFFIX='-$(GOARCH)' \
-	DOCKER_OPTS='--env CGO_ENABLED=0 --env GOOS=linux --env GOARCH=$(GOARCH)' \
+	DOCKER_OPTS='--env CGO_ENABLED=$(CGO_ENABLED) --env GOOS=linux --env GOARCH=$(GOARCH)' \
 	$(MAKE) package-via-docker

+package-via-docker-goarch-cgo:
+	CGO_ENABLED=1 $(MAKE) package-via-docker-goarch
+
+package-via-docker-goarch-nocgo:
+	CGO_ENABLED=0 $(MAKE) package-via-docker-goarch
+
 package-via-docker-pure:
 	APP_SUFFIX='-pure' DOCKER_OPTS='--env CGO_ENABLED=0' $(MAKE) package-via-docker

 package-via-docker-amd64:
-	GOARCH=amd64 $(MAKE) package-via-docker-goarch
+	GOARCH=amd64 $(MAKE) package-via-docker-goarch-cgo

 package-via-docker-arm:
-	GOARCH=arm $(MAKE) package-via-docker-goarch
+	GOARCH=arm $(MAKE) package-via-docker-goarch-nocgo

 package-via-docker-arm64:
-	GOARCH=arm64 $(MAKE) package-via-docker-goarch
+	GOARCH=arm64 $(MAKE) package-via-docker-goarch-nocgo

 package-via-docker-ppc64le:
-	GOARCH=ppc64le $(MAKE) package-via-docker-goarch
+	GOARCH=ppc64le $(MAKE) package-via-docker-goarch-nocgo

 package-via-docker-386:
-	GOARCH=386 $(MAKE) package-via-docker-goarch
+	GOARCH=386 $(MAKE) package-via-docker-goarch-nocgo

 remove-docker-images:
 	docker image ls --format '{{.Repository}}\t{{.ID}}' | grep $(DOCKER_NAMESPACE)/ | grep -v /builder | awk '{print $$2}' | xargs docker image rm -f
--- a/docs/CaseStudies.md
+++ b/docs/CaseStudies.md
@@ -1,8 +1,14 @@
-## Case studies
+## Case studies and talks

-Below are approved public case studies from VictoriaMetrics users. Join our [community Slack channel](http://slack.victoriametrics.com/)
+Below are approved public case studies and talks from VictoriaMetrics users. Join our [community Slack channel](http://slack.victoriametrics.com/)
 and feel free asking for references, reviews and additional case studies from real VictoriaMetrics users there.

+### Adidas
+
+See [slides](https://promcon.io/2019-munich/slides/remote-write-storage-wars.pdf) and [video](https://youtu.be/OsH6gPdxR4s)
+from [Remote Write Storage Wars](https://promcon.io/2019-munich/talks/remote-write-storage-wars/) talk at [PromCon 2019](https://promcon.io/2019-munich/).
+VictoriaMetrics is compared to Thanos, Corex and M3DB in the talk.
+

 ### Wix.com

--- a/docs/Cluster-VictoriaMetrics.md
+++ b/docs/Cluster-VictoriaMetrics.md
@@ -3,7 +3,7 @@
 VictoriaMetrics is fast, cost-effective and scalable time series database. It can be used as a long-term remote storage for Prometheus.

 It is recommended using [single-node version](https://github.com/VictoriaMetrics/VictoriaMetrics) instead of cluster version
-for ingestion rates lower than 10 million of data points per second.
+for ingestion rates lower than a million of data points per second.
 Single-node version [scales perfectly](https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae)
 with the number of CPU cores, RAM and available storage space.
 Single-node version is easier to configure and operate comparing to cluster version, so think twice before sticking to cluster version.
@@ -133,6 +133,7 @@ with [the official Grafana dashboard for VictoriaMetrics cluster](https://grafan
  - `<suffix>` may have the following values:
     - `prometheus` - for inserting data with [Prometheus remote write API](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write)
     - `influx/write` or `influx/api/v2/write` - for inserting data with [Influx line protocol](https://docs.influxdata.com/influxdb/v1.7/write_protocols/line_protocol_tutorial/)
+     - `opentsdb/api/put` - for accepting [OpenTSDB HTTP /api/put requests](http://opentsdb.net/docs/build/html/api_http/put.html).
     - `prometheus/api/v1/import` - for importing data obtained via `api/v1/export` on `vmselect` (see below).

 * URLs for querying: `http://<vmselect>:8481/select/<accountID>/prometheus/<suffix>`, where:
--- a/docs/ExtendedPromQL.md
+++ b/docs/ExtendedPromQL.md
@@ -1,9 +1,25 @@
-# PromQL extensions
+# MetricsQL

-VictoriaMetrics supports [standard PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics/)
-including [subqueries](https://prometheus.io/blog/2019/01/28/subquery-support/).
-Additionally it supports useful extensions mentioned below.
-Try these extensions on [an editable Grafana dashboard](http://play-grafana.victoriametrics.com:3000/d/4ome8yJmz/node-exporter-on-victoriametrics-demo).
+VictoriaMetrics implements MetricsQL - query language inspired by [PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics/).
+It is backwards compatible with PromQL, so Grafana dashboards backed by Prometheus datasource should work the same after switching from Prometheus to VictoriaMetrics.
+[Standalone MetricsQL package](https://godoc.org/github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql) can be used for parsing MetricsQL in external apps.
+
+The following functionality is implemented differently in MetricsQL comparing to PromQL in order to improve user experience:
+* MetricsQL takes into account the previous point before the window in square brackets for range functions such as `rate` and `increase`.
+  It also doesn't extrapolate range function results. This addresses [this issue from Prometheus](https://github.com/prometheus/prometheus/issues/3746).
+* MetricsQL returns the expected non-empty responses for requests with `step` values smaller than scrape interval. This addresses [this issue from Grafana](https://github.com/grafana/grafana/issues/11451).
+* MetricsQL treats `scalar` type the same as `instant vector` without labels, since subtle difference between these types usually confuses users.
+  See [the corresponding Prometheus docs](https://prometheus.io/docs/prometheus/latest/querying/basics/#expression-language-data-types) for details.
+
+Other PromQL functionality should work the same in MetricsQL. [File an issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues)
+if you notice discrepancies between PromQL and MetricsQL results other than mentioned above.
+
+MetricsQL provides additional functionality mentioned below, which is aimed towards solving practical cases.
+Feel free [filing a feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues) if you think MetricsQL misses certain useful functionality.
+
+*Note that the functionality mentioned below doesn't work in PromQL, so it is impossible switching back to Prometheus after you start using it.*
+
+This functionality can be tried at [an editable Grafana dashboard](http://play-grafana.victoriametrics.com:3000/d/4ome8yJmz/node-exporter-on-victoriametrics-demo).

 - [`WITH` templates](https://play.victoriametrics.com/promql/expand-with-exprs). This feature simplifies writing and managing complex queries. Go to [`WITH` templates playground](https://victoriametrics.com/promql/expand-with-exprs) and try it.
 - Metric names and metric labels may contain escaped chars. For instance, `foo\-bar{baz\=aa="b"}` is valid expression. It returns time series with name `foo-bar` containing label `baz=aa` with value `b`. Additionally, `\xXX` escape sequence is supported, where `XX` is hexadecimal representation of escaped char.
@@ -63,6 +79,11 @@ Try these extensions on [an editable Grafana dashboard](http://play-grafana.vict
 - `increases_over_time(m[d])` and `decreases_over_time(m[d])` - returns the number of `m` increases or decreases over the given duration `d`.
 - `prometheus_buckets(q)` - converts [VictoriaMetrics histogram](https://godoc.org/github.com/VictoriaMetrics/metrics#Histogram) buckets to Prometheus buckets with `le` labels.
 - `histogram(q)` - calculates aggregate histogram over `q` time series for each point on the graph. See [this article](https://medium.com/@valyala/improving-histogram-usability-for-prometheus-and-grafana-bc7e5df0e350) for more details.
+- `histogram_over_time(m[d])` - calculates [VictoriaMetrics histogram](https://godoc.org/github.com/VictoriaMetrics/metrics#Histogram) for `m` over `d`.
+  For example, the following query calculates median temperature by country over the last 24 hours:
+  `histogram_quantile(0.5, sum(histogram_over_time(temperature[24h])) by (vmbucket, country))`.
+- `histogram_share(le, buckets)` - returns share (in the range 0..1) for `buckets`. Useful for calculating SLI and SLO.
+  For instance, the following query returns the share of requests which are performed under 1.5 seconds: `histogram_share(1.5, sum(request_duration_seconds_bucket) by (le))`.
 - `topk_*` and `bottomk_*` aggregate functions, which return up to K time series. Note that the standard `topk` function may return more than K time series -
   see [this article](https://www.robustperception.io/graph-top-n-time-series-in-grafana) for details.
   - `topk_min(k, q)` - returns top K time series with the max minimums on the given time range
@@ -73,3 +94,12 @@ Try these extensions on [an editable Grafana dashboard](http://play-grafana.vict
   - `bottomk_max(k, q)` - returns bottom K time series with the min maximums on the given time range
   - `bottomk_avg(k, q)` - returns bottom K time series with the min averages on the given time range
   - `bottomk_median(k, q)` - returns bottom K time series with the min medians on the given time range
+- `share_le_over_time(m[d], le)` - returns share (in the range 0..1) of values in `m` over `d`, which are smaller or equal to `le`. Useful for calculating SLI and SLO.
+  Example: `share_le_over_time(memory_usage_bytes[24h], 100*1024*1024)` returns the share of time series values for the last 24 hours when memory usage was below or equal to 100MB.
+- `share_gt_over_time(m[d], gt)` - returns share (in the range 0..1) of values in `m` over `d`, which are bigger than `gt`. Useful for calculating SLI and SLO.
+  Example: `share_gt_over_time(up[24h], 0)` - returns service availability for the last 24 hours.
+- `tmin_over_time(m[d])` - returns timestamp for the minimum value for `m` over `d` time range.
+- `tmax_over_time(m[d])` - returns timestamp for the maximum value for `m` over `d` time range.
+- `aggr_over_time(("aggr_func1", "aggr_func2", ...), m[d])` - simultaneously calculates all the listed `aggr_func*` for `m` over `d` time range.
+  `aggr_func*` can contain any functions that accept range vector. For instance, `aggr_over_time(("min_over_time", "max_over_time", "rate"), m[d])`
+  would calculate `min_over_time`, `max_over_time` and `rate` for `m[d]`.
--- a/docs/FAQ.md
+++ b/docs/FAQ.md
@@ -16,7 +16,7 @@ To provide the best long-term [remote storage](https://prometheus.io/docs/operat
 * High data compression, so [up to 70x more data points](https://medium.com/@valyala/when-size-matters-benchmarking-victoriametrics-vs-timescale-and-influxdb-6035811952d4)
  may be crammed into a limited storage comparing to TimescaleDB.
 * Optimized for storage with high-latency IO and low iops (HDD and network storage in AWS, Google Cloud, Microsoft Azure, etc). See [graphs from these benchmarks](https://medium.com/@valyala/high-cardinality-tsdb-benchmarks-victoriametrics-vs-timescaledb-vs-influxdb-13e6ee64dd6b).
-* A single-node VictoriaMetrics may substitute moderately sized clusters built with competing solutions such as Thanos, Uber M3, Cortex, InfluxDB or TimescaleDB.
+* A single-node VictoriaMetrics may substitute moderately sized clusters built with competing solutions such as Thanos, M3DB, Cortex, InfluxDB or TimescaleDB.
  See [vertical scalability benchmarks](https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae)
  and [comparing Thanos to VictoriaMetrics](https://medium.com/@valyala/comparing-thanos-to-victoriametrics-cluster-b193bea1683).
 * Easy operation:
@@ -56,12 +56,15 @@ and new data is available for querying via Prometheus as usual.

 ### How does VictoriaMetrics compare to other clustered TSDBs on top of Prometheus such as [M3 from Uber](https://eng.uber.com/m3/), [Thanos](https://github.com/improbable-eng/thanos), [Cortex](https://github.com/cortexproject/cortex), etc.?

-VictoriaMetrics is simpler, faster, more cost-effective and it provides [useful extensions for PromQL](ExtendedPromQL). The simplicity is twofold:
+VictoriaMetrics is simpler, faster, more cost-effective and it provides [MetricsQL with useful extensions for PromQL](ExtendedPromQL). The simplicity is twofold:
 - It is simpler to configure and operate. There is no need in configuring third-party [sidecars](https://github.com/improbable-eng/thanos/blob/master/docs/components/sidecar.md)
  or fighting with [gossip protocol](https://github.com/improbable-eng/thanos/blob/master/docs/proposals/completed/201809_gossip-removal.md).
- VictoriaMetrics has simpler architecture, which means less bugs and more useful features in a long run comparing to competing TSDBs.
+- VictoriaMetrics has simpler architecture, which means less bugs and more useful features in the long run comparing to competing TSDBs.

-See [comparing Thanos to VictoriaMetrics cluster](https://medium.com/@valyala/comparing-thanos-to-victoriametrics-cluster-b193bea1683).
+See [comparing Thanos to VictoriaMetrics cluster](https://medium.com/@valyala/comparing-thanos-to-victoriametrics-cluster-b193bea1683)
+and [Remote Write Storage Wars](https://promcon.io/2019-munich/talks/remote-write-storage-wars/) talk from [PromCon 2019](https://promcon.io/2019-munich/talks/remote-write-storage-wars/).
+
+VictoriaMetrics also [uses less RAM than Thanos components](https://github.com/thanos-io/thanos/issues/448).


 ### How does VictoriaMetrics compare to [InfluxDB](https://www.influxdata.com/time-series-platform/influxdb/)?
--- a/docs/Home.md
+++ b/docs/Home.md
@@ -3,10 +3,11 @@
 * [Quick start](Quick-Start)
 * [`WITH` templates playground](https://play.victoriametrics.com/promql/expand-with-exprs)
 * [Grafana playground](http://play-grafana.victoriametrics.com:3000/d/4ome8yJmz/node-exporter-on-victoriametrics-demo)
-* [PromQL extensions](ExtendedPromQL)
+* [MetricsQL](ExtendedPromQL)
 * [Single-node version](Single-server-VictoriaMetrics)
 * [FAQ](FAQ)
 * [Cluster version](Cluster-VictoriaMetrics)
 * [Articles](Articles)
+* [Case Studies](CaseStudies)
 * [vmbackup](vmbackup)
 * [vmrestore](vmrestore)
--- a/docs/Single-server-VictoriaMetrics.md
+++ b/docs/Single-server-VictoriaMetrics.md
@@ -18,7 +18,7 @@ Cluster version is available [here](https://github.com/VictoriaMetrics/VictoriaM
 ## Prominent features

 * Supports [Prometheus querying API](https://prometheus.io/docs/prometheus/latest/querying/api/), so it can be used as Prometheus drop-in replacement in Grafana.
-  Additionally, VictoriaMetrics extends PromQL with opt-in [useful features](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/ExtendedPromQL).
+  VictoriaMetrics implements [MetricsQL](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/ExtendedPromQL) query language, which is inspired by PromQL.
 * Supports global query view. Multiple Prometheus instances may write data into VictoriaMetrics. Later this data may be used in a single query.
 * High performance and good scalability for both [inserts](https://medium.com/@valyala/high-cardinality-tsdb-benchmarks-victoriametrics-vs-timescaledb-vs-influxdb-13e6ee64dd6b)
  and [selects](https://medium.com/@valyala/when-size-matters-benchmarking-victoriametrics-vs-timescale-and-influxdb-6035811952d4).
@@ -28,9 +28,11 @@ Cluster version is available [here](https://github.com/VictoriaMetrics/VictoriaM
 * High data compression, so [up to 70x more data points](https://medium.com/@valyala/when-size-matters-benchmarking-victoriametrics-vs-timescale-and-influxdb-6035811952d4)
  may be crammed into limited storage comparing to TimescaleDB.
 * Optimized for storage with high-latency IO and low IOPS (HDD and network storage in AWS, Google Cloud, Microsoft Azure, etc). See [graphs from these benchmarks](https://medium.com/@valyala/high-cardinality-tsdb-benchmarks-victoriametrics-vs-timescaledb-vs-influxdb-13e6ee64dd6b).
-* A single-node VictoriaMetrics may substitute moderately sized clusters built with competing solutions such as Thanos, Uber M3, Cortex, InfluxDB or TimescaleDB.
-  See [vertical scalability benchmarks](https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae)
-  and [comparing Thanos to VictoriaMetrics cluster](https://medium.com/@valyala/comparing-thanos-to-victoriametrics-cluster-b193bea1683).
+* A single-node VictoriaMetrics may substitute moderately sized clusters built with competing solutions such as Thanos, M3DB, Cortex, InfluxDB or TimescaleDB.
+  See [vertical scalability benchmarks](https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae),
+  [comparing Thanos to VictoriaMetrics cluster](https://medium.com/@valyala/comparing-thanos-to-victoriametrics-cluster-b193bea1683)
+  and [Remote Write Storage Wars](https://promcon.io/2019-munich/talks/remote-write-storage-wars/) talk
+  from [PromCon 2019](https://promcon.io/2019-munich/talks/remote-write-storage-wars/).
 * Easy operation:
  * VictoriaMetrics consists of a single [small executable](https://medium.com/@valyala/stripping-dependency-bloat-in-victoriametrics-docker-image-983fb5912b0d) without external dependencies.
  * All the configuration is done via explicit command-line flags with reasonable defaults.
@@ -126,14 +128,13 @@ It is recommended setting up [monitoring](#monitoring) for VictoriaMetrics.

 ### Prometheus setup

-Add the following lines to Prometheus config file (it is usually located at `/etc/prometheus/prometheus.yml`):
+Prometheus must be configured with [remote_write](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write) 
+in order to send data to VictoriaMetrics. Add the following lines 
+to Prometheus config file (it is usually located at `/etc/prometheus/prometheus.yml`):

 ```yml
 remote_write:
  - url: http://<victoriametrics-addr>:8428/api/v1/write
-    queue_config:
-      max_samples_per_send: 10000
-      max_shards: 30
 ```

 Substitute `<victoriametrics-addr>` with the hostname or IP address of VictoriaMetrics.
@@ -160,6 +161,22 @@ This instructs Prometheus to add `datacenter=dc-123` label to each time series s
 The label name may be arbitrary - `datacenter` is just an example. The label value must be unique
 across Prometheus instances, so those time series may be filtered and grouped by this label.

+For highly loaded Prometheus instances (400k+ samples per second)
+the following tuning may be applied:
+```
+remote_write:
+  - url: http://<victoriametrics-addr>:8428/api/v1/write
+    queue_config:
+      max_samples_per_send: 10000
+      capacity: 20000
+      max_shards: 30
+```
+
+Using remote write increases memory usage for Prometheus up to ~25%
+and depends on the shape of data. If you are experiencing issues with
+too high memory consumption try to lower `max_samples_per_send` 
+and `capacity` params (keep in mind that these two params are tightly connected).
+Read more about tuning remote write for Prometheus [here](https://prometheus.io/docs/practices/remote_write).

 It is recommended upgrading Prometheus to [v2.12.0](https://github.com/prometheus/prometheus/releases) or newer,
 since the previous versions may have issues with `remote_write`.
@@ -395,6 +412,18 @@ VictoriaMetrics supports the following handlers from [Prometheus querying API](h

 These handlers can be queried from Prometheus-compatible clients such as Grafana or curl.

+VictoriaMetrics accepts additional args for `/api/v1/labels` and `/api/v1/label/.../values` handlers.
+See [this feature request](https://github.com/prometheus/prometheus/issues/6178) for details:
+
+* Any number [time series selectors](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors) via `match[]` query arg.
+* Optional `start` and `end` query args for limiting the time range for the selected labels or label values.
+
+Additionally VictoriaMetrics provides the following handlers:
+
+* `/api/v1/series/count` - it returns the total number of time series in the database. Note that this handler scans all the inverted index,
+  so it can be slow if the database contains tens of millions of time series.
+* `/api/v1/labels/count` - it returns a list of `label: values_count` entries. It can be used for determining labels with the maximum number of values.
+

 ### How to build from sources

@@ -522,6 +551,15 @@ Each JSON line would contain data for a single time series. An example output:
 Optional `start` and `end` args may be added to the request in order to limit the time frame for the exported data. These args may contain either
 unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) values.

+Pass `Accept-Encoding: gzip` HTTP header in the request to `/api/v1/export` in order to reduce network bandwidth during exporing big amounts
+of time series data. This enables gzip compression for the exported data. Example for exporting gzipped data:
+
+```
+curl -H 'Accept-Encoding: gzip' http://localhost:8428/api/v1/export -d 'match[]={__name__!=""}' > data.jsonl.gz
+```
+
+The maximum duration for each request to `/api/v1/export` is limited by `-search.maxExportDuration` command-line flag.
+
 Exported data can be imported via POST'ing it to [/api/v1/import](#how-to-import-time-series-data).


@@ -540,10 +578,20 @@ The most efficient protocol for importing data into VictoriaMetrics is `/api/v1/

 ```
 # Export the data from <source-victoriametrics>:
-curl -s 'http://source-victoriametrics:8428/api/v1/export' -d 'match={__name__!=""}' > exported_data.jsonl
+curl http://source-victoriametrics:8428/api/v1/export -d 'match={__name__!=""}' > exported_data.jsonl

 # Import the data to <destination-victoriametrics>:
-curl -X POST 'http://destination-victoriametrics:8428/api/v1/import' -T exported_data.jsonl
+curl -X POST http://destination-victoriametrics:8428/api/v1/import -T exported_data.jsonl
+```
+
+Pass `Content-Encoding: gzip` HTTP request header to `/api/v1/import` for importing gzipped data:
+
+```
+# Export gzipped data from <source-victoriametrics>:
+curl -H 'Accept-Encoding: gzip' http://source-victoriametrics:8428/api/v1/export -d 'match={__name__!=""}' > exported_data.jsonl.gz
+
+# Import gzipped data to <destination-victoriametrics>:
+curl -X POST -H 'Content-Encoding: gzip' http://destination-victoriametrics:8428/api/v1/import -T exported_data.jsonl.gz
 ```

 Each request to `/api/v1/import` can load up to a single vCPU core on VictoriaMetrics. Import speed can be improved by splitting the original file into smaller parts
@@ -569,7 +617,7 @@ A rough estimation of the required resources for ingestion path:
 * RAM size: less than 1KB per active time series. So, ~1GB of RAM is required for 1M active time series.
  Time series is considered active if new data points have been added to it recently or if it has been recently queried.
  The number of active time series may be obtained from `vm_cache_entries{type="storage/hour_metric_ids"}` metric
-  exproted on the `/metrics` page.
+  exported on the `/metrics` page.
  VictoriaMetrics stores various caches in RAM. Memory size for these caches may be limited by `-memory.allowedPercent` flag.

 * CPU cores: a CPU core per 300K inserted data points per second. So, ~4 CPU cores are required for processing
@@ -650,7 +698,7 @@ There is no downsampling support at the moment, but:
 - VictoriaMetrics has good compression for on-disk data. See [this article](https://medium.com/@valyala/victoriametrics-achieving-better-compression-for-time-series-data-than-gorilla-317bc1f95932)
  for details.

-These properties reduce the need in downsampling. We plan to implement downsampling in the future.
+These properties reduce the need of downsampling. We plan to implement downsampling in the future.
 See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/36) for details.


@@ -672,8 +720,10 @@ horizontally scalable long-term remote storage for really large Prometheus deplo

 ### Alerting

-VictoriaMetrics doesn't support rule evaluation and alerting yet, so these actions must be performed either
-on [Prometheus side](https://prometheus.io/docs/alerting/overview/) or on [Grafana side](https://grafana.com/docs/alerting/rules/).
+VictoriaMetrics doesn't support rule evaluation and alerting yet, so these actions can be performed at the following places:
+* At Prometheus - see [the corresponding docs](https://prometheus.io/docs/alerting/overview/).
+* At Promxy - see [the corresponding docs](https://github.com/jacksontj/promxy/blob/master/README.md#how-do-i-use-alertingrecording-rules-in-promxy).
+* At Grafana - see [the corresponding docs](https://grafana.com/docs/alerting/rules/).


 ### Security
@@ -693,14 +743,14 @@ For example, substitute `-graphiteListenAddr=:2003` with `-graphiteListenAddr=<i

 ### Tuning

-* There is no need in VictoriaMetrics tuning since it uses reasonable defaults for command-line flags,
+* There is no need for VictoriaMetrics tuning since it uses reasonable defaults for command-line flags,
  which are automatically adjusted for the available CPU and RAM resources.
-* There is no need in Operating System tuning since VictoriaMetrics is optimized for default OS settings.
+* There is no need for Operating System tuning since VictoriaMetrics is optimized for default OS settings.
  The only option is increasing the limit on [the number of open files in the OS](https://medium.com/@muhammadtriwibowo/set-permanently-ulimit-n-open-files-in-ubuntu-4d61064429a),
  so Prometheus instances could establish more connections to VictoriaMetrics.
 * The recommended filesystem is `ext4`, the recommended persistent storage is [persistent HDD-based disk on GCP](https://cloud.google.com/compute/docs/disks/#pdspecs),
  since it is protected from hardware failures via internal replication and it can be [resized on the fly](https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd).
-  If you plan storing more than 1TB of data on `ext4` partition or plan extending it to more than 16TB,
+  If you plan to store more than 1TB of data on `ext4` partition or plan extending it to more than 16TB,
  then the following options are recommended to pass to `mkfs.ext4`:

 ```
@@ -731,7 +781,7 @@ The most interesting metrics are:
 ### Troubleshooting

 * It is recommended to use default command-line flag values (i.e. don't set them explicitly) until the need
-  in tweaking these flag values arises.
+  of tweaking these flag values arises.

 * If VictoriaMetrics works slowly and eats more than a CPU core per 100K ingested data points per second,
  then it is likely you have too many active time series for the current amount of RAM.
@@ -745,7 +795,7 @@ The most interesting metrics are:
  has at least 20% of free space comparing to disk size.

 * If VictoriaMetrics doesn't work because of certain parts are corrupted due to disk errors,
-  then just remove directoreis with broken parts. This will recover VictoriaMetrics at the cost
+  then just remove directories with broken parts. This will recover VictoriaMetrics at the cost
  of data loss stored in the broken parts. In the future, `vmrecover` tool will be created
  for automatic recovering from such errors.

@@ -796,7 +846,7 @@ The collected profiles may be analyzed with [go tool pprof](https://github.com/g
 - [ ] CLI tool for data migration, re-balancing and adding/removing nodes [#103](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/103)


-The discussion happens [here](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/129). Feel free to comment any item or add own one.
+The discussion happens [here](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/129). Feel free to comment on any item or add you own one.


 ## Contacts
@@ -839,7 +889,7 @@ Report bugs and propose new features [here](https://github.com/VictoriaMetrics/V

 ## Victoria Metrics Logo

-[Zip](VM_logo.zip) contains three folders with different image orientation (main color and inverted version).
+[Zip](VM_logo.zip) contains three folders with different image orientations (main color and inverted version).

 Files included in each folder:

--- a/go.mod
+++ b/go.mod
@@ -1,32 +1,31 @@
 module github.com/VictoriaMetrics/VictoriaMetrics

 require (
-	cloud.google.com/go v0.49.0 // indirect
+	cloud.google.com/go v0.50.0 // indirect
 	cloud.google.com/go/storage v1.4.0
-	github.com/VictoriaMetrics/fastcache v1.5.4
+	github.com/VictoriaMetrics/fastcache v1.5.5
 	github.com/VictoriaMetrics/metrics v1.9.3
-	github.com/aws/aws-sdk-go v1.26.0
+	github.com/aws/aws-sdk-go v1.26.8
 	github.com/cespare/xxhash/v2 v2.1.1
 	github.com/golang/groupcache v0.0.0-20191027212112-611e8accdfc9 // indirect
 	github.com/golang/snappy v0.0.1
 	github.com/jstemmer/go-junit-report v0.9.1 // indirect
 	github.com/klauspost/compress v1.9.4
-	github.com/valyala/fastjson v1.4.1
+	github.com/valyala/fastjson v1.4.5
 	github.com/valyala/fastrand v1.0.0
-	github.com/valyala/gozstd v1.6.3
+	github.com/valyala/gozstd v1.6.4
 	github.com/valyala/histogram v1.0.1
 	github.com/valyala/quicktemplate v1.4.1
 	go.opencensus.io v0.22.2 // indirect
-	golang.org/x/exp v0.0.0-20191129062945-2f5052295587 // indirect
-	golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f // indirect
+	golang.org/x/exp v0.0.0-20191224044220-1fea468a75e9 // indirect
 	golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553 // indirect
 	golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6 // indirect
-	golang.org/x/sys v0.0.0-20191210023423-ac6580df4449
-	golang.org/x/tools v0.0.0-20191212051200-825cb0626375 // indirect
-	google.golang.org/api v0.14.0
+	golang.org/x/sys v0.0.0-20191224085550-c709ea063b76
+	golang.org/x/tools v0.0.0-20191224055732-dd894d0a8a40 // indirect
+	google.golang.org/api v0.15.0
 	google.golang.org/appengine v1.6.5 // indirect
-	google.golang.org/genproto v0.0.0-20191206224255-0243a4be9c8f // indirect
-	google.golang.org/grpc v1.25.1 // indirect
+	google.golang.org/genproto v0.0.0-20191223191004-3caeed10a8bf // indirect
+	google.golang.org/grpc v1.26.0 // indirect
 )

 go 1.12
--- a/go.sum
+++ b/go.sum
@@ -5,8 +5,8 @@ cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6A
 cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
 cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=
 cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=
-cloud.google.com/go v0.49.0 h1:CH+lkubJzcPYB1Ggupcq0+k8Ni2ILdG2lYjDIgavDBQ=
-cloud.google.com/go v0.49.0/go.mod h1:hGvAdzcWNbyuxS3nWhD7H2cIJxjRRTRLQVB0bdputVY=
+cloud.google.com/go v0.50.0 h1:0E3eE8MX426vUOs7aHfI7aN1BrIzzzf4ccKCSfSjGmc=
+cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=
 cloud.google.com/go/bigquery v1.0.1 h1:hL+ycaJpVE9M7nLoiXb/Pn10ENE2u+oddxbD8uu0ZVU=
 cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
 cloud.google.com/go/datastore v1.0.0 h1:Kt+gOPPp2LEPWp8CSfxhsM8ik9CcyE/gYu+0r+RnZvM=
@@ -20,14 +20,14 @@ dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7
 github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
 github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
 github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
-github.com/VictoriaMetrics/fastcache v1.5.4 h1:0BaXbRH01RycJk79OOBwMCXlNryko9z4yEf6RqbP+Xo=
-github.com/VictoriaMetrics/fastcache v1.5.4/go.mod h1:ptDBkNMQI4RtmVo8VS/XwRY6RoTu1dAWCbrk+6WsEM8=
+github.com/VictoriaMetrics/fastcache v1.5.5 h1:HsBlzPgzKG0566YOl1mmfyz8SCU0zLKfbl9RDLsiLD8=
+github.com/VictoriaMetrics/fastcache v1.5.5/go.mod h1:ptDBkNMQI4RtmVo8VS/XwRY6RoTu1dAWCbrk+6WsEM8=
 github.com/VictoriaMetrics/metrics v1.9.3 h1:+1kZnOIb8RY825Nb9q9yMrPcOYuPE2GrZWxUh59XnHI=
 github.com/VictoriaMetrics/metrics v1.9.3/go.mod h1:LU2j9qq7xqZYXz8tF3/RQnB2z2MbZms5TDiIg9/NHiQ=
 github.com/allegro/bigcache v1.2.1-0.20190218064605-e24eb225f156 h1:eMwmnE/GDgah4HI848JfFxHt+iPb26b4zyfspmqY0/8=
 github.com/allegro/bigcache v1.2.1-0.20190218064605-e24eb225f156/go.mod h1:Cb/ax3seSYIx7SuZdm2G2xzfwmv3TPSk2ucNfQESPXM=
-github.com/aws/aws-sdk-go v1.26.0 h1:uddjCeunG1nroOMjcMO57g/vfwY7jJ1zYSEGmdeCeEs=
-github.com/aws/aws-sdk-go v1.26.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
+github.com/aws/aws-sdk-go v1.26.8 h1:W+MPuCFLSO/itZkZ5GFOui0YC1j3lZ507/m5DFPtzE4=
+github.com/aws/aws-sdk-go v1.26.8/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
 github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
 github.com/cespare/xxhash/v2 v2.1.1 h1:6MnRN8NT7+YBpUIWxHtefFZOKTAPgGjpQSxqLNn0+qY=
 github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
@@ -35,7 +35,7 @@ github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDk
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
+github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
 github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
 github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
 github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
@@ -97,12 +97,12 @@ github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81P
 github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
 github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
 github.com/valyala/fasthttp v1.2.0/go.mod h1:4vX61m6KN+xDduDNwXrhIAVZaZaZiQ1luJk8LWSxF3s=
-github.com/valyala/fastjson v1.4.1 h1:hrltpHpIpkaxll8QltMU8c3QZ5+qIiCL8yKqPFJI/yE=
-github.com/valyala/fastjson v1.4.1/go.mod h1:nV6MsjxL2IMJQUoHDIrjEI7oLyeqK6aBD7EFWPsvP8o=
+github.com/valyala/fastjson v1.4.5 h1:uSuLfXk2LzRtzwd3Fy5zGRBe0Vs7zhs11vjdko32xb4=
+github.com/valyala/fastjson v1.4.5/go.mod h1:nV6MsjxL2IMJQUoHDIrjEI7oLyeqK6aBD7EFWPsvP8o=
 github.com/valyala/fastrand v1.0.0 h1:LUKT9aKer2dVQNUi3waewTbKV+7H17kvWFNKs2ObdkI=
 github.com/valyala/fastrand v1.0.0/go.mod h1:HWqCzkrkg6QXT8V2EXWvXCoow7vLwOFN002oeRzjapQ=
-github.com/valyala/gozstd v1.6.3 h1:kr3oF/F1RvxYr8wgPjrH04gvHuMEL99pPC9e+5pPQdU=
-github.com/valyala/gozstd v1.6.3/go.mod h1:y5Ew47GLlP37EkTB+B4s7r6A5rdaeB7ftbl9zoYiIPQ=
+github.com/valyala/gozstd v1.6.4 h1:nFLddjEf90SFl5cVWyElSHozQDsbvLljPK703/skBS0=
+github.com/valyala/gozstd v1.6.4/go.mod h1:y5Ew47GLlP37EkTB+B4s7r6A5rdaeB7ftbl9zoYiIPQ=
 github.com/valyala/histogram v1.0.1 h1:FzA7n2Tz/wKRMejgu3PV1vw3htAklTjjuoI6z3d4KDg=
 github.com/valyala/histogram v1.0.1/go.mod h1:lQy0xA4wUz2+IUnf97SivorsJIp8FxsnRd6x25q7Mto=
 github.com/valyala/quicktemplate v1.4.1 h1:tEtkSN6mTCJlYVT7As5x4wjtkk2hj2thsb0M+AcAVeM=
@@ -115,6 +115,7 @@ go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
 golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -123,6 +124,8 @@ golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136 h1:A1gGSx58LAGVHUUsOf7IiR0u8
 golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=
 golang.org/x/exp v0.0.0-20191129062945-2f5052295587 h1:5Uz0rkjCFu9BC9gCRN7EkwVvhNyQgGWb8KNJrPwBoHY=
 golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
+golang.org/x/exp v0.0.0-20191224044220-1fea468a75e9 h1:HLuLY2KniBsHW28uXd1i2UZKjifeJUy//P/wTK6AJwI=
+golang.org/x/exp v0.0.0-20191224044220-1fea468a75e9/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
 golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
 golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
 golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
@@ -139,6 +142,7 @@ golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU
 golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
 golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
 golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
+golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
 golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20180911220305-26e67e76b6c3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -150,6 +154,7 @@ golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn
 golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
 golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553 h1:efeOvDhwQ29Dj3SdAV/MJf8oukgn+8D8WgaCaRMchF8=
 golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
@@ -164,6 +169,8 @@ golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJ
 golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
 golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e h1:vcxGaoTs7kV8m5Np9uUNQin4BrLOthgV7252N8V+FwY=
+golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -172,8 +179,9 @@ golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20191210023423-ac6580df4449 h1:gSbV7h1NRL2G1xTg/owz62CST1oJBmxy4QpMMregXVQ=
-golang.org/x/sys v0.0.0-20191210023423-ac6580df4449/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191224085550-c709ea063b76 h1:Dho5nD6R3PcW2SH1or8vS0dszDaXRxIw55lBX7XiE5g=
+golang.org/x/sys v0.0.0-20191224085550-c709ea063b76/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
@@ -197,15 +205,19 @@ golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtn
 golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
 golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
 golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20191212051200-825cb0626375 h1:hrBCt+pkb1irxFjg/GeadSn24gLxi5/Z861I53OkQbc=
-golang.org/x/tools v0.0.0-20191212051200-825cb0626375/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.0.0-20191224055732-dd894d0a8a40 h1:UyP2XDSgSc8ldYCxAK735zQxeH3Gd81sK7Iy7AoaVxk=
+golang.org/x/tools v0.0.0-20191224055732-dd894d0a8a40/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
 google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
 google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
 google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
 google.golang.org/api v0.14.0 h1:uMf5uLi4eQMRrMKhCplNik4U4H8Z6C1br3zOtAa/aDE=
 google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
+google.golang.org/api v0.15.0 h1:yzlyyDW/J0w8yNFJIhiAJy4kq74S+1DOLdawELNxFMA=
+google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
 google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
 google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
 google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
@@ -221,14 +233,15 @@ google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98
 google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
 google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=
 google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
-google.golang.org/genproto v0.0.0-20191206224255-0243a4be9c8f h1:naitw5DILWPQvG0oG04mR9jF8fmKpRdW3E3zzKA4D0Y=
-google.golang.org/genproto v0.0.0-20191206224255-0243a4be9c8f/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
+google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
+google.golang.org/genproto v0.0.0-20191223191004-3caeed10a8bf h1:1x8rC5/IgdLMPbPTvlQTN28+rcy8XL9Q19UWUMDyqYs=
+google.golang.org/genproto v0.0.0-20191223191004-3caeed10a8bf/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
 google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
 google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
 google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
 google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
-google.golang.org/grpc v1.25.1 h1:wdKvqQk7IttEw92GoRyKG2IDrUIpgpj6H6m81yfeMW0=
-google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
+google.golang.org/grpc v1.26.0 h1:2dTRdpdFEEhJYQD8EMLB61nnrzSCTbG38PhqdhvOltg=
+google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
--- a/lib/backup/actions/backup.go
+++ b/lib/backup/actions/backup.go
@@ -7,6 +7,7 @@ import (
 	"time"

 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/backup/common"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/backup/fscommon"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/backup/fslocal"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/backup/fsnil"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
@@ -41,17 +42,33 @@ type Backup struct {

 // Run runs b with the provided settings.
 func (b *Backup) Run() error {
-	startTime := time.Now()
-
 	concurrency := b.Concurrency
 	src := b.Src
 	dst := b.Dst
 	origin := b.Origin

+	if origin != nil && origin.String() == dst.String() {
+		origin = nil
+	}
 	if origin == nil {
 		origin = &fsnil.FS{}
 	}

+	if err := dst.DeleteFile(fscommon.BackupCompleteFilename); err != nil {
+		return fmt.Errorf("cannot delete `backup complete` file at %s: %s", dst, err)
+	}
+	if err := runBackup(src, dst, origin, concurrency); err != nil {
+		return err
+	}
+	if err := dst.CreateFile(fscommon.BackupCompleteFilename, []byte("ok")); err != nil {
+		return fmt.Errorf("cannot create `backup complete` file at %s: %s", dst, err)
+	}
+	return nil
+}
+
+func runBackup(src *fslocal.FS, dst common.RemoteFS, origin common.OriginFS, concurrency int) error {
+	startTime := time.Now()
+
 	logger.Infof("starting backup from %s to %s using origin %s", src, dst, origin)

 	logger.Infof("obtaining list of parts at %s", src)
--- a/lib/backup/actions/restore.go
+++ b/lib/backup/actions/restore.go
@@ -7,6 +7,7 @@ import (
 	"time"

 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/backup/common"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/backup/fscommon"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/backup/fslocal"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/fs"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
@@ -29,6 +30,11 @@ type Restore struct {
 	// If dst points to existing directory, then incremental restore is performed,
 	// i.e. only new data is downloaded from src.
 	Dst *fslocal.FS
+
+	// SkipBackupCompleteCheck may be set in order to skip for `backup complete` file in Src.
+	//
+	// This may be needed for restoring from old backups with missing `backup complete` file.
+	SkipBackupCompleteCheck bool
 }

 // Run runs r with the provided settings.
@@ -48,6 +54,18 @@ func (r *Restore) Run() error {
 	concurrency := r.Concurrency
 	src := r.Src
 	dst := r.Dst
+
+	if !r.SkipBackupCompleteCheck {
+		ok, err := src.HasFile(fscommon.BackupCompleteFilename)
+		if err != nil {
+			return err
+		}
+		if !ok {
+			return fmt.Errorf("cannot find %s file in %s; this means either incomplete backup or old backup; "+
+				"pass `-skipBackupCompleteCheck` command-line flag if you still need restoring from this backup", fscommon.BackupCompleteFilename, src)
+		}
+	}
+
 	logger.Infof("starting restore from %s to %s", src, dst)

 	logger.Infof("obtaining list of parts at %s", src)
--- a/lib/backup/common/fs.go
+++ b/lib/backup/common/fs.go
@@ -38,4 +38,13 @@ type RemoteFS interface {

 	// UploadPart must upload part p from r to RemoteFS.
 	UploadPart(p Part, r io.Reader) error
+
+	// DeleteFile deletes filePath at RemoteFS
+	DeleteFile(filePath string) error
+
+	// CreateFile creates filePath at RemoteFS and puts data into it.
+	CreateFile(filePath string, data []byte) error
+
+	// HasFile returns true if filePath exists at RemoteFS.
+	HasFile(filePath string) (bool, error)
 }
--- a/lib/backup/fscommon/fscommon.go
+++ b/lib/backup/fscommon/fscommon.go
@@ -260,3 +260,11 @@ func removeEmptyDirsInternal(d *os.File) (bool, error) {
 	}
 	return true, nil
 }
+
+// IgnorePath returns true if the given path must be ignored.
+func IgnorePath(path string) bool {
+	return strings.HasSuffix(path, ".ignore")
+}
+
+// BackupCompleteFilename is a filename, which is created in the destination fs when backup is complete.
+const BackupCompleteFilename = "backup_complete.ignore"
--- a/lib/backup/fsremote/fsremote.go
+++ b/lib/backup/fsremote/fsremote.go
@@ -3,6 +3,7 @@ package fsremote
 import (
 	"fmt"
 	"io"
+	"io/ioutil"
 	"os"
 	"path/filepath"
 	"strings"
@@ -48,6 +49,9 @@ func (fs *FS) ListParts() ([]common.Part, error) {
 		if !strings.HasPrefix(file, dir) {
 			logger.Panicf("BUG: unexpected prefix for file %q; want %q", file, dir)
 		}
+		if fscommon.IgnorePath(file) {
+			continue
+		}
 		var p common.Part
 		if !p.ParseFromRemotePath(file[len(dir):]) {
 			logger.Infof("skipping unknown file %s", file)
@@ -188,3 +192,42 @@ func (fs *FS) mkdirAll(filePath string) error {
 func (fs *FS) path(p common.Part) string {
 	return p.RemotePath(fs.Dir)
 }
+
+// DeleteFile deletes filePath at fs.
+//
+// The function does nothing if the filePath doesn't exist.
+func (fs *FS) DeleteFile(filePath string) error {
+	path := filepath.Join(fs.Dir, filePath)
+	err := os.Remove(path)
+	if err != nil && !os.IsNotExist(err) {
+		return fmt.Errorf("cannot remove %q: %s", path, err)
+	}
+	return nil
+}
+
+// CreateFile creates filePath at fs and puts data into it.
+//
+// The file is overwritten if it exists.
+func (fs *FS) CreateFile(filePath string, data []byte) error {
+	path := filepath.Join(fs.Dir, filePath)
+	if err := ioutil.WriteFile(path, data, 0600); err != nil {
+		return fmt.Errorf("cannot write %d bytes to %q: %s", len(data), path, err)
+	}
+	return nil
+}
+
+// HasFile returns true if filePath exists at fs.
+func (fs *FS) HasFile(filePath string) (bool, error) {
+	path := filepath.Join(fs.Dir, filePath)
+	fi, err := os.Stat(path)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return false, nil
+		}
+		return false, fmt.Errorf("cannot stat %q: %s", path, err)
+	}
+	if fi.IsDir() {
+		return false, fmt.Errorf("%q is directory, while file is needed", path)
+	}
+	return true, nil
+}
--- a/lib/backup/gcsremote/gcs.go
+++ b/lib/backup/gcsremote/gcs.go
@@ -8,6 +8,7 @@ import (

 	"cloud.google.com/go/storage"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/backup/common"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/backup/fscommon"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
 	"google.golang.org/api/iterator"
 	"google.golang.org/api/option"
@@ -97,6 +98,9 @@ func (fs *FS) ListParts() ([]common.Part, error) {
 		if !strings.HasPrefix(file, dir) {
 			return nil, fmt.Errorf("unexpected prefix for gcs key %q; want %q", file, dir)
 		}
+		if fscommon.IgnorePath(file) {
+			continue
+		}
 		var p common.Part
 		if !p.ParseFromRemotePath(file[len(dir):]) {
 			logger.Infof("skipping unknown object %q", file)
@@ -187,3 +191,56 @@ func (fs *FS) object(p common.Part) *storage.ObjectHandle {
 	path := p.RemotePath(fs.Dir)
 	return fs.bkt.Object(path)
 }
+
+// DeleteFile deletes filePath at fs if it exists.
+//
+// The function does nothing if the filePath doesn't exists.
+func (fs *FS) DeleteFile(filePath string) error {
+	path := fs.Dir + filePath
+	o := fs.bkt.Object(path)
+	ctx := context.Background()
+	if err := o.Delete(ctx); err != nil {
+		if err != storage.ErrObjectNotExist {
+			return fmt.Errorf("cannot delete %q at %s (remote path %q): %s", filePath, fs, o.ObjectName(), err)
+		}
+	}
+	return nil
+}
+
+// CreateFile creates filePath at fs and puts data into it.
+//
+// The file is overwritten if it exists.
+func (fs *FS) CreateFile(filePath string, data []byte) error {
+	path := fs.Dir + filePath
+	o := fs.bkt.Object(path)
+	ctx := context.Background()
+	w := o.NewWriter(ctx)
+	n, err := w.Write(data)
+	if err != nil {
+		_ = w.Close()
+		return fmt.Errorf("cannot upload %d bytes to %q at %s (remote path %q): %s", len(data), filePath, fs, o.ObjectName(), err)
+	}
+	if n != len(data) {
+		_ = w.Close()
+		return fmt.Errorf("wrong data size uploaded to %q at %s (remote path %q); got %d bytes; want %d bytes", filePath, fs, o.ObjectName(), n, len(data))
+	}
+	if err := w.Close(); err != nil {
+		return fmt.Errorf("cannot close %q at %s (remote path %q): %s", filePath, fs, o.ObjectName(), err)
+	}
+	return nil
+}
+
+// HasFile returns ture if filePath exists at fs.
+func (fs *FS) HasFile(filePath string) (bool, error) {
+	path := fs.Dir + filePath
+	o := fs.bkt.Object(path)
+	ctx := context.Background()
+	_, err := o.Attrs(ctx)
+	if err != nil {
+		if err == storage.ErrObjectNotExist {
+			return false, nil
+		}
+		return false, fmt.Errorf("unexpected error when obtaining attributes for %q at %s (remote path %q): %s", filePath, fs, o.ObjectName(), err)
+	}
+	return true, nil
+}
--- a/lib/backup/s3remote/s3.go
+++ b/lib/backup/s3remote/s3.go
@@ -1,14 +1,17 @@
 package s3remote

 import (
+	"bytes"
 	"context"
 	"fmt"
 	"io"
 	"strings"

 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/backup/common"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/backup/fscommon"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
 	"github.com/aws/aws-sdk-go/aws"
+	"github.com/aws/aws-sdk-go/aws/awserr"
 	"github.com/aws/aws-sdk-go/aws/session"
 	"github.com/aws/aws-sdk-go/service/s3"
 	"github.com/aws/aws-sdk-go/service/s3/s3manager"
@@ -113,6 +116,9 @@ func (fs *FS) ListParts() ([]common.Part, error) {
 				errOuter = fmt.Errorf("unexpected prefix for s3 key %q; want %q", file, dir)
 				return false
 			}
+			if fscommon.IgnorePath(file) {
+				continue
+			}
 			var p common.Part
 			if !p.ParseFromRemotePath(file[len(dir):]) {
 				logger.Infof("skipping unknown object %q", file)
@@ -220,6 +226,78 @@ func (fs *FS) UploadPart(p common.Part, r io.Reader) error {
 	return nil
 }

+// DeleteFile deletes filePath from fs if it exists.
+//
+// The function does nothing if the file doesn't exist.
+func (fs *FS) DeleteFile(filePath string) error {
+	// It looks like s3 may return `AccessDenied: Access Denied` instead of `s3.ErrCodeNoSuchKey`
+	// on an attempt to delete non-existing file.
+	// so just check whether the filePath exists before deleting it.
+	// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/284 for details.
+	ok, err := fs.HasFile(filePath)
+	if err != nil {
+		return err
+	}
+	if !ok {
+		// Missing file - nothing to delete.
+		return nil
+	}
+
+	path := fs.Dir + filePath
+	input := &s3.DeleteObjectInput{
+		Bucket: aws.String(fs.Bucket),
+		Key:    aws.String(path),
+	}
+	if _, err := fs.s3.DeleteObject(input); err != nil {
+		return fmt.Errorf("cannot delete %q at %s (remote path %q): %s", filePath, fs, path, err)
+	}
+	return nil
+}
+
+// CreateFile creates filePath at fs and puts data into it.
+//
+// The file is overwritten if it already exists.
+func (fs *FS) CreateFile(filePath string, data []byte) error {
+	path := fs.Dir + filePath
+	sr := &statReader{
+		r: bytes.NewReader(data),
+	}
+	input := &s3manager.UploadInput{
+		Bucket: aws.String(fs.Bucket),
+		Key:    aws.String(path),
+		Body:   sr,
+	}
+	_, err := fs.uploader.Upload(input)
+	if err != nil {
+		return fmt.Errorf("cannot upoad data to %q at %s (remote path %q): %s", filePath, fs, path, err)
+	}
+	l := int64(len(data))
+	if sr.size != l {
+		return fmt.Errorf("wrong data size uploaded to %q at %s; got %d bytes; want %d bytes", filePath, fs, sr.size, l)
+	}
+	return nil
+}
+
+// HasFile returns true if filePath exists at fs.
+func (fs *FS) HasFile(filePath string) (bool, error) {
+	path := fs.Dir + filePath
+	input := &s3.GetObjectInput{
+		Bucket: aws.String(fs.Bucket),
+		Key:    aws.String(path),
+	}
+	o, err := fs.s3.GetObject(input)
+	if err != nil {
+		if ae, ok := err.(awserr.Error); ok && ae.Code() == s3.ErrCodeNoSuchKey {
+			return false, nil
+		}
+		return false, fmt.Errorf("cannot open %q at %s (remote path %q): %s", filePath, fs, path, err)
+	}
+	if err := o.Body.Close(); err != nil {
+		return false, fmt.Errorf("cannot close %q at %s (remote path %q): %s", filePath, fs, path, err)
+	}
+	return true, nil
+}
+
 func (fs *FS) path(p common.Part) string {
 	return p.RemotePath(fs.Dir)
 }
--- a/lib/encoding/encoding.go
+++ b/lib/encoding/encoding.go
@@ -166,24 +166,24 @@ func unmarshalInt64Array(dst []int64, src []byte, mt MarshalType, firstValue int
 		bb := bbPool.Get()
 		bb.B, err = DecompressZSTD(bb.B[:0], src)
 		if err != nil {
-			return nil, fmt.Errorf("cannot decompress zstd data of size %d: %s", len(src), err)
+			return nil, fmt.Errorf("cannot decompress zstd data of size %d: %s; src_zstd=%X", len(src), err, src)
 		}
 		dst, err = unmarshalInt64NearestDelta(dst, bb.B, firstValue, itemsCount)
 		bbPool.Put(bb)
 		if err != nil {
-			return nil, fmt.Errorf("cannot unmarshal nearest delta data after zstd decompression: %s", err)
+			return nil, fmt.Errorf("cannot unmarshal nearest delta data after zstd decompression: %s; src_zstd=%X", err, src)
 		}
 		return dst, nil
 	case MarshalTypeZSTDNearestDelta2:
 		bb := bbPool.Get()
 		bb.B, err = DecompressZSTD(bb.B[:0], src)
 		if err != nil {
-			return nil, fmt.Errorf("cannot decompress zstd data of size %d: %s", len(src), err)
+			return nil, fmt.Errorf("cannot decompress zstd data of size %d: %s; src_zstd=%X", len(src), err, src)
 		}
 		dst, err = unmarshalInt64NearestDelta2(dst, bb.B, firstValue, itemsCount)
 		bbPool.Put(bb)
 		if err != nil {
-			return nil, fmt.Errorf("cannot unmarshal nearest delta2 data after zstd decompression: %s", err)
+			return nil, fmt.Errorf("cannot unmarshal nearest delta2 data after zstd decompression: %s; src_zstd=%X", err, src)
 		}
 		return dst, nil
 	case MarshalTypeNearestDelta:
--- a/lib/encoding/nearest_delta.go
+++ b/lib/encoding/nearest_delta.go
@@ -60,10 +60,10 @@ func unmarshalInt64NearestDelta(dst []int64, src []byte, firstValue int64, items

 	tail, err := UnmarshalVarInt64s(is.A, src)
 	if err != nil {
-		return nil, fmt.Errorf("cannot unmarshal nearest delta: %s", err)
+		return nil, fmt.Errorf("cannot unmarshal nearest delta from %d bytes; src=%X: %s", len(src), src, err)
 	}
 	if len(tail) > 0 {
-		return nil, fmt.Errorf("unexpected tail left after unmarshaling %d items; tail size=%d, value=%X", itemsCount, len(tail), tail)
+		return nil, fmt.Errorf("unexpected tail left after unmarshaling %d items from %d bytes; tail size=%d; src=%X; tail=%X", itemsCount, len(src), len(tail), src, tail)
 	}

 	v := firstValue
--- a/lib/encoding/nearest_delta2.go
+++ b/lib/encoding/nearest_delta2.go
@@ -63,10 +63,10 @@ func unmarshalInt64NearestDelta2(dst []int64, src []byte, firstValue int64, item

 	tail, err := UnmarshalVarInt64s(is.A, src)
 	if err != nil {
-		return nil, fmt.Errorf("cannot unmarshal nearest delta: %s", err)
+		return nil, fmt.Errorf("cannot unmarshal nearest delta from %d bytes; src=%X: %s", len(src), src, err)
 	}
 	if len(tail) > 0 {
-		return nil, fmt.Errorf("unexpected tail left after unmarshaling %d items; tail size=%d, value=%X", itemsCount, len(tail), tail)
+		return nil, fmt.Errorf("unexpected tail left after unmarshaling %d items from %d bytes; tail size=%d; src=%X; tail=%X", itemsCount, len(src), len(tail), src, tail)
 	}

 	v := firstValue
--- a/lib/encoding/zstd/zstd_pure.go
+++ b/lib/encoding/zstd/zstd_pure.go
@@ -53,14 +53,16 @@ func getEncoder(compressionLevel int) *zstd.Encoder {
 	mu.Lock()
 	// Create the encoder under lock in order to prevent from wasted work
 	// when concurrent goroutines create encoder for the same compressionLevel.
-	e = newEncoder(compressionLevel)
 	r1 := av.Load().(registry)
-	r2 := make(registry)
-	for k, v := range r1 {
-		r2[k] = v
+	if e = r1[compressionLevel]; e == nil {
+		e = newEncoder(compressionLevel)
+		r2 := make(registry)
+		for k, v := range r1 {
+			r2[k] = v
+		}
+		r2[compressionLevel] = e
+		av.Store(r2)
 	}
-	r2[compressionLevel] = e
-	av.Store(r2)
 	mu.Unlock()

 	return e
--- a/lib/fs/fadvise_darwin.go
+++ b/lib/fs/fadvise_darwin.go
@@ -0,0 +1,12 @@
+package fs
+
+import (
+	"os"
+)
+
+// MustFadviseSequentialRead hints the OS that f is read mostly sequentially.
+//
+// if prefetch is set, then the OS is hinted to prefetch f data.
+func MustFadviseSequentialRead(f *os.File, prefetch bool) {
+	// TODO: implement this properly
+}
--- a/lib/fs/fadvise_unix.go
+++ b/lib/fs/fadvise_unix.go
@@ -0,0 +1,24 @@
+// +build linux freebsd
+
+package fs
+
+import (
+	"os"
+
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
+	"golang.org/x/sys/unix"
+)
+
+// MustFadviseSequentialRead hints the OS that f is read mostly sequentially.
+//
+// if prefetch is set, then the OS is hinted to prefetch f data.
+func MustFadviseSequentialRead(f *os.File, prefetch bool) {
+	fd := int(f.Fd())
+	mode := unix.FADV_SEQUENTIAL
+	if prefetch {
+		mode |= unix.FADV_WILLNEED
+	}
+	if err := unix.Fadvise(int(fd), 0, 0, mode); err != nil {
+		logger.Panicf("FATAL: error returned from unix.Fadvise(%d): %s", mode, err)
+	}
+}
--- a/lib/httpserver/httpserver.go
+++ b/lib/httpserver/httpserver.go
@@ -155,9 +155,6 @@ var metricsHandlerDuration = metrics.NewHistogram(`vm_http_request_duration_seco

 func handlerWrapper(w http.ResponseWriter, r *http.Request, rh RequestHandler) {
 	requestsTotal.Inc()
-	if !checkAuth(w, r) {
-		return
-	}
 	switch r.URL.Path {
 	case "/health":
 		w.Header().Set("Content-Type", "text/plain")
@@ -172,25 +169,36 @@ func handlerWrapper(w http.ResponseWriter, r *http.Request, rh RequestHandler) {
 		}
 		w.WriteHeader(status)
 		return
-	case "/metrics":
-		startTime := time.Now()
-		metricsRequests.Inc()
-		w.Header().Set("Content-Type", "text/plain")
-		writePrometheusMetrics(w)
-		metricsHandlerDuration.UpdateDuration(startTime)
-		return
 	case "/favicon.ico":
 		faviconRequests.Inc()
 		w.WriteHeader(http.StatusNoContent)
 		return
+	case "/metrics":
+		metricsRequests.Inc()
+		if len(*metricsAuthKey) > 0 && r.FormValue("authKey") != *metricsAuthKey {
+			http.Error(w, "The provided authKey doesn't match -metricsAuthKey", http.StatusUnauthorized)
+			return
+		}
+		startTime := time.Now()
+		w.Header().Set("Content-Type", "text/plain")
+		writePrometheusMetrics(w)
+		metricsHandlerDuration.UpdateDuration(startTime)
+		return
 	default:
 		if strings.HasPrefix(r.URL.Path, "/debug/pprof/") {
 			pprofRequests.Inc()
+			if len(*pprofAuthKey) > 0 && r.FormValue("authKey") != *pprofAuthKey {
+				http.Error(w, "The provided authKey doesn't match -pprofAuthKey", http.StatusUnauthorized)
+				return
+			}
 			DisableResponseCompression(w)
 			pprofHandler(r.URL.Path[len("/debug/pprof/"):], w, r)
 			return
 		}

+		if !checkBasicAuth(w, r) {
+			return
+		}
 		if rh(w, r) {
 			return
 		}
@@ -201,27 +209,6 @@ func handlerWrapper(w http.ResponseWriter, r *http.Request, rh RequestHandler) {
 	}
 }

-func checkAuth(w http.ResponseWriter, r *http.Request) bool {
-	path := r.URL.Path
-	if path == "/metrics" && len(*metricsAuthKey) > 0 {
-		authKey := r.FormValue("authKey")
-		if *metricsAuthKey == authKey {
-			return true
-		}
-		http.Error(w, "The provided authKey doesn't match -metricsAuthKey", http.StatusUnauthorized)
-		return false
-	}
-	if strings.HasPrefix(path, "/debug/pprof/") && len(*pprofAuthKey) > 0 {
-		authKey := r.FormValue("authKey")
-		if *pprofAuthKey == authKey {
-			return true
-		}
-		http.Error(w, "The provided authKey doesn't match -pprofAuthKey", http.StatusUnauthorized)
-		return false
-	}
-	return checkBasicAuth(w, r)
-}
-
 func checkBasicAuth(w http.ResponseWriter, r *http.Request) bool {
 	if len(*httpAuthUsername) == 0 {
 		// HTTP Basic Auth is disabled.
--- a/lib/metricsql/aggr.go
+++ b/lib/metricsql/aggr.go
@@ -0,0 +1,51 @@
+package metricsql
+
+import (
+	"strings"
+)
+
+var aggrFuncs = map[string]bool{
+	// See https://prometheus.io/docs/prometheus/latest/querying/operators/#aggregation-operators
+	"sum":          true,
+	"min":          true,
+	"max":          true,
+	"avg":          true,
+	"stddev":       true,
+	"stdvar":       true,
+	"count":        true,
+	"count_values": true,
+	"bottomk":      true,
+	"topk":         true,
+	"quantile":     true,
+
+	// MetricsQL extension funcs
+	"median":         true,
+	"limitk":         true,
+	"distinct":       true,
+	"sum2":           true,
+	"geomean":        true,
+	"histogram":      true,
+	"topk_min":       true,
+	"topk_max":       true,
+	"topk_avg":       true,
+	"topk_median":    true,
+	"bottomk_min":    true,
+	"bottomk_max":    true,
+	"bottomk_avg":    true,
+	"bottomk_median": true,
+}
+
+func isAggrFunc(s string) bool {
+	s = strings.ToLower(s)
+	return aggrFuncs[s]
+}
+
+func isAggrFuncModifier(s string) bool {
+	s = strings.ToLower(s)
+	switch s {
+	case "by", "without":
+		return true
+	default:
+		return false
+	}
+}
--- a/app/vmselect/promql/aggr_test.go
+++ b/app/vmselect/promql/aggr_test.go
@@ -1,4 +1,4 @@
-package promql
+package metricsql

 import (
 	"testing"
--- a/lib/metricsql/binary_op.go
+++ b/lib/metricsql/binary_op.go
@@ -0,0 +1,205 @@
+package metricsql
+
+import (
+	"fmt"
+	"math"
+	"strings"
+
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql/binaryop"
+)
+
+var binaryOps = map[string]bool{
+	"+": true,
+	"-": true,
+	"*": true,
+	"/": true,
+	"%": true,
+	"^": true,
+
+	// cmp ops
+	"==": true,
+	"!=": true,
+	">":  true,
+	"<":  true,
+	">=": true,
+	"<=": true,
+
+	// logical set ops
+	"and":    true,
+	"or":     true,
+	"unless": true,
+
+	// New ops for MetricsQL
+	"if":      true,
+	"ifnot":   true,
+	"default": true,
+}
+
+var binaryOpPriorities = map[string]int{
+	"default": -1,
+
+	"if":    0,
+	"ifnot": 0,
+
+	// See https://prometheus.io/docs/prometheus/latest/querying/operators/#binary-operator-precedence
+	"or": 1,
+
+	"and":    2,
+	"unless": 2,
+
+	"==": 3,
+	"!=": 3,
+	"<":  3,
+	">":  3,
+	"<=": 3,
+	">=": 3,
+
+	"+": 4,
+	"-": 4,
+
+	"*": 5,
+	"/": 5,
+	"%": 5,
+
+	"^": 6,
+}
+
+func isBinaryOp(op string) bool {
+	op = strings.ToLower(op)
+	return binaryOps[op]
+}
+
+func binaryOpPriority(op string) int {
+	op = strings.ToLower(op)
+	return binaryOpPriorities[op]
+}
+
+func scanBinaryOpPrefix(s string) int {
+	n := 0
+	for op := range binaryOps {
+		if len(s) < len(op) {
+			continue
+		}
+		ss := strings.ToLower(s[:len(op)])
+		if ss == op && len(op) > n {
+			n = len(op)
+		}
+	}
+	return n
+}
+
+func isRightAssociativeBinaryOp(op string) bool {
+	// See https://prometheus.io/docs/prometheus/latest/querying/operators/#binary-operator-precedence
+	return op == "^"
+}
+
+func isBinaryOpGroupModifier(s string) bool {
+	s = strings.ToLower(s)
+	switch s {
+	// See https://prometheus.io/docs/prometheus/latest/querying/operators/#vector-matching
+	case "on", "ignoring":
+		return true
+	default:
+		return false
+	}
+}
+
+func isBinaryOpJoinModifier(s string) bool {
+	s = strings.ToLower(s)
+	switch s {
+	case "group_left", "group_right":
+		return true
+	default:
+		return false
+	}
+}
+
+func isBinaryOpBoolModifier(s string) bool {
+	s = strings.ToLower(s)
+	return s == "bool"
+}
+
+// IsBinaryOpCmp returns true if op is comparison operator such as '==', '!=', etc.
+func IsBinaryOpCmp(op string) bool {
+	switch op {
+	case "==", "!=", ">", "<", ">=", "<=":
+		return true
+	default:
+		return false
+	}
+}
+
+func isBinaryOpLogicalSet(op string) bool {
+	op = strings.ToLower(op)
+	switch op {
+	case "and", "or", "unless":
+		return true
+	default:
+		return false
+	}
+}
+
+func binaryOpEval(op string, left, right float64, isBool bool) float64 {
+	if IsBinaryOpCmp(op) {
+		evalCmp := func(cf func(left, right float64) bool) float64 {
+			if isBool {
+				if cf(left, right) {
+					return 1
+				}
+				return 0
+			}
+			if cf(left, right) {
+				return left
+			}
+			return nan
+		}
+		switch op {
+		case "==":
+			left = evalCmp(binaryop.Eq)
+		case "!=":
+			left = evalCmp(binaryop.Neq)
+		case ">":
+			left = evalCmp(binaryop.Gt)
+		case "<":
+			left = evalCmp(binaryop.Lt)
+		case ">=":
+			left = evalCmp(binaryop.Gte)
+		case "<=":
+			left = evalCmp(binaryop.Lte)
+		default:
+			panic(fmt.Errorf("BUG: unexpected comparison binaryOp: %q", op))
+		}
+	} else {
+		switch op {
+		case "+":
+			left = binaryop.Plus(left, right)
+		case "-":
+			left = binaryop.Minus(left, right)
+		case "*":
+			left = binaryop.Mul(left, right)
+		case "/":
+			left = binaryop.Div(left, right)
+		case "%":
+			left = binaryop.Mod(left, right)
+		case "^":
+			left = binaryop.Pow(left, right)
+		case "and":
+			// Nothing to do
+		case "or":
+			// Nothing to do
+		case "unless":
+			left = nan
+		case "default":
+			left = binaryop.Default(left, right)
+		case "if":
+			left = binaryop.If(left, right)
+		case "ifnot":
+			left = binaryop.Ifnot(left, right)
+		default:
+			panic(fmt.Errorf("BUG: unexpected non-comparison binaryOp: %q", op))
+		}
+	}
+	return left
+}
+
+var nan = math.NaN()
--- a/app/vmselect/promql/binary_op_test.go
+++ b/app/vmselect/promql/binary_op_test.go
@@ -1,4 +1,4 @@
-package promql
+package metricsql

 import (
 	"testing"
--- a/lib/metricsql/binaryop/funcs.go
+++ b/lib/metricsql/binaryop/funcs.go
@@ -0,0 +1,104 @@
+package binaryop
+
+import (
+	"math"
+)
+
+var nan = math.NaN()
+
+// Eq returns true of left == right.
+func Eq(left, right float64) bool {
+	// Special handling for nan == nan.
+	// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150 .
+	if math.IsNaN(left) {
+		return math.IsNaN(right)
+	}
+	return left == right
+}
+
+// Neq returns true of left != right.
+func Neq(left, right float64) bool {
+	// Special handling for comparison with nan.
+	// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150 .
+	if math.IsNaN(left) {
+		return !math.IsNaN(right)
+	}
+	if math.IsNaN(right) {
+		return true
+	}
+	return left != right
+}
+
+// Gt returns true of left > right
+func Gt(left, right float64) bool {
+	return left > right
+}
+
+// Lt returns true if left < right
+func Lt(left, right float64) bool {
+	return left < right
+}
+
+// Gte returns true if left >= right
+func Gte(left, right float64) bool {
+	return left >= right
+}
+
+// Lte returns true if left <= right
+func Lte(left, right float64) bool {
+	return left <= right
+}
+
+// Plus returns left + right
+func Plus(left, right float64) float64 {
+	return left + right
+}
+
+// Minus returns left - right
+func Minus(left, right float64) float64 {
+	return left - right
+}
+
+// Mul returns left * right
+func Mul(left, right float64) float64 {
+	return left * right
+}
+
+// Div returns left / right
+func Div(left, right float64) float64 {
+	return left / right
+}
+
+// Mod returns mod(left, right)
+func Mod(left, right float64) float64 {
+	return math.Mod(left, right)
+}
+
+// Pow returns pow(left, right)
+func Pow(left, right float64) float64 {
+	return math.Pow(left, right)
+}
+
+// Default returns left or right if left is NaN.
+func Default(left, right float64) float64 {
+	if math.IsNaN(left) {
+		return right
+	}
+	return left
+}
+
+// If returns left if right is not NaN. Otherwise NaN is returned.
+func If(left, right float64) float64 {
+	if math.IsNaN(right) {
+		return nan
+	}
+	return left
+}
+
+// Ifnot returns left if right is NaN. Otherwise NaN is returned.
+func Ifnot(left, right float64) float64 {
+	if math.IsNaN(right) {
+		return left
+	}
+	return nan
+}
--- a/lib/metricsql/doc.go
+++ b/lib/metricsql/doc.go
@@ -0,0 +1,15 @@
+// Package metricsql implements MetricsQL parser.
+//
+// This parser can parse PromQL. Additionally it can parse all the MetricsQL extensions.
+// See https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/ExtendedPromQL for details about MetricsQL extensions.
+//
+// Usage:
+//
+//    expr, err := metricsql.Parse(`sum(rate(foo{bar="baz"}[5m])) by (job)`)
+//    if err != nil {
+//        // parse error
+//    }
+//    // Now expr contains parsed MetricsQL as `*Expr` structs.
+//    // See Parse examples for more details.
+//
+package metricsql
--- a/app/vmselect/promql/lexer.go
+++ b/app/vmselect/promql/lexer.go
@@ -1,11 +1,9 @@
-package promql
+package metricsql

 import (
 	"fmt"
 	"strconv"
 	"strings"
-
-	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
 )

 type lexer struct {
@@ -222,7 +220,7 @@ func scanIdent(s string) string {
 		}
 	}
 	if i == 0 {
-		logger.Panicf("BUG: scanIdent couldn't find a single ident char; make sure isIdentPrefix called before scanIdent")
+		panic("BUG: scanIdent couldn't find a single ident char; make sure isIdentPrefix called before scanIdent")
 	}
 	return s[:i]
 }
@@ -281,7 +279,7 @@ func toHex(n byte) byte {
 	return 'a' + (n - 10)
 }

-func appendEscapedIdent(dst, s []byte) []byte {
+func appendEscapedIdent(dst []byte, s string) []byte {
 	for i := 0; i < len(s); i++ {
 		ch := s[i]
 		if isIdentChar(ch) {
--- a/app/vmselect/promql/lexer_test.go
+++ b/app/vmselect/promql/lexer_test.go
@@ -1,4 +1,4 @@
-package promql
+package metricsql

 import (
 	"reflect"
@@ -28,7 +28,7 @@ func TestUnescapeIdent(t *testing.T) {
 func TestAppendEscapedIdent(t *testing.T) {
 	f := func(s, resultExpected string) {
 		t.Helper()
-		result := appendEscapedIdent(nil, []byte(s))
+		result := appendEscapedIdent(nil, s)
 		if string(result) != resultExpected {
 			t.Fatalf("unexpected result for appendEscapedIdent(%q); got %q; want %q", s, result, resultExpected)
 		}
--- a/lib/metricsql/parser.go
+++ b/lib/metricsql/parser.go
--- a/lib/metricsql/parser_example_test.go
+++ b/lib/metricsql/parser_example_test.go
@@ -0,0 +1,35 @@
+package metricsql_test
+
+import (
+	"fmt"
+	"log"
+
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql"
+)
+
+func ExampleParse() {
+	expr, err := metricsql.Parse(`sum(rate(foo{bar="baz"}[5m])) by (x,y)`)
+	if err != nil {
+		log.Fatalf("parse error: %s", err)
+	}
+	fmt.Printf("parsed expr: %s\n", expr.AppendString(nil))
+
+	ae := expr.(*metricsql.AggrFuncExpr)
+	fmt.Printf("aggr func: name=%s, arg=%s, modifier=%s\n", ae.Name, ae.Args[0].AppendString(nil), ae.Modifier.AppendString(nil))
+
+	fe := ae.Args[0].(*metricsql.FuncExpr)
+	fmt.Printf("func: name=%s, arg=%s\n", fe.Name, fe.Args[0].AppendString(nil))
+
+	re := fe.Args[0].(*metricsql.RollupExpr)
+	fmt.Printf("rollup: expr=%s, window=%s\n", re.Expr.AppendString(nil), re.Window)
+
+	me := re.Expr.(*metricsql.MetricExpr)
+	fmt.Printf("metric: labelFilter1=%s, labelFilter2=%s", me.LabelFilters[0].AppendString(nil), me.LabelFilters[1].AppendString(nil))
+
+	// Output:
+	// parsed expr: sum(rate(foo{bar="baz"}[5m])) by (x, y)
+	// aggr func: name=sum, arg=rate(foo{bar="baz"}[5m]), modifier=by (x, y)
+	// func: name=rate, arg=foo{bar="baz"}[5m]
+	// rollup: expr=foo{bar="baz"}, window=5m
+	// metric: labelFilter1=__name__="foo", labelFilter2=bar="baz"
+}
--- a/lib/metricsql/parser_test.go
+++ b/lib/metricsql/parser_test.go
@@ -0,0 +1,679 @@
+package metricsql
+
+import (
+	"testing"
+)
+
+func TestParseSuccess(t *testing.T) {
+	another := func(s string, sExpected string) {
+		t.Helper()
+
+		e, err := Parse(s)
+		if err != nil {
+			t.Fatalf("unexpected error when parsing %q: %s", s, err)
+		}
+		res := e.AppendString(nil)
+		if string(res) != sExpected {
+			t.Fatalf("unexpected string constructed;\ngot\n%q\nwant\n%q", res, sExpected)
+		}
+	}
+	same := func(s string) {
+		t.Helper()
+		another(s, s)
+	}
+
+	// metricExpr
+	same(`{}`)
+	same(`{}[5m]`)
+	same(`{}[5m:]`)
+	same(`{}[:]`)
+	another(`{}[: ]`, `{}[:]`)
+	same(`{}[:3s]`)
+	another(`{}[: 3s ]`, `{}[:3s]`)
+	same(`{}[5m:3s]`)
+	another(`{}[ 5m : 3s ]`, `{}[5m:3s]`)
+	same(`{} offset 5m`)
+	same(`{} offset -5m`)
+	same(`{}[5m] offset 10y`)
+	same(`{}[5.3m:3.4s] offset 10y`)
+	same(`{}[:3.4s] offset 10y`)
+	same(`{}[:3.4s] offset -10y`)
+	same(`{Foo="bAR"}`)
+	same(`{foo="bar"}`)
+	same(`{foo="bar"}[5m]`)
+	same(`{foo="bar"}[5m:]`)
+	same(`{foo="bar"}[5m:3s]`)
+	same(`{foo="bar"} offset 10y`)
+	same(`{foo="bar"} offset -10y`)
+	same(`{foo="bar"}[5m] offset 10y`)
+	same(`{foo="bar"}[5m:3s] offset 10y`)
+	another(`{foo="bar"}[5m] oFFSEt 10y`, `{foo="bar"}[5m] offset 10y`)
+	same("METRIC")
+	same("metric")
+	same("m_e:tri44:_c123")
+	another("-metric", "0 - metric")
+	same(`metric offset 10h`)
+	same("metric[5m]")
+	same("metric[5m:3s]")
+	same("metric[5m] offset 10h")
+	same("metric[5m:3s] offset 10h")
+	same("metric[5i:3i] offset 10i")
+	same(`metric{foo="bar"}`)
+	same(`metric{foo="bar"} offset 10h`)
+	same(`metric{foo!="bar"}[2d]`)
+	same(`metric{foo="bar"}[2d] offset 10h`)
+	same(`metric{foo="bar", b="sdfsdf"}[2d:3h] offset 10h`)
+	another(`  metric  {  foo  = "bar"  }  [  2d ]   offset   10h  `, `metric{foo="bar"}[2d] offset 10h`)
+	// metric name matching keywords
+	same("rate")
+	same("RATE")
+	same("by")
+	same("BY")
+	same("bool")
+	same("BOOL")
+	same("unless")
+	same("UNLESS")
+	same("Ignoring")
+	same("with")
+	same("WITH")
+	same("With")
+	same("alias")
+	same(`alias{foo="bar"}`)
+	same(`aLIas{alias="aa"}`)
+	another(`al\ias`, `alias`)
+	// identifiers with with escape chars
+	same(`foo\ bar`)
+	same(`foo\-bar\{{baz\+bar="aa"}`)
+	another(`\x2E\x2ef\oo{b\xEF\ar="aa"}`, `\x2e.foo{b\xefar="aa"}`)
+	// Duplicate filters
+	same(`foo{__name__="bar"}`)
+	same(`foo{a="b", a="c", __name__="aaa", b="d"}`)
+	// Metric filters ending with comma
+	another(`m{foo="bar",}`, `m{foo="bar"}`)
+	// String concat in tag value
+	another(`m{foo="bar" + "baz"}`, `m{foo="barbaz"}`)
+
+	// Valid regexp
+	same(`foo{bar=~"x"}`)
+	same(`foo{bar=~"^x"}`)
+	same(`foo{bar=~"^x$"}`)
+	same(`foo{bar=~"^(a[bc]|d)$"}`)
+	same(`foo{bar!~"x"}`)
+	same(`foo{bar!~"^x"}`)
+	same(`foo{bar!~"^x$"}`)
+	same(`foo{bar!~"^(a[bc]|d)$"}`)
+
+	// stringExpr
+	same(`""`)
+	same(`"\n\t\r 12:{}[]()44"`)
+	another(`''`, `""`)
+	another("``", `""`)
+	another("   `foo\"b'ar`  ", "\"foo\\\"b'ar\"")
+	another(`  'foo\'bar"BAZ'  `, `"foo'bar\"BAZ"`)
+	// string concat
+	another(`"foo"+'bar'`, `"foobar"`)
+
+	// numberExpr
+	same(`1`)
+	same(`1.23`)
+	same(`0.23`)
+	same(`1.2e+45`)
+	same(`1.2e-45`)
+	same(`-1`)
+	same(`-1.23`)
+	same(`-0.23`)
+	same(`-1.2e+45`)
+	same(`-1.2e-45`)
+	same(`-1.2e-45`)
+	another(`12.5E34`, `1.25e+35`)
+	another(`-.2`, `-0.2`)
+	another(`-.2E-2`, `-0.002`)
+	same(`NaN`)
+	another(`nan`, `NaN`)
+	another(`NAN`, `NaN`)
+	another(`nAN`, `NaN`)
+	another(`Inf`, `+Inf`)
+	another(`INF`, `+Inf`)
+	another(`inf`, `+Inf`)
+	another(`+Inf`, `+Inf`)
+	another(`-Inf`, `-Inf`)
+	another(`-inF`, `-Inf`)
+
+	// binaryOpExpr
+	another(`nan == nan`, `NaN`)
+	another(`nan ==bool nan`, `1`)
+	another(`nan !=bool nan`, `0`)
+	another(`nan !=bool 2`, `1`)
+	another(`2 !=bool nan`, `1`)
+	another(`nan >bool nan`, `0`)
+	another(`nan <bool nan`, `0`)
+	another(`1 ==bool nan`, `0`)
+	another(`NaN !=bool 1`, `1`)
+	another(`inf >=bool 2`, `1`)
+	another(`-1 >bool -inf`, `1`)
+	another(`-1 <bool -inf`, `0`)
+	another(`nan + 2 *3 * inf`, `NaN`)
+	another(`INF - Inf`, `NaN`)
+	another(`Inf + inf`, `+Inf`)
+	another(`1/0`, `+Inf`)
+	another(`0/0`, `NaN`)
+	another(`-m`, `0 - m`)
+	same(`m + ignoring () n[5m]`)
+	another(`M + IGNORING () N[5m]`, `M + ignoring () N[5m]`)
+	same(`m + on (foo) n[5m]`)
+	another(`m + ON (Foo) n[5m]`, `m + on (Foo) n[5m]`)
+	same(`m + ignoring (a, b) n[5m]`)
+	another(`1 or 2`, `1`)
+	another(`1 and 2`, `1`)
+	another(`1 unless 2`, `NaN`)
+	another(`1 default 2`, `1`)
+	another(`1 default NaN`, `1`)
+	another(`NaN default 2`, `2`)
+	another(`1 > 2`, `NaN`)
+	another(`1 > bool 2`, `0`)
+	another(`3 >= 2`, `3`)
+	another(`3 <= bool 2`, `0`)
+	another(`1 + -2 - 3`, `-4`)
+	another(`1 / 0 + 2`, `+Inf`)
+	another(`2 + -1 / 0`, `-Inf`)
+	another(`-1 ^ 0.5`, `NaN`)
+	another(`512.5 - (1 + 3) * (2 ^ 2) ^ 3`, `256.5`)
+	another(`1 == bool 1 != bool 24 < bool 4 > bool -1`, `1`)
+	another(`1 == bOOl 1 != BOOL 24 < Bool 4 > booL -1`, `1`)
+	another(`m1+on(foo)group_left m2`, `m1 + on (foo) group_left () m2`)
+	another(`M1+ON(FOO)GROUP_left M2`, `M1 + on (FOO) group_left () M2`)
+	same(`m1 + on (foo) group_right () m2`)
+	same(`m1 + on (foo, bar) group_right (x, y) m2`)
+	another(`m1 + on (foo, bar,) group_right (x, y,) m2`, `m1 + on (foo, bar) group_right (x, y) m2`)
+	same(`m1 == bool on (foo, bar) group_right (x, y) m2`)
+	another(`5 - 1 + 3 * 2 ^ 2 ^ 3 - 2  OR Metric {Bar= "Baz", aaa!="bb",cc=~"dd" ,zz !~"ff" } `,
+		`770 or Metric{Bar="Baz", aaa!="bb", cc=~"dd", zz!~"ff"}`)
+	same(`"foo" + bar()`)
+	same(`"foo" + bar{x="y"}`)
+	same(`("foo"[3s] + bar{x="y"})[5m:3s] offset 10s`)
+	same(`("foo"[3s] + bar{x="y"})[5i:3i] offset 10i`)
+	same(`bar + "foo" offset 3s`)
+	same(`bar + "foo" offset 3i`)
+	another(`1+2 if 2>3`, `NaN`)
+	another(`1+4 if 2<3`, `5`)
+	another(`2+6 default 3 if 2>3`, `8`)
+	another(`2+6 if 2>3 default NaN`, `NaN`)
+	another(`42 if 3>2 if 2+2<5`, `42`)
+	another(`42 if 3>2 if 2+2>=5`, `NaN`)
+	another(`1+2 ifnot 2>3`, `3`)
+	another(`1+4 ifnot 2<3`, `NaN`)
+	another(`2+6 default 3 ifnot 2>3`, `8`)
+	another(`2+6 ifnot 2>3 default NaN`, `8`)
+	another(`42 if 3>2 ifnot 2+2<5`, `NaN`)
+	another(`42 if 3>2 ifnot 2+2>=5`, `42`)
+
+	// parensExpr
+	another(`(-foo + ((bar) / (baz))) + ((23))`, `((0 - foo) + (bar / baz)) + 23`)
+	another(`(FOO + ((Bar) / (baZ))) + ((23))`, `(FOO + (Bar / baZ)) + 23`)
+	same(`(foo, bar)`)
+	another(`((foo, bar),(baz))`, `((foo, bar), baz)`)
+	same(`(foo, (bar, baz), ((x, y), (z, y), xx))`)
+	another(`1+(foo, bar,)`, `1 + (foo, bar)`)
+	another(`((foo(bar,baz)), (1+(2)+(3,4)+()))`, `(foo(bar, baz), (3 + (3, 4)) + ())`)
+	same(`()`)
+
+	// funcExpr
+	same(`f()`)
+	another(`f(x,)`, `f(x)`)
+	another(`-f()-Ff()`, `(0 - f()) - Ff()`)
+	same(`F()`)
+	another(`+F()`, `F()`)
+	another(`++F()`, `F()`)
+	another(`--F()`, `0 - (0 - F())`)
+	same(`f(http_server_request)`)
+	same(`f(http_server_request)[4s:5m] offset 10m`)
+	same(`f(http_server_request)[4i:5i] offset 10i`)
+	same(`F(HttpServerRequest)`)
+	same(`f(job, foo)`)
+	same(`F(Job, Foo)`)
+	another(` FOO (bar) + f  (  m  (  ),ff(1 + (  2.5)) ,M[5m ]  , "ff"  )`, `FOO(bar) + f(m(), ff(3.5), M[5m], "ff")`)
+	// funcName matching keywords
+	same(`by(2)`)
+	same(`BY(2)`)
+	same(`or(2)`)
+	same(`OR(2)`)
+	same(`bool(2)`)
+	same(`BOOL(2)`)
+	same(`rate(rate(m))`)
+	same(`rate(rate(m[5m]))`)
+	same(`rate(rate(m[5m])[1h:])`)
+	same(`rate(rate(m[5m])[1h:3s])`)
+	// funcName with escape chars
+	same(`foo\(ba\-r()`)
+
+	// aggrFuncExpr
+	same(`sum(http_server_request) by ()`)
+	same(`sum(http_server_request) by (job)`)
+	same(`sum(http_server_request) without (job, foo)`)
+	another(`sum(x,y,) without (a,b,)`, `sum(x, y) without (a, b)`)
+	another(`sum by () (xx)`, `sum(xx) by ()`)
+	another(`sum by (s) (xx)[5s]`, `(sum(xx) by (s))[5s]`)
+	another(`SUM BY (ZZ, aa) (XX)`, `sum(XX) by (ZZ, aa)`)
+	another(`sum without (a, b) (xx,2+2)`, `sum(xx, 4) without (a, b)`)
+	another(`Sum WIthout (a, B) (XX,2+2)`, `sum(XX, 4) without (a, B)`)
+	same(`sum(a) or sum(b)`)
+	same(`sum(a) by () or sum(b) without (x, y)`)
+	same(`sum(a) + sum(b)`)
+	same(`sum(x) * (1 + sum(a))`)
+
+	// All the above
+	another(`Sum(Ff(M) * M{X=""}[5m] Offset 7m - 123, 35) BY (X, y) * F2("Test")`,
+		`sum((Ff(M) * M{X=""}[5m] offset 7m) - 123, 35) by (X, y) * F2("Test")`)
+	another(`# comment
+		Sum(Ff(M) * M{X=""}[5m] Offset 7m - 123, 35) BY (X, y) # yet another comment
+		* F2("Test")`,
+		`sum((Ff(M) * M{X=""}[5m] offset 7m) - 123, 35) by (X, y) * F2("Test")`)
+
+	// withExpr
+	another(`with () x`, `x`)
+	another(`with (x=1,) x`, `1`)
+	another(`with (x = m offset 5h) x + x`, `m offset 5h + m offset 5h`)
+	another(`with (x = m offset 5i) x + x`, `m offset 5i + m offset 5i`)
+	another(`with (foo = bar{x="x"}) 1`, `1`)
+	another(`with (foo = bar{x="x"}) "x"`, `"x"`)
+	another(`with (f="x") f`, `"x"`)
+	another(`with (foo = bar{x="x"}) x{x="y"}`, `x{x="y"}`)
+	another(`with (foo = bar{x="x"}) 1+1`, `2`)
+	another(`with (foo = bar{x="x"}) f()`, `f()`)
+	another(`with (foo = bar{x="x"}) sum(x)`, `sum(x)`)
+	another(`with (foo = bar{x="x"}) baz{foo="bar"}`, `baz{foo="bar"}`)
+	another(`with (foo = bar) baz`, `baz`)
+	another(`with (foo = bar) foo + foo{a="b"}`, `bar + bar{a="b"}`)
+	another(`with (foo = bar, bar=baz + f()) test`, `test`)
+	another(`with (ct={job="test"}) a{ct} + ct() + f({ct="x"})`, `(a{job="test"} + {job="test"}) + f({ct="x"})`)
+	another(`with (ct={job="test", i="bar"}) ct + {ct, x="d"} + foo{ct, ct} + ctx(1)`,
+		`(({job="test", i="bar"} + {job="test", i="bar", x="d"}) + foo{job="test", i="bar"}) + ctx(1)`)
+	another(`with (foo = bar) {__name__=~"foo"}`, `{__name__=~"foo"}`)
+	another(`with (foo = bar) foo{__name__="foo"}`, `bar`)
+	another(`with (foo = bar) {__name__="foo", x="y"}`, `bar{x="y"}`)
+	another(`with (foo(bar) = {__name__!="bar"}) foo(x)`, `{__name__!="bar"}`)
+	another(`with (foo(bar) = bar{__name__="bar"}) foo(x)`, `x`)
+	another(`with (foo\-bar(baz) = baz + baz) foo\-bar((x,y))`, `(x, y) + (x, y)`)
+	another(`with (foo\-bar(baz) = baz + baz) foo\-bar(x*y)`, `(x * y) + (x * y)`)
+	another(`with (foo\-bar(baz) = baz + baz) foo\-bar(x\*y)`, `x\*y + x\*y`)
+	another(`with (foo\-bar(b\ az) = b\ az + b\ az) foo\-bar(x\*y)`, `x\*y + x\*y`)
+	// override ttf to something new.
+	another(`with (ttf = a) ttf + b`, `a + b`)
+	// override ttf to ru
+	another(`with (ttf = ru(m, n)) ttf`, `(clamp_min(n - clamp_min(m, 0), 0) / clamp_min(n, 0)) * 100`)
+
+	// Verify withExpr recursion and forward reference
+	another(`with (x = x+y, y = x+x) y ^ 2`, `((x + y) + (x + y)) ^ 2`)
+	another(`with (f1(x)=f2(x), f2(x)=f1(x)^2) f1(foobar)`, `f2(foobar)`)
+	another(`with (f1(x)=f2(x), f2(x)=f1(x)^2) f2(foobar)`, `f2(foobar) ^ 2`)
+
+	// Verify withExpr funcs
+	another(`with (x() = y+1) x`, `y + 1`)
+	another(`with (x(foo) = foo+1) x(a)`, `a + 1`)
+	another(`with (x(a, b) = a + b) x(foo, bar)`, `foo + bar`)
+	another(`with (x(a, b) = a + b) x(foo, x(1, 2))`, `foo + 3`)
+	another(`with (x(a) = sum(a) by (b)) x(xx) / x(y)`, `sum(xx) by (b) / sum(y) by (b)`)
+	another(`with (f(a,f,x)=ff(x,f,a)) f(f(x,y,z),1,2)`, `ff(2, 1, ff(z, y, x))`)
+	another(`with (f(x)=1+f(x)) f(foo{bar="baz"})`, `1 + f(foo{bar="baz"})`)
+	another(`with (a=foo, y=bar, f(a)= a+a+y) f(x)`, `(x + x) + bar`)
+	another(`with (f(a, b) = m{a, b}) f({a="x", b="y"}, {c="d"})`, `m{a="x", b="y", c="d"}`)
+	another(`with (xx={a="x"}, f(a, b) = m{a, b}) f({xx, b="y"}, {c="d"})`, `m{a="x", b="y", c="d"}`)
+	another(`with (x() = {b="c"}) foo{x}`, `foo{b="c"}`)
+	another(`with (f(x)=x{foo="bar"} offset 5m) f(m offset 10m)`, `(m{foo="bar"} offset 10m) offset 5m`)
+	another(`with (f(x)=x{foo="bar",bas="a"}[5m]) f(m[10m] offset 3s)`, `(m{foo="bar", bas="a"}[10m] offset 3s)[5m]`)
+	another(`with (f(x)=x{foo="bar"}[5m] offset 10m) f(m{x="y"})`, `m{x="y", foo="bar"}[5m] offset 10m`)
+	another(`with (f(x)=x{foo="bar"}[5m] offset 10m) f({x="y", foo="bar", foo="bar"})`, `{x="y", foo="bar"}[5m] offset 10m`)
+	another(`with (f(m, x)=m{x}[5m] offset 10m) f(foo, {})`, `foo[5m] offset 10m`)
+	another(`with (f(m, x)=m{x, bar="baz"}[5m] offset 10m) f(foo, {})`, `foo{bar="baz"}[5m] offset 10m`)
+	another(`with (f(x)=x[5m] offset 3s) f(foo[3m]+bar)`, `(foo[3m] + bar)[5m] offset 3s`)
+	another(`with (f(x)=x[5m:3s] oFFsEt 1.5m) f(sum(s) by (a,b))`, `(sum(s) by (a, b))[5m:3s] offset 1.5m`)
+	another(`with (x="a", y=x) y+"bc"`, `"abc"`)
+	another(`with (x="a", y="b"+x) "we"+y+"z"+f()`, `"webaz" + f()`)
+	another(`with (f(x) = m{foo=x+"y", bar="y"+x, baz=x} + x) f("qwe")`, `m{foo="qwey", bar="yqwe", baz="qwe"} + "qwe"`)
+	another(`with (f(a)=a) f`, `f`)
+	another(`with (f\q(a)=a) f\q`, `fq`)
+
+	// Verify withExpr for aggr func modifiers
+	another(`with (f(x) = x, y = sum(m) by (f)) y`, `sum(m) by (f)`)
+	another(`with (f(x) = sum(m) by (x)) f(foo)`, `sum(m) by (foo)`)
+	another(`with (f(x) = sum(m) by (x)) f((foo, bar, foo))`, `sum(m) by (foo, bar)`)
+	another(`with (f(x) = sum(m) without (x,y)) f((a, b))`, `sum(m) without (a, b, y)`)
+	another(`with (f(x) = sum(m) without (y,x)) f((a, y))`, `sum(m) without (y, a)`)
+	another(`with (f(x,y) = a + on (x,y) group_left (y,bar) b) f(foo,())`, `a + on (foo) group_left (bar) b`)
+	another(`with (f(x,y) = a + on (x,y) group_left (y,bar) b) f((foo),())`, `a + on (foo) group_left (bar) b`)
+	another(`with (f(x,y) = a + on (x,y) group_left (y,bar) b) f((foo,xx),())`, `a + on (foo, xx) group_left (bar) b`)
+
+	// Verify nested with exprs
+	another(`with (f(x) = (with(x=y) x) + x) f(z)`, `y + z`)
+	another(`with (x=foo) f(a, with (y=x) y)`, `f(a, foo)`)
+	another(`with (x=foo) a * x + (with (y=x) y) / y`, `(a * foo) + (foo / y)`)
+	another(`with (x = with (y = foo) y + x) x/x`, `(foo + x) / (foo + x)`)
+	another(`with (
+		x = {foo="bar"},
+		q = m{x, y="1"},
+		f(x) =
+			with (
+				z(y) = x + y * q
+			)
+			z(foo) / f(x)
+	)
+	f(a)`, `(a + (foo * m{foo="bar", y="1"})) / f(a)`)
+
+	// complex withExpr
+	another(`WITH (
+		treshold = (0.9),
+		commonFilters = {job="cacher", instance=~"1.2.3.4"},
+		hits = rate(cache{type="hit", commonFilters}[5m]),
+		miss = rate(cache{type="miss", commonFilters}[5m]),
+		sumByInstance(arg) = sum(arg) by (instance),
+		hitRatio = sumByInstance(hits) / sumByInstance(hits + miss)
+	)
+	hitRatio < treshold`,
+		`(sum(rate(cache{type="hit", job="cacher", instance=~"1.2.3.4"}[5m])) by (instance) / sum(rate(cache{type="hit", job="cacher", instance=~"1.2.3.4"}[5m]) + rate(cache{type="miss", job="cacher", instance=~"1.2.3.4"}[5m])) by (instance)) < 0.9`)
+	another(`WITH (
+		x2(x) = x^2,
+		f(x, y) = x2(x) + x*y + x2(y)
+	)
+	f(a, 3)
+	`, `((a ^ 2) + (a * 3)) + 9`)
+	another(`WITH (
+		x2(x) = x^2,
+		f(x, y) = x2(x) + x*y + x2(y)
+	)
+	f(2, 3)
+	`, `19`)
+	another(`WITH (
+		commonFilters = {instance="foo"},
+		timeToFuckup(currv, maxv) = (maxv - currv) / rate(currv)
+	)
+	timeToFuckup(diskUsage{commonFilters}, maxDiskSize{commonFilters})`,
+		`(maxDiskSize{instance="foo"} - diskUsage{instance="foo"}) / rate(diskUsage{instance="foo"})`)
+	another(`WITH (
+	       commonFilters = {job="foo", instance="bar"},
+	       sumRate(m, cf) = sum(rate(m{cf})) by (job, instance),
+	       hitRate(hits, misses) = sumRate(hits, commonFilters) / (sumRate(hits, commonFilters) + sumRate(misses, commonFilters))
+	   )
+	   hitRate(cacheHits, cacheMisses)`,
+		`sum(rate(cacheHits{job="foo", instance="bar"})) by (job, instance) / (sum(rate(cacheHits{job="foo", instance="bar"})) by (job, instance) + sum(rate(cacheMisses{job="foo", instance="bar"})) by (job, instance))`)
+	another(`with(y=123,z=5) union(with(y=3,f(x)=x*y) f(2) + f(3), with(x=5,y=2) x*y*z)`, `union(15, 50)`)
+}
+
+func TestParseError(t *testing.T) {
+	f := func(s string) {
+		t.Helper()
+
+		e, err := Parse(s)
+		if err == nil {
+			t.Fatalf("expecting non-nil error when parsing %q", s)
+		}
+		if e != nil {
+			t.Fatalf("expecting nil expr when parsing %q", s)
+		}
+	}
+
+	// an empty string
+	f("")
+	f("  \t\b\r\n  ")
+
+	// invalid metricExpr
+	f(`{__name__="ff"} offset 55`)
+	f(`foo[55]`)
+	f(`m[-5m]`)
+	f(`{`)
+	f(`foo{`)
+	f(`foo{bar`)
+	f(`foo{bar=`)
+	f(`foo{bar="baz"`)
+	f(`foo{bar="baz",  `)
+	f(`foo{123="23"}`)
+	f(`foo{foo}`)
+	f(`foo{,}`)
+	f(`foo{,foo="bar"}`)
+	f(`foo{foo=}`)
+	f(`foo{foo="ba}`)
+	f(`foo{"foo"="bar"}`)
+	f(`foo{$`)
+	f(`foo{a $`)
+	f(`foo{a="b",$`)
+	f(`foo{a="b"}$`)
+	f(`[`)
+	f(`[]`)
+	f(`f[5m]$`)
+	f(`[5m]`)
+	f(`[5m] offset 4h`)
+	f(`m[5m] offset $`)
+	f(`m[5m] offset 5h $`)
+	f(`m[]`)
+	f(`m[-5m]`)
+	f(`m[5m:`)
+	f(`m[5m:-`)
+	f(`m[5m:-1`)
+	f(`m[5m:-1]`)
+	f(`m[5m:-1s]`)
+	f(`m[-5m:1s]`)
+	f(`m[-5m:-1s]`)
+	f(`m[:`)
+	f(`m[:-`)
+	f(`m[:1]`)
+	f(`m[:-1m]`)
+	f(`m[5]`)
+	f(`m[[5m]]`)
+	f(`m[foo]`)
+	f(`m["ff"]`)
+	f(`m[10m`)
+	f(`m[123`)
+	f(`m["ff`)
+	f(`m[(f`)
+	f(`fd}`)
+	f(`]`)
+	f(`m $`)
+	f(`m{,}`)
+	f(`m{x=y}`)
+	f(`m{x=y/5}`)
+	f(`m{x=y+5}`)
+
+	// Invalid regexp
+	f(`foo{bar=~"x["}`)
+	f(`foo{bar=~"x("}`)
+	f(`foo{bar=~"x)"}`)
+	f(`foo{bar!~"x["}`)
+	f(`foo{bar!~"x("}`)
+	f(`foo{bar!~"x)"}`)
+
+	// invalid stringExpr
+	f(`'`)
+	f(`"`)
+	f("`")
+	f(`"foo`)
+	f(`'foo`)
+	f("`foo")
+	f(`"foo\"bar`)
+	f(`'foo\'bar`)
+	f("`foo\\`bar")
+	f(`"" $`)
+	f(`"foo" +`)
+	f(`n{"foo" + m`)
+
+	// invalid numberExpr
+	f(`12.`)
+	f(`1.2e`)
+	f(`23e-`)
+	f(`23E+`)
+	f(`.`)
+	f(`-12.`)
+	f(`-1.2e`)
+	f(`-23e-`)
+	f(`-23E+`)
+	f(`-.`)
+	f(`-1$$`)
+	f(`-$$`)
+	f(`+$$`)
+	f(`23 $$`)
+
+	// invalid binaryOpExpr
+	f(`+`)
+	f(`1 +`)
+	f(`1 + 2.`)
+	f(`3 unless`)
+	f(`23 + on (foo)`)
+	f(`m + on (,) m`)
+	f(`3 * ignoring`)
+	f(`m * on (`)
+	f(`m * on (foo`)
+	f(`m * on (foo,`)
+	f(`m * on (foo,)`)
+	f(`m * on (,foo)`)
+	f(`m * on (,)`)
+	f(`m == bool (bar) baz`)
+	f(`m == bool () baz`)
+	f(`m * by (baz) n`)
+	f(`m + bool group_left m2`)
+	f(`m + on () group_left (`)
+	f(`m + on () group_left (,`)
+	f(`m + on () group_left (,foo`)
+	f(`m + on () group_left (foo,)`)
+	f(`m + on () group_left (,foo)`)
+	f(`m + on () group_left (foo)`)
+	f(`m + on () group_right (foo) (m`)
+	f(`m or ignoring () group_left () n`)
+	f(`1 + bool 2`)
+	f(`m % bool n`)
+	f(`m * bool baz`)
+	f(`M * BOoL BaZ`)
+	f(`foo unless ignoring (bar) group_left xxx`)
+	f(`foo or bool bar`)
+	f(`foo == bool $$`)
+	f(`"foo" + bar`)
+
+	// invalid parensExpr
+	f(`(`)
+	f(`($`)
+	f(`(+`)
+	f(`(1`)
+	f(`(m+`)
+	f(`1)`)
+	f(`(,)`)
+	f(`(1)$`)
+
+	// invalid funcExpr
+	f(`f $`)
+	f(`f($)`)
+	f(`f[`)
+	f(`f()$`)
+	f(`f(`)
+	f(`f(foo`)
+	f(`f(f,`)
+	f(`f(,`)
+	f(`f(,)`)
+	f(`f(,foo)`)
+	f(`f(,foo`)
+	f(`f(foo,$`)
+	f(`f() by (a)`)
+	f(`f without (x) (y)`)
+	f(`f() foo (a)`)
+	f(`f bar (x) (b)`)
+	f(`f bar (x)`)
+
+	// invalid aggrFuncExpr
+	f(`sum(`)
+	f(`sum $`)
+	f(`sum [`)
+	f(`sum($)`)
+	f(`sum()$`)
+	f(`sum(foo) ba`)
+	f(`sum(foo) ba()`)
+	f(`sum(foo) by`)
+	f(`sum(foo) without x`)
+	f(`sum(foo) aaa`)
+	f(`sum(foo) aaa x`)
+	f(`sum() by $`)
+	f(`sum() by (`)
+	f(`sum() by ($`)
+	f(`sum() by (a`)
+	f(`sum() by (a $`)
+	f(`sum() by (a ]`)
+	f(`sum() by (a)$`)
+	f(`sum() by (,`)
+	f(`sum() by (a,$`)
+	f(`sum() by (,)`)
+	f(`sum() by (,a`)
+	f(`sum() by (,a)`)
+	f(`sum() on (b)`)
+	f(`sum() bool`)
+	f(`sum() group_left`)
+	f(`sum() group_right(x)`)
+	f(`sum ba`)
+	f(`sum ba ()`)
+	f(`sum by (`)
+	f(`sum by (a`)
+	f(`sum by (,`)
+	f(`sum by (,)`)
+	f(`sum by (,a`)
+	f(`sum by (,a)`)
+	f(`sum by (a)`)
+	f(`sum by (a) (`)
+	f(`sum by (a) [`)
+	f(`sum by (a) {`)
+	f(`sum by (a) (b`)
+	f(`sum by (a) (b,`)
+	f(`sum by (a) (,)`)
+	f(`avg by (a) (,b)`)
+	f(`sum by (x) (y) by (z)`)
+	f(`sum(m) by (1)`)
+
+	// invalid withExpr
+	f(`with $`)
+	f(`with a`)
+	f(`with a=b c`)
+	f(`with (`)
+	f(`with (x=b)$`)
+	f(`with ($`)
+	f(`with (foo`)
+	f(`with (foo $`)
+	f(`with (x y`)
+	f(`with (x =`)
+	f(`with (x = $`)
+	f(`with (x= y`)
+	f(`with (x= y $`)
+	f(`with (x= y)`)
+	f(`with (x=(`)
+	f(`with (x=[)`)
+	f(`with (x=() x)`)
+	f(`with ($$)`)
+	f(`with (x $$`)
+	f(`with (x = $$)`)
+	f(`with (x = foo) bar{x}`)
+	f(`with (x = {foo="bar"}[5m]) bar{x}`)
+	f(`with (x = {foo="bar"} offset 5m) bar{x}`)
+	f(`with (x = a, x = b) c`)
+	f(`with (x(a, a) = b) c`)
+	f(`with (x=m{f="x"}) foo{x}`)
+	f(`with (sum = x) y`)
+	f(`with (rate(a) = b) c`)
+	f(`with (clamp_min=x) y`)
+	f(`with (f()`)
+	f(`with (a=b c=d) e`)
+	f(`with (f(x)=x^2) m{x}`)
+	f(`with (f(x)=ff()) m{x}`)
+	f(`with (f(x`)
+	f(`with (x=m) a{x} + b`)
+	f(`with (x=m) b + a{x}`)
+	f(`with (x=m) f(b, a{x})`)
+	f(`with (x=m) sum(a{x})`)
+	f(`with (x=m) (a{x})`)
+	f(`with (f(a)=a) f(1, 2)`)
+	f(`with (f(x)=x{foo="bar"}) f(1)`)
+	f(`with (f(x)=x{foo="bar"}) f(m + n)`)
+	f(`with (f = with`)
+	f(`with (,)`)
+	f(`with (1) 2`)
+	f(`with (f(1)=2) 3`)
+	f(`with (f(,)=x) x`)
+	f(`with (x(a) = {b="c"}) foo{x}`)
+	f(`with (f(x) = m{foo=xx}) f("qwe")`)
+	f(`a + with(f(x)=x) f(1,2)`)
+	f(`with (f(x) = sum(m) by (x)) f({foo="bar"})`)
+	f(`with (f(x) = sum(m) by (x)) f((xx(), {foo="bar"}))`)
+	f(`with (f(x) = m + on (x) n) f(xx())`)
+	f(`with (f(x) = m + on (a) group_right (x) n) f(xx())`)
+}
--- a/app/vmselect/promql/regexp_cache.go
+++ b/app/vmselect/promql/regexp_cache.go
@@ -1,4 +1,4 @@
-package promql
+package metricsql

 import (
 	"regexp"
@@ -8,12 +8,14 @@ import (
 	"github.com/VictoriaMetrics/metrics"
 )

-func compileRegexpAnchored(re string) (*regexp.Regexp, error) {
+// CompileRegexpAnchored returns compiled regexp `^re$`.
+func CompileRegexpAnchored(re string) (*regexp.Regexp, error) {
 	reAnchored := "^(?:" + re + ")$"
-	return compileRegexp(reAnchored)
+	return CompileRegexp(reAnchored)
 }

-func compileRegexp(re string) (*regexp.Regexp, error) {
+// CompileRegexp returns compile regexp re.
+func CompileRegexp(re string) (*regexp.Regexp, error) {
 	rcv := regexpCacheV.Get(re)
 	if rcv != nil {
 		return rcv.r, rcv.err
--- a/lib/metricsql/rollup.go
+++ b/lib/metricsql/rollup.go
@@ -0,0 +1,63 @@
+package metricsql
+
+import (
+	"strings"
+)
+
+var rollupFuncs = map[string]bool{
+	// Standard rollup funcs from PromQL.
+	// See funcs accepting range-vector on https://prometheus.io/docs/prometheus/latest/querying/functions/ .
+	"changes":            true,
+	"delta":              true,
+	"deriv":              true,
+	"deriv_fast":         true,
+	"holt_winters":       true,
+	"idelta":             true,
+	"increase":           true,
+	"irate":              true,
+	"predict_linear":     true,
+	"rate":               true,
+	"resets":             true,
+	"avg_over_time":      true,
+	"min_over_time":      true,
+	"max_over_time":      true,
+	"sum_over_time":      true,
+	"count_over_time":    true,
+	"quantile_over_time": true,
+	"stddev_over_time":   true,
+	"stdvar_over_time":   true,
+	"absent_over_time":   true,
+
+	// Additional rollup funcs.
+	"default_rollup":      true,
+	"sum2_over_time":      true,
+	"geomean_over_time":   true,
+	"first_over_time":     true,
+	"last_over_time":      true,
+	"distinct_over_time":  true,
+	"increases_over_time": true,
+	"decreases_over_time": true,
+	"integrate":           true,
+	"ideriv":              true,
+	"lifetime":            true,
+	"lag":                 true,
+	"scrape_interval":     true,
+	"tmin_over_time":      true,
+	"tmax_over_time":      true,
+	"share_le_over_time":  true,
+	"share_gt_over_time":  true,
+	"histogram_over_time": true,
+	"rollup":              true,
+	"rollup_rate":         true,
+	"rollup_deriv":        true,
+	"rollup_delta":        true,
+	"rollup_increase":     true,
+	"rollup_candlestick":  true,
+	"aggr_over_time":      true,
+}
+
+// IsRollupFunc returns whether funcName is known rollup function.
+func IsRollupFunc(funcName string) bool {
+	s := strings.ToLower(funcName)
+	return rollupFuncs[s]
+}
--- a/lib/metricsql/transform.go
+++ b/lib/metricsql/transform.go
@@ -0,0 +1,83 @@
+package metricsql
+
+import (
+	"strings"
+)
+
+var transformFuncs = map[string]bool{
+	// Standard promql funcs
+	// See funcs accepting instant-vector on https://prometheus.io/docs/prometheus/latest/querying/functions/ .
+	"abs":                true,
+	"absent":             true,
+	"ceil":               true,
+	"clamp_max":          true,
+	"clamp_min":          true,
+	"day_of_month":       true,
+	"day_of_week":        true,
+	"days_in_month":      true,
+	"exp":                true,
+	"floor":              true,
+	"histogram_quantile": true,
+	"hour":               true,
+	"label_join":         true,
+	"label_replace":      true,
+	"ln":                 true,
+	"log2":               true,
+	"log10":              true,
+	"minute":             true,
+	"month":              true,
+	"round":              true,
+	"scalar":             true,
+	"sort":               true,
+	"sort_desc":          true,
+	"sqrt":               true,
+	"time":               true,
+	"timestamp":          true,
+	"vector":             true,
+	"year":               true,
+
+	// New funcs from MetricsQL
+	"label_set":          true,
+	"label_del":          true,
+	"label_keep":         true,
+	"label_copy":         true,
+	"label_move":         true,
+	"label_transform":    true,
+	"label_value":        true,
+	"union":              true,
+	"":                   true, // empty func is a synonim to union
+	"keep_last_value":    true,
+	"start":              true,
+	"end":                true,
+	"step":               true,
+	"running_sum":        true,
+	"running_max":        true,
+	"running_min":        true,
+	"running_avg":        true,
+	"range_sum":          true,
+	"range_max":          true,
+	"range_min":          true,
+	"range_avg":          true,
+	"range_first":        true,
+	"range_last":         true,
+	"range_quantile":     true,
+	"smooth_exponential": true,
+	"remove_resets":      true,
+	"rand":               true,
+	"rand_normal":        true,
+	"rand_exponential":   true,
+	"pi":                 true,
+	"sin":                true,
+	"cos":                true,
+	"asin":               true,
+	"acos":               true,
+	"prometheus_buckets": true,
+	"histogram_share":    true,
+}
+
+// IsTransformFunc returns whether funcName is known transform function.
+func IsTransformFunc(funcName string) bool {
+	s := strings.ToLower(funcName)
+	return transformFuncs[s]
+
+}
--- a/lib/metricsql/utils.go
+++ b/lib/metricsql/utils.go
@@ -0,0 +1,12 @@
+package metricsql
+
+// ExpandWithExprs expands WITH expressions inside q and returns the resulting
+// PromQL without WITH expressions.
+func ExpandWithExprs(q string) (string, error) {
+	e, err := Parse(q)
+	if err != nil {
+		return "", err
+	}
+	buf := e.AppendString(nil)
+	return string(buf), nil
+}
--- a/lib/metricsql/utils_example_test.go
+++ b/lib/metricsql/utils_example_test.go
@@ -0,0 +1,28 @@
+package metricsql_test
+
+import (
+	"fmt"
+	"log"
+
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/metricsql"
+)
+
+func ExampleExpandWithExprs() {
+	// mql can contain arbitrary MetricsQL extensions - see https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/ExtendedPromQL
+	mql := `WITH (
+		commonFilters = {job="$job", instance="$instance"},
+		f(a, b) = 100*(a/b),
+	)
+	f(disk_free_bytes{commonFilters}, disk_total_bytes{commonFilters})`
+
+	// Convert mql to PromQL
+	pql, err := metricsql.ExpandWithExprs(mql)
+	if err != nil {
+		log.Fatalf("cannot expand with expressions: %s", err)
+	}
+
+	fmt.Printf("%s\n", pql)
+
+	// Output:
+	// 100 * (disk_free_bytes{job="$job", instance="$instance"} / disk_total_bytes{job="$job", instance="$instance"})
+}
--- a/lib/metricsql/utils_test.go
+++ b/lib/metricsql/utils_test.go
@@ -0,0 +1,43 @@
+package metricsql
+
+import (
+	"testing"
+)
+
+func TestExpandWithExprsSuccess(t *testing.T) {
+	f := func(q, qExpected string) {
+		t.Helper()
+		for i := 0; i < 3; i++ {
+			qExpanded, err := ExpandWithExprs(q)
+			if err != nil {
+				t.Fatalf("unexpected error when expanding %q: %s", q, err)
+			}
+			if qExpanded != qExpected {
+				t.Fatalf("unexpected expanded expression for %q;\ngot\n%q\nwant\n%q", q, qExpanded, qExpected)
+			}
+		}
+	}
+
+	f(`1`, `1`)
+	f(`foobar`, `foobar`)
+	f(`with (x = 1) x+x`, `2`)
+	f(`with (f(x) = x*x) 3+f(2)+2`, `9`)
+}
+
+func TestExpandWithExprsError(t *testing.T) {
+	f := func(q string) {
+		t.Helper()
+		for i := 0; i < 3; i++ {
+			qExpanded, err := ExpandWithExprs(q)
+			if err == nil {
+				t.Fatalf("expecting non-nil error when expanding %q", q)
+			}
+			if qExpanded != "" {
+				t.Fatalf("unexpected non-empty qExpanded=%q", qExpanded)
+			}
+		}
+	}
+
+	f(``)
+	f(`  with (`)
+}
--- a/lib/storage/partition.go
+++ b/lib/storage/partition.go
@@ -61,6 +61,11 @@ const defaultPartsToMerge = 15
 // write amplification.
 const finalPartsToMerge = 3

+// The number of shards for rawRow entries per partition.
+//
+// Higher number of shards reduces CPU contention and increases the max bandwidth on multi-core systems.
+var rawRowsShardsPerPartition = (runtime.GOMAXPROCS(-1) + 7) / 8
+
 // getMaxRowsPerPartition returns the maximum number of rows that haven't been converted into parts yet.
 func getMaxRawRowsPerPartition() int {
 	maxRawRowsPerPartitionOnce.Do(func() {
@@ -129,16 +134,10 @@ type partition struct {
 	// Contains file-based parts with big number of items.
 	bigParts []*partWrapper

-	// rawRowsLock protects rawRows.
-	rawRowsLock sync.Mutex
-
 	// rawRows contains recently added rows that haven't been converted into parts yet.
 	//
 	// rawRows aren't used in search for performance reasons.
-	rawRows []rawRow
-
-	// rawRowsLastFlushTime is the last time rawRows are flushed.
-	rawRowsLastFlushTime time.Time
+	rawRows rawRowsShards

 	snapshotLock sync.RWMutex

@@ -264,18 +263,18 @@ func openPartition(smallPartsPath, bigPartsPath string, getDeletedMetricIDs func
 }

 func newPartition(name, smallPartsPath, bigPartsPath string, getDeletedMetricIDs func() *uint64set.Set) *partition {
-	return &partition{
+	p := &partition{
 		name:           name,
 		smallPartsPath: smallPartsPath,
 		bigPartsPath:   bigPartsPath,

 		getDeletedMetricIDs: getDeletedMetricIDs,

-		rawRows: getRawRowsMaxSize().rows,
-
 		mergeIdx: uint64(time.Now().UnixNano()),
 		stopCh:   make(chan struct{}),
 	}
+	p.rawRows.init()
+	return p
 }

 // partitionMetrics contains essential metrics for the partition.
@@ -322,10 +321,9 @@ type partitionMetrics struct {

 // UpdateMetrics updates m with metrics from pt.
 func (pt *partition) UpdateMetrics(m *partitionMetrics) {
-	pt.rawRowsLock.Lock()
-	m.PendingRows += uint64(len(pt.rawRows))
-	m.SmallRowsCount += uint64(len(pt.rawRows))
-	pt.rawRowsLock.Unlock()
+	rawRowsLen := uint64(pt.rawRows.Len())
+	m.PendingRows += rawRowsLen
+	m.SmallRowsCount += rawRowsLen

 	pt.partsLock.Lock()

@@ -399,29 +397,82 @@ func (pt *partition) AddRows(rows []rawRow) {
 		}
 	}

-	// Try adding rows.
-	var rrs []*rawRows
-	pt.rawRowsLock.Lock()
+	pt.rawRows.addRows(pt, rows)
+}
+
+type rawRowsShards struct {
+	lock     sync.Mutex
+	shardIdx int
+
+	// Shards reduce lock contention when adding rows on multi-CPU systems.
+	shards []rawRowsShard
+}
+
+func (rrs *rawRowsShards) init() {
+	rrs.shards = make([]rawRowsShard, rawRowsShardsPerPartition)
+}
+
+func (rrs *rawRowsShards) addRows(pt *partition, rows []rawRow) {
+	rrs.lock.Lock()
+	rrs.shardIdx++
+	if rrs.shardIdx >= len(rrs.shards) {
+		rrs.shardIdx = 0
+	}
+	shard := &rrs.shards[rrs.shardIdx]
+	rrs.lock.Unlock()
+
+	shard.addRows(pt, rows)
+}
+
+func (rrs *rawRowsShards) Len() int {
+	n := 0
+	for i := range rrs.shards[:] {
+		n += rrs.shards[i].Len()
+	}
+	return n
+}
+
+type rawRowsShard struct {
+	lock          sync.Mutex
+	rows          []rawRow
+	lastFlushTime time.Time
+}
+
+func (rrs *rawRowsShard) Len() int {
+	rrs.lock.Lock()
+	n := len(rrs.rows)
+	rrs.lock.Unlock()
+	return n
+}
+
+func (rrs *rawRowsShard) addRows(pt *partition, rows []rawRow) {
+	var rrss []*rawRows
+
+	rrs.lock.Lock()
+	if cap(rrs.rows) == 0 {
+		rrs.rows = getRawRowsMaxSize().rows
+	}
+	maxRowsCount := getMaxRawRowsPerPartition()
 	for {
-		capacity := cap(pt.rawRows) - len(pt.rawRows)
+		capacity := maxRowsCount - len(rrs.rows)
 		if capacity >= len(rows) {
 			// Fast path - rows fit capacity.
-			pt.rawRows = append(pt.rawRows, rows...)
+			rrs.rows = append(rrs.rows, rows...)
 			break
 		}

 		// Slow path - rows don't fit capacity.
 		// Fill rawRows to capacity and convert it to a part.
-		pt.rawRows = append(pt.rawRows, rows[:capacity]...)
+		rrs.rows = append(rrs.rows, rows[:capacity]...)
 		rows = rows[capacity:]
 		rr := getRawRowsMaxSize()
-		pt.rawRows, rr.rows = rr.rows, pt.rawRows
-		rrs = append(rrs, rr)
-		pt.rawRowsLastFlushTime = time.Now()
+		rrs.rows, rr.rows = rr.rows, rrs.rows
+		rrss = append(rrss, rr)
+		rrs.lastFlushTime = time.Now()
 	}
-	pt.rawRowsLock.Unlock()
+	rrs.lock.Unlock()

-	for _, rr := range rrs {
+	for _, rr := range rrss {
 		pt.addRowsPart(rr.rows)
 		putRawRows(rr)
 	}
@@ -583,7 +634,7 @@ func (pt *partition) MustClose() {
 	startTime = time.Now()

 	// Flush raw rows the last time before exit.
-	pt.flushRawRows(nil, true)
+	pt.flushRawRows(true)

 	// Flush inmemory parts to disk.
 	var pws []*partWrapper
@@ -637,7 +688,6 @@ func (pt *partition) startRawRowsFlusher() {
 }

 func (pt *partition) rawRowsFlusher() {
-	var rawRows []rawRow
 	t := time.NewTimer(rawRowsFlushInterval)
 	for {
 		select {
@@ -646,29 +696,35 @@ func (pt *partition) rawRowsFlusher() {
 		case <-t.C:
 			t.Reset(rawRowsFlushInterval)
 		}
-
-		rawRows = pt.flushRawRows(rawRows[:0], false)
+		pt.flushRawRows(false)
 	}
 }

-func (pt *partition) flushRawRows(newRawRows []rawRow, isFinal bool) []rawRow {
-	oldRawRows := newRawRows[:0]
-	mustFlush := false
+func (pt *partition) flushRawRows(isFinal bool) {
+	pt.rawRows.flush(pt, isFinal)
+}
+
+func (rrs *rawRowsShards) flush(pt *partition, isFinal bool) {
+	for i := range rrs.shards[:] {
+		rrs.shards[i].flush(pt, isFinal)
+	}
+}
+
+func (rrs *rawRowsShard) flush(pt *partition, isFinal bool) {
+	var rr *rawRows
 	currentTime := time.Now()

-	pt.rawRowsLock.Lock()
-	if isFinal || currentTime.Sub(pt.rawRowsLastFlushTime) > rawRowsFlushInterval {
-		mustFlush = true
-		oldRawRows = pt.rawRows
-		pt.rawRows = newRawRows[:0]
-		pt.rawRowsLastFlushTime = currentTime
+	rrs.lock.Lock()
+	if isFinal || currentTime.Sub(rrs.lastFlushTime) > rawRowsFlushInterval {
+		rr = getRawRowsMaxSize()
+		rrs.rows, rr.rows = rr.rows, rrs.rows
 	}
-	pt.rawRowsLock.Unlock()
+	rrs.lock.Unlock()

-	if mustFlush {
-		pt.addRowsPart(oldRawRows)
+	if rr != nil {
+		pt.addRowsPart(rr.rows)
+		putRawRows(rr)
 	}
-	return oldRawRows
 }

 func (pt *partition) startInmemoryPartsFlusher() {
@@ -1334,7 +1390,7 @@ func (pt *partition) CreateSnapshotAt(smallPath, bigPath string) error {
 	startTime := time.Now()

 	// Flush inmemory data to disk.
-	pt.flushRawRows(nil, true)
+	pt.flushRawRows(true)
 	if _, err := pt.flushInmemoryParts(nil, true); err != nil {
 		return fmt.Errorf("cannot flush inmemory parts: %s", err)
 	}
--- a/lib/storage/partition_search_test.go
+++ b/lib/storage/partition_search_test.go
@@ -185,7 +185,7 @@ func testPartitionSearchEx(t *testing.T, ptt int64, tr TimeRange, partsCount, ma
 		pt.AddRows(rows)

 		// Flush just added rows to a separate partition.
-		pt.flushRawRows(nil, true)
+		pt.flushRawRows(true)
 	}
 	testPartitionSearch(t, pt, tsids, tr, rbsExpected, -1)
 	pt.MustClose()
--- a/lib/storage/storage.go
+++ b/lib/storage/storage.go
@@ -803,6 +803,11 @@ func (s *Storage) add(rows []rawRow, mrs []MetricRow, precisionBits uint8) ([]ra
 	}
 	rows = rows[:rowsLen+len(mrs)]
 	j := 0
+	var (
+		// These vars are used for speeding up bulk imports of multiple adjancent rows for the same metricName.
+		prevTSID          TSID
+		prevMetricNameRaw []byte
+	)
 	minTimestamp, maxTimestamp := s.tb.getMinMaxTimestamps()
 	// Return only the last error, since it has no sense in returning all errors.
 	var lastWarn error
@@ -830,9 +835,17 @@ func (s *Storage) add(rows []rawRow, mrs []MetricRow, precisionBits uint8) ([]ra
 		r.Timestamp = mr.Timestamp
 		r.Value = mr.Value
 		r.PrecisionBits = precisionBits
+		if string(mr.MetricNameRaw) == string(prevMetricNameRaw) {
+			// Fast path - the current mr contains the same metric name as the previous mr, so it contains the same TSID.
+			// This path should trigger on bulk imports when many rows contain the same MetricNameRaw.
+			r.TSID = prevTSID
+			continue
+		}
 		if s.getTSIDFromCache(&r.TSID, mr.MetricNameRaw) {
 			if !dmis.Has(r.TSID.MetricID) {
 				// Fast path - the TSID for the given MetricName has been found in cache and isn't deleted.
+				prevTSID = r.TSID
+				prevMetricNameRaw = mr.MetricNameRaw
 				continue
 			}
 		}
@@ -890,6 +903,12 @@ func (s *Storage) updatePerDateData(rows []rawRow, lastError error) error {
 	var date uint64
 	var hour uint64
 	var prevTimestamp int64
+	var (
+		// These vars are used for speeding up bulk imports when multiple adjancent rows
+		// contain the same (metricID, date) pairs.
+		prevMatchedDate     uint64
+		prevMatchedMetricID uint64
+	)
 	idb := s.idb()
 	hm := s.currHourMetricIDs.Load().(*hourMetricIDs)
 	for i := range rows {
@@ -913,8 +932,14 @@ func (s *Storage) updatePerDateData(rows []rawRow, lastError error) error {
 		}

 		// Slower path: check global cache for (date, metricID) entry.
+		if metricID == prevMatchedMetricID && date == prevMatchedDate {
+			// Fast path for bulk import of multiple rows with the same (date, metricID) pairs.
+			continue
+		}
 		if s.dateMetricIDCache.Has(date, metricID) {
 			// The metricID has been already added to per-day inverted index.
+			prevMatchedDate = date
+			prevMatchedMetricID = metricID
 			continue
 		}

--- a/lib/storage/table.go
+++ b/lib/storage/table.go
@@ -220,7 +220,7 @@ func (tb *table) flushRawRows() {
 	defer tb.PutPartitions(ptws)

 	for _, ptw := range ptws {
-		ptw.pt.flushRawRows(nil, true)
+		ptw.pt.flushRawRows(true)
 	}
 }

--- a/lib/uint64set/uint64set.go
+++ b/lib/uint64set/uint64set.go
@@ -13,13 +13,11 @@ import (
 //
 // It is unsafe calling Set methods from concurrent goroutines.
 type Set struct {
-	skipSmallPool bool
-	itemsCount    int
-	buckets       bucket32Sorter
-	smallPool     [5]uint64
+	itemsCount int
+	buckets    bucket32Sorter
 }

-type bucket32Sorter []*bucket32
+type bucket32Sorter []bucket32

 func (s *bucket32Sorter) Len() int { return len(*s) }
 func (s *bucket32Sorter) Less(i, j int) bool {
@@ -38,12 +36,12 @@ func (s *Set) Clone() *Set {
 		return &Set{}
 	}
 	var dst Set
-	dst.skipSmallPool = s.skipSmallPool
 	dst.itemsCount = s.itemsCount
-	dst.buckets = make([]*bucket32, len(s.buckets))
-	dst.smallPool = s.smallPool
-	for i, b32 := range s.buckets {
-		dst.buckets[i] = b32.clone()
+	if len(s.buckets) > 0 {
+		dst.buckets = make([]bucket32, len(s.buckets))
+		for i := range s.buckets {
+			s.buckets[i].copyTo(&dst.buckets[i])
+		}
 	}
 	return &dst
 }
@@ -54,9 +52,10 @@ func (s *Set) SizeBytes() uint64 {
 		return 0
 	}
 	n := uint64(unsafe.Sizeof(*s))
-	for _, b := range s.buckets {
-		n += uint64(unsafe.Sizeof(b))
-		n += b.sizeBytes()
+	for i := range s.buckets {
+		b32 := &s.buckets[i]
+		n += uint64(unsafe.Sizeof(b32))
+		n += b32.sizeBytes()
 	}
 	return n
 }
@@ -71,13 +70,10 @@ func (s *Set) Len() int {

 // Add adds x to s.
 func (s *Set) Add(x uint64) {
-	if !s.skipSmallPool {
-		s.addToSmallPool(x)
-		return
-	}
 	hi := uint32(x >> 32)
 	lo := uint32(x)
-	for _, b32 := range s.buckets {
+	for i := range s.buckets {
+		b32 := &s.buckets[i]
 		if b32.hi == hi {
 			if b32.add(lo) {
 				s.itemsCount++
@@ -88,29 +84,16 @@ func (s *Set) Add(x uint64) {
 	s.addAlloc(hi, lo)
 }

-func (s *Set) addToSmallPool(x uint64) {
-	if s.hasInSmallPool(x) {
-		return
-	}
-	if s.itemsCount < len(s.smallPool) {
-		s.smallPool[s.itemsCount] = x
-		s.itemsCount++
-		return
-	}
-	s.skipSmallPool = true
-	s.itemsCount = 0
-	for _, v := range s.smallPool[:] {
-		s.Add(v)
-	}
-	s.Add(x)
-}
-
 func (s *Set) addAlloc(hi, lo uint32) {
-	var b32 bucket32
+	b32 := s.addBucket32()
 	b32.hi = hi
 	_ = b32.add(lo)
 	s.itemsCount++
-	s.buckets = append(s.buckets, &b32)
+}
+
+func (s *Set) addBucket32() *bucket32 {
+	s.buckets = append(s.buckets, bucket32{})
+	return &s.buckets[len(s.buckets)-1]
 }

 // Has verifies whether x exists in s.
@@ -118,12 +101,10 @@ func (s *Set) Has(x uint64) bool {
 	if s == nil {
 		return false
 	}
-	if !s.skipSmallPool {
-		return s.hasInSmallPool(x)
-	}
 	hi := uint32(x >> 32)
 	lo := uint32(x)
-	for _, b32 := range s.buckets {
+	for i := range s.buckets {
+		b32 := &s.buckets[i]
 		if b32.hi == hi {
 			return b32.has(lo)
 		}
@@ -131,24 +112,12 @@ func (s *Set) Has(x uint64) bool {
 	return false
 }

-func (s *Set) hasInSmallPool(x uint64) bool {
-	for _, v := range s.smallPool[:s.itemsCount] {
-		if v == x {
-			return true
-		}
-	}
-	return false
-}
-
 // Del deletes x from s.
 func (s *Set) Del(x uint64) {
-	if !s.skipSmallPool {
-		s.delFromSmallPool(x)
-		return
-	}
 	hi := uint32(x >> 32)
 	lo := uint32(x)
-	for _, b32 := range s.buckets {
+	for i := range s.buckets {
+		b32 := &s.buckets[i]
 		if b32.hi == hi {
 			if b32.del(lo) {
 				s.itemsCount--
@@ -158,20 +127,6 @@ func (s *Set) Del(x uint64) {
 	}
 }

-func (s *Set) delFromSmallPool(x uint64) {
-	idx := -1
-	for i, v := range s.smallPool[:s.itemsCount] {
-		if v == x {
-			idx = i
-		}
-	}
-	if idx < 0 {
-		return
-	}
-	copy(s.smallPool[idx:], s.smallPool[idx+1:])
-	s.itemsCount--
-}
-
 // AppendTo appends all the items from the set to dst and returns the result.
 //
 // The returned items are sorted.
@@ -181,13 +136,6 @@ func (s *Set) AppendTo(dst []uint64) []uint64 {
 	if s == nil {
 		return dst
 	}
-	if !s.skipSmallPool {
-		a := s.smallPool[:s.itemsCount]
-		if len(a) > 1 {
-			sort.Slice(a, func(i, j int) bool { return a[i] < a[j] })
-		}
-		return append(dst, a...)
-	}

 	// pre-allocate memory for dst
 	dstLen := len(dst)
@@ -199,8 +147,8 @@ func (s *Set) AppendTo(dst []uint64) []uint64 {
 	if !sort.IsSorted(&s.buckets) {
 		sort.Sort(&s.buckets)
 	}
-	for _, b32 := range s.buckets {
-		dst = b32.appendTo(dst)
+	for i := range s.buckets {
+		dst = s.buckets[i].appendTo(dst)
 	}
 	return dst
 }
@@ -265,36 +213,33 @@ func (s *Set) Equal(a *Set) bool {
 }

 type bucket32 struct {
-	skipSmallPool bool
-	smallPoolLen  int
-	hi            uint32
-	b16his        []uint16
-	buckets       []*bucket16
-	smallPool     [14]uint32
+	hi      uint32
+	b16his  []uint16
+	buckets []bucket16
 }

 func (b *bucket32) sizeBytes() uint64 {
 	n := uint64(unsafe.Sizeof(*b))
 	n += 2 * uint64(len(b.b16his))
-	for _, b := range b.buckets {
-		n += uint64(unsafe.Sizeof(b))
-		n += b.sizeBytes()
+	for i := range b.buckets {
+		b16 := &b.buckets[i]
+		n += uint64(unsafe.Sizeof(b16))
+		n += b16.sizeBytes()
 	}
 	return n
 }

-func (b *bucket32) clone() *bucket32 {
-	var dst bucket32
-	dst.skipSmallPool = b.skipSmallPool
-	dst.smallPoolLen = b.smallPoolLen
+func (b *bucket32) copyTo(dst *bucket32) {
 	dst.hi = b.hi
 	dst.b16his = append(dst.b16his[:0], b.b16his...)
-	dst.buckets = make([]*bucket16, len(b.buckets))
-	dst.smallPool = b.smallPool
-	for i, b16 := range b.buckets {
-		dst.buckets[i] = b16.clone()
+	// Do not reuse dst.buckets, since it may be used in other places.
+	dst.buckets = nil
+	if len(b.buckets) > 0 {
+		dst.buckets = make([]bucket16, len(b.buckets))
+		for i := range b.buckets {
+			b.buckets[i].copyTo(&dst.buckets[i])
+		}
 	}
-	return &dst
 }

 // This is for sort.Interface
@@ -310,9 +255,6 @@ func (b *bucket32) Swap(i, j int) {
 const maxUnsortedBuckets = 32

 func (b *bucket32) add(x uint32) bool {
-	if !b.skipSmallPool {
-		return b.addToSmallPool(x)
-	}
 	hi := uint16(x >> 16)
 	lo := uint16(x)
 	if len(b.buckets) > maxUnsortedBuckets {
@@ -327,33 +269,20 @@ func (b *bucket32) add(x uint32) bool {
 	return true
 }

-func (b *bucket32) addToSmallPool(x uint32) bool {
-	if b.hasInSmallPool(x) {
-		return false
-	}
-	if b.smallPoolLen < len(b.smallPool) {
-		b.smallPool[b.smallPoolLen] = x
-		b.smallPoolLen++
-		return true
-	}
-	b.skipSmallPool = true
-	b.smallPoolLen = 0
-	for _, v := range b.smallPool[:] {
-		b.add(v)
-	}
-	return b.add(x)
-}
-
 func (b *bucket32) addAllocSmall(hi, lo uint16) {
-	var b16 bucket16
-	_ = b16.add(lo)
 	b.b16his = append(b.b16his, hi)
-	b.buckets = append(b.buckets, &b16)
+	b16 := b.addBucket16()
+	_ = b16.add(lo)
 	if len(b.buckets) > maxUnsortedBuckets {
 		sort.Sort(b)
 	}
 }

+func (b *bucket32) addBucket16() *bucket16 {
+	b.buckets = append(b.buckets, bucket16{})
+	return &b.buckets[len(b.buckets)-1]
+}
+
 func (b *bucket32) addSlow(hi, lo uint16) bool {
 	n := binarySearch16(b.b16his, hi)
 	if n < 0 || n >= len(b.b16his) || b.b16his[n] != hi {
@@ -368,23 +297,21 @@ func (b *bucket32) addAllocBig(hi, lo uint16, n int) {
 		// This is a hint to Go compiler to remove automatic bounds checks below.
 		return
 	}
-	var b16 bucket16
-	_ = b16.add(lo)
 	if n >= len(b.b16his) {
 		b.b16his = append(b.b16his, hi)
-		b.buckets = append(b.buckets, &b16)
+		b16 := b.addBucket16()
+		_ = b16.add(lo)
 		return
 	}
 	b.b16his = append(b.b16his[:n+1], b.b16his[n:]...)
 	b.b16his[n] = hi
 	b.buckets = append(b.buckets[:n+1], b.buckets[n:]...)
-	b.buckets[n] = &b16
+	b16 := &b.buckets[n]
+	*b16 = bucket16{}
+	_ = b16.add(lo)
 }

 func (b *bucket32) has(x uint32) bool {
-	if !b.skipSmallPool {
-		return b.hasInSmallPool(x)
-	}
 	hi := uint16(x >> 16)
 	lo := uint16(x)
 	if len(b.buckets) > maxUnsortedBuckets {
@@ -398,15 +325,6 @@ func (b *bucket32) has(x uint32) bool {
 	return false
 }

-func (b *bucket32) hasInSmallPool(x uint32) bool {
-	for _, v := range b.smallPool[:b.smallPoolLen] {
-		if v == x {
-			return true
-		}
-	}
-	return false
-}
-
 func (b *bucket32) hasSlow(hi, lo uint16) bool {
 	n := binarySearch16(b.b16his, hi)
 	if n < 0 || n >= len(b.b16his) || b.b16his[n] != hi {
@@ -416,9 +334,6 @@ func (b *bucket32) hasSlow(hi, lo uint16) bool {
 }

 func (b *bucket32) del(x uint32) bool {
-	if !b.skipSmallPool {
-		return b.delFromSmallPool(x)
-	}
 	hi := uint16(x >> 16)
 	lo := uint16(x)
 	if len(b.buckets) > maxUnsortedBuckets {
@@ -432,21 +347,6 @@ func (b *bucket32) del(x uint32) bool {
 	return false
 }

-func (b *bucket32) delFromSmallPool(x uint32) bool {
-	idx := -1
-	for i, v := range b.smallPool[:b.smallPoolLen] {
-		if v == x {
-			idx = i
-		}
-	}
-	if idx < 0 {
-		return false
-	}
-	copy(b.smallPool[idx:], b.smallPool[idx+1:])
-	b.smallPoolLen--
-	return true
-}
-
 func (b *bucket32) delSlow(hi, lo uint16) bool {
 	n := binarySearch16(b.b16his, hi)
 	if n < 0 || n >= len(b.b16his) || b.b16his[n] != hi {
@@ -456,24 +356,12 @@ func (b *bucket32) delSlow(hi, lo uint16) bool {
 }

 func (b *bucket32) appendTo(dst []uint64) []uint64 {
-	if !b.skipSmallPool {
-		a := b.smallPool[:b.smallPoolLen]
-		if len(a) > 1 {
-			sort.Slice(a, func(i, j int) bool { return a[i] < a[j] })
-		}
-		hi := uint64(b.hi) << 32
-		for _, lo32 := range a {
-			v := hi | uint64(lo32)
-			dst = append(dst, v)
-		}
-		return dst
-	}
 	if len(b.buckets) <= maxUnsortedBuckets && !sort.IsSorted(b) {
 		sort.Sort(b)
 	}
-	for i, b16 := range b.buckets {
+	for i := range b.buckets {
 		hi16 := b.b16his[i]
-		dst = b16.appendTo(dst, b.hi, hi16)
+		dst = b.buckets[i].appendTo(dst, b.hi, hi16)
 	}
 	return dst
 }
@@ -484,20 +372,30 @@ const (
 )

 type bucket16 struct {
-	bits [wordsPerBucket]uint64
+	bits         *[wordsPerBucket]uint64
+	smallPoolLen int
+	smallPool    [56]uint16
 }

 func (b *bucket16) sizeBytes() uint64 {
-	return uint64(unsafe.Sizeof(*b))
+	return uint64(unsafe.Sizeof(*b)) + uint64(unsafe.Sizeof(*b.bits))
 }

-func (b *bucket16) clone() *bucket16 {
-	var dst bucket16
-	copy(dst.bits[:], b.bits[:])
-	return &dst
+func (b *bucket16) copyTo(dst *bucket16) {
+	// Do not reuse dst.bits, since it may be used in other places.
+	dst.bits = nil
+	if b.bits != nil {
+		bits := *b.bits
+		dst.bits = &bits
+	}
+	dst.smallPoolLen = b.smallPoolLen
+	dst.smallPool = b.smallPool
 }

 func (b *bucket16) add(x uint16) bool {
+	if b.bits == nil {
+		return b.addToSmallPool(x)
+	}
 	wordNum, bitMask := getWordNumBitMask(x)
 	word := &b.bits[wordNum]
 	ok := *word&bitMask == 0
@@ -505,12 +403,46 @@ func (b *bucket16) add(x uint16) bool {
 	return ok
 }

+func (b *bucket16) addToSmallPool(x uint16) bool {
+	if b.hasInSmallPool(x) {
+		return false
+	}
+	if b.smallPoolLen < len(b.smallPool) {
+		b.smallPool[b.smallPoolLen] = x
+		b.smallPoolLen++
+		return true
+	}
+	b.smallPoolLen = 0
+	var bits [wordsPerBucket]uint64
+	b.bits = &bits
+	for _, v := range b.smallPool[:] {
+		b.add(v)
+	}
+	b.add(x)
+	return true
+}
+
 func (b *bucket16) has(x uint16) bool {
+	if b.bits == nil {
+		return b.hasInSmallPool(x)
+	}
 	wordNum, bitMask := getWordNumBitMask(x)
 	return b.bits[wordNum]&bitMask != 0
 }

+func (b *bucket16) hasInSmallPool(x uint16) bool {
+	for _, v := range b.smallPool[:b.smallPoolLen] {
+		if v == x {
+			return true
+		}
+	}
+	return false
+}
+
 func (b *bucket16) del(x uint16) bool {
+	if b.bits == nil {
+		return b.delFromSmallPool(x)
+	}
 	wordNum, bitMask := getWordNumBitMask(x)
 	word := &b.bits[wordNum]
 	ok := *word&bitMask != 0
@@ -518,8 +450,30 @@ func (b *bucket16) del(x uint16) bool {
 	return ok
 }

+func (b *bucket16) delFromSmallPool(x uint16) bool {
+	for i, v := range b.smallPool[:b.smallPoolLen] {
+		if v == x {
+			copy(b.smallPool[i:], b.smallPool[i+1:])
+			b.smallPoolLen--
+			return true
+		}
+	}
+	return false
+}
+
 func (b *bucket16) appendTo(dst []uint64, hi uint32, hi16 uint16) []uint64 {
 	hi64 := uint64(hi)<<32 | uint64(hi16)<<16
+	if b.bits == nil {
+		a := b.smallPool[:b.smallPoolLen]
+		if len(a) > 1 {
+			sort.Slice(a, func(i, j int) bool { return a[i] < a[j] })
+		}
+		for _, v := range a {
+			x := hi64 | uint64(v)
+			dst = append(dst, x)
+		}
+		return dst
+	}
 	var wordNum uint64
 	for _, word := range b.bits {
 		if word == 0 {
--- a/vendor/cloud.google.com/go/CHANGES.md
+++ b/vendor/cloud.google.com/go/CHANGES.md
@@ -1,8 +1,18 @@
 # Changes

+## v0.50.0
+
+- profiler:
+  - Support disabling CPU profile collection.
+  - Log when a profile creation attempt begins.
+- compute/metadata:
+  - Fix panic on malformed URLs.
+  - InstanceName returns actual instance name.
+- Various updates to autogenerated clients.
+
 ## v0.49.0

-functions/metadata:
+- functions/metadata:
  - Handle string resources in JSON unmarshaller.
 - Various updates to autogenerated clients.

--- a/vendor/cloud.google.com/go/README.md
+++ b/vendor/cloud.google.com/go/README.md
@@ -35,20 +35,20 @@ Google API                                      | Status       | Package
 [Automl][cloud-automl]                          | stable       | [`cloud.google.com/go/automl/apiv1`](https://godoc.org/cloud.google.com/go/automl/apiv1)
 [BigQuery][cloud-bigquery]                      | stable       | [`cloud.google.com/go/bigquery`](https://godoc.org/cloud.google.com/go/bigquery)
 [Bigtable][cloud-bigtable]                      | stable       | [`cloud.google.com/go/bigtable`](https://godoc.org/cloud.google.com/go/bigtable)
-[Cloudbuild][cloud-build]                       | alpha        | [`cloud.google.com/go/cloudbuild/apiv1`](https://godoc.org/cloud.google.com/go/cloudbuild/apiv1)
+[Cloudbuild][cloud-build]                       | stable       | [`cloud.google.com/go/cloudbuild/apiv1`](https://godoc.org/cloud.google.com/go/cloudbuild/apiv1)
 [Cloudtasks][cloud-tasks]                       | stable       | [`cloud.google.com/go/cloudtasks/apiv2`](https://godoc.org/cloud.google.com/go/cloudtasks/apiv2)
 [Container][cloud-container]                    | stable       | [`cloud.google.com/go/container/apiv1`](https://godoc.org/cloud.google.com/go/container/apiv1)
 [ContainerAnalysis][cloud-containeranalysis]    | beta         | [`cloud.google.com/go/containeranalysis/apiv1`](https://godoc.org/cloud.google.com/go/containeranalysis/apiv1)
 [Dataproc][cloud-dataproc]                      | stable       | [`cloud.google.com/go/dataproc/apiv1`](https://godoc.org/cloud.google.com/go/dataproc/apiv1)
 [Datastore][cloud-datastore]                    | stable       | [`cloud.google.com/go/datastore`](https://godoc.org/cloud.google.com/go/datastore)
-[Debugger][cloud-debugger]                      | alpha        | [`cloud.google.com/go/debugger/apiv2`](https://godoc.org/cloud.google.com/go/debugger/apiv2)
-[Dialogflow][cloud-dialogflow]                  | alpha        | [`cloud.google.com/go/dialogflow/apiv2`](https://godoc.org/cloud.google.com/go/dialogflow/apiv2)
-[Data Loss Prevention][cloud-dlp]               | alpha        | [`cloud.google.com/go/dlp/apiv2`](https://godoc.org/cloud.google.com/go/dlp/apiv2)
+[Debugger][cloud-debugger]                      | stable       | [`cloud.google.com/go/debugger/apiv2`](https://godoc.org/cloud.google.com/go/debugger/apiv2)
+[Dialogflow][cloud-dialogflow]                  | stable       | [`cloud.google.com/go/dialogflow/apiv2`](https://godoc.org/cloud.google.com/go/dialogflow/apiv2)
+[Data Loss Prevention][cloud-dlp]               | stable       | [`cloud.google.com/go/dlp/apiv2`](https://godoc.org/cloud.google.com/go/dlp/apiv2)
 [ErrorReporting][cloud-errors]                  | alpha        | [`cloud.google.com/go/errorreporting`](https://godoc.org/cloud.google.com/go/errorreporting)
 [Firestore][cloud-firestore]                    | stable       | [`cloud.google.com/go/firestore`](https://godoc.org/cloud.google.com/go/firestore)
 [IAM][cloud-iam]                                | stable       | [`cloud.google.com/go/iam`](https://godoc.org/cloud.google.com/go/iam)
-[IoT][cloud-iot]                                | alpha        | [`cloud.google.com/iot/apiv1`](https://godoc.org/cloud.google.com/iot/apiv1)
-[IRM][cloud-irm]                                | alpha        | [`cloud.google.com/irm/apiv1alpha2`](https://godoc.org/cloud.google.com/irm/apiv1alpha2)
+[IoT][cloud-iot]                                | stable       | [`cloud.google.com/go/iot/apiv1`](https://godoc.org/cloud.google.com/go/iot/apiv1)
+[IRM][cloud-irm]                                | alpha        | [`cloud.google.com/go/irm/apiv1alpha2`](https://godoc.org/cloud.google.com/go/irm/apiv1alpha2)
 [KMS][cloud-kms]                                | stable       | [`cloud.google.com/go/kms`](https://godoc.org/cloud.google.com/go/kms)
 [Natural Language][cloud-natural-language]      | stable       | [`cloud.google.com/go/language/apiv1`](https://godoc.org/cloud.google.com/go/language/apiv1)
 [Logging][cloud-logging]                        | stable       | [`cloud.google.com/go/logging`](https://godoc.org/cloud.google.com/go/logging)
--- a/vendor/cloud.google.com/go/compute/metadata/metadata.go
+++ b/vendor/cloud.google.com/go/compute/metadata/metadata.go
@@ -304,7 +304,10 @@ func (c *Client) getETag(suffix string) (value, etag string, err error) {
 		host = metadataIP
 	}
 	u := "http://" + host + "/computeMetadata/v1/" + suffix
-	req, _ := http.NewRequest("GET", u, nil)
+	req, err := http.NewRequest("GET", u, nil)
+	if err != nil {
+		return "", "", err
+	}
 	req.Header.Set("Metadata-Flavor", "Google")
 	req.Header.Set("User-Agent", userAgent)
 	res, err := c.hc.Do(req)
@@ -407,11 +410,7 @@ func (c *Client) InstanceTags() ([]string, error) {

 // InstanceName returns the current VM's instance ID string.
 func (c *Client) InstanceName() (string, error) {
-	host, err := c.Hostname()
-	if err != nil {
-		return "", err
-	}
-	return strings.Split(host, ".")[0], nil
+	return c.getTrimmed("instance/name")
 }

 // Zone returns the current VM's zone, such as "us-central1-b".
--- a/vendor/cloud.google.com/go/gapics.txt
+++ b/vendor/cloud.google.com/go/gapics.txt
@@ -1,48 +0,0 @@
-google/api/expr/artman_cel.yaml
-google/cloud/asset/artman_cloudasset_v1beta1.yaml
-google/cloud/asset/artman_cloudasset_v1p2beta1.yaml
-google/iam/credentials/artman_iamcredentials_v1.yaml
-google/cloud/automl/artman_automl_v1.yaml
-google/cloud/automl/artman_automl_v1beta1.yaml
-google/cloud/bigquery/datatransfer/artman_bigquerydatatransfer.yaml
-google/cloud/bigquery/storage/artman_bigquerystorage_v1beta1.yaml
-google/cloud/dataproc/artman_dataproc_v1.yaml
-google/cloud/dataproc/artman_dataproc_v1beta2.yaml
-google/cloud/dialogflow/v2/artman_dialogflow_v2.yaml
-google/cloud/iot/artman_cloudiot.yaml
-google/cloud/irm/artman_irm_v1alpha2.yaml
-google/cloud/kms/artman_cloudkms.yaml
-google/cloud/language/artman_language_v1beta2.yaml
-google/cloud/oslogin/artman_oslogin_v1.yaml
-google/cloud/oslogin/artman_oslogin_v1beta.yaml
-google/cloud/recaptchaenterprise/artman_recaptchaenterprise_v1beta1.yaml
-google/cloud/recommender/artman_recommender_v1beta1.yaml
-google/cloud/redis/artman_redis_v1beta1.yaml
-google/cloud/redis/artman_redis_v1.yaml
-google/cloud/securitycenter/artman_securitycenter_v1beta1.yaml
-google/cloud/securitycenter/artman_securitycenter_v1.yaml
-google/cloud/talent/artman_talent_v4beta1.yaml
-google/cloud/tasks/artman_cloudtasks_v2beta2.yaml
-google/cloud/tasks/artman_cloudtasks_v2beta3.yaml
-google/cloud/tasks/artman_cloudtasks_v2.yaml
-google/cloud/videointelligence/artman_videointelligence_v1.yaml
-google/cloud/videointelligence/artman_videointelligence_v1beta2.yaml
-google/cloud/vision/artman_vision_v1.yaml
-google/cloud/vision/artman_vision_v1p1beta1.yaml
-google/cloud/webrisk/artman_webrisk_v1beta1.yaml
-google/devtools/artman_clouddebugger.yaml
-google/devtools/cloudbuild/artman_cloudbuild.yaml
-google/devtools/clouderrorreporting/artman_errorreporting.yaml
-google/devtools/cloudtrace/artman_cloudtrace_v1.yaml
-google/devtools/cloudtrace/artman_cloudtrace_v2.yaml
-google/devtools/containeranalysis/artman_containeranalysis_v1beta1.yaml
-google/firestore/artman_firestore.yaml
-google/firestore/admin/artman_firestore_v1.yaml
-google/logging/artman_logging.yaml
-google/longrunning/artman_longrunning.yaml
-google/monitoring/artman_monitoring.yaml
-google/privacy/dlp/artman_dlp_v2.yaml
-google/pubsub/artman_pubsub.yaml
-google/spanner/admin/database/artman_spanner_admin_database.yaml
-google/spanner/admin/instance/artman_spanner_admin_instance.yaml
-google/spanner/artman_spanner.yaml
--- a/vendor/cloud.google.com/go/go.mod
+++ b/vendor/cloud.google.com/go/go.mod
@@ -15,14 +15,16 @@ require (
 	github.com/googleapis/gax-go/v2 v2.0.5
 	github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024
 	go.opencensus.io v0.22.0
-	golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136
-	golang.org/x/lint v0.0.0-20190930215403-16217165b5de
-	golang.org/x/net v0.0.0-20190620200207-3b0461eec859
+	golang.org/x/exp v0.0.0-20191129062945-2f5052295587
+	golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f
+	golang.org/x/net v0.0.0-20190724013045-ca1201d0de80
 	golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45
+	golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e // indirect
+	golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e // indirect
 	golang.org/x/text v0.3.2
-	golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2
+	golang.org/x/tools v0.0.0-20191216173652-a0e659d51361
 	google.golang.org/api v0.14.0
-	google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9
+	google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1
 	google.golang.org/grpc v1.21.1
 	honnef.co/go/tools v0.0.1-2019.2.3
 )
--- a/vendor/cloud.google.com/go/go.sum
+++ b/vendor/cloud.google.com/go/go.sum
@@ -18,7 +18,7 @@ github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ
 github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
 github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
 github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
-github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
+github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
 github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58=
 github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
 github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
@@ -57,8 +57,10 @@ github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ
 github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024 h1:rBMNdlhTLzJjJSDIjNEXX1Pz3Hmwmz91v+zycvx9PJc=
 github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
 github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
+github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
 github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
 github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
+github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
 github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
 github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
 go.opencensus.io v0.21.0 h1:mU6zScU4U1YAFPHEHYk+3JC4SY7JxgkqS10ZOSyksNg=
@@ -68,6 +70,7 @@ go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
 golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/exp v0.0.0-20190121172915-509febef88a4 h1:c2HOrn5iMezYjSlGPncknSEr/8x5LELb/ilJbXi9DEA=
 golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
@@ -75,8 +78,8 @@ golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522 h1:OeRHuibLsmZkFj773W4LcfAGs
 golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
 golang.org/x/exp v0.0.0-20190829153037-c13cbed26979 h1:Agxu5KLo8o7Bb634SVDnhIfpTvxmzUwhbYAzBvXt6h4=
 golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=
-golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136 h1:A1gGSx58LAGVHUUsOf7IiR0u8Xb6W51gRwfDBhkdcaw=
-golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=
+golang.org/x/exp v0.0.0-20191129062945-2f5052295587 h1:5Uz0rkjCFu9BC9gCRN7EkwVvhNyQgGWb8KNJrPwBoHY=
+golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
 golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
 golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
 golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
@@ -88,12 +91,13 @@ golang.org/x/lint v0.0.0-20190409202823-959b441ac422 h1:QzoH/1pFpZguR8NrRHLcO6jK
 golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
 golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac h1:8R1esu+8QioDxo4E4mX6bFztO+dMTM49DNAaWfO5OeY=
 golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
-golang.org/x/lint v0.0.0-20190930215403-16217165b5de h1:5hukYrvBGR8/eNkX5mdUezrA6JiaEZDtJb9Ei+1LlBs=
-golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
+golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f h1:J5lckAjkw6qYlOZNj90mLYNTEKDvWeuc1yieZ8qUzUE=
+golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
 golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
 golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
 golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
 golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
+golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
 golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -107,6 +111,8 @@ golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn
 golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
 golang.org/x/net v0.0.0-20190620200207-3b0461eec859 h1:R/3boaszxrf1GEUWTVDzSKVwLmSJpwZ1yqXm8j0v2QI=
 golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20190724013045-ca1201d0de80 h1:Ao/3l156eZf2AW5wK8a7/smtodRU+gha3+BeqJ69lRk=
+golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421 h1:Wo7BWFiOk0QRFMLYMqJGFMd9CgUAcGx7V+qEg/h5IBI=
 golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -119,6 +125,8 @@ golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6 h1:bjcUS9ztw9kFmmIxJInhon/0
 golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
 golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e h1:vcxGaoTs7kV8m5Np9uUNQin4BrLOthgV7252N8V+FwY=
+golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a h1:1BGLXjeY4akVXGgbC9HugT3Jv3hCI0z56oJR5vAMgBU=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -130,6 +138,8 @@ golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0 h1:HyfiK1WMnHj5FXFXatD+Qs1A/xC2Run6RzeW1SyHxpc=
 golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e h1:D5TXcfTk7xF7hvieo4QErS3qqCB4teTffacDWr7CI+0=
+golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2 h1:z99zHgr7hKfrUcX/KsoJk5FJfjTceCKIp96+biqP4To=
 golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -157,9 +167,11 @@ golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtn
 golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff h1:On1qIo75ByTwFJ4/W2bIqHcwJ9XAqtSWUs8GwRrIhtc=
 golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
 golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2 h1:EtTFh6h4SAKemS+CURDMTDIANuduG5zKEXShyy18bGA=
-golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20191216173652-a0e659d51361 h1:RIIXAeV6GvDBuADKumTODatUqANFZ+5BPMnzsy4hulY=
+golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
 google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
 google.golang.org/api v0.8.0 h1:VGGbLNyPF7dvYHhcUGYBBGCRDDK0RRJAI6KCvo0CL+E=
@@ -188,14 +200,15 @@ google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55 h1:gSJIx1SDwno+2El
 google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
 google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51 h1:Ex1mq5jaJof+kRnYi3SlYJ8KKa9Ao3NHyIT5XJ1gF6U=
 google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=
-google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9 h1:6XzpBoANz1NqMNfDXzc2QmHmbb1vyMsvRfoP5rM+K1I=
-google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
+google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1 h1:aQktFqmDE2yjveXJlVIfslDFmFnUXSqG0i6KRcJAeMc=
+google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
 google.golang.org/grpc v1.19.0 h1:cfg4PD8YEdSFnm7qLV4++93WcmhH2nIUhMjhdCvl3j8=
 google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
 google.golang.org/grpc v1.20.1 h1:Hz2g2wirWK7H0qIIhGIqRGTuMwTE8HEKFnDZZ7lm9NU=
 google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
 google.golang.org/grpc v1.21.1 h1:j6XxA85m/6txkUCHvzlV5f+HBNl/1r5cZ2A/3IEFOO8=
 google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
+gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
 honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
--- a/vendor/cloud.google.com/go/manuals.txt
+++ b/vendor/cloud.google.com/go/manuals.txt
@@ -1,8 +0,0 @@
-errorreporting/apiv1beta1
-firestore/apiv1beta1
-firestore/apiv1
-logging/apiv2
-longrunning/autogen
-pubsub/apiv1
-spanner/apiv1
-trace/apiv1
--- a/vendor/cloud.google.com/go/microgens.csv
+++ b/vendor/cloud.google.com/go/microgens.csv
@@ -1,10 +0,0 @@
-input directory path, go module;package flag, gRPC ServiceConfig path flag, API service config path flag, release level
-google/cloud/texttospeech/v1, --go-gapic-package cloud.google.com/go/texttospeech/apiv1;texttospeech, --grpc-service-config google/cloud/texttospeech/v1/texttospeech_grpc_service_config.json, --gapic-service-config google/cloud/texttospeech/v1/texttospeech_v1.yaml, --release-level alpha
-google/cloud/asset/v1, --go-gapic-package cloud.google.com/go/asset/apiv1;asset, --grpc-service-config google/cloud/asset/v1/cloudasset_grpc_service_config.json, --gapic-service-config google/cloud/asset/v1/cloudasset_v1.yaml, --release-level alpha
-google/cloud/language/v1, --go-gapic-package cloud.google.com/go/language/apiv1;language, --grpc-service-config google/cloud/language/v1/language_grpc_service_config.json, --gapic-service-config google/cloud/language/language_v1.yaml, --release-level alpha
-google/cloud/phishingprotection/v1beta1, --go-gapic-package cloud.google.com/go/phishingprotection/apiv1beta1;phishingprotection, --grpc-service-config google/cloud/phishingprotection/v1beta1/phishingprotection_grpc_service_config.json, --gapic-service-config google/cloud/phishingprotection/v1beta1/phishingprotection_v1beta1.yaml, --release-level beta
-google/cloud/translate/v3, --go-gapic-package cloud.google.com/go/translate/apiv3;translate, --grpc-service-config google/cloud/translate/v3/translate_grpc_service_config.json, --gapic-service-config google/cloud/translate/v3/translate_v3.yaml,
-google/cloud/scheduler/v1, --go-gapic-package cloud.google.com/go/scheduler/apiv1;scheduler, --grpc-service-config google/cloud/scheduler/v1/cloudscheduler_grpc_service_config.json, --gapic-service-config google/cloud/scheduler/v1/cloudscheduler_v1.yaml,
-google/cloud/scheduler/v1beta1, --go-gapic-package cloud.google.com/go/scheduler/apiv1beta1;scheduler, --grpc-service-config google/cloud/scheduler/v1beta1/cloudscheduler_grpc_service_config.json, --gapic-service-config google/cloud/scheduler/v1beta1/cloudscheduler_v1beta1.yaml, --release-level beta
-google/cloud/speech/v1, --go-gapic-package cloud.google.com/go/speech/apiv1;speech, --grpc-service-config google/cloud/speech/v1/speech_grpc_service_config.json, --gapic-service-config google/cloud/speech/v1/speech_v1.yaml,
-google/cloud/speech/v1p1beta1, --go-gapic-package cloud.google.com/go/speech/apiv1p1beta1;speech, --grpc-service-config google/cloud/speech/v1p1beta1/speech_grpc_service_config.json, --gapic-service-config google/cloud/speech/v1p1beta1/speech_v1p1beta1.yaml, --release-level beta
--- a/vendor/cloud.google.com/go/regen-gapic.sh
+++ b/vendor/cloud.google.com/go/regen-gapic.sh
@@ -1,81 +0,0 @@
-#!/bin/bash
-# Copyright 2019 Google LLC
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# This script generates all GAPIC clients in this repo.
-# See instructions at go/yoshi-site.
-
-set -ex
-
-GOCLOUD_DIR="$(dirname "$0")"
-HOST_MOUNT="$PWD"
-
-# need to mount the /var/folders properly for macos
-# https://stackoverflow.com/questions/45122459/docker-mounts-denied-the-paths-are-not-shared-from-os-x-and-are-not-known/45123074
-if [[ "$OSTYPE" == "darwin"* ]] && [[ "$HOST_MOUNT" == "/var/folders"* ]]; then
-  HOST_MOUNT=/private$HOST_MOUNT
-fi
-
-microgen() {
-  input=$1
-  options="${@:2}"
-
-  # see https://github.com/googleapis/gapic-generator-go/blob/master/README.md#docker-wrapper for details
-  docker run \
-    --user $UID \
-    --mount type=bind,source=$HOST_MOUNT,destination=/conf,readonly \
-    --mount type=bind,source=$HOST_MOUNT/$input,destination=/in/$input,readonly \
-    --mount type=bind,source=/tmp,destination=/out \
-    --rm \
-    gcr.io/gapic-images/gapic-generator-go:0.9.1 \
-    $options
-}
-
-for gencfg in $(cat $GOCLOUD_DIR/gapics.txt); do
-  rm -rf artman-genfiles/*
-  artman --config "$gencfg" generate go_gapic
-  cp -r artman-genfiles/gapi-*/cloud.google.com/go/* $GOCLOUD_DIR
-done
-
-rm -rf /tmp/cloud.google.com
-{
-  # skip the first line with column titles
-  read -r
-  while IFS=, read -r input mod retrycfg apicfg release
-  do
-    microgen $input "$mod" "$retrycfg" "$apicfg" "$release"
-  done
-} < $GOCLOUD_DIR/microgens.csv
-
-# copy generated code if any was created
-[ -d "/tmp/cloud.google.com/go" ] && cp -r /tmp/cloud.google.com/go/* $GOCLOUD_DIR
-
-pushd $GOCLOUD_DIR
-  gofmt -s -d -l -w . && goimports -w .
-
-  # NOTE(pongad): `sed -i` doesn't work on Macs, because -i option needs an argument.
-  # `-i ''` doesn't work on GNU, since the empty string is treated as a file name.
-  # So we just create the backup and delete it after.
-  ver=$(date +%Y%m%d)
-  git ls-files -mo | while read modified; do
-    dir=${modified%/*.*}
-    find . -path "*/$dir/doc.go" -exec sed -i.backup -e "s/^const versionClient.*/const versionClient = \"$ver\"/" '{}' +
-  done
-popd
-
-for manualdir in $(cat $GOCLOUD_DIR/manuals.txt); do
-  find "$GOCLOUD_DIR/$manualdir" -name '*.go' -exec sed -i.backup -e 's/setGoogleClientInfo/SetGoogleClientInfo/g' '{}' '+'
-done
-
-find $GOCLOUD_DIR -name '*.backup' -delete
--- a/vendor/cloud.google.com/go/tidyall.sh
+++ b/vendor/cloud.google.com/go/tidyall.sh
@@ -1,28 +0,0 @@
-#!/bin/bash
-# Copyright 2019 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# Run at repo root.
-
-go mod tidy
-for i in `find . -name go.mod`; do
-pushd `dirname $i`;
-    # Update genproto and api to latest for every module (latest version is
-    # always correct version). tidy will remove the dependencies if they're not
-    # actually used by the client.
-    go get -u google.golang.org/api
-    go get -u google.golang.org/genproto
-    go mod tidy;
-popd;
-done
--- a/vendor/github.com/VictoriaMetrics/fastcache/fastcache.go
+++ b/vendor/github.com/VictoriaMetrics/fastcache/fastcache.go
@@ -168,7 +168,7 @@ func (c *Cache) Get(dst, k []byte) []byte {
 func (c *Cache) HasGet(dst, k []byte) ([]byte, bool) {
 	h := xxhash.Sum64(k)
 	idx := h % bucketsCount
-	return c.buckets[idx].Get(nil, k, h, true)
+	return c.buckets[idx].Get(dst, k, h, true)
 }

 // Has returns true if entry for the given key k exists in the cache.
--- a/vendor/github.com/VictoriaMetrics/fastcache/file.go
+++ b/vendor/github.com/VictoriaMetrics/fastcache/file.go
@@ -205,6 +205,9 @@ func loadMetadata(dir string) (uint64, error) {
 	if err != nil {
 		return 0, fmt.Errorf("cannot read maxBucketChunks from %q: %s", metadataPath, err)
 	}
+	if maxBucketChunks == 0 {
+		return 0, fmt.Errorf("invalid maxBucketChunks=0 read from %q", metadataPath)
+	}
 	return maxBucketChunks, nil
 }

@@ -359,10 +362,16 @@ func (b *bucket) Load(r io.Reader, maxChunks uint64) error {
 	}
 	for chunkIdx := uint64(0); chunkIdx < chunksLen; chunkIdx++ {
 		chunk := getChunk()
+		chunks[chunkIdx] = chunk
 		if _, err := io.ReadFull(r, chunk); err != nil {
+			// Free up allocated chunks before returning the error.
+			for _, chunk := range chunks {
+				if chunk != nil {
+					putChunk(chunk)
+				}
+			}
 			return fmt.Errorf("cannot read b.chunks[%d]: %s", chunkIdx, err)
 		}
-		chunks[chunkIdx] = chunk
 	}
 	// Adjust len for the chunk pointed by currChunkIdx.
 	if chunksLen > 0 {
--- a/vendor/github.com/aws/aws-sdk-go/aws/endpoints/defaults.go
+++ b/vendor/github.com/aws/aws-sdk-go/aws/endpoints/defaults.go
@@ -179,17 +179,47 @@ var awsPartition = partition{
 				"ap-southeast-1": endpoint{},
 				"ap-southeast-2": endpoint{},
 				"ca-central-1":   endpoint{},
-				"eu-central-1":   endpoint{},
-				"eu-north-1":     endpoint{},
-				"eu-west-1":      endpoint{},
-				"eu-west-2":      endpoint{},
-				"eu-west-3":      endpoint{},
-				"me-south-1":     endpoint{},
-				"sa-east-1":      endpoint{},
-				"us-east-1":      endpoint{},
-				"us-east-2":      endpoint{},
-				"us-west-1":      endpoint{},
-				"us-west-2":      endpoint{},
+				"ca-central-1-fips": endpoint{
+					Hostname: "acm-fips.ca-central-1.amazonaws.com",
+					CredentialScope: credentialScope{
+						Region: "ca-central-1",
+					},
+				},
+				"eu-central-1": endpoint{},
+				"eu-north-1":   endpoint{},
+				"eu-west-1":    endpoint{},
+				"eu-west-2":    endpoint{},
+				"eu-west-3":    endpoint{},
+				"me-south-1":   endpoint{},
+				"sa-east-1":    endpoint{},
+				"us-east-1":    endpoint{},
+				"us-east-1-fips": endpoint{
+					Hostname: "acm-fips.us-east-1.amazonaws.com",
+					CredentialScope: credentialScope{
+						Region: "us-east-1",
+					},
+				},
+				"us-east-2": endpoint{},
+				"us-east-2-fips": endpoint{
+					Hostname: "acm-fips.us-east-2.amazonaws.com",
+					CredentialScope: credentialScope{
+						Region: "us-east-2",
+					},
+				},
+				"us-west-1": endpoint{},
+				"us-west-1-fips": endpoint{
+					Hostname: "acm-fips.us-west-1.amazonaws.com",
+					CredentialScope: credentialScope{
+						Region: "us-west-1",
+					},
+				},
+				"us-west-2": endpoint{},
+				"us-west-2-fips": endpoint{
+					Hostname: "acm-fips.us-west-2.amazonaws.com",
+					CredentialScope: credentialScope{
+						Region: "us-west-2",
+					},
+				},
 			},
 		},
 		"acm-pca": service{
@@ -553,6 +583,7 @@ var awsPartition = partition{
 				"eu-west-2":      endpoint{},
 				"eu-west-3":      endpoint{},
 				"me-south-1":     endpoint{},
+				"sa-east-1":      endpoint{},
 				"us-east-1":      endpoint{},
 				"us-east-2":      endpoint{},
 				"us-west-1":      endpoint{},
@@ -697,9 +728,15 @@ var awsPartition = partition{

 			Endpoints: endpoints{
 				"ap-northeast-1": endpoint{},
+				"ap-northeast-2": endpoint{},
+				"ap-south-1":     endpoint{},
 				"ap-southeast-1": endpoint{},
+				"ap-southeast-2": endpoint{},
+				"ca-central-1":   endpoint{},
 				"eu-central-1":   endpoint{},
+				"eu-north-1":     endpoint{},
 				"eu-west-1":      endpoint{},
+				"eu-west-2":      endpoint{},
 				"us-east-1":      endpoint{},
 				"us-east-2":      endpoint{},
 				"us-west-2":      endpoint{},
@@ -886,6 +923,7 @@ var awsPartition = partition{
 		"codecommit": service{

 			Endpoints: endpoints{
+				"ap-east-1":      endpoint{},
 				"ap-northeast-1": endpoint{},
 				"ap-northeast-2": endpoint{},
 				"ap-south-1":     endpoint{},
@@ -1101,8 +1139,10 @@ var awsPartition = partition{

 			Endpoints: endpoints{
 				"ap-northeast-1": endpoint{},
+				"ap-southeast-1": endpoint{},
 				"ap-southeast-2": endpoint{},
 				"eu-central-1":   endpoint{},
+				"eu-west-2":      endpoint{},
 				"us-east-1":      endpoint{},
 				"us-west-2":      endpoint{},
 			},
@@ -1309,6 +1349,12 @@ var awsPartition = partition{
 						Region: "ap-southeast-2",
 					},
 				},
+				"ca-central-1": endpoint{
+					Hostname: "rds.ca-central-1.amazonaws.com",
+					CredentialScope: credentialScope{
+						Region: "ca-central-1",
+					},
+				},
 				"eu-central-1": endpoint{
 					Hostname: "rds.eu-central-1.amazonaws.com",
 					CredentialScope: credentialScope{
@@ -1367,6 +1413,7 @@ var awsPartition = partition{
 				"eu-north-1":     endpoint{},
 				"eu-west-1":      endpoint{},
 				"eu-west-2":      endpoint{},
+				"eu-west-3":      endpoint{},
 				"sa-east-1":      endpoint{},
 				"us-east-1":      endpoint{},
 				"us-east-2":      endpoint{},
@@ -1884,8 +1931,10 @@ var awsPartition = partition{
 		"groundstation": service{

 			Endpoints: endpoints{
-				"us-east-2": endpoint{},
-				"us-west-2": endpoint{},
+				"eu-north-1": endpoint{},
+				"me-south-1": endpoint{},
+				"us-east-2":  endpoint{},
+				"us-west-2":  endpoint{},
 			},
 		},
 		"guardduty": service{
@@ -2196,11 +2245,13 @@ var awsPartition = partition{
 				"ap-south-1":     endpoint{},
 				"ap-southeast-1": endpoint{},
 				"ap-southeast-2": endpoint{},
+				"ca-central-1":   endpoint{},
 				"eu-central-1":   endpoint{},
 				"eu-north-1":     endpoint{},
 				"eu-west-1":      endpoint{},
 				"eu-west-2":      endpoint{},
 				"eu-west-3":      endpoint{},
+				"sa-east-1":      endpoint{},
 				"us-east-1":      endpoint{},
 				"us-east-2":      endpoint{},
 				"us-west-2":      endpoint{},
@@ -2209,11 +2260,20 @@ var awsPartition = partition{
 		"kinesisvideo": service{

 			Endpoints: endpoints{
+				"ap-east-1":      endpoint{},
 				"ap-northeast-1": endpoint{},
+				"ap-northeast-2": endpoint{},
+				"ap-south-1":     endpoint{},
+				"ap-southeast-1": endpoint{},
 				"ap-southeast-2": endpoint{},
+				"ca-central-1":   endpoint{},
 				"eu-central-1":   endpoint{},
 				"eu-west-1":      endpoint{},
+				"eu-west-2":      endpoint{},
+				"eu-west-3":      endpoint{},
+				"sa-east-1":      endpoint{},
 				"us-east-1":      endpoint{},
+				"us-east-2":      endpoint{},
 				"us-west-2":      endpoint{},
 			},
 		},
@@ -2536,6 +2596,7 @@ var awsPartition = partition{
 				"ap-southeast-2": endpoint{},
 				"ca-central-1":   endpoint{},
 				"eu-central-1":   endpoint{},
+				"eu-north-1":     endpoint{},
 				"eu-west-1":      endpoint{},
 				"eu-west-2":      endpoint{},
 				"eu-west-3":      endpoint{},
@@ -2563,6 +2624,7 @@ var awsPartition = partition{
 						Region: "us-west-2",
 					},
 				},
+				"sa-east-1": endpoint{},
 				"us-east-1": endpoint{},
 				"us-east-2": endpoint{},
 				"us-west-1": endpoint{},
@@ -2774,6 +2836,20 @@ var awsPartition = partition{
 				},
 			},
 		},
+		"outposts": service{
+
+			Endpoints: endpoints{
+				"ap-northeast-1": endpoint{},
+				"ap-northeast-2": endpoint{},
+				"ap-southeast-2": endpoint{},
+				"eu-central-1":   endpoint{},
+				"eu-west-1":      endpoint{},
+				"us-east-1":      endpoint{},
+				"us-east-2":      endpoint{},
+				"us-west-1":      endpoint{},
+				"us-west-2":      endpoint{},
+			},
+		},
 		"pinpoint": service{
 			Defaults: endpoint{
 				CredentialScope: credentialScope{
@@ -3093,6 +3169,7 @@ var awsPartition = partition{
 				"ap-southeast-2": endpoint{},
 				"ca-central-1":   endpoint{},
 				"eu-central-1":   endpoint{},
+				"eu-north-1":     endpoint{},
 				"eu-west-1":      endpoint{},
 				"eu-west-2":      endpoint{},
 				"eu-west-3":      endpoint{},
@@ -3189,7 +3266,8 @@ var awsPartition = partition{
 					SignatureVersions: []string{"s3", "s3v4"},
 				},
 				"aws-global": endpoint{
-					Hostname: "s3.amazonaws.com",
+					Hostname:          "s3.amazonaws.com",
+					SignatureVersions: []string{"s3", "s3v4"},
 					CredentialScope: credentialScope{
 						Region: "us-east-1",
 					},
@@ -3215,7 +3293,10 @@ var awsPartition = partition{
 					Hostname:          "s3.sa-east-1.amazonaws.com",
 					SignatureVersions: []string{"s3", "s3v4"},
 				},
-				"us-east-1": endpoint{},
+				"us-east-1": endpoint{
+					Hostname:          "s3.us-east-1.amazonaws.com",
+					SignatureVersions: []string{"s3", "s3v4"},
+				},
 				"us-east-2": endpoint{},
 				"us-west-1": endpoint{
 					Hostname:          "s3.us-west-1.amazonaws.com",
@@ -4611,6 +4692,13 @@ var awscnPartition = partition{
 				},
 			},
 		},
+		"secretsmanager": service{
+
+			Endpoints: endpoints{
+				"cn-north-1":     endpoint{},
+				"cn-northwest-1": endpoint{},
+			},
+		},
 		"sms": service{

 			Endpoints: endpoints{
@@ -4733,6 +4821,13 @@ var awscnPartition = partition{
 				"cn-northwest-1": endpoint{},
 			},
 		},
+		"xray": service{
+
+			Endpoints: endpoints{
+				"cn-north-1":     endpoint{},
+				"cn-northwest-1": endpoint{},
+			},
+		},
 	},
 }

--- a/vendor/github.com/aws/aws-sdk-go/aws/endpoints/endpoints.go
+++ b/vendor/github.com/aws/aws-sdk-go/aws/endpoints/endpoints.go
@@ -317,7 +317,7 @@ func (p Partition) EndpointFor(service, region string, opts ...func(*Options)) (
 // Regions returns a map of Regions indexed by their ID. This is useful for
 // enumerating over the regions in a partition.
 func (p Partition) Regions() map[string]Region {
-	rs := map[string]Region{}
+	rs := make(map[string]Region, len(p.p.Regions))
 	for id, r := range p.p.Regions {
 		rs[id] = Region{
 			id:   id,
@@ -332,7 +332,7 @@ func (p Partition) Regions() map[string]Region {
 // Services returns a map of Service indexed by their ID. This is useful for
 // enumerating over the services in a partition.
 func (p Partition) Services() map[string]Service {
-	ss := map[string]Service{}
+	ss := make(map[string]Service, len(p.p.Services))
 	for id := range p.p.Services {
 		ss[id] = Service{
 			id: id,
@@ -419,7 +419,7 @@ func (s Service) Regions() map[string]Region {
 // A region is the AWS region the service exists in. Whereas a Endpoint is
 // an URL that can be resolved to a instance of a service.
 func (s Service) Endpoints() map[string]Endpoint {
-	es := map[string]Endpoint{}
+	es := make(map[string]Endpoint, len(s.p.Services[s.id].Endpoints))
 	for id := range s.p.Services[s.id].Endpoints {
 		es[id] = Endpoint{
 			id:        id,
--- a/vendor/github.com/aws/aws-sdk-go/aws/request/retryer.go
+++ b/vendor/github.com/aws/aws-sdk-go/aws/request/retryer.go
@@ -177,7 +177,7 @@ func shouldRetryError(origErr error) bool {
 		origErr := err.OrigErr()
 		var shouldRetry bool
 		if origErr != nil {
-			shouldRetry := shouldRetryError(origErr)
+			shouldRetry = shouldRetryError(origErr)
 			if err.Code() == "RequestError" && !shouldRetry {
 				return false
 			}
--- a/vendor/github.com/aws/aws-sdk-go/aws/version.go
+++ b/vendor/github.com/aws/aws-sdk-go/aws/version.go
@@ -5,4 +5,4 @@ package aws
 const SDKName = "aws-sdk-go"

 // SDKVersion is the version of this SDK
-const SDKVersion = "1.26.0"
+const SDKVersion = "1.26.8"
--- a/vendor/github.com/aws/aws-sdk-go/service/s3/api.go
+++ b/vendor/github.com/aws/aws-sdk-go/service/s3/api.go
@@ -347,7 +347,7 @@ func (c *S3) CopyObjectRequest(input *CopyObjectInput) (req *request.Request, ou
 // encryption key. You can do this regardless of the form of server-side encryption
 // that was used to encrypt the source, or even if the source object was not
 // encrypted. For more information about server-side encryption, see Using Server-Side
-// Encryption (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html).
+// Encryption (https://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html).
 //
 // A copy request might return an error when Amazon S3 receives the copy request
 // or while Amazon S3 is copying the files. If the error occurs before the copy
@@ -429,13 +429,18 @@ func (c *S3) CopyObjectRequest(input *CopyObjectInput) (req *request.Request, ou
 //    * To encrypt the target object using server-side encryption with an AWS
 //    managed encryption key, provide the following request headers, as appropriate.
 //    x-amz-server-side-encryption x-amz-server-side-encryption-aws-kms-key-id
-//    x-amz-server-side-encryption-context If you specify x-amz-server-side-encryption:aws:kms
-//    but don't provide x-amz-server-side- encryption-aws-kms-key-id, Amazon
-//    S3 uses the AWS managed customer master key (CMK) in AWS KMS to protect
-//    the data. All GET and PUT requests for an object protected by AWS KMS
-//    fail if you don't make them with SSL or by using SigV4. For more information
-//    about server-side encryption with CMKs stored in AWS KMS (SSE-KMS), see
-//    Protecting Data Using Server-Side Encryption with CMKs stored in KMS (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html).
+//    x-amz-server-side-encryption-context If you specify x-amz-server-side-encryption:aws:kms,
+//    but don't provide x-amz-server-side-encryption-aws-kms-key-id, Amazon
+//    S3 uses the AWS managed CMK in AWS KMS to protect the data. If you want
+//    to use a customer managed AWS KMS CMK, you must provide the x-amz-server-side-encryption-aws-kms-key-id
+//    of the symmetric customer managed CMK. Amazon S3 only supports symmetric
+//    CMKs and not asymmetric CMKs. For more information, see Using Symmetric
+//    and Asymmetric Keys (https://docs.aws.amazon.com/kms/latest/developerguide/symmetric-asymmetric.html)
+//    in the AWS Key Management Service Developer Guide. All GET and PUT requests
+//    for an object protected by AWS KMS fail if you don't make them with SSL
+//    or by using SigV4. For more information about server-side encryption with
+//    CMKs stored in AWS KMS (SSE-KMS), see Protecting Data Using Server-Side
+//    Encryption with CMKs stored in KMS (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html).
 //
 //    * To encrypt the target object using server-side encryption with an encryption
 //    key that you provide, use the following headers. x-amz-server-side-encryption-customer-algorithm
@@ -790,7 +795,7 @@ func (c *S3) CreateMultipartUploadRequest(input *CreateMultipartUploadInput) (re
 //    manage the keys used to encrypt data, specify the following headers in
 //    the request. x-amz-server-side-encryption x-amz-server-side-encryption-aws-kms-key-id
 //    x-amz-server-side-encryption-context If you specify x-amz-server-side-encryption:aws:kms,
-//    but don't provide x-amz-server-side- encryption-aws-kms-key-id, Amazon
+//    but don't provide x-amz-server-side-encryption-aws-kms-key-id, Amazon
 //    S3 uses the AWS managed CMK in AWS KMS to protect the data. All GET and
 //    PUT requests for an object protected by AWS KMS fail if you don't make
 //    them with SSL or by using SigV4. For more information about server-side
@@ -2026,7 +2031,7 @@ func (c *S3) DeleteObjectTaggingRequest(input *DeleteObjectTaggingInput) (req *r
 // DeleteObjectTagging API operation for Amazon Simple Storage Service.
 //
 // Removes the entire tag set from the specified object. For more information
-// about managing object tags, see Object Tagging (https://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html#MultiFactorAuthenticationDelete).
+// about managing object tags, see Object Tagging (https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html).
 //
 // To use this operation, you must have permission to perform the s3:DeleteObjectTagging
 // action.
@@ -6818,9 +6823,9 @@ func (c *S3) PutBucketEncryptionRequest(input *PutBucketEncryptionInput) (req *r
 // This implementation of the PUT operation uses the encryption subresource
 // to set the default encryption state of an existing bucket.
 //
-// This implementation of the PUT operation sets default encryption for a buckets
+// This implementation of the PUT operation sets default encryption for a bucket
 // using server-side encryption with Amazon S3-managed keys SSE-S3 or AWS KMS
-// customer master keys (CMKs) (SSE-KMS) bucket.
+// customer master keys (CMKs) (SSE-KMS).
 //
 // This operation requires AWS Signature Version 4. For more information, see
 // Authenticating Requests (AWS Signature Version 4) (sig-v4-authenticating-requests.html).
@@ -8454,19 +8459,24 @@ func (c *S3) PutObjectRequest(input *PutObjectInput) (req *request.Request, outp
 //    manage the keys used to encrypt data, specify the following headers in
 //    the request. x-amz-server-side-encryption x-amz-server-side-encryption-aws-kms-key-id
 //    x-amz-server-side-encryption-context If you specify x-amz-server-side-encryption:aws:kms,
-//    but don't provide x-amz-server-side- encryption-aws-kms-key-id, Amazon
-//    S3 uses the AWS managed CMK in AWS KMS to protect the data. All GET and
-//    PUT requests for an object protected by AWS KMS fail if you don't make
-//    them with SSL or by using SigV4. For more information about server-side
-//    encryption with CMKs stored in AWS KMS (SSE-KMS), see Protecting Data
-//    Using Server-Side Encryption with CMKs stored in AWS (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html).
+//    but don't provide x-amz-server-side-encryption-aws-kms-key-id, Amazon
+//    S3 uses the AWS managed CMK in AWS KMS to protect the data. If you want
+//    to use a customer managed AWS KMS CMK, you must provide the x-amz-server-side-encryption-aws-kms-key-id
+//    of the symmetric customer managed CMK. Amazon S3 only supports symmetric
+//    CMKs and not asymmetric CMKs. For more information, see Using Symmetric
+//    and Asymmetric Keys (https://docs.aws.amazon.com/kms/latest/developerguide/symmetric-asymmetric.html)
+//    in the AWS Key Management Service Developer Guide. All GET and PUT requests
+//    for an object protected by AWS KMS fail if you don't make them with SSL
+//    or by using SigV4. For more information about server-side encryption with
+//    CMKs stored in AWS KMS (SSE-KMS), see Protecting Data Using Server-Side
+//    Encryption with CMKs stored in AWS (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html).
 //
 //    * Use customer-provided encryption keys – If you want to manage your
 //    own encryption keys, provide all the following headers in the request.
 //    x-amz-server-side-encryption-customer-algorithm x-amz-server-side-encryption-customer-key
 //    x-amz-server-side-encryption-customer-key-MD5 For more information
 //    about server-side encryption with CMKs stored in KMS (SSE-KMS), see Protecting
-//    Data Using Server-Side Encryption with CMKs stored in AWS KMS (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html).
+//    Data Using Server-Side Encryption with CMKs stored in AWS (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html).
 //
 // Access-Control-List (ACL)-Specific Request Headers
 //
@@ -8517,8 +8527,13 @@ func (c *S3) PutObjectRequest(input *PutObjectInput) (req *request.Request, outp
 //    manage the keys used to encrypt data, specify the following headers in
 //    the request. x-amz-server-side-encryption x-amz-server-side-encryption-aws-kms-key-id
 //    x-amz-server-side-encryption-context If you specify x-amz-server-side-encryption:aws:kms,
-//    but don't provide x-amz-server-side- encryption-aws-kms-key-id, Amazon
-//    S3 uses the default AWS KMS CMK to protect the data. All GET and PUT requests
+//    but don't provide x-amz-server-side-encryption-aws-kms-key-id, Amazon
+//    S3 uses the AWS managed CMK in AWS KMS to protect the data. If you want
+//    to use a customer managed AWS KMS CMK, you must provide the x-amz-server-side-encryption-aws-kms-key-id
+//    of the symmetric customer managed CMK. Amazon S3 only supports symmetric
+//    CMKs and not asymmetric CMKs. For more information, see Using Symmetric
+//    and Asymmetric Keys (https://docs.aws.amazon.com/kms/latest/developerguide/symmetric-asymmetric.html)
+//    in the AWS Key Management Service Developer Guide. All GET and PUT requests
 //    for an object protected by AWS KMS fail if you don't make them with SSL
 //    or by using SigV4. For more information about server-side encryption with
 //    CMKs stored in AWS KMS (SSE-KMS), see Protecting Data Using Server-Side
@@ -9967,10 +9982,10 @@ type AbortMultipartUploadInput struct {
 	// Key is a required field
 	Key *string `location:"uri" locationName:"Key" min:"1" type:"string" required:"true"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -11073,10 +11088,10 @@ type CompleteMultipartUploadInput struct {
 	// The container for the multipart upload request information.
 	MultipartUpload *CompletedMultipartUpload `locationName:"CompleteMultipartUpload" type:"structure" xmlURI:"http://s3.amazonaws.com/doc/2006-03-01/"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -11201,7 +11216,8 @@ type CompleteMultipartUploadOutput struct {
 	RequestCharged *string `location:"header" locationName:"x-amz-request-charged" type:"string" enum:"RequestCharged"`

 	// If present, specifies the ID of the AWS Key Management Service (AWS KMS)
-	// customer master key (CMK) that was used for the object.
+	// symmetric customer managed customer master key (CMK) that was used for the
+	// object.
 	SSEKMSKeyId *string `location:"header" locationName:"x-amz-server-side-encryption-aws-kms-key-id" type:"string" sensitive:"true"`

 	// If you specified server-side encryption either with an Amazon S3-managed
@@ -11513,10 +11529,10 @@ type CopyObjectInput struct {
 	// The date and time when you want the copied object's Object Lock to expire.
 	ObjectLockRetainUntilDate *time.Time `location:"header" locationName:"x-amz-object-lock-retain-until-date" type:"timestamp" timestampFormat:"iso8601"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -11545,7 +11561,7 @@ type CopyObjectInput struct {
 	// requests for an object protected by AWS KMS will fail if not made via SSL
 	// or using SigV4. For information about configuring using any of the officially
 	// supported AWS SDKs and AWS CLI, see Specifying the Signature Version in Request
-	// Authentication (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/UsingAWSSDK.html#specify-signature-version)
+	// Authentication (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingAWSSDK.html#specify-signature-version)
 	// in the Amazon S3 Developer Guide.
 	SSEKMSKeyId *string `location:"header" locationName:"x-amz-server-side-encryption-aws-kms-key-id" type:"string" sensitive:"true"`

@@ -11895,7 +11911,8 @@ type CopyObjectOutput struct {
 	SSEKMSEncryptionContext *string `location:"header" locationName:"x-amz-server-side-encryption-context" type:"string" sensitive:"true"`

 	// If present, specifies the ID of the AWS Key Management Service (AWS KMS)
-	// customer master key (CMK) that was used for the object.
+	// symmetric customer managed customer master key (CMK) that was used for the
+	// object.
 	SSEKMSKeyId *string `location:"header" locationName:"x-amz-server-side-encryption-aws-kms-key-id" type:"string" sensitive:"true"`

 	// The server-side encryption algorithm used when storing this object in Amazon
@@ -12275,10 +12292,10 @@ type CreateMultipartUploadInput struct {
 	// Specifies the date and time when you want the Object Lock to expire.
 	ObjectLockRetainUntilDate *time.Time `location:"header" locationName:"x-amz-object-lock-retain-until-date" type:"timestamp" timestampFormat:"iso8601"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -12303,11 +12320,11 @@ type CreateMultipartUploadInput struct {
 	// encryption context key-value pairs.
 	SSEKMSEncryptionContext *string `location:"header" locationName:"x-amz-server-side-encryption-context" type:"string" sensitive:"true"`

-	// Specifies the AWS KMS key ID to use for object encryption. All GET and PUT
-	// requests for an object protected by AWS KMS will fail if not made via SSL
-	// or using SigV4. For information about configuring using any of the officially
-	// supported AWS SDKs and AWS CLI, see Specifying the Signature Version in Request
-	// Authentication (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/UsingAWSSDK.html#specify-signature-version)
+	// Specifies the ID of the symmetric customer managed AWS KMS CMK to use for
+	// object encryption. All GET and PUT requests for an object protected by AWS
+	// KMS will fail if not made via SSL or using SigV4. For information about configuring
+	// using any of the officially supported AWS SDKs and AWS CLI, see Specifying
+	// the Signature Version in Request Authentication (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/UsingAWSSDK.html#specify-signature-version)
 	// in the Amazon S3 Developer Guide.
 	SSEKMSKeyId *string `location:"header" locationName:"x-amz-server-side-encryption-aws-kms-key-id" type:"string" sensitive:"true"`

@@ -12601,7 +12618,8 @@ type CreateMultipartUploadOutput struct {
 	SSEKMSEncryptionContext *string `location:"header" locationName:"x-amz-server-side-encryption-context" type:"string" sensitive:"true"`

 	// If present, specifies the ID of the AWS Key Management Service (AWS KMS)
-	// customer master key (CMK) that was used for the object.
+	// symmetric customer managed customer master key (CMK) that was used for the
+	// object.
 	SSEKMSKeyId *string `location:"header" locationName:"x-amz-server-side-encryption-aws-kms-key-id" type:"string" sensitive:"true"`

 	// The server-side encryption algorithm used when storing this object in Amazon
@@ -13805,10 +13823,10 @@ type DeleteObjectInput struct {
 	// delete enabled.
 	MFA *string `location:"header" locationName:"x-amz-mfa" type:"string"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -14098,10 +14116,10 @@ type DeleteObjectsInput struct {
 	// delete enabled.
 	MFA *string `location:"header" locationName:"x-amz-mfa" type:"string"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`
 }
@@ -14517,8 +14535,11 @@ type Encryption struct {
 	// the encryption context for the restore results.
 	KMSContext *string `type:"string"`

-	// If the encryption type is aws:kms, this optional value specifies the AWS
-	// KMS key ID to use for encryption of job results.
+	// If the encryption type is aws:kms, this optional value specifies the ID of
+	// the symmetric customer managed AWS KMS CMK to use for encryption of job results.
+	// Amazon S3 only supports symmetric CMKs. For more information, see Using Symmetric
+	// and Asymmetric Keys (https://docs.aws.amazon.com/kms/latest/developerguide/symmetric-asymmetric.html)
+	// in the AWS Key Management Service Developer Guide.
 	KMSKeyId *string `type:"string" sensitive:"true"`
 }

@@ -14568,8 +14589,12 @@ func (s *Encryption) SetKMSKeyId(v string) *Encryption {
 type EncryptionConfiguration struct {
 	_ struct{} `type:"structure"`

-	// Specifies the AWS KMS Key ID (Key ARN or Alias ARN) for the destination bucket.
-	// Amazon S3 uses this key to encrypt replica objects.
+	// Specifies the ID (Key ARN or Alias ARN) of the customer managed customer
+	// master key (CMK) stored in AWS Key Management Service (KMS) for the destination
+	// bucket. Amazon S3 uses this key to encrypt replica objects. Amazon S3 only
+	// supports symmetric customer managed CMKs. For more information, see Using
+	// Symmetric and Asymmetric Keys (https://docs.aws.amazon.com/kms/latest/developerguide/symmetric-asymmetric.html)
+	// in the AWS Key Management Service Developer Guide.
 	ReplicaKmsKeyID *string `type:"string"`
 }

@@ -16854,10 +16879,10 @@ type GetObjectAclInput struct {
 	// Key is a required field
 	Key *string `location:"uri" locationName:"Key" min:"1" type:"string" required:"true"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -17029,10 +17054,10 @@ type GetObjectInput struct {
 	// the HTTP Range header, see http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35.
 	Range *string `location:"header" locationName:"Range" type:"string"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -17269,10 +17294,10 @@ type GetObjectLegalHoldInput struct {
 	// Key is a required field
 	Key *string `location:"uri" locationName:"Key" min:"1" type:"string" required:"true"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -17562,7 +17587,8 @@ type GetObjectOutput struct {
 	SSECustomerKeyMD5 *string `location:"header" locationName:"x-amz-server-side-encryption-customer-key-MD5" type:"string"`

 	// If present, specifies the ID of the AWS Key Management Service (AWS KMS)
-	// customer master key (CMK) that was used for the object.
+	// symmetric customer managed customer master key (CMK) that was used for the
+	// object.
 	SSEKMSKeyId *string `location:"header" locationName:"x-amz-server-side-encryption-aws-kms-key-id" type:"string" sensitive:"true"`

 	// The server-side encryption algorithm used when storing this object in Amazon
@@ -17802,10 +17828,10 @@ type GetObjectRetentionInput struct {
 	// Key is a required field
 	Key *string `location:"uri" locationName:"Key" min:"1" type:"string" required:"true"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -18056,10 +18082,10 @@ type GetObjectTorrentInput struct {
 	// Key is a required field
 	Key *string `location:"uri" locationName:"Key" min:"1" type:"string" required:"true"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`
 }
@@ -18532,10 +18558,10 @@ type HeadObjectInput struct {
 	// the HTTP Range header, see http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35.
 	Range *string `location:"header" locationName:"Range" type:"string"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -18828,7 +18854,8 @@ type HeadObjectOutput struct {
 	SSECustomerKeyMD5 *string `location:"header" locationName:"x-amz-server-side-encryption-customer-key-MD5" type:"string"`

 	// If present, specifies the ID of the AWS Key Management Service (AWS KMS)
-	// customer master key (CMK) that was used for the object.
+	// symmetric customer managed customer master key (CMK) that was used for the
+	// object.
 	SSEKMSKeyId *string `location:"header" locationName:"x-amz-server-side-encryption-aws-kms-key-id" type:"string" sensitive:"true"`

 	// If the object is stored using server-side encryption either with an AWS KMS
@@ -21676,10 +21703,10 @@ type ListPartsInput struct {
 	// part numbers will be listed.
 	PartNumberMarker *int64 `location:"querystring" locationName:"part-number-marker" type:"integer"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -25272,10 +25299,10 @@ type PutObjectAclInput struct {
 	// Key is a required field
 	Key *string `location:"uri" locationName:"Key" min:"1" type:"string" required:"true"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -25522,10 +25549,10 @@ type PutObjectInput struct {
 	// The date and time when you want this object's Object Lock to expire.
 	ObjectLockRetainUntilDate *time.Time `location:"header" locationName:"x-amz-object-lock-retain-until-date" type:"timestamp" timestampFormat:"iso8601"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -25552,12 +25579,14 @@ type PutObjectInput struct {

 	// If x-amz-server-side-encryption is present and has the value of aws:kms,
 	// this header specifies the ID of the AWS Key Management Service (AWS KMS)
-	// customer master key (CMK) that was used for the object.
+	// symmetrical customer managed customer master key (CMK) that was used for
+	// the object.
 	//
 	// If the value of x-amz-server-side-encryption is aws:kms, this header specifies
-	// the ID of the AWS KMS CMK that will be used for the object. If you specify
-	// x-amz-server-side-encryption:aws:kms, but do not providex-amz-server-side-encryption-aws-kms-key-id,
-	// Amazon S3 uses the AWS managed CMK in AWS to protect the data.
+	// the ID of the symmetric customer managed AWS KMS CMK that will be used for
+	// the object. If you specify x-amz-server-side-encryption:aws:kms, but do not
+	// providex-amz-server-side-encryption-aws-kms-key-id, Amazon S3 uses the AWS
+	// managed CMK in AWS to protect the data.
 	SSEKMSKeyId *string `location:"header" locationName:"x-amz-server-side-encryption-aws-kms-key-id" type:"string" sensitive:"true"`

 	// The server-side encryption algorithm used when storing this object in Amazon
@@ -25858,10 +25887,10 @@ type PutObjectLegalHoldInput struct {
 	// specified object.
 	LegalHold *ObjectLockLegalHold `locationName:"LegalHold" type:"structure" xmlURI:"http://s3.amazonaws.com/doc/2006-03-01/"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -25987,10 +26016,10 @@ type PutObjectLockConfigurationInput struct {
 	// The Object Lock configuration that you want to apply to the specified bucket.
 	ObjectLockConfiguration *ObjectLockConfiguration `locationName:"ObjectLockConfiguration" type:"structure" xmlURI:"http://s3.amazonaws.com/doc/2006-03-01/"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -26126,7 +26155,8 @@ type PutObjectOutput struct {

 	// If x-amz-server-side-encryption is present and has the value of aws:kms,
 	// this header specifies the ID of the AWS Key Management Service (AWS KMS)
-	// customer master key (CMK) that was used for the object.
+	// symmetric customer managed customer master key (CMK) that was used for the
+	// object.
 	SSEKMSKeyId *string `location:"header" locationName:"x-amz-server-side-encryption-aws-kms-key-id" type:"string" sensitive:"true"`

 	// If you specified server-side encryption either with an AWS KMS customer master
@@ -26228,10 +26258,10 @@ type PutObjectRetentionInput struct {
 	// Key is a required field
 	Key *string `location:"uri" locationName:"Key" min:"1" type:"string" required:"true"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -27426,10 +27456,10 @@ type RestoreObjectInput struct {
 	// Key is a required field
 	Key *string `location:"uri" locationName:"Key" min:"1" type:"string" required:"true"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -27849,8 +27879,8 @@ func (s *Rule) SetTransition(v *Transition) *Rule {
 type SSEKMS struct {
 	_ struct{} `locationName:"SSE-KMS" type:"structure"`

-	// Specifies the ID of the AWS Key Management Service (AWS KMS) customer master
-	// key (CMK) to use for encrypting inventory reports.
+	// Specifies the ID of the AWS Key Management Service (AWS KMS) symmetric customer
+	// managed customer master key (CMK) to use for encrypting inventory reports.
 	//
 	// KeyId is a required field
 	KeyId *string `type:"string" required:"true" sensitive:"true"`
@@ -29305,10 +29335,10 @@ type UploadPartCopyInput struct {
 	// PartNumber is a required field
 	PartNumber *int64 `location:"querystring" locationName:"partNumber" type:"integer" required:"true"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -29538,7 +29568,8 @@ type UploadPartCopyOutput struct {
 	SSECustomerKeyMD5 *string `location:"header" locationName:"x-amz-server-side-encryption-customer-key-MD5" type:"string"`

 	// If present, specifies the ID of the AWS Key Management Service (AWS KMS)
-	// customer master key (CMK) that was used for the object.
+	// symmetric customer managed customer master key (CMK) that was used for the
+	// object.
 	SSEKMSKeyId *string `location:"header" locationName:"x-amz-server-side-encryption-aws-kms-key-id" type:"string" sensitive:"true"`

 	// The server-side encryption algorithm used when storing this object in Amazon
@@ -29629,10 +29660,10 @@ type UploadPartInput struct {
 	// PartNumber is a required field
 	PartNumber *int64 `location:"querystring" locationName:"partNumber" type:"integer" required:"true"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -29812,7 +29843,7 @@ type UploadPartOutput struct {
 	SSECustomerKeyMD5 *string `location:"header" locationName:"x-amz-server-side-encryption-customer-key-MD5" type:"string"`

 	// If present, specifies the ID of the AWS Key Management Service (AWS KMS)
-	// customer master key (CMK) was used for the object.
+	// symmetric customer managed customer master key (CMK) was used for the object.
 	SSEKMSKeyId *string `location:"header" locationName:"x-amz-server-side-encryption-aws-kms-key-id" type:"string" sensitive:"true"`

 	// The server-side encryption algorithm used when storing this object in Amazon
@@ -30461,10 +30492,10 @@ const (
 	RequestChargedRequester = "requester"
 )

-// Confirms that the requester knows that she or he will be charged for the
-// request. Bucket owners need not specify this parameter in their requests.
-// For information about downloading objects from Requester Pays buckets, see
-// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+// Confirms that the requester knows that they will be charged for the request.
+// Bucket owners need not specify this parameter in their requests. For information
+// about downloading objects from requester pays buckets, see Downloading Objects
+// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 // in the Amazon S3 Developer Guide.
 const (
 	// RequestPayerRequester is a RequestPayer enum value
--- a/vendor/github.com/aws/aws-sdk-go/service/s3/s3manager/upload.go
+++ b/vendor/github.com/aws/aws-sdk-go/service/s3/s3manager/upload.go
@@ -397,14 +397,18 @@ func (u *uploader) init() error {
 		u.cfg.MaxUploadParts = MaxUploadParts
 	}

+	// Try to get the total size for some optimizations
+	if err := u.initSize(); err != nil {
+		return err
+	}
+
 	// If PartSize was changed or partPool was never setup then we need to allocated a new pool
 	// so that we return []byte slices of the correct size
 	if u.cfg.partPool == nil || u.cfg.partPool.partSize != u.cfg.PartSize {
 		u.cfg.partPool = newPartPool(u.cfg.PartSize)
 	}

-	// Try to get the total size for some optimizations
-	return u.initSize()
+	return nil
 }

 // initSize tries to detect the total stream size, setting u.totalSize. If
--- a/vendor/github.com/aws/aws-sdk-go/service/s3/s3manager/upload_input.go
+++ b/vendor/github.com/aws/aws-sdk-go/service/s3/s3manager/upload_input.go
@@ -97,10 +97,10 @@ type UploadInput struct {
 	// The date and time when you want this object's Object Lock to expire.
 	ObjectLockRetainUntilDate *time.Time `location:"header" locationName:"x-amz-object-lock-retain-until-date" type:"timestamp" timestampFormat:"iso8601"`

-	// Confirms that the requester knows that she or he will be charged for the
-	// request. Bucket owners need not specify this parameter in their requests.
-	// For information about downloading objects from Requester Pays buckets, see
-	// Downloading Objects in Requestor Pays Buckets (https://docs.aws.amazon.com/http:/docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
+	// Confirms that the requester knows that they will be charged for the request.
+	// Bucket owners need not specify this parameter in their requests. For information
+	// about downloading objects from requester pays buckets, see Downloading Objects
+	// in Requestor Pays Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html)
 	// in the Amazon S3 Developer Guide.
 	RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string" enum:"RequestPayer"`

@@ -127,12 +127,14 @@ type UploadInput struct {

 	// If x-amz-server-side-encryption is present and has the value of aws:kms,
 	// this header specifies the ID of the AWS Key Management Service (AWS KMS)
-	// customer master key (CMK) that was used for the object.
+	// symmetrical customer managed customer master key (CMK) that was used for
+	// the object.
 	//
 	// If the value of x-amz-server-side-encryption is aws:kms, this header specifies
-	// the ID of the AWS KMS CMK that will be used for the object. If you specify
-	// x-amz-server-side-encryption:aws:kms, but do not providex-amz-server-side-encryption-aws-kms-key-id,
-	// Amazon S3 uses the AWS managed CMK in AWS to protect the data.
+	// the ID of the symmetric customer managed AWS KMS CMK that will be used for
+	// the object. If you specify x-amz-server-side-encryption:aws:kms, but do not
+	// providex-amz-server-side-encryption-aws-kms-key-id, Amazon S3 uses the AWS
+	// managed CMK in AWS to protect the data.
 	SSEKMSKeyId *string `location:"header" locationName:"x-amz-server-side-encryption-aws-kms-key-id" type:"string" sensitive:"true"`

 	// The server-side encryption algorithm used when storing this object in Amazon
--- a/vendor/github.com/valyala/fastjson/README.md
+++ b/vendor/github.com/valyala/fastjson/README.md
@@ -102,6 +102,21 @@ See also [examples](https://godoc.org/github.com/valyala/fastjson#pkg-examples).
  * Prefer iterating over array returned from [Value.GetArray](https://godoc.org/github.com/valyala/fastjson#Object.Visit)
    with a range loop instead of calling `Value.Get*` for each array item.

+## Fuzzing
+Install [go-fuzz](https://github.com/dvyukov/go-fuzz) & optionally the go-fuzz-corpus.
+
+```bash
+go get -u github.com/dvyukov/go-fuzz/go-fuzz github.com/dvyukov/go-fuzz/go-fuzz-build
+```
+
+Build using `go-fuzz-build` and run `go-fuzz` with an optional corpus.
+
+```bash
+mkdir -p workdir/corpus
+cp $GOPATH/src/github.com/dvyukov/go-fuzz-corpus/json/corpus/* workdir/corpus
+go-fuzz-build github.com/valyala/fastjson
+go-fuzz -bin=fastjson-fuzz.zip -workdir=workdir
+```

 ## Benchmarks

--- a/vendor/github.com/valyala/fastjson/fastfloat/parse.go
+++ b/vendor/github.com/valyala/fastjson/fastfloat/parse.go
@@ -2,6 +2,7 @@ package fastfloat

 import (
 	"math"
+	"strings"
 	"strconv"
 )

@@ -132,6 +133,15 @@ func ParseBestEffort(s string) float64 {
 		break
 	}
 	if i <= j {
+		if strings.EqualFold(s[i:], "inf") {
+			if minus {
+				return -inf
+			}
+			return inf
+		}
+		if strings.EqualFold(s[i:], "nan") {
+			return nan
+		}
 		return 0
 	}
 	f := float64(d)
@@ -229,3 +239,6 @@ func ParseBestEffort(s string) float64 {
 	}
 	return 0
 }
+
+var inf = math.Inf(1)
+var nan = math.NaN()
--- a/vendor/github.com/valyala/fastjson/fuzz.go
+++ b/vendor/github.com/valyala/fastjson/fuzz.go
@@ -0,0 +1,22 @@
+// +build gofuzz
+
+package fastjson
+
+func Fuzz(data []byte) int {
+	err := ValidateBytes(data)
+	if err != nil {
+		return 0
+	}
+
+	v := MustParseBytes(data)
+
+	dst := make([]byte, 0)
+	dst = v.MarshalTo(dst)
+
+	err = ValidateBytes(dst)
+	if err != nil {
+		panic(err)
+	}
+
+	return 1
+}
--- a/vendor/github.com/valyala/fastjson/handy.go
+++ b/vendor/github.com/valyala/fastjson/handy.go
@@ -159,7 +159,7 @@ func ParseBytes(b []byte) (*Value, error) {

 // MustParseBytes parses b containing json.
 //
-// The function banics if b cannot be parsed.
+// The function panics if b cannot be parsed.
 // The function is slower than the Parser.ParseBytes for re-used Parser.
 func MustParseBytes(b []byte) *Value {
 	v, err := ParseBytes(b)
--- a/vendor/github.com/valyala/fastjson/parser.go
+++ b/vendor/github.com/valyala/fastjson/parser.go
@@ -138,6 +138,13 @@ func parseValue(s string, c *cache) (*Value, string, error) {
 	}
 	if s[0] == 'n' {
 		if len(s) < len("null") || s[:len("null")] != "null" {
+			// Try parsing NaN
+			if len(s) >= 3 && strings.EqualFold(s[:3], "nan") {
+				v := c.getValue()
+				v.t = TypeNumber
+				v.s = s[:3]
+				return v, s[3:], nil
+			}
 			return nil, s, fmt.Errorf("unexpected value found: %q", s)
 		}
 		return valueNull, s[len("null"):], nil
@@ -415,7 +422,13 @@ func parseRawNumber(s string) (string, string, error) {
 		if (ch >= '0' && ch <= '9') || ch == '.' || ch == '-' || ch == 'e' || ch == 'E' || ch == '+' {
 			continue
 		}
-		if i == 0 {
+		if i == 0 || i == 1 && (s[0] == '-' || s[0] == '+') {
+			if len(s[i:]) >= 3 {
+				xs := s[i : i+3]
+				if strings.EqualFold(xs, "inf") || strings.EqualFold(xs, "nan") {
+					return s[:i+3], s[i+3:], nil
+				}
+			}
 			return "", s, fmt.Errorf("unexpected char: %q", s[:1])
 		}
 		ns := s[:i]
@@ -484,8 +497,9 @@ func (o *Object) unescapeKeys() {
 	if o.keysUnescaped {
 		return
 	}
-	for i := range o.kvs {
-		kv := &o.kvs[i]
+	kvs := o.kvs
+	for i := range kvs {
+		kv := &kvs[i]
 		kv.k = unescapeStringBestEffort(kv.k)
 	}
 	o.keysUnescaped = true
--- a/vendor/github.com/valyala/gozstd/gozstd.go
+++ b/vendor/github.com/valyala/gozstd/gozstd.go
@@ -122,9 +122,7 @@ func compress(cctx, cctxDict *cctxWrapper, dst, src []byte, cd *CDict, compressi
 	dstLen := len(dst)
 	if cap(dst) > dstLen {
 		// Fast path - try compressing without dst resize.
-		dst = dst[:cap(dst)]
-
-		result := compressInternal(cctx, cctxDict, dst[dstLen:], src, cd, compressionLevel, false)
+		result := compressInternal(cctx, cctxDict, dst[dstLen:cap(dst)], src, cd, compressionLevel, false)
 		compressedSize := int(result)
 		if compressedSize >= 0 {
 			// All OK.
@@ -143,9 +141,8 @@ func compress(cctx, cctxDict *cctxWrapper, dst, src []byte, cd *CDict, compressi
 		// This should be optimized since go 1.11 - see https://golang.org/doc/go1.11#performance-compiler.
 		dst = append(dst[:cap(dst)], make([]byte, n)...)
 	}
-	dst = dst[:dstLen+compressBound]

-	result := compressInternal(cctx, cctxDict, dst[dstLen:], src, cd, compressionLevel, true)
+	result := compressInternal(cctx, cctxDict, dst[dstLen:dstLen+compressBound], src, cd, compressionLevel, true)
 	compressedSize := int(result)
 	return dst[:dstLen+compressedSize]
 }
@@ -247,9 +244,7 @@ func decompress(dctx, dctxDict *dctxWrapper, dst, src []byte, dd *DDict) ([]byte
 	dstLen := len(dst)
 	if cap(dst) > dstLen {
 		// Fast path - try decompressing without dst resize.
-		dst = dst[:cap(dst)]
-
-		result := decompressInternal(dctx, dctxDict, dst[dstLen:], src, dd)
+		result := decompressInternal(dctx, dctxDict, dst[dstLen:cap(dst)], src, dd)
 		decompressedSize := int(result)
 		if decompressedSize >= 0 {
 			// All OK.
@@ -279,9 +274,8 @@ func decompress(dctx, dctxDict *dctxWrapper, dst, src []byte, dd *DDict) ([]byte
 		// This should be optimized since go 1.11 - see https://golang.org/doc/go1.11#performance-compiler.
 		dst = append(dst[:cap(dst)], make([]byte, n)...)
 	}
-	dst = dst[:dstLen+decompressBound]

-	result := decompressInternal(dctx, dctxDict, dst[dstLen:], src, dd)
+	result := decompressInternal(dctx, dctxDict, dst[dstLen:dstLen+decompressBound], src, dd)
 	decompressedSize := int(result)
 	if decompressedSize >= 0 {
 		// All OK.
--- a/vendor/golang.org/x/sys/unix/fcntl.go
+++ b/vendor/golang.org/x/sys/unix/fcntl.go
@@ -9,12 +9,11 @@ package unix
 import "unsafe"

 // fcntl64Syscall is usually SYS_FCNTL, but is overridden on 32-bit Linux
-// systems by flock_linux_32bit.go to be SYS_FCNTL64.
+// systems by fcntl_linux_32bit.go to be SYS_FCNTL64.
 var fcntl64Syscall uintptr = SYS_FCNTL

-// FcntlInt performs a fcntl syscall on fd with the provided command and argument.
-func FcntlInt(fd uintptr, cmd, arg int) (int, error) {
-	valptr, _, errno := Syscall(fcntl64Syscall, fd, uintptr(cmd), uintptr(arg))
+func fcntl(fd int, cmd, arg int) (int, error) {
+	valptr, _, errno := Syscall(fcntl64Syscall, uintptr(fd), uintptr(cmd), uintptr(arg))
 	var err error
 	if errno != 0 {
 		err = errno
@@ -22,6 +21,11 @@ func FcntlInt(fd uintptr, cmd, arg int) (int, error) {
 	return int(valptr), err
 }

+// FcntlInt performs a fcntl syscall on fd with the provided command and argument.
+func FcntlInt(fd uintptr, cmd, arg int) (int, error) {
+	return fcntl(int(fd), cmd, arg)
+}
+
 // FcntlFlock performs a fcntl syscall for the F_GETLK, F_SETLK or F_SETLKW command.
 func FcntlFlock(fd uintptr, cmd int, lk *Flock_t) error {
 	_, _, errno := Syscall(fcntl64Syscall, fd, uintptr(cmd), uintptr(unsafe.Pointer(lk)))
--- a/vendor/golang.org/x/sys/unix/mkerrors.sh
+++ b/vendor/golang.org/x/sys/unix/mkerrors.sh
@@ -186,6 +186,7 @@ struct ltchars {
 #include <sys/select.h>
 #include <sys/signalfd.h>
 #include <sys/socket.h>
+#include <sys/uio.h>
 #include <sys/xattr.h>
 #include <linux/bpf.h>
 #include <linux/can.h>
@@ -526,6 +527,7 @@ ccflags="$@"
 		$2 ~ /^WDIOC_/ ||
 		$2 ~ /^NFN/ ||
 		$2 ~ /^XDP_/ ||
+		$2 ~ /^RWF_/ ||
 		$2 ~ /^(HDIO|WIN|SMART)_/ ||
 		$2 ~ /^CRYPTO_/ ||
 		$2 ~ /^TIPC_/ ||
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Aliaksandr Valialkin	46bfdbe6cf	app/vmselect/promql: do not take into account the previous point before time window in square brackets for `min_over_time`, `max_over_time`, `rollup_first` and `rollup_last` functions This makes the behaviour for these functions similar to Prometheus when processing broken time series with irregular data points like `gitlab_runner_jobs`. See https://gitlab.com/gitlab-org/gitlab-exporter/issues/50 for details.	2020-01-11 00:26:26 +02:00
Aliaksandr Valialkin	4f0a645f77	vendor: update github.com/valyala/fastjson from v1.4.2 to v1.4.5 This should fix parsing Inf values in `/api/v1/import`. The previous attempt to fix this in VictoriaMetrics v1.32.1 was unsuccessful.	2020-01-10 23:15:15 +02:00
Aliaksandr Valialkin	b829fe5e39	app/vmselect/promql: properly handle `aggr(aggr_over_time(...))`	2020-01-10 21:57:18 +02:00
Aliaksandr Valialkin	164278151f	app/vmselect/promql: add `aggr_over_time(("aggr_func1", "aggr_func2", ...), m[d])` function This function can be used for simultaneous calculating of multiple `aggr_func*` functions that accept range vector. For example, `aggr_over_time(("min_over_time", "max_over_time"), m[d])` would calculate `min_over_time` and `max_over_time` for `m[d]`.	2020-01-10 21:18:06 +02:00
Aliaksandr Valialkin	c4632faa9d	app/vmselect/promql: add `tmin_over_time(m[d])` and `tmax_over_time(m[d])` functions These functions return timestamp in seconds for the minimum and maximum value for `m` over time range `d`	2020-01-10 19:39:28 +02:00
Aliaksandr Valialkin	a768198814	docs: fix spelling typos	2020-01-09 23:42:55 +02:00
Roman Khavronenko	57f4875024	fix spellcheck issues (#285 )	2020-01-09 23:41:52 +02:00
Aliaksandr Valialkin	b8038a14e7	lib/backup/s3remote: check whether the file exists before deleting it Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/284	2020-01-09 23:20:31 +02:00
Aliaksandr Valialkin	f358fb72d1	app/{vmbackup,vmrestore}: add `backup complete` file to backup when it is complete and check for this file before restoring from backup This should prevent from restoring from incomplete backups. Add `-skipBackupCompleteCheck` command-line flag to `vmrestore` in order to be able restoring from old backups without `backup complete` file.	2020-01-09 15:35:38 +02:00
Aliaksandr Valialkin	1c436b2723	vendor: update github.com/valyala/fastjson from v1.4.1 to v1.4.2 This fixes parsing of `inf` and `nan` values in json lines passed to `/api/v1/import`	2020-01-08 20:47:21 +02:00
Aliaksandr Valialkin	a973df6d79	README.md: remove `height="200px"` from logo image, since it is improperly displayed on smartphones	2020-01-08 20:29:11 +02:00
Aliaksandr Valialkin	d4132a6915	docs: typo fix	2020-01-08 14:45:27 +02:00
Aliaksandr Valialkin	d5aeda0e1a	app/vmselect/promql: skip `rate` calculation for the first point on time series	2020-01-08 14:42:53 +02:00
Aliaksandr Valialkin	bb71b6d47d	docs: add references to `Remote Write Storage Wars` Also mention than VictoriaMetrics uses less RAM than Thanos Store Gateway - see https://github.com/thanos-io/thanos/issues/448 for details.	2020-01-04 23:57:35 +02:00
Aliaksandr Valialkin	fc71602039	lib/storage: limit maxRaRowsPerPartition by 500K for any number of rawRowsShardsPerPartition This should reduce write amplification for high ingestion rate on multi-CPU systems	2020-01-04 23:57:31 +02:00
Aliaksandr Valialkin	c60fdbed30	docs/CaseStudies.md: add link to `Remote Write Storage Wars` talk from Adidas at PromCon 2019	2020-01-04 16:51:45 +02:00
Aliaksandr Valialkin	d410c78c7e	app/vmselect/promql: fix calculations for `histogram_share`	2020-01-04 14:44:48 +02:00
Aliaksandr Valialkin	66f3d1dac8	README.md: update `Alerting` section	2020-01-04 13:55:09 +02:00
Aliaksandr Valialkin	d9c4ac9978	lib/metricsql: export IsRollupFunc and IsTransformFunc, since they can be used by package users	2020-01-04 13:25:05 +02:00
Aliaksandr Valialkin	4567a59fa0	LICENSE: update year	2020-01-04 13:21:04 +02:00
Aliaksandr Valialkin	d64699bb9f	app/vmselect/promql: add missing MetricName into netstorage.Result in tests	2020-01-04 12:52:39 +02:00
Aliaksandr Valialkin	f409f2d050	app/vmselect/promql: add `histogram_share(le, buckets)` function	2020-01-04 12:45:55 +02:00
Aliaksandr Valialkin	b1ded7cf9a	app/vmselect/promql: add `absent_over_time(m[d])` func similar to the function in Prometheus 2.16 See https://github.com/prometheus/prometheus/issues/2882	2020-01-04 12:45:07 +02:00
Aliaksandr Valialkin	a8360d04c0	app/vmselect/promql: add `histogram_over_time(m[d])` rollup function	2020-01-04 12:44:56 +02:00
Aliaksandr Valialkin	3e09d38f29	app/vmselect/promql: fix results caching for multi-arg rollup functions such as `quantile_over_time` Previosly only a single arg was taken into account, so caching didn't work properly for multi-arg rollup funcs.	2020-01-03 20:49:08 +02:00
Aliaksandr Valialkin	a774120460	app/vmselect/promql: use scrapeInterval instead of window in denominator when calculating `rate` for the first point on the time series This should provide better estimation for `rate` in the beginning of time series.	2020-01-03 19:01:50 +02:00
Aliaksandr Valialkin	695682232f	lib/uint64set: reduce memory usage when storing big number of sparse metric_id values	2020-01-03 18:16:44 +02:00
Aliaksandr Valialkin	b5645ccbdf	app/vmselect/promql: increase the estimated number of time series returned by `aggr() by (something)` from 100 to 1K, since 100 may result in OOM for high number of time series	2020-01-03 01:02:21 +02:00
Aliaksandr Valialkin	cb3a342882	app/vmselect/promql: add `share_le_over_time` and `share_gt_over_time` functions for SLI and SLO calculations	2020-01-03 00:41:16 +02:00
Aliaksandr Valialkin	0038365206	docs: refer to standalone MetricsQL package	2020-01-02 23:43:35 +02:00
Aliaksandr Valialkin	61c9d320ed	vendor: update github.com/VictoriaMetrics/fastcache from v1.5.4 to v1.5.5	2019-12-29 18:17:49 +02:00
Aliaksandr Valialkin	a21d786d3c	lib/metricsql: add example for ExpandWithExprs	2019-12-26 21:32:11 +02:00
Aliaksandr Valialkin	192b51c246	vendor: `make vendor-update`	2019-12-26 19:41:02 +02:00
Aliaksandr Valialkin	17a4dc9782	vendor: update github.com/valyala/gozstd from v1.6.3 to v1.6.4 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/281	2019-12-26 19:30:08 +02:00
Aliaksandr Valialkin	6f67e0b56b	lib/metricsq: add ExpandWithExprs	2019-12-25 22:20:30 +02:00
Aliaksandr Valialkin	1925ee038d	Rename lib/promql to lib/metricsql and apply small fixes	2019-12-25 22:03:59 +02:00
Mike Poindexter	bec62e4e43	Split Extended PromQL parsing to a separate library	2019-12-25 22:03:51 +02:00
Aliaksandr Valialkin	d880325cf6	app/vmselect/promql: make sure AdjustStartEnd returns time range covering the same number of points as the initial time range This should prevent from the following panic at app/vmselect/promql/binary_op.go:255: BUG: len(leftVaues) must match len(rightValues) and len(dstValues)	2019-12-24 22:45:56 +02:00
Aliaksandr Valialkin	c18802af59	lib/fs: typo fix in fadvise_unix.go	2019-12-24 20:59:28 +02:00
Aliaksandr Valialkin	4ba4abe666	lib/encoding: log the compressed block contents if it cannot be decompressed or unmarshaled This should help detecting the root cause of https://github.com/VictoriaMetrics/VictoriaMetrics/issues/281	2019-12-24 20:48:31 +02:00
Aliaksandr Valialkin	5bb39e757b	lib/encoding: mention src contents in error message returned from unmarshalInt64NearestDelta* This should simplify detecting the root cause of the issue at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/281	2019-12-24 20:41:52 +02:00
Aliaksandr Valialkin	d5c9841220	lib/encoding: mention unpacked block size in the error message if unparsed tail left Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/281	2019-12-24 20:35:13 +02:00
Aliaksandr Valialkin	9e19949c6b	app/vmselect/promql: adjust calculations for `rate` and `increase` for the first value These calculations should trigger alerts on `/api/v1/query` for counters starting from values greater than 0.	2019-12-24 19:39:25 +02:00
Aliaksandr Valialkin	0455c03cb9	app/vmselect/promql: properly calculate `rate` on the first data point It is calculated as `value / scrape_interval`, since the value was missing on the previous scrape, i.e. we can assume its value was 0 at this time.	2019-12-24 15:55:52 +02:00
Aliaksandr Valialkin	5cb8d97743	all: use gozstd instead of pure Go zstd for GOARCH=amd64	2019-12-24 12:42:42 +02:00
Aliaksandr Valialkin	31d04fb5df	Revert "lib/logger: prevent from blocking when log output isn't consumed in timely manner" This reverts commit `e3c462f08a`. Reason to revert: this leaves incomplete logs on app shutdown.	2019-12-24 12:21:39 +02:00
Aliaksandr Valialkin	5b75984aa9	app/vmselect/netstorage: move MustAdviseSequentialRead to lib/fs	2019-12-23 23:16:11 +02:00
Aliaksandr Valialkin	097c21931c	docs: sync README.md with Single-server-VictoriaMetrics.md	2019-12-23 20:34:21 +02:00
Roman Khavronenko	85463a7199	update configuration recommendations for Prometheus remote_write (#277 )	2019-12-23 20:33:10 +02:00
Aliaksandr Valialkin	6a1499efa3	lib/encoding/zstd: prevent from possible encoder leak when concurrent goroutines create encoders for the same compressionLevel Thanks to @klauspost for the pointer to this issue. See https://github.com/klauspost/compress/issues/195 for details.	2019-12-23 18:05:41 +02:00
Aliaksandr Valialkin	bf4413e58d	README.md: document how to export and import gzipped data	2019-12-23 13:40:22 +02:00
Aliaksandr Valialkin	e3c462f08a	lib/logger: prevent from blocking when log output isn't consumed in timely manner Drop log messages instead of blocking and increment `vm_log_messages_dropped_total` metric.	2019-12-20 11:49:34 +02:00
Aliaksandr Valialkin	bea5a8700a	app/vmselect: add `-search.maxExportDuration` command-line flag for limiting `/api/v1/export` duration Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/275	2019-12-20 11:35:22 +02:00
Aliaksandr Valialkin	1825893eef	lib/storage: scale ingestion performance by sharding rawRows on systems with more than 8 CPU cores	2019-12-19 18:18:29 +02:00
Aliaksandr Valialkin	97f70ccda7	lib/storage: optimize bulk import performance when multiple data points are inserted for the same time series This should speed up `/api/v1/import` and make it more scalable on multi-core systems.	2019-12-19 18:18:29 +02:00
Andrii Dembitskyi	2fba7b6f35	Fix typo in log message	2019-12-19 14:33:20 +02:00
Aliaksandr Valialkin	d03827c57d	app/vminsert: return StatusNoContent http response for `/api/v1/import` to be consistent with other insert handlers	2019-12-19 01:21:54 +02:00
Aliaksandr Valialkin	bb530a0591	lib/httpserver: inline checkAuth code to make it more clear	2019-12-18 23:06:25 +02:00
koalaty-code	aea4c80dd7	Ignore /health endpoint when checking auth	2019-12-18 23:04:31 +02:00
Aliaksandr Valialkin	5e8e0fbc80	docs/ExtendedPromQL.md: rewording regarding `scalar` vs `instant vector` difference	2019-12-18 21:47:24 +02:00
Aliaksandr Valialkin	1e8aa89a3b	docs/Home.md: fix link to case studies	2019-12-18 01:04:20 +02:00
Aliaksandr Valialkin	56595ae12a	docs: renaming: `PromQL extensions -> MetricsQL`	2019-12-18 00:56:51 +02:00