app/vmselect: unconditionally offset -step to rollup_candlestick. This makes results more consistent

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/309

Makefile

@@ -79,16 +79,16 @@ install-errcheck:
 check-all: fmt vet lint errcheck golangci-lint
 
 test:
-	GO111MODULE=on go test -tags=integration -mod=vendor ./lib/... ./app/...
+	GO111MODULE=on go test -mod=vendor ./lib/... ./app/...
 
 test-pure:
-	GO111MODULE=on CGO_ENABLED=0 go test -tags=integration -mod=vendor ./lib/... ./app/...
+	GO111MODULE=on CGO_ENABLED=0 go test -mod=vendor ./lib/... ./app/...
 
 test-full:
-	GO111MODULE=on go test -tags=integration -mod=vendor -coverprofile=coverage.txt -covermode=atomic ./lib/... ./app/...
+	GO111MODULE=on go test -mod=vendor -coverprofile=coverage.txt -covermode=atomic ./lib/... ./app/...
 
 test-full-386:
-	GO111MODULE=on GOARCH=386 go test -tags=integration -mod=vendor -coverprofile=coverage.txt -covermode=atomic ./lib/... ./app/...
+	GO111MODULE=on GOARCH=386 go test -mod=vendor -coverprofile=coverage.txt -covermode=atomic ./lib/... ./app/...
 
 benchmark:
 	GO111MODULE=on go test -mod=vendor -bench=. ./lib/...

README.md

@@ -18,8 +18,10 @@ in [source code](https://github.com/VictoriaMetrics/VictoriaMetrics). Just downl
 Cluster version is available [here](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster).
 
 
-## Case studies
+## Case studies and talks
 
+* [Adidas](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/CaseStudies#adidas)
+* [COLOPL](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/CaseStudies#colopl)
 * [Wix.com](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/CaseStudies#wixcom)
 * [Wedos.com](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/CaseStudies#wedoscom)
 * [Dreamteam](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/CaseStudies#dreamteam)
@@ -86,7 +88,6 @@ Cluster version is available [here](https://github.com/VictoriaMetrics/VictoriaM
     - [Building docker images](#building-docker-images)
   - [Start with docker-compose](#start-with-docker-compose)
   - [Setting up service](#setting-up-service)
-  - [Third-party contributions](#third-party-contributions)
   - [How to work with snapshots?](#how-to-work-with-snapshots)
   - [How to delete time series?](#how-to-delete-time-series)
   - [How to export time series?](#how-to-export-time-series)
@@ -94,6 +95,8 @@ Cluster version is available [here](https://github.com/VictoriaMetrics/VictoriaM
   - [Federation](#federation)
   - [Capacity planning](#capacity-planning)
   - [High availability](#high-availability)
+  - [Deduplication](#deduplication)
+  - [Retention](#retention)
   - [Multiple retentions](#multiple-retentions)
   - [Downsampling](#downsampling)
   - [Multi-tenancy](#multi-tenancy)
@@ -109,6 +112,7 @@ Cluster version is available [here](https://github.com/VictoriaMetrics/VictoriaM
 - [Roadmap](#roadmap)
 - [Contacts](#contacts)
 - [Community and contributions](#community-and-contributions)
+- [Third-party contributions](#third-party-contributions)
 - [Reporting bugs](#reporting-bugs)
 - [Victoria Metrics Logo](#victoria-metrics-logo)
   - [Logo Usage Guidelines](#logo-usage-guidelines)
@@ -500,11 +504,6 @@ More details may be found [here](https://github.com/VictoriaMetrics/VictoriaMetr
 Read [these instructions](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/43) on how to set up VictoriaMetrics as a service in your OS.
 
 
-### Third-party contributions
-
-* [Unofficial yum repository](https://copr.fedorainfracloud.org/coprs/antonpatsev/VictoriaMetrics/) ([source code](https://github.com/patsevanton/victoriametrics-rpm))
-
-
 ### How to work with snapshots?
 
 VictoriaMetrics can create [instant snapshots](https://medium.com/@valyala/how-victoriametrics-makes-instant-snapshots-for-multi-terabyte-time-series-data-e1f3fb0e0282)
@@ -544,6 +543,19 @@ the deleted time series isn't freed instantly - it is freed during subsequent me
 It is recommended verifying which metrics will be deleted with the call to `http://<victoria-metrics-addr>:8428/api/v1/series?match[]=<timeseries_selector_for_delete>`
 before actually deleting the metrics.
 
+The delete API is intended mainly for the following cases:
+
+- One-off deleting of accidentally written invalid (or undesired) time series.
+- One-off deleting of user data due to [GDPR](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation).
+
+It isn't recommended using delete API for the following cases, since it brings non-zero overhead:
+
+- Regular cleanups for unneded data. Just prevent writing unneeded data into VictoriaMetrics.
+- Reducing disk space usage by deleting unneded time series. This doesn't work as expected, since the deleted
+  time series occupy disk space until the next merge operation, which can never occur.
+
+It is better using `-retentionPeriod` command-line flag for efficient pruning of old data.
+
 
 ### How to export time series?
 
@@ -690,6 +702,29 @@ kill -HUP `pidof prometheus`
 If you have Prometheus HA pairs with replicas `r1` and `r2` in each pair, then configure each `r1`
 to write data to `victoriametrics-addr-1`, while each `r2` should write data to `victoriametrics-addr-2`.
 
+Another option is to write data simultaneously from Prometheus HA pair to a pair of VictoriaMetrics instances
+with the enabled de-duplication. See [this section](#deduplication) for details.
+
+
+### Deduplication
+
+VictoriaMetrics de-duplicates data points if `-dedup.minScrapeInterval` command-line flag
+is set to positive duration. For example, `-dedup.minScrapeInterval=60s` would de-duplicate data points
+on the same time series if they are located closer than 60s to each other.
+The de-duplication reduces disk space usage if multiple identically configured Prometheus instances in HA pair
+write data to the same VictoriaMetrics instance. Note that these Prometheus instances must have identical
+`external_labels` section in their configs, so they write data to the same time series.
+
+
+### Retention
+
+Retention is configured with `-retentionPeriod` command-line flag. For instance, `-retentionPeriod=3` means
+that the data will be stored for 3 months and then deleted.
+Data is split in per-month subdirectories inside `<-storageDataPath>/data/small` and `<-storageDataPath>/data/big` folders.
+Directories for months outside the configured retention are deleted on the first day of new month.
+In order to keep data according to `-retentionPeriod` max disk space usage is going to be `-retentionPeriod` + 1 month.
+For example if `-retentionPeriod` is set to 1, data for January is deleted on March 1st.
+
 
 ### Multiple retentions
 
@@ -770,8 +805,11 @@ mkfs.ext4 ... -O 64bit,huge_file,extent -T huge
 
 ### Monitoring
 
-VictoriaMetrics exports internal metrics in Prometheus format on the `/metrics` page.
-Add this page to Prometheus' scrape config in order to collect VictoriaMetrics metrics.
+VictoriaMetrics exports internal metrics in Prometheus format at `/metrics` page.
+These metrics may be collected either via Prometheus by adding the corresponding scrape config to it.
+Alternatively they can be self-scraped by setting `-selfScrapeInterval` command-line flag to duration greater than 0.
+For example, `-scrapeInterval=10s` would enable self-scraping of `/metrics` page with 10 seconds interval.
+
 There are officials Grafana dashboards for [single-node VictoriaMetrics](https://grafana.com/dashboards/10229) and [clustered VictoriaMetrics](https://grafana.com/grafana/dashboards/11176).
 
 The most interesting metrics are:
@@ -812,6 +850,7 @@ The most interesting metrics are:
 
 ### Backfilling
 
+VictoriaMetrics accepts historical data in arbitrary order of time.
 Make sure that configured `-retentionPeriod` covers timestamps for the backfilled data.
 
 It is recommended disabling query cache with `-search.disableCache` command-line flag when writing
@@ -869,6 +908,7 @@ Contact us with any questions regarding VictoriaMetrics at [info@victoriametrics
 Feel free asking any questions regarding VictoriaMetrics:
 
 - [slack](http://slack.victoriametrics.com/)
+- [reddit](https://www.reddit.com/r/VictoriaMetrics/)
 - [telegram-en](https://t.me/VictoriaMetrics_en)
 - [telegram-ru](https://t.me/VictoriaMetrics_ru1)
 - [google groups](https://groups.google.com/forum/#!forum/victorametrics-users)
@@ -892,6 +932,13 @@ We are open to third-party pull requests provided they follow [KISS design princ
 Adhering `KISS` principle simplifies the resulting code and architecture, so it can be reviewed, understood and verified by many people.
 
 
+## Third-party contributions
+
+* [Unofficial yum repository](https://copr.fedorainfracloud.org/coprs/antonpatsev/VictoriaMetrics/) ([source code](https://github.com/patsevanton/victoriametrics-rpm))
+* [Prometheus -> VictoriaMetrics exporter #1](https://github.com/ryotarai/prometheus-tsdb-dump)
+* [Prometheus -> VictoriaMetrics exporter #2](https://github.com/AnchorFree/tsdb-remote-write)
+
+
 ## Reporting bugs
 
 Report bugs and propose new features [here](https://github.com/VictoriaMetrics/VictoriaMetrics/issues).

app/victoria-metrics/main.go

@@ -26,27 +26,30 @@ func main() {
 	vmstorage.Init()
 	vmselect.Init()
 	vminsert.Init()
+	startSelfScraper()
 
 	go httpserver.Serve(*httpListenAddr, requestHandler)
-	logger.Infof("started VictoriaMetrics in %s", time.Since(startTime))
+	logger.Infof("started VictoriaMetrics in %.3f seconds", time.Since(startTime).Seconds())
 
 	sig := procutil.WaitForSigterm()
 	logger.Infof("received signal %s", sig)
 
+	stopSelfScraper()
+
 	logger.Infof("gracefully shutting down webservice at %q", *httpListenAddr)
 	startTime = time.Now()
 	if err := httpserver.Stop(*httpListenAddr); err != nil {
 		logger.Fatalf("cannot stop the webservice: %s", err)
 	}
 	vminsert.Stop()
-	logger.Infof("successfully shut down the webservice in %s", time.Since(startTime))
+	logger.Infof("successfully shut down the webservice in %.3f seconds", time.Since(startTime).Seconds())
 
 	vmstorage.Stop()
 	vmselect.Stop()
 
 	fs.MustStopDirRemover()
 
-	logger.Infof("the VictoriaMetrics has been stopped in %s", time.Since(startTime))
+	logger.Infof("the VictoriaMetrics has been stopped in %.3f seconds", time.Since(startTime).Seconds())
 }
 
 func requestHandler(w http.ResponseWriter, r *http.Request) bool {

app/victoria-metrics/main_test.go

@@ -1,5 +1,3 @@
-// +build integration
-
 package main
 
 import (
@@ -302,6 +300,9 @@ func readIn(readFor string, t *testing.T, insertTime time.Time) []test {
 	s := newSuite(t)
 	var tt []test
 	s.noError(filepath.Walk(filepath.Join(testFixturesDir, readFor), func(path string, info os.FileInfo, err error) error {
+		if err != nil {
+			return err
+		}
 		if filepath.Ext(path) != ".json" {
 			return nil
 		}

app/victoria-metrics/self_scraper.go

@@ -0,0 +1,99 @@
+package main
+
+import (
+	"flag"
+	"sync"
+	"time"
+
+	"github.com/VictoriaMetrics/VictoriaMetrics/app/vmstorage"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/prompb"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/protoparser/prometheus"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
+)
+
+var selfScrapeInterval = flag.Duration("selfScrapeInterval", 0, "Interval for self-scraping own metrics at /metrics page")
+
+var selfScraperStopCh chan struct{}
+var selfScraperWG sync.WaitGroup
+
+func startSelfScraper() {
+	selfScraperStopCh = make(chan struct{})
+	selfScraperWG.Add(1)
+	go func() {
+		defer selfScraperWG.Done()
+		selfScraper(*selfScrapeInterval)
+	}()
+}
+
+func stopSelfScraper() {
+	close(selfScraperStopCh)
+	selfScraperWG.Wait()
+}
+
+func selfScraper(scrapeInterval time.Duration) {
+	if scrapeInterval <= 0 {
+		// Self-scrape is disabled.
+		return
+	}
+	logger.Infof("started self-scraping `/metrics` page with interval %.3f seconds", scrapeInterval.Seconds())
+
+	var bb bytesutil.ByteBuffer
+	var rows prometheus.Rows
+	var mrs []storage.MetricRow
+	var labels []prompb.Label
+	t := time.NewTicker(scrapeInterval)
+	var currentTimestamp int64
+	for {
+		select {
+		case <-selfScraperStopCh:
+			t.Stop()
+			logger.Infof("stopped self-scraping `/metrics` page")
+			return
+		case currentTime := <-t.C:
+			currentTimestamp = currentTime.UnixNano() / 1e6
+		}
+		bb.Reset()
+		httpserver.WritePrometheusMetrics(&bb)
+		s := bytesutil.ToUnsafeString(bb.B)
+		rows.Reset()
+		rows.Unmarshal(s)
+		mrs = mrs[:0]
+		for i := range rows.Rows {
+			r := &rows.Rows[i]
+			labels = labels[:0]
+			labels = addLabel(labels, "", r.Metric)
+			labels = addLabel(labels, "job", "victoria-metrics")
+			labels = addLabel(labels, "instance", "self")
+			for j := range r.Tags {
+				t := &r.Tags[j]
+				labels = addLabel(labels, t.Key, t.Value)
+			}
+			if len(mrs) < cap(mrs) {
+				mrs = mrs[:len(mrs)+1]
+			} else {
+				mrs = append(mrs, storage.MetricRow{})
+			}
+			mr := &mrs[len(mrs)-1]
+			mr.MetricNameRaw = storage.MarshalMetricNameRaw(mr.MetricNameRaw[:0], labels)
+			mr.Timestamp = currentTimestamp
+			mr.Value = r.Value
+		}
+		logger.Infof("writing %d rows at timestamp %d", len(mrs), currentTimestamp)
+		vmstorage.AddRows(mrs)
+	}
+}
+
+func addLabel(dst []prompb.Label, key, value string) []prompb.Label {
+	if len(dst) < cap(dst) {
+		dst = dst[:len(dst)+1]
+	} else {
+		dst = append(dst, prompb.Label{})
+	}
+	lb := &dst[len(dst)-1]
+	lb.Name = bytesutil.ToUnsafeBytes(key)
+	lb.Value = bytesutil.ToUnsafeBytes(value)
+	return dst
+}

app/victoria-metrics/test/prom_types.go

@@ -1,18 +1,18 @@
-// +build integration
-
-// Source https://github.com/prometheus/prometheus/blob/master/prompb/remote.pb.go . Code is copy pasted and cleaned up
 package test
 
+// Source https://github.com/prometheus/prometheus/blob/master/prompb/remote.pb.go . Code is copy pasted and cleaned up
 import (
 	"encoding/binary"
 	"math"
 	"math/bits"
 )
 
+// WriteRequest is write request
 type WriteRequest struct {
 	Timeseries []TimeSeries `protobuf:"bytes,1,rep,name=timeseries,proto3" json:"timeseries"`
 }
 
+// Size returns m size in bytes after marshaling.
 func (m *WriteRequest) Size() (n int) {
 	if m == nil {
 		return 0
@@ -31,6 +31,7 @@ func sovRemote(x uint64) (n int) {
 	return (bits.Len64(x|1) + 6) / 7
 }
 
+// Marshal marshals m.
 func (m *WriteRequest) Marshal() (dAtA []byte, err error) {
 	size := m.Size()
 	dAtA = make([]byte, size)
@@ -41,11 +42,13 @@ func (m *WriteRequest) Marshal() (dAtA []byte, err error) {
 	return dAtA[:n], nil
 }
 
+// MarshalTo marshals m to dAtA
 func (m *WriteRequest) MarshalTo(dAtA []byte) (int, error) {
 	size := m.Size()
 	return m.MarshalToSizedBuffer(dAtA[:size])
 }
 
+// MarshalToSizedBuffer marshals m to dAtA.
 func (m *WriteRequest) MarshalToSizedBuffer(dAtA []byte) (int, error) {
 	i := len(dAtA)
 	if len(m.Timeseries) > 0 {
@@ -77,11 +80,13 @@ func encodeVarintRemote(dAtA []byte, offset int, v uint64) int {
 	return base
 }
 
+// Sample is time series sample.
 type Sample struct {
 	Value     float64 `protobuf:"fixed64,1,opt,name=value,proto3" json:"value,omitempty"`
 	Timestamp int64   `protobuf:"varint,2,opt,name=timestamp,proto3" json:"timestamp,omitempty"`
 }
 
+// Reset resets m.
 func (m *Sample) Reset() { *m = Sample{} }
 
 // TimeSeries represents samples and labels for a single time series.
@@ -90,21 +95,27 @@ type TimeSeries struct {
 	Samples []Sample `protobuf:"bytes,2,rep,name=samples,proto3" json:"samples"`
 }
 
+// Reset resets m.
 func (m *TimeSeries) Reset() { *m = TimeSeries{} }
 
+// Label is time series label.
 type Label struct {
 	Name  string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
 	Value string `protobuf:"bytes,2,opt,name=value,proto3" json:"value,omitempty"`
 }
 
+// Reset resets m.
 func (m *Label) Reset() { *m = Label{} }
 
+// Labels is a set of labels.
 type Labels struct {
 	Labels []Label `protobuf:"bytes,1,rep,name=labels,proto3" json:"labels"`
 }
 
+// Reset resets m.
 func (m *Labels) Reset() { *m = Labels{} }
 
+// Marshal marshals m.
 func (m *Sample) Marshal() (dAtA []byte, err error) {
 	size := m.Size()
 	dAtA = make([]byte, size)
@@ -115,11 +126,13 @@ func (m *Sample) Marshal() (dAtA []byte, err error) {
 	return dAtA[:n], nil
 }
 
+// MarshalTo marshals m to dAtA.
 func (m *Sample) MarshalTo(dAtA []byte) (int, error) {
 	size := m.Size()
 	return m.MarshalToSizedBuffer(dAtA[:size])
 }
 
+// MarshalToSizedBuffer marshals m to dAtA.
 func (m *Sample) MarshalToSizedBuffer(dAtA []byte) (int, error) {
 	i := len(dAtA)
 	if m.Timestamp != 0 {
@@ -136,6 +149,7 @@ func (m *Sample) MarshalToSizedBuffer(dAtA []byte) (int, error) {
 	return len(dAtA) - i, nil
 }
 
+// Marshal marshals m.
 func (m *TimeSeries) Marshal() (dAtA []byte, err error) {
 	size := m.Size()
 	dAtA = make([]byte, size)
@@ -146,11 +160,13 @@ func (m *TimeSeries) Marshal() (dAtA []byte, err error) {
 	return dAtA[:n], nil
 }
 
+// MarshalTo marshals m to dAtA.
 func (m *TimeSeries) MarshalTo(dAtA []byte) (int, error) {
 	size := m.Size()
 	return m.MarshalToSizedBuffer(dAtA[:size])
 }
 
+// MarshalToSizedBuffer marshals m to dAtA.
 func (m *TimeSeries) MarshalToSizedBuffer(dAtA []byte) (int, error) {
 	i := len(dAtA)
 	if len(m.Samples) > 0 {
@@ -184,6 +200,7 @@ func (m *TimeSeries) MarshalToSizedBuffer(dAtA []byte) (int, error) {
 	return len(dAtA) - i, nil
 }
 
+// Marshal marshals m.
 func (m *Label) Marshal() (dAtA []byte, err error) {
 	size := m.Size()
 	dAtA = make([]byte, size)
@@ -194,11 +211,13 @@ func (m *Label) Marshal() (dAtA []byte, err error) {
 	return dAtA[:n], nil
 }
 
+// MarshalTo marshals m to dAtA.
 func (m *Label) MarshalTo(dAtA []byte) (int, error) {
 	size := m.Size()
 	return m.MarshalToSizedBuffer(dAtA[:size])
 }
 
+// MarshalToSizedBuffer marshals m to dAtA.
 func (m *Label) MarshalToSizedBuffer(dAtA []byte) (int, error) {
 	i := len(dAtA)
 	_ = i
@@ -221,6 +240,7 @@ func (m *Label) MarshalToSizedBuffer(dAtA []byte) (int, error) {
 	return len(dAtA) - i, nil
 }
 
+// Marshal marshals m.
 func (m *Labels) Marshal() (dAtA []byte, err error) {
 	size := m.Size()
 	dAtA = make([]byte, size)
@@ -231,11 +251,13 @@ func (m *Labels) Marshal() (dAtA []byte, err error) {
 	return dAtA[:n], nil
 }
 
+// MarshalTo marshals m to dAtA.
 func (m *Labels) MarshalTo(dAtA []byte) (int, error) {
 	size := m.Size()
 	return m.MarshalToSizedBuffer(dAtA[:size])
 }
 
+// MarshalToSizedBuffer marshals m to dAtA.
 func (m *Labels) MarshalToSizedBuffer(dAtA []byte) (int, error) {
 	i := len(dAtA)
 	if len(m.Labels) > 0 {
@@ -267,6 +289,7 @@ func encodeVarintTypes(dAtA []byte, offset int, v uint64) int {
 	return base
 }
 
+// Size returns the size of marshaled m.
 func (m *Sample) Size() (n int) {
 	if m == nil {
 		return 0
@@ -280,6 +303,7 @@ func (m *Sample) Size() (n int) {
 	return n
 }
 
+// Size returns the size of marshaled m.
 func (m *TimeSeries) Size() (n int) {
 	if m == nil {
 		return 0
@@ -301,6 +325,7 @@ func (m *TimeSeries) Size() (n int) {
 	return n
 }
 
+// Size returns the size of marshaled m.
 func (m *Label) Size() (n int) {
 	if m == nil {
 		return 0
@@ -318,6 +343,7 @@ func (m *Label) Size() (n int) {
 	return n
 }
 
+// Size returns the size of marshaled m.
 func (m *Labels) Size() (n int) {
 	if m == nil {
 		return 0

app/victoria-metrics/test/prom_writter.go

@@ -1,9 +1,8 @@
-// +build integration
-
 package test
 
 import "github.com/golang/snappy"
 
+// Compress marshals and compresses wr.
 func Compress(wr WriteRequest) ([]byte, error) {
 	data, err := wr.Marshal()
 	if err != nil {

app/victoria-metrics/testdata/graphite/max_lookback_set.json

@@ -13,11 +13,8 @@
     "data":{"resultType":"matrix",
       "result":[{"metric":{"__name__":"max_lookback_set"},"values":[
 	      ["{TIME_S-150s}","4"],
-	      ["{TIME_S-140s}","4"],
 	      ["{TIME_S-120s}","3"],
-	      ["{TIME_S-110s}","3"],
 	      ["{TIME_S-60s}","2"],
-	      ["{TIME_S-50s}","2"],
 	      ["{TIME_S-30s}","1"],
 	      ["{TIME_S-20s}","1"]
       ]}]}}

app/victoria-metrics/testdata/graphite/max_lookback_unset.json

@@ -19,14 +19,12 @@
 	      ["{TIME_S-110s}","3"],
 	      ["{TIME_S-100s}","3"],
 	      ["{TIME_S-90s}","3"],
-	      ["{TIME_S-80s}","3"],
-	      ["{TIME_S-70s}","3"],
 	      ["{TIME_S-60s}","2"],
 	      ["{TIME_S-50s}","2"],
 	      ["{TIME_S-40s}","2"],
 	      ["{TIME_S-30s}","1"],
 	      ["{TIME_S-20s}","1"],
 	      ["{TIME_S-10s}","1"],
-	      ["{TIME_S}","1"]
+	      ["{TIME_S-0s}","1"]
       ]}]}}
 }

app/vmalert/README.md

@@ -0,0 +1,41 @@
+## VM Alert
+
+#### Abstract
+The application which accepts the alert rules, executes them on given source, sends(fires) an alert to(in) alert management system
+
+### Components
+
+#### Alert Config Reader
+It accepts yaml config as input parameter in Prometheus format, parses it into Go struct.
+
+#### Source Caller
+Create own watchdog for every alert group (goroutines), which executes alert query on given source and issues an alert if source returns non-empty result. 
+Source can be any service which supports PromQL (MetricsQL). 
+
+#### Alert Management System Provider
+Send positive alert to alert management system, provides interface for every concrete implementation.
+Should be ingratiated with Prometheus alertmanager. 
+
+open questions:
+- do we really need alert group or can just run every alert in own goroutine?
+
+#### Web Server
+Expose metrics
+
+open questions:
+- should the tool provide API or UI for managing alerting rules? Where to store config updated via the API or UI?
+- should the tool provide “alerting rules validation mode” for validating and debugging alerting rules? This mode is useful when creating and debugging alerting rules.
+
+#### Requirements:
+- Stateless
+- Avoid external dependencies if possible
+- Reuse existing code from VictoriaMetrics repo
+- Makefile rules for common tasks – see Makefiles for other apps in the app/ dir 
+- Every package should be covered by tests 
+- Dockerfile
+- Graceful shutdown 
+- Helm template
+- Application uses command line flags for configuration
+
+
+<img alt="VM Alert" src="vmalert.png">

app/vminsert/common/gzip_reader.go

@@ -1,9 +1,10 @@
 package common
 
 import (
-	"compress/gzip"
 	"io"
 	"sync"
+
+	"github.com/klauspost/compress/gzip"
 )
 
 // GetGzipReader returns new gzip reader from the pool.

app/vminsert/concurrencylimiter/concurrencylimiter.go

@@ -12,17 +12,14 @@ import (
 	"github.com/VictoriaMetrics/metrics"
 )
 
-var maxConcurrentInserts = flag.Int("maxConcurrentInserts", runtime.GOMAXPROCS(-1)*4, "The maximum number of concurrent inserts")
-
 var (
-	// ch is the channel for limiting concurrent calls to Do.
-	ch chan struct{}
-
-	// waitDuration is the amount of time to wait until at least a single
-	// concurrent Do call out of cap(ch) inserts is complete.
-	waitDuration = time.Second * 30
+	maxConcurrentInserts = flag.Int("maxConcurrentInserts", runtime.GOMAXPROCS(-1)*4, "The maximum number of concurrent inserts; see also -insert.maxQueueDuration")
+	maxQueueDuration     = flag.Duration("insert.maxQueueDuration", time.Minute, "The maximum duration for waiting in the queue for insert requests due to -maxConcurrentInserts")
 )
 
+// ch is the channel for limiting concurrent calls to Do.
+var ch chan struct{}
+
 // Init initializes concurrencylimiter.
 //
 // Init must be called after flag.Parse call.
@@ -43,9 +40,9 @@ func Do(f func() error) error {
 	}
 
 	// All the workers are busy.
-	// Sleep for up to waitDuration.
+	// Sleep for up to *maxQueueDuration.
 	concurrencyLimitReached.Inc()
-	t := timerpool.Get(waitDuration)
+	t := timerpool.Get(*maxQueueDuration)
 	select {
 	case ch <- struct{}{}:
 		timerpool.Put(t)
@@ -56,7 +53,9 @@ func Do(f func() error) error {
 		timerpool.Put(t)
 		concurrencyLimitTimeout.Inc()
 		return &httpserver.ErrorWithStatusCode{
-			Err:        fmt.Errorf("the server is overloaded with %d concurrent inserts; either increase -maxConcurrentInserts or reduce the load", cap(ch)),
+			Err: fmt.Errorf("cannot handle more than %d concurrent inserts during %s; possible solutions: "+
+				"increase `-insert.maxQueueDuration`, increase `-maxConcurrentInserts`, "+
+				"decrease `-search.maxConcurrentRequests`, increase server capacity", *maxConcurrentInserts, *maxQueueDuration),
 			StatusCode: http.StatusServiceUnavailable,
 		}
 	}

app/vminsert/graphite/request_handler.go

@@ -11,6 +11,7 @@ import (
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vminsert/common"
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vminsert/concurrencylimiter"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/protoparser/graphite"
 	"github.com/VictoriaMetrics/metrics"
 )
 
@@ -106,7 +107,7 @@ func (ctx *pushCtx) Read(r io.Reader) bool {
 }
 
 type pushCtx struct {
-	Rows   Rows
+	Rows   graphite.Rows
 	Common common.InsertCtx
 
 	reqBuf  []byte

app/vminsert/influx/request_handler.go

@@ -12,13 +12,14 @@ import (
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vminsert/common"
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vminsert/concurrencylimiter"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/protoparser/influx"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
 	"github.com/VictoriaMetrics/metrics"
 )
 
 var (
-	measurementFieldSeparator = flag.String("influxMeasurementFieldSeparator", "_", "Separator for `{measurement}{separator}{field_name}` metric name when inserted via Influx line protocol")
-	skipSingleField           = flag.Bool("influxSkipSingleField", false, "Uses `{measurement}` instead of `{measurement}{separator}{field_name}` for metic name if Influx line contains only a single field")
+	measurementFieldSeparator = flag.String("influxMeasurementFieldSeparator", "_", "Separator for '{measurement}{separator}{field_name}' metric name when inserted via Influx line protocol")
+	skipSingleField           = flag.Bool("influxSkipSingleField", false, "Uses '{measurement}' instead of '{measurement}{separator}{field_name}' for metic name if Influx line contains only a single field")
 )
 
 var (
@@ -171,7 +172,7 @@ var (
 )
 
 type pushCtx struct {
-	Rows   Rows
+	Rows   influx.Rows
 	Common common.InsertCtx
 
 	reqBuf         []byte

app/vminsert/main.go

@@ -24,7 +24,6 @@ var (
 		"Telnet put messages and HTTP /api/put messages are simultaneously served on TCP port. "+
 		"Usually :4242 must be set. Doesn't work if empty")
 	opentsdbHTTPListenAddr = flag.String("opentsdbHTTPListenAddr", "", "TCP address to listen for OpentTSDB HTTP put requests. Usually :4242 must be set. Doesn't work if empty")
-	maxInsertRequestSize   = flag.Int("maxInsertRequestSize", 32*1024*1024, "The maximum size of a single insert request in bytes")
 	maxLabelsPerTimeseries = flag.Int("maxLabelsPerTimeseries", 30, "The maximum number of labels accepted per time series. Superflouos labels are dropped")
 )
 
@@ -43,10 +42,10 @@ func Init() {
 		graphiteServer = graphite.MustStart(*graphiteListenAddr)
 	}
 	if len(*opentsdbListenAddr) > 0 {
-		opentsdbServer = opentsdb.MustStart(*opentsdbListenAddr, int64(*maxInsertRequestSize))
+		opentsdbServer = opentsdb.MustStart(*opentsdbListenAddr)
 	}
 	if len(*opentsdbHTTPListenAddr) > 0 {
-		opentsdbhttpServer = opentsdbhttp.MustStart(*opentsdbHTTPListenAddr, int64(*maxInsertRequestSize))
+		opentsdbhttpServer = opentsdbhttp.MustStart(*opentsdbHTTPListenAddr)
 	}
 }
 
@@ -69,7 +68,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 	switch path {
 	case "/api/v1/write":
 		prometheusWriteRequests.Inc()
-		if err := prometheus.InsertHandler(r, int64(*maxInsertRequestSize)); err != nil {
+		if err := prometheus.InsertHandler(r); err != nil {
 			prometheusWriteErrors.Inc()
 			httpserver.Errorf(w, "error in %q: %s", r.URL.Path, err)
 			return true

app/vminsert/opentsdb/request_handler.go

@@ -11,6 +11,7 @@ import (
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vminsert/common"
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vminsert/concurrencylimiter"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/protoparser/opentsdb"
 	"github.com/VictoriaMetrics/metrics"
 )
 
@@ -105,7 +106,7 @@ func (ctx *pushCtx) Read(r io.Reader) bool {
 }
 
 type pushCtx struct {
-	Rows   Rows
+	Rows   opentsdb.Rows
 	Common common.InsertCtx
 
 	reqBuf  []byte

app/vminsert/opentsdb/server.go

@@ -36,7 +36,7 @@ type Server struct {
 // MustStart starts OpenTSDB collector on the given addr.
 //
 // MustStop must be called on the returned server when it is no longer needed.
-func MustStart(addr string, maxRequestSize int64) *Server {
+func MustStart(addr string) *Server {
 	logger.Infof("starting TCP OpenTSDB collector at %q", addr)
 	lnTCP, err := netutil.NewTCPListener("opentsdb", addr)
 	if err != nil {
@@ -45,7 +45,7 @@ func MustStart(addr string, maxRequestSize int64) *Server {
 	ls := newListenerSwitch(lnTCP)
 	lnHTTP := ls.newHTTPListener()
 	lnTelnet := ls.newTelnetListener()
-	httpServer := opentsdbhttp.MustServe(lnHTTP, maxRequestSize)
+	httpServer := opentsdbhttp.MustServe(lnHTTP)
 
 	logger.Infof("starting UDP OpenTSDB collector at %q", addr)
 	lnUDP, err := net.ListenPacket("udp4", addr)

app/vminsert/opentsdbhttp/request_handler.go

@@ -1,6 +1,7 @@
 package opentsdbhttp
 
 import (
+	"flag"
 	"fmt"
 	"io"
 	"net/http"
@@ -11,10 +12,12 @@ import (
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vminsert/common"
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vminsert/concurrencylimiter"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/protoparser/opentsdbhttp"
 	"github.com/VictoriaMetrics/metrics"
-	"github.com/valyala/fastjson"
 )
 
+var maxInsertRequestSize = flag.Int("opentsdbhttp.maxInsertRequestSize", 32*1024*1024, "The maximum size of OpenTSDB HTTP put request")
+
 var (
 	rowsInserted  = metrics.NewCounter(`vm_rows_inserted_total{type="opentsdb-http"}`)
 	rowsPerInsert = metrics.NewSummary(`vm_rows_per_insert{type="opentsdb-http"}`)
@@ -26,13 +29,13 @@ var (
 
 // insertHandler processes HTTP OpenTSDB put requests.
 // See http://opentsdb.net/docs/build/html/api_http/put.html
-func insertHandler(req *http.Request, maxSize int64) error {
+func insertHandler(req *http.Request) error {
 	return concurrencylimiter.Do(func() error {
-		return insertHandlerInternal(req, maxSize)
+		return insertHandlerInternal(req)
 	})
 }
 
-func insertHandlerInternal(req *http.Request, maxSize int64) error {
+func insertHandlerInternal(req *http.Request) error {
 	readCalls.Inc()
 
 	r := req.Body
@@ -50,20 +53,20 @@ func insertHandlerInternal(req *http.Request, maxSize int64) error {
 	defer putPushCtx(ctx)
 
 	// Read the request in ctx.reqBuf
-	lr := io.LimitReader(r, maxSize+1)
+	lr := io.LimitReader(r, int64(*maxInsertRequestSize)+1)
 	reqLen, err := ctx.reqBuf.ReadFrom(lr)
 	if err != nil {
 		readErrors.Inc()
 		return fmt.Errorf("cannot read HTTP OpenTSDB request: %s", err)
 	}
-	if reqLen > maxSize {
+	if reqLen > int64(*maxInsertRequestSize) {
 		readErrors.Inc()
-		return fmt.Errorf("too big HTTP OpenTSDB request; mustn't exceed %d bytes", maxSize)
+		return fmt.Errorf("too big HTTP OpenTSDB request; mustn't exceed `-opentsdbhttp.maxInsertRequestSize=%d` bytes", *maxInsertRequestSize)
 	}
 
 	// Unmarshal the request to ctx.Rows
-	p := parserPool.Get()
-	defer parserPool.Put(p)
+	p := opentsdbhttp.GetParser()
+	defer opentsdbhttp.PutParser(p)
 	v, err := p.ParseBytes(ctx.reqBuf.B)
 	if err != nil {
 		unmarshalErrors.Inc()
@@ -110,10 +113,8 @@ func insertHandlerInternal(req *http.Request, maxSize int64) error {
 
 const secondMask int64 = 0x7FFFFFFF00000000
 
-var parserPool fastjson.ParserPool
-
 type pushCtx struct {
-	Rows   Rows
+	Rows   opentsdbhttp.Rows
 	Common common.InsertCtx
 
 	reqBuf bytesutil.ByteBuffer

app/vminsert/opentsdbhttp/server.go

@@ -28,20 +28,20 @@ type Server struct {
 // MustStart starts HTTP OpenTSDB server on the given addr.
 //
 // MustStop must be called on the returned server when it is no longer needed.
-func MustStart(addr string, maxRequestSize int64) *Server {
+func MustStart(addr string) *Server {
 	logger.Infof("starting HTTP OpenTSDB server at %q", addr)
 	lnTCP, err := netutil.NewTCPListener("opentsdbhttp", addr)
 	if err != nil {
 		logger.Fatalf("cannot start HTTP OpenTSDB collector at %q: %s", addr, err)
 	}
-	return MustServe(lnTCP, maxRequestSize)
+	return MustServe(lnTCP)
 }
 
-// MustServe serves OpenTSDB HTTP put requests from ln with up to maxRequestSize size.
+// MustServe serves OpenTSDB HTTP put requests from ln.
 //
 // MustStop must be called on the returned server when it is no longer needed.
-func MustServe(ln net.Listener, maxRequestSize int64) *Server {
-	h := newRequestHandler(maxRequestSize)
+func MustServe(ln net.Listener) *Server {
+	h := newRequestHandler()
 	hs := &http.Server{
 		Handler:      h,
 		ReadTimeout:  30 * time.Second,
@@ -82,12 +82,12 @@ func (s *Server) MustStop() {
 	logger.Infof("OpenTSDB HTTP server at %q has been stopped", s.ln.Addr())
 }
 
-func newRequestHandler(maxRequestSize int64) http.Handler {
+func newRequestHandler() http.Handler {
 	rh := func(w http.ResponseWriter, r *http.Request) {
 		switch r.URL.Path {
 		case "/api/put":
 			writeRequests.Inc()
-			if err := insertHandler(r, maxRequestSize); err != nil {
+			if err := insertHandler(r); err != nil {
 				writeErrors.Inc()
 				httpserver.Errorf(w, "error in %q: %s", r.URL.Path, err)
 				return

app/vminsert/prometheus/request_handler.go

@@ -1,33 +1,39 @@
 package prometheus
 
 import (
+	"flag"
 	"fmt"
+	"io"
 	"net/http"
 	"runtime"
 	"sync"
 
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vminsert/common"
 	"github.com/VictoriaMetrics/VictoriaMetrics/app/vminsert/concurrencylimiter"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/prompb"
 	"github.com/VictoriaMetrics/metrics"
+	"github.com/golang/snappy"
 )
 
+var maxInsertRequestSize = flag.Int("maxInsertRequestSize", 32*1024*1024, "The maximum size in bytes of a single Prometheus remote_write API request")
+
 var (
 	rowsInserted  = metrics.NewCounter(`vm_rows_inserted_total{type="prometheus"}`)
 	rowsPerInsert = metrics.NewSummary(`vm_rows_per_insert{type="prometheus"}`)
 )
 
 // InsertHandler processes remote write for prometheus.
-func InsertHandler(r *http.Request, maxSize int64) error {
+func InsertHandler(r *http.Request) error {
 	return concurrencylimiter.Do(func() error {
-		return insertHandlerInternal(r, maxSize)
+		return insertHandlerInternal(r)
 	})
 }
 
-func insertHandlerInternal(r *http.Request, maxSize int64) error {
+func insertHandlerInternal(r *http.Request) error {
 	ctx := getPushCtx()
 	defer putPushCtx(ctx)
-	if err := ctx.Read(r, maxSize); err != nil {
+	if err := ctx.Read(r); err != nil {
 		return err
 	}
 	timeseries := ctx.req.Timeseries
@@ -65,11 +71,11 @@ func (ctx *pushCtx) reset() {
 	ctx.reqBuf = ctx.reqBuf[:0]
 }
 
-func (ctx *pushCtx) Read(r *http.Request, maxSize int64) error {
+func (ctx *pushCtx) Read(r *http.Request) error {
 	prometheusReadCalls.Inc()
 
 	var err error
-	ctx.reqBuf, err = prompb.ReadSnappy(ctx.reqBuf[:0], r.Body, maxSize)
+	ctx.reqBuf, err = readSnappy(ctx.reqBuf[:0], r.Body)
 	if err != nil {
 		prometheusReadErrors.Inc()
 		return fmt.Errorf("cannot read prompb.WriteRequest: %s", err)
@@ -110,3 +116,35 @@ func putPushCtx(ctx *pushCtx) {
 
 var pushCtxPool sync.Pool
 var pushCtxPoolCh = make(chan *pushCtx, runtime.GOMAXPROCS(-1))
+
+func readSnappy(dst []byte, r io.Reader) ([]byte, error) {
+	lr := io.LimitReader(r, int64(*maxInsertRequestSize)+1)
+	bb := bodyBufferPool.Get()
+	reqLen, err := bb.ReadFrom(lr)
+	if err != nil {
+		bodyBufferPool.Put(bb)
+		return dst, fmt.Errorf("cannot read compressed request: %s", err)
+	}
+	if reqLen > int64(*maxInsertRequestSize) {
+		return dst, fmt.Errorf("too big packed request; mustn't exceed `-maxInsertRequestSize=%d` bytes", *maxInsertRequestSize)
+	}
+
+	buf := dst[len(dst):cap(dst)]
+	buf, err = snappy.Decode(buf, bb.B)
+	bodyBufferPool.Put(bb)
+	if err != nil {
+		err = fmt.Errorf("cannot decompress request with length %d: %s", reqLen, err)
+		return dst, err
+	}
+	if len(buf) > *maxInsertRequestSize {
+		return dst, fmt.Errorf("too big unpacked request; mustn't exceed `-maxInsertRequestSize=%d` bytes", *maxInsertRequestSize)
+	}
+	if len(buf) > 0 && len(dst) < cap(dst) && &buf[0] == &dst[len(dst):cap(dst)][0] {
+		dst = dst[:len(dst)+len(buf)]
+	} else {
+		dst = append(dst, buf...)
+	}
+	return dst, nil
+}
+
+var bodyBufferPool bytesutil.ByteBufferPool

app/vminsert/vmimport/request_handler.go

@@ -15,7 +15,7 @@ import (
 	"github.com/VictoriaMetrics/metrics"
 )
 
-var maxLineLen = flag.Int("import.maxLineLen", 100*1024*1024, "The maximum length in bytes of a single line accepted by `/api/v1/import`")
+var maxLineLen = flag.Int("import.maxLineLen", 100*1024*1024, "The maximum length in bytes of a single line accepted by /api/v1/import")
 
 var (
 	rowsInserted  = metrics.NewCounter(`vm_rows_inserted_total{type="vmimport"}`)

app/vmrestore/main.go

@@ -18,7 +18,7 @@ var (
 		"VictoriaMetrics must be stopped when restoring from backup. -storageDataPath dir can be non-empty. In this case only missing data is downloaded from backup")
 	concurrency             = flag.Int("concurrency", 10, "The number of concurrent workers. Higher concurrency may reduce restore duration")
 	maxBytesPerSecond       = flag.Int("maxBytesPerSecond", 0, "The maximum download speed. There is no limit if it is set to 0")
-	skipBackupCompleteCheck = flag.Bool("skipBackupCompleteCheck", false, "Whether to skip checking for `backup complete` file in `-src`. This may be useful for restoring from old backups, which were created without `backup complete` file")
+	skipBackupCompleteCheck = flag.Bool("skipBackupCompleteCheck", false, "Whether to skip checking for 'backup complete' file in -src. This may be useful for restoring from old backups, which were created without 'backup complete' file")
 )
 
 func main() {

app/vmselect/main.go

@@ -21,10 +21,25 @@ import (
 
 var (
 	deleteAuthKey         = flag.String("deleteAuthKey", "", "authKey for metrics' deletion via /api/v1/admin/tsdb/delete_series")
-	maxConcurrentRequests = flag.Int("search.maxConcurrentRequests", runtime.GOMAXPROCS(-1)*2, "The maximum number of concurrent search requests. It shouldn't exceed 2*vCPUs for better performance. See also -search.maxQueueDuration")
-	maxQueueDuration      = flag.Duration("search.maxQueueDuration", 10*time.Second, "The maximum time the request waits for execution when -search.maxConcurrentRequests limit is reached")
+	maxConcurrentRequests = flag.Int("search.maxConcurrentRequests", getDefaultMaxConcurrentRequests(), "The maximum number of concurrent search requests. "+
+		"It shouldn't be high, since a single request can saturate all the CPU cores. See also -search.maxQueueDuration")
+	maxQueueDuration = flag.Duration("search.maxQueueDuration", 10*time.Second, "The maximum time the request waits for execution when -search.maxConcurrentRequests limit is reached")
 )
 
+func getDefaultMaxConcurrentRequests() int {
+	n := runtime.GOMAXPROCS(-1)
+	if n <= 4 {
+		n *= 2
+	}
+	if n > 16 {
+		// A single request can saturate all the CPU cores, so there is no sense
+		// in allowing higher number of concurrent requests - they will just contend
+		// for unavailable CPU time.
+		n = 16
+	}
+	return n
+}
+
 // Init initializes vmselect
 func Init() {
 	tmpDirPath := *vmstorage.DataPath + "/tmp"
@@ -56,6 +71,7 @@ var (
 
 // RequestHandler handles remote read API requests for Prometheus
 func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
+	startTime := time.Now()
 	// Limit the number of concurrent queries.
 	select {
 	case concurrencyCh <- struct{}{}:
@@ -72,7 +88,9 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 			timerpool.Put(t)
 			concurrencyLimitTimeout.Inc()
 			err := &httpserver.ErrorWithStatusCode{
-				Err:        fmt.Errorf("cannot handle more than %d concurrent requests", cap(concurrencyCh)),
+				Err: fmt.Errorf("cannot handle more than %d concurrent search requests during %s; possible solutions: "+
+					"increase `-search.maxQueueDuration`, increase `-search.maxConcurrentRequests`, increase server capacity",
+					*maxConcurrentRequests, *maxQueueDuration),
 				StatusCode: http.StatusServiceUnavailable,
 			}
 			httpserver.Errorf(w, "%s", err)
@@ -87,7 +105,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 			labelValuesRequests.Inc()
 			labelName := s[:len(s)-len("/values")]
 			httpserver.EnableCORS(w, r)
-			if err := prometheus.LabelValuesHandler(labelName, w, r); err != nil {
+			if err := prometheus.LabelValuesHandler(startTime, labelName, w, r); err != nil {
 				labelValuesErrors.Inc()
 				sendPrometheusError(w, r, err)
 				return true
@@ -100,7 +118,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 	case "/api/v1/query":
 		queryRequests.Inc()
 		httpserver.EnableCORS(w, r)
-		if err := prometheus.QueryHandler(w, r); err != nil {
+		if err := prometheus.QueryHandler(startTime, w, r); err != nil {
 			queryErrors.Inc()
 			sendPrometheusError(w, r, err)
 			return true
@@ -109,7 +127,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 	case "/api/v1/query_range":
 		queryRangeRequests.Inc()
 		httpserver.EnableCORS(w, r)
-		if err := prometheus.QueryRangeHandler(w, r); err != nil {
+		if err := prometheus.QueryRangeHandler(startTime, w, r); err != nil {
 			queryRangeErrors.Inc()
 			sendPrometheusError(w, r, err)
 			return true
@@ -118,7 +136,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 	case "/api/v1/series":
 		seriesRequests.Inc()
 		httpserver.EnableCORS(w, r)
-		if err := prometheus.SeriesHandler(w, r); err != nil {
+		if err := prometheus.SeriesHandler(startTime, w, r); err != nil {
 			seriesErrors.Inc()
 			sendPrometheusError(w, r, err)
 			return true
@@ -127,7 +145,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 	case "/api/v1/series/count":
 		seriesCountRequests.Inc()
 		httpserver.EnableCORS(w, r)
-		if err := prometheus.SeriesCountHandler(w, r); err != nil {
+		if err := prometheus.SeriesCountHandler(startTime, w, r); err != nil {
 			seriesCountErrors.Inc()
 			sendPrometheusError(w, r, err)
 			return true
@@ -136,7 +154,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 	case "/api/v1/labels":
 		labelsRequests.Inc()
 		httpserver.EnableCORS(w, r)
-		if err := prometheus.LabelsHandler(w, r); err != nil {
+		if err := prometheus.LabelsHandler(startTime, w, r); err != nil {
 			labelsErrors.Inc()
 			sendPrometheusError(w, r, err)
 			return true
@@ -145,7 +163,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 	case "/api/v1/labels/count":
 		labelsCountRequests.Inc()
 		httpserver.EnableCORS(w, r)
-		if err := prometheus.LabelsCountHandler(w, r); err != nil {
+		if err := prometheus.LabelsCountHandler(startTime, w, r); err != nil {
 			labelsCountErrors.Inc()
 			sendPrometheusError(w, r, err)
 			return true
@@ -153,7 +171,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 		return true
 	case "/api/v1/export":
 		exportRequests.Inc()
-		if err := prometheus.ExportHandler(w, r); err != nil {
+		if err := prometheus.ExportHandler(startTime, w, r); err != nil {
 			exportErrors.Inc()
 			httpserver.Errorf(w, "error in %q: %s", r.URL.Path, err)
 			return true
@@ -161,7 +179,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 		return true
 	case "/federate":
 		federateRequests.Inc()
-		if err := prometheus.FederateHandler(w, r); err != nil {
+		if err := prometheus.FederateHandler(startTime, w, r); err != nil {
 			federateErrors.Inc()
 			httpserver.Errorf(w, "error int %q: %s", r.URL.Path, err)
 			return true
@@ -179,6 +197,12 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 		w.Header().Set("Content-Type", "application/json")
 		fmt.Fprintf(w, "%s", `{"status":"success","data":{"alerts":[]}}`)
 		return true
+	case "/api/v1/metadata":
+		// Return dumb placeholder
+		metadataRequests.Inc()
+		w.Header().Set("Content-Type", "application/json")
+		fmt.Fprintf(w, "%s", `{"status":"success","data":{}}`)
+		return true
 	case "/api/v1/admin/tsdb/delete_series":
 		deleteRequests.Inc()
 		authKey := r.FormValue("authKey")
@@ -186,7 +210,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 			httpserver.Errorf(w, "invalid authKey %q. It must match the value from -deleteAuthKey command line flag", authKey)
 			return true
 		}
-		if err := prometheus.DeleteHandler(r); err != nil {
+		if err := prometheus.DeleteHandler(startTime, r); err != nil {
 			deleteErrors.Inc()
 			httpserver.Errorf(w, "error in %q: %s", r.URL.Path, err)
 			return true
@@ -199,7 +223,7 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
 }
 
 func sendPrometheusError(w http.ResponseWriter, r *http.Request, err error) {
-	logger.Errorf("error in %q: %s", r.URL.Path, err)
+	logger.Errorf("error in %q: %s", r.RequestURI, err)
 
 	w.Header().Set("Content-Type", "application/json")
 	statusCode := http.StatusUnprocessableEntity
@@ -241,6 +265,7 @@ var (
 	federateRequests = metrics.NewCounter(`vm_http_requests_total{path="/federate"}`)
 	federateErrors   = metrics.NewCounter(`vm_http_request_errors_total{path="/federate"}`)
 
-	rulesRequests  = metrics.NewCounter(`vm_http_requests_total{path="/api/v1/rules"}`)
-	alertsRequests = metrics.NewCounter(`vm_http_requests_total{path="/api/v1/alerts"}`)
+	rulesRequests    = metrics.NewCounter(`vm_http_requests_total{path="/api/v1/rules"}`)
+	alertsRequests   = metrics.NewCounter(`vm_http_requests_total{path="/api/v1/alerts"}`)
+	metadataRequests = metrics.NewCounter(`vm_http_requests_total{path="/api/v1/metadata"}`)
 )

app/vmselect/netstorage/netstorage.go

@@ -103,7 +103,7 @@ func (rss *Results) RunParallel(f func(rs *Result, workerID uint)) error {
 			rowsProcessed := 0
 			for pts := range workCh {
 				if time.Until(rss.deadline.Deadline) < 0 {
-					err = fmt.Errorf("timeout exceeded during query execution: %s", rss.deadline.Timeout)
+					err = fmt.Errorf("timeout exceeded during query execution: %s", rss.deadline.String())
 					break
 				}
 				if err = pts.Unpack(rss.tbf, rs, rss.tr, rss.fetchData, maxWorkersCount); err != nil {
@@ -266,7 +266,7 @@ func mergeSortBlocks(dst *Result, sbh sortBlocksHeap) {
 			dst.Timestamps = append(dst.Timestamps, top.Timestamps[top.NextIdx:]...)
 			dst.Values = append(dst.Values, top.Values[top.NextIdx:]...)
 			putSortBlock(top)
-			return
+			break
 		}
 		sbNext := sbh[0]
 		tsNext := sbNext.Timestamps[sbNext.NextIdx]
@@ -287,6 +287,8 @@ func mergeSortBlocks(dst *Result, sbh sortBlocksHeap) {
 			putSortBlock(top)
 		}
 	}
+
+	dst.Timestamps, dst.Values = storage.DeduplicateSamples(dst.Timestamps, dst.Values)
 }
 
 type sortBlock struct {
@@ -499,7 +501,7 @@ func ProcessSearchQuery(sq *storage.SearchQuery, fetchData bool, deadline Deadli
 		}
 		if time.Until(deadline.Deadline) < 0 {
 			putTmpBlocksFile(tbf)
-			return nil, fmt.Errorf("timeout exceeded while fetching data block #%d from storage: %s", blocksRead, deadline.Timeout)
+			return nil, fmt.Errorf("timeout exceeded while fetching data block #%d from storage: %s", blocksRead, deadline.String())
 		}
 		metricName := sr.MetricBlock.MetricName
 		m[string(metricName)] = append(m[string(metricName)], addr)
@@ -575,13 +577,24 @@ func setupTfss(tagFilterss [][]storage.TagFilter) ([]*storage.TagFilters, error)
 // Deadline contains deadline with the corresponding timeout for pretty error messages.
 type Deadline struct {
 	Deadline time.Time
-	Timeout  time.Duration
+
+	timeout  time.Duration
+	flagHint string
 }
 
 // NewDeadline returns deadline for the given timeout.
-func NewDeadline(timeout time.Duration) Deadline {
+//
+// flagHint must contain a hit for command-line flag, which could be used
+// in order to increase timeout.
+func NewDeadline(timeout time.Duration, flagHint string) Deadline {
 	return Deadline{
 		Deadline: time.Now().Add(timeout),
-		Timeout:  timeout,
+		timeout:  timeout,
+		flagHint: flagHint,
 	}
 }
+
+// String returns human-readable string representation for d.
+func (d *Deadline) String() string {
+	return fmt.Sprintf("%.3f seconds; the timeout can be adjusted with `%s` command-line flag", d.timeout.Seconds(), d.flagHint)
+}

app/vmselect/netstorage/tmp_blocks_file.go

@@ -36,6 +36,9 @@ func maxInmemoryTmpBlocksFile() int {
 	if maxLen < 64*1024 {
 		return 64 * 1024
 	}
+	if maxLen > 4*1024*1024 {
+		return 4 * 1024 * 1024
+	}
 	return maxLen
 }
 
@@ -47,6 +50,7 @@ type tmpBlocksFile struct {
 	buf []byte
 
 	f *os.File
+	r *fs.ReaderAt
 
 	offset uint64
 }
@@ -65,6 +69,7 @@ func putTmpBlocksFile(tbf *tmpBlocksFile) {
 	tbf.MustClose()
 	tbf.buf = tbf.buf[:0]
 	tbf.f = nil
+	tbf.r = nil
 	tbf.offset = 0
 	tmpBlocksFilePool.Put(tbf)
 }
@@ -118,17 +123,20 @@ func (tbf *tmpBlocksFile) Finalize() error {
 	if tbf.f == nil {
 		return nil
 	}
+	fname := tbf.f.Name()
 	if _, err := tbf.f.Write(tbf.buf); err != nil {
-		return fmt.Errorf("cannot flush the remaining %d bytes to tmpBlocksFile: %s", len(tbf.buf), err)
+		return fmt.Errorf("cannot write the remaining %d bytes to %q: %s", len(tbf.buf), fname, err)
 	}
 	tbf.buf = tbf.buf[:0]
-	if _, err := tbf.f.Seek(0, 0); err != nil {
-		logger.Panicf("FATAL: cannot seek to the start of file: %s", err)
+	r, err := fs.OpenReaderAt(fname)
+	if err != nil {
+		logger.Panicf("FATAL: cannot open %q: %s", fname, err)
 	}
 	// Hint the OS that the file is read almost sequentiallly.
 	// This should reduce the number of disk seeks, which is important
 	// for HDDs.
-	fs.MustFadviseSequentialRead(tbf.f, true)
+	r.MustFadviseSequentialRead(true)
+	tbf.r = r
 	return nil
 }
 
@@ -140,13 +148,7 @@ func (tbf *tmpBlocksFile) MustReadBlockAt(dst *storage.Block, addr tmpBlockAddr)
 		bb := tmpBufPool.Get()
 		defer tmpBufPool.Put(bb)
 		bb.B = bytesutil.Resize(bb.B, addr.size)
-		n, err := tbf.f.ReadAt(bb.B, int64(addr.offset))
-		if err != nil {
-			logger.Panicf("FATAL: cannot read from %q at %s: %s", tbf.f.Name(), addr, err)
-		}
-		if n != len(bb.B) {
-			logger.Panicf("FATAL: too short number of bytes read at %s; got %d; want %d", addr, n, len(bb.B))
-		}
+		tbf.r.MustReadAt(bb.B, int64(addr.offset))
 		buf = bb.B
 	}
 	tail, err := storage.UnmarshalBlock(dst, buf)
@@ -164,6 +166,10 @@ func (tbf *tmpBlocksFile) MustClose() {
 	if tbf.f == nil {
 		return
 	}
+	if tbf.r != nil {
+		// tbf.r could be nil if Finalize wasn't called.
+		tbf.r.MustClose()
+	}
 	fname := tbf.f.Name()
 
 	// Remove the file at first, then close it.

app/vmselect/prometheus/prometheus.go

@@ -24,19 +24,18 @@ import (
 var (
 	latencyOffset = flag.Duration("search.latencyOffset", time.Second*30, "The time when data points become visible in query results after the colection. "+
 		"Too small value can result in incomplete last points for query results")
-	maxExportDuration = flag.Duration("search.maxExportDuration", 10*time.Minute, "The maximum duration for `/api/v1/export` call")
+	maxExportDuration = flag.Duration("search.maxExportDuration", time.Hour*24*30, "The maximum duration for /api/v1/export call")
 	maxQueryDuration  = flag.Duration("search.maxQueryDuration", time.Second*30, "The maximum duration for search query execution")
 	maxQueryLen       = flag.Int("search.maxQueryLen", 16*1024, "The maximum search query length in bytes")
-	maxLookback       = flag.Duration("search.maxLookback", 0, "Synonim to `-search.lookback-delta` from Prometheus. "+
-		"The value is dynamically detected from interval between time series datapoints if not set. It can be overridden on per-query basis via `max_lookback` arg")
+	maxLookback       = flag.Duration("search.maxLookback", 0, "Synonim to -search.lookback-delta from Prometheus. "+
+		"The value is dynamically detected from interval between time series datapoints if not set. It can be overridden on per-query basis via max_lookback arg")
 )
 
 // Default step used if not set.
 const defaultStep = 5 * 60 * 1000
 
 // FederateHandler implements /federate . See https://prometheus.io/docs/prometheus/latest/federation/
-func FederateHandler(w http.ResponseWriter, r *http.Request) error {
-	startTime := time.Now()
+func FederateHandler(startTime time.Time, w http.ResponseWriter, r *http.Request) error {
 	ct := currentTime()
 	if err := r.ParseForm(); err != nil {
 		return fmt.Errorf("cannot parse request form values: %s", err)
@@ -107,8 +106,7 @@ func FederateHandler(w http.ResponseWriter, r *http.Request) error {
 var federateDuration = metrics.NewSummary(`vm_request_duration_seconds{path="/federate"}`)
 
 // ExportHandler exports data in raw format from /api/v1/export.
-func ExportHandler(w http.ResponseWriter, r *http.Request) error {
-	startTime := time.Now()
+func ExportHandler(startTime time.Time, w http.ResponseWriter, r *http.Request) error {
 	ct := currentTime()
 	if err := r.ParseForm(); err != nil {
 		return fmt.Errorf("cannot parse request form values: %s", err)
@@ -136,7 +134,7 @@ func ExportHandler(w http.ResponseWriter, r *http.Request) error {
 		end = start + defaultStep
 	}
 	if err := exportHandler(w, matches, start, end, format, deadline); err != nil {
-		return err
+		return fmt.Errorf("error when exporting data for queries=%q on the time range (start=%d, end=%d): %s", matches, start, end, err)
 	}
 	exportDuration.UpdateDuration(startTime)
 	return nil
@@ -200,8 +198,7 @@ func exportHandler(w http.ResponseWriter, matches []string, start, end int64, fo
 // DeleteHandler processes /api/v1/admin/tsdb/delete_series prometheus API request.
 //
 // See https://prometheus.io/docs/prometheus/latest/querying/api/#delete-series
-func DeleteHandler(r *http.Request) error {
-	startTime := time.Now()
+func DeleteHandler(startTime time.Time, r *http.Request) error {
 	if err := r.ParseForm(); err != nil {
 		return fmt.Errorf("cannot parse request form values: %s", err)
 	}
@@ -235,8 +232,7 @@ var deleteDuration = metrics.NewSummary(`vm_request_duration_seconds{path="/api/
 // LabelValuesHandler processes /api/v1/label/<labelName>/values request.
 //
 // See https://prometheus.io/docs/prometheus/latest/querying/api/#querying-label-values
-func LabelValuesHandler(labelName string, w http.ResponseWriter, r *http.Request) error {
-	startTime := time.Now()
+func LabelValuesHandler(startTime time.Time, labelName string, w http.ResponseWriter, r *http.Request) error {
 	deadline := getDeadlineForQuery(r)
 
 	if err := r.ParseForm(); err != nil {
@@ -333,8 +329,7 @@ func labelValuesWithMatches(labelName string, matches []string, start, end int64
 var labelValuesDuration = metrics.NewSummary(`vm_request_duration_seconds{path="/api/v1/label/{}/values"}`)
 
 // LabelsCountHandler processes /api/v1/labels/count request.
-func LabelsCountHandler(w http.ResponseWriter, r *http.Request) error {
-	startTime := time.Now()
+func LabelsCountHandler(startTime time.Time, w http.ResponseWriter, r *http.Request) error {
 	deadline := getDeadlineForQuery(r)
 	labelEntries, err := netstorage.GetLabelEntries(deadline)
 	if err != nil {
@@ -352,8 +347,7 @@ var labelsCountDuration = metrics.NewSummary(`vm_request_duration_seconds{path="
 // LabelsHandler processes /api/v1/labels request.
 //
 // See https://prometheus.io/docs/prometheus/latest/querying/api/#getting-label-names
-func LabelsHandler(w http.ResponseWriter, r *http.Request) error {
-	startTime := time.Now()
+func LabelsHandler(startTime time.Time, w http.ResponseWriter, r *http.Request) error {
 	deadline := getDeadlineForQuery(r)
 
 	if err := r.ParseForm(); err != nil {
@@ -442,8 +436,7 @@ func labelsWithMatches(matches []string, start, end int64, deadline netstorage.D
 var labelsDuration = metrics.NewSummary(`vm_request_duration_seconds{path="/api/v1/labels"}`)
 
 // SeriesCountHandler processes /api/v1/series/count request.
-func SeriesCountHandler(w http.ResponseWriter, r *http.Request) error {
-	startTime := time.Now()
+func SeriesCountHandler(startTime time.Time, w http.ResponseWriter, r *http.Request) error {
 	deadline := getDeadlineForQuery(r)
 	n, err := netstorage.GetSeriesCount(deadline)
 	if err != nil {
@@ -460,8 +453,7 @@ var seriesCountDuration = metrics.NewSummary(`vm_request_duration_seconds{path="
 // SeriesHandler processes /api/v1/series request.
 //
 // See https://prometheus.io/docs/prometheus/latest/querying/api/#finding-series-by-label-matchers
-func SeriesHandler(w http.ResponseWriter, r *http.Request) error {
-	startTime := time.Now()
+func SeriesHandler(startTime time.Time, w http.ResponseWriter, r *http.Request) error {
 	ct := currentTime()
 
 	if err := r.ParseForm(); err != nil {
@@ -536,8 +528,7 @@ var seriesDuration = metrics.NewSummary(`vm_request_duration_seconds{path="/api/
 // QueryHandler processes /api/v1/query request.
 //
 // See https://prometheus.io/docs/prometheus/latest/querying/api/#instant-queries
-func QueryHandler(w http.ResponseWriter, r *http.Request) error {
-	startTime := time.Now()
+func QueryHandler(startTime time.Time, w http.ResponseWriter, r *http.Request) error {
 	ct := currentTime()
 
 	query := r.FormValue("query")
@@ -560,7 +551,7 @@ func QueryHandler(w http.ResponseWriter, r *http.Request) error {
 	}
 
 	if len(query) > *maxQueryLen {
-		return fmt.Errorf(`too long query; got %d bytes; mustn't exceed %d bytes`, len(query), *maxQueryLen)
+		return fmt.Errorf("too long query; got %d bytes; mustn't exceed `-search.maxQueryLen=%d` bytes", len(query), *maxQueryLen)
 	}
 	if !getBool(r, "nocache") && ct-start < queryOffset {
 		// Adjust start time only if `nocache` arg isn't set.
@@ -580,7 +571,7 @@ func QueryHandler(w http.ResponseWriter, r *http.Request) error {
 		end := start
 		start = end - window
 		if err := exportHandler(w, []string{childQuery}, start, end, "promapi", deadline); err != nil {
-			return err
+			return fmt.Errorf("error when exporting data for query=%q on the time range (start=%d, end=%d): %s", childQuery, start, end, err)
 		}
 		queryDuration.UpdateDuration(startTime)
 		return nil
@@ -605,7 +596,7 @@ func QueryHandler(w http.ResponseWriter, r *http.Request) error {
 		end := start
 		start = end - window
 		if err := queryRangeHandler(w, childQuery, start, end, step, r, ct); err != nil {
-			return err
+			return fmt.Errorf("error when executing query=%q on the time range (start=%d, end=%d, step=%d): %s", childQuery, start, end, step, err)
 		}
 		queryDuration.UpdateDuration(startTime)
 		return nil
@@ -620,7 +611,7 @@ func QueryHandler(w http.ResponseWriter, r *http.Request) error {
 	}
 	result, err := promql.Exec(&ec, query, true)
 	if err != nil {
-		return fmt.Errorf("cannot execute %q: %s", query, err)
+		return fmt.Errorf("error when executing query=%q for (time=%d, step=%d): %s", query, start, step, err)
 	}
 
 	w.Header().Set("Content-Type", "application/json")
@@ -648,8 +639,7 @@ func parsePositiveDuration(s string, step int64) (int64, error) {
 // QueryRangeHandler processes /api/v1/query_range request.
 //
 // See https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries
-func QueryRangeHandler(w http.ResponseWriter, r *http.Request) error {
-	startTime := time.Now()
+func QueryRangeHandler(startTime time.Time, w http.ResponseWriter, r *http.Request) error {
 	ct := currentTime()
 
 	query := r.FormValue("query")
@@ -669,7 +659,7 @@ func QueryRangeHandler(w http.ResponseWriter, r *http.Request) error {
 		return err
 	}
 	if err := queryRangeHandler(w, query, start, end, step, r, ct); err != nil {
-		return err
+		return fmt.Errorf("error when executing query=%q on the time range (start=%d, end=%d, step=%d): %s", query, start, end, step, err)
 	}
 	queryRangeDuration.UpdateDuration(startTime)
 	return nil
@@ -685,7 +675,7 @@ func queryRangeHandler(w http.ResponseWriter, query string, start, end, step int
 
 	// Validate input args.
 	if len(query) > *maxQueryLen {
-		return fmt.Errorf(`too long query; got %d bytes; mustn't exceed %d bytes`, len(query), *maxQueryLen)
+		return fmt.Errorf("too long query; got %d bytes; mustn't exceed `-search.maxQueryLen=%d` bytes", len(query), *maxQueryLen)
 	}
 	if start > end {
 		end = start + defaultStep
@@ -707,7 +697,7 @@ func queryRangeHandler(w http.ResponseWriter, query string, start, end, step int
 	}
 	result, err := promql.Exec(&ec, query, false)
 	if err != nil {
-		return fmt.Errorf("cannot execute %q: %s", query, err)
+		return fmt.Errorf("cannot execute query: %s", err)
 	}
 	queryOffset := getLatencyOffsetMilliseconds()
 	if ct-end < queryOffset {
@@ -870,15 +860,15 @@ func getMaxLookback(r *http.Request) (int64, error) {
 
 func getDeadlineForQuery(r *http.Request) netstorage.Deadline {
 	dMax := int64(maxQueryDuration.Seconds() * 1e3)
-	return getDeadlineWithMaxDuration(r, dMax)
+	return getDeadlineWithMaxDuration(r, dMax, "-search.maxQueryDuration")
 }
 
 func getDeadlineForExport(r *http.Request) netstorage.Deadline {
 	dMax := int64(maxExportDuration.Seconds() * 1e3)
-	return getDeadlineWithMaxDuration(r, dMax)
+	return getDeadlineWithMaxDuration(r, dMax, "-search.maxExportDuration")
 }
 
-func getDeadlineWithMaxDuration(r *http.Request, dMax int64) netstorage.Deadline {
+func getDeadlineWithMaxDuration(r *http.Request, dMax int64, flagHint string) netstorage.Deadline {
 	d, err := getDuration(r, "timeout", 0)
 	if err != nil {
 		d = 0
@@ -887,7 +877,7 @@ func getDeadlineWithMaxDuration(r *http.Request, dMax int64) netstorage.Deadline
 		d = dMax
 	}
 	timeout := time.Duration(d) * time.Millisecond
-	return netstorage.NewDeadline(timeout)
+	return netstorage.NewDeadline(timeout, flagHint)
 }
 
 func getBool(r *http.Request, argKey string) bool {

app/vmselect/promql/binary_op.go

@@ -32,7 +32,7 @@ var binaryOpFuncs = map[string]binaryOpFunc{
 	"or":     binaryOpOr,
 	"unless": binaryOpUnless,
 
-	// New op
+	// New ops
 	"if":      newBinaryOpArithFunc(binaryop.If),
 	"ifnot":   newBinaryOpArithFunc(binaryop.Ifnot),
 	"default": newBinaryOpArithFunc(binaryop.Default),
@@ -285,10 +285,21 @@ func resetMetricGroupIfRequired(be *metricsql.BinaryOpExpr, ts *timeseries) {
 func binaryOpAnd(bfa *binaryOpFuncArg) ([]*timeseries, error) {
 	mLeft, mRight := createTimeseriesMapByTagSet(bfa.be, bfa.left, bfa.right)
 	var rvs []*timeseries
-	for k := range mRight {
-		if tss := mLeft[k]; tss != nil {
-			rvs = append(rvs, tss...)
+	for k, tssRight := range mRight {
+		tssLeft := mLeft[k]
+		if tssLeft == nil {
+			continue
 		}
+		for i := range tssLeft[0].Values {
+			if !isAllNaNs(tssRight, i) {
+				continue
+			}
+			for _, tsLeft := range tssLeft {
+				tsLeft.Values[i] = nan
+			}
+		}
+		tssLeft = removeNaNs(tssLeft)
+		rvs = append(rvs, tssLeft...)
 	}
 	return rvs, nil
 }
@@ -310,14 +321,35 @@ func binaryOpOr(bfa *binaryOpFuncArg) ([]*timeseries, error) {
 func binaryOpUnless(bfa *binaryOpFuncArg) ([]*timeseries, error) {
 	mLeft, mRight := createTimeseriesMapByTagSet(bfa.be, bfa.left, bfa.right)
 	var rvs []*timeseries
-	for k, tss := range mLeft {
-		if mRight[k] == nil {
-			rvs = append(rvs, tss...)
+	for k, tssLeft := range mLeft {
+		tssRight := mRight[k]
+		if tssRight == nil {
+			rvs = append(rvs, tssLeft...)
+			continue
 		}
+		for i := range tssLeft[0].Values {
+			if isAllNaNs(tssRight, i) {
+				continue
+			}
+			for _, tsLeft := range tssLeft {
+				tsLeft.Values[i] = nan
+			}
+		}
+		tssLeft = removeNaNs(tssLeft)
+		rvs = append(rvs, tssLeft...)
 	}
 	return rvs, nil
 }
 
+func isAllNaNs(tss []*timeseries, idx int) bool {
+	for _, ts := range tss {
+		if !math.IsNaN(ts.Values[idx]) {
+			return false
+		}
+	}
+	return true
+}
+
 func createTimeseriesMapByTagSet(be *metricsql.BinaryOpExpr, left, right []*timeseries) (map[string][]*timeseries, map[string][]*timeseries) {
 	groupTags := be.GroupModifier.Args
 	groupOp := strings.ToLower(be.GroupModifier.Op)

app/vmselect/promql/eval.go

@@ -323,6 +323,10 @@ func tryGetArgRollupFuncWithMetricExpr(ae *metricsql.AggrFuncExpr) (*metricsql.F
 		return nil, nil
 	}
 	rollupArgIdx := getRollupArgIdx(fe.Name)
+	if rollupArgIdx >= len(fe.Args) {
+		// Incorrect number of args for rollup func.
+		return nil, nil
+	}
 	arg := fe.Args[rollupArgIdx]
 	if me, ok := arg.(*metricsql.MetricExpr); ok {
 		if me.IsEmpty() {
@@ -360,7 +364,7 @@ func evalRollupFuncArgs(ec *EvalConfig, fe *metricsql.FuncExpr) ([]interface{},
 	var re *metricsql.RollupExpr
 	rollupArgIdx := getRollupArgIdx(fe.Name)
 	if len(fe.Args) <= rollupArgIdx {
-		return nil, nil, fmt.Errorf("expecting at least %d args to %q; got %d args; expr: %q", rollupArgIdx, fe.Name, len(fe.Args), fe.AppendString(nil))
+		return nil, nil, fmt.Errorf("expecting at least %d args to %q; got %d args; expr: %q", rollupArgIdx+1, fe.Name, len(fe.Args), fe.AppendString(nil))
 	}
 	args := make([]interface{}, len(fe.Args))
 	for i, arg := range fe.Args {
@@ -415,7 +419,7 @@ func evalRollupFunc(ec *EvalConfig, name string, rf rollupFunc, expr metricsql.E
 		if err != nil {
 			return nil, err
 		}
-		ecNew = newEvalConfig(ec)
+		ecNew = newEvalConfig(ecNew)
 		ecNew.Start -= offset
 		ecNew.End -= offset
 		if ecNew.MayCache {
@@ -425,6 +429,16 @@ func evalRollupFunc(ec *EvalConfig, name string, rf rollupFunc, expr metricsql.E
 			ecNew.End = end
 		}
 	}
+	if name == "rollup_candlestick" {
+		// Automatically apply `offset -step` to `rollup_candlestick` function
+		// in order to obtain expected OHLC results.
+		// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/309#issuecomment-582113462
+		step := ecNew.Step
+		ecNew = newEvalConfig(ecNew)
+		ecNew.Start += step
+		ecNew.End += step
+		offset -= step
+	}
 	var rvs []*timeseries
 	var err error
 	if me, ok := re.Expr.(*metricsql.MetricExpr); ok {
@@ -605,6 +619,14 @@ func evalRollupFuncWithMetricExpr(ec *EvalConfig, name string, rf rollupFunc,
 		rollupResultCacheMiss.Inc()
 	}
 
+	// Obtain rollup configs before fetching data from db,
+	// so type errors can be caught earlier.
+	sharedTimestamps := getTimestamps(start, ec.End, ec.Step)
+	preFunc, rcs, err := getRollupConfigs(name, rf, expr, start, ec.End, ec.Step, window, ec.LookbackDelta, sharedTimestamps)
+	if err != nil {
+		return nil, err
+	}
+
 	// Fetch the remaining part of the result.
 	tfs := toTagFilters(me.LabelFilters)
 	sq := &storage.SearchQuery{
@@ -629,12 +651,6 @@ func evalRollupFuncWithMetricExpr(ec *EvalConfig, name string, rf rollupFunc,
 		tss = mergeTimeseries(tssCached, tss, start, ec)
 		return tss, nil
 	}
-	sharedTimestamps := getTimestamps(start, ec.End, ec.Step)
-	preFunc, rcs, err := getRollupConfigs(name, rf, expr, start, ec.End, ec.Step, window, ec.LookbackDelta, sharedTimestamps)
-	if err != nil {
-		rss.Cancel()
-		return nil, err
-	}
 
 	// Verify timeseries fit available memory after the rollup.
 	// Take into account points from tssCached.

app/vmselect/promql/exec.go

@@ -26,8 +26,8 @@ func Exec(ec *EvalConfig, q string, isFirstPointOnly bool) ([]netstorage.Result,
 		defer func() {
 			d := time.Since(startTime)
 			if d >= *logSlowQueryDuration {
-				logger.Infof("slow query according to -search.logSlowQueryDuration=%s: duration=%s, start=%d, end=%d, step=%d, query=%q",
-					*logSlowQueryDuration, d, ec.Start/1000, ec.End/1000, ec.Step/1000, q)
+				logger.Infof("slow query according to -search.logSlowQueryDuration=%s: duration=%.3f seconds, start=%d, end=%d, step=%d, query=%q",
+					*logSlowQueryDuration, d.Seconds(), ec.Start/1000, ec.End/1000, ec.Step/1000, q)
 				slowQueries.Inc()
 			}
 		}()

app/vmselect/promql/exec_test.go

@@ -21,7 +21,7 @@ func TestExecSuccess(t *testing.T) {
 			Start:    start,
 			End:      end,
 			Step:     step,
-			Deadline: netstorage.NewDeadline(time.Minute),
+			Deadline: netstorage.NewDeadline(time.Minute, ""),
 		}
 		for i := 0; i < 5; i++ {
 			result, err := Exec(ec, q, false)
@@ -1466,6 +1466,38 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r}
 		f(q, resultExpected)
 	})
+	t.Run(`label_match()`, func(t *testing.T) {
+		t.Parallel()
+		q := `
+		label_match((
+			alias(time(), "foo"),
+			alias(2*time(), "bar"),
+		), "__name__", "f.+")`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{1000, 1200, 1400, 1600, 1800, 2000},
+			Timestamps: timestampsExpected,
+		}
+		r.MetricName.MetricGroup = []byte("foo")
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
+	t.Run(`label_mismatch()`, func(t *testing.T) {
+		t.Parallel()
+		q := `
+		label_mismatch((
+			alias(time(), "foo"),
+			alias(2*time(), "bar"),
+		), "__name__", "f.+")`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{2000, 2400, 2800, 3200, 3600, 4000},
+			Timestamps: timestampsExpected,
+		}
+		r.MetricName.MetricGroup = []byte("bar")
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
 	t.Run(`two_timeseries`, func(t *testing.T) {
 		t.Parallel()
 		q := `sort_desc(time() or label_set(2, "xx", "foo"))`
@@ -1688,12 +1720,34 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r}
 		f(q, resultExpected)
 	})
+	t.Run(`time() and time() > 1300`, func(t *testing.T) {
+		t.Parallel()
+		q := `time() and time() > 1300`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{nan, nan, 1400, 1600, 1800, 2000},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
 	t.Run(`time() unless 2`, func(t *testing.T) {
 		t.Parallel()
 		q := `time() unless 2`
 		resultExpected := []netstorage.Result{}
 		f(q, resultExpected)
 	})
+	t.Run(`time() unless time() > 1500`, func(t *testing.T) {
+		t.Parallel()
+		q := `time() unless time() > 1500`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{1000, 1200, 1400, nan, nan, nan},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
 	t.Run(`timseries-with-tags unless 2`, func(t *testing.T) {
 		t.Parallel()
 		q := `label_set(time(), "foo", "bar") unless 2`
@@ -2922,6 +2976,23 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r1, r2, r3, r4, r5, r6}
 		f(q, resultExpected)
 	})
+	t.Run(`prometheus_buckets(zero-vmrange-value)`, func(t *testing.T) {
+		t.Parallel()
+		q := `sort(prometheus_buckets(label_set(0, "vmrange", "0...0")))`
+		r1 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0, 0, 0, 0, 0, 0},
+			Timestamps: timestampsExpected,
+		}
+		r1.MetricName.Tags = []storage.Tag{
+			{
+				Key:   []byte("le"),
+				Value: []byte("+Inf"),
+			},
+		}
+		resultsExpected := []netstorage.Result{r1}
+		f(q, resultsExpected)
+	})
 	t.Run(`prometheus_buckets(valid)`, func(t *testing.T) {
 		t.Parallel()
 		q := `sort(prometheus_buckets((
@@ -3216,6 +3287,17 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r}
 		f(q, resultExpected)
 	})
+	t.Run(`range_over_time(time)`, func(t *testing.T) {
+		t.Parallel()
+		q := `range_over_time(alias(time()/100, "foobar")[3i])`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{4, 4, 4, 4, 4, 4},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
 	t.Run(`sum(multi-vector)`, func(t *testing.T) {
 		t.Parallel()
 		q := `sum(label_set(10, "foo", "bar") or label_set(time()/100, "baz", "sss"))`
@@ -3844,6 +3926,22 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r1}
 		f(q, resultExpected)
 	})
+	t.Run(`keep_next_value()`, func(t *testing.T) {
+		t.Parallel()
+		q := `keep_next_value(label_set(time() < 1300 default time() > 1700, "__name__", "foobar", "x", "y"))`
+		r1 := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{1000, 1200, 1800, 1800, 1800, 2000},
+			Timestamps: timestampsExpected,
+		}
+		r1.MetricName.MetricGroup = []byte("foobar")
+		r1.MetricName.Tags = []storage.Tag{{
+			Key:   []byte("x"),
+			Value: []byte("y"),
+		}}
+		resultExpected := []netstorage.Result{r1}
+		f(q, resultExpected)
+	})
 	t.Run(`distinct_over_time([500s])`, func(t *testing.T) {
 		t.Parallel()
 		q := `distinct_over_time((time() < 1700)[500s])`
@@ -4480,6 +4578,29 @@ func TestExecSuccess(t *testing.T) {
 		resultExpected := []netstorage.Result{r}
 		f(q, resultExpected)
 	})
+	t.Run(`hoeffding_bound_lower()`, func(t *testing.T) {
+		t.Parallel()
+		q := `hoeffding_bound_lower(0.9, rand(0)[:10s])`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0.2516770508510652, 0.2830570387745462, 0.27716232108436645, 0.3679356319931767, 0.3168460474120903, 0.23156726248243734},
+			Timestamps: timestampsExpected,
+		}
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
+	t.Run(`hoeffding_bound_upper()`, func(t *testing.T) {
+		t.Parallel()
+		q := `hoeffding_bound_upper(0.9, alias(rand(0), "foobar")[:10s])`
+		r := netstorage.Result{
+			MetricName: metricNameExpected,
+			Values:     []float64{0.6510581320042821, 0.7261021731890429, 0.7245290097397009, 0.8113950442584258, 0.7736122275568004, 0.6658564048254882},
+			Timestamps: timestampsExpected,
+		}
+		r.MetricName.MetricGroup = []byte("foobar")
+		resultExpected := []netstorage.Result{r}
+		f(q, resultExpected)
+	})
 	t.Run(`aggr_over_time(single-func)`, func(t *testing.T) {
 		t.Parallel()
 		q := `aggr_over_time("increase", rand(0)[:10s])`
@@ -4577,25 +4698,25 @@ func TestExecSuccess(t *testing.T) {
 		}}
 		r2 := netstorage.Result{
 			MetricName: metricNameExpected,
-			Values:     []float64{0.85, 0.15, 0.43, 0.76, 0.47, 0.21},
+			Values:     []float64{0.1, 0.04, 0.49, 0.46, 0.57, 0.92},
 			Timestamps: timestampsExpected,
 		}
 		r2.MetricName.Tags = []storage.Tag{{
 			Key:   []byte("rollup"),
-			Value: []byte("open"),
+			Value: []byte("close"),
 		}}
 		r3 := netstorage.Result{
 			MetricName: metricNameExpected,
-			Values:     []float64{0.32, 0.82, 0.13, 0.28, 0.86, 0.57},
+			Values:     []float64{0.9, 0.32, 0.82, 0.13, 0.28, 0.86},
 			Timestamps: timestampsExpected,
 		}
 		r3.MetricName.Tags = []storage.Tag{{
 			Key:   []byte("rollup"),
-			Value: []byte("close"),
+			Value: []byte("open"),
 		}}
 		r4 := netstorage.Result{
 			MetricName: metricNameExpected,
-			Values:     []float64{0.85, 0.94, 0.97, 0.93, 0.98, 0.92},
+			Values:     []float64{0.9, 0.94, 0.97, 0.93, 0.98, 0.92},
 			Timestamps: timestampsExpected,
 		}
 		r4.MetricName.Tags = []storage.Tag{{
@@ -5163,7 +5284,7 @@ func TestExecError(t *testing.T) {
 			Start:    1000,
 			End:      2000,
 			Step:     100,
-			Deadline: netstorage.NewDeadline(time.Minute),
+			Deadline: netstorage.NewDeadline(time.Minute, ""),
 		}
 		for i := 0; i < 4; i++ {
 			rv, err := Exec(ec, q, false)
@@ -5210,6 +5331,8 @@ func TestExecError(t *testing.T) {
 	f(`label_set(1, "foo")`)
 	f(`label_del()`)
 	f(`label_keep()`)
+	f(`label_match()`)
+	f(`label_mismatch()`)
 	f(`round()`)
 	f(`round(1,2,3)`)
 	f(`scalar()`)
@@ -5257,6 +5380,7 @@ func TestExecError(t *testing.T) {
 	f(`median()`)
 	f(`median("foo", "bar")`)
 	f(`keep_last_value()`)
+	f(`keep_next_value()`)
 	f(`distinct_over_time()`)
 	f(`distinct()`)
 	f(`alias()`)
@@ -5267,6 +5391,15 @@ func TestExecError(t *testing.T) {
 	f(`aggr_over_time()`)
 	f(`aggr_over_time(foo)`)
 	f(`aggr_over_time("foo", bar, 1)`)
+	f(`sum(aggr_over_time())`)
+	f(`sum(aggr_over_time(foo))`)
+	f(`count(aggr_over_time("foo", bar, 1))`)
+	f(`hoeffding_bound_lower()`)
+	f(`hoeffding_bound_lower(1)`)
+	f(`hoeffding_bound_lower(0.99, foo, 1)`)
+	f(`hoeffding_bound_upper()`)
+	f(`hoeffding_bound_upper(1)`)
+	f(`hoeffding_bound_upper(0.99, foo, 1)`)
 
 	// Invalid argument type
 	f(`median_over_time({}, 2)`)
@@ -5301,6 +5434,8 @@ func TestExecError(t *testing.T) {
 	f(`label_transform(1, "foo", 3, 4)`)
 	f(`label_transform(1, "foo", "bar", 4)`)
 	f(`label_transform(1, "foo", "invalid(regexp", "baz`)
+	f(`label_match(1, 2, 3)`)
+	f(`label_mismatch(1, 2, 3)`)
 	f(`alias(1, 2)`)
 	f(`aggr_over_time(1, 2)`)
 	f(`aggr_over_time(("foo", "bar"), 3)`)

app/vmselect/promql/rollup.go

@@ -15,8 +15,6 @@ import (
 )
 
 var rollupFuncs = map[string]newRollupFunc{
-	"default_rollup": newRollupFuncOneArg(rollupDefault), // default rollup func
-
 	// Standard rollup funcs from PromQL.
 	// See funcs accepting range-vector on https://prometheus.io/docs/prometheus/latest/querying/functions/ .
 	"changes":            newRollupFuncOneArg(rollupChanges),
@@ -41,30 +39,34 @@ var rollupFuncs = map[string]newRollupFunc{
 	"absent_over_time":   newRollupFuncOneArg(rollupAbsent),
 
 	// Additional rollup funcs.
-	"sum2_over_time":      newRollupFuncOneArg(rollupSum2),
-	"geomean_over_time":   newRollupFuncOneArg(rollupGeomean),
-	"first_over_time":     newRollupFuncOneArg(rollupFirst),
-	"last_over_time":      newRollupFuncOneArg(rollupLast),
-	"distinct_over_time":  newRollupFuncOneArg(rollupDistinct),
-	"increases_over_time": newRollupFuncOneArg(rollupIncreases),
-	"decreases_over_time": newRollupFuncOneArg(rollupDecreases),
-	"integrate":           newRollupFuncOneArg(rollupIntegrate),
-	"ideriv":              newRollupFuncOneArg(rollupIderiv),
-	"lifetime":            newRollupFuncOneArg(rollupLifetime),
-	"lag":                 newRollupFuncOneArg(rollupLag),
-	"scrape_interval":     newRollupFuncOneArg(rollupScrapeInterval),
-	"tmin_over_time":      newRollupFuncOneArg(rollupTmin),
-	"tmax_over_time":      newRollupFuncOneArg(rollupTmax),
-	"share_le_over_time":  newRollupShareLE,
-	"share_gt_over_time":  newRollupShareGT,
-	"histogram_over_time": newRollupFuncOneArg(rollupHistogram),
-	"rollup":              newRollupFuncOneArg(rollupFake),
-	"rollup_rate":         newRollupFuncOneArg(rollupFake), // + rollupFuncsRemoveCounterResets
-	"rollup_deriv":        newRollupFuncOneArg(rollupFake),
-	"rollup_delta":        newRollupFuncOneArg(rollupFake),
-	"rollup_increase":     newRollupFuncOneArg(rollupFake), // + rollupFuncsRemoveCounterResets
-	"rollup_candlestick":  newRollupFuncOneArg(rollupFake),
-	"aggr_over_time":      newRollupFuncTwoArgs(rollupFake),
+	"default_rollup":        newRollupFuncOneArg(rollupDefault), // default rollup func
+	"range_over_time":       newRollupFuncOneArg(rollupRange),
+	"sum2_over_time":        newRollupFuncOneArg(rollupSum2),
+	"geomean_over_time":     newRollupFuncOneArg(rollupGeomean),
+	"first_over_time":       newRollupFuncOneArg(rollupFirst),
+	"last_over_time":        newRollupFuncOneArg(rollupLast),
+	"distinct_over_time":    newRollupFuncOneArg(rollupDistinct),
+	"increases_over_time":   newRollupFuncOneArg(rollupIncreases),
+	"decreases_over_time":   newRollupFuncOneArg(rollupDecreases),
+	"integrate":             newRollupFuncOneArg(rollupIntegrate),
+	"ideriv":                newRollupFuncOneArg(rollupIderiv),
+	"lifetime":              newRollupFuncOneArg(rollupLifetime),
+	"lag":                   newRollupFuncOneArg(rollupLag),
+	"scrape_interval":       newRollupFuncOneArg(rollupScrapeInterval),
+	"tmin_over_time":        newRollupFuncOneArg(rollupTmin),
+	"tmax_over_time":        newRollupFuncOneArg(rollupTmax),
+	"share_le_over_time":    newRollupShareLE,
+	"share_gt_over_time":    newRollupShareGT,
+	"histogram_over_time":   newRollupFuncOneArg(rollupHistogram),
+	"rollup":                newRollupFuncOneArg(rollupFake),
+	"rollup_rate":           newRollupFuncOneArg(rollupFake), // + rollupFuncsRemoveCounterResets
+	"rollup_deriv":          newRollupFuncOneArg(rollupFake),
+	"rollup_delta":          newRollupFuncOneArg(rollupFake),
+	"rollup_increase":       newRollupFuncOneArg(rollupFake), // + rollupFuncsRemoveCounterResets
+	"rollup_candlestick":    newRollupFuncOneArg(rollupFake),
+	"aggr_over_time":        newRollupFuncTwoArgs(rollupFake),
+	"hoeffding_bound_upper": newRollupHoeffdingBoundUpper,
+	"hoeffding_bound_lower": newRollupHoeffdingBoundLower,
 }
 
 // rollupAggrFuncs are functions that can be passed to `aggr_over_time()`
@@ -89,6 +91,7 @@ var rollupAggrFuncs = map[string]rollupFunc{
 	"absent_over_time": rollupAbsent,
 
 	// Additional rollup funcs.
+	"range_over_time":     rollupRange,
 	"sum2_over_time":      rollupSum2,
 	"geomean_over_time":   rollupGeomean,
 	"first_over_time":     rollupFirst,
@@ -105,17 +108,26 @@ var rollupAggrFuncs = map[string]rollupFunc{
 	"tmax_over_time":      rollupTmax,
 }
 
-var rollupFuncsMayAdjustWindow = map[string]bool{
-	"default_rollup":  true,
-	"first_over_time": true,
-	"last_over_time":  true,
-	"deriv":           true,
-	"deriv_fast":      true,
-	"irate":           true,
-	"rate":            true,
-	"lifetime":        true,
-	"lag":             true,
-	"scrape_interval": true,
+var rollupFuncsCannotAdjustWindow = map[string]bool{
+	"changes":             true,
+	"delta":               true,
+	"holt_winters":        true,
+	"idelta":              true,
+	"increase":            true,
+	"predict_linear":      true,
+	"resets":              true,
+	"sum_over_time":       true,
+	"count_over_time":     true,
+	"quantile_over_time":  true,
+	"stddev_over_time":    true,
+	"stdvar_over_time":    true,
+	"absent_over_time":    true,
+	"sum2_over_time":      true,
+	"geomean_over_time":   true,
+	"distinct_over_time":  true,
+	"increases_over_time": true,
+	"decreases_over_time": true,
+	"integrate":           true,
 }
 
 var rollupFuncsRemoveCounterResets = map[string]bool{
@@ -127,13 +139,15 @@ var rollupFuncsRemoveCounterResets = map[string]bool{
 }
 
 var rollupFuncsKeepMetricGroup = map[string]bool{
-	"default_rollup":     true,
-	"avg_over_time":      true,
-	"min_over_time":      true,
-	"max_over_time":      true,
-	"quantile_over_time": true,
-	"rollup":             true,
-	"geomean_over_time":  true,
+	"default_rollup":        true,
+	"avg_over_time":         true,
+	"min_over_time":         true,
+	"max_over_time":         true,
+	"quantile_over_time":    true,
+	"rollup":                true,
+	"geomean_over_time":     true,
+	"hoeffding_bound_lower": true,
+	"hoeffding_bound_upper": true,
 }
 
 func getRollupAggrFuncNames(expr metricsql.Expr) ([]string, error) {
@@ -191,7 +205,8 @@ func getRollupArgIdx(funcName string) int {
 		logger.Panicf("BUG: getRollupArgIdx is called for non-rollup func %q", funcName)
 	}
 	switch funcName {
-	case "quantile_over_time", "aggr_over_time":
+	case "quantile_over_time", "aggr_over_time",
+		"hoeffding_bound_lower", "hoeffding_bound_upper":
 		return 1
 	default:
 		return 0
@@ -214,7 +229,7 @@ func getRollupConfigs(name string, rf rollupFunc, expr metricsql.Expr, start, en
 			End:             end,
 			Step:            step,
 			Window:          window,
-			MayAdjustWindow: rollupFuncsMayAdjustWindow[name],
+			MayAdjustWindow: !rollupFuncsCannotAdjustWindow[name],
 			LookbackDelta:   lookbackDelta,
 			Timestamps:      sharedTimestamps,
 		}
@@ -244,10 +259,10 @@ func getRollupConfigs(name string, rf rollupFunc, expr metricsql.Expr, start, en
 		}
 		rcs = appendRollupConfigs(rcs)
 	case "rollup_candlestick":
-		rcs = append(rcs, newRollupConfig(rollupFirst, "open"))
-		rcs = append(rcs, newRollupConfig(rollupLast, "close"))
-		rcs = append(rcs, newRollupConfig(rollupMin, "low"))
-		rcs = append(rcs, newRollupConfig(rollupMax, "high"))
+		rcs = append(rcs, newRollupConfig(rollupOpen, "open"))
+		rcs = append(rcs, newRollupConfig(rollupClose, "close"))
+		rcs = append(rcs, newRollupConfig(rollupLow, "low"))
+		rcs = append(rcs, newRollupConfig(rollupHigh, "high"))
 	case "aggr_over_time":
 		aggrFuncNames, err := getRollupAggrFuncNames(expr)
 		if err != nil {
@@ -819,6 +834,66 @@ func newRollupShareFilter(args []interface{}, countFilter func(values []float64,
 	return rf, nil
 }
 
+func newRollupHoeffdingBoundLower(args []interface{}) (rollupFunc, error) {
+	if err := expectRollupArgsNum(args, 2); err != nil {
+		return nil, err
+	}
+	phis, err := getScalar(args[0], 0)
+	if err != nil {
+		return nil, err
+	}
+	rf := func(rfa *rollupFuncArg) float64 {
+		bound, avg := rollupHoeffdingBoundInternal(rfa, phis)
+		return avg - bound
+	}
+	return rf, nil
+}
+
+func newRollupHoeffdingBoundUpper(args []interface{}) (rollupFunc, error) {
+	if err := expectRollupArgsNum(args, 2); err != nil {
+		return nil, err
+	}
+	phis, err := getScalar(args[0], 0)
+	if err != nil {
+		return nil, err
+	}
+	rf := func(rfa *rollupFuncArg) float64 {
+		bound, avg := rollupHoeffdingBoundInternal(rfa, phis)
+		return avg + bound
+	}
+	return rf, nil
+}
+
+func rollupHoeffdingBoundInternal(rfa *rollupFuncArg, phis []float64) (float64, float64) {
+	// There is no need in handling NaNs here, since they must be cleaned up
+	// before calling rollup funcs.
+	values := rfa.values
+	if len(values) == 0 {
+		return nan, nan
+	}
+	if len(values) == 1 {
+		return 0, values[0]
+	}
+	vMax := rollupMax(rfa)
+	vMin := rollupMin(rfa)
+	vAvg := rollupAvg(rfa)
+	vRange := vMax - vMin
+	if vRange <= 0 {
+		return 0, vAvg
+	}
+	phi := phis[rfa.idx]
+	if phi >= 1 {
+		return inf, vAvg
+	}
+	if phi <= 0 {
+		return 0, vAvg
+	}
+	// See https://en.wikipedia.org/wiki/Hoeffding%27s_inequality
+	// and https://www.youtube.com/watch?v=6UwcqiNsZ8U&feature=youtu.be&t=1237
+	bound := vRange * math.Sqrt(math.Log(1/(1-phi))/(2*float64(len(values))))
+	return bound, vAvg
+}
+
 func newRollupQuantile(args []interface{}) (rollupFunc, error) {
 	if err := expectRollupArgsNum(args, 2); err != nil {
 		return nil, err
@@ -873,7 +948,10 @@ func rollupAvg(rfa *rollupFuncArg) float64 {
 	// before calling rollup funcs.
 	values := rfa.values
 	if len(values) == 0 {
-		return rfa.prevValue
+		// Do not take into account rfa.prevValue, since it may lead
+		// to inconsistent results comparing to Prometheus on broken time series
+		// with irregular data points.
+		return nan
 	}
 	var sum float64
 	for _, v := range values {
@@ -975,6 +1053,12 @@ func rollupSum(rfa *rollupFuncArg) float64 {
 	return sum
 }
 
+func rollupRange(rfa *rollupFuncArg) float64 {
+	max := rollupMax(rfa)
+	min := rollupMin(rfa)
+	return max - min
+}
+
 func rollupSum2(rfa *rollupFuncArg) float64 {
 	// There is no need in handling NaNs here, since they must be cleaned up
 	// before calling rollup funcs.
@@ -1073,10 +1157,18 @@ func rollupDeltaInternal(rfa *rollupFuncArg, canUseRealPrevValue bool) float64 {
 		if len(values) == 0 {
 			return nan
 		}
-		// Assume that the previous non-existing value was 0.
-		prevValue = 0
-		if canUseRealPrevValue && !math.IsNaN(rfa.prevValue) {
-			prevValue = rfa.prevValue
+		// Assume that the previous non-existing value was 0
+		// only if the first value is quite small.
+		// This should prevent from improper increase() results for os-level counters
+		// such as cpu time or bytes sent over the network interface.
+		// These counters may start long ago before the first value appears in the db.
+		if values[0] < 1e6 {
+			prevValue = 0
+			if canUseRealPrevValue && !math.IsNaN(rfa.realPrevValue) {
+				prevValue = rfa.realPrevValue
+			}
+		} else {
+			prevValue = values[0]
 		}
 	}
 	if len(values) == 0 {
@@ -1327,6 +1419,74 @@ func rollupResets(rfa *rollupFuncArg) float64 {
 	return float64(n)
 }
 
+// getCandlestickValues returns a subset of rfa.values suitable for rollup_candlestick
+//
+// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/309 for details.
+func getCandlestickValues(rfa *rollupFuncArg) []float64 {
+	currTimestamp := rfa.currTimestamp
+	timestamps := rfa.timestamps
+	for len(timestamps) > 0 && timestamps[len(timestamps)-1] >= currTimestamp {
+		timestamps = timestamps[:len(timestamps)-1]
+	}
+	if len(timestamps) == 0 {
+		return nil
+	}
+	return rfa.values[:len(timestamps)]
+}
+
+func rollupOpen(rfa *rollupFuncArg) float64 {
+	if !math.IsNaN(rfa.prevValue) {
+		return rfa.prevValue
+	}
+	values := getCandlestickValues(rfa)
+	if len(values) == 0 {
+		return nan
+	}
+	return values[0]
+}
+
+func rollupClose(rfa *rollupFuncArg) float64 {
+	values := getCandlestickValues(rfa)
+	if len(values) == 0 {
+		return rfa.prevValue
+	}
+	return values[len(values)-1]
+}
+
+func rollupHigh(rfa *rollupFuncArg) float64 {
+	values := getCandlestickValues(rfa)
+	max := rfa.prevValue
+	if math.IsNaN(max) {
+		if len(values) == 0 {
+			return nan
+		}
+		max = values[0]
+	}
+	for _, v := range values {
+		if v > max {
+			max = v
+		}
+	}
+	return max
+}
+
+func rollupLow(rfa *rollupFuncArg) float64 {
+	values := getCandlestickValues(rfa)
+	min := rfa.prevValue
+	if math.IsNaN(min) {
+		if len(values) == 0 {
+			return nan
+		}
+		min = values[0]
+	}
+	for _, v := range values {
+		if v < min {
+			min = v
+		}
+	}
+	return min
+}
+
 func rollupFirst(rfa *rollupFuncArg) float64 {
 	// There is no need in handling NaNs here, since they must be cleaned up
 	// before calling rollup funcs.

app/vmselect/promql/rollup_result_cache.go

@@ -74,8 +74,8 @@ func InitRollupResultCache(cachePath string) {
 		return stats
 	}
 	if len(rollupResultCachePath) > 0 {
-		logger.Infof("loaded rollupResult cache from %q in %s; entriesCount: %d, sizeBytes: %d",
-			rollupResultCachePath, time.Since(startTime), fcs().EntriesCount, fcs().BytesSize)
+		logger.Infof("loaded rollupResult cache from %q in %.3f seconds; entriesCount: %d, sizeBytes: %d",
+			rollupResultCachePath, time.Since(startTime).Seconds(), fcs().EntriesCount, fcs().BytesSize)
 	}
 
 	metrics.NewGauge(`vm_cache_entries{type="promql/rollupResult"}`, func() float64 {
@@ -113,8 +113,8 @@ func StopRollupResultCache() {
 	rollupResultCacheV.c.UpdateStats(&fcs)
 	rollupResultCacheV.c.Stop()
 	rollupResultCacheV.c = nil
-	logger.Infof("saved rollupResult cache to %q in %s; entriesCount: %d, sizeBytes: %d",
-		rollupResultCachePath, time.Since(startTime), fcs.EntriesCount, fcs.BytesSize)
+	logger.Infof("saved rollupResult cache to %q in %.3f seconds; entriesCount: %d, sizeBytes: %d",
+		rollupResultCachePath, time.Since(startTime).Seconds(), fcs.EntriesCount, fcs.BytesSize)
 }
 
 type rollupResultCache struct {

app/vmselect/promql/rollup_test.go

@@ -310,6 +310,48 @@ func TestRollupHoltWinters(t *testing.T) {
 	f(0.9, 0.9, 33.99637566941818)
 }
 
+func TestRollupHoeffdingBoundLower(t *testing.T) {
+	f := func(phi, vExpected float64) {
+		t.Helper()
+		phis := []*timeseries{{
+			Values:     []float64{phi},
+			Timestamps: []int64{123},
+		}}
+		var me metricsql.MetricExpr
+		args := []interface{}{phis, &metricsql.RollupExpr{Expr: &me}}
+		testRollupFunc(t, "hoeffding_bound_lower", args, &me, vExpected)
+	}
+
+	f(0.5, 28.21949401521037)
+	f(-1, 47.083333333333336)
+	f(0, 47.083333333333336)
+	f(1, -inf)
+	f(2, -inf)
+	f(0.1, 39.72878000047643)
+	f(0.9, 12.701803086472331)
+}
+
+func TestRollupHoeffdingBoundUpper(t *testing.T) {
+	f := func(phi, vExpected float64) {
+		t.Helper()
+		phis := []*timeseries{{
+			Values:     []float64{phi},
+			Timestamps: []int64{123},
+		}}
+		var me metricsql.MetricExpr
+		args := []interface{}{phis, &metricsql.RollupExpr{Expr: &me}}
+		testRollupFunc(t, "hoeffding_bound_upper", args, &me, vExpected)
+	}
+
+	f(0.5, 65.9471726514563)
+	f(-1, 47.083333333333336)
+	f(0, 47.083333333333336)
+	f(1, inf)
+	f(2, inf)
+	f(0.1, 54.43788666619024)
+	f(0.9, 81.46486358019433)
+}
+
 func TestRollupNewRollupFuncSuccess(t *testing.T) {
 	f := func(funcName string, vExpected float64) {
 		t.Helper()
@@ -328,6 +370,7 @@ func TestRollupNewRollupFuncSuccess(t *testing.T) {
 	f("irate", 0)
 	f("rate", 2200)
 	f("resets", 5)
+	f("range_over_time", 111)
 	f("avg_over_time", 47.083333333333336)
 	f("min_over_time", 12)
 	f("max_over_time", 123)

app/vmselect/promql/transform.go

@@ -65,9 +65,12 @@ var transformFuncs = map[string]transformFunc{
 	"label_move":         transformLabelMove,
 	"label_transform":    transformLabelTransform,
 	"label_value":        transformLabelValue,
+	"label_match":        transformLabelMatch,
+	"label_mismatch":     transformLabelMismatch,
 	"union":              transformUnion,
 	"":                   transformUnion, // empty func is a synonim to union
 	"keep_last_value":    transformKeepLastValue,
+	"keep_next_value":    transformKeepNextValue,
 	"start":              newTransformFuncZeroArgs(transformStart),
 	"end":                newTransformFuncZeroArgs(transformEnd),
 	"step":               newTransformFuncZeroArgs(transformStep),
@@ -361,6 +364,7 @@ func vmrangeBucketsToLE(tss []*timeseries) []*timeseries {
 			ts := xs.ts
 			if isZeroTS(ts) {
 				// Skip time series with zeros. They are substituted by xssNew below.
+				xsPrev = xs
 				continue
 			}
 			if xs.start != xsPrev.end {
@@ -721,13 +725,37 @@ func transformKeepLastValue(tfa *transformFuncArg) ([]*timeseries, error) {
 		if len(values) == 0 {
 			continue
 		}
-		prevValue := values[0]
+		lastValue := values[0]
 		for i, v := range values {
-			if math.IsNaN(v) {
-				v = prevValue
+			if !math.IsNaN(v) {
+				lastValue = v
+				continue
 			}
-			values[i] = v
-			prevValue = v
+			values[i] = lastValue
+		}
+	}
+	return rvs, nil
+}
+
+func transformKeepNextValue(tfa *transformFuncArg) ([]*timeseries, error) {
+	args := tfa.args
+	if err := expectTransformArgsNum(args, 1); err != nil {
+		return nil, err
+	}
+	rvs := args[0]
+	for _, ts := range rvs {
+		values := ts.Values
+		if len(values) == 0 {
+			continue
+		}
+		nextValue := values[len(values)-1]
+		for i := len(values) - 1; i >= 0; i-- {
+			v := values[i]
+			if !math.IsNaN(v) {
+				nextValue = v
+				continue
+			}
+			values[i] = nextValue
 		}
 	}
 	return rvs, nil
@@ -1203,6 +1231,62 @@ func transformLabelValue(tfa *transformFuncArg) ([]*timeseries, error) {
 	return rvs, nil
 }
 
+func transformLabelMatch(tfa *transformFuncArg) ([]*timeseries, error) {
+	args := tfa.args
+	if err := expectTransformArgsNum(args, 3); err != nil {
+		return nil, err
+	}
+	labelName, err := getString(args[1], 1)
+	if err != nil {
+		return nil, fmt.Errorf("cannot get label name: %s", err)
+	}
+	labelRe, err := getString(args[2], 2)
+	if err != nil {
+		return nil, fmt.Errorf("cannot get regexp: %s", err)
+	}
+	r, err := metricsql.CompileRegexpAnchored(labelRe)
+	if err != nil {
+		return nil, fmt.Errorf(`cannot compile regexp %q: %s`, labelRe, err)
+	}
+	tss := args[0]
+	rvs := tss[:0]
+	for _, ts := range tss {
+		labelValue := ts.MetricName.GetTagValue(labelName)
+		if r.Match(labelValue) {
+			rvs = append(rvs, ts)
+		}
+	}
+	return rvs, nil
+}
+
+func transformLabelMismatch(tfa *transformFuncArg) ([]*timeseries, error) {
+	args := tfa.args
+	if err := expectTransformArgsNum(args, 3); err != nil {
+		return nil, err
+	}
+	labelName, err := getString(args[1], 1)
+	if err != nil {
+		return nil, fmt.Errorf("cannot get label name: %s", err)
+	}
+	labelRe, err := getString(args[2], 2)
+	if err != nil {
+		return nil, fmt.Errorf("cannot get regexp: %s", err)
+	}
+	r, err := metricsql.CompileRegexpAnchored(labelRe)
+	if err != nil {
+		return nil, fmt.Errorf(`cannot compile regexp %q: %s`, labelRe, err)
+	}
+	tss := args[0]
+	rvs := tss[:0]
+	for _, ts := range tss {
+		labelValue := ts.MetricName.GetTagValue(labelName)
+		if !r.Match(labelValue) {
+			rvs = append(rvs, ts)
+		}
+	}
+	return rvs, nil
+}
+
 func transformLn(v float64) float64 {
 	return math.Log(v)
 }

app/vmstorage/main.go

@@ -62,8 +62,8 @@ func InitWithoutMetrics() {
 	blocksCount := tm.SmallBlocksCount + tm.BigBlocksCount
 	rowsCount := tm.SmallRowsCount + tm.BigRowsCount
 	sizeBytes := tm.SmallSizeBytes + tm.BigSizeBytes
-	logger.Infof("successfully opened storage %q in %s; partsCount: %d; blocksCount: %d; rowsCount: %d; sizeBytes: %d",
-		*DataPath, time.Since(startTime), partsCount, blocksCount, rowsCount, sizeBytes)
+	logger.Infof("successfully opened storage %q in %.3f seconds; partsCount: %d; blocksCount: %d; rowsCount: %d; sizeBytes: %d",
+		*DataPath, time.Since(startTime).Seconds(), partsCount, blocksCount, rowsCount, sizeBytes)
 }
 
 // Storage is a storage.
@@ -133,7 +133,7 @@ func Stop() {
 	startTime := time.Now()
 	WG.WaitAndBlock()
 	Storage.MustClose()
-	logger.Infof("successfully closed the storage in %s", time.Since(startTime))
+	logger.Infof("successfully closed the storage in %.3f seconds", time.Since(startTime).Seconds())
 
 	logger.Infof("the storage has been stopped")
 }
@@ -461,6 +461,9 @@ func registerStorageMetrics() {
 	metrics.NewGauge(`vm_cache_entries{type="storage/regexps"}`, func() float64 {
 		return float64(storage.RegexpCacheSize())
 	})
+	metrics.NewGauge(`vm_cache_size_entries{type="storage/prefetchedMetricIDs"}`, func() float64 {
+		return float64(m().PrefetchedMetricIDsSize)
+	})
 
 	metrics.NewGauge(`vm_cache_size_bytes{type="storage/tsid"}`, func() float64 {
 		return float64(m().TSIDCacheSizeBytes)
@@ -483,6 +486,9 @@ func registerStorageMetrics() {
 	metrics.NewGauge(`vm_cache_size_bytes{type="indexdb/uselessTagFilters"}`, func() float64 {
 		return float64(idbm().UselessTagFiltersCacheSizeBytes)
 	})
+	metrics.NewGauge(`vm_cache_size_bytes{type="storage/prefetchedMetricIDs"}`, func() float64 {
+		return float64(m().PrefetchedMetricIDsSizeBytes)
+	})
 
 	metrics.NewGauge(`vm_cache_requests_total{type="storage/tsid"}`, func() float64 {
 		return float64(m().TSIDCacheRequests)

dashboards/victoriametrics.json

@@ -14,7 +14,7 @@
       "type": "grafana",
       "id": "grafana",
       "name": "Grafana",
-      "version": "6.5.0"
+      "version": "6.5.2"
     },
     {
       "type": "panel",
@@ -60,12 +60,12 @@
       }
     ]
   },
-  "description": "Overview for single node VictoriaMetrics v1.30.3 or higher",
+  "description": "Overview for single node VictoriaMetrics v1.32.8 or higher",
   "editable": true,
   "gnetId": 10229,
   "graphTooltip": 0,
   "id": null,
-  "iteration": 1575825261972,
+  "iteration": 1580634216692,
   "links": [
     {
       "icon": "doc",
@@ -1565,7 +1565,7 @@
       "dashLength": 10,
       "dashes": false,
       "datasource": "${DS_PROMETHEUS}",
-      "description": "How many datapoints are in RAM queue waiting to be written into storage. The less is better.",
+      "description": "How many datapoints are in RAM queue waiting to be written into storage. The number of pending data points should be in the range from 0 to `2*<ingestion_rate>`, since VictoriaMetrics pushes pending data to persistent storage every second.",
       "fill": 1,
       "fillGradient": 0,
       "gridPos": {
@@ -2930,7 +2930,7 @@
     ]
   },
   "time": {
-    "from": "now-1h",
+    "from": "now-30m",
     "to": "now"
   },
   "timepicker": {
@@ -2961,5 +2961,5 @@
   "timezone": "",
   "title": "VictoriaMetrics",
   "uid": "wNf0q_kZk",
-  "version": 1
+  "version": 2
 }

deployment/docker/Makefile

@@ -1,7 +1,7 @@
 # All these commands must run from repository root.
 
 DOCKER_NAMESPACE := docker.io/victoriametrics
-BUILDER_IMAGE := local/builder:go1.13.5
+BUILDER_IMAGE := local/builder:go1.13.7
 CERTS_IMAGE := local/certs:1.0.3
 
 package-certs:

deployment/docker/builder/Dockerfile

@@ -1,2 +1,2 @@
-FROM golang:1.13.5
+FROM golang:1.13.7
 STOPSIGNAL SIGINT

deployment/docker/docker-compose.yml

@@ -2,7 +2,7 @@ version: '3.5'
 services:
   prometheus:
     container_name: prometheus
-    image: prom/prometheus:v2.14.0
+    image: prom/prometheus:v2.15.2
     depends_on:
       - "victoriametrics"
     ports:
@@ -35,7 +35,7 @@ services:
     restart: always
   grafana:
     container_name: grafana
-    image: grafana/grafana:6.5.0
+    image: grafana/grafana:6.5.2
     entrypoint: >
       /bin/sh -c "
       cd /var/lib/grafana &&

docs/Articles.md

@@ -20,3 +20,4 @@
 * [Evaluation performance and correctness: VictoriaMetrics response](https://medium.com/@valyala/evaluating-performance-and-correctness-victoriametrics-response-e27315627e87)
 * [Improving histogram usability for Prometheus and Grafana](https://medium.com/@valyala/improving-histogram-usability-for-prometheus-and-grafana-bc7e5df0e350)
 * [Prometheus storage: tech terms for humans](https://medium.com/@valyala/prometheus-storage-technical-terms-for-humans-4ab4de6c3d48)
+* [Billy: how VictoriaMetrics deals with more than 500 billion rows](https://medium.com/@valyala/billy-how-victoriametrics-deals-with-more-than-500-billion-rows-e82ff8f725da)

docs/CaseStudies.md

@@ -10,6 +10,20 @@ from [Remote Write Storage Wars](https://promcon.io/2019-munich/talks/remote-wri
 VictoriaMetrics is compared to Thanos, Corex and M3DB in the talk.
 
 
+### COLOPL
+
+[COLOPL](http://www.colopl.co.jp/en/) is Japaneese Game Development company. It started using VictoriaMetrics
+after evaulating the following remote storage solutions for Prometheus:
+
+* Cortex
+* Thanos
+* M3DB
+* VictoriaMetrics
+
+See [slides](https://speakerdeck.com/inletorder/monitoring-platform-with-victoria-metrics) and [video](https://www.youtube.com/watch?v=hUpHIluxw80)
+from `Large-scale, super-load system monitoring platform built with VictoriaMetrics` talk at [Prometheus Meetup Tokyo #3](https://prometheus.connpass.com/event/157721/).
+
+
 ### Wix.com
 
 [Wix.com](https://en.wikipedia.org/wiki/Wix.com) is the leading web development platform.

docs/ExtendedPromQL.md

@@ -52,6 +52,7 @@ This functionality can be tried at [an editable Grafana dashboard](http://play-g
   - `label_move(q, src_label1, dst_label1, ... src_labelN, dst_labelN)` for moving label values from `src_*` to `dst_*`.
   - `label_transform(q, label, regexp, replacement)` for replacing all the `regexp` occurences with `replacement` in the `label` values from `q`.
   - `label_value(q, label)` - returns numeric values for the given `label` from `q`.
+- `label_match(q, label, regexp)` and `label_mismatch(q, label, regexp)` for filtering time series with labels matching (or not matching) the given regexps.
 - `step()` function for returning the step in seconds used in the query.
 - `start()` and `end()` functions for returning the start and end timestamps of the `[start ... end]` range used in the query.
 - `integrate(m[d])` for returning integral over the given duration `d` for the given metric `m`.
@@ -65,10 +66,12 @@ This functionality can be tried at [an editable Grafana dashboard](http://play-g
 - `lifetime(q[d])` - returns lifetime of `q` over `d` in seconds. It is expected that `d` exceeds the lifetime of `q`.
 - `scrape_interval(q[d])` - returns the average interval in seconds between data points of `q` over `d` aka `scrape interval`.
 - Trigonometric functions - `sin(q)`, `cos(q)`, `asin(q)`, `acos(q)` and `pi()`.
+- `range_over_time(m[d])` - returns value range for `m` over `d` time window, i.e. `max_over_time(m[d])-min_over_time(m[d])`.
 - `median_over_time(m[d])` - calculates median values for `m` over `d` time window. Shorthand to `quantile_over_time(0.5, m[d])`.
 - `median(q)` - median aggregate. Shorthand to `quantile(0.5, q)`.
 - `limitk(k, q)` - limits the number of time series returned from `q` to `k`.
-- `keep_last_value(q)` - fills missing data (gaps) in `q` with the previous value.
+- `keep_last_value(q)` - fills missing data (gaps) in `q` with the previous non-empty value.
+- `keep_next_value(q)` - fills missing data (gaps) in `q` with the next non-empty value.
 - `distinct_over_time(m[d])` - returns distinct number of values for `m` data points over `d` duration.
 - `distinct(q)` - returns a time series with the number of unique values for each timestamp in `q`.
 - `sum2_over_time(m[d])` - returns sum of squares for all the `m` values over `d` duration.
@@ -103,3 +106,5 @@ This functionality can be tried at [an editable Grafana dashboard](http://play-g
 - `aggr_over_time(("aggr_func1", "aggr_func2", ...), m[d])` - simultaneously calculates all the listed `aggr_func*` for `m` over `d` time range.
   `aggr_func*` can contain any functions that accept range vector. For instance, `aggr_over_time(("min_over_time", "max_over_time", "rate"), m[d])`
   would calculate `min_over_time`, `max_over_time` and `rate` for `m[d]`.
+- `hoeffding_bound_upper(phi, m[d])` and `hoeffding_bound_lower(phi, m[d])` - return upper and lower [Hoeffding bounds](https://en.wikipedia.org/wiki/Hoeffding%27s_inequality)
+  for the given `phi` in the range `[0..1]`.

docs/FAQ.md

@@ -54,11 +54,11 @@ Yes. Prometheus continues writing data to local storage after enabling remote st
 and new data is available for querying via Prometheus as usual.
 
 
-### How does VictoriaMetrics compare to other clustered TSDBs on top of Prometheus such as [M3 from Uber](https://eng.uber.com/m3/), [Thanos](https://github.com/improbable-eng/thanos), [Cortex](https://github.com/cortexproject/cortex), etc.?
+### How does VictoriaMetrics compare to other clustered TSDBs on top of Prometheus such as [M3 from Uber](https://eng.uber.com/m3/), [Thanos](https://github.com/thanos-io/thanos), [Cortex](https://github.com/cortexproject/cortex), etc.?
 
 VictoriaMetrics is simpler, faster, more cost-effective and it provides [MetricsQL with useful extensions for PromQL](ExtendedPromQL). The simplicity is twofold:
-- It is simpler to configure and operate. There is no need in configuring third-party [sidecars](https://github.com/improbable-eng/thanos/blob/master/docs/components/sidecar.md)
-  or fighting with [gossip protocol](https://github.com/improbable-eng/thanos/blob/master/docs/proposals/completed/201809_gossip-removal.md).
+- It is simpler to configure and operate. There is no need in configuring third-party [sidecars](https://github.com/thanos-io/thanos/blob/master/docs/components/sidecar.md)
+  or fighting with [gossip protocol](https://github.com/thanos-io/thanos/blob/master/docs/proposals/completed/201809_gossip-removal.md).
 - VictoriaMetrics has simpler architecture, which means less bugs and more useful features in the long run comparing to competing TSDBs.
 
 See [comparing Thanos to VictoriaMetrics cluster](https://medium.com/@valyala/comparing-thanos-to-victoriametrics-cluster-b193bea1683)
@@ -67,6 +67,45 @@ and [Remote Write Storage Wars](https://promcon.io/2019-munich/talks/remote-writ
 VictoriaMetrics also [uses less RAM than Thanos components](https://github.com/thanos-io/thanos/issues/448).
 
 
+### What is the difference between VictoriaMetrics and [Cortex](https://github.com/cortexproject/cortex)?
+
+VictoriaMetrics is similar to Cortex in the following aspects:
+- Both systems accept data from Prometheus via standard [remote_write API](https://prometheus.io/docs/practices/remote_write/),
+  i.e. there is no need in running sidecars unlike in [Thanos](https://github.com/thanos-io/thanos) case.
+- Both systems support multi-tenancy out of the box. See [the corresponding docs for VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/cluster/README.md#url-format).
+
+The main differences between Corex and VictoriaMetrics:
+- Cortex re-uses Prometheus source code, while VictoriaMetrics is written from scratch.
+- Cortex provides [Ruler](https://github.com/cortexproject/cortex/blob/master/docs/architecture.md#ruler) and [Alertmanager](https://github.com/cortexproject/cortex/blob/master/docs/architecture.md#alertmanager) components,
+  which are currently missing in VictoriaMetrics. However, these components can be substituted by [Promxy](https://github.com/jacksontj/promxy#how-do-i-use-alertingrecording-rules-in-promxy).
+- Cortex heavily relies on third-party services such as Consul, Memcache, DynamoDB, BigTable, Cassandra, etc.
+  This may increase operational complexity and reduce system reliability comparing to VictoriaMetrics' case,
+  which doesn't use any external services. Compare [Cortex Architecture](https://github.com/cortexproject/cortex/blob/master/docs/architecture.md)
+  to [VictoriaMetrics architecture](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/cluster/README.md#architecture-overview).
+- VictoriaMetrics provides [production-ready single-node solution](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/README.md),
+  which is much easier to setup and operate than Cortex cluster.
+- Cortex may lose up to 12 hours of recent data on Ingestor failure - see [the corresponding docs](https://github.com/cortexproject/cortex/blob/master/docs/architecture.md#ingesters-failure-and-data-loss).
+  VictoriaMetrics may lose only a few seconds of recent data, which isn't synced to persistent storage yet.
+  See [this article for details](https://medium.com/@valyala/wal-usage-looks-broken-in-modern-time-series-databases-b62a627ab704).
+- Cortex is usually slower and requires more CPU and RAM than VictoriaMetrics. See [this talk from Adidas at PromCon 2019](https://promcon.io/2019-munich/talks/remote-write-storage-wars/).
+
+
+### What is the difference between VictoriaMetrics and [Thanos](https://github.com/thanos-io/thanos)?
+
+- Thanos re-uses Prometheus source code, while VictoriaMetrics is written from scratch.
+- Thanos provides [Ruler component](https://github.com/thanos-io/thanos/blob/master/docs/components/rule.md),
+  while VictoriaMetrics relies on [Promxy for alerting and recording rules](https://github.com/jacksontj/promxy#how-do-i-use-alertingrecording-rules-in-promxy).
+- VictoriaMetrics accepts data via [standard remote_write API for Prometheus](https://prometheus.io/docs/practices/remote_write/),
+  while Thanos uses non-standard [Sidecar](https://github.com/thanos-io/thanos/blob/master/docs/components/sidecar.md), which must run alongside each Prometheus instance.
+- Thanos Sidecar requires disabling data compaction in Prometheus, which may hurt Prometheus performance and increase RAM usage.
+- Thanos stores data on object storage (Amazon S3 or Google GCS), while VictoriaMetrics stores data on block storage (GCP persistent disks, Amazon EBS or bare metal HDD).
+- Thanos may lose up to 2 hours of recent data, which wasn't uploaded yet to object storage. VictoriaMetrics may lose only a few seconds of recent data,
+  which isn't synced to persistent storage yet. See [this article for details](https://medium.com/@valyala/wal-usage-looks-broken-in-modern-time-series-databases-b62a627ab704).
+- Thanos may be harder to setup and operate comparing to VictoriaMetrics, since it has more moving parts, which can be connected with less reliable networks.
+  See [this article for details](https://medium.com/faun/comparing-thanos-to-victoriametrics-cluster-b193bea1683).
+- Thanos is usually slower and requires more CPU and RAM than VictoriaMetrics. See [this talk from Adidas at PromCon 2019](https://promcon.io/2019-munich/talks/remote-write-storage-wars/).
+
+
 ### How does VictoriaMetrics compare to [InfluxDB](https://www.influxdata.com/time-series-platform/influxdb/)?
 
 VictoriaMetrics requires [10x less RAM](https://medium.com/@valyala/insert-benchmarks-with-inch-influxdb-vs-victoriametrics-e31a41ae2893) and it [works faster](https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae).
@@ -79,12 +118,13 @@ TimescaleDB insists on using SQL as a query language. While SQL is more powerful
 Additionally, VictoriaMetrics requires [up to 70x less storage space comparing to TimescaleDB](https://medium.com/@valyala/when-size-matters-benchmarking-victoriametrics-vs-timescale-and-influxdb-6035811952d4) for storing the same amount of time series data.
 
 
-### Does VictoriaMetrics use Prometheus technologies like other clustered TSDBs built on top of Prometheus such as [M3 from Uber](https://eng.uber.com/m3/), [Thanos](https://github.com/improbable-eng/thanos), [Cortex](https://github.com/cortexproject/cortex)?
+### Does VictoriaMetrics use Prometheus technologies like other clustered TSDBs built on top of Prometheus such as [M3 from Uber](https://eng.uber.com/m3/), [Thanos](https://github.com/thanos-io/thanos), [Cortex](https://github.com/cortexproject/cortex)?
 
 No. VictoriaMetrics core is written in Go from scratch by [fasthttp](https://github.com/valyala/fasthttp) [author](https://github.com/valyala).
 The architecture is [optimized for storing and querying large amounts of time series data with high cardinality](https://medium.com/devopslinks/victoriametrics-creating-the-best-remote-storage-for-prometheus-5d92d66787ac). VictoriaMetrics storage uses [certain ideas from ClickHouse](https://medium.com/@valyala/how-victoriametrics-makes-instant-snapshots-for-multi-terabyte-time-series-data-e1f3fb0e0282). Special thanks to [Alexey Milovidov](https://github.com/alexey-milovidov).
 
 
+
 ### Are there performance comparisons with other solutions?
 
 Yes:
@@ -118,7 +158,7 @@ This is slow and expensive.
 Prometheus remote read API isn't intended for querying foreign data aka `global query view`. See [this issue](https://github.com/prometheus/prometheus/issues/4456) for details.
 
 So just query VictoriaMetrics directly via [Prometheus Querying API](https://prometheus.io/docs/prometheus/latest/querying/api/)
-or via [Prometheus datasoruce in Grafana](http://docs.grafana.org/features/datasources/prometheus/).
+or via [Prometheus datasource in Grafana](http://docs.grafana.org/features/datasources/prometheus/).
 
 
 ### Does VictoriaMetrics deduplicate data from Prometheus instances scraping the same targets (aka `HA pairs`)?
@@ -136,8 +176,8 @@ The deduplication for Prometheus HA pair may be easily implemented on top of Vic
 ### Where is the source code of VictoriaMetrics?
 
 Source code for the following versions is available in the following places:
-* [Single-node version](https://github.com/VictoriaMetrics/VictoriaMetrics).
-* [Cluster version](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster).
+* [Single-node version](https://github.com/VictoriaMetrics/VictoriaMetrics)
+* [Cluster version](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster)
 
 
 ### Does VictoriaMetrics fit for data from IoT sensors and industrial sensors?
@@ -150,7 +190,11 @@ and scales horizontally to multiple nodes.
 
 ### Where can I ask questions about VictoriaMetrics?
 
-See [VictoriaMetrics-users group](https://groups.google.com/forum/#!forum/victorametrics-users).
+Questions about VictoriaMetrics can be asked via the following channels:
+
+- [Slack channel](http://slack.victoriametrics.com/)
+- [Telegram channel](https://t.me/VictoriaMetrics_en)
+- [Google group](https://groups.google.com/forum/#!forum/victorametrics-users)
 
 
 ### Where can I file bugs and feature requests regarding VictoriaMetrics?

docs/Single-server-VictoriaMetrics.md

@@ -8,8 +8,10 @@ in [source code](https://github.com/VictoriaMetrics/VictoriaMetrics). Just downl
 Cluster version is available [here](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster).
 
 
-## Case studies
+## Case studies and talks
 
+* [Adidas](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/CaseStudies#adidas)
+* [COLOPL](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/CaseStudies#colopl)
 * [Wix.com](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/CaseStudies#wixcom)
 * [Wedos.com](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/CaseStudies#wedoscom)
 * [Dreamteam](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/CaseStudies#dreamteam)
@@ -76,7 +78,6 @@ Cluster version is available [here](https://github.com/VictoriaMetrics/VictoriaM
     - [Building docker images](#building-docker-images)
   - [Start with docker-compose](#start-with-docker-compose)
   - [Setting up service](#setting-up-service)
-  - [Third-party contributions](#third-party-contributions)
   - [How to work with snapshots?](#how-to-work-with-snapshots)
   - [How to delete time series?](#how-to-delete-time-series)
   - [How to export time series?](#how-to-export-time-series)
@@ -84,6 +85,7 @@ Cluster version is available [here](https://github.com/VictoriaMetrics/VictoriaM
   - [Federation](#federation)
   - [Capacity planning](#capacity-planning)
   - [High availability](#high-availability)
+  - [Retention](#retention)
   - [Multiple retentions](#multiple-retentions)
   - [Downsampling](#downsampling)
   - [Multi-tenancy](#multi-tenancy)
@@ -99,6 +101,7 @@ Cluster version is available [here](https://github.com/VictoriaMetrics/VictoriaM
 - [Roadmap](#roadmap)
 - [Contacts](#contacts)
 - [Community and contributions](#community-and-contributions)
+- [Third-party contributions](#third-party-contributions)
 - [Reporting bugs](#reporting-bugs)
 - [Victoria Metrics Logo](#victoria-metrics-logo)
   - [Logo Usage Guidelines](#logo-usage-guidelines)
@@ -490,11 +493,6 @@ More details may be found [here](https://github.com/VictoriaMetrics/VictoriaMetr
 Read [these instructions](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/43) on how to set up VictoriaMetrics as a service in your OS.
 
 
-### Third-party contributions
-
-* [Unofficial yum repository](https://copr.fedorainfracloud.org/coprs/antonpatsev/VictoriaMetrics/) ([source code](https://github.com/patsevanton/victoriametrics-rpm))
-
-
 ### How to work with snapshots?
 
 VictoriaMetrics can create [instant snapshots](https://medium.com/@valyala/how-victoriametrics-makes-instant-snapshots-for-multi-terabyte-time-series-data-e1f3fb0e0282)
@@ -534,6 +532,19 @@ the deleted time series isn't freed instantly - it is freed during subsequent me
 It is recommended verifying which metrics will be deleted with the call to `http://<victoria-metrics-addr>:8428/api/v1/series?match[]=<timeseries_selector_for_delete>`
 before actually deleting the metrics.
 
+The delete API is intended mainly for the following cases:
+
+- One-off deleting of accidentally written invalid (or undesired) time series.
+- One-off deleting of user data due to [GDPR](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation).
+
+It isn't recommended using delete API for the following cases, since it brings non-zero overhead:
+
+- Regular cleanups for unneded data. Just prevent writing unneeded data into VictoriaMetrics.
+- Reducing disk space usage by deleting unneded time series. This doesn't work as expected, since the deleted
+  time series occupy disk space until the next merge operation, which can never occur.
+
+It is better using `-retentionPeriod` command-line flag for efficient pruning of old data.
+
 
 ### How to export time series?
 
@@ -681,6 +692,16 @@ If you have Prometheus HA pairs with replicas `r1` and `r2` in each pair, then c
 to write data to `victoriametrics-addr-1`, while each `r2` should write data to `victoriametrics-addr-2`.
 
 
+### Retention
+
+Retention is configured with `-retentionPeriod` command-line flag. For instance, `-retentionPeriod=3` means
+that the data will be stored for 3 months and then deleted.
+Data is split in per-month subdirectories inside `<-storageDataPath>/data/small` and `<-storageDataPath>/data/big` folders.
+Directories for months outside the configured retention are deleted on the first day of new month.
+In order to keep data according to `-retentionPeriod` max disk space usage is going to be `-retentionPeriod` + 1 month.
+For example if `-retentionPeriod` is set to 1, data for January is deleted on March 1st.
+
+
 ### Multiple retentions
 
 Just start multiple VictoriaMetrics instances with distinct values for the following flags:
@@ -760,8 +781,11 @@ mkfs.ext4 ... -O 64bit,huge_file,extent -T huge
 
 ### Monitoring
 
-VictoriaMetrics exports internal metrics in Prometheus format on the `/metrics` page.
-Add this page to Prometheus' scrape config in order to collect VictoriaMetrics metrics.
+VictoriaMetrics exports internal metrics in Prometheus format at `/metrics` page.
+These metrics may be collected either via Prometheus by adding the corresponding scrape config to it.
+Alternatively they can be self-scraped by setting `-selfScrapeInterval` command-line flag to duration greater than 0.
+For example, `-scrapeInterval=10s` would enable self-scraping of `/metrics` page with 10 seconds interval.
+
 There are officials Grafana dashboards for [single-node VictoriaMetrics](https://grafana.com/dashboards/10229) and [clustered VictoriaMetrics](https://grafana.com/grafana/dashboards/11176).
 
 The most interesting metrics are:
@@ -802,6 +826,7 @@ The most interesting metrics are:
 
 ### Backfilling
 
+VictoriaMetrics accepts historical data in arbitrary order of time.
 Make sure that configured `-retentionPeriod` covers timestamps for the backfilled data.
 
 It is recommended disabling query cache with `-search.disableCache` command-line flag when writing
@@ -859,6 +884,7 @@ Contact us with any questions regarding VictoriaMetrics at [info@victoriametrics
 Feel free asking any questions regarding VictoriaMetrics:
 
 - [slack](http://slack.victoriametrics.com/)
+- [reddit](https://www.reddit.com/r/VictoriaMetrics/)
 - [telegram-en](https://t.me/VictoriaMetrics_en)
 - [telegram-ru](https://t.me/VictoriaMetrics_ru1)
 - [google groups](https://groups.google.com/forum/#!forum/victorametrics-users)
@@ -882,6 +908,13 @@ We are open to third-party pull requests provided they follow [KISS design princ
 Adhering `KISS` principle simplifies the resulting code and architecture, so it can be reviewed, understood and verified by many people.
 
 
+## Third-party contributions
+
+* [Unofficial yum repository](https://copr.fedorainfracloud.org/coprs/antonpatsev/VictoriaMetrics/) ([source code](https://github.com/patsevanton/victoriametrics-rpm))
+* [Prometheus -> VictoriaMetrics exporter #1](https://github.com/ryotarai/prometheus-tsdb-dump)
+* [Prometheus -> VictoriaMetrics exporter #2](https://github.com/AnchorFree/tsdb-remote-write)
+
+
 ## Reporting bugs
 
 Report bugs and propose new features [here](https://github.com/VictoriaMetrics/VictoriaMetrics/issues).

errcheck_excludes.txt

@@ -1,2 +1,3 @@
 fmt.Fprintf
+fmt.Fprint
 (net/http.ResponseWriter).Write

go.mod

@@ -1,31 +1,27 @@
 module github.com/VictoriaMetrics/VictoriaMetrics
 
 require (
-	cloud.google.com/go v0.50.0 // indirect
-	cloud.google.com/go/storage v1.4.0
-	github.com/VictoriaMetrics/fastcache v1.5.5
+	cloud.google.com/go v0.51.0 // indirect
+	cloud.google.com/go/storage v1.5.0
+	github.com/VictoriaMetrics/fastcache v1.5.7
 	github.com/VictoriaMetrics/metrics v1.9.3
-	github.com/aws/aws-sdk-go v1.26.8
+	github.com/aws/aws-sdk-go v1.28.7
 	github.com/cespare/xxhash/v2 v2.1.1
-	github.com/golang/groupcache v0.0.0-20191027212112-611e8accdfc9 // indirect
+	github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e // indirect
 	github.com/golang/snappy v0.0.1
-	github.com/jstemmer/go-junit-report v0.9.1 // indirect
-	github.com/klauspost/compress v1.9.4
+	github.com/klauspost/compress v1.9.8
 	github.com/valyala/fastjson v1.4.5
 	github.com/valyala/fastrand v1.0.0
 	github.com/valyala/gozstd v1.6.4
 	github.com/valyala/histogram v1.0.1
 	github.com/valyala/quicktemplate v1.4.1
-	go.opencensus.io v0.22.2 // indirect
-	golang.org/x/exp v0.0.0-20191224044220-1fea468a75e9 // indirect
-	golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553 // indirect
-	golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6 // indirect
-	golang.org/x/sys v0.0.0-20191224085550-c709ea063b76
-	golang.org/x/tools v0.0.0-20191224055732-dd894d0a8a40 // indirect
+	golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a // indirect
+	golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa // indirect
+	golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d // indirect
+	golang.org/x/sys v0.0.0-20200122134326-e047566fdf82
+	golang.org/x/tools v0.0.0-20200122042241-dc16b66866f1 // indirect
 	google.golang.org/api v0.15.0
-	google.golang.org/appengine v1.6.5 // indirect
-	google.golang.org/genproto v0.0.0-20191223191004-3caeed10a8bf // indirect
-	google.golang.org/grpc v1.26.0 // indirect
+	google.golang.org/genproto v0.0.0-20200117163144-32f20d992d24 // indirect
 )
 
 go 1.12

go.sum

@@ -5,32 +5,38 @@ cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6A
 cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
 cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=
 cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=
-cloud.google.com/go v0.50.0 h1:0E3eE8MX426vUOs7aHfI7aN1BrIzzzf4ccKCSfSjGmc=
 cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=
-cloud.google.com/go/bigquery v1.0.1 h1:hL+ycaJpVE9M7nLoiXb/Pn10ENE2u+oddxbD8uu0ZVU=
+cloud.google.com/go v0.51.0 h1:PvKAVQWCtlGUSlZkGW3QLelKaWq7KYv/MW1EboG8bfM=
+cloud.google.com/go v0.51.0/go.mod h1:hWtGJ6gnXH+KgDv+V0zFGDvpi07n3z8ZNj3T1RW0Gcw=
 cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
+cloud.google.com/go/bigquery v1.3.0 h1:sAbMqjY1PEQKZBWfbu6Y6bsupJ9c4QdHnzg/VvYTLcE=
+cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
 cloud.google.com/go/datastore v1.0.0 h1:Kt+gOPPp2LEPWp8CSfxhsM8ik9CcyE/gYu+0r+RnZvM=
 cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
-cloud.google.com/go/pubsub v1.0.1 h1:W9tAK3E57P75u0XLLR82LZyw8VpAnhmyTOxW9qzmyj8=
 cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
+cloud.google.com/go/pubsub v1.1.0 h1:9/vpR43S4aJaROxqQHQ3nH9lfyKKV0dC3vOmnw8ebQQ=
+cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=
 cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
-cloud.google.com/go/storage v1.4.0 h1:KDdqY5VTXBTqpSbctVTt0mVvfanP6JZzNzLE0qNY100=
-cloud.google.com/go/storage v1.4.0/go.mod h1:ZusYJWlOshgSBGbt6K3GnB3MT3H1xs2id9+TCl4fDBA=
+cloud.google.com/go/storage v1.5.0 h1:RPUcBvDeYgQFMfQu1eBMq6piD1SXmLH+vK3qjewZPus=
+cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
 dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
 github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
 github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
 github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
-github.com/VictoriaMetrics/fastcache v1.5.5 h1:HsBlzPgzKG0566YOl1mmfyz8SCU0zLKfbl9RDLsiLD8=
-github.com/VictoriaMetrics/fastcache v1.5.5/go.mod h1:ptDBkNMQI4RtmVo8VS/XwRY6RoTu1dAWCbrk+6WsEM8=
+github.com/VictoriaMetrics/fastcache v1.5.7 h1:4y6y0G8PRzszQUYIQHHssv/jgPHAb5qQuuDNdCbyAgw=
+github.com/VictoriaMetrics/fastcache v1.5.7/go.mod h1:ptDBkNMQI4RtmVo8VS/XwRY6RoTu1dAWCbrk+6WsEM8=
 github.com/VictoriaMetrics/metrics v1.9.3 h1:+1kZnOIb8RY825Nb9q9yMrPcOYuPE2GrZWxUh59XnHI=
 github.com/VictoriaMetrics/metrics v1.9.3/go.mod h1:LU2j9qq7xqZYXz8tF3/RQnB2z2MbZms5TDiIg9/NHiQ=
 github.com/allegro/bigcache v1.2.1-0.20190218064605-e24eb225f156 h1:eMwmnE/GDgah4HI848JfFxHt+iPb26b4zyfspmqY0/8=
 github.com/allegro/bigcache v1.2.1-0.20190218064605-e24eb225f156/go.mod h1:Cb/ax3seSYIx7SuZdm2G2xzfwmv3TPSk2ucNfQESPXM=
-github.com/aws/aws-sdk-go v1.26.8 h1:W+MPuCFLSO/itZkZ5GFOui0YC1j3lZ507/m5DFPtzE4=
-github.com/aws/aws-sdk-go v1.26.8/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
+github.com/aws/aws-sdk-go v1.28.7 h1:8RUfzsEmyXR8a9G7o2snfUKwrSuqks/k4C7TIfXDDrY=
+github.com/aws/aws-sdk-go v1.28.7/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
 github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
 github.com/cespare/xxhash/v2 v2.1.1 h1:6MnRN8NT7+YBpUIWxHtefFZOKTAPgGjpQSxqLNn0+qY=
 github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
+github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
+github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
+github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
 github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
@@ -42,8 +48,10 @@ github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2
 github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58=
 github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
 github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
-github.com/golang/groupcache v0.0.0-20191027212112-611e8accdfc9 h1:uHTyIjqVhYRhLbJ8nIiOJHkEZZ+5YoOsAbD3sk82NiE=
-github.com/golang/groupcache v0.0.0-20191027212112-611e8accdfc9/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
+github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7 h1:5ZkaAPbicIKTF2I64qf5Fh8Aa83Q/dnOafMYV0OMwjA=
+github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
+github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e h1:1r7pUrabqp18hOBcwBwiTsbnFeTZHV9eER/QT5JVZxY=
+github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
 github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
 github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
 github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
@@ -56,18 +64,21 @@ github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEW
 github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
 github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
 github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
-github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY=
 github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
+github.com/google/go-cmp v0.3.1 h1:Xye71clBPdm5HgqGwUkwhbynsUJZhDbS20FvLhQ2izg=
+github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
 github.com/google/martian v2.1.0+incompatible h1:/CP5g8u/VJHijgedC/Legn3BAbAaWPgecwXBIDzw5no=
 github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
 github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
 github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
+github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
 github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
 github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
 github.com/googleapis/gax-go/v2 v2.0.5 h1:sjZBwGj9Jlw33ImPtvFviGYvseOtDM7hkSKB7+Tv3SM=
 github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
 github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
 github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
+github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
 github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af h1:pmfjZENx5imkbgOkpRUYLnmbU7UEFbjtDA2hxJ1ichM=
 github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
 github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
@@ -76,8 +87,8 @@ github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/X
 github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
 github.com/klauspost/compress v1.4.0/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
 github.com/klauspost/compress v1.4.1/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
-github.com/klauspost/compress v1.9.4 h1:xhvAeUPQ2drNUhKtrGdTGNvV9nNafHMUkRyLkzxJoB4=
-github.com/klauspost/compress v1.9.4/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
+github.com/klauspost/compress v1.9.8 h1:VMAMUUOh+gaxKTMk+zqbjsSjsIcUcL/LF4o63i82QyA=
+github.com/klauspost/compress v1.9.8/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
 github.com/klauspost/cpuid v0.0.0-20180405133222-e7e905edc00e/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
 github.com/klauspost/cpuid v1.2.0/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
 github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
@@ -90,7 +101,6 @@ github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZN
 github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
 github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
-github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
 github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
 github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
 github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
@@ -120,12 +130,12 @@ golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL
 golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
 golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=
-golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136 h1:A1gGSx58LAGVHUUsOf7IiR0u8Xb6W51gRwfDBhkdcaw=
 golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=
-golang.org/x/exp v0.0.0-20191129062945-2f5052295587 h1:5Uz0rkjCFu9BC9gCRN7EkwVvhNyQgGWb8KNJrPwBoHY=
 golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
-golang.org/x/exp v0.0.0-20191224044220-1fea468a75e9 h1:HLuLY2KniBsHW28uXd1i2UZKjifeJUy//P/wTK6AJwI=
-golang.org/x/exp v0.0.0-20191224044220-1fea468a75e9/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
+golang.org/x/exp v0.0.0-20191227195350-da58074b4299 h1:zQpM52jfKHG6II1ISZY1ZcpygvuSFZpLwfluuF89XOg=
+golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
+golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a h1:7Wlg8L54In96HTWOaI4sreLJ6qfyGuvSau5el3fK41Y=
+golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
 golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
 golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
 golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
@@ -134,7 +144,6 @@ golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTk
 golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
 golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
 golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
-golang.org/x/lint v0.0.0-20190930215403-16217165b5de h1:5hukYrvBGR8/eNkX5mdUezrA6JiaEZDtJb9Ei+1LlBs=
 golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
 golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f h1:J5lckAjkw6qYlOZNj90mLYNTEKDvWeuc1yieZ8qUzUE=
 golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
@@ -155,19 +164,19 @@ golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn
 golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
 golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553 h1:efeOvDhwQ29Dj3SdAV/MJf8oukgn+8D8WgaCaRMchF8=
 golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa h1:F+8P+gmewFQYRk6JoLQLwjBCTu3mcIURZfNkVweuRKA=
+golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
-golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45 h1:SVwTIAaPC2U/AvvLNZ2a7OVsmBpC8L5BlwK1whH3hm0=
 golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
-golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6 h1:pE8b58s1HRDMi8RDc79m0HISf9D4TzseP40cEA6IGfs=
 golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
+golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d h1:TzXSXBo42m9gQenoE3b9BGiEpg5IG2JkU5FkPIawgtw=
+golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
 golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
 golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e h1:vcxGaoTs7kV8m5Np9uUNQin4BrLOthgV7252N8V+FwY=
 golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -180,8 +189,10 @@ golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20191224085550-c709ea063b76 h1:Dho5nD6R3PcW2SH1or8vS0dszDaXRxIw55lBX7XiE5g=
-golang.org/x/sys v0.0.0-20191224085550-c709ea063b76/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200122134326-e047566fdf82 h1:ywK/j/KkyTHcdyYSZNXGjMwgmDSfjglYZ3vStQ/gSCU=
+golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
@@ -203,18 +214,20 @@ golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgw
 golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
 golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
 golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
 golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
 golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
 golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20191224055732-dd894d0a8a40 h1:UyP2XDSgSc8ldYCxAK735zQxeH3Gd81sK7Iy7AoaVxk=
-golang.org/x/tools v0.0.0-20191224055732-dd894d0a8a40/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.0.0-20200122042241-dc16b66866f1 h1:468gVSKEm8NObiNTQ3it08aAGsPfuvz+WXUHmnq8Wws=
+golang.org/x/tools v0.0.0-20200122042241-dc16b66866f1/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
 google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
 google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
 google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
-google.golang.org/api v0.14.0 h1:uMf5uLi4eQMRrMKhCplNik4U4H8Z6C1br3zOtAa/aDE=
+google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
 google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
 google.golang.org/api v0.15.0 h1:yzlyyDW/J0w8yNFJIhiAJy4kq74S+1DOLdawELNxFMA=
 google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
@@ -232,10 +245,12 @@ google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRn
 google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
 google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
 google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=
+google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
 google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
 google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
-google.golang.org/genproto v0.0.0-20191223191004-3caeed10a8bf h1:1x8rC5/IgdLMPbPTvlQTN28+rcy8XL9Q19UWUMDyqYs=
-google.golang.org/genproto v0.0.0-20191223191004-3caeed10a8bf/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
+google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
+google.golang.org/genproto v0.0.0-20200117163144-32f20d992d24 h1:wDju+RU97qa0FZT0QnZDg9Uc2dH0Ql513kFvHocz+WM=
+google.golang.org/genproto v0.0.0-20200117163144-32f20d992d24/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
 google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
 google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
 google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=

lib/backup/actions/backup.go

@@ -166,8 +166,8 @@ func runBackup(src *fslocal.FS, dst common.RemoteFS, origin common.OriginFS, con
 		}
 	}
 
-	logger.Infof("backed up %d bytes in %s; deleted %d bytes; server-side copied %d bytes; uploaded %d bytes",
-		backupSize, time.Since(startTime), deleteSize, copySize, uploadSize)
+	logger.Infof("backed up %d bytes in %.3f seconds; deleted %d bytes; server-side copied %d bytes; uploaded %d bytes",
+		backupSize, time.Since(startTime).Seconds(), deleteSize, copySize, uploadSize)
 
 	return nil
 }

lib/backup/actions/restore.go

@@ -62,7 +62,7 @@ func (r *Restore) Run() error {
 		}
 		if !ok {
 			return fmt.Errorf("cannot find %s file in %s; this means either incomplete backup or old backup; "+
-				"pass `-skipBackupCompleteCheck` command-line flag if you still need restoring from this backup", fscommon.BackupCompleteFilename, src)
+				"pass -skipBackupCompleteCheck command-line flag if you still need restoring from this backup", fscommon.BackupCompleteFilename, src)
 		}
 	}
 
@@ -182,7 +182,8 @@ func (r *Restore) Run() error {
 		}
 	}
 
-	logger.Infof("restored %d bytes from backup in %s; deleted %d bytes; downloaded %d bytes", backupSize, time.Since(startTime), deleteSize, downloadSize)
+	logger.Infof("restored %d bytes from backup in %.3f seconds; deleted %d bytes; downloaded %d bytes",
+		backupSize, time.Since(startTime).Seconds(), deleteSize, downloadSize)
 
 	return nil
 }

lib/bytesutil/bytebuffer.go

@@ -11,9 +11,9 @@ import (
 
 var (
 	// Verify ByteBuffer implements the given interfaces.
-	_ io.Writer       = &ByteBuffer{}
-	_ fs.ReadAtCloser = &ByteBuffer{}
-	_ io.ReaderFrom   = &ByteBuffer{}
+	_ io.Writer           = &ByteBuffer{}
+	_ fs.MustReadAtCloser = &ByteBuffer{}
+	_ io.ReaderFrom       = &ByteBuffer{}
 
 	// Verify reader implement filestream.ReadCloser interface.
 	_ filestream.ReadCloser = &reader{}
@@ -36,8 +36,8 @@ func (bb *ByteBuffer) Write(p []byte) (int, error) {
 	return len(p), nil
 }
 
-// ReadAt reads len(p) bytes starting from the given offset.
-func (bb *ByteBuffer) ReadAt(p []byte, offset int64) {
+// MustReadAt reads len(p) bytes starting from the given offset.
+func (bb *ByteBuffer) MustReadAt(p []byte, offset int64) {
 	if offset < 0 {
 		logger.Panicf("BUG: cannot read at negative offset=%d", offset)
 	}

lib/bytesutil/bytebuffer_test.go

@@ -218,7 +218,7 @@ func TestByteBufferRead(t *testing.T) {
 	}
 }
 
-func TestByteBufferReadAt(t *testing.T) {
+func TestByteBufferMustReadAt(t *testing.T) {
 	testStr := "foobar baz"
 
 	var bb ByteBuffer
@@ -232,7 +232,7 @@ func TestByteBufferReadAt(t *testing.T) {
 				t.Fatalf("expecting non-nil error when reading at negative offset")
 			}
 		}()
-		bb.ReadAt(p, -1)
+		bb.MustReadAt(p, -1)
 	}()
 
 	// Try reading past the end of buffer
@@ -242,18 +242,18 @@ func TestByteBufferReadAt(t *testing.T) {
 				t.Fatalf("expecting non-nil error when reading past the end of buffer")
 			}
 		}()
-		bb.ReadAt(p, int64(len(testStr))+1)
+		bb.MustReadAt(p, int64(len(testStr))+1)
 	}()
 
 	// Try reading the first byte
 	n := len(p)
-	bb.ReadAt(p, 0)
+	bb.MustReadAt(p, 0)
 	if string(p) != testStr[:n] {
 		t.Fatalf("unexpected value read: %q; want %q", p, testStr[:n])
 	}
 
 	// Try reading the last byte
-	bb.ReadAt(p, int64(len(testStr))-1)
+	bb.MustReadAt(p, int64(len(testStr))-1)
 	if string(p) != testStr[len(testStr)-1:] {
 		t.Fatalf("unexpected value read: %q; want %q", p, testStr[len(testStr)-1:])
 	}
@@ -266,18 +266,18 @@ func TestByteBufferReadAt(t *testing.T) {
 			}
 		}()
 		p := make([]byte, 10)
-		bb.ReadAt(p, int64(len(testStr))-3)
+		bb.MustReadAt(p, int64(len(testStr))-3)
 	}()
 
 	// Try reading multiple bytes from the middle
 	p = make([]byte, 3)
-	bb.ReadAt(p, 2)
+	bb.MustReadAt(p, 2)
 	if string(p) != testStr[2:2+len(p)] {
 		t.Fatalf("unexpected value read: %q; want %q", p, testStr[2:2+len(p)])
 	}
 }
 
-func TestByteBufferReadAtParallel(t *testing.T) {
+func TestByteBufferMustReadAtParallel(t *testing.T) {
 	ch := make(chan error, 10)
 	var bb ByteBuffer
 	bb.B = []byte("foo bar baz adsf adsf dsakjlkjlkj2l34324")
@@ -285,7 +285,7 @@ func TestByteBufferReadAtParallel(t *testing.T) {
 		go func() {
 			p := make([]byte, 3)
 			for i := 0; i < len(bb.B)-len(p); i++ {
-				bb.ReadAt(p, int64(i))
+				bb.MustReadAt(p, int64(i))
 			}
 			ch <- nil
 		}()

lib/fs/copy_mmap_cgo.go

@@ -0,0 +1,34 @@
+// +build cgo
+
+package fs
+
+// #cgo CFLAGS: -O3
+//
+// #include <stdint.h>  // for uintptr_t
+// #include <string.h>  // for memcpy
+//
+// // The memcpy_wrapper allows avoiding memory allocations during calls from Go.
+// // See https://github.com/golang/go/issues/24450 .
+// static void memcpy_wrapper(uintptr_t dst, uintptr_t src, size_t n) {
+//     memcpy((void*)dst, (void*)src, n);
+// }
+import "C"
+
+import (
+	"runtime"
+	"unsafe"
+)
+
+// copyMmap copies len(dst) bytes from src to dst.
+func copyMmap(dst, src []byte) {
+	// Copy data from mmap'ed src via cgo call in order to protect from goroutine stalls
+	// when the copied data isn't available in RAM, so the OS triggers reading the data from file.
+	// See https://medium.com/@valyala/mmap-in-go-considered-harmful-d92a25cb161d for details.
+	dstPtr := C.uintptr_t(uintptr(unsafe.Pointer(&dst[0])))
+	srcPtr := C.uintptr_t(uintptr(unsafe.Pointer(&src[0])))
+	C.memcpy_wrapper(dstPtr, srcPtr, C.size_t(len(dst)))
+
+	// Prevent from GC'ing src or dst during C.memcpy_wrapper call.
+	runtime.KeepAlive(src)
+	runtime.KeepAlive(dst)
+}

lib/fs/copy_mmap_nocgo.go

@@ -0,0 +1,12 @@
+// +build !cgo
+
+package fs
+
+// copyMmap copies len(dst) bytes from src to dst.
+func copyMmap(dst, src []byte) {
+	// This may lead to goroutines stalls when the copied data isn't available in RAM.
+	// In this case the OS triggers reading the data from file.
+	// See https://medium.com/@valyala/mmap-in-go-considered-harmful-d92a25cb161d for details.
+	// TODO: fix this
+	copy(dst, src)
+}

lib/fs/fadvise_darwin.go

@@ -4,9 +4,7 @@ import (
 	"os"
 )
 
-// MustFadviseSequentialRead hints the OS that f is read mostly sequentially.
-//
-// if prefetch is set, then the OS is hinted to prefetch f data.
-func MustFadviseSequentialRead(f *os.File, prefetch bool) {
+func fadviseSequentialRead(f *os.File, prefetch bool) error {
 	// TODO: implement this properly
+	return nil
 }

lib/fs/fadvise_unix.go

@@ -3,22 +3,20 @@
 package fs
 
 import (
+	"fmt"
 	"os"
 
-	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
 	"golang.org/x/sys/unix"
 )
 
-// MustFadviseSequentialRead hints the OS that f is read mostly sequentially.
-//
-// if prefetch is set, then the OS is hinted to prefetch f data.
-func MustFadviseSequentialRead(f *os.File, prefetch bool) {
+func fadviseSequentialRead(f *os.File, prefetch bool) error {
 	fd := int(f.Fd())
 	mode := unix.FADV_SEQUENTIAL
 	if prefetch {
 		mode |= unix.FADV_WILLNEED
 	}
 	if err := unix.Fadvise(int(fd), 0, 0, mode); err != nil {
-		logger.Panicf("FATAL: error returned from unix.Fadvise(%d): %s", mode, err)
+		return fmt.Errorf("error returned from unix.Fadvise(%d): %s", mode, err)
 	}
+	return nil
 }

lib/fs/fs.go

@@ -10,69 +10,9 @@ import (
 
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/filestream"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
-	"github.com/VictoriaMetrics/metrics"
 	"golang.org/x/sys/unix"
 )
 
-// ReadAtCloser is rand-access read interface.
-type ReadAtCloser interface {
-	// ReadAt must read len(p) bytes from offset off to p.
-	ReadAt(p []byte, off int64)
-
-	// MustClose must close the reader.
-	MustClose()
-}
-
-// ReaderAt implements rand-access read.
-type ReaderAt struct {
-	f *os.File
-}
-
-// ReadAt reads len(p) bytes from off to p.
-func (ra *ReaderAt) ReadAt(p []byte, off int64) {
-	if len(p) == 0 {
-		return
-	}
-	n, err := ra.f.ReadAt(p, off)
-	if err != nil {
-		logger.Panicf("FATAL: cannot read %d bytes at offset %d of file %q: %s", len(p), off, ra.f.Name(), err)
-	}
-	if n != len(p) {
-		logger.Panicf("FATAL: unexpected number of bytes read; got %d; want %d", n, len(p))
-	}
-	readCalls.Inc()
-	readBytes.Add(len(p))
-}
-
-// MustClose closes ra.
-func (ra *ReaderAt) MustClose() {
-	if err := ra.f.Close(); err != nil {
-		logger.Panicf("FATAL: cannot close file %q: %s", ra.f.Name(), err)
-	}
-	readersCount.Dec()
-}
-
-// OpenReaderAt opens a file on the given path for random-read access.
-//
-// The file must be closed with MustClose when no longer needed.
-func OpenReaderAt(path string) (*ReaderAt, error) {
-	f, err := os.Open(path)
-	if err != nil {
-		return nil, err
-	}
-	readersCount.Inc()
-	ra := &ReaderAt{
-		f: f,
-	}
-	return ra, nil
-}
-
-var (
-	readCalls    = metrics.NewCounter(`vm_fs_read_calls_total`)
-	readBytes    = metrics.NewCounter(`vm_fs_read_bytes_total`)
-	readersCount = metrics.NewCounter(`vm_fs_readers`)
-)
-
 // MustSyncPath syncs contents of the given path.
 func MustSyncPath(path string) {
 	d, err := os.Open(path)

lib/fs/reader_at.go

@@ -0,0 +1,128 @@
+package fs
+
+import (
+	"flag"
+	"fmt"
+	"os"
+
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
+	"github.com/VictoriaMetrics/metrics"
+	"golang.org/x/sys/unix"
+)
+
+var disableMmap = flag.Bool("fs.disableMmap", false, "Whether to use pread() instead of mmap() for reading data files")
+
+// MustReadAtCloser is rand-access read interface.
+type MustReadAtCloser interface {
+	// MustReadAt must read len(p) bytes from offset off to p.
+	MustReadAt(p []byte, off int64)
+
+	// MustClose must close the reader.
+	MustClose()
+}
+
+// ReaderAt implements rand-access reader.
+type ReaderAt struct {
+	f        *os.File
+	mmapData []byte
+}
+
+// MustReadAt reads len(p) bytes at off from r.
+func (r *ReaderAt) MustReadAt(p []byte, off int64) {
+	if len(p) == 0 {
+		return
+	}
+	if len(r.mmapData) == 0 || len(p) > 8*1024*1024 {
+		// Read big blocks directly from file.
+		// This could be faster than reading these blocks from mmap,
+		// since it triggers less page faults.
+		n, err := r.f.ReadAt(p, off)
+		if err != nil {
+			logger.Panicf("FATAL: cannot read %d bytes at offset %d of file %q: %s", len(p), off, r.f.Name(), err)
+		}
+		if n != len(p) {
+			logger.Panicf("FATAL: unexpected number of bytes read; got %d; want %d", n, len(p))
+		}
+	} else {
+		if off < 0 || off > int64(len(r.mmapData)-len(p)) {
+			logger.Panicf("off=%d is out of allowed range [0...%d] for len(p)=%d", off, len(r.mmapData)-len(p), len(p))
+		}
+		copyMmap(p, r.mmapData[off:])
+	}
+	readCalls.Inc()
+	readBytes.Add(len(p))
+}
+
+// MustClose closes r.
+func (r *ReaderAt) MustClose() {
+	fname := r.f.Name()
+	if len(r.mmapData) > 0 {
+		if err := unix.Munmap(r.mmapData); err != nil {
+			logger.Panicf("FATAL: cannot unmap data for file %q: %s", fname, err)
+		}
+		r.mmapData = nil
+	}
+	MustClose(r.f)
+	r.f = nil
+	readersCount.Dec()
+}
+
+// MustFadviseSequentialRead hints the OS that f is read mostly sequentially.
+//
+// if prefetch is set, then the OS is hinted to prefetch f data.
+func (r *ReaderAt) MustFadviseSequentialRead(prefetch bool) {
+	if err := fadviseSequentialRead(r.f, prefetch); err != nil {
+		logger.Panicf("FATAL: error in fadviseSequentialRead(%q, %v): %s", r.f.Name(), prefetch, err)
+	}
+}
+
+// OpenReaderAt opens ReaderAt for reading from filename.
+//
+// MustClose must be called on the returned ReaderAt when it is no longer needed.
+func OpenReaderAt(path string) (*ReaderAt, error) {
+	f, err := os.Open(path)
+	if err != nil {
+		return nil, fmt.Errorf("cannot open file %q for reader: %s", path, err)
+	}
+	var r ReaderAt
+	r.f = f
+	if !*disableMmap {
+		data, err := mmapFile(f)
+		if err != nil {
+			MustClose(f)
+			return nil, fmt.Errorf("cannot init reader for %q: %s", path, err)
+		}
+		r.mmapData = data
+	}
+	r.MustFadviseSequentialRead(false)
+	readersCount.Inc()
+	return &r, nil
+}
+
+var (
+	readCalls    = metrics.NewCounter(`vm_fs_read_calls_total`)
+	readBytes    = metrics.NewCounter(`vm_fs_read_bytes_total`)
+	readersCount = metrics.NewCounter(`vm_fs_readers`)
+)
+
+func mmapFile(f *os.File) ([]byte, error) {
+	fi, err := f.Stat()
+	if err != nil {
+		return nil, fmt.Errorf("error in stat: %s", err)
+	}
+	size := fi.Size()
+	if size == 0 {
+		return nil, nil
+	}
+	if size < 0 {
+		return nil, fmt.Errorf("got negative file size: %d bytes", size)
+	}
+	if int64(int(size)) != size {
+		return nil, fmt.Errorf("file is too big to be mmap'ed: %d bytes", size)
+	}
+	data, err := unix.Mmap(int(f.Fd()), 0, int(size), unix.PROT_READ, unix.MAP_SHARED)
+	if err != nil {
+		return nil, fmt.Errorf("cannot mmap file with size %d: %s", size, err)
+	}
+	return data, nil
+}

lib/httpserver/httpserver.go

@@ -2,7 +2,6 @@ package httpserver
 
 import (
 	"bufio"
-	"compress/gzip"
 	"context"
 	"crypto/tls"
 	"flag"
@@ -20,12 +19,13 @@ import (
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/netutil"
 	"github.com/VictoriaMetrics/metrics"
+	"github.com/klauspost/compress/gzip"
 )
 
 var (
-	tlsEnable   = flag.Bool("tls", false, "Whether to enable TLS (aka HTTPS) for incoming requests. `-tlsCertFile` and `-tlsKeyFile` must be set if `-tls` is set")
-	tlsCertFile = flag.String("tlsCertFile", "", "Path to file with TLS certificate. Used only if `-tls` is set. Prefer ECDSA certs instead of RSA certs, since RSA certs are slow")
-	tlsKeyFile  = flag.String("tlsKeyFile", "", "Path to file with TLS key. Used only if `-tls` is set")
+	tlsEnable   = flag.Bool("tls", false, "Whether to enable TLS (aka HTTPS) for incoming requests. -tlsCertFile and -tlsKeyFile must be set if -tls is set")
+	tlsCertFile = flag.String("tlsCertFile", "", "Path to file with TLS certificate. Used only if -tls is set. Prefer ECDSA certs instead of RSA certs, since RSA certs are slow")
+	tlsKeyFile  = flag.String("tlsKeyFile", "", "Path to file with TLS key. Used only if -tls is set")
 
 	httpAuthUsername = flag.String("httpAuth.username", "", "Username for HTTP Basic Auth. The authentication is disabled if empty. See also -httpAuth.password")
 	httpAuthPassword = flag.String("httpAuth.password", "", "Password for HTTP Basic Auth. The authentication is disabled if -httpAuth.username is empty")
@@ -181,7 +181,7 @@ func handlerWrapper(w http.ResponseWriter, r *http.Request, rh RequestHandler) {
 		}
 		startTime := time.Now()
 		w.Header().Set("Content-Type", "text/plain")
-		writePrometheusMetrics(w)
+		WritePrometheusMetrics(w)
 		metricsHandlerDuration.UpdateDuration(startTime)
 		return
 	default:
@@ -418,7 +418,7 @@ var (
 // Errorf writes formatted error message to w and to logger.
 func Errorf(w http.ResponseWriter, format string, args ...interface{}) {
 	errStr := fmt.Sprintf(format, args...)
-	logger.Errorf("%s", errStr)
+	logger.ErrorfSkipframes(1, "%s", errStr)
 
 	// Extract statusCode from args
 	statusCode := http.StatusBadRequest

lib/httpserver/metrics.go

@@ -12,7 +12,8 @@ import (
 	"github.com/VictoriaMetrics/metrics"
 )
 
-func writePrometheusMetrics(w io.Writer) {
+// WritePrometheusMetrics writes all the registered metrics to w in Prometheus exposition format.
+func WritePrometheusMetrics(w io.Writer) {
 	metrics.WritePrometheus(w, true)
 
 	fmt.Fprintf(w, "vm_app_version{version=%q} 1\n", buildinfo.Version)

lib/logger/logger.go

@@ -4,6 +4,7 @@ import (
 	"errors"
 	"flag"
 	"fmt"
+	"io"
 	"log"
 	"os"
 	"runtime"
@@ -19,6 +20,7 @@ import (
 var (
 	loggerLevel  = flag.String("loggerLevel", "INFO", "Minimum level of errors to log. Possible values: INFO, ERROR, FATAL, PANIC")
 	loggerFormat = flag.String("loggerFormat", "default", "Format for logs. Possible values: default, json")
+	loggerOutput = flag.String("loggerOutput", "stderr", "Output for the logs. Supported values: stderr, stdout")
 )
 
 // Init initializes the logger.
@@ -27,12 +29,26 @@ var (
 //
 // There is no need in calling Init from tests.
 func Init() {
+	setLoggerOutput()
 	validateLoggerLevel()
 	validateLoggerFormat()
 	go errorsLoggedCleaner()
 	logAllFlags()
 }
 
+func setLoggerOutput() {
+	switch *loggerOutput {
+	case "stderr":
+		output = os.Stderr
+	case "stdout":
+		output = os.Stdout
+	default:
+		panic(fmt.Errorf("FATAL: unsupported `loggerOutput` value: %q; supported values are: stderr, stdout", *loggerOutput))
+	}
+}
+
+var output io.Writer = os.Stderr
+
 func validateLoggerLevel() {
 	switch *loggerLevel {
 	case "INFO", "ERROR", "FATAL", "PANIC":
@@ -68,6 +84,11 @@ func Errorf(format string, args ...interface{}) {
 	logLevel("ERROR", format, args...)
 }
 
+// ErrorfSkipframes logs error message and skips the given number of frames for the caller.
+func ErrorfSkipframes(skipframes int, format string, args ...interface{}) {
+	logLevelSkipframes(skipframes, "ERROR", format, args...)
+}
+
 // Fatalf logs fatal message and terminates the app.
 func Fatalf(format string, args ...interface{}) {
 	logLevel("FATAL", format, args...)
@@ -79,19 +100,15 @@ func Panicf(format string, args ...interface{}) {
 }
 
 func logLevel(level, format string, args ...interface{}) {
+	logLevelSkipframes(1, level, format, args...)
+}
+
+func logLevelSkipframes(skipframes int, level, format string, args ...interface{}) {
 	if shouldSkipLog(level) {
 		return
 	}
-
-	// rate limit ERROR log messages
-	if level == "ERROR" {
-		if n := atomic.AddUint64(&errorsLogged, 1); n > 10 {
-			return
-		}
-	}
-
 	msg := fmt.Sprintf(format, args...)
-	logMessage(level, msg, 3)
+	logMessage(level, msg, 3+skipframes)
 }
 
 func errorsLoggedCleaner() {
@@ -107,13 +124,18 @@ type logWriter struct {
 }
 
 func (lw *logWriter) Write(p []byte) (int, error) {
-	if !shouldSkipLog("ERROR") {
-		logMessage("ERROR", string(p), 4)
-	}
+	logLevelSkipframes(2, "ERROR", "%s", p)
 	return len(p), nil
 }
 
 func logMessage(level, msg string, skipframes int) {
+	// rate limit ERROR log messages
+	if level == "ERROR" {
+		if n := atomic.AddUint64(&errorsLogged, 1); n > 10 {
+			return
+		}
+	}
+
 	timestamp := time.Now().UTC().Format("2006-01-02T15:04:05.000Z")
 	levelLowercase := strings.ToLower(level)
 	_, file, line, ok := runtime.Caller(skipframes)
@@ -139,7 +161,7 @@ func logMessage(level, msg string, skipframes int) {
 
 	// Serialize writes to log.
 	mu.Lock()
-	fmt.Fprint(os.Stderr, logMsg)
+	fmt.Fprint(output, logMsg)
 	mu.Unlock()
 
 	// Increment vm_log_messages_total
@@ -149,6 +171,10 @@ func logMessage(level, msg string, skipframes int) {
 
 	switch level {
 	case "PANIC":
+		if *loggerFormat == "json" {
+			// Do not clutter `json` output with panic stack trace
+			os.Exit(-1)
+		}
 		panic(errors.New(msg))
 	case "FATAL":
 		os.Exit(-1)

lib/mergeset/part.go

@@ -5,7 +5,7 @@ import (
 	"path/filepath"
 	"sync"
 	"sync/atomic"
-	"unsafe"
+	"time"
 
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/filestream"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/fs"
@@ -44,7 +44,7 @@ var (
 	maxCachedInmemoryBlocksPerPartOnce sync.Once
 )
 
-type partInternals struct {
+type part struct {
 	ph partHeader
 
 	path string
@@ -53,19 +53,12 @@ type partInternals struct {
 
 	mrs []metaindexRow
 
-	indexFile fs.ReadAtCloser
-	itemsFile fs.ReadAtCloser
-	lensFile  fs.ReadAtCloser
-}
+	indexFile fs.MustReadAtCloser
+	itemsFile fs.MustReadAtCloser
+	lensFile  fs.MustReadAtCloser
 
-type part struct {
-	partInternals
-
-	// Align atomic counters inside caches by 8 bytes on 32-bit architectures.
-	// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/212 .
-	_         [(8 - (unsafe.Sizeof(partInternals{}) % 8)) % 8]byte
-	idxbCache indexBlockCache
-	ibCache   inmemoryBlockCache
+	idxbCache *indexBlockCache
+	ibCache   *inmemoryBlockCache
 }
 
 func openFilePart(path string) (*part, error) {
@@ -114,7 +107,7 @@ func openFilePart(path string) (*part, error) {
 	return newPart(&ph, path, size, metaindexFile, indexFile, itemsFile, lensFile)
 }
 
-func newPart(ph *partHeader, path string, size uint64, metaindexReader filestream.ReadCloser, indexFile, itemsFile, lensFile fs.ReadAtCloser) (*part, error) {
+func newPart(ph *partHeader, path string, size uint64, metaindexReader filestream.ReadCloser, indexFile, itemsFile, lensFile fs.MustReadAtCloser) (*part, error) {
 	var errors []error
 	mrs, err := unmarshalMetaindexRows(nil, metaindexReader)
 	if err != nil {
@@ -132,8 +125,8 @@ func newPart(ph *partHeader, path string, size uint64, metaindexReader filestrea
 	p.lensFile = lensFile
 
 	p.ph.CopyFrom(ph)
-	p.idxbCache.Init()
-	p.ibCache.Init()
+	p.idxbCache = newIndexBlockCache()
+	p.ibCache = newInmemoryBlockCache()
 
 	if len(errors) > 0 {
 		// Return only the first error, since it has no sense in returning all errors.
@@ -149,8 +142,8 @@ func (p *part) MustClose() {
 	p.itemsFile.MustClose()
 	p.lensFile.MustClose()
 
-	p.idxbCache.Reset()
-	p.ibCache.Reset()
+	p.idxbCache.MustClose()
+	p.ibCache.MustClose()
 }
 
 type indexBlock struct {
@@ -179,27 +172,72 @@ type indexBlockCache struct {
 	requests uint64
 	misses   uint64
 
-	m         map[uint64]*indexBlock
-	missesMap map[uint64]uint64
-	mu        sync.RWMutex
+	m  map[uint64]*indexBlockCacheEntry
+	mu sync.RWMutex
+
+	cleanerStopCh chan struct{}
+	cleanerWG     sync.WaitGroup
 }
 
-func (idxbc *indexBlockCache) Init() {
-	idxbc.m = make(map[uint64]*indexBlock)
-	idxbc.missesMap = make(map[uint64]uint64)
-	idxbc.requests = 0
-	idxbc.misses = 0
+type indexBlockCacheEntry struct {
+	// Atomically updated counters must go first in the struct, so they are properly
+	// aligned to 8 bytes on 32-bit architectures.
+	// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/212
+	lastAccessTime uint64
+
+	idxb *indexBlock
 }
 
-func (idxbc *indexBlockCache) Reset() {
+func newIndexBlockCache() *indexBlockCache {
+	var idxbc indexBlockCache
+	idxbc.m = make(map[uint64]*indexBlockCacheEntry)
+	idxbc.cleanerStopCh = make(chan struct{})
+	idxbc.cleanerWG.Add(1)
+	go func() {
+		defer idxbc.cleanerWG.Done()
+		idxbc.cleaner()
+	}()
+	return &idxbc
+}
+
+func (idxbc *indexBlockCache) MustClose() {
+	close(idxbc.cleanerStopCh)
+	idxbc.cleanerWG.Wait()
+
 	atomic.AddUint64(&indexBlockCacheRequests, idxbc.requests)
 	atomic.AddUint64(&indexBlockCacheMisses, idxbc.misses)
 	// It is safe returning idxbc.m to pool, since the Reset must be called
 	// when the idxbc entries are no longer accessed by concurrent goroutines.
-	for _, idxb := range idxbc.m {
-		putIndexBlock(idxb)
+	for _, idxbe := range idxbc.m {
+		putIndexBlock(idxbe.idxb)
 	}
-	idxbc.Init()
+	idxbc.m = nil
+}
+
+// cleaner periodically cleans least recently used items.
+func (idxbc *indexBlockCache) cleaner() {
+	t := time.NewTimer(5 * time.Second)
+	for {
+		select {
+		case <-t.C:
+			idxbc.cleanByTimeout()
+		case <-idxbc.cleanerStopCh:
+			t.Stop()
+			return
+		}
+	}
+}
+
+func (idxbc *indexBlockCache) cleanByTimeout() {
+	currentTime := atomic.LoadUint64(&currentTimestamp)
+	idxbc.mu.Lock()
+	for k, idxbe := range idxbc.m {
+		// Delete items accessed more than 10 minutes ago.
+		if currentTime-atomic.LoadUint64(&idxbe.lastAccessTime) > 10*60 {
+			delete(idxbc.m, k)
+		}
+	}
+	idxbc.mu.Unlock()
 }
 
 var (
@@ -210,16 +248,17 @@ var (
 func (idxbc *indexBlockCache) Get(k uint64) *indexBlock {
 	atomic.AddUint64(&idxbc.requests, 1)
 	idxbc.mu.RLock()
-	idxb := idxbc.m[k]
+	idxbe := idxbc.m[k]
 	idxbc.mu.RUnlock()
 
-	if idxb != nil {
-		return idxb
+	if idxbe != nil {
+		currentTime := atomic.LoadUint64(&currentTimestamp)
+		if atomic.LoadUint64(&idxbe.lastAccessTime) != currentTime {
+			atomic.StoreUint64(&idxbe.lastAccessTime, currentTime)
+		}
+		return idxbe.idxb
 	}
 	atomic.AddUint64(&idxbc.misses, 1)
-	idxbc.mu.Lock()
-	idxbc.missesMap[k]++
-	idxbc.mu.Unlock()
 	return nil
 }
 
@@ -229,13 +268,6 @@ func (idxbc *indexBlockCache) Get(k uint64) *indexBlock {
 func (idxbc *indexBlockCache) Put(k uint64, idxb *indexBlock) bool {
 	idxbc.mu.Lock()
 
-	if idxbc.missesMap[k] < 2 {
-		// Do not pollute cache with infrequently accessed items, since they may
-		// evict frequently accessed items.
-		idxbc.mu.Unlock()
-		return false
-	}
-
 	// Remove superflouos entries.
 	if overflow := len(idxbc.m) - getMaxCachedIndexBlocksPerPart(); overflow > 0 {
 		// Remove 10% of items from the cache.
@@ -250,21 +282,13 @@ func (idxbc *indexBlockCache) Put(k uint64, idxb *indexBlock) bool {
 			}
 		}
 	}
-	if overflow := len(idxbc.missesMap) - 8*getMaxCachedIndexBlocksPerPart(); overflow > 0 {
-		// Remove 10% of items from the cache.
-		overflow = int(float64(len(idxbc.missesMap)) * 0.1)
-		for k := range idxbc.missesMap {
-			delete(idxbc.missesMap, k)
-			overflow--
-			if overflow == 0 {
-				break
-			}
-		}
-	}
 
-	// Store the frequently accessed idxb in the cache.
-	delete(idxbc.missesMap, k)
-	idxbc.m[k] = idxb
+	// Store idxb in the cache.
+	idxbe := &indexBlockCacheEntry{
+		lastAccessTime: atomic.LoadUint64(&currentTimestamp),
+		idxb:           idxb,
+	}
+	idxbc.m[k] = idxbe
 	idxbc.mu.Unlock()
 	return true
 }
@@ -291,9 +315,11 @@ type inmemoryBlockCache struct {
 	requests uint64
 	misses   uint64
 
-	m         map[inmemoryBlockCacheKey]*inmemoryBlock
-	missesMap map[inmemoryBlockCacheKey]uint64
-	mu        sync.RWMutex
+	m  map[inmemoryBlockCacheKey]*inmemoryBlockCacheEntry
+	mu sync.RWMutex
+
+	cleanerStopCh chan struct{}
+	cleanerWG     sync.WaitGroup
 }
 
 type inmemoryBlockCacheKey struct {
@@ -309,22 +335,66 @@ func (ibck *inmemoryBlockCacheKey) Init(bh *blockHeader) {
 	ibck.itemsBlockOffset = bh.itemsBlockOffset
 }
 
-func (ibc *inmemoryBlockCache) Init() {
-	ibc.m = make(map[inmemoryBlockCacheKey]*inmemoryBlock)
-	ibc.missesMap = make(map[inmemoryBlockCacheKey]uint64)
-	ibc.requests = 0
-	ibc.misses = 0
+type inmemoryBlockCacheEntry struct {
+	// Atomically updated counters must go first in the struct, so they are properly
+	// aligned to 8 bytes on 32-bit architectures.
+	// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/212
+	lastAccessTime uint64
+
+	ib *inmemoryBlock
 }
 
-func (ibc *inmemoryBlockCache) Reset() {
+func newInmemoryBlockCache() *inmemoryBlockCache {
+	var ibc inmemoryBlockCache
+	ibc.m = make(map[inmemoryBlockCacheKey]*inmemoryBlockCacheEntry)
+
+	ibc.cleanerStopCh = make(chan struct{})
+	ibc.cleanerWG.Add(1)
+	go func() {
+		defer ibc.cleanerWG.Done()
+		ibc.cleaner()
+	}()
+	return &ibc
+}
+
+func (ibc *inmemoryBlockCache) MustClose() {
+	close(ibc.cleanerStopCh)
+	ibc.cleanerWG.Wait()
+
 	atomic.AddUint64(&inmemoryBlockCacheRequests, ibc.requests)
 	atomic.AddUint64(&inmemoryBlockCacheMisses, ibc.misses)
 	// It is safe returning ibc.m entries to pool, since the Reset function may be called
 	// only if no other goroutines access ibc entries.
-	for _, ib := range ibc.m {
-		putInmemoryBlock(ib)
+	for _, ibe := range ibc.m {
+		putInmemoryBlock(ibe.ib)
 	}
-	ibc.Init()
+	ibc.m = nil
+}
+
+// cleaner periodically cleans least recently used items.
+func (ibc *inmemoryBlockCache) cleaner() {
+	t := time.NewTimer(5 * time.Second)
+	for {
+		select {
+		case <-t.C:
+			ibc.cleanByTimeout()
+		case <-ibc.cleanerStopCh:
+			t.Stop()
+			return
+		}
+	}
+}
+
+func (ibc *inmemoryBlockCache) cleanByTimeout() {
+	currentTime := atomic.LoadUint64(&currentTimestamp)
+	ibc.mu.Lock()
+	for k, ibe := range ibc.m {
+		// Delete items accessed more than 10 minutes ago.
+		if currentTime-atomic.LoadUint64(&ibe.lastAccessTime) > 10*60 {
+			delete(ibc.m, k)
+		}
+	}
+	ibc.mu.Unlock()
 }
 
 var (
@@ -336,16 +406,17 @@ func (ibc *inmemoryBlockCache) Get(k inmemoryBlockCacheKey) *inmemoryBlock {
 	atomic.AddUint64(&ibc.requests, 1)
 
 	ibc.mu.RLock()
-	ib := ibc.m[k]
+	ibe := ibc.m[k]
 	ibc.mu.RUnlock()
 
-	if ib != nil {
-		return ib
+	if ibe != nil {
+		currentTime := atomic.LoadUint64(&currentTimestamp)
+		if atomic.LoadUint64(&ibe.lastAccessTime) != currentTime {
+			atomic.StoreUint64(&ibe.lastAccessTime, currentTime)
+		}
+		return ibe.ib
 	}
 	atomic.AddUint64(&ibc.misses, 1)
-	ibc.mu.Lock()
-	ibc.missesMap[k]++
-	ibc.mu.Unlock()
 	return nil
 }
 
@@ -355,14 +426,7 @@ func (ibc *inmemoryBlockCache) Get(k inmemoryBlockCacheKey) *inmemoryBlock {
 func (ibc *inmemoryBlockCache) Put(k inmemoryBlockCacheKey, ib *inmemoryBlock) bool {
 	ibc.mu.Lock()
 
-	if ibc.missesMap[k] < 2 {
-		// Do not cache entry with low number of accesses, since it may evict
-		// frequently accessed entries from the cache.
-		ibc.mu.Unlock()
-		return false
-	}
-
-	// Clean superflouos entries in ibc.m and ibc.missesMap.
+	// Clean superflouos entries in cache.
 	if overflow := len(ibc.m) - getMaxCachedInmemoryBlocksPerPart(); overflow > 0 {
 		// Remove 10% of items from the cache.
 		overflow = int(float64(len(ibc.m)) * 0.1)
@@ -376,21 +440,13 @@ func (ibc *inmemoryBlockCache) Put(k inmemoryBlockCacheKey, ib *inmemoryBlock) b
 			}
 		}
 	}
-	if overflow := len(ibc.missesMap) - 8*getMaxCachedInmemoryBlocksPerPart(); overflow > 0 {
-		// Remove 10% of items from the cache.
-		overflow = int(float64(len(ibc.missesMap)) * 0.1)
-		for k := range ibc.missesMap {
-			delete(ibc.missesMap, k)
-			overflow--
-			if overflow == 0 {
-				break
-			}
-		}
-	}
 
-	// The entry is frequently accessed, so store it in the cache.
-	delete(ibc.missesMap, k)
-	ibc.m[k] = ib
+	// Store ib in the cache.
+	ibe := &inmemoryBlockCacheEntry{
+		lastAccessTime: atomic.LoadUint64(&currentTimestamp),
+		ib:             ib,
+	}
+	ibc.m[k] = ibe
 	ibc.mu.Unlock()
 	return true
 }
@@ -409,3 +465,15 @@ func (ibc *inmemoryBlockCache) Requests() uint64 {
 func (ibc *inmemoryBlockCache) Misses() uint64 {
 	return atomic.LoadUint64(&ibc.misses)
 }
+
+func init() {
+	go func() {
+		t := time.NewTimer(time.Second)
+		for tm := range t.C {
+			t := uint64(tm.Unix())
+			atomic.StoreUint64(&currentTimestamp, t)
+		}
+	}()
+}
+
+var currentTimestamp uint64

lib/mergeset/part_search.go

@@ -82,8 +82,8 @@ func (ps *partSearch) Init(p *part, shouldCacheBlock func(item []byte) bool) {
 	ps.reset()
 
 	ps.p = p
-	ps.idxbCache = &p.idxbCache
-	ps.ibCache = &p.ibCache
+	ps.idxbCache = p.idxbCache
+	ps.ibCache = p.ibCache
 }
 
 // Seek seeks for the first item greater or equal to k in ps.
@@ -311,7 +311,7 @@ func (ps *partSearch) getIndexBlock(mr *metaindexRow) (*indexBlock, bool, error)
 
 func (ps *partSearch) readIndexBlock(mr *metaindexRow) (*indexBlock, error) {
 	ps.compressedIndexBuf = bytesutil.Resize(ps.compressedIndexBuf, int(mr.indexBlockSize))
-	ps.p.indexFile.ReadAt(ps.compressedIndexBuf, int64(mr.indexBlockOffset))
+	ps.p.indexFile.MustReadAt(ps.compressedIndexBuf, int64(mr.indexBlockOffset))
 
 	var err error
 	ps.indexBuf, err = encoding.DecompressZSTD(ps.indexBuf[:0], ps.compressedIndexBuf)
@@ -355,10 +355,10 @@ func (ps *partSearch) readInmemoryBlock(bh *blockHeader) (*inmemoryBlock, error)
 	ps.sb.Reset()
 
 	ps.sb.itemsData = bytesutil.Resize(ps.sb.itemsData, int(bh.itemsBlockSize))
-	ps.p.itemsFile.ReadAt(ps.sb.itemsData, int64(bh.itemsBlockOffset))
+	ps.p.itemsFile.MustReadAt(ps.sb.itemsData, int64(bh.itemsBlockOffset))
 
 	ps.sb.lensData = bytesutil.Resize(ps.sb.lensData, int(bh.lensBlockSize))
-	ps.p.lensFile.ReadAt(ps.sb.lensData, int64(bh.lensBlockOffset))
+	ps.p.lensFile.MustReadAt(ps.sb.lensData, int64(bh.lensBlockOffset))
 
 	ib := getInmemoryBlock()
 	if err := ib.UnmarshalData(&ps.sb, bh.firstItem, bh.commonPrefix, bh.itemsCount, bh.marshalType); err != nil {

lib/mergeset/table.go

@@ -187,8 +187,8 @@ func OpenTable(path string, flushCallback func(), prepareBlock PrepareBlockCallb
 
 	var m TableMetrics
 	tb.UpdateMetrics(&m)
-	logger.Infof("table %q has been opened in %s; partsCount: %d; blocksCount: %d, itemsCount: %d; sizeBytes: %d",
-		path, time.Since(startTime), m.PartsCount, m.BlocksCount, m.ItemsCount, m.SizeBytes)
+	logger.Infof("table %q has been opened in %.3f seconds; partsCount: %d; blocksCount: %d, itemsCount: %d; sizeBytes: %d",
+		path, time.Since(startTime).Seconds(), m.PartsCount, m.BlocksCount, m.ItemsCount, m.SizeBytes)
 
 	tb.convertersWG.Add(1)
 	go func() {
@@ -206,17 +206,17 @@ func (tb *Table) MustClose() {
 	logger.Infof("waiting for raw items flusher to stop on %q...", tb.path)
 	startTime := time.Now()
 	tb.rawItemsFlusherWG.Wait()
-	logger.Infof("raw items flusher stopped in %s on %q", time.Since(startTime), tb.path)
+	logger.Infof("raw items flusher stopped in %.3f seconds on %q", time.Since(startTime).Seconds(), tb.path)
 
 	logger.Infof("waiting for converters to stop on %q...", tb.path)
 	startTime = time.Now()
 	tb.convertersWG.Wait()
-	logger.Infof("converters stopped in %s on %q", time.Since(startTime), tb.path)
+	logger.Infof("converters stopped in %.3f seconds on %q", time.Since(startTime).Seconds(), tb.path)
 
 	logger.Infof("waiting for part mergers to stop on %q...", tb.path)
 	startTime = time.Now()
 	tb.partMergersWG.Wait()
-	logger.Infof("part mergers stopped in %s on %q", time.Since(startTime), tb.path)
+	logger.Infof("part mergers stopped in %.3f seconds on %q", time.Since(startTime).Seconds(), tb.path)
 
 	logger.Infof("flushing inmemory parts to files on %q...", tb.path)
 	startTime = time.Now()
@@ -242,7 +242,7 @@ func (tb *Table) MustClose() {
 	if err := tb.mergePartsOptimal(pws, nil); err != nil {
 		logger.Panicf("FATAL: cannot flush inmemory parts to files in %q: %s", tb.path, err)
 	}
-	logger.Infof("%d inmemory parts have been flushed to files in %s on %q", len(pws), time.Since(startTime), tb.path)
+	logger.Infof("%d inmemory parts have been flushed to files in %.3f seconds on %q", len(pws), time.Since(startTime).Seconds(), tb.path)
 
 	// Remove references to parts from the tb, so they may be eventually closed
 	// after all the searches are done.
@@ -447,7 +447,7 @@ func (tb *Table) convertToV1280() {
 			logger.Errorf("failed round 1 of background conversion of %q to v1.28.0 format: %s", tb.path, err)
 			return
 		}
-		logger.Infof("finished round 1 of background conversion of %q to v1.28.0 format in %s", tb.path, time.Since(startTime))
+		logger.Infof("finished round 1 of background conversion of %q to v1.28.0 format in %.3f seconds", tb.path, time.Since(startTime).Seconds())
 
 		// The second round is needed in order to merge small blocks
 		// with tag->metricIDs rows left after the first round.
@@ -460,7 +460,7 @@ func (tb *Table) convertToV1280() {
 				return
 			}
 		}
-		logger.Infof("finished round 2 of background conversion of %q to v1.28.0 format in %s", tb.path, time.Since(startTime))
+		logger.Infof("finished round 2 of background conversion of %q to v1.28.0 format in %.3f seconds", tb.path, time.Since(startTime).Seconds())
 	}
 
 	if err := fs.WriteFileAtomically(flagFilePath, []byte("ok")); err != nil {
@@ -853,7 +853,8 @@ func (tb *Table) mergeParts(pws []*partWrapper, stopCh <-chan struct{}, isOuterP
 
 	d := time.Since(startTime)
 	if d > 10*time.Second {
-		logger.Infof("merged %d items in %s at %d items/sec to %q; sizeBytes: %d", outItemsCount, d, int(float64(outItemsCount)/d.Seconds()), dstPartPath, newPSize)
+		logger.Infof("merged %d items in %.3f seconds at %d items/sec to %q; sizeBytes: %d",
+			outItemsCount, d.Seconds(), int(float64(outItemsCount)/d.Seconds()), dstPartPath, newPSize)
 	}
 
 	return nil
@@ -1057,7 +1058,7 @@ func (tb *Table) CreateSnapshotAt(dstDir string) error {
 	parentDir := filepath.Dir(dstDir)
 	fs.MustSyncPath(parentDir)
 
-	logger.Infof("created Table snapshot of %q at %q in %s", srcDir, dstDir, time.Since(startTime))
+	logger.Infof("created Table snapshot of %q at %q in %.3f seconds", srcDir, dstDir, time.Since(startTime).Seconds())
 	return nil
 }
 

lib/metricsql/rollup.go

@@ -29,31 +29,34 @@ var rollupFuncs = map[string]bool{
 	"absent_over_time":   true,
 
 	// Additional rollup funcs.
-	"default_rollup":      true,
-	"sum2_over_time":      true,
-	"geomean_over_time":   true,
-	"first_over_time":     true,
-	"last_over_time":      true,
-	"distinct_over_time":  true,
-	"increases_over_time": true,
-	"decreases_over_time": true,
-	"integrate":           true,
-	"ideriv":              true,
-	"lifetime":            true,
-	"lag":                 true,
-	"scrape_interval":     true,
-	"tmin_over_time":      true,
-	"tmax_over_time":      true,
-	"share_le_over_time":  true,
-	"share_gt_over_time":  true,
-	"histogram_over_time": true,
-	"rollup":              true,
-	"rollup_rate":         true,
-	"rollup_deriv":        true,
-	"rollup_delta":        true,
-	"rollup_increase":     true,
-	"rollup_candlestick":  true,
-	"aggr_over_time":      true,
+	"default_rollup":        true,
+	"range_over_time":       true,
+	"sum2_over_time":        true,
+	"geomean_over_time":     true,
+	"first_over_time":       true,
+	"last_over_time":        true,
+	"distinct_over_time":    true,
+	"increases_over_time":   true,
+	"decreases_over_time":   true,
+	"integrate":             true,
+	"ideriv":                true,
+	"lifetime":              true,
+	"lag":                   true,
+	"scrape_interval":       true,
+	"tmin_over_time":        true,
+	"tmax_over_time":        true,
+	"share_le_over_time":    true,
+	"share_gt_over_time":    true,
+	"histogram_over_time":   true,
+	"rollup":                true,
+	"rollup_rate":           true,
+	"rollup_deriv":          true,
+	"rollup_delta":          true,
+	"rollup_increase":       true,
+	"rollup_candlestick":    true,
+	"aggr_over_time":        true,
+	"hoeffding_bound_upper": true,
+	"hoeffding_bound_lower": true,
 }
 
 // IsRollupFunc returns whether funcName is known rollup function.

lib/metricsql/transform.go

@@ -44,9 +44,12 @@ var transformFuncs = map[string]bool{
 	"label_move":         true,
 	"label_transform":    true,
 	"label_value":        true,
+	"label_match":        true,
+	"label_mismatch":     true,
 	"union":              true,
 	"":                   true, // empty func is a synonim to union
 	"keep_last_value":    true,
+	"keep_next_value":    true,
 	"start":              true,
 	"end":                true,
 	"step":               true,

lib/prompb/util.go

@@ -1,47 +1,5 @@
 package prompb
 
-import (
-	"fmt"
-	"io"
-
-	"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
-	"github.com/golang/snappy"
-)
-
-// ReadSnappy reads r, unpacks it using snappy, appends it to dst
-// and returns the result.
-func ReadSnappy(dst []byte, r io.Reader, maxSize int64) ([]byte, error) {
-	lr := io.LimitReader(r, maxSize+1)
-	bb := bodyBufferPool.Get()
-	reqLen, err := bb.ReadFrom(lr)
-	if err != nil {
-		bodyBufferPool.Put(bb)
-		return dst, fmt.Errorf("cannot read compressed request: %s", err)
-	}
-	if reqLen > maxSize {
-		return dst, fmt.Errorf("too big packed request; mustn't exceed %d bytes", maxSize)
-	}
-
-	buf := dst[len(dst):cap(dst)]
-	buf, err = snappy.Decode(buf, bb.B)
-	bodyBufferPool.Put(bb)
-	if err != nil {
-		err = fmt.Errorf("cannot decompress request with length %d: %s", reqLen, err)
-		return dst, err
-	}
-	if int64(len(buf)) > maxSize {
-		return dst, fmt.Errorf("too big unpacked request; musn't exceed %d bytes", maxSize)
-	}
-	if len(buf) > 0 && len(dst) < cap(dst) && &buf[0] == &dst[len(dst):cap(dst)][0] {
-		dst = dst[:len(dst)+len(buf)]
-	} else {
-		dst = append(dst, buf...)
-	}
-	return dst, nil
-}
-
-var bodyBufferPool bytesutil.ByteBufferPool
-
 // Reset resets wr.
 func (wr *WriteRequest) Reset() {
 	for i := range wr.Timeseries {

lib/protoparser/opentsdbhttp/parser_pool.go

@@ -0,0 +1,21 @@
+package opentsdbhttp
+
+import (
+	"github.com/valyala/fastjson"
+)
+
+// GetParser returns JSON parser.
+//
+// The parser must be returned to the pool via PutParser when no longer needed.
+func GetParser() *fastjson.Parser {
+	return parserPool.Get()
+}
+
+// PutParser returns p to the pool.
+//
+// p cannot be used after returning to the pool.
+func PutParser(p *fastjson.Parser) {
+	parserPool.Put(p)
+}
+
+var parserPool fastjson.ParserPool

lib/protoparser/opentsdbhttp/parser_test.go

@@ -9,8 +9,8 @@ func TestRowsUnmarshalFailure(t *testing.T) {
 	f := func(s string) {
 		t.Helper()
 		var rows Rows
-		p := parserPool.Get()
-		defer parserPool.Put(p)
+		p := GetParser()
+		defer PutParser(p)
 		v, err := p.Parse(s)
 		if err != nil {
 			// Expected JSON parser error
@@ -84,8 +84,8 @@ func TestRowsUnmarshalSuccess(t *testing.T) {
 		t.Helper()
 		var rows Rows
 
-		p := parserPool.Get()
-		defer parserPool.Put(p)
+		p := GetParser()
+		defer PutParser(p)
 		v, err := p.Parse(s)
 		if err != nil {
 			t.Fatalf("cannot parse json %s: %s", s, err)

lib/protoparser/prometheus/parser.go

@@ -0,0 +1,292 @@
+package prometheus
+
+import (
+	"fmt"
+	"strconv"
+	"strings"
+
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
+	"github.com/VictoriaMetrics/metrics"
+	"github.com/valyala/fastjson/fastfloat"
+)
+
+// Rows contains parsed Prometheus rows.
+type Rows struct {
+	Rows []Row
+
+	tagsPool []Tag
+}
+
+// Reset resets rs.
+func (rs *Rows) Reset() {
+	// Reset items, so they can be GC'ed
+
+	for i := range rs.Rows {
+		rs.Rows[i].reset()
+	}
+	rs.Rows = rs.Rows[:0]
+
+	for i := range rs.tagsPool {
+		rs.tagsPool[i].reset()
+	}
+	rs.tagsPool = rs.tagsPool[:0]
+}
+
+// Unmarshal unmarshals Prometheus exposition text rows from s.
+//
+// See https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md#text-format-details
+//
+// s must be unchanged until rs is in use.
+func (rs *Rows) Unmarshal(s string) {
+	noEscapes := strings.IndexByte(s, '\\') < 0
+	rs.Rows, rs.tagsPool = unmarshalRows(rs.Rows[:0], s, rs.tagsPool[:0], noEscapes)
+}
+
+// Row is a single Prometheus row.
+type Row struct {
+	Metric    string
+	Tags      []Tag
+	Value     float64
+	Timestamp int64
+}
+
+func (r *Row) reset() {
+	r.Metric = ""
+	r.Tags = nil
+	r.Value = 0
+	r.Timestamp = 0
+}
+
+func skipLeadingWhitespace(s string) string {
+	// Prometheus treats ' ' and '\t' as whitespace
+	// according to https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md#text-format-details
+	for len(s) > 0 && (s[0] == ' ' || s[0] == '\t') {
+		s = s[1:]
+	}
+	return s
+}
+
+func skipTrailingWhitespace(s string) string {
+	// Prometheus treats ' ' and '\t' as whitespace
+	// according to https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md#text-format-details
+	for len(s) > 0 && (s[len(s)-1] == ' ' || s[len(s)-1] == '\t') {
+		s = s[:len(s)-1]
+	}
+	return s
+}
+
+func nextWhitespace(s string) int {
+	n := strings.IndexByte(s, ' ')
+	if n < 0 {
+		return strings.IndexByte(s, '\t')
+	}
+	n1 := strings.IndexByte(s, '\t')
+	if n1 < 0 || n1 > n {
+		return n
+	}
+	return n1
+}
+
+func (r *Row) unmarshal(s string, tagsPool []Tag, noEscapes bool) ([]Tag, error) {
+	r.reset()
+	s = skipLeadingWhitespace(s)
+	n := strings.IndexByte(s, '{')
+	if n >= 0 {
+		// Tags found. Parse them.
+		r.Metric = skipTrailingWhitespace(s[:n])
+		s = s[n+1:]
+		tagsStart := len(tagsPool)
+		var err error
+		s, tagsPool, err = unmarshalTags(tagsPool, s, noEscapes)
+		if err != nil {
+			return tagsPool, fmt.Errorf("cannot unmarshal tags: %s", err)
+		}
+		if len(s) > 0 && s[0] == ' ' {
+			// Fast path - skip whitespace.
+			s = s[1:]
+		}
+		tags := tagsPool[tagsStart:]
+		r.Tags = tags[:len(tags):len(tags)]
+	} else {
+		// Tags weren't found. Search for value after whitespace
+		n = nextWhitespace(s)
+		if n < 0 {
+			return tagsPool, fmt.Errorf("missing value")
+		}
+		r.Metric = s[:n]
+		s = s[n+1:]
+	}
+	if len(r.Metric) == 0 {
+		return tagsPool, fmt.Errorf("metric cannot be empty")
+	}
+	s = skipLeadingWhitespace(s)
+	if len(s) == 0 {
+		return tagsPool, fmt.Errorf("value cannot be empty")
+	}
+	n = nextWhitespace(s)
+	if n < 0 {
+		// There is no timestamp.
+		r.Value = fastfloat.ParseBestEffort(s)
+		return tagsPool, nil
+	}
+	// There is timestamp.
+	r.Value = fastfloat.ParseBestEffort(s[:n])
+	s = skipLeadingWhitespace(s[n+1:])
+	r.Timestamp = fastfloat.ParseInt64BestEffort(s)
+	return tagsPool, nil
+}
+
+func unmarshalRows(dst []Row, s string, tagsPool []Tag, noEscapes bool) ([]Row, []Tag) {
+	for len(s) > 0 {
+		n := strings.IndexByte(s, '\n')
+		if n < 0 {
+			// The last line.
+			return unmarshalRow(dst, s, tagsPool, noEscapes)
+		}
+		dst, tagsPool = unmarshalRow(dst, s[:n], tagsPool, noEscapes)
+		s = s[n+1:]
+	}
+	return dst, tagsPool
+}
+
+func unmarshalRow(dst []Row, s string, tagsPool []Tag, noEscapes bool) ([]Row, []Tag) {
+	if len(s) > 0 && s[len(s)-1] == '\r' {
+		s = s[:len(s)-1]
+	}
+	s = skipLeadingWhitespace(s)
+	if len(s) == 0 {
+		// Skip empty line
+		return dst, tagsPool
+	}
+	if s[0] == '#' {
+		// Skip comment
+		return dst, tagsPool
+	}
+	if cap(dst) > len(dst) {
+		dst = dst[:len(dst)+1]
+	} else {
+		dst = append(dst, Row{})
+	}
+	r := &dst[len(dst)-1]
+	var err error
+	tagsPool, err = r.unmarshal(s, tagsPool, noEscapes)
+	if err != nil {
+		dst = dst[:len(dst)-1]
+		logger.Errorf("cannot unmarshal Prometheus line %q: %s", s, err)
+		invalidLines.Inc()
+	}
+	return dst, tagsPool
+}
+
+var invalidLines = metrics.NewCounter(`vm_rows_invalid_total{type="prometheus"}`)
+
+func unmarshalTags(dst []Tag, s string, noEscapes bool) (string, []Tag, error) {
+	s = skipLeadingWhitespace(s)
+	if len(s) > 0 && s[0] == '}' {
+		// End of tags found.
+		return s[1:], dst, nil
+	}
+	for {
+		n := strings.IndexByte(s, '=')
+		if n < 0 {
+			return s, dst, fmt.Errorf("missing value for tag %q", s)
+		}
+		key := skipTrailingWhitespace(s[:n])
+		s = skipLeadingWhitespace(s[n+1:])
+		if len(s) == 0 || s[0] != '"' {
+			return s, dst, fmt.Errorf("expecting quoted value for tag %q; got %q", key, s)
+		}
+		value := s[1:]
+		if noEscapes {
+			// Fast path - the line has no escape chars
+			n = strings.IndexByte(value, '"')
+			if n < 0 {
+				return s, dst, fmt.Errorf("missing closing quote for tag value %q", s)
+			}
+			s = value[n+1:]
+			value = value[:n]
+		} else {
+			// Slow path - the line contains escape chars
+			n = findClosingQuote(s)
+			if n < 0 {
+				return s, dst, fmt.Errorf("missing closing quote for tag value %q", s)
+			}
+			var err error
+			value, err = unescapeValue(s[:n+1])
+			if err != nil {
+				return s, dst, fmt.Errorf("cannot unescape value %q for tag %q: %s", s[:n+1], key, err)
+			}
+			s = s[n+1:]
+		}
+		if len(key) > 0 && len(value) > 0 {
+			if cap(dst) > len(dst) {
+				dst = dst[:len(dst)+1]
+			} else {
+				dst = append(dst, Tag{})
+			}
+			tag := &dst[len(dst)-1]
+			tag.Key = key
+			tag.Value = value
+		}
+		s = skipLeadingWhitespace(s)
+		if len(s) > 0 && s[0] == '}' {
+			// End of tags found.
+			return s[1:], dst, nil
+		}
+		if len(s) == 0 || s[0] != ',' {
+			return s, dst, fmt.Errorf("missing comma after tag %s=%q", key, value)
+		}
+		s = s[1:]
+	}
+}
+
+// Tag is a Prometheus tag.
+type Tag struct {
+	Key   string
+	Value string
+}
+
+func (t *Tag) reset() {
+	t.Key = ""
+	t.Value = ""
+}
+
+func findClosingQuote(s string) int {
+	if len(s) == 0 || s[0] != '"' {
+		return -1
+	}
+	off := 1
+	s = s[1:]
+	for {
+		n := strings.IndexByte(s, '"')
+		if n < 0 {
+			return -1
+		}
+		if prevBackslashesCount(s[:n])%2 == 0 {
+			return off + n
+		}
+		off += n + 1
+		s = s[n+1:]
+	}
+}
+
+func unescapeValue(s string) (string, error) {
+	if len(s) < 2 || s[0] != '"' || s[len(s)-1] != '"' {
+		return "", fmt.Errorf("unexpected tag value: %q", s)
+	}
+	n := strings.IndexByte(s, '\\')
+	if n < 0 {
+		// Fast path - nothing to unescape
+		return s[1 : len(s)-1], nil
+	}
+	return strconv.Unquote(s)
+}
+
+func prevBackslashesCount(s string) int {
+	n := 0
+	for len(s) > 0 && s[len(s)-1] == '\\' {
+		n++
+		s = s[:len(s)-1]
+	}
+	return n
+}

lib/protoparser/prometheus/parser_test.go

@@ -0,0 +1,270 @@
+package prometheus
+
+import (
+	"reflect"
+	"testing"
+)
+
+func TestPrevBackslashesCount(t *testing.T) {
+	f := func(s string, nExpected int) {
+		t.Helper()
+		n := prevBackslashesCount(s)
+		if n != nExpected {
+			t.Fatalf("unexpected value returned from prevBackslashesCount(%q); got %d; want %d", s, n, nExpected)
+		}
+	}
+	f(``, 0)
+	f(`foo`, 0)
+	f(`\`, 1)
+	f(`\\`, 2)
+	f(`\\\`, 3)
+	f(`\\\a`, 0)
+	f(`foo\bar`, 0)
+	f(`foo\\`, 2)
+	f(`\\foo\`, 1)
+	f(`\\foo\\\\`, 4)
+}
+
+func TestFindClosingQuote(t *testing.T) {
+	f := func(s string, nExpected int) {
+		t.Helper()
+		n := findClosingQuote(s)
+		if n != nExpected {
+			t.Fatalf("unexpected value returned from findClosingQuote(%q); got %d; want %d", s, n, nExpected)
+		}
+	}
+	f(``, -1)
+	f(`x`, -1)
+	f(`"`, -1)
+	f(`""`, 1)
+	f(`foobar"`, -1)
+	f(`"foo"`, 4)
+	f(`"\""`, 3)
+	f(`"\\"`, 3)
+	f(`"\"`, -1)
+	f(`"foo\"bar\"baz"`, 14)
+}
+
+func TestUnescapeValueFailure(t *testing.T) {
+	f := func(s string) {
+		t.Helper()
+		ss, err := unescapeValue(s)
+		if err == nil {
+			t.Fatalf("expecting error")
+		}
+		if ss != "" {
+			t.Fatalf("expecting empty string; got %q", ss)
+		}
+	}
+	f(``)
+	f(`foobar`)
+	f(`"foobar`)
+	f(`foobar"`)
+	f(`"foobar\"`)
+	f(` "foobar"`)
+	f(`"foobar" `)
+}
+
+func TestUnescapeValueSuccess(t *testing.T) {
+	f := func(s, resultExpected string) {
+		t.Helper()
+		result, err := unescapeValue(s)
+		if err != nil {
+			t.Fatalf("unexpected error: %s", err)
+		}
+		if result != resultExpected {
+			t.Fatalf("unexpected result; got %q; want %q", result, resultExpected)
+		}
+	}
+	f(`""`, "")
+	f(`"f"`, "f")
+	f(`"foobar"`, "foobar")
+	f(`"\"\n\t"`, "\"\n\t")
+}
+
+func TestRowsUnmarshalFailure(t *testing.T) {
+	f := func(s string) {
+		t.Helper()
+		var rows Rows
+		rows.Unmarshal(s)
+		if len(rows.Rows) != 0 {
+			t.Fatalf("unexpected number of rows parsed; got %d; want 0;\nrows:%#v", len(rows.Rows), rows.Rows)
+		}
+
+		// Try again
+		rows.Unmarshal(s)
+		if len(rows.Rows) != 0 {
+			t.Fatalf("unexpected number of rows parsed; got %d; want 0;\nrows:%#v", len(rows.Rows), rows.Rows)
+		}
+	}
+
+	// Empty lines and comments
+	f("")
+	f(" ")
+	f("\t")
+	f("\t  \r")
+	f("\t\t  \n\n  # foobar")
+	f("#foobar")
+	f("#foobar\n")
+
+	// invalid tags
+	f("a{")
+	f("a { ")
+	f("a {foo")
+	f("a {foo}")
+	f("a {foo  =")
+	f(`a {foo  ="bar`)
+	f(`a {foo  ="b\ar`)
+	f(`a {foo  = "bar"`)
+	f(`a {foo  ="bar",`)
+	f(`a {foo  ="bar" , `)
+	f(`a {foo  ="bar" , }`)
+	f(`a {foo  ="bar" , baz }`)
+
+	// empty metric name
+	f(`{foo="bar"}`)
+
+	// Missing value
+	f("aaa")
+	f(" aaa")
+	f(" aaa ")
+	f(" aaa   \n")
+	f(` aa{foo="bar"}   ` + "\n")
+}
+
+func TestRowsUnmarshalSuccess(t *testing.T) {
+	f := func(s string, rowsExpected *Rows) {
+		t.Helper()
+		var rows Rows
+		rows.Unmarshal(s)
+		if !reflect.DeepEqual(rows.Rows, rowsExpected.Rows) {
+			t.Fatalf("unexpected rows;\ngot\n%+v;\nwant\n%+v", rows.Rows, rowsExpected.Rows)
+		}
+
+		// Try unmarshaling again
+		rows.Unmarshal(s)
+		if !reflect.DeepEqual(rows.Rows, rowsExpected.Rows) {
+			t.Fatalf("unexpected rows;\ngot\n%+v;\nwant\n%+v", rows.Rows, rowsExpected.Rows)
+		}
+
+		rows.Reset()
+		if len(rows.Rows) != 0 {
+			t.Fatalf("non-empty rows after reset: %+v", rows.Rows)
+		}
+	}
+
+	// Empty line or comment
+	f("", &Rows{})
+	f("\r", &Rows{})
+	f("\n\n", &Rows{})
+	f("\n\r\n", &Rows{})
+	f("\t  \t\n\r\n#foobar\n  # baz", &Rows{})
+
+	// Single line
+	f("foobar 78.9", &Rows{
+		Rows: []Row{{
+			Metric: "foobar",
+			Value:  78.9,
+		}},
+	})
+	f("foobar 123.456 789\n", &Rows{
+		Rows: []Row{{
+			Metric:    "foobar",
+			Value:     123.456,
+			Timestamp: 789,
+		}},
+	})
+	f("foobar{} 123.456 789\n", &Rows{
+		Rows: []Row{{
+			Metric:    "foobar",
+			Value:     123.456,
+			Timestamp: 789,
+		}},
+	})
+
+	// Timestamp bigger than 1<<31
+	f("aaa 1123 429496729600", &Rows{
+		Rows: []Row{{
+			Metric:    "aaa",
+			Value:     1123,
+			Timestamp: 429496729600,
+		}},
+	})
+
+	// Tags
+	f(`foo{bar="baz"} 1 2`, &Rows{
+		Rows: []Row{{
+			Metric: "foo",
+			Tags: []Tag{{
+				Key:   "bar",
+				Value: "baz",
+			}},
+			Value:     1,
+			Timestamp: 2,
+		}},
+	})
+	f(`foo{bar="b\"a\\z"} -1.2`, &Rows{
+		Rows: []Row{{
+			Metric: "foo",
+			Tags: []Tag{{
+				Key:   "bar",
+				Value: "b\"a\\z",
+			}},
+			Value: -1.2,
+		}},
+	})
+	// Empty tags
+	f(`foo {bar="baz",aa="",x="y",="z"} 1 2`, &Rows{
+		Rows: []Row{{
+			Metric: "foo",
+			Tags: []Tag{
+				{
+					Key:   "bar",
+					Value: "baz",
+				},
+				{
+					Key:   "x",
+					Value: "y",
+				},
+			},
+			Value:     1,
+			Timestamp: 2,
+		}},
+	})
+
+	// Multi lines
+	f("# foo\n # bar ba zzz\nfoo 0.3 2\naaa 3\nbar.baz 0.34 43\n", &Rows{
+		Rows: []Row{
+			{
+				Metric:    "foo",
+				Value:     0.3,
+				Timestamp: 2,
+			},
+			{
+				Metric: "aaa",
+				Value:  3,
+			},
+			{
+				Metric:    "bar.baz",
+				Value:     0.34,
+				Timestamp: 43,
+			},
+		},
+	})
+
+	// Multi lines with invalid line
+	f("\t foo\t {} 0.3\t 2\naaa\n  bar.baz 0.34 43\n", &Rows{
+		Rows: []Row{
+			{
+				Metric:    "foo",
+				Value:     0.3,
+				Timestamp: 2,
+			},
+			{
+				Metric:    "bar.baz",
+				Value:     0.34,
+				Timestamp: 43,
+			},
+		},
+	})
+}

lib/protoparser/prometheus/parser_timing_test.go

@@ -0,0 +1,25 @@
+package prometheus
+
+import (
+	"fmt"
+	"testing"
+)
+
+func BenchmarkRowsUnmarshal(b *testing.B) {
+	s := `cpu_usage{mode="user"} 1.23
+cpu_usage{mode="system"} 23.344
+cpu_usage{mode="iowait"} 3.3443
+cpu_usage{mode="irq"} 0.34432
+`
+	b.SetBytes(int64(len(s)))
+	b.ReportAllocs()
+	b.RunParallel(func(pb *testing.PB) {
+		var rows Rows
+		for pb.Next() {
+			rows.Unmarshal(s)
+			if len(rows.Rows) != 4 {
+				panic(fmt.Errorf("unexpected number of rows unmarshaled: got %d; want 4", len(rows.Rows)))
+			}
+		}
+	})
+}

lib/storage/block.go

@@ -158,6 +158,25 @@ func (b *Block) tooBig() bool {
 	return false
 }
 
+func (b *Block) deduplicateSamplesDuringMerge() {
+	if len(b.values) == 0 {
+		// Nothing to dedup or the data is already marshaled.
+		return
+	}
+	srcTimestamps := b.timestamps[b.nextIdx:]
+	srcValues := b.values[b.nextIdx:]
+	timestamps, values := deduplicateSamplesDuringMerge(srcTimestamps, srcValues)
+	b.timestamps = b.timestamps[:b.nextIdx+len(timestamps)]
+	b.values = b.values[:b.nextIdx+len(values)]
+}
+
+func (b *Block) rowsCount() int {
+	if len(b.values) == 0 {
+		return int(b.bh.RowsCount)
+	}
+	return len(b.values[b.nextIdx:])
+}
+
 // MarshalData marshals the block into binary representation.
 func (b *Block) MarshalData(timestampsBlockOffset, valuesBlockOffset uint64) ([]byte, []byte, []byte) {
 	if len(b.values) == 0 {
@@ -244,7 +263,10 @@ func (b *Block) UnmarshalData() error {
 	if err != nil {
 		return err
 	}
-	encoding.EnsureNonDecreasingSequence(b.timestamps, b.bh.MinTimestamp, b.bh.MaxTimestamp)
+	if b.bh.PrecisionBits < 64 {
+		// Recover timestamps order after lossy compression.
+		encoding.EnsureNonDecreasingSequence(b.timestamps, b.bh.MinTimestamp, b.bh.MaxTimestamp)
+	}
 	b.timestampsData = b.timestampsData[:0]
 
 	b.values, err = encoding.UnmarshalValues(b.values[:0], b.valuesData, b.bh.ValuesMarshalType, b.bh.FirstValue, int(b.bh.RowsCount))

lib/storage/block_stream_writer.go

@@ -171,6 +171,8 @@ func (bsw *blockStreamWriter) MustClose() {
 
 // WriteExternalBlock writes b to bsw and updates ph and rowsMerged.
 func (bsw *blockStreamWriter) WriteExternalBlock(b *Block, ph *partHeader, rowsMerged *uint64) {
+	atomic.AddUint64(rowsMerged, uint64(b.rowsCount()))
+	b.deduplicateSamplesDuringMerge()
 	headerData, timestampsData, valuesData := b.MarshalData(bsw.timestampsBlockOffset, bsw.valuesBlockOffset)
 
 	bsw.indexData = append(bsw.indexData, headerData...)
@@ -186,7 +188,6 @@ func (bsw *blockStreamWriter) WriteExternalBlock(b *Block, ph *partHeader, rowsM
 	bsw.valuesBlockOffset += uint64(len(valuesData))
 
 	updatePartHeader(b, ph)
-	atomic.AddUint64(rowsMerged, uint64(b.bh.RowsCount))
 }
 
 func updatePartHeader(b *Block, ph *partHeader) {

lib/storage/dedup.go

@@ -0,0 +1,104 @@
+package storage
+
+import (
+	"flag"
+
+	"github.com/VictoriaMetrics/metrics"
+)
+
+var minScrapeInterval = flag.Duration("dedup.minScrapeInterval", 0, "Remove superflouos samples from time series if they are located closer to each other than this duration. "+
+	"This may be useful for reducing overhead when multiple identically configured Prometheus instances write data to the same VictoriaMetrics. "+
+	"Deduplication is disabled if the -dedup.minScrapeInterval is 0")
+
+func getMinDelta() int64 {
+	// Divide minScrapeInterval by 2 in order to preserve proper data points.
+	// For instance, if minScrapeInterval=10, the following time series:
+	//    10 15 19 25 30 34 41
+	// Would be unexpectedly converted to:
+	//    10 25 41
+	// When dividing minScrapeInterval by 2, it will be converted to the expected:
+	//    10 19 30 41
+	return minScrapeInterval.Milliseconds() / 2
+}
+
+// DeduplicateSamples removes samples from src* if they are closer to each other than minScrapeInterval.
+func DeduplicateSamples(srcTimestamps []int64, srcValues []float64) ([]int64, []float64) {
+	if *minScrapeInterval <= 0 {
+		return srcTimestamps, srcValues
+	}
+	minDelta := getMinDelta()
+	if !needsDedup(srcTimestamps, minDelta) {
+		// Fast path - nothing to deduplicate
+		return srcTimestamps, srcValues
+	}
+
+	// Slow path - dedup data points.
+	prevTimestamp := srcTimestamps[0]
+	dstTimestamps := srcTimestamps[:1]
+	dstValues := srcValues[:1]
+	dedups := 0
+	for i := 1; i < len(srcTimestamps); i++ {
+		ts := srcTimestamps[i]
+		if ts-prevTimestamp < minDelta {
+			dedups++
+			continue
+		}
+		dstTimestamps = append(dstTimestamps, ts)
+		dstValues = append(dstValues, srcValues[i])
+		prevTimestamp = ts
+	}
+	dedupsDuringSelect.Add(dedups)
+	return dstTimestamps, dstValues
+}
+
+var dedupsDuringSelect = metrics.NewCounter(`deduplicated_samples_total{type="select"}`)
+
+func deduplicateSamplesDuringMerge(srcTimestamps []int64, srcValues []int64) ([]int64, []int64) {
+	if *minScrapeInterval <= 0 {
+		return srcTimestamps, srcValues
+	}
+	if len(srcTimestamps) < 32 {
+		// Do not de-duplicate small number of samples during merge
+		// in order to improve deduplication accuracy on later stages.
+		return srcTimestamps, srcValues
+	}
+	minDelta := getMinDelta()
+	if !needsDedup(srcTimestamps, minDelta) {
+		// Fast path - nothing to deduplicate
+		return srcTimestamps, srcValues
+	}
+
+	// Slow path - dedup data points.
+	prevTimestamp := srcTimestamps[0]
+	dstTimestamps := srcTimestamps[:1]
+	dstValues := srcValues[:1]
+	dedups := 0
+	for i := 1; i < len(srcTimestamps); i++ {
+		ts := srcTimestamps[i]
+		if ts-prevTimestamp < minDelta {
+			dedups++
+			continue
+		}
+		dstTimestamps = append(dstTimestamps, ts)
+		dstValues = append(dstValues, srcValues[i])
+		prevTimestamp = ts
+	}
+	dedupsDuringMerge.Add(dedups)
+	return dstTimestamps, dstValues
+}
+
+var dedupsDuringMerge = metrics.NewCounter(`deduplicated_samples_total{type="merge"}`)
+
+func needsDedup(timestamps []int64, minDelta int64) bool {
+	if len(timestamps) == 0 {
+		return false
+	}
+	prevTimestamp := timestamps[0]
+	for _, ts := range timestamps[1:] {
+		if ts-prevTimestamp < minDelta {
+			return true
+		}
+		prevTimestamp = ts
+	}
+	return false
+}

lib/storage/index_db.go

@@ -1749,7 +1749,7 @@ func (is *indexSearch) updateMetricIDsForTagFilters(metricIDs *uint64set.Set, tf
 		}
 		minMetricIDs = mIDs
 	}
-	metricIDs.Union(minMetricIDs)
+	metricIDs.UnionMayOwn(minMetricIDs)
 	return nil
 }
 
@@ -2068,7 +2068,7 @@ func (is *indexSearch) getMetricIDsForTimeRange(tr TimeRange, maxMetrics int) (*
 			err := isLocal.getMetricIDsForDate(date, &result, maxMetrics)
 			mu.Lock()
 			if metricIDs.Len() < maxMetrics {
-				metricIDs.Union(&result)
+				metricIDs.UnionMayOwn(&result)
 			}
 			if err != nil {
 				errGlobal = err
@@ -2114,7 +2114,7 @@ func (is *indexSearch) tryUpdatingMetricIDsForDateRange(metricIDs *uint64set.Set
 			ok, err := isLocal.tryUpdatingMetricIDsForDate(date, &result, tfs, maxMetrics)
 			mu.Lock()
 			if metricIDs.Len() < maxMetrics {
-				metricIDs.Union(&result)
+				metricIDs.UnionMayOwn(&result)
 			}
 			if !ok {
 				okGlobal = ok
@@ -2203,7 +2203,7 @@ func (is *indexSearch) tryUpdatingMetricIDsForDate(date uint64, metricIDs *uint6
 			return true, nil
 		}
 	}
-	metricIDs.Union(result)
+	metricIDs.UnionMayOwn(result)
 	return true, nil
 }
 

lib/storage/part.go

@@ -5,7 +5,7 @@ import (
 	"path/filepath"
 	"sync"
 	"sync/atomic"
-	"unsafe"
+	"time"
 
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/filestream"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/fs"
@@ -28,7 +28,8 @@ var (
 	maxCachedIndexBlocksPerPartOnce sync.Once
 )
 
-type partInternals struct {
+// part represents a searchable part containing time series data.
+type part struct {
 	ph partHeader
 
 	// Filesystem path to the part.
@@ -39,21 +40,13 @@ type partInternals struct {
 	// Total size in bytes of part data.
 	size uint64
 
-	timestampsFile fs.ReadAtCloser
-	valuesFile     fs.ReadAtCloser
-	indexFile      fs.ReadAtCloser
+	timestampsFile fs.MustReadAtCloser
+	valuesFile     fs.MustReadAtCloser
+	indexFile      fs.MustReadAtCloser
 
 	metaindex []metaindexRow
-}
 
-// part represents a searchable part containing time series data.
-type part struct {
-	partInternals
-
-	// Align ibCache to 8 bytes in order to align internal counters on 32-bit architectures.
-	// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/212
-	_       [(8 - (unsafe.Sizeof(partInternals{}) % 8)) % 8]byte
-	ibCache indexBlockCache
+	ibCache *indexBlockCache
 }
 
 // openFilePart opens file-based part from the given path.
@@ -107,7 +100,7 @@ func openFilePart(path string) (*part, error) {
 //
 // The returned part calls MustClose on all the files passed to newPart
 // when calling part.MustClose.
-func newPart(ph *partHeader, path string, size uint64, metaindexReader filestream.ReadCloser, timestampsFile, valuesFile, indexFile fs.ReadAtCloser) (*part, error) {
+func newPart(ph *partHeader, path string, size uint64, metaindexReader filestream.ReadCloser, timestampsFile, valuesFile, indexFile fs.MustReadAtCloser) (*part, error) {
 	var errors []error
 	metaindex, err := unmarshalMetaindexRows(nil, metaindexReader)
 	if err != nil {
@@ -132,7 +125,7 @@ func newPart(ph *partHeader, path string, size uint64, metaindexReader filestrea
 		return nil, err
 	}
 
-	p.ibCache.Init()
+	p.ibCache = newIndexBlockCache()
 
 	return &p, nil
 }
@@ -152,7 +145,7 @@ func (p *part) MustClose() {
 	p.indexFile.MustClose()
 
 	isBig := p.ph.RowsCount > maxRowsPerSmallPart()
-	p.ibCache.Reset(isBig)
+	p.ibCache.MustClose(isBig)
 }
 
 type indexBlock struct {
@@ -180,19 +173,39 @@ type indexBlockCache struct {
 	requests uint64
 	misses   uint64
 
-	m         map[uint64]*indexBlock
-	missesMap map[uint64]uint64
-	mu        sync.RWMutex
+	m  map[uint64]*indexBlockCacheEntry
+	mu sync.RWMutex
+
+	cleanerStopCh chan struct{}
+	cleanerWG     sync.WaitGroup
 }
 
-func (ibc *indexBlockCache) Init() {
-	ibc.m = make(map[uint64]*indexBlock)
-	ibc.missesMap = make(map[uint64]uint64)
-	ibc.requests = 0
-	ibc.misses = 0
+type indexBlockCacheEntry struct {
+	// Atomically updated counters must go first in the struct, so they are properly
+	// aligned to 8 bytes on 32-bit architectures.
+	// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/212
+	lastAccessTime uint64
+
+	ib *indexBlock
 }
 
-func (ibc *indexBlockCache) Reset(isBig bool) {
+func newIndexBlockCache() *indexBlockCache {
+	var ibc indexBlockCache
+	ibc.m = make(map[uint64]*indexBlockCacheEntry)
+
+	ibc.cleanerStopCh = make(chan struct{})
+	ibc.cleanerWG.Add(1)
+	go func() {
+		defer ibc.cleanerWG.Done()
+		ibc.cleaner()
+	}()
+	return &ibc
+}
+
+func (ibc *indexBlockCache) MustClose(isBig bool) {
+	close(ibc.cleanerStopCh)
+	ibc.cleanerWG.Wait()
+
 	if isBig {
 		atomic.AddUint64(&bigIndexBlockCacheRequests, ibc.requests)
 		atomic.AddUint64(&bigIndexBlockCacheMisses, ibc.misses)
@@ -202,10 +215,36 @@ func (ibc *indexBlockCache) Reset(isBig bool) {
 	}
 	// It is safe returning ibc.m itemst to the pool, since Reset must
 	// be called only when no other goroutines access ibc entries.
-	for _, ib := range ibc.m {
-		putIndexBlock(ib)
+	for _, ibe := range ibc.m {
+		putIndexBlock(ibe.ib)
 	}
-	ibc.Init()
+	ibc.m = nil
+}
+
+// cleaner periodically cleans least recently used items.
+func (ibc *indexBlockCache) cleaner() {
+	t := time.NewTimer(5 * time.Second)
+	for {
+		select {
+		case <-t.C:
+			ibc.cleanByTimeout()
+		case <-ibc.cleanerStopCh:
+			t.Stop()
+			return
+		}
+	}
+}
+
+func (ibc *indexBlockCache) cleanByTimeout() {
+	currentTime := atomic.LoadUint64(&currentTimestamp)
+	ibc.mu.Lock()
+	for k, ibe := range ibc.m {
+		// Delete items accessed more than 10 minutes ago.
+		if currentTime-atomic.LoadUint64(&ibe.lastAccessTime) > 10*60 {
+			delete(ibc.m, k)
+		}
+	}
+	ibc.mu.Unlock()
 }
 
 var (
@@ -220,29 +259,23 @@ func (ibc *indexBlockCache) Get(k uint64) *indexBlock {
 	atomic.AddUint64(&ibc.requests, 1)
 
 	ibc.mu.RLock()
-	ib := ibc.m[k]
+	ibe := ibc.m[k]
 	ibc.mu.RUnlock()
 
-	if ib != nil {
-		return ib
+	if ibe != nil {
+		currentTime := atomic.LoadUint64(&currentTimestamp)
+		if atomic.LoadUint64(&ibe.lastAccessTime) != currentTime {
+			atomic.StoreUint64(&ibe.lastAccessTime, currentTime)
+		}
+		return ibe.ib
 	}
 	atomic.AddUint64(&ibc.misses, 1)
-	ibc.mu.Lock()
-	ibc.missesMap[k]++
-	ibc.mu.Unlock()
 	return nil
 }
 
 func (ibc *indexBlockCache) Put(k uint64, ib *indexBlock) bool {
 	ibc.mu.Lock()
 
-	if ibc.missesMap[k] < 2 {
-		// Do not store infrequently accessed ib in the cache,
-		// so it don't evict frequently accessed items.
-		ibc.mu.Unlock()
-		return false
-	}
-
 	// Clean superflouos cache entries.
 	if overflow := len(ibc.m) - getMaxCachedIndexBlocksPerPart(); overflow > 0 {
 		// Remove 10% of items from the cache.
@@ -256,21 +289,13 @@ func (ibc *indexBlockCache) Put(k uint64, ib *indexBlock) bool {
 			}
 		}
 	}
-	if overflow := len(ibc.missesMap) - 8*getMaxCachedIndexBlocksPerPart(); overflow > 0 {
-		// Remove 10% of items from the cache.
-		overflow = int(float64(len(ibc.missesMap)) * 0.1)
-		for k := range ibc.missesMap {
-			delete(ibc.missesMap, k)
-			overflow--
-			if overflow == 0 {
-				break
-			}
-		}
-	}
 
 	// Store frequently requested ib in the cache.
-	delete(ibc.missesMap, k)
-	ibc.m[k] = ib
+	ibe := &indexBlockCacheEntry{
+		lastAccessTime: atomic.LoadUint64(&currentTimestamp),
+		ib:             ib,
+	}
+	ibc.m[k] = ibe
 	ibc.mu.Unlock()
 	return true
 }
@@ -289,3 +314,15 @@ func (ibc *indexBlockCache) Len() uint64 {
 	ibc.mu.Unlock()
 	return n
 }
+
+func init() {
+	go func() {
+		t := time.NewTimer(time.Second)
+		for tm := range t.C {
+			t := uint64(tm.Unix())
+			atomic.StoreUint64(&currentTimestamp, t)
+		}
+	}()
+}
+
+var currentTimestamp uint64

lib/storage/part_search.go

@@ -88,7 +88,7 @@ func (ps *partSearch) Init(p *part, tsids []TSID, tr TimeRange, fetchData bool)
 	ps.tr = tr
 	ps.fetchData = fetchData
 	ps.metaindex = p.metaindex
-	ps.ibCache = &p.ibCache
+	ps.ibCache = p.ibCache
 
 	// Advance to the first tsid. There is no need in checking
 	// the returned result, since it will be checked in NextBlock.
@@ -229,7 +229,7 @@ func skipSmallMetaindexRows(metaindex []metaindexRow, tsid *TSID) []metaindexRow
 
 func (ps *partSearch) readIndexBlock(mr *metaindexRow) (*indexBlock, error) {
 	ps.compressedIndexBuf = bytesutil.Resize(ps.compressedIndexBuf[:0], int(mr.IndexBlockSize))
-	ps.p.indexFile.ReadAt(ps.compressedIndexBuf, int64(mr.IndexBlockOffset))
+	ps.p.indexFile.MustReadAt(ps.compressedIndexBuf, int64(mr.IndexBlockOffset))
 
 	var err error
 	ps.indexBuf, err = encoding.DecompressZSTD(ps.indexBuf[:0], ps.compressedIndexBuf)
@@ -302,8 +302,8 @@ func (ps *partSearch) readBlock(bh *blockHeader) {
 	}
 
 	ps.Block.timestampsData = bytesutil.Resize(ps.Block.timestampsData[:0], int(bh.TimestampsBlockSize))
-	ps.p.timestampsFile.ReadAt(ps.Block.timestampsData, int64(bh.TimestampsBlockOffset))
+	ps.p.timestampsFile.MustReadAt(ps.Block.timestampsData, int64(bh.TimestampsBlockOffset))
 
 	ps.Block.valuesData = bytesutil.Resize(ps.Block.valuesData[:0], int(bh.ValuesBlockSize))
-	ps.p.valuesFile.ReadAt(ps.Block.valuesData, int64(bh.ValuesBlockOffset))
+	ps.p.valuesFile.MustReadAt(ps.Block.valuesData, int64(bh.ValuesBlockOffset))
 }

lib/storage/partition.go

@@ -613,22 +613,22 @@ func (pt *partition) MustClose() {
 	logger.Infof("waiting for inmemory parts flusher to stop on %q...", pt.smallPartsPath)
 	startTime := time.Now()
 	pt.inmemoryPartsFlusherWG.Wait()
-	logger.Infof("inmemory parts flusher stopped in %s on %q", time.Since(startTime), pt.smallPartsPath)
+	logger.Infof("inmemory parts flusher stopped in %.3f seconds on %q", time.Since(startTime).Seconds(), pt.smallPartsPath)
 
 	logger.Infof("waiting for raw rows flusher to stop on %q...", pt.smallPartsPath)
 	startTime = time.Now()
 	pt.rawRowsFlusherWG.Wait()
-	logger.Infof("raw rows flusher stopped in %s on %q", time.Since(startTime), pt.smallPartsPath)
+	logger.Infof("raw rows flusher stopped in %.3f seconds on %q", time.Since(startTime).Seconds(), pt.smallPartsPath)
 
 	logger.Infof("waiting for small part mergers to stop on %q...", pt.smallPartsPath)
 	startTime = time.Now()
 	pt.smallPartsMergerWG.Wait()
-	logger.Infof("small part mergers stopped in %s on %q", time.Since(startTime), pt.smallPartsPath)
+	logger.Infof("small part mergers stopped in %.3f seconds on %q", time.Since(startTime).Seconds(), pt.smallPartsPath)
 
 	logger.Infof("waiting for big part mergers to stop on %q...", pt.bigPartsPath)
 	startTime = time.Now()
 	pt.bigPartsMergerWG.Wait()
-	logger.Infof("big part mergers stopped in %s on %q", time.Since(startTime), pt.bigPartsPath)
+	logger.Infof("big part mergers stopped in %.3f seconds on %q", time.Since(startTime).Seconds(), pt.bigPartsPath)
 
 	logger.Infof("flushing inmemory parts to files on %q...", pt.smallPartsPath)
 	startTime = time.Now()
@@ -654,7 +654,7 @@ func (pt *partition) MustClose() {
 	if err := pt.mergePartsOptimal(pws); err != nil {
 		logger.Panicf("FATAL: cannot flush %d inmemory parts to files on %q: %s", len(pws), pt.smallPartsPath, err)
 	}
-	logger.Infof("%d inmemory parts have been flushed to files in %s on %q", len(pws), time.Since(startTime), pt.smallPartsPath)
+	logger.Infof("%d inmemory parts have been flushed to files in %.3f seconds on %q", len(pws), time.Since(startTime).Seconds(), pt.smallPartsPath)
 
 	// Remove references to smallParts from the pt, so they may be eventually closed
 	// after all the searches are done.
@@ -1167,7 +1167,8 @@ func (pt *partition) mergeParts(pws []*partWrapper, stopCh <-chan struct{}) erro
 
 	d := time.Since(startTime)
 	if d > 10*time.Second {
-		logger.Infof("merged %d rows in %s at %d rows/sec to %q; sizeBytes: %d", outRowsCount, d, int(float64(outRowsCount)/d.Seconds()), dstPartPath, newPSize)
+		logger.Infof("merged %d rows in %.3f seconds at %d rows/sec to %q; sizeBytes: %d",
+			outRowsCount, d.Seconds(), int(float64(outRowsCount)/d.Seconds()), dstPartPath, newPSize)
 	}
 
 	return nil
@@ -1359,8 +1360,7 @@ func openParts(pathPrefix1, pathPrefix2, path string) ([]*partWrapper, error) {
 			mustCloseParts(pws)
 			return nil, fmt.Errorf("cannot open part %q: %s", partPath, err)
 		}
-		d := time.Since(startTime)
-		logger.Infof("opened part %q in %s", partPath, d)
+		logger.Infof("opened part %q in %.3f seconds", partPath, time.Since(startTime).Seconds())
 
 		pw := &partWrapper{
 			p:        p,
@@ -1407,7 +1407,8 @@ func (pt *partition) CreateSnapshotAt(smallPath, bigPath string) error {
 		return fmt.Errorf("cannot create snapshot for %q: %s", pt.bigPartsPath, err)
 	}
 
-	logger.Infof("created partition snapshot of %q and %q at %q and %q in %s", pt.smallPartsPath, pt.bigPartsPath, smallPath, bigPath, time.Since(startTime))
+	logger.Infof("created partition snapshot of %q and %q at %q and %q in %.3f seconds",
+		pt.smallPartsPath, pt.bigPartsPath, smallPath, bigPath, time.Since(startTime).Seconds())
 	return nil
 }
 

lib/storage/partition_test.go

@@ -110,10 +110,8 @@ func testAppendPartsToMerge(t *testing.T, maxPartsToMerge int, initialRowsCount,
 	prefix := []*partWrapper{
 		{
 			p: &part{
-				partInternals: partInternals{
-					ph: partHeader{
-						RowsCount: 1234,
-					},
+				ph: partHeader{
+					RowsCount: 1234,
 				},
 			},
 		},
@@ -146,10 +144,8 @@ func newTestPartWrappersForRowsCount(rowsCount []uint64) []*partWrapper {
 	for _, rc := range rowsCount {
 		pw := &partWrapper{
 			p: &part{
-				partInternals: partInternals{
-					ph: partHeader{
-						RowsCount: rc,
-					},
+				ph: partHeader{
+					RowsCount: rc,
 				},
 			},
 		}

lib/storage/search.go

@@ -112,7 +112,9 @@ func (s *Search) Init(storage *Storage, tfss []*TagFilters, tr TimeRange, fetchD
 	s.needClosing = true
 
 	tsids, err := storage.searchTSIDs(tfss, tr, maxMetrics)
-
+	if err == nil {
+		err = storage.prefetchMetricNames(tsids)
+	}
 	// It is ok to call Init on error from storage.searchTSIDs.
 	// Init must be called before returning because it will fail
 	// on Seach.MustClose otherwise.

lib/storage/storage.go

@@ -2,6 +2,7 @@ package storage
 
 import (
 	"fmt"
+	"io"
 	"io/ioutil"
 	"math"
 	"os"
@@ -71,10 +72,14 @@ type Storage struct {
 	pendingHourEntriesLock sync.Mutex
 	pendingHourEntries     *uint64set.Set
 
+	// metricIDs for pre-fetched metricNames in the prefetchMetricNames function.
+	prefetchedMetricIDs atomic.Value
+
 	stop chan struct{}
 
-	currHourMetricIDsUpdaterWG sync.WaitGroup
-	retentionWatcherWG         sync.WaitGroup
+	currHourMetricIDsUpdaterWG   sync.WaitGroup
+	retentionWatcherWG           sync.WaitGroup
+	prefetchedMetricIDsCleanerWG sync.WaitGroup
 }
 
 // OpenStorage opens storage on the given path with the given number of retention months.
@@ -127,6 +132,8 @@ func OpenStorage(path string, retentionMonths int) (*Storage, error) {
 	s.prevHourMetricIDs.Store(hmPrev)
 	s.pendingHourEntries = &uint64set.Set{}
 
+	s.prefetchedMetricIDs.Store(&uint64set.Set{})
+
 	// Load indexdb
 	idbPath := path + "/indexdb"
 	idbSnapshotsPath := idbPath + "/snapshots"
@@ -151,6 +158,7 @@ func OpenStorage(path string, retentionMonths int) (*Storage, error) {
 
 	s.startCurrHourMetricIDsUpdater()
 	s.startRetentionWatcher()
+	s.startPrefetchedMetricIDsCleaner()
 
 	return s, nil
 }
@@ -216,7 +224,7 @@ func (s *Storage) CreateSnapshot() (string, error) {
 	fs.MustSyncPath(dstDir)
 	fs.MustSyncPath(srcDir + "/snapshots")
 
-	logger.Infof("created Storage snapshot for %q at %q in %s", srcDir, dstDir, time.Since(startTime))
+	logger.Infof("created Storage snapshot for %q at %q in %.3f seconds", srcDir, dstDir, time.Since(startTime).Seconds())
 	return snapshotName, nil
 }
 
@@ -261,7 +269,7 @@ func (s *Storage) DeleteSnapshot(snapshotName string) error {
 	fs.MustRemoveAll(idbPath)
 	fs.MustRemoveAll(snapshotPath)
 
-	logger.Infof("deleted snapshot %q in %s", snapshotPath, time.Since(startTime))
+	logger.Infof("deleted snapshot %q in %.3f seconds", snapshotPath, time.Since(startTime).Seconds())
 
 	return nil
 }
@@ -313,6 +321,9 @@ type Metrics struct {
 	HourMetricIDCacheSize      uint64
 	HourMetricIDCacheSizeBytes uint64
 
+	PrefetchedMetricIDsSize      uint64
+	PrefetchedMetricIDsSizeBytes uint64
+
 	IndexDBMetrics IndexDBMetrics
 	TableMetrics   TableMetrics
 }
@@ -372,10 +383,35 @@ func (s *Storage) UpdateMetrics(m *Metrics) {
 	m.HourMetricIDCacheSizeBytes += hmCurr.m.SizeBytes()
 	m.HourMetricIDCacheSizeBytes += hmPrev.m.SizeBytes()
 
+	prefetchedMetricIDs := s.prefetchedMetricIDs.Load().(*uint64set.Set)
+	m.PrefetchedMetricIDsSize += uint64(prefetchedMetricIDs.Len())
+	m.PrefetchedMetricIDsSizeBytes += uint64(prefetchedMetricIDs.SizeBytes())
+
 	s.idb().UpdateMetrics(&m.IndexDBMetrics)
 	s.tb.UpdateMetrics(&m.TableMetrics)
 }
 
+func (s *Storage) startPrefetchedMetricIDsCleaner() {
+	s.prefetchedMetricIDsCleanerWG.Add(1)
+	go func() {
+		s.prefetchedMetricIDsCleaner()
+		s.prefetchedMetricIDsCleanerWG.Done()
+	}()
+}
+
+func (s *Storage) prefetchedMetricIDsCleaner() {
+	t := time.NewTicker(7 * time.Minute)
+	for {
+		select {
+		case <-s.stop:
+			t.Stop()
+			return
+		case <-t.C:
+			s.prefetchedMetricIDs.Store(&uint64set.Set{})
+		}
+	}
+}
+
 func (s *Storage) startRetentionWatcher() {
 	s.retentionWatcherWG.Add(1)
 	go func() {
@@ -529,7 +565,7 @@ func (s *Storage) mustLoadHourMetricIDs(hour uint64, name string) *hourMetricIDs
 		m.Add(metricID)
 	}
 
-	logger.Infof("loaded %s from %q in %s; entriesCount: %d; sizeBytes: %d", name, path, time.Since(startTime), hmLen, srcOrigLen)
+	logger.Infof("loaded %s from %q in %.3f seconds; entriesCount: %d; sizeBytes: %d", name, path, time.Since(startTime).Seconds(), hmLen, srcOrigLen)
 	return &hourMetricIDs{
 		m:      m,
 		hour:   hourLoaded,
@@ -553,14 +589,17 @@ func (s *Storage) mustSaveHourMetricIDs(hm *hourMetricIDs, name string) {
 
 	// Marshal hm.m
 	dst = encoding.MarshalUint64(dst, uint64(hm.m.Len()))
-	for _, metricID := range hm.m.AppendTo(nil) {
-		dst = encoding.MarshalUint64(dst, metricID)
-	}
+	hm.m.ForEach(func(part []uint64) bool {
+		for _, metricID := range part {
+			dst = encoding.MarshalUint64(dst, metricID)
+		}
+		return true
+	})
 
 	if err := ioutil.WriteFile(path, dst, 0644); err != nil {
 		logger.Panicf("FATAL: cannot write %d bytes to %q: %s", len(dst), path, err)
 	}
-	logger.Infof("saved %s to %q in %s; entriesCount: %d; sizeBytes: %d", name, path, time.Since(startTime), hm.m.Len(), len(dst))
+	logger.Infof("saved %s to %q in %.3f seconds; entriesCount: %d; sizeBytes: %d", name, path, time.Since(startTime).Seconds(), hm.m.Len(), len(dst))
 }
 
 func (s *Storage) mustLoadCache(info, name string, sizeBytes int) *workingsetcache.Cache {
@@ -570,8 +609,8 @@ func (s *Storage) mustLoadCache(info, name string, sizeBytes int) *workingsetcac
 	c := workingsetcache.Load(path, sizeBytes, time.Hour)
 	var cs fastcache.Stats
 	c.UpdateStats(&cs)
-	logger.Infof("loaded %s cache from %q in %s; entriesCount: %d; sizeBytes: %d",
-		info, path, time.Since(startTime), cs.EntriesCount, cs.BytesSize)
+	logger.Infof("loaded %s cache from %q in %.3f seconds; entriesCount: %d; sizeBytes: %d",
+		info, path, time.Since(startTime).Seconds(), cs.EntriesCount, cs.BytesSize)
 	return c
 }
 
@@ -585,8 +624,8 @@ func (s *Storage) mustSaveAndStopCache(c *workingsetcache.Cache, info, name stri
 	var cs fastcache.Stats
 	c.UpdateStats(&cs)
 	c.Stop()
-	logger.Infof("saved %s cache to %q in %s; entriesCount: %d; sizeBytes: %d",
-		info, path, time.Since(startTime), cs.EntriesCount, cs.BytesSize)
+	logger.Infof("saved %s cache to %q in %.3f seconds; entriesCount: %d; sizeBytes: %d",
+		info, path, time.Since(startTime).Seconds(), cs.EntriesCount, cs.BytesSize)
 }
 
 func nextRetentionDuration(retentionMonths int) time.Duration {
@@ -613,6 +652,47 @@ func (s *Storage) searchTSIDs(tfss []*TagFilters, tr TimeRange, maxMetrics int)
 	return tsids, nil
 }
 
+// prefetchMetricNames pre-fetches metric names for the given tsids into metricID->metricName cache.
+//
+// This should speed-up further searchMetricName calls for metricIDs from tsids.
+func (s *Storage) prefetchMetricNames(tsids []TSID) error {
+	var metricIDs uint64Sorter
+	prefetchedMetricIDs := s.prefetchedMetricIDs.Load().(*uint64set.Set)
+	for i := range tsids {
+		metricID := tsids[i].MetricID
+		if prefetchedMetricIDs.Has(metricID) {
+			continue
+		}
+		metricIDs = append(metricIDs, metricID)
+	}
+	if len(metricIDs) < 500 {
+		// It is cheaper to skip pre-fetching and obtain metricNames inline.
+		return nil
+	}
+
+	// Pre-fetch metricIDs.
+	sort.Sort(metricIDs)
+	var metricName []byte
+	var err error
+	idb := s.idb()
+	is := idb.getIndexSearch()
+	defer idb.putIndexSearch(is)
+	for _, metricID := range metricIDs {
+		metricName, err = is.searchMetricName(metricName[:0], metricID)
+		if err != nil && err != io.EOF {
+			return fmt.Errorf("error in pre-fetching metricName for metricID=%d: %s", metricID, err)
+		}
+	}
+
+	// Store the pre-fetched metricIDs, so they aren't pre-fetched next time.
+	prefetchedMetricIDsNew := prefetchedMetricIDs.Clone()
+	for _, metricID := range metricIDs {
+		prefetchedMetricIDsNew.Add(metricID)
+	}
+	s.prefetchedMetricIDs.Store(prefetchedMetricIDsNew)
+	return nil
+}
+
 // DeleteMetrics deletes all the metrics matching the given tfss.
 //
 // Returns the number of metrics deleted.

lib/storage/storage_test.go

@@ -245,10 +245,12 @@ func TestUpdateCurrHourMetricIDs(t *testing.T) {
 			return
 		}
 		m := pendingHourEntries.Clone()
-		origMetricIDs := hmOrig.m.AppendTo(nil)
-		for _, metricID := range origMetricIDs {
-			m.Add(metricID)
-		}
+		hmOrig.m.ForEach(func(part []uint64) bool {
+			for _, metricID := range part {
+				m.Add(metricID)
+			}
+			return true
+		})
 		if !hmCurr.m.Equal(m) {
 			t.Fatalf("unexpected hm.m; got %v; want %v", hmCurr.m, m)
 		}

lib/storage/table.go

@@ -170,7 +170,7 @@ func (tb *table) CreateSnapshot(snapshotName string) (string, string, error) {
 	fs.MustSyncPath(filepath.Dir(dstSmallDir))
 	fs.MustSyncPath(filepath.Dir(dstBigDir))
 
-	logger.Infof("created table snapshot for %q at (%q, %q) in %s", tb.path, dstSmallDir, dstBigDir, time.Since(startTime))
+	logger.Infof("created table snapshot for %q at (%q, %q) in %.3f seconds", tb.path, dstSmallDir, dstBigDir, time.Since(startTime).Seconds())
 	return dstSmallDir, dstBigDir, nil
 }
 

lib/uint64set/uint64set.go

@@ -3,6 +3,7 @@ package uint64set
 import (
 	"math/bits"
 	"sort"
+	"sync"
 	"unsafe"
 )
 
@@ -31,21 +32,34 @@ func (s *bucket32Sorter) Swap(i, j int) {
 
 // Clone returns an independent copy of s.
 func (s *Set) Clone() *Set {
-	if s == nil {
+	if s == nil || s.itemsCount == 0 {
 		// Return an empty set, so data could be added into it later.
 		return &Set{}
 	}
 	var dst Set
 	dst.itemsCount = s.itemsCount
-	if len(s.buckets) > 0 {
-		dst.buckets = make([]bucket32, len(s.buckets))
-		for i := range s.buckets {
-			s.buckets[i].copyTo(&dst.buckets[i])
-		}
+	dst.buckets = make([]bucket32, len(s.buckets))
+	for i := range s.buckets {
+		s.buckets[i].copyTo(&dst.buckets[i])
 	}
 	return &dst
 }
 
+func (s *Set) fixItemsCount() {
+	n := 0
+	for i := range s.buckets {
+		n += s.buckets[i].getLen()
+	}
+	s.itemsCount = n
+}
+
+func (s *Set) cloneShallow() *Set {
+	var dst Set
+	dst.itemsCount = s.itemsCount
+	dst.buckets = append(dst.buckets[:0], s.buckets...)
+	return &dst
+}
+
 // SizeBytes returns an estimate size of s in RAM.
 func (s *Set) SizeBytes() uint64 {
 	if s == nil {
@@ -143,57 +157,136 @@ func (s *Set) AppendTo(dst []uint64) []uint64 {
 		dst = append(dst[:cap(dst)], make([]uint64, n)...)
 		dst = dst[:dstLen]
 	}
-	// sort s.buckets if it isn't sorted yet
-	if !sort.IsSorted(&s.buckets) {
-		sort.Sort(&s.buckets)
-	}
+	s.sort()
 	for i := range s.buckets {
 		dst = s.buckets[i].appendTo(dst)
 	}
 	return dst
 }
 
+func (s *Set) sort() {
+	// sort s.buckets if it isn't sorted yet
+	if !sort.IsSorted(&s.buckets) {
+		sort.Sort(&s.buckets)
+	}
+}
+
 // Union adds all the items from a to s.
 func (s *Set) Union(a *Set) {
-	// Clone a, since AppendTo may mutate it below.
-	aCopy := a.Clone()
-	if s.Len() == 0 {
-		// Fast path if the initial set is empty.
-		*s = *aCopy
+	s.union(a, false)
+}
+
+// UnionMayOwn adds all the items from a to s.
+//
+// It may own a if s is empty. This means that `a` cannot be used
+// after the call to UnionMayOwn.
+func (s *Set) UnionMayOwn(a *Set) {
+	s.union(a, true)
+}
+
+func (s *Set) union(a *Set, mayOwn bool) {
+	if a.Len() == 0 {
+		// Fast path - nothing to union.
 		return
 	}
-	// TODO: optimize it
-	for _, x := range aCopy.AppendTo(nil) {
-		s.Add(x)
+	if s.Len() == 0 {
+		// Fast path - copy `a` to `s`.
+		if !mayOwn {
+			a = a.Clone()
+		}
+		*s = *a
+		return
 	}
+	// Make shallow copy of `a`, since it can be modified by a.sort().
+	if !mayOwn {
+		a = a.cloneShallow()
+	}
+	a.sort()
+	s.sort()
+	i := 0
+	j := 0
+	sbuckets := s.buckets
+	for {
+		for i < len(sbuckets) && j < len(a.buckets) && sbuckets[i].hi < a.buckets[j].hi {
+			i++
+		}
+		if i >= len(sbuckets) {
+			for j < len(a.buckets) {
+				b32 := s.addBucket32()
+				a.buckets[j].copyTo(b32)
+				j++
+			}
+			break
+		}
+		for j < len(a.buckets) && a.buckets[j].hi < sbuckets[i].hi {
+			b32 := s.addBucket32()
+			a.buckets[j].copyTo(b32)
+			j++
+		}
+		if j >= len(a.buckets) {
+			break
+		}
+		if sbuckets[i].hi == a.buckets[j].hi {
+			sbuckets[i].union(&a.buckets[j], mayOwn)
+			i++
+			j++
+		}
+	}
+	s.fixItemsCount()
 }
 
 // Intersect removes all the items missing in a from s.
 func (s *Set) Intersect(a *Set) {
-	if a.Len() == 0 {
-		// Fast path
+	if s.Len() == 0 || a.Len() == 0 {
+		// Fast path - the result is empty.
 		*s = Set{}
 		return
 	}
-	// TODO: optimize it
-	for _, x := range s.AppendTo(nil) {
-		if !a.Has(x) {
-			s.Del(x)
+	// Make shallow copy of `a`, since it can be modified by a.sort().
+	a = a.cloneShallow()
+	a.sort()
+	s.sort()
+	i := 0
+	j := 0
+	for {
+		for i < len(s.buckets) && j < len(a.buckets) && s.buckets[i].hi < a.buckets[j].hi {
+			s.buckets[i] = bucket32{}
+			i++
+		}
+		if i >= len(s.buckets) {
+			break
+		}
+		for j < len(a.buckets) && a.buckets[j].hi < s.buckets[i].hi {
+			j++
+		}
+		if j >= len(a.buckets) {
+			for i < len(s.buckets) {
+				s.buckets[i] = bucket32{}
+				i++
+			}
+			break
+		}
+		if s.buckets[i].hi == a.buckets[j].hi {
+			s.buckets[i].intersect(&a.buckets[j])
+			i++
+			j++
 		}
 	}
+	s.fixItemsCount()
 }
 
 // Subtract removes from s all the shared items between s and a.
 func (s *Set) Subtract(a *Set) {
-	if s.Len() == 0 {
+	if s.Len() == 0 || a.Len() == 0 {
+		// Fast path - nothing to subtract.
 		return
 	}
-	// Copy a because AppendTo below can mutate a.
-	aCopy := a.Clone()
-	// TODO: optimize it
-	for _, x := range aCopy.AppendTo(nil) {
-		s.Del(x)
-	}
+	a.ForEach(func(part []uint64) bool {
+		for _, x := range part {
+			s.Del(x)
+		}
+		return true
+	})
 }
 
 // Equal returns true if s contains the same items as a.
@@ -201,21 +294,153 @@ func (s *Set) Equal(a *Set) bool {
 	if s.Len() != a.Len() {
 		return false
 	}
-	// Copy a because AppendTo below can mutate a
-	aCopy := a.Clone()
-	// TODO: optimize it
-	for _, x := range aCopy.AppendTo(nil) {
-		if !s.Has(x) {
-			return false
+	equal := true
+	a.ForEach(func(part []uint64) bool {
+		for _, x := range part {
+			if !s.Has(x) {
+				equal = false
+				return false
+			}
+		}
+		return true
+	})
+	return equal
+}
+
+// ForEach calls f for all the items stored in s.
+//
+// Each call to f contains part with arbitrary part of items stored in the set.
+// The iteration is stopped if f returns false.
+func (s *Set) ForEach(f func(part []uint64) bool) {
+	if s == nil {
+		return
+	}
+	for i := range s.buckets {
+		if !s.buckets[i].forEach(f) {
+			return
 		}
 	}
-	return true
 }
 
 type bucket32 struct {
 	hi      uint32
 	b16his  []uint16
 	buckets []bucket16
+
+	// hint may contain bucket index for the last successful add or del operation.
+	// This allows saving CPU time on subsequent calls to the same bucket.
+	hint int
+}
+
+func (b *bucket32) cloneShallow() *bucket32 {
+	var dst bucket32
+	dst.hi = b.hi
+	dst.b16his = append(dst.b16his[:0], b.b16his...)
+	dst.buckets = append(dst.buckets[:0], b.buckets...)
+	dst.hint = b.hint
+	return &dst
+}
+
+func (b *bucket32) getLen() int {
+	n := 0
+	for i := range b.buckets {
+		n += b.buckets[i].getLen()
+	}
+	return n
+}
+
+func (b *bucket32) union(a *bucket32, mayOwn bool) {
+	if !mayOwn {
+		a = a.cloneShallow() // clone a, since it is sorted below.
+	}
+	a.sort()
+	b.sort()
+	i := 0
+	j := 0
+	bb16his := b.b16his
+	for {
+		for i < len(bb16his) && j < len(a.b16his) && bb16his[i] < a.b16his[j] {
+			i++
+		}
+		if i >= len(bb16his) {
+			for j < len(a.b16his) {
+				b.b16his = append(b.b16his, a.b16his[j])
+				b16 := b.addBucket16()
+				a.buckets[j].copyTo(b16)
+				j++
+			}
+			break
+		}
+		for j < len(a.b16his) && a.b16his[j] < bb16his[i] {
+			b.b16his = append(b.b16his, a.b16his[j])
+			b16 := b.addBucket16()
+			a.buckets[j].copyTo(b16)
+			j++
+		}
+		if j >= len(a.b16his) {
+			break
+		}
+		if bb16his[i] == a.b16his[j] {
+			b.buckets[i].union(&a.buckets[j])
+			i++
+			j++
+		}
+	}
+	b.sort()
+}
+
+func (b *bucket32) intersect(a *bucket32) {
+	a = a.cloneShallow() // clone a, since it is sorted below.
+	a.sort()
+	b.sort()
+	i := 0
+	j := 0
+	for {
+		for i < len(b.b16his) && j < len(a.b16his) && b.b16his[i] < a.b16his[j] {
+			b.buckets[i] = bucket16{}
+			i++
+		}
+		if i >= len(b.b16his) {
+			break
+		}
+		for j < len(a.b16his) && a.b16his[j] < b.b16his[i] {
+			j++
+		}
+		if j >= len(a.b16his) {
+			for i < len(b.b16his) {
+				b.buckets[i] = bucket16{}
+				i++
+			}
+			break
+		}
+		if b.b16his[i] == a.b16his[j] {
+			b.buckets[i].intersect(&a.buckets[j])
+			i++
+			j++
+		}
+	}
+}
+
+func (b *bucket32) forEach(f func(part []uint64) bool) bool {
+	xbuf := partBufPool.Get().(*[]uint64)
+	buf := *xbuf
+	for i := range b.buckets {
+		hi16 := b.b16his[i]
+		buf = b.buckets[i].appendTo(buf[:0], b.hi, hi16)
+		if !f(buf) {
+			return false
+		}
+	}
+	*xbuf = buf
+	partBufPool.Put(xbuf)
+	return true
+}
+
+var partBufPool = &sync.Pool{
+	New: func() interface{} {
+		buf := make([]uint64, 0, bitsPerBucket)
+		return &buf
+	},
 }
 
 func (b *bucket32) sizeBytes() uint64 {
@@ -240,6 +465,7 @@ func (b *bucket32) copyTo(dst *bucket32) {
 			b.buckets[i].copyTo(&dst.buckets[i])
 		}
 	}
+	dst.hint = b.hint
 }
 
 // This is for sort.Interface
@@ -257,11 +483,26 @@ const maxUnsortedBuckets = 32
 func (b *bucket32) add(x uint32) bool {
 	hi := uint16(x >> 16)
 	lo := uint16(x)
+	if n := b.hint; n < len(b.b16his) && b.b16his[n] == hi {
+		// Fast path - add to the previously used bucket.
+		return n < len(b.buckets) && b.buckets[n].add(lo)
+	}
+	return b.addSlow(hi, lo)
+}
+
+func (b *bucket32) addSlow(hi, lo uint16) bool {
 	if len(b.buckets) > maxUnsortedBuckets {
-		return b.addSlow(hi, lo)
+		n := binarySearch16(b.b16his, hi)
+		b.hint = n
+		if n < 0 || n >= len(b.b16his) || b.b16his[n] != hi {
+			b.addAllocBig(hi, lo, n)
+			return true
+		}
+		return n < len(b.buckets) && b.buckets[n].add(lo)
 	}
 	for i, hi16 := range b.b16his {
 		if hi16 == hi {
+			b.hint = i
 			return i < len(b.buckets) && b.buckets[i].add(lo)
 		}
 	}
@@ -283,15 +524,6 @@ func (b *bucket32) addBucket16() *bucket16 {
 	return &b.buckets[len(b.buckets)-1]
 }
 
-func (b *bucket32) addSlow(hi, lo uint16) bool {
-	n := binarySearch16(b.b16his, hi)
-	if n < 0 || n >= len(b.b16his) || b.b16his[n] != hi {
-		b.addAllocBig(hi, lo, n)
-		return true
-	}
-	return n < len(b.buckets) && b.buckets[n].add(lo)
-}
-
 func (b *bucket32) addAllocBig(hi, lo uint16, n int) {
 	if n < 0 {
 		// This is a hint to Go compiler to remove automatic bounds checks below.
@@ -336,28 +568,34 @@ func (b *bucket32) hasSlow(hi, lo uint16) bool {
 func (b *bucket32) del(x uint32) bool {
 	hi := uint16(x >> 16)
 	lo := uint16(x)
+	if n := b.hint; n < len(b.b16his) && b.b16his[n] == hi {
+		// Fast path - use the bucket from the previous operation.
+		return n < len(b.buckets) && b.buckets[n].del(lo)
+	}
+	return b.delSlow(hi, lo)
+}
+
+func (b *bucket32) delSlow(hi, lo uint16) bool {
 	if len(b.buckets) > maxUnsortedBuckets {
-		return b.delSlow(hi, lo)
+		n := binarySearch16(b.b16his, hi)
+		b.hint = n
+		if n < 0 || n >= len(b.b16his) || b.b16his[n] != hi {
+			return false
+		}
+		return n < len(b.buckets) && b.buckets[n].del(lo)
 	}
 	for i, hi16 := range b.b16his {
 		if hi16 == hi {
+			b.hint = i
 			return i < len(b.buckets) && b.buckets[i].del(lo)
 		}
 	}
 	return false
 }
 
-func (b *bucket32) delSlow(hi, lo uint16) bool {
-	n := binarySearch16(b.b16his, hi)
-	if n < 0 || n >= len(b.b16his) || b.b16his[n] != hi {
-		return false
-	}
-	return n < len(b.buckets) && b.buckets[n].del(lo)
-}
-
 func (b *bucket32) appendTo(dst []uint64) []uint64 {
-	if len(b.buckets) <= maxUnsortedBuckets && !sort.IsSorted(b) {
-		sort.Sort(b)
+	if len(b.buckets) <= maxUnsortedBuckets {
+		b.sort()
 	}
 	for i := range b.buckets {
 		hi16 := b.b16his[i]
@@ -366,6 +604,12 @@ func (b *bucket32) appendTo(dst []uint64) []uint64 {
 	return dst
 }
 
+func (b *bucket32) sort() {
+	if !sort.IsSorted(b) {
+		sort.Sort(b)
+	}
+}
+
 const (
 	bitsPerBucket  = 1 << 16
 	wordsPerBucket = bitsPerBucket / 64
@@ -377,6 +621,67 @@ type bucket16 struct {
 	smallPool    [56]uint16
 }
 
+func (b *bucket16) getLen() int {
+	if b.bits == nil {
+		return b.smallPoolLen
+	}
+	n := 0
+	for _, x := range b.bits {
+		if x > 0 {
+			n += bits.OnesCount64(x)
+		}
+	}
+	return n
+}
+
+func (b *bucket16) union(a *bucket16) {
+	if a.bits != nil && b.bits != nil {
+		// Fast path - use bitwise ops.
+		for i, ax := range a.bits {
+			bx := b.bits[i]
+			bx |= ax
+			b.bits[i] = bx
+		}
+		return
+	}
+
+	// Slow path
+	xbuf := partBufPool.Get().(*[]uint64)
+	buf := *xbuf
+	buf = a.appendTo(buf[:0], 0, 0)
+	for _, x := range buf {
+		x16 := uint16(x)
+		b.add(x16)
+	}
+	*xbuf = buf
+	partBufPool.Put(xbuf)
+}
+
+func (b *bucket16) intersect(a *bucket16) {
+	if a.bits != nil && b.bits != nil {
+		// Fast path - use bitwise ops
+		for i, ax := range a.bits {
+			bx := b.bits[i]
+			bx &= ax
+			b.bits[i] = bx
+		}
+		return
+	}
+
+	// Slow path
+	xbuf := partBufPool.Get().(*[]uint64)
+	buf := *xbuf
+	buf = b.appendTo(buf[:0], 0, 0)
+	for _, x := range buf {
+		x16 := uint16(x)
+		if !a.has(x16) {
+			b.del(x16)
+		}
+	}
+	*xbuf = buf
+	partBufPool.Put(xbuf)
+}
+
 func (b *bucket16) sizeBytes() uint64 {
 	return uint64(unsafe.Sizeof(*b)) + uint64(unsafe.Sizeof(*b.bits))
 }
@@ -464,14 +769,18 @@ func (b *bucket16) delFromSmallPool(x uint16) bool {
 func (b *bucket16) appendTo(dst []uint64, hi uint32, hi16 uint16) []uint64 {
 	hi64 := uint64(hi)<<32 | uint64(hi16)<<16
 	if b.bits == nil {
-		a := b.smallPool[:b.smallPoolLen]
-		if len(a) > 1 {
-			sort.Slice(a, func(i, j int) bool { return a[i] < a[j] })
+		// Use uint16Sorter instead of sort.Slice here in order to reduce memory allocations.
+		a := uint16SorterPool.Get().(*uint16Sorter)
+		*a = uint16Sorter(b.smallPool[:b.smallPoolLen])
+		if len(*a) > 1 && !sort.IsSorted(a) {
+			sort.Sort(a)
 		}
-		for _, v := range a {
+		for _, v := range *a {
 			x := hi64 | uint64(v)
 			dst = append(dst, x)
 		}
+		*a = nil
+		uint16SorterPool.Put(a)
 		return dst
 	}
 	var wordNum uint64
@@ -495,6 +804,22 @@ func (b *bucket16) appendTo(dst []uint64, hi uint32, hi16 uint16) []uint64 {
 	return dst
 }
 
+var uint16SorterPool = &sync.Pool{
+	New: func() interface{} {
+		return &uint16Sorter{}
+	},
+}
+
+type uint16Sorter []uint16
+
+func (s uint16Sorter) Len() int { return len(s) }
+func (s uint16Sorter) Less(i, j int) bool {
+	return s[i] < s[j]
+}
+func (s uint16Sorter) Swap(i, j int) {
+	s[i], s[j] = s[j], s[i]
+}
+
 func getWordNumBitMask(x uint16) (uint16, uint64) {
 	wordNum := x / 64
 	bitMask := uint64(1) << (x & 63)

lib/uint64set/uint64set_test.go

@@ -166,6 +166,48 @@ func testSetBasicOps(t *testing.T, itemsCount int) {
 		}
 	}
 
+	// Verify ForEach
+	{
+		var s Set
+		m := make(map[uint64]bool)
+		for i := 0; i < itemsCount; i++ {
+			v := uint64(i) + offset
+			s.Add(v)
+			m[v] = true
+		}
+
+		// Verify visiting all the items.
+		s.ForEach(func(part []uint64) bool {
+			for _, v := range part {
+				if !m[v] {
+					t.Fatalf("unexpected value v=%d passed to ForEach", v)
+				}
+				delete(m, v)
+			}
+			return true
+		})
+		if len(m) != 0 {
+			t.Fatalf("ForEach didn't visit %d items; items: %v", len(m), m)
+		}
+
+		// Verify fast stop
+		calls := 0
+		s.ForEach(func(part []uint64) bool {
+			calls++
+			return false
+		})
+		if itemsCount > 0 && calls != 1 {
+			t.Fatalf("Unexpected number of ForEach callback calls; got %d; want %d", calls, 1)
+		}
+
+		// Verify ForEach on nil set.
+		var s1 *Set
+		s1.ForEach(func(part []uint64) bool {
+			t.Fatalf("callback shouldn't be called on empty set")
+			return true
+		})
+	}
+
 	// Verify union
 	{
 		const unionOffset = 12345
@@ -190,32 +232,84 @@ func testSetBasicOps(t *testing.T, itemsCount int) {
 		if n := s3.Len(); n != expectedLen {
 			t.Fatalf("unexpected s3.Len() after union with empty set; got %d; want %d", n, expectedLen)
 		}
+		var s4 Set
+		expectedLen = s3.Len()
+		s3.Union(&s4)
+		if n := s3.Len(); n != expectedLen {
+			t.Fatalf("unexpected s3.Len() after union with empty set; got %d; want %d", n, expectedLen)
+		}
+	}
+
+	// Verify UnionMayOwn
+	{
+		const unionOffset = 12345
+		var s1, s2 Set
+		for i := 0; i < itemsCount; i++ {
+			s1.Add(uint64(i) + offset)
+			s2.Add(uint64(i) + offset + unionOffset)
+		}
+		s1.UnionMayOwn(&s2)
+		expectedLen := 2 * itemsCount
+		if itemsCount > unionOffset {
+			expectedLen = itemsCount + unionOffset
+		}
+		if n := s1.Len(); n != expectedLen {
+			t.Fatalf("unexpected s1.Len() after union; got %d; want %d", n, expectedLen)
+		}
+
+		// Verify union on empty set.
+		var s3 Set
+		expectedLen = s1.Len()
+		s3.UnionMayOwn(&s1)
+		if n := s3.Len(); n != expectedLen {
+			t.Fatalf("unexpected s3.Len() after union with empty set; got %d; want %d", n, expectedLen)
+		}
+		var s4 Set
+		expectedLen = s3.Len()
+		s3.UnionMayOwn(&s4)
+		if n := s3.Len(); n != expectedLen {
+			t.Fatalf("unexpected s3.Len() after union with empty set; got %d; want %d", n, expectedLen)
+		}
 	}
 
 	// Verify intersect
 	{
-		const intersectOffset = 12345
+		// Verify s1.Intersect(s2) and s2.Intersect(s1)
 		var s1, s2 Set
-		for i := 0; i < itemsCount; i++ {
-			s1.Add(uint64(i) + offset)
-			s2.Add(uint64(i) + offset + intersectOffset)
-		}
-		s1.Intersect(&s2)
-		expectedLen := 0
-		if itemsCount > intersectOffset {
-			expectedLen = itemsCount - intersectOffset
-		}
-		if n := s1.Len(); n != expectedLen {
-			t.Fatalf("unexpected s1.Len() after intersect; got %d; want %d", n, expectedLen)
+		for _, intersectOffset := range []uint64{123, 12345, 1<<32 + 4343} {
+			s1 = Set{}
+			s2 = Set{}
+			for i := 0; i < itemsCount; i++ {
+				s1.Add(uint64(i) + offset)
+				s2.Add(uint64(i) + offset + intersectOffset)
+			}
+			expectedLen := 0
+			if uint64(itemsCount) > intersectOffset {
+				expectedLen = int(uint64(itemsCount) - intersectOffset)
+			}
+			s1Copy := s1.Clone()
+			s1Copy.Intersect(&s2)
+			if n := s1Copy.Len(); n != expectedLen {
+				t.Fatalf("unexpected s1.Len() after intersect; got %d; want %d", n, expectedLen)
+			}
+			s2.Intersect(&s1)
+			if n := s2.Len(); n != expectedLen {
+				t.Fatalf("unexpected s2.Len() after intersect; got %d; want %d", n, expectedLen)
+			}
 		}
 
 		// Verify intersect on empty set.
 		var s3 Set
 		s2.Intersect(&s3)
-		expectedLen = 0
-		if n := s2.Len(); n != 0 {
+		expectedLen := 0
+		if n := s2.Len(); n != expectedLen {
 			t.Fatalf("unexpected s3.Len() after intersect with empty set; got %d; want %d", n, expectedLen)
 		}
+		var s4 Set
+		s4.Intersect(&s1)
+		if n := s4.Len(); n != expectedLen {
+			t.Fatalf("unexpected s4.Len() after intersect with empty set; got %d; want %d", n, expectedLen)
+		}
 	}
 
 	// Verify subtract

lib/uint64set/uint64set_timing_test.go

@@ -8,6 +8,107 @@ import (
 	"github.com/valyala/fastrand"
 )
 
+func BenchmarkUnionNoOverlap(b *testing.B) {
+	for _, itemsCount := range []int{1e3, 1e4, 1e5, 1e6, 1e7} {
+		start := uint64(time.Now().UnixNano())
+		sa := createRangeSet(start, itemsCount)
+		sb := createRangeSet(start+uint64(itemsCount), itemsCount)
+		b.Run(fmt.Sprintf("items_%d", itemsCount), func(b *testing.B) {
+			benchmarkUnion(b, sa, sb)
+		})
+	}
+}
+
+func BenchmarkUnionPartialOverlap(b *testing.B) {
+	for _, itemsCount := range []int{1e3, 1e4, 1e5, 1e6, 1e7} {
+		start := uint64(time.Now().UnixNano())
+		sa := createRangeSet(start, itemsCount)
+		sb := createRangeSet(start+uint64(itemsCount/2), itemsCount)
+		b.Run(fmt.Sprintf("items_%d", itemsCount), func(b *testing.B) {
+			benchmarkUnion(b, sa, sb)
+		})
+	}
+}
+
+func BenchmarkUnionFullOverlap(b *testing.B) {
+	for _, itemsCount := range []int{1e3, 1e4, 1e5, 1e6, 1e7} {
+		start := uint64(time.Now().UnixNano())
+		sa := createRangeSet(start, itemsCount)
+		sb := createRangeSet(start, itemsCount)
+		b.Run(fmt.Sprintf("items_%d", itemsCount), func(b *testing.B) {
+			benchmarkUnion(b, sa, sb)
+		})
+	}
+}
+
+func benchmarkUnion(b *testing.B, sa, sb *Set) {
+	b.ReportAllocs()
+	b.SetBytes(int64(sa.Len() + sb.Len()))
+	b.RunParallel(func(pb *testing.PB) {
+		for pb.Next() {
+			saCopy := sa.Clone()
+			sbCopy := sb.Clone()
+			saCopy.Union(sb)
+			sbCopy.Union(sa)
+		}
+	})
+}
+
+func BenchmarkIntersectNoOverlap(b *testing.B) {
+	for _, itemsCount := range []int{1e3, 1e4, 1e5, 1e6, 1e7} {
+		start := uint64(time.Now().UnixNano())
+		sa := createRangeSet(start, itemsCount)
+		sb := createRangeSet(start+uint64(itemsCount), itemsCount)
+		b.Run(fmt.Sprintf("items_%d", itemsCount), func(b *testing.B) {
+			benchmarkIntersect(b, sa, sb)
+		})
+	}
+}
+
+func BenchmarkIntersectPartialOverlap(b *testing.B) {
+	for _, itemsCount := range []int{1e3, 1e4, 1e5, 1e6, 1e7} {
+		start := uint64(time.Now().UnixNano())
+		sa := createRangeSet(start, itemsCount)
+		sb := createRangeSet(start+uint64(itemsCount/2), itemsCount)
+		b.Run(fmt.Sprintf("items_%d", itemsCount), func(b *testing.B) {
+			benchmarkIntersect(b, sa, sb)
+		})
+	}
+}
+
+func BenchmarkIntersectFullOverlap(b *testing.B) {
+	for _, itemsCount := range []int{1e3, 1e4, 1e5, 1e6, 1e7} {
+		start := uint64(time.Now().UnixNano())
+		sa := createRangeSet(start, itemsCount)
+		sb := createRangeSet(start, itemsCount)
+		b.Run(fmt.Sprintf("items_%d", itemsCount), func(b *testing.B) {
+			benchmarkIntersect(b, sa, sb)
+		})
+	}
+}
+
+func benchmarkIntersect(b *testing.B, sa, sb *Set) {
+	b.ReportAllocs()
+	b.SetBytes(int64(sa.Len() + sb.Len()))
+	b.RunParallel(func(pb *testing.PB) {
+		for pb.Next() {
+			saCopy := sa.Clone()
+			sbCopy := sb.Clone()
+			saCopy.Intersect(sb)
+			sbCopy.Intersect(sa)
+		}
+	})
+}
+
+func createRangeSet(start uint64, itemsCount int) *Set {
+	var s Set
+	for i := 0; i < itemsCount; i++ {
+		n := start + uint64(i)
+		s.Add(n)
+	}
+	return &s
+}
+
 func BenchmarkSetAddRandomLastBits(b *testing.B) {
 	const itemsCount = 1e5
 	for _, lastBits := range []uint64{20, 24, 28, 32} {
@@ -16,10 +117,10 @@ func BenchmarkSetAddRandomLastBits(b *testing.B) {
 			b.ReportAllocs()
 			b.SetBytes(int64(itemsCount))
 			b.RunParallel(func(pb *testing.PB) {
+				var rng fastrand.RNG
 				for pb.Next() {
 					start := uint64(time.Now().UnixNano())
 					var s Set
-					var rng fastrand.RNG
 					for i := 0; i < itemsCount; i++ {
 						n := start | (uint64(rng.Uint32()) & mask)
 						s.Add(n)
@@ -38,10 +139,10 @@ func BenchmarkMapAddRandomLastBits(b *testing.B) {
 			b.ReportAllocs()
 			b.SetBytes(int64(itemsCount))
 			b.RunParallel(func(pb *testing.PB) {
+				var rng fastrand.RNG
 				for pb.Next() {
 					start := uint64(time.Now().UnixNano())
 					m := make(map[uint64]struct{})
-					var rng fastrand.RNG
 					for i := 0; i < itemsCount; i++ {
 						n := start | (uint64(rng.Uint32()) & mask)
 						m[n] = struct{}{}

vendor/cloud.google.com/go/CHANGES.md

@@ -1,5 +1,13 @@
 # Changes
 
+## v0.51.0
+
+- secretmanager:
+  - add IAM helper for generic resource IAM handle
+- cloudbuild:
+  - migrate to microgen in a major version
+- Various updates to autogenerated clients.
+
 ## v0.50.0
 
 - profiler:

vendor/cloud.google.com/go/go.mod

@@ -7,24 +7,26 @@ require (
 	cloud.google.com/go/datastore v1.0.0
 	cloud.google.com/go/pubsub v1.0.1
 	cloud.google.com/go/storage v1.0.0
+	github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7 // indirect
 	github.com/golang/mock v1.3.1
 	github.com/golang/protobuf v1.3.2
-	github.com/google/go-cmp v0.3.0
+	github.com/google/go-cmp v0.3.1
 	github.com/google/martian v2.1.0+incompatible
-	github.com/google/pprof v0.0.0-20190515194954-54271f7e092f
+	github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc
 	github.com/googleapis/gax-go/v2 v2.0.5
-	github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024
-	go.opencensus.io v0.22.0
-	golang.org/x/exp v0.0.0-20191129062945-2f5052295587
+	github.com/jstemmer/go-junit-report v0.9.1
+	go.opencensus.io v0.22.2
+	golang.org/x/exp v0.0.0-20191227195350-da58074b4299
 	golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f
-	golang.org/x/net v0.0.0-20190724013045-ca1201d0de80
-	golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45
+	golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553
+	golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6
 	golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e // indirect
-	golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e // indirect
+	golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8 // indirect
 	golang.org/x/text v0.3.2
-	golang.org/x/tools v0.0.0-20191216173652-a0e659d51361
-	google.golang.org/api v0.14.0
-	google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1
-	google.golang.org/grpc v1.21.1
+	golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4
+	google.golang.org/api v0.15.0
+	google.golang.org/appengine v1.6.5 // indirect
+	google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb
+	google.golang.org/grpc v1.26.0
 	honnef.co/go/tools v0.0.1-2019.2.3
 )