e26de23739
Add the support of storage and retrieval of samples with future timestamps as requested in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/827 What to expect: - By default, the max future timestamp is still limited to `now+2d`. To change it, set the `-futureRetention` flag in `vmstorage`. The max flag value is currently limited to `100y`. It can be extended if we see a demand for this, but it can't be more than `~ 290y` due to how the time duration is implemented in Go. The flag value can't be less than `2d`. - downsampling and retention filters (available in enterprise edition) are currently not supported for future timestamps - If `vmstorage` restarts with a smaller value of `-futureRetention` flag, any future partitions that are outside the new future retention will be automatically deleted. - Data ingestion, data retrieval, backup/restore, timeseries (soft) deletion, and other operations work with future timestamps the same way as with the historical timestamps. - In the cluster version, the affected binaries are `vmstorage` and `vmselect`. This means that `vmselect` version must match `vmstorage` version if you want to query future timestamps. `vminsert` was not affected, so its version can be a lower one. - If you downgrade the `vmstorage`, the data with future timestamps will remain on disk and memory (per-partition caches) but won't be available for querying. Signed-off-by: Artem Fetishev <rtm@victoriametrics.com> Signed-off-by: Artem Fetishev <149964189+rtm0@users.noreply.github.com> Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
App Integration Tests
The apptest package contains the integration tests for the VictoriaMetrics
applications (such as vmstorage, vminsert, and vmselect).
An integration test aims at verifying the behavior of an application as a whole, as apposed to a unit test that verifies the behavior of a building block of an application.
To achieve that an integration test starts an application in a separate process and then issues HTTP requests to it and verifies the responses, examines the metrics the app exposes and/or files it creates, etc.
Note that an object of testing may be not just a single app, but several apps working together. A good example is VictoriaMetrics cluster. An integration test may reproduce an arbitrary cluster configuration and verify how the components work together as a system.
The package provides a collection of helpers to start applications and make queries to them:
app.go- contains the generic code for staring an application and should not be used by integration tests directly.{vmstorage,vminsert,etc}.go- build on top ofapp.goand provide the code for staring a specific application.client.go- provides helper functions for sending HTTP requests to applications.
The integration tests themselves reside in tests/*_test.go files. Apart from having
the _test suffix, there are no strict rules of how to name a file, but the
name should reflect the prevailing purpose of the tests located in that file.
For example, sharding_test.go aims at testing data sharding.
Since integration tests start applications in a separate process, they require
the application binary files to be built and put into the bin directory. The
build rule used for running integration tests, make apptest,
accounts for that, it builds all application binaries before running the tests.
But if you want to run the tests without make, i.e. by executing
go test ./app/apptest, you will need to build the binaries first (for example,
by executing make all).
Not all binaries can be built from master branch, cluster binaries can be built
only from cluster branch. Hence, not all test cases suitable to run in both branches:
- If test is using binaries from
clusterbranch, then test name should be prefixed withTestClusterword - If test is using binaries from
masterbranch, then test name should be prefixed withTestSingleword.