From 6ec422160bf3e7f5aec78c1401970359c996bcdf Mon Sep 17 00:00:00 2001 From: Roman Khavronenko Date: Mon, 5 May 2025 15:32:44 +0200 Subject: [PATCH] docs: add integration page (#8870) The rationale behind this change was explained here https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8595#issuecomment-2841788436 This PR moves only Prom and Grafana docs to show the change and limit the amount of changes across docs. Once accepted, the follow-up PRs will move Graphite, InfluxDB, OpenTSDB, Datadog and NewRelic docs. --------- Signed-off-by: hagen1778 --- README.md | 2 +- docs/guides/migrate-from-influx/README.md | 2 +- .../understand-your-setup-size/README.md | 2 +- docs/victoriametrics/FAQ.md | 6 +- docs/victoriametrics/Quick-Start.md | 6 +- docs/victoriametrics/README.md | 86 ++---------------- .../changelog/CHANGELOG_2023.md | 2 +- .../data-ingestion/Prometheus.md | 2 +- docs/victoriametrics/integrations/_index.md | 16 ++++ .../grafana-datasource-prometheus.webp | Bin docs/victoriametrics/integrations/grafana.md | 60 ++++++++++++ .../integrations/prometheus.md | 66 ++++++++++++++ .../keyConcepts/keyConcepts.md | 6 +- 13 files changed, 163 insertions(+), 93 deletions(-) create mode 100644 docs/victoriametrics/integrations/_index.md rename docs/victoriametrics/{ => integrations}/grafana-datasource-prometheus.webp (100%) create mode 100644 docs/victoriametrics/integrations/grafana.md create mode 100644 docs/victoriametrics/integrations/prometheus.md diff --git a/README.md b/README.md index b3d2f86a26..f6be78b306 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,7 @@ VictoriaMetrics is optimized for timeseries data, even when old time series are * **Easy to setup**: No dependencies, single [small binary](https://medium.com/@valyala/stripping-dependency-bloat-in-victoriametrics-docker-image-983fb5912b0d), configuration through command-line flags, but the default is also fine-tuned; backup and restore with [instant snapshots](https://medium.com/@valyala/how-victoriametrics-makes-instant-snapshots-for-multi-terabyte-time-series-data-e1f3fb0e0282). * **Global query view**: Multiple Prometheus instances or any other data sources may ingest data into VictoriaMetrics and queried via a single query. * **Various Protocols**: Support metric scraping, ingestion and backfilling in various protocol. - * [Prometheus exporters](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-scrape-prometheus-exporters-such-as-node-exporter), [Prometheus remote write API](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-setup), [Prometheus exposition format](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-import-data-in-prometheus-exposition-format). + * [Prometheus exporters](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-scrape-prometheus-exporters-such-as-node-exporter), [Prometheus remote write API](https://docs.victoriametrics.com/victoriametrics/integrations/prometheus), [Prometheus exposition format](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-import-data-in-prometheus-exposition-format). * [InfluxDB line protocol](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-send-data-from-influxdb-compatible-agents-such-as-telegraf) over HTTP, TCP and UDP. * [Graphite plaintext protocol](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-send-data-from-graphite-compatible-agents-such-as-statsd) with [tags](https://graphite.readthedocs.io/en/latest/tags.html#carbon). * [OpenTSDB put message](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#sending-data-via-telnet-put-protocol). diff --git a/docs/guides/migrate-from-influx/README.md b/docs/guides/migrate-from-influx/README.md index 461519bb50..ccb78eaefa 100644 --- a/docs/guides/migrate-from-influx/README.md +++ b/docs/guides/migrate-from-influx/README.md @@ -117,7 +117,7 @@ querying and visualizing metrics: ![Migrate from Influx](vmui.webp) -VictoriaMetrics can be configured as [Grafana datasource](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#grafana-setup). +VictoriaMetrics can be configured as [Grafana datasource](https://docs.victoriametrics.com/victoriametrics/integrations/grafana). See more about [how to query data in VictoriaMetrics](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#query-data). ### Basic concepts diff --git a/docs/guides/understand-your-setup-size/README.md b/docs/guides/understand-your-setup-size/README.md index 1bac192c4b..00ddb8a314 100644 --- a/docs/guides/understand-your-setup-size/README.md +++ b/docs/guides/understand-your-setup-size/README.md @@ -100,7 +100,7 @@ It is hard to predict the amount of compute resources (CPU, Mem) or cluster size Active Time Series. The much better approach is to run tests for your type of load (ingestion and reads) and extrapolate from there. -For example, if you already run [Prometheus](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-setup) +For example, if you already run [Prometheus](https://docs.victoriametrics.com/victoriametrics/integrations/prometheus) or [Telegraf](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-send-data-from-influxdb-compatible-agents-such-as-telegraf) for metrics collection then just configure them (or a part of them) to replicate data to VictoriaMetrics. In this way, you'd have the most precise simulation of your production environment. diff --git a/docs/victoriametrics/FAQ.md b/docs/victoriametrics/FAQ.md index db62f9f8b0..11d9371d92 100644 --- a/docs/victoriametrics/FAQ.md +++ b/docs/victoriametrics/FAQ.md @@ -47,7 +47,7 @@ Yes in most cases. VictoriaMetrics can substitute Prometheus in the following as * Prometheus-compatible service discovery and target scraping can be done with [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) and with single-node VictoriaMetrics. See [these docs](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-scrape-prometheus-exporters-such-as-node-exporter). * Prometheus-compatible alerting rules and recording rules can be processed with [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/). -* Prometheus-compatible querying in Grafana is supported by VictoriaMetrics. See [these docs](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#grafana-setup). +* Prometheus-compatible querying in Grafana is supported by VictoriaMetrics. See [these docs](https://docs.victoriametrics.com/victoriametrics/integrations/grafana). ## What is the difference between vmagent and Prometheus? @@ -121,7 +121,7 @@ VictoriaMetrics also [uses less RAM than Thanos components](https://github.com/t * QuestDB is much harder to set up and operate than VictoriaMetrics. Compare [setup instructions for QuestDB](https://questdb.io/docs/get-started/binaries) to [setup instructions for VictoriaMetrics](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-start-victoriametrics). * VictoriaMetrics provides the [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/) query language, which is better suited for typical queries over time series data than the SQL-like query language provided by QuestDB. See [this article](https://valyala.medium.com/promql-tutorial-for-beginners-9ab455142085) for details. * VictoriaMetrics can be queried via the [Prometheus querying API](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-querying-api-usage) and via [Graphite's API](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#graphite-api-usage). -* Thanks to PromQL support, VictoriaMetrics [can be used as a drop-in replacement for Prometheus in Grafana](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#grafana-setup), while QuestDB needs a full rewrite of existing dashboards in Grafana. +* Thanks to PromQL support, VictoriaMetrics [can be used as a drop-in replacement for Prometheus in Grafana](https://docs.victoriametrics.com/victoriametrics/integrations/grafana), while QuestDB needs a full rewrite of existing dashboards in Grafana. * Thanks to Prometheus' remote_write API support, VictoriaMetrics can be used as a long-term storage for Prometheus or for [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/), while QuestDB has no integration with Prometheus. * QuestDB [supports a smaller range of popular data ingestion protocols](https://questdb.io/docs/develop/insert-data) compared to VictoriaMetrics (compare to [the list of supported data ingestion protocols for VictoriaMetrics](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-import-time-series-data)). * [VictoriaMetrics supports backfilling (e.g. storing historical data) out of the box](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#backfilling), while QuestDB provides [very limited support for backfilling](https://questdb.io/blog/2021/05/10/questdb-release-6-0-tsbs-benchmark#the-problem-with-out-of-order-data). @@ -236,7 +236,7 @@ This is slow and expensive. Prometheus' remote read API isn't intended for querying foreign data – aka `global query view`. See [this issue](https://github.com/prometheus/prometheus/issues/4456) for details. So just query VictoriaMetrics directly via [vmui](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui), the [Prometheus Querying API](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-querying-api-usage) -or via [Prometheus datasource in Grafana](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#grafana-setup). +or via [Prometheus datasource in Grafana](https://docs.victoriametrics.com/victoriametrics/integrations/grafana). ## Does VictoriaMetrics deduplicate data from Prometheus instances scraping the same targets (aka `HA pairs`)? diff --git a/docs/victoriametrics/Quick-Start.md b/docs/victoriametrics/Quick-Start.md index 7398898bef..4a9a1f81fb 100644 --- a/docs/victoriametrics/Quick-Start.md +++ b/docs/victoriametrics/Quick-Start.md @@ -40,8 +40,8 @@ VictoriaMetrics is available as: * [VictoriaMetrics on DigitalOcean](https://marketplace.digitalocean.com/apps/victoriametrics-single) Just download VictoriaMetrics and follow [these instructions](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-start-victoriametrics). -Then read [Prometheus setup](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-setup) -and [Grafana setup](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#grafana-setup) docs. +Then read [Prometheus](https://docs.victoriametrics.com/victoriametrics/integrations/prometheus) +and [Grafana](https://docs.victoriametrics.com/victoriametrics/integrations/grafana) setup docs. VictoriaMetrics is developed at a fast pace, so it is recommended periodically checking the [CHANGELOG](https://docs.victoriametrics.com/victoriametrics/changelog/) and performing [regular upgrades](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-upgrade-victoriametrics). @@ -371,7 +371,7 @@ User Interface for querying and visualizing metrics. [MetricsQL](https://docs.vi query language for executing read queries in VictoriaMetrics. See examples of MetricsQL queries [here](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#metricsql). VictoriaMetrics provides an [HTTP API](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-querying-api-usage) -for serving read queries. The API is used in various integrations such as [Grafana](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#grafana-setup). +for serving read queries. The API is used in various integrations such as [Grafana](https://docs.victoriametrics.com/victoriametrics/integrations/grafana). See more details on [querying data here](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#query-data). diff --git a/docs/victoriametrics/README.md b/docs/victoriametrics/README.md index 8adc4b14bc..4cb5f487ba 100644 --- a/docs/victoriametrics/README.md +++ b/docs/victoriametrics/README.md @@ -31,7 +31,7 @@ We apply strict security measures in everything we do. See [Security page](https VictoriaMetrics has the following prominent features: -* It can be used as long-term storage for Prometheus. See [these docs](#prometheus-setup) for details. +* It can be used as long-term storage for Prometheus. See [these docs](https://docs.victoriametrics.com/victoriametrics/integrations/prometheus) for details. * It can be used as a drop-in replacement for Prometheus in Grafana, because it supports the [Prometheus querying API](#prometheus-querying-api-usage). * It can be used as a drop-in replacement for Graphite in Grafana, because it supports the [Graphite API](#graphite-api-usage). VictoriaMetrics allows reducing infrastructure costs by more than 10x comparing to Graphite - see [this case study](https://docs.victoriametrics.com/victoriametrics/casestudies/#grammarly). @@ -68,7 +68,7 @@ VictoriaMetrics has the following prominent features: [the storage architecture](https://medium.com/@valyala/how-victoriametrics-makes-instant-snapshots-for-multi-terabyte-time-series-data-e1f3fb0e0282). * It supports metrics scraping, ingestion and [backfilling](#backfilling) via the following protocols: * [Metrics scraping from Prometheus exporters](#how-to-scrape-prometheus-exporters-such-as-node-exporter). - * [Prometheus remote write API](#prometheus-setup). + * [Prometheus remote write API](https://docs.victoriametrics.com/victoriametrics/integrations/prometheus). * [Prometheus exposition format](#how-to-import-data-in-prometheus-exposition-format). * [InfluxDB line protocol](#how-to-send-data-from-influxdb-compatible-agents-such-as-telegraf) over HTTP, TCP and UDP. * [Graphite plaintext protocol](#how-to-send-data-from-graphite-compatible-agents-such-as-statsd) with [tags](https://graphite.readthedocs.io/en/latest/tags.html#carbon). @@ -137,8 +137,8 @@ Other flags have good enough default values, so set them only if you really need The following docs may be useful during initial VictoriaMetrics setup: * [How to set up scraping of Prometheus-compatible targets](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-scrape-prometheus-exporters-such-as-node-exporter) * [How to ingest data to VictoriaMetrics](#how-to-import-time-series-data) -* [How to set up Prometheus to write data to VictoriaMetrics](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-setup) -* [How to query VictoriaMetrics via Grafana](#grafana-setup) +* [How to set up Prometheus to write data to VictoriaMetrics](https://docs.victoriametrics.com/victoriametrics/integrations/prometheus) +* [How to query VictoriaMetrics via Grafana](https://docs.victoriametrics.com/victoriametrics/integrations/grafana) * [How to query VictoriaMetrics via Graphite API](#graphite-api-usage) * [How to handle alerts](#alerting) @@ -245,83 +245,11 @@ It can be used as an example for a [quick start](https://docs.victoriametrics.co ## Prometheus setup -Add the following lines to Prometheus config file (it is usually located at `/etc/prometheus/prometheus.yml`) in order to send data to VictoriaMetrics: -```yaml -remote_write: - - url: http://:8428/api/v1/write -``` - -Substitute `` with hostname or IP address of VictoriaMetrics. -Then apply new config via the following command: -```sh -kill -HUP `pidof prometheus` -``` - -Prometheus writes incoming data to local storage and replicates it to remote storage in parallel. -This means that data remains available in local storage for `--storage.tsdb.retention.time` duration -even if remote storage is unavailable. - -If you plan sending data to VictoriaMetrics from multiple Prometheus instances, then add the following lines into `global` section -of [Prometheus config](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#configuration-file): - -```yaml -global: - external_labels: - datacenter: dc-123 -``` - -This instructs Prometheus to add `datacenter=dc-123` label to each sample before sending it to remote storage. -The label name can be arbitrary - `datacenter` is just an example. The label value must be unique -across Prometheus instances, so time series could be filtered and grouped by this label. - -For highly loaded Prometheus instances (200k+ samples per second) the following tuning may be applied: -```yaml -remote_write: - - url: http://:8428/api/v1/write - queue_config: - max_samples_per_send: 10000 - capacity: 20000 - max_shards: 30 -``` - -Using remote write increases memory usage for Prometheus by up to ~25%. If you are experiencing issues with -too high memory consumption of Prometheus, then try to lower `max_samples_per_send` and `capacity` params. -Keep in mind that these two params are tightly connected. -Read more about tuning remote write for Prometheus [here](https://prometheus.io/docs/practices/remote_write). - -It is recommended upgrading Prometheus to [v2.12.0](https://github.com/prometheus/prometheus/releases/latest) or newer, -since previous versions may have issues with `remote_write`. - -Take a look also at [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) -and [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/), -which can be used as faster and less resource-hungry alternative to Prometheus. +Moved to [integrations/prometheus](https://docs.victoriametrics.com/victoriametrics/integrations/prometheus). ## Grafana setup -Create [Prometheus datasource](https://grafana.com/docs/grafana/latest/datasources/prometheus/configure-prometheus-data-source/) -in Grafana with the following url: -```url -http://:8428 -``` - -Substitute `` with the hostname or IP address of VictoriaMetrics. - -In the "Type and version" section it is recommended to set the type to "Prometheus" and the version to at least "2.24.x": - -![Datasource](grafana-datasource-prometheus.webp) - -This allows Grafana to use a more efficient API to get label values. - -Then build graphs and dashboards for the created datasource using [PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics/) -or [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/). - -Alternatively, use VictoriaMetrics [datasource plugin](https://github.com/VictoriaMetrics/victoriametrics-datasource) with support of extra features. -See more in [description](https://docs.victoriametrics.com/victoriametrics/victoriametrics-datasource/). - -Creating a datasource may require [specific permissions](https://grafana.com/docs/grafana/latest/administration/data-source-management/). -If you don't see an option to create a data source - try contacting system administrator. - -Grafana playground is available for viewing at our [sandbox](https://play-grafana.victoriametrics.com). +Moved to [integrations/grafana](https://docs.victoriametrics.com/victoriametrics/integrations/grafana). ## How to upgrade VictoriaMetrics @@ -1367,7 +1295,7 @@ VictoriaMetrics can discover and scrape metrics from Prometheus-compatible targe see [these docs](#how-to-scrape-prometheus-exporters-such-as-node-exporter). Additionally, VictoriaMetrics can accept metrics via the following popular data ingestion protocols (aka "push" protocols): -* [Prometheus remote_write API](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write). See [these docs](#prometheus-setup) for details. +* [Prometheus remote_write API](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write). See [these docs](https://docs.victoriametrics.com/victoriametrics/integrations/prometheus) for details. * DataDog `submit metrics` API. See [these docs](#how-to-send-data-from-datadog-agent) for details. * InfluxDB line protocol. See [these docs](#how-to-send-data-from-influxdb-compatible-agents-such-as-telegraf) for details. * Graphite plaintext protocol. See [these docs](#how-to-send-data-from-graphite-compatible-agents-such-as-statsd) for details. diff --git a/docs/victoriametrics/changelog/CHANGELOG_2023.md b/docs/victoriametrics/changelog/CHANGELOG_2023.md index 9ea7a4e316..2481a99fda 100644 --- a/docs/victoriametrics/changelog/CHANGELOG_2023.md +++ b/docs/victoriametrics/changelog/CHANGELOG_2023.md @@ -47,7 +47,7 @@ Released at 2023-12-13 * `go_scavenge_cpu_seconds_total` - the [counter](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#counter), which shows the total CPU time spent by Go runtime for returning memory to the Operating System. * `go_memlimit_bytes` - the value of [GOMEMLIMIT](https://pkg.go.dev/runtime#hdr-Environment_Variables) environment variable. * FEATURE: [vmui](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui): enhance autocomplete functionality with caching. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5348). -* FEATURE: add field `version` to the response for `/api/v1/status/buildinfo` API for using more efficient API in Grafana for receiving label values. Add additional info about setup Grafana datasource. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5370) and [these docs](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#grafana-setup) for details. +* FEATURE: add field `version` to the response for `/api/v1/status/buildinfo` API for using more efficient API in Grafana for receiving label values. Add additional info about setup Grafana datasource. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5370) and [these docs](https://docs.victoriametrics.com/victoriametrics/integrations/grafana) for details. * FEATURE: add `-search.maxResponseSeries` command-line flag for limiting the number of time series a single query to [`/api/v1/query`](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#instant-query) or [`/api/v1/query_range`](https://docs.victoriametrics.com/victoriametrics/keyconcepts/#range-query) can return. This limit can protect Grafana from high memory usage when the query returns too many series. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5372). * FEATURE: [Alerting rules for VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker#alerts): ease aggregation for certain alerting rules to keep more useful labels for the context. Before, all extra labels except `job` and `instance` were ignored. See this [pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5429) and this [follow-up commit](https://github.com/VictoriaMetrics/VictoriaMetrics/commit/8fb68152e67712ed2c16dcfccf7cf4d0af140835). Thanks to @7840vz. * FEATURE: [vmctl](https://docs.victoriametrics.com/victoriametrics/vmctl/): allow reversing the migrating order from the newest to the oldest data for [vm-native](https://docs.victoriametrics.com/victoriametrics/vmctl/#migrating-data-from-victoriametrics) and [remote-read](https://docs.victoriametrics.com/victoriametrics/vmctl/#migrating-data-by-remote-read-protocol) modes via `--vm-native-filter-time-reverse` and `--remote-read-filter-time-reverse` command-line flags respectively. See: https://docs.victoriametrics.com/victoriametrics/vmctl/#using-time-based-chunking-of-migration and [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5376). diff --git a/docs/victoriametrics/data-ingestion/Prometheus.md b/docs/victoriametrics/data-ingestion/Prometheus.md index 5cc2c85561..6df9a61536 100644 --- a/docs/victoriametrics/data-ingestion/Prometheus.md +++ b/docs/victoriametrics/data-ingestion/Prometheus.md @@ -16,7 +16,7 @@ aliases: --- VictoriaMetrics supports ingesting data from Prometheus via the Prometheus remote write protocol. -The documentation for pushing data from Prometheus to VictoriaMetrics is located [here](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-setup) +The documentation for pushing data from Prometheus to VictoriaMetrics is located [here](https://docs.victoriametrics.com/victoriametrics/integrations/prometheus) ## References diff --git a/docs/victoriametrics/integrations/_index.md b/docs/victoriametrics/integrations/_index.md new file mode 100644 index 0000000000..1fe437f520 --- /dev/null +++ b/docs/victoriametrics/integrations/_index.md @@ -0,0 +1,16 @@ +--- +title: Integrations +weight: 11 +menu: + docs: + parent: 'victoriametrics' + identifier: 'integrations-vm' + weight: 11 +--- + +VictoriaMetrics integrates with many popular monitoring solutions as remote storage or datasource: + +* [Grafana](https://docs.victoriametrics.com/victoriametrics/integrations/grafana) +* [Prometheus](https://docs.victoriametrics.com/victoriametrics/integrations/prometheus) + +If you think that community will benefit from new integrations, open a [feature request on GitHub](https://github.com/VictoriaMetrics/VictoriaMetrics/issues). \ No newline at end of file diff --git a/docs/victoriametrics/grafana-datasource-prometheus.webp b/docs/victoriametrics/integrations/grafana-datasource-prometheus.webp similarity index 100% rename from docs/victoriametrics/grafana-datasource-prometheus.webp rename to docs/victoriametrics/integrations/grafana-datasource-prometheus.webp diff --git a/docs/victoriametrics/integrations/grafana.md b/docs/victoriametrics/integrations/grafana.md new file mode 100644 index 0000000000..c2c90b0cad --- /dev/null +++ b/docs/victoriametrics/integrations/grafana.md @@ -0,0 +1,60 @@ +--- +title: Grafana +weight: 1 +menu: + docs: + parent: "integrations-vm" + identifier: "integrations-grafana-vm" + weight: 1 +--- + +VictoriaMetrics integrates with Grafana using either [Prometheus datasource](https://grafana.com/docs/grafana/latest/datasources/prometheus/) +or [VictoriaMetrics datasource](https://grafana.com/grafana/plugins/victoriametrics-metrics-datasource/) plugins. + +Resources: +* [VictoriaMetrics Grafana playground](https://play-grafana.victoriametrics.com) +* [VictoriaMetrics and Grafana in docker-compose environment](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker#docker-compose-environment-for-victoriametrics) + +## VictoriaMetrics datasource + +Create [VictoriaMetrics datasource](https://grafana.com/grafana/plugins/victoriametrics-metrics-datasource/) +in Grafana with the following URL for single-server: +``` + +http://:8428 +``` +_Replace `` with the VictoriaMetrics hostname or IP address._ + +For the cluster version, use `vmselect` address: +``` +http://:8481/select//prometheus +``` +_Replace `` with the hostname or IP address of vmselect service._ + +If you have more than 1 vmselect, configure [load-balancing](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#cluster-setup). +Replace `` based on your [multitenancy settings](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy). + +Once connected, you can start building graphs and dashboards using [PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics/) +or [MetricsQL](https://docs.victoriametrics.com/metricsql/). + +VictoriaMetrics datasource is publicly available on [GitHub](https://github.com/VictoriaMetrics/victoriametrics-datasource). +See more in [plugin docs](https://docs.victoriametrics.com/victoriametrics-datasource/). + +_Creating a datasource may require [specific permissions](https://grafana.com/docs/grafana/latest/administration/data-source-management/). +If you don't see an option to create a data source - try contacting system administrator._ + + +## Prometheus datasource + +Create [Prometheus datasource](https://grafana.com/docs/grafana/latest/datasources/prometheus/configure-prometheus-data-source/) +in Grafana. Follow the same connection instructions as for [VictoriaMetrics datasource](#VictoriaMetrics-datasource). + +In the "Type and version" section set the type to "Prometheus" and the version to at least "2.24.x". +This allows Grafana to use a more efficient API to get label values: + +![Datasource](grafana-datasource-prometheus.webp) + +Once connected, you can build graphs and dashboards using [PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics/). + +_Creating a datasource may require [specific permissions](https://grafana.com/docs/grafana/latest/administration/data-source-management/). +If you don't see an option to create a data source - try contacting system administrator._ diff --git a/docs/victoriametrics/integrations/prometheus.md b/docs/victoriametrics/integrations/prometheus.md new file mode 100644 index 0000000000..da103ace70 --- /dev/null +++ b/docs/victoriametrics/integrations/prometheus.md @@ -0,0 +1,66 @@ +--- +title: Prometheus +weight: 2 +menu: + docs: + parent: "integrations-vm" + weight: 2 +--- + +VictoriaMetrics integrates with Prometheus as [remote storage for writes](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage). + +To send data from Prometheus to VictoriaMetrics, add the following lines to your Prometheus config file: +```yaml +remote_write: + - url: http://:8428/api/v1/write +``` +_Replace `` with the VictoriaMetrics hostname or IP address._ + +For cluster version use vminsert address: +``` +http://:8480/insert//prometheus +``` +_Replace `` with the hostname or IP address of vminsert service._ + +If you have more than 1 vminsert, configure [load-balancing](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#cluster-setup). +Replace `` based on your [multitenancy settings](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy). + +Then apply the new config restart Prometheus or [hot-reload](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#configuration) it: +```sh +kill -HUP `pidof prometheus` +``` + +Prometheus stores incoming data locally and also sends a copy to the remote storage. +This means even if the remote storage is down, data is still available locally for as long as set in `--storage.tsdb.retention.time`. + +If you send data from more than one Prometheus instance, add the following lines into `global` section +of [Prometheus config](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#configuration-file): +```yaml +global: + external_labels: + datacenter: dc-123 +``` + +This adds a label `datacenter=dc-123` to each sample sent to remote storage. +You can use any label name; datacenter is just an example. The value should be different for each Prometheus instance +so you can filter and group time series by it. + +For Prometheus servers handling a high load (200k+ samples per second), use these settings: +```yaml +remote_write: + - url: http://:8428/api/v1/write + queue_config: + max_samples_per_send: 10000 + capacity: 20000 + max_shards: 30 +``` + +Using remote write can increase Prometheus memory use by around 25%. If memory usage is too high, try reducing +`max_samples_per_send` and `capacity` params. These two settings work closely together, so adjust carefully. +Read more about tuning [remote write](https://prometheus.io/docs/practices/remote_write) for Prometheus. + +It is recommended upgrading Prometheus to [v2.12.0](https://github.com/prometheus/prometheus/releases/latest) or newer, +since previous versions may have issues with `remote_write`. + +Take a look at [vmagent](https://docs.victoriametrics.com/vmagent/) and [vmalert](https://docs.victoriametrics.com/vmalert/), +which can be used as faster and less resource-hungry alternative to Prometheus. \ No newline at end of file diff --git a/docs/victoriametrics/keyConcepts/keyConcepts.md b/docs/victoriametrics/keyConcepts/keyConcepts.md index fe772b3644..7308ebf5dd 100644 --- a/docs/victoriametrics/keyConcepts/keyConcepts.md +++ b/docs/victoriametrics/keyConcepts/keyConcepts.md @@ -45,7 +45,7 @@ requests_total{path="/", code="200"} Labels can be automatically attached to the [time series](#time-series) written via [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/#adding-labels-to-metrics) -or [Prometheus](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-setup). +or [Prometheus](https://docs.victoriametrics.com/victoriametrics/integrations/prometheus). VictoriaMetrics supports enforcing of label filters for [query API](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-querying-api-enhancements) to emulate data isolation. However, the real data isolation can be achieved via [multi-tenancy](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy). @@ -464,7 +464,7 @@ or [VictoriaMetrics Cluster](https://docs.victoriametrics.com/victoriametrics/cl VictoriaMetrics provides an [HTTP API](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-querying-api-usage) for serving read queries. The API is used in various integrations such as -[Grafana](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#grafana-setup). The same API is also used by +[Grafana](https://docs.victoriametrics.com/victoriametrics/integrations/grafana). The same API is also used by [VMUI](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui) - a graphical User Interface for querying and visualizing metrics. @@ -949,7 +949,7 @@ Open `http://victoriametrics:8428/vmui` page, type the query and see the results ![vmui](vmui.webp) VictoriaMetrics supports [Prometheus HTTP API](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#prometheus-querying-api-usage) -which makes it possible to [query it with Grafana](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#grafana-setup) +which makes it possible to [query it with Grafana](https://docs.victoriametrics.com/victoriametrics/integrations/grafana) in the same way as Grafana queries Prometheus. ## Modify data