mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2026-05-17 08:36:55 +03:00
db69fc686a
Previously, creation of compressedLabels could require extra memory due
to re-allocation of tmpBuf and clone of 3 extra fields. It could result
into extra CPU usage for garbage-collection.
This commit adds sync.Pool for labels escape with JSON marshal and
allocates dedicated buffer for job, address and ID strings.
Optimisations was made based on the following profiles from reported
issue:
1) CPU:
```
Showing top 10 nodes out of 172
flat flat% sum% cum cum%
12.17s 17.19% 17.19% 12.25s 17.30% runtime.cgocall
5.87s 8.29% 25.48% 5.87s 8.29% runtime.memmove
3.45s 4.87% 30.35% 6.66s 9.41% runtime.tryDeferToSpanScan
```
2) memory go tool pprof -alloc_objects heap_profile.txt
```
Showing top 10 nodes out of 94
flat flat% sum% cum cum%
3673568660 26.09% 26.09% 4147984949 29.46% github.com/valyala/quicktemplate.AppendJSONString
1657933055 11.77% 37.86% 1657933055 11.77% internal/stringslite.Clone (inline)
1555166274 11.04% 48.91% 1555166274 11.04% github.com/valyala/gozstd.compress
1254756359 8.91% 57.82% 9433313305 66.99% github.com/VictoriaMetrics/VictoriaMetrics/lib/promscrape.newCompressedLabels
1067036870 7.58% 65.39% 1067036870 7.58% github.com/VictoriaMetrics/VictoriaMetrics/lib/promscrape.appendExtraLabels
```
results of benchstat:
```
benchstat before after
goos: darwin
goarch: arm64
pkg: github.com/VictoriaMetrics/VictoriaMetrics/lib/promscrape
cpu: Apple M1 Pro
│ 134/before │ after │
│ sec/op │ sec/op vs base │
NewCompressedLabels-10 981.3n ± 2% 908.6n ± 2% -7.40% (p=0.000 n=10)
│ 134/before │ after │
│ B/op │ B/op vs base │
NewCompressedLabels-10 891.5 ± 0% 772.0 ± 0% -13.40% (p=0.000 n=10)
│ 134/before │ after │
│ allocs/op │ allocs/op vs base │
NewCompressedLabels-10 10.000 ± 0% 3.000 ± 0% -70.00% (p=0.000 n=10)
```
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10919
VictoriaMetrics
VictoriaMetrics is a fast, cost-effective, and scalable solution for monitoring and managing time series data. It delivers high performance and reliability, making it an ideal choice for businesses of all sizes.
Here are some resources and information about VictoriaMetrics:
- Case studies: Grammarly, Roblox, Wix, Spotify,....
- Available: Binary releases, Docker images on Docker Hub and Quay, Source code.
- Deployment types: Single-node version and Cluster version under Apache License 2.0.
- Getting started: Read key concepts and follow the quick start guide.
- Community: Slack (join via Slack Inviter), X (Twitter), YouTube. See full list here.
- Changelog: Project evolves fast - check the CHANGELOG, and How to upgrade.
- Enterprise support: Contact us for commercial support with additional enterprise features.
- Enterprise releases: Enterprise and long-term support releases (LTS) are publicly available and can be evaluated for free using a free trial license.
- Security: we achieved security certifications for Database Software Development and Software-Based Monitoring Services.
Yes, we open-source both the single-node VictoriaMetrics and the cluster version.
Prominent features
VictoriaMetrics is optimized for timeseries data, even when old time series are constantly replaced by new ones at a high rate, it offers a lot of features:
- Long-term storage for Prometheus or as a drop-in replacement for Prometheus and Graphite in Grafana.
- Powerful stream aggregation: Can be used as a StatsD alternative.
- Ideal for big data: Works well with large amounts of time series data from APM, Kubernetes, IoT sensors, connected cars, industrial telemetry, financial data and various Enterprise workloads.
- Query language: Supports both PromQL and the more performant MetricsQL.
- Easy to setup: No dependencies, single small binary, configuration through command-line flags, but the default is also fine-tuned; backup and restore with instant snapshots.
- Global query view: Multiple Prometheus instances or any other data sources may ingest data into VictoriaMetrics and queried via a single query.
- Various Protocols: Support metric scraping, ingestion and backfilling in various protocol.
- Prometheus exporters, Prometheus remote write API, Prometheus exposition format.
- InfluxDB line protocol over HTTP, TCP and UDP.
- Graphite plaintext protocol with tags.
- OpenTSDB put message.
- HTTP OpenTSDB /api/put requests.
- JSON line format.
- Arbitrary CSV data.
- Native binary format.
- DataDog agent or DogStatsD.
- NewRelic infrastructure agent.
- OpenTelemetry metrics format.
- NFS-based storages: Supports storing data on NFS-based storages such as Amazon EFS, Google Filestore.
- And many other features such as metrics relabeling, cardinality limiter, etc.
Enterprise version
In addition, the Enterprise version includes extra features:
- Anomaly detection: Automation and simplification of your alerting rules, covering complex anomalies found in metrics data.
- Backup automation: Automates regular backup procedures.
- Multiple retentions: Reducing storage costs by specifying different retentions for different datasets.
- Downsampling: Reducing storage costs and increasing performance for queries over historical data.
- Stable releases with long-term support lines (LTS).
- Comprehensive support: First-class consulting, feature requests and technical support provided by the core VictoriaMetrics dev team.
- Many other features, which you can read about on the Enterprise page.
Contact us if you need enterprise support for VictoriaMetrics. Or you can request a free trial license here, downloaded Enterprise binaries are available at Github Releases.
We strictly apply security measures in everything we do. VictoriaMetrics has achieved security certifications for Database Software Development and Software-Based Monitoring Services. See Security page for more details.
Benchmarks
Some good benchmarks VictoriaMetrics achieved:
- Minimal memory footprint: handling millions of unique timeseries with 10x less RAM than InfluxDB, up to 7x less RAM than Prometheus, Thanos or Cortex.
- Highly scalable and performance for data ingestion and querying, 20x outperforms InfluxDB and TimescaleDB.
- High data compression: 70x more data points may be stored into limited storage than TimescaleDB, 7x less storage space is required than Prometheus, Thanos or Cortex.
- Reducing storage costs: 10x more effective than Graphite according to the Grammarly case study.
- A single-node VictoriaMetrics can replace medium-sized clusters built with competing solutions such as Thanos, M3DB, Cortex, InfluxDB or TimescaleDB. See VictoriaMetrics vs Thanos, Measuring vertical scalability, Remote write storage wars - PromCon 2019.
- Optimized for storage: Works well with high-latency IO and low IOPS (HDD and network storage in AWS, Google Cloud, Microsoft Azure, etc.).
Community and contributions
Feel free asking any questions regarding VictoriaMetrics:
If you like VictoriaMetrics and want to contribute, then please read these docs.
VictoriaMetrics Logo
The provided ZIP file contains three folders with different logo orientations. Each folder includes the following file types:
- JPEG: Preview files
- PNG: Preview files with transparent background
- AI: Adobe Illustrator files
VictoriaMetrics Logo Usage Guidelines
Font
- Font Used: Lato Black
- Download here: Lato Font
Color Palette
Logo Usage Rules
- Only use the Lato Black font as specified.
- Maintain sufficient clear space around the logo for visibility.
- Do not modify the spacing, alignment, or positioning of design elements.
- You may resize the logo as needed, but ensure all proportions remain intact.
Thank you for your cooperation!