mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2026-07-22 00:30:22 +03:00
c82127b6d4
## Description We have a simple vmagent setup using `remotewrite.shardByURL` to horizontally scale stream aggregations, as suggested [here](https://docs.victoriametrics.com/victoriametrics/vmagent/#sharding-among-remote-storages) with minimal additional configuration and noticed there was some optimization opportunities. ## Investigation We took a [pprof](https://github.com/user-attachments/files/28968177/cpu.pprof.zip) and found that our CPU time was in a few primary places. 1. payload protobuf marshaling 2. zstd compresssion 3. Map access in sharding 4. Hashing in sharding 5. Memory copies for timeseries ## Changes Zstd compression is pretty intractable(besides using `remotewrite.vmProtoCompressLevel`, which we already do), but there were some relatively easy gains for the other paths. 1. inline sov() when l < 128 on marshal. Skips calling a bunch of byte math in a common case 2. Use slices instead of maps for shard keys, in the common case where there are less than ~10 keys a slice iteration will be faster 3. use `xxhash.Digest.WriteString` rather than buffering values, avoids some extra allocations and GC work 4. In the simple case where `insertRows` doesn't modify label names or values construct the remotewrite request from the passed values instead of copying them. Avoids a bunch of memory copy and GC work. ## Results Modest but effective, we saw a ~10% reduction in total cpu cores used by vmagent with these changes. <img width="2358" height="709" alt="Screenshot 2026-06-15 at 2 24 43 PM" src="https://github.com/user-attachments/assets/d2931242-c7ed-447b-9cfd-73b29cd0b893" /> Related PR https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11113