mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2026-06-23 02:28:07 +03:00
5eae13fbe9
### Describe Your Changes The current one-second timeout for individual read or write operations during the handshake phase has proven to be insufficient in some scenarios https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9345. For example, short-lived CPU spikes lasting a few seconds can cause handshake failures due to the low timeout threshold. While a small timeout may work well in environments with fast and reliable networking, such as within a single datacenter, it becomes problematic in more complex setups—particularly in a [multi-level cluster setup](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multi-level-cluster-setup) where the top-level vmselect may reside in a different availability zone and work on a less reliable network. Another issue with the per-operation timeout approach is that it allows the total time for a handshake to accumulate significantly in the worst-case scenario. If each operation experiences a delay just under the timeout threshold, the entire handshake process could take up to 6s. Which accounts for 60% of `-search.maxQueueDuration` and leaves only 4s for the actual query. Introducing a single timeout for the entire handshake process would provide more predictable behavior and improve usability from a configuration standpoint. The timeout for the whole handshake op is also easier to understand from the operator's point of view. Increasing the timeout value and providing a configuration option for it would make the system more resilient to transient conditions like CPU contention and better suited for use cases involving cross-AZ communication. Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9345 ### Checklist The following checks are **mandatory**: - [x] My change adheres to [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist). - [x] My change adheres to [VictoriaMetrics development goals](https://docs.victoriametrics.com/victoriametrics/goals/).