app/vmctl: set MaxIdleConnsPerHost equal to concurrency value

Default value for MaxIdleConnsPerHost is `2`.
If `vm-concurrency` is >2, then vmctl will suboptimally re-establish
connections over and over. Which could have significant impact when
importing data via big number of small HTTP requests, like in per-series mode.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

app/vmctl/main.go

@@ -421,6 +421,11 @@ func main() {
 
 					disableKeepAlive := c.Bool(vmNativeDisableHTTPKeepAlive)
 
+					cc := c.Int(vmConcurrency)
+					if cc <= 0 {
+						cc = 1
+					}
+
 					var srcExtraLabels []string
 					srcAddr := strings.Trim(c.String(vmNativeSrcAddr), "/")
 					srcAuthConfig, err := auth.Generate(
@@ -446,6 +451,8 @@ func main() {
 					trSrc := httputil.NewTransport(false, "vmctl_src")
 					trSrc.DisableKeepAlives = disableKeepAlive
 					trSrc.TLSClientConfig = srcTC
+					// Keep an idle connection per worker to reduce connections churn.
+					trSrc.MaxIdleConnsPerHost = cc
 
 					srcHTTPClient := &http.Client{
 						Transport: trSrc,
@@ -476,6 +483,8 @@ func main() {
 					trDst := httputil.NewTransport(false, "vmctl_dst")
 					trDst.DisableKeepAlives = disableKeepAlive
 					trDst.TLSClientConfig = dstTC
+					// Keep an idle connection per worker to reduce connections churn.
+					trDst.MaxIdleConnsPerHost = cc
 
 					dstHTTPClient := &http.Client{
 						Transport: trDst,
@@ -504,7 +513,7 @@ func main() {
 							HTTPClient:  dstHTTPClient,
 						},
 						backoff:                  bf,
-						cc:                       c.Int(vmConcurrency),
+						cc:                       cc,
 						disablePerMetricRequests: c.Bool(vmNativeDisablePerMetricMigration),
 						isNative:                 !c.Bool(vmNativeDisableBinaryProtocol),
 					}

docs/victoriametrics/changelog/CHANGELOG.md

@@ -31,6 +31,7 @@ See also [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-rel
 * BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) and `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): fix infinite loop in the OpenTelemetry Firehose ingestion endpoint (`/opentelemetry/api/v1/push`) when receiving a malformed record with an incomplete varint in the `data` field. Previously this caused the goroutine to spin forever, permanently consuming CPU until the process was restarted.
 * BUGFIX: [vmalert-tool](https://docs.victoriametrics.com/victoriametrics/vmalert-tool/): reuse connections to `-remoteWrite.url` when writing the results of recording rules and alerts. Previously every series was sent over a new connection, which left a lot of sockets in `TIME_WAIT` state and could exhaust the ephemeral port range. The number of idle connections can be tuned via the new `-remoteWrite.maxIdleConnections` command-line flag. Thanks @evkuzin for contribution.
 * BUGFIX: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): prevent process crash in `sort_by_label_numeric()` and `sort_by_label_numeric_desc()` when a label value contains a number with 309 or more digits. See [#11423](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11423).
+* BUGFIX: [vmctl](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui): reuse connections in [vm-native mode](https://docs.victoriametrics.com/victoriametrics/vmctl/#migrating-data-from-victoriametrics) when `--vm-concurrency` exceeds 2. Previously the number of idle connections was limited to 2 per host, which was insufficient when `--vm-concurrency` was bigger than 2.
 
 ## [v1.150.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.150.0)