rms/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2026-05-17 00:26:36 +03:00
Code Issues Packages Projects Releases Wiki Activity

Add log which will report dropped log count (#9752)

Browse Source 
### Describe Your Changes

I have added a counter for the throttled logs which gets logged every 1
minute.
Fixes #9498

### Checklist

The following checks are **mandatory**:

- [x] My change adheres to [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/victoriametrics/contributing/#pull-request-checklist).
- [x] My change adheres to [VictoriaMetrics development
goals](https://docs.victoriametrics.com/victoriametrics/goals/).

---------

Co-authored-by: Hui Wang <haley@victoriametrics.com>
This commit is contained in:
Samarth Bagga
2025-11-12 23:02:33 +05:30
committed by GitHub
parent 8dd905c7a9
commit 96c1392b45
2 changed files with 11 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/victoriametrics/changelog/CHANGELOG.md
View File

@@ -61,6 +61,7 @@ Released at 2025-10-31
* FEATURE: [vmauth](https://docs.victoriametrics.com/victoriametrics/vmauth/): make the load distribution more even among the backends which execute queries with various durations. See [#9712](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9712).
* FEATURE: [dashboards/all](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/dashboards): enable column filters in `Non-default flags` panel. See [#9910](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9910). Thanks to @SamarthBagga for enhancement!
* FEATURE: add linux/s390x artifact to releases. See [#9697](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9697) for the details.
FEATURE: Attach additional information to logs when internal throttling occurs. For example, vmagent throttles printing of the OpenTelemetry row parse error to every 5 seconds. See [#9498](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9498). Thanks to @SamarthBagga for the [PR 9752](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/9752).
* BUGFIX: [vmbackup](https://docs.victoriametrics.com/victoriametrics/vmbackup/), [vmrestore](https://docs.victoriametrics.com/victoriametrics/vmrestore/), [vmbackupmanager](https://docs.victoriametrics.com/victoriametrics/vmbackupmanager/): complete a fix of environment variables configuration parsing for connection to AWS S3. Previously, such settings were ignored starting from [v1.115.0](https://docs.victoriametrics.com/victoriametrics/changelog/#v11150) and releases [v1.128.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.128.0), [v1.122.6](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.122.6) and [v1.110.21](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.110.21) did not fix an issue completely. See this issue [#9858](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9858) for details.
* BUGFIX: [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): fix search over group names and other attributes in vmalert's WEB UI. This functionality was broken since [v1.117.0](https://docs.victoriametrics.com/CHANGELOG.html#v11170). See [#9886](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9886) for details.

11
lib/logger/throttler.go
View File

@@ -1,7 +1,9 @@
package logger
import (
"fmt"
"sync"
"sync/atomic"
"time"
)
@@ -32,7 +34,8 @@ func WithThrottler(name string, throttle time.Duration) *LogThrottler {
//
// LogThrottler must be created via WithThrottler() call.
type LogThrottler struct {
ch chan struct{}
ch chan struct{}
dropped atomic.Uint64
}
func newLogThrottler(throttle time.Duration) *LogThrottler {
@@ -52,8 +55,11 @@ func newLogThrottler(throttle time.Duration) *LogThrottler {
func (lt *LogThrottler) Errorf(format string, args ...any) {
select {
case lt.ch <- struct{}{}:
dropped := lt.dropped.Swap(0)
format = fmt.Sprintf("%s. (Similar %d log messages were dropped due to throttling)", format, dropped)
ErrorfSkipframes(1, format, args...)
default:
lt.dropped.Add(1)
}
}
@@ -61,7 +67,10 @@ func (lt *LogThrottler) Errorf(format string, args ...any) {
func (lt *LogThrottler) Warnf(format string, args ...any) {
select {
case lt.ch <- struct{}{}:
dropped := lt.dropped.Swap(0)
format = fmt.Sprintf("%s. (Similar %d log messages were dropped due to throttling)", format, dropped)
WarnfSkipframes(1, format, args...)
default:
lt.dropped.Add(1)
}
}