Compare commits

...

15 Commits

Author SHA1 Message Date
Artem Fetishev
d0c6aa681f vmsingle: rename DataPath variable to storageDataPath (#11018)
This way the variable will match the corresponding name in cluster
branch which will reduce diff between branches.

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2026-05-26 19:17:05 +02:00
Hui Wang
c3525bf0bc lib/protoparser/opentelemetry: support disable scope and resource attributes label promotions
This commit adds four flags to allow managing label promotion for
resource attributes and OTel scope metadata, while staying compatible
with
[Prometheus](https://prometheus.io/docs/prometheus/latest/configuration/configuration/):
- `-opentelemetry.promoteScopeMetadata` - promote OTel scope metadata
(i.e. name, version, schema URL, and attributes) to metric labels.
- `opentelemetry.promoteAllResourceAttributes` - promote all resource
attributes to labels, except for the ones configured with
`-opentelemetry.ignoreResourceAttributes`.
- `opentelemetry.promoteResourceAttributes` - promote specific list of
resource attributes to labels. It cannot be configured simultaneously
with `opentelemetry.promoteAllResourceAttributes`.
- `opentelemetry.ignoreResourceAttributes` - which resource attributes
to ignore, can only be set when
`opentelemetry.promoteAllResourceAttributes` is true.

`-opentelemetry.promoteScopeMetadata` and
`opentelemetry.promoteAllResourceAttributes` are enabled by default in
order to preserve the current behavior.

fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10931.
2026-05-26 17:47:07 +02:00
Artem Fetishev
9e4bfebb74 changelog: revert previous release version
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2026-05-25 15:50:33 +02:00
Artem Fetishev
c2079a7880 docs: update flags with actual v1.144.0 binaries
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2026-05-25 14:12:55 +02:00
Artem Fetishev
3586757707 docs: bump version to v1.144.0
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2026-05-25 14:10:32 +02:00
Artem Fetishev
30cb4e831e deplyoment/docker: bump version to v1.144.0
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2026-05-25 14:09:16 +02:00
Artem Fetishev
ecfae87e4d docs: forward port LTS v1.122.23 changelog to upstream
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2026-05-25 13:32:58 +02:00
Artem Fetishev
bff02a6284 docs: forward port LTS v1.136.10 changelog to upstream
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2026-05-25 13:32:14 +02:00
Artem Fetishev
5e2bdf8220 Bump golang.org/x/net to v0.55.0
This fixes https://pkg.go.dev/vuln/GO-2026-5026

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2026-05-25 12:31:08 +02:00
Artem Fetishev
d82ad68f60 docs/changelog: cut release v1.144.0
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2026-05-22 17:05:50 +02:00
Artem Fetishev
dcbd8ef721 app/vmselect: run make vmui-update
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2026-05-22 17:01:56 +02:00
JAYICE
886c7762eb app/vmui: add timestamp and value column for csv data exported from table view (#10979)
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10975

before:
<img width="342" height="67" alt="image"
src="https://github.com/user-attachments/assets/a972857b-6a6c-4ec3-86fb-371b72a4d6d9"
/>

after:
<img width="426" height="115" alt="image"
src="https://github.com/user-attachments/assets/ddb540ed-5c1b-4431-a5c5-3576292dd21e"
/>

---------

Signed-off-by: JAYICE <1185430411@qq.com>
Co-authored-by: Max Kotliar <mkotlyar@victoriametrics.com>
2026-05-22 15:35:19 +03:00
Roman Khavronenko
d3006b25e6 app/vmui: display null/NaN/staleNaN values on RawQuery chart (#10986)
`null` values can be actual `NaN` or `null` values exposed by the
exporter, or stale markers

https://docs.victoriametrics.com/victoriametrics/vmagent/#prometheus-staleness-markers

 Before, vmui Raw Query was silently dropping non-numeric values.
 Displaying such values on chart could improve debugging experience.

Screenshots: 
<img width="1487" height="833" alt="image"
src="https://github.com/user-attachments/assets/2c80cd52-9d37-41f7-ad73-a48335b8aef2"
/>

Decisions:
1. Since null value can't be mapped on Y axis - it is placed on the very
bottom as `X` sign. It is ok to have it placed on top or as a dotted
line
2. It has a tooltip explaining what it is
3. Its value is not accounted in cumulative min/max/avg tolltip of the
whole time series on the graph
4. Null value can't exactly state it is a stale marker - such a value
can come from many places

Values to import as example dataset:
```
{"metric":{"__name__":"cpu_usage","job":"test","instance":"127.0.0.1:8080","cpu":"1","exported_instance":"foo","exported_job":"foo"},"values":[null,0.2,0.2,null,0.2,null,0.2,0.2,null,0.2,null,0.2,null,0.2,null,0.2,null,0.2,null,0.2,null,0.2,null,0.2,0.2,0.2,0.2,null,0.2,0.2,null,0.2,0.2,null,0.2,0.2,null,0.2,null,0.2,null,0.2,null,0.2,null,0.2,null,0.2,0.2],"timestamps":[1779362323611,1779362323617,1779362333617,1779362343610,1779362343617,1779362353610,1779362353617,1779362363617,1779362373610,1779362373617,1779362383610,1779362383617,1779362393610,1779362393617,1779362403610,1779362403617,1779362413610,1779362413617,1779362423610,1779362423617,1779362433610,1779362433617,1779362443610,1779362443617,1779362453617,1779362463617,1779362473617,1779362483610,1779362483617,1779362493617,1779362503610,1779362503617,1779362513617,1779362523610,1779362523617,1779362533617,1779362543609,1779362543617,1779362553609,1779362553617,1779362563609,1779362563617,1779362573609,1779362573617,1779362583609,1779362583617,1779362593609,1779362593617,1779362603617]}
```
------------

Feature like that would significantly help in debugging issues like
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10893

@Loori-R I am sorry, but this PR is rather a PoC - it was mostly
generated with AI help.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Max Kotliar <mkotlyar@victoriametrics.com>
2026-05-22 15:29:31 +03:00
andriibeee
c41e967ee1 vmstorage: vmselect RPC server must respect -futureRetention (#10881)
Flag `-denyQueriesOutsideRetention` now also rejects queries past future
retention limit. Default behavior is unchanged.

Note that HTTP error code has been changed from 503 to 400.

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10879.

---------

Signed-off-by: Andrii Beskomornyi <andriibeee@gmail.com>
Co-authored-by: Artem Fetishev <rtm@victoriametrics.com>
2026-05-22 13:00:21 +02:00
andriibeee
0c9a011e0a lib/timeutil: handle int64 overflow in relative duration parsing (#10883)
Fix `int64` overflow in `ParseTimeAt()` in `lib/timeutil/time.go` when
converting time expressed as string into nanoseconds.

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10880.

---------

Signed-off-by: Andrii Beskomornyi <andriibeee@gmail.com>
Co-authored-by: Artem Fetishev <rtm@victoriametrics.com>
2026-05-22 12:17:04 +02:00
175 changed files with 10460 additions and 26166 deletions

View File

@@ -118,6 +118,7 @@ func main() {
remotewrite.InitSecretFlags()
buildinfo.Init()
logger.Init()
opentelemetry.Init()
timeserieslimits.Init(*maxLabelsPerTimeseries, *maxLabelNameLen, *maxLabelValueLen)
if promscrape.IsDryRun() {

View File

@@ -25,6 +25,11 @@ var (
rowsPerInsert = metrics.NewHistogram(`vmagent_rows_per_insert{type="opentelemetry"}`)
)
// Init must be called after flag.Parse and before using the opentelemetry package.
func Init() {
stream.InitDecodeOptions()
}
// InsertHandlerForReader processes metrics from given reader.
func InsertHandlerForReader(at *auth.Token, r io.Reader, encoding string) error {
return stream.ParseStream(r, encoding, nil, func(tss []prompb.TimeSeries, mms []prompb.MetricMetadata) error {

View File

@@ -20,6 +20,9 @@ func TestGetTime_Failure(t *testing.T) {
// negative time
f("-292273086-05-16T16:47:06Z")
// relative duration that resolves to a timestamp before 1970
f("-9223372036.855")
}
func TestGetTime_Success(t *testing.T) {
@@ -77,9 +80,6 @@ func TestGetTime_Success(t *testing.T) {
// float timestamp representation",
f("1562529662.324", time.Date(2019, 7, 7, 20, 01, 02, 324e6, time.UTC))
// negative timestamp
f("-9223372036.855", time.Date(1970, 01, 01, 00, 00, 00, 00, time.UTC))
// big timestamp
f("1223372036855", time.Date(2008, 10, 7, 9, 33, 56, 855e6, time.UTC))

View File

@@ -89,6 +89,7 @@ var staticServer = http.FileServer(http.FS(staticFiles))
func Init() {
relabel.Init()
common.InitStreamAggr()
opentelemetry.Init()
protoparserutil.StartUnmarshalWorkers()
if len(*graphiteListenAddr) > 0 {
graphiteServer = graphiteserver.MustStart(*graphiteListenAddr, *graphiteUseProxyProtocol, graphite.InsertHandler)

View File

@@ -20,6 +20,11 @@ var (
metadataInserted = metrics.NewCounter(`vm_metadata_rows_inserted_total{type="opentelemetry"}`)
)
// Init must be called after flag.Parse and before using the opentelemetry package.
func Init() {
stream.InitDecodeOptions()
}
// InsertHandler processes opentelemetry metrics.
func InsertHandler(req *http.Request) error {
extraLabels, err := protoparserutil.GetExtraLabels(req)

View File

@@ -60,10 +60,10 @@ func getDefaultMaxConcurrentRequests() int {
// Init initializes vmselect
func Init() {
tmpDirPath := *vmstorage.DataPath + "/tmp"
tmpDirPath := vmstorage.DataPath() + "/tmp"
fs.MustRemoveDirContents(tmpDirPath)
netstorage.InitTmpBlocksDir(tmpDirPath)
promql.InitRollupResultCache(*vmstorage.DataPath + "/cache/rollupResult")
promql.InitRollupResultCache(vmstorage.DataPath() + "/cache/rollupResult")
prometheus.InitMaxUniqueTimeseries(*maxConcurrentRequests)
concurrencyLimitCh = make(chan struct{}, *maxConcurrentRequests)

View File

@@ -990,9 +990,6 @@ func ExportBlocks(qt *querytracer.Tracer, sq *storage.SearchQuery, deadline sear
return fmt.Errorf("timeout exceeded before starting data export: %s", deadline.String())
}
tr := sq.GetTimeRange()
if err := vmstorage.CheckTimeRange(tr); err != nil {
return err
}
tfss, err := setupTfss(qt, tr, sq.TagFilterss, sq.MaxMetrics, deadline)
if err != nil {
return err
@@ -1098,9 +1095,6 @@ func SearchMetricNames(qt *querytracer.Tracer, sq *storage.SearchQuery, deadline
// Setup search.
tr := sq.GetTimeRange()
if err := vmstorage.CheckTimeRange(tr); err != nil {
return nil, err
}
tfss, err := setupTfss(qt, tr, sq.TagFilterss, sq.MaxMetrics, deadline)
if err != nil {
return nil, err
@@ -1127,9 +1121,6 @@ func ProcessSearchQuery(qt *querytracer.Tracer, sq *storage.SearchQuery, deadlin
// Setup search.
tr := sq.GetTimeRange()
if err := vmstorage.CheckTimeRange(tr); err != nil {
return nil, err
}
tfss, err := setupTfss(qt, tr, sq.TagFilterss, sq.MaxMetrics, deadline)
if err != nil {
return nil, err

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -37,11 +37,11 @@
<meta property="og:title" content="UI for VictoriaMetrics">
<meta property="og:url" content="https://victoriametrics.com/">
<meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data">
<script type="module" crossorigin src="./assets/index-C7gvW_Zn.js"></script>
<script type="module" crossorigin src="./assets/index-BjJ7fDL7.js"></script>
<link rel="modulepreload" crossorigin href="./assets/rolldown-runtime-COnpUsM8.js">
<link rel="modulepreload" crossorigin href="./assets/vendor-C8Kwp93_.js">
<link rel="stylesheet" crossorigin href="./assets/vendor-CnsZ1jie.css">
<link rel="stylesheet" crossorigin href="./assets/index-D2OEy8Ra.css">
<link rel="stylesheet" crossorigin href="./assets/index-BL7jEFBa.css">
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>

View File

@@ -31,6 +31,7 @@ import (
)
var (
storageDataPath = flag.String("storageDataPath", "victoria-metrics-data", "Path to storage data")
retentionPeriod = flagutil.NewRetentionDuration("retentionPeriod", "1M", "Data with timestamps outside the retentionPeriod is automatically deleted. The minimum retentionPeriod is 24h or 1d. "+
"See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#retention. See also -retentionFilter")
futureRetention = flagutil.NewRetentionDuration("futureRetention", "2d", "Data with timestamps bigger than now+futureRetention is automatically deleted. "+
@@ -43,9 +44,6 @@ var (
precisionBits = flag.Int("precisionBits", 64, "The number of precision bits to store per each value. Lower precision bits improves data compression at the cost of precision loss")
// DataPath is a path to storage data.
DataPath = flag.String("storageDataPath", "victoria-metrics-data", "Path to storage data")
_ = flag.Duration("finalMergeDelay", 0, "Deprecated: this flag does nothing")
_ = flag.Int("bigMergeConcurrency", 0, "Deprecated: this flag does nothing")
_ = flag.Int("smallMergeConcurrency", 0, "Deprecated: this flag does nothing")
@@ -56,8 +54,8 @@ var (
logNewSeries = flag.Bool("logNewSeries", false, "Whether to log new series. This option is for debug purposes only. It can lead to performance issues "+
"when big number of new series are ingested into VictoriaMetrics")
denyQueriesOutsideRetention = flag.Bool("denyQueriesOutsideRetention", false, "Whether to deny queries outside the configured -retentionPeriod. "+
"When set, then /api/v1/query_range would return '503 Service Unavailable' error for queries with 'from' value outside -retentionPeriod. "+
denyQueriesOutsideRetention = flag.Bool("denyQueriesOutsideRetention", false, "Whether to deny queries outside the configured -retentionPeriod and -futureRetention. "+
"When set, then /api/v1/query_range will return an error for queries with 'from' value outside -retentionPeriod or 'to' value beyond -futureRetention. "+
"This may be useful when multiple data sources with distinct retentions are hidden behind query-tee")
maxHourlySeries = flag.Int64("storage.maxHourlySeries", 0, "The maximum number of unique series can be added to the storage during the last hour. "+
"Excess series are logged and dropped. This can be useful for limiting series cardinality. See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#cardinality-limiter . "+
@@ -103,19 +101,8 @@ var (
"If set to 0 or a negative value, defaults to 1% of allowed memory.")
)
// CheckTimeRange returns true if the given tr is denied for querying.
func CheckTimeRange(tr storage.TimeRange) error {
if !*denyQueriesOutsideRetention {
return nil
}
minAllowedTimestamp := int64(fasttime.UnixTimestamp()*1000) - retentionPeriod.Milliseconds()
if tr.MinTimestamp > minAllowedTimestamp {
return nil
}
return &httpserver.ErrorWithStatusCode{
Err: fmt.Errorf("the given time range %s is outside the allowed -retentionPeriod=%s according to -denyQueriesOutsideRetention", &tr, retentionPeriod),
StatusCode: http.StatusServiceUnavailable,
}
func DataPath() string {
return *storageDataPath
}
// Init initializes vmstorage.
@@ -147,20 +134,21 @@ func Init(resetCacheIfNeeded func(mrs []storage.MetricRow)) {
if *idbPrefillStart > 23*time.Hour {
logger.Panicf("-storage.idbPrefillStart cannot exceed 23 hours; got %s", idbPrefillStart)
}
logger.Infof("opening storage at %q with -retentionPeriod=%s", *DataPath, retentionPeriod)
logger.Infof("opening storage at %q with -retentionPeriod=%s", *storageDataPath, retentionPeriod)
startTime := time.Now()
WG = syncwg.WaitGroup{}
opts := storage.OpenOptions{
Retention: retentionPeriod.Duration(),
FutureRetention: futureRetention.Duration(),
MaxHourlySeries: getMaxHourlySeries(),
MaxDailySeries: getMaxDailySeries(),
DisablePerDayIndex: *disablePerDayIndex,
TrackMetricNamesStats: *trackMetricNamesStats,
IDBPrefillStart: *idbPrefillStart,
LogNewSeries: *logNewSeries,
Retention: retentionPeriod.Duration(),
FutureRetention: futureRetention.Duration(),
DenyQueriesOutsideRetention: *denyQueriesOutsideRetention,
MaxHourlySeries: getMaxHourlySeries(),
MaxDailySeries: getMaxDailySeries(),
DisablePerDayIndex: *disablePerDayIndex,
TrackMetricNamesStats: *trackMetricNamesStats,
IDBPrefillStart: *idbPrefillStart,
LogNewSeries: *logNewSeries,
}
strg := storage.MustOpenStorage(*DataPath, opts)
strg := storage.MustOpenStorage(*storageDataPath, opts)
Storage = strg
initStaleSnapshotsRemover(strg)
@@ -172,7 +160,7 @@ func Init(resetCacheIfNeeded func(mrs []storage.MetricRow)) {
rowsCount := tm.SmallRowsCount + tm.BigRowsCount
sizeBytes := tm.SmallSizeBytes + tm.BigSizeBytes
logger.Infof("successfully opened storage %q in %.3f seconds; partsCount: %d; blocksCount: %d; rowsCount: %d; sizeBytes: %d",
*DataPath, time.Since(startTime).Seconds(), partsCount, blocksCount, rowsCount, sizeBytes)
*storageDataPath, time.Since(startTime).Seconds(), partsCount, blocksCount, rowsCount, sizeBytes)
// register storage metrics
storageMetrics = metrics.NewSet()
@@ -180,7 +168,7 @@ func Init(resetCacheIfNeeded func(mrs []storage.MetricRow)) {
writeStorageMetrics(w, strg)
})
metrics.RegisterSet(storageMetrics)
fs.RegisterPathFsMetrics(*DataPath)
fs.RegisterPathFsMetrics(*storageDataPath)
}
var storageMetrics *metrics.Set
@@ -325,7 +313,7 @@ func Stop() {
metrics.UnregisterSet(storageMetrics, true)
storageMetrics = nil
logger.Infof("gracefully closing the storage at %s", *DataPath)
logger.Infof("gracefully closing the storage at %s", *storageDataPath)
startTime := time.Now()
WG.WaitAndBlock()
stopStaleSnapshotsRemover()
@@ -529,15 +517,15 @@ func writeStorageMetrics(w io.Writer, strg *storage.Storage) {
tm := &m.TableMetrics
idbm := &m.TableMetrics.IndexDBMetrics
metrics.WriteGaugeUint64(w, fmt.Sprintf(`vm_free_disk_space_bytes{path=%q}`, *DataPath), fs.MustGetFreeSpace(*DataPath))
metrics.WriteGaugeUint64(w, fmt.Sprintf(`vm_free_disk_space_limit_bytes{path=%q}`, *DataPath), uint64(minFreeDiskSpaceBytes.N))
metrics.WriteGaugeUint64(w, fmt.Sprintf(`vm_total_disk_space_bytes{path=%q}`, *DataPath), fs.MustGetTotalSpace(*DataPath))
metrics.WriteGaugeUint64(w, fmt.Sprintf(`vm_free_disk_space_bytes{path=%q}`, *storageDataPath), fs.MustGetFreeSpace(*storageDataPath))
metrics.WriteGaugeUint64(w, fmt.Sprintf(`vm_free_disk_space_limit_bytes{path=%q}`, *storageDataPath), uint64(minFreeDiskSpaceBytes.N))
metrics.WriteGaugeUint64(w, fmt.Sprintf(`vm_total_disk_space_bytes{path=%q}`, *storageDataPath), fs.MustGetTotalSpace(*storageDataPath))
isReadOnly := 0
if strg.IsReadOnly() {
isReadOnly = 1
}
metrics.WriteGaugeUint64(w, fmt.Sprintf(`vm_storage_is_read_only{path=%q}`, *DataPath), uint64(isReadOnly))
metrics.WriteGaugeUint64(w, fmt.Sprintf(`vm_storage_is_read_only{path=%q}`, *storageDataPath), uint64(isReadOnly))
metrics.WriteGaugeUint64(w, `vm_active_merges{type="storage/inmemory"}`, tm.ActiveInmemoryMerges)
metrics.WriteGaugeUint64(w, `vm_active_merges{type="storage/small"}`, tm.ActiveSmallMerges)

View File

@@ -3,6 +3,7 @@ export interface MetricBase {
metric: {
[key: string]: string;
};
nullTimestamps?: number[];
}
export interface MetricResult extends MetricBase {

View File

@@ -16,6 +16,7 @@ export interface ChartTooltipProps {
point: { top: number, left: number };
unit?: string;
statsFormatted?: SeriesItemStatsFormatted;
description?: ReactNode;
isSticky?: boolean;
info?: ReactNode;
marker?: string;
@@ -34,6 +35,7 @@ const ChartTooltip: FC<ChartTooltipProps> = ({
unit = "",
info,
statsFormatted,
description,
isSticky,
marker,
duplicateCount = 0,
@@ -173,6 +175,7 @@ const ChartTooltip: FC<ChartTooltipProps> = ({
))}
</table>
)}
{description && <p className="vm-chart-tooltip__description">{description}</p>}
{info && <p className="vm-chart-tooltip__info">{info}</p>}
</div>
), u.root);

View File

@@ -143,4 +143,10 @@ $chart-tooltip-y: -1 * ($padding-global + $chart-tooltip-half-icon);
word-break: break-all;
white-space: pre-wrap;
}
&__description {
word-break: break-word;
white-space: pre-wrap;
opacity: 0.85;
}
}

View File

@@ -15,19 +15,65 @@ interface LineTooltipHook {
unit?: string;
}
// Pixel proximity for detecting hover over null-timestamp X markers drawn at chart bottom.
const NULL_HOVER_PROX = 8;
// Half the visual marker height in CSS px (BASE_POINT_SIZE * 1.4 / 2 from scatter.ts).
// scatter.ts lifts the marker center by this amount above yMin so the icon sits inside
// the plot area; the hover y-anchor must match that offset.
const NULL_MARKER_HALF_CSS = 2.8;
interface NullHover {
seriesIdx: number;
timestamp: number;
}
const findNullHover = (u: uPlot): NullHover | null => {
const cursorLeft = u.cursor.left ?? -1;
const cursorTop = u.cursor.top ?? -1;
if (cursorLeft < 0 || cursorTop < 0) return null;
const scaleY = u.scales["1"];
if (!scaleY || scaleY.min == null) return null;
const yPos = u.valToPos(scaleY.min, "1") - NULL_MARKER_HALF_CSS;
if (Math.abs(cursorTop - yPos) > NULL_HOVER_PROX) return null;
let best: { seriesIdx: number; timestamp: number; dist: number } | null = null;
for (let s = 1; s < u.series.length; s++) {
const seriesItem = u.series[s] as SeriesItem;
if (!seriesItem.show) continue;
const nullTs = seriesItem.nullTimestamps;
if (!nullTs || !nullTs.length) continue;
for (let i = 0; i < nullTs.length; i++) {
const t = nullTs[i];
const xPos = u.valToPos(t, "x");
const dist = Math.abs(cursorLeft - xPos);
if (dist < NULL_HOVER_PROX && (best === null || dist < best.dist)) {
best = { seriesIdx: s, timestamp: t, dist };
}
}
}
return best ? { seriesIdx: best.seriesIdx, timestamp: best.timestamp } : null;
};
const NULL_DESCRIPTION = "\"null\" can be a staleness marker or an actual NaN/null value produced by exporter.";
const useLineTooltip = ({ u, metrics, series, unit }: LineTooltipHook) => {
const [showTooltip, setShowTooltip] = useState(false);
const [tooltipIdx, setTooltipIdx] = useState({ seriesIdx: -1, dataIdx: -1 });
const [nullTooltip, setNullTooltip] = useState<NullHover | null>(null);
const [stickyTooltips, setStickyToolTips] = useState<ChartTooltipProps[]>([]);
const resetTooltips = () => {
setStickyToolTips([]);
setTooltipIdx({ seriesIdx: -1, dataIdx: -1 });
setNullTooltip(null);
};
const setCursor = (u: uPlot) => {
const dataIdx = u.cursor.idx ?? -1;
setTooltipIdx(prev => ({ ...prev, dataIdx }));
setNullTooltip(findNullHover(u));
};
const seriesFocus = (u: uPlot, sidx: (number | null)) => {
@@ -35,7 +81,36 @@ const useLineTooltip = ({ u, metrics, series, unit }: LineTooltipHook) => {
setTooltipIdx(prev => ({ ...prev, seriesIdx }));
};
const getNullTooltipProps = (hit: NullHover): ChartTooltipProps => {
const { seriesIdx, timestamp } = hit;
const metricItem = metrics[seriesIdx - 1];
const seriesItem = series[seriesIdx] as SeriesItem;
const groups = new Set(metrics.map(m => m.group));
const group = metricItem?.group || 0;
const yMin = u?.scales?.[1]?.min ?? 0;
const point = {
top: u ? u.valToPos(yMin, seriesItem?.scale || "1") - NULL_MARKER_HALF_CSS : 0,
left: u ? u.valToPos(timestamp, "x") : 0,
};
return {
u,
id: `null_${seriesIdx}_${timestamp}`,
title: groups.size > 1 ? `Query ${group}` : "",
dates: [dayjs(timestamp * 1000).tz().format(DATE_FULL_TIMEZONE_FORMAT)],
value: "null",
info: getMetricName(metricItem, seriesItem),
description: NULL_DESCRIPTION,
marker: `${seriesItem?.stroke}`,
point,
};
};
const getTooltipProps = useCallback((): ChartTooltipProps => {
if (nullTooltip) return getNullTooltipProps(nullTooltip);
const { seriesIdx, dataIdx } = tooltipIdx;
const metricItem = metrics[seriesIdx - 1];
const seriesItem = series[seriesIdx] as SeriesItem;
@@ -84,7 +159,7 @@ const useLineTooltip = ({ u, metrics, series, unit }: LineTooltipHook) => {
marker: `${seriesItem?.stroke}`,
duplicateCount,
};
}, [u, tooltipIdx, metrics, series, unit]);
}, [u, tooltipIdx, metrics, series, unit, nullTooltip]);
const handleClick = useCallback(() => {
if (!showTooltip) return;
@@ -99,8 +174,9 @@ const useLineTooltip = ({ u, metrics, series, unit }: LineTooltipHook) => {
};
useEffect(() => {
setShowTooltip(tooltipIdx.dataIdx !== -1 && tooltipIdx.seriesIdx !== -1);
}, [tooltipIdx]);
const normalHit = tooltipIdx.dataIdx !== -1 && tooltipIdx.seriesIdx !== -1;
setShowTooltip(normalHit || nullTooltip !== null);
}, [tooltipIdx, nullTooltip]);
useEventListener("click", handleClick);

View File

@@ -20,7 +20,7 @@ describe("convertMetricsDataToCSV", () => {
},
];
const result = convertMetricsDataToCSV(data);
expect(result).toBe("header1,header2\n123,value2");
expect(result).toBe("header1,header2,__timestamp__,__value__\n123,value2,1623945600,123");
});
it("should return a valid CSV string for multiple metric entries with values", () => {
@@ -43,7 +43,7 @@ describe("convertMetricsDataToCSV", () => {
},
];
const result = convertMetricsDataToCSV(data);
expect(result).toBe("header1,header2\n123,value2\n456,value4");
expect(result).toBe("header1,header2,__timestamp__,__value__\n123,value2,1623945600,123\n456,value4,1623949200,456");
});
it("should handle metric entries with multiple values field", () => {
@@ -58,7 +58,7 @@ describe("convertMetricsDataToCSV", () => {
},
];
const result = convertMetricsDataToCSV(data);
expect(result).toBe("header1,header2\n123-456,values");
expect(result).toBe("header1,header2,__timestamp__,__value__\n123-456,values,-,-");
});
it("should handle a combination of metric entries with value and values", () => {
@@ -81,6 +81,19 @@ describe("convertMetricsDataToCSV", () => {
},
];
const result = convertMetricsDataToCSV(data);
expect(result).toBe("header1,header2\n123,first\n456-789,second");
expect(result).toBe("header1,header2,__timestamp__,__value__\n123,first,1623945600,123\n456-789,second,-,-");
});
it("should return value and timestamp if metric field is empty", () => {
const data: InstantMetricResult[] = [
{
value: [1623945600, "123"],
group: 0,
metric: {}
},
];
const result = convertMetricsDataToCSV(data);
expect(result).toBe("__timestamp__,__value__\n1623945600,123");
});
});

View File

@@ -3,16 +3,22 @@ import { getColumns, MetricCategory } from "../../hooks/useSortedCategories";
import { formatValueToCSV } from "../../utils/csv";
const getHeaders = (data: InstantMetricResult[]): string => {
return getColumns(data).map(({ key }) => key).join(",");
const metricHeaders = getColumns(data).map(({ key }) => key);
return [...metricHeaders, "__timestamp__", "__value__"].join(",");
};
const getRows = (data: InstantMetricResult[], headers: MetricCategory[]) => {
return data?.map(d => headers.map(c => formatValueToCSV(d.metric[c.key] || "-")).join(","));
return data?.map(d => {
const metricPart = headers.map(c => formatValueToCSV(d.metric[c.key] || "-"));
const timestamp = d.value ? formatValueToCSV(String(d.value[0])) : "-";
const value = d.value ? formatValueToCSV(d.value[1]) : "-";
return [...metricPart, timestamp, value].join(",");
});
};
export const convertMetricsDataToCSV = (data: InstantMetricResult[]): string => {
if (!data.length) return "";
const headers = getHeaders(data);
if (!headers.length) return "";
const rows = getRows(data, getColumns(data));
return [headers, ...rows].join("\n");
};

View File

@@ -149,15 +149,21 @@ export const useFetchExport = ({ hideQuery, showAllSeries }: FetchQueryParams):
const pointsToTake = shouldDownsample ? maxPointsPerSeries : totalPoints;
const step = shouldDownsample ? totalPoints / maxPointsPerSeries : 1;
const values: [number, number][] = Array.from({ length: pointsToTake }, (_, i) => {
const values: [number, number][] = new Array(pointsToTake);
const nullTimestamps: number[] = [];
for (let i = 0; i < pointsToTake; i++) {
const idx = shouldDownsample ? Math.floor(i * step) : i;
return [rawTimestamps[idx] / 1000, rawValues[idx]];
});
const ts = rawTimestamps[idx] / 1000;
const raw = rawValues[idx];
if (raw === null) nullTimestamps.push(ts);
values[i] = [ts, raw as number];
}
tempData.push({
group: counter,
metric: jsonLine.metric,
values,
nullTimestamps,
} as MetricBase);
}

View File

@@ -11,6 +11,7 @@ export interface SeriesItem extends Series {
statsFormatted: SeriesItemStatsFormatted;
median: number;
hasAlias?: boolean;
nullTimestamps?: number[];
}
export interface HideSeriesArgs {

View File

@@ -103,6 +103,28 @@ export const drawPoints = (u: uPlot, seriesIdx: number) => {
u.ctx.lineWidth = 1.4 * uPlot.pxRatio;
u.ctx.strokeStyle = u.ctx.fillStyle;
u.ctx.stroke(squaresPath);
const nullTs = (series as unknown as { nullTimestamps?: number[] }).nullTimestamps;
if (nullTs && nullTs.length) {
const xSize = BASE_POINT_SIZE * 1.4 * uPlot.pxRatio;
const xHalf = xSize / 2;
// Lift the marker by half its size so the entire icon sits inside the plot area
// (yMin maps to the plot's bottom edge, so centering on it would clip the lower half).
const cy = valToPosY(yMin, scaleY, yDim, yOff) - xHalf;
const xPath = new Path2D();
for (let i = 0; i < nullTs.length; i++) {
const t = nullTs[i];
if (t < xMin || t > xMax) continue;
const cx = valToPosX(t, scaleX, xDim, xOff);
xPath.moveTo(cx - xHalf, cy - xHalf);
xPath.lineTo(cx + xHalf, cy + xHalf);
xPath.moveTo(cx + xHalf, cy - xHalf);
xPath.lineTo(cx - xHalf, cy + xHalf);
}
u.ctx.lineWidth = 1.6 * uPlot.pxRatio;
u.ctx.strokeStyle = u.ctx.fillStyle;
u.ctx.stroke(xPath);
}
};
uPlot.orient(u, seriesIdx, orientCallback);

View File

@@ -38,6 +38,7 @@ export const getSeriesItemContext = (data: MetricResult[], hideSeries: string[],
show: !includesHideSeries(label, hideSeries),
scale: "1",
paths: isRawQuery ? drawPoints : undefined,
nullTimestamps: d.nullTimestamps,
...getSeriesStatistics(d),
};
};

View File

@@ -3,7 +3,7 @@ services:
# It scrapes targets defined in --promscrape.config
# And forward them to --remoteWrite.url
vmagent:
image: victoriametrics/vmagent:v1.143.0
image: victoriametrics/vmagent:v1.144.0
depends_on:
- "vmauth"
ports:
@@ -42,14 +42,14 @@ services:
# vmstorage shards. Each shard receives 1/N of all metrics sent to vminserts,
# where N is number of vmstorages (2 in this case).
vmstorage-1:
image: victoriametrics/vmstorage:v1.143.0-cluster
image: victoriametrics/vmstorage:v1.144.0-cluster
volumes:
- strgdata-1:/storage
command:
- "--storageDataPath=/storage"
restart: always
vmstorage-2:
image: victoriametrics/vmstorage:v1.143.0-cluster
image: victoriametrics/vmstorage:v1.144.0-cluster
volumes:
- strgdata-2:/storage
command:
@@ -59,7 +59,7 @@ services:
# vminsert is ingestion frontend. It receives metrics pushed by vmagent,
# pre-process them and distributes across configured vmstorage shards.
vminsert-1:
image: victoriametrics/vminsert:v1.143.0-cluster
image: victoriametrics/vminsert:v1.144.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -68,7 +68,7 @@ services:
- "--storageNode=vmstorage-2:8400"
restart: always
vminsert-2:
image: victoriametrics/vminsert:v1.143.0-cluster
image: victoriametrics/vminsert:v1.144.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -80,7 +80,7 @@ services:
# vmselect is a query fronted. It serves read queries in MetricsQL or PromQL.
# vmselect collects results from configured `--storageNode` shards.
vmselect-1:
image: victoriametrics/vmselect:v1.143.0-cluster
image: victoriametrics/vmselect:v1.144.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -90,7 +90,7 @@ services:
- "--vmalert.proxyURL=http://vmalert:8880"
restart: always
vmselect-2:
image: victoriametrics/vmselect:v1.143.0-cluster
image: victoriametrics/vmselect:v1.144.0-cluster
depends_on:
- "vmstorage-1"
- "vmstorage-2"
@@ -105,7 +105,7 @@ services:
# read requests from Grafana, vmui, vmalert among vmselects.
# It can be used as an authentication proxy.
vmauth:
image: victoriametrics/vmauth:v1.143.0
image: victoriametrics/vmauth:v1.144.0
depends_on:
- "vmselect-1"
- "vmselect-2"
@@ -119,7 +119,7 @@ services:
# vmalert executes alerting and recording rules
vmalert:
image: victoriametrics/vmalert:v1.143.0
image: victoriametrics/vmalert:v1.144.0
depends_on:
- "vmauth"
ports:

View File

@@ -3,7 +3,7 @@ services:
# It scrapes targets defined in --promscrape.config
# And forward them to --remoteWrite.url
vmagent:
image: victoriametrics/vmagent:v1.143.0
image: victoriametrics/vmagent:v1.144.0
depends_on:
- "victoriametrics"
ports:
@@ -18,7 +18,7 @@ services:
# VictoriaMetrics instance, a single process responsible for
# storing metrics and serve read requests.
victoriametrics:
image: victoriametrics/victoria-metrics:v1.143.0
image: victoriametrics/victoria-metrics:v1.144.0
ports:
- 8428:8428
- 8089:8089
@@ -59,7 +59,7 @@ services:
# vmalert executes alerting and recording rules
vmalert:
image: victoriametrics/vmalert:v1.143.0
image: victoriametrics/vmalert:v1.144.0
depends_on:
- "victoriametrics"
- "alertmanager"

View File

@@ -1,6 +1,6 @@
services:
vmagent:
image: victoriametrics/vmagent:v1.143.0
image: victoriametrics/vmagent:v1.144.0
depends_on:
- "victoriametrics"
ports:
@@ -14,7 +14,7 @@ services:
restart: always
victoriametrics:
image: victoriametrics/victoria-metrics:v1.143.0
image: victoriametrics/victoria-metrics:v1.144.0
ports:
- 8428:8428
volumes:
@@ -40,7 +40,7 @@ services:
restart: always
vmalert:
image: victoriametrics/vmalert:v1.143.0
image: victoriametrics/vmalert:v1.144.0
depends_on:
- "victoriametrics"
ports:

View File

@@ -240,23 +240,23 @@ vmagent will write data into VictoriaMetrics single-node and cluster (with tenan
# compose.yaml
services:
vmsingle:
image: victoriametrics/victoria-metrics:v1.143.0
image: victoriametrics/victoria-metrics:v1.144.0
vmstorage:
image: victoriametrics/vmstorage:v1.143.0-cluster
image: victoriametrics/vmstorage:v1.144.0-cluster
vminsert:
image: victoriametrics/vminsert:v1.143.0-cluster
image: victoriametrics/vminsert:v1.144.0-cluster
command:
- -storageNode=vmstorage:8400
vmselect:
image: victoriametrics/vmselect:v1.143.0-cluster
image: victoriametrics/vmselect:v1.144.0-cluster
command:
- -storageNode=vmstorage:8401
vmagent:
image: victoriametrics/vmagent:v1.143.0
image: victoriametrics/vmagent:v1.144.0
volumes:
- ./scrape.yaml:/etc/vmagent/config.yaml
command:
@@ -308,7 +308,7 @@ Now add the vmauth service to `compose.yaml`:
# compose.yaml
services:
vmauth:
image: docker.io/victoriametrics/vmauth:v1.143.0
image: docker.io/victoriametrics/vmauth:v1.144.0
ports:
- 8427:8427
volumes:

View File

@@ -155,15 +155,15 @@ These services will store and query the metrics scraped by vmagent.
# compose.yaml
services:
vmstorage:
image: victoriametrics/vmstorage:v1.143.0-cluster
image: victoriametrics/vmstorage:v1.144.0-cluster
vminsert:
image: victoriametrics/vminsert:v1.143.0-cluster
image: victoriametrics/vminsert:v1.144.0-cluster
command:
- -storageNode=vmstorage:8400
vmselect:
image: victoriametrics/vmselect:v1.143.0-cluster
image: victoriametrics/vmselect:v1.144.0-cluster
command:
- -storageNode=vmstorage:8401
ports:
@@ -196,7 +196,7 @@ Add the vmauth service to `compose.yaml`:
# compose.yaml
services:
vmauth:
image: victoriametrics/vmauth:v1.143.0-enterprise
image: victoriametrics/vmauth:v1.144.0-enterprise
ports:
- 8427:8427
volumes:
@@ -251,7 +251,7 @@ Add the vmagent service to `compose.yaml` with OAuth2 configuration:
# compose.yaml
services:
vmagent:
image: victoriametrics/vmagent:v1.143.0
image: victoriametrics/vmagent:v1.144.0
volumes:
- ./scrape.yaml:/etc/vmagent/config.yaml
command:

View File

@@ -107,7 +107,7 @@ The final piece is the Docker Compose file. This ties all the services together
# compose.yml
services:
victoriametrics:
image: victoriametrics/victoria-metrics:v1.143.0
image: victoriametrics/victoria-metrics:v1.144.0
command:
- "--storageDataPath=/victoria-metrics-data"
- "--selfScrapeInterval=10s"
@@ -128,7 +128,7 @@ services:
- ./alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro
vmalert:
image: victoriametrics/vmalert:v1.143.0
image: victoriametrics/vmalert:v1.144.0
depends_on:
- victoriametrics
- alertmanager

View File

@@ -58,9 +58,9 @@ Download the newest available [VictoriaMetrics release](https://docs.victoriamet
from [DockerHub](https://hub.docker.com/r/victoriametrics/victoria-metrics) or [Quay](https://quay.io/repository/victoriametrics/victoria-metrics?tab=tags):
```sh
docker pull victoriametrics/victoria-metrics:v1.143.0
docker pull victoriametrics/victoria-metrics:v1.144.0
docker run -it --rm -v `pwd`/victoria-metrics-data:/victoria-metrics-data -p 8428:8428 \
victoriametrics/victoria-metrics:v1.143.0 --selfScrapeInterval=5s -storageDataPath=victoria-metrics-data
victoriametrics/victoria-metrics:v1.144.0 --selfScrapeInterval=5s -storageDataPath=victoria-metrics-data
```
_For Enterprise images, see [this link](https://docs.victoriametrics.com/victoriametrics/enterprise/#docker-images)._

View File

@@ -26,6 +26,12 @@ See also [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-rel
## tip
* FEATURE: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/), [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): add `-opentelemetry.promoteAllResourceAttributes` and `-opentelemetry.promoteScopeMetadata` command-line flags to allow managing label promotion for resource attributes and OTel scope metadata. See [OpenTelemetry](https://docs.victoriametrics.com/victoriametrics/integrations/opentelemetry/) docs and [#10931](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10931).
## [v1.144.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.144.0)
Released at 2026-05-22
* FEATURE: all VictoriaMetrics components: improve logging for the `-memory.allowedBytes` flag to warn about excessively low value (less than 1MB). See issue [#10935](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10935).
* FEATURE: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/) and [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): add `basicAuth.usernameFile` command-line flags for reading basic auth username from a file, similar to the existing `basicAuth.passwordFile`. The file is re-read every second. See [#9436](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9436). Thanks to @kimjune01 for the contribution.
* FEATURE: `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): add `clusternative.tls` `vminsert` configuration flags for [multi-level cluster setups](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multi-level-cluster-setup). See [#10958](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10958).
@@ -34,6 +40,7 @@ See also [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-rel
* FEATURE: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): drain in-memory remote write queue on shutdown within the 5-second grace period before falling back to persisting blocks to disk. See [#9996](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/9996)
* FEATURE: `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): Improve [slowness-based rerouting](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#slowness-based-re-routing) to prevent rerouting storms under high cluster load. Previously, rerouting could cascade across storage nodes when the whole cluster was saturated, making the situation worse. Now rerouting only activates when the cluster p90 saturation is below 60%, and the slowest node is more than 20% slower than p90. See [#10876](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10876).
* FEATURE: [vmauth](https://docs.victoriametrics.com/victoriametrics/vmauth/): add `{{.MetricsAccountID}}` and `{{.MetricsProjectID}}` [JWT claim placeholders](https://docs.victoriametrics.com/victoriametrics/vmauth/#jwt-claim-based-request-templating) for use in `headers` and `url_prefix` config fields. Previously, only the combined `{{.MetricsTenant}}` (`accountID:projectID`) JWT placeholder was supported, making it impossible to configure [multitenancy via headers](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy-via-headers). See [#10927](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10927). Thanks to @Vinayak9769 for the contribution.
* FEATURE: [vmui](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui): display `null` values on `Raw Query` chart. `null` values can be actual `NaN` or `null` values exposed by the exporter, or [stale markers](https://docs.victoriametrics.com/victoriametrics/vmagent/#prometheus-staleness-markers). Before, vmui Raw Query was silently dropping non-numeric values. Displaying such values on the chart could improve the debugging experience. See [#10986](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10986).
* BUGFIX: [stream aggregation](https://docs.victoriametrics.com/victoriametrics/stream-aggregation/): stop emitting stale values for `quantiles(...)` outputs when a time series has no samples during the current aggregation interval. See [#10918](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10918). Thanks to @alexei38 for the contribution.
* BUGFIX: [stream aggregation](https://docs.victoriametrics.com/victoriametrics/stream-aggregation/): extend delay on aggregation windows flush by the biggest lag among pushed samples. Before, the delay was calculated as 95th percentile across samples, which could underrepresent outliers and reject them from aggregation as "too old". See [#10402](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10402).
@@ -47,6 +54,9 @@ See also [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-rel
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): return error on startup if `-remoteWrite.disableOnDiskQueue` is not configured uniformly across all `-remoteWrite.url` targets when `-remoteWrite.shardByURL` is enabled. Either all targets must have it enabled or all must have it disabled. See [#10507](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10507).
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): hide values passed to `vmalert.proxyURL` in startup logs, `/metrics`, and `/flags`, since they can contain sensitive HTTP headers such as `Authorization` and API keys.
* BUGFIX: [vmui](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui): preserve exact series values in graph tooltips instead of rounding them by significant digits. See [#10952](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10952).
* BUGFIX: all VictoriaMetrics components: fix int64 overflow when parsing [timestamp parameters](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#timestamp-formats) with relative durations. See [#10880](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10880).
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): `-denyQueriesOutsideRetention` now also rejects queries whose end time is beyond `-futureRetention`. See [#10879](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10879).
* BUGFIX: [vmui](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui): add missing `__timestamp__` and `__value__` columns to CSV exported from the table view on the Query tab. See [#10975](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10975).
## [v1.143.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.143.0)
@@ -232,6 +242,28 @@ It enables back `Discovered targets` debug UI by default.
* BUGFIX: `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): properly apply `extra_filters[]` filter when querying `vm_account_id` or `vm_project_id` labels via [multitenant](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#multitenancy) request for `/api/v1/label/…/values` API. Before, `extra_filters` was ignored. See [#10503](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10503).
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): revert the use of rollup result cache for [instant queries](https://docs.victoriametrics.com/keyConcepts.html#instant-query) that contain [`rate`](https://docs.victoriametrics.com/MetricsQL.html#rate) function with a lookbehind window larger than `-search.minWindowForInstantRollupOptimization`. The cache usage was removed since [v1.132.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.132.0). See [#10098](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10098#issuecomment-3895011084) for more details.
## [v1.136.10](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.136.10)
Released at 2026-05-22
**v1.136.x is a line of [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-releases/). It contains important up-to-date bugfixes for [VictoriaMetrics enterprise](https://docs.victoriametrics.com/victoriametrics/enterprise/).
All these fixes are also included in [the latest community release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest).
The v1.136.x line will be supported for at least 12 months since [v1.136.0](https://docs.victoriametrics.com/victoriametrics/changelog/#v11360) release**
* BUGFIX: [stream aggregation](https://docs.victoriametrics.com/victoriametrics/stream-aggregation/): stop emitting stale values for `quantiles(...)` outputs when a time series has no samples during the current aggregation interval. See [#10918](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10918). Thanks to @alexei38 for the contribution.
* BUGFIX: [stream aggregation](https://docs.victoriametrics.com/victoriametrics/stream-aggregation/): extend delay on aggregation windows flush by the biggest lag among pushed samples. Before, the delay was calculated as 95th percentile across samples, which could underrepresent outliers and reject them from aggregation as "too old". See [#10402](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10402).
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): fix a bug in [cardinality limiters](https://docs.victoriametrics.com/victoriametrics/vmagent/#cardinality-limiter) where series with different labels, like `{a="bc"}` and `{ab="c"}`, could be incorrectly treated as identical and dropped. See [#10937](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10937).
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): hide values passed to `-remoteWrite.headers` in startup logs, `/metrics`, and `/flags`, since they can contain sensitive HTTP headers such as `Authorization` and API keys.
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): hide values passed to `-remoteWrite.proxyURL` in startup logs, `/metrics`, and `/flags`, since they can contain sensitive credentials.
* BUGFIX: [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): hide values passed to `-remoteWrite.headers`,`remoteRead.headers`, `datasource.headers` and `notifier.headers` in startup logs, `/metrics`, and `/flags`, since they can contain sensitive HTTP headers such as `Authorization` and API keys.
* BUGFIX: `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): properly establish [mtls](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#mtls-protection) connection between vmstorage and vminsert. Regression was introduced in v1.130.0 release for the enterprise version of vmstorage. See [#10972](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10972).
* BUGFIX: [vmrestore](https://docs.victoriametrics.com/victoriametrics/vmrestore/): fix a bug where specifying `-storageDataPath` with a trailing slash could cause `vmrestore` to panic. See [#10823](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10823). Thanks to @utafrali for the contribution.
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): prevent unintentional rerouting of samples to other sharding targets when one of the `-remoteWrite.url` targets with `-remoteWrite.disableOnDiskQueue` becomes blocked. Previously this could break the sharding guarantee by sending samples to wrong targets instead of dropping or retrying them. See [#10507](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10507).
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): return error on startup if `-remoteWrite.disableOnDiskQueue` is not configured uniformly across all `-remoteWrite.url` targets when `-remoteWrite.shardByURL` is enabled. Either all targets must have it enabled or all must have it disabled. See [#10507](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10507).
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): hide values passed to `vmalert.proxyURL` in startup logs, `/metrics`, and `/flags`, since they can contain sensitive HTTP headers such as `Authorization` and API keys.
* BUGFIX: [vmui](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui): preserve exact series values in graph tooltips instead of rounding them by significant digits. See [#10952](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10952).
* BUGFIX: [vmui](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui): add missing `__timestamp__` and `__value__` columns to CSV exported from the table view on the Query tab. See [#10975](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10975).
## [v1.136.9](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.136.9)
Released at 2026-05-08
@@ -553,6 +585,26 @@ See changes [here](https://docs.victoriametrics.com/victoriametrics/changelog/ch
See changes [here](https://docs.victoriametrics.com/victoriametrics/changelog/changelog_2025/#v11230)
## [v1.122.23](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.122.23)
Released at 2026-05-22
**v1.122.x is a line of [LTS releases](https://docs.victoriametrics.com/victoriametrics/lts-releases/). It contains important up-to-date bugfixes for [VictoriaMetrics enterprise](https://docs.victoriametrics.com/victoriametrics/enterprise/).
All these fixes are also included in [the latest community release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest).
The v1.122.x line will be supported for at least 12 months since [v1.122.0](https://docs.victoriametrics.com/victoriametrics/changelog/#v11220) release**
* BUGFIX: [stream aggregation](https://docs.victoriametrics.com/victoriametrics/stream-aggregation/): stop emitting stale values for `quantiles(...)` outputs when a time series has no samples during the current aggregation interval. See [#10918](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10918). Thanks to @alexei38 for the contribution.
* BUGFIX: [stream aggregation](https://docs.victoriametrics.com/victoriametrics/stream-aggregation/): extend delay on aggregation windows flush by the biggest lag among pushed samples. Before, the delay was calculated as 95th percentile across samples, which could underrepresent outliers and reject them from aggregation as "too old". See [#10402](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10402).
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): fix a bug in [cardinality limiters](https://docs.victoriametrics.com/victoriametrics/vmagent/#cardinality-limiter) where series with different labels, like `{a="bc"}` and `{ab="c"}`, could be incorrectly treated as identical and dropped. See [#10937](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/10937).
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): hide values passed to `-remoteWrite.headers` in startup logs, `/metrics`, and `/flags`, since they can contain sensitive HTTP headers such as `Authorization` and API keys.
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): hide values passed to `-remoteWrite.proxyURL` in startup logs, `/metrics`, and `/flags`, since they can contain sensitive credentials.
* BUGFIX: [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/): hide values passed to `-remoteWrite.headers`,`remoteRead.headers`, `datasource.headers` and `notifier.headers` in startup logs, `/metrics`, and `/flags`, since they can contain sensitive HTTP headers such as `Authorization` and API keys.
* BUGFIX: [vmrestore](https://docs.victoriametrics.com/victoriametrics/vmrestore/): fix a bug where specifying `-storageDataPath` with a trailing slash could cause `vmrestore` to panic. See [#10823](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10823). Thanks to @utafrali for the contribution.
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): prevent unintentional rerouting of samples to other sharding targets when one of the `-remoteWrite.url` targets with `-remoteWrite.disableOnDiskQueue` becomes blocked. Previously this could break the sharding guarantee by sending samples to wrong targets instead of dropping or retrying them. See [#10507](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10507).
* BUGFIX: [vmagent](https://docs.victoriametrics.com/victoriametrics/vmagent/): return error on startup if `-remoteWrite.disableOnDiskQueue` is not configured uniformly across all `-remoteWrite.url` targets when `-remoteWrite.shardByURL` is enabled. Either all targets must have it enabled or all must have it disabled. See [#10507](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10507).
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/): hide values passed to `vmalert.proxyURL` in startup logs, `/metrics`, and `/flags`, since they can contain sensitive HTTP headers such as `Authorization` and API keys.
* BUGFIX: [vmui](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#vmui): preserve exact series values in graph tooltips instead of rounding them by significant digits. See [#10952](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/10952).
## [v1.122.22](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.122.22)
Released at 2026-05-08

View File

@@ -121,7 +121,7 @@ It is allowed to run Enterprise components in [cases listed here](https://docs.v
Binary releases of Enterprise components are available at [the releases page for VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest),
[the releases page for VictoriaLogs](https://github.com/VictoriaMetrics/VictoriaLogs/releases/latest)
and [the releases page for VictoriaTraces](https://github.com/VictoriaMetrics/VictoriaTraces/releases/latest).
Enterprise binaries and packages have `enterprise` suffix in their names. For example, `victoria-metrics-linux-amd64-v1.143.0-enterprise.tar.gz`.
Enterprise binaries and packages have `enterprise` suffix in their names. For example, `victoria-metrics-linux-amd64-v1.144.0-enterprise.tar.gz`.
In order to run binary release of Enterprise component, please download the `*-enterprise.tar.gz` archive for your OS and architecture
from the corresponding releases page and unpack it. Then run the unpacked binary.
@@ -139,8 +139,8 @@ For example, the following command runs VictoriaMetrics Enterprise binary with t
obtained at [this page](https://victoriametrics.com/products/enterprise/trial/):
```sh
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.143.0/victoria-metrics-linux-amd64-v1.143.0-enterprise.tar.gz
tar -xzf victoria-metrics-linux-amd64-v1.143.0-enterprise.tar.gz
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.144.0/victoria-metrics-linux-amd64-v1.144.0-enterprise.tar.gz
tar -xzf victoria-metrics-linux-amd64-v1.144.0-enterprise.tar.gz
./victoria-metrics-prod -license=BASE64_ENCODED_LICENSE_KEY
```
@@ -155,7 +155,7 @@ Alternatively, VictoriaMetrics Enterprise license can be stored in the file and
It is allowed to run Enterprise components in [cases listed here](https://docs.victoriametrics.com/victoriametrics/enterprise/#valid-cases-for-victoriametrics-enterprise).
Docker images for Enterprise components are available at [VictoriaMetrics Docker Hub](https://hub.docker.com/u/victoriametrics) and [VictoriaMetrics Quay](https://quay.io/organization/victoriametrics).
Enterprise docker images have `enterprise` suffix in their names. For example, `victoriametrics/victoria-metrics:v1.143.0-enterprise`.
Enterprise docker images have `enterprise` suffix in their names. For example, `victoriametrics/victoria-metrics:v1.144.0-enterprise`.
In order to run Docker image of VictoriaMetrics Enterprise component, it is required to provide the license key via the command-line
flag as described in the [binary-releases](https://docs.victoriametrics.com/victoriametrics/enterprise/#binary-releases) section.
@@ -165,13 +165,13 @@ Enterprise license key can be obtained at [this page](https://victoriametrics.co
For example, the following command runs VictoriaMetrics Enterprise Docker image with the specified license key:
```sh
docker run --name=victoria-metrics victoriametrics/victoria-metrics:v1.143.0-enterprise -license=BASE64_ENCODED_LICENSE_KEY
docker run --name=victoria-metrics victoriametrics/victoria-metrics:v1.144.0-enterprise -license=BASE64_ENCODED_LICENSE_KEY
```
Alternatively, the license code can be stored in the file and then referred via `-licenseFile` command-line flag:
```sh
docker run --name=victoria-metrics -v /vm-license:/vm-license victoriametrics/victoria-metrics:v1.143.0-enterprise -licenseFile=/path/to/vm-license
docker run --name=victoria-metrics -v /vm-license:/vm-license victoriametrics/victoria-metrics:v1.144.0-enterprise -licenseFile=/path/to/vm-license
```
Example docker-compose configuration:
@@ -181,7 +181,7 @@ version: "3.5"
services:
victoriametrics:
container_name: victoriametrics
image: victoriametrics/victoria-metrics:v1.143.0
image: victoriametrics/victoria-metrics:v1.144.0
ports:
- 8428:8428
volumes:
@@ -213,7 +213,7 @@ is used to provide the license key in plain-text:
```yaml
server:
image:
tag: v1.143.0-enterprise
tag: v1.144.0-enterprise
license:
key: {BASE64_ENCODED_LICENSE_KEY}
@@ -224,7 +224,7 @@ In order to provide the license key via existing secret, the following values fi
```yaml
server:
image:
tag: v1.143.0-enterprise
tag: v1.144.0-enterprise
license:
secret:
@@ -274,7 +274,7 @@ spec:
license:
key: {BASE64_ENCODED_LICENSE_KEY}
image:
tag: v1.143.0-enterprise
tag: v1.144.0-enterprise
```
In order to provide the license key via an existing secret, the following custom resource is used:
@@ -291,7 +291,7 @@ spec:
name: vm-license
key: license
image:
tag: v1.143.0-enterprise
tag: v1.144.0-enterprise
```
Example secret with license key:
@@ -342,7 +342,7 @@ Builds are available for amd64 and arm64 architectures.
Example archive:
`victoria-metrics-linux-amd64-v1.143.0-enterprise.tar.gz`
`victoria-metrics-linux-amd64-v1.144.0-enterprise.tar.gz`
Includes:
@@ -351,7 +351,7 @@ Includes:
Example Docker image:
`victoriametrics/victoria-metrics:v1.143.0-enterprise-fips` uses the FIPS-compatible binary and based on `scratch` image.
`victoriametrics/victoria-metrics:v1.144.0-enterprise-fips` uses the FIPS-compatible binary and based on `scratch` image.
## What Happens to Licensed Components When a License Expires

View File

@@ -28,9 +28,17 @@ The following label sanitization options can be enabled:
> These flags can be applied on vmagent, vminsert or VictoriaMetrics single-node.
## Instrumentation Scope
By default, VictoriaMetrics promotes [OTel scope metadata](https://opentelemetry.io/docs/specs/otel/common/instrumentation-scope/) to metric labels. This behavior can be disabled via `-opentelemetry.promoteScopeMetadata`.
## Resource Attributes
By default, VictoriaMetrics promotes all [OpenTelemetry resource](https://opentelemetry.io/docs/specs/otel/resource/data-model/) attributes to labels and attaches them to all ingested OTLP metrics.
The following attribute promotion options can be configured:
* `-opentelemetry.promoteAllResourceAttributes` - promotes all resource attributes to labels, except for the ones configured with `-opentelemetry.ignoreResourceAttributes`.
* `-opentelemetry.promoteResourceAttributes` - promotes specific list of resource attributes to labels. It cannot be configured simultaneously with `-opentelemetry.promoteAllResourceAttributes`.
* `-opentelemetry.ignoreResourceAttributes` - controls which resource attributes to ignore, can only be set when `-opentelemetry.promoteAllResourceAttributes` is true.
## Exponential histograms

View File

@@ -35,8 +35,8 @@ scrape_configs:
After you created the `scrape.yaml` file, download and unpack [single-node VictoriaMetrics](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/) to the same directory:
```sh
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.143.0/victoria-metrics-linux-amd64-v1.143.0.tar.gz
tar xzf victoria-metrics-linux-amd64-v1.143.0.tar.gz
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.144.0/victoria-metrics-linux-amd64-v1.144.0.tar.gz
tar xzf victoria-metrics-linux-amd64-v1.144.0.tar.gz
```
Then start VictoriaMetrics and instruct it to scrape targets defined in `scrape.yaml` and save scraped metrics
@@ -150,8 +150,8 @@ Then start [single-node VictoriaMetrics](https://docs.victoriametrics.com/victor
```yaml
# Download and unpack single-node VictoriaMetrics
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.143.0/victoria-metrics-linux-amd64-v1.143.0.tar.gz
tar xzf victoria-metrics-linux-amd64-v1.143.0.tar.gz
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.144.0/victoria-metrics-linux-amd64-v1.144.0.tar.gz
tar xzf victoria-metrics-linux-amd64-v1.144.0.tar.gz
# Run single-node VictoriaMetrics with the given scrape.yaml
./victoria-metrics-prod -promscrape.config=scrape.yaml

View File

@@ -37,7 +37,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/
Flag value can be read from the given file when using -deleteAuthKey=file:///abs/path/to/file or -deleteAuthKey=file://./relative/path/to/file.
Flag value can be read from the given http/https url when using -deleteAuthKey=http://host/path or -deleteAuthKey=https://host/path
-denyQueriesOutsideRetention
Whether to deny queries outside the configured -retentionPeriod. When set, then /api/v1/query_range would return '503 Service Unavailable' error for queries with 'from' value outside -retentionPeriod. This may be useful when multiple data sources with distinct retentions are hidden behind query-tee
Whether to deny queries outside the configured -retentionPeriod and -futureRetention. When set, then /api/v1/query_range will return an error for queries with 'from' value outside -retentionPeriod or 'to' value beyond -futureRetention. This may be useful when multiple data sources with distinct retentions are hidden behind query-tee
-denyQueryTracing
Whether to disable the ability to trace queries. See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#query-tracing
-disablePerDayIndex
@@ -198,7 +198,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/
-maxLabelsPerTimeseries int
The maximum number of labels per time series to be accepted. Series with superfluous labels are ignored. In this case the vm_rows_ignored_total{reason="too_many_labels"} metric at /metrics page is incremented (default 40)
-memory.allowedBytes size
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage. The process may behave unexpectedly if this flag is set too small (e.g., 1 byte).
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
-memory.allowedPercent float
Allowed percent of system memory VictoriaMetrics caches may occupy. See also -memory.allowedBytes. Too low a value may increase cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache which will result in higher disk IO usage (default 60)
@@ -217,9 +217,19 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 67108864)
-opentelemetry.convertMetricNamesToPrometheus
Whether to convert only metric names into Prometheus-compatible format for the metrics ingested via OpenTelemetry protocol; see https://docs.victoriametrics.com/victoriametrics/integrations/opentelemetry/
-opentelemetry.ignoreResourceAttributes array
Control which resource attributes to ignore, can only be set when 'opentelemetry.promoteAllResourceAttributes' is true.
-opentelemetry.labelNameUnderscoreSanitization
Whether to enable prepending of 'key' to labels starting with '_' when -opentelemetry.usePrometheusNaming is enabled. Reserved labels starting with '__' are not modified. See https://docs.victoriametrics.com/victoriametrics/integrations/opentelemetry/ (default true)
-opentelemetry.maxRequestSize size
The maximum size in bytes of a single OpenTelemetry request
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 67108864)
-opentelemetry.promoteAllResourceAttributes
Whether to promote all resource attributes to labels, except for the ones configured with 'opentelemetry.ignoreResourceAttributes'.
-opentelemetry.promoteResourceAttributes array
Promote specific list of resource attributes to labels.
-opentelemetry.promoteScopeMetadata
Whether to promote OTel scope metadata (i.e. name, version, schema URL, and attributes) to metric labels.
-opentelemetry.usePrometheusNaming
Whether to convert metric names and labels into Prometheus-compatible format for the metrics ingested via OpenTelemetry protocol; see https://docs.victoriametrics.com/victoriametrics/integrations/opentelemetry/
-opentsdbHTTPListenAddr string

View File

@@ -169,7 +169,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/vmagent/ .
-maxLabelsPerTimeseries int
The maximum number of labels per time series to be accepted. Series with superfluous labels are ignored. In this case the vm_rows_ignored_total{reason="too_many_labels"} metric at /metrics page is incremented
-memory.allowedBytes size
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage. The process may behave unexpectedly if this flag is set too small (e.g., 1 byte).
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
-memory.allowedPercent float
Allowed percent of system memory VictoriaMetrics caches may occupy. See also -memory.allowedBytes. Too low a value may increase cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache which will result in higher disk IO usage (default 60)
@@ -184,9 +184,19 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/vmagent/ .
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 67108864)
-opentelemetry.convertMetricNamesToPrometheus
Whether to convert only metric names into Prometheus-compatible format for the metrics ingested via OpenTelemetry protocol; see https://docs.victoriametrics.com/victoriametrics/integrations/opentelemetry/
-opentelemetry.ignoreResourceAttributes array
Control which resource attributes to ignore, can only be set when 'opentelemetry.promoteAllResourceAttributes' is true.
-opentelemetry.labelNameUnderscoreSanitization
Whether to enable prepending of 'key' to labels starting with '_' when -opentelemetry.usePrometheusNaming is enabled. Reserved labels starting with '__' are not modified. See https://docs.victoriametrics.com/victoriametrics/integrations/opentelemetry/ (default true)
-opentelemetry.maxRequestSize size
The maximum size in bytes of a single OpenTelemetry request
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 67108864)
-opentelemetry.promoteAllResourceAttributes
Whether to promote all resource attributes to labels, except for the ones configured with 'opentelemetry.ignoreResourceAttributes'.
-opentelemetry.promoteResourceAttributes array
Promote specific list of resource attributes to labels.
-opentelemetry.promoteScopeMetadata
Whether to promote OTel scope metadata (i.e. name, version, schema URL, and attributes) to metric labels.
-opentelemetry.usePrometheusNaming
Whether to convert metric names and labels into Prometheus-compatible format for the metrics ingested via OpenTelemetry protocol; see https://docs.victoriametrics.com/victoriametrics/integrations/opentelemetry/
-opentsdbHTTPListenAddr string

View File

@@ -166,7 +166,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/vmalert/ .
-loggerWarnsPerSecondLimit int
Per-second limit on the number of WARN messages. If more than the given number of warns are emitted per second, then the remaining warns are suppressed. Zero values disable the rate limit
-memory.allowedBytes size
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage. The process may behave unexpectedly if this flag is set too small (e.g., 1 byte).
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
-memory.allowedPercent float
Allowed percent of system memory VictoriaMetrics caches may occupy. See also -memory.allowedBytes. Too low a value may increase cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache which will result in higher disk IO usage (default 60)

View File

@@ -135,7 +135,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/vmauth/ .
The maximum request body size to buffer in memory for potential retries at other backends. Request bodies larger than this size cannot be retried if the backend fails. Zero or negative value disables retries. See also -requestBufferSize
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 16384)
-memory.allowedBytes size
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage. The process may behave unexpectedly if this flag is set too small (e.g., 1 byte).
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
-memory.allowedPercent float
Allowed percent of system memory VictoriaMetrics caches may occupy. See also -memory.allowedBytes. Too low a value may increase cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache which will result in higher disk IO usage (default 60)

View File

@@ -34,9 +34,9 @@ vmctl command-line tool is available as:
Download and unpack vmctl:
```sh
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.143.0/vmutils-darwin-arm64-v1.143.0.tar.gz
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.144.0/vmutils-darwin-arm64-v1.144.0.tar.gz
tar xzf vmutils-darwin-arm64-v1.143.0.tar.gz
tar xzf vmutils-darwin-arm64-v1.144.0.tar.gz
```
Once binary is unpacked, see the full list of supported modes by running the following command:

View File

@@ -169,7 +169,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/cluster-victori
-maxLabelsPerTimeseries int
The maximum number of labels per time series to be accepted. Series with superfluous labels are ignored. In this case the vm_rows_ignored_total{reason="too_many_labels"} metric at /metrics page is incremented (default 40)
-memory.allowedBytes size
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage. The process may behave unexpectedly if this flag is set too small (e.g., 1 byte).
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
-memory.allowedPercent float
Allowed percent of system memory VictoriaMetrics caches may occupy. See also -memory.allowedBytes. Too low a value may increase cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache which will result in higher disk IO usage (default 60)
@@ -184,9 +184,19 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/cluster-victori
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 67108864)
-opentelemetry.convertMetricNamesToPrometheus
Whether to convert only metric names into Prometheus-compatible format for the metrics ingested via OpenTelemetry protocol; see https://docs.victoriametrics.com/victoriametrics/integrations/opentelemetry/
-opentelemetry.ignoreResourceAttributes array
Control which resource attributes to ignore, can only be set when 'opentelemetry.promoteAllResourceAttributes' is true.
-opentelemetry.labelNameUnderscoreSanitization
Whether to enable prepending of 'key' to labels starting with '_' when -opentelemetry.usePrometheusNaming is enabled. Reserved labels starting with '__' are not modified. See https://docs.victoriametrics.com/victoriametrics/integrations/opentelemetry/ (default true)
-opentelemetry.maxRequestSize size
The maximum size in bytes of a single OpenTelemetry request
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 67108864)
-opentelemetry.promoteAllResourceAttributes
Whether to promote all resource attributes to labels, except for the ones configured with 'opentelemetry.ignoreResourceAttributes'.
-opentelemetry.promoteResourceAttributes array
Promote specific list of resource attributes to labels.
-opentelemetry.promoteScopeMetadata
Whether to promote OTel scope metadata (i.e. name, version, schema URL, and attributes) to metric labels.
-opentelemetry.usePrometheusNaming
Whether to convert metric names and labels into Prometheus-compatible format for the metrics ingested via OpenTelemetry protocol; see https://docs.victoriametrics.com/victoriametrics/integrations/opentelemetry/
-opentsdbHTTPListenAddr string

View File

@@ -18,6 +18,20 @@ sitemap:
Whether to skip verification of TLS certificates provided by -storageNode nodes if -cluster.tls flag is set. Note that disabled TLS certificate verification breaks security. This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/victoriametrics/enterprise/
-cluster.tlsKeyFile string
Path to client-side TLS key file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/victoriametrics/enterprise/
-clusternative.tls
Whether to use TLS when accepting connections at -clusternativeListenAddr. See https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/victoriametrics/enterprise/
-clusternative.tlsCAFile string
Path to TLS CA file to use for verifying certificates provided by vminsert, which connects at -clusternativeListenAddr if -clusternative.tls flag is set. By default system CA is used. See https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/victoriametrics/enterprise/
-clusternative.tlsCertFile string
Path to server-side TLS certificate file to use when accepting connections at -clusternativeListenAddr if -clusternative.tls flag is set. See https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/victoriametrics/enterprise/
-clusternative.tlsCipherSuites array
Optional list of TLS cipher suites used for connections at -clusternativeListenAddr if -clusternative.tls flag is set. See the list of supported cipher suites at https://pkg.go.dev/crypto/tls#pkg-constants . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/victoriametrics/enterprise/
Supports an array of values separated by comma or specified via multiple flags.
Each array item can contain comma inside single-quoted or double-quoted string, {}, [] and () braces.
-clusternative.tlsInsecureSkipVerify
Whether to skip verification of TLS certificates provided by vminsert, which connects to -clusternativeListenAddr if -clusternative.tls flag is set. Note that disabled TLS certificate verification breaks security. This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/victoriametrics/enterprise/
-clusternative.tlsKeyFile string
Path to server-side TLS key file to use when accepting connections at -clusternativeListenAddr if -clusternative.tls flag is set. See https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/victoriametrics/enterprise/
-eula
Deprecated, please use -license or -licenseFile flags instead. By specifying this flag, you confirm that you have an enterprise license and accept the ESA https://victoriametrics.com/legal/esa/ . This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/victoriametrics/enterprise/
-license string

View File

@@ -126,7 +126,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/cluster-victori
-loggerWarnsPerSecondLimit int
Per-second limit on the number of WARN messages. If more than the given number of warns are emitted per second, then the remaining warns are suppressed. Zero values disable the rate limit
-memory.allowedBytes size
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage. The process may behave unexpectedly if this flag is set too small (e.g., 1 byte).
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
-memory.allowedPercent float
Allowed percent of system memory VictoriaMetrics caches may occupy. See also -memory.allowedBytes. Too low a value may increase cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache which will result in higher disk IO usage (default 60)

View File

@@ -22,7 +22,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/cluster-victori
-dedup.minScrapeInterval duration
Leave only the last sample in every time series per each discrete interval equal to -dedup.minScrapeInterval > 0. See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#deduplication for details
-denyQueriesOutsideRetention
Whether to deny queries outside of the configured -retentionPeriod. When set, then /api/v1/query_range would return '503 Service Unavailable' error for queries with 'from' value outside -retentionPeriod. This may be useful when multiple data sources with distinct retentions are hidden behind query-tee
Whether to deny queries outside the configured -retentionPeriod and -futureRetention. When set, then /api/v1/query_range will return an error for queries with 'from' value outside -retentionPeriod or 'to' value beyond -futureRetention. This may be useful when multiple data sources with distinct retentions are hidden behind query-tee
-denyQueryTracing
Whether to disable the ability to trace queries. See https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#query-tracing
-disablePerDayIndex
@@ -131,7 +131,7 @@ See the docs at https://docs.victoriametrics.com/victoriametrics/cluster-victori
-maxConcurrentInserts int
The maximum number of concurrent insert requests. Set higher value when clients send data over slow networks. Default value depends on the number of available CPU cores. It should work fine in most cases since it minimizes resource usage. See also -insert.maxQueueDuration (default 2*cgroup.AvailableCPUs())
-memory.allowedBytes size
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage. The process may behave unexpectedly if this flag is set too small (e.g., 1 byte).
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
-memory.allowedPercent float
Allowed percent of system memory VictoriaMetrics caches may occupy. See also -memory.allowedBytes. Too low a value may increase cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache which will result in higher disk IO usage (default 60)

10
go.mod
View File

@@ -33,9 +33,9 @@ require (
github.com/valyala/gozstd v1.24.0
github.com/valyala/histogram v1.2.0
github.com/valyala/quicktemplate v1.8.0
golang.org/x/net v0.53.0
golang.org/x/net v0.55.0
golang.org/x/oauth2 v0.36.0
golang.org/x/sys v0.43.0
golang.org/x/sys v0.45.0
google.golang.org/api v0.276.0
gopkg.in/yaml.v2 v2.4.0
)
@@ -155,11 +155,11 @@ require (
go.uber.org/zap v1.27.1 // indirect
go.yaml.in/yaml/v2 v2.4.4 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto v0.50.0 // indirect
golang.org/x/crypto v0.51.0 // indirect
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f // indirect
golang.org/x/sync v0.20.0 // indirect
golang.org/x/term v0.42.0 // indirect
golang.org/x/text v0.36.0 // indirect
golang.org/x/term v0.43.0 // indirect
golang.org/x/text v0.37.0 // indirect
golang.org/x/time v0.15.0 // indirect
google.golang.org/genproto v0.0.0-20260414002931-afd174a4e478 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260414002931-afd174a4e478 // indirect

22
go.sum
View File

@@ -376,8 +376,6 @@ github.com/prometheus/otlptranslator v1.0.0 h1:s0LJW/iN9dkIH+EnhiD3BlkkP5QVIUVEo
github.com/prometheus/otlptranslator v1.0.0/go.mod h1:vRYWnXvI6aWGpsdY/mOT/cbeVRBlPWtBNDb7kGR3uKM=
github.com/prometheus/procfs v0.20.1 h1:XwbrGOIplXW/AU3YhIhLODXMJYyC1isLFfYCsTEycfc=
github.com/prometheus/procfs v0.20.1/go.mod h1:o9EMBZGRyvDrSPH1RqdxhojkuXstoe4UlK79eF5TGGo=
github.com/prometheus/prometheus v0.311.2 h1:6fBxp93y08GAZGNT1o3bIhgV/AMYvBFfU+ltDNEsHg8=
github.com/prometheus/prometheus v0.311.2/go.mod h1:gjsCxTKtHO1Q8T9333u1s+lUR1OjPyM7ruuGH8RvVyo=
github.com/prometheus/prometheus v0.311.3 h1:3IrVxQv6v5i/ZCGi6OrYeBhtCwaPTn6Z3DYruXoYm3M=
github.com/prometheus/prometheus v0.311.3/go.mod h1:gjsCxTKtHO1Q8T9333u1s+lUR1OjPyM7ruuGH8RvVyo=
github.com/prometheus/sigv4 v0.4.1 h1:EIc3j+8NBea9u1iV6O5ZAN8uvPq2xOIUPcqCTivHuXs=
@@ -511,8 +509,8 @@ go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
golang.org/x/crypto v0.51.0 h1:IBPXwPfKxY7cWQZ38ZCIRPI50YLeevDLlLnyC5wRGTI=
golang.org/x/crypto v0.51.0/go.mod h1:8AdwkbraGNABw2kOX6YFPs3WM22XqI4EXEd8g+x7Oc8=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f h1:W3F4c+6OLc6H2lb//N1q4WpJkhzJCK5J6kUi1NTVXfM=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f/go.mod h1:J1xhfL/vlindoeF/aINzNzt2Bket5bjo9sdOYzOsU80=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
@@ -523,8 +521,8 @@ golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.53.0 h1:d+qAbo5L0orcWAr0a9JweQpjXF19LMXJE8Ey7hwOdUA=
golang.org/x/net v0.53.0/go.mod h1:JvMuJH7rrdiCfbeHoo3fCQU24Lf5JJwT9W3sJFulfgs=
golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8=
golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -536,14 +534,14 @@ golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.42.0 h1:UiKe+zDFmJobeJ5ggPwOshJIVt6/Ft0rcfrXZDLWAWY=
golang.org/x/term v0.42.0/go.mod h1:Dq/D+snpsbazcBG5+F9Q1n2rXV8Ma+71xEjTRufARgY=
golang.org/x/sys v0.45.0 h1:dO4czNzziLiiXplLQgBCEpCvXQ3dnkn0SdaZSYdQ+FY=
golang.org/x/sys v0.45.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.43.0 h1:S4RLU2sB31O/NCl+zFN9Aru9A/Cq2aqKpTZJ6B+DwT4=
golang.org/x/term v0.43.0/go.mod h1:lrhlHNdQJHO+1qVYiHfFKVuVioJIheAc3fBSMFYEIsk=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=

View File

@@ -51,8 +51,6 @@ func TestGetTimeSuccess(t *testing.T) {
f("-292273086-05-16T16:47:06Z", minTimeMsecs)
f("292277025-08-18T07:12:54.999999999Z", maxTimeMsecs)
f("1562529662.324", 1562529662324)
f("-9223372036.854", minTimeMsecs)
f("-9223372036.855", minTimeMsecs)
f("1223372036.855", 1223372036855)
}
@@ -85,4 +83,8 @@ func TestGetTimeError(t *testing.T) {
f("292277025-08-18T07:12:54.999999998Z")
f("123md")
f("-12.3md")
// relative duration that resolves to a timestamp before 1970
f("-9223372036.854")
f("-9223372036.855")
}

View File

@@ -14,6 +14,14 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/lib/promutil"
)
// DecodeMetricsOptions defines options for DecodeMetricsData
type DecodeMetricsOptions struct {
DisableScopeMetadata bool
DisableResourceAttributes bool
// ResourceAttributesList stored a list of resource attributes to ignore or promote based on the value of DisableResourceAttributes.
ResourceAttributesList map[string]struct{}
}
// MetricPusher must push the parsed samples and metric metadata to the underlying storage.
type MetricPusher interface {
// PushSample must store a sample with the given args.
@@ -73,8 +81,8 @@ func (r *MetricsData) marshalProtobuf(mm *easyproto.MessageMarshaler) {
}
}
// DecodeMetricsData decodes metricsData from src and sends the decoded data to mp.
func DecodeMetricsData(src []byte, mp MetricPusher) (err error) {
// DecodeMetricsData decodes metricsData with given options from src and sends the decoded data to mp.
func DecodeMetricsData(src []byte, mp MetricPusher, options DecodeMetricsOptions) (err error) {
// See https://github.com/open-telemetry/opentelemetry-proto/blob/049d4332834935792fd4dbd392ecd31904f99ba2/opentelemetry/proto/metrics/v1/metrics.proto#L56
//
// message MetricsData {
@@ -96,7 +104,7 @@ func DecodeMetricsData(src []byte, mp MetricPusher) (err error) {
if !ok {
return fmt.Errorf("cannot read ResourceMetrics data")
}
if err := dctx.decodeResourceMetrics(data); err != nil {
if err := dctx.decodeResourceMetrics(data, options); err != nil {
return fmt.Errorf("cannot unmarshal ResourceMetrics: %w", err)
}
}
@@ -121,7 +129,7 @@ func (rm *ResourceMetrics) marshalProtobuf(mm *easyproto.MessageMarshaler) {
}
}
func (dctx *decoderContext) decodeResourceMetrics(src []byte) error {
func (dctx *decoderContext) decodeResourceMetrics(src []byte, options DecodeMetricsOptions) error {
// See https://github.com/open-telemetry/opentelemetry-proto/blob/049d4332834935792fd4dbd392ecd31904f99ba2/opentelemetry/proto/metrics/v1/metrics.proto#L66
//
// message ResourceMetrics {
@@ -137,7 +145,7 @@ func (dctx *decoderContext) decodeResourceMetrics(src []byte) error {
return fmt.Errorf("cannot read Resource data: %w", err)
}
if ok {
if err := dctx.decodeResource(resourceData); err != nil {
if err := dctx.decodeResource(resourceData, options.DisableResourceAttributes, options.ResourceAttributesList); err != nil {
return fmt.Errorf("cannot decode Resource: %w", err)
}
}
@@ -157,7 +165,7 @@ func (dctx *decoderContext) decodeResourceMetrics(src []byte) error {
return fmt.Errorf("cannot read ScopeMetrics data")
}
if err := dctx.decodeScopeMetrics(data); err != nil {
if err := dctx.decodeScopeMetrics(data, options.DisableScopeMetadata); err != nil {
return fmt.Errorf("cannot unmarshal ScopeMetrics: %w", err)
}
@@ -180,7 +188,7 @@ func (r *Resource) marshalProtobuf(mm *easyproto.MessageMarshaler) {
}
}
func (dctx *decoderContext) decodeResource(src []byte) (err error) {
func (dctx *decoderContext) decodeResource(src []byte, disableResourceAttributes bool, attributeKeys map[string]struct{}) (err error) {
// See https://github.com/open-telemetry/opentelemetry-proto/blob/049d4332834935792fd4dbd392ecd31904f99ba2/opentelemetry/proto/resource/v1/resource.proto#L28
//
// message Resource {
@@ -199,7 +207,34 @@ func (dctx *decoderContext) decodeResource(src []byte) (err error) {
if !ok {
return fmt.Errorf("cannot read Attributes")
}
if err := decodeKeyValue(data, &dctx.ls, &dctx.fb, ""); err != nil {
keySuffix, ok, err := easyproto.GetString(data, 1)
if err != nil {
return fmt.Errorf("cannot find Key in KeyValue: %w", err)
}
if !ok {
// Key is missing, skip it.
// See https://github.com/VictoriaMetrics/VictoriaLogs/issues/869#issuecomment-3631307996
continue
}
if _, ok := attributeKeys[keySuffix]; ok != disableResourceAttributes {
// Skip the attribute if:
// 1. it is in the list of ignore attributes when disableResourceAttributes is false,
// 2. it isn't in the list of promote attributes when disableResourceAttributes is true.
continue
}
key := dctx.fb.formatSubFieldName("", keySuffix)
// Decode value
value, ok, err := easyproto.GetMessageData(data, 2)
if err != nil {
return fmt.Errorf("cannot find Value in KeyValue: %w", err)
}
if !ok {
// Value is null, skip it.
continue
}
if err := decodeAnyValue(value, &dctx.ls, &dctx.fb, key); err != nil {
return fmt.Errorf("cannot unmarshal Attributes: %w", err)
}
}
@@ -257,7 +292,6 @@ func decodeKeyValue(src []byte, ls *promutil.Labels, fb *fmtBuffer, keyPrefix st
if err := decodeAnyValue(valueData, ls, fb, key); err != nil {
return fmt.Errorf("cannot decode AnyValue: %w", err)
}
return nil
}
@@ -452,7 +486,7 @@ func (sm *ScopeMetrics) marshalProtobuf(mm *easyproto.MessageMarshaler) {
}
}
func (dctx *decoderContext) decodeScopeMetrics(src []byte) error {
func (dctx *decoderContext) decodeScopeMetrics(src []byte, disableScopeMetadata bool) error {
// See https://github.com/open-telemetry/opentelemetry-proto/blob/049d4332834935792fd4dbd392ecd31904f99ba2/opentelemetry/proto/metrics/v1/metrics.proto#L86
//
// message ScopeMetrics {
@@ -460,19 +494,22 @@ func (dctx *decoderContext) decodeScopeMetrics(src []byte) error {
// repeated Metric metrics = 2;
// }
scopeData, ok, err := easyproto.GetMessageData(src, 1)
if err != nil {
return fmt.Errorf("cannot read InstrumentationScope: %w", err)
}
if ok {
if err := dctx.decodeInstrumentationScope(scopeData); err != nil {
return fmt.Errorf("cannot decode InstrumentationScope: %w", err)
if !disableScopeMetadata {
scopeData, ok, err := easyproto.GetMessageData(src, 1)
if err != nil {
return fmt.Errorf("cannot read InstrumentationScope: %w", err)
}
if ok {
if err := dctx.decodeInstrumentationScope(scopeData); err != nil {
return fmt.Errorf("cannot decode InstrumentationScope: %w", err)
}
}
}
dctxSnapshot := dctx.getSnapshot()
var fc easyproto.FieldContext
var err error
for len(src) > 0 {
src, err = fc.NextField(src)
if err != nil {

View File

@@ -0,0 +1,166 @@
package pb
import (
"reflect"
"testing"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/prompb"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/promutil"
)
type testMetricPusher struct {
samples []testSample
metadata []MetricMetadata
}
type testSample struct {
mm MetricMetadata
suffix string
labels []prompb.Label
ts uint64
value float64
}
func (p *testMetricPusher) PushSample(mm *MetricMetadata, suffix string, ls *promutil.Labels, timestampNsecs uint64, value float64, _ uint32) {
labels := make([]prompb.Label, len(ls.Labels))
copy(labels, ls.Labels)
p.samples = append(p.samples, testSample{
mm: *mm,
suffix: suffix,
labels: labels,
ts: timestampNsecs,
value: value,
})
}
func (p *testMetricPusher) PushMetricMetadata(mm *MetricMetadata) {
p.metadata = append(p.metadata, *mm)
}
func TestDecodeScopeMetrics(t *testing.T) {
scopeName := "my-scope"
scopeVersion := "v1.0"
envVal := "prod"
intVal := int64(1)
md := &MetricsData{
ResourceMetrics: []*ResourceMetrics{
{
Resource: &Resource{
Attributes: []*KeyValue{
{Key: "job", Value: &AnyValue{StringValue: new("vm")}},
{Key: "region", Value: &AnyValue{StringValue: new("us-east-1")}},
},
},
ScopeMetrics: []*ScopeMetrics{
{
Scope: &InstrumentationScope{
Name: &scopeName,
Version: &scopeVersion,
Attributes: []*KeyValue{
{Key: "env", Value: &AnyValue{StringValue: &envVal}},
},
},
Metrics: []*Metric{
{
Name: "my-gauge",
Description: "a test gauge",
Gauge: &Gauge{
DataPoints: []*NumberDataPoint{
{
Attributes: []*KeyValue{{Key: "label1", Value: &AnyValue{StringValue: new("value1")}}},
IntValue: &intVal,
TimeUnixNano: 1000,
},
},
},
},
},
},
},
},
},
}
data := md.MarshalProtobuf(nil)
f := func(options DecodeMetricsOptions, wantLabels map[string]string) {
t.Helper()
mp := &testMetricPusher{}
if err := DecodeMetricsData(data, mp, options); err != nil {
t.Fatalf("DecodeMetricsData error: %v", err)
}
if len(mp.samples) != 1 {
t.Fatalf("expected 1 sample, got %d", len(mp.samples))
}
gotMap := make(map[string]string, len(mp.samples[0].labels))
for _, l := range mp.samples[0].labels {
gotMap[l.Name] = l.Value
}
if !reflect.DeepEqual(gotMap, wantLabels) {
t.Errorf("unexpected labels:\n got: %v\n want: %v", gotMap, wantLabels)
}
}
// PromoteScopeMetadata=true + PromoteAllResourceAttributes=true:
// got all scope labels and resource attrs
f(DecodeMetricsOptions{
DisableScopeMetadata: false,
DisableResourceAttributes: false,
}, map[string]string{
"job": "vm",
"region": "us-east-1",
"scope.name": "my-scope",
"scope.version": "v1.0",
"scope.attributes.env": "prod",
"label1": "value1",
})
// PromoteScopeMetadata=false + PromoteAllResourceAttributes=true:
// got all resource attrs, no scope labels
f(DecodeMetricsOptions{
DisableScopeMetadata: true,
DisableResourceAttributes: false,
}, map[string]string{
"job": "vm",
"region": "us-east-1",
"label1": "value1",
})
// PromoteScopeMetadata=true + PromoteAllResourceAttributes=false + ResourceAttributesList=[region]:
// got only the `region` attr from resource
f(DecodeMetricsOptions{
DisableScopeMetadata: false,
DisableResourceAttributes: true,
ResourceAttributesList: map[string]struct{}{"region": {}},
}, map[string]string{
"region": "us-east-1",
"scope.name": "my-scope",
"scope.version": "v1.0",
"scope.attributes.env": "prod",
"label1": "value1",
})
// PromoteScopeMetadata=true + PromoteAllResourceAttributes=true + ResourceAttributesList=[region]:
// got all resource attrs except `region` (ignore list)
f(DecodeMetricsOptions{
DisableScopeMetadata: false,
DisableResourceAttributes: false,
ResourceAttributesList: map[string]struct{}{"region": {}},
}, map[string]string{
"job": "vm",
"scope.name": "my-scope",
"scope.version": "v1.0",
"scope.attributes.env": "prod",
"label1": "value1",
})
// PromoteScopeMetadata=false + PromoteAllResourceAttributes=false + ResourceAttributesList=[job]:
// got only `job` attr
f(DecodeMetricsOptions{
DisableScopeMetadata: true,
DisableResourceAttributes: true,
ResourceAttributesList: map[string]struct{}{"job": {}},
}, map[string]string{
"job": "vm",
"label1": "value1",
})
}

View File

@@ -1,6 +1,7 @@
package stream
import (
"flag"
"fmt"
"io"
"sync"
@@ -10,13 +11,43 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/decimal"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/flagutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/prompb"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/promutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/protoparser/opentelemetry/pb"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/protoparser/protoparserutil"
)
var maxRequestSize = flagutil.NewBytes("opentelemetry.maxRequestSize", 64*1024*1024, "The maximum size in bytes of a single OpenTelemetry request")
var (
maxRequestSize = flagutil.NewBytes("opentelemetry.maxRequestSize", 64*1024*1024, "The maximum size in bytes of a single OpenTelemetry request")
promoteScopeMetadata = flag.Bool("opentelemetry.promoteScopeMetadata", true, "Whether to promote OTel scope metadata (i.e. name, version, schema URL, and attributes) to metric labels.")
promoteAllResourceAttributes = flag.Bool("opentelemetry.promoteAllResourceAttributes", true, "Whether to promote all resource attributes to labels, except for the ones configured with 'opentelemetry.ignoreResourceAttributes'.")
promoteResourceAttributes = flagutil.NewArrayString("opentelemetry.promoteResourceAttributes", "Promote specific list of resource attributes to labels.")
ignoreResourceAttributes = flagutil.NewArrayString("opentelemetry.ignoreResourceAttributes", "Control which resource attributes to ignore, can only be set when 'opentelemetry.promoteAllResourceAttributes' is true.")
)
// InitDecodeOptions configures decoding settings for the parser
func InitDecodeOptions() {
if *promoteAllResourceAttributes && len(*promoteResourceAttributes) > 0 {
logger.Fatalf("cannot set both '-opentelemetry.promoteAllResourceAttributes' and '-opentelemetry.promoteResourceAttributes'")
}
if !*promoteAllResourceAttributes && len(*ignoreResourceAttributes) > 0 {
logger.Fatalf("'-opentelemetry.ignoreResourceAttributes' can only be set when '-opentelemetry.promoteAllResourceAttributes' is true.")
}
defaultDecodeMetricsOptions.DisableScopeMetadata = !*promoteScopeMetadata
defaultDecodeMetricsOptions.DisableResourceAttributes = !*promoteAllResourceAttributes
attributes := *ignoreResourceAttributes
if !*promoteAllResourceAttributes {
attributes = *promoteResourceAttributes
}
defaultDecodeMetricsOptions.ResourceAttributesList = make(map[string]struct{}, len(attributes))
for _, a := range attributes {
defaultDecodeMetricsOptions.ResourceAttributesList[a] = struct{}{}
}
}
var defaultDecodeMetricsOptions = pb.DecodeMetricsOptions{}
// ParseStream parses OpenTelemetry protobuf or json data from r and calls callback for the parsed rows.
//
@@ -47,7 +78,7 @@ func parseData(data []byte, callback func(tss []prompb.TimeSeries, mms []prompb.
// the flushFunc will be called multiple time if the request is big, to avoid over allocating memory for such request.
wctx.flushFunc = callback
if err := pb.DecodeMetricsData(data, wctx); err != nil {
if err := pb.DecodeMetricsData(data, wctx, defaultDecodeMetricsOptions); err != nil {
return fmt.Errorf("cannot unmarshal request from %d bytes: %w", len(data), err)
}

View File

@@ -228,6 +228,7 @@ func testAssertSearchResult(st *Storage, tr TimeRange, tfs *TagFilters, want []M
var s Search
s.Init(nil, st, []*TagFilters{tfs}, tr, 1e5, noDeadline)
defer s.MustClose()
var mbs []metricBlock
for s.NextMetricBlock() {
var b Block
@@ -241,7 +242,6 @@ func testAssertSearchResult(st *Storage, tr TimeRange, tfs *TagFilters, want []M
if err := s.Error(); err != nil {
return fmt.Errorf("search error: %w", err)
}
s.MustClose()
var got []MetricRow
var mn MetricName

View File

@@ -61,10 +61,11 @@ type Storage struct {
// indexdb rotation.
legacyNextRotationTimestamp atomic.Int64
path string
cachePath string
retentionMsecs int64
futureRetentionMsecs int64
path string
cachePath string
retentionMsecs int64
futureRetentionMsecs int64
denyQueriesOutsideRetention bool
// lock file for exclusive access to the storage on the given path.
flockF *os.File
@@ -161,14 +162,15 @@ type Storage struct {
// OpenOptions optional args for MustOpenStorage
type OpenOptions struct {
Retention time.Duration
FutureRetention time.Duration
MaxHourlySeries int
MaxDailySeries int
DisablePerDayIndex bool
TrackMetricNamesStats bool
IDBPrefillStart time.Duration
LogNewSeries bool
Retention time.Duration
FutureRetention time.Duration
DenyQueriesOutsideRetention bool
MaxHourlySeries int
MaxDailySeries int
DisablePerDayIndex bool
TrackMetricNamesStats bool
IDBPrefillStart time.Duration
LogNewSeries bool
}
// MustOpenStorage opens storage on the given path with the given retentionMsecs.
@@ -190,12 +192,13 @@ func MustOpenStorage(path string, opts OpenOptions) *Storage {
idbPrefillStart = time.Hour
}
s := &Storage{
path: path,
cachePath: filepath.Join(path, cacheDirname),
retentionMsecs: retention.Milliseconds(),
futureRetentionMsecs: futureRetention.Milliseconds(),
stopCh: make(chan struct{}),
idbPrefillStartSeconds: idbPrefillStart.Milliseconds() / 1000,
path: path,
cachePath: filepath.Join(path, cacheDirname),
retentionMsecs: retention.Milliseconds(),
futureRetentionMsecs: futureRetention.Milliseconds(),
denyQueriesOutsideRetention: opts.DenyQueriesOutsideRetention,
stopCh: make(chan struct{}),
idbPrefillStartSeconds: idbPrefillStart.Milliseconds() / 1000,
}
s.logNewSeries.Store(opts.LogNewSeries)
@@ -1225,6 +1228,25 @@ func searchAndMergeUniq(qt *querytracer.Tracer, s *Storage, tr TimeRange, search
return res, nil
}
// checkTimeRange returns an error if time range is outside the allowed
// -retentionPeriod or -futureRetention window when
// -denyQueriesOutsideRetention flag is set
func (s *Storage) checkTimeRange(tr TimeRange) error {
if !s.denyQueriesOutsideRetention {
return nil
}
minTimestamp, maxTimestamp := s.tb.getMinMaxTimestamps()
if minTimestamp <= tr.MinTimestamp && tr.MaxTimestamp <= maxTimestamp {
return nil
}
retention := time.Duration(s.retentionMsecs) * time.Millisecond
futureRetention := time.Duration(s.futureRetentionMsecs) * time.Millisecond
return fmt.Errorf("the given time range %s is outside the allowed -retentionPeriod=%s, -futureRetention=%s "+
"according to -denyQueriesOutsideRetention", &tr, retention, futureRetention)
}
// SearchTSIDs searches the TSIDs that correspond to filters within the given
// time range.
//
@@ -1236,6 +1258,10 @@ func (s *Storage) SearchTSIDs(qt *querytracer.Tracer, tfss []*TagFilters, tr Tim
qt = qt.NewChild("search TSIDs: filters=%s, timeRange=%s, maxMetrics=%d", tfss, &tr, maxMetrics)
defer qt.Done()
if err := s.checkTimeRange(tr); err != nil {
return nil, err
}
search := func(qt *querytracer.Tracer, idb *indexDB, tr TimeRange) ([]TSID, error) {
return idb.SearchTSIDs(qt, tfss, tr, maxMetrics, deadline)
}
@@ -1271,6 +1297,9 @@ func (s *Storage) SearchTSIDs(qt *querytracer.Tracer, tfss []*TagFilters, tr Tim
// MetricName.UnmarshalString().
func (s *Storage) SearchMetricNames(qt *querytracer.Tracer, tfss []*TagFilters, tr TimeRange, maxMetrics int, deadline uint64) ([]string, error) {
qt = qt.NewChild("search metric names: filters=%s, timeRange=%s, maxMetrics: %d", tfss, &tr, maxMetrics)
if err := s.checkTimeRange(tr); err != nil {
return nil, err
}
search := func(qt *querytracer.Tracer, idb *indexDB, tr TimeRange) ([]string, error) {
return idb.SearchMetricNames(qt, tfss, tr, maxMetrics, deadline)
}

View File

@@ -6,6 +6,7 @@ import (
"fmt"
"math/rand"
"path/filepath"
"slices"
"testing"
"testing/synctest"
"time"
@@ -1144,3 +1145,268 @@ func TestStorage_partitionsOutsideRetentionAreRemoved(t *testing.T) {
s.MustClose()
})
}
func TestStorage_denyQueriesOutsideRetention(t *testing.T) {
defer testRemoveAll(t)
tfsAll := NewTagFilters()
if err := tfsAll.Add(nil, []byte(".*"), false, true); err != nil {
t.Fatalf("TagFilters.Add() failed unexpectedly: %v", err)
}
tfssAll := []*TagFilters{tfsAll}
assertData := func(t *testing.T, s *Storage, tr TimeRange, wantData []MetricRow, wantErr bool) {
t.Helper()
err := testAssertSearchResult(s, tr, tfsAll, wantData)
gotErr := err != nil
if gotErr != wantErr {
t.Fatalf("Search: unmet error expectation for timeRange=%v: got %t, want %t (err: %v)", &tr, gotErr, wantErr, err)
}
}
assertMetricNames := func(t *testing.T, s *Storage, tr TimeRange, wantData []string, wantErr bool) {
t.Helper()
metricNames, err := s.SearchMetricNames(nil, tfssAll, tr, 1e9, noDeadline)
gotErr := err != nil
if gotErr != wantErr {
t.Fatalf("SearchMetricNames(): unmet error expectation for timeRange=%v: got %t, want %t (err: %v)", &tr, gotErr, wantErr, err)
}
var gotData []string
for _, name := range metricNames {
var mn MetricName
if err := mn.UnmarshalString(name); err != nil {
t.Fatalf("Could not unmarshal metric name %q: %v", name, err)
}
gotData = append(gotData, string(mn.MetricGroup))
}
if diff := cmp.Diff(wantData, gotData); diff != "" {
t.Fatalf("unexpected metric names (-want, +got):\n%s", diff)
}
}
assertLabelNames := func(t *testing.T, s *Storage, tr TimeRange, wantData []string, wantErr bool) {
t.Helper()
gotData, err := s.SearchLabelNames(nil, nil, tr, 1e9, 1e9, noDeadline)
gotErr := err != nil
if gotErr != wantErr {
t.Fatalf("SearchLabelNames(): unmet error expectation for timeRange=%v: got %t, want %t (err: %v)", &tr, gotErr, wantErr, err)
}
slices.Sort(gotData)
slices.Sort(wantData)
if diff := cmp.Diff(wantData, gotData); diff != "" {
t.Fatalf("unexpected label names (-want, +got):\n%s", diff)
}
}
assertLabelValues := func(t *testing.T, s *Storage, tr TimeRange, wantData []string, wantErr bool) {
t.Helper()
gotData, err := s.SearchLabelValues(nil, "__name__", nil, tr, 1e9, 1e9, noDeadline)
gotErr := err != nil
if gotErr != wantErr {
t.Fatalf("SearchLabelValues(): unmet error expectation for timeRange=%v: got %t, want %t (err: %v)", &tr, gotErr, wantErr, err)
}
slices.Sort(gotData)
slices.Sort(wantData)
if diff := cmp.Diff(wantData, gotData); diff != "" {
t.Fatalf("unexpected label values (-want, +got):\n%s", diff)
}
}
assertTagValueSuffixes := func(t *testing.T, s *Storage, tr TimeRange, wantData []string, wantErr bool) {
t.Helper()
gotData, err := s.SearchTagValueSuffixes(nil, tr, "", "", '.', 1e9, noDeadline)
gotErr := err != nil
if gotErr != wantErr {
t.Fatalf("SearchTagValueSuffixes(): unmet error expectation for timeRange=%v: got %t, want %t (err: %v)", &tr, gotErr, wantErr, err)
}
slices.Sort(gotData)
if diff := cmp.Diff(wantData, gotData); diff != "" {
t.Errorf("unexpected tag value suffixes (-want, +got):\n%s", diff)
}
}
assertGraphitePaths := func(t *testing.T, s *Storage, tr TimeRange, wantData []string, wantErr bool) {
t.Helper()
gotData, err := s.SearchGraphitePaths(nil, tr, []byte("*"), 1e9, noDeadline)
gotErr := err != nil
if gotErr != wantErr {
t.Fatalf("SearchTagValueSuffixes(): unmet error expectation for timeRange=%v: got %t, want %t (err: %v)", &tr, gotErr, wantErr, err)
}
slices.Sort(gotData)
if diff := cmp.Diff(wantData, gotData); diff != "" {
t.Errorf("unexpected graphite paths (-want, +got):\n%s", diff)
}
}
synctest.Test(t, func(t *testing.T) {
// synctests start at 2000-01-01T00:00:00Z
now := time.Now().UTC()
retention := 30 * 24 * time.Hour
futureRetention := 30 * 24 * time.Hour
day := 24 * time.Hour
trMatches := TimeRange{
MinTimestamp: now.Add(-retention).UnixMilli(),
MaxTimestamp: now.Add(futureRetention).UnixMilli(),
}
trContains := TimeRange{
MinTimestamp: now.Add(-(retention + day)).UnixMilli(),
MaxTimestamp: now.Add(futureRetention + day).UnixMilli(),
}
trInside := TimeRange{
MinTimestamp: now.Add(-(retention - day)).UnixMilli(),
MaxTimestamp: now.Add(futureRetention - day).UnixMilli(),
}
trOverlapsLeft := TimeRange{
MinTimestamp: now.Add(-(retention + day)).UnixMilli(),
MaxTimestamp: now.Add(futureRetention - day).UnixMilli(),
}
trOverlapsRight := TimeRange{
MinTimestamp: now.Add(-(retention - day)).UnixMilli(),
MaxTimestamp: now.Add(futureRetention + day).UnixMilli(),
}
trOutsideLeft := TimeRange{
MinTimestamp: now.Add(-(retention + 2*day)).UnixMilli(),
MaxTimestamp: now.Add(-(retention + day)).UnixMilli(),
}
trOutsideRight := TimeRange{
MinTimestamp: now.Add(futureRetention + day).UnixMilli(),
MaxTimestamp: now.Add(futureRetention + 2*day).UnixMilli(),
}
mn := MetricName{
MetricGroup: []byte("metric"),
}
metricNameRaw := mn.marshalRaw(nil)
mr := MetricRow{
MetricNameRaw: metricNameRaw,
Timestamp: now.UnixMilli(),
Value: 123,
}
wantMRs := []MetricRow{mr}
wantMetricNames := []string{"metric"}
wantLabelNames := []string{"__name__"}
emptyLabelNames := []string{}
wantLabelValues := []string{"metric"}
emptyLabelValues := []string{}
wantTagValueSuffixes := []string{"metric"}
emptyTagValueSuffixes := []string{}
wantGraphitePaths := []string{"metric"}
emptyGraphitePaths := []string{}
s := MustOpenStorage(t.Name(), OpenOptions{
Retention: retention,
FutureRetention: futureRetention,
})
s.AddRows([]MetricRow{mr}, defaultPrecisionBits)
s.DebugFlush()
assertData(t, s, trInside, wantMRs, false)
assertData(t, s, trMatches, wantMRs, false)
assertData(t, s, trContains, wantMRs, false)
assertData(t, s, trOverlapsLeft, wantMRs, false)
assertData(t, s, trOverlapsRight, wantMRs, false)
assertData(t, s, trOutsideLeft, nil, false)
assertData(t, s, trOutsideRight, nil, false)
assertMetricNames(t, s, trInside, wantMetricNames, false)
assertMetricNames(t, s, trMatches, wantMetricNames, false)
assertMetricNames(t, s, trContains, wantMetricNames, false)
assertMetricNames(t, s, trOverlapsLeft, wantMetricNames, false)
assertMetricNames(t, s, trOverlapsRight, wantMetricNames, false)
assertMetricNames(t, s, trOutsideLeft, nil, false)
assertMetricNames(t, s, trOutsideRight, nil, false)
assertLabelNames(t, s, trInside, wantLabelNames, false)
assertLabelNames(t, s, trMatches, wantLabelNames, false)
assertLabelNames(t, s, trContains, wantLabelNames, false)
assertLabelNames(t, s, trOverlapsLeft, wantLabelNames, false)
assertLabelNames(t, s, trOverlapsRight, wantLabelNames, false)
assertLabelNames(t, s, trOutsideLeft, emptyLabelNames, false)
assertLabelNames(t, s, trOutsideRight, emptyLabelNames, false)
assertLabelValues(t, s, trInside, wantLabelValues, false)
assertLabelValues(t, s, trMatches, wantLabelValues, false)
assertLabelValues(t, s, trContains, wantLabelValues, false)
assertLabelValues(t, s, trOverlapsLeft, wantLabelValues, false)
assertLabelValues(t, s, trOverlapsRight, wantLabelValues, false)
assertLabelValues(t, s, trOutsideLeft, emptyLabelValues, false)
assertLabelValues(t, s, trOutsideRight, emptyLabelValues, false)
assertTagValueSuffixes(t, s, trInside, wantTagValueSuffixes, false)
assertTagValueSuffixes(t, s, trMatches, wantTagValueSuffixes, false)
assertTagValueSuffixes(t, s, trContains, wantTagValueSuffixes, false)
assertTagValueSuffixes(t, s, trOverlapsLeft, wantTagValueSuffixes, false)
assertTagValueSuffixes(t, s, trOverlapsRight, wantTagValueSuffixes, false)
assertTagValueSuffixes(t, s, trOutsideLeft, emptyTagValueSuffixes, false)
assertTagValueSuffixes(t, s, trOutsideRight, emptyTagValueSuffixes, false)
assertGraphitePaths(t, s, trInside, wantGraphitePaths, false)
assertGraphitePaths(t, s, trMatches, wantGraphitePaths, false)
assertGraphitePaths(t, s, trContains, wantGraphitePaths, false)
assertGraphitePaths(t, s, trOverlapsLeft, wantGraphitePaths, false)
assertGraphitePaths(t, s, trOverlapsRight, wantGraphitePaths, false)
assertGraphitePaths(t, s, trOutsideLeft, emptyGraphitePaths, false)
assertGraphitePaths(t, s, trOutsideRight, emptyGraphitePaths, false)
// Restart storage with DenyQueriesOutsideRetention on.
s.MustClose()
s = MustOpenStorage(t.Name(), OpenOptions{
Retention: retention,
FutureRetention: futureRetention,
DenyQueriesOutsideRetention: true,
})
defer s.MustClose()
assertData(t, s, trInside, wantMRs, false)
assertData(t, s, trMatches, wantMRs, false)
assertData(t, s, trContains, nil, true)
assertData(t, s, trOverlapsLeft, nil, true)
assertData(t, s, trOverlapsRight, nil, true)
assertData(t, s, trOutsideLeft, nil, true)
assertData(t, s, trOutsideRight, nil, true)
assertMetricNames(t, s, trInside, wantMetricNames, false)
assertMetricNames(t, s, trMatches, wantMetricNames, false)
assertMetricNames(t, s, trContains, nil, true)
assertMetricNames(t, s, trOverlapsLeft, nil, true)
assertMetricNames(t, s, trOverlapsRight, nil, true)
assertMetricNames(t, s, trOutsideLeft, nil, true)
assertMetricNames(t, s, trOutsideRight, nil, true)
// DenyQueriesOutsideRetention does not apply to searching label names.
assertLabelNames(t, s, trInside, wantLabelNames, false)
assertLabelNames(t, s, trMatches, wantLabelNames, false)
assertLabelNames(t, s, trContains, wantLabelNames, false)
assertLabelNames(t, s, trOverlapsLeft, wantLabelNames, false)
assertLabelNames(t, s, trOverlapsRight, wantLabelNames, false)
assertLabelNames(t, s, trOutsideLeft, emptyLabelNames, false)
assertLabelNames(t, s, trOutsideRight, emptyLabelNames, false)
// DenyQueriesOutsideRetention does not apply to searching label values.
assertLabelValues(t, s, trInside, wantLabelValues, false)
assertLabelValues(t, s, trMatches, wantLabelValues, false)
assertLabelValues(t, s, trContains, wantLabelValues, false)
assertLabelValues(t, s, trOverlapsLeft, wantLabelValues, false)
assertLabelValues(t, s, trOverlapsRight, wantLabelValues, false)
assertLabelValues(t, s, trOutsideLeft, emptyLabelValues, false)
assertLabelValues(t, s, trOutsideRight, emptyLabelValues, false)
// DenyQueriesOutsideRetention does not apply to searching tag value suffixes.
assertTagValueSuffixes(t, s, trInside, wantTagValueSuffixes, false)
assertTagValueSuffixes(t, s, trMatches, wantTagValueSuffixes, false)
assertTagValueSuffixes(t, s, trContains, wantTagValueSuffixes, false)
assertTagValueSuffixes(t, s, trOverlapsLeft, wantTagValueSuffixes, false)
assertTagValueSuffixes(t, s, trOverlapsRight, wantTagValueSuffixes, false)
assertTagValueSuffixes(t, s, trOutsideLeft, emptyTagValueSuffixes, false)
assertTagValueSuffixes(t, s, trOutsideRight, emptyTagValueSuffixes, false)
// DenyQueriesOutsideRetention does not apply to searching graphite paths.
assertGraphitePaths(t, s, trInside, wantGraphitePaths, false)
assertGraphitePaths(t, s, trMatches, wantGraphitePaths, false)
assertGraphitePaths(t, s, trContains, wantGraphitePaths, false)
assertGraphitePaths(t, s, trOverlapsLeft, wantGraphitePaths, false)
assertGraphitePaths(t, s, trOverlapsRight, wantGraphitePaths, false)
assertGraphitePaths(t, s, trOutsideLeft, emptyGraphitePaths, false)
assertGraphitePaths(t, s, trOutsideRight, emptyGraphitePaths, false)
})
}

View File

@@ -1,16 +1,25 @@
package timeutil
import (
"fmt"
"time"
"github.com/VictoriaMetrics/metricsql"
)
var (
minDuration = time.Duration(minValidMilli * time.Millisecond)
maxDuration = time.Duration(maxValidMilli * time.Millisecond)
)
// ParseDuration parses duration string in Prometheus format
func ParseDuration(s string) (time.Duration, error) {
ms, err := metricsql.DurationValue(s, 0)
if err != nil {
return 0, err
}
if ms < minValidMilli || maxValidMilli < ms {
return 0, fmt.Errorf("duration %q must be in the range [%v, %v]", s, minDuration, maxDuration)
}
return time.Duration(ms) * time.Millisecond, nil
}

View File

@@ -1,6 +1,7 @@
package timeutil
import (
"fmt"
"testing"
"time"
)
@@ -27,3 +28,126 @@ func TestParseDuration(t *testing.T) {
f("-1m30s", -(time.Minute + time.Second*30))
f("1d-4h", time.Hour*20)
}
func TestParseDurationLimits(t *testing.T) {
f := func(s string, want time.Duration) {
t.Helper()
got, err := ParseDuration(s)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got != want {
t.Fatalf("unexpected result: got %v, want %v", got, want)
}
}
var s string
var want time.Duration
s = fmt.Sprintf("%dms", int64(minValidMilli))
f(s, minDuration)
s = fmt.Sprintf("%dms", int64(maxValidMilli))
f(s, maxDuration)
s = fmt.Sprintf("%ds", int64(minValidSecond))
want = minValidSecond * time.Second
f(s, want)
s = fmt.Sprintf("%ds", int64(maxValidSecond))
want = maxValidSecond * time.Second
f(s, want)
// When no unit is specified, seconds are assumed.
s = fmt.Sprintf("%d", int64(minValidSecond))
want = minValidSecond * time.Second
f(s, want)
s = fmt.Sprintf("%d", int64(maxValidSecond))
want = maxValidSecond * time.Second
f(s, want)
minValidMinute := int64(minValidSecond) / 60
maxValidMinute := int64(maxValidSecond) / 60
s = fmt.Sprintf("%dm", minValidMinute)
want = time.Duration(minValidMinute) * time.Minute
f(s, want)
s = fmt.Sprintf("%dm", maxValidMinute)
want = time.Duration(maxValidMinute) * time.Minute
f(s, want)
minValidHour := minValidMinute / 60
maxValidHour := maxValidMinute / 60
s = fmt.Sprintf("%dh", minValidHour)
want = time.Duration(minValidHour) * time.Hour
f(s, want)
s = fmt.Sprintf("%dh", maxValidHour)
want = time.Duration(maxValidHour) * time.Hour
f(s, want)
minValidDay := minValidHour / 24
maxValidDay := maxValidHour / 24
s = fmt.Sprintf("%dd", minValidDay)
want = time.Duration(minValidDay) * 24 * time.Hour
f(s, want)
s = fmt.Sprintf("%dd", maxValidDay)
want = time.Duration(maxValidDay) * 24 * time.Hour
f(s, want)
minValidWeek := minValidDay / 7
maxValidWeek := maxValidDay / 7
s = fmt.Sprintf("%dw", minValidWeek)
want = time.Duration(minValidWeek) * 7 * 24 * time.Hour
f(s, want)
s = fmt.Sprintf("%dw", maxValidWeek)
want = time.Duration(maxValidWeek) * 7 * 24 * time.Hour
f(s, want)
minValidYear := minValidDay / 365
maxValidYear := maxValidDay / 365
s = fmt.Sprintf("%dy", minValidYear)
want = time.Duration(minValidYear) * 365 * 24 * time.Hour
f(s, want)
s = fmt.Sprintf("%dy", maxValidYear)
want = time.Duration(maxValidYear) * 365 * 24 * time.Hour
f(s, want)
}
func TestParseDurationOutsideLimits(t *testing.T) {
f := func(s string) {
t.Helper()
got, err := ParseDuration(s)
gotDuration := time.Duration(got) * time.Millisecond
if err == nil {
t.Fatalf("ParseDuration(%s) unexpected result: got %d (%s), want error", s, got, gotDuration)
}
}
f(fmt.Sprintf("%dms", int64(minValidMilli)-1))
f(fmt.Sprintf("%dms", int64(maxValidMilli)+1))
f(fmt.Sprintf("%ds", int64(minValidSecond)-1))
f(fmt.Sprintf("%ds", int64(maxValidSecond)+1))
minValidMinute := int64(minValidSecond)/60 - 1
f(fmt.Sprintf("%dm", minValidMinute))
maxValidMinute := int64(maxValidSecond)/60 + 1
f(fmt.Sprintf("%dm", maxValidMinute))
minValidHour := minValidMinute/60 - 1
f(fmt.Sprintf("%dh", minValidHour))
maxValidHour := maxValidMinute/60 + 2
f(fmt.Sprintf("%dh", maxValidHour))
minValidDay := minValidHour/24 - 1
f(fmt.Sprintf("%dd", minValidDay))
maxValidDay := maxValidHour/24 + 1
f(fmt.Sprintf("%dd", maxValidDay))
minValidWeek := minValidDay/7 - 1
f(fmt.Sprintf("%dw", minValidWeek))
maxValidWeek := maxValidDay/7 + 1
f(fmt.Sprintf("%dw", maxValidWeek))
minValidYear := minValidDay/365 - 1
f(fmt.Sprintf("%dy", minValidYear))
maxValidYear := maxValidDay/365 + 1
f(fmt.Sprintf("%dy", maxValidYear))
}

View File

@@ -74,7 +74,11 @@ func ParseTimeAt(s string, currentTimestamp int64) (int64, error) {
if d > 0 {
d = -d
}
return currentTimestamp + int64(d), nil
nsec := currentTimestamp + int64(d)
if nsec < 0 {
return 0, fmt.Errorf("time %s (%v) must be in the range [%v, %v]", sOrig, time.Unix(0, nsec).UTC(), minTime, maxTime)
}
return nsec, nil
}
if len(s) == 4 {
// Parse YYYY

View File

@@ -1,6 +1,8 @@
package timeutil
import (
"fmt"
"math"
"strings"
"testing"
"time"
@@ -204,10 +206,11 @@ func TestParseTimeAtSuccess(t *testing.T) {
}
func TestParseTimeAtLimits(t *testing.T) {
now := time.Date(2025, 1, 1, 0, 0, 0, 0, time.UTC)
f := func(s string, wantTime time.Time) {
t.Helper()
currentTimestamp := time.Now().UnixNano()
got, err := ParseTimeAt(s, currentTimestamp)
got, err := ParseTimeAt(s, now.UnixNano())
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -226,6 +229,14 @@ func TestParseTimeAtLimits(t *testing.T) {
}
east := location(t, "Etc/GMT-14") // UTC+14:00
west := location(t, "Etc/GMT+12") // UTC-12:00
var s string
// min timestamp
f("0", time.Date(1970, 1, 1, 0, 0, 0, 0, time.UTC))
s = fmt.Sprintf("-%d", now.Unix())
f(s, time.Date(1970, 1, 1, 0, 0, 0, 0, time.UTC))
s = fmt.Sprintf("now-%d", now.Unix())
f(s, time.Date(1970, 1, 1, 0, 0, 0, 0, time.UTC))
// min year
f("1970Z", time.Date(1970, 1, 1, 0, 0, 0, 0, time.UTC))
@@ -286,13 +297,39 @@ func TestParseTimeAtLimits(t *testing.T) {
f("2262-04-11T23:47:16Z", time.Date(2262, 4, 11, 23, 47, 16, 0, time.UTC))
f("2262-04-12T13:47:16+14:00", time.Date(2262, 4, 12, 13, 47, 16, 0, east))
f("2262-04-11T11:47:16-12:00", time.Date(2262, 4, 11, 11, 47, 16, 0, west))
// max timestamp
s = fmt.Sprintf("%d", int64(maxValidSecond))
f(s, time.Date(2262, 4, 11, 23, 47, 16, 0, time.UTC))
s = fmt.Sprintf("%d", int64(maxValidMilli))
f(s, time.Date(2262, 4, 11, 23, 47, 16, 854_000_000, time.UTC))
s = fmt.Sprintf("%d", int64(maxValidMicro))
f(s, time.Date(2262, 4, 11, 23, 47, 16, 854_775_000, time.UTC))
s = fmt.Sprintf("%d", int64(math.MaxInt64))
f(s, time.Date(2262, 4, 11, 23, 47, 16, 854_775_807, time.UTC))
// timestamps beyond max valid second are still valid but are treated as
// milliseconds.
s = fmt.Sprintf("%d", int64(maxValidSecond)+1)
f(s, time.Date(1970, 4, 17, 18, 2, 52, 37_000_000, time.UTC))
// timestamps beyond max valid millisecond are still valid but are treated
// as microseconds.
s = fmt.Sprintf("%d", int64(maxValidMilli)+1)
f(s, time.Date(1970, 4, 17, 18, 2, 52, 36_855_000, time.UTC))
// timestamps beyond max valid microsecond are still valid but are treated
// as nanoseconds.
s = fmt.Sprintf("%d", int64(maxValidMicro)+1)
f(s, time.Date(1970, 4, 17, 18, 2, 52, 36_854_776, time.UTC))
}
func TestParseTimeAtOutsideLimits(t *testing.T) {
now := time.Date(2025, 1, 1, 0, 0, 0, 0, time.UTC)
f := func(s string) {
t.Helper()
currentTimestamp := time.Now().UnixNano()
got, err := ParseTimeAt(s, currentTimestamp)
got, err := ParseTimeAt(s, now.UnixNano())
if err == nil {
t.Fatalf("expected error but got %d", got)
}
@@ -301,6 +338,10 @@ func TestParseTimeAtOutsideLimits(t *testing.T) {
}
}
// min timestamp
f(fmt.Sprintf("-%d", now.Unix()+1))
f(fmt.Sprintf("now-%d", now.Unix()+1))
// min year
f("1969Z")
f("1970+14:00")
@@ -362,6 +403,24 @@ func TestParseTimeAtOutsideLimits(t *testing.T) {
f("2262-04-11T11:47:17-12:00")
}
func TestParseTimeAtOutsideLimits_Nanos(t *testing.T) {
now := time.Date(2025, 1, 1, 0, 0, 0, 0, time.UTC)
f := func(s string) {
t.Helper()
got, err := ParseTimeAt(s, now.UnixNano())
if err == nil {
t.Fatalf("expected error but got %d", got)
}
if !strings.Contains(err.Error(), "cannot parse numeric timestamp") {
t.Fatalf("expected error: %v", err)
}
}
// max unix nano
f(fmt.Sprintf("%d", uint64(math.MaxInt64+1)))
}
func TestParseTimeMsecFailure(t *testing.T) {
f := func(s string) {
t.Helper()

View File

@@ -11,6 +11,7 @@
package hkdf
import (
"crypto/hkdf"
"crypto/hmac"
"errors"
"hash"
@@ -24,15 +25,19 @@ import (
// Expand invocations and different context values. Most common scenarios,
// including the generation of multiple keys, should use New instead.
func Extract(hash func() hash.Hash, secret, salt []byte) []byte {
if salt == nil {
salt = make([]byte, hash().Size())
// Use the stdlib Extract, which disables FIPS 140 enforcement of the HMAC
// key (which in HKDF is the salt). The only possible error is FIPS 140
// enforcement of the hash, which had to panic under this API anyway. We
// don't use the stdlib Expand, because it switched to returning a []byte
// instead of an io.Reader, and Expand uses the HMAC key as a key.
out, err := hkdf.Extract(hash, secret, salt)
if err != nil {
panic(err)
}
extractor := hmac.New(hash, salt)
extractor.Write(secret)
return extractor.Sum(nil)
return out
}
type hkdf struct {
type hkdfReader struct {
expander hash.Hash
size int
@@ -43,7 +48,7 @@ type hkdf struct {
buf []byte
}
func (f *hkdf) Read(p []byte) (int, error) {
func (f *hkdfReader) Read(p []byte) (int, error) {
// Check whether enough data can be generated
need := len(p)
remains := len(f.buf) + int(255-f.counter+1)*f.size
@@ -84,7 +89,7 @@ func (f *hkdf) Read(p []byte) (int, error) {
// 3.3. Most common scenarios will want to use New instead.
func Expand(hash func() hash.Hash, pseudorandomKey, info []byte) io.Reader {
expander := hmac.New(hash, pseudorandomKey)
return &hkdf{expander, expander.Size(), info, 1, nil, nil}
return &hkdfReader{expander, expander.Size(), info, 1, nil, nil}
}
// New returns a Reader, from which keys can be read, using the given hash,

19
vendor/golang.org/x/net/http2/README.md generated vendored Normal file
View File

@@ -0,0 +1,19 @@
This package (golang.org/x/net/http2) is the original source of truth
of the Go HTTP/2 implementation.
As of Go 1.27, the source of truth has moved to the standard library
package net/http/internal/http2.
All new feature development should happen in that package.
Only critical bug fixes and security fixes will be backported to x/net.
The x/net package contains two implementations of the HTTP/2 transport and server:
The original implementation (no longer the source of truth).
A reimplementation of the x/net/http2 APIs in terms of net/http.
This is called "the wrapping implementation", since it wraps net/http.
The original implementation is used when the Go version is less than 1.27.
The wrapping implementation is used when the Go version is at least 1.27.
The build tag "http2legacy" may be set to use the original implementation.

View File

@@ -2,6 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build !(go1.27 && !http2legacy)
// Transport code's client connection pooling.
package http2
@@ -14,18 +16,6 @@ import (
"sync"
)
// ClientConnPool manages a pool of HTTP/2 client connections.
type ClientConnPool interface {
// GetClientConn returns a specific HTTP/2 connection (usually
// a TLS-TCP connection) to an HTTP/2 server. On success, the
// returned ClientConn accounts for the upcoming RoundTrip
// call, so the caller should not omit it. If the caller needs
// to, ClientConn.RoundTrip can be called with a bogus
// new(http.Request) to release the stream reservation.
GetClientConn(req *http.Request, addr string) (*ClientConn, error)
MarkDead(*ClientConn)
}
// clientConnPoolIdleCloser is the interface implemented by ClientConnPool
// implementations which can close their idle connections.
type clientConnPoolIdleCloser interface {

57
vendor/golang.org/x/net/http2/clientconn.go generated vendored Normal file
View File

@@ -0,0 +1,57 @@
// Copyright 2026 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package http2
import (
"context"
"net/http"
)
func (cc *ClientConn) RoundTrip(req *http.Request) (*http.Response, error) {
return cc.roundTrip(req)
}
// SetDoNotReuse marks cc as not reusable for future HTTP requests.
func (cc *ClientConn) SetDoNotReuse() {
cc.setDoNotReuse()
}
// CanTakeNewRequest reports whether the connection can take a new request,
// meaning it has not been closed or received or sent a GOAWAY.
//
// If the caller is going to immediately make a new request on this
// connection, use ReserveNewRequest instead.
func (cc *ClientConn) CanTakeNewRequest() bool {
return cc.canTakeNewRequest()
}
// ReserveNewRequest is like CanTakeNewRequest but also reserves a
// concurrent stream in cc. The reservation is decremented on the
// next call to RoundTrip.
func (cc *ClientConn) ReserveNewRequest() bool {
return cc.reserveNewRequest()
}
// State returns a snapshot of cc's state.
func (cc *ClientConn) State() ClientConnState {
return cc.state()
}
// Shutdown gracefully closes the client connection, waiting for running streams to complete.
func (cc *ClientConn) Shutdown(ctx context.Context) error {
return cc.shutdown(ctx)
}
// Close closes the client connection immediately.
//
// In-flight requests are interrupted. For a graceful shutdown, use Shutdown instead.
func (cc *ClientConn) Close() error {
return cc.close()
}
// Ping sends a PING frame to the server and waits for the ack.
func (cc *ClientConn) Ping(ctx context.Context) error {
return cc.ping(ctx)
}

View File

@@ -2,6 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build !(go1.27 && !http2legacy)
package http2
import (

View File

@@ -6,7 +6,6 @@ package hpack
import (
"fmt"
"strings"
)
// headerFieldTable implements a list of HeaderFields.
@@ -55,16 +54,10 @@ func (t *headerFieldTable) len() int {
// addEntry adds a new entry.
func (t *headerFieldTable) addEntry(f HeaderField) {
// Prevent f from escaping to the heap.
f2 := HeaderField{
Name: strings.Clone(f.Name),
Value: strings.Clone(f.Value),
Sensitive: f.Sensitive,
}
id := uint64(t.len()) + t.evictCount + 1
t.byName[f2.Name] = id
t.byNameValue[pairNameValue{f2.Name, f2.Value}] = id
t.ents = append(t.ents, f2)
t.byName[f.Name] = id
t.byNameValue[pairNameValue{f.Name, f.Value}] = id
t.ents = append(t.ents, f)
}
// evictOldest evicts the n oldest entries in the table.

View File

@@ -195,7 +195,7 @@ func (s SettingID) String() string {
}
// validWireHeaderFieldName reports whether v is a valid header field
// name (key). See httpguts.ValidHeaderName for the base rules.
// name (key). See httpguts.ValidHeaderFieldName for the base rules.
//
// Further, http2 says:
//

View File

@@ -2,6 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build !(go1.27 && !http2legacy)
// TODO: turn off the serve goroutine when idle, so
// an idle conn only has the readFrames goroutine active. (which could
// also be optimized probably to pin less memory in crypto/tls). This
@@ -88,98 +90,6 @@ var (
testHookOnPanic func(sc *serverConn, panicVal interface{}) (rePanic bool)
)
// Server is an HTTP/2 server.
type Server struct {
// MaxHandlers limits the number of http.Handler ServeHTTP goroutines
// which may run at a time over all connections.
// Negative or zero no limit.
// TODO: implement
MaxHandlers int
// MaxConcurrentStreams optionally specifies the number of
// concurrent streams that each client may have open at a
// time. This is unrelated to the number of http.Handler goroutines
// which may be active globally, which is MaxHandlers.
// If zero, MaxConcurrentStreams defaults to at least 100, per
// the HTTP/2 spec's recommendations.
MaxConcurrentStreams uint32
// MaxDecoderHeaderTableSize optionally specifies the http2
// SETTINGS_HEADER_TABLE_SIZE to send in the initial settings frame. It
// informs the remote endpoint of the maximum size of the header compression
// table used to decode header blocks, in octets. If zero, the default value
// of 4096 is used.
MaxDecoderHeaderTableSize uint32
// MaxEncoderHeaderTableSize optionally specifies an upper limit for the
// header compression table used for encoding request headers. Received
// SETTINGS_HEADER_TABLE_SIZE settings are capped at this limit. If zero,
// the default value of 4096 is used.
MaxEncoderHeaderTableSize uint32
// MaxReadFrameSize optionally specifies the largest frame
// this server is willing to read. A valid value is between
// 16k and 16M, inclusive. If zero or otherwise invalid, a
// default value is used.
MaxReadFrameSize uint32
// PermitProhibitedCipherSuites, if true, permits the use of
// cipher suites prohibited by the HTTP/2 spec.
PermitProhibitedCipherSuites bool
// IdleTimeout specifies how long until idle clients should be
// closed with a GOAWAY frame. PING frames are not considered
// activity for the purposes of IdleTimeout.
// If zero or negative, there is no timeout.
IdleTimeout time.Duration
// ReadIdleTimeout is the timeout after which a health check using a ping
// frame will be carried out if no frame is received on the connection.
// If zero, no health check is performed.
ReadIdleTimeout time.Duration
// PingTimeout is the timeout after which the connection will be closed
// if a response to a ping is not received.
// If zero, a default of 15 seconds is used.
PingTimeout time.Duration
// WriteByteTimeout is the timeout after which a connection will be
// closed if no data can be written to it. The timeout begins when data is
// available to write, and is extended whenever any bytes are written.
// If zero or negative, there is no timeout.
WriteByteTimeout time.Duration
// MaxUploadBufferPerConnection is the size of the initial flow
// control window for each connections. The HTTP/2 spec does not
// allow this to be smaller than 65535 or larger than 2^32-1.
// If the value is outside this range, a default value will be
// used instead.
MaxUploadBufferPerConnection int32
// MaxUploadBufferPerStream is the size of the initial flow control
// window for each stream. The HTTP/2 spec does not allow this to
// be larger than 2^32-1. If the value is zero or larger than the
// maximum, a default value will be used instead.
MaxUploadBufferPerStream int32
// NewWriteScheduler constructs a write scheduler for a connection.
// If nil, a default scheduler is chosen.
//
// Deprecated: User-provided write schedulers are deprecated.
NewWriteScheduler func() WriteScheduler
// CountError, if non-nil, is called on HTTP/2 server errors.
// It's intended to increment a metric for monitoring, such
// as an expvar or Prometheus metric.
// The errType consists of only ASCII word characters.
CountError func(errType string)
// Internal state. This is a pointer (rather than embedded directly)
// so that we don't embed a Mutex in this struct, which will make the
// struct non-copyable, which might break some callers.
state *serverInternalState
}
type serverInternalState struct {
mu sync.Mutex
activeConns map[*serverConn]struct{}
@@ -187,6 +97,9 @@ type serverInternalState struct {
// Pool of error channels. This is per-Server rather than global
// because channels can't be reused across synctest bubbles.
errChanPool sync.Pool
// Used in tests.
testNewConn func(*serverConn)
}
func (s *serverInternalState) registerConn(sc *serverConn) {
@@ -239,12 +152,7 @@ func (s *serverInternalState) putErrChan(ch chan error) {
s.errChanPool.Put(ch)
}
// ConfigureServer adds HTTP/2 support to a net/http Server.
//
// The configuration conf may be nil.
//
// ConfigureServer must be called before s begins serving.
func ConfigureServer(s *http.Server, conf *Server) error {
func configureServer(s *http.Server, conf *Server) error {
if s == nil {
panic("nil *http.Server")
}
@@ -349,83 +257,6 @@ func ConfigureServer(s *http.Server, conf *Server) error {
return nil
}
// ServeConnOpts are options for the Server.ServeConn method.
type ServeConnOpts struct {
// Context is the base context to use.
// If nil, context.Background is used.
Context context.Context
// BaseConfig optionally sets the base configuration
// for values. If nil, defaults are used.
BaseConfig *http.Server
// Handler specifies which handler to use for processing
// requests. If nil, BaseConfig.Handler is used. If BaseConfig
// or BaseConfig.Handler is nil, http.DefaultServeMux is used.
Handler http.Handler
// UpgradeRequest is an initial request received on a connection
// undergoing an h2c upgrade. The request body must have been
// completely read from the connection before calling ServeConn,
// and the 101 Switching Protocols response written.
UpgradeRequest *http.Request
// Settings is the decoded contents of the HTTP2-Settings header
// in an h2c upgrade request.
Settings []byte
// SawClientPreface is set if the HTTP/2 connection preface
// has already been read from the connection.
SawClientPreface bool
}
func (o *ServeConnOpts) context() context.Context {
if o != nil && o.Context != nil {
return o.Context
}
return context.Background()
}
func (o *ServeConnOpts) baseConfig() *http.Server {
if o != nil && o.BaseConfig != nil {
return o.BaseConfig
}
return new(http.Server)
}
func (o *ServeConnOpts) handler() http.Handler {
if o != nil {
if o.Handler != nil {
return o.Handler
}
if o.BaseConfig != nil && o.BaseConfig.Handler != nil {
return o.BaseConfig.Handler
}
}
return http.DefaultServeMux
}
// ServeConn serves HTTP/2 requests on the provided connection and
// blocks until the connection is no longer readable.
//
// ServeConn starts speaking HTTP/2 assuming that c has not had any
// reads or writes. It writes its initial settings frame and expects
// to be able to read the preface and settings frame from the
// client. If c has a ConnectionState method like a *tls.Conn, the
// ConnectionState is used to verify the TLS ciphersuite and to set
// the Request.TLS field in Handlers.
//
// ServeConn does not support h2c by itself. Any h2c support must be
// implemented in terms of providing a suitably-behaving net.Conn.
//
// The opts parameter is optional. If nil, default values are used.
func (s *Server) ServeConn(c net.Conn, opts *ServeConnOpts) {
if opts == nil {
opts = &ServeConnOpts{}
}
s.serveConn(c, opts, nil)
}
func (s *Server) serveConn(c net.Conn, opts *ServeConnOpts, newf func(*serverConn)) {
baseCtx, cancel := serverConnBaseContext(c, opts)
defer cancel()
@@ -461,6 +292,9 @@ func (s *Server) serveConn(c net.Conn, opts *ServeConnOpts, newf func(*serverCon
if newf != nil {
newf(sc)
}
if s.state != nil && s.state.testNewConn != nil {
s.state.testNewConn(sc)
}
s.state.registerConn(sc)
defer s.state.unregisterConn(sc)
@@ -570,15 +404,6 @@ func (s *Server) serveConn(c net.Conn, opts *ServeConnOpts, newf func(*serverCon
sc.serve(conf)
}
func serverConnBaseContext(c net.Conn, opts *ServeConnOpts) (ctx context.Context, cancel func()) {
ctx, cancel = context.WithCancel(opts.context())
ctx = context.WithValue(ctx, http.LocalAddrContextKey, c.LocalAddr())
if hs := opts.baseConfig(); hs != nil {
ctx = context.WithValue(ctx, http.ServerContextKey, hs)
}
return
}
func (sc *serverConn) rejectConn(err ErrCode, debug string) {
sc.vlogf("http2: server rejecting conn: %v, %s", err, debug)
// ignoring errors. hanging up anyway.
@@ -2832,21 +2657,6 @@ func (rws *responseWriterState) writeChunk(p []byte) (n int, err error) {
return len(p), nil
}
// TrailerPrefix is a magic prefix for ResponseWriter.Header map keys
// that, if present, signals that the map entry is actually for
// the response trailers, and not the response headers. The prefix
// is stripped after the ServeHTTP call finishes and the values are
// sent in the trailers.
//
// This mechanism is intended only for trailers that are not known
// prior to the headers being written. If the set of trailers is fixed
// or known before the header is written, the normal Go trailers mechanism
// is preferred:
//
// https://golang.org/pkg/net/http/#ResponseWriter
// https://golang.org/pkg/net/http/#example_ResponseWriter_trailers
const TrailerPrefix = "Trailer:"
// promoteUndeclaredTrailers permits http.Handlers to set trailers
// after the header has already been flushed. Because the Go
// ResponseWriter interface has no way to set Trailers (only the
@@ -3123,12 +2933,6 @@ func (w *responseWriter) handlerDone() {
responseWriterStatePool.Put(rws)
}
// Push errors.
var (
ErrRecursivePush = errors.New("http2: recursive push not allowed")
ErrPushLimitReached = errors.New("http2: push would exceed peer's SETTINGS_MAX_CONCURRENT_STREAMS")
)
var _ http.Pusher = (*responseWriter)(nil)
func (w *responseWriter) Push(target string, opts *http.PushOptions) error {

221
vendor/golang.org/x/net/http2/server_common.go generated vendored Normal file
View File

@@ -0,0 +1,221 @@
// Copyright 2026 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package http2
import (
"context"
"errors"
"net"
"net/http"
"time"
)
// TrailerPrefix is a magic prefix for ResponseWriter.Header map keys
// that, if present, signals that the map entry is actually for
// the response trailers, and not the response headers. The prefix
// is stripped after the ServeHTTP call finishes and the values are
// sent in the trailers.
//
// This mechanism is intended only for trailers that are not known
// prior to the headers being written. If the set of trailers is fixed
// or known before the header is written, the normal Go trailers mechanism
// is preferred:
//
// https://golang.org/pkg/net/http/#ResponseWriter
// https://golang.org/pkg/net/http/#example_ResponseWriter_trailers
const TrailerPrefix = "Trailer:"
// Push errors.
var (
ErrRecursivePush = errors.New("http2: recursive push not allowed")
ErrPushLimitReached = errors.New("http2: push would exceed peer's SETTINGS_MAX_CONCURRENT_STREAMS")
)
// ConfigureServer adds HTTP/2 support to a net/http Server.
//
// The configuration conf may be nil.
//
// ConfigureServer must be called before s begins serving.
func ConfigureServer(s *http.Server, conf *Server) error {
return configureServer(s, conf)
}
// Server is an HTTP/2 server.
type Server struct {
// MaxHandlers limits the number of http.Handler ServeHTTP goroutines
// which may run at a time over all connections.
// Negative or zero no limit.
// TODO: implement
MaxHandlers int
// MaxConcurrentStreams optionally specifies the number of
// concurrent streams that each client may have open at a
// time. This is unrelated to the number of http.Handler goroutines
// which may be active globally, which is MaxHandlers.
// If zero, MaxConcurrentStreams defaults to at least 100, per
// the HTTP/2 spec's recommendations.
MaxConcurrentStreams uint32
// MaxDecoderHeaderTableSize optionally specifies the http2
// SETTINGS_HEADER_TABLE_SIZE to send in the initial settings frame. It
// informs the remote endpoint of the maximum size of the header compression
// table used to decode header blocks, in octets. If zero, the default value
// of 4096 is used.
MaxDecoderHeaderTableSize uint32
// MaxEncoderHeaderTableSize optionally specifies an upper limit for the
// header compression table used for encoding request headers. Received
// SETTINGS_HEADER_TABLE_SIZE settings are capped at this limit. If zero,
// the default value of 4096 is used.
MaxEncoderHeaderTableSize uint32
// MaxReadFrameSize optionally specifies the largest frame
// this server is willing to read. A valid value is between
// 16k and 16M, inclusive. If zero or otherwise invalid, a
// default value is used.
MaxReadFrameSize uint32
// PermitProhibitedCipherSuites, if true, permits the use of
// cipher suites prohibited by the HTTP/2 spec.
PermitProhibitedCipherSuites bool
// IdleTimeout specifies how long until idle clients should be
// closed with a GOAWAY frame. PING frames are not considered
// activity for the purposes of IdleTimeout.
// If zero or negative, there is no timeout.
IdleTimeout time.Duration
// ReadIdleTimeout is the timeout after which a health check using a ping
// frame will be carried out if no frame is received on the connection.
// If zero, no health check is performed.
ReadIdleTimeout time.Duration
// PingTimeout is the timeout after which the connection will be closed
// if a response to a ping is not received.
// If zero, a default of 15 seconds is used.
PingTimeout time.Duration
// WriteByteTimeout is the timeout after which a connection will be
// closed if no data can be written to it. The timeout begins when data is
// available to write, and is extended whenever any bytes are written.
// If zero or negative, there is no timeout.
WriteByteTimeout time.Duration
// MaxUploadBufferPerConnection is the size of the initial flow
// control window for each connections. The HTTP/2 spec does not
// allow this to be smaller than 65535 or larger than 2^32-1.
// If the value is outside this range, a default value will be
// used instead.
MaxUploadBufferPerConnection int32
// MaxUploadBufferPerStream is the size of the initial flow control
// window for each stream. The HTTP/2 spec does not allow this to
// be larger than 2^32-1. If the value is zero or larger than the
// maximum, a default value will be used instead.
MaxUploadBufferPerStream int32
// NewWriteScheduler constructs a write scheduler for a connection.
// If nil, a default scheduler is chosen.
//
// Deprecated: User-provided write schedulers are deprecated.
NewWriteScheduler func() WriteScheduler
// CountError, if non-nil, is called on HTTP/2 server errors.
// It's intended to increment a metric for monitoring, such
// as an expvar or Prometheus metric.
// The errType consists of only ASCII word characters.
CountError func(errType string)
// Internal state. This is a pointer (rather than embedded directly)
// so that we don't embed a Mutex in this struct, which will make the
// struct non-copyable, which might break some callers.
state *serverInternalState
}
// ServeConnOpts are options for the Server.ServeConn method.
type ServeConnOpts struct {
// Context is the base context to use.
// If nil, context.Background is used.
Context context.Context
// BaseConfig optionally sets the base configuration
// for values. If nil, defaults are used.
BaseConfig *http.Server
// Handler specifies which handler to use for processing
// requests. If nil, BaseConfig.Handler is used. If BaseConfig
// or BaseConfig.Handler is nil, http.DefaultServeMux is used.
Handler http.Handler
// UpgradeRequest is an initial request received on a connection
// undergoing an h2c upgrade. The request body must have been
// completely read from the connection before calling ServeConn,
// and the 101 Switching Protocols response written.
UpgradeRequest *http.Request
// Settings is the decoded contents of the HTTP2-Settings header
// in an h2c upgrade request.
Settings []byte
// SawClientPreface is set if the HTTP/2 connection preface
// has already been read from the connection.
SawClientPreface bool
}
// ServeConn serves HTTP/2 requests on the provided connection and
// blocks until the connection is no longer readable.
//
// ServeConn starts speaking HTTP/2 assuming that c has not had any
// reads or writes. It writes its initial settings frame and expects
// to be able to read the preface and settings frame from the
// client. If c has a ConnectionState method like a *tls.Conn, the
// ConnectionState is used to verify the TLS ciphersuite and to set
// the Request.TLS field in Handlers.
//
// ServeConn does not support h2c by itself. Any h2c support must be
// implemented in terms of providing a suitably-behaving net.Conn.
//
// The opts parameter is optional. If nil, default values are used.
func (s *Server) ServeConn(c net.Conn, opts *ServeConnOpts) {
if opts == nil {
opts = &ServeConnOpts{}
}
s.serveConn(c, opts, nil)
}
func (o *ServeConnOpts) context() context.Context {
if o != nil && o.Context != nil {
return o.Context
}
return context.Background()
}
func (o *ServeConnOpts) baseConfig() *http.Server {
if o != nil && o.BaseConfig != nil {
return o.BaseConfig
}
return new(http.Server)
}
func (o *ServeConnOpts) handler() http.Handler {
if o != nil {
if o.Handler != nil {
return o.Handler
}
if o.BaseConfig != nil && o.BaseConfig.Handler != nil {
return o.BaseConfig.Handler
}
}
return http.DefaultServeMux
}
func serverConnBaseContext(c net.Conn, opts *ServeConnOpts) (ctx context.Context, cancel func()) {
ctx, cancel = context.WithCancel(opts.context())
ctx = context.WithValue(ctx, http.LocalAddrContextKey, c.LocalAddr())
if hs := opts.baseConfig(); hs != nil {
ctx = context.WithValue(ctx, http.ServerContextKey, hs)
}
return
}

201
vendor/golang.org/x/net/http2/server_wrap.go generated vendored Normal file
View File

@@ -0,0 +1,201 @@
// Copyright 2026 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build go1.27 && !http2legacy
// Server wrapping a net/http.Server.
package http2
import (
"context"
"errors"
"net"
"net/http"
"sync"
"time"
)
type serverInternalState struct {
s1 *http.Server
initOnce sync.Once
serveConnFunc func(context.Context, net.Conn, http.Handler, bool, *http.Request, []byte)
}
func configureServer(s *http.Server, conf *Server) error {
if s == nil {
panic("nil *http.Server")
}
if conf == nil {
conf = new(Server)
}
if conf.state != nil {
// This isn't a panic in the pre-wrapping implementation,
// but calling ConfigureServer twice with the same http2.Server
// overwrites internal state on the server.
// Make the error explicit and early here.
panic("ConfigureServer may be called only once per Server")
}
if h1, h2 := s, conf; h2.IdleTimeout == 0 {
if h1.IdleTimeout != 0 {
h2.IdleTimeout = h1.IdleTimeout
} else {
h2.IdleTimeout = h1.ReadTimeout
}
}
conf.state = &serverInternalState{
s1: s,
}
sconfig := &serverConfig{s: conf}
if err := s.Serve(sconfig); err != nil || sconfig.serveConnFunc == nil {
panic("http2: net/http does not support this version of x/net/http2")
}
conf.state.serveConnFunc = sconfig.serveConnFunc
return nil
}
type serverConfig struct {
s *Server
serveConnFunc func(context.Context, net.Conn, http.Handler, bool, *http.Request, []byte)
}
func (*serverConfig) Accept() (net.Conn, error) {
return nil, errors.New("unexpected call to Accept")
}
func (*serverConfig) Close() error {
return nil
}
func (*serverConfig) Addr() net.Addr {
return nil
}
func (s *serverConfig) ServeConnFunc(f func(context.Context, net.Conn, http.Handler, bool, *http.Request, []byte)) {
s.serveConnFunc = f
}
func (s *serverConfig) HTTP2Config() http.HTTP2Config {
return http.HTTP2Config{
MaxConcurrentStreams: int(s.s.MaxConcurrentStreams),
MaxDecoderHeaderTableSize: int(s.s.MaxDecoderHeaderTableSize),
MaxEncoderHeaderTableSize: int(s.s.MaxEncoderHeaderTableSize),
MaxReadFrameSize: int(s.s.MaxReadFrameSize),
PermitProhibitedCipherSuites: s.s.PermitProhibitedCipherSuites,
MaxReceiveBufferPerConnection: int(s.s.MaxUploadBufferPerConnection),
MaxReceiveBufferPerStream: int(s.s.MaxUploadBufferPerStream),
SendPingTimeout: s.s.ReadIdleTimeout,
PingTimeout: s.s.PingTimeout,
WriteByteTimeout: s.s.WriteByteTimeout,
CountError: s.s.CountError,
}
}
func (s *serverConfig) IdleTimeout() time.Duration {
return s.s.IdleTimeout
}
type serverConn struct{}
func (s *Server) serveConn(c net.Conn, opts *ServeConnOpts, _ func(*serverConn)) {
var serveConnFunc func(context.Context, net.Conn, http.Handler, bool, *http.Request, []byte)
switch {
case opts.BaseConfig != nil:
// The user has provided us with an http.Server to take configuration from.
//
// We can't send our request to opts.BaseConfig, because an http.Server can
// only be associated with a single http2.Server and the user might
// use this one with several http.Servers.
//
// We can't send our request to s.state.s1, because it doesn't contain
// the right configuration.
//
// So create a one-off copy of opts.BaseConfig and use it.
h1 := &http.Server{
TLSConfig: opts.BaseConfig.TLSConfig,
ReadTimeout: opts.BaseConfig.ReadTimeout,
ReadHeaderTimeout: opts.BaseConfig.ReadHeaderTimeout,
WriteTimeout: opts.BaseConfig.WriteTimeout,
IdleTimeout: opts.BaseConfig.IdleTimeout,
MaxHeaderBytes: opts.BaseConfig.MaxHeaderBytes,
ConnState: opts.BaseConfig.ConnState,
ErrorLog: opts.BaseConfig.ErrorLog,
BaseContext: opts.BaseConfig.BaseContext,
ConnContext: opts.BaseConfig.ConnContext,
HTTP2: opts.BaseConfig.HTTP2,
}
sconfig := &serverConfig{s: s}
if err := h1.Serve(sconfig); err != nil || sconfig.serveConnFunc == nil {
panic("http2: net/http does not support this version of x/net/http2")
}
serveConnFunc = sconfig.serveConnFunc
case s.state != nil:
serveConnFunc = s.state.serveConnFunc
default:
// Strange-but-true: Server has no concurrency-safe way to initialize
// its internal state, so historically ServeConn just doesn't use any
// persistent state if you don't call ConfigureServer first.
//
// If ConfigureServer hasn't been called, create a one-off http.Server
// for the connection, since we don't have any way to keep one around for reuse.
h1 := &http.Server{}
sconfig := &serverConfig{s: s}
if err := h1.Serve(sconfig); err != nil || sconfig.serveConnFunc == nil {
panic("http2: net/http does not support this version of x/net/http2")
}
serveConnFunc = sconfig.serveConnFunc
}
ctx, cancel := serverConnBaseContext(c, opts)
defer cancel()
serveConnFunc(ctx, c, opts.handler(), opts.SawClientPreface, opts.UpgradeRequest, opts.Settings)
}
// FrameWriteRequest is a request to write a frame.
//
// Deprecated: User-provided write schedulers are deprecated.
type FrameWriteRequest struct {
// Ideally we'd define this in writesched_common.go,
// to avoid duplicating an exported symbol across two files,
// but the changes required to make this work are fairly large.
}
func (wr FrameWriteRequest) StreamID() uint32 {
return 0
}
func (wr FrameWriteRequest) DataSize() int {
return 0
}
func (wr FrameWriteRequest) Consume(n int32) (FrameWriteRequest, FrameWriteRequest, int) {
return FrameWriteRequest{}, FrameWriteRequest{}, 0
}
func (wr FrameWriteRequest) String() string {
return ""
}
// NewPriorityWriteScheduler is deprecated.
//
// Deprecated: User-provided write schedulers are deprecated.
func NewPriorityWriteScheduler(cfg *PriorityWriteSchedulerConfig) WriteScheduler {
return unsupportedWriteScheduler{}
}
// NewRandomWriteScheduler is deprecated.
//
// Deprecated: User-provided write schedulers are deprecated.
func NewRandomWriteScheduler() WriteScheduler {
return unsupportedWriteScheduler{}
}
type unsupportedWriteScheduler struct{}
func (unsupportedWriteScheduler) OpenStream(streamID uint32, options OpenStreamOptions) {}
func (unsupportedWriteScheduler) CloseStream(streamID uint32) {}
func (unsupportedWriteScheduler) AdjustStream(streamID uint32, priority PriorityParam) {}
func (unsupportedWriteScheduler) Push(wr FrameWriteRequest) {}
func (unsupportedWriteScheduler) Pop() (wr FrameWriteRequest, ok bool) {
return FrameWriteRequest{}, false
}

View File

@@ -2,6 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build !(go1.27 && !http2legacy)
// Transport code.
package http2
@@ -21,20 +23,17 @@ import (
"log"
"math"
"math/bits"
mathrand "math/rand"
"net"
"net/http"
"net/http/httptrace"
"net/textproto"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"golang.org/x/net/http/httpguts"
"golang.org/x/net/http2/hpack"
"golang.org/x/net/idna"
"golang.org/x/net/internal/httpcommon"
)
@@ -60,123 +59,7 @@ const (
defaultMaxConcurrentStreams = 1000
)
// Transport is an HTTP/2 Transport.
//
// A Transport internally caches connections to servers. It is safe
// for concurrent use by multiple goroutines.
type Transport struct {
// DialTLSContext specifies an optional dial function with context for
// creating TLS connections for requests.
//
// If DialTLSContext and DialTLS is nil, tls.Dial is used.
//
// If the returned net.Conn has a ConnectionState method like tls.Conn,
// it will be used to set http.Response.TLS.
DialTLSContext func(ctx context.Context, network, addr string, cfg *tls.Config) (net.Conn, error)
// DialTLS specifies an optional dial function for creating
// TLS connections for requests.
//
// If DialTLSContext and DialTLS is nil, tls.Dial is used.
//
// Deprecated: Use DialTLSContext instead, which allows the transport
// to cancel dials as soon as they are no longer needed.
// If both are set, DialTLSContext takes priority.
DialTLS func(network, addr string, cfg *tls.Config) (net.Conn, error)
// TLSClientConfig specifies the TLS configuration to use with
// tls.Client. If nil, the default configuration is used.
TLSClientConfig *tls.Config
// ConnPool optionally specifies an alternate connection pool to use.
// If nil, the default is used.
ConnPool ClientConnPool
// DisableCompression, if true, prevents the Transport from
// requesting compression with an "Accept-Encoding: gzip"
// request header when the Request contains no existing
// Accept-Encoding value. If the Transport requests gzip on
// its own and gets a gzipped response, it's transparently
// decoded in the Response.Body. However, if the user
// explicitly requested gzip it is not automatically
// uncompressed.
DisableCompression bool
// AllowHTTP, if true, permits HTTP/2 requests using the insecure,
// plain-text "http" scheme. Note that this does not enable h2c support.
AllowHTTP bool
// MaxHeaderListSize is the http2 SETTINGS_MAX_HEADER_LIST_SIZE to
// send in the initial settings frame. It is how many bytes
// of response headers are allowed. Unlike the http2 spec, zero here
// means to use a default limit (currently 10MB). If you actually
// want to advertise an unlimited value to the peer, Transport
// interprets the highest possible value here (0xffffffff or 1<<32-1)
// to mean no limit.
MaxHeaderListSize uint32
// MaxReadFrameSize is the http2 SETTINGS_MAX_FRAME_SIZE to send in the
// initial settings frame. It is the size in bytes of the largest frame
// payload that the sender is willing to receive. If 0, no setting is
// sent, and the value is provided by the peer, which should be 16384
// according to the spec:
// https://datatracker.ietf.org/doc/html/rfc7540#section-6.5.2.
// Values are bounded in the range 16k to 16M.
MaxReadFrameSize uint32
// MaxDecoderHeaderTableSize optionally specifies the http2
// SETTINGS_HEADER_TABLE_SIZE to send in the initial settings frame. It
// informs the remote endpoint of the maximum size of the header compression
// table used to decode header blocks, in octets. If zero, the default value
// of 4096 is used.
MaxDecoderHeaderTableSize uint32
// MaxEncoderHeaderTableSize optionally specifies an upper limit for the
// header compression table used for encoding request headers. Received
// SETTINGS_HEADER_TABLE_SIZE settings are capped at this limit. If zero,
// the default value of 4096 is used.
MaxEncoderHeaderTableSize uint32
// StrictMaxConcurrentStreams controls whether the server's
// SETTINGS_MAX_CONCURRENT_STREAMS should be respected
// globally. If false, new TCP connections are created to the
// server as needed to keep each under the per-connection
// SETTINGS_MAX_CONCURRENT_STREAMS limit. If true, the
// server's SETTINGS_MAX_CONCURRENT_STREAMS is interpreted as
// a global limit and callers of RoundTrip block when needed,
// waiting for their turn.
StrictMaxConcurrentStreams bool
// IdleConnTimeout is the maximum amount of time an idle
// (keep-alive) connection will remain idle before closing
// itself.
// Zero means no limit.
IdleConnTimeout time.Duration
// ReadIdleTimeout is the timeout after which a health check using ping
// frame will be carried out if no frame is received on the connection.
// Note that a ping response will is considered a received frame, so if
// there is no other traffic on the connection, the health check will
// be performed every ReadIdleTimeout interval.
// If zero, no health check is performed.
ReadIdleTimeout time.Duration
// PingTimeout is the timeout after which the connection will be closed
// if a response to Ping is not received.
// Defaults to 15s.
PingTimeout time.Duration
// WriteByteTimeout is the timeout after which the connection will be
// closed no data can be written to it. The timeout begins when data is
// available to write, and is extended whenever any bytes are written.
WriteByteTimeout time.Duration
// CountError, if non-nil, is called on HTTP/2 transport errors.
// It's intended to increment a metric for monitoring, such
// as an expvar or Prometheus metric.
// The errType consists of only ASCII word characters.
CountError func(errType string)
type transportInternal struct {
// t1, if non-nil, is the standard library Transport using
// this transport. Its settings are used (but not its
// RoundTrip method, etc).
@@ -217,27 +100,18 @@ func (t *Transport) disableCompression() bool {
return t.DisableCompression || (t.t1 != nil && t.t1.DisableCompression)
}
// ConfigureTransport configures a net/http HTTP/1 Transport to use HTTP/2.
// It returns an error if t1 has already been HTTP/2-enabled.
//
// Use ConfigureTransports instead to configure the HTTP/2 Transport.
func ConfigureTransport(t1 *http.Transport) error {
_, err := ConfigureTransports(t1)
func configureTransport(t1 *http.Transport) error {
_, err := configureTransports(t1)
return err
}
// ConfigureTransports configures a net/http HTTP/1 Transport to use HTTP/2.
// It returns a new HTTP/2 Transport for further configuration.
// It returns an error if t1 has already been HTTP/2-enabled.
func ConfigureTransports(t1 *http.Transport) (*Transport, error) {
return configureTransports(t1)
}
func configureTransports(t1 *http.Transport) (*Transport, error) {
connPool := new(clientConnPool)
t2 := &Transport{
ConnPool: noDialClientConnPool{connPool},
t1: t1,
transportInternal: transportInternal{
t1: t1,
},
}
connPool.t = t2
if err := registerHTTPSProtocol(t1, noDialH2RoundTripper{t2}); err != nil {
@@ -525,68 +399,7 @@ func (sew stickyErrWriter) Write(p []byte) (n int, err error) {
return n, err
}
// noCachedConnError is the concrete type of ErrNoCachedConn, which
// needs to be detected by net/http regardless of whether it's its
// bundled version (in h2_bundle.go with a rewritten type name) or
// from a user's x/net/http2. As such, as it has a unique method name
// (IsHTTP2NoCachedConnError) that net/http sniffs for via func
// isNoCachedConnError.
type noCachedConnError struct{}
func (noCachedConnError) IsHTTP2NoCachedConnError() {}
func (noCachedConnError) Error() string { return "http2: no cached connection was available" }
// isNoCachedConnError reports whether err is of type noCachedConnError
// or its equivalent renamed type in net/http2's h2_bundle.go. Both types
// may coexist in the same running program.
func isNoCachedConnError(err error) bool {
_, ok := err.(interface{ IsHTTP2NoCachedConnError() })
return ok
}
var ErrNoCachedConn error = noCachedConnError{}
// RoundTripOpt are options for the Transport.RoundTripOpt method.
type RoundTripOpt struct {
// OnlyCachedConn controls whether RoundTripOpt may
// create a new TCP connection. If set true and
// no cached connection is available, RoundTripOpt
// will return ErrNoCachedConn.
OnlyCachedConn bool
allowHTTP bool // allow http:// URLs
}
func (t *Transport) RoundTrip(req *http.Request) (*http.Response, error) {
return t.RoundTripOpt(req, RoundTripOpt{})
}
// authorityAddr returns a given authority (a host/IP, or host:port / ip:port)
// and returns a host:port. The port 443 is added if needed.
func authorityAddr(scheme string, authority string) (addr string) {
host, port, err := net.SplitHostPort(authority)
if err != nil { // authority didn't have a port
host = authority
port = ""
}
if port == "" { // authority's port was empty
port = "443"
if scheme == "http" {
port = "80"
}
}
if a, err := idna.ToASCII(host); err == nil {
host = a
}
// IPv6 address literal, without a port:
if strings.HasPrefix(host, "[") && strings.HasSuffix(host, "]") {
return host + ":" + port
}
return net.JoinHostPort(host, port)
}
// RoundTripOpt is like RoundTrip, but takes options.
func (t *Transport) RoundTripOpt(req *http.Request, opt RoundTripOpt) (*http.Response, error) {
func (t *Transport) roundTripOpt(req *http.Request, opt RoundTripOpt) (*http.Response, error) {
switch req.URL.Scheme {
case "https":
// Always okay.
@@ -597,126 +410,15 @@ func (t *Transport) RoundTripOpt(req *http.Request, opt RoundTripOpt) (*http.Res
default:
return nil, errors.New("http2: unsupported scheme")
}
addr := authorityAddr(req.URL.Scheme, req.URL.Host)
for retry := 0; ; retry++ {
cc, err := t.connPool().GetClientConn(req, addr)
if err != nil {
t.vlogf("http2: Transport failed to get client conn for %s: %v", addr, err)
return nil, err
}
reused := !atomic.CompareAndSwapUint32(&cc.atomicReused, 0, 1)
traceGotConn(req, cc, reused)
res, err := cc.RoundTrip(req)
if err != nil && retry <= 6 {
roundTripErr := err
if req, err = shouldRetryRequest(req, err); err == nil {
// After the first retry, do exponential backoff with 10% jitter.
if retry == 0 {
t.vlogf("RoundTrip retrying after failure: %v", roundTripErr)
continue
}
backoff := float64(uint(1) << (uint(retry) - 1))
backoff += backoff * (0.1 * mathrand.Float64())
d := time.Second * time.Duration(backoff)
tm := time.NewTimer(d)
select {
case <-tm.C:
t.vlogf("RoundTrip retrying after failure: %v", roundTripErr)
continue
case <-req.Context().Done():
tm.Stop()
err = req.Context().Err()
}
}
}
if err == errClientConnNotEstablished {
// This ClientConn was created recently,
// this is the first request to use it,
// and the connection is closed and not usable.
//
// In this state, cc.idleTimer will remove the conn from the pool
// when it fires. Stop the timer and remove it here so future requests
// won't try to use this connection.
//
// If the timer has already fired and we're racing it, the redundant
// call to MarkDead is harmless.
if cc.idleTimer != nil {
cc.idleTimer.Stop()
}
t.connPool().MarkDead(cc)
}
if err != nil {
t.vlogf("RoundTrip failure: %v", err)
return nil, err
}
return res, nil
}
return t.roundTripViaPool(req, opt, t.connPool())
}
// CloseIdleConnections closes any connections which were previously
// connected from previous requests but are now sitting idle.
// It does not interrupt any connections currently in use.
func (t *Transport) CloseIdleConnections() {
func (t *Transport) closeIdleConnections() {
if cp, ok := t.connPool().(clientConnPoolIdleCloser); ok {
cp.closeIdleConnections()
}
}
var (
errClientConnClosed = errors.New("http2: client conn is closed")
errClientConnUnusable = errors.New("http2: client conn not usable")
errClientConnNotEstablished = errors.New("http2: client conn could not be established")
errClientConnGotGoAway = errors.New("http2: Transport received Server's graceful shutdown GOAWAY")
errClientConnForceClosed = errors.New("http2: client connection force closed via ClientConn.Close")
)
// shouldRetryRequest is called by RoundTrip when a request fails to get
// response headers. It is always called with a non-nil error.
// It returns either a request to retry (either the same request, or a
// modified clone), or an error if the request can't be replayed.
func shouldRetryRequest(req *http.Request, err error) (*http.Request, error) {
if !canRetryError(err) {
return nil, err
}
// If the Body is nil (or http.NoBody), it's safe to reuse
// this request and its Body.
if req.Body == nil || req.Body == http.NoBody {
return req, nil
}
// If the request body can be reset back to its original
// state via the optional req.GetBody, do that.
if req.GetBody != nil {
body, err := req.GetBody()
if err != nil {
return nil, err
}
newReq := *req
newReq.Body = body
return &newReq, nil
}
// The Request.Body can't reset back to the beginning, but we
// don't seem to have started to read from it yet, so reuse
// the request directly.
if err == errClientConnUnusable {
return req, nil
}
return nil, fmt.Errorf("http2: Transport: cannot retry err [%v] after Request.Body was written; define Request.GetBody to avoid this error", err)
}
func canRetryError(err error) bool {
if err == errClientConnUnusable || err == errClientConnGotGoAway {
return true
}
if se, ok := err.(StreamError); ok {
return se.Code == ErrCodeRefusedStream
}
return false
}
func (t *Transport) dialClientConn(ctx context.Context, addr string, singleUse bool) (*ClientConn, error) {
host, _, err := net.SplitHostPort(addr)
if err != nil {
@@ -743,27 +445,6 @@ func (t *Transport) newTLSConfig(host string) *tls.Config {
return cfg
}
func (t *Transport) dialTLS(ctx context.Context, network, addr string, tlsCfg *tls.Config) (net.Conn, error) {
if t.DialTLSContext != nil {
return t.DialTLSContext(ctx, network, addr, tlsCfg)
} else if t.DialTLS != nil {
return t.DialTLS(network, addr, tlsCfg)
}
tlsCn, err := t.dialTLSWithContext(ctx, network, addr, tlsCfg)
if err != nil {
return nil, err
}
state := tlsCn.ConnectionState()
if p := state.NegotiatedProtocol; p != NextProtoTLS {
return nil, fmt.Errorf("http2: unexpected ALPN protocol %q; want %q", p, NextProtoTLS)
}
if !state.NegotiatedProtocolIsMutual {
return nil, errors.New("http2: could not negotiate protocol mutually")
}
return tlsCn, nil
}
// disableKeepAlives reports whether connections should be closed as
// soon as possible after handling the first request.
func (t *Transport) disableKeepAlives() bool {
@@ -777,7 +458,7 @@ func (t *Transport) expectContinueTimeout() time.Duration {
return t.t1.ExpectContinueTimeout
}
func (t *Transport) NewClientConn(c net.Conn) (*ClientConn, error) {
func (t *Transport) newUserClientConn(c net.Conn) (*ClientConn, error) {
return t.newClientConn(c, t.disableKeepAlives(), nil)
}
@@ -890,8 +571,7 @@ func (cc *ClientConn) healthCheck() {
}
}
// SetDoNotReuse marks cc as not reusable for future HTTP requests.
func (cc *ClientConn) SetDoNotReuse() {
func (cc *ClientConn) setDoNotReuse() {
cc.mu.Lock()
defer cc.mu.Unlock()
cc.doNotReuse = true
@@ -932,21 +612,13 @@ func (cc *ClientConn) setGoAway(f *GoAwayFrame) {
}
}
// CanTakeNewRequest reports whether the connection can take a new request,
// meaning it has not been closed or received or sent a GOAWAY.
//
// If the caller is going to immediately make a new request on this
// connection, use ReserveNewRequest instead.
func (cc *ClientConn) CanTakeNewRequest() bool {
func (cc *ClientConn) canTakeNewRequest() bool {
cc.mu.Lock()
defer cc.mu.Unlock()
return cc.canTakeNewRequestLocked()
}
// ReserveNewRequest is like CanTakeNewRequest but also reserves a
// concurrent stream in cc. The reservation is decremented on the
// next call to RoundTrip.
func (cc *ClientConn) ReserveNewRequest() bool {
func (cc *ClientConn) reserveNewRequest() bool {
cc.mu.Lock()
defer cc.mu.Unlock()
if st := cc.idleStateLocked(); !st.canTakeNewRequest {
@@ -956,41 +628,7 @@ func (cc *ClientConn) ReserveNewRequest() bool {
return true
}
// ClientConnState describes the state of a ClientConn.
type ClientConnState struct {
// Closed is whether the connection is closed.
Closed bool
// Closing is whether the connection is in the process of
// closing. It may be closing due to shutdown, being a
// single-use connection, being marked as DoNotReuse, or
// having received a GOAWAY frame.
Closing bool
// StreamsActive is how many streams are active.
StreamsActive int
// StreamsReserved is how many streams have been reserved via
// ClientConn.ReserveNewRequest.
StreamsReserved int
// StreamsPending is how many requests have been sent in excess
// of the peer's advertised MaxConcurrentStreams setting and
// are waiting for other streams to complete.
StreamsPending int
// MaxConcurrentStreams is how many concurrent streams the
// peer advertised as acceptable. Zero means no SETTINGS
// frame has been received yet.
MaxConcurrentStreams uint32
// LastIdle, if non-zero, is when the connection last
// transitioned to idle state.
LastIdle time.Time
}
// State returns a snapshot of cc's state.
func (cc *ClientConn) State() ClientConnState {
func (cc *ClientConn) state() ClientConnState {
cc.wmu.Lock()
maxConcurrent := cc.maxConcurrentStreams
if !cc.seenSettings {
@@ -1161,6 +799,12 @@ func (cc *ClientConn) closeIfIdle() {
cc.closeConn()
}
func (cc *ClientConn) stopIdleTimer() {
if cc.idleTimer != nil {
cc.idleTimer.Stop()
}
}
func (cc *ClientConn) isDoNotReuseAndIdle() bool {
cc.mu.Lock()
defer cc.mu.Unlock()
@@ -1169,8 +813,7 @@ func (cc *ClientConn) isDoNotReuseAndIdle() bool {
var shutdownEnterWaitStateHook = func() {}
// Shutdown gracefully closes the client connection, waiting for running streams to complete.
func (cc *ClientConn) Shutdown(ctx context.Context) error {
func (cc *ClientConn) shutdown(ctx context.Context) error {
if err := cc.sendGoAway(); err != nil {
return err
}
@@ -1244,10 +887,7 @@ func (cc *ClientConn) closeForError(err error) {
cc.closeConn()
}
// Close closes the client connection immediately.
//
// In-flight requests are interrupted. For a graceful shutdown, use Shutdown instead.
func (cc *ClientConn) Close() error {
func (cc *ClientConn) close() error {
cc.closeForError(errClientConnForceClosed)
return nil
}
@@ -1301,11 +941,11 @@ func (cc *ClientConn) decrStreamReservationsLocked() {
}
}
func (cc *ClientConn) RoundTrip(req *http.Request) (*http.Response, error) {
return cc.roundTrip(req, nil)
func (cc *ClientConn) roundTrip(req *http.Request) (*http.Response, error) {
return cc.internalRoundTrip(req, nil)
}
func (cc *ClientConn) roundTrip(req *http.Request, streamf func(*clientStream)) (*http.Response, error) {
func (cc *ClientConn) internalRoundTrip(req *http.Request, streamf func(*clientStream)) (*http.Response, error) {
ctx := req.Context()
cs := &clientStream{
cc: cc,
@@ -2125,19 +1765,6 @@ func (cc *ClientConn) readLoop() {
}
}
// GoAwayError is returned by the Transport when the server closes the
// TCP connection after sending a GOAWAY frame.
type GoAwayError struct {
LastStreamID uint32
ErrCode ErrCode
DebugData string
}
func (e GoAwayError) Error() string {
return fmt.Sprintf("http2: server sent GOAWAY and closed the connection; LastStreamID=%v, ErrCode=%v, debug=%q",
e.LastStreamID, e.ErrCode, e.DebugData)
}
func isEOFOrNetReadError(err error) bool {
if err == io.EOF {
return true
@@ -2978,7 +2605,7 @@ func (rl *clientConnReadLoop) processResetStream(f *RSTStreamFrame) error {
}
// Ping sends a PING frame to the server and waits for the ack.
func (cc *ClientConn) Ping(ctx context.Context) error {
func (cc *ClientConn) ping(ctx context.Context) error {
c := make(chan struct{})
// Generate a random payload
var p [8]byte
@@ -3092,16 +2719,6 @@ func (cc *ClientConn) vlogf(format string, args ...interface{}) {
cc.t.vlogf(format, args...)
}
func (t *Transport) vlogf(format string, args ...interface{}) {
if VerboseLogs {
t.logf(format, args...)
}
}
func (t *Transport) logf(format string, args ...interface{}) {
log.Printf(format, args...)
}
var noBody io.ReadCloser = noBodyReader{}
type noBodyReader struct{}
@@ -3417,17 +3034,3 @@ func traceGot1xxResponseFunc(trace *httptrace.ClientTrace) func(int, textproto.M
}
return nil
}
// dialTLSWithContext uses tls.Dialer, added in Go 1.15, to open a TLS
// connection.
func (t *Transport) dialTLSWithContext(ctx context.Context, network, addr string, cfg *tls.Config) (*tls.Conn, error) {
dialer := &tls.Dialer{
Config: cfg,
}
cn, err := dialer.DialContext(ctx, network, addr)
if err != nil {
return nil, err
}
tlsCn := cn.(*tls.Conn) // DialContext comment promises this will always succeed
return tlsCn, nil
}

447
vendor/golang.org/x/net/http2/transport_common.go generated vendored Normal file
View File

@@ -0,0 +1,447 @@
// Copyright 2026 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package http2
import (
"context"
"crypto/tls"
"errors"
"fmt"
"log"
mathrand "math/rand"
"net"
"net/http"
"strings"
"sync/atomic"
"time"
"golang.org/x/net/idna"
)
// ConfigureTransport configures a net/http HTTP/1 Transport to use HTTP/2.
// It returns an error if t1 has already been HTTP/2-enabled.
//
// Use ConfigureTransports instead to configure the HTTP/2 Transport.
func ConfigureTransport(t1 *http.Transport) error {
return configureTransport(t1)
}
// ConfigureTransports configures a net/http HTTP/1 Transport to use HTTP/2.
// It returns a new HTTP/2 Transport for further configuration.
// It returns an error if t1 has already been HTTP/2-enabled.
func ConfigureTransports(t1 *http.Transport) (*Transport, error) {
return configureTransports(t1)
}
// Transport is an HTTP/2 Transport.
//
// A Transport internally caches connections to servers. It is safe
// for concurrent use by multiple goroutines.
type Transport struct {
// DialTLSContext specifies an optional dial function with context for
// creating TLS connections for requests.
//
// If DialTLSContext and DialTLS is nil, tls.Dial is used.
//
// If the returned net.Conn has a ConnectionState method like tls.Conn,
// it will be used to set http.Response.TLS.
DialTLSContext func(ctx context.Context, network, addr string, cfg *tls.Config) (net.Conn, error)
// DialTLS specifies an optional dial function for creating
// TLS connections for requests.
//
// If DialTLSContext and DialTLS is nil, tls.Dial is used.
//
// Deprecated: Use DialTLSContext instead, which allows the transport
// to cancel dials as soon as they are no longer needed.
// If both are set, DialTLSContext takes priority.
DialTLS func(network, addr string, cfg *tls.Config) (net.Conn, error)
// TLSClientConfig specifies the TLS configuration to use with
// tls.Client. If nil, the default configuration is used.
TLSClientConfig *tls.Config
// ConnPool optionally specifies an alternate connection pool to use.
// If nil, the default is used.
ConnPool ClientConnPool
// DisableCompression, if true, prevents the Transport from
// requesting compression with an "Accept-Encoding: gzip"
// request header when the Request contains no existing
// Accept-Encoding value. If the Transport requests gzip on
// its own and gets a gzipped response, it's transparently
// decoded in the Response.Body. However, if the user
// explicitly requested gzip it is not automatically
// uncompressed.
DisableCompression bool
// AllowHTTP, if true, permits HTTP/2 requests using the insecure,
// plain-text "http" scheme. Note that this does not enable h2c support.
AllowHTTP bool
// MaxHeaderListSize is the http2 SETTINGS_MAX_HEADER_LIST_SIZE to
// send in the initial settings frame. It is how many bytes
// of response headers are allowed. Unlike the http2 spec, zero here
// means to use a default limit (currently 10MB). If you actually
// want to advertise an unlimited value to the peer, Transport
// interprets the highest possible value here (0xffffffff or 1<<32-1)
// to mean no limit.
MaxHeaderListSize uint32
// MaxReadFrameSize is the http2 SETTINGS_MAX_FRAME_SIZE to send in the
// initial settings frame. It is the size in bytes of the largest frame
// payload that the sender is willing to receive. If 0, no setting is
// sent, and the value is provided by the peer, which should be 16384
// according to the spec:
// https://datatracker.ietf.org/doc/html/rfc7540#section-6.5.2.
// Values are bounded in the range 16k to 16M.
MaxReadFrameSize uint32
// MaxDecoderHeaderTableSize optionally specifies the http2
// SETTINGS_HEADER_TABLE_SIZE to send in the initial settings frame. It
// informs the remote endpoint of the maximum size of the header compression
// table used to decode header blocks, in octets. If zero, the default value
// of 4096 is used.
MaxDecoderHeaderTableSize uint32
// MaxEncoderHeaderTableSize optionally specifies an upper limit for the
// header compression table used for encoding request headers. Received
// SETTINGS_HEADER_TABLE_SIZE settings are capped at this limit. If zero,
// the default value of 4096 is used.
MaxEncoderHeaderTableSize uint32
// StrictMaxConcurrentStreams controls whether the server's
// SETTINGS_MAX_CONCURRENT_STREAMS should be respected
// globally. If false, new TCP connections are created to the
// server as needed to keep each under the per-connection
// SETTINGS_MAX_CONCURRENT_STREAMS limit. If true, the
// server's SETTINGS_MAX_CONCURRENT_STREAMS is interpreted as
// a global limit and callers of RoundTrip block when needed,
// waiting for their turn.
StrictMaxConcurrentStreams bool
// IdleConnTimeout is the maximum amount of time an idle
// (keep-alive) connection will remain idle before closing
// itself.
// Zero means no limit.
IdleConnTimeout time.Duration
// ReadIdleTimeout is the timeout after which a health check using ping
// frame will be carried out if no frame is received on the connection.
// Note that a ping response will is considered a received frame, so if
// there is no other traffic on the connection, the health check will
// be performed every ReadIdleTimeout interval.
// If zero, no health check is performed.
ReadIdleTimeout time.Duration
// PingTimeout is the timeout after which the connection will be closed
// if a response to Ping is not received.
// Defaults to 15s.
PingTimeout time.Duration
// WriteByteTimeout is the timeout after which the connection will be
// closed no data can be written to it. The timeout begins when data is
// available to write, and is extended whenever any bytes are written.
WriteByteTimeout time.Duration
// CountError, if non-nil, is called on HTTP/2 transport errors.
// It's intended to increment a metric for monitoring, such
// as an expvar or Prometheus metric.
// The errType consists of only ASCII word characters.
CountError func(errType string)
// Internal state, differs between wrapped and non-wrapped implementations.
transportInternal
}
var (
errClientConnClosed = errors.New("http2: client conn is closed")
errClientConnNotEstablished = errors.New("http2: client conn could not be established")
errClientConnGotGoAway = errors.New("http2: Transport received Server's graceful shutdown GOAWAY")
errClientConnForceClosed = errors.New("http2: client connection force closed via ClientConn.Close")
errClientConnUnusable = errors.New("http2: client conn not usable")
)
// ClientConnPool manages a pool of HTTP/2 client connections.
type ClientConnPool interface {
// GetClientConn returns a specific HTTP/2 connection (usually
// a TLS-TCP connection) to an HTTP/2 server. On success, the
// returned ClientConn accounts for the upcoming RoundTrip
// call, so the caller should not omit it. If the caller needs
// to, ClientConn.RoundTrip can be called with a bogus
// new(http.Request) to release the stream reservation.
GetClientConn(req *http.Request, addr string) (*ClientConn, error)
MarkDead(*ClientConn)
}
// ClientConnState describes the state of a ClientConn.
type ClientConnState struct {
// Closed is whether the connection is closed.
Closed bool
// Closing is whether the connection is in the process of
// closing. It may be closing due to shutdown, being a
// single-use connection, being marked as DoNotReuse, or
// having received a GOAWAY frame.
Closing bool
// StreamsActive is how many streams are active.
StreamsActive int
// StreamsReserved is how many streams have been reserved via
// ClientConn.ReserveNewRequest.
StreamsReserved int
// StreamsPending is how many requests have been sent in excess
// of the peer's advertised MaxConcurrentStreams setting and
// are waiting for other streams to complete.
StreamsPending int
// MaxConcurrentStreams is how many concurrent streams the
// peer advertised as acceptable. Zero means no SETTINGS
// frame has been received yet.
MaxConcurrentStreams uint32
// LastIdle, if non-zero, is when the connection last
// transitioned to idle state.
LastIdle time.Time
}
// RoundTripOpt are options for the Transport.RoundTripOpt method.
type RoundTripOpt struct {
// OnlyCachedConn controls whether RoundTripOpt may
// create a new TCP connection. If set true and
// no cached connection is available, RoundTripOpt
// will return ErrNoCachedConn.
// OnlyCachedConn was broken in https://go.dev/cl/16699.
OnlyCachedConn bool
allowHTTP bool // allow http:// URLs
}
func (t *Transport) RoundTrip(req *http.Request) (*http.Response, error) {
return t.RoundTripOpt(req, RoundTripOpt{})
}
// RoundTripOpt is like RoundTrip, but takes options.
func (t *Transport) RoundTripOpt(req *http.Request, opt RoundTripOpt) (*http.Response, error) {
return t.roundTripOpt(req, opt)
}
// CloseIdleConnections closes any connections which were previously
// connected from previous requests but are now sitting idle.
// It does not interrupt any connections currently in use.
func (t *Transport) CloseIdleConnections() {
t.closeIdleConnections()
}
func (t *Transport) NewClientConn(c net.Conn) (*ClientConn, error) {
return t.newUserClientConn(c)
}
// authorityAddr returns a given authority (a host/IP, or host:port / ip:port)
// and returns a host:port. The port 443 is added if needed.
func authorityAddr(scheme string, authority string) (addr string) {
host, port, err := net.SplitHostPort(authority)
if err != nil { // authority didn't have a port
host = authority
port = ""
}
if port == "" { // authority's port was empty
port = "443"
if scheme == "http" {
port = "80"
}
}
if a, err := idna.ToASCII(host); err == nil {
host = a
}
// IPv6 address literal, without a port:
if strings.HasPrefix(host, "[") && strings.HasSuffix(host, "]") {
return host + ":" + port
}
return net.JoinHostPort(host, port)
}
func (t *Transport) roundTripViaPool(req *http.Request, opt RoundTripOpt, pool ClientConnPool) (*http.Response, error) {
addr := authorityAddr(req.URL.Scheme, req.URL.Host)
for retry := 0; ; retry++ {
cc, err := pool.GetClientConn(req, addr)
if err != nil {
t.vlogf("http2: Transport failed to get client conn for %s: %v", addr, err)
return nil, err
}
reused := !atomic.CompareAndSwapUint32(&cc.atomicReused, 0, 1)
traceGotConn(req, cc, reused)
res, err := cc.RoundTrip(req)
if err != nil && retry <= 6 {
roundTripErr := err
if req, err = shouldRetryRequest(req, err); err == nil {
// After the first retry, do exponential backoff with 10% jitter.
if retry == 0 {
t.vlogf("RoundTrip retrying after failure: %v", roundTripErr)
continue
}
backoff := float64(uint(1) << (uint(retry) - 1))
backoff += backoff * (0.1 * mathrand.Float64())
d := time.Second * time.Duration(backoff)
tm := time.NewTimer(d)
select {
case <-tm.C:
t.vlogf("RoundTrip retrying after failure: %v", roundTripErr)
continue
case <-req.Context().Done():
tm.Stop()
err = req.Context().Err()
}
}
}
if err == errClientConnNotEstablished {
// This ClientConn was created recently,
// this is the first request to use it,
// and the connection is closed and not usable.
//
// In this state, cc.idleTimer will remove the conn from the pool
// when it fires. Stop the timer and remove it here so future requests
// won't try to use this connection.
//
// If the timer has already fired and we're racing it, the redundant
// call to MarkDead is harmless.
cc.stopIdleTimer()
pool.MarkDead(cc)
}
if err != nil {
t.vlogf("RoundTrip failure: %v", err)
return nil, err
}
return res, nil
}
}
// shouldRetryRequest is called by RoundTrip when a request fails to get
// response headers. It is always called with a non-nil error.
// It returns either a request to retry (either the same request, or a
// modified clone), or an error if the request can't be replayed.
func shouldRetryRequest(req *http.Request, err error) (*http.Request, error) {
if !canRetryError(err) {
return nil, err
}
// If the Body is nil (or http.NoBody), it's safe to reuse
// this request and its Body.
if req.Body == nil || req.Body == http.NoBody {
return req, nil
}
// If the request body can be reset back to its original
// state via the optional req.GetBody, do that.
if req.GetBody != nil {
body, err := req.GetBody()
if err != nil {
return nil, err
}
newReq := *req
newReq.Body = body
return &newReq, nil
}
// The Request.Body can't reset back to the beginning, but we
// don't seem to have started to read from it yet, so reuse
// the request directly.
if err == errClientConnUnusable {
return req, nil
}
return nil, fmt.Errorf("http2: Transport: cannot retry err [%v] after Request.Body was written; define Request.GetBody to avoid this error", err)
}
func canRetryError(err error) bool {
if err == errClientConnUnusable || err == errClientConnGotGoAway {
return true
}
if se, ok := err.(StreamError); ok {
return se.Code == ErrCodeRefusedStream
}
return false
}
func (t *Transport) vlogf(format string, args ...interface{}) {
if VerboseLogs {
t.logf(format, args...)
}
}
func (t *Transport) logf(format string, args ...interface{}) {
log.Printf(format, args...)
}
func (t *Transport) dialTLS(ctx context.Context, network, addr string, tlsCfg *tls.Config) (net.Conn, error) {
if t.DialTLSContext != nil {
return t.DialTLSContext(ctx, network, addr, tlsCfg)
} else if t.DialTLS != nil {
return t.DialTLS(network, addr, tlsCfg)
}
tlsCn, err := t.dialTLSWithContext(ctx, network, addr, tlsCfg)
if err != nil {
return nil, err
}
state := tlsCn.ConnectionState()
if p := state.NegotiatedProtocol; p != NextProtoTLS {
return nil, fmt.Errorf("http2: unexpected ALPN protocol %q; want %q", p, NextProtoTLS)
}
if !state.NegotiatedProtocolIsMutual {
return nil, errors.New("http2: could not negotiate protocol mutually")
}
return tlsCn, nil
}
// dialTLSWithContext uses tls.Dialer, added in Go 1.15, to open a TLS
// connection.
func (t *Transport) dialTLSWithContext(ctx context.Context, network, addr string, cfg *tls.Config) (*tls.Conn, error) {
dialer := &tls.Dialer{
Config: cfg,
}
cn, err := dialer.DialContext(ctx, network, addr)
if err != nil {
return nil, err
}
tlsCn := cn.(*tls.Conn) // DialContext comment promises this will always succeed
return tlsCn, nil
}
// GoAwayError is returned by the Transport when the server closes the
// TCP connection after sending a GOAWAY frame.
type GoAwayError struct {
LastStreamID uint32
ErrCode ErrCode
DebugData string
}
func (e GoAwayError) Error() string {
return fmt.Sprintf("http2: server sent GOAWAY and closed the connection; LastStreamID=%v, ErrCode=%v, debug=%q",
e.LastStreamID, e.ErrCode, e.DebugData)
}
// noCachedConnError is the concrete type of ErrNoCachedConn, which
// needs to be detected by net/http regardless of whether it's its
// bundled version (in h2_bundle.go with a rewritten type name) or
// from a user's x/net/http2. As such, as it has a unique method name
// (IsHTTP2NoCachedConnError) that net/http sniffs for via func
// isNoCachedConnError.
type noCachedConnError struct{}
func (noCachedConnError) IsHTTP2NoCachedConnError() {}
func (noCachedConnError) Error() string { return "http2: no cached connection was available" }
// isNoCachedConnError reports whether err is of type noCachedConnError
// or its equivalent renamed type in net/http2's h2_bundle.go. Both types
// may coexist in the same running program.
func isNoCachedConnError(err error) bool {
_, ok := err.(interface{ IsHTTP2NoCachedConnError() })
return ok
}
var ErrNoCachedConn error = noCachedConnError{}

381
vendor/golang.org/x/net/http2/transport_wrap.go generated vendored Normal file
View File

@@ -0,0 +1,381 @@
// Copyright 2026 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build go1.27 && !http2legacy
// Transport wrapping a net/http.Transport.
package http2
import (
"context"
"crypto/tls"
"errors"
"math"
"net"
"net/http"
"net/http/httptrace"
"slices"
"sync"
"time"
)
func configureTransport(t1 *http.Transport) error {
// ConfigureTransport is a no-op: The http.Transport already supports HTTP/2.
return nil
}
func configureTransports(t1 *http.Transport) (*Transport, error) {
// ConfigureTransport returns an http2.Transport with a configuration
// linked to the http.Transport's.
tr2 := &Transport{}
tr2.configure(t1)
return tr2, nil
}
// transportConfig is passed to net/http.Transport.RegisterProtocol("http/2", config).
// It provides the net/http.Transport with access to the configuration in the
// x/net/http2.Transport.
type transportConfig struct {
t *Transport
}
// Registered is called by net/http.Transport.RegisterProtocol,
// to let us know that it understands the registration mechanism we're using.
func (t transportConfig) Registered(t1 *http.Transport) {
t.t.t1 = t1
}
func (t transportConfig) DisableCompression() bool {
return t.t.DisableCompression
}
func (t transportConfig) MaxHeaderListSize() int64 {
return int64(t.t.MaxHeaderListSize)
}
func (t transportConfig) IdleConnTimeout() time.Duration {
return t.t.IdleConnTimeout
}
func (t transportConfig) HTTP2Config() http.HTTP2Config {
return http.HTTP2Config{
StrictMaxConcurrentRequests: t.t.StrictMaxConcurrentStreams,
MaxDecoderHeaderTableSize: int(t.t.MaxDecoderHeaderTableSize),
MaxEncoderHeaderTableSize: int(t.t.MaxEncoderHeaderTableSize),
MaxReadFrameSize: int(t.t.MaxReadFrameSize),
SendPingTimeout: t.t.ReadIdleTimeout,
PingTimeout: t.t.PingTimeout,
WriteByteTimeout: t.t.WriteByteTimeout,
CountError: t.t.CountError,
}
}
// ExternalRoundTrip reports whether the Transport wants to take control of the RoundTrip call.
// If the user hasn't configured a custom connection pool, we leave the RoundTrip up to net/http.
func (t transportConfig) ExternalRoundTrip() bool {
return t.t.ConnPool != nil
}
// RoundTrip is used when the http.Transport is passing control of the full
// RoundTrip to us--connection pooling, retries, etc.
//
// This is only used when the http2.Transport has a user-provided ConnPool.
// Any other time, net/http handles everything.
func (t transportConfig) RoundTrip(req *http.Request) (*http.Response, error) {
if t.t.ConnPool == nil {
return nil, http.ErrSkipAltProtocol
}
return t.t.RoundTrip(req)
}
// netConnContextKey passes a net.Conn to http.Transport.NewClientConn.
// See http2.Transport.NewClientConn.
type netConnContextKey struct{}
// ConnFromContext lets the http.Transport fetch a net.Conn out of a context
// passed to NewClientConn. See http2.Transport.NewClientConn.
func (t transportConfig) ConnFromContext(ctx context.Context) net.Conn {
nc, _ := ctx.Value(netConnContextKey{}).(net.Conn)
return nc
}
// http2TransportContextKey marks a RoundTrip as needing its dial handled by the http2.Transport.
// We set this for http2.RoundTrip calls, where the historical behavior is to use the
// http2.Transport's dialer.
type http2TransportContextKey struct{}
// DialFromContext dials a new connection using the http2.Transport's DialTLS/DialTLSContext.
func (t transportConfig) DialFromContext(ctx context.Context, network, address string) (net.Conn, error) {
if ctx.Value(http2TransportContextKey{}) == nil {
// We're being called from a RoundTrip that did not start with an http2.Transport.
// Use the http.Transport's dialer.
return nil, errors.ErrUnsupported
}
tlsConf := t.t.TLSClientConfig
if tlsConf == nil {
tlsConf = &tls.Config{}
} else {
tlsConf = tlsConf.Clone()
}
if !slices.Contains(tlsConf.NextProtos, "h2") {
tlsConf.NextProtos = append([]string{"h2"}, tlsConf.NextProtos...)
}
if tlsConf.ServerName == "" {
host, _, err := net.SplitHostPort(address)
if err == nil {
tlsConf.ServerName = host
}
}
return t.t.dialTLS(ctx, network, address, tlsConf)
}
type transportInternal struct {
initOnce sync.Once
t1 *http.Transport
}
func (t *Transport) init() {
t.initOnce.Do(func() {
if t.t1 != nil {
return
}
t1 := &http.Transport{}
t.configure(t1)
})
}
func (t *Transport) configure(t1 *http.Transport) {
t1.RegisterProtocol("http/2", transportConfig{t})
// tr2.t1 is set by transportConfig.Registered.
if t.t1 != t1 {
panic("http2: net/http does not support this version of x/net/http2")
}
}
func (t *Transport) roundTripOpt(req *http.Request, opt RoundTripOpt) (*http.Response, error) {
t.init()
if req.URL.Scheme == "http" && !t.AllowHTTP {
return nil, errors.New("http2: unencrypted HTTP/2 not enabled")
}
// When the Transport has a user-provided connection pool (unusual, deprecated),
// we need to handle picking a connection, retrys, etc.
if t.ConnPool != nil {
return t.roundTripViaPool(req, opt, t.ConnPool)
}
// Setting this context key lets net/http know that if it is necessary to dial
// a new connection, we should handle the net.Dial.
//
// Both http.Transport and http2.Transport allow the user to provide a custom
// dial function, and historically you only get the dial function from the
// Transport you're calling RoundTrip on.
ctx := context.WithValue(req.Context(), http2TransportContextKey{}, t)
req = req.WithContext(ctx)
return t.t1.RoundTrip(req)
}
func (t *Transport) closeIdleConnections() {
t.init()
t.t1.CloseIdleConnections()
}
func (t *Transport) newUserClientConn(c net.Conn) (*ClientConn, error) {
// http.Transport's NewClientConn doesn't provide a supported way to create
// a connection from a net.Conn. (This might be useful to add in the future?)
// We're going to craftily sneak one in via the context key, with the
// scheme of "http/2" telling NewClientConn to look for it.
ctx := context.WithValue(context.Background(), netConnContextKey{}, c)
nhcc, err := t.t1.NewClientConn(ctx, "http/2", "")
if err != nil {
return nil, err
}
cc := &ClientConn{cc: nhcc, tr: t, tconn: c}
nhcc.SetStateHook(cc.stateHook)
return cc, nil
}
// ClientConn is the state of a single HTTP/2 client connection to an
// HTTP/2 server.
type ClientConn struct {
cc *http.ClientConn
tconn net.Conn
tr *Transport
doNotReuse bool
mu sync.Mutex
closing bool
closed bool
roundTrips int
reserved int
starting int
pending int
maxConcurrent int
lastIdle time.Time
shutdownc chan struct{}
atomicReused uint32 // whether conn is being reused; atomic
}
func (cc *ClientConn) roundTrip(req *http.Request) (*http.Response, error) {
err := func() error {
cc.mu.Lock()
defer cc.mu.Unlock()
if cc.doNotReuse {
return errClientConnUnusable
}
cc.roundTrips++
if cc.reserved > 0 {
// We've already reserved a concurrency slot for this request.
cc.reserved--
} else if cc.cc.Reserve() != nil {
// We don't seem to have an available concurrency slot,
// so bump the pending count (requests waiting for a slot).
cc.pending++
}
// ClientConn.Shutdown will not shut down the conn while
// cc.starting > 0 or cc.cc.InFlight() > 0.
//
// The starting state covers the gap between us deciding to
// start sending the request, and actually sending it.
cc.starting++
return nil
}()
if err != nil {
return nil, err
}
resp, err := cc.cc.RoundTrip(req)
cc.mu.Lock()
cc.starting--
if cc.pending > 0 {
// A request completing frees up a concurrency slot for
// a pending request to start.
cc.pending--
}
cc.updateStateLocked()
cc.mu.Unlock()
return resp, err
}
func (cc *ClientConn) canTakeNewRequest() bool {
return cc.cc.Available() > 0 && !cc.doNotReuse
}
func (cc *ClientConn) close() error {
return cc.cc.Close()
}
func (cc *ClientConn) ping(ctx context.Context) error {
// Ask net/http to ping its connection by sending a request with a method of ":ping".
_, err := cc.cc.RoundTrip((&http.Request{
Method: ":ping",
}).WithContext(ctx))
return err
}
func (cc *ClientConn) reserveNewRequest() bool {
cc.mu.Lock()
defer cc.mu.Unlock()
if cc.doNotReuse {
return false
}
if err := cc.cc.Reserve(); err != nil {
return false
}
cc.reserved++
return true
}
func (cc *ClientConn) setDoNotReuse() {
cc.mu.Lock()
defer cc.mu.Unlock()
cc.doNotReuse = true
cc.closing = true
}
func (cc *ClientConn) shutdown(ctx context.Context) error {
cc.mu.Lock()
inFlight := cc.cc.InFlight() + cc.starting
if inFlight > 0 && cc.shutdownc == nil {
cc.shutdownc = make(chan struct{})
}
shutdownc := cc.shutdownc
cc.mu.Unlock()
if shutdownc != nil {
// Wait for in-flight requests to finish.
select {
case <-shutdownc:
case <-ctx.Done():
return ctx.Err()
}
}
cc.cc.Close()
return nil
}
func (cc *ClientConn) state() ClientConnState {
cc.mu.Lock()
defer cc.mu.Unlock()
cc.updateStateLocked()
return ClientConnState{
Closed: cc.closed,
Closing: cc.closing,
StreamsActive: cc.cc.InFlight() - cc.reserved,
StreamsReserved: cc.reserved,
StreamsPending: cc.pending,
MaxConcurrentStreams: uint32(min(int64(cc.maxConcurrent), math.MaxUint32)),
LastIdle: cc.lastIdle,
}
}
// stateHook is the http.ClientConn's state hook.
func (cc *ClientConn) stateHook(*http.ClientConn) {
cc.mu.Lock()
defer cc.mu.Unlock()
cc.updateStateLocked()
}
func (cc *ClientConn) updateStateLocked() {
if cc.cc.Err() != nil && !cc.closed {
cc.closing = true
cc.closed = true
if cc.tr.ConnPool != nil {
// Do the ConnPool update in another goroutine,
// to avoid holding the conn mutex while it runs.
go cc.tr.ConnPool.MarkDead(cc)
}
}
if cc.cc.InFlight() == 0 && cc.roundTrips > 0 && cc.starting == 0 {
cc.lastIdle = time.Now()
}
if !cc.closed {
// This is slightly racy (a request could start or finish in between
// the Available and InFlight calls), but the best we can do given that
// the net/http ClientConn API doesn't expose the conn's max concurrency.
cc.maxConcurrent = cc.cc.Available() + cc.cc.InFlight()
}
if cc.shutdownc != nil && cc.cc.InFlight()+cc.starting == 0 {
close(cc.shutdownc)
cc.shutdownc = nil
}
}
func (cc *ClientConn) stopIdleTimer() {}
// traceGotConn is (when http2legacy is not enabled) only used for tracing
// connections acquired while using a user-provided ClientConnPool.
func traceGotConn(req *http.Request, cc *ClientConn, reused bool) {
trace := httptrace.ContextClientTrace(req.Context())
if trace == nil || trace.GotConn == nil {
return
}
ci := httptrace.GotConnInfo{Conn: cc.tconn}
ci.Reused = reused
trace.GotConn(ci)
}

View File

@@ -2,54 +2,12 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build !(go1.27 && !http2legacy)
package http2
import "fmt"
// WriteScheduler is the interface implemented by HTTP/2 write schedulers.
// Methods are never called concurrently.
//
// Deprecated: User-provided write schedulers are deprecated.
type WriteScheduler interface {
// OpenStream opens a new stream in the write scheduler.
// It is illegal to call this with streamID=0 or with a streamID that is
// already open -- the call may panic.
OpenStream(streamID uint32, options OpenStreamOptions)
// CloseStream closes a stream in the write scheduler. Any frames queued on
// this stream should be discarded. It is illegal to call this on a stream
// that is not open -- the call may panic.
CloseStream(streamID uint32)
// AdjustStream adjusts the priority of the given stream. This may be called
// on a stream that has not yet been opened or has been closed. Note that
// RFC 7540 allows PRIORITY frames to be sent on streams in any state. See:
// https://tools.ietf.org/html/rfc7540#section-5.1
AdjustStream(streamID uint32, priority PriorityParam)
// Push queues a frame in the scheduler. In most cases, this will not be
// called with wr.StreamID()!=0 unless that stream is currently open. The one
// exception is RST_STREAM frames, which may be sent on idle or closed streams.
Push(wr FrameWriteRequest)
// Pop dequeues the next frame to write. Returns false if no frames can
// be written. Frames with a given wr.StreamID() are Pop'd in the same
// order they are Push'd, except RST_STREAM frames. No frames should be
// discarded except by CloseStream.
Pop() (wr FrameWriteRequest, ok bool)
}
// OpenStreamOptions specifies extra options for WriteScheduler.OpenStream.
//
// Deprecated: User-provided write schedulers are deprecated.
type OpenStreamOptions struct {
// PusherID is zero if the stream was initiated by the client. Otherwise,
// PusherID names the stream that pushed the newly opened stream.
PusherID uint32
// priority is used to set the priority of the newly opened stream.
priority PriorityParam
}
// FrameWriteRequest is a request to write a frame.
//
// Deprecated: User-provided write schedulers are deprecated.

90
vendor/golang.org/x/net/http2/writesched_common.go generated vendored Normal file
View File

@@ -0,0 +1,90 @@
// Copyright 2026 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package http2
// WriteScheduler is the interface implemented by HTTP/2 write schedulers.
// Methods are never called concurrently.
//
// Deprecated: User-provided write schedulers are deprecated.
type WriteScheduler interface {
// OpenStream opens a new stream in the write scheduler.
// It is illegal to call this with streamID=0 or with a streamID that is
// already open -- the call may panic.
OpenStream(streamID uint32, options OpenStreamOptions)
// CloseStream closes a stream in the write scheduler. Any frames queued on
// this stream should be discarded. It is illegal to call this on a stream
// that is not open -- the call may panic.
CloseStream(streamID uint32)
// AdjustStream adjusts the priority of the given stream. This may be called
// on a stream that has not yet been opened or has been closed. Note that
// RFC 7540 allows PRIORITY frames to be sent on streams in any state. See:
// https://tools.ietf.org/html/rfc7540#section-5.1
AdjustStream(streamID uint32, priority PriorityParam)
// Push queues a frame in the scheduler. In most cases, this will not be
// called with wr.StreamID()!=0 unless that stream is currently open. The one
// exception is RST_STREAM frames, which may be sent on idle or closed streams.
Push(wr FrameWriteRequest)
// Pop dequeues the next frame to write. Returns false if no frames can
// be written. Frames with a given wr.StreamID() are Pop'd in the same
// order they are Push'd, except RST_STREAM frames. No frames should be
// discarded except by CloseStream.
Pop() (wr FrameWriteRequest, ok bool)
}
// OpenStreamOptions specifies extra options for WriteScheduler.OpenStream.
//
// Deprecated: User-provided write schedulers are deprecated.
type OpenStreamOptions struct {
// PusherID is zero if the stream was initiated by the client. Otherwise,
// PusherID names the stream that pushed the newly opened stream.
PusherID uint32
// priority is used to set the priority of the newly opened stream.
priority PriorityParam
}
// PriorityWriteSchedulerConfig configures a priorityWriteScheduler.
//
// Deprecated: User-provided write schedulers are deprecated.
type PriorityWriteSchedulerConfig struct {
// MaxClosedNodesInTree controls the maximum number of closed streams to
// retain in the priority tree. Setting this to zero saves a small amount
// of memory at the cost of performance.
//
// See RFC 7540, Section 5.3.4:
// "It is possible for a stream to become closed while prioritization
// information ... is in transit. ... This potentially creates suboptimal
// prioritization, since the stream could be given a priority that is
// different from what is intended. To avoid these problems, an endpoint
// SHOULD retain stream prioritization state for a period after streams
// become closed. The longer state is retained, the lower the chance that
// streams are assigned incorrect or default priority values."
MaxClosedNodesInTree int
// MaxIdleNodesInTree controls the maximum number of idle streams to
// retain in the priority tree. Setting this to zero saves a small amount
// of memory at the cost of performance.
//
// See RFC 7540, Section 5.3.4:
// Similarly, streams that are in the "idle" state can be assigned
// priority or become a parent of other streams. This allows for the
// creation of a grouping node in the dependency tree, which enables
// more flexible expressions of priority. Idle streams begin with a
// default priority (Section 5.3.5).
MaxIdleNodesInTree int
// ThrottleOutOfOrderWrites enables write throttling to help ensure that
// data is delivered in priority order. This works around a race where
// stream B depends on stream A and both streams are about to call Write
// to queue DATA frames. If B wins the race, a naive scheduler would eagerly
// write as much data from B as possible, but this is suboptimal because A
// is a higher-priority stream. With throttling enabled, we write a small
// amount of data from B to minimize the amount of bandwidth that B can
// steal from A.
ThrottleOutOfOrderWrites bool
}

View File

@@ -2,6 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build !(go1.27 && !http2legacy)
package http2
import (
@@ -13,47 +15,6 @@ import (
// RFC 7540, Section 5.3.5: the default weight is 16.
const priorityDefaultWeightRFC7540 = 15 // 16 = 15 + 1
// PriorityWriteSchedulerConfig configures a priorityWriteScheduler.
//
// Deprecated: User-provided write schedulers are deprecated.
type PriorityWriteSchedulerConfig struct {
// MaxClosedNodesInTree controls the maximum number of closed streams to
// retain in the priority tree. Setting this to zero saves a small amount
// of memory at the cost of performance.
//
// See RFC 7540, Section 5.3.4:
// "It is possible for a stream to become closed while prioritization
// information ... is in transit. ... This potentially creates suboptimal
// prioritization, since the stream could be given a priority that is
// different from what is intended. To avoid these problems, an endpoint
// SHOULD retain stream prioritization state for a period after streams
// become closed. The longer state is retained, the lower the chance that
// streams are assigned incorrect or default priority values."
MaxClosedNodesInTree int
// MaxIdleNodesInTree controls the maximum number of idle streams to
// retain in the priority tree. Setting this to zero saves a small amount
// of memory at the cost of performance.
//
// See RFC 7540, Section 5.3.4:
// Similarly, streams that are in the "idle" state can be assigned
// priority or become a parent of other streams. This allows for the
// creation of a grouping node in the dependency tree, which enables
// more flexible expressions of priority. Idle streams begin with a
// default priority (Section 5.3.5).
MaxIdleNodesInTree int
// ThrottleOutOfOrderWrites enables write throttling to help ensure that
// data is delivered in priority order. This works around a race where
// stream B depends on stream A and both streams are about to call Write
// to queue DATA frames. If B wins the race, a naive scheduler would eagerly
// write as much data from B as possible, but this is suboptimal because A
// is a higher-priority stream. With throttling enabled, we write a small
// amount of data from B to minimize the amount of bandwidth that B can
// steal from A.
ThrottleOutOfOrderWrites bool
}
// NewPriorityWriteScheduler constructs a WriteScheduler that schedules
// frames by following HTTP/2 priorities as described in RFC 7540 Section 5.3.
// If cfg is nil, default options are used.

View File

@@ -2,6 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build !(go1.27 && !http2legacy)
package http2
import (

View File

@@ -2,6 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build !(go1.27 && !http2legacy)
package http2
import "math"

View File

@@ -2,6 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build !(go1.27 && !http2legacy)
package http2
import (

View File

@@ -1,13 +0,0 @@
// Code generated by running "go generate" in golang.org/x/text. DO NOT EDIT.
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build go1.18
package idna
// Transitional processing is disabled by default in Go 1.18.
// https://golang.org/issue/47510
const transitionalLookup = false

View File

@@ -4,8 +4,6 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build go1.10
// Package idna implements IDNA2008 using the compatibility processing
// defined by UTS (Unicode Technical Standard) #46, which defines a standard to
// deal with the transition from IDNA2003.
@@ -20,6 +18,7 @@ package idna // import "golang.org/x/net/idna"
import (
"fmt"
"strings"
"unicode"
"unicode/utf8"
"golang.org/x/text/secure/bidirule"
@@ -27,6 +26,8 @@ import (
"golang.org/x/text/unicode/norm"
)
const unicode16 = unicode.Version >= "16.0.0"
// NOTE: Unlike common practice in Go APIs, the functions will return a
// sanitized domain name in case of errors. Browsers sometimes use a partially
// evaluated string as lookup.
@@ -101,6 +102,11 @@ func ValidateLabels(enable bool) Option {
}
}
// validateLabels reports whether the ValidateLabels option is enabled.
func (p *Profile) validateLabels() bool {
return p.fromPuny != nil
}
// CheckHyphens sets whether to check for correct use of hyphens ('-') in
// labels. Most web browsers do not have this option set, since labels such as
// "r3---sn-apo3qvuoxuxbt-j5pe" are in common use.
@@ -263,6 +269,10 @@ func (p *Profile) String() string {
return s
}
// Transitional processing is disabled by default as of Go 1.18.
// https://golang.org/issue/47510
const transitionalLookup = false
var (
// Punycode is a Profile that does raw punycode processing with a minimum
// of validation.
@@ -324,15 +334,30 @@ func (e labelError) Error() string {
return fmt.Sprintf("idna: invalid label %q", e.label)
}
type runeError rune
func (e runeError) code() string { return "P1" }
func (e runeError) Error() string {
return fmt.Sprintf("idna: disallowed rune %U", e)
type runeError struct {
r rune
code_ string
}
// process implements the algorithm described in section 4 of UTS #46,
// see https://www.unicode.org/reports/tr46.
func (e runeError) code() string { return e.code_ }
func (e runeError) Error() string {
return fmt.Sprintf("idna: disallowed rune %U", e.r)
}
// code16 returns old for Unicode < 16, new for Unicode >= 16.
func code16(old, new string) string {
if unicode16 {
return new
}
return old
}
// process10 implements the algorithm described in section 4 of UTS #46.
// It implements both the Unicode 10 algorithm
// (https://www.unicode.org/reports/tr46/tr46-19.html)
// and the Unicode 16 algorithm
// (https://www.unicode.org/reports/tr46/tr46-35.html)
// depending on unicode16, which in turn depends on unicode.Version.
func (p *Profile) process(s string, toASCII bool) (string, error) {
var err error
var isBidi bool
@@ -347,8 +372,12 @@ func (p *Profile) process(s string, toASCII bool) (string, error) {
// TODO: allow for a quick check of the tables data.
// It seems like we should only create this error on ToASCII, but the
// UTS 46 conformance tests suggests we should always check this.
labelCode := "X4_2"
if !unicode16 || toASCII {
labelCode = "A4"
}
if err == nil && p.verifyDNSLength && s == "" {
err = &labelError{s, "A4"}
err = labelError{s, labelCode}
}
labels := labelIter{orig: s}
for ; !labels.done(); labels.next() {
@@ -357,12 +386,13 @@ func (p *Profile) process(s string, toASCII bool) (string, error) {
// Empty labels are not okay. The label iterator skips the last
// label if it is empty.
if err == nil && p.verifyDNSLength {
err = &labelError{s, "A4"}
err = labelError{s, labelCode}
}
continue
}
if strings.HasPrefix(label, acePrefix) {
u, err2 := decode(label[len(acePrefix):])
enc := label[len(acePrefix):]
u, err2 := decode(enc)
if err2 != nil {
if err == nil {
err = err2
@@ -370,6 +400,9 @@ func (p *Profile) process(s string, toASCII bool) (string, error) {
// Spec says keep the old label.
continue
}
if unicode16 && err == nil && len(u) > 0 && isASCII(u) {
err = punyError(enc)
}
isBidi = isBidi || bidirule.DirectionString(u) != bidi.LeftToRight
labels.set(u)
if err == nil && p.fromPuny != nil {
@@ -379,16 +412,16 @@ func (p *Profile) process(s string, toASCII bool) (string, error) {
// This should be called on NonTransitional, according to the
// spec, but that currently does not have any effect. Use the
// original profile to preserve options.
err = p.validateLabel(u)
err = p.validateLabel(u, labelCode)
}
} else if err == nil {
err = p.validateLabel(label)
err = p.validateLabel(label, labelCode)
}
}
if isBidi && p.bidirule != nil && err == nil {
for labels.reset(); !labels.done(); labels.next() {
if !p.bidirule(labels.label()) {
err = &labelError{s, "B"}
err = labelError{s, "B"}
break
}
}
@@ -406,24 +439,36 @@ func (p *Profile) process(s string, toASCII bool) (string, error) {
}
n := len(label)
if p.verifyDNSLength && err == nil && (n == 0 || n > 63) {
err = &labelError{label, "A4"}
err = labelError{label, labelCode}
}
}
}
s = labels.result()
if toASCII && p.verifyDNSLength && err == nil {
if unicode16 && strings.HasSuffix(s, ".") {
err = labelError{s, labelCode}
}
// Compute the length of the domain name minus the root label and its dot.
n := len(s)
if n > 0 && s[n-1] == '.' {
n--
}
if len(s) < 1 || n > 253 {
err = &labelError{s, "A4"}
err = labelError{s, labelCode}
}
}
return s, err
}
func isASCII(s string) bool {
for _, c := range []byte(s) {
if c >= 0x80 {
return false
}
}
return true
}
func normalize(p *Profile, s string) (mapped string, isBidi bool, err error) {
// TODO: consider first doing a quick check to see if any of these checks
// need to be done. This will make it slower in the general case, but
@@ -436,12 +481,12 @@ func normalize(p *Profile, s string) (mapped string, isBidi bool, err error) {
func validateRegistration(p *Profile, s string) (idem string, bidi bool, err error) {
// TODO: filter need for normalization in loop below.
if !norm.NFC.IsNormalString(s) {
return s, false, &labelError{s, "V1"}
return s, false, labelError{s, "V1"}
}
for i := 0; i < len(s); {
v, sz := trie.lookupString(s[i:])
if sz == 0 {
return s, bidi, runeError(utf8.RuneError)
return s, bidi, runeError{utf8.RuneError, "P1"}
}
bidi = bidi || info(v).isBidi(s[i:])
// Copy bytes not copied so far.
@@ -449,9 +494,12 @@ func validateRegistration(p *Profile, s string) (idem string, bidi bool, err err
// TODO: handle the NV8 defined in the Unicode idna data set to allow
// for strict conformance to IDNA2008.
case valid, deviation:
if sz == 1 && p.useSTD3Rules && !allowedSTD3(rune(s[i])) {
return s, bidi, runeError{rune(s[i]), "P1"}
}
case disallowed, mapped, unknown, ignored:
r, _ := utf8.DecodeRuneInString(s[i:])
return s, bidi, runeError(r)
return s, bidi, runeError{r, "P1"}
}
i += sz
}
@@ -489,7 +537,7 @@ func validateAndMap(p *Profile, s string) (vm string, bidi bool, err error) {
b = append(b, "\ufffd"...)
k = len(s)
if err == nil {
err = runeError(utf8.RuneError)
err = runeError{utf8.RuneError, "P1"}
}
break
}
@@ -502,14 +550,26 @@ func validateAndMap(p *Profile, s string) (vm string, bidi bool, err error) {
case valid:
continue
case disallowed:
if err == nil {
// Unicode 16 delays the error until validateLabels.
// Unicode 10 gave an error now.
if !unicode16 && err == nil {
r, _ := utf8.DecodeRuneInString(s[start:])
err = runeError(r)
err = runeError{r, "P1"}
}
continue
case mapped, deviation:
case deviation:
if unicode16 && !p.transitional {
break
}
fallthrough
case mapped:
b = append(b, s[k:start]...)
b = info(v).appendMapping(b, s[start:i])
// Unicode 16 requires a special case to handle ẞ -> ss in transitional mode.
if unicode16 && p.transitional && s[start:start+sz] == "ẞ" {
b = append(b, "ss"...)
} else {
b = info(v).appendMapping(b, s[start:i])
}
case ignored:
b = append(b, s[k:start]...)
// drop the rune
@@ -600,13 +660,13 @@ const acePrefix = "xn--"
func (p *Profile) simplify(cat category) category {
switch cat {
case disallowedSTD3Mapped:
case disallowedSTD3Mapped: // only happens for pre-Unicode 16
if p.useSTD3Rules {
cat = disallowed
} else {
cat = mapped
}
case disallowedSTD3Valid:
case disallowedSTD3Valid: // only happens for pre-Unicode 16
if p.useSTD3Rules {
cat = disallowed
} else {
@@ -625,17 +685,18 @@ func (p *Profile) simplify(cat category) category {
func validateFromPunycode(p *Profile, s string) error {
if !norm.NFC.IsNormalString(s) {
return &labelError{s, "V1"}
return labelError{s, "V1"}
}
// TODO: detect whether string may have to be normalized in the following
// loop.
for i := 0; i < len(s); {
v, sz := trie.lookupString(s[i:])
if sz == 0 {
return runeError(utf8.RuneError)
return runeError{utf8.RuneError, "P1"}
}
if c := p.simplify(info(v).category()); c != valid && c != deviation {
return &labelError{s, "V6"}
cat := info(v).category()
if c := p.simplify(cat); c != valid && c != deviation {
return labelError{s, code16("V6", "V7")}
}
i += sz
}
@@ -704,23 +765,51 @@ var joinStates = [][numJoinTypes]joinState{
},
}
// allowedSTD3 reports whether r is a rune that can appear in a domain name
// according to STD3. We allow all non-ASCII runes and then letters, digits, hyphens.
// We also add dot so that this can be run against the whole name and not just
// a single name element (label). The surrounding code checks dots well enough.
func allowedSTD3(r rune) bool {
return r >= 0x80 || 'a' <= r && r <= 'z' || '0' <= r && r <= '9' || r == '-' || r == '.'
}
// validateLabel validates the criteria from Section 4.1. Item 1, 4, and 6 are
// already implicitly satisfied by the overall implementation.
func (p *Profile) validateLabel(s string) (err error) {
func (p *Profile) validateLabel(s string, labelCode string) (err error) {
if s == "" {
if p.verifyDNSLength {
return &labelError{s, "A4"}
return labelError{s, labelCode}
}
return nil
}
if p.checkHyphens {
if len(s) > 4 && s[2] == '-' && s[3] == '-' {
return &labelError{s, "V2"}
return labelError{s, "V2"}
}
if s[0] == '-' || s[len(s)-1] == '-' {
return &labelError{s, "V3"}
return labelError{s, "V3"}
}
}
// Unicode 16's TR 46 delays the rune validity checks until after the label is decoded.
// (validateAndMap did not reject them earlier.)
if unicode16 && p.validateLabels() {
for i := 0; i < len(s); {
v, sz := trie.lookupString(s[i:])
if sz == 0 {
return runeError{utf8.RuneError, "P1"}
}
cat := info(v).category()
if c := p.simplify(cat); c != valid && (!p.transitional || c != deviation) {
return labelError{s, "V7"}
}
if sz == 1 && p.useSTD3Rules && !allowedSTD3(rune(s[i])) {
return runeError{rune(s[i]), "U1"}
}
i += sz
}
}
if !p.checkJoiners {
return nil
}
@@ -729,7 +818,7 @@ func (p *Profile) validateLabel(s string) (err error) {
v, sz := trie.lookupString(s)
x := info(v)
if x.isModifier() {
return &labelError{s, "V5"}
return labelError{s, code16("V5", "V6")}
}
// Quickly return in the absence of zero-width (non) joiners.
if strings.Index(s, zwj) == -1 && strings.Index(s, zwnj) == -1 {
@@ -754,8 +843,9 @@ func (p *Profile) validateLabel(s string) (err error) {
x = info(v)
}
if st == stateFAIL || st == stateAfter {
return &labelError{s, "C"}
return labelError{s, "C"}
}
return nil
}
@@ -767,3 +857,24 @@ func ascii(s string) bool {
}
return true
}
// appendMapping appends the mapping for the respective rune. isMapped must be
// true. A mapping is a categorization of a rune as defined in UTS #46.
func (c info) appendMapping(b []byte, s string) []byte {
index := int(c >> indexShift)
if c&xorBit == 0 {
p := index
return append(b, mappings[mappingIndex[p]:mappingIndex[p+1]]...)
}
b = append(b, s...)
if c&inlineXOR == inlineXOR {
// TODO: support and handle two-byte inline masks
b[len(b)-1] ^= byte(index)
} else {
for p := len(b) - int(xorData[index]); p < len(b); p++ {
index++
b[p] ^= xorData[index]
}
}
return b
}

View File

@@ -1,717 +0,0 @@
// Code generated by running "go generate" in golang.org/x/text. DO NOT EDIT.
// Copyright 2016 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build !go1.10
// Package idna implements IDNA2008 using the compatibility processing
// defined by UTS (Unicode Technical Standard) #46, which defines a standard to
// deal with the transition from IDNA2003.
//
// IDNA2008 (Internationalized Domain Names for Applications), is defined in RFC
// 5890, RFC 5891, RFC 5892, RFC 5893 and RFC 5894.
// UTS #46 is defined in https://www.unicode.org/reports/tr46.
// See https://unicode.org/cldr/utility/idna.jsp for a visualization of the
// differences between these two standards.
package idna // import "golang.org/x/net/idna"
import (
"fmt"
"strings"
"unicode/utf8"
"golang.org/x/text/secure/bidirule"
"golang.org/x/text/unicode/norm"
)
// NOTE: Unlike common practice in Go APIs, the functions will return a
// sanitized domain name in case of errors. Browsers sometimes use a partially
// evaluated string as lookup.
// TODO: the current error handling is, in my opinion, the least opinionated.
// Other strategies are also viable, though:
// Option 1) Return an empty string in case of error, but allow the user to
// specify explicitly which errors to ignore.
// Option 2) Return the partially evaluated string if it is itself a valid
// string, otherwise return the empty string in case of error.
// Option 3) Option 1 and 2.
// Option 4) Always return an empty string for now and implement Option 1 as
// needed, and document that the return string may not be empty in case of
// error in the future.
// I think Option 1 is best, but it is quite opinionated.
// ToASCII is a wrapper for Punycode.ToASCII.
func ToASCII(s string) (string, error) {
return Punycode.process(s, true)
}
// ToUnicode is a wrapper for Punycode.ToUnicode.
func ToUnicode(s string) (string, error) {
return Punycode.process(s, false)
}
// An Option configures a Profile at creation time.
type Option func(*options)
// Transitional sets a Profile to use the Transitional mapping as defined in UTS
// #46. This will cause, for example, "ß" to be mapped to "ss". Using the
// transitional mapping provides a compromise between IDNA2003 and IDNA2008
// compatibility. It is used by some browsers when resolving domain names. This
// option is only meaningful if combined with MapForLookup.
func Transitional(transitional bool) Option {
return func(o *options) { o.transitional = transitional }
}
// VerifyDNSLength sets whether a Profile should fail if any of the IDN parts
// are longer than allowed by the RFC.
//
// This option corresponds to the VerifyDnsLength flag in UTS #46.
func VerifyDNSLength(verify bool) Option {
return func(o *options) { o.verifyDNSLength = verify }
}
// RemoveLeadingDots removes leading label separators. Leading runes that map to
// dots, such as U+3002 IDEOGRAPHIC FULL STOP, are removed as well.
func RemoveLeadingDots(remove bool) Option {
return func(o *options) { o.removeLeadingDots = remove }
}
// ValidateLabels sets whether to check the mandatory label validation criteria
// as defined in Section 5.4 of RFC 5891. This includes testing for correct use
// of hyphens ('-'), normalization, validity of runes, and the context rules.
// In particular, ValidateLabels also sets the CheckHyphens and CheckJoiners flags
// in UTS #46.
func ValidateLabels(enable bool) Option {
return func(o *options) {
// Don't override existing mappings, but set one that at least checks
// normalization if it is not set.
if o.mapping == nil && enable {
o.mapping = normalize
}
o.trie = trie
o.checkJoiners = enable
o.checkHyphens = enable
if enable {
o.fromPuny = validateFromPunycode
} else {
o.fromPuny = nil
}
}
}
// CheckHyphens sets whether to check for correct use of hyphens ('-') in
// labels. Most web browsers do not have this option set, since labels such as
// "r3---sn-apo3qvuoxuxbt-j5pe" are in common use.
//
// This option corresponds to the CheckHyphens flag in UTS #46.
func CheckHyphens(enable bool) Option {
return func(o *options) { o.checkHyphens = enable }
}
// CheckJoiners sets whether to check the ContextJ rules as defined in Appendix
// A of RFC 5892, concerning the use of joiner runes.
//
// This option corresponds to the CheckJoiners flag in UTS #46.
func CheckJoiners(enable bool) Option {
return func(o *options) {
o.trie = trie
o.checkJoiners = enable
}
}
// StrictDomainName limits the set of permissible ASCII characters to those
// allowed in domain names as defined in RFC 1034 (A-Z, a-z, 0-9 and the
// hyphen). This is set by default for MapForLookup and ValidateForRegistration,
// but is only useful if ValidateLabels is set.
//
// This option is useful, for instance, for browsers that allow characters
// outside this range, for example a '_' (U+005F LOW LINE). See
// http://www.rfc-editor.org/std/std3.txt for more details.
//
// This option corresponds to the UseSTD3ASCIIRules flag in UTS #46.
func StrictDomainName(use bool) Option {
return func(o *options) { o.useSTD3Rules = use }
}
// NOTE: the following options pull in tables. The tables should not be linked
// in as long as the options are not used.
// BidiRule enables the Bidi rule as defined in RFC 5893. Any application
// that relies on proper validation of labels should include this rule.
//
// This option corresponds to the CheckBidi flag in UTS #46.
func BidiRule() Option {
return func(o *options) { o.bidirule = bidirule.ValidString }
}
// ValidateForRegistration sets validation options to verify that a given IDN is
// properly formatted for registration as defined by Section 4 of RFC 5891.
func ValidateForRegistration() Option {
return func(o *options) {
o.mapping = validateRegistration
StrictDomainName(true)(o)
ValidateLabels(true)(o)
VerifyDNSLength(true)(o)
BidiRule()(o)
}
}
// MapForLookup sets validation and mapping options such that a given IDN is
// transformed for domain name lookup according to the requirements set out in
// Section 5 of RFC 5891. The mappings follow the recommendations of RFC 5894,
// RFC 5895 and UTS 46. It does not add the Bidi Rule. Use the BidiRule option
// to add this check.
//
// The mappings include normalization and mapping case, width and other
// compatibility mappings.
func MapForLookup() Option {
return func(o *options) {
o.mapping = validateAndMap
StrictDomainName(true)(o)
ValidateLabels(true)(o)
RemoveLeadingDots(true)(o)
}
}
type options struct {
transitional bool
useSTD3Rules bool
checkHyphens bool
checkJoiners bool
verifyDNSLength bool
removeLeadingDots bool
trie *idnaTrie
// fromPuny calls validation rules when converting A-labels to U-labels.
fromPuny func(p *Profile, s string) error
// mapping implements a validation and mapping step as defined in RFC 5895
// or UTS 46, tailored to, for example, domain registration or lookup.
mapping func(p *Profile, s string) (string, error)
// bidirule, if specified, checks whether s conforms to the Bidi Rule
// defined in RFC 5893.
bidirule func(s string) bool
}
// A Profile defines the configuration of a IDNA mapper.
type Profile struct {
options
}
func apply(o *options, opts []Option) {
for _, f := range opts {
f(o)
}
}
// New creates a new Profile.
//
// With no options, the returned Profile is the most permissive and equals the
// Punycode Profile. Options can be passed to further restrict the Profile. The
// MapForLookup and ValidateForRegistration options set a collection of options,
// for lookup and registration purposes respectively, which can be tailored by
// adding more fine-grained options, where later options override earlier
// options.
func New(o ...Option) *Profile {
p := &Profile{}
apply(&p.options, o)
return p
}
// ToASCII converts a domain or domain label to its ASCII form. For example,
// ToASCII("bücher.example.com") is "xn--bcher-kva.example.com", and
// ToASCII("golang") is "golang". If an error is encountered it will return
// an error and a (partially) processed result.
func (p *Profile) ToASCII(s string) (string, error) {
return p.process(s, true)
}
// ToUnicode converts a domain or domain label to its Unicode form. For example,
// ToUnicode("xn--bcher-kva.example.com") is "bücher.example.com", and
// ToUnicode("golang") is "golang". If an error is encountered it will return
// an error and a (partially) processed result.
func (p *Profile) ToUnicode(s string) (string, error) {
pp := *p
pp.transitional = false
return pp.process(s, false)
}
// String reports a string with a description of the profile for debugging
// purposes. The string format may change with different versions.
func (p *Profile) String() string {
s := ""
if p.transitional {
s = "Transitional"
} else {
s = "NonTransitional"
}
if p.useSTD3Rules {
s += ":UseSTD3Rules"
}
if p.checkHyphens {
s += ":CheckHyphens"
}
if p.checkJoiners {
s += ":CheckJoiners"
}
if p.verifyDNSLength {
s += ":VerifyDNSLength"
}
return s
}
var (
// Punycode is a Profile that does raw punycode processing with a minimum
// of validation.
Punycode *Profile = punycode
// Lookup is the recommended profile for looking up domain names, according
// to Section 5 of RFC 5891. The exact configuration of this profile may
// change over time.
Lookup *Profile = lookup
// Display is the recommended profile for displaying domain names.
// The configuration of this profile may change over time.
Display *Profile = display
// Registration is the recommended profile for checking whether a given
// IDN is valid for registration, according to Section 4 of RFC 5891.
Registration *Profile = registration
punycode = &Profile{}
lookup = &Profile{options{
transitional: true,
removeLeadingDots: true,
useSTD3Rules: true,
checkHyphens: true,
checkJoiners: true,
trie: trie,
fromPuny: validateFromPunycode,
mapping: validateAndMap,
bidirule: bidirule.ValidString,
}}
display = &Profile{options{
useSTD3Rules: true,
removeLeadingDots: true,
checkHyphens: true,
checkJoiners: true,
trie: trie,
fromPuny: validateFromPunycode,
mapping: validateAndMap,
bidirule: bidirule.ValidString,
}}
registration = &Profile{options{
useSTD3Rules: true,
verifyDNSLength: true,
checkHyphens: true,
checkJoiners: true,
trie: trie,
fromPuny: validateFromPunycode,
mapping: validateRegistration,
bidirule: bidirule.ValidString,
}}
// TODO: profiles
// Register: recommended for approving domain names: don't do any mappings
// but rather reject on invalid input. Bundle or block deviation characters.
)
type labelError struct{ label, code_ string }
func (e labelError) code() string { return e.code_ }
func (e labelError) Error() string {
return fmt.Sprintf("idna: invalid label %q", e.label)
}
type runeError rune
func (e runeError) code() string { return "P1" }
func (e runeError) Error() string {
return fmt.Sprintf("idna: disallowed rune %U", e)
}
// process implements the algorithm described in section 4 of UTS #46,
// see https://www.unicode.org/reports/tr46.
func (p *Profile) process(s string, toASCII bool) (string, error) {
var err error
if p.mapping != nil {
s, err = p.mapping(p, s)
}
// Remove leading empty labels.
if p.removeLeadingDots {
for ; len(s) > 0 && s[0] == '.'; s = s[1:] {
}
}
// It seems like we should only create this error on ToASCII, but the
// UTS 46 conformance tests suggests we should always check this.
if err == nil && p.verifyDNSLength && s == "" {
err = &labelError{s, "A4"}
}
labels := labelIter{orig: s}
for ; !labels.done(); labels.next() {
label := labels.label()
if label == "" {
// Empty labels are not okay. The label iterator skips the last
// label if it is empty.
if err == nil && p.verifyDNSLength {
err = &labelError{s, "A4"}
}
continue
}
if strings.HasPrefix(label, acePrefix) {
u, err2 := decode(label[len(acePrefix):])
if err2 != nil {
if err == nil {
err = err2
}
// Spec says keep the old label.
continue
}
labels.set(u)
if err == nil && p.fromPuny != nil {
err = p.fromPuny(p, u)
}
if err == nil {
// This should be called on NonTransitional, according to the
// spec, but that currently does not have any effect. Use the
// original profile to preserve options.
err = p.validateLabel(u)
}
} else if err == nil {
err = p.validateLabel(label)
}
}
if toASCII {
for labels.reset(); !labels.done(); labels.next() {
label := labels.label()
if !ascii(label) {
a, err2 := encode(acePrefix, label)
if err == nil {
err = err2
}
label = a
labels.set(a)
}
n := len(label)
if p.verifyDNSLength && err == nil && (n == 0 || n > 63) {
err = &labelError{label, "A4"}
}
}
}
s = labels.result()
if toASCII && p.verifyDNSLength && err == nil {
// Compute the length of the domain name minus the root label and its dot.
n := len(s)
if n > 0 && s[n-1] == '.' {
n--
}
if len(s) < 1 || n > 253 {
err = &labelError{s, "A4"}
}
}
return s, err
}
func normalize(p *Profile, s string) (string, error) {
return norm.NFC.String(s), nil
}
func validateRegistration(p *Profile, s string) (string, error) {
if !norm.NFC.IsNormalString(s) {
return s, &labelError{s, "V1"}
}
for i := 0; i < len(s); {
v, sz := trie.lookupString(s[i:])
// Copy bytes not copied so far.
switch p.simplify(info(v).category()) {
// TODO: handle the NV8 defined in the Unicode idna data set to allow
// for strict conformance to IDNA2008.
case valid, deviation:
case disallowed, mapped, unknown, ignored:
r, _ := utf8.DecodeRuneInString(s[i:])
return s, runeError(r)
}
i += sz
}
return s, nil
}
func validateAndMap(p *Profile, s string) (string, error) {
var (
err error
b []byte
k int
)
for i := 0; i < len(s); {
v, sz := trie.lookupString(s[i:])
start := i
i += sz
// Copy bytes not copied so far.
switch p.simplify(info(v).category()) {
case valid:
continue
case disallowed:
if err == nil {
r, _ := utf8.DecodeRuneInString(s[start:])
err = runeError(r)
}
continue
case mapped, deviation:
b = append(b, s[k:start]...)
b = info(v).appendMapping(b, s[start:i])
case ignored:
b = append(b, s[k:start]...)
// drop the rune
case unknown:
b = append(b, s[k:start]...)
b = append(b, "\ufffd"...)
}
k = i
}
if k == 0 {
// No changes so far.
s = norm.NFC.String(s)
} else {
b = append(b, s[k:]...)
if norm.NFC.QuickSpan(b) != len(b) {
b = norm.NFC.Bytes(b)
}
// TODO: the punycode converters require strings as input.
s = string(b)
}
return s, err
}
// A labelIter allows iterating over domain name labels.
type labelIter struct {
orig string
slice []string
curStart int
curEnd int
i int
}
func (l *labelIter) reset() {
l.curStart = 0
l.curEnd = 0
l.i = 0
}
func (l *labelIter) done() bool {
return l.curStart >= len(l.orig)
}
func (l *labelIter) result() string {
if l.slice != nil {
return strings.Join(l.slice, ".")
}
return l.orig
}
func (l *labelIter) label() string {
if l.slice != nil {
return l.slice[l.i]
}
p := strings.IndexByte(l.orig[l.curStart:], '.')
l.curEnd = l.curStart + p
if p == -1 {
l.curEnd = len(l.orig)
}
return l.orig[l.curStart:l.curEnd]
}
// next sets the value to the next label. It skips the last label if it is empty.
func (l *labelIter) next() {
l.i++
if l.slice != nil {
if l.i >= len(l.slice) || l.i == len(l.slice)-1 && l.slice[l.i] == "" {
l.curStart = len(l.orig)
}
} else {
l.curStart = l.curEnd + 1
if l.curStart == len(l.orig)-1 && l.orig[l.curStart] == '.' {
l.curStart = len(l.orig)
}
}
}
func (l *labelIter) set(s string) {
if l.slice == nil {
l.slice = strings.Split(l.orig, ".")
}
l.slice[l.i] = s
}
// acePrefix is the ASCII Compatible Encoding prefix.
const acePrefix = "xn--"
func (p *Profile) simplify(cat category) category {
switch cat {
case disallowedSTD3Mapped:
if p.useSTD3Rules {
cat = disallowed
} else {
cat = mapped
}
case disallowedSTD3Valid:
if p.useSTD3Rules {
cat = disallowed
} else {
cat = valid
}
case deviation:
if !p.transitional {
cat = valid
}
case validNV8, validXV8:
// TODO: handle V2008
cat = valid
}
return cat
}
func validateFromPunycode(p *Profile, s string) error {
if !norm.NFC.IsNormalString(s) {
return &labelError{s, "V1"}
}
for i := 0; i < len(s); {
v, sz := trie.lookupString(s[i:])
if c := p.simplify(info(v).category()); c != valid && c != deviation {
return &labelError{s, "V6"}
}
i += sz
}
return nil
}
const (
zwnj = "\u200c"
zwj = "\u200d"
)
type joinState int8
const (
stateStart joinState = iota
stateVirama
stateBefore
stateBeforeVirama
stateAfter
stateFAIL
)
var joinStates = [][numJoinTypes]joinState{
stateStart: {
joiningL: stateBefore,
joiningD: stateBefore,
joinZWNJ: stateFAIL,
joinZWJ: stateFAIL,
joinVirama: stateVirama,
},
stateVirama: {
joiningL: stateBefore,
joiningD: stateBefore,
},
stateBefore: {
joiningL: stateBefore,
joiningD: stateBefore,
joiningT: stateBefore,
joinZWNJ: stateAfter,
joinZWJ: stateFAIL,
joinVirama: stateBeforeVirama,
},
stateBeforeVirama: {
joiningL: stateBefore,
joiningD: stateBefore,
joiningT: stateBefore,
},
stateAfter: {
joiningL: stateFAIL,
joiningD: stateBefore,
joiningT: stateAfter,
joiningR: stateStart,
joinZWNJ: stateFAIL,
joinZWJ: stateFAIL,
joinVirama: stateAfter, // no-op as we can't accept joiners here
},
stateFAIL: {
0: stateFAIL,
joiningL: stateFAIL,
joiningD: stateFAIL,
joiningT: stateFAIL,
joiningR: stateFAIL,
joinZWNJ: stateFAIL,
joinZWJ: stateFAIL,
joinVirama: stateFAIL,
},
}
// validateLabel validates the criteria from Section 4.1. Item 1, 4, and 6 are
// already implicitly satisfied by the overall implementation.
func (p *Profile) validateLabel(s string) error {
if s == "" {
if p.verifyDNSLength {
return &labelError{s, "A4"}
}
return nil
}
if p.bidirule != nil && !p.bidirule(s) {
return &labelError{s, "B"}
}
if p.checkHyphens {
if len(s) > 4 && s[2] == '-' && s[3] == '-' {
return &labelError{s, "V2"}
}
if s[0] == '-' || s[len(s)-1] == '-' {
return &labelError{s, "V3"}
}
}
if !p.checkJoiners {
return nil
}
trie := p.trie // p.checkJoiners is only set if trie is set.
// TODO: merge the use of this in the trie.
v, sz := trie.lookupString(s)
x := info(v)
if x.isModifier() {
return &labelError{s, "V5"}
}
// Quickly return in the absence of zero-width (non) joiners.
if strings.Index(s, zwj) == -1 && strings.Index(s, zwnj) == -1 {
return nil
}
st := stateStart
for i := 0; ; {
jt := x.joinType()
if s[i:i+sz] == zwj {
jt = joinZWJ
} else if s[i:i+sz] == zwnj {
jt = joinZWNJ
}
st = joinStates[st][jt]
if x.isViramaModifier() {
st = joinStates[st][joinVirama]
}
if i += sz; i == len(s) {
break
}
v, sz = trie.lookupString(s[i:])
x = info(v)
}
if st == stateFAIL || st == stateAfter {
return &labelError{s, "C"}
}
return nil
}
func ascii(s string) bool {
for i := 0; i < len(s); i++ {
if s[i] >= utf8.RuneSelf {
return false
}
}
return true
}

View File

@@ -1,11 +0,0 @@
// Code generated by running "go generate" in golang.org/x/text. DO NOT EDIT.
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build !go1.18
package idna
const transitionalLookup = true

View File

@@ -28,7 +28,7 @@ const (
tmin int32 = 1
)
func punyError(s string) error { return &labelError{s, "A3"} }
func punyError(s string) error { return &labelError{s, code16("A3", "P4")} }
// decode decodes a string as specified in section 6.2.
func decode(encoded string) (string, error) {
@@ -108,6 +108,9 @@ func encode(prefix, s string) (string, error) {
delta, n, bias := int32(0), initialN, initialBias
b, remaining := int32(0), int32(0)
for _, r := range s {
if unicode16 && r == 0xfffd {
return s, &labelError{s, "A3"}
}
if r < 0x80 {
b++
output = append(output, byte(r))

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
// Code generated by running "go generate" in golang.org/x/text. DO NOT EDIT.
//go:build go1.21
//go:build !go1.27
package idna

5302
vendor/golang.org/x/net/idna/tables17.0.0.go generated vendored Normal file

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,30 +0,0 @@
// Code generated by running "go generate" in golang.org/x/text. DO NOT EDIT.
// Copyright 2016 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build !go1.16
package idna
// appendMapping appends the mapping for the respective rune. isMapped must be
// true. A mapping is a categorization of a rune as defined in UTS #46.
func (c info) appendMapping(b []byte, s string) []byte {
index := int(c >> indexShift)
if c&xorBit == 0 {
s := mappings[index:]
return append(b, s[1:s[0]+1]...)
}
b = append(b, s...)
if c&inlineXOR == inlineXOR {
// TODO: support and handle two-byte inline masks
b[len(b)-1] ^= byte(index)
} else {
for p := len(b) - int(xorData[index]); p < len(b); p++ {
index++
b[p] ^= xorData[index]
}
}
return b
}

View File

@@ -1,30 +0,0 @@
// Code generated by running "go generate" in golang.org/x/text. DO NOT EDIT.
// Copyright 2016 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build go1.16
package idna
// appendMapping appends the mapping for the respective rune. isMapped must be
// true. A mapping is a categorization of a rune as defined in UTS #46.
func (c info) appendMapping(b []byte, s string) []byte {
index := int(c >> indexShift)
if c&xorBit == 0 {
p := index
return append(b, mappings[mappingIndex[p]:mappingIndex[p+1]]...)
}
b = append(b, s...)
if c&inlineXOR == inlineXOR {
// TODO: support and handle two-byte inline masks
b[len(b)-1] ^= byte(index)
} else {
for p := len(b) - int(xorData[index]); p < len(b); p++ {
index++
b[p] ^= xorData[index]
}
}
return b
}

View File

@@ -448,6 +448,14 @@ func NewServerRequest(rp ServerRequestParam) ServerRequestResult {
url_ = &url.URL{Host: rp.Authority}
requestURI = rp.Authority // mimic HTTP/1 server behavior
} else {
// "[The :path] pseudo-header field MUST NOT be empty [...]"
// https://www.rfc-editor.org/rfc/rfc9113.html#section-8.3.1-2.4.2
if rp.Path == "" || (rp.Path[0] != '/' && rp.Path != "*") {
return ServerRequestResult{
InvalidReason: "bad_path",
}
}
var err error
url_, err = url.ParseRequestURI(rp.Path)
if err != nil {

19
vendor/golang.org/x/sys/cpu/cpu.go generated vendored
View File

@@ -152,13 +152,17 @@ var ARM struct {
// The booleans in Loong64 contain the correspondingly named cpu feature bit.
// The struct is padded to avoid false sharing.
var Loong64 struct {
_ CacheLinePad
HasLSX bool // support 128-bit vector extension
HasLASX bool // support 256-bit vector extension
HasCRC32 bool // support CRC instruction
HasLAM_BH bool // support AM{SWAP/ADD}[_DB].{B/H} instruction
HasLAMCAS bool // support AMCAS[_DB].{B/H/W/D} instruction
_ CacheLinePad
_ CacheLinePad
HasLSX bool // support 128-bit vector extension
HasLASX bool // support 256-bit vector extension
HasCRC32 bool // support CRC instruction
HasLAMCAS bool // support AMCAS[_DB].{B/H/W/D}
HasLAM_BH bool // support AM{SWAP/ADD}[_DB].{B/H} instruction
HasLLACQ_SCREL bool // support LLACQ.{W/D}, SCREL.{W/D} instruction
HasSCQ bool // support SC.Q instruction
HasDBAR_HINTS bool // supports finer-grained DBAR hints
_ CacheLinePad
}
// MIPS64X contains the supported CPU features of the current mips64/mips64le
@@ -232,6 +236,7 @@ var RISCV64 struct {
HasZba bool // Address generation instructions extension
HasZbb bool // Basic bit-manipulation extension
HasZbs bool // Single-bit instructions extension
HasZbc bool // Carryless multiplication extension
HasZvbb bool // Vector Basic Bit-manipulation
HasZvbc bool // Vector Carryless Multiplication
HasZvkb bool // Vector Cryptography Bit-manipulation

View File

@@ -58,6 +58,7 @@ const (
riscv_HWPROBE_EXT_ZBA = 0x8
riscv_HWPROBE_EXT_ZBB = 0x10
riscv_HWPROBE_EXT_ZBS = 0x20
riscv_HWPROBE_EXT_ZBC = 0x80
riscv_HWPROBE_EXT_ZVBB = 0x20000
riscv_HWPROBE_EXT_ZVBC = 0x40000
riscv_HWPROBE_EXT_ZVKB = 0x80000
@@ -108,6 +109,7 @@ func doinit() {
RISCV64.HasZba = isSet(v, riscv_HWPROBE_EXT_ZBA)
RISCV64.HasZbb = isSet(v, riscv_HWPROBE_EXT_ZBB)
RISCV64.HasZbs = isSet(v, riscv_HWPROBE_EXT_ZBS)
RISCV64.HasZbc = isSet(v, riscv_HWPROBE_EXT_ZBC)
RISCV64.HasZvbb = isSet(v, riscv_HWPROBE_EXT_ZVBB)
RISCV64.HasZvbc = isSet(v, riscv_HWPROBE_EXT_ZVBC)
RISCV64.HasZvkb = isSet(v, riscv_HWPROBE_EXT_ZVKB)

View File

@@ -15,8 +15,13 @@ const (
cpucfg1_CRC32 = 1 << 25
// CPUCFG2 bits
cpucfg2_LAM_BH = 1 << 27
cpucfg2_LAMCAS = 1 << 28
cpucfg2_LAM_BH = 1 << 27
cpucfg2_LAMCAS = 1 << 28
cpucfg2_LLACQ_SCREL = 1 << 29
cpucfg2_SCQ = 1 << 30
// CPUCFG3 bits
cpucfg3_DBAR_HINTS = 1 << 17
)
func initOptions() {
@@ -26,6 +31,9 @@ func initOptions() {
{Name: "crc32", Feature: &Loong64.HasCRC32},
{Name: "lam_bh", Feature: &Loong64.HasLAM_BH},
{Name: "lamcas", Feature: &Loong64.HasLAMCAS},
{Name: "llacq_screl", Feature: &Loong64.HasLLACQ_SCREL},
{Name: "scq", Feature: &Loong64.HasSCQ},
{Name: "dbar_hints", Feature: &Loong64.HasDBAR_HINTS},
}
// The CPUCFG data on Loong64 only reflects the hardware capabilities,
@@ -37,10 +45,14 @@ func initOptions() {
// through CPUCFG
cfg1 := get_cpucfg(1)
cfg2 := get_cpucfg(2)
cfg3 := get_cpucfg(3)
Loong64.HasCRC32 = cfgIsSet(cfg1, cpucfg1_CRC32)
Loong64.HasLAMCAS = cfgIsSet(cfg2, cpucfg2_LAMCAS)
Loong64.HasLAM_BH = cfgIsSet(cfg2, cpucfg2_LAM_BH)
Loong64.HasLLACQ_SCREL = cfgIsSet(cfg2, cpucfg2_LLACQ_SCREL)
Loong64.HasSCQ = cfgIsSet(cfg2, cpucfg2_SCQ)
Loong64.HasDBAR_HINTS = cfgIsSet(cfg3, cpucfg3_DBAR_HINTS)
}
func get_cpucfg(reg uint32) uint32

View File

@@ -2,7 +2,7 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build !darwin && !linux && !netbsd && !openbsd && arm64
//go:build !darwin && !linux && !netbsd && !openbsd && !windows && arm64
package cpu

View File

@@ -16,6 +16,7 @@ func initOptions() {
{Name: "zba", Feature: &RISCV64.HasZba},
{Name: "zbb", Feature: &RISCV64.HasZbb},
{Name: "zbs", Feature: &RISCV64.HasZbs},
{Name: "zbc", Feature: &RISCV64.HasZbc},
// RISC-V Cryptography Extensions
{Name: "zvbb", Feature: &RISCV64.HasZvbb},
{Name: "zvbc", Feature: &RISCV64.HasZvbc},

26
vendor/golang.org/x/sys/cpu/cpu_windows.go generated vendored Normal file
View File

@@ -0,0 +1,26 @@
// Copyright 2026 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package cpu
//go:generate go run golang.org/x/sys/windows/mkwinsyscall -systemdll=false -output zcpu_windows.go cpu_windows.go
//sys isProcessorFeaturePresent(ProcessorFeature uint32) (ret bool) = kernel32.IsProcessorFeaturePresent
// The processor features to be tested for IsProcessorFeaturePresent, see
// https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-isprocessorfeaturepresent
const (
_PF_ARM_V8_CRYPTO_INSTRUCTIONS_AVAILABLE = 30
_PF_ARM_V8_CRC32_INSTRUCTIONS_AVAILABLE = 31
_PF_ARM_V81_ATOMIC_INSTRUCTIONS_AVAILABLE = 34
_PF_ARM_V82_DP_INSTRUCTIONS_AVAILABLE = 43
_PF_ARM_V83_JSCVT_INSTRUCTIONS_AVAILABLE = 44
_PF_ARM_V83_LRCPC_INSTRUCTIONS_AVAILABLE = 45
_PF_ARM_SVE_INSTRUCTIONS_AVAILABLE = 46
_PF_ARM_SVE2_INSTRUCTIONS_AVAILABLE = 47
_PF_ARM_SHA3_INSTRUCTIONS_AVAILABLE = 64
_PF_ARM_SHA512_INSTRUCTIONS_AVAILABLE = 65
)

38
vendor/golang.org/x/sys/cpu/cpu_windows_arm64.go generated vendored Normal file
View File

@@ -0,0 +1,38 @@
// Copyright 2026 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package cpu
func doinit() {
// set HasASIMD and HasFP to true as per
// https://learn.microsoft.com/en-us/cpp/build/arm64-windows-abi-conventions?view=msvc-170#base-requirements
//
// The ARM64 version of Windows always presupposes that it's running on an ARMv8 or later architecture.
// Both floating-point and NEON support are presumed to be present in hardware.
//
ARM64.HasASIMD = true
ARM64.HasFP = true
if isProcessorFeaturePresent(_PF_ARM_V8_CRYPTO_INSTRUCTIONS_AVAILABLE) {
ARM64.HasAES = true
ARM64.HasPMULL = true
ARM64.HasSHA1 = true
ARM64.HasSHA2 = true
}
ARM64.HasSHA3 = isProcessorFeaturePresent(_PF_ARM_SHA3_INSTRUCTIONS_AVAILABLE)
ARM64.HasCRC32 = isProcessorFeaturePresent(_PF_ARM_V8_CRC32_INSTRUCTIONS_AVAILABLE)
ARM64.HasSHA512 = isProcessorFeaturePresent(_PF_ARM_SHA512_INSTRUCTIONS_AVAILABLE)
ARM64.HasATOMICS = isProcessorFeaturePresent(_PF_ARM_V81_ATOMIC_INSTRUCTIONS_AVAILABLE)
if isProcessorFeaturePresent(_PF_ARM_V82_DP_INSTRUCTIONS_AVAILABLE) {
ARM64.HasASIMDDP = true
ARM64.HasASIMDRDM = true
}
if isProcessorFeaturePresent(_PF_ARM_V83_LRCPC_INSTRUCTIONS_AVAILABLE) {
ARM64.HasLRCPC = true
ARM64.HasSM3 = true
}
ARM64.HasSVE = isProcessorFeaturePresent(_PF_ARM_SVE_INSTRUCTIONS_AVAILABLE)
ARM64.HasSVE2 = isProcessorFeaturePresent(_PF_ARM_SVE2_INSTRUCTIONS_AVAILABLE)
ARM64.HasJSCVT = isProcessorFeaturePresent(_PF_ARM_V83_JSCVT_INSTRUCTIONS_AVAILABLE)
}

48
vendor/golang.org/x/sys/cpu/zcpu_windows.go generated vendored Normal file
View File

@@ -0,0 +1,48 @@
// Code generated by 'go generate'; DO NOT EDIT.
package cpu
import (
"syscall"
"unsafe"
)
var _ unsafe.Pointer
// Do the interface allocations only once for common
// Errno values.
const (
errnoERROR_IO_PENDING = 997
)
var (
errERROR_IO_PENDING error = syscall.Errno(errnoERROR_IO_PENDING)
errERROR_EINVAL error = syscall.EINVAL
)
// errnoErr returns common boxed Errno values, to prevent
// allocations at runtime.
func errnoErr(e syscall.Errno) error {
switch e {
case 0:
return errERROR_EINVAL
case errnoERROR_IO_PENDING:
return errERROR_IO_PENDING
}
// TODO: add more here, after collecting data on the common
// error values see on Windows. (perhaps when running
// all.bat?)
return e
}
var (
modkernel32 = syscall.NewLazyDLL("kernel32.dll")
procIsProcessorFeaturePresent = modkernel32.NewProc("IsProcessorFeaturePresent")
)
func isProcessorFeaturePresent(ProcessorFeature uint32) (ret bool) {
r0, _, _ := syscall.SyscallN(procIsProcessorFeaturePresent.Addr(), uintptr(ProcessorFeature))
ret = r0 != 0
return
}

View File

@@ -13,11 +13,19 @@ import (
const cpuSetSize = _CPU_SETSIZE / _NCPUBITS
// CPUSet represents a CPU affinity mask.
// CPUSet represents a bit mask of CPUs, to be used with [SchedGetaffinity], [SchedSetaffinity],
// and [SetMemPolicy].
//
// Note this type can only represent CPU IDs 0 through 1023.
// Use [CPUSetDynamic]/[NewCPUSet] instead to avoid this limit.
type CPUSet [cpuSetSize]cpuMask
func schedAffinity(trap uintptr, pid int, set *CPUSet) error {
_, _, e := RawSyscall(trap, uintptr(pid), uintptr(unsafe.Sizeof(*set)), uintptr(unsafe.Pointer(set)))
// CPUSetDynamic represents a bit mask of CPUs, to be used with [SchedGetaffinityDynamic],
// [SchedSetaffinityDynamic], and [SetMemPolicyDynamic]. Use [NewCPUSet] to allocate.
type CPUSetDynamic []cpuMask
func schedAffinity(trap uintptr, pid int, size uintptr, ptr unsafe.Pointer) error {
_, _, e := RawSyscall(trap, uintptr(pid), uintptr(size), uintptr(ptr))
if e != 0 {
return errnoErr(e)
}
@@ -27,13 +35,13 @@ func schedAffinity(trap uintptr, pid int, set *CPUSet) error {
// SchedGetaffinity gets the CPU affinity mask of the thread specified by pid.
// If pid is 0 the calling thread is used.
func SchedGetaffinity(pid int, set *CPUSet) error {
return schedAffinity(SYS_SCHED_GETAFFINITY, pid, set)
return schedAffinity(SYS_SCHED_GETAFFINITY, pid, unsafe.Sizeof(*set), unsafe.Pointer(set))
}
// SchedSetaffinity sets the CPU affinity mask of the thread specified by pid.
// If pid is 0 the calling thread is used.
func SchedSetaffinity(pid int, set *CPUSet) error {
return schedAffinity(SYS_SCHED_SETAFFINITY, pid, set)
return schedAffinity(SYS_SCHED_SETAFFINITY, pid, unsafe.Sizeof(*set), unsafe.Pointer(set))
}
// Zero clears the set s, so that it contains no CPUs.
@@ -45,9 +53,7 @@ func (s *CPUSet) Zero() {
// will silently ignore any invalid CPU bits in [CPUSet] so this is an
// efficient way of resetting the CPU affinity of a process.
func (s *CPUSet) Fill() {
for i := range s {
s[i] = ^cpuMask(0)
}
cpuMaskFill(s[:])
}
func cpuBitsIndex(cpu int) int {
@@ -58,24 +64,27 @@ func cpuBitsMask(cpu int) cpuMask {
return cpuMask(1 << (uint(cpu) % _NCPUBITS))
}
// Set adds cpu to the set s.
func (s *CPUSet) Set(cpu int) {
func cpuMaskFill(s []cpuMask) {
for i := range s {
s[i] = ^cpuMask(0)
}
}
func cpuMaskSet(s []cpuMask, cpu int) {
i := cpuBitsIndex(cpu)
if i < len(s) {
s[i] |= cpuBitsMask(cpu)
}
}
// Clear removes cpu from the set s.
func (s *CPUSet) Clear(cpu int) {
func cpuMaskClear(s []cpuMask, cpu int) {
i := cpuBitsIndex(cpu)
if i < len(s) {
s[i] &^= cpuBitsMask(cpu)
}
}
// IsSet reports whether cpu is in the set s.
func (s *CPUSet) IsSet(cpu int) bool {
func cpuMaskIsSet(s []cpuMask, cpu int) bool {
i := cpuBitsIndex(cpu)
if i < len(s) {
return s[i]&cpuBitsMask(cpu) != 0
@@ -83,11 +92,98 @@ func (s *CPUSet) IsSet(cpu int) bool {
return false
}
// Count returns the number of CPUs in the set s.
func (s *CPUSet) Count() int {
func cpuMaskCount(s []cpuMask) int {
c := 0
for _, b := range s {
c += bits.OnesCount64(uint64(b))
}
return c
}
// Set adds cpu to the set s. If cpu is out of bounds for s, no action is taken.
func (s *CPUSet) Set(cpu int) {
cpuMaskSet(s[:], cpu)
}
// Clear removes cpu from the set s. If cpu is out of bounds for s, no action is taken.
func (s *CPUSet) Clear(cpu int) {
cpuMaskClear(s[:], cpu)
}
// IsSet reports whether cpu is in the set s.
func (s *CPUSet) IsSet(cpu int) bool {
return cpuMaskIsSet(s[:], cpu)
}
// Count returns the number of CPUs in the set s.
func (s *CPUSet) Count() int {
return cpuMaskCount(s[:])
}
// NewCPUSet creates a CPU affinity mask capable of representing CPU IDs
// up to maxCPU (exclusive).
func NewCPUSet(maxCPU int) CPUSetDynamic {
numMasks := (maxCPU + _NCPUBITS - 1) / _NCPUBITS
if numMasks == 0 {
numMasks = 1
}
return make(CPUSetDynamic, numMasks)
}
// Zero clears the set s, so that it contains no CPUs.
func (s CPUSetDynamic) Zero() {
clear(s)
}
// Fill adds all possible CPU bits to the set s. On Linux, [SchedSetaffinityDynamic]
// will silently ignore any invalid CPU bits in [CPUSetDynamic] so this is an
// efficient way of resetting the CPU affinity of a process.
func (s CPUSetDynamic) Fill() {
cpuMaskFill(s)
}
// Set adds cpu to the set s. If cpu is out of bounds for s, no action is taken.
func (s CPUSetDynamic) Set(cpu int) {
cpuMaskSet(s, cpu)
}
// Clear removes cpu from the set s. If cpu is out of bounds for s, no action is taken.
func (s CPUSetDynamic) Clear(cpu int) {
cpuMaskClear(s, cpu)
}
// IsSet reports whether cpu is in the set s.
func (s CPUSetDynamic) IsSet(cpu int) bool {
return cpuMaskIsSet(s, cpu)
}
// Count returns the number of CPUs in the set s.
func (s CPUSetDynamic) Count() int {
return cpuMaskCount(s)
}
func (s CPUSetDynamic) size() uintptr {
return uintptr(len(s)) * unsafe.Sizeof(cpuMask(0))
}
func (s CPUSetDynamic) pointer() unsafe.Pointer {
if len(s) == 0 {
return nil
}
return unsafe.Pointer(&s[0])
}
// SchedGetaffinityDynamic gets the CPU affinity mask of the thread specified by pid.
// If pid is 0 the calling thread is used.
//
// If the set is smaller than the size of the affinity mask used by the kernel,
// [EINVAL] is returned.
func SchedGetaffinityDynamic(pid int, set CPUSetDynamic) error {
return schedAffinity(SYS_SCHED_GETAFFINITY, pid, set.size(), set.pointer())
}
// SchedSetaffinityDynamic sets the CPU affinity mask of the thread specified by pid.
// If pid is 0 the calling thread is used.
func SchedSetaffinityDynamic(pid int, set CPUSetDynamic) error {
return schedAffinity(SYS_SCHED_SETAFFINITY, pid, set.size(), set.pointer())
}

View File

@@ -51,7 +51,7 @@ if [[ "$GOOS" = "linux" ]]; then
# Files generated through docker (use $cmd so you can Ctl-C the build or run)
set -e
$cmd docker build --tag generate:$GOOS $GOOS
$cmd docker run --interactive --tty --volume $(cd -- "$(dirname -- "$0")/.." && pwd):/build generate:$GOOS
$cmd docker run --rm --interactive --tty --volume $(cd -- "$(dirname -- "$0")/.." && pwd):/build generate:$GOOS
exit
fi

Some files were not shown because too many files have changed in this diff Show More