Compare commits

..

18 Commits

Author SHA1 Message Date
Zakhar Bessarab
3e5527ae57 apptest/tests: add test to verify sparse cache usage
Sparse cache is only used for "final merges" - merge of data of previous months, so tests verify that.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-29 17:19:40 +04:00
Github Actions
0e08a7a125 Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@0d6e500 (#8180) 2025-01-29 13:27:11 +01:00
Roman Khavronenko
13c4324bb5 lib/cgroup: warn users about using fractional CPU quotas (#8175)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7988

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-29 13:19:08 +01:00
Aliaksandr Valialkin
95f182053b lib/logstorage: remove unnecesary abstraction - RowsFormatter
It is better to use the AppendFieldsToJSON function directly
instead of hiding it under RowsFormatter abstraction.
2025-01-28 18:03:18 +01:00
Aliaksandr Valialkin
ec64a1fd7c docs/Articles.md: add a link to https://tanmay-bhat.medium.com/reducing-inter-az-traffic-in-victoriametrics-with-zonekeeper-3bd7e1526796
Thanks to Tanmay Bhat ( https://tanmay-bhat.medium.com/ ) for the great article about VictoriaMetrics and vmagent!
2025-01-28 17:03:45 +01:00
Aliaksandr Valialkin
f61b5da617 go.mod: update Go from v1.23.5 to 1.23.5
This is a follow-up for 489631b227
2025-01-28 16:55:48 +01:00
Aliaksandr Valialkin
3c036e0d31 lib/logstorage: ignore logs with too long field names during data ingestion
Previously too long field names were silently truncated. This is not what most users expect.
It is better ignoring the whole log entry in this case and logging it with the WARNING message,
so human operator could notice and fix the ingestion of incorrect logs ASAP.

The commit also adds and updates the following entries to VictoriaLogs faq:

- https://docs.victoriametrics.com/victorialogs/faq/#how-many-fields-a-single-log-entry-may-contain
- https://docs.victoriametrics.com/victorialogs/faq/#what-is-the-maximum-supported-field-name-length
- https://docs.victoriametrics.com/victorialogs/faq/#what-length-a-log-record-is-expected-to-have

These entries are referred at `-insert.maxLineSizeBytes` and `-insert.maxFieldsPerLine` command-line descriptions
and at the WARNING messages, which are emitted when log entries are ignored because of some of these limits
are exceeded.
2025-01-28 16:55:48 +01:00
Aliaksandr Valialkin
aacb9b2726 app/vlogscli: show compact output mode line in the help output to be consistent with the naming for other output modes 2025-01-28 16:55:47 +01:00
Alexander Marshalov
c753f75e18 vmcloud docs: actualize parameters description due to changes in v1.108.0 (#6928) (#8172)
vmcloud docs: actualize parameters description due to changes in
v1.108.0 (#6928)
2025-01-28 16:05:07 +01:00
f41gh7
330461c635 docs/changelog: mention v1.102.12 and v1.97.17 releases
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2025-01-28 14:27:44 +01:00
Roman Khavronenko
5a1a28ba87 dashboards: consistently use process_cpu_cores_available for CPU usage
Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7988

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-28 13:58:06 +01:00
hagen1778
5b18dc0214 docs: re-order changes for better readability
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-28 09:57:15 +01:00
Aliaksandr Valialkin
10abdf34ab vendor: run make vendor-update 2025-01-27 23:10:31 +01:00
Fred Navruzov
e55b5fc605 docs/vmanomaly: release v1.19.2 (#8165)
### Describe Your Changes

docs/vmanomaly: release v1.19.2 (patch that addresses some of the bugs
found in 1.19.0)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Mathias Palmersheim <mathias@victoriametrics.com>
2025-01-27 21:03:34 +02:00
Aliaksandr Valialkin
3b34241380 lib/fs/fsutil: move lib/envutil to the more appropriate place at lib/fs/fsutil
This is a follow-up for 043d066133
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6871
2025-01-27 18:47:39 +01:00
Zakhar Bessarab
de144899b2 docs/{vmauth,VictoriaLogs}: add examples of using headers per-user
### Describe Your Changes
- add example a generic example to vmauth docs
- add an multi-tenancy usage example to VictoriaLogs docs
### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-27 20:14:43 +04:00
Aliaksandr Valialkin
dc9dba71b2 lib/storage: open per-month partitions in parallel
This should reduce the time needed for opening the storage with retentions exceeding a few months.

While at at, limit the concurrency of opening partitions in parallel to the number of available CPU cores,
since higher concurrency may increase RAM usage and CPU usage without performance improvements
if opening a single partition is CPU-bound task.

This is a follow-up for 17988942ab
2025-01-27 16:07:14 +01:00
Aliaksandr Valialkin
fd1568531d lib/filestream: use correct formatting option for error type in the error message 2025-01-27 15:23:52 +01:00
278 changed files with 3927 additions and 1679 deletions

View File

@@ -199,8 +199,8 @@ func (lmp *logMessageProcessor) AddRow(timestamp int64, fields, streamFields []l
lmp.bytesIngestedTotal.Add(n)
if len(fields) > *MaxFieldsPerLine {
rf := logstorage.RowFormatter(fields)
logger.Warnf("dropping log line with %d fields; it exceeds -insert.maxFieldsPerLine=%d; %s", len(fields), *MaxFieldsPerLine, rf)
line := logstorage.MarshalFieldsToJSON(nil, fields)
logger.Warnf("dropping log line with %d fields; it exceeds -insert.maxFieldsPerLine=%d; %s", len(fields), *MaxFieldsPerLine, line)
rowsDroppedTotalTooManyFields.Inc()
return
}

View File

@@ -8,8 +8,10 @@ import (
var (
// MaxLineSizeBytes is the maximum length of a single line for /insert/* handlers
MaxLineSizeBytes = flagutil.NewBytes("insert.maxLineSizeBytes", 256*1024, "The maximum size of a single line, which can be read by /insert/* handlers")
MaxLineSizeBytes = flagutil.NewBytes("insert.maxLineSizeBytes", 256*1024, "The maximum size of a single line, which can be read by /insert/* handlers; "+
"see https://docs.victoriametrics.com/victorialogs/faq/#what-length-a-log-record-is-expected-to-have")
// MaxFieldsPerLine is the maximum number of fields per line for /insert/* handlers
MaxFieldsPerLine = flag.Int("insert.maxFieldsPerLine", 1000, "The maximum number of log fields per line, which can be read by /insert/* handlers")
MaxFieldsPerLine = flag.Int("insert.maxFieldsPerLine", 1000, "The maximum number of log fields per line, which can be read by /insert/* handlers; "+
"see https://docs.victoriametrics.com/victorialogs/faq/#how-many-fields-a-single-log-entry-may-contain")
)

View File

@@ -270,7 +270,7 @@ func printCommandsHelp(w io.Writer) {
\h - show this help
\s - singleline json output mode
\m - multiline json output mode
\c - compact output
\c - compact output mode
\logfmt - logfmt output mode
\wrap_long_lines - toggles wrapping long lines
\tail <query> - live tail <query> results

View File

@@ -0,0 +1,76 @@
package tests
import (
"fmt"
"math/rand/v2"
"testing"
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/apptest"
)
func TestStorageUsesSparseCacheForFinalMerge(t *testing.T) {
tc := apptest.NewTestCase(t)
defer tc.Stop()
sut := tc.MustStartVmsingle("sparse-cache-final-merge", []string{`-retentionPeriod=100y`, `-downsampling.period={__name__=~"metric.*"}:5m:1s`})
// insert metrics daily over past 2 months
const metricsPerStep = 10
const steps = 35
const stepSize = int64(1 * 24 * 60 * 60 * 1000)
records := make([]string, metricsPerStep)
ts := time.Now().Add(-2 * 30 * 24 * time.Hour).UnixMilli()
for range steps {
for i := range metricsPerStep {
name := fmt.Sprintf("metric_%d", i)
records[i] = fmt.Sprintf("%s %d %d", name, rand.IntN(1000), ts)
}
sut.PrometheusAPIV1ImportPrometheus(t, records, apptest.QueryOpts{})
ts += stepSize
}
sut.ForceFlush(t)
sut.ForceMerge(t)
// todo: replace with a more reliable way to check if the merge is completed
// wait for merge to be completed
time.Sleep(5 * time.Second)
v := sut.GetIntMetric(t, `vm_cache_requests_total{type="indexdb/dataBlocksSparse"}`)
if v <= 0 {
t.Fatalf(`unexpected vm_cache_requests_total{type="indexdb/dataBlocksSparse"} value: %d`, v)
}
}
func TestStorageDoesNotUseSparseCacheForRegularMerge(t *testing.T) {
tc := apptest.NewTestCase(t)
defer tc.Stop()
sut := tc.MustStartVmsingle("sparse-cache-regular-merge", []string{`-retentionPeriod=100y`, `-downsampling.period={__name__=~"metric.*"}:5m:1s`})
// insert metrics into current month only
const metricsPerStep = 10
const steps = 2
const stepSize = 60 * 1000
records := make([]string, metricsPerStep)
ts := time.Now().Add(-1 * time.Hour).UnixMilli()
for range steps {
for i := range metricsPerStep {
name := fmt.Sprintf("metric_%d", i)
records[i] = fmt.Sprintf("%s %d %d", name, rand.IntN(1000), ts)
}
sut.PrometheusAPIV1ImportPrometheus(t, records, apptest.QueryOpts{})
ts += stepSize
}
sut.ForceFlush(t)
sut.ForceMerge(t)
// todo: replace with a more reliable way to check if the merge is completed
// wait for merge to be completed
time.Sleep(5 * time.Second)
v := sut.GetIntMetric(t, `vm_cache_requests_total{type="indexdb/dataBlocksSparse"}`)
if v == 0 {
t.Fatalf(`unexpected vm_cache_requests_total{type="indexdb/dataBlocksSparse"} value: %d`, v)
}
}

View File

@@ -24,6 +24,7 @@ type Vmsingle struct {
// vmstorage URLs.
forceFlushURL string
forceMergeURL string
// vminsert URLs.
influxLineWriteURL string
@@ -65,6 +66,7 @@ func StartVmsingle(instance string, flags []string, cli *Client) (*Vmsingle, err
httpListenAddr: stderrExtracts[1],
forceFlushURL: fmt.Sprintf("http://%s/internal/force_flush", stderrExtracts[1]),
forceMergeURL: fmt.Sprintf("http://%s/internal/force_merge", stderrExtracts[1]),
influxLineWriteURL: fmt.Sprintf("http://%s/influx/write", stderrExtracts[1]),
prometheusAPIV1ImportPrometheusURL: fmt.Sprintf("http://%s/prometheus/api/v1/import/prometheus", stderrExtracts[1]),
prometheusAPIV1WriteURL: fmt.Sprintf("http://%s/prometheus/api/v1/write", stderrExtracts[1]),
@@ -83,6 +85,13 @@ func (app *Vmsingle) ForceFlush(t *testing.T) {
app.cli.Get(t, app.forceFlushURL, http.StatusOK)
}
// ForceMerge is a test helper function that forces the merging of parts.
func (app *Vmsingle) ForceMerge(t *testing.T) {
t.Helper()
app.cli.Get(t, app.forceMergeURL, http.StatusOK)
}
// InfluxWrite is a test helper function that inserts a
// collection of records in Influx line format by sending a HTTP
// POST request to /influx/write vmsingle endpoint.

View File

@@ -21,6 +21,7 @@ type Vmstorage struct {
vmselectAddr string
forceFlushURL string
forceMergeURL string
}
// StartVmstorage starts an instance of vmstorage with the given flags. It also
@@ -57,6 +58,7 @@ func StartVmstorage(instance string, flags []string, cli *Client) (*Vmstorage, e
vmselectAddr: stderrExtracts[3],
forceFlushURL: fmt.Sprintf("http://%s/internal/force_flush", stderrExtracts[1]),
forceMergeURL: fmt.Sprintf("http://%s/internal/force_merge", stderrExtracts[1]),
}, nil
}
@@ -80,6 +82,12 @@ func (app *Vmstorage) ForceFlush(t *testing.T) {
app.cli.Get(t, app.forceFlushURL, http.StatusOK)
}
// ForceMerge is a test helper function that forces the merging of parts.
func (app *Vmstorage) ForceMerge(t *testing.T) {
t.Helper()
app.cli.Get(t, app.forceMergeURL, http.StatusOK)
}
// String returns the string representation of the vmstorage app state.
func (app *Vmstorage) String() string {
return fmt.Sprintf("{app: %s storageDataPath: %q httpListenAddr: %q vminsertAddr: %q vmselectAddr: %q}", []any{

View File

@@ -1686,7 +1686,7 @@
},
"editorMode": "code",
"exemplar": false,
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n vm_available_cpu_cores{job=~\"$job\", instance=~\"$instance\"}\n) by(instance)",
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n process_cpu_cores_available{job=~\"$job\", instance=~\"$instance\"}\n) by(instance)",
"format": "time_series",
"interval": "",
"intervalFactor": 1,

View File

@@ -1933,7 +1933,7 @@
},
"editorMode": "code",
"exemplar": true,
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n vm_available_cpu_cores{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n process_cpu_cores_available{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"format": "time_series",
"interval": "",
"intervalFactor": 1,

View File

@@ -1981,7 +1981,7 @@
},
"editorMode": "code",
"exemplar": false,
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n vm_available_cpu_cores{job=~\"$job\", instance=~\"$instance\"}\n) by(instance)",
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n process_cpu_cores_available{job=~\"$job\", instance=~\"$instance\"}\n) by(instance)",
"format": "time_series",
"interval": "",
"intervalFactor": 1,

View File

@@ -1687,7 +1687,7 @@
},
"editorMode": "code",
"exemplar": false,
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n vm_available_cpu_cores{job=~\"$job\", instance=~\"$instance\"}\n) by(instance)",
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n process_cpu_cores_available{job=~\"$job\", instance=~\"$instance\"}\n) by(instance)",
"format": "time_series",
"interval": "",
"intervalFactor": 1,

View File

@@ -1934,7 +1934,7 @@
},
"editorMode": "code",
"exemplar": true,
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n vm_available_cpu_cores{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n process_cpu_cores_available{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"format": "time_series",
"interval": "",
"intervalFactor": 1,

View File

@@ -1982,7 +1982,7 @@
},
"editorMode": "code",
"exemplar": false,
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n vm_available_cpu_cores{job=~\"$job\", instance=~\"$instance\"}\n) by(instance)",
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n process_cpu_cores_available{job=~\"$job\", instance=~\"$instance\"}\n) by(instance)",
"format": "time_series",
"interval": "",
"intervalFactor": 1,

View File

@@ -1666,7 +1666,7 @@
},
"editorMode": "code",
"exemplar": true,
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n vm_available_cpu_cores{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n process_cpu_cores_available{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"format": "time_series",
"interval": "",
"intervalFactor": 1,

View File

@@ -1473,7 +1473,7 @@
},
"editorMode": "code",
"exemplar": false,
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval]) \n / \n vm_available_cpu_cores{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval]) \n / \n process_cpu_cores_available{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"format": "time_series",
"interval": "",
"intervalFactor": 1,

View File

@@ -1361,7 +1361,7 @@
},
"editorMode": "code",
"exemplar": false,
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n vm_available_cpu_cores{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n process_cpu_cores_available{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"format": "time_series",
"interval": "",
"intervalFactor": 1,

View File

@@ -1665,7 +1665,7 @@
},
"editorMode": "code",
"exemplar": true,
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n vm_available_cpu_cores{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n process_cpu_cores_available{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"format": "time_series",
"interval": "",
"intervalFactor": 1,

View File

@@ -1472,7 +1472,7 @@
},
"editorMode": "code",
"exemplar": false,
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval]) \n / \n vm_available_cpu_cores{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval]) \n / \n process_cpu_cores_available{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"format": "time_series",
"interval": "",
"intervalFactor": 1,

View File

@@ -1360,7 +1360,7 @@
},
"editorMode": "code",
"exemplar": false,
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n vm_available_cpu_cores{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"expr": "max(\n rate(process_cpu_seconds_total{job=~\"$job\", instance=~\"$instance\"}[$__rate_interval])\n /\n process_cpu_cores_available{job=~\"$job\", instance=~\"$instance\"}\n) by(job)",
"format": "time_series",
"interval": "",
"intervalFactor": 1,

View File

@@ -72,7 +72,7 @@ services:
restart: always
vmanomaly:
container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.19.1
image: victoriametrics/vmanomaly:v1.19.2
depends_on:
- "victoriametrics"
ports:

View File

@@ -86,6 +86,7 @@ See also [case studies](https://docs.victoriametrics.com/casestudies/).
* [Persistent Data Structures in VictoriaMetrics (Part 2): vmselect](https://medium.com/@jiekun/persistent-data-structures-in-victoriametrics-part-2-vmselect-9e3de39a4d20)
* [Migrating to VictoriaMetrics (by Zomato): A Complete Overhaul for Enhanced Observability](https://blog.zomato.com/migrating-to-victoriametrics-a-complete-overhaul-for-enhanced-observability)
* [Harness the Power of VictoriaMetrics and Grafana Operators for Metrics Management](https://blog.ogenki.io/post/series/observability/metrics/)
* [Reducing Inter-AZ traffic in VictoriaMetrics with Zonekeeper](https://tanmay-bhat.medium.com/reducing-inter-az-traffic-in-victoriametrics-with-zonekeeper-3bd7e1526796)
## Our articles

View File

@@ -48,6 +48,21 @@ VictoriaMetrics can run also on MacOS for testing and development purposes.
* **MacOS**: amd64, arm64 (for testing and development purposes)
* **Windows**: amd64
## Kubernetes
VictoriaMetrics natively supports deployment in Kubernetes via [helm charts](https://docs.victoriametrics.com/helm/)
and [kubernetes operator](https://docs.victoriametrics.com/operator/). See how to [start using k8s operator](https://docs.victoriametrics.com/guides/getting-started-with-vm-operator/).
Common recommendations:
1. Prefer setting [requests equal to limits](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits)
for stateful components like [vmstorage](https://docs.victoriametrics.com/cluster-victoriametrics/#architecture-overview) to avoid unnecessary
component restarts.
1. Avoid using [fractional CPU units](https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-resource/#cpu-units)
when setting resources for optimal performance. VictoriaMetrics is written in Go and its runtime requires specifying
[integer number](https://pkg.go.dev/runtime#GOMAXPROCS) of concurrently running threads.
When fractional CPU unit is specified, VictoriaMetrics will automatically round it down.
## Upgrade procedure
It is safe to upgrade VictoriaMetrics to new versions unless the [release notes](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest) say otherwise.

View File

@@ -19,6 +19,8 @@ according to [these docs](https://docs.victoriametrics.com/victorialogs/quicksta
* FEATURE: [`block_stats` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#block_stats-pipe): return the path to the part where every data block is stored. The path to the part is returned in the `part_path` field. This allows investigating the distribution of data blocks among parts.
* FEATURE: reduce VictoriaLogs startup time by multiple times when it opens a large datastore with big [retention](https://docs.victoriametrics.com/victorialogs/#retention).
* BUGFIX: [data ingestion](https://docs.victoriametrics.com/victorialogs/data-ingestion/): drop log entries with too long field names and log the dropped log entries with the `ignoring log entry with too long field name` message, so human operators could notice and fix the ingestion of incorrect logs ASAP. Previously too long field names were silently truncated to shorter values. This isn't what most users expect. See [why VictoriaLogs has a limit on the field name length](https://docs.victoriametrics.com/victorialogs/faq/#what-is-the-maximum-supported-field-name-length).
## [v1.8.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.8.0-victorialogs)
Released at 2025-01-24

View File

@@ -184,16 +184,36 @@ VictoriaLogs works optimally with log records of up to `10KB`. It works OK with
log records of up to `100KB`. It works not so optimal with log records exceeding
`100KB`.
The max size of a log record VictoriaLogs can handle is `2MB`. This is
because VictoriaLogs stores log records in blocks and `2MB` is the max size of a
block. Blocks of this size fit the L2 cache of a typical CPU, which gives an
optimal processing performance.
The max size of a log record VictoriaLogs can accept during [data ingestion](https://docs.victoriametrics.com/victorialogs/data-ingestion/)
is `2MB`, because log records are stored in blocks of up to `2MB` size.
Blocks of this size fit the L2 cache of a typical CPU, which gives an
optimal performance during data ingestion and querying.
However, log records whose size is close to `2MB` aren't handled efficiently by
Note that log records with sizes close to `2MB` aren't handled efficiently by
VictoriaLogs because per-block overhead translates to a single log record, and
this overhead is big.
this overhead is big.
The `2MB` limit is hadrcoded and is unlikely to change.
The `2MB` limit is hadrcoded and is unlikely to increase.
The limit can be set to the lower value during [data ingestion](https://docs.victoriametrics.com/victorialogs/data-ingestion/)
via `-insert.maxLineSizeBytes` command-line flag.
## What is the maximum supported field name length
VictoriaLogs limits [log field](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) name length to 128 bytes -
Log entries with longer field names are ignored during [date ingestion](https://docs.victoriametrics.com/victorialogs/data-ingestion/).
The maximum length of a field name is hardcoded and is unikely to increase, since this may increase RAM and CPU usage.
## How many fields a single log entry may contain
A single log entry may contain up to 2000 fields. This fits well the majority of use cases for structured logs and
for [wide events](https://jeremymorrell.dev/blog/a-practitioners-guide-to-wide-events/).
The maximum number of fields per log entry is hardcoded and is unlikely to increase, since this may increase RAM and CPU usage.
The limit can be set to the lower value during [data ingestion](https://docs.victoriametrics.com/victorialogs/data-ingestion/)
via `-insert.maxFieldsPerLine` command-line flag.
## How to determine which log fields occupy the most of disk space?

View File

@@ -249,6 +249,37 @@ VictoriaLogs has very low overhead for per-tenant management, so it is OK to hav
VictoriaLogs doesn't perform per-tenant authorization. Use [vmauth](https://docs.victoriametrics.com/vmauth/) or similar tools for per-tenant authorization.
### Multitenancy access control
Enforce access control for tenants by using [vmauth](https://docs.victoriametrics.com/vmauth/). Access control can be configured for each tenant by setting up the following rules:
```yaml
users:
- username: "foo"
password: "bar"
url_map:
- src_paths:
- "/select/.*"
- "/insert/.*"
headers:
- "AccountID: 1"
- "ProjectID: 0"
url_prefix:
- "http://localhost:9428/"
- username: "baz"
password: "bar"
url_map:
- src_paths: ["/select/.*"]
headers:
- "AccountID: 2"
- "ProjectID: 0"
url_prefix:
- "http://localhost:9428/"
```
This configuration allows `foo` to use the `/select/.*` and `/insert/.*` endpoints with `AccountID: 1` and `ProjectID: 0`, while `baz` can only use the `/select/.*` endpoint with `AccountID: 2` and `ProjectID: 0`.
## Security
It is expected that VictoriaLogs runs in a protected environment, which is unreachable from the Internet without proper authorization.

View File

@@ -11,14 +11,29 @@ aliases:
---
Please find the changelog for VictoriaMetrics Anomaly Detection below.
## v1.19.2
Released: 2025-01-27
- IMPROVEMENT: Added the `complete` option to the `--splitBy` argument in `config_splitter.py` [util](https://docs.victoriametrics.com/anomaly-detection/faq/index.html#splitting-the-config). This allows splitting a parent configuration into the smallest possible sub-configurations, each containing exactly one scheduler, one model, and either one or multiple queries (depending on whether the model is [multivariate](https://docs.victoriametrics.com/anomaly-detection/components/models/#multivariate-models) or not).
- FIX: Resolved an issue where duplicate log messages were generated during sub-config validation of the parent configuration.
- FIX: Corrected usage of `AccountID` and `ProjectID` extracted from `tenant_id`, which are appended as labels `vm_account_id` and `vm_project_id`, respectively (previously swapped) by `VmReader` when using the per-query `tenant_id` feature. **This issue affected versions [v1.19.0](#v1190) and [v1.19.1](#v1191).**
- FIX: Resolved an issue with the `VmReader` instance string representation that caused errors when `vmanomaly` was run with `--loggerLevel DEBUG`.
## v1.19.1
Released: 2025-01-21
> **Note**: There is a known bug in [v1.19.0](#v1190) - the `AccountID` and `ProjectID` are swapped when they are extracted from the `tenant_id` argument in `VMReader`. This can cause correctly read results being written to the wrong tenant when using the per-query `tenant_id` feature with `AccountID` != `ProjectID`. Please update to patch [v1.19.2](#v1192), which resolves this issue.
- FIX: Resolved writer warnings for configurations where `reader.tenant_id` equals `writer.tenant_id` and **is not** `multitenant`, as this is a valid setup. Enhanced tenant_id-related log messages across config validation, reader, and writer for improved clarity.
## v1.19.0
Released: 2025-01-20
> **Note**: There is a known bug in [v1.19.0](#v1190) - the `AccountID` and `ProjectID` are swapped when they are extracted from the `tenant_id` argument in `VMReader`. This can cause correctly read results being written to the wrong tenant when using the per-query `tenant_id` feature with `AccountID` != `ProjectID`. Please update to patch [v1.19.2](#v1192), which resolves this issue.
- FEATURE: Added support for per-query `tenant_id` in the [`VmReader`](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader). This allows overriding the reader-level `tenant_id` within a single global `vmanomaly` configuration on a *per-query* basis, enabling isolation of data for different tenants in separate queries when querying the [VictoriaMetrics cluster version](https://docs.victoriametrics.com/cluster-victoriametrics/). For details, see the [documentation](https://docs.victoriametrics.com/anomaly-detection/components/reader/?highlight=tenant_id#per-query-parameters).
- IMPROVEMEMT: Speedup the model infer stage on multicore systems.
- IMPROVEMEMT: Speedup the model fitting stage by 1.253x, depending on configuration complexity.

View File

@@ -158,7 +158,7 @@ services:
# ...
vmanomaly:
container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.19.1
image: victoriametrics/vmanomaly:v1.19.2
# ...
ports:
- "8490:8490"
@@ -337,10 +337,10 @@ For **horizontal** scalability, `vmanomaly` can be deployed as multiple independ
### Splitting the config
CLI utility named `config_splitter` is available in `vmanomaly` {{% available_from "v1.18.5" anomaly %}}. The config splitter tool enables splitting a parent vmanomaly YAML configuration file into multiple sub-configurations based on logical entities such as `schedulers`, `queries`, `models`, `extra_filters`. The resulting sub-configurations are fully validated, functional, account for many-to-many relationships between models and their associated queries, and the schedulers they are linked to. These sub-configurations can then be saved to a specified directory for further use:
CLI utility named `config_splitter` is available in `vmanomaly` {{% available_from "v1.18.5" anomaly %}}. The config splitter tool enables splitting a parent vmanomaly YAML configuration file into multiple sub-configurations based on logical entities such as `schedulers`, `queries`, `models`, `extra_filters` and `complete` {{% available_from "v1.19.2" anomaly %}}. The resulting sub-configurations are fully validated, functional, account for many-to-many relationships between models and their associated queries, and the schedulers they are linked to. These sub-configurations can then be saved to a specified directory for further use:
```shellhelp
usage: config_splitter.py [-h] --splitBy {schedulers,models,queries,extra_filters} --outputDir OUTPUT_DIR [--fileNameFormat {raw,hash,int}] [--loggerLevel {WARNING,INFO,ERROR,FATAL,DEBUG}]
usage: config_splitter.py [-h] --splitBy {schedulers,models,queries,extra_filters,complete} --outputDir OUTPUT_DIR [--fileNameFormat {raw,hash,int}] [--loggerLevel {WARNING,ERROR,FATAL,INFO,DEBUG}]
config [config ...]
Splits the configuration of VictoriaMetrics Anomaly Detection service by a logical entity.
@@ -350,21 +350,22 @@ positional arguments:
options:
-h show this help message and exit
--splitBy {schedulers,models,queries,extra_filters}
The logical entity to split by. Choices: ['schedulers', 'models', 'queries', 'extra_filters'].
--splitBy {schedulers,models,queries,extra_filters,complete}
The logical entity to split by. Choices: ['schedulers', 'models', 'queries', 'extra_filters', 'complete']. `complete` produces configurations based on combinations of
(scheduler, model, queries). Default: complete.
--outputDir output_dir
Directory where the split configuration files will be saved.
--fileNameFormat {raw,hash,int}
The naming format for the output configuration files. Choices: raw (use the entity alias), hash (use hashed alias), int (use a sequential integer from 0 to N for N
produced sub-configs). Default: raw.
--loggerLevel {WARNING,INFO,ERROR,FATAL,DEBUG}
The naming format for the output configuration files. Choices: raw (use the entity alias), hash (use hashed alias), int (use a sequential integer from 0 to N for N produced
sub-configs). Default: raw.
--loggerLevel {WARNING,ERROR,FATAL,INFO,DEBUG}
Minimum level to log. Default: INFO
```
Heres an example of using the config splitter to divide configurations based on the `extra_filters` argument from the reader section:
```sh
docker pull victoriametrics/vmanomaly:v1.19.1 && docker image tag victoriametrics/vmanomaly:v1.19.1 vmanomaly
docker pull victoriametrics/vmanomaly:v1.19.2 && docker image tag victoriametrics/vmanomaly:v1.19.2 vmanomaly
```
```sh

View File

@@ -101,13 +101,13 @@ Below are the steps to get `vmanomaly` up and running inside a Docker container:
1. Pull Docker image:
```sh
docker pull victoriametrics/vmanomaly:v1.19.1
docker pull victoriametrics/vmanomaly:v1.19.2
```
2. (Optional step) tag the `vmanomaly` Docker image:
```sh
docker image tag victoriametrics/vmanomaly:v1.19.1 vmanomaly
docker image tag victoriametrics/vmanomaly:v1.19.2 vmanomaly
```
3. Start the `vmanomaly` Docker container with a *license file*, use the command below.
@@ -141,7 +141,7 @@ docker run -it --user 1000:1000 \
services:
# ...
vmanomaly:
image: victoriametrics/vmanomaly:v1.19.1
image: victoriametrics/vmanomaly:v1.19.2
volumes:
$YOUR_LICENSE_FILE_PATH:/license
$YOUR_CONFIG_FILE_PATH:/config.yml

View File

@@ -994,7 +994,7 @@ monitoring:
Let's pull the docker image for `vmanomaly`:
```sh
docker pull victoriametrics/vmanomaly:v1.19.1
docker pull victoriametrics/vmanomaly:v1.19.2
```
Now we can run the docker container putting as volumes both config and model file:
@@ -1008,7 +1008,7 @@ docker run -it \
-v $(PWD)/license:/license \
-v $(PWD)/custom_model.py:/vmanomaly/model/custom.py \
-v $(PWD)/custom.yaml:/config.yaml \
victoriametrics/vmanomaly:v1.19.1 /config.yaml \
victoriametrics/vmanomaly:v1.19.2 /config.yaml \
--licenseFile=/license
```

View File

@@ -387,7 +387,7 @@ services:
restart: always
vmanomaly:
container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.19.1
image: victoriametrics/vmanomaly:v1.19.2
depends_on:
- "victoriametrics"
ports:

View File

@@ -18,6 +18,30 @@ See also [LTS releases](https://docs.victoriametrics.com/lts-releases/).
## tip
* FEATURE: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and [vmstorage](https://docs.victoriametrics.com/cluster-victoriametrics/): improve startup times when opening a storage with the [retention](https://docs.victoriametrics.com/#retention) exceeding a few months.
## [v1.102.12](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.102.12)
Released at 2025-01-28
**v1.102.x is a line of [LTS releases](https://docs.victoriametrics.com/lts-releases/). It contains important up-to-date bugfixes for [VictoriaMetrics enterprise](https://docs.victoriametrics.com/enterprise.html).
All these fixes are also included in [the latest community release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest).
The v1.102.x line will be supported for at least 12 months since [v1.102.0](https://docs.victoriametrics.com/changelog/#v11020) release**
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/single-server-victoriametrics/), `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/) and [vmagent](https://docs.victoriametrics.com/vmagent/): log metric names for signals with unsupported delta temporality on ingestion via [OpenTelemetry protocol for metrics](https://docs.victoriametrics.com/#sending-data-via-opentelemetry). Thanks to @chenlujjj for [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8018).
* BUGFIX: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and [vmselect](https://docs.victoriametrics.com/cluster-victoriametrics/): respect staleness detection in increase, increase_pure and delta functions when time series has gaps and `-search.maxStalenessInterval` is set. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8072) for details.
## [v1.97.17](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.97.17)
Released at 2025-01-28
**v1.97.x is a line of [LTS releases](https://docs.victoriametrics.com/lts-releases/). It contains important up-to-date bugfixes for [VictoriaMetrics enterprise](https://docs.victoriametrics.com/enterprise.html).
All these fixes are also included in [the latest community release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest).
The v1.97.x line will be supported for at least 12 months since [v1.97.0](https://docs.victoriametrics.com/CHANGELOG.html#v1970) release**
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/single-server-victoriametrics/), `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/) and [vmagent](https://docs.victoriametrics.com/vmagent/): log metric names for signals with unsupported delta temporality on ingestion via [OpenTelemetry protocol for metrics](https://docs.victoriametrics.com/#sending-data-via-opentelemetry). Thanks to @chenlujjj for [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8018).
* BUGFIX: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and [vmselect](https://docs.victoriametrics.com/cluster-victoriametrics/): respect staleness detection in increase, increase_pure and delta functions when time series has gaps and `-search.maxStalenessInterval` is set. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8072) for details.
## [v1.110.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.110.0)
Released at 2025-01-24
@@ -29,12 +53,12 @@ Released at 2025-01-24
* FEATURE: [dashboards](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/dashboards): all dashboards that use [VictoriaMetrics Grafana datasource](https://github.com/VictoriaMetrics/victoriametrics-datasource) were updated to use a [new datasource ID](https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/tag/v0.12.0).
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): reflect column settings for the table view in URL, so the table view can be shared via link. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7662).
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/single-server-victoriametrics/), `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/) and [vmagent](https://docs.victoriametrics.com/vmagent/): allow ingesting histograms with missing `_sum` metric via [OpenTelemetry ingestion protocol](https://docs.victoriametrics.com/#sending-data-via-opentelemetry) in the same way as Prometheus does.
* BUGFIX: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and [vmselect](https://docs.victoriametrics.com/cluster-victoriametrics/): respect staleness detection in increase, increase_pure and delta functions when time series has gaps and `-search.maxStalenessInterval` is set. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8072) for details.
* BUGFIX: all VictoriaMetrics [enterprise](https://docs.victoriametrics.com/enterprise/) components: properly trim whitespaces at the end of license provided via `-license` and `-licenseFile` command-line flags. Previously, the trailing whitespaces could cause the license verification to fail.
* BUGFIX: [vmauth](https://docs.victoriametrics.com/vmauth/): fix possible runtime panic during requests processing under heavy load. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8051) for details.
* BUGFIX: [vmselect](https://docs.victoriametrics.com/cluster-victoriametrics/): fix panic when trying to delete series by using [multitenant read](https://docs.victoriametrics.com/cluster-victoriametrics/#multitenancy-via-labels) endpoint. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8126) for the details.
* BUGFIX: [vmselect](https://docs.victoriametrics.com/cluster-victoriametrics/): prevent panic when `vmselect` receives an error response from `vmstorage` during the query execution and request processing for other `vmstorage` nodes is still in progress. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8114) for the details.
* BUGFIX: all VictoriaMetrics [enterprise](https://docs.victoriametrics.com/enterprise/) components: properly trim whitespaces at the end of license provided via `-license` and `-licenseFile` command-line flags. Previously, the trailing whitespaces could cause the license verification to fail.
* BUGFIX: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and [vmselect](https://docs.victoriametrics.com/cluster-victoriametrics/): respect staleness detection in increase, increase_pure and delta functions when time series has gaps and `-search.maxStalenessInterval` is set. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8072) for details.
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/single-server-victoriametrics/), `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/) and [vmagent](https://docs.victoriametrics.com/vmagent/): allow ingesting histograms with missing `_sum` metric via [OpenTelemetry ingestion protocol](https://docs.victoriametrics.com/#sending-data-via-opentelemetry) in the same way as Prometheus does.
* BUGFIX: [vmui](https://docs.victoriametrics.com/#vmui): fix an issue where pressing the "Enter" key in the query editor did not execute the query. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8058).
## [v1.102.11](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.102.11)
@@ -47,11 +71,11 @@ The v1.102.x line will be supported for at least 12 months since [v1.102.0](http
* SECURITY: upgrade Go builder from Go1.23.4 to Go1.23.5. See the list of issues addressed in [Go1.23.5](https://github.com/golang/go/issues?q=milestone%3AGo1.23.5+label%3ACherryPickApproved).
* BUGFIX: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and [vmselect](https://docs.victoriametrics.com/cluster-victoriametrics/): don't take into account the last raw sample before the lookbehind window is sample exceeds the staleness interval. This affects correctness of increase, increase_pure, delta functions when preforming calculations on time series with gaps. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8002) for details.
* BUGFIX: all VictoriaMetrics [enterprise](https://docs.victoriametrics.com/enterprise/) components: remove unnecessary delay before failing if all online verification attempts have failed. This should reduce the time required for the component to proceed if all online verification attempts have failed.
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/single-server-victoriametrics/), `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/) and [vmagent](https://docs.victoriametrics.com/vmagent/): allow ingesting histograms with missing `_sum` metric via [OpenTelemetry ingestion protocol](https://docs.victoriametrics.com/#sending-data-via-opentelemetry) in the same way as Prometheus does.
* BUGFIX: all VictoriaMetrics [enterprise](https://docs.victoriametrics.com/enterprise/) components: properly trim whitespaces at the end of license provided via `-license` and `-licenseFile` command-line flags. Previously, the trailing whitespaces could cause the license verification to fail.
* BUGFIX: [vmauth](https://docs.victoriametrics.com/vmauth/): fix possible runtime panic during requests processing under heavy load. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8051) for details.
* BUGFIX: all VictoriaMetrics [enterprise](https://docs.victoriametrics.com/enterprise/) components: properly trim whitespaces at the end of license provided via `-license` and `-licenseFile` command-line flags. Previously, the trailing whitespaces could cause the license verification to fail.
* BUGFIX: all VictoriaMetrics [enterprise](https://docs.victoriametrics.com/enterprise/) components: remove unnecessary delay before failing if all online verification attempts have failed. This should reduce the time required for the component to proceed if all online verification attempts have failed.
* BUGFIX: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and [vmselect](https://docs.victoriametrics.com/cluster-victoriametrics/): don't take into account the last raw sample before the lookbehind window is sample exceeds the staleness interval. This affects correctness of increase, increase_pure, delta functions when preforming calculations on time series with gaps. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8002) for details.
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/single-server-victoriametrics/), `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/) and [vmagent](https://docs.victoriametrics.com/vmagent/): allow ingesting histograms with missing `_sum` metric via [OpenTelemetry ingestion protocol](https://docs.victoriametrics.com/#sending-data-via-opentelemetry) in the same way as Prometheus does.
## [v1.97.16](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.97.16)
@@ -61,9 +85,9 @@ Released at 2025-01-24
All these fixes are also included in [the latest community release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest).
The v1.97.x line will be supported for at least 12 months since [v1.97.0](https://docs.victoriametrics.com/CHANGELOG.html#v1970) release**
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/single-server-victoriametrics/), `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/) and [vmagent](https://docs.victoriametrics.com/vmagent/): allow ingesting histograms with missing `_sum` metric via [OpenTelemetry ingestion protocol](https://docs.victoriametrics.com/#sending-data-via-opentelemetry) in the same way as Prometheus does.
* BUGFIX: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and [vmselect](https://docs.victoriametrics.com/cluster-victoriametrics/): don't take into account the last raw sample before the lookbehind window is sample exceeds the staleness interval. This affects correctness of increase, increase_pure, delta functions when preforming calculations on time series with gaps. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8002) for details.
* BUGFIX: all VictoriaMetrics [enterprise](https://docs.victoriametrics.com/enterprise/) components: remove unnecessary delay before failing if all online verification attempts have failed. This should reduce the time required for the component to proceed if all online verification attempts have failed.
* BUGFIX: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and [vmselect](https://docs.victoriametrics.com/cluster-victoriametrics/): don't take into account the last raw sample before the lookbehind window is sample exceeds the staleness interval. This affects correctness of increase, increase_pure, delta functions when preforming calculations on time series with gaps. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8002) for details.
* BUGFIX: [vmsingle](https://docs.victoriametrics.com/single-server-victoriametrics/), `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/) and [vmagent](https://docs.victoriametrics.com/vmagent/): allow ingesting histograms with missing `_sum` metric via [OpenTelemetry ingestion protocol](https://docs.victoriametrics.com/#sending-data-via-opentelemetry) in the same way as Prometheus does.
## [v1.109.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.109.1)

View File

@@ -24,10 +24,10 @@ For a detailed explanation of each parameter, visit the guide on [Understanding
## Flag Parameters Configuration
| **Flag** | **Default Value** | **Description** |
|-----------------------------------|---------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Max Label Value Length** | `<= 1kb` (Default: `4kb`) | Maximum length of label values. Longer values are truncated. Large label values can lead to high RAM consumption. This can be adjusted via [support](mailto:support-cloud@victoriametrics.com). |
| **Max Labels per Time Series** | `<= 30` | Maximum number of labels per time series. Excess labels are dropped. Higher values can increase [cardinality](https://docs.victoriametrics.com/keyconcepts/#cardinality) and resource usage. This can be configured in [deployment settings](https://docs.victoriametrics.com/victoriametrics-cloud/quickstart/#modifying-an-existing-deployment). |
| **Flag** | **Default Value** | **Description** |
|-----------------------------------|---------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Max Label Value Length** | `<= 1kb` (Default: `4kb`) | Maximum length of label values. Time series with longer values are dropped. Large label values can lead to high RAM consumption. This can be adjusted via [support](mailto:support-cloud@victoriametrics.com). |
| **Max Labels per Time Series** | `<= 30` | Maximum number of labels per time series. Time series with excess labels are dropped. Higher values can increase [cardinality](https://docs.victoriametrics.com/keyconcepts/#cardinality) and resource usage. This can be configured in [deployment settings](https://docs.victoriametrics.com/victoriametrics-cloud/quickstart/#modifying-an-existing-deployment). |
## Terms and definitions:

View File

@@ -46,7 +46,7 @@ Installing VictoriaMetrics Grafana datasource [requires](https://grafana.com/doc
``` ini
[plugins]
allow_loading_unsigned_plugins = victoriametrics-datasource
allow_loading_unsigned_plugins = victoriametrics-metrics-datasource
```
For `grafana-operator` users, please adjust `config:` section in your `kind=Grafana` resource as below
@@ -54,7 +54,7 @@ For `grafana-operator` users, please adjust `config:` section in your `kind=Graf
```
config:
plugins:
allow_loading_unsigned_plugins: "victoriametrics-datasource"
allow_loading_unsigned_plugins: "victoriametrics-metrics-datasource"
```
See [why VictoriaMetrics datasource is unsigned](#why-victoriaMetrics-datasource-is-unsigned).
@@ -83,7 +83,7 @@ datasources:
# displayed in Grafana panels and queries.
- name: VictoriaMetrics
# <string, required> Sets the data source type.
type: victoriametrics-datasource
type: victoriametrics-metrics-datasource
# <string, required> Sets the access mode, either
# proxy or direct (Server or Browser in the UI).
# Some data sources are incompatible with any setting
@@ -99,7 +99,7 @@ datasources:
# displayed in Grafana panels and queries.
- name: VictoriaMetrics - cluster
# <string, required> Sets the data source type.
type: victoriametrics-datasource
type: victoriametrics-metrics-datasource
# <string, required> Sets the access mode, either
# proxy or direct (Server or Browser in the UI).
# Some data sources are incompatible with any setting
@@ -124,8 +124,8 @@ Please find the example of provisioning Grafana instance with VictoriaMetrics da
grafana:
image: grafana/grafana:11.0.0
environment:
- GF_INSTALL_PLUGINS=https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/download/v0.10.3/victoriametrics-datasource-v0.10.3.zip;victoriametrics-datasource
- GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS=victoriametrics-datasource
- GF_INSTALL_PLUGINS=https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/download/v0.13.0/victoriametrics-metrics-datasource-v0.13.0.zip;victoriametrics-metrics-datasource
- GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS=victoriametrics-metrics-datasource
ports:
- 3000:3000/tcp
volumes:
@@ -152,14 +152,14 @@ Option 1. Using Grafana provisioning:
``` yaml
env:
GF_INSTALL_PLUGINS: "https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/download/v0.10.3/victoriametrics-datasource-v0.10.3.zip;victoriametrics-datasource"
GF_INSTALL_PLUGINS: "https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/download/v0.13.0/victoriametrics-metrics-datasource-v0.13.0.zip;victoriametrics-metrics-datasource"
```
Option 2. Using Grafana plugins section in `values.yaml`:
``` yaml
plugins:
- https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/download/v0.10.3/victoriametrics-datasource-v0.10.3.zip;victoriametrics-datasource
- https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/download/v0.13.0/victoriametrics-metrics-datasource-v0.13.0.zip;victoriametrics-metrics-datasource
```
Option 3. Using init container:
@@ -179,7 +179,7 @@ extraInitContainers:
set -ex
mkdir -p /var/lib/grafana/plugins/
ver=$(curl -s -L https://api.github.com/repos/VictoriaMetrics/victoriametrics-datasource/releases/latest | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+' | head -1)
curl -L https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/download/$ver/victoriametrics-datasource-$ver.tar.gz -o /var/lib/grafana/plugins/vm-plugin.tar.gz
curl -L https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/download/$ver/victoriametrics-metrics-datasource-$ver.tar.gz -o /var/lib/grafana/plugins/vm-plugin.tar.gz
tar -xf /var/lib/grafana/plugins/vm-plugin.tar.gz -C /var/lib/grafana/plugins/
rm /var/lib/grafana/plugins/vm-plugin.tar.gz
volumeMounts:
@@ -239,7 +239,7 @@ spec:
set -ex
mkdir -p /var/lib/grafana/plugins/
ver=$(curl -s https://api.github.com/repos/VictoriaMetrics/victoriametrics-datasource/releases/latest | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+' | head -1)
curl -L https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/download/$ver/victoriametrics-datasource-$ver.tar.gz -o /var/lib/grafana/plugins/vm-plugin.tar.gz
curl -L https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/download/$ver/victoriametrics-metrics-datasource-$ver.tar.gz -o /var/lib/grafana/plugins/vm-plugin.tar.gz
tar -xf /var/lib/grafana/plugins/vm-plugin.tar.gz -C /var/lib/grafana/plugins/
rm /var/lib/grafana/plugins/vm-plugin.tar.gz
volumeMounts:
@@ -247,7 +247,7 @@ spec:
mountPath: /var/lib/grafana
config:
plugins:
allow_loading_unsigned_plugins: victoriametrics-datasource
allow_loading_unsigned_plugins: victoriametrics-metrics-datasource
```
See [Grafana operator reference](https://grafana-operator.github.io/grafana-operator/docs/grafana/) to find more about Grafana operator.
@@ -259,7 +259,7 @@ This example uses init container to download and install plugin.
```sh
ver=$(curl -s https://api.github.com/repos/VictoriaMetrics/victoriametrics-datasource/releases/latest | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+' | head -1)
curl -L https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/download/$ver/victoriametrics-datasource-$ver.tar.gz -o /var/lib/grafana/plugins/vm-plugin.tar.gz
curl -L https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/download/$ver/victoriametrics-metrics-datasource-$ver.tar.gz -o /var/lib/grafana/plugins/vm-plugin.tar.gz
tar -xf /var/lib/grafana/plugins/vm-plugin.tar.gz -C /var/lib/grafana/plugins/
rm /var/lib/grafana/plugins/vm-plugin.tar.gz
```
@@ -279,7 +279,7 @@ plugins = {{path to directory with plugin}}
``` ini
[plugins]
allow_loading_unsigned_plugins = victoriametrics-datasource
allow_loading_unsigned_plugins = victoriametrics-metrics-datasource
```
### 2. Run the plugin

View File

@@ -661,6 +661,19 @@ unauthorized_user:
headers:
- "TenantID: foobar"
- "X-Forwarded-For:"
users:
- username: "foo"
password: "bar"
dump_request_on_errors: true
url_map:
- src_paths: ["/select/.*"]
headers:
- "AccountID: 1"
- "ProjectID: 0"
url_prefix:
- "http://backend:9428/"
```
`vmauth` also supports the ability to set and remove HTTP response headers before returning the response from the backend to client.

87
go.mod
View File

@@ -1,9 +1,6 @@
module github.com/VictoriaMetrics/VictoriaMetrics
go 1.23.3
// See // See https://github.com/googleapis/google-cloud-go/issues/11283#issuecomment-2558515586
exclude google.golang.org/grpc/stats/opentelemetry v0.0.0-20240907200651-3ffb98b2c93a
go 1.23.5
// This is needed in order to avoid vmbackup and vmrestore binary size increase by 20MB
// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8008
@@ -13,19 +10,19 @@ replace cloud.google.com/go/storage => cloud.google.com/go/storage v1.43.0
require (
cloud.google.com/go/storage v1.50.0
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.17.0
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.0
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.5.0
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.1
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.0
github.com/VictoriaMetrics/easyproto v0.1.4
github.com/VictoriaMetrics/fastcache v1.12.2
github.com/VictoriaMetrics/metrics v1.35.1
github.com/VictoriaMetrics/metricsql v0.82.0
github.com/aws/aws-sdk-go-v2 v1.32.8
github.com/aws/aws-sdk-go-v2/config v1.28.10
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.48
github.com/aws/aws-sdk-go-v2/service/s3 v1.72.2
github.com/bmatcuk/doublestar/v4 v4.7.1
github.com/aws/aws-sdk-go-v2 v1.34.0
github.com/aws/aws-sdk-go-v2/config v1.29.2
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.54
github.com/aws/aws-sdk-go-v2/service/s3 v1.74.1
github.com/bmatcuk/doublestar/v4 v4.8.1
github.com/cespare/xxhash/v2 v2.3.0
github.com/cheggaaa/pb/v3 v3.1.5
github.com/cheggaaa/pb/v3 v3.1.6
github.com/ergochat/readline v0.1.3
github.com/gogo/protobuf v1.3.2
github.com/golang/snappy v0.0.4
@@ -45,7 +42,7 @@ require (
golang.org/x/net v0.34.0
golang.org/x/oauth2 v0.25.0
golang.org/x/sys v0.29.0
google.golang.org/api v0.216.0
google.golang.org/api v0.218.0
gopkg.in/yaml.v2 v2.4.0
)
@@ -59,22 +56,22 @@ require (
github.com/AzureAD/microsoft-authentication-library-for-go v1.3.2 // indirect
github.com/VividCortex/ewma v1.2.0 // indirect
github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b // indirect
github.com/aws/aws-sdk-go v1.55.5 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.7 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.17.51 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.23 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.27 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.27 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.27 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.1 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.4.8 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.8 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.8 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.24.9 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.8 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.33.6 // indirect
github.com/aws/smithy-go v1.22.1 // indirect
github.com/aws/aws-sdk-go v1.55.6 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.8 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.17.55 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.25 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.29 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.29 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.2 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.29 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.2 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.5.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.10 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.10 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.24.12 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.11 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.33.10 // indirect
github.com/aws/smithy-go v1.22.2 // indirect
github.com/bboreham/go-loser v0.0.0-20230920113527-fcc2c21820a3 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cncf/xds/go v0.0.0-20241223141626-cff3c89139a3 // indirect
@@ -107,7 +104,7 @@ require (
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/client_golang v1.20.5 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.61.0 // indirect
github.com/prometheus/common v0.62.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/prometheus/sigv4 v0.1.1 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
@@ -116,15 +113,15 @@ require (
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/collector/pdata v1.23.0 // indirect
go.opentelemetry.io/collector/semconv v0.117.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.58.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.58.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.58.0 // indirect
go.opentelemetry.io/otel v1.33.0 // indirect
go.opentelemetry.io/otel/metric v1.33.0 // indirect
go.opentelemetry.io/collector/pdata v1.24.0 // indirect
go.opentelemetry.io/collector/semconv v0.118.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.59.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.59.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.59.0 // indirect
go.opentelemetry.io/otel v1.34.0 // indirect
go.opentelemetry.io/otel/metric v1.34.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.33.0 // indirect
go.opentelemetry.io/otel/trace v1.33.0 // indirect
go.opentelemetry.io/otel/trace v1.34.0 // indirect
go.uber.org/atomic v1.11.0 // indirect
go.uber.org/goleak v1.3.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
@@ -133,14 +130,14 @@ require (
golang.org/x/sync v0.10.0 // indirect
golang.org/x/text v0.21.0 // indirect
golang.org/x/time v0.9.0 // indirect
google.golang.org/genproto v0.0.0-20250106144421-5f5ef82da422 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250106144421-5f5ef82da422 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250106144421-5f5ef82da422 // indirect
google.golang.org/grpc v1.69.2 // indirect
google.golang.org/protobuf v1.36.2 // indirect
google.golang.org/genproto v0.0.0-20250127172529-29210b9bc287 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250127172529-29210b9bc287 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250127172529-29210b9bc287 // indirect
google.golang.org/grpc v1.70.0 // indirect
google.golang.org/protobuf v1.36.4 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/apimachinery v0.32.0 // indirect
k8s.io/client-go v0.32.0 // indirect
k8s.io/apimachinery v0.32.1 // indirect
k8s.io/client-go v0.32.1 // indirect
k8s.io/klog/v2 v2.130.1 // indirect
k8s.io/utils v0.0.0-20241210054802-24370beab758 // indirect
)

176
go.sum
View File

@@ -14,10 +14,10 @@ cloud.google.com/go/storage v1.43.0 h1:CcxnSohZwizt4LCzQHWvBf1/kvtHUn7gk9QERXPyX
cloud.google.com/go/storage v1.43.0/go.mod h1:ajvxEa7WmZS1PxvKRq4bq0tFT3vMd502JwstCcYv0Q0=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.17.0 h1:g0EZJwz7xkXQiZAI5xi9f3WWFYBlX1CPTrR+NDToRkQ=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.17.0/go.mod h1:XCW7KnZet0Opnr7HccfUw1PLc4CjHqpcaxW8DHklNkQ=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.0 h1:B/dfvscEQtew9dVuoxqxrUKKv8Ih2f55PydknDamU+g=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.0/go.mod h1:fiPSssYvltE08HJchL04dOy+RD4hgrjph0cwGGMntdI=
github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.0 h1:+m0M/LFxN43KvULkDNfdXOgrjtg6UYJPFBJyuEcRCAw=
github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.0/go.mod h1:PwOyop78lveYMRs6oCxjiVyBdyCgIYH6XHIVZO9/SFQ=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.1 h1:1mvYtZfWQAnwNah/C+Z+Jb9rQH95LPE2vlmMuWAHJk8=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.1/go.mod h1:75I/mXtme1JyWFtz8GocPHVFyH421IBoZErnO16dd0k=
github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.1 h1:Bk5uOhSAenHyR5P61D/NzeQCv+4fEVV8mOkJ82NqpWw=
github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.1/go.mod h1:QZ4pw3or1WPmRBxf0cHd1tknzrT54WPBOQoGutCPvSU=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 h1:ywEEhmNahHBihViHepv3xPBn1663uRv2t2q/ESv9seY=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0/go.mod h1:iZDifYGJTIgIIkYRNWPENUnqx6bJ2xnSDFI2tjwZNuY=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5 v5.7.0 h1:LkHbJbgF3YyvC53aqYGR+wWQDn2Rdp9AQdGndf9QvY4=
@@ -26,8 +26,8 @@ github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/network/armnetwork/v4 v4.3
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/network/armnetwork/v4 v4.3.0/go.mod h1:Y/HgrePTmGy9HjdSGTqZNa+apUpTVIEVKXJyARP2lrk=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.6.0 h1:PiSrjRPpkQNjrM8H0WwKMnZUdu1RGMtd/LdGKUrOo+c=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.6.0/go.mod h1:oDrbWx4ewMylP7xHivfgixbfGBT6APAwsSoHRKotnIc=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.5.0 h1:mlmW46Q0B79I+Aj4azKC6xDMFN9a9SyZWESlGWYXbFs=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.5.0/go.mod h1:PXe2h+LKcWTX9afWdZoHyODqR4fBa5boUM/8uJfZ0Jo=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.0 h1:UXT0o77lXQrikd1kgwIPQOUect7EoR/+sbP4wQKdzxM=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.0/go.mod h1:cTvi54pg19DoT07ekoeMgE/taAwNtCShVeZqA+Iv2xI=
github.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1 h1:WJTmL004Abzc5wDB5VtZG2PJk5ndYDgVacGqfirKxjM=
github.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1/go.mod h1:tCcJZ0uHAmvjsVYzEFivsRTN00oz5BEsRgQHu5JZ9WE=
github.com/AzureAD/microsoft-authentication-library-for-go v1.3.2 h1:kYRSnvJju5gYVyhkij+RTJ/VR6QIUaCfWeaFm2ycsjQ=
@@ -53,57 +53,57 @@ github.com/allegro/bigcache v1.2.1-0.20190218064605-e24eb225f156 h1:eMwmnE/GDgah
github.com/allegro/bigcache v1.2.1-0.20190218064605-e24eb225f156/go.mod h1:Cb/ax3seSYIx7SuZdm2G2xzfwmv3TPSk2ucNfQESPXM=
github.com/armon/go-metrics v0.4.1 h1:hR91U9KYmb6bLBYLQjyM+3j+rcd/UhE+G78SFnF8gJA=
github.com/armon/go-metrics v0.4.1/go.mod h1:E6amYzXo6aW1tqzoZGT755KkbgrJsSdpwZ+3JqfkOG4=
github.com/aws/aws-sdk-go v1.55.5 h1:KKUZBfBoyqy5d3swXyiC7Q76ic40rYcbqH7qjh59kzU=
github.com/aws/aws-sdk-go v1.55.5/go.mod h1:eRwEWoyTWFMVYVQzKMNHWP5/RV4xIUGMQfXQHfHkpNU=
github.com/aws/aws-sdk-go-v2 v1.32.8 h1:cZV+NUS/eGxKXMtmyhtYPJ7Z4YLoI/V8bkTdRZfYhGo=
github.com/aws/aws-sdk-go-v2 v1.32.8/go.mod h1:P5WJBrYqqbWVaOxgH0X/FYYD47/nooaPOZPlQdmiN2U=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.7 h1:lL7IfaFzngfx0ZwUGOZdsFFnQ5uLvR0hWqqhyE7Q9M8=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.7/go.mod h1:QraP0UcVlQJsmHfioCrveWOC1nbiWUl3ej08h4mXWoc=
github.com/aws/aws-sdk-go-v2/config v1.28.10 h1:fKODZHfqQu06pCzR69KJ3GuttraRJkhlC8g80RZ0Dfg=
github.com/aws/aws-sdk-go-v2/config v1.28.10/go.mod h1:PvdxRYZ5Um9QMq9PQ0zHHNdtKK+he2NHtFCUFMXWXeg=
github.com/aws/aws-sdk-go-v2/credentials v1.17.51 h1:F/9Sm6Y6k4LqDesZDPJCLxQGXNNHd/ZtJiWd0lCZKRk=
github.com/aws/aws-sdk-go-v2/credentials v1.17.51/go.mod h1:TKbzCHm43AoPyA+iLGGcruXd4AFhF8tOmLex2R9jWNQ=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.23 h1:IBAoD/1d8A8/1aA8g4MBVtTRHhXRiNAgwdbo/xRM2DI=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.23/go.mod h1:vfENuCM7dofkgKpYzuzf1VT1UKkA/YL3qanfBn7HCaA=
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.48 h1:XnXVe2zRyPf0+fAW5L05esmngvBpC6DQZK7oZB/z/Co=
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.48/go.mod h1:S3wey90OrS4f7kYxH6PT175YyEcHTORY07++HurMaRM=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.27 h1:jSJjSBzw8VDIbWv+mmvBSP8ezsztMYJGH+eKqi9AmNs=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.27/go.mod h1:/DAhLbFRgwhmvJdOfSm+WwikZrCuUJiA4WgJG0fTNSw=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.27 h1:l+X4K77Dui85pIj5foXDhPlnqcNRG2QUyvca300lXh8=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.27/go.mod h1:KvZXSFEXm6x84yE8qffKvT3x8J5clWnVFXphpohhzJ8=
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 h1:VaRN3TlFdd6KxX1x3ILT5ynH6HvKgqdiXoTxAF4HQcQ=
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1/go.mod h1:FbtygfRFze9usAadmnGJNc8KsP346kEe+y2/oyhGAGc=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.27 h1:AmB5QxnD+fBFrg9LcqzkgF/CaYvMyU/BTlejG4t1S7Q=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.27/go.mod h1:Sai7P3xTiyv9ZUYO3IFxMnmiIP759/67iQbU4kdmkyU=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.1 h1:iXtILhvDxB6kPvEXgsDhGaZCSC6LQET5ZHSdJozeI0Y=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.1/go.mod h1:9nu0fVANtYiAePIBh2/pFUSwtJ402hLnp854CNoDOeE=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.4.8 h1:iwYS40JnrBeA9e9aI5S6KKN4EB2zR4iUVYN0nwVivz4=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.4.8/go.mod h1:Fm9Mi+ApqmFiknZtGpohVcBGvpTu542VC4XO9YudRi0=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.8 h1:cWno7lefSH6Pp+mSznagKCgfDGeZRin66UvYUqAkyeA=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.8/go.mod h1:tPD+VjU3ABTBoEJ3nctu5Nyg4P4yjqSH5bJGGkY4+XE=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.8 h1:/Mn7gTedG86nbpjT4QEKsN1D/fThiYe1qvq7WsBGNHg=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.8/go.mod h1:Ae3va9LPmvjj231ukHB6UeT8nS7wTPfC3tMZSZMwNYg=
github.com/aws/aws-sdk-go-v2/service/s3 v1.72.2 h1:a7aQ3RW+ug4IbhoQp29NZdc7vqrzKZZfWZSaQAXOZvQ=
github.com/aws/aws-sdk-go-v2/service/s3 v1.72.2/go.mod h1:xMekrnhmJ5aqmyxtmALs7mlvXw5xRh+eYjOjvrIIFJ4=
github.com/aws/aws-sdk-go-v2/service/sso v1.24.9 h1:YqtxripbjWb2QLyzRK9pByfEDvgg95gpC2AyDq4hFE8=
github.com/aws/aws-sdk-go-v2/service/sso v1.24.9/go.mod h1:lV8iQpg6OLOfBnqbGMBKYjilBlf633qwHnBEiMSPoHY=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.8 h1:6dBT1Lz8fK11m22R+AqfRsFn8320K0T5DTGxxOQBSMw=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.8/go.mod h1:/kiBvRQXBc6xeJTYzhSdGvJ5vm1tjaDEjH+MSeRJnlY=
github.com/aws/aws-sdk-go-v2/service/sts v1.33.6 h1:VwhTrsTuVn52an4mXx29PqRzs2Dvu921NpGk7y43tAM=
github.com/aws/aws-sdk-go-v2/service/sts v1.33.6/go.mod h1:+8h7PZb3yY5ftmVLD7ocEoE98hdc8PoKS0H3wfx1dlc=
github.com/aws/smithy-go v1.22.1 h1:/HPHZQ0g7f4eUeK6HKglFz8uwVfZKgoI25rb/J+dnro=
github.com/aws/smithy-go v1.22.1/go.mod h1:irrKGvNn1InZwb2d7fkIRNucdfwR8R+Ts3wxYa/cJHg=
github.com/aws/aws-sdk-go v1.55.6 h1:cSg4pvZ3m8dgYcgqB97MrcdjUmZ1BeMYKUxMMB89IPk=
github.com/aws/aws-sdk-go v1.55.6/go.mod h1:eRwEWoyTWFMVYVQzKMNHWP5/RV4xIUGMQfXQHfHkpNU=
github.com/aws/aws-sdk-go-v2 v1.34.0 h1:9iyL+cjifckRGEVpRKZP3eIxVlL06Qk1Tk13vreaVQU=
github.com/aws/aws-sdk-go-v2 v1.34.0/go.mod h1:JgstGg0JjWU1KpVJjD5H0y0yyAIpSdKEq556EI6yOOM=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.8 h1:zAxi9p3wsZMIaVCdoiQp2uZ9k1LsZvmAnoTBeZPXom0=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.8/go.mod h1:3XkePX5dSaxveLAYY7nsbsZZrKxCyEuE5pM4ziFxyGg=
github.com/aws/aws-sdk-go-v2/config v1.29.2 h1:JuIxOEPcSKpMB0J+khMjznG9LIhIBdmqNiEcPclnwqc=
github.com/aws/aws-sdk-go-v2/config v1.29.2/go.mod h1:HktTHregOZwNSM/e7WTfVSu9RCX+3eOv+6ij27PtaYs=
github.com/aws/aws-sdk-go-v2/credentials v1.17.55 h1:CDhKnDEaGkLA5ZszV/qw5uwN5M8rbv9Cl0JRN+PRsaM=
github.com/aws/aws-sdk-go-v2/credentials v1.17.55/go.mod h1:kPD/vj+RB5MREDUky376+zdnjZpR+WgdBBvwrmnlmKE=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.25 h1:kU7tmXNaJ07LsyN3BUgGqAmVmQtq0w6duVIHAKfp0/w=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.25/go.mod h1:OiC8+OiqrURb1wrwmr/UbOVLFSWEGxjinj5C299VQdo=
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.54 h1:6BWOAho3Cgdy4cmNJ4HWY8VZgqODEU7Gw78XXireNZI=
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.54/go.mod h1:n+t/oyYErOV3jf/GxNTVlizSM9RMV1yH7jvcIvld3Do=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.29 h1:Ej0Rf3GMv50Qh4G4852j2djtoDb7AzQ7MuQeFHa3D70=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.29/go.mod h1:oeNTC7PwJNoM5AznVr23wxhLnuJv0ZDe5v7w0wqIs9M=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.29 h1:6e8a71X+9GfghragVevC5bZqvATtc3mAMgxpSNbgzF0=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.29/go.mod h1:c4jkZiQ+BWpNqq7VtrxjwISrLrt/VvPq3XiopkUIolI=
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.2 h1:Pg9URiobXy85kgFev3og2CuOZ8JZUBENF+dcgWBaYNk=
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.2/go.mod h1:FbtygfRFze9usAadmnGJNc8KsP346kEe+y2/oyhGAGc=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.29 h1:g9OUETuxA8i/Www5Cby0R3WSTe7ppFTZXHVLNskNS4w=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.29/go.mod h1:CQk+koLR1QeY1+vm7lqNfFii07DEderKq6T3F1L2pyc=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.2 h1:D4oz8/CzT9bAEYtVhSBmFj2dNOtaHOtMKc2vHBwYizA=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.2/go.mod h1:Za3IHqTQ+yNcRHxu1OFucBh0ACZT4j4VQFF0BqpZcLY=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.5.3 h1:EP1ITDgYVPM2dL1bBBntJ7AW5yTjuWGz9XO+CZwpALU=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.5.3/go.mod h1:5lWNWeAgWenJ/BZ/CP9k9DjLbC0pjnM045WjXRPPi14=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.10 h1:hN4yJBGswmFTOVYqmbz1GBs9ZMtQe8SrYxPwrkrlRv8=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.10/go.mod h1:TsxON4fEZXyrKY+D+3d2gSTyJkGORexIYab9PTf56DA=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.10 h1:fXoWC2gi7tdJYNTPnnlSGzEVwewUchOi8xVq/dkg8Qs=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.10/go.mod h1:cvzBApD5dVazHU8C2rbBQzzzsKc8m5+wNJ9mCRZLKPc=
github.com/aws/aws-sdk-go-v2/service/s3 v1.74.1 h1:9LawY3cDJ3HE+v2GMd5SOkNLDwgN4K7TsCjyVBYu/L4=
github.com/aws/aws-sdk-go-v2/service/s3 v1.74.1/go.mod h1:hHnELVnIHltd8EOF3YzahVX6F6y2C6dNqpRj1IMkS5I=
github.com/aws/aws-sdk-go-v2/service/sso v1.24.12 h1:kznaW4f81mNMlREkU9w3jUuJvU5g/KsqDV43ab7Rp6s=
github.com/aws/aws-sdk-go-v2/service/sso v1.24.12/go.mod h1:bZy9r8e0/s0P7BSDHgMLXK2KvdyRRBIQ2blKlvLt0IU=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.11 h1:mUwIpAvILeKFnRx4h1dEgGEFGuV8KJ3pEScZWVFYuZA=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.11/go.mod h1:JDJtD+b8HNVv71axz8+S5492KM8wTzHRFpMKQbPlYxw=
github.com/aws/aws-sdk-go-v2/service/sts v1.33.10 h1:g9d+TOsu3ac7SgmY2dUf1qMgu/uJVTlQ4VCbH6hRxSw=
github.com/aws/aws-sdk-go-v2/service/sts v1.33.10/go.mod h1:WZfNmntu92HO44MVZAubQaz3qCuIdeOdog2sADfU6hU=
github.com/aws/smithy-go v1.22.2 h1:6D9hW43xKFrRx/tXXfAlIZc4JI+yQe6snnWcQyxSyLQ=
github.com/aws/smithy-go v1.22.2/go.mod h1:irrKGvNn1InZwb2d7fkIRNucdfwR8R+Ts3wxYa/cJHg=
github.com/bboreham/go-loser v0.0.0-20230920113527-fcc2c21820a3 h1:6df1vn4bBlDDo4tARvBm7l6KA9iVMnE3NWizDeWSrps=
github.com/bboreham/go-loser v0.0.0-20230920113527-fcc2c21820a3/go.mod h1:CIWtjkly68+yqLPbvwwR/fjNJA/idrtULjZWh2v1ys0=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/bmatcuk/doublestar/v4 v4.7.1 h1:fdDeAqgT47acgwd9bd9HxJRDmc9UAmPpc+2m0CXv75Q=
github.com/bmatcuk/doublestar/v4 v4.7.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/bmatcuk/doublestar/v4 v4.8.1 h1:54Bopc5c2cAvhLRAzqOGCYHYyhcDHsFF4wWIR5wKP38=
github.com/bmatcuk/doublestar/v4 v4.8.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cheggaaa/pb/v3 v3.1.5 h1:QuuUzeM2WsAqG2gMqtzaWithDJv0i+i6UlnwSCI4QLk=
github.com/cheggaaa/pb/v3 v3.1.5/go.mod h1:CrxkeghYTXi1lQBEI7jSn+3svI3cuc19haAj6jM60XI=
github.com/cheggaaa/pb/v3 v3.1.6 h1:h0x+vd7EiUohAJ29DJtJy+SNAc55t/elW3jCD086EXk=
github.com/cheggaaa/pb/v3 v3.1.6/go.mod h1:urxmfVtaxT+9aWk92DbsvXFZtNSWQSO5TRAp+MJ3l1s=
github.com/cncf/xds/go v0.0.0-20241223141626-cff3c89139a3 h1:boJj011Hh+874zpIySeApCX4GeOjPl9qhRF3QuIZq+Q=
github.com/cncf/xds/go v0.0.0-20241223141626-cff3c89139a3/go.mod h1:W+zGtBO5Y1IgJhy4+A9GOqVhqLpfZi+vwmdNXUehLA8=
github.com/cpuguy83/go-md2man/v2 v2.0.6 h1:XJtiaUW6dEEqVuZiMTn1ldk455QWwEIsMIJlo5vtkx0=
@@ -298,16 +298,16 @@ github.com/prometheus/client_golang v1.20.5 h1:cxppBPuYhUnsO6yo/aoRol4L7q7UFfdm+
github.com/prometheus/client_golang v1.20.5/go.mod h1:PIEt8X02hGcP8JWbeHyeZ53Y/jReSnHgO035n//V5WE=
github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E=
github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY=
github.com/prometheus/common v0.61.0 h1:3gv/GThfX0cV2lpO7gkTUwZru38mxevy90Bj8YFSRQQ=
github.com/prometheus/common v0.61.0/go.mod h1:zr29OCN/2BsJRaFwG8QOBr41D6kkchKbpeNH7pAjb/s=
github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ2Io=
github.com/prometheus/common v0.62.0/go.mod h1:vyBcEuLSvWos9B1+CyL7JZ2up+uFzXhkqml0W5zIY1I=
github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
github.com/prometheus/prometheus v0.301.0 h1:0z8dgegmILivNomCd79RKvVkIols8vBGPKmcIBc7OyY=
github.com/prometheus/prometheus v0.301.0/go.mod h1:BJLjWCKNfRfjp7Q48DrAjARnCi7GhfUVvUFEAWTssZM=
github.com/prometheus/sigv4 v0.1.1 h1:UJxjOqVcXctZlwDjpUpZ2OiMWJdFijgSofwLzO1Xk0Q=
github.com/prometheus/sigv4 v0.1.1/go.mod h1:RAmWVKqx0bwi0Qm4lrKMXFM0nhpesBcenfCtz9qRyH8=
github.com/redis/go-redis/v9 v9.6.1 h1:HHDteefn6ZkTtY5fGUE8tj8uy85AHk6zP7CpzIAM0y4=
github.com/redis/go-redis/v9 v9.6.1/go.mod h1:0C0c6ycQsdpVNQpxb1njEQIqkx5UcsM8FJCQLgE9+RA=
github.com/redis/go-redis/v9 v9.7.0 h1:HhLSs+B6O021gwzl+locl0zEDnyNkxMtf/Z3NNBMa9E=
github.com/redis/go-redis/v9 v9.7.0/go.mod h1:f6zhXITC7JUJIlPEiBOTXxJgPLdZcA93GewI7inzyWw=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
@@ -359,26 +359,26 @@ go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/collector/pdata v1.23.0 h1:tEk0dkfB8RdSukoOMfEa8duB938gfZowdfRkrJxGDrw=
go.opentelemetry.io/collector/pdata v1.23.0/go.mod h1:I2jggpBMiO8A+7TXhzNpcJZkJtvi1cU0iVNIi+6bc+o=
go.opentelemetry.io/collector/semconv v0.117.0 h1:SavOvSbHPVD/QdAnXlI/cMca+yxCNyXStY1mQzerHs4=
go.opentelemetry.io/collector/semconv v0.117.0/go.mod h1:N6XE8Q0JKgBN2fAhkUQtqK9LT7rEGR6+Wu/Rtbal1iI=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.58.0 h1:PS8wXpbyaDJQ2VDHHncMe9Vct0Zn1fEjpsjrLxGJoSc=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.58.0/go.mod h1:HDBUsEjOuRC0EzKZ1bSaRGZWUBAzo+MhAcUUORSr4D0=
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.58.0 h1:xwH3QJv6zL4u+gkPUu59NeT1Gyw9nScWT8FQpKLUJJI=
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.58.0/go.mod h1:uosvgpqTcTXtcPQORTbEkZNDQTCDOgTz1fe6aLSyqrQ=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.58.0 h1:yd02MEjBdJkG3uabWP9apV+OuWRIXGDuJEUJbOHmCFU=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.58.0/go.mod h1:umTcuxiv1n/s/S6/c2AT/g2CQ7u5C59sHDNmfSwgz7Q=
go.opentelemetry.io/otel v1.33.0 h1:/FerN9bax5LoK51X/sI0SVYrjSE0/yUL7DpxW4K3FWw=
go.opentelemetry.io/otel v1.33.0/go.mod h1:SUUkR6csvUQl+yjReHu5uM3EtVV7MBm5FHKRlNx4I8I=
go.opentelemetry.io/otel/metric v1.33.0 h1:r+JOocAyeRVXD8lZpjdQjzMadVZp2M4WmQ+5WtEnklQ=
go.opentelemetry.io/otel/metric v1.33.0/go.mod h1:L9+Fyctbp6HFTddIxClbQkjtubW6O9QS3Ann/M82u6M=
go.opentelemetry.io/collector/pdata v1.24.0 h1:D6j92eAzmAbQgivNBUnt8r9juOl8ugb+ihYynoFZIEg=
go.opentelemetry.io/collector/pdata v1.24.0/go.mod h1:cf3/W9E/uIvPS4MR26SnMFJhraUCattzzM6qusuONuc=
go.opentelemetry.io/collector/semconv v0.118.0 h1:V4vlMIK7TIaemrrn2VawvQPwruIKpj7Xgw9P5+BL56w=
go.opentelemetry.io/collector/semconv v0.118.0/go.mod h1:N6XE8Q0JKgBN2fAhkUQtqK9LT7rEGR6+Wu/Rtbal1iI=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.59.0 h1:rgMkmiGfix9vFJDcDi1PK8WEQP4FLQwLDfhp5ZLpFeE=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.59.0/go.mod h1:ijPqXp5P6IRRByFVVg9DY8P5HkxkHE5ARIa+86aXPf4=
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.59.0 h1:iQZYNQ7WwIcYXzOPR46FQv9O0dS1PW16RjvR0TjDOe8=
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.59.0/go.mod h1:54CaSNqYEXvpzDh8KPjiMVoWm60t5R0dZRt0leEPgAs=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.59.0 h1:CV7UdSGJt/Ao6Gp4CXckLxVRRsRgDHoI8XjbL3PDl8s=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.59.0/go.mod h1:FRmFuRJfag1IZ2dPkHnEoSFVgTVPUd2qf5Vi69hLb8I=
go.opentelemetry.io/otel v1.34.0 h1:zRLXxLCgL1WyKsPVrgbSdMN4c0FMkDAskSTQP+0hdUY=
go.opentelemetry.io/otel v1.34.0/go.mod h1:OWFPOQ+h4G8xpyjgqo4SxJYdDQ/qmRH+wivy7zzx9oI=
go.opentelemetry.io/otel/metric v1.34.0 h1:+eTR3U0MyfWjRDhmFMxe2SsW64QrZ84AOhvqS7Y+PoQ=
go.opentelemetry.io/otel/metric v1.34.0/go.mod h1:CEDrp0fy2D0MvkXE+dPV7cMi8tWZwX3dmaIhwPOaqHE=
go.opentelemetry.io/otel/sdk v1.33.0 h1:iax7M131HuAm9QkZotNHEfstof92xM+N8sr3uHXc2IM=
go.opentelemetry.io/otel/sdk v1.33.0/go.mod h1:A1Q5oi7/9XaMlIWzPSxLRWOI8nG3FnzHJNbiENQuihM=
go.opentelemetry.io/otel/sdk/metric v1.33.0 h1:Gs5VK9/WUJhNXZgn8MR6ITatvAmKeIuCtNbsP3JkNqU=
go.opentelemetry.io/otel/sdk/metric v1.33.0/go.mod h1:dL5ykHZmm1B1nVRk9dDjChwDmt81MjVp3gLkQRwKf/Q=
go.opentelemetry.io/otel/trace v1.33.0 h1:cCJuF7LRjUFso9LPnEAHJDB2pqzp+hbO8eu1qqW2d/s=
go.opentelemetry.io/otel/trace v1.33.0/go.mod h1:uIcdVUZMpTAmz0tI1z04GoVSezK37CbGV4fr1f2nBck=
go.opentelemetry.io/otel/trace v1.34.0 h1:+ouXS2V8Rd4hp4580a8q23bg0azF2nI8cqLYnC8mh/k=
go.opentelemetry.io/otel/trace v1.34.0/go.mod h1:Svm7lSjQD7kG7KJ/MUHPVXSDGz2OX4h0M2jHBhmSfRE=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
@@ -437,18 +437,18 @@ golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8T
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.216.0 h1:xnEHy+xWFrtYInWPy8OdGFsyIfWJjtVnO39g7pz2BFY=
google.golang.org/api v0.216.0/go.mod h1:K9wzQMvWi47Z9IU7OgdOofvZuw75Ge3PPITImZR/UyI=
google.golang.org/genproto v0.0.0-20250106144421-5f5ef82da422 h1:6GUHKGv2huWOHKmDXLMNE94q3fBDlEHI+oTRIZSebK0=
google.golang.org/genproto v0.0.0-20250106144421-5f5ef82da422/go.mod h1:1NPAxoesyw/SgLPqaUp9u1f9PWCLAk/jVmhx7gJZStg=
google.golang.org/genproto/googleapis/api v0.0.0-20250106144421-5f5ef82da422 h1:GVIKPyP/kLIyVOgOnTwFOrvQaQUzOzGMCxgFUOEmm24=
google.golang.org/genproto/googleapis/api v0.0.0-20250106144421-5f5ef82da422/go.mod h1:b6h1vNKhxaSoEI+5jc3PJUCustfli/mRab7295pY7rw=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250106144421-5f5ef82da422 h1:3UsHvIr4Wc2aW4brOaSCmcxh9ksica6fHEr8P1XhkYw=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250106144421-5f5ef82da422/go.mod h1:3ENsm/5D1mzDyhpzeRi1NR784I0BcofWBoSc5QqqMK4=
google.golang.org/grpc v1.69.2 h1:U3S9QEtbXC0bYNvRtcoklF3xGtLViumSYxWykJS+7AU=
google.golang.org/grpc v1.69.2/go.mod h1:vyjdE6jLBI76dgpDojsFGNaHlxdjXN9ghpnd2o7JGZ4=
google.golang.org/protobuf v1.36.2 h1:R8FeyR1/eLmkutZOM5CWghmo5itiG9z0ktFlTVLuTmU=
google.golang.org/protobuf v1.36.2/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
google.golang.org/api v0.218.0 h1:x6JCjEWeZ9PFCRe9z0FBrNwj7pB7DOAqT35N+IPnAUA=
google.golang.org/api v0.218.0/go.mod h1:5VGHBAkxrA/8EFjLVEYmMUJ8/8+gWWQ3s4cFH0FxG2M=
google.golang.org/genproto v0.0.0-20250127172529-29210b9bc287 h1:WoUI1G0DQ648FKvSl756SKxHQR/bI+y4HyyIQfxMWI8=
google.golang.org/genproto v0.0.0-20250127172529-29210b9bc287/go.mod h1:wkQ2Aj/xvshAUDtO/JHvu9y+AaN9cqs28QuSVSHtZSY=
google.golang.org/genproto/googleapis/api v0.0.0-20250127172529-29210b9bc287 h1:A2ni10G3UlplFrWdCDJTl7D7mJ7GSRm37S+PDimaKRw=
google.golang.org/genproto/googleapis/api v0.0.0-20250127172529-29210b9bc287/go.mod h1:iYONQfRdizDB8JJBybql13nArx91jcUk7zCXEsOofM4=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250127172529-29210b9bc287 h1:J1H9f+LEdWAfHcez/4cvaVBox7cOYT+IU6rgqj5x++8=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250127172529-29210b9bc287/go.mod h1:8BS3B93F/U1juMFq9+EDk+qOT5CO1R9IzXxG3PTqiRk=
google.golang.org/grpc v1.70.0 h1:pWFv03aZoHzlRKHWicjsZytKAiYCtNS0dHbXnIdq7jQ=
google.golang.org/grpc v1.70.0/go.mod h1:ofIJqVKDXx/JiXrwr2IG4/zwdH9txy3IlF40RmcJSQw=
google.golang.org/protobuf v1.36.4 h1:6A3ZDJHn/eNqc1i+IdefRzy/9PokBTPvcqMySR7NNIM=
google.golang.org/protobuf v1.36.4/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
@@ -464,12 +464,12 @@ gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
k8s.io/api v0.32.0 h1:OL9JpbvAU5ny9ga2fb24X8H6xQlVp+aJMFlgtQjR9CE=
k8s.io/api v0.32.0/go.mod h1:4LEwHZEf6Q/cG96F3dqR965sYOfmPM7rq81BLgsE0p0=
k8s.io/apimachinery v0.32.0 h1:cFSE7N3rmEEtv4ei5X6DaJPHHX0C+upp+v5lVPiEwpg=
k8s.io/apimachinery v0.32.0/go.mod h1:GpHVgxoKlTxClKcteaeuF1Ul/lDVb74KpZcxcmLDElE=
k8s.io/client-go v0.32.0 h1:DimtMcnN/JIKZcrSrstiwvvZvLjG0aSxy8PxN8IChp8=
k8s.io/client-go v0.32.0/go.mod h1:boDWvdM1Drk4NJj/VddSLnx59X3OPgwrOo0vGbtq9+8=
k8s.io/api v0.32.1 h1:f562zw9cy+GvXzXf0CKlVQ7yHJVYzLfL6JAS4kOAaOc=
k8s.io/api v0.32.1/go.mod h1:/Yi/BqkuueW1BgpoePYBRdDYfjPF5sgTr5+YqDZra5k=
k8s.io/apimachinery v0.32.1 h1:683ENpaCBjma4CYqsmZyhEzrGz6cjn1MY/X2jB2hkZs=
k8s.io/apimachinery v0.32.1/go.mod h1:GpHVgxoKlTxClKcteaeuF1Ul/lDVb74KpZcxcmLDElE=
k8s.io/client-go v0.32.1 h1:otM0AxdhdBIaQh7l1Q0jQpmo7WOFIk5FFa4bg6YMdUU=
k8s.io/client-go v0.32.1/go.mod h1:aTTKZY7MdxUaJ/KiUs8D+GssR9zJZi77ZqtzcGXIiDg=
k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=
k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE=
k8s.io/kube-openapi v0.0.0-20241105132330-32ad38e42d3f h1:GA7//TjRY9yWGy1poLzYYJJ4JRdzg3+O6e8I+e+8T5Y=

View File

@@ -8,6 +8,8 @@ import (
"strings"
"github.com/VictoriaMetrics/metrics"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
)
// AvailableCPUs returns the number of available CPU cores for the app.
@@ -44,6 +46,9 @@ func updateGOMAXPROCSToCPUQuota(cpuQuota float64) {
if gomaxprocs <= 0 {
gomaxprocs = 1
}
if cpuQuota > float64(gomaxprocs) {
logger.Warnf("rounding CPU quota %.1f to %d CPUs for performance reasons - see https://docs.victoriametrics.com/bestpractices/#kubernetes", cpuQuota, gomaxprocs)
}
numCPU := runtime.NumCPU()
if gomaxprocs > numCPU {

View File

@@ -9,7 +9,7 @@ import (
"sync"
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/envutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fs/fsutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/memory"
"github.com/VictoriaMetrics/metrics"
@@ -242,9 +242,9 @@ func (w *Writer) MustClose() {
putBufioWriter(w.bw)
w.bw = nil
if !envutil.IsFsyncDisabled() {
if !fsutil.IsFsyncDisabled() {
if err := w.f.Sync(); err != nil {
logger.Panicf("FATAL: cannot sync file %q: %d", w.f.Name(), err)
logger.Panicf("FATAL: cannot sync file %q: %s", w.f.Name(), err)
}
}
if err := w.st.close(); err != nil {

View File

@@ -4,7 +4,7 @@ import (
"fmt"
"os"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/envutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fs/fsutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"golang.org/x/sys/unix"
)
@@ -31,7 +31,7 @@ func mustSyncPath(path string) {
if err != nil {
logger.Panicf("FATAL: cannot open file for fsync: %s", err)
}
if !envutil.IsFsyncDisabled() {
if !fsutil.IsFsyncDisabled() {
if err := d.Sync(); err != nil {
_ = d.Close()
logger.Panicf("FATAL: cannot flush %q to storage: %s", path, err)

View File

@@ -6,7 +6,7 @@ import (
"fmt"
"os"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/envutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fs/fsutil"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"golang.org/x/sys/unix"
)
@@ -24,7 +24,7 @@ func mustSyncPath(path string) {
if err != nil {
logger.Panicf("FATAL: cannot open file for fsync: %s", err)
}
if !envutil.IsFsyncDisabled() {
if !fsutil.IsFsyncDisabled() {
if err := d.Sync(); err != nil {
_ = d.Close()
logger.Panicf("FATAL: cannot flush %q to storage: %s", path, err)

View File

@@ -1,4 +1,4 @@
package envutil
package fsutil
import (
"os"
@@ -14,6 +14,12 @@ import (
// The fsync is enabled for ordinary programs. It can be disabled by setting DISABLE_FSYNC_FOR_TESTING
// environment variable to true.
func IsFsyncDisabled() bool {
return isFsyncDisabled
}
var isFsyncDisabled = isFsyncDisabledInternal()
func isFsyncDisabledInternal() bool {
s := os.Getenv("DISABLE_FSYNC_FOR_TESTING")
if s == "" {
return testing.Testing()

View File

@@ -1,18 +1,18 @@
package envutil
package fsutil
import (
"os"
"testing"
)
func TestIsFsyncDisabled(t *testing.T) {
func TestIsFsyncDisabledInternal(t *testing.T) {
f := func(envVarValue string, resultExpected bool) {
t.Helper()
os.Setenv("DISABLE_FSYNC_FOR_TESTING", envVarValue)
defer os.Unsetenv("DISABLE_FSYNC_FOR_TESTING")
result := IsFsyncDisabled()
result := isFsyncDisabledInternal()
if result != resultExpected {
t.Errorf("unexpected value for DISABLE_FSYNC_FOR_TESTING=%q; got %v; want %v", envVarValue, result, resultExpected)
}

View File

@@ -29,10 +29,10 @@ const maxRowsPerBlock = 8 * 1024 * 1024
// in excess memory usage during data ingestion and significant slowdown during query execution.
const maxColumnsPerBlock = 2_000
// MaxFieldNameSize is the maximum size in bytes for field name.
// maxFieldNameSize is the maximum size in bytes for field name.
//
// Longer field names are truncated during data ingestion to MaxFieldNameSize length.
const MaxFieldNameSize = 128
// Log entries with longer field names are rejected during data ingestion.
const maxFieldNameSize = 128
// maxConstColumnValueSize is the maximum size in bytes for const column value.
//

View File

@@ -65,15 +65,6 @@ func (sf *sortedFields) Swap(i, j int) {
a[i], a[j] = a[j], a[i]
}
// RowFormatter implementes fmt.Stringer for []Field aka a single log row
type RowFormatter []Field
// String returns user-readable representation for rf
func (rf *RowFormatter) String() string {
result := MarshalFieldsToJSON(nil, *rf)
return string(result)
}
// Reset resets lr with all its settings.
//
// Call ResetKeepSettings() for resetting lr without resetting its settings.
@@ -142,23 +133,33 @@ func (lr *LogRows) NeedFlush() bool {
// It is OK to modify the args after returning from the function,
// since lr copies all the args to internal data.
//
// Field names longer than MaxFieldNameSize are automatically truncated to MaxFieldNameSize length.
//
// Log entries with too big number of fields are ignored.
// Loo long log entries are ignored.
// Log entries are dropped with the warning message in the following cases:
// - if there are too many log fields
// - if there are too long log field names
// - if the total length of log entries is too long
func (lr *LogRows) MustAdd(tenantID TenantID, timestamp int64, fields, streamFields []Field) {
// Verify that the log entry doesn't exceed limits.
if len(fields) > maxColumnsPerBlock {
fieldNames := make([]string, len(fields))
for i, f := range fields {
fieldNames[i] = f.Name
}
logger.Infof("ignoring log entry with too big number of fields, which exceeds %d; fieldNames=%q", maxColumnsPerBlock, fieldNames)
line := MarshalFieldsToJSON(nil, fields)
logger.Warnf("ignoring log entry with too big number of fields %d, since it exceeds the limit %d; "+
"see https://docs.victoriametrics.com/victorialogs/faq/#how-many-fields-a-single-log-entry-may-contain ; log entry: %s", len(fields), maxColumnsPerBlock, line)
return
}
for i := range fields {
fieldName := fields[i].Name
if len(fieldName) > maxFieldNameSize {
line := MarshalFieldsToJSON(nil, fields)
logger.Warnf("ignoring log entry with too long field name %q, since its length (%d) exceeds the limit %d bytes; "+
"see https://docs.victoriametrics.com/victorialogs/faq/#what-is-the-maximum-supported-field-name-length ; log entry: %s",
fieldName, len(fieldName), maxFieldNameSize, line)
return
}
}
rowLen := uncompressedRowSizeBytes(fields)
if rowLen > maxUncompressedBlockSize {
logger.Infof("ignoring too long log record with the estimated size %d bytes, since it exceeds the limit %d; "+
"see https://docs.victoriametrics.com/victorialogs/faq/#what-length-a-log-record-is-expected-to-have", rowLen, maxUncompressedBlockSize)
line := MarshalFieldsToJSON(nil, fields)
logger.Warnf("ignoring too long log entry with the estimated length of %d bytes, since it exceeds the limit %d bytes; "+
"see https://docs.victoriametrics.com/victorialogs/faq/#what-length-a-log-record-is-expected-to-have ; log entry: %s", rowLen, maxUncompressedBlockSize, line)
return
}
@@ -248,9 +249,6 @@ func (lr *LogRows) addFieldsInternal(fields []Field, ignoreFields map[string]str
dstField := &fb[len(fb)-1]
fieldName := f.Name
if len(fieldName) > MaxFieldNameSize {
fieldName = fieldName[:MaxFieldNameSize]
}
if fieldName == "_msg" {
fieldName = ""
hasMsgField = true
@@ -267,20 +265,21 @@ func (lr *LogRows) addFieldsInternal(fields []Field, ignoreFields map[string]str
func (lr *LogRows) GetRowString(idx int) string {
tf := TimeFormatter(lr.timestamps[idx])
streamTags := getStreamTagsString(lr.streamTagsCanonicals[idx])
var rf RowFormatter
rf = append(rf[:0], lr.rows[idx]...)
rf = append(rf, Field{
var fields []Field
fields = append(fields[:0], lr.rows[idx]...)
fields = append(fields, Field{
Name: "_time",
Value: tf.String(),
})
rf = append(rf, Field{
fields = append(fields, Field{
Name: "_stream",
Value: streamTags,
})
sort.Slice(rf, func(i, j int) bool {
return rf[i].Name < rf[j].Name
sort.Slice(fields, func(i, j int) bool {
return fields[i].Name < fields[j].Name
})
return rf.String()
line := MarshalFieldsToJSON(nil, fields)
return string(line)
}
// GetLogRows returns LogRows from the pool for the given streamFields.

View File

@@ -147,8 +147,8 @@ func (pt *partition) mustAddRows(lr *LogRows) {
func (pt *partition) logNewStream(streamTagsCanonical []byte, fields []Field) {
streamTags := getStreamTagsString(streamTagsCanonical)
rf := RowFormatter(fields)
logger.Infof("partition %s: new stream %s for log entry %s", pt.path, streamTags, &rf)
line := MarshalFieldsToJSON(nil, fields)
logger.Infof("partition %s: new stream %s for log entry %s", pt.path, streamTags, line)
}
func (pt *partition) logIngestedRows(lr *LogRows) {

View File

@@ -9,6 +9,7 @@ import (
"sync/atomic"
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/cgroup"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fs"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/timeutil"
@@ -268,12 +269,17 @@ func MustOpenStorage(path string, cfg *StorageConfig) *Storage {
// Open partitions in parallel. This should improve VictoriaLogs initializiation duration
// when it opens many partitions.
var wg sync.WaitGroup
concurrencyLimiterCh := make(chan struct{}, cgroup.AvailableCPUs())
for i, de := range des {
fname := de.Name()
wg.Add(1)
concurrencyLimiterCh <- struct{}{}
go func(idx int) {
defer wg.Done()
defer func() {
<-concurrencyLimiterCh
wg.Done()
}()
t, err := time.Parse(partitionNameFormat, fname)
if err != nil {
@@ -538,22 +544,22 @@ func (s *Storage) MustAddRows(lr *LogRows) {
for i, ts := range lr.timestamps {
day := ts / nsecsPerDay
if day < minAllowedDay {
rf := RowFormatter(lr.rows[i])
line := MarshalFieldsToJSON(nil, lr.rows[i])
tsf := TimeFormatter(ts)
minAllowedTsf := TimeFormatter(minAllowedDay * nsecsPerDay)
tooSmallTimestampLogger.Warnf("skipping log entry with too small timestamp=%s; it must be bigger than %s according "+
"to the configured -retentionPeriod=%dd. See https://docs.victoriametrics.com/victorialogs/#retention ; "+
"log entry: %s", &tsf, &minAllowedTsf, durationToDays(s.retention), &rf)
"log entry: %s", &tsf, &minAllowedTsf, durationToDays(s.retention), line)
s.rowsDroppedTooSmallTimestamp.Add(1)
continue
}
if day > maxAllowedDay {
rf := RowFormatter(lr.rows[i])
line := MarshalFieldsToJSON(nil, lr.rows[i])
tsf := TimeFormatter(ts)
maxAllowedTsf := TimeFormatter(maxAllowedDay * nsecsPerDay)
tooBigTimestampLogger.Warnf("skipping log entry with too big timestamp=%s; it must be smaller than %s according "+
"to the configured -futureRetention=%dd; see https://docs.victoriametrics.com/victorialogs/#retention ; "+
"log entry: %s", &tsf, &maxAllowedTsf, durationToDays(s.futureRetention), &rf)
"log entry: %s", &tsf, &maxAllowedTsf, durationToDays(s.futureRetention), line)
s.rowsDroppedTooBigTimestamp.Add(1)
continue
}

View File

@@ -8,6 +8,7 @@ import (
"sync/atomic"
"time"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/cgroup"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fasttime"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fs"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
@@ -523,12 +524,31 @@ func mustOpenPartitions(smallPartitionsPath, bigPartitionsPath string, s *Storag
mustPopulatePartitionNames(smallPartitionsPath, ptNames)
mustPopulatePartitionNames(bigPartitionsPath, ptNames)
var pts []*partition
var ptsLock sync.Mutex
// Open partitions in parallel. This should reduce the time needed for opening multiple partitions.
var wg sync.WaitGroup
concurrencyLimiterCh := make(chan struct{}, cgroup.AvailableCPUs())
for ptName := range ptNames {
smallPartsPath := filepath.Join(smallPartitionsPath, ptName)
bigPartsPath := filepath.Join(bigPartitionsPath, ptName)
pt := mustOpenPartition(smallPartsPath, bigPartsPath, s)
pts = append(pts, pt)
wg.Add(1)
concurrencyLimiterCh <- struct{}{}
go func(ptName string) {
defer func() {
<-concurrencyLimiterCh
wg.Done()
}()
smallPartsPath := filepath.Join(smallPartitionsPath, ptName)
bigPartsPath := filepath.Join(bigPartitionsPath, ptName)
pt := mustOpenPartition(smallPartsPath, bigPartsPath, s)
ptsLock.Lock()
pts = append(pts, pt)
ptsLock.Unlock()
}(ptName)
}
wg.Wait()
return pts
}

View File

@@ -1,5 +1,15 @@
# Breaking Changes
## v1.8.0
### New errors from `NewManagedIdentityCredential` in some environments
`NewManagedIdentityCredential` now returns an error when `ManagedIdentityCredentialOptions.ID` is set in a hosting environment whose managed identity API doesn't support user-assigned identities. `ManagedIdentityCredential.GetToken()` formerly logged a warning in these cases. Returning an error instead prevents the credential authenticating an unexpected identity. The affected hosting environments are:
* Azure Arc
* Azure ML (when a resource or object ID is specified; client IDs are supported)
* Cloud Shell
* Service Fabric
## v1.6.0
### Behavioral change to `DefaultAzureCredential` in IMDS managed identity scenarios

View File

@@ -1,5 +1,19 @@
# Release History
## 1.8.1 (2025-01-15)
### Bugs Fixed
* User credential types inconsistently log access token scopes
* `DefaultAzureCredential` skips managed identity in Azure Container Instances
* Credentials having optional tenant IDs such as `AzureCLICredential` and
`InteractiveBrowserCredential` require setting `AdditionallyAllowedTenants`
when used with some clients
### Other Changes
* `ChainedTokenCredential` and `DefaultAzureCredential` continue to their next
credential after `ManagedIdentityCredential` receives an unexpected response
from IMDS, indicating the response is from something else such as a proxy
## 1.8.0 (2024-10-08)
### Other Changes

View File

@@ -54,17 +54,7 @@ The `azidentity` module focuses on OAuth authentication with Microsoft Entra ID.
### DefaultAzureCredential
`DefaultAzureCredential` simplifies authentication while developing applications that deploy to Azure by combining credentials used in Azure hosting environments and credentials used in local development. In production, it's better to use a specific credential type so authentication is more predictable and easier to debug. `DefaultAzureCredential` attempts to authenticate via the following mechanisms in this order, stopping when one succeeds:
![DefaultAzureCredential authentication flow](img/mermaidjs/DefaultAzureCredentialAuthFlow.svg)
1. **Environment** - `DefaultAzureCredential` will read account information specified via [environment variables](#environment-variables) and use it to authenticate.
1. **Workload Identity** - If the app is deployed on Kubernetes with environment variables set by the workload identity webhook, `DefaultAzureCredential` will authenticate the configured identity.
1. **Managed Identity** - If the app is deployed to an Azure host with managed identity enabled, `DefaultAzureCredential` will authenticate with it.
1. **Azure CLI** - If a user or service principal has authenticated via the Azure CLI `az login` command, `DefaultAzureCredential` will authenticate that identity.
1. **Azure Developer CLI** - If the developer has authenticated via the Azure Developer CLI `azd auth login` command, the `DefaultAzureCredential` will authenticate with that account.
> Note: `DefaultAzureCredential` is intended to simplify getting started with the SDK by handling common scenarios with reasonable default behaviors. Developers who want more control or whose scenario isn't served by the default settings should use other credential types.
`DefaultAzureCredential` simplifies authentication while developing apps that deploy to Azure by combining credentials used in Azure hosting environments with credentials used in local development. For more information, see [DefaultAzureCredential overview][dac_overview].
## Managed Identity
@@ -128,10 +118,10 @@ client := armresources.NewResourceGroupsClient("subscription ID", chain, nil)
### Credential chains
|Credential|Usage
|-|-
|[DefaultAzureCredential](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#DefaultAzureCredential)|Simplified authentication experience for getting started developing Azure apps
|[ChainedTokenCredential](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#ChainedTokenCredential)|Define custom authentication flows, composing multiple credentials
|Credential|Usage|Reference
|-|-|-
|[DefaultAzureCredential](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#DefaultAzureCredential)|Simplified authentication experience for getting started developing Azure apps|[DefaultAzureCredential overview][dac_overview]|
|[ChainedTokenCredential](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#ChainedTokenCredential)|Define custom authentication flows, composing multiple credentials|[ChainedTokenCredential overview][ctc_overview]|
### Authenticating Azure-Hosted Applications
@@ -260,4 +250,8 @@ For more information, see the
or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any
additional questions or comments.
<!-- LINKS -->
[ctc_overview]: https://aka.ms/azsdk/go/identity/credential-chains#chainedtokencredential-overview
[dac_overview]: https://aka.ms/azsdk/go/identity/credential-chains#defaultazurecredential-overview
![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-go%2Fsdk%2Fazidentity%2FREADME.png)

View File

@@ -22,13 +22,13 @@ Some credential types support opt-in persistent token caching (see [the below ta
Persistent caches are encrypted at rest using a mechanism that depends on the operating system:
| Operating system | Encryption facility |
|------------------|---------------------------------------|
| Linux | kernel key retention service (keyctl) |
| macOS | Keychain |
| Windows | Data Protection API (DPAPI) |
| Operating system | Encryption facility |
| ---------------- | ---------------------------------------------- |
| Linux | kernel key retention service (keyctl) |
| macOS | Keychain (requires cgo and native build tools) |
| Windows | Data Protection API (DPAPI) |
Persistent caching requires encryption. When the required encryption facility is unuseable, or the application is running on an unsupported OS, the persistent cache constructor returns an error. This doesn't mean that authentication is impossible, only that credentials can't persist authentication data and the application will need to reauthenticate the next time it runs. See the [package documentation][example] for example code showing how to configure persistent caching and access cached data.
Persistent caching requires encryption. When the required encryption facility is unuseable, or the application is running on an unsupported OS, the persistent cache constructor returns an error. This doesn't mean that authentication is impossible, only that credentials can't persist authentication data and the application will need to reauthenticate the next time it runs. See the package documentation for examples showing how to configure persistent caching and access cached data for [users][user_example] and [service principals][sp_example].
### Credentials supporting token caching
@@ -37,7 +37,7 @@ The following table indicates the state of in-memory and persistent caching in e
**Note:** in-memory caching is enabled by default for every type supporting it. Persistent token caching must be enabled explicitly. See the [package documentation][user_example] for an example showing how to do this for credential types authenticating users. For types that authenticate service principals, set the `Cache` field on the constructor's options as shown in [this example][sp_example].
| Credential | In-memory token caching | Persistent token caching |
|--------------------------------|---------------------------------------------------------------------|--------------------------|
| ------------------------------ | ------------------------------------------------------------------- | ------------------------ |
| `AzureCLICredential` | Not Supported | Not Supported |
| `AzureDeveloperCLICredential` | Not Supported | Not Supported |
| `AzurePipelinesCredential` | Supported | Supported |

View File

@@ -8,6 +8,7 @@ This troubleshooting guide covers failure investigation techniques, common error
- [Permission issues](#permission-issues)
- [Find relevant information in errors](#find-relevant-information-in-errors)
- [Enable and configure logging](#enable-and-configure-logging)
- [Troubleshoot persistent token caching issues](#troubleshoot-persistent-token-caching-issues)
- [Troubleshoot AzureCLICredential authentication issues](#troubleshoot-azureclicredential-authentication-issues)
- [Troubleshoot AzureDeveloperCLICredential authentication issues](#troubleshoot-azuredeveloperclicredential-authentication-issues)
- [Troubleshoot AzurePipelinesCredential authentication issues](#troubleshoot-azurepipelinescredential-authentication-issues)
@@ -236,6 +237,29 @@ azd auth token --output json --scope https://management.core.windows.net/.defaul
| No service connection found with identifier |The `serviceConnectionID` argument to `NewAzurePipelinesCredential` is incorrect| Verify the service connection ID. This parameter refers to the `resourceId` of the Azure Service Connection. It can also be found in the query string of the service connection's configuration in Azure DevOps. [Azure Pipelines documentation](https://learn.microsoft.com/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml) has more information about service connections.|
|401 (Unauthorized) response from OIDC endpoint|The `systemAccessToken` argument to `NewAzurePipelinesCredential` is incorrect|Check pipeline configuration. This value comes from the predefined variable `System.AccessToken` [as described in Azure Pipelines documentation](https://learn.microsoft.com/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#systemaccesstoken).|
## Troubleshoot persistent token caching issues
### macOS
[azidentity/cache](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache) encrypts persistent caches with the system Keychain on macOS. You may see build and runtime errors there because calling the Keychain API requires cgo and macOS prohibits Keychain access in some scenarios.
#### Build errors
Build errors about undefined `accessor` symbols indicate that cgo wasn't enabled. For example:
```
$ GOOS=darwin go build
# github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache
../../go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azidentity/cache@v0.3.0/darwin.go:18:19: undefined: accessor.New
../../go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azidentity/cache@v0.3.0/darwin.go:18:38: undefined: accessor.WithAccount
```
Try `go build` again with `CGO_ENABLED=1`. You may need to install native build tools.
#### Runtime errors
macOS prohibits Keychain access from environments without a GUI such as SSH sessions. If your application calls the persistent cache constructor ([cache.New](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache#New)) from an SSH session on a macOS host, you'll see an error like
`persistent storage isn't available due to error "User interaction is not allowed. (-25308)"`. This doesn't mean authentication is impossible, only that credentials can't persist data and the application must reauthenticate the next time it runs.
## Get additional help
Additional information on ways to reach out for support can be found in [SUPPORT.md](https://github.com/Azure/azure-sdk-for-go/blob/main/SUPPORT.md).

View File

@@ -42,6 +42,8 @@ const (
developerSignOnClientID = "04b07795-8ddb-461a-bbee-02f9e1bf7b46"
defaultSuffix = "/.default"
scopeLogFmt = "%s.GetToken() acquired a token for scope %q"
traceNamespace = "Microsoft.Entra"
traceOpGetToken = "GetToken"
traceOpAuthenticate = "Authenticate"
@@ -103,7 +105,16 @@ func resolveAdditionalTenants(tenants []string) []string {
return cp
}
// resolveTenant returns the correct tenant for a token request
// resolveTenant returns the correct tenant for a token request, or "" when the calling credential doesn't
// have an explicitly configured tenant and the caller didn't specify a tenant for the token request.
//
// - defaultTenant: tenant set when constructing the credential, if any. "" is valid for credentials
// having an optional or implicit tenant such as dev tool and interactive user credentials. Those
// default to the tool's configured tenant or the user's home tenant, respectively.
// - specified: tenant specified for this token request i.e., TokenRequestOptions.TenantID. May be "".
// - credName: name of the calling credential type; for error messages
// - additionalTenants: optional allow list of tenants the credential may acquire tokens from in
// addition to defaultTenant i.e., the credential's AdditionallyAllowedTenants option
func resolveTenant(defaultTenant, specified, credName string, additionalTenants []string) (string, error) {
if specified == "" || specified == defaultTenant {
return defaultTenant, nil
@@ -119,6 +130,17 @@ func resolveTenant(defaultTenant, specified, credName string, additionalTenants
return specified, nil
}
}
if len(additionalTenants) == 0 {
switch defaultTenant {
case "", organizationsTenantID:
// The application didn't specify a tenant or allow list when constructing the credential. Allow the
// tenant specified for this token request because we have nothing to compare it to (i.e., it vacuously
// satisfies the credential's configuration); don't know whether the application is multitenant; and
// don't want to return an error in the common case that the specified tenant matches the credential's
// default tenant determined elsewhere e.g., in some dev tool's configuration.
return specified, nil
}
}
return "", fmt.Errorf(`%s isn't configured to acquire tokens for tenant %q. To enable acquiring tokens for this tenant add it to the AdditionallyAllowedTenants on the credential options, or add "*" to allow acquiring tokens for any tenant`, credName, specified)
}

View File

@@ -30,9 +30,9 @@ type azTokenProvider func(ctx context.Context, scopes []string, tenant, subscrip
// AzureCLICredentialOptions contains optional parameters for AzureCLICredential.
type AzureCLICredentialOptions struct {
// AdditionallyAllowedTenants specifies tenants for which the credential may acquire tokens, in addition
// to TenantID. Add the wildcard value "*" to allow the credential to acquire tokens for any tenant the
// logged in account can access.
// AdditionallyAllowedTenants specifies tenants to which the credential may authenticate, in addition to
// TenantID. When TenantID is empty, this option has no effect and the credential will authenticate to
// any requested tenant. Add the wildcard value "*" to allow the credential to authenticate to any tenant.
AdditionallyAllowedTenants []string
// Subscription is the name or ID of a subscription. Set this to acquire tokens for an account other

View File

@@ -30,9 +30,9 @@ type azdTokenProvider func(ctx context.Context, scopes []string, tenant string)
// AzureDeveloperCLICredentialOptions contains optional parameters for AzureDeveloperCLICredential.
type AzureDeveloperCLICredentialOptions struct {
// AdditionallyAllowedTenants specifies tenants for which the credential may acquire tokens, in addition
// to TenantID. Add the wildcard value "*" to allow the credential to acquire tokens for any tenant the
// logged in account can access.
// AdditionallyAllowedTenants specifies tenants to which the credential may authenticate, in addition to
// TenantID. When TenantID is empty, this option has no effect and the credential will authenticate to
// any requested tenant. Add the wildcard value "*" to allow the credential to authenticate to any tenant.
AdditionallyAllowedTenants []string
// TenantID identifies the tenant the credential should authenticate in. Defaults to the azd environment,

View File

@@ -27,7 +27,10 @@ type ChainedTokenCredentialOptions struct {
}
// ChainedTokenCredential links together multiple credentials and tries them sequentially when authenticating. By default,
// it tries all the credentials until one authenticates, after which it always uses that credential.
// it tries all the credentials until one authenticates, after which it always uses that credential. For more information,
// see [ChainedTokenCredential overview].
//
// [ChainedTokenCredential overview]: https://aka.ms/azsdk/go/identity/credential-chains#chainedtokencredential-overview
type ChainedTokenCredential struct {
cond *sync.Cond
iterating bool
@@ -46,6 +49,9 @@ func NewChainedTokenCredential(sources []azcore.TokenCredential, options *Chaine
if source == nil { // cannot have a nil credential in the chain or else the application will panic when GetToken() is called on nil
return nil, errors.New("sources cannot contain nil")
}
if mc, ok := source.(*ManagedIdentityCredential); ok {
mc.mic.chained = true
}
}
cp := make([]azcore.TokenCredential, len(sources))
copy(cp, sources)

View File

@@ -26,27 +26,16 @@ extends:
parameters:
CloudConfig:
Public:
ServiceConnection: azure-sdk-tests
SubscriptionConfigurationFilePaths:
- eng/common/TestResources/sub-config/AzurePublicMsft.json
SubscriptionConfigurations:
- $(sub-config-azure-cloud-test-resources)
- $(sub-config-identity-test-resources)
EnableRaceDetector: true
Location: westus2
RunLiveTests: true
ServiceDirectory: azidentity
UsePipelineProxy: false
${{ if endsWith(variables['Build.DefinitionName'], 'weekly') }}:
PreSteps:
- task: AzureCLI@2
displayName: Set OIDC token
inputs:
addSpnToEnvironment: true
azureSubscription: azure-sdk-tests
inlineScript: Write-Host "##vso[task.setvariable variable=OIDC_TOKEN;]$($env:idToken)"
scriptLocation: inlineScript
scriptType: pscore
PersistOidcToken: true
MatrixConfigs:
- Name: managed_identity_matrix
GenerateVMJobs: true

View File

@@ -115,7 +115,7 @@ func (c *confidentialClient) GetToken(ctx context.Context, tro policy.TokenReque
err = newAuthenticationFailedErrorFromMSAL(c.name, err)
}
} else {
msg := fmt.Sprintf("%s.GetToken() acquired a token for scope %q", c.name, strings.Join(ar.GrantedScopes, ", "))
msg := fmt.Sprintf(scopeLogFmt, c.name, strings.Join(ar.GrantedScopes, ", "))
log.Write(EventAuthentication, msg)
}
return azcore.AccessToken{Token: ar.AccessToken, ExpiresOn: ar.ExpiresOn.UTC()}, err

View File

@@ -23,15 +23,19 @@ type DefaultAzureCredentialOptions struct {
// to credential types that authenticate via external tools such as the Azure CLI.
azcore.ClientOptions
// AdditionallyAllowedTenants specifies additional tenants for which the credential may acquire tokens. Add
// the wildcard value "*" to allow the credential to acquire tokens for any tenant. This value can also be
// set as a semicolon delimited list of tenants in the environment variable AZURE_ADDITIONALLY_ALLOWED_TENANTS.
// AdditionallyAllowedTenants specifies tenants to which the credential may authenticate, in addition to
// TenantID. When TenantID is empty, this option has no effect and the credential will authenticate to
// any requested tenant. Add the wildcard value "*" to allow the credential to authenticate to any tenant.
// This value can also be set as a semicolon delimited list of tenants in the environment variable
// AZURE_ADDITIONALLY_ALLOWED_TENANTS.
AdditionallyAllowedTenants []string
// DisableInstanceDiscovery should be set true only by applications authenticating in disconnected clouds, or
// private clouds such as Azure Stack. It determines whether the credential requests Microsoft Entra instance metadata
// from https://login.microsoft.com before authenticating. Setting this to true will skip this request, making
// the application responsible for ensuring the configured authority is valid and trustworthy.
DisableInstanceDiscovery bool
// TenantID sets the default tenant for authentication via the Azure CLI and workload identity.
TenantID string
}
@@ -39,7 +43,7 @@ type DefaultAzureCredentialOptions struct {
// DefaultAzureCredential simplifies authentication while developing applications that deploy to Azure by
// combining credentials used in Azure hosting environments and credentials used in local development. In
// production, it's better to use a specific credential type so authentication is more predictable and easier
// to debug.
// to debug. For more information, see [DefaultAzureCredential overview].
//
// DefaultAzureCredential attempts to authenticate with each of these credential types, in the following order,
// stopping when one provides a token:
@@ -55,6 +59,8 @@ type DefaultAzureCredentialOptions struct {
// Consult the documentation for these credential types for more information on how they authenticate.
// Once a credential has successfully authenticated, DefaultAzureCredential will use that credential for
// every subsequent authentication.
//
// [DefaultAzureCredential overview]: https://aka.ms/azsdk/go/identity/credential-chains#defaultazurecredential-overview
type DefaultAzureCredential struct {
chain *ChainedTokenCredential
}

View File

@@ -21,8 +21,9 @@ const credNameDeviceCode = "DeviceCodeCredential"
type DeviceCodeCredentialOptions struct {
azcore.ClientOptions
// AdditionallyAllowedTenants specifies additional tenants for which the credential may acquire
// tokens. Add the wildcard value "*" to allow the credential to acquire tokens for any tenant.
// AdditionallyAllowedTenants specifies tenants to which the credential may authenticate, in addition to
// TenantID. When TenantID is empty, this option has no effect and the credential will authenticate to
// any requested tenant. Add the wildcard value "*" to allow the credential to authenticate to any tenant.
AdditionallyAllowedTenants []string
// AuthenticationRecord returned by a call to a credential's Authenticate method. Set this option

View File

@@ -20,8 +20,9 @@ const credNameBrowser = "InteractiveBrowserCredential"
type InteractiveBrowserCredentialOptions struct {
azcore.ClientOptions
// AdditionallyAllowedTenants specifies additional tenants for which the credential may acquire
// tokens. Add the wildcard value "*" to allow the credential to acquire tokens for any tenant.
// AdditionallyAllowedTenants specifies tenants to which the credential may authenticate, in addition to
// TenantID. When TenantID is empty, this option has no effect and the credential will authenticate to
// any requested tenant. Add the wildcard value "*" to allow the credential to authenticate to any tenant.
AdditionallyAllowedTenants []string
// AuthenticationRecord returned by a call to a credential's Authenticate method. Set this option

View File

@@ -65,6 +65,9 @@ type managedIdentityClient struct {
id ManagedIDKind
msiType msiType
probeIMDS bool
// chained indicates whether the client is part of a credential chain. If true, the client will return
// a credentialUnavailableError instead of an AuthenticationFailedError for an unexpected IMDS response.
chained bool
}
// arcKeyDirectory returns the directory expected to contain Azure Arc keys
@@ -144,7 +147,7 @@ func newManagedIdentityClient(options *ManagedIdentityCredentialOptions) (*manag
if _, ok := os.LookupEnv(identityHeader); ok {
if _, ok := os.LookupEnv(identityServerThumbprint); ok {
if options.ID != nil {
return nil, errors.New("the Service Fabric API doesn't support specifying a user-assigned managed identity at runtime")
return nil, errors.New("the Service Fabric API doesn't support specifying a user-assigned identity at runtime. The identity is determined by cluster resource configuration. See https://aka.ms/servicefabricmi")
}
env = "Service Fabric"
c.endpoint = endpoint
@@ -215,6 +218,7 @@ func (c *managedIdentityClient) authenticate(ctx context.Context, id ManagedIDKi
// no need to synchronize around this value because it's true only when DefaultAzureCredential constructed the client,
// and in that case ChainedTokenCredential.GetToken synchronizes goroutines that would execute this block
if c.probeIMDS {
// send a malformed request (no Metadata header) to IMDS to determine whether the endpoint is available
cx, cancel := context.WithTimeout(ctx, imdsProbeTimeout)
defer cancel()
cx = policy.WithRetryOptions(cx, policy.RetryOptions{MaxRetries: -1})
@@ -222,24 +226,14 @@ func (c *managedIdentityClient) authenticate(ctx context.Context, id ManagedIDKi
if err != nil {
return azcore.AccessToken{}, fmt.Errorf("failed to create IMDS probe request: %s", err)
}
res, err := c.azClient.Pipeline().Do(req)
if err != nil {
if _, err = c.azClient.Pipeline().Do(req); err != nil {
msg := err.Error()
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
msg = "managed identity timed out. See https://aka.ms/azsdk/go/identity/troubleshoot#dac for more information"
}
return azcore.AccessToken{}, newCredentialUnavailableError(credNameManagedIdentity, msg)
}
// because IMDS always responds with JSON, assume a non-JSON response is from something else, such
// as a proxy, and return credentialUnavailableError so DefaultAzureCredential continues iterating
b, err := azruntime.Payload(res)
if err != nil {
return azcore.AccessToken{}, newCredentialUnavailableError(credNameManagedIdentity, fmt.Sprintf("failed to read IMDS probe response: %s", err))
}
if !json.Valid(b) {
return azcore.AccessToken{}, newCredentialUnavailableError(credNameManagedIdentity, "unexpected response to IMDS probe")
}
// send normal token requests from now on because IMDS responded
// send normal token requests from now on because something responded
c.probeIMDS = false
}
@@ -254,13 +248,21 @@ func (c *managedIdentityClient) authenticate(ctx context.Context, id ManagedIDKi
}
if azruntime.HasStatusCode(resp, http.StatusOK, http.StatusCreated) {
return c.createAccessToken(resp)
tk, err := c.createAccessToken(resp)
if err != nil && c.chained && c.msiType == msiTypeIMDS {
// failure to unmarshal a 2xx implies the response is from something other than IMDS such as a proxy listening at
// the same address. Return a credentialUnavailableError so credential chains continue to their next credential
err = newCredentialUnavailableError(credNameManagedIdentity, err.Error())
}
return tk, err
}
if c.msiType == msiTypeIMDS {
switch resp.StatusCode {
case http.StatusBadRequest:
if id != nil {
// return authenticationFailedError, halting any encompassing credential chain,
// because the explicit user-assigned identity implies the developer expected this to work
return azcore.AccessToken{}, newAuthenticationFailedError(credNameManagedIdentity, "the requested identity isn't assigned to this resource", resp)
}
msg := "failed to authenticate a system assigned identity"
@@ -276,6 +278,13 @@ func (c *managedIdentityClient) authenticate(ctx context.Context, id ManagedIDKi
return azcore.AccessToken{}, newCredentialUnavailableError(credNameManagedIdentity, fmt.Sprintf("unexpected response %q", string(body)))
}
}
if c.chained {
// the response may be from something other than IMDS, for example a proxy returning
// 404. Return credentialUnavailableError so credential chains continue to their
// next credential, include the response in the error message to help debugging
err = newAuthenticationFailedError(credNameManagedIdentity, "", resp)
return azcore.AccessToken{}, newCredentialUnavailableError(credNameManagedIdentity, err.Error())
}
}
return azcore.AccessToken{}, newAuthenticationFailedError(credNameManagedIdentity, "", resp)
@@ -290,7 +299,7 @@ func (c *managedIdentityClient) createAccessToken(res *http.Response) (azcore.Ac
ExpiresOn interface{} `json:"expires_on,omitempty"` // the value returned in this field varies between a number and a date string
}{}
if err := azruntime.UnmarshalAsJSON(res, &value); err != nil {
return azcore.AccessToken{}, fmt.Errorf("internal AccessToken: %v", err)
return azcore.AccessToken{}, newAuthenticationFailedError(credNameManagedIdentity, "Unexpected response content", res)
}
if value.ExpiresIn != "" {
expiresIn, err := json.Number(value.ExpiresIn).Int64()

View File

@@ -154,12 +154,7 @@ func (p *publicClient) GetToken(ctx context.Context, tro policy.TokenRequestOpti
if p.opts.DisableAutomaticAuthentication {
return azcore.AccessToken{}, newAuthenticationRequiredError(p.name, tro)
}
at, err := p.reqToken(ctx, client, tro)
if err == nil {
msg := fmt.Sprintf("%s.GetToken() acquired a token for scope %q", p.name, strings.Join(ar.GrantedScopes, ", "))
log.Write(EventAuthentication, msg)
}
return at, err
return p.reqToken(ctx, client, tro)
}
// reqToken requests a token from the MSAL public client. It's separate from GetToken() to enable Authenticate() to bypass the cache.
@@ -242,6 +237,8 @@ func (p *publicClient) newMSALClient(enableCAE bool) (msalPublicClient, error) {
func (p *publicClient) token(ar public.AuthResult, err error) (azcore.AccessToken, error) {
if err == nil {
msg := fmt.Sprintf(scopeLogFmt, p.name, strings.Join(ar.GrantedScopes, ", "))
log.Write(EventAuthentication, msg)
p.record, err = newAuthenticationRecord(ar)
} else {
err = newAuthenticationFailedErrorFromMSAL(p.name, err)

View File

@@ -7,6 +7,10 @@ param (
[hashtable] $AdditionalParameters = @{},
[hashtable] $DeploymentOutputs,
[Parameter(Mandatory = $true)]
[ValidateNotNullOrEmpty()]
[string] $SubscriptionId,
[Parameter(ParameterSetName = 'Provisioner', Mandatory = $true)]
[ValidateNotNullOrEmpty()]
[string] $TenantId,
@@ -15,6 +19,10 @@ param (
[ValidatePattern('^[0-9a-f]{8}(-[0-9a-f]{4}){3}-[0-9a-f]{12}$')]
[string] $TestApplicationId,
[Parameter(Mandatory = $true)]
[ValidateNotNullOrEmpty()]
[string] $Environment,
# Captures any arguments from eng/New-TestResources.ps1 not declared here (no parameter errors).
[Parameter(ValueFromRemainingArguments = $true)]
$RemainingArguments
@@ -28,8 +36,9 @@ if ($CI) {
Write-Host "Skipping post-provisioning script because resources weren't deployed"
return
}
az login --federated-token $env:OIDC_TOKEN --service-principal -t $TenantId -u $TestApplicationId
az account set --subscription $DeploymentOutputs['AZIDENTITY_SUBSCRIPTION_ID']
az cloud set -n $Environment
az login --federated-token $env:ARM_OIDC_TOKEN --service-principal -t $TenantId -u $TestApplicationId
az account set --subscription $SubscriptionId
}
Write-Host "Building container"
@@ -62,6 +71,9 @@ $aciName = "azidentity-test"
az container create -g $rg -n $aciName --image $image `
--acr-identity $($DeploymentOutputs['AZIDENTITY_USER_ASSIGNED_IDENTITY']) `
--assign-identity [system] $($DeploymentOutputs['AZIDENTITY_USER_ASSIGNED_IDENTITY']) `
--cpu 1 `
--memory 1.0 `
--os-type Linux `
--role "Storage Blob Data Reader" `
--scope $($DeploymentOutputs['AZIDENTITY_STORAGE_ID']) `
-e AZIDENTITY_STORAGE_NAME=$($DeploymentOutputs['AZIDENTITY_STORAGE_NAME']) `

View File

@@ -14,5 +14,5 @@ const (
module = "github.com/Azure/azure-sdk-for-go/sdk/" + component
// Version is the semantic version (see http://semver.org) of this module.
version = "v1.8.0"
version = "v1.8.1"
)

View File

@@ -1,10 +1,23 @@
# Release History
## 1.6.0 (2025-01-23)
### Features Added
* Upgraded service version to `2025-01-05`.
## 1.6.0-beta.1 (2025-01-13)
### Features Added
* Added permissions & resourcetype parameters in listblob response.
* Added BlobProperties field in BlobPrefix definition in listblob response.
### Bugs Fixed
* Fix FilterBlob API if Query contains a space character. Fixes [#23546](https://github.com/Azure/azure-sdk-for-go/issues/23546)
## 1.5.0 (2024-11-13)
### Features Added
* Fix compareHeaders custom sorting algorithm for String To Sign.
* Added permissions & resourcetype parameters in listblob response.
## 1.5.0-beta.1 (2024-10-22)

View File

@@ -95,7 +95,7 @@ func (c *Client) ServiceClient() *service.Client {
}
// CreateContainer is a lifecycle method to creates a new container under the specified account.
// If the container with the same name already exists, a ResourceExistsError will be raised.
// If the container with the same name already exists, a ContainerAlreadyExists Error will be raised.
// This method returns a client with which to interact with the newly created container.
func (c *Client) CreateContainer(ctx context.Context, containerName string, o *CreateContainerOptions) (CreateContainerResponse, error) {
return c.svc.CreateContainer(ctx, containerName, o)

View File

@@ -8,5 +8,5 @@ package exported
const (
ModuleName = "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
ModuleVersion = "v1.5.0"
ModuleVersion = "v1.6.0"
)

View File

@@ -7,7 +7,7 @@ go: true
clear-output-folder: false
version: "^3.0.0"
license-header: MICROSOFT_MIT_NO_VERSION
input-file: "https://raw.githubusercontent.com/Azure/azure-rest-api-specs/f6f50c6388fd5836fa142384641b8353a99874ef/specification/storage/data-plane/Microsoft.BlobStorage/stable/2024-08-04/blob.json"
input-file: "https://raw.githubusercontent.com/Azure/azure-rest-api-specs/ae95eb6a4701d844bada7d1c4f5ecf4a7444e5b8/specification/storage/data-plane/Microsoft.BlobStorage/stable/2025-01-05/blob.json"
credential-scope: "https://storage.azure.com/.default"
output-folder: ../generated
file-prefix: "zz_"
@@ -22,6 +22,18 @@ export-clients: true
use: "@autorest/go@4.0.0-preview.65"
```
### Add a Properties field to the BlobPrefix definition
```yaml
directive:
- from: swagger-document
where: $.definitions
transform: >
$.BlobPrefix.properties["Properties"] = {
"type": "object",
"$ref": "#/definitions/BlobPropertiesInternal"
};
```
### Add Owner,Group,Permissions,Acl,ResourceType in ListBlob Response
``` yaml
directive:
@@ -29,19 +41,19 @@ directive:
where: $.definitions
transform: >
$.BlobPropertiesInternal.properties["Owner"] = {
"type" : "string",
"type" : "string",
};
$.BlobPropertiesInternal.properties["Group"] = {
"type" : "string",
"type" : "string",
};
$.BlobPropertiesInternal.properties["Permissions"] = {
"type" : "string",
"type" : "string",
};
$.BlobPropertiesInternal.properties["Acl"] = {
"type" : "string",
"type" : "string",
};
$.BlobPropertiesInternal.properties["ResourceType"] = {
"type" : "string",
"type" : "string",
};
```
@@ -55,7 +67,7 @@ directive:
$.items.enum.push("permissions");
```
### Updating service version to 2024-11-04
### Updating service version to 2025-01-05
```yaml
directive:
- from:
@@ -68,7 +80,7 @@ directive:
where: $
transform: >-
return $.
replaceAll(`[]string{"2024-08-04"}`, `[]string{ServiceVersion}`);
replaceAll(`[]string{"2021-12-02"}`, `[]string{ServiceVersion}`);
```
### Fix CRC Response Header in PutBlob response
@@ -410,11 +422,13 @@ directive:
``` yaml
directive:
- from: zz_service_client.go
where: $
transform: >-
return $.
replace(/req.Raw\(\).URL.RawQuery \= reqQP.Encode\(\)/, `req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)`)
- from:
- zz_service_client.go
- zz_container_client.go
where: $
transform: >-
return $.
replace(/req.Raw\(\).URL.RawQuery \= reqQP.Encode\(\)/g, `req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)`);
```
### Change `where` parameter in blob filtering to be required

View File

@@ -6,4 +6,4 @@
package generated
const ServiceVersion = "2024-11-04"
const ServiceVersion = "2025-01-05"

View File

@@ -29,7 +29,7 @@ type AppendBlobClient struct {
// AppendBlob. Append Block is supported only on version 2015-02-21 version or later.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - contentLength - The length of the request.
// - body - Initial data
// - options - AppendBlobClientAppendBlockOptions contains the optional parameters for the AppendBlobClient.AppendBlock method.
@@ -116,7 +116,13 @@ func (client *AppendBlobClient) appendBlockCreateRequest(ctx context.Context, co
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
if options != nil && options.StructuredBodyType != nil {
req.Raw().Header["x-ms-structured-body"] = []string{*options.StructuredBodyType}
}
if options != nil && options.StructuredContentLength != nil {
req.Raw().Header["x-ms-structured-content-length"] = []string{strconv.FormatInt(*options.StructuredContentLength, 10)}
}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
if err := req.SetBody(body, "application/octet-stream"); err != nil {
return nil, err
}
@@ -187,6 +193,9 @@ func (client *AppendBlobClient) appendBlockHandleResponse(resp *http.Response) (
if val := resp.Header.Get("x-ms-request-id"); val != "" {
result.RequestID = &val
}
if val := resp.Header.Get("x-ms-structured-body"); val != "" {
result.StructuredBodyType = &val
}
if val := resp.Header.Get("x-ms-version"); val != "" {
result.Version = &val
}
@@ -198,7 +207,7 @@ func (client *AppendBlobClient) appendBlockHandleResponse(resp *http.Response) (
// created with x-ms-blob-type set to AppendBlob. Append Block is supported only on version 2015-02-21 version or later.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - sourceURL - Specify a URL to the copy source.
// - contentLength - The length of the request.
// - options - AppendBlobClientAppendBlockFromURLOptions contains the optional parameters for the AppendBlobClient.AppendBlockFromURL
@@ -310,7 +319,7 @@ func (client *AppendBlobClient) appendBlockFromURLCreateRequest(ctx context.Cont
if options != nil && options.SourceRange != nil {
req.Raw().Header["x-ms-source-range"] = []string{*options.SourceRange}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -384,7 +393,7 @@ func (client *AppendBlobClient) appendBlockFromURLHandleResponse(resp *http.Resp
// Create - The Create Append Blob operation creates a new append blob.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - contentLength - The length of the request.
// - options - AppendBlobClientCreateOptions contains the optional parameters for the AppendBlobClient.Create method.
// - BlobHTTPHeaders - BlobHTTPHeaders contains a group of parameters for the BlobClient.SetHTTPHeaders method.
@@ -494,7 +503,7 @@ func (client *AppendBlobClient) createCreateRequest(ctx context.Context, content
if options != nil && options.BlobTagsString != nil {
req.Raw().Header["x-ms-tags"] = []string{*options.BlobTagsString}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -557,7 +566,7 @@ func (client *AppendBlobClient) createHandleResponse(resp *http.Response) (Appen
// or later.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - AppendBlobClientSealOptions contains the optional parameters for the AppendBlobClient.Seal method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
// - ModifiedAccessConditions - ModifiedAccessConditions contains a group of parameters for the ContainerClient.Delete method.
@@ -615,7 +624,7 @@ func (client *AppendBlobClient) sealCreateRequest(ctx context.Context, options *
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}

View File

@@ -29,7 +29,7 @@ type BlobClient struct {
// blob with zero length and full metadata.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - copyID - The copy identifier provided in the x-ms-copy-id header of the original Copy Blob operation.
// - options - BlobClientAbortCopyFromURLOptions contains the optional parameters for the BlobClient.AbortCopyFromURL method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
@@ -72,7 +72,7 @@ func (client *BlobClient) abortCopyFromURLCreateRequest(ctx context.Context, cop
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -101,7 +101,7 @@ func (client *BlobClient) abortCopyFromURLHandleResponse(resp *http.Response) (B
// AcquireLease - [Update] The Lease Blob operation establishes and manages a lock on a blob for write and delete operations
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - duration - Specifies the duration of the lease, in seconds, or negative one (-1) for a lease that never expires. A non-infinite
// lease can be between 15 and 60 seconds. A lease duration cannot be changed using
// renew or change.
@@ -161,7 +161,7 @@ func (client *BlobClient) acquireLeaseCreateRequest(ctx context.Context, duratio
if options != nil && options.ProposedLeaseID != nil {
req.Raw().Header["x-ms-proposed-lease-id"] = []string{*options.ProposedLeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -203,7 +203,7 @@ func (client *BlobClient) acquireLeaseHandleResponse(resp *http.Response) (BlobC
// BreakLease - [Update] The Lease Blob operation establishes and manages a lock on a blob for write and delete operations
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - BlobClientBreakLeaseOptions contains the optional parameters for the BlobClient.BreakLease method.
// - ModifiedAccessConditions - ModifiedAccessConditions contains a group of parameters for the ContainerClient.Delete method.
func (client *BlobClient) BreakLease(ctx context.Context, options *BlobClientBreakLeaseOptions, modifiedAccessConditions *ModifiedAccessConditions) (BlobClientBreakLeaseResponse, error) {
@@ -259,7 +259,7 @@ func (client *BlobClient) breakLeaseCreateRequest(ctx context.Context, options *
if options != nil && options.BreakPeriod != nil {
req.Raw().Header["x-ms-lease-break-period"] = []string{strconv.FormatInt(int64(*options.BreakPeriod), 10)}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -306,7 +306,7 @@ func (client *BlobClient) breakLeaseHandleResponse(resp *http.Response) (BlobCli
// ChangeLease - [Update] The Lease Blob operation establishes and manages a lock on a blob for write and delete operations
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - leaseID - Specifies the current lease ID on the resource.
// - proposedLeaseID - Proposed lease ID, in a GUID string format. The Blob service returns 400 (Invalid request) if the proposed
// lease ID is not in the correct format. See Guid Constructor (String) for a list of valid GUID
@@ -365,7 +365,7 @@ func (client *BlobClient) changeLeaseCreateRequest(ctx context.Context, leaseID
req.Raw().Header["x-ms-lease-action"] = []string{"change"}
req.Raw().Header["x-ms-lease-id"] = []string{leaseID}
req.Raw().Header["x-ms-proposed-lease-id"] = []string{proposedLeaseID}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -408,7 +408,7 @@ func (client *BlobClient) changeLeaseHandleResponse(resp *http.Response) (BlobCl
// until the copy is complete.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - copySource - Specifies the name of the source page blob snapshot. This value is a URL of up to 2 KB in length that specifies
// a page blob snapshot. The value should be URL-encoded as it would appear in a request
// URI. The source blob must either be public or must be authenticated via a shared access signature.
@@ -517,7 +517,7 @@ func (client *BlobClient) copyFromURLCreateRequest(ctx context.Context, copySour
if options != nil && options.BlobTagsString != nil {
req.Raw().Header["x-ms-tags"] = []string{*options.BlobTagsString}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -582,7 +582,7 @@ func (client *BlobClient) copyFromURLHandleResponse(resp *http.Response) (BlobCl
// CreateSnapshot - The Create Snapshot operation creates a read-only snapshot of a blob
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - BlobClientCreateSnapshotOptions contains the optional parameters for the BlobClient.CreateSnapshot method.
// - CPKInfo - CPKInfo contains a group of parameters for the BlobClient.Download method.
// - CPKScopeInfo - CPKScopeInfo contains a group of parameters for the BlobClient.SetMetadata method.
@@ -659,7 +659,7 @@ func (client *BlobClient) createSnapshotCreateRequest(ctx context.Context, optio
}
}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -721,7 +721,7 @@ func (client *BlobClient) createSnapshotHandleResponse(resp *http.Response) (Blo
// return an HTTP status code of 404 (ResourceNotFound).
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - BlobClientDeleteOptions contains the optional parameters for the BlobClient.Delete method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
// - ModifiedAccessConditions - ModifiedAccessConditions contains a group of parameters for the ContainerClient.Delete method.
@@ -788,7 +788,7 @@ func (client *BlobClient) deleteCreateRequest(ctx context.Context, options *Blob
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -817,7 +817,7 @@ func (client *BlobClient) deleteHandleResponse(resp *http.Response) (BlobClientD
// DeleteImmutabilityPolicy - The Delete Immutability Policy operation deletes the immutability policy on the blob
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - BlobClientDeleteImmutabilityPolicyOptions contains the optional parameters for the BlobClient.DeleteImmutabilityPolicy
// method.
func (client *BlobClient) DeleteImmutabilityPolicy(ctx context.Context, options *BlobClientDeleteImmutabilityPolicyOptions) (BlobClientDeleteImmutabilityPolicyResponse, error) {
@@ -846,15 +846,21 @@ func (client *BlobClient) deleteImmutabilityPolicyCreateRequest(ctx context.Cont
}
reqQP := req.Raw().URL.Query()
reqQP.Set("comp", "immutabilityPolicies")
if options != nil && options.Snapshot != nil {
reqQP.Set("snapshot", *options.Snapshot)
}
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
if options != nil && options.VersionID != nil {
reqQP.Set("versionid", *options.VersionID)
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -884,7 +890,7 @@ func (client *BlobClient) deleteImmutabilityPolicyHandleResponse(resp *http.Resp
// can also call Download to read a snapshot.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - BlobClientDownloadOptions contains the optional parameters for the BlobClient.Download method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
// - CPKInfo - CPKInfo contains a group of parameters for the BlobClient.Download method.
@@ -965,7 +971,10 @@ func (client *BlobClient) downloadCreateRequest(ctx context.Context, options *Bl
if options != nil && options.RangeGetContentMD5 != nil {
req.Raw().Header["x-ms-range-get-content-md5"] = []string{strconv.FormatBool(*options.RangeGetContentMD5)}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
if options != nil && options.StructuredBodyType != nil {
req.Raw().Header["x-ms-structured-body"] = []string{*options.StructuredBodyType}
}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -1173,6 +1182,16 @@ func (client *BlobClient) downloadHandleResponse(resp *http.Response) (BlobClien
if val := resp.Header.Get("x-ms-request-id"); val != "" {
result.RequestID = &val
}
if val := resp.Header.Get("x-ms-structured-body"); val != "" {
result.StructuredBodyType = &val
}
if val := resp.Header.Get("x-ms-structured-content-length"); val != "" {
structuredContentLength, err := strconv.ParseInt(val, 10, 64)
if err != nil {
return BlobClientDownloadResponse{}, err
}
result.StructuredContentLength = &structuredContentLength
}
if val := resp.Header.Get("x-ms-tag-count"); val != "" {
tagCount, err := strconv.ParseInt(val, 10, 64)
if err != nil {
@@ -1192,7 +1211,7 @@ func (client *BlobClient) downloadHandleResponse(resp *http.Response) (BlobClien
// GetAccountInfo - Returns the sku name and account kind
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - BlobClientGetAccountInfoOptions contains the optional parameters for the BlobClient.GetAccountInfo method.
func (client *BlobClient) GetAccountInfo(ctx context.Context, options *BlobClientGetAccountInfoOptions) (BlobClientGetAccountInfoResponse, error) {
var err error
@@ -1229,7 +1248,7 @@ func (client *BlobClient) getAccountInfoCreateRequest(ctx context.Context, optio
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -1272,7 +1291,7 @@ func (client *BlobClient) getAccountInfoHandleResponse(resp *http.Response) (Blo
// for the blob. It does not return the content of the blob.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - BlobClientGetPropertiesOptions contains the optional parameters for the BlobClient.GetProperties method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
// - CPKInfo - CPKInfo contains a group of parameters for the BlobClient.Download method.
@@ -1343,7 +1362,7 @@ func (client *BlobClient) getPropertiesCreateRequest(ctx context.Context, option
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -1590,7 +1609,7 @@ func (client *BlobClient) getPropertiesHandleResponse(resp *http.Response) (Blob
// GetTags - The Get Tags operation enables users to get the tags associated with a blob.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - BlobClientGetTagsOptions contains the optional parameters for the BlobClient.GetTags method.
// - ModifiedAccessConditions - ModifiedAccessConditions contains a group of parameters for the ContainerClient.Delete method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
@@ -1640,7 +1659,7 @@ func (client *BlobClient) getTagsCreateRequest(ctx context.Context, options *Blo
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -1672,7 +1691,7 @@ func (client *BlobClient) getTagsHandleResponse(resp *http.Response) (BlobClient
// Query - The Query operation enables users to select/project on blob data by providing simple query expressions.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - BlobClientQueryOptions contains the optional parameters for the BlobClient.Query method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
// - CPKInfo - CPKInfo contains a group of parameters for the BlobClient.Download method.
@@ -1742,7 +1761,7 @@ func (client *BlobClient) queryCreateRequest(ctx context.Context, options *BlobC
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
if options != nil && options.QueryRequest != nil {
if err := runtime.MarshalAsXML(req, *options.QueryRequest); err != nil {
return nil, err
@@ -1906,7 +1925,7 @@ func (client *BlobClient) queryHandleResponse(resp *http.Response) (BlobClientQu
// ReleaseLease - [Update] The Lease Blob operation establishes and manages a lock on a blob for write and delete operations
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - leaseID - Specifies the current lease ID on the resource.
// - options - BlobClientReleaseLeaseOptions contains the optional parameters for the BlobClient.ReleaseLease method.
// - ModifiedAccessConditions - ModifiedAccessConditions contains a group of parameters for the ContainerClient.Delete method.
@@ -1961,7 +1980,7 @@ func (client *BlobClient) releaseLeaseCreateRequest(ctx context.Context, leaseID
}
req.Raw().Header["x-ms-lease-action"] = []string{"release"}
req.Raw().Header["x-ms-lease-id"] = []string{leaseID}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -2000,7 +2019,7 @@ func (client *BlobClient) releaseLeaseHandleResponse(resp *http.Response) (BlobC
// RenewLease - [Update] The Lease Blob operation establishes and manages a lock on a blob for write and delete operations
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - leaseID - Specifies the current lease ID on the resource.
// - options - BlobClientRenewLeaseOptions contains the optional parameters for the BlobClient.RenewLease method.
// - ModifiedAccessConditions - ModifiedAccessConditions contains a group of parameters for the ContainerClient.Delete method.
@@ -2055,7 +2074,7 @@ func (client *BlobClient) renewLeaseCreateRequest(ctx context.Context, leaseID s
}
req.Raw().Header["x-ms-lease-action"] = []string{"renew"}
req.Raw().Header["x-ms-lease-id"] = []string{leaseID}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -2097,7 +2116,7 @@ func (client *BlobClient) renewLeaseHandleResponse(resp *http.Response) (BlobCli
// SetExpiry - Sets the time a blob will expire and be deleted.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - expiryOptions - Required. Indicates mode of the expiry time
// - options - BlobClientSetExpiryOptions contains the optional parameters for the BlobClient.SetExpiry method.
func (client *BlobClient) SetExpiry(ctx context.Context, expiryOptions ExpiryOptions, options *BlobClientSetExpiryOptions) (BlobClientSetExpiryResponse, error) {
@@ -2138,7 +2157,7 @@ func (client *BlobClient) setExpiryCreateRequest(ctx context.Context, expiryOpti
if options != nil && options.ExpiresOn != nil {
req.Raw().Header["x-ms-expiry-time"] = []string{*options.ExpiresOn}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -2177,7 +2196,7 @@ func (client *BlobClient) setExpiryHandleResponse(resp *http.Response) (BlobClie
// SetHTTPHeaders - The Set HTTP Headers operation sets system properties on the blob
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - BlobClientSetHTTPHeadersOptions contains the optional parameters for the BlobClient.SetHTTPHeaders method.
// - BlobHTTPHeaders - BlobHTTPHeaders contains a group of parameters for the BlobClient.SetHTTPHeaders method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
@@ -2252,7 +2271,7 @@ func (client *BlobClient) setHTTPHeadersCreateRequest(ctx context.Context, optio
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -2298,7 +2317,7 @@ func (client *BlobClient) setHTTPHeadersHandleResponse(resp *http.Response) (Blo
// SetImmutabilityPolicy - The Set Immutability Policy operation sets the immutability policy on the blob
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - BlobClientSetImmutabilityPolicyOptions contains the optional parameters for the BlobClient.SetImmutabilityPolicy
// method.
// - ModifiedAccessConditions - ModifiedAccessConditions contains a group of parameters for the ContainerClient.Delete method.
@@ -2328,9 +2347,15 @@ func (client *BlobClient) setImmutabilityPolicyCreateRequest(ctx context.Context
}
reqQP := req.Raw().URL.Query()
reqQP.Set("comp", "immutabilityPolicies")
if options != nil && options.Snapshot != nil {
reqQP.Set("snapshot", *options.Snapshot)
}
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
if options != nil && options.VersionID != nil {
reqQP.Set("versionid", *options.VersionID)
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().Header["Accept"] = []string{"application/xml"}
if modifiedAccessConditions != nil && modifiedAccessConditions.IfUnmodifiedSince != nil {
@@ -2345,7 +2370,7 @@ func (client *BlobClient) setImmutabilityPolicyCreateRequest(ctx context.Context
if options != nil && options.ImmutabilityPolicyExpiry != nil {
req.Raw().Header["x-ms-immutability-policy-until-date"] = []string{(*options.ImmutabilityPolicyExpiry).In(gmt).Format(time.RFC1123)}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -2384,7 +2409,7 @@ func (client *BlobClient) setImmutabilityPolicyHandleResponse(resp *http.Respons
// SetLegalHold - The Set Legal Hold operation sets a legal hold on the blob.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - legalHold - Specified if a legal hold should be set on the blob.
// - options - BlobClientSetLegalHoldOptions contains the optional parameters for the BlobClient.SetLegalHold method.
func (client *BlobClient) SetLegalHold(ctx context.Context, legalHold bool, options *BlobClientSetLegalHoldOptions) (BlobClientSetLegalHoldResponse, error) {
@@ -2413,16 +2438,22 @@ func (client *BlobClient) setLegalHoldCreateRequest(ctx context.Context, legalHo
}
reqQP := req.Raw().URL.Query()
reqQP.Set("comp", "legalhold")
if options != nil && options.Snapshot != nil {
reqQP.Set("snapshot", *options.Snapshot)
}
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
if options != nil && options.VersionID != nil {
reqQP.Set("versionid", *options.VersionID)
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-legal-hold"] = []string{strconv.FormatBool(legalHold)}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -2459,7 +2490,7 @@ func (client *BlobClient) setLegalHoldHandleResponse(resp *http.Response) (BlobC
// pairs
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - BlobClientSetMetadataOptions contains the optional parameters for the BlobClient.SetMetadata method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
// - CPKInfo - CPKInfo contains a group of parameters for the BlobClient.Download method.
@@ -2536,7 +2567,7 @@ func (client *BlobClient) setMetadataCreateRequest(ctx context.Context, options
}
}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -2591,7 +2622,7 @@ func (client *BlobClient) setMetadataHandleResponse(resp *http.Response) (BlobCl
// SetTags - The Set Tags operation enables users to set tags on a blob.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - tags - Blob tags
// - options - BlobClientSetTagsOptions contains the optional parameters for the BlobClient.SetTags method.
// - ModifiedAccessConditions - ModifiedAccessConditions contains a group of parameters for the ContainerClient.Delete method.
@@ -2645,7 +2676,7 @@ func (client *BlobClient) setTagsCreateRequest(ctx context.Context, tags BlobTag
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
if err := runtime.MarshalAsXML(req, tags); err != nil {
return nil, err
}
@@ -2680,7 +2711,7 @@ func (client *BlobClient) setTagsHandleResponse(resp *http.Response) (BlobClient
// storage type. This operation does not update the blob's ETag.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - tier - Indicates the tier to be set on the blob.
// - options - BlobClientSetTierOptions contains the optional parameters for the BlobClient.SetTier method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
@@ -2735,7 +2766,7 @@ func (client *BlobClient) setTierCreateRequest(ctx context.Context, tier AccessT
if options != nil && options.RehydratePriority != nil {
req.Raw().Header["x-ms-rehydrate-priority"] = []string{string(*options.RehydratePriority)}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -2757,7 +2788,7 @@ func (client *BlobClient) setTierHandleResponse(resp *http.Response) (BlobClient
// StartCopyFromURL - The Start Copy From URL operation copies a blob or an internet resource to a new blob.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - copySource - Specifies the name of the source page blob snapshot. This value is a URL of up to 2 KB in length that specifies
// a page blob snapshot. The value should be URL-encoded as it would appear in a request
// URI. The source blob must either be public or must be authenticated via a shared access signature.
@@ -2861,7 +2892,7 @@ func (client *BlobClient) startCopyFromURLCreateRequest(ctx context.Context, cop
if options != nil && options.BlobTagsString != nil {
req.Raw().Header["x-ms-tags"] = []string{*options.BlobTagsString}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -2909,7 +2940,7 @@ func (client *BlobClient) startCopyFromURLHandleResponse(resp *http.Response) (B
// Undelete - Undelete a blob that was previously soft deleted
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - BlobClientUndeleteOptions contains the optional parameters for the BlobClient.Undelete method.
func (client *BlobClient) Undelete(ctx context.Context, options *BlobClientUndeleteOptions) (BlobClientUndeleteResponse, error) {
var err error
@@ -2945,7 +2976,7 @@ func (client *BlobClient) undeleteCreateRequest(ctx context.Context, options *Bl
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}

View File

@@ -33,7 +33,7 @@ type BlockBlobClient struct {
// belong to.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - blocks - Blob Blocks.
// - options - BlockBlobClientCommitBlockListOptions contains the optional parameters for the BlockBlobClient.CommitBlockList
// method.
@@ -152,7 +152,7 @@ func (client *BlockBlobClient) commitBlockListCreateRequest(ctx context.Context,
if options != nil && options.BlobTagsString != nil {
req.Raw().Header["x-ms-tags"] = []string{*options.BlobTagsString}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
if err := runtime.MarshalAsXML(req, blocks); err != nil {
return nil, err
}
@@ -224,7 +224,7 @@ func (client *BlockBlobClient) commitBlockListHandleResponse(resp *http.Response
// GetBlockList - The Get Block List operation retrieves the list of blocks that have been uploaded as part of a block blob
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - listType - Specifies whether to return the list of committed blocks, the list of uncommitted blocks, or both lists together.
// - options - BlockBlobClientGetBlockListOptions contains the optional parameters for the BlockBlobClient.GetBlockList method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
@@ -273,7 +273,7 @@ func (client *BlockBlobClient) getBlockListCreateRequest(ctx context.Context, li
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -329,7 +329,7 @@ func (client *BlockBlobClient) getBlockListHandleResponse(resp *http.Response) (
// Block from URL API in conjunction with Put Block List.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - contentLength - The length of the request.
// - copySource - Specifies the name of the source page blob snapshot. This value is a URL of up to 2 KB in length that specifies
// a page blob snapshot. The value should be URL-encoded as it would appear in a request
@@ -470,7 +470,7 @@ func (client *BlockBlobClient) putBlobFromURLCreateRequest(ctx context.Context,
if options != nil && options.BlobTagsString != nil {
req.Raw().Header["x-ms-tags"] = []string{*options.BlobTagsString}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -532,7 +532,7 @@ func (client *BlockBlobClient) putBlobFromURLHandleResponse(resp *http.Response)
// StageBlock - The Stage Block operation creates a new block to be committed as part of a blob
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - blockID - A valid Base64 string value that identifies the block. Prior to encoding, the string must be less than or equal
// to 64 bytes in size. For a given blob, the length of the value specified for the blockid
// parameter must be the same size for each block.
@@ -599,7 +599,13 @@ func (client *BlockBlobClient) stageBlockCreateRequest(ctx context.Context, bloc
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
if options != nil && options.StructuredBodyType != nil {
req.Raw().Header["x-ms-structured-body"] = []string{*options.StructuredBodyType}
}
if options != nil && options.StructuredContentLength != nil {
req.Raw().Header["x-ms-structured-content-length"] = []string{strconv.FormatInt(*options.StructuredContentLength, 10)}
}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
if err := req.SetBody(body, "application/octet-stream"); err != nil {
return nil, err
}
@@ -649,6 +655,9 @@ func (client *BlockBlobClient) stageBlockHandleResponse(resp *http.Response) (Bl
if val := resp.Header.Get("x-ms-request-id"); val != "" {
result.RequestID = &val
}
if val := resp.Header.Get("x-ms-structured-body"); val != "" {
result.StructuredBodyType = &val
}
if val := resp.Header.Get("x-ms-version"); val != "" {
result.Version = &val
}
@@ -659,7 +668,7 @@ func (client *BlockBlobClient) stageBlockHandleResponse(resp *http.Response) (Bl
// are read from a URL.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - blockID - A valid Base64 string value that identifies the block. Prior to encoding, the string must be less than or equal
// to 64 bytes in size. For a given blob, the length of the value specified for the blockid
// parameter must be the same size for each block.
@@ -748,7 +757,7 @@ func (client *BlockBlobClient) stageBlockFromURLCreateRequest(ctx context.Contex
if options != nil && options.SourceRange != nil {
req.Raw().Header["x-ms-source-range"] = []string{*options.SourceRange}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -807,7 +816,7 @@ func (client *BlockBlobClient) stageBlockFromURLHandleResponse(resp *http.Respon
// the content of a block blob, use the Put Block List operation.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - contentLength - The length of the request.
// - body - Initial data
// - options - BlockBlobClientUploadOptions contains the optional parameters for the BlockBlobClient.Upload method.
@@ -924,10 +933,16 @@ func (client *BlockBlobClient) uploadCreateRequest(ctx context.Context, contentL
}
}
}
if options != nil && options.StructuredBodyType != nil {
req.Raw().Header["x-ms-structured-body"] = []string{*options.StructuredBodyType}
}
if options != nil && options.StructuredContentLength != nil {
req.Raw().Header["x-ms-structured-content-length"] = []string{strconv.FormatInt(*options.StructuredContentLength, 10)}
}
if options != nil && options.BlobTagsString != nil {
req.Raw().Header["x-ms-tags"] = []string{*options.BlobTagsString}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
if err := req.SetBody(body, "application/octet-stream"); err != nil {
return nil, err
}
@@ -987,6 +1002,9 @@ func (client *BlockBlobClient) uploadHandleResponse(resp *http.Response) (BlockB
if val := resp.Header.Get("x-ms-request-id"); val != "" {
result.RequestID = &val
}
if val := resp.Header.Get("x-ms-structured-body"); val != "" {
result.StructuredBodyType = &val
}
if val := resp.Header.Get("x-ms-version"); val != "" {
result.Version = &val
}

View File

@@ -522,6 +522,7 @@ const (
StorageErrorCodeAuthorizationResourceTypeMismatch StorageErrorCode = "AuthorizationResourceTypeMismatch"
StorageErrorCodeAuthorizationServiceMismatch StorageErrorCode = "AuthorizationServiceMismatch"
StorageErrorCodeAuthorizationSourceIPMismatch StorageErrorCode = "AuthorizationSourceIPMismatch"
StorageErrorCodeBlobAccessTierNotSupportedForAccountType StorageErrorCode = "BlobAccessTierNotSupportedForAccountType"
StorageErrorCodeBlobAlreadyExists StorageErrorCode = "BlobAlreadyExists"
StorageErrorCodeBlobArchived StorageErrorCode = "BlobArchived"
StorageErrorCodeBlobBeingRehydrated StorageErrorCode = "BlobBeingRehydrated"
@@ -640,6 +641,7 @@ func PossibleStorageErrorCodeValues() []StorageErrorCode {
StorageErrorCodeAuthorizationResourceTypeMismatch,
StorageErrorCodeAuthorizationServiceMismatch,
StorageErrorCodeAuthorizationSourceIPMismatch,
StorageErrorCodeBlobAccessTierNotSupportedForAccountType,
StorageErrorCodeBlobAlreadyExists,
StorageErrorCodeBlobArchived,
StorageErrorCodeBlobBeingRehydrated,

View File

@@ -31,7 +31,7 @@ type ContainerClient struct {
// to 60 seconds, or can be infinite
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - duration - Specifies the duration of the lease, in seconds, or negative one (-1) for a lease that never expires. A non-infinite
// lease can be between 15 and 60 seconds. A lease duration cannot be changed using
// renew or change.
@@ -67,7 +67,7 @@ func (client *ContainerClient) acquireLeaseCreateRequest(ctx context.Context, du
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if modifiedAccessConditions != nil && modifiedAccessConditions.IfModifiedSince != nil {
req.Raw().Header["If-Modified-Since"] = []string{(*modifiedAccessConditions.IfModifiedSince).In(gmt).Format(time.RFC1123)}
@@ -83,7 +83,7 @@ func (client *ContainerClient) acquireLeaseCreateRequest(ctx context.Context, du
if options != nil && options.ProposedLeaseID != nil {
req.Raw().Header["x-ms-proposed-lease-id"] = []string{*options.ProposedLeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -126,7 +126,7 @@ func (client *ContainerClient) acquireLeaseHandleResponse(resp *http.Response) (
// to 60 seconds, or can be infinite
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - ContainerClientBreakLeaseOptions contains the optional parameters for the ContainerClient.BreakLease method.
// - ModifiedAccessConditions - ModifiedAccessConditions contains a group of parameters for the ContainerClient.Delete method.
func (client *ContainerClient) BreakLease(ctx context.Context, options *ContainerClientBreakLeaseOptions, modifiedAccessConditions *ModifiedAccessConditions) (ContainerClientBreakLeaseResponse, error) {
@@ -159,7 +159,7 @@ func (client *ContainerClient) breakLeaseCreateRequest(ctx context.Context, opti
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if modifiedAccessConditions != nil && modifiedAccessConditions.IfModifiedSince != nil {
req.Raw().Header["If-Modified-Since"] = []string{(*modifiedAccessConditions.IfModifiedSince).In(gmt).Format(time.RFC1123)}
@@ -174,7 +174,7 @@ func (client *ContainerClient) breakLeaseCreateRequest(ctx context.Context, opti
if options != nil && options.BreakPeriod != nil {
req.Raw().Header["x-ms-lease-break-period"] = []string{strconv.FormatInt(int64(*options.BreakPeriod), 10)}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -222,7 +222,7 @@ func (client *ContainerClient) breakLeaseHandleResponse(resp *http.Response) (Co
// to 60 seconds, or can be infinite
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - leaseID - Specifies the current lease ID on the resource.
// - proposedLeaseID - Proposed lease ID, in a GUID string format. The Blob service returns 400 (Invalid request) if the proposed
// lease ID is not in the correct format. See Guid Constructor (String) for a list of valid GUID
@@ -259,7 +259,7 @@ func (client *ContainerClient) changeLeaseCreateRequest(ctx context.Context, lea
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if modifiedAccessConditions != nil && modifiedAccessConditions.IfModifiedSince != nil {
req.Raw().Header["If-Modified-Since"] = []string{(*modifiedAccessConditions.IfModifiedSince).In(gmt).Format(time.RFC1123)}
@@ -273,7 +273,7 @@ func (client *ContainerClient) changeLeaseCreateRequest(ctx context.Context, lea
req.Raw().Header["x-ms-lease-action"] = []string{"change"}
req.Raw().Header["x-ms-lease-id"] = []string{leaseID}
req.Raw().Header["x-ms-proposed-lease-id"] = []string{proposedLeaseID}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -316,7 +316,7 @@ func (client *ContainerClient) changeLeaseHandleResponse(resp *http.Response) (C
// fails
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - ContainerClientCreateOptions contains the optional parameters for the ContainerClient.Create method.
// - ContainerCPKScopeInfo - ContainerCPKScopeInfo contains a group of parameters for the ContainerClient.Create method.
func (client *ContainerClient) Create(ctx context.Context, options *ContainerClientCreateOptions, containerCPKScopeInfo *ContainerCPKScopeInfo) (ContainerClientCreateResponse, error) {
@@ -348,7 +348,7 @@ func (client *ContainerClient) createCreateRequest(ctx context.Context, options
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.Access != nil {
req.Raw().Header["x-ms-blob-public-access"] = []string{string(*options.Access)}
@@ -369,7 +369,7 @@ func (client *ContainerClient) createCreateRequest(ctx context.Context, options
}
}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -409,7 +409,7 @@ func (client *ContainerClient) createHandleResponse(resp *http.Response) (Contai
// deleted during garbage collection
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - ContainerClientDeleteOptions contains the optional parameters for the ContainerClient.Delete method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
// - ModifiedAccessConditions - ModifiedAccessConditions contains a group of parameters for the ContainerClient.Delete method.
@@ -442,7 +442,7 @@ func (client *ContainerClient) deleteCreateRequest(ctx context.Context, options
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if modifiedAccessConditions != nil && modifiedAccessConditions.IfModifiedSince != nil {
req.Raw().Header["If-Modified-Since"] = []string{(*modifiedAccessConditions.IfModifiedSince).In(gmt).Format(time.RFC1123)}
@@ -456,7 +456,7 @@ func (client *ContainerClient) deleteCreateRequest(ctx context.Context, options
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -486,7 +486,7 @@ func (client *ContainerClient) deleteHandleResponse(resp *http.Response) (Contai
// Filter blobs searches within the given container.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - where - Filters the results to return only to return only blobs whose tags match the specified expression.
// - options - ContainerClientFilterBlobsOptions contains the optional parameters for the ContainerClient.FilterBlobs method.
func (client *ContainerClient) FilterBlobs(ctx context.Context, where string, options *ContainerClientFilterBlobsOptions) (ContainerClientFilterBlobsResponse, error) {
@@ -529,12 +529,12 @@ func (client *ContainerClient) filterBlobsCreateRequest(ctx context.Context, whe
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
reqQP.Set("where", where)
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -567,7 +567,7 @@ func (client *ContainerClient) filterBlobsHandleResponse(resp *http.Response) (C
// be accessed publicly.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - ContainerClientGetAccessPolicyOptions contains the optional parameters for the ContainerClient.GetAccessPolicy
// method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
@@ -601,7 +601,7 @@ func (client *ContainerClient) getAccessPolicyCreateRequest(ctx context.Context,
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
@@ -609,7 +609,7 @@ func (client *ContainerClient) getAccessPolicyCreateRequest(ctx context.Context,
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -654,7 +654,7 @@ func (client *ContainerClient) getAccessPolicyHandleResponse(resp *http.Response
// GetAccountInfo - Returns the sku name and account kind
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - ContainerClientGetAccountInfoOptions contains the optional parameters for the ContainerClient.GetAccountInfo
// method.
func (client *ContainerClient) GetAccountInfo(ctx context.Context, options *ContainerClientGetAccountInfoOptions) (ContainerClientGetAccountInfoResponse, error) {
@@ -687,12 +687,12 @@ func (client *ContainerClient) getAccountInfoCreateRequest(ctx context.Context,
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -735,7 +735,7 @@ func (client *ContainerClient) getAccountInfoHandleResponse(resp *http.Response)
// does not include the container's list of blobs
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - ContainerClientGetPropertiesOptions contains the optional parameters for the ContainerClient.GetProperties method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
func (client *ContainerClient) GetProperties(ctx context.Context, options *ContainerClientGetPropertiesOptions, leaseAccessConditions *LeaseAccessConditions) (ContainerClientGetPropertiesResponse, error) {
@@ -767,7 +767,7 @@ func (client *ContainerClient) getPropertiesCreateRequest(ctx context.Context, o
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
@@ -775,7 +775,7 @@ func (client *ContainerClient) getPropertiesCreateRequest(ctx context.Context, o
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -864,7 +864,7 @@ func (client *ContainerClient) getPropertiesHandleResponse(resp *http.Response)
// NewListBlobFlatSegmentPager - [Update] The List Blobs operation returns a list of the blobs under the specified container
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - ContainerClientListBlobFlatSegmentOptions contains the optional parameters for the ContainerClient.NewListBlobFlatSegmentPager
// method.
//
@@ -892,12 +892,12 @@ func (client *ContainerClient) ListBlobFlatSegmentCreateRequest(ctx context.Cont
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -931,7 +931,7 @@ func (client *ContainerClient) ListBlobFlatSegmentHandleResponse(resp *http.Resp
// NewListBlobHierarchySegmentPager - [Update] The List Blobs operation returns a list of the blobs under the specified container
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - delimiter - When the request includes this parameter, the operation returns a BlobPrefix element in the response body that
// acts as a placeholder for all blobs whose names begin with the same substring up to the
// appearance of the delimiter character. The delimiter may be a single character or a string.
@@ -983,12 +983,12 @@ func (client *ContainerClient) ListBlobHierarchySegmentCreateRequest(ctx context
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -1024,7 +1024,7 @@ func (client *ContainerClient) ListBlobHierarchySegmentHandleResponse(resp *http
// to 60 seconds, or can be infinite
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - leaseID - Specifies the current lease ID on the resource.
// - options - ContainerClientReleaseLeaseOptions contains the optional parameters for the ContainerClient.ReleaseLease method.
// - ModifiedAccessConditions - ModifiedAccessConditions contains a group of parameters for the ContainerClient.Delete method.
@@ -1058,7 +1058,7 @@ func (client *ContainerClient) releaseLeaseCreateRequest(ctx context.Context, le
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if modifiedAccessConditions != nil && modifiedAccessConditions.IfModifiedSince != nil {
req.Raw().Header["If-Modified-Since"] = []string{(*modifiedAccessConditions.IfModifiedSince).In(gmt).Format(time.RFC1123)}
@@ -1071,7 +1071,7 @@ func (client *ContainerClient) releaseLeaseCreateRequest(ctx context.Context, le
}
req.Raw().Header["x-ms-lease-action"] = []string{"release"}
req.Raw().Header["x-ms-lease-id"] = []string{leaseID}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -1110,7 +1110,7 @@ func (client *ContainerClient) releaseLeaseHandleResponse(resp *http.Response) (
// Rename - Renames an existing container.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - sourceContainerName - Required. Specifies the name of the container to rename.
// - options - ContainerClientRenameOptions contains the optional parameters for the ContainerClient.Rename method.
func (client *ContainerClient) Rename(ctx context.Context, sourceContainerName string, options *ContainerClientRenameOptions) (ContainerClientRenameResponse, error) {
@@ -1143,7 +1143,7 @@ func (client *ContainerClient) renameCreateRequest(ctx context.Context, sourceCo
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
@@ -1152,7 +1152,7 @@ func (client *ContainerClient) renameCreateRequest(ctx context.Context, sourceCo
if options != nil && options.SourceLeaseID != nil {
req.Raw().Header["x-ms-source-lease-id"] = []string{*options.SourceLeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -1182,7 +1182,7 @@ func (client *ContainerClient) renameHandleResponse(resp *http.Response) (Contai
// to 60 seconds, or can be infinite
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - leaseID - Specifies the current lease ID on the resource.
// - options - ContainerClientRenewLeaseOptions contains the optional parameters for the ContainerClient.RenewLease method.
// - ModifiedAccessConditions - ModifiedAccessConditions contains a group of parameters for the ContainerClient.Delete method.
@@ -1216,7 +1216,7 @@ func (client *ContainerClient) renewLeaseCreateRequest(ctx context.Context, leas
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if modifiedAccessConditions != nil && modifiedAccessConditions.IfModifiedSince != nil {
req.Raw().Header["If-Modified-Since"] = []string{(*modifiedAccessConditions.IfModifiedSince).In(gmt).Format(time.RFC1123)}
@@ -1229,7 +1229,7 @@ func (client *ContainerClient) renewLeaseCreateRequest(ctx context.Context, leas
}
req.Raw().Header["x-ms-lease-action"] = []string{"renew"}
req.Raw().Header["x-ms-lease-id"] = []string{leaseID}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -1271,7 +1271,7 @@ func (client *ContainerClient) renewLeaseHandleResponse(resp *http.Response) (Co
// Restore - Restores a previously-deleted container.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - ContainerClientRestoreOptions contains the optional parameters for the ContainerClient.Restore method.
func (client *ContainerClient) Restore(ctx context.Context, options *ContainerClientRestoreOptions) (ContainerClientRestoreResponse, error) {
var err error
@@ -1303,7 +1303,7 @@ func (client *ContainerClient) restoreCreateRequest(ctx context.Context, options
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
@@ -1314,7 +1314,7 @@ func (client *ContainerClient) restoreCreateRequest(ctx context.Context, options
if options != nil && options.DeletedContainerVersion != nil {
req.Raw().Header["x-ms-deleted-container-version"] = []string{*options.DeletedContainerVersion}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -1344,7 +1344,7 @@ func (client *ContainerClient) restoreHandleResponse(resp *http.Response) (Conta
// may be accessed publicly.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - containerACL - the acls for the container
// - options - ContainerClientSetAccessPolicyOptions contains the optional parameters for the ContainerClient.SetAccessPolicy
// method.
@@ -1380,7 +1380,7 @@ func (client *ContainerClient) setAccessPolicyCreateRequest(ctx context.Context,
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if modifiedAccessConditions != nil && modifiedAccessConditions.IfModifiedSince != nil {
req.Raw().Header["If-Modified-Since"] = []string{(*modifiedAccessConditions.IfModifiedSince).In(gmt).Format(time.RFC1123)}
@@ -1397,7 +1397,7 @@ func (client *ContainerClient) setAccessPolicyCreateRequest(ctx context.Context,
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
type wrapper struct {
XMLName xml.Name `xml:"SignedIdentifiers"`
ContainerACL *[]*SignedIdentifier `xml:"SignedIdentifier"`
@@ -1443,7 +1443,7 @@ func (client *ContainerClient) setAccessPolicyHandleResponse(resp *http.Response
// SetMetadata - operation sets one or more user-defined name-value pairs for the specified container.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - ContainerClientSetMetadataOptions contains the optional parameters for the ContainerClient.SetMetadata method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
// - ModifiedAccessConditions - ModifiedAccessConditions contains a group of parameters for the ContainerClient.Delete method.
@@ -1477,7 +1477,7 @@ func (client *ContainerClient) setMetadataCreateRequest(ctx context.Context, opt
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if modifiedAccessConditions != nil && modifiedAccessConditions.IfModifiedSince != nil {
req.Raw().Header["If-Modified-Since"] = []string{(*modifiedAccessConditions.IfModifiedSince).In(gmt).Format(time.RFC1123)}
@@ -1495,7 +1495,7 @@ func (client *ContainerClient) setMetadataCreateRequest(ctx context.Context, opt
}
}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -1534,7 +1534,7 @@ func (client *ContainerClient) setMetadataHandleResponse(resp *http.Response) (C
// SubmitBatch - The Batch operation allows multiple API calls to be embedded into a single HTTP request.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - contentLength - The length of the request.
// - multipartContentType - Required. The value of this header must be multipart/mixed with a batch boundary. Example header
// value: multipart/mixed; boundary=batch_
@@ -1570,7 +1570,7 @@ func (client *ContainerClient) submitBatchCreateRequest(ctx context.Context, con
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
runtime.SkipBodyDownload(req)
req.Raw().Header["Accept"] = []string{"application/xml"}
req.Raw().Header["Content-Length"] = []string{strconv.FormatInt(contentLength, 10)}
@@ -1578,7 +1578,7 @@ func (client *ContainerClient) submitBatchCreateRequest(ctx context.Context, con
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
if err := req.SetBody(body, multipartContentType); err != nil {
return nil, err
}

View File

@@ -86,6 +86,9 @@ type BlobName struct {
type BlobPrefix struct {
// REQUIRED
Name *string `xml:"Name"`
// Properties of a blob
Properties *BlobProperties `xml:"Properties"`
}
// BlobProperties - Properties of a blob

View File

@@ -43,6 +43,13 @@ type AppendBlobClientAppendBlockOptions struct {
// analytics logging is enabled.
RequestID *string
// Required if the request body is a structured message. Specifies the message schema version and properties.
StructuredBodyType *string
// Required if the request body is a structured message. Specifies the length of the blob/file content inside the message
// body. Will always be smaller than Content-Length.
StructuredContentLength *int64
// The timeout parameter is expressed in seconds. For more information, see Setting Timeouts for Blob Service Operations.
// [https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/setting-timeouts-for-blob-service-operations]
Timeout *int32
@@ -236,9 +243,18 @@ type BlobClientDeleteImmutabilityPolicyOptions struct {
// analytics logging is enabled.
RequestID *string
// The snapshot parameter is an opaque DateTime value that, when present, specifies the blob snapshot to retrieve. For more
// information on working with blob snapshots, see Creating a Snapshot of a Blob.
// [https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/creating-a-snapshot-of-a-blob]
Snapshot *string
// The timeout parameter is expressed in seconds. For more information, see Setting Timeouts for Blob Service Operations.
// [https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/setting-timeouts-for-blob-service-operations]
Timeout *int32
// The version id parameter is an opaque DateTime value that, when present, specifies the version of the blob to operate on.
// It's for service version 2019-10-10 and newer.
VersionID *string
}
// BlobClientDeleteOptions contains the optional parameters for the BlobClient.Delete method.
@@ -291,6 +307,10 @@ type BlobClientDownloadOptions struct {
// [https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/creating-a-snapshot-of-a-blob]
Snapshot *string
// Specifies the response content should be returned as a structured message and specifies the message schema version and
// properties.
StructuredBodyType *string
// The timeout parameter is expressed in seconds. For more information, see Setting Timeouts for Blob Service Operations.
// [https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/setting-timeouts-for-blob-service-operations]
Timeout *int32
@@ -429,9 +449,18 @@ type BlobClientSetImmutabilityPolicyOptions struct {
// analytics logging is enabled.
RequestID *string
// The snapshot parameter is an opaque DateTime value that, when present, specifies the blob snapshot to retrieve. For more
// information on working with blob snapshots, see Creating a Snapshot of a Blob.
// [https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/creating-a-snapshot-of-a-blob]
Snapshot *string
// The timeout parameter is expressed in seconds. For more information, see Setting Timeouts for Blob Service Operations.
// [https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/setting-timeouts-for-blob-service-operations]
Timeout *int32
// The version id parameter is an opaque DateTime value that, when present, specifies the version of the blob to operate on.
// It's for service version 2019-10-10 and newer.
VersionID *string
}
// BlobClientSetLegalHoldOptions contains the optional parameters for the BlobClient.SetLegalHold method.
@@ -440,9 +469,18 @@ type BlobClientSetLegalHoldOptions struct {
// analytics logging is enabled.
RequestID *string
// The snapshot parameter is an opaque DateTime value that, when present, specifies the blob snapshot to retrieve. For more
// information on working with blob snapshots, see Creating a Snapshot of a Blob.
// [https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/creating-a-snapshot-of-a-blob]
Snapshot *string
// The timeout parameter is expressed in seconds. For more information, see Setting Timeouts for Blob Service Operations.
// [https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/setting-timeouts-for-blob-service-operations]
Timeout *int32
// The version id parameter is an opaque DateTime value that, when present, specifies the version of the blob to operate on.
// It's for service version 2019-10-10 and newer.
VersionID *string
}
// BlobClientSetMetadataOptions contains the optional parameters for the BlobClient.SetMetadata method.
@@ -708,6 +746,13 @@ type BlockBlobClientStageBlockOptions struct {
// analytics logging is enabled.
RequestID *string
// Required if the request body is a structured message. Specifies the message schema version and properties.
StructuredBodyType *string
// Required if the request body is a structured message. Specifies the length of the blob/file content inside the message
// body. Will always be smaller than Content-Length.
StructuredContentLength *int64
// The timeout parameter is expressed in seconds. For more information, see Setting Timeouts for Blob Service Operations.
// [https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/setting-timeouts-for-blob-service-operations]
Timeout *int32
@@ -745,6 +790,13 @@ type BlockBlobClientUploadOptions struct {
// analytics logging is enabled.
RequestID *string
// Required if the request body is a structured message. Specifies the message schema version and properties.
StructuredBodyType *string
// Required if the request body is a structured message. Specifies the length of the blob/file content inside the message
// body. Will always be smaller than Content-Length.
StructuredContentLength *int64
// Optional. Indicates the tier to be set on the blob.
Tier *AccessTier
@@ -1316,6 +1368,13 @@ type PageBlobClientUploadPagesOptions struct {
// analytics logging is enabled.
RequestID *string
// Required if the request body is a structured message. Specifies the message schema version and properties.
StructuredBodyType *string
// Required if the request body is a structured message. Specifies the length of the blob/file content inside the message
// body. Will always be smaller than Content-Length.
StructuredContentLength *int64
// The timeout parameter is expressed in seconds. For more information, see Setting Timeouts for Blob Service Operations.
// [https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/setting-timeouts-for-blob-service-operations]
Timeout *int32

View File

@@ -27,7 +27,7 @@ type PageBlobClient struct {
// ClearPages - The Clear Pages operation clears a set of pages from a page blob
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - contentLength - The length of the request.
// - options - PageBlobClientClearPagesOptions contains the optional parameters for the PageBlobClient.ClearPages method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
@@ -114,7 +114,7 @@ func (client *PageBlobClient) clearPagesCreateRequest(ctx context.Context, conte
if options != nil && options.Range != nil {
req.Raw().Header["x-ms-range"] = []string{*options.Range}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -178,7 +178,7 @@ func (client *PageBlobClient) clearPagesHandleResponse(resp *http.Response) (Pag
// 2016-05-31.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - copySource - Specifies the name of the source page blob snapshot. This value is a URL of up to 2 KB in length that specifies
// a page blob snapshot. The value should be URL-encoded as it would appear in a request
// URI. The source blob must either be public or must be authenticated via a shared access signature.
@@ -235,7 +235,7 @@ func (client *PageBlobClient) copyIncrementalCreateRequest(ctx context.Context,
if modifiedAccessConditions != nil && modifiedAccessConditions.IfTags != nil {
req.Raw().Header["x-ms-if-tags"] = []string{*modifiedAccessConditions.IfTags}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -280,7 +280,7 @@ func (client *PageBlobClient) copyIncrementalHandleResponse(resp *http.Response)
// Create - The Create operation creates a new page blob.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - contentLength - The length of the request.
// - blobContentLength - This header specifies the maximum size for the page blob, up to 1 TB. The page blob size must be aligned
// to a 512-byte boundary.
@@ -399,7 +399,7 @@ func (client *PageBlobClient) createCreateRequest(ctx context.Context, contentLe
if options != nil && options.BlobTagsString != nil {
req.Raw().Header["x-ms-tags"] = []string{*options.BlobTagsString}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -461,7 +461,7 @@ func (client *PageBlobClient) createHandleResponse(resp *http.Response) (PageBlo
// NewGetPageRangesPager - The Get Page Ranges operation returns the list of valid page ranges for a page blob or snapshot
// of a page blob
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - PageBlobClientGetPageRangesOptions contains the optional parameters for the PageBlobClient.NewGetPageRangesPager
// method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
@@ -533,7 +533,7 @@ func (client *PageBlobClient) GetPageRangesCreateRequest(ctx context.Context, op
if options != nil && options.Range != nil {
req.Raw().Header["x-ms-range"] = []string{*options.Range}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -582,7 +582,7 @@ func (client *PageBlobClient) GetPageRangesHandleResponse(resp *http.Response) (
// NewGetPageRangesDiffPager - The Get Page Ranges Diff operation returns the list of valid page ranges for a page blob that
// were changed between target blob and previous snapshot.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - PageBlobClientGetPageRangesDiffOptions contains the optional parameters for the PageBlobClient.NewGetPageRangesDiffPager
// method.
// - LeaseAccessConditions - LeaseAccessConditions contains a group of parameters for the ContainerClient.GetProperties method.
@@ -660,7 +660,7 @@ func (client *PageBlobClient) GetPageRangesDiffCreateRequest(ctx context.Context
if options != nil && options.Range != nil {
req.Raw().Header["x-ms-range"] = []string{*options.Range}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -709,7 +709,7 @@ func (client *PageBlobClient) GetPageRangesDiffHandleResponse(resp *http.Respons
// Resize - Resize the Blob
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - blobContentLength - This header specifies the maximum size for the page blob, up to 1 TB. The page blob size must be aligned
// to a 512-byte boundary.
// - options - PageBlobClientResizeOptions contains the optional parameters for the PageBlobClient.Resize method.
@@ -782,7 +782,7 @@ func (client *PageBlobClient) resizeCreateRequest(ctx context.Context, blobConte
if leaseAccessConditions != nil && leaseAccessConditions.LeaseID != nil {
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -828,7 +828,7 @@ func (client *PageBlobClient) resizeHandleResponse(resp *http.Response) (PageBlo
// UpdateSequenceNumber - Update the sequence number of the blob
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - sequenceNumberAction - Required if the x-ms-blob-sequence-number header is set for the request. This property applies to
// page blobs only. This property indicates how the service should modify the blob's sequence number
// - options - PageBlobClientUpdateSequenceNumberOptions contains the optional parameters for the PageBlobClient.UpdateSequenceNumber
@@ -891,7 +891,7 @@ func (client *PageBlobClient) updateSequenceNumberCreateRequest(ctx context.Cont
req.Raw().Header["x-ms-lease-id"] = []string{*leaseAccessConditions.LeaseID}
}
req.Raw().Header["x-ms-sequence-number-action"] = []string{string(sequenceNumberAction)}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -937,7 +937,7 @@ func (client *PageBlobClient) updateSequenceNumberHandleResponse(resp *http.Resp
// UploadPages - The Upload Pages operation writes a range of pages to a page blob
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - contentLength - The length of the request.
// - body - Initial data
// - options - PageBlobClientUploadPagesOptions contains the optional parameters for the PageBlobClient.UploadPages method.
@@ -1031,7 +1031,13 @@ func (client *PageBlobClient) uploadPagesCreateRequest(ctx context.Context, cont
if options != nil && options.Range != nil {
req.Raw().Header["x-ms-range"] = []string{*options.Range}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
if options != nil && options.StructuredBodyType != nil {
req.Raw().Header["x-ms-structured-body"] = []string{*options.StructuredBodyType}
}
if options != nil && options.StructuredContentLength != nil {
req.Raw().Header["x-ms-structured-content-length"] = []string{strconv.FormatInt(*options.StructuredContentLength, 10)}
}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
if err := req.SetBody(body, "application/octet-stream"); err != nil {
return nil, err
}
@@ -1098,6 +1104,9 @@ func (client *PageBlobClient) uploadPagesHandleResponse(resp *http.Response) (Pa
if val := resp.Header.Get("x-ms-request-id"); val != "" {
result.RequestID = &val
}
if val := resp.Header.Get("x-ms-structured-body"); val != "" {
result.StructuredBodyType = &val
}
if val := resp.Header.Get("x-ms-version"); val != "" {
result.Version = &val
}
@@ -1108,7 +1117,7 @@ func (client *PageBlobClient) uploadPagesHandleResponse(resp *http.Response) (Pa
// a URL
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - sourceURL - Specify a URL to the copy source.
// - sourceRange - Bytes of source data in the specified range. The length of this range should match the ContentLength header
// and x-ms-range/Range destination range header.
@@ -1224,7 +1233,7 @@ func (client *PageBlobClient) uploadPagesFromURLCreateRequest(ctx context.Contex
req.Raw().Header["x-ms-source-if-unmodified-since"] = []string{(*sourceModifiedAccessConditions.SourceIfUnmodifiedSince).In(gmt).Format(time.RFC1123)}
}
req.Raw().Header["x-ms-source-range"] = []string{sourceRange}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}

View File

@@ -88,6 +88,9 @@ type AppendBlobClientAppendBlockResponse struct {
// RequestID contains the information returned from the x-ms-request-id header response.
RequestID *string
// StructuredBodyType contains the information returned from the x-ms-structured-body header response.
StructuredBodyType *string
// Version contains the information returned from the x-ms-version header response.
Version *string
}
@@ -469,6 +472,12 @@ type BlobClientDownloadResponse struct {
// RequestID contains the information returned from the x-ms-request-id header response.
RequestID *string
// StructuredBodyType contains the information returned from the x-ms-structured-body header response.
StructuredBodyType *string
// StructuredContentLength contains the information returned from the x-ms-structured-content-length header response.
StructuredContentLength *int64
// TagCount contains the information returned from the x-ms-tag-count header response.
TagCount *int64
@@ -1170,6 +1179,9 @@ type BlockBlobClientStageBlockResponse struct {
// RequestID contains the information returned from the x-ms-request-id header response.
RequestID *string
// StructuredBodyType contains the information returned from the x-ms-structured-body header response.
StructuredBodyType *string
// Version contains the information returned from the x-ms-version header response.
Version *string
}
@@ -1206,6 +1218,9 @@ type BlockBlobClientUploadResponse struct {
// RequestID contains the information returned from the x-ms-request-id header response.
RequestID *string
// StructuredBodyType contains the information returned from the x-ms-structured-body header response.
StructuredBodyType *string
// Version contains the information returned from the x-ms-version header response.
Version *string
@@ -1882,6 +1897,9 @@ type PageBlobClientUploadPagesResponse struct {
// RequestID contains the information returned from the x-ms-request-id header response.
RequestID *string
// StructuredBodyType contains the information returned from the x-ms-structured-body header response.
StructuredBodyType *string
// Version contains the information returned from the x-ms-version header response.
Version *string
}

View File

@@ -30,7 +30,7 @@ type ServiceClient struct {
// be scoped within the expression to a single container.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - where - Filters the results to return only to return only blobs whose tags match the specified expression.
// - options - ServiceClientFilterBlobsOptions contains the optional parameters for the ServiceClient.FilterBlobs method.
func (client *ServiceClient) FilterBlobs(ctx context.Context, where string, options *ServiceClientFilterBlobsOptions) (ServiceClientFilterBlobsResponse, error) {
@@ -77,7 +77,7 @@ func (client *ServiceClient) filterBlobsCreateRequest(ctx context.Context, where
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -109,7 +109,7 @@ func (client *ServiceClient) filterBlobsHandleResponse(resp *http.Response) (Ser
// GetAccountInfo - Returns the sku name and account kind
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - ServiceClientGetAccountInfoOptions contains the optional parameters for the ServiceClient.GetAccountInfo method.
func (client *ServiceClient) GetAccountInfo(ctx context.Context, options *ServiceClientGetAccountInfoOptions) (ServiceClientGetAccountInfoResponse, error) {
var err error
@@ -141,12 +141,12 @@ func (client *ServiceClient) getAccountInfoCreateRequest(ctx context.Context, op
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -189,7 +189,7 @@ func (client *ServiceClient) getAccountInfoHandleResponse(resp *http.Response) (
// CORS (Cross-Origin Resource Sharing) rules.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - ServiceClientGetPropertiesOptions contains the optional parameters for the ServiceClient.GetProperties method.
func (client *ServiceClient) GetProperties(ctx context.Context, options *ServiceClientGetPropertiesOptions) (ServiceClientGetPropertiesResponse, error) {
var err error
@@ -221,12 +221,12 @@ func (client *ServiceClient) getPropertiesCreateRequest(ctx context.Context, opt
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -252,7 +252,7 @@ func (client *ServiceClient) getPropertiesHandleResponse(resp *http.Response) (S
// location endpoint when read-access geo-redundant replication is enabled for the storage account.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - ServiceClientGetStatisticsOptions contains the optional parameters for the ServiceClient.GetStatistics method.
func (client *ServiceClient) GetStatistics(ctx context.Context, options *ServiceClientGetStatisticsOptions) (ServiceClientGetStatisticsResponse, error) {
var err error
@@ -284,12 +284,12 @@ func (client *ServiceClient) getStatisticsCreateRequest(ctx context.Context, opt
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -322,7 +322,7 @@ func (client *ServiceClient) getStatisticsHandleResponse(resp *http.Response) (S
// bearer token authentication.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - keyInfo - Key information
// - options - ServiceClientGetUserDelegationKeyOptions contains the optional parameters for the ServiceClient.GetUserDelegationKey
// method.
@@ -356,12 +356,12 @@ func (client *ServiceClient) getUserDelegationKeyCreateRequest(ctx context.Conte
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
if err := runtime.MarshalAsXML(req, keyInfo); err != nil {
return nil, err
}
@@ -396,7 +396,7 @@ func (client *ServiceClient) getUserDelegationKeyHandleResponse(resp *http.Respo
// NewListContainersSegmentPager - The List Containers Segment operation returns a list of the containers under the specified
// account
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - options - ServiceClientListContainersSegmentOptions contains the optional parameters for the ServiceClient.NewListContainersSegmentPager
// method.
//
@@ -423,12 +423,12 @@ func (client *ServiceClient) ListContainersSegmentCreateRequest(ctx context.Cont
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
return req, nil
}
@@ -454,7 +454,7 @@ func (client *ServiceClient) ListContainersSegmentHandleResponse(resp *http.Resp
// and CORS (Cross-Origin Resource Sharing) rules
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - storageServiceProperties - The StorageService properties.
// - options - ServiceClientSetPropertiesOptions contains the optional parameters for the ServiceClient.SetProperties method.
func (client *ServiceClient) SetProperties(ctx context.Context, storageServiceProperties StorageServiceProperties, options *ServiceClientSetPropertiesOptions) (ServiceClientSetPropertiesResponse, error) {
@@ -487,12 +487,12 @@ func (client *ServiceClient) setPropertiesCreateRequest(ctx context.Context, sto
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
req.Raw().Header["Accept"] = []string{"application/xml"}
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
if err := runtime.MarshalAsXML(req, storageServiceProperties); err != nil {
return nil, err
}
@@ -517,7 +517,7 @@ func (client *ServiceClient) setPropertiesHandleResponse(resp *http.Response) (S
// SubmitBatch - The Batch operation allows multiple API calls to be embedded into a single HTTP request.
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2024-08-04
// Generated from API version 2025-01-05
// - contentLength - The length of the request.
// - multipartContentType - Required. The value of this header must be multipart/mixed with a batch boundary. Example header
// value: multipart/mixed; boundary=batch_
@@ -552,7 +552,7 @@ func (client *ServiceClient) submitBatchCreateRequest(ctx context.Context, conte
if options != nil && options.Timeout != nil {
reqQP.Set("timeout", strconv.FormatInt(int64(*options.Timeout), 10))
}
req.Raw().URL.RawQuery = reqQP.Encode()
req.Raw().URL.RawQuery = strings.Replace(reqQP.Encode(), "+", "%20", -1)
runtime.SkipBodyDownload(req)
req.Raw().Header["Accept"] = []string{"application/xml"}
req.Raw().Header["Content-Length"] = []string{strconv.FormatInt(contentLength, 10)}
@@ -560,7 +560,7 @@ func (client *ServiceClient) submitBatchCreateRequest(ctx context.Context, conte
if options != nil && options.RequestID != nil {
req.Raw().Header["x-ms-client-request-id"] = []string{*options.RequestID}
}
req.Raw().Header["x-ms-version"] = []string{ServiceVersion}
req.Raw().Header["x-ms-version"] = []string{"2025-01-05"}
if err := req.SetBody(body, multipartContentType); err != nil {
return nil, err
}

View File

@@ -1,6 +1,6 @@
//go:build go1.18 && (linux || darwin || dragonfly || freebsd || openbsd || netbsd || solaris || aix)
//go:build go1.18 && (linux || darwin || dragonfly || freebsd || openbsd || netbsd || solaris || aix || zos)
// +build go1.18
// +build linux darwin dragonfly freebsd openbsd netbsd solaris aix
// +build linux darwin dragonfly freebsd openbsd netbsd solaris aix zos
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License. See License.txt in the project root for license information.

33
vendor/github.com/aws/aws-sdk-go-v2/aws/checksum.go generated vendored Normal file
View File

@@ -0,0 +1,33 @@
package aws
// RequestChecksumCalculation controls request checksum calculation workflow
type RequestChecksumCalculation int
const (
// RequestChecksumCalculationUnset is the unset value for RequestChecksumCalculation
RequestChecksumCalculationUnset RequestChecksumCalculation = iota
// RequestChecksumCalculationWhenSupported indicates request checksum will be calculated
// if the operation supports input checksums
RequestChecksumCalculationWhenSupported
// RequestChecksumCalculationWhenRequired indicates request checksum will be calculated
// if required by the operation or if user elects to set a checksum algorithm in request
RequestChecksumCalculationWhenRequired
)
// ResponseChecksumValidation controls response checksum validation workflow
type ResponseChecksumValidation int
const (
// ResponseChecksumValidationUnset is the unset value for ResponseChecksumValidation
ResponseChecksumValidationUnset ResponseChecksumValidation = iota
// ResponseChecksumValidationWhenSupported indicates response checksum will be validated
// if the operation supports output checksums
ResponseChecksumValidationWhenSupported
// ResponseChecksumValidationWhenRequired indicates response checksum will only
// be validated if the operation requires output checksum validation
ResponseChecksumValidationWhenRequired
)

View File

@@ -165,6 +165,33 @@ type Config struct {
// Controls how a resolved AWS account ID is handled for endpoint routing.
AccountIDEndpointMode AccountIDEndpointMode
// RequestChecksumCalculation determines when request checksum calculation is performed.
//
// There are two possible values for this setting:
//
// 1. RequestChecksumCalculationWhenSupported (default): The checksum is always calculated
// if the operation supports it, regardless of whether the user sets an algorithm in the request.
//
// 2. RequestChecksumCalculationWhenRequired: The checksum is only calculated if the user
// explicitly sets a checksum algorithm in the request.
//
// This setting is sourced from the environment variable AWS_REQUEST_CHECKSUM_CALCULATION
// or the shared config profile attribute "request_checksum_calculation".
RequestChecksumCalculation RequestChecksumCalculation
// ResponseChecksumValidation determines when response checksum validation is performed
//
// There are two possible values for this setting:
//
// 1. ResponseChecksumValidationWhenSupported (default): The checksum is always validated
// if the operation supports it, regardless of whether the user sets the validation mode to ENABLED in request.
//
// 2. ResponseChecksumValidationWhenRequired: The checksum is only validated if the user
// explicitly sets the validation mode to ENABLED in the request
// This variable is sourced from environment variable AWS_RESPONSE_CHECKSUM_VALIDATION or
// the shared config profile attribute "response_checksum_validation".
ResponseChecksumValidation ResponseChecksumValidation
}
// NewConfig returns a new Config pointer that can be chained with builder

View File

@@ -3,4 +3,4 @@
package aws
// goModuleVersion is the tagged release for this module
const goModuleVersion = "1.32.8"
const goModuleVersion = "1.34.0"

View File

@@ -76,19 +76,28 @@ type UserAgentFeature string
// Enumerates UserAgentFeature.
const (
UserAgentFeatureResourceModel UserAgentFeature = "A" // n/a (we don't generate separate resource types)
UserAgentFeatureWaiter = "B"
UserAgentFeaturePaginator = "C"
UserAgentFeatureRetryModeLegacy = "D" // n/a (equivalent to standard)
UserAgentFeatureRetryModeStandard = "E"
UserAgentFeatureRetryModeAdaptive = "F"
UserAgentFeatureS3Transfer = "G"
UserAgentFeatureS3CryptoV1N = "H" // n/a (crypto client is external)
UserAgentFeatureS3CryptoV2 = "I" // n/a
UserAgentFeatureS3ExpressBucket = "J"
UserAgentFeatureS3AccessGrants = "K" // not yet implemented
UserAgentFeatureGZIPRequestCompression = "L"
UserAgentFeatureProtocolRPCV2CBOR = "M"
UserAgentFeatureResourceModel UserAgentFeature = "A" // n/a (we don't generate separate resource types)
UserAgentFeatureWaiter = "B"
UserAgentFeaturePaginator = "C"
UserAgentFeatureRetryModeLegacy = "D" // n/a (equivalent to standard)
UserAgentFeatureRetryModeStandard = "E"
UserAgentFeatureRetryModeAdaptive = "F"
UserAgentFeatureS3Transfer = "G"
UserAgentFeatureS3CryptoV1N = "H" // n/a (crypto client is external)
UserAgentFeatureS3CryptoV2 = "I" // n/a
UserAgentFeatureS3ExpressBucket = "J"
UserAgentFeatureS3AccessGrants = "K" // not yet implemented
UserAgentFeatureGZIPRequestCompression = "L"
UserAgentFeatureProtocolRPCV2CBOR = "M"
UserAgentFeatureRequestChecksumCRC32 = "U"
UserAgentFeatureRequestChecksumCRC32C = "V"
UserAgentFeatureRequestChecksumCRC64 = "W"
UserAgentFeatureRequestChecksumSHA1 = "X"
UserAgentFeatureRequestChecksumSHA256 = "Y"
UserAgentFeatureRequestChecksumWhenSupported = "Z"
UserAgentFeatureRequestChecksumWhenRequired = "a"
UserAgentFeatureResponseChecksumWhenSupported = "b"
UserAgentFeatureResponseChecksumWhenRequired = "c"
)
// RequestUserAgent is a build middleware that set the User-Agent for the request.

View File

@@ -1,3 +1,7 @@
# v1.6.8 (2025-01-24)
* **Dependency Update**: Upgrade to smithy-go v1.22.2.
# v1.6.7 (2024-11-18)
* **Dependency Update**: Update to smithy-go v1.22.1.

View File

@@ -3,4 +3,4 @@
package eventstream
// goModuleVersion is the tagged release for this module
const goModuleVersion = "1.6.7"
const goModuleVersion = "1.6.8"

View File

@@ -1,8 +1,8 @@
package query
import (
"fmt"
"net/url"
"strconv"
)
// Array represents the encoding of Query lists and sets. A Query array is a
@@ -21,19 +21,8 @@ type Array struct {
// keys for each element in the list. For example, an entry might have the
// key "ParentStructure.ListName.member.MemberName.1".
//
// While this is currently represented as a string that gets added to, it
// could also be represented as a stack that only gets condensed into a
// string when a finalized key is created. This could potentially reduce
// allocations.
// When the array is not flat the prefix will contain the memberName otherwise the memberName is ignored
prefix string
// Whether the list is flat or not. A list that is not flat will produce the
// following entry to the url.Values for a given entry:
// ListName.MemberName.1=value
// A list that is flat will produce the following:
// ListName.1=value
flat bool
// The location name of the member. In most cases this should be "member".
memberName string
// Elements are stored in values, so we keep track of the list size here.
size int32
// Empty lists are encoded as "<prefix>=", if we add a value later we will
@@ -45,11 +34,14 @@ func newArray(values url.Values, prefix string, flat bool, memberName string) *A
emptyValue := newValue(values, prefix, flat)
emptyValue.String("")
if !flat {
// This uses string concatenation in place of fmt.Sprintf as fmt.Sprintf has a much higher resource overhead
prefix = prefix + keySeparator + memberName
}
return &Array{
values: values,
prefix: prefix,
flat: flat,
memberName: memberName,
emptyValue: emptyValue,
}
}
@@ -63,10 +55,7 @@ func (a *Array) Value() Value {
// Query lists start a 1, so adjust the size first
a.size++
prefix := a.prefix
if !a.flat {
prefix = fmt.Sprintf("%s.%s", prefix, a.memberName)
}
// Lists can't have flat members
return newValue(a.values, fmt.Sprintf("%s.%d", prefix, a.size), false)
// This uses string concatenation in place of fmt.Sprintf as fmt.Sprintf has a much higher resource overhead
return newValue(a.values, a.prefix+keySeparator+strconv.FormatInt(int64(a.size), 10), false)
}

View File

@@ -1,9 +1,6 @@
package query
import (
"fmt"
"net/url"
)
import "net/url"
// Object represents the encoding of Query structures and unions. A Query
// object is a representation of a mapping of string keys to arbitrary
@@ -56,14 +53,16 @@ func (o *Object) FlatKey(name string) Value {
func (o *Object) key(name string, flatValue bool) Value {
if o.prefix != "" {
return newValue(o.values, fmt.Sprintf("%s.%s", o.prefix, name), flatValue)
// This uses string concatenation in place of fmt.Sprintf as fmt.Sprintf has a much higher resource overhead
return newValue(o.values, o.prefix+keySeparator+name, flatValue)
}
return newValue(o.values, name, flatValue)
}
func (o *Object) keyWithValues(name string, flatValue bool) Value {
if o.prefix != "" {
return newAppendValue(o.values, fmt.Sprintf("%s.%s", o.prefix, name), flatValue)
// This uses string concatenation in place of fmt.Sprintf as fmt.Sprintf has a much higher resource overhead
return newAppendValue(o.values, o.prefix+keySeparator+name, flatValue)
}
return newAppendValue(o.values, name, flatValue)
}

View File

@@ -7,6 +7,8 @@ import (
"github.com/aws/smithy-go/encoding/httpbinding"
)
const keySeparator = "."
// Value represents a Query Value type.
type Value struct {
// The query values to add the value to.

View File

@@ -1,3 +1,22 @@
# v1.29.2 (2025-01-24)
* **Bug Fix**: Fix env config naming and usage of deprecated ioutil
* **Dependency Update**: Updated to the latest SDK module versions
* **Dependency Update**: Upgrade to smithy-go v1.22.2.
# v1.29.1 (2025-01-17)
* **Dependency Update**: Updated to the latest SDK module versions
# v1.29.0 (2025-01-15)
* **Feature**: S3 client behavior is updated to always calculate a checksum by default for operations that support it (such as PutObject or UploadPart), or require it (such as DeleteObjects). The checksum algorithm used by default now becomes CRC32. Checksum behavior can be configured using `when_supported` and `when_required` options - in code using RequestChecksumCalculation, in shared config using request_checksum_calculation, or as env variable using AWS_REQUEST_CHECKSUM_CALCULATION. The S3 client attempts to validate response checksums for all S3 API operations that support checksums. However, if the SDK has not implemented the specified checksum algorithm then this validation is skipped. Checksum validation behavior can be configured using `when_supported` and `when_required` options - in code using ResponseChecksumValidation, in shared config using response_checksum_validation, or as env variable using AWS_RESPONSE_CHECKSUM_VALIDATION.
* **Dependency Update**: Updated to the latest SDK module versions
# v1.28.11 (2025-01-14)
* **Dependency Update**: Updated to the latest SDK module versions
# v1.28.10 (2025-01-10)
* **Dependency Update**: Updated to the latest SDK module versions

View File

@@ -83,6 +83,12 @@ var defaultAWSConfigResolvers = []awsConfigResolver{
// Sets the AccountIDEndpointMode if present in env var or shared config profile
resolveAccountIDEndpointMode,
// Sets the RequestChecksumCalculation if present in env var or shared config profile
resolveRequestChecksumCalculation,
// Sets the ResponseChecksumValidation if present in env var or shared config profile
resolveResponseChecksumValidation,
}
// A Config represents a generic configuration value or set of values. This type
@@ -212,7 +218,7 @@ func resolveConfigLoaders(options *LoadOptions) []loader {
loaders[0] = loadEnvConfig
// specification of a profile should cause a load failure if it doesn't exist
if os.Getenv(awsProfileEnvVar) != "" || options.SharedConfigProfile != "" {
if os.Getenv(awsProfileEnv) != "" || options.SharedConfigProfile != "" {
loaders[1] = loadSharedConfig
} else {
loaders[1] = loadSharedConfigIgnoreNotExist

View File

@@ -5,7 +5,6 @@ import (
"context"
"fmt"
"io"
"io/ioutil"
"os"
"strconv"
"strings"
@@ -21,86 +20,89 @@ const CredentialsSourceName = "EnvConfigCredentials"
// Environment variables that will be read for configuration values.
const (
awsAccessKeyIDEnvVar = "AWS_ACCESS_KEY_ID"
awsAccessKeyEnvVar = "AWS_ACCESS_KEY"
awsAccessKeyIDEnv = "AWS_ACCESS_KEY_ID"
awsAccessKeyEnv = "AWS_ACCESS_KEY"
awsSecretAccessKeyEnvVar = "AWS_SECRET_ACCESS_KEY"
awsSecretKeyEnvVar = "AWS_SECRET_KEY"
awsSecretAccessKeyEnv = "AWS_SECRET_ACCESS_KEY"
awsSecretKeyEnv = "AWS_SECRET_KEY"
awsSessionTokenEnvVar = "AWS_SESSION_TOKEN"
awsSessionTokenEnv = "AWS_SESSION_TOKEN"
awsContainerCredentialsEndpointEnvVar = "AWS_CONTAINER_CREDENTIALS_FULL_URI"
awsContainerCredentialsRelativePathEnvVar = "AWS_CONTAINER_CREDENTIALS_RELATIVE_URI"
awsContainerPProviderAuthorizationEnvVar = "AWS_CONTAINER_AUTHORIZATION_TOKEN"
awsContainerCredentialsFullURIEnv = "AWS_CONTAINER_CREDENTIALS_FULL_URI"
awsContainerCredentialsRelativeURIEnv = "AWS_CONTAINER_CREDENTIALS_RELATIVE_URI"
awsContainerAuthorizationTokenEnv = "AWS_CONTAINER_AUTHORIZATION_TOKEN"
awsRegionEnvVar = "AWS_REGION"
awsDefaultRegionEnvVar = "AWS_DEFAULT_REGION"
awsRegionEnv = "AWS_REGION"
awsDefaultRegionEnv = "AWS_DEFAULT_REGION"
awsProfileEnvVar = "AWS_PROFILE"
awsDefaultProfileEnvVar = "AWS_DEFAULT_PROFILE"
awsProfileEnv = "AWS_PROFILE"
awsDefaultProfileEnv = "AWS_DEFAULT_PROFILE"
awsSharedCredentialsFileEnvVar = "AWS_SHARED_CREDENTIALS_FILE"
awsSharedCredentialsFileEnv = "AWS_SHARED_CREDENTIALS_FILE"
awsConfigFileEnvVar = "AWS_CONFIG_FILE"
awsConfigFileEnv = "AWS_CONFIG_FILE"
awsCustomCABundleEnvVar = "AWS_CA_BUNDLE"
awsCABundleEnv = "AWS_CA_BUNDLE"
awsWebIdentityTokenFilePathEnvVar = "AWS_WEB_IDENTITY_TOKEN_FILE"
awsWebIdentityTokenFileEnv = "AWS_WEB_IDENTITY_TOKEN_FILE"
awsRoleARNEnvVar = "AWS_ROLE_ARN"
awsRoleSessionNameEnvVar = "AWS_ROLE_SESSION_NAME"
awsRoleARNEnv = "AWS_ROLE_ARN"
awsRoleSessionNameEnv = "AWS_ROLE_SESSION_NAME"
awsEnableEndpointDiscoveryEnvVar = "AWS_ENABLE_ENDPOINT_DISCOVERY"
awsEnableEndpointDiscoveryEnv = "AWS_ENABLE_ENDPOINT_DISCOVERY"
awsS3UseARNRegionEnvVar = "AWS_S3_USE_ARN_REGION"
awsS3UseARNRegionEnv = "AWS_S3_USE_ARN_REGION"
awsEc2MetadataServiceEndpointModeEnvVar = "AWS_EC2_METADATA_SERVICE_ENDPOINT_MODE"
awsEc2MetadataServiceEndpointModeEnv = "AWS_EC2_METADATA_SERVICE_ENDPOINT_MODE"
awsEc2MetadataServiceEndpointEnvVar = "AWS_EC2_METADATA_SERVICE_ENDPOINT"
awsEc2MetadataServiceEndpointEnv = "AWS_EC2_METADATA_SERVICE_ENDPOINT"
awsEc2MetadataDisabled = "AWS_EC2_METADATA_DISABLED"
awsEc2MetadataV1DisabledEnvVar = "AWS_EC2_METADATA_V1_DISABLED"
awsEc2MetadataDisabledEnv = "AWS_EC2_METADATA_DISABLED"
awsEc2MetadataV1DisabledEnv = "AWS_EC2_METADATA_V1_DISABLED"
awsS3DisableMultiRegionAccessPointEnvVar = "AWS_S3_DISABLE_MULTIREGION_ACCESS_POINTS"
awsS3DisableMultiRegionAccessPointsEnv = "AWS_S3_DISABLE_MULTIREGION_ACCESS_POINTS"
awsUseDualStackEndpoint = "AWS_USE_DUALSTACK_ENDPOINT"
awsUseDualStackEndpointEnv = "AWS_USE_DUALSTACK_ENDPOINT"
awsUseFIPSEndpoint = "AWS_USE_FIPS_ENDPOINT"
awsUseFIPSEndpointEnv = "AWS_USE_FIPS_ENDPOINT"
awsDefaultMode = "AWS_DEFAULTS_MODE"
awsDefaultsModeEnv = "AWS_DEFAULTS_MODE"
awsRetryMaxAttempts = "AWS_MAX_ATTEMPTS"
awsRetryMode = "AWS_RETRY_MODE"
awsSdkAppID = "AWS_SDK_UA_APP_ID"
awsMaxAttemptsEnv = "AWS_MAX_ATTEMPTS"
awsRetryModeEnv = "AWS_RETRY_MODE"
awsSdkUaAppIDEnv = "AWS_SDK_UA_APP_ID"
awsIgnoreConfiguredEndpoints = "AWS_IGNORE_CONFIGURED_ENDPOINT_URLS"
awsEndpointURL = "AWS_ENDPOINT_URL"
awsIgnoreConfiguredEndpointURLEnv = "AWS_IGNORE_CONFIGURED_ENDPOINT_URLS"
awsEndpointURLEnv = "AWS_ENDPOINT_URL"
awsDisableRequestCompression = "AWS_DISABLE_REQUEST_COMPRESSION"
awsRequestMinCompressionSizeBytes = "AWS_REQUEST_MIN_COMPRESSION_SIZE_BYTES"
awsDisableRequestCompressionEnv = "AWS_DISABLE_REQUEST_COMPRESSION"
awsRequestMinCompressionSizeBytesEnv = "AWS_REQUEST_MIN_COMPRESSION_SIZE_BYTES"
awsS3DisableExpressSessionAuthEnv = "AWS_S3_DISABLE_EXPRESS_SESSION_AUTH"
awsAccountIDEnv = "AWS_ACCOUNT_ID"
awsAccountIDEndpointModeEnv = "AWS_ACCOUNT_ID_ENDPOINT_MODE"
awsRequestChecksumCalculation = "AWS_REQUEST_CHECKSUM_CALCULATION"
awsResponseChecksumValidation = "AWS_RESPONSE_CHECKSUM_VALIDATION"
)
var (
credAccessEnvKeys = []string{
awsAccessKeyIDEnvVar,
awsAccessKeyEnvVar,
awsAccessKeyIDEnv,
awsAccessKeyEnv,
}
credSecretEnvKeys = []string{
awsSecretAccessKeyEnvVar,
awsSecretKeyEnvVar,
awsSecretAccessKeyEnv,
awsSecretKeyEnv,
}
regionEnvKeys = []string{
awsRegionEnvVar,
awsDefaultRegionEnvVar,
awsRegionEnv,
awsDefaultRegionEnv,
}
profileEnvKeys = []string{
awsProfileEnvVar,
awsDefaultProfileEnvVar,
awsProfileEnv,
awsDefaultProfileEnv,
}
)
@@ -296,6 +298,12 @@ type EnvConfig struct {
// Indicates whether account ID will be required/ignored in endpoint2.0 routing
AccountIDEndpointMode aws.AccountIDEndpointMode
// Indicates whether request checksum should be calculated
RequestChecksumCalculation aws.RequestChecksumCalculation
// Indicates whether response checksum should be validated
ResponseChecksumValidation aws.ResponseChecksumValidation
}
// loadEnvConfig reads configuration values from the OS's environment variables.
@@ -316,79 +324,79 @@ func NewEnvConfig() (EnvConfig, error) {
setStringFromEnvVal(&creds.SecretAccessKey, credSecretEnvKeys)
if creds.HasKeys() {
creds.AccountID = os.Getenv(awsAccountIDEnv)
creds.SessionToken = os.Getenv(awsSessionTokenEnvVar)
creds.SessionToken = os.Getenv(awsSessionTokenEnv)
cfg.Credentials = creds
}
cfg.ContainerCredentialsEndpoint = os.Getenv(awsContainerCredentialsEndpointEnvVar)
cfg.ContainerCredentialsRelativePath = os.Getenv(awsContainerCredentialsRelativePathEnvVar)
cfg.ContainerAuthorizationToken = os.Getenv(awsContainerPProviderAuthorizationEnvVar)
cfg.ContainerCredentialsEndpoint = os.Getenv(awsContainerCredentialsFullURIEnv)
cfg.ContainerCredentialsRelativePath = os.Getenv(awsContainerCredentialsRelativeURIEnv)
cfg.ContainerAuthorizationToken = os.Getenv(awsContainerAuthorizationTokenEnv)
setStringFromEnvVal(&cfg.Region, regionEnvKeys)
setStringFromEnvVal(&cfg.SharedConfigProfile, profileEnvKeys)
cfg.SharedCredentialsFile = os.Getenv(awsSharedCredentialsFileEnvVar)
cfg.SharedConfigFile = os.Getenv(awsConfigFileEnvVar)
cfg.SharedCredentialsFile = os.Getenv(awsSharedCredentialsFileEnv)
cfg.SharedConfigFile = os.Getenv(awsConfigFileEnv)
cfg.CustomCABundle = os.Getenv(awsCustomCABundleEnvVar)
cfg.CustomCABundle = os.Getenv(awsCABundleEnv)
cfg.WebIdentityTokenFilePath = os.Getenv(awsWebIdentityTokenFilePathEnvVar)
cfg.WebIdentityTokenFilePath = os.Getenv(awsWebIdentityTokenFileEnv)
cfg.RoleARN = os.Getenv(awsRoleARNEnvVar)
cfg.RoleSessionName = os.Getenv(awsRoleSessionNameEnvVar)
cfg.RoleARN = os.Getenv(awsRoleARNEnv)
cfg.RoleSessionName = os.Getenv(awsRoleSessionNameEnv)
cfg.AppID = os.Getenv(awsSdkAppID)
cfg.AppID = os.Getenv(awsSdkUaAppIDEnv)
if err := setBoolPtrFromEnvVal(&cfg.DisableRequestCompression, []string{awsDisableRequestCompression}); err != nil {
if err := setBoolPtrFromEnvVal(&cfg.DisableRequestCompression, []string{awsDisableRequestCompressionEnv}); err != nil {
return cfg, err
}
if err := setInt64PtrFromEnvVal(&cfg.RequestMinCompressSizeBytes, []string{awsRequestMinCompressionSizeBytes}, smithyrequestcompression.MaxRequestMinCompressSizeBytes); err != nil {
if err := setInt64PtrFromEnvVal(&cfg.RequestMinCompressSizeBytes, []string{awsRequestMinCompressionSizeBytesEnv}, smithyrequestcompression.MaxRequestMinCompressSizeBytes); err != nil {
return cfg, err
}
if err := setEndpointDiscoveryTypeFromEnvVal(&cfg.EnableEndpointDiscovery, []string{awsEnableEndpointDiscoveryEnvVar}); err != nil {
if err := setEndpointDiscoveryTypeFromEnvVal(&cfg.EnableEndpointDiscovery, []string{awsEnableEndpointDiscoveryEnv}); err != nil {
return cfg, err
}
if err := setBoolPtrFromEnvVal(&cfg.S3UseARNRegion, []string{awsS3UseARNRegionEnvVar}); err != nil {
if err := setBoolPtrFromEnvVal(&cfg.S3UseARNRegion, []string{awsS3UseARNRegionEnv}); err != nil {
return cfg, err
}
setEC2IMDSClientEnableState(&cfg.EC2IMDSClientEnableState, []string{awsEc2MetadataDisabled})
if err := setEC2IMDSEndpointMode(&cfg.EC2IMDSEndpointMode, []string{awsEc2MetadataServiceEndpointModeEnvVar}); err != nil {
setEC2IMDSClientEnableState(&cfg.EC2IMDSClientEnableState, []string{awsEc2MetadataDisabledEnv})
if err := setEC2IMDSEndpointMode(&cfg.EC2IMDSEndpointMode, []string{awsEc2MetadataServiceEndpointModeEnv}); err != nil {
return cfg, err
}
cfg.EC2IMDSEndpoint = os.Getenv(awsEc2MetadataServiceEndpointEnvVar)
if err := setBoolPtrFromEnvVal(&cfg.EC2IMDSv1Disabled, []string{awsEc2MetadataV1DisabledEnvVar}); err != nil {
cfg.EC2IMDSEndpoint = os.Getenv(awsEc2MetadataServiceEndpointEnv)
if err := setBoolPtrFromEnvVal(&cfg.EC2IMDSv1Disabled, []string{awsEc2MetadataV1DisabledEnv}); err != nil {
return cfg, err
}
if err := setBoolPtrFromEnvVal(&cfg.S3DisableMultiRegionAccessPoints, []string{awsS3DisableMultiRegionAccessPointEnvVar}); err != nil {
if err := setBoolPtrFromEnvVal(&cfg.S3DisableMultiRegionAccessPoints, []string{awsS3DisableMultiRegionAccessPointsEnv}); err != nil {
return cfg, err
}
if err := setUseDualStackEndpointFromEnvVal(&cfg.UseDualStackEndpoint, []string{awsUseDualStackEndpoint}); err != nil {
if err := setUseDualStackEndpointFromEnvVal(&cfg.UseDualStackEndpoint, []string{awsUseDualStackEndpointEnv}); err != nil {
return cfg, err
}
if err := setUseFIPSEndpointFromEnvVal(&cfg.UseFIPSEndpoint, []string{awsUseFIPSEndpoint}); err != nil {
if err := setUseFIPSEndpointFromEnvVal(&cfg.UseFIPSEndpoint, []string{awsUseFIPSEndpointEnv}); err != nil {
return cfg, err
}
if err := setDefaultsModeFromEnvVal(&cfg.DefaultsMode, []string{awsDefaultMode}); err != nil {
if err := setDefaultsModeFromEnvVal(&cfg.DefaultsMode, []string{awsDefaultsModeEnv}); err != nil {
return cfg, err
}
if err := setIntFromEnvVal(&cfg.RetryMaxAttempts, []string{awsRetryMaxAttempts}); err != nil {
if err := setIntFromEnvVal(&cfg.RetryMaxAttempts, []string{awsMaxAttemptsEnv}); err != nil {
return cfg, err
}
if err := setRetryModeFromEnvVal(&cfg.RetryMode, []string{awsRetryMode}); err != nil {
if err := setRetryModeFromEnvVal(&cfg.RetryMode, []string{awsRetryModeEnv}); err != nil {
return cfg, err
}
setStringFromEnvVal(&cfg.BaseEndpoint, []string{awsEndpointURL})
setStringFromEnvVal(&cfg.BaseEndpoint, []string{awsEndpointURLEnv})
if err := setBoolPtrFromEnvVal(&cfg.IgnoreConfiguredEndpoints, []string{awsIgnoreConfiguredEndpoints}); err != nil {
if err := setBoolPtrFromEnvVal(&cfg.IgnoreConfiguredEndpoints, []string{awsIgnoreConfiguredEndpointURLEnv}); err != nil {
return cfg, err
}
@@ -400,6 +408,13 @@ func NewEnvConfig() (EnvConfig, error) {
return cfg, err
}
if err := setRequestChecksumCalculationFromEnvVal(&cfg.RequestChecksumCalculation, []string{awsRequestChecksumCalculation}); err != nil {
return cfg, err
}
if err := setResponseChecksumValidationFromEnvVal(&cfg.ResponseChecksumValidation, []string{awsResponseChecksumValidation}); err != nil {
return cfg, err
}
return cfg, nil
}
@@ -432,6 +447,14 @@ func (c EnvConfig) getAccountIDEndpointMode(context.Context) (aws.AccountIDEndpo
return c.AccountIDEndpointMode, len(c.AccountIDEndpointMode) > 0, nil
}
func (c EnvConfig) getRequestChecksumCalculation(context.Context) (aws.RequestChecksumCalculation, bool, error) {
return c.RequestChecksumCalculation, c.RequestChecksumCalculation > 0, nil
}
func (c EnvConfig) getResponseChecksumValidation(context.Context) (aws.ResponseChecksumValidation, bool, error) {
return c.ResponseChecksumValidation, c.ResponseChecksumValidation > 0, nil
}
// GetRetryMaxAttempts returns the value of AWS_MAX_ATTEMPTS if was specified,
// and not 0.
func (c EnvConfig) GetRetryMaxAttempts(ctx context.Context) (int, bool, error) {
@@ -528,6 +551,45 @@ func setAIDEndPointModeFromEnvVal(m *aws.AccountIDEndpointMode, keys []string) e
return nil
}
func setRequestChecksumCalculationFromEnvVal(m *aws.RequestChecksumCalculation, keys []string) error {
for _, k := range keys {
value := os.Getenv(k)
if len(value) == 0 {
continue
}
switch strings.ToLower(value) {
case checksumWhenSupported:
*m = aws.RequestChecksumCalculationWhenSupported
case checksumWhenRequired:
*m = aws.RequestChecksumCalculationWhenRequired
default:
return fmt.Errorf("invalid value for environment variable, %s=%s, must be when_supported/when_required", k, value)
}
}
return nil
}
func setResponseChecksumValidationFromEnvVal(m *aws.ResponseChecksumValidation, keys []string) error {
for _, k := range keys {
value := os.Getenv(k)
if len(value) == 0 {
continue
}
switch strings.ToLower(value) {
case checksumWhenSupported:
*m = aws.ResponseChecksumValidationWhenSupported
case checksumWhenRequired:
*m = aws.ResponseChecksumValidationWhenRequired
default:
return fmt.Errorf("invalid value for environment variable, %s=%s, must be when_supported/when_required", k, value)
}
}
return nil
}
// GetRegion returns the AWS Region if set in the environment. Returns an empty
// string if not set.
func (c EnvConfig) getRegion(ctx context.Context) (string, bool, error) {
@@ -584,7 +646,7 @@ func (c EnvConfig) getCustomCABundle(context.Context) (io.Reader, bool, error) {
return nil, false, nil
}
b, err := ioutil.ReadFile(c.CustomCABundle)
b, err := os.ReadFile(c.CustomCABundle)
if err != nil {
return nil, false, err
}
@@ -608,7 +670,7 @@ func (c EnvConfig) getBaseEndpoint(context.Context) (string, bool, error) {
// GetServiceBaseEndpoint is used to retrieve a normalized SDK ID for use
// with configured endpoints.
func (c EnvConfig) GetServiceBaseEndpoint(ctx context.Context, sdkID string) (string, bool, error) {
if endpt := os.Getenv(fmt.Sprintf("%s_%s", awsEndpointURL, normalizeEnv(sdkID))); endpt != "" {
if endpt := os.Getenv(fmt.Sprintf("%s_%s", awsEndpointURLEnv, normalizeEnv(sdkID))); endpt != "" {
return endpt, true, nil
}
return "", false, nil

View File

@@ -3,4 +3,4 @@
package config
// goModuleVersion is the tagged release for this module
const goModuleVersion = "1.28.10"
const goModuleVersion = "1.29.2"

View File

@@ -216,8 +216,15 @@ type LoadOptions struct {
// Whether S3 Express auth is disabled.
S3DisableExpressAuth *bool
// Whether account id should be built into endpoint resolution
AccountIDEndpointMode aws.AccountIDEndpointMode
// Specify if request checksum should be calculated
RequestChecksumCalculation aws.RequestChecksumCalculation
// Specifies if response checksum should be validated
ResponseChecksumValidation aws.ResponseChecksumValidation
// Service endpoint override. This value is not necessarily final and is
// passed to the service's EndpointResolverV2 for further delegation.
BaseEndpoint string
@@ -288,6 +295,14 @@ func (o LoadOptions) getAccountIDEndpointMode(ctx context.Context) (aws.AccountI
return o.AccountIDEndpointMode, len(o.AccountIDEndpointMode) > 0, nil
}
func (o LoadOptions) getRequestChecksumCalculation(ctx context.Context) (aws.RequestChecksumCalculation, bool, error) {
return o.RequestChecksumCalculation, o.RequestChecksumCalculation > 0, nil
}
func (o LoadOptions) getResponseChecksumValidation(ctx context.Context) (aws.ResponseChecksumValidation, bool, error) {
return o.ResponseChecksumValidation, o.ResponseChecksumValidation > 0, nil
}
func (o LoadOptions) getBaseEndpoint(context.Context) (string, bool, error) {
return o.BaseEndpoint, o.BaseEndpoint != "", nil
}
@@ -357,6 +372,26 @@ func WithAccountIDEndpointMode(m aws.AccountIDEndpointMode) LoadOptionsFunc {
}
}
// WithRequestChecksumCalculation is a helper function to construct functional options
// that sets RequestChecksumCalculation on config's LoadOptions
func WithRequestChecksumCalculation(c aws.RequestChecksumCalculation) LoadOptionsFunc {
return func(o *LoadOptions) error {
if c > 0 {
o.RequestChecksumCalculation = c
}
return nil
}
}
// WithResponseChecksumValidation is a helper function to construct functional options
// that sets ResponseChecksumValidation on config's LoadOptions
func WithResponseChecksumValidation(v aws.ResponseChecksumValidation) LoadOptionsFunc {
return func(o *LoadOptions) error {
o.ResponseChecksumValidation = v
return nil
}
}
// getDefaultRegion returns DefaultRegion from config's LoadOptions
func (o LoadOptions) getDefaultRegion(ctx context.Context) (string, bool, error) {
if len(o.DefaultRegion) == 0 {

View File

@@ -242,6 +242,40 @@ func getAccountIDEndpointMode(ctx context.Context, configs configs) (value aws.A
return
}
// requestChecksumCalculationProvider provides access to the RequestChecksumCalculation
type requestChecksumCalculationProvider interface {
getRequestChecksumCalculation(context.Context) (aws.RequestChecksumCalculation, bool, error)
}
func getRequestChecksumCalculation(ctx context.Context, configs configs) (value aws.RequestChecksumCalculation, found bool, err error) {
for _, cfg := range configs {
if p, ok := cfg.(requestChecksumCalculationProvider); ok {
value, found, err = p.getRequestChecksumCalculation(ctx)
if err != nil || found {
break
}
}
}
return
}
// responseChecksumValidationProvider provides access to the ResponseChecksumValidation
type responseChecksumValidationProvider interface {
getResponseChecksumValidation(context.Context) (aws.ResponseChecksumValidation, bool, error)
}
func getResponseChecksumValidation(ctx context.Context, configs configs) (value aws.ResponseChecksumValidation, found bool, err error) {
for _, cfg := range configs {
if p, ok := cfg.(responseChecksumValidationProvider); ok {
value, found, err = p.getResponseChecksumValidation(ctx)
if err != nil || found {
break
}
}
}
return
}
// ec2IMDSRegionProvider provides access to the ec2 imds region
// configuration value
type ec2IMDSRegionProvider interface {

View File

@@ -182,6 +182,36 @@ func resolveAccountIDEndpointMode(ctx context.Context, cfg *aws.Config, configs
return nil
}
// resolveRequestChecksumCalculation extracts the RequestChecksumCalculation from the configs slice's
// SharedConfig or EnvConfig
func resolveRequestChecksumCalculation(ctx context.Context, cfg *aws.Config, configs configs) error {
c, found, err := getRequestChecksumCalculation(ctx, configs)
if err != nil {
return err
}
if !found {
c = aws.RequestChecksumCalculationWhenSupported
}
cfg.RequestChecksumCalculation = c
return nil
}
// resolveResponseValidation extracts the ResponseChecksumValidation from the configs slice's
// SharedConfig or EnvConfig
func resolveResponseChecksumValidation(ctx context.Context, cfg *aws.Config, configs configs) error {
c, found, err := getResponseChecksumValidation(ctx, configs)
if err != nil {
return err
}
if !found {
c = aws.ResponseChecksumValidationWhenSupported
}
cfg.ResponseChecksumValidation = c
return nil
}
// resolveDefaultRegion extracts the first instance of a default region and sets `aws.Config.Region` to the default
// region if region had not been resolved from other sources.
func resolveDefaultRegion(ctx context.Context, cfg *aws.Config, configs configs) error {

View File

@@ -118,6 +118,11 @@ const (
accountIDKey = "aws_account_id"
accountIDEndpointMode = "account_id_endpoint_mode"
requestChecksumCalculationKey = "request_checksum_calculation"
responseChecksumValidationKey = "response_checksum_validation"
checksumWhenSupported = "when_supported"
checksumWhenRequired = "when_required"
)
// defaultSharedConfigProfile allows for swapping the default profile for testing
@@ -346,6 +351,12 @@ type SharedConfig struct {
S3DisableExpressAuth *bool
AccountIDEndpointMode aws.AccountIDEndpointMode
// RequestChecksumCalculation indicates if the request checksum should be calculated
RequestChecksumCalculation aws.RequestChecksumCalculation
// ResponseChecksumValidation indicates if the response checksum should be validated
ResponseChecksumValidation aws.ResponseChecksumValidation
}
func (c SharedConfig) getDefaultsMode(ctx context.Context) (value aws.DefaultsMode, ok bool, err error) {
@@ -1133,6 +1144,13 @@ func (c *SharedConfig) setFromIniSection(profile string, section ini.Section) er
return fmt.Errorf("failed to load %s from shared config, %w", accountIDEndpointMode, err)
}
if err := updateRequestChecksumCalculation(&c.RequestChecksumCalculation, section, requestChecksumCalculationKey); err != nil {
return fmt.Errorf("failed to load %s from shared config, %w", requestChecksumCalculationKey, err)
}
if err := updateResponseChecksumValidation(&c.ResponseChecksumValidation, section, responseChecksumValidationKey); err != nil {
return fmt.Errorf("failed to load %s from shared config, %w", responseChecksumValidationKey, err)
}
// Shared Credentials
creds := aws.Credentials{
AccessKeyID: section.String(accessKeyIDKey),
@@ -1207,6 +1225,42 @@ func updateAIDEndpointMode(m *aws.AccountIDEndpointMode, sec ini.Section, key st
return nil
}
func updateRequestChecksumCalculation(m *aws.RequestChecksumCalculation, sec ini.Section, key string) error {
if !sec.Has(key) {
return nil
}
v := sec.String(key)
switch strings.ToLower(v) {
case checksumWhenSupported:
*m = aws.RequestChecksumCalculationWhenSupported
case checksumWhenRequired:
*m = aws.RequestChecksumCalculationWhenRequired
default:
return fmt.Errorf("invalid value for shared config profile field, %s=%s, must be when_supported/when_required", key, v)
}
return nil
}
func updateResponseChecksumValidation(m *aws.ResponseChecksumValidation, sec ini.Section, key string) error {
if !sec.Has(key) {
return nil
}
v := sec.String(key)
switch strings.ToLower(v) {
case checksumWhenSupported:
*m = aws.ResponseChecksumValidationWhenSupported
case checksumWhenRequired:
*m = aws.ResponseChecksumValidationWhenRequired
default:
return fmt.Errorf("invalid value for shared config profile field, %s=%s, must be when_supported/when_required", key, v)
}
return nil
}
func (c SharedConfig) getRequestMinCompressSizeBytes(ctx context.Context) (int64, bool, error) {
if c.RequestMinCompressSizeBytes == nil {
return 0, false, nil
@@ -1225,6 +1279,14 @@ func (c SharedConfig) getAccountIDEndpointMode(ctx context.Context) (aws.Account
return c.AccountIDEndpointMode, len(c.AccountIDEndpointMode) > 0, nil
}
func (c SharedConfig) getRequestChecksumCalculation(ctx context.Context) (aws.RequestChecksumCalculation, bool, error) {
return c.RequestChecksumCalculation, c.RequestChecksumCalculation > 0, nil
}
func (c SharedConfig) getResponseChecksumValidation(ctx context.Context) (aws.ResponseChecksumValidation, bool, error) {
return c.ResponseChecksumValidation, c.ResponseChecksumValidation > 0, nil
}
func updateDefaultsMode(mode *aws.DefaultsMode, section ini.Section, key string) error {
if !section.Has(key) {
return nil

View File

@@ -1,3 +1,20 @@
# v1.17.55 (2025-01-24)
* **Dependency Update**: Updated to the latest SDK module versions
* **Dependency Update**: Upgrade to smithy-go v1.22.2.
# v1.17.54 (2025-01-17)
* **Dependency Update**: Updated to the latest SDK module versions
# v1.17.53 (2025-01-15)
* **Dependency Update**: Updated to the latest SDK module versions
# v1.17.52 (2025-01-14)
* **Dependency Update**: Updated to the latest SDK module versions
# v1.17.51 (2025-01-10)
* **Dependency Update**: Updated to the latest SDK module versions

View File

@@ -3,4 +3,4 @@
package credentials
// goModuleVersion is the tagged release for this module
const goModuleVersion = "1.17.51"
const goModuleVersion = "1.17.55"

View File

@@ -1,3 +1,12 @@
# v1.16.25 (2025-01-24)
* **Dependency Update**: Updated to the latest SDK module versions
* **Dependency Update**: Upgrade to smithy-go v1.22.2.
# v1.16.24 (2025-01-15)
* **Dependency Update**: Updated to the latest SDK module versions
# v1.16.23 (2025-01-09)
* **Dependency Update**: Updated to the latest SDK module versions

Some files were not shown because too many files have changed in this diff Show More