mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2026-07-13 12:20:33 +03:00
Compare commits
10 Commits
lockless-p
...
docs/file-
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c469f5d9d4 | ||
|
|
de098add09 | ||
|
|
627eb7c804 | ||
|
|
ed715a89ac | ||
|
|
f16bee6973 | ||
|
|
c0edf0c983 | ||
|
|
81262b2fb5 | ||
|
|
2aec74f23b | ||
|
|
d3f86b24a1 | ||
|
|
60de3bad8e |
426
docs/guides/connecting-vm-components-to-cloud-storage/README.md
Normal file
426
docs/guides/connecting-vm-components-to-cloud-storage/README.md
Normal file
@@ -0,0 +1,426 @@
|
||||
Several VictoriaMetrics components can connect to cloud storage to read or write object data.
|
||||
|
||||
The following table shows the supported types of storage for each component:
|
||||
|
||||
| Component | AWS S3 and S3-compatible | Google Cloud Storage | Azure Blob Storage |
|
||||
|-----------|----|----------------------|--------------------|
|
||||
| [vmbackup](https://docs.victoriametrics.com/victoriametrics/vmbackup/) | ✅ | ✅ | ✅ |
|
||||
| [vmrestore](https://docs.victoriametrics.com/victoriametrics/vmrestore/) | ✅ | ✅ | ✅ |
|
||||
| [vmbackupmanager](https://docs.victoriametrics.com/victoriametrics/vmbackupmanager/) | ✅ | ✅ | ✅ |
|
||||
| [vmalert](https://docs.victoriametrics.com/victoriametrics/vmalert/) | ✅ | ✅ | ❌ |
|
||||
|
||||
All these components use the same underlying libraries, so the authentication setup is largely the same. The main difference is in the command-line flags:
|
||||
|
||||
- vmalert uses `-s3.*` prefixed flags (e.g., `-s3.credsFilePath`)
|
||||
- backup and restore tools use unprefixed flags (e.g., `-credsFilePath`)
|
||||
|
||||
See the [component reference](https://docs.victoriametrics.com/guides/connecting-vm-components-to-cloud-storage/#per-component-flag-reference) for details.
|
||||
|
||||
## Obtaining credentials
|
||||
|
||||
You need to supply credentials so the component can connect to the cloud storage. The setup differs by provider; the sections below cover AWS S3, S3‑compatible systems, GCS, and Azure Blob Storage.
|
||||
|
||||
### AWS S3
|
||||
|
||||
1. In AWS, [create an IAM user](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html) or role with the minimum permissions for the target bucket (see table below).
|
||||
1. [Create an access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) for that IAM identity.
|
||||
1. Copy the **Access key ID** and **Secret access key** values. You will use them in the credentials file or environment variables.
|
||||
|
||||
| Component | S3 usage | Minimum S3 permissions |
|
||||
|-----------------|-------------------------------------------------------------|------------------------|
|
||||
| vmalert (Enterprise) | Reads alerting/recording rules from bucket. | `s3:GetObject`, `s3:ListBucket` on the rules bucket/prefix.|
|
||||
| vmrestore | Restores backups from S3 buckets | `s3:GetObject`, `s3:ListBucket` on the backup bucket/prefix.|
|
||||
| vmbackup | Uploads backups to S3 and may delete old backup objects during maintenance.| `s3:PutObject`, `s3:GetObject`, `s3:ListBucket`, `s3:DeleteObject` on the backup bucket/prefix.|
|
||||
| vmbackupmanager (Enterprise) | Automates backups using vmbackup behavior. | Same as vmbackup. |
|
||||
|
||||
|
||||
### S3-compatible storage (MinIO, Ceph)
|
||||
|
||||
Generate access keys using your storage system's admin interface or CLI. The credentials follow the same format as AWS S3.
|
||||
|
||||
### Google Cloud Storage
|
||||
|
||||
1. Open the Google Cloud Console and go to **IAM & Admin > Service Accounts**.
|
||||
1. Click **Create service account**, enter a name, and assign a Storage role (for example, Storage Object Admin).
|
||||
1. Open the service account, go to **Keys**, then click **Add key > Create new key**.
|
||||
1. Choose **JSON** as the key type and click **Create**.
|
||||
1. Store the JSON file on the machine running the VictoriaMetrics component.
|
||||
|
||||
### Azure Blob Storage
|
||||
|
||||
> Azure does not use credential files.
|
||||
|
||||
1. Log in to the Azure Portal.
|
||||
1. Search for and select **Storage accounts**.
|
||||
1. Click on your specific storage account name. (This is your `AZURE_STORAGE_ACCOUNT_NAME`).
|
||||
1. In the left menu under **Security + networking**, click **Access keys**.
|
||||
1. Copy the key value from either **key1** or **key2** (this is your `AZURE_STORAGE_ACCOUNT_KEY`).
|
||||
1. Define the access keys as environment variables.
|
||||
```sh
|
||||
export AZURE_STORAGE_ACCOUNT_NAME=mystorageaccount
|
||||
export AZURE_STORAGE_ACCOUNT_KEY=myaccountkey
|
||||
```
|
||||
|
||||
## Authenticating with the cloud provider
|
||||
|
||||
Provide the credentials as a file or with environment variables, along with the path to the cloud storage bucket. The syntax for the bucket name depends on the cloud provider:
|
||||
|
||||
- `s3://`: for AWS S3 and self-hosted S3-compatible storage (MinIO, Ceph)
|
||||
- `gs://`: Google Cloud Storage
|
||||
- `azblob://`: Azure Blob Storage
|
||||
|
||||
### vmbackup and vmrestore
|
||||
|
||||
The following example backups to an AWS S3 bucket using a credentials file:
|
||||
|
||||
```sh
|
||||
vmbackup \
|
||||
-storageDataPath=/data \
|
||||
-snapshot.createURL=http://localhost:8428/snapshot/create \
|
||||
-dst=s3://victoriametrics-backup/backup01 \
|
||||
-credsFilePath=/etc/credentials
|
||||
```
|
||||
|
||||
In order to restore from the same backup from AWS S3:
|
||||
|
||||
```sh
|
||||
vmrestore \
|
||||
-src=s3://victoriametrics-backup/backup01 \
|
||||
-storageDataPath=/data \
|
||||
-credsFilePath=/etc/credentials
|
||||
|
||||
```
|
||||
|
||||
> To use non-AWS S3 buckets such as MinIO or Ceph, you must [supply the `-customS3Endpoint` argument](https://docs.victoriametrics.com/guides/connecting-vm-components-to-cloud-storage/#s3-compatible).
|
||||
|
||||
Alternatively, you can set the access keys as environment variables instead of using a credential file:
|
||||
|
||||
```sh
|
||||
export AWS_ACCESS_KEY_ID=YOUR_AWS_ACCESS_KEY
|
||||
export AWS_SECRET_ACCESS_KEY=YOUR_SECRET_AWS_ACCESS_KEY
|
||||
|
||||
vmbackup \
|
||||
-storageDataPath=/data \
|
||||
-snapshot.createURL=http://localhost:8428/snapshot/create \
|
||||
-dst=s3://victoriametrics-backup/backup01
|
||||
|
||||
vmrestore \
|
||||
-src=s3://victoriametrics-backup/backup01 \
|
||||
-storageDataPath=/data
|
||||
```
|
||||
|
||||
|
||||
Backups on Google Cloud Storage use the `gs://` prefix in the destination:
|
||||
|
||||
```sh
|
||||
vmbackup \
|
||||
-storageDataPath=/data \
|
||||
-snapshot.createURL=http://localhost:8428/snapshot/create \
|
||||
-dst=gs://victoriametrics-backup/backup01 \
|
||||
-credsFilePath=/etc/credentials
|
||||
```
|
||||
|
||||
You can restore this backup with:
|
||||
|
||||
```sh
|
||||
vmrestore \
|
||||
-src=gs://victoriametrics-backup/backup01 \
|
||||
-storageDataPath=/data \
|
||||
-credsFilePath=/etc/credentials
|
||||
```
|
||||
|
||||
On Google Cloud, you can define the path to the JSON credential file with `GOOGLE_APPLICATION_CREDENTIALS`. For example:
|
||||
|
||||
```sh
|
||||
export GOOGLE_APPLICATION_CREDENTIALS=/etc/credentials
|
||||
|
||||
vmbackup \
|
||||
-storageDataPath=/data \
|
||||
-snapshot.createURL=http://localhost:8428/snapshot/create \
|
||||
-dst=gs://victoriametrics-backup/backup01
|
||||
|
||||
vmrestore \
|
||||
-src=gs://victoriametrics-backup/backup01 \
|
||||
-storageDataPath=/data
|
||||
```
|
||||
|
||||
For Azure Blob Storage, use the `azblob://` prefix and rely on environment variables instead of `-credsFilePath`.
|
||||
|
||||
```sh
|
||||
export AZURE_STORAGE_ACCOUNT_NAME=myaccount
|
||||
export AZURE_STORAGE_ACCOUNT_KEY=mykey
|
||||
|
||||
vmbackup \
|
||||
-storageDataPath=/data \
|
||||
-snapshot.createURL=http://localhost:8428/snapshot/create \
|
||||
-dst=azblob://victoriametrics-backup/backup01
|
||||
|
||||
vmrestore \
|
||||
-src=azblob://victoriametrics-backup/backup01 \
|
||||
-storageDataPath=/data
|
||||
```
|
||||
|
||||
### vmbackupmanager
|
||||
|
||||
> vmbackupmanager only works in the [Enterprise](https://docs.victoriametrics.com/victoriametrics/enterprise/) edition.
|
||||
|
||||
To manage backups with vmbackupmanager on AWS S3, add the credentials with the `-credsFilePath` flag:
|
||||
|
||||
```sh
|
||||
vmbackupmanager \
|
||||
-dst=s3://vmstorage-data/backups \
|
||||
-credsFilePath=/etc/credentials \
|
||||
-storageDataPath=/vmstorage-data \
|
||||
-snapshot.createURL=http://vmstorage:8482/snapshot/create \
|
||||
-licenseFile=/etc/vm-license
|
||||
```
|
||||
|
||||
Or define the access keys using environment variables:
|
||||
|
||||
```sh
|
||||
export AWS_ACCESS_KEY_ID=YOUR_AWS_ACCESS_KEY
|
||||
export AWS_SECRET_ACCESS_KEY=YOUR_SECRET_AWS_ACCESS_KEY
|
||||
|
||||
vmbackupmanager \
|
||||
-dst=s3://vmstorage-data/backups \
|
||||
-storageDataPath=/vmstorage-data \
|
||||
-snapshot.createURL=http://vmstorage:8482/snapshot/create \
|
||||
-licenseFile=/etc/vm-license
|
||||
```
|
||||
|
||||
> To use non-AWS S3 buckets, you must [supply the `-customS3Endpoint` argument](#s3-compatible-endpoints).
|
||||
|
||||
Automated backups on Google Cloud Storage take the following form:
|
||||
|
||||
```sh
|
||||
vmbackupmanager \
|
||||
-dst=gs://vmstorage-data/backups \
|
||||
-credsFilePath=/etc/credentials \
|
||||
-storageDataPath=/vmstorage-data \
|
||||
-snapshot.createURL=http://vmstorage:8482/snapshot/create \
|
||||
-licenseFile=/etc/vm-license
|
||||
```
|
||||
|
||||
As with vmbackup and vmrestore, you can also define the path to the JSON credential file with `GOOGLE_APPLICATION_CREDENTIALS`:
|
||||
|
||||
```sh
|
||||
export GOOGLE_APPLICATION_CREDENTIALS=/etc/credentials
|
||||
|
||||
vmbackupmanager \
|
||||
-dst=gs://vmstorage-data/backups \
|
||||
-storageDataPath=/vmstorage-data \
|
||||
-snapshot.createURL=http://vmstorage:8482/snapshot/create \
|
||||
-licenseFile=/etc/vm-license
|
||||
```
|
||||
|
||||
vmbackupmanager can also use Azure Blob Storage by defining environment variables:
|
||||
|
||||
```sh
|
||||
export AZURE_STORAGE_ACCOUNT_NAME=mystorageaccount
|
||||
export AZURE_STORAGE_ACCOUNT_KEY=myaccountkey
|
||||
|
||||
vmbackupmanager \
|
||||
-dst=azblob://vmstorage-data/backups \
|
||||
-storageDataPath=/vmstorage-data \
|
||||
-snapshot.createURL=http://vmstorage:8482/snapshot/create \
|
||||
-licenseFile=/etc/vm-license
|
||||
```
|
||||
|
||||
### vmalert
|
||||
|
||||
> - vmalert cloud storage command line flags are prefixed with `-s3.` for S3 buckets *and* Google Cloud Storage.
|
||||
> - Cloud storage only works in the [Enterprise](https://docs.victoriametrics.com/victoriametrics/enterprise/) edition.
|
||||
|
||||
Read alerting rules from an S3 bucket. The `-rule` flag accepts a prefix, so it matches all files starting with `alerts_` in the `rules` folder:
|
||||
|
||||
```sh
|
||||
vmalert \
|
||||
-rule=s3://my-alert-bucket/rules/alerts_ \
|
||||
-s3.credsFilePath=/etc/vmalert/aws-credentials \
|
||||
-datasource.url=http://vmselect:8481/select/0/prometheus \
|
||||
-notifier.url=http://alertmanager:9093 \
|
||||
-licenseFile=/etc/vm-license
|
||||
```
|
||||
|
||||
Instead of a credential file, you can supply the access keys using environment variables:
|
||||
|
||||
```sh
|
||||
export AWS_ACCESS_KEY_ID=YOUR_AWS_ACCESS_KEY
|
||||
export AWS_SECRET_ACCESS_KEY=YOUR_SECRET_AWS_ACCESS_KEY
|
||||
|
||||
vmalert \
|
||||
-rule=s3://my-alert-bucket/rules/alerts_ \
|
||||
-datasource.url=http://vmselect:8481/select/0/prometheus \
|
||||
-notifier.url=http://alertmanager:9093 \
|
||||
-licenseFile=/etc/vm-license
|
||||
```
|
||||
|
||||
> To use non-AWS S3 buckets, you must [supply the `-s3.customEndpoint` argument](#s3-compatible-endpoints).
|
||||
|
||||
To read rules from Google Cloud Storage:
|
||||
|
||||
```sh
|
||||
vmalert \
|
||||
-rule=gs://my-alert-bucket/rules/alerts_ \
|
||||
-s3.credsFilePath=/etc/vmalert/gcp-service-account.json \
|
||||
-datasource.url=http://vmselect:8481/select/0/prometheus \
|
||||
-notifier.url=http://alertmanager:9093 \
|
||||
-licenseFile=/etc/vm-license
|
||||
```
|
||||
|
||||
If you prefer, you can supply the path to the JSON credential file with the `GOOGLE_APPLICATION_CREDENTIALS` environment variable:
|
||||
|
||||
```sh
|
||||
export GOOGLE_APPLICATION_CREDENTIALS=/etc/credentials
|
||||
|
||||
vmalert \
|
||||
-rule=gs://my-alert-bucket/rules/alerts_ \
|
||||
-datasource.url=http://vmselect:8481/select/0/prometheus \
|
||||
-notifier.url=http://alertmanager:9093 \
|
||||
-licenseFile=/etc/vm-license
|
||||
```
|
||||
|
||||
## Credentials files format
|
||||
|
||||
The file format depends on the storage provider.
|
||||
|
||||
### S3 credentials
|
||||
|
||||
The file uses the standard AWS shared credentials format used by the [AWS CLI](https://docs.aws.amazon.com/cli/v1/userguide/cli-configure-files.html) and [AWS SDKs](https://docs.aws.amazon.com/sdkref/latest/guide/file-format.html):
|
||||
|
||||
```ini
|
||||
[default]
|
||||
aws_access_key_id=YOUR_AWS_ACCESS_KEY
|
||||
aws_secret_access_key=YOUR_AWS_SECRET_ACCESS_KEY
|
||||
```
|
||||
|
||||
You can define multiple profiles in a single file:
|
||||
|
||||
```ini
|
||||
[default]
|
||||
aws_access_key_id=DEFAULT_ACCESS_KEY
|
||||
aws_secret_access_key=DEFAULT_SECRET_KEY
|
||||
|
||||
[monitoring]
|
||||
aws_access_key_id=MONITORING_ACCESS_KEY
|
||||
aws_secret_access_key=MONITORING_SECRET_KEY
|
||||
|
||||
[alerts]
|
||||
aws_access_key_id=ALERTS_ACCESS_KEY
|
||||
aws_secret_access_key=ALERTS_SECRET_KEY
|
||||
```
|
||||
|
||||
Use the `-configProfile` flag (or `-s3.configProfile` in vmalert) to select a non-default profile:
|
||||
|
||||
```sh
|
||||
-configProfile=alerts
|
||||
```
|
||||
|
||||
You can separate credentials from other configuration settings. Put credentials in one file:
|
||||
|
||||
```ini
|
||||
[default]
|
||||
aws_access_key_id=DEFAULT_ACCESS_KEY
|
||||
aws_secret_access_key=DEFAULT_SECRET_KEY
|
||||
```
|
||||
|
||||
And non-sensitive settings in another:
|
||||
|
||||
```ini
|
||||
[default]
|
||||
region=us-east-1
|
||||
```
|
||||
|
||||
Then pass both files:
|
||||
|
||||
```sh
|
||||
-configFilePath=/etc/aws-config \
|
||||
-credsFilePath=/etc/credentials
|
||||
```
|
||||
|
||||
### GCS credentials file format
|
||||
|
||||
The file is the JSON key downloaded from Google Cloud Console. Its content looks like this:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "service_account",
|
||||
"project_id": "project-id",
|
||||
"private_key_id": "key-id",
|
||||
"private_key": "-----BEGIN PRIVATE KEY-----\nprivate-key\n-----END PRIVATE KEY-----\n",
|
||||
"client_email": "service-account-email",
|
||||
"client_id": "client-id",
|
||||
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
|
||||
"token_uri": "https://accounts.google.com/o/oauth2/token",
|
||||
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
|
||||
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/service-account-email"
|
||||
}
|
||||
```
|
||||
|
||||
This is the standard service account key format defined by [Google Cloud IAM](https://developers.google.com/workspace/guides/create-credentials).
|
||||
|
||||
### Azure Blob Storage
|
||||
|
||||
Azure does not support credentials via file. Use environment variables instead.
|
||||
|
||||
```sh
|
||||
export AZURE_STORAGE_ACCOUNT_NAME=mystorageaccount
|
||||
export AZURE_STORAGE_ACCOUNT_KEY=myaccountkey
|
||||
```
|
||||
|
||||
## Self-hosted and S3-compatible endpoints {#s3-compatible}
|
||||
|
||||
For S3-compatible storage such as MinIO or Ceph, set a custom endpoint with the `-customS3Endpoint` flag for vmbackup, vmrestore, and vmbackupmanager. For example:
|
||||
|
||||
```sh
|
||||
vmbackup \
|
||||
-storageDataPath=/data \
|
||||
-snapshot.createURL=http://localhost:8428/snapshot/create \
|
||||
-dst=s3://victoriametrics-backup/backup01 \
|
||||
-customS3Endpoint=http://minio.example.local:9000
|
||||
```
|
||||
|
||||
On vmalert, use the `-s3.customEndpoint` flag instead:
|
||||
|
||||
```sh
|
||||
vmalert \
|
||||
-rule=s3://my-alert-bucket/rules/alerts_ \
|
||||
-s3.customEndpoint=http://minio.example.local:9000 \
|
||||
-s3.credsFilePath=/etc/vmalert/aws-credentials \
|
||||
-datasource.url=http://vmselect:8481/select/0/prometheus \
|
||||
-notifier.url=http://alertmanager:9093 \
|
||||
-licenseFile=/etc/vm-license
|
||||
```
|
||||
|
||||
### Addressing S3-compatible buckets
|
||||
|
||||
When connecting to non-AWS S3-compatible buckets, there is an additional flag you might need to configure:
|
||||
|
||||
- `-s3ForcePathStyle`: on vmbackupmanager, vmbackup, and vmrestore.
|
||||
- `-s3.forcePathStyle`: on vmalert.
|
||||
|
||||
The flag changes the expected URL pattern for a bucket.
|
||||
|
||||
| Flag value | Address-style | Example | Use with |
|
||||
|------------|---------------|---------|----------|
|
||||
| `true` (default) | Path-style | `https://endpoint/bucket/key` | MinIO, Ceph, most S3-compatible storages |
|
||||
| `false` | Virtual host-style | `https://bucket.endpoint/key` | [Aliyun OSS](https://www.aliyun.com/product/oss) and other endpoints that require it |
|
||||
|
||||
> The flag only takes effect when you use a custom endpoint (`-customS3Endpoint` or `-s3.customEndpoint` on vmalert). When connecting to real AWS S3, the SDK handles addressing automatically.
|
||||
|
||||
## Per-component flag reference
|
||||
|
||||
The table below shows how the same concept maps to different flag names across components.
|
||||
|
||||
| Concept | vmalert | vmbackup, vmrestore, and vmbackupmanager |
|
||||
|---|---|---|---|---|
|
||||
| Credentials file | `-s3.credsFilePath` | `-credsFilePath` |
|
||||
| Config file | `-s3.configFilePath` | `-configFilePath` |
|
||||
| Profile selection | `-s3.configProfile` | `-configProfile` |
|
||||
| Custom endpoint | `-s3.customEndpoint` | `-customS3Endpoint` |
|
||||
| Force path style | `-s3.forcePathStyle` | `-s3ForcePathStyle` |
|
||||
| TLS insecure | N/A | `-s3TLSInsecureSkipVerify` |
|
||||
| Storage class | N/A | `-s3StorageClass` |
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
---
|
||||
weight: 5
|
||||
title: Connecting VictoriaMetrics components to cloud storage
|
||||
menu:
|
||||
docs:
|
||||
parent: guides
|
||||
weight: 5
|
||||
tags:
|
||||
- metrics
|
||||
- guide
|
||||
---
|
||||
{{% content "README.md" %}}
|
||||
@@ -546,7 +546,7 @@ tags at [Docker Hub](https://hub.docker.com/r/victoriametrics/vmalert/tags) and
|
||||
|
||||
## Reading rules from object storage
|
||||
|
||||
[Enterprise version](https://docs.victoriametrics.com/victoriametrics/enterprise/) of `vmalert` may read alerting and recording rules
|
||||
The [Enterprise version](https://docs.victoriametrics.com/victoriametrics/enterprise/) of `vmalert` may read alerting and recording rules
|
||||
from object storage:
|
||||
|
||||
* `./bin/vmalert -rule=s3://bucket/dir/alert.rules` would read rules from the given path at S3 bucket
|
||||
@@ -563,6 +563,48 @@ The following [command-line flags](#flags) can be used for fine-tuning access to
|
||||
* `-s3.customEndpoint` - custom S3 endpoint for use with S3-compatible storages (e.g. MinIO). S3 is used if not set.
|
||||
* `-s3.forcePathStyle` - prefixing endpoint with bucket name when set false, true by default.
|
||||
|
||||
### S3 (AWS and S3-compatible)
|
||||
|
||||
The following example reads rules from an S3 bucket:
|
||||
|
||||
```sh
|
||||
./bin/vmalert \
|
||||
-rule=s3://my-alert-bucket/rules/alerts_ \
|
||||
-s3.credsFilePath=/etc/vmalert/aws-credentials \
|
||||
-datasource.url=http://vmselect:8481/select/0/prometheus \
|
||||
-notifier.url=http://alertmanager:9093 \
|
||||
-licenseFile=/etc/vm-license
|
||||
```
|
||||
|
||||
For S3-compatible backends such as [MinIO](https://www.min.io/) or [Ceph](https://ceph.io/), add `-s3.customEndpoint`:
|
||||
|
||||
```sh
|
||||
./bin/vmalert \
|
||||
-rule=s3://victoriametrics-alert-rules/rules_ \
|
||||
-s3.credsFilePath=/etc/vmalert/aws-credentials \
|
||||
-s3.customEndpoint=http://minio.example.local:9000 \
|
||||
-datasource.url=http://vmselect:8481/select/0/prometheus \
|
||||
-notifier.url=http://alertmanager:9093 \
|
||||
-licenseFile=/etc/vm-license
|
||||
```
|
||||
|
||||
See [Connecting VM components to cloud storage](https://docs.victoriametrics.com/guides/connecting-vm-components-to-cloud-storage/) for details on creating IAM users, credentials file format, environment variables, and IAM roles.
|
||||
|
||||
### Google Cloud Storage (GCS)
|
||||
|
||||
The following example reads rules from a GCS bucket:
|
||||
|
||||
```sh
|
||||
./bin/vmalert \
|
||||
-rule=gs://my-alert-bucket/rules/alerts_ \
|
||||
-s3.credsFilePath=/etc/vmalert/gcp-service-account.json \
|
||||
-datasource.url=http://vmselect:8481/select/0/prometheus \
|
||||
-notifier.url=http://alertmanager:9093 \
|
||||
-licenseFile=/etc/vm-license
|
||||
```
|
||||
|
||||
See [Connecting VM components to cloud storage](https://docs.victoriametrics.com/guides/connecting-vm-components-to-cloud-storage/) for details on creating service accounts and downloading JSON keys.
|
||||
|
||||
## Topology examples
|
||||
|
||||
The following sections are showing how `vmalert` may be used and configured
|
||||
|
||||
@@ -204,61 +204,45 @@ See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time-
|
||||
|
||||
### Providing credentials as a file
|
||||
|
||||
Obtaining credentials from a file.
|
||||
See [Connecting VM components to cloud storage](https://docs.victoriametrics.com/guides/connecting-vm-components-to-cloud-storage/) for instructions on obtaining credentials and providing them via credential files, environment variables, cloud provider metadata service, or Kubernetes secrets and IAM roles.
|
||||
|
||||
Add flag `-credsFilePath=/etc/credentials` with the following content:
|
||||
The following examples show the most common authentication patterns for each storage provider.
|
||||
|
||||
* for S3 (AWS, MinIO or other S3 compatible storages):
|
||||
#### S3 (AWS and S3-compatible)
|
||||
|
||||
```sh
|
||||
[default]
|
||||
aws_access_key_id=theaccesskey
|
||||
aws_secret_access_key=thesecretaccesskeyvalue
|
||||
```
|
||||
```sh
|
||||
vmbackup \
|
||||
-storageDataPath=/data \
|
||||
-snapshot.createURL=http://localhost:8428/snapshot/create \
|
||||
-dst=s3://victoriametrics-backup/backup01 \
|
||||
-credsFilePath=/etc/credentials
|
||||
```
|
||||
|
||||
* for GCP cloud storage:
|
||||
#### Google Cloud Storage (GCS)
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "service_account",
|
||||
"project_id": "project-id",
|
||||
"private_key_id": "key-id",
|
||||
"private_key": "-----BEGIN PRIVATE KEY-----\nprivate-key\n-----END PRIVATE KEY-----\n",
|
||||
"client_email": "service-account-email",
|
||||
"client_id": "client-id",
|
||||
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
|
||||
"token_uri": "https://accounts.google.com/o/oauth2/token",
|
||||
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
|
||||
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/service-account-email"
|
||||
}
|
||||
```
|
||||
```sh
|
||||
vmbackup \
|
||||
-storageDataPath=/data \
|
||||
-snapshot.createURL=http://localhost:8428/snapshot/create \
|
||||
-dst=gs://victoriametrics-backup/backup01 \
|
||||
-credsFilePath=/etc/credentials
|
||||
```
|
||||
|
||||
### Providing credentials via env variables
|
||||
#### Azure Blob Storage
|
||||
|
||||
Obtaining credentials from env variables.
|
||||
```sh
|
||||
export AZURE_STORAGE_ACCOUNT_NAME=mystorageaccount
|
||||
export AZURE_STORAGE_ACCOUNT_KEY=myaccountkey
|
||||
|
||||
* For AWS S3 compatible storages set env variable `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`.
|
||||
Also you can set env variable `AWS_SHARED_CREDENTIALS_FILE` with path to credentials file.
|
||||
* For GCE cloud storage set env variable `GOOGLE_APPLICATION_CREDENTIALS` with path to credentials file.
|
||||
* For Azure storage use one of these env variables:
|
||||
* `AZURE_STORAGE_ACCOUNT_CONNECTION_STRING`: use a connection string (must be either SAS Token or Account/Key)
|
||||
* `AZURE_STORAGE_ACCOUNT_NAME` and `AZURE_STORAGE_ACCOUNT_KEY`: use a specific account name and key (either primary or secondary)
|
||||
* `AZURE_USE_DEFAULT_CREDENTIAL` and `AZURE_STORAGE_ACCOUNT_NAME`: use the `DefaultAzureCredential` to allow the Azure library
|
||||
to search for multiple options (for example, managed identity related variables). Note that if multiple credentials are available,
|
||||
it is required to specify the `AZURE_CLIENT_ID` to select specific credentials.
|
||||
|
||||
The `AZURE_STORAGE_DOMAIN` can be used for optionally overriding the default domain for the Azure storage service.
|
||||
|
||||
Please, note that `vmbackup` will use credentials provided by cloud providers metadata service [when applicable](https://docs.victoriametrics.com/victoriametrics/vmbackup/#using-cloud-providers-metadata-service).
|
||||
|
||||
### Using cloud providers metadata service
|
||||
|
||||
`vmbackup` and `vmbackupmanager` will automatically use cloud providers metadata service in order to obtain credentials if they are running in cloud environment and credentials are not explicitly provided via flags or env variables.
|
||||
vmbackup \
|
||||
-storageDataPath=/data \
|
||||
-snapshot.createURL=http://localhost:8428/snapshot/create \
|
||||
-dst=azblob://victoriametrics-backup/backup01
|
||||
```
|
||||
|
||||
### Providing credentials in Kubernetes
|
||||
|
||||
The simplest way to provide credentials in Kubernetes is to use [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/)
|
||||
and inject them into the pod as environment variables. For example, the following secret can be used for AWS S3 credentials:
|
||||
The simplest way to provide credentials in Kubernetes is to use [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) and inject them into the pod as environment variables. For example, the following secret can be used for AWS S3 credentials:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
@@ -266,14 +250,13 @@ kind: Secret
|
||||
metadata:
|
||||
name: vmbackup-credentials
|
||||
data:
|
||||
access_key: key
|
||||
secret_key: secret
|
||||
access_key: <base64-encoded-key>
|
||||
secret_key: <base64-encoded-secret>
|
||||
```
|
||||
|
||||
And then it can be injected into the pod as environment variables:
|
||||
|
||||
```yaml
|
||||
...
|
||||
env:
|
||||
- name: AWS_ACCESS_KEY_ID
|
||||
valueFrom:
|
||||
@@ -285,7 +268,6 @@ env:
|
||||
secretKeyRef:
|
||||
key: secret_key
|
||||
name: vmbackup-credentials
|
||||
...
|
||||
```
|
||||
|
||||
A more secure way is to use IAM roles to provide tokens for pods instead of managing credentials manually.
|
||||
|
||||
@@ -88,34 +88,21 @@ There are two flags which could help with performance tuning:
|
||||
|
||||
### Example of Usage
|
||||
|
||||
GCS and cluster version. You need to have a credentials file in json format with following structure:
|
||||
|
||||
credentials.json
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "service_account",
|
||||
"project_id": "<project>",
|
||||
"private_key_id": "",
|
||||
"private_key": "-----BEGIN PRIVATE KEY-----\-----END PRIVATE KEY-----\n",
|
||||
"client_email": "test@<project>.iam.gserviceaccount.com",
|
||||
"client_id": "",
|
||||
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
|
||||
"token_uri": "https://oauth2.googleapis.com/token",
|
||||
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
|
||||
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/test%40<project>.iam.gserviceaccount.com"
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
Backup manager launched with the following configuration:
|
||||
The following command backs up data to a GCS bucket:
|
||||
|
||||
```sh
|
||||
export NODE_IP=192.168.0.10
|
||||
export VMSTORAGE_ENDPOINT=http://127.0.0.1:8428
|
||||
./vmbackupmanager -dst=gs://vmstorage-data/$NODE_IP -credsFilePath=credentials.json -storageDataPath=/vmstorage-data -snapshot.createURL=$VMSTORAGE_ENDPOINT/snapshot/create -licenseFile=/path/to/vm-license
|
||||
./vmbackupmanager \
|
||||
-dst=gs://vmstorage-data/$NODE_IP \
|
||||
-credsFilePath=credentials.json \
|
||||
-storageDataPath=/vmstorage-data \
|
||||
-snapshot.createURL=$VMSTORAGE_ENDPOINT/snapshot/create \
|
||||
-licenseFile=/path/to/vm-license
|
||||
```
|
||||
|
||||
See [Connecting VM components to cloud storage](https://docs.victoriametrics.com/guides/connecting-vm-components-to-cloud-storage/) for instructions on obtaining credentials and configuring access to S3, GCS, and Azure Blob Storage.
|
||||
|
||||
Expected logs in vmbackupmanager:
|
||||
|
||||
```sh
|
||||
@@ -147,8 +134,7 @@ objects. Typical object storage systems implement server-side copy by creating n
|
||||
This is very fast and efficient. Unfortunately there are systems such as [S3 Glacier](https://aws.amazon.com/s3/storage-classes/glacier/),
|
||||
which perform full object copy during server-side copying. This may be slow and expensive.
|
||||
|
||||
Please, see [vmbackup docs](https://docs.victoriametrics.com/victoriametrics/vmbackup/#advanced-usage) for more examples of authentication with different
|
||||
storage types.
|
||||
See [Connecting VM components to cloud storage](https://docs.victoriametrics.com/guides/connecting-vm-components-to-cloud-storage/) for more examples of authentication with different storage types.
|
||||
|
||||
### Backup Retention Policy
|
||||
|
||||
|
||||
@@ -47,13 +47,13 @@ i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/q
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
* See [how to setup credentials via environment variables](https://docs.victoriametrics.com/victoriametrics/vmbackup/#providing-credentials-via-env-variables).
|
||||
* See [Connecting VM components to cloud storage](https://docs.victoriametrics.com/guides/connecting-vm-components-to-cloud-storage/) for instructions on configuring credentials.
|
||||
* If `vmrestore` consumes all the network bandwidth, then set `-maxBytesPerSecond` to the desired value.
|
||||
* If `vmrestore` has been interrupted due to temporary error, then just restart it with the same args. It will resume the restore process.
|
||||
|
||||
## Advanced usage
|
||||
|
||||
Please, see [vmbackup docs](https://docs.victoriametrics.com/victoriametrics/vmbackup/#advanced-usage) for examples of authentication
|
||||
See [Connecting VM components to cloud storage](https://docs.victoriametrics.com/guides/connecting-vm-components-to-cloud-storage/) for examples of authentication
|
||||
with different storage types.
|
||||
|
||||
Run `vmrestore -help` in order to see all the available options:
|
||||
|
||||
Reference in New Issue
Block a user