Cloud-Sync Mode | XMigrate Docs

Overview

The cloud-sync service uploads snapshots and chunks to S3-compatible cloud object storage. It reads configuration from cloud-config.yaml in the repository directory.

Manual Usage


# Dry-run (default) — prints planned actions without uploading
xreplicator cloud-sync --repo /var/lib/backup/repo
 
# Actually upload
xreplicator cloud-sync --repo /var/lib/backup/repo --dry-run=false
 
# Run continuously on an interval
xreplicator cloud-sync --repo /var/lib/backup/repo --once=false --interval 1h
 
# Use a custom cloud config
xreplicator cloud-sync --repo /var/lib/backup/repo --cloud-config /etc/xreplicator/cloud-config.yaml

Flag	Default	Description
`--repo`	—	Repository path (required)
`--cloud-config`	`<repo>/cloud-config.yaml`	Cloud config path
`--once`	`true`	Run one sync iteration and exit
`--interval`	—	Repeat interval when `--once=false` (e.g., `"1h"`)
`--dry-run`	`true`	Print planned actions without uploading
`--concurrency`	`4`	Parallel chunk upload workers per iteration

You can also schedule Cloud Sync from the Web UI: go to Settings and configure the Cloud Sync schedule there. You can also create and update Cloud Sync policies/mappings from the UI itself.

Cloud Configuration File

The cloud config file (cloud-config.yaml) contains three sections:

storage_configs — Cloud storage backends
target_mappings — Which devices go to which storage
compaction — Shared with the compactor (see Compactor Mode)

Storage Configuration

Each entry defines a cloud storage backend.

Parameters

Parameter	Type	Required	Description
`name`	string	Yes	Unique name, referenced by target mappings
`provider`	string	Yes	Storage provider — currently `"s3"`
`endpoint`	string	No	Custom endpoint for MinIO or self-hosted S3
`region`	string	No	Storage region (required for AWS)
`bucket`	string	Yes	Bucket name
`prefix`	string	No	Key prefix for all objects (e.g., `"backups/"`)
`use_ssl`	boolean	No	Use SSL/TLS — recommended `true` for production
`credentials`	object	Yes	Auth credentials (`static` or `secret_ref`)

Static Credentials


credentials:
  static:
    access_key: "YOUR_ACCESS_KEY"
    secret_key: "YOUR_SECRET_KEY"
    session_token: "OPTIONAL_SESSION_TOKEN"  # for temporary credentials

Kubernetes Secret References


credentials:
  secret_ref:
    mount_path: "/etc/replicator/secrets"
    access_key_file: "access_key"
    secret_key_file: "secret_key"

Secrets Mode


secrets_mode: "standalone"  # Use static credentials (for VMs)
# or
secrets_mode: "k8s"         # Use Kubernetes secret references (default)

Target Mappings

Target mappings route devices to storage backends.

Parameters

Parameter	Type	Description
`name`	string	Unique mapping name
`hostnames`	string[]	Exact hostnames to match
`hostname_globs`	string[]	Glob patterns for hostnames
`device_paths`	string[]	Exact device paths to match
`device_path_globs`	string[]	Glob patterns for device paths
`storage_config_name`	string	Must match a name in `storage_configs`

How Cloud Sync Works

Config Loading — Reads the cloud configuration file
Target Discovery — Finds all hostnames and devices in the repository
Mapping Evaluation — Matches targets to storage configs
Snapshot Upload — Uploads snapshot metadata
Chunk Upload — Uploads referenced chunks (with deduplication)
Progress Tracking — Tracks upload progress and errors

Configuration Examples

AWS S3


secrets_mode: "standalone"
 
storage_configs:
  - name: "aws-backup"
    provider: "s3"
    region: "us-west-2"
    bucket: "my-backup-bucket"
    prefix: "replicator/"
    use_ssl: true
    credentials:
      static:
        access_key: "AKIAIOSFODNN7EXAMPLE"
        secret_key: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
 
target_mappings:
  - name: "all-devices"
    hostname_globs: ["*"]
    device_path_globs: ["*"]
    storage_config_name: "aws-backup"

MinIO (Local)


secrets_mode: "standalone"
 
storage_configs:
  - name: "local-minio"
    provider: "s3"
    endpoint: "localhost:9000"
    region: "us-east-1"
    bucket: "backup-test"
    prefix: "replicator/"
    use_ssl: false
    credentials:
      static:
        access_key: "minioadmin"
        secret_key: "minioadmin"
 
target_mappings:
  - name: "all-to-minio"
    hostname_globs: ["*"]
    device_path_globs: ["*"]
    storage_config_name: "local-minio"

Multiple Storage Targets


secrets_mode: "standalone"
 
storage_configs:
  - name: "aws-production"
    provider: "s3"
    region: "us-west-2"
    bucket: "prod-backups"
    use_ssl: true
    credentials:
      static:
        access_key: "PROD_KEY"
        secret_key: "PROD_SECRET"
 
  - name: "aws-archive"
    provider: "s3"
    region: "us-east-1"
    bucket: "archive-backups"
    use_ssl: true
    credentials:
      static:
        access_key: "ARCHIVE_KEY"
        secret_key: "ARCHIVE_SECRET"
 
target_mappings:
  - name: "production-servers"
    hostname_globs: ["prod-*"]
    storage_config_name: "aws-production"
 
  - name: "archive-servers"
    hostname_globs: ["archive-*"]
    storage_config_name: "aws-archive"

Security Considerations

Credentials — Use Kubernetes secrets (k8s mode) where possible; avoid hardcoding in config files
TLS — Always set use_ssl: true in production
IAM — Use IAM roles with minimal required permissions
Encryption — Enable bucket-level encryption on your cloud storage provider