Skip to main content
The S3 storage backend supports Amazon S3 and any S3-compatible service, including Cloudflare R2, MinIO, DigitalOcean Spaces, and more.

​
Configuration

​
Required Settings

​
CLOUDSTIC_STORE
string
required
Set to s3
​
CLOUDSTIC_STORE_PATH
string
required
The S3 bucket name (e.g., my-backup-bucket)
​
CLOUDSTIC_STORE_PREFIX
string
default:""
Optional prefix for all objects (e.g., backups/)
​
CLOUDSTIC_S3_REGION
string
default:"us-east-1"
AWS region for the bucket

​
Authentication

​
AWS_ACCESS_KEY_ID
string
required
AWS access key ID
​
AWS_SECRET_ACCESS_KEY
string
required
AWS secret access key
You can also use:
  • -s3-access-key and -s3-secret-key flags
  • AWS credential profiles (~/.aws/credentials)
  • IAM instance roles (EC2, ECS, Lambda)

​
Examples

# Set credentials
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# Initialize repository
cloudstic init \
  -store s3 \
  -store-path my-backup-bucket \
  -s3-region us-west-2

# Backup
cloudstic backup -source local -source-path ~/Documents

​
Features

​
High-Concurrency Uploads

The S3 store uses optimized connection pooling:
  • 128 concurrent uploads by default
  • HTTP connection reuse for reduced latency
  • Parallel PUT operations for chunk uploads
This provides significantly faster backups compared to sequential uploads.

​
Path-Style Addressing

For custom endpoints (R2, MinIO), Cloudstic automatically uses path-style addressing: 
# Virtual-hosted style (AWS S3 default)
https://my-bucket.s3.amazonaws.com/chunk/abc123

# Path style (MinIO, R2)
https://s3.example.com/my-bucket/chunk/abc123

​
Packfile Optimization

Packfiles bundle small objects to reduce PUT requests:
  • Default enabled for S3 backends
  • Reduces API costs significantly
  • Bundles objects less than 512KB into 8MB packs
  • Catalog stored in index/packs
cloudstic init \
  -store s3 \
  -store-path my-bucket \
  -enable-packfile=false

​
S3 Bucket Setup

​
Create a Bucket

# Using AWS CLI
aws s3 mb s3://my-backup-bucket --region us-west-2

# Enable versioning (recommended)
aws s3api put-bucket-versioning \
  --bucket my-backup-bucket \
  --versioning-configuration Status=Enabled

​
IAM Policy (AWS S3)

Create a minimal IAM policy for Cloudstic: 
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:ListBucket",
        "s3:HeadObject"
      ],
      "Resource": [
        "arn:aws:s3:::my-backup-bucket",
        "arn:aws:s3:::my-backup-bucket/*"
      ]
    }
  ]
}

​
Performance Optimization

​
Region Selection

Choose a region close to your location: 
# US West (faster for West Coast)
cloudstic init -store s3 -store-path my-bucket -s3-region us-west-1

# EU (faster for Europe)
cloudstic init -store s3 -store-path my-bucket -s3-region eu-west-1

​
Transfer Acceleration (AWS S3)

Enable S3 Transfer Acceleration for faster uploads over long distances: 
# Enable on the bucket
aws s3api put-bucket-accelerate-configuration \
  --bucket my-backup-bucket \
  --accelerate-configuration Status=Enabled

# Use accelerated endpoint
export CLOUDSTIC_S3_ENDPOINT=https://my-backup-bucket.s3-accelerate.amazonaws.com

​
Cost Optimization

​
Storage Classes

Use lifecycle policies to transition old backups to cheaper storage: 
# Example: Transition to Glacier after 90 days
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-backup-bucket \
  --lifecycle-configuration file://lifecycle.json
lifecycle.json: 
{
  "Rules": [
    {
      "Id": "ArchiveOldBackups",
      "Status": "Enabled",
      "Prefix": "snapshot/",
      "Transitions": [
        {
          "Days": 90,
          "StorageClass": "GLACIER"
        }
      ]
    }
  ]
}
If using lifecycle policies, remember that Glacier retrieval takes hours and has additional costs.

​
Request Reduction

Enable packfiles to reduce PUT request costs:
  • Without packfiles: 1 PUT per chunk/content/filemeta object
  • With packfiles: Objects bundled into 8MB packs
For 10,000 files, this can reduce from 30,000+ PUTs to just a few hundred.

​
Troubleshooting

​
Credential Errors

If you see “Unable to locate credentials”: 
# Verify credentials are set
echo $AWS_ACCESS_KEY_ID
echo $AWS_SECRET_ACCESS_KEY

# Test with AWS CLI
aws s3 ls s3://my-backup-bucket

# Or set explicitly
cloudstic backup \
  -s3-access-key AKIAIOSFODNN7EXAMPLE \
  -s3-secret-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

​
Region Mismatch

If you see “bucket is in this region” errors: 
# Get the correct region
aws s3api get-bucket-location --bucket my-backup-bucket

# Use the correct region
cloudstic init -store s3 -store-path my-backup-bucket -s3-region us-west-2

​
Endpoint Configuration (R2/MinIO)

If custom endpoints fail: 
# Ensure endpoint includes protocol
export CLOUDSTIC_S3_ENDPOINT=https://minio.example.com:9000  # Correct
export CLOUDSTIC_S3_ENDPOINT=minio.example.com:9000          # Wrong

# Test connectivity
curl -I $CLOUDSTIC_S3_ENDPOINT

Backup Command

Create backups to S3

B2 Storage

Alternative cloud storage

Local Storage

Use local filesystem

Encryption

Data encryption details