SFTP Source - Cloudstic CLI

The SFTP source backs up files from a remote server via SSH File Transfer Protocol. This allows you to back up remote servers without installing Cloudstic on them.

Basic Usage

Back up a remote directory via SFTP:

cloudstic backup -source sftp \
  -source-path /home/user/data \
  -source-sftp-host example.com \
  -source-sftp-user myuser \
  -source-sftp-password "mypassword"

Configuration Options

Required Flags

Flag	Description	Environment Variable
`-source sftp`	Specifies SFTP source	`CLOUDSTIC_SOURCE`
`-source-path <path>`	Remote directory path	`CLOUDSTIC_SOURCE_PATH`
`-source-sftp-host <host>`	SFTP server hostname	`CLOUDSTIC_SFTP_HOST`
`-source-sftp-user <user>`	SSH username	`CLOUDSTIC_SFTP_USER`

Authentication Flags

Flag	Description	Environment Variable
`-source-sftp-password <pw>`	SSH password	`CLOUDSTIC_SFTP_PASSWORD`
`-source-sftp-key <path>`	Path to SSH private key	`CLOUDSTIC_SFTP_KEY`
`-source-sftp-port <port>`	SSH port (default: 22)	`CLOUDSTIC_SFTP_PORT`

Optional Flags

Flag	Description
`-exclude <pattern>`	Exclude pattern (gitignore syntax, repeatable)
`-exclude-file <path>`	Load exclude patterns from file
`-tag <tag>`	Tag to apply to the snapshot (repeatable)
`-dry-run`	Scan source without writing to store

Authentication Methods

Cloudstic supports three SSH authentication methods:

1. Private Key (Recommended)

Use an SSH private key for authentication:

cloudstic backup -source sftp \
  -source-path /var/www \
  -source-sftp-host web-server.com \
  -source-sftp-user deploy \
  -source-sftp-key ~/.ssh/id_rsa

2. Password

Use password authentication:

cloudstic backup -source sftp \
  -source-path /home/user/files \
  -source-sftp-host example.com \
  -source-sftp-user myuser \
  -source-sftp-password "mypassword"

Avoid hardcoding passwords in scripts. Use environment variables or SSH keys instead.

3. SSH Agent (Automatic)

If SSH_AUTH_SOCK is set, Cloudstic automatically uses your SSH agent:

# Start SSH agent and add your key
ssh-add ~/.ssh/id_rsa

# Cloudstic will automatically use the agent
cloudstic backup -source sftp \
  -source-path /home/user/files \
  -source-sftp-host example.com \
  -source-sftp-user myuser

Examples

Backup Web Server

Back up a web server’s document root:

cloudstic backup -source sftp \
  -source-path /var/www/html \
  -source-sftp-host web01.example.com \
  -source-sftp-user www-data \
  -source-sftp-key ~/.ssh/deploy_key \
  -tag "webserver" \
  -tag "production"

Backup with Custom Port

Connect to a non-standard SSH port:

cloudstic backup -source sftp \
  -source-path /data \
  -source-sftp-host server.example.com \
  -source-sftp-port 2222 \
  -source-sftp-user backup \
  -source-sftp-key ~/.ssh/id_ed25519

Backup with Exclusions

Exclude cache and temporary files:

cloudstic backup -source sftp \
  -source-path /home/user/app \
  -source-sftp-host app-server.com \
  -source-sftp-user appuser \
  -source-sftp-password "$SFTP_PASSWORD" \
  -exclude "*.log" \
  -exclude "cache/" \
  -exclude "tmp/"

Multiple Server Backups

Back up multiple servers sequentially:

#!/bin/bash
set -e

SERVERS=("web01" "web02" "web03")

for server in "${SERVERS[@]}"; do
  echo "Backing up $server..."
  cloudstic backup -source sftp \
    -source-path /var/www \
    -source-sftp-host "$server.example.com" \
    -source-sftp-user deploy \
    -source-sftp-key ~/.ssh/deploy_key \
    -tag "webserver" \
    -tag "$server"
done

How It Works

Connection Process

Establish SSH connection

Cloudstic connects to the SFTP server using the provided credentials.

Authenticate

Authentication is attempted in order:

Private key (if specified)
Password (if specified)
SSH agent (if available)

Initialize SFTP session

An SFTP session is created over the SSH connection.

Walk directory tree

The remote directory is recursively scanned.

Stream files

Files are streamed over SFTP as they are backed up.

File Identification

SFTP files are identified by their relative path from the source root:

Source path: /home/user/data
File: /home/user/data/reports/Q1.pdf
File ID: reports/Q1.pdf

Source Information

Each snapshot records:

Type: sftp
Account: user@host (e.g., deploy@web-server.com)
Path: Remote directory path (e.g., /var/www/html)

Security Considerations

SSH Host Key Verification

By default, Cloudstic accepts any SSH host key (equivalent to StrictHostKeyChecking=no). For production use, consider:

Pre-adding the server’s host key to ~/.ssh/known_hosts
Using SSH certificates
Implementing custom host key verification

Credential Management

Best practices for managing SFTP credentials:

Use SSH keys instead of passwords

Generate a dedicated key pair:

ssh-keygen -t ed25519 -f ~/.ssh/cloudstic_backup -C "cloudstic backup"
ssh-copy-id -i ~/.ssh/cloudstic_backup.pub user@server

Use environment variables

Avoid hardcoding credentials:

export CLOUDSTIC_SFTP_PASSWORD="$VAULT_SFTP_PASS"
cloudstic backup -source sftp ...

Restrict key permissions

Ensure private keys are secure:

chmod 600 ~/.ssh/cloudstic_backup

Use a dedicated backup user

Create a read-only user for backups:

# On the remote server
useradd -m -s /bin/bash backup-user
# Configure read-only access to backup directories

Performance Considerations

Network Overhead

SFTP backups are slower than local backups due to:

Network latency
SSH encryption overhead
Remote filesystem access

Typical Performance

Small files: 100-500 files/second
Large files: Limited by network bandwidth
Network: 10-100 MB/s depending on connection

Optimizing SFTP Backups

Use exclude patterns

Reduce data transfer:

-exclude "*.log" -exclude "cache/" -exclude "tmp/"

Backup during off-peak hours

Schedule backups when network usage is low.

Use compression

Cloudstic automatically compresses data before storage.

Enable packfiles

Bundle small objects to reduce round trips (enabled by default).

Common Use Cases

Web Server Backups

cloudstic backup -source sftp \
  -source-path /var/www \
  -source-sftp-host web01.example.com \
  -source-sftp-user www-data \
  -source-sftp-key ~/.ssh/deploy_key \
  -exclude "cache/" \
  -exclude "*.log" \
  -tag "webserver"

Database Server Backups

# Backup database dumps
cloudstic backup -source sftp \
  -source-path /backup/mysql \
  -source-sftp-host db01.example.com \
  -source-sftp-user backup \
  -source-sftp-key ~/.ssh/backup_key \
  -tag "database" \
  -tag "mysql"

Application Server Backups

cloudstic backup -source sftp \
  -source-path /opt/app \
  -source-sftp-host app-server.com \
  -source-sftp-user appuser \
  -source-sftp-key ~/.ssh/app_key \
  -exclude "logs/" \
  -exclude "temp/" \
  -tag "application"

Troubleshooting

Connection Refused

Error: ssh dial example.com:22: connection refused

Solutions:

Check that SSH service is running on the remote server
Verify the hostname and port are correct
Check firewall rules

Authentication Failed

Error: ssh: unable to authenticate

Solutions:

Verify username and credentials
Check SSH key permissions (must be 600)
Ensure the public key is in ~/.ssh/authorized_keys on the server
Try password authentication if key auth fails

Permission Denied

Error: permission denied

Solutions:

Check that the user has read access to the source directory
Verify the source path is correct
Check filesystem permissions on the remote server

Slow Backups

If SFTP backups are taking too long:

Use exclude patterns to reduce data transfer
Check network bandwidth between source and destination
Consider local backups if possible (install Cloudstic on the remote server)
Run backups during off-peak hours

Environment Variables

Set default SFTP credentials:

export CLOUDSTIC_SOURCE=sftp
export CLOUDSTIC_SFTP_HOST=example.com
export CLOUDSTIC_SFTP_USER=backup
export CLOUDSTIC_SFTP_KEY=~/.ssh/backup_key
export CLOUDSTIC_SFTP_PORT=22

# Now run backup with minimal flags
cloudstic backup -source-path /var/www

Next Steps

Local Source

Learn about backing up local directories

SSH Key Setup

Set up SSH keys for authentication

Scheduling Backups

Automate backups with cron or systemd

Exclude Patterns

Master exclude pattern syntax

Get Started

Core Concepts

Sources

Storage Backends

Guides

​Basic Usage

​Configuration Options

​Required Flags

​Authentication Flags

​Optional Flags

​Authentication Methods

​1. Private Key (Recommended)

​2. Password

​3. SSH Agent (Automatic)

​Examples

​Backup Web Server

​Backup with Custom Port

​Backup with Exclusions

​Multiple Server Backups

​How It Works

​Connection Process

​File Identification

​Source Information

​Security Considerations

​SSH Host Key Verification

​Credential Management

​Performance Considerations

​Network Overhead

​Typical Performance

​Optimizing SFTP Backups

​Common Use Cases

​Web Server Backups

​Database Server Backups

​Application Server Backups

​Troubleshooting

​Connection Refused

​Authentication Failed

​Permission Denied

​Slow Backups

​Environment Variables

​Next Steps

Local Source

SSH Key Setup

Scheduling Backups

Exclude Patterns

Basic Usage

Configuration Options

Required Flags

Authentication Flags

Optional Flags

Authentication Methods

1. Private Key (Recommended)

2. Password

3. SSH Agent (Automatic)

Examples

Backup Web Server

Backup with Custom Port

Backup with Exclusions

Multiple Server Backups

How It Works

Connection Process

File Identification

Source Information

Security Considerations

SSH Host Key Verification

Credential Management

Performance Considerations

Network Overhead

Typical Performance

Optimizing SFTP Backups

Common Use Cases

Web Server Backups

Database Server Backups

Application Server Backups

Troubleshooting

Connection Refused

Authentication Failed

Permission Denied

Slow Backups

Environment Variables

Next Steps