cloudstic break-lock - Cloudstic CLI

Overview

The cloudstic break-lock command forcibly removes lock files from a Cloudstic repository. Locks are automatically acquired by operations like backup, restore, and prune to prevent concurrent writes that could corrupt the repository.

Only use break-lock if you are certain no other Cloudstic process is currently running. Breaking a lock held by an active process can cause repository corruption or data loss.

When to Use

Use break-lock when:

A Cloudstic process was killed (SIGKILL, crash, system reboot) and left a stale lock
You receive an error like: repository is locked by another operation
You have verified that no other Cloudstic process is running (check with ps, htop, or your process manager)

Locks automatically expire after 1 minute if the holding process stops refreshing them (e.g. after a crash). If possible, wait for the lock to expire naturally instead of breaking it.

Basic Usage

cloudstic break-lock [options]

Terminal Output Examples

Stale Lock Removed

$ cloudstic break-lock
Locks removed:
  Operation:  backup
  Holder:     hostname (pid 12345)
  Acquired:   2026-03-01T10:30:45Z
  Expired at: 2026-03-01T10:31:45Z
  Shared:     false

This indicates a backup operation’s lock was successfully removed. The lock was acquired on March 1 at 10:30 and would have expired at 11:30.

No Lock Found

$ cloudstic break-lock
No lock found. Repository is not locked.

The repository is not currently locked. No action was taken.

Multiple Locks Removed

$ cloudstic break-lock
Locks removed:
  Operation:  backup
  Holder:     server1 (pid 67890)
  Acquired:   2026-03-01T09:30:00Z
  Expired at: 2026-03-01T09:31:00Z
  Shared:     false

  Operation:  prune
  Holder:     server2 (pid 11223)
  Acquired:   2026-03-01T09:50:00Z
  Expired at: 2026-03-01T09:51:00Z
  Shared:     false

Multiple stale locks from different processes were removed.

Command Flags

break-lock has no command-specific flags. It uses global flags for repository access.

Global Flags

-store

string

default:"local:./backup_store"

Storage backend URI. Formats: local:<path>, s3:<bucket>[/<prefix>], b2:<bucket>[/<prefix>], sftp://[user@]host[:port]/<path>.

-encryption-key

string

default:""

Platform key (64 hex chars = 32 bytes) for encrypted repositories.

-password

string

default:""

Repository password for encrypted repositories.

-recovery-key

string

default:""

24-word BIP39 recovery phrase.

-kms-key-arn

string

default:""

AWS KMS key ARN for KMS-encrypted repositories.

-prompt

boolean

default:"false"

Prompt for password interactively (use alongside -encryption-key or -kms-key-arn to add a password layer).

-verbose

boolean

default:"false"

Log detailed operations.

-quiet

boolean

default:"false"

Suppress output (not recommended for this command).

-json

boolean

default:"false"

Write the command result as JSON to stdout. This suppresses the human-readable lock summary.

-debug

boolean

default:"false"

Log every store request (network calls, timing, sizes).

S3-Specific Flags

-s3-endpoint

string

default:""

S3-compatible endpoint (MinIO, R2, etc.).

-s3-region

string

default:"us-east-1"

S3 region.

-s3-access-key

string

default:""

S3 access key ID.

-s3-secret-key

string

default:""

S3 secret access key.

SFTP Store Credentials

-store-sftp-password

string

default:""

SFTP store password.

-store-sftp-key

string

default:""

Path to SSH private key for SFTP store authentication.

-store-sftp-known-hosts

string

default:""

Path to custom known_hosts file for host key validation.

-store-sftp-insecure

boolean

default:"false"

Skip host key validation (INSECURE).

Examples

Local Repository

cloudstic break-lock

Remote S3 Repository

export AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

cloudstic break-lock \
  -store s3:my-backup-bucket \
  -s3-region us-west-2 \
  -password "my-passphrase"

SFTP Repository

cloudstic break-lock \
  -store sftp://backupuser@backup.example.com/home/backupuser/cloudstic \
  -store-sftp-key ~/.ssh/id_rsa

With Debug Logging

cloudstic break-lock -debug

Output:

[DEBUG] GET lock/backup-server1-12345.lock (234 bytes, 45ms)
[DEBUG] DELETE lock/backup-server1-12345.lock (0 bytes, 32ms)
Locks removed:
  Operation:  backup
  Holder:     server1 (pid 12345)
  Acquired:   2026-03-01T10:30:45Z
  Expired at: 2026-03-01T10:31:45Z
  Shared:     false

How Locks Work

Lock Types

Cloudstic uses a reader-writer lock protocol stored directly in the repository:

Type	Key	Operations
Shared	`index/lock.shared/<timestamp>`	`backup`, `restore`; multiple can coexist
Exclusive	`index/lock.exclusive`	`prune`; blocks all shared locks

forget acquires no lock. check acquires no lock.

Lock Acquisition

When an operation starts, it:

Checks for conflicts: shared ops check for an exclusive lock; exclusive ops check for any shared or exclusive lock
Writes its lock object: JSON metadata stored at the key above
Re-reads to verify ownership: mitigates TOCTOU races on stores without atomic writes
Starts a refresh goroutine: extends expires_at every 30 seconds

Lock Expiration

TTL: 1 minute from acquisition, refreshed every 30 seconds while the process is alive
On crash: The refresh goroutine stops; the lock expires after at most 1 minute
Automatic recovery: The next operation sees an expired expires_at and proceeds normally: no manual intervention required

Lock Storage

Locks are stored as JSON objects in the repository under index/:

Exclusive key: index/lock.exclusive
Shared key: index/lock.shared/<timestamp>
Content: JSON metadata (operation, holder, acquired_at, expires_at, is_shared)
Unencrypted: Lock objects are stored in plaintext for debuggability

Understanding Lock Information

Operation

The Cloudstic command that held the lock:

backup: Backup operation (shared lock)
restore: Restore operation (shared lock)
prune: Prune operation (exclusive lock)

Holder

Unique identifier for the process that acquired the lock:

hostname (pid 12345)

hostname: Machine hostname
pid: Process ID (PID)

Acquired / Expired At

Acquired: When the lock was first created
Expired at: When the lock would automatically become invalid

If the current time is past “Expired at”, the lock is already stale and safe to break.

Shared

true: Shared lock (backup or restore): multiple can coexist
false: Exclusive lock (prune): blocks all other operations

Safety Considerations

Breaking an active lock can corrupt your repository. Always verify no Cloudstic process is running before using break-lock.

How to Verify No Active Processes

Linux/macOS

# Check for cloudstic processes
ps aux | grep cloudstic

# Or use pgrep
pgrep -fl cloudstic

# Check system-wide
htop  # Search for "cloudstic"

Windows

# PowerShell
Get-Process | Where-Object { $_.ProcessName -like "*cloudstic*" }

# Task Manager
# Look for "cloudstic.exe" in Processes tab

Systemd/Cron

Check if Cloudstic is running as a scheduled task:

# Systemd timers
systemctl list-timers --all | grep cloudstic

# Cron jobs
crontab -l | grep cloudstic
sudo crontab -l | grep cloudstic  # System-wide cron

When It’s Safe to Break a Lock

✅ Safe scenarios:

Process was killed (SIGKILL, crash, reboot)
Lock expiration time has passed
You confirmed no active Cloudstic processes
Lock holder hostname is a decommissioned machine

❌ Unsafe scenarios:

Active backup is in progress on another machine
Scheduled backup might be running (check cron/systemd)
Uncertain about process status

Troubleshooting

Error: Failed to init store

Failed to init store: no such host

Cause: Cannot connect to storage backend. Resolution: Verify -store URI and credentials (e.g., -s3-access-key).

Error: Failed to break lock

Failed to break lock: permission denied

Cause: Insufficient permissions to delete lock objects. Resolution:

Check storage backend permissions (S3 IAM policy, SFTP file permissions)
Verify you have write access to the repository

Lock reappears immediately after breaking

Cause: Another process is actively creating locks (e.g., running backup). Resolution:

Find and stop the active process
Wait for the lock to expire naturally (check “Expired at” time)
Do not repeatedly break locks: this indicates an underlying issue

Repository is corrupted after breaking lock

Cause: Lock was broken while another process was writing. Resolution:

Run cloudstic check to assess damage
If errors found, restore from an earlier snapshot or backup
In the future, always verify no processes are running before breaking locks

Use Cases

Crashed Backup Job

# Backup process was killed by system OOM killer
$ cloudstic backup -source local:/data
Killed

# Later, trying to run backup again:
$ cloudstic backup -source local:/data
Error: repository is locked by another operation

# Verify no processes running:
$ ps aux | grep cloudstic
# (no results)

# Break the stale lock:
$ cloudstic break-lock
Locks removed:
  Operation:  backup
  Holder:     server1 (pid 12345)
  ...

# Retry backup:
$ cloudstic backup -source local:/data
# Success!

Remote Repository Lock After Server Reboot

# Server rebooted unexpectedly while backup was running
# Next backup attempt:
$ cloudstic backup -store s3:my-bucket -source local:/data
Error: repository is locked by another operation

# Check lock details:
$ cloudstic break-lock -store s3:my-bucket
Locks removed:
  Operation:  backup
  Holder:     old-server (pid 78901)
  Acquired:   2026-02-28T15:30:00Z
  Expired at: 2026-02-28T15:31:00Z  # Already expired!
  Shared:     false

# Lock was already expired; safe to remove

CI/CD Pipeline Failure

# CI job was canceled mid-backup
# Next CI run:
- cloudstic backup -source local:/workspace
  Error: repository is locked by another operation

# CI script includes lock recovery:
- cloudstic break-lock || true  # Ignore errors if no lock
- cloudstic backup -source local:/workspace

For automated environments (CI/CD, cron), consider adding break-lock to your recovery logic. However, ensure jobs don’t run concurrently (e.g., use job queues or single-instance constraints).

Exit Codes

0: Success (lock removed or no lock found)
1: Error (store initialization failed, permission denied, or command error)

cloudstic backup: Acquires a shared lock
cloudstic restore: Acquires a shared lock
cloudstic prune: Acquires an exclusive lock; blocks backup and restore

Repository Management

Backup & Restore

Maintenance

Key Management

Configuration

Utilities

​Overview

​When to Use

​Basic Usage

​Terminal Output Examples

​Stale Lock Removed

​No Lock Found

​Multiple Locks Removed

​Command Flags

​Global Flags

​S3-Specific Flags

​SFTP Store Credentials

​Examples

​Local Repository

​Remote S3 Repository

​SFTP Repository

​With Debug Logging

​How Locks Work

​Lock Types

​Lock Acquisition

​Lock Expiration

​Lock Storage

​Understanding Lock Information

​Operation

​Holder

​Acquired / Expired At

​Shared

​Safety Considerations

​How to Verify No Active Processes

​Linux/macOS

​Windows

​Systemd/Cron

​When It’s Safe to Break a Lock

​Troubleshooting

​Error: Failed to init store

​Error: Failed to break lock

​Lock reappears immediately after breaking

​Repository is corrupted after breaking lock

​Use Cases

​Crashed Backup Job

​Remote Repository Lock After Server Reboot

​CI/CD Pipeline Failure

​Exit Codes

​Related Commands

Overview

When to Use

Basic Usage

Terminal Output Examples

Stale Lock Removed

No Lock Found

Multiple Locks Removed

Command Flags

Global Flags

S3-Specific Flags

SFTP Store Credentials

Examples

Local Repository

Remote S3 Repository

SFTP Repository

With Debug Logging

How Locks Work

Lock Types

Lock Acquisition

Lock Expiration

Lock Storage

Understanding Lock Information

Operation

Holder

Acquired / Expired At

Shared

Safety Considerations

How to Verify No Active Processes

Linux/macOS

Windows

Systemd/Cron

When It’s Safe to Break a Lock

Troubleshooting

Error: Failed to init store

Error: Failed to break lock

Lock reappears immediately after breaking

Repository is corrupted after breaking lock

Use Cases

Crashed Backup Job

Remote Repository Lock After Server Reboot

CI/CD Pipeline Failure

Exit Codes

Related Commands