> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cloudstic.com/llms.txt
> Use this file to discover all available pages before exploring further.

# cloudstic break-lock

> Remove stale repository locks left by crashed or killed processes

## Overview

The `cloudstic break-lock` command forcibly removes lock files from a Cloudstic repository. Locks are automatically acquired by operations like `backup`, `restore`, and `prune` to prevent concurrent writes that could corrupt the repository.

<Warning>
  **Only use `break-lock` if you are certain no other Cloudstic process is currently running.** Breaking a lock held by an active process can cause repository corruption or data loss.
</Warning>

## When to Use

Use `break-lock` when:

* A Cloudstic process was killed (SIGKILL, crash, system reboot) and left a stale lock
* You receive an error like: `repository is locked by another operation`
* You have verified that no other Cloudstic process is running (check with `ps`, `htop`, or your process manager)

<Tip>
  Locks automatically expire after **1 minute** if the holding process stops refreshing them (e.g. after a crash). If possible, wait for the lock to expire naturally instead of breaking it.
</Tip>

## Basic Usage

```bash theme={null}
cloudstic break-lock [options]
```

## Terminal Output Examples

### Stale Lock Removed

```bash theme={null}
$ cloudstic break-lock
Locks removed:
  Operation:  backup
  Holder:     hostname (pid 12345)
  Acquired:   2026-03-01T10:30:45Z
  Expired at: 2026-03-01T10:31:45Z
  Shared:     false
```

This indicates a backup operation's lock was successfully removed. The lock was acquired on March 1 at 10:30 and would have expired at 11:30.

### No Lock Found

```bash theme={null}
$ cloudstic break-lock
No lock found. Repository is not locked.
```

The repository is not currently locked. No action was taken.

### Multiple Locks Removed

```bash theme={null}
$ cloudstic break-lock
Locks removed:
  Operation:  backup
  Holder:     server1 (pid 67890)
  Acquired:   2026-03-01T09:30:00Z
  Expired at: 2026-03-01T09:31:00Z
  Shared:     false

  Operation:  prune
  Holder:     server2 (pid 11223)
  Acquired:   2026-03-01T09:50:00Z
  Expired at: 2026-03-01T09:51:00Z
  Shared:     false
```

Multiple stale locks from different processes were removed.

## Command Flags

`break-lock` has no command-specific flags. It uses global flags for repository access.

### Global Flags

<ParamField path="-store" type="string" default="local:./backup_store" env="CLOUDSTIC_STORE">
  Storage backend URI. Formats: `local:<path>`, `s3:<bucket>[/<prefix>]`, `b2:<bucket>[/<prefix>]`, `sftp://[user@]host[:port]/<path>`.
</ParamField>

<ParamField path="-encryption-key" type="string" default="" env="CLOUDSTIC_ENCRYPTION_KEY">
  Platform key (64 hex chars = 32 bytes) for encrypted repositories.
</ParamField>

<ParamField path="-password" type="string" default="" env="CLOUDSTIC_PASSWORD">
  Repository password for encrypted repositories.
</ParamField>

<ParamField path="-recovery-key" type="string" default="" env="CLOUDSTIC_RECOVERY_KEY">
  24-word BIP39 recovery phrase.
</ParamField>

<ParamField path="-kms-key-arn" type="string" default="" env="CLOUDSTIC_KMS_KEY_ARN">
  AWS KMS key ARN for KMS-encrypted repositories.
</ParamField>

<ParamField path="-prompt" type="boolean" default="false">
  Prompt for password interactively (use alongside `-encryption-key` or `-kms-key-arn` to add a password layer).
</ParamField>

<ParamField path="-verbose" type="boolean" default="false">
  Log detailed operations.
</ParamField>

<ParamField path="-quiet" type="boolean" default="false">
  Suppress output (not recommended for this command).
</ParamField>

<ParamField path="-json" type="boolean" default="false">
  Write the command result as JSON to stdout. This suppresses the human-readable lock summary.
</ParamField>

<ParamField path="-debug" type="boolean" default="false">
  Log every store request (network calls, timing, sizes).
</ParamField>

### S3-Specific Flags

<ParamField path="-s3-endpoint" type="string" default="" env="CLOUDSTIC_S3_ENDPOINT">
  S3-compatible endpoint (MinIO, R2, etc.).
</ParamField>

<ParamField path="-s3-region" type="string" default="us-east-1" env="CLOUDSTIC_S3_REGION">
  S3 region.
</ParamField>

<ParamField path="-s3-access-key" type="string" default="" env="AWS_ACCESS_KEY_ID">
  S3 access key ID.
</ParamField>

<ParamField path="-s3-secret-key" type="string" default="" env="AWS_SECRET_ACCESS_KEY">
  S3 secret access key.
</ParamField>

### SFTP Store Credentials

<ParamField path="-store-sftp-password" type="string" default="" env="CLOUDSTIC_STORE_SFTP_PASSWORD">
  SFTP store password.
</ParamField>

<ParamField path="-store-sftp-key" type="string" default="" env="CLOUDSTIC_STORE_SFTP_KEY">
  Path to SSH private key for SFTP store authentication.
</ParamField>

<ParamField path="-store-sftp-known-hosts" type="string" default="" env="CLOUDSTIC_STORE_SFTP_KNOWN_HOSTS">
  Path to custom `known_hosts` file for host key validation.
</ParamField>

<ParamField path="-store-sftp-insecure" type="boolean" default="false" env="CLOUDSTIC_STORE_SFTP_INSECURE">
  Skip host key validation (INSECURE).
</ParamField>

## Examples

### Local Repository

```bash theme={null}
cloudstic break-lock
```

### Remote S3 Repository

```bash theme={null}
export AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

cloudstic break-lock \
  -store s3:my-backup-bucket \
  -s3-region us-west-2 \
  -password "my-passphrase"
```

### SFTP Repository

```bash theme={null}
cloudstic break-lock \
  -store sftp://backupuser@backup.example.com/home/backupuser/cloudstic \
  -store-sftp-key ~/.ssh/id_rsa
```

### With Debug Logging

```bash theme={null}
cloudstic break-lock -debug
```

**Output:**

```
[DEBUG] GET lock/backup-server1-12345.lock (234 bytes, 45ms)
[DEBUG] DELETE lock/backup-server1-12345.lock (0 bytes, 32ms)
Locks removed:
  Operation:  backup
  Holder:     server1 (pid 12345)
  Acquired:   2026-03-01T10:30:45Z
  Expired at: 2026-03-01T10:31:45Z
  Shared:     false
```

## How Locks Work

### Lock Types

Cloudstic uses a reader-writer lock protocol stored directly in the repository:

| Type          | Key                             | Operations                                |
| ------------- | ------------------------------- | ----------------------------------------- |
| **Shared**    | `index/lock.shared/<timestamp>` | `backup`, `restore`; multiple can coexist |
| **Exclusive** | `index/lock.exclusive`          | `prune`; blocks all shared locks          |

`forget` acquires no lock. `check` acquires no lock.

### Lock Acquisition

When an operation starts, it:

1. **Checks for conflicts**: shared ops check for an exclusive lock; exclusive ops check for any shared or exclusive lock
2. **Writes its lock object**: JSON metadata stored at the key above
3. **Re-reads to verify ownership**: mitigates TOCTOU races on stores without atomic writes
4. **Starts a refresh goroutine**: extends `expires_at` every 30 seconds

### Lock Expiration

* **TTL:** 1 minute from acquisition, refreshed every 30 seconds while the process is alive
* **On crash:** The refresh goroutine stops; the lock expires after at most 1 minute
* **Automatic recovery:** The next operation sees an expired `expires_at` and proceeds normally: no manual intervention required

### Lock Storage

Locks are stored as JSON objects in the repository under `index/`:

* **Exclusive key:** `index/lock.exclusive`
* **Shared key:** `index/lock.shared/<timestamp>`
* **Content:** JSON metadata (operation, holder, acquired\_at, expires\_at, is\_shared)
* **Unencrypted:** Lock objects are stored in plaintext for debuggability

## Understanding Lock Information

### Operation

The Cloudstic command that held the lock:

* `backup`: Backup operation (shared lock)
* `restore`: Restore operation (shared lock)
* `prune`: Prune operation (exclusive lock)

### Holder

Unique identifier for the process that acquired the lock:

```
hostname (pid 12345)
```

* **hostname:** Machine hostname
* **pid:** Process ID (PID)

### Acquired / Expired At

* **Acquired:** When the lock was first created
* **Expired at:** When the lock would automatically become invalid

<Tip>
  If the current time is past "Expired at", the lock is already stale and safe to break.
</Tip>

### Shared

* **true:** Shared lock (`backup` or `restore`): multiple can coexist
* **false:** Exclusive lock (`prune`): blocks all other operations

## Safety Considerations

<Warning>
  **Breaking an active lock can corrupt your repository.** Always verify no Cloudstic process is running before using `break-lock`.
</Warning>

### How to Verify No Active Processes

#### Linux/macOS

```bash theme={null}
# Check for cloudstic processes
ps aux | grep cloudstic

# Or use pgrep
pgrep -fl cloudstic

# Check system-wide
htop  # Search for "cloudstic"
```

#### Windows

```powershell theme={null}
# PowerShell
Get-Process | Where-Object { $_.ProcessName -like "*cloudstic*" }

# Task Manager
# Look for "cloudstic.exe" in Processes tab
```

#### Systemd/Cron

Check if Cloudstic is running as a scheduled task:

```bash theme={null}
# Systemd timers
systemctl list-timers --all | grep cloudstic

# Cron jobs
crontab -l | grep cloudstic
sudo crontab -l | grep cloudstic  # System-wide cron
```

### When It's Safe to Break a Lock

✅ **Safe scenarios:**

* Process was killed (SIGKILL, crash, reboot)
* Lock expiration time has passed
* You confirmed no active Cloudstic processes
* Lock holder hostname is a decommissioned machine

❌ **Unsafe scenarios:**

* Active backup is in progress on another machine
* Scheduled backup might be running (check cron/systemd)
* Uncertain about process status

## Troubleshooting

### Error: Failed to init store

```
Failed to init store: no such host
```

**Cause:** Cannot connect to storage backend.

**Resolution:** Verify `-store` URI and credentials (e.g., `-s3-access-key`).

### Error: Failed to break lock

```
Failed to break lock: permission denied
```

**Cause:** Insufficient permissions to delete lock objects.

**Resolution:**

* Check storage backend permissions (S3 IAM policy, SFTP file permissions)
* Verify you have write access to the repository

### Lock reappears immediately after breaking

**Cause:** Another process is actively creating locks (e.g., running backup).

**Resolution:**

1. Find and stop the active process
2. Wait for the lock to expire naturally (check "Expired at" time)
3. Do not repeatedly break locks: this indicates an underlying issue

### Repository is corrupted after breaking lock

**Cause:** Lock was broken while another process was writing.

**Resolution:**

1. Run `cloudstic check` to assess damage
2. If errors found, restore from an earlier snapshot or backup
3. In the future, always verify no processes are running before breaking locks

## Use Cases

### Crashed Backup Job

```bash theme={null}
# Backup process was killed by system OOM killer
$ cloudstic backup -source local:/data
Killed

# Later, trying to run backup again:
$ cloudstic backup -source local:/data
Error: repository is locked by another operation

# Verify no processes running:
$ ps aux | grep cloudstic
# (no results)

# Break the stale lock:
$ cloudstic break-lock
Locks removed:
  Operation:  backup
  Holder:     server1 (pid 12345)
  ...

# Retry backup:
$ cloudstic backup -source local:/data
# Success!
```

### Remote Repository Lock After Server Reboot

```bash theme={null}
# Server rebooted unexpectedly while backup was running
# Next backup attempt:
$ cloudstic backup -store s3:my-bucket -source local:/data
Error: repository is locked by another operation

# Check lock details:
$ cloudstic break-lock -store s3:my-bucket
Locks removed:
  Operation:  backup
  Holder:     old-server (pid 78901)
  Acquired:   2026-02-28T15:30:00Z
  Expired at: 2026-02-28T15:31:00Z  # Already expired!
  Shared:     false

# Lock was already expired; safe to remove
```

### CI/CD Pipeline Failure

```bash theme={null}
# CI job was canceled mid-backup
# Next CI run:
- cloudstic backup -source local:/workspace
  Error: repository is locked by another operation

# CI script includes lock recovery:
- cloudstic break-lock || true  # Ignore errors if no lock
- cloudstic backup -source local:/workspace
```

<Tip>
  For automated environments (CI/CD, cron), consider adding `break-lock` to your recovery logic. However, ensure jobs don't run concurrently (e.g., use job queues or single-instance constraints).
</Tip>

## Exit Codes

* **0**: Success (lock removed or no lock found)
* **1**: Error (store initialization failed, permission denied, or command error)

## Related Commands

* [cloudstic backup](/commands/backup): Acquires a shared lock
* [cloudstic restore](/commands/restore): Acquires a shared lock
* [cloudstic prune](/commands/prune): Acquires an exclusive lock; blocks backup and restore