Go Client API - Cloudstic CLI

The github.com/cloudstic/cli package exposes a high-level Go API that mirrors the CLI commands. Use it to embed backup, restore, and repository management directly into your Go programs.

Installation

go get github.com/cloudstic/cli

Import the root package:

import cloudstic "github.com/cloudstic/cli"

Storage backends and keychain helpers live in sub-packages:

import (
    "github.com/cloudstic/cli/pkg/store"
    "github.com/cloudstic/cli/pkg/keychain"
    "github.com/cloudstic/cli/pkg/source"
)

Two further packages turn user-facing configuration into live objects, and the split between them decides what importing costs you:

import (
    "github.com/cloudstic/cli/pkg/config" // what did the user configure
    "github.com/cloudstic/cli/pkg/open"   // connect to it
)

pkg/config holds resolved configuration values and performs no I/O against a store or a provider, so reading and validating configuration pulls in no cloud SDK. pkg/open constructs stores, sources, keychains and clients from those values, and necessarily links whichever provider SDK you use. If you only need to inspect configuration, import the first and not the second.

Quick Start

package main

import (
    "bytes"
    "context"
    "fmt"
    "log"

    cloudstic "github.com/cloudstic/cli"
    "github.com/cloudstic/cli/pkg/keychain"
    "github.com/cloudstic/cli/pkg/source"
    "github.com/cloudstic/cli/pkg/store"
)

func main() {
    ctx := context.Background()

    // 1. Open a storage backend
    rawStore, err := store.NewLocalStore("./my-repo")
    if err != nil {
        log.Fatal(err)
    }

    // 2. Build a keychain with your credentials
    kc := keychain.Chain{keychain.WithPassword("my-passphrase")}

    // 3. Initialize the repository (first time only)
    _, err = cloudstic.InitRepo(ctx, rawStore,
        cloudstic.WithInitCredentials(kc),
    )
    if err != nil {
        log.Fatal(err)
    }

    // 4. Create a client: auto-resolves encryption via the keychain
    client, err := cloudstic.NewClient(ctx, rawStore, cloudstic.WithKeychain(kc))
    if err != nil {
        log.Fatal(err)
    }

    // 5. Back up a local directory
    src := source.NewLocalSource("./documents")
    result, err := client.Backup(ctx, src)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Snapshot: %s (%d new files)\n", result.SnapshotHash, result.FilesNew)

    // 6. Restore to a ZIP archive
    var buf bytes.Buffer
    _, err = client.Restore(ctx, &buf, "latest")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Restored %d bytes\n", buf.Len())
}

Storage Backends

All backends implement store.ObjectStore. Pass the raw store to InitRepo and NewClient.

Local

s, err := store.NewLocalStore("/path/to/repo")

Amazon S3 (and S3-compatible)

s, err := store.NewS3Store(ctx, "my-bucket",
    store.WithS3Region("us-east-1"),
    // For MinIO, Cloudflare R2, Wasabi, etc.:
    store.WithS3Endpoint("https://minio.example.com"),
    // Explicit credentials (or use the AWS SDK default credential chain):
    store.WithS3Credentials(accessKeyID, secretAccessKey),
    store.WithS3Prefix("backups/"),
)

Backblaze B2

s, err := store.NewB2Store("my-bucket",
    store.WithCredentials(keyID, appKey),
    store.WithPrefix("backups/"),
)

SFTP

s, err := store.NewSFTPStore("backup.example.com",
    store.WithSFTPBasePath("/home/backup/repo"),
    store.WithSFTPUser("backupuser"),
    store.WithSFTPKey("~/.ssh/id_ed25519"),
    store.WithSFTPPort("22"),
)

Keychain

The keychain resolves credentials to a master key. Build a keychain.Chain before calling InitRepo or NewClient.

// Password-based
kc := keychain.Chain{keychain.WithPassword("my-passphrase")}

// Platform key (raw 32-byte key)
platformKey, _ := hex.DecodeString("64-hex-chars...")
kc := keychain.Chain{keychain.WithPlatformKey(platformKey)}

// Recovery key (24-word BIP39 mnemonic)
kc := keychain.Chain{keychain.WithRecoveryKey("word1 word2 ... word24")}

// Multiple credentials (tried in order until one succeeds)
kc := keychain.Chain{
    keychain.WithPassword("my-passphrase"),
    keychain.WithRecoveryKey("word1 word2 ... word24"),
}

// AWS KMS
kmsClient, _ := crypto.NewAWSKMSClient(ctx, "arn:aws:kms:us-east-1:123456789:key/abc-123")
kc := keychain.Chain{keychain.WithKMSClient(kmsClient)}

Initializing a Repository

InitRepo is a package-level function that runs on the raw (unencrypted) store. It only needs to be called once per repository.

func InitRepo(ctx context.Context, rawStore store.ObjectStore, opts ...InitOption) (*InitResult, error)

Init Options

Option	Description
`WithInitCredentials(kc keychain.Chain)`	Key slots to create (password, platform key, KMS)
`WithInitRecovery()`	Also generate a 24-word recovery key slot
`WithInitNoEncryption()`	Create an unencrypted repository
`WithInitAdoptSlots()`	Adopt existing slots if already initialized (prevents error on re-init)

InitResult

type InitResult struct {
    Encrypted    bool   // whether the repository uses encryption
    RecoveryKey  string // 24-word mnemonic (empty if WithInitRecovery() was not requested)
    AdoptedSlots bool   // true if existing slots were adopted
}

Example

result, err := cloudstic.InitRepo(ctx, rawStore,
    cloudstic.WithInitCredentials(kc),
    cloudstic.WithInitRecovery(),
)
if err != nil {
    log.Fatal(err)
}
if result.RecoveryKey != "" {
    fmt.Println("Recovery key:", result.RecoveryKey)
    // Store this securely: it is displayed only once!
}

Inspecting a Repository

These package-level functions read repository state from the raw store without requiring the encryption key — useful for deciding whether to prompt for credentials before calling NewClient.

func InspectRepo(ctx context.Context, rawStore store.ObjectStore) (RepoStatus, error)

type RepoStatus struct {
    Initialized bool // whether a config marker exists at all
    Encrypted   bool // whether the repository uses encryption
    Sealed      bool // whether the marker itself is sealed (only true for newer encrypted repos)
}

status, err := cloudstic.InspectRepo(ctx, rawStore)
if err != nil {
    log.Fatal(err)
}
if !status.Initialized {
    fmt.Println("Run InitRepo first.")
} else if status.Encrypted {
    fmt.Println("Repository is encrypted; resolve a keychain before NewClient.")
}

LoadRepoConfig(ctx, rawStore, encryptionKey) (*RepoConfig, error) reads the full marker (not just its status) and requires the encryption key for a sealed repository. UpgradeRepoFormat stamps the on-disk format version; NewClient/Backup/Prune/Forget call it for you as part of normal operation, so most callers never need it directly.

Creating a Client

func NewClient(ctx context.Context, base store.ObjectStore, opts ...ClientOption) (*Client, error)

NewClient reads the repository config, resolves the master key via the keychain, and builds the encryption/compression/packfile decorator chain internally.

Client Options

Option	Description
`WithKeychain(kc keychain.Chain)`	Keychain for automatic master key resolution
`WithEncryptionKey(key []byte)`	Direct 32-byte AES key. Bypasses keychain and config detection
`WithReporter(r Reporter)`	Progress reporter for UI feedback
`WithPackfile(enable bool)`	Bundle small objects into 8MB packs (default: `true`)

client, err := cloudstic.NewClient(ctx, rawStore,
    cloudstic.WithKeychain(kc),
    cloudstic.WithReporter(myReporter),
    cloudstic.WithPackfile(true),
)

WithEncryptionKey is intended for SaaS scenarios where the master key is resolved externally. For typical use, prefer WithKeychain.

client.Store() returns the underlying store.ObjectStore (the fully decorated store NewClient built), for callers that need lower-level access alongside the Client API.

Backup

func (c *Client) Backup(ctx context.Context, src source.Source, opts ...BackupOption) (*BackupResult, error)

Creates a new backup snapshot from the given source.

Backup Options

Option	Description
`WithVerbose()`	Log per-file operations
`WithBackupDryRun()`	Scan without writing to the repository
`WithIgnoreEmptySnapshot()`	Skip writing a snapshot if nothing changed since the previous one
`WithTags(tags ...string)`	Apply tags to the snapshot
`WithGenerator(name string)`	Record the tool/script that produced the backup (e.g. for automation)
`WithMeta(key, value string)`	Attach an arbitrary key/value pair to the snapshot
`WithExcludeHash(hash string)`	Record the exclude pattern fingerprint in the snapshot

BackupResult

type BackupResult struct {
    SnapshotRef          string        // "snapshot/<hash>"
    SnapshotHash         string        // bare content hash
    Root                 string        // HAMT root ref
    FilesNew             int64
    FilesChanged         int64
    FilesUnmodified      int64
    FilesRemoved         int64
    DirsNew              int64
    DirsChanged          int64
    DirsUnmodified       int64
    DirsRemoved          int64
    BytesAddedRaw        int64         // uncompressed bytes written
    BytesAddedStored     int64         // bytes written to the store (post-compression and encryption)
    Duration             time.Duration
    DryRun               bool
    EmptySnapshotIgnored bool          // true if WithIgnoreEmptySnapshot() skipped an unchanged backup
}

Example

src := source.NewLocalSource("./documents",
    source.WithLocalExcludePatterns([]string{"*.log", "node_modules/"}),
)

result, err := client.Backup(ctx, src,
    cloudstic.WithTags("production", "weekly"),
)
if err != nil {
    log.Fatal(err)
}
fmt.Printf("Snapshot %s: %d new, %d changed files\n",
    result.SnapshotHash, result.FilesNew, result.FilesChanged)

Sources

Sources implement source.Source. Available implementations:

// Local filesystem
src := source.NewLocalSource("/path/to/dir",
    source.WithLocalExcludePatterns([]string{"*.tmp"}),
)

// SFTP remote directory
src, err := source.NewSFTPSource("backup.example.com",
    source.WithSFTPSourceBasePath("/remote/path"),
    source.WithSFTPSourceUser("user"),
    source.WithSFTPSourceKey("~/.ssh/id_rsa"),
)

// Google Drive (full scan)
src, err := source.NewGDriveSource(ctx,
    source.WithTokenPath("/path/to/google_token.json"),
    source.WithDriveID("sharedDriveID"), // omit for My Drive
    source.WithRootFolderID("folderID"), // omit for entire drive
)

// Google Drive (incremental via Changes API: recommended)
src, err := source.NewGDriveChangeSource(ctx,
    source.WithTokenPath("/path/to/google_token.json"),
)

// OneDrive (full scan)
src, err := source.NewOneDriveSource(ctx,
    source.WithOneDriveTokenPath("/path/to/onedrive_token.json"),
)

// OneDrive (incremental via Delta API: recommended)
src, err := source.NewOneDriveChangeSource(ctx,
    source.WithOneDriveTokenPath("/path/to/onedrive_token.json"),
)

Restore

func (c *Client) Restore(ctx context.Context, w io.Writer, snapshotRef string, opts ...RestoreOption) (*RestoreResult, error)

Writes the snapshot’s file tree as a ZIP archive to w. Pass io.Discard for a dry run. snapshotRef accepts "", "latest", a bare hash or unique hash prefix, or "snapshot/<hash-or-prefix>".

Restore Options

Option	Description
`WithRestoreDryRun()`	Count files/bytes without writing ZIP data
`WithRestoreVerbose()`	Log per-file operations
`WithRestorePath(path string)`	Restore only the given file or subtree (e.g. `"Documents/"`)
`WithRestoreNoVerify()`	Skip content-hash verification of restored chunks

RestoreResult

type RestoreResult struct {
    SnapshotRef  string
    Root         string
    FilesWritten int
    DirsWritten  int
    BytesWritten int64
    Errors       int    // number of non-fatal errors
    Warnings     int    // number of non-fatal warnings (e.g. skipped attributes)
    DryRun       bool
}

Example

f, err := os.Create("restore.zip")
if err != nil {
    log.Fatal(err)
}
defer f.Close()

result, err := client.Restore(ctx, f, "latest",
    cloudstic.WithRestorePath("Documents/"),
)
if err != nil {
    os.Remove("restore.zip")
    log.Fatal(err)
}
fmt.Printf("Restored %d files (%d bytes)\n", result.FilesWritten, result.BytesWritten)

Restoring directly to a directory

func (c *Client) RestoreToDir(ctx context.Context, outputDir, snapshotRef string, opts ...RestoreOption) (*RestoreResult, error)

Writes files directly to outputDir instead of a ZIP archive. Takes the same RestoreOptions and returns the same RestoreResult.

result, err := client.RestoreToDir(ctx, "/restore/target", "latest")
if err != nil {
    log.Fatal(err)
}
fmt.Printf("Restored %d files to disk\n", result.FilesWritten)

List

func (c *Client) List(ctx context.Context, opts ...ListOption) (*ListResult, error)

Lists all snapshots in the repository. Returns them sorted oldest-first.

List Options

Option	Description
`WithListVerbose()`	Log progress while the snapshot catalog is loaded

ListResult

type ListResult struct {
    Snapshots []SnapshotEntry
}

type SnapshotEntry struct {
    Ref     string        // "snapshot/<hash>"
    Snap    core.Snapshot // the decoded snapshot object (Seq, Source, Tags, Meta, ...)
    Created time.Time
}

SnapshotEntry.Snap is a core.Snapshot, which carries Seq int, Source *core.SourceInfo (Type, Account, Path, …), Tags []string, and Meta map[string]string. SourceInfo may be nil for snapshots written before source identity was tracked.

These types are declared in github.com/cloudstic/cli/internal/core, which external modules cannot import directly — Go’s internal/ rule blocks the import statement. You don’t need to. The root cloudstic package re-exports each of them as a type alias, so you can name them explicitly:

var snap cloudstic.Snapshot         // = core.Snapshot
var info *cloudstic.SourceInfo      // = *core.SourceInfo
var meta cloudstic.FileMeta         // = core.FileMeta
var entry cloudstic.SnapshotEntry   // = engine.SnapshotEntry

Because a Go alias denotes the identical type, these are interchangeable with what the Client API returns — you can declare variables, write helper function signatures, and implement interfaces against them. Reading exported fields off returned values (entry.Snap.Seq, meta.Name) works without naming the types at all, as this example does.

Example

result, err := client.List(ctx)
if err != nil {
    log.Fatal(err)
}
for _, entry := range result.Snapshots {
    source := "unknown"
    if entry.Snap.Source != nil {
        source = entry.Snap.Source.Type
    }
    fmt.Printf("[%d] %s  %-10s  %s\n",
        entry.Snap.Seq, entry.Created.Format(time.DateTime), source, entry.Ref)
}

LsSnapshot

func (c *Client) LsSnapshot(ctx context.Context, snapshotID string, opts ...LsSnapshotOption) (*LsSnapshotResult, error)

Loads all file metadata from a snapshot and returns the full directory tree. Accepts "latest", a bare hash or unique hash prefix, or "snapshot/<hash-or-prefix>".

LsSnapshot Options

Option	Description
`WithLsVerbose()`	Log progress while the tree is loaded

LsSnapshotResult

type LsSnapshotResult struct {
    Ref       string
    Snapshot  core.Snapshot
    RootRefs  []string                  // top-level entry refs
    RefToMeta map[string]core.FileMeta  // ref → file metadata (name, size, type, mode, owner, ...)
    ChildRefs map[string][]string       // parent ref → ordered child refs
}

See the note above about core.Snapshot/core.FileMeta and the internal/ import restriction.

Find

func (c *Client) Find(ctx context.Context, opts ...FindOption) (*FindResult, error)

Locates files across every snapshot in the repository without you having to know which snapshot holds them. Unlike the other read operations, Find takes a snapshot as output rather than input: by default it searches every snapshot and reports, for each matching file, the versions it has had and which snapshots each version lives in. It is a pure read path — no lock is taken and nothing is written.

Find Options

Entry predicates (all given predicates must match):

Option	Description
`WithFindPattern(pattern string)`	Positional pattern; routed to `Name` or `Path` depending on whether it contains a path separator
`WithFindName(pattern string)`	Match by basename glob
`WithFindPath(pattern string)`	Match by full path glob
`WithFindRegex(expr string)`	Match by regular expression
`WithFindIgnoreCase()`	Case-insensitive matching for name/path/regex
`WithFindFileID(id string)`	Match a specific source file ID
`WithFindContentHash(hash string)`	Match files with this exact content hash
`WithFindRef(ref string)`	Match a specific `filemeta/<hash>` ref
`WithFindType(t core.FileType)`	Restrict to a file type (file, directory, …)
`WithFindSize(cmp SizeCompare)`	Size predicate — see `ParseSizeCompare` below
`WithFindNewer(spec string)`	File `Mtime` at or after `spec` (RFC3339 or a duration like `"7d"`)
`WithFindOlder(spec string)`	File `Mtime` at or before `spec`

Snapshot selectors (which snapshots are searched):

Option	Description
`WithFindSnapshots(refs ...string)`	Restrict to specific snapshots (`"latest"`, full hash, or unambiguous prefix)
`WithFindSource(uri string)`	Restrict to snapshots from a source URI
`WithFindTags(tags ...string)`	Restrict to snapshots carrying any of these tags
`WithFindLatest(n int)`	Restrict to the `n` newest selected snapshots
`WithFindSince(spec string)`	Only snapshots created at or after `spec`
`WithFindUntil(spec string)`	Only snapshots created at or before `spec`

Presentation and execution:

Option	Description
`WithFindGroupByContent()`	Group matches by content hash instead of file identity (finds duplicate content)
`WithFindMaxResults(n int)`	Cap the number of distinct files reported (default 1000; scanning continues so counters stay accurate)
`WithFindNoDelta()`	Force a full per-snapshot walk instead of the delta scan between related snapshots
`WithFindVerbose()`	Log scan progress

Two helpers parse the string forms the CLI accepts, for callers building SizeCompare/time values programmatically:

func ParseSizeCompare(spec string) (SizeCompare, error) // "+10M", "-10M", "10M"
func ParseFindTime(spec string) (time.Time, error)      // RFC3339 or a duration like "7d"

FindResult

type FindResult struct {
    Query             FindQuery
    SnapshotsSearched int
    EntriesScanned    int
    MetaFetched       int         // filemeta objects actually read
    Matches           []FileMatch
    Truncated         bool        // true if MaxResults was reached
    Warnings          []string
    GroupedBy         string      // "file" or "content"
    Elapsed           string
}

// FileMatch is one file (or, under WithFindGroupByContent, one distinct
// content hash), with every version the query matched.
type FileMatch struct {
    FileID      string           // empty when grouped by content
    ContentHash string           // empty unless grouped by content
    Source      *core.SourceInfo
    Type        core.FileType
    Versions    []FileVersion    // newest first
}

// FileVersion is one immutable state of a file.
type FileVersion struct {
    Ref         string
    FileID      string
    Name        string
    Paths       []string      // more than one path if the same file lives under multiple parents
    ContentHash string
    Type        core.FileType
    Size        int64
    Mtime       int64
    Mode        uint32
    Snapshots   []SnapshotRef // every snapshot holding this exact version
    FirstSeen   string        // ISO8601, earliest containing snapshot
    LastSeen    string        // ISO8601, latest containing snapshot
}

type SnapshotRef struct {
    Ref     string
    Seq     int
    Created string // ISO8601
}

FileMatch has two convenience methods: Path() returns the newest version’s first path, and LatestSnapshot() returns the newest snapshot holding the newest version — the one a follow-up Restore would target.

Example

// Find every file over 100MB modified in the last 30 days, across all snapshots.
size, err := cloudstic.ParseSizeCompare("+100M")
if err != nil {
    log.Fatal(err)
}

result, err := client.Find(ctx,
    cloudstic.WithFindSize(size),
    cloudstic.WithFindNewer("30d"),
)
if err != nil {
    log.Fatal(err)
}

for _, m := range result.Matches {
    v := m.Versions[0]
    fmt.Printf("%s (%d bytes) — %d version(s), latest in %d snapshot(s)\n",
        m.Path(), v.Size, len(m.Versions), len(v.Snapshots))
}
if result.Truncated {
    fmt.Printf("... and more (raise WithFindMaxResults to see them)\n")
}

Finding duplicate content:

result, err := client.Find(ctx, cloudstic.WithFindGroupByContent())
if err != nil {
    log.Fatal(err)
}
for _, m := range result.Matches {
    if len(m.Versions) > 1 {
        fmt.Printf("%s duplicated across %d files\n", m.ContentHash, len(m.Versions))
    }
}

Diff

func (c *Client) Diff(ctx context.Context, snap1, snap2 string, opts ...DiffOption) (*DiffResult, error)

Compares two snapshots and returns the set of added, modified, and removed files. Each argument accepts "latest", a full hash, or a unique hash prefix.

Diff Options

Option	Description
`WithDiffVerbose()`	Log progress while both snapshots are loaded

Snapshot reference errors

Snapshot readers return sentinel errors that you can inspect with errors.Is:

result, err := client.Diff(ctx, "abc123", "latest")
switch {
case errors.Is(err, cloudstic.ErrSnapshotNotFound):
    // No snapshot matched the reference.
case errors.Is(err, cloudstic.ErrSnapshotRefAmbiguous):
    // More than one snapshot matched the prefix.
case err != nil:
    log.Fatal(err)
}

The same sentinels apply to Restore, RestoreToDir, LsSnapshot, and snapshot selectors passed to Find.

DiffResult

type DiffResult struct {
    Ref1    string
    Ref2    string
    Changes []Change
}

// Change.Type is one of: "added", "modified", "removed", "unchanged"
type Change struct {
    Type string
    Path string
}

Example

result, err := client.Diff(ctx, "abc123", "latest")
if err != nil {
    log.Fatal(err)
}
for _, c := range result.Changes {
    if c.Type != "unchanged" {
        fmt.Printf("%s %s\n", c.Type, c.Path)
    }
}

Prune

func (c *Client) Prune(ctx context.Context, opts ...PruneOption) (*PruneResult, error)

Removes unreachable objects (mark-and-sweep garbage collection). Run after Forget to reclaim storage.

Prune Options

Option	Description
`WithPruneDryRun()`	Count deletions without removing anything
`WithPruneVerbose()`	Log each deleted object

PruneResult

type PruneResult struct {
    ObjectsScanned int
    ObjectsDeleted int
    BytesReclaimed int64
    DryRun         bool
}

Forget

Remove a specific snapshot

func (c *Client) Forget(ctx context.Context, snapshotID string, opts ...ForgetOption) (*ForgetResult, error)

Apply a retention policy

func (c *Client) ForgetPolicy(ctx context.Context, opts ...ForgetOption) (*PolicyResult, error)

Forget Options

Option	Description
`WithPrune()`	Run prune after forgetting
`WithDryRun()`	Show what would be removed without deleting
`WithForgetVerbose()`	Verbose logging
`WithKeepLast(n int)`	Keep the N most recent snapshots
`WithKeepHourly(n int)`	Keep one snapshot per hour for the last N hours
`WithKeepDaily(n int)`	Keep one snapshot per day for the last N days
`WithKeepWeekly(n int)`	Keep one snapshot per week for the last N weeks
`WithKeepMonthly(n int)`	Keep one snapshot per month for the last N months
`WithKeepYearly(n int)`	Keep one snapshot per year for the last N years
`WithFilterTag(tag string)`	Only consider snapshots with this tag (repeatable)
`WithFilterSource(src string)`	Only consider snapshots from this source type
`WithFilterAccount(acct string)`	Only consider snapshots from this account
`WithFilterPath(path string)`	Only consider snapshots from this path
`WithGroupBy(fields string)`	Comma-separated grouping fields (default: `"source,account,path"`)

PolicyResult

type PolicyResult struct {
    Groups []PolicyGroupResult
    Prune  *PruneResult // set only when WithPrune() was passed
    DryRun bool
}

type PolicyGroupResult struct {
    Key    GroupKey
    Keep   []KeepReason
    Remove []SnapshotEntry
}

GroupKey and KeepReason are internal types you can read fields from (as in the example below) without importing them — see the note under List.

Example

// Retention policy: keep last 10, 7 daily, 4 weekly, 12 monthly: then prune
result, err := client.ForgetPolicy(ctx,
    cloudstic.WithKeepLast(10),
    cloudstic.WithKeepDaily(7),
    cloudstic.WithKeepWeekly(4),
    cloudstic.WithKeepMonthly(12),
    cloudstic.WithPrune(),
)
if err != nil {
    log.Fatal(err)
}
for _, group := range result.Groups {
    fmt.Printf("Group %s: keep %d, remove %d\n",
        group.Key, len(group.Keep), len(group.Remove))
}
if result.Prune != nil {
    fmt.Printf("Reclaimed %d bytes\n", result.Prune.BytesReclaimed)
}

Check

func (c *Client) Check(ctx context.Context, opts ...CheckOption) (*CheckResult, error)

Verifies repository integrity by walking the full reference chain (snapshots → HAMT nodes → filemeta → content → chunks).

Check Options

Option	Description
`WithReadData()`	Re-hash all chunk data for byte-level verification
`WithCheckVerbose()`	Log each object as it is verified
`WithSnapshotRef(ref string)`	Check only the specified snapshot (default: all)

CheckResult

type CheckResult struct {
    SnapshotsChecked int
    ObjectsVerified  int
    Errors           []CheckError
}

type CheckError struct {
    Type    string // "missing", "corrupt", "unreadable"
    Key     string // object key
    Message string
}

Example

result, err := client.Check(ctx, cloudstic.WithReadData())
if err != nil {
    log.Fatal(err)
}
if len(result.Errors) > 0 {
    for _, e := range result.Errors {
        fmt.Printf("[%s] %s: %s\n", e.Type, e.Key, e.Message)
    }
}

BreakLock

func (c *Client) BreakLock(ctx context.Context) ([]*RepoLock, error)

Removes stale repository lock files. Returns the list of removed locks (empty slice if none found).

removed, err := client.BreakLock(ctx)
if err != nil {
    log.Fatal(err)
}
for _, lock := range removed {
    fmt.Printf("Removed %s lock (held by %s)\n", lock.Operation, lock.Holder)
}

ErrRepoLocked

Backup, Restore, and Prune return ErrRepoLocked when the repository is already held by another operation. Check for it with errors.Is and prompt the caller toward BreakLock rather than treating it as a generic failure:

result, err := client.Backup(ctx, src)
if errors.Is(err, cloudstic.ErrRepoLocked) {
    // Another operation holds the repository. Only break the lock if you are
    // certain nothing is actually running against it.
    removed, breakErr := client.BreakLock(ctx)
    // ...
}

Cat

func (c *Client) Cat(ctx context.Context, keys ...string) ([]*CatResult, error)

Fetches the raw (decrypted, decompressed) data for one or more object keys. Useful for debugging and inspection.

CatResult

type CatResult struct {
    Key  string // the object key requested
    Data []byte // raw object data (typically JSON)
}

Object Key Namespaces

Key	Description
`config`	Repository configuration marker
`index/latest`	Latest snapshot pointer
`index/snapshots`	Snapshot catalog
`snapshot/<hash>`	Snapshot manifest
`node/<hash>`	HAMT tree node
`filemeta/<hash>`	File metadata object
`content/<hash>`	Content manifest (list of chunk refs)
`chunk/<hash>`	Raw data chunk
`keys/<slot>`	Encryption key slot
`lock/<id>`	Repository lock file

Example

results, err := client.Cat(ctx, "config", "index/latest")
if err != nil {
    log.Fatal(err)
}
for _, r := range results {
    fmt.Printf("=== %s ===\n%s\n\n", r.Key, r.Data)
}

Key Management

These package-level functions operate on the raw (unencrypted) store.

List key slots

func ListKeySlots(ctx context.Context, rawStore store.ObjectStore) ([]KeySlot, error)

Returns metadata for all key slots. Does not require authentication. Slot metadata is stored unencrypted.

slots, err := cloudstic.ListKeySlots(ctx, rawStore)
for _, s := range slots {
    fmt.Printf("Type: %-12s  Label: %s\n", s.SlotType, s.Label)
}

Change password

func ChangePassword(ctx context.Context, rawStore store.ObjectStore, kc keychain.Chain, pwd PasswordProvider) error

Replaces the password key slot. kc must unlock the current master key. pwd supplies the new password. Use PasswordString for a known value at call time:

err := cloudstic.ChangePassword(ctx, rawStore, kc,
    cloudstic.PasswordString("new-passphrase"),
)

Use PasswordProviderFunc when the new password must be obtained lazily (e.g. interactive prompt):

err := cloudstic.ChangePassword(ctx, rawStore, kc,
    cloudstic.PasswordProviderFunc(func(ctx context.Context) (string, error) {
        return promptUser("New password: ")
    }),
)

Add recovery key

func AddRecoveryKey(ctx context.Context, rawStore store.ObjectStore, kc keychain.Chain, opts AddRecoveryKeyOptions) (string, error)

Generates a BIP39 recovery key slot and returns the 24-word mnemonic. kc must unlock the current master key.

type AddRecoveryKeyOptions struct {
    Label   string // names the slot; empty means the default slot
    Replace bool   // overwrite an existing slot with the same label
}

A distinct Label lets a repository hold several recovery keys, all valid at once. Without Replace: true, adding a recovery key under a label that already exists returns a *keychain.SlotExistsError and writes nothing — this protects a previously issued mnemonic from being silently invalidated.

mnemonic, err := cloudstic.AddRecoveryKey(ctx, rawStore, kc, cloudstic.AddRecoveryKeyOptions{})
if err != nil {
    log.Fatal(err)
}
fmt.Println("Recovery key:", mnemonic)
// Store this securely: it is returned only once!

Workstation Setup and Source Discovery

These are no longer part of the client library. Workstation onboarding — discovering local backup candidates, proposing a set of profiles, and applying that plan — moved to an internal package, because it is a CLI wizard rather than a library capability: none of it needed a repository client, and the entry points were package-level functions rather than Client methods.Use the CLI instead:

cloudstic source discover        # list local backup candidates
cloudstic setup                  # plan and apply a set of profiles

The profiles file those commands write is still readable and writable programmatically — see Backup Profiles below.

Backup Profiles

The profiles YAML format lives in github.com/cloudstic/cli/pkg/profile. It is a separate package from the client precisely so that reading or writing a user’s profiles does not require opening a repository:

import "github.com/cloudstic/cli/pkg/profile"

cfg, err := profile.Load(profilesPath)      // or profile.LoadOrEmpty, which
if err != nil {                             // treats a missing file as empty
    log.Fatal(err)
}

profile.EnsureMaps(cfg)
cfg.Profiles["documents"] = profile.Profile{
    Source: "local:/Users/me/Documents",
    Store:  "primary",
    Tags:   []string{"workstation"},
}

if err := profile.Save(profilesPath, cfg); err != nil {
    log.Fatal(err)
}

Save validates before writing, so a malformed secret reference is rejected there rather than at backup time. The failure is a *secretref.Error, so you can branch on Kind rather than matching on the message:

var refErr *secretref.Error
if errors.As(err, &refErr) && refErr.Kind == secretref.KindInvalidRef {
    // the scheme://path in a profile field is malformed
}

Type	Purpose
`profile.Config`	The whole file: `Version`, `Stores`, `Profiles`, `Auth`
`profile.Profile`	One backup job — source, store, tags, exclusions
`profile.Store`	A named store definition, including secret references
`profile.Auth`	A reusable cloud auth entry

profile.DefaultPath("") returns where the file lives when the user names no path — profiles.yaml inside the config directory, honouring CLOUDSTIC_CONFIG_DIR. Use it so your program reads the same file the cloudstic CLI does rather than a second one that silently disagrees. It resolves a path without touching the filesystem, so asking the question creates nothing.

Acting on a Profile

Reading a profile is one thing; using one is another. pkg/open turns a named profile into a connected client in one call:

import "github.com/cloudstic/cli/pkg/open"

// An empty path means the default location.
client, err := open.FromProfile(ctx, "", "documents")
if err != nil {
    log.Fatal(err)
}

This is the one-call form of the explicit sequence, which you want instead whenever something has to happen in between — displaying the resolved configuration, diffing it, or deciding something from it before connecting:

cfg, err := profile.Load(profilesPath)
if err != nil {
    log.Fatal(err)
}

// Which store does this profile select? nil means the profile names none,
// which is legal — the CLI then takes it from -store or CLOUDSTIC_STORE.
storeCfg, err := cfg.StoreFor("documents")
if err != nil {
    log.Fatal(err)
}

clientCfg, err := config.FromProfileStore(ctx, *storeCfg, resolver)
if err != nil {
    log.Fatal(err)
}
fmt.Printf("connecting to %s\n", clientCfg.Store.URI)

client, err := open.Client(ctx, clientCfg)

A client, once returned, is already connected — so anything you want to override must be decided before this point, not after.

Layering Your Own Configuration

If your program has a configuration mechanism of its own — command-line flags, its own file, a form — WithDecided layers it over the profile, with the same precedence the cloudstic CLI applies to its flags. Every field you name is taken from your value, and the profile supplies the rest:

mine := config.Client{Store: config.Store{URI: "local:/my-own-choice"}}

client, err := open.FromProfile(ctx, "", "documents",
    open.WithDecided(mine, config.FieldsSetIn(mine)))

config.FieldsSetIn reports which fields your value is non-empty for, which is the right answer when your mechanism has no notion of “present but empty”. When empty is a choice you need to keep — -password "" meaning “no password”, as the CLI treats it — name the fields explicitly instead:

decided := config.NewFieldSet(config.FieldPassword, config.FieldS3AccessKey)

The field names are typed constants rather than strings, because the profiles file and the CLI spell the same field differently — s3_access_key against -s3-access-key — and a misspelled string silently meant “not decided”, which would connect you with a credential you had not chosen. config.StoreFields() and config.BackupFields() enumerate the complete set, so iterating them keeps your mapping current as fields are added; Field.ProfileKey() renders the profiles-file spelling for an error message.

Two groups of fields behave differently on purpose. Location and KMS settings are taken only when the profile actually names one, so a silent profile leaves what you had. Credentials are taken whenever you have not decided them, empty included — selecting a profile clears an ambient credential, because a profile is an explicit choice of which store to talk to, and reaching it with half a credential set inherited from the environment would be worse than failing to reach it.

A field you have decided is never resolved, so a broken secret reference on a field you are about to replace is not an error.

Backing Up From a Profile

The same layering applies to what a profile says to back up. config.Backup holds the resolved form, and open.Backup constructs both the source and the options a backup runs with:

bcfg, err := config.MergeProfileBackup(config.Backup{}, nil, "documents", cfg)
if err != nil {
    log.Fatal(err)
}

job, err := open.Backup(ctx, bcfg)
if err != nil {
    log.Fatal(err)
}

result, err := client.Backup(ctx, job.Source, job.Options...)

open.Backup returns the source and the options together rather than as two calls, because they share a derived value. The snapshot records a hash of the active exclude patterns, and the next backup compares its own against it to decide between an incremental scan and a full rescan. The patterns come from two places — Excludes and the file named by ExcludeFile — so a hash computed without reading that file describes a different exclude set than the one in force, and every subsequent run reads the difference as “the patterns changed”. Getting the source and the options from one call is what keeps them consistent. If you supply your own source.Source implementation, use open.BackupOptions(cfg) for the options and apply cfg.Source.Excludes yourself — otherwise the recorded hash describes filtering that is not happening. source.ExcludeHash is exported if you need to compute it directly.

Function	Purpose
`open.FromProfile(ctx, path, name, opts...)`	Profile → connected client, in one call
`open.Client(ctx, config.Client, opts...)`	Resolved configuration → connected client
`open.Store(ctx, config.Store, opts...)`	Store URI + credentials → raw `store.ObjectStore`
`open.Source(ctx, config.Source, opts...)`	Source URI + credentials → `source.Source`
`open.Backup(ctx, config.Backup, opts...)`	→ `BackupJob{Source, Options}`
`config.FromProfileStore(ctx, s, r)`	Profile store definition → `config.Client`
`config.MergeProfileStore(ctx, base, decided, s, r)`	The same, layered under your own values
`config.MergeProfileBackup(base, decided, name, cfg)`	Profile backup settings → `config.Backup`

Secret References

Store and source credentials can be written as scheme://path references instead of inline secrets. github.com/cloudstic/cli/pkg/secretref resolves them, and pkg/secretref/backends holds the schemes Cloudstic ships with: env://, file://, config-token://, keychain://, secret-service:// and wincred://.

import (
    "github.com/cloudstic/cli/pkg/secretref"
    "github.com/cloudstic/cli/pkg/secretref/backends"
)

resolver := backends.NewDefaultResolver()
password, err := resolver.Resolve(ctx, "keychain://cloudstic/store/prod")

Failures are typed, so you can tell a malformed reference from a missing value from a backend that is not available on this platform:

`Kind`	Meaning
`secretref.KindInvalidRef`	The `scheme://path` is malformed
`secretref.KindNotFound`	The backend works, but has no value at that path
`secretref.KindBackendUnavailable`	No backend is registered for that scheme, or it is unsupported here

Adding your own scheme

Backend is a single method, so a custom scheme — Vault, a cloud KMS, your own service — is small. backends.Default() returns a fresh map on every call, which is what lets you extend the built-in set rather than replace it:

type vaultBackend struct{ client *vault.Client }

func (v *vaultBackend) Resolve(ctx context.Context, ref secretref.Ref) (string, error) {
    secret, err := v.client.Read(ctx, ref.Path)
    if err != nil {
        return "", secretref.NewError(secretref.KindNotFound, ref.Raw, "no such vault secret", err)
    }
    return secret, nil
}

b := backends.Default()
b["vault"] = &vaultBackend{client: c}
resolver := secretref.NewResolver(b)

Use secretref.NewError for failures so callers can branch on Kind the same way they do for the built-in backends.

Implement WritableBackend as well (adding Scheme, DisplayName, WriteSupported, DefaultRef, Exists and Store) if the scheme should also be offered when the CLI and TUI write a secret, rather than only when resolving one.

Choosing where managed tokens live

The config-token:// backend stores tokens itself, encrypted at rest, and by default puts them in Cloudstic’s own config directory — CLOUDSTIC_CONFIG_DIR or the OS default. For an embedding program that is usually the wrong place, and an environment variable is a poor way for a library to be configured:

b := backends.Default()
b["config-token"] = backends.NewConfigTokenBackend(
    backends.WithConfigDir("/var/lib/myapp/secrets"),
)
resolver := secretref.NewResolver(b)

The salt that derives the at-rest encryption key lives in that directory too. Two backends pointed at different directories therefore hold different keys — tokens written under one are not readable under the other. Changing the directory of an existing deployment orphans whatever was already stored there.

Progress Reporting

cloudstic.Reporter receives progress events during long-running operations:

type Reporter interface {
    StartPhase(name string, total int64) Phase
}

type Phase interface {
    Increment(n int64)
    Done()
}

Pass your reporter to NewClient:

client, err := cloudstic.NewClient(ctx, rawStore,
    cloudstic.WithKeychain(kc),
    cloudstic.WithReporter(myReporter),
)

If WithReporter is omitted, all progress output is suppressed (no-op default).

Complete Example: Automated Backup

package main

import (
    "context"
    "encoding/hex"
    "fmt"
    "log"
    "os"

    cloudstic "github.com/cloudstic/cli"
    "github.com/cloudstic/cli/pkg/keychain"
    "github.com/cloudstic/cli/pkg/source"
    "github.com/cloudstic/cli/pkg/store"
)

func main() {
    ctx := context.Background()

    // Load platform key from environment
    encKeyHex := os.Getenv("CLOUDSTIC_ENCRYPTION_KEY")
    encKey, err := hex.DecodeString(encKeyHex)
    if err != nil {
        log.Fatalf("invalid CLOUDSTIC_ENCRYPTION_KEY: %v", err)
    }

    // Open S3 backend
    rawStore, err := store.NewS3Store(ctx, "my-backup-bucket",
        store.WithS3Region("us-east-1"),
    )
    if err != nil {
        log.Fatal(err)
    }

    // Build keychain
    kc := keychain.Chain{keychain.WithPlatformKey(encKey)}

    // Initialize if not already done (adopt-slots makes it idempotent)
    if _, err := cloudstic.InitRepo(ctx, rawStore,
        cloudstic.WithInitCredentials(kc),
        cloudstic.WithInitAdoptSlots(),
    ); err != nil {
        log.Fatal(err)
    }

    // Create client
    client, err := cloudstic.NewClient(ctx, rawStore, cloudstic.WithKeychain(kc))
    if err != nil {
        log.Fatal(err)
    }

    // Backup
    src := source.NewLocalSource("/data",
        source.WithLocalExcludePatterns([]string{"*.tmp", ".cache/"}),
    )
    result, err := client.Backup(ctx, src, cloudstic.WithTags("automated"))
    if err != nil {
        log.Fatalf("backup failed: %v", err)
    }
    fmt.Printf("Backup complete: %s (%d new, %d changed files)\n",
        result.SnapshotHash, result.FilesNew, result.FilesChanged)

    // Apply retention policy and prune
    _, err = client.ForgetPolicy(ctx,
        cloudstic.WithKeepLast(7),
        cloudstic.WithKeepWeekly(4),
        cloudstic.WithKeepMonthly(12),
        cloudstic.WithPrune(),
    )
    if err != nil {
        log.Fatalf("retention policy failed: %v", err)
    }
}

​Installation

​Quick Start

​Storage Backends

​Local

​Amazon S3 (and S3-compatible)

​Backblaze B2

​SFTP

​Keychain

​Initializing a Repository

​Init Options

​InitResult

​Example

​Inspecting a Repository

​Creating a Client

​Client Options

​Backup

​Backup Options

​BackupResult

​Example

​Sources

​Restore

​Restore Options

​RestoreResult

​Example

​Restoring directly to a directory

​List

​List Options

​ListResult

​Example

​LsSnapshot

​LsSnapshot Options

​LsSnapshotResult

​Find

​Find Options

​FindResult

​Example

​Diff

​Diff Options

​Snapshot reference errors

​DiffResult

​Example

​Prune

​Prune Options

​PruneResult

​Forget

​Remove a specific snapshot

​Apply a retention policy

​Forget Options

​PolicyResult

​Example

​Check

​Check Options

​CheckResult

​Example

​BreakLock

​ErrRepoLocked

​Cat

​CatResult

​Object Key Namespaces

​Example

​Key Management

​List key slots

​Change password

​Add recovery key

​Workstation Setup and Source Discovery

​Backup Profiles

​Acting on a Profile

​Layering Your Own Configuration

​Backing Up From a Profile

​Secret References

​Adding your own scheme

​Choosing where managed tokens live

​Progress Reporting

​Complete Example: Automated Backup

​See Also

Installation

Quick Start

Storage Backends

Local

Amazon S3 (and S3-compatible)

Backblaze B2

SFTP

Keychain

Initializing a Repository

Init Options

InitResult

Example

Inspecting a Repository

Creating a Client

Client Options

Backup

Backup Options

BackupResult

Example

Sources

Restore

Restore Options

RestoreResult

Example

Restoring directly to a directory

List

List Options

ListResult

Example

LsSnapshot

LsSnapshot Options

LsSnapshotResult

Find

Find Options

FindResult

Example

Diff

Diff Options

Snapshot reference errors

DiffResult

Example

Prune

Prune Options

PruneResult

Forget

Remove a specific snapshot

Apply a retention policy

Forget Options

PolicyResult

Example

Check

Check Options

CheckResult

Example

BreakLock

ErrRepoLocked

Cat

CatResult

Object Key Namespaces

Example

Key Management

List key slots

Change password

Add recovery key

Workstation Setup and Source Discovery

Backup Profiles

Acting on a Profile

Layering Your Own Configuration

Backing Up From a Profile

Secret References

Adding your own scheme

Choosing where managed tokens live

Progress Reporting

Complete Example: Automated Backup

See Also